2021-03-10T13:39:43.099 INFO:root:teuthology version: 1.1.0-b6eecfe8 2021-03-10T13:39:43.100 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_sage@teuthology --verbose --archive /home/teuthworker/archive/sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba/5953062 --name sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba --description rados:cephadm:upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait fixed-2 mon_election/classic} -- /home/teuthworker/archive/sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba/5953062/orig.config.yaml 2021-03-10T13:39:43.133 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-03-10T13:39:43.202 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba/5953062 branch: wip-sage-testing-2021-03-09-1434 description: rados:cephadm:upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait fixed-2 mon_election/classic} email: null first_in_suite: false job_id: '5953062' kernel: kdb: true sha1: distro last_in_suite: false machine_type: gibba name: sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-sage-testing-2021-03-09-1434 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd shutdown pgref assert: true log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b26fc080937a021ce826cdc94d1adb52c064381a ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: b26fc080937a021ce826cdc94d1adb52c064381a workunit: branch: cephadm-rgw-simplification sha1: fd39f244c71b258f156a77730a6203886f27efb2 owner: scheduled_sage@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b sha1: b26fc080937a021ce826cdc94d1adb52c064381a sleep_before_teardown: 0 suite: rados:cephadm:upgrade suite_branch: cephadm-rgw-simplification suite_path: /home/teuthworker/src/github.com_liewegas_ceph_fd39f244c71b258f156a77730a6203886f27efb2/qa suite_relpath: qa suite_repo: https://github.com/liewegas/ceph suite_sha1: fd39f244c71b258f156a77730a6203886f27efb2 targets: gibba011.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrZA9V10K5DHyrAlUPR02boyYl6R1YLrLTe3t70IG1T04oMo+85UZ+fW9CoVBgcCY+j/55IM+BW4xfjcRD9G7E8AYws+lRU9oIxAyIZ1fY45dGBbp/561iB9PhvJ08/cFZno/2fRcDqq5hK1f3xm5ZaGIBNK8Eqi4yNZ/HLBYF5iNDlsAVBdrMJaPIN4yyh6FK40JVH7XPdci61JHkqPK/6Aw9kvyrvS6+cN5Tae0/C5DIg6go0Q99dGY6MEiNqUcBBVuzqNBShULmjiHYcXB0BOct+Ln2SrQMsgDoL5jZhbw3Jh5575ODxCcsad0VGLEHT4mmdu/O8+rno293wE0NVY0VMlmsuJq5u0zx8As02LoyacRUJ9P9oYdhTEK/aPjz6bYVZc0hHjP62UH3sLtWPWUgcPo8CC6TY5teymELlnDVU+H5ULA0pYE5pXO/atmkVHpq3cYOKpLMeHEMi+93OnZVHW9QeNcflkMTurJkWhuXcUJP1CByB2rtp2w+eK8= gibba032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOA+IXDlnlUyR4FnUXgT5yGBdMAcp0YEgde461H/CB8nKiLpOMRMslg8YVNPTGLfDAX347YaTXoXtaR3uDOet/7VkDDeC52gr7VSDZgnM48Nv4qJFVLsPI35KNjdrPmgZoNwXiE04X4E+E+TzRspQlb2TVuiLfMA5O0qYc1zUZMCG+/y+nnBOwvuacl0+VhulGmCUKz14Zq/eExoisT+pSzOOZnABckVex0o0iw6NpAOuTWaiHfmACNV3Q6pMrqLuTr2jWRF0wgaB5UcatD87D/IE3SSArDQOvSQwcHSEKgRSyKR+wjYBveC6iRHjVvuJekrbP96eX7WLTGUPzVcD/RJvTuKlup0kVGFeIz5XDkzjJnYwsPBA4DjLbB4Q0rHu8IOPg5L80RU0ZiNjlfCjq9WjrdmKJyQ6UgFHYniWgtBjvj4Ewsu8M+KWejQ+qMf50YRoO45Nq8mXlyngcWFF2CfbRPYCiHgxbzcNeYUInuuwCh+eKGE3dK0l8uW3JtQ8= tasks: - cephadm: allow_ptrace: false cephadm_branch: v15.2.0 cephadm_git_url: https://github.com/ceph/ceph image: docker.io/ceph/ceph:v15.2.0 - cephadm.shell: mon.a: - ceph config set mgr mgr/cephadm/use_repo_digest false --force - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw r z --placement=1 - sleep 120 - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 teuthology_branch: master teuthology_sha1: b6eecfe89a4363750d696160843a096384b357a8 timestamp: 2021-03-10_13:21:20 tube: gibba user: sage verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.gibba.26181 2021-03-10T13:39:43.203 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_liewegas_ceph_fd39f244c71b258f156a77730a6203886f27efb2/qa; will attempt to use it 2021-03-10T13:39:43.203 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_liewegas_ceph_fd39f244c71b258f156a77730a6203886f27efb2/qa/tasks 2021-03-10T13:39:43.204 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-03-10T13:39:43.205 INFO:teuthology.task.internal:Checking packages... 2021-03-10T13:39:43.225 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'b26fc080937a021ce826cdc94d1adb52c064381a' 2021-03-10T13:39:43.225 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-10T13:39:43.226 INFO:teuthology.packaging:ref: None 2021-03-10T13:39:43.226 INFO:teuthology.packaging:tag: None 2021-03-10T13:39:43.227 INFO:teuthology.packaging:branch: wip-sage-testing-2021-03-09-1434 2021-03-10T13:39:43.227 INFO:teuthology.packaging:sha1: b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:39:43.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-sage-testing-2021-03-09-1434 2021-03-10T13:39:43.359 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-1749-gb26fc080-1focal 2021-03-10T13:39:43.360 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-03-10T13:39:43.371 INFO:teuthology.task.internal:no buildpackages task found 2021-03-10T13:39:43.371 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-03-10T13:39:43.397 INFO:teuthology.task.internal:Saving configuration 2021-03-10T13:39:43.409 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-03-10T13:39:43.422 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-10T13:39:43.441 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-10 13:34:32.116852', 'description': '/home/teuthworker/archive/sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba/5953062', 'locked_by': 'scheduled_sage@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP8C0wkGFbbGHg4OFmoeLX+Ju3wm3VoO+VuyAb2qNph5', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba011.front.sepia.ceph.com'} 2021-03-10T13:39:43.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-10 13:34:32.116833', 'description': '/home/teuthworker/archive/sage-2021-03-10_13:21:20-rados:cephadm:upgrade-wip-sage-testing-2021-03-09-1434-distro-basic-gibba/5953062', 'locked_by': 'scheduled_sage@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIONp26M7cAbPbprEtrdSrhIJ7+nKtGSpcSLDDAomN/8a', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba032.front.sepia.ceph.com'} 2021-03-10T13:39:43.458 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-03-10T13:39:43.468 INFO:teuthology.task.internal:roles: ubuntu@gibba011.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-03-10T13:39:43.469 INFO:teuthology.task.internal:roles: ubuntu@gibba032.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2021-03-10T13:39:43.469 INFO:teuthology.run_tasks:Running task console_log... 2021-03-10T13:39:43.530 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdf77516598>, signals=[15]) 2021-03-10T13:39:43.532 INFO:teuthology.run_tasks:Running task internal.connect... 2021-03-10T13:39:43.542 INFO:teuthology.task.internal:Opening connections... 2021-03-10T13:39:43.543 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba011.front.sepia.ceph.com 2021-03-10T13:39:43.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-10T13:39:43.587 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba032.front.sepia.ceph.com 2021-03-10T13:39:43.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-10T13:39:43.622 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-03-10T13:39:43.633 DEBUG:teuthology.orchestra.run.gibba011:> uname -m 2021-03-10T13:39:43.638 INFO:teuthology.orchestra.run.gibba011.stdout:x86_64 2021-03-10T13:39:43.638 DEBUG:teuthology.orchestra.run.gibba011:> cat /etc/os-release 2021-03-10T13:39:43.685 INFO:teuthology.orchestra.run.gibba011.stdout:NAME="Ubuntu" 2021-03-10T13:39:43.686 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-03-10T13:39:43.686 INFO:teuthology.orchestra.run.gibba011.stdout:ID=ubuntu 2021-03-10T13:39:43.686 INFO:teuthology.orchestra.run.gibba011.stdout:ID_LIKE=debian 2021-03-10T13:39:43.687 INFO:teuthology.orchestra.run.gibba011.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-03-10T13:39:43.687 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION_ID="20.04" 2021-03-10T13:39:43.687 INFO:teuthology.orchestra.run.gibba011.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-10T13:39:43.687 INFO:teuthology.orchestra.run.gibba011.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-10T13:39:43.687 INFO:teuthology.orchestra.run.gibba011.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-10T13:39:43.688 INFO:teuthology.orchestra.run.gibba011.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-10T13:39:43.688 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION_CODENAME=focal 2021-03-10T13:39:43.688 INFO:teuthology.orchestra.run.gibba011.stdout:UBUNTU_CODENAME=focal 2021-03-10T13:39:43.689 INFO:teuthology.lock.ops:Updating gibba011.front.sepia.ceph.com on lock server 2021-03-10T13:39:43.704 DEBUG:teuthology.orchestra.run.gibba032:> uname -m 2021-03-10T13:39:43.708 INFO:teuthology.orchestra.run.gibba032.stdout:x86_64 2021-03-10T13:39:43.709 DEBUG:teuthology.orchestra.run.gibba032:> cat /etc/os-release 2021-03-10T13:39:43.756 INFO:teuthology.orchestra.run.gibba032.stdout:NAME="Ubuntu" 2021-03-10T13:39:43.756 INFO:teuthology.orchestra.run.gibba032.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-03-10T13:39:43.756 INFO:teuthology.orchestra.run.gibba032.stdout:ID=ubuntu 2021-03-10T13:39:43.757 INFO:teuthology.orchestra.run.gibba032.stdout:ID_LIKE=debian 2021-03-10T13:39:43.757 INFO:teuthology.orchestra.run.gibba032.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-03-10T13:39:43.757 INFO:teuthology.orchestra.run.gibba032.stdout:VERSION_ID="20.04" 2021-03-10T13:39:43.757 INFO:teuthology.orchestra.run.gibba032.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-10T13:39:43.757 INFO:teuthology.orchestra.run.gibba032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-10T13:39:43.758 INFO:teuthology.orchestra.run.gibba032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-10T13:39:43.758 INFO:teuthology.orchestra.run.gibba032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-10T13:39:43.758 INFO:teuthology.orchestra.run.gibba032.stdout:VERSION_CODENAME=focal 2021-03-10T13:39:43.758 INFO:teuthology.orchestra.run.gibba032.stdout:UBUNTU_CODENAME=focal 2021-03-10T13:39:43.759 INFO:teuthology.lock.ops:Updating gibba032.front.sepia.ceph.com on lock server 2021-03-10T13:39:43.774 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-03-10T13:39:43.787 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-03-10T13:39:43.800 INFO:teuthology.task.internal:Checking for old test directory... 2021-03-10T13:39:43.800 DEBUG:teuthology.orchestra.run.gibba011:> test '!' -e /home/ubuntu/cephtest 2021-03-10T13:39:43.803 DEBUG:teuthology.orchestra.run.gibba032:> test '!' -e /home/ubuntu/cephtest 2021-03-10T13:39:43.811 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-03-10T13:39:43.824 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-03-10T13:39:43.824 DEBUG:teuthology.orchestra.run.gibba011:> test -z $(ls -A /var/lib/ceph) 2021-03-10T13:39:43.827 DEBUG:teuthology.orchestra.run.gibba032:> test -z $(ls -A /var/lib/ceph) 2021-03-10T13:39:43.844 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-03-10T13:39:43.889 INFO:teuthology.run_tasks:Running task kernel... 2021-03-10T13:39:43.906 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.a': {'kdb': True, 'sha1': 'distro'}, 'alertmanager.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'prometheus.a': {'kdb': True, 'sha1': 'distro'}, 'grafana.a': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.b': {'kdb': True, 'sha1': 'distro'}} 2021-03-10T13:39:43.907 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-03-10T13:39:43.907 DEBUG:teuthology.orchestra.run.gibba011:> uname -r 2021-03-10T13:39:43.916 INFO:teuthology.orchestra.run.gibba011.stdout:5.4.0-66-generic 2021-03-10T13:39:43.916 INFO:teuthology.task.kernel:Running kernel on gibba011: 5.4.0-66-generic 2021-03-10T13:39:43.916 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get clean 2021-03-10T13:39:43.988 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get update 2021-03-10T13:39:44.089 INFO:teuthology.orchestra.run.gibba011.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB] 2021-03-10T13:39:44.197 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-03-10T13:39:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-03-10T13:39:44.322 INFO:teuthology.orchestra.run.gibba011.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [199 kB] 2021-03-10T13:39:44.401 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [527 kB] 2021-03-10T13:39:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [113 kB] 2021-03-10T13:39:44.430 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [7,208 B] 2021-03-10T13:39:44.430 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [133 kB] 2021-03-10T13:39:44.434 INFO:teuthology.orchestra.run.gibba011.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [19.5 kB] 2021-03-10T13:39:44.435 INFO:teuthology.orchestra.run.gibba011.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [431 kB] 2021-03-10T13:39:44.441 INFO:teuthology.orchestra.run.gibba011.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [545 kB] 2021-03-10T13:39:44.451 INFO:teuthology.orchestra.run.gibba011.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [79.6 kB] 2021-03-10T13:39:44.452 INFO:teuthology.orchestra.run.gibba011.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [10.5 kB] 2021-03-10T13:39:44.607 INFO:teuthology.orchestra.run.gibba011.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-03-10T13:39:44.716 INFO:teuthology.orchestra.run.gibba011.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [436 kB] 2021-03-10T13:39:44.850 INFO:teuthology.orchestra.run.gibba011.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [844 kB] 2021-03-10T13:39:44.965 INFO:teuthology.orchestra.run.gibba011.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [202 kB] 2021-03-10T13:39:44.977 INFO:teuthology.orchestra.run.gibba011.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [12.8 kB] 2021-03-10T13:39:44.978 INFO:teuthology.orchestra.run.gibba011.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [156 kB] 2021-03-10T13:39:44.985 INFO:teuthology.orchestra.run.gibba011.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [23.3 kB] 2021-03-10T13:39:44.987 INFO:teuthology.orchestra.run.gibba011.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [558 kB] 2021-03-10T13:39:45.025 INFO:teuthology.orchestra.run.gibba011.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [749 kB] 2021-03-10T13:39:45.055 INFO:teuthology.orchestra.run.gibba011.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [157 kB] 2021-03-10T13:39:45.058 INFO:teuthology.orchestra.run.gibba011.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [16.2 kB] 2021-03-10T13:39:46.782 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 5,542 kB in 1s (4,761 kB/s) 2021-03-10T13:39:47.697 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-03-10T13:39:47.708 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-10T13:39:47.743 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-03-10T13:39:47.836 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-03-10T13:39:47.836 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-03-10T13:39:47.879 INFO:teuthology.orchestra.run.gibba011.stderr:E: Unable to locate package linux-image-current-generic 2021-03-10T13:39:47.880 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-10T13:39:47.881 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-10T13:39:47.916 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-03-10T13:39:48.008 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-03-10T13:39:48.008 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-03-10T13:39:48.093 INFO:teuthology.orchestra.run.gibba011.stdout:linux-image-generic is already the newest version (5.4.0.66.69). 2021-03-10T13:39:48.094 INFO:teuthology.orchestra.run.gibba011.stdout:linux-image-generic set to manually installed. 2021-03-10T13:39:48.094 INFO:teuthology.orchestra.run.gibba011.stdout:0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. 2021-03-10T13:39:48.095 DEBUG:teuthology.orchestra.run.gibba011:> dpkg -s linux-image-generic 2021-03-10T13:39:48.105 INFO:teuthology.orchestra.run.gibba011.stdout:Package: linux-image-generic 2021-03-10T13:39:48.105 INFO:teuthology.orchestra.run.gibba011.stdout:Status: install ok installed 2021-03-10T13:39:48.105 INFO:teuthology.orchestra.run.gibba011.stdout:Priority: optional 2021-03-10T13:39:48.106 INFO:teuthology.orchestra.run.gibba011.stdout:Section: kernel 2021-03-10T13:39:48.106 INFO:teuthology.orchestra.run.gibba011.stdout:Installed-Size: 18 2021-03-10T13:39:48.106 INFO:teuthology.orchestra.run.gibba011.stdout:Maintainer: Ubuntu Kernel Team 2021-03-10T13:39:48.106 INFO:teuthology.orchestra.run.gibba011.stdout:Architecture: amd64 2021-03-10T13:39:48.107 INFO:teuthology.orchestra.run.gibba011.stdout:Source: linux-meta 2021-03-10T13:39:48.107 INFO:teuthology.orchestra.run.gibba011.stdout:Version: 5.4.0.66.69 2021-03-10T13:39:48.107 INFO:teuthology.orchestra.run.gibba011.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.6) 2021-03-10T13:39:48.107 INFO:teuthology.orchestra.run.gibba011.stdout:Depends: linux-image-5.4.0-66-generic, linux-modules-extra-5.4.0-66-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-10T13:39:48.107 INFO:teuthology.orchestra.run.gibba011.stdout:Recommends: thermald 2021-03-10T13:39:48.108 INFO:teuthology.orchestra.run.gibba011.stdout:Description: Generic Linux kernel image 2021-03-10T13:39:48.108 INFO:teuthology.orchestra.run.gibba011.stdout: This package will always depend on the latest generic kernel image 2021-03-10T13:39:48.108 INFO:teuthology.orchestra.run.gibba011.stdout: available. 2021-03-10T13:39:48.108 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-10T13:39:48.109 DEBUG:teuthology.orchestra.run.gibba032:> uname -r 2021-03-10T13:39:48.117 INFO:teuthology.orchestra.run.gibba032.stdout:5.4.0-66-generic 2021-03-10T13:39:48.118 INFO:teuthology.task.kernel:Running kernel on gibba032: 5.4.0-66-generic 2021-03-10T13:39:48.118 DEBUG:teuthology.orchestra.run.gibba032:> sudo apt-get clean 2021-03-10T13:39:48.207 DEBUG:teuthology.orchestra.run.gibba032:> sudo apt-get update 2021-03-10T13:39:48.443 INFO:teuthology.orchestra.run.gibba032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-03-10T13:39:48.450 INFO:teuthology.orchestra.run.gibba032.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB] 2021-03-10T13:39:48.509 INFO:teuthology.orchestra.run.gibba032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-03-10T13:39:48.832 INFO:teuthology.orchestra.run.gibba032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-03-10T13:39:48.840 INFO:teuthology.orchestra.run.gibba032.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [527 kB] 2021-03-10T13:39:48.950 INFO:teuthology.orchestra.run.gibba032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [436 kB] 2021-03-10T13:39:49.075 INFO:teuthology.orchestra.run.gibba032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [844 kB] 2021-03-10T13:39:49.133 INFO:teuthology.orchestra.run.gibba032.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [199 kB] 2021-03-10T13:39:49.150 INFO:teuthology.orchestra.run.gibba032.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [113 kB] 2021-03-10T13:39:49.161 INFO:teuthology.orchestra.run.gibba032.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [7,208 B] 2021-03-10T13:39:49.161 INFO:teuthology.orchestra.run.gibba032.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [133 kB] 2021-03-10T13:39:49.173 INFO:teuthology.orchestra.run.gibba032.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [19.5 kB] 2021-03-10T13:39:49.175 INFO:teuthology.orchestra.run.gibba032.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [545 kB] 2021-03-10T13:39:49.189 INFO:teuthology.orchestra.run.gibba032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [202 kB] 2021-03-10T13:39:49.203 INFO:teuthology.orchestra.run.gibba032.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [12.8 kB] 2021-03-10T13:39:49.204 INFO:teuthology.orchestra.run.gibba032.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [156 kB] 2021-03-10T13:39:49.211 INFO:teuthology.orchestra.run.gibba032.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [23.3 kB] 2021-03-10T13:39:49.213 INFO:teuthology.orchestra.run.gibba032.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [749 kB] 2021-03-10T13:39:49.244 INFO:teuthology.orchestra.run.gibba032.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [431 kB] 2021-03-10T13:39:49.252 INFO:teuthology.orchestra.run.gibba032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [558 kB] 2021-03-10T13:39:49.275 INFO:teuthology.orchestra.run.gibba032.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [79.6 kB] 2021-03-10T13:39:49.279 INFO:teuthology.orchestra.run.gibba032.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [157 kB] 2021-03-10T13:39:49.281 INFO:teuthology.orchestra.run.gibba032.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [10.5 kB] 2021-03-10T13:39:49.284 INFO:teuthology.orchestra.run.gibba032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [16.2 kB] 2021-03-10T13:39:51.084 INFO:teuthology.orchestra.run.gibba032.stdout:Fetched 5,542 kB in 1s (4,362 kB/s) 2021-03-10T13:39:52.038 INFO:teuthology.orchestra.run.gibba032.stdout:Reading package lists... 2021-03-10T13:39:52.048 DEBUG:teuthology.orchestra.run.gibba032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-10T13:39:52.092 INFO:teuthology.orchestra.run.gibba032.stdout:Reading package lists... 2021-03-10T13:39:52.187 INFO:teuthology.orchestra.run.gibba032.stdout:Building dependency tree... 2021-03-10T13:39:52.188 INFO:teuthology.orchestra.run.gibba032.stdout:Reading state information... 2021-03-10T13:39:52.231 INFO:teuthology.orchestra.run.gibba032.stderr:E: Unable to locate package linux-image-current-generic 2021-03-10T13:39:52.232 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-10T13:39:52.233 DEBUG:teuthology.orchestra.run.gibba032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-10T13:39:52.314 INFO:teuthology.orchestra.run.gibba032.stdout:Reading package lists... 2021-03-10T13:39:52.409 INFO:teuthology.orchestra.run.gibba032.stdout:Building dependency tree... 2021-03-10T13:39:52.410 INFO:teuthology.orchestra.run.gibba032.stdout:Reading state information... 2021-03-10T13:39:52.496 INFO:teuthology.orchestra.run.gibba032.stdout:linux-image-generic is already the newest version (5.4.0.66.69). 2021-03-10T13:39:52.497 INFO:teuthology.orchestra.run.gibba032.stdout:linux-image-generic set to manually installed. 2021-03-10T13:39:52.497 INFO:teuthology.orchestra.run.gibba032.stdout:0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. 2021-03-10T13:39:52.498 DEBUG:teuthology.orchestra.run.gibba032:> dpkg -s linux-image-generic 2021-03-10T13:39:52.508 INFO:teuthology.orchestra.run.gibba032.stdout:Package: linux-image-generic 2021-03-10T13:39:52.508 INFO:teuthology.orchestra.run.gibba032.stdout:Status: install ok installed 2021-03-10T13:39:52.508 INFO:teuthology.orchestra.run.gibba032.stdout:Priority: optional 2021-03-10T13:39:52.509 INFO:teuthology.orchestra.run.gibba032.stdout:Section: kernel 2021-03-10T13:39:52.509 INFO:teuthology.orchestra.run.gibba032.stdout:Installed-Size: 18 2021-03-10T13:39:52.509 INFO:teuthology.orchestra.run.gibba032.stdout:Maintainer: Ubuntu Kernel Team 2021-03-10T13:39:52.509 INFO:teuthology.orchestra.run.gibba032.stdout:Architecture: amd64 2021-03-10T13:39:52.509 INFO:teuthology.orchestra.run.gibba032.stdout:Source: linux-meta 2021-03-10T13:39:52.510 INFO:teuthology.orchestra.run.gibba032.stdout:Version: 5.4.0.66.69 2021-03-10T13:39:52.510 INFO:teuthology.orchestra.run.gibba032.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.6) 2021-03-10T13:39:52.510 INFO:teuthology.orchestra.run.gibba032.stdout:Depends: linux-image-5.4.0-66-generic, linux-modules-extra-5.4.0-66-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-10T13:39:52.510 INFO:teuthology.orchestra.run.gibba032.stdout:Recommends: thermald 2021-03-10T13:39:52.511 INFO:teuthology.orchestra.run.gibba032.stdout:Description: Generic Linux kernel image 2021-03-10T13:39:52.511 INFO:teuthology.orchestra.run.gibba032.stdout: This package will always depend on the latest generic kernel image 2021-03-10T13:39:52.511 INFO:teuthology.orchestra.run.gibba032.stdout: available. 2021-03-10T13:39:52.511 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-10T13:39:52.512 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-03-10T13:39:52.512 DEBUG:teuthology.orchestra.run.gibba011:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-10T13:39:52.538 INFO:teuthology.orchestra.run.gibba011.stdout:ttyS1 2021-03-10T13:39:52.591 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-03-10T13:39:52.591 DEBUG:teuthology.orchestra.run.gibba032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-10T13:39:52.606 INFO:teuthology.orchestra.run.gibba032.stdout:ttyS1 2021-03-10T13:39:52.637 INFO:teuthology.run_tasks:Running task internal.base... 2021-03-10T13:39:52.645 INFO:teuthology.task.internal:Creating test directory... 2021-03-10T13:39:52.646 DEBUG:teuthology.orchestra.run.gibba011:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-10T13:39:52.649 DEBUG:teuthology.orchestra.run.gibba032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-10T13:39:52.686 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-03-10T13:39:52.696 INFO:teuthology.run_tasks:Running task internal.archive... 2021-03-10T13:39:52.710 INFO:teuthology.task.internal:Creating archive directory... 2021-03-10T13:39:52.710 DEBUG:teuthology.orchestra.run.gibba011:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-10T13:39:52.713 DEBUG:teuthology.orchestra.run.gibba032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-10T13:39:52.726 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-03-10T13:39:52.737 INFO:teuthology.task.internal:Enabling coredump saving... 2021-03-10T13:39:52.738 DEBUG:teuthology.orchestra.run.gibba011:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-10T13:39:52.740 DEBUG:teuthology.orchestra.run.gibba032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-10T13:39:52.760 INFO:teuthology.orchestra.run.gibba011.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-10T13:39:52.769 INFO:teuthology.orchestra.run.gibba032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-10T13:39:52.769 INFO:teuthology.orchestra.run.gibba011.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-10T13:39:52.772 INFO:teuthology.orchestra.run.gibba032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-10T13:39:52.773 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-03-10T13:39:52.835 INFO:teuthology.task.internal:Configuring sudo... 2021-03-10T13:39:52.836 DEBUG:teuthology.orchestra.run.gibba011:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-10T13:39:52.838 DEBUG:teuthology.orchestra.run.gibba032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-10T13:39:52.858 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-03-10T13:39:52.873 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-03-10T13:39:52.873 DEBUG:teuthology.orchestra.run.gibba011:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-10T13:39:52.876 DEBUG:teuthology.orchestra.run.gibba032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-10T13:39:52.885 DEBUG:teuthology.orchestra.run.gibba011:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-10T13:39:52.934 DEBUG:teuthology.orchestra.run.gibba011:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-10T13:39:52.942 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:39:52.943 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-10T13:39:52.956 DEBUG:teuthology.orchestra.run.gibba032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-10T13:39:52.965 DEBUG:teuthology.orchestra.run.gibba032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-10T13:39:52.974 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:39:52.975 DEBUG:teuthology.orchestra.run.gibba032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-10T13:39:53.001 DEBUG:teuthology.orchestra.run.gibba011:> sudo service rsyslog restart 2021-03-10T13:39:53.004 DEBUG:teuthology.orchestra.run.gibba032:> sudo service rsyslog restart 2021-03-10T13:39:53.103 INFO:teuthology.run_tasks:Running task internal.timer... 2021-03-10T13:39:53.115 INFO:teuthology.task.internal:Starting timer... 2021-03-10T13:39:53.115 INFO:teuthology.run_tasks:Running task pcp... 2021-03-10T13:39:53.155 INFO:teuthology.run_tasks:Running task selinux... 2021-03-10T13:39:53.170 DEBUG:teuthology.task.selinux:Excluding gibba011: OS 'ubuntu' does not support SELinux 2021-03-10T13:39:53.171 DEBUG:teuthology.task.selinux:Excluding gibba032: OS 'ubuntu' does not support SELinux 2021-03-10T13:39:53.171 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-03-10T13:39:53.171 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-03-10T13:39:53.171 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-03-10T13:39:53.172 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-03-10T13:39:53.190 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2021-03-10T13:39:53.196 INFO:teuthology.repo_utils:Fetching master from origin 2021-03-10T13:39:53.235 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-03-10T13:39:53.251 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-03-10T13:39:53.254 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit gibba011.front.sepia.ceph.com,gibba032.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-03-10T13:39:53.594 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-03-10T13:39:54.301 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-03-10T13:39:56.192 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-03-10T13:39:56.212 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:56 +0000 (0:00:00.035) 0:00:00.035 ******* 2021-03-10T13:39:56.222 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:56 +0000 (0:00:00.010) 0:00:00.045 ******* 2021-03-10T13:39:58.011 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-03-10T13:39:58.013 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba011.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba011.front.sepia.ceph.com] 2021-03-10T13:39:58.059 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:01.837) 0:00:01.882 ******* 2021-03-10T13:39:58.070 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba032.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba032.front.sepia.ceph.com] 2021-03-10T13:39:58.120 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:00.060) 0:00:01.943 ******* 2021-03-10T13:39:58.301 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-03-10T13:39:58.302 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-10T13:39:58.323 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:00.203) 0:00:02.147 ******* 2021-03-10T13:39:58.342 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:00.018) 0:00:02.165 ******* 2021-03-10T13:39:58.887 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-03-10T13:39:58.888 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-03-10T13:39:58.909 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:00.567) 0:00:02.732 ******* 2021-03-10T13:39:58.916 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-03-10T13:39:58.940 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:58 +0000 (0:00:00.030) 0:00:02.763 ******* 2021-03-10T13:39:59.202 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-03-10T13:39:59.204 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.045207", "end": "2021-03-10 13:39:58.929137", "rc": 0, "start": "2021-03-10 13:39:58.883930", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.050768", "end": "2021-03-10 13:39:59.118821", "rc": 0, "start": "2021-03-10 13:39:59.068053", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-03-10T13:39:59.224 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:59 +0000 (0:00:00.284) 0:00:03.048 ******* 2021-03-10T13:39:59.246 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:59 +0000 (0:00:00.021) 0:00:03.069 ******* 2021-03-10T13:39:59.457 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-03-10T13:39:59.459 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:39:59.480 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:59 +0000 (0:00:00.234) 0:00:03.303 ******* 2021-03-10T13:39:59.498 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:39:59 +0000 (0:00:00.017) 0:00:03.321 ******* 2021-03-10T13:40:00.205 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-03-10T13:40:00.207 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383599.507932-4724-255127903721487/source", "state": "file", "uid": 0} changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383599.525945-4727-191798870112163/source", "state": "file", "uid": 0} 2021-03-10T13:40:00.227 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:00 +0000 (0:00:00.729) 0:00:04.051 ******* 2021-03-10T13:40:00.245 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:00 +0000 (0:00:00.017) 0:00:04.069 ******* 2021-03-10T13:40:00.633 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-03-10T13:40:00.635 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-03-10T13:40:00.636 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-03-10T13:40:00.655 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-03-10T13:40:00.656 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-03-10T13:40:00.728 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-03-10T13:40:00.731 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-03-10T13:40:00.869 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:00 +0000 (0:00:00.623) 0:00:04.693 ******* 2021-03-10T13:40:00.889 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:00 +0000 (0:00:00.019) 0:00:04.712 ******* 2021-03-10T13:40:01.520 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba011.front.sepia.ceph.com] 2021-03-10T13:40:01.575 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-03-10T13:40:01.576 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:01 +0000 (0:00:00.685) 0:00:05.398 ******* 2021-03-10T13:40:02.128 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-03-10T13:40:02.130 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.325211", "end": "2021-03-10 13:40:02.107166", "rc": 0, "start": "2021-03-10 13:40:01.781955", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} 2021-03-10T13:40:02.152 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.576) 0:00:05.975 ******* 2021-03-10T13:40:02.249 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-10T13:40:02.251 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.007088", "end": "2021-03-10 13:40:01.979811", "failed_when_result": false, "rc": 0, "start": "2021-03-10 13:40:01.972723", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-10T13:40:02.272 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.120) 0:00:06.095 ******* 2021-03-10T13:40:02.339 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-10T13:40:02.341 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:02.361 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.089) 0:00:06.184 ******* 2021-03-10T13:40:02.427 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-03-10T13:40:02.429 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:02.449 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.088) 0:00:06.272 ******* 2021-03-10T13:40:02.515 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-03-10T13:40:02.517 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:02.537 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.088) 0:00:06.361 ******* 2021-03-10T13:40:02.604 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-03-10T13:40:02.605 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:02.627 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.089) 0:00:06.450 ******* 2021-03-10T13:40:02.637 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-03-10T13:40:02.638 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] 2021-03-10T13:40:02.689 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.062) 0:00:06.513 ******* 2021-03-10T13:40:02.705 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-03-10T13:40:02.707 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:02.729 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.039) 0:00:06.552 ******* 2021-03-10T13:40:02.870 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-03-10T13:40:02.873 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:40:02.893 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:02 +0000 (0:00:00.164) 0:00:06.716 ******* 2021-03-10T13:40:03.081 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:03.082 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-10T13:40:03.176 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-10T13:40:03.270 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-10T13:40:03.354 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-10T13:40:03.458 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-10T13:40:03.542 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-03-10T13:40:03.544 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.627477", "end": "2021-03-10 13:40:03.510780", "rc": 0, "start": "2021-03-10 13:40:02.883303", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:03.544 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-10T13:40:03.577 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:03 +0000 (0:00:00.683) 0:00:07.400 ******* 2021-03-10T13:40:03.636 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-10T13:40:03.678 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-10T13:40:03.680 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008159", "end": "2021-03-10 13:40:03.591641", "failed_when_result": false, "rc": 0, "start": "2021-03-10 13:40:03.583482", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-10T13:40:03.700 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:03 +0000 (0:00:00.123) 0:00:07.524 ******* 2021-03-10T13:40:03.737 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:03.739 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-10T13:40:03.769 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-10T13:40:03.771 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:03.791 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:03 +0000 (0:00:00.090) 0:00:07.614 ******* 2021-03-10T13:40:03.827 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:03.829 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-10T13:40:03.859 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:03.881 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:03 +0000 (0:00:00.090) 0:00:07.705 ******* 2021-03-10T13:40:03.918 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:03.920 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-10T13:40:03.950 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-03-10T13:40:03.952 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:03.972 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:03 +0000 (0:00:00.090) 0:00:07.795 ******* 2021-03-10T13:40:04.008 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:04.010 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-10T13:40:04.040 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:04.064 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:04 +0000 (0:00:00.092) 0:00:07.888 ******* 2021-03-10T13:40:04.110 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:04.112 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-10T13:40:04.122 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:04.143 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:04 +0000 (0:00:00.078) 0:00:07.967 ******* 2021-03-10T13:40:04.190 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:04.191 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-10T13:40:04.274 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-03-10T13:40:04.276 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:40:04.296 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:04 +0000 (0:00:00.153) 0:00:08.120 ******* 2021-03-10T13:40:04.302 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:04.304 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-10T13:40:04.376 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-10T13:40:04.463 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-10T13:40:04.496 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-10T13:40:04.551 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-10T13:40:04.595 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-10T13:40:04.648 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-10T13:40:04.680 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-10T13:40:04.733 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-10T13:40:04.775 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-10T13:40:04.818 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-10T13:40:04.860 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-10T13:40:04.913 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-10T13:40:04.955 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-10T13:40:04.998 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-10T13:40:05.040 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-10T13:40:05.092 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-10T13:40:05.145 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-10T13:40:05.177 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-10T13:40:05.230 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-10T13:40:05.272 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-10T13:40:05.325 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-10T13:40:05.357 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-10T13:40:05.410 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-10T13:40:05.442 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-10T13:40:05.506 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-10T13:40:05.528 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-10T13:40:05.591 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-10T13:40:05.623 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-10T13:40:05.676 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-10T13:40:05.708 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-10T13:40:05.761 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-10T13:40:05.793 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-10T13:40:05.855 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-10T13:40:05.887 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-10T13:40:05.940 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-10T13:40:05.972 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-10T13:40:06.035 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-10T13:40:06.067 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-10T13:40:06.130 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-10T13:40:06.152 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-10T13:40:06.216 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-10T13:40:06.249 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-10T13:40:06.312 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-10T13:40:06.334 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-10T13:40:06.417 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-10T13:40:06.429 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-10T13:40:06.502 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-10T13:40:06.514 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-10T13:40:06.608 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-10T13:40:06.609 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-10T13:40:06.692 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-10T13:40:06.704 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-10T13:40:06.788 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-10T13:40:06.799 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-10T13:40:06.883 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-10T13:40:06.884 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-10T13:40:06.972 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-10T13:40:06.983 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-10T13:40:07.067 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-10T13:40:07.068 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-10T13:40:07.162 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-10T13:40:07.163 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-10T13:40:07.258 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-10T13:40:07.291 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-10T13:40:07.354 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-10T13:40:07.376 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-10T13:40:07.439 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-10T13:40:07.471 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-10T13:40:07.535 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-10T13:40:07.567 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-10T13:40:07.620 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-10T13:40:07.652 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-10T13:40:07.705 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-10T13:40:07.737 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-10T13:40:07.800 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-10T13:40:07.832 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-10T13:40:07.884 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-10T13:40:07.916 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-10T13:40:07.969 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-10T13:40:08.002 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-10T13:40:08.055 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-10T13:40:08.087 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-10T13:40:08.150 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-10T13:40:08.182 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-10T13:40:08.235 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-10T13:40:08.267 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-10T13:40:08.320 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-10T13:40:08.352 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-10T13:40:08.415 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-10T13:40:08.447 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-10T13:40:08.500 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-10T13:40:08.533 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-10T13:40:08.596 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-10T13:40:08.628 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-10T13:40:08.681 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-10T13:40:08.754 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-10T13:40:08.776 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-10T13:40:08.840 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-10T13:40:08.872 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-10T13:40:08.936 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-10T13:40:08.958 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-10T13:40:09.021 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-10T13:40:09.053 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-10T13:40:09.137 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-10T13:40:09.148 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-10T13:40:09.223 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-10T13:40:09.234 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-10T13:40:09.318 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-10T13:40:09.330 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-10T13:40:09.403 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-10T13:40:09.414 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-10T13:40:09.499 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-10T13:40:09.501 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-10T13:40:09.585 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-10T13:40:09.596 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-10T13:40:09.680 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-10T13:40:09.682 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-10T13:40:09.767 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-10T13:40:09.778 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-10T13:40:09.862 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-10T13:40:09.864 INFO:teuthology.task.ansible.out: 2021-03-10T13:40:09.866 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-10T13:40:09.949 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-10T13:40:09.970 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-10T13:40:10.044 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-10T13:40:10.066 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-10T13:40:10.152 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-10T13:40:10.154 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-10T13:40:10.248 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-10T13:40:10.250 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-10T13:40:10.336 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-10T13:40:10.348 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-10T13:40:10.432 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-10T13:40:10.454 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-10T13:40:10.538 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-03-10T13:40:10.550 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-10T13:40:10.643 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-10T13:40:10.644 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-10T13:40:10.738 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-10T13:40:10.739 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-10T13:40:10.823 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-10T13:40:10.865 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-10T13:40:10.918 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-10T13:40:10.961 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-10T13:40:11.015 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-10T13:40:11.047 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-10T13:40:11.100 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-10T13:40:11.143 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-10T13:40:11.197 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-10T13:40:11.229 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-10T13:40:11.292 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-10T13:40:11.324 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-10T13:40:11.377 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-10T13:40:11.410 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-10T13:40:11.484 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-10T13:40:11.506 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-10T13:40:11.570 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-10T13:40:11.593 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-10T13:40:11.667 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-10T13:40:11.688 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-10T13:40:11.752 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-10T13:40:11.775 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-10T13:40:11.849 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-10T13:40:11.871 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-10T13:40:11.934 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-10T13:40:11.956 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-10T13:40:12.030 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-10T13:40:12.052 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-10T13:40:12.126 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-10T13:40:12.148 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-03-10T13:40:12.212 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-10T13:40:12.235 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-10T13:40:12.309 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-10T13:40:12.331 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-10T13:40:12.405 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-10T13:40:12.428 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-10T13:40:12.491 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-10T13:40:12.523 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-10T13:40:12.586 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-10T13:40:12.608 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-10T13:40:12.671 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-10T13:40:12.703 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-10T13:40:12.767 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-10T13:40:12.788 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-10T13:40:12.852 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-10T13:40:12.886 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-10T13:40:12.939 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-10T13:40:12.972 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-10T13:40:13.035 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-10T13:40:13.068 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-10T13:40:13.131 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-10T13:40:13.153 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-10T13:40:13.237 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-10T13:40:13.248 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-10T13:40:13.332 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-10T13:40:13.344 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-10T13:40:13.428 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-10T13:40:13.439 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-10T13:40:13.523 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-10T13:40:13.534 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-10T13:40:13.618 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-10T13:40:13.629 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-10T13:40:13.703 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-10T13:40:13.714 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-10T13:40:13.798 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-10T13:40:13.830 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-10T13:40:13.966 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-10T13:40:14.008 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-10T13:40:14.061 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-10T13:40:14.103 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-10T13:40:14.146 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-10T13:40:14.188 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-10T13:40:14.240 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-10T13:40:14.283 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-10T13:40:14.336 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-10T13:40:14.378 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-10T13:40:14.431 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-10T13:40:14.463 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-10T13:40:14.526 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-10T13:40:14.557 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-10T13:40:14.621 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-10T13:40:14.642 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-10T13:40:14.706 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-10T13:40:14.738 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-10T13:40:14.801 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-10T13:40:14.824 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-10T13:40:14.898 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-10T13:40:14.920 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-10T13:40:14.993 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-10T13:40:15.004 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-10T13:40:15.082 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-10T13:40:15.104 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-10T13:40:15.178 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-10T13:40:15.199 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-10T13:40:15.262 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-10T13:40:15.295 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-10T13:40:15.362 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-10T13:40:15.384 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-10T13:40:15.448 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-10T13:40:15.469 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-10T13:40:15.563 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-10T13:40:15.575 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-10T13:40:15.649 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-10T13:40:15.660 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-10T13:40:15.744 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-10T13:40:15.766 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-10T13:40:15.839 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-10T13:40:15.851 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-10T13:40:15.952 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:15 +0000 (0:00:11.655) 0:00:19.775 ******* 2021-03-10T13:40:15.960 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-10T13:40:16.048 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-10T13:40:16.122 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-03-10T13:40:16.144 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:16 +0000 (0:00:00.192) 0:00:19.968 ******* 2021-03-10T13:40:16.152 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:16.152 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-10T13:40:16.228 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-03-10T13:40:16.229 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-03-10T13:40:16.231 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-03-10T13:40:16.233 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-03-10T13:40:16.234 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-03-10T13:40:16.236 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-10T13:40:16.237 INFO:teuthology.task.ansible.out:4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953 2021-03-10T13:40:16.239 INFO:teuthology.task.ansible.out:cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7 2021-03-10T13:40:16.241 INFO:teuthology.task.ansible.out:cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef148 2021-03-10T13:40:16.242 INFO:teuthology.task.ansible.out:0c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017 2021-03-10T13:40:16.244 INFO:teuthology.task.ansible.out:b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214b 2021-03-10T13:40:16.245 INFO:teuthology.task.ansible.out:bea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ov 2021-03-10T13:40:16.246 INFO:teuthology.task.ansible.out:pn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez 2021-03-10T13:40:16.247 INFO:teuthology.task.ansible.out:@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed1 2021-03-10T13:40:16.249 INFO:teuthology.task.ansible.out:9b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} TASK [Create all admin users with sudo access.] ******************************** ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-10T13:40:16.273 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:16 +0000 (0:00:00.128) 0:00:20.097 ******* 2021-03-10T13:40:16.456 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-10T13:40:16.550 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-10T13:40:16.644 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-10T13:40:16.738 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-10T13:40:16.844 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-10T13:40:16.938 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-10T13:40:17.042 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-10T13:40:17.105 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-03-10T13:40:17.107 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-10T13:40:17.127 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:17 +0000 (0:00:00.853) 0:00:20.951 ******* 2021-03-10T13:40:17.145 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:17.146 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-10T13:40:17.230 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-10T13:40:17.323 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-10T13:40:17.417 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:17.419 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-10T13:40:17.459 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:17.461 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-10T13:40:17.522 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-10T13:40:17.544 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:17.545 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-10T13:40:17.637 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:17.639 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-10T13:40:17.649 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-10T13:40:17.651 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-10T13:40:17.818 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:17.820 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-03-10T13:40:17.822 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-03-10T13:40:17.840 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:17 +0000 (0:00:00.712) 0:00:21.663 ******* 2021-03-10T13:40:17.881 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-10T13:40:17.902 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-03-10T13:40:17.924 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:17 +0000 (0:00:00.084) 0:00:21.747 ******* 2021-03-10T13:40:17.991 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:17.993 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-03-10T13:40:17.994 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-10T13:40:18.004 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-03-10T13:40:18.005 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-03-10T13:40:18.007 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-03-10T13:40:18.009 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-03-10T13:40:18.010 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-03-10T13:40:18.012 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name" 2021-03-10T13:40:18.012 INFO:teuthology.task.ansible.out:: "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-10T13:40:18.014 INFO:teuthology.task.ansible.out:4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953 2021-03-10T13:40:18.016 INFO:teuthology.task.ansible.out:cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7 2021-03-10T13:40:18.018 INFO:teuthology.task.ansible.out:cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef148 2021-03-10T13:40:18.019 INFO:teuthology.task.ansible.out:0c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017 2021-03-10T13:40:18.021 INFO:teuthology.task.ansible.out:b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214b 2021-03-10T13:40:18.022 INFO:teuthology.task.ansible.out:bea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez 2021-03-10T13:40:18.024 INFO:teuthology.task.ansible.out:@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed1 2021-03-10T13:40:18.026 INFO:teuthology.task.ansible.out:9b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-03-10T13:40:18.045 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:18 +0000 (0:00:00.121) 0:00:21.868 ******* 2021-03-10T13:40:18.103 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:18.105 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-10T13:40:18.217 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-10T13:40:18.352 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-10T13:40:18.353 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:18.467 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-03-10T13:40:18.469 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-03-10T13:40:18.592 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-03-10T13:40:18.594 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-03-10T13:40:18.603 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:18.605 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:18.614 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-03-10T13:40:18.616 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-10T13:40:18.638 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:18 +0000 (0:00:00.592) 0:00:22.461 ******* 2021-03-10T13:40:18.736 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:18.738 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-10T13:40:18.820 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-10T13:40:18.872 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-03-10T13:40:18.874 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-10T13:40:18.935 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-10T13:40:18.977 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-10T13:40:19.060 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-10T13:40:19.082 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-03-10T13:40:19.083 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-10T13:40:19.166 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-03-10T13:40:19.168 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-10T13:40:19.260 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-10T13:40:19.272 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-10T13:40:19.365 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-10T13:40:19.386 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-03-10T13:40:19.388 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-10T13:40:19.459 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-03-10T13:40:19.461 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-10T13:40:19.502 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-10T13:40:19.504 INFO:teuthology.task.ansible.out: 2021-03-10T13:40:19.556 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-03-10T13:40:19.558 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-10T13:40:19.609 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-10T13:40:19.651 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-10T13:40:19.714 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-10T13:40:19.725 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:19.736 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-10T13:40:19.830 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-03-10T13:40:19.832 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAA 2021-03-10T13:40:19.833 INFO:teuthology.task.ansible.out:AADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-03-10T13:40:19.945 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-03-10T13:40:19.947 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-03-10T13:40:19.956 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:19.957 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-10T13:40:19.968 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:20.071 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-10T13:40:20.113 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-10T13:40:20.166 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-03-10T13:40:20.168 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-10T13:40:20.239 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-03-10T13:40:20.302 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-10T13:40:20.396 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-10T13:40:20.407 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-03-10T13:40:20.409 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-10T13:40:20.491 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-10T13:40:20.502 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-03-10T13:40:20.576 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-10T13:40:20.587 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-03-10T13:40:20.589 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-03-10T13:40:20.671 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-10T13:40:20.671 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-10T13:40:20.755 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-03-10T13:40:20.757 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-10T13:40:20.776 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-10T13:40:20.860 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-03-10T13:40:20.861 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-10T13:40:20.871 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-10T13:40:20.955 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-10T13:40:20.966 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-10T13:40:21.049 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-10T13:40:21.060 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-10T13:40:21.155 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-10T13:40:21.166 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-03-10T13:40:21.168 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-03-10T13:40:21.239 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-10T13:40:21.251 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-10T13:40:21.334 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-10T13:40:21.345 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-10T13:40:21.429 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-10T13:40:21.440 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-03-10T13:40:21.514 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-03-10T13:40:21.515 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-03-10T13:40:21.535 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-10T13:40:21.618 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-03-10T13:40:21.620 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validat 2021-03-10T13:40:21.622 INFO:teuthology.task.ansible.out:e_certs": true} 2021-03-10T13:40:21.703 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-10T13:40:21.714 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-03-10T13:40:21.715 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-03-10T13:40:21.797 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-10T13:40:21.808 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-10T13:40:21.881 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-10T13:40:21.913 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-10T13:40:21.975 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-10T13:40:22.007 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-03-10T13:40:22.008 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-10T13:40:22.059 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-10T13:40:22.102 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-03-10T13:40:22.104 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-03-10T13:40:22.105 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-10T13:40:22.154 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-10T13:40:22.197 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-10T13:40:22.250 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-10T13:40:22.281 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-10T13:40:22.344 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-10T13:40:22.376 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-10T13:40:22.428 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-10T13:40:22.470 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-10T13:40:22.533 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-10T13:40:22.575 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-10T13:40:22.627 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-03-10T13:40:22.629 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-03-10T13:40:22.669 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-03-10T13:40:22.671 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-03-10T13:40:22.722 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-03-10T13:40:22.724 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-03-10T13:40:22.764 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-10T13:40:22.806 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-10T13:40:22.859 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-10T13:40:22.901 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-10T13:40:22.953 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-03-10T13:40:22.955 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-03-10T13:40:22.985 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-10T13:40:23.048 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-10T13:40:23.121 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-03-10T13:40:23.123 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-10T13:40:23.143 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-10T13:40:23.226 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-03-10T13:40:23.228 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-03-10T13:40:23.230 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-10T13:40:23.238 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-03-10T13:40:23.240 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-10T13:40:23.311 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-10T13:40:23.322 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-10T13:40:23.416 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-10T13:40:23.437 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-10T13:40:23.511 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-10T13:40:23.532 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-10T13:40:23.605 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-10T13:40:23.617 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-10T13:40:23.700 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-10T13:40:23.722 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-10T13:40:23.796 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-10T13:40:23.849 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-03-10T13:40:23.851 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-03-10T13:40:23.891 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-10T13:40:23.944 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-10T13:40:23.986 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-10T13:40:24.029 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-10T13:40:24.071 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-10T13:40:24.124 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-03-10T13:40:24.126 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-03-10T13:40:24.167 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-10T13:40:24.209 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-10T13:40:24.262 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-10T13:40:24.305 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-10T13:40:24.357 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-10T13:40:24.389 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-03-10T13:40:24.391 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-10T13:40:24.452 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-10T13:40:24.494 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-10T13:40:24.536 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-10T13:40:24.578 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-10T13:40:24.631 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-10T13:40:24.662 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-10T13:40:24.725 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-10T13:40:24.746 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-10T13:40:24.819 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-10T13:40:24.841 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-10T13:40:24.914 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-10T13:40:24.935 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-10T13:40:25.008 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-10T13:40:25.030 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-10T13:40:25.106 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-03-10T13:40:25.127 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-10T13:40:25.210 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-03-10T13:40:25.212 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-03-10T13:40:25.222 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-10T13:40:25.305 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-10T13:40:25.306 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-10T13:40:25.390 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-10T13:40:25.401 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-10T13:40:25.475 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-10T13:40:25.496 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-10T13:40:25.569 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-03-10T13:40:25.591 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-10T13:40:25.664 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-03-10T13:40:25.666 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-10T13:40:25.685 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-10T13:40:25.758 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-03-10T13:40:25.760 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-10T13:40:25.790 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-10T13:40:25.852 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-10T13:40:25.884 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-10T13:40:25.947 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-10T13:40:25.989 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-10T13:40:26.031 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-10T13:40:26.083 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-10T13:40:26.126 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-10T13:40:26.219 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-10T13:40:26.221 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-10T13:40:26.303 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-03-10T13:40:26.304 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-10T13:40:26.398 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-03-10T13:40:26.400 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-10T13:40:26.493 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-03-10T13:40:26.495 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-10T13:40:26.577 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-10T13:40:26.589 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-10T13:40:26.672 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-10T13:40:26.683 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-10T13:40:26.767 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-10T13:40:26.809 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-03-10T13:40:26.851 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-10T13:40:26.904 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-03-10T13:40:26.905 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-10T13:40:26.946 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-10T13:40:26.988 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-03-10T13:40:26.990 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-10T13:40:27.040 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-10T13:40:27.082 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-10T13:40:27.137 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-03-10T13:40:27.138 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-10T13:40:27.220 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-10T13:40:27.231 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-10T13:40:27.314 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-10T13:40:27.325 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-10T13:40:27.409 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-10T13:40:27.420 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-10T13:40:27.493 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-10T13:40:27.504 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-10T13:40:27.588 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-10T13:40:27.599 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-10T13:40:27.672 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-10T13:40:27.693 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-10T13:40:27.766 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-03-10T13:40:27.768 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-03-10T13:40:27.777 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-10T13:40:27.871 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-10T13:40:27.872 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-10T13:40:27.955 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-10T13:40:27.957 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-10T13:40:28.050 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-10T13:40:28.051 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-10T13:40:28.144 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-10T13:40:28.146 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-10T13:40:28.228 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-10T13:40:28.239 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-10T13:40:28.323 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-10T13:40:28.325 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-10T13:40:28.417 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-10T13:40:28.419 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-03-10T13:40:28.421 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-10T13:40:28.512 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-10T13:40:28.514 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-10T13:40:28.596 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-10T13:40:28.607 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-10T13:40:28.691 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-10T13:40:28.692 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-10T13:40:28.785 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-10T13:40:28.786 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-10T13:40:28.880 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-10T13:40:28.882 INFO:teuthology.task.ansible.out: ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmj 2021-03-10T13:40:28.884 INFO:teuthology.task.ansible.out:Ld+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-10T13:40:28.965 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-10T13:40:28.976 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-10T13:40:29.059 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-10T13:40:29.070 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-10T13:40:29.155 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-10T13:40:29.157 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-10T13:40:29.250 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-10T13:40:29.251 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-10T13:40:29.334 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-03-10T13:40:29.345 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-10T13:40:29.428 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-10T13:40:29.439 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-10T13:40:29.523 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-10T13:40:29.524 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-10T13:40:29.607 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-10T13:40:29.618 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-10T13:40:29.701 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-03-10T13:40:29.754 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-10T13:40:29.786 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-03-10T13:40:29.838 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-10T13:40:29.891 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-10T13:40:29.933 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-10T13:40:29.976 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-03-10T13:40:30.029 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-10T13:40:30.071 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-03-10T13:40:30.123 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-10T13:40:30.156 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-03-10T13:40:30.219 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-03-10T13:40:30.221 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-10T13:40:30.251 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-03-10T13:40:30.303 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-10T13:40:30.346 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-10T13:40:30.398 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-10T13:40:30.430 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options": 2021-03-10T13:40:30.432 INFO:teuthology.task.ansible.out: null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-10T13:40:30.494 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-10T13:40:30.525 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-10T13:40:30.597 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:30 +0000 (0:00:11.959) 0:00:34.421 ******* 2021-03-10T13:40:30.604 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-10T13:40:30.670 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.681 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.683 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.693 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.694 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.705 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.716 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.718 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.727 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.769 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:30.771 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-10T13:40:30.832 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:30.834 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e19 2021-03-10T13:40:30.836 INFO:teuthology.task.ansible.out:02a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.844 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:30.865 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:30.867 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-10T13:40:30.939 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:30.941 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-10T13:40:30.961 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:30.962 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-10T13:40:31.034 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:31.036 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-03-10T13:40:31.045 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.047 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.057 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.057 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.059 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247 2021-03-10T13:40:31.060 INFO:teuthology.task.ansible.out:f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.070 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.072 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.081 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.083 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.092 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.103 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.105 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.114 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.116 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.126 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.127 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.137 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.138 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.148 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.150 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.160 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.161 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.171 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.182 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.183 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.193 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.195 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.204 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.206 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.216 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.227 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.228 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.238 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.239 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.239 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-10T13:40:31.250 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:31.252 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.262 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.263 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.273 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.274 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.284 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.286 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.296 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.306 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.308 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.318 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.319 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.329 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.330 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.340 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.342 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.352 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.354 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.363 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.365 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.374 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.376 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.386 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.397 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.398 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.408 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.409 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.419 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.421 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.430 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.432 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.443 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.444 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.454 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.455 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.465 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.467 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.476 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.487 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.489 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.499 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.500 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.510 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.510 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.521 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.523 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.534 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.535 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.545 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.547 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.556 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.567 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.569 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.579 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.580 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.590 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.592 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.601 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.603 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.612 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.614 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.625 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.626 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.636 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.637 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.647 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.649 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.659 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.669 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.671 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.681 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.682 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.692 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.694 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.704 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.705 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.715 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.716 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.726 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.728 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.738 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.738 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.739 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-10T13:40:31.739 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:31.739 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.750 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.761 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.762 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.772 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.773 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.783 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.785 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.795 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.796 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.806 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.844 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:31 +0000 (0:00:01.246) 0:00:35.667 ******* 2021-03-10T13:40:31.850 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.852 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options": 2021-03-10T13:40:31.853 INFO:teuthology.task.ansible.out: null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-10T13:40:31.904 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:31.926 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:31 +0000 (0:00:00.082) 0:00:35.749 ******* 2021-03-10T13:40:31.932 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-10T13:40:31.934 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-10T13:40:32.012 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:32 +0000 (0:00:00.086) 0:00:35.835 ******* 2021-03-10T13:40:32.090 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:32.092 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.103 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.103 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.114 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.125 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.127 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.137 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.138 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.179 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:32.181 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-10T13:40:32.242 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:32.244 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-03-10T13:40:32.253 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.255 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.265 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:32.266 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-10T13:40:32.338 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:32.339 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-10T13:40:32.349 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:32.351 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-10T13:40:32.423 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:32.424 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-03-10T13:40:32.434 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.445 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.447 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.457 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.460 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.468 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.470 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.480 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.481 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.491 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.502 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.504 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.513 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.515 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.524 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.526 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.536 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.538 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.548 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.549 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.559 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.570 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.571 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.581 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.583 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.592 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.594 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.604 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.614 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.616 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-10T13:40:32.627 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.628 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.638 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.640 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.649 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.651 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.661 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.671 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.673 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.683 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.684 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.694 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.696 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.705 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.707 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.718 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.719 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.729 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.731 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.740 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.751 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.753 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.763 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.764 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.774 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.775 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.785 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.796 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.797 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.808 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.810 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.819 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.821 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.831 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.841 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.843 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.853 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.863 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.865 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.875 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.876 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.886 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.887 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-10T13:40:32.898 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:32.900 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.909 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.921 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.922 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.932 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.934 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.943 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.945 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.955 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.966 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.966 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.977 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:32.979 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:32.989 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.000 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.002 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.011 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.013 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.022 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.024 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.034 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.045 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.046 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.056 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.057 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.067 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.069 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.079 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.081 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.091 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.092 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.102 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.103 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.113 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.115 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.125 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.136 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.137 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.147 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.149 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.158 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.160 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-10T13:40:33.170 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-10T13:40:33.172 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.182 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.184 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.193 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.195 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.204 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.206 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.216 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.255 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:33 +0000 (0:00:01.242) 0:00:37.078 ******* 2021-03-10T13:40:33.262 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:33.263 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-10T13:40:33.326 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-03-10T13:40:33.328 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:33.350 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:33 +0000 (0:00:00.094) 0:00:37.173 ******* 2021-03-10T13:40:33.356 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:33.357 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-10T13:40:33.441 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-10T13:40:33.514 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-03-10T13:40:33.525 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-10T13:40:33.599 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-10T13:40:33.610 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-10T13:40:33.694 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-10T13:40:33.695 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-10T13:40:33.778 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-10T13:40:33.779 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-10T13:40:33.862 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-03-10T13:40:33.874 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-10T13:40:33.957 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-10T13:40:33.978 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-03-10T13:40:34.051 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-10T13:40:34.073 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-10T13:40:34.136 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-10T13:40:34.157 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-03-10T13:40:34.220 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-10T13:40:34.241 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-03-10T13:40:34.314 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-10T13:40:34.336 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-10T13:40:34.399 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-10T13:40:34.420 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-03-10T13:40:34.494 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-10T13:40:34.505 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-03-10T13:40:34.600 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-10T13:40:34.611 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-10T13:40:34.684 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-10T13:40:34.726 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-10T13:40:34.768 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-10T13:40:34.821 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-10T13:40:34.852 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-10T13:40:34.936 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-10T13:40:34.947 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-10T13:40:35.031 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-10T13:40:35.062 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-10T13:40:35.115 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-10T13:40:35.147 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-10T13:40:35.209 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-10T13:40:35.241 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-10T13:40:35.294 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-10T13:40:35.336 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-10T13:40:35.388 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-10T13:40:35.420 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-10T13:40:35.473 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-10T13:40:35.505 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-10T13:40:35.588 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-10T13:40:35.609 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-10T13:40:35.683 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-10T13:40:35.695 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-10T13:40:35.768 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-10T13:40:35.779 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-10T13:40:35.852 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-10T13:40:35.863 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-10T13:40:35.946 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-10T13:40:35.947 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-10T13:40:36.031 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-10T13:40:36.033 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-10T13:40:36.115 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-10T13:40:36.137 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-10T13:40:36.200 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-10T13:40:36.242 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-10T13:40:36.294 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-10T13:40:36.326 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-10T13:40:36.399 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-10T13:40:36.410 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-10T13:40:36.483 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-10T13:40:36.517 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-10T13:40:36.540 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:36 +0000 (0:00:03.190) 0:00:40.364 ******* 2021-03-10T13:40:36.589 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-10T13:40:36.683 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-10T13:40:36.766 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-10T13:40:36.829 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-03-10T13:40:36.831 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383636.6133442-7237-262095991004752/source", "state": "file", "uid": 0} 2021-03-10T13:40:36.852 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:36 +0000 (0:00:00.312) 0:00:40.676 ******* 2021-03-10T13:40:36.868 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:36.870 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-10T13:40:36.921 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-03-10T13:40:36.922 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => { "msg": "Host gibba011.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-03-10T13:40:36.943 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:36 +0000 (0:00:00.091) 0:00:40.767 ******* 2021-03-10T13:40:36.950 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:36.951 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-10T13:40:37.014 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-03-10T13:40:37.015 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-10T13:40:37.036 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.092) 0:00:40.859 ******* 2021-03-10T13:40:37.051 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.053 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-10T13:40:37.114 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-03-10T13:40:37.116 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.137 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.101) 0:00:40.960 ******* 2021-03-10T13:40:37.144 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.145 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-10T13:40:37.207 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.230 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.093) 0:00:41.054 ******* 2021-03-10T13:40:37.237 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.239 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-10T13:40:37.301 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.324 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.094) 0:00:41.148 ******* 2021-03-10T13:40:37.331 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.333 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-10T13:40:37.384 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.406 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.081) 0:00:41.230 ******* 2021-03-10T13:40:37.422 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.424 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-10T13:40:37.465 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.488 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.081) 0:00:41.311 ******* 2021-03-10T13:40:37.504 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.506 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-10T13:40:37.557 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.581 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.093) 0:00:41.404 ******* 2021-03-10T13:40:37.608 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.609 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-10T13:40:37.640 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-03-10T13:40:37.641 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.663 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.081) 0:00:41.486 ******* 2021-03-10T13:40:37.700 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.702 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-10T13:40:37.722 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.745 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.081) 0:00:41.568 ******* 2021-03-10T13:40:37.782 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.784 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-10T13:40:37.814 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-03-10T13:40:37.816 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:37.836 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.091) 0:00:41.660 ******* 2021-03-10T13:40:37.874 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-10T13:40:37.876 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-10T13:40:37.914 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.077) 0:00:41.737 ******* 2021-03-10T13:40:37.920 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-03-10T13:40:37.921 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:37.946 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:37 +0000 (0:00:00.032) 0:00:41.769 ******* 2021-03-10T13:40:38.014 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-03-10T13:40:38.016 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.037 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.091) 0:00:41.861 ******* 2021-03-10T13:40:38.105 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-03-10T13:40:38.106 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:38.128 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.090) 0:00:41.951 ******* 2021-03-10T13:40:38.196 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-03-10T13:40:38.198 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.230 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.102) 0:00:42.053 ******* 2021-03-10T13:40:38.237 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-03-10T13:40:38.238 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383638.0032742-7322-120754794672433/source", "state": "file", "uid": 0} 2021-03-10T13:40:38.264 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.033) 0:00:42.087 ******* 2021-03-10T13:40:38.291 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-03-10T13:40:38.292 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:38.315 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.051) 0:00:42.139 ******* 2021-03-10T13:40:38.332 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-03-10T13:40:38.334 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => { "msg": "Host gibba032.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-03-10T13:40:38.355 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.039) 0:00:42.178 ******* 2021-03-10T13:40:38.381 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-03-10T13:40:38.383 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.405 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.049) 0:00:42.228 ******* 2021-03-10T13:40:38.421 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-03-10T13:40:38.423 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-10T13:40:38.446 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.041) 0:00:42.270 ******* 2021-03-10T13:40:38.473 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-03-10T13:40:38.475 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.497 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.050) 0:00:42.320 ******* 2021-03-10T13:40:38.514 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-03-10T13:40:38.515 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.537 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.040) 0:00:42.361 ******* 2021-03-10T13:40:38.564 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-03-10T13:40:38.566 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.587 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.049) 0:00:42.410 ******* 2021-03-10T13:40:38.603 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-03-10T13:40:38.604 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.626 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.039) 0:00:42.450 ******* 2021-03-10T13:40:38.653 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-03-10T13:40:38.654 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.676 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.049) 0:00:42.499 ******* 2021-03-10T13:40:38.692 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-03-10T13:40:38.694 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.715 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.039) 0:00:42.539 ******* 2021-03-10T13:40:38.741 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-03-10T13:40:38.743 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:38.769 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.053) 0:00:42.592 ******* 2021-03-10T13:40:38.785 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-03-10T13:40:38.787 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.809 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.040) 0:00:42.632 ******* 2021-03-10T13:40:38.825 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-10T13:40:38.827 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.849 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.040) 0:00:42.673 ******* 2021-03-10T13:40:38.876 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-03-10T13:40:38.878 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.900 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.050) 0:00:42.723 ******* 2021-03-10T13:40:38.916 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-03-10T13:40:38.918 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.940 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.040) 0:00:42.764 ******* 2021-03-10T13:40:38.956 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-03-10T13:40:38.958 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:38.981 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:38 +0000 (0:00:00.040) 0:00:42.804 ******* 2021-03-10T13:40:39.007 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-03-10T13:40:39.009 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.032 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.051) 0:00:42.855 ******* 2021-03-10T13:40:39.047 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-03-10T13:40:39.049 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.071 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.039) 0:00:42.895 ******* 2021-03-10T13:40:39.098 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-03-10T13:40:39.100 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.121 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.050) 0:00:42.945 ******* 2021-03-10T13:40:39.138 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.162 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.040) 0:00:42.985 ******* 2021-03-10T13:40:39.189 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-03-10T13:40:39.190 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.216 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.054) 0:00:43.039 ******* 2021-03-10T13:40:39.232 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-03-10T13:40:39.234 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:39.257 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.040) 0:00:43.080 ******* 2021-03-10T13:40:39.283 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-03-10T13:40:39.285 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.310 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.053) 0:00:43.133 ******* 2021-03-10T13:40:39.327 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-03-10T13:40:39.328 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.351 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.040) 0:00:43.174 ******* 2021-03-10T13:40:39.387 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-03-10T13:40:39.389 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:39.411 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.060) 0:00:43.235 ******* 2021-03-10T13:40:39.418 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-03-10T13:40:39.420 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.443 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.031) 0:00:43.267 ******* 2021-03-10T13:40:39.470 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-03-10T13:40:39.472 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.494 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.050) 0:00:43.317 ******* 2021-03-10T13:40:39.510 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-03-10T13:40:39.512 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:39.535 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.040) 0:00:43.358 ******* 2021-03-10T13:40:39.551 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-03-10T13:40:39.553 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.575 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.040) 0:00:43.398 ******* 2021-03-10T13:40:39.601 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-03-10T13:40:39.603 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.625 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.050) 0:00:43.448 ******* 2021-03-10T13:40:39.641 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-03-10T13:40:39.643 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.665 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.039) 0:00:43.488 ******* 2021-03-10T13:40:39.692 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-03-10T13:40:39.693 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:39.716 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.050) 0:00:43.539 ******* 2021-03-10T13:40:39.752 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-03-10T13:40:39.754 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.777 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.061) 0:00:43.601 ******* 2021-03-10T13:40:39.784 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-03-10T13:40:39.785 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.808 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.030) 0:00:43.631 ******* 2021-03-10T13:40:39.844 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-03-10T13:40:39.846 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.872 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.063) 0:00:43.695 ******* 2021-03-10T13:40:39.878 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-03-10T13:40:39.880 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.903 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.031) 0:00:43.727 ******* 2021-03-10T13:40:39.929 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-10T13:40:39.931 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.953 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.049) 0:00:43.776 ******* 2021-03-10T13:40:39.969 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-03-10T13:40:39.970 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:39.993 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:39 +0000 (0:00:00.039) 0:00:43.816 ******* 2021-03-10T13:40:40.060 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.083 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.090) 0:00:43.906 ******* 2021-03-10T13:40:40.089 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-10T13:40:40.090 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.003154", "end": "2021-03-10 13:40:39.814659", "rc": 0, "start": "2021-03-10 13:40:39.811505", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-10T13:40:40.115 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.031) 0:00:43.938 ******* 2021-03-10T13:40:40.141 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-03-10T13:40:40.143 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:40.166 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.051) 0:00:43.989 ******* 2021-03-10T13:40:40.182 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-03-10T13:40:40.184 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.206 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.040) 0:00:44.030 ******* 2021-03-10T13:40:40.233 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-10T13:40:40.234 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.257 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.050) 0:00:44.080 ******* 2021-03-10T13:40:40.272 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-03-10T13:40:40.274 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.297 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.040) 0:00:44.120 ******* 2021-03-10T13:40:40.324 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-03-10T13:40:40.325 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.348 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.051) 0:00:44.172 ******* 2021-03-10T13:40:40.416 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.439 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.090) 0:00:44.262 ******* 2021-03-10T13:40:40.507 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.533 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.094) 0:00:44.357 ******* 2021-03-10T13:40:40.590 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-03-10T13:40:40.592 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.613 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.079) 0:00:44.436 ******* 2021-03-10T13:40:40.680 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.702 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.088) 0:00:44.525 ******* 2021-03-10T13:40:40.769 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-03-10T13:40:40.771 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:40.791 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.089) 0:00:44.615 ******* 2021-03-10T13:40:40.848 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-10T13:40:40.850 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.871 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.080) 0:00:44.695 ******* 2021-03-10T13:40:40.939 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-03-10T13:40:40.941 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:40.961 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:40 +0000 (0:00:00.089) 0:00:44.785 ******* 2021-03-10T13:40:40.967 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-10T13:40:40.967 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:41.031 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.053 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.091) 0:00:44.876 ******* 2021-03-10T13:40:41.079 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-10T13:40:41.079 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:41.132 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-03-10T13:40:41.134 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.155 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.102) 0:00:44.979 ******* 2021-03-10T13:40:41.202 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-10T13:40:41.203 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:41.230 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.075) 0:00:45.054 ******* 2021-03-10T13:40:41.237 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-03-10T13:40:41.238 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.249 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.272 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.041) 0:00:45.095 ******* 2021-03-10T13:40:41.298 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-10T13:40:41.301 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.322 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.049) 0:00:45.145 ******* 2021-03-10T13:40:41.338 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-10T13:40:41.340 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.364 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.042) 0:00:45.188 ******* 2021-03-10T13:40:41.391 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-03-10T13:40:41.393 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.415 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.050) 0:00:45.238 ******* 2021-03-10T13:40:41.472 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-03-10T13:40:41.475 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.495 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.080) 0:00:45.318 ******* 2021-03-10T13:40:41.521 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-10T13:40:41.523 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.017042", "end": "2021-03-10 13:40:41.427211", "rc": 0, "start": "2021-03-10 13:40:41.410169", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-10T13:40:41.544 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.049) 0:00:45.368 ******* 2021-03-10T13:40:41.560 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-03-10T13:40:41.562 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.584 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.039) 0:00:45.407 ******* 2021-03-10T13:40:41.611 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-03-10T13:40:41.612 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.634 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.049) 0:00:45.457 ******* 2021-03-10T13:40:41.650 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-03-10T13:40:41.652 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.674 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.039) 0:00:45.497 ******* 2021-03-10T13:40:41.701 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-03-10T13:40:41.702 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:41.724 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:41 +0000 (0:00:00.050) 0:00:45.548 ******* 2021-03-10T13:40:41.926 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-10T13:40:41.928 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:42.053 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:42.167 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:42.282 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-03-10T13:40:42.308 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.583) 0:00:46.131 ******* 2021-03-10T13:40:42.375 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-10T13:40:42.377 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.397 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.089) 0:00:46.221 ******* 2021-03-10T13:40:42.465 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.493 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.095) 0:00:46.316 ******* 2021-03-10T13:40:42.560 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-03-10T13:40:42.561 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.582 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.089) 0:00:46.406 ******* 2021-03-10T13:40:42.649 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-03-10T13:40:42.651 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.674 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.091) 0:00:46.497 ******* 2021-03-10T13:40:42.732 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.755 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.081) 0:00:46.578 ******* 2021-03-10T13:40:42.771 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-03-10T13:40:42.773 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.009583", "end": "2021-03-10 13:40:42.500912", "rc": 0, "start": "2021-03-10 13:40:41.491329", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:42.798 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.042) 0:00:46.621 ******* 2021-03-10T13:40:42.857 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-03-10T13:40:42.858 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:42.879 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:42 +0000 (0:00:00.081) 0:00:46.703 ******* 2021-03-10T13:40:43.030 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-03-10T13:40:43.031 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.007466", "end": "2021-03-10 13:40:42.759926", "rc": 0, "start": "2021-03-10 13:40:42.752460", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:43.052 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:43 +0000 (0:00:00.172) 0:00:46.876 ******* 2021-03-10T13:40:43.585 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-03-10T13:40:43.587 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.739730", "end": "2021-03-10 13:40:43.500884", "rc": 0, "start": "2021-03-10 13:40:42.761154", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:43.609 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:43 +0000 (0:00:00.556) 0:00:47.433 ******* 2021-03-10T13:40:43.677 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-03-10T13:40:43.679 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:43.700 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:43 +0000 (0:00:00.090) 0:00:47.523 ******* 2021-03-10T13:40:43.860 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-03-10T13:40:43.862 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.007662", "end": "2021-03-10 13:40:43.768025", "rc": 0, "start": "2021-03-10 13:40:43.760363", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:43.883 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:43 +0000 (0:00:00.183) 0:00:47.707 ******* 2021-03-10T13:40:44.907 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-03-10T13:40:44.909 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:40:44.929 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:44 +0000 (0:00:01.044) 0:00:48.752 ******* 2021-03-10T13:40:45.380 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-03-10T13:40:45.381 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:40:45.402 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:45 +0000 (0:00:00.473) 0:00:49.225 ******* 2021-03-10T13:40:45.429 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-03-10T13:40:45.430 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:40:45.453 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:45 +0000 (0:00:00.051) 0:00:49.277 ******* 2021-03-10T13:40:45.469 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-03-10T13:40:45.470 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:45.492 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:45 +0000 (0:00:00.039) 0:00:49.316 ******* 2021-03-10T13:40:45.894 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-03-10T13:40:45.896 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:40:45.919 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:45 +0000 (0:00:00.426) 0:00:49.742 ******* 2021-03-10T13:40:45.996 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-03-10T13:40:45.998 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:46.020 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:46 +0000 (0:00:00.101) 0:00:49.843 ******* 2021-03-10T13:40:46.058 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-03-10T13:40:46.060 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-03-10T13:40:46.083 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:46 +0000 (0:00:00.063) 0:00:49.907 ******* 2021-03-10T13:40:46.298 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-03-10T13:40:46.325 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:46 +0000 (0:00:00.241) 0:00:50.148 ******* 2021-03-10T13:40:46.528 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-03-10T13:40:46.530 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:40:46.554 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:46 +0000 (0:00:00.228) 0:00:50.377 ******* 2021-03-10T13:40:46.819 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:40:46.843 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:46 +0000 (0:00:00.289) 0:00:50.666 ******* 2021-03-10T13:40:46.891 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-03-10T13:40:46.893 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-10T13:40:47.141 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-10T13:40:47.152 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-10T13:40:47.370 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-10T13:40:47.396 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:47 +0000 (0:00:00.553) 0:00:51.220 ******* 2021-03-10T13:40:47.414 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-10T13:40:47.487 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-03-10T13:40:47.489 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:40:47.512 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:47 +0000 (0:00:00.115) 0:00:51.335 ******* 2021-03-10T13:40:47.622 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-03-10T13:40:47.624 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-10T13:40:47.648 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:47 +0000 (0:00:00.136) 0:00:51.472 ******* 2021-03-10T13:40:47.780 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-03-10T13:40:47.782 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:40:47.805 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:47 +0000 (0:00:00.156) 0:00:51.628 ******* 2021-03-10T13:40:47.812 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-03-10T13:40:47.813 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-10T13:40:48.012 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-10T13:40:48.086 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-10T13:40:48.202 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-10T13:40:48.227 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.421) 0:00:52.050 ******* 2021-03-10T13:40:48.286 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-10T13:40:48.297 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-03-10T13:40:48.299 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-10T13:40:48.323 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.096) 0:00:52.147 ******* 2021-03-10T13:40:48.391 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-03-10T13:40:48.393 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.414 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.090) 0:00:52.238 ******* 2021-03-10T13:40:48.473 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-03-10T13:40:48.475 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-10T13:40:48.499 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.084) 0:00:52.322 ******* 2021-03-10T13:40:48.505 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-03-10T13:40:48.507 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:48.531 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.032) 0:00:52.355 ******* 2021-03-10T13:40:48.590 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-03-10T13:40:48.591 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-10T13:40:48.615 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.083) 0:00:52.438 ******* 2021-03-10T13:40:48.632 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-03-10T13:40:48.633 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.643 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.654 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.679 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.064) 0:00:52.502 ******* 2021-03-10T13:40:48.685 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-03-10T13:40:48.687 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.711 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.032) 0:00:52.535 ******* 2021-03-10T13:40:48.759 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-03-10T13:40:48.762 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.784 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.072) 0:00:52.607 ******* 2021-03-10T13:40:48.789 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-03-10T13:40:48.791 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:48.815 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.031) 0:00:52.639 ******* 2021-03-10T13:40:48.895 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-03-10T13:40:48.897 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.906 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.908 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.917 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:48.941 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:48 +0000 (0:00:00.125) 0:00:52.765 ******* 2021-03-10T13:40:49.009 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-03-10T13:40:49.011 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:49.032 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.090) 0:00:52.855 ******* 2021-03-10T13:40:49.255 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-03-10T13:40:49.257 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:40:49.280 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.248) 0:00:53.103 ******* 2021-03-10T13:40:49.499 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:40:49.525 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.245) 0:00:53.349 ******* 2021-03-10T13:40:49.584 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-03-10T13:40:49.586 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-10T13:40:49.609 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.083) 0:00:53.432 ******* 2021-03-10T13:40:49.803 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-10T13:40:49.827 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.218) 0:00:53.651 ******* 2021-03-10T13:40:49.917 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-03-10T13:40:49.918 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-10T13:40:49.952 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:49 +0000 (0:00:00.124) 0:00:53.775 ******* 2021-03-10T13:40:50.116 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-10T13:40:50.118 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-10T13:40:50.138 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-03-10T13:40:50.139 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-10T13:40:50.163 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.211) 0:00:53.986 ******* 2021-03-10T13:40:50.190 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-10T13:40:50.192 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-10T13:40:50.226 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.063) 0:00:54.049 ******* 2021-03-10T13:40:50.295 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-03-10T13:40:50.297 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:50.320 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.093) 0:00:54.143 ******* 2021-03-10T13:40:50.327 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-10T13:40:50.328 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-10T13:40:50.391 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:50.393 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [gibba032.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-10T13:40:50.415 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.095) 0:00:54.239 ******* 2021-03-10T13:40:50.448 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.032) 0:00:54.271 ******* 2021-03-10T13:40:50.487 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-03-10T13:40:50.488 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:50.510 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.062) 0:00:54.334 ******* 2021-03-10T13:40:50.527 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-03-10T13:40:50.529 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:50.555 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.044) 0:00:54.378 ******* 2021-03-10T13:40:50.626 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-03-10T13:40:50.628 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-10T13:40:50.661 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.105) 0:00:54.484 ******* 2021-03-10T13:40:50.732 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-03-10T13:40:50.733 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:50.756 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.094) 0:00:54.579 ******* 2021-03-10T13:40:50.782 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-03-10T13:40:50.784 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-10T13:40:50.808 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:50 +0000 (0:00:00.052) 0:00:54.631 ******* 2021-03-10T13:40:51.003 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-03-10T13:40:51.004 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:05 UTC", "ActiveEnterTimestampMonotonic": "14557023", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-journald.socket local-fs.target -.mount sysinit.target nss-lookup.target system.slice var-run.mount remote-fs.target time-sync.target basic.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:05 UTC", "AssertTimestampMonotonic": "14535010", "Before": "multi-user.target xdm.service getty@tty1.service shutdown.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:05 UTC", "Con 2021-03-10T13:40:51.006 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "14535010", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1375", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:05 UTC", "ExecMainStartTimestampMonotonic": "14556908", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", 2021-03-10T13:40:51.007 INFO:teuthology.task.ansible.out: "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:05 UTC", "InactiveExitTimestampMonotonic": "14557023", "InvocationID": "c991330b305242c0b5df5500e186c829", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1375", "MemoryAccounting": "yes", "MemoryCurrent": "1269760", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", " 2021-03-10T13:40:51.009 INFO:teuthology.task.ansible.out:MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:05 UTC", "StateChangeTimestampMonotonic": "14557023", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", " 2021-03-10T13:40:51.010 INFO:teuthology.task.ansible.out:SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38348", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:40:51.028 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.220) 0:00:54.851 ******* 2021-03-10T13:40:51.034 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-03-10T13:40:51.036 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-10T13:40:51.061 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.032) 0:00:54.884 ******* 2021-03-10T13:40:51.119 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-03-10T13:40:51.121 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.143 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.082) 0:00:54.967 ******* 2021-03-10T13:40:51.212 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-03-10T13:40:51.214 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.238 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.094) 0:00:55.061 ******* 2021-03-10T13:40:51.265 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-03-10T13:40:51.266 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:12 UTC", "ActiveEnterTimestampMonotonic": "14059487", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service systemd-journald.socket var-run.mount basic.target sysinit.target local-fs.target -.mount remote-fs.target network.target system.slice time-sync.target nss-lookup.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:12 UTC", "AssertTimestampMonotonic": "14043025", "Before": "multi-user.target getty@tty1.service shutdown.target xdm.service plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:12 UTC", "Con 2021-03-10T13:40:51.268 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "14043024", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1359", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:12 UTC", "ExecMainStartTimestampMonotonic": "14059382", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", 2021-03-10T13:40:51.269 INFO:teuthology.task.ansible.out: "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:12 UTC", "InactiveExitTimestampMonotonic": "14059487", "InvocationID": "e83f21c0a58f4c01b62a6131780ffb35", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1359", "MemoryAccounting": "yes", "MemoryCurrent": "1269760", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", " 2021-03-10T13:40:51.271 INFO:teuthology.task.ansible.out:MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:12 UTC", "StateChangeTimestampMonotonic": "14059487", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", " 2021-03-10T13:40:51.272 INFO:teuthology.task.ansible.out:SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38348", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:40:51.289 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.051) 0:00:55.113 ******* 2021-03-10T13:40:51.315 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-03-10T13:40:51.317 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.337 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.047) 0:00:55.160 ******* 2021-03-10T13:40:51.363 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-03-10T13:40:51.365 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.389 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.051) 0:00:55.212 ******* 2021-03-10T13:40:51.405 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-03-10T13:40:51.407 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.430 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.040) 0:00:55.253 ******* 2021-03-10T13:40:51.478 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-03-10T13:40:51.480 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.500 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.070) 0:00:55.324 ******* 2021-03-10T13:40:51.506 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-03-10T13:40:51.507 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.529 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.028) 0:00:55.353 ******* 2021-03-10T13:40:51.566 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-03-10T13:40:51.567 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.588 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.059) 0:00:55.412 ******* 2021-03-10T13:40:51.614 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-03-10T13:40:51.616 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.637 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.048) 0:00:55.460 ******* 2021-03-10T13:40:51.652 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-03-10T13:40:51.654 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.675 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.038) 0:00:55.498 ******* 2021-03-10T13:40:51.701 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-03-10T13:40:51.703 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.724 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.048) 0:00:55.547 ******* 2021-03-10T13:40:51.874 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-03-10T13:40:51.876 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.897 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.172) 0:00:55.720 ******* 2021-03-10T13:40:51.924 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-03-10T13:40:51.925 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:51.947 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:51 +0000 (0:00:00.050) 0:00:55.770 ******* 2021-03-10T13:40:52.087 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-03-10T13:40:52.089 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.110 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.162) 0:00:55.933 ******* 2021-03-10T13:40:52.136 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-03-10T13:40:52.138 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.161 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.050) 0:00:55.984 ******* 2021-03-10T13:40:52.197 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-03-10T13:40:52.199 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.225 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.064) 0:00:56.048 ******* 2021-03-10T13:40:52.242 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-03-10T13:40:52.244 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.267 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.041) 0:00:56.090 ******* 2021-03-10T13:40:52.293 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-03-10T13:40:52.295 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.318 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.050) 0:00:56.141 ******* 2021-03-10T13:40:52.334 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-03-10T13:40:52.336 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-10T13:40:52.359 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.041) 0:00:56.182 ******* 2021-03-10T13:40:52.386 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-03-10T13:40:52.388 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.411 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.051) 0:00:56.234 ******* 2021-03-10T13:40:52.427 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-03-10T13:40:52.429 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-10T13:40:52.453 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.042) 0:00:56.277 ******* 2021-03-10T13:40:52.480 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-03-10T13:40:52.482 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:52.504 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.050) 0:00:56.328 ******* 2021-03-10T13:40:52.531 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-03-10T13:40:52.533 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-03-10T13:40:52.556 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.051) 0:00:56.379 ******* 2021-03-10T13:40:52.698 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-03-10T13:40:52.700 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-10T13:40:52.721 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.165) 0:00:56.545 ******* 2021-03-10T13:40:52.748 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-03-10T13:40:52.750 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-10T13:40:52.771 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.050) 0:00:56.595 ******* 2021-03-10T13:40:52.870 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-03-10T13:40:52.871 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-10T13:40:52.892 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.120) 0:00:56.715 ******* 2021-03-10T13:40:52.909 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-03-10T13:40:52.910 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-10T13:40:52.934 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:52 +0000 (0:00:00.042) 0:00:56.758 ******* 2021-03-10T13:40:53.117 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-03-10T13:40:53.119 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-03-10T13:40:53.144 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.209) 0:00:56.967 ******* 2021-03-10T13:40:53.150 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-03-10T13:40:53.152 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-10T13:40:53.282 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.138) 0:00:57.106 ******* 2021-03-10T13:40:53.320 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-03-10T13:40:53.322 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-10T13:40:53.344 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.061) 0:00:57.167 ******* 2021-03-10T13:40:53.360 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-03-10T13:40:53.362 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:53.385 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.041) 0:00:57.209 ******* 2021-03-10T13:40:53.443 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-03-10T13:40:53.444 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-10T13:40:53.465 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.079) 0:00:57.288 ******* 2021-03-10T13:40:53.470 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-03-10T13:40:53.472 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:53.495 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.030) 0:00:57.318 ******* 2021-03-10T13:40:53.604 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-03-10T13:40:53.606 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-10T13:40:53.628 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.132) 0:00:57.451 ******* 2021-03-10T13:40:53.654 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-03-10T13:40:53.656 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-10T13:40:53.678 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.050) 0:00:57.501 ******* 2021-03-10T13:40:53.695 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-03-10T13:40:53.696 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:53.719 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.040) 0:00:57.542 ******* 2021-03-10T13:40:53.786 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-03-10T13:40:53.788 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:53.809 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.090) 0:00:57.633 ******* 2021-03-10T13:40:53.980 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-03-10T13:40:53.981 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-10T13:40:54.002 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:53 +0000 (0:00:00.192) 0:00:57.825 ******* 2021-03-10T13:40:54.028 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-03-10T13:40:54.030 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-10T13:40:54.052 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.050) 0:00:57.876 ******* 2021-03-10T13:40:54.131 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-03-10T13:40:54.132 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba011"}, "changed": false} 2021-03-10T13:40:54.154 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.101) 0:00:57.977 ******* 2021-03-10T13:40:54.347 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-03-10T13:40:54.349 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-10T13:40:54.375 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.220) 0:00:58.198 ******* 2021-03-10T13:40:54.443 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-03-10T13:40:54.445 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba032"}, "changed": false} 2021-03-10T13:40:54.466 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.091) 0:00:58.289 ******* 2021-03-10T13:40:54.803 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-03-10T13:40:54.807 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "gibba011", "ansible_hostname": "gibba011", "ansible_nodename": "gibba011"}, "changed": false, "name": "gibba011"} 2021-03-10T13:40:54.829 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.362) 0:00:58.652 ******* 2021-03-10T13:40:54.835 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "gibba032", "ansible_hostname": "gibba032", "ansible_nodename": "gibba032"}, "changed": false, "name": "gibba032"} 2021-03-10T13:40:54.860 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:54 +0000 (0:00:00.031) 0:00:58.683 ******* 2021-03-10T13:40:55.115 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-03-10T13:40:55.116 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-10T13:40:55.139 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.278) 0:00:58.962 ******* 2021-03-10T13:40:55.146 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-10T13:40:55.172 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.032) 0:00:58.995 ******* 2021-03-10T13:40:55.439 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-03-10T13:40:55.441 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-10T13:40:55.462 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.289) 0:00:59.285 ******* 2021-03-10T13:40:55.468 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-10T13:40:55.493 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.031) 0:00:59.316 ******* 2021-03-10T13:40:55.851 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-03-10T13:40:55.853 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-10T13:40:55.877 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.383) 0:00:59.700 ******* 2021-03-10T13:40:55.884 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-10T13:40:55.908 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.031) 0:00:59.732 ******* 2021-03-10T13:40:55.935 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-03-10T13:40:55.937 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:55.959 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.050) 0:00:59.782 ******* 2021-03-10T13:40:55.975 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.000 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:55 +0000 (0:00:00.040) 0:00:59.823 ******* 2021-03-10T13:40:56.027 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-10T13:40:56.028 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.050 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.050) 0:00:59.874 ******* 2021-03-10T13:40:56.067 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.091 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.040) 0:00:59.914 ******* 2021-03-10T13:40:56.117 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-03-10T13:40:56.118 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.140 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.049) 0:00:59.963 ******* 2021-03-10T13:40:56.155 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.179 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.039) 0:01:00.002 ******* 2021-03-10T13:40:56.216 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-10T13:40:56.217 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.238 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.059) 0:01:00.062 ******* 2021-03-10T13:40:56.254 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.277 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.038) 0:01:00.101 ******* 2021-03-10T13:40:56.304 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-10T13:40:56.306 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.328 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.050) 0:01:00.151 ******* 2021-03-10T13:40:56.343 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.368 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.039) 0:01:00.191 ******* 2021-03-10T13:40:56.395 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-10T13:40:56.397 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.422 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.054) 0:01:00.246 ******* 2021-03-10T13:40:56.439 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.462 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.039) 0:01:00.286 ******* 2021-03-10T13:40:56.477 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-10T13:40:56.479 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.502 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.039) 0:01:00.325 ******* 2021-03-10T13:40:56.528 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.552 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.050) 0:01:00.376 ******* 2021-03-10T13:40:56.569 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-03-10T13:40:56.570 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.593 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.041) 0:01:00.417 ******* 2021-03-10T13:40:56.621 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.645 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.051) 0:01:00.468 ******* 2021-03-10T13:40:56.661 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-03-10T13:40:56.686 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.041) 0:01:00.510 ******* 2021-03-10T13:40:56.738 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.051) 0:01:00.561 ******* 2021-03-10T13:40:56.754 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-03-10T13:40:56.779 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.041) 0:01:00.602 ******* 2021-03-10T13:40:56.840 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.061) 0:01:00.663 ******* 2021-03-10T13:40:56.857 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-03-10T13:40:56.881 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.041) 0:01:00.705 ******* 2021-03-10T13:40:56.932 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.050) 0:01:00.755 ******* 2021-03-10T13:40:56.948 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-03-10T13:40:56.950 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:56.976 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:56 +0000 (0:00:00.044) 0:01:00.799 ******* 2021-03-10T13:40:57.004 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.028 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.051) 0:01:00.851 ******* 2021-03-10T13:40:57.044 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-03-10T13:40:57.046 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.055 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.080 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.052) 0:01:00.903 ******* 2021-03-10T13:40:57.096 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.098 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.121 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.040) 0:01:00.944 ******* 2021-03-10T13:40:57.137 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-03-10T13:40:57.139 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.162 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.041) 0:01:00.985 ******* 2021-03-10T13:40:57.189 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.213 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.051) 0:01:01.036 ******* 2021-03-10T13:40:57.230 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:40:57.232 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.254 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.040) 0:01:01.077 ******* 2021-03-10T13:40:57.280 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.304 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.050) 0:01:01.128 ******* 2021-03-10T13:40:57.321 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-03-10T13:40:57.322 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.345 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.040) 0:01:01.168 ******* 2021-03-10T13:40:57.375 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.399 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.054) 0:01:01.223 ******* 2021-03-10T13:40:57.416 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-03-10T13:40:57.418 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.442 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.042) 0:01:01.265 ******* 2021-03-10T13:40:57.468 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.490 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.048) 0:01:01.314 ******* 2021-03-10T13:40:57.505 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-03-10T13:40:57.507 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.532 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.041) 0:01:01.355 ******* 2021-03-10T13:40:57.559 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.582 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.050) 0:01:01.406 ******* 2021-03-10T13:40:57.609 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-03-10T13:40:57.611 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.633 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.050) 0:01:01.456 ******* 2021-03-10T13:40:57.649 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.672 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.039) 0:01:01.495 ******* 2021-03-10T13:40:57.687 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-10T13:40:57.689 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.711 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.038) 0:01:01.534 ******* 2021-03-10T13:40:57.738 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.761 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.049) 0:01:01.584 ******* 2021-03-10T13:40:57.777 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-10T13:40:57.779 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.801 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.039) 0:01:01.624 ******* 2021-03-10T13:40:57.826 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.850 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.048) 0:01:01.673 ******* 2021-03-10T13:40:57.865 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-10T13:40:57.867 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.888 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.038) 0:01:01.712 ******* 2021-03-10T13:40:57.914 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.938 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.049) 0:01:01.762 ******* 2021-03-10T13:40:57.966 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-03-10T13:40:57.968 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:57.990 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:57 +0000 (0:00:00.051) 0:01:01.814 ******* 2021-03-10T13:40:58.007 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.030 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.040) 0:01:01.854 ******* 2021-03-10T13:40:58.057 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-10T13:40:58.058 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.083 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.052) 0:01:01.907 ******* 2021-03-10T13:40:58.100 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.124 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.040) 0:01:01.947 ******* 2021-03-10T13:40:58.140 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-10T13:40:58.142 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.163 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.039) 0:01:01.987 ******* 2021-03-10T13:40:58.190 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.213 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.049) 0:01:02.036 ******* 2021-03-10T13:40:58.229 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-03-10T13:40:58.230 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.253 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.039) 0:01:02.076 ******* 2021-03-10T13:40:58.279 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.303 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.049) 0:01:02.126 ******* 2021-03-10T13:40:58.319 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-03-10T13:40:58.320 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.342 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.039) 0:01:02.166 ******* 2021-03-10T13:40:58.369 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.391 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.049) 0:01:02.215 ******* 2021-03-10T13:40:58.407 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-03-10T13:40:58.409 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.432 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.040) 0:01:02.255 ******* 2021-03-10T13:40:58.459 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.484 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.051) 0:01:02.307 ******* 2021-03-10T13:40:58.501 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-03-10T13:40:58.502 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.526 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.042) 0:01:02.349 ******* 2021-03-10T13:40:58.563 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.586 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.059) 0:01:02.409 ******* 2021-03-10T13:40:58.603 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.630 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.043) 0:01:02.453 ******* 2021-03-10T13:40:58.657 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.681 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.051) 0:01:02.504 ******* 2021-03-10T13:40:58.697 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-03-10T13:40:58.699 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.723 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.041) 0:01:02.546 ******* 2021-03-10T13:40:58.750 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.773 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.050) 0:01:02.596 ******* 2021-03-10T13:40:58.789 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-03-10T13:40:58.791 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.812 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.039) 0:01:02.636 ******* 2021-03-10T13:40:58.839 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.862 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.049) 0:01:02.686 ******* 2021-03-10T13:40:58.879 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-03-10T13:40:58.880 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.903 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.040) 0:01:02.727 ******* 2021-03-10T13:40:58.931 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:58.954 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:58 +0000 (0:00:00.050) 0:01:02.777 ******* 2021-03-10T13:40:59.044 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-03-10T13:40:59.046 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001304", "end": "2021-03-10 13:40:58.957868", "rc": 0, "start": "2021-03-10 13:40:58.956564", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:59.069 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.114) 0:01:02.892 ******* 2021-03-10T13:40:59.096 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001356", "end": "2021-03-10 13:40:58.826008", "rc": 0, "start": "2021-03-10 13:40:58.824652", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:40:59.119 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.050) 0:01:02.943 ******* 2021-03-10T13:40:59.136 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-03-10T13:40:59.160 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.040) 0:01:02.983 ******* 2021-03-10T13:40:59.209 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.049) 0:01:03.032 ******* 2021-03-10T13:40:59.444 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-03-10T13:40:59.445 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-10T13:40:59.470 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.261) 0:01:03.294 ******* 2021-03-10T13:40:59.498 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-10T13:40:59.523 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.052) 0:01:03.346 ******* 2021-03-10T13:40:59.539 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-03-10T13:40:59.541 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:59.563 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.040) 0:01:03.386 ******* 2021-03-10T13:40:59.589 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:40:59.613 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:40:59 +0000 (0:00:00.050) 0:01:03.437 ******* 2021-03-10T13:41:00.415 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-03-10T13:41:00.417 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-03-10T13:41:00.530 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-03-10T13:41:01.058 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-10T13:41:01.084 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:01.470) 0:01:04.907 ******* 2021-03-10T13:41:01.255 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-10T13:41:01.279 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.194) 0:01:05.102 ******* 2021-03-10T13:41:01.522 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-03-10T13:41:01.524 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:01.549 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.270) 0:01:05.372 ******* 2021-03-10T13:41:01.608 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-03-10T13:41:01.631 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.081) 0:01:05.454 ******* 2021-03-10T13:41:01.729 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-03-10T13:41:01.731 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:01.754 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.122) 0:01:05.577 ******* 2021-03-10T13:41:01.781 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-03-10T13:41:01.783 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.010309", "end": "2021-03-10 13:41:01.508341", "rc": 0, "start": "2021-03-10 13:41:01.498032", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:01.806 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.052) 0:01:05.630 ******* 2021-03-10T13:41:01.823 INFO:teuthology.task.ansible.out: 2021-03-10T13:41:01.825 INFO:teuthology.task.ansible.out:TASK [testnode : Add local apt repos.] ***************************************** 2021-03-10T13:41:01.850 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:01 +0000 (0:00:00.043) 0:01:05.673 ******* 2021-03-10T13:41:01.851 INFO:teuthology.task.ansible.out: 2021-03-10T13:41:02.012 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-03-10T13:41:02.013 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011115", "end": "2021-03-10 13:41:01.920992", "rc": 0, "start": "2021-03-10 13:41:01.909877", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:02.037 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:02 +0000 (0:00:00.186) 0:01:05.860 ******* 2021-03-10T13:41:03.346 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-03-10T13:41:03.348 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:03.369 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:03 +0000 (0:00:01.332) 0:01:07.192 ******* 2021-03-10T13:41:03.679 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:03.706 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:03 +0000 (0:00:00.336) 0:01:07.529 ******* 2021-03-10T13:41:11.835 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-03-10T13:41:11.837 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101691 files and directories currently installed.)\r\nR 2021-03-10T13:41:11.839 INFO:teuthology.task.ansible.out:emoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading 2021-03-10T13:41:11.840 INFO:teuthology.task.ansible.out: database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101691 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-03-10T13:41:11.858 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:11 +0000 (0:00:08.152) 0:01:15.682 ******* 2021-03-10T13:41:12.299 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-10T13:41:12.301 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:12.321 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:12 +0000 (0:00:00.462) 0:01:16.145 ******* 2021-03-10T13:41:12.388 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-03-10T13:41:12.390 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:12.411 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:12 +0000 (0:00:00.089) 0:01:16.234 ******* 2021-03-10T13:41:12.541 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-03-10T13:41:12.542 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101691 files and directories currently installed.)\r\nR 2021-03-10T13:41:12.544 INFO:teuthology.task.ansible.out:emoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading 2021-03-10T13:41:12.545 INFO:teuthology.task.ansible.out: database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101691 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-03-10T13:41:12.565 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:12 +0000 (0:00:00.153) 0:01:16.388 ******* 2021-03-10T13:41:13.003 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-10T13:41:13.005 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:13.025 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:13 +0000 (0:00:00.459) 0:01:16.848 ******* 2021-03-10T13:41:13.092 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-03-10T13:41:13.094 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:13.114 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:13 +0000 (0:00:00.089) 0:01:16.938 ******* 2021-03-10T13:41:22.261 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-10T13:41:22.263 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 8 not upgraded.\nNeed to get 18.2 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.8-0ubuntu0.20.04.1 [3209 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.8-0ubuntu0.20.04.1 [1615 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.14 [53.1 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.14 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.14 [6726 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.14 [975 kB]\nFetched 18.2 MB in 1s (13.9 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading databas 2021-03-10T13:41:22.265 INFO:teuthology.task.ansible.out:e ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101619 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.8-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.8-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.14) ...\r\nSetting up librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) 2021-03-10T13:41:22.266 INFO:teuthology.task.ansible.out:...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.14) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 8 not upgraded.", "Need to get 18.2 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.8-0ubuntu0.20.04.1 [3209 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.8-0ubuntu0.20.04.1 [1615 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.14 [53.1 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.14 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.14 [6726 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.14 [975 kB]", "Fetched 18.2 MB in 1s (13.9 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", " 2021-03-10T13:41:22.268 INFO:teuthology.task.ansible.out:(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101619 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.8-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.8-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.14) ...", "Setting up librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Setting up git (1:2.25.1-1ubuntu3.1) ...", "Setting up librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Sett 2021-03-10T13:41:22.270 INFO:teuthology.task.ansible.out:ing up qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.14) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Setting up qemu-utils (1 2021-03-10T13:41:22.270 INFO:teuthology.task.ansible.out::4.2-3ubuntu6.14) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-03-10T13:41:22.292 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:22 +0000 (0:00:09.178) 0:01:26.116 ******* 2021-03-10T13:41:22.733 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 8 not upgraded.\nNeed to get 18.2 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.8-0ubuntu0.20.04.1 [3209 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.8-0ubuntu0.20.04.1 [1615 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.14 [53.1 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.14 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.14 [6726 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.14 [975 kB]\nFetched 18.2 MB in 2s (7700 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading databas 2021-03-10T13:41:22.735 INFO:teuthology.task.ansible.out:e ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101619 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.8-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.8-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a4.2-3ubuntu6.14_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.14) ...\r\nSetting up librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) 2021-03-10T13:41:22.736 INFO:teuthology.task.ansible.out:...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.14) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 8 not upgraded.", "Need to get 18.2 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.8-0ubuntu0.20.04.1 [3209 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.8-0ubuntu0.20.04.1 [1615 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.14 [53.1 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.14 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.14 [6726 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.14 [975 kB]", "Fetched 18.2 MB in 2s (7700 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", " 2021-03-10T13:41:22.738 INFO:teuthology.task.ansible.out:(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101619 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.8-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.8-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a4.2-3ubuntu6.14_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.14) ...", "Setting up librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Setting up git (1:2.25.1-1ubuntu3.1) 2021-03-10T13:41:22.738 INFO:teuthology.task.ansible.out: ...", "Setting up librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Sett 2021-03-10T13:41:22.740 INFO:teuthology.task.ansible.out:ing up qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.14) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.14) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-03-10T13:41:22.762 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:22 +0000 (0:00:00.469) 0:01:26.585 ******* 2021-03-10T13:41:22.779 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-03-10T13:41:22.780 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:22.803 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:22 +0000 (0:00:00.041) 0:01:26.626 ******* 2021-03-10T13:41:23.203 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:23.225 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.422) 0:01:27.049 ******* 2021-03-10T13:41:23.241 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-03-10T13:41:23.243 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:23.265 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.040) 0:01:27.089 ******* 2021-03-10T13:41:23.447 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-03-10T13:41:23.449 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-10T13:41:23.469 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.203) 0:01:27.293 ******* 2021-03-10T13:41:23.536 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-03-10T13:41:23.537 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:23.558 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.088) 0:01:27.382 ******* 2021-03-10T13:41:23.666 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-03-10T13:41:23.667 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:23.688 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.130) 0:01:27.512 ******* 2021-03-10T13:41:23.704 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-03-10T13:41:23.706 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-10T13:41:23.728 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.039) 0:01:27.551 ******* 2021-03-10T13:41:23.827 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-03-10T13:41:23.828 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-10T13:41:23.849 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.121) 0:01:27.673 ******* 2021-03-10T13:41:23.916 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:23.939 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:23 +0000 (0:00:00.089) 0:01:27.762 ******* 2021-03-10T13:41:24.017 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-03-10T13:41:24.019 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-10T13:41:24.040 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.101) 0:01:27.863 ******* 2021-03-10T13:41:24.139 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-03-10T13:41:24.140 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-10T13:41:24.166 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.126) 0:01:27.989 ******* 2021-03-10T13:41:24.214 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-03-10T13:41:24.216 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-10T13:41:24.239 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.073) 0:01:28.063 ******* 2021-03-10T13:41:24.380 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-03-10T13:41:24.381 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:24.403 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.163) 0:01:28.226 ******* 2021-03-10T13:41:24.460 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-03-10T13:41:24.462 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-10T13:41:24.484 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.081) 0:01:28.307 ******* 2021-03-10T13:41:24.615 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-03-10T13:41:24.616 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:13 UTC", "ActiveEnterTimestampMonotonic": "15111137", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target -.mount systemd-journald.socket systemd-tmpfiles-setup.service nss-lookup.target sysinit.target network.target system.slice remote-fs.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:12 UTC", "AssertTimestampMonotonic": "13954812", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:12 UTC", "ConditionTimestampMonotonic": "13954812", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": 2021-03-10T13:41:24.618 INFO:teuthology.task.ansible.out: "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1521", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:13 UTC", "ExecMainStartTimestampMonotonic": "15111119", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709 2021-03-10T13:41:24.619 INFO:teuthology.task.ansible.out:551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:12 UTC", "InactiveExitTimestampMonotonic": "13955795", "InvocationID": "27ff87f7615349e79280bf054d17b126", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1521", "MemoryAccounting": "yes", "MemoryCurrent": "10862592", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPoli 2021-03-10T13:41:24.621 INFO:teuthology.task.ansible.out:cy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:13 UTC", "StateChangeTimestampMonotonic": "15111137", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38348", "TimeoutAbortUSec": 2021-03-10T13:41:24.623 INFO:teuthology.task.ansible.out: "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:41:24.640 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.156) 0:01:28.463 ******* 2021-03-10T13:41:24.678 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-03-10T13:41:24.680 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-10T13:41:24.700 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.060) 0:01:28.524 ******* 2021-03-10T13:41:24.706 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-10T13:41:24.708 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:24.730 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.029) 0:01:28.554 ******* 2021-03-10T13:41:24.798 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:24.820 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.089) 0:01:28.643 ******* 2021-03-10T13:41:24.845 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-03-10T13:41:24.847 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:24.868 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.048) 0:01:28.692 ******* 2021-03-10T13:41:24.884 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-10T13:41:24.886 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:24.908 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.039) 0:01:28.731 ******* 2021-03-10T13:41:24.975 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-10T13:41:24.977 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:24.998 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:24 +0000 (0:00:00.089) 0:01:28.821 ******* 2021-03-10T13:41:25.064 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-10T13:41:25.066 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.090 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.092) 0:01:28.914 ******* 2021-03-10T13:41:25.097 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-03-10T13:41:25.099 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:06 UTC", "ActiveEnterTimestampMonotonic": "15544091", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount remote-fs.target systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket basic.target system.slice nss-lookup.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:05 UTC", "AssertTimestampMonotonic": "14454954", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:05 UTC", "ConditionTimestampMonotonic": "14454953", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": 2021-03-10T13:41:25.101 INFO:teuthology.task.ansible.out: "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1536", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:06 UTC", "ExecMainStartTimestampMonotonic": "15544078", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709 2021-03-10T13:41:25.102 INFO:teuthology.task.ansible.out:551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:05 UTC", "InactiveExitTimestampMonotonic": "14455590", "InvocationID": "7be7943bfab94223a816f9831372b66e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1536", "MemoryAccounting": "yes", "MemoryCurrent": "10547200", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPoli 2021-03-10T13:41:25.104 INFO:teuthology.task.ansible.out:cy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:06 UTC", "StateChangeTimestampMonotonic": "15544091", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38348", "TimeoutAbortUSec": 2021-03-10T13:41:25.105 INFO:teuthology.task.ansible.out: "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:41:25.125 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.034) 0:01:28.948 ******* 2021-03-10T13:41:25.151 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-10T13:41:25.152 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.174 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.049) 0:01:28.997 ******* 2021-03-10T13:41:25.190 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-10T13:41:25.191 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.214 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.040) 0:01:29.038 ******* 2021-03-10T13:41:25.230 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-10T13:41:25.232 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.254 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.040) 0:01:29.078 ******* 2021-03-10T13:41:25.270 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-03-10T13:41:25.272 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.295 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.040) 0:01:29.118 ******* 2021-03-10T13:41:25.322 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-10T13:41:25.324 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.346 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.051) 0:01:29.170 ******* 2021-03-10T13:41:25.363 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-10T13:41:25.364 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.386 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.040) 0:01:29.210 ******* 2021-03-10T13:41:25.413 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:25.415 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.436 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.049) 0:01:29.259 ******* 2021-03-10T13:41:25.452 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-10T13:41:25.454 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.476 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.039) 0:01:29.299 ******* 2021-03-10T13:41:25.503 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-10T13:41:25.505 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.526 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.049) 0:01:29.349 ******* 2021-03-10T13:41:25.542 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-10T13:41:25.544 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.566 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.039) 0:01:29.389 ******* 2021-03-10T13:41:25.597 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-10T13:41:25.599 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.624 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.058) 0:01:29.447 ******* 2021-03-10T13:41:25.641 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-10T13:41:25.642 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.665 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.041) 0:01:29.489 ******* 2021-03-10T13:41:25.682 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-10T13:41:25.684 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.706 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.040) 0:01:29.529 ******* 2021-03-10T13:41:25.733 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-10T13:41:25.735 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.758 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.051) 0:01:29.581 ******* 2021-03-10T13:41:25.774 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-10T13:41:25.776 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.799 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.041) 0:01:29.622 ******* 2021-03-10T13:41:25.826 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.850 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.051) 0:01:29.673 ******* 2021-03-10T13:41:25.866 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:25.868 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.890 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.039) 0:01:29.713 ******* 2021-03-10T13:41:25.916 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:25.917 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.938 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.048) 0:01:29.762 ******* 2021-03-10T13:41:25.955 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-10T13:41:25.956 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:25.979 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:25 +0000 (0:00:00.039) 0:01:29.802 ******* 2021-03-10T13:41:26.005 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-10T13:41:26.007 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.028 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.049) 0:01:29.852 ******* 2021-03-10T13:41:26.045 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-10T13:41:26.047 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.069 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.040) 0:01:29.892 ******* 2021-03-10T13:41:26.095 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-10T13:41:26.097 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.119 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.050) 0:01:29.942 ******* 2021-03-10T13:41:26.135 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-10T13:41:26.137 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.162 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.042) 0:01:29.985 ******* 2021-03-10T13:41:26.188 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-10T13:41:26.190 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.212 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.049) 0:01:30.035 ******* 2021-03-10T13:41:26.227 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-10T13:41:26.228 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.252 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.040) 0:01:30.075 ******* 2021-03-10T13:41:26.278 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-10T13:41:26.280 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.303 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.051) 0:01:30.126 ******* 2021-03-10T13:41:26.319 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-10T13:41:26.321 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.344 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.041) 0:01:30.167 ******* 2021-03-10T13:41:26.371 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:26.373 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.394 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.050) 0:01:30.218 ******* 2021-03-10T13:41:26.411 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:26.412 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.434 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.039) 0:01:30.257 ******* 2021-03-10T13:41:26.460 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-10T13:41:26.462 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.485 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.050) 0:01:30.308 ******* 2021-03-10T13:41:26.501 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-10T13:41:26.502 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.525 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.040) 0:01:30.348 ******* 2021-03-10T13:41:26.552 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-10T13:41:26.553 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.576 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.050) 0:01:30.399 ******* 2021-03-10T13:41:26.592 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-10T13:41:26.593 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.616 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.039) 0:01:30.439 ******* 2021-03-10T13:41:26.642 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-10T13:41:26.643 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.665 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.048) 0:01:30.488 ******* 2021-03-10T13:41:26.680 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:26.682 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.707 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.042) 0:01:30.531 ******* 2021-03-10T13:41:26.734 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-10T13:41:26.735 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.758 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.050) 0:01:30.581 ******* 2021-03-10T13:41:26.774 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-10T13:41:26.776 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.799 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.040) 0:01:30.622 ******* 2021-03-10T13:41:26.825 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-10T13:41:26.826 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.848 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.049) 0:01:30.671 ******* 2021-03-10T13:41:26.863 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-10T13:41:26.865 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.887 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.038) 0:01:30.710 ******* 2021-03-10T13:41:26.913 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:26.915 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.937 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.049) 0:01:30.760 ******* 2021-03-10T13:41:26.952 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-03-10T13:41:26.954 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:26.976 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:26 +0000 (0:00:00.039) 0:01:30.799 ******* 2021-03-10T13:41:27.002 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-10T13:41:27.004 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.025 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.049) 0:01:30.849 ******* 2021-03-10T13:41:27.092 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.115 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.089) 0:01:30.938 ******* 2021-03-10T13:41:27.121 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-03-10T13:41:27.123 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-10T13:41:27.146 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.030) 0:01:30.969 ******* 2021-03-10T13:41:27.182 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-10T13:41:27.184 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.206 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.059) 0:01:31.029 ******* 2021-03-10T13:41:27.274 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.296 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.090) 0:01:31.120 ******* 2021-03-10T13:41:27.365 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.390 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.093) 0:01:31.214 ******* 2021-03-10T13:41:27.427 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-10T13:41:27.429 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-10T13:41:27.453 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.062) 0:01:31.276 ******* 2021-03-10T13:41:27.459 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-03-10T13:41:27.461 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.484 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.031) 0:01:31.308 ******* 2021-03-10T13:41:27.511 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-10T13:41:27.512 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:27.534 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.049) 0:01:31.358 ******* 2021-03-10T13:41:27.623 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-03-10T13:41:27.624 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-10T13:41:27.646 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.111) 0:01:31.469 ******* 2021-03-10T13:41:27.827 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-03-10T13:41:27.829 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-10T13:41:27.850 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.204) 0:01:31.673 ******* 2021-03-10T13:41:27.897 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-10T13:41:27.899 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-10T13:41:27.921 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:27 +0000 (0:00:00.070) 0:01:31.744 ******* 2021-03-10T13:41:27.988 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-10T13:41:27.990 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:28.010 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.089) 0:01:31.834 ******* 2021-03-10T13:41:28.150 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-03-10T13:41:28.151 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-03-10T13:41:28.173 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.162) 0:01:31.997 ******* 2021-03-10T13:41:28.271 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-03-10T13:41:28.273 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-10T13:41:28.295 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.121) 0:01:32.118 ******* 2021-03-10T13:41:28.424 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-03-10T13:41:28.426 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-10T13:41:28.448 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.153) 0:01:32.271 ******* 2021-03-10T13:41:28.578 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-03-10T13:41:28.579 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-03-10T13:41:28.601 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.153) 0:01:32.424 ******* 2021-03-10T13:41:28.637 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-03-10T13:41:28.639 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:28.664 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.063) 0:01:32.488 ******* 2021-03-10T13:41:28.732 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:28.755 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.090) 0:01:32.578 ******* 2021-03-10T13:41:28.821 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-03-10T13:41:28.823 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:28.843 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.088) 0:01:32.667 ******* 2021-03-10T13:41:28.890 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-03-10T13:41:28.891 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-10T13:41:28.913 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.069) 0:01:32.736 ******* 2021-03-10T13:41:28.919 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-03-10T13:41:28.921 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:28.944 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:28 +0000 (0:00:00.031) 0:01:32.768 ******* 2021-03-10T13:41:29.012 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-03-10T13:41:29.014 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.034 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.090) 0:01:32.858 ******* 2021-03-10T13:41:29.050 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-03-10T13:41:29.052 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:29.074 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.039) 0:01:32.898 ******* 2021-03-10T13:41:29.101 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-03-10T13:41:29.103 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.125 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.050) 0:01:32.949 ******* 2021-03-10T13:41:29.140 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-03-10T13:41:29.142 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.165 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.039) 0:01:32.988 ******* 2021-03-10T13:41:29.191 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:29.193 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-10T13:41:29.214 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.049) 0:01:33.038 ******* 2021-03-10T13:41:29.230 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-03-10T13:41:29.231 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.254 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.039) 0:01:33.077 ******* 2021-03-10T13:41:29.290 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-03-10T13:41:29.292 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.301 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.303 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.312 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.314 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.324 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.325 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba032.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.347 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.092) 0:01:33.170 ******* 2021-03-10T13:41:29.353 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-03-10T13:41:29.354 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.354 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.356 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.356 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.367 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.368 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.378 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.408 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.061) 0:01:33.232 ******* 2021-03-10T13:41:29.414 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-03-10T13:41:29.415 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.439 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.030) 0:01:33.262 ******* 2021-03-10T13:41:29.496 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-03-10T13:41:29.498 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.518 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.079) 0:01:33.341 ******* 2021-03-10T13:41:29.584 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:29.586 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-10T13:41:29.607 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.089) 0:01:33.431 ******* 2021-03-10T13:41:29.624 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-03-10T13:41:29.625 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002838", "end": "2021-03-10 13:41:29.594720", "rc": 0, "start": "2021-03-10 13:41:29.591882", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock"]} 2021-03-10T13:41:29.650 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.042) 0:01:33.473 ******* 2021-03-10T13:41:29.687 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-03-10T13:41:29.688 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.698 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.700 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.709 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.711 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.720 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.722 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** ok: [gibba011.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [gibba011.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.746 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.095) 0:01:33.569 ******* 2021-03-10T13:41:29.751 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.752 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.752 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.754 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.764 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.766 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.792 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.046) 0:01:33.616 ******* 2021-03-10T13:41:29.809 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-03-10T13:41:29.810 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.832 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.040) 0:01:33.656 ******* 2021-03-10T13:41:29.900 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-03-10T13:41:29.902 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:29.922 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:29 +0000 (0:00:00.090) 0:01:33.746 ******* 2021-03-10T13:41:30.010 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-03-10T13:41:30.012 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003135", "end": "2021-03-10 13:41:29.985573", "rc": 0, "start": "2021-03-10 13:41:29.982438", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock"]} 2021-03-10T13:41:30.036 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.113) 0:01:33.859 ******* 2021-03-10T13:41:30.102 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.127 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.091) 0:01:33.950 ******* 2021-03-10T13:41:30.194 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-03-10T13:41:30.196 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.216 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.088) 0:01:34.039 ******* 2021-03-10T13:41:30.283 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.309 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.093) 0:01:34.132 ******* 2021-03-10T13:41:30.366 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-03-10T13:41:30.368 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:30.390 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.081) 0:01:34.213 ******* 2021-03-10T13:41:30.448 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-03-10T13:41:30.449 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.470 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.079) 0:01:34.293 ******* 2021-03-10T13:41:30.547 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-10T13:41:30.548 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item={'block_used': 13728326, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366782, 'size_available': 927198339072, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113647, 'block_size': 4096, 'inode_available': 60941329}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366782, "block_size": 4096, "block_total": 240095108, "block_used": 13728326, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941329, "inode_total": 61054976, "inode_used": 113647, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198339072, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-03-10T13:41:30.569 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.099) 0:01:34.393 ******* 2021-03-10T13:41:30.708 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-03-10T13:41:30.710 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002394", "end": "2021-03-10 13:41:30.623507", "rc": 0, "start": "2021-03-10 13:41:30.621113", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:30.731 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.161) 0:01:34.554 ******* 2021-03-10T13:41:30.737 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-03-10T13:41:30.739 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:30.763 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.031) 0:01:34.586 ******* 2021-03-10T13:41:30.810 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-03-10T13:41:30.812 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'block_used': 13728326, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366782, 'size_available': 927198339072, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113647, 'block_size': 4096, 'inode_available': 60941329}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366782, "block_size": 4096, "block_total": 240095108, "block_used": 13728326, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941329, "inode_total": 61054976, "inode_used": 113647, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198339072, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.834 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.071) 0:01:34.658 ******* 2021-03-10T13:41:30.840 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-03-10T13:41:30.842 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:30.865 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.030) 0:01:34.688 ******* 2021-03-10T13:41:30.902 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-03-10T13:41:30.925 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.060) 0:01:34.749 ******* 2021-03-10T13:41:30.942 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-10T13:41:30.943 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13728330, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366778, 'size_available': 927198322688, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113650, 'block_size': 4096, 'inode_available': 60941326}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366778, "block_size": 4096, "block_total": 240095108, "block_used": 13728330, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941326, "inode_total": 61054976, "inode_used": 113650, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198322688, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-03-10T13:41:30.969 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:30 +0000 (0:00:00.043) 0:01:34.793 ******* 2021-03-10T13:41:31.016 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-10T13:41:31.018 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:31.110 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-03-10T13:41:31.112 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002483", "end": "2021-03-10 13:41:30.841101", "rc": 0, "start": "2021-03-10 13:41:30.838618", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:31.132 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:31 +0000 (0:00:00.162) 0:01:34.955 ******* 2021-03-10T13:41:31.209 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-03-10T13:41:31.211 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13728330, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366778, 'size_available': 927198322688, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113650, 'block_size': 4096, 'inode_available': 60941326}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366778, "block_size": 4096, "block_total": 240095108, "block_used": 13728330, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941326, "inode_total": 61054976, "inode_used": 113650, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198322688, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:31.232 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:31 +0000 (0:00:00.100) 0:01:35.055 ******* 2021-03-10T13:41:31.289 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-03-10T13:41:31.315 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:31 +0000 (0:00:00.082) 0:01:35.138 ******* 2021-03-10T13:41:31.392 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-10T13:41:31.394 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:32.848 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzB2rAaXhwI6f9cPuM2Ydiozj3x7zHT4zR'], 'uuids': ['6dd3e237-1b02-4477-8f47-53e4d5b2807e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.740548", "end": "2021-03-10 13:41:32.748114", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzB2rAaXhwI6f9cPuM2Ydiozj3x7zHT4zR"], "labels": [], "masters": [], "uuids": ["6dd3e237-1b02-4477-8f47-53e4d5b2807e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:31.007566", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:32.859 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:32.860 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:33.098 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HomwyWA1csZfyQQfLXEdt99s45c3iWl1l6'], 'uuids': ['2b6015c9-5244-4c12-8e3b-907b652d4b93']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.615948", "end": "2021-03-10 13:41:32.818156", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HomwyWA1csZfyQQfLXEdt99s45c3iWl1l6"], "labels": [], "masters": [], "uuids": ["2b6015c9-5244-4c12-8e3b-907b652d4b93"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:31.202208", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:33.110 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:33.111 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:34.083 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzvYxQrOsFUei28fGBOIOlchxxNhOBargv'], 'uuids': ['5bd43f39-cef4-4875-9394-dfe6591b7f5a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.131309", "end": "2021-03-10 13:41:33.982622", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzvYxQrOsFUei28fGBOIOlchxxNhOBargv"], "labels": [], "masters": [], "uuids": ["5bd43f39-cef4-4875-9394-dfe6591b7f5a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:32.851313", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:34.084 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF', 'wwn-0x50000399ab801e8b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF-part1', 'wwn-0x50000399ab801e8b-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e8b', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PRF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF", "wwn-0x50000399ab801e8b"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF-part1", "wwn-0x50000399ab801e8b-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PRF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e8b"}}, "skip_reason": "Cond 2021-03-10T13:41:34.086 INFO:teuthology.task.ansible.out:itional result was False"} 2021-03-10T13:41:34.302 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Hog6XkO8bAq43NksphnNHEZixMbl33c0eo'], 'uuids': ['fa51aac1-2c88-48e9-a075-352c4af66e5a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.106396", "end": "2021-03-10 13:41:34.021405", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Hog6XkO8bAq43NksphnNHEZixMbl33c0eo"], "labels": [], "masters": [], "uuids": ["fa51aac1-2c88-48e9-a075-352c4af66e5a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:32.915009", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:34.400 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF', 'wwn-0x50000399ab9813ca'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1', 'wwn-0x50000399ab9813ca-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ca', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JWF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF", "wwn-0x50000399ab9813ca"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1", "wwn-0x50000399ab9813ca-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JWF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ca"}}, "skip_reason": "Cond 2021-03-10T13:41:34.402 INFO:teuthology.task.ansible.out:itional result was False"} 2021-03-10T13:41:35.314 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-buxRwe-MhRd-r6px-E3Au-vmFJ-Pghf-2iBbjk', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400C3375A', 'nvme-nvme.8086-50484b4d393133343030433333373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.132982", "end": "2021-03-10 13:41:35.217681", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-buxRwe-MhRd-r6px-E3Au-vmFJ-Pghf-2iBbjk", "nvme-INTEL_SSDPEL1K375GA_PHKM913400C3375A", "nvme-nvme.8086-50484b4d393133343030433333373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:34.084699", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-03-10T13:41:35.315 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:35.325 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:35.326 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:35.513 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-TyPJjT-NXI2-fewA-GH7e-nbn4-vZSE-2Ne97w', 'nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A', 'nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.127315", "end": "2021-03-10 13:41:35.233324", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-TyPJjT-NXI2-fewA-GH7e-nbn4-vZSE-2Ne97w", "nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A", "nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:34.106009", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-03-10T13:41:35.515 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:35.525 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:35.526 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:36.528 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzOPZjM4UDqk7wUdZOM1vOIIJ0Ikg3hSkc'], 'uuids': ['825d01fc-267a-4645-a9a7-64f4f7ba4004']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.119296", "end": "2021-03-10 13:41:36.435446", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzOPZjM4UDqk7wUdZOM1vOIIJ0Ikg3hSkc"], "labels": [], "masters": [], "uuids": ["825d01fc-267a-4645-a9a7-64f4f7ba4004"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:35.316150", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:36.539 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:36.551 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:36.748 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoN0ZRee9VewUPjLUsXycL7RvSgddm1rUg'], 'uuids': ['c5327882-8b8e-462a-837c-26b84770584f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.109080", "end": "2021-03-10 13:41:36.465081", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoN0ZRee9VewUPjLUsXycL7RvSgddm1rUg"], "labels": [], "masters": [], "uuids": ["c5327882-8b8e-462a-837c-26b84770584f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:35.356001", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:36.749 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:36.761 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:37.744 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOz1NtvlA0CRJZtZkaan8Qqn821YOy8DvH5'], 'uuids': ['fd35a206-29c4-4654-9438-8affc27f0d83']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.106759", "end": "2021-03-10 13:41:37.643380", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOz1NtvlA0CRJZtZkaan8Qqn821YOy8DvH5"], "labels": [], "masters": [], "uuids": ["fd35a206-29c4-4654-9438-8affc27f0d83"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:36.536621", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:37.757 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:37.966 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoiIeWBSJ8lo3EvuKoSWfynPVbqvVdSPgA'], 'uuids': ['6d4381fd-e94c-4663-8556-034efe71f0ea']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.110380", "end": "2021-03-10 13:41:37.681128", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoiIeWBSJ8lo3EvuKoSWfynPVbqvVdSPgA"], "labels": [], "masters": [], "uuids": ["6d4381fd-e94c-4663-8556-034efe71f0ea"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:36.570748", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:37.967 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:38.939 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzV7wFSE5jjNuBEwunQNgSTtEtW6nxK2Y9'], 'uuids': ['d615c9c2-3ae6-4585-ae5c-18371056919d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.109002", "end": "2021-03-10 13:41:38.844102", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzV7wFSE5jjNuBEwunQNgSTtEtW6nxK2Y9"], "labels": [], "masters": [], "uuids": ["d615c9c2-3ae6-4585-ae5c-18371056919d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:37.735100", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:38.983 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:38 +0000 (0:00:07.668) 0:01:42.806 ******* 2021-03-10T13:41:39.061 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-10T13:41:39.063 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.166 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-10T13:41:39.168 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Ho7a2JI3mvOFJ6yLhjqtk4hClcTcwtpzVS'], 'uuids': ['2e86dcaf-7a6b-4bea-9cd0-56fc5ca31f5e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.104307", "end": "2021-03-10 13:41:38.881633", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Ho7a2JI3mvOFJ6yLhjqtk4hClcTcwtpzVS"], "labels": [], "masters": [], "uuids": ["2e86dcaf-7a6b-4bea-9cd0-56fc5ca31f5e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:37.777326", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-10T13:41:39.201 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:39 +0000 (0:00:00.217) 0:01:43.024 ******* 2021-03-10T13:41:39.228 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-10T13:41:39.229 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzB2rAaXhwI6f9cPuM2Ydiozj3x7zHT4zR'], 'uuids': ['6dd3e237-1b02-4477-8f47-53e4d5b2807e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077482", "end": "2021-03-10 13:41:39.129679", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzB2rAaXhwI6f9cPuM2Ydiozj3x7zHT4zR"], "labels": [], "masters": [], "uuids": ["6dd3e237-1b02-4477-8f47-53e4d5b2807e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.052197", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0762301 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0762301 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 2021-03-10T13:41:39.231 INFO:teuthology.task.ansible.out:'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.241 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.283 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.409 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzvYxQrOsFUei28fGBOIOlchxxNhOBargv'], 'uuids': ['5bd43f39-cef4-4875-9394-dfe6591b7f5a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.077432", "end": "2021-03-10 13:41:39.311026", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzvYxQrOsFUei28fGBOIOlchxxNhOBargv"], "labels": [], "masters": [], "uuids": ["5bd43f39-cef4-4875-9394-dfe6591b7f5a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.233594", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0761975 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0761975 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.410 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF', 'wwn-0x50000399ab801e8b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF-part1', 'wwn-0x50000399ab801e8b-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e8b', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PRF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF", "wwn-0x50000399ab801e8b"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF-part1", "wwn-0x50000399ab801e8b-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PRF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e8b"}}, "skip_reason": "Cond 2021-03-10T13:41:39.412 INFO:teuthology.task.ansible.out:itional result was False"} 2021-03-10T13:41:39.443 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HomwyWA1csZfyQQfLXEdt99s45c3iWl1l6'], 'uuids': ['2b6015c9-5244-4c12-8e3b-907b652d4b93']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077573", "end": "2021-03-10 13:41:39.169861", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HomwyWA1csZfyQQfLXEdt99s45c3iWl1l6"], "labels": [], "masters": [], "uuids": ["2b6015c9-5244-4c12-8e3b-907b652d4b93"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.092288", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0763349 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0763349 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.455 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.466 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.529 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-buxRwe-MhRd-r6px-E3Au-vmFJ-Pghf-2iBbjk', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400C3375A', 'nvme-nvme.8086-50484b4d393133343030433333373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.024726", "end": "2021-03-10 13:41:39.435634", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-buxRwe-MhRd-r6px-E3Au-vmFJ-Pghf-2iBbjk", "nvme-INTEL_SSDPEL1K375GA_PHKM913400C3375A", "nvme-nvme.8086-50484b4d393133343030433333373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.410908", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0234808 s, 4.9 GB/s", "stderr_lines": ["1 2021-03-10T13:41:39.531 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0234808 s, 4.9 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.541 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.542 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.615 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Hog6XkO8bAq43NksphnNHEZixMbl33c0eo'], 'uuids': ['fa51aac1-2c88-48e9-a075-352c4af66e5a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078783", "end": "2021-03-10 13:41:39.343317", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Hog6XkO8bAq43NksphnNHEZixMbl33c0eo"], "labels": [], "masters": [], "uuids": ["fa51aac1-2c88-48e9-a075-352c4af66e5a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.264534", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0775624 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0775624 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.627 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF', 'wwn-0x50000399ab9813ca'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1', 'wwn-0x50000399ab9813ca-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ca', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JWF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF", "wwn-0x50000399ab9813ca"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1", "wwn-0x50000399ab9813ca-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JWF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ca"}}, "skip_reason": "Cond 2021-03-10T13:41:39.628 INFO:teuthology.task.ansible.out:itional result was False"} 2021-03-10T13:41:39.700 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzOPZjM4UDqk7wUdZOM1vOIIJ0Ikg3hSkc'], 'uuids': ['825d01fc-267a-4645-a9a7-64f4f7ba4004']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078543", "end": "2021-03-10 13:41:39.608159", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzOPZjM4UDqk7wUdZOM1vOIIJ0Ikg3hSkc"], "labels": [], "masters": [], "uuids": ["825d01fc-267a-4645-a9a7-64f4f7ba4004"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.529616", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0771908 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0771908 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.712 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.713 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.745 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-TyPJjT-NXI2-fewA-GH7e-nbn4-vZSE-2Ne97w', 'nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A', 'nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025417", "end": "2021-03-10 13:41:39.466829", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-TyPJjT-NXI2-fewA-GH7e-nbn4-vZSE-2Ne97w", "nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A", "nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.441412", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0241109 s, 4.8 GB/s", "stderr_lines": ["1 2021-03-10T13:41:39.747 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0241109 s, 4.8 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.758 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.873 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOz1NtvlA0CRJZtZkaan8Qqn821YOy8DvH5'], 'uuids': ['fd35a206-29c4-4654-9438-8affc27f0d83']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078197", "end": "2021-03-10 13:41:39.779123", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOz1NtvlA0CRJZtZkaan8Qqn821YOy8DvH5"], "labels": [], "masters": [], "uuids": ["fd35a206-29c4-4654-9438-8affc27f0d83"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.700926", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0769039 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0769039 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.884 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.916 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoN0ZRee9VewUPjLUsXycL7RvSgddm1rUg'], 'uuids': ['c5327882-8b8e-462a-837c-26b84770584f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.077906", "end": "2021-03-10 13:41:39.638034", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoN0ZRee9VewUPjLUsXycL7RvSgddm1rUg"], "labels": [], "masters": [], "uuids": ["c5327882-8b8e-462a-837c-26b84770584f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.560128", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0765681 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0765681 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:39.917 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:39.928 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:40.053 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzV7wFSE5jjNuBEwunQNgSTtEtW6nxK2Y9'], 'uuids': ['d615c9c2-3ae6-4585-ae5c-18371056919d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.077364", "end": "2021-03-10 13:41:39.958253", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-1I5D86GEa6XBkSDJKTbz8Vswhc3BHuOzV7wFSE5jjNuBEwunQNgSTtEtW6nxK2Y9"], "labels": [], "masters": [], "uuids": ["d615c9c2-3ae6-4585-ae5c-18371056919d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.880889", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0761529 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0761529 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.086 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.885) 0:01:43.909 ******* 2021-03-10T13:41:40.092 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoiIeWBSJ8lo3EvuKoSWfynPVbqvVdSPgA'], 'uuids': ['6d4381fd-e94c-4663-8556-034efe71f0ea']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.077942", "end": "2021-03-10 13:41:39.810382", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0HoiIeWBSJ8lo3EvuKoSWfynPVbqvVdSPgA"], "labels": [], "masters": [], "uuids": ["6d4381fd-e94c-4663-8556-034efe71f0ea"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.732440", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0767414 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0767414 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.105 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:40.262 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Ho7a2JI3mvOFJ6yLhjqtk4hClcTcwtpzVS'], 'uuids': ['2e86dcaf-7a6b-4bea-9cd0-56fc5ca31f5e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.077587", "end": "2021-03-10 13:41:39.982773", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-f6ucpgDUXt0fjd4OIkwKh9U4yWy3e0Ho7a2JI3mvOFJ6yLhjqtk4hClcTcwtpzVS"], "labels": [], "masters": [], "uuids": ["2e86dcaf-7a6b-4bea-9cd0-56fc5ca31f5e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-10 13:41:39.905186", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0763736 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0763736 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.294 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.208) 0:01:44.118 ******* 2021-03-10T13:41:40.456 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-03-10T13:41:40.457 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.194947", "end": "2021-03-10 13:41:40.313961", "rc": 0, "start": "2021-03-10 13:41:40.119014", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.480 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.185) 0:01:44.303 ******* 2021-03-10T13:41:40.651 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.156111", "end": "2021-03-10 13:41:40.313849", "rc": 0, "start": "2021-03-10 13:41:40.157738", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.653 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.073977", "end": "2021-03-10 13:41:40.561099", "rc": 0, "start": "2021-03-10 13:41:40.487122", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.675 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.195) 0:01:44.498 ******* 2021-03-10T13:41:40.696 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.020) 0:01:44.519 ******* 2021-03-10T13:41:40.753 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-10T13:41:40.814 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.118) 0:01:44.637 ******* 2021-03-10T13:41:40.841 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-10T13:41:40.842 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:40.864 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.050) 0:01:44.688 ******* 2021-03-10T13:41:40.870 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-03-10T13:41:40.871 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.076044", "end": "2021-03-10 13:41:40.578701", "rc": 0, "start": "2021-03-10 13:41:40.502657", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:40.894 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.029) 0:01:44.717 ******* 2021-03-10T13:41:40.902 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-03-10T13:41:40.903 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:40.926 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.032) 0:01:44.750 ******* 2021-03-10T13:41:40.932 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-10T13:41:40.958 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.032) 0:01:44.782 ******* 2021-03-10T13:41:40.964 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-10T13:41:40.966 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:40.990 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:40 +0000 (0:00:00.031) 0:01:44.813 ******* 2021-03-10T13:41:40.995 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-10T13:41:40.997 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.021 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.030) 0:01:44.844 ******* 2021-03-10T13:41:41.026 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-03-10T13:41:41.028 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.050 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:44.874 ******* 2021-03-10T13:41:41.056 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-03-10T13:41:41.058 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.080 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.030) 0:01:44.904 ******* 2021-03-10T13:41:41.107 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-10T13:41:41.109 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.132 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.051) 0:01:44.956 ******* 2021-03-10T13:41:41.149 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-10T13:41:41.150 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:41.171 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.039) 0:01:44.995 ******* 2021-03-10T13:41:41.177 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-03-10T13:41:41.179 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.205 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.033) 0:01:45.028 ******* 2021-03-10T13:41:41.222 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-10T13:41:41.223 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'block_used': 13728326, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366782, 'size_available': 927198339072, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113647, 'block_size': 4096, 'inode_available': 60941329}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366782, "block_size": 4096, "block_total": 240095108, "block_used": 13728326, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941329, "inode_total": 61054976, "inode_used": 113647, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198339072, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.247 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.042) 0:01:45.071 ******* 2021-03-10T13:41:41.273 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-10T13:41:41.275 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.296 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.048) 0:01:45.119 ******* 2021-03-10T13:41:41.313 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-10T13:41:41.314 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-10T13:41:41.335 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.039) 0:01:45.159 ******* 2021-03-10T13:41:41.341 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:41.343 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.366 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.030) 0:01:45.189 ******* 2021-03-10T13:41:41.372 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-10T13:41:41.374 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13728330, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226366778, 'size_available': 927198322688, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 113650, 'block_size': 4096, 'inode_available': 60941326}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226366778, "block_size": 4096, "block_total": 240095108, "block_used": 13728330, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941326, "inode_total": 61054976, "inode_used": 113650, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 927198322688, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.398 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.031) 0:01:45.221 ******* 2021-03-10T13:41:41.404 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:41.406 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.428 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:45.251 ******* 2021-03-10T13:41:41.433 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-10T13:41:41.434 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.457 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:45.280 ******* 2021-03-10T13:41:41.463 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-03-10T13:41:41.464 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.487 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:45.310 ******* 2021-03-10T13:41:41.492 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:41.494 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.516 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:45.339 ******* 2021-03-10T13:41:41.521 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-03-10T13:41:41.523 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.545 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.029) 0:01:45.368 ******* 2021-03-10T13:41:41.550 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:41.552 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.573 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.028) 0:01:45.397 ******* 2021-03-10T13:41:41.579 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-03-10T13:41:41.580 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.605 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.031) 0:01:45.429 ******* 2021-03-10T13:41:41.612 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-03-10T13:41:41.614 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.770 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.163) 0:01:45.593 ******* 2021-03-10T13:41:41.797 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-03-10T13:41:41.798 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.819 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.049) 0:01:45.643 ******* 2021-03-10T13:41:41.845 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-03-10T13:41:41.847 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:41.868 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:41 +0000 (0:00:00.048) 0:01:45.691 ******* 2021-03-10T13:41:42.347 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-03-10T13:41:42.349 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-10T13:41:42.370 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:42 +0000 (0:00:00.502) 0:01:46.194 ******* 2021-03-10T13:41:42.397 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-10T13:41:42.422 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:42 +0000 (0:00:00.051) 0:01:46.246 ******* 2021-03-10T13:41:42.873 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-10T13:41:42.875 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:42.895 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:43.267 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:43.278 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:43.641 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:43.662 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:44.005 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:44.016 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:44.369 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:44.395 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:01.972) 0:01:48.218 ******* 2021-03-10T13:41:44.473 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-10T13:41:44.501 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.106) 0:01:48.325 ******* 2021-03-10T13:41:44.508 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-10T13:41:44.509 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-10T13:41:44.533 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.031) 0:01:48.356 ******* 2021-03-10T13:41:44.601 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-10T13:41:44.626 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.093) 0:01:48.450 ******* 2021-03-10T13:41:44.643 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-10T13:41:44.645 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.717 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.760 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.792 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.835 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.867 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.878 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:44.904 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.277) 0:01:48.727 ******* 2021-03-10T13:41:44.911 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:44.933 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-03-10T13:41:44.935 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:44.956 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.051) 0:01:48.779 ******* 2021-03-10T13:41:44.982 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-03-10T13:41:44.983 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-10T13:41:45.005 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:44 +0000 (0:00:00.049) 0:01:48.828 ******* 2021-03-10T13:41:45.011 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.038 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.033) 0:01:48.862 ******* 2021-03-10T13:41:45.064 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-03-10T13:41:45.066 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.087 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.048) 0:01:48.911 ******* 2021-03-10T13:41:45.113 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-03-10T13:41:45.115 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:45.137 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.049) 0:01:48.960 ******* 2021-03-10T13:41:45.157 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.019) 0:01:48.980 ******* 2021-03-10T13:41:45.194 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-03-10T13:41:45.195 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-10T13:41:45.225 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.068) 0:01:49.048 ******* 2021-03-10T13:41:45.241 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-03-10T13:41:45.242 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:45.266 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.040) 0:01:49.089 ******* 2021-03-10T13:41:45.303 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-03-10T13:41:45.305 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-10T13:41:45.326 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.060) 0:01:49.150 ******* 2021-03-10T13:41:45.486 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-03-10T13:41:45.487 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true} 2021-03-10T13:41:45.510 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.183) 0:01:49.333 ******* 2021-03-10T13:41:45.515 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true} 2021-03-10T13:41:45.540 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.030) 0:01:49.363 ******* 2021-03-10T13:41:45.783 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-03-10T13:41:45.785 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-03-10T13:41:45.807 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.267) 0:01:49.630 ******* 2021-03-10T13:41:45.813 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-03-10T13:41:45.835 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.028) 0:01:49.659 ******* 2021-03-10T13:41:45.840 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-03-10T13:41:45.841 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.863 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.027) 0:01:49.686 ******* 2021-03-10T13:41:45.868 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.890 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.027) 0:01:49.713 ******* 2021-03-10T13:41:45.895 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-03-10T13:41:45.896 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.917 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.027) 0:01:49.741 ******* 2021-03-10T13:41:45.922 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.944 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.027) 0:01:49.768 ******* 2021-03-10T13:41:45.949 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-10T13:41:45.951 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:45.975 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.030) 0:01:49.798 ******* 2021-03-10T13:41:45.981 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:46.003 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:45 +0000 (0:00:00.028) 0:01:49.827 ******* 2021-03-10T13:41:46.008 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-10T13:41:46.010 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:46.031 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.027) 0:01:49.855 ******* 2021-03-10T13:41:46.037 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:46.061 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.029) 0:01:49.884 ******* 2021-03-10T13:41:46.149 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-03-10T13:41:46.150 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-10T13:41:46.172 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.111) 0:01:49.996 ******* 2021-03-10T13:41:46.178 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-10T13:41:46.203 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.030) 0:01:50.027 ******* 2021-03-10T13:41:46.303 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-03-10T13:41:46.304 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:46.326 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.122) 0:01:50.149 ******* 2021-03-10T13:41:46.345 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.018) 0:01:50.168 ******* 2021-03-10T13:41:46.568 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-03-10T13:41:46.570 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-10T13:41:46.591 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.246) 0:01:50.415 ******* 2021-03-10T13:41:46.618 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-10T13:41:46.642 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.051) 0:01:50.466 ******* 2021-03-10T13:41:46.866 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-03-10T13:41:46.868 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-10T13:41:46.889 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.246) 0:01:50.712 ******* 2021-03-10T13:41:46.895 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-10T13:41:46.926 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:46 +0000 (0:00:00.037) 0:01:50.749 ******* 2021-03-10T13:41:47.306 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-03-10T13:41:47.308 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:47.329 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.403) 0:01:51.152 ******* 2021-03-10T13:41:47.376 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:47.400 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.070) 0:01:51.223 ******* 2021-03-10T13:41:47.582 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-03-10T13:41:47.584 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.139403", "end": "2021-03-10 13:41:47.306742", "rc": 0, "start": "2021-03-10 13:41:47.167339", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-03-10T13:41:47.606 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.206) 0:01:51.429 ******* 2021-03-10T13:41:47.632 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:47.655 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.048) 0:01:51.478 ******* 2021-03-10T13:41:47.661 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-03-10T13:41:47.663 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.155411", "end": "2021-03-10 13:41:47.561710", "rc": 0, "start": "2021-03-10 13:41:47.406299", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-03-10T13:41:47.685 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.030) 0:01:51.508 ******* 2021-03-10T13:41:47.691 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-03-10T13:41:47.692 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:47.716 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.030) 0:01:51.539 ******* 2021-03-10T13:41:47.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-03-10T13:41:47.723 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:47.746 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.030) 0:01:51.569 ******* 2021-03-10T13:41:47.772 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:47.795 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:47 +0000 (0:00:00.048) 0:01:51.618 ******* 2021-03-10T13:41:48.118 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-03-10T13:41:48.120 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:48.142 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.347) 0:01:51.966 ******* 2021-03-10T13:41:48.169 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-03-10T13:41:48.170 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.195 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.052) 0:01:52.018 ******* 2021-03-10T13:41:48.201 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-03-10T13:41:48.202 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:48.227 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.032) 0:01:52.050 ******* 2021-03-10T13:41:48.252 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-03-10T13:41:48.254 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.275 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.048) 0:01:52.099 ******* 2021-03-10T13:41:48.425 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-03-10T13:41:48.427 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-10T13:41:48.449 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.173) 0:01:52.272 ******* 2021-03-10T13:41:48.475 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.498 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.048) 0:01:52.321 ******* 2021-03-10T13:41:48.513 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-03-10T13:41:48.515 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-10T13:41:48.538 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.040) 0:01:52.362 ******* 2021-03-10T13:41:48.564 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-03-10T13:41:48.566 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.587 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.048) 0:01:52.410 ******* 2021-03-10T13:41:48.685 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-03-10T13:41:48.687 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:04 UTC", "ActiveEnterTimestampMonotonic": "12803008", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-journald.socket sysinit.target systemd-tmpfiles-setup.service -.mount system.slice basic.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:04 UTC", "AssertTimestampMonotonic": "12654691", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:04 UTC", "ConditionTimestampMonotonic": "12654691", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": 2021-03-10T13:41:48.689 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1030", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:04 UTC", "ExecMainStartTimestampMonotonic": "12802998", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:04 UTC", "InactiveExitTimestampMonotonic": "12655273", "InvocationID": "97519c8d21ad42cd9b586a9a702d22a0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Lim 2021-03-10T13:41:48.690 INFO:teuthology.task.ansible.out:itCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1030", "MemoryAccounting": "yes", "MemoryCurrent": "3825664", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "Requ 2021-03-10T13:41:48.692 INFO:teuthology.task.ansible.out:iresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:04 UTC", "StateChangeTimestampMonotonic": "12803008", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38348", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:41:48.712 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.124) 0:01:52.535 ******* 2021-03-10T13:41:48.737 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:48.739 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-10T13:41:48.760 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.048) 0:01:52.583 ******* 2021-03-10T13:41:48.766 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-03-10T13:41:48.768 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-03-10 13:39:11 UTC", "ActiveEnterTimestampMonotonic": "12707802", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service basic.target sysinit.target system.slice systemd-journald.socket network.target -.mount", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-03-10 13:39:10 UTC", "AssertTimestampMonotonic": "12522822", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-03-10 13:39:10 UTC", "ConditionTimestampMonotonic": "12522821", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": 2021-03-10T13:41:48.770 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "962", "ExecMainStartTimestamp": "Wed 2021-03-10 13:39:11 UTC", "ExecMainStartTimestampMonotonic": "12707792", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-03-10 13:39:10 UTC", "InactiveExitTimestampMonotonic": "12523494", "InvocationID": "b62b78242c7a4cd7b230e6788bfe822e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-03-10T13:41:48.771 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127828", "LimitNPROCSoft": "127828", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "962", "MemoryAccounting": "yes", "MemoryCurrent": "4075520", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "Requir 2021-03-10T13:41:48.773 INFO:teuthology.task.ansible.out:esMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:11 UTC", "StateChangeTimestampMonotonic": "12707802", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38348", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:41:48.793 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.032) 0:01:52.616 ******* 2021-03-10T13:41:48.799 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:48.800 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-03-10T13:41:48.822 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.029) 0:01:52.645 ******* 2021-03-10T13:41:48.827 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:48.830 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-10T13:41:48.854 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.032) 0:01:52.677 ******* 2021-03-10T13:41:48.860 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-03-10T13:41:48.862 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.885 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.031) 0:01:52.709 ******* 2021-03-10T13:41:48.891 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:48.893 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-03-10T13:41:48.917 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.032) 0:01:52.741 ******* 2021-03-10T13:41:48.943 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-03-10T13:41:48.945 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:48.967 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:48 +0000 (0:00:00.049) 0:01:52.790 ******* 2021-03-10T13:41:49.708 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-03-10T13:41:49.709 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:49.732 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:49 +0000 (0:00:00.764) 0:01:53.555 ******* 2021-03-10T13:41:49.758 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-03-10T13:41:49.760 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:49.782 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:49 +0000 (0:00:00.050) 0:01:53.605 ******* 2021-03-10T13:41:49.808 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-03-10T13:41:49.809 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:49.833 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:49 +0000 (0:00:00.050) 0:01:53.656 ******* 2021-03-10T13:41:49.859 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-03-10T13:41:49.861 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:49.882 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:49 +0000 (0:00:00.048) 0:01:53.705 ******* 2021-03-10T13:41:49.907 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-03-10T13:41:49.909 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-10T13:41:49.932 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:49 +0000 (0:00:00.050) 0:01:53.755 ******* 2021-03-10T13:41:50.011 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-10T13:41:50.035 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:50 +0000 (0:00:00.102) 0:01:53.858 ******* 2021-03-10T13:41:50.268 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-03-10T13:41:50.270 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-10T13:41:50.295 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:50 +0000 (0:00:00.259) 0:01:54.118 ******* 2021-03-10T13:41:50.354 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-10T13:41:50.380 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:50 +0000 (0:00:00.085) 0:01:54.204 ******* 2021-03-10T13:41:52.189 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-03-10T13:41:52.191 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-03-10T13:41:52.214 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:01.833) 0:01:56.038 ******* 2021-03-10T13:41:52.242 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-03-10T13:41:52.267 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.052) 0:01:56.090 ******* 2021-03-10T13:41:52.561 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-03-10T13:41:52.563 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false} 2021-03-10T13:41:52.585 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.318) 0:01:56.408 ******* 2021-03-10T13:41:52.611 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-03-10T13:41:52.613 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-03-10T13:41:52.632 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.047) 0:01:56.456 ******* 2021-03-10T13:41:52.638 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-03-10T13:41:52.640 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false} 2021-03-10T13:41:52.665 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.032) 0:01:56.489 ******* 2021-03-10T13:41:52.692 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-03-10T13:41:52.693 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-03-10T13:41:52.715 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.049) 0:01:56.538 ******* 2021-03-10T13:41:52.764 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-03-10T13:41:52.765 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615383561.368323, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615383561.364323, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615383561.364323, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-10T13:41:52.791 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.075) 0:01:56.614 ******* 2021-03-10T13:41:52.849 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615383567.9840934, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615383567.9800935, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615383567.9800935, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-10T13:41:52.873 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:52 +0000 (0:00:00.082) 0:01:56.697 ******* 2021-03-10T13:41:53.035 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-03-10T13:41:53.037 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-03-10T13:41:53.058 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.184) 0:01:56.881 ******* 2021-03-10T13:41:53.065 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-03-10T13:41:53.090 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.031) 0:01:56.913 ******* 2021-03-10T13:41:53.095 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-03-10T13:41:53.097 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-10T13:41:53.120 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.030) 0:01:56.943 ******* 2021-03-10T13:41:53.126 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-10T13:41:53.150 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.029) 0:01:56.973 ******* 2021-03-10T13:41:53.155 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-03-10T13:41:53.156 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.179 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.029) 0:01:57.002 ******* 2021-03-10T13:41:53.185 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.210 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.030) 0:01:57.033 ******* 2021-03-10T13:41:53.298 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-03-10T13:41:53.300 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-10T13:41:53.323 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.113) 0:01:57.146 ******* 2021-03-10T13:41:53.340 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-10T13:41:53.365 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.041) 0:01:57.188 ******* 2021-03-10T13:41:53.464 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-03-10T13:41:53.466 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-10T13:41:53.497 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.132) 0:01:57.321 ******* 2021-03-10T13:41:53.515 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-10T13:41:53.538 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.040) 0:01:57.361 ******* 2021-03-10T13:41:53.544 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-03-10T13:41:53.546 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.568 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.030) 0:01:57.392 ******* 2021-03-10T13:41:53.574 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.600 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.031) 0:01:57.423 ******* 2021-03-10T13:41:53.607 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-03-10T13:41:53.608 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.620 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.621 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.645 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.045) 0:01:57.468 ******* 2021-03-10T13:41:53.651 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.652 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.652 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.654 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.679 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.034) 0:01:57.503 ******* 2021-03-10T13:41:53.686 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-03-10T13:41:53.688 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.698 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.699 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.700 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.726 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.046) 0:01:57.549 ******* 2021-03-10T13:41:53.733 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.733 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.735 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.735 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.760 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.034) 0:01:57.583 ******* 2021-03-10T13:41:53.765 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-10T13:41:53.767 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.793 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.033) 0:01:57.617 ******* 2021-03-10T13:41:53.801 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.826 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.032) 0:01:57.649 ******* 2021-03-10T13:41:53.831 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-03-10T13:41:53.833 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.854 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.028) 0:01:57.678 ******* 2021-03-10T13:41:53.860 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.883 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.028) 0:01:57.707 ******* 2021-03-10T13:41:53.890 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-03-10T13:41:53.891 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.919 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.036) 0:01:57.743 ******* 2021-03-10T13:41:53.927 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.954 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.034) 0:01:57.777 ******* 2021-03-10T13:41:53.959 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-03-10T13:41:53.961 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:53.984 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:53 +0000 (0:00:00.029) 0:01:57.807 ******* 2021-03-10T13:41:53.991 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.015 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.030) 0:01:57.838 ******* 2021-03-10T13:41:54.021 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-03-10T13:41:54.023 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.048 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.033) 0:01:57.871 ******* 2021-03-10T13:41:54.056 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.082 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.033) 0:01:57.905 ******* 2021-03-10T13:41:54.088 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-10T13:41:54.090 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.114 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.032) 0:01:57.938 ******* 2021-03-10T13:41:54.120 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.146 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.031) 0:01:57.969 ******* 2021-03-10T13:41:54.151 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-03-10T13:41:54.153 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.179 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.032) 0:01:58.002 ******* 2021-03-10T13:41:54.185 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.211 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.031) 0:01:58.034 ******* 2021-03-10T13:41:54.216 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-10T13:41:54.218 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.239 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.028) 0:01:58.062 ******* 2021-03-10T13:41:54.245 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:54.269 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.029) 0:01:58.092 ******* 2021-03-10T13:41:54.377 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-10T13:41:54.379 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002563", "end": "2021-03-10 13:41:54.092944", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-10 13:41:54.090381", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.410 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002618", "end": "2021-03-10 13:41:54.304120", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-10 13:41:54.301502", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.472 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001746", "end": "2021-03-10 13:41:54.194031", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-10 13:41:54.192285", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.505 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001783", "end": "2021-03-10 13:41:54.412219", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-10 13:41:54.410436", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.576 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001654", "end": "2021-03-10 13:41:54.291246", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-10 13:41:54.289592", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.608 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001719", "end": "2021-03-10 13:41:54.509518", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-10 13:41:54.507799", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.711 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.030558", "end": "2021-03-10 13:41:54.428444", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-10 13:41:54.397886", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.736 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.467) 0:01:58.559 ******* 2021-03-10T13:41:54.743 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.031390", "end": "2021-03-10 13:41:54.636994", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-10 13:41:54.605604", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:41:54.768 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.031) 0:01:58.591 ******* 2021-03-10T13:41:54.812 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.028) 0:01:58.620 ******* 2021-03-10T13:41:54.828 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:54 +0000 (0:00:00.030) 0:01:58.651 ******* 2021-03-10T13:41:54.926 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-03-10T13:41:54.928 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-10T13:41:54.973 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-10T13:41:55.039 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-03-10T13:41:55.055 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:55 +0000 (0:00:00.227) 0:01:58.878 ******* 2021-03-10T13:41:55.080 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:55 +0000 (0:00:00.024) 0:01:58.903 ******* 2021-03-10T13:41:55.875 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba011.front.sepia.ceph.com] 2021-03-10T13:41:55.925 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:55 +0000 (0:00:00.845) 0:01:59.748 ******* 2021-03-10T13:41:55.952 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:55.953 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-10T13:41:55.973 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:55 +0000 (0:00:00.048) 0:01:59.797 ******* 2021-03-10T13:41:55.984 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba032.front.sepia.ceph.com] 2021-03-10T13:41:56.014 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.015 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:56.034 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.060) 0:01:59.857 ******* 2021-03-10T13:41:56.054 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.019) 0:01:59.877 ******* 2021-03-10T13:41:56.072 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-03-10T13:41:56.073 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-10T13:41:56.095 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.040) 0:01:59.918 ******* 2021-03-10T13:41:56.101 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.102 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-10T13:41:56.126 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.031) 0:01:59.950 ******* 2021-03-10T13:41:56.154 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.156 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:56.176 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.049) 0:01:59.999 ******* 2021-03-10T13:41:56.213 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-03-10T13:41:56.215 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-10T13:41:56.237 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.060) 0:02:00.060 ******* 2021-03-10T13:41:56.562 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-03-10T13:41:56.563 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1615383584, "cache_updated": false, "changed": false} 2021-03-10T13:41:56.585 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.347) 0:02:00.408 ******* 2021-03-10T13:41:56.610 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.612 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:56.640 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.055) 0:02:00.464 ******* 2021-03-10T13:41:56.668 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.670 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-10T13:41:56.691 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.050) 0:02:00.515 ******* 2021-03-10T13:41:56.698 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-03-10T13:41:56.699 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"cache_update_time": 1615383589, "cache_updated": false, "changed": false} 2021-03-10T13:41:56.722 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.030) 0:02:00.545 ******* 2021-03-10T13:41:56.747 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.749 INFO:teuthology.task.ansible.out:skipping: [gibba032.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-10T13:41:56.776 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.053) 0:02:00.599 ******* 2021-03-10T13:41:56.802 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-10T13:41:56.804 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-10T13:41:56.825 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.049) 0:02:00.649 ******* 2021-03-10T13:41:56.842 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-03-10T13:41:56.843 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:56.957 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:56.986 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:56 +0000 (0:00:00.160) 0:02:00.809 ******* 2021-03-10T13:41:56.996 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:57.081 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-10T13:41:57.107 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:57 +0000 (0:00:00.121) 0:02:00.930 ******* 2021-03-10T13:41:57.320 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-03-10T13:41:57.322 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383717.040557-11676-39246534846803/source", "state": "file", "uid": 0} 2021-03-10T13:41:57.446 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383717.1624396-11684-16248439357026/source", "state": "file", "uid": 0} 2021-03-10T13:41:57.622 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383717.3416116-11676-241681942369902/source", "state": "file", "uid": 0} 2021-03-10T13:41:57.647 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:57 +0000 (0:00:00.539) 0:02:01.470 ******* 2021-03-10T13:41:57.746 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383717.4646373-11684-200303346088611/source", "state": "file", "uid": 0} 2021-03-10T13:41:57.771 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:41:57 +0000 (0:00:00.124) 0:02:01.594 ******* 2021-03-10T13:42:00.935 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-03-10T13:42:00.937 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-5kri6_s1\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-5kri6_s1"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-5kri6_s1\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=2719a363fa68c5e15884c474190b774860194c26cf25238ca1bd6054ca01c8f2\n Stored in directory: /tmp/pip-ephem-wheel-cache-58njccco/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5fb9a2102a404eb78ebc94683259c69a35ce9ddda91557c75dcd0d39c215b3f7\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.c 2021-03-10T13:42:00.939 INFO:teuthology.task.ansible.out:om/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-5kri6_s1", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=2719a363fa68c5e15884c474190b774860194c26cf25238ca1bd6054ca01c8f2", " Stored in directory: /tmp/pip-ephem-wheel-cache-58njccco/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5fb9a2102a404eb78ebc94683259c69a35ce9ddda91557c75dcd0d39c215b3f7", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-03-10T13:42:00.957 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:00 +0000 (0:00:03.186) 0:02:04.780 ******* 2021-03-10T13:42:00.994 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-dqpcv6r0\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-dqpcv6r0"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-dqpcv6r0\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=2719a363fa68c5e15884c474190b774860194c26cf25238ca1bd6054ca01c8f2\n Stored in directory: /tmp/pip-ephem-wheel-cache-zjvv61pr/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5fb9a2102a404eb78ebc94683259c69a35ce9ddda91557c75dcd0d39c215b3f7\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.c 2021-03-10T13:42:00.996 INFO:teuthology.task.ansible.out:om/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-dqpcv6r0", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=2719a363fa68c5e15884c474190b774860194c26cf25238ca1bd6054ca01c8f2", " Stored in directory: /tmp/pip-ephem-wheel-cache-zjvv61pr/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5fb9a2102a404eb78ebc94683259c69a35ce9ddda91557c75dcd0d39c215b3f7", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-03-10T13:42:01.017 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.060) 0:02:04.840 ******* 2021-03-10T13:42:01.106 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-03-10T13:42:01.108 INFO:teuthology.task.ansible.out:ok: [gibba032.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-10T13:42:01.131 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.114) 0:02:04.955 ******* 2021-03-10T13:42:01.157 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-10T13:42:01.181 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.049) 0:02:05.004 ******* 2021-03-10T13:42:01.434 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-03-10T13:42:01.436 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383721.1773314-11928-181756852117046/source", "state": "file", "uid": 0} 2021-03-10T13:42:01.458 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.277) 0:02:05.281 ******* 2021-03-10T13:42:01.474 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615383721.2223098-11934-16575600370073/source", "state": "file", "uid": 0} 2021-03-10T13:42:01.498 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.040) 0:02:05.322 ******* 2021-03-10T13:42:01.628 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-03-10T13:42:01.629 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.018893", "end": "2021-03-10 13:42:01.535588", "rc": 0, "start": "2021-03-10 13:42:01.516695", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:42:01.652 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.153) 0:02:05.475 ******* 2021-03-10T13:42:01.669 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.018995", "end": "2021-03-10 13:42:01.398095", "rc": 0, "start": "2021-03-10 13:42:01.379100", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-10T13:42:01.697 INFO:teuthology.task.ansible.out:Wednesday 10 March 2021 13:42:01 +0000 (0:00:00.044) 0:02:05.520 ******* 2021-03-10T13:42:05.567 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-03-10T13:42:05.570 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "containerd.service sysinit.target network-online.target docker.socket systemd-journald.socket basic.target system.slice firewalld.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-fire 2021-03-10T13:42:05.572 INFO:teuthology.task.ansible.out:wall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", " 2021-03-10T13:42:05.573 INFO:teuthology.task.ansible.out:LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", 2021-03-10T13:42:05.575 INFO:teuthology.task.ansible.out:"RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:35 UTC", "StateChangeTimestampMonotonic": "43700601", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:42:05.748 INFO:teuthology.task.ansible.out:changed: [gibba032.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "docker.socket sysinit.target basic.target systemd-journald.socket network-online.target system.slice firewalld.service containerd.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-fire 2021-03-10T13:42:05.750 INFO:teuthology.task.ansible.out:wall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", " 2021-03-10T13:42:05.752 INFO:teuthology.task.ansible.out:LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127828", "LimitSIGPENDINGSoft": "127828", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", 2021-03-10T13:42:05.754 INFO:teuthology.task.ansible.out:"RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-03-10 13:39:35 UTC", "StateChangeTimestampMonotonic": "37159243", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-10T13:42:05.816 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-03-10T13:42:05.817 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-03-10T13:42:05.819 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* gibba011.front.sepia.ceph.com : ok=129 changed=35 unreachable=0 failed=0 skipped=182 rescued=0 ignored=0 gibba032.front.sepia.ceph.com : ok=129 changed=35 unreachable=0 failed=0 skipped=182 rescued=0 ignored=0 Wednesday 10 March 2021 13:42:05 +0000 (0:00:04.121) 0:02:09.642 ******* =============================================================================== 2021-03-10T13:42:05.820 INFO:teuthology.task.ansible.out:users : Update authorized_keys using the keys repo --------------------- 11.96s Create all admin users with sudo access. ------------------------------- 11.66s testnode : Install packages --------------------------------------------- 9.18s testnode : Ensure packages are not present. ----------------------------- 8.15s testnode : Zap all non-root disks --------------------------------------- 7.67s container-host : Restart docker service --------------------------------- 4.12s Remove revoked users ---------------------------------------------------- 3.19s container-host : Install registries-conf-ctl ---------------------------- 3.19s testnode : Create logical volume(s) ------------------------------------- 1.97s Gathering Facts --------------------------------------------------------- 1.84s testnode : Install packages via pip ------------------------------------- 1.83s testnode : Install apt keys --------------------------------------------- 1.47s testnode : Update apt cache. -------------------------------------------- 1.33s users : Update authorized_keys for each user with literal keys ---------- 1.24s common : Update apt cache ----------------------------------------------- 1.05s testnode : Blow away lingering OSD data and FSIDs ----------------------- 0.89s Gathering Facts --------------------------------------------------------- 0.85s testnode : Instal 2021-03-10T13:42:05.822 INFO:teuthology.task.ansible.out:l python-pip on apt based systems. --------------------- 0.76s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.73s Gathering Facts --------------------------------------------------------- 0.69s 2021-03-10T13:42:06.126 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@gibba011.front.sepia.ceph.com'), Remote(name='ubuntu@gibba032.front.sepia.ceph.com')] 2021-03-10T13:42:06.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-10T13:42:06.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-10T13:42:06.164 DEBUG:teuthology.orchestra.run.gibba011:> true 2021-03-10T13:42:06.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-10T13:42:06.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-10T13:42:06.276 DEBUG:teuthology.orchestra.run.gibba032:> true 2021-03-10T13:42:06.362 INFO:teuthology.run_tasks:Running task clock... 2021-03-10T13:42:06.386 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-03-10T13:42:06.387 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-10T13:42:06.389 DEBUG:teuthology.orchestra.run.gibba011:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-10T13:42:06.409 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-03-10T13:42:06.409 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Command line: ntpd -gq 2021-03-10T13:42:06.410 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: proto: precision = 0.035 usec (-25) 2021-03-10T13:42:06.411 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-10T13:42:06.411 INFO:teuthology.orchestra.run.gibba011.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-10T13:42:06.412 INFO:teuthology.orchestra.run.gibba011.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-10T13:42:06.413 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: restrict ::: KOD does nothing without LIMITED. 2021-03-10T13:42:06.413 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen and drop on 0 v6wildcard [::]:123 2021-03-10T13:42:06.414 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-10T13:42:06.414 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen normally on 2 lo 127.0.0.1:123 2021-03-10T13:42:06.414 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen normally on 3 enp1s0f0 172.21.2.111:123 2021-03-10T13:42:06.415 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen normally on 4 lo [::1]:123 2021-03-10T13:42:06.415 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8e0a%4]:123 2021-03-10T13:42:06.415 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:06 ntpd[8655]: Listening on routing socket on fd #22 for interface updates 2021-03-10T13:42:07.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:07.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:07.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:07.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:07.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:07.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:07.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:07.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:07.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:07.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:07.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:07.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:07 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:09.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:09.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:09.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:09.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:09.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:09.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:09.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:09.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:09.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:09.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:09.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:09.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:09 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:10.410 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:10.410 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:10.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:10.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:10.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:10.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:10 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:11.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:11.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:11.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:11.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:11.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:11.415 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:11.415 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:11.415 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:11 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:12.410 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:12.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:12.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:12.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:12.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:12.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:12 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:13.411 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:13.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:13.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:13.412 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:13.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:13.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:13.413 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:13.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:13.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:13.414 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:13.415 INFO:teuthology.orchestra.run.gibba011.stdout:10 Mar 13:42:13 ntpd[8655]: ntpd: time set +0.259801 s 2021-03-10T13:42:13.415 INFO:teuthology.orchestra.run.gibba011.stdout:ntpd: time set +0.259801s 2021-03-10T13:42:13.416 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-10T13:42:13.416 INFO:teuthology.orchestra.run.gibba011.stderr:10 Mar 13:42:13 ntpd[8655]: can't open /var/log/ntpstats/loopstats.20210310: Permission denied 2021-03-10T13:42:13.448 INFO:teuthology.orchestra.run.gibba011.stdout: remote refid st t when poll reach delay offset jitter 2021-03-10T13:42:13.448 INFO:teuthology.orchestra.run.gibba011.stdout:============================================================================== 2021-03-10T13:42:13.449 INFO:teuthology.orchestra.run.gibba011.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:13.449 INFO:teuthology.orchestra.run.gibba011.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:13.449 INFO:teuthology.orchestra.run.gibba011.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:13.450 INFO:teuthology.orchestra.run.gibba011.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:13.450 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-10T13:42:13.451 DEBUG:teuthology.orchestra.run.gibba032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-10T13:42:13.471 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-03-10T13:42:13.472 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Command line: ntpd -gq 2021-03-10T13:42:13.472 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: proto: precision = 0.033 usec (-25) 2021-03-10T13:42:13.472 INFO:teuthology.orchestra.run.gibba032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-10T13:42:13.473 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-10T13:42:13.473 INFO:teuthology.orchestra.run.gibba032.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-10T13:42:13.474 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: restrict ::: KOD does nothing without LIMITED. 2021-03-10T13:42:13.475 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen and drop on 0 v6wildcard [::]:123 2021-03-10T13:42:13.475 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-10T13:42:13.475 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen normally on 2 lo 127.0.0.1:123 2021-03-10T13:42:13.475 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen normally on 3 enp1s0f0 172.21.2.132:123 2021-03-10T13:42:13.476 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen normally on 4 lo [::1]:123 2021-03-10T13:42:13.476 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:9240%4]:123 2021-03-10T13:42:13.476 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:13 ntpd[8651]: Listening on routing socket on fd #22 for interface updates 2021-03-10T13:42:14.472 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:14.472 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:14.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:14.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:14.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:14.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:14 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:15.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:15.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:15.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:15.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:15.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:15.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:15.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:15.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:15.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:15.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:15.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:15.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:15 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:16.472 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:16.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:16.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:16.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:16.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:16.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:16 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:17.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:17.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:17.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:17.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:17.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:17.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:17.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:17.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:17.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:17.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:17.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:17.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:17 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:18.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:18.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:18.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:18.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:18.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:18.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:18.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:18.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:18.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:18.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:18.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:18.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:18 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:19.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:19.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:19.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:19.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:19.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:19.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:19.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:19.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:19.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:19.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:19.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:19.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:19 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:20.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:20.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:20.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:20.473 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:20.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-10T13:42:20.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: can't open /var/log/ntpstats/rawstats.20210310: Permission denied 2021-03-10T13:42:20.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-10T13:42:20.474 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: can't open /var/log/ntpstats/peerstats.20210310: Permission denied 2021-03-10T13:42:20.475 INFO:teuthology.orchestra.run.gibba032.stdout:10 Mar 13:42:20 ntpd[8651]: ntpd: time set +0.076248 s 2021-03-10T13:42:20.475 INFO:teuthology.orchestra.run.gibba032.stdout:ntpd: time set +0.076248s 2021-03-10T13:42:20.475 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-10T13:42:20.476 INFO:teuthology.orchestra.run.gibba032.stderr:10 Mar 13:42:20 ntpd[8651]: can't open /var/log/ntpstats/loopstats.20210310: Permission denied 2021-03-10T13:42:20.501 INFO:teuthology.orchestra.run.gibba032.stdout: remote refid st t when poll reach delay offset jitter 2021-03-10T13:42:20.501 INFO:teuthology.orchestra.run.gibba032.stdout:============================================================================== 2021-03-10T13:42:20.501 INFO:teuthology.orchestra.run.gibba032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:20.501 INFO:teuthology.orchestra.run.gibba032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:20.501 INFO:teuthology.orchestra.run.gibba032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:20.502 INFO:teuthology.orchestra.run.gibba032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-10T13:42:20.502 INFO:teuthology.run_tasks:Running task cephadm... 2021-03-10T13:42:20.560 INFO:tasks.cephadm:Config: {'allow_ptrace': False, 'cephadm_branch': 'v15.2.0', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'docker.io/ceph/ceph:v15.2.0', 'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'osd shutdown pgref assert': True}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b26fc080937a021ce826cdc94d1adb52c064381a'} 2021-03-10T13:42:20.560 INFO:tasks.cephadm:Cluster image is docker.io/ceph/ceph:v15.2.0 2021-03-10T13:42:20.561 INFO:tasks.cephadm:Cluster fsid is 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:20.561 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2021-03-10T13:42:20.562 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.2.111', 'mon.c': '[v2:172.21.2.111:3301,v1:172.21.2.111:6790]', 'mon.b': '172.21.2.132'} 2021-03-10T13:42:20.562 INFO:tasks.cephadm:First mon is mon.a on gibba011 2021-03-10T13:42:20.563 INFO:tasks.cephadm:First mgr is y 2021-03-10T13:42:20.563 INFO:tasks.cephadm:Normalizing hostnames... 2021-03-10T13:42:20.563 DEBUG:teuthology.orchestra.run.gibba011:> sudo hostname $(hostname -s) 2021-03-10T13:42:20.579 DEBUG:teuthology.orchestra.run.gibba032:> sudo hostname $(hostname -s) 2021-03-10T13:42:20.586 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v15.2.0)... 2021-03-10T13:42:20.587 DEBUG:teuthology.orchestra.run.gibba011:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v15.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.674 INFO:teuthology.orchestra.run.gibba011.stdout:-rw-rw-r-- 1 ubuntu ubuntu 151122 Mar 10 13:42 /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.675 DEBUG:teuthology.orchestra.run.gibba032:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v15.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.788 INFO:teuthology.orchestra.run.gibba032.stdout:-rw-rw-r-- 1 ubuntu ubuntu 151122 Mar 10 13:42 /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.788 DEBUG:teuthology.orchestra.run.gibba011:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.793 DEBUG:teuthology.orchestra.run.gibba032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2021-03-10T13:42:20.810 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /etc/ceph 2021-03-10T13:42:20.848 DEBUG:teuthology.orchestra.run.gibba032:> sudo mkdir -p /etc/ceph 2021-03-10T13:42:20.856 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 777 /etc/ceph 2021-03-10T13:42:20.876 DEBUG:teuthology.orchestra.run.gibba032:> sudo chmod 777 /etc/ceph 2021-03-10T13:42:20.886 INFO:tasks.cephadm:Writing seed config... 2021-03-10T13:42:20.887 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2021-03-10T13:42:20.888 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2021-03-10T13:42:20.888 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2021-03-10T13:42:20.888 INFO:tasks.cephadm: override: [mon] debug mon = 20 2021-03-10T13:42:20.889 INFO:tasks.cephadm: override: [mon] debug ms = 1 2021-03-10T13:42:20.889 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2021-03-10T13:42:20.889 INFO:tasks.cephadm: override: [osd] debug filestore = 20 2021-03-10T13:42:20.889 INFO:tasks.cephadm: override: [osd] debug journal = 20 2021-03-10T13:42:20.890 INFO:tasks.cephadm: override: [osd] debug ms = 1 2021-03-10T13:42:20.890 INFO:tasks.cephadm: override: [osd] debug osd = 25 2021-03-10T13:42:20.890 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2021-03-10T13:42:20.891 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:42:20.892 DEBUG:teuthology.orchestra.run.gibba011:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2021-03-10T13:42:20.901 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6fd36d94-81a6-11eb-9080-001a4aab830c mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug filestore = 20 debug journal = 20 debug ms = 1 debug osd = 25 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2021-03-10T13:42:20.901 DEBUG:teuthology.orchestra.run.gibba011:mon.a> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service 2021-03-10T13:42:20.947 DEBUG:teuthology.orchestra.run.gibba011:mgr.y> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service 2021-03-10T13:42:20.948 INFO:tasks.cephadm:Bootstrapping... 2021-03-10T13:42:20.949 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 -v bootstrap --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.2.111 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2021-03-10T13:42:20.978 INFO:journalctl@ceph.mgr.y.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:42:20.986 INFO:journalctl@ceph.mon.a.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:42:21.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Could not locate podman: podman not found 2021-03-10T13:42:21.020 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Verifying podman|docker is present... 2021-03-10T13:42:21.020 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Verifying lvm2 is present... 2021-03-10T13:42:21.020 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Verifying time synchronization is in place... 2021-03-10T13:42:21.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chrony.service 2021-03-10T13:42:21.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chrony.service: No such file or directory 2021-03-10T13:42:21.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active chrony.service 2021-03-10T13:42:21.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chronyd.service 2021-03-10T13:42:21.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chronyd.service: No such file or directory 2021-03-10T13:42:21.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active chronyd.service 2021-03-10T13:42:21.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled systemd-timesyncd.service 2021-03-10T13:42:21.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout masked 2021-03-10T13:42:21.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active systemd-timesyncd.service 2021-03-10T13:42:21.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntpd.service 2021-03-10T13:42:21.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for ntpd.service: No such file or directory 2021-03-10T13:42:21.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active ntpd.service 2021-03-10T13:42:21.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntp.service 2021-03-10T13:42:21.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout enabled 2021-03-10T13:42:21.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active ntp.service 2021-03-10T13:42:21.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout active 2021-03-10T13:42:21.039 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Unit ntp.service is enabled and running 2021-03-10T13:42:21.039 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Repeating the final host check... 2021-03-10T13:42:21.039 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:podman|docker (/usr/bin/docker) is present 2021-03-10T13:42:21.040 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:systemctl is present 2021-03-10T13:42:21.040 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:lvcreate is present 2021-03-10T13:42:21.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chrony.service 2021-03-10T13:42:21.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chrony.service: No such file or directory 2021-03-10T13:42:21.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active chrony.service 2021-03-10T13:42:21.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chronyd.service 2021-03-10T13:42:21.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chronyd.service: No such file or directory 2021-03-10T13:42:21.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active chronyd.service 2021-03-10T13:42:21.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled systemd-timesyncd.service 2021-03-10T13:42:21.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout masked 2021-03-10T13:42:21.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active systemd-timesyncd.service 2021-03-10T13:42:21.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntpd.service 2021-03-10T13:42:21.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for ntpd.service: No such file or directory 2021-03-10T13:42:21.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active ntpd.service 2021-03-10T13:42:21.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout inactive 2021-03-10T13:42:21.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntp.service 2021-03-10T13:42:21.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout enabled 2021-03-10T13:42:21.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl is-active ntp.service 2021-03-10T13:42:21.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stdout active 2021-03-10T13:42:21.058 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Unit ntp.service is enabled and running 2021-03-10T13:42:21.058 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Host looks OK 2021-03-10T13:42:21.059 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:root:Cluster fsid: 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:21.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Acquiring lock 140189835284832 on /run/cephadm/6fd36d94-81a6-11eb-9080-001a4aab830c.lock 2021-03-10T13:42:21.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Lock 140189835284832 acquired on /run/cephadm/6fd36d94-81a6-11eb-9080-001a4aab830c.lock 2021-03-10T13:42:21.059 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Verifying IP 172.21.2.111 port 3300 ... 2021-03-10T13:42:21.060 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Verifying IP 172.21.2.111 port 6789 ... 2021-03-10T13:42:21.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Base mon IP is 172.21.2.111, final addrv is [v2:172.21.2.111:3300,v1:172.21.2.111:6789] 2021-03-10T13:42:21.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/sbin/ip route ls 2021-03-10T13:42:21.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/sbin/ip:stdout default via 172.21.15.254 dev enp1s0f0 2021-03-10T13:42:21.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/sbin/ip:stdout default via 172.21.15.254 dev enp1s0f0 proto dhcp src 172.21.2.111 metric 100 2021-03-10T13:42:21.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/sbin/ip:stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2021-03-10T13:42:21.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/sbin/ip:stdout 172.21.0.0/20 dev enp1s0f0 proto kernel scope link src 172.21.2.111 2021-03-10T13:42:21.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/sbin/ip:stdout 172.21.15.254 dev enp1s0f0 proto dhcp scope link src 172.21.2.111 metric 100 2021-03-10T13:42:21.062 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Mon IP 172.21.2.111 is in CIDR network 172.21.0.0/20 2021-03-10T13:42:21.062 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Pulling latest docker.io/ceph/ceph:v15.2.0 container... 2021-03-10T13:42:21.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker pull docker.io/ceph/ceph:v15.2.0 2021-03-10T13:42:22.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout v15.2.0: Pulling from ceph/ceph 2021-03-10T13:42:22.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 8a29a15cefae: Pulling fs layer 2021-03-10T13:42:22.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 120f0bd0d532: Pulling fs layer 2021-03-10T13:42:22.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 8a29a15cefae: Verifying Checksum 2021-03-10T13:42:22.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 8a29a15cefae: Download complete 2021-03-10T13:42:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 120f0bd0d532: Verifying Checksum 2021-03-10T13:42:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 120f0bd0d532: Download complete 2021-03-10T13:42:23.820 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 8a29a15cefae: Pull complete 2021-03-10T13:42:27.708 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout 120f0bd0d532: Pull complete 2021-03-10T13:42:27.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout Digest: sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60 2021-03-10T13:42:27.833 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout Status: Downloaded newer image for ceph/ceph:v15.2.0 2021-03-10T13:42:27.835 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/docker:stdout docker.io/ceph/ceph:v15.2.0 2021-03-10T13:42:27.841 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Extracting ceph user uid/gid from container image... 2021-03-10T13:42:27.841 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '--entrypoint', 'stat', 'docker.io/ceph/ceph:v15.2.0', '-c', '%u %g', '/var/lib/ceph'] 2021-03-10T13:42:27.841 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 --entrypoint stat docker.io/ceph/ceph:v15.2.0 -c %u %g /var/lib/ceph 2021-03-10T13:42:36.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:stat:stdout 167 167 2021-03-10T13:42:36.688 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Creating initial keys... 2021-03-10T13:42:36.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '--entrypoint', '/usr/bin/ceph-authtool', 'docker.io/ceph/ceph:v15.2.0', '--gen-print-key'] 2021-03-10T13:42:36.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 --entrypoint /usr/bin/ceph-authtool docker.io/ceph/ceph:v15.2.0 --gen-print-key 2021-03-10T13:42:37.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQDNzEhgADfvFxAAMVONMVdYEGOTva+8jh9IPw== 2021-03-10T13:42:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '--entrypoint', '/usr/bin/ceph-authtool', 'docker.io/ceph/ceph:v15.2.0', '--gen-print-key'] 2021-03-10T13:42:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 --entrypoint /usr/bin/ceph-authtool docker.io/ceph/ceph:v15.2.0 --gen-print-key 2021-03-10T13:42:38.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQDOzEhgJz6+JRAAYuHGm/lyLcHzGDu1a0oGhA== 2021-03-10T13:42:39.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '--entrypoint', '/usr/bin/ceph-authtool', 'docker.io/ceph/ceph:v15.2.0', '--gen-print-key'] 2021-03-10T13:42:39.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 --entrypoint /usr/bin/ceph-authtool docker.io/ceph/ceph:v15.2.0 --gen-print-key 2021-03-10T13:42:39.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQDPzEhgPvsiOBAASy1/DP0TtZ7AvjGBSOnGmg== 2021-03-10T13:42:40.449 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Creating initial monmap... 2021-03-10T13:42:40.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/tmp/ceph-tmpydiz64ul:/tmp/monmap:z', '--entrypoint', '/usr/bin/monmaptool', 'docker.io/ceph/ceph:v15.2.0', '--create', '--clobber', '--fsid', '6fd36d94-81a6-11eb-9080-001a4aab830c', '--addv', 'a', '[v2:172.21.2.111:3300,v1:172.21.2.111:6789]', '/tmp/monmap'] 2021-03-10T13:42:40.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /tmp/ceph-tmpydiz64ul:/tmp/monmap:z --entrypoint /usr/bin/monmaptool docker.io/ceph/ceph:v15.2.0 --create --clobber --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c --addv a [v2:172.21.2.111:3300,v1:172.21.2.111:6789] /tmp/monmap 2021-03-10T13:42:41.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2021-03-10T13:42:41.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: set fsid to 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:41.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2021-03-10T13:42:41.766 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Creating mon... 2021-03-10T13:42:41.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '-v', '/tmp/ceph-tmpr9f8dj9t:/tmp/keyring:z', '-v', '/tmp/ceph-tmpydiz64ul:/tmp/monmap:z', '--entrypoint', '/usr/bin/ceph-mon', 'docker.io/ceph/ceph:v15.2.0', '--mkfs', '-i', 'a', '--fsid', '6fd36d94-81a6-11eb-9080-001a4aab830c', '-c', '/dev/null', '--monmap', '/tmp/monmap', '--keyring', '/tmp/keyring', '--setuser', 'ceph', '--setgroup', 'ceph', '--default-log-to-file=false', '--default-log-to-stderr=true', '--default-log-stderr-prefix="debug "', '--default-mon-cluster-log-to-file=false', '--default-mon-cluster-log-to-stderr=true'] 2021-03-10T13:42:41.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpr9f8dj9t:/tmp/keyring:z -v /tmp/ceph-tmpydiz64ul:/tmp/monmap:z --entrypoint /usr/bin/ceph-mon docker.io/ceph/ceph:v15.2.0 --mkfs -i a --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix="debug " --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2021-03-10T13:42:42.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.464+0000 7f93b86266c0 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:42:42.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.464+0000 7f93b86266c0 1 imported monmap: 2021-03-10T13:42:42.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr epoch 0 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr last_changed 2021-03-10T13:42:41.205251+0000 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr created 2021-03-10T13:42:41.205251+0000 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr min_mon_release 0 (unknown) 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 0: [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] mon.a 2021-03-10T13:42:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.464+0000 7f93b86266c0 0 /usr/bin/ceph-mon: set fsid to 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:42:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:42:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:42:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:42:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:42:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:42:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: RocksDB version: 6.1.2 2021-03-10T13:42:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:42:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Compile date Mar 23 2020 2021-03-10T13:42:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: DB SUMMARY 2021-03-10T13:42:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2021-03-10T13:42:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.create_if_missing: 1 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:42:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.env: 0x56132dc9d1a0 2021-03-10T13:42:42.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.info_log: 0x56132eb7ba80 2021-03-10T13:42:42.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:42:42.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:42:42.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:42:42.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:42:42.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:42:42.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:42:42.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:42:42.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:42:42.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:42:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:42:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:42:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:42:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:42:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.db_log_dir: 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:42:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:42:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:42:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:42:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:42:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:42:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:42:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.write_buffer_manager: 0x56132f71c120 2021-03-10T13:42:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:42:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:42:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:42:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:42:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:42:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:42:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:42:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:42:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:42:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:42:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:42:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:42:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:42:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.row_cache: None 2021-03-10T13:42:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.wal_filter: None 2021-03-10T13:42:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:42:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:42:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:42:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:42:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:42:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:42:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:42:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:42:42.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:42:42.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:42:42.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:42:42.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:42:42.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Compression algorithms supported: 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:42:42.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kZSTD supported: 0 2021-03-10T13:42:42.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:42:42.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:42:42.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:42:42.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:42:42.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:42:42.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:42:42.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:42:42.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: [db/db_impl_open.cc:228] Creating manifest 1 2021-03-10T13:42:42.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.merge_operator: 2021-03-10T13:42:42.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:42:42.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:42:42.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:42:42.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:42:42.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56132eaa4130) 2021-03-10T13:42:42.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr cache_index_and_filter_blocks: 1 2021-03-10T13:42:42.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr cache_index_and_filter_blocks_with_high_priority: 1 2021-03-10T13:42:42.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:42:42.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr pin_top_level_index_and_filter: 1 2021-03-10T13:42:42.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr index_type: 0 2021-03-10T13:42:42.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr data_block_index_type: 0 2021-03-10T13:42:42.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:42:42.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr hash_index_allow_collision: 1 2021-03-10T13:42:42.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr checksum: 1 2021-03-10T13:42:42.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr no_block_cache: 0 2021-03-10T13:42:42.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache: 0x56132eada510 2021-03-10T13:42:42.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_name: BinnedLRUCache 2021-03-10T13:42:42.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_options: 2021-03-10T13:42:42.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr capacity : 536870912 2021-03-10T13:42:42.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr num_shard_bits : 4 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr strict_capacity_limit : 0 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr high_pri_pool_ratio: 0.000 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_compressed: (nil) 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr persistent_cache: (nil) 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_size: 4096 2021-03-10T13:42:42.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_size_deviation: 10 2021-03-10T13:42:42.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_restart_interval: 16 2021-03-10T13:42:42.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr index_block_restart_interval: 1 2021-03-10T13:42:42.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr metadata_block_size: 4096 2021-03-10T13:42:42.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr partition_filters: 0 2021-03-10T13:42:42.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr use_delta_encoding: 1 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr whole_key_filtering: 1 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr verify_compression: 0 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr read_amp_bytes_per_bit: 0 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr format_version: 2 2021-03-10T13:42:42.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr enable_index_compression: 1 2021-03-10T13:42:42.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_align: 0 2021-03-10T13:42:42.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:42:42.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:42:42.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:42:42.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:42:42.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:42:42.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:42:42.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.num_levels: 7 2021-03-10T13:42:42.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:42:42.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:42:42.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:42:42.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:42:42.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:42:42.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:42:42.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:42:42.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:42:42.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:42:42.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:42:42.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:42:42.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:42:42.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:42:42.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:42:42.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:42:42.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:42:42.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:42:42.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:42:42.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:42:42.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:42:42.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:42:42.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:42:42.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:42:42.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:42:42.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:42:42.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:42:42.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:42:42.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:42:42.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:42:42.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:42:42.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:42:42.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:42:42.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:42:42.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:42:42.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:42:42.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:42:42.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:42:42.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:42:42.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:42:42.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:42:42.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:42:42.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:42:42.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:42:42.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.468+0000 7f93b86266c0 4 rocksdb: Options.ttl: 0 2021-03-10T13:42:42.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:42:42.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 0 2021-03-10T13:42:42.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 4 rocksdb: DB pointer 0x56132eb75200 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93a17e7700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93a17e7700 4 rocksdb: [db/db_impl.cc:778] 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** DB Stats ** 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:42:42.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:42:42.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:42:42.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:42:42.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:42:42.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:42:42.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2021-03-10T13:42:42.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:42.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:42.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:42:42.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:42:42.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2021-03-10T13:42:42.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:42.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:42.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:42:42.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:42.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:42.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:42:42.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:42:42.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2021-03-10T13:42:42.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:42.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:42.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:42:42.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:42:42.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2021-03-10T13:42:42.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:42.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:42.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:42.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:42.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:42.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:42:42.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:42:42.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:42:42.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2021-03-10T13:42:42.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 4 rocksdb: [db/db_impl.cc:390] Shutdown: canceling all background work 2021-03-10T13:42:42.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 4 rocksdb: [db/db_impl.cc:563] Shutdown complete 2021-03-10T13:42:42.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2021-03-10T13:42:42.472+0000 7f93b86266c0 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2021-03-10T13:42:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl disable ceph.target 2021-03-10T13:42:43.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl enable ceph.target 2021-03-10T13:42:43.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2021-03-10T13:42:43.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl start ceph.target 2021-03-10T13:42:43.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target 2021-03-10T13:42:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target -> /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target. 2021-03-10T13:42:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target -> /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target. 2021-03-10T13:42:43.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl start ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target 2021-03-10T13:42:43.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl daemon-reload 2021-03-10T13:42:43.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl stop ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a 2021-03-10T13:42:43.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl reset-failed ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a 2021-03-10T13:42:43.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to reset failed state of unit ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service: Unit ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service not loaded. 2021-03-10T13:42:43.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a 2021-03-10T13:42:43.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target.wants/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service -> /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@.service. 2021-03-10T13:42:43.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl start ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a 2021-03-10T13:42:43.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:firewalld does not appear to be present 2021-03-10T13:42:43.647 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for mon to start... 2021-03-10T13:42:43.647 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for mon... 2021-03-10T13:42:43.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 status 2021-03-10T13:42:44.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:43 gibba011 docker[9399]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.a 2021-03-10T13:42:44.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:43 gibba011 systemd[1]: Started Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:42:44.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout cluster: 2021-03-10T13:42:44.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout id: 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:44.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout health: HEALTH_OK 2021-03-10T13:42:44.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:44.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout services: 2021-03-10T13:42:44.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon: 1 daemons, quorum a (age 0.0981068s) 2021-03-10T13:42:44.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mgr: no daemons active 2021-03-10T13:42:44.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd: 0 osds: 0 up, 0 in 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout data: 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout pools: 0 pools, 0 pgs 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout objects: 0 objects, 0 B 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout usage: 0 B used, 0 B / 0 B avail 2021-03-10T13:42:44.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout pgs: 2021-03-10T13:42:44.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:44.780 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:44 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.617191+0000 mon.a (mon.0) 0 : [INF] mkfs 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:44.781 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:44 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.613467+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2021-03-10T13:42:45.291 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Assimilating anything we can from ceph.conf... 2021-03-10T13:42:45.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '-v', '/var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'config', 'assimilate-conf', '-i', '/var/lib/ceph/mon/ceph-a/config'] 2021-03-10T13:42:45.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z -v /var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 config assimilate-conf -i /var/lib/ceph/mon/ceph-a/config 2021-03-10T13:42:45.928 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.616383+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2021-03-10T13:42:45.928 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.616723+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0]} 2021-03-10T13:42:45.929 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.618430+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2021-03-10T13:42:45.929 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.620253+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2021-03-10T13:42:45.929 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: cluster 2021-03-10T13:42:44.620698+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2021-03-10T13:42:45.929 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:45 gibba011 bash[9414]: audit 2021-03-10T13:42:44.714464+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.2.111:0/3244329254' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2021-03-10T13:42:46.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:46.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [global] 2021-03-10T13:42:46.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout fsid = 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:46.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_election_default_strategy = 1 2021-03-10T13:42:46.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_host = [v2:172.21.2.111:3300,v1:172.21.2.111:6789] 2021-03-10T13:42:46.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_osd_allow_pg_remap = true 2021-03-10T13:42:46.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_osd_allow_primary_affinity = true 2021-03-10T13:42:46.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_warn_on_no_sortbitwise = false 2021-03-10T13:42:46.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_crush_chooseleaf_type = 0 2021-03-10T13:42:46.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:46.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [mgr] 2021-03-10T13:42:46.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mgr/telemetry/nag = false 2021-03-10T13:42:46.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:46.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [osd] 2021-03-10T13:42:46.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_map_max_advance = 10 2021-03-10T13:42:46.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_sloppy_crc = true 2021-03-10T13:42:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Generating new minimal ceph.conf... 2021-03-10T13:42:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '-v', '/var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'config', 'generate-minimal-conf', '-o', '/var/lib/ceph/mon/ceph-a/config'] 2021-03-10T13:42:46.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z -v /var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-a/config 2021-03-10T13:42:47.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:46 gibba011 bash[9414]: audit 2021-03-10T13:42:46.167669+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.2.111:0/3543021683' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2021-03-10T13:42:47.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:46 gibba011 bash[9414]: audit 2021-03-10T13:42:46.169796+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.2.111:0/3543021683' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2021-03-10T13:42:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:47 gibba011 bash[9414]: audit 2021-03-10T13:42:47.702736+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.2.111:0/3244486949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:42:48.270 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Restarting the monitor... 2021-03-10T13:42:48.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl restart ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a 2021-03-10T13:42:48.571 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 systemd[1]: Stopping Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:42:48.571 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 bash[9414]: debug 2021-03-10T13:42:48.344+0000 7f4514f36700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:42:48.571 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 bash[9414]: debug 2021-03-10T13:42:48.344+0000 7f4514f36700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2021-03-10T13:42:48.988 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Setting mon public_network... 2021-03-10T13:42:48.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'config', 'set', 'mon', 'public_network', '172.21.0.0/20'] 2021-03-10T13:42:48.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 config set mon public_network 172.21.0.0/20 2021-03-10T13:42:49.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 docker[9759]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.a 2021-03-10T13:42:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service: Succeeded. 2021-03-10T13:42:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 systemd[1]: Stopped Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:42:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 systemd[1]: Starting Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:42:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 docker[9794]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.a 2021-03-10T13:42:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:48 gibba011 systemd[1]: Started Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:42:50.214 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.012+0000 7f64b12026c0 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:42:50.214 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.012+0000 7f64b12026c0 0 ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc), process ceph-mon, pid 1 2021-03-10T13:42:50.214 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.012+0000 7f64b12026c0 0 pidfile_write: ignore empty --pid-file 2021-03-10T13:42:50.214 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 load: jerasure load: lrc load: isa 2021-03-10T13:42:50.215 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:42:50.215 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:42:50.215 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:42:50.215 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:42:50.215 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:42:50.216 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:42:50.216 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 1 rocksdb: do_open column families: [default] 2021-03-10T13:42:50.216 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: RocksDB version: 6.1.2 2021-03-10T13:42:50.216 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:42:50.216 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Compile date Mar 23 2020 2021-03-10T13:42:50.217 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: DB SUMMARY 2021-03-10T13:42:50.217 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: CURRENT file: CURRENT 2021-03-10T13:42:50.217 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: IDENTITY file: IDENTITY 2021-03-10T13:42:50.217 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 109 Bytes 2021-03-10T13:42:50.217 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2021-03-10T13:42:50.218 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 69025 ; 2021-03-10T13:42:50.218 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:42:50.218 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.create_if_missing: 0 2021-03-10T13:42:50.218 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.env: 0x5639356821a0 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.info_log: 0x5639362249a0 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:42:50.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:42:50.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:42:50.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:42:50.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:42:50.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:42:50.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:42:50.221 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:42:50.221 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:42:50.221 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:42:50.221 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:42:50.221 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:42:50.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.db_log_dir: 2021-03-10T13:42:50.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2021-03-10T13:42:50.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:42:50.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:42:50.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:42:50.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:42:50.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:42:50.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:42:50.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:42:50.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:42:50.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:42:50.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.write_buffer_manager: 0x563936dbeae0 2021-03-10T13:42:50.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:42:50.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:42:50.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:42:50.225 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:42:50.225 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:42:50.225 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:42:50.225 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:42:50.226 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:42:50.226 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:42:50.226 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:42:50.226 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:42:50.226 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:42:50.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:42:50.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.row_cache: None 2021-03-10T13:42:50.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.wal_filter: None 2021-03-10T13:42:50.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:42:50.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:42:50.228 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:42:50.228 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:42:50.228 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:42:50.228 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:42:50.228 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:42:50.229 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:42:50.229 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:42:50.229 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:42:50.229 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:42:50.229 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:42:50.230 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:42:50.230 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:42:50.230 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:42:50.230 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:42:50.231 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:42:50.231 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:42:50.231 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:42:50.231 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:42:50.231 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Compression algorithms supported: 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kZSTD supported: 0 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:42:50.232 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:42:50.233 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:42:50.233 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:42:50.233 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:42:50.233 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:42:50.233 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2021-03-10T13:42:50.234 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2021-03-10T13:42:50.234 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:42:50.234 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.merge_operator: 2021-03-10T13:42:50.234 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:42:50.234 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:42:50.235 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:42:50.235 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:42:50.235 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563936144138) 2021-03-10T13:42:50.235 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cache_index_and_filter_blocks: 1 2021-03-10T13:42:50.236 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cache_index_and_filter_blocks_with_high_priority: 1 2021-03-10T13:42:50.236 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:42:50.236 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: pin_top_level_index_and_filter: 1 2021-03-10T13:42:50.236 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: index_type: 0 2021-03-10T13:42:50.236 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: data_block_index_type: 0 2021-03-10T13:42:50.237 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:42:50.237 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: hash_index_allow_collision: 1 2021-03-10T13:42:50.237 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: checksum: 1 2021-03-10T13:42:50.237 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: no_block_cache: 0 2021-03-10T13:42:50.237 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_cache: 0x56393617a610 2021-03-10T13:42:50.238 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_cache_name: BinnedLRUCache 2021-03-10T13:42:50.238 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_cache_options: 2021-03-10T13:42:50.238 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: capacity : 536870912 2021-03-10T13:42:50.238 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: num_shard_bits : 4 2021-03-10T13:42:50.238 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: strict_capacity_limit : 0 2021-03-10T13:42:50.239 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: high_pri_pool_ratio: 0.000 2021-03-10T13:42:50.239 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_cache_compressed: (nil) 2021-03-10T13:42:50.239 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: persistent_cache: (nil) 2021-03-10T13:42:50.239 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_size: 4096 2021-03-10T13:42:50.239 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_size_deviation: 10 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_restart_interval: 16 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: index_block_restart_interval: 1 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: metadata_block_size: 4096 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: partition_filters: 0 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: use_delta_encoding: 1 2021-03-10T13:42:50.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:42:50.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: whole_key_filtering: 1 2021-03-10T13:42:50.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: verify_compression: 0 2021-03-10T13:42:50.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: read_amp_bytes_per_bit: 0 2021-03-10T13:42:50.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: format_version: 2 2021-03-10T13:42:50.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: enable_index_compression: 1 2021-03-10T13:42:50.242 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: block_align: 0 2021-03-10T13:42:50.242 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:42:50.242 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:42:50.242 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:42:50.242 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:42:50.243 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:42:50.243 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:42:50.245 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.num_levels: 7 2021-03-10T13:42:50.246 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:42:50.246 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:42:50.246 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:42:50.246 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:42:50.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:42:50.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:42:50.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:42:50.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:42:50.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:42:50.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:42:50.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:42:50.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:42:50.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:42:50.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:42:50.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:42:50.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:42:50.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:42:50.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:42:50.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:42:50.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:42:50.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:42:50.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:42:50.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:42:50.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:42:50.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:42:50.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:42:50.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:42:50.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:42:50.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:42:50.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:42:50.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:42:50.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:42:50.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:42:50.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:42:50.253 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:42:50.253 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:42:50.253 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:42:50.253 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:42:50.253 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:42:50.254 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:42:50.254 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:42:50.254 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:42:50.254 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:42:50.255 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:42:50.255 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:42:50.255 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:42:50.255 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:42:50.255 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:42:50.256 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:42:50.256 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:42:50.256 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:42:50.256 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:42:50.256 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:42:50.257 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:42:50.257 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:42:50.257 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:42:50.257 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:42:50.257 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:42:50.258 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: Options.ttl: 0 2021-03-10T13:42:50.258 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:42:50.258 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 4 2021-03-10T13:42:50.258 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383770022626, "job": 1, "event": "recovery_started", "log_files": [6]} 2021-03-10T13:42:50.259 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.016+0000 7f64b12026c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #6 mode 2 2021-03-10T13:42:50.259 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383770023402, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 66120, "table_properties": {"data_size": 64415, "index_size": 310, "filter_size": 581, "raw_key_size": 8602, "raw_average_key_size": 46, "raw_value_size": 59469, "raw_average_value_size": 321, "num_data_blocks": 10, "num_entries": 185, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2021-03-10T13:42:50.259 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 8 2021-03-10T13:42:50.259 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383770023952, "job": 1, "event": "recovery_finished"} 2021-03-10T13:42:50.259 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 4 rocksdb: DB pointer 0x563936215200 2021-03-10T13:42:50.260 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f649834f700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2021-03-10T13:42:50.260 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f649834f700 4 rocksdb: [db/db_impl.cc:778] 2021-03-10T13:42:50.260 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** DB Stats ** 2021-03-10T13:42:50.260 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:50.260 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:42:50.261 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:42:50.261 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:42:50.261 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:42:50.261 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:42:50.261 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:42:50.262 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** Compaction Stats [default] ** 2021-03-10T13:42:50.262 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:50.262 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:50.262 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: L0 2/0 66.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.262 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Sum 2/0 66.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.263 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.263 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** Compaction Stats [default] ** 2021-03-10T13:42:50.263 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:50.263 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:50.263 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:50.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:50.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:50.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:42:50.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:42:50.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:42:50.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Cumulative compaction: 0.00 GB write, 22.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:50.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Interval compaction: 0.00 GB write, 22.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:50.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:42:50.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:42:50.266 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** Compaction Stats [default] ** 2021-03-10T13:42:50.266 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:50.266 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:50.266 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: L0 2/0 66.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.266 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Sum 2/0 66.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** Compaction Stats [default] ** 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 113.6 0.00 0.00 1 0.001 0 0 2021-03-10T13:42:50.267 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:42:50.268 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:50.268 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:42:50.268 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:42:50.268 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:42:50.269 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:42:50.269 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Cumulative compaction: 0.00 GB write, 22.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:50.269 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:42:50.269 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:42:50.269 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:42:50.270 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 starting mon.a rank 0 at public addrs [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] at bind addrs [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:50.270 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 1 mon.a@-1(???) e1 preinit fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:42:50.270 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).mds e1 new map 2021-03-10T13:42:50.270 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).mds e1 print_map 2021-03-10T13:42:50.270 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: e1 2021-03-10T13:42:50.271 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: enable_multiple, ever_enabled_multiple: 0,0 2021-03-10T13:42:50.271 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-03-10T13:42:50.271 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: legacy client fscid: -1 2021-03-10T13:42:50.271 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: 2021-03-10T13:42:50.271 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: No filesystems configured 2021-03-10T13:42:50.272 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-03-10T13:42:50.272 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-03-10T13:42:50.272 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2021-03-10T13:42:50.272 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-03-10T13:42:50.272 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: audit 2021-03-10T13:42:47.702736+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.2.111:0/3244486949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:42:50.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2021-03-10T13:42:50.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expand map: {default=false} 2021-03-10T13:42:50.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta from 'false' to 'false' 2021-03-10T13:42:50.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expanded map: {default=false} 2021-03-10T13:42:50.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expand map: {default=info} 2021-03-10T13:42:50.274 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta from 'info' to 'info' 2021-03-10T13:42:50.274 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expanded map: {default=info} 2021-03-10T13:42:50.274 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expand map: {default=daemon} 2021-03-10T13:42:50.274 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta from 'daemon' to 'daemon' 2021-03-10T13:42:50.275 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expanded map: {default=daemon} 2021-03-10T13:42:50.275 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expand map: {default=debug} 2021-03-10T13:42:50.275 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta from 'debug' to 'debug' 2021-03-10T13:42:50.275 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: debug 2021-03-10T13:42:50.020+0000 7f64b12026c0 20 expand_channel_meta expanded map: {default=debug} 2021-03-10T13:42:50.275 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cluster 2021-03-10T13:42:50.028904+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2021-03-10T13:42:50.276 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cluster 2021-03-10T13:42:50.028936+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0]} 2021-03-10T13:42:50.276 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cluster 2021-03-10T13:42:50.028964+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2021-03-10T13:42:50.276 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cluster 2021-03-10T13:42:50.028979+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2021-03-10T13:42:50.276 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:50 gibba011 bash[9813]: cluster 2021-03-10T13:42:50.029237+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2021-03-10T13:42:50.721 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Creating mgr... 2021-03-10T13:42:50.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl daemon-reload 2021-03-10T13:42:50.841 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl stop ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y 2021-03-10T13:42:50.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl reset-failed ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y 2021-03-10T13:42:50.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Failed to reset failed state of unit ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service: Unit ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service not loaded. 2021-03-10T13:42:50.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y 2021-03-10T13:42:50.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c.target.wants/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service -> /etc/systemd/system/ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@.service. 2021-03-10T13:42:50.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: systemctl start ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y 2021-03-10T13:42:50.973 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:50 gibba011 docker[10028]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mgr.y 2021-03-10T13:42:50.974 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:50 gibba011 systemd[1]: Started Ceph mgr.y for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:42:50.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:firewalld does not appear to be present 2021-03-10T13:42:50.975 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2021-03-10T13:42:50.975 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Wrote config to /etc/ceph/ceph.conf 2021-03-10T13:42:50.975 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for mgr to start... 2021-03-10T13:42:50.975 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for mgr... 2021-03-10T13:42:50.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'status', '-f', 'json-pretty'] 2021-03-10T13:42:50.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 status -f json-pretty 2021-03-10T13:42:51.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:51 gibba011 bash[9813]: audit 2021-03-10T13:42:50.158681+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.2.111:0/297813401' entity='client.admin' cmd=[{"prefix": "config set", "who": "mon", "name": "public_network", "value": "172.21.0.0/20"}]: dispatch 2021-03-10T13:42:51.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:51 gibba011 bash[9813]: audit 2021-03-10T13:42:50.160663+0000 mon.a (mon.0) 7 : audit [INF] from='client.? 172.21.2.111:0/297813401' entity='client.admin' cmd='[{"prefix": "config set", "who": "mon", "name": "public_network", "value": "172.21.0.0/20"}]': finished 2021-03-10T13:42:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:42:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "6fd36d94-81a6-11eb-9080-001a4aab830c", 2021-03-10T13:42:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2021-03-10T13:42:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2021-03-10T13:42:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2021-03-10T13:42:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2021-03-10T13:42:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2021-03-10T13:42:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2021-03-10T13:42:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2021-03-10T13:42:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2021-03-10T13:42:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2021-03-10T13:42:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 2, 2021-03-10T13:42:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2021-03-10T13:42:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "octopus", 2021-03-10T13:42:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2021-03-10T13:42:52.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2021-03-10T13:42:52.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:52.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2021-03-10T13:42:52.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2021-03-10T13:42:52.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2021-03-10T13:42:52.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2021-03-10T13:42:52.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2021-03-10T13:42:52.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2021-03-10T13:42:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2021-03-10T13:42:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2021-03-10T13:42:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2021-03-10T13:42:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2021-03-10T13:42:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2021-03-10T13:42:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2021-03-10T13:42:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2021-03-10T13:42:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2021-03-10T13:42:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2021-03-10T13:42:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2021-03-10T13:42:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2021-03-10T13:42:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2021-03-10T13:42:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2021-03-10T13:42:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": false, 2021-03-10T13:42:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2021-03-10T13:42:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2021-03-10T13:42:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2021-03-10T13:42:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2021-03-10T13:42:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2021-03-10T13:42:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2021-03-10T13:42:44.617469+0000", 2021-03-10T13:42:52.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:52.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:52.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2021-03-10T13:42:52.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:42:52.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:52 gibba011 bash[10046]: debug 2021-03-10T13:42:52.172+0000 7f175ef98040 1 -- 172.21.2.111:0/163453645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5594ead00800 msgr2=0x5594eaca9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-03-10T13:42:52.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:52 gibba011 bash[10046]: debug 2021-03-10T13:42:52.172+0000 7f175ef98040 1 --2- 172.21.2.111:0/163453645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5594ead00800 0x5594eaca9a80 secure :-1 s=READY pgs=3 cs=0 l=1 rx=0x5594eacc8420 tx=0x5594eaccad70).stop 2021-03-10T13:42:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:52 gibba011 bash[9813]: audit 2021-03-10T13:42:52.313148+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.2.111:0/565844359' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-03-10T13:42:52.867 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:mgr not available, waiting (1/10)... 2021-03-10T13:42:53.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'status', '-f', 'json-pretty'] 2021-03-10T13:42:53.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 status -f json-pretty 2021-03-10T13:42:54.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:54.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:42:54.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "6fd36d94-81a6-11eb-9080-001a4aab830c", 2021-03-10T13:42:54.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2021-03-10T13:42:54.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2021-03-10T13:42:54.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2021-03-10T13:42:54.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2021-03-10T13:42:54.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2021-03-10T13:42:54.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2021-03-10T13:42:54.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2021-03-10T13:42:54.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:54.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2021-03-10T13:42:54.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2021-03-10T13:42:54.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:54.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 4, 2021-03-10T13:42:54.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2021-03-10T13:42:54.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:54.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "octopus", 2021-03-10T13:42:54.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2021-03-10T13:42:54.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2021-03-10T13:42:54.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:54.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2021-03-10T13:42:54.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2021-03-10T13:42:54.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2021-03-10T13:42:54.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2021-03-10T13:42:54.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2021-03-10T13:42:54.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2021-03-10T13:42:54.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2021-03-10T13:42:54.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2021-03-10T13:42:54.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2021-03-10T13:42:54.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2021-03-10T13:42:54.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2021-03-10T13:42:54.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:54.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2021-03-10T13:42:54.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2021-03-10T13:42:54.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2021-03-10T13:42:54.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": false, 2021-03-10T13:42:54.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2021-03-10T13:42:54.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2021-03-10T13:42:54.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:54.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2021-03-10T13:42:44.617469+0000", 2021-03-10T13:42:54.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:54.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:54.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2021-03-10T13:42:54.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:42:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:54 gibba011 bash[9813]: audit 2021-03-10T13:42:54.766630+0000 mon.a (mon.0) 9 : audit [DBG] from='client.? 172.21.2.111:0/403941028' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-03-10T13:42:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:mgr not available, waiting (2/10)... 2021-03-10T13:42:56.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: cluster 2021-03-10T13:42:54.888024+0000 mon.a (mon.0) 10 : cluster [INF] Activating manager daemon y 2021-03-10T13:42:56.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: cluster 2021-03-10T13:42:54.889962+0000 mon.a (mon.0) 11 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00199986s) 2021-03-10T13:42:56.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.890644+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:42:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.890772+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:42:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.890884+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:42:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.891125+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:42:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.891246+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:42:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: cluster 2021-03-10T13:42:54.892673+0000 mon.a (mon.0) 17 : cluster [INF] Manager daemon y is now available 2021-03-10T13:42:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.900074+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-03-10T13:42:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.970325+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2021-03-10T13:42:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: cluster 2021-03-10T13:42:54.970378+0000 mon.a (mon.0) 20 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2021-03-10T13:42:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.971349+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-03-10T13:42:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.974873+0000 mon.a (mon.0) 22 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/telemetry/report_id","val":"77308058-2e53-4243-973c-a50706b32e65"}]: dispatch 2021-03-10T13:42:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.976168+0000 mon.a (mon.0) 23 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/telemetry/report_id","val":"77308058-2e53-4243-973c-a50706b32e65"}]': finished 2021-03-10T13:42:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.976539+0000 mon.a (mon.0) 24 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/telemetry/salt","val":"375089f0-52d3-4061-8dad-633fa4cf8ac9"}]: dispatch 2021-03-10T13:42:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:55 gibba011 bash[9813]: audit 2021-03-10T13:42:54.977963+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/telemetry/salt","val":"375089f0-52d3-4061-8dad-633fa4cf8ac9"}]': finished 2021-03-10T13:42:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'status', '-f', 'json-pretty'] 2021-03-10T13:42:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 status -f json-pretty 2021-03-10T13:42:57.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2021-03-10T13:42:57.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:42:57.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "6fd36d94-81a6-11eb-9080-001a4aab830c", 2021-03-10T13:42:57.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2021-03-10T13:42:57.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2021-03-10T13:42:57.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2021-03-10T13:42:57.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2021-03-10T13:42:57.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2021-03-10T13:42:57.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2021-03-10T13:42:57.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2021-03-10T13:42:57.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:57.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2021-03-10T13:42:57.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2021-03-10T13:42:57.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:57.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 7, 2021-03-10T13:42:57.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2021-03-10T13:42:57.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:57.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "octopus", 2021-03-10T13:42:57.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2021-03-10T13:42:57.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2021-03-10T13:42:57.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 3, 2021-03-10T13:42:57.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2021-03-10T13:42:57.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2021-03-10T13:42:57.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2021-03-10T13:42:57.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2021-03-10T13:42:57.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2021-03-10T13:42:57.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2021-03-10T13:42:57.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2021-03-10T13:42:57.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2021-03-10T13:42:57.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2021-03-10T13:42:57.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2021-03-10T13:42:57.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2021-03-10T13:42:57.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2021-03-10T13:42:57.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2021-03-10T13:42:57.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2021-03-10T13:42:57.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2021-03-10T13:42:57.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2021-03-10T13:42:57.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:57.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2021-03-10T13:42:57.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2021-03-10T13:42:57.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2021-03-10T13:42:57.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2021-03-10T13:42:57.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2021-03-10T13:42:57.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2021-03-10T13:42:57.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2021-03-10T13:42:57.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2021-03-10T13:42:57.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:42:57.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:57.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2021-03-10T13:42:57.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2021-03-10T13:42:57.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2021-03-10T13:42:44.617469+0000", 2021-03-10T13:42:57.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2021-03-10T13:42:57.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:42:57.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2021-03-10T13:42:57.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:42:57.282 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:56 gibba011 bash[9813]: cluster 2021-03-10T13:42:55.895803+0000 mon.a (mon.0) 26 : cluster [DBG] mgrmap e3: y(active, since 1.00783s) 2021-03-10T13:42:57.282 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:56 gibba011 bash[9813]: audit 2021-03-10T13:42:55.976019+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.14100 172.21.2.111:0/2730113995' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-03-10T13:42:57.282 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:56 gibba011 bash[9813]: cluster 2021-03-10T13:42:55.976132+0000 mon.a (mon.0) 28 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2021-03-10T13:42:57.283 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:56 gibba011 bash[9813]: audit 2021-03-10T13:42:55.983594+0000 mon.a (mon.0) 29 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-03-10T13:42:57.283 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:56 gibba011 bash[9813]: audit 2021-03-10T13:42:56.138681+0000 mon.a (mon.0) 30 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-03-10T13:42:57.649 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Enabling cephadm module... 2021-03-10T13:42:57.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'mgr', 'module', 'enable', 'cephadm'] 2021-03-10T13:42:57.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 mgr module enable cephadm 2021-03-10T13:42:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:57 gibba011 bash[9813]: audit 2021-03-10T13:42:57.227432+0000 mon.a (mon.0) 31 : audit [DBG] from='client.? 172.21.2.111:0/1438556759' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-03-10T13:42:59.327 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:58 gibba011 bash[10046]: ignoring --setuser ceph since I am not root 2021-03-10T13:42:59.327 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:42:58 gibba011 bash[10046]: ignoring --setgroup ceph since I am not root 2021-03-10T13:42:59.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:58 gibba011 bash[9813]: cluster 2021-03-10T13:42:57.906136+0000 mon.a (mon.0) 32 : cluster [DBG] mgrmap e4: y(active, since 3s) 2021-03-10T13:42:59.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:58 gibba011 bash[9813]: audit 2021-03-10T13:42:58.760254+0000 mon.a (mon.0) 33 : audit [INF] from='client.? 172.21.2.111:0/2150233669' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2021-03-10T13:42:59.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'mgr', 'dump'] 2021-03-10T13:42:59.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 mgr dump 2021-03-10T13:43:00.182 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:59 gibba011 bash[9813]: audit 2021-03-10T13:42:58.907008+0000 mon.a (mon.0) 34 : audit [INF] from='client.? 172.21.2.111:0/2150233669' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2021-03-10T13:43:00.183 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:42:59 gibba011 bash[9813]: cluster 2021-03-10T13:42:58.907108+0000 mon.a (mon.0) 35 : cluster [DBG] mgrmap e5: y(active, since 4s) 2021-03-10T13:43:00.843 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.843 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 5, 2021-03-10T13:43:00.844 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_gid": 14100, 2021-03-10T13:43:00.844 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_name": "y", 2021-03-10T13:43:00.844 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addrs": { 2021-03-10T13:43:00.844 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:00.844 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.845 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:00.845 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:6800", 2021-03-10T13:43:00.845 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 2764398316 2021-03-10T13:43:00.845 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v1", 2021-03-10T13:43:00.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:6801", 2021-03-10T13:43:00.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 2764398316 2021-03-10T13:43:00.846 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.847 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:00.847 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.847 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addr": "172.21.2.111:6801/2764398316", 2021-03-10T13:43:00.847 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_change": "2021-03-10T13:42:54.887954+0000", 2021-03-10T13:43:00.847 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_mgr_features": 4540138292836696063, 2021-03-10T13:43:00.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2021-03-10T13:43:00.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standbys": [], 2021-03-10T13:43:00.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2021-03-10T13:43:00.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2021-03-10T13:43:00.848 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2021-03-10T13:43:00.849 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2021-03-10T13:43:00.849 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.849 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available_modules": [ 2021-03-10T13:43:00.849 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.849 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "alerts", 2021-03-10T13:43:00.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:00.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:00.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:00.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:00.850 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:00.851 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:00.851 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.851 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.851 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:00.851 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.852 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.852 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.852 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "How frequently to reexamine health status", 2021-03-10T13:43:00.852 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.852 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.853 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.854 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.854 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.854 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.854 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.854 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.855 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.855 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.855 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.855 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.855 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.856 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.856 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.856 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.856 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.856 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.857 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.858 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.858 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.858 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.858 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.859 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.859 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.859 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.859 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.859 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.860 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.860 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:00.860 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:00.860 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.860 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:00.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:00.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_destination": { 2021-03-10T13:43:00.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_destination", 2021-03-10T13:43:00.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email address to send alerts to", 2021-03-10T13:43:00.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_from_name": { 2021-03-10T13:43:00.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_from_name", 2021-03-10T13:43:00.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "Ceph", 2021-03-10T13:43:00.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email From: name", 2021-03-10T13:43:00.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_host": { 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_host", 2021-03-10T13:43:00.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP server", 2021-03-10T13:43:00.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_password": { 2021-03-10T13:43:00.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_password", 2021-03-10T13:43:00.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Password to authenticate with", 2021-03-10T13:43:00.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_port": { 2021-03-10T13:43:00.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_port", 2021-03-10T13:43:00.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:00.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "465", 2021-03-10T13:43:00.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP port", 2021-03-10T13:43:00.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_sender": { 2021-03-10T13:43:00.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_sender", 2021-03-10T13:43:00.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP envelope sender", 2021-03-10T13:43:00.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_ssl": { 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_ssl", 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:00.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Use SSL to connect to SMTP server", 2021-03-10T13:43:00.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_user": { 2021-03-10T13:43:00.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_user", 2021-03-10T13:43:00.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "User to authenticate as", 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "balancer", 2021-03-10T13:43:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:00.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active": { 2021-03-10T13:43:00.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "active", 2021-03-10T13:43:00.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically balance PGs across cluster", 2021-03-10T13:43:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_time": { 2021-03-10T13:43:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_time", 2021-03-10T13:43:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0000", 2021-03-10T13:43:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "beginning time of day to automatically balance", 2021-03-10T13:43:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_weekday": { 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_weekday", 2021-03-10T13:43:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:00.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:00.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2021-03-10T13:43:00.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2021-03-10T13:43:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to this day of the week or later", 2021-03-10T13:43:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-03-10T13:43:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_max_iterations": { 2021-03-10T13:43:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_max_iterations", 2021-03-10T13:43:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "25", 2021-03-10T13:43:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2021-03-10T13:43:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "250", 2021-03-10T13:43:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum number of iterations to attempt optimization", 2021-03-10T13:43:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_metrics": { 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_metrics", 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "pgs,objects,bytes", 2021-03-10T13:43:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "metrics with which to calculate OSD utilization", 2021-03-10T13:43:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2021-03-10T13:43:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_step": { 2021-03-10T13:43:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_step", 2021-03-10T13:43:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0.5", 2021-03-10T13:43:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0.001", 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "0.999", 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "aggressiveness of optimization", 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": ".99 is very aggressive, .01 is less aggressive", 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_time": { 2021-03-10T13:43:00.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_time", 2021-03-10T13:43:00.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2400", 2021-03-10T13:43:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "ending time of day to automatically balance", 2021-03-10T13:43:00.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2021-03-10T13:43:00.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_weekday": { 2021-03-10T13:43:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_weekday", 2021-03-10T13:43:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2021-03-10T13:43:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2021-03-10T13:43:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2021-03-10T13:43:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to days of the week earlier than this", 2021-03-10T13:43:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-03-10T13:43:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:00.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:00.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:00.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_score": { 2021-03-10T13:43:00.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_score", 2021-03-10T13:43:00.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:00.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "minimum score, below which no optimization is attempted", 2021-03-10T13:43:00.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2021-03-10T13:43:00.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2021-03-10T13:43:00.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "none", 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush-compat", 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "none", 2021-03-10T13:43:00.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap" 2021-03-10T13:43:00.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Balancer mode", 2021-03-10T13:43:00.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_ids": { 2021-03-10T13:43:00.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_ids", 2021-03-10T13:43:00.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "pools which the automatic balancing will be limited to", 2021-03-10T13:43:00.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:00.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:00.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:00.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:00.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and attempt optimization", 2021-03-10T13:43:00.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_deviation": { 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_deviation", 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:00.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2021-03-10T13:43:00.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "deviation below which no optimization is attempted", 2021-03-10T13:43:00.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_optimizations": { 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_optimizations", 2021-03-10T13:43:00.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:00.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:00.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum upmap optimizations to make per attempt", 2021-03-10T13:43:00.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "cephadm", 2021-03-10T13:43:00.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:00.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:00.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:00.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "allow_ptrace": { 2021-03-10T13:43:00.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "allow_ptrace", 2021-03-10T13:43:00.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "allow SYS_PTRACE capability on ceph containers", 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_base": { 2021-03-10T13:43:00.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_base", 2021-03-10T13:43:00.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph/ceph", 2021-03-10T13:43:00.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Container image name, without the tag", 2021-03-10T13:43:00.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "daemon_cache_timeout": { 2021-03-10T13:43:00.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "daemon_cache_timeout", 2021-03-10T13:43:00.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:00.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache service (daemon) inventory", 2021-03-10T13:43:00.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_cache_timeout": { 2021-03-10T13:43:00.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_cache_timeout", 2021-03-10T13:43:00.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:00.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1800", 2021-03-10T13:43:00.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache device inventory", 2021-03-10T13:43:00.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "host_check_interval": { 2021-03-10T13:43:00.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "host_check_interval", 2021-03-10T13:43:00.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:00.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:00.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to perform a host check", 2021-03-10T13:43:00.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:00.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:00.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:00.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:00.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:00.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "log to the \"cephadm\" cluster log channel\"", 2021-03-10T13:43:00.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:00.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:00.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:00.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:00.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:00.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "root", 2021-03-10T13:43:00.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm-package", 2021-03-10T13:43:00.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "root" 2021-03-10T13:43:00.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "mode for remote execution of cephadm", 2021-03-10T13:43:00.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssh_config_file": { 2021-03-10T13:43:00.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssh_config_file", 2021-03-10T13:43:00.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:00.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "customized SSH config file to connect to managed hosts", 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_failed_host_check": { 2021-03-10T13:43:00.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_failed_host_check", 2021-03-10T13:43:00.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:00.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if the host check fails", 2021-03-10T13:43:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_daemons": { 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_daemons", 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:00.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2021-03-10T13:43:00.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_hosts": { 2021-03-10T13:43:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_hosts", 2021-03-10T13:43:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crash", 2021-03-10T13:43:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:00.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:00.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:00.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:00.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:00.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:00.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:00.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:00.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:00.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:00.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:00.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:00.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:00.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:00.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:00.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retain_interval": { 2021-03-10T13:43:01.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retain_interval", 2021-03-10T13:43:01.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "31536000", 2021-03-10T13:43:01.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain crashes before pruning them", 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_recent_interval": { 2021-03-10T13:43:01.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_recent_interval", 2021-03-10T13:43:01.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1209600", 2021-03-10T13:43:01.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "time interval in which to warn about recent crashes", 2021-03-10T13:43:01.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "dashboard", 2021-03-10T13:43:01.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ALERTMANAGER_API_HOST": { 2021-03-10T13:43:01.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ALERTMANAGER_API_HOST", 2021-03-10T13:43:01.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_ENABLED": { 2021-03-10T13:43:01.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_ENABLED", 2021-03-10T13:43:01.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_LOG_PAYLOAD": { 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_LOG_PAYLOAD", 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ENABLE_BROWSABLE_API": { 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ENABLE_BROWSABLE_API", 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_cephfs": { 2021-03-10T13:43:01.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_cephfs", 2021-03-10T13:43:01.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_iscsi": { 2021-03-10T13:43:01.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_iscsi", 2021-03-10T13:43:01.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_mirroring": { 2021-03-10T13:43:01.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_mirroring", 2021-03-10T13:43:01.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_nfs": { 2021-03-10T13:43:01.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_nfs", 2021-03-10T13:43:01.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rbd": { 2021-03-10T13:43:01.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rbd", 2021-03-10T13:43:01.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rgw": { 2021-03-10T13:43:01.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rgw", 2021-03-10T13:43:01.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2021-03-10T13:43:01.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2021-03-10T13:43:01.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_PASSWORD": { 2021-03-10T13:43:01.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_PASSWORD", 2021-03-10T13:43:01.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:01.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_SSL_VERIFY": { 2021-03-10T13:43:01.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_SSL_VERIFY", 2021-03-10T13:43:01.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_URL": { 2021-03-10T13:43:01.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_URL", 2021-03-10T13:43:01.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_USERNAME": { 2021-03-10T13:43:01.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_USERNAME", 2021-03-10T13:43:01.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:01.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_UPDATE_DASHBOARDS": { 2021-03-10T13:43:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_UPDATE_DASHBOARDS", 2021-03-10T13:43:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ISCSI_API_SSL_VERIFICATION": { 2021-03-10T13:43:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ISCSI_API_SSL_VERIFICATION", 2021-03-10T13:43:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PROMETHEUS_API_HOST": { 2021-03-10T13:43:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PROMETHEUS_API_HOST", 2021-03-10T13:43:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2021-03-10T13:43:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2021-03-10T13:43:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2021-03-10T13:43:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2021-03-10T13:43:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2021-03-10T13:43:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2021-03-10T13:43:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2021-03-10T13:43:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2021-03-10T13:43:01.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2021-03-10T13:43:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2021-03-10T13:43:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2021-03-10T13:43:01.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2021-03-10T13:43:01.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_ENABLED": { 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_ENABLED", 2021-03-10T13:43:01.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_EXCLUSION_LIST": { 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_EXCLUSION_LIST", 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2021-03-10T13:43:01.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_COMPLEXITY": { 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_COMPLEXITY", 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:01.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_LENGTH": { 2021-03-10T13:43:01.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_LENGTH", 2021-03-10T13:43:01.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8", 2021-03-10T13:43:01.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "REST_REQUESTS_TIMEOUT": { 2021-03-10T13:43:01.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "REST_REQUESTS_TIMEOUT", 2021-03-10T13:43:01.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "45", 2021-03-10T13:43:01.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ACCESS_KEY": { 2021-03-10T13:43:01.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ACCESS_KEY", 2021-03-10T13:43:01.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ADMIN_RESOURCE": { 2021-03-10T13:43:01.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ADMIN_RESOURCE", 2021-03-10T13:43:01.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:01.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_HOST": { 2021-03-10T13:43:01.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_HOST", 2021-03-10T13:43:01.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_PORT": { 2021-03-10T13:43:01.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_PORT", 2021-03-10T13:43:01.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "80", 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SCHEME": { 2021-03-10T13:43:01.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SCHEME", 2021-03-10T13:43:01.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "http", 2021-03-10T13:43:01.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SECRET_KEY": { 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SECRET_KEY", 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SSL_VERIFY": { 2021-03-10T13:43:01.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SSL_VERIFY", 2021-03-10T13:43:01.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_USER_ID": { 2021-03-10T13:43:01.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_USER_ID", 2021-03-10T13:43:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_SPAN": { 2021-03-10T13:43:01.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_SPAN", 2021-03-10T13:43:01.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_1": { 2021-03-10T13:43:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_1", 2021-03-10T13:43:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_2": { 2021-03-10T13:43:01.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_2", 2021-03-10T13:43:01.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:01.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crt_file": { 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crt_file", 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug": { 2021-03-10T13:43:01.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "debug", 2021-03-10T13:43:01.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Enable/disable debug options", 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "jwt_token_ttl": { 2021-03-10T13:43:01.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "jwt_token_ttl", 2021-03-10T13:43:01.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "28800", 2021-03-10T13:43:01.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2021-03-10T13:43:01.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2021-03-10T13:43:01.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "::", 2021-03-10T13:43:01.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:01.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:01.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8080", 2021-03-10T13:43:01.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2021-03-10T13:43:01.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl_server_port": { 2021-03-10T13:43:01.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl_server_port", 2021-03-10T13:43:01.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8443", 2021-03-10T13:43:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_behaviour": { 2021-03-10T13:43:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_behaviour", 2021-03-10T13:43:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "redirect", 2021-03-10T13:43:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "redirect" 2021-03-10T13:43:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_error_status_code": { 2021-03-10T13:43:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_error_status_code", 2021-03-10T13:43:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "500", 2021-03-10T13:43:01.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "400", 2021-03-10T13:43:01.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "599", 2021-03-10T13:43:01.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url_prefix": { 2021-03-10T13:43:01.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url_prefix", 2021-03-10T13:43:01.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "devicehealth", 2021-03-10T13:43:01.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_monitoring": { 2021-03-10T13:43:01.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_monitoring", 2021-03-10T13:43:01.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "monitor device health metrics", 2021-03-10T13:43:01.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mark_out_threshold": { 2021-03-10T13:43:01.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mark_out_threshold", 2021-03-10T13:43:01.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2419200", 2021-03-10T13:43:01.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically mark OSD if it may fail before this long", 2021-03-10T13:43:01.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_name": { 2021-03-10T13:43:01.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_name", 2021-03-10T13:43:01.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "device_health_metrics", 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name of pool in which to store device health metrics", 2021-03-10T13:43:01.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retention_period": { 2021-03-10T13:43:01.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retention_period", 2021-03-10T13:43:01.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15552000", 2021-03-10T13:43:01.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain device health metrics", 2021-03-10T13:43:01.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_frequency": { 2021-03-10T13:43:01.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_frequency", 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to scrape device health metrics", 2021-03-10T13:43:01.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "self_heal": { 2021-03-10T13:43:01.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "self_heal", 2021-03-10T13:43:01.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "preemptively heal cluster around devices that may fail", 2021-03-10T13:43:01.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:01.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and check device health", 2021-03-10T13:43:01.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_threshold": { 2021-03-10T13:43:01.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_threshold", 2021-03-10T13:43:01.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7257600", 2021-03-10T13:43:01.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise health warning if OSD may fail before this long", 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "diskprediction_local", 2021-03-10T13:43:01.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predict_interval": { 2021-03-10T13:43:01.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predict_interval", 2021-03-10T13:43:01.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2021-03-10T13:43:01.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predictor_model": { 2021-03-10T13:43:01.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predictor_model", 2021-03-10T13:43:01.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "prophetstor", 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:01.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:01.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:01.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "influx", 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": false, 2021-03-10T13:43:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "influxdb python module not found", 2021-03-10T13:43:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "batch_size": { 2021-03-10T13:43:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "batch_size", 2021-03-10T13:43:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5000", 2021-03-10T13:43:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "database": { 2021-03-10T13:43:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "database", 2021-03-10T13:43:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph", 2021-03-10T13:43:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "hostname": { 2021-03-10T13:43:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "hostname", 2021-03-10T13:43:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "30", 2021-03-10T13:43:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "port": { 2021-03-10T13:43:01.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "port", 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8086", 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2021-03-10T13:43:01.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2021-03-10T13:43:01.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "false", 2021-03-10T13:43:01.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "threads": { 2021-03-10T13:43:01.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "threads", 2021-03-10T13:43:01.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:01.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2021-03-10T13:43:01.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2021-03-10T13:43:01.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "verify_ssl": { 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "verify_ssl", 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "true", 2021-03-10T13:43:01.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "insights", 2021-03-10T13:43:01.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "iostat", 2021-03-10T13:43:01.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "k8sevents", 2021-03-10T13:43:01.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ceph_event_retention_days": { 2021-03-10T13:43:01.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ceph_event_retention_days", 2021-03-10T13:43:01.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2021-03-10T13:43:01.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Days to hold ceph event information within local cache", 2021-03-10T13:43:01.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_check_secs": { 2021-03-10T13:43:01.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_check_secs", 2021-03-10T13:43:01.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:01.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "10", 2021-03-10T13:43:01.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "interval (secs) to check for cluster configuration changes", 2021-03-10T13:43:01.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "localpool", 2021-03-10T13:43:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "failure_domain": { 2021-03-10T13:43:01.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "failure_domain", 2021-03-10T13:43:01.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "host", 2021-03-10T13:43:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "failure domain for any created local pool", 2021-03-10T13:43:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_size": { 2021-03-10T13:43:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_size", 2021-03-10T13:43:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default min_size for any created local pool", 2021-03-10T13:43:01.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_rep": { 2021-03-10T13:43:01.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "num_rep", 2021-03-10T13:43:01.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "3", 2021-03-10T13:43:01.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default replica count for any created local pool", 2021-03-10T13:43:01.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_num": { 2021-03-10T13:43:01.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_num", 2021-03-10T13:43:01.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "128", 2021-03-10T13:43:01.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default pg_num for any created local pool", 2021-03-10T13:43:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prefix": { 2021-03-10T13:43:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prefix", 2021-03-10T13:43:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name prefix for any created local pool", 2021-03-10T13:43:01.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "subtree": { 2021-03-10T13:43:01.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "subtree", 2021-03-10T13:43:01.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "rack", 2021-03-10T13:43:01.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "CRUSH level for which to create a local pool", 2021-03-10T13:43:01.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2021-03-10T13:43:01.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator": { 2021-03-10T13:43:01.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2021-03-10T13:43:01.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2021-03-10T13:43:01.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rook", 2021-03-10T13:43:01.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "test_orchestrator" 2021-03-10T13:43:01.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Orchestrator backend", 2021-03-10T13:43:01.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "osd_support", 2021-03-10T13:43:01.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_autoscaler", 2021-03-10T13:43:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "progress", 2021-03-10T13:43:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max_completed_events": { 2021-03-10T13:43:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "max_completed_events", 2021-03-10T13:43:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "50", 2021-03-10T13:43:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "number of past completed events to remember", 2021-03-10T13:43:01.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "persist_interval": { 2021-03-10T13:43:01.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "persist_interval", 2021-03-10T13:43:01.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:01.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to persist completed events", 2021-03-10T13:43:01.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus", 2021-03-10T13:43:01.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools": { 2021-03-10T13:43:01.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools", 2021-03-10T13:43:01.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.451 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.451 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.451 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.451 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.451 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools_refresh_interval": { 2021-03-10T13:43:01.452 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools_refresh_interval", 2021-03-10T13:43:01.452 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.452 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.452 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.452 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_interval": { 2021-03-10T13:43:01.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_interval", 2021-03-10T13:43:01.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:01.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:01.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:01.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:01.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_support", 2021-03-10T13:43:01.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mirror_snapshot_schedule": { 2021-03-10T13:43:01.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mirror_snapshot_schedule", 2021-03-10T13:43:01.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "trash_purge_schedule": { 2021-03-10T13:43:01.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "trash_purge_schedule", 2021-03-10T13:43:01.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "restful", 2021-03-10T13:43:01.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2021-03-10T13:43:01.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2021-03-10T13:43:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:01.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:01.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:01.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:01.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rook", 2021-03-10T13:43:01.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "selftest", 2021-03-10T13:43:01.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption1": { 2021-03-10T13:43:01.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption1", 2021-03-10T13:43:01.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption2": { 2021-03-10T13:43:01.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption2", 2021-03-10T13:43:01.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "xyz", 2021-03-10T13:43:01.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption1": { 2021-03-10T13:43:01.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption1", 2021-03-10T13:43:01.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption2": { 2021-03-10T13:43:01.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption2", 2021-03-10T13:43:01.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption3": { 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption3", 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:01.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption4": { 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption4", 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption5": { 2021-03-10T13:43:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption5", 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption6": { 2021-03-10T13:43:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption6", 2021-03-10T13:43:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testkey": { 2021-03-10T13:43:01.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testkey", 2021-03-10T13:43:01.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testlkey": { 2021-03-10T13:43:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testlkey", 2021-03-10T13:43:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testnewline": { 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testnewline", 2021-03-10T13:43:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "status", 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telegraf", 2021-03-10T13:43:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "address": { 2021-03-10T13:43:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "address", 2021-03-10T13:43:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "unixgram:///tmp/telegraf.sock", 2021-03-10T13:43:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:01.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:01.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15", 2021-03-10T13:43:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telemetry", 2021-03-10T13:43:01.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_basic": { 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_basic", 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Share basic cluster information (size, version)", 2021-03-10T13:43:01.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_crash": { 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_crash", 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_device": { 2021-03-10T13:43:01.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_device", 2021-03-10T13:43:01.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:01.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_ident": { 2021-03-10T13:43:01.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_ident", 2021-03-10T13:43:01.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "contact": { 2021-03-10T13:43:01.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "contact", 2021-03-10T13:43:01.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "description": { 2021-03-10T13:43:01.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "description", 2021-03-10T13:43:01.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_url": { 2021-03-10T13:43:01.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_url", 2021-03-10T13:43:01.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/device", 2021-03-10T13:43:01.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enabled": { 2021-03-10T13:43:01.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enabled", 2021-03-10T13:43:01.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:01.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:01.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "24", 2021-03-10T13:43:01.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "8", 2021-03-10T13:43:01.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_opt_revision": { 2021-03-10T13:43:01.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "last_opt_revision", 2021-03-10T13:43:01.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1", 2021-03-10T13:43:01.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "leaderboard": { 2021-03-10T13:43:01.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "leaderboard", 2021-03-10T13:43:01.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "organization": { 2021-03-10T13:43:01.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "organization", 2021-03-10T13:43:01.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "proxy": { 2021-03-10T13:43:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "proxy", 2021-03-10T13:43:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url": { 2021-03-10T13:43:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url", 2021-03-10T13:43:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/report", 2021-03-10T13:43:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "test_orchestrator", 2021-03-10T13:43:01.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.676 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.676 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.676 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.676 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.676 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.677 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.677 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.677 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.677 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.677 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "volumes", 2021-03-10T13:43:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.684 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.684 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.684 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.684 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.685 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.685 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.685 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.685 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.685 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.686 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.686 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.686 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.686 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.686 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.687 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.687 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.687 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.687 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.687 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.688 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.689 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.689 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.689 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.689 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.689 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.690 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.690 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.690 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.690 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.691 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.692 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.692 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.692 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.692 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.693 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.693 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.693 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.693 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.693 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.694 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix", 2021-03-10T13:43:01.694 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:01.694 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:01.694 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:01.694 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "discovery_interval": { 2021-03-10T13:43:01.695 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "discovery_interval", 2021-03-10T13:43:01.695 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.695 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.695 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.695 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "100", 2021-03-10T13:43:01.696 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.696 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.696 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.696 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.696 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.697 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.697 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.697 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.697 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "identifier": { 2021-03-10T13:43:01.697 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "identifier", 2021-03-10T13:43:01.698 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.698 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.698 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.698 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.698 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.699 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.699 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.699 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.699 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.699 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.700 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.700 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.700 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:01.700 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:01.700 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:01.701 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.701 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.701 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:01.701 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.701 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.702 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.702 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.702 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.702 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.702 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.703 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.703 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:01.703 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:01.703 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.703 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.704 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.704 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:01.704 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.704 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.704 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.705 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.705 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.705 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.705 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.705 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.706 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.706 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.706 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.706 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.706 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.707 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.707 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.707 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:01.707 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:01.707 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.708 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.708 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.708 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.708 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.709 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.709 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.709 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.709 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.709 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.710 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.710 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.710 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:01.710 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:01.710 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.711 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.711 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.711 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:01.711 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.711 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.712 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:01.712 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:01.712 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:01.712 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:01.712 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:01.713 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:01.713 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:01.713 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.713 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.713 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.714 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.714 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.714 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.714 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:01.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:01.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:01.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:01.715 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.716 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_host": { 2021-03-10T13:43:01.717 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_host", 2021-03-10T13:43:01.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:01.718 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.719 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.719 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.719 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.719 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.719 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.720 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.720 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.720 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_port": { 2021-03-10T13:43:01.720 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_port", 2021-03-10T13:43:01.720 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10051", 2021-03-10T13:43:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_sender": { 2021-03-10T13:43:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_sender", 2021-03-10T13:43:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/usr/bin/zabbix_sender", 2021-03-10T13:43:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {}, 2021-03-10T13:43:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "always_on_modules": { 2021-03-10T13:43:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nautilus": [ 2021-03-10T13:43:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2021-03-10T13:43:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2021-03-10T13:43:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2021-03-10T13:43:01.728 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator_cli", 2021-03-10T13:43:01.728 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2021-03-10T13:43:01.728 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2021-03-10T13:43:01.728 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2021-03-10T13:43:01.729 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2021-03-10T13:43:01.729 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.729 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "octopus": [ 2021-03-10T13:43:01.729 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2021-03-10T13:43:01.729 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2021-03-10T13:43:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2021-03-10T13:43:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2021-03-10T13:43:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_support", 2021-03-10T13:43:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2021-03-10T13:43:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2021-03-10T13:43:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2021-03-10T13:43:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2021-03-10T13:43:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2021-03-10T13:43:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2021-03-10T13:43:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_failure_osd_epoch": 0, 2021-03-10T13:43:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_clients": [ 2021-03-10T13:43:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3059131095 2021-03-10T13:43:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 363049641 2021-03-10T13:43:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:01.737 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:01.737 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 308369833 2021-03-10T13:43:01.737 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.737 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:01.737 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.738 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:01.738 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.738 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:01.739 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for the mgr to restart... 2021-03-10T13:43:01.739 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for Mgr epoch 5... 2021-03-10T13:43:01.740 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'tell', 'mgr', 'mgr_status'] 2021-03-10T13:43:01.740 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 tell mgr mgr_status 2021-03-10T13:43:01.740 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:01 gibba011 bash[9813]: audit 2021-03-10T13:43:00.443885+0000 mon.a (mon.0) 36 : audit [DBG] from='client.? 172.21.2.111:0/3448635594' entity='client.admin' cmd=[{"prefix": "mgr dump"}]: dispatch 2021-03-10T13:43:02.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:02 gibba011 bash[9813]: cluster 2021-03-10T13:43:01.684785+0000 mon.a (mon.0) 37 : cluster [INF] Active manager daemon y restarted 2021-03-10T13:43:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:02 gibba011 bash[9813]: cluster 2021-03-10T13:43:01.685372+0000 mon.a (mon.0) 38 : cluster [INF] Activating manager daemon y 2021-03-10T13:43:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:02 gibba011 bash[9813]: cluster 2021-03-10T13:43:01.687045+0000 mon.a (mon.0) 39 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2021-03-10T13:43:03.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:03.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap_epoch": 7, 2021-03-10T13:43:03.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "initialized": true 2021-03-10T13:43:03.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: cluster 2021-03-10T13:43:02.041833+0000 mon.a (mon.0) 40 : cluster [DBG] mgrmap e6: y(active, starting, since 0.356501s) 2021-03-10T13:43:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.045943+0000 mon.a (mon.0) 41 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:43:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.048252+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:43:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.048622+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:43:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.048957+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:43:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.050214+0000 mon.a (mon.0) 45 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:43:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.052131+0000 mon.a (mon.0) 46 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:43:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.053303+0000 mon.a (mon.0) 47 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:43:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: cluster 2021-03-10T13:43:02.059012+0000 mon.a (mon.0) 48 : cluster [INF] Manager daemon y is now available 2021-03-10T13:43:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.079958+0000 mon.a (mon.0) 49 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-03-10T13:43:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:03 gibba011 bash[9813]: audit 2021-03-10T13:43:02.235089+0000 mon.a (mon.0) 50 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-03-10T13:43:03.556 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Setting orchestrator backend to cephadm... 2021-03-10T13:43:03.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'orch', 'set', 'backend', 'cephadm'] 2021-03-10T13:43:03.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 orch set backend cephadm 2021-03-10T13:43:04.310 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:04 gibba011 bash[9813]: cluster 2021-03-10T13:43:03.047295+0000 mon.a (mon.0) 51 : cluster [DBG] mgrmap e7: y(active, since 1.36198s) 2021-03-10T13:43:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:05 gibba011 bash[9813]: cluster 2021-03-10T13:43:04.061468+0000 mon.a (mon.0) 52 : cluster [DBG] mgrmap e8: y(active, since 2s) 2021-03-10T13:43:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:05 gibba011 bash[9813]: audit 2021-03-10T13:43:04.630121+0000 mgr.y (mgr.14117) 1 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:05 gibba011 bash[9813]: audit 2021-03-10T13:43:04.630944+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config set","value":"cephadm","who":"mgr","name":"mgr/orchestrator/orchestrator"}]: dispatch 2021-03-10T13:43:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:05 gibba011 bash[9813]: audit 2021-03-10T13:43:04.633048+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config set","value":"cephadm","who":"mgr","name":"mgr/orchestrator/orchestrator"}]': finished 2021-03-10T13:43:05.205 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Generating ssh key... 2021-03-10T13:43:05.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'cephadm', 'generate-key'] 2021-03-10T13:43:05.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 cephadm generate-key 2021-03-10T13:43:06.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[9813]: audit 2021-03-10T13:43:06.461807+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_key","val":"-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn\nNhAAAAAwEAAQAAAYEAuPXAJ8YGdSz3m28dsvPNfgv6AT9Ij8Gog963H8b3mjMUObWEXRir\nSttRgYdDwmn6FU3jZiGEHdEe9scAd6Tj1B37ppNW3lk8hrOI6C68A3fSF29JLZJEhCPev1\nPRhKiT6nQuYmVk+PD4a6tC/o/2BH7EeOOrU64sbYXiJHYmBoP1UCJco9hSy9vhV2WabQOG\ncnm4Bg8U7GI3dNG2OXYrlpZT3iAVk37cFHyUqnInqj0wqxRi7huhrxgqPNpUPs2P2Apean\ncVg7RKiwxkdjpXMqVl2sj6c/Hl1JWKGhvj7c0DVxPGHPhZYUqrbS3Ge7tGir0nobSVHa7P\nDSELnwWwJmN2s3RXX3Z0x5bW1BYu7VR2EyV2NxRYrL5ZywCAz+aXBQKm4kXxSMeUpI4JsN\n/boaVHOc1fjfQJQVVaLdv/DyDX/Fu5VVXpunMOKE8aR2yrt3L6i5UAQt7mRf/+q9kgi/Gc\n8WPtSHDLwrdqL3zE4RvcYOqt/k4PeSnD4V04iTsdAAAFoJxuDyGcbg8hAAAAB3NzaC1yc2\nEAAAGBALj1wCfGBnUs95tvHbLzzX4L+gE/SI/BqIPetx/G95ozFDm1hF0Yq0rbUYGHQ8Jp\n+hVN42YhhB3RHvbHAHek49Qd+6aTVt5ZPIaziOguvAN30hdvSS2SRIQj3r9T0YSok+p0Lm\nJlZPjw+GurQv6P9gR+xHjjq1OuLG2F4iR2JgaD9VAiXKPYUsvb4Vdlmm0DhnJ5uAYPFOxi\nN3TRtjl2K5aWU94gFZN+3BR8lKpyJ6o9MKsUYu4boa8YKjzaVD7Nj9gKXmp3FYO0SosMZH\nY6VzKlZdrI+nPx5dSVihob4+3NA1cTxhz4WWFKq20txnu7Roq9J6G0lR2uzw0hC58FsCZj\ndrN0V192dMeW1tQWLu1UdhMldjcUWKy+WcsAgM/mlwUCpuJF8UjHlKSOCbDf26GlRznNX4\n30CUFVWi3b/w8g1/xbuVVV6bpzDihPGkdsq7dy+ouVAELe5kX//qvZIIvxnPFj7Uhwy8K3\nai98xOEb3GDqrf5OD3kpw+FdOIk7HQAAAAMBAAEAAAGAG0Yhz7ujuCOpX4JDHuBUrQGXHB\nf42znzX364AjeUy2g800Ww/wlkOOWeXztTApgeCIYzwhuvH7Z2mm095GasMds6AJY5eRAv\nnK1cwuAhWWGR2WLPp6ip8dJYRl5VxvoU1b3MGTZabVsbRsU6W89et8r5rdnkqbJFWuEjN7\n00XkGXKvnmL0L28pi54elZnDmo2biZKkuPCkxxYQiLlm2Wf94q/9Y7WEScr2ivg3LR5oif\nZJrc+7+XV/p4QADx61DkZP5Wi+bjM8YKp8NEmgcUPFFqPCjoIWoTjHLEoSA+7tKlcWO/X2\ndgwO3mqtnvg8hgnor9ZZhgwUrPAqHGan8nzR6rrykcrqveKrYXmsHVtgIrQYxQHlWYpYrP\nMKZT/V1Iknskq+ZsM00BEQ6PJIjMRaq+bYsJEmeFQodu+WOErqucLsoHtINZ30AAQUKY8k\n+ZopZyoFwXSQ4G/h/aU0eZXm8dZEaOZ3yOeSrtYa6rUTJ7Tyo/iDHeF2IVaNEU+KABAAAA\nwGfk7IqlT7y4fIaOv0hScGLtP6yehG1UIpHnJjFNL2oH+Wp28agc6udhV7rGFNu5QsRfq7\nN5VIVXjy/ACW6UfLJkPYqV3DkJOmmbys2TPgLMbWAog85ec+Lp5KvzeWyf9WWT7EmSI0cz\nFC4mKbk9pU3/QdotncBoK7oPrUUDG9gq/u0CyIVF1rK95UKAe2efEkLfmqdqRk2f91W0Wr\nhnJBMbs1Md1VWgGssCt/WHTmGbx+R7AIDNon6Kh7EJzuaf7AAAAMEA4hOeZocxY/FCnrEu\nhPNQa3HY0VTpVUV7Oi8hn6TPVdGsPNVk+gjzrP4nINxLU7IGx4TTDnO9BbxK+D0KwN38Dp\nQ5a5wzw+lwAez5+MiY0gdhUbv4n0AkP62ndr/wdRPNtVQ0ujMBD7tUvR8JwpUTke0b+UL8\nCobZ7ybqXQhvIYKE5Pu7v3MOz50IbrInHyIQbuTo38kwBsqiEbEePFfk954UARYJFpheXz\nGBgY6cERdBtfi7OmpqYWFgAj3FkyChAAAAwQDRcO823bCOkDYuG06YzyAeXOnsIAhenXNU\nuQdQzLHdBqvDiWENkHPlE4VWeXisfUPRUrdQjqmjgjCnGFHrkgbXNtjjRFQkphvousFI0+\n57TeUv1YaisolSsg3/wvvpn8SnvfBGS/E8nlvtzRZP/TJ8dfD3Y6TaTFiXWuZ4LXe2Fmhu\nhYpW3WNTQ2LvygfLqu0G235q+4FDkUxb98P0ijjgZJlg8P/L1U1y9MgiKtkiTSxs6scLez\namMFDvDo3UfP0AAAApY2VwaC02ZmQzNmQ5NC04MWE2LTExZWItOTA4MC0wMDFhNGFhYjgz\nMGMBAg==\n-----END OPENSSH PRIVATE KEY-----\n"}]: dispatch 2021-03-10T13:43:06.829 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: Generating public/private rsa key pair. 2021-03-10T13:43:06.829 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: Your identification has been saved in /tmp/tmphv573svq/key. 2021-03-10T13:43:06.829 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: Your public key has been saved in /tmp/tmphv573svq/key.pub. 2021-03-10T13:43:06.830 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: The key fingerprint is: 2021-03-10T13:43:06.830 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: SHA256:TaRBflkidNpNXP6U0/x961l0WE3nKLIX+NyCs4vQXn4 ceph-6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:43:06.830 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: The key's randomart image is: 2021-03-10T13:43:06.830 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: +---[RSA 3072]----+ 2021-03-10T13:43:06.830 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | o= +.o.. o| 2021-03-10T13:43:06.831 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | . O B.. ==| 2021-03-10T13:43:06.831 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | + B + ooB| 2021-03-10T13:43:06.831 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | + * + =+| 2021-03-10T13:43:06.831 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | S = = o B| 2021-03-10T13:43:06.831 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | . + . .+| 2021-03-10T13:43:06.832 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | . . o ..| 2021-03-10T13:43:06.832 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | o + .E . o| 2021-03-10T13:43:06.832 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: | o o. o | 2021-03-10T13:43:06.832 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[10046]: +----[SHA256]-----+ 2021-03-10T13:43:06.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[9813]: audit 2021-03-10T13:43:06.463897+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_key","val":"-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn\nNhAAAAAwEAAQAAAYEAuPXAJ8YGdSz3m28dsvPNfgv6AT9Ij8Gog963H8b3mjMUObWEXRir\nSttRgYdDwmn6FU3jZiGEHdEe9scAd6Tj1B37ppNW3lk8hrOI6C68A3fSF29JLZJEhCPev1\nPRhKiT6nQuYmVk+PD4a6tC/o/2BH7EeOOrU64sbYXiJHYmBoP1UCJco9hSy9vhV2WabQOG\ncnm4Bg8U7GI3dNG2OXYrlpZT3iAVk37cFHyUqnInqj0wqxRi7huhrxgqPNpUPs2P2Apean\ncVg7RKiwxkdjpXMqVl2sj6c/Hl1JWKGhvj7c0DVxPGHPhZYUqrbS3Ge7tGir0nobSVHa7P\nDSELnwWwJmN2s3RXX3Z0x5bW1BYu7VR2EyV2NxRYrL5ZywCAz+aXBQKm4kXxSMeUpI4JsN\n/boaVHOc1fjfQJQVVaLdv/DyDX/Fu5VVXpunMOKE8aR2yrt3L6i5UAQt7mRf/+q9kgi/Gc\n8WPtSHDLwrdqL3zE4RvcYOqt/k4PeSnD4V04iTsdAAAFoJxuDyGcbg8hAAAAB3NzaC1yc2\nEAAAGBALj1wCfGBnUs95tvHbLzzX4L+gE/SI/BqIPetx/G95ozFDm1hF0Yq0rbUYGHQ8Jp\n+hVN42YhhB3RHvbHAHek49Qd+6aTVt5ZPIaziOguvAN30hdvSS2SRIQj3r9T0YSok+p0Lm\nJlZPjw+GurQv6P9gR+xHjjq1OuLG2F4iR2JgaD9VAiXKPYUsvb4Vdlmm0DhnJ5uAYPFOxi\nN3TRtjl2K5aWU94gFZN+3BR8lKpyJ6o9MKsUYu4boa8YKjzaVD7Nj9gKXmp3FYO0SosMZH\nY6VzKlZdrI+nPx5dSVihob4+3NA1cTxhz4WWFKq20txnu7Roq9J6G0lR2uzw0hC58FsCZj\ndrN0V192dMeW1tQWLu1UdhMldjcUWKy+WcsAgM/mlwUCpuJF8UjHlKSOCbDf26GlRznNX4\n30CUFVWi3b/w8g1/xbuVVV6bpzDihPGkdsq7dy+ouVAELe5kX//qvZIIvxnPFj7Uhwy8K3\nai98xOEb3GDqrf5OD3kpw+FdOIk7HQAAAAMBAAEAAAGAG0Yhz7ujuCOpX4JDHuBUrQGXHB\nf42znzX364AjeUy2g800Ww/wlkOOWeXztTApgeCIYzwhuvH7Z2mm095GasMds6AJY5eRAv\nnK1cwuAhWWGR2WLPp6ip8dJYRl5VxvoU1b3MGTZabVsbRsU6W89et8r5rdnkqbJFWuEjN7\n00XkGXKvnmL0L28pi54elZnDmo2biZKkuPCkxxYQiLlm2Wf94q/9Y7WEScr2ivg3LR5oif\nZJrc+7+XV/p4QADx61DkZP5Wi+bjM8YKp8NEmgcUPFFqPCjoIWoTjHLEoSA+7tKlcWO/X2\ndgwO3mqtnvg8hgnor9ZZhgwUrPAqHGan8nzR6rrykcrqveKrYXmsHVtgIrQYxQHlWYpYrP\nMKZT/V1Iknskq+ZsM00BEQ6PJIjMRaq+bYsJEmeFQodu+WOErqucLsoHtINZ30AAQUKY8k\n+ZopZyoFwXSQ4G/h/aU0eZXm8dZEaOZ3yOeSrtYa6rUTJ7Tyo/iDHeF2IVaNEU+KABAAAA\nwGfk7IqlT7y4fIaOv0hScGLtP6yehG1UIpHnJjFNL2oH+Wp28agc6udhV7rGFNu5QsRfq7\nN5VIVXjy/ACW6UfLJkPYqV3DkJOmmbys2TPgLMbWAog85ec+Lp5KvzeWyf9WWT7EmSI0cz\nFC4mKbk9pU3/QdotncBoK7oPrUUDG9gq/u0CyIVF1rK95UKAe2efEkLfmqdqRk2f91W0Wr\nhnJBMbs1Md1VWgGssCt/WHTmGbx+R7AIDNon6Kh7EJzuaf7AAAAMEA4hOeZocxY/FCnrEu\nhPNQa3HY0VTpVUV7Oi8hn6TPVdGsPNVk+gjzrP4nINxLU7IGx4TTDnO9BbxK+D0KwN38Dp\nQ5a5wzw+lwAez5+MiY0gdhUbv4n0AkP62ndr/wdRPNtVQ0ujMBD7tUvR8JwpUTke0b+UL8\nCobZ7ybqXQhvIYKE5Pu7v3MOz50IbrInHyIQbuTo38kwBsqiEbEePFfk954UARYJFpheXz\nGBgY6cERdBtfi7OmpqYWFgAj3FkyChAAAAwQDRcO823bCOkDYuG06YzyAeXOnsIAhenXNU\nuQdQzLHdBqvDiWENkHPlE4VWeXisfUPRUrdQjqmjgjCnGFHrkgbXNtjjRFQkphvousFI0+\n57TeUv1YaisolSsg3/wvvpn8SnvfBGS/E8nlvtzRZP/TJ8dfD3Y6TaTFiXWuZ4LXe2Fmhu\nhYpW3WNTQ2LvygfLqu0G235q+4FDkUxb98P0ijjgZJlg8P/L1U1y9MgiKtkiTSxs6scLez\namMFDvDo3UfP0AAAApY2VwaC02ZmQzNmQ5NC04MWE2LTExZWItOTA4MC0wMDFhNGFhYjgz\nMGMBAg==\n-----END OPENSSH PRIVATE KEY-----\n"}]': finished 2021-03-10T13:43:06.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[9813]: audit 2021-03-10T13:43:06.464408+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_pub","val":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c\n"}]: dispatch 2021-03-10T13:43:06.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:06 gibba011 bash[9813]: audit 2021-03-10T13:43:06.465864+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_pub","val":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c\n"}]': finished 2021-03-10T13:43:06.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'cephadm', 'get-pub-key'] 2021-03-10T13:43:06.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 cephadm get-pub-key 2021-03-10T13:43:07.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:07 gibba011 bash[9813]: audit 2021-03-10T13:43:06.220532+0000 mgr.y (mgr.14117) 2 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:07.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:07 gibba011 bash[9813]: cephadm 2021-03-10T13:43:06.220704+0000 mgr.y (mgr.14117) 3 : cephadm [INF] Generating ssh key... 2021-03-10T13:43:07.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:43:08.551 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Wrote public SSH key to to /home/ubuntu/cephtest/ceph.pub 2021-03-10T13:43:08.551 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Adding key to root@localhost's authorized_keys... 2021-03-10T13:43:08.552 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Adding host gibba011... 2021-03-10T13:43:08.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'orch', 'host', 'add', 'gibba011'] 2021-03-10T13:43:08.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 orch host add gibba011 2021-03-10T13:43:09.281 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:09 gibba011 bash[9813]: audit 2021-03-10T13:43:07.986407+0000 mgr.y (mgr.14117) 4 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:09.827 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:09 gibba011 bash[10046]: Warning: Permanently added 'gibba011,172.21.2.111' (ECDSA) to the list of known hosts. 2021-03-10T13:43:10.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.530807+0000 mgr.y (mgr.14117) 5 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "gibba011", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:10.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.861436+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"gibba011\": {\"hostname\": \"gibba011\", \"addr\": \"gibba011\", \"labels\": [], \"status\": \"\"}}"}]: dispatch 2021-03-10T13:43:10.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.863267+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"gibba011\": {\"hostname\": \"gibba011\", \"addr\": \"gibba011\", \"labels\": [], \"status\": \"\"}}"}]': finished 2021-03-10T13:43:10.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: cephadm 2021-03-10T13:43:09.863711+0000 mgr.y (mgr.14117) 6 : cephadm [INF] Added host gibba011 2021-03-10T13:43:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.976002+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.977666+0000 mon.a (mon.0) 62 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:43:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:10 gibba011 bash[9813]: audit 2021-03-10T13:43:09.978173+0000 mon.a (mon.0) 63 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:10.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout Added host 'gibba011' 2021-03-10T13:43:11.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:11 gibba011 bash[9813]: audit 2021-03-10T13:43:10.286250+0000 mon.a (mon.0) 64 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:10.284793\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:10.285802\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:10.285866\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:11 gibba011 bash[9813]: audit 2021-03-10T13:43:10.287900+0000 mon.a (mon.0) 65 : audit [INF] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:10.284793\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:10.285802\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:10.285866\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:43:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:11 gibba011 bash[9813]: audit 2021-03-10T13:43:10.288436+0000 mon.a (mon.0) 66 : audit [DBG] from='mgr.14117 172.21.2.111:0/4078150364' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:43:11.559 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Enabling the dashboard module... 2021-03-10T13:43:11.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'mgr', 'module', 'enable', 'dashboard'] 2021-03-10T13:43:11.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 mgr module enable dashboard 2021-03-10T13:43:13.710 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:13 gibba011 bash[9813]: audit 2021-03-10T13:43:13.365954+0000 mon.a (mon.0) 67 : audit [INF] from='client.? 172.21.2.111:0/2446304869' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2021-03-10T13:43:13.710 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:13 gibba011 bash[10046]: ignoring --setuser ceph since I am not root 2021-03-10T13:43:13.711 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:13 gibba011 bash[10046]: ignoring --setgroup ceph since I am not root 2021-03-10T13:43:14.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'mgr', 'dump'] 2021-03-10T13:43:14.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 mgr dump 2021-03-10T13:43:14.539 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:14 gibba011 bash[9813]: audit 2021-03-10T13:43:13.418740+0000 mon.a (mon.0) 68 : audit [INF] from='client.? 172.21.2.111:0/2446304869' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2021-03-10T13:43:14.539 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:14 gibba011 bash[9813]: cluster 2021-03-10T13:43:13.418770+0000 mon.a (mon.0) 69 : cluster [DBG] mgrmap e9: y(active, since 11s) 2021-03-10T13:43:15.861 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 9, 2021-03-10T13:43:15.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_gid": 14117, 2021-03-10T13:43:15.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_name": "y", 2021-03-10T13:43:15.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addrs": { 2021-03-10T13:43:15.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:15.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:15.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:6800", 2021-03-10T13:43:15.863 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 767229161 2021-03-10T13:43:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v1", 2021-03-10T13:43:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:6801", 2021-03-10T13:43:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 767229161 2021-03-10T13:43:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addr": "172.21.2.111:6801/767229161", 2021-03-10T13:43:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_change": "2021-03-10T13:43:01.685310+0000", 2021-03-10T13:43:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_mgr_features": 4540138292836696063, 2021-03-10T13:43:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2021-03-10T13:43:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standbys": [], 2021-03-10T13:43:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2021-03-10T13:43:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2021-03-10T13:43:15.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "dashboard", 2021-03-10T13:43:15.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2021-03-10T13:43:15.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2021-03-10T13:43:15.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.867 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available_modules": [ 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "alerts", 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:15.868 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:15.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:15.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:15.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.869 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:15.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "How frequently to reexamine health status", 2021-03-10T13:43:15.870 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:15.871 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:15.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.872 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.873 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.874 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.875 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.876 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.877 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:15.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:15.878 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:15.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.879 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.880 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.882 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.883 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.884 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_destination": { 2021-03-10T13:43:15.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_destination", 2021-03-10T13:43:15.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.885 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email address to send alerts to", 2021-03-10T13:43:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_from_name": { 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_from_name", 2021-03-10T13:43:15.887 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "Ceph", 2021-03-10T13:43:15.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.888 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email From: name", 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_host": { 2021-03-10T13:43:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_host", 2021-03-10T13:43:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP server", 2021-03-10T13:43:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_password": { 2021-03-10T13:43:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_password", 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.893 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Password to authenticate with", 2021-03-10T13:43:15.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.894 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_port": { 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_port", 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.895 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "465", 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP port", 2021-03-10T13:43:15.896 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_sender": { 2021-03-10T13:43:15.897 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_sender", 2021-03-10T13:43:15.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.898 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP envelope sender", 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.899 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_ssl": { 2021-03-10T13:43:15.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_ssl", 2021-03-10T13:43:15.900 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.901 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Use SSL to connect to SMTP server", 2021-03-10T13:43:15.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.902 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_user": { 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_user", 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.903 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "User to authenticate as", 2021-03-10T13:43:15.904 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.905 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "balancer", 2021-03-10T13:43:15.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:15.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:15.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:15.906 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active": { 2021-03-10T13:43:15.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "active", 2021-03-10T13:43:15.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.907 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically balance PGs across cluster", 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.908 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_time": { 2021-03-10T13:43:15.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_time", 2021-03-10T13:43:15.909 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0000", 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.910 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "beginning time of day to automatically balance", 2021-03-10T13:43:15.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2021-03-10T13:43:15.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.911 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_weekday": { 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_weekday", 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.912 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:15.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2021-03-10T13:43:15.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2021-03-10T13:43:15.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to this day of the week or later", 2021-03-10T13:43:15.913 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_max_iterations": { 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_max_iterations", 2021-03-10T13:43:15.914 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:15.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "25", 2021-03-10T13:43:15.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2021-03-10T13:43:15.915 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "250", 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum number of iterations to attempt optimization", 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.916 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_metrics": { 2021-03-10T13:43:15.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_metrics", 2021-03-10T13:43:15.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.917 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "pgs,objects,bytes", 2021-03-10T13:43:15.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.918 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "metrics with which to calculate OSD utilization", 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_step": { 2021-03-10T13:43:15.919 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_step", 2021-03-10T13:43:15.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:15.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0.5", 2021-03-10T13:43:15.920 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0.001", 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "0.999", 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "aggressiveness of optimization", 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": ".99 is very aggressive, .01 is less aggressive", 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.921 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_time": { 2021-03-10T13:43:15.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_time", 2021-03-10T13:43:15.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.922 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2400", 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.923 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "ending time of day to automatically balance", 2021-03-10T13:43:15.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2021-03-10T13:43:15.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.924 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_weekday": { 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_weekday", 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2021-03-10T13:43:15.925 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2021-03-10T13:43:15.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2021-03-10T13:43:15.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to days of the week earlier than this", 2021-03-10T13:43:15.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-03-10T13:43:15.926 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:15.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:15.927 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.928 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.929 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.930 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.932 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.933 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:15.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:15.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.934 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.935 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_score": { 2021-03-10T13:43:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_score", 2021-03-10T13:43:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "minimum score, below which no optimization is attempted", 2021-03-10T13:43:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "none", 2021-03-10T13:43:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush-compat", 2021-03-10T13:43:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "none", 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap" 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Balancer mode", 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_ids": { 2021-03-10T13:43:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_ids", 2021-03-10T13:43:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "pools which the automatic balancing will be limited to", 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and attempt optimization", 2021-03-10T13:43:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_deviation": { 2021-03-10T13:43:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_deviation", 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2021-03-10T13:43:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "deviation below which no optimization is attempted", 2021-03-10T13:43:15.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2021-03-10T13:43:15.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.953 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_optimizations": { 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_optimizations", 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.954 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:15.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.955 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum upmap optimizations to make per attempt", 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.956 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "cephadm", 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "allow_ptrace": { 2021-03-10T13:43:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "allow_ptrace", 2021-03-10T13:43:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "allow SYS_PTRACE capability on ceph containers", 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2021-03-10T13:43:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_base": { 2021-03-10T13:43:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_base", 2021-03-10T13:43:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph/ceph", 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Container image name, without the tag", 2021-03-10T13:43:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "daemon_cache_timeout": { 2021-03-10T13:43:15.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "daemon_cache_timeout", 2021-03-10T13:43:15.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:15.963 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache service (daemon) inventory", 2021-03-10T13:43:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_cache_timeout": { 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_cache_timeout", 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1800", 2021-03-10T13:43:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache device inventory", 2021-03-10T13:43:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "host_check_interval": { 2021-03-10T13:43:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "host_check_interval", 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to perform a host check", 2021-03-10T13:43:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.971 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.972 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.973 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.974 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.975 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:15.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.976 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "log to the \"cephadm\" cluster log channel\"", 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:15.977 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:15.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:15.978 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:15.979 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:15.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:15.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:15.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.980 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:15.981 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:15.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.982 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.983 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "root", 2021-03-10T13:43:15.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:15.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm-package", 2021-03-10T13:43:15.985 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "root" 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "mode for remote execution of cephadm", 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssh_config_file": { 2021-03-10T13:43:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssh_config_file", 2021-03-10T13:43:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "customized SSH config file to connect to managed hosts", 2021-03-10T13:43:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_failed_host_check": { 2021-03-10T13:43:15.989 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_failed_host_check", 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.990 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if the host check fails", 2021-03-10T13:43:15.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.991 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_daemons": { 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_daemons", 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.992 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:15.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.993 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_hosts": { 2021-03-10T13:43:15.994 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_hosts", 2021-03-10T13:43:15.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:15.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:15.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:15.995 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:15.996 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:15.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:15.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:15.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:15.997 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crash", 2021-03-10T13:43:15.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:15.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:15.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:15.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:15.998 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:15.999 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.000 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.001 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.002 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.003 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.004 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.005 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.006 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.007 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.008 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.009 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.010 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.011 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retain_interval": { 2021-03-10T13:43:16.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retain_interval", 2021-03-10T13:43:16.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.012 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "31536000", 2021-03-10T13:43:16.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.013 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain crashes before pruning them", 2021-03-10T13:43:16.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.014 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_recent_interval": { 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_recent_interval", 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1209600", 2021-03-10T13:43:16.015 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "time interval in which to warn about recent crashes", 2021-03-10T13:43:16.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.016 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.017 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "dashboard", 2021-03-10T13:43:16.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.018 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ALERTMANAGER_API_HOST": { 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ALERTMANAGER_API_HOST", 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.019 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.020 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_ENABLED": { 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_ENABLED", 2021-03-10T13:43:16.021 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.022 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.023 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_LOG_PAYLOAD": { 2021-03-10T13:43:16.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_LOG_PAYLOAD", 2021-03-10T13:43:16.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.024 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.025 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ENABLE_BROWSABLE_API": { 2021-03-10T13:43:16.026 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ENABLE_BROWSABLE_API", 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.027 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.028 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_cephfs": { 2021-03-10T13:43:16.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_cephfs", 2021-03-10T13:43:16.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.029 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.030 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.031 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_iscsi": { 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_iscsi", 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.032 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_mirroring": { 2021-03-10T13:43:16.034 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_mirroring", 2021-03-10T13:43:16.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.035 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.036 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_nfs": { 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_nfs", 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.037 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.038 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.039 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rbd": { 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rbd", 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.040 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.041 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rgw": { 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rgw", 2021-03-10T13:43:16.042 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.043 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.044 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2021-03-10T13:43:16.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2021-03-10T13:43:16.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.045 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.046 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_PASSWORD": { 2021-03-10T13:43:16.047 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_PASSWORD", 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.048 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.049 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_SSL_VERIFY": { 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_SSL_VERIFY", 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.050 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.051 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_URL": { 2021-03-10T13:43:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_URL", 2021-03-10T13:43:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_USERNAME": { 2021-03-10T13:43:16.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_USERNAME", 2021-03-10T13:43:16.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.055 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.056 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_UPDATE_DASHBOARDS": { 2021-03-10T13:43:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_UPDATE_DASHBOARDS", 2021-03-10T13:43:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ISCSI_API_SSL_VERIFICATION": { 2021-03-10T13:43:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ISCSI_API_SSL_VERIFICATION", 2021-03-10T13:43:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PROMETHEUS_API_HOST": { 2021-03-10T13:43:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PROMETHEUS_API_HOST", 2021-03-10T13:43:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2021-03-10T13:43:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2021-03-10T13:43:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2021-03-10T13:43:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2021-03-10T13:43:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2021-03-10T13:43:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.073 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2021-03-10T13:43:16.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2021-03-10T13:43:16.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.074 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.075 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2021-03-10T13:43:16.076 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.077 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.078 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2021-03-10T13:43:16.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2021-03-10T13:43:16.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.079 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.080 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2021-03-10T13:43:16.081 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.082 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.083 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_ENABLED": { 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_ENABLED", 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.084 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.085 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.086 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_EXCLUSION_LIST": { 2021-03-10T13:43:16.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_EXCLUSION_LIST", 2021-03-10T13:43:16.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.087 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_COMPLEXITY": { 2021-03-10T13:43:16.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_COMPLEXITY", 2021-03-10T13:43:16.089 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:16.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.090 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.091 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_LENGTH": { 2021-03-10T13:43:16.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_LENGTH", 2021-03-10T13:43:16.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.092 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8", 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "REST_REQUESTS_TIMEOUT": { 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "REST_REQUESTS_TIMEOUT", 2021-03-10T13:43:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "45", 2021-03-10T13:43:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ACCESS_KEY": { 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ACCESS_KEY", 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.098 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.099 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ADMIN_RESOURCE": { 2021-03-10T13:43:16.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ADMIN_RESOURCE", 2021-03-10T13:43:16.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.100 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.101 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_HOST": { 2021-03-10T13:43:16.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_HOST", 2021-03-10T13:43:16.102 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.103 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.104 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_PORT": { 2021-03-10T13:43:16.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_PORT", 2021-03-10T13:43:16.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.105 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "80", 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.106 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.107 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SCHEME": { 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SCHEME", 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "http", 2021-03-10T13:43:16.108 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.109 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SECRET_KEY": { 2021-03-10T13:43:16.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SECRET_KEY", 2021-03-10T13:43:16.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.110 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.111 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.112 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SSL_VERIFY": { 2021-03-10T13:43:16.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SSL_VERIFY", 2021-03-10T13:43:16.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.113 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.114 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_USER_ID": { 2021-03-10T13:43:16.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_USER_ID", 2021-03-10T13:43:16.115 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.116 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.117 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_SPAN": { 2021-03-10T13:43:16.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_SPAN", 2021-03-10T13:43:16.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.118 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2021-03-10T13:43:16.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.119 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_1": { 2021-03-10T13:43:16.120 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_1", 2021-03-10T13:43:16.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:16.121 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.122 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_2": { 2021-03-10T13:43:16.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_2", 2021-03-10T13:43:16.123 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:16.124 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.125 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crt_file": { 2021-03-10T13:43:16.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crt_file", 2021-03-10T13:43:16.126 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.127 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug": { 2021-03-10T13:43:16.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "debug", 2021-03-10T13:43:16.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.129 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.130 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Enable/disable debug options", 2021-03-10T13:43:16.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.131 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "jwt_token_ttl": { 2021-03-10T13:43:16.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "jwt_token_ttl", 2021-03-10T13:43:16.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.132 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "28800", 2021-03-10T13:43:16.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.133 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2021-03-10T13:43:16.134 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2021-03-10T13:43:16.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.135 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.136 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.137 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.138 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.139 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.140 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.141 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.142 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.143 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.144 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.145 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.146 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.147 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.148 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.149 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.150 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.152 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:16.153 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "::", 2021-03-10T13:43:16.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.154 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:16.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:16.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.156 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8080", 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2021-03-10T13:43:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2021-03-10T13:43:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl_server_port": { 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl_server_port", 2021-03-10T13:43:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8443", 2021-03-10T13:43:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_behaviour": { 2021-03-10T13:43:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_behaviour", 2021-03-10T13:43:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "redirect", 2021-03-10T13:43:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "redirect" 2021-03-10T13:43:16.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.166 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_error_status_code": { 2021-03-10T13:43:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_error_status_code", 2021-03-10T13:43:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "500", 2021-03-10T13:43:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "400", 2021-03-10T13:43:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "599", 2021-03-10T13:43:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url_prefix": { 2021-03-10T13:43:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url_prefix", 2021-03-10T13:43:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2021-03-10T13:43:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2021-03-10T13:43:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.174 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.175 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.176 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "devicehealth", 2021-03-10T13:43:16.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_monitoring": { 2021-03-10T13:43:16.177 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_monitoring", 2021-03-10T13:43:16.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.178 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "monitor device health metrics", 2021-03-10T13:43:16.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.179 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.180 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.181 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.182 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.183 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.184 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.185 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.186 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.187 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.188 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.189 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.190 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.191 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.192 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.193 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.194 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mark_out_threshold": { 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mark_out_threshold", 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.195 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2419200", 2021-03-10T13:43:16.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.196 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically mark OSD if it may fail before this long", 2021-03-10T13:43:16.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.197 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_name": { 2021-03-10T13:43:16.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_name", 2021-03-10T13:43:16.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.198 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "device_health_metrics", 2021-03-10T13:43:16.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name of pool in which to store device health metrics", 2021-03-10T13:43:16.199 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retention_period": { 2021-03-10T13:43:16.200 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retention_period", 2021-03-10T13:43:16.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15552000", 2021-03-10T13:43:16.201 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain device health metrics", 2021-03-10T13:43:16.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.202 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_frequency": { 2021-03-10T13:43:16.203 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_frequency", 2021-03-10T13:43:16.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.204 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2021-03-10T13:43:16.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.205 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to scrape device health metrics", 2021-03-10T13:43:16.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.206 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "self_heal": { 2021-03-10T13:43:16.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "self_heal", 2021-03-10T13:43:16.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.207 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.208 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "preemptively heal cluster around devices that may fail", 2021-03-10T13:43:16.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.209 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:16.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:16.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.210 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:16.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and check device health", 2021-03-10T13:43:16.211 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_threshold": { 2021-03-10T13:43:16.212 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_threshold", 2021-03-10T13:43:16.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7257600", 2021-03-10T13:43:16.213 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise health warning if OSD may fail before this long", 2021-03-10T13:43:16.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.214 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.215 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "diskprediction_local", 2021-03-10T13:43:16.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.216 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.217 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.218 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.219 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.220 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.221 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.222 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.223 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.224 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.225 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.226 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.227 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.228 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.229 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.230 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predict_interval": { 2021-03-10T13:43:16.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predict_interval", 2021-03-10T13:43:16.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.231 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2021-03-10T13:43:16.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.232 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predictor_model": { 2021-03-10T13:43:16.233 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predictor_model", 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "prophetstor", 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.234 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.235 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:16.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:16.236 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2021-03-10T13:43:16.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.237 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.238 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.239 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "influx", 2021-03-10T13:43:16.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": false, 2021-03-10T13:43:16.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "influxdb python module not found", 2021-03-10T13:43:16.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "batch_size": { 2021-03-10T13:43:16.240 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "batch_size", 2021-03-10T13:43:16.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5000", 2021-03-10T13:43:16.241 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.242 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "database": { 2021-03-10T13:43:16.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "database", 2021-03-10T13:43:16.243 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph", 2021-03-10T13:43:16.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.244 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.245 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "hostname": { 2021-03-10T13:43:16.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "hostname", 2021-03-10T13:43:16.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.246 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.247 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.248 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:16.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:16.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.249 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "30", 2021-03-10T13:43:16.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.250 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.251 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.252 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.253 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.254 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.255 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.256 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.257 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.258 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.259 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.260 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.261 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.262 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.263 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.264 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.265 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.266 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.267 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "port": { 2021-03-10T13:43:16.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "port", 2021-03-10T13:43:16.268 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8086", 2021-03-10T13:43:16.269 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.270 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "false", 2021-03-10T13:43:16.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.272 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.273 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "threads": { 2021-03-10T13:43:16.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "threads", 2021-03-10T13:43:16.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.274 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:16.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.275 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.276 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2021-03-10T13:43:16.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2021-03-10T13:43:16.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.277 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.278 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "verify_ssl": { 2021-03-10T13:43:16.279 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "verify_ssl", 2021-03-10T13:43:16.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "true", 2021-03-10T13:43:16.280 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.281 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.282 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "insights", 2021-03-10T13:43:16.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.283 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.284 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.285 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.286 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.287 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.288 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.295 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.296 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.297 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.298 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "iostat", 2021-03-10T13:43:16.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.299 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.300 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.301 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.302 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.303 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.304 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.305 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.306 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.307 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.308 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.309 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.310 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.311 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.312 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.313 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "k8sevents", 2021-03-10T13:43:16.314 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ceph_event_retention_days": { 2021-03-10T13:43:16.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ceph_event_retention_days", 2021-03-10T13:43:16.315 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2021-03-10T13:43:16.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.316 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Days to hold ceph event information within local cache", 2021-03-10T13:43:16.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.317 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_check_secs": { 2021-03-10T13:43:16.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_check_secs", 2021-03-10T13:43:16.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.318 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2021-03-10T13:43:16.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "10", 2021-03-10T13:43:16.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.319 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "interval (secs) to check for cluster configuration changes", 2021-03-10T13:43:16.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.320 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.322 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.333 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.334 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "localpool", 2021-03-10T13:43:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "failure_domain": { 2021-03-10T13:43:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "failure_domain", 2021-03-10T13:43:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "host", 2021-03-10T13:43:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "failure domain for any created local pool", 2021-03-10T13:43:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.343 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.344 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.345 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.346 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.347 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.348 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.349 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.350 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.351 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.352 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_size": { 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_size", 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.353 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default min_size for any created local pool", 2021-03-10T13:43:16.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.354 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_rep": { 2021-03-10T13:43:16.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "num_rep", 2021-03-10T13:43:16.355 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "3", 2021-03-10T13:43:16.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.356 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default replica count for any created local pool", 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.357 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_num": { 2021-03-10T13:43:16.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_num", 2021-03-10T13:43:16.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.358 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "128", 2021-03-10T13:43:16.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.359 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default pg_num for any created local pool", 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prefix": { 2021-03-10T13:43:16.360 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prefix", 2021-03-10T13:43:16.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.361 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name prefix for any created local pool", 2021-03-10T13:43:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "subtree": { 2021-03-10T13:43:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "subtree", 2021-03-10T13:43:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "rack", 2021-03-10T13:43:16.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.364 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "CRUSH level for which to create a local pool", 2021-03-10T13:43:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2021-03-10T13:43:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.373 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator": { 2021-03-10T13:43:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2021-03-10T13:43:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rook", 2021-03-10T13:43:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "test_orchestrator" 2021-03-10T13:43:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Orchestrator backend", 2021-03-10T13:43:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "osd_support", 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.387 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.395 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_autoscaler", 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.404 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.405 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.406 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.407 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.408 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.409 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.410 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.411 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2021-03-10T13:43:16.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2021-03-10T13:43:16.412 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:16.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.413 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.414 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "progress", 2021-03-10T13:43:16.415 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.416 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.417 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.418 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.419 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.420 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.421 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.422 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.423 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.424 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.425 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.426 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max_completed_events": { 2021-03-10T13:43:16.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "max_completed_events", 2021-03-10T13:43:16.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.429 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "50", 2021-03-10T13:43:16.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.430 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "number of past completed events to remember", 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.431 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "persist_interval": { 2021-03-10T13:43:16.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "persist_interval", 2021-03-10T13:43:16.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.432 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to persist completed events", 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.433 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.434 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus", 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.435 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.436 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.437 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.438 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.439 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.440 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.441 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.442 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.443 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.444 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.445 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.446 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.447 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools": { 2021-03-10T13:43:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools", 2021-03-10T13:43:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.450 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.453 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.454 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools_refresh_interval": { 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools_refresh_interval", 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_interval": { 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_interval", 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_support", 2021-03-10T13:43:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.477 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.478 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.479 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mirror_snapshot_schedule": { 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mirror_snapshot_schedule", 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.480 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.481 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "trash_purge_schedule": { 2021-03-10T13:43:16.482 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "trash_purge_schedule", 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.483 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.484 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "restful", 2021-03-10T13:43:16.485 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2021-03-10T13:43:16.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2021-03-10T13:43:16.486 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.487 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.488 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.489 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.490 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.491 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.492 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.493 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.494 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.495 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.496 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.497 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.498 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.499 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.500 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.501 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2021-03-10T13:43:16.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2021-03-10T13:43:16.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.502 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.503 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2021-03-10T13:43:16.504 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2021-03-10T13:43:16.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.505 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.506 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.507 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rook", 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.508 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.509 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.510 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.511 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.512 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.513 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.514 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.515 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.516 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.519 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "selftest", 2021-03-10T13:43:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.523 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.524 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.525 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.526 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.527 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.528 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.529 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.530 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.531 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.532 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.533 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.534 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.535 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption1": { 2021-03-10T13:43:16.536 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption1", 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.537 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.538 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption2": { 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption2", 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.539 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "xyz", 2021-03-10T13:43:16.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.540 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.541 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption1": { 2021-03-10T13:43:16.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption1", 2021-03-10T13:43:16.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.542 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption2": { 2021-03-10T13:43:16.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption2", 2021-03-10T13:43:16.544 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.545 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.546 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption3": { 2021-03-10T13:43:16.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption3", 2021-03-10T13:43:16.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2021-03-10T13:43:16.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.547 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.548 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption4": { 2021-03-10T13:43:16.549 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption4", 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.550 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption5": { 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption5", 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption6": { 2021-03-10T13:43:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption6", 2021-03-10T13:43:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testkey": { 2021-03-10T13:43:16.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testkey", 2021-03-10T13:43:16.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.557 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.558 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.559 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testlkey": { 2021-03-10T13:43:16.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testlkey", 2021-03-10T13:43:16.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.560 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.561 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testnewline": { 2021-03-10T13:43:16.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testnewline", 2021-03-10T13:43:16.562 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.563 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.564 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "status", 2021-03-10T13:43:16.565 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.566 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.567 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.568 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.569 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.570 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.571 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.572 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.573 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.574 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.575 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.576 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.577 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.578 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.579 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telegraf", 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.580 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "address": { 2021-03-10T13:43:16.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "address", 2021-03-10T13:43:16.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.581 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "unixgram:///tmp/telegraf.sock", 2021-03-10T13:43:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15", 2021-03-10T13:43:16.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.584 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.592 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telemetry", 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_basic": { 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_basic", 2021-03-10T13:43:16.600 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.601 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Share basic cluster information (size, version)", 2021-03-10T13:43:16.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.602 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_crash": { 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_crash", 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.603 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.604 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.605 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_device": { 2021-03-10T13:43:16.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_device", 2021-03-10T13:43:16.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.606 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2021-03-10T13:43:16.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.607 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_ident": { 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_ident", 2021-03-10T13:43:16.608 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.609 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.610 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "contact": { 2021-03-10T13:43:16.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "contact", 2021-03-10T13:43:16.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.611 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.612 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "description": { 2021-03-10T13:43:16.613 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "description", 2021-03-10T13:43:16.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.614 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.615 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_url": { 2021-03-10T13:43:16.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_url", 2021-03-10T13:43:16.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.616 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/device", 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.617 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enabled": { 2021-03-10T13:43:16.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enabled", 2021-03-10T13:43:16.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.619 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.620 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.621 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "24", 2021-03-10T13:43:16.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "8", 2021-03-10T13:43:16.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.622 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.623 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_opt_revision": { 2021-03-10T13:43:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "last_opt_revision", 2021-03-10T13:43:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1", 2021-03-10T13:43:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "leaderboard": { 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "leaderboard", 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "organization": { 2021-03-10T13:43:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "organization", 2021-03-10T13:43:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.644 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "proxy": { 2021-03-10T13:43:16.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "proxy", 2021-03-10T13:43:16.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.645 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url": { 2021-03-10T13:43:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url", 2021-03-10T13:43:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/report", 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.650 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "test_orchestrator", 2021-03-10T13:43:16.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.651 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.652 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.653 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.654 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.655 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.656 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.657 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.658 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.659 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.660 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.661 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.662 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "volumes", 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.666 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.667 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.668 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.669 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.670 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.671 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.673 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.674 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.675 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.758 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.763 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.763 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix", 2021-03-10T13:43:16.763 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2021-03-10T13:43:16.763 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2021-03-10T13:43:16.763 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "discovery_interval": { 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "discovery_interval", 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "100", 2021-03-10T13:43:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "identifier": { 2021-03-10T13:43:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "identifier", 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2021-03-10T13:43:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.771 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2021-03-10T13:43:16.772 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.773 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.774 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.775 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.776 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.777 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2021-03-10T13:43:16.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2021-03-10T13:43:16.778 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2021-03-10T13:43:16.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.779 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2021-03-10T13:43:16.780 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2021-03-10T13:43:16.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2021-03-10T13:43:16.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.781 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2021-03-10T13:43:16.782 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2021-03-10T13:43:16.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2021-03-10T13:43:16.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.783 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.784 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.784 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.784 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.784 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.784 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.785 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_host": { 2021-03-10T13:43:16.785 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_host", 2021-03-10T13:43:16.785 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.785 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.785 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.786 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2021-03-10T13:43:16.786 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.786 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.786 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.786 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.787 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.787 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.787 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.787 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.787 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_port": { 2021-03-10T13:43:16.788 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_port", 2021-03-10T13:43:16.788 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2021-03-10T13:43:16.788 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.788 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.788 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10051", 2021-03-10T13:43:16.789 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.789 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.789 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.789 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.789 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.790 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.790 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.790 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.790 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_sender": { 2021-03-10T13:43:16.790 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_sender", 2021-03-10T13:43:16.791 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2021-03-10T13:43:16.791 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2021-03-10T13:43:16.791 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2021-03-10T13:43:16.791 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/usr/bin/zabbix_sender", 2021-03-10T13:43:16.791 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2021-03-10T13:43:16.792 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2021-03-10T13:43:16.792 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2021-03-10T13:43:16.792 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2021-03-10T13:43:16.792 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2021-03-10T13:43:16.792 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.793 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {}, 2021-03-10T13:43:16.794 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "always_on_modules": { 2021-03-10T13:43:16.794 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nautilus": [ 2021-03-10T13:43:16.794 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2021-03-10T13:43:16.794 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2021-03-10T13:43:16.794 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2021-03-10T13:43:16.795 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator_cli", 2021-03-10T13:43:16.795 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2021-03-10T13:43:16.795 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2021-03-10T13:43:16.795 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2021-03-10T13:43:16.795 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2021-03-10T13:43:16.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "octopus": [ 2021-03-10T13:43:16.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2021-03-10T13:43:16.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2021-03-10T13:43:16.796 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2021-03-10T13:43:16.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2021-03-10T13:43:16.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_support", 2021-03-10T13:43:16.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2021-03-10T13:43:16.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2021-03-10T13:43:16.797 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2021-03-10T13:43:16.798 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2021-03-10T13:43:16.798 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2021-03-10T13:43:16.798 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2021-03-10T13:43:16.798 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2021-03-10T13:43:16.799 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_failure_osd_epoch": 4, 2021-03-10T13:43:16.799 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_clients": [ 2021-03-10T13:43:16.799 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.799 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:16.799 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3399696856 2021-03-10T13:43:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:16.801 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.801 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.801 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:16.801 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.801 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3970935116 2021-03-10T13:43:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2021-03-10T13:43:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2021-03-10T13:43:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.2.111:0", 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 1453015932 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2021-03-10T13:43:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for the mgr to restart... 2021-03-10T13:43:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Waiting for Mgr epoch 9... 2021-03-10T13:43:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'tell', 'mgr', 'mgr_status'] 2021-03-10T13:43:16.807 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 tell mgr mgr_status 2021-03-10T13:43:16.807 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:16 gibba011 bash[9813]: audit 2021-03-10T13:43:15.467314+0000 mon.a (mon.0) 70 : audit [DBG] from='client.? 172.21.2.111:0/572017450' entity='client.admin' cmd=[{"prefix": "mgr dump"}]: dispatch 2021-03-10T13:43:17.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:17 gibba011 bash[9813]: cluster 2021-03-10T13:43:16.222461+0000 mon.a (mon.0) 71 : cluster [INF] Active manager daemon y restarted 2021-03-10T13:43:17.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:17 gibba011 bash[9813]: cluster 2021-03-10T13:43:16.222859+0000 mon.a (mon.0) 72 : cluster [INF] Activating manager daemon y 2021-03-10T13:43:17.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:17 gibba011 bash[9813]: cluster 2021-03-10T13:43:16.224177+0000 mon.a (mon.0) 73 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2021-03-10T13:43:17.327 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:17 gibba011 bash[10046]: Warning: Permanently added 'gibba011,172.21.2.111' (ECDSA) to the list of known hosts. 2021-03-10T13:43:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2021-03-10T13:43:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap_epoch": 11, 2021-03-10T13:43:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "initialized": true 2021-03-10T13:43:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2021-03-10T13:43:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: cluster 2021-03-10T13:43:17.038248+0000 mon.a (mon.0) 74 : cluster [DBG] mgrmap e10: y(active, starting, since 0.815443s) 2021-03-10T13:43:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.039801+0000 mon.a (mon.0) 75 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:43:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.040501+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:43:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.040618+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:43:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.040735+0000 mon.a (mon.0) 78 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:43:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.041137+0000 mon.a (mon.0) 79 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:43:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.041775+0000 mon.a (mon.0) 80 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:43:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.042173+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:43:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.042549+0000 mon.a (mon.0) 82 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:43:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.042898+0000 mon.a (mon.0) 83 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.043206+0000 mon.a (mon.0) 84 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.043488+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: cluster 2021-03-10T13:43:17.045277+0000 mon.a (mon.0) 86 : cluster [INF] Manager daemon y is now available 2021-03-10T13:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.064284+0000 mon.a (mon.0) 87 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-03-10T13:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.224486+0000 mon.a (mon.0) 88 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-03-10T13:43:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.224705+0000 mon.a (mon.0) 89 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.521150+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:17.520763\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.522926+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:17.520763\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:43:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:18 gibba011 bash[9813]: audit 2021-03-10T13:43:17.523381+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:43:18.862 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Generating a dashboard self-signed certificate... 2021-03-10T13:43:18.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'dashboard', 'create-self-signed-cert'] 2021-03-10T13:43:18.862 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 dashboard create-self-signed-cert 2021-03-10T13:43:19.243 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:19 gibba011 bash[9813]: cluster 2021-03-10T13:43:18.045753+0000 mon.a (mon.0) 93 : cluster [DBG] mgrmap e11: y(active, since 1.82295s) 2021-03-10T13:43:20.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:20 gibba011 bash[9813]: cluster 2021-03-10T13:43:19.054009+0000 mon.a (mon.0) 94 : cluster [DBG] mgrmap e12: y(active, since 2s) 2021-03-10T13:43:20.725 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout Self-signed certificate created 2021-03-10T13:43:21.301 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:21 gibba011 bash[9813]: audit 2021-03-10T13:43:20.723047+0000 mon.a (mon.0) 95 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/crt","val":"-----BEGIN CERTIFICATE-----\nMIICzzCCAbcCEEv2QeE4mEDnoeJEA2U49EMwDQYJKoZIhvcNAQENBQAwJjELMAkG\nA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgtZGFzaGJvYXJkMB4XDTIxMDMxMDEzNDMy\nMFoXDTMxMDMwODEzNDMyMFowJjELMAkGA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgt\nZGFzaGJvYXJkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxlCln3eT\nxnhJWehpuPcjYTMkT/IoLsnl2ex3JoJVQ70GgWlhuKwN4SnTHwEtykrWAVx3ln8w\nMtgYCr9HhoxZg/Z80dQJdkw+aILrfdvr7bxozCyHLOJOt56ZKAoDynzSfgMYO7ZC\nXTUJLMGX+HFdWlLw9Yxw6HQh9JD3TJVKeuj2b8A6c7COXPOKHS1EFv8iZSxUQrBp\nWxf9vAeaIHIiFbE6+seKIMizbkuKzxcnjoQwT9Yll1BIfOexQsVxTNj89VTgwYXl\nsXTindecv7/TmuIV29maUHQhYsJfbkyln5xSxanXs0E43eLYJwGG7aR9C5X1JSpW\n2MfH20QlSqMQNwIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQB7kSYao6SmQ1TeO3Eo\nVjLdkoCDA1LpR7T0QLVh223IZPZfQLYVe3A12SkS1df5lyB5n/3L0/u4F9Iam4Hh\np25x4jmMaY/Y+CtyKAfhmkXNkQs9Bj7bhT4SmOjcfiOwI1EQTShAEU89Df/3KinT\nTWozdHeTTzHb76v3P6Vh7+HOPHjROzO7RldEFmsovSLExBElzqCDrQxEV2whjJMq\n6ac8dH7Xlzoy7vQXdATdzK2otJNo/qag0E6p25KYaGRrW9mseypI/k/7WQ+JNL09\nuN6MD2Dk/WjlIwWzt7+W8OvL7DDKYQTZxuNDhUmkkOahuXr0Dy0z4f3UoRgglzJ+\nuqlv\n-----END CERTIFICATE-----\n"}]: dispatch 2021-03-10T13:43:21.302 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:21 gibba011 bash[9813]: audit 2021-03-10T13:43:20.724676+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/crt","val":"-----BEGIN CERTIFICATE-----\nMIICzzCCAbcCEEv2QeE4mEDnoeJEA2U49EMwDQYJKoZIhvcNAQENBQAwJjELMAkG\nA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgtZGFzaGJvYXJkMB4XDTIxMDMxMDEzNDMy\nMFoXDTMxMDMwODEzNDMyMFowJjELMAkGA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgt\nZGFzaGJvYXJkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxlCln3eT\nxnhJWehpuPcjYTMkT/IoLsnl2ex3JoJVQ70GgWlhuKwN4SnTHwEtykrWAVx3ln8w\nMtgYCr9HhoxZg/Z80dQJdkw+aILrfdvr7bxozCyHLOJOt56ZKAoDynzSfgMYO7ZC\nXTUJLMGX+HFdWlLw9Yxw6HQh9JD3TJVKeuj2b8A6c7COXPOKHS1EFv8iZSxUQrBp\nWxf9vAeaIHIiFbE6+seKIMizbkuKzxcnjoQwT9Yll1BIfOexQsVxTNj89VTgwYXl\nsXTindecv7/TmuIV29maUHQhYsJfbkyln5xSxanXs0E43eLYJwGG7aR9C5X1JSpW\n2MfH20QlSqMQNwIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQB7kSYao6SmQ1TeO3Eo\nVjLdkoCDA1LpR7T0QLVh223IZPZfQLYVe3A12SkS1df5lyB5n/3L0/u4F9Iam4Hh\np25x4jmMaY/Y+CtyKAfhmkXNkQs9Bj7bhT4SmOjcfiOwI1EQTShAEU89Df/3KinT\nTWozdHeTTzHb76v3P6Vh7+HOPHjROzO7RldEFmsovSLExBElzqCDrQxEV2whjJMq\n6ac8dH7Xlzoy7vQXdATdzK2otJNo/qag0E6p25KYaGRrW9mseypI/k/7WQ+JNL09\nuN6MD2Dk/WjlIwWzt7+W8OvL7DDKYQTZxuNDhUmkkOahuXr0Dy0z4f3UoRgglzJ+\nuqlv\n-----END CERTIFICATE-----\n"}]': finished 2021-03-10T13:43:21.302 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:21 gibba011 bash[9813]: audit 2021-03-10T13:43:20.725187+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDGUKWfd5PGeElZ\n6Gm49yNhMyRP8iguyeXZ7HcmglVDvQaBaWG4rA3hKdMfAS3KStYBXHeWfzAy2BgK\nv0eGjFmD9nzR1Al2TD5ogut92+vtvGjMLIcs4k63npkoCgPKfNJ+Axg7tkJdNQks\nwZf4cV1aUvD1jHDodCH0kPdMlUp66PZvwDpzsI5c84odLUQW/yJlLFRCsGlbF/28\nB5ogciIVsTr6x4ogyLNuS4rPFyeOhDBP1iWXUEh857FCxXFM2Pz1VODBheWxdOKd\n15y/v9Oa4hXb2ZpQdCFiwl9uTKWfnFLFqdezQTjd4tgnAYbtpH0LlfUlKlbYx8fb\nRCVKoxA3AgMBAAECggEBAKN1Ya5ZH2KdTsheepXMjkrVnn6OuHQoUd4Xu/Xvw1u7\n+Bq+w581oZMPqEaL8DQnDtG8Z42SdCXUW6ywEMJj+Hx+tE83GD17ho0aLARnOsWP\nYEm6AgkK15sdjFumNY226MCYyGafSYAN8OwE/b/3oNyMNLQP+BiBcGNnAY8MFoHL\nA60+cRmWLnV/zgHha708y9dHDC8JcdXbSwPjIGh/RghEIEuprfQIN/UI9DHEM0HH\nCGy7xygkUVTCIlwxoHYuVFtQTrmbZavWO78aaCqT53VP2LYkrGng/QFqzy80k9/w\ngX86KIUrUPI7H6tzci6KJWy24DYr4DxiHmCll9REh3ECgYEA5DGDoCQqmfU7UdX8\n+OAMnu0+b0jZbGFqqoQYfdlKNC/m4EQt59K62SJCbM7yOi5DqqD+m6wEPXDe75xB\nZD/O7+Jd+XWAdfFueQUc6s6lVL4xrrEBBjw0Jbd9Te7sr+F0dxJv6CEHuuzJojuy\np8VdAE8HMqt1x61g9ZhhPb3OlOMCgYEA3nsT2CgT8JPSLH9+Gmha8zewwwYxGpub\n3KObaly2EE5dMJojCJioPFW32B96SyxuxBbgA0yEyGs1ZOAFV3dVu6DonpbS1c6L\n2aviXYXw9hzQLwb+EAPYheO0YQBsXxUZOfG/OJ+xrTGA20Z3Hk+8ZwvLhVNnKrUj\nkawf0cUEC50CgYAv2g4/E7Z3SIyi8Zu12M3sudVg4sbtSlRi44Z7RYig/vaMSt/R\ngEg8ZknFpXP4LWF1M4OQw3a0J6ooVv0Dd7n+ZdrNCA3jLbi72g2hU6op5iRm/Kk5\nVICe9CZrpFGv5DhXC9+5rst/ME5sHdauBd4lxWy9/GSbdckiV0z09IuQfwKBgBIN\nnrCdn/CXrihaOBtgeHDgBgHtULK+Ccm/pO7ekw+DrGAzQcCC0rQCqKZq93t+IYbb\n7FWYMfYpbO3Al+5fCp/tELM9veIHnqWNIxM/uSvfbq1CPf+CsGE89vggFqsohSGE\nkmW/36UULUad0zCse7dI57zr5aU7iaVaeEKHYCqpAoGAOfu2LUrdmnRB+YjPo8FB\nbGLbyMtlh+KnQ+R1B0ConuNjZrUwn5ihQCaKyulo5ZtakvKIw3lw07CFSlAhZZ1V\nOl++dfqo+OPAojlz5L4x6c5tTQX+0EUTc7PVrL1gXrA1zjROMBB30BwSPDKeEanW\njZ7gWM0ghi3Ti4G75ZMo7vI=\n-----END PRIVATE KEY-----\n"}]: dispatch 2021-03-10T13:43:21.302 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:21 gibba011 bash[9813]: audit 2021-03-10T13:43:20.726396+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDGUKWfd5PGeElZ\n6Gm49yNhMyRP8iguyeXZ7HcmglVDvQaBaWG4rA3hKdMfAS3KStYBXHeWfzAy2BgK\nv0eGjFmD9nzR1Al2TD5ogut92+vtvGjMLIcs4k63npkoCgPKfNJ+Axg7tkJdNQks\nwZf4cV1aUvD1jHDodCH0kPdMlUp66PZvwDpzsI5c84odLUQW/yJlLFRCsGlbF/28\nB5ogciIVsTr6x4ogyLNuS4rPFyeOhDBP1iWXUEh857FCxXFM2Pz1VODBheWxdOKd\n15y/v9Oa4hXb2ZpQdCFiwl9uTKWfnFLFqdezQTjd4tgnAYbtpH0LlfUlKlbYx8fb\nRCVKoxA3AgMBAAECggEBAKN1Ya5ZH2KdTsheepXMjkrVnn6OuHQoUd4Xu/Xvw1u7\n+Bq+w581oZMPqEaL8DQnDtG8Z42SdCXUW6ywEMJj+Hx+tE83GD17ho0aLARnOsWP\nYEm6AgkK15sdjFumNY226MCYyGafSYAN8OwE/b/3oNyMNLQP+BiBcGNnAY8MFoHL\nA60+cRmWLnV/zgHha708y9dHDC8JcdXbSwPjIGh/RghEIEuprfQIN/UI9DHEM0HH\nCGy7xygkUVTCIlwxoHYuVFtQTrmbZavWO78aaCqT53VP2LYkrGng/QFqzy80k9/w\ngX86KIUrUPI7H6tzci6KJWy24DYr4DxiHmCll9REh3ECgYEA5DGDoCQqmfU7UdX8\n+OAMnu0+b0jZbGFqqoQYfdlKNC/m4EQt59K62SJCbM7yOi5DqqD+m6wEPXDe75xB\nZD/O7+Jd+XWAdfFueQUc6s6lVL4xrrEBBjw0Jbd9Te7sr+F0dxJv6CEHuuzJojuy\np8VdAE8HMqt1x61g9ZhhPb3OlOMCgYEA3nsT2CgT8JPSLH9+Gmha8zewwwYxGpub\n3KObaly2EE5dMJojCJioPFW32B96SyxuxBbgA0yEyGs1ZOAFV3dVu6DonpbS1c6L\n2aviXYXw9hzQLwb+EAPYheO0YQBsXxUZOfG/OJ+xrTGA20Z3Hk+8ZwvLhVNnKrUj\nkawf0cUEC50CgYAv2g4/E7Z3SIyi8Zu12M3sudVg4sbtSlRi44Z7RYig/vaMSt/R\ngEg8ZknFpXP4LWF1M4OQw3a0J6ooVv0Dd7n+ZdrNCA3jLbi72g2hU6op5iRm/Kk5\nVICe9CZrpFGv5DhXC9+5rst/ME5sHdauBd4lxWy9/GSbdckiV0z09IuQfwKBgBIN\nnrCdn/CXrihaOBtgeHDgBgHtULK+Ccm/pO7ekw+DrGAzQcCC0rQCqKZq93t+IYbb\n7FWYMfYpbO3Al+5fCp/tELM9veIHnqWNIxM/uSvfbq1CPf+CsGE89vggFqsohSGE\nkmW/36UULUad0zCse7dI57zr5aU7iaVaeEKHYCqpAoGAOfu2LUrdmnRB+YjPo8FB\nbGLbyMtlh+KnQ+R1B0ConuNjZrUwn5ihQCaKyulo5ZtakvKIw3lw07CFSlAhZZ1V\nOl++dfqo+OPAojlz5L4x6c5tTQX+0EUTc7PVrL1gXrA1zjROMBB30BwSPDKeEanW\njZ7gWM0ghi3Ti4G75ZMo7vI=\n-----END PRIVATE KEY-----\n"}]': finished 2021-03-10T13:43:21.462 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Creating initial admin user... 2021-03-10T13:43:21.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'dashboard', 'ac-user-create', 'admin', 'btxkuunij9', 'administrator', '--force-password', '--pwd-update-required'] 2021-03-10T13:43:21.462 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 dashboard ac-user-create admin btxkuunij9 administrator --force-password --pwd-update-required 2021-03-10T13:43:22.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:22 gibba011 bash[9813]: audit 2021-03-10T13:43:20.697579+0000 mgr.y (mgr.14140) 1 : audit [DBG] from='client.14153 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:22.672 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout {"username": "admin", "password": "$2b$12$sQ26gIQt47yRRnkjC1TLkuk6uqsSxTk6sKexy6qx2x.cTxDFCZDSy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1615383802, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2021-03-10T13:43:23.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:23 gibba011 bash[9813]: audit 2021-03-10T13:43:22.671317+0000 mon.a (mon.0) 99 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/accessdb_v2","val":"{\"users\": {\"admin\": {\"username\": \"admin\", \"password\": \"$2b$12$sQ26gIQt47yRRnkjC1TLkuk6uqsSxTk6sKexy6qx2x.cTxDFCZDSy\", \"roles\": [\"administrator\"], \"name\": null, \"email\": null, \"lastUpdate\": 1615383802, \"enabled\": true, \"pwdExpirationDate\": null, \"pwdUpdateRequired\": true}}, \"roles\": {}, \"version\": 2}"}]: dispatch 2021-03-10T13:43:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:23 gibba011 bash[9813]: audit 2021-03-10T13:43:22.673008+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/accessdb_v2","val":"{\"users\": {\"admin\": {\"username\": \"admin\", \"password\": \"$2b$12$sQ26gIQt47yRRnkjC1TLkuk6uqsSxTk6sKexy6qx2x.cTxDFCZDSy\", \"roles\": [\"administrator\"], \"name\": null, \"email\": null, \"lastUpdate\": 1615383802, \"enabled\": true, \"pwdExpirationDate\": null, \"pwdUpdateRequired\": true}}, \"roles\": {}, \"version\": 2}"}]': finished 2021-03-10T13:43:23.554 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Fetching dashboard port number... 2021-03-10T13:43:23.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '-e', 'CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0', '-e', 'NODE_NAME=gibba011', '-v', '/var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'docker.io/ceph/ceph:v15.2.0', 'config', 'get', 'mgr', 'mgr/dashboard/ssl_server_port'] 2021-03-10T13:43:23.555 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15.2.0 -e NODE_NAME=gibba011 -v /var/log/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpx5216ufe:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpyl2h0som:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15.2.0 config get mgr mgr/dashboard/ssl_server_port 2021-03-10T13:43:24.319 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:24 gibba011 bash[9813]: audit 2021-03-10T13:43:22.503283+0000 mgr.y (mgr.14140) 2 : audit [DBG] from='client.14155 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "password": "btxkuunij9", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:24.320 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:24 gibba011 bash[9813]: cluster 2021-03-10T13:43:23.073024+0000 mon.a (mon.0) 101 : cluster [DBG] mgrmap e13: y(active, since 6s) 2021-03-10T13:43:24.320 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:24 gibba011 bash[9813]: audit 2021-03-10T13:43:23.432798+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:24.320 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:24 gibba011 bash[9813]: audit 2021-03-10T13:43:23.530378+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:17.520763\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:24.320 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:24 gibba011 bash[9813]: audit 2021-03-10T13:43:23.532068+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:17.520763\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:43:24.543 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 8443 2021-03-10T13:43:25.124 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Ceph Dashboard is now available at: 2021-03-10T13:43:25.124 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.124 INFO:teuthology.orchestra.run.gibba011.stderr: URL: https://gibba011:8443/ 2021-03-10T13:43:25.125 INFO:teuthology.orchestra.run.gibba011.stderr: User: admin 2021-03-10T13:43:25.125 INFO:teuthology.orchestra.run.gibba011.stderr: Password: btxkuunij9 2021-03-10T13:43:25.125 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.125 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:You can access the Ceph CLI with: 2021-03-10T13:43:25.126 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.126 INFO:teuthology.orchestra.run.gibba011.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2021-03-10T13:43:25.126 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.126 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Please consider enabling telemetry to help improve Ceph: 2021-03-10T13:43:25.126 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.127 INFO:teuthology.orchestra.run.gibba011.stderr: ceph telemetry on 2021-03-10T13:43:25.127 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:For more information see: 2021-03-10T13:43:25.127 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.127 INFO:teuthology.orchestra.run.gibba011.stderr: https://docs.ceph.com/docs/master/mgr/telemetry/ 2021-03-10T13:43:25.128 INFO:teuthology.orchestra.run.gibba011.stderr: 2021-03-10T13:43:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:INFO:cephadm:Bootstrap complete. 2021-03-10T13:43:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Releasing lock 140189835284832 on /run/cephadm/6fd36d94-81a6-11eb-9080-001a4aab830c.lock 2021-03-10T13:43:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:DEBUG:cephadm:Lock 140189835284832 released on /run/cephadm/6fd36d94-81a6-11eb-9080-001a4aab830c.lock 2021-03-10T13:43:25.133 INFO:tasks.cephadm:Fetching config... 2021-03-10T13:43:25.133 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:43:25.134 DEBUG:teuthology.orchestra.run.gibba011:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2021-03-10T13:43:25.137 INFO:tasks.cephadm:Fetching client.admin keyring... 2021-03-10T13:43:25.138 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:43:25.138 DEBUG:teuthology.orchestra.run.gibba011:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2021-03-10T13:43:25.185 INFO:tasks.cephadm:Fetching mon keyring... 2021-03-10T13:43:25.185 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:43:25.186 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd if=/var/lib/ceph/6fd36d94-81a6-11eb-9080-001a4aab830c/mon.a/keyring of=/dev/stdout 2021-03-10T13:43:25.199 INFO:tasks.cephadm:Fetching pub ssh key... 2021-03-10T13:43:25.199 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:43:25.200 DEBUG:teuthology.orchestra.run.gibba011:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2021-03-10T13:43:25.249 INFO:tasks.cephadm:Installing pub ssh key for root users... 2021-03-10T13:43:25.250 DEBUG:teuthology.orchestra.run.gibba011:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2021-03-10T13:43:25.258 INFO:teuthology.orchestra.run.gibba011.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:43:25.262 DEBUG:teuthology.orchestra.run.gibba032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2021-03-10T13:43:25.278 INFO:teuthology.orchestra.run.gibba032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49cAnxgZ1LPebbx2y881+C/oBP0iPwaiD3rcfxveaMxQ5tYRdGKtK21GBh0PCafoVTeNmIYQd0R72xwB3pOPUHfumk1beWTyGs4joLrwDd9IXb0ktkkSEI96/U9GEqJPqdC5iZWT48Phrq0L+j/YEfsR446tTrixtheIkdiYGg/VQIlyj2FLL2+FXZZptA4ZyebgGDxTsYjd00bY5diuWllPeIBWTftwUfJSqcieqPTCrFGLuG6GvGCo82lQ+zY/YCl5qdxWDtEqLDGR2OlcypWXayPpz8eXUlYoaG+PtzQNXE8Yc+FlhSqttLcZ7u0aKvSehtJUdrs8NIQufBbAmY3azdFdfdnTHltbUFi7tVHYTJXY3FFisvlnLAIDP5pcFAqbiRfFIx5Skjgmw39uhpUc5zV+N9AlBVVot2/8PINf8W7lVVem6cw4oTxpHbKu3cvqLlQBC3uZF//6r2SCL8ZzxY+1IcMvCt2ovfMThG9xg6q3+Tg95KcPhXTiJOx0= ceph-6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:43:25.281 INFO:tasks.cephadm:Writing (initial) conf and keyring to gibba032 2021-03-10T13:43:25.282 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:43:25.282 DEBUG:teuthology.orchestra.run.gibba032:> dd of=/etc/ceph/ceph.conf 2021-03-10T13:43:25.291 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:43:25.292 DEBUG:teuthology.orchestra.run.gibba032:> dd of=/etc/ceph/ceph.client.admin.keyring 2021-03-10T13:43:25.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:25 gibba011 bash[9813]: audit 2021-03-10T13:43:24.544594+0000 mon.a (mon.0) 105 : audit [DBG] from='client.? 172.21.2.111:0/1267732238' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2021-03-10T13:43:25.339 INFO:tasks.cephadm:Adding host gibba032 to orchestrator... 2021-03-10T13:43:25.340 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch host add gibba032 2021-03-10T13:43:38.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:38 gibba011 bash[9813]: cluster 2021-03-10T13:43:37.044687+0000 mgr.y (mgr.14140) 3 : cluster [DBG] pgmap v3: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:40.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:40 gibba011 bash[9813]: cluster 2021-03-10T13:43:39.045685+0000 mgr.y (mgr.14140) 4 : cluster [DBG] pgmap v4: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:42 gibba011 bash[9813]: cluster 2021-03-10T13:43:41.046623+0000 mgr.y (mgr.14140) 5 : cluster [DBG] pgmap v5: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:42.827 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:43:42 gibba011 bash[10046]: Warning: Permanently added 'gibba032,172.21.2.132' (ECDSA) to the list of known hosts. 2021-03-10T13:43:43.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:43 gibba011 bash[9813]: audit 2021-03-10T13:43:42.794065+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"gibba011\": {\"hostname\": \"gibba011\", \"addr\": \"gibba011\", \"labels\": [], \"status\": \"\"}, \"gibba032\": {\"hostname\": \"gibba032\", \"addr\": \"gibba032\", \"labels\": [], \"status\": \"\"}}"}]: dispatch 2021-03-10T13:43:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:43 gibba011 bash[9813]: audit 2021-03-10T13:43:42.800091+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"gibba011\": {\"hostname\": \"gibba011\", \"addr\": \"gibba011\", \"labels\": [], \"status\": \"\"}, \"gibba032\": {\"hostname\": \"gibba032\", \"addr\": \"gibba032\", \"labels\": [], \"status\": \"\"}}"}]': finished 2021-03-10T13:43:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:43 gibba011 bash[9813]: audit 2021-03-10T13:43:42.933615+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:43:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:43 gibba011 bash[9813]: audit 2021-03-10T13:43:42.939391+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:43:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:43 gibba011 bash[9813]: audit 2021-03-10T13:43:42.941478+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:43.461 INFO:teuthology.orchestra.run.gibba032.stdout:Added host 'gibba032' 2021-03-10T13:43:44.105 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch host ls --format=json 2021-03-10T13:43:44.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: audit 2021-03-10T13:43:42.456798+0000 mgr.y (mgr.14140) 6 : audit [DBG] from='client.14159 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "gibba032", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: cephadm 2021-03-10T13:43:42.801373+0000 mgr.y (mgr.14140) 7 : cephadm [INF] Added host gibba032 2021-03-10T13:43:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: cluster 2021-03-10T13:43:43.047651+0000 mgr.y (mgr.14140) 8 : cluster [DBG] pgmap v6: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: audit 2021-03-10T13:43:43.069282+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:43.068977\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:43:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: audit 2021-03-10T13:43:43.071036+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:43.068977\", \"networks\": {}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:43:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:44 gibba011 bash[9813]: audit 2021-03-10T13:43:43.071550+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:43:46.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:46 gibba011 bash[9813]: cluster 2021-03-10T13:43:45.048374+0000 mgr.y (mgr.14140) 9 : cluster [DBG] pgmap v7: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:47.856 INFO:teuthology.orchestra.run.gibba032.stdout: 2021-03-10T13:43:47.856 INFO:teuthology.orchestra.run.gibba032.stdout:[{"addr": "gibba011", "hostname": "gibba011", "labels": [], "status": ""}, {"addr": "gibba032", "hostname": "gibba032", "labels": [], "status": ""}] 2021-03-10T13:43:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:48 gibba011 bash[9813]: cluster 2021-03-10T13:43:47.048954+0000 mgr.y (mgr.14140) 10 : cluster [DBG] pgmap v8: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:48.632 INFO:tasks.cephadm:Setting crush tunables to default 2021-03-10T13:43:48.632 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd crush tunables default 2021-03-10T13:43:49.484 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:49 gibba011 bash[9813]: audit 2021-03-10T13:43:47.855098+0000 mgr.y (mgr.14140) 11 : audit [DBG] from='client.14161 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:43:49.485 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:49 gibba011 bash[9813]: audit 2021-03-10T13:43:48.989660+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:49.485 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:49 gibba011 bash[9813]: audit 2021-03-10T13:43:49.102554+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:43.068977\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:43:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:50 gibba011 bash[9813]: cluster 2021-03-10T13:43:49.049513+0000 mgr.y (mgr.14140) 12 : cluster [DBG] pgmap v9: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:50 gibba011 bash[9813]: audit 2021-03-10T13:43:49.104310+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2021-03-10T13:43:43.068977\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:43:52.119 INFO:teuthology.orchestra.run.gibba011.stderr:adjusted tunables profile to default 2021-03-10T13:43:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:52 gibba011 bash[9813]: cluster 2021-03-10T13:43:51.049733+0000 mgr.y (mgr.14140) 13 : cluster [DBG] pgmap v10: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:52 gibba011 bash[9813]: audit 2021-03-10T13:43:51.193554+0000 mon.a (mon.0) 117 : audit [INF] from='client.? 172.21.2.111:0/248087182' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2021-03-10T13:43:52.759 INFO:tasks.cephadm:Adding mon.c on gibba011 2021-03-10T13:43:52.760 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add mon 'gibba011:[v2:172.21.2.111:3301,v1:172.21.2.111:6790]=c' 2021-03-10T13:43:53.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:53 gibba011 bash[9813]: audit 2021-03-10T13:43:52.118904+0000 mon.a (mon.0) 118 : audit [INF] from='client.? 172.21.2.111:0/248087182' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2021-03-10T13:43:53.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:53 gibba011 bash[9813]: cluster 2021-03-10T13:43:52.118930+0000 mon.a (mon.0) 119 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:43:54.292 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:54 gibba011 bash[9813]: cluster 2021-03-10T13:43:53.050195+0000 mgr.y (mgr.14140) 14 : cluster [DBG] pgmap v12: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: cluster 2021-03-10T13:43:55.050787+0000 mgr.y (mgr.14140) 15 : cluster [DBG] pgmap v13: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: audit 2021-03-10T13:43:55.329799+0000 mgr.y (mgr.14140) 16 : audit [DBG] from='client.14165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "gibba011:[v2:172.21.2.111:3301,v1:172.21.2.111:6790]=c", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:43:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: audit 2021-03-10T13:43:55.331042+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:43:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: audit 2021-03-10T13:43:55.331838+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:43:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: cephadm 2021-03-10T13:43:55.332494+0000 mgr.y (mgr.14140) 17 : cephadm [INF] Deploying daemon mon.c on gibba011 2021-03-10T13:43:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:56 gibba011 bash[9813]: audit 2021-03-10T13:43:55.332661+0000 mon.a (mon.0) 122 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2021-03-10T13:43:58.265 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:58 gibba011 bash[9813]: cluster 2021-03-10T13:43:57.051412+0000 mgr.y (mgr.14140) 18 : cluster [DBG] pgmap v14: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:43:59.335 INFO:teuthology.orchestra.run.gibba011.stdout:Deployed mon.c on host 'gibba011' 2021-03-10T13:43:59.403 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:59 gibba011 bash[9813]: audit 2021-03-10T13:43:58.533339+0000 mon.a (mon.0) 123 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:59.404 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:59 gibba011 bash[9813]: audit 2021-03-10T13:43:58.533830+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:43:59.404 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:59 gibba011 bash[9813]: audit 2021-03-10T13:43:58.535104+0000 mon.a (mon.0) 125 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520614\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:17.520691\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:43:59.404 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:59 gibba011 bash[9813]: audit 2021-03-10T13:43:58.942749+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942190\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}}, \"last_daemon_update\": \"2021-03-10T13:43:58.942270\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:43:59.404 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:43:59 gibba011 bash[9813]: audit 2021-03-10T13:43:58.944678+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942190\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}}, \"last_daemon_update\": \"2021-03-10T13:43:58.942270\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:00.048 DEBUG:teuthology.orchestra.run.gibba011:mon.c> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.c.service 2021-03-10T13:44:00.050 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-03-10T13:44:00.054 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph mon dump -f json 2021-03-10T13:44:00.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:44:05.036 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:44:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":2,"fsid":"6fd36d94-81a6-11eb-9080-001a4aab830c","modified":"2021-03-10T13:43:59.550843Z","created":"2021-03-10T13:42:41.205251Z","min_mon_release":15,"min_mon_release_name":"octopus","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:3300","nonce":0},{"type":"v1","addr":"172.21.2.111:6789","nonce":0}]},"addr":"172.21.2.111:6789/0","public_addr":"172.21.2.111:6789/0","priority":0,"weight":0},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:3301","nonce":0},{"type":"v1","addr":"172.21.2.111:6790","nonce":0}]},"addr":"172.21.2.111:6790/0","public_addr":"172.21.2.111:6790/0","priority":0,"weight":0}],"quorum":[0,1]} 2021-03-10T13:44:05.038 INFO:teuthology.orchestra.run.gibba011.stderr:dumped monmap epoch 2 2021-03-10T13:44:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:43:59.552650+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2021-03-10T13:44:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:01.052464+0000 mgr.y (mgr.14140) 20 : cluster [DBG] pgmap v16: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:01.554940+0000 mon.c (mon.1) 1 : cluster [INF] mon.c calling monitor election 2021-03-10T13:44:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:03.052966+0000 mgr.y (mgr.14140) 21 : cluster [DBG] pgmap v17: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.559059+0000 mon.a (mon.0) 129 : cluster [INF] mon.a is new leader, mons a,c in quorum (ranks 0,1) 2021-03-10T13:44:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.567901+0000 mon.a (mon.0) 130 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:44:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.568023+0000 mon.a (mon.0) 131 : cluster [DBG] fsmap 2021-03-10T13:44:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.568089+0000 mon.a (mon.0) 132 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:44:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.568537+0000 mon.a (mon.0) 133 : cluster [DBG] mgrmap e13: y(active, since 48s) 2021-03-10T13:44:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[9813]: cluster 2021-03-10T13:44:04.571129+0000 mon.a (mon.0) 134 : cluster [INF] overall HEALTH_OK 2021-03-10T13:44:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:43:59.552650+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2021-03-10T13:44:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:01.052464+0000 mgr.y (mgr.14140) 20 : cluster [DBG] pgmap v16: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:01.554940+0000 mon.c (mon.1) 1 : cluster [INF] mon.c calling monitor election 2021-03-10T13:44:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:03.052966+0000 mgr.y (mgr.14140) 21 : cluster [DBG] pgmap v17: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.559059+0000 mon.a (mon.0) 129 : cluster [INF] mon.a is new leader, mons a,c in quorum (ranks 0,1) 2021-03-10T13:44:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.567901+0000 mon.a (mon.0) 130 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:44:05.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.568023+0000 mon.a (mon.0) 131 : cluster [DBG] fsmap 2021-03-10T13:44:05.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.568089+0000 mon.a (mon.0) 132 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:44:05.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.568537+0000 mon.a (mon.0) 133 : cluster [DBG] mgrmap e13: y(active, since 48s) 2021-03-10T13:44:05.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:04 gibba011 bash[13015]: cluster 2021-03-10T13:44:04.571129+0000 mon.a (mon.0) 134 : cluster [INF] overall HEALTH_OK 2021-03-10T13:44:05.629 INFO:tasks.cephadm:Adding mon.b on gibba032 2021-03-10T13:44:05.630 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add mon gibba032:172.21.2.132=b 2021-03-10T13:44:06.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:05 gibba011 bash[9813]: audit 2021-03-10T13:44:05.038334+0000 mon.a (mon.0) 135 : audit [DBG] from='client.? 172.21.2.111:0/3459301526' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-03-10T13:44:06.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:05 gibba011 bash[9813]: cluster 2021-03-10T13:44:05.053468+0000 mgr.y (mgr.14140) 22 : cluster [DBG] pgmap v18: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:06.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:05 gibba011 bash[13015]: audit 2021-03-10T13:44:05.038334+0000 mon.a (mon.0) 135 : audit [DBG] from='client.? 172.21.2.111:0/3459301526' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-03-10T13:44:06.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:05 gibba011 bash[13015]: cluster 2021-03-10T13:44:05.053468+0000 mgr.y (mgr.14140) 22 : cluster [DBG] pgmap v18: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[9813]: cluster 2021-03-10T13:44:07.053973+0000 mgr.y (mgr.14140) 23 : cluster [DBG] pgmap v19: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[9813]: audit 2021-03-10T13:44:08.086663+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:44:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[9813]: audit 2021-03-10T13:44:08.089468+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[9813]: audit 2021-03-10T13:44:08.092749+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2021-03-10T13:44:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[13015]: cluster 2021-03-10T13:44:07.053973+0000 mgr.y (mgr.14140) 23 : cluster [DBG] pgmap v19: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[13015]: audit 2021-03-10T13:44:08.086663+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:44:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[13015]: audit 2021-03-10T13:44:08.089468+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:08 gibba011 bash[13015]: audit 2021-03-10T13:44:08.092749+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2021-03-10T13:44:09.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:09 gibba011 bash[9813]: audit 2021-03-10T13:44:08.083029+0000 mgr.y (mgr.14140) 24 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "gibba032:172.21.2.132=b", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:09.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:09 gibba011 bash[9813]: cephadm 2021-03-10T13:44:08.091793+0000 mgr.y (mgr.14140) 25 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:44:09.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:09 gibba011 bash[13015]: audit 2021-03-10T13:44:08.083029+0000 mgr.y (mgr.14140) 24 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "gibba032:172.21.2.132=b", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:09.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:09 gibba011 bash[13015]: cephadm 2021-03-10T13:44:08.091793+0000 mgr.y (mgr.14140) 25 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:44:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:10 gibba011 bash[9813]: cluster 2021-03-10T13:44:09.054845+0000 mgr.y (mgr.14140) 26 : cluster [DBG] pgmap v20: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:10 gibba011 bash[13015]: cluster 2021-03-10T13:44:09.054845+0000 mgr.y (mgr.14140) 26 : cluster [DBG] pgmap v20: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:12.093 INFO:teuthology.orchestra.run.gibba032.stdout:Deployed mon.b on host 'gibba032' 2021-03-10T13:44:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: cluster 2021-03-10T13:44:11.055749+0000 mgr.y (mgr.14140) 27 : cluster [DBG] pgmap v21: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: audit 2021-03-10T13:44:11.663109+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: audit 2021-03-10T13:44:11.664516+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: cluster 2021-03-10T13:44:11.055749+0000 mgr.y (mgr.14140) 27 : cluster [DBG] pgmap v21: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: audit 2021-03-10T13:44:11.663109+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: audit 2021-03-10T13:44:11.664516+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: audit 2021-03-10T13:44:11.671029+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: audit 2021-03-10T13:44:11.671029+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: audit 2021-03-10T13:44:11.899207+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[9813]: audit 2021-03-10T13:44:11.907003+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: audit 2021-03-10T13:44:11.899207+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:12 gibba011 bash[13015]: audit 2021-03-10T13:44:11.907003+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:12.916 DEBUG:teuthology.orchestra.run.gibba032:mon.b> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.b.service 2021-03-10T13:44:12.919 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2021-03-10T13:44:12.920 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph mon dump -f json 2021-03-10T13:44:12.968 INFO:journalctl@ceph.mon.b.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:44:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.929+0000 7f88756056c0 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:44:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.929+0000 7f88756056c0 0 ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc), process ceph-mon, pid 1 2021-03-10T13:44:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.929+0000 7f88756056c0 0 pidfile_write: ignore empty --pid-file 2021-03-10T13:44:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 load: jerasure load: lrc load: isa 2021-03-10T13:44:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:44:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:44:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:44:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option compression = kNoCompression 2021-03-10T13:44:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2021-03-10T13:44:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 set rocksdb option write_buffer_size = 33554432 2021-03-10T13:44:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 1 rocksdb: do_open column families: [default] 2021-03-10T13:44:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: RocksDB version: 6.1.2 2021-03-10T13:44:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:44:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Compile date Mar 23 2020 2021-03-10T13:44:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: DB SUMMARY 2021-03-10T13:44:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: CURRENT file: CURRENT 2021-03-10T13:44:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: IDENTITY file: IDENTITY 2021-03-10T13:44:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2021-03-10T13:44:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2021-03-10T13:44:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 610 ; 2021-03-10T13:44:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:44:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.create_if_missing: 0 2021-03-10T13:44:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:44:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.env: 0x55597e4db1a0 2021-03-10T13:44:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.info_log: 0x55598034a9a0 2021-03-10T13:44:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:44:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:44:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:44:13.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:44:13.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:44:13.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:44:13.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:44:13.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:44:13.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:44:13.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:44:13.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:44:13.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:44:13.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:44:13.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:44:13.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.db_log_dir: 2021-03-10T13:44:13.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2021-03-10T13:44:13.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:44:13.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:44:13.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:44:13.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:44:13.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:44:13.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:44:13.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:44:13.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:44:13.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:44:13.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.write_buffer_manager: 0x555980ee4a80 2021-03-10T13:44:13.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:44:13.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:44:13.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:44:13.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:44:13.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:44:13.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:44:13.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:44:13.272 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:44:13.272 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:44:13.272 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:44:13.272 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:44:13.272 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:44:13.273 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:44:13.273 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.row_cache: None 2021-03-10T13:44:13.273 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.wal_filter: None 2021-03-10T13:44:13.273 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:44:13.273 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:44:13.274 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:44:13.274 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:44:13.274 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:44:13.274 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:44:13.274 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:44:13.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:44:13.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:44:13.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:44:13.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:44:13.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:44:13.276 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:44:13.276 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:44:13.276 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:44:13.276 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:44:13.277 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:44:13.277 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:44:13.277 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:44:13.277 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:44:13.277 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:44:13.278 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Compression algorithms supported: 2021-03-10T13:44:13.278 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:44:13.278 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kZSTD supported: 0 2021-03-10T13:44:13.278 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:44:13.278 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:44:13.279 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:44:13.279 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:44:13.279 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:44:13.279 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:44:13.279 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:44:13.280 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2021-03-10T13:44:13.280 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2021-03-10T13:44:13.280 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:44:13.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.merge_operator: 2021-03-10T13:44:13.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:44:13.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:44:13.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:44:13.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:44:13.282 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55598026a138) 2021-03-10T13:44:13.282 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: cache_index_and_filter_blocks: 1 2021-03-10T13:44:13.282 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: cache_index_and_filter_blocks_with_high_priority: 1 2021-03-10T13:44:13.282 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:44:13.282 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: pin_top_level_index_and_filter: 1 2021-03-10T13:44:13.283 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: index_type: 0 2021-03-10T13:44:13.283 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: data_block_index_type: 0 2021-03-10T13:44:13.283 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:44:13.283 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: hash_index_allow_collision: 1 2021-03-10T13:44:13.283 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: checksum: 1 2021-03-10T13:44:13.284 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: no_block_cache: 0 2021-03-10T13:44:13.284 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_cache: 0x5559802a0610 2021-03-10T13:44:13.284 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_cache_name: BinnedLRUCache 2021-03-10T13:44:13.284 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_cache_options: 2021-03-10T13:44:13.284 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: capacity : 536870912 2021-03-10T13:44:13.285 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: num_shard_bits : 4 2021-03-10T13:44:13.285 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: strict_capacity_limit : 0 2021-03-10T13:44:13.285 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: high_pri_pool_ratio: 0.000 2021-03-10T13:44:13.285 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_cache_compressed: (nil) 2021-03-10T13:44:13.285 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: persistent_cache: (nil) 2021-03-10T13:44:13.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_size: 4096 2021-03-10T13:44:13.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_size_deviation: 10 2021-03-10T13:44:13.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_restart_interval: 16 2021-03-10T13:44:13.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: index_block_restart_interval: 1 2021-03-10T13:44:13.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: metadata_block_size: 4096 2021-03-10T13:44:13.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: partition_filters: 0 2021-03-10T13:44:13.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: use_delta_encoding: 1 2021-03-10T13:44:13.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:44:13.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: whole_key_filtering: 1 2021-03-10T13:44:13.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: verify_compression: 0 2021-03-10T13:44:13.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: read_amp_bytes_per_bit: 0 2021-03-10T13:44:13.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: format_version: 2 2021-03-10T13:44:13.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: enable_index_compression: 1 2021-03-10T13:44:13.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: block_align: 0 2021-03-10T13:44:13.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:44:13.289 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:44:13.289 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:44:13.289 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:44:13.289 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:44:13.289 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:44:13.290 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.num_levels: 7 2021-03-10T13:44:13.290 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:44:13.290 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:44:13.290 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:44:13.290 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:44:13.291 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:44:13.291 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:44:13.291 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:44:13.291 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:44:13.291 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:44:13.292 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:44:13.292 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:44:13.292 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:44:13.292 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:44:13.292 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:44:13.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:44:13.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:44:13.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:44:13.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:44:13.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:44:13.294 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:44:13.294 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:44:13.294 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:44:13.294 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:44:13.294 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:44:13.295 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:44:13.295 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:44:13.295 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:44:13.295 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:44:13.295 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:44:13.296 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:44:13.296 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:44:13.296 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:44:13.296 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:44:13.297 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:44:13.297 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:44:13.297 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:44:13.297 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:44:13.297 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:44:13.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:44:13.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:44:13.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:44:13.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:44:13.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:44:13.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:44:13.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:44:13.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:44:13.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:44:13.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:44:13.300 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:44:13.300 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:44:13.300 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:44:13.300 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:44:13.300 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:44:13.301 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:44:13.301 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:44:13.301 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:44:13.301 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:44:13.301 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:44:13.302 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: Options.ttl: 0 2021-03-10T13:44:13.302 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:44:13.302 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 0 2021-03-10T13:44:13.302 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383852941145, "job": 1, "event": "recovery_started", "log_files": [3]} 2021-03-10T13:44:13.303 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3 mode 2 2021-03-10T13:44:13.303 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383852941590, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1524, "table_properties": {"data_size": 622, "index_size": 28, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 500, "raw_average_value_size": 100, "num_data_blocks": 1, "num_entries": 5, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2021-03-10T13:44:13.303 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 5 2021-03-10T13:44:13.303 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615383852942182, "job": 1, "event": "recovery_finished"} 2021-03-10T13:44:13.303 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 4 rocksdb: DB pointer 0x55598033b200 2021-03-10T13:44:13.304 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f885c752700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2021-03-10T13:44:13.304 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f885c752700 4 rocksdb: [db/db_impl.cc:778] 2021-03-10T13:44:13.304 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** DB Stats ** 2021-03-10T13:44:13.304 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:44:13.304 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:44:13.305 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:44:13.305 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:44:13.305 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:44:13.305 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:44:13.305 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:44:13.306 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** Compaction Stats [default] ** 2021-03-10T13:44:13.306 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:44:13.306 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:44:13.306 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: L0 1/0 1.49 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.306 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Sum 1/0 1.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** Compaction Stats [default] ** 2021-03-10T13:44:13.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:44:13.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:44:13.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.308 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:44:13.308 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:44:13.308 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:44:13.308 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:44:13.308 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:44:13.309 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:44:13.309 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:44:13.309 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Interval compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:44:13.309 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:44:13.310 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:44:13.310 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** Compaction Stats [default] ** 2021-03-10T13:44:13.310 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:44:13.310 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:44:13.310 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: L0 1/0 1.49 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.311 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Sum 1/0 1.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.311 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:44:13.311 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** Compaction Stats [default] ** 2021-03-10T13:44:13.311 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:44:13.311 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:44:13.312 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2021-03-10T13:44:13.312 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:44:13.312 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Flush(GB): cumulative 0.000, interval 0.000 2021-03-10T13:44:13.312 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:44:13.312 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:44:13.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:44:13.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:44:13.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:44:13.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:44:13.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:44:13.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:44:13.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2021-03-10T13:44:13.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 using public_addr v2:172.21.2.132:0/0 -> [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] 2021-03-10T13:44:13.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.937+0000 7f88756056c0 0 starting mon.b rank -1 at public addrs [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] at bind addrs [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:44:13.315 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.941+0000 7f88756056c0 1 mon.b@-1(???) e0 preinit fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:44:13.315 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).mds e1 new map 2021-03-10T13:44:13.315 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).mds e1 print_map 2021-03-10T13:44:13.315 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: e1 2021-03-10T13:44:13.315 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: enable_multiple, ever_enabled_multiple: 0,0 2021-03-10T13:44:13.316 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-03-10T13:44:13.316 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: legacy client fscid: -1 2021-03-10T13:44:13.316 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: 2021-03-10T13:44:13.316 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: No filesystems configured 2021-03-10T13:44:13.316 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2021-03-10T13:44:13.317 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2021-03-10T13:44:13.317 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2021-03-10T13:44:13.317 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2021-03-10T13:44:13.317 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2021-03-10T13:44:13.317 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2021-03-10T13:44:13.318 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2021-03-10T13:44:13.318 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).osd e6 e6: 0 total, 0 up, 0 in 2021-03-10T13:44:13.318 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:44:13.318 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:44:13.319 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).osd e6 crush map has features 3314933000852226048, adjusting msgr requires 2021-03-10T13:44:13.319 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 0 mon.b@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:44:13.319 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: cluster 2021-03-10T13:44:11.055749+0000 mgr.y (mgr.14140) 27 : cluster [DBG] pgmap v21: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:13.319 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: audit 2021-03-10T13:44:11.663109+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:13.319 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: audit 2021-03-10T13:44:11.664516+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:13.320 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: audit 2021-03-10T13:44:11.671029+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:13.320 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: audit 2021-03-10T13:44:11.899207+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:13.320 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: audit 2021-03-10T13:44:11.907003+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}}, \"last_daemon_update\": \"2021-03-10T13:44:11.897620\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:13.320 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:12 gibba032 bash[9829]: debug 2021-03-10T13:44:12.973+0000 7f885ef57700 1 mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2021-03-10T13:44:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:12.985487+0000 mon.a (mon.0) 144 : cluster [INF] mon.a calling monitor election 2021-03-10T13:44:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:12.986429+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:44:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:13.056184+0000 mgr.y (mgr.14140) 28 : cluster [DBG] pgmap v22: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:14.985202+0000 mon.b (mon.2) 1 : cluster [INF] mon.b calling monitor election 2021-03-10T13:44:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:15.056958+0000 mgr.y (mgr.14140) 29 : cluster [DBG] pgmap v23: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:17.057629+0000 mgr.y (mgr.14140) 30 : cluster [DBG] pgmap v24: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.012301+0000 mon.a (mon.0) 145 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:44:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.022167+0000 mon.a (mon.0) 146 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:44:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.022353+0000 mon.a (mon.0) 147 : cluster [DBG] fsmap 2021-03-10T13:44:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.022422+0000 mon.a (mon.0) 148 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:44:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.022838+0000 mon.a (mon.0) 149 : cluster [DBG] mgrmap e13: y(active, since 61s) 2021-03-10T13:44:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.026578+0000 mon.a (mon.0) 150 : cluster [INF] overall HEALTH_OK 2021-03-10T13:44:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:12.985487+0000 mon.a (mon.0) 144 : cluster [INF] mon.a calling monitor election 2021-03-10T13:44:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:12.986429+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:44:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:13.056184+0000 mgr.y (mgr.14140) 28 : cluster [DBG] pgmap v22: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:14.985202+0000 mon.b (mon.2) 1 : cluster [INF] mon.b calling monitor election 2021-03-10T13:44:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:15.056958+0000 mgr.y (mgr.14140) 29 : cluster [DBG] pgmap v23: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:17.057629+0000 mgr.y (mgr.14140) 30 : cluster [DBG] pgmap v24: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.012301+0000 mon.a (mon.0) 145 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:44:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.022167+0000 mon.a (mon.0) 146 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:44:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.022353+0000 mon.a (mon.0) 147 : cluster [DBG] fsmap 2021-03-10T13:44:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.022422+0000 mon.a (mon.0) 148 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:44:18.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.022838+0000 mon.a (mon.0) 149 : cluster [DBG] mgrmap e13: y(active, since 61s) 2021-03-10T13:44:18.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:18 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.026578+0000 mon.a (mon.0) 150 : cluster [INF] overall HEALTH_OK 2021-03-10T13:44:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:12.985487+0000 mon.a (mon.0) 144 : cluster [INF] mon.a calling monitor election 2021-03-10T13:44:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:12.986429+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:44:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:13.056184+0000 mgr.y (mgr.14140) 28 : cluster [DBG] pgmap v22: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:14.985202+0000 mon.b (mon.2) 1 : cluster [INF] mon.b calling monitor election 2021-03-10T13:44:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:15.056958+0000 mgr.y (mgr.14140) 29 : cluster [DBG] pgmap v23: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:17.057629+0000 mgr.y (mgr.14140) 30 : cluster [DBG] pgmap v24: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.012301+0000 mon.a (mon.0) 145 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:44:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.022167+0000 mon.a (mon.0) 146 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.022353+0000 mon.a (mon.0) 147 : cluster [DBG] fsmap 2021-03-10T13:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.022422+0000 mon.a (mon.0) 148 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2021-03-10T13:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.022838+0000 mon.a (mon.0) 149 : cluster [DBG] mgrmap e13: y(active, since 61s) 2021-03-10T13:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:18 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.026578+0000 mon.a (mon.0) 150 : cluster [INF] overall HEALTH_OK 2021-03-10T13:44:19.458 INFO:teuthology.orchestra.run.gibba032.stdout: 2021-03-10T13:44:19.459 INFO:teuthology.orchestra.run.gibba032.stdout:{"epoch":3,"fsid":"6fd36d94-81a6-11eb-9080-001a4aab830c","modified":"2021-03-10T13:44:12.982620Z","created":"2021-03-10T13:42:41.205251Z","min_mon_release":15,"min_mon_release_name":"octopus","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:3300","nonce":0},{"type":"v1","addr":"172.21.2.111:6789","nonce":0}]},"addr":"172.21.2.111:6789/0","public_addr":"172.21.2.111:6789/0","priority":0,"weight":0},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:3301","nonce":0},{"type":"v1","addr":"172.21.2.111:6790","nonce":0}]},"addr":"172.21.2.111:6790/0","public_addr":"172.21.2.111:6790/0","priority":0,"weight":0},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:3300","nonce":0},{"type":"v1","addr":"172.21.2.132:6789","nonce":0}]},"addr":"172.21.2.132:6789/0","public_addr":"172.21.2.132:6789/0","priority":0,"weight":0}],"quorum":[0,1,2]} 2021-03-10T13:44:19.461 INFO:teuthology.orchestra.run.gibba032.stderr:dumped monmap epoch 3 2021-03-10T13:44:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:19 gibba032 bash[9829]: cluster 2021-03-10T13:44:18.074516+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:44:19.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:19 gibba011 bash[9813]: cluster 2021-03-10T13:44:18.074516+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:44:19.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:19 gibba011 bash[13015]: cluster 2021-03-10T13:44:18.074516+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:44:20.136 INFO:tasks.cephadm:Generating final ceph.conf file... 2021-03-10T13:44:20.137 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph config generate-minimal-conf 2021-03-10T13:44:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:20 gibba032 bash[9829]: cluster 2021-03-10T13:44:19.058588+0000 mgr.y (mgr.14140) 31 : cluster [DBG] pgmap v25: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:20 gibba032 bash[9829]: audit 2021-03-10T13:44:19.459441+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.132:0/2273526844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-03-10T13:44:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:20 gibba011 bash[9813]: cluster 2021-03-10T13:44:19.058588+0000 mgr.y (mgr.14140) 31 : cluster [DBG] pgmap v25: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:20 gibba011 bash[9813]: audit 2021-03-10T13:44:19.459441+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.132:0/2273526844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-03-10T13:44:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:20 gibba011 bash[13015]: cluster 2021-03-10T13:44:19.058588+0000 mgr.y (mgr.14140) 31 : cluster [DBG] pgmap v25: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:20 gibba011 bash[13015]: audit 2021-03-10T13:44:19.459441+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.132:0/2273526844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-03-10T13:44:22.465 INFO:teuthology.orchestra.run.gibba011.stdout:# minimal ceph.conf for 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:44:22.465 INFO:teuthology.orchestra.run.gibba011.stdout:[global] 2021-03-10T13:44:22.465 INFO:teuthology.orchestra.run.gibba011.stdout: fsid = 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:44:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: mon_host = [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] [v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0] 2021-03-10T13:44:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:22 gibba032 bash[9829]: cluster 2021-03-10T13:44:21.058891+0000 mgr.y (mgr.14140) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:22.520 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:22 gibba011 bash[9813]: cluster 2021-03-10T13:44:21.058891+0000 mgr.y (mgr.14140) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:22.521 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:22 gibba011 bash[13015]: cluster 2021-03-10T13:44:21.058891+0000 mgr.y (mgr.14140) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:23.033 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2021-03-10T13:44:23.033 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:44:23.034 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/ceph/ceph.conf 2021-03-10T13:44:23.040 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:44:23.041 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2021-03-10T13:44:23.096 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:44:23.096 DEBUG:teuthology.orchestra.run.gibba032:> sudo dd of=/etc/ceph/ceph.conf 2021-03-10T13:44:23.108 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:44:23.108 DEBUG:teuthology.orchestra.run.gibba032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2021-03-10T13:44:23.160 INFO:tasks.cephadm:Adding mgr.y on gibba011 2021-03-10T13:44:23.160 INFO:tasks.cephadm:Adding mgr.x on gibba032 2021-03-10T13:44:23.161 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch apply mgr '3;gibba011=y;gibba032=x' 2021-03-10T13:44:23.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:23 gibba011 bash[9813]: audit 2021-03-10T13:44:22.466619+0000 mon.a (mon.0) 153 : audit [DBG] from='client.? 172.21.2.111:0/3836502902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:23.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:23 gibba011 bash[13015]: audit 2021-03-10T13:44:22.466619+0000 mon.a (mon.0) 153 : audit [DBG] from='client.? 172.21.2.111:0/3836502902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:23.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:23 gibba032 bash[9829]: audit 2021-03-10T13:44:22.466619+0000 mon.a (mon.0) 153 : audit [DBG] from='client.? 172.21.2.111:0/3836502902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:24 gibba032 bash[9829]: cluster 2021-03-10T13:44:23.059426+0000 mgr.y (mgr.14140) 33 : cluster [DBG] pgmap v27: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:24 gibba011 bash[9813]: cluster 2021-03-10T13:44:23.059426+0000 mgr.y (mgr.14140) 33 : cluster [DBG] pgmap v27: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:24 gibba011 bash[13015]: cluster 2021-03-10T13:44:23.059426+0000 mgr.y (mgr.14140) 33 : cluster [DBG] pgmap v27: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:25.721 INFO:teuthology.orchestra.run.gibba032.stdout:Scheduled mgr update... 2021-03-10T13:44:26.142 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: cluster 2021-03-10T13:44:25.060031+0000 mgr.y (mgr.14140) 34 : cluster [DBG] pgmap v28: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:26.142 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.713570+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:44:26.143 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.721362+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:44:26.143 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.722894+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:26.143 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.725383+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:26.143 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.726100+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:26.143 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:26 gibba032 bash[9829]: audit 2021-03-10T13:44:25.726985+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:44:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: cluster 2021-03-10T13:44:25.060031+0000 mgr.y (mgr.14140) 34 : cluster [DBG] pgmap v28: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.713570+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:44:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.721362+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:44:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.722894+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.725383+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.726100+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[9813]: audit 2021-03-10T13:44:25.726985+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:44:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: cluster 2021-03-10T13:44:25.060031+0000 mgr.y (mgr.14140) 34 : cluster [DBG] pgmap v28: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.713570+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:44:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.721362+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:44:25.712574\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"y\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"x\", \"network\": \"\"}]}, \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:44:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.722894+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.725383+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.726100+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:26 gibba011 bash[13015]: audit 2021-03-10T13:44:25.726985+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:44:26.818 DEBUG:teuthology.orchestra.run.gibba032:mgr.x> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.x.service 2021-03-10T13:44:26.821 INFO:tasks.cephadm:Deploying OSDs... 2021-03-10T13:44:26.821 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:44:26.822 DEBUG:teuthology.orchestra.run.gibba011:> dd if=/scratch_devs of=/dev/stdout 2021-03-10T13:44:26.831 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-10T13:44:26.832 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-10T13:44:26.835 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-10T13:44:26.835 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.836 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 348 Links: 1 2021-03-10T13:44:26.836 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.836 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-03-10 13:41:42.696321645 +0000 2021-03-10T13:44:26.836 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-03-10 13:41:42.608321645 +0000 2021-03-10T13:44:26.836 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-03-10 13:41:42.608321645 +0000 2021-03-10T13:44:26.837 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-03-10T13:44:26.838 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-03-10T13:44:26.838 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-03-10T13:44:26.838 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000267508 s, 1.9 MB/s 2021-03-10T13:44:26.839 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-10T13:44:26.885 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-10T13:44:26.885 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.885 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 356 Links: 1 2021-03-10T13:44:26.886 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.886 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-03-10 13:41:43.100321641 +0000 2021-03-10T13:44:26.886 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-03-10 13:41:42.988321642 +0000 2021-03-10T13:44:26.886 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-03-10 13:41:42.988321642 +0000 2021-03-10T13:44:26.887 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-03-10T13:44:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-03-10T13:44:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-03-10T13:44:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000286357 s, 1.8 MB/s 2021-03-10T13:44:26.893 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-10T13:44:26.899 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-10T13:44:26.900 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.900 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 362 Links: 1 2021-03-10T13:44:26.900 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.900 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-03-10 13:41:43.496321637 +0000 2021-03-10T13:44:26.900 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-03-10 13:41:43.356321638 +0000 2021-03-10T13:44:26.901 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-03-10 13:41:43.356321638 +0000 2021-03-10T13:44:26.901 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-03-10T13:44:26.906 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-03-10T13:44:26.906 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-03-10T13:44:26.907 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000235823 s, 2.2 MB/s 2021-03-10T13:44:26.908 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-10T13:44:26.910 INFO:journalctl@ceph.mgr.x.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:44:26.911 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-10T13:44:26.911 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.912 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 368 Links: 1 2021-03-10T13:44:26.912 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.912 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-03-10 13:41:43.856321634 +0000 2021-03-10T13:44:26.912 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-03-10 13:41:43.684321635 +0000 2021-03-10T13:44:26.913 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-03-10 13:41:43.684321635 +0000 2021-03-10T13:44:26.913 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-03-10T13:44:26.914 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-03-10T13:44:26.914 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-03-10T13:44:26.914 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.00016728 s, 3.1 MB/s 2021-03-10T13:44:26.916 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:44:26.916 DEBUG:teuthology.orchestra.run.gibba032:> dd if=/scratch_devs of=/dev/stdout 2021-03-10T13:44:26.923 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-10T13:44:26.923 DEBUG:teuthology.orchestra.run.gibba032:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-10T13:44:26.933 INFO:teuthology.orchestra.run.gibba032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-10T13:44:26.933 INFO:teuthology.orchestra.run.gibba032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.933 INFO:teuthology.orchestra.run.gibba032.stdout:Device: 6h/6d Inode: 346 Links: 1 2021-03-10T13:44:26.934 INFO:teuthology.orchestra.run.gibba032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.934 INFO:teuthology.orchestra.run.gibba032.stdout:Access: 2021-03-10 13:41:42.868089926 +0000 2021-03-10T13:44:26.934 INFO:teuthology.orchestra.run.gibba032.stdout:Modify: 2021-03-10 13:41:42.780089928 +0000 2021-03-10T13:44:26.934 INFO:teuthology.orchestra.run.gibba032.stdout:Change: 2021-03-10 13:41:42.780089928 +0000 2021-03-10T13:44:26.934 INFO:teuthology.orchestra.run.gibba032.stdout: Birth: - 2021-03-10T13:44:26.941 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records in 2021-03-10T13:44:26.941 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records out 2021-03-10T13:44:26.941 INFO:teuthology.orchestra.run.gibba032.stderr:512 bytes copied, 0.000255814 s, 2.0 MB/s 2021-03-10T13:44:26.942 DEBUG:teuthology.orchestra.run.gibba032:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-10T13:44:26.945 INFO:teuthology.orchestra.run.gibba032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-10T13:44:26.946 INFO:teuthology.orchestra.run.gibba032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:26.946 INFO:teuthology.orchestra.run.gibba032.stdout:Device: 6h/6d Inode: 356 Links: 1 2021-03-10T13:44:26.946 INFO:teuthology.orchestra.run.gibba032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:26.946 INFO:teuthology.orchestra.run.gibba032.stdout:Access: 2021-03-10 13:41:43.260089916 +0000 2021-03-10T13:44:26.947 INFO:teuthology.orchestra.run.gibba032.stdout:Modify: 2021-03-10 13:41:43.188089918 +0000 2021-03-10T13:44:26.947 INFO:teuthology.orchestra.run.gibba032.stdout:Change: 2021-03-10 13:41:43.188089918 +0000 2021-03-10T13:44:26.947 INFO:teuthology.orchestra.run.gibba032.stdout: Birth: - 2021-03-10T13:44:26.948 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records in 2021-03-10T13:44:26.948 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records out 2021-03-10T13:44:26.948 INFO:teuthology.orchestra.run.gibba032.stderr:512 bytes copied, 0.000146115 s, 3.5 MB/s 2021-03-10T13:44:26.950 DEBUG:teuthology.orchestra.run.gibba032:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-10T13:44:27.001 INFO:teuthology.orchestra.run.gibba032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-10T13:44:27.001 INFO:teuthology.orchestra.run.gibba032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:27.002 INFO:teuthology.orchestra.run.gibba032.stdout:Device: 6h/6d Inode: 360 Links: 1 2021-03-10T13:44:27.002 INFO:teuthology.orchestra.run.gibba032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:27.002 INFO:teuthology.orchestra.run.gibba032.stdout:Access: 2021-03-10 13:41:43.644089906 +0000 2021-03-10T13:44:27.002 INFO:teuthology.orchestra.run.gibba032.stdout:Modify: 2021-03-10 13:41:43.528089909 +0000 2021-03-10T13:44:27.003 INFO:teuthology.orchestra.run.gibba032.stdout:Change: 2021-03-10 13:41:43.528089909 +0000 2021-03-10T13:44:27.003 INFO:teuthology.orchestra.run.gibba032.stdout: Birth: - 2021-03-10T13:44:27.008 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records in 2021-03-10T13:44:27.008 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records out 2021-03-10T13:44:27.008 INFO:teuthology.orchestra.run.gibba032.stderr:512 bytes copied, 0.000158541 s, 3.2 MB/s 2021-03-10T13:44:27.010 DEBUG:teuthology.orchestra.run.gibba032:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-10T13:44:27.013 INFO:teuthology.orchestra.run.gibba032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-10T13:44:27.013 INFO:teuthology.orchestra.run.gibba032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-10T13:44:27.014 INFO:teuthology.orchestra.run.gibba032.stdout:Device: 6h/6d Inode: 367 Links: 1 2021-03-10T13:44:27.014 INFO:teuthology.orchestra.run.gibba032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-10T13:44:27.014 INFO:teuthology.orchestra.run.gibba032.stdout:Access: 2021-03-10 13:41:44.020089896 +0000 2021-03-10T13:44:27.014 INFO:teuthology.orchestra.run.gibba032.stdout:Modify: 2021-03-10 13:41:43.896089899 +0000 2021-03-10T13:44:27.014 INFO:teuthology.orchestra.run.gibba032.stdout:Change: 2021-03-10 13:41:43.896089899 +0000 2021-03-10T13:44:27.015 INFO:teuthology.orchestra.run.gibba032.stdout: Birth: - 2021-03-10T13:44:27.016 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records in 2021-03-10T13:44:27.016 INFO:teuthology.orchestra.run.gibba032.stderr:1+0 records out 2021-03-10T13:44:27.016 INFO:teuthology.orchestra.run.gibba032.stderr:512 bytes copied, 0.000157867 s, 3.2 MB/s 2021-03-10T13:44:27.017 INFO:tasks.cephadm:Deploying osd.0 on gibba011 with /dev/vg_nvme/lv_4... 2021-03-10T13:44:27.018 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2021-03-10T13:44:27.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[9813]: audit 2021-03-10T13:44:25.710998+0000 mgr.y (mgr.14140) 35 : audit [DBG] from='client.34103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;gibba011=y;gibba032=x", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:27.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[9813]: cephadm 2021-03-10T13:44:25.712540+0000 mgr.y (mgr.14140) 36 : cephadm [INF] Saving service mgr spec with placement count:3 gibba011=y,gibba032=x 2021-03-10T13:44:27.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[9813]: cephadm 2021-03-10T13:44:25.726777+0000 mgr.y (mgr.14140) 37 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:44:27.112 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[13015]: audit 2021-03-10T13:44:25.710998+0000 mgr.y (mgr.14140) 35 : audit [DBG] from='client.34103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;gibba011=y;gibba032=x", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:27.112 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[13015]: cephadm 2021-03-10T13:44:25.712540+0000 mgr.y (mgr.14140) 36 : cephadm [INF] Saving service mgr spec with placement count:3 gibba011=y,gibba032=x 2021-03-10T13:44:27.112 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:27 gibba011 bash[13015]: cephadm 2021-03-10T13:44:25.726777+0000 mgr.y (mgr.14140) 37 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:44:27.326 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:27 gibba032 bash[9829]: audit 2021-03-10T13:44:25.710998+0000 mgr.y (mgr.14140) 35 : audit [DBG] from='client.34103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;gibba011=y;gibba032=x", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:27.327 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:27 gibba032 bash[9829]: cephadm 2021-03-10T13:44:25.712540+0000 mgr.y (mgr.14140) 36 : cephadm [INF] Saving service mgr spec with placement count:3 gibba011=y,gibba032=x 2021-03-10T13:44:27.327 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:27 gibba032 bash[9829]: cephadm 2021-03-10T13:44:25.726777+0000 mgr.y (mgr.14140) 37 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:44:28.134 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:28 gibba032 bash[9829]: cluster 2021-03-10T13:44:27.060359+0000 mgr.y (mgr.14140) 38 : cluster [DBG] pgmap v29: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:28.135 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:44:28 gibba032 systemd[1]: Starting Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:44:28.202 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:28 gibba011 bash[9813]: cluster 2021-03-10T13:44:27.060359+0000 mgr.y (mgr.14140) 38 : cluster [DBG] pgmap v29: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:28.203 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:28 gibba011 bash[13015]: cluster 2021-03-10T13:44:27.060359+0000 mgr.y (mgr.14140) 38 : cluster [DBG] pgmap v29: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:28.506 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:44:28 gibba032 docker[10437]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mgr.x 2021-03-10T13:44:28.507 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:44:28 gibba032 systemd[1]: Started Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:44:29.336 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.150065+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:29.337 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.157819+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:29.337 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.159669+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:29.337 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.168110+0000 mon.a (mon.0) 163 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:29.337 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.170622+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:29.338 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: cephadm 2021-03-10T13:44:28.172714+0000 mgr.y (mgr.14140) 39 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:44:29.338 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[9829]: audit 2021-03-10T13:44:28.174104+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:44:29.369 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.150065+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:29.370 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.150065+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:29.370 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.157819+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:29.370 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.159669+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:29.371 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.168110+0000 mon.a (mon.0) 163 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:29.371 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.170622+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:29.371 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: cephadm 2021-03-10T13:44:28.172714+0000 mgr.y (mgr.14140) 39 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:44:29.371 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[9813]: audit 2021-03-10T13:44:28.174104+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:44:29.372 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.157819+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:29.372 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.159669+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:44:29.372 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.168110+0000 mon.a (mon.0) 163 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-03-10T13:44:29.372 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.170622+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:29.372 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: cephadm 2021-03-10T13:44:28.172714+0000 mgr.y (mgr.14140) 39 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:44:29.373 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:29 gibba011 bash[13015]: audit 2021-03-10T13:44:28.174104+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:44:29.756 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[10454]: debug 2021-03-10T13:44:29.329+0000 7f8697e9c040 1 -- 172.21.2.132:0/506448683 >> [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] conn(0x5587256bd000 msgr2=0x558726332480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-03-10T13:44:29.757 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:44:29 gibba032 bash[10454]: debug 2021-03-10T13:44:29.329+0000 7f8697e9c040 1 --2- 172.21.2.132:0/506448683 >> [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] conn(0x5587256bd000 0x558726332480 secure :-1 s=READY pgs=7 cs=0 l=1 rx=0x5587256af140 tx=0x5587256b17c0).stop 2021-03-10T13:44:29.980 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping: /dev/vg_nvme/lv_4 2021-03-10T13:44:29.980 INFO:teuthology.orchestra.run.gibba011.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2021-03-10T13:44:29.981 INFO:teuthology.orchestra.run.gibba011.stderr: stderr: 10+0 records in 2021-03-10T13:44:29.981 INFO:teuthology.orchestra.run.gibba011.stderr:10+0 records out 2021-03-10T13:44:29.981 INFO:teuthology.orchestra.run.gibba011.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0247248 s, 424 MB/s 2021-03-10T13:44:29.981 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping successful for: 2021-03-10T13:44:30.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:30 gibba032 bash[9829]: cluster 2021-03-10T13:44:29.061316+0000 mgr.y (mgr.14140) 40 : cluster [DBG] pgmap v30: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:30 gibba011 bash[9813]: cluster 2021-03-10T13:44:29.061316+0000 mgr.y (mgr.14140) 40 : cluster [DBG] pgmap v30: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:30 gibba011 bash[13015]: cluster 2021-03-10T13:44:29.061316+0000 mgr.y (mgr.14140) 40 : cluster [DBG] pgmap v30: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:30.629 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba011:vg_nvme/lv_4 2021-03-10T13:44:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.336826+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.344761+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.346848+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.781493+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.784976+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.786509+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2021-03-10T13:44:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.787842+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:31.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:31 gibba032 bash[9829]: audit 2021-03-10T13:44:30.789423+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2021-03-10T13:44:31.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.336826+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.336826+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.344761+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.346848+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.344761+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:11.897285\", \"created\": \"2021-03-10T13:44:11.645118\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.346848+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:31.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.781493+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.781493+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.784976+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.786509+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2021-03-10T13:44:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.787842+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[9813]: audit 2021-03-10T13:44:30.789423+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2021-03-10T13:44:31.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.784976+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780491\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780590\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:30.780646\", \"created\": \"2021-03-10T13:44:30.317118\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:30.780733\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:31.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.786509+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2021-03-10T13:44:31.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.787842+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:31.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:31 gibba011 bash[13015]: audit 2021-03-10T13:44:30.789423+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2021-03-10T13:44:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:32 gibba032 bash[9829]: cephadm 2021-03-10T13:44:30.786181+0000 mgr.y (mgr.14140) 41 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2021-03-10T13:44:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:32 gibba032 bash[9829]: cephadm 2021-03-10T13:44:30.788848+0000 mgr.y (mgr.14140) 42 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:44:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:32 gibba032 bash[9829]: cluster 2021-03-10T13:44:31.062229+0000 mgr.y (mgr.14140) 43 : cluster [DBG] pgmap v31: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[9813]: cephadm 2021-03-10T13:44:30.786181+0000 mgr.y (mgr.14140) 41 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2021-03-10T13:44:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[9813]: cephadm 2021-03-10T13:44:30.788848+0000 mgr.y (mgr.14140) 42 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:44:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[9813]: cluster 2021-03-10T13:44:31.062229+0000 mgr.y (mgr.14140) 43 : cluster [DBG] pgmap v31: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[13015]: cephadm 2021-03-10T13:44:30.786181+0000 mgr.y (mgr.14140) 41 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2021-03-10T13:44:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[13015]: cephadm 2021-03-10T13:44:30.788848+0000 mgr.y (mgr.14140) 42 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:44:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:32 gibba011 bash[13015]: cluster 2021-03-10T13:44:31.062229+0000 mgr.y (mgr.14140) 43 : cluster [DBG] pgmap v31: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:33.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:33 gibba032 bash[9829]: audit 2021-03-10T13:44:32.193615+0000 mon.b (mon.2) 2 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:33.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:33 gibba032 bash[9829]: audit 2021-03-10T13:44:32.194532+0000 mon.b (mon.2) 3 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:33 gibba032 bash[9829]: cluster 2021-03-10T13:44:32.194946+0000 mon.a (mon.0) 174 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:44:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[9813]: audit 2021-03-10T13:44:32.193615+0000 mon.b (mon.2) 2 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[9813]: audit 2021-03-10T13:44:32.194532+0000 mon.b (mon.2) 3 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[9813]: cluster 2021-03-10T13:44:32.194946+0000 mon.a (mon.0) 174 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:44:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[13015]: audit 2021-03-10T13:44:32.193615+0000 mon.b (mon.2) 2 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[13015]: audit 2021-03-10T13:44:32.194532+0000 mon.b (mon.2) 3 : audit [DBG] from='mgr.? 172.21.2.132:0/1583580272' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:33.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:33 gibba011 bash[13015]: cluster 2021-03-10T13:44:32.194946+0000 mon.a (mon.0) 174 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:44:34.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: cluster 2021-03-10T13:44:33.063005+0000 mgr.y (mgr.14140) 44 : cluster [DBG] pgmap v32: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:34.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: cluster 2021-03-10T13:44:33.191963+0000 mon.a (mon.0) 175 : cluster [DBG] mgrmap e14: y(active, since 76s), standbys: x 2021-03-10T13:44:34.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.192055+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:44:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.198288+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: cluster 2021-03-10T13:44:33.063005+0000 mgr.y (mgr.14140) 44 : cluster [DBG] pgmap v32: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:34.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: cluster 2021-03-10T13:44:33.191963+0000 mon.a (mon.0) 175 : cluster [DBG] mgrmap e14: y(active, since 76s), standbys: x 2021-03-10T13:44:34.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.192055+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:44:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.198288+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.200882+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.201643+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.200882+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.201643+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.666679+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.666679+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.669256+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.669706+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.976368+0000 mon.b (mon.2) 4 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:33.977303+0000 mon.b (mon.2) 5 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.669256+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.669706+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.976368+0000 mon.b (mon.2) 4 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:34.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: cluster 2021-03-10T13:44:33.978320+0000 mon.a (mon.0) 183 : cluster [DBG] Standby manager daemon gibba032.whopeo started 2021-03-10T13:44:34.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:33.977303+0000 mon.b (mon.2) 5 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:34.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: cluster 2021-03-10T13:44:33.978320+0000 mon.a (mon.0) 183 : cluster [DBG] Standby manager daemon gibba032.whopeo started 2021-03-10T13:44:34.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:34.004799+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:34.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[9813]: audit 2021-03-10T13:44:34.012990+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:34.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:34.004799+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:34.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:34 gibba011 bash[13015]: audit 2021-03-10T13:44:34.012990+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: cluster 2021-03-10T13:44:33.063005+0000 mgr.y (mgr.14140) 44 : cluster [DBG] pgmap v32: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:34.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: cluster 2021-03-10T13:44:33.191963+0000 mon.a (mon.0) 175 : cluster [DBG] mgrmap e14: y(active, since 76s), standbys: x 2021-03-10T13:44:34.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.192055+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:44:34.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.198288+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.200882+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942126\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:43:58.942220\", \"created\": \"2021-03-10T13:43:58.520406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.201643+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.666679+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.669256+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}}, \"last_daemon_update\": \"2021-03-10T13:44:33.666237\", \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.669706+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.976368+0000 mon.b (mon.2) 4 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:44:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:33.977303+0000 mon.b (mon.2) 5 : audit [DBG] from='mgr.? 172.21.2.132:0/231087624' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:44:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: cluster 2021-03-10T13:44:33.978320+0000 mon.a (mon.0) 183 : cluster [DBG] Standby manager daemon gibba032.whopeo started 2021-03-10T13:44:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:34.004799+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:44:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:34 gibba032 bash[9829]: audit 2021-03-10T13:44:34.012990+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}}, \"last_daemon_update\": \"2021-03-10T13:44:34.002866\", \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:44:35.428 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: audit 2021-03-10T13:44:34.316784+0000 mgr.y (mgr.14140) 45 : audit [DBG] from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:35.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: cephadm 2021-03-10T13:44:34.317355+0000 mgr.y (mgr.14140) 46 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:35.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: audit 2021-03-10T13:44:34.317587+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:35.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: audit 2021-03-10T13:44:34.318460+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:35.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: audit 2021-03-10T13:44:34.319328+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:35.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: cluster 2021-03-10T13:44:34.682863+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e15: y(active, since 78s), standbys: x, gibba032.whopeo 2021-03-10T13:44:35.430 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[9813]: audit 2021-03-10T13:44:34.682968+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:44:35.430 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: audit 2021-03-10T13:44:34.316784+0000 mgr.y (mgr.14140) 45 : audit [DBG] from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:35.430 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: cephadm 2021-03-10T13:44:34.317355+0000 mgr.y (mgr.14140) 46 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:35.430 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: audit 2021-03-10T13:44:34.317587+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:35.431 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: audit 2021-03-10T13:44:34.318460+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:35.431 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: audit 2021-03-10T13:44:34.319328+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:35.431 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: cluster 2021-03-10T13:44:34.682863+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e15: y(active, since 78s), standbys: x, gibba032.whopeo 2021-03-10T13:44:35.431 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:35 gibba011 bash[13015]: audit 2021-03-10T13:44:34.682968+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:44:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: audit 2021-03-10T13:44:34.316784+0000 mgr.y (mgr.14140) 45 : audit [DBG] from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: cephadm 2021-03-10T13:44:34.317355+0000 mgr.y (mgr.14140) 46 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: audit 2021-03-10T13:44:34.317587+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: audit 2021-03-10T13:44:34.318460+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: audit 2021-03-10T13:44:34.319328+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: cluster 2021-03-10T13:44:34.682863+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e15: y(active, since 78s), standbys: x, gibba032.whopeo 2021-03-10T13:44:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:35 gibba032 bash[9829]: audit 2021-03-10T13:44:34.682968+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:44:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:36 gibba032 bash[9829]: cluster 2021-03-10T13:44:35.005271+0000 mgr.gibba032.whopeo (mgr.34115) 1 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:44:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:36 gibba032 bash[9829]: cluster 2021-03-10T13:44:35.063646+0000 mgr.y (mgr.14140) 47 : cluster [DBG] pgmap v33: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:36.528 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:36 gibba011 bash[9813]: cluster 2021-03-10T13:44:35.005271+0000 mgr.gibba032.whopeo (mgr.34115) 1 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:44:36.528 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:36 gibba011 bash[9813]: cluster 2021-03-10T13:44:35.063646+0000 mgr.y (mgr.14140) 47 : cluster [DBG] pgmap v33: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:36.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:36 gibba011 bash[13015]: cluster 2021-03-10T13:44:35.005271+0000 mgr.gibba032.whopeo (mgr.34115) 1 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:44:36.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:36 gibba011 bash[13015]: cluster 2021-03-10T13:44:35.063646+0000 mgr.y (mgr.14140) 47 : cluster [DBG] pgmap v33: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:37 gibba032 bash[9829]: audit 2021-03-10T13:44:37.189420+0000 mon.c (mon.1) 3 : audit [INF] from='client.? 172.21.2.111:0/996012095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:37 gibba032 bash[9829]: audit 2021-03-10T13:44:37.189744+0000 mon.a (mon.0) 191 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:37 gibba032 bash[9829]: audit 2021-03-10T13:44:37.194808+0000 mon.a (mon.0) 192 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]': finished 2021-03-10T13:44:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:37 gibba032 bash[9829]: cluster 2021-03-10T13:44:37.194836+0000 mon.a (mon.0) 193 : cluster [DBG] osdmap e7: 1 total, 0 up, 0 in 2021-03-10T13:44:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:37 gibba032 bash[9829]: audit 2021-03-10T13:44:37.194936+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:37.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[9813]: audit 2021-03-10T13:44:37.189420+0000 mon.c (mon.1) 3 : audit [INF] from='client.? 172.21.2.111:0/996012095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[9813]: audit 2021-03-10T13:44:37.189744+0000 mon.a (mon.0) 191 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[9813]: audit 2021-03-10T13:44:37.194808+0000 mon.a (mon.0) 192 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]': finished 2021-03-10T13:44:37.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[9813]: cluster 2021-03-10T13:44:37.194836+0000 mon.a (mon.0) 193 : cluster [DBG] osdmap e7: 1 total, 0 up, 0 in 2021-03-10T13:44:37.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[9813]: audit 2021-03-10T13:44:37.194936+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:37.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[13015]: audit 2021-03-10T13:44:37.189420+0000 mon.c (mon.1) 3 : audit [INF] from='client.? 172.21.2.111:0/996012095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[13015]: audit 2021-03-10T13:44:37.189744+0000 mon.a (mon.0) 191 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]: dispatch 2021-03-10T13:44:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[13015]: audit 2021-03-10T13:44:37.194808+0000 mon.a (mon.0) 192 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924baa6c-3f7e-44d7-a126-60ca7e5787ad"}]': finished 2021-03-10T13:44:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[13015]: cluster 2021-03-10T13:44:37.194836+0000 mon.a (mon.0) 193 : cluster [DBG] osdmap e7: 1 total, 0 up, 0 in 2021-03-10T13:44:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:37 gibba011 bash[13015]: audit 2021-03-10T13:44:37.194936+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:38 gibba032 bash[9829]: cluster 2021-03-10T13:44:37.063954+0000 mgr.y (mgr.14140) 48 : cluster [DBG] pgmap v34: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:38 gibba011 bash[9813]: cluster 2021-03-10T13:44:37.063954+0000 mgr.y (mgr.14140) 48 : cluster [DBG] pgmap v34: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:38 gibba011 bash[13015]: cluster 2021-03-10T13:44:37.063954+0000 mgr.y (mgr.14140) 48 : cluster [DBG] pgmap v34: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:39.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:39 gibba032 bash[9829]: audit 2021-03-10T13:44:39.079484+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3125079972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:44:39.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:39 gibba011 bash[9813]: audit 2021-03-10T13:44:39.079484+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3125079972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:44:39.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:39 gibba011 bash[13015]: audit 2021-03-10T13:44:39.079484+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3125079972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:44:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:40 gibba032 bash[9829]: cluster 2021-03-10T13:44:39.064152+0000 mgr.y (mgr.14140) 49 : cluster [DBG] pgmap v36: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:40 gibba011 bash[9813]: cluster 2021-03-10T13:44:39.064152+0000 mgr.y (mgr.14140) 49 : cluster [DBG] pgmap v36: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:40.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:40 gibba011 bash[13015]: cluster 2021-03-10T13:44:39.064152+0000 mgr.y (mgr.14140) 49 : cluster [DBG] pgmap v36: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:42.560 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:42 gibba011 bash[9813]: cluster 2021-03-10T13:44:41.064682+0000 mgr.y (mgr.14140) 50 : cluster [DBG] pgmap v37: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:42.561 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:42 gibba011 bash[9813]: audit 2021-03-10T13:44:41.706799+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:42.561 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:42 gibba011 bash[13015]: cluster 2021-03-10T13:44:41.064682+0000 mgr.y (mgr.14140) 50 : cluster [DBG] pgmap v37: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:42.561 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:42 gibba011 bash[13015]: audit 2021-03-10T13:44:41.706799+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:42 gibba032 bash[9829]: cluster 2021-03-10T13:44:41.064682+0000 mgr.y (mgr.14140) 50 : cluster [DBG] pgmap v37: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:42 gibba032 bash[9829]: audit 2021-03-10T13:44:41.706799+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:44 gibba011 bash[9813]: cluster 2021-03-10T13:44:43.065356+0000 mgr.y (mgr.14140) 51 : cluster [DBG] pgmap v38: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:44 gibba011 bash[13015]: cluster 2021-03-10T13:44:43.065356+0000 mgr.y (mgr.14140) 51 : cluster [DBG] pgmap v38: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:44 gibba032 bash[9829]: cluster 2021-03-10T13:44:43.065356+0000 mgr.y (mgr.14140) 51 : cluster [DBG] pgmap v38: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:45.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[9813]: audit 2021-03-10T13:44:44.835093+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2021-03-10T13:44:45.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[9813]: audit 2021-03-10T13:44:44.835732+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:45.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[9813]: audit 2021-03-10T13:44:44.836461+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2021-03-10T13:44:45.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[13015]: audit 2021-03-10T13:44:44.835093+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2021-03-10T13:44:45.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[13015]: audit 2021-03-10T13:44:44.835732+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:45.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:45 gibba011 bash[13015]: audit 2021-03-10T13:44:44.836461+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2021-03-10T13:44:45.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:45 gibba032 bash[9829]: audit 2021-03-10T13:44:44.835093+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2021-03-10T13:44:45.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:45 gibba032 bash[9829]: audit 2021-03-10T13:44:44.835732+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:45.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:45 gibba032 bash[9829]: audit 2021-03-10T13:44:44.836461+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2021-03-10T13:44:46.536 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:46 gibba011 bash[9813]: cephadm 2021-03-10T13:44:44.836296+0000 mgr.y (mgr.14140) 52 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:44:46.536 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:46 gibba011 bash[9813]: cluster 2021-03-10T13:44:45.065693+0000 mgr.y (mgr.14140) 53 : cluster [DBG] pgmap v39: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:46.536 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:46 gibba011 bash[13015]: cephadm 2021-03-10T13:44:44.836296+0000 mgr.y (mgr.14140) 52 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:44:46.537 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:46 gibba011 bash[13015]: cluster 2021-03-10T13:44:45.065693+0000 mgr.y (mgr.14140) 53 : cluster [DBG] pgmap v39: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:46.570 INFO:teuthology.orchestra.run.gibba011.stdout:Created osd(s) 0 on host 'gibba011' 2021-03-10T13:44:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:46 gibba032 bash[9829]: cephadm 2021-03-10T13:44:44.836296+0000 mgr.y (mgr.14140) 52 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:44:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:46 gibba032 bash[9829]: cluster 2021-03-10T13:44:45.065693+0000 mgr.y (mgr.14140) 53 : cluster [DBG] pgmap v39: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:47.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:46.568628+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:46.569186+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:46.568628+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:46.569186+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:46.572546+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:46.572546+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:47.050341+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:47.054507+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[9813]: audit 2021-03-10T13:44:47.055035+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:47.050341+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:47.054507+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:47 gibba011 bash[13015]: audit 2021-03-10T13:44:47.055035+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:46.568628+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:46.569186+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:46.572546+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666093\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666152\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:33.666181\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_device_update\": \"2021-03-10T13:43:23.529911\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:47.050341+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:47.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:47.054507+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:47.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:47 gibba032 bash[9829]: audit 2021-03-10T13:44:47.055035+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:47.839 DEBUG:teuthology.orchestra.run.gibba011:osd.0> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.0.service 2021-03-10T13:44:47.841 INFO:tasks.cephadm:Deploying osd.1 on gibba011 with /dev/vg_nvme/lv_3... 2021-03-10T13:44:47.842 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2021-03-10T13:44:47.953 INFO:journalctl@ceph.osd.0.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:44:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[9813]: cluster 2021-03-10T13:44:47.065887+0000 mgr.y (mgr.14140) 54 : cluster [DBG] pgmap v40: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[13015]: cluster 2021-03-10T13:44:47.065887+0000 mgr.y (mgr.14140) 54 : cluster [DBG] pgmap v40: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:48 gibba032 bash[9829]: cluster 2021-03-10T13:44:47.065887+0000 mgr.y (mgr.14140) 54 : cluster [DBG] pgmap v40: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:49.247 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-03-10T13:44:49.247 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2021-03-10T13:44:49.247 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2021-03-10T13:44:49.247 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2021-03-10T13:44:49.248 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-03-10T13:44:49.248 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-03-10T13:44:49.248 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:48 gibba011 bash[14634]: --> ceph-volume lvm activate successful for osd ID: 0 2021-03-10T13:44:50.611 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:50 gibba011 bash[9813]: cluster 2021-03-10T13:44:49.066410+0000 mgr.y (mgr.14140) 55 : cluster [DBG] pgmap v41: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:50.611 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:50 gibba011 bash[13015]: cluster 2021-03-10T13:44:49.066410+0000 mgr.y (mgr.14140) 55 : cluster [DBG] pgmap v41: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:50 gibba032 bash[9829]: cluster 2021-03-10T13:44:49.066410+0000 mgr.y (mgr.14140) 55 : cluster [DBG] pgmap v41: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:52 gibba011 bash[9813]: cluster 2021-03-10T13:44:51.067187+0000 mgr.y (mgr.14140) 56 : cluster [DBG] pgmap v42: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:52 gibba011 bash[13015]: cluster 2021-03-10T13:44:51.067187+0000 mgr.y (mgr.14140) 56 : cluster [DBG] pgmap v42: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:52 gibba032 bash[9829]: cluster 2021-03-10T13:44:51.067187+0000 mgr.y (mgr.14140) 56 : cluster [DBG] pgmap v42: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:53.050 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:52 gibba011 bash[14938]: debug 2021-03-10T13:44:52.752+0000 7fb65220aec0 -1 Falling back to public interface 2021-03-10T13:44:53.511 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping: /dev/vg_nvme/lv_3 2021-03-10T13:44:53.512 INFO:teuthology.orchestra.run.gibba011.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2021-03-10T13:44:53.512 INFO:teuthology.orchestra.run.gibba011.stderr: stderr: 10+0 records in 2021-03-10T13:44:53.512 INFO:teuthology.orchestra.run.gibba011.stderr:10+0 records out 2021-03-10T13:44:53.512 INFO:teuthology.orchestra.run.gibba011.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00918223 s, 1.1 GB/s 2021-03-10T13:44:53.512 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping successful for: 2021-03-10T13:44:53.710 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:53 gibba011 bash[14938]: debug 2021-03-10T13:44:53.432+0000 7fb65220aec0 -1 osd.0 0 log_to_monitors {default=true} 2021-03-10T13:44:54.255 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba011:vg_nvme/lv_3 2021-03-10T13:44:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:54 gibba011 bash[9813]: cluster 2021-03-10T13:44:53.067444+0000 mgr.y (mgr.14140) 57 : cluster [DBG] pgmap v43: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:54 gibba011 bash[9813]: audit 2021-03-10T13:44:53.437697+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:44:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:54 gibba011 bash[13015]: cluster 2021-03-10T13:44:53.067444+0000 mgr.y (mgr.14140) 57 : cluster [DBG] pgmap v43: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:54 gibba011 bash[13015]: audit 2021-03-10T13:44:53.437697+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:44:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:54 gibba032 bash[9829]: cluster 2021-03-10T13:44:53.067444+0000 mgr.y (mgr.14140) 57 : cluster [DBG] pgmap v43: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:54 gibba032 bash[9829]: audit 2021-03-10T13:44:53.437697+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:44:55.577 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[14938]: debug 2021-03-10T13:44:55.320+0000 7fb63afbf700 -1 osd.0 0 waiting for initial osdmap 2021-03-10T13:44:55.578 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[14938]: debug 2021-03-10T13:44:55.320+0000 7fb6461db700 -1 osd.0 9 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:44:55.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:54.311991+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:44:55.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: cluster 2021-03-10T13:44:54.312044+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2021-03-10T13:44:55.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:54.312793+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:55.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:54.313095+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:44:55.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:55.017734+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:54.311991+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:44:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: cluster 2021-03-10T13:44:54.312044+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2021-03-10T13:44:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:54.312793+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:54.313095+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:44:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:55.017734+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:55.116588+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:55.116588+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:55.120221+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:55.320888+0000 mon.a (mon.0) 213 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:44:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: cluster 2021-03-10T13:44:55.320945+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2021-03-10T13:44:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[9813]: audit 2021-03-10T13:44:55.321074+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:55.120221+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:55.320888+0000 mon.a (mon.0) 213 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:44:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: cluster 2021-03-10T13:44:55.320945+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2021-03-10T13:44:55.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:55 gibba011 bash[13015]: audit 2021-03-10T13:44:55.321074+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:55.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:54.311991+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:44:55.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: cluster 2021-03-10T13:44:54.312044+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2021-03-10T13:44:55.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:54.312793+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:55.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:54.313095+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:44:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:55.017734+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:44:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:55.116588+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:44:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:55.120221+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}}, \"last_daemon_update\": \"2021-03-10T13:44:47.049808\", \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:44:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:55.320888+0000 mon.a (mon.0) 213 : audit [INF] from='osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:44:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: cluster 2021-03-10T13:44:55.320945+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2021-03-10T13:44:55.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:55 gibba032 bash[9829]: audit 2021-03-10T13:44:55.321074+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:56 gibba032 bash[9829]: cluster 2021-03-10T13:44:54.458502+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:44:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:56 gibba032 bash[9829]: cluster 2021-03-10T13:44:54.458600+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:44:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:56 gibba032 bash[9829]: cluster 2021-03-10T13:44:55.067619+0000 mgr.y (mgr.14140) 58 : cluster [DBG] pgmap v45: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:56.768 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[9813]: cluster 2021-03-10T13:44:54.458502+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:44:56.768 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[9813]: cluster 2021-03-10T13:44:54.458600+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:44:56.769 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[9813]: cluster 2021-03-10T13:44:55.067619+0000 mgr.y (mgr.14140) 58 : cluster [DBG] pgmap v45: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:56.769 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[13015]: cluster 2021-03-10T13:44:54.458502+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:44:56.769 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[13015]: cluster 2021-03-10T13:44:54.458600+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:44:56.770 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:56 gibba011 bash[13015]: cluster 2021-03-10T13:44:55.067619+0000 mgr.y (mgr.14140) 58 : cluster [DBG] pgmap v45: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: cluster 2021-03-10T13:44:56.330467+0000 mon.a (mon.0) 216 : cluster [INF] osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926] boot 2021-03-10T13:44:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: cluster 2021-03-10T13:44:56.330502+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2021-03-10T13:44:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: audit 2021-03-10T13:44:56.331262+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: audit 2021-03-10T13:44:57.025977+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: audit 2021-03-10T13:44:57.026839+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[9813]: audit 2021-03-10T13:44:57.027860+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: cluster 2021-03-10T13:44:56.330467+0000 mon.a (mon.0) 216 : cluster [INF] osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926] boot 2021-03-10T13:44:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: cluster 2021-03-10T13:44:56.330502+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2021-03-10T13:44:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: audit 2021-03-10T13:44:56.331262+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: audit 2021-03-10T13:44:57.025977+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: audit 2021-03-10T13:44:57.026839+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:57 gibba011 bash[13015]: audit 2021-03-10T13:44:57.027860+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: cluster 2021-03-10T13:44:56.330467+0000 mon.a (mon.0) 216 : cluster [INF] osd.0 [v2:172.21.2.111:6802/388310926,v1:172.21.2.111:6803/388310926] boot 2021-03-10T13:44:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: cluster 2021-03-10T13:44:56.330502+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2021-03-10T13:44:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: audit 2021-03-10T13:44:56.331262+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:44:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: audit 2021-03-10T13:44:57.025977+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:44:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: audit 2021-03-10T13:44:57.026839+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:44:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:57 gibba032 bash[9829]: audit 2021-03-10T13:44:57.027860+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[9813]: audit 2021-03-10T13:44:57.025287+0000 mgr.y (mgr.14140) 59 : audit [DBG] from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[9813]: cephadm 2021-03-10T13:44:57.025735+0000 mgr.y (mgr.14140) 60 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[9813]: cluster 2021-03-10T13:44:57.067841+0000 mgr.y (mgr.14140) 61 : cluster [DBG] pgmap v48: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[9813]: cluster 2021-03-10T13:44:57.339354+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2021-03-10T13:44:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[9813]: cluster 2021-03-10T13:44:58.343164+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2021-03-10T13:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[13015]: audit 2021-03-10T13:44:57.025287+0000 mgr.y (mgr.14140) 59 : audit [DBG] from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[13015]: cephadm 2021-03-10T13:44:57.025735+0000 mgr.y (mgr.14140) 60 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[13015]: cluster 2021-03-10T13:44:57.067841+0000 mgr.y (mgr.14140) 61 : cluster [DBG] pgmap v48: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[13015]: cluster 2021-03-10T13:44:57.339354+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2021-03-10T13:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:58 gibba011 bash[13015]: cluster 2021-03-10T13:44:58.343164+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2021-03-10T13:44:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:58 gibba032 bash[9829]: audit 2021-03-10T13:44:57.025287+0000 mgr.y (mgr.14140) 59 : audit [DBG] from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:44:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:58 gibba032 bash[9829]: cephadm 2021-03-10T13:44:57.025735+0000 mgr.y (mgr.14140) 60 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:44:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:58 gibba032 bash[9829]: cluster 2021-03-10T13:44:57.067841+0000 mgr.y (mgr.14140) 61 : cluster [DBG] pgmap v48: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2021-03-10T13:44:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:58 gibba032 bash[9829]: cluster 2021-03-10T13:44:57.339354+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2021-03-10T13:44:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:58 gibba032 bash[9829]: cluster 2021-03-10T13:44:58.343164+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2021-03-10T13:44:59.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:44:59 gibba032 bash[9829]: cluster 2021-03-10T13:44:59.350298+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:44:59.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:44:59 gibba011 bash[9813]: cluster 2021-03-10T13:44:59.350298+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:44:59.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:44:59 gibba011 bash[13015]: cluster 2021-03-10T13:44:59.350298+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:45:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:00 gibba032 bash[9829]: cluster 2021-03-10T13:44:59.068278+0000 mgr.y (mgr.14140) 62 : cluster [DBG] pgmap v51: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:00 gibba032 bash[9829]: audit 2021-03-10T13:44:59.993224+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]: dispatch 2021-03-10T13:45:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:00 gibba032 bash[9829]: audit 2021-03-10T13:44:59.998136+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]': finished 2021-03-10T13:45:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:00 gibba032 bash[9829]: cluster 2021-03-10T13:44:59.998179+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e13: 2 total, 1 up, 1 in 2021-03-10T13:45:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:00 gibba032 bash[9829]: audit 2021-03-10T13:44:59.998267+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[9813]: cluster 2021-03-10T13:44:59.068278+0000 mgr.y (mgr.14140) 62 : cluster [DBG] pgmap v51: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[9813]: audit 2021-03-10T13:44:59.993224+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]: dispatch 2021-03-10T13:45:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[9813]: audit 2021-03-10T13:44:59.998136+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]': finished 2021-03-10T13:45:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[9813]: cluster 2021-03-10T13:44:59.998179+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e13: 2 total, 1 up, 1 in 2021-03-10T13:45:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[9813]: audit 2021-03-10T13:44:59.998267+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[13015]: cluster 2021-03-10T13:44:59.068278+0000 mgr.y (mgr.14140) 62 : cluster [DBG] pgmap v51: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[13015]: audit 2021-03-10T13:44:59.993224+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]: dispatch 2021-03-10T13:45:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[13015]: audit 2021-03-10T13:44:59.998136+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.2.111:0/1712676864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a81ef55-f29d-4fd7-81a8-384de101e141"}]': finished 2021-03-10T13:45:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[13015]: cluster 2021-03-10T13:44:59.998179+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e13: 2 total, 1 up, 1 in 2021-03-10T13:45:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:00 gibba011 bash[13015]: audit 2021-03-10T13:44:59.998267+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:02 gibba032 bash[9829]: cluster 2021-03-10T13:45:01.068636+0000 mgr.y (mgr.14140) 63 : cluster [DBG] pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:02 gibba032 bash[9829]: audit 2021-03-10T13:45:01.913486+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.2.111:0/2422143881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:02 gibba011 bash[9813]: cluster 2021-03-10T13:45:01.068636+0000 mgr.y (mgr.14140) 63 : cluster [DBG] pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:02 gibba011 bash[9813]: audit 2021-03-10T13:45:01.913486+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.2.111:0/2422143881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:02 gibba011 bash[13015]: cluster 2021-03-10T13:45:01.068636+0000 mgr.y (mgr.14140) 63 : cluster [DBG] pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 96 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:02 gibba011 bash[13015]: audit 2021-03-10T13:45:01.913486+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.2.111:0/2422143881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:04 gibba032 bash[9829]: cluster 2021-03-10T13:45:03.068996+0000 mgr.y (mgr.14140) 64 : cluster [DBG] pgmap v54: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:04 gibba011 bash[9813]: cluster 2021-03-10T13:45:03.068996+0000 mgr.y (mgr.14140) 64 : cluster [DBG] pgmap v54: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:04.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:04 gibba011 bash[13015]: cluster 2021-03-10T13:45:03.068996+0000 mgr.y (mgr.14140) 64 : cluster [DBG] pgmap v54: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:05.611 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:05 gibba011 bash[9813]: audit 2021-03-10T13:45:04.467886+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:05.611 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:05 gibba011 bash[9813]: cluster 2021-03-10T13:45:05.058792+0000 mon.a (mon.0) 231 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:45:05.612 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:05 gibba011 bash[13015]: audit 2021-03-10T13:45:04.467886+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:05.612 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:05 gibba011 bash[13015]: cluster 2021-03-10T13:45:05.058792+0000 mon.a (mon.0) 231 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:45:05.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:05 gibba032 bash[9829]: audit 2021-03-10T13:45:04.467886+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:05.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:05 gibba032 bash[9829]: cluster 2021-03-10T13:45:05.058792+0000 mon.a (mon.0) 231 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2021-03-10T13:45:06.712 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:06 gibba011 bash[13015]: cluster 2021-03-10T13:45:05.069254+0000 mgr.y (mgr.14140) 65 : cluster [DBG] pgmap v55: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:06.713 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:06 gibba011 bash[9813]: cluster 2021-03-10T13:45:05.069254+0000 mgr.y (mgr.14140) 65 : cluster [DBG] pgmap v55: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:06 gibba032 bash[9829]: cluster 2021-03-10T13:45:05.069254+0000 mgr.y (mgr.14140) 65 : cluster [DBG] pgmap v55: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:08 gibba032 bash[9829]: cluster 2021-03-10T13:45:07.069870+0000 mgr.y (mgr.14140) 66 : cluster [DBG] pgmap v56: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:08 gibba032 bash[9829]: audit 2021-03-10T13:45:07.755052+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2021-03-10T13:45:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:08 gibba032 bash[9829]: audit 2021-03-10T13:45:07.755929+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:08 gibba032 bash[9829]: audit 2021-03-10T13:45:07.756884+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2021-03-10T13:45:08.786 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[9813]: cluster 2021-03-10T13:45:07.069870+0000 mgr.y (mgr.14140) 66 : cluster [DBG] pgmap v56: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:08.786 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[9813]: audit 2021-03-10T13:45:07.755052+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2021-03-10T13:45:08.786 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[9813]: audit 2021-03-10T13:45:07.755929+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:08.787 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[9813]: audit 2021-03-10T13:45:07.756884+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2021-03-10T13:45:08.787 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[13015]: cluster 2021-03-10T13:45:07.069870+0000 mgr.y (mgr.14140) 66 : cluster [DBG] pgmap v56: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:08.787 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[13015]: audit 2021-03-10T13:45:07.755052+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2021-03-10T13:45:08.787 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[13015]: audit 2021-03-10T13:45:07.755929+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:08.788 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:08 gibba011 bash[13015]: audit 2021-03-10T13:45:07.756884+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2021-03-10T13:45:09.463 INFO:teuthology.orchestra.run.gibba011.stdout:Created osd(s) 1 on host 'gibba011' 2021-03-10T13:45:09.727 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:09 gibba011 bash[9813]: cephadm 2021-03-10T13:45:07.756650+0000 mgr.y (mgr.14140) 67 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:45:09.727 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:09 gibba011 bash[13015]: cephadm 2021-03-10T13:45:07.756650+0000 mgr.y (mgr.14140) 67 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:45:09.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:09 gibba032 bash[9829]: cephadm 2021-03-10T13:45:07.756650+0000 mgr.y (mgr.14140) 67 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:45:10.606 DEBUG:teuthology.orchestra.run.gibba011:osd.1> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.1.service 2021-03-10T13:45:10.608 INFO:tasks.cephadm:Deploying osd.2 on gibba011 with /dev/vg_nvme/lv_2... 2021-03-10T13:45:10.609 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2021-03-10T13:45:10.739 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: cluster 2021-03-10T13:45:09.070811+0000 mgr.y (mgr.14140) 68 : cluster [DBG] pgmap v57: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:10.739 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.455441+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.739 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.455610+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.740 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: cluster 2021-03-10T13:45:09.070811+0000 mgr.y (mgr.14140) 68 : cluster [DBG] pgmap v57: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:10.740 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.464699+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.741 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.455441+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.741 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.455610+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.742 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.993680+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.742 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.464699+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.743 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.997557+0000 mon.a (mon.0) 239 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.743 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[9813]: audit 2021-03-10T13:45:09.998107+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.744 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.993680+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.744 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.997557+0000 mon.a (mon.0) 239 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.744 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:10 gibba011 bash[13015]: audit 2021-03-10T13:45:09.998107+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: cluster 2021-03-10T13:45:09.070811+0000 mgr.y (mgr.14140) 68 : cluster [DBG] pgmap v57: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.455441+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.455610+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.464699+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049646\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049704\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049733\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:47.049760\", \"created\": \"2021-03-10T13:44:46.556406\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_device_update\": \"2021-03-10T13:44:55.116075\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.993680+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.997557+0000 mon.a (mon.0) 239 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:10 gibba032 bash[9829]: audit 2021-03-10T13:45:09.998107+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:10.822 INFO:journalctl@ceph.osd.1.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:45:11.905 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-03-10T13:45:11.905 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2021-03-10T13:45:11.906 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2021-03-10T13:45:11.906 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2021-03-10T13:45:11.906 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-03-10T13:45:11.907 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-03-10T13:45:11.907 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:11 gibba011 bash[16082]: --> ceph-volume lvm activate successful for osd ID: 1 2021-03-10T13:45:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:12 gibba032 bash[9829]: cluster 2021-03-10T13:45:11.071384+0000 mgr.y (mgr.14140) 69 : cluster [DBG] pgmap v58: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:12 gibba011 bash[9813]: cluster 2021-03-10T13:45:11.071384+0000 mgr.y (mgr.14140) 69 : cluster [DBG] pgmap v58: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:12.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:12 gibba011 bash[13015]: cluster 2021-03-10T13:45:11.071384+0000 mgr.y (mgr.14140) 69 : cluster [DBG] pgmap v58: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:14 gibba032 bash[9829]: cluster 2021-03-10T13:45:13.071986+0000 mgr.y (mgr.14140) 70 : cluster [DBG] pgmap v59: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:14.766 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:14 gibba011 bash[9813]: cluster 2021-03-10T13:45:13.071986+0000 mgr.y (mgr.14140) 70 : cluster [DBG] pgmap v59: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:14.767 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:14 gibba011 bash[13015]: cluster 2021-03-10T13:45:13.071986+0000 mgr.y (mgr.14140) 70 : cluster [DBG] pgmap v59: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:15.827 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:15 gibba011 bash[16394]: debug 2021-03-10T13:45:15.596+0000 7fe9a177bec0 -1 Falling back to public interface 2021-03-10T13:45:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping: /dev/vg_nvme/lv_2 2021-03-10T13:45:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2021-03-10T13:45:15.950 INFO:teuthology.orchestra.run.gibba011.stderr: stderr: 10+0 records in 2021-03-10T13:45:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:10+0 records out 2021-03-10T13:45:15.950 INFO:teuthology.orchestra.run.gibba011.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142435 s, 736 MB/s 2021-03-10T13:45:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping successful for: 2021-03-10T13:45:16.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:16 gibba011 bash[9813]: cluster 2021-03-10T13:45:15.072521+0000 mgr.y (mgr.14140) 71 : cluster [DBG] pgmap v60: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:16 gibba011 bash[13015]: cluster 2021-03-10T13:45:15.072521+0000 mgr.y (mgr.14140) 71 : cluster [DBG] pgmap v60: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:16.328 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:16 gibba011 bash[16394]: debug 2021-03-10T13:45:16.272+0000 7fe9a177bec0 -1 osd.1 0 log_to_monitors {default=true} 2021-03-10T13:45:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:16 gibba032 bash[9829]: cluster 2021-03-10T13:45:15.072521+0000 mgr.y (mgr.14140) 71 : cluster [DBG] pgmap v60: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:16.562 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba011:vg_nvme/lv_2 2021-03-10T13:45:17.186 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:17 gibba011 bash[9813]: audit 2021-03-10T13:45:16.280146+0000 mon.c (mon.1) 5 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:17.186 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:17 gibba011 bash[9813]: audit 2021-03-10T13:45:16.280546+0000 mon.a (mon.0) 241 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:17.187 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:17 gibba011 bash[13015]: audit 2021-03-10T13:45:16.280146+0000 mon.c (mon.1) 5 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:17.187 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:17 gibba011 bash[13015]: audit 2021-03-10T13:45:16.280546+0000 mon.a (mon.0) 241 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:17 gibba032 bash[9829]: audit 2021-03-10T13:45:16.280146+0000 mon.c (mon.1) 5 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:17 gibba032 bash[9829]: audit 2021-03-10T13:45:16.280546+0000 mon.a (mon.0) 241 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:45:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: cluster 2021-03-10T13:45:17.073162+0000 mgr.y (mgr.14140) 72 : cluster [DBG] pgmap v61: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.092508+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:45:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: cluster 2021-03-10T13:45:17.092542+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2021-03-10T13:45:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.092631+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.093216+0000 mon.c (mon.1) 6 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.093451+0000 mon.a (mon.0) 245 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: cluster 2021-03-10T13:45:17.263120+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: cluster 2021-03-10T13:45:17.263252+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.719876+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.817448+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:18 gibba032 bash[9829]: audit 2021-03-10T13:45:17.821973+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:18.577 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[16394]: debug 2021-03-10T13:45:18.108+0000 7fe986528700 -1 osd.1 0 waiting for initial osdmap 2021-03-10T13:45:18.577 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[16394]: debug 2021-03-10T13:45:18.112+0000 7fe991744700 -1 osd.1 15 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:45:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: cluster 2021-03-10T13:45:17.073162+0000 mgr.y (mgr.14140) 72 : cluster [DBG] pgmap v61: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.092508+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:45:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: cluster 2021-03-10T13:45:17.092542+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2021-03-10T13:45:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.092631+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.093216+0000 mon.c (mon.1) 6 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.093451+0000 mon.a (mon.0) 245 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: cluster 2021-03-10T13:45:17.263120+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: cluster 2021-03-10T13:45:17.263252+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.719876+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: cluster 2021-03-10T13:45:17.073162+0000 mgr.y (mgr.14140) 72 : cluster [DBG] pgmap v61: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.092508+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:45:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: cluster 2021-03-10T13:45:17.092542+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2021-03-10T13:45:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.092631+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.093216+0000 mon.c (mon.1) 6 : audit [INF] from='osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.093451+0000 mon.a (mon.0) 245 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: cluster 2021-03-10T13:45:17.263120+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: cluster 2021-03-10T13:45:17.263252+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.719876+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.817448+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:18.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[9813]: audit 2021-03-10T13:45:17.821973+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:18.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.817448+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:18.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:18 gibba011 bash[13015]: audit 2021-03-10T13:45:17.821973+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}}, \"last_daemon_update\": \"2021-03-10T13:45:09.993140\", \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:19.393 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[9813]: audit 2021-03-10T13:45:18.100298+0000 mon.a (mon.0) 249 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:19.394 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[9813]: cluster 2021-03-10T13:45:18.100393+0000 mon.a (mon.0) 250 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2021-03-10T13:45:19.394 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[9813]: audit 2021-03-10T13:45:18.101560+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:19.394 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[13015]: audit 2021-03-10T13:45:18.100298+0000 mon.a (mon.0) 249 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:19.395 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[13015]: cluster 2021-03-10T13:45:18.100393+0000 mon.a (mon.0) 250 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2021-03-10T13:45:19.395 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:19 gibba011 bash[13015]: audit 2021-03-10T13:45:18.101560+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:19 gibba032 bash[9829]: audit 2021-03-10T13:45:18.100298+0000 mon.a (mon.0) 249 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:19 gibba032 bash[9829]: cluster 2021-03-10T13:45:18.100393+0000 mon.a (mon.0) 250 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2021-03-10T13:45:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:19 gibba032 bash[9829]: audit 2021-03-10T13:45:18.101560+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: cluster 2021-03-10T13:45:19.074000+0000 mgr.y (mgr.14140) 73 : cluster [DBG] pgmap v64: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:20.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: cluster 2021-03-10T13:45:19.116869+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352] boot 2021-03-10T13:45:20.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: cluster 2021-03-10T13:45:19.116900+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e16: 2 total, 2 up, 2 in 2021-03-10T13:45:20.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: audit 2021-03-10T13:45:19.117446+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:20.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: audit 2021-03-10T13:45:19.653878+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: audit 2021-03-10T13:45:19.654904+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:20 gibba032 bash[9829]: audit 2021-03-10T13:45:19.656007+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: cluster 2021-03-10T13:45:19.074000+0000 mgr.y (mgr.14140) 73 : cluster [DBG] pgmap v64: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: cluster 2021-03-10T13:45:19.116869+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352] boot 2021-03-10T13:45:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: cluster 2021-03-10T13:45:19.116900+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e16: 2 total, 2 up, 2 in 2021-03-10T13:45:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: audit 2021-03-10T13:45:19.117446+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: audit 2021-03-10T13:45:19.653878+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: audit 2021-03-10T13:45:19.654904+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[9813]: audit 2021-03-10T13:45:19.656007+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: cluster 2021-03-10T13:45:19.074000+0000 mgr.y (mgr.14140) 73 : cluster [DBG] pgmap v64: 1 pgs: 1 undersized+peered; 0 B data, 120 KiB used, 83 GiB / 84 GiB avail 2021-03-10T13:45:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: cluster 2021-03-10T13:45:19.116869+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.2.111:6810/4088389352,v1:172.21.2.111:6811/4088389352] boot 2021-03-10T13:45:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: cluster 2021-03-10T13:45:19.116900+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e16: 2 total, 2 up, 2 in 2021-03-10T13:45:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: audit 2021-03-10T13:45:19.117446+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:45:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: audit 2021-03-10T13:45:19.653878+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: audit 2021-03-10T13:45:19.654904+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:20 gibba011 bash[13015]: audit 2021-03-10T13:45:19.656007+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:21.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[9813]: audit 2021-03-10T13:45:19.653021+0000 mgr.y (mgr.14140) 74 : audit [DBG] from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[9813]: cephadm 2021-03-10T13:45:19.653619+0000 mgr.y (mgr.14140) 75 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[9813]: cluster 2021-03-10T13:45:20.135209+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2021-03-10T13:45:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[9813]: audit 2021-03-10T13:45:20.137282+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]: dispatch 2021-03-10T13:45:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[9813]: audit 2021-03-10T13:45:20.139728+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]': finished 2021-03-10T13:45:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[13015]: audit 2021-03-10T13:45:19.653021+0000 mgr.y (mgr.14140) 74 : audit [DBG] from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[13015]: cephadm 2021-03-10T13:45:19.653619+0000 mgr.y (mgr.14140) 75 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[13015]: cluster 2021-03-10T13:45:20.135209+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2021-03-10T13:45:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[13015]: audit 2021-03-10T13:45:20.137282+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]: dispatch 2021-03-10T13:45:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:21 gibba011 bash[13015]: audit 2021-03-10T13:45:20.139728+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]': finished 2021-03-10T13:45:21.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:21 gibba032 bash[9829]: audit 2021-03-10T13:45:19.653021+0000 mgr.y (mgr.14140) 74 : audit [DBG] from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:21 gibba032 bash[9829]: cephadm 2021-03-10T13:45:19.653619+0000 mgr.y (mgr.14140) 75 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:21 gibba032 bash[9829]: cluster 2021-03-10T13:45:20.135209+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2021-03-10T13:45:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:21 gibba032 bash[9829]: audit 2021-03-10T13:45:20.137282+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]: dispatch 2021-03-10T13:45:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:21 gibba032 bash[9829]: audit 2021-03-10T13:45:20.139728+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20210310-134317"}]': finished 2021-03-10T13:45:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:22 gibba032 bash[9829]: cluster 2021-03-10T13:45:21.074901+0000 mgr.y (mgr.14140) 76 : cluster [DBG] pgmap v67: 1 pgs: 1 undersized+peered; 0 B data, 264 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:22 gibba011 bash[9813]: cluster 2021-03-10T13:45:21.074901+0000 mgr.y (mgr.14140) 76 : cluster [DBG] pgmap v67: 1 pgs: 1 undersized+peered; 0 B data, 264 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:22 gibba011 bash[13015]: cluster 2021-03-10T13:45:21.074901+0000 mgr.y (mgr.14140) 76 : cluster [DBG] pgmap v67: 1 pgs: 1 undersized+peered; 0 B data, 264 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:23 gibba032 bash[9829]: audit 2021-03-10T13:45:22.798792+0000 mon.a (mon.0) 261 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]: dispatch 2021-03-10T13:45:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:23 gibba032 bash[9829]: audit 2021-03-10T13:45:22.804155+0000 mon.a (mon.0) 262 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]': finished 2021-03-10T13:45:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:23 gibba032 bash[9829]: cluster 2021-03-10T13:45:22.804180+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e18: 3 total, 2 up, 2 in 2021-03-10T13:45:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:23 gibba032 bash[9829]: audit 2021-03-10T13:45:22.804321+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:23.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[9813]: audit 2021-03-10T13:45:22.798792+0000 mon.a (mon.0) 261 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]: dispatch 2021-03-10T13:45:23.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[9813]: audit 2021-03-10T13:45:22.804155+0000 mon.a (mon.0) 262 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]': finished 2021-03-10T13:45:23.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[9813]: cluster 2021-03-10T13:45:22.804180+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e18: 3 total, 2 up, 2 in 2021-03-10T13:45:23.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[9813]: audit 2021-03-10T13:45:22.804321+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:23.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[13015]: audit 2021-03-10T13:45:22.798792+0000 mon.a (mon.0) 261 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]: dispatch 2021-03-10T13:45:23.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[13015]: audit 2021-03-10T13:45:22.804155+0000 mon.a (mon.0) 262 : audit [INF] from='client.? 172.21.2.111:0/2487506376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26948f0-d140-49d5-b246-b225db5e8c93"}]': finished 2021-03-10T13:45:23.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[13015]: cluster 2021-03-10T13:45:22.804180+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e18: 3 total, 2 up, 2 in 2021-03-10T13:45:23.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:23 gibba011 bash[13015]: audit 2021-03-10T13:45:22.804321+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:24 gibba032 bash[9829]: cluster 2021-03-10T13:45:23.075477+0000 mgr.y (mgr.14140) 77 : cluster [DBG] pgmap v69: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:24 gibba011 bash[9813]: cluster 2021-03-10T13:45:23.075477+0000 mgr.y (mgr.14140) 77 : cluster [DBG] pgmap v69: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:24 gibba011 bash[13015]: cluster 2021-03-10T13:45:23.075477+0000 mgr.y (mgr.14140) 77 : cluster [DBG] pgmap v69: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:25.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:25 gibba032 bash[9829]: audit 2021-03-10T13:45:24.799419+0000 mon.c (mon.1) 7 : audit [DBG] from='client.? 172.21.2.111:0/2822200684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:25.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:25 gibba011 bash[9813]: audit 2021-03-10T13:45:24.799419+0000 mon.c (mon.1) 7 : audit [DBG] from='client.? 172.21.2.111:0/2822200684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:25.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:25 gibba011 bash[13015]: audit 2021-03-10T13:45:24.799419+0000 mon.c (mon.1) 7 : audit [DBG] from='client.? 172.21.2.111:0/2822200684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:26 gibba032 bash[9829]: cluster 2021-03-10T13:45:25.076010+0000 mgr.y (mgr.14140) 78 : cluster [DBG] pgmap v70: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:26 gibba011 bash[9813]: cluster 2021-03-10T13:45:25.076010+0000 mgr.y (mgr.14140) 78 : cluster [DBG] pgmap v70: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:26 gibba011 bash[13015]: cluster 2021-03-10T13:45:25.076010+0000 mgr.y (mgr.14140) 78 : cluster [DBG] pgmap v70: 1 pgs: 1 undersized+peered; 0 B data, 296 KiB used, 166 GiB / 168 GiB avail 2021-03-10T13:45:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:27 gibba032 bash[9829]: cluster 2021-03-10T13:45:27.173603+0000 mon.a (mon.0) 265 : cluster [WRN] Health check failed: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:45:27.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:27 gibba032 bash[9829]: cluster 2021-03-10T13:45:27.173702+0000 mon.a (mon.0) 266 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2021-03-10T13:45:27.530 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:27 gibba011 bash[9813]: cluster 2021-03-10T13:45:27.173603+0000 mon.a (mon.0) 265 : cluster [WRN] Health check failed: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:45:27.530 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:27 gibba011 bash[9813]: cluster 2021-03-10T13:45:27.173702+0000 mon.a (mon.0) 266 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2021-03-10T13:45:27.531 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:27 gibba011 bash[13015]: cluster 2021-03-10T13:45:27.173603+0000 mon.a (mon.0) 265 : cluster [WRN] Health check failed: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:45:27.531 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:27 gibba011 bash[13015]: cluster 2021-03-10T13:45:27.173702+0000 mon.a (mon.0) 266 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2021-03-10T13:45:28.483 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:28 gibba011 bash[9813]: cluster 2021-03-10T13:45:27.076987+0000 mgr.y (mgr.14140) 79 : cluster [DBG] pgmap v71: 1 pgs: 1 active+undersized+degraded; 0 B data, 312 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:28.483 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:28 gibba011 bash[9813]: audit 2021-03-10T13:45:27.422532+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:28.483 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:28 gibba011 bash[13015]: cluster 2021-03-10T13:45:27.076987+0000 mgr.y (mgr.14140) 79 : cluster [DBG] pgmap v71: 1 pgs: 1 active+undersized+degraded; 0 B data, 312 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:28.484 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:28 gibba011 bash[13015]: audit 2021-03-10T13:45:27.422532+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:28 gibba032 bash[9829]: cluster 2021-03-10T13:45:27.076987+0000 mgr.y (mgr.14140) 79 : cluster [DBG] pgmap v71: 1 pgs: 1 active+undersized+degraded; 0 B data, 312 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:28 gibba032 bash[9829]: audit 2021-03-10T13:45:27.422532+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:30 gibba032 bash[9829]: cluster 2021-03-10T13:45:29.077535+0000 mgr.y (mgr.14140) 80 : cluster [DBG] pgmap v72: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:30 gibba011 bash[9813]: cluster 2021-03-10T13:45:29.077535+0000 mgr.y (mgr.14140) 80 : cluster [DBG] pgmap v72: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:30 gibba011 bash[13015]: cluster 2021-03-10T13:45:29.077535+0000 mgr.y (mgr.14140) 80 : cluster [DBG] pgmap v72: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:31 gibba032 bash[9829]: audit 2021-03-10T13:45:30.750512+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2021-03-10T13:45:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:31 gibba032 bash[9829]: audit 2021-03-10T13:45:30.751359+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:31 gibba032 bash[9829]: audit 2021-03-10T13:45:30.752246+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2021-03-10T13:45:31.560 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[9813]: audit 2021-03-10T13:45:30.750512+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2021-03-10T13:45:31.560 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[9813]: audit 2021-03-10T13:45:30.751359+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:31.560 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[9813]: audit 2021-03-10T13:45:30.752246+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2021-03-10T13:45:31.560 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[13015]: audit 2021-03-10T13:45:30.750512+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2021-03-10T13:45:31.561 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[13015]: audit 2021-03-10T13:45:30.751359+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:31.561 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:31 gibba011 bash[13015]: audit 2021-03-10T13:45:30.752246+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2021-03-10T13:45:32.542 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:32 gibba011 bash[9813]: cephadm 2021-03-10T13:45:30.752006+0000 mgr.y (mgr.14140) 81 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:45:32.543 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:32 gibba011 bash[9813]: cluster 2021-03-10T13:45:31.078388+0000 mgr.y (mgr.14140) 82 : cluster [DBG] pgmap v73: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:32.543 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:32 gibba011 bash[13015]: cephadm 2021-03-10T13:45:30.752006+0000 mgr.y (mgr.14140) 81 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:45:32.543 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:32 gibba011 bash[13015]: cluster 2021-03-10T13:45:31.078388+0000 mgr.y (mgr.14140) 82 : cluster [DBG] pgmap v73: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:32.578 INFO:teuthology.orchestra.run.gibba011.stdout:Created osd(s) 2 on host 'gibba011' 2021-03-10T13:45:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:32 gibba032 bash[9829]: cephadm 2021-03-10T13:45:30.752006+0000 mgr.y (mgr.14140) 81 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:45:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:32 gibba032 bash[9829]: cluster 2021-03-10T13:45:31.078388+0000 mgr.y (mgr.14140) 82 : cluster [DBG] pgmap v73: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:33.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[13015]: audit 2021-03-10T13:45:32.573863+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:33.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[13015]: audit 2021-03-10T13:45:32.574412+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[13015]: audit 2021-03-10T13:45:32.578058+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[9813]: audit 2021-03-10T13:45:32.573863+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[9813]: audit 2021-03-10T13:45:32.574412+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:33 gibba011 bash[9813]: audit 2021-03-10T13:45:32.578058+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:33 gibba032 bash[9829]: audit 2021-03-10T13:45:32.573863+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:33 gibba032 bash[9829]: audit 2021-03-10T13:45:32.574412+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:33 gibba032 bash[9829]: audit 2021-03-10T13:45:32.578058+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.992955\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993014\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993042\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993068\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:09.993094\", \"created\": \"2021-03-10T13:45:09.440405\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_device_update\": \"2021-03-10T13:45:17.816890\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:33.916 DEBUG:teuthology.orchestra.run.gibba011:osd.2> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.2.service 2021-03-10T13:45:33.919 INFO:tasks.cephadm:Deploying osd.3 on gibba011 with /dev/vg_nvme/lv_1... 2021-03-10T13:45:33.919 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2021-03-10T13:45:33.964 INFO:journalctl@ceph.osd.2.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:45:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[9813]: cluster 2021-03-10T13:45:33.078692+0000 mgr.y (mgr.14140) 83 : cluster [DBG] pgmap v74: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[13015]: cluster 2021-03-10T13:45:33.078692+0000 mgr.y (mgr.14140) 83 : cluster [DBG] pgmap v74: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[9813]: audit 2021-03-10T13:45:33.277569+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[13015]: audit 2021-03-10T13:45:33.277569+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[9813]: audit 2021-03-10T13:45:33.280200+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[9813]: audit 2021-03-10T13:45:33.280781+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[13015]: audit 2021-03-10T13:45:33.280200+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[13015]: audit 2021-03-10T13:45:33.280781+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:34 gibba032 bash[9829]: cluster 2021-03-10T13:45:33.078692+0000 mgr.y (mgr.14140) 83 : cluster [DBG] pgmap v74: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:34 gibba032 bash[9829]: audit 2021-03-10T13:45:33.277569+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:34 gibba032 bash[9829]: audit 2021-03-10T13:45:33.280200+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:34 gibba032 bash[9829]: audit 2021-03-10T13:45:33.280781+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:35.219 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-03-10T13:45:35.219 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2021-03-10T13:45:35.220 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2021-03-10T13:45:35.220 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2021-03-10T13:45:35.220 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-03-10T13:45:35.220 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-03-10T13:45:35.220 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:34 gibba011 bash[17517]: --> ceph-volume lvm activate successful for osd ID: 2 2021-03-10T13:45:36.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:36 gibba011 bash[9813]: cluster 2021-03-10T13:45:35.079264+0000 mgr.y (mgr.14140) 84 : cluster [DBG] pgmap v75: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:36.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:36 gibba011 bash[13015]: cluster 2021-03-10T13:45:35.079264+0000 mgr.y (mgr.14140) 84 : cluster [DBG] pgmap v75: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:36 gibba032 bash[9829]: cluster 2021-03-10T13:45:35.079264+0000 mgr.y (mgr.14140) 84 : cluster [DBG] pgmap v75: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:38 gibba032 bash[9829]: cluster 2021-03-10T13:45:37.080031+0000 mgr.y (mgr.14140) 85 : cluster [DBG] pgmap v76: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:38 gibba011 bash[9813]: cluster 2021-03-10T13:45:37.080031+0000 mgr.y (mgr.14140) 85 : cluster [DBG] pgmap v76: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:38 gibba011 bash[13015]: cluster 2021-03-10T13:45:37.080031+0000 mgr.y (mgr.14140) 85 : cluster [DBG] pgmap v76: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:39.080 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:38 gibba011 bash[17842]: debug 2021-03-10T13:45:38.813+0000 7f35867cbec0 -1 Falling back to public interface 2021-03-10T13:45:39.734 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping: /dev/vg_nvme/lv_1 2021-03-10T13:45:39.735 INFO:teuthology.orchestra.run.gibba011.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2021-03-10T13:45:39.735 INFO:teuthology.orchestra.run.gibba011.stderr: stderr: 10+0 records in 2021-03-10T13:45:39.735 INFO:teuthology.orchestra.run.gibba011.stderr:10+0 records out 2021-03-10T13:45:39.735 INFO:teuthology.orchestra.run.gibba011.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00876856 s, 1.2 GB/s 2021-03-10T13:45:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:--> Zapping successful for: 2021-03-10T13:45:39.789 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:39 gibba011 bash[17842]: debug 2021-03-10T13:45:39.409+0000 7f35867cbec0 -1 osd.2 0 log_to_monitors {default=true} 2021-03-10T13:45:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:40 gibba011 bash[9813]: cluster 2021-03-10T13:45:39.080373+0000 mgr.y (mgr.14140) 86 : cluster [DBG] pgmap v77: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:40 gibba011 bash[9813]: audit 2021-03-10T13:45:39.416742+0000 mon.a (mon.0) 277 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:45:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:40 gibba011 bash[13015]: cluster 2021-03-10T13:45:39.080373+0000 mgr.y (mgr.14140) 86 : cluster [DBG] pgmap v77: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:40 gibba011 bash[13015]: audit 2021-03-10T13:45:39.416742+0000 mon.a (mon.0) 277 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:45:40.379 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba011:vg_nvme/lv_1 2021-03-10T13:45:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:40 gibba032 bash[9829]: cluster 2021-03-10T13:45:39.080373+0000 mgr.y (mgr.14140) 86 : cluster [DBG] pgmap v77: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:40 gibba032 bash[9829]: audit 2021-03-10T13:45:39.416742+0000 mon.a (mon.0) 277 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:45:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:41 gibba032 bash[9829]: audit 2021-03-10T13:45:40.120173+0000 mon.a (mon.0) 278 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:45:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:41 gibba032 bash[9829]: cluster 2021-03-10T13:45:40.120283+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e19: 3 total, 2 up, 2 in 2021-03-10T13:45:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:41 gibba032 bash[9829]: audit 2021-03-10T13:45:40.120617+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:41 gibba032 bash[9829]: audit 2021-03-10T13:45:40.121796+0000 mon.a (mon.0) 281 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:41.580 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[17842]: debug 2021-03-10T13:45:41.125+0000 7f356b578700 -1 osd.2 0 waiting for initial osdmap 2021-03-10T13:45:41.581 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[17842]: debug 2021-03-10T13:45:41.129+0000 7f3576794700 -1 osd.2 20 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:45:41.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[9813]: audit 2021-03-10T13:45:40.120173+0000 mon.a (mon.0) 278 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:45:41.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[9813]: cluster 2021-03-10T13:45:40.120283+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e19: 3 total, 2 up, 2 in 2021-03-10T13:45:41.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[9813]: audit 2021-03-10T13:45:40.120617+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:41.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[9813]: audit 2021-03-10T13:45:40.121796+0000 mon.a (mon.0) 281 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:41.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[13015]: audit 2021-03-10T13:45:40.120173+0000 mon.a (mon.0) 278 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:45:41.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[13015]: cluster 2021-03-10T13:45:40.120283+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e19: 3 total, 2 up, 2 in 2021-03-10T13:45:41.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[13015]: audit 2021-03-10T13:45:40.120617+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:41.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:41 gibba011 bash[13015]: audit 2021-03-10T13:45:40.121796+0000 mon.a (mon.0) 281 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:45:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: cluster 2021-03-10T13:45:40.454123+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: cluster 2021-03-10T13:45:40.454156+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: cluster 2021-03-10T13:45:41.080985+0000 mgr.y (mgr.14140) 87 : cluster [DBG] pgmap v79: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: audit 2021-03-10T13:45:41.125133+0000 mon.a (mon.0) 282 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: cluster 2021-03-10T13:45:41.125201+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2021-03-10T13:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: audit 2021-03-10T13:45:41.125839+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: audit 2021-03-10T13:45:41.347214+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: audit 2021-03-10T13:45:41.447190+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:42 gibba032 bash[9829]: audit 2021-03-10T13:45:41.451460+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: cluster 2021-03-10T13:45:40.454123+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: cluster 2021-03-10T13:45:40.454156+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: cluster 2021-03-10T13:45:41.080985+0000 mgr.y (mgr.14140) 87 : cluster [DBG] pgmap v79: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: audit 2021-03-10T13:45:41.125133+0000 mon.a (mon.0) 282 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: cluster 2021-03-10T13:45:41.125201+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2021-03-10T13:45:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: audit 2021-03-10T13:45:41.125839+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:42.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: audit 2021-03-10T13:45:41.347214+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: cluster 2021-03-10T13:45:40.454123+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:45:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: cluster 2021-03-10T13:45:40.454156+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:45:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: cluster 2021-03-10T13:45:41.080985+0000 mgr.y (mgr.14140) 87 : cluster [DBG] pgmap v79: 1 pgs: 1 active+undersized+degraded; 0 B data, 320 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: audit 2021-03-10T13:45:41.125133+0000 mon.a (mon.0) 282 : audit [INF] from='osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:45:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: cluster 2021-03-10T13:45:41.125201+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2021-03-10T13:45:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: audit 2021-03-10T13:45:41.125839+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: audit 2021-03-10T13:45:41.347214+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: audit 2021-03-10T13:45:41.447190+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[13015]: audit 2021-03-10T13:45:41.451460+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:42.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: audit 2021-03-10T13:45:41.447190+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:42.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:42 gibba011 bash[9813]: audit 2021-03-10T13:45:41.451460+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}}, \"last_daemon_update\": \"2021-03-10T13:45:33.277019\", \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:43 gibba032 bash[9829]: cluster 2021-03-10T13:45:42.139719+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540] boot 2021-03-10T13:45:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:43 gibba032 bash[9829]: cluster 2021-03-10T13:45:42.139743+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2021-03-10T13:45:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:43 gibba032 bash[9829]: audit 2021-03-10T13:45:42.140544+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:43 gibba032 bash[9829]: cluster 2021-03-10T13:45:43.143098+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2021-03-10T13:45:43.512 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[9813]: cluster 2021-03-10T13:45:42.139719+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540] boot 2021-03-10T13:45:43.512 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[9813]: cluster 2021-03-10T13:45:42.139743+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2021-03-10T13:45:43.512 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[9813]: audit 2021-03-10T13:45:42.140544+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:43.513 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[9813]: cluster 2021-03-10T13:45:43.143098+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2021-03-10T13:45:43.513 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[13015]: cluster 2021-03-10T13:45:42.139719+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 [v2:172.21.2.111:6818/2275782540,v1:172.21.2.111:6819/2275782540] boot 2021-03-10T13:45:43.513 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[13015]: cluster 2021-03-10T13:45:42.139743+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2021-03-10T13:45:43.513 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[13015]: audit 2021-03-10T13:45:42.140544+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:45:43.514 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:43 gibba011 bash[13015]: cluster 2021-03-10T13:45:43.143098+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2021-03-10T13:45:44.458 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: cluster 2021-03-10T13:45:43.081737+0000 mgr.y (mgr.14140) 88 : cluster [DBG] pgmap v82: 1 pgs: 1 active+undersized+degraded; 0 B data, 336 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:44.459 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: audit 2021-03-10T13:45:43.399049+0000 mgr.y (mgr.14140) 89 : audit [DBG] from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:44.459 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: cephadm 2021-03-10T13:45:43.399499+0000 mgr.y (mgr.14140) 90 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:44.459 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: audit 2021-03-10T13:45:43.399716+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:44.459 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: audit 2021-03-10T13:45:43.400634+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:44.460 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[9813]: audit 2021-03-10T13:45:43.401546+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:44.460 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: cluster 2021-03-10T13:45:43.081737+0000 mgr.y (mgr.14140) 88 : cluster [DBG] pgmap v82: 1 pgs: 1 active+undersized+degraded; 0 B data, 336 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:44.460 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: audit 2021-03-10T13:45:43.399049+0000 mgr.y (mgr.14140) 89 : audit [DBG] from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:44.460 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: cephadm 2021-03-10T13:45:43.399499+0000 mgr.y (mgr.14140) 90 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:44.461 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: audit 2021-03-10T13:45:43.399716+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:44.461 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: audit 2021-03-10T13:45:43.400634+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:44.461 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:44 gibba011 bash[13015]: audit 2021-03-10T13:45:43.401546+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: cluster 2021-03-10T13:45:43.081737+0000 mgr.y (mgr.14140) 88 : cluster [DBG] pgmap v82: 1 pgs: 1 active+undersized+degraded; 0 B data, 336 KiB used, 166 GiB / 168 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: audit 2021-03-10T13:45:43.399049+0000 mgr.y (mgr.14140) 89 : audit [DBG] from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba011:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:45:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: cephadm 2021-03-10T13:45:43.399499+0000 mgr.y (mgr.14140) 90 : cephadm [INF] Applying osd on host gibba011... 2021-03-10T13:45:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: audit 2021-03-10T13:45:43.399716+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:45:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: audit 2021-03-10T13:45:43.400634+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:44 gibba032 bash[9829]: audit 2021-03-10T13:45:43.401546+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:46 gibba011 bash[9813]: cluster 2021-03-10T13:45:45.082083+0000 mgr.y (mgr.14140) 91 : cluster [DBG] pgmap v84: 1 pgs: 1 active+undersized+degraded; 0 B data, 520 KiB used, 249 GiB / 252 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:46 gibba011 bash[13015]: cluster 2021-03-10T13:45:45.082083+0000 mgr.y (mgr.14140) 91 : cluster [DBG] pgmap v84: 1 pgs: 1 active+undersized+degraded; 0 B data, 520 KiB used, 249 GiB / 252 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:46 gibba032 bash[9829]: cluster 2021-03-10T13:45:45.082083+0000 mgr.y (mgr.14140) 91 : cluster [DBG] pgmap v84: 1 pgs: 1 active+undersized+degraded; 0 B data, 520 KiB used, 249 GiB / 252 GiB avail; 1/3 objects degraded (33.333%) 2021-03-10T13:45:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:47 gibba032 bash[9829]: audit 2021-03-10T13:45:46.524044+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]: dispatch 2021-03-10T13:45:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:47 gibba032 bash[9829]: audit 2021-03-10T13:45:46.529483+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]': finished 2021-03-10T13:45:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:47 gibba032 bash[9829]: cluster 2021-03-10T13:45:46.529528+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e23: 4 total, 3 up, 3 in 2021-03-10T13:45:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:47 gibba032 bash[9829]: audit 2021-03-10T13:45:46.529694+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:45:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[9813]: audit 2021-03-10T13:45:46.524044+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]: dispatch 2021-03-10T13:45:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[9813]: audit 2021-03-10T13:45:46.529483+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]': finished 2021-03-10T13:45:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[9813]: cluster 2021-03-10T13:45:46.529528+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e23: 4 total, 3 up, 3 in 2021-03-10T13:45:47.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[9813]: audit 2021-03-10T13:45:46.529694+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:45:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[13015]: audit 2021-03-10T13:45:46.524044+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]: dispatch 2021-03-10T13:45:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[13015]: audit 2021-03-10T13:45:46.529483+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.2.111:0/2144617670' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c387e310-fde0-48e0-971a-0a016244271f"}]': finished 2021-03-10T13:45:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[13015]: cluster 2021-03-10T13:45:46.529528+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e23: 4 total, 3 up, 3 in 2021-03-10T13:45:47.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:47 gibba011 bash[13015]: audit 2021-03-10T13:45:46.529694+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:45:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:48 gibba032 bash[9829]: cluster 2021-03-10T13:45:47.082315+0000 mgr.y (mgr.14140) 92 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 552 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:48 gibba032 bash[9829]: cluster 2021-03-10T13:45:47.527630+0000 mon.a (mon.0) 299 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded) 2021-03-10T13:45:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:48 gibba032 bash[9829]: cluster 2021-03-10T13:45:47.527727+0000 mon.a (mon.0) 300 : cluster [INF] Cluster is now healthy 2021-03-10T13:45:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[9813]: cluster 2021-03-10T13:45:47.082315+0000 mgr.y (mgr.14140) 92 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 552 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[9813]: cluster 2021-03-10T13:45:47.527630+0000 mon.a (mon.0) 299 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded) 2021-03-10T13:45:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[9813]: cluster 2021-03-10T13:45:47.527727+0000 mon.a (mon.0) 300 : cluster [INF] Cluster is now healthy 2021-03-10T13:45:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[13015]: cluster 2021-03-10T13:45:47.082315+0000 mgr.y (mgr.14140) 92 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 552 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[13015]: cluster 2021-03-10T13:45:47.527630+0000 mon.a (mon.0) 299 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/3 objects degraded (33.333%), 1 pg degraded) 2021-03-10T13:45:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:48 gibba011 bash[13015]: cluster 2021-03-10T13:45:47.527727+0000 mon.a (mon.0) 300 : cluster [INF] Cluster is now healthy 2021-03-10T13:45:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:49 gibba032 bash[9829]: audit 2021-03-10T13:45:48.605837+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.2.111:0/4106286892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:49 gibba011 bash[9813]: audit 2021-03-10T13:45:48.605837+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.2.111:0/4106286892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:49 gibba011 bash[13015]: audit 2021-03-10T13:45:48.605837+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.2.111:0/4106286892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:45:50.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:50 gibba032 bash[9829]: cluster 2021-03-10T13:45:49.083208+0000 mgr.y (mgr.14140) 93 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 584 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:50 gibba011 bash[9813]: cluster 2021-03-10T13:45:49.083208+0000 mgr.y (mgr.14140) 93 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 584 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:50 gibba011 bash[13015]: cluster 2021-03-10T13:45:49.083208+0000 mgr.y (mgr.14140) 93 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 584 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:52.411 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:52 gibba011 bash[9813]: cluster 2021-03-10T13:45:51.083914+0000 mgr.y (mgr.14140) 94 : cluster [DBG] pgmap v88: 1 pgs: 1 active+clean; 0 B data, 592 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:52.411 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:52 gibba011 bash[9813]: audit 2021-03-10T13:45:51.263654+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:52.411 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:52 gibba011 bash[13015]: cluster 2021-03-10T13:45:51.083914+0000 mgr.y (mgr.14140) 94 : cluster [DBG] pgmap v88: 1 pgs: 1 active+clean; 0 B data, 592 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:52.412 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:52 gibba011 bash[13015]: audit 2021-03-10T13:45:51.263654+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:52 gibba032 bash[9829]: cluster 2021-03-10T13:45:51.083914+0000 mgr.y (mgr.14140) 94 : cluster [DBG] pgmap v88: 1 pgs: 1 active+clean; 0 B data, 592 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:52 gibba032 bash[9829]: audit 2021-03-10T13:45:51.263654+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:54.411 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:54 gibba011 bash[9813]: cluster 2021-03-10T13:45:53.084597+0000 mgr.y (mgr.14140) 95 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:54.411 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:54 gibba011 bash[13015]: cluster 2021-03-10T13:45:53.084597+0000 mgr.y (mgr.14140) 95 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:54 gibba032 bash[9829]: cluster 2021-03-10T13:45:53.084597+0000 mgr.y (mgr.14140) 95 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[9813]: audit 2021-03-10T13:45:54.879439+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2021-03-10T13:45:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[9813]: audit 2021-03-10T13:45:54.880358+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[9813]: audit 2021-03-10T13:45:54.881272+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2021-03-10T13:45:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[13015]: audit 2021-03-10T13:45:54.879439+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2021-03-10T13:45:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[13015]: audit 2021-03-10T13:45:54.880358+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:55 gibba011 bash[13015]: audit 2021-03-10T13:45:54.881272+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2021-03-10T13:45:55.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:55 gibba032 bash[9829]: audit 2021-03-10T13:45:54.879439+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2021-03-10T13:45:55.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:55 gibba032 bash[9829]: audit 2021-03-10T13:45:54.880358+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:45:55.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:55 gibba032 bash[9829]: audit 2021-03-10T13:45:54.881272+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2021-03-10T13:45:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:56 gibba032 bash[9829]: cephadm 2021-03-10T13:45:54.881064+0000 mgr.y (mgr.14140) 96 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:45:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:56 gibba032 bash[9829]: cluster 2021-03-10T13:45:55.085253+0000 mgr.y (mgr.14140) 97 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:56.541 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:56 gibba011 bash[9813]: cephadm 2021-03-10T13:45:54.881064+0000 mgr.y (mgr.14140) 96 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:45:56.542 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:56 gibba011 bash[9813]: cluster 2021-03-10T13:45:55.085253+0000 mgr.y (mgr.14140) 97 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:56.542 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:56 gibba011 bash[13015]: cephadm 2021-03-10T13:45:54.881064+0000 mgr.y (mgr.14140) 96 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:45:56.542 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:56 gibba011 bash[13015]: cluster 2021-03-10T13:45:55.085253+0000 mgr.y (mgr.14140) 97 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:45:56.811 INFO:teuthology.orchestra.run.gibba011.stdout:Created osd(s) 3 on host 'gibba011' 2021-03-10T13:45:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:57 gibba032 bash[9829]: audit 2021-03-10T13:45:56.805593+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:57 gibba032 bash[9829]: audit 2021-03-10T13:45:56.806007+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:57 gibba032 bash[9829]: audit 2021-03-10T13:45:56.809357+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:57.536 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[13015]: audit 2021-03-10T13:45:56.805593+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:57.537 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[13015]: audit 2021-03-10T13:45:56.806007+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:57.537 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[13015]: audit 2021-03-10T13:45:56.809357+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:57.538 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[9813]: audit 2021-03-10T13:45:56.805593+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:57.538 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[9813]: audit 2021-03-10T13:45:56.806007+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:45:57.538 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:57 gibba011 bash[9813]: audit 2021-03-10T13:45:56.809357+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276815\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276871\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276899\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276924\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276949\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:33.276973\", \"created\": \"2021-03-10T13:45:32.562625\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_device_update\": \"2021-03-10T13:45:41.446623\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:58.037 DEBUG:teuthology.orchestra.run.gibba011:osd.3> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.3.service 2021-03-10T13:45:58.039 INFO:tasks.cephadm:Deploying osd.4 on gibba032 with /dev/vg_nvme/lv_4... 2021-03-10T13:45:58.040 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2021-03-10T13:45:58.120 INFO:journalctl@ceph.osd.3.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:45:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[9813]: cluster 2021-03-10T13:45:57.086069+0000 mgr.y (mgr.14140) 98 : cluster [DBG] pgmap v91: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:45:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[13015]: cluster 2021-03-10T13:45:57.086069+0000 mgr.y (mgr.14140) 98 : cluster [DBG] pgmap v91: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:45:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[9813]: audit 2021-03-10T13:45:57.401535+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[9813]: audit 2021-03-10T13:45:57.405614+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[9813]: audit 2021-03-10T13:45:57.406084+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[13015]: audit 2021-03-10T13:45:57.401535+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[13015]: audit 2021-03-10T13:45:57.405614+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[13015]: audit 2021-03-10T13:45:57.406084+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:58 gibba032 bash[9829]: cluster 2021-03-10T13:45:57.086069+0000 mgr.y (mgr.14140) 98 : cluster [DBG] pgmap v91: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:45:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:58 gibba032 bash[9829]: audit 2021-03-10T13:45:57.401535+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:45:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:58 gibba032 bash[9829]: audit 2021-03-10T13:45:57.405614+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:45:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:45:58 gibba032 bash[9829]: audit 2021-03-10T13:45:57.406084+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:45:59.303 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-03-10T13:45:59.303 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2021-03-10T13:45:59.304 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3/block 2021-03-10T13:45:59.304 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2021-03-10T13:45:59.304 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2021-03-10T13:45:59.304 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-03-10T13:45:59.304 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:45:58 gibba011 bash[18972]: --> ceph-volume lvm activate successful for osd ID: 3 2021-03-10T13:46:00.420 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:00 gibba032 bash[9829]: cluster 2021-03-10T13:45:59.086423+0000 mgr.y (mgr.14140) 99 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:00 gibba011 bash[9813]: cluster 2021-03-10T13:45:59.086423+0000 mgr.y (mgr.14140) 99 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:00.652 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:00 gibba011 bash[13015]: cluster 2021-03-10T13:45:59.086423+0000 mgr.y (mgr.14140) 99 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:00.827 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2021-03-10T13:46:00.827 INFO:teuthology.orchestra.run.gibba032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2021-03-10T13:46:00.828 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 10+0 records in 2021-03-10T13:46:00.828 INFO:teuthology.orchestra.run.gibba032.stderr:10+0 records out 2021-03-10T13:46:00.828 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00901513 s, 1.2 GB/s 2021-03-10T13:46:00.828 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping successful for: 2021-03-10T13:46:01.447 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba032:vg_nvme/lv_4 2021-03-10T13:46:02.168 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:46:01 gibba011 bash[19312]: debug 2021-03-10T13:46:01.893+0000 7fbc26fc4ec0 -1 Falling back to public interface 2021-03-10T13:46:02.485 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:02 gibba032 bash[9829]: cluster 2021-03-10T13:46:01.087173+0000 mgr.y (mgr.14140) 100 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:02.498 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:02 gibba011 bash[9813]: cluster 2021-03-10T13:46:01.087173+0000 mgr.y (mgr.14140) 100 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:02.499 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:02 gibba011 bash[13015]: cluster 2021-03-10T13:46:01.087173+0000 mgr.y (mgr.14140) 100 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:02.830 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:46:02 gibba011 bash[19312]: debug 2021-03-10T13:46:02.601+0000 7fbc26fc4ec0 -1 osd.3 0 log_to_monitors {default=true} 2021-03-10T13:46:03.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:03 gibba032 bash[9829]: audit 2021-03-10T13:46:02.613209+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:03 gibba032 bash[9829]: audit 2021-03-10T13:46:02.613384+0000 mon.a (mon.0) 312 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:03 gibba032 bash[9829]: audit 2021-03-10T13:46:03.036754+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:03 gibba032 bash[9829]: audit 2021-03-10T13:46:03.135304+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:46:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:03 gibba032 bash[9829]: audit 2021-03-10T13:46:03.138329+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:46:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[9813]: audit 2021-03-10T13:46:02.613209+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[9813]: audit 2021-03-10T13:46:02.613384+0000 mon.a (mon.0) 312 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[9813]: audit 2021-03-10T13:46:03.036754+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[13015]: audit 2021-03-10T13:46:02.613209+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[13015]: audit 2021-03-10T13:46:02.613384+0000 mon.a (mon.0) 312 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:46:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[13015]: audit 2021-03-10T13:46:03.036754+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[9813]: audit 2021-03-10T13:46:03.135304+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:46:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[9813]: audit 2021-03-10T13:46:03.138329+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:46:03.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[13015]: audit 2021-03-10T13:46:03.135304+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:46:03.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:03 gibba011 bash[13015]: audit 2021-03-10T13:46:03.138329+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}}, \"last_daemon_update\": \"2021-03-10T13:45:57.400909\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:46:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: cluster 2021-03-10T13:46:03.087522+0000 mgr.y (mgr.14140) 101 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:03.182017+0000 mon.a (mon.0) 316 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:46:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: cluster 2021-03-10T13:46:03.182040+0000 mon.a (mon.0) 317 : cluster [DBG] osdmap e24: 4 total, 3 up, 3 in 2021-03-10T13:46:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:03.182117+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:03.182533+0000 mon.c (mon.1) 9 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:03.182695+0000 mon.a (mon.0) 319 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:03.999012+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:04.001905+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:04 gibba032 bash[9829]: audit 2021-03-10T13:46:04.005840+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:04.580 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[19312]: debug 2021-03-10T13:46:04.193+0000 7fbc0bd71700 -1 osd.3 0 waiting for initial osdmap 2021-03-10T13:46:04.580 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[19312]: debug 2021-03-10T13:46:04.197+0000 7fbc16f8d700 -1 osd.3 25 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: cluster 2021-03-10T13:46:03.087522+0000 mgr.y (mgr.14140) 101 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:03.182017+0000 mon.a (mon.0) 316 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: cluster 2021-03-10T13:46:03.182040+0000 mon.a (mon.0) 317 : cluster [DBG] osdmap e24: 4 total, 3 up, 3 in 2021-03-10T13:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:03.182117+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:03.182533+0000 mon.c (mon.1) 9 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:03.182695+0000 mon.a (mon.0) 319 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:03.999012+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:04.001905+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[9813]: audit 2021-03-10T13:46:04.005840+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: cluster 2021-03-10T13:46:03.087522+0000 mgr.y (mgr.14140) 101 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:03.182017+0000 mon.a (mon.0) 316 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:46:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: cluster 2021-03-10T13:46:03.182040+0000 mon.a (mon.0) 317 : cluster [DBG] osdmap e24: 4 total, 3 up, 3 in 2021-03-10T13:46:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:03.182117+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:03.182533+0000 mon.c (mon.1) 9 : audit [INF] from='osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:03.182695+0000 mon.a (mon.0) 319 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:46:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:03.999012+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:04.001905+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:04 gibba011 bash[13015]: audit 2021-03-10T13:46:04.005840+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:05.226 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: cluster 2021-03-10T13:46:03.647500+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:05.226 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: cluster 2021-03-10T13:46:03.647600+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:05.226 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: audit 2021-03-10T13:46:03.995704+0000 mgr.y (mgr.14140) 102 : audit [DBG] from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:05.227 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: cephadm 2021-03-10T13:46:03.998213+0000 mgr.y (mgr.14140) 103 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:05.227 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: audit 2021-03-10T13:46:04.190912+0000 mon.a (mon.0) 323 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:46:05.227 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: cluster 2021-03-10T13:46:04.190938+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2021-03-10T13:46:05.227 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:05 gibba032 bash[9829]: audit 2021-03-10T13:46:04.191711+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: cluster 2021-03-10T13:46:03.647500+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: cluster 2021-03-10T13:46:03.647600+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: audit 2021-03-10T13:46:03.995704+0000 mgr.y (mgr.14140) 102 : audit [DBG] from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: cephadm 2021-03-10T13:46:03.998213+0000 mgr.y (mgr.14140) 103 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: audit 2021-03-10T13:46:04.190912+0000 mon.a (mon.0) 323 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:46:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: cluster 2021-03-10T13:46:04.190938+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2021-03-10T13:46:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[9813]: audit 2021-03-10T13:46:04.191711+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:05.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: cluster 2021-03-10T13:46:03.647500+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: cluster 2021-03-10T13:46:03.647600+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: audit 2021-03-10T13:46:03.995704+0000 mgr.y (mgr.14140) 102 : audit [DBG] from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: cephadm 2021-03-10T13:46:03.998213+0000 mgr.y (mgr.14140) 103 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: audit 2021-03-10T13:46:04.190912+0000 mon.a (mon.0) 323 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]': finished 2021-03-10T13:46:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: cluster 2021-03-10T13:46:04.190938+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2021-03-10T13:46:05.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:05 gibba011 bash[13015]: audit 2021-03-10T13:46:04.191711+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:06.471 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:06 gibba032 bash[9829]: cluster 2021-03-10T13:46:05.088035+0000 mgr.y (mgr.14140) 104 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:06.471 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:06 gibba032 bash[9829]: cluster 2021-03-10T13:46:05.211419+0000 mon.a (mon.0) 326 : cluster [INF] osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943] boot 2021-03-10T13:46:06.472 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:06 gibba032 bash[9829]: cluster 2021-03-10T13:46:05.211442+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2021-03-10T13:46:06.472 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:06 gibba032 bash[9829]: audit 2021-03-10T13:46:05.211510+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[9813]: cluster 2021-03-10T13:46:05.088035+0000 mgr.y (mgr.14140) 104 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[9813]: cluster 2021-03-10T13:46:05.211419+0000 mon.a (mon.0) 326 : cluster [INF] osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943] boot 2021-03-10T13:46:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[9813]: cluster 2021-03-10T13:46:05.211442+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2021-03-10T13:46:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[9813]: audit 2021-03-10T13:46:05.211510+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[13015]: cluster 2021-03-10T13:46:05.088035+0000 mgr.y (mgr.14140) 104 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 600 KiB used, 249 GiB / 252 GiB avail 2021-03-10T13:46:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[13015]: cluster 2021-03-10T13:46:05.211419+0000 mon.a (mon.0) 326 : cluster [INF] osd.3 [v2:172.21.2.111:6826/2123819943,v1:172.21.2.111:6827/2123819943] boot 2021-03-10T13:46:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[13015]: cluster 2021-03-10T13:46:05.211442+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2021-03-10T13:46:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:06 gibba011 bash[13015]: audit 2021-03-10T13:46:05.211510+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:46:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: cluster 2021-03-10T13:46:06.226782+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2021-03-10T13:46:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: audit 2021-03-10T13:46:07.051205+0000 mon.b (mon.2) 6 : audit [INF] from='client.? 172.21.2.132:0/1342785789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: audit 2021-03-10T13:46:07.051371+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: audit 2021-03-10T13:46:07.062923+0000 mon.a (mon.0) 331 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]': finished 2021-03-10T13:46:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: cluster 2021-03-10T13:46:07.063005+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e28: 5 total, 4 up, 4 in 2021-03-10T13:46:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:07 gibba032 bash[9829]: audit 2021-03-10T13:46:07.063182+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: cluster 2021-03-10T13:46:06.226782+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2021-03-10T13:46:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: audit 2021-03-10T13:46:07.051205+0000 mon.b (mon.2) 6 : audit [INF] from='client.? 172.21.2.132:0/1342785789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: audit 2021-03-10T13:46:07.051371+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: audit 2021-03-10T13:46:07.062923+0000 mon.a (mon.0) 331 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]': finished 2021-03-10T13:46:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: cluster 2021-03-10T13:46:07.063005+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e28: 5 total, 4 up, 4 in 2021-03-10T13:46:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[9813]: audit 2021-03-10T13:46:07.063182+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: cluster 2021-03-10T13:46:06.226782+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2021-03-10T13:46:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: audit 2021-03-10T13:46:07.051205+0000 mon.b (mon.2) 6 : audit [INF] from='client.? 172.21.2.132:0/1342785789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: audit 2021-03-10T13:46:07.051371+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]: dispatch 2021-03-10T13:46:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: audit 2021-03-10T13:46:07.062923+0000 mon.a (mon.0) 331 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d2b345-b3a1-4ff1-bb20-44ec0c354123"}]': finished 2021-03-10T13:46:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: cluster 2021-03-10T13:46:07.063005+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e28: 5 total, 4 up, 4 in 2021-03-10T13:46:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:07 gibba011 bash[13015]: audit 2021-03-10T13:46:07.063182+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:08 gibba032 bash[9829]: cluster 2021-03-10T13:46:07.088494+0000 mgr.y (mgr.14140) 105 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 840 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:08 gibba011 bash[9813]: cluster 2021-03-10T13:46:07.088494+0000 mgr.y (mgr.14140) 105 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 840 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:08 gibba011 bash[13015]: cluster 2021-03-10T13:46:07.088494+0000 mgr.y (mgr.14140) 105 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 840 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:09.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:09 gibba032 bash[9829]: audit 2021-03-10T13:46:09.141533+0000 mon.b (mon.2) 7 : audit [DBG] from='client.? 172.21.2.132:0/3060128125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:09 gibba011 bash[9813]: audit 2021-03-10T13:46:09.141533+0000 mon.b (mon.2) 7 : audit [DBG] from='client.? 172.21.2.132:0/3060128125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:09.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:09 gibba011 bash[13015]: audit 2021-03-10T13:46:09.141533+0000 mon.b (mon.2) 7 : audit [DBG] from='client.? 172.21.2.132:0/3060128125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:10 gibba011 bash[9813]: cluster 2021-03-10T13:46:09.089345+0000 mgr.y (mgr.14140) 106 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 912 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:10 gibba011 bash[13015]: cluster 2021-03-10T13:46:09.089345+0000 mgr.y (mgr.14140) 106 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 912 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:10 gibba032 bash[9829]: cluster 2021-03-10T13:46:09.089345+0000 mgr.y (mgr.14140) 106 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 912 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:12.528 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:12 gibba032 bash[9829]: cluster 2021-03-10T13:46:11.090456+0000 mgr.y (mgr.14140) 107 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 936 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:12.528 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:12 gibba032 bash[9829]: audit 2021-03-10T13:46:11.772815+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:12 gibba011 bash[9813]: cluster 2021-03-10T13:46:11.090456+0000 mgr.y (mgr.14140) 107 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 936 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:12 gibba011 bash[9813]: audit 2021-03-10T13:46:11.772815+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:12 gibba011 bash[13015]: cluster 2021-03-10T13:46:11.090456+0000 mgr.y (mgr.14140) 107 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 936 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:12 gibba011 bash[13015]: audit 2021-03-10T13:46:11.772815+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:14 gibba011 bash[9813]: cluster 2021-03-10T13:46:13.091401+0000 mgr.y (mgr.14140) 108 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:14 gibba011 bash[13015]: cluster 2021-03-10T13:46:13.091401+0000 mgr.y (mgr.14140) 108 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:14 gibba032 bash[9829]: cluster 2021-03-10T13:46:13.091401+0000 mgr.y (mgr.14140) 108 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:15.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:15 gibba032 bash[9829]: audit 2021-03-10T13:46:14.910947+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2021-03-10T13:46:15.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:15 gibba032 bash[9829]: audit 2021-03-10T13:46:14.913958+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:15 gibba032 bash[9829]: audit 2021-03-10T13:46:14.917442+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2021-03-10T13:46:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[9813]: audit 2021-03-10T13:46:14.910947+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2021-03-10T13:46:15.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[9813]: audit 2021-03-10T13:46:14.913958+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:15.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[9813]: audit 2021-03-10T13:46:14.917442+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2021-03-10T13:46:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[13015]: audit 2021-03-10T13:46:14.910947+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2021-03-10T13:46:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[13015]: audit 2021-03-10T13:46:14.913958+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:15 gibba011 bash[13015]: audit 2021-03-10T13:46:14.917442+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2021-03-10T13:46:16.558 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:16 gibba032 bash[9829]: cephadm 2021-03-10T13:46:14.916456+0000 mgr.y (mgr.14140) 109 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:46:16.558 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:16 gibba032 bash[9829]: cluster 2021-03-10T13:46:15.092077+0000 mgr.y (mgr.14140) 110 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:16 gibba011 bash[9813]: cephadm 2021-03-10T13:46:14.916456+0000 mgr.y (mgr.14140) 109 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:46:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:16 gibba011 bash[9813]: cluster 2021-03-10T13:46:15.092077+0000 mgr.y (mgr.14140) 110 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:16 gibba011 bash[13015]: cephadm 2021-03-10T13:46:14.916456+0000 mgr.y (mgr.14140) 109 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:46:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:16 gibba011 bash[13015]: cluster 2021-03-10T13:46:15.092077+0000 mgr.y (mgr.14140) 110 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:16.721 INFO:teuthology.orchestra.run.gibba032.stdout:Created osd(s) 4 on host 'gibba032' 2021-03-10T13:46:17.367 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:16.709849+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.367 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:16.711297+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:17.368 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:16.718482+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.368 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:17.128905+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.369 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:17.132997+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.369 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:17 gibba032 bash[9829]: audit 2021-03-10T13:46:17.133794+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:16.709849+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:16.711297+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:16.709849+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:16.711297+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:17.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:16.718482+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:16.718482+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002079\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002342\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:44:34.002508\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_device_update\": \"2021-03-10T13:43:49.102128\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:17.128905+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:17.132997+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[9813]: audit 2021-03-10T13:46:17.133794+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:17.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:17.128905+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:17.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:17.132997+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:17.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:17 gibba011 bash[13015]: audit 2021-03-10T13:46:17.133794+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:18.150 DEBUG:teuthology.orchestra.run.gibba032:osd.4> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.4.service 2021-03-10T13:46:18.152 INFO:tasks.cephadm:Deploying osd.5 on gibba032 with /dev/vg_nvme/lv_3... 2021-03-10T13:46:18.152 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2021-03-10T13:46:18.189 INFO:journalctl@ceph.osd.4.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:46:18.566 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:18 gibba032 bash[9829]: cluster 2021-03-10T13:46:17.092602+0000 mgr.y (mgr.14140) 111 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:18 gibba011 bash[9813]: cluster 2021-03-10T13:46:17.092602+0000 mgr.y (mgr.14140) 111 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:18 gibba011 bash[13015]: cluster 2021-03-10T13:46:17.092602+0000 mgr.y (mgr.14140) 111 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:19.506 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-03-10T13:46:19.506 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2021-03-10T13:46:19.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2021-03-10T13:46:19.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2021-03-10T13:46:19.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-03-10T13:46:19.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-03-10T13:46:19.508 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:19 gibba032 bash[11905]: --> ceph-volume lvm activate successful for osd ID: 4 2021-03-10T13:46:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:20 gibba011 bash[9813]: cluster 2021-03-10T13:46:19.093412+0000 mgr.y (mgr.14140) 112 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:20 gibba011 bash[13015]: cluster 2021-03-10T13:46:19.093412+0000 mgr.y (mgr.14140) 112 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:20.597 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:20 gibba032 bash[9829]: cluster 2021-03-10T13:46:19.093412+0000 mgr.y (mgr.14140) 112 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:22 gibba011 bash[9813]: cluster 2021-03-10T13:46:21.094377+0000 mgr.y (mgr.14140) 113 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:22 gibba011 bash[13015]: cluster 2021-03-10T13:46:21.094377+0000 mgr.y (mgr.14140) 113 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:22.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:22 gibba032 bash[9829]: cluster 2021-03-10T13:46:21.094377+0000 mgr.y (mgr.14140) 113 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:23.255 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:22 gibba032 bash[12211]: debug 2021-03-10T13:46:22.814+0000 7fa6c057eec0 -1 Falling back to public interface 2021-03-10T13:46:23.474 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2021-03-10T13:46:23.475 INFO:teuthology.orchestra.run.gibba032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2021-03-10T13:46:23.475 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 10+0 records in 2021-03-10T13:46:23.475 INFO:teuthology.orchestra.run.gibba032.stderr:10+0 records out 2021-03-10T13:46:23.475 INFO:teuthology.orchestra.run.gibba032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00890631 s, 1.2 GB/s 2021-03-10T13:46:23.475 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 2021-03-10T13:46:23.476 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping successful for: 2021-03-10T13:46:23.756 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:23 gibba032 bash[12211]: debug 2021-03-10T13:46:23.462+0000 7fa6c057eec0 -1 osd.4 0 log_to_monitors {default=true} 2021-03-10T13:46:24.272 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba032:vg_nvme/lv_3 2021-03-10T13:46:24.384 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:24 gibba032 bash[9829]: cluster 2021-03-10T13:46:23.095490+0000 mgr.y (mgr.14140) 114 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:24.385 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:24 gibba032 bash[9829]: audit 2021-03-10T13:46:23.468044+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:24.385 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:24 gibba032 bash[9829]: audit 2021-03-10T13:46:23.468256+0000 mon.b (mon.2) 8 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[9813]: cluster 2021-03-10T13:46:23.095490+0000 mgr.y (mgr.14140) 114 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[9813]: audit 2021-03-10T13:46:23.468044+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[9813]: audit 2021-03-10T13:46:23.468256+0000 mon.b (mon.2) 8 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[13015]: cluster 2021-03-10T13:46:23.095490+0000 mgr.y (mgr.14140) 114 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[13015]: audit 2021-03-10T13:46:23.468044+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:24 gibba011 bash[13015]: audit 2021-03-10T13:46:23.468256+0000 mon.b (mon.2) 8 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:46:25.585 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[12211]: debug 2021-03-10T13:46:25.342+0000 7fa6a9333700 -1 osd.4 0 waiting for initial osdmap 2021-03-10T13:46:25.585 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[12211]: debug 2021-03-10T13:46:25.350+0000 7fa6b454f700 -1 osd.4 30 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:46:25.586 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:24.335304+0000 mon.a (mon.0) 345 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:46:25.586 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: cluster 2021-03-10T13:46:24.335365+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e29: 5 total, 4 up, 4 in 2021-03-10T13:46:25.586 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:24.335517+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:25.587 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:24.336228+0000 mon.a (mon.0) 348 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.587 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:24.336984+0000 mon.b (mon.2) 9 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.587 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:25.023977+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:25.587 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:25.134335+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:25.588 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:25 gibba032 bash[9829]: audit 2021-03-10T13:46:25.142921+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:25.684 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:24.335304+0000 mon.a (mon.0) 345 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:46:25.685 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: cluster 2021-03-10T13:46:24.335365+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e29: 5 total, 4 up, 4 in 2021-03-10T13:46:25.685 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:24.335517+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:25.685 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:24.336228+0000 mon.a (mon.0) 348 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.685 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:24.336984+0000 mon.b (mon.2) 9 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.685 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:25.023977+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:25.686 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:24.335304+0000 mon.a (mon.0) 345 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:46:25.686 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: cluster 2021-03-10T13:46:24.335365+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e29: 5 total, 4 up, 4 in 2021-03-10T13:46:25.686 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:24.335517+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:25.687 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:24.336228+0000 mon.a (mon.0) 348 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.687 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:24.336984+0000 mon.b (mon.2) 9 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:25.687 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:25.023977+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:25.688 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:25.134335+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:25.688 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[9813]: audit 2021-03-10T13:46:25.142921+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:25.688 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:25.134335+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:25.689 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:25 gibba011 bash[13015]: audit 2021-03-10T13:46:25.142921+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}}, \"last_daemon_update\": \"2021-03-10T13:46:17.128136\", \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:26.695 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:24.508827+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:26.696 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:24.508924+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:26.696 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:25.096009+0000 mgr.y (mgr.14140) 115 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:26.696 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: audit 2021-03-10T13:46:25.344499+0000 mon.a (mon.0) 352 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:26.696 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:25.344546+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 4 up, 4 in 2021-03-10T13:46:26.697 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: audit 2021-03-10T13:46:25.345044+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:26.697 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:26.348590+0000 mon.a (mon.0) 355 : cluster [INF] osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226] boot 2021-03-10T13:46:26.697 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:26 gibba032 bash[9829]: cluster 2021-03-10T13:46:26.348627+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2021-03-10T13:46:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:24.508827+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:24.508924+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:25.096009+0000 mgr.y (mgr.14140) 115 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: audit 2021-03-10T13:46:25.344499+0000 mon.a (mon.0) 352 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:25.344546+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 4 up, 4 in 2021-03-10T13:46:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: audit 2021-03-10T13:46:25.345044+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:26.348590+0000 mon.a (mon.0) 355 : cluster [INF] osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226] boot 2021-03-10T13:46:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[9813]: cluster 2021-03-10T13:46:26.348627+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2021-03-10T13:46:26.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:24.508827+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:26.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:24.508924+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:26.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:25.096009+0000 mgr.y (mgr.14140) 115 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 960 KiB used, 332 GiB / 336 GiB avail 2021-03-10T13:46:26.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: audit 2021-03-10T13:46:25.344499+0000 mon.a (mon.0) 352 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:26.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:25.344546+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 4 up, 4 in 2021-03-10T13:46:26.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: audit 2021-03-10T13:46:25.345044+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:26.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:26.348590+0000 mon.a (mon.0) 355 : cluster [INF] osd.4 [v2:172.21.2.132:6800/337025226,v1:172.21.2.132:6801/337025226] boot 2021-03-10T13:46:26.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:26 gibba011 bash[13015]: cluster 2021-03-10T13:46:26.348627+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2021-03-10T13:46:27.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:27 gibba032 bash[9829]: audit 2021-03-10T13:46:26.348715+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:27.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:27 gibba032 bash[9829]: audit 2021-03-10T13:46:26.975857+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:27 gibba032 bash[9829]: audit 2021-03-10T13:46:26.978994+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:27 gibba032 bash[9829]: audit 2021-03-10T13:46:26.983398+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:27 gibba032 bash[9829]: cluster 2021-03-10T13:46:27.353797+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2021-03-10T13:46:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[9813]: audit 2021-03-10T13:46:26.348715+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[9813]: audit 2021-03-10T13:46:26.975857+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[9813]: audit 2021-03-10T13:46:26.978994+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:27.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[9813]: audit 2021-03-10T13:46:26.983398+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:27.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[9813]: cluster 2021-03-10T13:46:27.353797+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2021-03-10T13:46:27.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[13015]: audit 2021-03-10T13:46:26.348715+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:46:27.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[13015]: audit 2021-03-10T13:46:26.975857+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:27.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[13015]: audit 2021-03-10T13:46:26.978994+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:27.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[13015]: audit 2021-03-10T13:46:26.983398+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:27.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:27 gibba011 bash[13015]: cluster 2021-03-10T13:46:27.353797+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2021-03-10T13:46:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:28 gibba032 bash[9829]: audit 2021-03-10T13:46:26.972710+0000 mgr.y (mgr.14140) 116 : audit [DBG] from='client.34151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:28 gibba032 bash[9829]: cephadm 2021-03-10T13:46:26.974959+0000 mgr.y (mgr.14140) 117 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:28 gibba032 bash[9829]: cluster 2021-03-10T13:46:27.096520+0000 mgr.y (mgr.14140) 118 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 1.2 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[9813]: audit 2021-03-10T13:46:26.972710+0000 mgr.y (mgr.14140) 116 : audit [DBG] from='client.34151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[9813]: cephadm 2021-03-10T13:46:26.974959+0000 mgr.y (mgr.14140) 117 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[9813]: cluster 2021-03-10T13:46:27.096520+0000 mgr.y (mgr.14140) 118 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 1.2 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[13015]: audit 2021-03-10T13:46:26.972710+0000 mgr.y (mgr.14140) 116 : audit [DBG] from='client.34151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[13015]: cephadm 2021-03-10T13:46:26.974959+0000 mgr.y (mgr.14140) 117 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:28 gibba011 bash[13015]: cluster 2021-03-10T13:46:27.096520+0000 mgr.y (mgr.14140) 118 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 1.2 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: cluster 2021-03-10T13:46:29.097294+0000 mgr.y (mgr.14140) 119 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: audit 2021-03-10T13:46:29.944380+0000 mon.a (mon.0) 362 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: audit 2021-03-10T13:46:29.944394+0000 mon.b (mon.2) 10 : audit [INF] from='client.? 172.21.2.132:0/2564455724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: audit 2021-03-10T13:46:29.956091+0000 mon.a (mon.0) 363 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]': finished 2021-03-10T13:46:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: cluster 2021-03-10T13:46:29.956181+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e33: 6 total, 5 up, 5 in 2021-03-10T13:46:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:30 gibba032 bash[9829]: audit 2021-03-10T13:46:29.956351+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: cluster 2021-03-10T13:46:29.097294+0000 mgr.y (mgr.14140) 119 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: audit 2021-03-10T13:46:29.944380+0000 mon.a (mon.0) 362 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: audit 2021-03-10T13:46:29.944394+0000 mon.b (mon.2) 10 : audit [INF] from='client.? 172.21.2.132:0/2564455724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: audit 2021-03-10T13:46:29.956091+0000 mon.a (mon.0) 363 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]': finished 2021-03-10T13:46:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: cluster 2021-03-10T13:46:29.956181+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e33: 6 total, 5 up, 5 in 2021-03-10T13:46:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[9813]: audit 2021-03-10T13:46:29.956351+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: cluster 2021-03-10T13:46:29.097294+0000 mgr.y (mgr.14140) 119 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: audit 2021-03-10T13:46:29.944380+0000 mon.a (mon.0) 362 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: audit 2021-03-10T13:46:29.944394+0000 mon.b (mon.2) 10 : audit [INF] from='client.? 172.21.2.132:0/2564455724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]: dispatch 2021-03-10T13:46:30.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: audit 2021-03-10T13:46:29.956091+0000 mon.a (mon.0) 363 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8682e3b2-5b44-47b9-8181-83be4f601dfe"}]': finished 2021-03-10T13:46:30.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: cluster 2021-03-10T13:46:29.956181+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e33: 6 total, 5 up, 5 in 2021-03-10T13:46:30.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:30 gibba011 bash[13015]: audit 2021-03-10T13:46:29.956351+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:32 gibba032 bash[9829]: cluster 2021-03-10T13:46:31.098235+0000 mgr.y (mgr.14140) 120 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:32 gibba032 bash[9829]: audit 2021-03-10T13:46:31.838057+0000 mon.b (mon.2) 11 : audit [DBG] from='client.? 172.21.2.132:0/622337074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:32 gibba011 bash[9813]: cluster 2021-03-10T13:46:31.098235+0000 mgr.y (mgr.14140) 120 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:32 gibba011 bash[9813]: audit 2021-03-10T13:46:31.838057+0000 mon.b (mon.2) 11 : audit [DBG] from='client.? 172.21.2.132:0/622337074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:32 gibba011 bash[13015]: cluster 2021-03-10T13:46:31.098235+0000 mgr.y (mgr.14140) 120 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.3 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:32 gibba011 bash[13015]: audit 2021-03-10T13:46:31.838057+0000 mon.b (mon.2) 11 : audit [DBG] from='client.? 172.21.2.132:0/622337074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:34 gibba032 bash[9829]: cluster 2021-03-10T13:46:33.099247+0000 mgr.y (mgr.14140) 121 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:34 gibba011 bash[9813]: cluster 2021-03-10T13:46:33.099247+0000 mgr.y (mgr.14140) 121 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:34 gibba011 bash[13015]: cluster 2021-03-10T13:46:33.099247+0000 mgr.y (mgr.14140) 121 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:35.577 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:35 gibba032 bash[9829]: audit 2021-03-10T13:46:34.404885+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:35 gibba011 bash[9813]: audit 2021-03-10T13:46:34.404885+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:35.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:35 gibba011 bash[13015]: audit 2021-03-10T13:46:34.404885+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:36.679 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:36 gibba032 bash[9829]: cluster 2021-03-10T13:46:35.100013+0000 mgr.y (mgr.14140) 122 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:36 gibba011 bash[9813]: cluster 2021-03-10T13:46:35.100013+0000 mgr.y (mgr.14140) 122 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:36 gibba011 bash[13015]: cluster 2021-03-10T13:46:35.100013+0000 mgr.y (mgr.14140) 122 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:38.752 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:38 gibba032 bash[9829]: cluster 2021-03-10T13:46:37.100782+0000 mgr.y (mgr.14140) 123 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:38.753 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:38 gibba032 bash[9829]: audit 2021-03-10T13:46:37.724291+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2021-03-10T13:46:38.753 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:38 gibba032 bash[9829]: audit 2021-03-10T13:46:37.727281+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:38.753 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:38 gibba032 bash[9829]: audit 2021-03-10T13:46:37.730530+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2021-03-10T13:46:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[9813]: cluster 2021-03-10T13:46:37.100782+0000 mgr.y (mgr.14140) 123 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[9813]: audit 2021-03-10T13:46:37.724291+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2021-03-10T13:46:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[9813]: audit 2021-03-10T13:46:37.727281+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[9813]: audit 2021-03-10T13:46:37.730530+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2021-03-10T13:46:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[13015]: cluster 2021-03-10T13:46:37.100782+0000 mgr.y (mgr.14140) 123 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[13015]: audit 2021-03-10T13:46:37.724291+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2021-03-10T13:46:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[13015]: audit 2021-03-10T13:46:37.727281+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:38 gibba011 bash[13015]: audit 2021-03-10T13:46:37.730530+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2021-03-10T13:46:39.570 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:39 gibba032 bash[9829]: cephadm 2021-03-10T13:46:37.729687+0000 mgr.y (mgr.14140) 124 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:46:39.615 INFO:teuthology.orchestra.run.gibba032.stdout:Created osd(s) 5 on host 'gibba032' 2021-03-10T13:46:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:39 gibba011 bash[9813]: cephadm 2021-03-10T13:46:37.729687+0000 mgr.y (mgr.14140) 124 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:46:39.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:39 gibba011 bash[13015]: cephadm 2021-03-10T13:46:37.729687+0000 mgr.y (mgr.14140) 124 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:46:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: cluster 2021-03-10T13:46:39.101832+0000 mgr.y (mgr.14140) 125 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:39.603262+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:39.605030+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:39.612258+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:40.088834+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:40.097620+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:40 gibba032 bash[9829]: audit 2021-03-10T13:46:40.098873+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: cluster 2021-03-10T13:46:39.101832+0000 mgr.y (mgr.14140) 125 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: cluster 2021-03-10T13:46:39.101832+0000 mgr.y (mgr.14140) 125 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:39.603262+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:39.605030+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:39.603262+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:39.605030+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:39.612258+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:39.612258+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127849\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.127946\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128000\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:17.128050\", \"created\": \"2021-03-10T13:46:16.698595\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_device_update\": \"2021-03-10T13:46:25.132220\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:40.088834+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:40.097620+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[9813]: audit 2021-03-10T13:46:40.098873+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:40.088834+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:40.097620+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:40 gibba011 bash[13015]: audit 2021-03-10T13:46:40.098873+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:41.025 DEBUG:teuthology.orchestra.run.gibba032:osd.5> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.5.service 2021-03-10T13:46:41.027 INFO:tasks.cephadm:Deploying osd.6 on gibba032 with /dev/vg_nvme/lv_2... 2021-03-10T13:46:41.028 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2021-03-10T13:46:41.071 INFO:journalctl@ceph.osd.5.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:46:42.303 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-03-10T13:46:42.303 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2021-03-10T13:46:42.304 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5/block 2021-03-10T13:46:42.304 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2021-03-10T13:46:42.304 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-03-10T13:46:42.304 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-03-10T13:46:42.304 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[13359]: --> ceph-volume lvm activate successful for osd ID: 5 2021-03-10T13:46:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[9829]: cluster 2021-03-10T13:46:41.102862+0000 mgr.y (mgr.14140) 126 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[9829]: audit 2021-03-10T13:46:42.091723+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:46:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:42 gibba032 bash[9829]: audit 2021-03-10T13:46:42.100248+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:46:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[9813]: cluster 2021-03-10T13:46:41.102862+0000 mgr.y (mgr.14140) 126 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[9813]: audit 2021-03-10T13:46:42.091723+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:46:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[9813]: audit 2021-03-10T13:46:42.100248+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:46:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[13015]: cluster 2021-03-10T13:46:41.102862+0000 mgr.y (mgr.14140) 126 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2021-03-10T13:46:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[13015]: audit 2021-03-10T13:46:42.091723+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:46:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:42 gibba011 bash[13015]: audit 2021-03-10T13:46:42.100248+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:46:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:44 gibba032 bash[9829]: cluster 2021-03-10T13:46:43.103771+0000 mgr.y (mgr.14140) 127 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:44 gibba011 bash[9813]: cluster 2021-03-10T13:46:43.103771+0000 mgr.y (mgr.14140) 127 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:44 gibba011 bash[13015]: cluster 2021-03-10T13:46:43.103771+0000 mgr.y (mgr.14140) 127 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:46.314 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2021-03-10T13:46:46.314 INFO:teuthology.orchestra.run.gibba032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2021-03-10T13:46:46.314 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 10+0 records in 2021-03-10T13:46:46.315 INFO:teuthology.orchestra.run.gibba032.stderr:10+0 records out 2021-03-10T13:46:46.315 INFO:teuthology.orchestra.run.gibba032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0273753 s, 383 MB/s 2021-03-10T13:46:46.315 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping successful for: 2021-03-10T13:46:46.383 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:46 gibba032 bash[9829]: cluster 2021-03-10T13:46:45.104465+0000 mgr.y (mgr.14140) 128 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:46.384 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:46 gibba032 bash[13724]: debug 2021-03-10T13:46:46.214+0000 7f4d8d311ec0 -1 Falling back to public interface 2021-03-10T13:46:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:46 gibba011 bash[9813]: cluster 2021-03-10T13:46:45.104465+0000 mgr.y (mgr.14140) 128 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:46 gibba011 bash[13015]: cluster 2021-03-10T13:46:45.104465+0000 mgr.y (mgr.14140) 128 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:47.083 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba032:vg_nvme/lv_2 2021-03-10T13:46:47.099 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:46 gibba032 bash[13724]: debug 2021-03-10T13:46:46.914+0000 7f4d8d311ec0 -1 osd.5 0 log_to_monitors {default=true} 2021-03-10T13:46:47.404 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:47 gibba032 bash[9829]: audit 2021-03-10T13:46:46.926167+0000 mon.b (mon.2) 12 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:47.405 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:47 gibba032 bash[9829]: audit 2021-03-10T13:46:46.926862+0000 mon.a (mon.0) 378 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:47 gibba011 bash[9813]: audit 2021-03-10T13:46:46.926167+0000 mon.b (mon.2) 12 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:47.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:47 gibba011 bash[9813]: audit 2021-03-10T13:46:46.926862+0000 mon.a (mon.0) 378 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:47 gibba011 bash[13015]: audit 2021-03-10T13:46:46.926167+0000 mon.b (mon.2) 12 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:47 gibba011 bash[13015]: audit 2021-03-10T13:46:46.926862+0000 mon.a (mon.0) 378 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:46:48.443 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[13724]: debug 2021-03-10T13:46:48.142+0000 7f4d760c6700 -1 osd.5 0 waiting for initial osdmap 2021-03-10T13:46:48.444 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[13724]: debug 2021-03-10T13:46:48.146+0000 7f4d812e2700 -1 osd.5 35 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:46:48.445 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: cluster 2021-03-10T13:46:47.105435+0000 mgr.y (mgr.14140) 129 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:48.445 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.130985+0000 mon.a (mon.0) 379 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:46:48.446 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: cluster 2021-03-10T13:46:47.131010+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e34: 6 total, 5 up, 5 in 2021-03-10T13:46:48.446 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.131077+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:48.447 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.137662+0000 mon.b (mon.2) 13 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.447 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.137782+0000 mon.a (mon.0) 382 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.448 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.869356+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:48.448 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.976928+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:48.449 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:48 gibba032 bash[9829]: audit 2021-03-10T13:46:47.982560+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: cluster 2021-03-10T13:46:47.105435+0000 mgr.y (mgr.14140) 129 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.130985+0000 mon.a (mon.0) 379 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:46:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: cluster 2021-03-10T13:46:47.131010+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e34: 6 total, 5 up, 5 in 2021-03-10T13:46:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.131077+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.137662+0000 mon.b (mon.2) 13 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.137782+0000 mon.a (mon.0) 382 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.869356+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: cluster 2021-03-10T13:46:47.105435+0000 mgr.y (mgr.14140) 129 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.130985+0000 mon.a (mon.0) 379 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:46:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: cluster 2021-03-10T13:46:47.131010+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e34: 6 total, 5 up, 5 in 2021-03-10T13:46:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.131077+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.137662+0000 mon.b (mon.2) 13 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.137782+0000 mon.a (mon.0) 382 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:46:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.869356+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:46:48.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.976928+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:48.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[9813]: audit 2021-03-10T13:46:47.982560+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:48.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.976928+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:46:48.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:48 gibba011 bash[13015]: audit 2021-03-10T13:46:47.982560+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}}, \"last_daemon_update\": \"2021-03-10T13:46:40.086544\", \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:46:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:49 gibba032 bash[9829]: cluster 2021-03-10T13:46:47.943604+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:49 gibba032 bash[9829]: cluster 2021-03-10T13:46:47.943639+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:49 gibba032 bash[9829]: audit 2021-03-10T13:46:48.140820+0000 mon.a (mon.0) 386 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:49 gibba032 bash[9829]: cluster 2021-03-10T13:46:48.140841+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e35: 6 total, 5 up, 5 in 2021-03-10T13:46:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:49 gibba032 bash[9829]: audit 2021-03-10T13:46:48.141589+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[9813]: cluster 2021-03-10T13:46:47.943604+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[9813]: cluster 2021-03-10T13:46:47.943639+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[9813]: audit 2021-03-10T13:46:48.140820+0000 mon.a (mon.0) 386 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[9813]: cluster 2021-03-10T13:46:48.140841+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e35: 6 total, 5 up, 5 in 2021-03-10T13:46:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[9813]: audit 2021-03-10T13:46:48.141589+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[13015]: cluster 2021-03-10T13:46:47.943604+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:46:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[13015]: cluster 2021-03-10T13:46:47.943639+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:46:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[13015]: audit 2021-03-10T13:46:48.140820+0000 mon.a (mon.0) 386 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:46:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[13015]: cluster 2021-03-10T13:46:48.140841+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e35: 6 total, 5 up, 5 in 2021-03-10T13:46:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:49 gibba011 bash[13015]: audit 2021-03-10T13:46:48.141589+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: cluster 2021-03-10T13:46:49.106377+0000 mgr.y (mgr.14140) 130 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: cluster 2021-03-10T13:46:49.145990+0000 mon.a (mon.0) 389 : cluster [INF] osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493] boot 2021-03-10T13:46:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: cluster 2021-03-10T13:46:49.146016+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2021-03-10T13:46:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: audit 2021-03-10T13:46:49.146169+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: audit 2021-03-10T13:46:49.878200+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: audit 2021-03-10T13:46:49.881249+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:50 gibba032 bash[9829]: audit 2021-03-10T13:46:49.885540+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: cluster 2021-03-10T13:46:49.106377+0000 mgr.y (mgr.14140) 130 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: cluster 2021-03-10T13:46:49.145990+0000 mon.a (mon.0) 389 : cluster [INF] osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493] boot 2021-03-10T13:46:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: cluster 2021-03-10T13:46:49.146016+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2021-03-10T13:46:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: audit 2021-03-10T13:46:49.146169+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: audit 2021-03-10T13:46:49.878200+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: audit 2021-03-10T13:46:49.881249+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[9813]: audit 2021-03-10T13:46:49.885540+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: cluster 2021-03-10T13:46:49.106377+0000 mgr.y (mgr.14140) 130 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.4 MiB used, 415 GiB / 420 GiB avail 2021-03-10T13:46:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: cluster 2021-03-10T13:46:49.145990+0000 mon.a (mon.0) 389 : cluster [INF] osd.5 [v2:172.21.2.132:6808/4109259493,v1:172.21.2.132:6809/4109259493] boot 2021-03-10T13:46:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: cluster 2021-03-10T13:46:49.146016+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2021-03-10T13:46:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: audit 2021-03-10T13:46:49.146169+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:46:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: audit 2021-03-10T13:46:49.878200+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:46:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: audit 2021-03-10T13:46:49.881249+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:46:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:50 gibba011 bash[13015]: audit 2021-03-10T13:46:49.885540+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:51.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:51 gibba032 bash[9829]: audit 2021-03-10T13:46:49.875435+0000 mgr.y (mgr.14140) 131 : audit [DBG] from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:51 gibba032 bash[9829]: cephadm 2021-03-10T13:46:49.877379+0000 mgr.y (mgr.14140) 132 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:51 gibba032 bash[9829]: cluster 2021-03-10T13:46:50.163934+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2021-03-10T13:46:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[9813]: audit 2021-03-10T13:46:49.875435+0000 mgr.y (mgr.14140) 131 : audit [DBG] from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[9813]: cephadm 2021-03-10T13:46:49.877379+0000 mgr.y (mgr.14140) 132 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[9813]: cluster 2021-03-10T13:46:50.163934+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2021-03-10T13:46:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[13015]: audit 2021-03-10T13:46:49.875435+0000 mgr.y (mgr.14140) 131 : audit [DBG] from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:46:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[13015]: cephadm 2021-03-10T13:46:49.877379+0000 mgr.y (mgr.14140) 132 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:46:51.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:51 gibba011 bash[13015]: cluster 2021-03-10T13:46:50.163934+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2021-03-10T13:46:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:52 gibba032 bash[9829]: cluster 2021-03-10T13:46:51.107395+0000 mgr.y (mgr.14140) 133 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.7 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:52 gibba011 bash[9813]: cluster 2021-03-10T13:46:51.107395+0000 mgr.y (mgr.14140) 133 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.7 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:52 gibba011 bash[13015]: cluster 2021-03-10T13:46:51.107395+0000 mgr.y (mgr.14140) 133 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.7 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:53 gibba032 bash[9829]: audit 2021-03-10T13:46:52.891864+0000 mon.b (mon.2) 14 : audit [INF] from='client.? 172.21.2.132:0/3815153604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:53 gibba032 bash[9829]: audit 2021-03-10T13:46:52.892830+0000 mon.a (mon.0) 396 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:53 gibba032 bash[9829]: audit 2021-03-10T13:46:52.904697+0000 mon.a (mon.0) 397 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]': finished 2021-03-10T13:46:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:53 gibba032 bash[9829]: cluster 2021-03-10T13:46:52.904738+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e38: 7 total, 6 up, 6 in 2021-03-10T13:46:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:53 gibba032 bash[9829]: audit 2021-03-10T13:46:52.904903+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:46:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[9813]: audit 2021-03-10T13:46:52.891864+0000 mon.b (mon.2) 14 : audit [INF] from='client.? 172.21.2.132:0/3815153604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[9813]: audit 2021-03-10T13:46:52.892830+0000 mon.a (mon.0) 396 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[9813]: audit 2021-03-10T13:46:52.904697+0000 mon.a (mon.0) 397 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]': finished 2021-03-10T13:46:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[9813]: cluster 2021-03-10T13:46:52.904738+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e38: 7 total, 6 up, 6 in 2021-03-10T13:46:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[9813]: audit 2021-03-10T13:46:52.904903+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:46:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[13015]: audit 2021-03-10T13:46:52.891864+0000 mon.b (mon.2) 14 : audit [INF] from='client.? 172.21.2.132:0/3815153604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[13015]: audit 2021-03-10T13:46:52.892830+0000 mon.a (mon.0) 396 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]: dispatch 2021-03-10T13:46:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[13015]: audit 2021-03-10T13:46:52.904697+0000 mon.a (mon.0) 397 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95e33995-e46c-4204-9623-3f2b7b02beb8"}]': finished 2021-03-10T13:46:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[13015]: cluster 2021-03-10T13:46:52.904738+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e38: 7 total, 6 up, 6 in 2021-03-10T13:46:53.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:53 gibba011 bash[13015]: audit 2021-03-10T13:46:52.904903+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:46:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:54 gibba032 bash[9829]: cluster 2021-03-10T13:46:53.108089+0000 mgr.y (mgr.14140) 134 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.9 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:54 gibba011 bash[9813]: cluster 2021-03-10T13:46:53.108089+0000 mgr.y (mgr.14140) 134 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.9 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:54 gibba011 bash[13015]: cluster 2021-03-10T13:46:53.108089+0000 mgr.y (mgr.14140) 134 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.9 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:55.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:55 gibba032 bash[9829]: audit 2021-03-10T13:46:54.923330+0000 mon.b (mon.2) 15 : audit [DBG] from='client.? 172.21.2.132:0/3444061475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:55 gibba011 bash[9813]: audit 2021-03-10T13:46:54.923330+0000 mon.b (mon.2) 15 : audit [DBG] from='client.? 172.21.2.132:0/3444061475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:55 gibba011 bash[13015]: audit 2021-03-10T13:46:54.923330+0000 mon.b (mon.2) 15 : audit [DBG] from='client.? 172.21.2.132:0/3444061475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:46:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:56 gibba032 bash[9829]: cluster 2021-03-10T13:46:55.108761+0000 mgr.y (mgr.14140) 135 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:56 gibba011 bash[9813]: cluster 2021-03-10T13:46:55.108761+0000 mgr.y (mgr.14140) 135 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:56 gibba011 bash[13015]: cluster 2021-03-10T13:46:55.108761+0000 mgr.y (mgr.14140) 135 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:58.502 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:58 gibba032 bash[9829]: cluster 2021-03-10T13:46:57.109742+0000 mgr.y (mgr.14140) 136 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:58.503 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:46:58 gibba032 bash[9829]: audit 2021-03-10T13:46:57.467171+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:58 gibba011 bash[9813]: cluster 2021-03-10T13:46:57.109742+0000 mgr.y (mgr.14140) 136 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:46:58 gibba011 bash[9813]: audit 2021-03-10T13:46:57.467171+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:46:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:58 gibba011 bash[13015]: cluster 2021-03-10T13:46:57.109742+0000 mgr.y (mgr.14140) 136 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:46:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:46:58 gibba011 bash[13015]: audit 2021-03-10T13:46:57.467171+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:00 gibba032 bash[9829]: cluster 2021-03-10T13:46:59.110778+0000 mgr.y (mgr.14140) 137 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:00 gibba011 bash[9813]: cluster 2021-03-10T13:46:59.110778+0000 mgr.y (mgr.14140) 137 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:00 gibba011 bash[13015]: cluster 2021-03-10T13:46:59.110778+0000 mgr.y (mgr.14140) 137 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:01 gibba032 bash[9829]: audit 2021-03-10T13:47:00.717364+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2021-03-10T13:47:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:01 gibba032 bash[9829]: audit 2021-03-10T13:47:00.720376+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:01 gibba032 bash[9829]: audit 2021-03-10T13:47:00.723732+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2021-03-10T13:47:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[9813]: audit 2021-03-10T13:47:00.717364+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2021-03-10T13:47:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[9813]: audit 2021-03-10T13:47:00.720376+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[9813]: audit 2021-03-10T13:47:00.723732+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2021-03-10T13:47:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[13015]: audit 2021-03-10T13:47:00.717364+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2021-03-10T13:47:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[13015]: audit 2021-03-10T13:47:00.720376+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:01 gibba011 bash[13015]: audit 2021-03-10T13:47:00.723732+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2021-03-10T13:47:02.441 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:02 gibba032 bash[9829]: cephadm 2021-03-10T13:47:00.722803+0000 mgr.y (mgr.14140) 138 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:47:02.441 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:02 gibba032 bash[9829]: cluster 2021-03-10T13:47:01.111798+0000 mgr.y (mgr.14140) 139 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:02.484 INFO:teuthology.orchestra.run.gibba032.stdout:Created osd(s) 6 on host 'gibba032' 2021-03-10T13:47:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:02 gibba011 bash[9813]: cephadm 2021-03-10T13:47:00.722803+0000 mgr.y (mgr.14140) 138 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:47:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:02 gibba011 bash[9813]: cluster 2021-03-10T13:47:01.111798+0000 mgr.y (mgr.14140) 139 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:02 gibba011 bash[13015]: cephadm 2021-03-10T13:47:00.722803+0000 mgr.y (mgr.14140) 138 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:47:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:02 gibba011 bash[13015]: cluster 2021-03-10T13:47:01.111798+0000 mgr.y (mgr.14140) 139 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:03.549 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:02.473230+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.550 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:02.474809+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.550 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:02.482202+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.551 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:03.002837+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.551 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:03.011535+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.551 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:03 gibba032 bash[9829]: audit 2021-03-10T13:47:03.012111+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:02.473230+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:02.474809+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:02.473230+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:02.474809+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:02.482202+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:02.482202+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085555\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085821\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.085986\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086138\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:46:40.086286\", \"created\": \"2021-03-10T13:46:39.590594\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_device_update\": \"2021-03-10T13:46:47.975731\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:03.002837+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:03.011535+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[9813]: audit 2021-03-10T13:47:03.012111+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:03.002837+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:03.011535+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:03 gibba011 bash[13015]: audit 2021-03-10T13:47:03.012111+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:03.727 DEBUG:teuthology.orchestra.run.gibba032:osd.6> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.6.service 2021-03-10T13:47:03.729 INFO:tasks.cephadm:Deploying osd.7 on gibba032 with /dev/vg_nvme/lv_1... 2021-03-10T13:47:03.730 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2021-03-10T13:47:03.791 INFO:journalctl@ceph.osd.6.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:04.542 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[9829]: cluster 2021-03-10T13:47:03.112810+0000 mgr.y (mgr.14140) 140 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:04 gibba011 bash[9813]: cluster 2021-03-10T13:47:03.112810+0000 mgr.y (mgr.14140) 140 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:04 gibba011 bash[13015]: cluster 2021-03-10T13:47:03.112810+0000 mgr.y (mgr.14140) 140 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:04.895 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-03-10T13:47:04.895 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-6 --no-mon-config 2021-03-10T13:47:04.896 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6/block 2021-03-10T13:47:04.896 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-6/block 2021-03-10T13:47:04.896 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-03-10T13:47:04.897 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-03-10T13:47:04.897 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:04 gibba032 bash[14801]: --> ceph-volume lvm activate successful for osd ID: 6 2021-03-10T13:47:06.428 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:06 gibba032 bash[9829]: cluster 2021-03-10T13:47:05.113503+0000 mgr.y (mgr.14140) 141 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:06 gibba011 bash[9813]: cluster 2021-03-10T13:47:05.113503+0000 mgr.y (mgr.14140) 141 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:06 gibba011 bash[13015]: cluster 2021-03-10T13:47:05.113503+0000 mgr.y (mgr.14140) 141 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:07 gibba032 bash[9829]: audit 2021-03-10T13:47:07.104247+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:07 gibba032 bash[9829]: audit 2021-03-10T13:47:07.113271+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:07 gibba011 bash[9813]: audit 2021-03-10T13:47:07.104247+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:07 gibba011 bash[9813]: audit 2021-03-10T13:47:07.113271+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:07 gibba011 bash[13015]: audit 2021-03-10T13:47:07.104247+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:07 gibba011 bash[13015]: audit 2021-03-10T13:47:07.113271+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:08 gibba032 bash[9829]: cluster 2021-03-10T13:47:07.113853+0000 mgr.y (mgr.14140) 142 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:08 gibba011 bash[9813]: cluster 2021-03-10T13:47:07.113853+0000 mgr.y (mgr.14140) 142 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:08 gibba011 bash[13015]: cluster 2021-03-10T13:47:07.113853+0000 mgr.y (mgr.14140) 142 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:08.964 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:08 gibba032 bash[15138]: debug 2021-03-10T13:47:08.634+0000 7f21ba0d5ec0 -1 Falling back to public interface 2021-03-10T13:47:09.483 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping: /dev/vg_nvme/lv_1 2021-03-10T13:47:09.483 INFO:teuthology.orchestra.run.gibba032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2021-03-10T13:47:09.483 INFO:teuthology.orchestra.run.gibba032.stderr: stderr: 10+0 records in 2021-03-10T13:47:09.484 INFO:teuthology.orchestra.run.gibba032.stderr:10+0 records out 2021-03-10T13:47:09.484 INFO:teuthology.orchestra.run.gibba032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0100934 s, 1.0 GB/s 2021-03-10T13:47:09.484 INFO:teuthology.orchestra.run.gibba032.stderr:--> Zapping successful for: 2021-03-10T13:47:09.564 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:09 gibba032 bash[15138]: debug 2021-03-10T13:47:09.274+0000 7f21ba0d5ec0 -1 osd.6 0 log_to_monitors {default=true} 2021-03-10T13:47:10.129 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch daemon add osd gibba032:vg_nvme/lv_1 2021-03-10T13:47:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:10 gibba032 bash[9829]: cluster 2021-03-10T13:47:09.114599+0000 mgr.y (mgr.14140) 143 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:10 gibba032 bash[9829]: audit 2021-03-10T13:47:09.282033+0000 mon.b (mon.2) 16 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:10.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:10 gibba032 bash[9829]: audit 2021-03-10T13:47:09.282329+0000 mon.a (mon.0) 412 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[9813]: cluster 2021-03-10T13:47:09.114599+0000 mgr.y (mgr.14140) 143 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[9813]: audit 2021-03-10T13:47:09.282033+0000 mon.b (mon.2) 16 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[9813]: audit 2021-03-10T13:47:09.282329+0000 mon.a (mon.0) 412 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[13015]: cluster 2021-03-10T13:47:09.114599+0000 mgr.y (mgr.14140) 143 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[13015]: audit 2021-03-10T13:47:09.282033+0000 mon.b (mon.2) 16 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:10 gibba011 bash[13015]: audit 2021-03-10T13:47:09.282329+0000 mon.a (mon.0) 412 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:47:11.431 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[15138]: debug 2021-03-10T13:47:11.154+0000 7f21a2e8a700 -1 osd.6 0 waiting for initial osdmap 2021-03-10T13:47:11.432 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[15138]: debug 2021-03-10T13:47:11.158+0000 7f21ae0a6700 -1 osd.6 40 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:47:11.432 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: audit 2021-03-10T13:47:10.141698+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:47:11.432 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: cluster 2021-03-10T13:47:10.141733+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e39: 7 total, 6 up, 6 in 2021-03-10T13:47:11.433 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: audit 2021-03-10T13:47:10.141815+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:11.433 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: audit 2021-03-10T13:47:10.142430+0000 mon.a (mon.0) 416 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.433 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: audit 2021-03-10T13:47:10.142687+0000 mon.b (mon.2) 17 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.433 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: cluster 2021-03-10T13:47:10.232610+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:11.434 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:11 gibba032 bash[9829]: cluster 2021-03-10T13:47:10.232707+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: audit 2021-03-10T13:47:10.141698+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:47:11.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: cluster 2021-03-10T13:47:10.141733+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e39: 7 total, 6 up, 6 in 2021-03-10T13:47:11.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: audit 2021-03-10T13:47:10.141815+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:11.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: audit 2021-03-10T13:47:10.142430+0000 mon.a (mon.0) 416 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: audit 2021-03-10T13:47:10.142687+0000 mon.b (mon.2) 17 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: cluster 2021-03-10T13:47:10.232610+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[9813]: cluster 2021-03-10T13:47:10.232707+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: audit 2021-03-10T13:47:10.141698+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:47:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: cluster 2021-03-10T13:47:10.141733+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e39: 7 total, 6 up, 6 in 2021-03-10T13:47:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: audit 2021-03-10T13:47:10.141815+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: audit 2021-03-10T13:47:10.142430+0000 mon.a (mon.0) 416 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: audit 2021-03-10T13:47:10.142687+0000 mon.b (mon.2) 17 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:11.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: cluster 2021-03-10T13:47:10.232610+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:11.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:11 gibba011 bash[13015]: cluster 2021-03-10T13:47:10.232707+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: cluster 2021-03-10T13:47:11.115712+0000 mgr.y (mgr.14140) 144 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: audit 2021-03-10T13:47:11.151059+0000 mon.a (mon.0) 417 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: cluster 2021-03-10T13:47:11.151101+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e40: 7 total, 6 up, 6 in 2021-03-10T13:47:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: audit 2021-03-10T13:47:11.151686+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: audit 2021-03-10T13:47:11.316387+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: audit 2021-03-10T13:47:11.441359+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:12 gibba032 bash[9829]: audit 2021-03-10T13:47:11.450281+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: cluster 2021-03-10T13:47:11.115712+0000 mgr.y (mgr.14140) 144 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: audit 2021-03-10T13:47:11.151059+0000 mon.a (mon.0) 417 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: cluster 2021-03-10T13:47:11.151101+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e40: 7 total, 6 up, 6 in 2021-03-10T13:47:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: audit 2021-03-10T13:47:11.151686+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: audit 2021-03-10T13:47:11.316387+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: cluster 2021-03-10T13:47:11.115712+0000 mgr.y (mgr.14140) 144 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 2.0 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: audit 2021-03-10T13:47:11.151059+0000 mon.a (mon.0) 417 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: cluster 2021-03-10T13:47:11.151101+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e40: 7 total, 6 up, 6 in 2021-03-10T13:47:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: audit 2021-03-10T13:47:11.151686+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:12.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: audit 2021-03-10T13:47:11.316387+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:12.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: audit 2021-03-10T13:47:11.441359+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:12.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[9813]: audit 2021-03-10T13:47:11.450281+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:12.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: audit 2021-03-10T13:47:11.441359+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:12.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:12 gibba011 bash[13015]: audit 2021-03-10T13:47:11.450281+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}}, \"last_daemon_update\": \"2021-03-10T13:47:03.000405\", \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:13 gibba032 bash[9829]: cluster 2021-03-10T13:47:12.167166+0000 mon.a (mon.0) 423 : cluster [INF] osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840] boot 2021-03-10T13:47:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:13 gibba032 bash[9829]: cluster 2021-03-10T13:47:12.167193+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e41: 7 total, 7 up, 7 in 2021-03-10T13:47:13.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:13 gibba032 bash[9829]: audit 2021-03-10T13:47:12.167281+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[9813]: cluster 2021-03-10T13:47:12.167166+0000 mon.a (mon.0) 423 : cluster [INF] osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840] boot 2021-03-10T13:47:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[9813]: cluster 2021-03-10T13:47:12.167193+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e41: 7 total, 7 up, 7 in 2021-03-10T13:47:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[9813]: audit 2021-03-10T13:47:12.167281+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:13.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[13015]: cluster 2021-03-10T13:47:12.167166+0000 mon.a (mon.0) 423 : cluster [INF] osd.6 [v2:172.21.2.132:6816/837531840,v1:172.21.2.132:6817/837531840] boot 2021-03-10T13:47:13.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[13015]: cluster 2021-03-10T13:47:12.167193+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e41: 7 total, 7 up, 7 in 2021-03-10T13:47:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:13 gibba011 bash[13015]: audit 2021-03-10T13:47:12.167281+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:47:14.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: cluster 2021-03-10T13:47:13.116539+0000 mgr.y (mgr.14140) 145 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 2.1 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:14.462 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: cluster 2021-03-10T13:47:13.178884+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2021-03-10T13:47:14.463 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: audit 2021-03-10T13:47:13.232308+0000 mgr.y (mgr.14140) 146 : audit [DBG] from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:14.463 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: cephadm 2021-03-10T13:47:13.234287+0000 mgr.y (mgr.14140) 147 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:47:14.463 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: audit 2021-03-10T13:47:13.235172+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:47:14.464 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: audit 2021-03-10T13:47:13.238231+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:14.465 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:14 gibba032 bash[9829]: audit 2021-03-10T13:47:13.242696+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: cluster 2021-03-10T13:47:13.116539+0000 mgr.y (mgr.14140) 145 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 2.1 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: cluster 2021-03-10T13:47:13.178884+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2021-03-10T13:47:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: audit 2021-03-10T13:47:13.232308+0000 mgr.y (mgr.14140) 146 : audit [DBG] from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: cephadm 2021-03-10T13:47:13.234287+0000 mgr.y (mgr.14140) 147 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:47:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: audit 2021-03-10T13:47:13.235172+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:47:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: audit 2021-03-10T13:47:13.238231+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[9813]: audit 2021-03-10T13:47:13.242696+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: cluster 2021-03-10T13:47:13.116539+0000 mgr.y (mgr.14140) 145 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 2.1 MiB used, 498 GiB / 504 GiB avail 2021-03-10T13:47:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: cluster 2021-03-10T13:47:13.178884+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2021-03-10T13:47:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: audit 2021-03-10T13:47:13.232308+0000 mgr.y (mgr.14140) 146 : audit [DBG] from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "gibba032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: cephadm 2021-03-10T13:47:13.234287+0000 mgr.y (mgr.14140) 147 : cephadm [INF] Applying osd on host gibba032... 2021-03-10T13:47:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: audit 2021-03-10T13:47:13.235172+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-03-10T13:47:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: audit 2021-03-10T13:47:13.238231+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:14.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:14 gibba011 bash[13015]: audit 2021-03-10T13:47:13.242696+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:16 gibba032 bash[9829]: cluster 2021-03-10T13:47:15.117348+0000 mgr.y (mgr.14140) 148 : cluster [DBG] pgmap v149: 1 pgs: 1 peering; 0 B data, 2.5 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:16 gibba011 bash[9813]: cluster 2021-03-10T13:47:15.117348+0000 mgr.y (mgr.14140) 148 : cluster [DBG] pgmap v149: 1 pgs: 1 peering; 0 B data, 2.5 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:16 gibba011 bash[13015]: cluster 2021-03-10T13:47:15.117348+0000 mgr.y (mgr.14140) 148 : cluster [DBG] pgmap v149: 1 pgs: 1 peering; 0 B data, 2.5 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: cluster 2021-03-10T13:47:16.117112+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:47:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: audit 2021-03-10T13:47:16.293124+0000 mon.b (mon.2) 18 : audit [INF] from='client.? 172.21.2.132:0/1891264973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: audit 2021-03-10T13:47:16.293894+0000 mon.a (mon.0) 431 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: audit 2021-03-10T13:47:16.306242+0000 mon.a (mon.0) 432 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]': finished 2021-03-10T13:47:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: cluster 2021-03-10T13:47:16.306322+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e43: 8 total, 7 up, 7 in 2021-03-10T13:47:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:17 gibba032 bash[9829]: audit 2021-03-10T13:47:16.306479+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: cluster 2021-03-10T13:47:16.117112+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:47:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: audit 2021-03-10T13:47:16.293124+0000 mon.b (mon.2) 18 : audit [INF] from='client.? 172.21.2.132:0/1891264973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: audit 2021-03-10T13:47:16.293894+0000 mon.a (mon.0) 431 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: audit 2021-03-10T13:47:16.306242+0000 mon.a (mon.0) 432 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]': finished 2021-03-10T13:47:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: cluster 2021-03-10T13:47:16.306322+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e43: 8 total, 7 up, 7 in 2021-03-10T13:47:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[9813]: audit 2021-03-10T13:47:16.306479+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: cluster 2021-03-10T13:47:16.117112+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2021-03-10T13:47:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: audit 2021-03-10T13:47:16.293124+0000 mon.b (mon.2) 18 : audit [INF] from='client.? 172.21.2.132:0/1891264973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: audit 2021-03-10T13:47:16.293894+0000 mon.a (mon.0) 431 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]: dispatch 2021-03-10T13:47:17.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: audit 2021-03-10T13:47:16.306242+0000 mon.a (mon.0) 432 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0503e7-615c-4f41-bc6c-7eb89d164d5e"}]': finished 2021-03-10T13:47:17.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: cluster 2021-03-10T13:47:16.306322+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e43: 8 total, 7 up, 7 in 2021-03-10T13:47:17.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:17 gibba011 bash[13015]: audit 2021-03-10T13:47:16.306479+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:18 gibba011 bash[9813]: cluster 2021-03-10T13:47:17.117949+0000 mgr.y (mgr.14140) 149 : cluster [DBG] pgmap v151: 1 pgs: 1 peering; 0 B data, 2.6 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:18 gibba011 bash[13015]: cluster 2021-03-10T13:47:17.117949+0000 mgr.y (mgr.14140) 149 : cluster [DBG] pgmap v151: 1 pgs: 1 peering; 0 B data, 2.6 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:18 gibba032 bash[9829]: cluster 2021-03-10T13:47:17.117949+0000 mgr.y (mgr.14140) 149 : cluster [DBG] pgmap v151: 1 pgs: 1 peering; 0 B data, 2.6 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:19 gibba011 bash[9813]: audit 2021-03-10T13:47:18.441376+0000 mon.b (mon.2) 19 : audit [DBG] from='client.? 172.21.2.132:0/1194907369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:47:19.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:19 gibba011 bash[13015]: audit 2021-03-10T13:47:18.441376+0000 mon.b (mon.2) 19 : audit [DBG] from='client.? 172.21.2.132:0/1194907369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:47:19.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:19 gibba032 bash[9829]: audit 2021-03-10T13:47:18.441376+0000 mon.b (mon.2) 19 : audit [DBG] from='client.? 172.21.2.132:0/1194907369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-03-10T13:47:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[9813]: cluster 2021-03-10T13:47:19.118900+0000 mgr.y (mgr.14140) 150 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[9813]: cluster 2021-03-10T13:47:19.307133+0000 mon.a (mon.0) 435 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:47:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[9813]: cluster 2021-03-10T13:47:19.307239+0000 mon.a (mon.0) 436 : cluster [INF] Cluster is now healthy 2021-03-10T13:47:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[13015]: cluster 2021-03-10T13:47:19.118900+0000 mgr.y (mgr.14140) 150 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[13015]: cluster 2021-03-10T13:47:19.307133+0000 mon.a (mon.0) 435 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:47:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:20 gibba011 bash[13015]: cluster 2021-03-10T13:47:19.307239+0000 mon.a (mon.0) 436 : cluster [INF] Cluster is now healthy 2021-03-10T13:47:20.661 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:20 gibba032 bash[9829]: cluster 2021-03-10T13:47:19.118900+0000 mgr.y (mgr.14140) 150 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:20.661 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:20 gibba032 bash[9829]: cluster 2021-03-10T13:47:19.307133+0000 mon.a (mon.0) 435 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:47:20.661 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:20 gibba032 bash[9829]: cluster 2021-03-10T13:47:19.307239+0000 mon.a (mon.0) 436 : cluster [INF] Cluster is now healthy 2021-03-10T13:47:21.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:21 gibba032 bash[9829]: audit 2021-03-10T13:47:21.121311+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:21 gibba011 bash[9813]: audit 2021-03-10T13:47:21.121311+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:21 gibba011 bash[13015]: audit 2021-03-10T13:47:21.121311+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:22.703 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:22 gibba032 bash[9829]: cluster 2021-03-10T13:47:21.119606+0000 mgr.y (mgr.14140) 151 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:22 gibba011 bash[9813]: cluster 2021-03-10T13:47:21.119606+0000 mgr.y (mgr.14140) 151 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:22 gibba011 bash[13015]: cluster 2021-03-10T13:47:21.119606+0000 mgr.y (mgr.14140) 151 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 2.7 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:24 gibba032 bash[9829]: cluster 2021-03-10T13:47:23.120730+0000 mgr.y (mgr.14140) 152 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:24 gibba011 bash[9813]: cluster 2021-03-10T13:47:23.120730+0000 mgr.y (mgr.14140) 152 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:24 gibba011 bash[13015]: cluster 2021-03-10T13:47:23.120730+0000 mgr.y (mgr.14140) 152 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:25.635 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:25 gibba032 bash[9829]: audit 2021-03-10T13:47:24.660662+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2021-03-10T13:47:25.635 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:25 gibba032 bash[9829]: audit 2021-03-10T13:47:24.663640+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:25.635 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:25 gibba032 bash[9829]: audit 2021-03-10T13:47:24.667051+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2021-03-10T13:47:25.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[9813]: audit 2021-03-10T13:47:24.660662+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2021-03-10T13:47:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[9813]: audit 2021-03-10T13:47:24.663640+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[9813]: audit 2021-03-10T13:47:24.667051+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2021-03-10T13:47:25.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[13015]: audit 2021-03-10T13:47:24.660662+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2021-03-10T13:47:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[13015]: audit 2021-03-10T13:47:24.663640+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:25 gibba011 bash[13015]: audit 2021-03-10T13:47:24.667051+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2021-03-10T13:47:26.544 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:26 gibba032 bash[9829]: cephadm 2021-03-10T13:47:24.666115+0000 mgr.y (mgr.14140) 153 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:47:26.544 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:26 gibba032 bash[9829]: cluster 2021-03-10T13:47:25.121450+0000 mgr.y (mgr.14140) 154 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:26.588 INFO:teuthology.orchestra.run.gibba032.stdout:Created osd(s) 7 on host 'gibba032' 2021-03-10T13:47:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:26 gibba011 bash[9813]: cephadm 2021-03-10T13:47:24.666115+0000 mgr.y (mgr.14140) 153 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:47:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:26 gibba011 bash[9813]: cluster 2021-03-10T13:47:25.121450+0000 mgr.y (mgr.14140) 154 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:26 gibba011 bash[13015]: cephadm 2021-03-10T13:47:24.666115+0000 mgr.y (mgr.14140) 153 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:47:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:26 gibba011 bash[13015]: cluster 2021-03-10T13:47:25.121450+0000 mgr.y (mgr.14140) 154 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:26.577004+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:26.578652+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:27.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:26.585724+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:27.116612+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:27.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:27.125277+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:27.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:27.210017+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:27.216153+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:27 gibba032 bash[9829]: audit 2021-03-10T13:47:27.217715+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:27.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:26.577004+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:26.578652+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:27.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:26.577004+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:26.578652+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:27.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:26.585724+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:27.116612+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:27.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:27.125277+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:27.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:26.585724+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999190\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999497\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999670\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999828\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:02.999977\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:03.000126\", \"created\": \"2021-03-10T13:47:02.458594\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_device_update\": \"2021-03-10T13:47:11.438807\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:27.116612+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:27.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:27.125277+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:27.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:27.210017+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:27.216153+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[9813]: audit 2021-03-10T13:47:27.217715+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:27.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:27.210017+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:27.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:27.216153+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:27.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:27 gibba011 bash[13015]: audit 2021-03-10T13:47:27.217715+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2021-03-10T13:47:28.066 DEBUG:teuthology.orchestra.run.gibba032:osd.7> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.7.service 2021-03-10T13:47:28.068 INFO:tasks.cephadm:Adding prometheus.a on gibba032 2021-03-10T13:47:28.069 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch apply prometheus '1;gibba032=a' 2021-03-10T13:47:28.073 INFO:journalctl@ceph.osd.7.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:28 gibba032 bash[9829]: cluster 2021-03-10T13:47:27.122063+0000 mgr.y (mgr.14140) 155 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:28 gibba011 bash[9813]: cluster 2021-03-10T13:47:27.122063+0000 mgr.y (mgr.14140) 155 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:28 gibba011 bash[13015]: cluster 2021-03-10T13:47:27.122063+0000 mgr.y (mgr.14140) 155 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:29.654 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-03-10T13:47:29.654 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-7 --no-mon-config 2021-03-10T13:47:29.655 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-7/block 2021-03-10T13:47:29.655 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-7/block 2021-03-10T13:47:29.655 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2021-03-10T13:47:29.655 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-03-10T13:47:29.656 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:29 gibba032 bash[16261]: --> ceph-volume lvm activate successful for osd ID: 7 2021-03-10T13:47:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:30 gibba032 bash[9829]: cluster 2021-03-10T13:47:29.123074+0000 mgr.y (mgr.14140) 156 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:30 gibba011 bash[9813]: cluster 2021-03-10T13:47:29.123074+0000 mgr.y (mgr.14140) 156 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:30 gibba011 bash[13015]: cluster 2021-03-10T13:47:29.123074+0000 mgr.y (mgr.14140) 156 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:32.723 INFO:teuthology.orchestra.run.gibba032.stdout:Scheduled prometheus update... 2021-03-10T13:47:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:32 gibba032 bash[9829]: cluster 2021-03-10T13:47:31.123936+0000 mgr.y (mgr.14140) 157 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:32 gibba011 bash[9813]: cluster 2021-03-10T13:47:31.123936+0000 mgr.y (mgr.14140) 157 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:32 gibba011 bash[13015]: cluster 2021-03-10T13:47:31.123936+0000 mgr.y (mgr.14140) 157 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:33.257 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:32 gibba032 bash[16653]: debug 2021-03-10T13:47:32.986+0000 7f0542da9ec0 -1 Falling back to public interface 2021-03-10T13:47:33.342 DEBUG:teuthology.orchestra.run.gibba032:prometheus.a> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@prometheus.a.service 2021-03-10T13:47:33.344 INFO:tasks.cephadm:Adding node-exporter.a on gibba011 2021-03-10T13:47:33.344 INFO:tasks.cephadm:Adding node-exporter.b on gibba032 2021-03-10T13:47:33.344 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch apply node-exporter '2;gibba011=a;gibba032=b' 2021-03-10T13:47:33.357 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:33.612 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:33 gibba032 bash[9829]: audit 2021-03-10T13:47:32.713500+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]: dispatch 2021-03-10T13:47:33.612 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:33 gibba032 bash[9829]: audit 2021-03-10T13:47:32.721896+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]': finished 2021-03-10T13:47:33.756 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:33 gibba032 bash[16653]: debug 2021-03-10T13:47:33.606+0000 7f0542da9ec0 -1 osd.7 0 log_to_monitors {default=true} 2021-03-10T13:47:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:33 gibba011 bash[9813]: audit 2021-03-10T13:47:32.713500+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]: dispatch 2021-03-10T13:47:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:33 gibba011 bash[9813]: audit 2021-03-10T13:47:32.721896+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]': finished 2021-03-10T13:47:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:33 gibba011 bash[13015]: audit 2021-03-10T13:47:32.713500+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]: dispatch 2021-03-10T13:47:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:33 gibba011 bash[13015]: audit 2021-03-10T13:47:32.721896+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.prometheus","val":"{\"created\": \"2021-03-10T13:47:32.712523\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"prometheus\"}}"}]': finished 2021-03-10T13:47:34.693 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:34 gibba032 bash[9829]: audit 2021-03-10T13:47:32.711078+0000 mgr.y (mgr.14140) 158 : audit [DBG] from='client.34229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:34.694 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:34 gibba032 bash[9829]: cephadm 2021-03-10T13:47:32.712489+0000 mgr.y (mgr.14140) 159 : cephadm [INF] Saving service prometheus spec with placement count:1 gibba032=a 2021-03-10T13:47:34.694 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:34 gibba032 bash[9829]: cluster 2021-03-10T13:47:33.124656+0000 mgr.y (mgr.14140) 160 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:34.695 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:34 gibba032 bash[9829]: audit 2021-03-10T13:47:33.613332+0000 mon.b (mon.2) 20 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:34.695 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:34 gibba032 bash[9829]: audit 2021-03-10T13:47:33.613941+0000 mon.a (mon.0) 451 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[9813]: audit 2021-03-10T13:47:32.711078+0000 mgr.y (mgr.14140) 158 : audit [DBG] from='client.34229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[9813]: cephadm 2021-03-10T13:47:32.712489+0000 mgr.y (mgr.14140) 159 : cephadm [INF] Saving service prometheus spec with placement count:1 gibba032=a 2021-03-10T13:47:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[9813]: cluster 2021-03-10T13:47:33.124656+0000 mgr.y (mgr.14140) 160 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[9813]: audit 2021-03-10T13:47:33.613332+0000 mon.b (mon.2) 20 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[9813]: audit 2021-03-10T13:47:33.613941+0000 mon.a (mon.0) 451 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[13015]: audit 2021-03-10T13:47:32.711078+0000 mgr.y (mgr.14140) 158 : audit [DBG] from='client.34229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[13015]: cephadm 2021-03-10T13:47:32.712489+0000 mgr.y (mgr.14140) 159 : cephadm [INF] Saving service prometheus spec with placement count:1 gibba032=a 2021-03-10T13:47:34.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[13015]: cluster 2021-03-10T13:47:33.124656+0000 mgr.y (mgr.14140) 160 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:34.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[13015]: audit 2021-03-10T13:47:33.613332+0000 mon.b (mon.2) 20 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:34.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:34 gibba011 bash[13015]: audit 2021-03-10T13:47:33.613941+0000 mon.a (mon.0) 451 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:47:35.757 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[16653]: debug 2021-03-10T13:47:35.422+0000 7f052bb5e700 -1 osd.7 0 waiting for initial osdmap 2021-03-10T13:47:35.757 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[16653]: debug 2021-03-10T13:47:35.426+0000 7f0536d7a700 -1 osd.7 45 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:47:35.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.407470+0000 mon.a (mon.0) 452 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:47:35.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: cluster 2021-03-10T13:47:34.407515+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e44: 8 total, 7 up, 7 in 2021-03-10T13:47:35.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.408238+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:35.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.410020+0000 mon.b (mon.2) 21 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.412667+0000 mon.a (mon.0) 455 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.640640+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:35.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.754145+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:35 gibba032 bash[9829]: audit 2021-03-10T13:47:34.763227+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.407470+0000 mon.a (mon.0) 452 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:47:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: cluster 2021-03-10T13:47:34.407515+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e44: 8 total, 7 up, 7 in 2021-03-10T13:47:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.408238+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.410020+0000 mon.b (mon.2) 21 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.412667+0000 mon.a (mon.0) 455 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.640640+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:35.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.407470+0000 mon.a (mon.0) 452 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:47:35.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: cluster 2021-03-10T13:47:34.407515+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e44: 8 total, 7 up, 7 in 2021-03-10T13:47:35.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.408238+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:35.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.410020+0000 mon.b (mon.2) 21 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.412667+0000 mon.a (mon.0) 455 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:47:35.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.640640+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:35.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.754145+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:35.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[9813]: audit 2021-03-10T13:47:34.763227+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:35.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.754145+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:35.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:35 gibba011 bash[13015]: audit 2021-03-10T13:47:34.763227+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}}, \"last_daemon_update\": \"2021-03-10T13:47:27.208682\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: cluster 2021-03-10T13:47:34.652515+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: cluster 2021-03-10T13:47:34.652553+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: cephadm 2021-03-10T13:47:34.766122+0000 mgr.y (mgr.14140) 161 : cephadm [INF] Deploying daemon prometheus.a on gibba032 2021-03-10T13:47:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: cluster 2021-03-10T13:47:35.125311+0000 mgr.y (mgr.14140) 162 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: audit 2021-03-10T13:47:35.421688+0000 mon.a (mon.0) 459 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: cluster 2021-03-10T13:47:35.421760+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e45: 8 total, 7 up, 7 in 2021-03-10T13:47:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:36 gibba032 bash[9829]: audit 2021-03-10T13:47:35.422433+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: cluster 2021-03-10T13:47:34.652515+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: cluster 2021-03-10T13:47:34.652553+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: cephadm 2021-03-10T13:47:34.766122+0000 mgr.y (mgr.14140) 161 : cephadm [INF] Deploying daemon prometheus.a on gibba032 2021-03-10T13:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: cluster 2021-03-10T13:47:35.125311+0000 mgr.y (mgr.14140) 162 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: audit 2021-03-10T13:47:35.421688+0000 mon.a (mon.0) 459 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:36.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: cluster 2021-03-10T13:47:35.421760+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e45: 8 total, 7 up, 7 in 2021-03-10T13:47:36.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[9813]: audit 2021-03-10T13:47:35.422433+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: cluster 2021-03-10T13:47:34.652515+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2021-03-10T13:47:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: cluster 2021-03-10T13:47:34.652553+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2021-03-10T13:47:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: cephadm 2021-03-10T13:47:34.766122+0000 mgr.y (mgr.14140) 161 : cephadm [INF] Deploying daemon prometheus.a on gibba032 2021-03-10T13:47:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: cluster 2021-03-10T13:47:35.125311+0000 mgr.y (mgr.14140) 162 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 2.8 MiB used, 581 GiB / 588 GiB avail 2021-03-10T13:47:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: audit 2021-03-10T13:47:35.421688+0000 mon.a (mon.0) 459 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]': finished 2021-03-10T13:47:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: cluster 2021-03-10T13:47:35.421760+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e45: 8 total, 7 up, 7 in 2021-03-10T13:47:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:36 gibba011 bash[13015]: audit 2021-03-10T13:47:35.422433+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:37.170 INFO:teuthology.orchestra.run.gibba032.stdout:Scheduled node-exporter update... 2021-03-10T13:47:37.756 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[15138]: debug 2021-03-10T13:47:37.442+0000 7f219807a700 -1 osd.6 47 build_incremental_map_msg missing incremental map 47 2021-03-10T13:47:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[9829]: cluster 2021-03-10T13:47:36.426909+0000 mon.a (mon.0) 462 : cluster [INF] osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098] boot 2021-03-10T13:47:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[9829]: cluster 2021-03-10T13:47:36.427009+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e46: 8 total, 8 up, 8 in 2021-03-10T13:47:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[9829]: audit 2021-03-10T13:47:36.427205+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:37.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[9829]: audit 2021-03-10T13:47:37.160474+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]: dispatch 2021-03-10T13:47:37.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:37 gibba032 bash[9829]: audit 2021-03-10T13:47:37.168901+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]': finished 2021-03-10T13:47:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[9813]: cluster 2021-03-10T13:47:36.426909+0000 mon.a (mon.0) 462 : cluster [INF] osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098] boot 2021-03-10T13:47:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[9813]: cluster 2021-03-10T13:47:36.427009+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e46: 8 total, 8 up, 8 in 2021-03-10T13:47:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[9813]: audit 2021-03-10T13:47:36.427205+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[9813]: audit 2021-03-10T13:47:37.160474+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]: dispatch 2021-03-10T13:47:37.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[9813]: audit 2021-03-10T13:47:37.168901+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]': finished 2021-03-10T13:47:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[13015]: cluster 2021-03-10T13:47:36.426909+0000 mon.a (mon.0) 462 : cluster [INF] osd.7 [v2:172.21.2.132:6824/189669098,v1:172.21.2.132:6825/189669098] boot 2021-03-10T13:47:37.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[13015]: cluster 2021-03-10T13:47:36.427009+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e46: 8 total, 8 up, 8 in 2021-03-10T13:47:37.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[13015]: audit 2021-03-10T13:47:36.427205+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:47:37.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[13015]: audit 2021-03-10T13:47:37.160474+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]: dispatch 2021-03-10T13:47:37.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:37 gibba011 bash[13015]: audit 2021-03-10T13:47:37.168901+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.node-exporter","val":"{\"created\": \"2021-03-10T13:47:37.159464\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}, {\"hostname\": \"gibba032\", \"name\": \"b\", \"network\": \"\"}]}, \"service_type\": \"node-exporter\"}}"}]': finished 2021-03-10T13:47:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:38 gibba032 bash[9829]: cluster 2021-03-10T13:47:37.126206+0000 mgr.y (mgr.14140) 163 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 3.2 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:38 gibba032 bash[9829]: audit 2021-03-10T13:47:37.157967+0000 mgr.y (mgr.14140) 164 : audit [DBG] from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;gibba011=a;gibba032=b", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:38 gibba032 bash[9829]: cephadm 2021-03-10T13:47:37.159429+0000 mgr.y (mgr.14140) 165 : cephadm [INF] Saving service node-exporter spec with placement count:2 gibba011=a,gibba032=b 2021-03-10T13:47:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:38 gibba032 bash[9829]: cluster 2021-03-10T13:47:37.443289+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e47: 8 total, 8 up, 8 in 2021-03-10T13:47:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[9813]: cluster 2021-03-10T13:47:37.126206+0000 mgr.y (mgr.14140) 163 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 3.2 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[9813]: audit 2021-03-10T13:47:37.157967+0000 mgr.y (mgr.14140) 164 : audit [DBG] from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;gibba011=a;gibba032=b", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[9813]: cephadm 2021-03-10T13:47:37.159429+0000 mgr.y (mgr.14140) 165 : cephadm [INF] Saving service node-exporter spec with placement count:2 gibba011=a,gibba032=b 2021-03-10T13:47:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[9813]: cluster 2021-03-10T13:47:37.443289+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e47: 8 total, 8 up, 8 in 2021-03-10T13:47:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[13015]: cluster 2021-03-10T13:47:37.126206+0000 mgr.y (mgr.14140) 163 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 3.2 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:38.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[13015]: audit 2021-03-10T13:47:37.157967+0000 mgr.y (mgr.14140) 164 : audit [DBG] from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;gibba011=a;gibba032=b", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:38.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[13015]: cephadm 2021-03-10T13:47:37.159429+0000 mgr.y (mgr.14140) 165 : cephadm [INF] Saving service node-exporter spec with placement count:2 gibba011=a,gibba032=b 2021-03-10T13:47:38.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:38 gibba011 bash[13015]: cluster 2021-03-10T13:47:37.443289+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e47: 8 total, 8 up, 8 in 2021-03-10T13:47:39.261 DEBUG:teuthology.orchestra.run.gibba011:node-exporter.a> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@node-exporter.a.service 2021-03-10T13:47:39.265 DEBUG:teuthology.orchestra.run.gibba032:node-exporter.b> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@node-exporter.b.service 2021-03-10T13:47:39.267 INFO:tasks.cephadm:Adding alertmanager.a on gibba011 2021-03-10T13:47:39.267 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch apply alertmanager '1;gibba011=a' 2021-03-10T13:47:39.284 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:39.316 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:40 gibba032 bash[9829]: cluster 2021-03-10T13:47:39.126913+0000 mgr.y (mgr.14140) 166 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 3.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:40 gibba011 bash[9813]: cluster 2021-03-10T13:47:39.126913+0000 mgr.y (mgr.14140) 166 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 3.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:40 gibba011 bash[13015]: cluster 2021-03-10T13:47:39.126913+0000 mgr.y (mgr.14140) 166 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 3.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:42.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:42 gibba032 bash[9829]: cluster 2021-03-10T13:47:41.127787+0000 mgr.y (mgr.14140) 167 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 3.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:42 gibba011 bash[9813]: cluster 2021-03-10T13:47:41.127787+0000 mgr.y (mgr.14140) 167 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 3.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:42 gibba011 bash[13015]: cluster 2021-03-10T13:47:41.127787+0000 mgr.y (mgr.14140) 167 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 3.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:43.328 INFO:teuthology.orchestra.run.gibba032.stdout:Scheduled alertmanager update... 2021-03-10T13:47:43.650 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:43 gibba032 bash[9829]: audit 2021-03-10T13:47:43.317524+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]: dispatch 2021-03-10T13:47:43.651 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:43 gibba032 bash[9829]: audit 2021-03-10T13:47:43.326004+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]': finished 2021-03-10T13:47:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:43 gibba011 bash[9813]: audit 2021-03-10T13:47:43.317524+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]: dispatch 2021-03-10T13:47:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:43 gibba011 bash[9813]: audit 2021-03-10T13:47:43.326004+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]': finished 2021-03-10T13:47:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:43 gibba011 bash[13015]: audit 2021-03-10T13:47:43.317524+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]: dispatch 2021-03-10T13:47:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:43 gibba011 bash[13015]: audit 2021-03-10T13:47:43.326004+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.alertmanager","val":"{\"created\": \"2021-03-10T13:47:43.316544\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba011\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"alertmanager\"}}"}]': finished 2021-03-10T13:47:44.199 DEBUG:teuthology.orchestra.run.gibba011:alertmanager.a> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@alertmanager.a.service 2021-03-10T13:47:44.203 INFO:tasks.cephadm:Adding grafana.a on gibba032 2021-03-10T13:47:44.203 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph orch apply grafana '1;gibba032=a' 2021-03-10T13:47:44.222 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:44.613 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:44 gibba032 bash[9829]: cluster 2021-03-10T13:47:43.128557+0000 mgr.y (mgr.14140) 168 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:44.613 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:44 gibba032 bash[9829]: audit 2021-03-10T13:47:43.315191+0000 mgr.y (mgr.14140) 169 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;gibba011=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:44.613 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:44 gibba032 bash[9829]: cephadm 2021-03-10T13:47:43.316513+0000 mgr.y (mgr.14140) 170 : cephadm [INF] Saving service alertmanager spec with placement count:1 gibba011=a 2021-03-10T13:47:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[9813]: cluster 2021-03-10T13:47:43.128557+0000 mgr.y (mgr.14140) 168 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[9813]: audit 2021-03-10T13:47:43.315191+0000 mgr.y (mgr.14140) 169 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;gibba011=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[9813]: cephadm 2021-03-10T13:47:43.316513+0000 mgr.y (mgr.14140) 170 : cephadm [INF] Saving service alertmanager spec with placement count:1 gibba011=a 2021-03-10T13:47:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[13015]: cluster 2021-03-10T13:47:43.128557+0000 mgr.y (mgr.14140) 168 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[13015]: audit 2021-03-10T13:47:43.315191+0000 mgr.y (mgr.14140) 169 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;gibba011=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:44 gibba011 bash[13015]: cephadm 2021-03-10T13:47:43.316513+0000 mgr.y (mgr.14140) 170 : cephadm [INF] Saving service alertmanager spec with placement count:1 gibba011=a 2021-03-10T13:47:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:46 gibba032 bash[9829]: cluster 2021-03-10T13:47:45.129258+0000 mgr.y (mgr.14140) 171 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:46 gibba011 bash[9813]: cluster 2021-03-10T13:47:45.129258+0000 mgr.y (mgr.14140) 171 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:46 gibba011 bash[13015]: cluster 2021-03-10T13:47:45.129258+0000 mgr.y (mgr.14140) 171 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:47.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:47 gibba032 bash[9829]: audit 2021-03-10T13:47:47.128649+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:47.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:47 gibba032 bash[9829]: audit 2021-03-10T13:47:47.137439+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:47.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:47 gibba011 bash[9813]: audit 2021-03-10T13:47:47.128649+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:47.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:47 gibba011 bash[9813]: audit 2021-03-10T13:47:47.137439+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:47.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:47 gibba011 bash[13015]: audit 2021-03-10T13:47:47.128649+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2021-03-10T13:47:47.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:47 gibba011 bash[13015]: audit 2021-03-10T13:47:47.137439+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"e6796fd6-ecf5-4fb0-af68-113a8ec371e8\", \"message\": \"Rebalancing after osd.4 marked in\", \"refs\": [[\"osd\", 4]], \"started_at\": 1615383986.3485992, \"finished_at\": 1615383999.6166334}, {\"id\": \"fbd81c2a-977c-4295-8d27-420817e36c14\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1615384009.145623, \"finished_at\": 1615384022.4862714}, {\"id\": \"1afb9f95-eae4-4f58-bce7-cb2ad3af99c7\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1615384032.167104, \"finished_at\": 1615384046.589522}, {\"id\": \"9c473dfe-68af-450c-ae96-da266203b842\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1615384056.4268894, \"finished_at\": 1615384064.3235571}], \"version\": 2, \"compat_version\": 2}"}]': finished 2021-03-10T13:47:48.593 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:48 gibba032 bash[9829]: cluster 2021-03-10T13:47:47.130061+0000 mgr.y (mgr.14140) 172 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:48.639 INFO:teuthology.orchestra.run.gibba032.stdout:Scheduled grafana update... 2021-03-10T13:47:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:48 gibba011 bash[9813]: cluster 2021-03-10T13:47:47.130061+0000 mgr.y (mgr.14140) 172 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:48 gibba011 bash[13015]: cluster 2021-03-10T13:47:47.130061+0000 mgr.y (mgr.14140) 172 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:48.981 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:48 gibba032 docker[17894]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-prometheus.a 2021-03-10T13:47:48.981 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:48 gibba032 systemd[1]: Started Ceph prometheus.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:47:49.669 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 docker[19684]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-node-exporter.a 2021-03-10T13:47:49.670 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 systemd[1]: Started Ceph node-exporter.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:47:49.671 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:48.626711+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.671 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:48.626711+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.672 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:48.635698+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.672 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:48.635828+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]: dispatch 2021-03-10T13:47:49.672 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:48.636536+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:49.673 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:48.638163+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]': finished 2021-03-10T13:47:49.673 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:48.635698+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.674 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:48.635828+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]: dispatch 2021-03-10T13:47:49.674 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:48.636536+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:49.674 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:48.638163+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]': finished 2021-03-10T13:47:49.674 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:49.298508+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.675 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:49.301920+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.675 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:49.302979+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:49.675 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[9813]: audit 2021-03-10T13:47:49.303882+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:49.676 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:49.298508+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.676 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:49.301920+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.676 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:49.302979+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:49.676 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[13015]: audit 2021-03-10T13:47:49.303882+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:49.754 DEBUG:teuthology.orchestra.run.gibba032:grafana.a> sudo journalctl -f -n 0 -u ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@grafana.a.service 2021-03-10T13:47:49.757 INFO:tasks.cephadm:Setting up client nodes... 2021-03-10T13:47:49.758 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2021-03-10T13:47:49.774 INFO:journalctl@ceph.grafana.a.gibba032.stdout:-- Logs begin at Wed 2021-03-03 23:49:35 UTC. -- 2021-03-10T13:47:49.783 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:48.626711+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.784 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:48.635698+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208044\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208183\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208264\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208339\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208411\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208481\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:27.208551\", \"created\": \"2021-03-10T13:47:26.562593\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.784 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:48.635828+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]: dispatch 2021-03-10T13:47:49.784 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:48.636536+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:47:49.785 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:48.638163+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.grafana","val":"{\"created\": \"2021-03-10T13:47:48.634232\", \"spec\": {\"placement\": {\"count\": 1, \"hosts\": [{\"hostname\": \"gibba032\", \"name\": \"a\", \"network\": \"\"}]}, \"service_type\": \"grafana\"}}"}]': finished 2021-03-10T13:47:49.785 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:49.298508+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:49.785 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:49.301920+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}}, \"last_daemon_update\": \"2021-03-10T13:47:49.296886\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:49.786 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:49.302979+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:49.786 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:49 gibba032 bash[9829]: audit 2021-03-10T13:47:49.303882+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:49.962 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[19706]: Unable to find image 'prom/node-exporter:latest' locally 2021-03-10T13:47:50.065 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:366 msg="No time or size retention was set so using the default time retention" duration=15d 2021-03-10T13:47:50.065 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:404 msg="Starting Prometheus" version="(version=2.25.0, branch=HEAD, revision=a6be548dbc17780d562a39c0e4bd0bd4c00ad6e2)" 2021-03-10T13:47:50.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:409 build_context="(go=go1.15.8, user=root@615f028225c9, date=20210217-14:17:24)" 2021-03-10T13:47:50.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:410 host_details="(Linux 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021 x86_64 gibba032 (none))" 2021-03-10T13:47:50.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:411 fd_limits="(soft=1048576, hard=1048576)" 2021-03-10T13:47:50.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.021Z caller=main.go:412 vm_limits="(soft=unlimited, hard=unlimited)" 2021-03-10T13:47:50.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.023Z caller=web.go:532 component=web msg="Start listening for connections" address=:9095 2021-03-10T13:47:50.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.023Z caller=main.go:779 msg="Starting TSDB ..." 2021-03-10T13:47:50.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.024Z caller=tls_config.go:191 component=web msg="TLS is disabled." http2=false 2021-03-10T13:47:50.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.026Z caller=head.go:668 component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2021-03-10T13:47:50.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.026Z caller=head.go:682 component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.186µs 2021-03-10T13:47:50.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.026Z caller=head.go:688 component=tsdb msg="Replaying WAL, this may take a while" 2021-03-10T13:47:50.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.026Z caller=head.go:740 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2021-03-10T13:47:50.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.026Z caller=head.go:745 component=tsdb msg="WAL replay completed" checkpoint_replay_duration=12.461µs wal_replay_duration=144.276µs total_replay_duration=166.735µs 2021-03-10T13:47:50.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.027Z caller=main.go:799 fs_type=XFS_SUPER_MAGIC 2021-03-10T13:47:50.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.027Z caller=main.go:802 msg="TSDB started" 2021-03-10T13:47:50.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.027Z caller=main.go:928 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2021-03-10T13:47:50.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.027Z caller=main.go:959 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=360.879µs remote_storage=808ns web_handler=131ns query_engine=355ns scrape=196.591µs scrape_sd=13.401µs notify=365ns notify_sd=1.257µs rules=16.616µs 2021-03-10T13:47:50.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[17912]: level=info ts=2021-03-10T13:47:50.027Z caller=main.go:751 msg="Server is ready to receive web requests." 2021-03-10T13:47:50.330 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[19706]: latest: Pulling from prom/node-exporter 2021-03-10T13:47:50.330 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[19706]: dc6dd4561653: Pulling fs layer 2021-03-10T13:47:50.331 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[19706]: 613f88646930: Pulling fs layer 2021-03-10T13:47:50.331 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:49 gibba011 bash[19706]: edad907fb257: Pulling fs layer 2021-03-10T13:47:50.331 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: 613f88646930: Verifying Checksum 2021-03-10T13:47:50.331 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: 613f88646930: Download complete 2021-03-10T13:47:50.331 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: dc6dd4561653: Verifying Checksum 2021-03-10T13:47:50.332 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: dc6dd4561653: Download complete 2021-03-10T13:47:50.332 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: edad907fb257: Verifying Checksum 2021-03-10T13:47:50.332 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: edad907fb257: Download complete 2021-03-10T13:47:50.455 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 systemd[1]: Started Ceph node-exporter.b for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:47:50.456 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: Unable to find image 'prom/node-exporter:latest' locally 2021-03-10T13:47:50.456 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: latest: Pulling from prom/node-exporter 2021-03-10T13:47:50.741 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: dc6dd4561653: Pulling fs layer 2021-03-10T13:47:50.742 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: 613f88646930: Pulling fs layer 2021-03-10T13:47:50.742 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: edad907fb257: Pulling fs layer 2021-03-10T13:47:50.742 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: 613f88646930: Download complete 2021-03-10T13:47:50.743 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: dc6dd4561653: Verifying Checksum 2021-03-10T13:47:50.743 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: dc6dd4561653: Download complete 2021-03-10T13:47:50.743 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: edad907fb257: Verifying Checksum 2021-03-10T13:47:50.743 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: edad907fb257: Download complete 2021-03-10T13:47:50.744 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[18247]: dc6dd4561653: Pull complete 2021-03-10T13:47:50.745 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:48.632995+0000 mgr.y (mgr.14140) 173 : audit [DBG] from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:50.745 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: cephadm 2021-03-10T13:47:48.634201+0000 mgr.y (mgr.14140) 174 : cephadm [INF] Saving service grafana spec with placement count:1 gibba032=a 2021-03-10T13:47:50.745 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: cluster 2021-03-10T13:47:49.131073+0000 mgr.y (mgr.14140) 175 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:50.745 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: cephadm 2021-03-10T13:47:49.304520+0000 mgr.y (mgr.14140) 176 : cephadm [INF] Deploying daemon node-exporter.a on gibba011 2021-03-10T13:47:50.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:49.656831+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:47:50.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:49.663507+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:47:50.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:49.663948+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:50.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:49.664815+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:50.747 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:50.168226+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:50.747 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:50 gibba032 bash[9829]: audit 2021-03-10T13:47:50.171016+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:50.830 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[19706]: dc6dd4561653: Pull complete 2021-03-10T13:47:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:48.632995+0000 mgr.y (mgr.14140) 173 : audit [DBG] from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: cephadm 2021-03-10T13:47:48.634201+0000 mgr.y (mgr.14140) 174 : cephadm [INF] Saving service grafana spec with placement count:1 gibba032=a 2021-03-10T13:47:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: cluster 2021-03-10T13:47:49.131073+0000 mgr.y (mgr.14140) 175 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: cephadm 2021-03-10T13:47:49.304520+0000 mgr.y (mgr.14140) 176 : cephadm [INF] Deploying daemon node-exporter.a on gibba011 2021-03-10T13:47:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:48.632995+0000 mgr.y (mgr.14140) 173 : audit [DBG] from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;gibba032=a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:47:50.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: cephadm 2021-03-10T13:47:48.634201+0000 mgr.y (mgr.14140) 174 : cephadm [INF] Saving service grafana spec with placement count:1 gibba032=a 2021-03-10T13:47:50.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: cluster 2021-03-10T13:47:49.131073+0000 mgr.y (mgr.14140) 175 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:50.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: cephadm 2021-03-10T13:47:49.304520+0000 mgr.y (mgr.14140) 176 : cephadm [INF] Deploying daemon node-exporter.a on gibba011 2021-03-10T13:47:50.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:49.656831+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:47:50.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:49.656831+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:47:50.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:49.663507+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:47:50.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:49.663948+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:50.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:49.664815+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:50.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:49.663507+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:47:50.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:49.663948+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon"}]: dispatch 2021-03-10T13:47:50.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:49.664815+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:47:50.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:50.168226+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:50.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[13015]: audit 2021-03-10T13:47:50.171016+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:50.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:50.168226+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:47:50.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:50 gibba011 bash[9813]: audit 2021-03-10T13:47:50.171016+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:47:51.334 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[18247]: 613f88646930: Pull complete 2021-03-10T13:47:51.506 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[19706]: 613f88646930: Pull complete 2021-03-10T13:47:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[9829]: cephadm 2021-03-10T13:47:49.665228+0000 mgr.y (mgr.14140) 177 : cephadm [INF] Deploying daemon node-exporter.b on gibba032 2021-03-10T13:47:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[9829]: cephadm 2021-03-10T13:47:50.173284+0000 mgr.y (mgr.14140) 178 : cephadm [INF] Deploying daemon alertmanager.a on gibba011 2021-03-10T13:47:51.757 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[18247]: edad907fb257: Pull complete 2021-03-10T13:47:51.758 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[18247]: Digest: sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810 2021-03-10T13:47:51.758 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:51 gibba032 bash[18247]: Status: Downloaded newer image for prom/node-exporter:latest 2021-03-10T13:47:51.829 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[19706]: edad907fb257: Pull complete 2021-03-10T13:47:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[9813]: cephadm 2021-03-10T13:47:49.665228+0000 mgr.y (mgr.14140) 177 : cephadm [INF] Deploying daemon node-exporter.b on gibba032 2021-03-10T13:47:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[9813]: cephadm 2021-03-10T13:47:50.173284+0000 mgr.y (mgr.14140) 178 : cephadm [INF] Deploying daemon alertmanager.a on gibba011 2021-03-10T13:47:51.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[13015]: cephadm 2021-03-10T13:47:49.665228+0000 mgr.y (mgr.14140) 177 : cephadm [INF] Deploying daemon node-exporter.b on gibba032 2021-03-10T13:47:51.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[13015]: cephadm 2021-03-10T13:47:50.173284+0000 mgr.y (mgr.14140) 178 : cephadm [INF] Deploying daemon alertmanager.a on gibba011 2021-03-10T13:47:52.330 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:51 gibba011 bash[19706]: Digest: sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810 2021-03-10T13:47:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:52 gibba011 bash[9813]: cluster 2021-03-10T13:47:51.131809+0000 mgr.y (mgr.14140) 179 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:52 gibba011 bash[13015]: cluster 2021-03-10T13:47:51.131809+0000 mgr.y (mgr.14140) 179 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:52.831 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:52 gibba011 bash[19706]: Status: Downloaded newer image for prom/node-exporter:latest 2021-03-10T13:47:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[9829]: cluster 2021-03-10T13:47:51.131809+0000 mgr.y (mgr.14140) 179 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:53.007 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.612Z caller=node_exporter.go:178 msg="Starting node_exporter" version="(version=1.1.2, branch=HEAD, revision=b597c1244d7bef49e6f3359c87a56dd7707f6719)" 2021-03-10T13:47:53.008 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.612Z caller=node_exporter.go:179 msg="Build context" build_context="(go=go1.15.8, user=root@f07de8ca602a, date=20210305-09:29:10)" 2021-03-10T13:47:53.008 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=filesystem_common.go:74 collector=filesystem msg="Parsed flag --collector.filesystem.ignored-mount-points" flag=^/(dev|proc|sys|var/lib/docker/.+)($|/) 2021-03-10T13:47:53.008 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=filesystem_common.go:76 collector=filesystem msg="Parsed flag --collector.filesystem.ignored-fs-types" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2021-03-10T13:47:53.008 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:106 msg="Enabled collectors" 2021-03-10T13:47:53.009 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=arp 2021-03-10T13:47:53.009 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=bcache 2021-03-10T13:47:53.009 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=bonding 2021-03-10T13:47:53.009 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=btrfs 2021-03-10T13:47:53.010 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=conntrack 2021-03-10T13:47:53.010 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=cpu 2021-03-10T13:47:53.010 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=cpufreq 2021-03-10T13:47:53.010 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=diskstats 2021-03-10T13:47:53.010 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=edac 2021-03-10T13:47:53.011 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=entropy 2021-03-10T13:47:53.011 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=fibrechannel 2021-03-10T13:47:53.011 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=filefd 2021-03-10T13:47:53.011 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=filesystem 2021-03-10T13:47:53.011 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=hwmon 2021-03-10T13:47:53.012 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=infiniband 2021-03-10T13:47:53.012 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=ipvs 2021-03-10T13:47:53.012 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=loadavg 2021-03-10T13:47:53.012 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=mdadm 2021-03-10T13:47:53.013 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=meminfo 2021-03-10T13:47:53.013 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=netclass 2021-03-10T13:47:53.013 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=netdev 2021-03-10T13:47:53.013 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=netstat 2021-03-10T13:47:53.013 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=nfs 2021-03-10T13:47:53.014 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=nfsd 2021-03-10T13:47:53.014 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=powersupplyclass 2021-03-10T13:47:53.014 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=pressure 2021-03-10T13:47:53.015 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=rapl 2021-03-10T13:47:53.015 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=schedstat 2021-03-10T13:47:53.015 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=sockstat 2021-03-10T13:47:53.015 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=softnet 2021-03-10T13:47:53.015 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=stat 2021-03-10T13:47:53.016 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=textfile 2021-03-10T13:47:53.016 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=thermal_zone 2021-03-10T13:47:53.016 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=time 2021-03-10T13:47:53.016 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=udp_queues 2021-03-10T13:47:53.016 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=uname 2021-03-10T13:47:53.017 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=vmstat 2021-03-10T13:47:53.017 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=xfs 2021-03-10T13:47:53.017 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:113 collector=zfs 2021-03-10T13:47:53.017 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=node_exporter.go:195 msg="Listening on" address=:9100 2021-03-10T13:47:53.017 INFO:journalctl@ceph.node-exporter.b.gibba032.stdout:Mar 10 13:47:52 gibba032 bash[18247]: level=info ts=2021-03-10T13:47:52.613Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2021-03-10T13:47:54.822 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[9813]: cluster 2021-03-10T13:47:53.132217+0000 mgr.y (mgr.14140) 180 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:54.823 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[13015]: cluster 2021-03-10T13:47:53.132217+0000 mgr.y (mgr.14140) 180 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:54 gibba032 bash[9829]: cluster 2021-03-10T13:47:53.132217+0000 mgr.y (mgr.14140) 180 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:55.086 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.821Z caller=node_exporter.go:178 msg="Starting node_exporter" version="(version=1.1.2, branch=HEAD, revision=b597c1244d7bef49e6f3359c87a56dd7707f6719)" 2021-03-10T13:47:55.086 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.821Z caller=node_exporter.go:179 msg="Build context" build_context="(go=go1.15.8, user=root@f07de8ca602a, date=20210305-09:29:10)" 2021-03-10T13:47:55.086 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=filesystem_common.go:74 collector=filesystem msg="Parsed flag --collector.filesystem.ignored-mount-points" flag=^/(dev|proc|sys|var/lib/docker/.+)($|/) 2021-03-10T13:47:55.086 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=filesystem_common.go:76 collector=filesystem msg="Parsed flag --collector.filesystem.ignored-fs-types" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2021-03-10T13:47:55.087 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:106 msg="Enabled collectors" 2021-03-10T13:47:55.087 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=arp 2021-03-10T13:47:55.087 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=bcache 2021-03-10T13:47:55.087 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=bonding 2021-03-10T13:47:55.087 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=btrfs 2021-03-10T13:47:55.088 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=conntrack 2021-03-10T13:47:55.088 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=cpu 2021-03-10T13:47:55.088 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=cpufreq 2021-03-10T13:47:55.088 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=diskstats 2021-03-10T13:47:55.088 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=edac 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=entropy 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=fibrechannel 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=filefd 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=filesystem 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=hwmon 2021-03-10T13:47:55.089 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=infiniband 2021-03-10T13:47:55.090 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=ipvs 2021-03-10T13:47:55.090 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=loadavg 2021-03-10T13:47:55.090 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=mdadm 2021-03-10T13:47:55.090 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=meminfo 2021-03-10T13:47:55.091 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=netclass 2021-03-10T13:47:55.091 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=netdev 2021-03-10T13:47:55.091 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=netstat 2021-03-10T13:47:55.091 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=nfs 2021-03-10T13:47:55.092 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=nfsd 2021-03-10T13:47:55.092 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=powersupplyclass 2021-03-10T13:47:55.092 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=pressure 2021-03-10T13:47:55.092 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=rapl 2021-03-10T13:47:55.092 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=schedstat 2021-03-10T13:47:55.093 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=sockstat 2021-03-10T13:47:55.093 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=softnet 2021-03-10T13:47:55.093 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=stat 2021-03-10T13:47:55.093 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=textfile 2021-03-10T13:47:55.093 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=thermal_zone 2021-03-10T13:47:55.094 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=time 2021-03-10T13:47:55.094 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=udp_queues 2021-03-10T13:47:55.094 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=uname 2021-03-10T13:47:55.094 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=vmstat 2021-03-10T13:47:55.094 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=xfs 2021-03-10T13:47:55.095 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:113 collector=zfs 2021-03-10T13:47:55.095 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.822Z caller=node_exporter.go:195 msg="Listening on" address=:9100 2021-03-10T13:47:55.095 INFO:journalctl@ceph.node-exporter.a.gibba011.stdout:Mar 10 13:47:54 gibba011 bash[19706]: level=info ts=2021-03-10T13:47:54.823Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2021-03-10T13:47:55.335 INFO:teuthology.orchestra.run.gibba011.stdout:[client.0] 2021-03-10T13:47:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQALzkhgpWCiExAAcBXLV3IkWcjmwqdE+oPmXQ== 2021-03-10T13:47:56.454 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[9813]: cluster 2021-03-10T13:47:55.132487+0000 mgr.y (mgr.14140) 181 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:56.454 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[9813]: audit 2021-03-10T13:47:55.329321+0000 mon.a (mon.0) 487 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:47:56.454 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[9813]: audit 2021-03-10T13:47:55.333996+0000 mon.a (mon.0) 488 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:47:56.455 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[13015]: cluster 2021-03-10T13:47:55.132487+0000 mgr.y (mgr.14140) 181 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:56.455 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[13015]: audit 2021-03-10T13:47:55.329321+0000 mon.a (mon.0) 487 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:47:56.455 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:56 gibba011 bash[13015]: audit 2021-03-10T13:47:55.333996+0000 mon.a (mon.0) 488 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:47:56.456 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-03-10T13:47:56.456 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2021-03-10T13:47:56.457 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-03-10T13:47:56.465 DEBUG:teuthology.orchestra.run.gibba032:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2021-03-10T13:47:56.475 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:56 gibba032 bash[9829]: cluster 2021-03-10T13:47:55.132487+0000 mgr.y (mgr.14140) 181 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:56.476 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:56 gibba032 bash[9829]: audit 2021-03-10T13:47:55.329321+0000 mon.a (mon.0) 487 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:47:56.476 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:56 gibba032 bash[9829]: audit 2021-03-10T13:47:55.333996+0000 mon.a (mon.0) 488 : audit [INF] from='client.? 172.21.2.111:0/2344027963' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:47:58.404 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:47:58 gibba011 bash[9813]: cluster 2021-03-10T13:47:57.132868+0000 mgr.y (mgr.14140) 182 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:58.404 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:47:58 gibba011 bash[13015]: cluster 2021-03-10T13:47:57.132868+0000 mgr.y (mgr.14140) 182 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:47:58 gibba032 bash[9829]: cluster 2021-03-10T13:47:57.132868+0000 mgr.y (mgr.14140) 182 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:47:59.212 INFO:teuthology.orchestra.run.gibba032.stdout:[client.1] 2021-03-10T13:47:59.213 INFO:teuthology.orchestra.run.gibba032.stdout: key = AQAPzkhgySdjDBAAaBPSSnHH1pRgzCUrf1ETmg== 2021-03-10T13:47:59.250 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:47:59 gibba011 systemd[1]: Starting Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:47:59.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:47:59 gibba011 docker[20313]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-alertmanager.a 2021-03-10T13:47:59.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:47:59 gibba011 systemd[1]: Started Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:47:59.779 DEBUG:teuthology.orchestra.run.gibba032:> set -ex 2021-03-10T13:47:59.780 DEBUG:teuthology.orchestra.run.gibba032:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2021-03-10T13:47:59.780 DEBUG:teuthology.orchestra.run.gibba032:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-03-10T13:47:59.789 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd dump --format=json 2021-03-10T13:48:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: cluster 2021-03-10T13:47:59.133089+0000 mgr.y (mgr.14140) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.207430+0000 mon.b (mon.2) 22 : audit [INF] from='client.? 172.21.2.132:0/470966605' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.207714+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.210941+0000 mon.a (mon.0) 490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:48:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.261272+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.264037+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.328481+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]: dispatch 2021-03-10T13:48:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.332101+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]': finished 2021-03-10T13:48:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.332512+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]: dispatch 2021-03-10T13:48:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: audit 2021-03-10T13:47:59.339676+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]': finished 2021-03-10T13:48:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:00 gibba032 bash[9829]: cephadm 2021-03-10T13:47:59.340076+0000 mgr.y (mgr.14140) 184 : cephadm [INF] Deploying daemon grafana.a on gibba032 2021-03-10T13:48:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: cluster 2021-03-10T13:47:59.133089+0000 mgr.y (mgr.14140) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.207430+0000 mon.b (mon.2) 22 : audit [INF] from='client.? 172.21.2.132:0/470966605' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.207714+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.210941+0000 mon.a (mon.0) 490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:48:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: cluster 2021-03-10T13:47:59.133089+0000 mgr.y (mgr.14140) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.207430+0000 mon.b (mon.2) 22 : audit [INF] from='client.? 172.21.2.132:0/470966605' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.207714+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-03-10T13:48:00.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.210941+0000 mon.a (mon.0) 490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-03-10T13:48:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.261272+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:00.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.261272+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:00.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.264037+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.264037+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400673\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400732\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400760\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400786\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400812\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400837\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:45:57.400862\", \"created\": \"2021-03-10T13:45:56.793082\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:00.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.328481+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]: dispatch 2021-03-10T13:48:00.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.332101+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]': finished 2021-03-10T13:48:00.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.332512+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]: dispatch 2021-03-10T13:48:00.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: audit 2021-03-10T13:47:59.339676+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]': finished 2021-03-10T13:48:00.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[9813]: cephadm 2021-03-10T13:47:59.340076+0000 mgr.y (mgr.14140) 184 : cephadm [INF] Deploying daemon grafana.a on gibba032 2021-03-10T13:48:00.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.328481+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]: dispatch 2021-03-10T13:48:00.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.332101+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_crt","val":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa4CEQCv2ZSvhJpDJIDm8IoRG+ZDMA0GCSqGSIb3DQEBDQUAMCExDTAL\nBgNVBAoMBENlcGgxEDAOBgNVBAMMB2NlcGhhZG0wHhcNMjEwMzEwMTM0NzU5WhcN\nMzEwMzA4MTM0NzU5WjAhMQ0wCwYDVQQKDARDZXBoMRAwDgYDVQQDDAdjZXBoYWRt\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwF5my/LYZA74efyr6Ttb\n47DP9UgyFWGHgMROtA82ZteVKWiRm0MRq+qqBt9xvs888rf2Qs5NxeQ/Vkx4zdgt\nUktPSJvJDIMm8VKuDOtde2UASyDKvZWP16u6NeaAK1wPH9GaZJgOBiTPqu6UVMfV\nq/VYpYxYaixuPlHdeQT1wVn84LrDI+L2hDjfj1/FDJZ5Y06109/e6Qd+nZZbg7Jd\nGLjhl5RWoY9b5NbANBRFLa/w7yS8tt6HzwfxKjvIKf4MXej1sfgLSqj4ERFOpLuH\n2nZJWoYyfgzKN9w5kXrDILlxAs4c/HEJbDCXW2ixSXXsB38W2xM0CVVB1Fa7GuIq\n/QIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQCvf/yg4fSR4ZGYjvxBx8NWQ4sVEZMw\nIvXsMZ0l3qTXDx7abYmRly2qo8Y3kDRBc5SVHHi42ar3759ELTKXoZfMYlvQ7+WZ\nKKTTEecn83TungOVw4FOGm0KLUIhGXKGW6JLCyhpGltnSFW+np0iG925GR6u3GqO\ngdNHO3AXKQWkOrfUj7WnpIJvQBILLQ+YtP2OUqkQNpw18lFjehRotIayqHnDikHT\nx0NtukwOBJDCd2rvQOgB2eMEBq7Ls4o7u2uEOxpDOCRWgU3nrS0KytrrjoywTdxo\nBqH56Gsb4DcH7/IrQLM7c1aJ+1tCSsEf3mDLualp/mJE95iG83bF/I9F\n-----END CERTIFICATE-----\n"}]': finished 2021-03-10T13:48:00.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.332512+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]: dispatch 2021-03-10T13:48:00.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: audit 2021-03-10T13:47:59.339676+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/grafana_key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDAXmbL8thkDvh5\n/KvpO1vjsM/1SDIVYYeAxE60DzZm15UpaJGbQxGr6qoG33G+zzzyt/ZCzk3F5D9W\nTHjN2C1SS09Im8kMgybxUq4M6117ZQBLIMq9lY/Xq7o15oArXA8f0ZpkmA4GJM+q\n7pRUx9Wr9ViljFhqLG4+Ud15BPXBWfzgusMj4vaEON+PX8UMlnljTrXT397pB36d\nlluDsl0YuOGXlFahj1vk1sA0FEUtr/DvJLy23ofPB/EqO8gp/gxd6PWx+AtKqPgR\nEU6ku4fadklahjJ+DMo33DmResMguXECzhz8cQlsMJdbaLFJdewHfxbbEzQJVUHU\nVrsa4ir9AgMBAAECggEAWsszkYcpAvDd7hWHvik6EZleTdKpcNX7cpc03l9fueJ+\nkHC+aJ2bNZ0oat4DfZ+qkJSrMpTUuFYmXsDxM5aYrYLGBOsiVTmBo9z2MTrVgGwH\nW246wohYv95ZoR8N12k36jivulLaK6QsL+0zLWC0RNkj4d7/y/z0e42ADqJ8q6jP\n6azb9Yg4KaD+/mDbFKYA6r+adysYqZtwx3f1DR9HAVpyn1lkd8HXxHLWbeebiNwq\n/KXy+ELoQHgoqJaseJ5OTMNTatOy0Rk9vmBMc+ELq249g+w2vZ+lQAm5Mx12g03G\nbRqVr9z4WU1jw5kI8kE4iMr71d8tvLnsbW8ePMrGMQKBgQDfTsxUTvxHL4GCX7qi\nrsRFsI+KHICLGLUL4z7fZ45I7FqY5nhqxiVyu+f80xn4nr+elEWZT1Vrzq2izoZe\nqy2clIJI3CKZbPZhO2eszJj74X+vQMHisHGaRISHFBC9JLlKSx4kCTp2VIsTS072\nq2DKG0kLI3VW7J4JabiJ6LmWFwKBgQDciA9K5J/cClHs7sLAQ6jGWLr01q05fVPJ\nJAGjuj9BwOXZPvadF1B9e982k+BMwMwSuHwGhMjl8j8ml9UmoR74axJjfXvy3n3c\n2sboF2LOm9LOySRkpN4Rh6C3YepCGLM7+uo/PpqmG96c6p6CFIwFquYxDW7CO1cs\ncBWBPogICwKBgAMz8iJTb7Llr4nbLaYfs5j8S1LDzeQw2oftHn290oChQztmMtv1\nM5xNraomrPyv6jewEL9+L894C22GYM8CzYitaKD651QkB/SlEXudA6S5O0ecOiuy\nNC7uRKhbVVIhMLcsFKaXq0r/M293aQ6Irr63XXjXbVtt0WZ5KbhwqhjJAoGBAM1n\n5WaKf+NBsYEP+lF+M4tbRfeapQnxE8M7SFOpvBWQiAXLQYScHJpkcEmzfq2+vdWK\nA+tGbqDO+8cEkMJEPEaB5ETKslJKrcyx8aZncd6jAL0eWEltznFhdG6aQsmBnB/3\n24tA0mZ5euWJGk5quOjmfzWdH1bGUrHPTBhvl+B1AoGAAWB50pK6LBXYEPwr+sLd\nRarpD4hlCuDOD/q7yEmecWqMaTO+hOIpEZNKiERT+WG0tkgTjdp/RqOAd4q4k4N0\nSCyw5nBxeaxY6IesOdTF9ZRi3QmPiN12Tnw0DQMVdBM7sPfFiRUoqhGwXBdWvnMI\n9scr/ORSEJGMcdv4lfSDRvE=\n-----END PRIVATE KEY-----\n"}]': finished 2021-03-10T13:48:00.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:00 gibba011 bash[13015]: cephadm 2021-03-10T13:47:59.340076+0000 mgr.y (mgr.14140) 184 : cephadm [INF] Deploying daemon grafana.a on gibba032 2021-03-10T13:48:01.242 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.030Z caller=main.go:216 msg="Starting Alertmanager" version="(version=0.21.0, branch=HEAD, revision=4c6c03ebfe21009c546e4d1e9b92c371d67c021d)" 2021-03-10T13:48:01.243 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.030Z caller=main.go:217 build_context="(go=go1.14.4, user=root@dee35927357f, date=20200617-08:54:02)" 2021-03-10T13:48:01.243 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.031Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.21.2.111 port=9094 2021-03-10T13:48:01.243 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.032Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s 2021-03-10T13:48:01.243 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.046Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:48:01.243 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.046Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:48:01.244 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:01 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:01.048Z caller=main.go:485 msg=Listening address=:9093 2021-03-10T13:48:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:02 gibba032 bash[9829]: cluster 2021-03-10T13:48:01.133784+0000 mgr.y (mgr.14140) 185 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:02 gibba011 bash[9813]: cluster 2021-03-10T13:48:01.133784+0000 mgr.y (mgr.14140) 185 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:02 gibba011 bash[13015]: cluster 2021-03-10T13:48:01.133784+0000 mgr.y (mgr.14140) 185 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:02.833 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:48:02.834 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":47,"fsid":"6fd36d94-81a6-11eb-9080-001a4aab830c","created":"2021-03-10T13:42:44.617082+0000","modified":"2021-03-10T13:47:37.431108+0000","last_up_change":"2021-03-10T13:47:36.415267+0000","last_in_change":"2021-03-10T13:47:36.415267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-10T13:42:54.968165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"924baa6c-3f7e-44d7-a126-60ca7e5787ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6803","nonce":388310926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6805","nonce":388310926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6809","nonce":388310926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6807","nonce":388310926}]},"public_addr":"172.21.2.111:6803/388310926","cluster_addr":"172.21.2.111:6805/388310926","heartbeat_back_addr":"172.21.2.111:6809/388310926","heartbeat_front_addr":"172.21.2.111:6807/388310926","state":["exists","up"]},{"osd":1,"uuid":"1a81ef55-f29d-4fd7-81a8-384de101e141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6811","nonce":4088389352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6813","nonce":4088389352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6817","nonce":4088389352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6814","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6815","nonce":4088389352}]},"public_addr":"172.21.2.111:6811/4088389352","cluster_addr":"172.21.2.111:6813/4088389352","heartbeat_back_addr":"172.21.2.111:6817/4088389352","heartbeat_front_addr":"172.21.2.111:6815/4088389352","state":["exists","up"]},{"osd":2,"uuid":"c26948f0-d140-49d5-b246-b225db5e8c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6818","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6819","nonce":2275782540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6821","nonce":2275782540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6825","nonce":2275782540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6822","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6823","nonce":2275782540}]},"public_addr":"172.21.2.111:6819/2275782540","cluster_addr":"172.21.2.111:6821/2275782540","heartbeat_back_addr":"172.21.2.111:6825/2275782540","heartbeat_front_addr":"172.21.2.111:6823/2275782540","state":["exists","up"]},{"osd":3,"uuid":"c387e310-fde0-48e0-971a-0a016244271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6827","nonce":2123819943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6829","nonce":2123819943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6833","nonce":2123819943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6831","nonce":2123819943}]},"public_addr":"172.21.2.111:6827/2123819943","cluster_addr":"172.21.2.111:6829/2123819943","heartbeat_back_addr":"172.21.2.111:6833/2123819943","heartbeat_front_addr":"172.21.2.111:6831/2123819943","state":["exists","up"]},{"osd":4,"uuid":"68d2b345-b3a1-4ff1-bb20-44ec0c354123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6800","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6801","nonce":337025226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6802","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6803","nonce":337025226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6806","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6807","nonce":337025226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6804","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6805","nonce":337025226}]},"public_addr":"172.21.2.132:6801/337025226","cluster_addr":"172.21.2.132:6803/337025226","heartbeat_back_addr":"172.21.2.132:6807/337025226","heartbeat_front_addr":"172.21.2.132:6805/337025226","state":["exists","up"]},{"osd":5,"uuid":"8682e3b2-5b44-47b9-8181-83be4f601dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6808","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6809","nonce":4109259493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6810","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6811","nonce":4109259493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6814","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6815","nonce":4109259493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6812","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6813","nonce":4109259493}]},"public_addr":"172.21.2.132:6809/4109259493","cluster_addr":"172.21.2.132:6811/4109259493","heartbeat_back_addr":"172.21.2.132:6815/4109259493","heartbeat_front_addr":"172.21.2.132:6813/4109259493","state":["exists","up"]},{"osd":6,"uuid":"95e33995-e46c-4204-9623-3f2b7b02beb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6816","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6817","nonce":837531840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6818","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6819","nonce":837531840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6822","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6823","nonce":837531840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6820","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6821","nonce":837531840}]},"public_addr":"172.21.2.132:6817/837531840","cluster_addr":"172.21.2.132:6819/837531840","heartbeat_back_addr":"172.21.2.132:6823/837531840","heartbeat_front_addr":"172.21.2.132:6821/837531840","state":["exists","up"]},{"osd":7,"uuid":"ce0503e7-615c-4f41-bc6c-7eb89d164d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6824","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6825","nonce":189669098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6826","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6827","nonce":189669098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6830","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6831","nonce":189669098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6828","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6829","nonce":189669098}]},"public_addr":"172.21.2.132:6825/189669098","cluster_addr":"172.21.2.132:6827/189669098","heartbeat_back_addr":"172.21.2.132:6831/189669098","heartbeat_front_addr":"172.21.2.132:6829/189669098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:44:54.458607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:17.263260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:40.454158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:03.647607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:24.508931+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:47.943640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:10.232714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:34.652555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.111:6800/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3399696856":"2021-03-11T13:43:16.222472+0000","172.21.2.111:6801/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/1453015932":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3970935116":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/308369833":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/363049641":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6800/2764398316":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/3059131095":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6801/2764398316":"2021-03-11T13:43:01.684797+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-10T13:48:03.174 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:03 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:03.032Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000041843s 2021-03-10T13:48:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:03 gibba011 bash[9813]: audit 2021-03-10T13:48:02.832226+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1285476083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:03 gibba011 bash[13015]: audit 2021-03-10T13:48:02.832226+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1285476083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:03 gibba032 bash[9829]: audit 2021-03-10T13:48:02.832226+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1285476083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:03.522 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-03-10T13:42:54.968165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2021-03-10T13:48:03.523 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd pool get device_health_metrics pg_num 2021-03-10T13:48:04.453 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:04 gibba011 bash[9813]: cluster 2021-03-10T13:48:03.134752+0000 mgr.y (mgr.14140) 186 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:04.454 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:04 gibba011 bash[13015]: cluster 2021-03-10T13:48:03.134752+0000 mgr.y (mgr.14140) 186 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:04 gibba032 bash[9829]: cluster 2021-03-10T13:48:03.134752+0000 mgr.y (mgr.14140) 186 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:06.181 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-03-10T13:48:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:06 gibba032 bash[9829]: cluster 2021-03-10T13:48:05.135203+0000 mgr.y (mgr.14140) 187 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:06 gibba032 bash[9829]: audit 2021-03-10T13:48:06.180662+0000 mon.a (mon.0) 497 : audit [DBG] from='client.? 172.21.2.111:0/4227942460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-03-10T13:48:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:06 gibba011 bash[9813]: cluster 2021-03-10T13:48:05.135203+0000 mgr.y (mgr.14140) 187 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:06 gibba011 bash[9813]: audit 2021-03-10T13:48:06.180662+0000 mon.a (mon.0) 497 : audit [DBG] from='client.? 172.21.2.111:0/4227942460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-03-10T13:48:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:06 gibba011 bash[13015]: cluster 2021-03-10T13:48:05.135203+0000 mgr.y (mgr.14140) 187 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:06 gibba011 bash[13015]: audit 2021-03-10T13:48:06.180662+0000 mon.a (mon.0) 497 : audit [DBG] from='client.? 172.21.2.111:0/4227942460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-03-10T13:48:06.872 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-03-10T13:48:06.873 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2021-03-10T13:48:06.873 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph mgr dump --format=json 2021-03-10T13:48:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:08 gibba011 bash[9813]: cluster 2021-03-10T13:48:07.136186+0000 mgr.y (mgr.14140) 188 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:08.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:08 gibba011 bash[13015]: cluster 2021-03-10T13:48:07.136186+0000 mgr.y (mgr.14140) 188 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:08 gibba032 bash[9829]: cluster 2021-03-10T13:48:07.136186+0000 mgr.y (mgr.14140) 188 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:09.601 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:48:10.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:09 gibba032 docker[18774]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-grafana.a 2021-03-10T13:48:10.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:09 gibba032 systemd[1]: Started Ceph grafana.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:48:10.089 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":15,"active_gid":14140,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6800","nonce":1131343133},{"type":"v1","addr":"172.21.2.111:6801","nonce":1131343133}]},"active_addr":"172.21.2.111:6801/1131343133","active_change":"2021-03-10T13:43:16.222794+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":34109,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34115,"name":"gibba032.whopeo","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://gibba011:8443/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":5,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":1701951737}]},{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":3586213168}]},{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":3248476364}]}]}} 2021-03-10T13:48:10.095 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2021-03-10T13:48:10.095 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2021-03-10T13:48:10.096 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd dump --format=json 2021-03-10T13:48:10.481 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[9813]: cluster 2021-03-10T13:48:09.136950+0000 mgr.y (mgr.14140) 189 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:10.481 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[9813]: audit 2021-03-10T13:48:09.595766+0000 mon.a (mon.0) 498 : audit [DBG] from='client.? 172.21.2.111:0/2048744858' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2021-03-10T13:48:10.481 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[13015]: cluster 2021-03-10T13:48:09.136950+0000 mgr.y (mgr.14140) 189 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:10.482 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[13015]: audit 2021-03-10T13:48:09.595766+0000 mon.a (mon.0) 498 : audit [DBG] from='client.? 172.21.2.111:0/2048744858' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2021-03-10T13:48:10.482 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[9813]: audit 2021-03-10T13:48:09.617692+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:10.483 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[9813]: audit 2021-03-10T13:48:09.620582+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:10.483 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[9813]: audit 2021-03-10T13:48:09.621243+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:10.484 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[13015]: audit 2021-03-10T13:48:09.617692+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:10.484 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[13015]: audit 2021-03-10T13:48:09.620582+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:10.484 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:10 gibba011 bash[13015]: audit 2021-03-10T13:48:09.621243+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:10.485 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[9829]: cluster 2021-03-10T13:48:09.136950+0000 mgr.y (mgr.14140) 189 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:10.485 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[9829]: audit 2021-03-10T13:48:09.595766+0000 mon.a (mon.0) 498 : audit [DBG] from='client.? 172.21.2.111:0/2048744858' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2021-03-10T13:48:10.486 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[9829]: audit 2021-03-10T13:48:09.617692+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:10.486 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[9829]: audit 2021-03-10T13:48:09.620582+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295439\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295706\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.295873\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296030\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296180\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296328\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296474\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:47:49.296622\", \"created\": \"2021-03-10T13:47:48.610593\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"status\": 1, \"status_desc\": \"starting\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:10.486 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[9829]: audit 2021-03-10T13:48:09.621243+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:10.756 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Starting Grafana" logger=server version=6.6.2 commit=3fa63cfc34 branch=HEAD compiled=2020-02-20T12:03:49+0000 2021-03-10T13:48:10.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2021-03-10T13:48:10.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2021-03-10T13:48:10.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2021-03-10T13:48:10.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2021-03-10T13:48:10.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2021-03-10T13:48:10.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2021-03-10T13:48:10.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2021-03-10T13:48:10.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2021-03-10T13:48:10.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2021-03-10T13:48:10.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2021-03-10T13:48:10.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2021-03-10T13:48:10.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="App mode production" logger=settings 2021-03-10T13:48:10.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Initializing SqlStore" logger=server 2021-03-10T13:48:10.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2021-03-10T13:48:10.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Starting DB migration" logger=migrator 2021-03-10T13:48:10.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2021-03-10T13:48:10.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2021-03-10T13:48:11.045 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2021-03-10T13:48:11.045 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2021-03-10T13:48:11.045 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2021-03-10T13:48:11.046 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2021-03-10T13:48:11.046 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2021-03-10T13:48:11.046 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:10 gibba032 bash[18790]: t=2021-03-10T13:48:10+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2021-03-10T13:48:11.046 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2021-03-10T13:48:11.297 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2021-03-10T13:48:11.297 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2021-03-10T13:48:11.297 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2021-03-10T13:48:11.297 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2021-03-10T13:48:11.297 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2021-03-10T13:48:11.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[9829]: audit 2021-03-10T13:48:10.583086+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:11.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[9829]: audit 2021-03-10T13:48:10.585741+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:11.299 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[9829]: audit 2021-03-10T13:48:10.586250+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:11.330 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:11.033Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.001100576s 2021-03-10T13:48:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[9813]: audit 2021-03-10T13:48:10.583086+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:11.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[13015]: audit 2021-03-10T13:48:10.583086+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[13015]: audit 2021-03-10T13:48:10.585741+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[13015]: audit 2021-03-10T13:48:10.586250+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[9813]: audit 2021-03-10T13:48:10.585741+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"e9c6e2b550a5\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582316\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:47:59.273898\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:50.171412\"}}, \"last_daemon_update\": \"2021-03-10T13:48:10.582389\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:11 gibba011 bash[9813]: audit 2021-03-10T13:48:10.586250+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:11.566 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2021-03-10T13:48:11.566 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2021-03-10T13:48:11.566 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2021-03-10T13:48:11.817 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2021-03-10T13:48:11.817 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2021-03-10T13:48:11.818 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2021-03-10T13:48:11.818 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2021-03-10T13:48:11.818 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2021-03-10T13:48:12.149 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2021-03-10T13:48:12.149 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2021-03-10T13:48:12.150 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:11 gibba032 bash[18790]: t=2021-03-10T13:48:11+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2021-03-10T13:48:12.150 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2021-03-10T13:48:12.150 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2021-03-10T13:48:12.409 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2021-03-10T13:48:12.409 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2021-03-10T13:48:12.410 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2021-03-10T13:48:12.410 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2021-03-10T13:48:12.410 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2021-03-10T13:48:12.411 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[9829]: cluster 2021-03-10T13:48:11.137671+0000 mgr.y (mgr.14140) 190 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:12.411 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[9829]: audit 2021-03-10T13:48:11.423231+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:12.412 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[9829]: audit 2021-03-10T13:48:11.432205+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:12.412 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[9829]: cephadm 2021-03-10T13:48:11.438801+0000 mgr.y (mgr.14140) 191 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:48:12.412 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[9829]: cephadm 2021-03-10T13:48:11.448578+0000 mgr.y (mgr.14140) 192 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:48:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[9813]: cluster 2021-03-10T13:48:11.137671+0000 mgr.y (mgr.14140) 190 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[13015]: cluster 2021-03-10T13:48:11.137671+0000 mgr.y (mgr.14140) 190 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:12.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[9813]: audit 2021-03-10T13:48:11.423231+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:12.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[9813]: audit 2021-03-10T13:48:11.432205+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:12.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[9813]: cephadm 2021-03-10T13:48:11.438801+0000 mgr.y (mgr.14140) 191 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:48:12.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[9813]: cephadm 2021-03-10T13:48:11.448578+0000 mgr.y (mgr.14140) 192 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:48:12.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[13015]: audit 2021-03-10T13:48:11.423231+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:12.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[13015]: audit 2021-03-10T13:48:11.432205+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"b2e6d29eb45d\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419325\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:47:48.635634\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:47:34.764304\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:11.420019\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:12.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[13015]: cephadm 2021-03-10T13:48:11.438801+0000 mgr.y (mgr.14140) 191 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:48:12.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:12 gibba011 bash[13015]: cephadm 2021-03-10T13:48:11.448578+0000 mgr.y (mgr.14140) 192 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:48:12.720 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2021-03-10T13:48:12.720 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2021-03-10T13:48:12.720 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2021-03-10T13:48:12.720 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2021-03-10T13:48:12.721 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2021-03-10T13:48:12.988 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2021-03-10T13:48:12.988 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2021-03-10T13:48:12.988 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2021-03-10T13:48:13.239 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:12 gibba032 bash[18790]: t=2021-03-10T13:48:12+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2021-03-10T13:48:13.239 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2021-03-10T13:48:13.240 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2021-03-10T13:48:13.240 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2021-03-10T13:48:13.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2021-03-10T13:48:13.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2021-03-10T13:48:13.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2021-03-10T13:48:13.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2021-03-10T13:48:13.800 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2021-03-10T13:48:13.800 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2021-03-10T13:48:13.801 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2021-03-10T13:48:13.801 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2021-03-10T13:48:13.801 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2021-03-10T13:48:13.802 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:48:13.802 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":47,"fsid":"6fd36d94-81a6-11eb-9080-001a4aab830c","created":"2021-03-10T13:42:44.617082+0000","modified":"2021-03-10T13:47:37.431108+0000","last_up_change":"2021-03-10T13:47:36.415267+0000","last_in_change":"2021-03-10T13:47:36.415267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-10T13:42:54.968165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"924baa6c-3f7e-44d7-a126-60ca7e5787ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6803","nonce":388310926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6805","nonce":388310926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6809","nonce":388310926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6807","nonce":388310926}]},"public_addr":"172.21.2.111:6803/388310926","cluster_addr":"172.21.2.111:6805/388310926","heartbeat_back_addr":"172.21.2.111:6809/388310926","heartbeat_front_addr":"172.21.2.111:6807/388310926","state":["exists","up"]},{"osd":1,"uuid":"1a81ef55-f29d-4fd7-81a8-384de101e141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6811","nonce":4088389352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6813","nonce":4088389352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6817","nonce":4088389352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6814","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6815","nonce":4088389352}]},"public_addr":"172.21.2.111:6811/4088389352","cluster_addr":"172.21.2.111:6813/4088389352","heartbeat_back_addr":"172.21.2.111:6817/4088389352","heartbeat_front_addr":"172.21.2.111:6815/4088389352","state":["exists","up"]},{"osd":2,"uuid":"c26948f0-d140-49d5-b246-b225db5e8c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6818","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6819","nonce":2275782540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6821","nonce":2275782540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6825","nonce":2275782540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6822","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6823","nonce":2275782540}]},"public_addr":"172.21.2.111:6819/2275782540","cluster_addr":"172.21.2.111:6821/2275782540","heartbeat_back_addr":"172.21.2.111:6825/2275782540","heartbeat_front_addr":"172.21.2.111:6823/2275782540","state":["exists","up"]},{"osd":3,"uuid":"c387e310-fde0-48e0-971a-0a016244271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6827","nonce":2123819943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6829","nonce":2123819943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6833","nonce":2123819943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6831","nonce":2123819943}]},"public_addr":"172.21.2.111:6827/2123819943","cluster_addr":"172.21.2.111:6829/2123819943","heartbeat_back_addr":"172.21.2.111:6833/2123819943","heartbeat_front_addr":"172.21.2.111:6831/2123819943","state":["exists","up"]},{"osd":4,"uuid":"68d2b345-b3a1-4ff1-bb20-44ec0c354123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6800","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6801","nonce":337025226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6802","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6803","nonce":337025226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6806","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6807","nonce":337025226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6804","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6805","nonce":337025226}]},"public_addr":"172.21.2.132:6801/337025226","cluster_addr":"172.21.2.132:6803/337025226","heartbeat_back_addr":"172.21.2.132:6807/337025226","heartbeat_front_addr":"172.21.2.132:6805/337025226","state":["exists","up"]},{"osd":5,"uuid":"8682e3b2-5b44-47b9-8181-83be4f601dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6808","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6809","nonce":4109259493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6810","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6811","nonce":4109259493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6814","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6815","nonce":4109259493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6812","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6813","nonce":4109259493}]},"public_addr":"172.21.2.132:6809/4109259493","cluster_addr":"172.21.2.132:6811/4109259493","heartbeat_back_addr":"172.21.2.132:6815/4109259493","heartbeat_front_addr":"172.21.2.132:6813/4109259493","state":["exists","up"]},{"osd":6,"uuid":"95e33995-e46c-4204-9623-3f2b7b02beb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6816","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6817","nonce":837531840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6818","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6819","nonce":837531840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6822","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6823","nonce":837531840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6820","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6821","nonce":837531840}]},"public_addr":"172.21.2.132:6817/837531840","cluster_addr":"172.21.2.132:6819/837531840","heartbeat_back_addr":"172.21.2.132:6823/837531840","heartbeat_front_addr":"172.21.2.132:6821/837531840","state":["exists","up"]},{"osd":7,"uuid":"ce0503e7-615c-4f41-bc6c-7eb89d164d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6824","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6825","nonce":189669098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6826","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6827","nonce":189669098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6830","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6831","nonce":189669098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6828","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6829","nonce":189669098}]},"public_addr":"172.21.2.132:6825/189669098","cluster_addr":"172.21.2.132:6827/189669098","heartbeat_back_addr":"172.21.2.132:6831/189669098","heartbeat_front_addr":"172.21.2.132:6829/189669098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:44:54.458607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:17.263260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:40.454158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:03.647607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:24.508931+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:47.943640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:10.232714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:34.652555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.111:6800/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3399696856":"2021-03-11T13:43:16.222472+0000","172.21.2.111:6801/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/1453015932":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3970935116":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/308369833":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/363049641":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6800/2764398316":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/3059131095":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6801/2764398316":"2021-03-11T13:43:01.684797+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-10T13:48:14.051 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2021-03-10T13:48:14.051 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2021-03-10T13:48:14.052 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:13 gibba032 bash[18790]: t=2021-03-10T13:48:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2021-03-10T13:48:14.052 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2021-03-10T13:48:14.344 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2021-03-10T13:48:14.345 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2021-03-10T13:48:14.345 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2021-03-10T13:48:14.345 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[9829]: cluster 2021-03-10T13:48:13.138464+0000 mgr.y (mgr.14140) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:14.345 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[9829]: audit 2021-03-10T13:48:13.801090+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/3688977592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:14.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:14 gibba011 bash[9813]: cluster 2021-03-10T13:48:13.138464+0000 mgr.y (mgr.14140) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:14.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:14 gibba011 bash[9813]: audit 2021-03-10T13:48:13.801090+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/3688977592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:14.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:14 gibba011 bash[13015]: cluster 2021-03-10T13:48:13.138464+0000 mgr.y (mgr.14140) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:14.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:14 gibba011 bash[13015]: audit 2021-03-10T13:48:13.801090+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/3688977592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:14.605 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2021-03-10T13:48:14.605 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2021-03-10T13:48:14.605 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2021-03-10T13:48:14.605 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2021-03-10T13:48:14.605 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2021-03-10T13:48:14.881 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2021-03-10T13:48:14.882 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2021-03-10T13:48:14.882 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2021-03-10T13:48:14.882 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2021-03-10T13:48:14.883 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2021-03-10T13:48:15.099 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2021-03-10T13:48:15.100 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd dump --format=json 2021-03-10T13:48:15.199 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:14 gibba032 bash[18790]: t=2021-03-10T13:48:14+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2021-03-10T13:48:15.199 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2021-03-10T13:48:15.199 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2021-03-10T13:48:15.200 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2021-03-10T13:48:15.200 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2021-03-10T13:48:15.501 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2021-03-10T13:48:15.502 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2021-03-10T13:48:15.502 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2021-03-10T13:48:15.502 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2021-03-10T13:48:15.502 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2021-03-10T13:48:15.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2021-03-10T13:48:15.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2021-03-10T13:48:15.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2021-03-10T13:48:15.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2021-03-10T13:48:16.130 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2021-03-10T13:48:16.131 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2021-03-10T13:48:16.131 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2021-03-10T13:48:16.131 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:15 gibba032 bash[18790]: t=2021-03-10T13:48:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2021-03-10T13:48:16.131 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2021-03-10T13:48:16.132 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2021-03-10T13:48:16.406 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:16 gibba011 bash[9813]: cluster 2021-03-10T13:48:15.138898+0000 mgr.y (mgr.14140) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:16.406 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:16 gibba011 bash[13015]: cluster 2021-03-10T13:48:15.138898+0000 mgr.y (mgr.14140) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:16.440 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[9829]: cluster 2021-03-10T13:48:15.138898+0000 mgr.y (mgr.14140) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:16.440 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2021-03-10T13:48:16.441 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2021-03-10T13:48:16.441 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2021-03-10T13:48:16.691 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2021-03-10T13:48:16.692 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2021-03-10T13:48:16.692 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2021-03-10T13:48:16.692 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2021-03-10T13:48:16.942 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2021-03-10T13:48:16.943 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2021-03-10T13:48:16.943 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2021-03-10T13:48:16.943 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2021-03-10T13:48:17.211 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:16 gibba032 bash[18790]: t=2021-03-10T13:48:16+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2021-03-10T13:48:17.212 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2021-03-10T13:48:17.212 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2021-03-10T13:48:17.212 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2021-03-10T13:48:17.481 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2021-03-10T13:48:17.481 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2021-03-10T13:48:17.482 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2021-03-10T13:48:17.482 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2021-03-10T13:48:17.482 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2021-03-10T13:48:17.534 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:17 gibba011 systemd[1]: Stopping Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:48:17.534 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:17 gibba011 bash[20330]: level=info ts=2021-03-10T13:48:17.413Z caller=main.go:524 msg="Received SIGTERM, exiting gracefully..." 2021-03-10T13:48:17.732 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2021-03-10T13:48:17.732 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2021-03-10T13:48:17.732 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2021-03-10T13:48:17.733 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2021-03-10T13:48:18.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2021-03-10T13:48:18.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2021-03-10T13:48:18.008 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2021-03-10T13:48:18.008 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:17 gibba032 bash[18790]: t=2021-03-10T13:48:17+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2021-03-10T13:48:18.293 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[9829]: cluster 2021-03-10T13:48:17.139789+0000 mgr.y (mgr.14140) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:18.294 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2021-03-10T13:48:18.294 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2021-03-10T13:48:18.295 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2021-03-10T13:48:18.295 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2021-03-10T13:48:18.464 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:18 gibba011 bash[9813]: cluster 2021-03-10T13:48:17.139789+0000 mgr.y (mgr.14140) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:18.465 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:18 gibba011 bash[13015]: cluster 2021-03-10T13:48:17.139789+0000 mgr.y (mgr.14140) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:18.560 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2021-03-10T13:48:18.561 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2021-03-10T13:48:18.561 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2021-03-10T13:48:18.561 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2021-03-10T13:48:18.561 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2021-03-10T13:48:18.562 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2021-03-10T13:48:18.830 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 docker[21498]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-alertmanager.a 2021-03-10T13:48:18.830 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@alertmanager.a.service: Succeeded. 2021-03-10T13:48:18.831 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 systemd[1]: Stopped Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:48:18.831 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 systemd[1]: Starting Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:48:18.831 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 docker[21547]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-alertmanager.a 2021-03-10T13:48:18.831 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:18 gibba011 systemd[1]: Started Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:48:19.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2021-03-10T13:48:19.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:18 gibba032 bash[18790]: t=2021-03-10T13:48:18+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2021-03-10T13:48:19.230 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:48:19.230 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":47,"fsid":"6fd36d94-81a6-11eb-9080-001a4aab830c","created":"2021-03-10T13:42:44.617082+0000","modified":"2021-03-10T13:47:37.431108+0000","last_up_change":"2021-03-10T13:47:36.415267+0000","last_in_change":"2021-03-10T13:47:36.415267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-10T13:42:54.968165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"924baa6c-3f7e-44d7-a126-60ca7e5787ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6803","nonce":388310926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6805","nonce":388310926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6809","nonce":388310926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":388310926},{"type":"v1","addr":"172.21.2.111:6807","nonce":388310926}]},"public_addr":"172.21.2.111:6803/388310926","cluster_addr":"172.21.2.111:6805/388310926","heartbeat_back_addr":"172.21.2.111:6809/388310926","heartbeat_front_addr":"172.21.2.111:6807/388310926","state":["exists","up"]},{"osd":1,"uuid":"1a81ef55-f29d-4fd7-81a8-384de101e141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6811","nonce":4088389352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6813","nonce":4088389352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6817","nonce":4088389352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6814","nonce":4088389352},{"type":"v1","addr":"172.21.2.111:6815","nonce":4088389352}]},"public_addr":"172.21.2.111:6811/4088389352","cluster_addr":"172.21.2.111:6813/4088389352","heartbeat_back_addr":"172.21.2.111:6817/4088389352","heartbeat_front_addr":"172.21.2.111:6815/4088389352","state":["exists","up"]},{"osd":2,"uuid":"c26948f0-d140-49d5-b246-b225db5e8c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6818","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6819","nonce":2275782540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6821","nonce":2275782540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6825","nonce":2275782540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6822","nonce":2275782540},{"type":"v1","addr":"172.21.2.111:6823","nonce":2275782540}]},"public_addr":"172.21.2.111:6819/2275782540","cluster_addr":"172.21.2.111:6821/2275782540","heartbeat_back_addr":"172.21.2.111:6825/2275782540","heartbeat_front_addr":"172.21.2.111:6823/2275782540","state":["exists","up"]},{"osd":3,"uuid":"c387e310-fde0-48e0-971a-0a016244271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6827","nonce":2123819943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6829","nonce":2123819943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6833","nonce":2123819943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":2123819943},{"type":"v1","addr":"172.21.2.111:6831","nonce":2123819943}]},"public_addr":"172.21.2.111:6827/2123819943","cluster_addr":"172.21.2.111:6829/2123819943","heartbeat_back_addr":"172.21.2.111:6833/2123819943","heartbeat_front_addr":"172.21.2.111:6831/2123819943","state":["exists","up"]},{"osd":4,"uuid":"68d2b345-b3a1-4ff1-bb20-44ec0c354123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6800","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6801","nonce":337025226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6802","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6803","nonce":337025226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6806","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6807","nonce":337025226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6804","nonce":337025226},{"type":"v1","addr":"172.21.2.132:6805","nonce":337025226}]},"public_addr":"172.21.2.132:6801/337025226","cluster_addr":"172.21.2.132:6803/337025226","heartbeat_back_addr":"172.21.2.132:6807/337025226","heartbeat_front_addr":"172.21.2.132:6805/337025226","state":["exists","up"]},{"osd":5,"uuid":"8682e3b2-5b44-47b9-8181-83be4f601dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6808","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6809","nonce":4109259493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6810","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6811","nonce":4109259493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6814","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6815","nonce":4109259493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6812","nonce":4109259493},{"type":"v1","addr":"172.21.2.132:6813","nonce":4109259493}]},"public_addr":"172.21.2.132:6809/4109259493","cluster_addr":"172.21.2.132:6811/4109259493","heartbeat_back_addr":"172.21.2.132:6815/4109259493","heartbeat_front_addr":"172.21.2.132:6813/4109259493","state":["exists","up"]},{"osd":6,"uuid":"95e33995-e46c-4204-9623-3f2b7b02beb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6816","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6817","nonce":837531840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6818","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6819","nonce":837531840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6822","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6823","nonce":837531840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6820","nonce":837531840},{"type":"v1","addr":"172.21.2.132:6821","nonce":837531840}]},"public_addr":"172.21.2.132:6817/837531840","cluster_addr":"172.21.2.132:6819/837531840","heartbeat_back_addr":"172.21.2.132:6823/837531840","heartbeat_front_addr":"172.21.2.132:6821/837531840","state":["exists","up"]},{"osd":7,"uuid":"ce0503e7-615c-4f41-bc6c-7eb89d164d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6824","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6825","nonce":189669098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6826","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6827","nonce":189669098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6830","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6831","nonce":189669098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.132:6828","nonce":189669098},{"type":"v1","addr":"172.21.2.132:6829","nonce":189669098}]},"public_addr":"172.21.2.132:6825/189669098","cluster_addr":"172.21.2.132:6827/189669098","heartbeat_back_addr":"172.21.2.132:6831/189669098","heartbeat_front_addr":"172.21.2.132:6829/189669098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:44:54.458607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:17.263260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:45:40.454158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:03.647607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:24.508931+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:46:47.943640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:10.232714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2021-03-10T13:47:34.652555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.111:6800/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3399696856":"2021-03-11T13:43:16.222472+0000","172.21.2.111:6801/767229161":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/1453015932":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/3970935116":"2021-03-11T13:43:16.222472+0000","172.21.2.111:0/308369833":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/363049641":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6800/2764398316":"2021-03-11T13:43:01.684797+0000","172.21.2.111:0/3059131095":"2021-03-11T13:43:01.684797+0000","172.21.2.111:6801/2764398316":"2021-03-11T13:43:01.684797+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-10T13:48:19.292 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[9813]: audit 2021-03-10T13:48:18.502496+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:19.292 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[13015]: audit 2021-03-10T13:48:18.502496+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:19.293 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[9813]: audit 2021-03-10T13:48:18.506642+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:19.293 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[9813]: cephadm 2021-03-10T13:48:18.507177+0000 mgr.y (mgr.14140) 196 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2021-03-10T13:48:19.294 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[9813]: cephadm 2021-03-10T13:48:18.509396+0000 mgr.y (mgr.14140) 197 : cephadm [INF] Reconfiguring daemon prometheus.a on gibba032 2021-03-10T13:48:19.294 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[13015]: audit 2021-03-10T13:48:18.506642+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:19.294 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[13015]: cephadm 2021-03-10T13:48:18.507177+0000 mgr.y (mgr.14140) 196 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2021-03-10T13:48:19.295 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:19 gibba011 bash[13015]: cephadm 2021-03-10T13:48:18.509396+0000 mgr.y (mgr.14140) 197 : cephadm [INF] Reconfiguring daemon prometheus.a on gibba032 2021-03-10T13:48:19.410 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[18790]: t=2021-03-10T13:48:19+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2021-03-10T13:48:19.410 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[18790]: t=2021-03-10T13:48:19+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2021-03-10T13:48:19.411 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[9829]: audit 2021-03-10T13:48:18.502496+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:19.411 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[9829]: audit 2021-03-10T13:48:18.506642+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582048\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582126\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582156\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582184\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582210\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582236\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582262\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:10.582290\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:19.412 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[9829]: cephadm 2021-03-10T13:48:18.507177+0000 mgr.y (mgr.14140) 196 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2021-03-10T13:48:19.412 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[9829]: cephadm 2021-03-10T13:48:18.509396+0000 mgr.y (mgr.14140) 197 : cephadm [INF] Reconfiguring daemon prometheus.a on gibba032 2021-03-10T13:48:19.733 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[18790]: t=2021-03-10T13:48:19+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2021-03-10T13:48:20.008 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[18790]: t=2021-03-10T13:48:19+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2021-03-10T13:48:20.009 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:19 gibba032 bash[18790]: t=2021-03-10T13:48:19+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2021-03-10T13:48:20.137 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2021-03-10T13:48:20.422 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[9829]: cluster 2021-03-10T13:48:19.140119+0000 mgr.y (mgr.14140) 198 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:20.422 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[9829]: audit 2021-03-10T13:48:19.229009+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3614336196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:20.423 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2021-03-10T13:48:20.423 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2021-03-10T13:48:20.423 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2021-03-10T13:48:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[9813]: cluster 2021-03-10T13:48:19.140119+0000 mgr.y (mgr.14140) 198 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[9813]: audit 2021-03-10T13:48:19.229009+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3614336196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[13015]: cluster 2021-03-10T13:48:19.140119+0000 mgr.y (mgr.14140) 198 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[13015]: audit 2021-03-10T13:48:19.229009+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3614336196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-03-10T13:48:20.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.224Z caller=main.go:216 msg="Starting Alertmanager" version="(version=0.21.0, branch=HEAD, revision=4c6c03ebfe21009c546e4d1e9b92c371d67c021d)" 2021-03-10T13:48:20.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.224Z caller=main.go:217 build_context="(go=go1.14.4, user=root@dee35927357f, date=20200617-08:54:02)" 2021-03-10T13:48:20.582 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.225Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.21.2.111 port=9094 2021-03-10T13:48:20.582 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.226Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s 2021-03-10T13:48:20.582 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.240Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:48:20.582 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.241Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:48:20.582 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:20.242Z caller=main.go:485 msg=Listening address=:9093 2021-03-10T13:48:20.676 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2021-03-10T13:48:20.676 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2021-03-10T13:48:20.677 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2021-03-10T13:48:21.040 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2021-03-10T13:48:21.041 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:20 gibba032 bash[18790]: t=2021-03-10T13:48:20+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2021-03-10T13:48:21.257 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:21 gibba032 bash[18790]: t=2021-03-10T13:48:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2021-03-10T13:48:21.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:21 gibba032 bash[18790]: t=2021-03-10T13:48:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2021-03-10T13:48:21.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:21 gibba032 bash[18790]: t=2021-03-10T13:48:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2021-03-10T13:48:21.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:21 gibba032 bash[18790]: t=2021-03-10T13:48:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2021-03-10T13:48:22.225 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:21 gibba032 bash[18790]: t=2021-03-10T13:48:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2021-03-10T13:48:22.226 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2021-03-10T13:48:22.226 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[9829]: cluster 2021-03-10T13:48:21.140714+0000 mgr.y (mgr.14140) 199 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:22.227 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:22 gibba011 bash[9813]: cluster 2021-03-10T13:48:21.140714+0000 mgr.y (mgr.14140) 199 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:22.228 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:22 gibba011 bash[13015]: cluster 2021-03-10T13:48:21.140714+0000 mgr.y (mgr.14140) 199 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:22.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2021-03-10T13:48:22.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:22 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:22.226Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000127345s 2021-03-10T13:48:22.886 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2021-03-10T13:48:22.887 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2021-03-10T13:48:22.887 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2021-03-10T13:48:22.887 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2021-03-10T13:48:23.161 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:22 gibba032 bash[18790]: t=2021-03-10T13:48:22+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2021-03-10T13:48:23.161 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2021-03-10T13:48:23.506 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2021-03-10T13:48:23.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2021-03-10T13:48:23.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2021-03-10T13:48:23.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2021-03-10T13:48:23.523 INFO:teuthology.orchestra.run.gibba011.stdout:42949673004 2021-03-10T13:48:23.523 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2021-03-10T13:48:24.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2021-03-10T13:48:24.007 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:23 gibba032 bash[18790]: t=2021-03-10T13:48:23+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2021-03-10T13:48:24.367 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[9829]: cluster 2021-03-10T13:48:23.141415+0000 mgr.y (mgr.14140) 200 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:24.367 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 systemd[1]: Stopping Ceph prometheus.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:48:24.368 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2021-03-10T13:48:24.368 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2021-03-10T13:48:24.368 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2021-03-10T13:48:24.458 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:24 gibba011 bash[9813]: cluster 2021-03-10T13:48:23.141415+0000 mgr.y (mgr.14140) 200 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:24.458 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:24 gibba011 bash[13015]: cluster 2021-03-10T13:48:23.141415+0000 mgr.y (mgr.14140) 200 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:24.701 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2021-03-10T13:48:24.702 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2021-03-10T13:48:24.702 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2021-03-10T13:48:24.702 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2021-03-10T13:48:24.703 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=warn ts=2021-03-10T13:48:24.401Z caller=main.go:629 msg="Received SIGTERM, exiting gracefully..." 2021-03-10T13:48:24.703 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.401Z caller=main.go:652 msg="Stopping scrape discovery manager..." 2021-03-10T13:48:24.703 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.401Z caller=main.go:666 msg="Stopping notify discovery manager..." 2021-03-10T13:48:24.703 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.401Z caller=main.go:688 msg="Stopping scrape manager..." 2021-03-10T13:48:24.703 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.401Z caller=manager.go:924 component="rule manager" msg="Stopping rule manager..." 2021-03-10T13:48:24.704 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=manager.go:934 component="rule manager" msg="Rule manager stopped" 2021-03-10T13:48:24.704 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.401Z caller=main.go:648 msg="Scrape discovery manager stopped" 2021-03-10T13:48:24.704 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=main.go:662 msg="Notify discovery manager stopped" 2021-03-10T13:48:24.704 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=main.go:682 msg="Scrape manager stopped" 2021-03-10T13:48:24.704 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=notifier.go:601 component=notifier msg="Stopping notification manager..." 2021-03-10T13:48:24.705 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=main.go:856 msg="Notifier manager stopped" 2021-03-10T13:48:24.705 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[17912]: level=info ts=2021-03-10T13:48:24.402Z caller=main.go:868 msg="See you next time!" 2021-03-10T13:48:24.999 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2021-03-10T13:48:25.257 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:24 gibba032 bash[18790]: t=2021-03-10T13:48:24+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2021-03-10T13:48:25.257 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:25 gibba032 bash[18790]: t=2021-03-10T13:48:25+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2021-03-10T13:48:25.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:25 gibba032 bash[18790]: t=2021-03-10T13:48:25+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2021-03-10T13:48:25.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:25 gibba032 bash[18790]: t=2021-03-10T13:48:25+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2021-03-10T13:48:25.758 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 docker[19363]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-prometheus.a 2021-03-10T13:48:25.758 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@prometheus.a.service: Succeeded. 2021-03-10T13:48:25.758 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 systemd[1]: Stopped Ceph prometheus.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:48:25.759 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 systemd[1]: Starting Ceph prometheus.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:48:25.759 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 docker[19400]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-prometheus.a 2021-03-10T13:48:25.759 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:25 gibba032 systemd[1]: Started Ceph prometheus.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:48:26.154 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:25 gibba032 bash[18790]: t=2021-03-10T13:48:25+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2021-03-10T13:48:26.154 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:25 gibba032 bash[18790]: t=2021-03-10T13:48:25+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2021-03-10T13:48:26.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2021-03-10T13:48:26.507 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2021-03-10T13:48:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: cluster 2021-03-10T13:48:25.141808+0000 mgr.y (mgr.14140) 201 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:25.400589+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:25.409305+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: cephadm 2021-03-10T13:48:25.409780+0000 mgr.y (mgr.14140) 202 : cephadm [INF] Setting dashboard grafana config to https://gibba032:3000 2021-03-10T13:48:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:25.410039+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]: dispatch 2021-03-10T13:48:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:25.413334+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]': finished 2021-03-10T13:48:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:25.414698+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:26.060634+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:26.064077+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[9829]: audit 2021-03-10T13:48:26.064544+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: cluster 2021-03-10T13:48:25.141808+0000 mgr.y (mgr.14140) 201 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: cluster 2021-03-10T13:48:25.141808+0000 mgr.y (mgr.14140) 201 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:25.400589+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:25.400589+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:26.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:25.409305+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:26.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: cephadm 2021-03-10T13:48:25.409780+0000 mgr.y (mgr.14140) 202 : cephadm [INF] Setting dashboard grafana config to https://gibba032:3000 2021-03-10T13:48:26.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:25.410039+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]: dispatch 2021-03-10T13:48:26.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:25.413334+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]': finished 2021-03-10T13:48:26.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:25.414698+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:25.409305+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418146\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418406\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418573\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418728\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.418880\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419028\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419178\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"status\": 1, \"status_desc\": \"starting\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419521\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:11.419667\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: cephadm 2021-03-10T13:48:25.409780+0000 mgr.y (mgr.14140) 202 : cephadm [INF] Setting dashboard grafana config to https://gibba032:3000 2021-03-10T13:48:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:25.410039+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]: dispatch 2021-03-10T13:48:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:25.413334+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config set","value":"https://gibba032:3000","who":"mgr","name":"mgr/dashboard/GRAFANA_API_URL"}]': finished 2021-03-10T13:48:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:25.414698+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:26.060634+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:26.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:26.064077+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:26.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[9813]: audit 2021-03-10T13:48:26.064544+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:26.060634+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:26.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:26.064077+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059663\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059723\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059753\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059779\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059806\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059833\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059859\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059885\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:26.059911\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:26.059973\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:26.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:26 gibba011 bash[13015]: audit 2021-03-10T13:48:26.064544+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:26.686 INFO:teuthology.orchestra.run.gibba011.stdout:68719476776 2021-03-10T13:48:26.687 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2021-03-10T13:48:26.778 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2021-03-10T13:48:26.779 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2021-03-10T13:48:26.779 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2021-03-10T13:48:27.064 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2021-03-10T13:48:27.064 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2021-03-10T13:48:27.064 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2021-03-10T13:48:27.064 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[18790]: t=2021-03-10T13:48:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2021-03-10T13:48:27.065 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2021-03-10T13:48:27.065 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:366 msg="No time or size retention was set so using the default time retention" duration=15d 2021-03-10T13:48:27.065 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:404 msg="Starting Prometheus" version="(version=2.25.0, branch=HEAD, revision=a6be548dbc17780d562a39c0e4bd0bd4c00ad6e2)" 2021-03-10T13:48:27.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:409 build_context="(go=go1.15.8, user=root@615f028225c9, date=20210217-14:17:24)" 2021-03-10T13:48:27.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:410 host_details="(Linux 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021 x86_64 gibba032 (none))" 2021-03-10T13:48:27.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:411 fd_limits="(soft=1048576, hard=1048576)" 2021-03-10T13:48:27.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.875Z caller=main.go:412 vm_limits="(soft=unlimited, hard=unlimited)" 2021-03-10T13:48:27.066 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.876Z caller=web.go:532 component=web msg="Start listening for connections" address=:9095 2021-03-10T13:48:27.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.876Z caller=main.go:779 msg="Starting TSDB ..." 2021-03-10T13:48:27.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.877Z caller=tls_config.go:191 component=web msg="TLS is disabled." http2=false 2021-03-10T13:48:27.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.878Z caller=head.go:668 component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2021-03-10T13:48:27.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.878Z caller=head.go:682 component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.066µs 2021-03-10T13:48:27.067 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.878Z caller=head.go:688 component=tsdb msg="Replaying WAL, this may take a while" 2021-03-10T13:48:27.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.879Z caller=head.go:740 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2021-03-10T13:48:27.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.879Z caller=head.go:740 component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2021-03-10T13:48:27.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.879Z caller=head.go:745 component=tsdb msg="WAL replay completed" checkpoint_replay_duration=30.758µs wal_replay_duration=632.638µs total_replay_duration=670.818µs 2021-03-10T13:48:27.068 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.880Z caller=main.go:799 fs_type=XFS_SUPER_MAGIC 2021-03-10T13:48:27.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.880Z caller=main.go:802 msg="TSDB started" 2021-03-10T13:48:27.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.880Z caller=main.go:928 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2021-03-10T13:48:27.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.880Z caller=main.go:959 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=501.14µs remote_storage=818ns web_handler=202ns query_engine=307ns scrape=181.032µs scrape_sd=62.949µs notify=28.151µs notify_sd=5.699µs rules=19.232µs 2021-03-10T13:48:27.069 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 13:48:26 gibba032 bash[19417]: level=info ts=2021-03-10T13:48:26.880Z caller=main.go:751 msg="Server is ready to receive web requests." 2021-03-10T13:48:27.366 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2021-03-10T13:48:27.367 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2021-03-10T13:48:27.367 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2021-03-10T13:48:27.652 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2021-03-10T13:48:27.653 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2021-03-10T13:48:27.653 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2021-03-10T13:48:27.653 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2021-03-10T13:48:27.653 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2021-03-10T13:48:27.654 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2021-03-10T13:48:27.920 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2021-03-10T13:48:27.921 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2021-03-10T13:48:27.921 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2021-03-10T13:48:28.179 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2021-03-10T13:48:28.180 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:27 gibba032 bash[18790]: t=2021-03-10T13:48:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2021-03-10T13:48:28.180 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2021-03-10T13:48:28.180 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2021-03-10T13:48:28.181 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[9829]: cluster 2021-03-10T13:48:27.142421+0000 mgr.y (mgr.14140) 203 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:28.181 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[9829]: audit 2021-03-10T13:48:27.355227+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:28.181 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[9829]: audit 2021-03-10T13:48:27.364199+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:28.181 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[9829]: audit 2021-03-10T13:48:27.365819+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[9813]: cluster 2021-03-10T13:48:27.142421+0000 mgr.y (mgr.14140) 203 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[13015]: cluster 2021-03-10T13:48:27.142421+0000 mgr.y (mgr.14140) 203 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[9813]: audit 2021-03-10T13:48:27.355227+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[13015]: audit 2021-03-10T13:48:27.355227+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[9813]: audit 2021-03-10T13:48:27.364199+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[9813]: audit 2021-03-10T13:48:27.365819+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[13015]: audit 2021-03-10T13:48:27.364199+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350311\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350570\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350733\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.350885\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351033\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351175\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351321\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351504\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351647\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:27.351789\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:27.352108\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:28.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:28 gibba011 bash[13015]: audit 2021-03-10T13:48:27.365819+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:28.472 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2021-03-10T13:48:28.472 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2021-03-10T13:48:28.473 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2021-03-10T13:48:28.473 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2021-03-10T13:48:28.756 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2021-03-10T13:48:28.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2021-03-10T13:48:28.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2021-03-10T13:48:28.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2021-03-10T13:48:28.757 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2021-03-10T13:48:29.060 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2021-03-10T13:48:29.060 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2021-03-10T13:48:29.061 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2021-03-10T13:48:29.061 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:28 gibba032 bash[18790]: t=2021-03-10T13:48:28+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2021-03-10T13:48:29.061 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2021-03-10T13:48:29.061 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2021-03-10T13:48:29.312 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2021-03-10T13:48:29.312 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2021-03-10T13:48:29.313 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2021-03-10T13:48:29.313 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[9829]: audit 2021-03-10T13:48:28.300465+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:29.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[9829]: audit 2021-03-10T13:48:28.304238+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:29.314 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[9829]: audit 2021-03-10T13:48:28.304732+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:29.436 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[9813]: audit 2021-03-10T13:48:28.300465+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:29.436 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[13015]: audit 2021-03-10T13:48:28.300465+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:48:29.437 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[9813]: audit 2021-03-10T13:48:28.304238+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:29.437 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[9813]: audit 2021-03-10T13:48:28.304732+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:29.438 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[13015]: audit 2021-03-10T13:48:28.304238+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}}, \"last_daemon_update\": \"2021-03-10T13:48:28.299826\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:48:29.438 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:29 gibba011 bash[13015]: audit 2021-03-10T13:48:28.304732+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:48:29.606 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2021-03-10T13:48:29.606 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2021-03-10T13:48:29.606 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2021-03-10T13:48:29.607 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2021-03-10T13:48:29.607 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2021-03-10T13:48:29.607 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2021-03-10T13:48:29.899 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2021-03-10T13:48:29.900 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2021-03-10T13:48:29.900 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2021-03-10T13:48:30.022 INFO:teuthology.orchestra.run.gibba011.stdout:90194313252 2021-03-10T13:48:30.022 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2021-03-10T13:48:30.257 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2021-03-10T13:48:30.258 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:29 gibba032 bash[18790]: t=2021-03-10T13:48:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2021-03-10T13:48:30.258 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2021-03-10T13:48:30.258 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2021-03-10T13:48:30.314 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[21563]: level=info ts=2021-03-10T13:48:30.227Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.000821221s 2021-03-10T13:48:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[9813]: cluster 2021-03-10T13:48:29.143127+0000 mgr.y (mgr.14140) 204 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[13015]: cluster 2021-03-10T13:48:29.143127+0000 mgr.y (mgr.14140) 204 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[9813]: audit 2021-03-10T13:48:29.351203+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[9813]: audit 2021-03-10T13:48:29.353829+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[13015]: audit 2021-03-10T13:48:29.351203+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:30 gibba011 bash[13015]: audit 2021-03-10T13:48:29.353829+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[9829]: cluster 2021-03-10T13:48:29.143127+0000 mgr.y (mgr.14140) 204 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:30.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Created default admin" logger=sqlstore user=admin 2021-03-10T13:48:30.758 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing HTTPServer" logger=server 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing BackendPluginManager" logger=server 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing PluginManager" logger=server 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Starting plugin search" logger=plugins 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Registering plugin" logger=plugins name="Pie Chart" 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Registering plugin" logger=plugins name="Status Panel" 2021-03-10T13:48:30.759 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing HooksService" logger=server 2021-03-10T13:48:30.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing OSSLicensingService" logger=server 2021-03-10T13:48:30.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing InternalMetricsService" logger=server 2021-03-10T13:48:30.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing RemoteCache" logger=server 2021-03-10T13:48:30.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing QuotaService" logger=server 2021-03-10T13:48:30.760 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing ServerLockService" logger=server 2021-03-10T13:48:30.761 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing UserAuthTokenService" logger=server 2021-03-10T13:48:30.761 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing RenderingService" logger=server 2021-03-10T13:48:30.761 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing AlertEngine" logger=server 2021-03-10T13:48:30.761 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing DatasourceCacheService" logger=server 2021-03-10T13:48:30.761 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing LoginService" logger=server 2021-03-10T13:48:30.762 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing SearchService" logger=server 2021-03-10T13:48:30.762 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing TracingService" logger=server 2021-03-10T13:48:30.762 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing UsageStatsService" logger=server 2021-03-10T13:48:30.762 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing CleanUpService" logger=server 2021-03-10T13:48:30.762 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing NotificationService" logger=server 2021-03-10T13:48:30.763 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing provisioningServiceImpl" logger=server 2021-03-10T13:48:30.763 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2021-03-10T13:48:30.763 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name="Dashboard 1" 2021-03-10T13:48:30.763 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Backend rendering via phantomJS" logger=rendering renderer=phantomJS 2021-03-10T13:48:30.763 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=warn msg="phantomJS is deprecated and will be removed in a future release. You should consider migrating from phantomJS to grafana-image-renderer plugin." logger=rendering renderer=phantomJS 2021-03-10T13:48:30.764 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="Initializing Stream Manager" 2021-03-10T13:48:30.764 INFO:journalctl@ceph.grafana.a.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[18790]: t=2021-03-10T13:48:30+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=127.0.0.1:3000 protocol=https subUrl= socket= 2021-03-10T13:48:30.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[9829]: audit 2021-03-10T13:48:29.351203+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:48:30.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:30 gibba032 bash[9829]: audit 2021-03-10T13:48:29.353829+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"c0962a4e2967\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350229\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:44:28.158112\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:25.725968\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:48:29.350501\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:48:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:32 gibba011 bash[9813]: cluster 2021-03-10T13:48:31.143718+0000 mgr.y (mgr.14140) 205 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:32 gibba011 bash[13015]: cluster 2021-03-10T13:48:31.143718+0000 mgr.y (mgr.14140) 205 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:32 gibba032 bash[9829]: cluster 2021-03-10T13:48:31.143718+0000 mgr.y (mgr.14140) 205 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:33.281 INFO:teuthology.orchestra.run.gibba011.stdout:111669149727 2021-03-10T13:48:33.282 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2021-03-10T13:48:34.485 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:34 gibba011 bash[9813]: cluster 2021-03-10T13:48:33.144480+0000 mgr.y (mgr.14140) 206 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:34.485 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:34 gibba011 bash[13015]: cluster 2021-03-10T13:48:33.144480+0000 mgr.y (mgr.14140) 206 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:34 gibba032 bash[9829]: cluster 2021-03-10T13:48:33.144480+0000 mgr.y (mgr.14140) 206 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:36 gibba011 bash[9813]: cluster 2021-03-10T13:48:35.145247+0000 mgr.y (mgr.14140) 207 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:36.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:36 gibba011 bash[13015]: cluster 2021-03-10T13:48:35.145247+0000 mgr.y (mgr.14140) 207 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:36 gibba032 bash[9829]: cluster 2021-03-10T13:48:35.145247+0000 mgr.y (mgr.14140) 207 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:36.572 INFO:teuthology.orchestra.run.gibba011.stdout:133143986204 2021-03-10T13:48:36.572 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2021-03-10T13:48:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:38 gibba032 bash[9829]: cluster 2021-03-10T13:48:37.146146+0000 mgr.y (mgr.14140) 208 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:38 gibba011 bash[9813]: cluster 2021-03-10T13:48:37.146146+0000 mgr.y (mgr.14140) 208 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:38 gibba011 bash[13015]: cluster 2021-03-10T13:48:37.146146+0000 mgr.y (mgr.14140) 208 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:39.879 INFO:teuthology.orchestra.run.gibba011.stdout:154618822681 2021-03-10T13:48:39.880 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2021-03-10T13:48:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:40 gibba011 bash[9813]: cluster 2021-03-10T13:48:39.146882+0000 mgr.y (mgr.14140) 209 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:40 gibba011 bash[13015]: cluster 2021-03-10T13:48:39.146882+0000 mgr.y (mgr.14140) 209 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:40 gibba032 bash[9829]: cluster 2021-03-10T13:48:39.146882+0000 mgr.y (mgr.14140) 209 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:42 gibba032 bash[9829]: cluster 2021-03-10T13:48:41.147842+0000 mgr.y (mgr.14140) 210 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:42 gibba011 bash[9813]: cluster 2021-03-10T13:48:41.147842+0000 mgr.y (mgr.14140) 210 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:42 gibba011 bash[13015]: cluster 2021-03-10T13:48:41.147842+0000 mgr.y (mgr.14140) 210 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:43.182 INFO:teuthology.orchestra.run.gibba011.stdout:176093659157 2021-03-10T13:48:43.183 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2021-03-10T13:48:44.360 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:44 gibba011 bash[9813]: cluster 2021-03-10T13:48:43.148592+0000 mgr.y (mgr.14140) 211 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:44.361 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:44 gibba011 bash[13015]: cluster 2021-03-10T13:48:43.148592+0000 mgr.y (mgr.14140) 211 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:44 gibba032 bash[9829]: cluster 2021-03-10T13:48:43.148592+0000 mgr.y (mgr.14140) 211 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:46 gibba011 bash[9813]: cluster 2021-03-10T13:48:45.149243+0000 mgr.y (mgr.14140) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:46.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:46 gibba011 bash[13015]: cluster 2021-03-10T13:48:45.149243+0000 mgr.y (mgr.14140) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:46.388 INFO:teuthology.orchestra.run.gibba011.stdout:197568495631 2021-03-10T13:48:46.389 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.0 2021-03-10T13:48:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:46 gibba032 bash[9829]: cluster 2021-03-10T13:48:45.149243+0000 mgr.y (mgr.14140) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:48 gibba011 bash[9813]: cluster 2021-03-10T13:48:47.149598+0000 mgr.y (mgr.14140) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:48.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:48 gibba011 bash[13015]: cluster 2021-03-10T13:48:47.149598+0000 mgr.y (mgr.14140) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:48 gibba032 bash[9829]: cluster 2021-03-10T13:48:47.149598+0000 mgr.y (mgr.14140) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:49.069 INFO:teuthology.orchestra.run.gibba011.stdout:42949673008 2021-03-10T13:48:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:49 gibba032 bash[9829]: audit 2021-03-10T13:48:49.068347+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/2784937770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2021-03-10T13:48:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:49 gibba011 bash[9813]: audit 2021-03-10T13:48:49.068347+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/2784937770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2021-03-10T13:48:49.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:49 gibba011 bash[13015]: audit 2021-03-10T13:48:49.068347+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/2784937770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2021-03-10T13:48:49.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42949673004 got 42949673008 for osd.0 2021-03-10T13:48:49.710 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.1 2021-03-10T13:48:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:50 gibba032 bash[9829]: cluster 2021-03-10T13:48:49.150226+0000 mgr.y (mgr.14140) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:50 gibba011 bash[9813]: cluster 2021-03-10T13:48:49.150226+0000 mgr.y (mgr.14140) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:50 gibba011 bash[13015]: cluster 2021-03-10T13:48:49.150226+0000 mgr.y (mgr.14140) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:52.395 INFO:teuthology.orchestra.run.gibba011.stdout:68719476781 2021-03-10T13:48:52.446 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:52 gibba011 bash[9813]: cluster 2021-03-10T13:48:51.151093+0000 mgr.y (mgr.14140) 215 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:52.447 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:52 gibba011 bash[13015]: cluster 2021-03-10T13:48:51.151093+0000 mgr.y (mgr.14140) 215 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:52 gibba032 bash[9829]: cluster 2021-03-10T13:48:51.151093+0000 mgr.y (mgr.14140) 215 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:53.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476776 got 68719476781 for osd.1 2021-03-10T13:48:53.087 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.2 2021-03-10T13:48:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:53 gibba032 bash[9829]: audit 2021-03-10T13:48:52.394016+0000 mon.a (mon.0) 525 : audit [DBG] from='client.? 172.21.2.111:0/2458998019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2021-03-10T13:48:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:53 gibba011 bash[9813]: audit 2021-03-10T13:48:52.394016+0000 mon.a (mon.0) 525 : audit [DBG] from='client.? 172.21.2.111:0/2458998019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2021-03-10T13:48:53.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:53 gibba011 bash[13015]: audit 2021-03-10T13:48:52.394016+0000 mon.a (mon.0) 525 : audit [DBG] from='client.? 172.21.2.111:0/2458998019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2021-03-10T13:48:54.374 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:54 gibba011 bash[9813]: cluster 2021-03-10T13:48:53.151515+0000 mgr.y (mgr.14140) 216 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:54.374 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:54 gibba011 bash[13015]: cluster 2021-03-10T13:48:53.151515+0000 mgr.y (mgr.14140) 216 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:54 gibba032 bash[9829]: cluster 2021-03-10T13:48:53.151515+0000 mgr.y (mgr.14140) 216 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:55.925 INFO:teuthology.orchestra.run.gibba011.stdout:90194313257 2021-03-10T13:48:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:56 gibba032 bash[9829]: cluster 2021-03-10T13:48:55.152120+0000 mgr.y (mgr.14140) 217 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:56 gibba032 bash[9829]: audit 2021-03-10T13:48:55.924241+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/4143507934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2021-03-10T13:48:56.555 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:56 gibba011 bash[9813]: cluster 2021-03-10T13:48:55.152120+0000 mgr.y (mgr.14140) 217 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:56.555 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:56 gibba011 bash[9813]: audit 2021-03-10T13:48:55.924241+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/4143507934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2021-03-10T13:48:56.555 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:56 gibba011 bash[13015]: cluster 2021-03-10T13:48:55.152120+0000 mgr.y (mgr.14140) 217 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:56.556 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:56 gibba011 bash[13015]: audit 2021-03-10T13:48:55.924241+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/4143507934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2021-03-10T13:48:56.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313252 got 90194313257 for osd.2 2021-03-10T13:48:56.557 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.3 2021-03-10T13:48:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:48:58 gibba032 bash[9829]: cluster 2021-03-10T13:48:57.153056+0000 mgr.y (mgr.14140) 218 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:48:58 gibba011 bash[9813]: cluster 2021-03-10T13:48:57.153056+0000 mgr.y (mgr.14140) 218 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:48:58 gibba011 bash[13015]: cluster 2021-03-10T13:48:57.153056+0000 mgr.y (mgr.14140) 218 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:48:59.263 INFO:teuthology.orchestra.run.gibba011.stdout:111669149732 2021-03-10T13:48:59.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149727 got 111669149732 for osd.3 2021-03-10T13:48:59.926 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.4 2021-03-10T13:49:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:00 gibba011 bash[9813]: cluster 2021-03-10T13:48:59.153381+0000 mgr.y (mgr.14140) 219 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:00 gibba011 bash[9813]: audit 2021-03-10T13:48:59.262200+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/3547773104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2021-03-10T13:49:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:00 gibba011 bash[13015]: cluster 2021-03-10T13:48:59.153381+0000 mgr.y (mgr.14140) 219 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:00 gibba011 bash[13015]: audit 2021-03-10T13:48:59.262200+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/3547773104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2021-03-10T13:49:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:00 gibba032 bash[9829]: cluster 2021-03-10T13:48:59.153381+0000 mgr.y (mgr.14140) 219 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:00 gibba032 bash[9829]: audit 2021-03-10T13:48:59.262200+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/3547773104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2021-03-10T13:49:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:02 gibba011 bash[9813]: cluster 2021-03-10T13:49:01.153969+0000 mgr.y (mgr.14140) 220 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:02 gibba011 bash[13015]: cluster 2021-03-10T13:49:01.153969+0000 mgr.y (mgr.14140) 220 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:02.656 INFO:teuthology.orchestra.run.gibba011.stdout:133143986209 2021-03-10T13:49:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:02 gibba032 bash[9829]: cluster 2021-03-10T13:49:01.153969+0000 mgr.y (mgr.14140) 220 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:03.312 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986204 got 133143986209 for osd.4 2021-03-10T13:49:03.313 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.5 2021-03-10T13:49:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:03 gibba011 bash[9813]: audit 2021-03-10T13:49:02.655171+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/3761541043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2021-03-10T13:49:03.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:03 gibba011 bash[13015]: audit 2021-03-10T13:49:02.655171+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/3761541043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2021-03-10T13:49:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:03 gibba032 bash[9829]: audit 2021-03-10T13:49:02.655171+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/3761541043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2021-03-10T13:49:04.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:04 gibba011 bash[9813]: cluster 2021-03-10T13:49:03.154687+0000 mgr.y (mgr.14140) 221 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:04.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:04 gibba011 bash[13015]: cluster 2021-03-10T13:49:03.154687+0000 mgr.y (mgr.14140) 221 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:04 gibba032 bash[9829]: cluster 2021-03-10T13:49:03.154687+0000 mgr.y (mgr.14140) 221 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:06.140 INFO:teuthology.orchestra.run.gibba011.stdout:154618822686 2021-03-10T13:49:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:06 gibba011 bash[9813]: cluster 2021-03-10T13:49:05.155303+0000 mgr.y (mgr.14140) 222 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:06 gibba011 bash[9813]: audit 2021-03-10T13:49:06.138772+0000 mon.a (mon.0) 526 : audit [DBG] from='client.? 172.21.2.111:0/1441267221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2021-03-10T13:49:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:06 gibba011 bash[13015]: cluster 2021-03-10T13:49:05.155303+0000 mgr.y (mgr.14140) 222 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:06 gibba011 bash[13015]: audit 2021-03-10T13:49:06.138772+0000 mon.a (mon.0) 526 : audit [DBG] from='client.? 172.21.2.111:0/1441267221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2021-03-10T13:49:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:06 gibba032 bash[9829]: cluster 2021-03-10T13:49:05.155303+0000 mgr.y (mgr.14140) 222 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:06 gibba032 bash[9829]: audit 2021-03-10T13:49:06.138772+0000 mon.a (mon.0) 526 : audit [DBG] from='client.? 172.21.2.111:0/1441267221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2021-03-10T13:49:06.799 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822681 got 154618822686 for osd.5 2021-03-10T13:49:06.800 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.6 2021-03-10T13:49:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:08 gibba011 bash[9813]: cluster 2021-03-10T13:49:07.156091+0000 mgr.y (mgr.14140) 223 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:08 gibba011 bash[13015]: cluster 2021-03-10T13:49:07.156091+0000 mgr.y (mgr.14140) 223 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:08 gibba032 bash[9829]: cluster 2021-03-10T13:49:07.156091+0000 mgr.y (mgr.14140) 223 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:09.549 INFO:teuthology.orchestra.run.gibba011.stdout:176093659163 2021-03-10T13:49:10.229 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659157 got 176093659163 for osd.6 2021-03-10T13:49:10.230 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph osd last-stat-seq osd.7 2021-03-10T13:49:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:10 gibba011 bash[9813]: cluster 2021-03-10T13:49:09.156839+0000 mgr.y (mgr.14140) 224 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:10 gibba011 bash[9813]: audit 2021-03-10T13:49:09.548927+0000 mon.a (mon.0) 527 : audit [DBG] from='client.? 172.21.2.111:0/2250257804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2021-03-10T13:49:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:10 gibba011 bash[13015]: cluster 2021-03-10T13:49:09.156839+0000 mgr.y (mgr.14140) 224 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:10 gibba011 bash[13015]: audit 2021-03-10T13:49:09.548927+0000 mon.a (mon.0) 527 : audit [DBG] from='client.? 172.21.2.111:0/2250257804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2021-03-10T13:49:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:10 gibba032 bash[9829]: cluster 2021-03-10T13:49:09.156839+0000 mgr.y (mgr.14140) 224 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:10 gibba032 bash[9829]: audit 2021-03-10T13:49:09.548927+0000 mon.a (mon.0) 527 : audit [DBG] from='client.? 172.21.2.111:0/2250257804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2021-03-10T13:49:12.681 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:12 gibba011 bash[9813]: cluster 2021-03-10T13:49:11.157689+0000 mgr.y (mgr.14140) 225 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:12.681 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:12 gibba011 bash[13015]: cluster 2021-03-10T13:49:11.157689+0000 mgr.y (mgr.14140) 225 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:12 gibba032 bash[9829]: cluster 2021-03-10T13:49:11.157689+0000 mgr.y (mgr.14140) 225 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:12.938 INFO:teuthology.orchestra.run.gibba011.stdout:197568495636 2021-03-10T13:49:13.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:13 gibba011 bash[9813]: audit 2021-03-10T13:49:12.936926+0000 mon.a (mon.0) 528 : audit [DBG] from='client.? 172.21.2.111:0/2993933000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2021-03-10T13:49:13.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:13 gibba011 bash[13015]: audit 2021-03-10T13:49:12.936926+0000 mon.a (mon.0) 528 : audit [DBG] from='client.? 172.21.2.111:0/2993933000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2021-03-10T13:49:13.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495631 got 197568495636 for osd.7 2021-03-10T13:49:13.578 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2021-03-10T13:49:13.578 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph pg dump --format=json 2021-03-10T13:49:13.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:13 gibba032 bash[9829]: audit 2021-03-10T13:49:12.936926+0000 mon.a (mon.0) 528 : audit [DBG] from='client.? 172.21.2.111:0/2993933000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2021-03-10T13:49:14.655 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:14 gibba011 bash[9813]: cluster 2021-03-10T13:49:13.158337+0000 mgr.y (mgr.14140) 226 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:14.655 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:14 gibba011 bash[13015]: cluster 2021-03-10T13:49:13.158337+0000 mgr.y (mgr.14140) 226 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:14 gibba032 bash[9829]: cluster 2021-03-10T13:49:13.158337+0000 mgr.y (mgr.14140) 226 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:16.271 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:49:16.271 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-03-10T13:49:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:16 gibba032 bash[9829]: cluster 2021-03-10T13:49:15.159039+0000 mgr.y (mgr.14140) 227 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:16 gibba011 bash[9813]: cluster 2021-03-10T13:49:15.159039+0000 mgr.y (mgr.14140) 227 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:16 gibba011 bash[13015]: cluster 2021-03-10T13:49:15.159039+0000 mgr.y (mgr.14140) 227 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:16.919 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2021-03-10T13:49:15.158697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":704643072,"kb_used":8392220,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":696250852,"statfs":{"total":721554505728,"available":712960872448,"internally_reserved":8589934592,"allocated":3633152,"data_stored":1442378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004370"},"pg_stats":[{"pgid":"1.0","version":"17'1","reported_seq":"14","reported_epoch":"47","state":"active+clean","last_fresh":"2021-03-10T13:47:37.447063+0000","last_change":"2021-03-10T13:47:37.447063+0000","last_active":"2021-03-10T13:47:37.447063+0000","last_peered":"2021-03-10T13:47:37.447063+0000","last_clean":"2021-03-10T13:47:37.447063+0000","last_became_active":"2021-03-10T13:47:37.445429+0000","last_became_peered":"2021-03-10T13:47:37.445429+0000","last_unstale":"2021-03-10T13:47:37.447063+0000","last_undegraded":"2021-03-10T13:47:37.447063+0000","last_fullsized":"2021-03-10T13:47:37.447063+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-10T13:42:54.968183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-10T13:42:54.968183+0000","last_clean_scrub_stamp":"2021-03-10T13:42:54.968183+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":46,"seq":197568495637,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.879},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.048}]},{"osd":1,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.89800000000000002}]},{"osd":2,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.548,"15min":1.548},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.131}]},{"osd":3,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.218},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":1.004}]},{"osd":4,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.749},{"interface":"front","average":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.81000000000000005}]}]},{"osd":6,"up_from":41,"seq":176093659164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049024,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031360,"statfs":{"total":90194313216,"available":89120112640,"internally_reserved":1073741824,"allocated":450560,"data_stored":180040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.728},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.2130000000000001}]},{"osd":1,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.345},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":2.1880000000000002}]},{"osd":2,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":2.5190000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.901}]},{"osd":3,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":2.335}]},{"osd":4,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.0409999999999999}]},{"osd":5,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.4870000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:40 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":2.0329999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.591}]}]},{"osd":1,"up_from":16,"seq":68719476786,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:21 2021","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72999999999999998,"15min":0.71099999999999997},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":1.1559999999999999,"5min":1.306,"15min":1.306},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0329999999999999,"15min":0.998},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":2.032,"5min":2.153,"15min":2.153},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":0.97899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.748,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.97999999999999998,"15min":1.004},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":1.1719999999999999,"5min":1.728,"15min":1.728},"last":1.022}]},{"osd":3,"last update":"Wed Mar 10 13:49:12 2021","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.2030000000000001,"15min":1.2809999999999999},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.659,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.173,"15min":1.2649999999999999},"min":{"1min":0.25700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.421}]},{"osd":4,"last update":"Wed Mar 10 13:48:29 2021","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4299999999999999,"15min":1.3939999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":2.032},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.3169999999999999,"15min":1.2649999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.996}]},{"osd":5,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.254,"15min":1.2310000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.302},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6240000000000001,"15min":1.6120000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.9219999999999999}]},{"osd":6,"last update":"Wed Mar 10 13:48:14 2021","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":2.081},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.6850000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:39 2021","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":2.181},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.8520000000000001}]}]},{"osd":0,"up_from":10,"seq":42949673014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 10 13:48:27 2021","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.81699999999999995,"15min":0.71899999999999997},"min":{"1min":0.20000000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.323},{"interface":"front","average":{"1min":1.2010000000000001,"5min":0.94299999999999995,"15min":0.82799999999999996},"min":{"1min":0.308,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.242}]},{"osd":2,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.88700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.9079999999999999,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":1.325,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.83399999999999996}]},{"osd":3,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.20799999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.5940000000000001,"5min":1.677,"15min":1.677},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.079,"15min":1.103},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.8080000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0629999999999999}]},{"osd":4,"last update":"Wed Mar 10 13:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3740000000000001,"15min":1.411},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.968,"5min":2.008,"15min":2.008},"last":1.502},{"interface":"front","average":{"1min":1.325,"5min":1.474,"15min":1.4990000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.4570000000000001}]},{"osd":5,"last update":"Wed Mar 10 13:48:53 2021","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.5309999999999999,"15min":1.5669999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.786},{"interface":"front","average":{"1min":1.391,"5min":1.5649999999999999,"15min":1.5940000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.988}]},{"osd":6,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.865},{"interface":"front","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.595}]},{"osd":7,"last update":"Wed Mar 10 13:48:36 2021","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.708}]}]},{"osd":2,"up_from":21,"seq":90194313261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.79600000000000004},"min":{"1min":0.155,"5min":0.124,"15min":0.124},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.219,"5min":0.158,"15min":0.158},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.76400000000000001}]},{"osd":1,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.96099999999999997,"15min":0.96999999999999997},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":1.5620000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.008},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0640000000000001,"15min":1.056},"min":{"1min":0.186,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.95199999999999996}]},{"osd":3,"last update":"Wed Mar 10 13:49:09 2021","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.95699999999999996,"15min":0.99299999999999999},"min":{"1min":0.27700000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.5329999999999999,"5min":1.984,"15min":1.984},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.232,"15min":1.256},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.0910000000000002,"5min":2.238,"15min":2.238},"last":1.1839999999999999}]},{"osd":4,"last update":"Wed Mar 10 13:48:27 2021","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3999999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.653},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5529999999999999,"15min":1.5600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Mar 10 13:48:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.506,"15min":1.524},"min":{"1min":0.42699999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.2679999999999998,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.536,"5min":1.4199999999999999,"15min":1.3999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.5269999999999999}]},{"osd":6,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.389},{"interface":"front","average":{"1min":1.645,"5min":1.645,"15min":1.645},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.9810000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":2.1579999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92600000000000005,"15min":0.99399999999999999},"min":{"1min":0.19,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.8100000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.98499999999999999,"15min":1},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":1.5700000000000001,"5min":1.742,"15min":1.742},"last":0.77900000000000003}]},{"osd":1,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.115,"15min":1.1599999999999999},"min":{"1min":0.21199999999999999,"5min":0.189,"15min":0.189},"max":{"1min":1.641,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.135},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.02,"15min":1.034},"min":{"1min":0.23300000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.605,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.425}]},{"osd":2,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.177,"15min":1.2330000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98999999999999999,"15min":0.99299999999999999},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.333,"5min":1.768,"15min":1.768},"last":1.2509999999999999}]},{"osd":4,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.419},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":2.2719999999999998}]},{"osd":5,"last update":"Wed Mar 10 13:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.5309999999999999,"15min":1.5149999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.681},{"interface":"front","average":{"1min":1.5,"5min":1.4710000000000001,"15min":1.466},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1760000000000002,"5min":2.415,"15min":2.415},"last":1.607}]},{"osd":6,"last update":"Wed Mar 10 13:48:14 2021","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":2.101},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":2.016}]},{"osd":7,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.9630000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":2.3740000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.232,"15min":1.234},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.8680000000000001,"5min":2.254,"15min":2.254},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.462,"5min":1.4930000000000001,"15min":1.498},"min":{"1min":0.41899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.266,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.1799999999999999}]},{"osd":1,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.2989999999999999,"15min":1.2869999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.827},{"interface":"front","average":{"1min":1.248,"5min":1.3440000000000001,"15min":1.3600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.96399999999999997}]},{"osd":2,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.51,"15min":1.5209999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.595}]},{"osd":3,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4179999999999999,"15min":1.4199999999999999},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":2.0779999999999998},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4279999999999999,"15min":1.421},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":2.0110000000000001}]},{"osd":5,"last update":"Wed Mar 10 13:48:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.0309999999999999,"15min":1.008},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1619999999999999,"15min":1.165},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.9790000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.4870000000000001}]},{"osd":6,"last update":"Wed Mar 10 13:48:12 2021","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.746},{"interface":"front","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.304}]},{"osd":7,"last update":"Wed Mar 10 13:48:36 2021","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.671},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.911}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4750000000000001,"15min":1.486},"min":{"1min":0.56499999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.2470000000000001,"15min":1.2310000000000001},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.0349999999999999}]},{"osd":1,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.49,"15min":1.484},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.4750000000000001,"5min":2.637,"15min":2.637},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.363,"15min":1.333},"min":{"1min":0.81799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.4950000000000001}]},{"osd":2,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.385,"15min":1.395},"min":{"1min":0.77500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.21,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4630000000000001,"15min":1.458},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.627}]},{"osd":3,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.446,"15min":1.4430000000000001},"min":{"1min":0.82899999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.2370000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.589,"5min":1.51,"15min":1.4970000000000001},"min":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.8180000000000001}]},{"osd":4,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.88400000000000001,"15min":0.84499999999999997},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.0169999999999999,"15min":0.95499999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.7330000000000001}]},{"osd":6,"last update":"Wed Mar 10 13:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.1539999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.2799999999999998}]},{"osd":7,"last update":"Wed Mar 10 13:48:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.4420000000000002},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":2.5670000000000002}]}]}]}} 2021-03-10T13:49:16.920 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph pg dump --format=json 2021-03-10T13:49:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:17 gibba032 bash[9829]: audit 2021-03-10T13:49:16.267715+0000 mgr.y (mgr.14140) 228 : audit [DBG] from='client.14439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:17.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:17 gibba011 bash[9813]: audit 2021-03-10T13:49:16.267715+0000 mgr.y (mgr.14140) 228 : audit [DBG] from='client.14439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:17.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:17 gibba011 bash[13015]: audit 2021-03-10T13:49:16.267715+0000 mgr.y (mgr.14140) 228 : audit [DBG] from='client.14439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:18 gibba011 bash[9813]: cluster 2021-03-10T13:49:17.159829+0000 mgr.y (mgr.14140) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:18 gibba011 bash[13015]: cluster 2021-03-10T13:49:17.159829+0000 mgr.y (mgr.14140) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:18 gibba032 bash[9829]: cluster 2021-03-10T13:49:17.159829+0000 mgr.y (mgr.14140) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:19.759 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:49:19.760 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-03-10T13:49:20.382 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2021-03-10T13:49:19.160160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":704643072,"kb_used":8392220,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":696250852,"statfs":{"total":721554505728,"available":712960872448,"internally_reserved":8589934592,"allocated":3633152,"data_stored":1442378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004441"},"pg_stats":[{"pgid":"1.0","version":"17'1","reported_seq":"14","reported_epoch":"47","state":"active+clean","last_fresh":"2021-03-10T13:47:37.447063+0000","last_change":"2021-03-10T13:47:37.447063+0000","last_active":"2021-03-10T13:47:37.447063+0000","last_peered":"2021-03-10T13:47:37.447063+0000","last_clean":"2021-03-10T13:47:37.447063+0000","last_became_active":"2021-03-10T13:47:37.445429+0000","last_became_peered":"2021-03-10T13:47:37.445429+0000","last_unstale":"2021-03-10T13:47:37.447063+0000","last_undegraded":"2021-03-10T13:47:37.447063+0000","last_fullsized":"2021-03-10T13:47:37.447063+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-10T13:42:54.968183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-10T13:42:54.968183+0000","last_clean_scrub_stamp":"2021-03-10T13:42:54.968183+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":46,"seq":197568495638,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.88},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.996}]},{"osd":1,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.403}]},{"osd":2,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.548,"15min":1.548},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.1100000000000001}]},{"osd":3,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.726},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":1.246}]},{"osd":4,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":1.202},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.3200000000000001}]},{"osd":5,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.8180000000000001}]},{"osd":6,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.78},{"interface":"front","average":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.8919999999999999}]}]},{"osd":6,"up_from":41,"seq":176093659165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049024,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031360,"statfs":{"total":90194313216,"available":89120112640,"internally_reserved":1073741824,"allocated":450560,"data_stored":180040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4490000000000001,"15min":1.4510000000000001},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":2.2839999999999998,"5min":2.456,"15min":2.456},"last":1.343},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.369,"15min":1.3480000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":1.5760000000000001}]},{"osd":1,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.552,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.502,"5min":1.4379999999999999,"15min":1.427},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.2010000000000001}]},{"osd":2,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.77,"5min":1.55,"15min":1.514},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.7810000000000001,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":2.161},{"interface":"front","average":{"1min":1.335,"5min":1.3779999999999999,"15min":1.385},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.86399999999999999}]},{"osd":3,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.165,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.077},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.627,"15min":1.617},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.7440000000000002,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.756}]},{"osd":4,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.0509999999999999,"15min":1.0369999999999999},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":1.7490000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.2110000000000001,"15min":1.1799999999999999},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":2.6789999999999998}]},{"osd":5,"last update":"Wed Mar 10 13:49:15 2021","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1299999999999999,"15min":1.1080000000000001},"min":{"1min":0.27200000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.897},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.446,"15min":1.4179999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":2.3119999999999998}]},{"osd":7,"last update":"Wed Mar 10 13:48:40 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":2.5129999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":2.0390000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476786,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:21 2021","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72999999999999998,"15min":0.71099999999999997},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":1.1559999999999999,"5min":1.306,"15min":1.306},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0329999999999999,"15min":0.998},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":2.032,"5min":2.153,"15min":2.153},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":0.97899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.748,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.97999999999999998,"15min":1.004},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":1.1719999999999999,"5min":1.728,"15min":1.728},"last":1.022}]},{"osd":3,"last update":"Wed Mar 10 13:49:12 2021","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.2030000000000001,"15min":1.2809999999999999},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.659,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.173,"15min":1.2649999999999999},"min":{"1min":0.25700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.421}]},{"osd":4,"last update":"Wed Mar 10 13:48:29 2021","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4299999999999999,"15min":1.3939999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":2.032},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.3169999999999999,"15min":1.2649999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.996}]},{"osd":5,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.254,"15min":1.2310000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.302},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6240000000000001,"15min":1.6120000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.9219999999999999}]},{"osd":6,"last update":"Wed Mar 10 13:48:14 2021","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":2.081},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.6850000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:39 2021","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":2.181},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.8520000000000001}]}]},{"osd":0,"up_from":10,"seq":42949673015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 10 13:48:27 2021","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.81699999999999995,"15min":0.71899999999999997},"min":{"1min":0.20000000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":0.94299999999999995,"15min":0.82799999999999996},"min":{"1min":0.308,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.2949999999999999}]},{"osd":2,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.88700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.9079999999999999,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":1.325,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.20799999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.5940000000000001,"5min":1.677,"15min":1.677},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.079,"15min":1.103},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.8080000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0049999999999999}]},{"osd":4,"last update":"Wed Mar 10 13:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3740000000000001,"15min":1.411},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.968,"5min":2.008,"15min":2.008},"last":2.1240000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.474,"15min":1.4990000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":2}]},{"osd":5,"last update":"Wed Mar 10 13:48:53 2021","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.5309999999999999,"15min":1.5669999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.5649999999999999,"15min":1.5940000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.722}]},{"osd":6,"last update":"Wed Mar 10 13:49:17 2021","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.7050000000000001,"15min":1.7210000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":2.4390000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.4339999999999999,"15min":1.448},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.637}]},{"osd":7,"last update":"Wed Mar 10 13:48:36 2021","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":2.2709999999999999}]}]},{"osd":2,"up_from":21,"seq":90194313262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.79600000000000004},"min":{"1min":0.155,"5min":0.124,"15min":0.124},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.219,"5min":0.158,"15min":0.158},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Mar 10 13:48:46 2021","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.96099999999999997,"15min":0.96999999999999997},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":1.5620000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.219},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0640000000000001,"15min":1.056},"min":{"1min":0.186,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.24399999999999999}]},{"osd":3,"last update":"Wed Mar 10 13:49:09 2021","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.95699999999999996,"15min":0.99299999999999999},"min":{"1min":0.27700000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.5329999999999999,"5min":1.984,"15min":1.984},"last":0.182},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.232,"15min":1.256},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.0910000000000002,"5min":2.238,"15min":2.238},"last":0.16600000000000001}]},{"osd":4,"last update":"Wed Mar 10 13:48:27 2021","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3999999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5529999999999999,"15min":1.5600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.33700000000000002}]},{"osd":5,"last update":"Wed Mar 10 13:48:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.506,"15min":1.524},"min":{"1min":0.42699999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.2679999999999998,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.536,"5min":1.4199999999999999,"15min":1.3999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.35299999999999998}]},{"osd":6,"last update":"Wed Mar 10 13:49:13 2021","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.478,"15min":1.454},"min":{"1min":0.40799999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.782,"5min":1.6719999999999999,"15min":1.6539999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.30599999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92600000000000005,"15min":0.99399999999999999},"min":{"1min":0.19,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.154},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.98499999999999999,"15min":1},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":1.5700000000000001,"5min":1.742,"15min":1.742},"last":0.215}]},{"osd":1,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.115,"15min":1.1599999999999999},"min":{"1min":0.21199999999999999,"5min":0.189,"15min":0.189},"max":{"1min":1.641,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":0.253},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.02,"15min":1.034},"min":{"1min":0.23300000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.605,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.17999999999999999}]},{"osd":2,"last update":"Wed Mar 10 13:49:11 2021","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.177,"15min":1.2330000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98999999999999999,"15min":0.99299999999999999},"min":{"1min":0.219,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.333,"5min":1.768,"15min":1.768},"last":0.24099999999999999}]},{"osd":4,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.419},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.752},{"interface":"front","average":{"1min":1.462,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Mar 10 13:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.5309999999999999,"15min":1.5149999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.5,"5min":1.4710000000000001,"15min":1.466},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1760000000000002,"5min":2.415,"15min":2.415},"last":0.85499999999999998}]},{"osd":6,"last update":"Wed Mar 10 13:49:14 2021","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.484,"15min":1.458},"min":{"1min":0.75900000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.7529999999999999,"5min":1.6359999999999999,"15min":1.6160000000000001},"min":{"1min":0.57699999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Mar 10 13:48:38 2021","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.72399999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.232,"15min":1.234},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.8680000000000001,"5min":2.254,"15min":2.254},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.4930000000000001,"15min":1.498},"min":{"1min":0.41899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.266,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.6779999999999999}]},{"osd":1,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.2989999999999999,"15min":1.2869999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.3440000000000001,"15min":1.3600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":2.0350000000000001}]},{"osd":2,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.51,"15min":1.5209999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":2.4300000000000002},{"interface":"front","average":{"1min":1.47,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.903}]},{"osd":3,"last update":"Wed Mar 10 13:48:30 2021","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4179999999999999,"15min":1.4199999999999999},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":2.8500000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4279999999999999,"15min":1.421},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":2.5659999999999998}]},{"osd":5,"last update":"Wed Mar 10 13:48:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.0309999999999999,"15min":1.008},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.363},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1619999999999999,"15min":1.165},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.9790000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.165}]},{"osd":6,"last update":"Wed Mar 10 13:49:13 2021","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.1859999999999999,"15min":1.1459999999999999},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.1930000000000001,"15min":1.1699999999999999},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":2.4870000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Wed Mar 10 13:48:36 2021","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":2.1549999999999998},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":2.2999999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88080384,"kb_used":1049028,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":87031356,"statfs":{"total":90194313216,"available":89120108544,"internally_reserved":1073741824,"allocated":454656,"data_stored":180334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4750000000000001,"15min":1.486},"min":{"1min":0.56499999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.2470000000000001,"15min":1.2310000000000001},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.0349999999999999}]},{"osd":1,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.49,"15min":1.484},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.4750000000000001,"5min":2.637,"15min":2.637},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.363,"15min":1.333},"min":{"1min":0.81799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.4950000000000001}]},{"osd":2,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.385,"15min":1.395},"min":{"1min":0.77500000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.21,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4630000000000001,"15min":1.458},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.627}]},{"osd":3,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.446,"15min":1.4430000000000001},"min":{"1min":0.82899999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.2370000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.589,"5min":1.51,"15min":1.4970000000000001},"min":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.8180000000000001}]},{"osd":4,"last update":"Wed Mar 10 13:48:50 2021","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.88400000000000001,"15min":0.84499999999999997},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.0169999999999999,"15min":0.95499999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.7330000000000001}]},{"osd":6,"last update":"Wed Mar 10 13:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.1539999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.2799999999999998}]},{"osd":7,"last update":"Wed Mar 10 13:48:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.4420000000000002},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":2.5670000000000002}]}]}]}} 2021-03-10T13:49:20.384 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2021-03-10T13:49:20.384 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-03-10T13:49:20.385 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2021-03-10T13:49:20.385 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- ceph health --format=json 2021-03-10T13:49:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:20 gibba032 bash[9829]: cluster 2021-03-10T13:49:19.160546+0000 mgr.y (mgr.14140) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:20 gibba011 bash[9813]: cluster 2021-03-10T13:49:19.160546+0000 mgr.y (mgr.14140) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:20 gibba011 bash[13015]: cluster 2021-03-10T13:49:19.160546+0000 mgr.y (mgr.14140) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:21.556 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:21 gibba011 bash[9813]: audit 2021-03-10T13:49:19.756249+0000 mgr.y (mgr.14140) 231 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:21.557 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:21 gibba011 bash[13015]: audit 2021-03-10T13:49:19.756249+0000 mgr.y (mgr.14140) 231 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:21 gibba032 bash[9829]: audit 2021-03-10T13:49:19.756249+0000 mgr.y (mgr.14140) 231 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-03-10T13:49:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:22 gibba011 bash[9813]: cluster 2021-03-10T13:49:21.161391+0000 mgr.y (mgr.14140) 232 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:22 gibba011 bash[13015]: cluster 2021-03-10T13:49:21.161391+0000 mgr.y (mgr.14140) 232 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:22 gibba032 bash[9829]: cluster 2021-03-10T13:49:21.161391+0000 mgr.y (mgr.14140) 232 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:23.285 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-03-10T13:49:23.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:23 gibba032 bash[9829]: audit 2021-03-10T13:49:23.284520+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/3810087781' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2021-03-10T13:49:23.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:23 gibba011 bash[9813]: audit 2021-03-10T13:49:23.284520+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/3810087781' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2021-03-10T13:49:23.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:23 gibba011 bash[13015]: audit 2021-03-10T13:49:23.284520+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/3810087781' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2021-03-10T13:49:23.978 INFO:teuthology.orchestra.run.gibba011.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-03-10T13:49:23.978 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2021-03-10T13:49:23.979 INFO:tasks.cephadm:Setup complete, yielding 2021-03-10T13:49:23.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-03-10T13:49:23.990 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@gibba011.front.sepia.ceph.com 2021-03-10T13:49:23.991 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest false --force' 2021-03-10T13:49:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:24 gibba032 bash[9829]: cluster 2021-03-10T13:49:23.161765+0000 mgr.y (mgr.14140) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:24 gibba011 bash[9813]: cluster 2021-03-10T13:49:23.161765+0000 mgr.y (mgr.14140) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:24 gibba011 bash[13015]: cluster 2021-03-10T13:49:23.161765+0000 mgr.y (mgr.14140) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:26.739 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:26 gibba011 bash[9813]: cluster 2021-03-10T13:49:25.162129+0000 mgr.y (mgr.14140) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:26.739 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:26 gibba011 bash[13015]: cluster 2021-03-10T13:49:25.162129+0000 mgr.y (mgr.14140) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:26 gibba032 bash[9829]: cluster 2021-03-10T13:49:25.162129+0000 mgr.y (mgr.14140) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:27.334 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-03-10T13:49:27.343 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@gibba011.front.sepia.ceph.com 2021-03-10T13:49:27.343 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2021-03-10T13:49:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:27 gibba032 bash[9829]: audit 2021-03-10T13:49:26.684334+0000 mon.c (mon.1) 18 : audit [INF] from='client.? 172.21.2.111:0/163774221' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:27 gibba032 bash[9829]: audit 2021-03-10T13:49:26.684581+0000 mon.a (mon.0) 529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:27 gibba032 bash[9829]: audit 2021-03-10T13:49:26.689281+0000 mon.a (mon.0) 530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]': finished 2021-03-10T13:49:27.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[9813]: audit 2021-03-10T13:49:26.684334+0000 mon.c (mon.1) 18 : audit [INF] from='client.? 172.21.2.111:0/163774221' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[9813]: audit 2021-03-10T13:49:26.684581+0000 mon.a (mon.0) 529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[9813]: audit 2021-03-10T13:49:26.689281+0000 mon.a (mon.0) 530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]': finished 2021-03-10T13:49:27.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[13015]: audit 2021-03-10T13:49:26.684334+0000 mon.c (mon.1) 18 : audit [INF] from='client.? 172.21.2.111:0/163774221' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[13015]: audit 2021-03-10T13:49:26.684581+0000 mon.a (mon.0) 529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]: dispatch 2021-03-10T13:49:27.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:27 gibba011 bash[13015]: audit 2021-03-10T13:49:26.689281+0000 mon.a (mon.0) 530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/use_repo_digest", "value": "false", "force": true}]': finished 2021-03-10T13:49:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:28 gibba011 bash[9813]: cluster 2021-03-10T13:49:27.162927+0000 mgr.y (mgr.14140) 235 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:28 gibba011 bash[13015]: cluster 2021-03-10T13:49:27.162927+0000 mgr.y (mgr.14140) 235 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:28 gibba032 bash[9829]: cluster 2021-03-10T13:49:27.162927+0000 mgr.y (mgr.14140) 235 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:30 gibba032 bash[9829]: cluster 2021-03-10T13:49:29.163550+0000 mgr.y (mgr.14140) 236 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:30 gibba011 bash[9813]: cluster 2021-03-10T13:49:29.163550+0000 mgr.y (mgr.14140) 236 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:30 gibba011 bash[13015]: cluster 2021-03-10T13:49:29.163550+0000 mgr.y (mgr.14140) 236 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:31.492 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:49:31.492 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "5167b4b4-efd7-44e8-b5d4-438b3197b20a", 2021-03-10T13:49:31.493 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "r", 2021-03-10T13:49:31.493 INFO:teuthology.orchestra.run.gibba011.stdout: "current_period": "69964699-9689-4633-8a17-bdc6f2d63cb5", 2021-03-10T13:49:31.493 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 1 2021-03-10T13:49:31.493 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:49:31.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:31 gibba032 bash[9829]: cluster 2021-03-10T13:49:30.466410+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2021-03-10T13:49:31.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:31 gibba032 bash[9829]: audit 2021-03-10T13:49:30.467025+0000 mon.c (mon.1) 19 : audit [INF] from='client.? 172.21.2.111:0/4256597429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:31.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:31 gibba032 bash[9829]: audit 2021-03-10T13:49:30.467544+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[9813]: cluster 2021-03-10T13:49:30.466410+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2021-03-10T13:49:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[9813]: audit 2021-03-10T13:49:30.467025+0000 mon.c (mon.1) 19 : audit [INF] from='client.? 172.21.2.111:0/4256597429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[9813]: audit 2021-03-10T13:49:30.467544+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[13015]: cluster 2021-03-10T13:49:30.466410+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2021-03-10T13:49:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[13015]: audit 2021-03-10T13:49:30.467025+0000 mon.c (mon.1) 19 : audit [INF] from='client.? 172.21.2.111:0/4256597429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:31 gibba011 bash[13015]: audit 2021-03-10T13:49:30.467544+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2021-03-10T13:49:32.181 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2021-03-10T13:49:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[9813]: cluster 2021-03-10T13:49:31.164156+0000 mgr.y (mgr.14140) 237 : cluster [DBG] pgmap v223: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[9813]: audit 2021-03-10T13:49:31.471959+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2021-03-10T13:49:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[9813]: cluster 2021-03-10T13:49:31.472014+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2021-03-10T13:49:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[13015]: cluster 2021-03-10T13:49:31.164156+0000 mgr.y (mgr.14140) 237 : cluster [DBG] pgmap v223: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[13015]: audit 2021-03-10T13:49:31.471959+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2021-03-10T13:49:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:32 gibba011 bash[13015]: cluster 2021-03-10T13:49:31.472014+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2021-03-10T13:49:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:32 gibba032 bash[9829]: cluster 2021-03-10T13:49:31.164156+0000 mgr.y (mgr.14140) 237 : cluster [DBG] pgmap v223: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 3.5 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:32 gibba032 bash[9829]: audit 2021-03-10T13:49:31.471959+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2021-03-10T13:49:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:32 gibba032 bash[9829]: cluster 2021-03-10T13:49:31.472014+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2021-03-10T13:49:33.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:33 gibba032 bash[9829]: cluster 2021-03-10T13:49:32.485825+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2021-03-10T13:49:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:33 gibba011 bash[9813]: cluster 2021-03-10T13:49:32.485825+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2021-03-10T13:49:33.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:33 gibba011 bash[13015]: cluster 2021-03-10T13:49:32.485825+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2021-03-10T13:49:34.765 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:34 gibba011 bash[9813]: cluster 2021-03-10T13:49:33.164937+0000 mgr.y (mgr.14140) 238 : cluster [DBG] pgmap v226: 33 pgs: 7 active+clean, 10 creating+peering, 16 unknown; 181 B data, 3.6 MiB used, 664 GiB / 672 GiB avail; 682 B/s wr, 0 op/s 2021-03-10T13:49:34.766 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:34 gibba011 bash[13015]: cluster 2021-03-10T13:49:33.164937+0000 mgr.y (mgr.14140) 238 : cluster [DBG] pgmap v226: 33 pgs: 7 active+clean, 10 creating+peering, 16 unknown; 181 B data, 3.6 MiB used, 664 GiB / 672 GiB avail; 682 B/s wr, 0 op/s 2021-03-10T13:49:34.887 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:49:34.887 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "c2caafc6-293a-4a5b-a747-96bc075177e7", 2021-03-10T13:49:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "default", 2021-03-10T13:49:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "api_name": "default", 2021-03-10T13:49:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "is_master": "true", 2021-03-10T13:49:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "endpoints": [], 2021-03-10T13:49:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "hostnames": [], 2021-03-10T13:49:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "hostnames_s3website": [], 2021-03-10T13:49:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "master_zone": "", 2021-03-10T13:49:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "zones": [], 2021-03-10T13:49:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "placement_targets": [], 2021-03-10T13:49:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "default_placement": "", 2021-03-10T13:49:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_id": "5167b4b4-efd7-44e8-b5d4-438b3197b20a", 2021-03-10T13:49:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "sync_policy": { 2021-03-10T13:49:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "groups": [] 2021-03-10T13:49:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:34.891 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:49:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:34 gibba032 bash[9829]: cluster 2021-03-10T13:49:33.164937+0000 mgr.y (mgr.14140) 238 : cluster [DBG] pgmap v226: 33 pgs: 7 active+clean, 10 creating+peering, 16 unknown; 181 B data, 3.6 MiB used, 664 GiB / 672 GiB avail; 682 B/s wr, 0 op/s 2021-03-10T13:49:35.553 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2021-03-10T13:49:36.742 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:36 gibba011 bash[9813]: cluster 2021-03-10T13:49:35.165967+0000 mgr.y (mgr.14140) 239 : cluster [DBG] pgmap v227: 33 pgs: 23 active+clean, 10 creating+peering; 273 B data, 3.8 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s wr, 1 op/s 2021-03-10T13:49:36.743 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:36 gibba011 bash[13015]: cluster 2021-03-10T13:49:35.165967+0000 mgr.y (mgr.14140) 239 : cluster [DBG] pgmap v227: 33 pgs: 23 active+clean, 10 creating+peering; 273 B data, 3.8 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s wr, 1 op/s 2021-03-10T13:49:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:36 gibba032 bash[9829]: cluster 2021-03-10T13:49:35.165967+0000 mgr.y (mgr.14140) 239 : cluster [DBG] pgmap v227: 33 pgs: 23 active+clean, 10 creating+peering; 273 B data, 3.8 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s wr, 1 op/s 2021-03-10T13:49:38.028 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:49:38.029 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "10194b89-4d9a-4f0c-8a0c-134c2d981d08", 2021-03-10T13:49:38.029 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "z", 2021-03-10T13:49:38.029 INFO:teuthology.orchestra.run.gibba011.stdout: "domain_root": "z.rgw.meta:root", 2021-03-10T13:49:38.030 INFO:teuthology.orchestra.run.gibba011.stdout: "control_pool": "z.rgw.control", 2021-03-10T13:49:38.030 INFO:teuthology.orchestra.run.gibba011.stdout: "gc_pool": "z.rgw.log:gc", 2021-03-10T13:49:38.030 INFO:teuthology.orchestra.run.gibba011.stdout: "lc_pool": "z.rgw.log:lc", 2021-03-10T13:49:38.030 INFO:teuthology.orchestra.run.gibba011.stdout: "log_pool": "z.rgw.log", 2021-03-10T13:49:38.030 INFO:teuthology.orchestra.run.gibba011.stdout: "intent_log_pool": "z.rgw.log:intent", 2021-03-10T13:49:38.031 INFO:teuthology.orchestra.run.gibba011.stdout: "usage_log_pool": "z.rgw.log:usage", 2021-03-10T13:49:38.031 INFO:teuthology.orchestra.run.gibba011.stdout: "roles_pool": "z.rgw.meta:roles", 2021-03-10T13:49:38.031 INFO:teuthology.orchestra.run.gibba011.stdout: "reshard_pool": "z.rgw.log:reshard", 2021-03-10T13:49:38.031 INFO:teuthology.orchestra.run.gibba011.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2021-03-10T13:49:38.031 INFO:teuthology.orchestra.run.gibba011.stdout: "user_email_pool": "z.rgw.meta:users.email", 2021-03-10T13:49:38.032 INFO:teuthology.orchestra.run.gibba011.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2021-03-10T13:49:38.032 INFO:teuthology.orchestra.run.gibba011.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2021-03-10T13:49:38.032 INFO:teuthology.orchestra.run.gibba011.stdout: "otp_pool": "z.rgw.otp", 2021-03-10T13:49:38.032 INFO:teuthology.orchestra.run.gibba011.stdout: "system_key": { 2021-03-10T13:49:38.032 INFO:teuthology.orchestra.run.gibba011.stdout: "access_key": "", 2021-03-10T13:49:38.033 INFO:teuthology.orchestra.run.gibba011.stdout: "secret_key": "" 2021-03-10T13:49:38.033 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:49:38.033 INFO:teuthology.orchestra.run.gibba011.stdout: "placement_pools": [ 2021-03-10T13:49:38.033 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-03-10T13:49:38.034 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "default-placement", 2021-03-10T13:49:38.034 INFO:teuthology.orchestra.run.gibba011.stdout: "val": { 2021-03-10T13:49:38.034 INFO:teuthology.orchestra.run.gibba011.stdout: "index_pool": "z.rgw.buckets.index", 2021-03-10T13:49:38.034 INFO:teuthology.orchestra.run.gibba011.stdout: "storage_classes": { 2021-03-10T13:49:38.034 INFO:teuthology.orchestra.run.gibba011.stdout: "STANDARD": { 2021-03-10T13:49:38.035 INFO:teuthology.orchestra.run.gibba011.stdout: "data_pool": "z.rgw.buckets.data" 2021-03-10T13:49:38.035 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:38.035 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:49:38.035 INFO:teuthology.orchestra.run.gibba011.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2021-03-10T13:49:38.036 INFO:teuthology.orchestra.run.gibba011.stdout: "index_type": 0 2021-03-10T13:49:38.036 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:38.036 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:38.036 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-03-10T13:49:38.036 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_id": "5167b4b4-efd7-44e8-b5d4-438b3197b20a" 2021-03-10T13:49:38.037 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:49:38.646 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2021-03-10T13:49:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:38 gibba011 bash[9813]: cluster 2021-03-10T13:49:37.166559+0000 mgr.y (mgr.14140) 240 : cluster [DBG] pgmap v228: 33 pgs: 33 active+clean; 608 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 1.5 KiB/s wr, 1 op/s 2021-03-10T13:49:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:38 gibba011 bash[13015]: cluster 2021-03-10T13:49:37.166559+0000 mgr.y (mgr.14140) 240 : cluster [DBG] pgmap v228: 33 pgs: 33 active+clean; 608 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 1.5 KiB/s wr, 1 op/s 2021-03-10T13:49:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:38 gibba032 bash[9829]: cluster 2021-03-10T13:49:37.166559+0000 mgr.y (mgr.14140) 240 : cluster [DBG] pgmap v228: 33 pgs: 33 active+clean; 608 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 1.5 KiB/s wr, 1 op/s 2021-03-10T13:49:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:40 gibba011 bash[9813]: cluster 2021-03-10T13:49:39.167155+0000 mgr.y (mgr.14140) 241 : cluster [DBG] pgmap v229: 33 pgs: 33 active+clean; 654 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 127 B/s rd, 1.5 KiB/s wr, 2 op/s 2021-03-10T13:49:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:40 gibba011 bash[13015]: cluster 2021-03-10T13:49:39.167155+0000 mgr.y (mgr.14140) 241 : cluster [DBG] pgmap v229: 33 pgs: 33 active+clean; 654 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 127 B/s rd, 1.5 KiB/s wr, 2 op/s 2021-03-10T13:49:41.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:40 gibba032 bash[9829]: cluster 2021-03-10T13:49:39.167155+0000 mgr.y (mgr.14140) 241 : cluster [DBG] pgmap v229: 33 pgs: 33 active+clean; 654 B data, 3.9 MiB used, 664 GiB / 672 GiB avail; 127 B/s rd, 1.5 KiB/s wr, 2 op/s 2021-03-10T13:49:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[9813]: cluster 2021-03-10T13:49:41.167562+0000 mgr.y (mgr.14140) 242 : cluster [DBG] pgmap v230: 33 pgs: 33 active+clean; 1.8 KiB data, 3.9 MiB used, 664 GiB / 672 GiB avail; 422 B/s rd, 1.6 KiB/s wr, 2 op/s 2021-03-10T13:49:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[9813]: cluster 2021-03-10T13:49:41.553019+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2021-03-10T13:49:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[9813]: audit 2021-03-10T13:49:41.553862+0000 mon.a (mon.0) 537 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2021-03-10T13:49:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[13015]: cluster 2021-03-10T13:49:41.167562+0000 mgr.y (mgr.14140) 242 : cluster [DBG] pgmap v230: 33 pgs: 33 active+clean; 1.8 KiB data, 3.9 MiB used, 664 GiB / 672 GiB avail; 422 B/s rd, 1.6 KiB/s wr, 2 op/s 2021-03-10T13:49:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[13015]: cluster 2021-03-10T13:49:41.553019+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2021-03-10T13:49:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:42 gibba011 bash[13015]: audit 2021-03-10T13:49:41.553862+0000 mon.a (mon.0) 537 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2021-03-10T13:49:43.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:42 gibba032 bash[9829]: cluster 2021-03-10T13:49:41.167562+0000 mgr.y (mgr.14140) 242 : cluster [DBG] pgmap v230: 33 pgs: 33 active+clean; 1.8 KiB data, 3.9 MiB used, 664 GiB / 672 GiB avail; 422 B/s rd, 1.6 KiB/s wr, 2 op/s 2021-03-10T13:49:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:42 gibba032 bash[9829]: cluster 2021-03-10T13:49:41.553019+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2021-03-10T13:49:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:42 gibba032 bash[9829]: audit 2021-03-10T13:49:41.553862+0000 mon.a (mon.0) 537 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2021-03-10T13:49:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:43 gibba011 bash[9813]: audit 2021-03-10T13:49:42.563259+0000 mon.a (mon.0) 538 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2021-03-10T13:49:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:43 gibba011 bash[9813]: cluster 2021-03-10T13:49:42.563381+0000 mon.a (mon.0) 539 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2021-03-10T13:49:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:43 gibba011 bash[13015]: audit 2021-03-10T13:49:42.563259+0000 mon.a (mon.0) 538 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2021-03-10T13:49:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:43 gibba011 bash[13015]: cluster 2021-03-10T13:49:42.563381+0000 mon.a (mon.0) 539 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2021-03-10T13:49:44.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:43 gibba032 bash[9829]: audit 2021-03-10T13:49:42.563259+0000 mon.a (mon.0) 538 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2021-03-10T13:49:44.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:43 gibba032 bash[9829]: cluster 2021-03-10T13:49:42.563381+0000 mon.a (mon.0) 539 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2021-03-10T13:49:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:44 gibba032 bash[9829]: cluster 2021-03-10T13:49:43.168243+0000 mgr.y (mgr.14140) 243 : cluster [DBG] pgmap v233: 65 pgs: 6 creating+peering, 26 unknown, 33 active+clean; 1.8 KiB data, 4.0 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2021-03-10T13:49:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:44 gibba032 bash[9829]: cluster 2021-03-10T13:49:43.591085+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2021-03-10T13:49:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:44 gibba032 bash[9829]: audit 2021-03-10T13:49:43.592010+0000 mon.a (mon.0) 541 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2021-03-10T13:49:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[9813]: cluster 2021-03-10T13:49:43.168243+0000 mgr.y (mgr.14140) 243 : cluster [DBG] pgmap v233: 65 pgs: 6 creating+peering, 26 unknown, 33 active+clean; 1.8 KiB data, 4.0 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2021-03-10T13:49:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[9813]: cluster 2021-03-10T13:49:43.591085+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2021-03-10T13:49:45.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[9813]: audit 2021-03-10T13:49:43.592010+0000 mon.a (mon.0) 541 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2021-03-10T13:49:45.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[13015]: cluster 2021-03-10T13:49:43.168243+0000 mgr.y (mgr.14140) 243 : cluster [DBG] pgmap v233: 65 pgs: 6 creating+peering, 26 unknown, 33 active+clean; 1.8 KiB data, 4.0 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2021-03-10T13:49:45.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[13015]: cluster 2021-03-10T13:49:43.591085+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2021-03-10T13:49:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:44 gibba011 bash[13015]: audit 2021-03-10T13:49:43.592010+0000 mon.a (mon.0) 541 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2021-03-10T13:49:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:45 gibba032 bash[9829]: audit 2021-03-10T13:49:44.595004+0000 mon.a (mon.0) 542 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2021-03-10T13:49:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:45 gibba032 bash[9829]: cluster 2021-03-10T13:49:44.595138+0000 mon.a (mon.0) 543 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2021-03-10T13:49:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:45 gibba032 bash[9829]: cluster 2021-03-10T13:49:45.168623+0000 mgr.y (mgr.14140) 244 : cluster [DBG] pgmap v236: 97 pgs: 8 creating+peering, 41 unknown, 48 active+clean; 1.8 KiB data, 4.1 MiB used, 664 GiB / 672 GiB avail; 4.5 KiB/s rd, 4 op/s 2021-03-10T13:49:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[9813]: audit 2021-03-10T13:49:44.595004+0000 mon.a (mon.0) 542 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2021-03-10T13:49:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[9813]: cluster 2021-03-10T13:49:44.595138+0000 mon.a (mon.0) 543 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2021-03-10T13:49:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[9813]: cluster 2021-03-10T13:49:45.168623+0000 mgr.y (mgr.14140) 244 : cluster [DBG] pgmap v236: 97 pgs: 8 creating+peering, 41 unknown, 48 active+clean; 1.8 KiB data, 4.1 MiB used, 664 GiB / 672 GiB avail; 4.5 KiB/s rd, 4 op/s 2021-03-10T13:49:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[13015]: audit 2021-03-10T13:49:44.595004+0000 mon.a (mon.0) 542 : audit [INF] from='client.? 172.21.2.111:0/2272124426' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2021-03-10T13:49:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[13015]: cluster 2021-03-10T13:49:44.595138+0000 mon.a (mon.0) 543 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2021-03-10T13:49:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:45 gibba011 bash[13015]: cluster 2021-03-10T13:49:45.168623+0000 mgr.y (mgr.14140) 244 : cluster [DBG] pgmap v236: 97 pgs: 8 creating+peering, 41 unknown, 48 active+clean; 1.8 KiB data, 4.1 MiB used, 664 GiB / 672 GiB avail; 4.5 KiB/s rd, 4 op/s 2021-03-10T13:49:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:46 gibba032 bash[9829]: cluster 2021-03-10T13:49:45.613337+0000 mon.a (mon.0) 544 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2021-03-10T13:49:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:46 gibba032 bash[9829]: audit 2021-03-10T13:49:45.615447+0000 mon.c (mon.1) 20 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:46 gibba032 bash[9829]: audit 2021-03-10T13:49:45.615627+0000 mon.a (mon.0) 545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:47.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[9813]: cluster 2021-03-10T13:49:45.613337+0000 mon.a (mon.0) 544 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2021-03-10T13:49:47.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[9813]: audit 2021-03-10T13:49:45.615447+0000 mon.c (mon.1) 20 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:47.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[9813]: audit 2021-03-10T13:49:45.615627+0000 mon.a (mon.0) 545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:47.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[13015]: cluster 2021-03-10T13:49:45.613337+0000 mon.a (mon.0) 544 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2021-03-10T13:49:47.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[13015]: audit 2021-03-10T13:49:45.615447+0000 mon.c (mon.1) 20 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:47.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:46 gibba011 bash[13015]: audit 2021-03-10T13:49:45.615627+0000 mon.a (mon.0) 545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2021-03-10T13:49:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:47 gibba032 bash[9829]: audit 2021-03-10T13:49:46.623528+0000 mon.a (mon.0) 546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2021-03-10T13:49:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:47 gibba032 bash[9829]: cluster 2021-03-10T13:49:46.623591+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2021-03-10T13:49:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:47 gibba032 bash[9829]: audit 2021-03-10T13:49:46.625126+0000 mon.c (mon.1) 21 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:47 gibba032 bash[9829]: audit 2021-03-10T13:49:46.629360+0000 mon.a (mon.0) 548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:47 gibba032 bash[9829]: cluster 2021-03-10T13:49:47.169647+0000 mgr.y (mgr.14140) 245 : cluster [DBG] pgmap v239: 129 pgs: 20 creating+peering, 29 unknown, 80 active+clean; 1.8 KiB data, 4.2 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 1 op/s 2021-03-10T13:49:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[9813]: audit 2021-03-10T13:49:46.623528+0000 mon.a (mon.0) 546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2021-03-10T13:49:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[9813]: cluster 2021-03-10T13:49:46.623591+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2021-03-10T13:49:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[9813]: audit 2021-03-10T13:49:46.625126+0000 mon.c (mon.1) 21 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[9813]: audit 2021-03-10T13:49:46.629360+0000 mon.a (mon.0) 548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[9813]: cluster 2021-03-10T13:49:47.169647+0000 mgr.y (mgr.14140) 245 : cluster [DBG] pgmap v239: 129 pgs: 20 creating+peering, 29 unknown, 80 active+clean; 1.8 KiB data, 4.2 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 1 op/s 2021-03-10T13:49:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[13015]: audit 2021-03-10T13:49:46.623528+0000 mon.a (mon.0) 546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2021-03-10T13:49:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[13015]: cluster 2021-03-10T13:49:46.623591+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2021-03-10T13:49:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[13015]: audit 2021-03-10T13:49:46.625126+0000 mon.c (mon.1) 21 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[13015]: audit 2021-03-10T13:49:46.629360+0000 mon.a (mon.0) 548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2021-03-10T13:49:48.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:47 gibba011 bash[13015]: cluster 2021-03-10T13:49:47.169647+0000 mgr.y (mgr.14140) 245 : cluster [DBG] pgmap v239: 129 pgs: 20 creating+peering, 29 unknown, 80 active+clean; 1.8 KiB data, 4.2 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 1 op/s 2021-03-10T13:49:48.875 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:49:48.875 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "cb3b83ed-262f-4dd4-90e7-3c6ccea7e390", 2021-03-10T13:49:48.875 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 1, 2021-03-10T13:49:48.875 INFO:teuthology.orchestra.run.gibba011.stdout: "predecessor_uuid": "69964699-9689-4633-8a17-bdc6f2d63cb5", 2021-03-10T13:49:48.876 INFO:teuthology.orchestra.run.gibba011.stdout: "sync_status": [], 2021-03-10T13:49:48.876 INFO:teuthology.orchestra.run.gibba011.stdout: "period_map": { 2021-03-10T13:49:48.876 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "cb3b83ed-262f-4dd4-90e7-3c6ccea7e390", 2021-03-10T13:49:48.876 INFO:teuthology.orchestra.run.gibba011.stdout: "zonegroups": [ 2021-03-10T13:49:48.877 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-03-10T13:49:48.877 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "c2caafc6-293a-4a5b-a747-96bc075177e7", 2021-03-10T13:49:48.877 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "default", 2021-03-10T13:49:48.877 INFO:teuthology.orchestra.run.gibba011.stdout: "api_name": "default", 2021-03-10T13:49:48.877 INFO:teuthology.orchestra.run.gibba011.stdout: "is_master": "true", 2021-03-10T13:49:48.878 INFO:teuthology.orchestra.run.gibba011.stdout: "endpoints": [], 2021-03-10T13:49:48.878 INFO:teuthology.orchestra.run.gibba011.stdout: "hostnames": [], 2021-03-10T13:49:48.878 INFO:teuthology.orchestra.run.gibba011.stdout: "hostnames_s3website": [], 2021-03-10T13:49:48.878 INFO:teuthology.orchestra.run.gibba011.stdout: "master_zone": "10194b89-4d9a-4f0c-8a0c-134c2d981d08", 2021-03-10T13:49:48.878 INFO:teuthology.orchestra.run.gibba011.stdout: "zones": [ 2021-03-10T13:49:48.879 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-03-10T13:49:48.879 INFO:teuthology.orchestra.run.gibba011.stdout: "id": "10194b89-4d9a-4f0c-8a0c-134c2d981d08", 2021-03-10T13:49:48.879 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "z", 2021-03-10T13:49:48.879 INFO:teuthology.orchestra.run.gibba011.stdout: "endpoints": [], 2021-03-10T13:49:48.879 INFO:teuthology.orchestra.run.gibba011.stdout: "log_meta": "false", 2021-03-10T13:49:48.880 INFO:teuthology.orchestra.run.gibba011.stdout: "log_data": "false", 2021-03-10T13:49:48.880 INFO:teuthology.orchestra.run.gibba011.stdout: "bucket_index_max_shards": 11, 2021-03-10T13:49:48.880 INFO:teuthology.orchestra.run.gibba011.stdout: "read_only": "false", 2021-03-10T13:49:48.880 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_type": "", 2021-03-10T13:49:48.880 INFO:teuthology.orchestra.run.gibba011.stdout: "sync_from_all": "true", 2021-03-10T13:49:48.881 INFO:teuthology.orchestra.run.gibba011.stdout: "sync_from": [], 2021-03-10T13:49:48.881 INFO:teuthology.orchestra.run.gibba011.stdout: "redirect_zone": "" 2021-03-10T13:49:48.881 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.881 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-03-10T13:49:48.881 INFO:teuthology.orchestra.run.gibba011.stdout: "placement_targets": [ 2021-03-10T13:49:48.882 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-03-10T13:49:48.882 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "default-placement", 2021-03-10T13:49:48.882 INFO:teuthology.orchestra.run.gibba011.stdout: "tags": [], 2021-03-10T13:49:48.882 INFO:teuthology.orchestra.run.gibba011.stdout: "storage_classes": [ 2021-03-10T13:49:48.882 INFO:teuthology.orchestra.run.gibba011.stdout: "STANDARD" 2021-03-10T13:49:48.883 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-03-10T13:49:48.883 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.883 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-03-10T13:49:48.883 INFO:teuthology.orchestra.run.gibba011.stdout: "default_placement": "default-placement", 2021-03-10T13:49:48.883 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_id": "5167b4b4-efd7-44e8-b5d4-438b3197b20a", 2021-03-10T13:49:48.884 INFO:teuthology.orchestra.run.gibba011.stdout: "sync_policy": { 2021-03-10T13:49:48.884 INFO:teuthology.orchestra.run.gibba011.stdout: "groups": [] 2021-03-10T13:49:48.884 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.884 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.884 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-03-10T13:49:48.885 INFO:teuthology.orchestra.run.gibba011.stdout: "short_zone_ids": [ 2021-03-10T13:49:48.885 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-03-10T13:49:48.885 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "10194b89-4d9a-4f0c-8a0c-134c2d981d08", 2021-03-10T13:49:48.885 INFO:teuthology.orchestra.run.gibba011.stdout: "val": 2375366110 2021-03-10T13:49:48.886 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.886 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-03-10T13:49:48.886 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:49:48.886 INFO:teuthology.orchestra.run.gibba011.stdout: "master_zonegroup": "c2caafc6-293a-4a5b-a747-96bc075177e7", 2021-03-10T13:49:48.886 INFO:teuthology.orchestra.run.gibba011.stdout: "master_zone": "10194b89-4d9a-4f0c-8a0c-134c2d981d08", 2021-03-10T13:49:48.887 INFO:teuthology.orchestra.run.gibba011.stdout: "period_config": { 2021-03-10T13:49:48.887 INFO:teuthology.orchestra.run.gibba011.stdout: "bucket_quota": { 2021-03-10T13:49:48.887 INFO:teuthology.orchestra.run.gibba011.stdout: "enabled": false, 2021-03-10T13:49:48.887 INFO:teuthology.orchestra.run.gibba011.stdout: "check_on_raw": false, 2021-03-10T13:49:48.888 INFO:teuthology.orchestra.run.gibba011.stdout: "max_size": -1, 2021-03-10T13:49:48.888 INFO:teuthology.orchestra.run.gibba011.stdout: "max_size_kb": 0, 2021-03-10T13:49:48.888 INFO:teuthology.orchestra.run.gibba011.stdout: "max_objects": -1 2021-03-10T13:49:48.888 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:49:48.888 INFO:teuthology.orchestra.run.gibba011.stdout: "user_quota": { 2021-03-10T13:49:48.889 INFO:teuthology.orchestra.run.gibba011.stdout: "enabled": false, 2021-03-10T13:49:48.889 INFO:teuthology.orchestra.run.gibba011.stdout: "check_on_raw": false, 2021-03-10T13:49:48.889 INFO:teuthology.orchestra.run.gibba011.stdout: "max_size": -1, 2021-03-10T13:49:48.889 INFO:teuthology.orchestra.run.gibba011.stdout: "max_size_kb": 0, 2021-03-10T13:49:48.889 INFO:teuthology.orchestra.run.gibba011.stdout: "max_objects": -1 2021-03-10T13:49:48.890 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:49:48.890 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:49:48.890 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_id": "5167b4b4-efd7-44e8-b5d4-438b3197b20a", 2021-03-10T13:49:48.890 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_name": "r", 2021-03-10T13:49:48.890 INFO:teuthology.orchestra.run.gibba011.stdout: "realm_epoch": 2 2021-03-10T13:49:48.891 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:49:48.959 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: audit 2021-03-10T13:49:47.627720+0000 mon.a (mon.0) 549 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2021-03-10T13:49:48.960 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: cluster 2021-03-10T13:49:47.627770+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2021-03-10T13:49:48.960 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: audit 2021-03-10T13:49:47.628253+0000 mon.c (mon.1) 22 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:48.960 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: audit 2021-03-10T13:49:47.628707+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:48.960 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: audit 2021-03-10T13:49:48.629845+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2021-03-10T13:49:48.960 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[9813]: cluster 2021-03-10T13:49:48.629879+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2021-03-10T13:49:48.961 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: audit 2021-03-10T13:49:47.627720+0000 mon.a (mon.0) 549 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2021-03-10T13:49:48.961 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: cluster 2021-03-10T13:49:47.627770+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2021-03-10T13:49:48.961 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: audit 2021-03-10T13:49:47.628253+0000 mon.c (mon.1) 22 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:48.962 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: audit 2021-03-10T13:49:47.628707+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:48.962 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: audit 2021-03-10T13:49:48.629845+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2021-03-10T13:49:48.962 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:48 gibba011 bash[13015]: cluster 2021-03-10T13:49:48.629879+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2021-03-10T13:49:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: audit 2021-03-10T13:49:47.627720+0000 mon.a (mon.0) 549 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2021-03-10T13:49:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: cluster 2021-03-10T13:49:47.627770+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2021-03-10T13:49:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: audit 2021-03-10T13:49:47.628253+0000 mon.c (mon.1) 22 : audit [INF] from='client.? 172.21.2.111:0/43862420' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: audit 2021-03-10T13:49:47.628707+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2021-03-10T13:49:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: audit 2021-03-10T13:49:48.629845+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2021-03-10T13:49:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:48 gibba032 bash[9829]: cluster 2021-03-10T13:49:48.629879+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2021-03-10T13:49:49.651 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'ceph orch apply rgw r z --placement=1' 2021-03-10T13:49:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:49 gibba032 bash[9829]: cluster 2021-03-10T13:49:49.170520+0000 mgr.y (mgr.14140) 246 : cluster [DBG] pgmap v242: 129 pgs: 14 creating+peering, 9 unknown, 106 active+clean; 1.8 KiB data, 4.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:49 gibba011 bash[9813]: cluster 2021-03-10T13:49:49.170520+0000 mgr.y (mgr.14140) 246 : cluster [DBG] pgmap v242: 129 pgs: 14 creating+peering, 9 unknown, 106 active+clean; 1.8 KiB data, 4.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:49 gibba011 bash[13015]: cluster 2021-03-10T13:49:49.170520+0000 mgr.y (mgr.14140) 246 : cluster [DBG] pgmap v242: 129 pgs: 14 creating+peering, 9 unknown, 106 active+clean; 1.8 KiB data, 4.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:49:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:52 gibba011 bash[13015]: cluster 2021-03-10T13:49:51.171884+0000 mgr.y (mgr.14140) 247 : cluster [DBG] pgmap v243: 129 pgs: 12 creating+peering, 117 active+clean; 3.7 KiB data, 4.6 MiB used, 664 GiB / 672 GiB avail; 368 B/s rd, 2.3 KiB/s wr, 8 op/s 2021-03-10T13:49:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:52 gibba011 bash[9813]: cluster 2021-03-10T13:49:51.171884+0000 mgr.y (mgr.14140) 247 : cluster [DBG] pgmap v243: 129 pgs: 12 creating+peering, 117 active+clean; 3.7 KiB data, 4.6 MiB used, 664 GiB / 672 GiB avail; 368 B/s rd, 2.3 KiB/s wr, 8 op/s 2021-03-10T13:49:52.342 INFO:teuthology.orchestra.run.gibba011.stdout:Scheduled rgw update... 2021-03-10T13:49:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:52 gibba032 bash[9829]: cluster 2021-03-10T13:49:51.171884+0000 mgr.y (mgr.14140) 247 : cluster [DBG] pgmap v243: 129 pgs: 12 creating+peering, 117 active+clean; 3.7 KiB data, 4.6 MiB used, 664 GiB / 672 GiB avail; 368 B/s rd, 2.3 KiB/s wr, 8 op/s 2021-03-10T13:49:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.337023+0000 mgr.y (mgr.14140) 248 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "realm_name": "r", "zone_name": "z", "placement": "1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:49:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: cephadm 2021-03-10T13:49:52.337369+0000 mgr.y (mgr.14140) 249 : cephadm [INF] Saving service rgw.r.z spec with placement count:1 2021-03-10T13:49:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.337638+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]: dispatch 2021-03-10T13:49:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.341588+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]': finished 2021-03-10T13:49:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.343572+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]: dispatch 2021-03-10T13:49:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.349780+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]': finished 2021-03-10T13:49:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.350784+0000 mon.a (mon.0) 558 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]: dispatch 2021-03-10T13:49:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.353916+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]': finished 2021-03-10T13:49:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.354790+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]: dispatch 2021-03-10T13:49:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.357976+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]': finished 2021-03-10T13:49:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.358716+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]: dispatch 2021-03-10T13:49:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.361197+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]': finished 2021-03-10T13:49:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.361921+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:49:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: cephadm 2021-03-10T13:49:52.362539+0000 mgr.y (mgr.14140) 250 : cephadm [INF] Deploying daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:49:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:53 gibba032 bash[9829]: audit 2021-03-10T13:49:52.363016+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "client.rgw.r.z.gibba011.xbljmx", "key": "container_image"}]: dispatch 2021-03-10T13:49:53.523 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.337023+0000 mgr.y (mgr.14140) 248 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "realm_name": "r", "zone_name": "z", "placement": "1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:49:53.523 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: cephadm 2021-03-10T13:49:52.337369+0000 mgr.y (mgr.14140) 249 : cephadm [INF] Saving service rgw.r.z spec with placement count:1 2021-03-10T13:49:53.524 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.337638+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]: dispatch 2021-03-10T13:49:53.524 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.341588+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]': finished 2021-03-10T13:49:53.524 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.343572+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]: dispatch 2021-03-10T13:49:53.524 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.349780+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]': finished 2021-03-10T13:49:53.524 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.350784+0000 mon.a (mon.0) 558 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]: dispatch 2021-03-10T13:49:53.525 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.353916+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]': finished 2021-03-10T13:49:53.525 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.354790+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]: dispatch 2021-03-10T13:49:53.525 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.357976+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]': finished 2021-03-10T13:49:53.525 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.358716+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]: dispatch 2021-03-10T13:49:53.526 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.361197+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]': finished 2021-03-10T13:49:53.526 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.361921+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:49:53.526 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: cephadm 2021-03-10T13:49:52.362539+0000 mgr.y (mgr.14140) 250 : cephadm [INF] Deploying daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:49:53.526 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[9813]: audit 2021-03-10T13:49:52.363016+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "client.rgw.r.z.gibba011.xbljmx", "key": "container_image"}]: dispatch 2021-03-10T13:49:53.527 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.337023+0000 mgr.y (mgr.14140) 248 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "realm_name": "r", "zone_name": "z", "placement": "1", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:49:53.527 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: cephadm 2021-03-10T13:49:52.337369+0000 mgr.y (mgr.14140) 249 : cephadm [INF] Saving service rgw.r.z spec with placement count:1 2021-03-10T13:49:53.527 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.337638+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]: dispatch 2021-03-10T13:49:53.527 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.341588+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.rgw.r.z","val":"{\"created\": \"2021-03-10T13:49:52.337377\", \"spec\": {\"placement\": {\"count\": 1}, \"rgw_realm\": \"r\", \"rgw_zone\": \"z\", \"service_id\": \"r.z\", \"service_type\": \"rgw\"}}"}]': finished 2021-03-10T13:49:53.528 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.343572+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]: dispatch 2021-03-10T13:49:53.528 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.349780+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_zone", "value": "z"}]': finished 2021-03-10T13:49:53.528 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.350784+0000 mon.a (mon.0) 558 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]: dispatch 2021-03-10T13:49:53.528 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.353916+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r", "name": "rgw_realm", "value": "r"}]': finished 2021-03-10T13:49:53.528 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.354790+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]: dispatch 2021-03-10T13:49:53.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.357976+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "who": "client.rgw.r.z", "name": "rgw_frontends", "value": "beast port=80"}]': finished 2021-03-10T13:49:53.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.358716+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]: dispatch 2021-03-10T13:49:53.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.361197+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow rw", "mgr", "allow rw", "osd", "allow rwx"]}]': finished 2021-03-10T13:49:53.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.361921+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:49:53.529 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: cephadm 2021-03-10T13:49:52.362539+0000 mgr.y (mgr.14140) 250 : cephadm [INF] Deploying daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:49:53.530 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:53 gibba011 bash[13015]: audit 2021-03-10T13:49:52.363016+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "client.rgw.r.z.gibba011.xbljmx", "key": "container_image"}]: dispatch 2021-03-10T13:49:53.531 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'sleep 120' 2021-03-10T13:49:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:54 gibba032 bash[9829]: cluster 2021-03-10T13:49:53.173238+0000 mgr.y (mgr.14140) 251 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 6.8 KiB data, 5.0 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2021-03-10T13:49:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:54 gibba011 bash[9813]: cluster 2021-03-10T13:49:53.173238+0000 mgr.y (mgr.14140) 251 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 6.8 KiB data, 5.0 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2021-03-10T13:49:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:54 gibba011 bash[13015]: cluster 2021-03-10T13:49:53.173238+0000 mgr.y (mgr.14140) 251 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 6.8 KiB data, 5.0 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2021-03-10T13:49:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:56 gibba032 bash[9829]: cluster 2021-03-10T13:49:55.173805+0000 mgr.y (mgr.14140) 252 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 895 B/s rd, 4.9 KiB/s wr, 18 op/s 2021-03-10T13:49:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:56 gibba032 bash[9829]: audit 2021-03-10T13:49:55.671677+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:56 gibba032 bash[9829]: audit 2021-03-10T13:49:55.674526+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:49:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:56 gibba032 bash[9829]: audit 2021-03-10T13:49:55.674973+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:49:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[9813]: cluster 2021-03-10T13:49:55.173805+0000 mgr.y (mgr.14140) 252 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 895 B/s rd, 4.9 KiB/s wr, 18 op/s 2021-03-10T13:49:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[13015]: cluster 2021-03-10T13:49:55.173805+0000 mgr.y (mgr.14140) 252 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 895 B/s rd, 4.9 KiB/s wr, 18 op/s 2021-03-10T13:49:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[9813]: audit 2021-03-10T13:49:55.671677+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[9813]: audit 2021-03-10T13:49:55.674526+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:49:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[9813]: audit 2021-03-10T13:49:55.674973+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:49:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[13015]: audit 2021-03-10T13:49:55.671677+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[13015]: audit 2021-03-10T13:49:55.674526+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299522\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299580\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299609\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299636\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299662\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299688\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299714\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299740\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:28.299766\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"status\": 1, \"status_desc\": \"starting\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:49:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:56 gibba011 bash[13015]: audit 2021-03-10T13:49:55.674973+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:49:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:58 gibba032 bash[9829]: cluster 2021-03-10T13:49:57.175126+0000 mgr.y (mgr.14140) 253 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 750 B/s rd, 4.1 KiB/s wr, 15 op/s 2021-03-10T13:49:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:58 gibba032 bash[9829]: audit 2021-03-10T13:49:57.546677+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:49:58 gibba032 bash[9829]: audit 2021-03-10T13:49:57.551457+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:49:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[9813]: cluster 2021-03-10T13:49:57.175126+0000 mgr.y (mgr.14140) 253 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 750 B/s rd, 4.1 KiB/s wr, 15 op/s 2021-03-10T13:49:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[13015]: cluster 2021-03-10T13:49:57.175126+0000 mgr.y (mgr.14140) 253 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 750 B/s rd, 4.1 KiB/s wr, 15 op/s 2021-03-10T13:49:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[9813]: audit 2021-03-10T13:49:57.546677+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[9813]: audit 2021-03-10T13:49:57.551457+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:49:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[13015]: audit 2021-03-10T13:49:57.546677+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]: dispatch 2021-03-10T13:49:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:49:58 gibba011 bash[13015]: audit 2021-03-10T13:49:57.551457+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"a\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544827\", \"created\": \"2021-03-10T13:42:43.644407\", \"started\": \"2021-03-10T13:42:49.012640\"}, \"mgr.y\": {\"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"y\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544937\", \"created\": \"2021-03-10T13:42:50.972407\", \"started\": \"2021-03-10T13:42:50.997585\"}, \"mon.c\": {\"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"c\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.544999\", \"created\": \"2021-03-10T13:43:58.520406\", \"started\": \"2021-03-10T13:43:58.546236\"}, \"osd.0\": {\"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"0\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545057\", \"created\": \"2021-03-10T13:44:46.556406\", \"started\": \"2021-03-10T13:44:49.809781\"}, \"osd.1\": {\"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"1\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545112\", \"created\": \"2021-03-10T13:45:09.440405\", \"started\": \"2021-03-10T13:45:12.629074\"}, \"osd.2\": {\"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"2\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545166\", \"created\": \"2021-03-10T13:45:32.562625\", \"started\": \"2021-03-10T13:45:35.953114\"}, \"osd.3\": {\"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"3\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545220\", \"created\": \"2021-03-10T13:45:56.793082\", \"started\": \"2021-03-10T13:45:59.870484\"}, \"node-exporter.a\": {\"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"a\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545274\", \"created\": \"2021-03-10T13:47:49.641081\", \"started\": \"2021-03-10T13:47:52.381825\"}, \"alertmanager.a\": {\"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"daemon_id\": \"a\", \"daemon_type\": \"alertmanager\", \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545327\", \"created\": \"2021-03-10T13:47:59.249081\", \"started\": \"2021-03-10T13:48:18.516422\"}, \"rgw.r.z.gibba011.xbljmx\": {\"hostname\": \"gibba011\", \"container_id\": \"6f9c0a2748ad\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"daemon_type\": \"rgw\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:49:57.545381\", \"created\": \"2021-03-10T13:49:55.661080\", \"started\": \"2021-03-10T13:49:55.686321\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\"}], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740\"}}, \"last_daemon_update\": \"2021-03-10T13:49:57.545507\", \"last_device_update\": \"2021-03-10T13:46:03.134728\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724\"}"}]': finished 2021-03-10T13:50:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:00 gibba032 bash[9829]: cluster 2021-03-10T13:49:59.176499+0000 mgr.y (mgr.14140) 254 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2021-03-10T13:50:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:00 gibba011 bash[9813]: cluster 2021-03-10T13:49:59.176499+0000 mgr.y (mgr.14140) 254 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2021-03-10T13:50:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:00 gibba011 bash[13015]: cluster 2021-03-10T13:49:59.176499+0000 mgr.y (mgr.14140) 254 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 6.9 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2021-03-10T13:50:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:02 gibba032 bash[9829]: cluster 2021-03-10T13:50:01.177893+0000 mgr.y (mgr.14140) 255 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2021-03-10T13:50:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:02 gibba011 bash[9813]: cluster 2021-03-10T13:50:01.177893+0000 mgr.y (mgr.14140) 255 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2021-03-10T13:50:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:02 gibba011 bash[13015]: cluster 2021-03-10T13:50:01.177893+0000 mgr.y (mgr.14140) 255 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2021-03-10T13:50:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:04 gibba032 bash[9829]: cluster 2021-03-10T13:50:03.179402+0000 mgr.y (mgr.14140) 256 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2021-03-10T13:50:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:04 gibba011 bash[9813]: cluster 2021-03-10T13:50:03.179402+0000 mgr.y (mgr.14140) 256 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2021-03-10T13:50:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:04 gibba011 bash[13015]: cluster 2021-03-10T13:50:03.179402+0000 mgr.y (mgr.14140) 256 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2021-03-10T13:50:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:06 gibba032 bash[9829]: cluster 2021-03-10T13:50:05.180923+0000 mgr.y (mgr.14140) 257 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2021-03-10T13:50:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:06 gibba011 bash[9813]: cluster 2021-03-10T13:50:05.180923+0000 mgr.y (mgr.14140) 257 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2021-03-10T13:50:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:06 gibba011 bash[13015]: cluster 2021-03-10T13:50:05.180923+0000 mgr.y (mgr.14140) 257 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2021-03-10T13:50:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:08 gibba032 bash[9829]: cluster 2021-03-10T13:50:07.182484+0000 mgr.y (mgr.14140) 258 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:08 gibba011 bash[9813]: cluster 2021-03-10T13:50:07.182484+0000 mgr.y (mgr.14140) 258 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:08 gibba011 bash[13015]: cluster 2021-03-10T13:50:07.182484+0000 mgr.y (mgr.14140) 258 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:10 gibba032 bash[9829]: cluster 2021-03-10T13:50:09.183348+0000 mgr.y (mgr.14140) 259 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:10 gibba011 bash[9813]: cluster 2021-03-10T13:50:09.183348+0000 mgr.y (mgr.14140) 259 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:10 gibba011 bash[13015]: cluster 2021-03-10T13:50:09.183348+0000 mgr.y (mgr.14140) 259 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:12 gibba032 bash[9829]: cluster 2021-03-10T13:50:11.184412+0000 mgr.y (mgr.14140) 260 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2021-03-10T13:50:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:12 gibba011 bash[9813]: cluster 2021-03-10T13:50:11.184412+0000 mgr.y (mgr.14140) 260 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2021-03-10T13:50:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:12 gibba011 bash[13015]: cluster 2021-03-10T13:50:11.184412+0000 mgr.y (mgr.14140) 260 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2021-03-10T13:50:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:14 gibba032 bash[9829]: cluster 2021-03-10T13:50:13.185811+0000 mgr.y (mgr.14140) 261 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.1 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:14 gibba011 bash[9813]: cluster 2021-03-10T13:50:13.185811+0000 mgr.y (mgr.14140) 261 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.1 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:14 gibba011 bash[13015]: cluster 2021-03-10T13:50:13.185811+0000 mgr.y (mgr.14140) 261 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 3.1 KiB/s rd, 85 B/s wr, 4 op/s 2021-03-10T13:50:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:16 gibba032 bash[9829]: cluster 2021-03-10T13:50:15.187402+0000 mgr.y (mgr.14140) 262 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:16 gibba011 bash[9813]: cluster 2021-03-10T13:50:15.187402+0000 mgr.y (mgr.14140) 262 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:16 gibba011 bash[13015]: cluster 2021-03-10T13:50:15.187402+0000 mgr.y (mgr.14140) 262 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:18 gibba032 bash[9829]: cluster 2021-03-10T13:50:17.189011+0000 mgr.y (mgr.14140) 263 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:18 gibba011 bash[9813]: cluster 2021-03-10T13:50:17.189011+0000 mgr.y (mgr.14140) 263 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:18 gibba011 bash[13015]: cluster 2021-03-10T13:50:17.189011+0000 mgr.y (mgr.14140) 263 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:20 gibba032 bash[9829]: cluster 2021-03-10T13:50:19.190409+0000 mgr.y (mgr.14140) 264 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:20 gibba011 bash[9813]: cluster 2021-03-10T13:50:19.190409+0000 mgr.y (mgr.14140) 264 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:20 gibba011 bash[13015]: cluster 2021-03-10T13:50:19.190409+0000 mgr.y (mgr.14140) 264 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:22 gibba011 bash[9813]: cluster 2021-03-10T13:50:21.191775+0000 mgr.y (mgr.14140) 265 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:22 gibba011 bash[13015]: cluster 2021-03-10T13:50:21.191775+0000 mgr.y (mgr.14140) 265 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:22 gibba032 bash[9829]: cluster 2021-03-10T13:50:21.191775+0000 mgr.y (mgr.14140) 265 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:24 gibba011 bash[9813]: cluster 2021-03-10T13:50:23.193370+0000 mgr.y (mgr.14140) 266 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:24 gibba011 bash[13015]: cluster 2021-03-10T13:50:23.193370+0000 mgr.y (mgr.14140) 266 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:24 gibba032 bash[9829]: cluster 2021-03-10T13:50:23.193370+0000 mgr.y (mgr.14140) 266 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:26 gibba011 bash[9813]: cluster 2021-03-10T13:50:25.194514+0000 mgr.y (mgr.14140) 267 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:26 gibba011 bash[13015]: cluster 2021-03-10T13:50:25.194514+0000 mgr.y (mgr.14140) 267 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:26 gibba032 bash[9829]: cluster 2021-03-10T13:50:25.194514+0000 mgr.y (mgr.14140) 267 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:28 gibba011 bash[9813]: cluster 2021-03-10T13:50:27.195038+0000 mgr.y (mgr.14140) 268 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:28.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:28 gibba011 bash[13015]: cluster 2021-03-10T13:50:27.195038+0000 mgr.y (mgr.14140) 268 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:28 gibba032 bash[9829]: cluster 2021-03-10T13:50:27.195038+0000 mgr.y (mgr.14140) 268 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:30 gibba011 bash[9813]: cluster 2021-03-10T13:50:29.196170+0000 mgr.y (mgr.14140) 269 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:30 gibba011 bash[13015]: cluster 2021-03-10T13:50:29.196170+0000 mgr.y (mgr.14140) 269 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:30 gibba032 bash[9829]: cluster 2021-03-10T13:50:29.196170+0000 mgr.y (mgr.14140) 269 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:32 gibba011 bash[9813]: cluster 2021-03-10T13:50:31.197444+0000 mgr.y (mgr.14140) 270 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:32 gibba011 bash[13015]: cluster 2021-03-10T13:50:31.197444+0000 mgr.y (mgr.14140) 270 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:32 gibba032 bash[9829]: cluster 2021-03-10T13:50:31.197444+0000 mgr.y (mgr.14140) 270 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:34 gibba011 bash[9813]: cluster 2021-03-10T13:50:33.199188+0000 mgr.y (mgr.14140) 271 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:34 gibba011 bash[13015]: cluster 2021-03-10T13:50:33.199188+0000 mgr.y (mgr.14140) 271 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:34 gibba032 bash[9829]: cluster 2021-03-10T13:50:33.199188+0000 mgr.y (mgr.14140) 271 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:36 gibba011 bash[9813]: cluster 2021-03-10T13:50:35.200447+0000 mgr.y (mgr.14140) 272 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:36.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:36 gibba011 bash[13015]: cluster 2021-03-10T13:50:35.200447+0000 mgr.y (mgr.14140) 272 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:36 gibba032 bash[9829]: cluster 2021-03-10T13:50:35.200447+0000 mgr.y (mgr.14140) 272 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:38 gibba032 bash[9829]: cluster 2021-03-10T13:50:37.202064+0000 mgr.y (mgr.14140) 273 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:38 gibba011 bash[9813]: cluster 2021-03-10T13:50:37.202064+0000 mgr.y (mgr.14140) 273 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:38 gibba011 bash[13015]: cluster 2021-03-10T13:50:37.202064+0000 mgr.y (mgr.14140) 273 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:40 gibba032 bash[9829]: cluster 2021-03-10T13:50:39.203595+0000 mgr.y (mgr.14140) 274 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:40 gibba011 bash[9813]: cluster 2021-03-10T13:50:39.203595+0000 mgr.y (mgr.14140) 274 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:40 gibba011 bash[13015]: cluster 2021-03-10T13:50:39.203595+0000 mgr.y (mgr.14140) 274 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:42.857 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:42 gibba011 bash[9813]: cluster 2021-03-10T13:50:41.204819+0000 mgr.y (mgr.14140) 275 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:42.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:42 gibba011 bash[13015]: cluster 2021-03-10T13:50:41.204819+0000 mgr.y (mgr.14140) 275 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:42 gibba032 bash[9829]: cluster 2021-03-10T13:50:41.204819+0000 mgr.y (mgr.14140) 275 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:44 gibba032 bash[9829]: cluster 2021-03-10T13:50:43.206545+0000 mgr.y (mgr.14140) 276 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:44 gibba011 bash[9813]: cluster 2021-03-10T13:50:43.206545+0000 mgr.y (mgr.14140) 276 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:44 gibba011 bash[13015]: cluster 2021-03-10T13:50:43.206545+0000 mgr.y (mgr.14140) 276 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:46 gibba032 bash[9829]: cluster 2021-03-10T13:50:45.208174+0000 mgr.y (mgr.14140) 277 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:46 gibba011 bash[9813]: cluster 2021-03-10T13:50:45.208174+0000 mgr.y (mgr.14140) 277 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:46 gibba011 bash[13015]: cluster 2021-03-10T13:50:45.208174+0000 mgr.y (mgr.14140) 277 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2021-03-10T13:50:48.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:48 gibba032 bash[9829]: cluster 2021-03-10T13:50:47.209874+0000 mgr.y (mgr.14140) 278 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:48 gibba011 bash[9813]: cluster 2021-03-10T13:50:47.209874+0000 mgr.y (mgr.14140) 278 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:48 gibba011 bash[13015]: cluster 2021-03-10T13:50:47.209874+0000 mgr.y (mgr.14140) 278 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:50 gibba032 bash[9829]: cluster 2021-03-10T13:50:49.211417+0000 mgr.y (mgr.14140) 279 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:50 gibba011 bash[9813]: cluster 2021-03-10T13:50:49.211417+0000 mgr.y (mgr.14140) 279 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:50 gibba011 bash[13015]: cluster 2021-03-10T13:50:49.211417+0000 mgr.y (mgr.14140) 279 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:52 gibba032 bash[9829]: cluster 2021-03-10T13:50:51.212902+0000 mgr.y (mgr.14140) 280 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:52 gibba011 bash[9813]: cluster 2021-03-10T13:50:51.212902+0000 mgr.y (mgr.14140) 280 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:52 gibba011 bash[13015]: cluster 2021-03-10T13:50:51.212902+0000 mgr.y (mgr.14140) 280 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:54.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:54 gibba032 bash[9829]: cluster 2021-03-10T13:50:53.214363+0000 mgr.y (mgr.14140) 281 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:54 gibba011 bash[9813]: cluster 2021-03-10T13:50:53.214363+0000 mgr.y (mgr.14140) 281 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:54 gibba011 bash[13015]: cluster 2021-03-10T13:50:53.214363+0000 mgr.y (mgr.14140) 281 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:50:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:56 gibba011 bash[9813]: cluster 2021-03-10T13:50:55.215545+0000 mgr.y (mgr.14140) 282 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:56 gibba011 bash[13015]: cluster 2021-03-10T13:50:55.215545+0000 mgr.y (mgr.14140) 282 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:56 gibba032 bash[9829]: cluster 2021-03-10T13:50:55.215545+0000 mgr.y (mgr.14140) 282 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:50:58 gibba011 bash[9813]: cluster 2021-03-10T13:50:57.216959+0000 mgr.y (mgr.14140) 283 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:50:58 gibba011 bash[13015]: cluster 2021-03-10T13:50:57.216959+0000 mgr.y (mgr.14140) 283 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:50:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:50:58 gibba032 bash[9829]: cluster 2021-03-10T13:50:57.216959+0000 mgr.y (mgr.14140) 283 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:00 gibba011 bash[9813]: cluster 2021-03-10T13:50:59.218105+0000 mgr.y (mgr.14140) 284 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:00 gibba011 bash[13015]: cluster 2021-03-10T13:50:59.218105+0000 mgr.y (mgr.14140) 284 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:00 gibba032 bash[9829]: cluster 2021-03-10T13:50:59.218105+0000 mgr.y (mgr.14140) 284 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:02 gibba011 bash[9813]: cluster 2021-03-10T13:51:01.219468+0000 mgr.y (mgr.14140) 285 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:02 gibba011 bash[13015]: cluster 2021-03-10T13:51:01.219468+0000 mgr.y (mgr.14140) 285 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:02 gibba032 bash[9829]: cluster 2021-03-10T13:51:01.219468+0000 mgr.y (mgr.14140) 285 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:04 gibba011 bash[9813]: cluster 2021-03-10T13:51:03.221041+0000 mgr.y (mgr.14140) 286 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:04 gibba011 bash[13015]: cluster 2021-03-10T13:51:03.221041+0000 mgr.y (mgr.14140) 286 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:04 gibba032 bash[9829]: cluster 2021-03-10T13:51:03.221041+0000 mgr.y (mgr.14140) 286 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:06 gibba032 bash[9829]: cluster 2021-03-10T13:51:05.222159+0000 mgr.y (mgr.14140) 287 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:06 gibba011 bash[9813]: cluster 2021-03-10T13:51:05.222159+0000 mgr.y (mgr.14140) 287 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:06 gibba011 bash[13015]: cluster 2021-03-10T13:51:05.222159+0000 mgr.y (mgr.14140) 287 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:08.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:08 gibba032 bash[9829]: cluster 2021-03-10T13:51:07.223646+0000 mgr.y (mgr.14140) 288 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:08 gibba011 bash[9813]: cluster 2021-03-10T13:51:07.223646+0000 mgr.y (mgr.14140) 288 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:08 gibba011 bash[13015]: cluster 2021-03-10T13:51:07.223646+0000 mgr.y (mgr.14140) 288 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:10.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:10 gibba032 bash[9829]: cluster 2021-03-10T13:51:09.224952+0000 mgr.y (mgr.14140) 289 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:10 gibba011 bash[9813]: cluster 2021-03-10T13:51:09.224952+0000 mgr.y (mgr.14140) 289 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:10 gibba011 bash[13015]: cluster 2021-03-10T13:51:09.224952+0000 mgr.y (mgr.14140) 289 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:12 gibba011 bash[9813]: cluster 2021-03-10T13:51:11.226029+0000 mgr.y (mgr.14140) 290 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:12 gibba011 bash[13015]: cluster 2021-03-10T13:51:11.226029+0000 mgr.y (mgr.14140) 290 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:12 gibba032 bash[9829]: cluster 2021-03-10T13:51:11.226029+0000 mgr.y (mgr.14140) 290 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:14 gibba011 bash[9813]: cluster 2021-03-10T13:51:13.227652+0000 mgr.y (mgr.14140) 291 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:14 gibba011 bash[13015]: cluster 2021-03-10T13:51:13.227652+0000 mgr.y (mgr.14140) 291 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:14 gibba032 bash[9829]: cluster 2021-03-10T13:51:13.227652+0000 mgr.y (mgr.14140) 291 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:16 gibba032 bash[9829]: cluster 2021-03-10T13:51:15.228799+0000 mgr.y (mgr.14140) 292 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:16 gibba011 bash[9813]: cluster 2021-03-10T13:51:15.228799+0000 mgr.y (mgr.14140) 292 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:16 gibba011 bash[13015]: cluster 2021-03-10T13:51:15.228799+0000 mgr.y (mgr.14140) 292 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:18 gibba032 bash[9829]: cluster 2021-03-10T13:51:17.230337+0000 mgr.y (mgr.14140) 293 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:18 gibba011 bash[9813]: cluster 2021-03-10T13:51:17.230337+0000 mgr.y (mgr.14140) 293 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:18 gibba011 bash[13015]: cluster 2021-03-10T13:51:17.230337+0000 mgr.y (mgr.14140) 293 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:20 gibba011 bash[9813]: cluster 2021-03-10T13:51:19.231964+0000 mgr.y (mgr.14140) 294 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:20 gibba011 bash[13015]: cluster 2021-03-10T13:51:19.231964+0000 mgr.y (mgr.14140) 294 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:20 gibba032 bash[9829]: cluster 2021-03-10T13:51:19.231964+0000 mgr.y (mgr.14140) 294 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:22 gibba011 bash[9813]: cluster 2021-03-10T13:51:21.233287+0000 mgr.y (mgr.14140) 295 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:22 gibba011 bash[13015]: cluster 2021-03-10T13:51:21.233287+0000 mgr.y (mgr.14140) 295 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:22 gibba032 bash[9829]: cluster 2021-03-10T13:51:21.233287+0000 mgr.y (mgr.14140) 295 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:24 gibba011 bash[9813]: cluster 2021-03-10T13:51:23.234924+0000 mgr.y (mgr.14140) 296 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:24 gibba011 bash[13015]: cluster 2021-03-10T13:51:23.234924+0000 mgr.y (mgr.14140) 296 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:24 gibba032 bash[9829]: cluster 2021-03-10T13:51:23.234924+0000 mgr.y (mgr.14140) 296 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:26 gibba032 bash[9829]: cluster 2021-03-10T13:51:25.236308+0000 mgr.y (mgr.14140) 297 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:26.540 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:26 gibba011 bash[9813]: cluster 2021-03-10T13:51:25.236308+0000 mgr.y (mgr.14140) 297 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:26.540 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:26 gibba011 bash[13015]: cluster 2021-03-10T13:51:25.236308+0000 mgr.y (mgr.14140) 297 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:28 gibba011 bash[9813]: cluster 2021-03-10T13:51:27.237667+0000 mgr.y (mgr.14140) 298 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:28 gibba011 bash[13015]: cluster 2021-03-10T13:51:27.237667+0000 mgr.y (mgr.14140) 298 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:28 gibba032 bash[9829]: cluster 2021-03-10T13:51:27.237667+0000 mgr.y (mgr.14140) 298 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:30 gibba011 bash[9813]: cluster 2021-03-10T13:51:29.239302+0000 mgr.y (mgr.14140) 299 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:30 gibba011 bash[13015]: cluster 2021-03-10T13:51:29.239302+0000 mgr.y (mgr.14140) 299 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:30 gibba032 bash[9829]: cluster 2021-03-10T13:51:29.239302+0000 mgr.y (mgr.14140) 299 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:32 gibba011 bash[9813]: cluster 2021-03-10T13:51:31.240371+0000 mgr.y (mgr.14140) 300 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:32 gibba011 bash[13015]: cluster 2021-03-10T13:51:31.240371+0000 mgr.y (mgr.14140) 300 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:32 gibba032 bash[9829]: cluster 2021-03-10T13:51:31.240371+0000 mgr.y (mgr.14140) 300 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 2 op/s 2021-03-10T13:51:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:34 gibba011 bash[9813]: cluster 2021-03-10T13:51:33.241966+0000 mgr.y (mgr.14140) 301 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:34 gibba011 bash[13015]: cluster 2021-03-10T13:51:33.241966+0000 mgr.y (mgr.14140) 301 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:34 gibba032 bash[9829]: cluster 2021-03-10T13:51:33.241966+0000 mgr.y (mgr.14140) 301 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:36 gibba032 bash[9829]: cluster 2021-03-10T13:51:35.242486+0000 mgr.y (mgr.14140) 302 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:36 gibba011 bash[9813]: cluster 2021-03-10T13:51:35.242486+0000 mgr.y (mgr.14140) 302 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:36 gibba011 bash[13015]: cluster 2021-03-10T13:51:35.242486+0000 mgr.y (mgr.14140) 302 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:38 gibba011 bash[9813]: cluster 2021-03-10T13:51:37.244041+0000 mgr.y (mgr.14140) 303 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:38 gibba011 bash[13015]: cluster 2021-03-10T13:51:37.244041+0000 mgr.y (mgr.14140) 303 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:38 gibba032 bash[9829]: cluster 2021-03-10T13:51:37.244041+0000 mgr.y (mgr.14140) 303 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:40 gibba011 bash[9813]: cluster 2021-03-10T13:51:39.245656+0000 mgr.y (mgr.14140) 304 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:40 gibba011 bash[13015]: cluster 2021-03-10T13:51:39.245656+0000 mgr.y (mgr.14140) 304 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:40 gibba032 bash[9829]: cluster 2021-03-10T13:51:39.245656+0000 mgr.y (mgr.14140) 304 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2021-03-10T13:51:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:42 gibba011 bash[9813]: cluster 2021-03-10T13:51:41.246970+0000 mgr.y (mgr.14140) 305 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:42 gibba011 bash[13015]: cluster 2021-03-10T13:51:41.246970+0000 mgr.y (mgr.14140) 305 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:42 gibba032 bash[9829]: cluster 2021-03-10T13:51:41.246970+0000 mgr.y (mgr.14140) 305 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2021-03-10T13:51:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:44 gibba011 bash[9813]: cluster 2021-03-10T13:51:43.248420+0000 mgr.y (mgr.14140) 306 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2021-03-10T13:51:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:44 gibba011 bash[13015]: cluster 2021-03-10T13:51:43.248420+0000 mgr.y (mgr.14140) 306 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2021-03-10T13:51:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:44 gibba032 bash[9829]: cluster 2021-03-10T13:51:43.248420+0000 mgr.y (mgr.14140) 306 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2021-03-10T13:51:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:46 gibba032 bash[9829]: cluster 2021-03-10T13:51:45.249042+0000 mgr.y (mgr.14140) 307 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:46 gibba011 bash[9813]: cluster 2021-03-10T13:51:45.249042+0000 mgr.y (mgr.14140) 307 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:46 gibba011 bash[13015]: cluster 2021-03-10T13:51:45.249042+0000 mgr.y (mgr.14140) 307 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:48 gibba011 bash[9813]: cluster 2021-03-10T13:51:47.250439+0000 mgr.y (mgr.14140) 308 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:48 gibba011 bash[13015]: cluster 2021-03-10T13:51:47.250439+0000 mgr.y (mgr.14140) 308 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:48 gibba032 bash[9829]: cluster 2021-03-10T13:51:47.250439+0000 mgr.y (mgr.14140) 308 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:50 gibba011 bash[9813]: cluster 2021-03-10T13:51:49.251823+0000 mgr.y (mgr.14140) 309 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:50 gibba011 bash[13015]: cluster 2021-03-10T13:51:49.251823+0000 mgr.y (mgr.14140) 309 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:50 gibba032 bash[9829]: cluster 2021-03-10T13:51:49.251823+0000 mgr.y (mgr.14140) 309 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:52 gibba011 bash[9813]: cluster 2021-03-10T13:51:51.253262+0000 mgr.y (mgr.14140) 310 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:52 gibba011 bash[13015]: cluster 2021-03-10T13:51:51.253262+0000 mgr.y (mgr.14140) 310 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:52 gibba032 bash[9829]: cluster 2021-03-10T13:51:51.253262+0000 mgr.y (mgr.14140) 310 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:54 gibba011 bash[9813]: cluster 2021-03-10T13:51:53.254711+0000 mgr.y (mgr.14140) 311 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:54 gibba011 bash[13015]: cluster 2021-03-10T13:51:53.254711+0000 mgr.y (mgr.14140) 311 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:54 gibba032 bash[9829]: cluster 2021-03-10T13:51:53.254711+0000 mgr.y (mgr.14140) 311 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:56 gibba011 bash[9813]: cluster 2021-03-10T13:51:55.255187+0000 mgr.y (mgr.14140) 312 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:56 gibba011 bash[13015]: cluster 2021-03-10T13:51:55.255187+0000 mgr.y (mgr.14140) 312 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:56 gibba032 bash[9829]: cluster 2021-03-10T13:51:55.255187+0000 mgr.y (mgr.14140) 312 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:58.448 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2021-03-10T13:51:58.531 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:51:58 gibba011 bash[9813]: cluster 2021-03-10T13:51:57.256640+0000 mgr.y (mgr.14140) 313 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:58.532 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:51:58 gibba011 bash[13015]: cluster 2021-03-10T13:51:57.256640+0000 mgr.y (mgr.14140) 313 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:51:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:51:58 gibba032 bash[9829]: cluster 2021-03-10T13:51:57.256640+0000 mgr.y (mgr.14140) 313 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:00 gibba011 bash[9813]: cluster 2021-03-10T13:51:59.257958+0000 mgr.y (mgr.14140) 314 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:00 gibba011 bash[13015]: cluster 2021-03-10T13:51:59.257958+0000 mgr.y (mgr.14140) 314 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:00 gibba032 bash[9829]: cluster 2021-03-10T13:51:59.257958+0000 mgr.y (mgr.14140) 314 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:01.390 INFO:teuthology.orchestra.run.gibba011.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:52:02.015 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-03-10T13:52:02.024 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@gibba011.front.sepia.ceph.com 2021-03-10T13:52:02.025 DEBUG:teuthology.orchestra.run.gibba011:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6fd36d94-81a6-11eb-9080-001a4aab830c -e sha1=b26fc080937a021ce826cdc94d1adb52c064381a -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done' 2021-03-10T13:52:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[9813]: cluster 2021-03-10T13:52:01.258387+0000 mgr.y (mgr.14140) 315 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[9813]: audit 2021-03-10T13:52:01.383869+0000 mgr.y (mgr.14140) 316 : audit [DBG] from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[9813]: audit 2021-03-10T13:52:01.384505+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]: dispatch 2021-03-10T13:52:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[9813]: audit 2021-03-10T13:52:01.388995+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]': finished 2021-03-10T13:52:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[9813]: cephadm 2021-03-10T13:52:01.391220+0000 mgr.y (mgr.14140) 317 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:52:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[13015]: cluster 2021-03-10T13:52:01.258387+0000 mgr.y (mgr.14140) 315 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[13015]: audit 2021-03-10T13:52:01.383869+0000 mgr.y (mgr.14140) 316 : audit [DBG] from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[13015]: audit 2021-03-10T13:52:01.384505+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]: dispatch 2021-03-10T13:52:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[13015]: audit 2021-03-10T13:52:01.388995+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]': finished 2021-03-10T13:52:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:02 gibba011 bash[13015]: cephadm 2021-03-10T13:52:01.391220+0000 mgr.y (mgr.14140) 317 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:52:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:02 gibba032 bash[9829]: cluster 2021-03-10T13:52:01.258387+0000 mgr.y (mgr.14140) 315 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:02 gibba032 bash[9829]: audit 2021-03-10T13:52:01.383869+0000 mgr.y (mgr.14140) 316 : audit [DBG] from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:02 gibba032 bash[9829]: audit 2021-03-10T13:52:01.384505+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]: dispatch 2021-03-10T13:52:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:02 gibba032 bash[9829]: audit 2021-03-10T13:52:01.388995+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\"}"}]': finished 2021-03-10T13:52:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:02 gibba032 bash[9829]: cephadm 2021-03-10T13:52:01.391220+0000 mgr.y (mgr.14140) 317 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:52:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:04 gibba011 bash[9813]: cluster 2021-03-10T13:52:03.258796+0000 mgr.y (mgr.14140) 318 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:04 gibba011 bash[13015]: cluster 2021-03-10T13:52:03.258796+0000 mgr.y (mgr.14140) 318 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:04 gibba032 bash[9829]: cluster 2021-03-10T13:52:03.258796+0000 mgr.y (mgr.14140) 318 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:06 gibba011 bash[9813]: cluster 2021-03-10T13:52:05.259150+0000 mgr.y (mgr.14140) 319 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:06 gibba011 bash[13015]: cluster 2021-03-10T13:52:05.259150+0000 mgr.y (mgr.14140) 319 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:06 gibba032 bash[9829]: cluster 2021-03-10T13:52:05.259150+0000 mgr.y (mgr.14140) 319 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:08 gibba011 bash[9813]: cluster 2021-03-10T13:52:07.259479+0000 mgr.y (mgr.14140) 320 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:08 gibba011 bash[13015]: cluster 2021-03-10T13:52:07.259479+0000 mgr.y (mgr.14140) 320 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:08 gibba032 bash[9829]: cluster 2021-03-10T13:52:07.259479+0000 mgr.y (mgr.14140) 320 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:10 gibba011 bash[9813]: cluster 2021-03-10T13:52:09.260666+0000 mgr.y (mgr.14140) 321 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:10 gibba011 bash[13015]: cluster 2021-03-10T13:52:09.260666+0000 mgr.y (mgr.14140) 321 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:10 gibba032 bash[9829]: cluster 2021-03-10T13:52:09.260666+0000 mgr.y (mgr.14140) 321 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:12 gibba011 bash[9813]: cluster 2021-03-10T13:52:11.261822+0000 mgr.y (mgr.14140) 322 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:12 gibba011 bash[13015]: cluster 2021-03-10T13:52:11.261822+0000 mgr.y (mgr.14140) 322 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:12 gibba032 bash[9829]: cluster 2021-03-10T13:52:11.261822+0000 mgr.y (mgr.14140) 322 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:14 gibba011 bash[9813]: cluster 2021-03-10T13:52:13.263151+0000 mgr.y (mgr.14140) 323 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:14 gibba011 bash[13015]: cluster 2021-03-10T13:52:13.263151+0000 mgr.y (mgr.14140) 323 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:14 gibba032 bash[9829]: cluster 2021-03-10T13:52:13.263151+0000 mgr.y (mgr.14140) 323 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:15.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[9813]: audit 2021-03-10T13:52:15.133936+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]: dispatch 2021-03-10T13:52:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[9813]: audit 2021-03-10T13:52:15.137783+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]': finished 2021-03-10T13:52:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[9813]: audit 2021-03-10T13:52:15.138229+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:15.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[13015]: audit 2021-03-10T13:52:15.133936+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]: dispatch 2021-03-10T13:52:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[13015]: audit 2021-03-10T13:52:15.137783+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]': finished 2021-03-10T13:52:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:15 gibba011 bash[13015]: audit 2021-03-10T13:52:15.138229+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:15 gibba032 bash[9829]: audit 2021-03-10T13:52:15.133936+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]: dispatch 2021-03-10T13:52:15.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:15 gibba032 bash[9829]: audit 2021-03-10T13:52:15.137783+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_version\": \"ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)\"}"}]': finished 2021-03-10T13:52:15.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:15 gibba032 bash[9829]: audit 2021-03-10T13:52:15.138229+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:16.395 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:52:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[9813]: cephadm 2021-03-10T13:52:15.138015+0000 mgr.y (mgr.14140) 324 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[9813]: cephadm 2021-03-10T13:52:15.138866+0000 mgr.y (mgr.14140) 325 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[9813]: cephadm 2021-03-10T13:52:15.138940+0000 mgr.y (mgr.14140) 326 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[9813]: cephadm 2021-03-10T13:52:15.260030+0000 mgr.y (mgr.14140) 327 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a on gibba032 2021-03-10T13:52:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[9813]: cluster 2021-03-10T13:52:15.263754+0000 mgr.y (mgr.14140) 328 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[13015]: cephadm 2021-03-10T13:52:15.138015+0000 mgr.y (mgr.14140) 324 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[13015]: cephadm 2021-03-10T13:52:15.138866+0000 mgr.y (mgr.14140) 325 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[13015]: cephadm 2021-03-10T13:52:15.138940+0000 mgr.y (mgr.14140) 326 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[13015]: cephadm 2021-03-10T13:52:15.260030+0000 mgr.y (mgr.14140) 327 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a on gibba032 2021-03-10T13:52:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:16 gibba011 bash[13015]: cluster 2021-03-10T13:52:15.263754+0000 mgr.y (mgr.14140) 328 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:52:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3m) 2m ago 4m 0.21.0 prom/alertmanager c876f5897d7b 4b2fdb052cac 2021-03-10T13:52:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (4m) 3m ago 4m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:52:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (7m) 3m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 aeae5c17250e 2021-03-10T13:52:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (7m) 3m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 c0962a4e2967 2021-03-10T13:52:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (9m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 9865ee42339d 2021-03-10T13:52:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (9m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cdbdd153dd01 2021-03-10T13:52:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (8m) 3m ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b1f2ea2c99b4 2021-03-10T13:52:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (8m) 2m ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 d494ae59299f 2021-03-10T13:52:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (4m) 2m ago 4m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:52:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (4m) 3m ago 4m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:52:16.590 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (7m) 2m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:52:16.590 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (7m) 2m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:52:16.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (6m) 2m ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:52:16.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (6m) 2m ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:52:16.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (5m) 3m ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:52:16.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (5m) 3m ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:52:16.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (5m) 3m ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:52:16.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (4m) 3m ago 4m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:52:16.593 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3m) 3m ago 4m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:52:16.594 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 running (2m) 2m ago 2m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 6f9c0a2748ad 2021-03-10T13:52:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:16 gibba032 bash[9829]: cephadm 2021-03-10T13:52:15.138015+0000 mgr.y (mgr.14140) 324 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:16 gibba032 bash[9829]: cephadm 2021-03-10T13:52:15.138866+0000 mgr.y (mgr.14140) 325 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:16 gibba032 bash[9829]: cephadm 2021-03-10T13:52:15.138940+0000 mgr.y (mgr.14140) 326 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:16 gibba032 bash[9829]: cephadm 2021-03-10T13:52:15.260030+0000 mgr.y (mgr.14140) 327 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a on gibba032 2021-03-10T13:52:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:16 gibba032 bash[9829]: cluster 2021-03-10T13:52:15.263754+0000 mgr.y (mgr.14140) 328 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:16.809 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:52:16.809 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:52:16.809 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:52:16.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:16.810 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:52:16.810 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:52:16.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:16.810 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:52:16.811 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:52:16.811 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:16.811 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:52:16.811 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:52:16.811 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 14 2021-03-10T13:52:16.812 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:52:16.812 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:52:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[9813]: audit 2021-03-10T13:52:16.386709+0000 mgr.y (mgr.14140) 329 : audit [DBG] from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[9813]: audit 2021-03-10T13:52:16.582019+0000 mgr.y (mgr.14140) 330 : audit [DBG] from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[9813]: audit 2021-03-10T13:52:16.808842+0000 mon.a (mon.0) 576 : audit [DBG] from='client.? 172.21.2.111:0/1238982344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:17.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[13015]: audit 2021-03-10T13:52:16.386709+0000 mgr.y (mgr.14140) 329 : audit [DBG] from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[13015]: audit 2021-03-10T13:52:16.582019+0000 mgr.y (mgr.14140) 330 : audit [DBG] from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:17 gibba011 bash[13015]: audit 2021-03-10T13:52:16.808842+0000 mon.a (mon.0) 576 : audit [DBG] from='client.? 172.21.2.111:0/1238982344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:17 gibba032 bash[9829]: audit 2021-03-10T13:52:16.386709+0000 mgr.y (mgr.14140) 329 : audit [DBG] from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:17 gibba032 bash[9829]: audit 2021-03-10T13:52:16.582019+0000 mgr.y (mgr.14140) 330 : audit [DBG] from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:17 gibba032 bash[9829]: audit 2021-03-10T13:52:16.808842+0000 mon.a (mon.0) 576 : audit [DBG] from='client.? 172.21.2.111:0/1238982344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:18 gibba011 bash[9813]: cluster 2021-03-10T13:52:17.264295+0000 mgr.y (mgr.14140) 331 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:18 gibba011 bash[13015]: cluster 2021-03-10T13:52:17.264295+0000 mgr.y (mgr.14140) 331 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:18 gibba032 bash[9829]: cluster 2021-03-10T13:52:17.264295+0000 mgr.y (mgr.14140) 331 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:20 gibba011 bash[9813]: cluster 2021-03-10T13:52:19.265517+0000 mgr.y (mgr.14140) 332 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:20 gibba011 bash[13015]: cluster 2021-03-10T13:52:19.265517+0000 mgr.y (mgr.14140) 332 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:20 gibba032 bash[9829]: cluster 2021-03-10T13:52:19.265517+0000 mgr.y (mgr.14140) 332 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:22 gibba011 bash[9813]: cluster 2021-03-10T13:52:21.266837+0000 mgr.y (mgr.14140) 333 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:22 gibba011 bash[13015]: cluster 2021-03-10T13:52:21.266837+0000 mgr.y (mgr.14140) 333 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:22 gibba032 bash[9829]: cluster 2021-03-10T13:52:21.266837+0000 mgr.y (mgr.14140) 333 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:24 gibba011 bash[9813]: cluster 2021-03-10T13:52:23.268351+0000 mgr.y (mgr.14140) 334 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:24 gibba011 bash[13015]: cluster 2021-03-10T13:52:23.268351+0000 mgr.y (mgr.14140) 334 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:24 gibba032 bash[9829]: cluster 2021-03-10T13:52:23.268351+0000 mgr.y (mgr.14140) 334 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:26 gibba011 bash[9813]: cluster 2021-03-10T13:52:25.269466+0000 mgr.y (mgr.14140) 335 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:26 gibba011 bash[13015]: cluster 2021-03-10T13:52:25.269466+0000 mgr.y (mgr.14140) 335 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:26 gibba032 bash[9829]: cluster 2021-03-10T13:52:25.269466+0000 mgr.y (mgr.14140) 335 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:28 gibba011 bash[9813]: cluster 2021-03-10T13:52:27.270728+0000 mgr.y (mgr.14140) 336 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:28 gibba011 bash[13015]: cluster 2021-03-10T13:52:27.270728+0000 mgr.y (mgr.14140) 336 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:28 gibba032 bash[9829]: cluster 2021-03-10T13:52:27.270728+0000 mgr.y (mgr.14140) 336 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: cluster 2021-03-10T13:52:29.272331+0000 mgr.y (mgr.14140) 337 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: audit 2021-03-10T13:52:30.172588+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]: dispatch 2021-03-10T13:52:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: audit 2021-03-10T13:52:30.182218+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]': finished 2021-03-10T13:52:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: audit 2021-03-10T13:52:30.183344+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.x"}]: dispatch 2021-03-10T13:52:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: audit 2021-03-10T13:52:30.184362+0000 mon.a (mon.0) 580 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[13015]: audit 2021-03-10T13:52:30.185148+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:52:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: cluster 2021-03-10T13:52:29.272331+0000 mgr.y (mgr.14140) 337 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: audit 2021-03-10T13:52:30.172588+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]: dispatch 2021-03-10T13:52:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: audit 2021-03-10T13:52:30.182218+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]': finished 2021-03-10T13:52:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: audit 2021-03-10T13:52:30.183344+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.x"}]: dispatch 2021-03-10T13:52:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: audit 2021-03-10T13:52:30.184362+0000 mon.a (mon.0) 580 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:30 gibba011 bash[9813]: audit 2021-03-10T13:52:30.185148+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:52:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: cluster 2021-03-10T13:52:29.272331+0000 mgr.y (mgr.14140) 337 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: audit 2021-03-10T13:52:30.172588+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]: dispatch 2021-03-10T13:52:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: audit 2021-03-10T13:52:30.182218+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.x"}]': finished 2021-03-10T13:52:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: audit 2021-03-10T13:52:30.183344+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.x"}]: dispatch 2021-03-10T13:52:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: audit 2021-03-10T13:52:30.184362+0000 mon.a (mon.0) 580 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:30 gibba032 bash[9829]: audit 2021-03-10T13:52:30.185148+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2021-03-10T13:52:31.466 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:31 gibba032 bash[9829]: cephadm 2021-03-10T13:52:30.171421+0000 mgr.y (mgr.14140) 338 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.x 2021-03-10T13:52:31.467 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:31 gibba032 bash[9829]: cephadm 2021-03-10T13:52:30.171649+0000 mgr.y (mgr.14140) 339 : cephadm [INF] Upgrade: Redeploying mgr.x 2021-03-10T13:52:31.467 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:31 gibba032 bash[9829]: cephadm 2021-03-10T13:52:30.184975+0000 mgr.y (mgr.14140) 340 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:52:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[9813]: cephadm 2021-03-10T13:52:30.171421+0000 mgr.y (mgr.14140) 338 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.x 2021-03-10T13:52:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[9813]: cephadm 2021-03-10T13:52:30.171649+0000 mgr.y (mgr.14140) 339 : cephadm [INF] Upgrade: Redeploying mgr.x 2021-03-10T13:52:31.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[9813]: cephadm 2021-03-10T13:52:30.184975+0000 mgr.y (mgr.14140) 340 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:52:31.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[13015]: cephadm 2021-03-10T13:52:30.171421+0000 mgr.y (mgr.14140) 338 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.x 2021-03-10T13:52:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[13015]: cephadm 2021-03-10T13:52:30.171649+0000 mgr.y (mgr.14140) 339 : cephadm [INF] Upgrade: Redeploying mgr.x 2021-03-10T13:52:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:31 gibba011 bash[13015]: cephadm 2021-03-10T13:52:30.184975+0000 mgr.y (mgr.14140) 340 : cephadm [INF] Deploying daemon mgr.x on gibba032 2021-03-10T13:52:32.316 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:32 gibba032 systemd[1]: Stopping Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:52:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:32 gibba011 bash[9813]: cluster 2021-03-10T13:52:31.273471+0000 mgr.y (mgr.14140) 341 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:32 gibba011 bash[13015]: cluster 2021-03-10T13:52:31.273471+0000 mgr.y (mgr.14140) 341 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:32 gibba032 bash[9829]: cluster 2021-03-10T13:52:31.273471+0000 mgr.y (mgr.14140) 341 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:34 gibba011 bash[9813]: cluster 2021-03-10T13:52:33.274838+0000 mgr.y (mgr.14140) 342 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:34 gibba011 bash[13015]: cluster 2021-03-10T13:52:33.274838+0000 mgr.y (mgr.14140) 342 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:34 gibba032 bash[9829]: cluster 2021-03-10T13:52:33.274838+0000 mgr.y (mgr.14140) 342 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:36 gibba011 bash[9813]: cluster 2021-03-10T13:52:35.275790+0000 mgr.y (mgr.14140) 343 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:36 gibba011 bash[13015]: cluster 2021-03-10T13:52:35.275790+0000 mgr.y (mgr.14140) 343 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:36 gibba032 bash[9829]: cluster 2021-03-10T13:52:35.275790+0000 mgr.y (mgr.14140) 343 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:38 gibba011 bash[9813]: cluster 2021-03-10T13:52:37.276961+0000 mgr.y (mgr.14140) 344 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:38 gibba011 bash[13015]: cluster 2021-03-10T13:52:37.276961+0000 mgr.y (mgr.14140) 344 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:38 gibba032 bash[9829]: cluster 2021-03-10T13:52:37.276961+0000 mgr.y (mgr.14140) 344 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:40 gibba011 bash[9813]: cluster 2021-03-10T13:52:39.278540+0000 mgr.y (mgr.14140) 345 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:40 gibba011 bash[13015]: cluster 2021-03-10T13:52:39.278540+0000 mgr.y (mgr.14140) 345 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:40 gibba032 bash[9829]: cluster 2021-03-10T13:52:39.278540+0000 mgr.y (mgr.14140) 345 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:42 gibba032 bash[9829]: cluster 2021-03-10T13:52:41.280075+0000 mgr.y (mgr.14140) 346 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:42 gibba011 bash[9813]: cluster 2021-03-10T13:52:41.280075+0000 mgr.y (mgr.14140) 346 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:42 gibba011 bash[13015]: cluster 2021-03-10T13:52:41.280075+0000 mgr.y (mgr.14140) 346 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:43.256 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:42 gibba032 docker[20585]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mgr.x 2021-03-10T13:52:43.256 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:42 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.x.service: Main process exited, code=exited, status=137/n/a 2021-03-10T13:52:43.256 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:42 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.x.service: Failed with result 'exit-code'. 2021-03-10T13:52:43.257 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:42 gibba032 systemd[1]: Stopped Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:52:43.257 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:43 gibba032 systemd[1]: Starting Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:52:43.257 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:43 gibba032 docker[20663]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mgr.x 2021-03-10T13:52:43.257 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:43 gibba032 systemd[1]: Started Ceph mgr.x for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:52:43.371 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:43 gibba032 bash[9829]: audit 2021-03-10T13:52:43.106030+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:43.371 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:43 gibba032 bash[9829]: audit 2021-03-10T13:52:43.114866+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:43.372 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:43 gibba032 bash[9829]: audit 2021-03-10T13:52:43.115573+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:52:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[9813]: audit 2021-03-10T13:52:43.106030+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[9813]: audit 2021-03-10T13:52:43.114866+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[9813]: audit 2021-03-10T13:52:43.115573+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:52:43.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[13015]: audit 2021-03-10T13:52:43.106030+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:43.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[13015]: audit 2021-03-10T13:52:43.114866+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350170\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350259\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350286\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350312\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350338\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350363\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350389\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350415\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:48:29.350440\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:43.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:43 gibba011 bash[13015]: audit 2021-03-10T13:52:43.115573+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:52:44.506 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:52:44 gibba032 bash[20680]: debug 2021-03-10T13:52:44.165+0000 7f7024eb0700 1 -- 172.21.2.132:0/1131565183 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b784744e0 con 0x561b78563000 2021-03-10T13:52:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:44 gibba032 bash[9829]: cluster 2021-03-10T13:52:43.281406+0000 mgr.y (mgr.14140) 347 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:44 gibba032 bash[9829]: audit 2021-03-10T13:52:44.184922+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:44 gibba032 bash[9829]: audit 2021-03-10T13:52:44.193657+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:44.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:44 gibba032 bash[9829]: audit 2021-03-10T13:52:44.195499+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[9813]: cluster 2021-03-10T13:52:43.281406+0000 mgr.y (mgr.14140) 347 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[13015]: cluster 2021-03-10T13:52:43.281406+0000 mgr.y (mgr.14140) 347 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[9813]: audit 2021-03-10T13:52:44.184922+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[9813]: audit 2021-03-10T13:52:44.193657+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[9813]: audit 2021-03-10T13:52:44.195499+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[13015]: audit 2021-03-10T13:52:44.184922+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:52:44.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[13015]: audit 2021-03-10T13:52:44.193657+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"aeae5c17250e\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180377\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:44:30.344651\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:28.170023\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:52:44.181740\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:52:44.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:44 gibba011 bash[13015]: audit 2021-03-10T13:52:44.195499+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:52:45.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:45 gibba032 bash[9829]: cephadm 2021-03-10T13:52:44.195306+0000 mgr.y (mgr.14140) 348 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:45.520 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:45 gibba032 bash[9829]: cephadm 2021-03-10T13:52:44.196380+0000 mgr.y (mgr.14140) 349 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:45.520 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:45 gibba032 bash[9829]: cephadm 2021-03-10T13:52:44.196839+0000 mgr.y (mgr.14140) 350 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:45.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[9813]: cephadm 2021-03-10T13:52:44.195306+0000 mgr.y (mgr.14140) 348 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[9813]: cephadm 2021-03-10T13:52:44.196380+0000 mgr.y (mgr.14140) 349 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[9813]: cephadm 2021-03-10T13:52:44.196839+0000 mgr.y (mgr.14140) 350 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:45.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[13015]: cephadm 2021-03-10T13:52:44.195306+0000 mgr.y (mgr.14140) 348 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:52:45.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[13015]: cephadm 2021-03-10T13:52:44.196380+0000 mgr.y (mgr.14140) 349 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:52:45.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:45 gibba011 bash[13015]: cephadm 2021-03-10T13:52:44.196839+0000 mgr.y (mgr.14140) 350 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:52:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:46 gibba011 bash[9813]: cluster 2021-03-10T13:52:45.281826+0000 mgr.y (mgr.14140) 351 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:46 gibba011 bash[13015]: cluster 2021-03-10T13:52:45.281826+0000 mgr.y (mgr.14140) 351 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:46 gibba032 bash[9829]: cluster 2021-03-10T13:52:45.281826+0000 mgr.y (mgr.14140) 351 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:47.022 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:52:47.212 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:52:47.212 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (4m) 2m ago 4m 0.21.0 prom/alertmanager c876f5897d7b 4b2fdb052cac 2021-03-10T13:52:47.212 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (4m) 3s ago 4m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:52:47.212 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (8m) 3s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 aeae5c17250e 2021-03-10T13:52:47.213 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running 3s ago 8m quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:52:47.213 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (9m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 9865ee42339d 2021-03-10T13:52:47.213 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (9m) 2m ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cdbdd153dd01 2021-03-10T13:52:47.213 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (8m) 3s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b1f2ea2c99b4 2021-03-10T13:52:47.213 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (8m) 2m ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 d494ae59299f 2021-03-10T13:52:47.214 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (4m) 2m ago 4m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:52:47.214 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (4m) 3s ago 4m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:52:47.214 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (7m) 2m ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:52:47.214 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (7m) 2m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:52:47.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (7m) 2m ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:52:47.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (6m) 2m ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:52:47.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (6m) 3s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:52:47.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (6m) 3s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:52:47.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (5m) 3s ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:52:47.216 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (5m) 3s ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:52:47.216 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (4m) 3s ago 4m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:52:47.216 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 running (2m) 2m ago 2m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 6f9c0a2748ad 2021-03-10T13:52:47.427 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:52:47.427 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:52:47.427 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:52:47.428 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:52:47.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:52:47.429 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:52:47.429 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:52:47.429 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 14 2021-03-10T13:52:47.429 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:52:47.430 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:52:48.615 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.014468+0000 mgr.y (mgr.14140) 352 : audit [DBG] from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.616 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.208185+0000 mgr.y (mgr.14140) 353 : audit [DBG] from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.616 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cluster 2021-03-10T13:52:47.282229+0000 mgr.y (mgr.14140) 354 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:48.616 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.426386+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/2126269234' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:48.616 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cephadm 2021-03-10T13:52:47.523578+0000 mgr.y (mgr.14140) 355 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.gibba032.whopeo 2021-03-10T13:52:48.617 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cephadm 2021-03-10T13:52:47.523773+0000 mgr.y (mgr.14140) 356 : cephadm [INF] Upgrade: Redeploying mgr.gibba032.whopeo 2021-03-10T13:52:48.617 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.524570+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.617 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.533492+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:52:48.617 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.534399+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.617 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.535140+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:48.618 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cephadm 2021-03-10T13:52:47.535696+0000 mgr.y (mgr.14140) 357 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:52:48.618 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:47.536004+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:52:48.618 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cluster 2021-03-10T13:52:48.013920+0000 mon.a (mon.0) 593 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:52:48.618 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: cluster 2021-03-10T13:52:48.013990+0000 mon.a (mon.0) 594 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:52:48.618 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:48.014923+0000 mon.b (mon.2) 23 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:52:48.619 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:48 gibba032 bash[9829]: audit 2021-03-10T13:52:48.017147+0000 mon.b (mon.2) 24 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:52:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.014468+0000 mgr.y (mgr.14140) 352 : audit [DBG] from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.208185+0000 mgr.y (mgr.14140) 353 : audit [DBG] from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cluster 2021-03-10T13:52:47.282229+0000 mgr.y (mgr.14140) 354 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.426386+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/2126269234' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cephadm 2021-03-10T13:52:47.523578+0000 mgr.y (mgr.14140) 355 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.gibba032.whopeo 2021-03-10T13:52:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cephadm 2021-03-10T13:52:47.523773+0000 mgr.y (mgr.14140) 356 : cephadm [INF] Upgrade: Redeploying mgr.gibba032.whopeo 2021-03-10T13:52:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.524570+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.533492+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:52:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.534399+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.535140+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cephadm 2021-03-10T13:52:47.535696+0000 mgr.y (mgr.14140) 357 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:52:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:47.536004+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:52:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cluster 2021-03-10T13:52:48.013920+0000 mon.a (mon.0) 593 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:52:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: cluster 2021-03-10T13:52:48.013990+0000 mon.a (mon.0) 594 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:52:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:48.014923+0000 mon.b (mon.2) 23 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:52:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[9813]: audit 2021-03-10T13:52:48.017147+0000 mon.b (mon.2) 24 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:52:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.014468+0000 mgr.y (mgr.14140) 352 : audit [DBG] from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.208185+0000 mgr.y (mgr.14140) 353 : audit [DBG] from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:52:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cluster 2021-03-10T13:52:47.282229+0000 mgr.y (mgr.14140) 354 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.426386+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/2126269234' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:52:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cephadm 2021-03-10T13:52:47.523578+0000 mgr.y (mgr.14140) 355 : cephadm [INF] Upgrade: It is presumed safe to stop mgr.gibba032.whopeo 2021-03-10T13:52:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cephadm 2021-03-10T13:52:47.523773+0000 mgr.y (mgr.14140) 356 : cephadm [INF] Upgrade: Redeploying mgr.gibba032.whopeo 2021-03-10T13:52:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.524570+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.533492+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:52:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.534399+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:52:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.535140+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:52:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cephadm 2021-03-10T13:52:47.535696+0000 mgr.y (mgr.14140) 357 : cephadm [INF] Deploying daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:52:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:47.536004+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.gibba032.whopeo", "key": "container_image"}]: dispatch 2021-03-10T13:52:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cluster 2021-03-10T13:52:48.013920+0000 mon.a (mon.0) 593 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:52:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: cluster 2021-03-10T13:52:48.013990+0000 mon.a (mon.0) 594 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:52:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:48.014923+0000 mon.b (mon.2) 23 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:52:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:48 gibba011 bash[13015]: audit 2021-03-10T13:52:48.017147+0000 mon.b (mon.2) 24 : audit [DBG] from='mgr.? 172.21.2.132:0/1948562518' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:52:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:49 gibba011 bash[9813]: cluster 2021-03-10T13:52:48.541140+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: gibba032.whopeo, x 2021-03-10T13:52:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:49 gibba011 bash[13015]: cluster 2021-03-10T13:52:48.541140+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: gibba032.whopeo, x 2021-03-10T13:52:50.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:49 gibba032 bash[9829]: cluster 2021-03-10T13:52:48.541140+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: gibba032.whopeo, x 2021-03-10T13:52:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:50 gibba011 bash[9813]: cluster 2021-03-10T13:52:49.283783+0000 mgr.y (mgr.14140) 358 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:50 gibba011 bash[13015]: cluster 2021-03-10T13:52:49.283783+0000 mgr.y (mgr.14140) 358 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:50 gibba032 bash[9829]: cluster 2021-03-10T13:52:49.283783+0000 mgr.y (mgr.14140) 358 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:52 gibba011 bash[9813]: cluster 2021-03-10T13:52:51.285203+0000 mgr.y (mgr.14140) 359 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:52 gibba011 bash[13015]: cluster 2021-03-10T13:52:51.285203+0000 mgr.y (mgr.14140) 359 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:52 gibba032 bash[9829]: cluster 2021-03-10T13:52:51.285203+0000 mgr.y (mgr.14140) 359 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:54 gibba011 bash[9813]: cluster 2021-03-10T13:52:53.286716+0000 mgr.y (mgr.14140) 360 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:54 gibba011 bash[13015]: cluster 2021-03-10T13:52:53.286716+0000 mgr.y (mgr.14140) 360 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:54 gibba032 bash[9829]: cluster 2021-03-10T13:52:53.286716+0000 mgr.y (mgr.14140) 360 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:56 gibba011 bash[9813]: cluster 2021-03-10T13:52:55.287152+0000 mgr.y (mgr.14140) 361 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:56 gibba011 bash[13015]: cluster 2021-03-10T13:52:55.287152+0000 mgr.y (mgr.14140) 361 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:56 gibba032 bash[9829]: cluster 2021-03-10T13:52:55.287152+0000 mgr.y (mgr.14140) 361 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:52:58 gibba011 bash[9813]: cluster 2021-03-10T13:52:57.288676+0000 mgr.y (mgr.14140) 362 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:52:58 gibba011 bash[13015]: cluster 2021-03-10T13:52:57.288676+0000 mgr.y (mgr.14140) 362 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:52:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:52:58 gibba032 bash[9829]: cluster 2021-03-10T13:52:57.288676+0000 mgr.y (mgr.14140) 362 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:00 gibba032 bash[9829]: cluster 2021-03-10T13:52:59.290178+0000 mgr.y (mgr.14140) 363 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:00 gibba032 bash[9829]: audit 2021-03-10T13:53:00.206619+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:00 gibba032 bash[9829]: audit 2021-03-10T13:53:00.215894+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:00 gibba032 bash[9829]: audit 2021-03-10T13:53:00.216753+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:53:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[9813]: cluster 2021-03-10T13:52:59.290178+0000 mgr.y (mgr.14140) 363 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[13015]: cluster 2021-03-10T13:52:59.290178+0000 mgr.y (mgr.14140) 363 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[13015]: audit 2021-03-10T13:53:00.206619+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[13015]: audit 2021-03-10T13:53:00.215894+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[13015]: audit 2021-03-10T13:53:00.216753+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:53:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[9813]: audit 2021-03-10T13:53:00.206619+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[9813]: audit 2021-03-10T13:53:00.215894+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180005\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180272\", \"created\": \"2021-03-10T13:44:28.133118\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"status\": 1, \"status_desc\": \"starting\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180536\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180686\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180834\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.180980\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181125\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181271\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:52:44.181416\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:00 gibba011 bash[9813]: audit 2021-03-10T13:53:00.216753+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2021-03-10T13:53:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: cluster 2021-03-10T13:53:01.291364+0000 mgr.y (mgr.14140) 364 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: cluster 2021-03-10T13:53:01.291364+0000 mgr.y (mgr.14140) 364 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: audit 2021-03-10T13:53:01.787984+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: audit 2021-03-10T13:53:01.791293+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: audit 2021-03-10T13:53:01.792964+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: audit 2021-03-10T13:53:01.795845+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2021-03-10T13:53:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[13015]: cluster 2021-03-10T13:53:01.799276+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2021-03-10T13:53:02.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: audit 2021-03-10T13:53:01.787984+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:02.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: audit 2021-03-10T13:53:01.791293+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:02.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: audit 2021-03-10T13:53:01.792964+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:02.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: audit 2021-03-10T13:53:01.795845+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2021-03-10T13:53:02.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[9813]: cluster 2021-03-10T13:53:01.799276+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2021-03-10T13:53:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: cluster 2021-03-10T13:53:01.291364+0000 mgr.y (mgr.14140) 364 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 5.1 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: audit 2021-03-10T13:53:01.787984+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]: dispatch 2021-03-10T13:53:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: audit 2021-03-10T13:53:01.791293+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"b\", \"daemon_type\": \"mon\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783542\", \"created\": \"2021-03-10T13:44:11.645118\", \"started\": \"2021-03-10T13:44:11.670632\"}, \"mgr.x\": {\"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"x\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783809\", \"created\": \"2021-03-10T13:44:28.133118\", \"started\": \"2021-03-10T13:52:43.114887\"}, \"mgr.gibba032.whopeo\": {\"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"daemon_id\": \"gibba032.whopeo\", \"daemon_type\": \"mgr\", \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.783973\", \"created\": \"2021-03-10T13:44:30.317118\", \"started\": \"2021-03-10T13:53:00.215719\"}, \"osd.4\": {\"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"4\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784127\", \"created\": \"2021-03-10T13:46:16.698595\", \"started\": \"2021-03-10T13:46:20.099278\"}, \"osd.5\": {\"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"5\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784275\", \"created\": \"2021-03-10T13:46:39.590594\", \"started\": \"2021-03-10T13:46:43.393092\"}, \"osd.6\": {\"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"6\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784420\", \"created\": \"2021-03-10T13:47:02.458594\", \"started\": \"2021-03-10T13:47:05.825608\"}, \"osd.7\": {\"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"daemon_id\": \"7\", \"daemon_type\": \"osd\", \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784565\", \"created\": \"2021-03-10T13:47:26.562593\", \"started\": \"2021-03-10T13:47:30.546484\"}, \"prometheus.a\": {\"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"prometheus\", \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784709\", \"created\": \"2021-03-10T13:47:48.610593\", \"started\": \"2021-03-10T13:48:25.410465\"}, \"node-exporter.b\": {\"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"daemon_id\": \"b\", \"daemon_type\": \"node-exporter\", \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784852\", \"created\": \"2021-03-10T13:47:50.158592\", \"started\": \"2021-03-10T13:47:51.465647\"}, \"grafana.a\": {\"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"daemon_id\": \"a\", \"daemon_type\": \"grafana\", \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"last_refresh\": \"2021-03-10T13:53:01.784994\", \"created\": \"2021-03-10T13:48:09.606592\", \"started\": \"2021-03-10T13:48:09.629942\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\"}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\"}], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489\"}}, \"last_daemon_update\": \"2021-03-10T13:53:01.785317\", \"last_device_update\": \"2021-03-10T13:47:34.751511\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806\"}"}]': finished 2021-03-10T13:53:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: audit 2021-03-10T13:53:01.792964+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: audit 2021-03-10T13:53:01.795845+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2021-03-10T13:53:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:02 gibba032 bash[9829]: cluster 2021-03-10T13:53:01.799276+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2021-03-10T13:53:03.079 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[10046]: debug 2021-03-10T13:53:02.796+0000 7f9f07a57700 -1 mgr handle_mgr_map I was active but no longer am 2021-03-10T13:53:03.079 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[10046]: ignoring --setuser ceph since I am not root 2021-03-10T13:53:03.080 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:02 gibba011 bash[10046]: ignoring --setgroup ceph since I am not root 2021-03-10T13:53:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: cephadm 2021-03-10T13:53:01.792797+0000 mgr.y (mgr.14140) 365 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:53:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: cephadm 2021-03-10T13:53:01.793798+0000 mgr.y (mgr.14140) 366 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:53:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: cephadm 2021-03-10T13:53:01.793863+0000 mgr.y (mgr.14140) 367 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:53:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: cephadm 2021-03-10T13:53:01.795506+0000 mgr.y (mgr.14140) 368 : cephadm [INF] Upgrade: there are 2 other already-upgraded standby mgrs, failing over 2021-03-10T13:53:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: audit 2021-03-10T13:53:02.802552+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2021-03-10T13:53:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[9813]: cluster 2021-03-10T13:53:02.802594+0000 mon.a (mon.0) 605 : cluster [DBG] mgrmap e17: gibba032.whopeo(active, starting, since 1.00624s), standbys: x 2021-03-10T13:53:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: cephadm 2021-03-10T13:53:01.792797+0000 mgr.y (mgr.14140) 365 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:53:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: cephadm 2021-03-10T13:53:01.793798+0000 mgr.y (mgr.14140) 366 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:53:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: cephadm 2021-03-10T13:53:01.793863+0000 mgr.y (mgr.14140) 367 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:53:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: cephadm 2021-03-10T13:53:01.795506+0000 mgr.y (mgr.14140) 368 : cephadm [INF] Upgrade: there are 2 other already-upgraded standby mgrs, failing over 2021-03-10T13:53:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: audit 2021-03-10T13:53:02.802552+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2021-03-10T13:53:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:03 gibba011 bash[13015]: cluster 2021-03-10T13:53:02.802594+0000 mon.a (mon.0) 605 : cluster [DBG] mgrmap e17: gibba032.whopeo(active, starting, since 1.00624s), standbys: x 2021-03-10T13:53:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: cephadm 2021-03-10T13:53:01.792797+0000 mgr.y (mgr.14140) 365 : cephadm [INF] Upgrade: Target is quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a with id d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14 2021-03-10T13:53:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: cephadm 2021-03-10T13:53:01.793798+0000 mgr.y (mgr.14140) 366 : cephadm [INF] Upgrade: Checking mgr daemons... 2021-03-10T13:53:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: cephadm 2021-03-10T13:53:01.793863+0000 mgr.y (mgr.14140) 367 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.y) 2021-03-10T13:53:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: cephadm 2021-03-10T13:53:01.795506+0000 mgr.y (mgr.14140) 368 : cephadm [INF] Upgrade: there are 2 other already-upgraded standby mgrs, failing over 2021-03-10T13:53:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: audit 2021-03-10T13:53:02.802552+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14140 172.21.2.111:0/2060711532' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2021-03-10T13:53:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:03 gibba032 bash[9829]: cluster 2021-03-10T13:53:02.802594+0000 mon.a (mon.0) 605 : cluster [DBG] mgrmap e17: gibba032.whopeo(active, starting, since 1.00624s), standbys: x 2021-03-10T13:53:05.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:05 gibba032 bash[9829]: cluster 2021-03-10T13:53:05.022070+0000 mon.a (mon.0) 606 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:05 gibba032 bash[9829]: cluster 2021-03-10T13:53:05.022783+0000 mon.a (mon.0) 607 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:05 gibba032 bash[9829]: audit 2021-03-10T13:53:05.022851+0000 mon.b (mon.2) 25 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:05 gibba032 bash[9829]: cluster 2021-03-10T13:53:05.026177+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:05 gibba032 bash[9829]: audit 2021-03-10T13:53:05.027012+0000 mon.b (mon.2) 26 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[9813]: cluster 2021-03-10T13:53:05.022070+0000 mon.a (mon.0) 606 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[9813]: cluster 2021-03-10T13:53:05.022783+0000 mon.a (mon.0) 607 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[9813]: audit 2021-03-10T13:53:05.022851+0000 mon.b (mon.2) 25 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[9813]: cluster 2021-03-10T13:53:05.026177+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[9813]: audit 2021-03-10T13:53:05.027012+0000 mon.b (mon.2) 26 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[13015]: cluster 2021-03-10T13:53:05.022070+0000 mon.a (mon.0) 606 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[13015]: cluster 2021-03-10T13:53:05.022783+0000 mon.a (mon.0) 607 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[13015]: audit 2021-03-10T13:53:05.022851+0000 mon.b (mon.2) 25 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[13015]: cluster 2021-03-10T13:53:05.026177+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:05 gibba011 bash[13015]: audit 2021-03-10T13:53:05.027012+0000 mon.b (mon.2) 26 : audit [DBG] from='mgr.? 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: cluster 2021-03-10T13:53:05.078165+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e18: gibba032.whopeo(active, starting, since 0.0555818s), standbys: x 2021-03-10T13:53:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083038+0000 mon.b (mon.2) 27 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083187+0000 mon.b (mon.2) 28 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083296+0000 mon.b (mon.2) 29 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083437+0000 mon.b (mon.2) 30 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083579+0000 mon.b (mon.2) 31 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083721+0000 mon.b (mon.2) 32 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083846+0000 mon.b (mon.2) 33 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.083978+0000 mon.b (mon.2) 34 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.084118+0000 mon.b (mon.2) 35 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.084255+0000 mon.b (mon.2) 36 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.084409+0000 mon.b (mon.2) 37 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.084517+0000 mon.b (mon.2) 38 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.084846+0000 mon.b (mon.2) 39 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.085782+0000 mon.b (mon.2) 40 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.086513+0000 mon.b (mon.2) 41 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:06.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.086875+0000 mon.b (mon.2) 42 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:06.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.087248+0000 mon.b (mon.2) 43 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:06.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.087658+0000 mon.b (mon.2) 44 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:06.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.087983+0000 mon.b (mon.2) 45 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:06.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.088416+0000 mon.b (mon.2) 46 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:06.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.088859+0000 mon.b (mon.2) 47 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:06.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.089281+0000 mon.b (mon.2) 48 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:06.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.089698+0000 mon.b (mon.2) 49 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:06.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.090087+0000 mon.b (mon.2) 50 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:06.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.090470+0000 mon.b (mon.2) 51 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:06.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.090803+0000 mon.b (mon.2) 52 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:06.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.091129+0000 mon.b (mon.2) 53 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:06.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.091515+0000 mon.b (mon.2) 54 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:06.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.091874+0000 mon.b (mon.2) 55 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:06.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.092199+0000 mon.b (mon.2) 56 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.092533+0000 mon.b (mon.2) 57 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.092888+0000 mon.b (mon.2) 58 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:06.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.093219+0000 mon.b (mon.2) 59 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:06.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.093578+0000 mon.b (mon.2) 60 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:06.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.093931+0000 mon.b (mon.2) 61 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:06.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.569811+0000 mon.b (mon.2) 62 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: cluster 2021-03-10T13:53:05.570237+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:06.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:05.571443+0000 mon.b (mon.2) 63 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: cluster 2021-03-10T13:53:06.064895+0000 mon.a (mon.0) 611 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:06.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:06.071457+0000 mon.b (mon.2) 64 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:06.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:06 gibba032 bash[9829]: audit 2021-03-10T13:53:06.072450+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: cluster 2021-03-10T13:53:05.078165+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e18: gibba032.whopeo(active, starting, since 0.0555818s), standbys: x 2021-03-10T13:53:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083038+0000 mon.b (mon.2) 27 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083187+0000 mon.b (mon.2) 28 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083296+0000 mon.b (mon.2) 29 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083437+0000 mon.b (mon.2) 30 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083579+0000 mon.b (mon.2) 31 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083721+0000 mon.b (mon.2) 32 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083846+0000 mon.b (mon.2) 33 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.083978+0000 mon.b (mon.2) 34 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.084118+0000 mon.b (mon.2) 35 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.084255+0000 mon.b (mon.2) 36 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.084409+0000 mon.b (mon.2) 37 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.084517+0000 mon.b (mon.2) 38 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.084846+0000 mon.b (mon.2) 39 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.085782+0000 mon.b (mon.2) 40 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.086513+0000 mon.b (mon.2) 41 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.086875+0000 mon.b (mon.2) 42 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: cluster 2021-03-10T13:53:05.078165+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e18: gibba032.whopeo(active, starting, since 0.0555818s), standbys: x 2021-03-10T13:53:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083038+0000 mon.b (mon.2) 27 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083187+0000 mon.b (mon.2) 28 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083296+0000 mon.b (mon.2) 29 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083437+0000 mon.b (mon.2) 30 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083579+0000 mon.b (mon.2) 31 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083721+0000 mon.b (mon.2) 32 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083846+0000 mon.b (mon.2) 33 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:06.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.083978+0000 mon.b (mon.2) 34 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:06.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.084118+0000 mon.b (mon.2) 35 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:06.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.084255+0000 mon.b (mon.2) 36 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:06.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.084409+0000 mon.b (mon.2) 37 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:06.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.084517+0000 mon.b (mon.2) 38 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:06.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.084846+0000 mon.b (mon.2) 39 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:06.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.085782+0000 mon.b (mon.2) 40 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:06.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.086513+0000 mon.b (mon.2) 41 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:06.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.086875+0000 mon.b (mon.2) 42 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:06.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.087248+0000 mon.b (mon.2) 43 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:06.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.087658+0000 mon.b (mon.2) 44 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:06.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.087983+0000 mon.b (mon.2) 45 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:06.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.088416+0000 mon.b (mon.2) 46 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:06.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.088859+0000 mon.b (mon.2) 47 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:06.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.089281+0000 mon.b (mon.2) 48 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:06.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.089698+0000 mon.b (mon.2) 49 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:06.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.090087+0000 mon.b (mon.2) 50 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.090470+0000 mon.b (mon.2) 51 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.090803+0000 mon.b (mon.2) 52 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.091129+0000 mon.b (mon.2) 53 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.091515+0000 mon.b (mon.2) 54 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.091874+0000 mon.b (mon.2) 55 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:06.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.092199+0000 mon.b (mon.2) 56 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.092533+0000 mon.b (mon.2) 57 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.092888+0000 mon.b (mon.2) 58 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:06.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.087248+0000 mon.b (mon.2) 43 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:06.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.087658+0000 mon.b (mon.2) 44 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:06.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.087983+0000 mon.b (mon.2) 45 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:06.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.088416+0000 mon.b (mon.2) 46 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:06.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.088859+0000 mon.b (mon.2) 47 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:06.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.089281+0000 mon.b (mon.2) 48 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:06.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.089698+0000 mon.b (mon.2) 49 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:06.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.090087+0000 mon.b (mon.2) 50 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:06.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.090470+0000 mon.b (mon.2) 51 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:06.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.090803+0000 mon.b (mon.2) 52 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:06.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.091129+0000 mon.b (mon.2) 53 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:06.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.091515+0000 mon.b (mon.2) 54 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:06.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.091874+0000 mon.b (mon.2) 55 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:06.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.092199+0000 mon.b (mon.2) 56 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.092533+0000 mon.b (mon.2) 57 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.092888+0000 mon.b (mon.2) 58 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:06.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.093219+0000 mon.b (mon.2) 59 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:06.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.093578+0000 mon.b (mon.2) 60 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:06.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.093931+0000 mon.b (mon.2) 61 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:06.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.569811+0000 mon.b (mon.2) 62 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: cluster 2021-03-10T13:53:05.570237+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:06.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:05.571443+0000 mon.b (mon.2) 63 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: cluster 2021-03-10T13:53:06.064895+0000 mon.a (mon.0) 611 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:06.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:06.071457+0000 mon.b (mon.2) 64 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:06.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[13015]: audit 2021-03-10T13:53:06.072450+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:06.596 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.093219+0000 mon.b (mon.2) 59 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:06.596 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.093578+0000 mon.b (mon.2) 60 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:06.596 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.093931+0000 mon.b (mon.2) 61 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:06.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.569811+0000 mon.b (mon.2) 62 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:06.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: cluster 2021-03-10T13:53:05.570237+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:06.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:05.571443+0000 mon.b (mon.2) 63 : audit [DBG] from='mgr.? 172.21.2.111:0/2247773876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:06.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: cluster 2021-03-10T13:53:06.064895+0000 mon.a (mon.0) 611 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:06.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:06.071457+0000 mon.b (mon.2) 64 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:06.598 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:06 gibba011 bash[9813]: audit 2021-03-10T13:53:06.072450+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.081139+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: cephadm 2021-03-10T13:53:06.084315+0000 mgr.gibba032.whopeo (mgr.34343) 2 : cephadm [INF] Migrating to explicit placement 2021-03-10T13:53:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.087825+0000 mon.b (mon.2) 65 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.087885+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.090074+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:53:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.091442+0000 mon.b (mon.2) 66 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.091561+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.094438+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.095922+0000 mon.b (mon.2) 67 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.096029+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.099094+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.201311+0000 mon.b (mon.2) 68 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.201549+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.203324+0000 mon.b (mon.2) 69 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.203476+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: cluster 2021-03-10T13:53:06.288379+0000 mon.a (mon.0) 622 : cluster [DBG] mgrmap e19: gibba032.whopeo(active, since 1.26578s), standbys: x, y 2021-03-10T13:53:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.290573+0000 mon.b (mon.2) 70 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.331016+0000 mon.b (mon.2) 71 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:07 gibba032 bash[9829]: audit 2021-03-10T13:53:06.334042+0000 mon.b (mon.2) 72 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.081139+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: cephadm 2021-03-10T13:53:06.084315+0000 mgr.gibba032.whopeo (mgr.34343) 2 : cephadm [INF] Migrating to explicit placement 2021-03-10T13:53:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.087825+0000 mon.b (mon.2) 65 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.087885+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.090074+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:53:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.091442+0000 mon.b (mon.2) 66 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.091561+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.094438+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.095922+0000 mon.b (mon.2) 67 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.096029+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.099094+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.201311+0000 mon.b (mon.2) 68 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.201549+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.081139+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: cephadm 2021-03-10T13:53:06.084315+0000 mgr.gibba032.whopeo (mgr.34343) 2 : cephadm [INF] Migrating to explicit placement 2021-03-10T13:53:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.087825+0000 mon.b (mon.2) 65 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.087885+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2021-03-10T13:53:07.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.090074+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2021-03-10T13:53:06.084323Z\", \"spec\": {\"placement\": {\"count\": 3, \"hosts\": [\"gibba011=y\", \"gibba032=x\", \"gibba032=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2021-03-10T13:53:07.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.091442+0000 mon.b (mon.2) 66 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.091561+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.094438+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.095922+0000 mon.b (mon.2) 67 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.096029+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2021-03-10T13:53:07.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.099094+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2021-03-10T13:53:07.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.201311+0000 mon.b (mon.2) 68 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.201549+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:07.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.203324+0000 mon.b (mon.2) 69 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.203476+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: cluster 2021-03-10T13:53:06.288379+0000 mon.a (mon.0) 622 : cluster [DBG] mgrmap e19: gibba032.whopeo(active, since 1.26578s), standbys: x, y 2021-03-10T13:53:07.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.290573+0000 mon.b (mon.2) 70 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:07.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.331016+0000 mon.b (mon.2) 71 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:07.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: audit 2021-03-10T13:53:06.334042+0000 mon.b (mon.2) 72 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:07.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[9813]: debug 2021-03-10T13:53:07.280+0000 7f649d359700 -1 log_channel(cluster) log [ERR] : Health check failed: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) (MGR_MODULE_ERROR) 2021-03-10T13:53:07.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.203324+0000 mon.b (mon.2) 69 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.203476+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:07.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: cluster 2021-03-10T13:53:06.288379+0000 mon.a (mon.0) 622 : cluster [DBG] mgrmap e19: gibba032.whopeo(active, since 1.26578s), standbys: x, y 2021-03-10T13:53:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.290573+0000 mon.b (mon.2) 70 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.331016+0000 mon.b (mon.2) 71 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:07 gibba011 bash[13015]: audit 2021-03-10T13:53:06.334042+0000 mon.b (mon.2) 72 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:08 gibba032 bash[9829]: cluster 2021-03-10T13:53:06.291743+0000 mgr.gibba032.whopeo (mgr.34343) 3 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:08 gibba032 bash[9829]: cluster 2021-03-10T13:53:07.084244+0000 mgr.gibba032.whopeo (mgr.34343) 4 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:08 gibba032 bash[9829]: cluster 2021-03-10T13:53:07.286037+0000 mon.a (mon.0) 623 : cluster [ERR] Health check failed: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) (MGR_MODULE_ERROR) 2021-03-10T13:53:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:08 gibba032 bash[9829]: cluster 2021-03-10T13:53:07.300538+0000 mon.a (mon.0) 624 : cluster [DBG] mgrmap e20: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:53:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[9813]: cluster 2021-03-10T13:53:06.291743+0000 mgr.gibba032.whopeo (mgr.34343) 3 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[9813]: cluster 2021-03-10T13:53:07.084244+0000 mgr.gibba032.whopeo (mgr.34343) 4 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[9813]: cluster 2021-03-10T13:53:07.286037+0000 mon.a (mon.0) 623 : cluster [ERR] Health check failed: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) (MGR_MODULE_ERROR) 2021-03-10T13:53:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[9813]: cluster 2021-03-10T13:53:07.300538+0000 mon.a (mon.0) 624 : cluster [DBG] mgrmap e20: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:53:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[13015]: cluster 2021-03-10T13:53:06.291743+0000 mgr.gibba032.whopeo (mgr.34343) 3 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[13015]: cluster 2021-03-10T13:53:07.084244+0000 mgr.gibba032.whopeo (mgr.34343) 4 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[13015]: cluster 2021-03-10T13:53:07.286037+0000 mon.a (mon.0) 623 : cluster [ERR] Health check failed: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) (MGR_MODULE_ERROR) 2021-03-10T13:53:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:08 gibba011 bash[13015]: cluster 2021-03-10T13:53:07.300538+0000 mon.a (mon.0) 624 : cluster [DBG] mgrmap e20: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:53:09.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:09 gibba032 bash[9829]: cluster 2021-03-10T13:53:07.337276+0000 mgr.gibba032.whopeo (mgr.34343) 5 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:09.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:09 gibba011 bash[9813]: cluster 2021-03-10T13:53:07.337276+0000 mgr.gibba032.whopeo (mgr.34343) 5 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:09.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:09 gibba011 bash[13015]: cluster 2021-03-10T13:53:07.337276+0000 mgr.gibba032.whopeo (mgr.34343) 5 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:10.458 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:10 gibba011 bash[9813]: cluster 2021-03-10T13:53:09.085088+0000 mgr.gibba032.whopeo (mgr.34343) 6 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:10.458 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:10 gibba011 bash[13015]: cluster 2021-03-10T13:53:09.085088+0000 mgr.gibba032.whopeo (mgr.34343) 6 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:10 gibba032 bash[9829]: cluster 2021-03-10T13:53:09.085088+0000 mgr.gibba032.whopeo (mgr.34343) 6 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.277787+0000 mon.b (mon.2) 73 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.277787+0000 mon.b (mon.2) 73 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.278431+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.278431+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.281662+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.281662+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.611189+0000 mon.b (mon.2) 74 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.611189+0000 mon.b (mon.2) 74 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.612783+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.621939+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.824902+0000 mon.b (mon.2) 75 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.612783+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.827636+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.836717+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.839516+0000 mon.b (mon.2) 76 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.840375+0000 mon.b (mon.2) 77 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.841137+0000 mon.b (mon.2) 78 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[13015]: audit 2021-03-10T13:53:10.841980+0000 mon.b (mon.2) 79 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:11.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.621939+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.824902+0000 mon.b (mon.2) 75 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.827636+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.836717+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:11.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.839516+0000 mon.b (mon.2) 76 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:11.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.840375+0000 mon.b (mon.2) 77 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:11.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.841137+0000 mon.b (mon.2) 78 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:11.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:11 gibba011 bash[9813]: audit 2021-03-10T13:53:10.841980+0000 mon.b (mon.2) 79 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:11.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.277787+0000 mon.b (mon.2) 73 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.278431+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.281662+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.611189+0000 mon.b (mon.2) 74 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.612783+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.621939+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:08.088777Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.824902+0000 mon.b (mon.2) 75 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.827636+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.836717+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.839516+0000 mon.b (mon.2) 76 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.840375+0000 mon.b (mon.2) 77 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.841137+0000 mon.b (mon.2) 78 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:11.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:11 gibba032 bash[9829]: audit 2021-03-10T13:53:10.841980+0000 mon.b (mon.2) 79 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:12.366 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cephadm 2021-03-10T13:53:10.282581+0000 mgr.gibba032.whopeo (mgr.34343) 7 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:53:12.366 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cephadm 2021-03-10T13:53:10.623222+0000 mgr.gibba032.whopeo (mgr.34343) 8 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:53:12.366 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cluster 2021-03-10T13:53:10.824556+0000 mgr.gibba032.whopeo (mgr.34343) 9 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:12.367 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cephadm 2021-03-10T13:53:10.839299+0000 mgr.gibba032.whopeo (mgr.34343) 10 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2021-03-10T13:53:12.367 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cephadm 2021-03-10T13:53:10.841751+0000 mgr.gibba032.whopeo (mgr.34343) 11 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:53:12.367 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[9813]: cluster 2021-03-10T13:53:11.624183+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:53:12.367 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cephadm 2021-03-10T13:53:10.282581+0000 mgr.gibba032.whopeo (mgr.34343) 7 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:53:12.368 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cephadm 2021-03-10T13:53:10.623222+0000 mgr.gibba032.whopeo (mgr.34343) 8 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:53:12.368 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cluster 2021-03-10T13:53:10.824556+0000 mgr.gibba032.whopeo (mgr.34343) 9 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:12.368 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cephadm 2021-03-10T13:53:10.839299+0000 mgr.gibba032.whopeo (mgr.34343) 10 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2021-03-10T13:53:12.368 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cephadm 2021-03-10T13:53:10.841751+0000 mgr.gibba032.whopeo (mgr.34343) 11 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:53:12.368 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:12 gibba011 bash[13015]: cluster 2021-03-10T13:53:11.624183+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:53:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cephadm 2021-03-10T13:53:10.282581+0000 mgr.gibba032.whopeo (mgr.34343) 7 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:53:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cephadm 2021-03-10T13:53:10.623222+0000 mgr.gibba032.whopeo (mgr.34343) 8 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:53:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cluster 2021-03-10T13:53:10.824556+0000 mgr.gibba032.whopeo (mgr.34343) 9 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cephadm 2021-03-10T13:53:10.839299+0000 mgr.gibba032.whopeo (mgr.34343) 10 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2021-03-10T13:53:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cephadm 2021-03-10T13:53:10.841751+0000 mgr.gibba032.whopeo (mgr.34343) 11 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:53:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:12 gibba032 bash[9829]: cluster 2021-03-10T13:53:11.624183+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:53:13.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.865906+0000 mon.b (mon.2) 80 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.865906+0000 mon.b (mon.2) 80 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.866994+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.866994+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.870877+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:13.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.877896+0000 mon.b (mon.2) 81 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:13.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.878991+0000 mon.b (mon.2) 82 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:13.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.879823+0000 mon.b (mon.2) 83 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:13.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[13015]: audit 2021-03-10T13:53:12.880702+0000 mon.b (mon.2) 84 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:53:13.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.870877+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:13.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.877896+0000 mon.b (mon.2) 81 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:13.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.878991+0000 mon.b (mon.2) 82 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:13.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.879823+0000 mon.b (mon.2) 83 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:13.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:13 gibba011 bash[9813]: audit 2021-03-10T13:53:12.880702+0000 mon.b (mon.2) 84 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:53:13.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.865906+0000 mon.b (mon.2) 80 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.866994+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.870877+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:43:55.331709Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.877896+0000 mon.b (mon.2) 81 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.878991+0000 mon.b (mon.2) 82 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.879823+0000 mon.b (mon.2) 83 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:13 gibba032 bash[9829]: audit 2021-03-10T13:53:12.880702+0000 mon.b (mon.2) 84 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:53:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:14 gibba032 bash[9829]: cluster 2021-03-10T13:53:12.826549+0000 mgr.gibba032.whopeo (mgr.34343) 12 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:14 gibba032 bash[9829]: cephadm 2021-03-10T13:53:12.877534+0000 mgr.gibba032.whopeo (mgr.34343) 13 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:53:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:14 gibba032 bash[9829]: cephadm 2021-03-10T13:53:12.880405+0000 mgr.gibba032.whopeo (mgr.34343) 14 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:53:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[9813]: cluster 2021-03-10T13:53:12.826549+0000 mgr.gibba032.whopeo (mgr.34343) 12 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[9813]: cephadm 2021-03-10T13:53:12.877534+0000 mgr.gibba032.whopeo (mgr.34343) 13 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:53:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[9813]: cephadm 2021-03-10T13:53:12.880405+0000 mgr.gibba032.whopeo (mgr.34343) 14 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:53:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[13015]: cluster 2021-03-10T13:53:12.826549+0000 mgr.gibba032.whopeo (mgr.34343) 12 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[13015]: cephadm 2021-03-10T13:53:12.877534+0000 mgr.gibba032.whopeo (mgr.34343) 13 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:53:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:14 gibba011 bash[13015]: cephadm 2021-03-10T13:53:12.880405+0000 mgr.gibba032.whopeo (mgr.34343) 14 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:53:15.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:15 gibba032 bash[9829]: audit 2021-03-10T13:53:14.655692+0000 mon.b (mon.2) 85 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:15 gibba032 bash[9829]: audit 2021-03-10T13:53:14.656716+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:15 gibba032 bash[9829]: audit 2021-03-10T13:53:14.660760+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[13015]: audit 2021-03-10T13:53:14.655692+0000 mon.b (mon.2) 85 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[9813]: audit 2021-03-10T13:53:14.655692+0000 mon.b (mon.2) 85 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[13015]: audit 2021-03-10T13:53:14.656716+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[13015]: audit 2021-03-10T13:53:14.660760+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:15.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[9813]: audit 2021-03-10T13:53:14.656716+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:15.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:15 gibba011 bash[9813]: audit 2021-03-10T13:53:14.660760+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4b2fdb052cac\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 14187233, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276471Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:48:18.516422Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\"], \"last_config\": \"2021-03-10T13:48:11.438831Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.276691Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:16 gibba032 bash[9829]: cephadm 2021-03-10T13:53:14.667433+0000 mgr.gibba032.whopeo (mgr.34343) 15 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:53:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:16 gibba032 bash[9829]: cephadm 2021-03-10T13:53:14.673516+0000 mgr.gibba032.whopeo (mgr.34343) 16 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:53:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:16 gibba032 bash[9829]: cluster 2021-03-10T13:53:14.827753+0000 mgr.gibba032.whopeo (mgr.34343) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[9813]: cephadm 2021-03-10T13:53:14.667433+0000 mgr.gibba032.whopeo (mgr.34343) 15 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:53:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[9813]: cephadm 2021-03-10T13:53:14.673516+0000 mgr.gibba032.whopeo (mgr.34343) 16 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:53:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[9813]: cluster 2021-03-10T13:53:14.827753+0000 mgr.gibba032.whopeo (mgr.34343) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[13015]: cephadm 2021-03-10T13:53:14.667433+0000 mgr.gibba032.whopeo (mgr.34343) 15 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2021-03-10T13:53:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[13015]: cephadm 2021-03-10T13:53:14.673516+0000 mgr.gibba032.whopeo (mgr.34343) 16 : cephadm [INF] Reconfiguring daemon alertmanager.a on gibba011 2021-03-10T13:53:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:16 gibba011 bash[13015]: cluster 2021-03-10T13:53:14.827753+0000 mgr.gibba032.whopeo (mgr.34343) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:17.674 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:53:17.906 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:53:17.906 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (4m) 7s ago 5m 0.21.0 prom/alertmanager c876f5897d7b 4b2fdb052cac 2021-03-10T13:53:17.906 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (5m) 7s ago 5m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:53:17.907 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (17s) 7s ago 8m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:53:17.907 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (34s) 7s ago 8m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:53:17.907 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (10m) 7s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 9865ee42339d 2021-03-10T13:53:17.907 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (10m) 7s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cdbdd153dd01 2021-03-10T13:53:17.907 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (9m) 7s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b1f2ea2c99b4 2021-03-10T13:53:17.908 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (9m) 7s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 d494ae59299f 2021-03-10T13:53:17.908 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (5m) 7s ago 5m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:53:17.908 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (5m) 7s ago 5m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:53:17.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (8m) 7s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:53:17.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (8m) 7s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (7m) 7s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (7m) 7s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (6m) 7s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (6m) 7s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (6m) 7s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:53:17.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (5m) 7s ago 5m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:53:17.910 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (4m) 7s ago 5m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:53:17.910 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7s ago 3m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:53:18.187 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:53:18.188 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:53:18.188 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:53:18.188 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:18.189 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:53:18.189 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 1, 2021-03-10T13:53:18.189 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 2 2021-03-10T13:53:18.189 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:18.190 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:53:18.190 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:53:18.190 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:18.190 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:53:18.190 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:53:18.191 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 12, 2021-03-10T13:53:18.191 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 2 2021-03-10T13:53:18.191 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:53:18.191 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:53:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:18 gibba011 bash[9813]: cluster 2021-03-10T13:53:16.829883+0000 mgr.gibba032.whopeo (mgr.34343) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:18 gibba011 bash[13015]: cluster 2021-03-10T13:53:16.829883+0000 mgr.gibba032.whopeo (mgr.34343) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:18 gibba032 bash[9829]: cluster 2021-03-10T13:53:16.829883+0000 mgr.gibba032.whopeo (mgr.34343) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[9813]: audit 2021-03-10T13:53:17.664838+0000 mgr.gibba032.whopeo (mgr.34343) 19 : audit [DBG] from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[9813]: audit 2021-03-10T13:53:17.894327+0000 mgr.gibba032.whopeo (mgr.34343) 20 : audit [DBG] from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[9813]: audit 2021-03-10T13:53:18.187088+0000 mon.a (mon.0) 636 : audit [DBG] from='client.? 172.21.2.111:0/2126212838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[13015]: audit 2021-03-10T13:53:17.664838+0000 mgr.gibba032.whopeo (mgr.34343) 19 : audit [DBG] from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[13015]: audit 2021-03-10T13:53:17.894327+0000 mgr.gibba032.whopeo (mgr.34343) 20 : audit [DBG] from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:19 gibba011 bash[13015]: audit 2021-03-10T13:53:18.187088+0000 mon.a (mon.0) 636 : audit [DBG] from='client.? 172.21.2.111:0/2126212838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:19 gibba032 bash[9829]: audit 2021-03-10T13:53:17.664838+0000 mgr.gibba032.whopeo (mgr.34343) 19 : audit [DBG] from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:19 gibba032 bash[9829]: audit 2021-03-10T13:53:17.894327+0000 mgr.gibba032.whopeo (mgr.34343) 20 : audit [DBG] from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:19 gibba032 bash[9829]: audit 2021-03-10T13:53:18.187088+0000 mon.a (mon.0) 636 : audit [DBG] from='client.? 172.21.2.111:0/2126212838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:20.499 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:20 gibba011 bash[9813]: cluster 2021-03-10T13:53:18.831002+0000 mgr.gibba032.whopeo (mgr.34343) 21 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:20.500 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:20 gibba011 bash[13015]: cluster 2021-03-10T13:53:18.831002+0000 mgr.gibba032.whopeo (mgr.34343) 21 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:20.500 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:20 gibba011 systemd[1]: Stopping Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:53:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:20 gibba032 bash[9829]: cluster 2021-03-10T13:53:18.831002+0000 mgr.gibba032.whopeo (mgr.34343) 21 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:20.766 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:20 gibba011 bash[21563]: level=info ts=2021-03-10T13:53:20.498Z caller=main.go:524 msg="Received SIGTERM, exiting gracefully..." 2021-03-10T13:53:21.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:21 gibba011 bash[9813]: cluster 2021-03-10T13:53:20.831856+0000 mgr.gibba032.whopeo (mgr.34343) 22 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:21 gibba011 bash[13015]: cluster 2021-03-10T13:53:20.831856+0000 mgr.gibba032.whopeo (mgr.34343) 22 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:21.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 docker[29300]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-alertmanager.a 2021-03-10T13:53:21.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@alertmanager.a.service: Succeeded. 2021-03-10T13:53:21.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 systemd[1]: Stopped Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:21.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 systemd[1]: Starting Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:53:21.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 docker[29333]: Error: No such container: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-alertmanager.a 2021-03-10T13:53:21.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:21 gibba011 systemd[1]: Started Ceph alertmanager.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:21.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:21 gibba032 bash[9829]: cluster 2021-03-10T13:53:20.831856+0000 mgr.gibba032.whopeo (mgr.34343) 22 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:22.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.214Z caller=main.go:216 msg="Starting Alertmanager" version="(version=0.21.0, branch=HEAD, revision=4c6c03ebfe21009c546e4d1e9b92c371d67c021d)" 2021-03-10T13:53:22.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.214Z caller=main.go:217 build_context="(go=go1.14.4, user=root@dee35927357f, date=20200617-08:54:02)" 2021-03-10T13:53:22.580 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.215Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.21.2.111 port=9094 2021-03-10T13:53:22.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.216Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s 2021-03-10T13:53:22.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.230Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:53:22.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.230Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2021-03-10T13:53:22.581 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:22.232Z caller=main.go:485 msg=Listening address=:9093 2021-03-10T13:53:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.320362+0000 mon.b (mon.2) 86 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.320362+0000 mon.b (mon.2) 86 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.320481+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.323300+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: cephadm 2021-03-10T13:53:21.325121+0000 mgr.gibba032.whopeo (mgr.34343) 23 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:53:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.325374+0000 mon.b (mon.2) 87 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.326381+0000 mon.b (mon.2) 88 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.327248+0000 mon.b (mon.2) 89 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: cephadm 2021-03-10T13:53:21.328120+0000 mgr.gibba032.whopeo (mgr.34343) 24 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:53:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[13015]: audit 2021-03-10T13:53:21.328433+0000 mon.b (mon.2) 90 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:53:22.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.320481+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.323300+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: cephadm 2021-03-10T13:53:21.325121+0000 mgr.gibba032.whopeo (mgr.34343) 23 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:53:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.325374+0000 mon.b (mon.2) 87 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.326381+0000 mon.b (mon.2) 88 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:22.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.327248+0000 mon.b (mon.2) 89 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:22.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: cephadm 2021-03-10T13:53:21.328120+0000 mgr.gibba032.whopeo (mgr.34343) 24 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:53:22.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:22 gibba011 bash[9813]: audit 2021-03-10T13:53:21.328433+0000 mon.b (mon.2) 90 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:53:22.592 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.320362+0000 mon.b (mon.2) 86 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.593 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.320481+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:22.593 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.323300+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"9865ee42339d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 307652198, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275545Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:42:50.997585Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:30.786192Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:22.594 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: cephadm 2021-03-10T13:53:21.325121+0000 mgr.gibba032.whopeo (mgr.34343) 23 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:53:22.594 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.325374+0000 mon.b (mon.2) 87 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:22.594 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.326381+0000 mon.b (mon.2) 88 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:22.594 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.327248+0000 mon.b (mon.2) 89 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:22.594 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: cephadm 2021-03-10T13:53:21.328120+0000 mgr.gibba032.whopeo (mgr.34343) 24 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:53:22.595 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:22 gibba032 bash[9829]: audit 2021-03-10T13:53:21.328433+0000 mon.b (mon.2) 90 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:53:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: cluster 2021-03-10T13:53:22.833869+0000 mgr.gibba032.whopeo (mgr.34343) 25 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: audit 2021-03-10T13:53:23.105417+0000 mon.b (mon.2) 91 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: audit 2021-03-10T13:53:23.106445+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: audit 2021-03-10T13:53:23.115407+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: audit 2021-03-10T13:53:23.121185+0000 mon.b (mon.2) 92 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: audit 2021-03-10T13:53:23.121572+0000 mgr.gibba032.whopeo (mgr.34343) 26 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: cephadm 2021-03-10T13:53:23.122753+0000 mgr.gibba032.whopeo (mgr.34343) 27 : cephadm [WRN] Failed to get Dashboard config for AlertManager: dashboard get-alertmanager-api-host failed: Module 'dashboard' has experienced an error and cannot handle commands: Timeout('Port 8443 not free on ::.',) retval: -5 2021-03-10T13:53:23.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:23 gibba032 bash[9829]: cephadm 2021-03-10T13:53:23.122846+0000 mgr.gibba032.whopeo (mgr.34343) 28 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:53:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: cluster 2021-03-10T13:53:22.833869+0000 mgr.gibba032.whopeo (mgr.34343) 25 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: cluster 2021-03-10T13:53:22.833869+0000 mgr.gibba032.whopeo (mgr.34343) 25 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: audit 2021-03-10T13:53:23.105417+0000 mon.b (mon.2) 91 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: audit 2021-03-10T13:53:23.105417+0000 mon.b (mon.2) 91 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: audit 2021-03-10T13:53:23.106445+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: audit 2021-03-10T13:53:23.115407+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: audit 2021-03-10T13:53:23.121185+0000 mon.b (mon.2) 92 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: audit 2021-03-10T13:53:23.121572+0000 mgr.gibba032.whopeo (mgr.34343) 26 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: cephadm 2021-03-10T13:53:23.122753+0000 mgr.gibba032.whopeo (mgr.34343) 27 : cephadm [WRN] Failed to get Dashboard config for AlertManager: dashboard get-alertmanager-api-host failed: Module 'dashboard' has experienced an error and cannot handle commands: Timeout('Port 8443 not free on ::.',) retval: -5 2021-03-10T13:53:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[13015]: cephadm 2021-03-10T13:53:23.122846+0000 mgr.gibba032.whopeo (mgr.34343) 28 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:53:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: audit 2021-03-10T13:53:23.106445+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: audit 2021-03-10T13:53:23.115407+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:10.610287Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:43:42.932806Z\", \"scheduled_daemon_actions\": {}}"}]': finished 2021-03-10T13:53:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: audit 2021-03-10T13:53:23.121185+0000 mon.b (mon.2) 92 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: audit 2021-03-10T13:53:23.121572+0000 mgr.gibba032.whopeo (mgr.34343) 26 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-03-10T13:53:23.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: cephadm 2021-03-10T13:53:23.122753+0000 mgr.gibba032.whopeo (mgr.34343) 27 : cephadm [WRN] Failed to get Dashboard config for AlertManager: dashboard get-alertmanager-api-host failed: Module 'dashboard' has experienced an error and cannot handle commands: Timeout('Port 8443 not free on ::.',) retval: -5 2021-03-10T13:53:23.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:23 gibba011 bash[9813]: cephadm 2021-03-10T13:53:23.122846+0000 mgr.gibba032.whopeo (mgr.34343) 28 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a 2021-03-10T13:53:24.526 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:24 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:24.216Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000041916s 2021-03-10T13:53:26.240 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[9813]: cluster 2021-03-10T13:53:24.835040+0000 mgr.gibba032.whopeo (mgr.34343) 29 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:26.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[9813]: audit 2021-03-10T13:53:25.284516+0000 mon.b (mon.2) 93 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[9813]: audit 2021-03-10T13:53:25.285036+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[9813]: audit 2021-03-10T13:53:25.288750+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]': finished 2021-03-10T13:53:26.241 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[9813]: audit 2021-03-10T13:53:25.296615+0000 mon.b (mon.2) 94 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:26.242 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[13015]: cluster 2021-03-10T13:53:24.835040+0000 mgr.gibba032.whopeo (mgr.34343) 29 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:26.242 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[13015]: audit 2021-03-10T13:53:25.284516+0000 mon.b (mon.2) 93 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.242 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[13015]: audit 2021-03-10T13:53:25.285036+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.243 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[13015]: audit 2021-03-10T13:53:25.288750+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]': finished 2021-03-10T13:53:26.243 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:25 gibba011 bash[13015]: audit 2021-03-10T13:53:25.296615+0000 mon.b (mon.2) 94 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:26.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:25 gibba032 bash[9829]: cluster 2021-03-10T13:53:24.835040+0000 mgr.gibba032.whopeo (mgr.34343) 29 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:25 gibba032 bash[9829]: audit 2021-03-10T13:53:25.284516+0000 mon.b (mon.2) 93 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:25 gibba032 bash[9829]: audit 2021-03-10T13:53:25.285036+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]: dispatch 2021-03-10T13:53:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:25 gibba032 bash[9829]: audit 2021-03-10T13:53:25.288750+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/upgrade_state","val":"{\"target_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"progress_id\": \"1d0cc364-757b-416b-972f-aa5a8507a7dd\", \"target_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"target_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"target_version\": \"17.0.0-1749-gb26fc080\", \"fs_original_max_mds\": null, \"error\": null, \"paused\": false}"}]': finished 2021-03-10T13:53:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:25 gibba032 bash[9829]: audit 2021-03-10T13:53:25.296615+0000 mon.b (mon.2) 94 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:27.218 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[13015]: cephadm 2021-03-10T13:53:25.294787+0000 mgr.gibba032.whopeo (mgr.34343) 30 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:27.219 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[13015]: cephadm 2021-03-10T13:53:25.297752+0000 mgr.gibba032.whopeo (mgr.34343) 31 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:27.219 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[13015]: cephadm 2021-03-10T13:53:25.297944+0000 mgr.gibba032.whopeo (mgr.34343) 32 : cephadm [INF] Upgrade: It is presumed safe to stop ['mgr.y'] 2021-03-10T13:53:27.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[9813]: cephadm 2021-03-10T13:53:25.294787+0000 mgr.gibba032.whopeo (mgr.34343) 30 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:27.219 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[9813]: cephadm 2021-03-10T13:53:25.297752+0000 mgr.gibba032.whopeo (mgr.34343) 31 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:27.220 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:26 gibba011 bash[9813]: cephadm 2021-03-10T13:53:25.297944+0000 mgr.gibba032.whopeo (mgr.34343) 32 : cephadm [INF] Upgrade: It is presumed safe to stop ['mgr.y'] 2021-03-10T13:53:27.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:26 gibba032 bash[9829]: cephadm 2021-03-10T13:53:25.294787+0000 mgr.gibba032.whopeo (mgr.34343) 30 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:27.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:26 gibba032 bash[9829]: cephadm 2021-03-10T13:53:25.297752+0000 mgr.gibba032.whopeo (mgr.34343) 31 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:27.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:26 gibba032 bash[9829]: cephadm 2021-03-10T13:53:25.297944+0000 mgr.gibba032.whopeo (mgr.34343) 32 : cephadm [INF] Upgrade: It is presumed safe to stop ['mgr.y'] 2021-03-10T13:53:28.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: cluster 2021-03-10T13:53:26.837259+0000 mgr.gibba032.whopeo (mgr.34343) 33 : cluster [DBG] pgmap v14: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:28.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: cephadm 2021-03-10T13:53:27.007119+0000 mgr.gibba032.whopeo (mgr.34343) 34 : cephadm [INF] Upgrade: Updating mgr.y 2021-03-10T13:53:28.222 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.008200+0000 mon.b (mon.2) 95 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.008428+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.012787+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]': finished 2021-03-10T13:53:28.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.023815+0000 mon.b (mon.2) 96 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.024309+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.223 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.027923+0000 mon.b (mon.2) 97 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:53:28.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.031249+0000 mon.b (mon.2) 98 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:28.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: cephadm 2021-03-10T13:53:27.033843+0000 mgr.gibba032.whopeo (mgr.34343) 35 : cephadm [INF] Deploying daemon mgr.y on gibba011 2021-03-10T13:53:28.224 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[9813]: audit 2021-03-10T13:53:27.035102+0000 mon.b (mon.2) 99 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:53:28.225 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: cluster 2021-03-10T13:53:26.837259+0000 mgr.gibba032.whopeo (mgr.34343) 33 : cluster [DBG] pgmap v14: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:28.225 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: cephadm 2021-03-10T13:53:27.007119+0000 mgr.gibba032.whopeo (mgr.34343) 34 : cephadm [INF] Upgrade: Updating mgr.y 2021-03-10T13:53:28.225 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.008200+0000 mon.b (mon.2) 95 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.225 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.008428+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.225 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.012787+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]': finished 2021-03-10T13:53:28.226 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.023815+0000 mon.b (mon.2) 96 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.226 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.024309+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.226 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.027923+0000 mon.b (mon.2) 97 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:53:28.226 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.031249+0000 mon.b (mon.2) 98 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:28.227 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: cephadm 2021-03-10T13:53:27.033843+0000 mgr.gibba032.whopeo (mgr.34343) 35 : cephadm [INF] Deploying daemon mgr.y on gibba011 2021-03-10T13:53:28.227 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:27 gibba011 bash[13015]: audit 2021-03-10T13:53:27.035102+0000 mon.b (mon.2) 99 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:53:28.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: cluster 2021-03-10T13:53:26.837259+0000 mgr.gibba032.whopeo (mgr.34343) 33 : cluster [DBG] pgmap v14: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:28.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: cephadm 2021-03-10T13:53:27.007119+0000 mgr.gibba032.whopeo (mgr.34343) 34 : cephadm [INF] Upgrade: Updating mgr.y 2021-03-10T13:53:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.008200+0000 mon.b (mon.2) 95 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.008428+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.012787+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr.y"}]': finished 2021-03-10T13:53:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.023815+0000 mon.b (mon.2) 96 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.024309+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:53:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.027923+0000 mon.b (mon.2) 97 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:53:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.031249+0000 mon.b (mon.2) 98 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: cephadm 2021-03-10T13:53:27.033843+0000 mgr.gibba032.whopeo (mgr.34343) 35 : cephadm [INF] Deploying daemon mgr.y on gibba011 2021-03-10T13:53:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:27 gibba032 bash[9829]: audit 2021-03-10T13:53:27.035102+0000 mon.b (mon.2) 99 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:53:29.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:28 gibba011 systemd[1]: Stopping Ceph mgr.y for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:53:30.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:29 gibba032 bash[9829]: cluster 2021-03-10T13:53:28.838255+0000 mgr.gibba032.whopeo (mgr.34343) 36 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:29 gibba011 bash[9813]: cluster 2021-03-10T13:53:28.838255+0000 mgr.gibba032.whopeo (mgr.34343) 36 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:30.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:29 gibba011 bash[13015]: cluster 2021-03-10T13:53:28.838255+0000 mgr.gibba032.whopeo (mgr.34343) 36 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:31 gibba011 bash[9813]: cluster 2021-03-10T13:53:30.839377+0000 mgr.gibba032.whopeo (mgr.34343) 37 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:31 gibba011 bash[13015]: cluster 2021-03-10T13:53:30.839377+0000 mgr.gibba032.whopeo (mgr.34343) 37 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:31 gibba032 bash[9829]: cluster 2021-03-10T13:53:30.839377+0000 mgr.gibba032.whopeo (mgr.34343) 37 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:32.579 INFO:journalctl@ceph.alertmanager.a.gibba011.stdout:Mar 10 13:53:32 gibba011 bash[29350]: level=info ts=2021-03-10T13:53:32.217Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.000877s 2021-03-10T13:53:34.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:33 gibba032 bash[9829]: cluster 2021-03-10T13:53:32.841550+0000 mgr.gibba032.whopeo (mgr.34343) 38 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:34.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:33 gibba011 bash[9813]: cluster 2021-03-10T13:53:32.841550+0000 mgr.gibba032.whopeo (mgr.34343) 38 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:33 gibba011 bash[13015]: cluster 2021-03-10T13:53:32.841550+0000 mgr.gibba032.whopeo (mgr.34343) 38 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:36.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:35 gibba032 bash[9829]: cluster 2021-03-10T13:53:34.842666+0000 mgr.gibba032.whopeo (mgr.34343) 39 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:36.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:35 gibba011 bash[9813]: cluster 2021-03-10T13:53:34.842666+0000 mgr.gibba032.whopeo (mgr.34343) 39 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:36.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:35 gibba011 bash[13015]: cluster 2021-03-10T13:53:34.842666+0000 mgr.gibba032.whopeo (mgr.34343) 39 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:38.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:37 gibba032 bash[9829]: cluster 2021-03-10T13:53:36.844566+0000 mgr.gibba032.whopeo (mgr.34343) 40 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:37 gibba011 bash[9813]: cluster 2021-03-10T13:53:36.844566+0000 mgr.gibba032.whopeo (mgr.34343) 40 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:37 gibba011 bash[13015]: cluster 2021-03-10T13:53:36.844566+0000 mgr.gibba032.whopeo (mgr.34343) 40 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:40.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:39 gibba032 bash[9829]: cluster 2021-03-10T13:53:38.845675+0000 mgr.gibba032.whopeo (mgr.34343) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:39 gibba011 bash[9813]: cluster 2021-03-10T13:53:38.845675+0000 mgr.gibba032.whopeo (mgr.34343) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:39 gibba011 bash[13015]: cluster 2021-03-10T13:53:38.845675+0000 mgr.gibba032.whopeo (mgr.34343) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:40.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:39 gibba011 docker[29687]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mgr.y 2021-03-10T13:53:40.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:39 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=137/n/a 2021-03-10T13:53:40.330 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:39 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mgr.y.service: Failed with result 'exit-code'. 2021-03-10T13:53:40.331 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:39 gibba011 systemd[1]: Stopped Ceph mgr.y for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:40.331 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:40 gibba011 systemd[1]: Started Ceph mgr.y for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.038120+0000 mon.b (mon.2) 100 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.038445+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.042071+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.235664+0000 mon.b (mon.2) 101 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.236403+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.240344+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:40 gibba032 bash[9829]: audit 2021-03-10T13:53:40.249058+0000 mon.b (mon.2) 102 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:41.329 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:41 gibba011 bash[29783]: debug 2021-03-10T13:53:41.032+0000 7fd6e3eed700 1 -- 172.21.2.111:0/2639702089 <== mon.0 v2:172.21.2.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5574cf04a4e0 con 0x5574cf13b000 2021-03-10T13:53:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.038120+0000 mon.b (mon.2) 100 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.038120+0000 mon.b (mon.2) 100 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.038445+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.038445+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]: dispatch 2021-03-10T13:53:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.042071+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.042071+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:43:09.975724Z\"}"}]': finished 2021-03-10T13:53:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.235664+0000 mon.b (mon.2) 101 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.235664+0000 mon.b (mon.2) 101 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.236403+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.240344+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[13015]: audit 2021-03-10T13:53:40.249058+0000 mon.b (mon.2) 102 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:41.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.236403+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.240344+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 97066680, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275411Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 93941923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275613Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48234496, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275674Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49325015, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.275867Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45560627, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276027Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47909437, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276167Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24285020, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276362Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.276567Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:40 gibba011 bash[9813]: audit 2021-03-10T13:53:40.249058+0000 mon.b (mon.2) 102 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:42.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:41 gibba032 bash[9829]: cluster 2021-03-10T13:53:40.846652+0000 mgr.gibba032.whopeo (mgr.34343) 42 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:41 gibba011 bash[9813]: cluster 2021-03-10T13:53:40.846652+0000 mgr.gibba032.whopeo (mgr.34343) 42 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:41 gibba011 bash[13015]: cluster 2021-03-10T13:53:40.846652+0000 mgr.gibba032.whopeo (mgr.34343) 42 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: cluster 2021-03-10T13:53:42.848821+0000 mgr.gibba032.whopeo (mgr.34343) 43 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.907271+0000 mon.b (mon.2) 103 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.907324+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.910308+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.911563+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.911922+0000 mon.b (mon.2) 104 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.913760+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.919333+0000 mon.b (mon.2) 105 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.920591+0000 mon.b (mon.2) 106 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.921468+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.921707+0000 mon.b (mon.2) 107 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.924500+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]': finished 2021-03-10T13:53:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.926333+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.926654+0000 mon.b (mon.2) 108 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.929222+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:53:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.931322+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.931542+0000 mon.b (mon.2) 109 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.934514+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2021-03-10T13:53:44.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.935761+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.936033+0000 mon.b (mon.2) 110 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.938796+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2021-03-10T13:53:44.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:43 gibba032 bash[9829]: audit 2021-03-10T13:53:43.940480+0000 mon.b (mon.2) 111 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2021-03-10T13:53:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: cluster 2021-03-10T13:53:42.848821+0000 mgr.gibba032.whopeo (mgr.34343) 43 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:44.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: cluster 2021-03-10T13:53:42.848821+0000 mgr.gibba032.whopeo (mgr.34343) 43 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.907271+0000 mon.b (mon.2) 103 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.907271+0000 mon.b (mon.2) 103 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.907324+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.907324+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.910308+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.911563+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.910308+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"cdbdd153dd01\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 103536394, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901302Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:42:49.012640Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:10.841696Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:43.906047Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]': finished 2021-03-10T13:53:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.911563+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.911922+0000 mon.b (mon.2) 104 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.913760+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.919333+0000 mon.b (mon.2) 105 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.920591+0000 mon.b (mon.2) 106 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.921468+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.921707+0000 mon.b (mon.2) 107 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.924500+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]': finished 2021-03-10T13:53:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.926333+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.926654+0000 mon.b (mon.2) 108 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.929222+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:53:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.931322+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.931542+0000 mon.b (mon.2) 109 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.934514+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2021-03-10T13:53:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.935761+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.936033+0000 mon.b (mon.2) 110 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.938796+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2021-03-10T13:53:44.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[13015]: audit 2021-03-10T13:53:43.940480+0000 mon.b (mon.2) 111 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2021-03-10T13:53:44.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.911922+0000 mon.b (mon.2) 104 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:53:44.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.913760+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2021-03-10T13:53:44.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.919333+0000 mon.b (mon.2) 105 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.920591+0000 mon.b (mon.2) 106 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.921468+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.921707+0000 mon.b (mon.2) 107 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]: dispatch 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.924500+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mgr"}]': finished 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.926333+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.926654+0000 mon.b (mon.2) 108 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]: dispatch 2021-03-10T13:53:44.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.929222+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.gibba032.whopeo"}]': finished 2021-03-10T13:53:44.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.931322+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.931542+0000 mon.b (mon.2) 109 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2021-03-10T13:53:44.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.934514+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2021-03-10T13:53:44.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.935761+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.936033+0000 mon.b (mon.2) 110 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2021-03-10T13:53:44.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.938796+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2021-03-10T13:53:44.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:43 gibba011 bash[9813]: audit 2021-03-10T13:53:43.940480+0000 mon.b (mon.2) 111 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2021-03-10T13:53:45.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.918735+0000 mgr.gibba032.whopeo (mgr.34343) 44 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:45.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.920248+0000 mgr.gibba032.whopeo (mgr.34343) 45 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:45.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.921422+0000 mgr.gibba032.whopeo (mgr.34343) 46 : cephadm [WRN] Upgrade: 1 mgr daemon(s) are 15.2.0 != target 17.0.0-1749-gb26fc080 2021-03-10T13:53:45.250 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.921468+0000 mgr.gibba032.whopeo (mgr.34343) 47 : cephadm [INF] Upgrade: Setting container_image for all mgr 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.940149+0000 mgr.gibba032.whopeo (mgr.34343) 48 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.940185+0000 mgr.gibba032.whopeo (mgr.34343) 49 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.941231+0000 mgr.gibba032.whopeo (mgr.34343) 50 : cephadm [INF] It is presumed safe to stop mon.a 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cephadm 2021-03-10T13:53:43.941304+0000 mgr.gibba032.whopeo (mgr.34343) 51 : cephadm [INF] Upgrade: It is presumed safe to stop mon.a 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cluster 2021-03-10T13:53:44.920439+0000 mon.a (mon.0) 662 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:45.251 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: cluster 2021-03-10T13:53:44.920495+0000 mon.a (mon.0) 663 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:45.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: audit 2021-03-10T13:53:44.921680+0000 mon.a (mon.0) 664 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:45.252 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[9813]: audit 2021-03-10T13:53:44.922593+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:45.252 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.918735+0000 mgr.gibba032.whopeo (mgr.34343) 44 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:45.252 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.920248+0000 mgr.gibba032.whopeo (mgr.34343) 45 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:45.253 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.921422+0000 mgr.gibba032.whopeo (mgr.34343) 46 : cephadm [WRN] Upgrade: 1 mgr daemon(s) are 15.2.0 != target 17.0.0-1749-gb26fc080 2021-03-10T13:53:45.253 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.921468+0000 mgr.gibba032.whopeo (mgr.34343) 47 : cephadm [INF] Upgrade: Setting container_image for all mgr 2021-03-10T13:53:45.253 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.940149+0000 mgr.gibba032.whopeo (mgr.34343) 48 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:53:45.253 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.940185+0000 mgr.gibba032.whopeo (mgr.34343) 49 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:53:45.253 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.941231+0000 mgr.gibba032.whopeo (mgr.34343) 50 : cephadm [INF] It is presumed safe to stop mon.a 2021-03-10T13:53:45.254 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cephadm 2021-03-10T13:53:43.941304+0000 mgr.gibba032.whopeo (mgr.34343) 51 : cephadm [INF] Upgrade: It is presumed safe to stop mon.a 2021-03-10T13:53:45.254 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cluster 2021-03-10T13:53:44.920439+0000 mon.a (mon.0) 662 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:45.254 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: cluster 2021-03-10T13:53:44.920495+0000 mon.a (mon.0) 663 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:45.254 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: audit 2021-03-10T13:53:44.921680+0000 mon.a (mon.0) 664 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:45.254 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:44 gibba011 bash[13015]: audit 2021-03-10T13:53:44.922593+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:45.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.918735+0000 mgr.gibba032.whopeo (mgr.34343) 44 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:53:45.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.920248+0000 mgr.gibba032.whopeo (mgr.34343) 45 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.921422+0000 mgr.gibba032.whopeo (mgr.34343) 46 : cephadm [WRN] Upgrade: 1 mgr daemon(s) are 15.2.0 != target 17.0.0-1749-gb26fc080 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.921468+0000 mgr.gibba032.whopeo (mgr.34343) 47 : cephadm [INF] Upgrade: Setting container_image for all mgr 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.940149+0000 mgr.gibba032.whopeo (mgr.34343) 48 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.940185+0000 mgr.gibba032.whopeo (mgr.34343) 49 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.941231+0000 mgr.gibba032.whopeo (mgr.34343) 50 : cephadm [INF] It is presumed safe to stop mon.a 2021-03-10T13:53:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cephadm 2021-03-10T13:53:43.941304+0000 mgr.gibba032.whopeo (mgr.34343) 51 : cephadm [INF] Upgrade: It is presumed safe to stop mon.a 2021-03-10T13:53:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cluster 2021-03-10T13:53:44.920439+0000 mon.a (mon.0) 662 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: cluster 2021-03-10T13:53:44.920495+0000 mon.a (mon.0) 663 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: audit 2021-03-10T13:53:44.921680+0000 mon.a (mon.0) 664 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:44 gibba032 bash[9829]: audit 2021-03-10T13:53:44.922593+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.? 172.21.2.111:0/1984522590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:46.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: cluster 2021-03-10T13:53:44.849990+0000 mgr.gibba032.whopeo (mgr.34343) 52 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: cluster 2021-03-10T13:53:44.970201+0000 mon.a (mon.0) 666 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 39s), standbys: y, x 2021-03-10T13:53:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.893743+0000 mon.b (mon.2) 112 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.894151+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.898661+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]': finished 2021-03-10T13:53:46.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.905227+0000 mon.b (mon.2) 113 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:46.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.906168+0000 mon.b (mon.2) 114 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:46.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.906933+0000 mon.b (mon.2) 115 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:46.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:45 gibba032 bash[9829]: audit 2021-03-10T13:53:45.907926+0000 mon.b (mon.2) 116 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: cluster 2021-03-10T13:53:44.849990+0000 mgr.gibba032.whopeo (mgr.34343) 52 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: cluster 2021-03-10T13:53:44.970201+0000 mon.a (mon.0) 666 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 39s), standbys: y, x 2021-03-10T13:53:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.893743+0000 mon.b (mon.2) 112 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.894151+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.898661+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]': finished 2021-03-10T13:53:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.905227+0000 mon.b (mon.2) 113 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.906168+0000 mon.b (mon.2) 114 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.906933+0000 mon.b (mon.2) 115 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[9813]: audit 2021-03-10T13:53:45.907926+0000 mon.b (mon.2) 116 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: cluster 2021-03-10T13:53:44.849990+0000 mgr.gibba032.whopeo (mgr.34343) 52 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: cluster 2021-03-10T13:53:44.970201+0000 mon.a (mon.0) 666 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 39s), standbys: y, x 2021-03-10T13:53:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.893743+0000 mon.b (mon.2) 112 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.894151+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]: dispatch 2021-03-10T13:53:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.898661+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.34343 ' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.a"}]': finished 2021-03-10T13:53:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.905227+0000 mon.b (mon.2) 113 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:53:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.906168+0000 mon.b (mon.2) 114 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:53:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.906933+0000 mon.b (mon.2) 115 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:53:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:45 gibba011 bash[13015]: audit 2021-03-10T13:53:45.907926+0000 mon.b (mon.2) 116 : audit [DBG] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:53:47.246 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:46 gibba011 bash[9813]: cephadm 2021-03-10T13:53:45.892666+0000 mgr.gibba032.whopeo (mgr.34343) 53 : cephadm [INF] Upgrade: Updating mon.a 2021-03-10T13:53:47.247 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:46 gibba011 bash[9813]: cephadm 2021-03-10T13:53:45.907662+0000 mgr.gibba032.whopeo (mgr.34343) 54 : cephadm [INF] Deploying daemon mon.a on gibba011 2021-03-10T13:53:47.247 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:46 gibba011 bash[13015]: cephadm 2021-03-10T13:53:45.892666+0000 mgr.gibba032.whopeo (mgr.34343) 53 : cephadm [INF] Upgrade: Updating mon.a 2021-03-10T13:53:47.248 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:46 gibba011 bash[13015]: cephadm 2021-03-10T13:53:45.907662+0000 mgr.gibba032.whopeo (mgr.34343) 54 : cephadm [INF] Deploying daemon mon.a on gibba011 2021-03-10T13:53:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:46 gibba032 bash[9829]: cephadm 2021-03-10T13:53:45.892666+0000 mgr.gibba032.whopeo (mgr.34343) 53 : cephadm [INF] Upgrade: Updating mon.a 2021-03-10T13:53:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:46 gibba032 bash[9829]: cephadm 2021-03-10T13:53:45.907662+0000 mgr.gibba032.whopeo (mgr.34343) 54 : cephadm [INF] Deploying daemon mon.a on gibba011 2021-03-10T13:53:48.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:47 gibba011 systemd[1]: Stopping Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:53:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:47 gibba011 bash[9813]: debug 2021-03-10T13:53:47.932+0000 7f64a1361700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:53:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:47 gibba011 bash[9813]: debug 2021-03-10T13:53:47.932+0000 7f64a1361700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2021-03-10T13:53:48.471 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:53:48.701 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:53:48.701 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (27s) 4s ago 5m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:53:48.701 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (5m) 38s ago 5m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:53:48.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (48s) 38s ago 9m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:53:48.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (65s) 38s ago 9m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:53:48.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (8s) 4s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:53:48.702 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (10m) 4s ago 11m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cdbdd153dd01 2021-03-10T13:53:48.702 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (9m) 38s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b1f2ea2c99b4 2021-03-10T13:53:48.703 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (9m) 4s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 d494ae59299f 2021-03-10T13:53:48.703 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (5m) 4s ago 5m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:53:48.703 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (5m) 38s ago 5m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:53:48.703 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (8m) 4s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:53:48.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (8m) 4s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:53:48.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (8m) 4s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:53:48.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (7m) 4s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:53:48.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (7m) 38s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:53:48.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (7m) 38s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:53:48.705 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (6m) 38s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:53:48.705 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (6m) 38s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:53:48.705 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (5m) 38s ago 6m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:53:48.705 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4s ago 3m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:53:48.959 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:53:48.960 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:53:48.960 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3 2021-03-10T13:53:48.960 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:48.960 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:53:48.961 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:53:48.961 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:48.961 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:53:48.961 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:53:48.962 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:53:48.962 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:53:48.962 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:53:48.962 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 11, 2021-03-10T13:53:48.962 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:53:48.963 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:53:48.963 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:53:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:48 gibba011 docker[30622]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.a 2021-03-10T13:53:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:48 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.a.service: Succeeded. 2021-03-10T13:53:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:48 gibba011 systemd[1]: Stopped Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:48 gibba011 systemd[1]: Started Ceph mon.a for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:53:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.780+0000 7efeb1a2a700 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:53:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.780+0000 7efeb1a2a700 0 ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev), process ceph-mon, pid 6 2021-03-10T13:53:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.780+0000 7efeb1a2a700 0 pidfile_write: ignore empty --pid-file 2021-03-10T13:53:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 0 load: jerasure load: lrc load: isa 2021-03-10T13:53:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: RocksDB version: 6.8.1 2021-03-10T13:53:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:53:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Compile date Mar 9 2021 2021-03-10T13:53:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: DB SUMMARY 2021-03-10T13:53:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: CURRENT file: CURRENT 2021-03-10T13:53:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: IDENTITY file: IDENTITY 2021-03-10T13:53:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: MANIFEST file: MANIFEST-000008 size: 909 Bytes 2021-03-10T13:53:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 2, files: 000026.sst 000027.sst 2021-03-10T13:53:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000024.log size: 10685480 ; 2021-03-10T13:53:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:53:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.create_if_missing: 0 2021-03-10T13:53:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:53:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.env: 0x5574abf351c0 2021-03-10T13:53:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.fs: Posix File System 2021-03-10T13:53:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.info_log: 0x5574ad9f71e0 2021-03-10T13:53:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:53:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:53:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:53:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:53:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:53:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:53:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:53:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:53:50.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:53:50.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:53:50.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:53:50.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:53:50.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:53:50.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:53:50.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.db_log_dir: 2021-03-10T13:53:50.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2021-03-10T13:53:50.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:53:50.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:53:50.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:53:50.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:53:50.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:53:50.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-03-10T13:53:50.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:53:50.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:53:50.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:53:50.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:53:50.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.write_buffer_manager: 0x5574ada4bc50 2021-03-10T13:53:50.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:53:50.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:53:50.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:53:50.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:53:50.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:53:50.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:53:50.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:53:50.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:53:50.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:53:50.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.unordered_write: 0 2021-03-10T13:53:50.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:53:50.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:53:50.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:53:50.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:53:50.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.row_cache: None 2021-03-10T13:53:50.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.wal_filter: None 2021-03-10T13:53:50.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:53:50.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:53:50.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:53:50.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:53:50.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:53:50.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:53:50.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:53:50.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-03-10T13:53:50.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-03-10T13:53:50.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.log_readahead_size: 0 2021-03-10T13:53:50.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-03-10T13:53:50.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:53:50.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:53:50.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:53:50.099 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:53:50.099 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:53:50.099 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:53:50.100 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:53:50.100 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:53:50.100 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:53:50.100 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:53:50.100 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:53:50.101 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:53:50.101 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:53:50.101 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-03-10T13:53:50.101 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:53:50.101 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Compression algorithms supported: 2021-03-10T13:53:50.102 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:53:50.102 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kZSTD supported: 0 2021-03-10T13:53:50.102 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:53:50.102 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:53:50.103 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:53:50.103 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:53:50.103 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:53:50.103 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:53:50.103 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:53:50.104 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000008 2021-03-10T13:53:50.104 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-03-10T13:53:50.104 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:53:50.104 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.merge_operator: 2021-03-10T13:53:50.104 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:53:50.105 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:53:50.105 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:53:50.105 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:53:50.105 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5574ad99a230) 2021-03-10T13:53:50.106 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: cache_index_and_filter_blocks: 1 2021-03-10T13:53:50.106 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: cache_index_and_filter_blocks_with_high_priority: 0 2021-03-10T13:53:50.106 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:53:50.106 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: pin_top_level_index_and_filter: 1 2021-03-10T13:53:50.106 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: index_type: 0 2021-03-10T13:53:50.107 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: data_block_index_type: 0 2021-03-10T13:53:50.107 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: index_shortening: 1 2021-03-10T13:53:50.107 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:53:50.107 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: hash_index_allow_collision: 1 2021-03-10T13:53:50.107 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: checksum: 1 2021-03-10T13:53:50.108 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: no_block_cache: 0 2021-03-10T13:53:50.109 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_cache: 0x5574ad9d2510 2021-03-10T13:53:50.109 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_cache_name: BinnedLRUCache 2021-03-10T13:53:50.109 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_cache_options: 2021-03-10T13:53:50.109 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: capacity : 536870912 2021-03-10T13:53:50.110 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: num_shard_bits : 4 2021-03-10T13:53:50.110 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: strict_capacity_limit : 0 2021-03-10T13:53:50.110 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: high_pri_pool_ratio: 0.000 2021-03-10T13:53:50.110 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_cache_compressed: (nil) 2021-03-10T13:53:50.110 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: persistent_cache: (nil) 2021-03-10T13:53:50.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_size: 4096 2021-03-10T13:53:50.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_size_deviation: 10 2021-03-10T13:53:50.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_restart_interval: 16 2021-03-10T13:53:50.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: index_block_restart_interval: 1 2021-03-10T13:53:50.111 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: metadata_block_size: 4096 2021-03-10T13:53:50.112 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: partition_filters: 0 2021-03-10T13:53:50.112 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: use_delta_encoding: 1 2021-03-10T13:53:50.112 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:53:50.112 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: whole_key_filtering: 1 2021-03-10T13:53:50.113 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: verify_compression: 0 2021-03-10T13:53:50.113 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: read_amp_bytes_per_bit: 0 2021-03-10T13:53:50.113 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: format_version: 2 2021-03-10T13:53:50.113 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: enable_index_compression: 1 2021-03-10T13:53:50.113 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: block_align: 0 2021-03-10T13:53:50.114 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:53:50.114 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:53:50.114 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:53:50.114 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:53:50.114 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:53:50.115 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:53:50.115 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.num_levels: 7 2021-03-10T13:53:50.115 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:53:50.115 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:53:50.115 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-03-10T13:53:50.116 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:53:50.116 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:53:50.116 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:53:50.116 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:53:50.117 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:53:50.117 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:53:50.117 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:53:50.117 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:53:50.117 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:53:50.118 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:53:50.118 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:53:50.118 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:53:50.118 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:53:50.118 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:53:50.119 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:53:50.119 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:53:50.119 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:53:50.119 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:53:50.119 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:53:50.120 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:53:50.120 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:53:50.120 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:53:50.120 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:53:50.121 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:53:50.121 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:53:50.121 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:53:50.121 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:53:50.121 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:53:50.122 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:53:50.122 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:53:50.122 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:53:50.122 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:53:50.123 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:53:50.123 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:53:50.123 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:53:50.123 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:53:50.123 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:53:50.124 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:53:50.124 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:53:50.124 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:53:50.124 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:53:50.125 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:53:50.125 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:53:50.125 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:53:50.125 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:53:50.125 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:53:50.126 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:53:50.126 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:53:50.126 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:53:50.126 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:53:50.126 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:53:50.127 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:53:50.127 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:53:50.127 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:53:50.127 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:53:50.128 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:53:50.128 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.ttl: 2592000 2021-03-10T13:53:50.128 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-03-10T13:53:50.128 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000008 succeeded,manifest_file_number is 8, next_file_number is 29, last_sequence is 8735, log_number is 24,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:53:50.128 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 24 2021-03-10T13:53:50.129 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384429791108, "job": 1, "event": "recovery_started", "log_files": [24]} 2021-03-10T13:53:50.129 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.784+0000 7efeb1a2a700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #24 mode 2 2021-03-10T13:53:50.129 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.792+0000 7efeb1a2a700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-03-10T13:53:50.129 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.800+0000 7efeb1a2a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384429806501, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 29, "file_size": 10628008, "table_properties": {"data_size": 10622981, "index_size": 3628, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 4130, "raw_average_key_size": 22, "raw_value_size": 10617478, "raw_average_value_size": 58660, "num_data_blocks": 112, "num_entries": 181, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1615384429, "oldest_key_time": 3, "file_creation_time": 0}} 2021-03-10T13:53:50.129 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.800+0000 7efeb1a2a700 4 rocksdb: [version_set.cc:3826] Creating manifest 30 2021-03-10T13:53:50.130 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.800+0000 7efeb1a2a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384429807224, "job": 1, "event": "recovery_finished"} 2021-03-10T13:53:50.130 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 4 rocksdb: DB pointer 0x5574ada97800 2021-03-10T13:53:50.130 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efe9883d700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-03-10T13:53:50.130 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efe9883d700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-03-10T13:53:50.131 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** DB Stats ** 2021-03-10T13:53:50.131 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:53:50.131 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:53:50.131 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:53:50.131 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:53:50.132 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:53:50.132 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:53:50.132 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:53:50.132 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** Compaction Stats [default] ** 2021-03-10T13:53:50.132 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:53:50.133 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:53:50.133 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: L0 1/0 10.14 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.133 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:53:50.133 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Sum 3/0 74.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.133 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.134 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** Compaction Stats [default] ** 2021-03-10T13:53:50.134 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:53:50.134 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:53:50.134 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.134 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:53:50.135 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Flush(GB): cumulative 0.010, interval 0.010 2021-03-10T13:53:50.136 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:53:50.137 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:53:50.137 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:53:50.137 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:53:50.137 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Cumulative compaction: 0.01 GB write, 548.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:53:50.138 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Interval compaction: 0.01 GB write, 548.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:53:50.138 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:53:50.138 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:53:50.138 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** Compaction Stats [default] ** 2021-03-10T13:53:50.138 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:53:50.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:53:50.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: L0 1/0 10.14 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:53:50.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Sum 3/0 74.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.140 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:53:50.140 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** Compaction Stats [default] ** 2021-03-10T13:53:50.140 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:53:50.140 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:53:50.141 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1102.5 0.01 0.00 1 0.009 0 0 2021-03-10T13:53:50.141 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:53:50.141 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Flush(GB): cumulative 0.010, interval 0.000 2021-03-10T13:53:50.141 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:53:50.141 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:53:50.142 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:53:50.142 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:53:50.142 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Cumulative compaction: 0.01 GB write, 547.10 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:53:50.142 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:53:50.143 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:53:50.143 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:53:50.143 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 starting mon.a rank 0 at public addrs [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] at bind addrs [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:53:50.143 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 1 mon.a@-1(???) e3 preinit fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:53:50.143 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).mds e1 new map 2021-03-10T13:53:50.144 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).mds e1 print_map 2021-03-10T13:53:50.144 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: e1 2021-03-10T13:53:50.144 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: enable_multiple, ever_enabled_multiple: 0,0 2021-03-10T13:53:50.144 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-03-10T13:53:50.145 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: legacy client fscid: -1 2021-03-10T13:53:50.145 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: 2021-03-10T13:53:50.145 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: No filesystems configured 2021-03-10T13:53:50.145 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).osd e60 crush map has features 3314933000852226048, adjusting msgr requires 2021-03-10T13:53:50.146 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).osd e60 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:53:50.146 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).osd e60 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:53:50.146 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 0 mon.a@-1(???).osd e60 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:53:50.146 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: cephadm 2021-03-10T13:53:45.892666+0000 mgr.gibba032.whopeo (mgr.34343) 53 : cephadm [INF] Upgrade: Updating mon.a 2021-03-10T13:53:50.146 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: cephadm 2021-03-10T13:53:45.907662+0000 mgr.gibba032.whopeo (mgr.34343) 54 : cephadm [INF] Deploying daemon mon.a on gibba011 2021-03-10T13:53:50.147 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:49 gibba011 bash[30791]: debug 2021-03-10T13:53:49.804+0000 7efeb1a2a700 1 mon.a@-1(???).paxosservice(auth 1..19) refresh upgraded, format 0 -> 3 2021-03-10T13:53:51.579 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[29783]: ignoring --setuser ceph since I am not root 2021-03-10T13:53:51.580 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[29783]: ignoring --setgroup ceph since I am not root 2021-03-10T13:53:51.580 INFO:journalctl@ceph.mgr.y.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[29783]: debug 2021-03-10T13:53:51.276+0000 7fcdf0669700 1 -- 172.21.2.111:0/1124487043 <== mon.0 v2:172.21.2.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a630584e0 con 0x564a63148800 2021-03-10T13:53:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: audit 2021-03-10T13:53:48.462540+0000 mgr.gibba032.whopeo (mgr.34343) 56 : audit [DBG] from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: audit 2021-03-10T13:53:48.689512+0000 mgr.gibba032.whopeo (mgr.34343) 57 : audit [DBG] from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:48.853158+0000 mgr.gibba032.whopeo (mgr.34343) 58 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: debug 2021-03-10T13:53:51.248+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: debug 2021-03-10T13:53:51.248+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:53:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: debug 2021-03-10T13:53:51.248+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:53:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: debug 2021-03-10T13:53:51.248+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: debug 2021-03-10T13:53:51.248+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: audit 2021-03-10T13:53:48.462540+0000 mgr.gibba032.whopeo (mgr.34343) 56 : audit [DBG] from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: audit 2021-03-10T13:53:48.689512+0000 mgr.gibba032.whopeo (mgr.34343) 57 : audit [DBG] from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:48.853158+0000 mgr.gibba032.whopeo (mgr.34343) 58 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: audit 2021-03-10T13:53:48.876927+0000 mon.b (mon.2) 117 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:51.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: audit 2021-03-10T13:53:48.958972+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/442604960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:51.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:50.019021+0000 mon.a (mon.0) 1 : cluster [INF] mon.a calling monitor election 2021-03-10T13:53:51.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:50.854053+0000 mgr.gibba032.whopeo (mgr.34343) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.230017+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:53:51.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.240906+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:53:51.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.240932+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2021-03-10T13:53:51.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.240950+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:51.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.241812+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.253795+0000 mon.a (mon.0) 7 : cluster [DBG] mgrmap e22: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.254311+0000 mon.a (mon.0) 8 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.254326+0000 mon.a (mon.0) 9 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:53:51.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.254338+0000 mon.a (mon.0) 10 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:53:51.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.254349+0000 mon.a (mon.0) 11 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: cluster 2021-03-10T13:53:51.254359+0000 mon.a (mon.0) 12 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[30791]: audit 2021-03-10T13:53:51.258590+0000 mon.a (mon.0) 13 : audit [INF] from='mgr.34343 ' entity='' 2021-03-10T13:53:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: audit 2021-03-10T13:53:48.876927+0000 mon.b (mon.2) 117 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: audit 2021-03-10T13:53:48.958972+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/442604960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:50.019021+0000 mon.a (mon.0) 1 : cluster [INF] mon.a calling monitor election 2021-03-10T13:53:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:50.854053+0000 mgr.gibba032.whopeo (mgr.34343) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.230017+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:53:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.240906+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:53:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.240932+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2021-03-10T13:53:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.240950+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.241812+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.253795+0000 mon.a (mon.0) 7 : cluster [DBG] mgrmap e22: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.254311+0000 mon.a (mon.0) 8 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.254326+0000 mon.a (mon.0) 9 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:53:51.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.254338+0000 mon.a (mon.0) 10 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:53:51.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.254349+0000 mon.a (mon.0) 11 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: cluster 2021-03-10T13:53:51.254359+0000 mon.a (mon.0) 12 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:51 gibba011 bash[13015]: audit 2021-03-10T13:53:51.258590+0000 mon.a (mon.0) 13 : audit [INF] from='mgr.34343 ' entity='' 2021-03-10T13:53:51.755 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[20680]: ignoring --setuser ceph since I am not root 2021-03-10T13:53:51.756 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[20680]: ignoring --setgroup ceph since I am not root 2021-03-10T13:53:51.756 INFO:journalctl@ceph.mgr.x.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[20680]: debug 2021-03-10T13:53:51.278+0000 7fe93f89a700 1 -- 172.21.2.132:0/1109255562 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f76e284e0 con 0x564f76f18800 2021-03-10T13:53:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: audit 2021-03-10T13:53:48.462540+0000 mgr.gibba032.whopeo (mgr.34343) 56 : audit [DBG] from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: audit 2021-03-10T13:53:48.689512+0000 mgr.gibba032.whopeo (mgr.34343) 57 : audit [DBG] from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:53:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:48.853158+0000 mgr.gibba032.whopeo (mgr.34343) 58 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: audit 2021-03-10T13:53:48.876927+0000 mon.b (mon.2) 117 : audit [INF] from='mgr.34343 172.21.2.132:0/1586375760' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 165884723, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.901789Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94235525, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902114Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48391782, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.902413Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49566187, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903120Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45371883, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.903769Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47919923, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.904438Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24379392, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905062Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12603883, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:43.905387Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:43.905681Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\"}"}]: dispatch 2021-03-10T13:53:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: audit 2021-03-10T13:53:48.958972+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/442604960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:53:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:50.019021+0000 mon.a (mon.0) 1 : cluster [INF] mon.a calling monitor election 2021-03-10T13:53:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:50.854053+0000 mgr.gibba032.whopeo (mgr.34343) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.3 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.230017+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:53:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.240906+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:53:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.240932+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2021-03-10T13:53:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.240950+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2021-03-10T13:53:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.241812+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e21: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.253795+0000 mon.a (mon.0) 7 : cluster [DBG] mgrmap e22: gibba032.whopeo(active, since 46s), standbys: y, x 2021-03-10T13:53:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.254311+0000 mon.a (mon.0) 8 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.254326+0000 mon.a (mon.0) 9 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:53:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.254338+0000 mon.a (mon.0) 10 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:53:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.254349+0000 mon.a (mon.0) 11 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: cluster 2021-03-10T13:53:51.254359+0000 mon.a (mon.0) 12 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:51 gibba032 bash[9829]: audit 2021-03-10T13:53:51.258590+0000 mon.a (mon.0) 13 : audit [INF] from='mgr.34343 ' entity='' 2021-03-10T13:53:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.004605+0000 mon.a (mon.0) 14 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.004683+0000 mon.a (mon.0) 15 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: audit 2021-03-10T13:53:55.006190+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: audit 2021-03-10T13:53:55.007104+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.122562+0000 mon.a (mon.0) 18 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:53:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.122691+0000 mon.a (mon.0) 19 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:53:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: audit 2021-03-10T13:53:55.122939+0000 mon.b (mon.2) 118 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: audit 2021-03-10T13:53:55.123835+0000 mon.b (mon.2) 119 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.142699+0000 mon.a (mon.0) 20 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.144097+0000 mon.a (mon.0) 21 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:56.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.152542+0000 mon.a (mon.0) 22 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:53:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.004605+0000 mon.a (mon.0) 14 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.004683+0000 mon.a (mon.0) 15 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: audit 2021-03-10T13:53:55.006190+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: audit 2021-03-10T13:53:55.007104+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.122562+0000 mon.a (mon.0) 18 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:53:56.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.122691+0000 mon.a (mon.0) 19 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:53:56.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: audit 2021-03-10T13:53:55.122939+0000 mon.b (mon.2) 118 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: audit 2021-03-10T13:53:55.123835+0000 mon.b (mon.2) 119 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.142699+0000 mon.a (mon.0) 20 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:56.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.144097+0000 mon.a (mon.0) 21 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:56.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:55 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.152542+0000 mon.a (mon.0) 22 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:53:56.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.004605+0000 mon.a (mon.0) 14 : cluster [DBG] Standby manager daemon y restarted 2021-03-10T13:53:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.004683+0000 mon.a (mon.0) 15 : cluster [DBG] Standby manager daemon y started 2021-03-10T13:53:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: audit 2021-03-10T13:53:55.006190+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: audit 2021-03-10T13:53:55.007104+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.? 172.21.2.111:0/4024603322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.122562+0000 mon.a (mon.0) 18 : cluster [DBG] Standby manager daemon x restarted 2021-03-10T13:53:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.122691+0000 mon.a (mon.0) 19 : cluster [DBG] Standby manager daemon x started 2021-03-10T13:53:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: audit 2021-03-10T13:53:55.122939+0000 mon.b (mon.2) 118 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: audit 2021-03-10T13:53:55.123835+0000 mon.b (mon.2) 119 : audit [DBG] from='mgr.? 172.21.2.132:0/3887873143' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.142699+0000 mon.a (mon.0) 20 : cluster [INF] Active manager daemon gibba032.whopeo restarted 2021-03-10T13:53:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.144097+0000 mon.a (mon.0) 21 : cluster [INF] Activating manager daemon gibba032.whopeo 2021-03-10T13:53:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:55 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.152542+0000 mon.a (mon.0) 22 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:53:57.126 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.836449+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e23: gibba032.whopeo(active, starting, since 0.692564s), standbys: x, y 2021-03-10T13:53:57.126 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841037+0000 mon.b (mon.2) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:57.126 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841206+0000 mon.b (mon.2) 121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:57.126 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841349+0000 mon.b (mon.2) 122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:57.127 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841479+0000 mon.b (mon.2) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:57.127 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841652+0000 mon.b (mon.2) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:57.127 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841803+0000 mon.b (mon.2) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:57.127 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.841951+0000 mon.b (mon.2) 126 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:57.127 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842099+0000 mon.b (mon.2) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:57.128 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842243+0000 mon.b (mon.2) 128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:57.128 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842391+0000 mon.b (mon.2) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:57.128 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842543+0000 mon.b (mon.2) 130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:57.128 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842706+0000 mon.b (mon.2) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:57.128 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.842821+0000 mon.b (mon.2) 132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:57.129 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.843138+0000 mon.b (mon.2) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:57.129 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.843809+0000 mon.b (mon.2) 134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:57.129 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.844624+0000 mon.b (mon.2) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:57.129 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.845065+0000 mon.b (mon.2) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:57.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.845507+0000 mon.b (mon.2) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:57.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.845899+0000 mon.b (mon.2) 138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:57.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.846181+0000 mon.b (mon.2) 139 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:57.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.846474+0000 mon.b (mon.2) 140 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2021-03-10T13:53:57.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.846725+0000 mon.b (mon.2) 141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:57.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.847079+0000 mon.b (mon.2) 142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:57.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.847395+0000 mon.b (mon.2) 143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:57.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.847733+0000 mon.b (mon.2) 144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:57.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.847953+0000 mon.b (mon.2) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:57.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.848256+0000 mon.b (mon.2) 146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:57.132 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.848654+0000 mon.b (mon.2) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:57.132 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.848906+0000 mon.b (mon.2) 148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:57.132 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.849098+0000 mon.b (mon.2) 149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:57.132 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.849291+0000 mon.b (mon.2) 150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:57.132 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.849479+0000 mon.b (mon.2) 151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:57.133 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.849705+0000 mon.b (mon.2) 152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:57.133 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.849884+0000 mon.b (mon.2) 153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:57.133 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.850046+0000 mon.b (mon.2) 154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:57.133 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.850202+0000 mon.b (mon.2) 155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:57.133 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.850379+0000 mon.b (mon.2) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:57.134 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: cluster 2021-03-10T13:53:55.852062+0000 mon.a (mon.0) 24 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:57.134 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.954395+0000 mon.b (mon.2) 157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.134 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.954926+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.134 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.966716+0000 mon.b (mon.2) 158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.135 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:55.966931+0000 mon.a (mon.0) 26 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.135 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:56.102815+0000 mon.b (mon.2) 159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:57.135 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:56.290561+0000 mon.b (mon.2) 160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:57.135 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:56.294191+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:53:57.136 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:56 gibba032 bash[9829]: audit 2021-03-10T13:53:56.296120+0000 mon.b (mon.2) 161 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.836449+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e23: gibba032.whopeo(active, starting, since 0.692564s), standbys: x, y 2021-03-10T13:53:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841037+0000 mon.b (mon.2) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841206+0000 mon.b (mon.2) 121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841349+0000 mon.b (mon.2) 122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841479+0000 mon.b (mon.2) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841652+0000 mon.b (mon.2) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841803+0000 mon.b (mon.2) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.841951+0000 mon.b (mon.2) 126 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842099+0000 mon.b (mon.2) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842243+0000 mon.b (mon.2) 128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842391+0000 mon.b (mon.2) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842543+0000 mon.b (mon.2) 130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842706+0000 mon.b (mon.2) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.842821+0000 mon.b (mon.2) 132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.843138+0000 mon.b (mon.2) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.843809+0000 mon.b (mon.2) 134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:57.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.844624+0000 mon.b (mon.2) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.836449+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e23: gibba032.whopeo(active, starting, since 0.692564s), standbys: x, y 2021-03-10T13:53:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841037+0000 mon.b (mon.2) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "gibba032.whopeo", "id": "gibba032.whopeo"}]: dispatch 2021-03-10T13:53:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841206+0000 mon.b (mon.2) 121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2021-03-10T13:53:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841349+0000 mon.b (mon.2) 122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2021-03-10T13:53:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841479+0000 mon.b (mon.2) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:53:57.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841652+0000 mon.b (mon.2) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:53:57.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841803+0000 mon.b (mon.2) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:53:57.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.841951+0000 mon.b (mon.2) 126 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:53:57.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842099+0000 mon.b (mon.2) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:53:57.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842243+0000 mon.b (mon.2) 128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:53:57.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842391+0000 mon.b (mon.2) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:53:57.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842543+0000 mon.b (mon.2) 130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:53:57.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842706+0000 mon.b (mon.2) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-03-10T13:53:57.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.842821+0000 mon.b (mon.2) 132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-03-10T13:53:57.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.843138+0000 mon.b (mon.2) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-03-10T13:53:57.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.843809+0000 mon.b (mon.2) 134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key ls"}]: dispatch 2021-03-10T13:53:57.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.844624+0000 mon.b (mon.2) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_crt"}]: dispatch 2021-03-10T13:53:57.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.845065+0000 mon.b (mon.2) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:57.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.845507+0000 mon.b (mon.2) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:57.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.845899+0000 mon.b (mon.2) 138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:57.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.846181+0000 mon.b (mon.2) 139 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:57.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.846474+0000 mon.b (mon.2) 140 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2021-03-10T13:53:57.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.846725+0000 mon.b (mon.2) 141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:57.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.847079+0000 mon.b (mon.2) 142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:57.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.847395+0000 mon.b (mon.2) 143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:57.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.847733+0000 mon.b (mon.2) 144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:57.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.847953+0000 mon.b (mon.2) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:57.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.848256+0000 mon.b (mon.2) 146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:57.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.848654+0000 mon.b (mon.2) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:57.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.848906+0000 mon.b (mon.2) 148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:57.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.849098+0000 mon.b (mon.2) 149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:57.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.849291+0000 mon.b (mon.2) 150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:57.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.849479+0000 mon.b (mon.2) 151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:57.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.845065+0000 mon.b (mon.2) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/grafana_key"}]: dispatch 2021-03-10T13:53:57.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.845507+0000 mon.b (mon.2) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba011"}]: dispatch 2021-03-10T13:53:57.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.845899+0000 mon.b (mon.2) 138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.gibba032"}]: dispatch 2021-03-10T13:53:57.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.846181+0000 mon.b (mon.2) 139 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2021-03-10T13:53:57.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.846474+0000 mon.b (mon.2) 140 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2021-03-10T13:53:57.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.846725+0000 mon.b (mon.2) 141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.alertmanager"}]: dispatch 2021-03-10T13:53:57.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.847079+0000 mon.b (mon.2) 142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.grafana"}]: dispatch 2021-03-10T13:53:57.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.847395+0000 mon.b (mon.2) 143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2021-03-10T13:53:57.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.847733+0000 mon.b (mon.2) 144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.node-exporter"}]: dispatch 2021-03-10T13:53:57.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.847953+0000 mon.b (mon.2) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.prometheus"}]: dispatch 2021-03-10T13:53:57.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.848256+0000 mon.b (mon.2) 146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.rgw.r.z"}]: dispatch 2021-03-10T13:53:57.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.848654+0000 mon.b (mon.2) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2021-03-10T13:53:57.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.848906+0000 mon.b (mon.2) 148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2021-03-10T13:53:57.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.849098+0000 mon.b (mon.2) 149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/upgrade_state"}]: dispatch 2021-03-10T13:53:57.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.849291+0000 mon.b (mon.2) 150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2021-03-10T13:53:57.347 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.849479+0000 mon.b (mon.2) 151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-03-10T13:53:57.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.849705+0000 mon.b (mon.2) 152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:57.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.849884+0000 mon.b (mon.2) 153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:57.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.850046+0000 mon.b (mon.2) 154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:57.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.850202+0000 mon.b (mon.2) 155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:57.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.850379+0000 mon.b (mon.2) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:57.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: cluster 2021-03-10T13:53:55.852062+0000 mon.a (mon.0) 24 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:57.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.954395+0000 mon.b (mon.2) 157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.954926+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.966716+0000 mon.b (mon.2) 158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:55.966931+0000 mon.a (mon.0) 26 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:56.102815+0000 mon.b (mon.2) 159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:57.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:56.290561+0000 mon.b (mon.2) 160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:57.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:56.294191+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:53:57.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[13015]: audit 2021-03-10T13:53:56.296120+0000 mon.b (mon.2) 161 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:57.351 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.849705+0000 mon.b (mon.2) 152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-03-10T13:53:57.351 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.849884+0000 mon.b (mon.2) 153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/devicehealth/last_scrape"}]: dispatch 2021-03-10T13:53:57.351 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.850046+0000 mon.b (mon.2) 154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/progress/completed"}]: dispatch 2021-03-10T13:53:57.352 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.850202+0000 mon.b (mon.2) 155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2021-03-10T13:53:57.352 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.850379+0000 mon.b (mon.2) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2021-03-10T13:53:57.352 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: cluster 2021-03-10T13:53:55.852062+0000 mon.a (mon.0) 24 : cluster [INF] Manager daemon gibba032.whopeo is now available 2021-03-10T13:53:57.352 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.954395+0000 mon.b (mon.2) 157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.353 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.954926+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:53:57.353 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.966716+0000 mon.b (mon.2) 158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.353 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:55.966931+0000 mon.a (mon.0) 26 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:53:57.353 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:56.102815+0000 mon.b (mon.2) 159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:57.354 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:56.290561+0000 mon.b (mon.2) 160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102110330, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609379Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 372978483, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609487Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 400346316, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609546Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48622469, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609600Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47448064, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609746Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 46252687, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609861Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47940894, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.609975Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 33900462, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610087Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 23645388, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610145Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 29842472, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:10.610198Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:53:57.354 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:56.294191+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:53:57.354 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:56 gibba011 bash[30791]: audit 2021-03-10T13:53:56.296120+0000 mon.b (mon.2) 161 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:53:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:57 gibba032 bash[9829]: cluster 2021-03-10T13:53:56.854703+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e24: gibba032.whopeo(active, since 1.71082s), standbys: x, y 2021-03-10T13:53:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:57 gibba032 bash[9829]: cluster 2021-03-10T13:53:56.859369+0000 mgr.gibba032.whopeo (mgr.24625) 1 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:57 gibba032 bash[9829]: cluster 2021-03-10T13:53:57.125344+0000 mgr.gibba032.whopeo (mgr.24625) 2 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[30791]: cluster 2021-03-10T13:53:56.854703+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e24: gibba032.whopeo(active, since 1.71082s), standbys: x, y 2021-03-10T13:53:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[30791]: cluster 2021-03-10T13:53:56.859369+0000 mgr.gibba032.whopeo (mgr.24625) 1 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[30791]: cluster 2021-03-10T13:53:57.125344+0000 mgr.gibba032.whopeo (mgr.24625) 2 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[13015]: cluster 2021-03-10T13:53:56.854703+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e24: gibba032.whopeo(active, since 1.71082s), standbys: x, y 2021-03-10T13:53:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[13015]: cluster 2021-03-10T13:53:56.859369+0000 mgr.gibba032.whopeo (mgr.24625) 1 : cluster [DBG] pgmap v3: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:57 gibba011 bash[13015]: cluster 2021-03-10T13:53:57.125344+0000 mgr.gibba032.whopeo (mgr.24625) 2 : cluster [ERR] Unhandled exception from module 'dashboard' while running on mgr.gibba032.whopeo: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:53:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:58 gibba032 bash[9829]: cluster 2021-03-10T13:53:57.843539+0000 mgr.gibba032.whopeo (mgr.24625) 3 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:59.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:58 gibba032 bash[9829]: cluster 2021-03-10T13:53:57.852858+0000 mon.a (mon.0) 29 : cluster [INF] Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-03-10T13:53:59.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:53:58 gibba032 bash[9829]: cluster 2021-03-10T13:53:57.867108+0000 mon.a (mon.0) 30 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:53:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[30791]: cluster 2021-03-10T13:53:57.843539+0000 mgr.gibba032.whopeo (mgr.24625) 3 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[30791]: cluster 2021-03-10T13:53:57.852858+0000 mon.a (mon.0) 29 : cluster [INF] Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-03-10T13:53:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[30791]: cluster 2021-03-10T13:53:57.867108+0000 mon.a (mon.0) 30 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:53:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[13015]: cluster 2021-03-10T13:53:57.843539+0000 mgr.gibba032.whopeo (mgr.24625) 3 : cluster [DBG] pgmap v4: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:53:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[13015]: cluster 2021-03-10T13:53:57.852858+0000 mon.a (mon.0) 29 : cluster [INF] Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-03-10T13:53:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:53:58 gibba011 bash[13015]: cluster 2021-03-10T13:53:57.867108+0000 mon.a (mon.0) 30 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 2s), standbys: x, y 2021-03-10T13:54:01.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:53:59.824141+0000 mon.b (mon.2) 162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94183096, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821164Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:59.822635Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:53:59.828883+0000 mon.a (mon.0) 31 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: cephadm 2021-03-10T13:53:59.830845+0000 mgr.gibba032.whopeo (mgr.24625) 4 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:54:01.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: cluster 2021-03-10T13:53:59.844080+0000 mgr.gibba032.whopeo (mgr.24625) 5 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:01.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.247527+0000 mon.b (mon.2) 163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102980648, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245735Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 375285350, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245841Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 403596902, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245898Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49754931, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245951Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48632954, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246097Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47123005, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246213Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48790241, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246330Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 34435235, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246441Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24589107, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246499Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 30188503, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246552Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:00.246641Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.260046+0000 mon.a (mon.0) 32 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.262170+0000 mgr.gibba032.whopeo (mgr.24625) 6 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:54:01.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.445826+0000 mon.b (mon.2) 164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:01.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.455769+0000 mon.a (mon.0) 33 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.461173+0000 mon.b (mon.2) 165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:01.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.462404+0000 mon.b (mon.2) 166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:01.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[30791]: audit 2021-03-10T13:54:00.463628+0000 mon.b (mon.2) 167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2021-03-10T13:54:01.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:53:59.824141+0000 mon.b (mon.2) 162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94183096, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821164Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:59.822635Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:53:59.828883+0000 mon.a (mon.0) 31 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: cephadm 2021-03-10T13:53:59.830845+0000 mgr.gibba032.whopeo (mgr.24625) 4 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:54:01.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: cluster 2021-03-10T13:53:59.844080+0000 mgr.gibba032.whopeo (mgr.24625) 5 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:01.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.247527+0000 mon.b (mon.2) 163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102980648, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245735Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 375285350, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245841Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 403596902, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245898Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49754931, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245951Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48632954, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246097Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47123005, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246213Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48790241, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246330Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 34435235, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246441Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24589107, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246499Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 30188503, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246552Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:00.246641Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.260046+0000 mon.a (mon.0) 32 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.262170+0000 mgr.gibba032.whopeo (mgr.24625) 6 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:54:01.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.445826+0000 mon.b (mon.2) 164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:01.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.455769+0000 mon.a (mon.0) 33 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.461173+0000 mon.b (mon.2) 165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:01.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.462404+0000 mon.b (mon.2) 166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:01.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:00 gibba011 bash[13015]: audit 2021-03-10T13:54:00.463628+0000 mon.b (mon.2) 167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2021-03-10T13:54:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:53:59.824141+0000 mon.b (mon.2) 162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"d494ae59299f\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 94183096, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821164Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:43:58.546236Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:12.880345Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:53:59.822635Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:53:59.828883+0000 mon.a (mon.0) 31 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: cephadm 2021-03-10T13:53:59.830845+0000 mgr.gibba032.whopeo (mgr.24625) 4 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:54:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: cluster 2021-03-10T13:53:59.844080+0000 mgr.gibba032.whopeo (mgr.24625) 5 : cluster [DBG] pgmap v5: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.247527+0000 mon.b (mon.2) 163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba032","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"b1f2ea2c99b4\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 102980648, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245735Z\", \"created\": \"2021-03-10T13:44:11.645118Z\", \"started\": \"2021-03-10T13:44:11.670632Z\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"gibba032\", \"container_id\": \"d6016a02942a\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 375285350, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245841Z\", \"created\": \"2021-03-10T13:44:28.133118Z\", \"started\": \"2021-03-10T13:52:43.114887Z\"}, \"mgr.gibba032.whopeo\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"gibba032.whopeo\", \"hostname\": \"gibba032\", \"container_id\": \"f91cc2266d59\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 403596902, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245898Z\", \"created\": \"2021-03-10T13:44:30.317118Z\", \"started\": \"2021-03-10T13:53:00.215719Z\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"gibba032\", \"container_id\": \"b06d31b74111\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 49754931, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.245951Z\", \"created\": \"2021-03-10T13:46:16.698595Z\", \"started\": \"2021-03-10T13:46:20.099278Z\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"gibba032\", \"container_id\": \"312fae1e3bf7\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48632954, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246097Z\", \"created\": \"2021-03-10T13:46:39.590594Z\", \"started\": \"2021-03-10T13:46:43.393092Z\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"gibba032\", \"container_id\": \"db54ff79c65d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 47123005, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246213Z\", \"created\": \"2021-03-10T13:47:02.458594Z\", \"started\": \"2021-03-10T13:47:05.825608Z\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"gibba032\", \"container_id\": \"afb8f661d684\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48790241, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246330Z\", \"created\": \"2021-03-10T13:47:26.562593Z\", \"started\": \"2021-03-10T13:47:30.546484Z\"}, \"prometheus.a\": {\"daemon_type\": \"prometheus\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"6a603218f777\", \"container_image_id\": \"a618f5685492e55adfb68a25b09f1d644d9123f204602f8617d7f2c13ec27d5e\", \"container_image_name\": \"prom/prometheus:latest\", \"container_image_digests\": [\"prom/prometheus@sha256:fd8b3c4c7ced91cbe96aa8a8dd4d02aa5aff7aefdaf0e579486127745c758c27\"], \"memory_usage\": 34435235, \"version\": \"2.25.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246441Z\", \"created\": \"2021-03-10T13:47:48.610593Z\", \"started\": \"2021-03-10T13:48:25.410465Z\"}, \"node-exporter.b\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"b\", \"hostname\": \"gibba032\", \"container_id\": \"f2425e6af6e6\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24589107, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246499Z\", \"created\": \"2021-03-10T13:47:50.158592Z\", \"started\": \"2021-03-10T13:47:51.465647Z\"}, \"grafana.a\": {\"daemon_type\": \"grafana\", \"daemon_id\": \"a\", \"hostname\": \"gibba032\", \"container_id\": \"c8432c2a7a94\", \"container_image_id\": \"87a51ecf0b1c9a7b187b21c1b071425dafea0d765a96d5bc371c791169b3d7f4\", \"container_image_name\": \"ceph/ceph-grafana:latest\", \"container_image_digests\": [\"ceph/ceph-grafana@sha256:f5ec32c0cf2585fc34af7f6be4d829a1ba6fe9c8eac6f4593d62f258dd718a8a\"], \"memory_usage\": 30188503, \"version\": \"6.6.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:00.246552Z\", \"created\": \"2021-03-10T13:48:09.606592Z\", \"started\": \"2021-03-10T13:48:09.629942Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ce0503e7-615c-4f41-bc6c-7eb89d164d5e\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"DrhacP-DpMS-4muL-88yP-mVvP-9ViL-dpLR1M\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"95e33995-e46c-4204-9623-3f2b7b02beb8\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"XN1VSi-gCei-vQot-X37S-aOao-ygY0-TmJy7j\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"8682e3b2-5b44-47b9-8181-83be4f601dfe\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"L8IfeA-SSX8-bPhx-aqZg-4QdF-ON4J-RYtFep\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"68d2b345-b3a1-4ff1-bb20-44ec0c354123\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"TeEjWi-s6HE-rmeY-qbin-AGdg-XycA-lcrLgr\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM913400C3375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I3K3PRF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:21.328051Z\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:30.183148Z\"}, \"mgr.gibba032.whopeo\": {\"deps\": [], \"last_config\": \"2021-03-10T13:52:47.534229Z\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:14.910108Z\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2021-03-10T13:46:37.723445Z\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:00.716513Z\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:24.659840Z\"}, \"prometheus.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\", \"node-exporter.a\", \"node-exporter.b\"], \"last_config\": \"2021-03-10T13:48:18.507184Z\"}, \"node-exporter.b\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.663777Z\"}, \"grafana.a\": {\"deps\": [\"prometheus.a\"], \"last_config\": \"2021-03-10T13:47:59.264489Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:00.246641Z\", \"last_device_update\": \"2021-03-10T13:47:34.751511Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.132\"], \"172.21.15.254\": [\"172.21.2.132\"]}, \"last_host_check\": \"2021-03-10T13:53:56.289631Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.260046+0000 mon.a (mon.0) 32 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.262170+0000 mgr.gibba032.whopeo (mgr.24625) 6 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:54:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.445826+0000 mon.b (mon.2) 164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.455769+0000 mon.a (mon.0) 33 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.461173+0000 mon.b (mon.2) 165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.462404+0000 mon.b (mon.2) 166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:00 gibba032 bash[9829]: audit 2021-03-10T13:54:00.463628+0000 mon.b (mon.2) 167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2021-03-10T13:54:02.032 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cluster 2021-03-10T13:54:00.445500+0000 mgr.gibba032.whopeo (mgr.24625) 7 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:02.033 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.460522+0000 mgr.gibba032.whopeo (mgr.24625) 8 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:02.033 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.462133+0000 mgr.gibba032.whopeo (mgr.24625) 9 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:02.033 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.463268+0000 mgr.gibba032.whopeo (mgr.24625) 10 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:02.033 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.463330+0000 mgr.gibba032.whopeo (mgr.24625) 11 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:02.034 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.464183+0000 mgr.gibba032.whopeo (mgr.24625) 12 : cephadm [INF] It is presumed safe to stop mon.c 2021-03-10T13:54:02.034 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cephadm 2021-03-10T13:54:00.464236+0000 mgr.gibba032.whopeo (mgr.24625) 13 : cephadm [INF] Upgrade: It is presumed safe to stop mon.c 2021-03-10T13:54:02.034 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[30791]: cluster 2021-03-10T13:54:00.827070+0000 mon.a (mon.0) 34 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:54:02.034 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cluster 2021-03-10T13:54:00.445500+0000 mgr.gibba032.whopeo (mgr.24625) 7 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:02.035 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.460522+0000 mgr.gibba032.whopeo (mgr.24625) 8 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:02.035 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.462133+0000 mgr.gibba032.whopeo (mgr.24625) 9 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:02.035 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.463268+0000 mgr.gibba032.whopeo (mgr.24625) 10 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:02.035 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.463330+0000 mgr.gibba032.whopeo (mgr.24625) 11 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:02.035 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.464183+0000 mgr.gibba032.whopeo (mgr.24625) 12 : cephadm [INF] It is presumed safe to stop mon.c 2021-03-10T13:54:02.036 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cephadm 2021-03-10T13:54:00.464236+0000 mgr.gibba032.whopeo (mgr.24625) 13 : cephadm [INF] Upgrade: It is presumed safe to stop mon.c 2021-03-10T13:54:02.036 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:01 gibba011 bash[13015]: cluster 2021-03-10T13:54:00.827070+0000 mon.a (mon.0) 34 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:54:02.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cluster 2021-03-10T13:54:00.445500+0000 mgr.gibba032.whopeo (mgr.24625) 7 : cluster [DBG] pgmap v6: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:02.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.460522+0000 mgr.gibba032.whopeo (mgr.24625) 8 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.462133+0000 mgr.gibba032.whopeo (mgr.24625) 9 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.463268+0000 mgr.gibba032.whopeo (mgr.24625) 10 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.463330+0000 mgr.gibba032.whopeo (mgr.24625) 11 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.464183+0000 mgr.gibba032.whopeo (mgr.24625) 12 : cephadm [INF] It is presumed safe to stop mon.c 2021-03-10T13:54:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cephadm 2021-03-10T13:54:00.464236+0000 mgr.gibba032.whopeo (mgr.24625) 13 : cephadm [INF] Upgrade: It is presumed safe to stop mon.c 2021-03-10T13:54:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:01 gibba032 bash[9829]: cluster 2021-03-10T13:54:00.827070+0000 mon.a (mon.0) 34 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-03-10T13:54:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.549982+0000 mon.b (mon.2) 168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.c"}]: dispatch 2021-03-10T13:54:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.554855+0000 mon.a (mon.0) 35 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.561992+0000 mon.b (mon.2) 169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.562763+0000 mon.b (mon.2) 170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.563448+0000 mon.b (mon.2) 171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[30791]: audit 2021-03-10T13:54:02.564345+0000 mon.b (mon.2) 172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:03.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.549982+0000 mon.b (mon.2) 168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.c"}]: dispatch 2021-03-10T13:54:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.554855+0000 mon.a (mon.0) 35 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.561992+0000 mon.b (mon.2) 169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.562763+0000 mon.b (mon.2) 170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.563448+0000 mon.b (mon.2) 171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:03.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:02 gibba011 bash[13015]: audit 2021-03-10T13:54:02.564345+0000 mon.b (mon.2) 172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:03.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.549982+0000 mon.b (mon.2) 168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.c"}]: dispatch 2021-03-10T13:54:03.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.554855+0000 mon.a (mon.0) 35 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.561992+0000 mon.b (mon.2) 169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.562763+0000 mon.b (mon.2) 170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.563448+0000 mon.b (mon.2) 171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:02 gibba032 bash[9829]: audit 2021-03-10T13:54:02.564345+0000 mon.b (mon.2) 172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:04.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:03 gibba032 bash[9829]: cluster 2021-03-10T13:54:02.447442+0000 mgr.gibba032.whopeo (mgr.24625) 14 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:04.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:03 gibba032 bash[9829]: cephadm 2021-03-10T13:54:02.548864+0000 mgr.gibba032.whopeo (mgr.24625) 15 : cephadm [INF] Upgrade: Updating mon.c 2021-03-10T13:54:04.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:03 gibba032 bash[9829]: cephadm 2021-03-10T13:54:02.564085+0000 mgr.gibba032.whopeo (mgr.24625) 16 : cephadm [INF] Deploying daemon mon.c on gibba011 2021-03-10T13:54:04.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[30791]: cluster 2021-03-10T13:54:02.447442+0000 mgr.gibba032.whopeo (mgr.24625) 14 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[30791]: cephadm 2021-03-10T13:54:02.548864+0000 mgr.gibba032.whopeo (mgr.24625) 15 : cephadm [INF] Upgrade: Updating mon.c 2021-03-10T13:54:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[30791]: cephadm 2021-03-10T13:54:02.564085+0000 mgr.gibba032.whopeo (mgr.24625) 16 : cephadm [INF] Deploying daemon mon.c on gibba011 2021-03-10T13:54:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[13015]: cluster 2021-03-10T13:54:02.447442+0000 mgr.gibba032.whopeo (mgr.24625) 14 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[13015]: cephadm 2021-03-10T13:54:02.548864+0000 mgr.gibba032.whopeo (mgr.24625) 15 : cephadm [INF] Upgrade: Updating mon.c 2021-03-10T13:54:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:03 gibba011 bash[13015]: cephadm 2021-03-10T13:54:02.564085+0000 mgr.gibba032.whopeo (mgr.24625) 16 : cephadm [INF] Deploying daemon mon.c on gibba011 2021-03-10T13:54:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:04 gibba011 systemd[1]: Stopping Ceph mon.c for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:54:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:04 gibba011 bash[13015]: debug 2021-03-10T13:54:04.708+0000 7fe95ae99700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:54:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:04 gibba011 bash[13015]: debug 2021-03-10T13:54:04.708+0000 7fe95ae99700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2021-03-10T13:54:05.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:05 gibba011 docker[31688]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.c 2021-03-10T13:54:05.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:05 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.c.service: Succeeded. 2021-03-10T13:54:05.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:05 gibba011 systemd[1]: Stopped Ceph mon.c for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:54:05.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:05 gibba011 systemd[1]: Started Ceph mon.c for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:54:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.572+0000 7f6769a1a700 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:54:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.572+0000 7f6769a1a700 0 ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev), process ceph-mon, pid 7 2021-03-10T13:54:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.572+0000 7f6769a1a700 0 pidfile_write: ignore empty --pid-file 2021-03-10T13:54:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 0 load: jerasure load: lrc load: isa 2021-03-10T13:54:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: RocksDB version: 6.8.1 2021-03-10T13:54:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:54:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Compile date Mar 9 2021 2021-03-10T13:54:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: DB SUMMARY 2021-03-10T13:54:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: CURRENT file: CURRENT 2021-03-10T13:54:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: IDENTITY file: IDENTITY 2021-03-10T13:54:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 782 Bytes 2021-03-10T13:54:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 2, files: 000022.sst 000023.sst 2021-03-10T13:54:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000020.log size: 20241837 ; 2021-03-10T13:54:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:54:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.create_if_missing: 0 2021-03-10T13:54:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:54:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.env: 0x5633284641c0 2021-03-10T13:54:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.fs: Posix File System 2021-03-10T13:54:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.info_log: 0x5633292691e0 2021-03-10T13:54:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:54:06.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:54:06.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:54:06.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:54:06.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:54:06.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:54:06.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:54:06.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:54:06.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:54:06.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:54:06.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:54:06.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:54:06.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:54:06.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:54:06.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.db_log_dir: 2021-03-10T13:54:06.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2021-03-10T13:54:06.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:54:06.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:54:06.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:54:06.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:54:06.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:54:06.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-03-10T13:54:06.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:54:06.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:54:06.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:54:06.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:54:06.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.write_buffer_manager: 0x5633292bdc50 2021-03-10T13:54:06.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:54:06.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:54:06.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:54:06.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:54:06.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:54:06.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:54:06.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:54:06.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:54:06.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:54:06.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.unordered_write: 0 2021-03-10T13:54:06.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:54:06.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:54:06.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:54:06.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:54:06.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.row_cache: None 2021-03-10T13:54:06.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.wal_filter: None 2021-03-10T13:54:06.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:54:06.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:54:06.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:54:06.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:54:06.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:54:06.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:54:06.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:54:06.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-03-10T13:54:06.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-03-10T13:54:06.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.log_readahead_size: 0 2021-03-10T13:54:06.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-03-10T13:54:06.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:54:06.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:54:06.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:54:06.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:54:06.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:54:06.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:54:06.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:54:06.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:54:06.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:54:06.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:54:06.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:54:06.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:54:06.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:54:06.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-03-10T13:54:06.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:54:06.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Compression algorithms supported: 2021-03-10T13:54:06.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kZSTD supported: 0 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:54:06.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:54:06.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:54:06.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:54:06.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2021-03-10T13:54:06.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-03-10T13:54:06.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:54:06.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.merge_operator: 2021-03-10T13:54:06.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:54:06.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:54:06.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:54:06.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:54:06.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56332920c230) 2021-03-10T13:54:06.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: cache_index_and_filter_blocks: 1 2021-03-10T13:54:06.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: cache_index_and_filter_blocks_with_high_priority: 0 2021-03-10T13:54:06.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:54:06.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: pin_top_level_index_and_filter: 1 2021-03-10T13:54:06.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: index_type: 0 2021-03-10T13:54:06.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: data_block_index_type: 0 2021-03-10T13:54:06.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: index_shortening: 1 2021-03-10T13:54:06.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:54:06.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: hash_index_allow_collision: 1 2021-03-10T13:54:06.856 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: checksum: 1 2021-03-10T13:54:06.856 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: no_block_cache: 0 2021-03-10T13:54:06.856 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_cache: 0x563329244510 2021-03-10T13:54:06.856 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_cache_name: BinnedLRUCache 2021-03-10T13:54:06.856 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_cache_options: 2021-03-10T13:54:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: capacity : 536870912 2021-03-10T13:54:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: num_shard_bits : 4 2021-03-10T13:54:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: strict_capacity_limit : 0 2021-03-10T13:54:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: high_pri_pool_ratio: 0.000 2021-03-10T13:54:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_cache_compressed: (nil) 2021-03-10T13:54:06.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: persistent_cache: (nil) 2021-03-10T13:54:06.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_size: 4096 2021-03-10T13:54:06.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_size_deviation: 10 2021-03-10T13:54:06.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_restart_interval: 16 2021-03-10T13:54:06.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: index_block_restart_interval: 1 2021-03-10T13:54:06.859 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: metadata_block_size: 4096 2021-03-10T13:54:06.859 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: partition_filters: 0 2021-03-10T13:54:06.859 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: use_delta_encoding: 1 2021-03-10T13:54:06.859 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:54:06.859 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: whole_key_filtering: 1 2021-03-10T13:54:06.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: verify_compression: 0 2021-03-10T13:54:06.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: read_amp_bytes_per_bit: 0 2021-03-10T13:54:06.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: format_version: 2 2021-03-10T13:54:06.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: enable_index_compression: 1 2021-03-10T13:54:06.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: block_align: 0 2021-03-10T13:54:06.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:54:06.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:54:06.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:54:06.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:54:06.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:54:06.862 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:54:06.862 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.num_levels: 7 2021-03-10T13:54:06.862 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:54:06.862 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:54:06.862 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-03-10T13:54:06.863 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:54:06.863 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:54:06.863 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:54:06.863 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:54:06.863 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:54:06.864 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:54:06.864 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:54:06.864 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:54:06.864 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:54:06.864 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:54:06.865 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:54:06.865 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:54:06.865 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:54:06.865 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:54:06.865 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:54:06.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:54:06.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:54:06.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:54:06.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:54:06.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:54:06.867 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:54:06.867 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:54:06.867 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:54:06.867 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:54:06.867 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:54:06.868 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:54:06.868 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:54:06.868 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:54:06.868 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:54:06.868 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:54:06.869 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:54:06.869 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:54:06.869 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:54:06.869 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:54:06.869 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:54:06.870 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:54:06.870 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:54:06.870 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:54:06.870 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:54:06.870 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:54:06.871 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:54:06.871 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:54:06.871 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:54:06.871 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:54:06.871 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:54:06.872 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:54:06.872 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:54:06.872 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:54:06.872 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:54:06.872 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:54:06.873 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:54:06.873 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:54:06.873 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:54:06.873 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:54:06.873 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:54:06.874 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:54:06.874 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.ttl: 2592000 2021-03-10T13:54:06.874 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-03-10T13:54:06.874 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 25, last_sequence is 9258, log_number is 20,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:54:06.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 20 2021-03-10T13:54:06.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384446582136, "job": 1, "event": "recovery_started", "log_files": [20]} 2021-03-10T13:54:06.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.576+0000 7f6769a1a700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #20 mode 2 2021-03-10T13:54:06.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.588+0000 7f6769a1a700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-03-10T13:54:06.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.596+0000 7f6769a1a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384446602375, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 25, "file_size": 13394140, "table_properties": {"data_size": 13387495, "index_size": 5117, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 5795, "raw_average_key_size": 22, "raw_value_size": 13379638, "raw_average_value_size": 50873, "num_data_blocks": 160, "num_entries": 263, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1615384446, "oldest_key_time": 3, "file_creation_time": 0}} 2021-03-10T13:54:06.876 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.596+0000 7f6769a1a700 4 rocksdb: [version_set.cc:3826] Creating manifest 26 2021-03-10T13:54:06.876 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.596+0000 7f6769a1a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384446603203, "job": 1, "event": "recovery_finished"} 2021-03-10T13:54:06.876 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 4 rocksdb: DB pointer 0x563329309800 2021-03-10T13:54:06.876 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f675082d700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-03-10T13:54:06.876 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f675082d700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-03-10T13:54:06.877 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** DB Stats ** 2021-03-10T13:54:06.877 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:06.877 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:54:06.877 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:54:06.877 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** Compaction Stats [default] ** 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:06.878 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:06.879 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: L0 1/0 12.77 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.879 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:06.879 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Sum 3/0 76.82 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.879 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.879 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** Compaction Stats [default] ** 2021-03-10T13:54:06.880 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:06.880 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:06.880 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.880 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:06.880 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Flush(GB): cumulative 0.012, interval 0.012 2021-03-10T13:54:06.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:54:06.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:54:06.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:54:06.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:54:06.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Cumulative compaction: 0.01 GB write, 548.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:06.882 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Interval compaction: 0.01 GB write, 548.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:06.882 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:54:06.882 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:54:06.882 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** Compaction Stats [default] ** 2021-03-10T13:54:06.883 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:06.883 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:06.883 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: L0 1/0 12.77 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.883 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:06.883 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Sum 3/0 76.82 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.884 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:06.884 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** Compaction Stats [default] ** 2021-03-10T13:54:06.884 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:06.884 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:06.884 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1199.6 0.01 0.00 1 0.011 0 0 2021-03-10T13:54:06.885 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:06.885 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Flush(GB): cumulative 0.012, interval 0.000 2021-03-10T13:54:06.885 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:54:06.885 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:54:06.885 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:54:06.886 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:54:06.886 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Cumulative compaction: 0.01 GB write, 548.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:06.886 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:06.886 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:54:06.886 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:54:06.887 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 starting mon.c rank 1 at public addrs [v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0] at bind addrs [v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:54:06.887 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 1 mon.c@-1(???) e3 preinit fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:54:06.887 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).mds e1 new map 2021-03-10T13:54:06.887 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).mds e1 print_map 2021-03-10T13:54:06.887 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: e1 2021-03-10T13:54:06.888 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: enable_multiple, ever_enabled_multiple: 0,0 2021-03-10T13:54:06.888 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-03-10T13:54:06.888 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: legacy client fscid: -1 2021-03-10T13:54:06.888 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: 2021-03-10T13:54:06.888 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: No filesystems configured 2021-03-10T13:54:06.889 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).osd e61 crush map has features 3314933000852226048, adjusting msgr requires 2021-03-10T13:54:06.889 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:06.889 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:06.889 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 0 mon.c@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:06.890 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: cluster 2021-03-10T13:54:02.447442+0000 mgr.gibba032.whopeo (mgr.24625) 14 : cluster [DBG] pgmap v7: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:06.890 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: cephadm 2021-03-10T13:54:02.548864+0000 mgr.gibba032.whopeo (mgr.24625) 15 : cephadm [INF] Upgrade: Updating mon.c 2021-03-10T13:54:06.890 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: cephadm 2021-03-10T13:54:02.564085+0000 mgr.gibba032.whopeo (mgr.24625) 16 : cephadm [INF] Deploying daemon mon.c on gibba011 2021-03-10T13:54:06.891 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.600+0000 7f6769a1a700 1 mon.c@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2021-03-10T13:54:06.891 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expand map: {default=false} 2021-03-10T13:54:06.891 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta from 'false' to 'false' 2021-03-10T13:54:06.891 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expanded map: {default=false} 2021-03-10T13:54:06.891 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expand map: {default=info} 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta from 'info' to 'info' 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expanded map: {default=info} 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expand map: {default=daemon} 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta from 'daemon' to 'daemon' 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expanded map: {default=daemon} 2021-03-10T13:54:06.892 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expand map: {default=debug} 2021-03-10T13:54:06.893 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta from 'debug' to 'debug' 2021-03-10T13:54:06.893 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:06 gibba011 bash[31774]: debug 2021-03-10T13:54:06.604+0000 7f6769a1a700 20 expand_channel_meta expanded map: {default=debug} 2021-03-10T13:54:08.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:07 gibba011 bash[30791]: debug 2021-03-10T13:54:07.856+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:07 gibba011 bash[30791]: debug 2021-03-10T13:54:07.856+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:07 gibba011 bash[30791]: debug 2021-03-10T13:54:07.856+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:07 gibba011 bash[30791]: debug 2021-03-10T13:54:07.856+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:07 gibba011 bash[30791]: debug 2021-03-10T13:54:07.856+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:04.448684+0000 mgr.gibba032.whopeo (mgr.24625) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: audit 2021-03-10T13:54:05.656348+0000 mon.b (mon.2) 173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:06.449828+0000 mgr.gibba032.whopeo (mgr.24625) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:06.816289+0000 mon.c (mon.1) 1 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.836344+0000 mon.a (mon.0) 36 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.842744+0000 mon.a (mon.0) 37 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.855881+0000 mon.a (mon.0) 38 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.855967+0000 mon.a (mon.0) 39 : cluster [DBG] fsmap 2021-03-10T13:54:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.855998+0000 mon.a (mon.0) 40 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.856381+0000 mon.a (mon.0) 41 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 12s), standbys: x, y 2021-03-10T13:54:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.860622+0000 mon.a (mon.0) 42 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.860632+0000 mon.a (mon.0) 43 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.860637+0000 mon.a (mon.0) 44 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.860655+0000 mon.a (mon.0) 45 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: cluster 2021-03-10T13:54:07.860659+0000 mon.a (mon.0) 46 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: audit 2021-03-10T13:54:07.863688+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.24625 ' entity='' 2021-03-10T13:54:09.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:08 gibba032 bash[9829]: audit 2021-03-10T13:54:07.865808+0000 mon.b (mon.2) 174 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:04.448684+0000 mgr.gibba032.whopeo (mgr.24625) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:04.448684+0000 mgr.gibba032.whopeo (mgr.24625) 17 : cluster [DBG] pgmap v8: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: audit 2021-03-10T13:54:05.656348+0000 mon.b (mon.2) 173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:06.449828+0000 mgr.gibba032.whopeo (mgr.24625) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:06.816289+0000 mon.c (mon.1) 1 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.836344+0000 mon.a (mon.0) 36 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.842744+0000 mon.a (mon.0) 37 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.855881+0000 mon.a (mon.0) 38 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.855967+0000 mon.a (mon.0) 39 : cluster [DBG] fsmap 2021-03-10T13:54:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.855998+0000 mon.a (mon.0) 40 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.856381+0000 mon.a (mon.0) 41 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 12s), standbys: x, y 2021-03-10T13:54:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.860622+0000 mon.a (mon.0) 42 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.860632+0000 mon.a (mon.0) 43 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:09.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.860637+0000 mon.a (mon.0) 44 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:09.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.860655+0000 mon.a (mon.0) 45 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: cluster 2021-03-10T13:54:07.860659+0000 mon.a (mon.0) 46 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: audit 2021-03-10T13:54:07.863688+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.24625 ' entity='' 2021-03-10T13:54:09.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[31774]: audit 2021-03-10T13:54:07.865808+0000 mon.b (mon.2) 174 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:09.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: audit 2021-03-10T13:54:05.656348+0000 mon.b (mon.2) 173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 38241566, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.820830Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374446489, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821045Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48580526, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821273Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50415534, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821559Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45990543, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.821805Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48601497, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822038Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24610078, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 12939427, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2021-03-10T13:53:59.822384Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:53:59.822491Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:09.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:06.449828+0000 mgr.gibba032.whopeo (mgr.24625) 18 : cluster [DBG] pgmap v9: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:09.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:06.816289+0000 mon.c (mon.1) 1 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:09.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.836344+0000 mon.a (mon.0) 36 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:09.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.842744+0000 mon.a (mon.0) 37 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:09.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.855881+0000 mon.a (mon.0) 38 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:09.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.855967+0000 mon.a (mon.0) 39 : cluster [DBG] fsmap 2021-03-10T13:54:09.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.855998+0000 mon.a (mon.0) 40 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:09.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.856381+0000 mon.a (mon.0) 41 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 12s), standbys: x, y 2021-03-10T13:54:09.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.860622+0000 mon.a (mon.0) 42 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.860632+0000 mon.a (mon.0) 43 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:09.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.860637+0000 mon.a (mon.0) 44 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:09.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.860655+0000 mon.a (mon.0) 45 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: cluster 2021-03-10T13:54:07.860659+0000 mon.a (mon.0) 46 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:09.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: audit 2021-03-10T13:54:07.863688+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.24625 ' entity='' 2021-03-10T13:54:09.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:08 gibba011 bash[30791]: audit 2021-03-10T13:54:07.865808+0000 mon.b (mon.2) 174 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:10.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:09 gibba032 bash[9829]: cluster 2021-03-10T13:54:08.451858+0000 mgr.gibba032.whopeo (mgr.24625) 19 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:10.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:09 gibba011 bash[30791]: cluster 2021-03-10T13:54:08.451858+0000 mgr.gibba032.whopeo (mgr.24625) 19 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:09 gibba011 bash[31774]: cluster 2021-03-10T13:54:08.451858+0000 mgr.gibba032.whopeo (mgr.24625) 19 : cluster [DBG] pgmap v10: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: cluster 2021-03-10T13:54:10.452943+0000 mgr.gibba032.whopeo (mgr.24625) 20 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.600325+0000 mon.b (mon.2) 175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 43568332, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598613Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374551347, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598715Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"6edd7f37c961\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 39227228, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598767Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:54:05.680495Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48737812, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598816Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50457477, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598954Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45969571, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599060Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48685383, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599164Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24735907, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 13274972, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599322Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599372Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:11.599453Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.605465+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.607566+0000 mon.b (mon.2) 176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.611411+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.626651+0000 mon.b (mon.2) 177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.627854+0000 mon.b (mon.2) 178 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:11 gibba032 bash[9829]: audit 2021-03-10T13:54:11.629059+0000 mon.b (mon.2) 179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2021-03-10T13:54:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: cluster 2021-03-10T13:54:10.452943+0000 mgr.gibba032.whopeo (mgr.24625) 20 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: cluster 2021-03-10T13:54:10.452943+0000 mgr.gibba032.whopeo (mgr.24625) 20 : cluster [DBG] pgmap v11: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.600325+0000 mon.b (mon.2) 175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 43568332, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598613Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374551347, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598715Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"6edd7f37c961\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 39227228, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598767Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:54:05.680495Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48737812, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598816Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50457477, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598954Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45969571, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599060Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48685383, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599164Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24735907, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 13274972, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599322Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599372Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:11.599453Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.605465+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.607566+0000 mon.b (mon.2) 176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.611411+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.626651+0000 mon.b (mon.2) 177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.627854+0000 mon.b (mon.2) 178 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:12.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[30791]: audit 2021-03-10T13:54:11.629059+0000 mon.b (mon.2) 179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2021-03-10T13:54:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.600325+0000 mon.b (mon.2) 175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.gibba011","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"4c594a5db324\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 43568332, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598613Z\", \"created\": \"2021-03-10T13:42:43.644407Z\", \"started\": \"2021-03-10T13:53:48.900261Z\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"gibba011\", \"container_id\": \"db0a8bf316f6\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 374551347, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598715Z\", \"created\": \"2021-03-10T13:42:50.972407Z\", \"started\": \"2021-03-10T13:53:40.062962Z\"}, \"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"gibba011\", \"container_id\": \"6edd7f37c961\", \"container_image_id\": \"d4615e7085da621d1ad059caeee4303a51b89331a28166bbd090a93d091f1a14\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a\", \"container_image_digests\": [\"quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54\"], \"memory_usage\": 39227228, \"version\": \"17.0.0-1749-gb26fc080\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598767Z\", \"created\": \"2021-03-10T13:43:58.520406Z\", \"started\": \"2021-03-10T13:54:05.680495Z\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"gibba011\", \"container_id\": \"cc98e62351ea\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48737812, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598816Z\", \"created\": \"2021-03-10T13:44:46.556406Z\", \"started\": \"2021-03-10T13:44:49.809781Z\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"gibba011\", \"container_id\": \"913379f50d2d\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 50457477, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.598954Z\", \"created\": \"2021-03-10T13:45:09.440405Z\", \"started\": \"2021-03-10T13:45:12.629074Z\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"gibba011\", \"container_id\": \"1446073f7732\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 45969571, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599060Z\", \"created\": \"2021-03-10T13:45:32.562625Z\", \"started\": \"2021-03-10T13:45:35.953114Z\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"gibba011\", \"container_id\": \"61c9f8aa39e9\", \"container_image_id\": \"204a01f9b0b6710dd0c0af7f37ce7139c47ff0f0105d778d7104c69282dfbbf1\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"container_image_digests\": [\"ceph/ceph@sha256:54b064d6a616656497df507997f8cd47f96ec7cc137f18b69f74ea03a1467c60\"], \"memory_usage\": 48685383, \"version\": \"15.2.0\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599164Z\", \"created\": \"2021-03-10T13:45:56.793082Z\", \"started\": \"2021-03-10T13:45:59.870484Z\"}, \"node-exporter.a\": {\"daemon_type\": \"node-exporter\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"c7b51a252180\", \"container_image_id\": \"c19ae228f0699185488b6a1c0debb9c6b79672181356ad455c9a7924a41a01bb\", \"container_image_name\": \"prom/node-exporter\", \"container_image_digests\": [\"prom/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810\"], \"memory_usage\": 24735907, \"version\": \"1.1.2\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599266Z\", \"created\": \"2021-03-10T13:47:49.641081Z\", \"started\": \"2021-03-10T13:47:52.381825Z\"}, \"alertmanager.a\": {\"daemon_type\": \"alertmanager\", \"daemon_id\": \"a\", \"hostname\": \"gibba011\", \"container_id\": \"6c1e707dd209\", \"container_image_id\": \"c876f5897d7b07a19836227e26de9053c4ae14c114e43635df6fdb4a4b251ba6\", \"container_image_name\": \"prom/alertmanager\", \"container_image_digests\": [\"prom/alertmanager@sha256:24a5204b418e8fa0214cfb628486749003b039c279c56b5bddb5b10cd100d926\"], \"memory_usage\": 13274972, \"version\": \"0.21.0\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599322Z\", \"created\": \"2021-03-10T13:47:59.249081Z\", \"started\": \"2021-03-10T13:53:21.328777Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"daemon_type\": \"rgw\", \"daemon_id\": \"r.z.gibba011.xbljmx\", \"hostname\": \"gibba011\", \"container_image_name\": \"docker.io/ceph/ceph:v15.2.0\", \"status\": -1, \"status_desc\": \"error\", \"is_active\": false, \"last_refresh\": \"2021-03-10T13:54:11.599372Z\", \"created\": \"2021-03-10T13:49:55.661080Z\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEL1K375GA\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 375083606016.0, \"human_readable_size\": \"349.32 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c387e310-fde0-48e0-971a-0a016244271f\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"fz2AT6-yjAV-Wwtt-wD83-UOrD-LvSJ-TPDY1s\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c26948f0-d140-49d5-b246-b225db5e8c93\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"WUl72N-1sPU-My7q-9Nxz-Hh9b-RM04-ZECN75\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"1a81ef55-f29d-4fd7-81a8-384de101e141\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"0NLLwF-e44R-ApVF-uNRZ-uYdG-PkUW-wjGYSM\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"924baa6c-3f7e-44d7-a126-60ca7e5787ad\", \"cluster_fsid\": \"6fd36d94-81a6-11eb-9080-001a4aab830c\", \"block_uuid\": \"kpXaNn-GXTL-lt8E-6gzZ-hF72-LCNz-esfZy9\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"INTEL SSDPEL1K375GA_PHKM920000BW375A\", \"lsm_data\": {}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"TOSHIBA MG04ACA1\", \"rev\": \"FJ5A\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"64\", \"scheduler_mode\": \"mq-deadline\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF\", \"lsm_data\": {}}], \"osdspec_previews\": [], \"osdspec_last_applied\": {}, \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2021-03-10T13:54:02.564013Z\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:27.033516Z\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2021-03-10T13:44:44.834875Z\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:07.754784Z\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:30.750240Z\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2021-03-10T13:45:54.879098Z\"}, \"node-exporter.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:47:49.302785Z\"}, \"alertmanager.a\": {\"deps\": [\"alertmanager.a\", \"mgr.gibba032.whopeo\", \"mgr.x\", \"mgr.y\"], \"last_config\": \"2021-03-10T13:53:14.673458Z\"}, \"rgw.r.z.gibba011.xbljmx\": {\"deps\": [], \"last_config\": \"2021-03-10T13:49:52.361740Z\"}, \"mon.a\": {\"deps\": [], \"last_config\": \"2021-03-10T13:53:45.907569Z\"}}, \"last_daemon_update\": \"2021-03-10T13:54:11.599453Z\", \"last_device_update\": \"2021-03-10T13:46:03.134728Z\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.2.111\"], \"172.21.15.254\": [\"172.21.2.111\"]}, \"last_host_check\": \"2021-03-10T13:53:40.231797Z\", \"scheduled_daemon_actions\": {}}"}]: dispatch 2021-03-10T13:54:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.605465+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.607566+0000 mon.b (mon.2) 176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2021-03-10T13:54:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.611411+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.626651+0000 mon.b (mon.2) 177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.627854+0000 mon.b (mon.2) 178 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:11 gibba011 bash[31774]: audit 2021-03-10T13:54:11.629059+0000 mon.b (mon.2) 179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2021-03-10T13:54:13.149 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.626000+0000 mgr.gibba032.whopeo (mgr.24625) 21 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:13.150 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.627595+0000 mgr.gibba032.whopeo (mgr.24625) 22 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:13.150 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.628681+0000 mgr.gibba032.whopeo (mgr.24625) 23 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:13.150 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.628721+0000 mgr.gibba032.whopeo (mgr.24625) 24 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:13.151 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.629641+0000 mgr.gibba032.whopeo (mgr.24625) 25 : cephadm [INF] It is presumed safe to stop mon.b 2021-03-10T13:54:13.151 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:12 gibba032 bash[9829]: cephadm 2021-03-10T13:54:11.629680+0000 mgr.gibba032.whopeo (mgr.24625) 26 : cephadm [INF] Upgrade: It is presumed safe to stop mon.b 2021-03-10T13:54:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.626000+0000 mgr.gibba032.whopeo (mgr.24625) 21 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.627595+0000 mgr.gibba032.whopeo (mgr.24625) 22 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.628681+0000 mgr.gibba032.whopeo (mgr.24625) 23 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.628721+0000 mgr.gibba032.whopeo (mgr.24625) 24 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.629641+0000 mgr.gibba032.whopeo (mgr.24625) 25 : cephadm [INF] It is presumed safe to stop mon.b 2021-03-10T13:54:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[30791]: cephadm 2021-03-10T13:54:11.629680+0000 mgr.gibba032.whopeo (mgr.24625) 26 : cephadm [INF] Upgrade: It is presumed safe to stop mon.b 2021-03-10T13:54:13.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.626000+0000 mgr.gibba032.whopeo (mgr.24625) 21 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:13.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.627595+0000 mgr.gibba032.whopeo (mgr.24625) 22 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.628681+0000 mgr.gibba032.whopeo (mgr.24625) 23 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.628721+0000 mgr.gibba032.whopeo (mgr.24625) 24 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.629641+0000 mgr.gibba032.whopeo (mgr.24625) 25 : cephadm [INF] It is presumed safe to stop mon.b 2021-03-10T13:54:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:12 gibba011 bash[31774]: cephadm 2021-03-10T13:54:11.629680+0000 mgr.gibba032.whopeo (mgr.24625) 26 : cephadm [INF] Upgrade: It is presumed safe to stop mon.b 2021-03-10T13:54:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: cluster 2021-03-10T13:54:12.453651+0000 mgr.gibba032.whopeo (mgr.24625) 27 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.616824+0000 mon.b (mon.2) 180 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.b"}]: dispatch 2021-03-10T13:54:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.626403+0000 mon.a (mon.0) 50 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.628476+0000 mon.b (mon.2) 181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.629312+0000 mon.b (mon.2) 182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.630151+0000 mon.b (mon.2) 183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:13 gibba032 bash[9829]: audit 2021-03-10T13:54:13.631115+0000 mon.b (mon.2) 184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: cluster 2021-03-10T13:54:12.453651+0000 mgr.gibba032.whopeo (mgr.24625) 27 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.616824+0000 mon.b (mon.2) 180 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.b"}]: dispatch 2021-03-10T13:54:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.626403+0000 mon.a (mon.0) 50 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.628476+0000 mon.b (mon.2) 181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.629312+0000 mon.b (mon.2) 182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.630151+0000 mon.b (mon.2) 183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[30791]: audit 2021-03-10T13:54:13.631115+0000 mon.b (mon.2) 184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: cluster 2021-03-10T13:54:12.453651+0000 mgr.gibba032.whopeo (mgr.24625) 27 : cluster [DBG] pgmap v12: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.616824+0000 mon.b (mon.2) 180 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config set", "name": "container_image", "value": "quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a", "who": "mon.b"}]: dispatch 2021-03-10T13:54:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.626403+0000 mon.a (mon.0) 50 : audit [INF] from='mgr.24625 ' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.628476+0000 mon.b (mon.2) 181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.629312+0000 mon.b (mon.2) 182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.630151+0000 mon.b (mon.2) 183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:13 gibba011 bash[31774]: audit 2021-03-10T13:54:13.631115+0000 mon.b (mon.2) 184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:14 gibba032 bash[9829]: cephadm 2021-03-10T13:54:13.616471+0000 mgr.gibba032.whopeo (mgr.24625) 28 : cephadm [INF] Upgrade: Updating mon.b 2021-03-10T13:54:15.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:14 gibba032 bash[9829]: cephadm 2021-03-10T13:54:13.630813+0000 mgr.gibba032.whopeo (mgr.24625) 29 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:54:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:14 gibba011 bash[30791]: cephadm 2021-03-10T13:54:13.616471+0000 mgr.gibba032.whopeo (mgr.24625) 28 : cephadm [INF] Upgrade: Updating mon.b 2021-03-10T13:54:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:14 gibba011 bash[30791]: cephadm 2021-03-10T13:54:13.630813+0000 mgr.gibba032.whopeo (mgr.24625) 29 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:54:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:14 gibba011 bash[31774]: cephadm 2021-03-10T13:54:13.616471+0000 mgr.gibba032.whopeo (mgr.24625) 28 : cephadm [INF] Upgrade: Updating mon.b 2021-03-10T13:54:15.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:14 gibba011 bash[31774]: cephadm 2021-03-10T13:54:13.630813+0000 mgr.gibba032.whopeo (mgr.24625) 29 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:54:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:15 gibba032 systemd[1]: Stopping Ceph mon.b for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:54:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:15 gibba032 bash[9829]: debug 2021-03-10T13:54:15.494+0000 7f8865764700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:54:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:15 gibba032 bash[9829]: debug 2021-03-10T13:54:15.494+0000 7f8865764700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2021-03-10T13:54:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:16 gibba032 docker[23913]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-mon.b 2021-03-10T13:54:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:16 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@mon.b.service: Succeeded. 2021-03-10T13:54:17.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:16 gibba032 systemd[1]: Stopped Ceph mon.b for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:54:17.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:16 gibba032 systemd[1]: Started Ceph mon.b for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:54:17.730 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.722+0000 7f48951bc700 0 set uid:gid to 167:167 (ceph:ceph) 2021-03-10T13:54:17.730 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.722+0000 7f48951bc700 0 ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev), process ceph-mon, pid 6 2021-03-10T13:54:17.731 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.722+0000 7f48951bc700 0 pidfile_write: ignore empty --pid-file 2021-03-10T13:54:17.731 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 0 load: jerasure load: lrc load: isa 2021-03-10T13:54:17.731 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: RocksDB version: 6.8.1 2021-03-10T13:54:17.731 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-03-10T13:54:17.731 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Compile date Mar 9 2021 2021-03-10T13:54:17.732 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: DB SUMMARY 2021-03-10T13:54:17.732 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: CURRENT file: CURRENT 2021-03-10T13:54:17.732 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: IDENTITY file: IDENTITY 2021-03-10T13:54:17.732 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 782 Bytes 2021-03-10T13:54:17.732 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 2, files: 000022.sst 000023.sst 2021-03-10T13:54:17.733 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000020.log size: 21980187 ; 2021-03-10T13:54:17.733 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.error_if_exists: 0 2021-03-10T13:54:17.733 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.create_if_missing: 0 2021-03-10T13:54:17.733 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.paranoid_checks: 1 2021-03-10T13:54:17.734 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.env: 0x5634419e01c0 2021-03-10T13:54:17.734 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.fs: Posix File System 2021-03-10T13:54:17.734 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.info_log: 0x5634426ab1e0 2021-03-10T13:54:17.734 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_file_opening_threads: 16 2021-03-10T13:54:17.734 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.statistics: (nil) 2021-03-10T13:54:17.735 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.use_fsync: 0 2021-03-10T13:54:17.735 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_log_file_size: 0 2021-03-10T13:54:17.735 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-03-10T13:54:17.735 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-03-10T13:54:17.735 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.keep_log_file_num: 1000 2021-03-10T13:54:17.736 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.recycle_log_file_num: 0 2021-03-10T13:54:17.736 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.allow_fallocate: 1 2021-03-10T13:54:17.737 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.allow_mmap_reads: 0 2021-03-10T13:54:17.737 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.allow_mmap_writes: 0 2021-03-10T13:54:17.737 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.use_direct_reads: 0 2021-03-10T13:54:17.737 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-03-10T13:54:17.737 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.create_missing_column_families: 0 2021-03-10T13:54:17.738 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.db_log_dir: 2021-03-10T13:54:17.738 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2021-03-10T13:54:17.738 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-03-10T13:54:17.738 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_subcompactions: 1 2021-03-10T13:54:17.739 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_background_flushes: -1 2021-03-10T13:54:17.739 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-03-10T13:54:17.739 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-03-10T13:54:17.739 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-03-10T13:54:17.739 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-03-10T13:54:17.740 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-03-10T13:54:17.740 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.advise_random_on_open: 1 2021-03-10T13:54:17.740 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.db_write_buffer_size: 0 2021-03-10T13:54:17.740 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.write_buffer_manager: 0x5634426ffc50 2021-03-10T13:54:17.740 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-03-10T13:54:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-03-10T13:54:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-03-10T13:54:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-03-10T13:54:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.rate_limiter: (nil) 2021-03-10T13:54:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-03-10T13:54:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.wal_recovery_mode: 2 2021-03-10T13:54:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.enable_thread_tracking: 0 2021-03-10T13:54:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.enable_pipelined_write: 0 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.unordered_write: 0 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-03-10T13:54:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.row_cache: None 2021-03-10T13:54:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.wal_filter: None 2021-03-10T13:54:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-03-10T13:54:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.allow_ingest_behind: 0 2021-03-10T13:54:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.preserve_deletes: 0 2021-03-10T13:54:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.two_write_queues: 0 2021-03-10T13:54:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.manual_wal_flush: 0 2021-03-10T13:54:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.atomic_flush: 0 2021-03-10T13:54:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-03-10T13:54:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-03-10T13:54:18.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-03-10T13:54:18.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.log_readahead_size: 0 2021-03-10T13:54:18.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-03-10T13:54:18.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_background_jobs: 2 2021-03-10T13:54:18.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_background_compactions: -1 2021-03-10T13:54:18.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-03-10T13:54:18.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-03-10T13:54:18.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-03-10T13:54:18.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_total_wal_size: 0 2021-03-10T13:54:18.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-03-10T13:54:18.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-03-10T13:54:18.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-03-10T13:54:18.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-03-10T13:54:18.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_open_files: -1 2021-03-10T13:54:18.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bytes_per_sync: 0 2021-03-10T13:54:18.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-03-10T13:54:18.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-03-10T13:54:18.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_readahead_size: 0 2021-03-10T13:54:18.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Compression algorithms supported: 2021-03-10T13:54:18.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-03-10T13:54:18.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kZSTD supported: 0 2021-03-10T13:54:18.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kXpressCompression supported: 0 2021-03-10T13:54:18.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kLZ4HCCompression supported: 1 2021-03-10T13:54:18.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kLZ4Compression supported: 1 2021-03-10T13:54:18.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kBZip2Compression supported: 0 2021-03-10T13:54:18.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kZlibCompression supported: 1 2021-03-10T13:54:18.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: kSnappyCompression supported: 1 2021-03-10T13:54:18.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-03-10T13:54:18.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2021-03-10T13:54:18.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-03-10T13:54:18.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-03-10T13:54:18.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.merge_operator: 2021-03-10T13:54:18.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_filter: None 2021-03-10T13:54:18.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_filter_factory: None 2021-03-10T13:54:18.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-03-10T13:54:18.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-03-10T13:54:18.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56344264e230) 2021-03-10T13:54:18.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: cache_index_and_filter_blocks: 1 2021-03-10T13:54:18.021 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: cache_index_and_filter_blocks_with_high_priority: 0 2021-03-10T13:54:18.021 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-03-10T13:54:18.021 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: pin_top_level_index_and_filter: 1 2021-03-10T13:54:18.021 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: index_type: 0 2021-03-10T13:54:18.021 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: data_block_index_type: 0 2021-03-10T13:54:18.022 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: index_shortening: 1 2021-03-10T13:54:18.022 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: data_block_hash_table_util_ratio: 0.750000 2021-03-10T13:54:18.022 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: hash_index_allow_collision: 1 2021-03-10T13:54:18.022 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: checksum: 1 2021-03-10T13:54:18.023 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: no_block_cache: 0 2021-03-10T13:54:18.023 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_cache: 0x563442686510 2021-03-10T13:54:18.023 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_cache_name: BinnedLRUCache 2021-03-10T13:54:18.023 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_cache_options: 2021-03-10T13:54:18.023 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: capacity : 536870912 2021-03-10T13:54:18.024 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: num_shard_bits : 4 2021-03-10T13:54:18.024 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: strict_capacity_limit : 0 2021-03-10T13:54:18.024 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: high_pri_pool_ratio: 0.000 2021-03-10T13:54:18.024 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_cache_compressed: (nil) 2021-03-10T13:54:18.024 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: persistent_cache: (nil) 2021-03-10T13:54:18.025 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_size: 4096 2021-03-10T13:54:18.025 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_size_deviation: 10 2021-03-10T13:54:18.025 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_restart_interval: 16 2021-03-10T13:54:18.025 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: index_block_restart_interval: 1 2021-03-10T13:54:18.025 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: metadata_block_size: 4096 2021-03-10T13:54:18.026 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: partition_filters: 0 2021-03-10T13:54:18.026 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: use_delta_encoding: 1 2021-03-10T13:54:18.026 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: filter_policy: rocksdb.BuiltinBloomFilter 2021-03-10T13:54:18.026 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: whole_key_filtering: 1 2021-03-10T13:54:18.026 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: verify_compression: 0 2021-03-10T13:54:18.027 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: read_amp_bytes_per_bit: 0 2021-03-10T13:54:18.027 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: format_version: 2 2021-03-10T13:54:18.027 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: enable_index_compression: 1 2021-03-10T13:54:18.027 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: block_align: 0 2021-03-10T13:54:18.027 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.write_buffer_size: 33554432 2021-03-10T13:54:18.028 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_write_buffer_number: 2 2021-03-10T13:54:18.028 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression: NoCompression 2021-03-10T13:54:18.028 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression: Disabled 2021-03-10T13:54:18.028 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.prefix_extractor: nullptr 2021-03-10T13:54:18.029 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-03-10T13:54:18.029 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.num_levels: 7 2021-03-10T13:54:18.029 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-03-10T13:54:18.029 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-03-10T13:54:18.029 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-03-10T13:54:18.030 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-03-10T13:54:18.030 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-03-10T13:54:18.030 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-03-10T13:54:18.030 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-03-10T13:54:18.031 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:54:18.031 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-03-10T13:54:18.031 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-03-10T13:54:18.031 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.level: 32767 2021-03-10T13:54:18.031 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.strategy: 0 2021-03-10T13:54:18.032 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-03-10T13:54:18.032 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-03-10T13:54:18.032 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compression_opts.enabled: false 2021-03-10T13:54:18.032 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-03-10T13:54:18.032 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-03-10T13:54:18.033 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-03-10T13:54:18.033 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.target_file_size_base: 67108864 2021-03-10T13:54:18.033 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-03-10T13:54:18.033 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-03-10T13:54:18.034 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-03-10T13:54:18.034 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-03-10T13:54:18.034 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-03-10T13:54:18.034 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-03-10T13:54:18.035 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-03-10T13:54:18.035 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-03-10T13:54:18.035 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-03-10T13:54:18.035 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-03-10T13:54:18.035 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-03-10T13:54:18.036 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-03-10T13:54:18.036 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-03-10T13:54:18.036 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.arena_block_size: 4194304 2021-03-10T13:54:18.036 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-03-10T13:54:18.036 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-03-10T13:54:18.037 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-03-10T13:54:18.037 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.disable_auto_compactions: 0 2021-03-10T13:54:18.037 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-03-10T13:54:18.037 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-03-10T13:54:18.037 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-03-10T13:54:18.038 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-03-10T13:54:18.038 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-03-10T13:54:18.038 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-03-10T13:54:18.038 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-03-10T13:54:18.038 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-03-10T13:54:18.039 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-03-10T13:54:18.039 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-03-10T13:54:18.039 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.table_properties_collectors: 2021-03-10T13:54:18.039 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.inplace_update_support: 0 2021-03-10T13:54:18.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-03-10T13:54:18.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-03-10T13:54:18.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-03-10T13:54:18.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-03-10T13:54:18.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.bloom_locality: 0 2021-03-10T13:54:18.041 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.max_successive_merges: 0 2021-03-10T13:54:18.041 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-03-10T13:54:18.041 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.paranoid_file_checks: 0 2021-03-10T13:54:18.041 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.force_consistency_checks: 0 2021-03-10T13:54:18.041 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.report_bg_io_stats: 0 2021-03-10T13:54:18.042 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.ttl: 2592000 2021-03-10T13:54:18.042 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-03-10T13:54:18.042 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 25, last_sequence is 9181, log_number is 20,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-03-10T13:54:18.042 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 20 2021-03-10T13:54:18.043 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384457730119, "job": 1, "event": "recovery_started", "log_files": [20]} 2021-03-10T13:54:18.043 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.726+0000 7f48951bc700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #20 mode 2 2021-03-10T13:54:18.043 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.738+0000 7f48951bc700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-03-10T13:54:18.043 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f48951bc700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384457752851, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 25, "file_size": 14527688, "table_properties": {"data_size": 14520033, "index_size": 5999, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 837, "raw_key_size": 6699, "raw_average_key_size": 21, "raw_value_size": 14510904, "raw_average_value_size": 47113, "num_data_blocks": 188, "num_entries": 308, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1615384457, "oldest_key_time": 3, "file_creation_time": 0}} 2021-03-10T13:54:18.043 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f48951bc700 4 rocksdb: [version_set.cc:3826] Creating manifest 26 2021-03-10T13:54:18.044 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f48951bc700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1615384457753847, "job": 1, "event": "recovery_finished"} 2021-03-10T13:54:18.044 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f48951bc700 4 rocksdb: DB pointer 0x56344274b800 2021-03-10T13:54:18.044 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f487bfcf700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-03-10T13:54:18.044 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.750+0000 7f487bfcf700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-03-10T13:54:18.044 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** DB Stats ** 2021-03-10T13:54:18.045 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:18.045 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-03-10T13:54:18.045 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-03-10T13:54:18.045 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:54:18.045 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-03-10T13:54:18.046 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-03-10T13:54:18.046 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-03-10T13:54:18.046 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** Compaction Stats [default] ** 2021-03-10T13:54:18.046 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:18.047 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:18.047 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: L0 1/0 13.85 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.047 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:18.047 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Sum 3/0 77.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.047 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.048 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** Compaction Stats [default] ** 2021-03-10T13:54:18.048 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:18.048 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:18.048 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.049 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:18.049 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Flush(GB): cumulative 0.014, interval 0.014 2021-03-10T13:54:18.049 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:54:18.049 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:54:18.049 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:54:18.050 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:54:18.050 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Cumulative compaction: 0.01 GB write, 527.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:18.050 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Interval compaction: 0.01 GB write, 527.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:18.050 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:54:18.051 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:54:18.051 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** Compaction Stats [default] ** 2021-03-10T13:54:18.051 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:18.051 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:18.051 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: L0 1/0 13.85 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.052 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: L6 2/0 64.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:18.052 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Sum 3/0 77.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.052 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-03-10T13:54:18.052 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** Compaction Stats [default] ** 2021-03-10T13:54:18.053 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-03-10T13:54:18.053 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-03-10T13:54:18.053 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1167.1 0.01 0.00 1 0.012 0 0 2021-03-10T13:54:18.053 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Uptime(secs): 0.0 total, 0.0 interval 2021-03-10T13:54:18.053 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Flush(GB): cumulative 0.014, interval 0.000 2021-03-10T13:54:18.054 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(GB): cumulative 0.000, interval 0.000 2021-03-10T13:54:18.054 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(Total Files): cumulative 0, interval 0 2021-03-10T13:54:18.054 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(L0 Files): cumulative 0, interval 0 2021-03-10T13:54:18.054 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: AddFile(Keys): cumulative 0, interval 0 2021-03-10T13:54:18.054 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Cumulative compaction: 0.01 GB write, 526.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:18.055 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-03-10T13:54:18.055 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-03-10T13:54:18.055 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: ** File Read Latency Histogram By Level [default] ** 2021-03-10T13:54:18.055 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 starting mon.b rank 2 at public addrs [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] at bind addrs [v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:54:18.056 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 1 mon.b@-1(???) e3 preinit fsid 6fd36d94-81a6-11eb-9080-001a4aab830c 2021-03-10T13:54:18.056 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).mds e1 new map 2021-03-10T13:54:18.056 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).mds e1 print_map 2021-03-10T13:54:18.056 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: e1 2021-03-10T13:54:18.056 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: enable_multiple, ever_enabled_multiple: 0,0 2021-03-10T13:54:18.057 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-03-10T13:54:18.057 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: legacy client fscid: -1 2021-03-10T13:54:18.057 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: 2021-03-10T13:54:18.057 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: No filesystems configured 2021-03-10T13:54:18.057 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).osd e61 crush map has features 3314933000852226048, adjusting msgr requires 2021-03-10T13:54:18.058 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:18.058 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:18.058 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 0 mon.b@-1(???).osd e61 crush map has features 288514051259236352, adjusting msgr requires 2021-03-10T13:54:18.058 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: cephadm 2021-03-10T13:54:13.616471+0000 mgr.gibba032.whopeo (mgr.24625) 28 : cephadm [INF] Upgrade: Updating mon.b 2021-03-10T13:54:18.059 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: cephadm 2021-03-10T13:54:13.630813+0000 mgr.gibba032.whopeo (mgr.24625) 29 : cephadm [INF] Deploying daemon mon.b on gibba032 2021-03-10T13:54:18.059 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:17 gibba032 bash[24004]: debug 2021-03-10T13:54:17.754+0000 7f48951bc700 1 mon.b@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2021-03-10T13:54:19.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:18 gibba032 bash[24004]: cluster 2021-03-10T13:54:14.454551+0000 mgr.gibba032.whopeo (mgr.24625) 30 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[31774]: cluster 2021-03-10T13:54:14.454551+0000 mgr.gibba032.whopeo (mgr.24625) 30 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: cluster 2021-03-10T13:54:14.454551+0000 mgr.gibba032.whopeo (mgr.24625) 30 : cluster [DBG] pgmap v13: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: debug 2021-03-10T13:54:18.964+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: debug 2021-03-10T13:54:18.964+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: debug 2021-03-10T13:54:18.964+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: debug 2021-03-10T13:54:18.964+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:18 gibba011 bash[30791]: debug 2021-03-10T13:54:18.964+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.968+0000 7efe9b042700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.968+0000 7efe9b042700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.968+0000 7efe9b042700 -1 log_channel(cluster) log [ERR] : daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.968+0000 7efe9b042700 -1 log_channel(cluster) log [ERR] : [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.968+0000 7efe9b042700 -1 log_channel(cluster) log [ERR] : Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.988+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.988+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.988+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.988+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:23 gibba011 bash[30791]: debug 2021-03-10T13:54:23.988+0000 7efe9983f700 -1 log_channel(cluster) log [ERR] : Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.457684+0000 mgr.gibba032.whopeo (mgr.24625) 32 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.968836+0000 mon.a (mon.0) 57 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.968846+0000 mon.a (mon.0) 58 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.968852+0000 mon.a (mon.0) 59 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.968856+0000 mon.a (mon.0) 60 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.968861+0000 mon.a (mon.0) 61 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.969266+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.969280+0000 mon.a (mon.0) 62 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:18.970685+0000 mon.b (mon.2) 2 : cluster [INF] mon.b calling monitor election 2021-03-10T13:54:24.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:20.458851+0000 mgr.gibba032.whopeo (mgr.24625) 33 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:22.461029+0000 mgr.gibba032.whopeo (mgr.24625) 34 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.457684+0000 mgr.gibba032.whopeo (mgr.24625) 32 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.968836+0000 mon.a (mon.0) 57 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.968846+0000 mon.a (mon.0) 58 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.968852+0000 mon.a (mon.0) 59 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.968856+0000 mon.a (mon.0) 60 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.968861+0000 mon.a (mon.0) 61 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.969266+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.969280+0000 mon.a (mon.0) 62 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:18.970685+0000 mon.b (mon.2) 2 : cluster [INF] mon.b calling monitor election 2021-03-10T13:54:24.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:20.458851+0000 mgr.gibba032.whopeo (mgr.24625) 33 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:22.461029+0000 mgr.gibba032.whopeo (mgr.24625) 34 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.971051+0000 mon.a (mon.0) 63 : cluster [INF] mon.a is new leader, mons a,c in quorum (ranks 0,1) 2021-03-10T13:54:24.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974125+0000 mon.a (mon.0) 64 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974204+0000 mon.a (mon.0) 65 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974241+0000 mon.a (mon.0) 66 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974273+0000 mon.a (mon.0) 67 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974331+0000 mon.a (mon.0) 68 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.974474+0000 mon.a (mon.0) 69 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.975150+0000 mon.c (mon.1) 3 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.980552+0000 mon.a (mon.0) 70 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:24.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.990729+0000 mon.a (mon.0) 71 : cluster [DBG] monmap e4: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:24.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.990757+0000 mon.a (mon.0) 72 : cluster [DBG] fsmap 2021-03-10T13:54:24.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.990774+0000 mon.a (mon.0) 73 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:24.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.991069+0000 mon.a (mon.0) 74 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 28s), standbys: x, y 2021-03-10T13:54:24.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.994825+0000 mon.a (mon.0) 75 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.994860+0000 mon.a (mon.0) 76 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.994865+0000 mon.a (mon.0) 77 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.994893+0000 mon.a (mon.0) 78 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: cluster 2021-03-10T13:54:23.994898+0000 mon.a (mon.0) 79 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: audit 2021-03-10T13:54:24.002222+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:24.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[31774]: audit 2021-03-10T13:54:24.003440+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:24.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.971051+0000 mon.a (mon.0) 63 : cluster [INF] mon.a is new leader, mons a,c in quorum (ranks 0,1) 2021-03-10T13:54:24.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974125+0000 mon.a (mon.0) 64 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974204+0000 mon.a (mon.0) 65 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974241+0000 mon.a (mon.0) 66 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974273+0000 mon.a (mon.0) 67 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974331+0000 mon.a (mon.0) 68 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.974474+0000 mon.a (mon.0) 69 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.975150+0000 mon.c (mon.1) 3 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.980552+0000 mon.a (mon.0) 70 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:24.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.990729+0000 mon.a (mon.0) 71 : cluster [DBG] monmap e4: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:24.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.990757+0000 mon.a (mon.0) 72 : cluster [DBG] fsmap 2021-03-10T13:54:24.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.990774+0000 mon.a (mon.0) 73 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:24.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.991069+0000 mon.a (mon.0) 74 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 28s), standbys: x, y 2021-03-10T13:54:24.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.994825+0000 mon.a (mon.0) 75 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.994860+0000 mon.a (mon.0) 76 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.994865+0000 mon.a (mon.0) 77 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.994893+0000 mon.a (mon.0) 78 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: cluster 2021-03-10T13:54:23.994898+0000 mon.a (mon.0) 79 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: audit 2021-03-10T13:54:24.002222+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:24.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:24 gibba011 bash[30791]: audit 2021-03-10T13:54:24.003440+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.457684+0000 mgr.gibba032.whopeo (mgr.24625) 32 : cluster [DBG] pgmap v15: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.968836+0000 mon.a (mon.0) 57 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.968846+0000 mon.a (mon.0) 58 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.968852+0000 mon.a (mon.0) 59 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.968856+0000 mon.a (mon.0) 60 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.968861+0000 mon.a (mon.0) 61 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.969266+0000 mon.c (mon.1) 2 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.969280+0000 mon.a (mon.0) 62 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:18.970685+0000 mon.b (mon.2) 2 : cluster [INF] mon.b calling monitor election 2021-03-10T13:54:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:20.458851+0000 mgr.gibba032.whopeo (mgr.24625) 33 : cluster [DBG] pgmap v16: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:22.461029+0000 mgr.gibba032.whopeo (mgr.24625) 34 : cluster [DBG] pgmap v17: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.971051+0000 mon.a (mon.0) 63 : cluster [INF] mon.a is new leader, mons a,c in quorum (ranks 0,1) 2021-03-10T13:54:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974125+0000 mon.a (mon.0) 64 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974204+0000 mon.a (mon.0) 65 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974241+0000 mon.a (mon.0) 66 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974273+0000 mon.a (mon.0) 67 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974331+0000 mon.a (mon.0) 68 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.974474+0000 mon.a (mon.0) 69 : cluster [INF] mon.a calling monitor election 2021-03-10T13:54:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.975150+0000 mon.c (mon.1) 3 : cluster [INF] mon.c calling monitor election 2021-03-10T13:54:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.980552+0000 mon.a (mon.0) 70 : cluster [INF] mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2021-03-10T13:54:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.990729+0000 mon.a (mon.0) 71 : cluster [DBG] monmap e4: 3 mons at {a=[v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0],b=[v2:172.21.2.132:3300/0,v1:172.21.2.132:6789/0],c=[v2:172.21.2.111:3301/0,v1:172.21.2.111:6790/0]} 2021-03-10T13:54:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.990757+0000 mon.a (mon.0) 72 : cluster [DBG] fsmap 2021-03-10T13:54:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.990774+0000 mon.a (mon.0) 73 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2021-03-10T13:54:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.991069+0000 mon.a (mon.0) 74 : cluster [DBG] mgrmap e25: gibba032.whopeo(active, since 28s), standbys: x, y 2021-03-10T13:54:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.994825+0000 mon.a (mon.0) 75 : cluster [ERR] Health detail: HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.994860+0000 mon.a (mon.0) 76 : cluster [ERR] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-03-10T13:54:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.994865+0000 mon.a (mon.0) 77 : cluster [ERR] daemon rgw.r.z.gibba011.xbljmx on gibba011 is in error state 2021-03-10T13:54:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.994893+0000 mon.a (mon.0) 78 : cluster [ERR] [ERR] MGR_MODULE_ERROR: Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: cluster 2021-03-10T13:54:23.994898+0000 mon.a (mon.0) 79 : cluster [ERR] Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T13:54:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: audit 2021-03-10T13:54:24.002222+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:24 gibba032 bash[24004]: audit 2021-03-10T13:54:24.003440+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:54:24.626 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:54:24.856 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:54:24.856 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (63s) 13s ago 6m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:54:24.856 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (6m) 24s ago 6m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:54:24.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (84s) 24s ago 9m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:54:24.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (101s) 24s ago 9m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:54:24.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (44s) 13s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:54:24.857 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (35s) 13s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:54:24.858 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 starting - - 2021-03-10T13:54:24.858 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (19s) 13s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:54:24.858 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (6m) 13s ago 6m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:54:24.858 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (6m) 24s ago 6m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:54:24.858 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (9m) 13s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:54:24.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (9m) 13s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:54:24.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (8m) 13s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:54:24.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (8m) 13s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:54:24.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (8m) 24s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:54:24.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (7m) 24s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:54:24.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (7m) 24s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:54:24.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (6m) 24s ago 6m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:54:24.860 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (5m) 24s ago 6m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:54:24.860 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 13s ago 4m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:54:25.104 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:54:25.104 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:54:25.104 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:54:25.105 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:25.105 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:54:25.105 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:54:25.105 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:25.105 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:54:25.106 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:54:25.106 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:25.106 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:54:25.106 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:54:25.107 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8, 2021-03-10T13:54:25.107 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 6 2021-03-10T13:54:25.107 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:54:25.107 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:54:26.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[30791]: cluster 2021-03-10T13:54:24.462052+0000 mgr.gibba032.whopeo (mgr.24625) 35 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:26.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[30791]: audit 2021-03-10T13:54:24.616875+0000 mgr.gibba032.whopeo (mgr.24625) 36 : audit [DBG] from='client.54103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[30791]: audit 2021-03-10T13:54:24.844657+0000 mgr.gibba032.whopeo (mgr.24625) 37 : audit [DBG] from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[30791]: audit 2021-03-10T13:54:25.103141+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3106368949' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[31774]: cluster 2021-03-10T13:54:24.462052+0000 mgr.gibba032.whopeo (mgr.24625) 35 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[31774]: audit 2021-03-10T13:54:24.616875+0000 mgr.gibba032.whopeo (mgr.24625) 36 : audit [DBG] from='client.54103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[31774]: audit 2021-03-10T13:54:24.844657+0000 mgr.gibba032.whopeo (mgr.24625) 37 : audit [DBG] from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:26 gibba011 bash[31774]: audit 2021-03-10T13:54:25.103141+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3106368949' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:26 gibba032 bash[24004]: cluster 2021-03-10T13:54:24.462052+0000 mgr.gibba032.whopeo (mgr.24625) 35 : cluster [DBG] pgmap v18: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:26 gibba032 bash[24004]: audit 2021-03-10T13:54:24.616875+0000 mgr.gibba032.whopeo (mgr.24625) 36 : audit [DBG] from='client.54103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:26 gibba032 bash[24004]: audit 2021-03-10T13:54:24.844657+0000 mgr.gibba032.whopeo (mgr.24625) 37 : audit [DBG] from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:26 gibba032 bash[24004]: audit 2021-03-10T13:54:25.103141+0000 mon.c (mon.1) 4 : audit [DBG] from='client.? 172.21.2.111:0/3106368949' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: cluster 2021-03-10T13:54:26.463155+0000 mgr.gibba032.whopeo (mgr.24625) 38 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.841524+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.846282+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.850082+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.851617+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.853183+0000 mon.a (mon.0) 86 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[30791]: audit 2021-03-10T13:54:27.855086+0000 mon.a (mon.0) 87 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:54:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: cluster 2021-03-10T13:54:26.463155+0000 mgr.gibba032.whopeo (mgr.24625) 38 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.841524+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.846282+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.850082+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.851617+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.853183+0000 mon.a (mon.0) 86 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:28 gibba011 bash[31774]: audit 2021-03-10T13:54:27.855086+0000 mon.a (mon.0) 87 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:54:28.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: cluster 2021-03-10T13:54:26.463155+0000 mgr.gibba032.whopeo (mgr.24625) 38 : cluster [DBG] pgmap v19: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.841524+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.846282+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.850082+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.851617+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.853183+0000 mon.a (mon.0) 86 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:28 gibba032 bash[24004]: audit 2021-03-10T13:54:27.855086+0000 mon.a (mon.0) 87 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.a","key": "container_image"}]: dispatch 2021-03-10T13:54:29.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:29 gibba011 bash[30791]: cephadm 2021-03-10T13:54:27.849952+0000 mgr.gibba032.whopeo (mgr.24625) 39 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2021-03-10T13:54:29.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:29 gibba011 bash[30791]: cephadm 2021-03-10T13:54:27.854886+0000 mgr.gibba032.whopeo (mgr.24625) 40 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:54:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:29 gibba011 bash[31774]: cephadm 2021-03-10T13:54:27.849952+0000 mgr.gibba032.whopeo (mgr.24625) 39 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2021-03-10T13:54:29.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:29 gibba011 bash[31774]: cephadm 2021-03-10T13:54:27.854886+0000 mgr.gibba032.whopeo (mgr.24625) 40 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:54:29.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:29 gibba032 bash[24004]: cephadm 2021-03-10T13:54:27.849952+0000 mgr.gibba032.whopeo (mgr.24625) 39 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2021-03-10T13:54:29.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:29 gibba032 bash[24004]: cephadm 2021-03-10T13:54:27.854886+0000 mgr.gibba032.whopeo (mgr.24625) 40 : cephadm [INF] Reconfiguring daemon mon.a on gibba011 2021-03-10T13:54:30.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: cluster 2021-03-10T13:54:28.465181+0000 mgr.gibba032.whopeo (mgr.24625) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:30.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: audit 2021-03-10T13:54:29.752520+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:30.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: audit 2021-03-10T13:54:29.755314+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: audit 2021-03-10T13:54:29.757431+0000 mon.a (mon.0) 90 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: audit 2021-03-10T13:54:29.758850+0000 mon.a (mon.0) 91 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[30791]: audit 2021-03-10T13:54:29.760099+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:54:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: cluster 2021-03-10T13:54:28.465181+0000 mgr.gibba032.whopeo (mgr.24625) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: audit 2021-03-10T13:54:29.752520+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: audit 2021-03-10T13:54:29.755314+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: audit 2021-03-10T13:54:29.757431+0000 mon.a (mon.0) 90 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: audit 2021-03-10T13:54:29.758850+0000 mon.a (mon.0) 91 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:30.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:30 gibba011 bash[31774]: audit 2021-03-10T13:54:29.760099+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:54:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: cluster 2021-03-10T13:54:28.465181+0000 mgr.gibba032.whopeo (mgr.24625) 41 : cluster [DBG] pgmap v20: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: audit 2021-03-10T13:54:29.752520+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: audit 2021-03-10T13:54:29.755314+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: audit 2021-03-10T13:54:29.757431+0000 mon.a (mon.0) 90 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: audit 2021-03-10T13:54:29.758850+0000 mon.a (mon.0) 91 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:30 gibba032 bash[24004]: audit 2021-03-10T13:54:29.760099+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.y","key": "container_image"}]: dispatch 2021-03-10T13:54:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:31 gibba011 bash[30791]: cephadm 2021-03-10T13:54:29.755144+0000 mgr.gibba032.whopeo (mgr.24625) 42 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2021-03-10T13:54:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:31 gibba011 bash[30791]: cephadm 2021-03-10T13:54:29.760235+0000 mgr.gibba032.whopeo (mgr.24625) 43 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:54:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:31 gibba011 bash[31774]: cephadm 2021-03-10T13:54:29.755144+0000 mgr.gibba032.whopeo (mgr.24625) 42 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2021-03-10T13:54:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:31 gibba011 bash[31774]: cephadm 2021-03-10T13:54:29.760235+0000 mgr.gibba032.whopeo (mgr.24625) 43 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:54:31.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:31 gibba032 bash[24004]: cephadm 2021-03-10T13:54:29.755144+0000 mgr.gibba032.whopeo (mgr.24625) 42 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2021-03-10T13:54:31.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:31 gibba032 bash[24004]: cephadm 2021-03-10T13:54:29.760235+0000 mgr.gibba032.whopeo (mgr.24625) 43 : cephadm [INF] Reconfiguring daemon mgr.y on gibba011 2021-03-10T13:54:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: cluster 2021-03-10T13:54:30.466252+0000 mgr.gibba032.whopeo (mgr.24625) 44 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: audit 2021-03-10T13:54:31.529294+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: audit 2021-03-10T13:54:31.530583+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: audit 2021-03-10T13:54:31.531714+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: audit 2021-03-10T13:54:31.532573+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:32 gibba032 bash[24004]: audit 2021-03-10T13:54:31.533754+0000 mon.a (mon.0) 97 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: cluster 2021-03-10T13:54:30.466252+0000 mgr.gibba032.whopeo (mgr.24625) 44 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: audit 2021-03-10T13:54:31.529294+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: audit 2021-03-10T13:54:31.530583+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: audit 2021-03-10T13:54:31.531714+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: audit 2021-03-10T13:54:31.532573+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[31774]: audit 2021-03-10T13:54:31.533754+0000 mon.a (mon.0) 97 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:32.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: cluster 2021-03-10T13:54:30.466252+0000 mgr.gibba032.whopeo (mgr.24625) 44 : cluster [DBG] pgmap v21: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:32.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: audit 2021-03-10T13:54:31.529294+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:32.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: audit 2021-03-10T13:54:31.530583+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:32.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: audit 2021-03-10T13:54:31.531714+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:32.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: audit 2021-03-10T13:54:31.532573+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:32.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:32 gibba011 bash[30791]: audit 2021-03-10T13:54:31.533754+0000 mon.a (mon.0) 97 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.c","key": "container_image"}]: dispatch 2021-03-10T13:54:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:33 gibba011 bash[30791]: cephadm 2021-03-10T13:54:31.530791+0000 mgr.gibba032.whopeo (mgr.24625) 45 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:54:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:33 gibba011 bash[30791]: cephadm 2021-03-10T13:54:31.533875+0000 mgr.gibba032.whopeo (mgr.24625) 46 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:54:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:33 gibba011 bash[31774]: cephadm 2021-03-10T13:54:31.530791+0000 mgr.gibba032.whopeo (mgr.24625) 45 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:54:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:33 gibba011 bash[31774]: cephadm 2021-03-10T13:54:31.533875+0000 mgr.gibba032.whopeo (mgr.24625) 46 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:54:33.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:33 gibba032 bash[24004]: cephadm 2021-03-10T13:54:31.530791+0000 mgr.gibba032.whopeo (mgr.24625) 45 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2021-03-10T13:54:33.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:33 gibba032 bash[24004]: cephadm 2021-03-10T13:54:31.533875+0000 mgr.gibba032.whopeo (mgr.24625) 46 : cephadm [INF] Reconfiguring daemon mon.c on gibba011 2021-03-10T13:54:34.481 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[31774]: cluster 2021-03-10T13:54:32.468182+0000 mgr.gibba032.whopeo (mgr.24625) 47 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:34.481 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[31774]: audit 2021-03-10T13:54:33.380521+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:34.481 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[31774]: audit 2021-03-10T13:54:33.385967+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:34.482 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[30791]: cluster 2021-03-10T13:54:32.468182+0000 mgr.gibba032.whopeo (mgr.24625) 47 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:34.482 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[30791]: audit 2021-03-10T13:54:33.380521+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:34.482 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:34 gibba011 bash[30791]: audit 2021-03-10T13:54:33.385967+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:34.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:34 gibba032 bash[24004]: cluster 2021-03-10T13:54:32.468182+0000 mgr.gibba032.whopeo (mgr.24625) 47 : cluster [DBG] pgmap v22: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:34.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:34 gibba032 bash[24004]: audit 2021-03-10T13:54:33.380521+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:34 gibba032 bash[24004]: audit 2021-03-10T13:54:33.385967+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:35.474 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:35 gibba011 bash[30791]: cephadm 2021-03-10T13:54:33.383176+0000 mgr.gibba032.whopeo (mgr.24625) 48 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2021-03-10T13:54:35.474 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:35 gibba011 bash[30791]: cephadm 2021-03-10T13:54:33.385477+0000 mgr.gibba032.whopeo (mgr.24625) 49 : cephadm [INF] Reconfiguring daemon osd.0 on gibba011 2021-03-10T13:54:35.475 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:35 gibba011 bash[31774]: cephadm 2021-03-10T13:54:33.383176+0000 mgr.gibba032.whopeo (mgr.24625) 48 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2021-03-10T13:54:35.475 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:35 gibba011 bash[31774]: cephadm 2021-03-10T13:54:33.385477+0000 mgr.gibba032.whopeo (mgr.24625) 49 : cephadm [INF] Reconfiguring daemon osd.0 on gibba011 2021-03-10T13:54:35.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:35 gibba032 bash[24004]: cephadm 2021-03-10T13:54:33.383176+0000 mgr.gibba032.whopeo (mgr.24625) 48 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2021-03-10T13:54:35.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:35 gibba032 bash[24004]: cephadm 2021-03-10T13:54:33.385477+0000 mgr.gibba032.whopeo (mgr.24625) 49 : cephadm [INF] Reconfiguring daemon osd.0 on gibba011 2021-03-10T13:54:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[30791]: cluster 2021-03-10T13:54:34.469344+0000 mgr.gibba032.whopeo (mgr.24625) 50 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[30791]: audit 2021-03-10T13:54:35.292699+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[30791]: cephadm 2021-03-10T13:54:35.295357+0000 mgr.gibba032.whopeo (mgr.24625) 51 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2021-03-10T13:54:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[30791]: cephadm 2021-03-10T13:54:35.297624+0000 mgr.gibba032.whopeo (mgr.24625) 52 : cephadm [INF] Reconfiguring daemon osd.1 on gibba011 2021-03-10T13:54:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[30791]: audit 2021-03-10T13:54:35.298402+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:54:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[31774]: cluster 2021-03-10T13:54:34.469344+0000 mgr.gibba032.whopeo (mgr.24625) 50 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[31774]: audit 2021-03-10T13:54:35.292699+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[31774]: cephadm 2021-03-10T13:54:35.295357+0000 mgr.gibba032.whopeo (mgr.24625) 51 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2021-03-10T13:54:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[31774]: cephadm 2021-03-10T13:54:35.297624+0000 mgr.gibba032.whopeo (mgr.24625) 52 : cephadm [INF] Reconfiguring daemon osd.1 on gibba011 2021-03-10T13:54:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:36 gibba011 bash[31774]: audit 2021-03-10T13:54:35.298402+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:54:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:36 gibba032 bash[24004]: cluster 2021-03-10T13:54:34.469344+0000 mgr.gibba032.whopeo (mgr.24625) 50 : cluster [DBG] pgmap v23: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:36 gibba032 bash[24004]: audit 2021-03-10T13:54:35.292699+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:36 gibba032 bash[24004]: cephadm 2021-03-10T13:54:35.295357+0000 mgr.gibba032.whopeo (mgr.24625) 51 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2021-03-10T13:54:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:36 gibba032 bash[24004]: cephadm 2021-03-10T13:54:35.297624+0000 mgr.gibba032.whopeo (mgr.24625) 52 : cephadm [INF] Reconfiguring daemon osd.1 on gibba011 2021-03-10T13:54:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:36 gibba032 bash[24004]: audit 2021-03-10T13:54:35.298402+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:54:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[30791]: cluster 2021-03-10T13:54:36.470502+0000 mgr.gibba032.whopeo (mgr.24625) 53 : cluster [DBG] pgmap v24: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[30791]: audit 2021-03-10T13:54:37.049968+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[30791]: cephadm 2021-03-10T13:54:37.052524+0000 mgr.gibba032.whopeo (mgr.24625) 54 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2021-03-10T13:54:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[30791]: cephadm 2021-03-10T13:54:37.054736+0000 mgr.gibba032.whopeo (mgr.24625) 55 : cephadm [INF] Reconfiguring daemon osd.2 on gibba011 2021-03-10T13:54:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[30791]: audit 2021-03-10T13:54:37.055362+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:54:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[31774]: cluster 2021-03-10T13:54:36.470502+0000 mgr.gibba032.whopeo (mgr.24625) 53 : cluster [DBG] pgmap v24: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[31774]: audit 2021-03-10T13:54:37.049968+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[31774]: cephadm 2021-03-10T13:54:37.052524+0000 mgr.gibba032.whopeo (mgr.24625) 54 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2021-03-10T13:54:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[31774]: cephadm 2021-03-10T13:54:37.054736+0000 mgr.gibba032.whopeo (mgr.24625) 55 : cephadm [INF] Reconfiguring daemon osd.2 on gibba011 2021-03-10T13:54:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:38 gibba011 bash[31774]: audit 2021-03-10T13:54:37.055362+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:54:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:38 gibba032 bash[24004]: cluster 2021-03-10T13:54:36.470502+0000 mgr.gibba032.whopeo (mgr.24625) 53 : cluster [DBG] pgmap v24: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:38 gibba032 bash[24004]: audit 2021-03-10T13:54:37.049968+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:38 gibba032 bash[24004]: cephadm 2021-03-10T13:54:37.052524+0000 mgr.gibba032.whopeo (mgr.24625) 54 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2021-03-10T13:54:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:38 gibba032 bash[24004]: cephadm 2021-03-10T13:54:37.054736+0000 mgr.gibba032.whopeo (mgr.24625) 55 : cephadm [INF] Reconfiguring daemon osd.2 on gibba011 2021-03-10T13:54:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:38 gibba032 bash[24004]: audit 2021-03-10T13:54:37.055362+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:54:40.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:39 gibba032 bash[24004]: cluster 2021-03-10T13:54:38.472548+0000 mgr.gibba032.whopeo (mgr.24625) 56 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:40.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:39 gibba032 bash[24004]: audit 2021-03-10T13:54:38.840055+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:39 gibba032 bash[24004]: cephadm 2021-03-10T13:54:38.842661+0000 mgr.gibba032.whopeo (mgr.24625) 57 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2021-03-10T13:54:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:39 gibba032 bash[24004]: cephadm 2021-03-10T13:54:38.844912+0000 mgr.gibba032.whopeo (mgr.24625) 58 : cephadm [INF] Reconfiguring daemon osd.3 on gibba011 2021-03-10T13:54:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:39 gibba032 bash[24004]: audit 2021-03-10T13:54:38.845444+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:54:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[30791]: cluster 2021-03-10T13:54:38.472548+0000 mgr.gibba032.whopeo (mgr.24625) 56 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[30791]: audit 2021-03-10T13:54:38.840055+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[30791]: cephadm 2021-03-10T13:54:38.842661+0000 mgr.gibba032.whopeo (mgr.24625) 57 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2021-03-10T13:54:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[30791]: cephadm 2021-03-10T13:54:38.844912+0000 mgr.gibba032.whopeo (mgr.24625) 58 : cephadm [INF] Reconfiguring daemon osd.3 on gibba011 2021-03-10T13:54:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[30791]: audit 2021-03-10T13:54:38.845444+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:54:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[31774]: cluster 2021-03-10T13:54:38.472548+0000 mgr.gibba032.whopeo (mgr.24625) 56 : cluster [DBG] pgmap v25: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[31774]: audit 2021-03-10T13:54:38.840055+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[31774]: cephadm 2021-03-10T13:54:38.842661+0000 mgr.gibba032.whopeo (mgr.24625) 57 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2021-03-10T13:54:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[31774]: cephadm 2021-03-10T13:54:38.844912+0000 mgr.gibba032.whopeo (mgr.24625) 58 : cephadm [INF] Reconfiguring daemon osd.3 on gibba011 2021-03-10T13:54:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:39 gibba011 bash[31774]: audit 2021-03-10T13:54:38.845444+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:54:42.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: cluster 2021-03-10T13:54:40.473566+0000 mgr.gibba032.whopeo (mgr.24625) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: audit 2021-03-10T13:54:40.613641+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: cephadm 2021-03-10T13:54:40.618575+0000 mgr.gibba032.whopeo (mgr.24625) 60 : cephadm [INF] Reconfiguring rgw.r.z.gibba011.xbljmx (monmap changed)... 2021-03-10T13:54:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: audit 2021-03-10T13:54:40.618836+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2021-03-10T13:54:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: audit 2021-03-10T13:54:40.620028+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "client.rgw.r.z.gibba011.xbljmx"}]: dispatch 2021-03-10T13:54:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: audit 2021-03-10T13:54:40.620953+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: audit 2021-03-10T13:54:40.622028+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "client.rgw.r.z.gibba011.xbljmx","key": "container_image"}]: dispatch 2021-03-10T13:54:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:41 gibba032 bash[24004]: cephadm 2021-03-10T13:54:40.622172+0000 mgr.gibba032.whopeo (mgr.24625) 61 : cephadm [INF] Reconfiguring daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:54:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: cluster 2021-03-10T13:54:40.473566+0000 mgr.gibba032.whopeo (mgr.24625) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: audit 2021-03-10T13:54:40.613641+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: cephadm 2021-03-10T13:54:40.618575+0000 mgr.gibba032.whopeo (mgr.24625) 60 : cephadm [INF] Reconfiguring rgw.r.z.gibba011.xbljmx (monmap changed)... 2021-03-10T13:54:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: audit 2021-03-10T13:54:40.618836+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2021-03-10T13:54:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: audit 2021-03-10T13:54:40.620028+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "client.rgw.r.z.gibba011.xbljmx"}]: dispatch 2021-03-10T13:54:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: audit 2021-03-10T13:54:40.620953+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: audit 2021-03-10T13:54:40.622028+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "client.rgw.r.z.gibba011.xbljmx","key": "container_image"}]: dispatch 2021-03-10T13:54:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[30791]: cephadm 2021-03-10T13:54:40.622172+0000 mgr.gibba032.whopeo (mgr.24625) 61 : cephadm [INF] Reconfiguring daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:54:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: cluster 2021-03-10T13:54:40.473566+0000 mgr.gibba032.whopeo (mgr.24625) 59 : cluster [DBG] pgmap v26: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: audit 2021-03-10T13:54:40.613641+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: cephadm 2021-03-10T13:54:40.618575+0000 mgr.gibba032.whopeo (mgr.24625) 60 : cephadm [INF] Reconfiguring rgw.r.z.gibba011.xbljmx (monmap changed)... 2021-03-10T13:54:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: audit 2021-03-10T13:54:40.618836+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.r.z.gibba011.xbljmx", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2021-03-10T13:54:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: audit 2021-03-10T13:54:40.620028+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "client.rgw.r.z.gibba011.xbljmx"}]: dispatch 2021-03-10T13:54:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: audit 2021-03-10T13:54:40.620953+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: audit 2021-03-10T13:54:40.622028+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "client.rgw.r.z.gibba011.xbljmx","key": "container_image"}]: dispatch 2021-03-10T13:54:42.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:41 gibba011 bash[31774]: cephadm 2021-03-10T13:54:40.622172+0000 mgr.gibba032.whopeo (mgr.24625) 61 : cephadm [INF] Reconfiguring daemon rgw.r.z.gibba011.xbljmx on gibba011 2021-03-10T13:54:43.697 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: audit 2021-03-10T13:54:42.424690+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:43.698 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: cephadm 2021-03-10T13:54:42.427231+0000 mgr.gibba032.whopeo (mgr.24625) 62 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:54:43.698 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: audit 2021-03-10T13:54:42.427365+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:43.698 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: audit 2021-03-10T13:54:42.429507+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:43.699 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: audit 2021-03-10T13:54:42.431031+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:43.699 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: audit 2021-03-10T13:54:42.432340+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:43.699 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: cephadm 2021-03-10T13:54:42.432461+0000 mgr.gibba032.whopeo (mgr.24625) 63 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:54:43.699 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:43 gibba032 bash[24004]: cluster 2021-03-10T13:54:42.474148+0000 mgr.gibba032.whopeo (mgr.24625) 64 : cluster [DBG] pgmap v27: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: audit 2021-03-10T13:54:42.424690+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: cephadm 2021-03-10T13:54:42.427231+0000 mgr.gibba032.whopeo (mgr.24625) 62 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:54:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: audit 2021-03-10T13:54:42.427365+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: audit 2021-03-10T13:54:42.429507+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: audit 2021-03-10T13:54:42.431031+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: audit 2021-03-10T13:54:42.432340+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: cephadm 2021-03-10T13:54:42.432461+0000 mgr.gibba032.whopeo (mgr.24625) 63 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:54:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[30791]: cluster 2021-03-10T13:54:42.474148+0000 mgr.gibba032.whopeo (mgr.24625) 64 : cluster [DBG] pgmap v27: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: audit 2021-03-10T13:54:42.424690+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: cephadm 2021-03-10T13:54:42.427231+0000 mgr.gibba032.whopeo (mgr.24625) 62 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2021-03-10T13:54:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: audit 2021-03-10T13:54:42.427365+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-03-10T13:54:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: audit 2021-03-10T13:54:42.429507+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-03-10T13:54:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: audit 2021-03-10T13:54:42.431031+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: audit 2021-03-10T13:54:42.432340+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon.b","key": "container_image"}]: dispatch 2021-03-10T13:54:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: cephadm 2021-03-10T13:54:42.432461+0000 mgr.gibba032.whopeo (mgr.24625) 63 : cephadm [INF] Reconfiguring daemon mon.b on gibba032 2021-03-10T13:54:43.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:43 gibba011 bash[31774]: cluster 2021-03-10T13:54:42.474148+0000 mgr.gibba032.whopeo (mgr.24625) 64 : cluster [DBG] pgmap v27: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:45.467 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: audit 2021-03-10T13:54:44.248913+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:45.467 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: audit 2021-03-10T13:54:44.250073+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:45.467 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: cephadm 2021-03-10T13:54:44.250111+0000 mgr.gibba032.whopeo (mgr.24625) 65 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2021-03-10T13:54:45.468 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: audit 2021-03-10T13:54:44.251328+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:45.468 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: audit 2021-03-10T13:54:44.252771+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:45.468 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: cephadm 2021-03-10T13:54:44.254302+0000 mgr.gibba032.whopeo (mgr.24625) 66 : cephadm [INF] Reconfiguring daemon mgr.x on gibba032 2021-03-10T13:54:45.468 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:45 gibba032 bash[24004]: audit 2021-03-10T13:54:44.254337+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.x","key": "container_image"}]: dispatch 2021-03-10T13:54:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: audit 2021-03-10T13:54:44.248913+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: audit 2021-03-10T13:54:44.250073+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: cephadm 2021-03-10T13:54:44.250111+0000 mgr.gibba032.whopeo (mgr.24625) 65 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2021-03-10T13:54:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: audit 2021-03-10T13:54:44.251328+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: audit 2021-03-10T13:54:44.252771+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: cephadm 2021-03-10T13:54:44.254302+0000 mgr.gibba032.whopeo (mgr.24625) 66 : cephadm [INF] Reconfiguring daemon mgr.x on gibba032 2021-03-10T13:54:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[30791]: audit 2021-03-10T13:54:44.254337+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.x","key": "container_image"}]: dispatch 2021-03-10T13:54:45.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: audit 2021-03-10T13:54:44.248913+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: audit 2021-03-10T13:54:44.250073+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: cephadm 2021-03-10T13:54:44.250111+0000 mgr.gibba032.whopeo (mgr.24625) 65 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2021-03-10T13:54:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: audit 2021-03-10T13:54:44.251328+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: audit 2021-03-10T13:54:44.252771+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: cephadm 2021-03-10T13:54:44.254302+0000 mgr.gibba032.whopeo (mgr.24625) 66 : cephadm [INF] Reconfiguring daemon mgr.x on gibba032 2021-03-10T13:54:45.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:45 gibba011 bash[31774]: audit 2021-03-10T13:54:44.254337+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.x","key": "container_image"}]: dispatch 2021-03-10T13:54:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: cluster 2021-03-10T13:54:44.474714+0000 mgr.gibba032.whopeo (mgr.24625) 67 : cluster [DBG] pgmap v28: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: audit 2021-03-10T13:54:45.981470+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: audit 2021-03-10T13:54:45.982854+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: audit 2021-03-10T13:54:45.984080+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: audit 2021-03-10T13:54:45.985224+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:46 gibba032 bash[24004]: audit 2021-03-10T13:54:45.986886+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.gibba032.whopeo","key": "container_image"}]: dispatch 2021-03-10T13:54:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: cluster 2021-03-10T13:54:44.474714+0000 mgr.gibba032.whopeo (mgr.24625) 67 : cluster [DBG] pgmap v28: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: audit 2021-03-10T13:54:45.981470+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: audit 2021-03-10T13:54:45.982854+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: audit 2021-03-10T13:54:45.984080+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: audit 2021-03-10T13:54:45.985224+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[30791]: audit 2021-03-10T13:54:45.986886+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.gibba032.whopeo","key": "container_image"}]: dispatch 2021-03-10T13:54:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: cluster 2021-03-10T13:54:44.474714+0000 mgr.gibba032.whopeo (mgr.24625) 67 : cluster [DBG] pgmap v28: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: audit 2021-03-10T13:54:45.981470+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: audit 2021-03-10T13:54:45.982854+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.gibba032.whopeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-03-10T13:54:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: audit 2021-03-10T13:54:45.984080+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "mgr services"}]: dispatch 2021-03-10T13:54:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: audit 2021-03-10T13:54:45.985224+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-03-10T13:54:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:46 gibba011 bash[31774]: audit 2021-03-10T13:54:45.986886+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mgr.gibba032.whopeo","key": "container_image"}]: dispatch 2021-03-10T13:54:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:47 gibba011 bash[30791]: cephadm 2021-03-10T13:54:45.982974+0000 mgr.gibba032.whopeo (mgr.24625) 68 : cephadm [INF] Reconfiguring mgr.gibba032.whopeo (monmap changed)... 2021-03-10T13:54:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:47 gibba011 bash[30791]: cephadm 2021-03-10T13:54:45.986774+0000 mgr.gibba032.whopeo (mgr.24625) 69 : cephadm [INF] Reconfiguring daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:54:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:47 gibba011 bash[31774]: cephadm 2021-03-10T13:54:45.982974+0000 mgr.gibba032.whopeo (mgr.24625) 68 : cephadm [INF] Reconfiguring mgr.gibba032.whopeo (monmap changed)... 2021-03-10T13:54:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:47 gibba011 bash[31774]: cephadm 2021-03-10T13:54:45.986774+0000 mgr.gibba032.whopeo (mgr.24625) 69 : cephadm [INF] Reconfiguring daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:54:47.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:47 gibba032 bash[24004]: cephadm 2021-03-10T13:54:45.982974+0000 mgr.gibba032.whopeo (mgr.24625) 68 : cephadm [INF] Reconfiguring mgr.gibba032.whopeo (monmap changed)... 2021-03-10T13:54:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:47 gibba032 bash[24004]: cephadm 2021-03-10T13:54:45.986774+0000 mgr.gibba032.whopeo (mgr.24625) 69 : cephadm [INF] Reconfiguring daemon mgr.gibba032.whopeo on gibba032 2021-03-10T13:54:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[30791]: cluster 2021-03-10T13:54:46.475596+0000 mgr.gibba032.whopeo (mgr.24625) 70 : cluster [DBG] pgmap v29: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[30791]: audit 2021-03-10T13:54:47.761519+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[30791]: audit 2021-03-10T13:54:47.763505+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:54:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[31774]: cluster 2021-03-10T13:54:46.475596+0000 mgr.gibba032.whopeo (mgr.24625) 70 : cluster [DBG] pgmap v29: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[31774]: audit 2021-03-10T13:54:47.761519+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:48 gibba011 bash[31774]: audit 2021-03-10T13:54:47.763505+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:54:48.660 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:48 gibba032 bash[24004]: cluster 2021-03-10T13:54:46.475596+0000 mgr.gibba032.whopeo (mgr.24625) 70 : cluster [DBG] pgmap v29: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:48.660 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:48 gibba032 bash[24004]: audit 2021-03-10T13:54:47.761519+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:48.661 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:48 gibba032 bash[24004]: audit 2021-03-10T13:54:47.763505+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:54:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:49 gibba011 bash[30791]: cephadm 2021-03-10T13:54:47.762956+0000 mgr.gibba032.whopeo (mgr.24625) 71 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2021-03-10T13:54:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:49 gibba011 bash[30791]: cephadm 2021-03-10T13:54:47.763495+0000 mgr.gibba032.whopeo (mgr.24625) 72 : cephadm [INF] Reconfiguring daemon osd.4 on gibba032 2021-03-10T13:54:49.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:49 gibba011 bash[31774]: cephadm 2021-03-10T13:54:47.762956+0000 mgr.gibba032.whopeo (mgr.24625) 71 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2021-03-10T13:54:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:49 gibba011 bash[31774]: cephadm 2021-03-10T13:54:47.763495+0000 mgr.gibba032.whopeo (mgr.24625) 72 : cephadm [INF] Reconfiguring daemon osd.4 on gibba032 2021-03-10T13:54:49.583 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:49 gibba032 bash[24004]: cephadm 2021-03-10T13:54:47.762956+0000 mgr.gibba032.whopeo (mgr.24625) 71 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2021-03-10T13:54:49.583 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:49 gibba032 bash[24004]: cephadm 2021-03-10T13:54:47.763495+0000 mgr.gibba032.whopeo (mgr.24625) 72 : cephadm [INF] Reconfiguring daemon osd.4 on gibba032 2021-03-10T13:54:50.569 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:50 gibba032 bash[24004]: cluster 2021-03-10T13:54:48.477496+0000 mgr.gibba032.whopeo (mgr.24625) 73 : cluster [DBG] pgmap v30: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:50.569 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:50 gibba032 bash[24004]: audit 2021-03-10T13:54:49.406909+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:50.569 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:50 gibba032 bash[24004]: audit 2021-03-10T13:54:49.408842+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:54:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[30791]: cluster 2021-03-10T13:54:48.477496+0000 mgr.gibba032.whopeo (mgr.24625) 73 : cluster [DBG] pgmap v30: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[30791]: audit 2021-03-10T13:54:49.406909+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[30791]: audit 2021-03-10T13:54:49.408842+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:54:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[31774]: cluster 2021-03-10T13:54:48.477496+0000 mgr.gibba032.whopeo (mgr.24625) 73 : cluster [DBG] pgmap v30: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[31774]: audit 2021-03-10T13:54:49.406909+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:50 gibba011 bash[31774]: audit 2021-03-10T13:54:49.408842+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:54:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[30791]: cephadm 2021-03-10T13:54:49.408253+0000 mgr.gibba032.whopeo (mgr.24625) 74 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2021-03-10T13:54:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[30791]: cephadm 2021-03-10T13:54:49.408769+0000 mgr.gibba032.whopeo (mgr.24625) 75 : cephadm [INF] Reconfiguring daemon osd.5 on gibba032 2021-03-10T13:54:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[30791]: audit 2021-03-10T13:54:51.175316+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[30791]: audit 2021-03-10T13:54:51.177495+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:54:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[31774]: cephadm 2021-03-10T13:54:49.408253+0000 mgr.gibba032.whopeo (mgr.24625) 74 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2021-03-10T13:54:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[31774]: cephadm 2021-03-10T13:54:49.408769+0000 mgr.gibba032.whopeo (mgr.24625) 75 : cephadm [INF] Reconfiguring daemon osd.5 on gibba032 2021-03-10T13:54:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[31774]: audit 2021-03-10T13:54:51.175316+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:51.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:51 gibba011 bash[31774]: audit 2021-03-10T13:54:51.177495+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:54:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:51 gibba032 bash[24004]: cephadm 2021-03-10T13:54:49.408253+0000 mgr.gibba032.whopeo (mgr.24625) 74 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2021-03-10T13:54:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:51 gibba032 bash[24004]: cephadm 2021-03-10T13:54:49.408769+0000 mgr.gibba032.whopeo (mgr.24625) 75 : cephadm [INF] Reconfiguring daemon osd.5 on gibba032 2021-03-10T13:54:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:51 gibba032 bash[24004]: audit 2021-03-10T13:54:51.175316+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:51 gibba032 bash[24004]: audit 2021-03-10T13:54:51.177495+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:54:52.464 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:52 gibba032 bash[24004]: cluster 2021-03-10T13:54:50.478199+0000 mgr.gibba032.whopeo (mgr.24625) 76 : cluster [DBG] pgmap v31: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:52.465 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:52 gibba032 bash[24004]: cephadm 2021-03-10T13:54:51.176672+0000 mgr.gibba032.whopeo (mgr.24625) 77 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2021-03-10T13:54:52.465 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:52 gibba032 bash[24004]: cephadm 2021-03-10T13:54:51.177354+0000 mgr.gibba032.whopeo (mgr.24625) 78 : cephadm [INF] Reconfiguring daemon osd.6 on gibba032 2021-03-10T13:54:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[30791]: cluster 2021-03-10T13:54:50.478199+0000 mgr.gibba032.whopeo (mgr.24625) 76 : cluster [DBG] pgmap v31: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[30791]: cephadm 2021-03-10T13:54:51.176672+0000 mgr.gibba032.whopeo (mgr.24625) 77 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2021-03-10T13:54:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[30791]: cephadm 2021-03-10T13:54:51.177354+0000 mgr.gibba032.whopeo (mgr.24625) 78 : cephadm [INF] Reconfiguring daemon osd.6 on gibba032 2021-03-10T13:54:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[31774]: cluster 2021-03-10T13:54:50.478199+0000 mgr.gibba032.whopeo (mgr.24625) 76 : cluster [DBG] pgmap v31: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[31774]: cephadm 2021-03-10T13:54:51.176672+0000 mgr.gibba032.whopeo (mgr.24625) 77 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2021-03-10T13:54:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:52 gibba011 bash[31774]: cephadm 2021-03-10T13:54:51.177354+0000 mgr.gibba032.whopeo (mgr.24625) 78 : cephadm [INF] Reconfiguring daemon osd.6 on gibba032 2021-03-10T13:54:54.106 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:53 gibba032 bash[24004]: cluster 2021-03-10T13:54:52.479996+0000 mgr.gibba032.whopeo (mgr.24625) 79 : cluster [DBG] pgmap v32: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:54.107 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:53 gibba032 bash[24004]: audit 2021-03-10T13:54:52.903438+0000 mon.a (mon.0) 132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:54.107 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:53 gibba032 bash[24004]: cephadm 2021-03-10T13:54:52.904630+0000 mgr.gibba032.whopeo (mgr.24625) 80 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2021-03-10T13:54:54.107 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:53 gibba032 bash[24004]: audit 2021-03-10T13:54:52.904991+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:54:54.108 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:53 gibba032 bash[24004]: cephadm 2021-03-10T13:54:52.905074+0000 mgr.gibba032.whopeo (mgr.24625) 81 : cephadm [INF] Reconfiguring daemon osd.7 on gibba032 2021-03-10T13:54:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[30791]: cluster 2021-03-10T13:54:52.479996+0000 mgr.gibba032.whopeo (mgr.24625) 79 : cluster [DBG] pgmap v32: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[30791]: audit 2021-03-10T13:54:52.903438+0000 mon.a (mon.0) 132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[30791]: cephadm 2021-03-10T13:54:52.904630+0000 mgr.gibba032.whopeo (mgr.24625) 80 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2021-03-10T13:54:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[30791]: audit 2021-03-10T13:54:52.904991+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:54:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[30791]: cephadm 2021-03-10T13:54:52.905074+0000 mgr.gibba032.whopeo (mgr.24625) 81 : cephadm [INF] Reconfiguring daemon osd.7 on gibba032 2021-03-10T13:54:54.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[31774]: cluster 2021-03-10T13:54:52.479996+0000 mgr.gibba032.whopeo (mgr.24625) 79 : cluster [DBG] pgmap v32: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:54.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[31774]: audit 2021-03-10T13:54:52.903438+0000 mon.a (mon.0) 132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[31774]: cephadm 2021-03-10T13:54:52.904630+0000 mgr.gibba032.whopeo (mgr.24625) 80 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2021-03-10T13:54:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[31774]: audit 2021-03-10T13:54:52.904991+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:54:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:53 gibba011 bash[31774]: cephadm 2021-03-10T13:54:52.905074+0000 mgr.gibba032.whopeo (mgr.24625) 81 : cephadm [INF] Reconfiguring daemon osd.7 on gibba032 2021-03-10T13:54:55.389 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:54:55.616 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:54:55.616 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (94s) 44s ago 6m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:54:55.617 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (6m) 27s ago 6m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:54:55.617 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (115s) 27s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:54:55.617 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2m) 27s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:54:55.617 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (75s) 44s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:54:55.618 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (66s) 44s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:54:55.618 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (38s) 27s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:54:55.618 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (49s) 44s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:54:55.618 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (7m) 44s ago 7m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:54:55.618 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (7m) 27s ago 7m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:54:55.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (10m) 44s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 cc98e62351ea 2021-03-10T13:54:55.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (9m) 44s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:54:55.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (9m) 44s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:54:55.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (8m) 44s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:54:55.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (8m) 27s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:54:55.620 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (8m) 27s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:54:55.620 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (7m) 27s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:54:55.620 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (7m) 27s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:54:55.620 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (6m) 27s ago 7m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:54:55.621 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 44s ago 4m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:54:55.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cluster 2021-03-10T13:54:54.480875+0000 mgr.gibba032.whopeo (mgr.24625) 82 : cluster [DBG] pgmap v33: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:55.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.567613+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.569322+0000 mgr.gibba032.whopeo (mgr.24625) 83 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:55.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.569640+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:55.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.571824+0000 mgr.gibba032.whopeo (mgr.24625) 84 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.571920+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.573525+0000 mgr.gibba032.whopeo (mgr.24625) 85 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.573568+0000 mgr.gibba032.whopeo (mgr.24625) 86 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.573780+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.575263+0000 mgr.gibba032.whopeo (mgr.24625) 87 : cephadm [INF] Upgrade: Setting container_image for all mon 2021-03-10T13:54:55.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.579477+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.580542+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2021-03-10T13:54:55.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.583559+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2021-03-10T13:54:55.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.584538+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2021-03-10T13:54:55.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.587710+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2021-03-10T13:54:55.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.588765+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2021-03-10T13:54:55.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.591566+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2021-03-10T13:54:55.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.592611+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cluster 2021-03-10T13:54:54.480875+0000 mgr.gibba032.whopeo (mgr.24625) 82 : cluster [DBG] pgmap v33: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:55.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.567613+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.569322+0000 mgr.gibba032.whopeo (mgr.24625) 83 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:55.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.569640+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:55.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.571824+0000 mgr.gibba032.whopeo (mgr.24625) 84 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:55.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.571920+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.573525+0000 mgr.gibba032.whopeo (mgr.24625) 85 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:55.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.573568+0000 mgr.gibba032.whopeo (mgr.24625) 86 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:55.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.573780+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.575263+0000 mgr.gibba032.whopeo (mgr.24625) 87 : cephadm [INF] Upgrade: Setting container_image for all mon 2021-03-10T13:54:55.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.579477+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.580542+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2021-03-10T13:54:55.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.583559+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2021-03-10T13:54:55.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.584538+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2021-03-10T13:54:55.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.587710+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2021-03-10T13:54:55.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.588765+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2021-03-10T13:54:55.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.591566+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2021-03-10T13:54:55.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.592611+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:55.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.592704+0000 mgr.gibba032.whopeo (mgr.24625) 88 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:54:55.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.592768+0000 mgr.gibba032.whopeo (mgr.24625) 89 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:54:55.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.593876+0000 mgr.gibba032.whopeo (mgr.24625) 90 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:54:55.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.597067+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.598530+0000 mgr.gibba032.whopeo (mgr.24625) 91 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:54:55.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.598602+0000 mgr.gibba032.whopeo (mgr.24625) 92 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:54:55.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.598898+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:55.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: audit 2021-03-10T13:54:54.599536+0000 mgr.gibba032.whopeo (mgr.24625) 93 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:55.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[30791]: cephadm 2021-03-10T13:54:54.601314+0000 mgr.gibba032.whopeo (mgr.24625) 94 : cephadm [INF] Upgrade: osd.0 is safe to restart 2021-03-10T13:54:55.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.592704+0000 mgr.gibba032.whopeo (mgr.24625) 88 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:54:55.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.592768+0000 mgr.gibba032.whopeo (mgr.24625) 89 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:54:55.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.593876+0000 mgr.gibba032.whopeo (mgr.24625) 90 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:54:55.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.597067+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:55.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.598530+0000 mgr.gibba032.whopeo (mgr.24625) 91 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:54:55.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.598602+0000 mgr.gibba032.whopeo (mgr.24625) 92 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:54:55.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.598898+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:55.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: audit 2021-03-10T13:54:54.599536+0000 mgr.gibba032.whopeo (mgr.24625) 93 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:55.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:55 gibba011 bash[31774]: cephadm 2021-03-10T13:54:54.601314+0000 mgr.gibba032.whopeo (mgr.24625) 94 : cephadm [INF] Upgrade: osd.0 is safe to restart 2021-03-10T13:54:55.892 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:54:55.893 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:54:55.893 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:54:55.893 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:55.894 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:54:55.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:54:55.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:55.894 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:54:55.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8 2021-03-10T13:54:55.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:54:55.895 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:54:55.895 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:54:55.895 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 8, 2021-03-10T13:54:55.896 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 6 2021-03-10T13:54:55.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:54:55.896 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:54:56.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cluster 2021-03-10T13:54:54.480875+0000 mgr.gibba032.whopeo (mgr.24625) 82 : cluster [DBG] pgmap v33: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:56.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.567613+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.569322+0000 mgr.gibba032.whopeo (mgr.24625) 83 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:54:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.569640+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:54:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.571824+0000 mgr.gibba032.whopeo (mgr.24625) 84 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:54:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.571920+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.573525+0000 mgr.gibba032.whopeo (mgr.24625) 85 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:54:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.573568+0000 mgr.gibba032.whopeo (mgr.24625) 86 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:54:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.573780+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.575263+0000 mgr.gibba032.whopeo (mgr.24625) 87 : cephadm [INF] Upgrade: Setting container_image for all mon 2021-03-10T13:54:56.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.579477+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:56.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.580542+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2021-03-10T13:54:56.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.583559+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2021-03-10T13:54:56.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.584538+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2021-03-10T13:54:56.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.587710+0000 mon.a (mon.0) 142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2021-03-10T13:54:56.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.588765+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2021-03-10T13:54:56.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.591566+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2021-03-10T13:54:56.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.592611+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:56.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.592704+0000 mgr.gibba032.whopeo (mgr.24625) 88 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:54:56.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.592768+0000 mgr.gibba032.whopeo (mgr.24625) 89 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:54:56.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.593876+0000 mgr.gibba032.whopeo (mgr.24625) 90 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:54:56.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.597067+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:56.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.598530+0000 mgr.gibba032.whopeo (mgr.24625) 91 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:54:56.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.598602+0000 mgr.gibba032.whopeo (mgr.24625) 92 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:54:56.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.598898+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:56.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: audit 2021-03-10T13:54:54.599536+0000 mgr.gibba032.whopeo (mgr.24625) 93 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-03-10T13:54:56.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:55 gibba032 bash[24004]: cephadm 2021-03-10T13:54:54.601314+0000 mgr.gibba032.whopeo (mgr.24625) 94 : cephadm [INF] Upgrade: osd.0 is safe to restart 2021-03-10T13:54:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.379482+0000 mgr.gibba032.whopeo (mgr.24625) 95 : audit [DBG] from='client.44136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.604123+0000 mgr.gibba032.whopeo (mgr.24625) 96 : audit [DBG] from='client.54127 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.861375+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T13:54:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.861688+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T13:54:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.861958+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T13:54:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.862463+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:54:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.892247+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.111:0/2567847293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.971007+0000 mon.a (mon.0) 153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:54:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[30791]: audit 2021-03-10T13:54:55.971742+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:54:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.379482+0000 mgr.gibba032.whopeo (mgr.24625) 95 : audit [DBG] from='client.44136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.604123+0000 mgr.gibba032.whopeo (mgr.24625) 96 : audit [DBG] from='client.54127 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.861375+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T13:54:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.861688+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T13:54:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.861958+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T13:54:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.862463+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:54:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.892247+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.111:0/2567847293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.971007+0000 mon.a (mon.0) 153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:54:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:56 gibba011 bash[31774]: audit 2021-03-10T13:54:55.971742+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:54:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.379482+0000 mgr.gibba032.whopeo (mgr.24625) 95 : audit [DBG] from='client.44136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.604123+0000 mgr.gibba032.whopeo (mgr.24625) 96 : audit [DBG] from='client.54127 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:54:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.861375+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T13:54:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.861688+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T13:54:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.861958+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T13:54:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.862463+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:54:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.892247+0000 mon.a (mon.0) 152 : audit [DBG] from='client.? 172.21.2.111:0/2567847293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:54:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.971007+0000 mon.a (mon.0) 153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:54:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:56 gibba032 bash[24004]: audit 2021-03-10T13:54:55.971742+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:54:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:57 gibba032 bash[24004]: cluster 2021-03-10T13:54:56.482104+0000 mgr.gibba032.whopeo (mgr.24625) 97 : cluster [DBG] pgmap v34: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:57 gibba032 bash[24004]: cephadm 2021-03-10T13:54:56.567660+0000 mgr.gibba032.whopeo (mgr.24625) 98 : cephadm [INF] Upgrade: Updating osd.0 2021-03-10T13:54:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:57 gibba032 bash[24004]: audit 2021-03-10T13:54:56.583185+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:57 gibba032 bash[24004]: audit 2021-03-10T13:54:56.584812+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:57 gibba032 bash[24004]: cephadm 2021-03-10T13:54:56.584862+0000 mgr.gibba032.whopeo (mgr.24625) 99 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:54:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[30791]: cluster 2021-03-10T13:54:56.482104+0000 mgr.gibba032.whopeo (mgr.24625) 97 : cluster [DBG] pgmap v34: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[30791]: cephadm 2021-03-10T13:54:56.567660+0000 mgr.gibba032.whopeo (mgr.24625) 98 : cephadm [INF] Upgrade: Updating osd.0 2021-03-10T13:54:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[30791]: audit 2021-03-10T13:54:56.583185+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[30791]: audit 2021-03-10T13:54:56.584812+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[30791]: cephadm 2021-03-10T13:54:56.584862+0000 mgr.gibba032.whopeo (mgr.24625) 99 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:54:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[31774]: cluster 2021-03-10T13:54:56.482104+0000 mgr.gibba032.whopeo (mgr.24625) 97 : cluster [DBG] pgmap v34: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[31774]: cephadm 2021-03-10T13:54:56.567660+0000 mgr.gibba032.whopeo (mgr.24625) 98 : cephadm [INF] Upgrade: Updating osd.0 2021-03-10T13:54:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[31774]: audit 2021-03-10T13:54:56.583185+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:54:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[31774]: audit 2021-03-10T13:54:56.584812+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.0","key": "container_image"}]: dispatch 2021-03-10T13:54:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:57 gibba011 bash[31774]: cephadm 2021-03-10T13:54:56.584862+0000 mgr.gibba032.whopeo (mgr.24625) 99 : cephadm [INF] Deploying daemon osd.0 on gibba011 2021-03-10T13:54:58.708 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:54:58 gibba011 systemd[1]: Stopping Ceph osd.0 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:54:58.709 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:54:58 gibba011 bash[14938]: debug 2021-03-10T13:54:58.556+0000 7fb649a52700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:54:58.709 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:54:58 gibba011 bash[14938]: debug 2021-03-10T13:54:58.556+0000 7fb649a52700 -1 osd.0 61 *** Got signal Terminated *** 2021-03-10T13:54:58.709 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:54:58 gibba011 bash[14938]: debug 2021-03-10T13:54:58.556+0000 7fb649a52700 -1 osd.0 61 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:54:59.580 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:54:59 gibba011 docker[33322]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.0 2021-03-10T13:54:59.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.483601+0000 mgr.gibba032.whopeo (mgr.24625) 100 : cluster [DBG] pgmap v35: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.854331+0000 mon.a (mon.0) 157 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.854452+0000 mon.a (mon.0) 158 : cluster [INF] osd.0 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:54:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.855126+0000 mon.a (mon.0) 159 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.855690+0000 mon.a (mon.0) 160 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.856222+0000 mon.a (mon.0) 161 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.856397+0000 mon.a (mon.0) 162 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.856778+0000 mon.a (mon.0) 163 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.856941+0000 mon.a (mon.0) 164 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.857338+0000 mon.a (mon.0) 165 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.857507+0000 mon.a (mon.0) 166 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.857948+0000 mon.a (mon.0) 167 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.858112+0000 mon.a (mon.0) 168 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.858457+0000 mon.a (mon.0) 169 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.858611+0000 mon.a (mon.0) 170 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.858974+0000 mon.a (mon.0) 171 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.859133+0000 mon.a (mon.0) 172 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.859615+0000 mon.a (mon.0) 173 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.859785+0000 mon.a (mon.0) 174 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.860135+0000 mon.a (mon.0) 175 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.860298+0000 mon.a (mon.0) 176 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:58.860646+0000 mon.a (mon.0) 177 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.255556+0000 mon.a (mon.0) 178 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.255963+0000 mon.a (mon.0) 179 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.256271+0000 mon.a (mon.0) 180 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.483601+0000 mgr.gibba032.whopeo (mgr.24625) 100 : cluster [DBG] pgmap v35: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:54:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.854331+0000 mon.a (mon.0) 157 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.854452+0000 mon.a (mon.0) 158 : cluster [INF] osd.0 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:54:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.855126+0000 mon.a (mon.0) 159 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.855690+0000 mon.a (mon.0) 160 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.856222+0000 mon.a (mon.0) 161 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.856397+0000 mon.a (mon.0) 162 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.856778+0000 mon.a (mon.0) 163 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.856941+0000 mon.a (mon.0) 164 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.857338+0000 mon.a (mon.0) 165 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.857507+0000 mon.a (mon.0) 166 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.857948+0000 mon.a (mon.0) 167 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.858112+0000 mon.a (mon.0) 168 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.858457+0000 mon.a (mon.0) 169 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.858611+0000 mon.a (mon.0) 170 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.858974+0000 mon.a (mon.0) 171 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.859133+0000 mon.a (mon.0) 172 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.859615+0000 mon.a (mon.0) 173 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.859785+0000 mon.a (mon.0) 174 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.860135+0000 mon.a (mon.0) 175 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.860298+0000 mon.a (mon.0) 176 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:58.860646+0000 mon.a (mon.0) 177 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.255556+0000 mon.a (mon.0) 178 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.255963+0000 mon.a (mon.0) 179 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.256271+0000 mon.a (mon.0) 180 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.256473+0000 mon.a (mon.0) 181 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.256968+0000 mon.a (mon.0) 182 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.257409+0000 mon.a (mon.0) 183 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.257761+0000 mon.a (mon.0) 184 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.258094+0000 mon.a (mon.0) 185 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.258515+0000 mon.a (mon.0) 186 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.259001+0000 mon.a (mon.0) 187 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.259482+0000 mon.a (mon.0) 188 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.260257+0000 mon.a (mon.0) 189 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.260603+0000 mon.a (mon.0) 190 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.260915+0000 mon.a (mon.0) 191 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261217+0000 mon.a (mon.0) 192 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261389+0000 mon.a (mon.0) 193 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261616+0000 mon.a (mon.0) 194 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261733+0000 mon.a (mon.0) 195 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261797+0000 mon.a (mon.0) 196 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.261903+0000 mon.a (mon.0) 197 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.498663+0000 mon.a (mon.0) 198 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.256473+0000 mon.a (mon.0) 181 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.256968+0000 mon.a (mon.0) 182 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:54:59.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.257409+0000 mon.a (mon.0) 183 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.257761+0000 mon.a (mon.0) 184 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.258094+0000 mon.a (mon.0) 185 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:54:59.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.258515+0000 mon.a (mon.0) 186 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.259001+0000 mon.a (mon.0) 187 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:54:59.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.259482+0000 mon.a (mon.0) 188 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:54:59.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.260257+0000 mon.a (mon.0) 189 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.260603+0000 mon.a (mon.0) 190 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.260915+0000 mon.a (mon.0) 191 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261217+0000 mon.a (mon.0) 192 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261389+0000 mon.a (mon.0) 193 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261616+0000 mon.a (mon.0) 194 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:54:59.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261733+0000 mon.a (mon.0) 195 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261797+0000 mon.a (mon.0) 196 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:54:59.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.261903+0000 mon.a (mon.0) 197 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:54:59.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:54:59 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.498663+0000 mon.a (mon.0) 198 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:00.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.483601+0000 mgr.gibba032.whopeo (mgr.24625) 100 : cluster [DBG] pgmap v35: 129 pgs: 129 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:55:00.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.854331+0000 mon.a (mon.0) 157 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:00.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.854452+0000 mon.a (mon.0) 158 : cluster [INF] osd.0 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:55:00.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.855126+0000 mon.a (mon.0) 159 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:00.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.855690+0000 mon.a (mon.0) 160 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.856222+0000 mon.a (mon.0) 161 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.856397+0000 mon.a (mon.0) 162 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.856778+0000 mon.a (mon.0) 163 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.856941+0000 mon.a (mon.0) 164 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.857338+0000 mon.a (mon.0) 165 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.857507+0000 mon.a (mon.0) 166 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.857948+0000 mon.a (mon.0) 167 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.858112+0000 mon.a (mon.0) 168 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.858457+0000 mon.a (mon.0) 169 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.858611+0000 mon.a (mon.0) 170 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.858974+0000 mon.a (mon.0) 171 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.859133+0000 mon.a (mon.0) 172 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.859615+0000 mon.a (mon.0) 173 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.859785+0000 mon.a (mon.0) 174 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.860135+0000 mon.a (mon.0) 175 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.860298+0000 mon.a (mon.0) 176 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:58.860646+0000 mon.a (mon.0) 177 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.255556+0000 mon.a (mon.0) 178 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.255963+0000 mon.a (mon.0) 179 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.256271+0000 mon.a (mon.0) 180 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.256473+0000 mon.a (mon.0) 181 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.256968+0000 mon.a (mon.0) 182 : cluster [DBG] osd.0 reported immediately failed by osd.6 2021-03-10T13:55:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.257409+0000 mon.a (mon.0) 183 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.257761+0000 mon.a (mon.0) 184 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.258094+0000 mon.a (mon.0) 185 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.258515+0000 mon.a (mon.0) 186 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.259001+0000 mon.a (mon.0) 187 : cluster [DBG] osd.0 reported immediately failed by osd.5 2021-03-10T13:55:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.259482+0000 mon.a (mon.0) 188 : cluster [DBG] osd.0 reported immediately failed by osd.4 2021-03-10T13:55:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.260257+0000 mon.a (mon.0) 189 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.260603+0000 mon.a (mon.0) 190 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.260915+0000 mon.a (mon.0) 191 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261217+0000 mon.a (mon.0) 192 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261389+0000 mon.a (mon.0) 193 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261616+0000 mon.a (mon.0) 194 : cluster [DBG] osd.0 reported immediately failed by osd.1 2021-03-10T13:55:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261733+0000 mon.a (mon.0) 195 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261797+0000 mon.a (mon.0) 196 : cluster [DBG] osd.0 reported immediately failed by osd.3 2021-03-10T13:55:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.261903+0000 mon.a (mon.0) 197 : cluster [DBG] osd.0 reported immediately failed by osd.2 2021-03-10T13:55:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:54:59 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.498663+0000 mon.a (mon.0) 198 : cluster [DBG] osd.0 reported immediately failed by osd.7 2021-03-10T13:55:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:00 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.593385+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:00 gibba032 bash[24004]: cluster 2021-03-10T13:54:59.606096+0000 mon.a (mon.0) 200 : cluster [DBG] osdmap e62: 8 total, 7 up, 8 in 2021-03-10T13:55:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:00 gibba032 bash[24004]: audit 2021-03-10T13:54:59.606282+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-03-10T13:55:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:00 gibba032 bash[24004]: audit 2021-03-10T13:54:59.607719+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-03-10T13:55:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.593385+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[30791]: cluster 2021-03-10T13:54:59.606096+0000 mon.a (mon.0) 200 : cluster [DBG] osdmap e62: 8 total, 7 up, 8 in 2021-03-10T13:55:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[30791]: audit 2021-03-10T13:54:59.606282+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-03-10T13:55:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[30791]: audit 2021-03-10T13:54:59.607719+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-03-10T13:55:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.593385+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[31774]: cluster 2021-03-10T13:54:59.606096+0000 mon.a (mon.0) 200 : cluster [DBG] osdmap e62: 8 total, 7 up, 8 in 2021-03-10T13:55:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[31774]: audit 2021-03-10T13:54:59.606282+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-03-10T13:55:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:00 gibba011 bash[31774]: audit 2021-03-10T13:54:59.607719+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-03-10T13:55:01.579 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:01 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.0.service: Succeeded. 2021-03-10T13:55:01.579 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:01 gibba011 systemd[1]: Stopped Ceph osd.0 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:01.580 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:01 gibba011 systemd[1]: Started Ceph osd.0 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:01.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[31774]: cluster 2021-03-10T13:55:00.484533+0000 mgr.gibba032.whopeo (mgr.24625) 101 : cluster [DBG] pgmap v37: 129 pgs: 16 stale+active+clean, 113 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:55:01.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[31774]: audit 2021-03-10T13:55:00.608835+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-03-10T13:55:01.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[31774]: cluster 2021-03-10T13:55:00.608942+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e63: 8 total, 7 up, 8 in 2021-03-10T13:55:01.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[31774]: audit 2021-03-10T13:55:01.442718+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:01.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[31774]: audit 2021-03-10T13:55:01.446345+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[30791]: cluster 2021-03-10T13:55:00.484533+0000 mgr.gibba032.whopeo (mgr.24625) 101 : cluster [DBG] pgmap v37: 129 pgs: 16 stale+active+clean, 113 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:55:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[30791]: audit 2021-03-10T13:55:00.608835+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-03-10T13:55:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[30791]: cluster 2021-03-10T13:55:00.608942+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e63: 8 total, 7 up, 8 in 2021-03-10T13:55:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[30791]: audit 2021-03-10T13:55:01.442718+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:01 gibba011 bash[30791]: audit 2021-03-10T13:55:01.446345+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:01.867 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:01 gibba032 bash[24004]: cluster 2021-03-10T13:55:00.484533+0000 mgr.gibba032.whopeo (mgr.24625) 101 : cluster [DBG] pgmap v37: 129 pgs: 16 stale+active+clean, 113 active+clean; 7.0 KiB data, 5.4 MiB used, 664 GiB / 672 GiB avail 2021-03-10T13:55:01.868 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:01 gibba032 bash[24004]: audit 2021-03-10T13:55:00.608835+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-03-10T13:55:01.868 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:01 gibba032 bash[24004]: cluster 2021-03-10T13:55:00.608942+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e63: 8 total, 7 up, 8 in 2021-03-10T13:55:01.868 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:01 gibba032 bash[24004]: audit 2021-03-10T13:55:01.442718+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:01.868 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:01 gibba032 bash[24004]: audit 2021-03-10T13:55:01.446345+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:02.948 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[30791]: cephadm 2021-03-10T13:55:01.446805+0000 mgr.gibba032.whopeo (mgr.24625) 102 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:55:02.949 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[31774]: cephadm 2021-03-10T13:55:01.446805+0000 mgr.gibba032.whopeo (mgr.24625) 102 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:55:02.950 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-03-10T13:55:02.950 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2021-03-10T13:55:02.950 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2021-03-10T13:55:02.950 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2021-03-10T13:55:02.950 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-03-10T13:55:02.951 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-03-10T13:55:02.951 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:02 gibba011 bash[33509]: --> ceph-volume lvm activate successful for osd ID: 0 2021-03-10T13:55:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:02 gibba032 bash[24004]: cephadm 2021-03-10T13:55:01.446805+0000 mgr.gibba032.whopeo (mgr.24625) 102 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:55:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:03 gibba011 bash[30791]: cluster 2021-03-10T13:55:02.487036+0000 mgr.gibba032.whopeo (mgr.24625) 103 : cluster [DBG] pgmap v39: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:03 gibba011 bash[30791]: cluster 2021-03-10T13:55:02.608860+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:03 gibba011 bash[31774]: cluster 2021-03-10T13:55:02.487036+0000 mgr.gibba032.whopeo (mgr.24625) 103 : cluster [DBG] pgmap v39: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:03 gibba011 bash[31774]: cluster 2021-03-10T13:55:02.608860+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:04.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:03 gibba032 bash[24004]: cluster 2021-03-10T13:55:02.487036+0000 mgr.gibba032.whopeo (mgr.24625) 103 : cluster [DBG] pgmap v39: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:04.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:03 gibba032 bash[24004]: cluster 2021-03-10T13:55:02.608860+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:04.580 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:04 gibba011 bash[33636]: debug 2021-03-10T13:55:04.312+0000 7f912e0a3700 1 -- 172.21.2.111:0/2045592220 <== mon.1 v2:172.21.2.111:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a157e964e0 con 0x55a157f13000 2021-03-10T13:55:05.329 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:04 gibba011 bash[33636]: debug 2021-03-10T13:55:04.920+0000 7f9135b38080 -1 Falling back to public interface 2021-03-10T13:55:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: cluster 2021-03-10T13:55:04.487737+0000 mgr.gibba032.whopeo (mgr.24625) 104 : cluster [DBG] pgmap v40: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.200764+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.201844+0000 mgr.gibba032.whopeo (mgr.24625) 105 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:55:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.401052+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.410545+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.412536+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.414132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.415593+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:06 gibba032 bash[24004]: audit 2021-03-10T13:55:05.418148+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: cluster 2021-03-10T13:55:04.487737+0000 mgr.gibba032.whopeo (mgr.24625) 104 : cluster [DBG] pgmap v40: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.200764+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.201844+0000 mgr.gibba032.whopeo (mgr.24625) 105 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:55:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.401052+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.410545+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.412536+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.414132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.415593+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[30791]: audit 2021-03-10T13:55:05.418148+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: cluster 2021-03-10T13:55:04.487737+0000 mgr.gibba032.whopeo (mgr.24625) 104 : cluster [DBG] pgmap v40: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.200764+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.201844+0000 mgr.gibba032.whopeo (mgr.24625) 105 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:55:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.401052+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.410545+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.412536+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.414132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.415593+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:06 gibba011 bash[31774]: audit 2021-03-10T13:55:05.418148+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.409911+0000 mgr.gibba032.whopeo (mgr.24625) 106 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.412345+0000 mgr.gibba032.whopeo (mgr.24625) 107 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.414024+0000 mgr.gibba032.whopeo (mgr.24625) 108 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.414061+0000 mgr.gibba032.whopeo (mgr.24625) 109 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.415492+0000 mgr.gibba032.whopeo (mgr.24625) 110 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.415528+0000 mgr.gibba032.whopeo (mgr.24625) 111 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.416787+0000 mgr.gibba032.whopeo (mgr.24625) 112 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.417945+0000 mgr.gibba032.whopeo (mgr.24625) 113 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.417983+0000 mgr.gibba032.whopeo (mgr.24625) 114 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: audit 2021-03-10T13:55:05.418666+0000 mgr.gibba032.whopeo (mgr.24625) 115 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:07 gibba032 bash[24004]: cephadm 2021-03-10T13:55:05.419736+0000 mgr.gibba032.whopeo (mgr.24625) 116 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2021-03-10T13:55:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.409911+0000 mgr.gibba032.whopeo (mgr.24625) 106 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.412345+0000 mgr.gibba032.whopeo (mgr.24625) 107 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.414024+0000 mgr.gibba032.whopeo (mgr.24625) 108 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.414061+0000 mgr.gibba032.whopeo (mgr.24625) 109 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.415492+0000 mgr.gibba032.whopeo (mgr.24625) 110 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.415528+0000 mgr.gibba032.whopeo (mgr.24625) 111 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.416787+0000 mgr.gibba032.whopeo (mgr.24625) 112 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.417945+0000 mgr.gibba032.whopeo (mgr.24625) 113 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.417983+0000 mgr.gibba032.whopeo (mgr.24625) 114 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: audit 2021-03-10T13:55:05.418666+0000 mgr.gibba032.whopeo (mgr.24625) 115 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[30791]: cephadm 2021-03-10T13:55:05.419736+0000 mgr.gibba032.whopeo (mgr.24625) 116 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2021-03-10T13:55:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.409911+0000 mgr.gibba032.whopeo (mgr.24625) 106 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.412345+0000 mgr.gibba032.whopeo (mgr.24625) 107 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.414024+0000 mgr.gibba032.whopeo (mgr.24625) 108 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.414061+0000 mgr.gibba032.whopeo (mgr.24625) 109 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.415492+0000 mgr.gibba032.whopeo (mgr.24625) 110 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.415528+0000 mgr.gibba032.whopeo (mgr.24625) 111 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.416787+0000 mgr.gibba032.whopeo (mgr.24625) 112 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.417945+0000 mgr.gibba032.whopeo (mgr.24625) 113 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.417983+0000 mgr.gibba032.whopeo (mgr.24625) 114 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: audit 2021-03-10T13:55:05.418666+0000 mgr.gibba032.whopeo (mgr.24625) 115 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:07.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:07 gibba011 bash[31774]: cephadm 2021-03-10T13:55:05.419736+0000 mgr.gibba032.whopeo (mgr.24625) 116 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2021-03-10T13:55:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:08 gibba032 bash[24004]: cluster 2021-03-10T13:55:06.488963+0000 mgr.gibba032.whopeo (mgr.24625) 117 : cluster [DBG] pgmap v41: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[30791]: cluster 2021-03-10T13:55:06.488963+0000 mgr.gibba032.whopeo (mgr.24625) 117 : cluster [DBG] pgmap v41: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[31774]: cluster 2021-03-10T13:55:06.488963+0000 mgr.gibba032.whopeo (mgr.24625) 117 : cluster [DBG] pgmap v41: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:08.581 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[33636]: debug 2021-03-10T13:55:08.120+0000 7f9135b38080 -1 bluestore(/var/lib/ceph/osd/ceph-0) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:55:08.581 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[33636]: debug 2021-03-10T13:55:08.124+0000 7f911f5fa700 -1 bluestore(/var/lib/ceph/osd/ceph-0) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:55:08.581 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[33636]: debug 2021-03-10T13:55:08.124+0000 7f911f5fa700 -1 bluestore(/var/lib/ceph/osd/ceph-0) fsck warning: #1:d525e5cb:::INTEL_SSDPEL1K375GA_PHKM920000BW375A:head# has omap that is not per-pg or pgmeta 2021-03-10T13:55:08.581 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:08 gibba011 bash[33636]: debug 2021-03-10T13:55:08.248+0000 7f9135b38080 -1 osd.0 61 log_to_monitors {default=true} 2021-03-10T13:55:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:09 gibba032 bash[24004]: audit 2021-03-10T13:55:08.256689+0000 mon.a (mon.0) 215 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:55:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:09 gibba011 bash[30791]: audit 2021-03-10T13:55:08.256689+0000 mon.a (mon.0) 215 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:55:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:09 gibba011 bash[31774]: audit 2021-03-10T13:55:08.256689+0000 mon.a (mon.0) 215 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-03-10T13:55:09.583 INFO:journalctl@ceph.osd.0.gibba011.stdout:Mar 10 13:55:09 gibba011 bash[33636]: debug 2021-03-10T13:55:09.232+0000 7f9125815700 -1 osd.0 61 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:55:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:10 gibba032 bash[24004]: cluster 2021-03-10T13:55:08.491286+0000 mgr.gibba032.whopeo (mgr.24625) 118 : cluster [DBG] pgmap v42: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:10 gibba032 bash[24004]: audit 2021-03-10T13:55:09.232832+0000 mon.a (mon.0) 216 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:55:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:10 gibba032 bash[24004]: cluster 2021-03-10T13:55:09.232869+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e64: 8 total, 7 up, 8 in 2021-03-10T13:55:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:10 gibba032 bash[24004]: audit 2021-03-10T13:55:09.233496+0000 mon.a (mon.0) 218 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[30791]: cluster 2021-03-10T13:55:08.491286+0000 mgr.gibba032.whopeo (mgr.24625) 118 : cluster [DBG] pgmap v42: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[30791]: audit 2021-03-10T13:55:09.232832+0000 mon.a (mon.0) 216 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:55:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[30791]: cluster 2021-03-10T13:55:09.232869+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e64: 8 total, 7 up, 8 in 2021-03-10T13:55:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[30791]: audit 2021-03-10T13:55:09.233496+0000 mon.a (mon.0) 218 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[31774]: cluster 2021-03-10T13:55:08.491286+0000 mgr.gibba032.whopeo (mgr.24625) 118 : cluster [DBG] pgmap v42: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 5.6 MiB used, 664 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[31774]: audit 2021-03-10T13:55:09.232832+0000 mon.a (mon.0) 216 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-03-10T13:55:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[31774]: cluster 2021-03-10T13:55:09.232869+0000 mon.a (mon.0) 217 : cluster [DBG] osdmap e64: 8 total, 7 up, 8 in 2021-03-10T13:55:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:10 gibba011 bash[31774]: audit 2021-03-10T13:55:09.233496+0000 mon.a (mon.0) 218 : audit [INF] from='osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:11.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:11 gibba032 bash[24004]: cluster 2021-03-10T13:55:10.232688+0000 mon.a (mon.0) 219 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:11 gibba032 bash[24004]: cluster 2021-03-10T13:55:10.243087+0000 mon.a (mon.0) 220 : cluster [INF] osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693] boot 2021-03-10T13:55:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:11 gibba032 bash[24004]: cluster 2021-03-10T13:55:10.243129+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2021-03-10T13:55:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:11 gibba032 bash[24004]: audit 2021-03-10T13:55:10.243477+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:55:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:11 gibba032 bash[24004]: cluster 2021-03-10T13:55:11.247481+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2021-03-10T13:55:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[30791]: cluster 2021-03-10T13:55:10.232688+0000 mon.a (mon.0) 219 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[30791]: cluster 2021-03-10T13:55:10.243087+0000 mon.a (mon.0) 220 : cluster [INF] osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693] boot 2021-03-10T13:55:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[30791]: cluster 2021-03-10T13:55:10.243129+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2021-03-10T13:55:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[30791]: audit 2021-03-10T13:55:10.243477+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:55:11.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[30791]: cluster 2021-03-10T13:55:11.247481+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2021-03-10T13:55:11.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[31774]: cluster 2021-03-10T13:55:10.232688+0000 mon.a (mon.0) 219 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:11.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[31774]: cluster 2021-03-10T13:55:10.243087+0000 mon.a (mon.0) 220 : cluster [INF] osd.0 [v2:172.21.2.111:6800/2589730693,v1:172.21.2.111:6801/2589730693] boot 2021-03-10T13:55:11.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[31774]: cluster 2021-03-10T13:55:10.243129+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2021-03-10T13:55:11.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[31774]: audit 2021-03-10T13:55:10.243477+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-03-10T13:55:11.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:11 gibba011 bash[31774]: cluster 2021-03-10T13:55:11.247481+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2021-03-10T13:55:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:12 gibba011 bash[30791]: cluster 2021-03-10T13:55:10.492828+0000 mgr.gibba032.whopeo (mgr.24625) 119 : cluster [DBG] pgmap v45: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:12 gibba011 bash[31774]: cluster 2021-03-10T13:55:10.492828+0000 mgr.gibba032.whopeo (mgr.24625) 119 : cluster [DBG] pgmap v45: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:12 gibba032 bash[24004]: cluster 2021-03-10T13:55:10.492828+0000 mgr.gibba032.whopeo (mgr.24625) 119 : cluster [DBG] pgmap v45: 129 pgs: 18 active+undersized+degraded, 31 active+undersized, 80 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:55:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:13 gibba011 bash[30791]: cluster 2021-03-10T13:55:13.257607+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:13.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:13 gibba011 bash[31774]: cluster 2021-03-10T13:55:13.257607+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:13.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:13 gibba032 bash[24004]: cluster 2021-03-10T13:55:13.257607+0000 mon.a (mon.0) 224 : cluster [WRN] Health check update: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:14 gibba011 bash[30791]: cluster 2021-03-10T13:55:12.494993+0000 mgr.gibba032.whopeo (mgr.24625) 120 : cluster [DBG] pgmap v47: 129 pgs: 6 active+undersized+degraded, 10 active+undersized, 113 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 6/192 objects degraded (3.125%) 2021-03-10T13:55:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:14 gibba011 bash[31774]: cluster 2021-03-10T13:55:12.494993+0000 mgr.gibba032.whopeo (mgr.24625) 120 : cluster [DBG] pgmap v47: 129 pgs: 6 active+undersized+degraded, 10 active+undersized, 113 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 6/192 objects degraded (3.125%) 2021-03-10T13:55:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:14 gibba032 bash[24004]: cluster 2021-03-10T13:55:12.494993+0000 mgr.gibba032.whopeo (mgr.24625) 120 : cluster [DBG] pgmap v47: 129 pgs: 6 active+undersized+degraded, 10 active+undersized, 113 active+clean; 7.0 KiB data, 10 MiB used, 665 GiB / 672 GiB avail; 6/192 objects degraded (3.125%) 2021-03-10T13:55:16.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:15 gibba032 bash[24004]: cluster 2021-03-10T13:55:14.496195+0000 mgr.gibba032.whopeo (mgr.24625) 121 : cluster [DBG] pgmap v48: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:15 gibba032 bash[24004]: cluster 2021-03-10T13:55:15.317213+0000 mon.a (mon.0) 225 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded) 2021-03-10T13:55:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:15 gibba011 bash[30791]: cluster 2021-03-10T13:55:14.496195+0000 mgr.gibba032.whopeo (mgr.24625) 121 : cluster [DBG] pgmap v48: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:15 gibba011 bash[30791]: cluster 2021-03-10T13:55:15.317213+0000 mon.a (mon.0) 225 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded) 2021-03-10T13:55:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:15 gibba011 bash[31774]: cluster 2021-03-10T13:55:14.496195+0000 mgr.gibba032.whopeo (mgr.24625) 121 : cluster [DBG] pgmap v48: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:15 gibba011 bash[31774]: cluster 2021-03-10T13:55:15.317213+0000 mon.a (mon.0) 225 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/192 objects degraded (3.125%), 6 pgs degraded) 2021-03-10T13:55:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:16 gibba032 bash[24004]: audit 2021-03-10T13:55:15.972248+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:16 gibba011 bash[30791]: audit 2021-03-10T13:55:15.972248+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:16 gibba011 bash[31774]: audit 2021-03-10T13:55:15.972248+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:18.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:17 gibba032 bash[24004]: cluster 2021-03-10T13:55:16.496804+0000 mgr.gibba032.whopeo (mgr.24625) 122 : cluster [DBG] pgmap v49: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:17 gibba011 bash[30791]: cluster 2021-03-10T13:55:16.496804+0000 mgr.gibba032.whopeo (mgr.24625) 122 : cluster [DBG] pgmap v49: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:17 gibba011 bash[31774]: cluster 2021-03-10T13:55:16.496804+0000 mgr.gibba032.whopeo (mgr.24625) 122 : cluster [DBG] pgmap v49: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:19 gibba032 bash[24004]: cluster 2021-03-10T13:55:18.498773+0000 mgr.gibba032.whopeo (mgr.24625) 123 : cluster [DBG] pgmap v50: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:19 gibba011 bash[30791]: cluster 2021-03-10T13:55:18.498773+0000 mgr.gibba032.whopeo (mgr.24625) 123 : cluster [DBG] pgmap v50: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:20.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:19 gibba011 bash[31774]: cluster 2021-03-10T13:55:18.498773+0000 mgr.gibba032.whopeo (mgr.24625) 123 : cluster [DBG] pgmap v50: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:21.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:21 gibba011 bash[30791]: audit 2021-03-10T13:55:20.426192+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:21 gibba011 bash[31774]: audit 2021-03-10T13:55:20.426192+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:21.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:21 gibba032 bash[24004]: audit 2021-03-10T13:55:20.426192+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[30791]: audit 2021-03-10T13:55:20.427165+0000 mgr.gibba032.whopeo (mgr.24625) 124 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[30791]: cephadm 2021-03-10T13:55:20.431744+0000 mgr.gibba032.whopeo (mgr.24625) 125 : cephadm [INF] Upgrade: osd.1 is safe to restart 2021-03-10T13:55:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[30791]: cluster 2021-03-10T13:55:20.499996+0000 mgr.gibba032.whopeo (mgr.24625) 126 : cluster [DBG] pgmap v51: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[31774]: audit 2021-03-10T13:55:20.427165+0000 mgr.gibba032.whopeo (mgr.24625) 124 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[31774]: cephadm 2021-03-10T13:55:20.431744+0000 mgr.gibba032.whopeo (mgr.24625) 125 : cephadm [INF] Upgrade: osd.1 is safe to restart 2021-03-10T13:55:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:22 gibba011 bash[31774]: cluster 2021-03-10T13:55:20.499996+0000 mgr.gibba032.whopeo (mgr.24625) 126 : cluster [DBG] pgmap v51: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:22 gibba032 bash[24004]: audit 2021-03-10T13:55:20.427165+0000 mgr.gibba032.whopeo (mgr.24625) 124 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-03-10T13:55:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:22 gibba032 bash[24004]: cephadm 2021-03-10T13:55:20.431744+0000 mgr.gibba032.whopeo (mgr.24625) 125 : cephadm [INF] Upgrade: osd.1 is safe to restart 2021-03-10T13:55:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:22 gibba032 bash[24004]: cluster 2021-03-10T13:55:20.499996+0000 mgr.gibba032.whopeo (mgr.24625) 126 : cluster [DBG] pgmap v51: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:23.743 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[30791]: cephadm 2021-03-10T13:55:22.442748+0000 mgr.gibba032.whopeo (mgr.24625) 127 : cephadm [INF] Upgrade: Updating osd.1 2021-03-10T13:55:23.743 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[30791]: audit 2021-03-10T13:55:22.449107+0000 mon.a (mon.0) 228 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:23.743 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[30791]: cephadm 2021-03-10T13:55:22.453506+0000 mgr.gibba032.whopeo (mgr.24625) 128 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:55:23.744 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[30791]: audit 2021-03-10T13:55:22.453642+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:55:23.744 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[30791]: cluster 2021-03-10T13:55:22.500846+0000 mgr.gibba032.whopeo (mgr.24625) 129 : cluster [DBG] pgmap v52: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:23.744 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[31774]: cephadm 2021-03-10T13:55:22.442748+0000 mgr.gibba032.whopeo (mgr.24625) 127 : cephadm [INF] Upgrade: Updating osd.1 2021-03-10T13:55:23.745 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[31774]: audit 2021-03-10T13:55:22.449107+0000 mon.a (mon.0) 228 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:23.745 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[31774]: cephadm 2021-03-10T13:55:22.453506+0000 mgr.gibba032.whopeo (mgr.24625) 128 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:55:23.745 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[31774]: audit 2021-03-10T13:55:22.453642+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:55:23.745 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:23 gibba011 bash[31774]: cluster 2021-03-10T13:55:22.500846+0000 mgr.gibba032.whopeo (mgr.24625) 129 : cluster [DBG] pgmap v52: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:23.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:23 gibba032 bash[24004]: cephadm 2021-03-10T13:55:22.442748+0000 mgr.gibba032.whopeo (mgr.24625) 127 : cephadm [INF] Upgrade: Updating osd.1 2021-03-10T13:55:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:23 gibba032 bash[24004]: audit 2021-03-10T13:55:22.449107+0000 mon.a (mon.0) 228 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:23 gibba032 bash[24004]: cephadm 2021-03-10T13:55:22.453506+0000 mgr.gibba032.whopeo (mgr.24625) 128 : cephadm [INF] Deploying daemon osd.1 on gibba011 2021-03-10T13:55:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:23 gibba032 bash[24004]: audit 2021-03-10T13:55:22.453642+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.1","key": "container_image"}]: dispatch 2021-03-10T13:55:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:23 gibba032 bash[24004]: cluster 2021-03-10T13:55:22.500846+0000 mgr.gibba032.whopeo (mgr.24625) 129 : cluster [DBG] pgmap v52: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:24.579 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:24 gibba011 systemd[1]: Stopping Ceph osd.1 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:55:24.579 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:24 gibba011 bash[16394]: debug 2021-03-10T13:55:24.468+0000 7fe998fc3700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:55:24.580 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:24 gibba011 bash[16394]: debug 2021-03-10T13:55:24.468+0000 7fe998fc3700 -1 osd.1 66 *** Got signal Terminated *** 2021-03-10T13:55:24.580 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:24 gibba011 bash[16394]: debug 2021-03-10T13:55:24.468+0000 7fe998fc3700 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:55:25.564 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:25 gibba011 docker[34493]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.1 2021-03-10T13:55:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.501959+0000 mgr.gibba032.whopeo (mgr.24625) 130 : cluster [DBG] pgmap v53: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.759437+0000 mon.a (mon.0) 230 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.759551+0000 mon.a (mon.0) 231 : cluster [INF] osd.1 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:55:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.760311+0000 mon.a (mon.0) 232 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.760599+0000 mon.a (mon.0) 233 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.761021+0000 mon.a (mon.0) 234 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.761205+0000 mon.a (mon.0) 235 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.761610+0000 mon.a (mon.0) 236 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.761789+0000 mon.a (mon.0) 237 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.762178+0000 mon.a (mon.0) 238 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.762478+0000 mon.a (mon.0) 239 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.763031+0000 mon.a (mon.0) 240 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.763207+0000 mon.a (mon.0) 241 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.763636+0000 mon.a (mon.0) 242 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.763815+0000 mon.a (mon.0) 243 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:24.764347+0000 mon.a (mon.0) 244 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.158187+0000 mon.a (mon.0) 245 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.158411+0000 mon.a (mon.0) 246 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.158971+0000 mon.a (mon.0) 247 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.159163+0000 mon.a (mon.0) 248 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.159706+0000 mon.a (mon.0) 249 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.160265+0000 mon.a (mon.0) 250 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.160593+0000 mon.a (mon.0) 251 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.161259+0000 mon.a (mon.0) 252 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.164223+0000 mon.a (mon.0) 253 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.501959+0000 mgr.gibba032.whopeo (mgr.24625) 130 : cluster [DBG] pgmap v53: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:25.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.759437+0000 mon.a (mon.0) 230 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.759551+0000 mon.a (mon.0) 231 : cluster [INF] osd.1 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:55:25.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.760311+0000 mon.a (mon.0) 232 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.760599+0000 mon.a (mon.0) 233 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.761021+0000 mon.a (mon.0) 234 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.761205+0000 mon.a (mon.0) 235 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.761610+0000 mon.a (mon.0) 236 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.761789+0000 mon.a (mon.0) 237 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.762178+0000 mon.a (mon.0) 238 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.762478+0000 mon.a (mon.0) 239 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.763031+0000 mon.a (mon.0) 240 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.763207+0000 mon.a (mon.0) 241 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.763636+0000 mon.a (mon.0) 242 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.763815+0000 mon.a (mon.0) 243 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:24.764347+0000 mon.a (mon.0) 244 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.158187+0000 mon.a (mon.0) 245 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.158411+0000 mon.a (mon.0) 246 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.158971+0000 mon.a (mon.0) 247 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.159163+0000 mon.a (mon.0) 248 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.159706+0000 mon.a (mon.0) 249 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:25.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.160265+0000 mon.a (mon.0) 250 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:25.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.160593+0000 mon.a (mon.0) 251 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:25.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.161259+0000 mon.a (mon.0) 252 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:25.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.164223+0000 mon.a (mon.0) 253 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.164444+0000 mon.a (mon.0) 254 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.164901+0000 mon.a (mon.0) 255 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.165085+0000 mon.a (mon.0) 256 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.165499+0000 mon.a (mon.0) 257 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.165688+0000 mon.a (mon.0) 258 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.164444+0000 mon.a (mon.0) 254 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:25.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.164901+0000 mon.a (mon.0) 255 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.165085+0000 mon.a (mon.0) 256 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:25.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.165499+0000 mon.a (mon.0) 257 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:25.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:25 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.165688+0000 mon.a (mon.0) 258 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:26.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.501959+0000 mgr.gibba032.whopeo (mgr.24625) 130 : cluster [DBG] pgmap v53: 129 pgs: 16 active+clean+wait, 113 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:26.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.759437+0000 mon.a (mon.0) 230 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:26.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.759551+0000 mon.a (mon.0) 231 : cluster [INF] osd.1 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:55:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.760311+0000 mon.a (mon.0) 232 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.760599+0000 mon.a (mon.0) 233 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.761021+0000 mon.a (mon.0) 234 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.761205+0000 mon.a (mon.0) 235 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.761610+0000 mon.a (mon.0) 236 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.761789+0000 mon.a (mon.0) 237 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.762178+0000 mon.a (mon.0) 238 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.762478+0000 mon.a (mon.0) 239 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.763031+0000 mon.a (mon.0) 240 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.763207+0000 mon.a (mon.0) 241 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.763636+0000 mon.a (mon.0) 242 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.763815+0000 mon.a (mon.0) 243 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:24.764347+0000 mon.a (mon.0) 244 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.158187+0000 mon.a (mon.0) 245 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.158411+0000 mon.a (mon.0) 246 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:26.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.158971+0000 mon.a (mon.0) 247 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:26.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.159163+0000 mon.a (mon.0) 248 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:26.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.159706+0000 mon.a (mon.0) 249 : cluster [DBG] osd.1 reported immediately failed by osd.6 2021-03-10T13:55:26.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.160265+0000 mon.a (mon.0) 250 : cluster [DBG] osd.1 reported immediately failed by osd.5 2021-03-10T13:55:26.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.160593+0000 mon.a (mon.0) 251 : cluster [DBG] osd.1 reported immediately failed by osd.7 2021-03-10T13:55:26.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.161259+0000 mon.a (mon.0) 252 : cluster [DBG] osd.1 reported immediately failed by osd.4 2021-03-10T13:55:26.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.164223+0000 mon.a (mon.0) 253 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:26.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.164444+0000 mon.a (mon.0) 254 : cluster [DBG] osd.1 reported immediately failed by osd.2 2021-03-10T13:55:26.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.164901+0000 mon.a (mon.0) 255 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:26.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.165085+0000 mon.a (mon.0) 256 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:26.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.165499+0000 mon.a (mon.0) 257 : cluster [DBG] osd.1 reported immediately failed by osd.3 2021-03-10T13:55:26.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:25 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.165688+0000 mon.a (mon.0) 258 : cluster [DBG] osd.1 reported immediately failed by osd.0 2021-03-10T13:55:26.140 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:55:26.375 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:55:26.375 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2m) 21s ago 7m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:55:26.375 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (7m) 58s ago 7m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:55:26.375 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2m) 58s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:55:26.376 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2m) 58s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:55:26.376 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (106s) 21s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:55:26.376 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (97s) 21s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:55:26.376 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (69s) 58s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:55:26.376 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (80s) 21s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:55:26.377 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (7m) 21s ago 7m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:55:26.377 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (7m) 58s ago 7m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:55:26.377 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (22s) 21s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:55:26.377 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (10m) 21s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 913379f50d2d 2021-03-10T13:55:26.377 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (9m) 21s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 1446073f7732 2021-03-10T13:55:26.378 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (9m) 21s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:55:26.378 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (9m) 58s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:55:26.378 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (8m) 58s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:55:26.378 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (8m) 58s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:55:26.378 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (7m) 58s ago 7m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:55:26.379 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (7m) 58s ago 7m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:55:26.379 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 21s ago 5m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:55:26.641 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:55:26.641 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:55:26.641 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:55:26.641 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:26.642 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:55:26.642 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:55:26.642 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:26.642 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:55:26.642 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 6, 2021-03-10T13:55:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 1 2021-03-10T13:55:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:55:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:55:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 6, 2021-03-10T13:55:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 7 2021-03-10T13:55:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:55:26.644 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:55:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.560817+0000 mon.a (mon.0) 259 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[30791]: cluster 2021-03-10T13:55:25.574143+0000 mon.a (mon.0) 260 : cluster [DBG] osdmap e67: 8 total, 7 up, 8 in 2021-03-10T13:55:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[30791]: audit 2021-03-10T13:55:26.130634+0000 mgr.gibba032.whopeo (mgr.24625) 131 : audit [DBG] from='client.54145 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.560817+0000 mon.a (mon.0) 259 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[31774]: cluster 2021-03-10T13:55:25.574143+0000 mon.a (mon.0) 260 : cluster [DBG] osdmap e67: 8 total, 7 up, 8 in 2021-03-10T13:55:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:26 gibba011 bash[31774]: audit 2021-03-10T13:55:26.130634+0000 mgr.gibba032.whopeo (mgr.24625) 131 : audit [DBG] from='client.54145 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:26 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.560817+0000 mon.a (mon.0) 259 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:26 gibba032 bash[24004]: cluster 2021-03-10T13:55:25.574143+0000 mon.a (mon.0) 260 : cluster [DBG] osdmap e67: 8 total, 7 up, 8 in 2021-03-10T13:55:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:26 gibba032 bash[24004]: audit 2021-03-10T13:55:26.130634+0000 mgr.gibba032.whopeo (mgr.24625) 131 : audit [DBG] from='client.54145 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:27.579 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:27 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.1.service: Succeeded. 2021-03-10T13:55:27.580 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:27 gibba011 systemd[1]: Stopped Ceph osd.1 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:27.580 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:27 gibba011 systemd[1]: Started Ceph osd.1 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:28.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: audit 2021-03-10T13:55:26.363278+0000 mgr.gibba032.whopeo (mgr.24625) 132 : audit [DBG] from='client.54151 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:28.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: cluster 2021-03-10T13:55:26.502993+0000 mgr.gibba032.whopeo (mgr.24625) 133 : cluster [DBG] pgmap v55: 129 pgs: 18 stale+active+clean, 16 active+clean+wait, 95 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:28.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: cluster 2021-03-10T13:55:26.579012+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e68: 8 total, 7 up, 8 in 2021-03-10T13:55:28.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: audit 2021-03-10T13:55:26.640182+0000 mon.c (mon.1) 5 : audit [DBG] from='client.? 172.21.2.111:0/1099942669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:28.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: audit 2021-03-10T13:55:27.235487+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:28.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:27 gibba032 bash[24004]: audit 2021-03-10T13:55:27.238602+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:28.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: audit 2021-03-10T13:55:26.363278+0000 mgr.gibba032.whopeo (mgr.24625) 132 : audit [DBG] from='client.54151 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: cluster 2021-03-10T13:55:26.502993+0000 mgr.gibba032.whopeo (mgr.24625) 133 : cluster [DBG] pgmap v55: 129 pgs: 18 stale+active+clean, 16 active+clean+wait, 95 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: cluster 2021-03-10T13:55:26.579012+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e68: 8 total, 7 up, 8 in 2021-03-10T13:55:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: audit 2021-03-10T13:55:26.640182+0000 mon.c (mon.1) 5 : audit [DBG] from='client.? 172.21.2.111:0/1099942669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: audit 2021-03-10T13:55:27.235487+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[30791]: audit 2021-03-10T13:55:27.238602+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: audit 2021-03-10T13:55:26.363278+0000 mgr.gibba032.whopeo (mgr.24625) 132 : audit [DBG] from='client.54151 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: cluster 2021-03-10T13:55:26.502993+0000 mgr.gibba032.whopeo (mgr.24625) 133 : cluster [DBG] pgmap v55: 129 pgs: 18 stale+active+clean, 16 active+clean+wait, 95 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail 2021-03-10T13:55:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: cluster 2021-03-10T13:55:26.579012+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e68: 8 total, 7 up, 8 in 2021-03-10T13:55:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: audit 2021-03-10T13:55:26.640182+0000 mon.c (mon.1) 5 : audit [DBG] from='client.? 172.21.2.111:0/1099942669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: audit 2021-03-10T13:55:27.235487+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:27 gibba011 bash[31774]: audit 2021-03-10T13:55:27.238602+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[30791]: cluster 2021-03-10T13:55:28.582488+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: Degraded data redundancy: 27/192 objects degraded (14.062%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:28.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[31774]: cluster 2021-03-10T13:55:28.582488+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: Degraded data redundancy: 27/192 objects degraded (14.062%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:28.831 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-03-10T13:55:28.831 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2021-03-10T13:55:28.831 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2021-03-10T13:55:28.831 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2021-03-10T13:55:28.831 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-03-10T13:55:28.832 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-03-10T13:55:28.832 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:28 gibba011 bash[34745]: --> ceph-volume lvm activate successful for osd ID: 1 2021-03-10T13:55:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:28 gibba032 bash[24004]: cluster 2021-03-10T13:55:28.582488+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: Degraded data redundancy: 27/192 objects degraded (14.062%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:29.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:29 gibba011 bash[30791]: cluster 2021-03-10T13:55:28.503909+0000 mgr.gibba032.whopeo (mgr.24625) 134 : cluster [DBG] pgmap v57: 129 pgs: 1 stale+active+clean, 19 active+undersized+degraded, 16 active+clean+wait, 32 active+undersized, 61 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 27/192 objects degraded (14.062%) 2021-03-10T13:55:29.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:29 gibba011 bash[31774]: cluster 2021-03-10T13:55:28.503909+0000 mgr.gibba032.whopeo (mgr.24625) 134 : cluster [DBG] pgmap v57: 129 pgs: 1 stale+active+clean, 19 active+undersized+degraded, 16 active+clean+wait, 32 active+undersized, 61 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 27/192 objects degraded (14.062%) 2021-03-10T13:55:30.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:29 gibba032 bash[24004]: cluster 2021-03-10T13:55:28.503909+0000 mgr.gibba032.whopeo (mgr.24625) 134 : cluster [DBG] pgmap v57: 129 pgs: 1 stale+active+clean, 19 active+undersized+degraded, 16 active+clean+wait, 32 active+undersized, 61 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 27/192 objects degraded (14.062%) 2021-03-10T13:55:30.329 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:30 gibba011 bash[34872]: debug 2021-03-10T13:55:30.100+0000 7f594beaa700 1 -- 172.21.2.111:0/3761648173 <== mon.1 v2:172.21.2.111:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ddc02024e0 con 0x55ddc027ec00 2021-03-10T13:55:31.079 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:30 gibba011 bash[34872]: debug 2021-03-10T13:55:30.672+0000 7f595393f080 -1 Falling back to public interface 2021-03-10T13:55:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cluster 2021-03-10T13:55:30.505190+0000 mgr.gibba032.whopeo (mgr.24625) 135 : cluster [DBG] pgmap v58: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.938713+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.944915+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.963487+0000 mgr.gibba032.whopeo (mgr.24625) 136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.965969+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.969304+0000 mgr.gibba032.whopeo (mgr.24625) 137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.970775+0000 mon.a (mon.0) 268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.973229+0000 mgr.gibba032.whopeo (mgr.24625) 138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.973618+0000 mgr.gibba032.whopeo (mgr.24625) 139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.974481+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.976817+0000 mgr.gibba032.whopeo (mgr.24625) 140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.977080+0000 mgr.gibba032.whopeo (mgr.24625) 141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.977670+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.980045+0000 mgr.gibba032.whopeo (mgr.24625) 142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.983011+0000 mgr.gibba032.whopeo (mgr.24625) 143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.983219+0000 mgr.gibba032.whopeo (mgr.24625) 144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.984971+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: audit 2021-03-10T13:55:30.985745+0000 mgr.gibba032.whopeo (mgr.24625) 145 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:31 gibba032 bash[24004]: cephadm 2021-03-10T13:55:30.988712+0000 mgr.gibba032.whopeo (mgr.24625) 146 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cluster 2021-03-10T13:55:30.505190+0000 mgr.gibba032.whopeo (mgr.24625) 135 : cluster [DBG] pgmap v58: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.938713+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.944915+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.963487+0000 mgr.gibba032.whopeo (mgr.24625) 136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.965969+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.969304+0000 mgr.gibba032.whopeo (mgr.24625) 137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.970775+0000 mon.a (mon.0) 268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.973229+0000 mgr.gibba032.whopeo (mgr.24625) 138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.973618+0000 mgr.gibba032.whopeo (mgr.24625) 139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:32.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.974481+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.976817+0000 mgr.gibba032.whopeo (mgr.24625) 140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:32.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.977080+0000 mgr.gibba032.whopeo (mgr.24625) 141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:32.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.977670+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.980045+0000 mgr.gibba032.whopeo (mgr.24625) 142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:32.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.983011+0000 mgr.gibba032.whopeo (mgr.24625) 143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:32.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.983219+0000 mgr.gibba032.whopeo (mgr.24625) 144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:32.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.984971+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: audit 2021-03-10T13:55:30.985745+0000 mgr.gibba032.whopeo (mgr.24625) 145 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[30791]: cephadm 2021-03-10T13:55:30.988712+0000 mgr.gibba032.whopeo (mgr.24625) 146 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cluster 2021-03-10T13:55:30.505190+0000 mgr.gibba032.whopeo (mgr.24625) 135 : cluster [DBG] pgmap v58: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.938713+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.944915+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.963487+0000 mgr.gibba032.whopeo (mgr.24625) 136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.965969+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.969304+0000 mgr.gibba032.whopeo (mgr.24625) 137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.970775+0000 mon.a (mon.0) 268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.973229+0000 mgr.gibba032.whopeo (mgr.24625) 138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:32.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.973618+0000 mgr.gibba032.whopeo (mgr.24625) 139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:32.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.974481+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.976817+0000 mgr.gibba032.whopeo (mgr.24625) 140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:32.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.977080+0000 mgr.gibba032.whopeo (mgr.24625) 141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.977670+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.980045+0000 mgr.gibba032.whopeo (mgr.24625) 142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.983011+0000 mgr.gibba032.whopeo (mgr.24625) 143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.983219+0000 mgr.gibba032.whopeo (mgr.24625) 144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.984971+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: audit 2021-03-10T13:55:30.985745+0000 mgr.gibba032.whopeo (mgr.24625) 145 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:32.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:31 gibba011 bash[31774]: cephadm 2021-03-10T13:55:30.988712+0000 mgr.gibba032.whopeo (mgr.24625) 146 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:34.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:33 gibba032 bash[24004]: cluster 2021-03-10T13:55:32.507475+0000 mgr.gibba032.whopeo (mgr.24625) 147 : cluster [DBG] pgmap v59: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:33 gibba011 bash[30791]: cluster 2021-03-10T13:55:32.507475+0000 mgr.gibba032.whopeo (mgr.24625) 147 : cluster [DBG] pgmap v59: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:33 gibba011 bash[31774]: cluster 2021-03-10T13:55:32.507475+0000 mgr.gibba032.whopeo (mgr.24625) 147 : cluster [DBG] pgmap v59: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:34.330 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:33 gibba011 bash[34872]: debug 2021-03-10T13:55:33.852+0000 7f595393f080 -1 bluestore(/var/lib/ceph/osd/ceph-1) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:55:34.331 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:33 gibba011 bash[34872]: debug 2021-03-10T13:55:33.856+0000 7f593ec04700 -1 bluestore(/var/lib/ceph/osd/ceph-1) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:55:34.331 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:33 gibba011 bash[34872]: debug 2021-03-10T13:55:33.980+0000 7f595393f080 -1 osd.1 66 log_to_monitors {default=true} 2021-03-10T13:55:35.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:34 gibba032 bash[24004]: audit 2021-03-10T13:55:33.988631+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:55:35.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:34 gibba032 bash[24004]: cluster 2021-03-10T13:55:34.854741+0000 mon.a (mon.0) 273 : cluster [WRN] Health check update: Degraded data redundancy: 28/192 objects degraded (14.583%), 20 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:35.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:34 gibba011 bash[30791]: audit 2021-03-10T13:55:33.988631+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:55:35.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:34 gibba011 bash[30791]: cluster 2021-03-10T13:55:34.854741+0000 mon.a (mon.0) 273 : cluster [WRN] Health check update: Degraded data redundancy: 28/192 objects degraded (14.583%), 20 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:35.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:34 gibba011 bash[31774]: audit 2021-03-10T13:55:33.988631+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-03-10T13:55:35.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:34 gibba011 bash[31774]: cluster 2021-03-10T13:55:34.854741+0000 mon.a (mon.0) 273 : cluster [WRN] Health check update: Degraded data redundancy: 28/192 objects degraded (14.583%), 20 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:35.332 INFO:journalctl@ceph.osd.1.gibba011.stdout:Mar 10 13:55:34 gibba011 bash[34872]: debug 2021-03-10T13:55:34.964+0000 7f5944e1f700 -1 osd.1 66 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:55:36.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:35 gibba032 bash[24004]: cluster 2021-03-10T13:55:34.508363+0000 mgr.gibba032.whopeo (mgr.24625) 148 : cluster [DBG] pgmap v60: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:35 gibba032 bash[24004]: audit 2021-03-10T13:55:34.963540+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:55:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:35 gibba032 bash[24004]: cluster 2021-03-10T13:55:34.963573+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e69: 8 total, 7 up, 8 in 2021-03-10T13:55:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:35 gibba032 bash[24004]: audit 2021-03-10T13:55:34.964060+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[30791]: cluster 2021-03-10T13:55:34.508363+0000 mgr.gibba032.whopeo (mgr.24625) 148 : cluster [DBG] pgmap v60: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[30791]: audit 2021-03-10T13:55:34.963540+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:55:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[30791]: cluster 2021-03-10T13:55:34.963573+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e69: 8 total, 7 up, 8 in 2021-03-10T13:55:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[30791]: audit 2021-03-10T13:55:34.964060+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[31774]: cluster 2021-03-10T13:55:34.508363+0000 mgr.gibba032.whopeo (mgr.24625) 148 : cluster [DBG] pgmap v60: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 11 MiB used, 665 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[31774]: audit 2021-03-10T13:55:34.963540+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-03-10T13:55:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[31774]: cluster 2021-03-10T13:55:34.963573+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e69: 8 total, 7 up, 8 in 2021-03-10T13:55:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:35 gibba011 bash[31774]: audit 2021-03-10T13:55:34.964060+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:55:37.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:36 gibba032 bash[24004]: cluster 2021-03-10T13:55:35.963719+0000 mon.a (mon.0) 277 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:36 gibba032 bash[24004]: cluster 2021-03-10T13:55:35.974067+0000 mon.a (mon.0) 278 : cluster [INF] osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918] boot 2021-03-10T13:55:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:36 gibba032 bash[24004]: cluster 2021-03-10T13:55:35.974115+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2021-03-10T13:55:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:36 gibba032 bash[24004]: audit 2021-03-10T13:55:35.974455+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:55:37.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[30791]: cluster 2021-03-10T13:55:35.963719+0000 mon.a (mon.0) 277 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:37.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[30791]: cluster 2021-03-10T13:55:35.974067+0000 mon.a (mon.0) 278 : cluster [INF] osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918] boot 2021-03-10T13:55:37.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[30791]: cluster 2021-03-10T13:55:35.974115+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2021-03-10T13:55:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[30791]: audit 2021-03-10T13:55:35.974455+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:55:37.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[31774]: cluster 2021-03-10T13:55:35.963719+0000 mon.a (mon.0) 277 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:55:37.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[31774]: cluster 2021-03-10T13:55:35.974067+0000 mon.a (mon.0) 278 : cluster [INF] osd.1 [v2:172.21.2.111:6808/499960918,v1:172.21.2.111:6809/499960918] boot 2021-03-10T13:55:37.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[31774]: cluster 2021-03-10T13:55:35.974115+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2021-03-10T13:55:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:36 gibba011 bash[31774]: audit 2021-03-10T13:55:35.974455+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-03-10T13:55:38.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:37 gibba032 bash[24004]: cluster 2021-03-10T13:55:36.509719+0000 mgr.gibba032.whopeo (mgr.24625) 149 : cluster [DBG] pgmap v63: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:37 gibba032 bash[24004]: cluster 2021-03-10T13:55:36.991603+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2021-03-10T13:55:38.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:37 gibba011 bash[30791]: cluster 2021-03-10T13:55:36.509719+0000 mgr.gibba032.whopeo (mgr.24625) 149 : cluster [DBG] pgmap v63: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:37 gibba011 bash[30791]: cluster 2021-03-10T13:55:36.991603+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2021-03-10T13:55:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:37 gibba011 bash[31774]: cluster 2021-03-10T13:55:36.509719+0000 mgr.gibba032.whopeo (mgr.24625) 149 : cluster [DBG] pgmap v63: 129 pgs: 20 active+undersized+degraded, 36 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 28/192 objects degraded (14.583%) 2021-03-10T13:55:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:37 gibba011 bash[31774]: cluster 2021-03-10T13:55:36.991603+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2021-03-10T13:55:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:40 gibba032 bash[24004]: cluster 2021-03-10T13:55:38.510817+0000 mgr.gibba032.whopeo (mgr.24625) 150 : cluster [DBG] pgmap v65: 129 pgs: 34 peering, 8 active+undersized+degraded, 14 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:55:40.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:40 gibba032 bash[24004]: cluster 2021-03-10T13:55:39.857471+0000 mon.a (mon.0) 282 : cluster [WRN] Health check update: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:40 gibba011 bash[30791]: cluster 2021-03-10T13:55:38.510817+0000 mgr.gibba032.whopeo (mgr.24625) 150 : cluster [DBG] pgmap v65: 129 pgs: 34 peering, 8 active+undersized+degraded, 14 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:55:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:40 gibba011 bash[30791]: cluster 2021-03-10T13:55:39.857471+0000 mon.a (mon.0) 282 : cluster [WRN] Health check update: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:40 gibba011 bash[31774]: cluster 2021-03-10T13:55:38.510817+0000 mgr.gibba032.whopeo (mgr.24625) 150 : cluster [DBG] pgmap v65: 129 pgs: 34 peering, 8 active+undersized+degraded, 14 active+undersized, 73 active+clean; 7.0 KiB data, 16 MiB used, 666 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:55:40.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:40 gibba011 bash[31774]: cluster 2021-03-10T13:55:39.857471+0000 mon.a (mon.0) 282 : cluster [WRN] Health check update: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded (PG_DEGRADED) 2021-03-10T13:55:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:41 gibba011 bash[30791]: cluster 2021-03-10T13:55:41.003559+0000 mon.a (mon.0) 283 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded) 2021-03-10T13:55:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:41 gibba011 bash[31774]: cluster 2021-03-10T13:55:41.003559+0000 mon.a (mon.0) 283 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded) 2021-03-10T13:55:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:41 gibba032 bash[24004]: cluster 2021-03-10T13:55:41.003559+0000 mon.a (mon.0) 283 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 8 pgs degraded) 2021-03-10T13:55:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:42 gibba011 bash[30791]: cluster 2021-03-10T13:55:40.512248+0000 mgr.gibba032.whopeo (mgr.24625) 151 : cluster [DBG] pgmap v66: 129 pgs: 18 active+clean+wait, 34 peering, 77 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:42 gibba011 bash[31774]: cluster 2021-03-10T13:55:40.512248+0000 mgr.gibba032.whopeo (mgr.24625) 151 : cluster [DBG] pgmap v66: 129 pgs: 18 active+clean+wait, 34 peering, 77 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:42 gibba032 bash[24004]: cluster 2021-03-10T13:55:40.512248+0000 mgr.gibba032.whopeo (mgr.24625) 151 : cluster [DBG] pgmap v66: 129 pgs: 18 active+clean+wait, 34 peering, 77 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:44 gibba011 bash[30791]: cluster 2021-03-10T13:55:42.513955+0000 mgr.gibba032.whopeo (mgr.24625) 152 : cluster [DBG] pgmap v67: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:44.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:44 gibba011 bash[31774]: cluster 2021-03-10T13:55:42.513955+0000 mgr.gibba032.whopeo (mgr.24625) 152 : cluster [DBG] pgmap v67: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:44 gibba032 bash[24004]: cluster 2021-03-10T13:55:42.513955+0000 mgr.gibba032.whopeo (mgr.24625) 152 : cluster [DBG] pgmap v67: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:46.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:46 gibba011 bash[30791]: cluster 2021-03-10T13:55:44.515168+0000 mgr.gibba032.whopeo (mgr.24625) 153 : cluster [DBG] pgmap v68: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:46.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:46 gibba011 bash[30791]: audit 2021-03-10T13:55:46.005196+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:46.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:46 gibba011 bash[31774]: cluster 2021-03-10T13:55:44.515168+0000 mgr.gibba032.whopeo (mgr.24625) 153 : cluster [DBG] pgmap v68: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:46.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:46 gibba011 bash[31774]: audit 2021-03-10T13:55:46.005196+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:46 gibba032 bash[24004]: cluster 2021-03-10T13:55:44.515168+0000 mgr.gibba032.whopeo (mgr.24625) 153 : cluster [DBG] pgmap v68: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:46 gibba032 bash[24004]: audit 2021-03-10T13:55:46.005196+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:47.429 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:47 gibba011 bash[30791]: audit 2021-03-10T13:55:46.006120+0000 mgr.gibba032.whopeo (mgr.24625) 154 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:47.430 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:47 gibba011 bash[30791]: cephadm 2021-03-10T13:55:46.011026+0000 mgr.gibba032.whopeo (mgr.24625) 155 : cephadm [INF] Upgrade: osd.2 is safe to restart 2021-03-10T13:55:47.430 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:47 gibba011 bash[31774]: audit 2021-03-10T13:55:46.006120+0000 mgr.gibba032.whopeo (mgr.24625) 154 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:47.430 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:47 gibba011 bash[31774]: cephadm 2021-03-10T13:55:46.011026+0000 mgr.gibba032.whopeo (mgr.24625) 155 : cephadm [INF] Upgrade: osd.2 is safe to restart 2021-03-10T13:55:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:47 gibba032 bash[24004]: audit 2021-03-10T13:55:46.006120+0000 mgr.gibba032.whopeo (mgr.24625) 154 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-03-10T13:55:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:47 gibba032 bash[24004]: cephadm 2021-03-10T13:55:46.011026+0000 mgr.gibba032.whopeo (mgr.24625) 155 : cephadm [INF] Upgrade: osd.2 is safe to restart 2021-03-10T13:55:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:48 gibba032 bash[24004]: cluster 2021-03-10T13:55:46.515897+0000 mgr.gibba032.whopeo (mgr.24625) 156 : cluster [DBG] pgmap v69: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:48 gibba032 bash[24004]: audit 2021-03-10T13:55:48.002335+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:48 gibba032 bash[24004]: audit 2021-03-10T13:55:48.006567+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:55:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[30791]: cluster 2021-03-10T13:55:46.515897+0000 mgr.gibba032.whopeo (mgr.24625) 156 : cluster [DBG] pgmap v69: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[30791]: audit 2021-03-10T13:55:48.002335+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[30791]: audit 2021-03-10T13:55:48.006567+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:55:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[31774]: cluster 2021-03-10T13:55:46.515897+0000 mgr.gibba032.whopeo (mgr.24625) 156 : cluster [DBG] pgmap v69: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[31774]: audit 2021-03-10T13:55:48.002335+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:48 gibba011 bash[31774]: audit 2021-03-10T13:55:48.006567+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.2","key": "container_image"}]: dispatch 2021-03-10T13:55:49.373 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:49 gibba011 bash[30791]: cephadm 2021-03-10T13:55:47.996378+0000 mgr.gibba032.whopeo (mgr.24625) 157 : cephadm [INF] Upgrade: Updating osd.2 2021-03-10T13:55:49.373 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:49 gibba011 bash[30791]: cephadm 2021-03-10T13:55:48.006279+0000 mgr.gibba032.whopeo (mgr.24625) 158 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:55:49.374 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:49 gibba011 bash[31774]: cephadm 2021-03-10T13:55:47.996378+0000 mgr.gibba032.whopeo (mgr.24625) 157 : cephadm [INF] Upgrade: Updating osd.2 2021-03-10T13:55:49.374 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:49 gibba011 bash[31774]: cephadm 2021-03-10T13:55:48.006279+0000 mgr.gibba032.whopeo (mgr.24625) 158 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:55:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:49 gibba032 bash[24004]: cephadm 2021-03-10T13:55:47.996378+0000 mgr.gibba032.whopeo (mgr.24625) 157 : cephadm [INF] Upgrade: Updating osd.2 2021-03-10T13:55:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:49 gibba032 bash[24004]: cephadm 2021-03-10T13:55:48.006279+0000 mgr.gibba032.whopeo (mgr.24625) 158 : cephadm [INF] Deploying daemon osd.2 on gibba011 2021-03-10T13:55:50.110 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:49 gibba011 systemd[1]: Stopping Ceph osd.2 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:55:50.111 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:50 gibba011 bash[17842]: debug 2021-03-10T13:55:50.036+0000 7f357e013700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:55:50.111 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:50 gibba011 bash[17842]: debug 2021-03-10T13:55:50.036+0000 7f357e013700 -1 osd.2 71 *** Got signal Terminated *** 2021-03-10T13:55:50.111 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:50 gibba011 bash[17842]: debug 2021-03-10T13:55:50.036+0000 7f357e013700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:55:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:50 gibba032 bash[24004]: cluster 2021-03-10T13:55:48.517671+0000 mgr.gibba032.whopeo (mgr.24625) 159 : cluster [DBG] pgmap v70: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:50 gibba011 bash[30791]: cluster 2021-03-10T13:55:48.517671+0000 mgr.gibba032.whopeo (mgr.24625) 159 : cluster [DBG] pgmap v70: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:50 gibba011 bash[31774]: cluster 2021-03-10T13:55:48.517671+0000 mgr.gibba032.whopeo (mgr.24625) 159 : cluster [DBG] pgmap v70: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:51.115 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:50 gibba011 docker[35714]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.2 2021-03-10T13:55:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.362920+0000 mon.a (mon.0) 287 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.362954+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:55:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363152+0000 mon.a (mon.0) 289 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363227+0000 mon.a (mon.0) 290 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363418+0000 mon.a (mon.0) 291 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363543+0000 mon.a (mon.0) 292 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363643+0000 mon.a (mon.0) 293 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363698+0000 mon.a (mon.0) 294 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363800+0000 mon.a (mon.0) 295 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363871+0000 mon.a (mon.0) 296 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.363994+0000 mon.a (mon.0) 297 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.364068+0000 mon.a (mon.0) 298 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.364139+0000 mon.a (mon.0) 299 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.364230+0000 mon.a (mon.0) 300 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.364321+0000 mon.a (mon.0) 301 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.762946+0000 mon.a (mon.0) 302 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.763274+0000 mon.a (mon.0) 303 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.763966+0000 mon.a (mon.0) 304 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.764285+0000 mon.a (mon.0) 305 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.764807+0000 mon.a (mon.0) 306 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.764990+0000 mon.a (mon.0) 307 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.765586+0000 mon.a (mon.0) 308 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.765829+0000 mon.a (mon.0) 309 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.768133+0000 mon.a (mon.0) 310 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.768467+0000 mon.a (mon.0) 311 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.768642+0000 mon.a (mon.0) 312 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.362920+0000 mon.a (mon.0) 287 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.362954+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:55:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363152+0000 mon.a (mon.0) 289 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363227+0000 mon.a (mon.0) 290 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363418+0000 mon.a (mon.0) 291 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363543+0000 mon.a (mon.0) 292 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363643+0000 mon.a (mon.0) 293 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363698+0000 mon.a (mon.0) 294 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363800+0000 mon.a (mon.0) 295 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363871+0000 mon.a (mon.0) 296 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.363994+0000 mon.a (mon.0) 297 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.364068+0000 mon.a (mon.0) 298 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.364139+0000 mon.a (mon.0) 299 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.364230+0000 mon.a (mon.0) 300 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.364321+0000 mon.a (mon.0) 301 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.762946+0000 mon.a (mon.0) 302 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.763274+0000 mon.a (mon.0) 303 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.763966+0000 mon.a (mon.0) 304 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.764285+0000 mon.a (mon.0) 305 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.764807+0000 mon.a (mon.0) 306 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.764990+0000 mon.a (mon.0) 307 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.765586+0000 mon.a (mon.0) 308 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.765829+0000 mon.a (mon.0) 309 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.768133+0000 mon.a (mon.0) 310 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.768467+0000 mon.a (mon.0) 311 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.768642+0000 mon.a (mon.0) 312 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.769035+0000 mon.a (mon.0) 313 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.769208+0000 mon.a (mon.0) 314 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.769587+0000 mon.a (mon.0) 315 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: audit 2021-03-10T13:55:50.983494+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:51.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.004652+0000 mon.a (mon.0) 317 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.005616+0000 mon.a (mon.0) 318 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.005839+0000 mon.a (mon.0) 319 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.344 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.006433+0000 mon.a (mon.0) 320 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.769035+0000 mon.a (mon.0) 313 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.769208+0000 mon.a (mon.0) 314 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.769587+0000 mon.a (mon.0) 315 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: audit 2021-03-10T13:55:50.983494+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:51.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.004652+0000 mon.a (mon.0) 317 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.005616+0000 mon.a (mon.0) 318 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.005839+0000 mon.a (mon.0) 319 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:51 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.006433+0000 mon.a (mon.0) 320 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.362920+0000 mon.a (mon.0) 287 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.362954+0000 mon.a (mon.0) 288 : cluster [INF] osd.2 failed (root=default,host=gibba011) (connection refused reported by osd.7) 2021-03-10T13:55:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363152+0000 mon.a (mon.0) 289 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363227+0000 mon.a (mon.0) 290 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363418+0000 mon.a (mon.0) 291 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363543+0000 mon.a (mon.0) 292 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363643+0000 mon.a (mon.0) 293 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363698+0000 mon.a (mon.0) 294 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363800+0000 mon.a (mon.0) 295 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363871+0000 mon.a (mon.0) 296 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.363994+0000 mon.a (mon.0) 297 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.364068+0000 mon.a (mon.0) 298 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.364139+0000 mon.a (mon.0) 299 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.364230+0000 mon.a (mon.0) 300 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.364321+0000 mon.a (mon.0) 301 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.762946+0000 mon.a (mon.0) 302 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.763274+0000 mon.a (mon.0) 303 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.763966+0000 mon.a (mon.0) 304 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.764285+0000 mon.a (mon.0) 305 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.764807+0000 mon.a (mon.0) 306 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.764990+0000 mon.a (mon.0) 307 : cluster [DBG] osd.2 reported immediately failed by osd.6 2021-03-10T13:55:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.765586+0000 mon.a (mon.0) 308 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.765829+0000 mon.a (mon.0) 309 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.768133+0000 mon.a (mon.0) 310 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.768467+0000 mon.a (mon.0) 311 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.768642+0000 mon.a (mon.0) 312 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.769035+0000 mon.a (mon.0) 313 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.769208+0000 mon.a (mon.0) 314 : cluster [DBG] osd.2 reported immediately failed by osd.0 2021-03-10T13:55:51.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.769587+0000 mon.a (mon.0) 315 : cluster [DBG] osd.2 reported immediately failed by osd.1 2021-03-10T13:55:51.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: audit 2021-03-10T13:55:50.983494+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:51.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.004652+0000 mon.a (mon.0) 317 : cluster [DBG] osd.2 reported immediately failed by osd.3 2021-03-10T13:55:51.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.005616+0000 mon.a (mon.0) 318 : cluster [DBG] osd.2 reported immediately failed by osd.5 2021-03-10T13:55:51.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.005839+0000 mon.a (mon.0) 319 : cluster [DBG] osd.2 reported immediately failed by osd.7 2021-03-10T13:55:51.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:51 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.006433+0000 mon.a (mon.0) 320 : cluster [DBG] osd.2 reported immediately failed by osd.4 2021-03-10T13:55:52.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[30791]: cluster 2021-03-10T13:55:50.519028+0000 mgr.gibba032.whopeo (mgr.24625) 160 : cluster [DBG] pgmap v71: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:52.248 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.111221+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:52.249 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[30791]: cluster 2021-03-10T13:55:51.135632+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e72: 8 total, 7 up, 8 in 2021-03-10T13:55:52.249 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[31774]: cluster 2021-03-10T13:55:50.519028+0000 mgr.gibba032.whopeo (mgr.24625) 160 : cluster [DBG] pgmap v71: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:52.249 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.111221+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:52.249 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:52 gibba011 bash[31774]: cluster 2021-03-10T13:55:51.135632+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e72: 8 total, 7 up, 8 in 2021-03-10T13:55:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:52 gibba032 bash[24004]: cluster 2021-03-10T13:55:50.519028+0000 mgr.gibba032.whopeo (mgr.24625) 160 : cluster [DBG] pgmap v71: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:52 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.111221+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:55:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:52 gibba032 bash[24004]: cluster 2021-03-10T13:55:51.135632+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e72: 8 total, 7 up, 8 in 2021-03-10T13:55:53.079 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:52 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.2.service: Succeeded. 2021-03-10T13:55:53.079 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:52 gibba011 systemd[1]: Stopped Ceph osd.2 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:53.079 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:52 gibba011 systemd[1]: Started Ceph osd.2 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:55:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[30791]: cluster 2021-03-10T13:55:52.131251+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e73: 8 total, 7 up, 8 in 2021-03-10T13:55:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[30791]: audit 2021-03-10T13:55:52.983841+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[30791]: audit 2021-03-10T13:55:52.987070+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[31774]: cluster 2021-03-10T13:55:52.131251+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e73: 8 total, 7 up, 8 in 2021-03-10T13:55:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[31774]: audit 2021-03-10T13:55:52.983841+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:53 gibba011 bash[31774]: audit 2021-03-10T13:55:52.987070+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:53 gibba032 bash[24004]: cluster 2021-03-10T13:55:52.131251+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e73: 8 total, 7 up, 8 in 2021-03-10T13:55:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:53 gibba032 bash[24004]: audit 2021-03-10T13:55:52.983841+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:53 gibba032 bash[24004]: audit 2021-03-10T13:55:52.987070+0000 mon.a (mon.0) 325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:55:54.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[30791]: cluster 2021-03-10T13:55:52.519735+0000 mgr.gibba032.whopeo (mgr.24625) 161 : cluster [DBG] pgmap v74: 129 pgs: 4 stale+active+clean, 21 peering, 18 active+clean+wait, 86 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:54.264 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[30791]: cluster 2021-03-10T13:55:53.129825+0000 mon.a (mon.0) 326 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2021-03-10T13:55:54.265 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[31774]: cluster 2021-03-10T13:55:52.519735+0000 mgr.gibba032.whopeo (mgr.24625) 161 : cluster [DBG] pgmap v74: 129 pgs: 4 stale+active+clean, 21 peering, 18 active+clean+wait, 86 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:54.265 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[31774]: cluster 2021-03-10T13:55:53.129825+0000 mon.a (mon.0) 326 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2021-03-10T13:55:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:54 gibba032 bash[24004]: cluster 2021-03-10T13:55:52.519735+0000 mgr.gibba032.whopeo (mgr.24625) 161 : cluster [DBG] pgmap v74: 129 pgs: 4 stale+active+clean, 21 peering, 18 active+clean+wait, 86 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail 2021-03-10T13:55:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:54 gibba032 bash[24004]: cluster 2021-03-10T13:55:53.129825+0000 mon.a (mon.0) 326 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2021-03-10T13:55:54.580 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-03-10T13:55:54.580 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2021-03-10T13:55:54.580 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2021-03-10T13:55:54.580 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2021-03-10T13:55:54.581 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-03-10T13:55:54.581 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-03-10T13:55:54.581 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:54 gibba011 bash[35892]: --> ceph-volume lvm activate successful for osd ID: 2 2021-03-10T13:55:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:55 gibba011 bash[30791]: cluster 2021-03-10T13:55:54.521113+0000 mgr.gibba032.whopeo (mgr.24625) 162 : cluster [DBG] pgmap v75: 129 pgs: 2 stale+active+clean, 1 active+undersized+degraded, 21 peering, 18 active+clean+wait, 4 active+undersized, 83 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 1/192 objects degraded (0.521%) 2021-03-10T13:55:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:55 gibba011 bash[30791]: cluster 2021-03-10T13:55:55.135043+0000 mon.a (mon.0) 327 : cluster [WRN] Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:55:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:55 gibba011 bash[31774]: cluster 2021-03-10T13:55:54.521113+0000 mgr.gibba032.whopeo (mgr.24625) 162 : cluster [DBG] pgmap v75: 129 pgs: 2 stale+active+clean, 1 active+undersized+degraded, 21 peering, 18 active+clean+wait, 4 active+undersized, 83 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 1/192 objects degraded (0.521%) 2021-03-10T13:55:56.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:55 gibba011 bash[31774]: cluster 2021-03-10T13:55:55.135043+0000 mon.a (mon.0) 327 : cluster [WRN] Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:55:56.081 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:55 gibba011 bash[36034]: debug 2021-03-10T13:55:55.724+0000 7f8986187700 1 -- 172.21.2.111:0/3301549813 <== mon.0 v2:172.21.2.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55987da8a4e0 con 0x55987db07400 2021-03-10T13:55:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:55 gibba032 bash[24004]: cluster 2021-03-10T13:55:54.521113+0000 mgr.gibba032.whopeo (mgr.24625) 162 : cluster [DBG] pgmap v75: 129 pgs: 2 stale+active+clean, 1 active+undersized+degraded, 21 peering, 18 active+clean+wait, 4 active+undersized, 83 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 1/192 objects degraded (0.521%) 2021-03-10T13:55:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:55 gibba032 bash[24004]: cluster 2021-03-10T13:55:55.135043+0000 mon.a (mon.0) 327 : cluster [WRN] Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2021-03-10T13:55:56.579 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[36034]: debug 2021-03-10T13:55:56.296+0000 7f898dc1c080 -1 Falling back to public interface 2021-03-10T13:55:56.910 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:55:57.140 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:55:57.140 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2m) 0s ago 7m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:55:57.141 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (7m) 89s ago 7m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:55:57.141 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2m) 89s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:55:57.141 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3m) 89s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:55:57.141 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2m) 0s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:55:57.142 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2m) 0s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:55:57.142 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (100s) 89s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:55:57.142 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (111s) 0s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:55:57.143 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (8m) 0s ago 8m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:55:57.143 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (8m) 89s ago 8m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:55:57.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (53s) 0s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:55:57.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (27s) 0s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:55:57.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2s) 0s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:55:57.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (9m) 0s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 61c9f8aa39e9 2021-03-10T13:55:57.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (9m) 89s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:55:57.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (9m) 89s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:55:57.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (8m) 89s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:55:57.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (8m) 89s ago 8m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:55:57.145 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (7m) 89s ago 8m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:55:57.145 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 0s ago 6m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:55:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:55.986222+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:55:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:55.987163+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:55:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.763909+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.767365+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.771432+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.772541+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.773567+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.774503+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:56 gibba032 bash[24004]: audit 2021-03-10T13:55:56.776536+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:55.986222+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:55:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:55.987163+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:55:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.763909+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.767365+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.771432+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.772541+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.773567+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.774503+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[30791]: audit 2021-03-10T13:55:56.776536+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:55.986222+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:55:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:55.987163+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:55:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.763909+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.767365+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:55:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.771432+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:55:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.772541+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.773567+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.774503+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:56 gibba011 bash[31774]: audit 2021-03-10T13:55:56.776536+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:57.393 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:55:57.394 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:55:57.394 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:55:57.394 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:57.394 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:55:57.395 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:55:57.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:57.395 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:55:57.395 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 5, 2021-03-10T13:55:57.396 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 2 2021-03-10T13:55:57.396 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:55:57.396 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:55:57.396 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:55:57.397 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 5, 2021-03-10T13:55:57.397 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T13:55:57.397 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:55:57.397 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:55:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cluster 2021-03-10T13:55:56.522716+0000 mgr.gibba032.whopeo (mgr.24625) 163 : cluster [DBG] pgmap v76: 129 pgs: 3 active+undersized+degraded, 21 peering, 8 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:55:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.770900+0000 mgr.gibba032.whopeo (mgr.24625) 164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.772382+0000 mgr.gibba032.whopeo (mgr.24625) 165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.773364+0000 mgr.gibba032.whopeo (mgr.24625) 166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.773408+0000 mgr.gibba032.whopeo (mgr.24625) 167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.774371+0000 mgr.gibba032.whopeo (mgr.24625) 168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.774415+0000 mgr.gibba032.whopeo (mgr.24625) 169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.775250+0000 mgr.gibba032.whopeo (mgr.24625) 170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.776141+0000 mgr.gibba032.whopeo (mgr.24625) 171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.776188+0000 mgr.gibba032.whopeo (mgr.24625) 172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: audit 2021-03-10T13:55:56.776839+0000 mgr.gibba032.whopeo (mgr.24625) 173 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: cephadm 2021-03-10T13:55:56.777903+0000 mgr.gibba032.whopeo (mgr.24625) 174 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: audit 2021-03-10T13:55:56.901780+0000 mgr.gibba032.whopeo (mgr.24625) 175 : audit [DBG] from='client.54172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: audit 2021-03-10T13:55:57.128389+0000 mgr.gibba032.whopeo (mgr.24625) 176 : audit [DBG] from='client.54178 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:57 gibba032 bash[24004]: audit 2021-03-10T13:55:57.392727+0000 mon.c (mon.1) 6 : audit [DBG] from='client.? 172.21.2.111:0/4133192320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cluster 2021-03-10T13:55:56.522716+0000 mgr.gibba032.whopeo (mgr.24625) 163 : cluster [DBG] pgmap v76: 129 pgs: 3 active+undersized+degraded, 21 peering, 8 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:55:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.770900+0000 mgr.gibba032.whopeo (mgr.24625) 164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.772382+0000 mgr.gibba032.whopeo (mgr.24625) 165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.773364+0000 mgr.gibba032.whopeo (mgr.24625) 166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.773408+0000 mgr.gibba032.whopeo (mgr.24625) 167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.774371+0000 mgr.gibba032.whopeo (mgr.24625) 168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.774415+0000 mgr.gibba032.whopeo (mgr.24625) 169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.775250+0000 mgr.gibba032.whopeo (mgr.24625) 170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.776141+0000 mgr.gibba032.whopeo (mgr.24625) 171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.776188+0000 mgr.gibba032.whopeo (mgr.24625) 172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: audit 2021-03-10T13:55:56.776839+0000 mgr.gibba032.whopeo (mgr.24625) 173 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: cephadm 2021-03-10T13:55:56.777903+0000 mgr.gibba032.whopeo (mgr.24625) 174 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: audit 2021-03-10T13:55:56.901780+0000 mgr.gibba032.whopeo (mgr.24625) 175 : audit [DBG] from='client.54172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: audit 2021-03-10T13:55:57.128389+0000 mgr.gibba032.whopeo (mgr.24625) 176 : audit [DBG] from='client.54178 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[30791]: audit 2021-03-10T13:55:57.392727+0000 mon.c (mon.1) 6 : audit [DBG] from='client.? 172.21.2.111:0/4133192320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cluster 2021-03-10T13:55:56.522716+0000 mgr.gibba032.whopeo (mgr.24625) 163 : cluster [DBG] pgmap v76: 129 pgs: 3 active+undersized+degraded, 21 peering, 8 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:55:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.770900+0000 mgr.gibba032.whopeo (mgr.24625) 164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:55:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.772382+0000 mgr.gibba032.whopeo (mgr.24625) 165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:55:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.773364+0000 mgr.gibba032.whopeo (mgr.24625) 166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:55:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.773408+0000 mgr.gibba032.whopeo (mgr.24625) 167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:55:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.774371+0000 mgr.gibba032.whopeo (mgr.24625) 168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:55:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.774415+0000 mgr.gibba032.whopeo (mgr.24625) 169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:55:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.775250+0000 mgr.gibba032.whopeo (mgr.24625) 170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:55:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.776141+0000 mgr.gibba032.whopeo (mgr.24625) 171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:55:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.776188+0000 mgr.gibba032.whopeo (mgr.24625) 172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:55:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: audit 2021-03-10T13:55:56.776839+0000 mgr.gibba032.whopeo (mgr.24625) 173 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:55:58.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: cephadm 2021-03-10T13:55:56.777903+0000 mgr.gibba032.whopeo (mgr.24625) 174 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2021-03-10T13:55:58.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: audit 2021-03-10T13:55:56.901780+0000 mgr.gibba032.whopeo (mgr.24625) 175 : audit [DBG] from='client.54172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: audit 2021-03-10T13:55:57.128389+0000 mgr.gibba032.whopeo (mgr.24625) 176 : audit [DBG] from='client.54178 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:55:58.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:57 gibba011 bash[31774]: audit 2021-03-10T13:55:57.392727+0000 mon.c (mon.1) 6 : audit [DBG] from='client.? 172.21.2.111:0/4133192320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:55:59.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:58 gibba032 bash[24004]: cluster 2021-03-10T13:55:58.882999+0000 mon.a (mon.0) 337 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 6 pgs peering) 2021-03-10T13:55:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:58 gibba011 bash[30791]: cluster 2021-03-10T13:55:58.882999+0000 mon.a (mon.0) 337 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 6 pgs peering) 2021-03-10T13:55:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:58 gibba011 bash[31774]: cluster 2021-03-10T13:55:58.882999+0000 mon.a (mon.0) 337 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 6 pgs peering) 2021-03-10T13:55:59.947 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[36034]: debug 2021-03-10T13:55:59.588+0000 7f898dc1c080 -1 bluestore(/var/lib/ceph/osd/ceph-2) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:55:59.948 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[36034]: debug 2021-03-10T13:55:59.588+0000 7f89796e2700 -1 bluestore(/var/lib/ceph/osd/ceph-2) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:55:59.948 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[36034]: debug 2021-03-10T13:55:59.708+0000 7f898dc1c080 -1 osd.2 71 log_to_monitors {default=true} 2021-03-10T13:56:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:59 gibba032 bash[24004]: cluster 2021-03-10T13:55:58.523594+0000 mgr.gibba032.whopeo (mgr.24625) 177 : cluster [DBG] pgmap v77: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:55:59 gibba032 bash[24004]: audit 2021-03-10T13:55:59.717237+0000 mon.a (mon.0) 338 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:56:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[30791]: cluster 2021-03-10T13:55:58.523594+0000 mgr.gibba032.whopeo (mgr.24625) 177 : cluster [DBG] pgmap v77: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[30791]: audit 2021-03-10T13:55:59.717237+0000 mon.a (mon.0) 338 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:56:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[31774]: cluster 2021-03-10T13:55:58.523594+0000 mgr.gibba032.whopeo (mgr.24625) 177 : cluster [DBG] pgmap v77: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 17 MiB used, 666 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[31774]: audit 2021-03-10T13:55:59.717237+0000 mon.a (mon.0) 338 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-03-10T13:56:00.331 INFO:journalctl@ceph.osd.2.gibba011.stdout:Mar 10 13:55:59 gibba011 bash[36034]: debug 2021-03-10T13:55:59.948+0000 7f897f8fd700 -1 osd.2 71 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:56:01.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:00 gibba032 bash[24004]: audit 2021-03-10T13:55:59.943878+0000 mon.a (mon.0) 339 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:56:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:00 gibba032 bash[24004]: cluster 2021-03-10T13:55:59.944035+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e74: 8 total, 7 up, 8 in 2021-03-10T13:56:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:00 gibba032 bash[24004]: audit 2021-03-10T13:55:59.945182+0000 mon.a (mon.0) 341 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:01.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[30791]: audit 2021-03-10T13:55:59.943878+0000 mon.a (mon.0) 339 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:56:01.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[30791]: cluster 2021-03-10T13:55:59.944035+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e74: 8 total, 7 up, 8 in 2021-03-10T13:56:01.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[30791]: audit 2021-03-10T13:55:59.945182+0000 mon.a (mon.0) 341 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:01.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[31774]: audit 2021-03-10T13:55:59.943878+0000 mon.a (mon.0) 339 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-03-10T13:56:01.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[31774]: cluster 2021-03-10T13:55:59.944035+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e74: 8 total, 7 up, 8 in 2021-03-10T13:56:01.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:00 gibba011 bash[31774]: audit 2021-03-10T13:55:59.945182+0000 mon.a (mon.0) 341 : audit [INF] from='osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:02.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:01 gibba032 bash[24004]: cluster 2021-03-10T13:56:00.525087+0000 mgr.gibba032.whopeo (mgr.24625) 178 : cluster [DBG] pgmap v79: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:01 gibba032 bash[24004]: cluster 2021-03-10T13:56:00.947890+0000 mon.a (mon.0) 342 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:01 gibba032 bash[24004]: cluster 2021-03-10T13:56:00.959149+0000 mon.a (mon.0) 343 : cluster [INF] osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099] boot 2021-03-10T13:56:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:01 gibba032 bash[24004]: cluster 2021-03-10T13:56:00.959209+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2021-03-10T13:56:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:01 gibba032 bash[24004]: audit 2021-03-10T13:56:00.959614+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:56:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[30791]: cluster 2021-03-10T13:56:00.525087+0000 mgr.gibba032.whopeo (mgr.24625) 178 : cluster [DBG] pgmap v79: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[30791]: cluster 2021-03-10T13:56:00.947890+0000 mon.a (mon.0) 342 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[30791]: cluster 2021-03-10T13:56:00.959149+0000 mon.a (mon.0) 343 : cluster [INF] osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099] boot 2021-03-10T13:56:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[30791]: cluster 2021-03-10T13:56:00.959209+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2021-03-10T13:56:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[30791]: audit 2021-03-10T13:56:00.959614+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:56:02.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[31774]: cluster 2021-03-10T13:56:00.525087+0000 mgr.gibba032.whopeo (mgr.24625) 178 : cluster [DBG] pgmap v79: 129 pgs: 10 active+undersized+degraded, 22 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 15/192 objects degraded (7.812%) 2021-03-10T13:56:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[31774]: cluster 2021-03-10T13:56:00.947890+0000 mon.a (mon.0) 342 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[31774]: cluster 2021-03-10T13:56:00.959149+0000 mon.a (mon.0) 343 : cluster [INF] osd.2 [v2:172.21.2.111:6816/2837477099,v1:172.21.2.111:6817/2837477099] boot 2021-03-10T13:56:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[31774]: cluster 2021-03-10T13:56:00.959209+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2021-03-10T13:56:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:01 gibba011 bash[31774]: audit 2021-03-10T13:56:00.959614+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-03-10T13:56:03.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:02 gibba032 bash[24004]: cluster 2021-03-10T13:56:01.962356+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2021-03-10T13:56:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:02 gibba011 bash[30791]: cluster 2021-03-10T13:56:01.962356+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2021-03-10T13:56:03.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:02 gibba011 bash[31774]: cluster 2021-03-10T13:56:01.962356+0000 mon.a (mon.0) 346 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2021-03-10T13:56:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:03 gibba032 bash[24004]: cluster 2021-03-10T13:56:02.526149+0000 mgr.gibba032.whopeo (mgr.24625) 179 : cluster [DBG] pgmap v82: 129 pgs: 15 peering, 5 active+undersized+degraded, 12 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:56:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:03 gibba032 bash[24004]: cluster 2021-03-10T13:56:02.966159+0000 mon.a (mon.0) 347 : cluster [WRN] Health check update: Degraded data redundancy: 9/192 objects degraded (4.688%), 5 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:04.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:03 gibba011 bash[30791]: cluster 2021-03-10T13:56:02.526149+0000 mgr.gibba032.whopeo (mgr.24625) 179 : cluster [DBG] pgmap v82: 129 pgs: 15 peering, 5 active+undersized+degraded, 12 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:56:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:03 gibba011 bash[30791]: cluster 2021-03-10T13:56:02.966159+0000 mon.a (mon.0) 347 : cluster [WRN] Health check update: Degraded data redundancy: 9/192 objects degraded (4.688%), 5 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:03 gibba011 bash[31774]: cluster 2021-03-10T13:56:02.526149+0000 mgr.gibba032.whopeo (mgr.24625) 179 : cluster [DBG] pgmap v82: 129 pgs: 15 peering, 5 active+undersized+degraded, 12 active+undersized, 97 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:56:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:03 gibba011 bash[31774]: cluster 2021-03-10T13:56:02.966159+0000 mon.a (mon.0) 347 : cluster [WRN] Health check update: Degraded data redundancy: 9/192 objects degraded (4.688%), 5 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:06.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:05 gibba032 bash[24004]: cluster 2021-03-10T13:56:04.526782+0000 mgr.gibba032.whopeo (mgr.24625) 180 : cluster [DBG] pgmap v83: 129 pgs: 15 peering, 4 active+undersized+degraded, 10 active+undersized, 100 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:56:06.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:05 gibba011 bash[30791]: cluster 2021-03-10T13:56:04.526782+0000 mgr.gibba032.whopeo (mgr.24625) 180 : cluster [DBG] pgmap v83: 129 pgs: 15 peering, 4 active+undersized+degraded, 10 active+undersized, 100 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:56:06.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:05 gibba011 bash[31774]: cluster 2021-03-10T13:56:04.526782+0000 mgr.gibba032.whopeo (mgr.24625) 180 : cluster [DBG] pgmap v83: 129 pgs: 15 peering, 4 active+undersized+degraded, 10 active+undersized, 100 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:56:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:07 gibba011 bash[30791]: cluster 2021-03-10T13:56:06.983116+0000 mon.a (mon.0) 348 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 4 pgs degraded) 2021-03-10T13:56:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:07 gibba011 bash[31774]: cluster 2021-03-10T13:56:06.983116+0000 mon.a (mon.0) 348 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 4 pgs degraded) 2021-03-10T13:56:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:07 gibba032 bash[24004]: cluster 2021-03-10T13:56:06.983116+0000 mon.a (mon.0) 348 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 4 pgs degraded) 2021-03-10T13:56:08.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:08 gibba011 bash[30791]: cluster 2021-03-10T13:56:06.528007+0000 mgr.gibba032.whopeo (mgr.24625) 181 : cluster [DBG] pgmap v84: 129 pgs: 10 active+clean+wait, 15 peering, 104 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:08.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:08 gibba011 bash[31774]: cluster 2021-03-10T13:56:06.528007+0000 mgr.gibba032.whopeo (mgr.24625) 181 : cluster [DBG] pgmap v84: 129 pgs: 10 active+clean+wait, 15 peering, 104 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:08 gibba032 bash[24004]: cluster 2021-03-10T13:56:06.528007+0000 mgr.gibba032.whopeo (mgr.24625) 181 : cluster [DBG] pgmap v84: 129 pgs: 10 active+clean+wait, 15 peering, 104 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:10.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:10 gibba011 bash[30791]: cluster 2021-03-10T13:56:08.529686+0000 mgr.gibba032.whopeo (mgr.24625) 182 : cluster [DBG] pgmap v85: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:10 gibba011 bash[31774]: cluster 2021-03-10T13:56:08.529686+0000 mgr.gibba032.whopeo (mgr.24625) 182 : cluster [DBG] pgmap v85: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:10 gibba032 bash[24004]: cluster 2021-03-10T13:56:08.529686+0000 mgr.gibba032.whopeo (mgr.24625) 182 : cluster [DBG] pgmap v85: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:12 gibba011 bash[30791]: cluster 2021-03-10T13:56:10.530499+0000 mgr.gibba032.whopeo (mgr.24625) 183 : cluster [DBG] pgmap v86: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:12 gibba011 bash[30791]: audit 2021-03-10T13:56:11.794666+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:12 gibba011 bash[31774]: cluster 2021-03-10T13:56:10.530499+0000 mgr.gibba032.whopeo (mgr.24625) 183 : cluster [DBG] pgmap v86: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:12 gibba011 bash[31774]: audit 2021-03-10T13:56:11.794666+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:12 gibba032 bash[24004]: cluster 2021-03-10T13:56:10.530499+0000 mgr.gibba032.whopeo (mgr.24625) 183 : cluster [DBG] pgmap v86: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:12 gibba032 bash[24004]: audit 2021-03-10T13:56:11.794666+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:13.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:13 gibba011 bash[30791]: audit 2021-03-10T13:56:11.795687+0000 mgr.gibba032.whopeo (mgr.24625) 184 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:13.273 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:13 gibba011 bash[30791]: cephadm 2021-03-10T13:56:11.801053+0000 mgr.gibba032.whopeo (mgr.24625) 185 : cephadm [INF] Upgrade: osd.3 is safe to restart 2021-03-10T13:56:13.274 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:13 gibba011 bash[31774]: audit 2021-03-10T13:56:11.795687+0000 mgr.gibba032.whopeo (mgr.24625) 184 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:13.274 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:13 gibba011 bash[31774]: cephadm 2021-03-10T13:56:11.801053+0000 mgr.gibba032.whopeo (mgr.24625) 185 : cephadm [INF] Upgrade: osd.3 is safe to restart 2021-03-10T13:56:13.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:13 gibba032 bash[24004]: audit 2021-03-10T13:56:11.795687+0000 mgr.gibba032.whopeo (mgr.24625) 184 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-03-10T13:56:13.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:13 gibba032 bash[24004]: cephadm 2021-03-10T13:56:11.801053+0000 mgr.gibba032.whopeo (mgr.24625) 185 : cephadm [INF] Upgrade: osd.3 is safe to restart 2021-03-10T13:56:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[30791]: cluster 2021-03-10T13:56:12.532224+0000 mgr.gibba032.whopeo (mgr.24625) 186 : cluster [DBG] pgmap v87: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[30791]: audit 2021-03-10T13:56:13.746885+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[30791]: audit 2021-03-10T13:56:13.751843+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:56:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[31774]: cluster 2021-03-10T13:56:12.532224+0000 mgr.gibba032.whopeo (mgr.24625) 186 : cluster [DBG] pgmap v87: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[31774]: audit 2021-03-10T13:56:13.746885+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:14 gibba011 bash[31774]: audit 2021-03-10T13:56:13.751843+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:56:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:14 gibba032 bash[24004]: cluster 2021-03-10T13:56:12.532224+0000 mgr.gibba032.whopeo (mgr.24625) 186 : cluster [DBG] pgmap v87: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:14 gibba032 bash[24004]: audit 2021-03-10T13:56:13.746885+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:14 gibba032 bash[24004]: audit 2021-03-10T13:56:13.751843+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.3","key": "container_image"}]: dispatch 2021-03-10T13:56:15.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[30791]: cephadm 2021-03-10T13:56:13.740997+0000 mgr.gibba032.whopeo (mgr.24625) 187 : cephadm [INF] Upgrade: Updating osd.3 2021-03-10T13:56:15.139 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[30791]: cephadm 2021-03-10T13:56:13.750876+0000 mgr.gibba032.whopeo (mgr.24625) 188 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:56:15.140 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[31774]: cephadm 2021-03-10T13:56:13.740997+0000 mgr.gibba032.whopeo (mgr.24625) 187 : cephadm [INF] Upgrade: Updating osd.3 2021-03-10T13:56:15.140 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[31774]: cephadm 2021-03-10T13:56:13.750876+0000 mgr.gibba032.whopeo (mgr.24625) 188 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:56:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:15 gibba032 bash[24004]: cephadm 2021-03-10T13:56:13.740997+0000 mgr.gibba032.whopeo (mgr.24625) 187 : cephadm [INF] Upgrade: Updating osd.3 2021-03-10T13:56:15.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:15 gibba032 bash[24004]: cephadm 2021-03-10T13:56:13.750876+0000 mgr.gibba032.whopeo (mgr.24625) 188 : cephadm [INF] Deploying daemon osd.3 on gibba011 2021-03-10T13:56:15.969 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:15 gibba011 systemd[1]: Stopping Ceph osd.3 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:56:15.970 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[19312]: debug 2021-03-10T13:56:15.804+0000 7fbc1e80c700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:56:15.970 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[19312]: debug 2021-03-10T13:56:15.804+0000 7fbc1e80c700 -1 osd.3 76 *** Got signal Terminated *** 2021-03-10T13:56:15.971 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:15 gibba011 bash[19312]: debug 2021-03-10T13:56:15.804+0000 7fbc1e80c700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:56:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:16 gibba011 bash[30791]: cluster 2021-03-10T13:56:14.533403+0000 mgr.gibba032.whopeo (mgr.24625) 189 : cluster [DBG] pgmap v88: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:16 gibba011 bash[30791]: audit 2021-03-10T13:56:15.993539+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:16 gibba011 bash[31774]: cluster 2021-03-10T13:56:14.533403+0000 mgr.gibba032.whopeo (mgr.24625) 189 : cluster [DBG] pgmap v88: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:16 gibba011 bash[31774]: audit 2021-03-10T13:56:15.993539+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:16 gibba032 bash[24004]: cluster 2021-03-10T13:56:14.533403+0000 mgr.gibba032.whopeo (mgr.24625) 189 : cluster [DBG] pgmap v88: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:16 gibba032 bash[24004]: audit 2021-03-10T13:56:15.993539+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:17.079 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:16 gibba011 docker[36953]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.3 2021-03-10T13:56:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.111524+0000 mon.a (mon.0) 353 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.111640+0000 mon.a (mon.0) 354 : cluster [INF] osd.3 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:56:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.112318+0000 mon.a (mon.0) 355 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.112510+0000 mon.a (mon.0) 356 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.112916+0000 mon.a (mon.0) 357 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.113079+0000 mon.a (mon.0) 358 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.113432+0000 mon.a (mon.0) 359 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.113598+0000 mon.a (mon.0) 360 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.114144+0000 mon.a (mon.0) 361 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.114317+0000 mon.a (mon.0) 362 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.114662+0000 mon.a (mon.0) 363 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.114821+0000 mon.a (mon.0) 364 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.115191+0000 mon.a (mon.0) 365 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.115386+0000 mon.a (mon.0) 366 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.115856+0000 mon.a (mon.0) 367 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.512408+0000 mon.a (mon.0) 368 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.512877+0000 mon.a (mon.0) 369 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.513219+0000 mon.a (mon.0) 370 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.513663+0000 mon.a (mon.0) 371 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.513977+0000 mon.a (mon.0) 372 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.514279+0000 mon.a (mon.0) 373 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.514580+0000 mon.a (mon.0) 374 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.515008+0000 mon.a (mon.0) 375 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.516097+0000 mon.a (mon.0) 376 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.516423+0000 mon.a (mon.0) 377 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.516733+0000 mon.a (mon.0) 378 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.517084+0000 mon.a (mon.0) 379 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.517409+0000 mon.a (mon.0) 380 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.517774+0000 mon.a (mon.0) 381 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.625639+0000 mon.a (mon.0) 382 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.827405+0000 mon.a (mon.0) 383 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.999313+0000 mon.a (mon.0) 384 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:17 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.999728+0000 mon.a (mon.0) 385 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.111524+0000 mon.a (mon.0) 353 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.111640+0000 mon.a (mon.0) 354 : cluster [INF] osd.3 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:56:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.112318+0000 mon.a (mon.0) 355 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.112510+0000 mon.a (mon.0) 356 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.112916+0000 mon.a (mon.0) 357 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.113079+0000 mon.a (mon.0) 358 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.113432+0000 mon.a (mon.0) 359 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.113598+0000 mon.a (mon.0) 360 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.114144+0000 mon.a (mon.0) 361 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.114317+0000 mon.a (mon.0) 362 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.114662+0000 mon.a (mon.0) 363 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.114821+0000 mon.a (mon.0) 364 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.115191+0000 mon.a (mon.0) 365 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.115386+0000 mon.a (mon.0) 366 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.115856+0000 mon.a (mon.0) 367 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.512408+0000 mon.a (mon.0) 368 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.512877+0000 mon.a (mon.0) 369 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.513219+0000 mon.a (mon.0) 370 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.513663+0000 mon.a (mon.0) 371 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.513977+0000 mon.a (mon.0) 372 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.514279+0000 mon.a (mon.0) 373 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.514580+0000 mon.a (mon.0) 374 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.515008+0000 mon.a (mon.0) 375 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.516097+0000 mon.a (mon.0) 376 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.516423+0000 mon.a (mon.0) 377 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.516733+0000 mon.a (mon.0) 378 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.111524+0000 mon.a (mon.0) 353 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.111640+0000 mon.a (mon.0) 354 : cluster [INF] osd.3 failed (root=default,host=gibba011) (connection refused reported by osd.5) 2021-03-10T13:56:17.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.112318+0000 mon.a (mon.0) 355 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.112510+0000 mon.a (mon.0) 356 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.112916+0000 mon.a (mon.0) 357 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.113079+0000 mon.a (mon.0) 358 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.113432+0000 mon.a (mon.0) 359 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.113598+0000 mon.a (mon.0) 360 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.114144+0000 mon.a (mon.0) 361 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.114317+0000 mon.a (mon.0) 362 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.114662+0000 mon.a (mon.0) 363 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.114821+0000 mon.a (mon.0) 364 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.115191+0000 mon.a (mon.0) 365 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.115386+0000 mon.a (mon.0) 366 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.115856+0000 mon.a (mon.0) 367 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.512408+0000 mon.a (mon.0) 368 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.512877+0000 mon.a (mon.0) 369 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.513219+0000 mon.a (mon.0) 370 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.513663+0000 mon.a (mon.0) 371 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.513977+0000 mon.a (mon.0) 372 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.514279+0000 mon.a (mon.0) 373 : cluster [DBG] osd.3 reported immediately failed by osd.7 2021-03-10T13:56:17.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.514580+0000 mon.a (mon.0) 374 : cluster [DBG] osd.3 reported immediately failed by osd.6 2021-03-10T13:56:17.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.515008+0000 mon.a (mon.0) 375 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.516097+0000 mon.a (mon.0) 376 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.516423+0000 mon.a (mon.0) 377 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.516733+0000 mon.a (mon.0) 378 : cluster [DBG] osd.3 reported immediately failed by osd.0 2021-03-10T13:56:17.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.517084+0000 mon.a (mon.0) 379 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.517409+0000 mon.a (mon.0) 380 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.517774+0000 mon.a (mon.0) 381 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.625639+0000 mon.a (mon.0) 382 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.827405+0000 mon.a (mon.0) 383 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.999313+0000 mon.a (mon.0) 384 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.999728+0000 mon.a (mon.0) 385 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.517084+0000 mon.a (mon.0) 379 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.517409+0000 mon.a (mon.0) 380 : cluster [DBG] osd.3 reported immediately failed by osd.2 2021-03-10T13:56:17.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.517774+0000 mon.a (mon.0) 381 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:17.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.625639+0000 mon.a (mon.0) 382 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.827405+0000 mon.a (mon.0) 383 : cluster [DBG] osd.3 reported immediately failed by osd.4 2021-03-10T13:56:17.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.999313+0000 mon.a (mon.0) 384 : cluster [DBG] osd.3 reported immediately failed by osd.5 2021-03-10T13:56:17.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:17 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.999728+0000 mon.a (mon.0) 385 : cluster [DBG] osd.3 reported immediately failed by osd.1 2021-03-10T13:56:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[30791]: cluster 2021-03-10T13:56:16.534209+0000 mgr.gibba032.whopeo (mgr.24625) 190 : cluster [DBG] pgmap v89: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[30791]: cluster 2021-03-10T13:56:17.081599+0000 mon.a (mon.0) 386 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[30791]: cluster 2021-03-10T13:56:17.100823+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e77: 8 total, 7 up, 8 in 2021-03-10T13:56:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[31774]: cluster 2021-03-10T13:56:16.534209+0000 mgr.gibba032.whopeo (mgr.24625) 190 : cluster [DBG] pgmap v89: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[31774]: cluster 2021-03-10T13:56:17.081599+0000 mon.a (mon.0) 386 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:18 gibba011 bash[31774]: cluster 2021-03-10T13:56:17.100823+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e77: 8 total, 7 up, 8 in 2021-03-10T13:56:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:18 gibba032 bash[24004]: cluster 2021-03-10T13:56:16.534209+0000 mgr.gibba032.whopeo (mgr.24625) 190 : cluster [DBG] pgmap v89: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:18 gibba032 bash[24004]: cluster 2021-03-10T13:56:17.081599+0000 mon.a (mon.0) 386 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:18 gibba032 bash[24004]: cluster 2021-03-10T13:56:17.100823+0000 mon.a (mon.0) 387 : cluster [DBG] osdmap e77: 8 total, 7 up, 8 in 2021-03-10T13:56:18.829 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:18 gibba011 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.3.service: Succeeded. 2021-03-10T13:56:18.829 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:18 gibba011 systemd[1]: Stopped Ceph osd.3 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:56:18.830 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:18 gibba011 systemd[1]: Started Ceph osd.3 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:56:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:19 gibba032 bash[24004]: cluster 2021-03-10T13:56:18.102998+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e78: 8 total, 7 up, 8 in 2021-03-10T13:56:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:19 gibba032 bash[24004]: audit 2021-03-10T13:56:18.712272+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:19 gibba032 bash[24004]: audit 2021-03-10T13:56:18.715575+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[30791]: cluster 2021-03-10T13:56:18.102998+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e78: 8 total, 7 up, 8 in 2021-03-10T13:56:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[30791]: audit 2021-03-10T13:56:18.712272+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[30791]: audit 2021-03-10T13:56:18.715575+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:19.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[31774]: cluster 2021-03-10T13:56:18.102998+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e78: 8 total, 7 up, 8 in 2021-03-10T13:56:19.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[31774]: audit 2021-03-10T13:56:18.712272+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:19.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:19 gibba011 bash[31774]: audit 2021-03-10T13:56:18.715575+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[30791]: cluster 2021-03-10T13:56:18.535160+0000 mgr.gibba032.whopeo (mgr.24625) 191 : cluster [DBG] pgmap v92: 129 pgs: 20 stale+active+clean, 10 active+clean+wait, 99 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[30791]: cephadm 2021-03-10T13:56:18.715657+0000 mgr.gibba032.whopeo (mgr.24625) 192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:56:20.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[31774]: cluster 2021-03-10T13:56:18.535160+0000 mgr.gibba032.whopeo (mgr.24625) 191 : cluster [DBG] pgmap v92: 129 pgs: 20 stale+active+clean, 10 active+clean+wait, 99 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[31774]: cephadm 2021-03-10T13:56:18.715657+0000 mgr.gibba032.whopeo (mgr.24625) 192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:56:20.331 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-03-10T13:56:20.331 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2021-03-10T13:56:20.332 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3/block 2021-03-10T13:56:20.332 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2021-03-10T13:56:20.332 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2021-03-10T13:56:20.332 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-03-10T13:56:20.332 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:20 gibba011 bash[37141]: --> ceph-volume lvm activate successful for osd ID: 3 2021-03-10T13:56:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:20 gibba032 bash[24004]: cluster 2021-03-10T13:56:18.535160+0000 mgr.gibba032.whopeo (mgr.24625) 191 : cluster [DBG] pgmap v92: 129 pgs: 20 stale+active+clean, 10 active+clean+wait, 99 active+clean; 7.0 KiB data, 22 MiB used, 667 GiB / 672 GiB avail 2021-03-10T13:56:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:20 gibba032 bash[24004]: cephadm 2021-03-10T13:56:18.715657+0000 mgr.gibba032.whopeo (mgr.24625) 192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:56:21.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:21 gibba032 bash[24004]: cluster 2021-03-10T13:56:21.124288+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Degraded data redundancy: 13/192 objects degraded (6.771%), 11 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:21.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:21 gibba011 bash[30791]: cluster 2021-03-10T13:56:21.124288+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Degraded data redundancy: 13/192 objects degraded (6.771%), 11 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:21 gibba011 bash[31774]: cluster 2021-03-10T13:56:21.124288+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Degraded data redundancy: 13/192 objects degraded (6.771%), 11 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:22.079 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:21 gibba011 bash[37275]: debug 2021-03-10T13:56:21.608+0000 7fe5c7bc4700 1 -- 172.21.2.111:0/2915624519 <== mon.0 v2:172.21.2.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dcd5d704e0 con 0x55dcd5ded400 2021-03-10T13:56:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:22 gibba032 bash[24004]: cluster 2021-03-10T13:56:20.536953+0000 mgr.gibba032.whopeo (mgr.24625) 193 : cluster [DBG] pgmap v93: 129 pgs: 9 stale+active+clean, 11 active+undersized+degraded, 17 active+undersized, 92 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:22 gibba011 bash[30791]: cluster 2021-03-10T13:56:20.536953+0000 mgr.gibba032.whopeo (mgr.24625) 193 : cluster [DBG] pgmap v93: 129 pgs: 9 stale+active+clean, 11 active+undersized+degraded, 17 active+undersized, 92 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:22 gibba011 bash[31774]: cluster 2021-03-10T13:56:20.536953+0000 mgr.gibba032.whopeo (mgr.24625) 193 : cluster [DBG] pgmap v93: 129 pgs: 9 stale+active+clean, 11 active+undersized+degraded, 17 active+undersized, 92 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:22.581 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:22 gibba011 bash[37275]: debug 2021-03-10T13:56:22.168+0000 7fe5cf659080 -1 Falling back to public interface 2021-03-10T13:56:23.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.195501+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.196111+0000 mgr.gibba032.whopeo (mgr.24625) 194 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:56:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.412672+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.421716+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.424136+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.425745+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.426876+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:23 gibba032 bash[24004]: audit 2021-03-10T13:56:22.429407+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.195501+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.196111+0000 mgr.gibba032.whopeo (mgr.24625) 194 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:56:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.412672+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.421716+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.424136+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.425745+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.426876+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[30791]: audit 2021-03-10T13:56:22.429407+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.195501+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.196111+0000 mgr.gibba032.whopeo (mgr.24625) 194 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:56:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.412672+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.421716+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.424136+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.425745+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.426876+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:23 gibba011 bash[31774]: audit 2021-03-10T13:56:22.429407+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.420646+0000 mgr.gibba032.whopeo (mgr.24625) 195 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.423462+0000 mgr.gibba032.whopeo (mgr.24625) 196 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.425370+0000 mgr.gibba032.whopeo (mgr.24625) 197 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.425408+0000 mgr.gibba032.whopeo (mgr.24625) 198 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.426609+0000 mgr.gibba032.whopeo (mgr.24625) 199 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.426651+0000 mgr.gibba032.whopeo (mgr.24625) 200 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.428050+0000 mgr.gibba032.whopeo (mgr.24625) 201 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.428981+0000 mgr.gibba032.whopeo (mgr.24625) 202 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.429019+0000 mgr.gibba032.whopeo (mgr.24625) 203 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: audit 2021-03-10T13:56:22.429594+0000 mgr.gibba032.whopeo (mgr.24625) 204 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cephadm 2021-03-10T13:56:22.430546+0000 mgr.gibba032.whopeo (mgr.24625) 205 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2021-03-10T13:56:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:24 gibba032 bash[24004]: cluster 2021-03-10T13:56:22.539006+0000 mgr.gibba032.whopeo (mgr.24625) 206 : cluster [DBG] pgmap v94: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.420646+0000 mgr.gibba032.whopeo (mgr.24625) 195 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.423462+0000 mgr.gibba032.whopeo (mgr.24625) 196 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.425370+0000 mgr.gibba032.whopeo (mgr.24625) 197 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.425408+0000 mgr.gibba032.whopeo (mgr.24625) 198 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.426609+0000 mgr.gibba032.whopeo (mgr.24625) 199 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.426651+0000 mgr.gibba032.whopeo (mgr.24625) 200 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.428050+0000 mgr.gibba032.whopeo (mgr.24625) 201 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.428981+0000 mgr.gibba032.whopeo (mgr.24625) 202 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.429019+0000 mgr.gibba032.whopeo (mgr.24625) 203 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: audit 2021-03-10T13:56:22.429594+0000 mgr.gibba032.whopeo (mgr.24625) 204 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cephadm 2021-03-10T13:56:22.430546+0000 mgr.gibba032.whopeo (mgr.24625) 205 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2021-03-10T13:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[30791]: cluster 2021-03-10T13:56:22.539006+0000 mgr.gibba032.whopeo (mgr.24625) 206 : cluster [DBG] pgmap v94: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.420646+0000 mgr.gibba032.whopeo (mgr.24625) 195 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.423462+0000 mgr.gibba032.whopeo (mgr.24625) 196 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.425370+0000 mgr.gibba032.whopeo (mgr.24625) 197 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.425408+0000 mgr.gibba032.whopeo (mgr.24625) 198 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.426609+0000 mgr.gibba032.whopeo (mgr.24625) 199 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.426651+0000 mgr.gibba032.whopeo (mgr.24625) 200 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.428050+0000 mgr.gibba032.whopeo (mgr.24625) 201 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.428981+0000 mgr.gibba032.whopeo (mgr.24625) 202 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.429019+0000 mgr.gibba032.whopeo (mgr.24625) 203 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:24.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: audit 2021-03-10T13:56:22.429594+0000 mgr.gibba032.whopeo (mgr.24625) 204 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:24.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cephadm 2021-03-10T13:56:22.430546+0000 mgr.gibba032.whopeo (mgr.24625) 205 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2021-03-10T13:56:24.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:24 gibba011 bash[31774]: cluster 2021-03-10T13:56:22.539006+0000 mgr.gibba032.whopeo (mgr.24625) 206 : cluster [DBG] pgmap v94: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:25.830 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:25 gibba011 bash[37275]: debug 2021-03-10T13:56:25.408+0000 7fe5cf659080 -1 bluestore(/var/lib/ceph/osd/ceph-3) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:56:25.830 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:25 gibba011 bash[37275]: debug 2021-03-10T13:56:25.412+0000 7fe5bb920700 -1 bluestore(/var/lib/ceph/osd/ceph-3) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:56:25.830 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:25 gibba011 bash[37275]: debug 2021-03-10T13:56:25.536+0000 7fe5cf659080 -1 osd.3 76 log_to_monitors {default=true} 2021-03-10T13:56:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:26 gibba032 bash[24004]: cluster 2021-03-10T13:56:24.540531+0000 mgr.gibba032.whopeo (mgr.24625) 207 : cluster [DBG] pgmap v95: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:26 gibba032 bash[24004]: audit 2021-03-10T13:56:25.543788+0000 mon.c (mon.1) 7 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:26 gibba032 bash[24004]: audit 2021-03-10T13:56:25.544020+0000 mon.a (mon.0) 399 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[30791]: cluster 2021-03-10T13:56:24.540531+0000 mgr.gibba032.whopeo (mgr.24625) 207 : cluster [DBG] pgmap v95: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[30791]: audit 2021-03-10T13:56:25.543788+0000 mon.c (mon.1) 7 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[30791]: audit 2021-03-10T13:56:25.544020+0000 mon.a (mon.0) 399 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:26.580 INFO:journalctl@ceph.osd.3.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[37275]: debug 2021-03-10T13:56:26.224+0000 7fe5c1b3b700 -1 osd.3 76 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:56:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[31774]: cluster 2021-03-10T13:56:24.540531+0000 mgr.gibba032.whopeo (mgr.24625) 207 : cluster [DBG] pgmap v95: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 23 MiB used, 667 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[31774]: audit 2021-03-10T13:56:25.543788+0000 mon.c (mon.1) 7 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:26 gibba011 bash[31774]: audit 2021-03-10T13:56:25.544020+0000 mon.a (mon.0) 399 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-03-10T13:56:27.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:27 gibba032 bash[24004]: audit 2021-03-10T13:56:26.218596+0000 mon.a (mon.0) 400 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:56:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:27 gibba032 bash[24004]: cluster 2021-03-10T13:56:26.218640+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e79: 8 total, 7 up, 8 in 2021-03-10T13:56:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:27 gibba032 bash[24004]: audit 2021-03-10T13:56:26.219671+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:27 gibba032 bash[24004]: audit 2021-03-10T13:56:26.222864+0000 mon.a (mon.0) 402 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[30791]: audit 2021-03-10T13:56:26.218596+0000 mon.a (mon.0) 400 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:56:27.600 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[30791]: cluster 2021-03-10T13:56:26.218640+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e79: 8 total, 7 up, 8 in 2021-03-10T13:56:27.601 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[30791]: audit 2021-03-10T13:56:26.219671+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.602 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[30791]: audit 2021-03-10T13:56:26.222864+0000 mon.a (mon.0) 402 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.604 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[31774]: audit 2021-03-10T13:56:26.218596+0000 mon.a (mon.0) 400 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-03-10T13:56:27.604 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[31774]: cluster 2021-03-10T13:56:26.218640+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e79: 8 total, 7 up, 8 in 2021-03-10T13:56:27.606 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[31774]: audit 2021-03-10T13:56:26.219671+0000 mon.c (mon.1) 8 : audit [INF] from='osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.607 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:27 gibba011 bash[31774]: audit 2021-03-10T13:56:26.222864+0000 mon.a (mon.0) 402 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0820, "args": ["host=gibba011", "root=default"]}]: dispatch 2021-03-10T13:56:27.641 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:56:27.868 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:56:27.868 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3m) 5s ago 8m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:56:27.869 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (8m) 2m ago 8m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:56:27.869 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3m) 2m ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:56:27.869 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3m) 2m ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:56:27.869 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2m) 5s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:56:27.869 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2m) 5s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:56:27.870 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2m) 2m ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:56:27.870 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2m) 5s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:56:27.870 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (8m) 5s ago 8m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:56:27.870 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (8m) 2m ago 8m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:56:27.870 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (84s) 5s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:56:27.871 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (58s) 5s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:56:27.871 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (33s) 5s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:56:27.871 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (7s) 5s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:56:27.871 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (10m) 2m ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 b06d31b74111 2021-03-10T13:56:27.871 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (9m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:56:27.872 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (9m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:56:27.872 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (8m) 2m ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:56:27.872 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (8m) 2m ago 8m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:56:27.872 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5s ago 6m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:56:28.125 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:56:28.126 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:56:28.126 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:56:28.126 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:28.126 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:56:28.127 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:56:28.127 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:28.127 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:56:28.127 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 4, 2021-03-10T13:56:28.128 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 4 2021-03-10T13:56:28.128 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:28.128 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:56:28.128 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:56:28.129 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 4, 2021-03-10T13:56:28.129 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 10 2021-03-10T13:56:28.129 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:56:28.129 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:56:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: cluster 2021-03-10T13:56:26.541737+0000 mgr.gibba032.whopeo (mgr.24625) 208 : cluster [DBG] pgmap v97: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: cluster 2021-03-10T13:56:27.223422+0000 mon.a (mon.0) 403 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: cluster 2021-03-10T13:56:27.235615+0000 mon.a (mon.0) 404 : cluster [INF] osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162] boot 2021-03-10T13:56:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: cluster 2021-03-10T13:56:27.235654+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2021-03-10T13:56:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: audit 2021-03-10T13:56:27.235922+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:56:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:28 gibba032 bash[24004]: audit 2021-03-10T13:56:28.125002+0000 mon.a (mon.0) 407 : audit [DBG] from='client.? 172.21.2.111:0/589462827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: cluster 2021-03-10T13:56:26.541737+0000 mgr.gibba032.whopeo (mgr.24625) 208 : cluster [DBG] pgmap v97: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: cluster 2021-03-10T13:56:27.223422+0000 mon.a (mon.0) 403 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: cluster 2021-03-10T13:56:27.235615+0000 mon.a (mon.0) 404 : cluster [INF] osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162] boot 2021-03-10T13:56:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: cluster 2021-03-10T13:56:27.235654+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2021-03-10T13:56:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: audit 2021-03-10T13:56:27.235922+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:56:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[30791]: audit 2021-03-10T13:56:28.125002+0000 mon.a (mon.0) 407 : audit [DBG] from='client.? 172.21.2.111:0/589462827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: cluster 2021-03-10T13:56:26.541737+0000 mgr.gibba032.whopeo (mgr.24625) 208 : cluster [DBG] pgmap v97: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: cluster 2021-03-10T13:56:27.223422+0000 mon.a (mon.0) 403 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: cluster 2021-03-10T13:56:27.235615+0000 mon.a (mon.0) 404 : cluster [INF] osd.3 [v2:172.21.2.111:6824/918718162,v1:172.21.2.111:6825/918718162] boot 2021-03-10T13:56:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: cluster 2021-03-10T13:56:27.235654+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2021-03-10T13:56:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: audit 2021-03-10T13:56:27.235922+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-03-10T13:56:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:28 gibba011 bash[31774]: audit 2021-03-10T13:56:28.125002+0000 mon.a (mon.0) 407 : audit [DBG] from='client.? 172.21.2.111:0/589462827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:29.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:29 gibba032 bash[24004]: audit 2021-03-10T13:56:27.630988+0000 mgr.gibba032.whopeo (mgr.24625) 209 : audit [DBG] from='client.54199 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:29 gibba032 bash[24004]: audit 2021-03-10T13:56:27.856004+0000 mgr.gibba032.whopeo (mgr.24625) 210 : audit [DBG] from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:29 gibba032 bash[24004]: cluster 2021-03-10T13:56:28.244254+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2021-03-10T13:56:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[30791]: audit 2021-03-10T13:56:27.630988+0000 mgr.gibba032.whopeo (mgr.24625) 209 : audit [DBG] from='client.54199 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[30791]: audit 2021-03-10T13:56:27.856004+0000 mgr.gibba032.whopeo (mgr.24625) 210 : audit [DBG] from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[30791]: cluster 2021-03-10T13:56:28.244254+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2021-03-10T13:56:29.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[31774]: audit 2021-03-10T13:56:27.630988+0000 mgr.gibba032.whopeo (mgr.24625) 209 : audit [DBG] from='client.54199 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[31774]: audit 2021-03-10T13:56:27.856004+0000 mgr.gibba032.whopeo (mgr.24625) 210 : audit [DBG] from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:56:29.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:29 gibba011 bash[31774]: cluster 2021-03-10T13:56:28.244254+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2021-03-10T13:56:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:30 gibba011 bash[30791]: cluster 2021-03-10T13:56:28.542525+0000 mgr.gibba032.whopeo (mgr.24625) 211 : cluster [DBG] pgmap v100: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:30 gibba011 bash[30791]: cluster 2021-03-10T13:56:29.877673+0000 mon.a (mon.0) 409 : cluster [WRN] Health check update: Degraded data redundancy: 22/192 objects degraded (11.458%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:30 gibba011 bash[31774]: cluster 2021-03-10T13:56:28.542525+0000 mgr.gibba032.whopeo (mgr.24625) 211 : cluster [DBG] pgmap v100: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:30 gibba011 bash[31774]: cluster 2021-03-10T13:56:29.877673+0000 mon.a (mon.0) 409 : cluster [WRN] Health check update: Degraded data redundancy: 22/192 objects degraded (11.458%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:30 gibba032 bash[24004]: cluster 2021-03-10T13:56:28.542525+0000 mgr.gibba032.whopeo (mgr.24625) 211 : cluster [DBG] pgmap v100: 129 pgs: 18 active+undersized+degraded, 32 active+undersized, 79 active+clean; 7.0 KiB data, 27 MiB used, 668 GiB / 672 GiB avail; 22/192 objects degraded (11.458%) 2021-03-10T13:56:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:30 gibba032 bash[24004]: cluster 2021-03-10T13:56:29.877673+0000 mon.a (mon.0) 409 : cluster [WRN] Health check update: Degraded data redundancy: 22/192 objects degraded (11.458%), 18 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:32 gibba011 bash[30791]: cluster 2021-03-10T13:56:30.544231+0000 mgr.gibba032.whopeo (mgr.24625) 212 : cluster [DBG] pgmap v101: 129 pgs: 11 active+undersized+degraded, 22 active+undersized, 96 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 14/192 objects degraded (7.292%) 2021-03-10T13:56:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:32 gibba011 bash[31774]: cluster 2021-03-10T13:56:30.544231+0000 mgr.gibba032.whopeo (mgr.24625) 212 : cluster [DBG] pgmap v101: 129 pgs: 11 active+undersized+degraded, 22 active+undersized, 96 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 14/192 objects degraded (7.292%) 2021-03-10T13:56:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:32 gibba032 bash[24004]: cluster 2021-03-10T13:56:30.544231+0000 mgr.gibba032.whopeo (mgr.24625) 212 : cluster [DBG] pgmap v101: 129 pgs: 11 active+undersized+degraded, 22 active+undersized, 96 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 14/192 objects degraded (7.292%) 2021-03-10T13:56:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:33 gibba032 bash[24004]: cluster 2021-03-10T13:56:33.273354+0000 mon.a (mon.0) 410 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/192 objects degraded (7.292%), 11 pgs degraded) 2021-03-10T13:56:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:33 gibba011 bash[30791]: cluster 2021-03-10T13:56:33.273354+0000 mon.a (mon.0) 410 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/192 objects degraded (7.292%), 11 pgs degraded) 2021-03-10T13:56:33.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:33 gibba011 bash[31774]: cluster 2021-03-10T13:56:33.273354+0000 mon.a (mon.0) 410 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/192 objects degraded (7.292%), 11 pgs degraded) 2021-03-10T13:56:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:34 gibba032 bash[24004]: cluster 2021-03-10T13:56:32.545905+0000 mgr.gibba032.whopeo (mgr.24625) 213 : cluster [DBG] pgmap v102: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:34 gibba011 bash[30791]: cluster 2021-03-10T13:56:32.545905+0000 mgr.gibba032.whopeo (mgr.24625) 213 : cluster [DBG] pgmap v102: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:34 gibba011 bash[31774]: cluster 2021-03-10T13:56:32.545905+0000 mgr.gibba032.whopeo (mgr.24625) 213 : cluster [DBG] pgmap v102: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:36 gibba032 bash[24004]: cluster 2021-03-10T13:56:34.546686+0000 mgr.gibba032.whopeo (mgr.24625) 214 : cluster [DBG] pgmap v103: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:36 gibba032 bash[24004]: audit 2021-03-10T13:56:36.006437+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:36 gibba011 bash[30791]: cluster 2021-03-10T13:56:34.546686+0000 mgr.gibba032.whopeo (mgr.24625) 214 : cluster [DBG] pgmap v103: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:36 gibba011 bash[30791]: audit 2021-03-10T13:56:36.006437+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:36.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:36 gibba011 bash[31774]: cluster 2021-03-10T13:56:34.546686+0000 mgr.gibba032.whopeo (mgr.24625) 214 : cluster [DBG] pgmap v103: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:36 gibba011 bash[31774]: audit 2021-03-10T13:56:36.006437+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:38.603 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:38 gibba032 bash[24004]: cluster 2021-03-10T13:56:36.548328+0000 mgr.gibba032.whopeo (mgr.24625) 215 : cluster [DBG] pgmap v104: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:38.603 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:38 gibba032 bash[24004]: audit 2021-03-10T13:56:37.447079+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:38 gibba011 bash[30791]: cluster 2021-03-10T13:56:36.548328+0000 mgr.gibba032.whopeo (mgr.24625) 215 : cluster [DBG] pgmap v104: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:38 gibba011 bash[30791]: audit 2021-03-10T13:56:37.447079+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:38 gibba011 bash[31774]: cluster 2021-03-10T13:56:36.548328+0000 mgr.gibba032.whopeo (mgr.24625) 215 : cluster [DBG] pgmap v104: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:38 gibba011 bash[31774]: audit 2021-03-10T13:56:37.447079+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: audit 2021-03-10T13:56:37.447868+0000 mgr.gibba032.whopeo (mgr.24625) 216 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: cephadm 2021-03-10T13:56:37.453026+0000 mgr.gibba032.whopeo (mgr.24625) 217 : cephadm [INF] Upgrade: osd.4 is safe to restart 2021-03-10T13:56:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: cluster 2021-03-10T13:56:38.548844+0000 mgr.gibba032.whopeo (mgr.24625) 218 : cluster [DBG] pgmap v105: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: cephadm 2021-03-10T13:56:39.238808+0000 mgr.gibba032.whopeo (mgr.24625) 219 : cephadm [INF] Upgrade: Updating osd.4 2021-03-10T13:56:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: audit 2021-03-10T13:56:39.249310+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: cephadm 2021-03-10T13:56:39.250789+0000 mgr.gibba032.whopeo (mgr.24625) 220 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:56:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:39 gibba032 bash[24004]: audit 2021-03-10T13:56:39.251392+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:56:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: audit 2021-03-10T13:56:37.447868+0000 mgr.gibba032.whopeo (mgr.24625) 216 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: cephadm 2021-03-10T13:56:37.453026+0000 mgr.gibba032.whopeo (mgr.24625) 217 : cephadm [INF] Upgrade: osd.4 is safe to restart 2021-03-10T13:56:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: cluster 2021-03-10T13:56:38.548844+0000 mgr.gibba032.whopeo (mgr.24625) 218 : cluster [DBG] pgmap v105: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: cephadm 2021-03-10T13:56:39.238808+0000 mgr.gibba032.whopeo (mgr.24625) 219 : cephadm [INF] Upgrade: Updating osd.4 2021-03-10T13:56:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: audit 2021-03-10T13:56:39.249310+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: cephadm 2021-03-10T13:56:39.250789+0000 mgr.gibba032.whopeo (mgr.24625) 220 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:56:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[30791]: audit 2021-03-10T13:56:39.251392+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:56:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: audit 2021-03-10T13:56:37.447868+0000 mgr.gibba032.whopeo (mgr.24625) 216 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-03-10T13:56:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: cephadm 2021-03-10T13:56:37.453026+0000 mgr.gibba032.whopeo (mgr.24625) 217 : cephadm [INF] Upgrade: osd.4 is safe to restart 2021-03-10T13:56:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: cluster 2021-03-10T13:56:38.548844+0000 mgr.gibba032.whopeo (mgr.24625) 218 : cluster [DBG] pgmap v105: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: cephadm 2021-03-10T13:56:39.238808+0000 mgr.gibba032.whopeo (mgr.24625) 219 : cephadm [INF] Upgrade: Updating osd.4 2021-03-10T13:56:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: audit 2021-03-10T13:56:39.249310+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:39.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: cephadm 2021-03-10T13:56:39.250789+0000 mgr.gibba032.whopeo (mgr.24625) 220 : cephadm [INF] Deploying daemon osd.4 on gibba032 2021-03-10T13:56:39.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:39 gibba011 bash[31774]: audit 2021-03-10T13:56:39.251392+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.4","key": "container_image"}]: dispatch 2021-03-10T13:56:41.257 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:41 gibba032 systemd[1]: Stopping Ceph osd.4 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:56:41.257 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[12211]: debug 2021-03-10T13:56:41.182+0000 7fa6b7dc6700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:56:41.257 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[12211]: debug 2021-03-10T13:56:41.182+0000 7fa6b7dc6700 -1 osd.4 81 *** Got signal Terminated *** 2021-03-10T13:56:41.257 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[12211]: debug 2021-03-10T13:56:41.182+0000 7fa6b7dc6700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:56:42.256 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:41 gibba032 docker[25399]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.4 2021-03-10T13:56:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:40.550166+0000 mgr.gibba032.whopeo (mgr.24625) 221 : cluster [DBG] pgmap v106: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.504307+0000 mon.a (mon.0) 415 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.504401+0000 mon.a (mon.0) 416 : cluster [INF] osd.4 failed (root=default,host=gibba032) (connection refused reported by osd.6) 2021-03-10T13:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.504938+0000 mon.a (mon.0) 417 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.505114+0000 mon.a (mon.0) 418 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.505525+0000 mon.a (mon.0) 419 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.505704+0000 mon.a (mon.0) 420 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.506130+0000 mon.a (mon.0) 421 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.508414+0000 mon.a (mon.0) 422 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.508609+0000 mon.a (mon.0) 423 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.509335+0000 mon.a (mon.0) 424 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.509548+0000 mon.a (mon.0) 425 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.509972+0000 mon.a (mon.0) 426 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.510281+0000 mon.a (mon.0) 427 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.510809+0000 mon.a (mon.0) 428 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:41 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.511105+0000 mon.a (mon.0) 429 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:40.550166+0000 mgr.gibba032.whopeo (mgr.24625) 221 : cluster [DBG] pgmap v106: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.504307+0000 mon.a (mon.0) 415 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.504401+0000 mon.a (mon.0) 416 : cluster [INF] osd.4 failed (root=default,host=gibba032) (connection refused reported by osd.6) 2021-03-10T13:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.504938+0000 mon.a (mon.0) 417 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.505114+0000 mon.a (mon.0) 418 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.505525+0000 mon.a (mon.0) 419 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.505704+0000 mon.a (mon.0) 420 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.506130+0000 mon.a (mon.0) 421 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.508414+0000 mon.a (mon.0) 422 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.508609+0000 mon.a (mon.0) 423 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.509335+0000 mon.a (mon.0) 424 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.509548+0000 mon.a (mon.0) 425 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.509972+0000 mon.a (mon.0) 426 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.510281+0000 mon.a (mon.0) 427 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.510809+0000 mon.a (mon.0) 428 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.511105+0000 mon.a (mon.0) 429 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:40.550166+0000 mgr.gibba032.whopeo (mgr.24625) 221 : cluster [DBG] pgmap v106: 129 pgs: 20 active+clean+wait, 109 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.504307+0000 mon.a (mon.0) 415 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.504401+0000 mon.a (mon.0) 416 : cluster [INF] osd.4 failed (root=default,host=gibba032) (connection refused reported by osd.6) 2021-03-10T13:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.504938+0000 mon.a (mon.0) 417 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.505114+0000 mon.a (mon.0) 418 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.505525+0000 mon.a (mon.0) 419 : cluster [DBG] osd.4 reported immediately failed by osd.6 2021-03-10T13:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.505704+0000 mon.a (mon.0) 420 : cluster [DBG] osd.4 reported immediately failed by osd.5 2021-03-10T13:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.506130+0000 mon.a (mon.0) 421 : cluster [DBG] osd.4 reported immediately failed by osd.7 2021-03-10T13:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.508414+0000 mon.a (mon.0) 422 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.508609+0000 mon.a (mon.0) 423 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.509335+0000 mon.a (mon.0) 424 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.509548+0000 mon.a (mon.0) 425 : cluster [DBG] osd.4 reported immediately failed by osd.1 2021-03-10T13:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.509972+0000 mon.a (mon.0) 426 : cluster [DBG] osd.4 reported immediately failed by osd.2 2021-03-10T13:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.510281+0000 mon.a (mon.0) 427 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.510809+0000 mon.a (mon.0) 428 : cluster [DBG] osd.4 reported immediately failed by osd.0 2021-03-10T13:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:41 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.511105+0000 mon.a (mon.0) 429 : cluster [DBG] osd.4 reported immediately failed by osd.3 2021-03-10T13:56:43.164 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:42 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.893231+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:43.165 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:42 gibba032 bash[24004]: cluster 2021-03-10T13:56:41.904097+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e82: 8 total, 7 up, 8 in 2021-03-10T13:56:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:42 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.893231+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:42 gibba011 bash[30791]: cluster 2021-03-10T13:56:41.904097+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e82: 8 total, 7 up, 8 in 2021-03-10T13:56:43.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:42 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.893231+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:56:43.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:42 gibba011 bash[31774]: cluster 2021-03-10T13:56:41.904097+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e82: 8 total, 7 up, 8 in 2021-03-10T13:56:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:43 gibba032 bash[24004]: cluster 2021-03-10T13:56:42.551280+0000 mgr.gibba032.whopeo (mgr.24625) 222 : cluster [DBG] pgmap v108: 129 pgs: 18 stale+active+clean, 20 active+clean+wait, 91 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:43 gibba032 bash[24004]: cluster 2021-03-10T13:56:42.908246+0000 mon.a (mon.0) 432 : cluster [DBG] osdmap e83: 8 total, 7 up, 8 in 2021-03-10T13:56:44.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:43 gibba032 bash[24004]: audit 2021-03-10T13:56:43.862966+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:44.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:43 gibba032 bash[24004]: audit 2021-03-10T13:56:43.864424+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:44.007 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:43 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.4.service: Succeeded. 2021-03-10T13:56:44.007 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:43 gibba032 systemd[1]: Stopped Ceph osd.4 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:56:44.008 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:43 gibba032 systemd[1]: Started Ceph osd.4 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:56:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[30791]: cluster 2021-03-10T13:56:42.551280+0000 mgr.gibba032.whopeo (mgr.24625) 222 : cluster [DBG] pgmap v108: 129 pgs: 18 stale+active+clean, 20 active+clean+wait, 91 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[30791]: cluster 2021-03-10T13:56:42.908246+0000 mon.a (mon.0) 432 : cluster [DBG] osdmap e83: 8 total, 7 up, 8 in 2021-03-10T13:56:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[30791]: audit 2021-03-10T13:56:43.862966+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[30791]: audit 2021-03-10T13:56:43.864424+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[31774]: cluster 2021-03-10T13:56:42.551280+0000 mgr.gibba032.whopeo (mgr.24625) 222 : cluster [DBG] pgmap v108: 129 pgs: 18 stale+active+clean, 20 active+clean+wait, 91 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail 2021-03-10T13:56:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[31774]: cluster 2021-03-10T13:56:42.908246+0000 mon.a (mon.0) 432 : cluster [DBG] osdmap e83: 8 total, 7 up, 8 in 2021-03-10T13:56:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[31774]: audit 2021-03-10T13:56:43.862966+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:43 gibba011 bash[31774]: audit 2021-03-10T13:56:43.864424+0000 mon.a (mon.0) 434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:56:45.506 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-03-10T13:56:45.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2021-03-10T13:56:45.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2021-03-10T13:56:45.507 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2021-03-10T13:56:45.508 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-03-10T13:56:45.508 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-03-10T13:56:45.508 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[25578]: --> ceph-volume lvm activate successful for osd ID: 4 2021-03-10T13:56:46.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[24004]: cluster 2021-03-10T13:56:44.552595+0000 mgr.gibba032.whopeo (mgr.24625) 223 : cluster [DBG] pgmap v110: 129 pgs: 16 stale+active+clean, 4 active+undersized+degraded, 20 active+clean+wait, 5 active+undersized, 84 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:56:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:45 gibba032 bash[24004]: cluster 2021-03-10T13:56:44.911822+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 7/192 objects degraded (3.646%), 4 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:45 gibba011 bash[30791]: cluster 2021-03-10T13:56:44.552595+0000 mgr.gibba032.whopeo (mgr.24625) 223 : cluster [DBG] pgmap v110: 129 pgs: 16 stale+active+clean, 4 active+undersized+degraded, 20 active+clean+wait, 5 active+undersized, 84 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:56:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:45 gibba011 bash[30791]: cluster 2021-03-10T13:56:44.911822+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 7/192 objects degraded (3.646%), 4 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:45 gibba011 bash[31774]: cluster 2021-03-10T13:56:44.552595+0000 mgr.gibba032.whopeo (mgr.24625) 223 : cluster [DBG] pgmap v110: 129 pgs: 16 stale+active+clean, 4 active+undersized+degraded, 20 active+clean+wait, 5 active+undersized, 84 active+clean; 7.0 KiB data, 28 MiB used, 668 GiB / 672 GiB avail; 7/192 objects degraded (3.646%) 2021-03-10T13:56:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:45 gibba011 bash[31774]: cluster 2021-03-10T13:56:44.911822+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 7/192 objects degraded (3.646%), 4 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:46.756 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:46 gibba032 bash[25706]: debug 2021-03-10T13:56:46.566+0000 7f3f8741b700 1 -- 172.21.2.132:0/3626396406 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566df4664e0 con 0x5566df4e2c00 2021-03-10T13:56:47.506 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[25706]: debug 2021-03-10T13:56:47.142+0000 7f3f8eeb0080 -1 Falling back to public interface 2021-03-10T13:56:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: cluster 2021-03-10T13:56:46.553165+0000 mgr.gibba032.whopeo (mgr.24625) 224 : cluster [DBG] pgmap v111: 129 pgs: 7 stale+active+clean, 12 active+undersized+degraded, 17 active+undersized, 93 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 16/192 objects degraded (8.333%) 2021-03-10T13:56:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.788874+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.792781+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.798477+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.800560+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.802277+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.803825+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:47 gibba032 bash[24004]: audit 2021-03-10T13:56:47.807546+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: cluster 2021-03-10T13:56:46.553165+0000 mgr.gibba032.whopeo (mgr.24625) 224 : cluster [DBG] pgmap v111: 129 pgs: 7 stale+active+clean, 12 active+undersized+degraded, 17 active+undersized, 93 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 16/192 objects degraded (8.333%) 2021-03-10T13:56:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.788874+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.792781+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.798477+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.800560+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.802277+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.803825+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[30791]: audit 2021-03-10T13:56:47.807546+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: cluster 2021-03-10T13:56:46.553165+0000 mgr.gibba032.whopeo (mgr.24625) 224 : cluster [DBG] pgmap v111: 129 pgs: 7 stale+active+clean, 12 active+undersized+degraded, 17 active+undersized, 93 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 16/192 objects degraded (8.333%) 2021-03-10T13:56:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.788874+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.792781+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:56:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.798477+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:56:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.800560+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.802277+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.803825+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:47 gibba011 bash[31774]: audit 2021-03-10T13:56:47.807546+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.797346+0000 mgr.gibba032.whopeo (mgr.24625) 225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.799844+0000 mgr.gibba032.whopeo (mgr.24625) 226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.801580+0000 mgr.gibba032.whopeo (mgr.24625) 227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.801652+0000 mgr.gibba032.whopeo (mgr.24625) 228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.803224+0000 mgr.gibba032.whopeo (mgr.24625) 229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.803303+0000 mgr.gibba032.whopeo (mgr.24625) 230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.804926+0000 mgr.gibba032.whopeo (mgr.24625) 231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.806370+0000 mgr.gibba032.whopeo (mgr.24625) 232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.806573+0000 mgr.gibba032.whopeo (mgr.24625) 233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: audit 2021-03-10T13:56:47.807778+0000 mgr.gibba032.whopeo (mgr.24625) 234 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:48 gibba032 bash[24004]: cephadm 2021-03-10T13:56:47.809552+0000 mgr.gibba032.whopeo (mgr.24625) 235 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2021-03-10T13:56:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.797346+0000 mgr.gibba032.whopeo (mgr.24625) 225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.799844+0000 mgr.gibba032.whopeo (mgr.24625) 226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.801580+0000 mgr.gibba032.whopeo (mgr.24625) 227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.801652+0000 mgr.gibba032.whopeo (mgr.24625) 228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.803224+0000 mgr.gibba032.whopeo (mgr.24625) 229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.803303+0000 mgr.gibba032.whopeo (mgr.24625) 230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.804926+0000 mgr.gibba032.whopeo (mgr.24625) 231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.806370+0000 mgr.gibba032.whopeo (mgr.24625) 232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.806573+0000 mgr.gibba032.whopeo (mgr.24625) 233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: audit 2021-03-10T13:56:47.807778+0000 mgr.gibba032.whopeo (mgr.24625) 234 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[30791]: cephadm 2021-03-10T13:56:47.809552+0000 mgr.gibba032.whopeo (mgr.24625) 235 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2021-03-10T13:56:49.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.797346+0000 mgr.gibba032.whopeo (mgr.24625) 225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:56:49.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.799844+0000 mgr.gibba032.whopeo (mgr.24625) 226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:56:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.801580+0000 mgr.gibba032.whopeo (mgr.24625) 227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:56:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.801652+0000 mgr.gibba032.whopeo (mgr.24625) 228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:56:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.803224+0000 mgr.gibba032.whopeo (mgr.24625) 229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:56:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.803303+0000 mgr.gibba032.whopeo (mgr.24625) 230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:56:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.804926+0000 mgr.gibba032.whopeo (mgr.24625) 231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:56:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.806370+0000 mgr.gibba032.whopeo (mgr.24625) 232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:56:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.806573+0000 mgr.gibba032.whopeo (mgr.24625) 233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:56:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: audit 2021-03-10T13:56:47.807778+0000 mgr.gibba032.whopeo (mgr.24625) 234 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:56:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:48 gibba011 bash[31774]: cephadm 2021-03-10T13:56:47.809552+0000 mgr.gibba032.whopeo (mgr.24625) 235 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2021-03-10T13:56:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:49 gibba032 bash[24004]: cluster 2021-03-10T13:56:48.554987+0000 mgr.gibba032.whopeo (mgr.24625) 236 : cluster [DBG] pgmap v112: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:49 gibba011 bash[30791]: cluster 2021-03-10T13:56:48.554987+0000 mgr.gibba032.whopeo (mgr.24625) 236 : cluster [DBG] pgmap v112: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:50.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:49 gibba011 bash[31774]: cluster 2021-03-10T13:56:48.554987+0000 mgr.gibba032.whopeo (mgr.24625) 236 : cluster [DBG] pgmap v112: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:50.756 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[25706]: debug 2021-03-10T13:56:50.398+0000 7f3f8eeb0080 -1 bluestore(/var/lib/ceph/osd/ceph-4) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:56:50.756 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[25706]: debug 2021-03-10T13:56:50.398+0000 7f3f7b978700 -1 bluestore(/var/lib/ceph/osd/ceph-4) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:56:50.757 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[25706]: debug 2021-03-10T13:56:50.522+0000 7f3f8eeb0080 -1 osd.4 81 log_to_monitors {default=true} 2021-03-10T13:56:51.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[24004]: audit 2021-03-10T13:56:50.528027+0000 mon.b (mon.2) 3 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:51.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[24004]: audit 2021-03-10T13:56:50.529117+0000 mon.a (mon.0) 443 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:51.257 INFO:journalctl@ceph.osd.4.gibba032.stdout:Mar 10 13:56:50 gibba032 bash[25706]: debug 2021-03-10T13:56:50.938+0000 7f3f81392700 -1 osd.4 81 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:56:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:50 gibba011 bash[30791]: audit 2021-03-10T13:56:50.528027+0000 mon.b (mon.2) 3 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:50 gibba011 bash[30791]: audit 2021-03-10T13:56:50.529117+0000 mon.a (mon.0) 443 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:50 gibba011 bash[31774]: audit 2021-03-10T13:56:50.528027+0000 mon.b (mon.2) 3 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:51.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:50 gibba011 bash[31774]: audit 2021-03-10T13:56:50.529117+0000 mon.a (mon.0) 443 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-03-10T13:56:52.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:51 gibba032 bash[24004]: cluster 2021-03-10T13:56:50.556417+0000 mgr.gibba032.whopeo (mgr.24625) 237 : cluster [DBG] pgmap v113: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:51 gibba032 bash[24004]: audit 2021-03-10T13:56:50.931761+0000 mon.a (mon.0) 444 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:56:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:51 gibba032 bash[24004]: cluster 2021-03-10T13:56:50.931868+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e84: 8 total, 7 up, 8 in 2021-03-10T13:56:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:51 gibba032 bash[24004]: audit 2021-03-10T13:56:50.932785+0000 mon.b (mon.2) 4 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:51 gibba032 bash[24004]: audit 2021-03-10T13:56:50.933151+0000 mon.a (mon.0) 446 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[30791]: cluster 2021-03-10T13:56:50.556417+0000 mgr.gibba032.whopeo (mgr.24625) 237 : cluster [DBG] pgmap v113: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[30791]: audit 2021-03-10T13:56:50.931761+0000 mon.a (mon.0) 444 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:56:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[30791]: cluster 2021-03-10T13:56:50.931868+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e84: 8 total, 7 up, 8 in 2021-03-10T13:56:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[30791]: audit 2021-03-10T13:56:50.932785+0000 mon.b (mon.2) 4 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[30791]: audit 2021-03-10T13:56:50.933151+0000 mon.a (mon.0) 446 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[31774]: cluster 2021-03-10T13:56:50.556417+0000 mgr.gibba032.whopeo (mgr.24625) 237 : cluster [DBG] pgmap v113: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 29 MiB used, 668 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[31774]: audit 2021-03-10T13:56:50.931761+0000 mon.a (mon.0) 444 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-03-10T13:56:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[31774]: cluster 2021-03-10T13:56:50.931868+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e84: 8 total, 7 up, 8 in 2021-03-10T13:56:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[31774]: audit 2021-03-10T13:56:50.932785+0000 mon.b (mon.2) 4 : audit [INF] from='osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:51 gibba011 bash[31774]: audit 2021-03-10T13:56:50.933151+0000 mon.a (mon.0) 446 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:56:53.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:52 gibba032 bash[24004]: cluster 2021-03-10T13:56:51.930262+0000 mon.a (mon.0) 447 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:52 gibba032 bash[24004]: cluster 2021-03-10T13:56:51.935493+0000 mon.a (mon.0) 448 : cluster [INF] osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622] boot 2021-03-10T13:56:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:52 gibba032 bash[24004]: cluster 2021-03-10T13:56:51.935565+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2021-03-10T13:56:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:52 gibba032 bash[24004]: audit 2021-03-10T13:56:51.936102+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:56:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[30791]: cluster 2021-03-10T13:56:51.930262+0000 mon.a (mon.0) 447 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[30791]: cluster 2021-03-10T13:56:51.935493+0000 mon.a (mon.0) 448 : cluster [INF] osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622] boot 2021-03-10T13:56:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[30791]: cluster 2021-03-10T13:56:51.935565+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2021-03-10T13:56:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[30791]: audit 2021-03-10T13:56:51.936102+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:56:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[31774]: cluster 2021-03-10T13:56:51.930262+0000 mon.a (mon.0) 447 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:56:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[31774]: cluster 2021-03-10T13:56:51.935493+0000 mon.a (mon.0) 448 : cluster [INF] osd.4 [v2:172.21.2.132:6800/3736877622,v1:172.21.2.132:6801/3736877622] boot 2021-03-10T13:56:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[31774]: cluster 2021-03-10T13:56:51.935565+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2021-03-10T13:56:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:52 gibba011 bash[31774]: audit 2021-03-10T13:56:51.936102+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-03-10T13:56:54.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:53 gibba032 bash[24004]: cluster 2021-03-10T13:56:52.557066+0000 mgr.gibba032.whopeo (mgr.24625) 238 : cluster [DBG] pgmap v116: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 33 MiB used, 669 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:53 gibba032 bash[24004]: cluster 2021-03-10T13:56:52.944150+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2021-03-10T13:56:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:53 gibba011 bash[30791]: cluster 2021-03-10T13:56:52.557066+0000 mgr.gibba032.whopeo (mgr.24625) 238 : cluster [DBG] pgmap v116: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 33 MiB used, 669 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:53 gibba011 bash[30791]: cluster 2021-03-10T13:56:52.944150+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2021-03-10T13:56:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:53 gibba011 bash[31774]: cluster 2021-03-10T13:56:52.557066+0000 mgr.gibba032.whopeo (mgr.24625) 238 : cluster [DBG] pgmap v116: 129 pgs: 23 active+undersized+degraded, 33 active+undersized, 73 active+clean; 7.0 KiB data, 33 MiB used, 669 GiB / 672 GiB avail; 31/192 objects degraded (16.146%) 2021-03-10T13:56:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:53 gibba011 bash[31774]: cluster 2021-03-10T13:56:52.944150+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2021-03-10T13:56:55.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:54 gibba032 bash[24004]: cluster 2021-03-10T13:56:54.890667+0000 mon.a (mon.0) 452 : cluster [WRN] Health check update: Degraded data redundancy: 31/192 objects degraded (16.146%), 23 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:54 gibba011 bash[30791]: cluster 2021-03-10T13:56:54.890667+0000 mon.a (mon.0) 452 : cluster [WRN] Health check update: Degraded data redundancy: 31/192 objects degraded (16.146%), 23 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:54 gibba011 bash[31774]: cluster 2021-03-10T13:56:54.890667+0000 mon.a (mon.0) 452 : cluster [WRN] Health check update: Degraded data redundancy: 31/192 objects degraded (16.146%), 23 pgs degraded (PG_DEGRADED) 2021-03-10T13:56:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:55 gibba032 bash[24004]: cluster 2021-03-10T13:56:54.558617+0000 mgr.gibba032.whopeo (mgr.24625) 239 : cluster [DBG] pgmap v118: 129 pgs: 21 active+undersized+degraded, 28 active+undersized, 80 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 29/192 objects degraded (15.104%) 2021-03-10T13:56:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:55 gibba011 bash[30791]: cluster 2021-03-10T13:56:54.558617+0000 mgr.gibba032.whopeo (mgr.24625) 239 : cluster [DBG] pgmap v118: 129 pgs: 21 active+undersized+degraded, 28 active+undersized, 80 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 29/192 objects degraded (15.104%) 2021-03-10T13:56:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:55 gibba011 bash[31774]: cluster 2021-03-10T13:56:54.558617+0000 mgr.gibba032.whopeo (mgr.24625) 239 : cluster [DBG] pgmap v118: 129 pgs: 21 active+undersized+degraded, 28 active+undersized, 80 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 29/192 objects degraded (15.104%) 2021-03-10T13:56:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:56 gibba032 bash[24004]: audit 2021-03-10T13:56:55.990656+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:56:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:56 gibba032 bash[24004]: audit 2021-03-10T13:56:55.993360+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:56:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:56 gibba011 bash[30791]: audit 2021-03-10T13:56:55.990656+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:56:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:56 gibba011 bash[30791]: audit 2021-03-10T13:56:55.993360+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:56:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:56 gibba011 bash[31774]: audit 2021-03-10T13:56:55.990656+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:56:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:56 gibba011 bash[31774]: audit 2021-03-10T13:56:55.993360+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:56:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:57 gibba032 bash[24004]: cluster 2021-03-10T13:56:56.560587+0000 mgr.gibba032.whopeo (mgr.24625) 240 : cluster [DBG] pgmap v119: 129 pgs: 18 active+clean+wait, 9 active+undersized+degraded, 11 active+undersized, 91 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:57 gibba011 bash[30791]: cluster 2021-03-10T13:56:56.560587+0000 mgr.gibba032.whopeo (mgr.24625) 240 : cluster [DBG] pgmap v119: 129 pgs: 18 active+clean+wait, 9 active+undersized+degraded, 11 active+undersized, 91 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:57 gibba011 bash[31774]: cluster 2021-03-10T13:56:56.560587+0000 mgr.gibba032.whopeo (mgr.24625) 240 : cluster [DBG] pgmap v119: 129 pgs: 18 active+clean+wait, 9 active+undersized+degraded, 11 active+undersized, 91 active+clean; 7.0 KiB data, 34 MiB used, 669 GiB / 672 GiB avail; 13/192 objects degraded (6.771%) 2021-03-10T13:56:58.410 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:56:58.638 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:56:58.638 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3m) 36s ago 8m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:56:58.639 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (8m) 10s ago 8m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:56:58.639 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3m) 10s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:56:58.639 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (4m) 10s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:56:58.639 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3m) 36s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:56:58.639 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3m) 36s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:56:58.640 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2m) 10s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:56:58.640 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2m) 36s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:56:58.640 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (9m) 36s ago 9m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:56:58.640 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (9m) 10s ago 9m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:56:58.640 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (115s) 36s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:56:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (89s) 36s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:56:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (63s) 36s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:56:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (37s) 36s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:56:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (12s) 10s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:56:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (10m) 10s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 312fae1e3bf7 2021-03-10T13:56:58.642 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (9m) 10s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:56:58.642 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (9m) 10s ago 9m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:56:58.642 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (8m) 10s ago 9m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:56:58.642 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 36s ago 7m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:56:58.886 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:56:58.911 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:56:58.911 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:56:58.912 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:58.912 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:56:58.912 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:56:58.912 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:58.912 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:56:58.913 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3, 2021-03-10T13:56:58.913 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 5 2021-03-10T13:56:58.913 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:56:58.913 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:56:58.913 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:56:58.914 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 3, 2021-03-10T13:56:58.914 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 11 2021-03-10T13:56:58.914 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:56:58.914 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:56:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:58 gibba032 bash[24004]: audit 2021-03-10T13:56:58.886052+0000 mon.a (mon.0) 455 : audit [DBG] from='client.? 172.21.2.111:0/1063637291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:58 gibba011 bash[30791]: audit 2021-03-10T13:56:58.886052+0000 mon.a (mon.0) 455 : audit [DBG] from='client.? 172.21.2.111:0/1063637291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:56:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:58 gibba011 bash[31774]: audit 2021-03-10T13:56:58.886052+0000 mon.a (mon.0) 455 : audit [DBG] from='client.? 172.21.2.111:0/1063637291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:59 gibba032 bash[24004]: audit 2021-03-10T13:56:58.400550+0000 mgr.gibba032.whopeo (mgr.24625) 241 : audit [DBG] from='client.44232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:59 gibba032 bash[24004]: cluster 2021-03-10T13:56:58.562040+0000 mgr.gibba032.whopeo (mgr.24625) 242 : cluster [DBG] pgmap v120: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:59 gibba032 bash[24004]: audit 2021-03-10T13:56:58.625530+0000 mgr.gibba032.whopeo (mgr.24625) 243 : audit [DBG] from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:56:59 gibba032 bash[24004]: cluster 2021-03-10T13:56:58.978610+0000 mon.a (mon.0) 456 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 9 pgs degraded) 2021-03-10T13:57:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[30791]: audit 2021-03-10T13:56:58.400550+0000 mgr.gibba032.whopeo (mgr.24625) 241 : audit [DBG] from='client.44232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[30791]: cluster 2021-03-10T13:56:58.562040+0000 mgr.gibba032.whopeo (mgr.24625) 242 : cluster [DBG] pgmap v120: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[30791]: audit 2021-03-10T13:56:58.625530+0000 mgr.gibba032.whopeo (mgr.24625) 243 : audit [DBG] from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[30791]: cluster 2021-03-10T13:56:58.978610+0000 mon.a (mon.0) 456 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 9 pgs degraded) 2021-03-10T13:57:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[31774]: audit 2021-03-10T13:56:58.400550+0000 mgr.gibba032.whopeo (mgr.24625) 241 : audit [DBG] from='client.44232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[31774]: cluster 2021-03-10T13:56:58.562040+0000 mgr.gibba032.whopeo (mgr.24625) 242 : cluster [DBG] pgmap v120: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[31774]: audit 2021-03-10T13:56:58.625530+0000 mgr.gibba032.whopeo (mgr.24625) 243 : audit [DBG] from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:56:59 gibba011 bash[31774]: cluster 2021-03-10T13:56:58.978610+0000 mon.a (mon.0) 456 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/192 objects degraded (6.771%), 9 pgs degraded) 2021-03-10T13:57:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:02 gibba011 bash[30791]: cluster 2021-03-10T13:57:00.563385+0000 mgr.gibba032.whopeo (mgr.24625) 244 : cluster [DBG] pgmap v121: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:02 gibba011 bash[30791]: audit 2021-03-10T13:57:01.014814+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:02.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:02 gibba011 bash[31774]: cluster 2021-03-10T13:57:00.563385+0000 mgr.gibba032.whopeo (mgr.24625) 244 : cluster [DBG] pgmap v121: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:02.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:02 gibba011 bash[31774]: audit 2021-03-10T13:57:01.014814+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:02 gibba032 bash[24004]: cluster 2021-03-10T13:57:00.563385+0000 mgr.gibba032.whopeo (mgr.24625) 244 : cluster [DBG] pgmap v121: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:02 gibba032 bash[24004]: audit 2021-03-10T13:57:01.014814+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:03 gibba011 bash[30791]: audit 2021-03-10T13:57:02.826489+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:03 gibba011 bash[31774]: audit 2021-03-10T13:57:02.826489+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:03.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:03 gibba032 bash[24004]: audit 2021-03-10T13:57:02.826489+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:04.160 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:04 gibba032 bash[24004]: cluster 2021-03-10T13:57:02.565169+0000 mgr.gibba032.whopeo (mgr.24625) 245 : cluster [DBG] pgmap v122: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:04.160 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:04 gibba032 bash[24004]: audit 2021-03-10T13:57:02.827330+0000 mgr.gibba032.whopeo (mgr.24625) 246 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:04.161 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:04 gibba032 bash[24004]: cephadm 2021-03-10T13:57:02.832355+0000 mgr.gibba032.whopeo (mgr.24625) 247 : cephadm [INF] Upgrade: osd.5 is safe to restart 2021-03-10T13:57:04.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[30791]: cluster 2021-03-10T13:57:02.565169+0000 mgr.gibba032.whopeo (mgr.24625) 245 : cluster [DBG] pgmap v122: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[30791]: audit 2021-03-10T13:57:02.827330+0000 mgr.gibba032.whopeo (mgr.24625) 246 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[30791]: cephadm 2021-03-10T13:57:02.832355+0000 mgr.gibba032.whopeo (mgr.24625) 247 : cephadm [INF] Upgrade: osd.5 is safe to restart 2021-03-10T13:57:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[31774]: cluster 2021-03-10T13:57:02.565169+0000 mgr.gibba032.whopeo (mgr.24625) 245 : cluster [DBG] pgmap v122: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[31774]: audit 2021-03-10T13:57:02.827330+0000 mgr.gibba032.whopeo (mgr.24625) 246 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-03-10T13:57:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:04 gibba011 bash[31774]: cephadm 2021-03-10T13:57:02.832355+0000 mgr.gibba032.whopeo (mgr.24625) 247 : cephadm [INF] Upgrade: osd.5 is safe to restart 2021-03-10T13:57:05.895 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:05 gibba032 bash[24004]: cluster 2021-03-10T13:57:04.566381+0000 mgr.gibba032.whopeo (mgr.24625) 248 : cluster [DBG] pgmap v123: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:05.895 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:05 gibba032 bash[24004]: cephadm 2021-03-10T13:57:04.681055+0000 mgr.gibba032.whopeo (mgr.24625) 249 : cephadm [INF] Upgrade: Updating osd.5 2021-03-10T13:57:05.896 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:05 gibba032 bash[24004]: audit 2021-03-10T13:57:04.691543+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:05.896 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:05 gibba032 bash[24004]: cephadm 2021-03-10T13:57:04.692663+0000 mgr.gibba032.whopeo (mgr.24625) 250 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:57:05.896 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:05 gibba032 bash[24004]: audit 2021-03-10T13:57:04.693099+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:57:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[30791]: cluster 2021-03-10T13:57:04.566381+0000 mgr.gibba032.whopeo (mgr.24625) 248 : cluster [DBG] pgmap v123: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[30791]: cephadm 2021-03-10T13:57:04.681055+0000 mgr.gibba032.whopeo (mgr.24625) 249 : cephadm [INF] Upgrade: Updating osd.5 2021-03-10T13:57:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[30791]: audit 2021-03-10T13:57:04.691543+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[30791]: cephadm 2021-03-10T13:57:04.692663+0000 mgr.gibba032.whopeo (mgr.24625) 250 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:57:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[30791]: audit 2021-03-10T13:57:04.693099+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:57:06.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[31774]: cluster 2021-03-10T13:57:04.566381+0000 mgr.gibba032.whopeo (mgr.24625) 248 : cluster [DBG] pgmap v123: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:06.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[31774]: cephadm 2021-03-10T13:57:04.681055+0000 mgr.gibba032.whopeo (mgr.24625) 249 : cephadm [INF] Upgrade: Updating osd.5 2021-03-10T13:57:06.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[31774]: audit 2021-03-10T13:57:04.691543+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:06.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[31774]: cephadm 2021-03-10T13:57:04.692663+0000 mgr.gibba032.whopeo (mgr.24625) 250 : cephadm [INF] Deploying daemon osd.5 on gibba032 2021-03-10T13:57:06.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:05 gibba011 bash[31774]: audit 2021-03-10T13:57:04.693099+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.5","key": "container_image"}]: dispatch 2021-03-10T13:57:06.756 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:06 gibba032 systemd[1]: Stopping Ceph osd.5 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:57:06.757 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:06 gibba032 bash[13724]: debug 2021-03-10T13:57:06.599+0000 7f4d84b59700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:57:06.757 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:06 gibba032 bash[13724]: debug 2021-03-10T13:57:06.599+0000 7f4d84b59700 -1 osd.5 86 *** Got signal Terminated *** 2021-03-10T13:57:06.757 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:06 gibba032 bash[13724]: debug 2021-03-10T13:57:06.599+0000 7f4d84b59700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:57:07.702 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:07 gibba032 docker[26603]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.5 2021-03-10T13:57:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.567056+0000 mgr.gibba032.whopeo (mgr.24625) 251 : cluster [DBG] pgmap v124: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.944452+0000 mon.a (mon.0) 461 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.944548+0000 mon.a (mon.0) 462 : cluster [INF] osd.5 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.945155+0000 mon.a (mon.0) 463 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.945503+0000 mon.a (mon.0) 464 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.945814+0000 mon.a (mon.0) 465 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.946115+0000 mon.a (mon.0) 466 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.946303+0000 mon.a (mon.0) 467 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.946854+0000 mon.a (mon.0) 468 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.947134+0000 mon.a (mon.0) 469 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.947591+0000 mon.a (mon.0) 470 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.947769+0000 mon.a (mon.0) 471 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.948166+0000 mon.a (mon.0) 472 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.948341+0000 mon.a (mon.0) 473 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.948894+0000 mon.a (mon.0) 474 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:06.949171+0000 mon.a (mon.0) 475 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.341852+0000 mon.a (mon.0) 476 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.342069+0000 mon.a (mon.0) 477 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.342508+0000 mon.a (mon.0) 478 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.342684+0000 mon.a (mon.0) 479 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.346398+0000 mon.a (mon.0) 480 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:07.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.346772+0000 mon.a (mon.0) 481 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:07.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.348345+0000 mon.a (mon.0) 482 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:07.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.348555+0000 mon.a (mon.0) 483 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:07.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.349062+0000 mon.a (mon.0) 484 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:07.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.349251+0000 mon.a (mon.0) 485 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:07.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.349650+0000 mon.a (mon.0) 486 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:07.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.349969+0000 mon.a (mon.0) 487 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:07.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.350395+0000 mon.a (mon.0) 488 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:07.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:07 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.350687+0000 mon.a (mon.0) 489 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.567056+0000 mgr.gibba032.whopeo (mgr.24625) 251 : cluster [DBG] pgmap v124: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.944452+0000 mon.a (mon.0) 461 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.944548+0000 mon.a (mon.0) 462 : cluster [INF] osd.5 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.945155+0000 mon.a (mon.0) 463 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.945503+0000 mon.a (mon.0) 464 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.945814+0000 mon.a (mon.0) 465 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.946115+0000 mon.a (mon.0) 466 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.946303+0000 mon.a (mon.0) 467 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.946854+0000 mon.a (mon.0) 468 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.947134+0000 mon.a (mon.0) 469 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.947591+0000 mon.a (mon.0) 470 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.947769+0000 mon.a (mon.0) 471 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.948166+0000 mon.a (mon.0) 472 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.948341+0000 mon.a (mon.0) 473 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.948894+0000 mon.a (mon.0) 474 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:06.949171+0000 mon.a (mon.0) 475 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.341852+0000 mon.a (mon.0) 476 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.342069+0000 mon.a (mon.0) 477 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.342508+0000 mon.a (mon.0) 478 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.342684+0000 mon.a (mon.0) 479 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.346398+0000 mon.a (mon.0) 480 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.346772+0000 mon.a (mon.0) 481 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.348345+0000 mon.a (mon.0) 482 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.348555+0000 mon.a (mon.0) 483 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.349062+0000 mon.a (mon.0) 484 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.349251+0000 mon.a (mon.0) 485 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.349650+0000 mon.a (mon.0) 486 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.349969+0000 mon.a (mon.0) 487 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.350395+0000 mon.a (mon.0) 488 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.350687+0000 mon.a (mon.0) 489 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.567056+0000 mgr.gibba032.whopeo (mgr.24625) 251 : cluster [DBG] pgmap v124: 129 pgs: 18 active+clean+wait, 111 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.944452+0000 mon.a (mon.0) 461 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.944548+0000 mon.a (mon.0) 462 : cluster [INF] osd.5 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.945155+0000 mon.a (mon.0) 463 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.945503+0000 mon.a (mon.0) 464 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.945814+0000 mon.a (mon.0) 465 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.946115+0000 mon.a (mon.0) 466 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.946303+0000 mon.a (mon.0) 467 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.946854+0000 mon.a (mon.0) 468 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.947134+0000 mon.a (mon.0) 469 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.947591+0000 mon.a (mon.0) 470 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.947769+0000 mon.a (mon.0) 471 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.948166+0000 mon.a (mon.0) 472 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.948341+0000 mon.a (mon.0) 473 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.948894+0000 mon.a (mon.0) 474 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:06.949171+0000 mon.a (mon.0) 475 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.341852+0000 mon.a (mon.0) 476 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.342069+0000 mon.a (mon.0) 477 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.342508+0000 mon.a (mon.0) 478 : cluster [DBG] osd.5 reported immediately failed by osd.6 2021-03-10T13:57:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.342684+0000 mon.a (mon.0) 479 : cluster [DBG] osd.5 reported immediately failed by osd.7 2021-03-10T13:57:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.346398+0000 mon.a (mon.0) 480 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.346772+0000 mon.a (mon.0) 481 : cluster [DBG] osd.5 reported immediately failed by osd.4 2021-03-10T13:57:08.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.348345+0000 mon.a (mon.0) 482 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.348555+0000 mon.a (mon.0) 483 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.349062+0000 mon.a (mon.0) 484 : cluster [DBG] osd.5 reported immediately failed by osd.0 2021-03-10T13:57:08.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.349251+0000 mon.a (mon.0) 485 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.349650+0000 mon.a (mon.0) 486 : cluster [DBG] osd.5 reported immediately failed by osd.2 2021-03-10T13:57:08.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.349969+0000 mon.a (mon.0) 487 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:08.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.350395+0000 mon.a (mon.0) 488 : cluster [DBG] osd.5 reported immediately failed by osd.1 2021-03-10T13:57:08.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:07 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.350687+0000 mon.a (mon.0) 489 : cluster [DBG] osd.5 reported immediately failed by osd.3 2021-03-10T13:57:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:08 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.696712+0000 mon.a (mon.0) 490 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:09.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:08 gibba032 bash[24004]: cluster 2021-03-10T13:57:07.710042+0000 mon.a (mon.0) 491 : cluster [DBG] osdmap e87: 8 total, 7 up, 8 in 2021-03-10T13:57:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:08 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.696712+0000 mon.a (mon.0) 490 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:08 gibba011 bash[30791]: cluster 2021-03-10T13:57:07.710042+0000 mon.a (mon.0) 491 : cluster [DBG] osdmap e87: 8 total, 7 up, 8 in 2021-03-10T13:57:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:08 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.696712+0000 mon.a (mon.0) 490 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:08 gibba011 bash[31774]: cluster 2021-03-10T13:57:07.710042+0000 mon.a (mon.0) 491 : cluster [DBG] osdmap e87: 8 total, 7 up, 8 in 2021-03-10T13:57:09.716 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:09 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.5.service: Succeeded. 2021-03-10T13:57:09.716 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:09 gibba032 systemd[1]: Stopped Ceph osd.5 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:57:09.717 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:09 gibba032 systemd[1]: Started Ceph osd.5 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:57:10.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:09 gibba032 bash[24004]: cluster 2021-03-10T13:57:08.567351+0000 mgr.gibba032.whopeo (mgr.24625) 252 : cluster [DBG] pgmap v126: 129 pgs: 17 stale+active+clean, 18 active+clean+wait, 94 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:09 gibba032 bash[24004]: cluster 2021-03-10T13:57:08.712654+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e88: 8 total, 7 up, 8 in 2021-03-10T13:57:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:09 gibba032 bash[24004]: audit 2021-03-10T13:57:09.526687+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:09 gibba032 bash[24004]: audit 2021-03-10T13:57:09.528062+0000 mon.a (mon.0) 494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[30791]: cluster 2021-03-10T13:57:08.567351+0000 mgr.gibba032.whopeo (mgr.24625) 252 : cluster [DBG] pgmap v126: 129 pgs: 17 stale+active+clean, 18 active+clean+wait, 94 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[30791]: cluster 2021-03-10T13:57:08.712654+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e88: 8 total, 7 up, 8 in 2021-03-10T13:57:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[30791]: audit 2021-03-10T13:57:09.526687+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[30791]: audit 2021-03-10T13:57:09.528062+0000 mon.a (mon.0) 494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[31774]: cluster 2021-03-10T13:57:08.567351+0000 mgr.gibba032.whopeo (mgr.24625) 252 : cluster [DBG] pgmap v126: 129 pgs: 17 stale+active+clean, 18 active+clean+wait, 94 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail 2021-03-10T13:57:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[31774]: cluster 2021-03-10T13:57:08.712654+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e88: 8 total, 7 up, 8 in 2021-03-10T13:57:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[31774]: audit 2021-03-10T13:57:09.526687+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:10.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:09 gibba011 bash[31774]: audit 2021-03-10T13:57:09.528062+0000 mon.a (mon.0) 494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[24004]: cluster 2021-03-10T13:57:10.903789+0000 mon.a (mon.0) 495 : cluster [WRN] Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:11.007 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-03-10T13:57:11.007 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2021-03-10T13:57:11.007 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5/block 2021-03-10T13:57:11.007 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2021-03-10T13:57:11.008 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-03-10T13:57:11.008 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-03-10T13:57:11.008 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:10 gibba032 bash[26778]: --> ceph-volume lvm activate successful for osd ID: 5 2021-03-10T13:57:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:10 gibba011 bash[30791]: cluster 2021-03-10T13:57:10.903789+0000 mon.a (mon.0) 495 : cluster [WRN] Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:10 gibba011 bash[31774]: cluster 2021-03-10T13:57:10.903789+0000 mon.a (mon.0) 495 : cluster [WRN] Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 6 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:12.238 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:11 gibba032 bash[24004]: cluster 2021-03-10T13:57:10.567783+0000 mgr.gibba032.whopeo (mgr.24625) 253 : cluster [DBG] pgmap v128: 129 pgs: 12 stale+active+clean, 6 active+undersized+degraded, 18 active+clean+wait, 13 active+undersized, 80 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:57:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:11 gibba011 bash[30791]: cluster 2021-03-10T13:57:10.567783+0000 mgr.gibba032.whopeo (mgr.24625) 253 : cluster [DBG] pgmap v128: 129 pgs: 12 stale+active+clean, 6 active+undersized+degraded, 18 active+clean+wait, 13 active+undersized, 80 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:57:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:11 gibba011 bash[31774]: cluster 2021-03-10T13:57:10.567783+0000 mgr.gibba032.whopeo (mgr.24625) 253 : cluster [DBG] pgmap v128: 129 pgs: 12 stale+active+clean, 6 active+undersized+degraded, 18 active+clean+wait, 13 active+undersized, 80 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 8/192 objects degraded (4.167%) 2021-03-10T13:57:12.756 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:12 gibba032 bash[26915]: debug 2021-03-10T13:57:12.255+0000 7fe4709e1700 1 -- 172.21.2.132:0/230510661 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5563f5b104e0 con 0x5563f5b8d400 2021-03-10T13:57:13.256 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:12 gibba032 bash[26915]: debug 2021-03-10T13:57:12.815+0000 7fe478476080 -1 Falling back to public interface 2021-03-10T13:57:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: cluster 2021-03-10T13:57:12.568284+0000 mgr.gibba032.whopeo (mgr.24625) 254 : cluster [DBG] pgmap v129: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.461644+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.464893+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.468218+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.469383+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.470357+0000 mon.a (mon.0) 500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.471209+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:13 gibba032 bash[24004]: audit 2021-03-10T13:57:13.473681+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: cluster 2021-03-10T13:57:12.568284+0000 mgr.gibba032.whopeo (mgr.24625) 254 : cluster [DBG] pgmap v129: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.461644+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.464893+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.468218+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.469383+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.470357+0000 mon.a (mon.0) 500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.471209+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[30791]: audit 2021-03-10T13:57:13.473681+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: cluster 2021-03-10T13:57:12.568284+0000 mgr.gibba032.whopeo (mgr.24625) 254 : cluster [DBG] pgmap v129: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.461644+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.464893+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.468218+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.469383+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.470357+0000 mon.a (mon.0) 500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.471209+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:13 gibba011 bash[31774]: audit 2021-03-10T13:57:13.473681+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:15.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.467530+0000 mgr.gibba032.whopeo (mgr.24625) 255 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:15.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.469014+0000 mgr.gibba032.whopeo (mgr.24625) 256 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:15.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.469963+0000 mgr.gibba032.whopeo (mgr.24625) 257 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.470004+0000 mgr.gibba032.whopeo (mgr.24625) 258 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.470890+0000 mgr.gibba032.whopeo (mgr.24625) 259 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.470927+0000 mgr.gibba032.whopeo (mgr.24625) 260 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.471812+0000 mgr.gibba032.whopeo (mgr.24625) 261 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.472866+0000 mgr.gibba032.whopeo (mgr.24625) 262 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.472907+0000 mgr.gibba032.whopeo (mgr.24625) 263 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: audit 2021-03-10T13:57:13.473863+0000 mgr.gibba032.whopeo (mgr.24625) 264 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:14 gibba032 bash[24004]: cephadm 2021-03-10T13:57:13.475549+0000 mgr.gibba032.whopeo (mgr.24625) 265 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2021-03-10T13:57:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.467530+0000 mgr.gibba032.whopeo (mgr.24625) 255 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.469014+0000 mgr.gibba032.whopeo (mgr.24625) 256 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.469963+0000 mgr.gibba032.whopeo (mgr.24625) 257 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.470004+0000 mgr.gibba032.whopeo (mgr.24625) 258 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.470890+0000 mgr.gibba032.whopeo (mgr.24625) 259 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.470927+0000 mgr.gibba032.whopeo (mgr.24625) 260 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.471812+0000 mgr.gibba032.whopeo (mgr.24625) 261 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.472866+0000 mgr.gibba032.whopeo (mgr.24625) 262 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.472907+0000 mgr.gibba032.whopeo (mgr.24625) 263 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: audit 2021-03-10T13:57:13.473863+0000 mgr.gibba032.whopeo (mgr.24625) 264 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[30791]: cephadm 2021-03-10T13:57:13.475549+0000 mgr.gibba032.whopeo (mgr.24625) 265 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2021-03-10T13:57:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.467530+0000 mgr.gibba032.whopeo (mgr.24625) 255 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.469014+0000 mgr.gibba032.whopeo (mgr.24625) 256 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.469963+0000 mgr.gibba032.whopeo (mgr.24625) 257 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.470004+0000 mgr.gibba032.whopeo (mgr.24625) 258 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.470890+0000 mgr.gibba032.whopeo (mgr.24625) 259 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.470927+0000 mgr.gibba032.whopeo (mgr.24625) 260 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:15.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.471812+0000 mgr.gibba032.whopeo (mgr.24625) 261 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:15.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.472866+0000 mgr.gibba032.whopeo (mgr.24625) 262 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:15.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.472907+0000 mgr.gibba032.whopeo (mgr.24625) 263 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:15.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: audit 2021-03-10T13:57:13.473863+0000 mgr.gibba032.whopeo (mgr.24625) 264 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:15.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:14 gibba011 bash[31774]: cephadm 2021-03-10T13:57:13.475549+0000 mgr.gibba032.whopeo (mgr.24625) 265 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2021-03-10T13:57:16.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:15 gibba032 bash[24004]: cluster 2021-03-10T13:57:14.569727+0000 mgr.gibba032.whopeo (mgr.24625) 266 : cluster [DBG] pgmap v130: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:16.257 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:16 gibba032 bash[26915]: debug 2021-03-10T13:57:16.055+0000 7fe478476080 -1 bluestore(/var/lib/ceph/osd/ceph-5) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:57:16.257 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:16 gibba032 bash[26915]: debug 2021-03-10T13:57:16.055+0000 7fe462739700 -1 bluestore(/var/lib/ceph/osd/ceph-5) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:57:16.257 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:16 gibba032 bash[26915]: debug 2021-03-10T13:57:16.179+0000 7fe478476080 -1 osd.5 86 log_to_monitors {default=true} 2021-03-10T13:57:16.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:15 gibba011 bash[30791]: cluster 2021-03-10T13:57:14.569727+0000 mgr.gibba032.whopeo (mgr.24625) 266 : cluster [DBG] pgmap v130: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:15 gibba011 bash[31774]: cluster 2021-03-10T13:57:14.569727+0000 mgr.gibba032.whopeo (mgr.24625) 266 : cluster [DBG] pgmap v130: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:16 gibba032 bash[24004]: audit 2021-03-10T13:57:16.183181+0000 mon.b (mon.2) 5 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:16 gibba032 bash[24004]: audit 2021-03-10T13:57:16.184137+0000 mon.a (mon.0) 503 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:17.257 INFO:journalctl@ceph.osd.5.gibba032.stdout:Mar 10 13:57:17 gibba032 bash[26915]: debug 2021-03-10T13:57:17.011+0000 7fe468153700 -1 osd.5 86 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:57:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:16 gibba011 bash[30791]: audit 2021-03-10T13:57:16.183181+0000 mon.b (mon.2) 5 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:16 gibba011 bash[30791]: audit 2021-03-10T13:57:16.184137+0000 mon.a (mon.0) 503 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:16 gibba011 bash[31774]: audit 2021-03-10T13:57:16.183181+0000 mon.b (mon.2) 5 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:17.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:16 gibba011 bash[31774]: audit 2021-03-10T13:57:16.184137+0000 mon.a (mon.0) 503 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-03-10T13:57:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[30791]: cluster 2021-03-10T13:57:16.571660+0000 mgr.gibba032.whopeo (mgr.24625) 267 : cluster [DBG] pgmap v131: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[30791]: audit 2021-03-10T13:57:17.003289+0000 mon.a (mon.0) 504 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:57:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[30791]: cluster 2021-03-10T13:57:17.003667+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e89: 8 total, 7 up, 8 in 2021-03-10T13:57:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[30791]: audit 2021-03-10T13:57:17.004159+0000 mon.b (mon.2) 6 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[30791]: audit 2021-03-10T13:57:17.004724+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[31774]: cluster 2021-03-10T13:57:16.571660+0000 mgr.gibba032.whopeo (mgr.24625) 267 : cluster [DBG] pgmap v131: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[31774]: audit 2021-03-10T13:57:17.003289+0000 mon.a (mon.0) 504 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:57:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[31774]: cluster 2021-03-10T13:57:17.003667+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e89: 8 total, 7 up, 8 in 2021-03-10T13:57:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[31774]: audit 2021-03-10T13:57:17.004159+0000 mon.b (mon.2) 6 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:18 gibba011 bash[31774]: audit 2021-03-10T13:57:17.004724+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:18 gibba032 bash[24004]: cluster 2021-03-10T13:57:16.571660+0000 mgr.gibba032.whopeo (mgr.24625) 267 : cluster [DBG] pgmap v131: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 35 MiB used, 669 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:18 gibba032 bash[24004]: audit 2021-03-10T13:57:17.003289+0000 mon.a (mon.0) 504 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-03-10T13:57:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:18 gibba032 bash[24004]: cluster 2021-03-10T13:57:17.003667+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e89: 8 total, 7 up, 8 in 2021-03-10T13:57:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:18 gibba032 bash[24004]: audit 2021-03-10T13:57:17.004159+0000 mon.b (mon.2) 6 : audit [INF] from='osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:18 gibba032 bash[24004]: audit 2021-03-10T13:57:17.004724+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[30791]: cluster 2021-03-10T13:57:17.998661+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[30791]: cluster 2021-03-10T13:57:18.008059+0000 mon.a (mon.0) 508 : cluster [INF] osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435] boot 2021-03-10T13:57:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[30791]: cluster 2021-03-10T13:57:18.008102+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2021-03-10T13:57:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[30791]: audit 2021-03-10T13:57:18.009309+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:57:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[30791]: cluster 2021-03-10T13:57:19.013272+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2021-03-10T13:57:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[31774]: cluster 2021-03-10T13:57:17.998661+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[31774]: cluster 2021-03-10T13:57:18.008059+0000 mon.a (mon.0) 508 : cluster [INF] osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435] boot 2021-03-10T13:57:19.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[31774]: cluster 2021-03-10T13:57:18.008102+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2021-03-10T13:57:19.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[31774]: audit 2021-03-10T13:57:18.009309+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:57:19.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:19 gibba011 bash[31774]: cluster 2021-03-10T13:57:19.013272+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2021-03-10T13:57:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:19 gibba032 bash[24004]: cluster 2021-03-10T13:57:17.998661+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:19 gibba032 bash[24004]: cluster 2021-03-10T13:57:18.008059+0000 mon.a (mon.0) 508 : cluster [INF] osd.5 [v2:172.21.2.132:6808/1065728435,v1:172.21.2.132:6809/1065728435] boot 2021-03-10T13:57:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:19 gibba032 bash[24004]: cluster 2021-03-10T13:57:18.008102+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2021-03-10T13:57:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:19 gibba032 bash[24004]: audit 2021-03-10T13:57:18.009309+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-03-10T13:57:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:19 gibba032 bash[24004]: cluster 2021-03-10T13:57:19.013272+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2021-03-10T13:57:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:20 gibba011 bash[30791]: cluster 2021-03-10T13:57:18.573052+0000 mgr.gibba032.whopeo (mgr.24625) 268 : cluster [DBG] pgmap v134: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:20 gibba011 bash[30791]: cluster 2021-03-10T13:57:19.903659+0000 mon.a (mon.0) 512 : cluster [WRN] Health check update: Degraded data redundancy: 19/192 objects degraded (9.896%), 15 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:20 gibba011 bash[31774]: cluster 2021-03-10T13:57:18.573052+0000 mgr.gibba032.whopeo (mgr.24625) 268 : cluster [DBG] pgmap v134: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:20 gibba011 bash[31774]: cluster 2021-03-10T13:57:19.903659+0000 mon.a (mon.0) 512 : cluster [WRN] Health check update: Degraded data redundancy: 19/192 objects degraded (9.896%), 15 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:20 gibba032 bash[24004]: cluster 2021-03-10T13:57:18.573052+0000 mgr.gibba032.whopeo (mgr.24625) 268 : cluster [DBG] pgmap v134: 129 pgs: 15 active+undersized+degraded, 29 active+undersized, 85 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 19/192 objects degraded (9.896%) 2021-03-10T13:57:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:20 gibba032 bash[24004]: cluster 2021-03-10T13:57:19.903659+0000 mon.a (mon.0) 512 : cluster [WRN] Health check update: Degraded data redundancy: 19/192 objects degraded (9.896%), 15 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:22 gibba011 bash[30791]: cluster 2021-03-10T13:57:20.574790+0000 mgr.gibba032.whopeo (mgr.24625) 269 : cluster [DBG] pgmap v136: 129 pgs: 10 active+undersized+degraded, 20 active+undersized, 99 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 12/192 objects degraded (6.250%) 2021-03-10T13:57:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:22 gibba011 bash[31774]: cluster 2021-03-10T13:57:20.574790+0000 mgr.gibba032.whopeo (mgr.24625) 269 : cluster [DBG] pgmap v136: 129 pgs: 10 active+undersized+degraded, 20 active+undersized, 99 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 12/192 objects degraded (6.250%) 2021-03-10T13:57:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:22 gibba032 bash[24004]: cluster 2021-03-10T13:57:20.574790+0000 mgr.gibba032.whopeo (mgr.24625) 269 : cluster [DBG] pgmap v136: 129 pgs: 10 active+undersized+degraded, 20 active+undersized, 99 active+clean; 7.0 KiB data, 40 MiB used, 670 GiB / 672 GiB avail; 12/192 objects degraded (6.250%) 2021-03-10T13:57:23.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:23 gibba032 bash[24004]: cluster 2021-03-10T13:57:23.031548+0000 mon.a (mon.0) 513 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/192 objects degraded (6.250%), 10 pgs degraded) 2021-03-10T13:57:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:23 gibba011 bash[30791]: cluster 2021-03-10T13:57:23.031548+0000 mon.a (mon.0) 513 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/192 objects degraded (6.250%), 10 pgs degraded) 2021-03-10T13:57:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:23 gibba011 bash[31774]: cluster 2021-03-10T13:57:23.031548+0000 mon.a (mon.0) 513 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/192 objects degraded (6.250%), 10 pgs degraded) 2021-03-10T13:57:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:24 gibba032 bash[24004]: cluster 2021-03-10T13:57:22.576643+0000 mgr.gibba032.whopeo (mgr.24625) 270 : cluster [DBG] pgmap v137: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:24 gibba011 bash[30791]: cluster 2021-03-10T13:57:22.576643+0000 mgr.gibba032.whopeo (mgr.24625) 270 : cluster [DBG] pgmap v137: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:24 gibba011 bash[31774]: cluster 2021-03-10T13:57:22.576643+0000 mgr.gibba032.whopeo (mgr.24625) 270 : cluster [DBG] pgmap v137: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:26 gibba032 bash[24004]: cluster 2021-03-10T13:57:24.578002+0000 mgr.gibba032.whopeo (mgr.24625) 271 : cluster [DBG] pgmap v138: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:26 gibba032 bash[24004]: audit 2021-03-10T13:57:26.027990+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:26 gibba011 bash[30791]: cluster 2021-03-10T13:57:24.578002+0000 mgr.gibba032.whopeo (mgr.24625) 271 : cluster [DBG] pgmap v138: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:26 gibba011 bash[30791]: audit 2021-03-10T13:57:26.027990+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:26 gibba011 bash[31774]: cluster 2021-03-10T13:57:24.578002+0000 mgr.gibba032.whopeo (mgr.24625) 271 : cluster [DBG] pgmap v138: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:26 gibba011 bash[31774]: audit 2021-03-10T13:57:26.027990+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:28 gibba032 bash[24004]: cluster 2021-03-10T13:57:26.579829+0000 mgr.gibba032.whopeo (mgr.24625) 272 : cluster [DBG] pgmap v139: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:28 gibba011 bash[30791]: cluster 2021-03-10T13:57:26.579829+0000 mgr.gibba032.whopeo (mgr.24625) 272 : cluster [DBG] pgmap v139: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:28 gibba011 bash[31774]: cluster 2021-03-10T13:57:26.579829+0000 mgr.gibba032.whopeo (mgr.24625) 272 : cluster [DBG] pgmap v139: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:29.129 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:57:29.358 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:57:29.358 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (4m) 67s ago 9m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:57:29.358 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (9m) 15s ago 9m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:57:29.359 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (4m) 15s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:57:29.359 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (4m) 15s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:57:29.359 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3m) 67s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:57:29.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3m) 67s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:57:29.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3m) 15s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:57:29.360 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3m) 67s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:57:29.360 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (9m) 67s ago 9m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:57:29.360 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (9m) 15s ago 9m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:57:29.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2m) 67s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:57:29.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2m) 67s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:57:29.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (94s) 67s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:57:29.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (68s) 67s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:57:29.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (43s) 15s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:57:29.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (18s) 15s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T13:57:29.362 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (10m) 15s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 db54ff79c65d 2021-03-10T13:57:29.362 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (9m) 15s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:57:29.362 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (9m) 15s ago 9m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:57:29.362 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 67s ago 7m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:57:29.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:29 gibba032 bash[24004]: audit 2021-03-10T13:57:28.489154+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:29.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:29 gibba011 bash[30791]: audit 2021-03-10T13:57:28.489154+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:29.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:29 gibba011 bash[31774]: audit 2021-03-10T13:57:28.489154+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:29.647 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:57:29.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:57:29.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:57:29.648 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:57:29.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:57:29.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:57:29.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:57:29.649 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:57:29.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 2, 2021-03-10T13:57:29.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 6 2021-03-10T13:57:29.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:57:29.650 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:57:29.650 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:57:29.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc)": 2, 2021-03-10T13:57:29.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 12 2021-03-10T13:57:29.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:57:29.651 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:57:30.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: audit 2021-03-10T13:57:28.489743+0000 mgr.gibba032.whopeo (mgr.24625) 273 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:30.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: cephadm 2021-03-10T13:57:28.491773+0000 mgr.gibba032.whopeo (mgr.24625) 274 : cephadm [INF] Upgrade: osd.6 is safe to restart 2021-03-10T13:57:30.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: cluster 2021-03-10T13:57:28.580338+0000 mgr.gibba032.whopeo (mgr.24625) 275 : cluster [DBG] pgmap v140: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:30.462 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: audit 2021-03-10T13:57:29.120489+0000 mgr.gibba032.whopeo (mgr.24625) 276 : audit [DBG] from='client.54235 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.462 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: audit 2021-03-10T13:57:29.344321+0000 mgr.gibba032.whopeo (mgr.24625) 277 : audit [DBG] from='client.54241 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.462 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:30 gibba032 bash[24004]: audit 2021-03-10T13:57:29.647097+0000 mon.c (mon.1) 9 : audit [DBG] from='client.? 172.21.2.111:0/229574961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: audit 2021-03-10T13:57:28.489743+0000 mgr.gibba032.whopeo (mgr.24625) 273 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: cephadm 2021-03-10T13:57:28.491773+0000 mgr.gibba032.whopeo (mgr.24625) 274 : cephadm [INF] Upgrade: osd.6 is safe to restart 2021-03-10T13:57:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: cluster 2021-03-10T13:57:28.580338+0000 mgr.gibba032.whopeo (mgr.24625) 275 : cluster [DBG] pgmap v140: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: audit 2021-03-10T13:57:29.120489+0000 mgr.gibba032.whopeo (mgr.24625) 276 : audit [DBG] from='client.54235 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: audit 2021-03-10T13:57:29.344321+0000 mgr.gibba032.whopeo (mgr.24625) 277 : audit [DBG] from='client.54241 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[30791]: audit 2021-03-10T13:57:29.647097+0000 mon.c (mon.1) 9 : audit [DBG] from='client.? 172.21.2.111:0/229574961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: audit 2021-03-10T13:57:28.489743+0000 mgr.gibba032.whopeo (mgr.24625) 273 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2021-03-10T13:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: cephadm 2021-03-10T13:57:28.491773+0000 mgr.gibba032.whopeo (mgr.24625) 274 : cephadm [INF] Upgrade: osd.6 is safe to restart 2021-03-10T13:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: cluster 2021-03-10T13:57:28.580338+0000 mgr.gibba032.whopeo (mgr.24625) 275 : cluster [DBG] pgmap v140: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: audit 2021-03-10T13:57:29.120489+0000 mgr.gibba032.whopeo (mgr.24625) 276 : audit [DBG] from='client.54235 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: audit 2021-03-10T13:57:29.344321+0000 mgr.gibba032.whopeo (mgr.24625) 277 : audit [DBG] from='client.54241 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:30 gibba011 bash[31774]: audit 2021-03-10T13:57:29.647097+0000 mon.c (mon.1) 9 : audit [DBG] from='client.? 172.21.2.111:0/229574961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[30791]: cephadm 2021-03-10T13:57:30.274846+0000 mgr.gibba032.whopeo (mgr.24625) 278 : cephadm [INF] Upgrade: Updating osd.6 2021-03-10T13:57:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[30791]: audit 2021-03-10T13:57:30.285588+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[30791]: cephadm 2021-03-10T13:57:30.286789+0000 mgr.gibba032.whopeo (mgr.24625) 279 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:57:31.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[30791]: audit 2021-03-10T13:57:30.287287+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:57:31.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[31774]: cephadm 2021-03-10T13:57:30.274846+0000 mgr.gibba032.whopeo (mgr.24625) 278 : cephadm [INF] Upgrade: Updating osd.6 2021-03-10T13:57:31.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[31774]: audit 2021-03-10T13:57:30.285588+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:31.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[31774]: cephadm 2021-03-10T13:57:30.286789+0000 mgr.gibba032.whopeo (mgr.24625) 279 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:57:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:31 gibba011 bash[31774]: audit 2021-03-10T13:57:30.287287+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:57:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:31 gibba032 bash[24004]: cephadm 2021-03-10T13:57:30.274846+0000 mgr.gibba032.whopeo (mgr.24625) 278 : cephadm [INF] Upgrade: Updating osd.6 2021-03-10T13:57:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:31 gibba032 bash[24004]: audit 2021-03-10T13:57:30.285588+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:31 gibba032 bash[24004]: cephadm 2021-03-10T13:57:30.286789+0000 mgr.gibba032.whopeo (mgr.24625) 279 : cephadm [INF] Deploying daemon osd.6 on gibba032 2021-03-10T13:57:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:31 gibba032 bash[24004]: audit 2021-03-10T13:57:30.287287+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.6","key": "container_image"}]: dispatch 2021-03-10T13:57:32.299 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:32 gibba032 systemd[1]: Stopping Ceph osd.6 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:57:32.299 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:32 gibba032 bash[15138]: debug 2021-03-10T13:57:32.263+0000 7f21b191d700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:57:32.299 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:32 gibba032 bash[15138]: debug 2021-03-10T13:57:32.267+0000 7f21b191d700 -1 osd.6 91 *** Got signal Terminated *** 2021-03-10T13:57:32.300 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:32 gibba032 bash[15138]: debug 2021-03-10T13:57:32.267+0000 7f21b191d700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:57:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:32 gibba011 bash[30791]: cluster 2021-03-10T13:57:30.581864+0000 mgr.gibba032.whopeo (mgr.24625) 280 : cluster [DBG] pgmap v141: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:32 gibba011 bash[31774]: cluster 2021-03-10T13:57:30.581864+0000 mgr.gibba032.whopeo (mgr.24625) 280 : cluster [DBG] pgmap v141: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:32.610 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:32 gibba032 bash[24004]: cluster 2021-03-10T13:57:30.581864+0000 mgr.gibba032.whopeo (mgr.24625) 280 : cluster [DBG] pgmap v141: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:33.506 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:33 gibba032 docker[27816]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.6 2021-03-10T13:57:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.600440+0000 mon.a (mon.0) 518 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.600547+0000 mon.a (mon.0) 519 : cluster [INF] osd.6 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.601075+0000 mon.a (mon.0) 520 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.601479+0000 mon.a (mon.0) 521 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.601796+0000 mon.a (mon.0) 522 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.602138+0000 mon.a (mon.0) 523 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.602589+0000 mon.a (mon.0) 524 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.603005+0000 mon.a (mon.0) 525 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.603187+0000 mon.a (mon.0) 526 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.603609+0000 mon.a (mon.0) 527 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.603787+0000 mon.a (mon.0) 528 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.604335+0000 mon.a (mon.0) 529 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.604608+0000 mon.a (mon.0) 530 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.604841+0000 mon.a (mon.0) 531 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.604927+0000 mon.a (mon.0) 532 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.954603+0000 mon.a (mon.0) 533 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.956256+0000 mon.a (mon.0) 534 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.956787+0000 mon.a (mon.0) 535 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.998161+0000 mon.a (mon.0) 536 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.998285+0000 mon.a (mon.0) 537 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.998487+0000 mon.a (mon.0) 538 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.999790+0000 mon.a (mon.0) 539 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.000201+0000 mon.a (mon.0) 540 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.000395+0000 mon.a (mon.0) 541 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.003917+0000 mon.a (mon.0) 542 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.004158+0000 mon.a (mon.0) 543 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.004359+0000 mon.a (mon.0) 544 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.004744+0000 mon.a (mon.0) 545 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.004937+0000 mon.a (mon.0) 546 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.005200+0000 mon.a (mon.0) 547 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.005513+0000 mon.a (mon.0) 548 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.005753+0000 mon.a (mon.0) 549 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.153648+0000 mon.a (mon.0) 550 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.155355+0000 mon.a (mon.0) 551 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:33 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.160388+0000 mon.a (mon.0) 552 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.600440+0000 mon.a (mon.0) 518 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.600547+0000 mon.a (mon.0) 519 : cluster [INF] osd.6 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.601075+0000 mon.a (mon.0) 520 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.601479+0000 mon.a (mon.0) 521 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.601796+0000 mon.a (mon.0) 522 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.602138+0000 mon.a (mon.0) 523 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.602589+0000 mon.a (mon.0) 524 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.603005+0000 mon.a (mon.0) 525 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.603187+0000 mon.a (mon.0) 526 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.603609+0000 mon.a (mon.0) 527 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.603787+0000 mon.a (mon.0) 528 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.604335+0000 mon.a (mon.0) 529 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.604608+0000 mon.a (mon.0) 530 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.604841+0000 mon.a (mon.0) 531 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.604927+0000 mon.a (mon.0) 532 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.954603+0000 mon.a (mon.0) 533 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.956256+0000 mon.a (mon.0) 534 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.956787+0000 mon.a (mon.0) 535 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.998161+0000 mon.a (mon.0) 536 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.998285+0000 mon.a (mon.0) 537 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.998487+0000 mon.a (mon.0) 538 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.999790+0000 mon.a (mon.0) 539 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.000201+0000 mon.a (mon.0) 540 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.000395+0000 mon.a (mon.0) 541 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.003917+0000 mon.a (mon.0) 542 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.004158+0000 mon.a (mon.0) 543 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.600440+0000 mon.a (mon.0) 518 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.600547+0000 mon.a (mon.0) 519 : cluster [INF] osd.6 failed (root=default,host=gibba032) (connection refused reported by osd.7) 2021-03-10T13:57:33.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.601075+0000 mon.a (mon.0) 520 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.601479+0000 mon.a (mon.0) 521 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.601796+0000 mon.a (mon.0) 522 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.602138+0000 mon.a (mon.0) 523 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.602589+0000 mon.a (mon.0) 524 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.603005+0000 mon.a (mon.0) 525 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.603187+0000 mon.a (mon.0) 526 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.603609+0000 mon.a (mon.0) 527 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.603787+0000 mon.a (mon.0) 528 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.604335+0000 mon.a (mon.0) 529 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.604608+0000 mon.a (mon.0) 530 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.604841+0000 mon.a (mon.0) 531 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.604927+0000 mon.a (mon.0) 532 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.954603+0000 mon.a (mon.0) 533 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.956256+0000 mon.a (mon.0) 534 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.956787+0000 mon.a (mon.0) 535 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.998161+0000 mon.a (mon.0) 536 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.998285+0000 mon.a (mon.0) 537 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.998487+0000 mon.a (mon.0) 538 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.999790+0000 mon.a (mon.0) 539 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.000201+0000 mon.a (mon.0) 540 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.000395+0000 mon.a (mon.0) 541 : cluster [DBG] osd.6 reported immediately failed by osd.5 2021-03-10T13:57:33.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.003917+0000 mon.a (mon.0) 542 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.004158+0000 mon.a (mon.0) 543 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.004359+0000 mon.a (mon.0) 544 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.004744+0000 mon.a (mon.0) 545 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.004937+0000 mon.a (mon.0) 546 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.005200+0000 mon.a (mon.0) 547 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.005513+0000 mon.a (mon.0) 548 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.005753+0000 mon.a (mon.0) 549 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.153648+0000 mon.a (mon.0) 550 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.155355+0000 mon.a (mon.0) 551 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.160388+0000 mon.a (mon.0) 552 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.004359+0000 mon.a (mon.0) 544 : cluster [DBG] osd.6 reported immediately failed by osd.2 2021-03-10T13:57:33.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.004744+0000 mon.a (mon.0) 545 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.004937+0000 mon.a (mon.0) 546 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.005200+0000 mon.a (mon.0) 547 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:33.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.005513+0000 mon.a (mon.0) 548 : cluster [DBG] osd.6 reported immediately failed by osd.3 2021-03-10T13:57:33.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.005753+0000 mon.a (mon.0) 549 : cluster [DBG] osd.6 reported immediately failed by osd.0 2021-03-10T13:57:33.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.153648+0000 mon.a (mon.0) 550 : cluster [DBG] osd.6 reported immediately failed by osd.4 2021-03-10T13:57:33.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.155355+0000 mon.a (mon.0) 551 : cluster [DBG] osd.6 reported immediately failed by osd.7 2021-03-10T13:57:33.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:33 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.160388+0000 mon.a (mon.0) 552 : cluster [DBG] osd.6 reported immediately failed by osd.1 2021-03-10T13:57:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:34 gibba032 bash[24004]: cluster 2021-03-10T13:57:32.583877+0000 mgr.gibba032.whopeo (mgr.24625) 281 : cluster [DBG] pgmap v142: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:34 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.299200+0000 mon.a (mon.0) 553 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:34 gibba032 bash[24004]: cluster 2021-03-10T13:57:33.324755+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e92: 8 total, 7 up, 8 in 2021-03-10T13:57:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[30791]: cluster 2021-03-10T13:57:32.583877+0000 mgr.gibba032.whopeo (mgr.24625) 281 : cluster [DBG] pgmap v142: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.299200+0000 mon.a (mon.0) 553 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[30791]: cluster 2021-03-10T13:57:33.324755+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e92: 8 total, 7 up, 8 in 2021-03-10T13:57:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[31774]: cluster 2021-03-10T13:57:32.583877+0000 mgr.gibba032.whopeo (mgr.24625) 281 : cluster [DBG] pgmap v142: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.299200+0000 mon.a (mon.0) 553 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:57:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:34 gibba011 bash[31774]: cluster 2021-03-10T13:57:33.324755+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e92: 8 total, 7 up, 8 in 2021-03-10T13:57:35.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:35 gibba032 bash[24004]: cluster 2021-03-10T13:57:34.319849+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e93: 8 total, 7 up, 8 in 2021-03-10T13:57:35.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:35 gibba032 bash[24004]: audit 2021-03-10T13:57:35.300039+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:35 gibba032 bash[24004]: audit 2021-03-10T13:57:35.301503+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:35.507 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:35 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.6.service: Succeeded. 2021-03-10T13:57:35.507 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:35 gibba032 systemd[1]: Stopped Ceph osd.6 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:57:35.507 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:35 gibba032 systemd[1]: Started Ceph osd.6 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:57:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[30791]: cluster 2021-03-10T13:57:34.319849+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e93: 8 total, 7 up, 8 in 2021-03-10T13:57:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[30791]: audit 2021-03-10T13:57:35.300039+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[30791]: audit 2021-03-10T13:57:35.301503+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[31774]: cluster 2021-03-10T13:57:34.319849+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e93: 8 total, 7 up, 8 in 2021-03-10T13:57:35.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[31774]: audit 2021-03-10T13:57:35.300039+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:35.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:35 gibba011 bash[31774]: audit 2021-03-10T13:57:35.301503+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:57:36.481 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[24004]: cluster 2021-03-10T13:57:34.585007+0000 mgr.gibba032.whopeo (mgr.24625) 282 : cluster [DBG] pgmap v145: 129 pgs: 11 stale+active+clean, 3 peering, 17 active+clean+wait, 98 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:36.482 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[24004]: cephadm 2021-03-10T13:57:35.300774+0000 mgr.gibba032.whopeo (mgr.24625) 283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:57:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:36 gibba011 bash[30791]: cluster 2021-03-10T13:57:34.585007+0000 mgr.gibba032.whopeo (mgr.24625) 282 : cluster [DBG] pgmap v145: 129 pgs: 11 stale+active+clean, 3 peering, 17 active+clean+wait, 98 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:36 gibba011 bash[30791]: cephadm 2021-03-10T13:57:35.300774+0000 mgr.gibba032.whopeo (mgr.24625) 283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:57:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:36 gibba011 bash[31774]: cluster 2021-03-10T13:57:34.585007+0000 mgr.gibba032.whopeo (mgr.24625) 282 : cluster [DBG] pgmap v145: 129 pgs: 11 stale+active+clean, 3 peering, 17 active+clean+wait, 98 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail 2021-03-10T13:57:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:36 gibba011 bash[31774]: cephadm 2021-03-10T13:57:35.300774+0000 mgr.gibba032.whopeo (mgr.24625) 283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:57:36.757 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-03-10T13:57:36.757 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-6 --no-mon-config 2021-03-10T13:57:36.757 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6/block 2021-03-10T13:57:36.757 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-6/block 2021-03-10T13:57:36.757 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-03-10T13:57:36.758 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-03-10T13:57:36.758 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:36 gibba032 bash[27985]: --> ceph-volume lvm activate successful for osd ID: 6 2021-03-10T13:57:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:37 gibba032 bash[24004]: cluster 2021-03-10T13:57:36.586567+0000 mgr.gibba032.whopeo (mgr.24625) 284 : cluster [DBG] pgmap v146: 129 pgs: 3 stale+active+clean, 7 active+undersized+degraded, 3 peering, 17 active+clean+wait, 18 active+undersized, 81 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:57:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:37 gibba032 bash[24004]: cluster 2021-03-10T13:57:37.327256+0000 mon.a (mon.0) 558 : cluster [WRN] Health check failed: Degraded data redundancy: 11/192 objects degraded (5.729%), 7 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:37.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:37 gibba011 bash[30791]: cluster 2021-03-10T13:57:36.586567+0000 mgr.gibba032.whopeo (mgr.24625) 284 : cluster [DBG] pgmap v146: 129 pgs: 3 stale+active+clean, 7 active+undersized+degraded, 3 peering, 17 active+clean+wait, 18 active+undersized, 81 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:57:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:37 gibba011 bash[30791]: cluster 2021-03-10T13:57:37.327256+0000 mon.a (mon.0) 558 : cluster [WRN] Health check failed: Degraded data redundancy: 11/192 objects degraded (5.729%), 7 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:37.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:37 gibba011 bash[31774]: cluster 2021-03-10T13:57:36.586567+0000 mgr.gibba032.whopeo (mgr.24625) 284 : cluster [DBG] pgmap v146: 129 pgs: 3 stale+active+clean, 7 active+undersized+degraded, 3 peering, 17 active+clean+wait, 18 active+undersized, 81 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:57:37.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:37 gibba011 bash[31774]: cluster 2021-03-10T13:57:37.327256+0000 mon.a (mon.0) 558 : cluster [WRN] Health check failed: Degraded data redundancy: 11/192 objects degraded (5.729%), 7 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:38.256 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:38 gibba032 bash[28111]: debug 2021-03-10T13:57:38.047+0000 7f1969bdf700 1 -- 172.21.2.132:0/639726362 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c1163664e0 con 0x55c1163e3400 2021-03-10T13:57:39.006 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:38 gibba032 bash[28111]: debug 2021-03-10T13:57:38.651+0000 7f1971674080 -1 Falling back to public interface 2021-03-10T13:57:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cluster 2021-03-10T13:57:38.587115+0000 mgr.gibba032.whopeo (mgr.24625) 285 : cluster [DBG] pgmap v147: 129 pgs: 13 active+undersized+degraded, 3 peering, 27 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.087880+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.088389+0000 mgr.gibba032.whopeo (mgr.24625) 286 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:57:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.284106+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.286567+0000 mgr.gibba032.whopeo (mgr.24625) 287 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.287282+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.288378+0000 mgr.gibba032.whopeo (mgr.24625) 288 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.289044+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.290028+0000 mgr.gibba032.whopeo (mgr.24625) 289 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.290072+0000 mgr.gibba032.whopeo (mgr.24625) 290 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.290691+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.291552+0000 mgr.gibba032.whopeo (mgr.24625) 291 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.291589+0000 mgr.gibba032.whopeo (mgr.24625) 292 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.292121+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.292964+0000 mgr.gibba032.whopeo (mgr.24625) 293 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:40.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.294360+0000 mgr.gibba032.whopeo (mgr.24625) 294 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:40.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.294398+0000 mgr.gibba032.whopeo (mgr.24625) 295 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:40.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.295010+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: audit 2021-03-10T13:57:39.295093+0000 mgr.gibba032.whopeo (mgr.24625) 296 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:40 gibba032 bash[24004]: cephadm 2021-03-10T13:57:39.296363+0000 mgr.gibba032.whopeo (mgr.24625) 297 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2021-03-10T13:57:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cluster 2021-03-10T13:57:38.587115+0000 mgr.gibba032.whopeo (mgr.24625) 285 : cluster [DBG] pgmap v147: 129 pgs: 13 active+undersized+degraded, 3 peering, 27 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.087880+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.088389+0000 mgr.gibba032.whopeo (mgr.24625) 286 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.284106+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.286567+0000 mgr.gibba032.whopeo (mgr.24625) 287 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.287282+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.288378+0000 mgr.gibba032.whopeo (mgr.24625) 288 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.289044+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.290028+0000 mgr.gibba032.whopeo (mgr.24625) 289 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.290072+0000 mgr.gibba032.whopeo (mgr.24625) 290 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.290691+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.291552+0000 mgr.gibba032.whopeo (mgr.24625) 291 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.291589+0000 mgr.gibba032.whopeo (mgr.24625) 292 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.292121+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.292964+0000 mgr.gibba032.whopeo (mgr.24625) 293 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.294360+0000 mgr.gibba032.whopeo (mgr.24625) 294 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.294398+0000 mgr.gibba032.whopeo (mgr.24625) 295 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.295010+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: audit 2021-03-10T13:57:39.295093+0000 mgr.gibba032.whopeo (mgr.24625) 296 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[30791]: cephadm 2021-03-10T13:57:39.296363+0000 mgr.gibba032.whopeo (mgr.24625) 297 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2021-03-10T13:57:40.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cluster 2021-03-10T13:57:38.587115+0000 mgr.gibba032.whopeo (mgr.24625) 285 : cluster [DBG] pgmap v147: 129 pgs: 13 active+undersized+degraded, 3 peering, 27 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:40.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.087880+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.088389+0000 mgr.gibba032.whopeo (mgr.24625) 286 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.284106+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.286567+0000 mgr.gibba032.whopeo (mgr.24625) 287 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.287282+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.288378+0000 mgr.gibba032.whopeo (mgr.24625) 288 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:57:40.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.289044+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.290028+0000 mgr.gibba032.whopeo (mgr.24625) 289 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:57:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.290072+0000 mgr.gibba032.whopeo (mgr.24625) 290 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:57:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.290691+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.291552+0000 mgr.gibba032.whopeo (mgr.24625) 291 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:57:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.291589+0000 mgr.gibba032.whopeo (mgr.24625) 292 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:57:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.292121+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:57:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.292964+0000 mgr.gibba032.whopeo (mgr.24625) 293 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:57:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.294360+0000 mgr.gibba032.whopeo (mgr.24625) 294 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:57:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.294398+0000 mgr.gibba032.whopeo (mgr.24625) 295 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:57:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.295010+0000 mon.a (mon.0) 565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: audit 2021-03-10T13:57:39.295093+0000 mgr.gibba032.whopeo (mgr.24625) 296 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:40 gibba011 bash[31774]: cephadm 2021-03-10T13:57:39.296363+0000 mgr.gibba032.whopeo (mgr.24625) 297 : cephadm [ERR] Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2021-03-10T13:57:42.106 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:41 gibba032 bash[28111]: debug 2021-03-10T13:57:41.843+0000 7f1971674080 -1 bluestore(/var/lib/ceph/osd/ceph-6) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:57:42.106 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:41 gibba032 bash[28111]: debug 2021-03-10T13:57:41.843+0000 7f195c939700 -1 bluestore(/var/lib/ceph/osd/ceph-6) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:57:42.106 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:41 gibba032 bash[28111]: debug 2021-03-10T13:57:41.847+0000 7f195c939700 -1 bluestore(/var/lib/ceph/osd/ceph-6) fsck warning: #1:d525e5cb:::INTEL_SSDPEL1K375GA_PHKM920000BW375A:head# has omap that is not per-pg or pgmeta 2021-03-10T13:57:42.107 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:41 gibba032 bash[28111]: debug 2021-03-10T13:57:41.967+0000 7f1971674080 -1 osd.6 91 log_to_monitors {default=true} 2021-03-10T13:57:42.507 INFO:journalctl@ceph.osd.6.gibba032.stdout:Mar 10 13:57:42 gibba032 bash[28111]: debug 2021-03-10T13:57:42.111+0000 7f1962b54700 -1 osd.6 91 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:57:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:42 gibba032 bash[24004]: cluster 2021-03-10T13:57:40.588581+0000 mgr.gibba032.whopeo (mgr.24625) 298 : cluster [DBG] pgmap v148: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:42 gibba032 bash[24004]: audit 2021-03-10T13:57:41.973507+0000 mon.b (mon.2) 7 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:42 gibba032 bash[24004]: audit 2021-03-10T13:57:41.974806+0000 mon.a (mon.0) 566 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[30791]: cluster 2021-03-10T13:57:40.588581+0000 mgr.gibba032.whopeo (mgr.24625) 298 : cluster [DBG] pgmap v148: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[30791]: audit 2021-03-10T13:57:41.973507+0000 mon.b (mon.2) 7 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[30791]: audit 2021-03-10T13:57:41.974806+0000 mon.a (mon.0) 566 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[31774]: cluster 2021-03-10T13:57:40.588581+0000 mgr.gibba032.whopeo (mgr.24625) 298 : cluster [DBG] pgmap v148: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 41 MiB used, 670 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[31774]: audit 2021-03-10T13:57:41.973507+0000 mon.b (mon.2) 7 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:42 gibba011 bash[31774]: audit 2021-03-10T13:57:41.974806+0000 mon.a (mon.0) 566 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2021-03-10T13:57:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:43 gibba032 bash[24004]: audit 2021-03-10T13:57:42.103371+0000 mon.a (mon.0) 567 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:57:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:43 gibba032 bash[24004]: cluster 2021-03-10T13:57:42.103437+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e94: 8 total, 7 up, 8 in 2021-03-10T13:57:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:43 gibba032 bash[24004]: audit 2021-03-10T13:57:42.104572+0000 mon.b (mon.2) 8 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:43 gibba032 bash[24004]: audit 2021-03-10T13:57:42.107419+0000 mon.a (mon.0) 569 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[30791]: audit 2021-03-10T13:57:42.103371+0000 mon.a (mon.0) 567 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:57:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[30791]: cluster 2021-03-10T13:57:42.103437+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e94: 8 total, 7 up, 8 in 2021-03-10T13:57:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[30791]: audit 2021-03-10T13:57:42.104572+0000 mon.b (mon.2) 8 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[30791]: audit 2021-03-10T13:57:42.107419+0000 mon.a (mon.0) 569 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:43.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[31774]: audit 2021-03-10T13:57:42.103371+0000 mon.a (mon.0) 567 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2021-03-10T13:57:43.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[31774]: cluster 2021-03-10T13:57:42.103437+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e94: 8 total, 7 up, 8 in 2021-03-10T13:57:43.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[31774]: audit 2021-03-10T13:57:42.104572+0000 mon.b (mon.2) 8 : audit [INF] from='osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:43.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:43 gibba011 bash[31774]: audit 2021-03-10T13:57:42.107419+0000 mon.a (mon.0) 569 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:57:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:44 gibba032 bash[24004]: cluster 2021-03-10T13:57:42.589878+0000 mgr.gibba032.whopeo (mgr.24625) 299 : cluster [DBG] pgmap v150: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:44 gibba032 bash[24004]: cluster 2021-03-10T13:57:43.106360+0000 mon.a (mon.0) 570 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:44 gibba032 bash[24004]: cluster 2021-03-10T13:57:43.116197+0000 mon.a (mon.0) 571 : cluster [INF] osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018] boot 2021-03-10T13:57:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:44 gibba032 bash[24004]: cluster 2021-03-10T13:57:43.116272+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2021-03-10T13:57:44.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:44 gibba032 bash[24004]: audit 2021-03-10T13:57:43.116862+0000 mon.a (mon.0) 573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:57:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[30791]: cluster 2021-03-10T13:57:42.589878+0000 mgr.gibba032.whopeo (mgr.24625) 299 : cluster [DBG] pgmap v150: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[30791]: cluster 2021-03-10T13:57:43.106360+0000 mon.a (mon.0) 570 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[30791]: cluster 2021-03-10T13:57:43.116197+0000 mon.a (mon.0) 571 : cluster [INF] osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018] boot 2021-03-10T13:57:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[30791]: cluster 2021-03-10T13:57:43.116272+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2021-03-10T13:57:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[30791]: audit 2021-03-10T13:57:43.116862+0000 mon.a (mon.0) 573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:57:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[31774]: cluster 2021-03-10T13:57:42.589878+0000 mgr.gibba032.whopeo (mgr.24625) 299 : cluster [DBG] pgmap v150: 129 pgs: 13 active+undersized+degraded, 30 active+undersized, 86 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[31774]: cluster 2021-03-10T13:57:43.106360+0000 mon.a (mon.0) 570 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:57:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[31774]: cluster 2021-03-10T13:57:43.116197+0000 mon.a (mon.0) 571 : cluster [INF] osd.6 [v2:172.21.2.132:6816/3838782018,v1:172.21.2.132:6817/3838782018] boot 2021-03-10T13:57:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[31774]: cluster 2021-03-10T13:57:43.116272+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2021-03-10T13:57:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:44 gibba011 bash[31774]: audit 2021-03-10T13:57:43.116862+0000 mon.a (mon.0) 573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2021-03-10T13:57:45.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:45 gibba032 bash[24004]: cluster 2021-03-10T13:57:44.120241+0000 mon.a (mon.0) 574 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2021-03-10T13:57:45.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:45 gibba032 bash[24004]: cluster 2021-03-10T13:57:44.916320+0000 mon.a (mon.0) 575 : cluster [WRN] Health check update: Degraded data redundancy: 18/192 objects degraded (9.375%), 13 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:45.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:45 gibba011 bash[30791]: cluster 2021-03-10T13:57:44.120241+0000 mon.a (mon.0) 574 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2021-03-10T13:57:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:45 gibba011 bash[30791]: cluster 2021-03-10T13:57:44.916320+0000 mon.a (mon.0) 575 : cluster [WRN] Health check update: Degraded data redundancy: 18/192 objects degraded (9.375%), 13 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:45.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:45 gibba011 bash[31774]: cluster 2021-03-10T13:57:44.120241+0000 mon.a (mon.0) 574 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2021-03-10T13:57:45.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:45 gibba011 bash[31774]: cluster 2021-03-10T13:57:44.916320+0000 mon.a (mon.0) 575 : cluster [WRN] Health check update: Degraded data redundancy: 18/192 objects degraded (9.375%), 13 pgs degraded (PG_DEGRADED) 2021-03-10T13:57:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:46 gibba032 bash[24004]: cluster 2021-03-10T13:57:44.591445+0000 mgr.gibba032.whopeo (mgr.24625) 300 : cluster [DBG] pgmap v153: 129 pgs: 13 active+undersized+degraded, 29 active+undersized, 87 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:46 gibba011 bash[30791]: cluster 2021-03-10T13:57:44.591445+0000 mgr.gibba032.whopeo (mgr.24625) 300 : cluster [DBG] pgmap v153: 129 pgs: 13 active+undersized+degraded, 29 active+undersized, 87 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:46 gibba011 bash[31774]: cluster 2021-03-10T13:57:44.591445+0000 mgr.gibba032.whopeo (mgr.24625) 300 : cluster [DBG] pgmap v153: 129 pgs: 13 active+undersized+degraded, 29 active+undersized, 87 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 18/192 objects degraded (9.375%) 2021-03-10T13:57:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:48 gibba032 bash[24004]: cluster 2021-03-10T13:57:46.593372+0000 mgr.gibba032.whopeo (mgr.24625) 301 : cluster [DBG] pgmap v154: 129 pgs: 8 active+undersized+degraded, 17 active+undersized, 104 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:57:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:48 gibba011 bash[30791]: cluster 2021-03-10T13:57:46.593372+0000 mgr.gibba032.whopeo (mgr.24625) 301 : cluster [DBG] pgmap v154: 129 pgs: 8 active+undersized+degraded, 17 active+undersized, 104 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:57:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:48 gibba011 bash[31774]: cluster 2021-03-10T13:57:46.593372+0000 mgr.gibba032.whopeo (mgr.24625) 301 : cluster [DBG] pgmap v154: 129 pgs: 8 active+undersized+degraded, 17 active+undersized, 104 active+clean; 7.0 KiB data, 46 MiB used, 671 GiB / 672 GiB avail; 9/192 objects degraded (4.688%) 2021-03-10T13:57:49.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:49 gibba032 bash[24004]: cluster 2021-03-10T13:57:49.153382+0000 mon.a (mon.0) 576 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/192 objects degraded (4.688%), 8 pgs degraded) 2021-03-10T13:57:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:49 gibba011 bash[30791]: cluster 2021-03-10T13:57:49.153382+0000 mon.a (mon.0) 576 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/192 objects degraded (4.688%), 8 pgs degraded) 2021-03-10T13:57:49.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:49 gibba011 bash[31774]: cluster 2021-03-10T13:57:49.153382+0000 mon.a (mon.0) 576 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/192 objects degraded (4.688%), 8 pgs degraded) 2021-03-10T13:57:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:50 gibba032 bash[24004]: cluster 2021-03-10T13:57:48.595011+0000 mgr.gibba032.whopeo (mgr.24625) 302 : cluster [DBG] pgmap v155: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:50 gibba011 bash[30791]: cluster 2021-03-10T13:57:48.595011+0000 mgr.gibba032.whopeo (mgr.24625) 302 : cluster [DBG] pgmap v155: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:50 gibba011 bash[31774]: cluster 2021-03-10T13:57:48.595011+0000 mgr.gibba032.whopeo (mgr.24625) 302 : cluster [DBG] pgmap v155: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:52 gibba011 bash[30791]: cluster 2021-03-10T13:57:50.596535+0000 mgr.gibba032.whopeo (mgr.24625) 303 : cluster [DBG] pgmap v156: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:52 gibba011 bash[30791]: audit 2021-03-10T13:57:51.033154+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:52 gibba011 bash[31774]: cluster 2021-03-10T13:57:50.596535+0000 mgr.gibba032.whopeo (mgr.24625) 303 : cluster [DBG] pgmap v156: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:52 gibba011 bash[31774]: audit 2021-03-10T13:57:51.033154+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:52 gibba032 bash[24004]: cluster 2021-03-10T13:57:50.596535+0000 mgr.gibba032.whopeo (mgr.24625) 303 : cluster [DBG] pgmap v156: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:52 gibba032 bash[24004]: audit 2021-03-10T13:57:51.033154+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:54 gibba011 bash[30791]: cluster 2021-03-10T13:57:52.598473+0000 mgr.gibba032.whopeo (mgr.24625) 304 : cluster [DBG] pgmap v157: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:54 gibba011 bash[31774]: cluster 2021-03-10T13:57:52.598473+0000 mgr.gibba032.whopeo (mgr.24625) 304 : cluster [DBG] pgmap v157: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:54 gibba032 bash[24004]: cluster 2021-03-10T13:57:52.598473+0000 mgr.gibba032.whopeo (mgr.24625) 304 : cluster [DBG] pgmap v157: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[30791]: audit 2021-03-10T13:57:54.313125+0000 mon.a (mon.0) 578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[30791]: audit 2021-03-10T13:57:54.313927+0000 mgr.gibba032.whopeo (mgr.24625) 305 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[30791]: cephadm 2021-03-10T13:57:54.319163+0000 mgr.gibba032.whopeo (mgr.24625) 306 : cephadm [INF] Upgrade: osd.7 is safe to restart 2021-03-10T13:57:55.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[31774]: audit 2021-03-10T13:57:54.313125+0000 mon.a (mon.0) 578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[31774]: audit 2021-03-10T13:57:54.313927+0000 mgr.gibba032.whopeo (mgr.24625) 305 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:55 gibba011 bash[31774]: cephadm 2021-03-10T13:57:54.319163+0000 mgr.gibba032.whopeo (mgr.24625) 306 : cephadm [INF] Upgrade: osd.7 is safe to restart 2021-03-10T13:57:55.387 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:55 gibba032 bash[24004]: audit 2021-03-10T13:57:54.313125+0000 mon.a (mon.0) 578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.387 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:55 gibba032 bash[24004]: audit 2021-03-10T13:57:54.313927+0000 mgr.gibba032.whopeo (mgr.24625) 305 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2021-03-10T13:57:55.388 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:55 gibba032 bash[24004]: cephadm 2021-03-10T13:57:54.319163+0000 mgr.gibba032.whopeo (mgr.24625) 306 : cephadm [INF] Upgrade: osd.7 is safe to restart 2021-03-10T13:57:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: cluster 2021-03-10T13:57:54.599739+0000 mgr.gibba032.whopeo (mgr.24625) 307 : cluster [DBG] pgmap v158: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.879751+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T13:57:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.880530+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.881228+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T13:57:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.881916+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.998578+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:57:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[30791]: audit 2021-03-10T13:57:55.999195+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:57:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: cluster 2021-03-10T13:57:54.599739+0000 mgr.gibba032.whopeo (mgr.24625) 307 : cluster [DBG] pgmap v158: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.879751+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T13:57:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.880530+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.881228+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T13:57:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.881916+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.998578+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:57:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:56 gibba011 bash[31774]: audit 2021-03-10T13:57:55.999195+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:57:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: cluster 2021-03-10T13:57:54.599739+0000 mgr.gibba032.whopeo (mgr.24625) 307 : cluster [DBG] pgmap v158: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.879751+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T13:57:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.880530+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.881228+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T13:57:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.881916+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T13:57:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.998578+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:57:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:56 gibba032 bash[24004]: audit 2021-03-10T13:57:55.999195+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:57:57.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:57 gibba032 bash[24004]: cephadm 2021-03-10T13:57:56.115628+0000 mgr.gibba032.whopeo (mgr.24625) 308 : cephadm [INF] Upgrade: Updating osd.7 2021-03-10T13:57:57.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:57 gibba032 bash[24004]: audit 2021-03-10T13:57:56.123833+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:57.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:57 gibba032 bash[24004]: cephadm 2021-03-10T13:57:56.125410+0000 mgr.gibba032.whopeo (mgr.24625) 309 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:57:57.462 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:57 gibba032 bash[24004]: audit 2021-03-10T13:57:56.126560+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:57:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[30791]: cephadm 2021-03-10T13:57:56.115628+0000 mgr.gibba032.whopeo (mgr.24625) 308 : cephadm [INF] Upgrade: Updating osd.7 2021-03-10T13:57:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[30791]: audit 2021-03-10T13:57:56.123833+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[30791]: cephadm 2021-03-10T13:57:56.125410+0000 mgr.gibba032.whopeo (mgr.24625) 309 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:57:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[30791]: audit 2021-03-10T13:57:56.126560+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:57:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[31774]: cephadm 2021-03-10T13:57:56.115628+0000 mgr.gibba032.whopeo (mgr.24625) 308 : cephadm [INF] Upgrade: Updating osd.7 2021-03-10T13:57:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[31774]: audit 2021-03-10T13:57:56.123833+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:57:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[31774]: cephadm 2021-03-10T13:57:56.125410+0000 mgr.gibba032.whopeo (mgr.24625) 309 : cephadm [INF] Deploying daemon osd.7 on gibba032 2021-03-10T13:57:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:57 gibba011 bash[31774]: audit 2021-03-10T13:57:56.126560+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd.7","key": "container_image"}]: dispatch 2021-03-10T13:57:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:58 gibba032 bash[24004]: cluster 2021-03-10T13:57:56.601576+0000 mgr.gibba032.whopeo (mgr.24625) 310 : cluster [DBG] pgmap v159: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:58.257 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:57:58 gibba032 systemd[1]: Stopping Ceph osd.7 for 6fd36d94-81a6-11eb-9080-001a4aab830c... 2021-03-10T13:57:58.257 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:57:58 gibba032 bash[16653]: debug 2021-03-10T13:57:58.171+0000 7f053a5f1700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2021-03-10T13:57:58.257 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:57:58 gibba032 bash[16653]: debug 2021-03-10T13:57:58.171+0000 7f053a5f1700 -1 osd.7 96 *** Got signal Terminated *** 2021-03-10T13:57:58.258 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:57:58 gibba032 bash[16653]: debug 2021-03-10T13:57:58.171+0000 7f053a5f1700 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-03-10T13:57:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:58 gibba011 bash[30791]: cluster 2021-03-10T13:57:56.601576+0000 mgr.gibba032.whopeo (mgr.24625) 310 : cluster [DBG] pgmap v159: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:58 gibba011 bash[31774]: cluster 2021-03-10T13:57:56.601576+0000 mgr.gibba032.whopeo (mgr.24625) 310 : cluster [DBG] pgmap v159: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:57:59.144 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:57:58 gibba032 docker[29014]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c-osd.7 2021-03-10T13:57:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.483279+0000 mon.a (mon.0) 587 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.483434+0000 mon.a (mon.0) 588 : cluster [INF] osd.7 failed (root=default,host=gibba032) (connection refused reported by osd.5) 2021-03-10T13:57:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.484221+0000 mon.a (mon.0) 589 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.484718+0000 mon.a (mon.0) 590 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.485158+0000 mon.a (mon.0) 591 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.485336+0000 mon.a (mon.0) 592 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.485852+0000 mon.a (mon.0) 593 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.486143+0000 mon.a (mon.0) 594 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.486656+0000 mon.a (mon.0) 595 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.486931+0000 mon.a (mon.0) 596 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.488262+0000 mon.a (mon.0) 597 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.488488+0000 mon.a (mon.0) 598 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.488934+0000 mon.a (mon.0) 599 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.489114+0000 mon.a (mon.0) 600 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.489873+0000 mon.a (mon.0) 601 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.884192+0000 mon.a (mon.0) 602 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.884523+0000 mon.a (mon.0) 603 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.885086+0000 mon.a (mon.0) 604 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.885362+0000 mon.a (mon.0) 605 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.885897+0000 mon.a (mon.0) 606 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.886190+0000 mon.a (mon.0) 607 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.888254+0000 mon.a (mon.0) 608 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.888465+0000 mon.a (mon.0) 609 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.889254+0000 mon.a (mon.0) 610 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.892202+0000 mon.a (mon.0) 611 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.892424+0000 mon.a (mon.0) 612 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.892879+0000 mon.a (mon.0) 613 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.893058+0000 mon.a (mon.0) 614 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.893634+0000 mon.a (mon.0) 615 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.021376+0000 mon.a (mon.0) 616 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.022834+0000 mon.a (mon.0) 617 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.023871+0000 mon.a (mon.0) 618 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:57:59 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.024616+0000 mon.a (mon.0) 619 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.483279+0000 mon.a (mon.0) 587 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.483434+0000 mon.a (mon.0) 588 : cluster [INF] osd.7 failed (root=default,host=gibba032) (connection refused reported by osd.5) 2021-03-10T13:57:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.484221+0000 mon.a (mon.0) 589 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.484718+0000 mon.a (mon.0) 590 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.485158+0000 mon.a (mon.0) 591 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.485336+0000 mon.a (mon.0) 592 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.485852+0000 mon.a (mon.0) 593 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.486143+0000 mon.a (mon.0) 594 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.486656+0000 mon.a (mon.0) 595 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.486931+0000 mon.a (mon.0) 596 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.488262+0000 mon.a (mon.0) 597 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.488488+0000 mon.a (mon.0) 598 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.488934+0000 mon.a (mon.0) 599 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.489114+0000 mon.a (mon.0) 600 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.489873+0000 mon.a (mon.0) 601 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.884192+0000 mon.a (mon.0) 602 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.884523+0000 mon.a (mon.0) 603 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.885086+0000 mon.a (mon.0) 604 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.885362+0000 mon.a (mon.0) 605 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.885897+0000 mon.a (mon.0) 606 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.886190+0000 mon.a (mon.0) 607 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.888254+0000 mon.a (mon.0) 608 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.888465+0000 mon.a (mon.0) 609 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.889254+0000 mon.a (mon.0) 610 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.892202+0000 mon.a (mon.0) 611 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.892424+0000 mon.a (mon.0) 612 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.483279+0000 mon.a (mon.0) 587 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.483434+0000 mon.a (mon.0) 588 : cluster [INF] osd.7 failed (root=default,host=gibba032) (connection refused reported by osd.5) 2021-03-10T13:57:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.484221+0000 mon.a (mon.0) 589 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.484718+0000 mon.a (mon.0) 590 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.485158+0000 mon.a (mon.0) 591 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.485336+0000 mon.a (mon.0) 592 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.485852+0000 mon.a (mon.0) 593 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.486143+0000 mon.a (mon.0) 594 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.486656+0000 mon.a (mon.0) 595 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.486931+0000 mon.a (mon.0) 596 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.488262+0000 mon.a (mon.0) 597 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.488488+0000 mon.a (mon.0) 598 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.488934+0000 mon.a (mon.0) 599 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.489114+0000 mon.a (mon.0) 600 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.489873+0000 mon.a (mon.0) 601 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.884192+0000 mon.a (mon.0) 602 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.884523+0000 mon.a (mon.0) 603 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.885086+0000 mon.a (mon.0) 604 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.885362+0000 mon.a (mon.0) 605 : cluster [DBG] osd.7 reported immediately failed by osd.6 2021-03-10T13:57:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.885897+0000 mon.a (mon.0) 606 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.886190+0000 mon.a (mon.0) 607 : cluster [DBG] osd.7 reported immediately failed by osd.4 2021-03-10T13:57:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.888254+0000 mon.a (mon.0) 608 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.888465+0000 mon.a (mon.0) 609 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.889254+0000 mon.a (mon.0) 610 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.892202+0000 mon.a (mon.0) 611 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.892424+0000 mon.a (mon.0) 612 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.892879+0000 mon.a (mon.0) 613 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.893058+0000 mon.a (mon.0) 614 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.893634+0000 mon.a (mon.0) 615 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.021376+0000 mon.a (mon.0) 616 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.022834+0000 mon.a (mon.0) 617 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.023871+0000 mon.a (mon.0) 618 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.024616+0000 mon.a (mon.0) 619 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.892879+0000 mon.a (mon.0) 613 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.893058+0000 mon.a (mon.0) 614 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.893634+0000 mon.a (mon.0) 615 : cluster [DBG] osd.7 reported immediately failed by osd.3 2021-03-10T13:57:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.021376+0000 mon.a (mon.0) 616 : cluster [DBG] osd.7 reported immediately failed by osd.1 2021-03-10T13:57:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.022834+0000 mon.a (mon.0) 617 : cluster [DBG] osd.7 reported immediately failed by osd.5 2021-03-10T13:57:59.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.023871+0000 mon.a (mon.0) 618 : cluster [DBG] osd.7 reported immediately failed by osd.2 2021-03-10T13:57:59.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:57:59 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.024616+0000 mon.a (mon.0) 619 : cluster [DBG] osd.7 reported immediately failed by osd.0 2021-03-10T13:57:59.887 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:58:00.110 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:58:00.111 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (4m) 97s ago 10m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:58:00.111 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (9m) 21s ago 9m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:58:00.111 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (4m) 21s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:58:00.111 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (5m) 21s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:58:00.112 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (4m) 97s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:58:00.112 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (4m) 97s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:58:00.112 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3m) 21s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:58:00.112 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3m) 97s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:58:00.112 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (10m) 97s ago 10m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:58:00.113 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (10m) 21s ago 10m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:58:00.113 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2m) 97s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:58:00.113 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2m) 97s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:58:00.113 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2m) 97s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:58:00.113 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (99s) 97s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:58:00.114 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (74s) 21s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:58:00.114 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (48s) 21s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T13:58:00.114 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (22s) 21s ago 10m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T13:58:00.114 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (10m) 21s ago 10m 15.2.0 docker.io/ceph/ceph:v15.2.0 204a01f9b0b6 afb8f661d684 2021-03-10T13:58:00.115 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (9m) 21s ago 10m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:58:00.115 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 97s ago 8m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:58:00.368 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:58:00.368 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:58:00.368 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:58:00.369 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:00.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:58:00.369 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:58:00.369 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:00.370 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:58:00.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 7 2021-03-10T13:58:00.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:00.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:58:00.371 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:58:00.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 13 2021-03-10T13:58:00.371 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:58:00.371 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:58:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:00 gibba032 bash[24004]: cluster 2021-03-10T13:57:58.603089+0000 mgr.gibba032.whopeo (mgr.24625) 311 : cluster [DBG] pgmap v160: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:00 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.139778+0000 mon.a (mon.0) 620 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:58:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:00 gibba032 bash[24004]: cluster 2021-03-10T13:57:59.152066+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e97: 8 total, 7 up, 8 in 2021-03-10T13:58:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[30791]: cluster 2021-03-10T13:57:58.603089+0000 mgr.gibba032.whopeo (mgr.24625) 311 : cluster [DBG] pgmap v160: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.139778+0000 mon.a (mon.0) 620 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:58:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[30791]: cluster 2021-03-10T13:57:59.152066+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e97: 8 total, 7 up, 8 in 2021-03-10T13:58:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[31774]: cluster 2021-03-10T13:57:58.603089+0000 mgr.gibba032.whopeo (mgr.24625) 311 : cluster [DBG] pgmap v160: 129 pgs: 13 active+clean+wait, 116 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.139778+0000 mon.a (mon.0) 620 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2021-03-10T13:58:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:00 gibba011 bash[31774]: cluster 2021-03-10T13:57:59.152066+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e97: 8 total, 7 up, 8 in 2021-03-10T13:58:00.976 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:00 gibba032 systemd[1]: ceph-6fd36d94-81a6-11eb-9080-001a4aab830c@osd.7.service: Succeeded. 2021-03-10T13:58:00.976 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:00 gibba032 systemd[1]: Stopped Ceph osd.7 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:58:00.976 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:00 gibba032 systemd[1]: Started Ceph osd.7 for 6fd36d94-81a6-11eb-9080-001a4aab830c. 2021-03-10T13:58:01.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: audit 2021-03-10T13:57:59.878325+0000 mgr.gibba032.whopeo (mgr.24625) 312 : audit [DBG] from='client.54256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: audit 2021-03-10T13:58:00.101221+0000 mgr.gibba032.whopeo (mgr.24625) 313 : audit [DBG] from='client.44277 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: cluster 2021-03-10T13:58:00.153114+0000 mon.a (mon.0) 622 : cluster [DBG] osdmap e98: 8 total, 7 up, 8 in 2021-03-10T13:58:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: audit 2021-03-10T13:58:00.367346+0000 mon.a (mon.0) 623 : audit [DBG] from='client.? 172.21.2.111:0/2029379635' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: audit 2021-03-10T13:58:00.956792+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:01 gibba032 bash[24004]: audit 2021-03-10T13:58:00.957860+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:58:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: audit 2021-03-10T13:57:59.878325+0000 mgr.gibba032.whopeo (mgr.24625) 312 : audit [DBG] from='client.54256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: audit 2021-03-10T13:58:00.101221+0000 mgr.gibba032.whopeo (mgr.24625) 313 : audit [DBG] from='client.44277 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: cluster 2021-03-10T13:58:00.153114+0000 mon.a (mon.0) 622 : cluster [DBG] osdmap e98: 8 total, 7 up, 8 in 2021-03-10T13:58:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: audit 2021-03-10T13:58:00.367346+0000 mon.a (mon.0) 623 : audit [DBG] from='client.? 172.21.2.111:0/2029379635' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: audit 2021-03-10T13:58:00.956792+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[30791]: audit 2021-03-10T13:58:00.957860+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:58:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: audit 2021-03-10T13:57:59.878325+0000 mgr.gibba032.whopeo (mgr.24625) 312 : audit [DBG] from='client.54256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: audit 2021-03-10T13:58:00.101221+0000 mgr.gibba032.whopeo (mgr.24625) 313 : audit [DBG] from='client.44277 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: cluster 2021-03-10T13:58:00.153114+0000 mon.a (mon.0) 622 : cluster [DBG] osdmap e98: 8 total, 7 up, 8 in 2021-03-10T13:58:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: audit 2021-03-10T13:58:00.367346+0000 mon.a (mon.0) 623 : audit [DBG] from='client.? 172.21.2.111:0/2029379635' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: audit 2021-03-10T13:58:00.956792+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:01 gibba011 bash[31774]: audit 2021-03-10T13:58:00.957860+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T13:58:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[24004]: cluster 2021-03-10T13:58:00.604290+0000 mgr.gibba032.whopeo (mgr.24625) 314 : cluster [DBG] pgmap v163: 129 pgs: 12 stale+active+clean, 19 peering, 13 active+clean+wait, 85 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[24004]: cluster 2021-03-10T13:58:01.152919+0000 mon.a (mon.0) 626 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2021-03-10T13:58:02.507 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-03-10T13:58:02.507 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-7 --no-mon-config 2021-03-10T13:58:02.508 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-7/block 2021-03-10T13:58:02.508 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-7/block 2021-03-10T13:58:02.508 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2021-03-10T13:58:02.508 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-03-10T13:58:02.508 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:02 gibba032 bash[29190]: --> ceph-volume lvm activate successful for osd ID: 7 2021-03-10T13:58:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:02 gibba011 bash[30791]: cluster 2021-03-10T13:58:00.604290+0000 mgr.gibba032.whopeo (mgr.24625) 314 : cluster [DBG] pgmap v163: 129 pgs: 12 stale+active+clean, 19 peering, 13 active+clean+wait, 85 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:02 gibba011 bash[30791]: cluster 2021-03-10T13:58:01.152919+0000 mon.a (mon.0) 626 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2021-03-10T13:58:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:02 gibba011 bash[31774]: cluster 2021-03-10T13:58:00.604290+0000 mgr.gibba032.whopeo (mgr.24625) 314 : cluster [DBG] pgmap v163: 129 pgs: 12 stale+active+clean, 19 peering, 13 active+clean+wait, 85 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail 2021-03-10T13:58:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:02 gibba011 bash[31774]: cluster 2021-03-10T13:58:01.152919+0000 mon.a (mon.0) 626 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2021-03-10T13:58:03.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:03 gibba032 bash[24004]: cluster 2021-03-10T13:58:03.160004+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:03 gibba011 bash[30791]: cluster 2021-03-10T13:58:03.160004+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:03.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:03 gibba011 bash[31774]: cluster 2021-03-10T13:58:03.160004+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: Degraded data redundancy: 25/192 objects degraded (13.021%), 19 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:04.006 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:03 gibba032 bash[29316]: debug 2021-03-10T13:58:03.655+0000 7f61ec35c700 1 -- 172.21.2.132:0/2625688948 <== mon.2 v2:172.21.2.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556386f664e0 con 0x556386fe3400 2021-03-10T13:58:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:04 gibba032 bash[24004]: cluster 2021-03-10T13:58:02.606358+0000 mgr.gibba032.whopeo (mgr.24625) 315 : cluster [DBG] pgmap v164: 129 pgs: 19 active+undersized+degraded, 19 peering, 19 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:58:04.523 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:04 gibba032 bash[29316]: debug 2021-03-10T13:58:04.259+0000 7f61f3df1080 -1 Falling back to public interface 2021-03-10T13:58:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:04 gibba011 bash[30791]: cluster 2021-03-10T13:58:02.606358+0000 mgr.gibba032.whopeo (mgr.24625) 315 : cluster [DBG] pgmap v164: 129 pgs: 19 active+undersized+degraded, 19 peering, 19 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:58:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:04 gibba011 bash[31774]: cluster 2021-03-10T13:58:02.606358+0000 mgr.gibba032.whopeo (mgr.24625) 315 : cluster [DBG] pgmap v164: 129 pgs: 19 active+undersized+degraded, 19 peering, 19 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 25/192 objects degraded (13.021%) 2021-03-10T13:58:06.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cluster 2021-03-10T13:58:04.606927+0000 mgr.gibba032.whopeo (mgr.24625) 316 : cluster [DBG] pgmap v165: 129 pgs: 24 active+undersized+degraded, 9 peering, 24 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 30/192 objects degraded (15.625%) 2021-03-10T13:58:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.750407+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.755456+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.759018+0000 mgr.gibba032.whopeo (mgr.24625) 317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:58:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.760057+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:58:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.761224+0000 mgr.gibba032.whopeo (mgr.24625) 318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:58:06.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.761774+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.762561+0000 mgr.gibba032.whopeo (mgr.24625) 319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:58:06.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.762636+0000 mgr.gibba032.whopeo (mgr.24625) 320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:58:06.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.763203+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.764010+0000 mgr.gibba032.whopeo (mgr.24625) 321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:58:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.764141+0000 mgr.gibba032.whopeo (mgr.24625) 322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:58:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.764597+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.765373+0000 mgr.gibba032.whopeo (mgr.24625) 323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:58:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.766718+0000 mgr.gibba032.whopeo (mgr.24625) 324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:58:06.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.766791+0000 mgr.gibba032.whopeo (mgr.24625) 325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:58:06.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.767534+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.768357+0000 mgr.gibba032.whopeo (mgr.24625) 326 : cephadm [INF] Upgrade: Setting container_image for all osd 2021-03-10T13:58:06.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.772483+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.773694+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2021-03-10T13:58:06.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.777025+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2021-03-10T13:58:06.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.778273+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2021-03-10T13:58:06.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.782134+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2021-03-10T13:58:06.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.783676+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2021-03-10T13:58:06.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.787876+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2021-03-10T13:58:06.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.789722+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2021-03-10T13:58:06.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.793894+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2021-03-10T13:58:06.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.796329+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2021-03-10T13:58:06.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.803314+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2021-03-10T13:58:06.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.806054+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2021-03-10T13:58:06.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.814378+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2021-03-10T13:58:06.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.817028+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2021-03-10T13:58:06.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.820738+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2021-03-10T13:58:06.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.822927+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2021-03-10T13:58:06.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.831471+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2021-03-10T13:58:06.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.833188+0000 mgr.gibba032.whopeo (mgr.24625) 327 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:58:06.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: cephadm 2021-03-10T13:58:04.835342+0000 mgr.gibba032.whopeo (mgr.24625) 328 : cephadm [INF] Upgrade: Setting require_osd_release to 17 quincy 2021-03-10T13:58:06.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:05 gibba032 bash[24004]: audit 2021-03-10T13:58:04.836255+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd require-osd-release", "release": "quincy"}]: dispatch 2021-03-10T13:58:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cluster 2021-03-10T13:58:04.606927+0000 mgr.gibba032.whopeo (mgr.24625) 316 : cluster [DBG] pgmap v165: 129 pgs: 24 active+undersized+degraded, 9 peering, 24 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 30/192 objects degraded (15.625%) 2021-03-10T13:58:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.750407+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.755456+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.759018+0000 mgr.gibba032.whopeo (mgr.24625) 317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:58:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.760057+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:58:06.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.761224+0000 mgr.gibba032.whopeo (mgr.24625) 318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:58:06.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.761774+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.762561+0000 mgr.gibba032.whopeo (mgr.24625) 319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:58:06.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.762636+0000 mgr.gibba032.whopeo (mgr.24625) 320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:58:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.763203+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.764010+0000 mgr.gibba032.whopeo (mgr.24625) 321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:58:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.764141+0000 mgr.gibba032.whopeo (mgr.24625) 322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:58:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.764597+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.765373+0000 mgr.gibba032.whopeo (mgr.24625) 323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:58:06.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.766718+0000 mgr.gibba032.whopeo (mgr.24625) 324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:58:06.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.766791+0000 mgr.gibba032.whopeo (mgr.24625) 325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:58:06.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.767534+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.768357+0000 mgr.gibba032.whopeo (mgr.24625) 326 : cephadm [INF] Upgrade: Setting container_image for all osd 2021-03-10T13:58:06.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.772483+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cluster 2021-03-10T13:58:04.606927+0000 mgr.gibba032.whopeo (mgr.24625) 316 : cluster [DBG] pgmap v165: 129 pgs: 24 active+undersized+degraded, 9 peering, 24 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 30/192 objects degraded (15.625%) 2021-03-10T13:58:06.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.750407+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.755456+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.759018+0000 mgr.gibba032.whopeo (mgr.24625) 317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:58:06.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.760057+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:58:06.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.761224+0000 mgr.gibba032.whopeo (mgr.24625) 318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:58:06.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.761774+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.762561+0000 mgr.gibba032.whopeo (mgr.24625) 319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:58:06.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.762636+0000 mgr.gibba032.whopeo (mgr.24625) 320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:58:06.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.763203+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.764010+0000 mgr.gibba032.whopeo (mgr.24625) 321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:58:06.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.764141+0000 mgr.gibba032.whopeo (mgr.24625) 322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:58:06.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.764597+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.765373+0000 mgr.gibba032.whopeo (mgr.24625) 323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:58:06.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.766718+0000 mgr.gibba032.whopeo (mgr.24625) 324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:58:06.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.766791+0000 mgr.gibba032.whopeo (mgr.24625) 325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:58:06.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.767534+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:06.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.768357+0000 mgr.gibba032.whopeo (mgr.24625) 326 : cephadm [INF] Upgrade: Setting container_image for all osd 2021-03-10T13:58:06.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.772483+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:06.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.773694+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2021-03-10T13:58:06.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.777025+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2021-03-10T13:58:06.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.778273+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2021-03-10T13:58:06.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.782134+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2021-03-10T13:58:06.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.783676+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2021-03-10T13:58:06.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.787876+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2021-03-10T13:58:06.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.789722+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2021-03-10T13:58:06.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.793894+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2021-03-10T13:58:06.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.796329+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2021-03-10T13:58:06.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.803314+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2021-03-10T13:58:06.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.806054+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2021-03-10T13:58:06.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.814378+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2021-03-10T13:58:06.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.817028+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2021-03-10T13:58:06.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.820738+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2021-03-10T13:58:06.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.822927+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2021-03-10T13:58:06.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.831471+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2021-03-10T13:58:06.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.833188+0000 mgr.gibba032.whopeo (mgr.24625) 327 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:58:06.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: cephadm 2021-03-10T13:58:04.835342+0000 mgr.gibba032.whopeo (mgr.24625) 328 : cephadm [INF] Upgrade: Setting require_osd_release to 17 quincy 2021-03-10T13:58:06.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[31774]: audit 2021-03-10T13:58:04.836255+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd require-osd-release", "release": "quincy"}]: dispatch 2021-03-10T13:58:06.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.773694+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2021-03-10T13:58:06.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.777025+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2021-03-10T13:58:06.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.778273+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2021-03-10T13:58:06.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.782134+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2021-03-10T13:58:06.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.783676+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2021-03-10T13:58:06.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.787876+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2021-03-10T13:58:06.095 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.789722+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2021-03-10T13:58:06.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.793894+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2021-03-10T13:58:06.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.796329+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2021-03-10T13:58:06.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.803314+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2021-03-10T13:58:06.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.806054+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2021-03-10T13:58:06.096 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.814378+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2021-03-10T13:58:06.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.817028+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2021-03-10T13:58:06.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.820738+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2021-03-10T13:58:06.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.822927+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2021-03-10T13:58:06.097 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.831471+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2021-03-10T13:58:06.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.833188+0000 mgr.gibba032.whopeo (mgr.24625) 327 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:58:06.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: cephadm 2021-03-10T13:58:04.835342+0000 mgr.gibba032.whopeo (mgr.24625) 328 : cephadm [INF] Upgrade: Setting require_osd_release to 17 quincy 2021-03-10T13:58:06.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:05 gibba011 bash[30791]: audit 2021-03-10T13:58:04.836255+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd require-osd-release", "release": "quincy"}]: dispatch 2021-03-10T13:58:07.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: audit 2021-03-10T13:58:05.840023+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd require-osd-release", "release": "quincy"}]': finished 2021-03-10T13:58:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cluster 2021-03-10T13:58:05.840065+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e99: 8 total, 7 up, 8 in 2021-03-10T13:58:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cephadm 2021-03-10T13:58:05.841206+0000 mgr.gibba032.whopeo (mgr.24625) 329 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:58:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: audit 2021-03-10T13:58:05.841713+0000 mon.a (mon.0) 655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cephadm 2021-03-10T13:58:05.842310+0000 mgr.gibba032.whopeo (mgr.24625) 330 : cephadm [INF] Upgrade: Setting container_image for all mds 2021-03-10T13:58:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: audit 2021-03-10T13:58:05.845826+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cephadm 2021-03-10T13:58:05.846325+0000 mgr.gibba032.whopeo (mgr.24625) 331 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:58:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cephadm 2021-03-10T13:58:05.846368+0000 mgr.gibba032.whopeo (mgr.24625) 332 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:58:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:06 gibba032 bash[24004]: cephadm 2021-03-10T13:58:05.846499+0000 mgr.gibba032.whopeo (mgr.24625) 333 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: audit 2021-03-10T13:58:05.840023+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd require-osd-release", "release": "quincy"}]': finished 2021-03-10T13:58:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cluster 2021-03-10T13:58:05.840065+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e99: 8 total, 7 up, 8 in 2021-03-10T13:58:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cephadm 2021-03-10T13:58:05.841206+0000 mgr.gibba032.whopeo (mgr.24625) 329 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:58:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: audit 2021-03-10T13:58:05.841713+0000 mon.a (mon.0) 655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cephadm 2021-03-10T13:58:05.842310+0000 mgr.gibba032.whopeo (mgr.24625) 330 : cephadm [INF] Upgrade: Setting container_image for all mds 2021-03-10T13:58:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: audit 2021-03-10T13:58:05.845826+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cephadm 2021-03-10T13:58:05.846325+0000 mgr.gibba032.whopeo (mgr.24625) 331 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:58:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cephadm 2021-03-10T13:58:05.846368+0000 mgr.gibba032.whopeo (mgr.24625) 332 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:58:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[30791]: cephadm 2021-03-10T13:58:05.846499+0000 mgr.gibba032.whopeo (mgr.24625) 333 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: audit 2021-03-10T13:58:05.840023+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd='[{"prefix": "osd require-osd-release", "release": "quincy"}]': finished 2021-03-10T13:58:07.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cluster 2021-03-10T13:58:05.840065+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e99: 8 total, 7 up, 8 in 2021-03-10T13:58:07.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cephadm 2021-03-10T13:58:05.841206+0000 mgr.gibba032.whopeo (mgr.24625) 329 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:58:07.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: audit 2021-03-10T13:58:05.841713+0000 mon.a (mon.0) 655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:07.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cephadm 2021-03-10T13:58:05.842310+0000 mgr.gibba032.whopeo (mgr.24625) 330 : cephadm [INF] Upgrade: Setting container_image for all mds 2021-03-10T13:58:07.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: audit 2021-03-10T13:58:05.845826+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:07.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cephadm 2021-03-10T13:58:05.846325+0000 mgr.gibba032.whopeo (mgr.24625) 331 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:58:07.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cephadm 2021-03-10T13:58:05.846368+0000 mgr.gibba032.whopeo (mgr.24625) 332 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:58:07.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:06 gibba011 bash[31774]: cephadm 2021-03-10T13:58:05.846499+0000 mgr.gibba032.whopeo (mgr.24625) 333 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:07.846 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[29316]: debug 2021-03-10T13:58:07.547+0000 7f61f3df1080 -1 bluestore(/var/lib/ceph/osd/ceph-7) fsck warning: store not yet converted to per-pg omap 2021-03-10T13:58:07.846 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[29316]: debug 2021-03-10T13:58:07.547+0000 7f61dc8b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7) fsck warning: #-1:c1a3fc6e:::purged_snaps:0# has omap that is not per-pg or pgmeta 2021-03-10T13:58:07.847 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[29316]: debug 2021-03-10T13:58:07.547+0000 7f61dc8b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7) fsck warning: #1:d525e5cb:::INTEL_SSDPEL1K375GA_PHKM920000BW375A:head# has omap that is not per-pg or pgmeta 2021-03-10T13:58:07.847 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[29316]: debug 2021-03-10T13:58:07.671+0000 7f61f3df1080 -1 osd.7 96 log_to_monitors {default=true} 2021-03-10T13:58:08.256 INFO:journalctl@ceph.osd.7.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[29316]: debug 2021-03-10T13:58:07.859+0000 7f61e2acc700 -1 osd.7 96 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-03-10T13:58:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[24004]: cluster 2021-03-10T13:58:06.608688+0000 mgr.gibba032.whopeo (mgr.24625) 334 : cluster [DBG] pgmap v167: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[24004]: cluster 2021-03-10T13:58:06.844831+0000 mon.a (mon.0) 657 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:58:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[24004]: audit 2021-03-10T13:58:07.678521+0000 mon.b (mon.2) 9 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:07 gibba032 bash[24004]: audit 2021-03-10T13:58:07.679647+0000 mon.a (mon.0) 658 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:08.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[30791]: cluster 2021-03-10T13:58:06.608688+0000 mgr.gibba032.whopeo (mgr.24625) 334 : cluster [DBG] pgmap v167: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[30791]: cluster 2021-03-10T13:58:06.844831+0000 mon.a (mon.0) 657 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:58:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[30791]: audit 2021-03-10T13:58:07.678521+0000 mon.b (mon.2) 9 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[30791]: audit 2021-03-10T13:58:07.679647+0000 mon.a (mon.0) 658 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[31774]: cluster 2021-03-10T13:58:06.608688+0000 mgr.gibba032.whopeo (mgr.24625) 334 : cluster [DBG] pgmap v167: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 47 MiB used, 671 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[31774]: cluster 2021-03-10T13:58:06.844831+0000 mon.a (mon.0) 657 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2021-03-10T13:58:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[31774]: audit 2021-03-10T13:58:07.678521+0000 mon.b (mon.2) 9 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:07 gibba011 bash[31774]: audit 2021-03-10T13:58:07.679647+0000 mon.a (mon.0) 658 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2021-03-10T13:58:09.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:08 gibba032 bash[24004]: audit 2021-03-10T13:58:07.852855+0000 mon.a (mon.0) 659 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:58:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:08 gibba032 bash[24004]: cluster 2021-03-10T13:58:07.852915+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e100: 8 total, 7 up, 8 in 2021-03-10T13:58:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:08 gibba032 bash[24004]: audit 2021-03-10T13:58:07.853916+0000 mon.b (mon.2) 10 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:08 gibba032 bash[24004]: audit 2021-03-10T13:58:07.854386+0000 mon.a (mon.0) 661 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[30791]: audit 2021-03-10T13:58:07.852855+0000 mon.a (mon.0) 659 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:58:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[30791]: cluster 2021-03-10T13:58:07.852915+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e100: 8 total, 7 up, 8 in 2021-03-10T13:58:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[30791]: audit 2021-03-10T13:58:07.853916+0000 mon.b (mon.2) 10 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[30791]: audit 2021-03-10T13:58:07.854386+0000 mon.a (mon.0) 661 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[31774]: audit 2021-03-10T13:58:07.852855+0000 mon.a (mon.0) 659 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2021-03-10T13:58:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[31774]: cluster 2021-03-10T13:58:07.852915+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e100: 8 total, 7 up, 8 in 2021-03-10T13:58:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[31774]: audit 2021-03-10T13:58:07.853916+0000 mon.b (mon.2) 10 : audit [INF] from='osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:08 gibba011 bash[31774]: audit 2021-03-10T13:58:07.854386+0000 mon.a (mon.0) 661 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0820, "args": ["host=gibba032", "root=default"]}]: dispatch 2021-03-10T13:58:10.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: cluster 2021-03-10T13:58:08.610181+0000 mgr.gibba032.whopeo (mgr.24625) 335 : cluster [DBG] pgmap v169: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 52 MiB used, 672 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: cluster 2021-03-10T13:58:08.853018+0000 mon.a (mon.0) 662 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:58:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: cluster 2021-03-10T13:58:08.864428+0000 mon.a (mon.0) 663 : cluster [INF] osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632] boot 2021-03-10T13:58:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: cluster 2021-03-10T13:58:08.864475+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2021-03-10T13:58:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: audit 2021-03-10T13:58:08.865123+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:58:10.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:09 gibba032 bash[24004]: cluster 2021-03-10T13:58:09.868300+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2021-03-10T13:58:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: cluster 2021-03-10T13:58:08.610181+0000 mgr.gibba032.whopeo (mgr.24625) 335 : cluster [DBG] pgmap v169: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 52 MiB used, 672 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: cluster 2021-03-10T13:58:08.853018+0000 mon.a (mon.0) 662 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:58:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: cluster 2021-03-10T13:58:08.864428+0000 mon.a (mon.0) 663 : cluster [INF] osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632] boot 2021-03-10T13:58:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: cluster 2021-03-10T13:58:08.864475+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2021-03-10T13:58:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: audit 2021-03-10T13:58:08.865123+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:58:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[31774]: cluster 2021-03-10T13:58:09.868300+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2021-03-10T13:58:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: cluster 2021-03-10T13:58:08.610181+0000 mgr.gibba032.whopeo (mgr.24625) 335 : cluster [DBG] pgmap v169: 129 pgs: 27 active+undersized+degraded, 30 active+undersized, 72 active+clean; 7.0 KiB data, 52 MiB used, 672 GiB / 672 GiB avail; 34/192 objects degraded (17.708%) 2021-03-10T13:58:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: cluster 2021-03-10T13:58:08.853018+0000 mon.a (mon.0) 662 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2021-03-10T13:58:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: cluster 2021-03-10T13:58:08.864428+0000 mon.a (mon.0) 663 : cluster [INF] osd.7 [v2:172.21.2.132:6824/4206927632,v1:172.21.2.132:6825/4206927632] boot 2021-03-10T13:58:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: cluster 2021-03-10T13:58:08.864475+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2021-03-10T13:58:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: audit 2021-03-10T13:58:08.865123+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2021-03-10T13:58:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:09 gibba011 bash[30791]: cluster 2021-03-10T13:58:09.868300+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2021-03-10T13:58:11.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:10 gibba032 bash[24004]: cluster 2021-03-10T13:58:09.927631+0000 mon.a (mon.0) 667 : cluster [WRN] Health check update: Degraded data redundancy: 34/192 objects degraded (17.708%), 27 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:10 gibba011 bash[30791]: cluster 2021-03-10T13:58:09.927631+0000 mon.a (mon.0) 667 : cluster [WRN] Health check update: Degraded data redundancy: 34/192 objects degraded (17.708%), 27 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:10 gibba011 bash[31774]: cluster 2021-03-10T13:58:09.927631+0000 mon.a (mon.0) 667 : cluster [WRN] Health check update: Degraded data redundancy: 34/192 objects degraded (17.708%), 27 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:12.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:11 gibba032 bash[24004]: cluster 2021-03-10T13:58:10.611737+0000 mgr.gibba032.whopeo (mgr.24625) 336 : cluster [DBG] pgmap v172: 129 pgs: 7 peering, 20 active+undersized+degraded, 23 active+undersized, 79 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 26/192 objects degraded (13.542%) 2021-03-10T13:58:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:11 gibba011 bash[30791]: cluster 2021-03-10T13:58:10.611737+0000 mgr.gibba032.whopeo (mgr.24625) 336 : cluster [DBG] pgmap v172: 129 pgs: 7 peering, 20 active+undersized+degraded, 23 active+undersized, 79 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 26/192 objects degraded (13.542%) 2021-03-10T13:58:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:11 gibba011 bash[31774]: cluster 2021-03-10T13:58:10.611737+0000 mgr.gibba032.whopeo (mgr.24625) 336 : cluster [DBG] pgmap v172: 129 pgs: 7 peering, 20 active+undersized+degraded, 23 active+undersized, 79 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 26/192 objects degraded (13.542%) 2021-03-10T13:58:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:13 gibba032 bash[24004]: cluster 2021-03-10T13:58:12.613693+0000 mgr.gibba032.whopeo (mgr.24625) 337 : cluster [DBG] pgmap v173: 129 pgs: 7 peering, 9 active+undersized+degraded, 8 active+undersized, 105 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:58:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:13 gibba011 bash[30791]: cluster 2021-03-10T13:58:12.613693+0000 mgr.gibba032.whopeo (mgr.24625) 337 : cluster [DBG] pgmap v173: 129 pgs: 7 peering, 9 active+undersized+degraded, 8 active+undersized, 105 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:58:14.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:13 gibba011 bash[31774]: cluster 2021-03-10T13:58:12.613693+0000 mgr.gibba032.whopeo (mgr.24625) 337 : cluster [DBG] pgmap v173: 129 pgs: 7 peering, 9 active+undersized+degraded, 8 active+undersized, 105 active+clean; 7.0 KiB data, 53 MiB used, 672 GiB / 672 GiB avail; 11/192 objects degraded (5.729%) 2021-03-10T13:58:15.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:14 gibba032 bash[24004]: cluster 2021-03-10T13:58:14.929066+0000 mon.a (mon.0) 668 : cluster [WRN] Health check update: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:14 gibba011 bash[30791]: cluster 2021-03-10T13:58:14.929066+0000 mon.a (mon.0) 668 : cluster [WRN] Health check update: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:14 gibba011 bash[31774]: cluster 2021-03-10T13:58:14.929066+0000 mon.a (mon.0) 668 : cluster [WRN] Health check update: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded (PG_DEGRADED) 2021-03-10T13:58:16.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:15 gibba032 bash[24004]: cluster 2021-03-10T13:58:14.615076+0000 mgr.gibba032.whopeo (mgr.24625) 338 : cluster [DBG] pgmap v174: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:15 gibba032 bash[24004]: cluster 2021-03-10T13:58:14.948977+0000 mon.a (mon.0) 669 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded) 2021-03-10T13:58:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:15 gibba011 bash[30791]: cluster 2021-03-10T13:58:14.615076+0000 mgr.gibba032.whopeo (mgr.24625) 338 : cluster [DBG] pgmap v174: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:15 gibba011 bash[30791]: cluster 2021-03-10T13:58:14.948977+0000 mon.a (mon.0) 669 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded) 2021-03-10T13:58:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:15 gibba011 bash[31774]: cluster 2021-03-10T13:58:14.615076+0000 mgr.gibba032.whopeo (mgr.24625) 338 : cluster [DBG] pgmap v174: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:15 gibba011 bash[31774]: cluster 2021-03-10T13:58:14.948977+0000 mon.a (mon.0) 669 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/192 objects degraded (5.729%), 9 pgs degraded) 2021-03-10T13:58:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:17 gibba011 bash[30791]: audit 2021-03-10T13:58:16.043157+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:17 gibba011 bash[31774]: audit 2021-03-10T13:58:16.043157+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:17 gibba032 bash[24004]: audit 2021-03-10T13:58:16.043157+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:58:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:18 gibba011 bash[30791]: cluster 2021-03-10T13:58:16.616772+0000 mgr.gibba032.whopeo (mgr.24625) 339 : cluster [DBG] pgmap v175: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:18 gibba011 bash[31774]: cluster 2021-03-10T13:58:16.616772+0000 mgr.gibba032.whopeo (mgr.24625) 339 : cluster [DBG] pgmap v175: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:18 gibba032 bash[24004]: cluster 2021-03-10T13:58:16.616772+0000 mgr.gibba032.whopeo (mgr.24625) 339 : cluster [DBG] pgmap v175: 129 pgs: 17 active+clean+wait, 112 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:20 gibba011 bash[30791]: cluster 2021-03-10T13:58:18.618392+0000 mgr.gibba032.whopeo (mgr.24625) 340 : cluster [DBG] pgmap v176: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:20.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:20 gibba011 bash[31774]: cluster 2021-03-10T13:58:18.618392+0000 mgr.gibba032.whopeo (mgr.24625) 340 : cluster [DBG] pgmap v176: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:20 gibba032 bash[24004]: cluster 2021-03-10T13:58:18.618392+0000 mgr.gibba032.whopeo (mgr.24625) 340 : cluster [DBG] pgmap v176: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:22 gibba011 bash[30791]: cluster 2021-03-10T13:58:20.619714+0000 mgr.gibba032.whopeo (mgr.24625) 341 : cluster [DBG] pgmap v177: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:22 gibba011 bash[30791]: cephadm 2021-03-10T13:58:20.862227+0000 mgr.gibba032.whopeo (mgr.24625) 342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:22 gibba011 bash[31774]: cluster 2021-03-10T13:58:20.619714+0000 mgr.gibba032.whopeo (mgr.24625) 341 : cluster [DBG] pgmap v177: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:22 gibba011 bash[31774]: cephadm 2021-03-10T13:58:20.862227+0000 mgr.gibba032.whopeo (mgr.24625) 342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:22 gibba032 bash[24004]: cluster 2021-03-10T13:58:20.619714+0000 mgr.gibba032.whopeo (mgr.24625) 341 : cluster [DBG] pgmap v177: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:22 gibba032 bash[24004]: cephadm 2021-03-10T13:58:20.862227+0000 mgr.gibba032.whopeo (mgr.24625) 342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:24 gibba032 bash[24004]: cluster 2021-03-10T13:58:22.621234+0000 mgr.gibba032.whopeo (mgr.24625) 343 : cluster [DBG] pgmap v178: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:24 gibba011 bash[30791]: cluster 2021-03-10T13:58:22.621234+0000 mgr.gibba032.whopeo (mgr.24625) 343 : cluster [DBG] pgmap v178: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:24 gibba011 bash[31774]: cluster 2021-03-10T13:58:22.621234+0000 mgr.gibba032.whopeo (mgr.24625) 343 : cluster [DBG] pgmap v178: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:26 gibba032 bash[24004]: cluster 2021-03-10T13:58:24.622594+0000 mgr.gibba032.whopeo (mgr.24625) 344 : cluster [DBG] pgmap v179: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:26 gibba011 bash[30791]: cluster 2021-03-10T13:58:24.622594+0000 mgr.gibba032.whopeo (mgr.24625) 344 : cluster [DBG] pgmap v179: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:26 gibba011 bash[31774]: cluster 2021-03-10T13:58:24.622594+0000 mgr.gibba032.whopeo (mgr.24625) 344 : cluster [DBG] pgmap v179: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:28 gibba032 bash[24004]: cluster 2021-03-10T13:58:26.624214+0000 mgr.gibba032.whopeo (mgr.24625) 345 : cluster [DBG] pgmap v180: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:28 gibba011 bash[30791]: cluster 2021-03-10T13:58:26.624214+0000 mgr.gibba032.whopeo (mgr.24625) 345 : cluster [DBG] pgmap v180: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:28 gibba011 bash[31774]: cluster 2021-03-10T13:58:26.624214+0000 mgr.gibba032.whopeo (mgr.24625) 345 : cluster [DBG] pgmap v180: 129 pgs: 10 active+clean+wait, 119 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:30 gibba032 bash[24004]: cluster 2021-03-10T13:58:28.625524+0000 mgr.gibba032.whopeo (mgr.24625) 346 : cluster [DBG] pgmap v181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:30 gibba011 bash[30791]: cluster 2021-03-10T13:58:28.625524+0000 mgr.gibba032.whopeo (mgr.24625) 346 : cluster [DBG] pgmap v181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:30 gibba011 bash[31774]: cluster 2021-03-10T13:58:28.625524+0000 mgr.gibba032.whopeo (mgr.24625) 346 : cluster [DBG] pgmap v181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:30.617 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:58:30.844 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:58:30.844 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (5m) 2m ago 10m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:58:30.844 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (10m) 26s ago 10m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (5m) 26s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (5m) 26s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (4m) 2m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (4m) 2m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (4m) 26s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:58:30.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (4m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:58:30.846 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (10m) 2m ago 10m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:58:30.846 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (10m) 26s ago 10m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:58:30.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:58:30.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:58:30.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2m) 2m ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:58:30.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2m) 2m ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:58:30.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (105s) 26s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:58:30.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (79s) 26s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T13:58:30.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (53s) 26s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T13:58:30.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (28s) 26s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T13:58:30.848 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (10m) 26s ago 10m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:58:30.848 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 8m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:58:31.097 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:58:31.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:58:31.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:58:31.098 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:31.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:58:31.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:58:31.099 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:31.099 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:58:31.099 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T13:58:31.099 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:58:31.099 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:58:31.100 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:58:31.100 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T13:58:31.100 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:58:31.100 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:58:31.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:31 gibba032 bash[24004]: audit 2021-03-10T13:58:31.096950+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1006261341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:31.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:31 gibba011 bash[30791]: audit 2021-03-10T13:58:31.096950+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1006261341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:31 gibba011 bash[31774]: audit 2021-03-10T13:58:31.096950+0000 mon.c (mon.1) 10 : audit [DBG] from='client.? 172.21.2.111:0/1006261341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:58:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:32 gibba032 bash[24004]: audit 2021-03-10T13:58:30.607752+0000 mgr.gibba032.whopeo (mgr.24625) 347 : audit [DBG] from='client.54274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:32 gibba032 bash[24004]: cluster 2021-03-10T13:58:30.627014+0000 mgr.gibba032.whopeo (mgr.24625) 348 : cluster [DBG] pgmap v182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:32 gibba032 bash[24004]: audit 2021-03-10T13:58:30.830526+0000 mgr.gibba032.whopeo (mgr.24625) 349 : audit [DBG] from='client.64139 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[30791]: audit 2021-03-10T13:58:30.607752+0000 mgr.gibba032.whopeo (mgr.24625) 347 : audit [DBG] from='client.54274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[30791]: cluster 2021-03-10T13:58:30.627014+0000 mgr.gibba032.whopeo (mgr.24625) 348 : cluster [DBG] pgmap v182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[30791]: audit 2021-03-10T13:58:30.830526+0000 mgr.gibba032.whopeo (mgr.24625) 349 : audit [DBG] from='client.64139 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[31774]: audit 2021-03-10T13:58:30.607752+0000 mgr.gibba032.whopeo (mgr.24625) 347 : audit [DBG] from='client.54274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[31774]: cluster 2021-03-10T13:58:30.627014+0000 mgr.gibba032.whopeo (mgr.24625) 348 : cluster [DBG] pgmap v182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:32 gibba011 bash[31774]: audit 2021-03-10T13:58:30.830526+0000 mgr.gibba032.whopeo (mgr.24625) 349 : audit [DBG] from='client.64139 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:58:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:34 gibba032 bash[24004]: cluster 2021-03-10T13:58:32.628673+0000 mgr.gibba032.whopeo (mgr.24625) 350 : cluster [DBG] pgmap v183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:34 gibba011 bash[30791]: cluster 2021-03-10T13:58:32.628673+0000 mgr.gibba032.whopeo (mgr.24625) 350 : cluster [DBG] pgmap v183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:34 gibba011 bash[31774]: cluster 2021-03-10T13:58:32.628673+0000 mgr.gibba032.whopeo (mgr.24625) 350 : cluster [DBG] pgmap v183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:36.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:36 gibba032 bash[24004]: cluster 2021-03-10T13:58:34.629912+0000 mgr.gibba032.whopeo (mgr.24625) 351 : cluster [DBG] pgmap v184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:36 gibba011 bash[30791]: cluster 2021-03-10T13:58:34.629912+0000 mgr.gibba032.whopeo (mgr.24625) 351 : cluster [DBG] pgmap v184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:36 gibba011 bash[31774]: cluster 2021-03-10T13:58:34.629912+0000 mgr.gibba032.whopeo (mgr.24625) 351 : cluster [DBG] pgmap v184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:37 gibba032 bash[24004]: cephadm 2021-03-10T13:58:35.876940+0000 mgr.gibba032.whopeo (mgr.24625) 352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:37.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:37 gibba011 bash[30791]: cephadm 2021-03-10T13:58:35.876940+0000 mgr.gibba032.whopeo (mgr.24625) 352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:37 gibba011 bash[31774]: cephadm 2021-03-10T13:58:35.876940+0000 mgr.gibba032.whopeo (mgr.24625) 352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:38 gibba032 bash[24004]: cluster 2021-03-10T13:58:36.631602+0000 mgr.gibba032.whopeo (mgr.24625) 353 : cluster [DBG] pgmap v185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:38 gibba011 bash[30791]: cluster 2021-03-10T13:58:36.631602+0000 mgr.gibba032.whopeo (mgr.24625) 353 : cluster [DBG] pgmap v185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:38 gibba011 bash[31774]: cluster 2021-03-10T13:58:36.631602+0000 mgr.gibba032.whopeo (mgr.24625) 353 : cluster [DBG] pgmap v185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:40 gibba032 bash[24004]: cluster 2021-03-10T13:58:38.633296+0000 mgr.gibba032.whopeo (mgr.24625) 354 : cluster [DBG] pgmap v186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:40 gibba011 bash[30791]: cluster 2021-03-10T13:58:38.633296+0000 mgr.gibba032.whopeo (mgr.24625) 354 : cluster [DBG] pgmap v186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:40 gibba011 bash[31774]: cluster 2021-03-10T13:58:38.633296+0000 mgr.gibba032.whopeo (mgr.24625) 354 : cluster [DBG] pgmap v186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:42 gibba032 bash[24004]: cluster 2021-03-10T13:58:40.634752+0000 mgr.gibba032.whopeo (mgr.24625) 355 : cluster [DBG] pgmap v187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:42 gibba011 bash[30791]: cluster 2021-03-10T13:58:40.634752+0000 mgr.gibba032.whopeo (mgr.24625) 355 : cluster [DBG] pgmap v187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:42 gibba011 bash[31774]: cluster 2021-03-10T13:58:40.634752+0000 mgr.gibba032.whopeo (mgr.24625) 355 : cluster [DBG] pgmap v187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:44 gibba032 bash[24004]: cluster 2021-03-10T13:58:42.636347+0000 mgr.gibba032.whopeo (mgr.24625) 356 : cluster [DBG] pgmap v188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:44 gibba011 bash[30791]: cluster 2021-03-10T13:58:42.636347+0000 mgr.gibba032.whopeo (mgr.24625) 356 : cluster [DBG] pgmap v188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:44 gibba011 bash[31774]: cluster 2021-03-10T13:58:42.636347+0000 mgr.gibba032.whopeo (mgr.24625) 356 : cluster [DBG] pgmap v188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:46 gibba032 bash[24004]: cluster 2021-03-10T13:58:44.637622+0000 mgr.gibba032.whopeo (mgr.24625) 357 : cluster [DBG] pgmap v189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:46 gibba011 bash[30791]: cluster 2021-03-10T13:58:44.637622+0000 mgr.gibba032.whopeo (mgr.24625) 357 : cluster [DBG] pgmap v189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:46 gibba011 bash[31774]: cluster 2021-03-10T13:58:44.637622+0000 mgr.gibba032.whopeo (mgr.24625) 357 : cluster [DBG] pgmap v189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:48 gibba032 bash[24004]: cluster 2021-03-10T13:58:46.639328+0000 mgr.gibba032.whopeo (mgr.24625) 358 : cluster [DBG] pgmap v190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:48 gibba011 bash[30791]: cluster 2021-03-10T13:58:46.639328+0000 mgr.gibba032.whopeo (mgr.24625) 358 : cluster [DBG] pgmap v190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:48 gibba011 bash[31774]: cluster 2021-03-10T13:58:46.639328+0000 mgr.gibba032.whopeo (mgr.24625) 358 : cluster [DBG] pgmap v190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:50 gibba032 bash[24004]: cluster 2021-03-10T13:58:48.640853+0000 mgr.gibba032.whopeo (mgr.24625) 359 : cluster [DBG] pgmap v191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:50 gibba011 bash[30791]: cluster 2021-03-10T13:58:48.640853+0000 mgr.gibba032.whopeo (mgr.24625) 359 : cluster [DBG] pgmap v191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:50 gibba011 bash[31774]: cluster 2021-03-10T13:58:48.640853+0000 mgr.gibba032.whopeo (mgr.24625) 359 : cluster [DBG] pgmap v191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:52 gibba032 bash[24004]: cluster 2021-03-10T13:58:50.642214+0000 mgr.gibba032.whopeo (mgr.24625) 360 : cluster [DBG] pgmap v192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:52 gibba032 bash[24004]: cephadm 2021-03-10T13:58:50.888789+0000 mgr.gibba032.whopeo (mgr.24625) 361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:52 gibba011 bash[30791]: cluster 2021-03-10T13:58:50.642214+0000 mgr.gibba032.whopeo (mgr.24625) 360 : cluster [DBG] pgmap v192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:52 gibba011 bash[30791]: cephadm 2021-03-10T13:58:50.888789+0000 mgr.gibba032.whopeo (mgr.24625) 361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:52 gibba011 bash[31774]: cluster 2021-03-10T13:58:50.642214+0000 mgr.gibba032.whopeo (mgr.24625) 360 : cluster [DBG] pgmap v192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:52 gibba011 bash[31774]: cephadm 2021-03-10T13:58:50.888789+0000 mgr.gibba032.whopeo (mgr.24625) 361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:58:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:54 gibba032 bash[24004]: cluster 2021-03-10T13:58:52.643887+0000 mgr.gibba032.whopeo (mgr.24625) 362 : cluster [DBG] pgmap v193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:54 gibba011 bash[30791]: cluster 2021-03-10T13:58:52.643887+0000 mgr.gibba032.whopeo (mgr.24625) 362 : cluster [DBG] pgmap v193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:54 gibba011 bash[31774]: cluster 2021-03-10T13:58:52.643887+0000 mgr.gibba032.whopeo (mgr.24625) 362 : cluster [DBG] pgmap v193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: cluster 2021-03-10T13:58:54.645411+0000 mgr.gibba032.whopeo (mgr.24625) 363 : cluster [DBG] pgmap v194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:55.890575+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T13:58:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:55.891464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T13:58:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:55.892211+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:55.892954+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:56.005120+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:58:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:56 gibba032 bash[24004]: audit 2021-03-10T13:58:56.005752+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:58:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: cluster 2021-03-10T13:58:54.645411+0000 mgr.gibba032.whopeo (mgr.24625) 363 : cluster [DBG] pgmap v194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:55.890575+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T13:58:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:55.891464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T13:58:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:55.892211+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:55.892954+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:56.005120+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:58:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[30791]: audit 2021-03-10T13:58:56.005752+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:58:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: cluster 2021-03-10T13:58:54.645411+0000 mgr.gibba032.whopeo (mgr.24625) 363 : cluster [DBG] pgmap v194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:55.890575+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T13:58:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:55.891464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T13:58:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:55.892211+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:55.892954+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T13:58:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:56.005120+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:58:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:56 gibba011 bash[31774]: audit 2021-03-10T13:58:56.005752+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:58:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:58:58 gibba032 bash[24004]: cluster 2021-03-10T13:58:56.647136+0000 mgr.gibba032.whopeo (mgr.24625) 364 : cluster [DBG] pgmap v195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:58:58 gibba011 bash[30791]: cluster 2021-03-10T13:58:56.647136+0000 mgr.gibba032.whopeo (mgr.24625) 364 : cluster [DBG] pgmap v195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:58:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:58:58 gibba011 bash[31774]: cluster 2021-03-10T13:58:56.647136+0000 mgr.gibba032.whopeo (mgr.24625) 364 : cluster [DBG] pgmap v195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:00 gibba011 bash[30791]: cluster 2021-03-10T13:58:58.648732+0000 mgr.gibba032.whopeo (mgr.24625) 365 : cluster [DBG] pgmap v196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:00 gibba011 bash[31774]: cluster 2021-03-10T13:58:58.648732+0000 mgr.gibba032.whopeo (mgr.24625) 365 : cluster [DBG] pgmap v196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:00 gibba032 bash[24004]: cluster 2021-03-10T13:58:58.648732+0000 mgr.gibba032.whopeo (mgr.24625) 365 : cluster [DBG] pgmap v196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:01.343 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:59:01.572 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:59:01.572 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (5m) 2m ago 11m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:59:01.572 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (10m) 56s ago 10m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:59:01.573 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (6m) 56s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:59:01.573 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (6m) 56s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:59:01.573 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (5m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:59:01.573 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (5m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:59:01.573 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (4m) 56s ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:59:01.574 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (4m) 2m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:59:01.574 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (11m) 2m ago 11m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:59:01.574 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (11m) 56s ago 11m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:59:01.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:59:01.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:59:01.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:59:01.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:59:01.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2m) 56s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:59:01.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (110s) 56s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T13:59:01.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (84s) 56s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T13:59:01.576 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (58s) 56s ago 11m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T13:59:01.576 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (10m) 56s ago 11m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:59:01.576 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 9m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:59:01.819 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:59:01.819 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:59:01.820 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:59:01.820 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:01.820 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:59:01.820 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:59:01.821 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:01.821 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:59:01.821 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T13:59:01.821 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:01.821 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:59:01.822 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:59:01.822 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T13:59:01.822 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:59:01.822 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:59:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[30791]: cluster 2021-03-10T13:59:00.650030+0000 mgr.gibba032.whopeo (mgr.24625) 366 : cluster [DBG] pgmap v197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[30791]: audit 2021-03-10T13:59:01.333359+0000 mgr.gibba032.whopeo (mgr.24625) 367 : audit [DBG] from='client.44307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[30791]: audit 2021-03-10T13:59:01.818688+0000 mon.a (mon.0) 677 : audit [DBG] from='client.? 172.21.2.111:0/1951132316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[31774]: cluster 2021-03-10T13:59:00.650030+0000 mgr.gibba032.whopeo (mgr.24625) 366 : cluster [DBG] pgmap v197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[31774]: audit 2021-03-10T13:59:01.333359+0000 mgr.gibba032.whopeo (mgr.24625) 367 : audit [DBG] from='client.44307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:02 gibba011 bash[31774]: audit 2021-03-10T13:59:01.818688+0000 mon.a (mon.0) 677 : audit [DBG] from='client.? 172.21.2.111:0/1951132316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:02 gibba032 bash[24004]: cluster 2021-03-10T13:59:00.650030+0000 mgr.gibba032.whopeo (mgr.24625) 366 : cluster [DBG] pgmap v197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:02 gibba032 bash[24004]: audit 2021-03-10T13:59:01.333359+0000 mgr.gibba032.whopeo (mgr.24625) 367 : audit [DBG] from='client.44307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:02 gibba032 bash[24004]: audit 2021-03-10T13:59:01.818688+0000 mon.a (mon.0) 677 : audit [DBG] from='client.? 172.21.2.111:0/1951132316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:03 gibba011 bash[30791]: audit 2021-03-10T13:59:01.558823+0000 mgr.gibba032.whopeo (mgr.24625) 368 : audit [DBG] from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:03.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:03 gibba011 bash[31774]: audit 2021-03-10T13:59:01.558823+0000 mgr.gibba032.whopeo (mgr.24625) 368 : audit [DBG] from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:03 gibba032 bash[24004]: audit 2021-03-10T13:59:01.558823+0000 mgr.gibba032.whopeo (mgr.24625) 368 : audit [DBG] from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:04 gibba011 bash[30791]: cluster 2021-03-10T13:59:02.651750+0000 mgr.gibba032.whopeo (mgr.24625) 369 : cluster [DBG] pgmap v198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:04 gibba011 bash[31774]: cluster 2021-03-10T13:59:02.651750+0000 mgr.gibba032.whopeo (mgr.24625) 369 : cluster [DBG] pgmap v198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:04 gibba032 bash[24004]: cluster 2021-03-10T13:59:02.651750+0000 mgr.gibba032.whopeo (mgr.24625) 369 : cluster [DBG] pgmap v198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: cluster 2021-03-10T13:59:04.653261+0000 mgr.gibba032.whopeo (mgr.24625) 370 : cluster [DBG] pgmap v199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.120029+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:59:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.123285+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:59:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.124383+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.125431+0000 mon.a (mon.0) 681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.126719+0000 mon.a (mon.0) 682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.128772+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:06 gibba032 bash[24004]: audit 2021-03-10T13:59:06.129981+0000 mon.a (mon.0) 684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: cluster 2021-03-10T13:59:04.653261+0000 mgr.gibba032.whopeo (mgr.24625) 370 : cluster [DBG] pgmap v199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.120029+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:59:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.123285+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:59:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.124383+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.125431+0000 mon.a (mon.0) 681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.126719+0000 mon.a (mon.0) 682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.128772+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[30791]: audit 2021-03-10T13:59:06.129981+0000 mon.a (mon.0) 684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: cluster 2021-03-10T13:59:04.653261+0000 mgr.gibba032.whopeo (mgr.24625) 370 : cluster [DBG] pgmap v199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.120029+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T13:59:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.123285+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T13:59:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.124383+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.125431+0000 mon.a (mon.0) 681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.126719+0000 mon.a (mon.0) 682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.128772+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:06.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:06 gibba011 bash[31774]: audit 2021-03-10T13:59:06.129981+0000 mon.a (mon.0) 684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:05.904443+0000 mgr.gibba032.whopeo (mgr.24625) 371 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:59:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:05.904906+0000 mgr.gibba032.whopeo (mgr.24625) 372 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:59:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.122388+0000 mgr.gibba032.whopeo (mgr.24625) 373 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:59:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.123827+0000 mgr.gibba032.whopeo (mgr.24625) 374 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:59:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.124811+0000 mgr.gibba032.whopeo (mgr.24625) 375 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:59:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.124845+0000 mgr.gibba032.whopeo (mgr.24625) 376 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:59:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.126101+0000 mgr.gibba032.whopeo (mgr.24625) 377 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:59:07.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.126137+0000 mgr.gibba032.whopeo (mgr.24625) 378 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:59:07.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.127139+0000 mgr.gibba032.whopeo (mgr.24625) 379 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:59:07.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.127992+0000 mgr.gibba032.whopeo (mgr.24625) 380 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:59:07.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.128081+0000 mgr.gibba032.whopeo (mgr.24625) 381 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:59:07.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.129204+0000 mgr.gibba032.whopeo (mgr.24625) 382 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:59:07.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.129512+0000 mgr.gibba032.whopeo (mgr.24625) 383 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:59:07.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.130484+0000 mgr.gibba032.whopeo (mgr.24625) 384 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:59:07.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.130629+0000 mgr.gibba032.whopeo (mgr.24625) 385 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:59:07.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[30791]: cephadm 2021-03-10T13:59:06.130735+0000 mgr.gibba032.whopeo (mgr.24625) 386 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:07.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:05.904443+0000 mgr.gibba032.whopeo (mgr.24625) 371 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:59:07.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:05.904906+0000 mgr.gibba032.whopeo (mgr.24625) 372 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:59:07.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.122388+0000 mgr.gibba032.whopeo (mgr.24625) 373 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:59:07.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.123827+0000 mgr.gibba032.whopeo (mgr.24625) 374 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:59:07.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.124811+0000 mgr.gibba032.whopeo (mgr.24625) 375 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:59:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.124845+0000 mgr.gibba032.whopeo (mgr.24625) 376 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:59:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.126101+0000 mgr.gibba032.whopeo (mgr.24625) 377 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:59:07.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.126137+0000 mgr.gibba032.whopeo (mgr.24625) 378 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:59:07.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.127139+0000 mgr.gibba032.whopeo (mgr.24625) 379 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:59:07.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.127992+0000 mgr.gibba032.whopeo (mgr.24625) 380 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:59:07.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.128081+0000 mgr.gibba032.whopeo (mgr.24625) 381 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:59:07.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.129204+0000 mgr.gibba032.whopeo (mgr.24625) 382 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:59:07.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.129512+0000 mgr.gibba032.whopeo (mgr.24625) 383 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:59:07.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.130484+0000 mgr.gibba032.whopeo (mgr.24625) 384 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:59:07.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.130629+0000 mgr.gibba032.whopeo (mgr.24625) 385 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:59:07.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:07 gibba011 bash[31774]: cephadm 2021-03-10T13:59:06.130735+0000 mgr.gibba032.whopeo (mgr.24625) 386 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:05.904443+0000 mgr.gibba032.whopeo (mgr.24625) 371 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T13:59:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:05.904906+0000 mgr.gibba032.whopeo (mgr.24625) 372 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T13:59:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.122388+0000 mgr.gibba032.whopeo (mgr.24625) 373 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T13:59:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.123827+0000 mgr.gibba032.whopeo (mgr.24625) 374 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T13:59:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.124811+0000 mgr.gibba032.whopeo (mgr.24625) 375 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T13:59:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.124845+0000 mgr.gibba032.whopeo (mgr.24625) 376 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T13:59:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.126101+0000 mgr.gibba032.whopeo (mgr.24625) 377 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T13:59:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.126137+0000 mgr.gibba032.whopeo (mgr.24625) 378 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T13:59:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.127139+0000 mgr.gibba032.whopeo (mgr.24625) 379 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T13:59:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.127992+0000 mgr.gibba032.whopeo (mgr.24625) 380 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T13:59:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.128081+0000 mgr.gibba032.whopeo (mgr.24625) 381 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T13:59:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.129204+0000 mgr.gibba032.whopeo (mgr.24625) 382 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T13:59:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.129512+0000 mgr.gibba032.whopeo (mgr.24625) 383 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T13:59:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.130484+0000 mgr.gibba032.whopeo (mgr.24625) 384 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T13:59:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.130629+0000 mgr.gibba032.whopeo (mgr.24625) 385 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T13:59:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:07 gibba032 bash[24004]: cephadm 2021-03-10T13:59:06.130735+0000 mgr.gibba032.whopeo (mgr.24625) 386 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:08 gibba011 bash[30791]: cluster 2021-03-10T13:59:06.655059+0000 mgr.gibba032.whopeo (mgr.24625) 387 : cluster [DBG] pgmap v200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:08 gibba011 bash[31774]: cluster 2021-03-10T13:59:06.655059+0000 mgr.gibba032.whopeo (mgr.24625) 387 : cluster [DBG] pgmap v200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:08 gibba032 bash[24004]: cluster 2021-03-10T13:59:06.655059+0000 mgr.gibba032.whopeo (mgr.24625) 387 : cluster [DBG] pgmap v200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:10 gibba011 bash[30791]: cluster 2021-03-10T13:59:08.656492+0000 mgr.gibba032.whopeo (mgr.24625) 388 : cluster [DBG] pgmap v201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:10 gibba011 bash[31774]: cluster 2021-03-10T13:59:08.656492+0000 mgr.gibba032.whopeo (mgr.24625) 388 : cluster [DBG] pgmap v201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:10 gibba032 bash[24004]: cluster 2021-03-10T13:59:08.656492+0000 mgr.gibba032.whopeo (mgr.24625) 388 : cluster [DBG] pgmap v201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:12 gibba011 bash[30791]: cluster 2021-03-10T13:59:10.657988+0000 mgr.gibba032.whopeo (mgr.24625) 389 : cluster [DBG] pgmap v202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:12 gibba011 bash[31774]: cluster 2021-03-10T13:59:10.657988+0000 mgr.gibba032.whopeo (mgr.24625) 389 : cluster [DBG] pgmap v202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:12 gibba032 bash[24004]: cluster 2021-03-10T13:59:10.657988+0000 mgr.gibba032.whopeo (mgr.24625) 389 : cluster [DBG] pgmap v202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:14 gibba032 bash[24004]: cluster 2021-03-10T13:59:12.659646+0000 mgr.gibba032.whopeo (mgr.24625) 390 : cluster [DBG] pgmap v203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:14 gibba011 bash[30791]: cluster 2021-03-10T13:59:12.659646+0000 mgr.gibba032.whopeo (mgr.24625) 390 : cluster [DBG] pgmap v203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:14 gibba011 bash[31774]: cluster 2021-03-10T13:59:12.659646+0000 mgr.gibba032.whopeo (mgr.24625) 390 : cluster [DBG] pgmap v203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:16 gibba032 bash[24004]: cluster 2021-03-10T13:59:14.661131+0000 mgr.gibba032.whopeo (mgr.24625) 391 : cluster [DBG] pgmap v204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:16 gibba011 bash[30791]: cluster 2021-03-10T13:59:14.661131+0000 mgr.gibba032.whopeo (mgr.24625) 391 : cluster [DBG] pgmap v204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:16 gibba011 bash[31774]: cluster 2021-03-10T13:59:14.661131+0000 mgr.gibba032.whopeo (mgr.24625) 391 : cluster [DBG] pgmap v204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:18 gibba032 bash[24004]: cluster 2021-03-10T13:59:16.662649+0000 mgr.gibba032.whopeo (mgr.24625) 392 : cluster [DBG] pgmap v205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:18 gibba011 bash[30791]: cluster 2021-03-10T13:59:16.662649+0000 mgr.gibba032.whopeo (mgr.24625) 392 : cluster [DBG] pgmap v205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:18 gibba011 bash[31774]: cluster 2021-03-10T13:59:16.662649+0000 mgr.gibba032.whopeo (mgr.24625) 392 : cluster [DBG] pgmap v205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:20 gibba032 bash[24004]: cluster 2021-03-10T13:59:18.664338+0000 mgr.gibba032.whopeo (mgr.24625) 393 : cluster [DBG] pgmap v206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:20 gibba011 bash[30791]: cluster 2021-03-10T13:59:18.664338+0000 mgr.gibba032.whopeo (mgr.24625) 393 : cluster [DBG] pgmap v206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:20 gibba011 bash[31774]: cluster 2021-03-10T13:59:18.664338+0000 mgr.gibba032.whopeo (mgr.24625) 393 : cluster [DBG] pgmap v206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:22 gibba032 bash[24004]: cluster 2021-03-10T13:59:20.664857+0000 mgr.gibba032.whopeo (mgr.24625) 394 : cluster [DBG] pgmap v207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:22 gibba032 bash[24004]: cephadm 2021-03-10T13:59:21.146622+0000 mgr.gibba032.whopeo (mgr.24625) 395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:22 gibba011 bash[30791]: cluster 2021-03-10T13:59:20.664857+0000 mgr.gibba032.whopeo (mgr.24625) 394 : cluster [DBG] pgmap v207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:22 gibba011 bash[30791]: cephadm 2021-03-10T13:59:21.146622+0000 mgr.gibba032.whopeo (mgr.24625) 395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:22 gibba011 bash[31774]: cluster 2021-03-10T13:59:20.664857+0000 mgr.gibba032.whopeo (mgr.24625) 394 : cluster [DBG] pgmap v207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:22 gibba011 bash[31774]: cephadm 2021-03-10T13:59:21.146622+0000 mgr.gibba032.whopeo (mgr.24625) 395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:24 gibba032 bash[24004]: cluster 2021-03-10T13:59:22.666358+0000 mgr.gibba032.whopeo (mgr.24625) 396 : cluster [DBG] pgmap v208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:24 gibba011 bash[30791]: cluster 2021-03-10T13:59:22.666358+0000 mgr.gibba032.whopeo (mgr.24625) 396 : cluster [DBG] pgmap v208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:24 gibba011 bash[31774]: cluster 2021-03-10T13:59:22.666358+0000 mgr.gibba032.whopeo (mgr.24625) 396 : cluster [DBG] pgmap v208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:26 gibba032 bash[24004]: cluster 2021-03-10T13:59:24.667611+0000 mgr.gibba032.whopeo (mgr.24625) 397 : cluster [DBG] pgmap v209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:26 gibba011 bash[30791]: cluster 2021-03-10T13:59:24.667611+0000 mgr.gibba032.whopeo (mgr.24625) 397 : cluster [DBG] pgmap v209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:26 gibba011 bash[31774]: cluster 2021-03-10T13:59:24.667611+0000 mgr.gibba032.whopeo (mgr.24625) 397 : cluster [DBG] pgmap v209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:28 gibba032 bash[24004]: cluster 2021-03-10T13:59:26.669323+0000 mgr.gibba032.whopeo (mgr.24625) 398 : cluster [DBG] pgmap v210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:28 gibba011 bash[30791]: cluster 2021-03-10T13:59:26.669323+0000 mgr.gibba032.whopeo (mgr.24625) 398 : cluster [DBG] pgmap v210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:28.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:28 gibba011 bash[31774]: cluster 2021-03-10T13:59:26.669323+0000 mgr.gibba032.whopeo (mgr.24625) 398 : cluster [DBG] pgmap v210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:30 gibba032 bash[24004]: cluster 2021-03-10T13:59:28.670893+0000 mgr.gibba032.whopeo (mgr.24625) 399 : cluster [DBG] pgmap v211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:30 gibba011 bash[30791]: cluster 2021-03-10T13:59:28.670893+0000 mgr.gibba032.whopeo (mgr.24625) 399 : cluster [DBG] pgmap v211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:30 gibba011 bash[31774]: cluster 2021-03-10T13:59:28.670893+0000 mgr.gibba032.whopeo (mgr.24625) 399 : cluster [DBG] pgmap v211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:32.062 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T13:59:32.287 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T13:59:32.288 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (6m) 3m ago 11m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T13:59:32.288 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (11m) 87s ago 11m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T13:59:32.288 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (6m) 87s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T13:59:32.289 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (6m) 87s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T13:59:32.289 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (5m) 3m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T13:59:32.289 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (5m) 3m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T13:59:32.289 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (5m) 87s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T13:59:32.289 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (5m) 3m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T13:59:32.290 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (11m) 3m ago 11m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T13:59:32.290 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (11m) 87s ago 11m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T13:59:32.290 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (4m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T13:59:32.290 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (4m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T13:59:32.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3m) 3m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T13:59:32.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3m) 3m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T13:59:32.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2m) 87s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T13:59:32.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2m) 87s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T13:59:32.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (115s) 87s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T13:59:32.292 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (89s) 87s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T13:59:32.292 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (11m) 87s ago 11m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T13:59:32.292 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 9m docker.io/ceph/ceph:v15.2.0 2021-03-10T13:59:32.537 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T13:59:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T13:59:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:59:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T13:59:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T13:59:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T13:59:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T13:59:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T13:59:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T13:59:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T13:59:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T13:59:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T13:59:32.540 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T13:59:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:32 gibba032 bash[24004]: cluster 2021-03-10T13:59:30.671380+0000 mgr.gibba032.whopeo (mgr.24625) 400 : cluster [DBG] pgmap v212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:32 gibba011 bash[30791]: cluster 2021-03-10T13:59:30.671380+0000 mgr.gibba032.whopeo (mgr.24625) 400 : cluster [DBG] pgmap v212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:32 gibba011 bash[31774]: cluster 2021-03-10T13:59:30.671380+0000 mgr.gibba032.whopeo (mgr.24625) 400 : cluster [DBG] pgmap v212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:33.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:33 gibba032 bash[24004]: audit 2021-03-10T13:59:32.051897+0000 mgr.gibba032.whopeo (mgr.24625) 401 : audit [DBG] from='client.54310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:33 gibba032 bash[24004]: audit 2021-03-10T13:59:32.274836+0000 mgr.gibba032.whopeo (mgr.24625) 402 : audit [DBG] from='client.54316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:33 gibba032 bash[24004]: audit 2021-03-10T13:59:32.536748+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.2.111:0/3453284114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[30791]: audit 2021-03-10T13:59:32.051897+0000 mgr.gibba032.whopeo (mgr.24625) 401 : audit [DBG] from='client.54310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[30791]: audit 2021-03-10T13:59:32.274836+0000 mgr.gibba032.whopeo (mgr.24625) 402 : audit [DBG] from='client.54316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[30791]: audit 2021-03-10T13:59:32.536748+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.2.111:0/3453284114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[31774]: audit 2021-03-10T13:59:32.051897+0000 mgr.gibba032.whopeo (mgr.24625) 401 : audit [DBG] from='client.54310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[31774]: audit 2021-03-10T13:59:32.274836+0000 mgr.gibba032.whopeo (mgr.24625) 402 : audit [DBG] from='client.54316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T13:59:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:33 gibba011 bash[31774]: audit 2021-03-10T13:59:32.536748+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.2.111:0/3453284114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T13:59:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:34 gibba032 bash[24004]: cluster 2021-03-10T13:59:32.673067+0000 mgr.gibba032.whopeo (mgr.24625) 403 : cluster [DBG] pgmap v213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:34 gibba011 bash[30791]: cluster 2021-03-10T13:59:32.673067+0000 mgr.gibba032.whopeo (mgr.24625) 403 : cluster [DBG] pgmap v213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:34 gibba011 bash[31774]: cluster 2021-03-10T13:59:32.673067+0000 mgr.gibba032.whopeo (mgr.24625) 403 : cluster [DBG] pgmap v213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:35 gibba032 bash[24004]: cluster 2021-03-10T13:59:34.674590+0000 mgr.gibba032.whopeo (mgr.24625) 404 : cluster [DBG] pgmap v214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:36.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:35 gibba011 bash[30791]: cluster 2021-03-10T13:59:34.674590+0000 mgr.gibba032.whopeo (mgr.24625) 404 : cluster [DBG] pgmap v214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:36.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:35 gibba011 bash[31774]: cluster 2021-03-10T13:59:34.674590+0000 mgr.gibba032.whopeo (mgr.24625) 404 : cluster [DBG] pgmap v214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:36 gibba032 bash[24004]: cephadm 2021-03-10T13:59:36.162393+0000 mgr.gibba032.whopeo (mgr.24625) 405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:36 gibba011 bash[30791]: cephadm 2021-03-10T13:59:36.162393+0000 mgr.gibba032.whopeo (mgr.24625) 405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:36 gibba011 bash[31774]: cephadm 2021-03-10T13:59:36.162393+0000 mgr.gibba032.whopeo (mgr.24625) 405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:37 gibba032 bash[24004]: cluster 2021-03-10T13:59:36.676374+0000 mgr.gibba032.whopeo (mgr.24625) 406 : cluster [DBG] pgmap v215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:38.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:37 gibba011 bash[30791]: cluster 2021-03-10T13:59:36.676374+0000 mgr.gibba032.whopeo (mgr.24625) 406 : cluster [DBG] pgmap v215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:37 gibba011 bash[31774]: cluster 2021-03-10T13:59:36.676374+0000 mgr.gibba032.whopeo (mgr.24625) 406 : cluster [DBG] pgmap v215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:40 gibba032 bash[24004]: cluster 2021-03-10T13:59:38.677951+0000 mgr.gibba032.whopeo (mgr.24625) 407 : cluster [DBG] pgmap v216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:40 gibba011 bash[30791]: cluster 2021-03-10T13:59:38.677951+0000 mgr.gibba032.whopeo (mgr.24625) 407 : cluster [DBG] pgmap v216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:40 gibba011 bash[31774]: cluster 2021-03-10T13:59:38.677951+0000 mgr.gibba032.whopeo (mgr.24625) 407 : cluster [DBG] pgmap v216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:42 gibba011 bash[30791]: cluster 2021-03-10T13:59:40.678539+0000 mgr.gibba032.whopeo (mgr.24625) 408 : cluster [DBG] pgmap v217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:42 gibba011 bash[31774]: cluster 2021-03-10T13:59:40.678539+0000 mgr.gibba032.whopeo (mgr.24625) 408 : cluster [DBG] pgmap v217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:42 gibba032 bash[24004]: cluster 2021-03-10T13:59:40.678539+0000 mgr.gibba032.whopeo (mgr.24625) 408 : cluster [DBG] pgmap v217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:44 gibba011 bash[30791]: cluster 2021-03-10T13:59:42.680119+0000 mgr.gibba032.whopeo (mgr.24625) 409 : cluster [DBG] pgmap v218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:44.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:44 gibba011 bash[31774]: cluster 2021-03-10T13:59:42.680119+0000 mgr.gibba032.whopeo (mgr.24625) 409 : cluster [DBG] pgmap v218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:44 gibba032 bash[24004]: cluster 2021-03-10T13:59:42.680119+0000 mgr.gibba032.whopeo (mgr.24625) 409 : cluster [DBG] pgmap v218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:46.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:46 gibba011 bash[30791]: cluster 2021-03-10T13:59:44.681621+0000 mgr.gibba032.whopeo (mgr.24625) 410 : cluster [DBG] pgmap v219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:46.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:46 gibba011 bash[31774]: cluster 2021-03-10T13:59:44.681621+0000 mgr.gibba032.whopeo (mgr.24625) 410 : cluster [DBG] pgmap v219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:46 gibba032 bash[24004]: cluster 2021-03-10T13:59:44.681621+0000 mgr.gibba032.whopeo (mgr.24625) 410 : cluster [DBG] pgmap v219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:48 gibba011 bash[30791]: cluster 2021-03-10T13:59:46.683397+0000 mgr.gibba032.whopeo (mgr.24625) 411 : cluster [DBG] pgmap v220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:48.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:48 gibba011 bash[31774]: cluster 2021-03-10T13:59:46.683397+0000 mgr.gibba032.whopeo (mgr.24625) 411 : cluster [DBG] pgmap v220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:48 gibba032 bash[24004]: cluster 2021-03-10T13:59:46.683397+0000 mgr.gibba032.whopeo (mgr.24625) 411 : cluster [DBG] pgmap v220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:50 gibba011 bash[30791]: cluster 2021-03-10T13:59:48.685003+0000 mgr.gibba032.whopeo (mgr.24625) 412 : cluster [DBG] pgmap v221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:50.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:50 gibba011 bash[31774]: cluster 2021-03-10T13:59:48.685003+0000 mgr.gibba032.whopeo (mgr.24625) 412 : cluster [DBG] pgmap v221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:50 gibba032 bash[24004]: cluster 2021-03-10T13:59:48.685003+0000 mgr.gibba032.whopeo (mgr.24625) 412 : cluster [DBG] pgmap v221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:52 gibba011 bash[30791]: cluster 2021-03-10T13:59:50.685663+0000 mgr.gibba032.whopeo (mgr.24625) 413 : cluster [DBG] pgmap v222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:52 gibba011 bash[30791]: cephadm 2021-03-10T13:59:51.168212+0000 mgr.gibba032.whopeo (mgr.24625) 414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:52 gibba011 bash[31774]: cluster 2021-03-10T13:59:50.685663+0000 mgr.gibba032.whopeo (mgr.24625) 413 : cluster [DBG] pgmap v222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:52 gibba011 bash[31774]: cephadm 2021-03-10T13:59:51.168212+0000 mgr.gibba032.whopeo (mgr.24625) 414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:52 gibba032 bash[24004]: cluster 2021-03-10T13:59:50.685663+0000 mgr.gibba032.whopeo (mgr.24625) 413 : cluster [DBG] pgmap v222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:52 gibba032 bash[24004]: cephadm 2021-03-10T13:59:51.168212+0000 mgr.gibba032.whopeo (mgr.24625) 414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T13:59:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:54 gibba011 bash[30791]: cluster 2021-03-10T13:59:52.687278+0000 mgr.gibba032.whopeo (mgr.24625) 415 : cluster [DBG] pgmap v223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:54 gibba011 bash[31774]: cluster 2021-03-10T13:59:52.687278+0000 mgr.gibba032.whopeo (mgr.24625) 415 : cluster [DBG] pgmap v223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:54 gibba032 bash[24004]: cluster 2021-03-10T13:59:52.687278+0000 mgr.gibba032.whopeo (mgr.24625) 415 : cluster [DBG] pgmap v223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: cluster 2021-03-10T13:59:54.688867+0000 mgr.gibba032.whopeo (mgr.24625) 416 : cluster [DBG] pgmap v224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:55.902844+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T13:59:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:55.903662+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:55.904354+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T13:59:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:55.905038+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:56.012876+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:59:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[30791]: audit 2021-03-10T13:59:56.013552+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:59:56.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: cluster 2021-03-10T13:59:54.688867+0000 mgr.gibba032.whopeo (mgr.24625) 416 : cluster [DBG] pgmap v224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:55.902844+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T13:59:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:55.903662+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:55.904354+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T13:59:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:55.905038+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:56.012876+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:59:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:56 gibba011 bash[31774]: audit 2021-03-10T13:59:56.013552+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:59:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: cluster 2021-03-10T13:59:54.688867+0000 mgr.gibba032.whopeo (mgr.24625) 416 : cluster [DBG] pgmap v224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:55.902844+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T13:59:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:55.903662+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:55.904354+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T13:59:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:55.905038+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T13:59:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:56.012876+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T13:59:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:56 gibba032 bash[24004]: audit 2021-03-10T13:59:56.013552+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T13:59:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 13:59:58 gibba011 bash[30791]: cluster 2021-03-10T13:59:56.690618+0000 mgr.gibba032.whopeo (mgr.24625) 417 : cluster [DBG] pgmap v225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 13:59:58 gibba011 bash[31774]: cluster 2021-03-10T13:59:56.690618+0000 mgr.gibba032.whopeo (mgr.24625) 417 : cluster [DBG] pgmap v225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T13:59:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 13:59:58 gibba032 bash[24004]: cluster 2021-03-10T13:59:56.690618+0000 mgr.gibba032.whopeo (mgr.24625) 417 : cluster [DBG] pgmap v225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:00 gibba011 bash[30791]: debug 2021-03-10T13:59:59.995+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:00:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:00 gibba011 bash[30791]: cluster 2021-03-10T13:59:58.692255+0000 mgr.gibba032.whopeo (mgr.24625) 418 : cluster [DBG] pgmap v226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:00 gibba011 bash[30791]: cluster 2021-03-10T14:00:00.000185+0000 mon.a (mon.0) 692 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:00:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:00 gibba011 bash[31774]: cluster 2021-03-10T13:59:58.692255+0000 mgr.gibba032.whopeo (mgr.24625) 418 : cluster [DBG] pgmap v226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:00 gibba011 bash[31774]: cluster 2021-03-10T14:00:00.000185+0000 mon.a (mon.0) 692 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:00:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:00 gibba032 bash[24004]: cluster 2021-03-10T13:59:58.692255+0000 mgr.gibba032.whopeo (mgr.24625) 418 : cluster [DBG] pgmap v226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:00 gibba032 bash[24004]: cluster 2021-03-10T14:00:00.000185+0000 mon.a (mon.0) 692 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:00:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:02 gibba032 bash[24004]: cluster 2021-03-10T14:00:00.692873+0000 mgr.gibba032.whopeo (mgr.24625) 419 : cluster [DBG] pgmap v227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:02 gibba011 bash[30791]: cluster 2021-03-10T14:00:00.692873+0000 mgr.gibba032.whopeo (mgr.24625) 419 : cluster [DBG] pgmap v227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:02 gibba011 bash[31774]: cluster 2021-03-10T14:00:00.692873+0000 mgr.gibba032.whopeo (mgr.24625) 419 : cluster [DBG] pgmap v227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:02.781 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:00:03.008 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:00:03.008 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (6m) 3m ago 12m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:00:03.008 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (11m) 118s ago 11m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:00:03.008 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (7m) 118s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:00:03.009 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (7m) 118s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:00:03.009 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (6m) 3m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:00:03.009 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (6m) 3m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:00:03.009 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (5m) 118s ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:00:03.010 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (5m) 3m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:00:03.010 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (12m) 3m ago 12m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:00:03.010 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (12m) 118s ago 12m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:00:03.010 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (4m) 3m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:00:03.011 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (4m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:00:03.011 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (4m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:00:03.011 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:00:03.011 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3m) 118s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:00:03.011 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2m) 118s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:00:03.012 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2m) 118s ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:00:03.012 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2m) 118s ago 12m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:00:03.012 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (11m) 118s ago 12m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:00:03.012 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 10m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:00:03.261 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:00:03.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:00:03.262 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:00:03.262 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:03.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:00:03.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:00:03.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:03.263 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:00:03.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:00:03.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:03.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:00:03.264 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:00:03.264 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:00:03.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:00:03.265 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:00:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:04 gibba032 bash[24004]: cluster 2021-03-10T14:00:02.694527+0000 mgr.gibba032.whopeo (mgr.24625) 420 : cluster [DBG] pgmap v228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:04 gibba032 bash[24004]: audit 2021-03-10T14:00:02.770984+0000 mgr.gibba032.whopeo (mgr.24625) 421 : audit [DBG] from='client.54328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:04 gibba032 bash[24004]: audit 2021-03-10T14:00:02.994847+0000 mgr.gibba032.whopeo (mgr.24625) 422 : audit [DBG] from='client.54334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:04 gibba032 bash[24004]: audit 2021-03-10T14:00:03.260802+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/1942680889' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[30791]: cluster 2021-03-10T14:00:02.694527+0000 mgr.gibba032.whopeo (mgr.24625) 420 : cluster [DBG] pgmap v228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[30791]: audit 2021-03-10T14:00:02.770984+0000 mgr.gibba032.whopeo (mgr.24625) 421 : audit [DBG] from='client.54328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[30791]: audit 2021-03-10T14:00:02.994847+0000 mgr.gibba032.whopeo (mgr.24625) 422 : audit [DBG] from='client.54334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[30791]: audit 2021-03-10T14:00:03.260802+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/1942680889' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[31774]: cluster 2021-03-10T14:00:02.694527+0000 mgr.gibba032.whopeo (mgr.24625) 420 : cluster [DBG] pgmap v228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[31774]: audit 2021-03-10T14:00:02.770984+0000 mgr.gibba032.whopeo (mgr.24625) 421 : audit [DBG] from='client.54328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[31774]: audit 2021-03-10T14:00:02.994847+0000 mgr.gibba032.whopeo (mgr.24625) 422 : audit [DBG] from='client.54334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:04 gibba011 bash[31774]: audit 2021-03-10T14:00:03.260802+0000 mon.c (mon.1) 11 : audit [DBG] from='client.? 172.21.2.111:0/1942680889' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:06.405 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:06 gibba032 bash[24004]: cluster 2021-03-10T14:00:04.696070+0000 mgr.gibba032.whopeo (mgr.24625) 423 : cluster [DBG] pgmap v229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:06 gibba011 bash[30791]: cluster 2021-03-10T14:00:04.696070+0000 mgr.gibba032.whopeo (mgr.24625) 423 : cluster [DBG] pgmap v229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:06 gibba011 bash[31774]: cluster 2021-03-10T14:00:04.696070+0000 mgr.gibba032.whopeo (mgr.24625) 423 : cluster [DBG] pgmap v229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.185066+0000 mgr.gibba032.whopeo (mgr.24625) 424 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:00:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.186612+0000 mgr.gibba032.whopeo (mgr.24625) 425 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:00:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.391845+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:00:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.393828+0000 mgr.gibba032.whopeo (mgr.24625) 426 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:00:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.395038+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:00:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.395667+0000 mgr.gibba032.whopeo (mgr.24625) 427 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:00:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.396503+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.397016+0000 mgr.gibba032.whopeo (mgr.24625) 428 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:00:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.397052+0000 mgr.gibba032.whopeo (mgr.24625) 429 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:00:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.397883+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.398362+0000 mgr.gibba032.whopeo (mgr.24625) 430 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:00:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.398399+0000 mgr.gibba032.whopeo (mgr.24625) 431 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:00:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.399162+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.399594+0000 mgr.gibba032.whopeo (mgr.24625) 432 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:00:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.400908+0000 mgr.gibba032.whopeo (mgr.24625) 433 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:00:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.400942+0000 mgr.gibba032.whopeo (mgr.24625) 434 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:00:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.401876+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.402316+0000 mgr.gibba032.whopeo (mgr.24625) 435 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:00:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.402677+0000 mgr.gibba032.whopeo (mgr.24625) 436 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:00:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: audit 2021-03-10T14:00:06.403627+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.404199+0000 mgr.gibba032.whopeo (mgr.24625) 437 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:00:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.404325+0000 mgr.gibba032.whopeo (mgr.24625) 438 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:00:07.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:07 gibba032 bash[24004]: cephadm 2021-03-10T14:00:06.404432+0000 mgr.gibba032.whopeo (mgr.24625) 439 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.185066+0000 mgr.gibba032.whopeo (mgr.24625) 424 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:00:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.186612+0000 mgr.gibba032.whopeo (mgr.24625) 425 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:00:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.391845+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:00:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.393828+0000 mgr.gibba032.whopeo (mgr.24625) 426 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:00:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.395038+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:00:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.395667+0000 mgr.gibba032.whopeo (mgr.24625) 427 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:00:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.396503+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.397016+0000 mgr.gibba032.whopeo (mgr.24625) 428 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:00:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.397052+0000 mgr.gibba032.whopeo (mgr.24625) 429 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:00:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.397883+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.398362+0000 mgr.gibba032.whopeo (mgr.24625) 430 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:00:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.398399+0000 mgr.gibba032.whopeo (mgr.24625) 431 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:00:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.399162+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.399594+0000 mgr.gibba032.whopeo (mgr.24625) 432 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:00:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.400908+0000 mgr.gibba032.whopeo (mgr.24625) 433 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:00:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.400942+0000 mgr.gibba032.whopeo (mgr.24625) 434 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:00:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.401876+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.402316+0000 mgr.gibba032.whopeo (mgr.24625) 435 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:00:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.402677+0000 mgr.gibba032.whopeo (mgr.24625) 436 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:00:07.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: audit 2021-03-10T14:00:06.403627+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.404199+0000 mgr.gibba032.whopeo (mgr.24625) 437 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:00:07.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.404325+0000 mgr.gibba032.whopeo (mgr.24625) 438 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:00:07.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[30791]: cephadm 2021-03-10T14:00:06.404432+0000 mgr.gibba032.whopeo (mgr.24625) 439 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.185066+0000 mgr.gibba032.whopeo (mgr.24625) 424 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:00:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.186612+0000 mgr.gibba032.whopeo (mgr.24625) 425 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:00:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.391845+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:00:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.393828+0000 mgr.gibba032.whopeo (mgr.24625) 426 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:00:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.395038+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:00:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.395667+0000 mgr.gibba032.whopeo (mgr.24625) 427 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:00:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.396503+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.397016+0000 mgr.gibba032.whopeo (mgr.24625) 428 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:00:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.397052+0000 mgr.gibba032.whopeo (mgr.24625) 429 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:00:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.397883+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.398362+0000 mgr.gibba032.whopeo (mgr.24625) 430 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:00:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.398399+0000 mgr.gibba032.whopeo (mgr.24625) 431 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:00:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.399162+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.399594+0000 mgr.gibba032.whopeo (mgr.24625) 432 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:00:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.400908+0000 mgr.gibba032.whopeo (mgr.24625) 433 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:00:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.400942+0000 mgr.gibba032.whopeo (mgr.24625) 434 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:00:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.401876+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.402316+0000 mgr.gibba032.whopeo (mgr.24625) 435 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:00:07.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.402677+0000 mgr.gibba032.whopeo (mgr.24625) 436 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:00:07.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: audit 2021-03-10T14:00:06.403627+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:07.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.404199+0000 mgr.gibba032.whopeo (mgr.24625) 437 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:00:07.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.404325+0000 mgr.gibba032.whopeo (mgr.24625) 438 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:00:07.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:07 gibba011 bash[31774]: cephadm 2021-03-10T14:00:06.404432+0000 mgr.gibba032.whopeo (mgr.24625) 439 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:08 gibba032 bash[24004]: cluster 2021-03-10T14:00:06.697811+0000 mgr.gibba032.whopeo (mgr.24625) 440 : cluster [DBG] pgmap v230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:08 gibba011 bash[30791]: cluster 2021-03-10T14:00:06.697811+0000 mgr.gibba032.whopeo (mgr.24625) 440 : cluster [DBG] pgmap v230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:08 gibba011 bash[31774]: cluster 2021-03-10T14:00:06.697811+0000 mgr.gibba032.whopeo (mgr.24625) 440 : cluster [DBG] pgmap v230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:10 gibba032 bash[24004]: cluster 2021-03-10T14:00:08.699268+0000 mgr.gibba032.whopeo (mgr.24625) 441 : cluster [DBG] pgmap v231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:10 gibba011 bash[30791]: cluster 2021-03-10T14:00:08.699268+0000 mgr.gibba032.whopeo (mgr.24625) 441 : cluster [DBG] pgmap v231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:10 gibba011 bash[31774]: cluster 2021-03-10T14:00:08.699268+0000 mgr.gibba032.whopeo (mgr.24625) 441 : cluster [DBG] pgmap v231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:12 gibba032 bash[24004]: cluster 2021-03-10T14:00:10.699808+0000 mgr.gibba032.whopeo (mgr.24625) 442 : cluster [DBG] pgmap v232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:12 gibba011 bash[30791]: cluster 2021-03-10T14:00:10.699808+0000 mgr.gibba032.whopeo (mgr.24625) 442 : cluster [DBG] pgmap v232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:12 gibba011 bash[31774]: cluster 2021-03-10T14:00:10.699808+0000 mgr.gibba032.whopeo (mgr.24625) 442 : cluster [DBG] pgmap v232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:13.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:13 gibba032 bash[24004]: cluster 2021-03-10T14:00:12.701459+0000 mgr.gibba032.whopeo (mgr.24625) 443 : cluster [DBG] pgmap v233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:13.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:13 gibba011 bash[30791]: cluster 2021-03-10T14:00:12.701459+0000 mgr.gibba032.whopeo (mgr.24625) 443 : cluster [DBG] pgmap v233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:13.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:13 gibba011 bash[31774]: cluster 2021-03-10T14:00:12.701459+0000 mgr.gibba032.whopeo (mgr.24625) 443 : cluster [DBG] pgmap v233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:16 gibba032 bash[24004]: cluster 2021-03-10T14:00:14.702943+0000 mgr.gibba032.whopeo (mgr.24625) 444 : cluster [DBG] pgmap v234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:16.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:15 gibba011 bash[30791]: cluster 2021-03-10T14:00:14.702943+0000 mgr.gibba032.whopeo (mgr.24625) 444 : cluster [DBG] pgmap v234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:16 gibba011 bash[31774]: cluster 2021-03-10T14:00:14.702943+0000 mgr.gibba032.whopeo (mgr.24625) 444 : cluster [DBG] pgmap v234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:18 gibba011 bash[30791]: cluster 2021-03-10T14:00:16.704707+0000 mgr.gibba032.whopeo (mgr.24625) 445 : cluster [DBG] pgmap v235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:18 gibba011 bash[31774]: cluster 2021-03-10T14:00:16.704707+0000 mgr.gibba032.whopeo (mgr.24625) 445 : cluster [DBG] pgmap v235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:18 gibba032 bash[24004]: cluster 2021-03-10T14:00:16.704707+0000 mgr.gibba032.whopeo (mgr.24625) 445 : cluster [DBG] pgmap v235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:20 gibba011 bash[30791]: cluster 2021-03-10T14:00:18.706271+0000 mgr.gibba032.whopeo (mgr.24625) 446 : cluster [DBG] pgmap v236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:20.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:20 gibba011 bash[31774]: cluster 2021-03-10T14:00:18.706271+0000 mgr.gibba032.whopeo (mgr.24625) 446 : cluster [DBG] pgmap v236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:20 gibba032 bash[24004]: cluster 2021-03-10T14:00:18.706271+0000 mgr.gibba032.whopeo (mgr.24625) 446 : cluster [DBG] pgmap v236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:22 gibba011 bash[30791]: cluster 2021-03-10T14:00:20.707604+0000 mgr.gibba032.whopeo (mgr.24625) 447 : cluster [DBG] pgmap v237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:22 gibba011 bash[30791]: cephadm 2021-03-10T14:00:21.420473+0000 mgr.gibba032.whopeo (mgr.24625) 448 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:22 gibba011 bash[31774]: cluster 2021-03-10T14:00:20.707604+0000 mgr.gibba032.whopeo (mgr.24625) 447 : cluster [DBG] pgmap v237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:22 gibba011 bash[31774]: cephadm 2021-03-10T14:00:21.420473+0000 mgr.gibba032.whopeo (mgr.24625) 448 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:22 gibba032 bash[24004]: cluster 2021-03-10T14:00:20.707604+0000 mgr.gibba032.whopeo (mgr.24625) 447 : cluster [DBG] pgmap v237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:22 gibba032 bash[24004]: cephadm 2021-03-10T14:00:21.420473+0000 mgr.gibba032.whopeo (mgr.24625) 448 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:24 gibba011 bash[30791]: cluster 2021-03-10T14:00:22.709248+0000 mgr.gibba032.whopeo (mgr.24625) 449 : cluster [DBG] pgmap v238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:24.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:24 gibba011 bash[31774]: cluster 2021-03-10T14:00:22.709248+0000 mgr.gibba032.whopeo (mgr.24625) 449 : cluster [DBG] pgmap v238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:24 gibba032 bash[24004]: cluster 2021-03-10T14:00:22.709248+0000 mgr.gibba032.whopeo (mgr.24625) 449 : cluster [DBG] pgmap v238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:26 gibba011 bash[30791]: cluster 2021-03-10T14:00:24.710501+0000 mgr.gibba032.whopeo (mgr.24625) 450 : cluster [DBG] pgmap v239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:26 gibba011 bash[31774]: cluster 2021-03-10T14:00:24.710501+0000 mgr.gibba032.whopeo (mgr.24625) 450 : cluster [DBG] pgmap v239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:26 gibba032 bash[24004]: cluster 2021-03-10T14:00:24.710501+0000 mgr.gibba032.whopeo (mgr.24625) 450 : cluster [DBG] pgmap v239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:28 gibba011 bash[30791]: cluster 2021-03-10T14:00:26.712322+0000 mgr.gibba032.whopeo (mgr.24625) 451 : cluster [DBG] pgmap v240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:28 gibba011 bash[31774]: cluster 2021-03-10T14:00:26.712322+0000 mgr.gibba032.whopeo (mgr.24625) 451 : cluster [DBG] pgmap v240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:28 gibba032 bash[24004]: cluster 2021-03-10T14:00:26.712322+0000 mgr.gibba032.whopeo (mgr.24625) 451 : cluster [DBG] pgmap v240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:30 gibba011 bash[30791]: cluster 2021-03-10T14:00:28.713939+0000 mgr.gibba032.whopeo (mgr.24625) 452 : cluster [DBG] pgmap v241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:30.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:30 gibba011 bash[31774]: cluster 2021-03-10T14:00:28.713939+0000 mgr.gibba032.whopeo (mgr.24625) 452 : cluster [DBG] pgmap v241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:30 gibba032 bash[24004]: cluster 2021-03-10T14:00:28.713939+0000 mgr.gibba032.whopeo (mgr.24625) 452 : cluster [DBG] pgmap v241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:32 gibba011 bash[30791]: cluster 2021-03-10T14:00:30.715264+0000 mgr.gibba032.whopeo (mgr.24625) 453 : cluster [DBG] pgmap v242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:32 gibba011 bash[31774]: cluster 2021-03-10T14:00:30.715264+0000 mgr.gibba032.whopeo (mgr.24625) 453 : cluster [DBG] pgmap v242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:32 gibba032 bash[24004]: cluster 2021-03-10T14:00:30.715264+0000 mgr.gibba032.whopeo (mgr.24625) 453 : cluster [DBG] pgmap v242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:33.506 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:00:33.732 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:00:33.732 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (7m) 4m ago 12m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:00:33.732 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (12m) 2m ago 12m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:00:33.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (7m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:00:33.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (7m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:00:33.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (6m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:00:33.733 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (6m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:00:33.734 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (6m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:00:33.734 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (6m) 4m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:00:33.734 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (12m) 4m ago 12m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:00:33.734 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (12m) 2m ago 12m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:00:33.734 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:00:33.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:00:33.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:00:33.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (4m) 4m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:00:33.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:00:33.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:00:33.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:00:33.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:00:33.736 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (12m) 2m ago 12m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:00:33.737 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 10m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:00:33.980 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:00:33.981 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:00:33.981 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:00:33.981 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:33.982 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:00:33.982 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:00:33.982 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:33.982 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:00:33.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:00:33.983 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:00:33.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:00:33.983 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:00:33.984 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:00:33.984 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:00:33.984 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:00:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:34 gibba011 bash[30791]: cluster 2021-03-10T14:00:32.717051+0000 mgr.gibba032.whopeo (mgr.24625) 454 : cluster [DBG] pgmap v243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:34.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:34 gibba011 bash[30791]: audit 2021-03-10T14:00:33.980147+0000 mon.a (mon.0) 700 : audit [DBG] from='client.? 172.21.2.111:0/850998294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:34 gibba011 bash[31774]: cluster 2021-03-10T14:00:32.717051+0000 mgr.gibba032.whopeo (mgr.24625) 454 : cluster [DBG] pgmap v243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:34 gibba011 bash[31774]: audit 2021-03-10T14:00:33.980147+0000 mon.a (mon.0) 700 : audit [DBG] from='client.? 172.21.2.111:0/850998294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:34 gibba032 bash[24004]: cluster 2021-03-10T14:00:32.717051+0000 mgr.gibba032.whopeo (mgr.24625) 454 : cluster [DBG] pgmap v243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:34 gibba032 bash[24004]: audit 2021-03-10T14:00:33.980147+0000 mon.a (mon.0) 700 : audit [DBG] from='client.? 172.21.2.111:0/850998294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:00:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:35 gibba011 bash[30791]: audit 2021-03-10T14:00:33.495978+0000 mgr.gibba032.whopeo (mgr.24625) 455 : audit [DBG] from='client.44361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:35.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:35 gibba011 bash[30791]: audit 2021-03-10T14:00:33.718413+0000 mgr.gibba032.whopeo (mgr.24625) 456 : audit [DBG] from='client.54352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:35 gibba011 bash[31774]: audit 2021-03-10T14:00:33.495978+0000 mgr.gibba032.whopeo (mgr.24625) 455 : audit [DBG] from='client.44361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:35.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:35 gibba011 bash[31774]: audit 2021-03-10T14:00:33.718413+0000 mgr.gibba032.whopeo (mgr.24625) 456 : audit [DBG] from='client.54352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:35 gibba032 bash[24004]: audit 2021-03-10T14:00:33.495978+0000 mgr.gibba032.whopeo (mgr.24625) 455 : audit [DBG] from='client.44361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:35 gibba032 bash[24004]: audit 2021-03-10T14:00:33.718413+0000 mgr.gibba032.whopeo (mgr.24625) 456 : audit [DBG] from='client.54352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:00:36.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:36 gibba011 bash[30791]: cluster 2021-03-10T14:00:34.718529+0000 mgr.gibba032.whopeo (mgr.24625) 457 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:36.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:36 gibba011 bash[31774]: cluster 2021-03-10T14:00:34.718529+0000 mgr.gibba032.whopeo (mgr.24625) 457 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:36.438 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:36 gibba032 bash[24004]: cluster 2021-03-10T14:00:34.718529+0000 mgr.gibba032.whopeo (mgr.24625) 457 : cluster [DBG] pgmap v244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:37 gibba032 bash[24004]: cephadm 2021-03-10T14:00:36.436603+0000 mgr.gibba032.whopeo (mgr.24625) 458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:37.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:37 gibba011 bash[30791]: cephadm 2021-03-10T14:00:36.436603+0000 mgr.gibba032.whopeo (mgr.24625) 458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:37 gibba011 bash[31774]: cephadm 2021-03-10T14:00:36.436603+0000 mgr.gibba032.whopeo (mgr.24625) 458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:38 gibba032 bash[24004]: cluster 2021-03-10T14:00:36.720251+0000 mgr.gibba032.whopeo (mgr.24625) 459 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:38 gibba011 bash[30791]: cluster 2021-03-10T14:00:36.720251+0000 mgr.gibba032.whopeo (mgr.24625) 459 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:38 gibba011 bash[31774]: cluster 2021-03-10T14:00:36.720251+0000 mgr.gibba032.whopeo (mgr.24625) 459 : cluster [DBG] pgmap v245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:40 gibba032 bash[24004]: cluster 2021-03-10T14:00:38.721873+0000 mgr.gibba032.whopeo (mgr.24625) 460 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:40 gibba011 bash[30791]: cluster 2021-03-10T14:00:38.721873+0000 mgr.gibba032.whopeo (mgr.24625) 460 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:40 gibba011 bash[31774]: cluster 2021-03-10T14:00:38.721873+0000 mgr.gibba032.whopeo (mgr.24625) 460 : cluster [DBG] pgmap v246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:42 gibba032 bash[24004]: cluster 2021-03-10T14:00:40.723196+0000 mgr.gibba032.whopeo (mgr.24625) 461 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:42 gibba011 bash[30791]: cluster 2021-03-10T14:00:40.723196+0000 mgr.gibba032.whopeo (mgr.24625) 461 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:42 gibba011 bash[31774]: cluster 2021-03-10T14:00:40.723196+0000 mgr.gibba032.whopeo (mgr.24625) 461 : cluster [DBG] pgmap v247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:44 gibba032 bash[24004]: cluster 2021-03-10T14:00:42.725026+0000 mgr.gibba032.whopeo (mgr.24625) 462 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:44 gibba011 bash[30791]: cluster 2021-03-10T14:00:42.725026+0000 mgr.gibba032.whopeo (mgr.24625) 462 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:44 gibba011 bash[31774]: cluster 2021-03-10T14:00:42.725026+0000 mgr.gibba032.whopeo (mgr.24625) 462 : cluster [DBG] pgmap v248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:46 gibba032 bash[24004]: cluster 2021-03-10T14:00:44.726554+0000 mgr.gibba032.whopeo (mgr.24625) 463 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:46 gibba011 bash[30791]: cluster 2021-03-10T14:00:44.726554+0000 mgr.gibba032.whopeo (mgr.24625) 463 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:46 gibba011 bash[31774]: cluster 2021-03-10T14:00:44.726554+0000 mgr.gibba032.whopeo (mgr.24625) 463 : cluster [DBG] pgmap v249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:48 gibba032 bash[24004]: cluster 2021-03-10T14:00:46.728062+0000 mgr.gibba032.whopeo (mgr.24625) 464 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:48 gibba011 bash[30791]: cluster 2021-03-10T14:00:46.728062+0000 mgr.gibba032.whopeo (mgr.24625) 464 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:48 gibba011 bash[31774]: cluster 2021-03-10T14:00:46.728062+0000 mgr.gibba032.whopeo (mgr.24625) 464 : cluster [DBG] pgmap v250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:50 gibba032 bash[24004]: cluster 2021-03-10T14:00:48.729450+0000 mgr.gibba032.whopeo (mgr.24625) 465 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:50 gibba011 bash[30791]: cluster 2021-03-10T14:00:48.729450+0000 mgr.gibba032.whopeo (mgr.24625) 465 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:50 gibba011 bash[31774]: cluster 2021-03-10T14:00:48.729450+0000 mgr.gibba032.whopeo (mgr.24625) 465 : cluster [DBG] pgmap v251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:52 gibba032 bash[24004]: cluster 2021-03-10T14:00:50.730794+0000 mgr.gibba032.whopeo (mgr.24625) 466 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:52 gibba011 bash[30791]: cluster 2021-03-10T14:00:50.730794+0000 mgr.gibba032.whopeo (mgr.24625) 466 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:52 gibba011 bash[31774]: cluster 2021-03-10T14:00:50.730794+0000 mgr.gibba032.whopeo (mgr.24625) 466 : cluster [DBG] pgmap v252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:53 gibba032 bash[24004]: cephadm 2021-03-10T14:00:51.451533+0000 mgr.gibba032.whopeo (mgr.24625) 467 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:53 gibba011 bash[30791]: cephadm 2021-03-10T14:00:51.451533+0000 mgr.gibba032.whopeo (mgr.24625) 467 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:53.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:53 gibba011 bash[31774]: cephadm 2021-03-10T14:00:51.451533+0000 mgr.gibba032.whopeo (mgr.24625) 467 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:00:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:54 gibba032 bash[24004]: cluster 2021-03-10T14:00:52.732556+0000 mgr.gibba032.whopeo (mgr.24625) 468 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:54 gibba011 bash[30791]: cluster 2021-03-10T14:00:52.732556+0000 mgr.gibba032.whopeo (mgr.24625) 468 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:54 gibba011 bash[31774]: cluster 2021-03-10T14:00:52.732556+0000 mgr.gibba032.whopeo (mgr.24625) 468 : cluster [DBG] pgmap v253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: cluster 2021-03-10T14:00:54.734026+0000 mgr.gibba032.whopeo (mgr.24625) 469 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:55.913353+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:00:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:55.914102+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:55.914780+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:00:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:55.915755+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:56.020001+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:00:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:56 gibba032 bash[24004]: audit 2021-03-10T14:00:56.020639+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:00:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: cluster 2021-03-10T14:00:54.734026+0000 mgr.gibba032.whopeo (mgr.24625) 469 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:55.913353+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:00:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:55.914102+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:55.914780+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:00:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:55.915755+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:56.020001+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:00:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[30791]: audit 2021-03-10T14:00:56.020639+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:00:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: cluster 2021-03-10T14:00:54.734026+0000 mgr.gibba032.whopeo (mgr.24625) 469 : cluster [DBG] pgmap v254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:55.913353+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:00:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:55.914102+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:55.914780+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:00:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:55.915755+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:00:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:56.020001+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:00:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:56 gibba011 bash[31774]: audit 2021-03-10T14:00:56.020639+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:00:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:00:58 gibba032 bash[24004]: cluster 2021-03-10T14:00:56.735832+0000 mgr.gibba032.whopeo (mgr.24625) 470 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:00:58 gibba011 bash[30791]: cluster 2021-03-10T14:00:56.735832+0000 mgr.gibba032.whopeo (mgr.24625) 470 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:00:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:00:58 gibba011 bash[31774]: cluster 2021-03-10T14:00:56.735832+0000 mgr.gibba032.whopeo (mgr.24625) 470 : cluster [DBG] pgmap v255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:00 gibba032 bash[24004]: cluster 2021-03-10T14:00:58.737316+0000 mgr.gibba032.whopeo (mgr.24625) 471 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:00 gibba011 bash[30791]: cluster 2021-03-10T14:00:58.737316+0000 mgr.gibba032.whopeo (mgr.24625) 471 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:00 gibba011 bash[31774]: cluster 2021-03-10T14:00:58.737316+0000 mgr.gibba032.whopeo (mgr.24625) 471 : cluster [DBG] pgmap v256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:02 gibba032 bash[24004]: cluster 2021-03-10T14:01:00.738788+0000 mgr.gibba032.whopeo (mgr.24625) 472 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:02 gibba011 bash[30791]: cluster 2021-03-10T14:01:00.738788+0000 mgr.gibba032.whopeo (mgr.24625) 472 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:02 gibba011 bash[31774]: cluster 2021-03-10T14:01:00.738788+0000 mgr.gibba032.whopeo (mgr.24625) 472 : cluster [DBG] pgmap v257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:04.226 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:01:04.488 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:01:04.489 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (7m) 4m ago 13m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:01:04.489 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (12m) 2m ago 12m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:01:04.489 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (8m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:01:04.489 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (8m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:01:04.490 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (7m) 4m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:01:04.490 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (7m) 4m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:01:04.490 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (6m) 2m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:01:04.491 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (6m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:01:04.491 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (13m) 4m ago 13m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:01:04.491 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (13m) 2m ago 13m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:01:04.491 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (6m) 4m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:01:04.492 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:01:04.492 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:01:04.492 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:01:04.492 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (4m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:01:04.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:01:04.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3m) 2m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:01:04.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3m) 2m ago 13m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:01:04.493 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (12m) 2m ago 13m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:01:04.494 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 11m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:01:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:04 gibba032 bash[24004]: cluster 2021-03-10T14:01:02.740579+0000 mgr.gibba032.whopeo (mgr.24625) 473 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:04 gibba011 bash[30791]: cluster 2021-03-10T14:01:02.740579+0000 mgr.gibba032.whopeo (mgr.24625) 473 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:04 gibba011 bash[31774]: cluster 2021-03-10T14:01:02.740579+0000 mgr.gibba032.whopeo (mgr.24625) 473 : cluster [DBG] pgmap v258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:04.737 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:01:04.737 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:01:04.738 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:01:04.738 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:04.738 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:01:04.738 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:01:04.739 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:04.739 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:01:04.739 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:01:04.739 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:04.739 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:01:04.740 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:01:04.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:01:04.740 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:01:04.740 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:01:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:05 gibba032 bash[24004]: audit 2021-03-10T14:01:04.217083+0000 mgr.gibba032.whopeo (mgr.24625) 474 : audit [DBG] from='client.44379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:05 gibba032 bash[24004]: audit 2021-03-10T14:01:04.737112+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3200291197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:05 gibba011 bash[30791]: audit 2021-03-10T14:01:04.217083+0000 mgr.gibba032.whopeo (mgr.24625) 474 : audit [DBG] from='client.44379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:05 gibba011 bash[30791]: audit 2021-03-10T14:01:04.737112+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3200291197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:05 gibba011 bash[31774]: audit 2021-03-10T14:01:04.217083+0000 mgr.gibba032.whopeo (mgr.24625) 474 : audit [DBG] from='client.44379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:05 gibba011 bash[31774]: audit 2021-03-10T14:01:04.737112+0000 mon.c (mon.1) 12 : audit [DBG] from='client.? 172.21.2.111:0/3200291197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:06 gibba032 bash[24004]: audit 2021-03-10T14:01:04.449096+0000 mgr.gibba032.whopeo (mgr.24625) 475 : audit [DBG] from='client.54370 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:06 gibba032 bash[24004]: cluster 2021-03-10T14:01:04.741819+0000 mgr.gibba032.whopeo (mgr.24625) 476 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:06 gibba011 bash[30791]: audit 2021-03-10T14:01:04.449096+0000 mgr.gibba032.whopeo (mgr.24625) 475 : audit [DBG] from='client.54370 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:06 gibba011 bash[30791]: cluster 2021-03-10T14:01:04.741819+0000 mgr.gibba032.whopeo (mgr.24625) 476 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:06 gibba011 bash[31774]: audit 2021-03-10T14:01:04.449096+0000 mgr.gibba032.whopeo (mgr.24625) 475 : audit [DBG] from='client.54370 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:06 gibba011 bash[31774]: cluster 2021-03-10T14:01:04.741819+0000 mgr.gibba032.whopeo (mgr.24625) 476 : cluster [DBG] pgmap v259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:08.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.468380+0000 mgr.gibba032.whopeo (mgr.24625) 477 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:01:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.469922+0000 mgr.gibba032.whopeo (mgr.24625) 478 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:01:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.664499+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:01:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.666254+0000 mgr.gibba032.whopeo (mgr.24625) 479 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:01:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.667963+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:01:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.669747+0000 mgr.gibba032.whopeo (mgr.24625) 480 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:01:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.671064+0000 mon.a (mon.0) 709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.673098+0000 mgr.gibba032.whopeo (mgr.24625) 481 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:01:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.673140+0000 mgr.gibba032.whopeo (mgr.24625) 482 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:01:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.674452+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.675440+0000 mgr.gibba032.whopeo (mgr.24625) 483 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:01:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.675474+0000 mgr.gibba032.whopeo (mgr.24625) 484 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:01:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.676696+0000 mon.a (mon.0) 711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.677656+0000 mgr.gibba032.whopeo (mgr.24625) 485 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:01:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.679579+0000 mgr.gibba032.whopeo (mgr.24625) 486 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:01:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.679651+0000 mgr.gibba032.whopeo (mgr.24625) 487 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:01:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.680977+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.682225+0000 mgr.gibba032.whopeo (mgr.24625) 488 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:01:08.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.682619+0000 mgr.gibba032.whopeo (mgr.24625) 489 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:01:08.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: audit 2021-03-10T14:01:06.683791+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.684989+0000 mgr.gibba032.whopeo (mgr.24625) 490 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:01:08.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.685029+0000 mgr.gibba032.whopeo (mgr.24625) 491 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:01:08.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cephadm 2021-03-10T14:01:06.685221+0000 mgr.gibba032.whopeo (mgr.24625) 492 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:08.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:07 gibba032 bash[24004]: cluster 2021-03-10T14:01:06.742843+0000 mgr.gibba032.whopeo (mgr.24625) 493 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.468380+0000 mgr.gibba032.whopeo (mgr.24625) 477 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:01:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.469922+0000 mgr.gibba032.whopeo (mgr.24625) 478 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:01:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.664499+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:01:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.666254+0000 mgr.gibba032.whopeo (mgr.24625) 479 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:01:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.667963+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:01:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.669747+0000 mgr.gibba032.whopeo (mgr.24625) 480 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:01:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.671064+0000 mon.a (mon.0) 709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.673098+0000 mgr.gibba032.whopeo (mgr.24625) 481 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:01:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.673140+0000 mgr.gibba032.whopeo (mgr.24625) 482 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:01:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.674452+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.675440+0000 mgr.gibba032.whopeo (mgr.24625) 483 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:01:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.675474+0000 mgr.gibba032.whopeo (mgr.24625) 484 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:01:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.676696+0000 mon.a (mon.0) 711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.677656+0000 mgr.gibba032.whopeo (mgr.24625) 485 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:01:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.679579+0000 mgr.gibba032.whopeo (mgr.24625) 486 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:01:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.679651+0000 mgr.gibba032.whopeo (mgr.24625) 487 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:01:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.680977+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.682225+0000 mgr.gibba032.whopeo (mgr.24625) 488 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:01:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.682619+0000 mgr.gibba032.whopeo (mgr.24625) 489 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:01:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: audit 2021-03-10T14:01:06.683791+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.684989+0000 mgr.gibba032.whopeo (mgr.24625) 490 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:01:08.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.468380+0000 mgr.gibba032.whopeo (mgr.24625) 477 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:01:08.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.469922+0000 mgr.gibba032.whopeo (mgr.24625) 478 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:01:08.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.664499+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:01:08.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.666254+0000 mgr.gibba032.whopeo (mgr.24625) 479 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:01:08.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.667963+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:01:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.669747+0000 mgr.gibba032.whopeo (mgr.24625) 480 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:01:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.671064+0000 mon.a (mon.0) 709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.673098+0000 mgr.gibba032.whopeo (mgr.24625) 481 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:01:08.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.673140+0000 mgr.gibba032.whopeo (mgr.24625) 482 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:01:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.674452+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.675440+0000 mgr.gibba032.whopeo (mgr.24625) 483 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:01:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.675474+0000 mgr.gibba032.whopeo (mgr.24625) 484 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:01:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.676696+0000 mon.a (mon.0) 711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.677656+0000 mgr.gibba032.whopeo (mgr.24625) 485 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:01:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.679579+0000 mgr.gibba032.whopeo (mgr.24625) 486 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:01:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.679651+0000 mgr.gibba032.whopeo (mgr.24625) 487 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:01:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.680977+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.682225+0000 mgr.gibba032.whopeo (mgr.24625) 488 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:01:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.682619+0000 mgr.gibba032.whopeo (mgr.24625) 489 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:01:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: audit 2021-03-10T14:01:06.683791+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.684989+0000 mgr.gibba032.whopeo (mgr.24625) 490 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:01:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.685029+0000 mgr.gibba032.whopeo (mgr.24625) 491 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:01:08.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cephadm 2021-03-10T14:01:06.685221+0000 mgr.gibba032.whopeo (mgr.24625) 492 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:08.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[31774]: cluster 2021-03-10T14:01:06.742843+0000 mgr.gibba032.whopeo (mgr.24625) 493 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:08.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.685029+0000 mgr.gibba032.whopeo (mgr.24625) 491 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:01:08.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cephadm 2021-03-10T14:01:06.685221+0000 mgr.gibba032.whopeo (mgr.24625) 492 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:08.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:07 gibba011 bash[30791]: cluster 2021-03-10T14:01:06.742843+0000 mgr.gibba032.whopeo (mgr.24625) 493 : cluster [DBG] pgmap v260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:09 gibba011 bash[30791]: cluster 2021-03-10T14:01:08.744256+0000 mgr.gibba032.whopeo (mgr.24625) 494 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:09 gibba011 bash[31774]: cluster 2021-03-10T14:01:08.744256+0000 mgr.gibba032.whopeo (mgr.24625) 494 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:09 gibba032 bash[24004]: cluster 2021-03-10T14:01:08.744256+0000 mgr.gibba032.whopeo (mgr.24625) 494 : cluster [DBG] pgmap v261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:12 gibba011 bash[30791]: cluster 2021-03-10T14:01:10.745624+0000 mgr.gibba032.whopeo (mgr.24625) 495 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:12 gibba011 bash[31774]: cluster 2021-03-10T14:01:10.745624+0000 mgr.gibba032.whopeo (mgr.24625) 495 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:12 gibba032 bash[24004]: cluster 2021-03-10T14:01:10.745624+0000 mgr.gibba032.whopeo (mgr.24625) 495 : cluster [DBG] pgmap v262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:14 gibba011 bash[30791]: cluster 2021-03-10T14:01:12.747197+0000 mgr.gibba032.whopeo (mgr.24625) 496 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:14.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:14 gibba011 bash[31774]: cluster 2021-03-10T14:01:12.747197+0000 mgr.gibba032.whopeo (mgr.24625) 496 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:14 gibba032 bash[24004]: cluster 2021-03-10T14:01:12.747197+0000 mgr.gibba032.whopeo (mgr.24625) 496 : cluster [DBG] pgmap v263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:16.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:16 gibba011 bash[30791]: cluster 2021-03-10T14:01:14.748698+0000 mgr.gibba032.whopeo (mgr.24625) 497 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:16.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:16 gibba011 bash[31774]: cluster 2021-03-10T14:01:14.748698+0000 mgr.gibba032.whopeo (mgr.24625) 497 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:16 gibba032 bash[24004]: cluster 2021-03-10T14:01:14.748698+0000 mgr.gibba032.whopeo (mgr.24625) 497 : cluster [DBG] pgmap v264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:18 gibba011 bash[30791]: cluster 2021-03-10T14:01:16.750217+0000 mgr.gibba032.whopeo (mgr.24625) 498 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:18 gibba011 bash[31774]: cluster 2021-03-10T14:01:16.750217+0000 mgr.gibba032.whopeo (mgr.24625) 498 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:18 gibba032 bash[24004]: cluster 2021-03-10T14:01:16.750217+0000 mgr.gibba032.whopeo (mgr.24625) 498 : cluster [DBG] pgmap v265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:20 gibba011 bash[30791]: cluster 2021-03-10T14:01:18.751659+0000 mgr.gibba032.whopeo (mgr.24625) 499 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:20.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:20 gibba011 bash[31774]: cluster 2021-03-10T14:01:18.751659+0000 mgr.gibba032.whopeo (mgr.24625) 499 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:20 gibba032 bash[24004]: cluster 2021-03-10T14:01:18.751659+0000 mgr.gibba032.whopeo (mgr.24625) 499 : cluster [DBG] pgmap v266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:22 gibba011 bash[30791]: cluster 2021-03-10T14:01:20.753117+0000 mgr.gibba032.whopeo (mgr.24625) 500 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:22 gibba011 bash[31774]: cluster 2021-03-10T14:01:20.753117+0000 mgr.gibba032.whopeo (mgr.24625) 500 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:22 gibba032 bash[24004]: cluster 2021-03-10T14:01:20.753117+0000 mgr.gibba032.whopeo (mgr.24625) 500 : cluster [DBG] pgmap v267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:23 gibba011 bash[30791]: cephadm 2021-03-10T14:01:21.701262+0000 mgr.gibba032.whopeo (mgr.24625) 501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:23.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:23 gibba011 bash[31774]: cephadm 2021-03-10T14:01:21.701262+0000 mgr.gibba032.whopeo (mgr.24625) 501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:23 gibba032 bash[24004]: cephadm 2021-03-10T14:01:21.701262+0000 mgr.gibba032.whopeo (mgr.24625) 501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:24 gibba011 bash[30791]: cluster 2021-03-10T14:01:22.754672+0000 mgr.gibba032.whopeo (mgr.24625) 502 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:24 gibba011 bash[31774]: cluster 2021-03-10T14:01:22.754672+0000 mgr.gibba032.whopeo (mgr.24625) 502 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:24 gibba032 bash[24004]: cluster 2021-03-10T14:01:22.754672+0000 mgr.gibba032.whopeo (mgr.24625) 502 : cluster [DBG] pgmap v268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:26 gibba032 bash[24004]: cluster 2021-03-10T14:01:24.756296+0000 mgr.gibba032.whopeo (mgr.24625) 503 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:26 gibba011 bash[30791]: cluster 2021-03-10T14:01:24.756296+0000 mgr.gibba032.whopeo (mgr.24625) 503 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:26 gibba011 bash[31774]: cluster 2021-03-10T14:01:24.756296+0000 mgr.gibba032.whopeo (mgr.24625) 503 : cluster [DBG] pgmap v269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:28 gibba032 bash[24004]: cluster 2021-03-10T14:01:26.757973+0000 mgr.gibba032.whopeo (mgr.24625) 504 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:28 gibba011 bash[30791]: cluster 2021-03-10T14:01:26.757973+0000 mgr.gibba032.whopeo (mgr.24625) 504 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:28 gibba011 bash[31774]: cluster 2021-03-10T14:01:26.757973+0000 mgr.gibba032.whopeo (mgr.24625) 504 : cluster [DBG] pgmap v270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:30 gibba032 bash[24004]: cluster 2021-03-10T14:01:28.759353+0000 mgr.gibba032.whopeo (mgr.24625) 505 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:30 gibba011 bash[30791]: cluster 2021-03-10T14:01:28.759353+0000 mgr.gibba032.whopeo (mgr.24625) 505 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:30 gibba011 bash[31774]: cluster 2021-03-10T14:01:28.759353+0000 mgr.gibba032.whopeo (mgr.24625) 505 : cluster [DBG] pgmap v271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:32 gibba032 bash[24004]: cluster 2021-03-10T14:01:30.760675+0000 mgr.gibba032.whopeo (mgr.24625) 506 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:32 gibba011 bash[30791]: cluster 2021-03-10T14:01:30.760675+0000 mgr.gibba032.whopeo (mgr.24625) 506 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:32 gibba011 bash[31774]: cluster 2021-03-10T14:01:30.760675+0000 mgr.gibba032.whopeo (mgr.24625) 506 : cluster [DBG] pgmap v272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:34 gibba032 bash[24004]: cluster 2021-03-10T14:01:32.762291+0000 mgr.gibba032.whopeo (mgr.24625) 507 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:34 gibba011 bash[30791]: cluster 2021-03-10T14:01:32.762291+0000 mgr.gibba032.whopeo (mgr.24625) 507 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:34 gibba011 bash[31774]: cluster 2021-03-10T14:01:32.762291+0000 mgr.gibba032.whopeo (mgr.24625) 507 : cluster [DBG] pgmap v273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:34.980 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:01:35.216 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:01:35.216 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (8m) 5m ago 13m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:01:35.217 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (13m) 3m ago 13m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:01:35.217 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (8m) 3m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:01:35.217 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (8m) 3m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:01:35.217 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (7m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:01:35.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (7m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:01:35.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (7m) 3m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:01:35.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (7m) 5m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:01:35.218 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (13m) 5m ago 13m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:01:35.219 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (13m) 3m ago 13m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:01:35.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:01:35.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:01:35.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (5m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:01:35.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (5m) 5m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:01:35.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (4m) 3m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:01:35.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (4m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:01:35.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:01:35.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3m) 3m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:01:35.220 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (13m) 3m ago 13m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:01:35.221 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 11m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:01:35.464 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:01:35.464 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:01:35.464 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:01:35.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:35.465 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:01:35.465 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:01:35.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:35.466 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:01:35.466 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:01:35.466 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:01:35.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:01:35.467 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:01:35.467 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:01:35.467 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:01:35.467 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:01:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:36 gibba032 bash[24004]: cluster 2021-03-10T14:01:34.763617+0000 mgr.gibba032.whopeo (mgr.24625) 508 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:36 gibba032 bash[24004]: audit 2021-03-10T14:01:34.969817+0000 mgr.gibba032.whopeo (mgr.24625) 509 : audit [DBG] from='client.44397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:36 gibba032 bash[24004]: audit 2021-03-10T14:01:35.194061+0000 mgr.gibba032.whopeo (mgr.24625) 510 : audit [DBG] from='client.54388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:36 gibba032 bash[24004]: audit 2021-03-10T14:01:35.463575+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/1310301345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[30791]: cluster 2021-03-10T14:01:34.763617+0000 mgr.gibba032.whopeo (mgr.24625) 508 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[30791]: audit 2021-03-10T14:01:34.969817+0000 mgr.gibba032.whopeo (mgr.24625) 509 : audit [DBG] from='client.44397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[30791]: audit 2021-03-10T14:01:35.194061+0000 mgr.gibba032.whopeo (mgr.24625) 510 : audit [DBG] from='client.54388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[30791]: audit 2021-03-10T14:01:35.463575+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/1310301345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[31774]: cluster 2021-03-10T14:01:34.763617+0000 mgr.gibba032.whopeo (mgr.24625) 508 : cluster [DBG] pgmap v274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[31774]: audit 2021-03-10T14:01:34.969817+0000 mgr.gibba032.whopeo (mgr.24625) 509 : audit [DBG] from='client.44397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[31774]: audit 2021-03-10T14:01:35.194061+0000 mgr.gibba032.whopeo (mgr.24625) 510 : audit [DBG] from='client.54388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:01:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:36 gibba011 bash[31774]: audit 2021-03-10T14:01:35.463575+0000 mon.c (mon.1) 13 : audit [DBG] from='client.? 172.21.2.111:0/1310301345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:01:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:38 gibba032 bash[24004]: cephadm 2021-03-10T14:01:36.717309+0000 mgr.gibba032.whopeo (mgr.24625) 511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:38 gibba032 bash[24004]: cluster 2021-03-10T14:01:36.765206+0000 mgr.gibba032.whopeo (mgr.24625) 512 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:38 gibba011 bash[30791]: cephadm 2021-03-10T14:01:36.717309+0000 mgr.gibba032.whopeo (mgr.24625) 511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:38 gibba011 bash[30791]: cluster 2021-03-10T14:01:36.765206+0000 mgr.gibba032.whopeo (mgr.24625) 512 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:38 gibba011 bash[31774]: cephadm 2021-03-10T14:01:36.717309+0000 mgr.gibba032.whopeo (mgr.24625) 511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:38 gibba011 bash[31774]: cluster 2021-03-10T14:01:36.765206+0000 mgr.gibba032.whopeo (mgr.24625) 512 : cluster [DBG] pgmap v275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:40 gibba032 bash[24004]: cluster 2021-03-10T14:01:38.766806+0000 mgr.gibba032.whopeo (mgr.24625) 513 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:40 gibba011 bash[30791]: cluster 2021-03-10T14:01:38.766806+0000 mgr.gibba032.whopeo (mgr.24625) 513 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:40 gibba011 bash[31774]: cluster 2021-03-10T14:01:38.766806+0000 mgr.gibba032.whopeo (mgr.24625) 513 : cluster [DBG] pgmap v276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:42 gibba032 bash[24004]: cluster 2021-03-10T14:01:40.768165+0000 mgr.gibba032.whopeo (mgr.24625) 514 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:42 gibba011 bash[30791]: cluster 2021-03-10T14:01:40.768165+0000 mgr.gibba032.whopeo (mgr.24625) 514 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:42 gibba011 bash[31774]: cluster 2021-03-10T14:01:40.768165+0000 mgr.gibba032.whopeo (mgr.24625) 514 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:44 gibba032 bash[24004]: cluster 2021-03-10T14:01:42.769914+0000 mgr.gibba032.whopeo (mgr.24625) 515 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:44 gibba011 bash[30791]: cluster 2021-03-10T14:01:42.769914+0000 mgr.gibba032.whopeo (mgr.24625) 515 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:44 gibba011 bash[31774]: cluster 2021-03-10T14:01:42.769914+0000 mgr.gibba032.whopeo (mgr.24625) 515 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:46 gibba032 bash[24004]: cluster 2021-03-10T14:01:44.771289+0000 mgr.gibba032.whopeo (mgr.24625) 516 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:46 gibba011 bash[30791]: cluster 2021-03-10T14:01:44.771289+0000 mgr.gibba032.whopeo (mgr.24625) 516 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:46 gibba011 bash[31774]: cluster 2021-03-10T14:01:44.771289+0000 mgr.gibba032.whopeo (mgr.24625) 516 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:48 gibba032 bash[24004]: cluster 2021-03-10T14:01:46.773053+0000 mgr.gibba032.whopeo (mgr.24625) 517 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:48 gibba011 bash[30791]: cluster 2021-03-10T14:01:46.773053+0000 mgr.gibba032.whopeo (mgr.24625) 517 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:48 gibba011 bash[31774]: cluster 2021-03-10T14:01:46.773053+0000 mgr.gibba032.whopeo (mgr.24625) 517 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:50 gibba032 bash[24004]: cluster 2021-03-10T14:01:48.774612+0000 mgr.gibba032.whopeo (mgr.24625) 518 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:50 gibba011 bash[30791]: cluster 2021-03-10T14:01:48.774612+0000 mgr.gibba032.whopeo (mgr.24625) 518 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:50 gibba011 bash[31774]: cluster 2021-03-10T14:01:48.774612+0000 mgr.gibba032.whopeo (mgr.24625) 518 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:52 gibba032 bash[24004]: cluster 2021-03-10T14:01:50.775941+0000 mgr.gibba032.whopeo (mgr.24625) 519 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:52 gibba011 bash[30791]: cluster 2021-03-10T14:01:50.775941+0000 mgr.gibba032.whopeo (mgr.24625) 519 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:52 gibba011 bash[31774]: cluster 2021-03-10T14:01:50.775941+0000 mgr.gibba032.whopeo (mgr.24625) 519 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:53 gibba032 bash[24004]: cephadm 2021-03-10T14:01:51.733343+0000 mgr.gibba032.whopeo (mgr.24625) 520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:53.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:53 gibba011 bash[30791]: cephadm 2021-03-10T14:01:51.733343+0000 mgr.gibba032.whopeo (mgr.24625) 520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:53.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:53 gibba011 bash[31774]: cephadm 2021-03-10T14:01:51.733343+0000 mgr.gibba032.whopeo (mgr.24625) 520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:01:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:54 gibba032 bash[24004]: cluster 2021-03-10T14:01:52.777501+0000 mgr.gibba032.whopeo (mgr.24625) 521 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:54 gibba011 bash[30791]: cluster 2021-03-10T14:01:52.777501+0000 mgr.gibba032.whopeo (mgr.24625) 521 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:54 gibba011 bash[31774]: cluster 2021-03-10T14:01:52.777501+0000 mgr.gibba032.whopeo (mgr.24625) 521 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: cluster 2021-03-10T14:01:54.778993+0000 mgr.gibba032.whopeo (mgr.24625) 522 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:55.924013+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:01:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:55.924772+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:01:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:55.925502+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:55.926215+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:56.027258+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:01:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:56 gibba032 bash[24004]: audit 2021-03-10T14:01:56.027905+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:01:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: cluster 2021-03-10T14:01:54.778993+0000 mgr.gibba032.whopeo (mgr.24625) 522 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:55.924013+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:01:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:55.924772+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:01:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:55.925502+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:55.926215+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:56.027258+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:01:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[30791]: audit 2021-03-10T14:01:56.027905+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:01:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: cluster 2021-03-10T14:01:54.778993+0000 mgr.gibba032.whopeo (mgr.24625) 522 : cluster [DBG] pgmap v284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:55.924013+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:01:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:55.924772+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:01:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:55.925502+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:55.926215+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:01:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:56.027258+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:01:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:56 gibba011 bash[31774]: audit 2021-03-10T14:01:56.027905+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:01:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:01:58 gibba032 bash[24004]: cluster 2021-03-10T14:01:56.780663+0000 mgr.gibba032.whopeo (mgr.24625) 523 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:01:58 gibba011 bash[30791]: cluster 2021-03-10T14:01:56.780663+0000 mgr.gibba032.whopeo (mgr.24625) 523 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:01:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:01:58 gibba011 bash[31774]: cluster 2021-03-10T14:01:56.780663+0000 mgr.gibba032.whopeo (mgr.24625) 523 : cluster [DBG] pgmap v285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:00 gibba032 bash[24004]: cluster 2021-03-10T14:01:58.782312+0000 mgr.gibba032.whopeo (mgr.24625) 524 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:00 gibba011 bash[30791]: cluster 2021-03-10T14:01:58.782312+0000 mgr.gibba032.whopeo (mgr.24625) 524 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:00 gibba011 bash[31774]: cluster 2021-03-10T14:01:58.782312+0000 mgr.gibba032.whopeo (mgr.24625) 524 : cluster [DBG] pgmap v286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:02 gibba032 bash[24004]: cluster 2021-03-10T14:02:00.783615+0000 mgr.gibba032.whopeo (mgr.24625) 525 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:02 gibba011 bash[30791]: cluster 2021-03-10T14:02:00.783615+0000 mgr.gibba032.whopeo (mgr.24625) 525 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:02 gibba011 bash[31774]: cluster 2021-03-10T14:02:00.783615+0000 mgr.gibba032.whopeo (mgr.24625) 525 : cluster [DBG] pgmap v287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:04 gibba011 bash[30791]: cluster 2021-03-10T14:02:02.785246+0000 mgr.gibba032.whopeo (mgr.24625) 526 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:04 gibba011 bash[31774]: cluster 2021-03-10T14:02:02.785246+0000 mgr.gibba032.whopeo (mgr.24625) 526 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:04 gibba032 bash[24004]: cluster 2021-03-10T14:02:02.785246+0000 mgr.gibba032.whopeo (mgr.24625) 526 : cluster [DBG] pgmap v288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:05.707 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:02:05.962 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:02:05.963 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (8m) 5m ago 14m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:02:05.963 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (13m) 4m ago 13m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:02:05.963 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (9m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:02:05.964 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (9m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:02:05.964 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (8m) 5m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:02:05.964 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (8m) 5m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:02:05.964 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (7m) 4m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:02:05.965 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (8m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:02:05.965 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (14m) 5m ago 14m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:02:05.965 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (14m) 4m ago 14m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:02:05.965 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (7m) 5m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:02:05.965 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:02:05.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:02:05.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (5m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:02:05.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:02:05.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:02:05.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:02:05.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (4m) 4m ago 14m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:02:05.967 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (13m) 4m ago 14m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:02:05.967 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 12m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:02:06.218 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:02:06.219 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:02:06.219 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:02:06.219 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:06.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:02:06.220 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:02:06.220 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:06.220 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:02:06.220 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:02:06.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:06.221 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:02:06.221 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:02:06.221 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:02:06.222 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:02:06.222 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:02:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:06 gibba011 bash[30791]: cluster 2021-03-10T14:02:04.786776+0000 mgr.gibba032.whopeo (mgr.24625) 527 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:06 gibba011 bash[30791]: audit 2021-03-10T14:02:06.218097+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/2237302704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:06 gibba011 bash[31774]: cluster 2021-03-10T14:02:04.786776+0000 mgr.gibba032.whopeo (mgr.24625) 527 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:06 gibba011 bash[31774]: audit 2021-03-10T14:02:06.218097+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/2237302704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:06 gibba032 bash[24004]: cluster 2021-03-10T14:02:04.786776+0000 mgr.gibba032.whopeo (mgr.24625) 527 : cluster [DBG] pgmap v289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:06 gibba032 bash[24004]: audit 2021-03-10T14:02:06.218097+0000 mon.c (mon.1) 14 : audit [DBG] from='client.? 172.21.2.111:0/2237302704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:05.696910+0000 mgr.gibba032.whopeo (mgr.24625) 528 : audit [DBG] from='client.54400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:05.921310+0000 mgr.gibba032.whopeo (mgr.24625) 529 : audit [DBG] from='client.54406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.946405+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:02:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.953728+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:02:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.955671+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.957037+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.958385+0000 mon.a (mon.0) 724 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.960768+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[30791]: audit 2021-03-10T14:02:06.962507+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:05.696910+0000 mgr.gibba032.whopeo (mgr.24625) 528 : audit [DBG] from='client.54400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:05.921310+0000 mgr.gibba032.whopeo (mgr.24625) 529 : audit [DBG] from='client.54406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.946405+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:02:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.953728+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:02:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.955671+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.957037+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.958385+0000 mon.a (mon.0) 724 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.960768+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:07 gibba011 bash[31774]: audit 2021-03-10T14:02:06.962507+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:05.696910+0000 mgr.gibba032.whopeo (mgr.24625) 528 : audit [DBG] from='client.54400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:05.921310+0000 mgr.gibba032.whopeo (mgr.24625) 529 : audit [DBG] from='client.54406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.946405+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:02:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.953728+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:02:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.955671+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.957037+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.958385+0000 mon.a (mon.0) 724 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.960768+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:07 gibba032 bash[24004]: audit 2021-03-10T14:02:06.962507+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.737493+0000 mgr.gibba032.whopeo (mgr.24625) 530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:02:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.739169+0000 mgr.gibba032.whopeo (mgr.24625) 531 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:02:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cluster 2021-03-10T14:02:06.787342+0000 mgr.gibba032.whopeo (mgr.24625) 532 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.952146+0000 mgr.gibba032.whopeo (mgr.24625) 533 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:02:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.954392+0000 mgr.gibba032.whopeo (mgr.24625) 534 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:02:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.955708+0000 mgr.gibba032.whopeo (mgr.24625) 535 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:02:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.955746+0000 mgr.gibba032.whopeo (mgr.24625) 536 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:02:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.957200+0000 mgr.gibba032.whopeo (mgr.24625) 537 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:02:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.957236+0000 mgr.gibba032.whopeo (mgr.24625) 538 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:02:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.958345+0000 mgr.gibba032.whopeo (mgr.24625) 539 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:02:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.959353+0000 mgr.gibba032.whopeo (mgr.24625) 540 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:02:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.959390+0000 mgr.gibba032.whopeo (mgr.24625) 541 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:02:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.960808+0000 mgr.gibba032.whopeo (mgr.24625) 542 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:02:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.961184+0000 mgr.gibba032.whopeo (mgr.24625) 543 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:02:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.962999+0000 mgr.gibba032.whopeo (mgr.24625) 544 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:02:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.963044+0000 mgr.gibba032.whopeo (mgr.24625) 545 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:02:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[30791]: cephadm 2021-03-10T14:02:06.963152+0000 mgr.gibba032.whopeo (mgr.24625) 546 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:08.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.737493+0000 mgr.gibba032.whopeo (mgr.24625) 530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:02:08.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.739169+0000 mgr.gibba032.whopeo (mgr.24625) 531 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:02:08.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cluster 2021-03-10T14:02:06.787342+0000 mgr.gibba032.whopeo (mgr.24625) 532 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:08.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.952146+0000 mgr.gibba032.whopeo (mgr.24625) 533 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:02:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.954392+0000 mgr.gibba032.whopeo (mgr.24625) 534 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:02:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.955708+0000 mgr.gibba032.whopeo (mgr.24625) 535 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:02:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.955746+0000 mgr.gibba032.whopeo (mgr.24625) 536 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:02:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.957200+0000 mgr.gibba032.whopeo (mgr.24625) 537 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:02:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.957236+0000 mgr.gibba032.whopeo (mgr.24625) 538 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:02:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.958345+0000 mgr.gibba032.whopeo (mgr.24625) 539 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:02:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.959353+0000 mgr.gibba032.whopeo (mgr.24625) 540 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:02:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.959390+0000 mgr.gibba032.whopeo (mgr.24625) 541 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:02:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.960808+0000 mgr.gibba032.whopeo (mgr.24625) 542 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:02:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.961184+0000 mgr.gibba032.whopeo (mgr.24625) 543 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:02:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.962999+0000 mgr.gibba032.whopeo (mgr.24625) 544 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:02:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.963044+0000 mgr.gibba032.whopeo (mgr.24625) 545 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:02:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:08 gibba011 bash[31774]: cephadm 2021-03-10T14:02:06.963152+0000 mgr.gibba032.whopeo (mgr.24625) 546 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.737493+0000 mgr.gibba032.whopeo (mgr.24625) 530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:02:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.739169+0000 mgr.gibba032.whopeo (mgr.24625) 531 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:02:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cluster 2021-03-10T14:02:06.787342+0000 mgr.gibba032.whopeo (mgr.24625) 532 : cluster [DBG] pgmap v290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.952146+0000 mgr.gibba032.whopeo (mgr.24625) 533 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:02:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.954392+0000 mgr.gibba032.whopeo (mgr.24625) 534 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:02:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.955708+0000 mgr.gibba032.whopeo (mgr.24625) 535 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:02:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.955746+0000 mgr.gibba032.whopeo (mgr.24625) 536 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:02:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.957200+0000 mgr.gibba032.whopeo (mgr.24625) 537 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:02:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.957236+0000 mgr.gibba032.whopeo (mgr.24625) 538 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:02:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.958345+0000 mgr.gibba032.whopeo (mgr.24625) 539 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:02:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.959353+0000 mgr.gibba032.whopeo (mgr.24625) 540 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:02:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.959390+0000 mgr.gibba032.whopeo (mgr.24625) 541 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:02:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.960808+0000 mgr.gibba032.whopeo (mgr.24625) 542 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:02:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.961184+0000 mgr.gibba032.whopeo (mgr.24625) 543 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:02:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.962999+0000 mgr.gibba032.whopeo (mgr.24625) 544 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:02:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.963044+0000 mgr.gibba032.whopeo (mgr.24625) 545 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:02:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:08 gibba032 bash[24004]: cephadm 2021-03-10T14:02:06.963152+0000 mgr.gibba032.whopeo (mgr.24625) 546 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:10 gibba011 bash[30791]: cluster 2021-03-10T14:02:08.788892+0000 mgr.gibba032.whopeo (mgr.24625) 547 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:10 gibba011 bash[31774]: cluster 2021-03-10T14:02:08.788892+0000 mgr.gibba032.whopeo (mgr.24625) 547 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:10 gibba032 bash[24004]: cluster 2021-03-10T14:02:08.788892+0000 mgr.gibba032.whopeo (mgr.24625) 547 : cluster [DBG] pgmap v291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:12 gibba011 bash[30791]: cluster 2021-03-10T14:02:10.790393+0000 mgr.gibba032.whopeo (mgr.24625) 548 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:12 gibba011 bash[31774]: cluster 2021-03-10T14:02:10.790393+0000 mgr.gibba032.whopeo (mgr.24625) 548 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:12 gibba032 bash[24004]: cluster 2021-03-10T14:02:10.790393+0000 mgr.gibba032.whopeo (mgr.24625) 548 : cluster [DBG] pgmap v292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:14 gibba011 bash[30791]: cluster 2021-03-10T14:02:12.792110+0000 mgr.gibba032.whopeo (mgr.24625) 549 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:14.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:14 gibba011 bash[31774]: cluster 2021-03-10T14:02:12.792110+0000 mgr.gibba032.whopeo (mgr.24625) 549 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:14 gibba032 bash[24004]: cluster 2021-03-10T14:02:12.792110+0000 mgr.gibba032.whopeo (mgr.24625) 549 : cluster [DBG] pgmap v293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:16 gibba011 bash[30791]: cluster 2021-03-10T14:02:14.793377+0000 mgr.gibba032.whopeo (mgr.24625) 550 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:16 gibba011 bash[31774]: cluster 2021-03-10T14:02:14.793377+0000 mgr.gibba032.whopeo (mgr.24625) 550 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:16 gibba032 bash[24004]: cluster 2021-03-10T14:02:14.793377+0000 mgr.gibba032.whopeo (mgr.24625) 550 : cluster [DBG] pgmap v294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:18 gibba011 bash[30791]: cluster 2021-03-10T14:02:16.794953+0000 mgr.gibba032.whopeo (mgr.24625) 551 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:18 gibba011 bash[31774]: cluster 2021-03-10T14:02:16.794953+0000 mgr.gibba032.whopeo (mgr.24625) 551 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:18 gibba032 bash[24004]: cluster 2021-03-10T14:02:16.794953+0000 mgr.gibba032.whopeo (mgr.24625) 551 : cluster [DBG] pgmap v295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:20 gibba032 bash[24004]: cluster 2021-03-10T14:02:18.796472+0000 mgr.gibba032.whopeo (mgr.24625) 552 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:20 gibba011 bash[30791]: cluster 2021-03-10T14:02:18.796472+0000 mgr.gibba032.whopeo (mgr.24625) 552 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:20 gibba011 bash[31774]: cluster 2021-03-10T14:02:18.796472+0000 mgr.gibba032.whopeo (mgr.24625) 552 : cluster [DBG] pgmap v296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:22 gibba032 bash[24004]: cluster 2021-03-10T14:02:20.797762+0000 mgr.gibba032.whopeo (mgr.24625) 553 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:22 gibba011 bash[30791]: cluster 2021-03-10T14:02:20.797762+0000 mgr.gibba032.whopeo (mgr.24625) 553 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:22 gibba011 bash[31774]: cluster 2021-03-10T14:02:20.797762+0000 mgr.gibba032.whopeo (mgr.24625) 553 : cluster [DBG] pgmap v297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:23 gibba032 bash[24004]: cephadm 2021-03-10T14:02:21.979049+0000 mgr.gibba032.whopeo (mgr.24625) 554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:23.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:23 gibba011 bash[30791]: cephadm 2021-03-10T14:02:21.979049+0000 mgr.gibba032.whopeo (mgr.24625) 554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:23.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:23 gibba011 bash[31774]: cephadm 2021-03-10T14:02:21.979049+0000 mgr.gibba032.whopeo (mgr.24625) 554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:24 gibba032 bash[24004]: cluster 2021-03-10T14:02:22.799400+0000 mgr.gibba032.whopeo (mgr.24625) 555 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:24 gibba011 bash[30791]: cluster 2021-03-10T14:02:22.799400+0000 mgr.gibba032.whopeo (mgr.24625) 555 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:24 gibba011 bash[31774]: cluster 2021-03-10T14:02:22.799400+0000 mgr.gibba032.whopeo (mgr.24625) 555 : cluster [DBG] pgmap v298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:26 gibba032 bash[24004]: cluster 2021-03-10T14:02:24.800814+0000 mgr.gibba032.whopeo (mgr.24625) 556 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:26 gibba011 bash[30791]: cluster 2021-03-10T14:02:24.800814+0000 mgr.gibba032.whopeo (mgr.24625) 556 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:26 gibba011 bash[31774]: cluster 2021-03-10T14:02:24.800814+0000 mgr.gibba032.whopeo (mgr.24625) 556 : cluster [DBG] pgmap v299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:28 gibba032 bash[24004]: cluster 2021-03-10T14:02:26.802319+0000 mgr.gibba032.whopeo (mgr.24625) 557 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:28 gibba011 bash[30791]: cluster 2021-03-10T14:02:26.802319+0000 mgr.gibba032.whopeo (mgr.24625) 557 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:28.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:28 gibba011 bash[31774]: cluster 2021-03-10T14:02:26.802319+0000 mgr.gibba032.whopeo (mgr.24625) 557 : cluster [DBG] pgmap v300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:30 gibba032 bash[24004]: cluster 2021-03-10T14:02:28.803809+0000 mgr.gibba032.whopeo (mgr.24625) 558 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:30 gibba011 bash[30791]: cluster 2021-03-10T14:02:28.803809+0000 mgr.gibba032.whopeo (mgr.24625) 558 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:30 gibba011 bash[31774]: cluster 2021-03-10T14:02:28.803809+0000 mgr.gibba032.whopeo (mgr.24625) 558 : cluster [DBG] pgmap v301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:32 gibba032 bash[24004]: cluster 2021-03-10T14:02:30.805161+0000 mgr.gibba032.whopeo (mgr.24625) 559 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:32 gibba011 bash[30791]: cluster 2021-03-10T14:02:30.805161+0000 mgr.gibba032.whopeo (mgr.24625) 559 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:32 gibba011 bash[31774]: cluster 2021-03-10T14:02:30.805161+0000 mgr.gibba032.whopeo (mgr.24625) 559 : cluster [DBG] pgmap v302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:34 gibba032 bash[24004]: cluster 2021-03-10T14:02:32.806773+0000 mgr.gibba032.whopeo (mgr.24625) 560 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:34 gibba011 bash[30791]: cluster 2021-03-10T14:02:32.806773+0000 mgr.gibba032.whopeo (mgr.24625) 560 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:34 gibba011 bash[31774]: cluster 2021-03-10T14:02:32.806773+0000 mgr.gibba032.whopeo (mgr.24625) 560 : cluster [DBG] pgmap v303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:36.462 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:02:36.690 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:02:36.691 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (9m) 6m ago 14m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:02:36.691 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (14m) 4m ago 14m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:02:36.691 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (9m) 4m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:02:36.691 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (9m) 4m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:02:36.692 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (8m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:02:36.692 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (8m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:02:36.692 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (8m) 4m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:02:36.692 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (8m) 6m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:02:36.692 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (14m) 6m ago 14m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:02:36.693 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (14m) 4m ago 14m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:02:36.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:02:36.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:02:36.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (6m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:02:36.694 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (6m) 6m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:02:36.694 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (5m) 4m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:02:36.694 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (5m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:02:36.694 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:02:36.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (4m) 4m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:02:36.695 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (14m) 4m ago 14m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:02:36.695 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 12m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:02:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:36 gibba032 bash[24004]: cluster 2021-03-10T14:02:34.808190+0000 mgr.gibba032.whopeo (mgr.24625) 561 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:36 gibba011 bash[31774]: cluster 2021-03-10T14:02:34.808190+0000 mgr.gibba032.whopeo (mgr.24625) 561 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:36 gibba011 bash[30791]: cluster 2021-03-10T14:02:34.808190+0000 mgr.gibba032.whopeo (mgr.24625) 561 : cluster [DBG] pgmap v304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:36.937 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:02:36.937 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:02:36.937 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:02:36.938 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:36.938 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:02:36.938 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:02:36.938 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:36.938 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:02:36.939 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:02:36.939 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:02:36.939 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:02:36.939 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:02:36.939 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:02:36.940 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:02:36.940 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:02:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:37 gibba032 bash[24004]: audit 2021-03-10T14:02:36.451606+0000 mgr.gibba032.whopeo (mgr.24625) 562 : audit [DBG] from='client.54418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:37 gibba032 bash[24004]: audit 2021-03-10T14:02:36.936841+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? 172.21.2.111:0/3377133802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:37.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:37 gibba011 bash[30791]: audit 2021-03-10T14:02:36.451606+0000 mgr.gibba032.whopeo (mgr.24625) 562 : audit [DBG] from='client.54418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:37 gibba011 bash[30791]: audit 2021-03-10T14:02:36.936841+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? 172.21.2.111:0/3377133802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:37.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:37 gibba011 bash[31774]: audit 2021-03-10T14:02:36.451606+0000 mgr.gibba032.whopeo (mgr.24625) 562 : audit [DBG] from='client.54418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:37.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:37 gibba011 bash[31774]: audit 2021-03-10T14:02:36.936841+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? 172.21.2.111:0/3377133802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:02:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:38 gibba032 bash[24004]: audit 2021-03-10T14:02:36.676449+0000 mgr.gibba032.whopeo (mgr.24625) 563 : audit [DBG] from='client.54424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:38 gibba032 bash[24004]: cluster 2021-03-10T14:02:36.809911+0000 mgr.gibba032.whopeo (mgr.24625) 564 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:38 gibba032 bash[24004]: cephadm 2021-03-10T14:02:36.992782+0000 mgr.gibba032.whopeo (mgr.24625) 565 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[30791]: audit 2021-03-10T14:02:36.676449+0000 mgr.gibba032.whopeo (mgr.24625) 563 : audit [DBG] from='client.54424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[30791]: cluster 2021-03-10T14:02:36.809911+0000 mgr.gibba032.whopeo (mgr.24625) 564 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[30791]: cephadm 2021-03-10T14:02:36.992782+0000 mgr.gibba032.whopeo (mgr.24625) 565 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[31774]: audit 2021-03-10T14:02:36.676449+0000 mgr.gibba032.whopeo (mgr.24625) 563 : audit [DBG] from='client.54424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:02:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[31774]: cluster 2021-03-10T14:02:36.809911+0000 mgr.gibba032.whopeo (mgr.24625) 564 : cluster [DBG] pgmap v305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:38 gibba011 bash[31774]: cephadm 2021-03-10T14:02:36.992782+0000 mgr.gibba032.whopeo (mgr.24625) 565 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:40 gibba032 bash[24004]: cluster 2021-03-10T14:02:38.811373+0000 mgr.gibba032.whopeo (mgr.24625) 566 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:40 gibba011 bash[30791]: cluster 2021-03-10T14:02:38.811373+0000 mgr.gibba032.whopeo (mgr.24625) 566 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:40 gibba011 bash[31774]: cluster 2021-03-10T14:02:38.811373+0000 mgr.gibba032.whopeo (mgr.24625) 566 : cluster [DBG] pgmap v306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:42 gibba032 bash[24004]: cluster 2021-03-10T14:02:40.812747+0000 mgr.gibba032.whopeo (mgr.24625) 567 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:42 gibba011 bash[30791]: cluster 2021-03-10T14:02:40.812747+0000 mgr.gibba032.whopeo (mgr.24625) 567 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:42 gibba011 bash[31774]: cluster 2021-03-10T14:02:40.812747+0000 mgr.gibba032.whopeo (mgr.24625) 567 : cluster [DBG] pgmap v307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:44 gibba032 bash[24004]: cluster 2021-03-10T14:02:42.814373+0000 mgr.gibba032.whopeo (mgr.24625) 568 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:44 gibba011 bash[30791]: cluster 2021-03-10T14:02:42.814373+0000 mgr.gibba032.whopeo (mgr.24625) 568 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:44 gibba011 bash[31774]: cluster 2021-03-10T14:02:42.814373+0000 mgr.gibba032.whopeo (mgr.24625) 568 : cluster [DBG] pgmap v308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:46 gibba032 bash[24004]: cluster 2021-03-10T14:02:44.815705+0000 mgr.gibba032.whopeo (mgr.24625) 569 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:46.997 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:46 gibba011 bash[30791]: cluster 2021-03-10T14:02:44.815705+0000 mgr.gibba032.whopeo (mgr.24625) 569 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:46.998 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:46 gibba011 bash[31774]: cluster 2021-03-10T14:02:44.815705+0000 mgr.gibba032.whopeo (mgr.24625) 569 : cluster [DBG] pgmap v309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:48 gibba032 bash[24004]: cluster 2021-03-10T14:02:46.817457+0000 mgr.gibba032.whopeo (mgr.24625) 570 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:48 gibba011 bash[30791]: cluster 2021-03-10T14:02:46.817457+0000 mgr.gibba032.whopeo (mgr.24625) 570 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:48 gibba011 bash[31774]: cluster 2021-03-10T14:02:46.817457+0000 mgr.gibba032.whopeo (mgr.24625) 570 : cluster [DBG] pgmap v310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:50 gibba032 bash[24004]: cluster 2021-03-10T14:02:48.819011+0000 mgr.gibba032.whopeo (mgr.24625) 571 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:50 gibba011 bash[30791]: cluster 2021-03-10T14:02:48.819011+0000 mgr.gibba032.whopeo (mgr.24625) 571 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:50 gibba011 bash[31774]: cluster 2021-03-10T14:02:48.819011+0000 mgr.gibba032.whopeo (mgr.24625) 571 : cluster [DBG] pgmap v311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:52 gibba032 bash[24004]: cluster 2021-03-10T14:02:50.820363+0000 mgr.gibba032.whopeo (mgr.24625) 572 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:52 gibba011 bash[30791]: cluster 2021-03-10T14:02:50.820363+0000 mgr.gibba032.whopeo (mgr.24625) 572 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:52 gibba011 bash[31774]: cluster 2021-03-10T14:02:50.820363+0000 mgr.gibba032.whopeo (mgr.24625) 572 : cluster [DBG] pgmap v312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:53.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:53 gibba032 bash[24004]: cephadm 2021-03-10T14:02:52.008812+0000 mgr.gibba032.whopeo (mgr.24625) 573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:53.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:53 gibba011 bash[30791]: cephadm 2021-03-10T14:02:52.008812+0000 mgr.gibba032.whopeo (mgr.24625) 573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:53.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:53 gibba011 bash[31774]: cephadm 2021-03-10T14:02:52.008812+0000 mgr.gibba032.whopeo (mgr.24625) 573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:02:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:54 gibba032 bash[24004]: cluster 2021-03-10T14:02:52.821982+0000 mgr.gibba032.whopeo (mgr.24625) 574 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:54 gibba011 bash[30791]: cluster 2021-03-10T14:02:52.821982+0000 mgr.gibba032.whopeo (mgr.24625) 574 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:54 gibba011 bash[31774]: cluster 2021-03-10T14:02:52.821982+0000 mgr.gibba032.whopeo (mgr.24625) 574 : cluster [DBG] pgmap v313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: cluster 2021-03-10T14:02:54.823231+0000 mgr.gibba032.whopeo (mgr.24625) 575 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:55.934338+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:02:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:55.935064+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:55.935791+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:55.936476+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:56.042385+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:02:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[30791]: audit 2021-03-10T14:02:56.045742+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:02:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: cluster 2021-03-10T14:02:54.823231+0000 mgr.gibba032.whopeo (mgr.24625) 575 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:55.934338+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:02:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:55.935064+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:55.935791+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:55.936476+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:56.042385+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:02:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:56 gibba011 bash[31774]: audit 2021-03-10T14:02:56.045742+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:02:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: cluster 2021-03-10T14:02:54.823231+0000 mgr.gibba032.whopeo (mgr.24625) 575 : cluster [DBG] pgmap v314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:55.934338+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:02:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:55.935064+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:55.935791+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:55.936476+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:02:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:56.042385+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:02:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:56 gibba032 bash[24004]: audit 2021-03-10T14:02:56.045742+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:02:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:02:58 gibba032 bash[24004]: cluster 2021-03-10T14:02:56.824984+0000 mgr.gibba032.whopeo (mgr.24625) 576 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:02:58 gibba011 bash[30791]: cluster 2021-03-10T14:02:56.824984+0000 mgr.gibba032.whopeo (mgr.24625) 576 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:02:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:02:58 gibba011 bash[31774]: cluster 2021-03-10T14:02:56.824984+0000 mgr.gibba032.whopeo (mgr.24625) 576 : cluster [DBG] pgmap v315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:00 gibba032 bash[24004]: cluster 2021-03-10T14:02:58.826439+0000 mgr.gibba032.whopeo (mgr.24625) 577 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:00 gibba011 bash[30791]: cluster 2021-03-10T14:02:58.826439+0000 mgr.gibba032.whopeo (mgr.24625) 577 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:00 gibba011 bash[31774]: cluster 2021-03-10T14:02:58.826439+0000 mgr.gibba032.whopeo (mgr.24625) 577 : cluster [DBG] pgmap v316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:02 gibba032 bash[24004]: cluster 2021-03-10T14:03:00.827939+0000 mgr.gibba032.whopeo (mgr.24625) 578 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:02 gibba011 bash[30791]: cluster 2021-03-10T14:03:00.827939+0000 mgr.gibba032.whopeo (mgr.24625) 578 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:02 gibba011 bash[31774]: cluster 2021-03-10T14:03:00.827939+0000 mgr.gibba032.whopeo (mgr.24625) 578 : cluster [DBG] pgmap v317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:04 gibba032 bash[24004]: cluster 2021-03-10T14:03:02.829460+0000 mgr.gibba032.whopeo (mgr.24625) 579 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:04 gibba011 bash[30791]: cluster 2021-03-10T14:03:02.829460+0000 mgr.gibba032.whopeo (mgr.24625) 579 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:04 gibba011 bash[31774]: cluster 2021-03-10T14:03:02.829460+0000 mgr.gibba032.whopeo (mgr.24625) 579 : cluster [DBG] pgmap v318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:06 gibba032 bash[24004]: cluster 2021-03-10T14:03:04.830935+0000 mgr.gibba032.whopeo (mgr.24625) 580 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:06 gibba011 bash[30791]: cluster 2021-03-10T14:03:04.830935+0000 mgr.gibba032.whopeo (mgr.24625) 580 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:06 gibba011 bash[31774]: cluster 2021-03-10T14:03:04.830935+0000 mgr.gibba032.whopeo (mgr.24625) 580 : cluster [DBG] pgmap v319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:07.177 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:03:07.436 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:03:07.437 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (9m) 6m ago 15m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:03:07.437 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (14m) 5m ago 14m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:03:07.437 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (10m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:03:07.438 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (10m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:03:07.438 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (9m) 6m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:03:07.438 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (9m) 6m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:03:07.438 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (8m) 5m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:03:07.439 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (9m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:03:07.439 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (15m) 6m ago 15m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:03:07.439 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (15m) 5m ago 15m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:03:07.439 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (8m) 6m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:03:07.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:03:07.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:03:07.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (6m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:03:07.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:03:07.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (5m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:03:07.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (5m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:03:07.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (5m) 5m ago 15m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:03:07.441 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (14m) 5m ago 15m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:03:07.442 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 13m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:03:07.692 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:03:07.692 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:03:07.692 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:03:07.692 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:07.693 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:03:07.693 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:03:07.693 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:07.693 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:03:07.694 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:03:07.694 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:07.694 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:03:07.694 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:03:07.695 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:03:07.695 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:03:07.695 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:03:08.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cluster 2021-03-10T14:03:06.832680+0000 mgr.gibba032.whopeo (mgr.24625) 581 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.025514+0000 mgr.gibba032.whopeo (mgr.24625) 582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:03:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.027074+0000 mgr.gibba032.whopeo (mgr.24625) 583 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:03:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.168167+0000 mgr.gibba032.whopeo (mgr.24625) 584 : audit [DBG] from='client.54436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.229335+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:03:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.230791+0000 mgr.gibba032.whopeo (mgr.24625) 585 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:03:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.232402+0000 mgr.gibba032.whopeo (mgr.24625) 586 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:03:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.232629+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:03:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.233750+0000 mgr.gibba032.whopeo (mgr.24625) 587 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:03:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.233786+0000 mgr.gibba032.whopeo (mgr.24625) 588 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:03:08.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.233976+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.235170+0000 mgr.gibba032.whopeo (mgr.24625) 589 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:03:08.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.235223+0000 mgr.gibba032.whopeo (mgr.24625) 590 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:03:08.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.235262+0000 mon.a (mon.0) 737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.236116+0000 mgr.gibba032.whopeo (mgr.24625) 591 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:03:08.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.236681+0000 mon.a (mon.0) 738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.237285+0000 mgr.gibba032.whopeo (mgr.24625) 592 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:03:08.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.237329+0000 mgr.gibba032.whopeo (mgr.24625) 593 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:03:08.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.238352+0000 mgr.gibba032.whopeo (mgr.24625) 594 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:03:08.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.238784+0000 mgr.gibba032.whopeo (mgr.24625) 595 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:03:08.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.238948+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.518 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.239567+0000 mgr.gibba032.whopeo (mgr.24625) 596 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:03:08.518 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.239632+0000 mgr.gibba032.whopeo (mgr.24625) 597 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:03:08.519 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: cephadm 2021-03-10T14:03:07.239754+0000 mgr.gibba032.whopeo (mgr.24625) 598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:08.519 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.240245+0000 mon.a (mon.0) 740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.520 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.395602+0000 mgr.gibba032.whopeo (mgr.24625) 599 : audit [DBG] from='client.54442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.520 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:08 gibba032 bash[24004]: audit 2021-03-10T14:03:07.691594+0000 mon.a (mon.0) 741 : audit [DBG] from='client.? 172.21.2.111:0/1872861432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cluster 2021-03-10T14:03:06.832680+0000 mgr.gibba032.whopeo (mgr.24625) 581 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.025514+0000 mgr.gibba032.whopeo (mgr.24625) 582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:03:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.027074+0000 mgr.gibba032.whopeo (mgr.24625) 583 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:03:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.168167+0000 mgr.gibba032.whopeo (mgr.24625) 584 : audit [DBG] from='client.54436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.229335+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:03:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.230791+0000 mgr.gibba032.whopeo (mgr.24625) 585 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:03:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.232402+0000 mgr.gibba032.whopeo (mgr.24625) 586 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:03:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.232629+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:03:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.233750+0000 mgr.gibba032.whopeo (mgr.24625) 587 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:03:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.233786+0000 mgr.gibba032.whopeo (mgr.24625) 588 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:03:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.233976+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.235170+0000 mgr.gibba032.whopeo (mgr.24625) 589 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:03:08.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.235223+0000 mgr.gibba032.whopeo (mgr.24625) 590 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:03:08.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.235262+0000 mon.a (mon.0) 737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.236116+0000 mgr.gibba032.whopeo (mgr.24625) 591 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:03:08.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.236681+0000 mon.a (mon.0) 738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.237285+0000 mgr.gibba032.whopeo (mgr.24625) 592 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:03:08.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.237329+0000 mgr.gibba032.whopeo (mgr.24625) 593 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:03:08.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.238352+0000 mgr.gibba032.whopeo (mgr.24625) 594 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:03:08.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.238784+0000 mgr.gibba032.whopeo (mgr.24625) 595 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:03:08.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cluster 2021-03-10T14:03:06.832680+0000 mgr.gibba032.whopeo (mgr.24625) 581 : cluster [DBG] pgmap v320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:08.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.025514+0000 mgr.gibba032.whopeo (mgr.24625) 582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:03:08.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.027074+0000 mgr.gibba032.whopeo (mgr.24625) 583 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:03:08.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.168167+0000 mgr.gibba032.whopeo (mgr.24625) 584 : audit [DBG] from='client.54436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.229335+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:03:08.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.230791+0000 mgr.gibba032.whopeo (mgr.24625) 585 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:03:08.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.232402+0000 mgr.gibba032.whopeo (mgr.24625) 586 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:03:08.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.232629+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:03:08.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.233750+0000 mgr.gibba032.whopeo (mgr.24625) 587 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:03:08.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.233786+0000 mgr.gibba032.whopeo (mgr.24625) 588 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:03:08.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.233976+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.235170+0000 mgr.gibba032.whopeo (mgr.24625) 589 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:03:08.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.235223+0000 mgr.gibba032.whopeo (mgr.24625) 590 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:03:08.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.235262+0000 mon.a (mon.0) 737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.236116+0000 mgr.gibba032.whopeo (mgr.24625) 591 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:03:08.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.236681+0000 mon.a (mon.0) 738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.237285+0000 mgr.gibba032.whopeo (mgr.24625) 592 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:03:08.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.237329+0000 mgr.gibba032.whopeo (mgr.24625) 593 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:03:08.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.238352+0000 mgr.gibba032.whopeo (mgr.24625) 594 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:03:08.597 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.238784+0000 mgr.gibba032.whopeo (mgr.24625) 595 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:03:08.598 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.238948+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.598 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.239567+0000 mgr.gibba032.whopeo (mgr.24625) 596 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:03:08.599 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.239632+0000 mgr.gibba032.whopeo (mgr.24625) 597 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:03:08.599 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: cephadm 2021-03-10T14:03:07.239754+0000 mgr.gibba032.whopeo (mgr.24625) 598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:08.600 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.240245+0000 mon.a (mon.0) 740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.600 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.395602+0000 mgr.gibba032.whopeo (mgr.24625) 599 : audit [DBG] from='client.54442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.601 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[30791]: audit 2021-03-10T14:03:07.691594+0000 mon.a (mon.0) 741 : audit [DBG] from='client.? 172.21.2.111:0/1872861432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.238948+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.239567+0000 mgr.gibba032.whopeo (mgr.24625) 596 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:03:08.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.239632+0000 mgr.gibba032.whopeo (mgr.24625) 597 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:03:08.603 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: cephadm 2021-03-10T14:03:07.239754+0000 mgr.gibba032.whopeo (mgr.24625) 598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:08.603 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.240245+0000 mon.a (mon.0) 740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:08.604 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.395602+0000 mgr.gibba032.whopeo (mgr.24625) 599 : audit [DBG] from='client.54442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:08.604 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:08 gibba011 bash[31774]: audit 2021-03-10T14:03:07.691594+0000 mon.a (mon.0) 741 : audit [DBG] from='client.? 172.21.2.111:0/1872861432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:10 gibba032 bash[24004]: cluster 2021-03-10T14:03:08.834060+0000 mgr.gibba032.whopeo (mgr.24625) 600 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:10 gibba011 bash[30791]: cluster 2021-03-10T14:03:08.834060+0000 mgr.gibba032.whopeo (mgr.24625) 600 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:10 gibba011 bash[31774]: cluster 2021-03-10T14:03:08.834060+0000 mgr.gibba032.whopeo (mgr.24625) 600 : cluster [DBG] pgmap v321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:12 gibba032 bash[24004]: cluster 2021-03-10T14:03:10.835508+0000 mgr.gibba032.whopeo (mgr.24625) 601 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:12 gibba011 bash[30791]: cluster 2021-03-10T14:03:10.835508+0000 mgr.gibba032.whopeo (mgr.24625) 601 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:12 gibba011 bash[31774]: cluster 2021-03-10T14:03:10.835508+0000 mgr.gibba032.whopeo (mgr.24625) 601 : cluster [DBG] pgmap v322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:14 gibba011 bash[30791]: cluster 2021-03-10T14:03:12.837190+0000 mgr.gibba032.whopeo (mgr.24625) 602 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:14 gibba011 bash[31774]: cluster 2021-03-10T14:03:12.837190+0000 mgr.gibba032.whopeo (mgr.24625) 602 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:14 gibba032 bash[24004]: cluster 2021-03-10T14:03:12.837190+0000 mgr.gibba032.whopeo (mgr.24625) 602 : cluster [DBG] pgmap v323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:16 gibba011 bash[30791]: cluster 2021-03-10T14:03:14.838428+0000 mgr.gibba032.whopeo (mgr.24625) 603 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:16 gibba011 bash[31774]: cluster 2021-03-10T14:03:14.838428+0000 mgr.gibba032.whopeo (mgr.24625) 603 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:16 gibba032 bash[24004]: cluster 2021-03-10T14:03:14.838428+0000 mgr.gibba032.whopeo (mgr.24625) 603 : cluster [DBG] pgmap v324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:18 gibba011 bash[30791]: cluster 2021-03-10T14:03:16.840173+0000 mgr.gibba032.whopeo (mgr.24625) 604 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:18 gibba011 bash[31774]: cluster 2021-03-10T14:03:16.840173+0000 mgr.gibba032.whopeo (mgr.24625) 604 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:18 gibba032 bash[24004]: cluster 2021-03-10T14:03:16.840173+0000 mgr.gibba032.whopeo (mgr.24625) 604 : cluster [DBG] pgmap v325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:20 gibba011 bash[30791]: cluster 2021-03-10T14:03:18.841740+0000 mgr.gibba032.whopeo (mgr.24625) 605 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:20 gibba011 bash[31774]: cluster 2021-03-10T14:03:18.841740+0000 mgr.gibba032.whopeo (mgr.24625) 605 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:20 gibba032 bash[24004]: cluster 2021-03-10T14:03:18.841740+0000 mgr.gibba032.whopeo (mgr.24625) 605 : cluster [DBG] pgmap v326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:22 gibba011 bash[30791]: cluster 2021-03-10T14:03:20.843059+0000 mgr.gibba032.whopeo (mgr.24625) 606 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:22 gibba011 bash[31774]: cluster 2021-03-10T14:03:20.843059+0000 mgr.gibba032.whopeo (mgr.24625) 606 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:22 gibba032 bash[24004]: cluster 2021-03-10T14:03:20.843059+0000 mgr.gibba032.whopeo (mgr.24625) 606 : cluster [DBG] pgmap v327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:23 gibba011 bash[30791]: cephadm 2021-03-10T14:03:22.255765+0000 mgr.gibba032.whopeo (mgr.24625) 607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:23 gibba011 bash[31774]: cephadm 2021-03-10T14:03:22.255765+0000 mgr.gibba032.whopeo (mgr.24625) 607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:23.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:23 gibba032 bash[24004]: cephadm 2021-03-10T14:03:22.255765+0000 mgr.gibba032.whopeo (mgr.24625) 607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:24 gibba011 bash[30791]: cluster 2021-03-10T14:03:22.844715+0000 mgr.gibba032.whopeo (mgr.24625) 608 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:24 gibba011 bash[31774]: cluster 2021-03-10T14:03:22.844715+0000 mgr.gibba032.whopeo (mgr.24625) 608 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:24 gibba032 bash[24004]: cluster 2021-03-10T14:03:22.844715+0000 mgr.gibba032.whopeo (mgr.24625) 608 : cluster [DBG] pgmap v328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:26 gibba011 bash[30791]: cluster 2021-03-10T14:03:24.846266+0000 mgr.gibba032.whopeo (mgr.24625) 609 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:26 gibba011 bash[31774]: cluster 2021-03-10T14:03:24.846266+0000 mgr.gibba032.whopeo (mgr.24625) 609 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:26 gibba032 bash[24004]: cluster 2021-03-10T14:03:24.846266+0000 mgr.gibba032.whopeo (mgr.24625) 609 : cluster [DBG] pgmap v329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:28 gibba011 bash[30791]: cluster 2021-03-10T14:03:26.848080+0000 mgr.gibba032.whopeo (mgr.24625) 610 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:28 gibba011 bash[31774]: cluster 2021-03-10T14:03:26.848080+0000 mgr.gibba032.whopeo (mgr.24625) 610 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:28 gibba032 bash[24004]: cluster 2021-03-10T14:03:26.848080+0000 mgr.gibba032.whopeo (mgr.24625) 610 : cluster [DBG] pgmap v330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:30 gibba032 bash[24004]: cluster 2021-03-10T14:03:28.849390+0000 mgr.gibba032.whopeo (mgr.24625) 611 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:30 gibba011 bash[30791]: cluster 2021-03-10T14:03:28.849390+0000 mgr.gibba032.whopeo (mgr.24625) 611 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:30 gibba011 bash[31774]: cluster 2021-03-10T14:03:28.849390+0000 mgr.gibba032.whopeo (mgr.24625) 611 : cluster [DBG] pgmap v331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:32 gibba032 bash[24004]: cluster 2021-03-10T14:03:30.850774+0000 mgr.gibba032.whopeo (mgr.24625) 612 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:32 gibba011 bash[30791]: cluster 2021-03-10T14:03:30.850774+0000 mgr.gibba032.whopeo (mgr.24625) 612 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:32 gibba011 bash[31774]: cluster 2021-03-10T14:03:30.850774+0000 mgr.gibba032.whopeo (mgr.24625) 612 : cluster [DBG] pgmap v332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:34 gibba032 bash[24004]: cluster 2021-03-10T14:03:32.852413+0000 mgr.gibba032.whopeo (mgr.24625) 613 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:34 gibba011 bash[30791]: cluster 2021-03-10T14:03:32.852413+0000 mgr.gibba032.whopeo (mgr.24625) 613 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:34 gibba011 bash[31774]: cluster 2021-03-10T14:03:32.852413+0000 mgr.gibba032.whopeo (mgr.24625) 613 : cluster [DBG] pgmap v333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:36 gibba032 bash[24004]: cluster 2021-03-10T14:03:34.853887+0000 mgr.gibba032.whopeo (mgr.24625) 614 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:36 gibba011 bash[30791]: cluster 2021-03-10T14:03:34.853887+0000 mgr.gibba032.whopeo (mgr.24625) 614 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:36 gibba011 bash[31774]: cluster 2021-03-10T14:03:34.853887+0000 mgr.gibba032.whopeo (mgr.24625) 614 : cluster [DBG] pgmap v334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:03:38.198 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:03:38.199 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (10m) 7m ago 15m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:03:38.199 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (15m) 5m ago 15m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:03:38.199 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (10m) 5m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:03:38.199 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (10m) 5m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:03:38.200 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (9m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:03:38.200 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (9m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:03:38.200 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (9m) 5m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:03:38.200 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (9m) 7m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:03:38.201 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (15m) 7m ago 15m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:03:38.201 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (15m) 5m ago 15m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:03:38.201 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:03:38.201 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:03:38.202 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (7m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:03:38.202 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (7m) 7m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:03:38.202 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (6m) 5m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:03:38.202 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:03:38.202 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (6m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:03:38.203 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (5m) 5m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:03:38.203 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (15m) 5m ago 15m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:03:38.203 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 13m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:03:38.450 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:03:38.451 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:03:38.452 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:03:38.453 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:38.455 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:03:38.455 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:03:38.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:38.456 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:03:38.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:03:38.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:03:38.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:03:38.457 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:03:38.457 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:03:38.457 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:03:38.457 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:03:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:38 gibba032 bash[24004]: cluster 2021-03-10T14:03:36.855390+0000 mgr.gibba032.whopeo (mgr.24625) 615 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:38 gibba032 bash[24004]: cephadm 2021-03-10T14:03:37.268780+0000 mgr.gibba032.whopeo (mgr.24625) 616 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:38 gibba011 bash[30791]: cluster 2021-03-10T14:03:36.855390+0000 mgr.gibba032.whopeo (mgr.24625) 615 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:38 gibba011 bash[30791]: cephadm 2021-03-10T14:03:37.268780+0000 mgr.gibba032.whopeo (mgr.24625) 616 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:38 gibba011 bash[31774]: cluster 2021-03-10T14:03:36.855390+0000 mgr.gibba032.whopeo (mgr.24625) 615 : cluster [DBG] pgmap v335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:38 gibba011 bash[31774]: cephadm 2021-03-10T14:03:37.268780+0000 mgr.gibba032.whopeo (mgr.24625) 616 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:39 gibba032 bash[24004]: audit 2021-03-10T14:03:37.926357+0000 mgr.gibba032.whopeo (mgr.24625) 617 : audit [DBG] from='client.54454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:39 gibba032 bash[24004]: audit 2021-03-10T14:03:38.153819+0000 mgr.gibba032.whopeo (mgr.24625) 618 : audit [DBG] from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:39 gibba032 bash[24004]: audit 2021-03-10T14:03:38.447429+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.2.111:0/1493738937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[30791]: audit 2021-03-10T14:03:37.926357+0000 mgr.gibba032.whopeo (mgr.24625) 617 : audit [DBG] from='client.54454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[30791]: audit 2021-03-10T14:03:38.153819+0000 mgr.gibba032.whopeo (mgr.24625) 618 : audit [DBG] from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[30791]: audit 2021-03-10T14:03:38.447429+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.2.111:0/1493738937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:39.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[31774]: audit 2021-03-10T14:03:37.926357+0000 mgr.gibba032.whopeo (mgr.24625) 617 : audit [DBG] from='client.54454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[31774]: audit 2021-03-10T14:03:38.153819+0000 mgr.gibba032.whopeo (mgr.24625) 618 : audit [DBG] from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:03:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:39 gibba011 bash[31774]: audit 2021-03-10T14:03:38.447429+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.2.111:0/1493738937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:03:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:40 gibba032 bash[24004]: cluster 2021-03-10T14:03:38.856961+0000 mgr.gibba032.whopeo (mgr.24625) 619 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:40 gibba011 bash[30791]: cluster 2021-03-10T14:03:38.856961+0000 mgr.gibba032.whopeo (mgr.24625) 619 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:40 gibba011 bash[31774]: cluster 2021-03-10T14:03:38.856961+0000 mgr.gibba032.whopeo (mgr.24625) 619 : cluster [DBG] pgmap v336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:42 gibba032 bash[24004]: cluster 2021-03-10T14:03:40.858290+0000 mgr.gibba032.whopeo (mgr.24625) 620 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:42 gibba011 bash[30791]: cluster 2021-03-10T14:03:40.858290+0000 mgr.gibba032.whopeo (mgr.24625) 620 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:42 gibba011 bash[31774]: cluster 2021-03-10T14:03:40.858290+0000 mgr.gibba032.whopeo (mgr.24625) 620 : cluster [DBG] pgmap v337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:44 gibba032 bash[24004]: cluster 2021-03-10T14:03:42.860093+0000 mgr.gibba032.whopeo (mgr.24625) 621 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:44 gibba011 bash[30791]: cluster 2021-03-10T14:03:42.860093+0000 mgr.gibba032.whopeo (mgr.24625) 621 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:44 gibba011 bash[31774]: cluster 2021-03-10T14:03:42.860093+0000 mgr.gibba032.whopeo (mgr.24625) 621 : cluster [DBG] pgmap v338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:46 gibba032 bash[24004]: cluster 2021-03-10T14:03:44.861585+0000 mgr.gibba032.whopeo (mgr.24625) 622 : cluster [DBG] pgmap v339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:46 gibba011 bash[30791]: cluster 2021-03-10T14:03:44.861585+0000 mgr.gibba032.whopeo (mgr.24625) 622 : cluster [DBG] pgmap v339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:46 gibba011 bash[31774]: cluster 2021-03-10T14:03:44.861585+0000 mgr.gibba032.whopeo (mgr.24625) 622 : cluster [DBG] pgmap v339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:48 gibba032 bash[24004]: cluster 2021-03-10T14:03:46.863335+0000 mgr.gibba032.whopeo (mgr.24625) 623 : cluster [DBG] pgmap v340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:48 gibba011 bash[30791]: cluster 2021-03-10T14:03:46.863335+0000 mgr.gibba032.whopeo (mgr.24625) 623 : cluster [DBG] pgmap v340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:48 gibba011 bash[31774]: cluster 2021-03-10T14:03:46.863335+0000 mgr.gibba032.whopeo (mgr.24625) 623 : cluster [DBG] pgmap v340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:50 gibba032 bash[24004]: cluster 2021-03-10T14:03:48.864952+0000 mgr.gibba032.whopeo (mgr.24625) 624 : cluster [DBG] pgmap v341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:50 gibba011 bash[30791]: cluster 2021-03-10T14:03:48.864952+0000 mgr.gibba032.whopeo (mgr.24625) 624 : cluster [DBG] pgmap v341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:50 gibba011 bash[31774]: cluster 2021-03-10T14:03:48.864952+0000 mgr.gibba032.whopeo (mgr.24625) 624 : cluster [DBG] pgmap v341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:52 gibba032 bash[24004]: cluster 2021-03-10T14:03:50.866318+0000 mgr.gibba032.whopeo (mgr.24625) 625 : cluster [DBG] pgmap v342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:52 gibba011 bash[30791]: cluster 2021-03-10T14:03:50.866318+0000 mgr.gibba032.whopeo (mgr.24625) 625 : cluster [DBG] pgmap v342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:52 gibba011 bash[31774]: cluster 2021-03-10T14:03:50.866318+0000 mgr.gibba032.whopeo (mgr.24625) 625 : cluster [DBG] pgmap v342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:53.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:53 gibba032 bash[24004]: cephadm 2021-03-10T14:03:52.284823+0000 mgr.gibba032.whopeo (mgr.24625) 626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:53.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:53 gibba011 bash[30791]: cephadm 2021-03-10T14:03:52.284823+0000 mgr.gibba032.whopeo (mgr.24625) 626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:53.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:53 gibba011 bash[31774]: cephadm 2021-03-10T14:03:52.284823+0000 mgr.gibba032.whopeo (mgr.24625) 626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:03:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:54 gibba032 bash[24004]: cluster 2021-03-10T14:03:52.867952+0000 mgr.gibba032.whopeo (mgr.24625) 627 : cluster [DBG] pgmap v343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:54 gibba011 bash[30791]: cluster 2021-03-10T14:03:52.867952+0000 mgr.gibba032.whopeo (mgr.24625) 627 : cluster [DBG] pgmap v343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:54 gibba011 bash[31774]: cluster 2021-03-10T14:03:52.867952+0000 mgr.gibba032.whopeo (mgr.24625) 627 : cluster [DBG] pgmap v343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: cluster 2021-03-10T14:03:54.869242+0000 mgr.gibba032.whopeo (mgr.24625) 628 : cluster [DBG] pgmap v344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:55.945445+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:03:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:55.946209+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:55.946926+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:55.947654+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T14:03:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:56.052358+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:03:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:56 gibba032 bash[24004]: audit 2021-03-10T14:03:56.053018+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:03:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: cluster 2021-03-10T14:03:54.869242+0000 mgr.gibba032.whopeo (mgr.24625) 628 : cluster [DBG] pgmap v344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:55.945445+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:03:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:55.946209+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:55.946926+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:55.947654+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T14:03:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:56.052358+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:03:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[30791]: audit 2021-03-10T14:03:56.053018+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:03:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: cluster 2021-03-10T14:03:54.869242+0000 mgr.gibba032.whopeo (mgr.24625) 628 : cluster [DBG] pgmap v344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:55.945445+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:03:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:55.946209+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:55.946926+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:03:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:55.947654+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T14:03:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:56.052358+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:03:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:56 gibba011 bash[31774]: audit 2021-03-10T14:03:56.053018+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:03:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:03:58 gibba032 bash[24004]: cluster 2021-03-10T14:03:56.870852+0000 mgr.gibba032.whopeo (mgr.24625) 629 : cluster [DBG] pgmap v345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:03:58 gibba011 bash[30791]: cluster 2021-03-10T14:03:56.870852+0000 mgr.gibba032.whopeo (mgr.24625) 629 : cluster [DBG] pgmap v345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:03:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:03:58 gibba011 bash[31774]: cluster 2021-03-10T14:03:56.870852+0000 mgr.gibba032.whopeo (mgr.24625) 629 : cluster [DBG] pgmap v345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:00 gibba032 bash[24004]: cluster 2021-03-10T14:03:58.872477+0000 mgr.gibba032.whopeo (mgr.24625) 630 : cluster [DBG] pgmap v346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:00 gibba011 bash[30791]: cluster 2021-03-10T14:03:58.872477+0000 mgr.gibba032.whopeo (mgr.24625) 630 : cluster [DBG] pgmap v346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:00 gibba011 bash[31774]: cluster 2021-03-10T14:03:58.872477+0000 mgr.gibba032.whopeo (mgr.24625) 630 : cluster [DBG] pgmap v346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:02 gibba032 bash[24004]: cluster 2021-03-10T14:04:00.873779+0000 mgr.gibba032.whopeo (mgr.24625) 631 : cluster [DBG] pgmap v347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:02 gibba011 bash[30791]: cluster 2021-03-10T14:04:00.873779+0000 mgr.gibba032.whopeo (mgr.24625) 631 : cluster [DBG] pgmap v347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:02.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:02 gibba011 bash[31774]: cluster 2021-03-10T14:04:00.873779+0000 mgr.gibba032.whopeo (mgr.24625) 631 : cluster [DBG] pgmap v347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:04 gibba011 bash[30791]: cluster 2021-03-10T14:04:02.875486+0000 mgr.gibba032.whopeo (mgr.24625) 632 : cluster [DBG] pgmap v348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:04 gibba011 bash[31774]: cluster 2021-03-10T14:04:02.875486+0000 mgr.gibba032.whopeo (mgr.24625) 632 : cluster [DBG] pgmap v348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:04 gibba032 bash[24004]: cluster 2021-03-10T14:04:02.875486+0000 mgr.gibba032.whopeo (mgr.24625) 632 : cluster [DBG] pgmap v348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:06 gibba011 bash[30791]: cluster 2021-03-10T14:04:04.877030+0000 mgr.gibba032.whopeo (mgr.24625) 633 : cluster [DBG] pgmap v349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:06 gibba011 bash[31774]: cluster 2021-03-10T14:04:04.877030+0000 mgr.gibba032.whopeo (mgr.24625) 633 : cluster [DBG] pgmap v349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:07.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:06 gibba032 bash[24004]: cluster 2021-03-10T14:04:04.877030+0000 mgr.gibba032.whopeo (mgr.24625) 633 : cluster [DBG] pgmap v349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:08.699 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:04:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: cluster 2021-03-10T14:04:06.878535+0000 mgr.gibba032.whopeo (mgr.24625) 634 : cluster [DBG] pgmap v350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.500681+0000 mgr.gibba032.whopeo (mgr.24625) 635 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:04:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.501657+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.503570+0000 mgr.gibba032.whopeo (mgr.24625) 636 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:04:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.504569+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.690694+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.694078+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:04:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.695923+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.697556+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.698910+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.701623+0000 mon.a (mon.0) 756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:08 gibba032 bash[24004]: audit 2021-03-10T14:04:07.703098+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: cluster 2021-03-10T14:04:06.878535+0000 mgr.gibba032.whopeo (mgr.24625) 634 : cluster [DBG] pgmap v350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.500681+0000 mgr.gibba032.whopeo (mgr.24625) 635 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:04:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.501657+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.503570+0000 mgr.gibba032.whopeo (mgr.24625) 636 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:04:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.504569+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.690694+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.694078+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:04:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.695923+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.697556+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.698910+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.701623+0000 mon.a (mon.0) 756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[30791]: audit 2021-03-10T14:04:07.703098+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: cluster 2021-03-10T14:04:06.878535+0000 mgr.gibba032.whopeo (mgr.24625) 634 : cluster [DBG] pgmap v350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.500681+0000 mgr.gibba032.whopeo (mgr.24625) 635 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:04:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.501657+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.503570+0000 mgr.gibba032.whopeo (mgr.24625) 636 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:04:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.504569+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.690694+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:04:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.694078+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:04:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.695923+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.697556+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.698910+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.701623+0000 mon.a (mon.0) 756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:08 gibba011 bash[31774]: audit 2021-03-10T14:04:07.703098+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:08.926 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:04:08.926 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (10m) 7m ago 16m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:04:08.927 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (15m) 6m ago 15m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:04:08.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (11m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:04:08.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (11m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:04:08.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (10m) 7m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:04:08.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (10m) 7m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:04:08.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (9m) 6m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:04:08.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (10m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:04:08.928 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (16m) 7m ago 16m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:04:08.929 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (16m) 6m ago 16m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:04:08.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (9m) 7m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:04:08.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:04:08.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:04:08.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (7m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:04:08.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:04:08.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (6m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:04:08.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (6m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:04:08.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (6m) 6m ago 16m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:04:08.931 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (15m) 6m ago 16m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:04:08.931 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 14m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:04:09.173 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:04:09.173 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:04:09.173 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:04:09.174 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:09.174 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:04:09.174 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:04:09.174 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:09.174 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:04:09.175 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:04:09.175 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:09.175 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:04:09.175 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:04:09.176 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:04:09.176 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:04:09.176 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:04:09.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.691536+0000 mgr.gibba032.whopeo (mgr.24625) 637 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:04:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.693909+0000 mgr.gibba032.whopeo (mgr.24625) 638 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:04:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.695545+0000 mgr.gibba032.whopeo (mgr.24625) 639 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:04:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.695583+0000 mgr.gibba032.whopeo (mgr.24625) 640 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:04:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.696995+0000 mgr.gibba032.whopeo (mgr.24625) 641 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:04:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.697030+0000 mgr.gibba032.whopeo (mgr.24625) 642 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:04:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.698340+0000 mgr.gibba032.whopeo (mgr.24625) 643 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:04:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.699576+0000 mgr.gibba032.whopeo (mgr.24625) 644 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:04:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.699613+0000 mgr.gibba032.whopeo (mgr.24625) 645 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:04:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.700960+0000 mgr.gibba032.whopeo (mgr.24625) 646 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:04:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.701273+0000 mgr.gibba032.whopeo (mgr.24625) 647 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:04:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.702540+0000 mgr.gibba032.whopeo (mgr.24625) 648 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:04:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.702578+0000 mgr.gibba032.whopeo (mgr.24625) 649 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:04:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: cephadm 2021-03-10T14:04:07.702680+0000 mgr.gibba032.whopeo (mgr.24625) 650 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[30791]: audit 2021-03-10T14:04:09.172592+0000 mon.a (mon.0) 758 : audit [DBG] from='client.? 172.21.2.111:0/795970470' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:09.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.691536+0000 mgr.gibba032.whopeo (mgr.24625) 637 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:04:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.693909+0000 mgr.gibba032.whopeo (mgr.24625) 638 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:04:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.695545+0000 mgr.gibba032.whopeo (mgr.24625) 639 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:04:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.695583+0000 mgr.gibba032.whopeo (mgr.24625) 640 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:04:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.696995+0000 mgr.gibba032.whopeo (mgr.24625) 641 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:04:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.697030+0000 mgr.gibba032.whopeo (mgr.24625) 642 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:04:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.698340+0000 mgr.gibba032.whopeo (mgr.24625) 643 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:04:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.699576+0000 mgr.gibba032.whopeo (mgr.24625) 644 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:04:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.699613+0000 mgr.gibba032.whopeo (mgr.24625) 645 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:04:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.700960+0000 mgr.gibba032.whopeo (mgr.24625) 646 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:04:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.701273+0000 mgr.gibba032.whopeo (mgr.24625) 647 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:04:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.702540+0000 mgr.gibba032.whopeo (mgr.24625) 648 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:04:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.702578+0000 mgr.gibba032.whopeo (mgr.24625) 649 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:04:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: cephadm 2021-03-10T14:04:07.702680+0000 mgr.gibba032.whopeo (mgr.24625) 650 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:09 gibba011 bash[31774]: audit 2021-03-10T14:04:09.172592+0000 mon.a (mon.0) 758 : audit [DBG] from='client.? 172.21.2.111:0/795970470' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.691536+0000 mgr.gibba032.whopeo (mgr.24625) 637 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:04:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.693909+0000 mgr.gibba032.whopeo (mgr.24625) 638 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:04:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.695545+0000 mgr.gibba032.whopeo (mgr.24625) 639 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:04:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.695583+0000 mgr.gibba032.whopeo (mgr.24625) 640 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:04:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.696995+0000 mgr.gibba032.whopeo (mgr.24625) 641 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:04:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.697030+0000 mgr.gibba032.whopeo (mgr.24625) 642 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:04:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.698340+0000 mgr.gibba032.whopeo (mgr.24625) 643 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:04:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.699576+0000 mgr.gibba032.whopeo (mgr.24625) 644 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:04:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.699613+0000 mgr.gibba032.whopeo (mgr.24625) 645 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:04:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.700960+0000 mgr.gibba032.whopeo (mgr.24625) 646 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:04:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.701273+0000 mgr.gibba032.whopeo (mgr.24625) 647 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:04:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.702540+0000 mgr.gibba032.whopeo (mgr.24625) 648 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:04:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.702578+0000 mgr.gibba032.whopeo (mgr.24625) 649 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:04:10.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: cephadm 2021-03-10T14:04:07.702680+0000 mgr.gibba032.whopeo (mgr.24625) 650 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:10.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:09 gibba032 bash[24004]: audit 2021-03-10T14:04:09.172592+0000 mon.a (mon.0) 758 : audit [DBG] from='client.? 172.21.2.111:0/795970470' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[30791]: audit 2021-03-10T14:04:08.688328+0000 mgr.gibba032.whopeo (mgr.24625) 651 : audit [DBG] from='client.64160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[30791]: cluster 2021-03-10T14:04:08.879993+0000 mgr.gibba032.whopeo (mgr.24625) 652 : cluster [DBG] pgmap v351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[30791]: audit 2021-03-10T14:04:08.912692+0000 mgr.gibba032.whopeo (mgr.24625) 653 : audit [DBG] from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[31774]: audit 2021-03-10T14:04:08.688328+0000 mgr.gibba032.whopeo (mgr.24625) 651 : audit [DBG] from='client.64160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[31774]: cluster 2021-03-10T14:04:08.879993+0000 mgr.gibba032.whopeo (mgr.24625) 652 : cluster [DBG] pgmap v351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:10 gibba011 bash[31774]: audit 2021-03-10T14:04:08.912692+0000 mgr.gibba032.whopeo (mgr.24625) 653 : audit [DBG] from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:10 gibba032 bash[24004]: audit 2021-03-10T14:04:08.688328+0000 mgr.gibba032.whopeo (mgr.24625) 651 : audit [DBG] from='client.64160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:10 gibba032 bash[24004]: cluster 2021-03-10T14:04:08.879993+0000 mgr.gibba032.whopeo (mgr.24625) 652 : cluster [DBG] pgmap v351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:10 gibba032 bash[24004]: audit 2021-03-10T14:04:08.912692+0000 mgr.gibba032.whopeo (mgr.24625) 653 : audit [DBG] from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:11 gibba011 bash[30791]: cluster 2021-03-10T14:04:10.881278+0000 mgr.gibba032.whopeo (mgr.24625) 654 : cluster [DBG] pgmap v352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:11 gibba011 bash[31774]: cluster 2021-03-10T14:04:10.881278+0000 mgr.gibba032.whopeo (mgr.24625) 654 : cluster [DBG] pgmap v352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:12.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:11 gibba032 bash[24004]: cluster 2021-03-10T14:04:10.881278+0000 mgr.gibba032.whopeo (mgr.24625) 654 : cluster [DBG] pgmap v352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:13 gibba032 bash[24004]: cluster 2021-03-10T14:04:12.883113+0000 mgr.gibba032.whopeo (mgr.24625) 655 : cluster [DBG] pgmap v353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:13 gibba011 bash[30791]: cluster 2021-03-10T14:04:12.883113+0000 mgr.gibba032.whopeo (mgr.24625) 655 : cluster [DBG] pgmap v353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:13 gibba011 bash[31774]: cluster 2021-03-10T14:04:12.883113+0000 mgr.gibba032.whopeo (mgr.24625) 655 : cluster [DBG] pgmap v353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:16 gibba032 bash[24004]: cluster 2021-03-10T14:04:14.884527+0000 mgr.gibba032.whopeo (mgr.24625) 656 : cluster [DBG] pgmap v354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:16 gibba011 bash[30791]: cluster 2021-03-10T14:04:14.884527+0000 mgr.gibba032.whopeo (mgr.24625) 656 : cluster [DBG] pgmap v354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:16 gibba011 bash[31774]: cluster 2021-03-10T14:04:14.884527+0000 mgr.gibba032.whopeo (mgr.24625) 656 : cluster [DBG] pgmap v354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:18 gibba032 bash[24004]: cluster 2021-03-10T14:04:16.886279+0000 mgr.gibba032.whopeo (mgr.24625) 657 : cluster [DBG] pgmap v355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:18 gibba011 bash[30791]: cluster 2021-03-10T14:04:16.886279+0000 mgr.gibba032.whopeo (mgr.24625) 657 : cluster [DBG] pgmap v355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:18 gibba011 bash[31774]: cluster 2021-03-10T14:04:16.886279+0000 mgr.gibba032.whopeo (mgr.24625) 657 : cluster [DBG] pgmap v355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:20 gibba032 bash[24004]: cluster 2021-03-10T14:04:18.887778+0000 mgr.gibba032.whopeo (mgr.24625) 658 : cluster [DBG] pgmap v356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:20 gibba011 bash[30791]: cluster 2021-03-10T14:04:18.887778+0000 mgr.gibba032.whopeo (mgr.24625) 658 : cluster [DBG] pgmap v356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:20 gibba011 bash[31774]: cluster 2021-03-10T14:04:18.887778+0000 mgr.gibba032.whopeo (mgr.24625) 658 : cluster [DBG] pgmap v356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:22 gibba032 bash[24004]: cluster 2021-03-10T14:04:20.889104+0000 mgr.gibba032.whopeo (mgr.24625) 659 : cluster [DBG] pgmap v357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:22 gibba011 bash[30791]: cluster 2021-03-10T14:04:20.889104+0000 mgr.gibba032.whopeo (mgr.24625) 659 : cluster [DBG] pgmap v357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:22 gibba011 bash[31774]: cluster 2021-03-10T14:04:20.889104+0000 mgr.gibba032.whopeo (mgr.24625) 659 : cluster [DBG] pgmap v357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:24 gibba032 bash[24004]: cephadm 2021-03-10T14:04:22.714801+0000 mgr.gibba032.whopeo (mgr.24625) 660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:24 gibba032 bash[24004]: cluster 2021-03-10T14:04:22.890717+0000 mgr.gibba032.whopeo (mgr.24625) 661 : cluster [DBG] pgmap v358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:24 gibba011 bash[30791]: cephadm 2021-03-10T14:04:22.714801+0000 mgr.gibba032.whopeo (mgr.24625) 660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:24 gibba011 bash[30791]: cluster 2021-03-10T14:04:22.890717+0000 mgr.gibba032.whopeo (mgr.24625) 661 : cluster [DBG] pgmap v358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:24 gibba011 bash[31774]: cephadm 2021-03-10T14:04:22.714801+0000 mgr.gibba032.whopeo (mgr.24625) 660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:24 gibba011 bash[31774]: cluster 2021-03-10T14:04:22.890717+0000 mgr.gibba032.whopeo (mgr.24625) 661 : cluster [DBG] pgmap v358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:26 gibba032 bash[24004]: cluster 2021-03-10T14:04:24.892151+0000 mgr.gibba032.whopeo (mgr.24625) 662 : cluster [DBG] pgmap v359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:26 gibba011 bash[30791]: cluster 2021-03-10T14:04:24.892151+0000 mgr.gibba032.whopeo (mgr.24625) 662 : cluster [DBG] pgmap v359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:26 gibba011 bash[31774]: cluster 2021-03-10T14:04:24.892151+0000 mgr.gibba032.whopeo (mgr.24625) 662 : cluster [DBG] pgmap v359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:28 gibba032 bash[24004]: cluster 2021-03-10T14:04:26.893692+0000 mgr.gibba032.whopeo (mgr.24625) 663 : cluster [DBG] pgmap v360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:28 gibba011 bash[30791]: cluster 2021-03-10T14:04:26.893692+0000 mgr.gibba032.whopeo (mgr.24625) 663 : cluster [DBG] pgmap v360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:28 gibba011 bash[31774]: cluster 2021-03-10T14:04:26.893692+0000 mgr.gibba032.whopeo (mgr.24625) 663 : cluster [DBG] pgmap v360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:30 gibba032 bash[24004]: cluster 2021-03-10T14:04:28.895183+0000 mgr.gibba032.whopeo (mgr.24625) 664 : cluster [DBG] pgmap v361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:30 gibba011 bash[30791]: cluster 2021-03-10T14:04:28.895183+0000 mgr.gibba032.whopeo (mgr.24625) 664 : cluster [DBG] pgmap v361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:30 gibba011 bash[31774]: cluster 2021-03-10T14:04:28.895183+0000 mgr.gibba032.whopeo (mgr.24625) 664 : cluster [DBG] pgmap v361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:32 gibba032 bash[24004]: cluster 2021-03-10T14:04:30.896535+0000 mgr.gibba032.whopeo (mgr.24625) 665 : cluster [DBG] pgmap v362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:32 gibba011 bash[30791]: cluster 2021-03-10T14:04:30.896535+0000 mgr.gibba032.whopeo (mgr.24625) 665 : cluster [DBG] pgmap v362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:32 gibba011 bash[31774]: cluster 2021-03-10T14:04:30.896535+0000 mgr.gibba032.whopeo (mgr.24625) 665 : cluster [DBG] pgmap v362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:34 gibba032 bash[24004]: cluster 2021-03-10T14:04:32.898250+0000 mgr.gibba032.whopeo (mgr.24625) 666 : cluster [DBG] pgmap v363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:34 gibba011 bash[30791]: cluster 2021-03-10T14:04:32.898250+0000 mgr.gibba032.whopeo (mgr.24625) 666 : cluster [DBG] pgmap v363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:34 gibba011 bash[31774]: cluster 2021-03-10T14:04:32.898250+0000 mgr.gibba032.whopeo (mgr.24625) 666 : cluster [DBG] pgmap v363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:36 gibba032 bash[24004]: cluster 2021-03-10T14:04:34.899801+0000 mgr.gibba032.whopeo (mgr.24625) 667 : cluster [DBG] pgmap v364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:36 gibba011 bash[30791]: cluster 2021-03-10T14:04:34.899801+0000 mgr.gibba032.whopeo (mgr.24625) 667 : cluster [DBG] pgmap v364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:36 gibba011 bash[31774]: cluster 2021-03-10T14:04:34.899801+0000 mgr.gibba032.whopeo (mgr.24625) 667 : cluster [DBG] pgmap v364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:38 gibba032 bash[24004]: cluster 2021-03-10T14:04:36.901711+0000 mgr.gibba032.whopeo (mgr.24625) 668 : cluster [DBG] pgmap v365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:38 gibba011 bash[30791]: cluster 2021-03-10T14:04:36.901711+0000 mgr.gibba032.whopeo (mgr.24625) 668 : cluster [DBG] pgmap v365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:38 gibba011 bash[31774]: cluster 2021-03-10T14:04:36.901711+0000 mgr.gibba032.whopeo (mgr.24625) 668 : cluster [DBG] pgmap v365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:39.418 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:04:39.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:39 gibba032 bash[24004]: cephadm 2021-03-10T14:04:37.730722+0000 mgr.gibba032.whopeo (mgr.24625) 669 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:39.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:39 gibba011 bash[30791]: cephadm 2021-03-10T14:04:37.730722+0000 mgr.gibba032.whopeo (mgr.24625) 669 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:39.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:39 gibba011 bash[31774]: cephadm 2021-03-10T14:04:37.730722+0000 mgr.gibba032.whopeo (mgr.24625) 669 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:39.645 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:04:39.645 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (11m) 8m ago 16m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:04:39.645 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (16m) 6m ago 16m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:04:39.645 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (11m) 6m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:04:39.646 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (11m) 6m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:04:39.646 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (10m) 8m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:04:39.646 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (10m) 8m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:04:39.646 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (10m) 6m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:04:39.647 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (10m) 8m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:04:39.647 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (16m) 8m ago 16m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:04:39.647 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (16m) 6m ago 16m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:04:39.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:04:39.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:04:39.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (8m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:04:39.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (8m) 8m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:04:39.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (7m) 6m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:04:39.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (7m) 6m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:04:39.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (7m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:04:39.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (6m) 6m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:04:39.649 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (16m) 6m ago 16m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:04:39.649 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 14m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:04:39.894 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:04:39.894 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:04:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:04:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:04:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:04:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:04:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:04:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:04:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:04:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:04:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:04:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:04:39.898 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:04:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:40 gibba032 bash[24004]: cluster 2021-03-10T14:04:38.903235+0000 mgr.gibba032.whopeo (mgr.24625) 670 : cluster [DBG] pgmap v366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:40 gibba032 bash[24004]: audit 2021-03-10T14:04:39.407234+0000 mgr.gibba032.whopeo (mgr.24625) 671 : audit [DBG] from='client.44505 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:40 gibba032 bash[24004]: audit 2021-03-10T14:04:39.893787+0000 mon.a (mon.0) 759 : audit [DBG] from='client.? 172.21.2.111:0/1918147258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[30791]: cluster 2021-03-10T14:04:38.903235+0000 mgr.gibba032.whopeo (mgr.24625) 670 : cluster [DBG] pgmap v366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[30791]: audit 2021-03-10T14:04:39.407234+0000 mgr.gibba032.whopeo (mgr.24625) 671 : audit [DBG] from='client.44505 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[30791]: audit 2021-03-10T14:04:39.893787+0000 mon.a (mon.0) 759 : audit [DBG] from='client.? 172.21.2.111:0/1918147258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[31774]: cluster 2021-03-10T14:04:38.903235+0000 mgr.gibba032.whopeo (mgr.24625) 670 : cluster [DBG] pgmap v366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[31774]: audit 2021-03-10T14:04:39.407234+0000 mgr.gibba032.whopeo (mgr.24625) 671 : audit [DBG] from='client.44505 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:40 gibba011 bash[31774]: audit 2021-03-10T14:04:39.893787+0000 mon.a (mon.0) 759 : audit [DBG] from='client.? 172.21.2.111:0/1918147258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:04:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:41 gibba032 bash[24004]: audit 2021-03-10T14:04:39.631138+0000 mgr.gibba032.whopeo (mgr.24625) 672 : audit [DBG] from='client.44511 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:41.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:41 gibba011 bash[30791]: audit 2021-03-10T14:04:39.631138+0000 mgr.gibba032.whopeo (mgr.24625) 672 : audit [DBG] from='client.44511 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:41 gibba011 bash[31774]: audit 2021-03-10T14:04:39.631138+0000 mgr.gibba032.whopeo (mgr.24625) 672 : audit [DBG] from='client.44511 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:04:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:42 gibba032 bash[24004]: cluster 2021-03-10T14:04:40.904576+0000 mgr.gibba032.whopeo (mgr.24625) 673 : cluster [DBG] pgmap v367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:42 gibba011 bash[30791]: cluster 2021-03-10T14:04:40.904576+0000 mgr.gibba032.whopeo (mgr.24625) 673 : cluster [DBG] pgmap v367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:42 gibba011 bash[31774]: cluster 2021-03-10T14:04:40.904576+0000 mgr.gibba032.whopeo (mgr.24625) 673 : cluster [DBG] pgmap v367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:44 gibba032 bash[24004]: cluster 2021-03-10T14:04:42.906083+0000 mgr.gibba032.whopeo (mgr.24625) 674 : cluster [DBG] pgmap v368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:44 gibba011 bash[30791]: cluster 2021-03-10T14:04:42.906083+0000 mgr.gibba032.whopeo (mgr.24625) 674 : cluster [DBG] pgmap v368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:44 gibba011 bash[31774]: cluster 2021-03-10T14:04:42.906083+0000 mgr.gibba032.whopeo (mgr.24625) 674 : cluster [DBG] pgmap v368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:46 gibba032 bash[24004]: cluster 2021-03-10T14:04:44.907512+0000 mgr.gibba032.whopeo (mgr.24625) 675 : cluster [DBG] pgmap v369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:46 gibba011 bash[30791]: cluster 2021-03-10T14:04:44.907512+0000 mgr.gibba032.whopeo (mgr.24625) 675 : cluster [DBG] pgmap v369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:46 gibba011 bash[31774]: cluster 2021-03-10T14:04:44.907512+0000 mgr.gibba032.whopeo (mgr.24625) 675 : cluster [DBG] pgmap v369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:48 gibba032 bash[24004]: cluster 2021-03-10T14:04:46.909071+0000 mgr.gibba032.whopeo (mgr.24625) 676 : cluster [DBG] pgmap v370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:48 gibba011 bash[30791]: cluster 2021-03-10T14:04:46.909071+0000 mgr.gibba032.whopeo (mgr.24625) 676 : cluster [DBG] pgmap v370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:48 gibba011 bash[31774]: cluster 2021-03-10T14:04:46.909071+0000 mgr.gibba032.whopeo (mgr.24625) 676 : cluster [DBG] pgmap v370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:50 gibba011 bash[30791]: cluster 2021-03-10T14:04:48.910637+0000 mgr.gibba032.whopeo (mgr.24625) 677 : cluster [DBG] pgmap v371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:50 gibba011 bash[31774]: cluster 2021-03-10T14:04:48.910637+0000 mgr.gibba032.whopeo (mgr.24625) 677 : cluster [DBG] pgmap v371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:50 gibba032 bash[24004]: cluster 2021-03-10T14:04:48.910637+0000 mgr.gibba032.whopeo (mgr.24625) 677 : cluster [DBG] pgmap v371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:52 gibba011 bash[30791]: cluster 2021-03-10T14:04:50.912002+0000 mgr.gibba032.whopeo (mgr.24625) 678 : cluster [DBG] pgmap v372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:52 gibba011 bash[31774]: cluster 2021-03-10T14:04:50.912002+0000 mgr.gibba032.whopeo (mgr.24625) 678 : cluster [DBG] pgmap v372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:52.749 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:52 gibba032 bash[24004]: cluster 2021-03-10T14:04:50.912002+0000 mgr.gibba032.whopeo (mgr.24625) 678 : cluster [DBG] pgmap v372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:54 gibba011 bash[30791]: cephadm 2021-03-10T14:04:52.746652+0000 mgr.gibba032.whopeo (mgr.24625) 679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:54 gibba011 bash[30791]: cluster 2021-03-10T14:04:52.913755+0000 mgr.gibba032.whopeo (mgr.24625) 680 : cluster [DBG] pgmap v373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:54 gibba011 bash[31774]: cephadm 2021-03-10T14:04:52.746652+0000 mgr.gibba032.whopeo (mgr.24625) 679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:54 gibba011 bash[31774]: cluster 2021-03-10T14:04:52.913755+0000 mgr.gibba032.whopeo (mgr.24625) 680 : cluster [DBG] pgmap v373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:54 gibba032 bash[24004]: cephadm 2021-03-10T14:04:52.746652+0000 mgr.gibba032.whopeo (mgr.24625) 679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:04:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:54 gibba032 bash[24004]: cluster 2021-03-10T14:04:52.913755+0000 mgr.gibba032.whopeo (mgr.24625) 680 : cluster [DBG] pgmap v373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: cluster 2021-03-10T14:04:54.915230+0000 mgr.gibba032.whopeo (mgr.24625) 681 : cluster [DBG] pgmap v374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:55.957146+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:04:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:55.957932+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:04:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:55.958661+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:55.959417+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:56.059856+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:04:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[30791]: audit 2021-03-10T14:04:56.060582+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:04:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: cluster 2021-03-10T14:04:54.915230+0000 mgr.gibba032.whopeo (mgr.24625) 681 : cluster [DBG] pgmap v374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:55.957146+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:04:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:55.957932+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:04:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:55.958661+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:55.959417+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:56.059856+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:04:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:56 gibba011 bash[31774]: audit 2021-03-10T14:04:56.060582+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:04:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: cluster 2021-03-10T14:04:54.915230+0000 mgr.gibba032.whopeo (mgr.24625) 681 : cluster [DBG] pgmap v374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:55.957146+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:04:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:55.957932+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:04:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:55.958661+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:55.959417+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:04:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:56.059856+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:04:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:56 gibba032 bash[24004]: audit 2021-03-10T14:04:56.060582+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:04:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:04:58 gibba011 bash[30791]: cluster 2021-03-10T14:04:56.916729+0000 mgr.gibba032.whopeo (mgr.24625) 682 : cluster [DBG] pgmap v375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:04:58 gibba011 bash[31774]: cluster 2021-03-10T14:04:56.916729+0000 mgr.gibba032.whopeo (mgr.24625) 682 : cluster [DBG] pgmap v375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:04:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:04:58 gibba032 bash[24004]: cluster 2021-03-10T14:04:56.916729+0000 mgr.gibba032.whopeo (mgr.24625) 682 : cluster [DBG] pgmap v375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:00 gibba011 bash[30791]: cluster 2021-03-10T14:04:58.918296+0000 mgr.gibba032.whopeo (mgr.24625) 683 : cluster [DBG] pgmap v376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:00 gibba011 bash[31774]: cluster 2021-03-10T14:04:58.918296+0000 mgr.gibba032.whopeo (mgr.24625) 683 : cluster [DBG] pgmap v376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:00 gibba032 bash[24004]: cluster 2021-03-10T14:04:58.918296+0000 mgr.gibba032.whopeo (mgr.24625) 683 : cluster [DBG] pgmap v376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:02 gibba011 bash[30791]: cluster 2021-03-10T14:05:00.919783+0000 mgr.gibba032.whopeo (mgr.24625) 684 : cluster [DBG] pgmap v377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:02 gibba011 bash[31774]: cluster 2021-03-10T14:05:00.919783+0000 mgr.gibba032.whopeo (mgr.24625) 684 : cluster [DBG] pgmap v377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:02 gibba032 bash[24004]: cluster 2021-03-10T14:05:00.919783+0000 mgr.gibba032.whopeo (mgr.24625) 684 : cluster [DBG] pgmap v377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:04 gibba011 bash[30791]: cluster 2021-03-10T14:05:02.921527+0000 mgr.gibba032.whopeo (mgr.24625) 685 : cluster [DBG] pgmap v378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:04 gibba011 bash[31774]: cluster 2021-03-10T14:05:02.921527+0000 mgr.gibba032.whopeo (mgr.24625) 685 : cluster [DBG] pgmap v378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:04 gibba032 bash[24004]: cluster 2021-03-10T14:05:02.921527+0000 mgr.gibba032.whopeo (mgr.24625) 685 : cluster [DBG] pgmap v378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:06 gibba032 bash[24004]: cluster 2021-03-10T14:05:04.923003+0000 mgr.gibba032.whopeo (mgr.24625) 686 : cluster [DBG] pgmap v379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:06 gibba011 bash[30791]: cluster 2021-03-10T14:05:04.923003+0000 mgr.gibba032.whopeo (mgr.24625) 686 : cluster [DBG] pgmap v379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:06 gibba011 bash[31774]: cluster 2021-03-10T14:05:04.923003+0000 mgr.gibba032.whopeo (mgr.24625) 686 : cluster [DBG] pgmap v379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: cluster 2021-03-10T14:05:06.924745+0000 mgr.gibba032.whopeo (mgr.24625) 687 : cluster [DBG] pgmap v380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.011195+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:05:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.014842+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:05:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.015883+0000 mon.a (mon.0) 768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.016996+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.018002+0000 mon.a (mon.0) 770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.019873+0000 mon.a (mon.0) 771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:08 gibba032 bash[24004]: audit 2021-03-10T14:05:08.021183+0000 mon.a (mon.0) 772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: cluster 2021-03-10T14:05:06.924745+0000 mgr.gibba032.whopeo (mgr.24625) 687 : cluster [DBG] pgmap v380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.011195+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:05:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.014842+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:05:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.015883+0000 mon.a (mon.0) 768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.016996+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.018002+0000 mon.a (mon.0) 770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.019873+0000 mon.a (mon.0) 771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[30791]: audit 2021-03-10T14:05:08.021183+0000 mon.a (mon.0) 772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: cluster 2021-03-10T14:05:06.924745+0000 mgr.gibba032.whopeo (mgr.24625) 687 : cluster [DBG] pgmap v380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.011195+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:05:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.014842+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:05:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.015883+0000 mon.a (mon.0) 768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.016996+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.018002+0000 mon.a (mon.0) 770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.019873+0000 mon.a (mon.0) 771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:08 gibba011 bash[31774]: audit 2021-03-10T14:05:08.021183+0000 mon.a (mon.0) 772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:09.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:07.753059+0000 mgr.gibba032.whopeo (mgr.24625) 688 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:05:09.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:07.754607+0000 mgr.gibba032.whopeo (mgr.24625) 689 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:05:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.012336+0000 mgr.gibba032.whopeo (mgr.24625) 690 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:05:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.013806+0000 mgr.gibba032.whopeo (mgr.24625) 691 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:05:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.014852+0000 mgr.gibba032.whopeo (mgr.24625) 692 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:05:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.014904+0000 mgr.gibba032.whopeo (mgr.24625) 693 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:05:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.015957+0000 mgr.gibba032.whopeo (mgr.24625) 694 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:05:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.015998+0000 mgr.gibba032.whopeo (mgr.24625) 695 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:05:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.016887+0000 mgr.gibba032.whopeo (mgr.24625) 696 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:05:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.017693+0000 mgr.gibba032.whopeo (mgr.24625) 697 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:05:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.017734+0000 mgr.gibba032.whopeo (mgr.24625) 698 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:05:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.018778+0000 mgr.gibba032.whopeo (mgr.24625) 699 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:05:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.019117+0000 mgr.gibba032.whopeo (mgr.24625) 700 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:05:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.020124+0000 mgr.gibba032.whopeo (mgr.24625) 701 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:05:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.020172+0000 mgr.gibba032.whopeo (mgr.24625) 702 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:05:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:09 gibba032 bash[24004]: cephadm 2021-03-10T14:05:08.020292+0000 mgr.gibba032.whopeo (mgr.24625) 703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:07.753059+0000 mgr.gibba032.whopeo (mgr.24625) 688 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:05:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:07.754607+0000 mgr.gibba032.whopeo (mgr.24625) 689 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:05:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.012336+0000 mgr.gibba032.whopeo (mgr.24625) 690 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:05:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.013806+0000 mgr.gibba032.whopeo (mgr.24625) 691 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:05:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.014852+0000 mgr.gibba032.whopeo (mgr.24625) 692 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:05:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.014904+0000 mgr.gibba032.whopeo (mgr.24625) 693 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:05:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.015957+0000 mgr.gibba032.whopeo (mgr.24625) 694 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:05:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.015998+0000 mgr.gibba032.whopeo (mgr.24625) 695 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:05:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.016887+0000 mgr.gibba032.whopeo (mgr.24625) 696 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:05:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.017693+0000 mgr.gibba032.whopeo (mgr.24625) 697 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:05:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.017734+0000 mgr.gibba032.whopeo (mgr.24625) 698 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:05:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.018778+0000 mgr.gibba032.whopeo (mgr.24625) 699 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:05:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.019117+0000 mgr.gibba032.whopeo (mgr.24625) 700 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:05:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.020124+0000 mgr.gibba032.whopeo (mgr.24625) 701 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:05:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.020172+0000 mgr.gibba032.whopeo (mgr.24625) 702 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:05:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[30791]: cephadm 2021-03-10T14:05:08.020292+0000 mgr.gibba032.whopeo (mgr.24625) 703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:09.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:07.753059+0000 mgr.gibba032.whopeo (mgr.24625) 688 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:05:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:07.754607+0000 mgr.gibba032.whopeo (mgr.24625) 689 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:05:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.012336+0000 mgr.gibba032.whopeo (mgr.24625) 690 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:05:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.013806+0000 mgr.gibba032.whopeo (mgr.24625) 691 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:05:09.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.014852+0000 mgr.gibba032.whopeo (mgr.24625) 692 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:05:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.014904+0000 mgr.gibba032.whopeo (mgr.24625) 693 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:05:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.015957+0000 mgr.gibba032.whopeo (mgr.24625) 694 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:05:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.015998+0000 mgr.gibba032.whopeo (mgr.24625) 695 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:05:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.016887+0000 mgr.gibba032.whopeo (mgr.24625) 696 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:05:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.017693+0000 mgr.gibba032.whopeo (mgr.24625) 697 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:05:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.017734+0000 mgr.gibba032.whopeo (mgr.24625) 698 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:05:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.018778+0000 mgr.gibba032.whopeo (mgr.24625) 699 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:05:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.019117+0000 mgr.gibba032.whopeo (mgr.24625) 700 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:05:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.020124+0000 mgr.gibba032.whopeo (mgr.24625) 701 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:05:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.020172+0000 mgr.gibba032.whopeo (mgr.24625) 702 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:05:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:09 gibba011 bash[31774]: cephadm 2021-03-10T14:05:08.020292+0000 mgr.gibba032.whopeo (mgr.24625) 703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:10.143 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:05:10.366 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:05:10.367 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (11m) 8m ago 17m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:05:10.367 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (17m) 7m ago 17m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:05:10.367 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (12m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:05:10.367 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (12m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:05:10.368 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (11m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:05:10.368 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (11m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:05:10.368 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (10m) 7m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:05:10.368 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (11m) 8m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:05:10.368 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (17m) 8m ago 17m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:05:10.369 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (17m) 7m ago 17m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:05:10.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (10m) 8m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:05:10.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (9m) 8m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:05:10.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:05:10.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (8m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:05:10.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:05:10.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (7m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:05:10.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (7m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:05:10.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (7m) 7m ago 17m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:05:10.371 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (16m) 7m ago 17m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:05:10.371 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 15m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:05:10.615 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:05:10.615 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:05:10.615 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:05:10.615 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:10.616 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:05:10.616 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:05:10.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:10.616 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:05:10.617 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:05:10.617 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:10.617 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:05:10.617 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:05:10.617 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:05:10.618 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:05:10.618 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:05:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:10 gibba032 bash[24004]: cluster 2021-03-10T14:05:08.926330+0000 mgr.gibba032.whopeo (mgr.24625) 704 : cluster [DBG] pgmap v381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:10 gibba011 bash[30791]: cluster 2021-03-10T14:05:08.926330+0000 mgr.gibba032.whopeo (mgr.24625) 704 : cluster [DBG] pgmap v381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:10 gibba011 bash[31774]: cluster 2021-03-10T14:05:08.926330+0000 mgr.gibba032.whopeo (mgr.24625) 704 : cluster [DBG] pgmap v381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:11 gibba032 bash[24004]: audit 2021-03-10T14:05:10.125463+0000 mgr.gibba032.whopeo (mgr.24625) 705 : audit [DBG] from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:11 gibba032 bash[24004]: audit 2021-03-10T14:05:10.356991+0000 mgr.gibba032.whopeo (mgr.24625) 706 : audit [DBG] from='client.54514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:11 gibba032 bash[24004]: audit 2021-03-10T14:05:10.614823+0000 mon.a (mon.0) 773 : audit [DBG] from='client.? 172.21.2.111:0/204764317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[30791]: audit 2021-03-10T14:05:10.125463+0000 mgr.gibba032.whopeo (mgr.24625) 705 : audit [DBG] from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[30791]: audit 2021-03-10T14:05:10.356991+0000 mgr.gibba032.whopeo (mgr.24625) 706 : audit [DBG] from='client.54514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[30791]: audit 2021-03-10T14:05:10.614823+0000 mon.a (mon.0) 773 : audit [DBG] from='client.? 172.21.2.111:0/204764317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[31774]: audit 2021-03-10T14:05:10.125463+0000 mgr.gibba032.whopeo (mgr.24625) 705 : audit [DBG] from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[31774]: audit 2021-03-10T14:05:10.356991+0000 mgr.gibba032.whopeo (mgr.24625) 706 : audit [DBG] from='client.54514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:11 gibba011 bash[31774]: audit 2021-03-10T14:05:10.614823+0000 mon.a (mon.0) 773 : audit [DBG] from='client.? 172.21.2.111:0/204764317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:12 gibba032 bash[24004]: cluster 2021-03-10T14:05:10.927667+0000 mgr.gibba032.whopeo (mgr.24625) 707 : cluster [DBG] pgmap v382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:12 gibba011 bash[30791]: cluster 2021-03-10T14:05:10.927667+0000 mgr.gibba032.whopeo (mgr.24625) 707 : cluster [DBG] pgmap v382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:12 gibba011 bash[31774]: cluster 2021-03-10T14:05:10.927667+0000 mgr.gibba032.whopeo (mgr.24625) 707 : cluster [DBG] pgmap v382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:14 gibba032 bash[24004]: cluster 2021-03-10T14:05:12.929393+0000 mgr.gibba032.whopeo (mgr.24625) 708 : cluster [DBG] pgmap v383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:14 gibba011 bash[30791]: cluster 2021-03-10T14:05:12.929393+0000 mgr.gibba032.whopeo (mgr.24625) 708 : cluster [DBG] pgmap v383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:14 gibba011 bash[31774]: cluster 2021-03-10T14:05:12.929393+0000 mgr.gibba032.whopeo (mgr.24625) 708 : cluster [DBG] pgmap v383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:16 gibba032 bash[24004]: cluster 2021-03-10T14:05:14.930767+0000 mgr.gibba032.whopeo (mgr.24625) 709 : cluster [DBG] pgmap v384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:16 gibba011 bash[30791]: cluster 2021-03-10T14:05:14.930767+0000 mgr.gibba032.whopeo (mgr.24625) 709 : cluster [DBG] pgmap v384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:16 gibba011 bash[31774]: cluster 2021-03-10T14:05:14.930767+0000 mgr.gibba032.whopeo (mgr.24625) 709 : cluster [DBG] pgmap v384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:18 gibba032 bash[24004]: cluster 2021-03-10T14:05:16.932554+0000 mgr.gibba032.whopeo (mgr.24625) 710 : cluster [DBG] pgmap v385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:18 gibba011 bash[30791]: cluster 2021-03-10T14:05:16.932554+0000 mgr.gibba032.whopeo (mgr.24625) 710 : cluster [DBG] pgmap v385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:18 gibba011 bash[31774]: cluster 2021-03-10T14:05:16.932554+0000 mgr.gibba032.whopeo (mgr.24625) 710 : cluster [DBG] pgmap v385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:20 gibba032 bash[24004]: cluster 2021-03-10T14:05:18.933936+0000 mgr.gibba032.whopeo (mgr.24625) 711 : cluster [DBG] pgmap v386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:20 gibba011 bash[30791]: cluster 2021-03-10T14:05:18.933936+0000 mgr.gibba032.whopeo (mgr.24625) 711 : cluster [DBG] pgmap v386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:20 gibba011 bash[31774]: cluster 2021-03-10T14:05:18.933936+0000 mgr.gibba032.whopeo (mgr.24625) 711 : cluster [DBG] pgmap v386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:22 gibba032 bash[24004]: cluster 2021-03-10T14:05:20.935295+0000 mgr.gibba032.whopeo (mgr.24625) 712 : cluster [DBG] pgmap v387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:22 gibba011 bash[30791]: cluster 2021-03-10T14:05:20.935295+0000 mgr.gibba032.whopeo (mgr.24625) 712 : cluster [DBG] pgmap v387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:22 gibba011 bash[31774]: cluster 2021-03-10T14:05:20.935295+0000 mgr.gibba032.whopeo (mgr.24625) 712 : cluster [DBG] pgmap v387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:24 gibba032 bash[24004]: cluster 2021-03-10T14:05:22.937246+0000 mgr.gibba032.whopeo (mgr.24625) 713 : cluster [DBG] pgmap v388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:24 gibba032 bash[24004]: cephadm 2021-03-10T14:05:23.032521+0000 mgr.gibba032.whopeo (mgr.24625) 714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:24 gibba011 bash[30791]: cluster 2021-03-10T14:05:22.937246+0000 mgr.gibba032.whopeo (mgr.24625) 713 : cluster [DBG] pgmap v388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:24 gibba011 bash[30791]: cephadm 2021-03-10T14:05:23.032521+0000 mgr.gibba032.whopeo (mgr.24625) 714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:24 gibba011 bash[31774]: cluster 2021-03-10T14:05:22.937246+0000 mgr.gibba032.whopeo (mgr.24625) 713 : cluster [DBG] pgmap v388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:24 gibba011 bash[31774]: cephadm 2021-03-10T14:05:23.032521+0000 mgr.gibba032.whopeo (mgr.24625) 714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:26 gibba032 bash[24004]: cluster 2021-03-10T14:05:24.938767+0000 mgr.gibba032.whopeo (mgr.24625) 715 : cluster [DBG] pgmap v389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:26 gibba011 bash[30791]: cluster 2021-03-10T14:05:24.938767+0000 mgr.gibba032.whopeo (mgr.24625) 715 : cluster [DBG] pgmap v389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:26 gibba011 bash[31774]: cluster 2021-03-10T14:05:24.938767+0000 mgr.gibba032.whopeo (mgr.24625) 715 : cluster [DBG] pgmap v389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:28 gibba032 bash[24004]: cluster 2021-03-10T14:05:26.940542+0000 mgr.gibba032.whopeo (mgr.24625) 716 : cluster [DBG] pgmap v390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:28 gibba011 bash[30791]: cluster 2021-03-10T14:05:26.940542+0000 mgr.gibba032.whopeo (mgr.24625) 716 : cluster [DBG] pgmap v390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:28 gibba011 bash[31774]: cluster 2021-03-10T14:05:26.940542+0000 mgr.gibba032.whopeo (mgr.24625) 716 : cluster [DBG] pgmap v390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:30 gibba032 bash[24004]: cluster 2021-03-10T14:05:28.942142+0000 mgr.gibba032.whopeo (mgr.24625) 717 : cluster [DBG] pgmap v391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:30 gibba011 bash[30791]: cluster 2021-03-10T14:05:28.942142+0000 mgr.gibba032.whopeo (mgr.24625) 717 : cluster [DBG] pgmap v391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:30 gibba011 bash[31774]: cluster 2021-03-10T14:05:28.942142+0000 mgr.gibba032.whopeo (mgr.24625) 717 : cluster [DBG] pgmap v391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:32 gibba032 bash[24004]: cluster 2021-03-10T14:05:30.943467+0000 mgr.gibba032.whopeo (mgr.24625) 718 : cluster [DBG] pgmap v392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:32 gibba011 bash[30791]: cluster 2021-03-10T14:05:30.943467+0000 mgr.gibba032.whopeo (mgr.24625) 718 : cluster [DBG] pgmap v392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:32 gibba011 bash[31774]: cluster 2021-03-10T14:05:30.943467+0000 mgr.gibba032.whopeo (mgr.24625) 718 : cluster [DBG] pgmap v392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:34 gibba032 bash[24004]: cluster 2021-03-10T14:05:32.945288+0000 mgr.gibba032.whopeo (mgr.24625) 719 : cluster [DBG] pgmap v393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:34 gibba011 bash[30791]: cluster 2021-03-10T14:05:32.945288+0000 mgr.gibba032.whopeo (mgr.24625) 719 : cluster [DBG] pgmap v393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:34 gibba011 bash[31774]: cluster 2021-03-10T14:05:32.945288+0000 mgr.gibba032.whopeo (mgr.24625) 719 : cluster [DBG] pgmap v393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:36 gibba032 bash[24004]: cluster 2021-03-10T14:05:34.946643+0000 mgr.gibba032.whopeo (mgr.24625) 720 : cluster [DBG] pgmap v394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:36 gibba011 bash[30791]: cluster 2021-03-10T14:05:34.946643+0000 mgr.gibba032.whopeo (mgr.24625) 720 : cluster [DBG] pgmap v394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:36 gibba011 bash[31774]: cluster 2021-03-10T14:05:34.946643+0000 mgr.gibba032.whopeo (mgr.24625) 720 : cluster [DBG] pgmap v394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:38 gibba032 bash[24004]: cluster 2021-03-10T14:05:36.948443+0000 mgr.gibba032.whopeo (mgr.24625) 721 : cluster [DBG] pgmap v395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:38 gibba011 bash[30791]: cluster 2021-03-10T14:05:36.948443+0000 mgr.gibba032.whopeo (mgr.24625) 721 : cluster [DBG] pgmap v395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:38 gibba011 bash[31774]: cluster 2021-03-10T14:05:36.948443+0000 mgr.gibba032.whopeo (mgr.24625) 721 : cluster [DBG] pgmap v395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:39 gibba032 bash[24004]: cephadm 2021-03-10T14:05:38.048324+0000 mgr.gibba032.whopeo (mgr.24625) 722 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:39 gibba011 bash[30791]: cephadm 2021-03-10T14:05:38.048324+0000 mgr.gibba032.whopeo (mgr.24625) 722 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:39.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:39 gibba011 bash[31774]: cephadm 2021-03-10T14:05:38.048324+0000 mgr.gibba032.whopeo (mgr.24625) 722 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:40 gibba032 bash[24004]: cluster 2021-03-10T14:05:38.950127+0000 mgr.gibba032.whopeo (mgr.24625) 723 : cluster [DBG] pgmap v396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:40 gibba011 bash[30791]: cluster 2021-03-10T14:05:38.950127+0000 mgr.gibba032.whopeo (mgr.24625) 723 : cluster [DBG] pgmap v396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:40 gibba011 bash[31774]: cluster 2021-03-10T14:05:38.950127+0000 mgr.gibba032.whopeo (mgr.24625) 723 : cluster [DBG] pgmap v396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:40.859 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:05:41.088 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:05:41.089 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (12m) 9m ago 17m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:05:41.089 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (17m) 7m ago 17m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:05:41.089 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (12m) 7m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:05:41.089 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (12m) 7m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:05:41.090 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (12m) 9m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:05:41.090 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (11m) 9m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:05:41.090 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (11m) 7m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:05:41.091 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (11m) 9m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:05:41.091 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (17m) 9m ago 17m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:05:41.091 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (17m) 7m ago 17m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:05:41.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:05:41.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:05:41.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (9m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:05:41.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (9m) 9m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:05:41.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (8m) 7m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:05:41.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (8m) 7m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:05:41.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (8m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:05:41.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (7m) 7m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:05:41.093 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (17m) 7m ago 17m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:05:41.093 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 15m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:05:41.337 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:05:41.337 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:05:41.337 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:05:41.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:41.338 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:05:41.338 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:05:41.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:41.338 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:05:41.339 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:05:41.339 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:05:41.339 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:05:41.339 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:05:41.340 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:05:41.340 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:05:41.340 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:05:41.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:41 gibba011 bash[30791]: audit 2021-03-10T14:05:41.336917+0000 mon.a (mon.0) 774 : audit [DBG] from='client.? 172.21.2.111:0/1443901755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:41.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:41 gibba011 bash[31774]: audit 2021-03-10T14:05:41.336917+0000 mon.a (mon.0) 774 : audit [DBG] from='client.? 172.21.2.111:0/1443901755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:41 gibba032 bash[24004]: audit 2021-03-10T14:05:41.336917+0000 mon.a (mon.0) 774 : audit [DBG] from='client.? 172.21.2.111:0/1443901755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:05:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[30791]: audit 2021-03-10T14:05:40.848651+0000 mgr.gibba032.whopeo (mgr.24625) 724 : audit [DBG] from='client.54526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[30791]: cluster 2021-03-10T14:05:40.951535+0000 mgr.gibba032.whopeo (mgr.24625) 725 : cluster [DBG] pgmap v397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[30791]: audit 2021-03-10T14:05:41.073898+0000 mgr.gibba032.whopeo (mgr.24625) 726 : audit [DBG] from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[31774]: audit 2021-03-10T14:05:40.848651+0000 mgr.gibba032.whopeo (mgr.24625) 724 : audit [DBG] from='client.54526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[31774]: cluster 2021-03-10T14:05:40.951535+0000 mgr.gibba032.whopeo (mgr.24625) 725 : cluster [DBG] pgmap v397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:42 gibba011 bash[31774]: audit 2021-03-10T14:05:41.073898+0000 mgr.gibba032.whopeo (mgr.24625) 726 : audit [DBG] from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:43.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:42 gibba032 bash[24004]: audit 2021-03-10T14:05:40.848651+0000 mgr.gibba032.whopeo (mgr.24625) 724 : audit [DBG] from='client.54526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:42 gibba032 bash[24004]: cluster 2021-03-10T14:05:40.951535+0000 mgr.gibba032.whopeo (mgr.24625) 725 : cluster [DBG] pgmap v397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:42 gibba032 bash[24004]: audit 2021-03-10T14:05:41.073898+0000 mgr.gibba032.whopeo (mgr.24625) 726 : audit [DBG] from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:05:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:44 gibba011 bash[30791]: cluster 2021-03-10T14:05:42.953412+0000 mgr.gibba032.whopeo (mgr.24625) 727 : cluster [DBG] pgmap v398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:44 gibba011 bash[31774]: cluster 2021-03-10T14:05:42.953412+0000 mgr.gibba032.whopeo (mgr.24625) 727 : cluster [DBG] pgmap v398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:44 gibba032 bash[24004]: cluster 2021-03-10T14:05:42.953412+0000 mgr.gibba032.whopeo (mgr.24625) 727 : cluster [DBG] pgmap v398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:46 gibba011 bash[30791]: cluster 2021-03-10T14:05:44.954699+0000 mgr.gibba032.whopeo (mgr.24625) 728 : cluster [DBG] pgmap v399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:46 gibba011 bash[31774]: cluster 2021-03-10T14:05:44.954699+0000 mgr.gibba032.whopeo (mgr.24625) 728 : cluster [DBG] pgmap v399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:46 gibba032 bash[24004]: cluster 2021-03-10T14:05:44.954699+0000 mgr.gibba032.whopeo (mgr.24625) 728 : cluster [DBG] pgmap v399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:48 gibba011 bash[30791]: cluster 2021-03-10T14:05:46.956478+0000 mgr.gibba032.whopeo (mgr.24625) 729 : cluster [DBG] pgmap v400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:48 gibba011 bash[31774]: cluster 2021-03-10T14:05:46.956478+0000 mgr.gibba032.whopeo (mgr.24625) 729 : cluster [DBG] pgmap v400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:49.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:48 gibba032 bash[24004]: cluster 2021-03-10T14:05:46.956478+0000 mgr.gibba032.whopeo (mgr.24625) 729 : cluster [DBG] pgmap v400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:50 gibba011 bash[30791]: cluster 2021-03-10T14:05:48.957921+0000 mgr.gibba032.whopeo (mgr.24625) 730 : cluster [DBG] pgmap v401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:50 gibba011 bash[31774]: cluster 2021-03-10T14:05:48.957921+0000 mgr.gibba032.whopeo (mgr.24625) 730 : cluster [DBG] pgmap v401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:50 gibba032 bash[24004]: cluster 2021-03-10T14:05:48.957921+0000 mgr.gibba032.whopeo (mgr.24625) 730 : cluster [DBG] pgmap v401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:52 gibba011 bash[30791]: cluster 2021-03-10T14:05:50.959265+0000 mgr.gibba032.whopeo (mgr.24625) 731 : cluster [DBG] pgmap v402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:52 gibba011 bash[31774]: cluster 2021-03-10T14:05:50.959265+0000 mgr.gibba032.whopeo (mgr.24625) 731 : cluster [DBG] pgmap v402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:52 gibba032 bash[24004]: cluster 2021-03-10T14:05:50.959265+0000 mgr.gibba032.whopeo (mgr.24625) 731 : cluster [DBG] pgmap v402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:53.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:53 gibba011 bash[30791]: cluster 2021-03-10T14:05:52.961162+0000 mgr.gibba032.whopeo (mgr.24625) 732 : cluster [DBG] pgmap v403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:53.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:53 gibba011 bash[30791]: cephadm 2021-03-10T14:05:53.063951+0000 mgr.gibba032.whopeo (mgr.24625) 733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:53.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:53 gibba011 bash[31774]: cluster 2021-03-10T14:05:52.961162+0000 mgr.gibba032.whopeo (mgr.24625) 732 : cluster [DBG] pgmap v403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:53.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:53 gibba011 bash[31774]: cephadm 2021-03-10T14:05:53.063951+0000 mgr.gibba032.whopeo (mgr.24625) 733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:54.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:53 gibba032 bash[24004]: cluster 2021-03-10T14:05:52.961162+0000 mgr.gibba032.whopeo (mgr.24625) 732 : cluster [DBG] pgmap v403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:54.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:53 gibba032 bash[24004]: cephadm 2021-03-10T14:05:53.063951+0000 mgr.gibba032.whopeo (mgr.24625) 733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:05:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: cluster 2021-03-10T14:05:54.962531+0000 mgr.gibba032.whopeo (mgr.24625) 734 : cluster [DBG] pgmap v404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:55.969318+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:05:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:55.970082+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:05:56.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:55.970777+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:55.971490+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:56.068431+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:05:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:56 gibba032 bash[24004]: audit 2021-03-10T14:05:56.075289+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:05:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: cluster 2021-03-10T14:05:54.962531+0000 mgr.gibba032.whopeo (mgr.24625) 734 : cluster [DBG] pgmap v404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:55.969318+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:05:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:55.970082+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:05:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:55.970777+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:55.971490+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:56.068431+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:05:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[30791]: audit 2021-03-10T14:05:56.075289+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:05:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: cluster 2021-03-10T14:05:54.962531+0000 mgr.gibba032.whopeo (mgr.24625) 734 : cluster [DBG] pgmap v404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:55.969318+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:05:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:55.970082+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:05:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:55.970777+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:55.971490+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:05:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:56.068431+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:05:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:56 gibba011 bash[31774]: audit 2021-03-10T14:05:56.075289+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:05:58 gibba032 bash[24004]: cluster 2021-03-10T14:05:56.964439+0000 mgr.gibba032.whopeo (mgr.24625) 735 : cluster [DBG] pgmap v405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:05:58 gibba011 bash[30791]: cluster 2021-03-10T14:05:56.964439+0000 mgr.gibba032.whopeo (mgr.24625) 735 : cluster [DBG] pgmap v405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:05:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:05:58 gibba011 bash[31774]: cluster 2021-03-10T14:05:56.964439+0000 mgr.gibba032.whopeo (mgr.24625) 735 : cluster [DBG] pgmap v405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:00 gibba032 bash[24004]: cluster 2021-03-10T14:05:58.966251+0000 mgr.gibba032.whopeo (mgr.24625) 736 : cluster [DBG] pgmap v406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:00 gibba011 bash[30791]: cluster 2021-03-10T14:05:58.966251+0000 mgr.gibba032.whopeo (mgr.24625) 736 : cluster [DBG] pgmap v406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:00 gibba011 bash[31774]: cluster 2021-03-10T14:05:58.966251+0000 mgr.gibba032.whopeo (mgr.24625) 736 : cluster [DBG] pgmap v406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:02 gibba032 bash[24004]: cluster 2021-03-10T14:06:00.967743+0000 mgr.gibba032.whopeo (mgr.24625) 737 : cluster [DBG] pgmap v407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:02 gibba011 bash[30791]: cluster 2021-03-10T14:06:00.967743+0000 mgr.gibba032.whopeo (mgr.24625) 737 : cluster [DBG] pgmap v407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:02 gibba011 bash[31774]: cluster 2021-03-10T14:06:00.967743+0000 mgr.gibba032.whopeo (mgr.24625) 737 : cluster [DBG] pgmap v407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:04 gibba032 bash[24004]: cluster 2021-03-10T14:06:02.969642+0000 mgr.gibba032.whopeo (mgr.24625) 738 : cluster [DBG] pgmap v408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:04 gibba011 bash[30791]: cluster 2021-03-10T14:06:02.969642+0000 mgr.gibba032.whopeo (mgr.24625) 738 : cluster [DBG] pgmap v408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:04 gibba011 bash[31774]: cluster 2021-03-10T14:06:02.969642+0000 mgr.gibba032.whopeo (mgr.24625) 738 : cluster [DBG] pgmap v408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:06 gibba032 bash[24004]: cluster 2021-03-10T14:06:04.971079+0000 mgr.gibba032.whopeo (mgr.24625) 739 : cluster [DBG] pgmap v409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:06 gibba011 bash[30791]: cluster 2021-03-10T14:06:04.971079+0000 mgr.gibba032.whopeo (mgr.24625) 739 : cluster [DBG] pgmap v409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:06 gibba011 bash[31774]: cluster 2021-03-10T14:06:04.971079+0000 mgr.gibba032.whopeo (mgr.24625) 739 : cluster [DBG] pgmap v409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:08.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:08 gibba032 bash[24004]: cluster 2021-03-10T14:06:06.973038+0000 mgr.gibba032.whopeo (mgr.24625) 740 : cluster [DBG] pgmap v410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:08 gibba011 bash[30791]: cluster 2021-03-10T14:06:06.973038+0000 mgr.gibba032.whopeo (mgr.24625) 740 : cluster [DBG] pgmap v410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:08 gibba011 bash[31774]: cluster 2021-03-10T14:06:06.973038+0000 mgr.gibba032.whopeo (mgr.24625) 740 : cluster [DBG] pgmap v410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.080729+0000 mgr.gibba032.whopeo (mgr.24625) 741 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:06:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.082290+0000 mgr.gibba032.whopeo (mgr.24625) 742 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:06:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.299084+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:06:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.299303+0000 mgr.gibba032.whopeo (mgr.24625) 743 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:06:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.300870+0000 mgr.gibba032.whopeo (mgr.24625) 744 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:06:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.302083+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:06:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.302187+0000 mgr.gibba032.whopeo (mgr.24625) 745 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:06:09.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.302226+0000 mgr.gibba032.whopeo (mgr.24625) 746 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:06:09.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.303248+0000 mon.a (mon.0) 783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.303306+0000 mgr.gibba032.whopeo (mgr.24625) 747 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:06:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.303346+0000 mgr.gibba032.whopeo (mgr.24625) 748 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:06:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.304357+0000 mgr.gibba032.whopeo (mgr.24625) 749 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:06:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.304632+0000 mon.a (mon.0) 784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.305294+0000 mgr.gibba032.whopeo (mgr.24625) 750 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:06:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.305336+0000 mgr.gibba032.whopeo (mgr.24625) 751 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:06:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.305681+0000 mon.a (mon.0) 785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.306402+0000 mgr.gibba032.whopeo (mgr.24625) 752 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:06:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.306719+0000 mgr.gibba032.whopeo (mgr.24625) 753 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:06:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.307660+0000 mgr.gibba032.whopeo (mgr.24625) 754 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:06:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.307700+0000 mgr.gibba032.whopeo (mgr.24625) 755 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:06:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: cephadm 2021-03-10T14:06:08.307805+0000 mgr.gibba032.whopeo (mgr.24625) 756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.307848+0000 mon.a (mon.0) 786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[30791]: audit 2021-03-10T14:06:08.309085+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.080729+0000 mgr.gibba032.whopeo (mgr.24625) 741 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:06:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.082290+0000 mgr.gibba032.whopeo (mgr.24625) 742 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:06:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.299084+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:06:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.299303+0000 mgr.gibba032.whopeo (mgr.24625) 743 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:06:09.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.300870+0000 mgr.gibba032.whopeo (mgr.24625) 744 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:06:09.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.302083+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:06:09.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.302187+0000 mgr.gibba032.whopeo (mgr.24625) 745 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:06:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.302226+0000 mgr.gibba032.whopeo (mgr.24625) 746 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:06:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.303248+0000 mon.a (mon.0) 783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.303306+0000 mgr.gibba032.whopeo (mgr.24625) 747 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:06:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.303346+0000 mgr.gibba032.whopeo (mgr.24625) 748 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:06:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.304357+0000 mgr.gibba032.whopeo (mgr.24625) 749 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:06:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.304632+0000 mon.a (mon.0) 784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.305294+0000 mgr.gibba032.whopeo (mgr.24625) 750 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:06:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.305336+0000 mgr.gibba032.whopeo (mgr.24625) 751 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:06:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.305681+0000 mon.a (mon.0) 785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.306402+0000 mgr.gibba032.whopeo (mgr.24625) 752 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:06:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.306719+0000 mgr.gibba032.whopeo (mgr.24625) 753 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:06:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.307660+0000 mgr.gibba032.whopeo (mgr.24625) 754 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:06:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.307700+0000 mgr.gibba032.whopeo (mgr.24625) 755 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:06:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: cephadm 2021-03-10T14:06:08.307805+0000 mgr.gibba032.whopeo (mgr.24625) 756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.307848+0000 mon.a (mon.0) 786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:09 gibba011 bash[31774]: audit 2021-03-10T14:06:08.309085+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.080729+0000 mgr.gibba032.whopeo (mgr.24625) 741 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:06:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.082290+0000 mgr.gibba032.whopeo (mgr.24625) 742 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:06:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.299084+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:06:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.299303+0000 mgr.gibba032.whopeo (mgr.24625) 743 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:06:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.300870+0000 mgr.gibba032.whopeo (mgr.24625) 744 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:06:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.302083+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:06:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.302187+0000 mgr.gibba032.whopeo (mgr.24625) 745 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:06:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.302226+0000 mgr.gibba032.whopeo (mgr.24625) 746 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:06:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.303248+0000 mon.a (mon.0) 783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.303306+0000 mgr.gibba032.whopeo (mgr.24625) 747 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:06:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.303346+0000 mgr.gibba032.whopeo (mgr.24625) 748 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:06:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.304357+0000 mgr.gibba032.whopeo (mgr.24625) 749 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:06:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.304632+0000 mon.a (mon.0) 784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.305294+0000 mgr.gibba032.whopeo (mgr.24625) 750 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:06:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.305336+0000 mgr.gibba032.whopeo (mgr.24625) 751 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:06:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.305681+0000 mon.a (mon.0) 785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.306402+0000 mgr.gibba032.whopeo (mgr.24625) 752 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:06:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.306719+0000 mgr.gibba032.whopeo (mgr.24625) 753 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:06:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.307660+0000 mgr.gibba032.whopeo (mgr.24625) 754 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:06:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.307700+0000 mgr.gibba032.whopeo (mgr.24625) 755 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:06:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: cephadm 2021-03-10T14:06:08.307805+0000 mgr.gibba032.whopeo (mgr.24625) 756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.307848+0000 mon.a (mon.0) 786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:09 gibba032 bash[24004]: audit 2021-03-10T14:06:08.309085+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:10 gibba011 bash[30791]: cluster 2021-03-10T14:06:08.974687+0000 mgr.gibba032.whopeo (mgr.24625) 757 : cluster [DBG] pgmap v411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:10 gibba011 bash[31774]: cluster 2021-03-10T14:06:08.974687+0000 mgr.gibba032.whopeo (mgr.24625) 757 : cluster [DBG] pgmap v411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:10 gibba032 bash[24004]: cluster 2021-03-10T14:06:08.974687+0000 mgr.gibba032.whopeo (mgr.24625) 757 : cluster [DBG] pgmap v411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:11.576 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:06:11.801 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:06:11.801 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (12m) 9m ago 18m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:06:11.802 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (18m) 8m ago 18m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:06:11.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (13m) 8m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:06:11.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (13m) 8m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:06:11.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (12m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:06:11.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (12m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:06:11.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (11m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:06:11.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (12m) 9m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:06:11.803 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (18m) 9m ago 18m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:06:11.803 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (18m) 8m ago 18m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:06:11.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (11m) 9m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:06:11.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (10m) 9m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:06:11.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:06:11.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (9m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:06:11.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:06:11.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:06:11.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (8m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:06:11.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (8m) 8m ago 18m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:06:11.805 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (17m) 8m ago 18m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:06:11.805 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 16m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:06:12.055 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:06:12.055 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:06:12.055 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:06:12.056 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:12.056 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:06:12.056 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:06:12.056 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:12.056 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:06:12.057 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:06:12.057 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:12.057 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:06:12.057 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:06:12.057 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:06:12.058 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:06:12.058 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:06:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:12 gibba011 bash[30791]: cluster 2021-03-10T14:06:10.976157+0000 mgr.gibba032.whopeo (mgr.24625) 758 : cluster [DBG] pgmap v412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:12 gibba011 bash[30791]: audit 2021-03-10T14:06:12.054881+0000 mon.a (mon.0) 788 : audit [DBG] from='client.? 172.21.2.111:0/3814334495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:12 gibba011 bash[31774]: cluster 2021-03-10T14:06:10.976157+0000 mgr.gibba032.whopeo (mgr.24625) 758 : cluster [DBG] pgmap v412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:12 gibba011 bash[31774]: audit 2021-03-10T14:06:12.054881+0000 mon.a (mon.0) 788 : audit [DBG] from='client.? 172.21.2.111:0/3814334495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:12 gibba032 bash[24004]: cluster 2021-03-10T14:06:10.976157+0000 mgr.gibba032.whopeo (mgr.24625) 758 : cluster [DBG] pgmap v412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:12 gibba032 bash[24004]: audit 2021-03-10T14:06:12.054881+0000 mon.a (mon.0) 788 : audit [DBG] from='client.? 172.21.2.111:0/3814334495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:13 gibba032 bash[24004]: audit 2021-03-10T14:06:11.564817+0000 mgr.gibba032.whopeo (mgr.24625) 759 : audit [DBG] from='client.54544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:13 gibba032 bash[24004]: audit 2021-03-10T14:06:11.786947+0000 mgr.gibba032.whopeo (mgr.24625) 760 : audit [DBG] from='client.54550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:13.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:13 gibba011 bash[30791]: audit 2021-03-10T14:06:11.564817+0000 mgr.gibba032.whopeo (mgr.24625) 759 : audit [DBG] from='client.54544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:13.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:13 gibba011 bash[30791]: audit 2021-03-10T14:06:11.786947+0000 mgr.gibba032.whopeo (mgr.24625) 760 : audit [DBG] from='client.54550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:13.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:13 gibba011 bash[31774]: audit 2021-03-10T14:06:11.564817+0000 mgr.gibba032.whopeo (mgr.24625) 759 : audit [DBG] from='client.54544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:13.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:13 gibba011 bash[31774]: audit 2021-03-10T14:06:11.786947+0000 mgr.gibba032.whopeo (mgr.24625) 760 : audit [DBG] from='client.54550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:14 gibba032 bash[24004]: cluster 2021-03-10T14:06:12.978110+0000 mgr.gibba032.whopeo (mgr.24625) 761 : cluster [DBG] pgmap v413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:14 gibba011 bash[30791]: cluster 2021-03-10T14:06:12.978110+0000 mgr.gibba032.whopeo (mgr.24625) 761 : cluster [DBG] pgmap v413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:14 gibba011 bash[31774]: cluster 2021-03-10T14:06:12.978110+0000 mgr.gibba032.whopeo (mgr.24625) 761 : cluster [DBG] pgmap v413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:16 gibba032 bash[24004]: cluster 2021-03-10T14:06:14.979477+0000 mgr.gibba032.whopeo (mgr.24625) 762 : cluster [DBG] pgmap v414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:16 gibba011 bash[30791]: cluster 2021-03-10T14:06:14.979477+0000 mgr.gibba032.whopeo (mgr.24625) 762 : cluster [DBG] pgmap v414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:16 gibba011 bash[31774]: cluster 2021-03-10T14:06:14.979477+0000 mgr.gibba032.whopeo (mgr.24625) 762 : cluster [DBG] pgmap v414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:18 gibba032 bash[24004]: cluster 2021-03-10T14:06:16.981279+0000 mgr.gibba032.whopeo (mgr.24625) 763 : cluster [DBG] pgmap v415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:18 gibba011 bash[30791]: cluster 2021-03-10T14:06:16.981279+0000 mgr.gibba032.whopeo (mgr.24625) 763 : cluster [DBG] pgmap v415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:18 gibba011 bash[31774]: cluster 2021-03-10T14:06:16.981279+0000 mgr.gibba032.whopeo (mgr.24625) 763 : cluster [DBG] pgmap v415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:20 gibba032 bash[24004]: cluster 2021-03-10T14:06:18.982838+0000 mgr.gibba032.whopeo (mgr.24625) 764 : cluster [DBG] pgmap v416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:20 gibba011 bash[30791]: cluster 2021-03-10T14:06:18.982838+0000 mgr.gibba032.whopeo (mgr.24625) 764 : cluster [DBG] pgmap v416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:20 gibba011 bash[31774]: cluster 2021-03-10T14:06:18.982838+0000 mgr.gibba032.whopeo (mgr.24625) 764 : cluster [DBG] pgmap v416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:22 gibba032 bash[24004]: cluster 2021-03-10T14:06:20.984234+0000 mgr.gibba032.whopeo (mgr.24625) 765 : cluster [DBG] pgmap v417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:22 gibba011 bash[30791]: cluster 2021-03-10T14:06:20.984234+0000 mgr.gibba032.whopeo (mgr.24625) 765 : cluster [DBG] pgmap v417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:22 gibba011 bash[31774]: cluster 2021-03-10T14:06:20.984234+0000 mgr.gibba032.whopeo (mgr.24625) 765 : cluster [DBG] pgmap v417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:24 gibba032 bash[24004]: cluster 2021-03-10T14:06:22.986162+0000 mgr.gibba032.whopeo (mgr.24625) 766 : cluster [DBG] pgmap v418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:24 gibba032 bash[24004]: cephadm 2021-03-10T14:06:23.323878+0000 mgr.gibba032.whopeo (mgr.24625) 767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:24 gibba011 bash[30791]: cluster 2021-03-10T14:06:22.986162+0000 mgr.gibba032.whopeo (mgr.24625) 766 : cluster [DBG] pgmap v418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:24 gibba011 bash[30791]: cephadm 2021-03-10T14:06:23.323878+0000 mgr.gibba032.whopeo (mgr.24625) 767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:24 gibba011 bash[31774]: cluster 2021-03-10T14:06:22.986162+0000 mgr.gibba032.whopeo (mgr.24625) 766 : cluster [DBG] pgmap v418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:24 gibba011 bash[31774]: cephadm 2021-03-10T14:06:23.323878+0000 mgr.gibba032.whopeo (mgr.24625) 767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:26 gibba032 bash[24004]: cluster 2021-03-10T14:06:24.987517+0000 mgr.gibba032.whopeo (mgr.24625) 768 : cluster [DBG] pgmap v419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:26 gibba011 bash[30791]: cluster 2021-03-10T14:06:24.987517+0000 mgr.gibba032.whopeo (mgr.24625) 768 : cluster [DBG] pgmap v419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:26 gibba011 bash[31774]: cluster 2021-03-10T14:06:24.987517+0000 mgr.gibba032.whopeo (mgr.24625) 768 : cluster [DBG] pgmap v419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:28 gibba032 bash[24004]: cluster 2021-03-10T14:06:26.989264+0000 mgr.gibba032.whopeo (mgr.24625) 769 : cluster [DBG] pgmap v420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:28 gibba011 bash[30791]: cluster 2021-03-10T14:06:26.989264+0000 mgr.gibba032.whopeo (mgr.24625) 769 : cluster [DBG] pgmap v420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:28 gibba011 bash[31774]: cluster 2021-03-10T14:06:26.989264+0000 mgr.gibba032.whopeo (mgr.24625) 769 : cluster [DBG] pgmap v420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:30 gibba032 bash[24004]: cluster 2021-03-10T14:06:28.990757+0000 mgr.gibba032.whopeo (mgr.24625) 770 : cluster [DBG] pgmap v421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:30 gibba011 bash[30791]: cluster 2021-03-10T14:06:28.990757+0000 mgr.gibba032.whopeo (mgr.24625) 770 : cluster [DBG] pgmap v421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:30 gibba011 bash[31774]: cluster 2021-03-10T14:06:28.990757+0000 mgr.gibba032.whopeo (mgr.24625) 770 : cluster [DBG] pgmap v421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:32 gibba032 bash[24004]: cluster 2021-03-10T14:06:30.992318+0000 mgr.gibba032.whopeo (mgr.24625) 771 : cluster [DBG] pgmap v422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:32 gibba011 bash[30791]: cluster 2021-03-10T14:06:30.992318+0000 mgr.gibba032.whopeo (mgr.24625) 771 : cluster [DBG] pgmap v422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:32 gibba011 bash[31774]: cluster 2021-03-10T14:06:30.992318+0000 mgr.gibba032.whopeo (mgr.24625) 771 : cluster [DBG] pgmap v422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:34 gibba032 bash[24004]: cluster 2021-03-10T14:06:32.994260+0000 mgr.gibba032.whopeo (mgr.24625) 772 : cluster [DBG] pgmap v423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:34 gibba011 bash[30791]: cluster 2021-03-10T14:06:32.994260+0000 mgr.gibba032.whopeo (mgr.24625) 772 : cluster [DBG] pgmap v423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:34 gibba011 bash[31774]: cluster 2021-03-10T14:06:32.994260+0000 mgr.gibba032.whopeo (mgr.24625) 772 : cluster [DBG] pgmap v423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:36 gibba032 bash[24004]: cluster 2021-03-10T14:06:34.995603+0000 mgr.gibba032.whopeo (mgr.24625) 773 : cluster [DBG] pgmap v424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:36 gibba011 bash[30791]: cluster 2021-03-10T14:06:34.995603+0000 mgr.gibba032.whopeo (mgr.24625) 773 : cluster [DBG] pgmap v424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:36 gibba011 bash[31774]: cluster 2021-03-10T14:06:34.995603+0000 mgr.gibba032.whopeo (mgr.24625) 773 : cluster [DBG] pgmap v424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:38 gibba032 bash[24004]: cluster 2021-03-10T14:06:36.997367+0000 mgr.gibba032.whopeo (mgr.24625) 774 : cluster [DBG] pgmap v425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:38 gibba011 bash[30791]: cluster 2021-03-10T14:06:36.997367+0000 mgr.gibba032.whopeo (mgr.24625) 774 : cluster [DBG] pgmap v425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:38 gibba011 bash[31774]: cluster 2021-03-10T14:06:36.997367+0000 mgr.gibba032.whopeo (mgr.24625) 774 : cluster [DBG] pgmap v425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:39 gibba032 bash[24004]: cephadm 2021-03-10T14:06:38.339851+0000 mgr.gibba032.whopeo (mgr.24625) 775 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:39.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:39 gibba011 bash[30791]: cephadm 2021-03-10T14:06:38.339851+0000 mgr.gibba032.whopeo (mgr.24625) 775 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:39.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:39 gibba011 bash[31774]: cephadm 2021-03-10T14:06:38.339851+0000 mgr.gibba032.whopeo (mgr.24625) 775 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:40 gibba032 bash[24004]: cluster 2021-03-10T14:06:38.998957+0000 mgr.gibba032.whopeo (mgr.24625) 776 : cluster [DBG] pgmap v426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:40 gibba011 bash[30791]: cluster 2021-03-10T14:06:38.998957+0000 mgr.gibba032.whopeo (mgr.24625) 776 : cluster [DBG] pgmap v426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:40 gibba011 bash[31774]: cluster 2021-03-10T14:06:38.998957+0000 mgr.gibba032.whopeo (mgr.24625) 776 : cluster [DBG] pgmap v426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:42.298 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:06:42.527 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:06:42.528 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (13m) 10m ago 18m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:06:42.528 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (18m) 8m ago 18m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:06:42.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (13m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:06:42.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (13m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:06:42.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (13m) 10m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:06:42.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (12m) 10m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:06:42.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (12m) 8m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:06:42.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (12m) 10m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:06:42.529 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (18m) 10m ago 18m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:06:42.529 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (18m) 8m ago 18m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:06:42.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (11m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:06:42.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (11m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:06:42.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (10m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:06:42.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (10m) 10m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:06:42.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (9m) 8m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:06:42.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (9m) 8m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:06:42.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (9m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:06:42.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (8m) 8m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:06:42.531 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (18m) 8m ago 18m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:06:42.532 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 16m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:06:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:42 gibba032 bash[24004]: cluster 2021-03-10T14:06:41.000312+0000 mgr.gibba032.whopeo (mgr.24625) 777 : cluster [DBG] pgmap v427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:42.774 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:06:42.775 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:06:42.775 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:06:42.775 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:42.775 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:06:42.776 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:06:42.776 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:42.776 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:06:42.776 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:06:42.776 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:06:42.777 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:06:42.777 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:06:42.777 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:06:42.777 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:06:42.777 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:06:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:42 gibba011 bash[30791]: cluster 2021-03-10T14:06:41.000312+0000 mgr.gibba032.whopeo (mgr.24625) 777 : cluster [DBG] pgmap v427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:42 gibba011 bash[31774]: cluster 2021-03-10T14:06:41.000312+0000 mgr.gibba032.whopeo (mgr.24625) 777 : cluster [DBG] pgmap v427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:43.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:43 gibba032 bash[24004]: audit 2021-03-10T14:06:42.287086+0000 mgr.gibba032.whopeo (mgr.24625) 778 : audit [DBG] from='client.54562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:43 gibba032 bash[24004]: audit 2021-03-10T14:06:42.514749+0000 mgr.gibba032.whopeo (mgr.24625) 779 : audit [DBG] from='client.44583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:43 gibba032 bash[24004]: audit 2021-03-10T14:06:42.775111+0000 mon.a (mon.0) 789 : audit [DBG] from='client.? 172.21.2.111:0/1262381007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:43.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[30791]: audit 2021-03-10T14:06:42.287086+0000 mgr.gibba032.whopeo (mgr.24625) 778 : audit [DBG] from='client.54562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[30791]: audit 2021-03-10T14:06:42.514749+0000 mgr.gibba032.whopeo (mgr.24625) 779 : audit [DBG] from='client.44583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[30791]: audit 2021-03-10T14:06:42.775111+0000 mon.a (mon.0) 789 : audit [DBG] from='client.? 172.21.2.111:0/1262381007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:43.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[31774]: audit 2021-03-10T14:06:42.287086+0000 mgr.gibba032.whopeo (mgr.24625) 778 : audit [DBG] from='client.54562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[31774]: audit 2021-03-10T14:06:42.514749+0000 mgr.gibba032.whopeo (mgr.24625) 779 : audit [DBG] from='client.44583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:06:43.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:43 gibba011 bash[31774]: audit 2021-03-10T14:06:42.775111+0000 mon.a (mon.0) 789 : audit [DBG] from='client.? 172.21.2.111:0/1262381007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:06:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:44 gibba032 bash[24004]: cluster 2021-03-10T14:06:43.001902+0000 mgr.gibba032.whopeo (mgr.24625) 780 : cluster [DBG] pgmap v428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:44 gibba011 bash[30791]: cluster 2021-03-10T14:06:43.001902+0000 mgr.gibba032.whopeo (mgr.24625) 780 : cluster [DBG] pgmap v428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:44 gibba011 bash[31774]: cluster 2021-03-10T14:06:43.001902+0000 mgr.gibba032.whopeo (mgr.24625) 780 : cluster [DBG] pgmap v428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:46 gibba032 bash[24004]: cluster 2021-03-10T14:06:45.003026+0000 mgr.gibba032.whopeo (mgr.24625) 781 : cluster [DBG] pgmap v429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:46 gibba011 bash[30791]: cluster 2021-03-10T14:06:45.003026+0000 mgr.gibba032.whopeo (mgr.24625) 781 : cluster [DBG] pgmap v429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:46 gibba011 bash[31774]: cluster 2021-03-10T14:06:45.003026+0000 mgr.gibba032.whopeo (mgr.24625) 781 : cluster [DBG] pgmap v429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:48 gibba011 bash[30791]: cluster 2021-03-10T14:06:47.004792+0000 mgr.gibba032.whopeo (mgr.24625) 782 : cluster [DBG] pgmap v430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:48 gibba011 bash[31774]: cluster 2021-03-10T14:06:47.004792+0000 mgr.gibba032.whopeo (mgr.24625) 782 : cluster [DBG] pgmap v430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:48 gibba032 bash[24004]: cluster 2021-03-10T14:06:47.004792+0000 mgr.gibba032.whopeo (mgr.24625) 782 : cluster [DBG] pgmap v430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:50 gibba011 bash[30791]: cluster 2021-03-10T14:06:49.006217+0000 mgr.gibba032.whopeo (mgr.24625) 783 : cluster [DBG] pgmap v431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:50 gibba011 bash[31774]: cluster 2021-03-10T14:06:49.006217+0000 mgr.gibba032.whopeo (mgr.24625) 783 : cluster [DBG] pgmap v431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:50 gibba032 bash[24004]: cluster 2021-03-10T14:06:49.006217+0000 mgr.gibba032.whopeo (mgr.24625) 783 : cluster [DBG] pgmap v431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:52 gibba011 bash[30791]: cluster 2021-03-10T14:06:51.007524+0000 mgr.gibba032.whopeo (mgr.24625) 784 : cluster [DBG] pgmap v432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:52 gibba011 bash[31774]: cluster 2021-03-10T14:06:51.007524+0000 mgr.gibba032.whopeo (mgr.24625) 784 : cluster [DBG] pgmap v432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:52 gibba032 bash[24004]: cluster 2021-03-10T14:06:51.007524+0000 mgr.gibba032.whopeo (mgr.24625) 784 : cluster [DBG] pgmap v432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:54 gibba011 bash[30791]: cluster 2021-03-10T14:06:53.009191+0000 mgr.gibba032.whopeo (mgr.24625) 785 : cluster [DBG] pgmap v433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:54 gibba011 bash[30791]: cephadm 2021-03-10T14:06:53.355892+0000 mgr.gibba032.whopeo (mgr.24625) 786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:54 gibba011 bash[31774]: cluster 2021-03-10T14:06:53.009191+0000 mgr.gibba032.whopeo (mgr.24625) 785 : cluster [DBG] pgmap v433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:54 gibba011 bash[31774]: cephadm 2021-03-10T14:06:53.355892+0000 mgr.gibba032.whopeo (mgr.24625) 786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:54 gibba032 bash[24004]: cluster 2021-03-10T14:06:53.009191+0000 mgr.gibba032.whopeo (mgr.24625) 785 : cluster [DBG] pgmap v433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:54 gibba032 bash[24004]: cephadm 2021-03-10T14:06:53.355892+0000 mgr.gibba032.whopeo (mgr.24625) 786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:06:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: cluster 2021-03-10T14:06:55.010389+0000 mgr.gibba032.whopeo (mgr.24625) 787 : cluster [DBG] pgmap v434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:55.979638+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:06:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:55.980415+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:06:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:55.981224+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:06:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:55.981936+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:06:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:56.085523+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:06:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[30791]: audit 2021-03-10T14:06:56.086202+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:06:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: cluster 2021-03-10T14:06:55.010389+0000 mgr.gibba032.whopeo (mgr.24625) 787 : cluster [DBG] pgmap v434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:55.979638+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:06:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:55.980415+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:06:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:55.981224+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:06:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:55.981936+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:06:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:56.085523+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:06:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:56 gibba011 bash[31774]: audit 2021-03-10T14:06:56.086202+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:06:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: cluster 2021-03-10T14:06:55.010389+0000 mgr.gibba032.whopeo (mgr.24625) 787 : cluster [DBG] pgmap v434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:55.979638+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:06:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:55.980415+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:06:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:55.981224+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:06:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:55.981936+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:06:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:56.085523+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:06:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:56 gibba032 bash[24004]: audit 2021-03-10T14:06:56.086202+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:06:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:06:58 gibba011 bash[30791]: cluster 2021-03-10T14:06:57.012058+0000 mgr.gibba032.whopeo (mgr.24625) 788 : cluster [DBG] pgmap v435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:06:58 gibba011 bash[31774]: cluster 2021-03-10T14:06:57.012058+0000 mgr.gibba032.whopeo (mgr.24625) 788 : cluster [DBG] pgmap v435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:06:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:06:58 gibba032 bash[24004]: cluster 2021-03-10T14:06:57.012058+0000 mgr.gibba032.whopeo (mgr.24625) 788 : cluster [DBG] pgmap v435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:00 gibba011 bash[30791]: cluster 2021-03-10T14:06:59.013366+0000 mgr.gibba032.whopeo (mgr.24625) 789 : cluster [DBG] pgmap v436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:00 gibba011 bash[31774]: cluster 2021-03-10T14:06:59.013366+0000 mgr.gibba032.whopeo (mgr.24625) 789 : cluster [DBG] pgmap v436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:00 gibba032 bash[24004]: cluster 2021-03-10T14:06:59.013366+0000 mgr.gibba032.whopeo (mgr.24625) 789 : cluster [DBG] pgmap v436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:02 gibba011 bash[30791]: cluster 2021-03-10T14:07:01.014681+0000 mgr.gibba032.whopeo (mgr.24625) 790 : cluster [DBG] pgmap v437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:02 gibba011 bash[31774]: cluster 2021-03-10T14:07:01.014681+0000 mgr.gibba032.whopeo (mgr.24625) 790 : cluster [DBG] pgmap v437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:02 gibba032 bash[24004]: cluster 2021-03-10T14:07:01.014681+0000 mgr.gibba032.whopeo (mgr.24625) 790 : cluster [DBG] pgmap v437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:04 gibba011 bash[30791]: cluster 2021-03-10T14:07:03.016635+0000 mgr.gibba032.whopeo (mgr.24625) 791 : cluster [DBG] pgmap v438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:04 gibba011 bash[31774]: cluster 2021-03-10T14:07:03.016635+0000 mgr.gibba032.whopeo (mgr.24625) 791 : cluster [DBG] pgmap v438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:04 gibba032 bash[24004]: cluster 2021-03-10T14:07:03.016635+0000 mgr.gibba032.whopeo (mgr.24625) 791 : cluster [DBG] pgmap v438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:06 gibba032 bash[24004]: cluster 2021-03-10T14:07:05.017979+0000 mgr.gibba032.whopeo (mgr.24625) 792 : cluster [DBG] pgmap v439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:06 gibba011 bash[30791]: cluster 2021-03-10T14:07:05.017979+0000 mgr.gibba032.whopeo (mgr.24625) 792 : cluster [DBG] pgmap v439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:06 gibba011 bash[31774]: cluster 2021-03-10T14:07:05.017979+0000 mgr.gibba032.whopeo (mgr.24625) 792 : cluster [DBG] pgmap v439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:07 gibba032 bash[24004]: cluster 2021-03-10T14:07:07.019672+0000 mgr.gibba032.whopeo (mgr.24625) 793 : cluster [DBG] pgmap v440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:08.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:07 gibba011 bash[30791]: cluster 2021-03-10T14:07:07.019672+0000 mgr.gibba032.whopeo (mgr.24625) 793 : cluster [DBG] pgmap v440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:08.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:07 gibba011 bash[31774]: cluster 2021-03-10T14:07:07.019672+0000 mgr.gibba032.whopeo (mgr.24625) 793 : cluster [DBG] pgmap v440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:08 gibba032 bash[24004]: cephadm 2021-03-10T14:07:08.373317+0000 mgr.gibba032.whopeo (mgr.24625) 794 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:07:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:08 gibba032 bash[24004]: audit 2021-03-10T14:07:08.376109+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:07:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:08 gibba011 bash[30791]: cephadm 2021-03-10T14:07:08.373317+0000 mgr.gibba032.whopeo (mgr.24625) 794 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:07:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:08 gibba011 bash[30791]: audit 2021-03-10T14:07:08.376109+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:07:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:08 gibba011 bash[31774]: cephadm 2021-03-10T14:07:08.373317+0000 mgr.gibba032.whopeo (mgr.24625) 794 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:07:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:08 gibba011 bash[31774]: audit 2021-03-10T14:07:08.376109+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:07:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:09 gibba032 bash[24004]: cluster 2021-03-10T14:07:09.021079+0000 mgr.gibba032.whopeo (mgr.24625) 795 : cluster [DBG] pgmap v441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:10.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:09 gibba011 bash[30791]: cluster 2021-03-10T14:07:09.021079+0000 mgr.gibba032.whopeo (mgr.24625) 795 : cluster [DBG] pgmap v441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:10.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:09 gibba011 bash[31774]: cluster 2021-03-10T14:07:09.021079+0000 mgr.gibba032.whopeo (mgr.24625) 795 : cluster [DBG] pgmap v441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: cluster 2021-03-10T14:07:11.022400+0000 mgr.gibba032.whopeo (mgr.24625) 796 : cluster [DBG] pgmap v442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:11.887625+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.095866+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.104699+0000 mon.a (mon.0) 799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:07:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.106620+0000 mon.a (mon.0) 800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.108719+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.110541+0000 mon.a (mon.0) 802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.114063+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:12 gibba032 bash[24004]: audit 2021-03-10T14:07:12.115797+0000 mon.a (mon.0) 804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: cluster 2021-03-10T14:07:11.022400+0000 mgr.gibba032.whopeo (mgr.24625) 796 : cluster [DBG] pgmap v442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:11.887625+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.095866+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.104699+0000 mon.a (mon.0) 799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:07:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.106620+0000 mon.a (mon.0) 800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.108719+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.110541+0000 mon.a (mon.0) 802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.114063+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[30791]: audit 2021-03-10T14:07:12.115797+0000 mon.a (mon.0) 804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: cluster 2021-03-10T14:07:11.022400+0000 mgr.gibba032.whopeo (mgr.24625) 796 : cluster [DBG] pgmap v442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:11.887625+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.095866+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:07:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.104699+0000 mon.a (mon.0) 799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:07:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.106620+0000 mon.a (mon.0) 800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.108719+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.110541+0000 mon.a (mon.0) 802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.114063+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:12 gibba011 bash[31774]: audit 2021-03-10T14:07:12.115797+0000 mon.a (mon.0) 804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:13.018 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:07:13.244 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:07:13.244 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (13m) 1s ago 19m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:07:13.244 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (19m) 9m ago 19m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:07:13.245 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (14m) 9m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:07:13.245 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (14m) 9m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:07:13.245 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (13m) 1s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:07:13.245 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (13m) 1s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:07:13.245 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (12m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:07:13.246 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (13m) 1s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:07:13.246 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (19m) 1s ago 19m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:07:13.246 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (19m) 9m ago 19m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:07:13.246 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (12m) 1s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:07:13.246 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (11m) 1s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:07:13.247 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (11m) 1s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:07:13.247 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (10m) 1s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:07:13.247 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:07:13.247 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:07:13.247 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (9m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:07:13.248 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (9m) 9m ago 19m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:07:13.248 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (18m) 9m ago 19m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:07:13.248 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 1s ago 17m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:07:13.493 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:07:13.493 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:07:13.493 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:07:13.494 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:13.494 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:07:13.494 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:07:13.494 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:07:13.495 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:07:13.496 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:07:13.496 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:07:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:11.885706+0000 mgr.gibba032.whopeo (mgr.24625) 797 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:07:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.101242+0000 mgr.gibba032.whopeo (mgr.24625) 798 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:07:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.103748+0000 mgr.gibba032.whopeo (mgr.24625) 799 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:07:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.105716+0000 mgr.gibba032.whopeo (mgr.24625) 800 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:07:13.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.105758+0000 mgr.gibba032.whopeo (mgr.24625) 801 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:07:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.107637+0000 mgr.gibba032.whopeo (mgr.24625) 802 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:07:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.107681+0000 mgr.gibba032.whopeo (mgr.24625) 803 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:07:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.109449+0000 mgr.gibba032.whopeo (mgr.24625) 804 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:07:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.111004+0000 mgr.gibba032.whopeo (mgr.24625) 805 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:07:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.111046+0000 mgr.gibba032.whopeo (mgr.24625) 806 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:07:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.112606+0000 mgr.gibba032.whopeo (mgr.24625) 807 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:07:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.113038+0000 mgr.gibba032.whopeo (mgr.24625) 808 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:07:13.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.114176+0000 mgr.gibba032.whopeo (mgr.24625) 809 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:07:13.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.114221+0000 mgr.gibba032.whopeo (mgr.24625) 810 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:07:13.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:13 gibba032 bash[24004]: cephadm 2021-03-10T14:07:12.114343+0000 mgr.gibba032.whopeo (mgr.24625) 811 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:11.885706+0000 mgr.gibba032.whopeo (mgr.24625) 797 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:07:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.101242+0000 mgr.gibba032.whopeo (mgr.24625) 798 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:07:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.103748+0000 mgr.gibba032.whopeo (mgr.24625) 799 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:07:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.105716+0000 mgr.gibba032.whopeo (mgr.24625) 800 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:07:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.105758+0000 mgr.gibba032.whopeo (mgr.24625) 801 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:07:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.107637+0000 mgr.gibba032.whopeo (mgr.24625) 802 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:07:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.107681+0000 mgr.gibba032.whopeo (mgr.24625) 803 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:07:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.109449+0000 mgr.gibba032.whopeo (mgr.24625) 804 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:07:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.111004+0000 mgr.gibba032.whopeo (mgr.24625) 805 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:07:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.111046+0000 mgr.gibba032.whopeo (mgr.24625) 806 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:07:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.112606+0000 mgr.gibba032.whopeo (mgr.24625) 807 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:07:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.113038+0000 mgr.gibba032.whopeo (mgr.24625) 808 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:07:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.114176+0000 mgr.gibba032.whopeo (mgr.24625) 809 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:07:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.114221+0000 mgr.gibba032.whopeo (mgr.24625) 810 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:07:13.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[30791]: cephadm 2021-03-10T14:07:12.114343+0000 mgr.gibba032.whopeo (mgr.24625) 811 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:11.885706+0000 mgr.gibba032.whopeo (mgr.24625) 797 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:07:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.101242+0000 mgr.gibba032.whopeo (mgr.24625) 798 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:07:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.103748+0000 mgr.gibba032.whopeo (mgr.24625) 799 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:07:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.105716+0000 mgr.gibba032.whopeo (mgr.24625) 800 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:07:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.105758+0000 mgr.gibba032.whopeo (mgr.24625) 801 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:07:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.107637+0000 mgr.gibba032.whopeo (mgr.24625) 802 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:07:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.107681+0000 mgr.gibba032.whopeo (mgr.24625) 803 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:07:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.109449+0000 mgr.gibba032.whopeo (mgr.24625) 804 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:07:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.111004+0000 mgr.gibba032.whopeo (mgr.24625) 805 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.111046+0000 mgr.gibba032.whopeo (mgr.24625) 806 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.112606+0000 mgr.gibba032.whopeo (mgr.24625) 807 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.113038+0000 mgr.gibba032.whopeo (mgr.24625) 808 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.114176+0000 mgr.gibba032.whopeo (mgr.24625) 809 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.114221+0000 mgr.gibba032.whopeo (mgr.24625) 810 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:07:13.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:13 gibba011 bash[31774]: cephadm 2021-03-10T14:07:12.114343+0000 mgr.gibba032.whopeo (mgr.24625) 811 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:14.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:14 gibba032 bash[24004]: audit 2021-03-10T14:07:13.006984+0000 mgr.gibba032.whopeo (mgr.24625) 812 : audit [DBG] from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:14 gibba032 bash[24004]: cluster 2021-03-10T14:07:13.023219+0000 mgr.gibba032.whopeo (mgr.24625) 813 : cluster [DBG] pgmap v443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:14 gibba032 bash[24004]: audit 2021-03-10T14:07:13.231557+0000 mgr.gibba032.whopeo (mgr.24625) 814 : audit [DBG] from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:14 gibba032 bash[24004]: audit 2021-03-10T14:07:13.493349+0000 mon.a (mon.0) 805 : audit [DBG] from='client.? 172.21.2.111:0/4224740694' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[30791]: audit 2021-03-10T14:07:13.006984+0000 mgr.gibba032.whopeo (mgr.24625) 812 : audit [DBG] from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[30791]: cluster 2021-03-10T14:07:13.023219+0000 mgr.gibba032.whopeo (mgr.24625) 813 : cluster [DBG] pgmap v443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[30791]: audit 2021-03-10T14:07:13.231557+0000 mgr.gibba032.whopeo (mgr.24625) 814 : audit [DBG] from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[30791]: audit 2021-03-10T14:07:13.493349+0000 mon.a (mon.0) 805 : audit [DBG] from='client.? 172.21.2.111:0/4224740694' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[31774]: audit 2021-03-10T14:07:13.006984+0000 mgr.gibba032.whopeo (mgr.24625) 812 : audit [DBG] from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[31774]: cluster 2021-03-10T14:07:13.023219+0000 mgr.gibba032.whopeo (mgr.24625) 813 : cluster [DBG] pgmap v443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[31774]: audit 2021-03-10T14:07:13.231557+0000 mgr.gibba032.whopeo (mgr.24625) 814 : audit [DBG] from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:14 gibba011 bash[31774]: audit 2021-03-10T14:07:13.493349+0000 mon.a (mon.0) 805 : audit [DBG] from='client.? 172.21.2.111:0/4224740694' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:16.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:16 gibba032 bash[24004]: cluster 2021-03-10T14:07:15.024475+0000 mgr.gibba032.whopeo (mgr.24625) 815 : cluster [DBG] pgmap v444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:16 gibba011 bash[30791]: cluster 2021-03-10T14:07:15.024475+0000 mgr.gibba032.whopeo (mgr.24625) 815 : cluster [DBG] pgmap v444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:16 gibba011 bash[31774]: cluster 2021-03-10T14:07:15.024475+0000 mgr.gibba032.whopeo (mgr.24625) 815 : cluster [DBG] pgmap v444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:18 gibba032 bash[24004]: cluster 2021-03-10T14:07:17.026231+0000 mgr.gibba032.whopeo (mgr.24625) 816 : cluster [DBG] pgmap v445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:18 gibba011 bash[30791]: cluster 2021-03-10T14:07:17.026231+0000 mgr.gibba032.whopeo (mgr.24625) 816 : cluster [DBG] pgmap v445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:18 gibba011 bash[31774]: cluster 2021-03-10T14:07:17.026231+0000 mgr.gibba032.whopeo (mgr.24625) 816 : cluster [DBG] pgmap v445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:20.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:20 gibba032 bash[24004]: cluster 2021-03-10T14:07:19.027802+0000 mgr.gibba032.whopeo (mgr.24625) 817 : cluster [DBG] pgmap v446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:20 gibba011 bash[30791]: cluster 2021-03-10T14:07:19.027802+0000 mgr.gibba032.whopeo (mgr.24625) 817 : cluster [DBG] pgmap v446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:20 gibba011 bash[31774]: cluster 2021-03-10T14:07:19.027802+0000 mgr.gibba032.whopeo (mgr.24625) 817 : cluster [DBG] pgmap v446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:22 gibba032 bash[24004]: cluster 2021-03-10T14:07:21.029180+0000 mgr.gibba032.whopeo (mgr.24625) 818 : cluster [DBG] pgmap v447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:22 gibba011 bash[30791]: cluster 2021-03-10T14:07:21.029180+0000 mgr.gibba032.whopeo (mgr.24625) 818 : cluster [DBG] pgmap v447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:22 gibba011 bash[31774]: cluster 2021-03-10T14:07:21.029180+0000 mgr.gibba032.whopeo (mgr.24625) 818 : cluster [DBG] pgmap v447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:24 gibba032 bash[24004]: cluster 2021-03-10T14:07:23.031065+0000 mgr.gibba032.whopeo (mgr.24625) 819 : cluster [DBG] pgmap v448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:24 gibba011 bash[30791]: cluster 2021-03-10T14:07:23.031065+0000 mgr.gibba032.whopeo (mgr.24625) 819 : cluster [DBG] pgmap v448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:24 gibba011 bash[31774]: cluster 2021-03-10T14:07:23.031065+0000 mgr.gibba032.whopeo (mgr.24625) 819 : cluster [DBG] pgmap v448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:26 gibba011 bash[30791]: cluster 2021-03-10T14:07:25.032462+0000 mgr.gibba032.whopeo (mgr.24625) 820 : cluster [DBG] pgmap v449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:26 gibba011 bash[31774]: cluster 2021-03-10T14:07:25.032462+0000 mgr.gibba032.whopeo (mgr.24625) 820 : cluster [DBG] pgmap v449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:26 gibba032 bash[24004]: cluster 2021-03-10T14:07:25.032462+0000 mgr.gibba032.whopeo (mgr.24625) 820 : cluster [DBG] pgmap v449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:28 gibba011 bash[30791]: cluster 2021-03-10T14:07:27.034208+0000 mgr.gibba032.whopeo (mgr.24625) 821 : cluster [DBG] pgmap v450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:28 gibba011 bash[30791]: cephadm 2021-03-10T14:07:27.130340+0000 mgr.gibba032.whopeo (mgr.24625) 822 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:28 gibba011 bash[31774]: cluster 2021-03-10T14:07:27.034208+0000 mgr.gibba032.whopeo (mgr.24625) 821 : cluster [DBG] pgmap v450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:28 gibba011 bash[31774]: cephadm 2021-03-10T14:07:27.130340+0000 mgr.gibba032.whopeo (mgr.24625) 822 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:28 gibba032 bash[24004]: cluster 2021-03-10T14:07:27.034208+0000 mgr.gibba032.whopeo (mgr.24625) 821 : cluster [DBG] pgmap v450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:28 gibba032 bash[24004]: cephadm 2021-03-10T14:07:27.130340+0000 mgr.gibba032.whopeo (mgr.24625) 822 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:30 gibba011 bash[30791]: cluster 2021-03-10T14:07:29.035794+0000 mgr.gibba032.whopeo (mgr.24625) 823 : cluster [DBG] pgmap v451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:30 gibba011 bash[31774]: cluster 2021-03-10T14:07:29.035794+0000 mgr.gibba032.whopeo (mgr.24625) 823 : cluster [DBG] pgmap v451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:30 gibba032 bash[24004]: cluster 2021-03-10T14:07:29.035794+0000 mgr.gibba032.whopeo (mgr.24625) 823 : cluster [DBG] pgmap v451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:32 gibba011 bash[30791]: cluster 2021-03-10T14:07:31.037156+0000 mgr.gibba032.whopeo (mgr.24625) 824 : cluster [DBG] pgmap v452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:32 gibba011 bash[31774]: cluster 2021-03-10T14:07:31.037156+0000 mgr.gibba032.whopeo (mgr.24625) 824 : cluster [DBG] pgmap v452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:32 gibba032 bash[24004]: cluster 2021-03-10T14:07:31.037156+0000 mgr.gibba032.whopeo (mgr.24625) 824 : cluster [DBG] pgmap v452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:34 gibba011 bash[30791]: cluster 2021-03-10T14:07:33.037890+0000 mgr.gibba032.whopeo (mgr.24625) 825 : cluster [DBG] pgmap v453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:34 gibba011 bash[31774]: cluster 2021-03-10T14:07:33.037890+0000 mgr.gibba032.whopeo (mgr.24625) 825 : cluster [DBG] pgmap v453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:34 gibba032 bash[24004]: cluster 2021-03-10T14:07:33.037890+0000 mgr.gibba032.whopeo (mgr.24625) 825 : cluster [DBG] pgmap v453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:36 gibba011 bash[30791]: cluster 2021-03-10T14:07:35.039053+0000 mgr.gibba032.whopeo (mgr.24625) 826 : cluster [DBG] pgmap v454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:36 gibba011 bash[31774]: cluster 2021-03-10T14:07:35.039053+0000 mgr.gibba032.whopeo (mgr.24625) 826 : cluster [DBG] pgmap v454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:36 gibba032 bash[24004]: cluster 2021-03-10T14:07:35.039053+0000 mgr.gibba032.whopeo (mgr.24625) 826 : cluster [DBG] pgmap v454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:38 gibba011 bash[30791]: cluster 2021-03-10T14:07:37.040790+0000 mgr.gibba032.whopeo (mgr.24625) 827 : cluster [DBG] pgmap v455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:38 gibba011 bash[31774]: cluster 2021-03-10T14:07:37.040790+0000 mgr.gibba032.whopeo (mgr.24625) 827 : cluster [DBG] pgmap v455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:38 gibba032 bash[24004]: cluster 2021-03-10T14:07:37.040790+0000 mgr.gibba032.whopeo (mgr.24625) 827 : cluster [DBG] pgmap v455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:40 gibba011 bash[30791]: cluster 2021-03-10T14:07:39.042353+0000 mgr.gibba032.whopeo (mgr.24625) 828 : cluster [DBG] pgmap v456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:40 gibba011 bash[31774]: cluster 2021-03-10T14:07:39.042353+0000 mgr.gibba032.whopeo (mgr.24625) 828 : cluster [DBG] pgmap v456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:40 gibba032 bash[24004]: cluster 2021-03-10T14:07:39.042353+0000 mgr.gibba032.whopeo (mgr.24625) 828 : cluster [DBG] pgmap v456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:42 gibba032 bash[24004]: cluster 2021-03-10T14:07:41.043763+0000 mgr.gibba032.whopeo (mgr.24625) 829 : cluster [DBG] pgmap v457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:42 gibba011 bash[30791]: cluster 2021-03-10T14:07:41.043763+0000 mgr.gibba032.whopeo (mgr.24625) 829 : cluster [DBG] pgmap v457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:42.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:42 gibba011 bash[31774]: cluster 2021-03-10T14:07:41.043763+0000 mgr.gibba032.whopeo (mgr.24625) 829 : cluster [DBG] pgmap v457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:43.736 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:07:43.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:43 gibba032 bash[24004]: cephadm 2021-03-10T14:07:42.146265+0000 mgr.gibba032.whopeo (mgr.24625) 830 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:43.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:43 gibba011 bash[30791]: cephadm 2021-03-10T14:07:42.146265+0000 mgr.gibba032.whopeo (mgr.24625) 830 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:43.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:43 gibba011 bash[31774]: cephadm 2021-03-10T14:07:42.146265+0000 mgr.gibba032.whopeo (mgr.24625) 830 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:43.964 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:07:43.964 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (14m) 32s ago 19m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:07:43.965 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (19m) 9m ago 19m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:07:43.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (14m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:07:43.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (15m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:07:43.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (14m) 32s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:07:43.965 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (13m) 32s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:07:43.966 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (13m) 9m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:07:43.966 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (13m) 32s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:07:43.966 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (19m) 32s ago 19m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:07:43.966 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (19m) 9m ago 19m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:07:43.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (12m) 32s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:07:43.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (12m) 32s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:07:43.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (11m) 32s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:07:43.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (11m) 32s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:07:43.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (10m) 9m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:07:43.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (10m) 9m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:07:43.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (10m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:07:43.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (9m) 9m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:07:43.968 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (19m) 9m ago 19m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:07:43.968 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 32s ago 17m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:07:44.215 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:07:44.215 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:07:44.216 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:07:44.216 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:44.216 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:07:44.216 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:07:44.216 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:44.217 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:07:44.217 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:07:44.217 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:07:44.217 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:07:44.217 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:07:44.218 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:07:44.218 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:07:44.218 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:07:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:44 gibba032 bash[24004]: cluster 2021-03-10T14:07:43.044398+0000 mgr.gibba032.whopeo (mgr.24625) 831 : cluster [DBG] pgmap v458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:44 gibba032 bash[24004]: audit 2021-03-10T14:07:44.215694+0000 mon.a (mon.0) 806 : audit [DBG] from='client.? 172.21.2.111:0/2422143268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:44.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:44 gibba011 bash[30791]: cluster 2021-03-10T14:07:43.044398+0000 mgr.gibba032.whopeo (mgr.24625) 831 : cluster [DBG] pgmap v458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:44 gibba011 bash[30791]: audit 2021-03-10T14:07:44.215694+0000 mon.a (mon.0) 806 : audit [DBG] from='client.? 172.21.2.111:0/2422143268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:44 gibba011 bash[31774]: cluster 2021-03-10T14:07:43.044398+0000 mgr.gibba032.whopeo (mgr.24625) 831 : cluster [DBG] pgmap v458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:44 gibba011 bash[31774]: audit 2021-03-10T14:07:44.215694+0000 mon.a (mon.0) 806 : audit [DBG] from='client.? 172.21.2.111:0/2422143268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:07:45.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:45 gibba032 bash[24004]: audit 2021-03-10T14:07:43.725031+0000 mgr.gibba032.whopeo (mgr.24625) 832 : audit [DBG] from='client.44613 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:45.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:45 gibba032 bash[24004]: audit 2021-03-10T14:07:43.950370+0000 mgr.gibba032.whopeo (mgr.24625) 833 : audit [DBG] from='client.54604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:45.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:45 gibba011 bash[30791]: audit 2021-03-10T14:07:43.725031+0000 mgr.gibba032.whopeo (mgr.24625) 832 : audit [DBG] from='client.44613 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:45.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:45 gibba011 bash[30791]: audit 2021-03-10T14:07:43.950370+0000 mgr.gibba032.whopeo (mgr.24625) 833 : audit [DBG] from='client.54604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:45.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:45 gibba011 bash[31774]: audit 2021-03-10T14:07:43.725031+0000 mgr.gibba032.whopeo (mgr.24625) 832 : audit [DBG] from='client.44613 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:45.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:45 gibba011 bash[31774]: audit 2021-03-10T14:07:43.950370+0000 mgr.gibba032.whopeo (mgr.24625) 833 : audit [DBG] from='client.54604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:07:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:46 gibba032 bash[24004]: cluster 2021-03-10T14:07:45.045652+0000 mgr.gibba032.whopeo (mgr.24625) 834 : cluster [DBG] pgmap v459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:46.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:46 gibba011 bash[30791]: cluster 2021-03-10T14:07:45.045652+0000 mgr.gibba032.whopeo (mgr.24625) 834 : cluster [DBG] pgmap v459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:46 gibba011 bash[31774]: cluster 2021-03-10T14:07:45.045652+0000 mgr.gibba032.whopeo (mgr.24625) 834 : cluster [DBG] pgmap v459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:48 gibba032 bash[24004]: cluster 2021-03-10T14:07:47.047390+0000 mgr.gibba032.whopeo (mgr.24625) 835 : cluster [DBG] pgmap v460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:48 gibba011 bash[30791]: cluster 2021-03-10T14:07:47.047390+0000 mgr.gibba032.whopeo (mgr.24625) 835 : cluster [DBG] pgmap v460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:48 gibba011 bash[31774]: cluster 2021-03-10T14:07:47.047390+0000 mgr.gibba032.whopeo (mgr.24625) 835 : cluster [DBG] pgmap v460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:50 gibba032 bash[24004]: cluster 2021-03-10T14:07:49.048979+0000 mgr.gibba032.whopeo (mgr.24625) 836 : cluster [DBG] pgmap v461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:50 gibba011 bash[30791]: cluster 2021-03-10T14:07:49.048979+0000 mgr.gibba032.whopeo (mgr.24625) 836 : cluster [DBG] pgmap v461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:50 gibba011 bash[31774]: cluster 2021-03-10T14:07:49.048979+0000 mgr.gibba032.whopeo (mgr.24625) 836 : cluster [DBG] pgmap v461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:52 gibba032 bash[24004]: cluster 2021-03-10T14:07:51.050401+0000 mgr.gibba032.whopeo (mgr.24625) 837 : cluster [DBG] pgmap v462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:52 gibba011 bash[30791]: cluster 2021-03-10T14:07:51.050401+0000 mgr.gibba032.whopeo (mgr.24625) 837 : cluster [DBG] pgmap v462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:52 gibba011 bash[31774]: cluster 2021-03-10T14:07:51.050401+0000 mgr.gibba032.whopeo (mgr.24625) 837 : cluster [DBG] pgmap v462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:54 gibba032 bash[24004]: cluster 2021-03-10T14:07:53.051426+0000 mgr.gibba032.whopeo (mgr.24625) 838 : cluster [DBG] pgmap v463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:54.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:54 gibba011 bash[30791]: cluster 2021-03-10T14:07:53.051426+0000 mgr.gibba032.whopeo (mgr.24625) 838 : cluster [DBG] pgmap v463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:54 gibba011 bash[31774]: cluster 2021-03-10T14:07:53.051426+0000 mgr.gibba032.whopeo (mgr.24625) 838 : cluster [DBG] pgmap v463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: cluster 2021-03-10T14:07:55.052777+0000 mgr.gibba032.whopeo (mgr.24625) 839 : cluster [DBG] pgmap v464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:55.989189+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:07:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:55.990341+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:55.991449+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:07:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:55.992448+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:56.091262+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:07:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:56 gibba032 bash[24004]: audit 2021-03-10T14:07:56.091984+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:07:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: cluster 2021-03-10T14:07:55.052777+0000 mgr.gibba032.whopeo (mgr.24625) 839 : cluster [DBG] pgmap v464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:55.989189+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:07:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:55.990341+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:55.991449+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:07:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:55.992448+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:56.091262+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:07:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[30791]: audit 2021-03-10T14:07:56.091984+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:07:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: cluster 2021-03-10T14:07:55.052777+0000 mgr.gibba032.whopeo (mgr.24625) 839 : cluster [DBG] pgmap v464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:55.989189+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:07:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:55.990341+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:55.991449+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:07:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:55.992448+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:07:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:56.091262+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:07:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:56 gibba011 bash[31774]: audit 2021-03-10T14:07:56.091984+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:07:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:58 gibba032 bash[24004]: cluster 2021-03-10T14:07:57.054387+0000 mgr.gibba032.whopeo (mgr.24625) 840 : cluster [DBG] pgmap v465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:07:58 gibba032 bash[24004]: cephadm 2021-03-10T14:07:57.162308+0000 mgr.gibba032.whopeo (mgr.24625) 841 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:58 gibba011 bash[30791]: cluster 2021-03-10T14:07:57.054387+0000 mgr.gibba032.whopeo (mgr.24625) 840 : cluster [DBG] pgmap v465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:07:58 gibba011 bash[30791]: cephadm 2021-03-10T14:07:57.162308+0000 mgr.gibba032.whopeo (mgr.24625) 841 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:07:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:58 gibba011 bash[31774]: cluster 2021-03-10T14:07:57.054387+0000 mgr.gibba032.whopeo (mgr.24625) 840 : cluster [DBG] pgmap v465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:07:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:07:58 gibba011 bash[31774]: cephadm 2021-03-10T14:07:57.162308+0000 mgr.gibba032.whopeo (mgr.24625) 841 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:00 gibba032 bash[24004]: cluster 2021-03-10T14:07:59.055986+0000 mgr.gibba032.whopeo (mgr.24625) 842 : cluster [DBG] pgmap v466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:00 gibba011 bash[30791]: cluster 2021-03-10T14:07:59.055986+0000 mgr.gibba032.whopeo (mgr.24625) 842 : cluster [DBG] pgmap v466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:00 gibba011 bash[31774]: cluster 2021-03-10T14:07:59.055986+0000 mgr.gibba032.whopeo (mgr.24625) 842 : cluster [DBG] pgmap v466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:02 gibba032 bash[24004]: cluster 2021-03-10T14:08:01.057476+0000 mgr.gibba032.whopeo (mgr.24625) 843 : cluster [DBG] pgmap v467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:02 gibba011 bash[30791]: cluster 2021-03-10T14:08:01.057476+0000 mgr.gibba032.whopeo (mgr.24625) 843 : cluster [DBG] pgmap v467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:02 gibba011 bash[31774]: cluster 2021-03-10T14:08:01.057476+0000 mgr.gibba032.whopeo (mgr.24625) 843 : cluster [DBG] pgmap v467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:04 gibba032 bash[24004]: cluster 2021-03-10T14:08:03.058280+0000 mgr.gibba032.whopeo (mgr.24625) 844 : cluster [DBG] pgmap v468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:04 gibba011 bash[30791]: cluster 2021-03-10T14:08:03.058280+0000 mgr.gibba032.whopeo (mgr.24625) 844 : cluster [DBG] pgmap v468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:04 gibba011 bash[31774]: cluster 2021-03-10T14:08:03.058280+0000 mgr.gibba032.whopeo (mgr.24625) 844 : cluster [DBG] pgmap v468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:06 gibba032 bash[24004]: cluster 2021-03-10T14:08:05.059585+0000 mgr.gibba032.whopeo (mgr.24625) 845 : cluster [DBG] pgmap v469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:06 gibba011 bash[30791]: cluster 2021-03-10T14:08:05.059585+0000 mgr.gibba032.whopeo (mgr.24625) 845 : cluster [DBG] pgmap v469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:06 gibba011 bash[31774]: cluster 2021-03-10T14:08:05.059585+0000 mgr.gibba032.whopeo (mgr.24625) 845 : cluster [DBG] pgmap v469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:08 gibba032 bash[24004]: cluster 2021-03-10T14:08:07.061253+0000 mgr.gibba032.whopeo (mgr.24625) 846 : cluster [DBG] pgmap v470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:08.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:08 gibba011 bash[30791]: cluster 2021-03-10T14:08:07.061253+0000 mgr.gibba032.whopeo (mgr.24625) 846 : cluster [DBG] pgmap v470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:08 gibba011 bash[31774]: cluster 2021-03-10T14:08:07.061253+0000 mgr.gibba032.whopeo (mgr.24625) 846 : cluster [DBG] pgmap v470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:10 gibba032 bash[24004]: cluster 2021-03-10T14:08:09.062834+0000 mgr.gibba032.whopeo (mgr.24625) 847 : cluster [DBG] pgmap v471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:10 gibba011 bash[30791]: cluster 2021-03-10T14:08:09.062834+0000 mgr.gibba032.whopeo (mgr.24625) 847 : cluster [DBG] pgmap v471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:10 gibba011 bash[31774]: cluster 2021-03-10T14:08:09.062834+0000 mgr.gibba032.whopeo (mgr.24625) 847 : cluster [DBG] pgmap v471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:12 gibba032 bash[24004]: cluster 2021-03-10T14:08:11.064392+0000 mgr.gibba032.whopeo (mgr.24625) 848 : cluster [DBG] pgmap v472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:12 gibba032 bash[24004]: audit 2021-03-10T14:08:12.183604+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:08:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:12 gibba011 bash[30791]: cluster 2021-03-10T14:08:11.064392+0000 mgr.gibba032.whopeo (mgr.24625) 848 : cluster [DBG] pgmap v472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:12 gibba011 bash[30791]: audit 2021-03-10T14:08:12.183604+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:08:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:12 gibba011 bash[31774]: cluster 2021-03-10T14:08:11.064392+0000 mgr.gibba032.whopeo (mgr.24625) 848 : cluster [DBG] pgmap v472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:12 gibba011 bash[31774]: audit 2021-03-10T14:08:12.183604+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:08:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:13 gibba032 bash[24004]: cephadm 2021-03-10T14:08:12.179113+0000 mgr.gibba032.whopeo (mgr.24625) 849 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:08:13.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:13 gibba011 bash[30791]: cephadm 2021-03-10T14:08:12.179113+0000 mgr.gibba032.whopeo (mgr.24625) 849 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:08:13.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:13 gibba011 bash[31774]: cephadm 2021-03-10T14:08:12.179113+0000 mgr.gibba032.whopeo (mgr.24625) 849 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:08:14.460 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:08:14.692 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:08:14.692 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (14m) 62s ago 20m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:08:14.693 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (20m) 10m ago 20m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:08:14.693 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (15m) 10m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:08:14.693 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (15m) 10m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:08:14.693 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (14m) 62s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:08:14.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (14m) 62s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:08:14.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (13m) 10m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:08:14.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (14m) 62s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:08:14.694 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (20m) 62s ago 20m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:08:14.694 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (20m) 10m ago 20m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:08:14.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (13m) 62s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:08:14.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (12m) 62s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:08:14.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (12m) 62s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:08:14.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (11m) 62s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:08:14.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (11m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:08:14.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (11m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:08:14.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (10m) 10m ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:08:14.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (10m) 10m ago 20m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:08:14.696 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (19m) 10m ago 20m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:08:14.697 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 62s ago 18m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:08:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:14 gibba032 bash[24004]: cluster 2021-03-10T14:08:13.065392+0000 mgr.gibba032.whopeo (mgr.24625) 850 : cluster [DBG] pgmap v473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:14 gibba011 bash[30791]: cluster 2021-03-10T14:08:13.065392+0000 mgr.gibba032.whopeo (mgr.24625) 850 : cluster [DBG] pgmap v473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:14 gibba011 bash[31774]: cluster 2021-03-10T14:08:13.065392+0000 mgr.gibba032.whopeo (mgr.24625) 850 : cluster [DBG] pgmap v473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:14.939 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:08:14.939 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:08:14.939 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:08:14.940 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:14.940 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:08:14.940 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:08:14.940 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:14.940 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:08:14.941 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:08:14.941 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:14.941 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:08:14.941 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:08:14.942 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:08:14.942 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:08:14.942 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:08:15.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:15 gibba032 bash[24004]: audit 2021-03-10T14:08:14.448871+0000 mgr.gibba032.whopeo (mgr.24625) 851 : audit [DBG] from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:15.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:15 gibba032 bash[24004]: audit 2021-03-10T14:08:14.939537+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/1612126543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:15.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:15 gibba011 bash[30791]: audit 2021-03-10T14:08:14.448871+0000 mgr.gibba032.whopeo (mgr.24625) 851 : audit [DBG] from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:15.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:15 gibba011 bash[30791]: audit 2021-03-10T14:08:14.939537+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/1612126543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:15.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:15 gibba011 bash[31774]: audit 2021-03-10T14:08:14.448871+0000 mgr.gibba032.whopeo (mgr.24625) 851 : audit [DBG] from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:15.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:15 gibba011 bash[31774]: audit 2021-03-10T14:08:14.939537+0000 mon.c (mon.1) 15 : audit [DBG] from='client.? 172.21.2.111:0/1612126543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:14.677389+0000 mgr.gibba032.whopeo (mgr.24625) 852 : audit [DBG] from='client.54622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: cluster 2021-03-10T14:08:15.066025+0000 mgr.gibba032.whopeo (mgr.24625) 853 : cluster [DBG] pgmap v474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:15.914978+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.112981+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.116927+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:08:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.118584+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.120274+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.121589+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.124807+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:16 gibba032 bash[24004]: audit 2021-03-10T14:08:16.126391+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:14.677389+0000 mgr.gibba032.whopeo (mgr.24625) 852 : audit [DBG] from='client.54622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: cluster 2021-03-10T14:08:15.066025+0000 mgr.gibba032.whopeo (mgr.24625) 853 : cluster [DBG] pgmap v474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:15.914978+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.112981+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.116927+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:08:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.118584+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.120274+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.121589+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.124807+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[30791]: audit 2021-03-10T14:08:16.126391+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:14.677389+0000 mgr.gibba032.whopeo (mgr.24625) 852 : audit [DBG] from='client.54622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: cluster 2021-03-10T14:08:15.066025+0000 mgr.gibba032.whopeo (mgr.24625) 853 : cluster [DBG] pgmap v474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:15.914978+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.112981+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:08:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.116927+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:08:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.118584+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.120274+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.121589+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.124807+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:16.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:16 gibba011 bash[31774]: audit 2021-03-10T14:08:16.126391+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:17.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:15.913383+0000 mgr.gibba032.whopeo (mgr.24625) 854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:08:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.113486+0000 mgr.gibba032.whopeo (mgr.24625) 855 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:08:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.115691+0000 mgr.gibba032.whopeo (mgr.24625) 856 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:08:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.117300+0000 mgr.gibba032.whopeo (mgr.24625) 857 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:08:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.117336+0000 mgr.gibba032.whopeo (mgr.24625) 858 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:08:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.118791+0000 mgr.gibba032.whopeo (mgr.24625) 859 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:08:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.118830+0000 mgr.gibba032.whopeo (mgr.24625) 860 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:08:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.120322+0000 mgr.gibba032.whopeo (mgr.24625) 861 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:08:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.121936+0000 mgr.gibba032.whopeo (mgr.24625) 862 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:08:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.121974+0000 mgr.gibba032.whopeo (mgr.24625) 863 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:08:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.123232+0000 mgr.gibba032.whopeo (mgr.24625) 864 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:08:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.123612+0000 mgr.gibba032.whopeo (mgr.24625) 865 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:08:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.124890+0000 mgr.gibba032.whopeo (mgr.24625) 866 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:08:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.124927+0000 mgr.gibba032.whopeo (mgr.24625) 867 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:08:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[30791]: cephadm 2021-03-10T14:08:16.125032+0000 mgr.gibba032.whopeo (mgr.24625) 868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:15.913383+0000 mgr.gibba032.whopeo (mgr.24625) 854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:08:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.113486+0000 mgr.gibba032.whopeo (mgr.24625) 855 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:08:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.115691+0000 mgr.gibba032.whopeo (mgr.24625) 856 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:08:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.117300+0000 mgr.gibba032.whopeo (mgr.24625) 857 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:08:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.117336+0000 mgr.gibba032.whopeo (mgr.24625) 858 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:08:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.118791+0000 mgr.gibba032.whopeo (mgr.24625) 859 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:08:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.118830+0000 mgr.gibba032.whopeo (mgr.24625) 860 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:08:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.120322+0000 mgr.gibba032.whopeo (mgr.24625) 861 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:08:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.121936+0000 mgr.gibba032.whopeo (mgr.24625) 862 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:08:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.121974+0000 mgr.gibba032.whopeo (mgr.24625) 863 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:08:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.123232+0000 mgr.gibba032.whopeo (mgr.24625) 864 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:08:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.123612+0000 mgr.gibba032.whopeo (mgr.24625) 865 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:08:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.124890+0000 mgr.gibba032.whopeo (mgr.24625) 866 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:08:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.124927+0000 mgr.gibba032.whopeo (mgr.24625) 867 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:08:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:17 gibba011 bash[31774]: cephadm 2021-03-10T14:08:16.125032+0000 mgr.gibba032.whopeo (mgr.24625) 868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:15.913383+0000 mgr.gibba032.whopeo (mgr.24625) 854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:08:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.113486+0000 mgr.gibba032.whopeo (mgr.24625) 855 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:08:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.115691+0000 mgr.gibba032.whopeo (mgr.24625) 856 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:08:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.117300+0000 mgr.gibba032.whopeo (mgr.24625) 857 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:08:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.117336+0000 mgr.gibba032.whopeo (mgr.24625) 858 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:08:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.118791+0000 mgr.gibba032.whopeo (mgr.24625) 859 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:08:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.118830+0000 mgr.gibba032.whopeo (mgr.24625) 860 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:08:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.120322+0000 mgr.gibba032.whopeo (mgr.24625) 861 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:08:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.121936+0000 mgr.gibba032.whopeo (mgr.24625) 862 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:08:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.121974+0000 mgr.gibba032.whopeo (mgr.24625) 863 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:08:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.123232+0000 mgr.gibba032.whopeo (mgr.24625) 864 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:08:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.123612+0000 mgr.gibba032.whopeo (mgr.24625) 865 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:08:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.124890+0000 mgr.gibba032.whopeo (mgr.24625) 866 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:08:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.124927+0000 mgr.gibba032.whopeo (mgr.24625) 867 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:08:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:17 gibba032 bash[24004]: cephadm 2021-03-10T14:08:16.125032+0000 mgr.gibba032.whopeo (mgr.24625) 868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:18 gibba011 bash[30791]: cluster 2021-03-10T14:08:17.067545+0000 mgr.gibba032.whopeo (mgr.24625) 869 : cluster [DBG] pgmap v475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:18 gibba011 bash[31774]: cluster 2021-03-10T14:08:17.067545+0000 mgr.gibba032.whopeo (mgr.24625) 869 : cluster [DBG] pgmap v475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:18 gibba032 bash[24004]: cluster 2021-03-10T14:08:17.067545+0000 mgr.gibba032.whopeo (mgr.24625) 869 : cluster [DBG] pgmap v475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:20 gibba011 bash[30791]: cluster 2021-03-10T14:08:19.069116+0000 mgr.gibba032.whopeo (mgr.24625) 870 : cluster [DBG] pgmap v476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:20 gibba011 bash[31774]: cluster 2021-03-10T14:08:19.069116+0000 mgr.gibba032.whopeo (mgr.24625) 870 : cluster [DBG] pgmap v476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:20 gibba032 bash[24004]: cluster 2021-03-10T14:08:19.069116+0000 mgr.gibba032.whopeo (mgr.24625) 870 : cluster [DBG] pgmap v476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:22 gibba011 bash[30791]: cluster 2021-03-10T14:08:21.070693+0000 mgr.gibba032.whopeo (mgr.24625) 871 : cluster [DBG] pgmap v477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:22 gibba011 bash[31774]: cluster 2021-03-10T14:08:21.070693+0000 mgr.gibba032.whopeo (mgr.24625) 871 : cluster [DBG] pgmap v477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:22 gibba032 bash[24004]: cluster 2021-03-10T14:08:21.070693+0000 mgr.gibba032.whopeo (mgr.24625) 871 : cluster [DBG] pgmap v477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:24 gibba011 bash[30791]: cluster 2021-03-10T14:08:23.071265+0000 mgr.gibba032.whopeo (mgr.24625) 872 : cluster [DBG] pgmap v478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:24 gibba011 bash[31774]: cluster 2021-03-10T14:08:23.071265+0000 mgr.gibba032.whopeo (mgr.24625) 872 : cluster [DBG] pgmap v478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:24 gibba032 bash[24004]: cluster 2021-03-10T14:08:23.071265+0000 mgr.gibba032.whopeo (mgr.24625) 872 : cluster [DBG] pgmap v478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:26 gibba011 bash[30791]: cluster 2021-03-10T14:08:25.072599+0000 mgr.gibba032.whopeo (mgr.24625) 873 : cluster [DBG] pgmap v479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:26 gibba011 bash[31774]: cluster 2021-03-10T14:08:25.072599+0000 mgr.gibba032.whopeo (mgr.24625) 873 : cluster [DBG] pgmap v479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:26 gibba032 bash[24004]: cluster 2021-03-10T14:08:25.072599+0000 mgr.gibba032.whopeo (mgr.24625) 873 : cluster [DBG] pgmap v479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:28 gibba011 bash[30791]: cluster 2021-03-10T14:08:27.074019+0000 mgr.gibba032.whopeo (mgr.24625) 874 : cluster [DBG] pgmap v480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:28 gibba011 bash[31774]: cluster 2021-03-10T14:08:27.074019+0000 mgr.gibba032.whopeo (mgr.24625) 874 : cluster [DBG] pgmap v480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:28 gibba032 bash[24004]: cluster 2021-03-10T14:08:27.074019+0000 mgr.gibba032.whopeo (mgr.24625) 874 : cluster [DBG] pgmap v480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:30 gibba011 bash[30791]: cluster 2021-03-10T14:08:29.075606+0000 mgr.gibba032.whopeo (mgr.24625) 875 : cluster [DBG] pgmap v481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:30 gibba011 bash[31774]: cluster 2021-03-10T14:08:29.075606+0000 mgr.gibba032.whopeo (mgr.24625) 875 : cluster [DBG] pgmap v481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:30 gibba032 bash[24004]: cluster 2021-03-10T14:08:29.075606+0000 mgr.gibba032.whopeo (mgr.24625) 875 : cluster [DBG] pgmap v481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:32 gibba032 bash[24004]: cluster 2021-03-10T14:08:31.077286+0000 mgr.gibba032.whopeo (mgr.24625) 876 : cluster [DBG] pgmap v482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:32 gibba032 bash[24004]: cephadm 2021-03-10T14:08:31.137657+0000 mgr.gibba032.whopeo (mgr.24625) 877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:32 gibba011 bash[30791]: cluster 2021-03-10T14:08:31.077286+0000 mgr.gibba032.whopeo (mgr.24625) 876 : cluster [DBG] pgmap v482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:32 gibba011 bash[30791]: cephadm 2021-03-10T14:08:31.137657+0000 mgr.gibba032.whopeo (mgr.24625) 877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:32 gibba011 bash[31774]: cluster 2021-03-10T14:08:31.077286+0000 mgr.gibba032.whopeo (mgr.24625) 876 : cluster [DBG] pgmap v482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:32 gibba011 bash[31774]: cephadm 2021-03-10T14:08:31.137657+0000 mgr.gibba032.whopeo (mgr.24625) 877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:34 gibba032 bash[24004]: cluster 2021-03-10T14:08:33.078113+0000 mgr.gibba032.whopeo (mgr.24625) 878 : cluster [DBG] pgmap v483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:35.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:34 gibba011 bash[30791]: cluster 2021-03-10T14:08:33.078113+0000 mgr.gibba032.whopeo (mgr.24625) 878 : cluster [DBG] pgmap v483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:34 gibba011 bash[31774]: cluster 2021-03-10T14:08:33.078113+0000 mgr.gibba032.whopeo (mgr.24625) 878 : cluster [DBG] pgmap v483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:36 gibba032 bash[24004]: cluster 2021-03-10T14:08:35.079676+0000 mgr.gibba032.whopeo (mgr.24625) 879 : cluster [DBG] pgmap v484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:36 gibba011 bash[30791]: cluster 2021-03-10T14:08:35.079676+0000 mgr.gibba032.whopeo (mgr.24625) 879 : cluster [DBG] pgmap v484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:36 gibba011 bash[31774]: cluster 2021-03-10T14:08:35.079676+0000 mgr.gibba032.whopeo (mgr.24625) 879 : cluster [DBG] pgmap v484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:38 gibba032 bash[24004]: cluster 2021-03-10T14:08:37.081412+0000 mgr.gibba032.whopeo (mgr.24625) 880 : cluster [DBG] pgmap v485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:39.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:38 gibba011 bash[30791]: cluster 2021-03-10T14:08:37.081412+0000 mgr.gibba032.whopeo (mgr.24625) 880 : cluster [DBG] pgmap v485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:38 gibba011 bash[31774]: cluster 2021-03-10T14:08:37.081412+0000 mgr.gibba032.whopeo (mgr.24625) 880 : cluster [DBG] pgmap v485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:41.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:40 gibba032 bash[24004]: cluster 2021-03-10T14:08:39.082976+0000 mgr.gibba032.whopeo (mgr.24625) 881 : cluster [DBG] pgmap v486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:40 gibba011 bash[30791]: cluster 2021-03-10T14:08:39.082976+0000 mgr.gibba032.whopeo (mgr.24625) 881 : cluster [DBG] pgmap v486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:40 gibba011 bash[31774]: cluster 2021-03-10T14:08:39.082976+0000 mgr.gibba032.whopeo (mgr.24625) 881 : cluster [DBG] pgmap v486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:42 gibba032 bash[24004]: cluster 2021-03-10T14:08:41.084648+0000 mgr.gibba032.whopeo (mgr.24625) 882 : cluster [DBG] pgmap v487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:43.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:42 gibba011 bash[30791]: cluster 2021-03-10T14:08:41.084648+0000 mgr.gibba032.whopeo (mgr.24625) 882 : cluster [DBG] pgmap v487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:43.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:42 gibba011 bash[31774]: cluster 2021-03-10T14:08:41.084648+0000 mgr.gibba032.whopeo (mgr.24625) 882 : cluster [DBG] pgmap v487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:44 gibba032 bash[24004]: cluster 2021-03-10T14:08:43.085523+0000 mgr.gibba032.whopeo (mgr.24625) 883 : cluster [DBG] pgmap v488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:44 gibba011 bash[30791]: cluster 2021-03-10T14:08:43.085523+0000 mgr.gibba032.whopeo (mgr.24625) 883 : cluster [DBG] pgmap v488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:44 gibba011 bash[31774]: cluster 2021-03-10T14:08:43.085523+0000 mgr.gibba032.whopeo (mgr.24625) 883 : cluster [DBG] pgmap v488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:45.187 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:08:45.446 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:08:45.446 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (15m) 93s ago 20m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:08:45.446 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (20m) 29s ago 20m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:08:45.447 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (15m) 29s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:08:45.447 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (16m) 29s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:08:45.447 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (15m) 93s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:08:45.447 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (14m) 93s ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:08:45.447 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (14m) 29s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:08:45.448 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (14m) 93s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:08:45.448 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (20m) 93s ago 20m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:08:45.448 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (20m) 29s ago 20m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:08:45.448 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (13m) 93s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:08:45.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (13m) 93s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:08:45.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (12m) 93s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:08:45.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (12m) 93s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:08:45.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (11m) 29s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:08:45.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (11m) 29s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:08:45.450 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (11m) 29s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:08:45.450 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (10m) 29s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:08:45.450 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (20m) 29s ago 20m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:08:45.450 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 93s ago 18m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:08:45.705 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:08:45.705 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:08:45.705 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:08:45.705 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:45.706 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:08:45.706 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:08:45.706 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:45.707 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:08:45.707 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:08:45.707 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:08:45.707 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:08:45.707 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:08:45.708 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:08:45.708 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:08:45.708 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:08:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:46 gibba032 bash[24004]: cluster 2021-03-10T14:08:45.087038+0000 mgr.gibba032.whopeo (mgr.24625) 884 : cluster [DBG] pgmap v489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:46 gibba032 bash[24004]: audit 2021-03-10T14:08:45.176798+0000 mgr.gibba032.whopeo (mgr.24625) 885 : audit [DBG] from='client.44649 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:46 gibba032 bash[24004]: audit 2021-03-10T14:08:45.403199+0000 mgr.gibba032.whopeo (mgr.24625) 886 : audit [DBG] from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:46 gibba032 bash[24004]: audit 2021-03-10T14:08:45.704937+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/439348323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[30791]: cluster 2021-03-10T14:08:45.087038+0000 mgr.gibba032.whopeo (mgr.24625) 884 : cluster [DBG] pgmap v489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[30791]: audit 2021-03-10T14:08:45.176798+0000 mgr.gibba032.whopeo (mgr.24625) 885 : audit [DBG] from='client.44649 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[30791]: audit 2021-03-10T14:08:45.403199+0000 mgr.gibba032.whopeo (mgr.24625) 886 : audit [DBG] from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[30791]: audit 2021-03-10T14:08:45.704937+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/439348323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[31774]: cluster 2021-03-10T14:08:45.087038+0000 mgr.gibba032.whopeo (mgr.24625) 884 : cluster [DBG] pgmap v489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[31774]: audit 2021-03-10T14:08:45.176798+0000 mgr.gibba032.whopeo (mgr.24625) 885 : audit [DBG] from='client.44649 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[31774]: audit 2021-03-10T14:08:45.403199+0000 mgr.gibba032.whopeo (mgr.24625) 886 : audit [DBG] from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:08:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:46 gibba011 bash[31774]: audit 2021-03-10T14:08:45.704937+0000 mon.c (mon.1) 16 : audit [DBG] from='client.? 172.21.2.111:0/439348323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:08:47.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:47 gibba032 bash[24004]: cephadm 2021-03-10T14:08:46.152799+0000 mgr.gibba032.whopeo (mgr.24625) 887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:47.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:47 gibba011 bash[30791]: cephadm 2021-03-10T14:08:46.152799+0000 mgr.gibba032.whopeo (mgr.24625) 887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:47.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:47 gibba011 bash[31774]: cephadm 2021-03-10T14:08:46.152799+0000 mgr.gibba032.whopeo (mgr.24625) 887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:08:48.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:48 gibba032 bash[24004]: cluster 2021-03-10T14:08:47.088535+0000 mgr.gibba032.whopeo (mgr.24625) 888 : cluster [DBG] pgmap v490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:48 gibba011 bash[30791]: cluster 2021-03-10T14:08:47.088535+0000 mgr.gibba032.whopeo (mgr.24625) 888 : cluster [DBG] pgmap v490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:48 gibba011 bash[31774]: cluster 2021-03-10T14:08:47.088535+0000 mgr.gibba032.whopeo (mgr.24625) 888 : cluster [DBG] pgmap v490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:50 gibba011 bash[30791]: cluster 2021-03-10T14:08:49.089993+0000 mgr.gibba032.whopeo (mgr.24625) 889 : cluster [DBG] pgmap v491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:50 gibba011 bash[31774]: cluster 2021-03-10T14:08:49.089993+0000 mgr.gibba032.whopeo (mgr.24625) 889 : cluster [DBG] pgmap v491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:50 gibba032 bash[24004]: cluster 2021-03-10T14:08:49.089993+0000 mgr.gibba032.whopeo (mgr.24625) 889 : cluster [DBG] pgmap v491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:52 gibba011 bash[30791]: cluster 2021-03-10T14:08:51.091503+0000 mgr.gibba032.whopeo (mgr.24625) 890 : cluster [DBG] pgmap v492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:52 gibba011 bash[31774]: cluster 2021-03-10T14:08:51.091503+0000 mgr.gibba032.whopeo (mgr.24625) 890 : cluster [DBG] pgmap v492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:52 gibba032 bash[24004]: cluster 2021-03-10T14:08:51.091503+0000 mgr.gibba032.whopeo (mgr.24625) 890 : cluster [DBG] pgmap v492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:54 gibba011 bash[30791]: cluster 2021-03-10T14:08:53.092386+0000 mgr.gibba032.whopeo (mgr.24625) 891 : cluster [DBG] pgmap v493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:54 gibba011 bash[31774]: cluster 2021-03-10T14:08:53.092386+0000 mgr.gibba032.whopeo (mgr.24625) 891 : cluster [DBG] pgmap v493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:54 gibba032 bash[24004]: cluster 2021-03-10T14:08:53.092386+0000 mgr.gibba032.whopeo (mgr.24625) 891 : cluster [DBG] pgmap v493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: cluster 2021-03-10T14:08:55.093717+0000 mgr.gibba032.whopeo (mgr.24625) 892 : cluster [DBG] pgmap v494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.000901+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:08:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.001623+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.002373+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.003096+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:08:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.098785+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:08:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[30791]: audit 2021-03-10T14:08:56.099508+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:08:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: cluster 2021-03-10T14:08:55.093717+0000 mgr.gibba032.whopeo (mgr.24625) 892 : cluster [DBG] pgmap v494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.000901+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:08:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.001623+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.002373+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.003096+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:08:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.098785+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:08:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:56 gibba011 bash[31774]: audit 2021-03-10T14:08:56.099508+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:08:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: cluster 2021-03-10T14:08:55.093717+0000 mgr.gibba032.whopeo (mgr.24625) 892 : cluster [DBG] pgmap v494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.000901+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:08:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.001623+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.002373+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:08:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.003096+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:08:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.098785+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:08:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:56 gibba032 bash[24004]: audit 2021-03-10T14:08:56.099508+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:08:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:08:58 gibba011 bash[30791]: cluster 2021-03-10T14:08:57.095242+0000 mgr.gibba032.whopeo (mgr.24625) 893 : cluster [DBG] pgmap v495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:08:58 gibba011 bash[31774]: cluster 2021-03-10T14:08:57.095242+0000 mgr.gibba032.whopeo (mgr.24625) 893 : cluster [DBG] pgmap v495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:08:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:08:58 gibba032 bash[24004]: cluster 2021-03-10T14:08:57.095242+0000 mgr.gibba032.whopeo (mgr.24625) 893 : cluster [DBG] pgmap v495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:00 gibba011 bash[30791]: cluster 2021-03-10T14:08:59.096877+0000 mgr.gibba032.whopeo (mgr.24625) 894 : cluster [DBG] pgmap v496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:00 gibba011 bash[31774]: cluster 2021-03-10T14:08:59.096877+0000 mgr.gibba032.whopeo (mgr.24625) 894 : cluster [DBG] pgmap v496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:00 gibba032 bash[24004]: cluster 2021-03-10T14:08:59.096877+0000 mgr.gibba032.whopeo (mgr.24625) 894 : cluster [DBG] pgmap v496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:02 gibba032 bash[24004]: cluster 2021-03-10T14:09:01.098389+0000 mgr.gibba032.whopeo (mgr.24625) 895 : cluster [DBG] pgmap v497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:02 gibba032 bash[24004]: cephadm 2021-03-10T14:09:01.156796+0000 mgr.gibba032.whopeo (mgr.24625) 896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:02 gibba011 bash[30791]: cluster 2021-03-10T14:09:01.098389+0000 mgr.gibba032.whopeo (mgr.24625) 895 : cluster [DBG] pgmap v497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:02 gibba011 bash[30791]: cephadm 2021-03-10T14:09:01.156796+0000 mgr.gibba032.whopeo (mgr.24625) 896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:02 gibba011 bash[31774]: cluster 2021-03-10T14:09:01.098389+0000 mgr.gibba032.whopeo (mgr.24625) 895 : cluster [DBG] pgmap v497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:02.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:02 gibba011 bash[31774]: cephadm 2021-03-10T14:09:01.156796+0000 mgr.gibba032.whopeo (mgr.24625) 896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:04 gibba032 bash[24004]: cluster 2021-03-10T14:09:03.100077+0000 mgr.gibba032.whopeo (mgr.24625) 897 : cluster [DBG] pgmap v498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:04 gibba011 bash[30791]: cluster 2021-03-10T14:09:03.100077+0000 mgr.gibba032.whopeo (mgr.24625) 897 : cluster [DBG] pgmap v498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:04 gibba011 bash[31774]: cluster 2021-03-10T14:09:03.100077+0000 mgr.gibba032.whopeo (mgr.24625) 897 : cluster [DBG] pgmap v498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:06 gibba032 bash[24004]: cluster 2021-03-10T14:09:05.101442+0000 mgr.gibba032.whopeo (mgr.24625) 898 : cluster [DBG] pgmap v499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:06 gibba011 bash[30791]: cluster 2021-03-10T14:09:05.101442+0000 mgr.gibba032.whopeo (mgr.24625) 898 : cluster [DBG] pgmap v499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:06 gibba011 bash[31774]: cluster 2021-03-10T14:09:05.101442+0000 mgr.gibba032.whopeo (mgr.24625) 898 : cluster [DBG] pgmap v499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:08 gibba032 bash[24004]: cluster 2021-03-10T14:09:07.102729+0000 mgr.gibba032.whopeo (mgr.24625) 899 : cluster [DBG] pgmap v500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:08.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:08 gibba011 bash[30791]: cluster 2021-03-10T14:09:07.102729+0000 mgr.gibba032.whopeo (mgr.24625) 899 : cluster [DBG] pgmap v500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:08 gibba011 bash[31774]: cluster 2021-03-10T14:09:07.102729+0000 mgr.gibba032.whopeo (mgr.24625) 899 : cluster [DBG] pgmap v500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:10 gibba032 bash[24004]: cluster 2021-03-10T14:09:09.104371+0000 mgr.gibba032.whopeo (mgr.24625) 900 : cluster [DBG] pgmap v501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:10 gibba011 bash[30791]: cluster 2021-03-10T14:09:09.104371+0000 mgr.gibba032.whopeo (mgr.24625) 900 : cluster [DBG] pgmap v501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:10 gibba011 bash[31774]: cluster 2021-03-10T14:09:09.104371+0000 mgr.gibba032.whopeo (mgr.24625) 900 : cluster [DBG] pgmap v501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:12 gibba032 bash[24004]: cluster 2021-03-10T14:09:11.105838+0000 mgr.gibba032.whopeo (mgr.24625) 901 : cluster [DBG] pgmap v502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:12.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:12 gibba011 bash[30791]: cluster 2021-03-10T14:09:11.105838+0000 mgr.gibba032.whopeo (mgr.24625) 901 : cluster [DBG] pgmap v502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:12 gibba011 bash[31774]: cluster 2021-03-10T14:09:11.105838+0000 mgr.gibba032.whopeo (mgr.24625) 901 : cluster [DBG] pgmap v502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:14 gibba032 bash[24004]: cluster 2021-03-10T14:09:13.106658+0000 mgr.gibba032.whopeo (mgr.24625) 902 : cluster [DBG] pgmap v503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:14 gibba011 bash[30791]: cluster 2021-03-10T14:09:13.106658+0000 mgr.gibba032.whopeo (mgr.24625) 902 : cluster [DBG] pgmap v503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:14 gibba011 bash[31774]: cluster 2021-03-10T14:09:13.106658+0000 mgr.gibba032.whopeo (mgr.24625) 902 : cluster [DBG] pgmap v503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:15.948 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:09:16.178 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:09:16.178 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (15m) 2m ago 21m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:09:16.179 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (21m) 60s ago 21m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:09:16.179 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (16m) 60s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:09:16.180 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (16m) 60s ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:09:16.180 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (15m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:09:16.180 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (15m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:09:16.181 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (14m) 60s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:09:16.181 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (15m) 2m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:09:16.182 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (21m) 2m ago 21m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:09:16.182 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (21m) 60s ago 21m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:09:16.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (14m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:09:16.183 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (13m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:09:16.183 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (13m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:09:16.184 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (12m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:09:16.184 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (12m) 60s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:09:16.184 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (12m) 60s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:09:16.185 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (11m) 60s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:09:16.185 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (11m) 60s ago 21m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:09:16.185 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (20m) 60s ago 21m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:09:16.186 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 19m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:09:16.435 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:09:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:09:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:09:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:09:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:09:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:09:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:09:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:09:16.438 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:09:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: cluster 2021-03-10T14:09:15.108210+0000 mgr.gibba032.whopeo (mgr.24625) 903 : cluster [DBG] pgmap v504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.365582+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:09:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.368736+0000 mon.a (mon.0) 829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:09:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.370040+0000 mon.a (mon.0) 830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.371215+0000 mon.a (mon.0) 831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.372313+0000 mon.a (mon.0) 832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.374558+0000 mon.a (mon.0) 833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:16 gibba032 bash[24004]: audit 2021-03-10T14:09:16.375904+0000 mon.a (mon.0) 834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: cluster 2021-03-10T14:09:15.108210+0000 mgr.gibba032.whopeo (mgr.24625) 903 : cluster [DBG] pgmap v504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.365582+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:09:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.368736+0000 mon.a (mon.0) 829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:09:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.370040+0000 mon.a (mon.0) 830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.371215+0000 mon.a (mon.0) 831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.372313+0000 mon.a (mon.0) 832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.374558+0000 mon.a (mon.0) 833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[30791]: audit 2021-03-10T14:09:16.375904+0000 mon.a (mon.0) 834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: cluster 2021-03-10T14:09:15.108210+0000 mgr.gibba032.whopeo (mgr.24625) 903 : cluster [DBG] pgmap v504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.365582+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:09:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.368736+0000 mon.a (mon.0) 829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:09:16.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.370040+0000 mon.a (mon.0) 830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.371215+0000 mon.a (mon.0) 831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.372313+0000 mon.a (mon.0) 832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.374558+0000 mon.a (mon.0) 833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:16.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:16 gibba011 bash[31774]: audit 2021-03-10T14:09:16.375904+0000 mon.a (mon.0) 834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:17.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: audit 2021-03-10T14:09:15.937403+0000 mgr.gibba032.whopeo (mgr.24625) 904 : audit [DBG] from='client.44667 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: audit 2021-03-10T14:09:16.162585+0000 mgr.gibba032.whopeo (mgr.24625) 905 : audit [DBG] from='client.54658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.175346+0000 mgr.gibba032.whopeo (mgr.24625) 906 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:09:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.175711+0000 mgr.gibba032.whopeo (mgr.24625) 907 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:09:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.365826+0000 mgr.gibba032.whopeo (mgr.24625) 908 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:09:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.367479+0000 mgr.gibba032.whopeo (mgr.24625) 909 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:09:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.368615+0000 mgr.gibba032.whopeo (mgr.24625) 910 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:09:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.368655+0000 mgr.gibba032.whopeo (mgr.24625) 911 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:09:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.369783+0000 mgr.gibba032.whopeo (mgr.24625) 912 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:09:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.369822+0000 mgr.gibba032.whopeo (mgr.24625) 913 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:09:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.370849+0000 mgr.gibba032.whopeo (mgr.24625) 914 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:09:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.371826+0000 mgr.gibba032.whopeo (mgr.24625) 915 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:09:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.371864+0000 mgr.gibba032.whopeo (mgr.24625) 916 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:09:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.373125+0000 mgr.gibba032.whopeo (mgr.24625) 917 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:09:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.373438+0000 mgr.gibba032.whopeo (mgr.24625) 918 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:09:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.374787+0000 mgr.gibba032.whopeo (mgr.24625) 919 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:09:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.374827+0000 mgr.gibba032.whopeo (mgr.24625) 920 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:09:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: cephadm 2021-03-10T14:09:16.374934+0000 mgr.gibba032.whopeo (mgr.24625) 921 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:17 gibba032 bash[24004]: audit 2021-03-10T14:09:16.435544+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/2699671763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:17.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: audit 2021-03-10T14:09:15.937403+0000 mgr.gibba032.whopeo (mgr.24625) 904 : audit [DBG] from='client.44667 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: audit 2021-03-10T14:09:16.162585+0000 mgr.gibba032.whopeo (mgr.24625) 905 : audit [DBG] from='client.54658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.175346+0000 mgr.gibba032.whopeo (mgr.24625) 906 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:09:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.175711+0000 mgr.gibba032.whopeo (mgr.24625) 907 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:09:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.365826+0000 mgr.gibba032.whopeo (mgr.24625) 908 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:09:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.367479+0000 mgr.gibba032.whopeo (mgr.24625) 909 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:09:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.368615+0000 mgr.gibba032.whopeo (mgr.24625) 910 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:09:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.368655+0000 mgr.gibba032.whopeo (mgr.24625) 911 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:09:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.369783+0000 mgr.gibba032.whopeo (mgr.24625) 912 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:09:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.369822+0000 mgr.gibba032.whopeo (mgr.24625) 913 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:09:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.370849+0000 mgr.gibba032.whopeo (mgr.24625) 914 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:09:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.371826+0000 mgr.gibba032.whopeo (mgr.24625) 915 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.371864+0000 mgr.gibba032.whopeo (mgr.24625) 916 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.373125+0000 mgr.gibba032.whopeo (mgr.24625) 917 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.373438+0000 mgr.gibba032.whopeo (mgr.24625) 918 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.374787+0000 mgr.gibba032.whopeo (mgr.24625) 919 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.374827+0000 mgr.gibba032.whopeo (mgr.24625) 920 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:09:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: cephadm 2021-03-10T14:09:16.374934+0000 mgr.gibba032.whopeo (mgr.24625) 921 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[30791]: audit 2021-03-10T14:09:16.435544+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/2699671763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: audit 2021-03-10T14:09:15.937403+0000 mgr.gibba032.whopeo (mgr.24625) 904 : audit [DBG] from='client.44667 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: audit 2021-03-10T14:09:16.162585+0000 mgr.gibba032.whopeo (mgr.24625) 905 : audit [DBG] from='client.54658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.175346+0000 mgr.gibba032.whopeo (mgr.24625) 906 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:09:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.175711+0000 mgr.gibba032.whopeo (mgr.24625) 907 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:09:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.365826+0000 mgr.gibba032.whopeo (mgr.24625) 908 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:09:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.367479+0000 mgr.gibba032.whopeo (mgr.24625) 909 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:09:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.368615+0000 mgr.gibba032.whopeo (mgr.24625) 910 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:09:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.368655+0000 mgr.gibba032.whopeo (mgr.24625) 911 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.369783+0000 mgr.gibba032.whopeo (mgr.24625) 912 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.369822+0000 mgr.gibba032.whopeo (mgr.24625) 913 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.370849+0000 mgr.gibba032.whopeo (mgr.24625) 914 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.371826+0000 mgr.gibba032.whopeo (mgr.24625) 915 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.371864+0000 mgr.gibba032.whopeo (mgr.24625) 916 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:09:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.373125+0000 mgr.gibba032.whopeo (mgr.24625) 917 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:09:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.373438+0000 mgr.gibba032.whopeo (mgr.24625) 918 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:09:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.374787+0000 mgr.gibba032.whopeo (mgr.24625) 919 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:09:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.374827+0000 mgr.gibba032.whopeo (mgr.24625) 920 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:09:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: cephadm 2021-03-10T14:09:16.374934+0000 mgr.gibba032.whopeo (mgr.24625) 921 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:17 gibba011 bash[31774]: audit 2021-03-10T14:09:16.435544+0000 mon.c (mon.1) 17 : audit [DBG] from='client.? 172.21.2.111:0/2699671763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:18 gibba032 bash[24004]: cluster 2021-03-10T14:09:17.109599+0000 mgr.gibba032.whopeo (mgr.24625) 922 : cluster [DBG] pgmap v505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:18 gibba011 bash[30791]: cluster 2021-03-10T14:09:17.109599+0000 mgr.gibba032.whopeo (mgr.24625) 922 : cluster [DBG] pgmap v505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:18 gibba011 bash[31774]: cluster 2021-03-10T14:09:17.109599+0000 mgr.gibba032.whopeo (mgr.24625) 922 : cluster [DBG] pgmap v505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:20 gibba032 bash[24004]: cluster 2021-03-10T14:09:19.111264+0000 mgr.gibba032.whopeo (mgr.24625) 923 : cluster [DBG] pgmap v506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:20 gibba011 bash[30791]: cluster 2021-03-10T14:09:19.111264+0000 mgr.gibba032.whopeo (mgr.24625) 923 : cluster [DBG] pgmap v506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:20 gibba011 bash[31774]: cluster 2021-03-10T14:09:19.111264+0000 mgr.gibba032.whopeo (mgr.24625) 923 : cluster [DBG] pgmap v506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:22 gibba032 bash[24004]: cluster 2021-03-10T14:09:21.112926+0000 mgr.gibba032.whopeo (mgr.24625) 924 : cluster [DBG] pgmap v507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:22 gibba011 bash[30791]: cluster 2021-03-10T14:09:21.112926+0000 mgr.gibba032.whopeo (mgr.24625) 924 : cluster [DBG] pgmap v507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:22 gibba011 bash[31774]: cluster 2021-03-10T14:09:21.112926+0000 mgr.gibba032.whopeo (mgr.24625) 924 : cluster [DBG] pgmap v507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:24 gibba032 bash[24004]: cluster 2021-03-10T14:09:23.114462+0000 mgr.gibba032.whopeo (mgr.24625) 925 : cluster [DBG] pgmap v508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:24 gibba011 bash[30791]: cluster 2021-03-10T14:09:23.114462+0000 mgr.gibba032.whopeo (mgr.24625) 925 : cluster [DBG] pgmap v508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:24 gibba011 bash[31774]: cluster 2021-03-10T14:09:23.114462+0000 mgr.gibba032.whopeo (mgr.24625) 925 : cluster [DBG] pgmap v508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:26 gibba032 bash[24004]: cluster 2021-03-10T14:09:25.115942+0000 mgr.gibba032.whopeo (mgr.24625) 926 : cluster [DBG] pgmap v509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:26 gibba011 bash[30791]: cluster 2021-03-10T14:09:25.115942+0000 mgr.gibba032.whopeo (mgr.24625) 926 : cluster [DBG] pgmap v509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:26 gibba011 bash[31774]: cluster 2021-03-10T14:09:25.115942+0000 mgr.gibba032.whopeo (mgr.24625) 926 : cluster [DBG] pgmap v509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:28 gibba032 bash[24004]: cluster 2021-03-10T14:09:27.117360+0000 mgr.gibba032.whopeo (mgr.24625) 927 : cluster [DBG] pgmap v510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:28 gibba011 bash[30791]: cluster 2021-03-10T14:09:27.117360+0000 mgr.gibba032.whopeo (mgr.24625) 927 : cluster [DBG] pgmap v510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:28 gibba011 bash[31774]: cluster 2021-03-10T14:09:27.117360+0000 mgr.gibba032.whopeo (mgr.24625) 927 : cluster [DBG] pgmap v510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:30 gibba032 bash[24004]: cluster 2021-03-10T14:09:29.118901+0000 mgr.gibba032.whopeo (mgr.24625) 928 : cluster [DBG] pgmap v511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:30 gibba011 bash[30791]: cluster 2021-03-10T14:09:29.118901+0000 mgr.gibba032.whopeo (mgr.24625) 928 : cluster [DBG] pgmap v511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:30 gibba011 bash[31774]: cluster 2021-03-10T14:09:29.118901+0000 mgr.gibba032.whopeo (mgr.24625) 928 : cluster [DBG] pgmap v511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:32 gibba032 bash[24004]: cluster 2021-03-10T14:09:31.120464+0000 mgr.gibba032.whopeo (mgr.24625) 929 : cluster [DBG] pgmap v512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:32 gibba032 bash[24004]: cephadm 2021-03-10T14:09:31.390206+0000 mgr.gibba032.whopeo (mgr.24625) 930 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:32 gibba011 bash[30791]: cluster 2021-03-10T14:09:31.120464+0000 mgr.gibba032.whopeo (mgr.24625) 929 : cluster [DBG] pgmap v512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:32 gibba011 bash[30791]: cephadm 2021-03-10T14:09:31.390206+0000 mgr.gibba032.whopeo (mgr.24625) 930 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:32 gibba011 bash[31774]: cluster 2021-03-10T14:09:31.120464+0000 mgr.gibba032.whopeo (mgr.24625) 929 : cluster [DBG] pgmap v512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:32 gibba011 bash[31774]: cephadm 2021-03-10T14:09:31.390206+0000 mgr.gibba032.whopeo (mgr.24625) 930 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:34 gibba032 bash[24004]: cluster 2021-03-10T14:09:33.122114+0000 mgr.gibba032.whopeo (mgr.24625) 931 : cluster [DBG] pgmap v513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:34.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:34 gibba011 bash[30791]: cluster 2021-03-10T14:09:33.122114+0000 mgr.gibba032.whopeo (mgr.24625) 931 : cluster [DBG] pgmap v513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:34 gibba011 bash[31774]: cluster 2021-03-10T14:09:33.122114+0000 mgr.gibba032.whopeo (mgr.24625) 931 : cluster [DBG] pgmap v513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:36 gibba032 bash[24004]: cluster 2021-03-10T14:09:35.123540+0000 mgr.gibba032.whopeo (mgr.24625) 932 : cluster [DBG] pgmap v514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:36 gibba011 bash[30791]: cluster 2021-03-10T14:09:35.123540+0000 mgr.gibba032.whopeo (mgr.24625) 932 : cluster [DBG] pgmap v514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:36 gibba011 bash[31774]: cluster 2021-03-10T14:09:35.123540+0000 mgr.gibba032.whopeo (mgr.24625) 932 : cluster [DBG] pgmap v514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:38 gibba032 bash[24004]: cluster 2021-03-10T14:09:37.125018+0000 mgr.gibba032.whopeo (mgr.24625) 933 : cluster [DBG] pgmap v515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:38 gibba011 bash[30791]: cluster 2021-03-10T14:09:37.125018+0000 mgr.gibba032.whopeo (mgr.24625) 933 : cluster [DBG] pgmap v515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:38 gibba011 bash[31774]: cluster 2021-03-10T14:09:37.125018+0000 mgr.gibba032.whopeo (mgr.24625) 933 : cluster [DBG] pgmap v515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:40 gibba032 bash[24004]: cluster 2021-03-10T14:09:39.126621+0000 mgr.gibba032.whopeo (mgr.24625) 934 : cluster [DBG] pgmap v516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:40 gibba011 bash[30791]: cluster 2021-03-10T14:09:39.126621+0000 mgr.gibba032.whopeo (mgr.24625) 934 : cluster [DBG] pgmap v516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:40 gibba011 bash[31774]: cluster 2021-03-10T14:09:39.126621+0000 mgr.gibba032.whopeo (mgr.24625) 934 : cluster [DBG] pgmap v516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:42 gibba011 bash[30791]: cluster 2021-03-10T14:09:41.128090+0000 mgr.gibba032.whopeo (mgr.24625) 935 : cluster [DBG] pgmap v517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:42 gibba011 bash[31774]: cluster 2021-03-10T14:09:41.128090+0000 mgr.gibba032.whopeo (mgr.24625) 935 : cluster [DBG] pgmap v517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:43.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:42 gibba032 bash[24004]: cluster 2021-03-10T14:09:41.128090+0000 mgr.gibba032.whopeo (mgr.24625) 935 : cluster [DBG] pgmap v517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:44.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:44 gibba011 bash[30791]: cluster 2021-03-10T14:09:43.129676+0000 mgr.gibba032.whopeo (mgr.24625) 936 : cluster [DBG] pgmap v518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:44 gibba011 bash[31774]: cluster 2021-03-10T14:09:43.129676+0000 mgr.gibba032.whopeo (mgr.24625) 936 : cluster [DBG] pgmap v518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:44 gibba032 bash[24004]: cluster 2021-03-10T14:09:43.129676+0000 mgr.gibba032.whopeo (mgr.24625) 936 : cluster [DBG] pgmap v518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:46.682 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:09:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:46 gibba032 bash[24004]: cluster 2021-03-10T14:09:45.131252+0000 mgr.gibba032.whopeo (mgr.24625) 937 : cluster [DBG] pgmap v519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:46.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:46 gibba011 bash[30791]: cluster 2021-03-10T14:09:45.131252+0000 mgr.gibba032.whopeo (mgr.24625) 937 : cluster [DBG] pgmap v519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:46 gibba011 bash[31774]: cluster 2021-03-10T14:09:45.131252+0000 mgr.gibba032.whopeo (mgr.24625) 937 : cluster [DBG] pgmap v519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:46.911 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:09:46.911 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (16m) 2m ago 21m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:09:46.911 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (21m) 91s ago 21m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:09:46.911 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (16m) 91s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (17m) 91s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (16m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (15m) 2m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (15m) 91s ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (15m) 2m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:09:46.912 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (21m) 2m ago 21m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:09:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (21m) 91s ago 21m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:09:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (14m) 2m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:09:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (14m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:09:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (13m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:09:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (13m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:09:46.914 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (13m) 91s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:09:46.914 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (12m) 91s ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:09:46.914 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (12m) 91s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:09:46.914 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (11m) 91s ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:09:46.914 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (21m) 91s ago 21m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:09:46.915 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 19m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:09:47.161 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:09:47.161 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:09:47.161 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:09:47.161 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:47.162 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:09:47.162 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:09:47.162 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:47.162 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:09:47.163 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:09:47.163 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:09:47.163 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:09:47.163 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:09:47.163 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:09:47.164 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:09:47.164 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:09:47.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:47 gibba011 bash[30791]: cephadm 2021-03-10T14:09:46.406189+0000 mgr.gibba032.whopeo (mgr.24625) 938 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:47.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:47 gibba011 bash[30791]: audit 2021-03-10T14:09:47.161000+0000 mon.c (mon.1) 18 : audit [DBG] from='client.? 172.21.2.111:0/4266588702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:47.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:47 gibba011 bash[31774]: cephadm 2021-03-10T14:09:46.406189+0000 mgr.gibba032.whopeo (mgr.24625) 938 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:47.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:47 gibba011 bash[31774]: audit 2021-03-10T14:09:47.161000+0000 mon.c (mon.1) 18 : audit [DBG] from='client.? 172.21.2.111:0/4266588702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:47 gibba032 bash[24004]: cephadm 2021-03-10T14:09:46.406189+0000 mgr.gibba032.whopeo (mgr.24625) 938 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:09:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:47 gibba032 bash[24004]: audit 2021-03-10T14:09:47.161000+0000 mon.c (mon.1) 18 : audit [DBG] from='client.? 172.21.2.111:0/4266588702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:09:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[30791]: audit 2021-03-10T14:09:46.671143+0000 mgr.gibba032.whopeo (mgr.24625) 939 : audit [DBG] from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[30791]: audit 2021-03-10T14:09:46.897107+0000 mgr.gibba032.whopeo (mgr.24625) 940 : audit [DBG] from='client.44691 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[30791]: cluster 2021-03-10T14:09:47.132817+0000 mgr.gibba032.whopeo (mgr.24625) 941 : cluster [DBG] pgmap v520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[31774]: audit 2021-03-10T14:09:46.671143+0000 mgr.gibba032.whopeo (mgr.24625) 939 : audit [DBG] from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[31774]: audit 2021-03-10T14:09:46.897107+0000 mgr.gibba032.whopeo (mgr.24625) 940 : audit [DBG] from='client.44691 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:48 gibba011 bash[31774]: cluster 2021-03-10T14:09:47.132817+0000 mgr.gibba032.whopeo (mgr.24625) 941 : cluster [DBG] pgmap v520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:48 gibba032 bash[24004]: audit 2021-03-10T14:09:46.671143+0000 mgr.gibba032.whopeo (mgr.24625) 939 : audit [DBG] from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:48 gibba032 bash[24004]: audit 2021-03-10T14:09:46.897107+0000 mgr.gibba032.whopeo (mgr.24625) 940 : audit [DBG] from='client.44691 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:09:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:48 gibba032 bash[24004]: cluster 2021-03-10T14:09:47.132817+0000 mgr.gibba032.whopeo (mgr.24625) 941 : cluster [DBG] pgmap v520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:50 gibba011 bash[30791]: cluster 2021-03-10T14:09:49.134201+0000 mgr.gibba032.whopeo (mgr.24625) 942 : cluster [DBG] pgmap v521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:50 gibba011 bash[31774]: cluster 2021-03-10T14:09:49.134201+0000 mgr.gibba032.whopeo (mgr.24625) 942 : cluster [DBG] pgmap v521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:50 gibba032 bash[24004]: cluster 2021-03-10T14:09:49.134201+0000 mgr.gibba032.whopeo (mgr.24625) 942 : cluster [DBG] pgmap v521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:52 gibba011 bash[30791]: cluster 2021-03-10T14:09:51.135730+0000 mgr.gibba032.whopeo (mgr.24625) 943 : cluster [DBG] pgmap v522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:52 gibba011 bash[31774]: cluster 2021-03-10T14:09:51.135730+0000 mgr.gibba032.whopeo (mgr.24625) 943 : cluster [DBG] pgmap v522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:52 gibba032 bash[24004]: cluster 2021-03-10T14:09:51.135730+0000 mgr.gibba032.whopeo (mgr.24625) 943 : cluster [DBG] pgmap v522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:54 gibba011 bash[30791]: cluster 2021-03-10T14:09:53.137492+0000 mgr.gibba032.whopeo (mgr.24625) 944 : cluster [DBG] pgmap v523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:54 gibba011 bash[31774]: cluster 2021-03-10T14:09:53.137492+0000 mgr.gibba032.whopeo (mgr.24625) 944 : cluster [DBG] pgmap v523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:54 gibba032 bash[24004]: cluster 2021-03-10T14:09:53.137492+0000 mgr.gibba032.whopeo (mgr.24625) 944 : cluster [DBG] pgmap v523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: cluster 2021-03-10T14:09:55.139073+0000 mgr.gibba032.whopeo (mgr.24625) 945 : cluster [DBG] pgmap v524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.010260+0000 mon.a (mon.0) 835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:09:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.010987+0000 mon.a (mon.0) 836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:09:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.011736+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.012408+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.105959+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:09:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:56 gibba032 bash[24004]: audit 2021-03-10T14:09:56.106573+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:09:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: cluster 2021-03-10T14:09:55.139073+0000 mgr.gibba032.whopeo (mgr.24625) 945 : cluster [DBG] pgmap v524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.010260+0000 mon.a (mon.0) 835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:09:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.010987+0000 mon.a (mon.0) 836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:09:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.011736+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.012408+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.105959+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:09:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[30791]: audit 2021-03-10T14:09:56.106573+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:09:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: cluster 2021-03-10T14:09:55.139073+0000 mgr.gibba032.whopeo (mgr.24625) 945 : cluster [DBG] pgmap v524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.010260+0000 mon.a (mon.0) 835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:09:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.010987+0000 mon.a (mon.0) 836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:09:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.011736+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.012408+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:09:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.105959+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:09:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:56 gibba011 bash[31774]: audit 2021-03-10T14:09:56.106573+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:09:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:09:58 gibba032 bash[24004]: cluster 2021-03-10T14:09:57.140476+0000 mgr.gibba032.whopeo (mgr.24625) 946 : cluster [DBG] pgmap v525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:59.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:09:58 gibba011 bash[30791]: cluster 2021-03-10T14:09:57.140476+0000 mgr.gibba032.whopeo (mgr.24625) 946 : cluster [DBG] pgmap v525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:09:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:09:58 gibba011 bash[31774]: cluster 2021-03-10T14:09:57.140476+0000 mgr.gibba032.whopeo (mgr.24625) 946 : cluster [DBG] pgmap v525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:00 gibba011 bash[30791]: debug 2021-03-10T14:09:59.998+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:10:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:00 gibba032 bash[24004]: cluster 2021-03-10T14:09:59.141926+0000 mgr.gibba032.whopeo (mgr.24625) 947 : cluster [DBG] pgmap v526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:00 gibba032 bash[24004]: cluster 2021-03-10T14:10:00.000347+0000 mon.a (mon.0) 841 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:10:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:00 gibba011 bash[30791]: cluster 2021-03-10T14:09:59.141926+0000 mgr.gibba032.whopeo (mgr.24625) 947 : cluster [DBG] pgmap v526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:00 gibba011 bash[30791]: cluster 2021-03-10T14:10:00.000347+0000 mon.a (mon.0) 841 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:10:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:00 gibba011 bash[31774]: cluster 2021-03-10T14:09:59.141926+0000 mgr.gibba032.whopeo (mgr.24625) 947 : cluster [DBG] pgmap v526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:00 gibba011 bash[31774]: cluster 2021-03-10T14:10:00.000347+0000 mon.a (mon.0) 841 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:10:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:02 gibba032 bash[24004]: cluster 2021-03-10T14:10:01.143337+0000 mgr.gibba032.whopeo (mgr.24625) 948 : cluster [DBG] pgmap v527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:02 gibba032 bash[24004]: cephadm 2021-03-10T14:10:01.422204+0000 mgr.gibba032.whopeo (mgr.24625) 949 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:02 gibba011 bash[30791]: cluster 2021-03-10T14:10:01.143337+0000 mgr.gibba032.whopeo (mgr.24625) 948 : cluster [DBG] pgmap v527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:02 gibba011 bash[30791]: cephadm 2021-03-10T14:10:01.422204+0000 mgr.gibba032.whopeo (mgr.24625) 949 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:02 gibba011 bash[31774]: cluster 2021-03-10T14:10:01.143337+0000 mgr.gibba032.whopeo (mgr.24625) 948 : cluster [DBG] pgmap v527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:02 gibba011 bash[31774]: cephadm 2021-03-10T14:10:01.422204+0000 mgr.gibba032.whopeo (mgr.24625) 949 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:04 gibba032 bash[24004]: cluster 2021-03-10T14:10:03.145004+0000 mgr.gibba032.whopeo (mgr.24625) 950 : cluster [DBG] pgmap v528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:04 gibba011 bash[30791]: cluster 2021-03-10T14:10:03.145004+0000 mgr.gibba032.whopeo (mgr.24625) 950 : cluster [DBG] pgmap v528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:04 gibba011 bash[31774]: cluster 2021-03-10T14:10:03.145004+0000 mgr.gibba032.whopeo (mgr.24625) 950 : cluster [DBG] pgmap v528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:06 gibba032 bash[24004]: cluster 2021-03-10T14:10:05.146602+0000 mgr.gibba032.whopeo (mgr.24625) 951 : cluster [DBG] pgmap v529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:06 gibba011 bash[30791]: cluster 2021-03-10T14:10:05.146602+0000 mgr.gibba032.whopeo (mgr.24625) 951 : cluster [DBG] pgmap v529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:06 gibba011 bash[31774]: cluster 2021-03-10T14:10:05.146602+0000 mgr.gibba032.whopeo (mgr.24625) 951 : cluster [DBG] pgmap v529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:08 gibba032 bash[24004]: cluster 2021-03-10T14:10:07.147954+0000 mgr.gibba032.whopeo (mgr.24625) 952 : cluster [DBG] pgmap v530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:08 gibba011 bash[30791]: cluster 2021-03-10T14:10:07.147954+0000 mgr.gibba032.whopeo (mgr.24625) 952 : cluster [DBG] pgmap v530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:08 gibba011 bash[31774]: cluster 2021-03-10T14:10:07.147954+0000 mgr.gibba032.whopeo (mgr.24625) 952 : cluster [DBG] pgmap v530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:10 gibba032 bash[24004]: cluster 2021-03-10T14:10:09.149568+0000 mgr.gibba032.whopeo (mgr.24625) 953 : cluster [DBG] pgmap v531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:10 gibba011 bash[30791]: cluster 2021-03-10T14:10:09.149568+0000 mgr.gibba032.whopeo (mgr.24625) 953 : cluster [DBG] pgmap v531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:10 gibba011 bash[31774]: cluster 2021-03-10T14:10:09.149568+0000 mgr.gibba032.whopeo (mgr.24625) 953 : cluster [DBG] pgmap v531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:13.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:12 gibba032 bash[24004]: cluster 2021-03-10T14:10:11.151069+0000 mgr.gibba032.whopeo (mgr.24625) 954 : cluster [DBG] pgmap v532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:12 gibba011 bash[30791]: cluster 2021-03-10T14:10:11.151069+0000 mgr.gibba032.whopeo (mgr.24625) 954 : cluster [DBG] pgmap v532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:12 gibba011 bash[31774]: cluster 2021-03-10T14:10:11.151069+0000 mgr.gibba032.whopeo (mgr.24625) 954 : cluster [DBG] pgmap v532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:14 gibba032 bash[24004]: cluster 2021-03-10T14:10:13.152907+0000 mgr.gibba032.whopeo (mgr.24625) 955 : cluster [DBG] pgmap v533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:14 gibba011 bash[30791]: cluster 2021-03-10T14:10:13.152907+0000 mgr.gibba032.whopeo (mgr.24625) 955 : cluster [DBG] pgmap v533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:14 gibba011 bash[31774]: cluster 2021-03-10T14:10:13.152907+0000 mgr.gibba032.whopeo (mgr.24625) 955 : cluster [DBG] pgmap v533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: cluster 2021-03-10T14:10:15.154542+0000 mgr.gibba032.whopeo (mgr.24625) 956 : cluster [DBG] pgmap v534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.648066+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:10:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.651376+0000 mon.a (mon.0) 843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:10:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.653378+0000 mon.a (mon.0) 844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.655190+0000 mon.a (mon.0) 845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.656894+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:16 gibba032 bash[24004]: audit 2021-03-10T14:10:16.660219+0000 mon.a (mon.0) 847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: cluster 2021-03-10T14:10:15.154542+0000 mgr.gibba032.whopeo (mgr.24625) 956 : cluster [DBG] pgmap v534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.648066+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:10:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.651376+0000 mon.a (mon.0) 843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:10:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.653378+0000 mon.a (mon.0) 844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.655190+0000 mon.a (mon.0) 845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.656894+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[30791]: audit 2021-03-10T14:10:16.660219+0000 mon.a (mon.0) 847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: cluster 2021-03-10T14:10:15.154542+0000 mgr.gibba032.whopeo (mgr.24625) 956 : cluster [DBG] pgmap v534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.648066+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:10:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.651376+0000 mon.a (mon.0) 843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:10:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.653378+0000 mon.a (mon.0) 844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.655190+0000 mon.a (mon.0) 845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.656894+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:16 gibba011 bash[31774]: audit 2021-03-10T14:10:16.660219+0000 mon.a (mon.0) 847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:17.403 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:10:17.631 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:10:17.632 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (16m) 3m ago 22m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:10:17.632 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (22m) 2m ago 22m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:10:17.632 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (17m) 2m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:10:17.632 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (17m) 2m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:10:17.632 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (16m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (16m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (16m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (16m) 3m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (22m) 3m ago 22m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (22m) 2m ago 22m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:10:17.633 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (15m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:10:17.634 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (14m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:10:17.634 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (14m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:10:17.634 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (13m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:10:17.634 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (13m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:10:17.635 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (13m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:10:17.635 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (12m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:10:17.635 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (12m) 2m ago 22m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:10:17.635 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (21m) 2m ago 22m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:10:17.636 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 20m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:10:17.882 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:10:17.882 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:10:17.882 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:10:17.883 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:17.883 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:10:17.883 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:10:17.883 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:17.883 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:10:17.884 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:10:17.884 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:17.884 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:10:17.884 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:10:17.884 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:10:17.885 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:10:17.885 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:10:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.438996+0000 mgr.gibba032.whopeo (mgr.24625) 957 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:10:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.440594+0000 mgr.gibba032.whopeo (mgr.24625) 958 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:10:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.648466+0000 mgr.gibba032.whopeo (mgr.24625) 959 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:10:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.650701+0000 mgr.gibba032.whopeo (mgr.24625) 960 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:10:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.652493+0000 mgr.gibba032.whopeo (mgr.24625) 961 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:10:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.652528+0000 mgr.gibba032.whopeo (mgr.24625) 962 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:10:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.654199+0000 mgr.gibba032.whopeo (mgr.24625) 963 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:10:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.654275+0000 mgr.gibba032.whopeo (mgr.24625) 964 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:10:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.655739+0000 mgr.gibba032.whopeo (mgr.24625) 965 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:10:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.657455+0000 mgr.gibba032.whopeo (mgr.24625) 966 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:10:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.657495+0000 mgr.gibba032.whopeo (mgr.24625) 967 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:10:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.659254+0000 mgr.gibba032.whopeo (mgr.24625) 968 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:10:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.659829+0000 mgr.gibba032.whopeo (mgr.24625) 969 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:10:18.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:17 gibba032 bash[24004]: audit 2021-03-10T14:10:16.662315+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:18.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.438996+0000 mgr.gibba032.whopeo (mgr.24625) 957 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:10:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.440594+0000 mgr.gibba032.whopeo (mgr.24625) 958 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:10:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.648466+0000 mgr.gibba032.whopeo (mgr.24625) 959 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:10:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.650701+0000 mgr.gibba032.whopeo (mgr.24625) 960 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:10:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.652493+0000 mgr.gibba032.whopeo (mgr.24625) 961 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:10:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.652528+0000 mgr.gibba032.whopeo (mgr.24625) 962 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:10:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.654199+0000 mgr.gibba032.whopeo (mgr.24625) 963 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:10:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.654275+0000 mgr.gibba032.whopeo (mgr.24625) 964 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:10:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.655739+0000 mgr.gibba032.whopeo (mgr.24625) 965 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:10:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.657455+0000 mgr.gibba032.whopeo (mgr.24625) 966 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:10:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.657495+0000 mgr.gibba032.whopeo (mgr.24625) 967 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:10:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.659254+0000 mgr.gibba032.whopeo (mgr.24625) 968 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:10:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.659829+0000 mgr.gibba032.whopeo (mgr.24625) 969 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:10:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[30791]: audit 2021-03-10T14:10:16.662315+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.438996+0000 mgr.gibba032.whopeo (mgr.24625) 957 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:10:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.440594+0000 mgr.gibba032.whopeo (mgr.24625) 958 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:10:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.648466+0000 mgr.gibba032.whopeo (mgr.24625) 959 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:10:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.650701+0000 mgr.gibba032.whopeo (mgr.24625) 960 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:10:18.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.652493+0000 mgr.gibba032.whopeo (mgr.24625) 961 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:10:18.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.652528+0000 mgr.gibba032.whopeo (mgr.24625) 962 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:10:18.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.654199+0000 mgr.gibba032.whopeo (mgr.24625) 963 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:10:18.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.654275+0000 mgr.gibba032.whopeo (mgr.24625) 964 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:10:18.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.655739+0000 mgr.gibba032.whopeo (mgr.24625) 965 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:10:18.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.657455+0000 mgr.gibba032.whopeo (mgr.24625) 966 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:10:18.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.657495+0000 mgr.gibba032.whopeo (mgr.24625) 967 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:10:18.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.659254+0000 mgr.gibba032.whopeo (mgr.24625) 968 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:10:18.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.659829+0000 mgr.gibba032.whopeo (mgr.24625) 969 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:10:18.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:17 gibba011 bash[31774]: audit 2021-03-10T14:10:16.662315+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.662655+0000 mgr.gibba032.whopeo (mgr.24625) 970 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:10:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.662696+0000 mgr.gibba032.whopeo (mgr.24625) 971 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:10:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: cephadm 2021-03-10T14:10:16.662809+0000 mgr.gibba032.whopeo (mgr.24625) 972 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: cluster 2021-03-10T14:10:17.156140+0000 mgr.gibba032.whopeo (mgr.24625) 973 : cluster [DBG] pgmap v535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: audit 2021-03-10T14:10:17.392369+0000 mgr.gibba032.whopeo (mgr.24625) 974 : audit [DBG] from='client.54688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: audit 2021-03-10T14:10:17.617576+0000 mgr.gibba032.whopeo (mgr.24625) 975 : audit [DBG] from='client.54694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:18 gibba032 bash[24004]: audit 2021-03-10T14:10:17.882203+0000 mon.c (mon.1) 19 : audit [DBG] from='client.? 172.21.2.111:0/2180707597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.662655+0000 mgr.gibba032.whopeo (mgr.24625) 970 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:10:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.662696+0000 mgr.gibba032.whopeo (mgr.24625) 971 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:10:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: cephadm 2021-03-10T14:10:16.662809+0000 mgr.gibba032.whopeo (mgr.24625) 972 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: cluster 2021-03-10T14:10:17.156140+0000 mgr.gibba032.whopeo (mgr.24625) 973 : cluster [DBG] pgmap v535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: audit 2021-03-10T14:10:17.392369+0000 mgr.gibba032.whopeo (mgr.24625) 974 : audit [DBG] from='client.54688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: audit 2021-03-10T14:10:17.617576+0000 mgr.gibba032.whopeo (mgr.24625) 975 : audit [DBG] from='client.54694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[30791]: audit 2021-03-10T14:10:17.882203+0000 mon.c (mon.1) 19 : audit [DBG] from='client.? 172.21.2.111:0/2180707597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.662655+0000 mgr.gibba032.whopeo (mgr.24625) 970 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:10:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.662696+0000 mgr.gibba032.whopeo (mgr.24625) 971 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:10:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: cephadm 2021-03-10T14:10:16.662809+0000 mgr.gibba032.whopeo (mgr.24625) 972 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: cluster 2021-03-10T14:10:17.156140+0000 mgr.gibba032.whopeo (mgr.24625) 973 : cluster [DBG] pgmap v535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: audit 2021-03-10T14:10:17.392369+0000 mgr.gibba032.whopeo (mgr.24625) 974 : audit [DBG] from='client.54688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: audit 2021-03-10T14:10:17.617576+0000 mgr.gibba032.whopeo (mgr.24625) 975 : audit [DBG] from='client.54694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:19.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:18 gibba011 bash[31774]: audit 2021-03-10T14:10:17.882203+0000 mon.c (mon.1) 19 : audit [DBG] from='client.? 172.21.2.111:0/2180707597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:20.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:19 gibba032 bash[24004]: cluster 2021-03-10T14:10:19.157663+0000 mgr.gibba032.whopeo (mgr.24625) 976 : cluster [DBG] pgmap v536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:19 gibba011 bash[30791]: cluster 2021-03-10T14:10:19.157663+0000 mgr.gibba032.whopeo (mgr.24625) 976 : cluster [DBG] pgmap v536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:19 gibba011 bash[31774]: cluster 2021-03-10T14:10:19.157663+0000 mgr.gibba032.whopeo (mgr.24625) 976 : cluster [DBG] pgmap v536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:22 gibba011 bash[30791]: cluster 2021-03-10T14:10:21.159179+0000 mgr.gibba032.whopeo (mgr.24625) 977 : cluster [DBG] pgmap v537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:22 gibba011 bash[31774]: cluster 2021-03-10T14:10:21.159179+0000 mgr.gibba032.whopeo (mgr.24625) 977 : cluster [DBG] pgmap v537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:22 gibba032 bash[24004]: cluster 2021-03-10T14:10:21.159179+0000 mgr.gibba032.whopeo (mgr.24625) 977 : cluster [DBG] pgmap v537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:24 gibba011 bash[30791]: cluster 2021-03-10T14:10:23.160891+0000 mgr.gibba032.whopeo (mgr.24625) 978 : cluster [DBG] pgmap v538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:24 gibba011 bash[31774]: cluster 2021-03-10T14:10:23.160891+0000 mgr.gibba032.whopeo (mgr.24625) 978 : cluster [DBG] pgmap v538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:24 gibba032 bash[24004]: cluster 2021-03-10T14:10:23.160891+0000 mgr.gibba032.whopeo (mgr.24625) 978 : cluster [DBG] pgmap v538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:26 gibba011 bash[30791]: cluster 2021-03-10T14:10:25.162462+0000 mgr.gibba032.whopeo (mgr.24625) 979 : cluster [DBG] pgmap v539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:26 gibba011 bash[31774]: cluster 2021-03-10T14:10:25.162462+0000 mgr.gibba032.whopeo (mgr.24625) 979 : cluster [DBG] pgmap v539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:26 gibba032 bash[24004]: cluster 2021-03-10T14:10:25.162462+0000 mgr.gibba032.whopeo (mgr.24625) 979 : cluster [DBG] pgmap v539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:28 gibba032 bash[24004]: cluster 2021-03-10T14:10:27.162958+0000 mgr.gibba032.whopeo (mgr.24625) 980 : cluster [DBG] pgmap v540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:28 gibba011 bash[30791]: cluster 2021-03-10T14:10:27.162958+0000 mgr.gibba032.whopeo (mgr.24625) 980 : cluster [DBG] pgmap v540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:28 gibba011 bash[31774]: cluster 2021-03-10T14:10:27.162958+0000 mgr.gibba032.whopeo (mgr.24625) 980 : cluster [DBG] pgmap v540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:30 gibba032 bash[24004]: cluster 2021-03-10T14:10:29.164127+0000 mgr.gibba032.whopeo (mgr.24625) 981 : cluster [DBG] pgmap v541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:30 gibba011 bash[30791]: cluster 2021-03-10T14:10:29.164127+0000 mgr.gibba032.whopeo (mgr.24625) 981 : cluster [DBG] pgmap v541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:30 gibba011 bash[31774]: cluster 2021-03-10T14:10:29.164127+0000 mgr.gibba032.whopeo (mgr.24625) 981 : cluster [DBG] pgmap v541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:32 gibba032 bash[24004]: cluster 2021-03-10T14:10:31.165597+0000 mgr.gibba032.whopeo (mgr.24625) 982 : cluster [DBG] pgmap v542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:32 gibba011 bash[30791]: cluster 2021-03-10T14:10:31.165597+0000 mgr.gibba032.whopeo (mgr.24625) 982 : cluster [DBG] pgmap v542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:32 gibba011 bash[31774]: cluster 2021-03-10T14:10:31.165597+0000 mgr.gibba032.whopeo (mgr.24625) 982 : cluster [DBG] pgmap v542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:33 gibba032 bash[24004]: cephadm 2021-03-10T14:10:31.667455+0000 mgr.gibba032.whopeo (mgr.24625) 983 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:33 gibba011 bash[30791]: cephadm 2021-03-10T14:10:31.667455+0000 mgr.gibba032.whopeo (mgr.24625) 983 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:33.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:33 gibba011 bash[31774]: cephadm 2021-03-10T14:10:31.667455+0000 mgr.gibba032.whopeo (mgr.24625) 983 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:34 gibba032 bash[24004]: cluster 2021-03-10T14:10:33.167334+0000 mgr.gibba032.whopeo (mgr.24625) 984 : cluster [DBG] pgmap v543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:34.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:34 gibba011 bash[30791]: cluster 2021-03-10T14:10:33.167334+0000 mgr.gibba032.whopeo (mgr.24625) 984 : cluster [DBG] pgmap v543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:34 gibba011 bash[31774]: cluster 2021-03-10T14:10:33.167334+0000 mgr.gibba032.whopeo (mgr.24625) 984 : cluster [DBG] pgmap v543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:36 gibba032 bash[24004]: cluster 2021-03-10T14:10:35.168887+0000 mgr.gibba032.whopeo (mgr.24625) 985 : cluster [DBG] pgmap v544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:36 gibba011 bash[30791]: cluster 2021-03-10T14:10:35.168887+0000 mgr.gibba032.whopeo (mgr.24625) 985 : cluster [DBG] pgmap v544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:36 gibba011 bash[31774]: cluster 2021-03-10T14:10:35.168887+0000 mgr.gibba032.whopeo (mgr.24625) 985 : cluster [DBG] pgmap v544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:38 gibba032 bash[24004]: cluster 2021-03-10T14:10:37.170529+0000 mgr.gibba032.whopeo (mgr.24625) 986 : cluster [DBG] pgmap v545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:38.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:38 gibba011 bash[30791]: cluster 2021-03-10T14:10:37.170529+0000 mgr.gibba032.whopeo (mgr.24625) 986 : cluster [DBG] pgmap v545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:38 gibba011 bash[31774]: cluster 2021-03-10T14:10:37.170529+0000 mgr.gibba032.whopeo (mgr.24625) 986 : cluster [DBG] pgmap v545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:40 gibba032 bash[24004]: cluster 2021-03-10T14:10:39.171965+0000 mgr.gibba032.whopeo (mgr.24625) 987 : cluster [DBG] pgmap v546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:40 gibba011 bash[30791]: cluster 2021-03-10T14:10:39.171965+0000 mgr.gibba032.whopeo (mgr.24625) 987 : cluster [DBG] pgmap v546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:40 gibba011 bash[31774]: cluster 2021-03-10T14:10:39.171965+0000 mgr.gibba032.whopeo (mgr.24625) 987 : cluster [DBG] pgmap v546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:42 gibba032 bash[24004]: cluster 2021-03-10T14:10:41.173733+0000 mgr.gibba032.whopeo (mgr.24625) 988 : cluster [DBG] pgmap v547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:42 gibba011 bash[30791]: cluster 2021-03-10T14:10:41.173733+0000 mgr.gibba032.whopeo (mgr.24625) 988 : cluster [DBG] pgmap v547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:42 gibba011 bash[31774]: cluster 2021-03-10T14:10:41.173733+0000 mgr.gibba032.whopeo (mgr.24625) 988 : cluster [DBG] pgmap v547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:44 gibba032 bash[24004]: cluster 2021-03-10T14:10:43.175370+0000 mgr.gibba032.whopeo (mgr.24625) 989 : cluster [DBG] pgmap v548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:44 gibba011 bash[30791]: cluster 2021-03-10T14:10:43.175370+0000 mgr.gibba032.whopeo (mgr.24625) 989 : cluster [DBG] pgmap v548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:44 gibba011 bash[31774]: cluster 2021-03-10T14:10:43.175370+0000 mgr.gibba032.whopeo (mgr.24625) 989 : cluster [DBG] pgmap v548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:46.686 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:46 gibba032 bash[24004]: cluster 2021-03-10T14:10:45.176729+0000 mgr.gibba032.whopeo (mgr.24625) 990 : cluster [DBG] pgmap v549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:46.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:46 gibba011 bash[30791]: cluster 2021-03-10T14:10:45.176729+0000 mgr.gibba032.whopeo (mgr.24625) 990 : cluster [DBG] pgmap v549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:46 gibba011 bash[31774]: cluster 2021-03-10T14:10:45.176729+0000 mgr.gibba032.whopeo (mgr.24625) 990 : cluster [DBG] pgmap v549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:48.127 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:10:48.357 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:10:48.358 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (17m) 3m ago 22m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:10:48.358 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (22m) 2m ago 22m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:10:48.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (17m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:10:48.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (18m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:10:48.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (17m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:10:48.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (16m) 3m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:10:48.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (16m) 2m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:10:48.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (16m) 3m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:10:48.359 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (22m) 3m ago 22m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:10:48.360 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (22m) 2m ago 22m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:10:48.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (15m) 3m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:10:48.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (15m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:10:48.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (14m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:10:48.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (14m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:10:48.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (14m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:10:48.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (13m) 2m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:10:48.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (13m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:10:48.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (12m) 2m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:10:48.361 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (22m) 2m ago 22m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:10:48.362 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 20m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:10:48.608 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:10:48.609 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:10:48.609 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:10:48.609 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:48.610 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:10:48.610 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:10:48.611 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:48.611 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:10:48.611 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:10:48.611 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:10:48.612 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:10:48.612 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:10:48.612 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:10:48.612 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:10:48.612 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:10:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:48 gibba032 bash[24004]: cephadm 2021-03-10T14:10:46.683369+0000 mgr.gibba032.whopeo (mgr.24625) 991 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:48 gibba032 bash[24004]: cluster 2021-03-10T14:10:47.178150+0000 mgr.gibba032.whopeo (mgr.24625) 992 : cluster [DBG] pgmap v550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:48 gibba011 bash[30791]: cephadm 2021-03-10T14:10:46.683369+0000 mgr.gibba032.whopeo (mgr.24625) 991 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:48 gibba011 bash[30791]: cluster 2021-03-10T14:10:47.178150+0000 mgr.gibba032.whopeo (mgr.24625) 992 : cluster [DBG] pgmap v550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:48 gibba011 bash[31774]: cephadm 2021-03-10T14:10:46.683369+0000 mgr.gibba032.whopeo (mgr.24625) 991 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:10:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:48 gibba011 bash[31774]: cluster 2021-03-10T14:10:47.178150+0000 mgr.gibba032.whopeo (mgr.24625) 992 : cluster [DBG] pgmap v550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:49.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:49 gibba032 bash[24004]: audit 2021-03-10T14:10:48.116800+0000 mgr.gibba032.whopeo (mgr.24625) 993 : audit [DBG] from='client.54706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:49 gibba032 bash[24004]: audit 2021-03-10T14:10:48.343505+0000 mgr.gibba032.whopeo (mgr.24625) 994 : audit [DBG] from='client.54712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:49 gibba032 bash[24004]: audit 2021-03-10T14:10:48.608726+0000 mon.a (mon.0) 849 : audit [DBG] from='client.? 172.21.2.111:0/639183207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[30791]: audit 2021-03-10T14:10:48.116800+0000 mgr.gibba032.whopeo (mgr.24625) 993 : audit [DBG] from='client.54706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[30791]: audit 2021-03-10T14:10:48.343505+0000 mgr.gibba032.whopeo (mgr.24625) 994 : audit [DBG] from='client.54712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[30791]: audit 2021-03-10T14:10:48.608726+0000 mon.a (mon.0) 849 : audit [DBG] from='client.? 172.21.2.111:0/639183207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:49.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[31774]: audit 2021-03-10T14:10:48.116800+0000 mgr.gibba032.whopeo (mgr.24625) 993 : audit [DBG] from='client.54706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[31774]: audit 2021-03-10T14:10:48.343505+0000 mgr.gibba032.whopeo (mgr.24625) 994 : audit [DBG] from='client.54712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:10:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:49 gibba011 bash[31774]: audit 2021-03-10T14:10:48.608726+0000 mon.a (mon.0) 849 : audit [DBG] from='client.? 172.21.2.111:0/639183207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:10:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:50 gibba032 bash[24004]: cluster 2021-03-10T14:10:49.179664+0000 mgr.gibba032.whopeo (mgr.24625) 995 : cluster [DBG] pgmap v551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:50 gibba011 bash[30791]: cluster 2021-03-10T14:10:49.179664+0000 mgr.gibba032.whopeo (mgr.24625) 995 : cluster [DBG] pgmap v551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:50 gibba011 bash[31774]: cluster 2021-03-10T14:10:49.179664+0000 mgr.gibba032.whopeo (mgr.24625) 995 : cluster [DBG] pgmap v551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:52 gibba032 bash[24004]: cluster 2021-03-10T14:10:51.181191+0000 mgr.gibba032.whopeo (mgr.24625) 996 : cluster [DBG] pgmap v552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:52 gibba011 bash[30791]: cluster 2021-03-10T14:10:51.181191+0000 mgr.gibba032.whopeo (mgr.24625) 996 : cluster [DBG] pgmap v552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:52 gibba011 bash[31774]: cluster 2021-03-10T14:10:51.181191+0000 mgr.gibba032.whopeo (mgr.24625) 996 : cluster [DBG] pgmap v552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:54 gibba032 bash[24004]: cluster 2021-03-10T14:10:53.183029+0000 mgr.gibba032.whopeo (mgr.24625) 997 : cluster [DBG] pgmap v553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:54 gibba011 bash[30791]: cluster 2021-03-10T14:10:53.183029+0000 mgr.gibba032.whopeo (mgr.24625) 997 : cluster [DBG] pgmap v553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:54 gibba011 bash[31774]: cluster 2021-03-10T14:10:53.183029+0000 mgr.gibba032.whopeo (mgr.24625) 997 : cluster [DBG] pgmap v553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: cluster 2021-03-10T14:10:55.184393+0000 mgr.gibba032.whopeo (mgr.24625) 998 : cluster [DBG] pgmap v554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.019860+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:10:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.020967+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.022020+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.023046+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.113578+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:10:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:56 gibba032 bash[24004]: audit 2021-03-10T14:10:56.114511+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:10:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: cluster 2021-03-10T14:10:55.184393+0000 mgr.gibba032.whopeo (mgr.24625) 998 : cluster [DBG] pgmap v554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.019860+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:10:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.020967+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.022020+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.023046+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.113578+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:10:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[30791]: audit 2021-03-10T14:10:56.114511+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:10:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: cluster 2021-03-10T14:10:55.184393+0000 mgr.gibba032.whopeo (mgr.24625) 998 : cluster [DBG] pgmap v554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.019860+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:10:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.020967+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.022020+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.023046+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:10:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.113578+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:10:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:56 gibba011 bash[31774]: audit 2021-03-10T14:10:56.114511+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:10:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:10:58 gibba032 bash[24004]: cluster 2021-03-10T14:10:57.186069+0000 mgr.gibba032.whopeo (mgr.24625) 999 : cluster [DBG] pgmap v555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:10:58 gibba011 bash[30791]: cluster 2021-03-10T14:10:57.186069+0000 mgr.gibba032.whopeo (mgr.24625) 999 : cluster [DBG] pgmap v555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:10:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:10:58 gibba011 bash[31774]: cluster 2021-03-10T14:10:57.186069+0000 mgr.gibba032.whopeo (mgr.24625) 999 : cluster [DBG] pgmap v555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:00 gibba032 bash[24004]: cluster 2021-03-10T14:10:59.187677+0000 mgr.gibba032.whopeo (mgr.24625) 1000 : cluster [DBG] pgmap v556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:00 gibba011 bash[30791]: cluster 2021-03-10T14:10:59.187677+0000 mgr.gibba032.whopeo (mgr.24625) 1000 : cluster [DBG] pgmap v556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:00 gibba011 bash[31774]: cluster 2021-03-10T14:10:59.187677+0000 mgr.gibba032.whopeo (mgr.24625) 1000 : cluster [DBG] pgmap v556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:02 gibba032 bash[24004]: cluster 2021-03-10T14:11:01.189242+0000 mgr.gibba032.whopeo (mgr.24625) 1001 : cluster [DBG] pgmap v557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:02 gibba011 bash[30791]: cluster 2021-03-10T14:11:01.189242+0000 mgr.gibba032.whopeo (mgr.24625) 1001 : cluster [DBG] pgmap v557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:02 gibba011 bash[31774]: cluster 2021-03-10T14:11:01.189242+0000 mgr.gibba032.whopeo (mgr.24625) 1001 : cluster [DBG] pgmap v557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:03 gibba032 bash[24004]: cephadm 2021-03-10T14:11:01.699300+0000 mgr.gibba032.whopeo (mgr.24625) 1002 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:03 gibba011 bash[30791]: cephadm 2021-03-10T14:11:01.699300+0000 mgr.gibba032.whopeo (mgr.24625) 1002 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:03.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:03 gibba011 bash[31774]: cephadm 2021-03-10T14:11:01.699300+0000 mgr.gibba032.whopeo (mgr.24625) 1002 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:04.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:04 gibba011 bash[30791]: cluster 2021-03-10T14:11:03.191197+0000 mgr.gibba032.whopeo (mgr.24625) 1003 : cluster [DBG] pgmap v558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:04 gibba011 bash[31774]: cluster 2021-03-10T14:11:03.191197+0000 mgr.gibba032.whopeo (mgr.24625) 1003 : cluster [DBG] pgmap v558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:04 gibba032 bash[24004]: cluster 2021-03-10T14:11:03.191197+0000 mgr.gibba032.whopeo (mgr.24625) 1003 : cluster [DBG] pgmap v558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:06.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:06 gibba011 bash[30791]: cluster 2021-03-10T14:11:05.192998+0000 mgr.gibba032.whopeo (mgr.24625) 1004 : cluster [DBG] pgmap v559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:06.889 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:06 gibba011 bash[31774]: cluster 2021-03-10T14:11:05.192998+0000 mgr.gibba032.whopeo (mgr.24625) 1004 : cluster [DBG] pgmap v559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:06 gibba032 bash[24004]: cluster 2021-03-10T14:11:05.192998+0000 mgr.gibba032.whopeo (mgr.24625) 1004 : cluster [DBG] pgmap v559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:08 gibba011 bash[30791]: cluster 2021-03-10T14:11:07.194848+0000 mgr.gibba032.whopeo (mgr.24625) 1005 : cluster [DBG] pgmap v560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:08 gibba011 bash[31774]: cluster 2021-03-10T14:11:07.194848+0000 mgr.gibba032.whopeo (mgr.24625) 1005 : cluster [DBG] pgmap v560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:08 gibba032 bash[24004]: cluster 2021-03-10T14:11:07.194848+0000 mgr.gibba032.whopeo (mgr.24625) 1005 : cluster [DBG] pgmap v560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:10 gibba011 bash[30791]: cluster 2021-03-10T14:11:09.196301+0000 mgr.gibba032.whopeo (mgr.24625) 1006 : cluster [DBG] pgmap v561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:10 gibba011 bash[31774]: cluster 2021-03-10T14:11:09.196301+0000 mgr.gibba032.whopeo (mgr.24625) 1006 : cluster [DBG] pgmap v561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:10 gibba032 bash[24004]: cluster 2021-03-10T14:11:09.196301+0000 mgr.gibba032.whopeo (mgr.24625) 1006 : cluster [DBG] pgmap v561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:12 gibba011 bash[30791]: cluster 2021-03-10T14:11:11.197833+0000 mgr.gibba032.whopeo (mgr.24625) 1007 : cluster [DBG] pgmap v562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:12 gibba011 bash[31774]: cluster 2021-03-10T14:11:11.197833+0000 mgr.gibba032.whopeo (mgr.24625) 1007 : cluster [DBG] pgmap v562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:13.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:12 gibba032 bash[24004]: cluster 2021-03-10T14:11:11.197833+0000 mgr.gibba032.whopeo (mgr.24625) 1007 : cluster [DBG] pgmap v562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:14 gibba011 bash[30791]: cluster 2021-03-10T14:11:13.199474+0000 mgr.gibba032.whopeo (mgr.24625) 1008 : cluster [DBG] pgmap v563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:14 gibba011 bash[31774]: cluster 2021-03-10T14:11:13.199474+0000 mgr.gibba032.whopeo (mgr.24625) 1008 : cluster [DBG] pgmap v563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:14 gibba032 bash[24004]: cluster 2021-03-10T14:11:13.199474+0000 mgr.gibba032.whopeo (mgr.24625) 1008 : cluster [DBG] pgmap v563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:16 gibba011 bash[30791]: cluster 2021-03-10T14:11:15.200964+0000 mgr.gibba032.whopeo (mgr.24625) 1009 : cluster [DBG] pgmap v564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:16 gibba011 bash[31774]: cluster 2021-03-10T14:11:15.200964+0000 mgr.gibba032.whopeo (mgr.24625) 1009 : cluster [DBG] pgmap v564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:16.961 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:16 gibba032 bash[24004]: cluster 2021-03-10T14:11:15.200964+0000 mgr.gibba032.whopeo (mgr.24625) 1009 : cluster [DBG] pgmap v564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.715961+0000 mgr.gibba032.whopeo (mgr.24625) 1010 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:11:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.717556+0000 mgr.gibba032.whopeo (mgr.24625) 1011 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:11:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.934061+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:11:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.934299+0000 mgr.gibba032.whopeo (mgr.24625) 1012 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:11:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.936177+0000 mgr.gibba032.whopeo (mgr.24625) 1013 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:11:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.937302+0000 mon.a (mon.0) 857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:11:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.937758+0000 mgr.gibba032.whopeo (mgr.24625) 1014 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:11:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.937797+0000 mgr.gibba032.whopeo (mgr.24625) 1015 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:11:18.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.938818+0000 mon.a (mon.0) 858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.940487+0000 mon.a (mon.0) 859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.940840+0000 mgr.gibba032.whopeo (mgr.24625) 1016 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:11:18.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.941328+0000 mgr.gibba032.whopeo (mgr.24625) 1017 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:11:18.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.944152+0000 mgr.gibba032.whopeo (mgr.24625) 1018 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:11:18.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.944293+0000 mon.a (mon.0) 860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.947239+0000 mgr.gibba032.whopeo (mgr.24625) 1019 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:11:18.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.947781+0000 mgr.gibba032.whopeo (mgr.24625) 1020 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:11:18.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.952111+0000 mon.a (mon.0) 861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.952185+0000 mgr.gibba032.whopeo (mgr.24625) 1021 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:11:18.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.954134+0000 mgr.gibba032.whopeo (mgr.24625) 1022 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:11:18.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: audit 2021-03-10T14:11:16.957357+0000 mon.a (mon.0) 862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.957457+0000 mgr.gibba032.whopeo (mgr.24625) 1023 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:11:18.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.957798+0000 mgr.gibba032.whopeo (mgr.24625) 1024 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:11:18.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cephadm 2021-03-10T14:11:16.958637+0000 mgr.gibba032.whopeo (mgr.24625) 1025 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:18.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:17 gibba032 bash[24004]: cluster 2021-03-10T14:11:17.202651+0000 mgr.gibba032.whopeo (mgr.24625) 1026 : cluster [DBG] pgmap v565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.715961+0000 mgr.gibba032.whopeo (mgr.24625) 1010 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:11:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.717556+0000 mgr.gibba032.whopeo (mgr.24625) 1011 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:11:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.934061+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:11:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.934299+0000 mgr.gibba032.whopeo (mgr.24625) 1012 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:11:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.936177+0000 mgr.gibba032.whopeo (mgr.24625) 1013 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:11:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.937302+0000 mon.a (mon.0) 857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:11:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.937758+0000 mgr.gibba032.whopeo (mgr.24625) 1014 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:11:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.937797+0000 mgr.gibba032.whopeo (mgr.24625) 1015 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:11:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.938818+0000 mon.a (mon.0) 858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.940487+0000 mon.a (mon.0) 859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.940840+0000 mgr.gibba032.whopeo (mgr.24625) 1016 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:11:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.941328+0000 mgr.gibba032.whopeo (mgr.24625) 1017 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:11:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.944152+0000 mgr.gibba032.whopeo (mgr.24625) 1018 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:11:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.944293+0000 mon.a (mon.0) 860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.947239+0000 mgr.gibba032.whopeo (mgr.24625) 1019 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:11:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.947781+0000 mgr.gibba032.whopeo (mgr.24625) 1020 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:11:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.952111+0000 mon.a (mon.0) 861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.952185+0000 mgr.gibba032.whopeo (mgr.24625) 1021 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:11:18.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.954134+0000 mgr.gibba032.whopeo (mgr.24625) 1022 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:11:18.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: audit 2021-03-10T14:11:16.957357+0000 mon.a (mon.0) 862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.957457+0000 mgr.gibba032.whopeo (mgr.24625) 1023 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:11:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.715961+0000 mgr.gibba032.whopeo (mgr.24625) 1010 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:11:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.717556+0000 mgr.gibba032.whopeo (mgr.24625) 1011 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:11:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.934061+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:11:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.934299+0000 mgr.gibba032.whopeo (mgr.24625) 1012 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:11:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.936177+0000 mgr.gibba032.whopeo (mgr.24625) 1013 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:11:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.937302+0000 mon.a (mon.0) 857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:11:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.937758+0000 mgr.gibba032.whopeo (mgr.24625) 1014 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:11:18.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.937797+0000 mgr.gibba032.whopeo (mgr.24625) 1015 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:11:18.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.938818+0000 mon.a (mon.0) 858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.940487+0000 mon.a (mon.0) 859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.940840+0000 mgr.gibba032.whopeo (mgr.24625) 1016 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:11:18.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.941328+0000 mgr.gibba032.whopeo (mgr.24625) 1017 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:11:18.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.944152+0000 mgr.gibba032.whopeo (mgr.24625) 1018 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:11:18.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.944293+0000 mon.a (mon.0) 860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.947239+0000 mgr.gibba032.whopeo (mgr.24625) 1019 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:11:18.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.947781+0000 mgr.gibba032.whopeo (mgr.24625) 1020 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:11:18.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.952111+0000 mon.a (mon.0) 861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.952185+0000 mgr.gibba032.whopeo (mgr.24625) 1021 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:11:18.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.954134+0000 mgr.gibba032.whopeo (mgr.24625) 1022 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:11:18.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: audit 2021-03-10T14:11:16.957357+0000 mon.a (mon.0) 862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:18.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.957457+0000 mgr.gibba032.whopeo (mgr.24625) 1023 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:11:18.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.957798+0000 mgr.gibba032.whopeo (mgr.24625) 1024 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:11:18.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cephadm 2021-03-10T14:11:16.958637+0000 mgr.gibba032.whopeo (mgr.24625) 1025 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:18.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[30791]: cluster 2021-03-10T14:11:17.202651+0000 mgr.gibba032.whopeo (mgr.24625) 1026 : cluster [DBG] pgmap v565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:18.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.957798+0000 mgr.gibba032.whopeo (mgr.24625) 1024 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:11:18.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cephadm 2021-03-10T14:11:16.958637+0000 mgr.gibba032.whopeo (mgr.24625) 1025 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:18.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:17 gibba011 bash[31774]: cluster 2021-03-10T14:11:17.202651+0000 mgr.gibba032.whopeo (mgr.24625) 1026 : cluster [DBG] pgmap v565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:18.854 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:11:19.079 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:11:19.079 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (17m) 4m ago 23m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:11:19.080 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (23m) 3m ago 23m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:11:19.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (18m) 3m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:11:19.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (18m) 3m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:11:19.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (17m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:11:19.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (17m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:11:19.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (17m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:11:19.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (17m) 4m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:11:19.081 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (23m) 4m ago 23m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:11:19.081 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (23m) 3m ago 23m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:11:19.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (16m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:11:19.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (15m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:11:19.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (15m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:11:19.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (14m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:11:19.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (14m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:11:19.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (14m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:11:19.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (13m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:11:19.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (13m) 3m ago 23m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:11:19.084 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (22m) 3m ago 23m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:11:19.084 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 21m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:11:19.332 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:11:19.333 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:11:19.333 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:11:19.333 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:19.333 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:11:19.334 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:11:19.334 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:19.334 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:11:19.334 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:11:19.335 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:19.335 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:11:19.335 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:11:19.335 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:11:19.335 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:11:19.336 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:11:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[30791]: audit 2021-03-10T14:11:18.842459+0000 mgr.gibba032.whopeo (mgr.24625) 1027 : audit [DBG] from='client.54724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[30791]: audit 2021-03-10T14:11:19.064920+0000 mgr.gibba032.whopeo (mgr.24625) 1028 : audit [DBG] from='client.54730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[30791]: cluster 2021-03-10T14:11:19.204089+0000 mgr.gibba032.whopeo (mgr.24625) 1029 : cluster [DBG] pgmap v566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[30791]: audit 2021-03-10T14:11:19.332801+0000 mon.c (mon.1) 20 : audit [DBG] from='client.? 172.21.2.111:0/993695408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[31774]: audit 2021-03-10T14:11:18.842459+0000 mgr.gibba032.whopeo (mgr.24625) 1027 : audit [DBG] from='client.54724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[31774]: audit 2021-03-10T14:11:19.064920+0000 mgr.gibba032.whopeo (mgr.24625) 1028 : audit [DBG] from='client.54730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[31774]: cluster 2021-03-10T14:11:19.204089+0000 mgr.gibba032.whopeo (mgr.24625) 1029 : cluster [DBG] pgmap v566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:20 gibba011 bash[31774]: audit 2021-03-10T14:11:19.332801+0000 mon.c (mon.1) 20 : audit [DBG] from='client.? 172.21.2.111:0/993695408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:20 gibba032 bash[24004]: audit 2021-03-10T14:11:18.842459+0000 mgr.gibba032.whopeo (mgr.24625) 1027 : audit [DBG] from='client.54724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:20 gibba032 bash[24004]: audit 2021-03-10T14:11:19.064920+0000 mgr.gibba032.whopeo (mgr.24625) 1028 : audit [DBG] from='client.54730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:20 gibba032 bash[24004]: cluster 2021-03-10T14:11:19.204089+0000 mgr.gibba032.whopeo (mgr.24625) 1029 : cluster [DBG] pgmap v566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:20 gibba032 bash[24004]: audit 2021-03-10T14:11:19.332801+0000 mon.c (mon.1) 20 : audit [DBG] from='client.? 172.21.2.111:0/993695408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:22 gibba032 bash[24004]: cluster 2021-03-10T14:11:21.205613+0000 mgr.gibba032.whopeo (mgr.24625) 1030 : cluster [DBG] pgmap v567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:22 gibba011 bash[30791]: cluster 2021-03-10T14:11:21.205613+0000 mgr.gibba032.whopeo (mgr.24625) 1030 : cluster [DBG] pgmap v567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:22 gibba011 bash[31774]: cluster 2021-03-10T14:11:21.205613+0000 mgr.gibba032.whopeo (mgr.24625) 1030 : cluster [DBG] pgmap v567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:24 gibba032 bash[24004]: cluster 2021-03-10T14:11:23.207366+0000 mgr.gibba032.whopeo (mgr.24625) 1031 : cluster [DBG] pgmap v568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:24 gibba011 bash[30791]: cluster 2021-03-10T14:11:23.207366+0000 mgr.gibba032.whopeo (mgr.24625) 1031 : cluster [DBG] pgmap v568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:24 gibba011 bash[31774]: cluster 2021-03-10T14:11:23.207366+0000 mgr.gibba032.whopeo (mgr.24625) 1031 : cluster [DBG] pgmap v568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:26 gibba032 bash[24004]: cluster 2021-03-10T14:11:25.208968+0000 mgr.gibba032.whopeo (mgr.24625) 1032 : cluster [DBG] pgmap v569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:26 gibba011 bash[30791]: cluster 2021-03-10T14:11:25.208968+0000 mgr.gibba032.whopeo (mgr.24625) 1032 : cluster [DBG] pgmap v569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:26 gibba011 bash[31774]: cluster 2021-03-10T14:11:25.208968+0000 mgr.gibba032.whopeo (mgr.24625) 1032 : cluster [DBG] pgmap v569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:28 gibba032 bash[24004]: cluster 2021-03-10T14:11:27.210415+0000 mgr.gibba032.whopeo (mgr.24625) 1033 : cluster [DBG] pgmap v570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:28 gibba011 bash[30791]: cluster 2021-03-10T14:11:27.210415+0000 mgr.gibba032.whopeo (mgr.24625) 1033 : cluster [DBG] pgmap v570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:28 gibba011 bash[31774]: cluster 2021-03-10T14:11:27.210415+0000 mgr.gibba032.whopeo (mgr.24625) 1033 : cluster [DBG] pgmap v570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:30 gibba032 bash[24004]: cluster 2021-03-10T14:11:29.211619+0000 mgr.gibba032.whopeo (mgr.24625) 1034 : cluster [DBG] pgmap v571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:30 gibba011 bash[30791]: cluster 2021-03-10T14:11:29.211619+0000 mgr.gibba032.whopeo (mgr.24625) 1034 : cluster [DBG] pgmap v571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:30 gibba011 bash[31774]: cluster 2021-03-10T14:11:29.211619+0000 mgr.gibba032.whopeo (mgr.24625) 1034 : cluster [DBG] pgmap v571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:32 gibba032 bash[24004]: cluster 2021-03-10T14:11:31.213174+0000 mgr.gibba032.whopeo (mgr.24625) 1035 : cluster [DBG] pgmap v572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:32 gibba011 bash[30791]: cluster 2021-03-10T14:11:31.213174+0000 mgr.gibba032.whopeo (mgr.24625) 1035 : cluster [DBG] pgmap v572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:32 gibba011 bash[31774]: cluster 2021-03-10T14:11:31.213174+0000 mgr.gibba032.whopeo (mgr.24625) 1035 : cluster [DBG] pgmap v572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:33 gibba032 bash[24004]: cephadm 2021-03-10T14:11:31.974698+0000 mgr.gibba032.whopeo (mgr.24625) 1036 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:33 gibba011 bash[30791]: cephadm 2021-03-10T14:11:31.974698+0000 mgr.gibba032.whopeo (mgr.24625) 1036 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:33.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:33 gibba011 bash[31774]: cephadm 2021-03-10T14:11:31.974698+0000 mgr.gibba032.whopeo (mgr.24625) 1036 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:34 gibba032 bash[24004]: cluster 2021-03-10T14:11:33.214672+0000 mgr.gibba032.whopeo (mgr.24625) 1037 : cluster [DBG] pgmap v573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:34 gibba011 bash[30791]: cluster 2021-03-10T14:11:33.214672+0000 mgr.gibba032.whopeo (mgr.24625) 1037 : cluster [DBG] pgmap v573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:34 gibba011 bash[31774]: cluster 2021-03-10T14:11:33.214672+0000 mgr.gibba032.whopeo (mgr.24625) 1037 : cluster [DBG] pgmap v573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:36 gibba032 bash[24004]: cluster 2021-03-10T14:11:35.216356+0000 mgr.gibba032.whopeo (mgr.24625) 1038 : cluster [DBG] pgmap v574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:36 gibba011 bash[30791]: cluster 2021-03-10T14:11:35.216356+0000 mgr.gibba032.whopeo (mgr.24625) 1038 : cluster [DBG] pgmap v574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:36 gibba011 bash[31774]: cluster 2021-03-10T14:11:35.216356+0000 mgr.gibba032.whopeo (mgr.24625) 1038 : cluster [DBG] pgmap v574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:38 gibba032 bash[24004]: cluster 2021-03-10T14:11:37.217790+0000 mgr.gibba032.whopeo (mgr.24625) 1039 : cluster [DBG] pgmap v575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:38 gibba011 bash[30791]: cluster 2021-03-10T14:11:37.217790+0000 mgr.gibba032.whopeo (mgr.24625) 1039 : cluster [DBG] pgmap v575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:38 gibba011 bash[31774]: cluster 2021-03-10T14:11:37.217790+0000 mgr.gibba032.whopeo (mgr.24625) 1039 : cluster [DBG] pgmap v575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:40 gibba032 bash[24004]: cluster 2021-03-10T14:11:39.219137+0000 mgr.gibba032.whopeo (mgr.24625) 1040 : cluster [DBG] pgmap v576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:40 gibba011 bash[30791]: cluster 2021-03-10T14:11:39.219137+0000 mgr.gibba032.whopeo (mgr.24625) 1040 : cluster [DBG] pgmap v576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:40 gibba011 bash[31774]: cluster 2021-03-10T14:11:39.219137+0000 mgr.gibba032.whopeo (mgr.24625) 1040 : cluster [DBG] pgmap v576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:42 gibba032 bash[24004]: cluster 2021-03-10T14:11:41.220983+0000 mgr.gibba032.whopeo (mgr.24625) 1041 : cluster [DBG] pgmap v577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:42 gibba011 bash[30791]: cluster 2021-03-10T14:11:41.220983+0000 mgr.gibba032.whopeo (mgr.24625) 1041 : cluster [DBG] pgmap v577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:42 gibba011 bash[31774]: cluster 2021-03-10T14:11:41.220983+0000 mgr.gibba032.whopeo (mgr.24625) 1041 : cluster [DBG] pgmap v577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:44 gibba032 bash[24004]: cluster 2021-03-10T14:11:43.222452+0000 mgr.gibba032.whopeo (mgr.24625) 1042 : cluster [DBG] pgmap v578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:44 gibba011 bash[30791]: cluster 2021-03-10T14:11:43.222452+0000 mgr.gibba032.whopeo (mgr.24625) 1042 : cluster [DBG] pgmap v578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:44 gibba011 bash[31774]: cluster 2021-03-10T14:11:43.222452+0000 mgr.gibba032.whopeo (mgr.24625) 1042 : cluster [DBG] pgmap v578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:46 gibba032 bash[24004]: cluster 2021-03-10T14:11:45.223929+0000 mgr.gibba032.whopeo (mgr.24625) 1043 : cluster [DBG] pgmap v579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:46 gibba011 bash[30791]: cluster 2021-03-10T14:11:45.223929+0000 mgr.gibba032.whopeo (mgr.24625) 1043 : cluster [DBG] pgmap v579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:46 gibba011 bash[31774]: cluster 2021-03-10T14:11:45.223929+0000 mgr.gibba032.whopeo (mgr.24625) 1043 : cluster [DBG] pgmap v579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:48 gibba032 bash[24004]: cephadm 2021-03-10T14:11:46.990737+0000 mgr.gibba032.whopeo (mgr.24625) 1044 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:48 gibba032 bash[24004]: cluster 2021-03-10T14:11:47.225482+0000 mgr.gibba032.whopeo (mgr.24625) 1045 : cluster [DBG] pgmap v580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:48 gibba011 bash[30791]: cephadm 2021-03-10T14:11:46.990737+0000 mgr.gibba032.whopeo (mgr.24625) 1044 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:48 gibba011 bash[30791]: cluster 2021-03-10T14:11:47.225482+0000 mgr.gibba032.whopeo (mgr.24625) 1045 : cluster [DBG] pgmap v580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:48 gibba011 bash[31774]: cephadm 2021-03-10T14:11:46.990737+0000 mgr.gibba032.whopeo (mgr.24625) 1044 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:11:48.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:48 gibba011 bash[31774]: cluster 2021-03-10T14:11:47.225482+0000 mgr.gibba032.whopeo (mgr.24625) 1045 : cluster [DBG] pgmap v580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:49.575 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:11:49.802 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:11:49.802 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (18m) 4m ago 23m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:11:49.802 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (23m) 3m ago 23m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:11:49.803 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (18m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:11:49.803 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (19m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:11:49.803 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (18m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:11:49.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (18m) 4m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:11:49.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (17m) 3m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:11:49.804 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (17m) 4m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:11:49.804 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (23m) 4m ago 24m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:11:49.804 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (23m) 3m ago 23m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:11:49.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (16m) 4m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:11:49.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (16m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:11:49.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (15m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:11:49.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (15m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:11:49.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (15m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:11:49.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (14m) 3m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:11:49.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (14m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:11:49.806 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (13m) 3m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:11:49.806 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (23m) 3m ago 24m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:11:49.806 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 21m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:11:50.050 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:11:50.051 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:11:50.051 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:11:50.051 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:50.051 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:11:50.052 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:11:50.052 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:50.052 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:11:50.052 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:11:50.052 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:11:50.053 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:11:50.053 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:11:50.053 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:11:50.053 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:11:50.053 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:11:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:50 gibba032 bash[24004]: cluster 2021-03-10T14:11:49.226963+0000 mgr.gibba032.whopeo (mgr.24625) 1046 : cluster [DBG] pgmap v581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:50 gibba032 bash[24004]: audit 2021-03-10T14:11:49.564211+0000 mgr.gibba032.whopeo (mgr.24625) 1047 : audit [DBG] from='client.54742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:50 gibba032 bash[24004]: audit 2021-03-10T14:11:50.051050+0000 mon.c (mon.1) 21 : audit [DBG] from='client.? 172.21.2.111:0/674635268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[30791]: cluster 2021-03-10T14:11:49.226963+0000 mgr.gibba032.whopeo (mgr.24625) 1046 : cluster [DBG] pgmap v581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[30791]: audit 2021-03-10T14:11:49.564211+0000 mgr.gibba032.whopeo (mgr.24625) 1047 : audit [DBG] from='client.54742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[30791]: audit 2021-03-10T14:11:50.051050+0000 mon.c (mon.1) 21 : audit [DBG] from='client.? 172.21.2.111:0/674635268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[31774]: cluster 2021-03-10T14:11:49.226963+0000 mgr.gibba032.whopeo (mgr.24625) 1046 : cluster [DBG] pgmap v581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[31774]: audit 2021-03-10T14:11:49.564211+0000 mgr.gibba032.whopeo (mgr.24625) 1047 : audit [DBG] from='client.54742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:50 gibba011 bash[31774]: audit 2021-03-10T14:11:50.051050+0000 mon.c (mon.1) 21 : audit [DBG] from='client.? 172.21.2.111:0/674635268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:11:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:51 gibba032 bash[24004]: audit 2021-03-10T14:11:49.787578+0000 mgr.gibba032.whopeo (mgr.24625) 1048 : audit [DBG] from='client.54748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:51 gibba011 bash[30791]: audit 2021-03-10T14:11:49.787578+0000 mgr.gibba032.whopeo (mgr.24625) 1048 : audit [DBG] from='client.54748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:51 gibba011 bash[31774]: audit 2021-03-10T14:11:49.787578+0000 mgr.gibba032.whopeo (mgr.24625) 1048 : audit [DBG] from='client.54748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:11:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:52 gibba032 bash[24004]: cluster 2021-03-10T14:11:51.228739+0000 mgr.gibba032.whopeo (mgr.24625) 1049 : cluster [DBG] pgmap v582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:52 gibba011 bash[30791]: cluster 2021-03-10T14:11:51.228739+0000 mgr.gibba032.whopeo (mgr.24625) 1049 : cluster [DBG] pgmap v582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:52 gibba011 bash[31774]: cluster 2021-03-10T14:11:51.228739+0000 mgr.gibba032.whopeo (mgr.24625) 1049 : cluster [DBG] pgmap v582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:54 gibba032 bash[24004]: cluster 2021-03-10T14:11:53.230382+0000 mgr.gibba032.whopeo (mgr.24625) 1050 : cluster [DBG] pgmap v583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:54 gibba011 bash[30791]: cluster 2021-03-10T14:11:53.230382+0000 mgr.gibba032.whopeo (mgr.24625) 1050 : cluster [DBG] pgmap v583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:54 gibba011 bash[31774]: cluster 2021-03-10T14:11:53.230382+0000 mgr.gibba032.whopeo (mgr.24625) 1050 : cluster [DBG] pgmap v583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: cluster 2021-03-10T14:11:55.231919+0000 mgr.gibba032.whopeo (mgr.24625) 1051 : cluster [DBG] pgmap v584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.031014+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:11:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.031857+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.032567+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.033254+0000 mon.a (mon.0) 866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:11:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.119680+0000 mon.a (mon.0) 867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:11:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:56 gibba032 bash[24004]: audit 2021-03-10T14:11:56.120350+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:11:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: cluster 2021-03-10T14:11:55.231919+0000 mgr.gibba032.whopeo (mgr.24625) 1051 : cluster [DBG] pgmap v584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.031014+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:11:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.031857+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.032567+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.033254+0000 mon.a (mon.0) 866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:11:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.119680+0000 mon.a (mon.0) 867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:11:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[30791]: audit 2021-03-10T14:11:56.120350+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:11:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: cluster 2021-03-10T14:11:55.231919+0000 mgr.gibba032.whopeo (mgr.24625) 1051 : cluster [DBG] pgmap v584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.031014+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:11:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.031857+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.032567+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:11:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.033254+0000 mon.a (mon.0) 866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:11:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.119680+0000 mon.a (mon.0) 867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:11:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:56 gibba011 bash[31774]: audit 2021-03-10T14:11:56.120350+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:11:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:11:58 gibba032 bash[24004]: cluster 2021-03-10T14:11:57.233487+0000 mgr.gibba032.whopeo (mgr.24625) 1052 : cluster [DBG] pgmap v585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:11:58 gibba011 bash[30791]: cluster 2021-03-10T14:11:57.233487+0000 mgr.gibba032.whopeo (mgr.24625) 1052 : cluster [DBG] pgmap v585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:11:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:11:58 gibba011 bash[31774]: cluster 2021-03-10T14:11:57.233487+0000 mgr.gibba032.whopeo (mgr.24625) 1052 : cluster [DBG] pgmap v585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:00 gibba011 bash[30791]: cluster 2021-03-10T14:11:59.234998+0000 mgr.gibba032.whopeo (mgr.24625) 1053 : cluster [DBG] pgmap v586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:00 gibba011 bash[31774]: cluster 2021-03-10T14:11:59.234998+0000 mgr.gibba032.whopeo (mgr.24625) 1053 : cluster [DBG] pgmap v586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:00 gibba032 bash[24004]: cluster 2021-03-10T14:11:59.234998+0000 mgr.gibba032.whopeo (mgr.24625) 1053 : cluster [DBG] pgmap v586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:02 gibba011 bash[30791]: cluster 2021-03-10T14:12:01.236693+0000 mgr.gibba032.whopeo (mgr.24625) 1054 : cluster [DBG] pgmap v587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:02 gibba011 bash[31774]: cluster 2021-03-10T14:12:01.236693+0000 mgr.gibba032.whopeo (mgr.24625) 1054 : cluster [DBG] pgmap v587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:03.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:02 gibba032 bash[24004]: cluster 2021-03-10T14:12:01.236693+0000 mgr.gibba032.whopeo (mgr.24625) 1054 : cluster [DBG] pgmap v587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:03 gibba011 bash[30791]: cephadm 2021-03-10T14:12:02.006571+0000 mgr.gibba032.whopeo (mgr.24625) 1055 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:03 gibba011 bash[31774]: cephadm 2021-03-10T14:12:02.006571+0000 mgr.gibba032.whopeo (mgr.24625) 1055 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:04.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:03 gibba032 bash[24004]: cephadm 2021-03-10T14:12:02.006571+0000 mgr.gibba032.whopeo (mgr.24625) 1055 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:04 gibba011 bash[30791]: cluster 2021-03-10T14:12:03.238262+0000 mgr.gibba032.whopeo (mgr.24625) 1056 : cluster [DBG] pgmap v588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:04 gibba011 bash[31774]: cluster 2021-03-10T14:12:03.238262+0000 mgr.gibba032.whopeo (mgr.24625) 1056 : cluster [DBG] pgmap v588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:04 gibba032 bash[24004]: cluster 2021-03-10T14:12:03.238262+0000 mgr.gibba032.whopeo (mgr.24625) 1056 : cluster [DBG] pgmap v588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:06 gibba011 bash[30791]: cluster 2021-03-10T14:12:05.239862+0000 mgr.gibba032.whopeo (mgr.24625) 1057 : cluster [DBG] pgmap v589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:06 gibba011 bash[31774]: cluster 2021-03-10T14:12:05.239862+0000 mgr.gibba032.whopeo (mgr.24625) 1057 : cluster [DBG] pgmap v589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:06 gibba032 bash[24004]: cluster 2021-03-10T14:12:05.239862+0000 mgr.gibba032.whopeo (mgr.24625) 1057 : cluster [DBG] pgmap v589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:08 gibba011 bash[30791]: cluster 2021-03-10T14:12:07.241565+0000 mgr.gibba032.whopeo (mgr.24625) 1058 : cluster [DBG] pgmap v590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:08 gibba011 bash[31774]: cluster 2021-03-10T14:12:07.241565+0000 mgr.gibba032.whopeo (mgr.24625) 1058 : cluster [DBG] pgmap v590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:08 gibba032 bash[24004]: cluster 2021-03-10T14:12:07.241565+0000 mgr.gibba032.whopeo (mgr.24625) 1058 : cluster [DBG] pgmap v590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:10 gibba011 bash[30791]: cluster 2021-03-10T14:12:09.243045+0000 mgr.gibba032.whopeo (mgr.24625) 1059 : cluster [DBG] pgmap v591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:10 gibba011 bash[31774]: cluster 2021-03-10T14:12:09.243045+0000 mgr.gibba032.whopeo (mgr.24625) 1059 : cluster [DBG] pgmap v591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:10 gibba032 bash[24004]: cluster 2021-03-10T14:12:09.243045+0000 mgr.gibba032.whopeo (mgr.24625) 1059 : cluster [DBG] pgmap v591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:12 gibba011 bash[30791]: cluster 2021-03-10T14:12:11.244824+0000 mgr.gibba032.whopeo (mgr.24625) 1060 : cluster [DBG] pgmap v592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:12 gibba011 bash[31774]: cluster 2021-03-10T14:12:11.244824+0000 mgr.gibba032.whopeo (mgr.24625) 1060 : cluster [DBG] pgmap v592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:13.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:12 gibba032 bash[24004]: cluster 2021-03-10T14:12:11.244824+0000 mgr.gibba032.whopeo (mgr.24625) 1060 : cluster [DBG] pgmap v592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:14 gibba032 bash[24004]: cluster 2021-03-10T14:12:13.246434+0000 mgr.gibba032.whopeo (mgr.24625) 1061 : cluster [DBG] pgmap v593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:14 gibba011 bash[30791]: cluster 2021-03-10T14:12:13.246434+0000 mgr.gibba032.whopeo (mgr.24625) 1061 : cluster [DBG] pgmap v593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:14 gibba011 bash[31774]: cluster 2021-03-10T14:12:13.246434+0000 mgr.gibba032.whopeo (mgr.24625) 1061 : cluster [DBG] pgmap v593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:16 gibba032 bash[24004]: cluster 2021-03-10T14:12:15.248062+0000 mgr.gibba032.whopeo (mgr.24625) 1062 : cluster [DBG] pgmap v594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:17.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:16 gibba011 bash[30791]: cluster 2021-03-10T14:12:15.248062+0000 mgr.gibba032.whopeo (mgr.24625) 1062 : cluster [DBG] pgmap v594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:16 gibba011 bash[31774]: cluster 2021-03-10T14:12:15.248062+0000 mgr.gibba032.whopeo (mgr.24625) 1062 : cluster [DBG] pgmap v594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.021580+0000 mgr.gibba032.whopeo (mgr.24625) 1063 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:12:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.023148+0000 mgr.gibba032.whopeo (mgr.24625) 1064 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:12:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cluster 2021-03-10T14:12:17.249795+0000 mgr.gibba032.whopeo (mgr.24625) 1065 : cluster [DBG] pgmap v595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.251222+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:12:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.257546+0000 mgr.gibba032.whopeo (mgr.24625) 1066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:12:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.259918+0000 mgr.gibba032.whopeo (mgr.24625) 1067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:12:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.260721+0000 mon.a (mon.0) 870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:12:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.261652+0000 mgr.gibba032.whopeo (mgr.24625) 1068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:12:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.261798+0000 mgr.gibba032.whopeo (mgr.24625) 1069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:12:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.262698+0000 mon.a (mon.0) 871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.263616+0000 mgr.gibba032.whopeo (mgr.24625) 1070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:12:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.263701+0000 mgr.gibba032.whopeo (mgr.24625) 1071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:12:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.264652+0000 mon.a (mon.0) 872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.265068+0000 mgr.gibba032.whopeo (mgr.24625) 1072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:12:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.266374+0000 mon.a (mon.0) 873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.266377+0000 mgr.gibba032.whopeo (mgr.24625) 1073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:12:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.266413+0000 mgr.gibba032.whopeo (mgr.24625) 1074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:12:18.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.267953+0000 mgr.gibba032.whopeo (mgr.24625) 1075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:12:18.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.268307+0000 mgr.gibba032.whopeo (mgr.24625) 1076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:12:18.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.269160+0000 mon.a (mon.0) 874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.269806+0000 mgr.gibba032.whopeo (mgr.24625) 1077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:12:18.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.021580+0000 mgr.gibba032.whopeo (mgr.24625) 1063 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:12:18.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.023148+0000 mgr.gibba032.whopeo (mgr.24625) 1064 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:12:18.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cluster 2021-03-10T14:12:17.249795+0000 mgr.gibba032.whopeo (mgr.24625) 1065 : cluster [DBG] pgmap v595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:18.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.251222+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:12:18.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.257546+0000 mgr.gibba032.whopeo (mgr.24625) 1066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:12:18.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.259918+0000 mgr.gibba032.whopeo (mgr.24625) 1067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:12:18.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.260721+0000 mon.a (mon.0) 870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:12:18.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.261652+0000 mgr.gibba032.whopeo (mgr.24625) 1068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:12:18.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.261798+0000 mgr.gibba032.whopeo (mgr.24625) 1069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:12:18.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.262698+0000 mon.a (mon.0) 871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.263616+0000 mgr.gibba032.whopeo (mgr.24625) 1070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:12:18.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.263701+0000 mgr.gibba032.whopeo (mgr.24625) 1071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:12:18.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.264652+0000 mon.a (mon.0) 872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.265068+0000 mgr.gibba032.whopeo (mgr.24625) 1072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:12:18.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.266374+0000 mon.a (mon.0) 873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.266377+0000 mgr.gibba032.whopeo (mgr.24625) 1073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:12:18.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.266413+0000 mgr.gibba032.whopeo (mgr.24625) 1074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:12:18.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.267953+0000 mgr.gibba032.whopeo (mgr.24625) 1075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:12:18.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.268307+0000 mgr.gibba032.whopeo (mgr.24625) 1076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:12:18.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.269160+0000 mon.a (mon.0) 874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.269806+0000 mgr.gibba032.whopeo (mgr.24625) 1077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:12:18.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.269848+0000 mgr.gibba032.whopeo (mgr.24625) 1078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:12:18.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: cephadm 2021-03-10T14:12:17.269976+0000 mgr.gibba032.whopeo (mgr.24625) 1079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:18.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[31774]: audit 2021-03-10T14:12:17.270910+0000 mon.a (mon.0) 875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.269848+0000 mgr.gibba032.whopeo (mgr.24625) 1078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:12:18.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: cephadm 2021-03-10T14:12:17.269976+0000 mgr.gibba032.whopeo (mgr.24625) 1079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:18.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:18 gibba011 bash[30791]: audit 2021-03-10T14:12:17.270910+0000 mon.a (mon.0) 875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.021580+0000 mgr.gibba032.whopeo (mgr.24625) 1063 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:12:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.023148+0000 mgr.gibba032.whopeo (mgr.24625) 1064 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:12:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cluster 2021-03-10T14:12:17.249795+0000 mgr.gibba032.whopeo (mgr.24625) 1065 : cluster [DBG] pgmap v595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.251222+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:12:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.257546+0000 mgr.gibba032.whopeo (mgr.24625) 1066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:12:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.259918+0000 mgr.gibba032.whopeo (mgr.24625) 1067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:12:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.260721+0000 mon.a (mon.0) 870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:12:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.261652+0000 mgr.gibba032.whopeo (mgr.24625) 1068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:12:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.261798+0000 mgr.gibba032.whopeo (mgr.24625) 1069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:12:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.262698+0000 mon.a (mon.0) 871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.263616+0000 mgr.gibba032.whopeo (mgr.24625) 1070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:12:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.263701+0000 mgr.gibba032.whopeo (mgr.24625) 1071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:12:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.264652+0000 mon.a (mon.0) 872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.265068+0000 mgr.gibba032.whopeo (mgr.24625) 1072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:12:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.266374+0000 mon.a (mon.0) 873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.266377+0000 mgr.gibba032.whopeo (mgr.24625) 1073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:12:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.266413+0000 mgr.gibba032.whopeo (mgr.24625) 1074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:12:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.267953+0000 mgr.gibba032.whopeo (mgr.24625) 1075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:12:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.268307+0000 mgr.gibba032.whopeo (mgr.24625) 1076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:12:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.269160+0000 mon.a (mon.0) 874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.269806+0000 mgr.gibba032.whopeo (mgr.24625) 1077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:12:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.269848+0000 mgr.gibba032.whopeo (mgr.24625) 1078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:12:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: cephadm 2021-03-10T14:12:17.269976+0000 mgr.gibba032.whopeo (mgr.24625) 1079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:18 gibba032 bash[24004]: audit 2021-03-10T14:12:17.270910+0000 mon.a (mon.0) 875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:20.291 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:12:20.545 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:12:20.545 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (18m) 5m ago 24m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:12:20.545 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (24m) 4m ago 24m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:12:20.546 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (19m) 4m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:12:20.546 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (19m) 4m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:12:20.546 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (18m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:12:20.546 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (18m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:12:20.547 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (18m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:12:20.547 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (18m) 5m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:12:20.547 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (24m) 5m ago 24m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:12:20.547 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (24m) 4m ago 24m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:12:20.547 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (17m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:12:20.548 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (16m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:12:20.548 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (16m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:12:20.548 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (15m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:12:20.548 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (15m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:12:20.549 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (15m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:12:20.549 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (14m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:12:20.549 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (14m) 4m ago 24m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:12:20.549 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (23m) 4m ago 24m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:12:20.549 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 22m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:12:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:20 gibba011 bash[31774]: cluster 2021-03-10T14:12:19.251189+0000 mgr.gibba032.whopeo (mgr.24625) 1080 : cluster [DBG] pgmap v596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:20 gibba011 bash[30791]: cluster 2021-03-10T14:12:19.251189+0000 mgr.gibba032.whopeo (mgr.24625) 1080 : cluster [DBG] pgmap v596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:20 gibba032 bash[24004]: cluster 2021-03-10T14:12:19.251189+0000 mgr.gibba032.whopeo (mgr.24625) 1080 : cluster [DBG] pgmap v596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:20.801 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:12:20.802 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:12:20.802 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:12:20.802 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:20.802 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:12:20.803 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:12:20.803 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:20.803 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:12:20.803 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:12:20.803 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:20.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:12:20.804 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:12:20.804 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:12:20.804 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:12:20.804 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:12:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:21 gibba032 bash[24004]: audit 2021-03-10T14:12:20.280857+0000 mgr.gibba032.whopeo (mgr.24625) 1081 : audit [DBG] from='client.44775 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:21 gibba032 bash[24004]: audit 2021-03-10T14:12:20.503981+0000 mgr.gibba032.whopeo (mgr.24625) 1082 : audit [DBG] from='client.54766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:21 gibba032 bash[24004]: audit 2021-03-10T14:12:20.801913+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? 172.21.2.111:0/4002912673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[30791]: audit 2021-03-10T14:12:20.280857+0000 mgr.gibba032.whopeo (mgr.24625) 1081 : audit [DBG] from='client.44775 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[30791]: audit 2021-03-10T14:12:20.503981+0000 mgr.gibba032.whopeo (mgr.24625) 1082 : audit [DBG] from='client.54766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[30791]: audit 2021-03-10T14:12:20.801913+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? 172.21.2.111:0/4002912673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:21.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[31774]: audit 2021-03-10T14:12:20.280857+0000 mgr.gibba032.whopeo (mgr.24625) 1081 : audit [DBG] from='client.44775 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[31774]: audit 2021-03-10T14:12:20.503981+0000 mgr.gibba032.whopeo (mgr.24625) 1082 : audit [DBG] from='client.54766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:21.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:21 gibba011 bash[31774]: audit 2021-03-10T14:12:20.801913+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? 172.21.2.111:0/4002912673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:22 gibba032 bash[24004]: cluster 2021-03-10T14:12:21.252899+0000 mgr.gibba032.whopeo (mgr.24625) 1083 : cluster [DBG] pgmap v597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:22 gibba011 bash[30791]: cluster 2021-03-10T14:12:21.252899+0000 mgr.gibba032.whopeo (mgr.24625) 1083 : cluster [DBG] pgmap v597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:22 gibba011 bash[31774]: cluster 2021-03-10T14:12:21.252899+0000 mgr.gibba032.whopeo (mgr.24625) 1083 : cluster [DBG] pgmap v597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:24 gibba032 bash[24004]: cluster 2021-03-10T14:12:23.254293+0000 mgr.gibba032.whopeo (mgr.24625) 1084 : cluster [DBG] pgmap v598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:24 gibba011 bash[30791]: cluster 2021-03-10T14:12:23.254293+0000 mgr.gibba032.whopeo (mgr.24625) 1084 : cluster [DBG] pgmap v598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:24 gibba011 bash[31774]: cluster 2021-03-10T14:12:23.254293+0000 mgr.gibba032.whopeo (mgr.24625) 1084 : cluster [DBG] pgmap v598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:26 gibba032 bash[24004]: cluster 2021-03-10T14:12:25.255759+0000 mgr.gibba032.whopeo (mgr.24625) 1085 : cluster [DBG] pgmap v599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:26 gibba011 bash[30791]: cluster 2021-03-10T14:12:25.255759+0000 mgr.gibba032.whopeo (mgr.24625) 1085 : cluster [DBG] pgmap v599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:26 gibba011 bash[31774]: cluster 2021-03-10T14:12:25.255759+0000 mgr.gibba032.whopeo (mgr.24625) 1085 : cluster [DBG] pgmap v599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:28 gibba032 bash[24004]: cluster 2021-03-10T14:12:27.257349+0000 mgr.gibba032.whopeo (mgr.24625) 1086 : cluster [DBG] pgmap v600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:28 gibba011 bash[30791]: cluster 2021-03-10T14:12:27.257349+0000 mgr.gibba032.whopeo (mgr.24625) 1086 : cluster [DBG] pgmap v600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:28 gibba011 bash[31774]: cluster 2021-03-10T14:12:27.257349+0000 mgr.gibba032.whopeo (mgr.24625) 1086 : cluster [DBG] pgmap v600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:30 gibba032 bash[24004]: cluster 2021-03-10T14:12:29.258591+0000 mgr.gibba032.whopeo (mgr.24625) 1087 : cluster [DBG] pgmap v601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:30 gibba011 bash[30791]: cluster 2021-03-10T14:12:29.258591+0000 mgr.gibba032.whopeo (mgr.24625) 1087 : cluster [DBG] pgmap v601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:30 gibba011 bash[31774]: cluster 2021-03-10T14:12:29.258591+0000 mgr.gibba032.whopeo (mgr.24625) 1087 : cluster [DBG] pgmap v601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:32 gibba032 bash[24004]: cluster 2021-03-10T14:12:31.260498+0000 mgr.gibba032.whopeo (mgr.24625) 1088 : cluster [DBG] pgmap v602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:32 gibba011 bash[30791]: cluster 2021-03-10T14:12:31.260498+0000 mgr.gibba032.whopeo (mgr.24625) 1088 : cluster [DBG] pgmap v602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:32 gibba011 bash[31774]: cluster 2021-03-10T14:12:31.260498+0000 mgr.gibba032.whopeo (mgr.24625) 1088 : cluster [DBG] pgmap v602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:33 gibba032 bash[24004]: cephadm 2021-03-10T14:12:32.285989+0000 mgr.gibba032.whopeo (mgr.24625) 1089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:33 gibba011 bash[30791]: cephadm 2021-03-10T14:12:32.285989+0000 mgr.gibba032.whopeo (mgr.24625) 1089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:33.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:33 gibba011 bash[31774]: cephadm 2021-03-10T14:12:32.285989+0000 mgr.gibba032.whopeo (mgr.24625) 1089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:34 gibba032 bash[24004]: cluster 2021-03-10T14:12:33.262136+0000 mgr.gibba032.whopeo (mgr.24625) 1090 : cluster [DBG] pgmap v603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:34 gibba011 bash[30791]: cluster 2021-03-10T14:12:33.262136+0000 mgr.gibba032.whopeo (mgr.24625) 1090 : cluster [DBG] pgmap v603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:34 gibba011 bash[31774]: cluster 2021-03-10T14:12:33.262136+0000 mgr.gibba032.whopeo (mgr.24625) 1090 : cluster [DBG] pgmap v603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:36 gibba032 bash[24004]: cluster 2021-03-10T14:12:35.263631+0000 mgr.gibba032.whopeo (mgr.24625) 1091 : cluster [DBG] pgmap v604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:36 gibba011 bash[30791]: cluster 2021-03-10T14:12:35.263631+0000 mgr.gibba032.whopeo (mgr.24625) 1091 : cluster [DBG] pgmap v604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:36 gibba011 bash[31774]: cluster 2021-03-10T14:12:35.263631+0000 mgr.gibba032.whopeo (mgr.24625) 1091 : cluster [DBG] pgmap v604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:38 gibba032 bash[24004]: cluster 2021-03-10T14:12:37.265329+0000 mgr.gibba032.whopeo (mgr.24625) 1092 : cluster [DBG] pgmap v605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:38 gibba011 bash[30791]: cluster 2021-03-10T14:12:37.265329+0000 mgr.gibba032.whopeo (mgr.24625) 1092 : cluster [DBG] pgmap v605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:38 gibba011 bash[31774]: cluster 2021-03-10T14:12:37.265329+0000 mgr.gibba032.whopeo (mgr.24625) 1092 : cluster [DBG] pgmap v605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:40 gibba032 bash[24004]: cluster 2021-03-10T14:12:39.266579+0000 mgr.gibba032.whopeo (mgr.24625) 1093 : cluster [DBG] pgmap v606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:40 gibba011 bash[30791]: cluster 2021-03-10T14:12:39.266579+0000 mgr.gibba032.whopeo (mgr.24625) 1093 : cluster [DBG] pgmap v606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:40 gibba011 bash[31774]: cluster 2021-03-10T14:12:39.266579+0000 mgr.gibba032.whopeo (mgr.24625) 1093 : cluster [DBG] pgmap v606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:42 gibba032 bash[24004]: cluster 2021-03-10T14:12:41.268286+0000 mgr.gibba032.whopeo (mgr.24625) 1094 : cluster [DBG] pgmap v607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:42 gibba011 bash[30791]: cluster 2021-03-10T14:12:41.268286+0000 mgr.gibba032.whopeo (mgr.24625) 1094 : cluster [DBG] pgmap v607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:42 gibba011 bash[31774]: cluster 2021-03-10T14:12:41.268286+0000 mgr.gibba032.whopeo (mgr.24625) 1094 : cluster [DBG] pgmap v607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:44 gibba032 bash[24004]: cluster 2021-03-10T14:12:43.269889+0000 mgr.gibba032.whopeo (mgr.24625) 1095 : cluster [DBG] pgmap v608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:44 gibba011 bash[30791]: cluster 2021-03-10T14:12:43.269889+0000 mgr.gibba032.whopeo (mgr.24625) 1095 : cluster [DBG] pgmap v608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:44 gibba011 bash[31774]: cluster 2021-03-10T14:12:43.269889+0000 mgr.gibba032.whopeo (mgr.24625) 1095 : cluster [DBG] pgmap v608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:46 gibba032 bash[24004]: cluster 2021-03-10T14:12:45.271530+0000 mgr.gibba032.whopeo (mgr.24625) 1096 : cluster [DBG] pgmap v609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:46 gibba011 bash[30791]: cluster 2021-03-10T14:12:45.271530+0000 mgr.gibba032.whopeo (mgr.24625) 1096 : cluster [DBG] pgmap v609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:46 gibba011 bash[31774]: cluster 2021-03-10T14:12:45.271530+0000 mgr.gibba032.whopeo (mgr.24625) 1096 : cluster [DBG] pgmap v609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:48 gibba032 bash[24004]: cluster 2021-03-10T14:12:47.273303+0000 mgr.gibba032.whopeo (mgr.24625) 1097 : cluster [DBG] pgmap v610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:48 gibba032 bash[24004]: cephadm 2021-03-10T14:12:47.301768+0000 mgr.gibba032.whopeo (mgr.24625) 1098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:48 gibba011 bash[30791]: cluster 2021-03-10T14:12:47.273303+0000 mgr.gibba032.whopeo (mgr.24625) 1097 : cluster [DBG] pgmap v610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:48 gibba011 bash[30791]: cephadm 2021-03-10T14:12:47.301768+0000 mgr.gibba032.whopeo (mgr.24625) 1098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:48 gibba011 bash[31774]: cluster 2021-03-10T14:12:47.273303+0000 mgr.gibba032.whopeo (mgr.24625) 1097 : cluster [DBG] pgmap v610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:48 gibba011 bash[31774]: cephadm 2021-03-10T14:12:47.301768+0000 mgr.gibba032.whopeo (mgr.24625) 1098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:12:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:50 gibba032 bash[24004]: cluster 2021-03-10T14:12:49.274802+0000 mgr.gibba032.whopeo (mgr.24625) 1099 : cluster [DBG] pgmap v611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:50.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:50 gibba011 bash[30791]: cluster 2021-03-10T14:12:49.274802+0000 mgr.gibba032.whopeo (mgr.24625) 1099 : cluster [DBG] pgmap v611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:50.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:50 gibba011 bash[31774]: cluster 2021-03-10T14:12:49.274802+0000 mgr.gibba032.whopeo (mgr.24625) 1099 : cluster [DBG] pgmap v611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:51.044 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:12:51.270 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:12:51.271 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (19m) 5m ago 24m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:12:51.271 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (24m) 4m ago 24m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:12:51.271 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (19m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:12:51.271 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (20m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:12:51.272 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (19m) 5m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:12:51.272 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (19m) 5m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:12:51.272 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (18m) 4m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:12:51.272 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (18m) 5m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:12:51.273 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (24m) 5m ago 25m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:12:51.273 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (24m) 4m ago 25m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:12:51.273 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (17m) 5m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:12:51.273 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (17m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:12:51.274 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (16m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:12:51.274 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (16m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:12:51.274 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (16m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:12:51.274 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (15m) 4m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:12:51.275 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (15m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:12:51.275 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (14m) 4m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:12:51.275 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (24m) 4m ago 25m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:12:51.275 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 22m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:12:51.519 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:12:51.519 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:12:51.520 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:12:51.520 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:51.520 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:12:51.520 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:12:51.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:51.521 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:12:51.521 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:12:51.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:12:51.522 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:12:51.522 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:12:51.522 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:12:51.523 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:12:51.523 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:12:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[30791]: audit 2021-03-10T14:12:51.033046+0000 mgr.gibba032.whopeo (mgr.24625) 1100 : audit [DBG] from='client.44793 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[30791]: audit 2021-03-10T14:12:51.256471+0000 mgr.gibba032.whopeo (mgr.24625) 1101 : audit [DBG] from='client.54784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[30791]: cluster 2021-03-10T14:12:51.275295+0000 mgr.gibba032.whopeo (mgr.24625) 1102 : cluster [DBG] pgmap v612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[30791]: audit 2021-03-10T14:12:51.519397+0000 mon.c (mon.1) 22 : audit [DBG] from='client.? 172.21.2.111:0/3486424138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[31774]: audit 2021-03-10T14:12:51.033046+0000 mgr.gibba032.whopeo (mgr.24625) 1100 : audit [DBG] from='client.44793 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[31774]: audit 2021-03-10T14:12:51.256471+0000 mgr.gibba032.whopeo (mgr.24625) 1101 : audit [DBG] from='client.54784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[31774]: cluster 2021-03-10T14:12:51.275295+0000 mgr.gibba032.whopeo (mgr.24625) 1102 : cluster [DBG] pgmap v612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:52 gibba011 bash[31774]: audit 2021-03-10T14:12:51.519397+0000 mon.c (mon.1) 22 : audit [DBG] from='client.? 172.21.2.111:0/3486424138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:52 gibba032 bash[24004]: audit 2021-03-10T14:12:51.033046+0000 mgr.gibba032.whopeo (mgr.24625) 1100 : audit [DBG] from='client.44793 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:52 gibba032 bash[24004]: audit 2021-03-10T14:12:51.256471+0000 mgr.gibba032.whopeo (mgr.24625) 1101 : audit [DBG] from='client.54784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:12:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:52 gibba032 bash[24004]: cluster 2021-03-10T14:12:51.275295+0000 mgr.gibba032.whopeo (mgr.24625) 1102 : cluster [DBG] pgmap v612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:52 gibba032 bash[24004]: audit 2021-03-10T14:12:51.519397+0000 mon.c (mon.1) 22 : audit [DBG] from='client.? 172.21.2.111:0/3486424138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:12:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:54 gibba011 bash[30791]: cluster 2021-03-10T14:12:53.276535+0000 mgr.gibba032.whopeo (mgr.24625) 1103 : cluster [DBG] pgmap v613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:54 gibba011 bash[31774]: cluster 2021-03-10T14:12:53.276535+0000 mgr.gibba032.whopeo (mgr.24625) 1103 : cluster [DBG] pgmap v613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:54 gibba032 bash[24004]: cluster 2021-03-10T14:12:53.276535+0000 mgr.gibba032.whopeo (mgr.24625) 1103 : cluster [DBG] pgmap v613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: cluster 2021-03-10T14:12:55.278167+0000 mgr.gibba032.whopeo (mgr.24625) 1104 : cluster [DBG] pgmap v614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.040974+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:12:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.041751+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.042446+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.043179+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.126499+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:12:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[30791]: audit 2021-03-10T14:12:56.127164+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:12:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: cluster 2021-03-10T14:12:55.278167+0000 mgr.gibba032.whopeo (mgr.24625) 1104 : cluster [DBG] pgmap v614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.040974+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:12:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.041751+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.042446+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.043179+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:12:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.126499+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:12:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:56 gibba011 bash[31774]: audit 2021-03-10T14:12:56.127164+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:12:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: cluster 2021-03-10T14:12:55.278167+0000 mgr.gibba032.whopeo (mgr.24625) 1104 : cluster [DBG] pgmap v614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.040974+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:12:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.041751+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:12:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.042446+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:12:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.043179+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:12:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.126499+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:12:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:56 gibba032 bash[24004]: audit 2021-03-10T14:12:56.127164+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:12:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:12:58 gibba011 bash[30791]: cluster 2021-03-10T14:12:57.279998+0000 mgr.gibba032.whopeo (mgr.24625) 1105 : cluster [DBG] pgmap v615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:12:58 gibba011 bash[31774]: cluster 2021-03-10T14:12:57.279998+0000 mgr.gibba032.whopeo (mgr.24625) 1105 : cluster [DBG] pgmap v615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:12:59.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:12:58 gibba032 bash[24004]: cluster 2021-03-10T14:12:57.279998+0000 mgr.gibba032.whopeo (mgr.24625) 1105 : cluster [DBG] pgmap v615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:00 gibba011 bash[30791]: cluster 2021-03-10T14:12:59.281568+0000 mgr.gibba032.whopeo (mgr.24625) 1106 : cluster [DBG] pgmap v616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:00 gibba011 bash[31774]: cluster 2021-03-10T14:12:59.281568+0000 mgr.gibba032.whopeo (mgr.24625) 1106 : cluster [DBG] pgmap v616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:00 gibba032 bash[24004]: cluster 2021-03-10T14:12:59.281568+0000 mgr.gibba032.whopeo (mgr.24625) 1106 : cluster [DBG] pgmap v616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:02 gibba032 bash[24004]: cluster 2021-03-10T14:13:01.283384+0000 mgr.gibba032.whopeo (mgr.24625) 1107 : cluster [DBG] pgmap v617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:02 gibba011 bash[30791]: cluster 2021-03-10T14:13:01.283384+0000 mgr.gibba032.whopeo (mgr.24625) 1107 : cluster [DBG] pgmap v617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:02 gibba011 bash[31774]: cluster 2021-03-10T14:13:01.283384+0000 mgr.gibba032.whopeo (mgr.24625) 1107 : cluster [DBG] pgmap v617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:03 gibba011 bash[30791]: cephadm 2021-03-10T14:13:02.306388+0000 mgr.gibba032.whopeo (mgr.24625) 1108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:03 gibba011 bash[31774]: cephadm 2021-03-10T14:13:02.306388+0000 mgr.gibba032.whopeo (mgr.24625) 1108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:04.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:03 gibba032 bash[24004]: cephadm 2021-03-10T14:13:02.306388+0000 mgr.gibba032.whopeo (mgr.24625) 1108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:04 gibba011 bash[30791]: cluster 2021-03-10T14:13:03.284968+0000 mgr.gibba032.whopeo (mgr.24625) 1109 : cluster [DBG] pgmap v618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:04 gibba011 bash[31774]: cluster 2021-03-10T14:13:03.284968+0000 mgr.gibba032.whopeo (mgr.24625) 1109 : cluster [DBG] pgmap v618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:04 gibba032 bash[24004]: cluster 2021-03-10T14:13:03.284968+0000 mgr.gibba032.whopeo (mgr.24625) 1109 : cluster [DBG] pgmap v618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:06 gibba032 bash[24004]: cluster 2021-03-10T14:13:05.286503+0000 mgr.gibba032.whopeo (mgr.24625) 1110 : cluster [DBG] pgmap v619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:06 gibba011 bash[30791]: cluster 2021-03-10T14:13:05.286503+0000 mgr.gibba032.whopeo (mgr.24625) 1110 : cluster [DBG] pgmap v619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:06 gibba011 bash[31774]: cluster 2021-03-10T14:13:05.286503+0000 mgr.gibba032.whopeo (mgr.24625) 1110 : cluster [DBG] pgmap v619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:08 gibba032 bash[24004]: cluster 2021-03-10T14:13:07.288332+0000 mgr.gibba032.whopeo (mgr.24625) 1111 : cluster [DBG] pgmap v620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:08 gibba011 bash[30791]: cluster 2021-03-10T14:13:07.288332+0000 mgr.gibba032.whopeo (mgr.24625) 1111 : cluster [DBG] pgmap v620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:08 gibba011 bash[31774]: cluster 2021-03-10T14:13:07.288332+0000 mgr.gibba032.whopeo (mgr.24625) 1111 : cluster [DBG] pgmap v620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:10 gibba032 bash[24004]: cluster 2021-03-10T14:13:09.289838+0000 mgr.gibba032.whopeo (mgr.24625) 1112 : cluster [DBG] pgmap v621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:11.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:10 gibba011 bash[30791]: cluster 2021-03-10T14:13:09.289838+0000 mgr.gibba032.whopeo (mgr.24625) 1112 : cluster [DBG] pgmap v621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:10 gibba011 bash[31774]: cluster 2021-03-10T14:13:09.289838+0000 mgr.gibba032.whopeo (mgr.24625) 1112 : cluster [DBG] pgmap v621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:12 gibba032 bash[24004]: cluster 2021-03-10T14:13:11.291366+0000 mgr.gibba032.whopeo (mgr.24625) 1113 : cluster [DBG] pgmap v622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:12 gibba011 bash[30791]: cluster 2021-03-10T14:13:11.291366+0000 mgr.gibba032.whopeo (mgr.24625) 1113 : cluster [DBG] pgmap v622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:12 gibba011 bash[31774]: cluster 2021-03-10T14:13:11.291366+0000 mgr.gibba032.whopeo (mgr.24625) 1113 : cluster [DBG] pgmap v622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:14 gibba032 bash[24004]: cluster 2021-03-10T14:13:13.292959+0000 mgr.gibba032.whopeo (mgr.24625) 1114 : cluster [DBG] pgmap v623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:14 gibba011 bash[30791]: cluster 2021-03-10T14:13:13.292959+0000 mgr.gibba032.whopeo (mgr.24625) 1114 : cluster [DBG] pgmap v623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:14 gibba011 bash[31774]: cluster 2021-03-10T14:13:13.292959+0000 mgr.gibba032.whopeo (mgr.24625) 1114 : cluster [DBG] pgmap v623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:16 gibba032 bash[24004]: cluster 2021-03-10T14:13:15.294619+0000 mgr.gibba032.whopeo (mgr.24625) 1115 : cluster [DBG] pgmap v624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:16 gibba011 bash[30791]: cluster 2021-03-10T14:13:15.294619+0000 mgr.gibba032.whopeo (mgr.24625) 1115 : cluster [DBG] pgmap v624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:16 gibba011 bash[31774]: cluster 2021-03-10T14:13:15.294619+0000 mgr.gibba032.whopeo (mgr.24625) 1115 : cluster [DBG] pgmap v624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cluster 2021-03-10T14:13:17.296413+0000 mgr.gibba032.whopeo (mgr.24625) 1116 : cluster [DBG] pgmap v625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.315485+0000 mgr.gibba032.whopeo (mgr.24625) 1117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:13:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.317122+0000 mgr.gibba032.whopeo (mgr.24625) 1118 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:13:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.540632+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:13:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.540730+0000 mgr.gibba032.whopeo (mgr.24625) 1119 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:13:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.542684+0000 mgr.gibba032.whopeo (mgr.24625) 1120 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:13:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.543884+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:13:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.544227+0000 mgr.gibba032.whopeo (mgr.24625) 1121 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:13:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.544264+0000 mgr.gibba032.whopeo (mgr.24625) 1122 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:13:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.545481+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.545669+0000 mgr.gibba032.whopeo (mgr.24625) 1123 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:13:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.545707+0000 mgr.gibba032.whopeo (mgr.24625) 1124 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:13:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.546916+0000 mgr.gibba032.whopeo (mgr.24625) 1125 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:13:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.547054+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.547764+0000 mgr.gibba032.whopeo (mgr.24625) 1126 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:13:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.547802+0000 mgr.gibba032.whopeo (mgr.24625) 1127 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:13:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.548402+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.549263+0000 mgr.gibba032.whopeo (mgr.24625) 1128 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:13:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.549591+0000 mgr.gibba032.whopeo (mgr.24625) 1129 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:13:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.550472+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.550803+0000 mgr.gibba032.whopeo (mgr.24625) 1130 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:13:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cluster 2021-03-10T14:13:17.296413+0000 mgr.gibba032.whopeo (mgr.24625) 1116 : cluster [DBG] pgmap v625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.315485+0000 mgr.gibba032.whopeo (mgr.24625) 1117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:13:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.317122+0000 mgr.gibba032.whopeo (mgr.24625) 1118 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:13:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.540632+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:13:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.540730+0000 mgr.gibba032.whopeo (mgr.24625) 1119 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:13:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.542684+0000 mgr.gibba032.whopeo (mgr.24625) 1120 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:13:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.543884+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:13:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.544227+0000 mgr.gibba032.whopeo (mgr.24625) 1121 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:13:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.544264+0000 mgr.gibba032.whopeo (mgr.24625) 1122 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:13:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.545481+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.545669+0000 mgr.gibba032.whopeo (mgr.24625) 1123 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:13:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.545707+0000 mgr.gibba032.whopeo (mgr.24625) 1124 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:13:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.546916+0000 mgr.gibba032.whopeo (mgr.24625) 1125 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:13:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.547054+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.547764+0000 mgr.gibba032.whopeo (mgr.24625) 1126 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:13:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.547802+0000 mgr.gibba032.whopeo (mgr.24625) 1127 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:13:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.548402+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.549263+0000 mgr.gibba032.whopeo (mgr.24625) 1128 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:13:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.549591+0000 mgr.gibba032.whopeo (mgr.24625) 1129 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:13:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.550472+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.550803+0000 mgr.gibba032.whopeo (mgr.24625) 1130 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:13:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.550843+0000 mgr.gibba032.whopeo (mgr.24625) 1131 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:13:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: cephadm 2021-03-10T14:13:17.550951+0000 mgr.gibba032.whopeo (mgr.24625) 1132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[31774]: audit 2021-03-10T14:13:17.552297+0000 mon.a (mon.0) 889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:18.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.550843+0000 mgr.gibba032.whopeo (mgr.24625) 1131 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:13:18.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: cephadm 2021-03-10T14:13:17.550951+0000 mgr.gibba032.whopeo (mgr.24625) 1132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:18.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:18 gibba011 bash[30791]: audit 2021-03-10T14:13:17.552297+0000 mon.a (mon.0) 889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cluster 2021-03-10T14:13:17.296413+0000 mgr.gibba032.whopeo (mgr.24625) 1116 : cluster [DBG] pgmap v625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.315485+0000 mgr.gibba032.whopeo (mgr.24625) 1117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:13:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.317122+0000 mgr.gibba032.whopeo (mgr.24625) 1118 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:13:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.540632+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:13:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.540730+0000 mgr.gibba032.whopeo (mgr.24625) 1119 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:13:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.542684+0000 mgr.gibba032.whopeo (mgr.24625) 1120 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:13:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.543884+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:13:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.544227+0000 mgr.gibba032.whopeo (mgr.24625) 1121 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:13:19.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.544264+0000 mgr.gibba032.whopeo (mgr.24625) 1122 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:13:19.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.545481+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:19.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.545669+0000 mgr.gibba032.whopeo (mgr.24625) 1123 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:13:19.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.545707+0000 mgr.gibba032.whopeo (mgr.24625) 1124 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:13:19.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.546916+0000 mgr.gibba032.whopeo (mgr.24625) 1125 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:13:19.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.547054+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:19.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.547764+0000 mgr.gibba032.whopeo (mgr.24625) 1126 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:13:19.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.547802+0000 mgr.gibba032.whopeo (mgr.24625) 1127 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:13:19.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.548402+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.549263+0000 mgr.gibba032.whopeo (mgr.24625) 1128 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.549591+0000 mgr.gibba032.whopeo (mgr.24625) 1129 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.550472+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.550803+0000 mgr.gibba032.whopeo (mgr.24625) 1130 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.550843+0000 mgr.gibba032.whopeo (mgr.24625) 1131 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:13:19.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: cephadm 2021-03-10T14:13:17.550951+0000 mgr.gibba032.whopeo (mgr.24625) 1132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:19.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:18 gibba032 bash[24004]: audit 2021-03-10T14:13:17.552297+0000 mon.a (mon.0) 889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:20 gibba011 bash[30791]: cluster 2021-03-10T14:13:19.297834+0000 mgr.gibba032.whopeo (mgr.24625) 1133 : cluster [DBG] pgmap v626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:20 gibba011 bash[31774]: cluster 2021-03-10T14:13:19.297834+0000 mgr.gibba032.whopeo (mgr.24625) 1133 : cluster [DBG] pgmap v626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:20 gibba032 bash[24004]: cluster 2021-03-10T14:13:19.297834+0000 mgr.gibba032.whopeo (mgr.24625) 1133 : cluster [DBG] pgmap v626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:21.765 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:13:21.993 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:13:21.994 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (20m) 6m ago 25m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:13:21.994 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (25m) 5m ago 25m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:13:21.994 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (20m) 5m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:13:21.995 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (20m) 5m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:13:21.995 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (19m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:13:21.995 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (19m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:13:21.995 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (19m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:13:21.996 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (19m) 6m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:13:21.996 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (25m) 6m ago 25m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:13:21.996 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (25m) 5m ago 25m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:13:21.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (18m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:13:21.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (17m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:13:21.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (17m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:13:21.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (17m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:13:21.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (16m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:13:21.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (16m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:13:21.998 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (15m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:13:21.998 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (15m) 5m ago 25m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:13:21.998 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (24m) 5m ago 25m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:13:21.998 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 23m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:13:22.245 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:13:22.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:13:22.246 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:13:22.246 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:22.246 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:13:22.246 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:13:22.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:22.247 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:13:22.247 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:13:22.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:22.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:13:22.248 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:13:22.248 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:13:22.248 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:13:22.249 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:13:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:22 gibba011 bash[30791]: cluster 2021-03-10T14:13:21.299588+0000 mgr.gibba032.whopeo (mgr.24625) 1134 : cluster [DBG] pgmap v627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:22 gibba011 bash[30791]: audit 2021-03-10T14:13:22.245411+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/920094008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:22 gibba011 bash[31774]: cluster 2021-03-10T14:13:21.299588+0000 mgr.gibba032.whopeo (mgr.24625) 1134 : cluster [DBG] pgmap v627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:22 gibba011 bash[31774]: audit 2021-03-10T14:13:22.245411+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/920094008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:22 gibba032 bash[24004]: cluster 2021-03-10T14:13:21.299588+0000 mgr.gibba032.whopeo (mgr.24625) 1134 : cluster [DBG] pgmap v627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:22 gibba032 bash[24004]: audit 2021-03-10T14:13:22.245411+0000 mon.c (mon.1) 23 : audit [DBG] from='client.? 172.21.2.111:0/920094008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:23.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:23 gibba011 bash[30791]: audit 2021-03-10T14:13:21.754249+0000 mgr.gibba032.whopeo (mgr.24625) 1135 : audit [DBG] from='client.54796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:23.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:23 gibba011 bash[30791]: audit 2021-03-10T14:13:21.979650+0000 mgr.gibba032.whopeo (mgr.24625) 1136 : audit [DBG] from='client.54802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:23.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:23 gibba011 bash[31774]: audit 2021-03-10T14:13:21.754249+0000 mgr.gibba032.whopeo (mgr.24625) 1135 : audit [DBG] from='client.54796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:23.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:23 gibba011 bash[31774]: audit 2021-03-10T14:13:21.979650+0000 mgr.gibba032.whopeo (mgr.24625) 1136 : audit [DBG] from='client.54802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:24.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:23 gibba032 bash[24004]: audit 2021-03-10T14:13:21.754249+0000 mgr.gibba032.whopeo (mgr.24625) 1135 : audit [DBG] from='client.54796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:24.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:23 gibba032 bash[24004]: audit 2021-03-10T14:13:21.979650+0000 mgr.gibba032.whopeo (mgr.24625) 1136 : audit [DBG] from='client.54802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:24 gibba011 bash[30791]: cluster 2021-03-10T14:13:23.301190+0000 mgr.gibba032.whopeo (mgr.24625) 1137 : cluster [DBG] pgmap v628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:24 gibba011 bash[31774]: cluster 2021-03-10T14:13:23.301190+0000 mgr.gibba032.whopeo (mgr.24625) 1137 : cluster [DBG] pgmap v628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:24 gibba032 bash[24004]: cluster 2021-03-10T14:13:23.301190+0000 mgr.gibba032.whopeo (mgr.24625) 1137 : cluster [DBG] pgmap v628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:26 gibba032 bash[24004]: cluster 2021-03-10T14:13:25.302803+0000 mgr.gibba032.whopeo (mgr.24625) 1138 : cluster [DBG] pgmap v629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:27.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:26 gibba011 bash[30791]: cluster 2021-03-10T14:13:25.302803+0000 mgr.gibba032.whopeo (mgr.24625) 1138 : cluster [DBG] pgmap v629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:26 gibba011 bash[31774]: cluster 2021-03-10T14:13:25.302803+0000 mgr.gibba032.whopeo (mgr.24625) 1138 : cluster [DBG] pgmap v629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:28 gibba032 bash[24004]: cluster 2021-03-10T14:13:27.304432+0000 mgr.gibba032.whopeo (mgr.24625) 1139 : cluster [DBG] pgmap v630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:28 gibba011 bash[30791]: cluster 2021-03-10T14:13:27.304432+0000 mgr.gibba032.whopeo (mgr.24625) 1139 : cluster [DBG] pgmap v630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:28 gibba011 bash[31774]: cluster 2021-03-10T14:13:27.304432+0000 mgr.gibba032.whopeo (mgr.24625) 1139 : cluster [DBG] pgmap v630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:30 gibba032 bash[24004]: cluster 2021-03-10T14:13:29.305712+0000 mgr.gibba032.whopeo (mgr.24625) 1140 : cluster [DBG] pgmap v631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:30 gibba011 bash[30791]: cluster 2021-03-10T14:13:29.305712+0000 mgr.gibba032.whopeo (mgr.24625) 1140 : cluster [DBG] pgmap v631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:30 gibba011 bash[31774]: cluster 2021-03-10T14:13:29.305712+0000 mgr.gibba032.whopeo (mgr.24625) 1140 : cluster [DBG] pgmap v631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:32 gibba032 bash[24004]: cluster 2021-03-10T14:13:31.307341+0000 mgr.gibba032.whopeo (mgr.24625) 1141 : cluster [DBG] pgmap v632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:32 gibba011 bash[30791]: cluster 2021-03-10T14:13:31.307341+0000 mgr.gibba032.whopeo (mgr.24625) 1141 : cluster [DBG] pgmap v632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:32 gibba011 bash[31774]: cluster 2021-03-10T14:13:31.307341+0000 mgr.gibba032.whopeo (mgr.24625) 1141 : cluster [DBG] pgmap v632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:34.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:33 gibba032 bash[24004]: cephadm 2021-03-10T14:13:32.566952+0000 mgr.gibba032.whopeo (mgr.24625) 1142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:34.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:33 gibba011 bash[30791]: cephadm 2021-03-10T14:13:32.566952+0000 mgr.gibba032.whopeo (mgr.24625) 1142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:34.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:33 gibba011 bash[31774]: cephadm 2021-03-10T14:13:32.566952+0000 mgr.gibba032.whopeo (mgr.24625) 1142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:34 gibba032 bash[24004]: cluster 2021-03-10T14:13:33.308729+0000 mgr.gibba032.whopeo (mgr.24625) 1143 : cluster [DBG] pgmap v633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:34 gibba011 bash[30791]: cluster 2021-03-10T14:13:33.308729+0000 mgr.gibba032.whopeo (mgr.24625) 1143 : cluster [DBG] pgmap v633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:34 gibba011 bash[31774]: cluster 2021-03-10T14:13:33.308729+0000 mgr.gibba032.whopeo (mgr.24625) 1143 : cluster [DBG] pgmap v633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:36 gibba032 bash[24004]: cluster 2021-03-10T14:13:35.310332+0000 mgr.gibba032.whopeo (mgr.24625) 1144 : cluster [DBG] pgmap v634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:37.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:36 gibba011 bash[30791]: cluster 2021-03-10T14:13:35.310332+0000 mgr.gibba032.whopeo (mgr.24625) 1144 : cluster [DBG] pgmap v634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:36 gibba011 bash[31774]: cluster 2021-03-10T14:13:35.310332+0000 mgr.gibba032.whopeo (mgr.24625) 1144 : cluster [DBG] pgmap v634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:38 gibba032 bash[24004]: cluster 2021-03-10T14:13:37.312133+0000 mgr.gibba032.whopeo (mgr.24625) 1145 : cluster [DBG] pgmap v635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:38 gibba011 bash[30791]: cluster 2021-03-10T14:13:37.312133+0000 mgr.gibba032.whopeo (mgr.24625) 1145 : cluster [DBG] pgmap v635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:38 gibba011 bash[31774]: cluster 2021-03-10T14:13:37.312133+0000 mgr.gibba032.whopeo (mgr.24625) 1145 : cluster [DBG] pgmap v635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:40 gibba032 bash[24004]: cluster 2021-03-10T14:13:39.313347+0000 mgr.gibba032.whopeo (mgr.24625) 1146 : cluster [DBG] pgmap v636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:40 gibba011 bash[30791]: cluster 2021-03-10T14:13:39.313347+0000 mgr.gibba032.whopeo (mgr.24625) 1146 : cluster [DBG] pgmap v636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:40 gibba011 bash[31774]: cluster 2021-03-10T14:13:39.313347+0000 mgr.gibba032.whopeo (mgr.24625) 1146 : cluster [DBG] pgmap v636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:42 gibba032 bash[24004]: cluster 2021-03-10T14:13:41.314879+0000 mgr.gibba032.whopeo (mgr.24625) 1147 : cluster [DBG] pgmap v637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:42 gibba011 bash[30791]: cluster 2021-03-10T14:13:41.314879+0000 mgr.gibba032.whopeo (mgr.24625) 1147 : cluster [DBG] pgmap v637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:42 gibba011 bash[31774]: cluster 2021-03-10T14:13:41.314879+0000 mgr.gibba032.whopeo (mgr.24625) 1147 : cluster [DBG] pgmap v637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:44 gibba032 bash[24004]: cluster 2021-03-10T14:13:43.316558+0000 mgr.gibba032.whopeo (mgr.24625) 1148 : cluster [DBG] pgmap v638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:44 gibba011 bash[30791]: cluster 2021-03-10T14:13:43.316558+0000 mgr.gibba032.whopeo (mgr.24625) 1148 : cluster [DBG] pgmap v638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:44 gibba011 bash[31774]: cluster 2021-03-10T14:13:43.316558+0000 mgr.gibba032.whopeo (mgr.24625) 1148 : cluster [DBG] pgmap v638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:46 gibba032 bash[24004]: cluster 2021-03-10T14:13:45.318144+0000 mgr.gibba032.whopeo (mgr.24625) 1149 : cluster [DBG] pgmap v639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:46 gibba011 bash[30791]: cluster 2021-03-10T14:13:45.318144+0000 mgr.gibba032.whopeo (mgr.24625) 1149 : cluster [DBG] pgmap v639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:46 gibba011 bash[31774]: cluster 2021-03-10T14:13:45.318144+0000 mgr.gibba032.whopeo (mgr.24625) 1149 : cluster [DBG] pgmap v639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:48 gibba032 bash[24004]: cluster 2021-03-10T14:13:47.319764+0000 mgr.gibba032.whopeo (mgr.24625) 1150 : cluster [DBG] pgmap v640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:48 gibba032 bash[24004]: cephadm 2021-03-10T14:13:47.583018+0000 mgr.gibba032.whopeo (mgr.24625) 1151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:49.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:48 gibba011 bash[30791]: cluster 2021-03-10T14:13:47.319764+0000 mgr.gibba032.whopeo (mgr.24625) 1150 : cluster [DBG] pgmap v640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:48 gibba011 bash[30791]: cephadm 2021-03-10T14:13:47.583018+0000 mgr.gibba032.whopeo (mgr.24625) 1151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:48 gibba011 bash[31774]: cluster 2021-03-10T14:13:47.319764+0000 mgr.gibba032.whopeo (mgr.24625) 1150 : cluster [DBG] pgmap v640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:48 gibba011 bash[31774]: cephadm 2021-03-10T14:13:47.583018+0000 mgr.gibba032.whopeo (mgr.24625) 1151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:13:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:50 gibba032 bash[24004]: cluster 2021-03-10T14:13:49.321171+0000 mgr.gibba032.whopeo (mgr.24625) 1152 : cluster [DBG] pgmap v641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:50 gibba011 bash[30791]: cluster 2021-03-10T14:13:49.321171+0000 mgr.gibba032.whopeo (mgr.24625) 1152 : cluster [DBG] pgmap v641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:50 gibba011 bash[31774]: cluster 2021-03-10T14:13:49.321171+0000 mgr.gibba032.whopeo (mgr.24625) 1152 : cluster [DBG] pgmap v641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:52.489 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:13:52.717 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:13:52.718 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (20m) 6m ago 25m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:13:52.718 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (25m) 5m ago 25m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:13:52.718 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (20m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:13:52.719 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (21m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:13:52.719 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (20m) 6m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:13:52.719 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (20m) 6m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:13:52.719 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (19m) 5m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:13:52.720 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (19m) 6m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:13:52.720 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (26m) 6m ago 26m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:13:52.720 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (26m) 5m ago 26m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:13:52.720 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (18m) 6m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:13:52.721 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (18m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:13:52.721 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (17m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:13:52.721 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (17m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:13:52.721 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (17m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:13:52.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (16m) 5m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:13:52.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (16m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:13:52.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (15m) 5m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:13:52.722 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (25m) 5m ago 26m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:13:52.723 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 23m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:13:52.969 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:13:52.969 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:13:52.969 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:13:52.970 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:52.970 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:13:52.970 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:13:52.971 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:52.971 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:13:52.971 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:13:52.971 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:13:52.972 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:13:52.972 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:13:52.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:13:52.973 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:13:52.973 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:13:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:52 gibba032 bash[24004]: cluster 2021-03-10T14:13:51.322648+0000 mgr.gibba032.whopeo (mgr.24625) 1153 : cluster [DBG] pgmap v642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:53.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:52 gibba011 bash[30791]: cluster 2021-03-10T14:13:51.322648+0000 mgr.gibba032.whopeo (mgr.24625) 1153 : cluster [DBG] pgmap v642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:52 gibba011 bash[31774]: cluster 2021-03-10T14:13:51.322648+0000 mgr.gibba032.whopeo (mgr.24625) 1153 : cluster [DBG] pgmap v642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:54.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:53 gibba032 bash[24004]: audit 2021-03-10T14:13:52.477585+0000 mgr.gibba032.whopeo (mgr.24625) 1154 : audit [DBG] from='client.44829 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:53 gibba032 bash[24004]: audit 2021-03-10T14:13:52.702951+0000 mgr.gibba032.whopeo (mgr.24625) 1155 : audit [DBG] from='client.54820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:53 gibba032 bash[24004]: audit 2021-03-10T14:13:52.968719+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/3668542012' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[30791]: audit 2021-03-10T14:13:52.477585+0000 mgr.gibba032.whopeo (mgr.24625) 1154 : audit [DBG] from='client.44829 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[30791]: audit 2021-03-10T14:13:52.702951+0000 mgr.gibba032.whopeo (mgr.24625) 1155 : audit [DBG] from='client.54820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[30791]: audit 2021-03-10T14:13:52.968719+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/3668542012' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:54.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[31774]: audit 2021-03-10T14:13:52.477585+0000 mgr.gibba032.whopeo (mgr.24625) 1154 : audit [DBG] from='client.44829 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[31774]: audit 2021-03-10T14:13:52.702951+0000 mgr.gibba032.whopeo (mgr.24625) 1155 : audit [DBG] from='client.54820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:13:54.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:53 gibba011 bash[31774]: audit 2021-03-10T14:13:52.968719+0000 mon.c (mon.1) 24 : audit [DBG] from='client.? 172.21.2.111:0/3668542012' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:13:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:54 gibba032 bash[24004]: cluster 2021-03-10T14:13:53.324121+0000 mgr.gibba032.whopeo (mgr.24625) 1156 : cluster [DBG] pgmap v643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:54 gibba011 bash[30791]: cluster 2021-03-10T14:13:53.324121+0000 mgr.gibba032.whopeo (mgr.24625) 1156 : cluster [DBG] pgmap v643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:54 gibba011 bash[31774]: cluster 2021-03-10T14:13:53.324121+0000 mgr.gibba032.whopeo (mgr.24625) 1156 : cluster [DBG] pgmap v643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: cluster 2021-03-10T14:13:55.325420+0000 mgr.gibba032.whopeo (mgr.24625) 1157 : cluster [DBG] pgmap v644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.050437+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:13:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.051614+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.052691+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:13:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.053761+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.133754+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:13:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:56 gibba032 bash[24004]: audit 2021-03-10T14:13:56.134686+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:13:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: cluster 2021-03-10T14:13:55.325420+0000 mgr.gibba032.whopeo (mgr.24625) 1157 : cluster [DBG] pgmap v644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.050437+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:13:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.051614+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.052691+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:13:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.053761+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.133754+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:13:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[30791]: audit 2021-03-10T14:13:56.134686+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:13:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: cluster 2021-03-10T14:13:55.325420+0000 mgr.gibba032.whopeo (mgr.24625) 1157 : cluster [DBG] pgmap v644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.050437+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:13:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.051614+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.052691+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:13:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.053761+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:13:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.133754+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:13:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:56 gibba011 bash[31774]: audit 2021-03-10T14:13:56.134686+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:13:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:13:58 gibba032 bash[24004]: cluster 2021-03-10T14:13:57.327218+0000 mgr.gibba032.whopeo (mgr.24625) 1158 : cluster [DBG] pgmap v645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:13:58 gibba011 bash[30791]: cluster 2021-03-10T14:13:57.327218+0000 mgr.gibba032.whopeo (mgr.24625) 1158 : cluster [DBG] pgmap v645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:13:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:13:58 gibba011 bash[31774]: cluster 2021-03-10T14:13:57.327218+0000 mgr.gibba032.whopeo (mgr.24625) 1158 : cluster [DBG] pgmap v645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:00 gibba032 bash[24004]: cluster 2021-03-10T14:13:59.328727+0000 mgr.gibba032.whopeo (mgr.24625) 1159 : cluster [DBG] pgmap v646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:00 gibba011 bash[30791]: cluster 2021-03-10T14:13:59.328727+0000 mgr.gibba032.whopeo (mgr.24625) 1159 : cluster [DBG] pgmap v646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:00 gibba011 bash[31774]: cluster 2021-03-10T14:13:59.328727+0000 mgr.gibba032.whopeo (mgr.24625) 1159 : cluster [DBG] pgmap v646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:01 gibba011 bash[30791]: cluster 2021-03-10T14:14:01.330272+0000 mgr.gibba032.whopeo (mgr.24625) 1160 : cluster [DBG] pgmap v647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:01 gibba011 bash[31774]: cluster 2021-03-10T14:14:01.330272+0000 mgr.gibba032.whopeo (mgr.24625) 1160 : cluster [DBG] pgmap v647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:01 gibba032 bash[24004]: cluster 2021-03-10T14:14:01.330272+0000 mgr.gibba032.whopeo (mgr.24625) 1160 : cluster [DBG] pgmap v647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:02 gibba032 bash[24004]: cephadm 2021-03-10T14:14:02.591043+0000 mgr.gibba032.whopeo (mgr.24625) 1161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:02 gibba011 bash[30791]: cephadm 2021-03-10T14:14:02.591043+0000 mgr.gibba032.whopeo (mgr.24625) 1161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:02 gibba011 bash[31774]: cephadm 2021-03-10T14:14:02.591043+0000 mgr.gibba032.whopeo (mgr.24625) 1161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:04.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:03 gibba011 bash[30791]: cluster 2021-03-10T14:14:03.331872+0000 mgr.gibba032.whopeo (mgr.24625) 1162 : cluster [DBG] pgmap v648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:04.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:03 gibba011 bash[31774]: cluster 2021-03-10T14:14:03.331872+0000 mgr.gibba032.whopeo (mgr.24625) 1162 : cluster [DBG] pgmap v648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:04.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:03 gibba032 bash[24004]: cluster 2021-03-10T14:14:03.331872+0000 mgr.gibba032.whopeo (mgr.24625) 1162 : cluster [DBG] pgmap v648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:06 gibba032 bash[24004]: cluster 2021-03-10T14:14:05.333301+0000 mgr.gibba032.whopeo (mgr.24625) 1163 : cluster [DBG] pgmap v649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:06.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:06 gibba011 bash[30791]: cluster 2021-03-10T14:14:05.333301+0000 mgr.gibba032.whopeo (mgr.24625) 1163 : cluster [DBG] pgmap v649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:06 gibba011 bash[31774]: cluster 2021-03-10T14:14:05.333301+0000 mgr.gibba032.whopeo (mgr.24625) 1163 : cluster [DBG] pgmap v649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:08 gibba032 bash[24004]: cluster 2021-03-10T14:14:07.335081+0000 mgr.gibba032.whopeo (mgr.24625) 1164 : cluster [DBG] pgmap v650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:08 gibba011 bash[30791]: cluster 2021-03-10T14:14:07.335081+0000 mgr.gibba032.whopeo (mgr.24625) 1164 : cluster [DBG] pgmap v650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:08 gibba011 bash[31774]: cluster 2021-03-10T14:14:07.335081+0000 mgr.gibba032.whopeo (mgr.24625) 1164 : cluster [DBG] pgmap v650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:10 gibba032 bash[24004]: cluster 2021-03-10T14:14:09.336637+0000 mgr.gibba032.whopeo (mgr.24625) 1165 : cluster [DBG] pgmap v651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:10 gibba011 bash[30791]: cluster 2021-03-10T14:14:09.336637+0000 mgr.gibba032.whopeo (mgr.24625) 1165 : cluster [DBG] pgmap v651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:10 gibba011 bash[31774]: cluster 2021-03-10T14:14:09.336637+0000 mgr.gibba032.whopeo (mgr.24625) 1165 : cluster [DBG] pgmap v651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:12 gibba032 bash[24004]: cluster 2021-03-10T14:14:11.338090+0000 mgr.gibba032.whopeo (mgr.24625) 1166 : cluster [DBG] pgmap v652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:12 gibba011 bash[30791]: cluster 2021-03-10T14:14:11.338090+0000 mgr.gibba032.whopeo (mgr.24625) 1166 : cluster [DBG] pgmap v652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:12 gibba011 bash[31774]: cluster 2021-03-10T14:14:11.338090+0000 mgr.gibba032.whopeo (mgr.24625) 1166 : cluster [DBG] pgmap v652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:14 gibba032 bash[24004]: cluster 2021-03-10T14:14:13.339407+0000 mgr.gibba032.whopeo (mgr.24625) 1167 : cluster [DBG] pgmap v653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:14 gibba011 bash[30791]: cluster 2021-03-10T14:14:13.339407+0000 mgr.gibba032.whopeo (mgr.24625) 1167 : cluster [DBG] pgmap v653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:14 gibba011 bash[31774]: cluster 2021-03-10T14:14:13.339407+0000 mgr.gibba032.whopeo (mgr.24625) 1167 : cluster [DBG] pgmap v653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:16 gibba032 bash[24004]: cluster 2021-03-10T14:14:15.341007+0000 mgr.gibba032.whopeo (mgr.24625) 1168 : cluster [DBG] pgmap v654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:16 gibba011 bash[30791]: cluster 2021-03-10T14:14:15.341007+0000 mgr.gibba032.whopeo (mgr.24625) 1168 : cluster [DBG] pgmap v654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:16.858 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:16 gibba011 bash[31774]: cluster 2021-03-10T14:14:15.341007+0000 mgr.gibba032.whopeo (mgr.24625) 1168 : cluster [DBG] pgmap v654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: cluster 2021-03-10T14:14:17.342738+0000 mgr.gibba032.whopeo (mgr.24625) 1169 : cluster [DBG] pgmap v655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:17.801451+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:17.806332+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.011136+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.014309+0000 mon.a (mon.0) 899 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.015289+0000 mon.a (mon.0) 900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.016239+0000 mon.a (mon.0) 901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.017169+0000 mon.a (mon.0) 902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.018889+0000 mon.a (mon.0) 903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:18 gibba032 bash[24004]: audit 2021-03-10T14:14:18.019960+0000 mon.a (mon.0) 904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: cluster 2021-03-10T14:14:17.342738+0000 mgr.gibba032.whopeo (mgr.24625) 1169 : cluster [DBG] pgmap v655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:17.801451+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:17.806332+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.011136+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.014309+0000 mon.a (mon.0) 899 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.015289+0000 mon.a (mon.0) 900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.016239+0000 mon.a (mon.0) 901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.017169+0000 mon.a (mon.0) 902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.018889+0000 mon.a (mon.0) 903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[30791]: audit 2021-03-10T14:14:18.019960+0000 mon.a (mon.0) 904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: cluster 2021-03-10T14:14:17.342738+0000 mgr.gibba032.whopeo (mgr.24625) 1169 : cluster [DBG] pgmap v655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:17.801451+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:17.806332+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.011136+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:14:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.014309+0000 mon.a (mon.0) 899 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:14:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.015289+0000 mon.a (mon.0) 900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.016239+0000 mon.a (mon.0) 901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.017169+0000 mon.a (mon.0) 902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.018889+0000 mon.a (mon.0) 903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:18.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:18 gibba011 bash[31774]: audit 2021-03-10T14:14:18.019960+0000 mon.a (mon.0) 904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:19.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:17.800307+0000 mgr.gibba032.whopeo (mgr.24625) 1170 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:14:19.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:17.804576+0000 mgr.gibba032.whopeo (mgr.24625) 1171 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:14:19.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.011219+0000 mgr.gibba032.whopeo (mgr.24625) 1172 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:14:19.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.012577+0000 mgr.gibba032.whopeo (mgr.24625) 1173 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:14:19.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.013497+0000 mgr.gibba032.whopeo (mgr.24625) 1174 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.013533+0000 mgr.gibba032.whopeo (mgr.24625) 1175 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.014438+0000 mgr.gibba032.whopeo (mgr.24625) 1176 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.014473+0000 mgr.gibba032.whopeo (mgr.24625) 1177 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.015362+0000 mgr.gibba032.whopeo (mgr.24625) 1178 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.016058+0000 mgr.gibba032.whopeo (mgr.24625) 1179 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.016092+0000 mgr.gibba032.whopeo (mgr.24625) 1180 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.016984+0000 mgr.gibba032.whopeo (mgr.24625) 1181 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.017307+0000 mgr.gibba032.whopeo (mgr.24625) 1182 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.018122+0000 mgr.gibba032.whopeo (mgr.24625) 1183 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.018159+0000 mgr.gibba032.whopeo (mgr.24625) 1184 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:14:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:19 gibba032 bash[24004]: cephadm 2021-03-10T14:14:18.018279+0000 mgr.gibba032.whopeo (mgr.24625) 1185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:19.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:17.800307+0000 mgr.gibba032.whopeo (mgr.24625) 1170 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:14:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:17.804576+0000 mgr.gibba032.whopeo (mgr.24625) 1171 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:14:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.011219+0000 mgr.gibba032.whopeo (mgr.24625) 1172 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:14:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.012577+0000 mgr.gibba032.whopeo (mgr.24625) 1173 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:14:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.013497+0000 mgr.gibba032.whopeo (mgr.24625) 1174 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:14:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.013533+0000 mgr.gibba032.whopeo (mgr.24625) 1175 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:14:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.014438+0000 mgr.gibba032.whopeo (mgr.24625) 1176 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:14:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.014473+0000 mgr.gibba032.whopeo (mgr.24625) 1177 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:14:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.015362+0000 mgr.gibba032.whopeo (mgr.24625) 1178 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:14:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.016058+0000 mgr.gibba032.whopeo (mgr.24625) 1179 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:14:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.016092+0000 mgr.gibba032.whopeo (mgr.24625) 1180 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:14:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.016984+0000 mgr.gibba032.whopeo (mgr.24625) 1181 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:14:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.017307+0000 mgr.gibba032.whopeo (mgr.24625) 1182 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:14:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.018122+0000 mgr.gibba032.whopeo (mgr.24625) 1183 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:14:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.018159+0000 mgr.gibba032.whopeo (mgr.24625) 1184 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:14:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[30791]: cephadm 2021-03-10T14:14:18.018279+0000 mgr.gibba032.whopeo (mgr.24625) 1185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:19.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:17.800307+0000 mgr.gibba032.whopeo (mgr.24625) 1170 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:14:19.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:17.804576+0000 mgr.gibba032.whopeo (mgr.24625) 1171 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:14:19.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.011219+0000 mgr.gibba032.whopeo (mgr.24625) 1172 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:14:19.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.012577+0000 mgr.gibba032.whopeo (mgr.24625) 1173 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:14:19.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.013497+0000 mgr.gibba032.whopeo (mgr.24625) 1174 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:14:19.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.013533+0000 mgr.gibba032.whopeo (mgr.24625) 1175 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:14:19.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.014438+0000 mgr.gibba032.whopeo (mgr.24625) 1176 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:14:19.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.014473+0000 mgr.gibba032.whopeo (mgr.24625) 1177 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:14:19.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.015362+0000 mgr.gibba032.whopeo (mgr.24625) 1178 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:14:19.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.016058+0000 mgr.gibba032.whopeo (mgr.24625) 1179 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:14:19.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.016092+0000 mgr.gibba032.whopeo (mgr.24625) 1180 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:14:19.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.016984+0000 mgr.gibba032.whopeo (mgr.24625) 1181 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:14:19.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.017307+0000 mgr.gibba032.whopeo (mgr.24625) 1182 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:14:19.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.018122+0000 mgr.gibba032.whopeo (mgr.24625) 1183 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:14:19.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.018159+0000 mgr.gibba032.whopeo (mgr.24625) 1184 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:14:19.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:19 gibba011 bash[31774]: cephadm 2021-03-10T14:14:18.018279+0000 mgr.gibba032.whopeo (mgr.24625) 1185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:20 gibba032 bash[24004]: cluster 2021-03-10T14:14:19.344215+0000 mgr.gibba032.whopeo (mgr.24625) 1186 : cluster [DBG] pgmap v656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:20 gibba011 bash[30791]: cluster 2021-03-10T14:14:19.344215+0000 mgr.gibba032.whopeo (mgr.24625) 1186 : cluster [DBG] pgmap v656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:20 gibba011 bash[31774]: cluster 2021-03-10T14:14:19.344215+0000 mgr.gibba032.whopeo (mgr.24625) 1186 : cluster [DBG] pgmap v656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:22 gibba032 bash[24004]: cluster 2021-03-10T14:14:21.345761+0000 mgr.gibba032.whopeo (mgr.24625) 1187 : cluster [DBG] pgmap v657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:22 gibba011 bash[30791]: cluster 2021-03-10T14:14:21.345761+0000 mgr.gibba032.whopeo (mgr.24625) 1187 : cluster [DBG] pgmap v657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:22 gibba011 bash[31774]: cluster 2021-03-10T14:14:21.345761+0000 mgr.gibba032.whopeo (mgr.24625) 1187 : cluster [DBG] pgmap v657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:23.221 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:14:23.448 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:14:23.448 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (21m) 7m ago 26m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:14:23.449 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (26m) 6m ago 26m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:14:23.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (21m) 6m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:14:23.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (21m) 6m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:14:23.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (20m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:14:23.449 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (20m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:14:23.450 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (20m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:14:23.450 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (20m) 7m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:14:23.450 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (26m) 7m ago 26m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:14:23.450 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (26m) 6m ago 26m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:14:23.450 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (19m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:14:23.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (18m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:14:23.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (18m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:14:23.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (18m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:14:23.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (17m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:14:23.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (17m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:14:23.452 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (16m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:14:23.452 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (16m) 6m ago 26m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:14:23.452 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (25m) 6m ago 26m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:14:23.452 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 24m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:14:23.700 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:14:23.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:14:23.701 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:14:23.701 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:23.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:14:23.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:14:23.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:23.702 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:14:23.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:14:23.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:23.703 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:14:23.703 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:14:23.703 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:14:23.703 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:14:23.703 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:14:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:24 gibba032 bash[24004]: audit 2021-03-10T14:14:23.209183+0000 mgr.gibba032.whopeo (mgr.24625) 1188 : audit [DBG] from='client.54832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:24 gibba032 bash[24004]: cluster 2021-03-10T14:14:23.347257+0000 mgr.gibba032.whopeo (mgr.24625) 1189 : cluster [DBG] pgmap v658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:24 gibba032 bash[24004]: audit 2021-03-10T14:14:23.433376+0000 mgr.gibba032.whopeo (mgr.24625) 1190 : audit [DBG] from='client.54838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:24 gibba032 bash[24004]: audit 2021-03-10T14:14:23.700684+0000 mon.c (mon.1) 25 : audit [DBG] from='client.? 172.21.2.111:0/912269617' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[30791]: audit 2021-03-10T14:14:23.209183+0000 mgr.gibba032.whopeo (mgr.24625) 1188 : audit [DBG] from='client.54832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[30791]: cluster 2021-03-10T14:14:23.347257+0000 mgr.gibba032.whopeo (mgr.24625) 1189 : cluster [DBG] pgmap v658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[30791]: audit 2021-03-10T14:14:23.433376+0000 mgr.gibba032.whopeo (mgr.24625) 1190 : audit [DBG] from='client.54838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[30791]: audit 2021-03-10T14:14:23.700684+0000 mon.c (mon.1) 25 : audit [DBG] from='client.? 172.21.2.111:0/912269617' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[31774]: audit 2021-03-10T14:14:23.209183+0000 mgr.gibba032.whopeo (mgr.24625) 1188 : audit [DBG] from='client.54832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[31774]: cluster 2021-03-10T14:14:23.347257+0000 mgr.gibba032.whopeo (mgr.24625) 1189 : cluster [DBG] pgmap v658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[31774]: audit 2021-03-10T14:14:23.433376+0000 mgr.gibba032.whopeo (mgr.24625) 1190 : audit [DBG] from='client.54838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:24 gibba011 bash[31774]: audit 2021-03-10T14:14:23.700684+0000 mon.c (mon.1) 25 : audit [DBG] from='client.? 172.21.2.111:0/912269617' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:26 gibba032 bash[24004]: cluster 2021-03-10T14:14:25.348748+0000 mgr.gibba032.whopeo (mgr.24625) 1191 : cluster [DBG] pgmap v659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:26 gibba011 bash[30791]: cluster 2021-03-10T14:14:25.348748+0000 mgr.gibba032.whopeo (mgr.24625) 1191 : cluster [DBG] pgmap v659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:26 gibba011 bash[31774]: cluster 2021-03-10T14:14:25.348748+0000 mgr.gibba032.whopeo (mgr.24625) 1191 : cluster [DBG] pgmap v659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:28 gibba011 bash[30791]: cluster 2021-03-10T14:14:27.350388+0000 mgr.gibba032.whopeo (mgr.24625) 1192 : cluster [DBG] pgmap v660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:28 gibba011 bash[31774]: cluster 2021-03-10T14:14:27.350388+0000 mgr.gibba032.whopeo (mgr.24625) 1192 : cluster [DBG] pgmap v660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:28 gibba032 bash[24004]: cluster 2021-03-10T14:14:27.350388+0000 mgr.gibba032.whopeo (mgr.24625) 1192 : cluster [DBG] pgmap v660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:30 gibba011 bash[30791]: cluster 2021-03-10T14:14:29.351794+0000 mgr.gibba032.whopeo (mgr.24625) 1193 : cluster [DBG] pgmap v661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:30 gibba011 bash[31774]: cluster 2021-03-10T14:14:29.351794+0000 mgr.gibba032.whopeo (mgr.24625) 1193 : cluster [DBG] pgmap v661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:30 gibba032 bash[24004]: cluster 2021-03-10T14:14:29.351794+0000 mgr.gibba032.whopeo (mgr.24625) 1193 : cluster [DBG] pgmap v661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:32 gibba011 bash[30791]: cluster 2021-03-10T14:14:31.353454+0000 mgr.gibba032.whopeo (mgr.24625) 1194 : cluster [DBG] pgmap v662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:32 gibba011 bash[31774]: cluster 2021-03-10T14:14:31.353454+0000 mgr.gibba032.whopeo (mgr.24625) 1194 : cluster [DBG] pgmap v662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:32 gibba032 bash[24004]: cluster 2021-03-10T14:14:31.353454+0000 mgr.gibba032.whopeo (mgr.24625) 1194 : cluster [DBG] pgmap v662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:34 gibba011 bash[30791]: cephadm 2021-03-10T14:14:33.032566+0000 mgr.gibba032.whopeo (mgr.24625) 1195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:34 gibba011 bash[30791]: cluster 2021-03-10T14:14:33.354950+0000 mgr.gibba032.whopeo (mgr.24625) 1196 : cluster [DBG] pgmap v663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:34 gibba011 bash[31774]: cephadm 2021-03-10T14:14:33.032566+0000 mgr.gibba032.whopeo (mgr.24625) 1195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:34 gibba011 bash[31774]: cluster 2021-03-10T14:14:33.354950+0000 mgr.gibba032.whopeo (mgr.24625) 1196 : cluster [DBG] pgmap v663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:34 gibba032 bash[24004]: cephadm 2021-03-10T14:14:33.032566+0000 mgr.gibba032.whopeo (mgr.24625) 1195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:34 gibba032 bash[24004]: cluster 2021-03-10T14:14:33.354950+0000 mgr.gibba032.whopeo (mgr.24625) 1196 : cluster [DBG] pgmap v663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:36 gibba011 bash[30791]: cluster 2021-03-10T14:14:35.356475+0000 mgr.gibba032.whopeo (mgr.24625) 1197 : cluster [DBG] pgmap v664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:36 gibba011 bash[31774]: cluster 2021-03-10T14:14:35.356475+0000 mgr.gibba032.whopeo (mgr.24625) 1197 : cluster [DBG] pgmap v664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:36 gibba032 bash[24004]: cluster 2021-03-10T14:14:35.356475+0000 mgr.gibba032.whopeo (mgr.24625) 1197 : cluster [DBG] pgmap v664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:38 gibba011 bash[30791]: cluster 2021-03-10T14:14:37.358243+0000 mgr.gibba032.whopeo (mgr.24625) 1198 : cluster [DBG] pgmap v665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:38 gibba011 bash[31774]: cluster 2021-03-10T14:14:37.358243+0000 mgr.gibba032.whopeo (mgr.24625) 1198 : cluster [DBG] pgmap v665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:38 gibba032 bash[24004]: cluster 2021-03-10T14:14:37.358243+0000 mgr.gibba032.whopeo (mgr.24625) 1198 : cluster [DBG] pgmap v665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:40 gibba011 bash[30791]: cluster 2021-03-10T14:14:39.359752+0000 mgr.gibba032.whopeo (mgr.24625) 1199 : cluster [DBG] pgmap v666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:40 gibba011 bash[31774]: cluster 2021-03-10T14:14:39.359752+0000 mgr.gibba032.whopeo (mgr.24625) 1199 : cluster [DBG] pgmap v666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:40 gibba032 bash[24004]: cluster 2021-03-10T14:14:39.359752+0000 mgr.gibba032.whopeo (mgr.24625) 1199 : cluster [DBG] pgmap v666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:42 gibba011 bash[30791]: cluster 2021-03-10T14:14:41.361543+0000 mgr.gibba032.whopeo (mgr.24625) 1200 : cluster [DBG] pgmap v667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:42 gibba011 bash[31774]: cluster 2021-03-10T14:14:41.361543+0000 mgr.gibba032.whopeo (mgr.24625) 1200 : cluster [DBG] pgmap v667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:42 gibba032 bash[24004]: cluster 2021-03-10T14:14:41.361543+0000 mgr.gibba032.whopeo (mgr.24625) 1200 : cluster [DBG] pgmap v667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:44 gibba011 bash[30791]: cluster 2021-03-10T14:14:43.363035+0000 mgr.gibba032.whopeo (mgr.24625) 1201 : cluster [DBG] pgmap v668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:44 gibba011 bash[31774]: cluster 2021-03-10T14:14:43.363035+0000 mgr.gibba032.whopeo (mgr.24625) 1201 : cluster [DBG] pgmap v668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:44 gibba032 bash[24004]: cluster 2021-03-10T14:14:43.363035+0000 mgr.gibba032.whopeo (mgr.24625) 1201 : cluster [DBG] pgmap v668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:46 gibba032 bash[24004]: cluster 2021-03-10T14:14:45.364625+0000 mgr.gibba032.whopeo (mgr.24625) 1202 : cluster [DBG] pgmap v669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:46 gibba011 bash[30791]: cluster 2021-03-10T14:14:45.364625+0000 mgr.gibba032.whopeo (mgr.24625) 1202 : cluster [DBG] pgmap v669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:46 gibba011 bash[31774]: cluster 2021-03-10T14:14:45.364625+0000 mgr.gibba032.whopeo (mgr.24625) 1202 : cluster [DBG] pgmap v669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:48 gibba032 bash[24004]: cluster 2021-03-10T14:14:47.366247+0000 mgr.gibba032.whopeo (mgr.24625) 1203 : cluster [DBG] pgmap v670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:48 gibba011 bash[30791]: cluster 2021-03-10T14:14:47.366247+0000 mgr.gibba032.whopeo (mgr.24625) 1203 : cluster [DBG] pgmap v670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:48 gibba011 bash[31774]: cluster 2021-03-10T14:14:47.366247+0000 mgr.gibba032.whopeo (mgr.24625) 1203 : cluster [DBG] pgmap v670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:49 gibba032 bash[24004]: cephadm 2021-03-10T14:14:48.048328+0000 mgr.gibba032.whopeo (mgr.24625) 1204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:49 gibba011 bash[30791]: cephadm 2021-03-10T14:14:48.048328+0000 mgr.gibba032.whopeo (mgr.24625) 1204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:50.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:49 gibba011 bash[31774]: cephadm 2021-03-10T14:14:48.048328+0000 mgr.gibba032.whopeo (mgr.24625) 1204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:14:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:50 gibba032 bash[24004]: cluster 2021-03-10T14:14:49.367769+0000 mgr.gibba032.whopeo (mgr.24625) 1205 : cluster [DBG] pgmap v671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:50 gibba011 bash[30791]: cluster 2021-03-10T14:14:49.367769+0000 mgr.gibba032.whopeo (mgr.24625) 1205 : cluster [DBG] pgmap v671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:50 gibba011 bash[31774]: cluster 2021-03-10T14:14:49.367769+0000 mgr.gibba032.whopeo (mgr.24625) 1205 : cluster [DBG] pgmap v671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:52 gibba032 bash[24004]: cluster 2021-03-10T14:14:51.369440+0000 mgr.gibba032.whopeo (mgr.24625) 1206 : cluster [DBG] pgmap v672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:52 gibba011 bash[30791]: cluster 2021-03-10T14:14:51.369440+0000 mgr.gibba032.whopeo (mgr.24625) 1206 : cluster [DBG] pgmap v672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:52 gibba011 bash[31774]: cluster 2021-03-10T14:14:51.369440+0000 mgr.gibba032.whopeo (mgr.24625) 1206 : cluster [DBG] pgmap v672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:53.943 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:14:54.179 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:14:54.179 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (21m) 7m ago 26m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:14:54.180 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (26m) 6m ago 26m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:14:54.180 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (21m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:14:54.180 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (22m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:14:54.180 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (21m) 7m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:14:54.180 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (21m) 7m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:14:54.181 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (20m) 6m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:14:54.181 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (20m) 7m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:14:54.181 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (27m) 7m ago 27m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:14:54.181 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (27m) 6m ago 27m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:14:54.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (19m) 7m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:14:54.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (19m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:14:54.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (18m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:14:54.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (18m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:14:54.182 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (18m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:14:54.183 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (17m) 6m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:14:54.183 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (17m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:14:54.183 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (16m) 6m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:14:54.183 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (26m) 6m ago 27m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:14:54.184 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 24m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:14:54.428 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:14:54.429 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:14:54.429 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:14:54.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:54.429 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:14:54.430 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:14:54.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:54.430 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:14:54.430 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:14:54.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:14:54.431 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:14:54.431 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:14:54.431 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:14:54.431 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:14:54.431 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:14:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:54 gibba032 bash[24004]: cluster 2021-03-10T14:14:53.370934+0000 mgr.gibba032.whopeo (mgr.24625) 1207 : cluster [DBG] pgmap v673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:54 gibba032 bash[24004]: audit 2021-03-10T14:14:54.428697+0000 mon.c (mon.1) 26 : audit [DBG] from='client.? 172.21.2.111:0/534011614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:54 gibba011 bash[30791]: cluster 2021-03-10T14:14:53.370934+0000 mgr.gibba032.whopeo (mgr.24625) 1207 : cluster [DBG] pgmap v673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:55.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:54 gibba011 bash[30791]: audit 2021-03-10T14:14:54.428697+0000 mon.c (mon.1) 26 : audit [DBG] from='client.? 172.21.2.111:0/534011614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:54 gibba011 bash[31774]: cluster 2021-03-10T14:14:53.370934+0000 mgr.gibba032.whopeo (mgr.24625) 1207 : cluster [DBG] pgmap v673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:55.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:54 gibba011 bash[31774]: audit 2021-03-10T14:14:54.428697+0000 mon.c (mon.1) 26 : audit [DBG] from='client.? 172.21.2.111:0/534011614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:14:56.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:55 gibba032 bash[24004]: audit 2021-03-10T14:14:53.931859+0000 mgr.gibba032.whopeo (mgr.24625) 1208 : audit [DBG] from='client.44865 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:56.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:55 gibba032 bash[24004]: audit 2021-03-10T14:14:54.164659+0000 mgr.gibba032.whopeo (mgr.24625) 1209 : audit [DBG] from='client.44871 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:55 gibba011 bash[30791]: audit 2021-03-10T14:14:53.931859+0000 mgr.gibba032.whopeo (mgr.24625) 1208 : audit [DBG] from='client.44865 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:56.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:55 gibba011 bash[30791]: audit 2021-03-10T14:14:54.164659+0000 mgr.gibba032.whopeo (mgr.24625) 1209 : audit [DBG] from='client.44871 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:55 gibba011 bash[31774]: audit 2021-03-10T14:14:53.931859+0000 mgr.gibba032.whopeo (mgr.24625) 1208 : audit [DBG] from='client.44865 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:55 gibba011 bash[31774]: audit 2021-03-10T14:14:54.164659+0000 mgr.gibba032.whopeo (mgr.24625) 1209 : audit [DBG] from='client.44871 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:14:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: cluster 2021-03-10T14:14:55.372441+0000 mgr.gibba032.whopeo (mgr.24625) 1210 : cluster [DBG] pgmap v674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.062329+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:14:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.063056+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:14:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.063783+0000 mon.a (mon.0) 907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.064477+0000 mon.a (mon.0) 908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.140513+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:14:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:56 gibba032 bash[24004]: audit 2021-03-10T14:14:56.141135+0000 mon.a (mon.0) 910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:14:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: cluster 2021-03-10T14:14:55.372441+0000 mgr.gibba032.whopeo (mgr.24625) 1210 : cluster [DBG] pgmap v674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.062329+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:14:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.063056+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:14:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.063783+0000 mon.a (mon.0) 907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.064477+0000 mon.a (mon.0) 908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.140513+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:14:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[30791]: audit 2021-03-10T14:14:56.141135+0000 mon.a (mon.0) 910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:14:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: cluster 2021-03-10T14:14:55.372441+0000 mgr.gibba032.whopeo (mgr.24625) 1210 : cluster [DBG] pgmap v674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.062329+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:14:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.063056+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:14:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.063783+0000 mon.a (mon.0) 907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.064477+0000 mon.a (mon.0) 908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:14:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.140513+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:14:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:56 gibba011 bash[31774]: audit 2021-03-10T14:14:56.141135+0000 mon.a (mon.0) 910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:14:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:14:58 gibba032 bash[24004]: cluster 2021-03-10T14:14:57.374211+0000 mgr.gibba032.whopeo (mgr.24625) 1211 : cluster [DBG] pgmap v675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:59.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:14:58 gibba011 bash[30791]: cluster 2021-03-10T14:14:57.374211+0000 mgr.gibba032.whopeo (mgr.24625) 1211 : cluster [DBG] pgmap v675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:14:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:14:58 gibba011 bash[31774]: cluster 2021-03-10T14:14:57.374211+0000 mgr.gibba032.whopeo (mgr.24625) 1211 : cluster [DBG] pgmap v675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:00 gibba032 bash[24004]: cluster 2021-03-10T14:14:59.375906+0000 mgr.gibba032.whopeo (mgr.24625) 1212 : cluster [DBG] pgmap v676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:00 gibba011 bash[30791]: cluster 2021-03-10T14:14:59.375906+0000 mgr.gibba032.whopeo (mgr.24625) 1212 : cluster [DBG] pgmap v676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:00 gibba011 bash[31774]: cluster 2021-03-10T14:14:59.375906+0000 mgr.gibba032.whopeo (mgr.24625) 1212 : cluster [DBG] pgmap v676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:02 gibba032 bash[24004]: cluster 2021-03-10T14:15:01.377382+0000 mgr.gibba032.whopeo (mgr.24625) 1213 : cluster [DBG] pgmap v677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:02 gibba011 bash[30791]: cluster 2021-03-10T14:15:01.377382+0000 mgr.gibba032.whopeo (mgr.24625) 1213 : cluster [DBG] pgmap v677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:02 gibba011 bash[31774]: cluster 2021-03-10T14:15:01.377382+0000 mgr.gibba032.whopeo (mgr.24625) 1213 : cluster [DBG] pgmap v677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:04 gibba032 bash[24004]: cephadm 2021-03-10T14:15:03.064296+0000 mgr.gibba032.whopeo (mgr.24625) 1214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:04 gibba032 bash[24004]: cluster 2021-03-10T14:15:03.379127+0000 mgr.gibba032.whopeo (mgr.24625) 1215 : cluster [DBG] pgmap v678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:04 gibba011 bash[30791]: cephadm 2021-03-10T14:15:03.064296+0000 mgr.gibba032.whopeo (mgr.24625) 1214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:04 gibba011 bash[30791]: cluster 2021-03-10T14:15:03.379127+0000 mgr.gibba032.whopeo (mgr.24625) 1215 : cluster [DBG] pgmap v678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:04 gibba011 bash[31774]: cephadm 2021-03-10T14:15:03.064296+0000 mgr.gibba032.whopeo (mgr.24625) 1214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:04 gibba011 bash[31774]: cluster 2021-03-10T14:15:03.379127+0000 mgr.gibba032.whopeo (mgr.24625) 1215 : cluster [DBG] pgmap v678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:06 gibba032 bash[24004]: cluster 2021-03-10T14:15:05.380950+0000 mgr.gibba032.whopeo (mgr.24625) 1216 : cluster [DBG] pgmap v679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:06 gibba011 bash[30791]: cluster 2021-03-10T14:15:05.380950+0000 mgr.gibba032.whopeo (mgr.24625) 1216 : cluster [DBG] pgmap v679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:06 gibba011 bash[31774]: cluster 2021-03-10T14:15:05.380950+0000 mgr.gibba032.whopeo (mgr.24625) 1216 : cluster [DBG] pgmap v679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:08 gibba032 bash[24004]: cluster 2021-03-10T14:15:07.382983+0000 mgr.gibba032.whopeo (mgr.24625) 1217 : cluster [DBG] pgmap v680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:08 gibba011 bash[30791]: cluster 2021-03-10T14:15:07.382983+0000 mgr.gibba032.whopeo (mgr.24625) 1217 : cluster [DBG] pgmap v680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:08 gibba011 bash[31774]: cluster 2021-03-10T14:15:07.382983+0000 mgr.gibba032.whopeo (mgr.24625) 1217 : cluster [DBG] pgmap v680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:10 gibba032 bash[24004]: cluster 2021-03-10T14:15:09.384725+0000 mgr.gibba032.whopeo (mgr.24625) 1218 : cluster [DBG] pgmap v681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:10 gibba011 bash[30791]: cluster 2021-03-10T14:15:09.384725+0000 mgr.gibba032.whopeo (mgr.24625) 1218 : cluster [DBG] pgmap v681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:10 gibba011 bash[31774]: cluster 2021-03-10T14:15:09.384725+0000 mgr.gibba032.whopeo (mgr.24625) 1218 : cluster [DBG] pgmap v681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:12 gibba032 bash[24004]: cluster 2021-03-10T14:15:11.386534+0000 mgr.gibba032.whopeo (mgr.24625) 1219 : cluster [DBG] pgmap v682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:12 gibba011 bash[30791]: cluster 2021-03-10T14:15:11.386534+0000 mgr.gibba032.whopeo (mgr.24625) 1219 : cluster [DBG] pgmap v682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:13.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:12 gibba011 bash[31774]: cluster 2021-03-10T14:15:11.386534+0000 mgr.gibba032.whopeo (mgr.24625) 1219 : cluster [DBG] pgmap v682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:14 gibba032 bash[24004]: cluster 2021-03-10T14:15:13.388328+0000 mgr.gibba032.whopeo (mgr.24625) 1220 : cluster [DBG] pgmap v683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:14 gibba011 bash[30791]: cluster 2021-03-10T14:15:13.388328+0000 mgr.gibba032.whopeo (mgr.24625) 1220 : cluster [DBG] pgmap v683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:14 gibba011 bash[31774]: cluster 2021-03-10T14:15:13.388328+0000 mgr.gibba032.whopeo (mgr.24625) 1220 : cluster [DBG] pgmap v683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:16 gibba032 bash[24004]: cluster 2021-03-10T14:15:15.390065+0000 mgr.gibba032.whopeo (mgr.24625) 1221 : cluster [DBG] pgmap v684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:16 gibba011 bash[30791]: cluster 2021-03-10T14:15:15.390065+0000 mgr.gibba032.whopeo (mgr.24625) 1221 : cluster [DBG] pgmap v684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:16 gibba011 bash[31774]: cluster 2021-03-10T14:15:15.390065+0000 mgr.gibba032.whopeo (mgr.24625) 1221 : cluster [DBG] pgmap v684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: cluster 2021-03-10T14:15:17.391960+0000 mgr.gibba032.whopeo (mgr.24625) 1222 : cluster [DBG] pgmap v685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.324459+0000 mon.a (mon.0) 911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:15:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.328058+0000 mon.a (mon.0) 912 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:15:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.329921+0000 mon.a (mon.0) 913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.331768+0000 mon.a (mon.0) 914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.333467+0000 mon.a (mon.0) 915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.336710+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:18 gibba032 bash[24004]: audit 2021-03-10T14:15:18.338797+0000 mon.a (mon.0) 917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: cluster 2021-03-10T14:15:17.391960+0000 mgr.gibba032.whopeo (mgr.24625) 1222 : cluster [DBG] pgmap v685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.324459+0000 mon.a (mon.0) 911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:15:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.328058+0000 mon.a (mon.0) 912 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:15:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.329921+0000 mon.a (mon.0) 913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.331768+0000 mon.a (mon.0) 914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.333467+0000 mon.a (mon.0) 915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.336710+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[30791]: audit 2021-03-10T14:15:18.338797+0000 mon.a (mon.0) 917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: cluster 2021-03-10T14:15:17.391960+0000 mgr.gibba032.whopeo (mgr.24625) 1222 : cluster [DBG] pgmap v685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:19.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.324459+0000 mon.a (mon.0) 911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:15:19.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.328058+0000 mon.a (mon.0) 912 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:15:19.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.329921+0000 mon.a (mon.0) 913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.331768+0000 mon.a (mon.0) 914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.333467+0000 mon.a (mon.0) 915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.336710+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:19.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:18 gibba011 bash[31774]: audit 2021-03-10T14:15:18.338797+0000 mon.a (mon.0) 917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:20.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.080143+0000 mgr.gibba032.whopeo (mgr.24625) 1223 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:15:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.080727+0000 mgr.gibba032.whopeo (mgr.24625) 1224 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:15:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.324686+0000 mgr.gibba032.whopeo (mgr.24625) 1225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:15:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.326641+0000 mgr.gibba032.whopeo (mgr.24625) 1226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:15:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.328439+0000 mgr.gibba032.whopeo (mgr.24625) 1227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:15:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.328489+0000 mgr.gibba032.whopeo (mgr.24625) 1228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:15:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.330352+0000 mgr.gibba032.whopeo (mgr.24625) 1229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:15:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.330402+0000 mgr.gibba032.whopeo (mgr.24625) 1230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:15:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.331945+0000 mgr.gibba032.whopeo (mgr.24625) 1231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:15:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.333421+0000 mgr.gibba032.whopeo (mgr.24625) 1232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:15:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.333474+0000 mgr.gibba032.whopeo (mgr.24625) 1233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:15:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.335196+0000 mgr.gibba032.whopeo (mgr.24625) 1234 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:15:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.335753+0000 mgr.gibba032.whopeo (mgr.24625) 1235 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:15:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.337282+0000 mgr.gibba032.whopeo (mgr.24625) 1236 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:15:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.337339+0000 mgr.gibba032.whopeo (mgr.24625) 1237 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:15:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:19 gibba032 bash[24004]: cephadm 2021-03-10T14:15:18.337498+0000 mgr.gibba032.whopeo (mgr.24625) 1238 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.080143+0000 mgr.gibba032.whopeo (mgr.24625) 1223 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:15:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.080727+0000 mgr.gibba032.whopeo (mgr.24625) 1224 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:15:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.324686+0000 mgr.gibba032.whopeo (mgr.24625) 1225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:15:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.326641+0000 mgr.gibba032.whopeo (mgr.24625) 1226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:15:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.328439+0000 mgr.gibba032.whopeo (mgr.24625) 1227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:15:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.328489+0000 mgr.gibba032.whopeo (mgr.24625) 1228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:15:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.330352+0000 mgr.gibba032.whopeo (mgr.24625) 1229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:15:20.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.330402+0000 mgr.gibba032.whopeo (mgr.24625) 1230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:15:20.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.331945+0000 mgr.gibba032.whopeo (mgr.24625) 1231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:15:20.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.333421+0000 mgr.gibba032.whopeo (mgr.24625) 1232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:15:20.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.333474+0000 mgr.gibba032.whopeo (mgr.24625) 1233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:15:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.335196+0000 mgr.gibba032.whopeo (mgr.24625) 1234 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:15:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.335753+0000 mgr.gibba032.whopeo (mgr.24625) 1235 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:15:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.337282+0000 mgr.gibba032.whopeo (mgr.24625) 1236 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:15:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.337339+0000 mgr.gibba032.whopeo (mgr.24625) 1237 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:15:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[30791]: cephadm 2021-03-10T14:15:18.337498+0000 mgr.gibba032.whopeo (mgr.24625) 1238 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:20.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.080143+0000 mgr.gibba032.whopeo (mgr.24625) 1223 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:15:20.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.080727+0000 mgr.gibba032.whopeo (mgr.24625) 1224 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:15:20.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.324686+0000 mgr.gibba032.whopeo (mgr.24625) 1225 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:15:20.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.326641+0000 mgr.gibba032.whopeo (mgr.24625) 1226 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:15:20.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.328439+0000 mgr.gibba032.whopeo (mgr.24625) 1227 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:15:20.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.328489+0000 mgr.gibba032.whopeo (mgr.24625) 1228 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:15:20.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.330352+0000 mgr.gibba032.whopeo (mgr.24625) 1229 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:15:20.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.330402+0000 mgr.gibba032.whopeo (mgr.24625) 1230 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:15:20.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.331945+0000 mgr.gibba032.whopeo (mgr.24625) 1231 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:15:20.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.333421+0000 mgr.gibba032.whopeo (mgr.24625) 1232 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:15:20.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.333474+0000 mgr.gibba032.whopeo (mgr.24625) 1233 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:15:20.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.335196+0000 mgr.gibba032.whopeo (mgr.24625) 1234 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:15:20.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.335753+0000 mgr.gibba032.whopeo (mgr.24625) 1235 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:15:20.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.337282+0000 mgr.gibba032.whopeo (mgr.24625) 1236 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:15:20.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.337339+0000 mgr.gibba032.whopeo (mgr.24625) 1237 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:15:20.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:19 gibba011 bash[31774]: cephadm 2021-03-10T14:15:18.337498+0000 mgr.gibba032.whopeo (mgr.24625) 1238 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:20 gibba011 bash[30791]: cluster 2021-03-10T14:15:19.393564+0000 mgr.gibba032.whopeo (mgr.24625) 1239 : cluster [DBG] pgmap v686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:20 gibba011 bash[31774]: cluster 2021-03-10T14:15:19.393564+0000 mgr.gibba032.whopeo (mgr.24625) 1239 : cluster [DBG] pgmap v686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:20 gibba032 bash[24004]: cluster 2021-03-10T14:15:19.393564+0000 mgr.gibba032.whopeo (mgr.24625) 1239 : cluster [DBG] pgmap v686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:23.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:22 gibba011 bash[30791]: cluster 2021-03-10T14:15:21.395336+0000 mgr.gibba032.whopeo (mgr.24625) 1240 : cluster [DBG] pgmap v687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:22 gibba011 bash[31774]: cluster 2021-03-10T14:15:21.395336+0000 mgr.gibba032.whopeo (mgr.24625) 1240 : cluster [DBG] pgmap v687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:23.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:22 gibba032 bash[24004]: cluster 2021-03-10T14:15:21.395336+0000 mgr.gibba032.whopeo (mgr.24625) 1240 : cluster [DBG] pgmap v687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:24.670 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:15:24.900 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:15:24.900 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (22m) 8m ago 27m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:15:24.901 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (27m) 7m ago 27m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:15:24.901 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (22m) 7m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:15:24.902 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (22m) 7m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:15:24.902 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (21m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:15:24.903 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (21m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:15:24.903 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (21m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:15:24.904 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (21m) 8m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:15:24.904 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (27m) 8m ago 27m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:15:24.905 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (27m) 7m ago 27m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:15:24.905 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (20m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:15:24.906 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (19m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:15:24.906 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (19m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:15:24.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (19m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:15:24.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (18m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:15:24.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (18m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:15:24.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (17m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:15:24.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (17m) 7m ago 27m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:15:24.909 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (26m) 7m ago 27m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:15:24.909 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 25m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:15:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:24 gibba011 bash[30791]: cluster 2021-03-10T14:15:23.396863+0000 mgr.gibba032.whopeo (mgr.24625) 1241 : cluster [DBG] pgmap v688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:24 gibba011 bash[31774]: cluster 2021-03-10T14:15:23.396863+0000 mgr.gibba032.whopeo (mgr.24625) 1241 : cluster [DBG] pgmap v688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:25.152 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:15:25.153 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:15:25.154 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:15:25.154 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:25.155 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:15:25.156 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:15:25.156 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:25.157 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:15:25.157 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:15:25.158 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:25.158 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:15:25.159 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:15:25.159 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:15:25.159 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:15:25.160 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:15:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:24 gibba032 bash[24004]: cluster 2021-03-10T14:15:23.396863+0000 mgr.gibba032.whopeo (mgr.24625) 1241 : cluster [DBG] pgmap v688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:26.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:25 gibba011 bash[30791]: audit 2021-03-10T14:15:24.658194+0000 mgr.gibba032.whopeo (mgr.24625) 1242 : audit [DBG] from='client.44883 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:26.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:25 gibba011 bash[30791]: audit 2021-03-10T14:15:25.151903+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.2.111:0/3630940523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:26.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:25 gibba011 bash[31774]: audit 2021-03-10T14:15:24.658194+0000 mgr.gibba032.whopeo (mgr.24625) 1242 : audit [DBG] from='client.44883 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:26.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:25 gibba011 bash[31774]: audit 2021-03-10T14:15:25.151903+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.2.111:0/3630940523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:26.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:25 gibba032 bash[24004]: audit 2021-03-10T14:15:24.658194+0000 mgr.gibba032.whopeo (mgr.24625) 1242 : audit [DBG] from='client.44883 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:26.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:25 gibba032 bash[24004]: audit 2021-03-10T14:15:25.151903+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.2.111:0/3630940523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:26 gibba011 bash[30791]: audit 2021-03-10T14:15:24.885220+0000 mgr.gibba032.whopeo (mgr.24625) 1243 : audit [DBG] from='client.44889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:26 gibba011 bash[30791]: cluster 2021-03-10T14:15:25.398449+0000 mgr.gibba032.whopeo (mgr.24625) 1244 : cluster [DBG] pgmap v689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:26 gibba011 bash[31774]: audit 2021-03-10T14:15:24.885220+0000 mgr.gibba032.whopeo (mgr.24625) 1243 : audit [DBG] from='client.44889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:26 gibba011 bash[31774]: cluster 2021-03-10T14:15:25.398449+0000 mgr.gibba032.whopeo (mgr.24625) 1244 : cluster [DBG] pgmap v689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:26 gibba032 bash[24004]: audit 2021-03-10T14:15:24.885220+0000 mgr.gibba032.whopeo (mgr.24625) 1243 : audit [DBG] from='client.44889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:26 gibba032 bash[24004]: cluster 2021-03-10T14:15:25.398449+0000 mgr.gibba032.whopeo (mgr.24625) 1244 : cluster [DBG] pgmap v689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:27 gibba032 bash[24004]: cluster 2021-03-10T14:15:27.400262+0000 mgr.gibba032.whopeo (mgr.24625) 1245 : cluster [DBG] pgmap v690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:27 gibba011 bash[30791]: cluster 2021-03-10T14:15:27.400262+0000 mgr.gibba032.whopeo (mgr.24625) 1245 : cluster [DBG] pgmap v690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:27 gibba011 bash[31774]: cluster 2021-03-10T14:15:27.400262+0000 mgr.gibba032.whopeo (mgr.24625) 1245 : cluster [DBG] pgmap v690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:30 gibba032 bash[24004]: cluster 2021-03-10T14:15:29.401820+0000 mgr.gibba032.whopeo (mgr.24625) 1246 : cluster [DBG] pgmap v691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:30 gibba011 bash[30791]: cluster 2021-03-10T14:15:29.401820+0000 mgr.gibba032.whopeo (mgr.24625) 1246 : cluster [DBG] pgmap v691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:30 gibba011 bash[31774]: cluster 2021-03-10T14:15:29.401820+0000 mgr.gibba032.whopeo (mgr.24625) 1246 : cluster [DBG] pgmap v691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:32 gibba032 bash[24004]: cluster 2021-03-10T14:15:31.403444+0000 mgr.gibba032.whopeo (mgr.24625) 1247 : cluster [DBG] pgmap v692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:32 gibba011 bash[30791]: cluster 2021-03-10T14:15:31.403444+0000 mgr.gibba032.whopeo (mgr.24625) 1247 : cluster [DBG] pgmap v692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:32 gibba011 bash[31774]: cluster 2021-03-10T14:15:31.403444+0000 mgr.gibba032.whopeo (mgr.24625) 1247 : cluster [DBG] pgmap v692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:34 gibba032 bash[24004]: cephadm 2021-03-10T14:15:33.352140+0000 mgr.gibba032.whopeo (mgr.24625) 1248 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:34 gibba032 bash[24004]: cluster 2021-03-10T14:15:33.404976+0000 mgr.gibba032.whopeo (mgr.24625) 1249 : cluster [DBG] pgmap v693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:34 gibba011 bash[30791]: cephadm 2021-03-10T14:15:33.352140+0000 mgr.gibba032.whopeo (mgr.24625) 1248 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:34 gibba011 bash[30791]: cluster 2021-03-10T14:15:33.404976+0000 mgr.gibba032.whopeo (mgr.24625) 1249 : cluster [DBG] pgmap v693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:34 gibba011 bash[31774]: cephadm 2021-03-10T14:15:33.352140+0000 mgr.gibba032.whopeo (mgr.24625) 1248 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:34 gibba011 bash[31774]: cluster 2021-03-10T14:15:33.404976+0000 mgr.gibba032.whopeo (mgr.24625) 1249 : cluster [DBG] pgmap v693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:36 gibba032 bash[24004]: cluster 2021-03-10T14:15:35.406622+0000 mgr.gibba032.whopeo (mgr.24625) 1250 : cluster [DBG] pgmap v694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:36 gibba011 bash[30791]: cluster 2021-03-10T14:15:35.406622+0000 mgr.gibba032.whopeo (mgr.24625) 1250 : cluster [DBG] pgmap v694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:36 gibba011 bash[31774]: cluster 2021-03-10T14:15:35.406622+0000 mgr.gibba032.whopeo (mgr.24625) 1250 : cluster [DBG] pgmap v694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:38 gibba032 bash[24004]: cluster 2021-03-10T14:15:37.408405+0000 mgr.gibba032.whopeo (mgr.24625) 1251 : cluster [DBG] pgmap v695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:38 gibba011 bash[30791]: cluster 2021-03-10T14:15:37.408405+0000 mgr.gibba032.whopeo (mgr.24625) 1251 : cluster [DBG] pgmap v695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:38 gibba011 bash[31774]: cluster 2021-03-10T14:15:37.408405+0000 mgr.gibba032.whopeo (mgr.24625) 1251 : cluster [DBG] pgmap v695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:40 gibba032 bash[24004]: cluster 2021-03-10T14:15:39.409946+0000 mgr.gibba032.whopeo (mgr.24625) 1252 : cluster [DBG] pgmap v696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:40 gibba011 bash[30791]: cluster 2021-03-10T14:15:39.409946+0000 mgr.gibba032.whopeo (mgr.24625) 1252 : cluster [DBG] pgmap v696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:40 gibba011 bash[31774]: cluster 2021-03-10T14:15:39.409946+0000 mgr.gibba032.whopeo (mgr.24625) 1252 : cluster [DBG] pgmap v696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:42 gibba011 bash[30791]: cluster 2021-03-10T14:15:41.411441+0000 mgr.gibba032.whopeo (mgr.24625) 1253 : cluster [DBG] pgmap v697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:42.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:42 gibba011 bash[31774]: cluster 2021-03-10T14:15:41.411441+0000 mgr.gibba032.whopeo (mgr.24625) 1253 : cluster [DBG] pgmap v697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:42 gibba032 bash[24004]: cluster 2021-03-10T14:15:41.411441+0000 mgr.gibba032.whopeo (mgr.24625) 1253 : cluster [DBG] pgmap v697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:44 gibba011 bash[30791]: cluster 2021-03-10T14:15:43.413010+0000 mgr.gibba032.whopeo (mgr.24625) 1254 : cluster [DBG] pgmap v698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:44 gibba011 bash[31774]: cluster 2021-03-10T14:15:43.413010+0000 mgr.gibba032.whopeo (mgr.24625) 1254 : cluster [DBG] pgmap v698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:44 gibba032 bash[24004]: cluster 2021-03-10T14:15:43.413010+0000 mgr.gibba032.whopeo (mgr.24625) 1254 : cluster [DBG] pgmap v698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:46 gibba011 bash[30791]: cluster 2021-03-10T14:15:45.414365+0000 mgr.gibba032.whopeo (mgr.24625) 1255 : cluster [DBG] pgmap v699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:46 gibba011 bash[31774]: cluster 2021-03-10T14:15:45.414365+0000 mgr.gibba032.whopeo (mgr.24625) 1255 : cluster [DBG] pgmap v699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:46 gibba032 bash[24004]: cluster 2021-03-10T14:15:45.414365+0000 mgr.gibba032.whopeo (mgr.24625) 1255 : cluster [DBG] pgmap v699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:48 gibba032 bash[24004]: cluster 2021-03-10T14:15:47.416085+0000 mgr.gibba032.whopeo (mgr.24625) 1256 : cluster [DBG] pgmap v700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:48 gibba011 bash[30791]: cluster 2021-03-10T14:15:47.416085+0000 mgr.gibba032.whopeo (mgr.24625) 1256 : cluster [DBG] pgmap v700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:48 gibba011 bash[31774]: cluster 2021-03-10T14:15:47.416085+0000 mgr.gibba032.whopeo (mgr.24625) 1256 : cluster [DBG] pgmap v700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:49 gibba011 bash[30791]: cephadm 2021-03-10T14:15:48.364956+0000 mgr.gibba032.whopeo (mgr.24625) 1257 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:49.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:49 gibba011 bash[31774]: cephadm 2021-03-10T14:15:48.364956+0000 mgr.gibba032.whopeo (mgr.24625) 1257 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:49 gibba032 bash[24004]: cephadm 2021-03-10T14:15:48.364956+0000 mgr.gibba032.whopeo (mgr.24625) 1257 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:15:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:50 gibba011 bash[30791]: cluster 2021-03-10T14:15:49.417315+0000 mgr.gibba032.whopeo (mgr.24625) 1258 : cluster [DBG] pgmap v701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:50 gibba011 bash[31774]: cluster 2021-03-10T14:15:49.417315+0000 mgr.gibba032.whopeo (mgr.24625) 1258 : cluster [DBG] pgmap v701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:50 gibba032 bash[24004]: cluster 2021-03-10T14:15:49.417315+0000 mgr.gibba032.whopeo (mgr.24625) 1258 : cluster [DBG] pgmap v701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:52 gibba011 bash[30791]: cluster 2021-03-10T14:15:51.419141+0000 mgr.gibba032.whopeo (mgr.24625) 1259 : cluster [DBG] pgmap v702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:52 gibba011 bash[31774]: cluster 2021-03-10T14:15:51.419141+0000 mgr.gibba032.whopeo (mgr.24625) 1259 : cluster [DBG] pgmap v702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:52 gibba032 bash[24004]: cluster 2021-03-10T14:15:51.419141+0000 mgr.gibba032.whopeo (mgr.24625) 1259 : cluster [DBG] pgmap v702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:54 gibba011 bash[30791]: cluster 2021-03-10T14:15:53.420534+0000 mgr.gibba032.whopeo (mgr.24625) 1260 : cluster [DBG] pgmap v703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:54 gibba011 bash[31774]: cluster 2021-03-10T14:15:53.420534+0000 mgr.gibba032.whopeo (mgr.24625) 1260 : cluster [DBG] pgmap v703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:54 gibba032 bash[24004]: cluster 2021-03-10T14:15:53.420534+0000 mgr.gibba032.whopeo (mgr.24625) 1260 : cluster [DBG] pgmap v703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:55.395 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:15:55.623 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:15:55.623 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (22m) 8m ago 27m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:15:55.623 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (27m) 7m ago 27m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:15:55.623 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (22m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:15:55.624 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (23m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:15:55.624 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (22m) 8m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:15:55.624 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (22m) 8m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:15:55.624 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (21m) 7m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:15:55.625 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (21m) 8m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:15:55.625 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (28m) 8m ago 28m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:15:55.625 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (28m) 7m ago 28m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:15:55.625 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (20m) 8m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:15:55.625 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (20m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:15:55.626 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (20m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:15:55.626 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (19m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:15:55.626 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (19m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:15:55.626 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (18m) 7m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:15:55.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (18m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:15:55.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (17m) 7m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:15:55.627 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (27m) 7m ago 28m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:15:55.627 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 25m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:15:55.871 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:15:55.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:15:55.872 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:15:55.872 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:55.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:15:55.873 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:15:55.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:55.873 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:15:55.873 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:15:55.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:15:55.874 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:15:55.874 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:15:55.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:15:55.874 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:15:55.875 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:15:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:55.383602+0000 mgr.gibba032.whopeo (mgr.24625) 1261 : audit [DBG] from='client.54886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: cluster 2021-03-10T14:15:55.422150+0000 mgr.gibba032.whopeo (mgr.24625) 1262 : cluster [DBG] pgmap v704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:55.608854+0000 mgr.gibba032.whopeo (mgr.24625) 1263 : audit [DBG] from='client.44907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:55.871493+0000 mon.a (mon.0) 919 : audit [DBG] from='client.? 172.21.2.111:0/2667164340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.072274+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:15:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.073064+0000 mon.a (mon.0) 921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.073769+0000 mon.a (mon.0) 922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:15:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.074450+0000 mon.a (mon.0) 923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.147379+0000 mon.a (mon.0) 924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:15:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:56 gibba032 bash[24004]: audit 2021-03-10T14:15:56.148020+0000 mon.a (mon.0) 925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:15:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:55.383602+0000 mgr.gibba032.whopeo (mgr.24625) 1261 : audit [DBG] from='client.54886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: cluster 2021-03-10T14:15:55.422150+0000 mgr.gibba032.whopeo (mgr.24625) 1262 : cluster [DBG] pgmap v704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:55.608854+0000 mgr.gibba032.whopeo (mgr.24625) 1263 : audit [DBG] from='client.44907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:55.871493+0000 mon.a (mon.0) 919 : audit [DBG] from='client.? 172.21.2.111:0/2667164340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.072274+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:15:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.073064+0000 mon.a (mon.0) 921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.073769+0000 mon.a (mon.0) 922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:15:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.074450+0000 mon.a (mon.0) 923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.147379+0000 mon.a (mon.0) 924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:15:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[30791]: audit 2021-03-10T14:15:56.148020+0000 mon.a (mon.0) 925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:15:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:55.383602+0000 mgr.gibba032.whopeo (mgr.24625) 1261 : audit [DBG] from='client.54886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: cluster 2021-03-10T14:15:55.422150+0000 mgr.gibba032.whopeo (mgr.24625) 1262 : cluster [DBG] pgmap v704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:55.608854+0000 mgr.gibba032.whopeo (mgr.24625) 1263 : audit [DBG] from='client.44907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:15:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:55.871493+0000 mon.a (mon.0) 919 : audit [DBG] from='client.? 172.21.2.111:0/2667164340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:15:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.072274+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T14:15:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.073064+0000 mon.a (mon.0) 921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.073769+0000 mon.a (mon.0) 922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:15:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.074450+0000 mon.a (mon.0) 923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:15:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.147379+0000 mon.a (mon.0) 924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:15:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:56 gibba011 bash[31774]: audit 2021-03-10T14:15:56.148020+0000 mon.a (mon.0) 925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:15:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:15:58 gibba032 bash[24004]: cluster 2021-03-10T14:15:57.423963+0000 mgr.gibba032.whopeo (mgr.24625) 1264 : cluster [DBG] pgmap v705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:15:58 gibba011 bash[30791]: cluster 2021-03-10T14:15:57.423963+0000 mgr.gibba032.whopeo (mgr.24625) 1264 : cluster [DBG] pgmap v705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:15:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:15:58 gibba011 bash[31774]: cluster 2021-03-10T14:15:57.423963+0000 mgr.gibba032.whopeo (mgr.24625) 1264 : cluster [DBG] pgmap v705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:00 gibba032 bash[24004]: cluster 2021-03-10T14:15:59.425430+0000 mgr.gibba032.whopeo (mgr.24625) 1265 : cluster [DBG] pgmap v706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:01.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:00 gibba011 bash[30791]: cluster 2021-03-10T14:15:59.425430+0000 mgr.gibba032.whopeo (mgr.24625) 1265 : cluster [DBG] pgmap v706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:00 gibba011 bash[31774]: cluster 2021-03-10T14:15:59.425430+0000 mgr.gibba032.whopeo (mgr.24625) 1265 : cluster [DBG] pgmap v706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:02 gibba032 bash[24004]: cluster 2021-03-10T14:16:01.427320+0000 mgr.gibba032.whopeo (mgr.24625) 1266 : cluster [DBG] pgmap v707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:02 gibba011 bash[30791]: cluster 2021-03-10T14:16:01.427320+0000 mgr.gibba032.whopeo (mgr.24625) 1266 : cluster [DBG] pgmap v707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:02 gibba011 bash[31774]: cluster 2021-03-10T14:16:01.427320+0000 mgr.gibba032.whopeo (mgr.24625) 1266 : cluster [DBG] pgmap v707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:04 gibba032 bash[24004]: cephadm 2021-03-10T14:16:03.381008+0000 mgr.gibba032.whopeo (mgr.24625) 1267 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:05.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:04 gibba032 bash[24004]: cluster 2021-03-10T14:16:03.428665+0000 mgr.gibba032.whopeo (mgr.24625) 1268 : cluster [DBG] pgmap v708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:04 gibba011 bash[30791]: cephadm 2021-03-10T14:16:03.381008+0000 mgr.gibba032.whopeo (mgr.24625) 1267 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:04 gibba011 bash[30791]: cluster 2021-03-10T14:16:03.428665+0000 mgr.gibba032.whopeo (mgr.24625) 1268 : cluster [DBG] pgmap v708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:04 gibba011 bash[31774]: cephadm 2021-03-10T14:16:03.381008+0000 mgr.gibba032.whopeo (mgr.24625) 1267 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:04 gibba011 bash[31774]: cluster 2021-03-10T14:16:03.428665+0000 mgr.gibba032.whopeo (mgr.24625) 1268 : cluster [DBG] pgmap v708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:07.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:06 gibba032 bash[24004]: cluster 2021-03-10T14:16:05.430224+0000 mgr.gibba032.whopeo (mgr.24625) 1269 : cluster [DBG] pgmap v709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:06 gibba011 bash[30791]: cluster 2021-03-10T14:16:05.430224+0000 mgr.gibba032.whopeo (mgr.24625) 1269 : cluster [DBG] pgmap v709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:06 gibba011 bash[31774]: cluster 2021-03-10T14:16:05.430224+0000 mgr.gibba032.whopeo (mgr.24625) 1269 : cluster [DBG] pgmap v709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:08 gibba032 bash[24004]: cluster 2021-03-10T14:16:07.431893+0000 mgr.gibba032.whopeo (mgr.24625) 1270 : cluster [DBG] pgmap v710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:08 gibba011 bash[30791]: cluster 2021-03-10T14:16:07.431893+0000 mgr.gibba032.whopeo (mgr.24625) 1270 : cluster [DBG] pgmap v710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:08 gibba011 bash[31774]: cluster 2021-03-10T14:16:07.431893+0000 mgr.gibba032.whopeo (mgr.24625) 1270 : cluster [DBG] pgmap v710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:10 gibba032 bash[24004]: cluster 2021-03-10T14:16:09.433298+0000 mgr.gibba032.whopeo (mgr.24625) 1271 : cluster [DBG] pgmap v711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:10 gibba011 bash[30791]: cluster 2021-03-10T14:16:09.433298+0000 mgr.gibba032.whopeo (mgr.24625) 1271 : cluster [DBG] pgmap v711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:10 gibba011 bash[31774]: cluster 2021-03-10T14:16:09.433298+0000 mgr.gibba032.whopeo (mgr.24625) 1271 : cluster [DBG] pgmap v711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:12 gibba032 bash[24004]: cluster 2021-03-10T14:16:11.434822+0000 mgr.gibba032.whopeo (mgr.24625) 1272 : cluster [DBG] pgmap v712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:12 gibba011 bash[30791]: cluster 2021-03-10T14:16:11.434822+0000 mgr.gibba032.whopeo (mgr.24625) 1272 : cluster [DBG] pgmap v712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:12 gibba011 bash[31774]: cluster 2021-03-10T14:16:11.434822+0000 mgr.gibba032.whopeo (mgr.24625) 1272 : cluster [DBG] pgmap v712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:14 gibba032 bash[24004]: cluster 2021-03-10T14:16:13.436503+0000 mgr.gibba032.whopeo (mgr.24625) 1273 : cluster [DBG] pgmap v713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:14 gibba011 bash[30791]: cluster 2021-03-10T14:16:13.436503+0000 mgr.gibba032.whopeo (mgr.24625) 1273 : cluster [DBG] pgmap v713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:14 gibba011 bash[31774]: cluster 2021-03-10T14:16:13.436503+0000 mgr.gibba032.whopeo (mgr.24625) 1273 : cluster [DBG] pgmap v713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:16 gibba032 bash[24004]: cluster 2021-03-10T14:16:15.438096+0000 mgr.gibba032.whopeo (mgr.24625) 1274 : cluster [DBG] pgmap v714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:16 gibba011 bash[30791]: cluster 2021-03-10T14:16:15.438096+0000 mgr.gibba032.whopeo (mgr.24625) 1274 : cluster [DBG] pgmap v714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:16 gibba011 bash[31774]: cluster 2021-03-10T14:16:15.438096+0000 mgr.gibba032.whopeo (mgr.24625) 1274 : cluster [DBG] pgmap v714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:18 gibba032 bash[24004]: cluster 2021-03-10T14:16:17.439861+0000 mgr.gibba032.whopeo (mgr.24625) 1275 : cluster [DBG] pgmap v715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:18 gibba032 bash[24004]: audit 2021-03-10T14:16:18.402002+0000 mon.a (mon.0) 926 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:16:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:18 gibba011 bash[30791]: cluster 2021-03-10T14:16:17.439861+0000 mgr.gibba032.whopeo (mgr.24625) 1275 : cluster [DBG] pgmap v715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:18 gibba011 bash[30791]: audit 2021-03-10T14:16:18.402002+0000 mon.a (mon.0) 926 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:16:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:18 gibba011 bash[31774]: cluster 2021-03-10T14:16:17.439861+0000 mgr.gibba032.whopeo (mgr.24625) 1275 : cluster [DBG] pgmap v715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:18 gibba011 bash[31774]: audit 2021-03-10T14:16:18.402002+0000 mon.a (mon.0) 926 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:16:20.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:19 gibba032 bash[24004]: cephadm 2021-03-10T14:16:18.398773+0000 mgr.gibba032.whopeo (mgr.24625) 1276 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:16:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:19 gibba011 bash[30791]: cephadm 2021-03-10T14:16:18.398773+0000 mgr.gibba032.whopeo (mgr.24625) 1276 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:16:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:19 gibba011 bash[31774]: cephadm 2021-03-10T14:16:18.398773+0000 mgr.gibba032.whopeo (mgr.24625) 1276 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:16:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:20 gibba032 bash[24004]: cluster 2021-03-10T14:16:19.441412+0000 mgr.gibba032.whopeo (mgr.24625) 1277 : cluster [DBG] pgmap v716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:20 gibba011 bash[30791]: cluster 2021-03-10T14:16:19.441412+0000 mgr.gibba032.whopeo (mgr.24625) 1277 : cluster [DBG] pgmap v716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:20 gibba011 bash[31774]: cluster 2021-03-10T14:16:19.441412+0000 mgr.gibba032.whopeo (mgr.24625) 1277 : cluster [DBG] pgmap v716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:22 gibba032 bash[24004]: cluster 2021-03-10T14:16:21.443209+0000 mgr.gibba032.whopeo (mgr.24625) 1278 : cluster [DBG] pgmap v717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:23.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:22 gibba011 bash[30791]: cluster 2021-03-10T14:16:21.443209+0000 mgr.gibba032.whopeo (mgr.24625) 1278 : cluster [DBG] pgmap v717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:22 gibba011 bash[31774]: cluster 2021-03-10T14:16:21.443209+0000 mgr.gibba032.whopeo (mgr.24625) 1278 : cluster [DBG] pgmap v717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:22.903754+0000 mon.a (mon.0) 927 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:16:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.090315+0000 mon.a (mon.0) 928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.344038+0000 mon.a (mon.0) 929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.353034+0000 mon.a (mon.0) 930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:16:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.354866+0000 mon.a (mon.0) 931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.356313+0000 mon.a (mon.0) 932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.357777+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.360816+0000 mon.a (mon.0) 934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:23 gibba032 bash[24004]: audit 2021-03-10T14:16:23.362659+0000 mon.a (mon.0) 935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:22.903754+0000 mon.a (mon.0) 927 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:16:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.090315+0000 mon.a (mon.0) 928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.344038+0000 mon.a (mon.0) 929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.353034+0000 mon.a (mon.0) 930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:16:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.354866+0000 mon.a (mon.0) 931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.356313+0000 mon.a (mon.0) 932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.357777+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.360816+0000 mon.a (mon.0) 934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[30791]: audit 2021-03-10T14:16:23.362659+0000 mon.a (mon.0) 935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:22.903754+0000 mon.a (mon.0) 927 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:16:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.090315+0000 mon.a (mon.0) 928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.344038+0000 mon.a (mon.0) 929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:16:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.353034+0000 mon.a (mon.0) 930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:16:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.354866+0000 mon.a (mon.0) 931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.356313+0000 mon.a (mon.0) 932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.357777+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.360816+0000 mon.a (mon.0) 934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:24.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:23 gibba011 bash[31774]: audit 2021-03-10T14:16:23.362659+0000 mon.a (mon.0) 935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:25.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.088917+0000 mgr.gibba032.whopeo (mgr.24625) 1279 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:16:25.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.349219+0000 mgr.gibba032.whopeo (mgr.24625) 1280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:16:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.351555+0000 mgr.gibba032.whopeo (mgr.24625) 1281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:16:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.352998+0000 mgr.gibba032.whopeo (mgr.24625) 1282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:16:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.353037+0000 mgr.gibba032.whopeo (mgr.24625) 1283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:16:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.354515+0000 mgr.gibba032.whopeo (mgr.24625) 1284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:16:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.354555+0000 mgr.gibba032.whopeo (mgr.24625) 1285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:16:25.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.355879+0000 mgr.gibba032.whopeo (mgr.24625) 1286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:16:25.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.357220+0000 mgr.gibba032.whopeo (mgr.24625) 1287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:16:25.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.357260+0000 mgr.gibba032.whopeo (mgr.24625) 1288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:16:25.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.359078+0000 mgr.gibba032.whopeo (mgr.24625) 1289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:16:25.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.359429+0000 mgr.gibba032.whopeo (mgr.24625) 1290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:16:25.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.360944+0000 mgr.gibba032.whopeo (mgr.24625) 1291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:16:25.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.360991+0000 mgr.gibba032.whopeo (mgr.24625) 1292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:16:25.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cephadm 2021-03-10T14:16:23.361116+0000 mgr.gibba032.whopeo (mgr.24625) 1293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:25.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:24 gibba032 bash[24004]: cluster 2021-03-10T14:16:23.445007+0000 mgr.gibba032.whopeo (mgr.24625) 1294 : cluster [DBG] pgmap v718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.088917+0000 mgr.gibba032.whopeo (mgr.24625) 1279 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:16:25.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.349219+0000 mgr.gibba032.whopeo (mgr.24625) 1280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:16:25.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.351555+0000 mgr.gibba032.whopeo (mgr.24625) 1281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:16:25.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.352998+0000 mgr.gibba032.whopeo (mgr.24625) 1282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:16:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.353037+0000 mgr.gibba032.whopeo (mgr.24625) 1283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:16:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.354515+0000 mgr.gibba032.whopeo (mgr.24625) 1284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:16:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.354555+0000 mgr.gibba032.whopeo (mgr.24625) 1285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:16:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.355879+0000 mgr.gibba032.whopeo (mgr.24625) 1286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:16:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.357220+0000 mgr.gibba032.whopeo (mgr.24625) 1287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:16:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.357260+0000 mgr.gibba032.whopeo (mgr.24625) 1288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:16:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.359078+0000 mgr.gibba032.whopeo (mgr.24625) 1289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:16:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.359429+0000 mgr.gibba032.whopeo (mgr.24625) 1290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:16:25.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.360944+0000 mgr.gibba032.whopeo (mgr.24625) 1291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:16:25.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.360991+0000 mgr.gibba032.whopeo (mgr.24625) 1292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:16:25.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cephadm 2021-03-10T14:16:23.361116+0000 mgr.gibba032.whopeo (mgr.24625) 1293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:25.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[30791]: cluster 2021-03-10T14:16:23.445007+0000 mgr.gibba032.whopeo (mgr.24625) 1294 : cluster [DBG] pgmap v718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:25.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.088917+0000 mgr.gibba032.whopeo (mgr.24625) 1279 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:16:25.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.349219+0000 mgr.gibba032.whopeo (mgr.24625) 1280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:16:25.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.351555+0000 mgr.gibba032.whopeo (mgr.24625) 1281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:16:25.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.352998+0000 mgr.gibba032.whopeo (mgr.24625) 1282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:16:25.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.353037+0000 mgr.gibba032.whopeo (mgr.24625) 1283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:16:25.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.354515+0000 mgr.gibba032.whopeo (mgr.24625) 1284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:16:25.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.354555+0000 mgr.gibba032.whopeo (mgr.24625) 1285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:16:25.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.355879+0000 mgr.gibba032.whopeo (mgr.24625) 1286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:16:25.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.357220+0000 mgr.gibba032.whopeo (mgr.24625) 1287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:16:25.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.357260+0000 mgr.gibba032.whopeo (mgr.24625) 1288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:16:25.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.359078+0000 mgr.gibba032.whopeo (mgr.24625) 1289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:16:25.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.359429+0000 mgr.gibba032.whopeo (mgr.24625) 1290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:16:25.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.360944+0000 mgr.gibba032.whopeo (mgr.24625) 1291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:16:25.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.360991+0000 mgr.gibba032.whopeo (mgr.24625) 1292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:16:25.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cephadm 2021-03-10T14:16:23.361116+0000 mgr.gibba032.whopeo (mgr.24625) 1293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:25.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:24 gibba011 bash[31774]: cluster 2021-03-10T14:16:23.445007+0000 mgr.gibba032.whopeo (mgr.24625) 1294 : cluster [DBG] pgmap v718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:26.121 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:16:26.348 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:16:26.348 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (23m) 9m ago 28m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:16:26.348 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (28m) 8m ago 28m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:16:26.349 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (23m) 8m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:16:26.349 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (23m) 8m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:16:26.349 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (22m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:16:26.349 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (22m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:16:26.349 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (22m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:16:26.350 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (22m) 9m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:16:26.350 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (28m) 9m ago 28m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:16:26.350 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (28m) 8m ago 28m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:16:26.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (21m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:16:26.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (20m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:16:26.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (20m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:16:26.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (20m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:16:26.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (19m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:16:26.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (19m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:16:26.352 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (18m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:16:26.352 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (18m) 8m ago 28m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:16:26.352 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (28m) 8m ago 28m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:16:26.352 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 26m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:16:26.598 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:16:26.598 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:16:26.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:16:26.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:26.599 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:16:26.599 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:16:26.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:26.599 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:16:26.600 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:16:26.600 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:26.600 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:16:26.600 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:16:26.600 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:16:26.601 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:16:26.601 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:16:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:26 gibba032 bash[24004]: cluster 2021-03-10T14:16:25.446674+0000 mgr.gibba032.whopeo (mgr.24625) 1295 : cluster [DBG] pgmap v719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:26 gibba032 bash[24004]: audit 2021-03-10T14:16:26.598032+0000 mon.a (mon.0) 936 : audit [DBG] from='client.? 172.21.2.111:0/1625833751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:26 gibba011 bash[30791]: cluster 2021-03-10T14:16:25.446674+0000 mgr.gibba032.whopeo (mgr.24625) 1295 : cluster [DBG] pgmap v719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:26 gibba011 bash[30791]: audit 2021-03-10T14:16:26.598032+0000 mon.a (mon.0) 936 : audit [DBG] from='client.? 172.21.2.111:0/1625833751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:26 gibba011 bash[31774]: cluster 2021-03-10T14:16:25.446674+0000 mgr.gibba032.whopeo (mgr.24625) 1295 : cluster [DBG] pgmap v719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:26 gibba011 bash[31774]: audit 2021-03-10T14:16:26.598032+0000 mon.a (mon.0) 936 : audit [DBG] from='client.? 172.21.2.111:0/1625833751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:28.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:27 gibba032 bash[24004]: audit 2021-03-10T14:16:26.108902+0000 mgr.gibba032.whopeo (mgr.24625) 1296 : audit [DBG] from='client.44919 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:27 gibba032 bash[24004]: audit 2021-03-10T14:16:26.333336+0000 mgr.gibba032.whopeo (mgr.24625) 1297 : audit [DBG] from='client.44925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:28.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:27 gibba011 bash[30791]: audit 2021-03-10T14:16:26.108902+0000 mgr.gibba032.whopeo (mgr.24625) 1296 : audit [DBG] from='client.44919 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:28.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:27 gibba011 bash[30791]: audit 2021-03-10T14:16:26.333336+0000 mgr.gibba032.whopeo (mgr.24625) 1297 : audit [DBG] from='client.44925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:28.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:27 gibba011 bash[31774]: audit 2021-03-10T14:16:26.108902+0000 mgr.gibba032.whopeo (mgr.24625) 1296 : audit [DBG] from='client.44919 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:28.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:27 gibba011 bash[31774]: audit 2021-03-10T14:16:26.333336+0000 mgr.gibba032.whopeo (mgr.24625) 1297 : audit [DBG] from='client.44925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:28 gibba032 bash[24004]: cluster 2021-03-10T14:16:27.448403+0000 mgr.gibba032.whopeo (mgr.24625) 1298 : cluster [DBG] pgmap v720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:28 gibba011 bash[30791]: cluster 2021-03-10T14:16:27.448403+0000 mgr.gibba032.whopeo (mgr.24625) 1298 : cluster [DBG] pgmap v720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:28 gibba011 bash[31774]: cluster 2021-03-10T14:16:27.448403+0000 mgr.gibba032.whopeo (mgr.24625) 1298 : cluster [DBG] pgmap v720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:30 gibba032 bash[24004]: cluster 2021-03-10T14:16:29.449910+0000 mgr.gibba032.whopeo (mgr.24625) 1299 : cluster [DBG] pgmap v721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:30 gibba011 bash[30791]: cluster 2021-03-10T14:16:29.449910+0000 mgr.gibba032.whopeo (mgr.24625) 1299 : cluster [DBG] pgmap v721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:30 gibba011 bash[31774]: cluster 2021-03-10T14:16:29.449910+0000 mgr.gibba032.whopeo (mgr.24625) 1299 : cluster [DBG] pgmap v721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:32 gibba032 bash[24004]: cluster 2021-03-10T14:16:31.451653+0000 mgr.gibba032.whopeo (mgr.24625) 1300 : cluster [DBG] pgmap v722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:32 gibba011 bash[30791]: cluster 2021-03-10T14:16:31.451653+0000 mgr.gibba032.whopeo (mgr.24625) 1300 : cluster [DBG] pgmap v722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:32 gibba011 bash[31774]: cluster 2021-03-10T14:16:31.451653+0000 mgr.gibba032.whopeo (mgr.24625) 1300 : cluster [DBG] pgmap v722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:34 gibba011 bash[30791]: cluster 2021-03-10T14:16:33.453214+0000 mgr.gibba032.whopeo (mgr.24625) 1301 : cluster [DBG] pgmap v723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:34 gibba011 bash[31774]: cluster 2021-03-10T14:16:33.453214+0000 mgr.gibba032.whopeo (mgr.24625) 1301 : cluster [DBG] pgmap v723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:34 gibba032 bash[24004]: cluster 2021-03-10T14:16:33.453214+0000 mgr.gibba032.whopeo (mgr.24625) 1301 : cluster [DBG] pgmap v723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:37.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:36 gibba011 bash[30791]: cluster 2021-03-10T14:16:35.454816+0000 mgr.gibba032.whopeo (mgr.24625) 1302 : cluster [DBG] pgmap v724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:36 gibba011 bash[31774]: cluster 2021-03-10T14:16:35.454816+0000 mgr.gibba032.whopeo (mgr.24625) 1302 : cluster [DBG] pgmap v724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:37.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:36 gibba032 bash[24004]: cluster 2021-03-10T14:16:35.454816+0000 mgr.gibba032.whopeo (mgr.24625) 1302 : cluster [DBG] pgmap v724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:38 gibba011 bash[30791]: cluster 2021-03-10T14:16:37.456628+0000 mgr.gibba032.whopeo (mgr.24625) 1303 : cluster [DBG] pgmap v725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:38 gibba011 bash[31774]: cluster 2021-03-10T14:16:37.456628+0000 mgr.gibba032.whopeo (mgr.24625) 1303 : cluster [DBG] pgmap v725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:39.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:38 gibba032 bash[24004]: cluster 2021-03-10T14:16:37.456628+0000 mgr.gibba032.whopeo (mgr.24625) 1303 : cluster [DBG] pgmap v725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:39 gibba011 bash[30791]: cephadm 2021-03-10T14:16:38.377122+0000 mgr.gibba032.whopeo (mgr.24625) 1304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:40.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:39 gibba011 bash[31774]: cephadm 2021-03-10T14:16:38.377122+0000 mgr.gibba032.whopeo (mgr.24625) 1304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:40.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:39 gibba032 bash[24004]: cephadm 2021-03-10T14:16:38.377122+0000 mgr.gibba032.whopeo (mgr.24625) 1304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:40 gibba011 bash[30791]: cluster 2021-03-10T14:16:39.457982+0000 mgr.gibba032.whopeo (mgr.24625) 1305 : cluster [DBG] pgmap v726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:40 gibba011 bash[31774]: cluster 2021-03-10T14:16:39.457982+0000 mgr.gibba032.whopeo (mgr.24625) 1305 : cluster [DBG] pgmap v726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:40 gibba032 bash[24004]: cluster 2021-03-10T14:16:39.457982+0000 mgr.gibba032.whopeo (mgr.24625) 1305 : cluster [DBG] pgmap v726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:42 gibba011 bash[30791]: cluster 2021-03-10T14:16:41.459623+0000 mgr.gibba032.whopeo (mgr.24625) 1306 : cluster [DBG] pgmap v727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:42 gibba011 bash[31774]: cluster 2021-03-10T14:16:41.459623+0000 mgr.gibba032.whopeo (mgr.24625) 1306 : cluster [DBG] pgmap v727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:42 gibba032 bash[24004]: cluster 2021-03-10T14:16:41.459623+0000 mgr.gibba032.whopeo (mgr.24625) 1306 : cluster [DBG] pgmap v727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:44 gibba011 bash[30791]: cluster 2021-03-10T14:16:43.461137+0000 mgr.gibba032.whopeo (mgr.24625) 1307 : cluster [DBG] pgmap v728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:44 gibba011 bash[31774]: cluster 2021-03-10T14:16:43.461137+0000 mgr.gibba032.whopeo (mgr.24625) 1307 : cluster [DBG] pgmap v728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:44 gibba032 bash[24004]: cluster 2021-03-10T14:16:43.461137+0000 mgr.gibba032.whopeo (mgr.24625) 1307 : cluster [DBG] pgmap v728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:46 gibba011 bash[30791]: cluster 2021-03-10T14:16:45.462700+0000 mgr.gibba032.whopeo (mgr.24625) 1308 : cluster [DBG] pgmap v729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:46 gibba011 bash[31774]: cluster 2021-03-10T14:16:45.462700+0000 mgr.gibba032.whopeo (mgr.24625) 1308 : cluster [DBG] pgmap v729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:46 gibba032 bash[24004]: cluster 2021-03-10T14:16:45.462700+0000 mgr.gibba032.whopeo (mgr.24625) 1308 : cluster [DBG] pgmap v729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:47 gibba032 bash[24004]: cluster 2021-03-10T14:16:47.464333+0000 mgr.gibba032.whopeo (mgr.24625) 1309 : cluster [DBG] pgmap v730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:47 gibba011 bash[30791]: cluster 2021-03-10T14:16:47.464333+0000 mgr.gibba032.whopeo (mgr.24625) 1309 : cluster [DBG] pgmap v730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:47 gibba011 bash[31774]: cluster 2021-03-10T14:16:47.464333+0000 mgr.gibba032.whopeo (mgr.24625) 1309 : cluster [DBG] pgmap v730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:50 gibba011 bash[30791]: cluster 2021-03-10T14:16:49.465838+0000 mgr.gibba032.whopeo (mgr.24625) 1310 : cluster [DBG] pgmap v731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:50 gibba011 bash[31774]: cluster 2021-03-10T14:16:49.465838+0000 mgr.gibba032.whopeo (mgr.24625) 1310 : cluster [DBG] pgmap v731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:50 gibba032 bash[24004]: cluster 2021-03-10T14:16:49.465838+0000 mgr.gibba032.whopeo (mgr.24625) 1310 : cluster [DBG] pgmap v731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:52 gibba011 bash[30791]: cluster 2021-03-10T14:16:51.467351+0000 mgr.gibba032.whopeo (mgr.24625) 1311 : cluster [DBG] pgmap v732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:52 gibba011 bash[31774]: cluster 2021-03-10T14:16:51.467351+0000 mgr.gibba032.whopeo (mgr.24625) 1311 : cluster [DBG] pgmap v732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:52 gibba032 bash[24004]: cluster 2021-03-10T14:16:51.467351+0000 mgr.gibba032.whopeo (mgr.24625) 1311 : cluster [DBG] pgmap v732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:54 gibba011 bash[30791]: cephadm 2021-03-10T14:16:53.393108+0000 mgr.gibba032.whopeo (mgr.24625) 1312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:54 gibba011 bash[30791]: cluster 2021-03-10T14:16:53.469011+0000 mgr.gibba032.whopeo (mgr.24625) 1313 : cluster [DBG] pgmap v733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:54 gibba011 bash[31774]: cephadm 2021-03-10T14:16:53.393108+0000 mgr.gibba032.whopeo (mgr.24625) 1312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:54 gibba011 bash[31774]: cluster 2021-03-10T14:16:53.469011+0000 mgr.gibba032.whopeo (mgr.24625) 1313 : cluster [DBG] pgmap v733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:54 gibba032 bash[24004]: cephadm 2021-03-10T14:16:53.393108+0000 mgr.gibba032.whopeo (mgr.24625) 1312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:16:55.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:54 gibba032 bash[24004]: cluster 2021-03-10T14:16:53.469011+0000 mgr.gibba032.whopeo (mgr.24625) 1313 : cluster [DBG] pgmap v733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:56.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: cluster 2021-03-10T14:16:55.470648+0000 mgr.gibba032.whopeo (mgr.24625) 1314 : cluster [DBG] pgmap v734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.082723+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:16:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.083540+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:16:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.084236+0000 mon.a (mon.0) 939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:16:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.084908+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:16:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.151477+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:16:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[30791]: audit 2021-03-10T14:16:56.152127+0000 mon.a (mon.0) 942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:16:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: cluster 2021-03-10T14:16:55.470648+0000 mgr.gibba032.whopeo (mgr.24625) 1314 : cluster [DBG] pgmap v734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.082723+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:16:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.083540+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:16:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.084236+0000 mon.a (mon.0) 939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:16:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.084908+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:16:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.151477+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:16:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:56 gibba011 bash[31774]: audit 2021-03-10T14:16:56.152127+0000 mon.a (mon.0) 942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:16:56.838 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:16:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: cluster 2021-03-10T14:16:55.470648+0000 mgr.gibba032.whopeo (mgr.24625) 1314 : cluster [DBG] pgmap v734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.082723+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:16:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.083540+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:16:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.084236+0000 mon.a (mon.0) 939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:16:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.084908+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:16:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.151477+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:16:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:56 gibba032 bash[24004]: audit 2021-03-10T14:16:56.152127+0000 mon.a (mon.0) 942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:16:57.069 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:16:57.069 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (23m) 9m ago 28m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:16:57.070 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (28m) 8m ago 28m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:16:57.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (23m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:16:57.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (24m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:16:57.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (23m) 9m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:16:57.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (23m) 9m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:16:57.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (22m) 8m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:16:57.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (22m) 9m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:16:57.071 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (29m) 9m ago 29m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:16:57.071 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (29m) 8m ago 29m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:16:57.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (21m) 9m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:16:57.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (21m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:16:57.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (21m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:16:57.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (20m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:16:57.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (20m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:16:57.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (19m) 8m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:16:57.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (19m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:16:57.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (18m) 8m ago 29m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:16:57.073 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (28m) 8m ago 29m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:16:57.073 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 27m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:16:57.318 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:16:57.319 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:16:57.319 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:16:57.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:57.319 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:16:57.319 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:16:57.320 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:57.320 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:16:57.320 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:16:57.320 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:16:57.321 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:16:57.321 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:16:57.321 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:16:57.321 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:16:57.321 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:16:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:57 gibba011 bash[30791]: audit 2021-03-10T14:16:57.318570+0000 mon.a (mon.0) 943 : audit [DBG] from='client.? 172.21.2.111:0/3797916145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:57 gibba011 bash[31774]: audit 2021-03-10T14:16:57.318570+0000 mon.a (mon.0) 943 : audit [DBG] from='client.? 172.21.2.111:0/3797916145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:57 gibba032 bash[24004]: audit 2021-03-10T14:16:57.318570+0000 mon.a (mon.0) 943 : audit [DBG] from='client.? 172.21.2.111:0/3797916145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:16:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[30791]: audit 2021-03-10T14:16:56.826664+0000 mgr.gibba032.whopeo (mgr.24625) 1315 : audit [DBG] from='client.54922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[30791]: audit 2021-03-10T14:16:57.053793+0000 mgr.gibba032.whopeo (mgr.24625) 1316 : audit [DBG] from='client.54928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[30791]: cluster 2021-03-10T14:16:57.472458+0000 mgr.gibba032.whopeo (mgr.24625) 1317 : cluster [DBG] pgmap v735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[31774]: audit 2021-03-10T14:16:56.826664+0000 mgr.gibba032.whopeo (mgr.24625) 1315 : audit [DBG] from='client.54922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[31774]: audit 2021-03-10T14:16:57.053793+0000 mgr.gibba032.whopeo (mgr.24625) 1316 : audit [DBG] from='client.54928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:16:58 gibba011 bash[31774]: cluster 2021-03-10T14:16:57.472458+0000 mgr.gibba032.whopeo (mgr.24625) 1317 : cluster [DBG] pgmap v735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:16:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:58 gibba032 bash[24004]: audit 2021-03-10T14:16:56.826664+0000 mgr.gibba032.whopeo (mgr.24625) 1315 : audit [DBG] from='client.54922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:58 gibba032 bash[24004]: audit 2021-03-10T14:16:57.053793+0000 mgr.gibba032.whopeo (mgr.24625) 1316 : audit [DBG] from='client.54928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:16:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:16:58 gibba032 bash[24004]: cluster 2021-03-10T14:16:57.472458+0000 mgr.gibba032.whopeo (mgr.24625) 1317 : cluster [DBG] pgmap v735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:00 gibba032 bash[24004]: cluster 2021-03-10T14:16:59.473966+0000 mgr.gibba032.whopeo (mgr.24625) 1318 : cluster [DBG] pgmap v736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:01.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:00 gibba011 bash[30791]: cluster 2021-03-10T14:16:59.473966+0000 mgr.gibba032.whopeo (mgr.24625) 1318 : cluster [DBG] pgmap v736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:00 gibba011 bash[31774]: cluster 2021-03-10T14:16:59.473966+0000 mgr.gibba032.whopeo (mgr.24625) 1318 : cluster [DBG] pgmap v736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:02 gibba032 bash[24004]: cluster 2021-03-10T14:17:01.475607+0000 mgr.gibba032.whopeo (mgr.24625) 1319 : cluster [DBG] pgmap v737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:02 gibba011 bash[30791]: cluster 2021-03-10T14:17:01.475607+0000 mgr.gibba032.whopeo (mgr.24625) 1319 : cluster [DBG] pgmap v737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:02 gibba011 bash[31774]: cluster 2021-03-10T14:17:01.475607+0000 mgr.gibba032.whopeo (mgr.24625) 1319 : cluster [DBG] pgmap v737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:04 gibba032 bash[24004]: cluster 2021-03-10T14:17:03.477179+0000 mgr.gibba032.whopeo (mgr.24625) 1320 : cluster [DBG] pgmap v738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:04 gibba011 bash[30791]: cluster 2021-03-10T14:17:03.477179+0000 mgr.gibba032.whopeo (mgr.24625) 1320 : cluster [DBG] pgmap v738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:04 gibba011 bash[31774]: cluster 2021-03-10T14:17:03.477179+0000 mgr.gibba032.whopeo (mgr.24625) 1320 : cluster [DBG] pgmap v738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:06 gibba032 bash[24004]: cluster 2021-03-10T14:17:05.478795+0000 mgr.gibba032.whopeo (mgr.24625) 1321 : cluster [DBG] pgmap v739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:06 gibba011 bash[30791]: cluster 2021-03-10T14:17:05.478795+0000 mgr.gibba032.whopeo (mgr.24625) 1321 : cluster [DBG] pgmap v739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:06 gibba011 bash[31774]: cluster 2021-03-10T14:17:05.478795+0000 mgr.gibba032.whopeo (mgr.24625) 1321 : cluster [DBG] pgmap v739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:08 gibba032 bash[24004]: cluster 2021-03-10T14:17:07.480379+0000 mgr.gibba032.whopeo (mgr.24625) 1322 : cluster [DBG] pgmap v740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:08 gibba011 bash[30791]: cluster 2021-03-10T14:17:07.480379+0000 mgr.gibba032.whopeo (mgr.24625) 1322 : cluster [DBG] pgmap v740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:08 gibba011 bash[31774]: cluster 2021-03-10T14:17:07.480379+0000 mgr.gibba032.whopeo (mgr.24625) 1322 : cluster [DBG] pgmap v740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:09 gibba032 bash[24004]: cephadm 2021-03-10T14:17:08.409043+0000 mgr.gibba032.whopeo (mgr.24625) 1323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:10.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:09 gibba011 bash[30791]: cephadm 2021-03-10T14:17:08.409043+0000 mgr.gibba032.whopeo (mgr.24625) 1323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:10.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:09 gibba011 bash[31774]: cephadm 2021-03-10T14:17:08.409043+0000 mgr.gibba032.whopeo (mgr.24625) 1323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:10 gibba032 bash[24004]: cluster 2021-03-10T14:17:09.481887+0000 mgr.gibba032.whopeo (mgr.24625) 1324 : cluster [DBG] pgmap v741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:10 gibba011 bash[30791]: cluster 2021-03-10T14:17:09.481887+0000 mgr.gibba032.whopeo (mgr.24625) 1324 : cluster [DBG] pgmap v741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:10 gibba011 bash[31774]: cluster 2021-03-10T14:17:09.481887+0000 mgr.gibba032.whopeo (mgr.24625) 1324 : cluster [DBG] pgmap v741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:12 gibba032 bash[24004]: cluster 2021-03-10T14:17:11.483612+0000 mgr.gibba032.whopeo (mgr.24625) 1325 : cluster [DBG] pgmap v742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:12 gibba011 bash[30791]: cluster 2021-03-10T14:17:11.483612+0000 mgr.gibba032.whopeo (mgr.24625) 1325 : cluster [DBG] pgmap v742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:12 gibba011 bash[31774]: cluster 2021-03-10T14:17:11.483612+0000 mgr.gibba032.whopeo (mgr.24625) 1325 : cluster [DBG] pgmap v742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:14 gibba032 bash[24004]: cluster 2021-03-10T14:17:13.484135+0000 mgr.gibba032.whopeo (mgr.24625) 1326 : cluster [DBG] pgmap v743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:14 gibba011 bash[30791]: cluster 2021-03-10T14:17:13.484135+0000 mgr.gibba032.whopeo (mgr.24625) 1326 : cluster [DBG] pgmap v743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:14 gibba011 bash[31774]: cluster 2021-03-10T14:17:13.484135+0000 mgr.gibba032.whopeo (mgr.24625) 1326 : cluster [DBG] pgmap v743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:16 gibba032 bash[24004]: cluster 2021-03-10T14:17:15.485343+0000 mgr.gibba032.whopeo (mgr.24625) 1327 : cluster [DBG] pgmap v744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:16 gibba011 bash[30791]: cluster 2021-03-10T14:17:15.485343+0000 mgr.gibba032.whopeo (mgr.24625) 1327 : cluster [DBG] pgmap v744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:16 gibba011 bash[31774]: cluster 2021-03-10T14:17:15.485343+0000 mgr.gibba032.whopeo (mgr.24625) 1327 : cluster [DBG] pgmap v744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:18 gibba032 bash[24004]: cluster 2021-03-10T14:17:17.487061+0000 mgr.gibba032.whopeo (mgr.24625) 1328 : cluster [DBG] pgmap v745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:18 gibba011 bash[30791]: cluster 2021-03-10T14:17:17.487061+0000 mgr.gibba032.whopeo (mgr.24625) 1328 : cluster [DBG] pgmap v745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:18 gibba011 bash[31774]: cluster 2021-03-10T14:17:17.487061+0000 mgr.gibba032.whopeo (mgr.24625) 1328 : cluster [DBG] pgmap v745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:20 gibba032 bash[24004]: cluster 2021-03-10T14:17:19.487806+0000 mgr.gibba032.whopeo (mgr.24625) 1329 : cluster [DBG] pgmap v746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:20 gibba011 bash[30791]: cluster 2021-03-10T14:17:19.487806+0000 mgr.gibba032.whopeo (mgr.24625) 1329 : cluster [DBG] pgmap v746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:20 gibba011 bash[31774]: cluster 2021-03-10T14:17:19.487806+0000 mgr.gibba032.whopeo (mgr.24625) 1329 : cluster [DBG] pgmap v746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:22 gibba032 bash[24004]: cluster 2021-03-10T14:17:21.489240+0000 mgr.gibba032.whopeo (mgr.24625) 1330 : cluster [DBG] pgmap v747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:23.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:22 gibba011 bash[30791]: cluster 2021-03-10T14:17:21.489240+0000 mgr.gibba032.whopeo (mgr.24625) 1330 : cluster [DBG] pgmap v747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:22 gibba011 bash[31774]: cluster 2021-03-10T14:17:21.489240+0000 mgr.gibba032.whopeo (mgr.24625) 1330 : cluster [DBG] pgmap v747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:24.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:23 gibba032 bash[24004]: audit 2021-03-10T14:17:23.429969+0000 mon.a (mon.0) 944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:17:24.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:23 gibba011 bash[30791]: audit 2021-03-10T14:17:23.429969+0000 mon.a (mon.0) 944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:17:24.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:23 gibba011 bash[31774]: audit 2021-03-10T14:17:23.429969+0000 mon.a (mon.0) 944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:17:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:24 gibba032 bash[24004]: cephadm 2021-03-10T14:17:23.426670+0000 mgr.gibba032.whopeo (mgr.24625) 1331 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:17:25.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:24 gibba032 bash[24004]: cluster 2021-03-10T14:17:23.489743+0000 mgr.gibba032.whopeo (mgr.24625) 1332 : cluster [DBG] pgmap v748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:24 gibba011 bash[30791]: cephadm 2021-03-10T14:17:23.426670+0000 mgr.gibba032.whopeo (mgr.24625) 1331 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:17:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:24 gibba011 bash[30791]: cluster 2021-03-10T14:17:23.489743+0000 mgr.gibba032.whopeo (mgr.24625) 1332 : cluster [DBG] pgmap v748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:24 gibba011 bash[31774]: cephadm 2021-03-10T14:17:23.426670+0000 mgr.gibba032.whopeo (mgr.24625) 1331 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:17:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:24 gibba011 bash[31774]: cluster 2021-03-10T14:17:23.489743+0000 mgr.gibba032.whopeo (mgr.24625) 1332 : cluster [DBG] pgmap v748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:26 gibba032 bash[24004]: cluster 2021-03-10T14:17:25.490363+0000 mgr.gibba032.whopeo (mgr.24625) 1333 : cluster [DBG] pgmap v749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:27.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:26 gibba011 bash[30791]: cluster 2021-03-10T14:17:25.490363+0000 mgr.gibba032.whopeo (mgr.24625) 1333 : cluster [DBG] pgmap v749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:26 gibba011 bash[31774]: cluster 2021-03-10T14:17:25.490363+0000 mgr.gibba032.whopeo (mgr.24625) 1333 : cluster [DBG] pgmap v749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:27.564 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:17:27.791 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:17:27.791 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (24m) 0s ago 29m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:17:27.792 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (29m) 9m ago 29m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:17:27.792 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (24m) 9m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:17:27.792 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (24m) 9m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:17:27.792 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (23m) 0s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:17:27.792 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (23m) 0s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:17:27.793 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (23m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:17:27.793 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (23m) 0s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:17:27.793 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (29m) 0s ago 29m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:17:27.793 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (29m) 9m ago 29m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:17:27.793 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (22m) 0s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:17:27.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (21m) 0s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:17:27.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (21m) 0s ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:17:27.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (21m) 0s ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:17:27.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (20m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:17:27.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (20m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:17:27.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (19m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:17:27.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (19m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:17:27.795 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (29m) 9m ago 29m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:17:27.795 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 0s ago 27m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:17:28.049 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:17:28.050 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:17:28.050 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:17:28.050 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:28.050 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:17:28.051 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:17:28.051 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:28.051 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:17:28.051 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:17:28.052 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:28.052 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:17:28.052 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:17:28.052 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:17:28.053 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:17:28.053 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:17:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:26.937278+0000 mgr.gibba032.whopeo (mgr.24625) 1334 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:17:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:26.939515+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.159052+0000 mon.a (mon.0) 946 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.163841+0000 mgr.gibba032.whopeo (mgr.24625) 1335 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:17:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.166343+0000 mgr.gibba032.whopeo (mgr.24625) 1336 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:17:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.167517+0000 mon.a (mon.0) 947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:17:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.168108+0000 mgr.gibba032.whopeo (mgr.24625) 1337 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:17:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.168147+0000 mgr.gibba032.whopeo (mgr.24625) 1338 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:17:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.169375+0000 mgr.gibba032.whopeo (mgr.24625) 1339 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:17:28.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.169444+0000 mgr.gibba032.whopeo (mgr.24625) 1340 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:17:28.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.169767+0000 mon.a (mon.0) 948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.170392+0000 mgr.gibba032.whopeo (mgr.24625) 1341 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:17:28.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.171292+0000 mon.a (mon.0) 949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.171804+0000 mgr.gibba032.whopeo (mgr.24625) 1342 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:17:28.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.171845+0000 mgr.gibba032.whopeo (mgr.24625) 1343 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:17:28.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.172592+0000 mon.a (mon.0) 950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.173139+0000 mgr.gibba032.whopeo (mgr.24625) 1344 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:17:28.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.173562+0000 mgr.gibba032.whopeo (mgr.24625) 1345 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:17:28.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.174762+0000 mgr.gibba032.whopeo (mgr.24625) 1346 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:17:28.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.174810+0000 mgr.gibba032.whopeo (mgr.24625) 1347 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:17:28.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cephadm 2021-03-10T14:17:27.174939+0000 mgr.gibba032.whopeo (mgr.24625) 1348 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:28.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.175180+0000 mon.a (mon.0) 951 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.176712+0000 mon.a (mon.0) 952 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: cluster 2021-03-10T14:17:27.491987+0000 mgr.gibba032.whopeo (mgr.24625) 1349 : cluster [DBG] pgmap v750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:28.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.551686+0000 mgr.gibba032.whopeo (mgr.24625) 1350 : audit [DBG] from='client.54940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:28.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:27 gibba032 bash[24004]: audit 2021-03-10T14:17:27.776574+0000 mgr.gibba032.whopeo (mgr.24625) 1351 : audit [DBG] from='client.54946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:26.937278+0000 mgr.gibba032.whopeo (mgr.24625) 1334 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:17:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:26.939515+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.159052+0000 mon.a (mon.0) 946 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.163841+0000 mgr.gibba032.whopeo (mgr.24625) 1335 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:17:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.166343+0000 mgr.gibba032.whopeo (mgr.24625) 1336 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:17:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.167517+0000 mon.a (mon.0) 947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:17:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.168108+0000 mgr.gibba032.whopeo (mgr.24625) 1337 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:17:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.168147+0000 mgr.gibba032.whopeo (mgr.24625) 1338 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:17:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.169375+0000 mgr.gibba032.whopeo (mgr.24625) 1339 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:17:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.169444+0000 mgr.gibba032.whopeo (mgr.24625) 1340 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:17:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.169767+0000 mon.a (mon.0) 948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.170392+0000 mgr.gibba032.whopeo (mgr.24625) 1341 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:17:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.171292+0000 mon.a (mon.0) 949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.171804+0000 mgr.gibba032.whopeo (mgr.24625) 1342 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:17:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.171845+0000 mgr.gibba032.whopeo (mgr.24625) 1343 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:17:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.172592+0000 mon.a (mon.0) 950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.173139+0000 mgr.gibba032.whopeo (mgr.24625) 1344 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:17:28.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.173562+0000 mgr.gibba032.whopeo (mgr.24625) 1345 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:17:28.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.174762+0000 mgr.gibba032.whopeo (mgr.24625) 1346 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:17:28.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.174810+0000 mgr.gibba032.whopeo (mgr.24625) 1347 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:17:28.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:26.937278+0000 mgr.gibba032.whopeo (mgr.24625) 1334 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:17:28.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:26.939515+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.159052+0000 mon.a (mon.0) 946 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:17:28.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.163841+0000 mgr.gibba032.whopeo (mgr.24625) 1335 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:17:28.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.166343+0000 mgr.gibba032.whopeo (mgr.24625) 1336 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:17:28.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.167517+0000 mon.a (mon.0) 947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:17:28.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.168108+0000 mgr.gibba032.whopeo (mgr.24625) 1337 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:17:28.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.168147+0000 mgr.gibba032.whopeo (mgr.24625) 1338 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:17:28.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.169375+0000 mgr.gibba032.whopeo (mgr.24625) 1339 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:17:28.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.169444+0000 mgr.gibba032.whopeo (mgr.24625) 1340 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:17:28.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.169767+0000 mon.a (mon.0) 948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.170392+0000 mgr.gibba032.whopeo (mgr.24625) 1341 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:17:28.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.171292+0000 mon.a (mon.0) 949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.171804+0000 mgr.gibba032.whopeo (mgr.24625) 1342 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:17:28.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.171845+0000 mgr.gibba032.whopeo (mgr.24625) 1343 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:17:28.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.172592+0000 mon.a (mon.0) 950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.173139+0000 mgr.gibba032.whopeo (mgr.24625) 1344 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:17:28.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.173562+0000 mgr.gibba032.whopeo (mgr.24625) 1345 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:17:28.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.174762+0000 mgr.gibba032.whopeo (mgr.24625) 1346 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:17:28.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.174810+0000 mgr.gibba032.whopeo (mgr.24625) 1347 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:17:28.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cephadm 2021-03-10T14:17:27.174939+0000 mgr.gibba032.whopeo (mgr.24625) 1348 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:28.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.175180+0000 mon.a (mon.0) 951 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.176712+0000 mon.a (mon.0) 952 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: cluster 2021-03-10T14:17:27.491987+0000 mgr.gibba032.whopeo (mgr.24625) 1349 : cluster [DBG] pgmap v750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:28.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.551686+0000 mgr.gibba032.whopeo (mgr.24625) 1350 : audit [DBG] from='client.54940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:28.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[30791]: audit 2021-03-10T14:17:27.776574+0000 mgr.gibba032.whopeo (mgr.24625) 1351 : audit [DBG] from='client.54946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:28.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cephadm 2021-03-10T14:17:27.174939+0000 mgr.gibba032.whopeo (mgr.24625) 1348 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:28.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.175180+0000 mon.a (mon.0) 951 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.176712+0000 mon.a (mon.0) 952 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:28.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: cluster 2021-03-10T14:17:27.491987+0000 mgr.gibba032.whopeo (mgr.24625) 1349 : cluster [DBG] pgmap v750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:28.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.551686+0000 mgr.gibba032.whopeo (mgr.24625) 1350 : audit [DBG] from='client.54940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:28.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:27 gibba011 bash[31774]: audit 2021-03-10T14:17:27.776574+0000 mgr.gibba032.whopeo (mgr.24625) 1351 : audit [DBG] from='client.54946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:17:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:28 gibba032 bash[24004]: audit 2021-03-10T14:17:28.049651+0000 mon.a (mon.0) 953 : audit [DBG] from='client.? 172.21.2.111:0/1077174513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:28 gibba011 bash[30791]: audit 2021-03-10T14:17:28.049651+0000 mon.a (mon.0) 953 : audit [DBG] from='client.? 172.21.2.111:0/1077174513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:28 gibba011 bash[31774]: audit 2021-03-10T14:17:28.049651+0000 mon.a (mon.0) 953 : audit [DBG] from='client.? 172.21.2.111:0/1077174513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:17:30.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:29 gibba032 bash[24004]: cluster 2021-03-10T14:17:29.493543+0000 mgr.gibba032.whopeo (mgr.24625) 1352 : cluster [DBG] pgmap v751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:29 gibba011 bash[30791]: cluster 2021-03-10T14:17:29.493543+0000 mgr.gibba032.whopeo (mgr.24625) 1352 : cluster [DBG] pgmap v751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:29 gibba011 bash[31774]: cluster 2021-03-10T14:17:29.493543+0000 mgr.gibba032.whopeo (mgr.24625) 1352 : cluster [DBG] pgmap v751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:32 gibba011 bash[30791]: cluster 2021-03-10T14:17:31.494305+0000 mgr.gibba032.whopeo (mgr.24625) 1353 : cluster [DBG] pgmap v752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:32 gibba011 bash[31774]: cluster 2021-03-10T14:17:31.494305+0000 mgr.gibba032.whopeo (mgr.24625) 1353 : cluster [DBG] pgmap v752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:32 gibba032 bash[24004]: cluster 2021-03-10T14:17:31.494305+0000 mgr.gibba032.whopeo (mgr.24625) 1353 : cluster [DBG] pgmap v752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:34 gibba011 bash[30791]: cluster 2021-03-10T14:17:33.495589+0000 mgr.gibba032.whopeo (mgr.24625) 1354 : cluster [DBG] pgmap v753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:34 gibba011 bash[31774]: cluster 2021-03-10T14:17:33.495589+0000 mgr.gibba032.whopeo (mgr.24625) 1354 : cluster [DBG] pgmap v753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:34 gibba032 bash[24004]: cluster 2021-03-10T14:17:33.495589+0000 mgr.gibba032.whopeo (mgr.24625) 1354 : cluster [DBG] pgmap v753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:36 gibba011 bash[30791]: cluster 2021-03-10T14:17:35.497094+0000 mgr.gibba032.whopeo (mgr.24625) 1355 : cluster [DBG] pgmap v754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:36 gibba011 bash[31774]: cluster 2021-03-10T14:17:35.497094+0000 mgr.gibba032.whopeo (mgr.24625) 1355 : cluster [DBG] pgmap v754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:36 gibba032 bash[24004]: cluster 2021-03-10T14:17:35.497094+0000 mgr.gibba032.whopeo (mgr.24625) 1355 : cluster [DBG] pgmap v754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:38 gibba011 bash[30791]: cluster 2021-03-10T14:17:37.497875+0000 mgr.gibba032.whopeo (mgr.24625) 1356 : cluster [DBG] pgmap v755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:38 gibba011 bash[31774]: cluster 2021-03-10T14:17:37.497875+0000 mgr.gibba032.whopeo (mgr.24625) 1356 : cluster [DBG] pgmap v755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:38 gibba032 bash[24004]: cluster 2021-03-10T14:17:37.497875+0000 mgr.gibba032.whopeo (mgr.24625) 1356 : cluster [DBG] pgmap v755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:40 gibba011 bash[30791]: cluster 2021-03-10T14:17:39.499133+0000 mgr.gibba032.whopeo (mgr.24625) 1357 : cluster [DBG] pgmap v756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:40 gibba011 bash[31774]: cluster 2021-03-10T14:17:39.499133+0000 mgr.gibba032.whopeo (mgr.24625) 1357 : cluster [DBG] pgmap v756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:40 gibba032 bash[24004]: cluster 2021-03-10T14:17:39.499133+0000 mgr.gibba032.whopeo (mgr.24625) 1357 : cluster [DBG] pgmap v756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:42 gibba011 bash[30791]: cluster 2021-03-10T14:17:41.500902+0000 mgr.gibba032.whopeo (mgr.24625) 1358 : cluster [DBG] pgmap v757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:42 gibba011 bash[31774]: cluster 2021-03-10T14:17:41.500902+0000 mgr.gibba032.whopeo (mgr.24625) 1358 : cluster [DBG] pgmap v757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:42 gibba032 bash[24004]: cluster 2021-03-10T14:17:41.500902+0000 mgr.gibba032.whopeo (mgr.24625) 1358 : cluster [DBG] pgmap v757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:44.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:43 gibba032 bash[24004]: cephadm 2021-03-10T14:17:42.190889+0000 mgr.gibba032.whopeo (mgr.24625) 1359 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:44.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:43 gibba011 bash[30791]: cephadm 2021-03-10T14:17:42.190889+0000 mgr.gibba032.whopeo (mgr.24625) 1359 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:44.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:43 gibba011 bash[31774]: cephadm 2021-03-10T14:17:42.190889+0000 mgr.gibba032.whopeo (mgr.24625) 1359 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:44 gibba032 bash[24004]: cluster 2021-03-10T14:17:43.501746+0000 mgr.gibba032.whopeo (mgr.24625) 1360 : cluster [DBG] pgmap v758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:44 gibba011 bash[30791]: cluster 2021-03-10T14:17:43.501746+0000 mgr.gibba032.whopeo (mgr.24625) 1360 : cluster [DBG] pgmap v758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:44 gibba011 bash[31774]: cluster 2021-03-10T14:17:43.501746+0000 mgr.gibba032.whopeo (mgr.24625) 1360 : cluster [DBG] pgmap v758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:46 gibba032 bash[24004]: cluster 2021-03-10T14:17:45.503169+0000 mgr.gibba032.whopeo (mgr.24625) 1361 : cluster [DBG] pgmap v759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:46 gibba011 bash[30791]: cluster 2021-03-10T14:17:45.503169+0000 mgr.gibba032.whopeo (mgr.24625) 1361 : cluster [DBG] pgmap v759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:46 gibba011 bash[31774]: cluster 2021-03-10T14:17:45.503169+0000 mgr.gibba032.whopeo (mgr.24625) 1361 : cluster [DBG] pgmap v759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:48 gibba032 bash[24004]: cluster 2021-03-10T14:17:47.504917+0000 mgr.gibba032.whopeo (mgr.24625) 1362 : cluster [DBG] pgmap v760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:48 gibba011 bash[30791]: cluster 2021-03-10T14:17:47.504917+0000 mgr.gibba032.whopeo (mgr.24625) 1362 : cluster [DBG] pgmap v760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:48 gibba011 bash[31774]: cluster 2021-03-10T14:17:47.504917+0000 mgr.gibba032.whopeo (mgr.24625) 1362 : cluster [DBG] pgmap v760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:50 gibba032 bash[24004]: cluster 2021-03-10T14:17:49.505539+0000 mgr.gibba032.whopeo (mgr.24625) 1363 : cluster [DBG] pgmap v761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:50 gibba011 bash[30791]: cluster 2021-03-10T14:17:49.505539+0000 mgr.gibba032.whopeo (mgr.24625) 1363 : cluster [DBG] pgmap v761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:50 gibba011 bash[31774]: cluster 2021-03-10T14:17:49.505539+0000 mgr.gibba032.whopeo (mgr.24625) 1363 : cluster [DBG] pgmap v761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:52 gibba032 bash[24004]: cluster 2021-03-10T14:17:51.507182+0000 mgr.gibba032.whopeo (mgr.24625) 1364 : cluster [DBG] pgmap v762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:52 gibba011 bash[30791]: cluster 2021-03-10T14:17:51.507182+0000 mgr.gibba032.whopeo (mgr.24625) 1364 : cluster [DBG] pgmap v762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:52 gibba011 bash[31774]: cluster 2021-03-10T14:17:51.507182+0000 mgr.gibba032.whopeo (mgr.24625) 1364 : cluster [DBG] pgmap v762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:54 gibba032 bash[24004]: cluster 2021-03-10T14:17:53.508837+0000 mgr.gibba032.whopeo (mgr.24625) 1365 : cluster [DBG] pgmap v763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:54 gibba011 bash[30791]: cluster 2021-03-10T14:17:53.508837+0000 mgr.gibba032.whopeo (mgr.24625) 1365 : cluster [DBG] pgmap v763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:54 gibba011 bash[31774]: cluster 2021-03-10T14:17:53.508837+0000 mgr.gibba032.whopeo (mgr.24625) 1365 : cluster [DBG] pgmap v763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: cluster 2021-03-10T14:17:55.509673+0000 mgr.gibba032.whopeo (mgr.24625) 1366 : cluster [DBG] pgmap v764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.093267+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:17:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.094056+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:17:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.094789+0000 mon.a (mon.0) 956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.095519+0000 mon.a (mon.0) 957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.159232+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:17:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:56 gibba032 bash[24004]: audit 2021-03-10T14:17:56.159982+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:17:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: cluster 2021-03-10T14:17:55.509673+0000 mgr.gibba032.whopeo (mgr.24625) 1366 : cluster [DBG] pgmap v764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.093267+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:17:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.094056+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:17:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.094789+0000 mon.a (mon.0) 956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.095519+0000 mon.a (mon.0) 957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.159232+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:17:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[30791]: audit 2021-03-10T14:17:56.159982+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:17:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: cluster 2021-03-10T14:17:55.509673+0000 mgr.gibba032.whopeo (mgr.24625) 1366 : cluster [DBG] pgmap v764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.093267+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:17:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.094056+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:17:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.094789+0000 mon.a (mon.0) 956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.095519+0000 mon.a (mon.0) 957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:17:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.159232+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:17:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:56 gibba011 bash[31774]: audit 2021-03-10T14:17:56.159982+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:17:58.293 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:17:58.516 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:17:58.516 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (24m) 31s ago 29m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:17:58.517 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (29m) 9m ago 29m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:17:58.517 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (24m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:17:58.517 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (25m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:17:58.517 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (24m) 31s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:17:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (24m) 31s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:17:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (23m) 9m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:17:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (23m) 31s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:17:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (30m) 31s ago 30m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:17:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (30m) 9m ago 30m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:17:58.519 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (22m) 31s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:17:58.519 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (22m) 31s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:17:58.519 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (22m) 31s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:17:58.519 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (21m) 31s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:17:58.519 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (21m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:17:58.520 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (20m) 9m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:17:58.520 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (20m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:17:58.520 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (19m) 9m ago 30m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:17:58.520 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (29m) 9m ago 30m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:17:58.520 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 31s ago 28m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:17:58.769 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:17:58.769 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:17:58.770 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:17:58.770 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:58.770 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:17:58.770 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:17:58.771 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:58.771 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:17:58.771 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:17:58.771 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:17:58.771 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:17:58.772 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:17:58.772 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:17:58.772 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:17:58.772 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:17:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:58 gibba032 bash[24004]: cephadm 2021-03-10T14:17:57.206933+0000 mgr.gibba032.whopeo (mgr.24625) 1367 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:58 gibba032 bash[24004]: cluster 2021-03-10T14:17:57.511187+0000 mgr.gibba032.whopeo (mgr.24625) 1368 : cluster [DBG] pgmap v765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:58 gibba011 bash[30791]: cephadm 2021-03-10T14:17:57.206933+0000 mgr.gibba032.whopeo (mgr.24625) 1367 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:58 gibba011 bash[30791]: cluster 2021-03-10T14:17:57.511187+0000 mgr.gibba032.whopeo (mgr.24625) 1368 : cluster [DBG] pgmap v765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:17:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:58 gibba011 bash[31774]: cephadm 2021-03-10T14:17:57.206933+0000 mgr.gibba032.whopeo (mgr.24625) 1367 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:17:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:58 gibba011 bash[31774]: cluster 2021-03-10T14:17:57.511187+0000 mgr.gibba032.whopeo (mgr.24625) 1368 : cluster [DBG] pgmap v765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:59 gibba032 bash[24004]: audit 2021-03-10T14:17:58.281602+0000 mgr.gibba032.whopeo (mgr.24625) 1369 : audit [DBG] from='client.54958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:59 gibba032 bash[24004]: audit 2021-03-10T14:17:58.504752+0000 mgr.gibba032.whopeo (mgr.24625) 1370 : audit [DBG] from='client.54964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:17:59 gibba032 bash[24004]: audit 2021-03-10T14:17:58.769452+0000 mon.c (mon.1) 27 : audit [DBG] from='client.? 172.21.2.111:0/60786390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:00.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[30791]: audit 2021-03-10T14:17:58.281602+0000 mgr.gibba032.whopeo (mgr.24625) 1369 : audit [DBG] from='client.54958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[30791]: audit 2021-03-10T14:17:58.504752+0000 mgr.gibba032.whopeo (mgr.24625) 1370 : audit [DBG] from='client.54964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[30791]: audit 2021-03-10T14:17:58.769452+0000 mon.c (mon.1) 27 : audit [DBG] from='client.? 172.21.2.111:0/60786390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:00.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[31774]: audit 2021-03-10T14:17:58.281602+0000 mgr.gibba032.whopeo (mgr.24625) 1369 : audit [DBG] from='client.54958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[31774]: audit 2021-03-10T14:17:58.504752+0000 mgr.gibba032.whopeo (mgr.24625) 1370 : audit [DBG] from='client.54964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:00.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:17:59 gibba011 bash[31774]: audit 2021-03-10T14:17:58.769452+0000 mon.c (mon.1) 27 : audit [DBG] from='client.? 172.21.2.111:0/60786390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:00 gibba032 bash[24004]: cluster 2021-03-10T14:17:59.512694+0000 mgr.gibba032.whopeo (mgr.24625) 1371 : cluster [DBG] pgmap v766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:00 gibba011 bash[30791]: cluster 2021-03-10T14:17:59.512694+0000 mgr.gibba032.whopeo (mgr.24625) 1371 : cluster [DBG] pgmap v766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:00 gibba011 bash[31774]: cluster 2021-03-10T14:17:59.512694+0000 mgr.gibba032.whopeo (mgr.24625) 1371 : cluster [DBG] pgmap v766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:02 gibba032 bash[24004]: cluster 2021-03-10T14:18:01.513353+0000 mgr.gibba032.whopeo (mgr.24625) 1372 : cluster [DBG] pgmap v767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:02 gibba011 bash[30791]: cluster 2021-03-10T14:18:01.513353+0000 mgr.gibba032.whopeo (mgr.24625) 1372 : cluster [DBG] pgmap v767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:02 gibba011 bash[31774]: cluster 2021-03-10T14:18:01.513353+0000 mgr.gibba032.whopeo (mgr.24625) 1372 : cluster [DBG] pgmap v767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:04 gibba032 bash[24004]: cluster 2021-03-10T14:18:03.514799+0000 mgr.gibba032.whopeo (mgr.24625) 1373 : cluster [DBG] pgmap v768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:04 gibba011 bash[30791]: cluster 2021-03-10T14:18:03.514799+0000 mgr.gibba032.whopeo (mgr.24625) 1373 : cluster [DBG] pgmap v768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:04 gibba011 bash[31774]: cluster 2021-03-10T14:18:03.514799+0000 mgr.gibba032.whopeo (mgr.24625) 1373 : cluster [DBG] pgmap v768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:06 gibba032 bash[24004]: cluster 2021-03-10T14:18:05.516424+0000 mgr.gibba032.whopeo (mgr.24625) 1374 : cluster [DBG] pgmap v769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:06 gibba011 bash[30791]: cluster 2021-03-10T14:18:05.516424+0000 mgr.gibba032.whopeo (mgr.24625) 1374 : cluster [DBG] pgmap v769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:06 gibba011 bash[31774]: cluster 2021-03-10T14:18:05.516424+0000 mgr.gibba032.whopeo (mgr.24625) 1374 : cluster [DBG] pgmap v769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:08 gibba032 bash[24004]: cluster 2021-03-10T14:18:07.517353+0000 mgr.gibba032.whopeo (mgr.24625) 1375 : cluster [DBG] pgmap v770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:08 gibba011 bash[30791]: cluster 2021-03-10T14:18:07.517353+0000 mgr.gibba032.whopeo (mgr.24625) 1375 : cluster [DBG] pgmap v770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:08 gibba011 bash[31774]: cluster 2021-03-10T14:18:07.517353+0000 mgr.gibba032.whopeo (mgr.24625) 1375 : cluster [DBG] pgmap v770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:10 gibba032 bash[24004]: cluster 2021-03-10T14:18:09.518607+0000 mgr.gibba032.whopeo (mgr.24625) 1376 : cluster [DBG] pgmap v771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:11.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:10 gibba011 bash[30791]: cluster 2021-03-10T14:18:09.518607+0000 mgr.gibba032.whopeo (mgr.24625) 1376 : cluster [DBG] pgmap v771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:10 gibba011 bash[31774]: cluster 2021-03-10T14:18:09.518607+0000 mgr.gibba032.whopeo (mgr.24625) 1376 : cluster [DBG] pgmap v771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:12 gibba032 bash[24004]: cluster 2021-03-10T14:18:11.520387+0000 mgr.gibba032.whopeo (mgr.24625) 1377 : cluster [DBG] pgmap v772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:12 gibba011 bash[30791]: cluster 2021-03-10T14:18:11.520387+0000 mgr.gibba032.whopeo (mgr.24625) 1377 : cluster [DBG] pgmap v772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:12 gibba011 bash[31774]: cluster 2021-03-10T14:18:11.520387+0000 mgr.gibba032.whopeo (mgr.24625) 1377 : cluster [DBG] pgmap v772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:14.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:13 gibba032 bash[24004]: cephadm 2021-03-10T14:18:12.220970+0000 mgr.gibba032.whopeo (mgr.24625) 1378 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:14.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:13 gibba011 bash[30791]: cephadm 2021-03-10T14:18:12.220970+0000 mgr.gibba032.whopeo (mgr.24625) 1378 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:14.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:13 gibba011 bash[31774]: cephadm 2021-03-10T14:18:12.220970+0000 mgr.gibba032.whopeo (mgr.24625) 1378 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:14 gibba032 bash[24004]: cluster 2021-03-10T14:18:13.521285+0000 mgr.gibba032.whopeo (mgr.24625) 1379 : cluster [DBG] pgmap v773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:14 gibba011 bash[30791]: cluster 2021-03-10T14:18:13.521285+0000 mgr.gibba032.whopeo (mgr.24625) 1379 : cluster [DBG] pgmap v773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:14 gibba011 bash[31774]: cluster 2021-03-10T14:18:13.521285+0000 mgr.gibba032.whopeo (mgr.24625) 1379 : cluster [DBG] pgmap v773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:16 gibba032 bash[24004]: cluster 2021-03-10T14:18:15.522695+0000 mgr.gibba032.whopeo (mgr.24625) 1380 : cluster [DBG] pgmap v774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:16 gibba011 bash[30791]: cluster 2021-03-10T14:18:15.522695+0000 mgr.gibba032.whopeo (mgr.24625) 1380 : cluster [DBG] pgmap v774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:16 gibba011 bash[31774]: cluster 2021-03-10T14:18:15.522695+0000 mgr.gibba032.whopeo (mgr.24625) 1380 : cluster [DBG] pgmap v774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:18 gibba032 bash[24004]: cluster 2021-03-10T14:18:17.524508+0000 mgr.gibba032.whopeo (mgr.24625) 1381 : cluster [DBG] pgmap v775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:18 gibba011 bash[30791]: cluster 2021-03-10T14:18:17.524508+0000 mgr.gibba032.whopeo (mgr.24625) 1381 : cluster [DBG] pgmap v775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:18 gibba011 bash[31774]: cluster 2021-03-10T14:18:17.524508+0000 mgr.gibba032.whopeo (mgr.24625) 1381 : cluster [DBG] pgmap v775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:21.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:20 gibba032 bash[24004]: cluster 2021-03-10T14:18:19.525155+0000 mgr.gibba032.whopeo (mgr.24625) 1382 : cluster [DBG] pgmap v776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:20 gibba011 bash[30791]: cluster 2021-03-10T14:18:19.525155+0000 mgr.gibba032.whopeo (mgr.24625) 1382 : cluster [DBG] pgmap v776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:20 gibba011 bash[31774]: cluster 2021-03-10T14:18:19.525155+0000 mgr.gibba032.whopeo (mgr.24625) 1382 : cluster [DBG] pgmap v776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:22 gibba011 bash[30791]: cluster 2021-03-10T14:18:21.526798+0000 mgr.gibba032.whopeo (mgr.24625) 1383 : cluster [DBG] pgmap v777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:22 gibba011 bash[31774]: cluster 2021-03-10T14:18:21.526798+0000 mgr.gibba032.whopeo (mgr.24625) 1383 : cluster [DBG] pgmap v777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:23.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:22 gibba032 bash[24004]: cluster 2021-03-10T14:18:21.526798+0000 mgr.gibba032.whopeo (mgr.24625) 1383 : cluster [DBG] pgmap v777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:24 gibba011 bash[30791]: cluster 2021-03-10T14:18:23.528207+0000 mgr.gibba032.whopeo (mgr.24625) 1384 : cluster [DBG] pgmap v778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:24 gibba011 bash[31774]: cluster 2021-03-10T14:18:23.528207+0000 mgr.gibba032.whopeo (mgr.24625) 1384 : cluster [DBG] pgmap v778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:24 gibba032 bash[24004]: cluster 2021-03-10T14:18:23.528207+0000 mgr.gibba032.whopeo (mgr.24625) 1384 : cluster [DBG] pgmap v778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:26 gibba011 bash[30791]: cluster 2021-03-10T14:18:25.528952+0000 mgr.gibba032.whopeo (mgr.24625) 1385 : cluster [DBG] pgmap v779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:26 gibba011 bash[31774]: cluster 2021-03-10T14:18:25.528952+0000 mgr.gibba032.whopeo (mgr.24625) 1385 : cluster [DBG] pgmap v779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:26 gibba032 bash[24004]: cluster 2021-03-10T14:18:25.528952+0000 mgr.gibba032.whopeo (mgr.24625) 1385 : cluster [DBG] pgmap v779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:28.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:27 gibba011 bash[30791]: audit 2021-03-10T14:18:27.242132+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:28.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:27 gibba011 bash[31774]: audit 2021-03-10T14:18:27.242132+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:27 gibba032 bash[24004]: audit 2021-03-10T14:18:27.242132+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:29.014 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:18:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:28 gibba011 bash[30791]: cephadm 2021-03-10T14:18:27.237541+0000 mgr.gibba032.whopeo (mgr.24625) 1386 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:18:29.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:28 gibba011 bash[30791]: cluster 2021-03-10T14:18:27.530589+0000 mgr.gibba032.whopeo (mgr.24625) 1387 : cluster [DBG] pgmap v780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:29.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:28 gibba011 bash[31774]: cephadm 2021-03-10T14:18:27.237541+0000 mgr.gibba032.whopeo (mgr.24625) 1386 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:18:29.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:28 gibba011 bash[31774]: cluster 2021-03-10T14:18:27.530589+0000 mgr.gibba032.whopeo (mgr.24625) 1387 : cluster [DBG] pgmap v780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:29.244 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:18:29.245 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (25m) 62s ago 30m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:18:29.246 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (30m) 10m ago 30m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:18:29.246 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (25m) 10m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:18:29.246 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (25m) 10m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:18:29.247 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (24m) 62s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:18:29.247 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (24m) 62s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:18:29.247 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (24m) 10m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:18:29.248 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (24m) 62s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:18:29.249 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (30m) 62s ago 30m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:18:29.249 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (30m) 10m ago 30m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:18:29.250 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (23m) 62s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:18:29.250 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (23m) 62s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:18:29.250 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (22m) 62s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:18:29.251 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (22m) 62s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:18:29.251 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (21m) 10m ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:18:29.252 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (21m) 10m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:18:29.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (20m) 10m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:18:29.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (20m) 10m ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:18:29.255 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (30m) 10m ago 30m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:18:29.256 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 62s ago 28m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:18:29.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:28 gibba032 bash[24004]: cephadm 2021-03-10T14:18:27.237541+0000 mgr.gibba032.whopeo (mgr.24625) 1386 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:18:29.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:28 gibba032 bash[24004]: cluster 2021-03-10T14:18:27.530589+0000 mgr.gibba032.whopeo (mgr.24625) 1387 : cluster [DBG] pgmap v780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:29.495 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:18:29.495 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:18:29.495 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:18:29.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:18:29.496 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:18:29.496 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:18:29.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:18:29.496 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:18:29.497 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:18:29.497 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:18:29.497 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:18:29.497 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:18:29.498 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:18:29.498 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:18:29.498 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:18:30.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:29 gibba011 bash[30791]: audit 2021-03-10T14:18:29.494956+0000 mon.c (mon.1) 28 : audit [DBG] from='client.? 172.21.2.111:0/1499058480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:29 gibba011 bash[31774]: audit 2021-03-10T14:18:29.494956+0000 mon.c (mon.1) 28 : audit [DBG] from='client.? 172.21.2.111:0/1499058480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:30.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:29 gibba032 bash[24004]: audit 2021-03-10T14:18:29.494956+0000 mon.c (mon.1) 28 : audit [DBG] from='client.? 172.21.2.111:0/1499058480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[30791]: audit 2021-03-10T14:18:29.001809+0000 mgr.gibba032.whopeo (mgr.24625) 1388 : audit [DBG] from='client.54976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[30791]: audit 2021-03-10T14:18:29.229698+0000 mgr.gibba032.whopeo (mgr.24625) 1389 : audit [DBG] from='client.54982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[30791]: cluster 2021-03-10T14:18:29.531773+0000 mgr.gibba032.whopeo (mgr.24625) 1390 : cluster [DBG] pgmap v781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[31774]: audit 2021-03-10T14:18:29.001809+0000 mgr.gibba032.whopeo (mgr.24625) 1388 : audit [DBG] from='client.54976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[31774]: audit 2021-03-10T14:18:29.229698+0000 mgr.gibba032.whopeo (mgr.24625) 1389 : audit [DBG] from='client.54982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:30 gibba011 bash[31774]: cluster 2021-03-10T14:18:29.531773+0000 mgr.gibba032.whopeo (mgr.24625) 1390 : cluster [DBG] pgmap v781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:31.173 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:30 gibba032 bash[24004]: audit 2021-03-10T14:18:29.001809+0000 mgr.gibba032.whopeo (mgr.24625) 1388 : audit [DBG] from='client.54976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.174 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:30 gibba032 bash[24004]: audit 2021-03-10T14:18:29.229698+0000 mgr.gibba032.whopeo (mgr.24625) 1389 : audit [DBG] from='client.54982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:18:31.174 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:30 gibba032 bash[24004]: cluster 2021-03-10T14:18:29.531773+0000 mgr.gibba032.whopeo (mgr.24625) 1390 : cluster [DBG] pgmap v781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:31 gibba032 bash[24004]: audit 2021-03-10T14:18:30.965186+0000 mon.a (mon.0) 961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:31 gibba032 bash[24004]: audit 2021-03-10T14:18:30.966211+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:18:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:31 gibba032 bash[24004]: cluster 2021-03-10T14:18:31.532742+0000 mgr.gibba032.whopeo (mgr.24625) 1391 : cluster [DBG] pgmap v782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[30791]: audit 2021-03-10T14:18:30.965186+0000 mon.a (mon.0) 961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[30791]: audit 2021-03-10T14:18:30.966211+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:18:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[30791]: cluster 2021-03-10T14:18:31.532742+0000 mgr.gibba032.whopeo (mgr.24625) 1391 : cluster [DBG] pgmap v782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[31774]: audit 2021-03-10T14:18:30.965186+0000 mon.a (mon.0) 961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[31774]: audit 2021-03-10T14:18:30.966211+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:18:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:31 gibba011 bash[31774]: cluster 2021-03-10T14:18:31.532742+0000 mgr.gibba032.whopeo (mgr.24625) 1391 : cluster [DBG] pgmap v782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:34 gibba032 bash[24004]: cluster 2021-03-10T14:18:33.534112+0000 mgr.gibba032.whopeo (mgr.24625) 1392 : cluster [DBG] pgmap v783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:34 gibba011 bash[30791]: cluster 2021-03-10T14:18:33.534112+0000 mgr.gibba032.whopeo (mgr.24625) 1392 : cluster [DBG] pgmap v783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:34 gibba011 bash[31774]: cluster 2021-03-10T14:18:33.534112+0000 mgr.gibba032.whopeo (mgr.24625) 1392 : cluster [DBG] pgmap v783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: cluster 2021-03-10T14:18:35.535464+0000 mgr.gibba032.whopeo (mgr.24625) 1393 : cluster [DBG] pgmap v784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.034572+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.210925+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.400933+0000 mon.a (mon.0) 965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.404724+0000 mon.a (mon.0) 966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:18:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.408424+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.412376+0000 mon.a (mon.0) 968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.415904+0000 mon.a (mon.0) 969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.422911+0000 mon.a (mon.0) 970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:36.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:36 gibba032 bash[24004]: audit 2021-03-10T14:18:36.428012+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: cluster 2021-03-10T14:18:35.535464+0000 mgr.gibba032.whopeo (mgr.24625) 1393 : cluster [DBG] pgmap v784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.034572+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.210925+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.400933+0000 mon.a (mon.0) 965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.404724+0000 mon.a (mon.0) 966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:18:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.408424+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.412376+0000 mon.a (mon.0) 968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.415904+0000 mon.a (mon.0) 969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.422911+0000 mon.a (mon.0) 970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[30791]: audit 2021-03-10T14:18:36.428012+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: cluster 2021-03-10T14:18:35.535464+0000 mgr.gibba032.whopeo (mgr.24625) 1393 : cluster [DBG] pgmap v784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.034572+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:18:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.210925+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.400933+0000 mon.a (mon.0) 965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:18:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.404724+0000 mon.a (mon.0) 966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:18:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.408424+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.412376+0000 mon.a (mon.0) 968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.415904+0000 mon.a (mon.0) 969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.422911+0000 mon.a (mon.0) 970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:37.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:36 gibba011 bash[31774]: audit 2021-03-10T14:18:36.428012+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:18:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.208885+0000 mgr.gibba032.whopeo (mgr.24625) 1394 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:18:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.401038+0000 mgr.gibba032.whopeo (mgr.24625) 1395 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:18:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.404469+0000 mgr.gibba032.whopeo (mgr.24625) 1396 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:18:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.408113+0000 mgr.gibba032.whopeo (mgr.24625) 1397 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:18:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.408600+0000 mgr.gibba032.whopeo (mgr.24625) 1398 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:18:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.412075+0000 mgr.gibba032.whopeo (mgr.24625) 1399 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:18:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.412367+0000 mgr.gibba032.whopeo (mgr.24625) 1400 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:18:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.415558+0000 mgr.gibba032.whopeo (mgr.24625) 1401 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:18:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.418286+0000 mgr.gibba032.whopeo (mgr.24625) 1402 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:18:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.418517+0000 mgr.gibba032.whopeo (mgr.24625) 1403 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:18:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.422572+0000 mgr.gibba032.whopeo (mgr.24625) 1404 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:18:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.424548+0000 mgr.gibba032.whopeo (mgr.24625) 1405 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:18:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.427638+0000 mgr.gibba032.whopeo (mgr.24625) 1406 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:18:38.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.427876+0000 mgr.gibba032.whopeo (mgr.24625) 1407 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:18:38.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:37 gibba032 bash[24004]: cephadm 2021-03-10T14:18:36.428741+0000 mgr.gibba032.whopeo (mgr.24625) 1408 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.208885+0000 mgr.gibba032.whopeo (mgr.24625) 1394 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:18:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.401038+0000 mgr.gibba032.whopeo (mgr.24625) 1395 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:18:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.404469+0000 mgr.gibba032.whopeo (mgr.24625) 1396 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:18:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.408113+0000 mgr.gibba032.whopeo (mgr.24625) 1397 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:18:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.408600+0000 mgr.gibba032.whopeo (mgr.24625) 1398 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:18:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.412075+0000 mgr.gibba032.whopeo (mgr.24625) 1399 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:18:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.412367+0000 mgr.gibba032.whopeo (mgr.24625) 1400 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:18:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.415558+0000 mgr.gibba032.whopeo (mgr.24625) 1401 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:18:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.418286+0000 mgr.gibba032.whopeo (mgr.24625) 1402 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:18:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.418517+0000 mgr.gibba032.whopeo (mgr.24625) 1403 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:18:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.422572+0000 mgr.gibba032.whopeo (mgr.24625) 1404 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:18:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.424548+0000 mgr.gibba032.whopeo (mgr.24625) 1405 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:18:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.427638+0000 mgr.gibba032.whopeo (mgr.24625) 1406 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:18:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.427876+0000 mgr.gibba032.whopeo (mgr.24625) 1407 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:18:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[30791]: cephadm 2021-03-10T14:18:36.428741+0000 mgr.gibba032.whopeo (mgr.24625) 1408 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.208885+0000 mgr.gibba032.whopeo (mgr.24625) 1394 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:18:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.401038+0000 mgr.gibba032.whopeo (mgr.24625) 1395 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:18:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.404469+0000 mgr.gibba032.whopeo (mgr.24625) 1396 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:18:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.408113+0000 mgr.gibba032.whopeo (mgr.24625) 1397 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:18:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.408600+0000 mgr.gibba032.whopeo (mgr.24625) 1398 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:18:38.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.412075+0000 mgr.gibba032.whopeo (mgr.24625) 1399 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:18:38.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.412367+0000 mgr.gibba032.whopeo (mgr.24625) 1400 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:18:38.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.415558+0000 mgr.gibba032.whopeo (mgr.24625) 1401 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:18:38.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.418286+0000 mgr.gibba032.whopeo (mgr.24625) 1402 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:18:38.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.418517+0000 mgr.gibba032.whopeo (mgr.24625) 1403 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:18:38.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.422572+0000 mgr.gibba032.whopeo (mgr.24625) 1404 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:18:38.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.424548+0000 mgr.gibba032.whopeo (mgr.24625) 1405 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:18:38.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.427638+0000 mgr.gibba032.whopeo (mgr.24625) 1406 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:18:38.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.427876+0000 mgr.gibba032.whopeo (mgr.24625) 1407 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:18:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:37 gibba011 bash[31774]: cephadm 2021-03-10T14:18:36.428741+0000 mgr.gibba032.whopeo (mgr.24625) 1408 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:38 gibba032 bash[24004]: cluster 2021-03-10T14:18:37.536530+0000 mgr.gibba032.whopeo (mgr.24625) 1409 : cluster [DBG] pgmap v785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:38 gibba011 bash[30791]: cluster 2021-03-10T14:18:37.536530+0000 mgr.gibba032.whopeo (mgr.24625) 1409 : cluster [DBG] pgmap v785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:38 gibba011 bash[31774]: cluster 2021-03-10T14:18:37.536530+0000 mgr.gibba032.whopeo (mgr.24625) 1409 : cluster [DBG] pgmap v785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:40 gibba032 bash[24004]: cluster 2021-03-10T14:18:39.537927+0000 mgr.gibba032.whopeo (mgr.24625) 1410 : cluster [DBG] pgmap v786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:40 gibba011 bash[30791]: cluster 2021-03-10T14:18:39.537927+0000 mgr.gibba032.whopeo (mgr.24625) 1410 : cluster [DBG] pgmap v786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:40 gibba011 bash[31774]: cluster 2021-03-10T14:18:39.537927+0000 mgr.gibba032.whopeo (mgr.24625) 1410 : cluster [DBG] pgmap v786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:42 gibba032 bash[24004]: cluster 2021-03-10T14:18:41.539647+0000 mgr.gibba032.whopeo (mgr.24625) 1411 : cluster [DBG] pgmap v787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:43.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:42 gibba011 bash[30791]: cluster 2021-03-10T14:18:41.539647+0000 mgr.gibba032.whopeo (mgr.24625) 1411 : cluster [DBG] pgmap v787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:42 gibba011 bash[31774]: cluster 2021-03-10T14:18:41.539647+0000 mgr.gibba032.whopeo (mgr.24625) 1411 : cluster [DBG] pgmap v787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:44 gibba032 bash[24004]: cluster 2021-03-10T14:18:43.540584+0000 mgr.gibba032.whopeo (mgr.24625) 1412 : cluster [DBG] pgmap v788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:44 gibba011 bash[30791]: cluster 2021-03-10T14:18:43.540584+0000 mgr.gibba032.whopeo (mgr.24625) 1412 : cluster [DBG] pgmap v788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:44 gibba011 bash[31774]: cluster 2021-03-10T14:18:43.540584+0000 mgr.gibba032.whopeo (mgr.24625) 1412 : cluster [DBG] pgmap v788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:46 gibba032 bash[24004]: cluster 2021-03-10T14:18:45.542079+0000 mgr.gibba032.whopeo (mgr.24625) 1413 : cluster [DBG] pgmap v789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:46 gibba011 bash[30791]: cluster 2021-03-10T14:18:45.542079+0000 mgr.gibba032.whopeo (mgr.24625) 1413 : cluster [DBG] pgmap v789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:46 gibba011 bash[31774]: cluster 2021-03-10T14:18:45.542079+0000 mgr.gibba032.whopeo (mgr.24625) 1413 : cluster [DBG] pgmap v789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:48 gibba032 bash[24004]: cluster 2021-03-10T14:18:47.543856+0000 mgr.gibba032.whopeo (mgr.24625) 1414 : cluster [DBG] pgmap v790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:48 gibba011 bash[30791]: cluster 2021-03-10T14:18:47.543856+0000 mgr.gibba032.whopeo (mgr.24625) 1414 : cluster [DBG] pgmap v790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:48 gibba011 bash[31774]: cluster 2021-03-10T14:18:47.543856+0000 mgr.gibba032.whopeo (mgr.24625) 1414 : cluster [DBG] pgmap v790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:50 gibba032 bash[24004]: cluster 2021-03-10T14:18:49.545302+0000 mgr.gibba032.whopeo (mgr.24625) 1415 : cluster [DBG] pgmap v791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:50 gibba011 bash[30791]: cluster 2021-03-10T14:18:49.545302+0000 mgr.gibba032.whopeo (mgr.24625) 1415 : cluster [DBG] pgmap v791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:50 gibba011 bash[31774]: cluster 2021-03-10T14:18:49.545302+0000 mgr.gibba032.whopeo (mgr.24625) 1415 : cluster [DBG] pgmap v791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:52 gibba032 bash[24004]: cephadm 2021-03-10T14:18:51.444997+0000 mgr.gibba032.whopeo (mgr.24625) 1416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:53.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:52 gibba032 bash[24004]: cluster 2021-03-10T14:18:51.547024+0000 mgr.gibba032.whopeo (mgr.24625) 1417 : cluster [DBG] pgmap v792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:52 gibba011 bash[30791]: cephadm 2021-03-10T14:18:51.444997+0000 mgr.gibba032.whopeo (mgr.24625) 1416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:52 gibba011 bash[30791]: cluster 2021-03-10T14:18:51.547024+0000 mgr.gibba032.whopeo (mgr.24625) 1417 : cluster [DBG] pgmap v792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:52 gibba011 bash[31774]: cephadm 2021-03-10T14:18:51.444997+0000 mgr.gibba032.whopeo (mgr.24625) 1416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:18:53.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:52 gibba011 bash[31774]: cluster 2021-03-10T14:18:51.547024+0000 mgr.gibba032.whopeo (mgr.24625) 1417 : cluster [DBG] pgmap v792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:54 gibba032 bash[24004]: cluster 2021-03-10T14:18:53.548392+0000 mgr.gibba032.whopeo (mgr.24625) 1418 : cluster [DBG] pgmap v793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:54 gibba011 bash[30791]: cluster 2021-03-10T14:18:53.548392+0000 mgr.gibba032.whopeo (mgr.24625) 1418 : cluster [DBG] pgmap v793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:54 gibba011 bash[31774]: cluster 2021-03-10T14:18:53.548392+0000 mgr.gibba032.whopeo (mgr.24625) 1418 : cluster [DBG] pgmap v793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: cluster 2021-03-10T14:18:55.549270+0000 mgr.gibba032.whopeo (mgr.24625) 1419 : cluster [DBG] pgmap v794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.103988+0000 mon.a (mon.0) 972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:18:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.104716+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.105464+0000 mon.a (mon.0) 974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:18:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.106159+0000 mon.a (mon.0) 975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.166282+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:18:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:56 gibba032 bash[24004]: audit 2021-03-10T14:18:56.166915+0000 mon.a (mon.0) 977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:18:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: cluster 2021-03-10T14:18:55.549270+0000 mgr.gibba032.whopeo (mgr.24625) 1419 : cluster [DBG] pgmap v794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.103988+0000 mon.a (mon.0) 972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:18:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.104716+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.105464+0000 mon.a (mon.0) 974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:18:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.106159+0000 mon.a (mon.0) 975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.166282+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:18:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[30791]: audit 2021-03-10T14:18:56.166915+0000 mon.a (mon.0) 977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:18:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: cluster 2021-03-10T14:18:55.549270+0000 mgr.gibba032.whopeo (mgr.24625) 1419 : cluster [DBG] pgmap v794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.103988+0000 mon.a (mon.0) 972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:18:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.104716+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.105464+0000 mon.a (mon.0) 974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:18:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.106159+0000 mon.a (mon.0) 975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:18:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.166282+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:18:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:56 gibba011 bash[31774]: audit 2021-03-10T14:18:56.166915+0000 mon.a (mon.0) 977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:18:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:18:58 gibba032 bash[24004]: cluster 2021-03-10T14:18:57.550812+0000 mgr.gibba032.whopeo (mgr.24625) 1420 : cluster [DBG] pgmap v795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:18:58 gibba011 bash[30791]: cluster 2021-03-10T14:18:57.550812+0000 mgr.gibba032.whopeo (mgr.24625) 1420 : cluster [DBG] pgmap v795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:18:58 gibba011 bash[31774]: cluster 2021-03-10T14:18:57.550812+0000 mgr.gibba032.whopeo (mgr.24625) 1420 : cluster [DBG] pgmap v795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:18:59.736 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:18:59.966 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:18:59.967 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (25m) 93s ago 31m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:18:59.967 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (30m) 29s ago 30m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:18:59.967 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (25m) 29s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:18:59.967 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (26m) 29s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:18:59.968 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (25m) 93s ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:18:59.968 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (25m) 93s ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:18:59.968 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (24m) 29s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:18:59.968 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (24m) 93s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:18:59.969 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (31m) 93s ago 31m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:18:59.969 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (31m) 29s ago 31m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:18:59.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (23m) 93s ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:18:59.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (23m) 93s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:18:59.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (23m) 93s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:18:59.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (22m) 93s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:18:59.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (22m) 29s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:18:59.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (21m) 29s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:18:59.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (21m) 29s ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:18:59.971 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (20m) 29s ago 31m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:18:59.971 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (30m) 29s ago 31m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:18:59.971 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 93s ago 29m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:19:00.219 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:19:00.219 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:19:00.219 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:19:00.220 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:00.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:19:00.220 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:19:00.220 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:00.221 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:19:00.221 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:19:00.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:00.221 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:19:00.222 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:19:00.222 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:19:00.222 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:19:00.222 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:19:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:00 gibba032 bash[24004]: cluster 2021-03-10T14:18:59.552366+0000 mgr.gibba032.whopeo (mgr.24625) 1421 : cluster [DBG] pgmap v796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:01.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:00 gibba032 bash[24004]: audit 2021-03-10T14:18:59.724814+0000 mgr.gibba032.whopeo (mgr.24625) 1422 : audit [DBG] from='client.54994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:01.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:00 gibba032 bash[24004]: audit 2021-03-10T14:19:00.218904+0000 mon.c (mon.1) 29 : audit [DBG] from='client.? 172.21.2.111:0/2943350559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[30791]: cluster 2021-03-10T14:18:59.552366+0000 mgr.gibba032.whopeo (mgr.24625) 1421 : cluster [DBG] pgmap v796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[30791]: audit 2021-03-10T14:18:59.724814+0000 mgr.gibba032.whopeo (mgr.24625) 1422 : audit [DBG] from='client.54994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[30791]: audit 2021-03-10T14:19:00.218904+0000 mon.c (mon.1) 29 : audit [DBG] from='client.? 172.21.2.111:0/2943350559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[31774]: cluster 2021-03-10T14:18:59.552366+0000 mgr.gibba032.whopeo (mgr.24625) 1421 : cluster [DBG] pgmap v796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[31774]: audit 2021-03-10T14:18:59.724814+0000 mgr.gibba032.whopeo (mgr.24625) 1422 : audit [DBG] from='client.54994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:00 gibba011 bash[31774]: audit 2021-03-10T14:19:00.218904+0000 mon.c (mon.1) 29 : audit [DBG] from='client.? 172.21.2.111:0/2943350559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:02.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:01 gibba032 bash[24004]: audit 2021-03-10T14:18:59.952373+0000 mgr.gibba032.whopeo (mgr.24625) 1423 : audit [DBG] from='client.55000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:01 gibba011 bash[30791]: audit 2021-03-10T14:18:59.952373+0000 mgr.gibba032.whopeo (mgr.24625) 1423 : audit [DBG] from='client.55000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:01 gibba011 bash[31774]: audit 2021-03-10T14:18:59.952373+0000 mgr.gibba032.whopeo (mgr.24625) 1423 : audit [DBG] from='client.55000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:02 gibba032 bash[24004]: cluster 2021-03-10T14:19:01.553317+0000 mgr.gibba032.whopeo (mgr.24625) 1424 : cluster [DBG] pgmap v797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:02 gibba011 bash[30791]: cluster 2021-03-10T14:19:01.553317+0000 mgr.gibba032.whopeo (mgr.24625) 1424 : cluster [DBG] pgmap v797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:02 gibba011 bash[31774]: cluster 2021-03-10T14:19:01.553317+0000 mgr.gibba032.whopeo (mgr.24625) 1424 : cluster [DBG] pgmap v797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:04 gibba032 bash[24004]: cluster 2021-03-10T14:19:03.554525+0000 mgr.gibba032.whopeo (mgr.24625) 1425 : cluster [DBG] pgmap v798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:04 gibba011 bash[30791]: cluster 2021-03-10T14:19:03.554525+0000 mgr.gibba032.whopeo (mgr.24625) 1425 : cluster [DBG] pgmap v798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:04 gibba011 bash[31774]: cluster 2021-03-10T14:19:03.554525+0000 mgr.gibba032.whopeo (mgr.24625) 1425 : cluster [DBG] pgmap v798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:06 gibba032 bash[24004]: cluster 2021-03-10T14:19:05.556173+0000 mgr.gibba032.whopeo (mgr.24625) 1426 : cluster [DBG] pgmap v799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:06 gibba011 bash[30791]: cluster 2021-03-10T14:19:05.556173+0000 mgr.gibba032.whopeo (mgr.24625) 1426 : cluster [DBG] pgmap v799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:06 gibba011 bash[31774]: cluster 2021-03-10T14:19:05.556173+0000 mgr.gibba032.whopeo (mgr.24625) 1426 : cluster [DBG] pgmap v799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:08.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:07 gibba011 bash[30791]: cephadm 2021-03-10T14:19:06.461082+0000 mgr.gibba032.whopeo (mgr.24625) 1427 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:08.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:07 gibba011 bash[31774]: cephadm 2021-03-10T14:19:06.461082+0000 mgr.gibba032.whopeo (mgr.24625) 1427 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:08.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:07 gibba032 bash[24004]: cephadm 2021-03-10T14:19:06.461082+0000 mgr.gibba032.whopeo (mgr.24625) 1427 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:08 gibba011 bash[30791]: cluster 2021-03-10T14:19:07.557232+0000 mgr.gibba032.whopeo (mgr.24625) 1428 : cluster [DBG] pgmap v800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:08 gibba011 bash[31774]: cluster 2021-03-10T14:19:07.557232+0000 mgr.gibba032.whopeo (mgr.24625) 1428 : cluster [DBG] pgmap v800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:08 gibba032 bash[24004]: cluster 2021-03-10T14:19:07.557232+0000 mgr.gibba032.whopeo (mgr.24625) 1428 : cluster [DBG] pgmap v800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:10 gibba011 bash[30791]: cluster 2021-03-10T14:19:09.558579+0000 mgr.gibba032.whopeo (mgr.24625) 1429 : cluster [DBG] pgmap v801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:10 gibba011 bash[31774]: cluster 2021-03-10T14:19:09.558579+0000 mgr.gibba032.whopeo (mgr.24625) 1429 : cluster [DBG] pgmap v801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:10 gibba032 bash[24004]: cluster 2021-03-10T14:19:09.558579+0000 mgr.gibba032.whopeo (mgr.24625) 1429 : cluster [DBG] pgmap v801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:12 gibba011 bash[30791]: cluster 2021-03-10T14:19:11.560416+0000 mgr.gibba032.whopeo (mgr.24625) 1430 : cluster [DBG] pgmap v802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:12 gibba011 bash[31774]: cluster 2021-03-10T14:19:11.560416+0000 mgr.gibba032.whopeo (mgr.24625) 1430 : cluster [DBG] pgmap v802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:12 gibba032 bash[24004]: cluster 2021-03-10T14:19:11.560416+0000 mgr.gibba032.whopeo (mgr.24625) 1430 : cluster [DBG] pgmap v802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:14 gibba011 bash[30791]: cluster 2021-03-10T14:19:13.561677+0000 mgr.gibba032.whopeo (mgr.24625) 1431 : cluster [DBG] pgmap v803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:14 gibba011 bash[31774]: cluster 2021-03-10T14:19:13.561677+0000 mgr.gibba032.whopeo (mgr.24625) 1431 : cluster [DBG] pgmap v803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:14 gibba032 bash[24004]: cluster 2021-03-10T14:19:13.561677+0000 mgr.gibba032.whopeo (mgr.24625) 1431 : cluster [DBG] pgmap v803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:16 gibba011 bash[30791]: cluster 2021-03-10T14:19:15.563166+0000 mgr.gibba032.whopeo (mgr.24625) 1432 : cluster [DBG] pgmap v804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:16 gibba011 bash[31774]: cluster 2021-03-10T14:19:15.563166+0000 mgr.gibba032.whopeo (mgr.24625) 1432 : cluster [DBG] pgmap v804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:16 gibba032 bash[24004]: cluster 2021-03-10T14:19:15.563166+0000 mgr.gibba032.whopeo (mgr.24625) 1432 : cluster [DBG] pgmap v804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:18 gibba011 bash[30791]: cluster 2021-03-10T14:19:17.564840+0000 mgr.gibba032.whopeo (mgr.24625) 1433 : cluster [DBG] pgmap v805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:18 gibba011 bash[31774]: cluster 2021-03-10T14:19:17.564840+0000 mgr.gibba032.whopeo (mgr.24625) 1433 : cluster [DBG] pgmap v805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:18 gibba032 bash[24004]: cluster 2021-03-10T14:19:17.564840+0000 mgr.gibba032.whopeo (mgr.24625) 1433 : cluster [DBG] pgmap v805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:20 gibba011 bash[30791]: cluster 2021-03-10T14:19:19.565970+0000 mgr.gibba032.whopeo (mgr.24625) 1434 : cluster [DBG] pgmap v806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:20 gibba011 bash[31774]: cluster 2021-03-10T14:19:19.565970+0000 mgr.gibba032.whopeo (mgr.24625) 1434 : cluster [DBG] pgmap v806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:20 gibba032 bash[24004]: cluster 2021-03-10T14:19:19.565970+0000 mgr.gibba032.whopeo (mgr.24625) 1434 : cluster [DBG] pgmap v806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:22 gibba032 bash[24004]: cephadm 2021-03-10T14:19:21.477115+0000 mgr.gibba032.whopeo (mgr.24625) 1435 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:22 gibba032 bash[24004]: cluster 2021-03-10T14:19:21.567730+0000 mgr.gibba032.whopeo (mgr.24625) 1436 : cluster [DBG] pgmap v807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:22 gibba011 bash[30791]: cephadm 2021-03-10T14:19:21.477115+0000 mgr.gibba032.whopeo (mgr.24625) 1435 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:22 gibba011 bash[30791]: cluster 2021-03-10T14:19:21.567730+0000 mgr.gibba032.whopeo (mgr.24625) 1436 : cluster [DBG] pgmap v807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:22 gibba011 bash[31774]: cephadm 2021-03-10T14:19:21.477115+0000 mgr.gibba032.whopeo (mgr.24625) 1435 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:23.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:22 gibba011 bash[31774]: cluster 2021-03-10T14:19:21.567730+0000 mgr.gibba032.whopeo (mgr.24625) 1436 : cluster [DBG] pgmap v807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:24 gibba032 bash[24004]: cluster 2021-03-10T14:19:23.569321+0000 mgr.gibba032.whopeo (mgr.24625) 1437 : cluster [DBG] pgmap v808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:24 gibba011 bash[30791]: cluster 2021-03-10T14:19:23.569321+0000 mgr.gibba032.whopeo (mgr.24625) 1437 : cluster [DBG] pgmap v808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:24 gibba011 bash[31774]: cluster 2021-03-10T14:19:23.569321+0000 mgr.gibba032.whopeo (mgr.24625) 1437 : cluster [DBG] pgmap v808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:26 gibba032 bash[24004]: cluster 2021-03-10T14:19:25.570712+0000 mgr.gibba032.whopeo (mgr.24625) 1438 : cluster [DBG] pgmap v809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:26 gibba011 bash[30791]: cluster 2021-03-10T14:19:25.570712+0000 mgr.gibba032.whopeo (mgr.24625) 1438 : cluster [DBG] pgmap v809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:26 gibba011 bash[31774]: cluster 2021-03-10T14:19:25.570712+0000 mgr.gibba032.whopeo (mgr.24625) 1438 : cluster [DBG] pgmap v809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:28 gibba032 bash[24004]: cluster 2021-03-10T14:19:27.572505+0000 mgr.gibba032.whopeo (mgr.24625) 1439 : cluster [DBG] pgmap v810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:28 gibba011 bash[30791]: cluster 2021-03-10T14:19:27.572505+0000 mgr.gibba032.whopeo (mgr.24625) 1439 : cluster [DBG] pgmap v810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:28 gibba011 bash[31774]: cluster 2021-03-10T14:19:27.572505+0000 mgr.gibba032.whopeo (mgr.24625) 1439 : cluster [DBG] pgmap v810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:30.464 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:19:30.693 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:19:30.694 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (26m) 2m ago 31m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:19:30.694 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (31m) 59s ago 31m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:19:30.694 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (26m) 59s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:19:30.694 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (26m) 59s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:19:30.694 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (25m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:19:30.695 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (25m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:19:30.695 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (25m) 59s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:19:30.695 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (25m) 2m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:19:30.695 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (31m) 2m ago 31m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:19:30.696 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (31m) 59s ago 31m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:19:30.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (24m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:19:30.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (24m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:19:30.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (23m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:19:30.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (23m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:19:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (22m) 59s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:19:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (22m) 59s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:19:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (21m) 59s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:19:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (21m) 59s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:19:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (31m) 59s ago 31m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:19:30.698 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 29m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:19:30.943 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:19:30.943 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:19:30.943 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:19:30.944 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:30.944 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:19:30.944 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:19:30.944 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:30.944 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:19:30.945 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:19:30.945 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:19:30.945 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:19:30.945 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:19:30.946 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:19:30.946 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:19:30.946 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:19:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:30 gibba032 bash[24004]: cluster 2021-03-10T14:19:29.573850+0000 mgr.gibba032.whopeo (mgr.24625) 1440 : cluster [DBG] pgmap v811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:30 gibba011 bash[30791]: cluster 2021-03-10T14:19:29.573850+0000 mgr.gibba032.whopeo (mgr.24625) 1440 : cluster [DBG] pgmap v811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:30 gibba011 bash[31774]: cluster 2021-03-10T14:19:29.573850+0000 mgr.gibba032.whopeo (mgr.24625) 1440 : cluster [DBG] pgmap v811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:31 gibba032 bash[24004]: audit 2021-03-10T14:19:30.451828+0000 mgr.gibba032.whopeo (mgr.24625) 1441 : audit [DBG] from='client.55012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:31 gibba032 bash[24004]: audit 2021-03-10T14:19:30.678100+0000 mgr.gibba032.whopeo (mgr.24625) 1442 : audit [DBG] from='client.45033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:31 gibba032 bash[24004]: audit 2021-03-10T14:19:30.943136+0000 mon.a (mon.0) 978 : audit [DBG] from='client.? 172.21.2.111:0/3422870379' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[30791]: audit 2021-03-10T14:19:30.451828+0000 mgr.gibba032.whopeo (mgr.24625) 1441 : audit [DBG] from='client.55012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[30791]: audit 2021-03-10T14:19:30.678100+0000 mgr.gibba032.whopeo (mgr.24625) 1442 : audit [DBG] from='client.45033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[30791]: audit 2021-03-10T14:19:30.943136+0000 mon.a (mon.0) 978 : audit [DBG] from='client.? 172.21.2.111:0/3422870379' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[31774]: audit 2021-03-10T14:19:30.451828+0000 mgr.gibba032.whopeo (mgr.24625) 1441 : audit [DBG] from='client.55012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[31774]: audit 2021-03-10T14:19:30.678100+0000 mgr.gibba032.whopeo (mgr.24625) 1442 : audit [DBG] from='client.45033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:19:32.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:31 gibba011 bash[31774]: audit 2021-03-10T14:19:30.943136+0000 mon.a (mon.0) 978 : audit [DBG] from='client.? 172.21.2.111:0/3422870379' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:32 gibba032 bash[24004]: cluster 2021-03-10T14:19:31.575615+0000 mgr.gibba032.whopeo (mgr.24625) 1443 : cluster [DBG] pgmap v812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:32 gibba011 bash[30791]: cluster 2021-03-10T14:19:31.575615+0000 mgr.gibba032.whopeo (mgr.24625) 1443 : cluster [DBG] pgmap v812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:32 gibba011 bash[31774]: cluster 2021-03-10T14:19:31.575615+0000 mgr.gibba032.whopeo (mgr.24625) 1443 : cluster [DBG] pgmap v812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:34.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:33 gibba032 bash[24004]: cluster 2021-03-10T14:19:33.577254+0000 mgr.gibba032.whopeo (mgr.24625) 1444 : cluster [DBG] pgmap v813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:33 gibba011 bash[30791]: cluster 2021-03-10T14:19:33.577254+0000 mgr.gibba032.whopeo (mgr.24625) 1444 : cluster [DBG] pgmap v813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:33 gibba011 bash[31774]: cluster 2021-03-10T14:19:33.577254+0000 mgr.gibba032.whopeo (mgr.24625) 1444 : cluster [DBG] pgmap v813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:36 gibba032 bash[24004]: cluster 2021-03-10T14:19:35.578722+0000 mgr.gibba032.whopeo (mgr.24625) 1445 : cluster [DBG] pgmap v814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:36 gibba011 bash[30791]: cluster 2021-03-10T14:19:35.578722+0000 mgr.gibba032.whopeo (mgr.24625) 1445 : cluster [DBG] pgmap v814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:36 gibba011 bash[31774]: cluster 2021-03-10T14:19:35.578722+0000 mgr.gibba032.whopeo (mgr.24625) 1445 : cluster [DBG] pgmap v814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:38.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.493735+0000 mgr.gibba032.whopeo (mgr.24625) 1446 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:19:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.495279+0000 mgr.gibba032.whopeo (mgr.24625) 1447 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:19:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.721609+0000 mgr.gibba032.whopeo (mgr.24625) 1448 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:19:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.721735+0000 mon.a (mon.0) 979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:19:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.724862+0000 mgr.gibba032.whopeo (mgr.24625) 1449 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:19:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.725233+0000 mon.a (mon.0) 980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:19:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.727561+0000 mgr.gibba032.whopeo (mgr.24625) 1450 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:19:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.727602+0000 mgr.gibba032.whopeo (mgr.24625) 1451 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:19:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.728101+0000 mon.a (mon.0) 981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.730206+0000 mgr.gibba032.whopeo (mgr.24625) 1452 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:19:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.730242+0000 mgr.gibba032.whopeo (mgr.24625) 1453 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:19:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.730820+0000 mon.a (mon.0) 982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.732410+0000 mgr.gibba032.whopeo (mgr.24625) 1454 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:19:38.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.733391+0000 mon.a (mon.0) 983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.734714+0000 mgr.gibba032.whopeo (mgr.24625) 1455 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:19:38.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.734752+0000 mgr.gibba032.whopeo (mgr.24625) 1456 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:19:38.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.737434+0000 mgr.gibba032.whopeo (mgr.24625) 1457 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:19:38.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.737747+0000 mgr.gibba032.whopeo (mgr.24625) 1458 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:19:38.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.738066+0000 mon.a (mon.0) 984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.739923+0000 mgr.gibba032.whopeo (mgr.24625) 1459 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:19:38.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.739962+0000 mgr.gibba032.whopeo (mgr.24625) 1460 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:19:38.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: cephadm 2021-03-10T14:19:36.740176+0000 mgr.gibba032.whopeo (mgr.24625) 1461 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:38.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:37 gibba032 bash[24004]: audit 2021-03-10T14:19:36.740873+0000 mon.a (mon.0) 985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.493735+0000 mgr.gibba032.whopeo (mgr.24625) 1446 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:19:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.495279+0000 mgr.gibba032.whopeo (mgr.24625) 1447 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:19:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.721609+0000 mgr.gibba032.whopeo (mgr.24625) 1448 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:19:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.721735+0000 mon.a (mon.0) 979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:19:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.724862+0000 mgr.gibba032.whopeo (mgr.24625) 1449 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:19:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.725233+0000 mon.a (mon.0) 980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:19:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.727561+0000 mgr.gibba032.whopeo (mgr.24625) 1450 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:19:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.727602+0000 mgr.gibba032.whopeo (mgr.24625) 1451 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:19:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.728101+0000 mon.a (mon.0) 981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.730206+0000 mgr.gibba032.whopeo (mgr.24625) 1452 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:19:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.730242+0000 mgr.gibba032.whopeo (mgr.24625) 1453 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:19:38.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.730820+0000 mon.a (mon.0) 982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.732410+0000 mgr.gibba032.whopeo (mgr.24625) 1454 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:19:38.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.733391+0000 mon.a (mon.0) 983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.734714+0000 mgr.gibba032.whopeo (mgr.24625) 1455 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:19:38.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.734752+0000 mgr.gibba032.whopeo (mgr.24625) 1456 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:19:38.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.737434+0000 mgr.gibba032.whopeo (mgr.24625) 1457 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:19:38.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.737747+0000 mgr.gibba032.whopeo (mgr.24625) 1458 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:19:38.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.738066+0000 mon.a (mon.0) 984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.739923+0000 mgr.gibba032.whopeo (mgr.24625) 1459 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:19:38.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.739962+0000 mgr.gibba032.whopeo (mgr.24625) 1460 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:19:38.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: cephadm 2021-03-10T14:19:36.740176+0000 mgr.gibba032.whopeo (mgr.24625) 1461 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:38.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[30791]: audit 2021-03-10T14:19:36.740873+0000 mon.a (mon.0) 985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.493735+0000 mgr.gibba032.whopeo (mgr.24625) 1446 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:19:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.495279+0000 mgr.gibba032.whopeo (mgr.24625) 1447 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:19:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.721609+0000 mgr.gibba032.whopeo (mgr.24625) 1448 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:19:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.721735+0000 mon.a (mon.0) 979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:19:38.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.724862+0000 mgr.gibba032.whopeo (mgr.24625) 1449 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:19:38.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.725233+0000 mon.a (mon.0) 980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:19:38.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.727561+0000 mgr.gibba032.whopeo (mgr.24625) 1450 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:19:38.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.727602+0000 mgr.gibba032.whopeo (mgr.24625) 1451 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:19:38.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.728101+0000 mon.a (mon.0) 981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.730206+0000 mgr.gibba032.whopeo (mgr.24625) 1452 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:19:38.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.730242+0000 mgr.gibba032.whopeo (mgr.24625) 1453 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:19:38.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.730820+0000 mon.a (mon.0) 982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.732410+0000 mgr.gibba032.whopeo (mgr.24625) 1454 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:19:38.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.733391+0000 mon.a (mon.0) 983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.734714+0000 mgr.gibba032.whopeo (mgr.24625) 1455 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:19:38.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.734752+0000 mgr.gibba032.whopeo (mgr.24625) 1456 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:19:38.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.737434+0000 mgr.gibba032.whopeo (mgr.24625) 1457 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:19:38.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.737747+0000 mgr.gibba032.whopeo (mgr.24625) 1458 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:19:38.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.738066+0000 mon.a (mon.0) 984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:38.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.739923+0000 mgr.gibba032.whopeo (mgr.24625) 1459 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:19:38.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.739962+0000 mgr.gibba032.whopeo (mgr.24625) 1460 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:19:38.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: cephadm 2021-03-10T14:19:36.740176+0000 mgr.gibba032.whopeo (mgr.24625) 1461 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:38.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:37 gibba011 bash[31774]: audit 2021-03-10T14:19:36.740873+0000 mon.a (mon.0) 985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:19:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:38 gibba032 bash[24004]: cluster 2021-03-10T14:19:37.580822+0000 mgr.gibba032.whopeo (mgr.24625) 1462 : cluster [DBG] pgmap v815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:38 gibba011 bash[30791]: cluster 2021-03-10T14:19:37.580822+0000 mgr.gibba032.whopeo (mgr.24625) 1462 : cluster [DBG] pgmap v815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:38 gibba011 bash[31774]: cluster 2021-03-10T14:19:37.580822+0000 mgr.gibba032.whopeo (mgr.24625) 1462 : cluster [DBG] pgmap v815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:40 gibba032 bash[24004]: cluster 2021-03-10T14:19:39.582294+0000 mgr.gibba032.whopeo (mgr.24625) 1463 : cluster [DBG] pgmap v816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:40 gibba011 bash[30791]: cluster 2021-03-10T14:19:39.582294+0000 mgr.gibba032.whopeo (mgr.24625) 1463 : cluster [DBG] pgmap v816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:40 gibba011 bash[31774]: cluster 2021-03-10T14:19:39.582294+0000 mgr.gibba032.whopeo (mgr.24625) 1463 : cluster [DBG] pgmap v816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:42 gibba011 bash[30791]: cluster 2021-03-10T14:19:41.583916+0000 mgr.gibba032.whopeo (mgr.24625) 1464 : cluster [DBG] pgmap v817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:42 gibba011 bash[31774]: cluster 2021-03-10T14:19:41.583916+0000 mgr.gibba032.whopeo (mgr.24625) 1464 : cluster [DBG] pgmap v817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:42 gibba032 bash[24004]: cluster 2021-03-10T14:19:41.583916+0000 mgr.gibba032.whopeo (mgr.24625) 1464 : cluster [DBG] pgmap v817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:44 gibba011 bash[30791]: cluster 2021-03-10T14:19:43.585559+0000 mgr.gibba032.whopeo (mgr.24625) 1465 : cluster [DBG] pgmap v818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:44 gibba011 bash[31774]: cluster 2021-03-10T14:19:43.585559+0000 mgr.gibba032.whopeo (mgr.24625) 1465 : cluster [DBG] pgmap v818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:44 gibba032 bash[24004]: cluster 2021-03-10T14:19:43.585559+0000 mgr.gibba032.whopeo (mgr.24625) 1465 : cluster [DBG] pgmap v818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:46 gibba032 bash[24004]: cluster 2021-03-10T14:19:45.591571+0000 mgr.gibba032.whopeo (mgr.24625) 1466 : cluster [DBG] pgmap v819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:46 gibba011 bash[30791]: cluster 2021-03-10T14:19:45.591571+0000 mgr.gibba032.whopeo (mgr.24625) 1466 : cluster [DBG] pgmap v819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:47.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:46 gibba011 bash[31774]: cluster 2021-03-10T14:19:45.591571+0000 mgr.gibba032.whopeo (mgr.24625) 1466 : cluster [DBG] pgmap v819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:48 gibba032 bash[24004]: cluster 2021-03-10T14:19:47.593283+0000 mgr.gibba032.whopeo (mgr.24625) 1467 : cluster [DBG] pgmap v820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:48 gibba011 bash[30791]: cluster 2021-03-10T14:19:47.593283+0000 mgr.gibba032.whopeo (mgr.24625) 1467 : cluster [DBG] pgmap v820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:48 gibba011 bash[31774]: cluster 2021-03-10T14:19:47.593283+0000 mgr.gibba032.whopeo (mgr.24625) 1467 : cluster [DBG] pgmap v820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:50 gibba032 bash[24004]: cluster 2021-03-10T14:19:49.594696+0000 mgr.gibba032.whopeo (mgr.24625) 1468 : cluster [DBG] pgmap v821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:50 gibba011 bash[30791]: cluster 2021-03-10T14:19:49.594696+0000 mgr.gibba032.whopeo (mgr.24625) 1468 : cluster [DBG] pgmap v821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:50 gibba011 bash[31774]: cluster 2021-03-10T14:19:49.594696+0000 mgr.gibba032.whopeo (mgr.24625) 1468 : cluster [DBG] pgmap v821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:52 gibba032 bash[24004]: cluster 2021-03-10T14:19:51.596365+0000 mgr.gibba032.whopeo (mgr.24625) 1469 : cluster [DBG] pgmap v822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:53.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:52 gibba032 bash[24004]: cephadm 2021-03-10T14:19:51.752744+0000 mgr.gibba032.whopeo (mgr.24625) 1470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:52 gibba011 bash[30791]: cluster 2021-03-10T14:19:51.596365+0000 mgr.gibba032.whopeo (mgr.24625) 1469 : cluster [DBG] pgmap v822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:52 gibba011 bash[30791]: cephadm 2021-03-10T14:19:51.752744+0000 mgr.gibba032.whopeo (mgr.24625) 1470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:52 gibba011 bash[31774]: cluster 2021-03-10T14:19:51.596365+0000 mgr.gibba032.whopeo (mgr.24625) 1469 : cluster [DBG] pgmap v822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:53.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:52 gibba011 bash[31774]: cephadm 2021-03-10T14:19:51.752744+0000 mgr.gibba032.whopeo (mgr.24625) 1470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:19:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:54 gibba032 bash[24004]: cluster 2021-03-10T14:19:53.598015+0000 mgr.gibba032.whopeo (mgr.24625) 1471 : cluster [DBG] pgmap v823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:54 gibba011 bash[30791]: cluster 2021-03-10T14:19:53.598015+0000 mgr.gibba032.whopeo (mgr.24625) 1471 : cluster [DBG] pgmap v823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:54 gibba011 bash[31774]: cluster 2021-03-10T14:19:53.598015+0000 mgr.gibba032.whopeo (mgr.24625) 1471 : cluster [DBG] pgmap v823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: cluster 2021-03-10T14:19:55.598478+0000 mgr.gibba032.whopeo (mgr.24625) 1472 : cluster [DBG] pgmap v824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.114275+0000 mon.a (mon.0) 986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:19:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.114996+0000 mon.a (mon.0) 987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.115726+0000 mon.a (mon.0) 988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.116445+0000 mon.a (mon.0) 989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.173334+0000 mon.a (mon.0) 990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:56 gibba032 bash[24004]: audit 2021-03-10T14:19:56.174034+0000 mon.a (mon.0) 991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:19:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: cluster 2021-03-10T14:19:55.598478+0000 mgr.gibba032.whopeo (mgr.24625) 1472 : cluster [DBG] pgmap v824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.114275+0000 mon.a (mon.0) 986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.114996+0000 mon.a (mon.0) 987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.115726+0000 mon.a (mon.0) 988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.116445+0000 mon.a (mon.0) 989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.173334+0000 mon.a (mon.0) 990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:19:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[30791]: audit 2021-03-10T14:19:56.174034+0000 mon.a (mon.0) 991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:19:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: cluster 2021-03-10T14:19:55.598478+0000 mgr.gibba032.whopeo (mgr.24625) 1472 : cluster [DBG] pgmap v824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.114275+0000 mon.a (mon.0) 986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T14:19:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.114996+0000 mon.a (mon.0) 987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.115726+0000 mon.a (mon.0) 988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:19:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.116445+0000 mon.a (mon.0) 989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:19:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.173334+0000 mon.a (mon.0) 990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:19:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:56 gibba011 bash[31774]: audit 2021-03-10T14:19:56.174034+0000 mon.a (mon.0) 991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:19:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:19:58 gibba032 bash[24004]: cluster 2021-03-10T14:19:57.600181+0000 mgr.gibba032.whopeo (mgr.24625) 1473 : cluster [DBG] pgmap v825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:19:58 gibba011 bash[30791]: cluster 2021-03-10T14:19:57.600181+0000 mgr.gibba032.whopeo (mgr.24625) 1473 : cluster [DBG] pgmap v825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:19:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:19:58 gibba011 bash[31774]: cluster 2021-03-10T14:19:57.600181+0000 mgr.gibba032.whopeo (mgr.24625) 1473 : cluster [DBG] pgmap v825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:00 gibba011 bash[30791]: debug 2021-03-10T14:19:59.996+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:20:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:00 gibba032 bash[24004]: cluster 2021-03-10T14:19:59.601650+0000 mgr.gibba032.whopeo (mgr.24625) 1474 : cluster [DBG] pgmap v826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:01.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:00 gibba032 bash[24004]: cluster 2021-03-10T14:20:00.000223+0000 mon.a (mon.0) 992 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:20:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:00 gibba011 bash[31774]: cluster 2021-03-10T14:19:59.601650+0000 mgr.gibba032.whopeo (mgr.24625) 1474 : cluster [DBG] pgmap v826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:00 gibba011 bash[31774]: cluster 2021-03-10T14:20:00.000223+0000 mon.a (mon.0) 992 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:20:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:00 gibba011 bash[30791]: cluster 2021-03-10T14:19:59.601650+0000 mgr.gibba032.whopeo (mgr.24625) 1474 : cluster [DBG] pgmap v826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:00 gibba011 bash[30791]: cluster 2021-03-10T14:20:00.000223+0000 mon.a (mon.0) 992 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:20:01.187 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:20:01.413 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:20:01.414 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (26m) 2m ago 32m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:20:01.414 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (31m) 90s ago 31m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:20:01.415 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (27m) 90s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:20:01.415 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (27m) 90s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:20:01.415 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (26m) 2m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:20:01.415 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (26m) 2m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:20:01.415 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (25m) 90s ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:20:01.416 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (25m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:20:01.416 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (32m) 2m ago 32m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:20:01.416 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (32m) 90s ago 32m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:20:01.416 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (24m) 2m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:20:01.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (24m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:20:01.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (24m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:20:01.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (23m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:20:01.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (23m) 90s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:20:01.418 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (22m) 90s ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:20:01.418 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (22m) 90s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:20:01.418 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (21m) 90s ago 32m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:20:01.418 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (31m) 90s ago 32m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:20:01.418 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 30m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:20:01.662 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:20:01.663 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:20:01.663 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:20:01.664 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:01.664 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:20:01.664 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:20:01.664 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:01.665 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:20:01.665 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:20:01.665 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:01.665 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:20:01.666 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:20:01.666 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:20:01.666 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:20:01.666 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:20:02.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:01 gibba032 bash[24004]: audit 2021-03-10T14:20:01.662437+0000 mon.a (mon.0) 993 : audit [DBG] from='client.? 172.21.2.111:0/2864688423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:01 gibba011 bash[30791]: audit 2021-03-10T14:20:01.662437+0000 mon.a (mon.0) 993 : audit [DBG] from='client.? 172.21.2.111:0/2864688423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:01 gibba011 bash[31774]: audit 2021-03-10T14:20:01.662437+0000 mon.a (mon.0) 993 : audit [DBG] from='client.? 172.21.2.111:0/2864688423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:02 gibba032 bash[24004]: audit 2021-03-10T14:20:01.175398+0000 mgr.gibba032.whopeo (mgr.24625) 1475 : audit [DBG] from='client.45045 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:02 gibba032 bash[24004]: audit 2021-03-10T14:20:01.398447+0000 mgr.gibba032.whopeo (mgr.24625) 1476 : audit [DBG] from='client.45051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:02 gibba032 bash[24004]: cluster 2021-03-10T14:20:01.603595+0000 mgr.gibba032.whopeo (mgr.24625) 1477 : cluster [DBG] pgmap v827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[30791]: audit 2021-03-10T14:20:01.175398+0000 mgr.gibba032.whopeo (mgr.24625) 1475 : audit [DBG] from='client.45045 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[30791]: audit 2021-03-10T14:20:01.398447+0000 mgr.gibba032.whopeo (mgr.24625) 1476 : audit [DBG] from='client.45051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[30791]: cluster 2021-03-10T14:20:01.603595+0000 mgr.gibba032.whopeo (mgr.24625) 1477 : cluster [DBG] pgmap v827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[31774]: audit 2021-03-10T14:20:01.175398+0000 mgr.gibba032.whopeo (mgr.24625) 1475 : audit [DBG] from='client.45045 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[31774]: audit 2021-03-10T14:20:01.398447+0000 mgr.gibba032.whopeo (mgr.24625) 1476 : audit [DBG] from='client.45051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:03.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:02 gibba011 bash[31774]: cluster 2021-03-10T14:20:01.603595+0000 mgr.gibba032.whopeo (mgr.24625) 1477 : cluster [DBG] pgmap v827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:04 gibba032 bash[24004]: cluster 2021-03-10T14:20:03.605319+0000 mgr.gibba032.whopeo (mgr.24625) 1478 : cluster [DBG] pgmap v828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:04 gibba011 bash[30791]: cluster 2021-03-10T14:20:03.605319+0000 mgr.gibba032.whopeo (mgr.24625) 1478 : cluster [DBG] pgmap v828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:04 gibba011 bash[31774]: cluster 2021-03-10T14:20:03.605319+0000 mgr.gibba032.whopeo (mgr.24625) 1478 : cluster [DBG] pgmap v828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:06 gibba032 bash[24004]: cluster 2021-03-10T14:20:05.606182+0000 mgr.gibba032.whopeo (mgr.24625) 1479 : cluster [DBG] pgmap v829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:06 gibba011 bash[30791]: cluster 2021-03-10T14:20:05.606182+0000 mgr.gibba032.whopeo (mgr.24625) 1479 : cluster [DBG] pgmap v829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:06 gibba011 bash[31774]: cluster 2021-03-10T14:20:05.606182+0000 mgr.gibba032.whopeo (mgr.24625) 1479 : cluster [DBG] pgmap v829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:07 gibba032 bash[24004]: cephadm 2021-03-10T14:20:06.768788+0000 mgr.gibba032.whopeo (mgr.24625) 1480 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:08.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:07 gibba011 bash[30791]: cephadm 2021-03-10T14:20:06.768788+0000 mgr.gibba032.whopeo (mgr.24625) 1480 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:08.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:07 gibba011 bash[31774]: cephadm 2021-03-10T14:20:06.768788+0000 mgr.gibba032.whopeo (mgr.24625) 1480 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:08 gibba032 bash[24004]: cluster 2021-03-10T14:20:07.607709+0000 mgr.gibba032.whopeo (mgr.24625) 1481 : cluster [DBG] pgmap v830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:08 gibba011 bash[30791]: cluster 2021-03-10T14:20:07.607709+0000 mgr.gibba032.whopeo (mgr.24625) 1481 : cluster [DBG] pgmap v830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:08 gibba011 bash[31774]: cluster 2021-03-10T14:20:07.607709+0000 mgr.gibba032.whopeo (mgr.24625) 1481 : cluster [DBG] pgmap v830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:10 gibba032 bash[24004]: cluster 2021-03-10T14:20:09.609184+0000 mgr.gibba032.whopeo (mgr.24625) 1482 : cluster [DBG] pgmap v831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:10 gibba011 bash[30791]: cluster 2021-03-10T14:20:09.609184+0000 mgr.gibba032.whopeo (mgr.24625) 1482 : cluster [DBG] pgmap v831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:10 gibba011 bash[31774]: cluster 2021-03-10T14:20:09.609184+0000 mgr.gibba032.whopeo (mgr.24625) 1482 : cluster [DBG] pgmap v831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:12 gibba032 bash[24004]: cluster 2021-03-10T14:20:11.610936+0000 mgr.gibba032.whopeo (mgr.24625) 1483 : cluster [DBG] pgmap v832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:12 gibba011 bash[30791]: cluster 2021-03-10T14:20:11.610936+0000 mgr.gibba032.whopeo (mgr.24625) 1483 : cluster [DBG] pgmap v832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:12 gibba011 bash[31774]: cluster 2021-03-10T14:20:11.610936+0000 mgr.gibba032.whopeo (mgr.24625) 1483 : cluster [DBG] pgmap v832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:14 gibba032 bash[24004]: cluster 2021-03-10T14:20:13.612443+0000 mgr.gibba032.whopeo (mgr.24625) 1484 : cluster [DBG] pgmap v833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:14 gibba011 bash[30791]: cluster 2021-03-10T14:20:13.612443+0000 mgr.gibba032.whopeo (mgr.24625) 1484 : cluster [DBG] pgmap v833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:14 gibba011 bash[31774]: cluster 2021-03-10T14:20:13.612443+0000 mgr.gibba032.whopeo (mgr.24625) 1484 : cluster [DBG] pgmap v833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:16 gibba032 bash[24004]: cluster 2021-03-10T14:20:15.613098+0000 mgr.gibba032.whopeo (mgr.24625) 1485 : cluster [DBG] pgmap v834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:16 gibba011 bash[30791]: cluster 2021-03-10T14:20:15.613098+0000 mgr.gibba032.whopeo (mgr.24625) 1485 : cluster [DBG] pgmap v834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:16 gibba011 bash[31774]: cluster 2021-03-10T14:20:15.613098+0000 mgr.gibba032.whopeo (mgr.24625) 1485 : cluster [DBG] pgmap v834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:18 gibba032 bash[24004]: cluster 2021-03-10T14:20:17.614481+0000 mgr.gibba032.whopeo (mgr.24625) 1486 : cluster [DBG] pgmap v835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:18 gibba011 bash[30791]: cluster 2021-03-10T14:20:17.614481+0000 mgr.gibba032.whopeo (mgr.24625) 1486 : cluster [DBG] pgmap v835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:18 gibba011 bash[31774]: cluster 2021-03-10T14:20:17.614481+0000 mgr.gibba032.whopeo (mgr.24625) 1486 : cluster [DBG] pgmap v835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:20 gibba032 bash[24004]: cluster 2021-03-10T14:20:19.615941+0000 mgr.gibba032.whopeo (mgr.24625) 1487 : cluster [DBG] pgmap v836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:20 gibba011 bash[30791]: cluster 2021-03-10T14:20:19.615941+0000 mgr.gibba032.whopeo (mgr.24625) 1487 : cluster [DBG] pgmap v836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:20 gibba011 bash[31774]: cluster 2021-03-10T14:20:19.615941+0000 mgr.gibba032.whopeo (mgr.24625) 1487 : cluster [DBG] pgmap v836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:22 gibba032 bash[24004]: cluster 2021-03-10T14:20:21.617737+0000 mgr.gibba032.whopeo (mgr.24625) 1488 : cluster [DBG] pgmap v837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:22 gibba032 bash[24004]: cephadm 2021-03-10T14:20:21.784755+0000 mgr.gibba032.whopeo (mgr.24625) 1489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:23.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:22 gibba011 bash[30791]: cluster 2021-03-10T14:20:21.617737+0000 mgr.gibba032.whopeo (mgr.24625) 1488 : cluster [DBG] pgmap v837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:22 gibba011 bash[30791]: cephadm 2021-03-10T14:20:21.784755+0000 mgr.gibba032.whopeo (mgr.24625) 1489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:22 gibba011 bash[31774]: cluster 2021-03-10T14:20:21.617737+0000 mgr.gibba032.whopeo (mgr.24625) 1488 : cluster [DBG] pgmap v837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:22 gibba011 bash[31774]: cephadm 2021-03-10T14:20:21.784755+0000 mgr.gibba032.whopeo (mgr.24625) 1489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:24 gibba032 bash[24004]: cluster 2021-03-10T14:20:23.619173+0000 mgr.gibba032.whopeo (mgr.24625) 1490 : cluster [DBG] pgmap v838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:24 gibba011 bash[30791]: cluster 2021-03-10T14:20:23.619173+0000 mgr.gibba032.whopeo (mgr.24625) 1490 : cluster [DBG] pgmap v838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:24 gibba011 bash[31774]: cluster 2021-03-10T14:20:23.619173+0000 mgr.gibba032.whopeo (mgr.24625) 1490 : cluster [DBG] pgmap v838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:26 gibba032 bash[24004]: cluster 2021-03-10T14:20:25.620458+0000 mgr.gibba032.whopeo (mgr.24625) 1491 : cluster [DBG] pgmap v839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:26 gibba011 bash[30791]: cluster 2021-03-10T14:20:25.620458+0000 mgr.gibba032.whopeo (mgr.24625) 1491 : cluster [DBG] pgmap v839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:26 gibba011 bash[31774]: cluster 2021-03-10T14:20:25.620458+0000 mgr.gibba032.whopeo (mgr.24625) 1491 : cluster [DBG] pgmap v839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:28 gibba032 bash[24004]: cluster 2021-03-10T14:20:27.622215+0000 mgr.gibba032.whopeo (mgr.24625) 1492 : cluster [DBG] pgmap v840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:28 gibba011 bash[30791]: cluster 2021-03-10T14:20:27.622215+0000 mgr.gibba032.whopeo (mgr.24625) 1492 : cluster [DBG] pgmap v840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:28 gibba011 bash[31774]: cluster 2021-03-10T14:20:27.622215+0000 mgr.gibba032.whopeo (mgr.24625) 1492 : cluster [DBG] pgmap v840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:30 gibba032 bash[24004]: cluster 2021-03-10T14:20:29.623570+0000 mgr.gibba032.whopeo (mgr.24625) 1493 : cluster [DBG] pgmap v841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:30 gibba011 bash[30791]: cluster 2021-03-10T14:20:29.623570+0000 mgr.gibba032.whopeo (mgr.24625) 1493 : cluster [DBG] pgmap v841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:30 gibba011 bash[31774]: cluster 2021-03-10T14:20:29.623570+0000 mgr.gibba032.whopeo (mgr.24625) 1493 : cluster [DBG] pgmap v841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:31.908 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:20:32.134 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:20:32.135 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (27m) 3m ago 32m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:20:32.135 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (32m) 2m ago 32m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:20:32.135 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (27m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:20:32.135 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (27m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:20:32.135 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (26m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:20:32.136 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (26m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:20:32.136 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (26m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:20:32.136 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (26m) 3m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:20:32.136 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (32m) 3m ago 32m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:20:32.136 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (32m) 2m ago 32m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:20:32.137 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (25m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:20:32.137 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (25m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:20:32.137 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (24m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:20:32.138 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (24m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:20:32.138 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (23m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:20:32.138 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (23m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:20:32.138 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (22m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:20:32.138 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (22m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:20:32.139 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (32m) 2m ago 32m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:20:32.139 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 30m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:20:32.388 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:20:32.389 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:20:32.389 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:20:32.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:32.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:20:32.390 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:20:32.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:32.390 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:20:32.390 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:20:32.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:20:32.391 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:20:32.391 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:20:32.391 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:20:32.391 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:20:32.392 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:20:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:32 gibba032 bash[24004]: cluster 2021-03-10T14:20:31.625274+0000 mgr.gibba032.whopeo (mgr.24625) 1494 : cluster [DBG] pgmap v842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:32 gibba032 bash[24004]: audit 2021-03-10T14:20:31.896113+0000 mgr.gibba032.whopeo (mgr.24625) 1495 : audit [DBG] from='client.55048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:32 gibba032 bash[24004]: audit 2021-03-10T14:20:32.388575+0000 mon.a (mon.0) 994 : audit [DBG] from='client.? 172.21.2.111:0/944284979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[30791]: cluster 2021-03-10T14:20:31.625274+0000 mgr.gibba032.whopeo (mgr.24625) 1494 : cluster [DBG] pgmap v842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[30791]: audit 2021-03-10T14:20:31.896113+0000 mgr.gibba032.whopeo (mgr.24625) 1495 : audit [DBG] from='client.55048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[30791]: audit 2021-03-10T14:20:32.388575+0000 mon.a (mon.0) 994 : audit [DBG] from='client.? 172.21.2.111:0/944284979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[31774]: cluster 2021-03-10T14:20:31.625274+0000 mgr.gibba032.whopeo (mgr.24625) 1494 : cluster [DBG] pgmap v842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[31774]: audit 2021-03-10T14:20:31.896113+0000 mgr.gibba032.whopeo (mgr.24625) 1495 : audit [DBG] from='client.55048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:32 gibba011 bash[31774]: audit 2021-03-10T14:20:32.388575+0000 mon.a (mon.0) 994 : audit [DBG] from='client.? 172.21.2.111:0/944284979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:34.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:33 gibba032 bash[24004]: audit 2021-03-10T14:20:32.120159+0000 mgr.gibba032.whopeo (mgr.24625) 1496 : audit [DBG] from='client.45069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:34.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:33 gibba011 bash[30791]: audit 2021-03-10T14:20:32.120159+0000 mgr.gibba032.whopeo (mgr.24625) 1496 : audit [DBG] from='client.45069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:34.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:33 gibba011 bash[31774]: audit 2021-03-10T14:20:32.120159+0000 mgr.gibba032.whopeo (mgr.24625) 1496 : audit [DBG] from='client.45069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:20:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:34 gibba032 bash[24004]: cluster 2021-03-10T14:20:33.626733+0000 mgr.gibba032.whopeo (mgr.24625) 1497 : cluster [DBG] pgmap v843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:34 gibba011 bash[30791]: cluster 2021-03-10T14:20:33.626733+0000 mgr.gibba032.whopeo (mgr.24625) 1497 : cluster [DBG] pgmap v843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:34 gibba011 bash[31774]: cluster 2021-03-10T14:20:33.626733+0000 mgr.gibba032.whopeo (mgr.24625) 1497 : cluster [DBG] pgmap v843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:36 gibba032 bash[24004]: cluster 2021-03-10T14:20:35.627510+0000 mgr.gibba032.whopeo (mgr.24625) 1498 : cluster [DBG] pgmap v844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:37.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:36 gibba011 bash[30791]: cluster 2021-03-10T14:20:35.627510+0000 mgr.gibba032.whopeo (mgr.24625) 1498 : cluster [DBG] pgmap v844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:36 gibba011 bash[31774]: cluster 2021-03-10T14:20:35.627510+0000 mgr.gibba032.whopeo (mgr.24625) 1498 : cluster [DBG] pgmap v844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:38.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: cephadm 2021-03-10T14:20:36.801589+0000 mgr.gibba032.whopeo (mgr.24625) 1499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:20:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: cephadm 2021-03-10T14:20:36.803180+0000 mgr.gibba032.whopeo (mgr.24625) 1500 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:20:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.031474+0000 mon.a (mon.0) 995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:20:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.034642+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:20:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.036079+0000 mon.a (mon.0) 997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.037723+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.039123+0000 mon.a (mon.0) 999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.041980+0000 mon.a (mon.0) 1000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:37 gibba032 bash[24004]: audit 2021-03-10T14:20:37.043660+0000 mon.a (mon.0) 1001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: cephadm 2021-03-10T14:20:36.801589+0000 mgr.gibba032.whopeo (mgr.24625) 1499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:20:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: cephadm 2021-03-10T14:20:36.803180+0000 mgr.gibba032.whopeo (mgr.24625) 1500 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:20:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.031474+0000 mon.a (mon.0) 995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:20:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.034642+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:20:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.036079+0000 mon.a (mon.0) 997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.037723+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.039123+0000 mon.a (mon.0) 999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.041980+0000 mon.a (mon.0) 1000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[30791]: audit 2021-03-10T14:20:37.043660+0000 mon.a (mon.0) 1001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: cephadm 2021-03-10T14:20:36.801589+0000 mgr.gibba032.whopeo (mgr.24625) 1499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:20:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: cephadm 2021-03-10T14:20:36.803180+0000 mgr.gibba032.whopeo (mgr.24625) 1500 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:20:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.031474+0000 mon.a (mon.0) 995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:20:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.034642+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:20:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.036079+0000 mon.a (mon.0) 997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.037723+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.039123+0000 mon.a (mon.0) 999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.041980+0000 mon.a (mon.0) 1000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:38.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:37 gibba011 bash[31774]: audit 2021-03-10T14:20:37.043660+0000 mon.a (mon.0) 1001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:20:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.031374+0000 mgr.gibba032.whopeo (mgr.24625) 1501 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:20:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.033055+0000 mgr.gibba032.whopeo (mgr.24625) 1502 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:20:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.034639+0000 mgr.gibba032.whopeo (mgr.24625) 1503 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:20:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.034680+0000 mgr.gibba032.whopeo (mgr.24625) 1504 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:20:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.036083+0000 mgr.gibba032.whopeo (mgr.24625) 1505 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:20:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.036124+0000 mgr.gibba032.whopeo (mgr.24625) 1506 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:20:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.037496+0000 mgr.gibba032.whopeo (mgr.24625) 1507 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:20:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.038803+0000 mgr.gibba032.whopeo (mgr.24625) 1508 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:20:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.038842+0000 mgr.gibba032.whopeo (mgr.24625) 1509 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:20:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.040241+0000 mgr.gibba032.whopeo (mgr.24625) 1510 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:20:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.040653+0000 mgr.gibba032.whopeo (mgr.24625) 1511 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:20:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.042155+0000 mgr.gibba032.whopeo (mgr.24625) 1512 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:20:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.042201+0000 mgr.gibba032.whopeo (mgr.24625) 1513 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:20:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cephadm 2021-03-10T14:20:37.042319+0000 mgr.gibba032.whopeo (mgr.24625) 1514 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:38 gibba032 bash[24004]: cluster 2021-03-10T14:20:37.628861+0000 mgr.gibba032.whopeo (mgr.24625) 1515 : cluster [DBG] pgmap v845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.031374+0000 mgr.gibba032.whopeo (mgr.24625) 1501 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:20:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.033055+0000 mgr.gibba032.whopeo (mgr.24625) 1502 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:20:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.034639+0000 mgr.gibba032.whopeo (mgr.24625) 1503 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:20:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.034680+0000 mgr.gibba032.whopeo (mgr.24625) 1504 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:20:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.036083+0000 mgr.gibba032.whopeo (mgr.24625) 1505 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:20:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.036124+0000 mgr.gibba032.whopeo (mgr.24625) 1506 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:20:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.037496+0000 mgr.gibba032.whopeo (mgr.24625) 1507 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:20:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.038803+0000 mgr.gibba032.whopeo (mgr.24625) 1508 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:20:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.038842+0000 mgr.gibba032.whopeo (mgr.24625) 1509 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:20:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.040241+0000 mgr.gibba032.whopeo (mgr.24625) 1510 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:20:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.040653+0000 mgr.gibba032.whopeo (mgr.24625) 1511 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:20:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.042155+0000 mgr.gibba032.whopeo (mgr.24625) 1512 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:20:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.042201+0000 mgr.gibba032.whopeo (mgr.24625) 1513 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:20:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cephadm 2021-03-10T14:20:37.042319+0000 mgr.gibba032.whopeo (mgr.24625) 1514 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:39.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[30791]: cluster 2021-03-10T14:20:37.628861+0000 mgr.gibba032.whopeo (mgr.24625) 1515 : cluster [DBG] pgmap v845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.031374+0000 mgr.gibba032.whopeo (mgr.24625) 1501 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:20:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.033055+0000 mgr.gibba032.whopeo (mgr.24625) 1502 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:20:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.034639+0000 mgr.gibba032.whopeo (mgr.24625) 1503 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:20:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.034680+0000 mgr.gibba032.whopeo (mgr.24625) 1504 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:20:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.036083+0000 mgr.gibba032.whopeo (mgr.24625) 1505 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:20:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.036124+0000 mgr.gibba032.whopeo (mgr.24625) 1506 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:20:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.037496+0000 mgr.gibba032.whopeo (mgr.24625) 1507 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:20:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.038803+0000 mgr.gibba032.whopeo (mgr.24625) 1508 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:20:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.038842+0000 mgr.gibba032.whopeo (mgr.24625) 1509 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:20:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.040241+0000 mgr.gibba032.whopeo (mgr.24625) 1510 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:20:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.040653+0000 mgr.gibba032.whopeo (mgr.24625) 1511 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:20:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.042155+0000 mgr.gibba032.whopeo (mgr.24625) 1512 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:20:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.042201+0000 mgr.gibba032.whopeo (mgr.24625) 1513 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:20:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cephadm 2021-03-10T14:20:37.042319+0000 mgr.gibba032.whopeo (mgr.24625) 1514 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:39.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:38 gibba011 bash[31774]: cluster 2021-03-10T14:20:37.628861+0000 mgr.gibba032.whopeo (mgr.24625) 1515 : cluster [DBG] pgmap v845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:40 gibba032 bash[24004]: cluster 2021-03-10T14:20:39.630372+0000 mgr.gibba032.whopeo (mgr.24625) 1516 : cluster [DBG] pgmap v846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:40 gibba011 bash[30791]: cluster 2021-03-10T14:20:39.630372+0000 mgr.gibba032.whopeo (mgr.24625) 1516 : cluster [DBG] pgmap v846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:40 gibba011 bash[31774]: cluster 2021-03-10T14:20:39.630372+0000 mgr.gibba032.whopeo (mgr.24625) 1516 : cluster [DBG] pgmap v846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:42 gibba032 bash[24004]: cluster 2021-03-10T14:20:41.632183+0000 mgr.gibba032.whopeo (mgr.24625) 1517 : cluster [DBG] pgmap v847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:42 gibba011 bash[30791]: cluster 2021-03-10T14:20:41.632183+0000 mgr.gibba032.whopeo (mgr.24625) 1517 : cluster [DBG] pgmap v847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:42 gibba011 bash[31774]: cluster 2021-03-10T14:20:41.632183+0000 mgr.gibba032.whopeo (mgr.24625) 1517 : cluster [DBG] pgmap v847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:44 gibba032 bash[24004]: cluster 2021-03-10T14:20:43.633846+0000 mgr.gibba032.whopeo (mgr.24625) 1518 : cluster [DBG] pgmap v848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:44 gibba011 bash[30791]: cluster 2021-03-10T14:20:43.633846+0000 mgr.gibba032.whopeo (mgr.24625) 1518 : cluster [DBG] pgmap v848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:44 gibba011 bash[31774]: cluster 2021-03-10T14:20:43.633846+0000 mgr.gibba032.whopeo (mgr.24625) 1518 : cluster [DBG] pgmap v848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:46 gibba032 bash[24004]: cluster 2021-03-10T14:20:45.634579+0000 mgr.gibba032.whopeo (mgr.24625) 1519 : cluster [DBG] pgmap v849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:46 gibba011 bash[30791]: cluster 2021-03-10T14:20:45.634579+0000 mgr.gibba032.whopeo (mgr.24625) 1519 : cluster [DBG] pgmap v849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:46 gibba011 bash[31774]: cluster 2021-03-10T14:20:45.634579+0000 mgr.gibba032.whopeo (mgr.24625) 1519 : cluster [DBG] pgmap v849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:48 gibba032 bash[24004]: cluster 2021-03-10T14:20:47.636148+0000 mgr.gibba032.whopeo (mgr.24625) 1520 : cluster [DBG] pgmap v850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:48 gibba011 bash[30791]: cluster 2021-03-10T14:20:47.636148+0000 mgr.gibba032.whopeo (mgr.24625) 1520 : cluster [DBG] pgmap v850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:48 gibba011 bash[31774]: cluster 2021-03-10T14:20:47.636148+0000 mgr.gibba032.whopeo (mgr.24625) 1520 : cluster [DBG] pgmap v850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:50 gibba032 bash[24004]: cluster 2021-03-10T14:20:49.637697+0000 mgr.gibba032.whopeo (mgr.24625) 1521 : cluster [DBG] pgmap v851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:50 gibba011 bash[30791]: cluster 2021-03-10T14:20:49.637697+0000 mgr.gibba032.whopeo (mgr.24625) 1521 : cluster [DBG] pgmap v851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:50 gibba011 bash[31774]: cluster 2021-03-10T14:20:49.637697+0000 mgr.gibba032.whopeo (mgr.24625) 1521 : cluster [DBG] pgmap v851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:52 gibba032 bash[24004]: cluster 2021-03-10T14:20:51.639387+0000 mgr.gibba032.whopeo (mgr.24625) 1522 : cluster [DBG] pgmap v852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:53.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:52 gibba011 bash[30791]: cluster 2021-03-10T14:20:51.639387+0000 mgr.gibba032.whopeo (mgr.24625) 1522 : cluster [DBG] pgmap v852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:52 gibba011 bash[31774]: cluster 2021-03-10T14:20:51.639387+0000 mgr.gibba032.whopeo (mgr.24625) 1522 : cluster [DBG] pgmap v852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:54.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:53 gibba032 bash[24004]: cephadm 2021-03-10T14:20:52.058342+0000 mgr.gibba032.whopeo (mgr.24625) 1523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:53 gibba011 bash[30791]: cephadm 2021-03-10T14:20:52.058342+0000 mgr.gibba032.whopeo (mgr.24625) 1523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:54.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:53 gibba011 bash[31774]: cephadm 2021-03-10T14:20:52.058342+0000 mgr.gibba032.whopeo (mgr.24625) 1523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:20:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:54 gibba032 bash[24004]: cluster 2021-03-10T14:20:53.640765+0000 mgr.gibba032.whopeo (mgr.24625) 1524 : cluster [DBG] pgmap v853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:54 gibba011 bash[30791]: cluster 2021-03-10T14:20:53.640765+0000 mgr.gibba032.whopeo (mgr.24625) 1524 : cluster [DBG] pgmap v853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:54 gibba011 bash[31774]: cluster 2021-03-10T14:20:53.640765+0000 mgr.gibba032.whopeo (mgr.24625) 1524 : cluster [DBG] pgmap v853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: cluster 2021-03-10T14:20:55.641596+0000 mgr.gibba032.whopeo (mgr.24625) 1525 : cluster [DBG] pgmap v854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.124410+0000 mon.a (mon.0) 1002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:20:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.125188+0000 mon.a (mon.0) 1003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.125881+0000 mon.a (mon.0) 1004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:20:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.126560+0000 mon.a (mon.0) 1005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.180088+0000 mon.a (mon.0) 1006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:20:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:56 gibba032 bash[24004]: audit 2021-03-10T14:20:56.180732+0000 mon.a (mon.0) 1007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:20:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: cluster 2021-03-10T14:20:55.641596+0000 mgr.gibba032.whopeo (mgr.24625) 1525 : cluster [DBG] pgmap v854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.124410+0000 mon.a (mon.0) 1002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:20:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.125188+0000 mon.a (mon.0) 1003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.125881+0000 mon.a (mon.0) 1004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:20:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.126560+0000 mon.a (mon.0) 1005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.180088+0000 mon.a (mon.0) 1006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:20:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[30791]: audit 2021-03-10T14:20:56.180732+0000 mon.a (mon.0) 1007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:20:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: cluster 2021-03-10T14:20:55.641596+0000 mgr.gibba032.whopeo (mgr.24625) 1525 : cluster [DBG] pgmap v854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.124410+0000 mon.a (mon.0) 1002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:20:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.125188+0000 mon.a (mon.0) 1003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.125881+0000 mon.a (mon.0) 1004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:20:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.126560+0000 mon.a (mon.0) 1005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:20:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.180088+0000 mon.a (mon.0) 1006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:20:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:56 gibba011 bash[31774]: audit 2021-03-10T14:20:56.180732+0000 mon.a (mon.0) 1007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:20:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:20:58 gibba032 bash[24004]: cluster 2021-03-10T14:20:57.643292+0000 mgr.gibba032.whopeo (mgr.24625) 1526 : cluster [DBG] pgmap v855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:20:58 gibba011 bash[30791]: cluster 2021-03-10T14:20:57.643292+0000 mgr.gibba032.whopeo (mgr.24625) 1526 : cluster [DBG] pgmap v855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:20:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:20:58 gibba011 bash[31774]: cluster 2021-03-10T14:20:57.643292+0000 mgr.gibba032.whopeo (mgr.24625) 1526 : cluster [DBG] pgmap v855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:00 gibba032 bash[24004]: cluster 2021-03-10T14:20:59.644723+0000 mgr.gibba032.whopeo (mgr.24625) 1527 : cluster [DBG] pgmap v856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:00 gibba011 bash[30791]: cluster 2021-03-10T14:20:59.644723+0000 mgr.gibba032.whopeo (mgr.24625) 1527 : cluster [DBG] pgmap v856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:00 gibba011 bash[31774]: cluster 2021-03-10T14:20:59.644723+0000 mgr.gibba032.whopeo (mgr.24625) 1527 : cluster [DBG] pgmap v856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:02.630 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:21:02.859 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:21:02.860 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (27m) 3m ago 33m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:21:02.860 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (32m) 2m ago 32m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:21:02.860 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (28m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:21:02.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (28m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:21:02.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (27m) 3m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:21:02.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (27m) 3m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:21:02.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (26m) 2m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:21:02.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (26m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:21:02.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (33m) 3m ago 33m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:21:02.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (33m) 2m ago 33m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:21:02.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (25m) 3m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:21:02.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (25m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:21:02.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (25m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:21:02.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (24m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:21:02.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (24m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:21:02.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (23m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:21:02.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (23m) 2m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:21:02.865 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (23m) 2m ago 33m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:21:02.865 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (32m) 2m ago 33m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:21:02.866 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 31m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:21:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:02 gibba032 bash[24004]: cluster 2021-03-10T14:21:01.646467+0000 mgr.gibba032.whopeo (mgr.24625) 1528 : cluster [DBG] pgmap v857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:02 gibba011 bash[30791]: cluster 2021-03-10T14:21:01.646467+0000 mgr.gibba032.whopeo (mgr.24625) 1528 : cluster [DBG] pgmap v857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:02 gibba011 bash[31774]: cluster 2021-03-10T14:21:01.646467+0000 mgr.gibba032.whopeo (mgr.24625) 1528 : cluster [DBG] pgmap v857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:03.112 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:21:03.112 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:21:03.112 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:21:03.113 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:03.113 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:21:03.113 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:21:03.113 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:03.114 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:21:03.114 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:21:03.115 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:03.121 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:21:03.121 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:21:03.122 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:21:03.122 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:21:03.122 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:21:04.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:03 gibba032 bash[24004]: audit 2021-03-10T14:21:02.618386+0000 mgr.gibba032.whopeo (mgr.24625) 1529 : audit [DBG] from='client.45081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:03 gibba032 bash[24004]: audit 2021-03-10T14:21:02.844866+0000 mgr.gibba032.whopeo (mgr.24625) 1530 : audit [DBG] from='client.45087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:03 gibba032 bash[24004]: audit 2021-03-10T14:21:03.112114+0000 mon.c (mon.1) 30 : audit [DBG] from='client.? 172.21.2.111:0/2956202903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:04.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[30791]: audit 2021-03-10T14:21:02.618386+0000 mgr.gibba032.whopeo (mgr.24625) 1529 : audit [DBG] from='client.45081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[30791]: audit 2021-03-10T14:21:02.844866+0000 mgr.gibba032.whopeo (mgr.24625) 1530 : audit [DBG] from='client.45087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[30791]: audit 2021-03-10T14:21:03.112114+0000 mon.c (mon.1) 30 : audit [DBG] from='client.? 172.21.2.111:0/2956202903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:04.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[31774]: audit 2021-03-10T14:21:02.618386+0000 mgr.gibba032.whopeo (mgr.24625) 1529 : audit [DBG] from='client.45081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[31774]: audit 2021-03-10T14:21:02.844866+0000 mgr.gibba032.whopeo (mgr.24625) 1530 : audit [DBG] from='client.45087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:04.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:03 gibba011 bash[31774]: audit 2021-03-10T14:21:03.112114+0000 mon.c (mon.1) 30 : audit [DBG] from='client.? 172.21.2.111:0/2956202903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:04 gibba032 bash[24004]: cluster 2021-03-10T14:21:03.648149+0000 mgr.gibba032.whopeo (mgr.24625) 1531 : cluster [DBG] pgmap v858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:04 gibba011 bash[30791]: cluster 2021-03-10T14:21:03.648149+0000 mgr.gibba032.whopeo (mgr.24625) 1531 : cluster [DBG] pgmap v858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:04 gibba011 bash[31774]: cluster 2021-03-10T14:21:03.648149+0000 mgr.gibba032.whopeo (mgr.24625) 1531 : cluster [DBG] pgmap v858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:06 gibba032 bash[24004]: cluster 2021-03-10T14:21:05.648796+0000 mgr.gibba032.whopeo (mgr.24625) 1532 : cluster [DBG] pgmap v859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:06 gibba011 bash[30791]: cluster 2021-03-10T14:21:05.648796+0000 mgr.gibba032.whopeo (mgr.24625) 1532 : cluster [DBG] pgmap v859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:06 gibba011 bash[31774]: cluster 2021-03-10T14:21:05.648796+0000 mgr.gibba032.whopeo (mgr.24625) 1532 : cluster [DBG] pgmap v859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:08 gibba032 bash[24004]: cephadm 2021-03-10T14:21:07.074395+0000 mgr.gibba032.whopeo (mgr.24625) 1533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:08 gibba032 bash[24004]: cluster 2021-03-10T14:21:07.650412+0000 mgr.gibba032.whopeo (mgr.24625) 1534 : cluster [DBG] pgmap v860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:08 gibba011 bash[30791]: cephadm 2021-03-10T14:21:07.074395+0000 mgr.gibba032.whopeo (mgr.24625) 1533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:08 gibba011 bash[30791]: cluster 2021-03-10T14:21:07.650412+0000 mgr.gibba032.whopeo (mgr.24625) 1534 : cluster [DBG] pgmap v860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:08 gibba011 bash[31774]: cephadm 2021-03-10T14:21:07.074395+0000 mgr.gibba032.whopeo (mgr.24625) 1533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:08 gibba011 bash[31774]: cluster 2021-03-10T14:21:07.650412+0000 mgr.gibba032.whopeo (mgr.24625) 1534 : cluster [DBG] pgmap v860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:10 gibba032 bash[24004]: cluster 2021-03-10T14:21:09.651917+0000 mgr.gibba032.whopeo (mgr.24625) 1535 : cluster [DBG] pgmap v861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:10 gibba011 bash[30791]: cluster 2021-03-10T14:21:09.651917+0000 mgr.gibba032.whopeo (mgr.24625) 1535 : cluster [DBG] pgmap v861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:10 gibba011 bash[31774]: cluster 2021-03-10T14:21:09.651917+0000 mgr.gibba032.whopeo (mgr.24625) 1535 : cluster [DBG] pgmap v861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:12 gibba032 bash[24004]: cluster 2021-03-10T14:21:11.653716+0000 mgr.gibba032.whopeo (mgr.24625) 1536 : cluster [DBG] pgmap v862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:12 gibba011 bash[30791]: cluster 2021-03-10T14:21:11.653716+0000 mgr.gibba032.whopeo (mgr.24625) 1536 : cluster [DBG] pgmap v862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:12 gibba011 bash[31774]: cluster 2021-03-10T14:21:11.653716+0000 mgr.gibba032.whopeo (mgr.24625) 1536 : cluster [DBG] pgmap v862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:14 gibba032 bash[24004]: cluster 2021-03-10T14:21:13.655062+0000 mgr.gibba032.whopeo (mgr.24625) 1537 : cluster [DBG] pgmap v863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:14 gibba011 bash[30791]: cluster 2021-03-10T14:21:13.655062+0000 mgr.gibba032.whopeo (mgr.24625) 1537 : cluster [DBG] pgmap v863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:14 gibba011 bash[31774]: cluster 2021-03-10T14:21:13.655062+0000 mgr.gibba032.whopeo (mgr.24625) 1537 : cluster [DBG] pgmap v863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:16 gibba032 bash[24004]: cluster 2021-03-10T14:21:15.655562+0000 mgr.gibba032.whopeo (mgr.24625) 1538 : cluster [DBG] pgmap v864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:17.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:16 gibba011 bash[30791]: cluster 2021-03-10T14:21:15.655562+0000 mgr.gibba032.whopeo (mgr.24625) 1538 : cluster [DBG] pgmap v864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:16 gibba011 bash[31774]: cluster 2021-03-10T14:21:15.655562+0000 mgr.gibba032.whopeo (mgr.24625) 1538 : cluster [DBG] pgmap v864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:18 gibba032 bash[24004]: cluster 2021-03-10T14:21:17.656973+0000 mgr.gibba032.whopeo (mgr.24625) 1539 : cluster [DBG] pgmap v865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:18 gibba011 bash[30791]: cluster 2021-03-10T14:21:17.656973+0000 mgr.gibba032.whopeo (mgr.24625) 1539 : cluster [DBG] pgmap v865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:18 gibba011 bash[31774]: cluster 2021-03-10T14:21:17.656973+0000 mgr.gibba032.whopeo (mgr.24625) 1539 : cluster [DBG] pgmap v865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:20 gibba032 bash[24004]: cluster 2021-03-10T14:21:19.658462+0000 mgr.gibba032.whopeo (mgr.24625) 1540 : cluster [DBG] pgmap v866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:20 gibba011 bash[30791]: cluster 2021-03-10T14:21:19.658462+0000 mgr.gibba032.whopeo (mgr.24625) 1540 : cluster [DBG] pgmap v866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:20 gibba011 bash[31774]: cluster 2021-03-10T14:21:19.658462+0000 mgr.gibba032.whopeo (mgr.24625) 1540 : cluster [DBG] pgmap v866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:22 gibba032 bash[24004]: cluster 2021-03-10T14:21:21.660129+0000 mgr.gibba032.whopeo (mgr.24625) 1541 : cluster [DBG] pgmap v867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:22 gibba011 bash[30791]: cluster 2021-03-10T14:21:21.660129+0000 mgr.gibba032.whopeo (mgr.24625) 1541 : cluster [DBG] pgmap v867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:22 gibba011 bash[31774]: cluster 2021-03-10T14:21:21.660129+0000 mgr.gibba032.whopeo (mgr.24625) 1541 : cluster [DBG] pgmap v867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:24.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:23 gibba032 bash[24004]: cephadm 2021-03-10T14:21:22.090415+0000 mgr.gibba032.whopeo (mgr.24625) 1542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:24.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:23 gibba011 bash[30791]: cephadm 2021-03-10T14:21:22.090415+0000 mgr.gibba032.whopeo (mgr.24625) 1542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:24.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:23 gibba011 bash[31774]: cephadm 2021-03-10T14:21:22.090415+0000 mgr.gibba032.whopeo (mgr.24625) 1542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:24 gibba011 bash[30791]: cluster 2021-03-10T14:21:23.661616+0000 mgr.gibba032.whopeo (mgr.24625) 1543 : cluster [DBG] pgmap v868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:24 gibba011 bash[31774]: cluster 2021-03-10T14:21:23.661616+0000 mgr.gibba032.whopeo (mgr.24625) 1543 : cluster [DBG] pgmap v868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:24 gibba032 bash[24004]: cluster 2021-03-10T14:21:23.661616+0000 mgr.gibba032.whopeo (mgr.24625) 1543 : cluster [DBG] pgmap v868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:26 gibba032 bash[24004]: cluster 2021-03-10T14:21:25.662129+0000 mgr.gibba032.whopeo (mgr.24625) 1544 : cluster [DBG] pgmap v869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:26 gibba011 bash[30791]: cluster 2021-03-10T14:21:25.662129+0000 mgr.gibba032.whopeo (mgr.24625) 1544 : cluster [DBG] pgmap v869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:26 gibba011 bash[31774]: cluster 2021-03-10T14:21:25.662129+0000 mgr.gibba032.whopeo (mgr.24625) 1544 : cluster [DBG] pgmap v869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:28 gibba032 bash[24004]: cluster 2021-03-10T14:21:27.663816+0000 mgr.gibba032.whopeo (mgr.24625) 1545 : cluster [DBG] pgmap v870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:28 gibba011 bash[30791]: cluster 2021-03-10T14:21:27.663816+0000 mgr.gibba032.whopeo (mgr.24625) 1545 : cluster [DBG] pgmap v870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:28 gibba011 bash[31774]: cluster 2021-03-10T14:21:27.663816+0000 mgr.gibba032.whopeo (mgr.24625) 1545 : cluster [DBG] pgmap v870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:30 gibba032 bash[24004]: cluster 2021-03-10T14:21:29.665205+0000 mgr.gibba032.whopeo (mgr.24625) 1546 : cluster [DBG] pgmap v871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:30 gibba011 bash[30791]: cluster 2021-03-10T14:21:29.665205+0000 mgr.gibba032.whopeo (mgr.24625) 1546 : cluster [DBG] pgmap v871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:30 gibba011 bash[31774]: cluster 2021-03-10T14:21:29.665205+0000 mgr.gibba032.whopeo (mgr.24625) 1546 : cluster [DBG] pgmap v871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:32 gibba032 bash[24004]: cluster 2021-03-10T14:21:31.666872+0000 mgr.gibba032.whopeo (mgr.24625) 1547 : cluster [DBG] pgmap v872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:32 gibba011 bash[30791]: cluster 2021-03-10T14:21:31.666872+0000 mgr.gibba032.whopeo (mgr.24625) 1547 : cluster [DBG] pgmap v872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:32 gibba011 bash[31774]: cluster 2021-03-10T14:21:31.666872+0000 mgr.gibba032.whopeo (mgr.24625) 1547 : cluster [DBG] pgmap v872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:33.358 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:21:33.583 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:21:33.583 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (28m) 4m ago 33m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:21:33.583 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (33m) 3m ago 33m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:21:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (28m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:21:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (28m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:21:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (27m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:21:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (27m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:21:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (27m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:21:33.585 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (27m) 4m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:21:33.585 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (33m) 4m ago 33m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:21:33.585 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (33m) 3m ago 33m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:21:33.585 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (26m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:21:33.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (26m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:21:33.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (25m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:21:33.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (25m) 4m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:21:33.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (24m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:21:33.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (24m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:21:33.587 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (23m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:21:33.587 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (23m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:21:33.587 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (33m) 3m ago 33m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:21:33.588 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 31m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:21:33.831 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:21:33.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:21:33.832 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:21:33.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:33.832 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:21:33.832 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:21:33.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:33.833 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:21:33.833 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:21:33.833 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:21:33.833 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:21:33.834 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:21:33.834 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:21:33.834 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:21:33.834 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:21:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:34 gibba032 bash[24004]: audit 2021-03-10T14:21:33.346468+0000 mgr.gibba032.whopeo (mgr.24625) 1548 : audit [DBG] from='client.55084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:34 gibba032 bash[24004]: audit 2021-03-10T14:21:33.568319+0000 mgr.gibba032.whopeo (mgr.24625) 1549 : audit [DBG] from='client.45105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:34 gibba032 bash[24004]: cluster 2021-03-10T14:21:33.668370+0000 mgr.gibba032.whopeo (mgr.24625) 1550 : cluster [DBG] pgmap v873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:35.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:34 gibba032 bash[24004]: audit 2021-03-10T14:21:33.831102+0000 mon.c (mon.1) 31 : audit [DBG] from='client.? 172.21.2.111:0/3075484051' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[30791]: audit 2021-03-10T14:21:33.346468+0000 mgr.gibba032.whopeo (mgr.24625) 1548 : audit [DBG] from='client.55084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[30791]: audit 2021-03-10T14:21:33.568319+0000 mgr.gibba032.whopeo (mgr.24625) 1549 : audit [DBG] from='client.45105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[30791]: cluster 2021-03-10T14:21:33.668370+0000 mgr.gibba032.whopeo (mgr.24625) 1550 : cluster [DBG] pgmap v873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:35.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[30791]: audit 2021-03-10T14:21:33.831102+0000 mon.c (mon.1) 31 : audit [DBG] from='client.? 172.21.2.111:0/3075484051' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[31774]: audit 2021-03-10T14:21:33.346468+0000 mgr.gibba032.whopeo (mgr.24625) 1548 : audit [DBG] from='client.55084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[31774]: audit 2021-03-10T14:21:33.568319+0000 mgr.gibba032.whopeo (mgr.24625) 1549 : audit [DBG] from='client.45105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:21:35.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[31774]: cluster 2021-03-10T14:21:33.668370+0000 mgr.gibba032.whopeo (mgr.24625) 1550 : cluster [DBG] pgmap v873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:35.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:34 gibba011 bash[31774]: audit 2021-03-10T14:21:33.831102+0000 mon.c (mon.1) 31 : audit [DBG] from='client.? 172.21.2.111:0/3075484051' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:36 gibba032 bash[24004]: cluster 2021-03-10T14:21:35.669685+0000 mgr.gibba032.whopeo (mgr.24625) 1551 : cluster [DBG] pgmap v874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:36 gibba011 bash[30791]: cluster 2021-03-10T14:21:35.669685+0000 mgr.gibba032.whopeo (mgr.24625) 1551 : cluster [DBG] pgmap v874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:36 gibba011 bash[31774]: cluster 2021-03-10T14:21:35.669685+0000 mgr.gibba032.whopeo (mgr.24625) 1551 : cluster [DBG] pgmap v874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.099358+0000 mgr.gibba032.whopeo (mgr.24625) 1552 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:21:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.101075+0000 mgr.gibba032.whopeo (mgr.24625) 1553 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:21:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.325582+0000 mgr.gibba032.whopeo (mgr.24625) 1554 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:21:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.325633+0000 mon.a (mon.0) 1008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:21:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.327859+0000 mgr.gibba032.whopeo (mgr.24625) 1555 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:21:38.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.329186+0000 mon.a (mon.0) 1009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:21:38.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.329331+0000 mgr.gibba032.whopeo (mgr.24625) 1556 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:21:38.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.329368+0000 mgr.gibba032.whopeo (mgr.24625) 1557 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:21:38.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.330828+0000 mgr.gibba032.whopeo (mgr.24625) 1558 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:21:38.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.330869+0000 mgr.gibba032.whopeo (mgr.24625) 1559 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:21:38.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.330996+0000 mon.a (mon.0) 1010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.332265+0000 mgr.gibba032.whopeo (mgr.24625) 1560 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:21:38.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.332449+0000 mon.a (mon.0) 1011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.333886+0000 mon.a (mon.0) 1012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.333987+0000 mgr.gibba032.whopeo (mgr.24625) 1561 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:21:38.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.334025+0000 mgr.gibba032.whopeo (mgr.24625) 1562 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:21:38.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.335298+0000 mgr.gibba032.whopeo (mgr.24625) 1563 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:21:38.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.335616+0000 mgr.gibba032.whopeo (mgr.24625) 1564 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:21:38.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.337055+0000 mon.a (mon.0) 1013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.337056+0000 mgr.gibba032.whopeo (mgr.24625) 1565 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:21:38.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.337094+0000 mgr.gibba032.whopeo (mgr.24625) 1566 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:21:38.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cephadm 2021-03-10T14:21:37.337230+0000 mgr.gibba032.whopeo (mgr.24625) 1567 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:38.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: audit 2021-03-10T14:21:37.338619+0000 mon.a (mon.0) 1014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:38 gibba032 bash[24004]: cluster 2021-03-10T14:21:37.671460+0000 mgr.gibba032.whopeo (mgr.24625) 1568 : cluster [DBG] pgmap v875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.099358+0000 mgr.gibba032.whopeo (mgr.24625) 1552 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:21:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.101075+0000 mgr.gibba032.whopeo (mgr.24625) 1553 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:21:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.325582+0000 mgr.gibba032.whopeo (mgr.24625) 1554 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:21:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.325633+0000 mon.a (mon.0) 1008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:21:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.327859+0000 mgr.gibba032.whopeo (mgr.24625) 1555 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:21:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.329186+0000 mon.a (mon.0) 1009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:21:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.329331+0000 mgr.gibba032.whopeo (mgr.24625) 1556 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:21:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.329368+0000 mgr.gibba032.whopeo (mgr.24625) 1557 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:21:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.330828+0000 mgr.gibba032.whopeo (mgr.24625) 1558 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:21:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.330869+0000 mgr.gibba032.whopeo (mgr.24625) 1559 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:21:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.330996+0000 mon.a (mon.0) 1010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.332265+0000 mgr.gibba032.whopeo (mgr.24625) 1560 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:21:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.332449+0000 mon.a (mon.0) 1011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.333886+0000 mon.a (mon.0) 1012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.333987+0000 mgr.gibba032.whopeo (mgr.24625) 1561 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:21:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.334025+0000 mgr.gibba032.whopeo (mgr.24625) 1562 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:21:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.335298+0000 mgr.gibba032.whopeo (mgr.24625) 1563 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:21:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.335616+0000 mgr.gibba032.whopeo (mgr.24625) 1564 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:21:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.337055+0000 mon.a (mon.0) 1013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.337056+0000 mgr.gibba032.whopeo (mgr.24625) 1565 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:21:38.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.337094+0000 mgr.gibba032.whopeo (mgr.24625) 1566 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:21:38.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.099358+0000 mgr.gibba032.whopeo (mgr.24625) 1552 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:21:38.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.101075+0000 mgr.gibba032.whopeo (mgr.24625) 1553 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:21:38.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.325582+0000 mgr.gibba032.whopeo (mgr.24625) 1554 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:21:38.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.325633+0000 mon.a (mon.0) 1008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:21:38.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.327859+0000 mgr.gibba032.whopeo (mgr.24625) 1555 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:21:38.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.329186+0000 mon.a (mon.0) 1009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:21:38.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.329331+0000 mgr.gibba032.whopeo (mgr.24625) 1556 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:21:38.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.329368+0000 mgr.gibba032.whopeo (mgr.24625) 1557 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:21:38.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.330828+0000 mgr.gibba032.whopeo (mgr.24625) 1558 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:21:38.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.330869+0000 mgr.gibba032.whopeo (mgr.24625) 1559 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:21:38.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.330996+0000 mon.a (mon.0) 1010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.332265+0000 mgr.gibba032.whopeo (mgr.24625) 1560 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:21:38.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.332449+0000 mon.a (mon.0) 1011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.333886+0000 mon.a (mon.0) 1012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.333987+0000 mgr.gibba032.whopeo (mgr.24625) 1561 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:21:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.334025+0000 mgr.gibba032.whopeo (mgr.24625) 1562 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:21:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.335298+0000 mgr.gibba032.whopeo (mgr.24625) 1563 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:21:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.335616+0000 mgr.gibba032.whopeo (mgr.24625) 1564 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:21:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.337055+0000 mon.a (mon.0) 1013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.337056+0000 mgr.gibba032.whopeo (mgr.24625) 1565 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:21:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.337094+0000 mgr.gibba032.whopeo (mgr.24625) 1566 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:21:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cephadm 2021-03-10T14:21:37.337230+0000 mgr.gibba032.whopeo (mgr.24625) 1567 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: audit 2021-03-10T14:21:37.338619+0000 mon.a (mon.0) 1014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[31774]: cluster 2021-03-10T14:21:37.671460+0000 mgr.gibba032.whopeo (mgr.24625) 1568 : cluster [DBG] pgmap v875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:38.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cephadm 2021-03-10T14:21:37.337230+0000 mgr.gibba032.whopeo (mgr.24625) 1567 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:38.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: audit 2021-03-10T14:21:37.338619+0000 mon.a (mon.0) 1014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:21:38.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:38 gibba011 bash[30791]: cluster 2021-03-10T14:21:37.671460+0000 mgr.gibba032.whopeo (mgr.24625) 1568 : cluster [DBG] pgmap v875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:40 gibba032 bash[24004]: cluster 2021-03-10T14:21:39.672943+0000 mgr.gibba032.whopeo (mgr.24625) 1569 : cluster [DBG] pgmap v876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:40 gibba011 bash[30791]: cluster 2021-03-10T14:21:39.672943+0000 mgr.gibba032.whopeo (mgr.24625) 1569 : cluster [DBG] pgmap v876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:40 gibba011 bash[31774]: cluster 2021-03-10T14:21:39.672943+0000 mgr.gibba032.whopeo (mgr.24625) 1569 : cluster [DBG] pgmap v876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:42 gibba032 bash[24004]: cluster 2021-03-10T14:21:41.674453+0000 mgr.gibba032.whopeo (mgr.24625) 1570 : cluster [DBG] pgmap v877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:42 gibba011 bash[30791]: cluster 2021-03-10T14:21:41.674453+0000 mgr.gibba032.whopeo (mgr.24625) 1570 : cluster [DBG] pgmap v877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:42 gibba011 bash[31774]: cluster 2021-03-10T14:21:41.674453+0000 mgr.gibba032.whopeo (mgr.24625) 1570 : cluster [DBG] pgmap v877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:44 gibba032 bash[24004]: cluster 2021-03-10T14:21:43.676048+0000 mgr.gibba032.whopeo (mgr.24625) 1571 : cluster [DBG] pgmap v878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:44 gibba011 bash[30791]: cluster 2021-03-10T14:21:43.676048+0000 mgr.gibba032.whopeo (mgr.24625) 1571 : cluster [DBG] pgmap v878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:44 gibba011 bash[31774]: cluster 2021-03-10T14:21:43.676048+0000 mgr.gibba032.whopeo (mgr.24625) 1571 : cluster [DBG] pgmap v878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:46 gibba032 bash[24004]: cluster 2021-03-10T14:21:45.676627+0000 mgr.gibba032.whopeo (mgr.24625) 1572 : cluster [DBG] pgmap v879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:46 gibba011 bash[30791]: cluster 2021-03-10T14:21:45.676627+0000 mgr.gibba032.whopeo (mgr.24625) 1572 : cluster [DBG] pgmap v879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:46 gibba011 bash[31774]: cluster 2021-03-10T14:21:45.676627+0000 mgr.gibba032.whopeo (mgr.24625) 1572 : cluster [DBG] pgmap v879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:48 gibba032 bash[24004]: cluster 2021-03-10T14:21:47.678022+0000 mgr.gibba032.whopeo (mgr.24625) 1573 : cluster [DBG] pgmap v880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:48 gibba011 bash[30791]: cluster 2021-03-10T14:21:47.678022+0000 mgr.gibba032.whopeo (mgr.24625) 1573 : cluster [DBG] pgmap v880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:48 gibba011 bash[31774]: cluster 2021-03-10T14:21:47.678022+0000 mgr.gibba032.whopeo (mgr.24625) 1573 : cluster [DBG] pgmap v880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:50 gibba032 bash[24004]: cluster 2021-03-10T14:21:49.679346+0000 mgr.gibba032.whopeo (mgr.24625) 1574 : cluster [DBG] pgmap v881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:50 gibba011 bash[30791]: cluster 2021-03-10T14:21:49.679346+0000 mgr.gibba032.whopeo (mgr.24625) 1574 : cluster [DBG] pgmap v881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:50 gibba011 bash[31774]: cluster 2021-03-10T14:21:49.679346+0000 mgr.gibba032.whopeo (mgr.24625) 1574 : cluster [DBG] pgmap v881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:52 gibba011 bash[30791]: cluster 2021-03-10T14:21:51.680920+0000 mgr.gibba032.whopeo (mgr.24625) 1575 : cluster [DBG] pgmap v882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:53.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:52 gibba011 bash[31774]: cluster 2021-03-10T14:21:51.680920+0000 mgr.gibba032.whopeo (mgr.24625) 1575 : cluster [DBG] pgmap v882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:53.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:52 gibba032 bash[24004]: cluster 2021-03-10T14:21:51.680920+0000 mgr.gibba032.whopeo (mgr.24625) 1575 : cluster [DBG] pgmap v882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:53 gibba011 bash[30791]: cephadm 2021-03-10T14:21:52.343421+0000 mgr.gibba032.whopeo (mgr.24625) 1576 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:54.242 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:53 gibba011 bash[31774]: cephadm 2021-03-10T14:21:52.343421+0000 mgr.gibba032.whopeo (mgr.24625) 1576 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:54.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:53 gibba032 bash[24004]: cephadm 2021-03-10T14:21:52.343421+0000 mgr.gibba032.whopeo (mgr.24625) 1576 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:21:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:54 gibba011 bash[30791]: cluster 2021-03-10T14:21:53.682205+0000 mgr.gibba032.whopeo (mgr.24625) 1577 : cluster [DBG] pgmap v883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:54 gibba011 bash[31774]: cluster 2021-03-10T14:21:53.682205+0000 mgr.gibba032.whopeo (mgr.24625) 1577 : cluster [DBG] pgmap v883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:54 gibba032 bash[24004]: cluster 2021-03-10T14:21:53.682205+0000 mgr.gibba032.whopeo (mgr.24625) 1577 : cluster [DBG] pgmap v883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: cluster 2021-03-10T14:21:55.683624+0000 mgr.gibba032.whopeo (mgr.24625) 1578 : cluster [DBG] pgmap v884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.134531+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.135253+0000 mon.a (mon.0) 1016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.136037+0000 mon.a (mon.0) 1017 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.136716+0000 mon.a (mon.0) 1018 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.185360+0000 mon.a (mon.0) 1019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:21:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:56 gibba032 bash[24004]: audit 2021-03-10T14:21:56.186043+0000 mon.a (mon.0) 1020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:21:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: cluster 2021-03-10T14:21:55.683624+0000 mgr.gibba032.whopeo (mgr.24625) 1578 : cluster [DBG] pgmap v884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.134531+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:21:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.135253+0000 mon.a (mon.0) 1016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:21:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.136037+0000 mon.a (mon.0) 1017 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.136716+0000 mon.a (mon.0) 1018 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.185360+0000 mon.a (mon.0) 1019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:21:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[30791]: audit 2021-03-10T14:21:56.186043+0000 mon.a (mon.0) 1020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:21:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: cluster 2021-03-10T14:21:55.683624+0000 mgr.gibba032.whopeo (mgr.24625) 1578 : cluster [DBG] pgmap v884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.134531+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:21:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.135253+0000 mon.a (mon.0) 1016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:21:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.136037+0000 mon.a (mon.0) 1017 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.136716+0000 mon.a (mon.0) 1018 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:21:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.185360+0000 mon.a (mon.0) 1019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:21:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:56 gibba011 bash[31774]: audit 2021-03-10T14:21:56.186043+0000 mon.a (mon.0) 1020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:21:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:21:58 gibba032 bash[24004]: cluster 2021-03-10T14:21:57.685544+0000 mgr.gibba032.whopeo (mgr.24625) 1579 : cluster [DBG] pgmap v885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:21:58 gibba011 bash[30791]: cluster 2021-03-10T14:21:57.685544+0000 mgr.gibba032.whopeo (mgr.24625) 1579 : cluster [DBG] pgmap v885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:21:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:21:58 gibba011 bash[31774]: cluster 2021-03-10T14:21:57.685544+0000 mgr.gibba032.whopeo (mgr.24625) 1579 : cluster [DBG] pgmap v885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:00 gibba011 bash[30791]: cluster 2021-03-10T14:21:59.687073+0000 mgr.gibba032.whopeo (mgr.24625) 1580 : cluster [DBG] pgmap v886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:00 gibba011 bash[31774]: cluster 2021-03-10T14:21:59.687073+0000 mgr.gibba032.whopeo (mgr.24625) 1580 : cluster [DBG] pgmap v886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:00 gibba032 bash[24004]: cluster 2021-03-10T14:21:59.687073+0000 mgr.gibba032.whopeo (mgr.24625) 1580 : cluster [DBG] pgmap v886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:02 gibba011 bash[30791]: cluster 2021-03-10T14:22:01.688805+0000 mgr.gibba032.whopeo (mgr.24625) 1581 : cluster [DBG] pgmap v887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:02 gibba011 bash[31774]: cluster 2021-03-10T14:22:01.688805+0000 mgr.gibba032.whopeo (mgr.24625) 1581 : cluster [DBG] pgmap v887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:03.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:02 gibba032 bash[24004]: cluster 2021-03-10T14:22:01.688805+0000 mgr.gibba032.whopeo (mgr.24625) 1581 : cluster [DBG] pgmap v887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:04.074 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:22:04.299 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:22:04.299 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (28m) 4m ago 34m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:22:04.300 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (33m) 3m ago 33m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:22:04.300 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (29m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:22:04.300 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (29m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:22:04.300 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (28m) 4m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:22:04.300 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (28m) 4m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:22:04.301 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (27m) 3m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:22:04.301 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (27m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:22:04.301 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (34m) 4m ago 34m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:22:04.301 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (34m) 3m ago 34m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:22:04.301 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (27m) 4m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:22:04.302 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (26m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:22:04.302 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (26m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:22:04.302 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (25m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:22:04.302 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (25m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:22:04.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (24m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:22:04.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (24m) 3m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:22:04.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (24m) 3m ago 34m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:22:04.303 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (33m) 3m ago 34m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:22:04.303 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 32m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:22:04.548 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:22:04.549 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:22:04.549 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:22:04.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:04.549 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:22:04.550 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:22:04.550 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:04.550 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:22:04.550 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:22:04.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:04.551 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:22:04.551 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:22:04.551 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:22:04.551 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:22:04.552 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:22:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:04 gibba011 bash[30791]: cluster 2021-03-10T14:22:03.690265+0000 mgr.gibba032.whopeo (mgr.24625) 1582 : cluster [DBG] pgmap v888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:04 gibba011 bash[30791]: audit 2021-03-10T14:22:04.548292+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.2.111:0/3426173707' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:04 gibba011 bash[31774]: cluster 2021-03-10T14:22:03.690265+0000 mgr.gibba032.whopeo (mgr.24625) 1582 : cluster [DBG] pgmap v888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:04 gibba011 bash[31774]: audit 2021-03-10T14:22:04.548292+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.2.111:0/3426173707' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:04 gibba032 bash[24004]: cluster 2021-03-10T14:22:03.690265+0000 mgr.gibba032.whopeo (mgr.24625) 1582 : cluster [DBG] pgmap v888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:05.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:04 gibba032 bash[24004]: audit 2021-03-10T14:22:04.548292+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.2.111:0/3426173707' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:06.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:05 gibba011 bash[30791]: audit 2021-03-10T14:22:04.061661+0000 mgr.gibba032.whopeo (mgr.24625) 1583 : audit [DBG] from='client.45117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:05 gibba011 bash[30791]: audit 2021-03-10T14:22:04.284441+0000 mgr.gibba032.whopeo (mgr.24625) 1584 : audit [DBG] from='client.55108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:06.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:05 gibba011 bash[31774]: audit 2021-03-10T14:22:04.061661+0000 mgr.gibba032.whopeo (mgr.24625) 1583 : audit [DBG] from='client.45117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:06.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:05 gibba011 bash[31774]: audit 2021-03-10T14:22:04.284441+0000 mgr.gibba032.whopeo (mgr.24625) 1584 : audit [DBG] from='client.55108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:06.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:05 gibba032 bash[24004]: audit 2021-03-10T14:22:04.061661+0000 mgr.gibba032.whopeo (mgr.24625) 1583 : audit [DBG] from='client.45117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:06.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:05 gibba032 bash[24004]: audit 2021-03-10T14:22:04.284441+0000 mgr.gibba032.whopeo (mgr.24625) 1584 : audit [DBG] from='client.55108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:07.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:06 gibba011 bash[30791]: cluster 2021-03-10T14:22:05.690943+0000 mgr.gibba032.whopeo (mgr.24625) 1585 : cluster [DBG] pgmap v889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:06 gibba011 bash[31774]: cluster 2021-03-10T14:22:05.690943+0000 mgr.gibba032.whopeo (mgr.24625) 1585 : cluster [DBG] pgmap v889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:06 gibba032 bash[24004]: cluster 2021-03-10T14:22:05.690943+0000 mgr.gibba032.whopeo (mgr.24625) 1585 : cluster [DBG] pgmap v889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:08 gibba011 bash[30791]: cephadm 2021-03-10T14:22:07.352136+0000 mgr.gibba032.whopeo (mgr.24625) 1586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:08 gibba011 bash[30791]: cluster 2021-03-10T14:22:07.692438+0000 mgr.gibba032.whopeo (mgr.24625) 1587 : cluster [DBG] pgmap v890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:08 gibba011 bash[31774]: cephadm 2021-03-10T14:22:07.352136+0000 mgr.gibba032.whopeo (mgr.24625) 1586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:08 gibba011 bash[31774]: cluster 2021-03-10T14:22:07.692438+0000 mgr.gibba032.whopeo (mgr.24625) 1587 : cluster [DBG] pgmap v890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:08 gibba032 bash[24004]: cephadm 2021-03-10T14:22:07.352136+0000 mgr.gibba032.whopeo (mgr.24625) 1586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:08 gibba032 bash[24004]: cluster 2021-03-10T14:22:07.692438+0000 mgr.gibba032.whopeo (mgr.24625) 1587 : cluster [DBG] pgmap v890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:10 gibba011 bash[30791]: cluster 2021-03-10T14:22:09.693927+0000 mgr.gibba032.whopeo (mgr.24625) 1588 : cluster [DBG] pgmap v891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:10 gibba011 bash[31774]: cluster 2021-03-10T14:22:09.693927+0000 mgr.gibba032.whopeo (mgr.24625) 1588 : cluster [DBG] pgmap v891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:10 gibba032 bash[24004]: cluster 2021-03-10T14:22:09.693927+0000 mgr.gibba032.whopeo (mgr.24625) 1588 : cluster [DBG] pgmap v891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:12 gibba011 bash[30791]: cluster 2021-03-10T14:22:11.695652+0000 mgr.gibba032.whopeo (mgr.24625) 1589 : cluster [DBG] pgmap v892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:12 gibba011 bash[31774]: cluster 2021-03-10T14:22:11.695652+0000 mgr.gibba032.whopeo (mgr.24625) 1589 : cluster [DBG] pgmap v892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:12 gibba032 bash[24004]: cluster 2021-03-10T14:22:11.695652+0000 mgr.gibba032.whopeo (mgr.24625) 1589 : cluster [DBG] pgmap v892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:14 gibba011 bash[30791]: cluster 2021-03-10T14:22:13.697269+0000 mgr.gibba032.whopeo (mgr.24625) 1590 : cluster [DBG] pgmap v893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:14 gibba011 bash[31774]: cluster 2021-03-10T14:22:13.697269+0000 mgr.gibba032.whopeo (mgr.24625) 1590 : cluster [DBG] pgmap v893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:14 gibba032 bash[24004]: cluster 2021-03-10T14:22:13.697269+0000 mgr.gibba032.whopeo (mgr.24625) 1590 : cluster [DBG] pgmap v893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:16 gibba032 bash[24004]: cluster 2021-03-10T14:22:15.698655+0000 mgr.gibba032.whopeo (mgr.24625) 1591 : cluster [DBG] pgmap v894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:16 gibba011 bash[30791]: cluster 2021-03-10T14:22:15.698655+0000 mgr.gibba032.whopeo (mgr.24625) 1591 : cluster [DBG] pgmap v894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:16 gibba011 bash[31774]: cluster 2021-03-10T14:22:15.698655+0000 mgr.gibba032.whopeo (mgr.24625) 1591 : cluster [DBG] pgmap v894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:18 gibba011 bash[30791]: cluster 2021-03-10T14:22:17.700311+0000 mgr.gibba032.whopeo (mgr.24625) 1592 : cluster [DBG] pgmap v895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:18 gibba011 bash[31774]: cluster 2021-03-10T14:22:17.700311+0000 mgr.gibba032.whopeo (mgr.24625) 1592 : cluster [DBG] pgmap v895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:18 gibba032 bash[24004]: cluster 2021-03-10T14:22:17.700311+0000 mgr.gibba032.whopeo (mgr.24625) 1592 : cluster [DBG] pgmap v895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:20 gibba011 bash[30791]: cluster 2021-03-10T14:22:19.701579+0000 mgr.gibba032.whopeo (mgr.24625) 1593 : cluster [DBG] pgmap v896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:20 gibba011 bash[31774]: cluster 2021-03-10T14:22:19.701579+0000 mgr.gibba032.whopeo (mgr.24625) 1593 : cluster [DBG] pgmap v896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:20 gibba032 bash[24004]: cluster 2021-03-10T14:22:19.701579+0000 mgr.gibba032.whopeo (mgr.24625) 1593 : cluster [DBG] pgmap v896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:22 gibba011 bash[30791]: cluster 2021-03-10T14:22:21.703229+0000 mgr.gibba032.whopeo (mgr.24625) 1594 : cluster [DBG] pgmap v897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:22 gibba011 bash[31774]: cluster 2021-03-10T14:22:21.703229+0000 mgr.gibba032.whopeo (mgr.24625) 1594 : cluster [DBG] pgmap v897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:22 gibba032 bash[24004]: cluster 2021-03-10T14:22:21.703229+0000 mgr.gibba032.whopeo (mgr.24625) 1594 : cluster [DBG] pgmap v897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:24.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:23 gibba011 bash[30791]: cephadm 2021-03-10T14:22:22.364716+0000 mgr.gibba032.whopeo (mgr.24625) 1595 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:24.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:23 gibba011 bash[31774]: cephadm 2021-03-10T14:22:22.364716+0000 mgr.gibba032.whopeo (mgr.24625) 1595 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:23 gibba032 bash[24004]: cephadm 2021-03-10T14:22:22.364716+0000 mgr.gibba032.whopeo (mgr.24625) 1595 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:24 gibba011 bash[30791]: cluster 2021-03-10T14:22:23.704837+0000 mgr.gibba032.whopeo (mgr.24625) 1596 : cluster [DBG] pgmap v898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:24 gibba011 bash[31774]: cluster 2021-03-10T14:22:23.704837+0000 mgr.gibba032.whopeo (mgr.24625) 1596 : cluster [DBG] pgmap v898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:24 gibba032 bash[24004]: cluster 2021-03-10T14:22:23.704837+0000 mgr.gibba032.whopeo (mgr.24625) 1596 : cluster [DBG] pgmap v898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:26 gibba032 bash[24004]: cluster 2021-03-10T14:22:25.706120+0000 mgr.gibba032.whopeo (mgr.24625) 1597 : cluster [DBG] pgmap v899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:26 gibba011 bash[30791]: cluster 2021-03-10T14:22:25.706120+0000 mgr.gibba032.whopeo (mgr.24625) 1597 : cluster [DBG] pgmap v899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:26 gibba011 bash[31774]: cluster 2021-03-10T14:22:25.706120+0000 mgr.gibba032.whopeo (mgr.24625) 1597 : cluster [DBG] pgmap v899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:28 gibba011 bash[30791]: cluster 2021-03-10T14:22:27.707872+0000 mgr.gibba032.whopeo (mgr.24625) 1598 : cluster [DBG] pgmap v900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:28 gibba011 bash[31774]: cluster 2021-03-10T14:22:27.707872+0000 mgr.gibba032.whopeo (mgr.24625) 1598 : cluster [DBG] pgmap v900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:28 gibba032 bash[24004]: cluster 2021-03-10T14:22:27.707872+0000 mgr.gibba032.whopeo (mgr.24625) 1598 : cluster [DBG] pgmap v900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:30 gibba011 bash[30791]: cluster 2021-03-10T14:22:29.709430+0000 mgr.gibba032.whopeo (mgr.24625) 1599 : cluster [DBG] pgmap v901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:30 gibba011 bash[31774]: cluster 2021-03-10T14:22:29.709430+0000 mgr.gibba032.whopeo (mgr.24625) 1599 : cluster [DBG] pgmap v901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:30 gibba032 bash[24004]: cluster 2021-03-10T14:22:29.709430+0000 mgr.gibba032.whopeo (mgr.24625) 1599 : cluster [DBG] pgmap v901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:32 gibba011 bash[30791]: cluster 2021-03-10T14:22:31.711189+0000 mgr.gibba032.whopeo (mgr.24625) 1600 : cluster [DBG] pgmap v902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:32 gibba011 bash[31774]: cluster 2021-03-10T14:22:31.711189+0000 mgr.gibba032.whopeo (mgr.24625) 1600 : cluster [DBG] pgmap v902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:32 gibba032 bash[24004]: cluster 2021-03-10T14:22:31.711189+0000 mgr.gibba032.whopeo (mgr.24625) 1600 : cluster [DBG] pgmap v902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:34.792 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:22:35.046 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:22:35.047 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (29m) 5m ago 34m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:22:35.047 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (34m) 4m ago 34m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:22:35.047 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (29m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:22:35.047 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (29m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:22:35.047 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (28m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:22:35.048 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (28m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:22:35.048 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (28m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:22:35.048 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (28m) 5m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:22:35.048 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (34m) 5m ago 34m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:22:35.048 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (34m) 4m ago 34m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:22:35.049 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (27m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:22:35.049 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (27m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:22:35.049 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (26m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:22:35.049 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (26m) 5m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:22:35.049 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (25m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:22:35.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (25m) 4m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:22:35.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (24m) 4m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:22:35.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (24m) 4m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:22:35.050 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (34m) 4m ago 34m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:22:35.051 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 32m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:22:35.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:34 gibba011 bash[30791]: cluster 2021-03-10T14:22:33.712826+0000 mgr.gibba032.whopeo (mgr.24625) 1601 : cluster [DBG] pgmap v903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:34 gibba011 bash[31774]: cluster 2021-03-10T14:22:33.712826+0000 mgr.gibba032.whopeo (mgr.24625) 1601 : cluster [DBG] pgmap v903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:34 gibba032 bash[24004]: cluster 2021-03-10T14:22:33.712826+0000 mgr.gibba032.whopeo (mgr.24625) 1601 : cluster [DBG] pgmap v903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:35.302 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:22:35.302 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:22:35.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:22:35.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:35.303 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:22:35.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:22:35.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:35.304 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:22:35.304 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:22:35.304 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:22:35.304 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:22:35.304 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:22:35.305 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:22:35.305 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:22:35.305 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:22:36.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:35 gibba011 bash[30791]: audit 2021-03-10T14:22:34.780116+0000 mgr.gibba032.whopeo (mgr.24625) 1602 : audit [DBG] from='client.55120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:36.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:35 gibba011 bash[30791]: audit 2021-03-10T14:22:35.299661+0000 mon.c (mon.1) 32 : audit [DBG] from='client.? 172.21.2.111:0/3794828945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:36.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:35 gibba011 bash[31774]: audit 2021-03-10T14:22:34.780116+0000 mgr.gibba032.whopeo (mgr.24625) 1602 : audit [DBG] from='client.55120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:36.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:35 gibba011 bash[31774]: audit 2021-03-10T14:22:35.299661+0000 mon.c (mon.1) 32 : audit [DBG] from='client.? 172.21.2.111:0/3794828945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:36.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:35 gibba032 bash[24004]: audit 2021-03-10T14:22:34.780116+0000 mgr.gibba032.whopeo (mgr.24625) 1602 : audit [DBG] from='client.55120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:36.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:35 gibba032 bash[24004]: audit 2021-03-10T14:22:35.299661+0000 mon.c (mon.1) 32 : audit [DBG] from='client.? 172.21.2.111:0/3794828945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:36 gibba011 bash[30791]: audit 2021-03-10T14:22:35.004289+0000 mgr.gibba032.whopeo (mgr.24625) 1603 : audit [DBG] from='client.45141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:37.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:36 gibba011 bash[30791]: cluster 2021-03-10T14:22:35.713709+0000 mgr.gibba032.whopeo (mgr.24625) 1604 : cluster [DBG] pgmap v904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:36 gibba011 bash[31774]: audit 2021-03-10T14:22:35.004289+0000 mgr.gibba032.whopeo (mgr.24625) 1603 : audit [DBG] from='client.45141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:36 gibba011 bash[31774]: cluster 2021-03-10T14:22:35.713709+0000 mgr.gibba032.whopeo (mgr.24625) 1604 : cluster [DBG] pgmap v904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:37.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:36 gibba032 bash[24004]: audit 2021-03-10T14:22:35.004289+0000 mgr.gibba032.whopeo (mgr.24625) 1603 : audit [DBG] from='client.45141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:22:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:36 gibba032 bash[24004]: cluster 2021-03-10T14:22:35.713709+0000 mgr.gibba032.whopeo (mgr.24625) 1604 : cluster [DBG] pgmap v904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.381508+0000 mgr.gibba032.whopeo (mgr.24625) 1605 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:22:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.383049+0000 mgr.gibba032.whopeo (mgr.24625) 1606 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:22:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.609606+0000 mgr.gibba032.whopeo (mgr.24625) 1607 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:22:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.609798+0000 mon.a (mon.0) 1022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:22:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.612608+0000 mgr.gibba032.whopeo (mgr.24625) 1608 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:22:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.613239+0000 mon.a (mon.0) 1023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:22:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.614878+0000 mgr.gibba032.whopeo (mgr.24625) 1609 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:22:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.614914+0000 mgr.gibba032.whopeo (mgr.24625) 1610 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:22:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.615838+0000 mon.a (mon.0) 1024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.617584+0000 mgr.gibba032.whopeo (mgr.24625) 1611 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:22:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.617624+0000 mgr.gibba032.whopeo (mgr.24625) 1612 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:22:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.618125+0000 mon.a (mon.0) 1025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.620426+0000 mgr.gibba032.whopeo (mgr.24625) 1613 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:22:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.621171+0000 mon.a (mon.0) 1026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.623372+0000 mgr.gibba032.whopeo (mgr.24625) 1614 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:22:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.623415+0000 mgr.gibba032.whopeo (mgr.24625) 1615 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:22:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.626322+0000 mgr.gibba032.whopeo (mgr.24625) 1616 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:22:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.626725+0000 mgr.gibba032.whopeo (mgr.24625) 1617 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:22:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.627041+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.630065+0000 mgr.gibba032.whopeo (mgr.24625) 1618 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:22:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.630108+0000 mgr.gibba032.whopeo (mgr.24625) 1619 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:22:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: audit 2021-03-10T14:22:37.630176+0000 mon.a (mon.0) 1028 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cephadm 2021-03-10T14:22:37.630234+0000 mgr.gibba032.whopeo (mgr.24625) 1620 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:38 gibba032 bash[24004]: cluster 2021-03-10T14:22:37.715310+0000 mgr.gibba032.whopeo (mgr.24625) 1621 : cluster [DBG] pgmap v905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.381508+0000 mgr.gibba032.whopeo (mgr.24625) 1605 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:22:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.383049+0000 mgr.gibba032.whopeo (mgr.24625) 1606 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:22:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.609606+0000 mgr.gibba032.whopeo (mgr.24625) 1607 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:22:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.609798+0000 mon.a (mon.0) 1022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:22:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.612608+0000 mgr.gibba032.whopeo (mgr.24625) 1608 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:22:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.613239+0000 mon.a (mon.0) 1023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:22:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.614878+0000 mgr.gibba032.whopeo (mgr.24625) 1609 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:22:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.614914+0000 mgr.gibba032.whopeo (mgr.24625) 1610 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:22:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.615838+0000 mon.a (mon.0) 1024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.617584+0000 mgr.gibba032.whopeo (mgr.24625) 1611 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:22:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.617624+0000 mgr.gibba032.whopeo (mgr.24625) 1612 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:22:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.618125+0000 mon.a (mon.0) 1025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.620426+0000 mgr.gibba032.whopeo (mgr.24625) 1613 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:22:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.621171+0000 mon.a (mon.0) 1026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.623372+0000 mgr.gibba032.whopeo (mgr.24625) 1614 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:22:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.623415+0000 mgr.gibba032.whopeo (mgr.24625) 1615 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:22:39.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.626322+0000 mgr.gibba032.whopeo (mgr.24625) 1616 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:22:39.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.626725+0000 mgr.gibba032.whopeo (mgr.24625) 1617 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:22:39.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.627041+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.630065+0000 mgr.gibba032.whopeo (mgr.24625) 1618 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:22:39.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.630108+0000 mgr.gibba032.whopeo (mgr.24625) 1619 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:22:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.381508+0000 mgr.gibba032.whopeo (mgr.24625) 1605 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:22:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.383049+0000 mgr.gibba032.whopeo (mgr.24625) 1606 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:22:39.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.609606+0000 mgr.gibba032.whopeo (mgr.24625) 1607 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:22:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.609798+0000 mon.a (mon.0) 1022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:22:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.612608+0000 mgr.gibba032.whopeo (mgr.24625) 1608 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:22:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.613239+0000 mon.a (mon.0) 1023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:22:39.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.614878+0000 mgr.gibba032.whopeo (mgr.24625) 1609 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:22:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.614914+0000 mgr.gibba032.whopeo (mgr.24625) 1610 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:22:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.615838+0000 mon.a (mon.0) 1024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.617584+0000 mgr.gibba032.whopeo (mgr.24625) 1611 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:22:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.617624+0000 mgr.gibba032.whopeo (mgr.24625) 1612 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:22:39.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.618125+0000 mon.a (mon.0) 1025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.620426+0000 mgr.gibba032.whopeo (mgr.24625) 1613 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:22:39.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.621171+0000 mon.a (mon.0) 1026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.623372+0000 mgr.gibba032.whopeo (mgr.24625) 1614 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:22:39.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.623415+0000 mgr.gibba032.whopeo (mgr.24625) 1615 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:22:39.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.626322+0000 mgr.gibba032.whopeo (mgr.24625) 1616 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:22:39.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.626725+0000 mgr.gibba032.whopeo (mgr.24625) 1617 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:22:39.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.627041+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.630065+0000 mgr.gibba032.whopeo (mgr.24625) 1618 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:22:39.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.630108+0000 mgr.gibba032.whopeo (mgr.24625) 1619 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:22:39.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: audit 2021-03-10T14:22:37.630176+0000 mon.a (mon.0) 1028 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cephadm 2021-03-10T14:22:37.630234+0000 mgr.gibba032.whopeo (mgr.24625) 1620 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:39.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[30791]: cluster 2021-03-10T14:22:37.715310+0000 mgr.gibba032.whopeo (mgr.24625) 1621 : cluster [DBG] pgmap v905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:39.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: audit 2021-03-10T14:22:37.630176+0000 mon.a (mon.0) 1028 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:22:39.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cephadm 2021-03-10T14:22:37.630234+0000 mgr.gibba032.whopeo (mgr.24625) 1620 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:39.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:38 gibba011 bash[31774]: cluster 2021-03-10T14:22:37.715310+0000 mgr.gibba032.whopeo (mgr.24625) 1621 : cluster [DBG] pgmap v905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:40 gibba011 bash[30791]: cluster 2021-03-10T14:22:39.716725+0000 mgr.gibba032.whopeo (mgr.24625) 1622 : cluster [DBG] pgmap v906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:40 gibba011 bash[31774]: cluster 2021-03-10T14:22:39.716725+0000 mgr.gibba032.whopeo (mgr.24625) 1622 : cluster [DBG] pgmap v906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:40 gibba032 bash[24004]: cluster 2021-03-10T14:22:39.716725+0000 mgr.gibba032.whopeo (mgr.24625) 1622 : cluster [DBG] pgmap v906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:42 gibba011 bash[30791]: cluster 2021-03-10T14:22:41.718368+0000 mgr.gibba032.whopeo (mgr.24625) 1623 : cluster [DBG] pgmap v907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:42 gibba011 bash[31774]: cluster 2021-03-10T14:22:41.718368+0000 mgr.gibba032.whopeo (mgr.24625) 1623 : cluster [DBG] pgmap v907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:42 gibba032 bash[24004]: cluster 2021-03-10T14:22:41.718368+0000 mgr.gibba032.whopeo (mgr.24625) 1623 : cluster [DBG] pgmap v907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:44 gibba011 bash[30791]: cluster 2021-03-10T14:22:43.720004+0000 mgr.gibba032.whopeo (mgr.24625) 1624 : cluster [DBG] pgmap v908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:44 gibba011 bash[31774]: cluster 2021-03-10T14:22:43.720004+0000 mgr.gibba032.whopeo (mgr.24625) 1624 : cluster [DBG] pgmap v908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:45.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:44 gibba032 bash[24004]: cluster 2021-03-10T14:22:43.720004+0000 mgr.gibba032.whopeo (mgr.24625) 1624 : cluster [DBG] pgmap v908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:46 gibba032 bash[24004]: cluster 2021-03-10T14:22:45.721353+0000 mgr.gibba032.whopeo (mgr.24625) 1625 : cluster [DBG] pgmap v909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:46 gibba011 bash[30791]: cluster 2021-03-10T14:22:45.721353+0000 mgr.gibba032.whopeo (mgr.24625) 1625 : cluster [DBG] pgmap v909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:46 gibba011 bash[31774]: cluster 2021-03-10T14:22:45.721353+0000 mgr.gibba032.whopeo (mgr.24625) 1625 : cluster [DBG] pgmap v909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:48 gibba011 bash[30791]: cluster 2021-03-10T14:22:47.722981+0000 mgr.gibba032.whopeo (mgr.24625) 1626 : cluster [DBG] pgmap v910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:48 gibba011 bash[31774]: cluster 2021-03-10T14:22:47.722981+0000 mgr.gibba032.whopeo (mgr.24625) 1626 : cluster [DBG] pgmap v910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:48 gibba032 bash[24004]: cluster 2021-03-10T14:22:47.722981+0000 mgr.gibba032.whopeo (mgr.24625) 1626 : cluster [DBG] pgmap v910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:50 gibba011 bash[30791]: cluster 2021-03-10T14:22:49.724389+0000 mgr.gibba032.whopeo (mgr.24625) 1627 : cluster [DBG] pgmap v911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:50 gibba011 bash[31774]: cluster 2021-03-10T14:22:49.724389+0000 mgr.gibba032.whopeo (mgr.24625) 1627 : cluster [DBG] pgmap v911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:50 gibba032 bash[24004]: cluster 2021-03-10T14:22:49.724389+0000 mgr.gibba032.whopeo (mgr.24625) 1627 : cluster [DBG] pgmap v911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:52 gibba032 bash[24004]: cluster 2021-03-10T14:22:51.725855+0000 mgr.gibba032.whopeo (mgr.24625) 1628 : cluster [DBG] pgmap v912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:53.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:52 gibba011 bash[30791]: cluster 2021-03-10T14:22:51.725855+0000 mgr.gibba032.whopeo (mgr.24625) 1628 : cluster [DBG] pgmap v912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:52 gibba011 bash[31774]: cluster 2021-03-10T14:22:51.725855+0000 mgr.gibba032.whopeo (mgr.24625) 1628 : cluster [DBG] pgmap v912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:54.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:53 gibba011 bash[30791]: cephadm 2021-03-10T14:22:52.643371+0000 mgr.gibba032.whopeo (mgr.24625) 1629 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:54.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:53 gibba011 bash[31774]: cephadm 2021-03-10T14:22:52.643371+0000 mgr.gibba032.whopeo (mgr.24625) 1629 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:54.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:53 gibba032 bash[24004]: cephadm 2021-03-10T14:22:52.643371+0000 mgr.gibba032.whopeo (mgr.24625) 1629 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:22:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:54 gibba011 bash[30791]: cluster 2021-03-10T14:22:53.727419+0000 mgr.gibba032.whopeo (mgr.24625) 1630 : cluster [DBG] pgmap v913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:54 gibba011 bash[31774]: cluster 2021-03-10T14:22:53.727419+0000 mgr.gibba032.whopeo (mgr.24625) 1630 : cluster [DBG] pgmap v913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:54 gibba032 bash[24004]: cluster 2021-03-10T14:22:53.727419+0000 mgr.gibba032.whopeo (mgr.24625) 1630 : cluster [DBG] pgmap v913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: cluster 2021-03-10T14:22:55.727972+0000 mgr.gibba032.whopeo (mgr.24625) 1631 : cluster [DBG] pgmap v914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.145376+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:22:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.146113+0000 mon.a (mon.0) 1030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.146766+0000 mon.a (mon.0) 1031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:22:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.147487+0000 mon.a (mon.0) 1032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.193685+0000 mon.a (mon.0) 1033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:22:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:56 gibba032 bash[24004]: audit 2021-03-10T14:22:56.194381+0000 mon.a (mon.0) 1034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:22:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: cluster 2021-03-10T14:22:55.727972+0000 mgr.gibba032.whopeo (mgr.24625) 1631 : cluster [DBG] pgmap v914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.145376+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:22:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.146113+0000 mon.a (mon.0) 1030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.146766+0000 mon.a (mon.0) 1031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:22:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.147487+0000 mon.a (mon.0) 1032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.193685+0000 mon.a (mon.0) 1033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:22:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[30791]: audit 2021-03-10T14:22:56.194381+0000 mon.a (mon.0) 1034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:22:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: cluster 2021-03-10T14:22:55.727972+0000 mgr.gibba032.whopeo (mgr.24625) 1631 : cluster [DBG] pgmap v914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.145376+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:22:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.146113+0000 mon.a (mon.0) 1030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.146766+0000 mon.a (mon.0) 1031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:22:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.147487+0000 mon.a (mon.0) 1032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:22:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.193685+0000 mon.a (mon.0) 1033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:22:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:56 gibba011 bash[31774]: audit 2021-03-10T14:22:56.194381+0000 mon.a (mon.0) 1034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:22:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:22:58 gibba011 bash[30791]: cluster 2021-03-10T14:22:57.729473+0000 mgr.gibba032.whopeo (mgr.24625) 1632 : cluster [DBG] pgmap v915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:22:58 gibba011 bash[31774]: cluster 2021-03-10T14:22:57.729473+0000 mgr.gibba032.whopeo (mgr.24625) 1632 : cluster [DBG] pgmap v915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:22:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:22:58 gibba032 bash[24004]: cluster 2021-03-10T14:22:57.729473+0000 mgr.gibba032.whopeo (mgr.24625) 1632 : cluster [DBG] pgmap v915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:00 gibba011 bash[30791]: cluster 2021-03-10T14:22:59.730950+0000 mgr.gibba032.whopeo (mgr.24625) 1633 : cluster [DBG] pgmap v916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:00 gibba011 bash[31774]: cluster 2021-03-10T14:22:59.730950+0000 mgr.gibba032.whopeo (mgr.24625) 1633 : cluster [DBG] pgmap v916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:00 gibba032 bash[24004]: cluster 2021-03-10T14:22:59.730950+0000 mgr.gibba032.whopeo (mgr.24625) 1633 : cluster [DBG] pgmap v916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:02 gibba011 bash[30791]: cluster 2021-03-10T14:23:01.732490+0000 mgr.gibba032.whopeo (mgr.24625) 1634 : cluster [DBG] pgmap v917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:02 gibba011 bash[31774]: cluster 2021-03-10T14:23:01.732490+0000 mgr.gibba032.whopeo (mgr.24625) 1634 : cluster [DBG] pgmap v917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:03.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:02 gibba032 bash[24004]: cluster 2021-03-10T14:23:01.732490+0000 mgr.gibba032.whopeo (mgr.24625) 1634 : cluster [DBG] pgmap v917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:04 gibba011 bash[30791]: cluster 2021-03-10T14:23:03.734080+0000 mgr.gibba032.whopeo (mgr.24625) 1635 : cluster [DBG] pgmap v918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:04 gibba011 bash[31774]: cluster 2021-03-10T14:23:03.734080+0000 mgr.gibba032.whopeo (mgr.24625) 1635 : cluster [DBG] pgmap v918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:05.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:04 gibba032 bash[24004]: cluster 2021-03-10T14:23:03.734080+0000 mgr.gibba032.whopeo (mgr.24625) 1635 : cluster [DBG] pgmap v918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:05.543 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:23:05.771 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:23:05.772 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (29m) 5m ago 35m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:23:05.772 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (34m) 4m ago 34m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:23:05.772 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (30m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:23:05.772 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (30m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:23:05.773 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (29m) 5m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:23:05.773 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (29m) 5m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:23:05.773 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (28m) 4m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:23:05.773 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (29m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:23:05.773 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (35m) 5m ago 35m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:23:05.774 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (35m) 4m ago 35m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:23:05.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (28m) 5m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:23:05.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (27m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:23:05.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (27m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:23:05.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (26m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:23:05.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (26m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:23:05.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (25m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:23:05.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (25m) 4m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:23:05.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (25m) 4m ago 35m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:23:05.776 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (34m) 4m ago 35m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:23:05.776 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 33m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:23:06.024 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:23:06.024 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:23:06.024 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:23:06.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:06.025 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:23:06.025 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:23:06.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:06.025 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:23:06.026 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:23:06.026 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:06.026 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:23:06.026 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:23:06.027 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:23:06.027 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:23:06.027 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:23:07.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:06 gibba032 bash[24004]: audit 2021-03-10T14:23:05.529242+0000 mgr.gibba032.whopeo (mgr.24625) 1636 : audit [DBG] from='client.55138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:06 gibba032 bash[24004]: cluster 2021-03-10T14:23:05.735130+0000 mgr.gibba032.whopeo (mgr.24625) 1637 : cluster [DBG] pgmap v919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:06 gibba032 bash[24004]: audit 2021-03-10T14:23:05.756336+0000 mgr.gibba032.whopeo (mgr.24625) 1638 : audit [DBG] from='client.45159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:06 gibba032 bash[24004]: audit 2021-03-10T14:23:06.023430+0000 mon.a (mon.0) 1035 : audit [DBG] from='client.? 172.21.2.111:0/1718925061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[30791]: audit 2021-03-10T14:23:05.529242+0000 mgr.gibba032.whopeo (mgr.24625) 1636 : audit [DBG] from='client.55138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[30791]: cluster 2021-03-10T14:23:05.735130+0000 mgr.gibba032.whopeo (mgr.24625) 1637 : cluster [DBG] pgmap v919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[30791]: audit 2021-03-10T14:23:05.756336+0000 mgr.gibba032.whopeo (mgr.24625) 1638 : audit [DBG] from='client.45159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[30791]: audit 2021-03-10T14:23:06.023430+0000 mon.a (mon.0) 1035 : audit [DBG] from='client.? 172.21.2.111:0/1718925061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[31774]: audit 2021-03-10T14:23:05.529242+0000 mgr.gibba032.whopeo (mgr.24625) 1636 : audit [DBG] from='client.55138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[31774]: cluster 2021-03-10T14:23:05.735130+0000 mgr.gibba032.whopeo (mgr.24625) 1637 : cluster [DBG] pgmap v919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:07.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[31774]: audit 2021-03-10T14:23:05.756336+0000 mgr.gibba032.whopeo (mgr.24625) 1638 : audit [DBG] from='client.45159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:07.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:06 gibba011 bash[31774]: audit 2021-03-10T14:23:06.023430+0000 mon.a (mon.0) 1035 : audit [DBG] from='client.? 172.21.2.111:0/1718925061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:08 gibba011 bash[30791]: cephadm 2021-03-10T14:23:07.659477+0000 mgr.gibba032.whopeo (mgr.24625) 1639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:08 gibba011 bash[30791]: cluster 2021-03-10T14:23:07.736934+0000 mgr.gibba032.whopeo (mgr.24625) 1640 : cluster [DBG] pgmap v920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:08 gibba011 bash[31774]: cephadm 2021-03-10T14:23:07.659477+0000 mgr.gibba032.whopeo (mgr.24625) 1639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:08 gibba011 bash[31774]: cluster 2021-03-10T14:23:07.736934+0000 mgr.gibba032.whopeo (mgr.24625) 1640 : cluster [DBG] pgmap v920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:08 gibba032 bash[24004]: cephadm 2021-03-10T14:23:07.659477+0000 mgr.gibba032.whopeo (mgr.24625) 1639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:08 gibba032 bash[24004]: cluster 2021-03-10T14:23:07.736934+0000 mgr.gibba032.whopeo (mgr.24625) 1640 : cluster [DBG] pgmap v920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:10 gibba011 bash[30791]: cluster 2021-03-10T14:23:09.738170+0000 mgr.gibba032.whopeo (mgr.24625) 1641 : cluster [DBG] pgmap v921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:10 gibba011 bash[31774]: cluster 2021-03-10T14:23:09.738170+0000 mgr.gibba032.whopeo (mgr.24625) 1641 : cluster [DBG] pgmap v921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:10 gibba032 bash[24004]: cluster 2021-03-10T14:23:09.738170+0000 mgr.gibba032.whopeo (mgr.24625) 1641 : cluster [DBG] pgmap v921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:12 gibba011 bash[30791]: cluster 2021-03-10T14:23:11.739797+0000 mgr.gibba032.whopeo (mgr.24625) 1642 : cluster [DBG] pgmap v922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:12 gibba011 bash[31774]: cluster 2021-03-10T14:23:11.739797+0000 mgr.gibba032.whopeo (mgr.24625) 1642 : cluster [DBG] pgmap v922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:12 gibba032 bash[24004]: cluster 2021-03-10T14:23:11.739797+0000 mgr.gibba032.whopeo (mgr.24625) 1642 : cluster [DBG] pgmap v922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:14 gibba032 bash[24004]: cluster 2021-03-10T14:23:13.741360+0000 mgr.gibba032.whopeo (mgr.24625) 1643 : cluster [DBG] pgmap v923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:14 gibba011 bash[30791]: cluster 2021-03-10T14:23:13.741360+0000 mgr.gibba032.whopeo (mgr.24625) 1643 : cluster [DBG] pgmap v923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:14 gibba011 bash[31774]: cluster 2021-03-10T14:23:13.741360+0000 mgr.gibba032.whopeo (mgr.24625) 1643 : cluster [DBG] pgmap v923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:16 gibba032 bash[24004]: cluster 2021-03-10T14:23:15.741872+0000 mgr.gibba032.whopeo (mgr.24625) 1644 : cluster [DBG] pgmap v924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:16 gibba011 bash[30791]: cluster 2021-03-10T14:23:15.741872+0000 mgr.gibba032.whopeo (mgr.24625) 1644 : cluster [DBG] pgmap v924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:16 gibba011 bash[31774]: cluster 2021-03-10T14:23:15.741872+0000 mgr.gibba032.whopeo (mgr.24625) 1644 : cluster [DBG] pgmap v924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:18 gibba011 bash[30791]: cluster 2021-03-10T14:23:17.743380+0000 mgr.gibba032.whopeo (mgr.24625) 1645 : cluster [DBG] pgmap v925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:18 gibba011 bash[31774]: cluster 2021-03-10T14:23:17.743380+0000 mgr.gibba032.whopeo (mgr.24625) 1645 : cluster [DBG] pgmap v925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:18 gibba032 bash[24004]: cluster 2021-03-10T14:23:17.743380+0000 mgr.gibba032.whopeo (mgr.24625) 1645 : cluster [DBG] pgmap v925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:20 gibba011 bash[30791]: cluster 2021-03-10T14:23:19.744812+0000 mgr.gibba032.whopeo (mgr.24625) 1646 : cluster [DBG] pgmap v926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:20 gibba011 bash[31774]: cluster 2021-03-10T14:23:19.744812+0000 mgr.gibba032.whopeo (mgr.24625) 1646 : cluster [DBG] pgmap v926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:20 gibba032 bash[24004]: cluster 2021-03-10T14:23:19.744812+0000 mgr.gibba032.whopeo (mgr.24625) 1646 : cluster [DBG] pgmap v926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:22 gibba032 bash[24004]: cluster 2021-03-10T14:23:21.746325+0000 mgr.gibba032.whopeo (mgr.24625) 1647 : cluster [DBG] pgmap v927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:22 gibba011 bash[30791]: cluster 2021-03-10T14:23:21.746325+0000 mgr.gibba032.whopeo (mgr.24625) 1647 : cluster [DBG] pgmap v927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:22 gibba011 bash[31774]: cluster 2021-03-10T14:23:21.746325+0000 mgr.gibba032.whopeo (mgr.24625) 1647 : cluster [DBG] pgmap v927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:23 gibba032 bash[24004]: cephadm 2021-03-10T14:23:22.675495+0000 mgr.gibba032.whopeo (mgr.24625) 1648 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:23 gibba011 bash[30791]: cephadm 2021-03-10T14:23:22.675495+0000 mgr.gibba032.whopeo (mgr.24625) 1648 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:23 gibba011 bash[31774]: cephadm 2021-03-10T14:23:22.675495+0000 mgr.gibba032.whopeo (mgr.24625) 1648 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:24 gibba032 bash[24004]: cluster 2021-03-10T14:23:23.747904+0000 mgr.gibba032.whopeo (mgr.24625) 1649 : cluster [DBG] pgmap v928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:24 gibba011 bash[30791]: cluster 2021-03-10T14:23:23.747904+0000 mgr.gibba032.whopeo (mgr.24625) 1649 : cluster [DBG] pgmap v928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:24 gibba011 bash[31774]: cluster 2021-03-10T14:23:23.747904+0000 mgr.gibba032.whopeo (mgr.24625) 1649 : cluster [DBG] pgmap v928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:26 gibba032 bash[24004]: cluster 2021-03-10T14:23:25.748595+0000 mgr.gibba032.whopeo (mgr.24625) 1650 : cluster [DBG] pgmap v929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:26 gibba011 bash[30791]: cluster 2021-03-10T14:23:25.748595+0000 mgr.gibba032.whopeo (mgr.24625) 1650 : cluster [DBG] pgmap v929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:26 gibba011 bash[31774]: cluster 2021-03-10T14:23:25.748595+0000 mgr.gibba032.whopeo (mgr.24625) 1650 : cluster [DBG] pgmap v929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:28 gibba011 bash[30791]: cluster 2021-03-10T14:23:27.750226+0000 mgr.gibba032.whopeo (mgr.24625) 1651 : cluster [DBG] pgmap v930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:28 gibba011 bash[31774]: cluster 2021-03-10T14:23:27.750226+0000 mgr.gibba032.whopeo (mgr.24625) 1651 : cluster [DBG] pgmap v930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:28 gibba032 bash[24004]: cluster 2021-03-10T14:23:27.750226+0000 mgr.gibba032.whopeo (mgr.24625) 1651 : cluster [DBG] pgmap v930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:30 gibba011 bash[30791]: cluster 2021-03-10T14:23:29.751618+0000 mgr.gibba032.whopeo (mgr.24625) 1652 : cluster [DBG] pgmap v931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:30 gibba011 bash[31774]: cluster 2021-03-10T14:23:29.751618+0000 mgr.gibba032.whopeo (mgr.24625) 1652 : cluster [DBG] pgmap v931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:30 gibba032 bash[24004]: cluster 2021-03-10T14:23:29.751618+0000 mgr.gibba032.whopeo (mgr.24625) 1652 : cluster [DBG] pgmap v931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:32 gibba032 bash[24004]: cluster 2021-03-10T14:23:31.753333+0000 mgr.gibba032.whopeo (mgr.24625) 1653 : cluster [DBG] pgmap v932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:32 gibba011 bash[30791]: cluster 2021-03-10T14:23:31.753333+0000 mgr.gibba032.whopeo (mgr.24625) 1653 : cluster [DBG] pgmap v932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:32 gibba011 bash[31774]: cluster 2021-03-10T14:23:31.753333+0000 mgr.gibba032.whopeo (mgr.24625) 1653 : cluster [DBG] pgmap v932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:35.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:34 gibba032 bash[24004]: cluster 2021-03-10T14:23:33.754980+0000 mgr.gibba032.whopeo (mgr.24625) 1654 : cluster [DBG] pgmap v933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:34 gibba011 bash[30791]: cluster 2021-03-10T14:23:33.754980+0000 mgr.gibba032.whopeo (mgr.24625) 1654 : cluster [DBG] pgmap v933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:34 gibba011 bash[31774]: cluster 2021-03-10T14:23:33.754980+0000 mgr.gibba032.whopeo (mgr.24625) 1654 : cluster [DBG] pgmap v933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:36.268 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:23:36.496 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:23:36.496 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (30m) 6m ago 35m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:23:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (35m) 5m ago 35m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:23:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (30m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:23:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (30m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:23:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (29m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:23:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (29m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:23:36.498 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (29m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:23:36.498 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (29m) 6m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:23:36.498 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (35m) 6m ago 35m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:23:36.498 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (35m) 5m ago 35m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:23:36.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (28m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:23:36.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (28m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:23:36.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (27m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:23:36.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (27m) 6m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:23:36.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (26m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:23:36.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (26m) 5m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:23:36.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (25m) 5m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:23:36.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (25m) 5m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:23:36.500 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (35m) 5m ago 35m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:23:36.500 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 33m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:23:36.746 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:23:36.747 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:23:36.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:23:36.747 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:36.747 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:23:36.748 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:23:36.748 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:36.748 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:23:36.748 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:23:36.749 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:23:36.749 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:23:36.749 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:23:36.749 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:23:36.750 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:23:36.750 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:23:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:36 gibba032 bash[24004]: cluster 2021-03-10T14:23:35.755684+0000 mgr.gibba032.whopeo (mgr.24625) 1655 : cluster [DBG] pgmap v934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:36 gibba011 bash[30791]: cluster 2021-03-10T14:23:35.755684+0000 mgr.gibba032.whopeo (mgr.24625) 1655 : cluster [DBG] pgmap v934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:36 gibba011 bash[31774]: cluster 2021-03-10T14:23:35.755684+0000 mgr.gibba032.whopeo (mgr.24625) 1655 : cluster [DBG] pgmap v934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:37 gibba032 bash[24004]: audit 2021-03-10T14:23:36.255492+0000 mgr.gibba032.whopeo (mgr.24625) 1656 : audit [DBG] from='client.45171 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:37 gibba032 bash[24004]: audit 2021-03-10T14:23:36.480738+0000 mgr.gibba032.whopeo (mgr.24625) 1657 : audit [DBG] from='client.45177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:37 gibba032 bash[24004]: audit 2021-03-10T14:23:36.746235+0000 mon.a (mon.0) 1036 : audit [DBG] from='client.? 172.21.2.111:0/2942895285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[30791]: audit 2021-03-10T14:23:36.255492+0000 mgr.gibba032.whopeo (mgr.24625) 1656 : audit [DBG] from='client.45171 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[30791]: audit 2021-03-10T14:23:36.480738+0000 mgr.gibba032.whopeo (mgr.24625) 1657 : audit [DBG] from='client.45177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[30791]: audit 2021-03-10T14:23:36.746235+0000 mon.a (mon.0) 1036 : audit [DBG] from='client.? 172.21.2.111:0/2942895285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:38.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[31774]: audit 2021-03-10T14:23:36.255492+0000 mgr.gibba032.whopeo (mgr.24625) 1656 : audit [DBG] from='client.45171 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[31774]: audit 2021-03-10T14:23:36.480738+0000 mgr.gibba032.whopeo (mgr.24625) 1657 : audit [DBG] from='client.45177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:23:38.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:37 gibba011 bash[31774]: audit 2021-03-10T14:23:36.746235+0000 mon.a (mon.0) 1036 : audit [DBG] from='client.? 172.21.2.111:0/2942895285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.681909+0000 mgr.gibba032.whopeo (mgr.24625) 1658 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:23:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.683525+0000 mgr.gibba032.whopeo (mgr.24625) 1659 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:23:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cluster 2021-03-10T14:23:37.756174+0000 mgr.gibba032.whopeo (mgr.24625) 1660 : cluster [DBG] pgmap v935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.909896+0000 mgr.gibba032.whopeo (mgr.24625) 1661 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:23:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.910089+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:23:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.911822+0000 mgr.gibba032.whopeo (mgr.24625) 1662 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:23:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.913068+0000 mgr.gibba032.whopeo (mgr.24625) 1663 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:23:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.913105+0000 mgr.gibba032.whopeo (mgr.24625) 1664 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:23:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.913406+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:23:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.914699+0000 mgr.gibba032.whopeo (mgr.24625) 1665 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:23:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.914737+0000 mgr.gibba032.whopeo (mgr.24625) 1666 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:23:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.915026+0000 mon.a (mon.0) 1039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.915982+0000 mgr.gibba032.whopeo (mgr.24625) 1667 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:23:39.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.916514+0000 mon.a (mon.0) 1040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.917141+0000 mgr.gibba032.whopeo (mgr.24625) 1668 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:23:39.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.917177+0000 mgr.gibba032.whopeo (mgr.24625) 1669 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:23:39.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.917875+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.918516+0000 mgr.gibba032.whopeo (mgr.24625) 1670 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:23:39.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.918827+0000 mgr.gibba032.whopeo (mgr.24625) 1671 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:23:39.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.919796+0000 mgr.gibba032.whopeo (mgr.24625) 1672 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:23:39.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.919834+0000 mgr.gibba032.whopeo (mgr.24625) 1673 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:23:39.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: cephadm 2021-03-10T14:23:37.919934+0000 mgr.gibba032.whopeo (mgr.24625) 1674 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:39.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.920436+0000 mon.a (mon.0) 1042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:38 gibba032 bash[24004]: audit 2021-03-10T14:23:37.921843+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.681909+0000 mgr.gibba032.whopeo (mgr.24625) 1658 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:23:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.683525+0000 mgr.gibba032.whopeo (mgr.24625) 1659 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:23:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cluster 2021-03-10T14:23:37.756174+0000 mgr.gibba032.whopeo (mgr.24625) 1660 : cluster [DBG] pgmap v935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.909896+0000 mgr.gibba032.whopeo (mgr.24625) 1661 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:23:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.910089+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:23:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.911822+0000 mgr.gibba032.whopeo (mgr.24625) 1662 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:23:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.913068+0000 mgr.gibba032.whopeo (mgr.24625) 1663 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:23:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.913105+0000 mgr.gibba032.whopeo (mgr.24625) 1664 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:23:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.913406+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:23:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.914699+0000 mgr.gibba032.whopeo (mgr.24625) 1665 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:23:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.914737+0000 mgr.gibba032.whopeo (mgr.24625) 1666 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:23:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.915026+0000 mon.a (mon.0) 1039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.915982+0000 mgr.gibba032.whopeo (mgr.24625) 1667 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:23:39.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.916514+0000 mon.a (mon.0) 1040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.917141+0000 mgr.gibba032.whopeo (mgr.24625) 1668 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:23:39.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.917177+0000 mgr.gibba032.whopeo (mgr.24625) 1669 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:23:39.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.917875+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.918516+0000 mgr.gibba032.whopeo (mgr.24625) 1670 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:23:39.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.918827+0000 mgr.gibba032.whopeo (mgr.24625) 1671 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:23:39.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.919796+0000 mgr.gibba032.whopeo (mgr.24625) 1672 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:23:39.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.919834+0000 mgr.gibba032.whopeo (mgr.24625) 1673 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:23:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.681909+0000 mgr.gibba032.whopeo (mgr.24625) 1658 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:23:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.683525+0000 mgr.gibba032.whopeo (mgr.24625) 1659 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:23:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cluster 2021-03-10T14:23:37.756174+0000 mgr.gibba032.whopeo (mgr.24625) 1660 : cluster [DBG] pgmap v935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.909896+0000 mgr.gibba032.whopeo (mgr.24625) 1661 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:23:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.910089+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:23:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.911822+0000 mgr.gibba032.whopeo (mgr.24625) 1662 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:23:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.913068+0000 mgr.gibba032.whopeo (mgr.24625) 1663 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:23:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.913105+0000 mgr.gibba032.whopeo (mgr.24625) 1664 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:23:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.913406+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:23:39.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.914699+0000 mgr.gibba032.whopeo (mgr.24625) 1665 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:23:39.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.914737+0000 mgr.gibba032.whopeo (mgr.24625) 1666 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:23:39.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.915026+0000 mon.a (mon.0) 1039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.915982+0000 mgr.gibba032.whopeo (mgr.24625) 1667 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:23:39.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.916514+0000 mon.a (mon.0) 1040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.917141+0000 mgr.gibba032.whopeo (mgr.24625) 1668 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:23:39.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.917177+0000 mgr.gibba032.whopeo (mgr.24625) 1669 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:23:39.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.917875+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.918516+0000 mgr.gibba032.whopeo (mgr.24625) 1670 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:23:39.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.918827+0000 mgr.gibba032.whopeo (mgr.24625) 1671 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:23:39.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.919796+0000 mgr.gibba032.whopeo (mgr.24625) 1672 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:23:39.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.919834+0000 mgr.gibba032.whopeo (mgr.24625) 1673 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:23:39.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: cephadm 2021-03-10T14:23:37.919934+0000 mgr.gibba032.whopeo (mgr.24625) 1674 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:39.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.920436+0000 mon.a (mon.0) 1042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[30791]: audit 2021-03-10T14:23:37.921843+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: cephadm 2021-03-10T14:23:37.919934+0000 mgr.gibba032.whopeo (mgr.24625) 1674 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:39.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.920436+0000 mon.a (mon.0) 1042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:39.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:38 gibba011 bash[31774]: audit 2021-03-10T14:23:37.921843+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:23:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:40 gibba032 bash[24004]: cluster 2021-03-10T14:23:39.757321+0000 mgr.gibba032.whopeo (mgr.24625) 1675 : cluster [DBG] pgmap v936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:40 gibba011 bash[30791]: cluster 2021-03-10T14:23:39.757321+0000 mgr.gibba032.whopeo (mgr.24625) 1675 : cluster [DBG] pgmap v936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:40 gibba011 bash[31774]: cluster 2021-03-10T14:23:39.757321+0000 mgr.gibba032.whopeo (mgr.24625) 1675 : cluster [DBG] pgmap v936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:42 gibba032 bash[24004]: cluster 2021-03-10T14:23:41.758959+0000 mgr.gibba032.whopeo (mgr.24625) 1676 : cluster [DBG] pgmap v937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:42 gibba011 bash[30791]: cluster 2021-03-10T14:23:41.758959+0000 mgr.gibba032.whopeo (mgr.24625) 1676 : cluster [DBG] pgmap v937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:42 gibba011 bash[31774]: cluster 2021-03-10T14:23:41.758959+0000 mgr.gibba032.whopeo (mgr.24625) 1676 : cluster [DBG] pgmap v937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:45.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:44 gibba032 bash[24004]: cluster 2021-03-10T14:23:43.760567+0000 mgr.gibba032.whopeo (mgr.24625) 1677 : cluster [DBG] pgmap v938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:44 gibba011 bash[30791]: cluster 2021-03-10T14:23:43.760567+0000 mgr.gibba032.whopeo (mgr.24625) 1677 : cluster [DBG] pgmap v938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:44 gibba011 bash[31774]: cluster 2021-03-10T14:23:43.760567+0000 mgr.gibba032.whopeo (mgr.24625) 1677 : cluster [DBG] pgmap v938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:46 gibba032 bash[24004]: cluster 2021-03-10T14:23:45.761858+0000 mgr.gibba032.whopeo (mgr.24625) 1678 : cluster [DBG] pgmap v939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:46 gibba011 bash[30791]: cluster 2021-03-10T14:23:45.761858+0000 mgr.gibba032.whopeo (mgr.24625) 1678 : cluster [DBG] pgmap v939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:46 gibba011 bash[31774]: cluster 2021-03-10T14:23:45.761858+0000 mgr.gibba032.whopeo (mgr.24625) 1678 : cluster [DBG] pgmap v939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:48 gibba032 bash[24004]: cluster 2021-03-10T14:23:47.763636+0000 mgr.gibba032.whopeo (mgr.24625) 1679 : cluster [DBG] pgmap v940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:48 gibba011 bash[30791]: cluster 2021-03-10T14:23:47.763636+0000 mgr.gibba032.whopeo (mgr.24625) 1679 : cluster [DBG] pgmap v940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:48 gibba011 bash[31774]: cluster 2021-03-10T14:23:47.763636+0000 mgr.gibba032.whopeo (mgr.24625) 1679 : cluster [DBG] pgmap v940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:50 gibba032 bash[24004]: cluster 2021-03-10T14:23:49.765040+0000 mgr.gibba032.whopeo (mgr.24625) 1680 : cluster [DBG] pgmap v941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:50 gibba011 bash[30791]: cluster 2021-03-10T14:23:49.765040+0000 mgr.gibba032.whopeo (mgr.24625) 1680 : cluster [DBG] pgmap v941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:50 gibba011 bash[31774]: cluster 2021-03-10T14:23:49.765040+0000 mgr.gibba032.whopeo (mgr.24625) 1680 : cluster [DBG] pgmap v941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:53.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:52 gibba032 bash[24004]: cluster 2021-03-10T14:23:51.766751+0000 mgr.gibba032.whopeo (mgr.24625) 1681 : cluster [DBG] pgmap v942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:52 gibba011 bash[30791]: cluster 2021-03-10T14:23:51.766751+0000 mgr.gibba032.whopeo (mgr.24625) 1681 : cluster [DBG] pgmap v942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:52 gibba011 bash[31774]: cluster 2021-03-10T14:23:51.766751+0000 mgr.gibba032.whopeo (mgr.24625) 1681 : cluster [DBG] pgmap v942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:54.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:53 gibba032 bash[24004]: cephadm 2021-03-10T14:23:52.935853+0000 mgr.gibba032.whopeo (mgr.24625) 1682 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:53 gibba011 bash[30791]: cephadm 2021-03-10T14:23:52.935853+0000 mgr.gibba032.whopeo (mgr.24625) 1682 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:53 gibba011 bash[31774]: cephadm 2021-03-10T14:23:52.935853+0000 mgr.gibba032.whopeo (mgr.24625) 1682 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:23:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:54 gibba032 bash[24004]: cluster 2021-03-10T14:23:53.768393+0000 mgr.gibba032.whopeo (mgr.24625) 1683 : cluster [DBG] pgmap v943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:54 gibba011 bash[30791]: cluster 2021-03-10T14:23:53.768393+0000 mgr.gibba032.whopeo (mgr.24625) 1683 : cluster [DBG] pgmap v943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:55.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:54 gibba011 bash[31774]: cluster 2021-03-10T14:23:53.768393+0000 mgr.gibba032.whopeo (mgr.24625) 1683 : cluster [DBG] pgmap v943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: cluster 2021-03-10T14:23:55.769234+0000 mgr.gibba032.whopeo (mgr.24625) 1684 : cluster [DBG] pgmap v944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:57.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.154870+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:23:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.155683+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.156393+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.157080+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T14:23:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.199198+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:23:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:56 gibba032 bash[24004]: audit 2021-03-10T14:23:56.199951+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:23:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: cluster 2021-03-10T14:23:55.769234+0000 mgr.gibba032.whopeo (mgr.24625) 1684 : cluster [DBG] pgmap v944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.154870+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:23:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.155683+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.156393+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.157080+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T14:23:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.199198+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:23:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[30791]: audit 2021-03-10T14:23:56.199951+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:23:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: cluster 2021-03-10T14:23:55.769234+0000 mgr.gibba032.whopeo (mgr.24625) 1684 : cluster [DBG] pgmap v944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.154870+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:23:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.155683+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.156393+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:23:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.157080+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T14:23:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.199198+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:23:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:56 gibba011 bash[31774]: audit 2021-03-10T14:23:56.199951+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:23:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:23:58 gibba032 bash[24004]: cluster 2021-03-10T14:23:57.770902+0000 mgr.gibba032.whopeo (mgr.24625) 1685 : cluster [DBG] pgmap v945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:23:58 gibba011 bash[30791]: cluster 2021-03-10T14:23:57.770902+0000 mgr.gibba032.whopeo (mgr.24625) 1685 : cluster [DBG] pgmap v945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:23:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:23:58 gibba011 bash[31774]: cluster 2021-03-10T14:23:57.770902+0000 mgr.gibba032.whopeo (mgr.24625) 1685 : cluster [DBG] pgmap v945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:00 gibba032 bash[24004]: cluster 2021-03-10T14:23:59.772426+0000 mgr.gibba032.whopeo (mgr.24625) 1686 : cluster [DBG] pgmap v946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:00 gibba011 bash[30791]: cluster 2021-03-10T14:23:59.772426+0000 mgr.gibba032.whopeo (mgr.24625) 1686 : cluster [DBG] pgmap v946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:00 gibba011 bash[31774]: cluster 2021-03-10T14:23:59.772426+0000 mgr.gibba032.whopeo (mgr.24625) 1686 : cluster [DBG] pgmap v946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:02 gibba032 bash[24004]: cluster 2021-03-10T14:24:01.774190+0000 mgr.gibba032.whopeo (mgr.24625) 1687 : cluster [DBG] pgmap v947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:02 gibba011 bash[30791]: cluster 2021-03-10T14:24:01.774190+0000 mgr.gibba032.whopeo (mgr.24625) 1687 : cluster [DBG] pgmap v947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:02 gibba011 bash[31774]: cluster 2021-03-10T14:24:01.774190+0000 mgr.gibba032.whopeo (mgr.24625) 1687 : cluster [DBG] pgmap v947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:05.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:04 gibba032 bash[24004]: cluster 2021-03-10T14:24:03.775674+0000 mgr.gibba032.whopeo (mgr.24625) 1688 : cluster [DBG] pgmap v948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:04 gibba011 bash[30791]: cluster 2021-03-10T14:24:03.775674+0000 mgr.gibba032.whopeo (mgr.24625) 1688 : cluster [DBG] pgmap v948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:04 gibba011 bash[31774]: cluster 2021-03-10T14:24:03.775674+0000 mgr.gibba032.whopeo (mgr.24625) 1688 : cluster [DBG] pgmap v948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:06.990 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:24:07.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:06 gibba032 bash[24004]: cluster 2021-03-10T14:24:05.777148+0000 mgr.gibba032.whopeo (mgr.24625) 1689 : cluster [DBG] pgmap v949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:06 gibba011 bash[30791]: cluster 2021-03-10T14:24:05.777148+0000 mgr.gibba032.whopeo (mgr.24625) 1689 : cluster [DBG] pgmap v949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:06 gibba011 bash[31774]: cluster 2021-03-10T14:24:05.777148+0000 mgr.gibba032.whopeo (mgr.24625) 1689 : cluster [DBG] pgmap v949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:07.219 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:24:07.219 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (30m) 6m ago 36m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:24:07.219 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (35m) 5m ago 35m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:24:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (31m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:24:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (31m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:24:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (30m) 6m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:24:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (30m) 6m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:24:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (29m) 5m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:24:07.221 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (30m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:24:07.221 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (36m) 6m ago 36m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:24:07.221 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (36m) 5m ago 36m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:24:07.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (29m) 6m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:24:07.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (28m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:24:07.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (28m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:24:07.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (27m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:24:07.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (27m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:24:07.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (26m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:24:07.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (26m) 5m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:24:07.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (26m) 5m ago 36m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:24:07.223 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (35m) 5m ago 36m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:24:07.223 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 34m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:24:07.468 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:24:07.468 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:24:07.469 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:24:07.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:07.469 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:24:07.469 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:24:07.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:07.470 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:24:07.470 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:24:07.470 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:07.470 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:24:07.470 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:24:07.471 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:24:07.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:24:07.471 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:24:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:07 gibba032 bash[24004]: audit 2021-03-10T14:24:06.978230+0000 mgr.gibba032.whopeo (mgr.24625) 1690 : audit [DBG] from='client.45189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:07 gibba032 bash[24004]: audit 2021-03-10T14:24:07.468051+0000 mon.a (mon.0) 1050 : audit [DBG] from='client.? 172.21.2.111:0/465065192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:08.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:07 gibba011 bash[30791]: audit 2021-03-10T14:24:06.978230+0000 mgr.gibba032.whopeo (mgr.24625) 1690 : audit [DBG] from='client.45189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:08.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:07 gibba011 bash[30791]: audit 2021-03-10T14:24:07.468051+0000 mon.a (mon.0) 1050 : audit [DBG] from='client.? 172.21.2.111:0/465065192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:08.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:07 gibba011 bash[31774]: audit 2021-03-10T14:24:06.978230+0000 mgr.gibba032.whopeo (mgr.24625) 1690 : audit [DBG] from='client.45189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:08.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:07 gibba011 bash[31774]: audit 2021-03-10T14:24:07.468051+0000 mon.a (mon.0) 1050 : audit [DBG] from='client.? 172.21.2.111:0/465065192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:08 gibba032 bash[24004]: audit 2021-03-10T14:24:07.204014+0000 mgr.gibba032.whopeo (mgr.24625) 1691 : audit [DBG] from='client.55180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:08 gibba032 bash[24004]: cluster 2021-03-10T14:24:07.778980+0000 mgr.gibba032.whopeo (mgr.24625) 1692 : cluster [DBG] pgmap v950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:09.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:08 gibba032 bash[24004]: cephadm 2021-03-10T14:24:07.951918+0000 mgr.gibba032.whopeo (mgr.24625) 1693 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[30791]: audit 2021-03-10T14:24:07.204014+0000 mgr.gibba032.whopeo (mgr.24625) 1691 : audit [DBG] from='client.55180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[30791]: cluster 2021-03-10T14:24:07.778980+0000 mgr.gibba032.whopeo (mgr.24625) 1692 : cluster [DBG] pgmap v950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[30791]: cephadm 2021-03-10T14:24:07.951918+0000 mgr.gibba032.whopeo (mgr.24625) 1693 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[31774]: audit 2021-03-10T14:24:07.204014+0000 mgr.gibba032.whopeo (mgr.24625) 1691 : audit [DBG] from='client.55180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[31774]: cluster 2021-03-10T14:24:07.778980+0000 mgr.gibba032.whopeo (mgr.24625) 1692 : cluster [DBG] pgmap v950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:09.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:08 gibba011 bash[31774]: cephadm 2021-03-10T14:24:07.951918+0000 mgr.gibba032.whopeo (mgr.24625) 1693 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:10 gibba032 bash[24004]: cluster 2021-03-10T14:24:09.780525+0000 mgr.gibba032.whopeo (mgr.24625) 1694 : cluster [DBG] pgmap v951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:10 gibba011 bash[30791]: cluster 2021-03-10T14:24:09.780525+0000 mgr.gibba032.whopeo (mgr.24625) 1694 : cluster [DBG] pgmap v951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:10 gibba011 bash[31774]: cluster 2021-03-10T14:24:09.780525+0000 mgr.gibba032.whopeo (mgr.24625) 1694 : cluster [DBG] pgmap v951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:12 gibba032 bash[24004]: cluster 2021-03-10T14:24:11.782026+0000 mgr.gibba032.whopeo (mgr.24625) 1695 : cluster [DBG] pgmap v952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:12 gibba011 bash[30791]: cluster 2021-03-10T14:24:11.782026+0000 mgr.gibba032.whopeo (mgr.24625) 1695 : cluster [DBG] pgmap v952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:12 gibba011 bash[31774]: cluster 2021-03-10T14:24:11.782026+0000 mgr.gibba032.whopeo (mgr.24625) 1695 : cluster [DBG] pgmap v952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:14 gibba032 bash[24004]: cluster 2021-03-10T14:24:13.783627+0000 mgr.gibba032.whopeo (mgr.24625) 1696 : cluster [DBG] pgmap v953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:14 gibba011 bash[30791]: cluster 2021-03-10T14:24:13.783627+0000 mgr.gibba032.whopeo (mgr.24625) 1696 : cluster [DBG] pgmap v953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:14 gibba011 bash[31774]: cluster 2021-03-10T14:24:13.783627+0000 mgr.gibba032.whopeo (mgr.24625) 1696 : cluster [DBG] pgmap v953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:16 gibba032 bash[24004]: cluster 2021-03-10T14:24:15.784939+0000 mgr.gibba032.whopeo (mgr.24625) 1697 : cluster [DBG] pgmap v954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:16 gibba011 bash[30791]: cluster 2021-03-10T14:24:15.784939+0000 mgr.gibba032.whopeo (mgr.24625) 1697 : cluster [DBG] pgmap v954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:16 gibba011 bash[31774]: cluster 2021-03-10T14:24:15.784939+0000 mgr.gibba032.whopeo (mgr.24625) 1697 : cluster [DBG] pgmap v954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:18 gibba032 bash[24004]: cluster 2021-03-10T14:24:17.786572+0000 mgr.gibba032.whopeo (mgr.24625) 1698 : cluster [DBG] pgmap v955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:18 gibba011 bash[30791]: cluster 2021-03-10T14:24:17.786572+0000 mgr.gibba032.whopeo (mgr.24625) 1698 : cluster [DBG] pgmap v955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:18 gibba011 bash[31774]: cluster 2021-03-10T14:24:17.786572+0000 mgr.gibba032.whopeo (mgr.24625) 1698 : cluster [DBG] pgmap v955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:20 gibba032 bash[24004]: cluster 2021-03-10T14:24:19.788116+0000 mgr.gibba032.whopeo (mgr.24625) 1699 : cluster [DBG] pgmap v956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:20 gibba011 bash[30791]: cluster 2021-03-10T14:24:19.788116+0000 mgr.gibba032.whopeo (mgr.24625) 1699 : cluster [DBG] pgmap v956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:20 gibba011 bash[31774]: cluster 2021-03-10T14:24:19.788116+0000 mgr.gibba032.whopeo (mgr.24625) 1699 : cluster [DBG] pgmap v956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:22 gibba032 bash[24004]: cluster 2021-03-10T14:24:21.789894+0000 mgr.gibba032.whopeo (mgr.24625) 1700 : cluster [DBG] pgmap v957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:22 gibba011 bash[30791]: cluster 2021-03-10T14:24:21.789894+0000 mgr.gibba032.whopeo (mgr.24625) 1700 : cluster [DBG] pgmap v957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:22 gibba011 bash[31774]: cluster 2021-03-10T14:24:21.789894+0000 mgr.gibba032.whopeo (mgr.24625) 1700 : cluster [DBG] pgmap v957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:23 gibba032 bash[24004]: cephadm 2021-03-10T14:24:22.965076+0000 mgr.gibba032.whopeo (mgr.24625) 1701 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:23 gibba011 bash[30791]: cephadm 2021-03-10T14:24:22.965076+0000 mgr.gibba032.whopeo (mgr.24625) 1701 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:23 gibba011 bash[31774]: cephadm 2021-03-10T14:24:22.965076+0000 mgr.gibba032.whopeo (mgr.24625) 1701 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:24 gibba032 bash[24004]: cluster 2021-03-10T14:24:23.791519+0000 mgr.gibba032.whopeo (mgr.24625) 1702 : cluster [DBG] pgmap v958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:24 gibba011 bash[30791]: cluster 2021-03-10T14:24:23.791519+0000 mgr.gibba032.whopeo (mgr.24625) 1702 : cluster [DBG] pgmap v958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:24 gibba011 bash[31774]: cluster 2021-03-10T14:24:23.791519+0000 mgr.gibba032.whopeo (mgr.24625) 1702 : cluster [DBG] pgmap v958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:26 gibba032 bash[24004]: cluster 2021-03-10T14:24:25.792968+0000 mgr.gibba032.whopeo (mgr.24625) 1703 : cluster [DBG] pgmap v959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:26 gibba011 bash[30791]: cluster 2021-03-10T14:24:25.792968+0000 mgr.gibba032.whopeo (mgr.24625) 1703 : cluster [DBG] pgmap v959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:26 gibba011 bash[31774]: cluster 2021-03-10T14:24:25.792968+0000 mgr.gibba032.whopeo (mgr.24625) 1703 : cluster [DBG] pgmap v959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:28 gibba032 bash[24004]: cluster 2021-03-10T14:24:27.794774+0000 mgr.gibba032.whopeo (mgr.24625) 1704 : cluster [DBG] pgmap v960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:28 gibba011 bash[30791]: cluster 2021-03-10T14:24:27.794774+0000 mgr.gibba032.whopeo (mgr.24625) 1704 : cluster [DBG] pgmap v960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:28 gibba011 bash[31774]: cluster 2021-03-10T14:24:27.794774+0000 mgr.gibba032.whopeo (mgr.24625) 1704 : cluster [DBG] pgmap v960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:30 gibba032 bash[24004]: cluster 2021-03-10T14:24:29.796080+0000 mgr.gibba032.whopeo (mgr.24625) 1705 : cluster [DBG] pgmap v961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:30 gibba011 bash[30791]: cluster 2021-03-10T14:24:29.796080+0000 mgr.gibba032.whopeo (mgr.24625) 1705 : cluster [DBG] pgmap v961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:30 gibba011 bash[31774]: cluster 2021-03-10T14:24:29.796080+0000 mgr.gibba032.whopeo (mgr.24625) 1705 : cluster [DBG] pgmap v961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:32 gibba032 bash[24004]: cluster 2021-03-10T14:24:31.797912+0000 mgr.gibba032.whopeo (mgr.24625) 1706 : cluster [DBG] pgmap v962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:32 gibba011 bash[30791]: cluster 2021-03-10T14:24:31.797912+0000 mgr.gibba032.whopeo (mgr.24625) 1706 : cluster [DBG] pgmap v962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:32 gibba011 bash[31774]: cluster 2021-03-10T14:24:31.797912+0000 mgr.gibba032.whopeo (mgr.24625) 1706 : cluster [DBG] pgmap v962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:35.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:34 gibba032 bash[24004]: cluster 2021-03-10T14:24:33.799534+0000 mgr.gibba032.whopeo (mgr.24625) 1707 : cluster [DBG] pgmap v963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:34 gibba011 bash[30791]: cluster 2021-03-10T14:24:33.799534+0000 mgr.gibba032.whopeo (mgr.24625) 1707 : cluster [DBG] pgmap v963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:34 gibba011 bash[31774]: cluster 2021-03-10T14:24:33.799534+0000 mgr.gibba032.whopeo (mgr.24625) 1707 : cluster [DBG] pgmap v963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:36 gibba032 bash[24004]: cluster 2021-03-10T14:24:35.800322+0000 mgr.gibba032.whopeo (mgr.24625) 1708 : cluster [DBG] pgmap v964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:36 gibba011 bash[30791]: cluster 2021-03-10T14:24:35.800322+0000 mgr.gibba032.whopeo (mgr.24625) 1708 : cluster [DBG] pgmap v964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:36 gibba011 bash[31774]: cluster 2021-03-10T14:24:35.800322+0000 mgr.gibba032.whopeo (mgr.24625) 1708 : cluster [DBG] pgmap v964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:37.709 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:24:37.935 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:24:37.935 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (31m) 7m ago 36m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:24:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (36m) 6m ago 36m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:24:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (31m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:24:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (31m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:24:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (30m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:24:37.936 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (30m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:24:37.937 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (30m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:24:37.937 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (30m) 7m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:24:37.937 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (36m) 7m ago 36m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:24:37.937 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (36m) 6m ago 36m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:24:37.937 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (29m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:24:37.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (29m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:24:37.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (28m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:24:37.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (28m) 7m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:24:37.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (27m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:24:37.939 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (27m) 6m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:24:37.939 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (27m) 6m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:24:37.939 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (26m) 6m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:24:37.939 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (36m) 6m ago 36m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:24:37.939 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 34m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:24:38.200 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:24:38.200 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:24:38.201 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:24:38.201 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:38.201 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:24:38.201 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:24:38.202 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:38.202 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:24:38.202 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:24:38.202 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:24:38.202 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:24:38.203 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:24:38.203 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:24:38.203 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:24:38.203 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:24:39.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:37.697614+0000 mgr.gibba032.whopeo (mgr.24625) 1709 : audit [DBG] from='client.55192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: cluster 2021-03-10T14:24:37.801819+0000 mgr.gibba032.whopeo (mgr.24625) 1710 : cluster [DBG] pgmap v965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:37.920874+0000 mgr.gibba032.whopeo (mgr.24625) 1711 : audit [DBG] from='client.45213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.171307+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.176407+0000 mon.a (mon.0) 1052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.200344+0000 mon.c (mon.1) 33 : audit [DBG] from='client.? 172.21.2.111:0/314987055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.373161+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.376724+0000 mon.a (mon.0) 1054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:24:39.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.378465+0000 mon.a (mon.0) 1055 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.380066+0000 mon.a (mon.0) 1056 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.381731+0000 mon.a (mon.0) 1057 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.384761+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:38 gibba032 bash[24004]: audit 2021-03-10T14:24:38.386397+0000 mon.a (mon.0) 1059 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:37.697614+0000 mgr.gibba032.whopeo (mgr.24625) 1709 : audit [DBG] from='client.55192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: cluster 2021-03-10T14:24:37.801819+0000 mgr.gibba032.whopeo (mgr.24625) 1710 : cluster [DBG] pgmap v965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:37.920874+0000 mgr.gibba032.whopeo (mgr.24625) 1711 : audit [DBG] from='client.45213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.171307+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.176407+0000 mon.a (mon.0) 1052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.200344+0000 mon.c (mon.1) 33 : audit [DBG] from='client.? 172.21.2.111:0/314987055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.373161+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.376724+0000 mon.a (mon.0) 1054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:24:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.378465+0000 mon.a (mon.0) 1055 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.380066+0000 mon.a (mon.0) 1056 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.381731+0000 mon.a (mon.0) 1057 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.384761+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[30791]: audit 2021-03-10T14:24:38.386397+0000 mon.a (mon.0) 1059 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:37.697614+0000 mgr.gibba032.whopeo (mgr.24625) 1709 : audit [DBG] from='client.55192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: cluster 2021-03-10T14:24:37.801819+0000 mgr.gibba032.whopeo (mgr.24625) 1710 : cluster [DBG] pgmap v965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:37.920874+0000 mgr.gibba032.whopeo (mgr.24625) 1711 : audit [DBG] from='client.45213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:24:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.171307+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.176407+0000 mon.a (mon.0) 1052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.200344+0000 mon.c (mon.1) 33 : audit [DBG] from='client.? 172.21.2.111:0/314987055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.373161+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:24:39.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.376724+0000 mon.a (mon.0) 1054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:24:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.378465+0000 mon.a (mon.0) 1055 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.380066+0000 mon.a (mon.0) 1056 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.381731+0000 mon.a (mon.0) 1057 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.384761+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:39.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:38 gibba011 bash[31774]: audit 2021-03-10T14:24:38.386397+0000 mon.a (mon.0) 1059 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:24:40.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.169030+0000 mgr.gibba032.whopeo (mgr.24625) 1712 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.174072+0000 mgr.gibba032.whopeo (mgr.24625) 1713 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.372909+0000 mgr.gibba032.whopeo (mgr.24625) 1714 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.375094+0000 mgr.gibba032.whopeo (mgr.24625) 1715 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.376504+0000 mgr.gibba032.whopeo (mgr.24625) 1716 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.376589+0000 mgr.gibba032.whopeo (mgr.24625) 1717 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:24:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.378288+0000 mgr.gibba032.whopeo (mgr.24625) 1718 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:24:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.378353+0000 mgr.gibba032.whopeo (mgr.24625) 1719 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:24:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.379931+0000 mgr.gibba032.whopeo (mgr.24625) 1720 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:24:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.381292+0000 mgr.gibba032.whopeo (mgr.24625) 1721 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:24:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.381324+0000 mgr.gibba032.whopeo (mgr.24625) 1722 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:24:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.382812+0000 mgr.gibba032.whopeo (mgr.24625) 1723 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:24:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.383123+0000 mgr.gibba032.whopeo (mgr.24625) 1724 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:24:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.384464+0000 mgr.gibba032.whopeo (mgr.24625) 1725 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:24:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.384503+0000 mgr.gibba032.whopeo (mgr.24625) 1726 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:24:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:39 gibba032 bash[24004]: cephadm 2021-03-10T14:24:38.384606+0000 mgr.gibba032.whopeo (mgr.24625) 1727 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.169030+0000 mgr.gibba032.whopeo (mgr.24625) 1712 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:24:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.174072+0000 mgr.gibba032.whopeo (mgr.24625) 1713 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:24:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.372909+0000 mgr.gibba032.whopeo (mgr.24625) 1714 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:24:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.375094+0000 mgr.gibba032.whopeo (mgr.24625) 1715 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:24:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.376504+0000 mgr.gibba032.whopeo (mgr.24625) 1716 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:24:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.376589+0000 mgr.gibba032.whopeo (mgr.24625) 1717 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:24:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.378288+0000 mgr.gibba032.whopeo (mgr.24625) 1718 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.378353+0000 mgr.gibba032.whopeo (mgr.24625) 1719 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.379931+0000 mgr.gibba032.whopeo (mgr.24625) 1720 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.381292+0000 mgr.gibba032.whopeo (mgr.24625) 1721 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.381324+0000 mgr.gibba032.whopeo (mgr.24625) 1722 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.382812+0000 mgr.gibba032.whopeo (mgr.24625) 1723 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:24:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.383123+0000 mgr.gibba032.whopeo (mgr.24625) 1724 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:24:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.384464+0000 mgr.gibba032.whopeo (mgr.24625) 1725 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:24:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.384503+0000 mgr.gibba032.whopeo (mgr.24625) 1726 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:24:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[31774]: cephadm 2021-03-10T14:24:38.384606+0000 mgr.gibba032.whopeo (mgr.24625) 1727 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.169030+0000 mgr.gibba032.whopeo (mgr.24625) 1712 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:24:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.174072+0000 mgr.gibba032.whopeo (mgr.24625) 1713 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:24:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.372909+0000 mgr.gibba032.whopeo (mgr.24625) 1714 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:24:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.375094+0000 mgr.gibba032.whopeo (mgr.24625) 1715 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:24:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.376504+0000 mgr.gibba032.whopeo (mgr.24625) 1716 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:24:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.376589+0000 mgr.gibba032.whopeo (mgr.24625) 1717 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:24:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.378288+0000 mgr.gibba032.whopeo (mgr.24625) 1718 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:24:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.378353+0000 mgr.gibba032.whopeo (mgr.24625) 1719 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:24:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.379931+0000 mgr.gibba032.whopeo (mgr.24625) 1720 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:24:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.381292+0000 mgr.gibba032.whopeo (mgr.24625) 1721 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:24:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.381324+0000 mgr.gibba032.whopeo (mgr.24625) 1722 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:24:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.382812+0000 mgr.gibba032.whopeo (mgr.24625) 1723 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:24:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.383123+0000 mgr.gibba032.whopeo (mgr.24625) 1724 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:24:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.384464+0000 mgr.gibba032.whopeo (mgr.24625) 1725 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:24:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.384503+0000 mgr.gibba032.whopeo (mgr.24625) 1726 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:24:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:39 gibba011 bash[30791]: cephadm 2021-03-10T14:24:38.384606+0000 mgr.gibba032.whopeo (mgr.24625) 1727 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:40 gibba032 bash[24004]: cluster 2021-03-10T14:24:39.803319+0000 mgr.gibba032.whopeo (mgr.24625) 1728 : cluster [DBG] pgmap v966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:40 gibba011 bash[30791]: cluster 2021-03-10T14:24:39.803319+0000 mgr.gibba032.whopeo (mgr.24625) 1728 : cluster [DBG] pgmap v966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:40 gibba011 bash[31774]: cluster 2021-03-10T14:24:39.803319+0000 mgr.gibba032.whopeo (mgr.24625) 1728 : cluster [DBG] pgmap v966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:42 gibba032 bash[24004]: cluster 2021-03-10T14:24:41.805144+0000 mgr.gibba032.whopeo (mgr.24625) 1729 : cluster [DBG] pgmap v967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:42 gibba011 bash[30791]: cluster 2021-03-10T14:24:41.805144+0000 mgr.gibba032.whopeo (mgr.24625) 1729 : cluster [DBG] pgmap v967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:42 gibba011 bash[31774]: cluster 2021-03-10T14:24:41.805144+0000 mgr.gibba032.whopeo (mgr.24625) 1729 : cluster [DBG] pgmap v967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:45.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:44 gibba032 bash[24004]: cluster 2021-03-10T14:24:43.806751+0000 mgr.gibba032.whopeo (mgr.24625) 1730 : cluster [DBG] pgmap v968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:44 gibba011 bash[30791]: cluster 2021-03-10T14:24:43.806751+0000 mgr.gibba032.whopeo (mgr.24625) 1730 : cluster [DBG] pgmap v968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:44 gibba011 bash[31774]: cluster 2021-03-10T14:24:43.806751+0000 mgr.gibba032.whopeo (mgr.24625) 1730 : cluster [DBG] pgmap v968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:46 gibba011 bash[30791]: cluster 2021-03-10T14:24:45.807907+0000 mgr.gibba032.whopeo (mgr.24625) 1731 : cluster [DBG] pgmap v969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:46 gibba011 bash[31774]: cluster 2021-03-10T14:24:45.807907+0000 mgr.gibba032.whopeo (mgr.24625) 1731 : cluster [DBG] pgmap v969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:46 gibba032 bash[24004]: cluster 2021-03-10T14:24:45.807907+0000 mgr.gibba032.whopeo (mgr.24625) 1731 : cluster [DBG] pgmap v969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:48 gibba032 bash[24004]: cluster 2021-03-10T14:24:47.809653+0000 mgr.gibba032.whopeo (mgr.24625) 1732 : cluster [DBG] pgmap v970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:48 gibba011 bash[30791]: cluster 2021-03-10T14:24:47.809653+0000 mgr.gibba032.whopeo (mgr.24625) 1732 : cluster [DBG] pgmap v970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:48 gibba011 bash[31774]: cluster 2021-03-10T14:24:47.809653+0000 mgr.gibba032.whopeo (mgr.24625) 1732 : cluster [DBG] pgmap v970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:50 gibba032 bash[24004]: cluster 2021-03-10T14:24:49.811137+0000 mgr.gibba032.whopeo (mgr.24625) 1733 : cluster [DBG] pgmap v971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:50 gibba011 bash[30791]: cluster 2021-03-10T14:24:49.811137+0000 mgr.gibba032.whopeo (mgr.24625) 1733 : cluster [DBG] pgmap v971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:50 gibba011 bash[31774]: cluster 2021-03-10T14:24:49.811137+0000 mgr.gibba032.whopeo (mgr.24625) 1733 : cluster [DBG] pgmap v971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:53.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:52 gibba032 bash[24004]: cluster 2021-03-10T14:24:51.812927+0000 mgr.gibba032.whopeo (mgr.24625) 1734 : cluster [DBG] pgmap v972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:52 gibba011 bash[30791]: cluster 2021-03-10T14:24:51.812927+0000 mgr.gibba032.whopeo (mgr.24625) 1734 : cluster [DBG] pgmap v972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:52 gibba011 bash[31774]: cluster 2021-03-10T14:24:51.812927+0000 mgr.gibba032.whopeo (mgr.24625) 1734 : cluster [DBG] pgmap v972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:54 gibba032 bash[24004]: cephadm 2021-03-10T14:24:53.400676+0000 mgr.gibba032.whopeo (mgr.24625) 1735 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:54 gibba032 bash[24004]: cluster 2021-03-10T14:24:53.814307+0000 mgr.gibba032.whopeo (mgr.24625) 1736 : cluster [DBG] pgmap v973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:54 gibba011 bash[30791]: cephadm 2021-03-10T14:24:53.400676+0000 mgr.gibba032.whopeo (mgr.24625) 1735 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:54 gibba011 bash[30791]: cluster 2021-03-10T14:24:53.814307+0000 mgr.gibba032.whopeo (mgr.24625) 1736 : cluster [DBG] pgmap v973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:55.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:54 gibba011 bash[31774]: cephadm 2021-03-10T14:24:53.400676+0000 mgr.gibba032.whopeo (mgr.24625) 1735 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:24:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:54 gibba011 bash[31774]: cluster 2021-03-10T14:24:53.814307+0000 mgr.gibba032.whopeo (mgr.24625) 1736 : cluster [DBG] pgmap v973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: cluster 2021-03-10T14:24:55.815559+0000 mgr.gibba032.whopeo (mgr.24625) 1737 : cluster [DBG] pgmap v974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.163702+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T14:24:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.164484+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.165154+0000 mon.a (mon.0) 1062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:24:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.165821+0000 mon.a (mon.0) 1063 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.206803+0000 mon.a (mon.0) 1064 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:24:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[30791]: audit 2021-03-10T14:24:56.207500+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:24:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: cluster 2021-03-10T14:24:55.815559+0000 mgr.gibba032.whopeo (mgr.24625) 1737 : cluster [DBG] pgmap v974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.163702+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T14:24:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.164484+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.165154+0000 mon.a (mon.0) 1062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:24:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.165821+0000 mon.a (mon.0) 1063 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.206803+0000 mon.a (mon.0) 1064 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:24:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:56 gibba011 bash[31774]: audit 2021-03-10T14:24:56.207500+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:24:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: cluster 2021-03-10T14:24:55.815559+0000 mgr.gibba032.whopeo (mgr.24625) 1737 : cluster [DBG] pgmap v974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.163702+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T14:24:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.164484+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.165154+0000 mon.a (mon.0) 1062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:24:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.165821+0000 mon.a (mon.0) 1063 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:24:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.206803+0000 mon.a (mon.0) 1064 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:24:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:56 gibba032 bash[24004]: audit 2021-03-10T14:24:56.207500+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:24:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:24:58 gibba032 bash[24004]: cluster 2021-03-10T14:24:57.817299+0000 mgr.gibba032.whopeo (mgr.24625) 1738 : cluster [DBG] pgmap v975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:24:58 gibba011 bash[30791]: cluster 2021-03-10T14:24:57.817299+0000 mgr.gibba032.whopeo (mgr.24625) 1738 : cluster [DBG] pgmap v975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:24:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:24:58 gibba011 bash[31774]: cluster 2021-03-10T14:24:57.817299+0000 mgr.gibba032.whopeo (mgr.24625) 1738 : cluster [DBG] pgmap v975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:00 gibba032 bash[24004]: cluster 2021-03-10T14:24:59.818563+0000 mgr.gibba032.whopeo (mgr.24625) 1739 : cluster [DBG] pgmap v976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:00 gibba011 bash[30791]: cluster 2021-03-10T14:24:59.818563+0000 mgr.gibba032.whopeo (mgr.24625) 1739 : cluster [DBG] pgmap v976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:00 gibba011 bash[31774]: cluster 2021-03-10T14:24:59.818563+0000 mgr.gibba032.whopeo (mgr.24625) 1739 : cluster [DBG] pgmap v976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:02 gibba032 bash[24004]: cluster 2021-03-10T14:25:01.820220+0000 mgr.gibba032.whopeo (mgr.24625) 1740 : cluster [DBG] pgmap v977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:02 gibba011 bash[30791]: cluster 2021-03-10T14:25:01.820220+0000 mgr.gibba032.whopeo (mgr.24625) 1740 : cluster [DBG] pgmap v977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:02 gibba011 bash[31774]: cluster 2021-03-10T14:25:01.820220+0000 mgr.gibba032.whopeo (mgr.24625) 1740 : cluster [DBG] pgmap v977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:04 gibba032 bash[24004]: cluster 2021-03-10T14:25:03.821798+0000 mgr.gibba032.whopeo (mgr.24625) 1741 : cluster [DBG] pgmap v978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:04 gibba011 bash[30791]: cluster 2021-03-10T14:25:03.821798+0000 mgr.gibba032.whopeo (mgr.24625) 1741 : cluster [DBG] pgmap v978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:04 gibba011 bash[31774]: cluster 2021-03-10T14:25:03.821798+0000 mgr.gibba032.whopeo (mgr.24625) 1741 : cluster [DBG] pgmap v978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:06 gibba011 bash[30791]: cluster 2021-03-10T14:25:05.823118+0000 mgr.gibba032.whopeo (mgr.24625) 1742 : cluster [DBG] pgmap v979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:06 gibba011 bash[31774]: cluster 2021-03-10T14:25:05.823118+0000 mgr.gibba032.whopeo (mgr.24625) 1742 : cluster [DBG] pgmap v979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:06 gibba032 bash[24004]: cluster 2021-03-10T14:25:05.823118+0000 mgr.gibba032.whopeo (mgr.24625) 1742 : cluster [DBG] pgmap v979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:08.446 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:25:08.676 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:25:08.677 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (31m) 7m ago 37m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:25:08.677 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (36m) 6m ago 36m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:25:08.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (32m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:25:08.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (32m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:25:08.678 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (31m) 7m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:25:08.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (31m) 7m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:25:08.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (30m) 6m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:25:08.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (31m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:25:08.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (37m) 7m ago 37m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:25:08.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (37m) 6m ago 37m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:25:08.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (30m) 7m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:25:08.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (29m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:25:08.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (29m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:25:08.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (28m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:25:08.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (28m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:25:08.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (27m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:25:08.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (27m) 6m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:25:08.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (27m) 6m ago 37m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:25:08.681 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (36m) 6m ago 37m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:25:08.681 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 35m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:25:08.926 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:25:08.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:25:08.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:25:08.927 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:08.928 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:25:08.928 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:25:08.929 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:08.929 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:25:08.929 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:25:08.930 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:08.930 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:25:08.930 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:25:08.930 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:25:08.931 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:25:08.931 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:25:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:08 gibba032 bash[24004]: cluster 2021-03-10T14:25:07.824920+0000 mgr.gibba032.whopeo (mgr.24625) 1743 : cluster [DBG] pgmap v980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:08 gibba011 bash[30791]: cluster 2021-03-10T14:25:07.824920+0000 mgr.gibba032.whopeo (mgr.24625) 1743 : cluster [DBG] pgmap v980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:08 gibba011 bash[31774]: cluster 2021-03-10T14:25:07.824920+0000 mgr.gibba032.whopeo (mgr.24625) 1743 : cluster [DBG] pgmap v980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:09 gibba032 bash[24004]: cephadm 2021-03-10T14:25:08.416730+0000 mgr.gibba032.whopeo (mgr.24625) 1744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:09 gibba032 bash[24004]: audit 2021-03-10T14:25:08.433712+0000 mgr.gibba032.whopeo (mgr.24625) 1745 : audit [DBG] from='client.55210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:09 gibba032 bash[24004]: audit 2021-03-10T14:25:08.661810+0000 mgr.gibba032.whopeo (mgr.24625) 1746 : audit [DBG] from='client.55216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:09 gibba032 bash[24004]: audit 2021-03-10T14:25:08.925709+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? 172.21.2.111:0/734641223' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[30791]: cephadm 2021-03-10T14:25:08.416730+0000 mgr.gibba032.whopeo (mgr.24625) 1744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[30791]: audit 2021-03-10T14:25:08.433712+0000 mgr.gibba032.whopeo (mgr.24625) 1745 : audit [DBG] from='client.55210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[30791]: audit 2021-03-10T14:25:08.661810+0000 mgr.gibba032.whopeo (mgr.24625) 1746 : audit [DBG] from='client.55216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[30791]: audit 2021-03-10T14:25:08.925709+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? 172.21.2.111:0/734641223' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[31774]: cephadm 2021-03-10T14:25:08.416730+0000 mgr.gibba032.whopeo (mgr.24625) 1744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[31774]: audit 2021-03-10T14:25:08.433712+0000 mgr.gibba032.whopeo (mgr.24625) 1745 : audit [DBG] from='client.55210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[31774]: audit 2021-03-10T14:25:08.661810+0000 mgr.gibba032.whopeo (mgr.24625) 1746 : audit [DBG] from='client.55216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:09 gibba011 bash[31774]: audit 2021-03-10T14:25:08.925709+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? 172.21.2.111:0/734641223' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:10 gibba032 bash[24004]: cluster 2021-03-10T14:25:09.826423+0000 mgr.gibba032.whopeo (mgr.24625) 1747 : cluster [DBG] pgmap v981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:10 gibba011 bash[30791]: cluster 2021-03-10T14:25:09.826423+0000 mgr.gibba032.whopeo (mgr.24625) 1747 : cluster [DBG] pgmap v981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:10 gibba011 bash[31774]: cluster 2021-03-10T14:25:09.826423+0000 mgr.gibba032.whopeo (mgr.24625) 1747 : cluster [DBG] pgmap v981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:12 gibba032 bash[24004]: cluster 2021-03-10T14:25:11.828174+0000 mgr.gibba032.whopeo (mgr.24625) 1748 : cluster [DBG] pgmap v982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:12 gibba011 bash[30791]: cluster 2021-03-10T14:25:11.828174+0000 mgr.gibba032.whopeo (mgr.24625) 1748 : cluster [DBG] pgmap v982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:12 gibba011 bash[31774]: cluster 2021-03-10T14:25:11.828174+0000 mgr.gibba032.whopeo (mgr.24625) 1748 : cluster [DBG] pgmap v982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:14 gibba032 bash[24004]: cluster 2021-03-10T14:25:13.829688+0000 mgr.gibba032.whopeo (mgr.24625) 1749 : cluster [DBG] pgmap v983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:14 gibba011 bash[30791]: cluster 2021-03-10T14:25:13.829688+0000 mgr.gibba032.whopeo (mgr.24625) 1749 : cluster [DBG] pgmap v983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:14 gibba011 bash[31774]: cluster 2021-03-10T14:25:13.829688+0000 mgr.gibba032.whopeo (mgr.24625) 1749 : cluster [DBG] pgmap v983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:16 gibba011 bash[30791]: cluster 2021-03-10T14:25:15.831049+0000 mgr.gibba032.whopeo (mgr.24625) 1750 : cluster [DBG] pgmap v984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:16 gibba011 bash[31774]: cluster 2021-03-10T14:25:15.831049+0000 mgr.gibba032.whopeo (mgr.24625) 1750 : cluster [DBG] pgmap v984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:16 gibba032 bash[24004]: cluster 2021-03-10T14:25:15.831049+0000 mgr.gibba032.whopeo (mgr.24625) 1750 : cluster [DBG] pgmap v984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:18 gibba032 bash[24004]: cluster 2021-03-10T14:25:17.832903+0000 mgr.gibba032.whopeo (mgr.24625) 1751 : cluster [DBG] pgmap v985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:18 gibba011 bash[30791]: cluster 2021-03-10T14:25:17.832903+0000 mgr.gibba032.whopeo (mgr.24625) 1751 : cluster [DBG] pgmap v985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:18 gibba011 bash[31774]: cluster 2021-03-10T14:25:17.832903+0000 mgr.gibba032.whopeo (mgr.24625) 1751 : cluster [DBG] pgmap v985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:20 gibba032 bash[24004]: cluster 2021-03-10T14:25:19.834331+0000 mgr.gibba032.whopeo (mgr.24625) 1752 : cluster [DBG] pgmap v986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:20 gibba011 bash[30791]: cluster 2021-03-10T14:25:19.834331+0000 mgr.gibba032.whopeo (mgr.24625) 1752 : cluster [DBG] pgmap v986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:20 gibba011 bash[31774]: cluster 2021-03-10T14:25:19.834331+0000 mgr.gibba032.whopeo (mgr.24625) 1752 : cluster [DBG] pgmap v986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:22 gibba032 bash[24004]: cluster 2021-03-10T14:25:21.835964+0000 mgr.gibba032.whopeo (mgr.24625) 1753 : cluster [DBG] pgmap v987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:22 gibba011 bash[30791]: cluster 2021-03-10T14:25:21.835964+0000 mgr.gibba032.whopeo (mgr.24625) 1753 : cluster [DBG] pgmap v987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:22 gibba011 bash[31774]: cluster 2021-03-10T14:25:21.835964+0000 mgr.gibba032.whopeo (mgr.24625) 1753 : cluster [DBG] pgmap v987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:24 gibba032 bash[24004]: cephadm 2021-03-10T14:25:23.419914+0000 mgr.gibba032.whopeo (mgr.24625) 1754 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:24 gibba032 bash[24004]: cluster 2021-03-10T14:25:23.837602+0000 mgr.gibba032.whopeo (mgr.24625) 1755 : cluster [DBG] pgmap v988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:24 gibba011 bash[30791]: cephadm 2021-03-10T14:25:23.419914+0000 mgr.gibba032.whopeo (mgr.24625) 1754 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:24 gibba011 bash[30791]: cluster 2021-03-10T14:25:23.837602+0000 mgr.gibba032.whopeo (mgr.24625) 1755 : cluster [DBG] pgmap v988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:24 gibba011 bash[31774]: cephadm 2021-03-10T14:25:23.419914+0000 mgr.gibba032.whopeo (mgr.24625) 1754 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:24 gibba011 bash[31774]: cluster 2021-03-10T14:25:23.837602+0000 mgr.gibba032.whopeo (mgr.24625) 1755 : cluster [DBG] pgmap v988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:26 gibba011 bash[30791]: cluster 2021-03-10T14:25:25.838980+0000 mgr.gibba032.whopeo (mgr.24625) 1756 : cluster [DBG] pgmap v989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:26 gibba011 bash[31774]: cluster 2021-03-10T14:25:25.838980+0000 mgr.gibba032.whopeo (mgr.24625) 1756 : cluster [DBG] pgmap v989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:26 gibba032 bash[24004]: cluster 2021-03-10T14:25:25.838980+0000 mgr.gibba032.whopeo (mgr.24625) 1756 : cluster [DBG] pgmap v989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:28 gibba032 bash[24004]: cluster 2021-03-10T14:25:27.840647+0000 mgr.gibba032.whopeo (mgr.24625) 1757 : cluster [DBG] pgmap v990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:28 gibba011 bash[30791]: cluster 2021-03-10T14:25:27.840647+0000 mgr.gibba032.whopeo (mgr.24625) 1757 : cluster [DBG] pgmap v990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:28 gibba011 bash[31774]: cluster 2021-03-10T14:25:27.840647+0000 mgr.gibba032.whopeo (mgr.24625) 1757 : cluster [DBG] pgmap v990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:31.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:30 gibba032 bash[24004]: cluster 2021-03-10T14:25:29.842179+0000 mgr.gibba032.whopeo (mgr.24625) 1758 : cluster [DBG] pgmap v991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:30 gibba011 bash[30791]: cluster 2021-03-10T14:25:29.842179+0000 mgr.gibba032.whopeo (mgr.24625) 1758 : cluster [DBG] pgmap v991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:30 gibba011 bash[31774]: cluster 2021-03-10T14:25:29.842179+0000 mgr.gibba032.whopeo (mgr.24625) 1758 : cluster [DBG] pgmap v991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:32 gibba032 bash[24004]: cluster 2021-03-10T14:25:31.843934+0000 mgr.gibba032.whopeo (mgr.24625) 1759 : cluster [DBG] pgmap v992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:32 gibba011 bash[30791]: cluster 2021-03-10T14:25:31.843934+0000 mgr.gibba032.whopeo (mgr.24625) 1759 : cluster [DBG] pgmap v992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:32 gibba011 bash[31774]: cluster 2021-03-10T14:25:31.843934+0000 mgr.gibba032.whopeo (mgr.24625) 1759 : cluster [DBG] pgmap v992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:35.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:34 gibba032 bash[24004]: cluster 2021-03-10T14:25:33.845456+0000 mgr.gibba032.whopeo (mgr.24625) 1760 : cluster [DBG] pgmap v993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:34 gibba011 bash[30791]: cluster 2021-03-10T14:25:33.845456+0000 mgr.gibba032.whopeo (mgr.24625) 1760 : cluster [DBG] pgmap v993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:34 gibba011 bash[31774]: cluster 2021-03-10T14:25:33.845456+0000 mgr.gibba032.whopeo (mgr.24625) 1760 : cluster [DBG] pgmap v993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:36 gibba011 bash[30791]: cluster 2021-03-10T14:25:35.846788+0000 mgr.gibba032.whopeo (mgr.24625) 1761 : cluster [DBG] pgmap v994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:36 gibba011 bash[31774]: cluster 2021-03-10T14:25:35.846788+0000 mgr.gibba032.whopeo (mgr.24625) 1761 : cluster [DBG] pgmap v994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:37.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:36 gibba032 bash[24004]: cluster 2021-03-10T14:25:35.846788+0000 mgr.gibba032.whopeo (mgr.24625) 1761 : cluster [DBG] pgmap v994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: cluster 2021-03-10T14:25:37.848564+0000 mgr.gibba032.whopeo (mgr.24625) 1762 : cluster [DBG] pgmap v995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.653253+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:25:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.656582+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:25:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.658154+0000 mon.a (mon.0) 1069 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.660494+0000 mon.a (mon.0) 1070 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.662717+0000 mon.a (mon.0) 1071 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.666923+0000 mon.a (mon.0) 1072 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:38 gibba032 bash[24004]: audit 2021-03-10T14:25:38.669425+0000 mon.a (mon.0) 1073 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.164 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:25:39.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: cluster 2021-03-10T14:25:37.848564+0000 mgr.gibba032.whopeo (mgr.24625) 1762 : cluster [DBG] pgmap v995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.653253+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:25:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.656582+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:25:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.658154+0000 mon.a (mon.0) 1069 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.660494+0000 mon.a (mon.0) 1070 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.662717+0000 mon.a (mon.0) 1071 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.666923+0000 mon.a (mon.0) 1072 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[30791]: audit 2021-03-10T14:25:38.669425+0000 mon.a (mon.0) 1073 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: cluster 2021-03-10T14:25:37.848564+0000 mgr.gibba032.whopeo (mgr.24625) 1762 : cluster [DBG] pgmap v995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:39.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.653253+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:25:39.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.656582+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:25:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.658154+0000 mon.a (mon.0) 1069 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.660494+0000 mon.a (mon.0) 1070 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.662717+0000 mon.a (mon.0) 1071 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.666923+0000 mon.a (mon.0) 1072 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:38 gibba011 bash[31774]: audit 2021-03-10T14:25:38.669425+0000 mon.a (mon.0) 1073 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:39.396 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:25:39.396 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (32m) 8m ago 37m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:25:39.396 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (37m) 7m ago 37m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:25:39.396 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (32m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:25:39.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (32m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:25:39.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (31m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:25:39.397 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (31m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:25:39.397 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (31m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:25:39.397 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (31m) 8m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (37m) 8m ago 37m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (37m) 7m ago 37m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (30m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (30m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (29m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:25:39.398 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (29m) 8m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:25:39.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (28m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:25:39.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (28m) 7m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:25:39.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (28m) 7m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:25:39.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (27m) 7m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:25:39.399 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (37m) 7m ago 37m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:25:39.400 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 35m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:25:39.648 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:25:39.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:25:39.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:25:39.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:39.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:25:39.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:25:39.650 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:39.650 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:25:39.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:25:39.650 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:25:39.651 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:25:39.651 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:25:39.651 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:25:39.651 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:25:39.651 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:25:40.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.428062+0000 mgr.gibba032.whopeo (mgr.24625) 1763 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:25:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.429608+0000 mgr.gibba032.whopeo (mgr.24625) 1764 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:25:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.652857+0000 mgr.gibba032.whopeo (mgr.24625) 1765 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:25:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.654550+0000 mgr.gibba032.whopeo (mgr.24625) 1766 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:25:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.656767+0000 mgr.gibba032.whopeo (mgr.24625) 1767 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:25:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.656805+0000 mgr.gibba032.whopeo (mgr.24625) 1768 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:25:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.659221+0000 mgr.gibba032.whopeo (mgr.24625) 1769 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:25:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.659259+0000 mgr.gibba032.whopeo (mgr.24625) 1770 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:25:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.661404+0000 mgr.gibba032.whopeo (mgr.24625) 1771 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:25:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.663241+0000 mgr.gibba032.whopeo (mgr.24625) 1772 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:25:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.663277+0000 mgr.gibba032.whopeo (mgr.24625) 1773 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:25:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.665649+0000 mgr.gibba032.whopeo (mgr.24625) 1774 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:25:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.665960+0000 mgr.gibba032.whopeo (mgr.24625) 1775 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:25:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.668108+0000 mgr.gibba032.whopeo (mgr.24625) 1776 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:25:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.668173+0000 mgr.gibba032.whopeo (mgr.24625) 1777 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:25:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: cephadm 2021-03-10T14:25:38.668272+0000 mgr.gibba032.whopeo (mgr.24625) 1778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:39 gibba032 bash[24004]: audit 2021-03-10T14:25:39.648061+0000 mon.c (mon.1) 34 : audit [DBG] from='client.? 172.21.2.111:0/3677668267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.428062+0000 mgr.gibba032.whopeo (mgr.24625) 1763 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:25:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.429608+0000 mgr.gibba032.whopeo (mgr.24625) 1764 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:25:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.652857+0000 mgr.gibba032.whopeo (mgr.24625) 1765 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:25:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.654550+0000 mgr.gibba032.whopeo (mgr.24625) 1766 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:25:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.656767+0000 mgr.gibba032.whopeo (mgr.24625) 1767 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:25:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.656805+0000 mgr.gibba032.whopeo (mgr.24625) 1768 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:25:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.659221+0000 mgr.gibba032.whopeo (mgr.24625) 1769 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:25:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.659259+0000 mgr.gibba032.whopeo (mgr.24625) 1770 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:25:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.661404+0000 mgr.gibba032.whopeo (mgr.24625) 1771 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:25:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.663241+0000 mgr.gibba032.whopeo (mgr.24625) 1772 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:25:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.663277+0000 mgr.gibba032.whopeo (mgr.24625) 1773 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:25:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.665649+0000 mgr.gibba032.whopeo (mgr.24625) 1774 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:25:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.665960+0000 mgr.gibba032.whopeo (mgr.24625) 1775 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:25:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.668108+0000 mgr.gibba032.whopeo (mgr.24625) 1776 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:25:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.668173+0000 mgr.gibba032.whopeo (mgr.24625) 1777 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:25:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: cephadm 2021-03-10T14:25:38.668272+0000 mgr.gibba032.whopeo (mgr.24625) 1778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[30791]: audit 2021-03-10T14:25:39.648061+0000 mon.c (mon.1) 34 : audit [DBG] from='client.? 172.21.2.111:0/3677668267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:40.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.428062+0000 mgr.gibba032.whopeo (mgr.24625) 1763 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:25:40.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.429608+0000 mgr.gibba032.whopeo (mgr.24625) 1764 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:25:40.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.652857+0000 mgr.gibba032.whopeo (mgr.24625) 1765 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:25:40.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.654550+0000 mgr.gibba032.whopeo (mgr.24625) 1766 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:25:40.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.656767+0000 mgr.gibba032.whopeo (mgr.24625) 1767 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:25:40.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.656805+0000 mgr.gibba032.whopeo (mgr.24625) 1768 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:25:40.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.659221+0000 mgr.gibba032.whopeo (mgr.24625) 1769 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:25:40.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.659259+0000 mgr.gibba032.whopeo (mgr.24625) 1770 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:25:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.661404+0000 mgr.gibba032.whopeo (mgr.24625) 1771 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:25:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.663241+0000 mgr.gibba032.whopeo (mgr.24625) 1772 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:25:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.663277+0000 mgr.gibba032.whopeo (mgr.24625) 1773 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:25:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.665649+0000 mgr.gibba032.whopeo (mgr.24625) 1774 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:25:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.665960+0000 mgr.gibba032.whopeo (mgr.24625) 1775 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:25:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.668108+0000 mgr.gibba032.whopeo (mgr.24625) 1776 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:25:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.668173+0000 mgr.gibba032.whopeo (mgr.24625) 1777 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:25:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: cephadm 2021-03-10T14:25:38.668272+0000 mgr.gibba032.whopeo (mgr.24625) 1778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:39 gibba011 bash[31774]: audit 2021-03-10T14:25:39.648061+0000 mon.c (mon.1) 34 : audit [DBG] from='client.? 172.21.2.111:0/3677668267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:25:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:40 gibba032 bash[24004]: audit 2021-03-10T14:25:39.152717+0000 mgr.gibba032.whopeo (mgr.24625) 1779 : audit [DBG] from='client.55228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:40 gibba032 bash[24004]: audit 2021-03-10T14:25:39.380582+0000 mgr.gibba032.whopeo (mgr.24625) 1780 : audit [DBG] from='client.45249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:40 gibba032 bash[24004]: cluster 2021-03-10T14:25:39.850128+0000 mgr.gibba032.whopeo (mgr.24625) 1781 : cluster [DBG] pgmap v996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[30791]: audit 2021-03-10T14:25:39.152717+0000 mgr.gibba032.whopeo (mgr.24625) 1779 : audit [DBG] from='client.55228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[30791]: audit 2021-03-10T14:25:39.380582+0000 mgr.gibba032.whopeo (mgr.24625) 1780 : audit [DBG] from='client.45249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[30791]: cluster 2021-03-10T14:25:39.850128+0000 mgr.gibba032.whopeo (mgr.24625) 1781 : cluster [DBG] pgmap v996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[31774]: audit 2021-03-10T14:25:39.152717+0000 mgr.gibba032.whopeo (mgr.24625) 1779 : audit [DBG] from='client.55228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[31774]: audit 2021-03-10T14:25:39.380582+0000 mgr.gibba032.whopeo (mgr.24625) 1780 : audit [DBG] from='client.45249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:25:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:40 gibba011 bash[31774]: cluster 2021-03-10T14:25:39.850128+0000 mgr.gibba032.whopeo (mgr.24625) 1781 : cluster [DBG] pgmap v996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:42 gibba032 bash[24004]: cluster 2021-03-10T14:25:41.851878+0000 mgr.gibba032.whopeo (mgr.24625) 1782 : cluster [DBG] pgmap v997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:42 gibba011 bash[30791]: cluster 2021-03-10T14:25:41.851878+0000 mgr.gibba032.whopeo (mgr.24625) 1782 : cluster [DBG] pgmap v997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:42 gibba011 bash[31774]: cluster 2021-03-10T14:25:41.851878+0000 mgr.gibba032.whopeo (mgr.24625) 1782 : cluster [DBG] pgmap v997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:45.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:44 gibba032 bash[24004]: cluster 2021-03-10T14:25:43.853477+0000 mgr.gibba032.whopeo (mgr.24625) 1783 : cluster [DBG] pgmap v998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:44 gibba011 bash[30791]: cluster 2021-03-10T14:25:43.853477+0000 mgr.gibba032.whopeo (mgr.24625) 1783 : cluster [DBG] pgmap v998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:45.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:44 gibba011 bash[31774]: cluster 2021-03-10T14:25:43.853477+0000 mgr.gibba032.whopeo (mgr.24625) 1783 : cluster [DBG] pgmap v998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:46 gibba011 bash[30791]: cluster 2021-03-10T14:25:45.854836+0000 mgr.gibba032.whopeo (mgr.24625) 1784 : cluster [DBG] pgmap v999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:46 gibba011 bash[31774]: cluster 2021-03-10T14:25:45.854836+0000 mgr.gibba032.whopeo (mgr.24625) 1784 : cluster [DBG] pgmap v999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:46 gibba032 bash[24004]: cluster 2021-03-10T14:25:45.854836+0000 mgr.gibba032.whopeo (mgr.24625) 1784 : cluster [DBG] pgmap v999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:48 gibba032 bash[24004]: cluster 2021-03-10T14:25:47.856635+0000 mgr.gibba032.whopeo (mgr.24625) 1785 : cluster [DBG] pgmap v1000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:48 gibba011 bash[30791]: cluster 2021-03-10T14:25:47.856635+0000 mgr.gibba032.whopeo (mgr.24625) 1785 : cluster [DBG] pgmap v1000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:48 gibba011 bash[31774]: cluster 2021-03-10T14:25:47.856635+0000 mgr.gibba032.whopeo (mgr.24625) 1785 : cluster [DBG] pgmap v1000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:50 gibba032 bash[24004]: cluster 2021-03-10T14:25:49.858140+0000 mgr.gibba032.whopeo (mgr.24625) 1786 : cluster [DBG] pgmap v1001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:50 gibba011 bash[30791]: cluster 2021-03-10T14:25:49.858140+0000 mgr.gibba032.whopeo (mgr.24625) 1786 : cluster [DBG] pgmap v1001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:50 gibba011 bash[31774]: cluster 2021-03-10T14:25:49.858140+0000 mgr.gibba032.whopeo (mgr.24625) 1786 : cluster [DBG] pgmap v1001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:53.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:52 gibba032 bash[24004]: cluster 2021-03-10T14:25:51.859893+0000 mgr.gibba032.whopeo (mgr.24625) 1787 : cluster [DBG] pgmap v1002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:52 gibba011 bash[30791]: cluster 2021-03-10T14:25:51.859893+0000 mgr.gibba032.whopeo (mgr.24625) 1787 : cluster [DBG] pgmap v1002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:52 gibba011 bash[31774]: cluster 2021-03-10T14:25:51.859893+0000 mgr.gibba032.whopeo (mgr.24625) 1787 : cluster [DBG] pgmap v1002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:54 gibba032 bash[24004]: cephadm 2021-03-10T14:25:53.684248+0000 mgr.gibba032.whopeo (mgr.24625) 1788 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:54 gibba032 bash[24004]: cluster 2021-03-10T14:25:53.861529+0000 mgr.gibba032.whopeo (mgr.24625) 1789 : cluster [DBG] pgmap v1003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:54 gibba011 bash[30791]: cephadm 2021-03-10T14:25:53.684248+0000 mgr.gibba032.whopeo (mgr.24625) 1788 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:54 gibba011 bash[30791]: cluster 2021-03-10T14:25:53.861529+0000 mgr.gibba032.whopeo (mgr.24625) 1789 : cluster [DBG] pgmap v1003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:54 gibba011 bash[31774]: cephadm 2021-03-10T14:25:53.684248+0000 mgr.gibba032.whopeo (mgr.24625) 1788 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:25:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:54 gibba011 bash[31774]: cluster 2021-03-10T14:25:53.861529+0000 mgr.gibba032.whopeo (mgr.24625) 1789 : cluster [DBG] pgmap v1003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: cluster 2021-03-10T14:25:55.862839+0000 mgr.gibba032.whopeo (mgr.24625) 1790 : cluster [DBG] pgmap v1004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.173198+0000 mon.a (mon.0) 1074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:25:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.173968+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T14:25:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.174697+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.175475+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.211522+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:25:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[30791]: audit 2021-03-10T14:25:56.212133+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:25:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: cluster 2021-03-10T14:25:55.862839+0000 mgr.gibba032.whopeo (mgr.24625) 1790 : cluster [DBG] pgmap v1004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.173198+0000 mon.a (mon.0) 1074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:25:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.173968+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T14:25:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.174697+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.175475+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.211522+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:25:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:56 gibba011 bash[31774]: audit 2021-03-10T14:25:56.212133+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:25:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: cluster 2021-03-10T14:25:55.862839+0000 mgr.gibba032.whopeo (mgr.24625) 1790 : cluster [DBG] pgmap v1004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.173198+0000 mon.a (mon.0) 1074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:25:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.173968+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T14:25:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.174697+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.175475+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:25:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.211522+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:25:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:56 gibba032 bash[24004]: audit 2021-03-10T14:25:56.212133+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:25:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:25:58 gibba032 bash[24004]: cluster 2021-03-10T14:25:57.864645+0000 mgr.gibba032.whopeo (mgr.24625) 1791 : cluster [DBG] pgmap v1005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:25:58 gibba011 bash[30791]: cluster 2021-03-10T14:25:57.864645+0000 mgr.gibba032.whopeo (mgr.24625) 1791 : cluster [DBG] pgmap v1005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:25:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:25:58 gibba011 bash[31774]: cluster 2021-03-10T14:25:57.864645+0000 mgr.gibba032.whopeo (mgr.24625) 1791 : cluster [DBG] pgmap v1005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:00 gibba032 bash[24004]: cluster 2021-03-10T14:25:59.866128+0000 mgr.gibba032.whopeo (mgr.24625) 1792 : cluster [DBG] pgmap v1006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:00 gibba011 bash[30791]: cluster 2021-03-10T14:25:59.866128+0000 mgr.gibba032.whopeo (mgr.24625) 1792 : cluster [DBG] pgmap v1006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:00 gibba011 bash[31774]: cluster 2021-03-10T14:25:59.866128+0000 mgr.gibba032.whopeo (mgr.24625) 1792 : cluster [DBG] pgmap v1006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:02 gibba032 bash[24004]: cluster 2021-03-10T14:26:01.867925+0000 mgr.gibba032.whopeo (mgr.24625) 1793 : cluster [DBG] pgmap v1007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:02 gibba011 bash[30791]: cluster 2021-03-10T14:26:01.867925+0000 mgr.gibba032.whopeo (mgr.24625) 1793 : cluster [DBG] pgmap v1007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:02 gibba011 bash[31774]: cluster 2021-03-10T14:26:01.867925+0000 mgr.gibba032.whopeo (mgr.24625) 1793 : cluster [DBG] pgmap v1007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:05.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:04 gibba032 bash[24004]: cluster 2021-03-10T14:26:03.869514+0000 mgr.gibba032.whopeo (mgr.24625) 1794 : cluster [DBG] pgmap v1008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:04 gibba011 bash[30791]: cluster 2021-03-10T14:26:03.869514+0000 mgr.gibba032.whopeo (mgr.24625) 1794 : cluster [DBG] pgmap v1008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:05.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:04 gibba011 bash[31774]: cluster 2021-03-10T14:26:03.869514+0000 mgr.gibba032.whopeo (mgr.24625) 1794 : cluster [DBG] pgmap v1008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:06 gibba011 bash[30791]: cluster 2021-03-10T14:26:05.870867+0000 mgr.gibba032.whopeo (mgr.24625) 1795 : cluster [DBG] pgmap v1009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:06 gibba011 bash[31774]: cluster 2021-03-10T14:26:05.870867+0000 mgr.gibba032.whopeo (mgr.24625) 1795 : cluster [DBG] pgmap v1009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:06 gibba032 bash[24004]: cluster 2021-03-10T14:26:05.870867+0000 mgr.gibba032.whopeo (mgr.24625) 1795 : cluster [DBG] pgmap v1009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:08 gibba032 bash[24004]: cluster 2021-03-10T14:26:07.872677+0000 mgr.gibba032.whopeo (mgr.24625) 1796 : cluster [DBG] pgmap v1010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:08 gibba011 bash[30791]: cluster 2021-03-10T14:26:07.872677+0000 mgr.gibba032.whopeo (mgr.24625) 1796 : cluster [DBG] pgmap v1010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:08 gibba011 bash[31774]: cluster 2021-03-10T14:26:07.872677+0000 mgr.gibba032.whopeo (mgr.24625) 1796 : cluster [DBG] pgmap v1010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:09.894 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:26:10.153 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:26:10.154 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (32m) 8m ago 38m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:26:10.154 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (38m) 7m ago 38m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:26:10.154 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (33m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:26:10.155 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (33m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:26:10.155 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (32m) 8m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:26:10.155 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (32m) 8m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:26:10.155 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (31m) 7m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:26:10.156 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (32m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:26:10.156 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (38m) 8m ago 38m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:26:10.156 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (38m) 7m ago 38m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:26:10.156 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (31m) 8m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:26:10.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (30m) 8m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:26:10.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (30m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:26:10.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (29m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:26:10.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (29m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:26:10.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (28m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:26:10.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (28m) 7m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:26:10.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (28m) 7m ago 38m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:26:10.158 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (37m) 7m ago 38m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:26:10.158 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 36m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:26:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:09 gibba032 bash[24004]: cephadm 2021-03-10T14:26:08.700325+0000 mgr.gibba032.whopeo (mgr.24625) 1797 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:09 gibba011 bash[30791]: cephadm 2021-03-10T14:26:08.700325+0000 mgr.gibba032.whopeo (mgr.24625) 1797 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:09 gibba011 bash[31774]: cephadm 2021-03-10T14:26:08.700325+0000 mgr.gibba032.whopeo (mgr.24625) 1797 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:10.411 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:26:10.411 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:26:10.411 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:26:10.412 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:10.412 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:26:10.412 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:26:10.412 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:10.413 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:26:10.413 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:26:10.413 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:10.413 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:26:10.413 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:26:10.414 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:26:10.414 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:26:10.414 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:26:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:10 gibba032 bash[24004]: cluster 2021-03-10T14:26:09.874253+0000 mgr.gibba032.whopeo (mgr.24625) 1798 : cluster [DBG] pgmap v1011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:10 gibba032 bash[24004]: audit 2021-03-10T14:26:09.881224+0000 mgr.gibba032.whopeo (mgr.24625) 1799 : audit [DBG] from='client.45261 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:11.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:10 gibba032 bash[24004]: audit 2021-03-10T14:26:10.411059+0000 mon.c (mon.1) 35 : audit [DBG] from='client.? 172.21.2.111:0/1714026351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[30791]: cluster 2021-03-10T14:26:09.874253+0000 mgr.gibba032.whopeo (mgr.24625) 1798 : cluster [DBG] pgmap v1011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[30791]: audit 2021-03-10T14:26:09.881224+0000 mgr.gibba032.whopeo (mgr.24625) 1799 : audit [DBG] from='client.45261 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[30791]: audit 2021-03-10T14:26:10.411059+0000 mon.c (mon.1) 35 : audit [DBG] from='client.? 172.21.2.111:0/1714026351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[31774]: cluster 2021-03-10T14:26:09.874253+0000 mgr.gibba032.whopeo (mgr.24625) 1798 : cluster [DBG] pgmap v1011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[31774]: audit 2021-03-10T14:26:09.881224+0000 mgr.gibba032.whopeo (mgr.24625) 1799 : audit [DBG] from='client.45261 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:10 gibba011 bash[31774]: audit 2021-03-10T14:26:10.411059+0000 mon.c (mon.1) 35 : audit [DBG] from='client.? 172.21.2.111:0/1714026351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:11 gibba032 bash[24004]: audit 2021-03-10T14:26:10.110988+0000 mgr.gibba032.whopeo (mgr.24625) 1800 : audit [DBG] from='client.45267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:11 gibba011 bash[30791]: audit 2021-03-10T14:26:10.110988+0000 mgr.gibba032.whopeo (mgr.24625) 1800 : audit [DBG] from='client.45267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:11 gibba011 bash[31774]: audit 2021-03-10T14:26:10.110988+0000 mgr.gibba032.whopeo (mgr.24625) 1800 : audit [DBG] from='client.45267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:12 gibba032 bash[24004]: cluster 2021-03-10T14:26:11.875725+0000 mgr.gibba032.whopeo (mgr.24625) 1801 : cluster [DBG] pgmap v1012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:12 gibba011 bash[30791]: cluster 2021-03-10T14:26:11.875725+0000 mgr.gibba032.whopeo (mgr.24625) 1801 : cluster [DBG] pgmap v1012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:12 gibba011 bash[31774]: cluster 2021-03-10T14:26:11.875725+0000 mgr.gibba032.whopeo (mgr.24625) 1801 : cluster [DBG] pgmap v1012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:15.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:14 gibba032 bash[24004]: cluster 2021-03-10T14:26:13.877311+0000 mgr.gibba032.whopeo (mgr.24625) 1802 : cluster [DBG] pgmap v1013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:14 gibba011 bash[30791]: cluster 2021-03-10T14:26:13.877311+0000 mgr.gibba032.whopeo (mgr.24625) 1802 : cluster [DBG] pgmap v1013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:14 gibba011 bash[31774]: cluster 2021-03-10T14:26:13.877311+0000 mgr.gibba032.whopeo (mgr.24625) 1802 : cluster [DBG] pgmap v1013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:16 gibba011 bash[30791]: cluster 2021-03-10T14:26:15.878614+0000 mgr.gibba032.whopeo (mgr.24625) 1803 : cluster [DBG] pgmap v1014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:16 gibba011 bash[31774]: cluster 2021-03-10T14:26:15.878614+0000 mgr.gibba032.whopeo (mgr.24625) 1803 : cluster [DBG] pgmap v1014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:16 gibba032 bash[24004]: cluster 2021-03-10T14:26:15.878614+0000 mgr.gibba032.whopeo (mgr.24625) 1803 : cluster [DBG] pgmap v1014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:18 gibba032 bash[24004]: cluster 2021-03-10T14:26:17.880405+0000 mgr.gibba032.whopeo (mgr.24625) 1804 : cluster [DBG] pgmap v1015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:18 gibba011 bash[30791]: cluster 2021-03-10T14:26:17.880405+0000 mgr.gibba032.whopeo (mgr.24625) 1804 : cluster [DBG] pgmap v1015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:18 gibba011 bash[31774]: cluster 2021-03-10T14:26:17.880405+0000 mgr.gibba032.whopeo (mgr.24625) 1804 : cluster [DBG] pgmap v1015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:20 gibba032 bash[24004]: cluster 2021-03-10T14:26:19.881648+0000 mgr.gibba032.whopeo (mgr.24625) 1805 : cluster [DBG] pgmap v1016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:20 gibba011 bash[30791]: cluster 2021-03-10T14:26:19.881648+0000 mgr.gibba032.whopeo (mgr.24625) 1805 : cluster [DBG] pgmap v1016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:20 gibba011 bash[31774]: cluster 2021-03-10T14:26:19.881648+0000 mgr.gibba032.whopeo (mgr.24625) 1805 : cluster [DBG] pgmap v1016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:22 gibba032 bash[24004]: cluster 2021-03-10T14:26:21.883234+0000 mgr.gibba032.whopeo (mgr.24625) 1806 : cluster [DBG] pgmap v1017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:22 gibba011 bash[30791]: cluster 2021-03-10T14:26:21.883234+0000 mgr.gibba032.whopeo (mgr.24625) 1806 : cluster [DBG] pgmap v1017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:22 gibba011 bash[31774]: cluster 2021-03-10T14:26:21.883234+0000 mgr.gibba032.whopeo (mgr.24625) 1806 : cluster [DBG] pgmap v1017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:24 gibba032 bash[24004]: cephadm 2021-03-10T14:26:23.716354+0000 mgr.gibba032.whopeo (mgr.24625) 1807 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:24 gibba032 bash[24004]: cluster 2021-03-10T14:26:23.884971+0000 mgr.gibba032.whopeo (mgr.24625) 1808 : cluster [DBG] pgmap v1018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:24 gibba011 bash[30791]: cephadm 2021-03-10T14:26:23.716354+0000 mgr.gibba032.whopeo (mgr.24625) 1807 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:24 gibba011 bash[30791]: cluster 2021-03-10T14:26:23.884971+0000 mgr.gibba032.whopeo (mgr.24625) 1808 : cluster [DBG] pgmap v1018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:24 gibba011 bash[31774]: cephadm 2021-03-10T14:26:23.716354+0000 mgr.gibba032.whopeo (mgr.24625) 1807 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:24 gibba011 bash[31774]: cluster 2021-03-10T14:26:23.884971+0000 mgr.gibba032.whopeo (mgr.24625) 1808 : cluster [DBG] pgmap v1018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:26 gibba011 bash[30791]: cluster 2021-03-10T14:26:25.886296+0000 mgr.gibba032.whopeo (mgr.24625) 1809 : cluster [DBG] pgmap v1019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:26 gibba011 bash[31774]: cluster 2021-03-10T14:26:25.886296+0000 mgr.gibba032.whopeo (mgr.24625) 1809 : cluster [DBG] pgmap v1019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:26 gibba032 bash[24004]: cluster 2021-03-10T14:26:25.886296+0000 mgr.gibba032.whopeo (mgr.24625) 1809 : cluster [DBG] pgmap v1019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:28 gibba032 bash[24004]: cluster 2021-03-10T14:26:27.888115+0000 mgr.gibba032.whopeo (mgr.24625) 1810 : cluster [DBG] pgmap v1020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:28 gibba011 bash[30791]: cluster 2021-03-10T14:26:27.888115+0000 mgr.gibba032.whopeo (mgr.24625) 1810 : cluster [DBG] pgmap v1020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:28 gibba011 bash[31774]: cluster 2021-03-10T14:26:27.888115+0000 mgr.gibba032.whopeo (mgr.24625) 1810 : cluster [DBG] pgmap v1020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:31.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:30 gibba032 bash[24004]: cluster 2021-03-10T14:26:29.889648+0000 mgr.gibba032.whopeo (mgr.24625) 1811 : cluster [DBG] pgmap v1021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:30 gibba011 bash[30791]: cluster 2021-03-10T14:26:29.889648+0000 mgr.gibba032.whopeo (mgr.24625) 1811 : cluster [DBG] pgmap v1021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:30 gibba011 bash[31774]: cluster 2021-03-10T14:26:29.889648+0000 mgr.gibba032.whopeo (mgr.24625) 1811 : cluster [DBG] pgmap v1021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:32 gibba032 bash[24004]: cluster 2021-03-10T14:26:31.891271+0000 mgr.gibba032.whopeo (mgr.24625) 1812 : cluster [DBG] pgmap v1022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:32 gibba011 bash[30791]: cluster 2021-03-10T14:26:31.891271+0000 mgr.gibba032.whopeo (mgr.24625) 1812 : cluster [DBG] pgmap v1022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:32 gibba011 bash[31774]: cluster 2021-03-10T14:26:31.891271+0000 mgr.gibba032.whopeo (mgr.24625) 1812 : cluster [DBG] pgmap v1022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:35.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:34 gibba032 bash[24004]: cluster 2021-03-10T14:26:33.892887+0000 mgr.gibba032.whopeo (mgr.24625) 1813 : cluster [DBG] pgmap v1023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:34 gibba011 bash[30791]: cluster 2021-03-10T14:26:33.892887+0000 mgr.gibba032.whopeo (mgr.24625) 1813 : cluster [DBG] pgmap v1023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:34 gibba011 bash[31774]: cluster 2021-03-10T14:26:33.892887+0000 mgr.gibba032.whopeo (mgr.24625) 1813 : cluster [DBG] pgmap v1023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:36 gibba011 bash[30791]: cluster 2021-03-10T14:26:35.894234+0000 mgr.gibba032.whopeo (mgr.24625) 1814 : cluster [DBG] pgmap v1024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:36 gibba011 bash[31774]: cluster 2021-03-10T14:26:35.894234+0000 mgr.gibba032.whopeo (mgr.24625) 1814 : cluster [DBG] pgmap v1024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:36 gibba032 bash[24004]: cluster 2021-03-10T14:26:35.894234+0000 mgr.gibba032.whopeo (mgr.24625) 1814 : cluster [DBG] pgmap v1024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:39.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:38 gibba032 bash[24004]: cluster 2021-03-10T14:26:37.896052+0000 mgr.gibba032.whopeo (mgr.24625) 1815 : cluster [DBG] pgmap v1025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:38 gibba011 bash[30791]: cluster 2021-03-10T14:26:37.896052+0000 mgr.gibba032.whopeo (mgr.24625) 1815 : cluster [DBG] pgmap v1025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:38 gibba011 bash[31774]: cluster 2021-03-10T14:26:37.896052+0000 mgr.gibba032.whopeo (mgr.24625) 1815 : cluster [DBG] pgmap v1025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.733175+0000 mgr.gibba032.whopeo (mgr.24625) 1816 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:26:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.734758+0000 mgr.gibba032.whopeo (mgr.24625) 1817 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:26:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.957096+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:26:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.957144+0000 mgr.gibba032.whopeo (mgr.24625) 1818 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:26:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.958765+0000 mgr.gibba032.whopeo (mgr.24625) 1819 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:26:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.959669+0000 mgr.gibba032.whopeo (mgr.24625) 1820 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:26:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.959702+0000 mgr.gibba032.whopeo (mgr.24625) 1821 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:26:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.960743+0000 mgr.gibba032.whopeo (mgr.24625) 1822 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:26:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.960780+0000 mgr.gibba032.whopeo (mgr.24625) 1823 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:26:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.961010+0000 mon.a (mon.0) 1081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:26:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.961991+0000 mgr.gibba032.whopeo (mgr.24625) 1824 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:26:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.962075+0000 mon.a (mon.0) 1082 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.963024+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.963216+0000 mgr.gibba032.whopeo (mgr.24625) 1825 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:26:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.963250+0000 mgr.gibba032.whopeo (mgr.24625) 1826 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:26:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.964106+0000 mon.a (mon.0) 1084 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.964585+0000 mgr.gibba032.whopeo (mgr.24625) 1827 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:26:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.964913+0000 mgr.gibba032.whopeo (mgr.24625) 1828 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:26:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.966146+0000 mgr.gibba032.whopeo (mgr.24625) 1829 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:26:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.966220+0000 mgr.gibba032.whopeo (mgr.24625) 1830 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:26:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: cephadm 2021-03-10T14:26:38.966328+0000 mgr.gibba032.whopeo (mgr.24625) 1831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.966752+0000 mon.a (mon.0) 1085 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:39 gibba032 bash[24004]: audit 2021-03-10T14:26:38.968328+0000 mon.a (mon.0) 1086 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.733175+0000 mgr.gibba032.whopeo (mgr.24625) 1816 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:26:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.734758+0000 mgr.gibba032.whopeo (mgr.24625) 1817 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:26:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.957096+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:26:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.957144+0000 mgr.gibba032.whopeo (mgr.24625) 1818 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:26:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.958765+0000 mgr.gibba032.whopeo (mgr.24625) 1819 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:26:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.959669+0000 mgr.gibba032.whopeo (mgr.24625) 1820 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:26:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.959702+0000 mgr.gibba032.whopeo (mgr.24625) 1821 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:26:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.960743+0000 mgr.gibba032.whopeo (mgr.24625) 1822 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:26:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.960780+0000 mgr.gibba032.whopeo (mgr.24625) 1823 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:26:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.961010+0000 mon.a (mon.0) 1081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:26:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.961991+0000 mgr.gibba032.whopeo (mgr.24625) 1824 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:26:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.962075+0000 mon.a (mon.0) 1082 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.963024+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.963216+0000 mgr.gibba032.whopeo (mgr.24625) 1825 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:26:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.963250+0000 mgr.gibba032.whopeo (mgr.24625) 1826 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:26:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.964106+0000 mon.a (mon.0) 1084 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.964585+0000 mgr.gibba032.whopeo (mgr.24625) 1827 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:26:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.964913+0000 mgr.gibba032.whopeo (mgr.24625) 1828 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:26:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.966146+0000 mgr.gibba032.whopeo (mgr.24625) 1829 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:26:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.966220+0000 mgr.gibba032.whopeo (mgr.24625) 1830 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:26:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: cephadm 2021-03-10T14:26:38.966328+0000 mgr.gibba032.whopeo (mgr.24625) 1831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.966752+0000 mon.a (mon.0) 1085 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[30791]: audit 2021-03-10T14:26:38.968328+0000 mon.a (mon.0) 1086 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.733175+0000 mgr.gibba032.whopeo (mgr.24625) 1816 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:26:40.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.734758+0000 mgr.gibba032.whopeo (mgr.24625) 1817 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:26:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.957096+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:26:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.957144+0000 mgr.gibba032.whopeo (mgr.24625) 1818 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:26:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.958765+0000 mgr.gibba032.whopeo (mgr.24625) 1819 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:26:40.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.959669+0000 mgr.gibba032.whopeo (mgr.24625) 1820 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:26:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.959702+0000 mgr.gibba032.whopeo (mgr.24625) 1821 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:26:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.960743+0000 mgr.gibba032.whopeo (mgr.24625) 1822 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:26:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.960780+0000 mgr.gibba032.whopeo (mgr.24625) 1823 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:26:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.961010+0000 mon.a (mon.0) 1081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:26:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.961991+0000 mgr.gibba032.whopeo (mgr.24625) 1824 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:26:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.962075+0000 mon.a (mon.0) 1082 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.963024+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.963216+0000 mgr.gibba032.whopeo (mgr.24625) 1825 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:26:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.963250+0000 mgr.gibba032.whopeo (mgr.24625) 1826 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:26:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.964106+0000 mon.a (mon.0) 1084 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.964585+0000 mgr.gibba032.whopeo (mgr.24625) 1827 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:26:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.964913+0000 mgr.gibba032.whopeo (mgr.24625) 1828 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:26:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.966146+0000 mgr.gibba032.whopeo (mgr.24625) 1829 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:26:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.966220+0000 mgr.gibba032.whopeo (mgr.24625) 1830 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:26:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: cephadm 2021-03-10T14:26:38.966328+0000 mgr.gibba032.whopeo (mgr.24625) 1831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.966752+0000 mon.a (mon.0) 1085 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:39 gibba011 bash[31774]: audit 2021-03-10T14:26:38.968328+0000 mon.a (mon.0) 1086 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:40.657 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:26:40.912 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:26:40.912 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (33m) 9m ago 38m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:26:40.913 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (38m) 8m ago 38m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:26:40.913 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (33m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:26:40.913 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (33m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:26:40.913 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (33m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:26:40.913 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (32m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:26:40.914 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (32m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:26:40.914 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (32m) 9m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:26:40.914 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (38m) 9m ago 38m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:26:40.914 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (38m) 8m ago 38m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:26:40.914 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (31m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:26:40.915 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (31m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:26:40.915 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (30m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:26:40.915 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (30m) 9m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:26:40.915 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (29m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:26:40.915 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (29m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:26:40.916 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (29m) 8m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:26:40.916 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (28m) 8m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:26:40.916 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (38m) 8m ago 38m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:26:40.916 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 36m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:26:41.166 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:26:41.166 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:26:41.166 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:26:41.167 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:41.167 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:26:41.167 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:26:41.167 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:41.168 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:26:41.168 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:26:41.168 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:26:41.168 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:26:41.168 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:26:41.169 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:26:41.169 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:26:41.169 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:26:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:40 gibba032 bash[24004]: cluster 2021-03-10T14:26:39.897553+0000 mgr.gibba032.whopeo (mgr.24625) 1832 : cluster [DBG] pgmap v1026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:40 gibba011 bash[30791]: cluster 2021-03-10T14:26:39.897553+0000 mgr.gibba032.whopeo (mgr.24625) 1832 : cluster [DBG] pgmap v1026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:40 gibba011 bash[31774]: cluster 2021-03-10T14:26:39.897553+0000 mgr.gibba032.whopeo (mgr.24625) 1832 : cluster [DBG] pgmap v1026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:41 gibba032 bash[24004]: audit 2021-03-10T14:26:40.644335+0000 mgr.gibba032.whopeo (mgr.24625) 1833 : audit [DBG] from='client.55264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:41 gibba032 bash[24004]: audit 2021-03-10T14:26:40.870068+0000 mgr.gibba032.whopeo (mgr.24625) 1834 : audit [DBG] from='client.55270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:41 gibba032 bash[24004]: audit 2021-03-10T14:26:41.165743+0000 mon.c (mon.1) 36 : audit [DBG] from='client.? 172.21.2.111:0/2535185353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[30791]: audit 2021-03-10T14:26:40.644335+0000 mgr.gibba032.whopeo (mgr.24625) 1833 : audit [DBG] from='client.55264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[30791]: audit 2021-03-10T14:26:40.870068+0000 mgr.gibba032.whopeo (mgr.24625) 1834 : audit [DBG] from='client.55270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[30791]: audit 2021-03-10T14:26:41.165743+0000 mon.c (mon.1) 36 : audit [DBG] from='client.? 172.21.2.111:0/2535185353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[31774]: audit 2021-03-10T14:26:40.644335+0000 mgr.gibba032.whopeo (mgr.24625) 1833 : audit [DBG] from='client.55264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[31774]: audit 2021-03-10T14:26:40.870068+0000 mgr.gibba032.whopeo (mgr.24625) 1834 : audit [DBG] from='client.55270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:26:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:41 gibba011 bash[31774]: audit 2021-03-10T14:26:41.165743+0000 mon.c (mon.1) 36 : audit [DBG] from='client.? 172.21.2.111:0/2535185353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:26:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:42 gibba032 bash[24004]: cluster 2021-03-10T14:26:41.899250+0000 mgr.gibba032.whopeo (mgr.24625) 1835 : cluster [DBG] pgmap v1027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:42 gibba011 bash[30791]: cluster 2021-03-10T14:26:41.899250+0000 mgr.gibba032.whopeo (mgr.24625) 1835 : cluster [DBG] pgmap v1027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:43.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:42 gibba011 bash[31774]: cluster 2021-03-10T14:26:41.899250+0000 mgr.gibba032.whopeo (mgr.24625) 1835 : cluster [DBG] pgmap v1027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:45.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:45 gibba032 bash[24004]: cluster 2021-03-10T14:26:43.900912+0000 mgr.gibba032.whopeo (mgr.24625) 1836 : cluster [DBG] pgmap v1028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:45 gibba011 bash[30791]: cluster 2021-03-10T14:26:43.900912+0000 mgr.gibba032.whopeo (mgr.24625) 1836 : cluster [DBG] pgmap v1028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:45 gibba011 bash[31774]: cluster 2021-03-10T14:26:43.900912+0000 mgr.gibba032.whopeo (mgr.24625) 1836 : cluster [DBG] pgmap v1028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:46 gibba011 bash[30791]: cluster 2021-03-10T14:26:45.902191+0000 mgr.gibba032.whopeo (mgr.24625) 1837 : cluster [DBG] pgmap v1029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:46 gibba011 bash[31774]: cluster 2021-03-10T14:26:45.902191+0000 mgr.gibba032.whopeo (mgr.24625) 1837 : cluster [DBG] pgmap v1029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:46 gibba032 bash[24004]: cluster 2021-03-10T14:26:45.902191+0000 mgr.gibba032.whopeo (mgr.24625) 1837 : cluster [DBG] pgmap v1029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:48 gibba032 bash[24004]: cluster 2021-03-10T14:26:47.903821+0000 mgr.gibba032.whopeo (mgr.24625) 1838 : cluster [DBG] pgmap v1030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:48 gibba011 bash[30791]: cluster 2021-03-10T14:26:47.903821+0000 mgr.gibba032.whopeo (mgr.24625) 1838 : cluster [DBG] pgmap v1030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:48 gibba011 bash[31774]: cluster 2021-03-10T14:26:47.903821+0000 mgr.gibba032.whopeo (mgr.24625) 1838 : cluster [DBG] pgmap v1030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:50 gibba032 bash[24004]: cluster 2021-03-10T14:26:49.905300+0000 mgr.gibba032.whopeo (mgr.24625) 1839 : cluster [DBG] pgmap v1031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:50 gibba011 bash[30791]: cluster 2021-03-10T14:26:49.905300+0000 mgr.gibba032.whopeo (mgr.24625) 1839 : cluster [DBG] pgmap v1031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:50 gibba011 bash[31774]: cluster 2021-03-10T14:26:49.905300+0000 mgr.gibba032.whopeo (mgr.24625) 1839 : cluster [DBG] pgmap v1031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:53.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:52 gibba032 bash[24004]: cluster 2021-03-10T14:26:51.907030+0000 mgr.gibba032.whopeo (mgr.24625) 1840 : cluster [DBG] pgmap v1032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:52 gibba011 bash[30791]: cluster 2021-03-10T14:26:51.907030+0000 mgr.gibba032.whopeo (mgr.24625) 1840 : cluster [DBG] pgmap v1032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:52 gibba011 bash[31774]: cluster 2021-03-10T14:26:51.907030+0000 mgr.gibba032.whopeo (mgr.24625) 1840 : cluster [DBG] pgmap v1032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:54 gibba032 bash[24004]: cluster 2021-03-10T14:26:53.908624+0000 mgr.gibba032.whopeo (mgr.24625) 1841 : cluster [DBG] pgmap v1033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:54 gibba032 bash[24004]: cephadm 2021-03-10T14:26:53.982337+0000 mgr.gibba032.whopeo (mgr.24625) 1842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:54 gibba011 bash[30791]: cluster 2021-03-10T14:26:53.908624+0000 mgr.gibba032.whopeo (mgr.24625) 1841 : cluster [DBG] pgmap v1033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:54 gibba011 bash[30791]: cephadm 2021-03-10T14:26:53.982337+0000 mgr.gibba032.whopeo (mgr.24625) 1842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:54 gibba011 bash[31774]: cluster 2021-03-10T14:26:53.908624+0000 mgr.gibba032.whopeo (mgr.24625) 1841 : cluster [DBG] pgmap v1033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:54 gibba011 bash[31774]: cephadm 2021-03-10T14:26:53.982337+0000 mgr.gibba032.whopeo (mgr.24625) 1842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:26:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: cluster 2021-03-10T14:26:55.909991+0000 mgr.gibba032.whopeo (mgr.24625) 1843 : cluster [DBG] pgmap v1034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.183621+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T14:26:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.184405+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:26:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.185068+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.185743+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.215559+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:26:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[30791]: audit 2021-03-10T14:26:56.216170+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:26:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: cluster 2021-03-10T14:26:55.909991+0000 mgr.gibba032.whopeo (mgr.24625) 1843 : cluster [DBG] pgmap v1034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:57.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.183621+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T14:26:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.184405+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:26:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.185068+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.185743+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.215559+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:26:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:56 gibba011 bash[31774]: audit 2021-03-10T14:26:56.216170+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:26:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: cluster 2021-03-10T14:26:55.909991+0000 mgr.gibba032.whopeo (mgr.24625) 1843 : cluster [DBG] pgmap v1034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.183621+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T14:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.184405+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.185068+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.185743+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.215559+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:26:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:56 gibba032 bash[24004]: audit 2021-03-10T14:26:56.216170+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:26:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:26:58 gibba032 bash[24004]: cluster 2021-03-10T14:26:57.911747+0000 mgr.gibba032.whopeo (mgr.24625) 1844 : cluster [DBG] pgmap v1035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:26:58 gibba011 bash[30791]: cluster 2021-03-10T14:26:57.911747+0000 mgr.gibba032.whopeo (mgr.24625) 1844 : cluster [DBG] pgmap v1035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:26:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:26:58 gibba011 bash[31774]: cluster 2021-03-10T14:26:57.911747+0000 mgr.gibba032.whopeo (mgr.24625) 1844 : cluster [DBG] pgmap v1035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:01.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:00 gibba032 bash[24004]: cluster 2021-03-10T14:26:59.913251+0000 mgr.gibba032.whopeo (mgr.24625) 1845 : cluster [DBG] pgmap v1036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:00 gibba011 bash[30791]: cluster 2021-03-10T14:26:59.913251+0000 mgr.gibba032.whopeo (mgr.24625) 1845 : cluster [DBG] pgmap v1036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:00 gibba011 bash[31774]: cluster 2021-03-10T14:26:59.913251+0000 mgr.gibba032.whopeo (mgr.24625) 1845 : cluster [DBG] pgmap v1036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:02 gibba032 bash[24004]: cluster 2021-03-10T14:27:01.914986+0000 mgr.gibba032.whopeo (mgr.24625) 1846 : cluster [DBG] pgmap v1037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:02 gibba011 bash[30791]: cluster 2021-03-10T14:27:01.914986+0000 mgr.gibba032.whopeo (mgr.24625) 1846 : cluster [DBG] pgmap v1037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:02 gibba011 bash[31774]: cluster 2021-03-10T14:27:01.914986+0000 mgr.gibba032.whopeo (mgr.24625) 1846 : cluster [DBG] pgmap v1037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:05.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:05 gibba032 bash[24004]: cluster 2021-03-10T14:27:03.916671+0000 mgr.gibba032.whopeo (mgr.24625) 1847 : cluster [DBG] pgmap v1038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:05 gibba011 bash[30791]: cluster 2021-03-10T14:27:03.916671+0000 mgr.gibba032.whopeo (mgr.24625) 1847 : cluster [DBG] pgmap v1038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:05 gibba011 bash[31774]: cluster 2021-03-10T14:27:03.916671+0000 mgr.gibba032.whopeo (mgr.24625) 1847 : cluster [DBG] pgmap v1038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:06 gibba032 bash[24004]: cluster 2021-03-10T14:27:05.918024+0000 mgr.gibba032.whopeo (mgr.24625) 1848 : cluster [DBG] pgmap v1039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:06 gibba011 bash[30791]: cluster 2021-03-10T14:27:05.918024+0000 mgr.gibba032.whopeo (mgr.24625) 1848 : cluster [DBG] pgmap v1039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:06 gibba011 bash[31774]: cluster 2021-03-10T14:27:05.918024+0000 mgr.gibba032.whopeo (mgr.24625) 1848 : cluster [DBG] pgmap v1039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:08 gibba032 bash[24004]: cluster 2021-03-10T14:27:07.919802+0000 mgr.gibba032.whopeo (mgr.24625) 1849 : cluster [DBG] pgmap v1040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:08 gibba011 bash[30791]: cluster 2021-03-10T14:27:07.919802+0000 mgr.gibba032.whopeo (mgr.24625) 1849 : cluster [DBG] pgmap v1040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:08 gibba011 bash[31774]: cluster 2021-03-10T14:27:07.919802+0000 mgr.gibba032.whopeo (mgr.24625) 1849 : cluster [DBG] pgmap v1040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:09 gibba032 bash[24004]: cephadm 2021-03-10T14:27:08.997944+0000 mgr.gibba032.whopeo (mgr.24625) 1850 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:09 gibba011 bash[30791]: cephadm 2021-03-10T14:27:08.997944+0000 mgr.gibba032.whopeo (mgr.24625) 1850 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:10.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:09 gibba011 bash[31774]: cephadm 2021-03-10T14:27:08.997944+0000 mgr.gibba032.whopeo (mgr.24625) 1850 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:11 gibba032 bash[24004]: cluster 2021-03-10T14:27:09.921323+0000 mgr.gibba032.whopeo (mgr.24625) 1851 : cluster [DBG] pgmap v1041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:11 gibba011 bash[30791]: cluster 2021-03-10T14:27:09.921323+0000 mgr.gibba032.whopeo (mgr.24625) 1851 : cluster [DBG] pgmap v1041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:11 gibba011 bash[31774]: cluster 2021-03-10T14:27:09.921323+0000 mgr.gibba032.whopeo (mgr.24625) 1851 : cluster [DBG] pgmap v1041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:11.408 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:27:11.634 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:27:11.634 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (33m) 9m ago 39m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:27:11.635 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (39m) 8m ago 39m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:27:11.635 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (34m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:27:11.635 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (34m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:27:11.635 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (33m) 9m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:27:11.635 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (33m) 9m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (32m) 8m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (33m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (39m) 9m ago 39m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (39m) 8m ago 39m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (32m) 9m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:27:11.636 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (31m) 9m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:27:11.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (31m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:27:11.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (30m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:27:11.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (30m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:27:11.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (30m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:27:11.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (29m) 8m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:27:11.638 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (29m) 8m ago 39m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:27:11.638 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (38m) 8m ago 39m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:27:11.638 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 37m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:27:11.885 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:27:11.886 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:27:11.886 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:27:11.886 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:11.886 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:27:11.887 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:27:11.887 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:11.887 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:27:11.887 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:27:11.887 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:11.888 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:27:11.888 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:27:11.888 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:27:11.888 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:27:11.888 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:27:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:12 gibba011 bash[30791]: audit 2021-03-10T14:27:11.885322+0000 mon.a (mon.0) 1093 : audit [DBG] from='client.? 172.21.2.111:0/1384985557' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:12 gibba011 bash[31774]: audit 2021-03-10T14:27:11.885322+0000 mon.a (mon.0) 1093 : audit [DBG] from='client.? 172.21.2.111:0/1384985557' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:12 gibba032 bash[24004]: audit 2021-03-10T14:27:11.885322+0000 mon.a (mon.0) 1093 : audit [DBG] from='client.? 172.21.2.111:0/1384985557' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[30791]: audit 2021-03-10T14:27:11.396190+0000 mgr.gibba032.whopeo (mgr.24625) 1852 : audit [DBG] from='client.45297 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[30791]: audit 2021-03-10T14:27:11.618370+0000 mgr.gibba032.whopeo (mgr.24625) 1853 : audit [DBG] from='client.45303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[30791]: cluster 2021-03-10T14:27:11.923027+0000 mgr.gibba032.whopeo (mgr.24625) 1854 : cluster [DBG] pgmap v1042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[31774]: audit 2021-03-10T14:27:11.396190+0000 mgr.gibba032.whopeo (mgr.24625) 1852 : audit [DBG] from='client.45297 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[31774]: audit 2021-03-10T14:27:11.618370+0000 mgr.gibba032.whopeo (mgr.24625) 1853 : audit [DBG] from='client.45303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:13 gibba011 bash[31774]: cluster 2021-03-10T14:27:11.923027+0000 mgr.gibba032.whopeo (mgr.24625) 1854 : cluster [DBG] pgmap v1042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:13 gibba032 bash[24004]: audit 2021-03-10T14:27:11.396190+0000 mgr.gibba032.whopeo (mgr.24625) 1852 : audit [DBG] from='client.45297 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:13 gibba032 bash[24004]: audit 2021-03-10T14:27:11.618370+0000 mgr.gibba032.whopeo (mgr.24625) 1853 : audit [DBG] from='client.45303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:13 gibba032 bash[24004]: cluster 2021-03-10T14:27:11.923027+0000 mgr.gibba032.whopeo (mgr.24625) 1854 : cluster [DBG] pgmap v1042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:15 gibba011 bash[30791]: cluster 2021-03-10T14:27:13.924648+0000 mgr.gibba032.whopeo (mgr.24625) 1855 : cluster [DBG] pgmap v1043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:15 gibba011 bash[31774]: cluster 2021-03-10T14:27:13.924648+0000 mgr.gibba032.whopeo (mgr.24625) 1855 : cluster [DBG] pgmap v1043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:15 gibba032 bash[24004]: cluster 2021-03-10T14:27:13.924648+0000 mgr.gibba032.whopeo (mgr.24625) 1855 : cluster [DBG] pgmap v1043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:16 gibba032 bash[24004]: cluster 2021-03-10T14:27:15.925978+0000 mgr.gibba032.whopeo (mgr.24625) 1856 : cluster [DBG] pgmap v1044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:16 gibba011 bash[30791]: cluster 2021-03-10T14:27:15.925978+0000 mgr.gibba032.whopeo (mgr.24625) 1856 : cluster [DBG] pgmap v1044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:16 gibba011 bash[31774]: cluster 2021-03-10T14:27:15.925978+0000 mgr.gibba032.whopeo (mgr.24625) 1856 : cluster [DBG] pgmap v1044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:18 gibba032 bash[24004]: cluster 2021-03-10T14:27:17.927722+0000 mgr.gibba032.whopeo (mgr.24625) 1857 : cluster [DBG] pgmap v1045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:18 gibba011 bash[30791]: cluster 2021-03-10T14:27:17.927722+0000 mgr.gibba032.whopeo (mgr.24625) 1857 : cluster [DBG] pgmap v1045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:18 gibba011 bash[31774]: cluster 2021-03-10T14:27:17.927722+0000 mgr.gibba032.whopeo (mgr.24625) 1857 : cluster [DBG] pgmap v1045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:21.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:20 gibba032 bash[24004]: cluster 2021-03-10T14:27:19.929242+0000 mgr.gibba032.whopeo (mgr.24625) 1858 : cluster [DBG] pgmap v1046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:21 gibba011 bash[30791]: cluster 2021-03-10T14:27:19.929242+0000 mgr.gibba032.whopeo (mgr.24625) 1858 : cluster [DBG] pgmap v1046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:21 gibba011 bash[31774]: cluster 2021-03-10T14:27:19.929242+0000 mgr.gibba032.whopeo (mgr.24625) 1858 : cluster [DBG] pgmap v1046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:23 gibba011 bash[30791]: cluster 2021-03-10T14:27:21.930992+0000 mgr.gibba032.whopeo (mgr.24625) 1859 : cluster [DBG] pgmap v1047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:23 gibba011 bash[31774]: cluster 2021-03-10T14:27:21.930992+0000 mgr.gibba032.whopeo (mgr.24625) 1859 : cluster [DBG] pgmap v1047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:23 gibba032 bash[24004]: cluster 2021-03-10T14:27:21.930992+0000 mgr.gibba032.whopeo (mgr.24625) 1859 : cluster [DBG] pgmap v1047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:25 gibba011 bash[30791]: cluster 2021-03-10T14:27:23.932687+0000 mgr.gibba032.whopeo (mgr.24625) 1860 : cluster [DBG] pgmap v1048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:25 gibba011 bash[30791]: cephadm 2021-03-10T14:27:24.012776+0000 mgr.gibba032.whopeo (mgr.24625) 1861 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:25 gibba011 bash[31774]: cluster 2021-03-10T14:27:23.932687+0000 mgr.gibba032.whopeo (mgr.24625) 1860 : cluster [DBG] pgmap v1048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:25 gibba011 bash[31774]: cephadm 2021-03-10T14:27:24.012776+0000 mgr.gibba032.whopeo (mgr.24625) 1861 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:25 gibba032 bash[24004]: cluster 2021-03-10T14:27:23.932687+0000 mgr.gibba032.whopeo (mgr.24625) 1860 : cluster [DBG] pgmap v1048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:25 gibba032 bash[24004]: cephadm 2021-03-10T14:27:24.012776+0000 mgr.gibba032.whopeo (mgr.24625) 1861 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:26 gibba032 bash[24004]: cluster 2021-03-10T14:27:25.934095+0000 mgr.gibba032.whopeo (mgr.24625) 1862 : cluster [DBG] pgmap v1049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:26 gibba011 bash[30791]: cluster 2021-03-10T14:27:25.934095+0000 mgr.gibba032.whopeo (mgr.24625) 1862 : cluster [DBG] pgmap v1049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:26 gibba011 bash[31774]: cluster 2021-03-10T14:27:25.934095+0000 mgr.gibba032.whopeo (mgr.24625) 1862 : cluster [DBG] pgmap v1049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:28 gibba032 bash[24004]: cluster 2021-03-10T14:27:27.935763+0000 mgr.gibba032.whopeo (mgr.24625) 1863 : cluster [DBG] pgmap v1050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:29 gibba011 bash[30791]: cluster 2021-03-10T14:27:27.935763+0000 mgr.gibba032.whopeo (mgr.24625) 1863 : cluster [DBG] pgmap v1050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:29 gibba011 bash[31774]: cluster 2021-03-10T14:27:27.935763+0000 mgr.gibba032.whopeo (mgr.24625) 1863 : cluster [DBG] pgmap v1050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:31 gibba011 bash[30791]: cluster 2021-03-10T14:27:29.937274+0000 mgr.gibba032.whopeo (mgr.24625) 1864 : cluster [DBG] pgmap v1051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:31 gibba011 bash[31774]: cluster 2021-03-10T14:27:29.937274+0000 mgr.gibba032.whopeo (mgr.24625) 1864 : cluster [DBG] pgmap v1051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:31.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:31 gibba032 bash[24004]: cluster 2021-03-10T14:27:29.937274+0000 mgr.gibba032.whopeo (mgr.24625) 1864 : cluster [DBG] pgmap v1051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:33 gibba011 bash[30791]: cluster 2021-03-10T14:27:31.939010+0000 mgr.gibba032.whopeo (mgr.24625) 1865 : cluster [DBG] pgmap v1052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:33 gibba011 bash[31774]: cluster 2021-03-10T14:27:31.939010+0000 mgr.gibba032.whopeo (mgr.24625) 1865 : cluster [DBG] pgmap v1052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:33 gibba032 bash[24004]: cluster 2021-03-10T14:27:31.939010+0000 mgr.gibba032.whopeo (mgr.24625) 1865 : cluster [DBG] pgmap v1052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:35 gibba011 bash[30791]: cluster 2021-03-10T14:27:33.940647+0000 mgr.gibba032.whopeo (mgr.24625) 1866 : cluster [DBG] pgmap v1053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:35 gibba011 bash[31774]: cluster 2021-03-10T14:27:33.940647+0000 mgr.gibba032.whopeo (mgr.24625) 1866 : cluster [DBG] pgmap v1053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:35 gibba032 bash[24004]: cluster 2021-03-10T14:27:33.940647+0000 mgr.gibba032.whopeo (mgr.24625) 1866 : cluster [DBG] pgmap v1053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:36 gibba032 bash[24004]: cluster 2021-03-10T14:27:35.941958+0000 mgr.gibba032.whopeo (mgr.24625) 1867 : cluster [DBG] pgmap v1054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:36 gibba011 bash[30791]: cluster 2021-03-10T14:27:35.941958+0000 mgr.gibba032.whopeo (mgr.24625) 1867 : cluster [DBG] pgmap v1054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:36 gibba011 bash[31774]: cluster 2021-03-10T14:27:35.941958+0000 mgr.gibba032.whopeo (mgr.24625) 1867 : cluster [DBG] pgmap v1054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:39 gibba011 bash[30791]: cluster 2021-03-10T14:27:37.943678+0000 mgr.gibba032.whopeo (mgr.24625) 1868 : cluster [DBG] pgmap v1055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:39 gibba011 bash[31774]: cluster 2021-03-10T14:27:37.943678+0000 mgr.gibba032.whopeo (mgr.24625) 1868 : cluster [DBG] pgmap v1055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:39.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:39 gibba032 bash[24004]: cluster 2021-03-10T14:27:37.943678+0000 mgr.gibba032.whopeo (mgr.24625) 1868 : cluster [DBG] pgmap v1055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:40 gibba011 bash[30791]: cephadm 2021-03-10T14:27:39.028698+0000 mgr.gibba032.whopeo (mgr.24625) 1869 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:27:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:40 gibba011 bash[30791]: audit 2021-03-10T14:27:39.032369+0000 mon.a (mon.0) 1094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:27:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:40 gibba011 bash[31774]: cephadm 2021-03-10T14:27:39.028698+0000 mgr.gibba032.whopeo (mgr.24625) 1869 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:27:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:40 gibba011 bash[31774]: audit 2021-03-10T14:27:39.032369+0000 mon.a (mon.0) 1094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:27:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:40 gibba032 bash[24004]: cephadm 2021-03-10T14:27:39.028698+0000 mgr.gibba032.whopeo (mgr.24625) 1869 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:27:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:40 gibba032 bash[24004]: audit 2021-03-10T14:27:39.032369+0000 mon.a (mon.0) 1094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:27:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:41 gibba011 bash[30791]: cluster 2021-03-10T14:27:39.945319+0000 mgr.gibba032.whopeo (mgr.24625) 1870 : cluster [DBG] pgmap v1056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:41 gibba011 bash[31774]: cluster 2021-03-10T14:27:39.945319+0000 mgr.gibba032.whopeo (mgr.24625) 1870 : cluster [DBG] pgmap v1056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:41.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:41 gibba032 bash[24004]: cluster 2021-03-10T14:27:39.945319+0000 mgr.gibba032.whopeo (mgr.24625) 1870 : cluster [DBG] pgmap v1056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:42.176 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:27:42.438 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:27:42.438 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (34m) 10m ago 39m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:27:42.439 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (39m) 9m ago 39m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:27:42.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (34m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:27:42.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (34m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:27:42.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (34m) 10m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:27:42.440 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (33m) 10m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:27:42.440 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (33m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:27:42.440 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (33m) 10m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:27:42.440 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (39m) 10m ago 39m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:27:42.441 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (39m) 9m ago 39m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:27:42.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (32m) 10m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:27:42.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (32m) 10m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:27:42.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (31m) 10m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:27:42.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (31m) 10m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:27:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (30m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:27:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (30m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:27:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (30m) 9m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:27:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (29m) 9m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:27:42.443 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (39m) 9m ago 39m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:27:42.443 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 37m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:27:42.723 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:27:42.723 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:27:42.723 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:27:42.724 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:42.724 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:27:42.724 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:27:42.724 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:42.724 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:27:42.725 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:27:42.725 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:27:42.725 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:27:42.725 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:27:42.725 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:27:42.726 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:27:42.726 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:27:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: cluster 2021-03-10T14:27:41.946863+0000 mgr.gibba032.whopeo (mgr.24625) 1871 : cluster [DBG] pgmap v1057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.521931+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.722675+0000 mon.c (mon.1) 37 : audit [DBG] from='client.? 172.21.2.111:0/1881911338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.739589+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.747780+0000 mon.a (mon.0) 1097 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:27:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.750357+0000 mon.a (mon.0) 1098 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.751993+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.753390+0000 mon.a (mon.0) 1100 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.755673+0000 mon.a (mon.0) 1101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[30791]: audit 2021-03-10T14:27:42.757341+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: cluster 2021-03-10T14:27:41.946863+0000 mgr.gibba032.whopeo (mgr.24625) 1871 : cluster [DBG] pgmap v1057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.521931+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.722675+0000 mon.c (mon.1) 37 : audit [DBG] from='client.? 172.21.2.111:0/1881911338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.739589+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.747780+0000 mon.a (mon.0) 1097 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:27:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.750357+0000 mon.a (mon.0) 1098 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.751993+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.753390+0000 mon.a (mon.0) 1100 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.755673+0000 mon.a (mon.0) 1101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:43 gibba011 bash[31774]: audit 2021-03-10T14:27:42.757341+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: cluster 2021-03-10T14:27:41.946863+0000 mgr.gibba032.whopeo (mgr.24625) 1871 : cluster [DBG] pgmap v1057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.521931+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.722675+0000 mon.c (mon.1) 37 : audit [DBG] from='client.? 172.21.2.111:0/1881911338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.739589+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:27:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.747780+0000 mon.a (mon.0) 1097 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:27:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.750357+0000 mon.a (mon.0) 1098 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.751993+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.753390+0000 mon.a (mon.0) 1100 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.755673+0000 mon.a (mon.0) 1101 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:43.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:43 gibba032 bash[24004]: audit 2021-03-10T14:27:42.757341+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:27:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: audit 2021-03-10T14:27:42.164599+0000 mgr.gibba032.whopeo (mgr.24625) 1872 : audit [DBG] from='client.45315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: audit 2021-03-10T14:27:42.422554+0000 mgr.gibba032.whopeo (mgr.24625) 1873 : audit [DBG] from='client.55306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.519251+0000 mgr.gibba032.whopeo (mgr.24625) 1874 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:27:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.743735+0000 mgr.gibba032.whopeo (mgr.24625) 1875 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:27:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.746450+0000 mgr.gibba032.whopeo (mgr.24625) 1876 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:27:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.748244+0000 mgr.gibba032.whopeo (mgr.24625) 1877 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:27:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.748281+0000 mgr.gibba032.whopeo (mgr.24625) 1878 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:27:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.749738+0000 mgr.gibba032.whopeo (mgr.24625) 1879 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:27:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.749775+0000 mgr.gibba032.whopeo (mgr.24625) 1880 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:27:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.750909+0000 mgr.gibba032.whopeo (mgr.24625) 1881 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:27:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.751913+0000 mgr.gibba032.whopeo (mgr.24625) 1882 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:27:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.751953+0000 mgr.gibba032.whopeo (mgr.24625) 1883 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:27:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.753311+0000 mgr.gibba032.whopeo (mgr.24625) 1884 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:27:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.753745+0000 mgr.gibba032.whopeo (mgr.24625) 1885 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:27:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.754972+0000 mgr.gibba032.whopeo (mgr.24625) 1886 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:27:44.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.755014+0000 mgr.gibba032.whopeo (mgr.24625) 1887 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:27:44.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[30791]: cephadm 2021-03-10T14:27:42.755131+0000 mgr.gibba032.whopeo (mgr.24625) 1888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: audit 2021-03-10T14:27:42.164599+0000 mgr.gibba032.whopeo (mgr.24625) 1872 : audit [DBG] from='client.45315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: audit 2021-03-10T14:27:42.422554+0000 mgr.gibba032.whopeo (mgr.24625) 1873 : audit [DBG] from='client.55306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.519251+0000 mgr.gibba032.whopeo (mgr.24625) 1874 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:27:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.743735+0000 mgr.gibba032.whopeo (mgr.24625) 1875 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:27:44.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.746450+0000 mgr.gibba032.whopeo (mgr.24625) 1876 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:27:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.748244+0000 mgr.gibba032.whopeo (mgr.24625) 1877 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:27:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.748281+0000 mgr.gibba032.whopeo (mgr.24625) 1878 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:27:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.749738+0000 mgr.gibba032.whopeo (mgr.24625) 1879 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:27:44.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.749775+0000 mgr.gibba032.whopeo (mgr.24625) 1880 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:27:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.750909+0000 mgr.gibba032.whopeo (mgr.24625) 1881 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:27:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.751913+0000 mgr.gibba032.whopeo (mgr.24625) 1882 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:27:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.751953+0000 mgr.gibba032.whopeo (mgr.24625) 1883 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:27:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.753311+0000 mgr.gibba032.whopeo (mgr.24625) 1884 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:27:44.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.753745+0000 mgr.gibba032.whopeo (mgr.24625) 1885 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:27:44.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.754972+0000 mgr.gibba032.whopeo (mgr.24625) 1886 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:27:44.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.755014+0000 mgr.gibba032.whopeo (mgr.24625) 1887 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:27:44.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:44 gibba011 bash[31774]: cephadm 2021-03-10T14:27:42.755131+0000 mgr.gibba032.whopeo (mgr.24625) 1888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: audit 2021-03-10T14:27:42.164599+0000 mgr.gibba032.whopeo (mgr.24625) 1872 : audit [DBG] from='client.45315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: audit 2021-03-10T14:27:42.422554+0000 mgr.gibba032.whopeo (mgr.24625) 1873 : audit [DBG] from='client.55306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:27:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.519251+0000 mgr.gibba032.whopeo (mgr.24625) 1874 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:27:44.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.743735+0000 mgr.gibba032.whopeo (mgr.24625) 1875 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:27:44.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.746450+0000 mgr.gibba032.whopeo (mgr.24625) 1876 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:27:44.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.748244+0000 mgr.gibba032.whopeo (mgr.24625) 1877 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:27:44.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.748281+0000 mgr.gibba032.whopeo (mgr.24625) 1878 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:27:44.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.749738+0000 mgr.gibba032.whopeo (mgr.24625) 1879 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:27:44.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.749775+0000 mgr.gibba032.whopeo (mgr.24625) 1880 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:27:44.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.750909+0000 mgr.gibba032.whopeo (mgr.24625) 1881 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:27:44.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.751913+0000 mgr.gibba032.whopeo (mgr.24625) 1882 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:27:44.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.751953+0000 mgr.gibba032.whopeo (mgr.24625) 1883 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:27:44.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.753311+0000 mgr.gibba032.whopeo (mgr.24625) 1884 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:27:44.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.753745+0000 mgr.gibba032.whopeo (mgr.24625) 1885 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:27:44.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.754972+0000 mgr.gibba032.whopeo (mgr.24625) 1886 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:27:44.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.755014+0000 mgr.gibba032.whopeo (mgr.24625) 1887 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:27:44.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:44 gibba032 bash[24004]: cephadm 2021-03-10T14:27:42.755131+0000 mgr.gibba032.whopeo (mgr.24625) 1888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:45 gibba011 bash[30791]: cluster 2021-03-10T14:27:43.948390+0000 mgr.gibba032.whopeo (mgr.24625) 1889 : cluster [DBG] pgmap v1058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:45 gibba011 bash[31774]: cluster 2021-03-10T14:27:43.948390+0000 mgr.gibba032.whopeo (mgr.24625) 1889 : cluster [DBG] pgmap v1058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:45.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:45 gibba032 bash[24004]: cluster 2021-03-10T14:27:43.948390+0000 mgr.gibba032.whopeo (mgr.24625) 1889 : cluster [DBG] pgmap v1058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:46 gibba032 bash[24004]: cluster 2021-03-10T14:27:45.949763+0000 mgr.gibba032.whopeo (mgr.24625) 1890 : cluster [DBG] pgmap v1059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:46 gibba011 bash[30791]: cluster 2021-03-10T14:27:45.949763+0000 mgr.gibba032.whopeo (mgr.24625) 1890 : cluster [DBG] pgmap v1059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:46 gibba011 bash[31774]: cluster 2021-03-10T14:27:45.949763+0000 mgr.gibba032.whopeo (mgr.24625) 1890 : cluster [DBG] pgmap v1059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:49 gibba011 bash[30791]: cluster 2021-03-10T14:27:47.951506+0000 mgr.gibba032.whopeo (mgr.24625) 1891 : cluster [DBG] pgmap v1060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:49 gibba011 bash[31774]: cluster 2021-03-10T14:27:47.951506+0000 mgr.gibba032.whopeo (mgr.24625) 1891 : cluster [DBG] pgmap v1060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:49 gibba032 bash[24004]: cluster 2021-03-10T14:27:47.951506+0000 mgr.gibba032.whopeo (mgr.24625) 1891 : cluster [DBG] pgmap v1060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:51 gibba011 bash[30791]: cluster 2021-03-10T14:27:49.952952+0000 mgr.gibba032.whopeo (mgr.24625) 1892 : cluster [DBG] pgmap v1061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:51 gibba011 bash[31774]: cluster 2021-03-10T14:27:49.952952+0000 mgr.gibba032.whopeo (mgr.24625) 1892 : cluster [DBG] pgmap v1061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:51 gibba032 bash[24004]: cluster 2021-03-10T14:27:49.952952+0000 mgr.gibba032.whopeo (mgr.24625) 1892 : cluster [DBG] pgmap v1061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:53 gibba011 bash[30791]: cluster 2021-03-10T14:27:51.954542+0000 mgr.gibba032.whopeo (mgr.24625) 1893 : cluster [DBG] pgmap v1062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:53 gibba011 bash[31774]: cluster 2021-03-10T14:27:51.954542+0000 mgr.gibba032.whopeo (mgr.24625) 1893 : cluster [DBG] pgmap v1062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:53 gibba032 bash[24004]: cluster 2021-03-10T14:27:51.954542+0000 mgr.gibba032.whopeo (mgr.24625) 1893 : cluster [DBG] pgmap v1062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:55 gibba011 bash[30791]: cluster 2021-03-10T14:27:53.956062+0000 mgr.gibba032.whopeo (mgr.24625) 1894 : cluster [DBG] pgmap v1063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:55.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:55 gibba011 bash[31774]: cluster 2021-03-10T14:27:53.956062+0000 mgr.gibba032.whopeo (mgr.24625) 1894 : cluster [DBG] pgmap v1063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:55 gibba032 bash[24004]: cluster 2021-03-10T14:27:53.956062+0000 mgr.gibba032.whopeo (mgr.24625) 1894 : cluster [DBG] pgmap v1063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: cluster 2021-03-10T14:27:55.957341+0000 mgr.gibba032.whopeo (mgr.24625) 1895 : cluster [DBG] pgmap v1064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.191897+0000 mon.a (mon.0) 1103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:27:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.192739+0000 mon.a (mon.0) 1104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.193604+0000 mon.a (mon.0) 1105 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T14:27:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.194438+0000 mon.a (mon.0) 1106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.219268+0000 mon.a (mon.0) 1107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:27:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:56 gibba032 bash[24004]: audit 2021-03-10T14:27:56.219965+0000 mon.a (mon.0) 1108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:27:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: cluster 2021-03-10T14:27:55.957341+0000 mgr.gibba032.whopeo (mgr.24625) 1895 : cluster [DBG] pgmap v1064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.191897+0000 mon.a (mon.0) 1103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:27:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.192739+0000 mon.a (mon.0) 1104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.193604+0000 mon.a (mon.0) 1105 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T14:27:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.194438+0000 mon.a (mon.0) 1106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.219268+0000 mon.a (mon.0) 1107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:27:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[30791]: audit 2021-03-10T14:27:56.219965+0000 mon.a (mon.0) 1108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:27:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: cluster 2021-03-10T14:27:55.957341+0000 mgr.gibba032.whopeo (mgr.24625) 1895 : cluster [DBG] pgmap v1064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.191897+0000 mon.a (mon.0) 1103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:27:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.192739+0000 mon.a (mon.0) 1104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.193604+0000 mon.a (mon.0) 1105 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T14:27:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.194438+0000 mon.a (mon.0) 1106 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:27:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.219268+0000 mon.a (mon.0) 1107 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:27:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:56 gibba011 bash[31774]: audit 2021-03-10T14:27:56.219965+0000 mon.a (mon.0) 1108 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:27:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:59 gibba011 bash[30791]: cephadm 2021-03-10T14:27:57.769089+0000 mgr.gibba032.whopeo (mgr.24625) 1896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:27:59 gibba011 bash[30791]: cluster 2021-03-10T14:27:57.958970+0000 mgr.gibba032.whopeo (mgr.24625) 1897 : cluster [DBG] pgmap v1065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:59 gibba011 bash[31774]: cephadm 2021-03-10T14:27:57.769089+0000 mgr.gibba032.whopeo (mgr.24625) 1896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:27:59 gibba011 bash[31774]: cluster 2021-03-10T14:27:57.958970+0000 mgr.gibba032.whopeo (mgr.24625) 1897 : cluster [DBG] pgmap v1065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:27:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:59 gibba032 bash[24004]: cephadm 2021-03-10T14:27:57.769089+0000 mgr.gibba032.whopeo (mgr.24625) 1896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:27:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:27:59 gibba032 bash[24004]: cluster 2021-03-10T14:27:57.958970+0000 mgr.gibba032.whopeo (mgr.24625) 1897 : cluster [DBG] pgmap v1065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:01 gibba011 bash[30791]: cluster 2021-03-10T14:27:59.960387+0000 mgr.gibba032.whopeo (mgr.24625) 1898 : cluster [DBG] pgmap v1066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:01 gibba011 bash[31774]: cluster 2021-03-10T14:27:59.960387+0000 mgr.gibba032.whopeo (mgr.24625) 1898 : cluster [DBG] pgmap v1066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:01 gibba032 bash[24004]: cluster 2021-03-10T14:27:59.960387+0000 mgr.gibba032.whopeo (mgr.24625) 1898 : cluster [DBG] pgmap v1066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:03 gibba011 bash[30791]: cluster 2021-03-10T14:28:01.962066+0000 mgr.gibba032.whopeo (mgr.24625) 1899 : cluster [DBG] pgmap v1067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:03.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:03 gibba011 bash[31774]: cluster 2021-03-10T14:28:01.962066+0000 mgr.gibba032.whopeo (mgr.24625) 1899 : cluster [DBG] pgmap v1067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:03 gibba032 bash[24004]: cluster 2021-03-10T14:28:01.962066+0000 mgr.gibba032.whopeo (mgr.24625) 1899 : cluster [DBG] pgmap v1067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:05 gibba011 bash[30791]: cluster 2021-03-10T14:28:03.963899+0000 mgr.gibba032.whopeo (mgr.24625) 1900 : cluster [DBG] pgmap v1068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:05 gibba011 bash[31774]: cluster 2021-03-10T14:28:03.963899+0000 mgr.gibba032.whopeo (mgr.24625) 1900 : cluster [DBG] pgmap v1068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:05 gibba032 bash[24004]: cluster 2021-03-10T14:28:03.963899+0000 mgr.gibba032.whopeo (mgr.24625) 1900 : cluster [DBG] pgmap v1068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:06 gibba032 bash[24004]: cluster 2021-03-10T14:28:05.965315+0000 mgr.gibba032.whopeo (mgr.24625) 1901 : cluster [DBG] pgmap v1069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:06 gibba011 bash[30791]: cluster 2021-03-10T14:28:05.965315+0000 mgr.gibba032.whopeo (mgr.24625) 1901 : cluster [DBG] pgmap v1069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:06 gibba011 bash[31774]: cluster 2021-03-10T14:28:05.965315+0000 mgr.gibba032.whopeo (mgr.24625) 1901 : cluster [DBG] pgmap v1069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:09 gibba011 bash[30791]: cluster 2021-03-10T14:28:07.967408+0000 mgr.gibba032.whopeo (mgr.24625) 1902 : cluster [DBG] pgmap v1070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:09 gibba011 bash[31774]: cluster 2021-03-10T14:28:07.967408+0000 mgr.gibba032.whopeo (mgr.24625) 1902 : cluster [DBG] pgmap v1070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:09.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:09 gibba032 bash[24004]: cluster 2021-03-10T14:28:07.967408+0000 mgr.gibba032.whopeo (mgr.24625) 1902 : cluster [DBG] pgmap v1070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:11 gibba011 bash[30791]: cluster 2021-03-10T14:28:09.968929+0000 mgr.gibba032.whopeo (mgr.24625) 1903 : cluster [DBG] pgmap v1071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:11 gibba011 bash[31774]: cluster 2021-03-10T14:28:09.968929+0000 mgr.gibba032.whopeo (mgr.24625) 1903 : cluster [DBG] pgmap v1071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:11.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:11 gibba032 bash[24004]: cluster 2021-03-10T14:28:09.968929+0000 mgr.gibba032.whopeo (mgr.24625) 1903 : cluster [DBG] pgmap v1071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:12.970 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:28:13.208 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:28:13.208 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (34m) 30s ago 40m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:28:13.208 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (40m) 9m ago 40m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:28:13.209 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (35m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:28:13.209 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (35m) 9m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:28:13.209 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (34m) 30s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:28:13.209 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (34m) 30s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:28:13.209 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (33m) 9m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:28:13.210 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (34m) 30s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:28:13.210 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (40m) 30s ago 40m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:28:13.210 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (40m) 9m ago 40m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:28:13.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (33m) 30s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:28:13.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (32m) 30s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:28:13.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (32m) 30s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:28:13.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (31m) 30s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:28:13.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (31m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:28:13.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (31m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:28:13.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (30m) 9m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:28:13.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (30m) 9m ago 40m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:28:13.212 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (39m) 9m ago 40m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:28:13.212 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 30s ago 38m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:28:13.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:13 gibba011 bash[30791]: cluster 2021-03-10T14:28:11.970487+0000 mgr.gibba032.whopeo (mgr.24625) 1904 : cluster [DBG] pgmap v1072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:13 gibba011 bash[31774]: cluster 2021-03-10T14:28:11.970487+0000 mgr.gibba032.whopeo (mgr.24625) 1904 : cluster [DBG] pgmap v1072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:13.485 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:28:13.485 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:28:13.485 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:28:13.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:13.486 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:28:13.486 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:28:13.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:13.486 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:28:13.486 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:28:13.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:13.487 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:28:13.487 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:28:13.487 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:28:13.487 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:28:13.488 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:28:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:13 gibba032 bash[24004]: cluster 2021-03-10T14:28:11.970487+0000 mgr.gibba032.whopeo (mgr.24625) 1904 : cluster [DBG] pgmap v1072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[30791]: cephadm 2021-03-10T14:28:12.785131+0000 mgr.gibba032.whopeo (mgr.24625) 1905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[30791]: audit 2021-03-10T14:28:12.957566+0000 mgr.gibba032.whopeo (mgr.24625) 1906 : audit [DBG] from='client.55318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[30791]: audit 2021-03-10T14:28:13.484620+0000 mon.a (mon.0) 1109 : audit [DBG] from='client.? 172.21.2.111:0/1723821767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:14.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[31774]: cephadm 2021-03-10T14:28:12.785131+0000 mgr.gibba032.whopeo (mgr.24625) 1905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:14.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[31774]: audit 2021-03-10T14:28:12.957566+0000 mgr.gibba032.whopeo (mgr.24625) 1906 : audit [DBG] from='client.55318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:14 gibba011 bash[31774]: audit 2021-03-10T14:28:13.484620+0000 mon.a (mon.0) 1109 : audit [DBG] from='client.? 172.21.2.111:0/1723821767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:14 gibba032 bash[24004]: cephadm 2021-03-10T14:28:12.785131+0000 mgr.gibba032.whopeo (mgr.24625) 1905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:14 gibba032 bash[24004]: audit 2021-03-10T14:28:12.957566+0000 mgr.gibba032.whopeo (mgr.24625) 1906 : audit [DBG] from='client.55318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:14 gibba032 bash[24004]: audit 2021-03-10T14:28:13.484620+0000 mon.a (mon.0) 1109 : audit [DBG] from='client.? 172.21.2.111:0/1723821767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:15 gibba011 bash[30791]: audit 2021-03-10T14:28:13.192119+0000 mgr.gibba032.whopeo (mgr.24625) 1907 : audit [DBG] from='client.55324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:15 gibba011 bash[30791]: cluster 2021-03-10T14:28:13.971977+0000 mgr.gibba032.whopeo (mgr.24625) 1908 : cluster [DBG] pgmap v1073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:15 gibba011 bash[31774]: audit 2021-03-10T14:28:13.192119+0000 mgr.gibba032.whopeo (mgr.24625) 1907 : audit [DBG] from='client.55324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:15 gibba011 bash[31774]: cluster 2021-03-10T14:28:13.971977+0000 mgr.gibba032.whopeo (mgr.24625) 1908 : cluster [DBG] pgmap v1073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:15 gibba032 bash[24004]: audit 2021-03-10T14:28:13.192119+0000 mgr.gibba032.whopeo (mgr.24625) 1907 : audit [DBG] from='client.55324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:15 gibba032 bash[24004]: cluster 2021-03-10T14:28:13.971977+0000 mgr.gibba032.whopeo (mgr.24625) 1908 : cluster [DBG] pgmap v1073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:16 gibba032 bash[24004]: cluster 2021-03-10T14:28:15.973574+0000 mgr.gibba032.whopeo (mgr.24625) 1909 : cluster [DBG] pgmap v1074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:16 gibba011 bash[30791]: cluster 2021-03-10T14:28:15.973574+0000 mgr.gibba032.whopeo (mgr.24625) 1909 : cluster [DBG] pgmap v1074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:16 gibba011 bash[31774]: cluster 2021-03-10T14:28:15.973574+0000 mgr.gibba032.whopeo (mgr.24625) 1909 : cluster [DBG] pgmap v1074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:19 gibba011 bash[30791]: cluster 2021-03-10T14:28:17.975531+0000 mgr.gibba032.whopeo (mgr.24625) 1910 : cluster [DBG] pgmap v1075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:19 gibba011 bash[31774]: cluster 2021-03-10T14:28:17.975531+0000 mgr.gibba032.whopeo (mgr.24625) 1910 : cluster [DBG] pgmap v1075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:19 gibba032 bash[24004]: cluster 2021-03-10T14:28:17.975531+0000 mgr.gibba032.whopeo (mgr.24625) 1910 : cluster [DBG] pgmap v1075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:21 gibba011 bash[30791]: cluster 2021-03-10T14:28:19.976715+0000 mgr.gibba032.whopeo (mgr.24625) 1911 : cluster [DBG] pgmap v1076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:21 gibba011 bash[31774]: cluster 2021-03-10T14:28:19.976715+0000 mgr.gibba032.whopeo (mgr.24625) 1911 : cluster [DBG] pgmap v1076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:21.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:21 gibba032 bash[24004]: cluster 2021-03-10T14:28:19.976715+0000 mgr.gibba032.whopeo (mgr.24625) 1911 : cluster [DBG] pgmap v1076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:23 gibba011 bash[30791]: cluster 2021-03-10T14:28:21.978322+0000 mgr.gibba032.whopeo (mgr.24625) 1912 : cluster [DBG] pgmap v1077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:23 gibba011 bash[31774]: cluster 2021-03-10T14:28:21.978322+0000 mgr.gibba032.whopeo (mgr.24625) 1912 : cluster [DBG] pgmap v1077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:23 gibba032 bash[24004]: cluster 2021-03-10T14:28:21.978322+0000 mgr.gibba032.whopeo (mgr.24625) 1912 : cluster [DBG] pgmap v1077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:25 gibba011 bash[30791]: cluster 2021-03-10T14:28:23.979926+0000 mgr.gibba032.whopeo (mgr.24625) 1913 : cluster [DBG] pgmap v1078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:25 gibba011 bash[31774]: cluster 2021-03-10T14:28:23.979926+0000 mgr.gibba032.whopeo (mgr.24625) 1913 : cluster [DBG] pgmap v1078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:25 gibba032 bash[24004]: cluster 2021-03-10T14:28:23.979926+0000 mgr.gibba032.whopeo (mgr.24625) 1913 : cluster [DBG] pgmap v1078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:26 gibba032 bash[24004]: cluster 2021-03-10T14:28:25.981219+0000 mgr.gibba032.whopeo (mgr.24625) 1914 : cluster [DBG] pgmap v1079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:26 gibba011 bash[30791]: cluster 2021-03-10T14:28:25.981219+0000 mgr.gibba032.whopeo (mgr.24625) 1914 : cluster [DBG] pgmap v1079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:26 gibba011 bash[31774]: cluster 2021-03-10T14:28:25.981219+0000 mgr.gibba032.whopeo (mgr.24625) 1914 : cluster [DBG] pgmap v1079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:29 gibba011 bash[30791]: cephadm 2021-03-10T14:28:27.801227+0000 mgr.gibba032.whopeo (mgr.24625) 1915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:29 gibba011 bash[30791]: cluster 2021-03-10T14:28:27.983120+0000 mgr.gibba032.whopeo (mgr.24625) 1916 : cluster [DBG] pgmap v1080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:29 gibba011 bash[31774]: cephadm 2021-03-10T14:28:27.801227+0000 mgr.gibba032.whopeo (mgr.24625) 1915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:29 gibba011 bash[31774]: cluster 2021-03-10T14:28:27.983120+0000 mgr.gibba032.whopeo (mgr.24625) 1916 : cluster [DBG] pgmap v1080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:29.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:29 gibba032 bash[24004]: cephadm 2021-03-10T14:28:27.801227+0000 mgr.gibba032.whopeo (mgr.24625) 1915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:29.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:29 gibba032 bash[24004]: cluster 2021-03-10T14:28:27.983120+0000 mgr.gibba032.whopeo (mgr.24625) 1916 : cluster [DBG] pgmap v1080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:31 gibba011 bash[30791]: cluster 2021-03-10T14:28:29.984506+0000 mgr.gibba032.whopeo (mgr.24625) 1917 : cluster [DBG] pgmap v1081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:31 gibba011 bash[31774]: cluster 2021-03-10T14:28:29.984506+0000 mgr.gibba032.whopeo (mgr.24625) 1917 : cluster [DBG] pgmap v1081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:31.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:31 gibba032 bash[24004]: cluster 2021-03-10T14:28:29.984506+0000 mgr.gibba032.whopeo (mgr.24625) 1917 : cluster [DBG] pgmap v1081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:33 gibba011 bash[30791]: cluster 2021-03-10T14:28:31.986063+0000 mgr.gibba032.whopeo (mgr.24625) 1918 : cluster [DBG] pgmap v1082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:33.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:33 gibba011 bash[31774]: cluster 2021-03-10T14:28:31.986063+0000 mgr.gibba032.whopeo (mgr.24625) 1918 : cluster [DBG] pgmap v1082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:33 gibba032 bash[24004]: cluster 2021-03-10T14:28:31.986063+0000 mgr.gibba032.whopeo (mgr.24625) 1918 : cluster [DBG] pgmap v1082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:34 gibba011 bash[30791]: cluster 2021-03-10T14:28:33.987549+0000 mgr.gibba032.whopeo (mgr.24625) 1919 : cluster [DBG] pgmap v1083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:34 gibba011 bash[31774]: cluster 2021-03-10T14:28:33.987549+0000 mgr.gibba032.whopeo (mgr.24625) 1919 : cluster [DBG] pgmap v1083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:34 gibba032 bash[24004]: cluster 2021-03-10T14:28:33.987549+0000 mgr.gibba032.whopeo (mgr.24625) 1919 : cluster [DBG] pgmap v1083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:36 gibba032 bash[24004]: cluster 2021-03-10T14:28:35.988900+0000 mgr.gibba032.whopeo (mgr.24625) 1920 : cluster [DBG] pgmap v1084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:36 gibba011 bash[30791]: cluster 2021-03-10T14:28:35.988900+0000 mgr.gibba032.whopeo (mgr.24625) 1920 : cluster [DBG] pgmap v1084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:36 gibba011 bash[31774]: cluster 2021-03-10T14:28:35.988900+0000 mgr.gibba032.whopeo (mgr.24625) 1920 : cluster [DBG] pgmap v1084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:39 gibba011 bash[30791]: cluster 2021-03-10T14:28:37.990818+0000 mgr.gibba032.whopeo (mgr.24625) 1921 : cluster [DBG] pgmap v1085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:39 gibba011 bash[31774]: cluster 2021-03-10T14:28:37.990818+0000 mgr.gibba032.whopeo (mgr.24625) 1921 : cluster [DBG] pgmap v1085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:39.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:39 gibba032 bash[24004]: cluster 2021-03-10T14:28:37.990818+0000 mgr.gibba032.whopeo (mgr.24625) 1921 : cluster [DBG] pgmap v1085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:41 gibba011 bash[30791]: cluster 2021-03-10T14:28:39.992313+0000 mgr.gibba032.whopeo (mgr.24625) 1922 : cluster [DBG] pgmap v1086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:41 gibba011 bash[31774]: cluster 2021-03-10T14:28:39.992313+0000 mgr.gibba032.whopeo (mgr.24625) 1922 : cluster [DBG] pgmap v1086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:41.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:41 gibba032 bash[24004]: cluster 2021-03-10T14:28:39.992313+0000 mgr.gibba032.whopeo (mgr.24625) 1922 : cluster [DBG] pgmap v1086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:42 gibba032 bash[24004]: cluster 2021-03-10T14:28:41.994100+0000 mgr.gibba032.whopeo (mgr.24625) 1923 : cluster [DBG] pgmap v1087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:42 gibba011 bash[30791]: cluster 2021-03-10T14:28:41.994100+0000 mgr.gibba032.whopeo (mgr.24625) 1923 : cluster [DBG] pgmap v1087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:42 gibba011 bash[31774]: cluster 2021-03-10T14:28:41.994100+0000 mgr.gibba032.whopeo (mgr.24625) 1923 : cluster [DBG] pgmap v1087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:43 gibba032 bash[24004]: cephadm 2021-03-10T14:28:42.804341+0000 mgr.gibba032.whopeo (mgr.24625) 1924 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:28:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:43 gibba032 bash[24004]: audit 2021-03-10T14:28:42.809661+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:28:43.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:43 gibba011 bash[30791]: cephadm 2021-03-10T14:28:42.804341+0000 mgr.gibba032.whopeo (mgr.24625) 1924 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:28:43.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:43 gibba011 bash[30791]: audit 2021-03-10T14:28:42.809661+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:28:43.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:43 gibba011 bash[31774]: cephadm 2021-03-10T14:28:42.804341+0000 mgr.gibba032.whopeo (mgr.24625) 1924 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:28:43.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:43 gibba011 bash[31774]: audit 2021-03-10T14:28:42.809661+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:28:43.724 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:28:43.952 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:28:43.953 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (35m) 61s ago 40m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:28:43.953 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (40m) 10m ago 40m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:28:43.953 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (35m) 10m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:28:43.953 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (36m) 10m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:28:43.954 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (35m) 61s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:28:43.954 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (34m) 61s ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:28:43.954 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (34m) 10m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:28:43.954 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (34m) 61s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:28:43.954 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (40m) 61s ago 40m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:28:43.955 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (40m) 10m ago 40m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:28:43.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (33m) 61s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:28:43.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (33m) 61s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:28:43.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (32m) 61s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:28:43.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (32m) 61s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:28:43.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (31m) 10m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:28:43.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (31m) 10m ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:28:43.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (31m) 10m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:28:43.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (30m) 10m ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:28:43.957 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (40m) 10m ago 40m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:28:43.957 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 61s ago 38m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:28:44.206 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:28:44.207 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:28:44.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:28:44.207 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:44.207 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:28:44.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:28:44.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:44.208 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:28:44.208 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:28:44.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:28:44.208 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:28:44.209 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:28:44.209 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:28:44.209 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:28:44.209 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:28:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:44 gibba032 bash[24004]: audit 2021-03-10T14:28:43.712217+0000 mgr.gibba032.whopeo (mgr.24625) 1925 : audit [DBG] from='client.45351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:44 gibba032 bash[24004]: audit 2021-03-10T14:28:43.936828+0000 mgr.gibba032.whopeo (mgr.24625) 1926 : audit [DBG] from='client.55342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:44 gibba032 bash[24004]: cluster 2021-03-10T14:28:43.994746+0000 mgr.gibba032.whopeo (mgr.24625) 1927 : cluster [DBG] pgmap v1088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[30791]: audit 2021-03-10T14:28:43.712217+0000 mgr.gibba032.whopeo (mgr.24625) 1925 : audit [DBG] from='client.45351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[30791]: audit 2021-03-10T14:28:43.936828+0000 mgr.gibba032.whopeo (mgr.24625) 1926 : audit [DBG] from='client.55342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[30791]: cluster 2021-03-10T14:28:43.994746+0000 mgr.gibba032.whopeo (mgr.24625) 1927 : cluster [DBG] pgmap v1088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[31774]: audit 2021-03-10T14:28:43.712217+0000 mgr.gibba032.whopeo (mgr.24625) 1925 : audit [DBG] from='client.45351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[31774]: audit 2021-03-10T14:28:43.936828+0000 mgr.gibba032.whopeo (mgr.24625) 1926 : audit [DBG] from='client.55342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:28:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:44 gibba011 bash[31774]: cluster 2021-03-10T14:28:43.994746+0000 mgr.gibba032.whopeo (mgr.24625) 1927 : cluster [DBG] pgmap v1088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:45.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:45 gibba032 bash[24004]: audit 2021-03-10T14:28:44.206786+0000 mon.c (mon.1) 38 : audit [DBG] from='client.? 172.21.2.111:0/230500403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:45.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:45 gibba011 bash[30791]: audit 2021-03-10T14:28:44.206786+0000 mon.c (mon.1) 38 : audit [DBG] from='client.? 172.21.2.111:0/230500403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:45.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:45 gibba011 bash[31774]: audit 2021-03-10T14:28:44.206786+0000 mon.c (mon.1) 38 : audit [DBG] from='client.? 172.21.2.111:0/230500403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: cluster 2021-03-10T14:28:45.995176+0000 mgr.gibba032.whopeo (mgr.24625) 1928 : cluster [DBG] pgmap v1089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.468446+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.675673+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.683805+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:28:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.685858+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.687515+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.689155+0000 mon.a (mon.0) 1116 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.691215+0000 mon.a (mon.0) 1117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:46 gibba032 bash[24004]: audit 2021-03-10T14:28:46.693053+0000 mon.a (mon.0) 1118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: cluster 2021-03-10T14:28:45.995176+0000 mgr.gibba032.whopeo (mgr.24625) 1928 : cluster [DBG] pgmap v1089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.468446+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.675673+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.683805+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:28:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.685858+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.687515+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.689155+0000 mon.a (mon.0) 1116 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.691215+0000 mon.a (mon.0) 1117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[30791]: audit 2021-03-10T14:28:46.693053+0000 mon.a (mon.0) 1118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: cluster 2021-03-10T14:28:45.995176+0000 mgr.gibba032.whopeo (mgr.24625) 1928 : cluster [DBG] pgmap v1089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.468446+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.675673+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:28:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.683805+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:28:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.685858+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.687515+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.689155+0000 mon.a (mon.0) 1116 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.691215+0000 mon.a (mon.0) 1117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:46 gibba011 bash[31774]: audit 2021-03-10T14:28:46.693053+0000 mon.a (mon.0) 1118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:28:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.465777+0000 mgr.gibba032.whopeo (mgr.24625) 1929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:28:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.679488+0000 mgr.gibba032.whopeo (mgr.24625) 1930 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:28:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.681908+0000 mgr.gibba032.whopeo (mgr.24625) 1931 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:28:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.683667+0000 mgr.gibba032.whopeo (mgr.24625) 1932 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:28:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.683710+0000 mgr.gibba032.whopeo (mgr.24625) 1933 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:28:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.685379+0000 mgr.gibba032.whopeo (mgr.24625) 1934 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:28:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.685461+0000 mgr.gibba032.whopeo (mgr.24625) 1935 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:28:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.686488+0000 mgr.gibba032.whopeo (mgr.24625) 1936 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:28:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.687418+0000 mgr.gibba032.whopeo (mgr.24625) 1937 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:28:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.687462+0000 mgr.gibba032.whopeo (mgr.24625) 1938 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:28:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.689034+0000 mgr.gibba032.whopeo (mgr.24625) 1939 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:28:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.689359+0000 mgr.gibba032.whopeo (mgr.24625) 1940 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:28:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.690611+0000 mgr.gibba032.whopeo (mgr.24625) 1941 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:28:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.690659+0000 mgr.gibba032.whopeo (mgr.24625) 1942 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:28:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:47 gibba032 bash[24004]: cephadm 2021-03-10T14:28:46.690778+0000 mgr.gibba032.whopeo (mgr.24625) 1943 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.465777+0000 mgr.gibba032.whopeo (mgr.24625) 1929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:28:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.679488+0000 mgr.gibba032.whopeo (mgr.24625) 1930 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:28:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.681908+0000 mgr.gibba032.whopeo (mgr.24625) 1931 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:28:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.683667+0000 mgr.gibba032.whopeo (mgr.24625) 1932 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:28:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.683710+0000 mgr.gibba032.whopeo (mgr.24625) 1933 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:28:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.685379+0000 mgr.gibba032.whopeo (mgr.24625) 1934 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:28:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.685461+0000 mgr.gibba032.whopeo (mgr.24625) 1935 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:28:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.686488+0000 mgr.gibba032.whopeo (mgr.24625) 1936 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:28:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.687418+0000 mgr.gibba032.whopeo (mgr.24625) 1937 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:28:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.687462+0000 mgr.gibba032.whopeo (mgr.24625) 1938 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:28:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.689034+0000 mgr.gibba032.whopeo (mgr.24625) 1939 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:28:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.689359+0000 mgr.gibba032.whopeo (mgr.24625) 1940 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:28:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.690611+0000 mgr.gibba032.whopeo (mgr.24625) 1941 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:28:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.690659+0000 mgr.gibba032.whopeo (mgr.24625) 1942 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:28:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[30791]: cephadm 2021-03-10T14:28:46.690778+0000 mgr.gibba032.whopeo (mgr.24625) 1943 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.465777+0000 mgr.gibba032.whopeo (mgr.24625) 1929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:28:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.679488+0000 mgr.gibba032.whopeo (mgr.24625) 1930 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:28:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.681908+0000 mgr.gibba032.whopeo (mgr.24625) 1931 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:28:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.683667+0000 mgr.gibba032.whopeo (mgr.24625) 1932 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:28:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.683710+0000 mgr.gibba032.whopeo (mgr.24625) 1933 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:28:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.685379+0000 mgr.gibba032.whopeo (mgr.24625) 1934 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:28:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.685461+0000 mgr.gibba032.whopeo (mgr.24625) 1935 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:28:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.686488+0000 mgr.gibba032.whopeo (mgr.24625) 1936 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:28:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.687418+0000 mgr.gibba032.whopeo (mgr.24625) 1937 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:28:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.687462+0000 mgr.gibba032.whopeo (mgr.24625) 1938 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:28:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.689034+0000 mgr.gibba032.whopeo (mgr.24625) 1939 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:28:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.689359+0000 mgr.gibba032.whopeo (mgr.24625) 1940 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:28:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.690611+0000 mgr.gibba032.whopeo (mgr.24625) 1941 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:28:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.690659+0000 mgr.gibba032.whopeo (mgr.24625) 1942 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:28:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:47 gibba011 bash[31774]: cephadm 2021-03-10T14:28:46.690778+0000 mgr.gibba032.whopeo (mgr.24625) 1943 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:28:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:48 gibba032 bash[24004]: cluster 2021-03-10T14:28:47.996879+0000 mgr.gibba032.whopeo (mgr.24625) 1944 : cluster [DBG] pgmap v1090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:48 gibba011 bash[30791]: cluster 2021-03-10T14:28:47.996879+0000 mgr.gibba032.whopeo (mgr.24625) 1944 : cluster [DBG] pgmap v1090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:48 gibba011 bash[31774]: cluster 2021-03-10T14:28:47.996879+0000 mgr.gibba032.whopeo (mgr.24625) 1944 : cluster [DBG] pgmap v1090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:51 gibba011 bash[30791]: cluster 2021-03-10T14:28:49.997937+0000 mgr.gibba032.whopeo (mgr.24625) 1945 : cluster [DBG] pgmap v1091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:51 gibba011 bash[31774]: cluster 2021-03-10T14:28:49.997937+0000 mgr.gibba032.whopeo (mgr.24625) 1945 : cluster [DBG] pgmap v1091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:51 gibba032 bash[24004]: cluster 2021-03-10T14:28:49.997937+0000 mgr.gibba032.whopeo (mgr.24625) 1945 : cluster [DBG] pgmap v1091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:53 gibba011 bash[30791]: cluster 2021-03-10T14:28:51.999662+0000 mgr.gibba032.whopeo (mgr.24625) 1946 : cluster [DBG] pgmap v1092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:53 gibba011 bash[31774]: cluster 2021-03-10T14:28:51.999662+0000 mgr.gibba032.whopeo (mgr.24625) 1946 : cluster [DBG] pgmap v1092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:53 gibba032 bash[24004]: cluster 2021-03-10T14:28:51.999662+0000 mgr.gibba032.whopeo (mgr.24625) 1946 : cluster [DBG] pgmap v1092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:54 gibba032 bash[24004]: cluster 2021-03-10T14:28:54.001356+0000 mgr.gibba032.whopeo (mgr.24625) 1947 : cluster [DBG] pgmap v1093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:54 gibba011 bash[30791]: cluster 2021-03-10T14:28:54.001356+0000 mgr.gibba032.whopeo (mgr.24625) 1947 : cluster [DBG] pgmap v1093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:54 gibba011 bash[31774]: cluster 2021-03-10T14:28:54.001356+0000 mgr.gibba032.whopeo (mgr.24625) 1947 : cluster [DBG] pgmap v1093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: cluster 2021-03-10T14:28:56.002954+0000 mgr.gibba032.whopeo (mgr.24625) 1948 : cluster [DBG] pgmap v1094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.202257+0000 mon.a (mon.0) 1119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:28:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.203009+0000 mon.a (mon.0) 1120 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.203772+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.204447+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:28:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.221951+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:28:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:56 gibba032 bash[24004]: audit 2021-03-10T14:28:56.223321+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:28:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: cluster 2021-03-10T14:28:56.002954+0000 mgr.gibba032.whopeo (mgr.24625) 1948 : cluster [DBG] pgmap v1094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.202257+0000 mon.a (mon.0) 1119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:28:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.203009+0000 mon.a (mon.0) 1120 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.203772+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.204447+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:28:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.221951+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:28:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[30791]: audit 2021-03-10T14:28:56.223321+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:28:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: cluster 2021-03-10T14:28:56.002954+0000 mgr.gibba032.whopeo (mgr.24625) 1948 : cluster [DBG] pgmap v1094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.202257+0000 mon.a (mon.0) 1119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:28:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.203009+0000 mon.a (mon.0) 1120 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.203772+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:28:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.204447+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T14:28:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.221951+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:28:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:56 gibba011 bash[31774]: audit 2021-03-10T14:28:56.223321+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:28:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:28:59 gibba011 bash[30791]: cluster 2021-03-10T14:28:58.004806+0000 mgr.gibba032.whopeo (mgr.24625) 1949 : cluster [DBG] pgmap v1095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:28:59 gibba011 bash[31774]: cluster 2021-03-10T14:28:58.004806+0000 mgr.gibba032.whopeo (mgr.24625) 1949 : cluster [DBG] pgmap v1095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:28:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:28:59 gibba032 bash[24004]: cluster 2021-03-10T14:28:58.004806+0000 mgr.gibba032.whopeo (mgr.24625) 1949 : cluster [DBG] pgmap v1095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:00 gibba032 bash[24004]: cluster 2021-03-10T14:29:00.005887+0000 mgr.gibba032.whopeo (mgr.24625) 1950 : cluster [DBG] pgmap v1096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:00 gibba011 bash[30791]: cluster 2021-03-10T14:29:00.005887+0000 mgr.gibba032.whopeo (mgr.24625) 1950 : cluster [DBG] pgmap v1096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:00 gibba011 bash[31774]: cluster 2021-03-10T14:29:00.005887+0000 mgr.gibba032.whopeo (mgr.24625) 1950 : cluster [DBG] pgmap v1096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:03 gibba011 bash[30791]: cephadm 2021-03-10T14:29:01.691650+0000 mgr.gibba032.whopeo (mgr.24625) 1951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:03 gibba011 bash[30791]: cluster 2021-03-10T14:29:02.007544+0000 mgr.gibba032.whopeo (mgr.24625) 1952 : cluster [DBG] pgmap v1097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:03 gibba011 bash[31774]: cephadm 2021-03-10T14:29:01.691650+0000 mgr.gibba032.whopeo (mgr.24625) 1951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:03 gibba011 bash[31774]: cluster 2021-03-10T14:29:02.007544+0000 mgr.gibba032.whopeo (mgr.24625) 1952 : cluster [DBG] pgmap v1097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:03 gibba032 bash[24004]: cephadm 2021-03-10T14:29:01.691650+0000 mgr.gibba032.whopeo (mgr.24625) 1951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:03 gibba032 bash[24004]: cluster 2021-03-10T14:29:02.007544+0000 mgr.gibba032.whopeo (mgr.24625) 1952 : cluster [DBG] pgmap v1097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:04 gibba032 bash[24004]: cluster 2021-03-10T14:29:04.009133+0000 mgr.gibba032.whopeo (mgr.24625) 1953 : cluster [DBG] pgmap v1098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:04 gibba011 bash[30791]: cluster 2021-03-10T14:29:04.009133+0000 mgr.gibba032.whopeo (mgr.24625) 1953 : cluster [DBG] pgmap v1098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:04 gibba011 bash[31774]: cluster 2021-03-10T14:29:04.009133+0000 mgr.gibba032.whopeo (mgr.24625) 1953 : cluster [DBG] pgmap v1098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:06 gibba032 bash[24004]: cluster 2021-03-10T14:29:06.010440+0000 mgr.gibba032.whopeo (mgr.24625) 1954 : cluster [DBG] pgmap v1099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:06 gibba011 bash[30791]: cluster 2021-03-10T14:29:06.010440+0000 mgr.gibba032.whopeo (mgr.24625) 1954 : cluster [DBG] pgmap v1099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:06 gibba011 bash[31774]: cluster 2021-03-10T14:29:06.010440+0000 mgr.gibba032.whopeo (mgr.24625) 1954 : cluster [DBG] pgmap v1099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:09 gibba011 bash[30791]: cluster 2021-03-10T14:29:08.012382+0000 mgr.gibba032.whopeo (mgr.24625) 1955 : cluster [DBG] pgmap v1100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:09 gibba011 bash[31774]: cluster 2021-03-10T14:29:08.012382+0000 mgr.gibba032.whopeo (mgr.24625) 1955 : cluster [DBG] pgmap v1100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:09.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:09 gibba032 bash[24004]: cluster 2021-03-10T14:29:08.012382+0000 mgr.gibba032.whopeo (mgr.24625) 1955 : cluster [DBG] pgmap v1100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:10 gibba032 bash[24004]: cluster 2021-03-10T14:29:10.013061+0000 mgr.gibba032.whopeo (mgr.24625) 1956 : cluster [DBG] pgmap v1101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:10 gibba011 bash[30791]: cluster 2021-03-10T14:29:10.013061+0000 mgr.gibba032.whopeo (mgr.24625) 1956 : cluster [DBG] pgmap v1101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:10 gibba011 bash[31774]: cluster 2021-03-10T14:29:10.013061+0000 mgr.gibba032.whopeo (mgr.24625) 1956 : cluster [DBG] pgmap v1101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:12 gibba032 bash[24004]: cluster 2021-03-10T14:29:12.014309+0000 mgr.gibba032.whopeo (mgr.24625) 1957 : cluster [DBG] pgmap v1102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:12 gibba011 bash[30791]: cluster 2021-03-10T14:29:12.014309+0000 mgr.gibba032.whopeo (mgr.24625) 1957 : cluster [DBG] pgmap v1102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:12 gibba011 bash[31774]: cluster 2021-03-10T14:29:12.014309+0000 mgr.gibba032.whopeo (mgr.24625) 1957 : cluster [DBG] pgmap v1102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:14.453 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:29:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:14 gibba032 bash[24004]: cluster 2021-03-10T14:29:14.015954+0000 mgr.gibba032.whopeo (mgr.24625) 1958 : cluster [DBG] pgmap v1103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:14.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:14 gibba011 bash[30791]: cluster 2021-03-10T14:29:14.015954+0000 mgr.gibba032.whopeo (mgr.24625) 1958 : cluster [DBG] pgmap v1103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:14.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:14 gibba011 bash[31774]: cluster 2021-03-10T14:29:14.015954+0000 mgr.gibba032.whopeo (mgr.24625) 1958 : cluster [DBG] pgmap v1103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:14.681 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:29:14.682 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (35m) 92s ago 41m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:29:14.682 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (41m) 28s ago 41m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:29:14.682 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (36m) 28s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:29:14.682 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (36m) 28s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:29:14.683 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (35m) 92s ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:29:14.683 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (35m) 92s ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:29:14.683 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (34m) 28s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:29:14.683 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (35m) 92s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:29:14.683 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (41m) 92s ago 41m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:29:14.684 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (41m) 28s ago 41m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:29:14.684 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (34m) 92s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:29:14.684 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (33m) 92s ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:29:14.684 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (33m) 92s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:29:14.684 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (32m) 92s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:29:14.685 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (32m) 28s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:29:14.685 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (32m) 28s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:29:14.685 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (31m) 28s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:29:14.685 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (31m) 28s ago 41m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:29:14.685 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (40m) 28s ago 41m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:29:14.686 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 92s ago 39m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:29:14.932 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:29:14.932 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:29:14.932 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:29:14.933 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:14.933 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:29:14.933 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:29:14.933 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:14.934 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:29:14.934 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:29:14.934 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:14.934 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:29:14.934 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:29:14.935 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:29:14.935 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:29:14.935 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:29:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:15 gibba032 bash[24004]: audit 2021-03-10T14:29:14.440772+0000 mgr.gibba032.whopeo (mgr.24625) 1959 : audit [DBG] from='client.45369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:15 gibba032 bash[24004]: audit 2021-03-10T14:29:14.666181+0000 mgr.gibba032.whopeo (mgr.24625) 1960 : audit [DBG] from='client.45375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:15 gibba032 bash[24004]: audit 2021-03-10T14:29:14.931903+0000 mon.c (mon.1) 39 : audit [DBG] from='client.? 172.21.2.111:0/2494429255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:15.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[30791]: audit 2021-03-10T14:29:14.440772+0000 mgr.gibba032.whopeo (mgr.24625) 1959 : audit [DBG] from='client.45369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[30791]: audit 2021-03-10T14:29:14.666181+0000 mgr.gibba032.whopeo (mgr.24625) 1960 : audit [DBG] from='client.45375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[30791]: audit 2021-03-10T14:29:14.931903+0000 mon.c (mon.1) 39 : audit [DBG] from='client.? 172.21.2.111:0/2494429255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:15.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[31774]: audit 2021-03-10T14:29:14.440772+0000 mgr.gibba032.whopeo (mgr.24625) 1959 : audit [DBG] from='client.45369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[31774]: audit 2021-03-10T14:29:14.666181+0000 mgr.gibba032.whopeo (mgr.24625) 1960 : audit [DBG] from='client.45375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:15 gibba011 bash[31774]: audit 2021-03-10T14:29:14.931903+0000 mon.c (mon.1) 39 : audit [DBG] from='client.? 172.21.2.111:0/2494429255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:17.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:16 gibba032 bash[24004]: cluster 2021-03-10T14:29:16.017332+0000 mgr.gibba032.whopeo (mgr.24625) 1961 : cluster [DBG] pgmap v1104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:16 gibba011 bash[30791]: cluster 2021-03-10T14:29:16.017332+0000 mgr.gibba032.whopeo (mgr.24625) 1961 : cluster [DBG] pgmap v1104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:16 gibba011 bash[31774]: cluster 2021-03-10T14:29:16.017332+0000 mgr.gibba032.whopeo (mgr.24625) 1961 : cluster [DBG] pgmap v1104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:17 gibba032 bash[24004]: cephadm 2021-03-10T14:29:16.707649+0000 mgr.gibba032.whopeo (mgr.24625) 1962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:17 gibba011 bash[30791]: cephadm 2021-03-10T14:29:16.707649+0000 mgr.gibba032.whopeo (mgr.24625) 1962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:18.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:17 gibba011 bash[31774]: cephadm 2021-03-10T14:29:16.707649+0000 mgr.gibba032.whopeo (mgr.24625) 1962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:18 gibba032 bash[24004]: cluster 2021-03-10T14:29:18.018633+0000 mgr.gibba032.whopeo (mgr.24625) 1963 : cluster [DBG] pgmap v1105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:18 gibba011 bash[30791]: cluster 2021-03-10T14:29:18.018633+0000 mgr.gibba032.whopeo (mgr.24625) 1963 : cluster [DBG] pgmap v1105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:18 gibba011 bash[31774]: cluster 2021-03-10T14:29:18.018633+0000 mgr.gibba032.whopeo (mgr.24625) 1963 : cluster [DBG] pgmap v1105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:21.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:21 gibba032 bash[24004]: cluster 2021-03-10T14:29:20.019302+0000 mgr.gibba032.whopeo (mgr.24625) 1964 : cluster [DBG] pgmap v1106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:21.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:21 gibba011 bash[30791]: cluster 2021-03-10T14:29:20.019302+0000 mgr.gibba032.whopeo (mgr.24625) 1964 : cluster [DBG] pgmap v1106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:21.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:21 gibba011 bash[31774]: cluster 2021-03-10T14:29:20.019302+0000 mgr.gibba032.whopeo (mgr.24625) 1964 : cluster [DBG] pgmap v1106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:22 gibba032 bash[24004]: cluster 2021-03-10T14:29:22.020841+0000 mgr.gibba032.whopeo (mgr.24625) 1965 : cluster [DBG] pgmap v1107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:22 gibba011 bash[30791]: cluster 2021-03-10T14:29:22.020841+0000 mgr.gibba032.whopeo (mgr.24625) 1965 : cluster [DBG] pgmap v1107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:22 gibba011 bash[31774]: cluster 2021-03-10T14:29:22.020841+0000 mgr.gibba032.whopeo (mgr.24625) 1965 : cluster [DBG] pgmap v1107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:24 gibba032 bash[24004]: cluster 2021-03-10T14:29:24.022328+0000 mgr.gibba032.whopeo (mgr.24625) 1966 : cluster [DBG] pgmap v1108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:24 gibba011 bash[30791]: cluster 2021-03-10T14:29:24.022328+0000 mgr.gibba032.whopeo (mgr.24625) 1966 : cluster [DBG] pgmap v1108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:24 gibba011 bash[31774]: cluster 2021-03-10T14:29:24.022328+0000 mgr.gibba032.whopeo (mgr.24625) 1966 : cluster [DBG] pgmap v1108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:26 gibba032 bash[24004]: cluster 2021-03-10T14:29:26.023793+0000 mgr.gibba032.whopeo (mgr.24625) 1967 : cluster [DBG] pgmap v1109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:26 gibba011 bash[30791]: cluster 2021-03-10T14:29:26.023793+0000 mgr.gibba032.whopeo (mgr.24625) 1967 : cluster [DBG] pgmap v1109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:26 gibba011 bash[31774]: cluster 2021-03-10T14:29:26.023793+0000 mgr.gibba032.whopeo (mgr.24625) 1967 : cluster [DBG] pgmap v1109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:28 gibba032 bash[24004]: cluster 2021-03-10T14:29:28.024346+0000 mgr.gibba032.whopeo (mgr.24625) 1968 : cluster [DBG] pgmap v1110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:28 gibba011 bash[30791]: cluster 2021-03-10T14:29:28.024346+0000 mgr.gibba032.whopeo (mgr.24625) 1968 : cluster [DBG] pgmap v1110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:28 gibba011 bash[31774]: cluster 2021-03-10T14:29:28.024346+0000 mgr.gibba032.whopeo (mgr.24625) 1968 : cluster [DBG] pgmap v1110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:30 gibba032 bash[24004]: cluster 2021-03-10T14:29:30.024864+0000 mgr.gibba032.whopeo (mgr.24625) 1969 : cluster [DBG] pgmap v1111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:30 gibba011 bash[30791]: cluster 2021-03-10T14:29:30.024864+0000 mgr.gibba032.whopeo (mgr.24625) 1969 : cluster [DBG] pgmap v1111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:30 gibba011 bash[31774]: cluster 2021-03-10T14:29:30.024864+0000 mgr.gibba032.whopeo (mgr.24625) 1969 : cluster [DBG] pgmap v1111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:32 gibba032 bash[24004]: cephadm 2021-03-10T14:29:31.723696+0000 mgr.gibba032.whopeo (mgr.24625) 1970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:32 gibba032 bash[24004]: cluster 2021-03-10T14:29:32.026132+0000 mgr.gibba032.whopeo (mgr.24625) 1971 : cluster [DBG] pgmap v1112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:32 gibba011 bash[30791]: cephadm 2021-03-10T14:29:31.723696+0000 mgr.gibba032.whopeo (mgr.24625) 1970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:32 gibba011 bash[30791]: cluster 2021-03-10T14:29:32.026132+0000 mgr.gibba032.whopeo (mgr.24625) 1971 : cluster [DBG] pgmap v1112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:32 gibba011 bash[31774]: cephadm 2021-03-10T14:29:31.723696+0000 mgr.gibba032.whopeo (mgr.24625) 1970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:32 gibba011 bash[31774]: cluster 2021-03-10T14:29:32.026132+0000 mgr.gibba032.whopeo (mgr.24625) 1971 : cluster [DBG] pgmap v1112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:34 gibba032 bash[24004]: cluster 2021-03-10T14:29:34.027568+0000 mgr.gibba032.whopeo (mgr.24625) 1972 : cluster [DBG] pgmap v1113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:34 gibba011 bash[30791]: cluster 2021-03-10T14:29:34.027568+0000 mgr.gibba032.whopeo (mgr.24625) 1972 : cluster [DBG] pgmap v1113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:34 gibba011 bash[31774]: cluster 2021-03-10T14:29:34.027568+0000 mgr.gibba032.whopeo (mgr.24625) 1972 : cluster [DBG] pgmap v1113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:36 gibba032 bash[24004]: cluster 2021-03-10T14:29:36.028925+0000 mgr.gibba032.whopeo (mgr.24625) 1973 : cluster [DBG] pgmap v1114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:36 gibba011 bash[30791]: cluster 2021-03-10T14:29:36.028925+0000 mgr.gibba032.whopeo (mgr.24625) 1973 : cluster [DBG] pgmap v1114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:36 gibba011 bash[31774]: cluster 2021-03-10T14:29:36.028925+0000 mgr.gibba032.whopeo (mgr.24625) 1973 : cluster [DBG] pgmap v1114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:38 gibba032 bash[24004]: cluster 2021-03-10T14:29:38.029785+0000 mgr.gibba032.whopeo (mgr.24625) 1974 : cluster [DBG] pgmap v1115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:38 gibba011 bash[30791]: cluster 2021-03-10T14:29:38.029785+0000 mgr.gibba032.whopeo (mgr.24625) 1974 : cluster [DBG] pgmap v1115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:38 gibba011 bash[31774]: cluster 2021-03-10T14:29:38.029785+0000 mgr.gibba032.whopeo (mgr.24625) 1974 : cluster [DBG] pgmap v1115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:40 gibba032 bash[24004]: cluster 2021-03-10T14:29:40.030426+0000 mgr.gibba032.whopeo (mgr.24625) 1975 : cluster [DBG] pgmap v1116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:40 gibba011 bash[30791]: cluster 2021-03-10T14:29:40.030426+0000 mgr.gibba032.whopeo (mgr.24625) 1975 : cluster [DBG] pgmap v1116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:40 gibba011 bash[31774]: cluster 2021-03-10T14:29:40.030426+0000 mgr.gibba032.whopeo (mgr.24625) 1975 : cluster [DBG] pgmap v1116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:42 gibba032 bash[24004]: cluster 2021-03-10T14:29:42.031792+0000 mgr.gibba032.whopeo (mgr.24625) 1976 : cluster [DBG] pgmap v1117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:42 gibba011 bash[30791]: cluster 2021-03-10T14:29:42.031792+0000 mgr.gibba032.whopeo (mgr.24625) 1976 : cluster [DBG] pgmap v1117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:42 gibba011 bash[31774]: cluster 2021-03-10T14:29:42.031792+0000 mgr.gibba032.whopeo (mgr.24625) 1976 : cluster [DBG] pgmap v1117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:44 gibba032 bash[24004]: cluster 2021-03-10T14:29:44.033404+0000 mgr.gibba032.whopeo (mgr.24625) 1977 : cluster [DBG] pgmap v1118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:44 gibba011 bash[30791]: cluster 2021-03-10T14:29:44.033404+0000 mgr.gibba032.whopeo (mgr.24625) 1977 : cluster [DBG] pgmap v1118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:44 gibba011 bash[31774]: cluster 2021-03-10T14:29:44.033404+0000 mgr.gibba032.whopeo (mgr.24625) 1977 : cluster [DBG] pgmap v1118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:45.174 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:29:45.403 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:29:45.403 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (36m) 2m ago 41m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:29:45.404 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (41m) 58s ago 41m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:29:45.404 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (36m) 58s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:29:45.404 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (37m) 58s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:29:45.404 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (36m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:29:45.404 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (35m) 2m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:29:45.405 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (35m) 58s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:29:45.405 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (35m) 2m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:29:45.405 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (41m) 2m ago 41m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:29:45.405 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (41m) 58s ago 41m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:29:45.405 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (34m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:29:45.406 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (34m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:29:45.406 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (33m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:29:45.406 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (33m) 2m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:29:45.406 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (32m) 58s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:29:45.406 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (32m) 58s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:29:45.407 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (32m) 58s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:29:45.407 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (31m) 58s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:29:45.407 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (41m) 58s ago 41m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:29:45.407 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 39m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:29:45.654 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:29:45.654 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:29:45.654 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:29:45.655 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:45.655 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:29:45.655 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:29:45.655 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:45.655 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:29:45.656 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:29:45.656 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:29:45.656 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:29:45.656 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:29:45.657 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:29:45.657 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:29:45.657 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:29:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:45 gibba032 bash[24004]: audit 2021-03-10T14:29:45.653662+0000 mon.a (mon.0) 1125 : audit [DBG] from='client.? 172.21.2.111:0/1207169721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:46.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:45 gibba011 bash[30791]: audit 2021-03-10T14:29:45.653662+0000 mon.a (mon.0) 1125 : audit [DBG] from='client.? 172.21.2.111:0/1207169721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:46.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:45 gibba011 bash[31774]: audit 2021-03-10T14:29:45.653662+0000 mon.a (mon.0) 1125 : audit [DBG] from='client.? 172.21.2.111:0/1207169721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:46 gibba032 bash[24004]: audit 2021-03-10T14:29:45.161586+0000 mgr.gibba032.whopeo (mgr.24625) 1978 : audit [DBG] from='client.45387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:46 gibba032 bash[24004]: audit 2021-03-10T14:29:45.387625+0000 mgr.gibba032.whopeo (mgr.24625) 1979 : audit [DBG] from='client.55378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:46 gibba032 bash[24004]: cluster 2021-03-10T14:29:46.035147+0000 mgr.gibba032.whopeo (mgr.24625) 1980 : cluster [DBG] pgmap v1119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[30791]: audit 2021-03-10T14:29:45.161586+0000 mgr.gibba032.whopeo (mgr.24625) 1978 : audit [DBG] from='client.45387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[30791]: audit 2021-03-10T14:29:45.387625+0000 mgr.gibba032.whopeo (mgr.24625) 1979 : audit [DBG] from='client.55378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[30791]: cluster 2021-03-10T14:29:46.035147+0000 mgr.gibba032.whopeo (mgr.24625) 1980 : cluster [DBG] pgmap v1119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[31774]: audit 2021-03-10T14:29:45.161586+0000 mgr.gibba032.whopeo (mgr.24625) 1978 : audit [DBG] from='client.45387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[31774]: audit 2021-03-10T14:29:45.387625+0000 mgr.gibba032.whopeo (mgr.24625) 1979 : audit [DBG] from='client.55378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:29:47.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:46 gibba011 bash[31774]: cluster 2021-03-10T14:29:46.035147+0000 mgr.gibba032.whopeo (mgr.24625) 1980 : cluster [DBG] pgmap v1119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.740471+0000 mgr.gibba032.whopeo (mgr.24625) 1981 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:29:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.741984+0000 mgr.gibba032.whopeo (mgr.24625) 1982 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:29:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.974557+0000 mgr.gibba032.whopeo (mgr.24625) 1983 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:29:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.975127+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:29:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.976026+0000 mgr.gibba032.whopeo (mgr.24625) 1984 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:29:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.976989+0000 mgr.gibba032.whopeo (mgr.24625) 1985 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:29:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.977030+0000 mgr.gibba032.whopeo (mgr.24625) 1986 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:29:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.978108+0000 mgr.gibba032.whopeo (mgr.24625) 1987 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:29:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.978148+0000 mgr.gibba032.whopeo (mgr.24625) 1988 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:29:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.978555+0000 mon.a (mon.0) 1127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:29:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.978954+0000 mgr.gibba032.whopeo (mgr.24625) 1989 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:29:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.979652+0000 mon.a (mon.0) 1128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.979660+0000 mgr.gibba032.whopeo (mgr.24625) 1990 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:29:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.979699+0000 mgr.gibba032.whopeo (mgr.24625) 1991 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:29:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.980775+0000 mon.a (mon.0) 1129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.980816+0000 mgr.gibba032.whopeo (mgr.24625) 1992 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:29:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.981153+0000 mgr.gibba032.whopeo (mgr.24625) 1993 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:29:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.981679+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.982258+0000 mgr.gibba032.whopeo (mgr.24625) 1994 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:29:48.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.982300+0000 mgr.gibba032.whopeo (mgr.24625) 1995 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:29:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: cephadm 2021-03-10T14:29:46.982416+0000 mgr.gibba032.whopeo (mgr.24625) 1996 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.983493+0000 mon.a (mon.0) 1131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:47 gibba032 bash[24004]: audit 2021-03-10T14:29:46.984753+0000 mon.a (mon.0) 1132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.740471+0000 mgr.gibba032.whopeo (mgr.24625) 1981 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:29:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.741984+0000 mgr.gibba032.whopeo (mgr.24625) 1982 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:29:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.974557+0000 mgr.gibba032.whopeo (mgr.24625) 1983 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.975127+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.976026+0000 mgr.gibba032.whopeo (mgr.24625) 1984 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.976989+0000 mgr.gibba032.whopeo (mgr.24625) 1985 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.977030+0000 mgr.gibba032.whopeo (mgr.24625) 1986 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.978108+0000 mgr.gibba032.whopeo (mgr.24625) 1987 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:29:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.978148+0000 mgr.gibba032.whopeo (mgr.24625) 1988 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:29:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.978555+0000 mon.a (mon.0) 1127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:29:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.978954+0000 mgr.gibba032.whopeo (mgr.24625) 1989 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:29:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.979652+0000 mon.a (mon.0) 1128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.979660+0000 mgr.gibba032.whopeo (mgr.24625) 1990 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:29:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.979699+0000 mgr.gibba032.whopeo (mgr.24625) 1991 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:29:48.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.980775+0000 mon.a (mon.0) 1129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.980816+0000 mgr.gibba032.whopeo (mgr.24625) 1992 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:29:48.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.981153+0000 mgr.gibba032.whopeo (mgr.24625) 1993 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:29:48.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.981679+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.982258+0000 mgr.gibba032.whopeo (mgr.24625) 1994 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:29:48.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.982300+0000 mgr.gibba032.whopeo (mgr.24625) 1995 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:29:48.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: cephadm 2021-03-10T14:29:46.982416+0000 mgr.gibba032.whopeo (mgr.24625) 1996 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:48.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.983493+0000 mon.a (mon.0) 1131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[30791]: audit 2021-03-10T14:29:46.984753+0000 mon.a (mon.0) 1132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.740471+0000 mgr.gibba032.whopeo (mgr.24625) 1981 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:29:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.741984+0000 mgr.gibba032.whopeo (mgr.24625) 1982 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:29:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.974557+0000 mgr.gibba032.whopeo (mgr.24625) 1983 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:29:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.975127+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:29:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.976026+0000 mgr.gibba032.whopeo (mgr.24625) 1984 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:29:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.976989+0000 mgr.gibba032.whopeo (mgr.24625) 1985 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:29:48.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.977030+0000 mgr.gibba032.whopeo (mgr.24625) 1986 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:29:48.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.978108+0000 mgr.gibba032.whopeo (mgr.24625) 1987 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:29:48.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.978148+0000 mgr.gibba032.whopeo (mgr.24625) 1988 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:29:48.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.978555+0000 mon.a (mon.0) 1127 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:29:48.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.978954+0000 mgr.gibba032.whopeo (mgr.24625) 1989 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:29:48.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.979652+0000 mon.a (mon.0) 1128 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.979660+0000 mgr.gibba032.whopeo (mgr.24625) 1990 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:29:48.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.979699+0000 mgr.gibba032.whopeo (mgr.24625) 1991 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:29:48.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.980775+0000 mon.a (mon.0) 1129 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.980816+0000 mgr.gibba032.whopeo (mgr.24625) 1992 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:29:48.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.981153+0000 mgr.gibba032.whopeo (mgr.24625) 1993 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:29:48.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.981679+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.982258+0000 mgr.gibba032.whopeo (mgr.24625) 1994 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:29:48.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.982300+0000 mgr.gibba032.whopeo (mgr.24625) 1995 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:29:48.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: cephadm 2021-03-10T14:29:46.982416+0000 mgr.gibba032.whopeo (mgr.24625) 1996 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:29:48.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.983493+0000 mon.a (mon.0) 1131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:48.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:47 gibba011 bash[31774]: audit 2021-03-10T14:29:46.984753+0000 mon.a (mon.0) 1132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:29:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:48 gibba032 bash[24004]: cluster 2021-03-10T14:29:48.036237+0000 mgr.gibba032.whopeo (mgr.24625) 1997 : cluster [DBG] pgmap v1120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:48 gibba011 bash[30791]: cluster 2021-03-10T14:29:48.036237+0000 mgr.gibba032.whopeo (mgr.24625) 1997 : cluster [DBG] pgmap v1120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:48 gibba011 bash[31774]: cluster 2021-03-10T14:29:48.036237+0000 mgr.gibba032.whopeo (mgr.24625) 1997 : cluster [DBG] pgmap v1120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:50 gibba032 bash[24004]: cluster 2021-03-10T14:29:50.036778+0000 mgr.gibba032.whopeo (mgr.24625) 1998 : cluster [DBG] pgmap v1121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:50 gibba011 bash[30791]: cluster 2021-03-10T14:29:50.036778+0000 mgr.gibba032.whopeo (mgr.24625) 1998 : cluster [DBG] pgmap v1121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:50 gibba011 bash[31774]: cluster 2021-03-10T14:29:50.036778+0000 mgr.gibba032.whopeo (mgr.24625) 1998 : cluster [DBG] pgmap v1121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:52 gibba032 bash[24004]: cluster 2021-03-10T14:29:52.038154+0000 mgr.gibba032.whopeo (mgr.24625) 1999 : cluster [DBG] pgmap v1122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:52 gibba011 bash[30791]: cluster 2021-03-10T14:29:52.038154+0000 mgr.gibba032.whopeo (mgr.24625) 1999 : cluster [DBG] pgmap v1122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:52 gibba011 bash[31774]: cluster 2021-03-10T14:29:52.038154+0000 mgr.gibba032.whopeo (mgr.24625) 1999 : cluster [DBG] pgmap v1122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:54 gibba032 bash[24004]: cluster 2021-03-10T14:29:54.038621+0000 mgr.gibba032.whopeo (mgr.24625) 2000 : cluster [DBG] pgmap v1123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:54 gibba011 bash[30791]: cluster 2021-03-10T14:29:54.038621+0000 mgr.gibba032.whopeo (mgr.24625) 2000 : cluster [DBG] pgmap v1123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:54 gibba011 bash[31774]: cluster 2021-03-10T14:29:54.038621+0000 mgr.gibba032.whopeo (mgr.24625) 2000 : cluster [DBG] pgmap v1123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: cluster 2021-03-10T14:29:56.039963+0000 mgr.gibba032.whopeo (mgr.24625) 2001 : cluster [DBG] pgmap v1124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.212772+0000 mon.a (mon.0) 1133 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:29:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.213627+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.214428+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.215196+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:29:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.226555+0000 mon.a (mon.0) 1137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:29:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:56 gibba032 bash[24004]: audit 2021-03-10T14:29:56.227208+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:29:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: cluster 2021-03-10T14:29:56.039963+0000 mgr.gibba032.whopeo (mgr.24625) 2001 : cluster [DBG] pgmap v1124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.212772+0000 mon.a (mon.0) 1133 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:29:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.213627+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.214428+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.215196+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:29:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.226555+0000 mon.a (mon.0) 1137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:29:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[30791]: audit 2021-03-10T14:29:56.227208+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:29:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: cluster 2021-03-10T14:29:56.039963+0000 mgr.gibba032.whopeo (mgr.24625) 2001 : cluster [DBG] pgmap v1124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.212772+0000 mon.a (mon.0) 1133 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:29:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.213627+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.214428+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:29:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.215196+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:29:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.226555+0000 mon.a (mon.0) 1137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:29:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:56 gibba011 bash[31774]: audit 2021-03-10T14:29:56.227208+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:29:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:29:58 gibba032 bash[24004]: cluster 2021-03-10T14:29:58.040795+0000 mgr.gibba032.whopeo (mgr.24625) 2002 : cluster [DBG] pgmap v1125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:29:58 gibba011 bash[30791]: cluster 2021-03-10T14:29:58.040795+0000 mgr.gibba032.whopeo (mgr.24625) 2002 : cluster [DBG] pgmap v1125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:29:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:29:58 gibba011 bash[31774]: cluster 2021-03-10T14:29:58.040795+0000 mgr.gibba032.whopeo (mgr.24625) 2002 : cluster [DBG] pgmap v1125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:00 gibba011 bash[31774]: cluster 2021-03-10T14:30:00.000213+0000 mon.a (mon.0) 1139 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:30:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:00 gibba011 bash[30791]: debug 2021-03-10T14:29:59.999+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:30:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:00 gibba011 bash[30791]: cluster 2021-03-10T14:30:00.000213+0000 mon.a (mon.0) 1139 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:30:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:00 gibba032 bash[24004]: cluster 2021-03-10T14:30:00.000213+0000 mon.a (mon.0) 1139 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:30:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:01 gibba011 bash[30791]: cluster 2021-03-10T14:30:00.041127+0000 mgr.gibba032.whopeo (mgr.24625) 2003 : cluster [DBG] pgmap v1126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:01 gibba011 bash[31774]: cluster 2021-03-10T14:30:00.041127+0000 mgr.gibba032.whopeo (mgr.24625) 2003 : cluster [DBG] pgmap v1126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:01 gibba032 bash[24004]: cluster 2021-03-10T14:30:00.041127+0000 mgr.gibba032.whopeo (mgr.24625) 2003 : cluster [DBG] pgmap v1126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:03 gibba011 bash[30791]: cephadm 2021-03-10T14:30:01.998313+0000 mgr.gibba032.whopeo (mgr.24625) 2004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:03 gibba011 bash[30791]: cluster 2021-03-10T14:30:02.041537+0000 mgr.gibba032.whopeo (mgr.24625) 2005 : cluster [DBG] pgmap v1127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:03 gibba011 bash[31774]: cephadm 2021-03-10T14:30:01.998313+0000 mgr.gibba032.whopeo (mgr.24625) 2004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:03.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:03 gibba011 bash[31774]: cluster 2021-03-10T14:30:02.041537+0000 mgr.gibba032.whopeo (mgr.24625) 2005 : cluster [DBG] pgmap v1127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:03 gibba032 bash[24004]: cephadm 2021-03-10T14:30:01.998313+0000 mgr.gibba032.whopeo (mgr.24625) 2004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:03 gibba032 bash[24004]: cluster 2021-03-10T14:30:02.041537+0000 mgr.gibba032.whopeo (mgr.24625) 2005 : cluster [DBG] pgmap v1127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:05 gibba011 bash[30791]: cluster 2021-03-10T14:30:04.042858+0000 mgr.gibba032.whopeo (mgr.24625) 2006 : cluster [DBG] pgmap v1128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:05 gibba011 bash[31774]: cluster 2021-03-10T14:30:04.042858+0000 mgr.gibba032.whopeo (mgr.24625) 2006 : cluster [DBG] pgmap v1128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:05 gibba032 bash[24004]: cluster 2021-03-10T14:30:04.042858+0000 mgr.gibba032.whopeo (mgr.24625) 2006 : cluster [DBG] pgmap v1128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:06 gibba032 bash[24004]: cluster 2021-03-10T14:30:06.044256+0000 mgr.gibba032.whopeo (mgr.24625) 2007 : cluster [DBG] pgmap v1129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:06 gibba011 bash[30791]: cluster 2021-03-10T14:30:06.044256+0000 mgr.gibba032.whopeo (mgr.24625) 2007 : cluster [DBG] pgmap v1129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:06 gibba011 bash[31774]: cluster 2021-03-10T14:30:06.044256+0000 mgr.gibba032.whopeo (mgr.24625) 2007 : cluster [DBG] pgmap v1129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:08 gibba032 bash[24004]: cluster 2021-03-10T14:30:08.045180+0000 mgr.gibba032.whopeo (mgr.24625) 2008 : cluster [DBG] pgmap v1130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:08 gibba011 bash[30791]: cluster 2021-03-10T14:30:08.045180+0000 mgr.gibba032.whopeo (mgr.24625) 2008 : cluster [DBG] pgmap v1130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:08 gibba011 bash[31774]: cluster 2021-03-10T14:30:08.045180+0000 mgr.gibba032.whopeo (mgr.24625) 2008 : cluster [DBG] pgmap v1130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:10 gibba032 bash[24004]: cluster 2021-03-10T14:30:10.045645+0000 mgr.gibba032.whopeo (mgr.24625) 2009 : cluster [DBG] pgmap v1131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:10 gibba011 bash[30791]: cluster 2021-03-10T14:30:10.045645+0000 mgr.gibba032.whopeo (mgr.24625) 2009 : cluster [DBG] pgmap v1131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:10 gibba011 bash[31774]: cluster 2021-03-10T14:30:10.045645+0000 mgr.gibba032.whopeo (mgr.24625) 2009 : cluster [DBG] pgmap v1131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:12 gibba032 bash[24004]: cluster 2021-03-10T14:30:12.047139+0000 mgr.gibba032.whopeo (mgr.24625) 2010 : cluster [DBG] pgmap v1132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:12 gibba011 bash[30791]: cluster 2021-03-10T14:30:12.047139+0000 mgr.gibba032.whopeo (mgr.24625) 2010 : cluster [DBG] pgmap v1132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:12 gibba011 bash[31774]: cluster 2021-03-10T14:30:12.047139+0000 mgr.gibba032.whopeo (mgr.24625) 2010 : cluster [DBG] pgmap v1132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:14 gibba032 bash[24004]: cluster 2021-03-10T14:30:14.048719+0000 mgr.gibba032.whopeo (mgr.24625) 2011 : cluster [DBG] pgmap v1133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:14 gibba011 bash[30791]: cluster 2021-03-10T14:30:14.048719+0000 mgr.gibba032.whopeo (mgr.24625) 2011 : cluster [DBG] pgmap v1133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:14 gibba011 bash[31774]: cluster 2021-03-10T14:30:14.048719+0000 mgr.gibba032.whopeo (mgr.24625) 2011 : cluster [DBG] pgmap v1133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:15.899 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:30:16.155 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:30:16.155 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (36m) 2m ago 42m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:30:16.155 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (42m) 89s ago 42m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:30:16.156 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (37m) 89s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:30:16.156 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (37m) 89s ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:30:16.156 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (36m) 2m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:30:16.156 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (36m) 2m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:30:16.156 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (35m) 89s ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:30:16.157 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (36m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:30:16.157 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (42m) 2m ago 42m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:30:16.157 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (42m) 89s ago 42m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:30:16.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (35m) 2m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:30:16.157 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (34m) 2m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:30:16.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (34m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:30:16.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (33m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:30:16.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (33m) 89s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:30:16.158 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (33m) 89s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:30:16.159 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (32m) 89s ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:30:16.159 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (32m) 89s ago 42m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:30:16.159 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (41m) 89s ago 42m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:30:16.159 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 40m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:30:16.418 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:30:16.418 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:30:16.418 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:30:16.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:16.419 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:30:16.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:30:16.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:16.420 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:30:16.420 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:30:16.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:16.420 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:30:16.421 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:30:16.421 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:30:16.421 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:30:16.422 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:30:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:16 gibba032 bash[24004]: audit 2021-03-10T14:30:15.885769+0000 mgr.gibba032.whopeo (mgr.24625) 2012 : audit [DBG] from='client.55390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:16 gibba032 bash[24004]: cluster 2021-03-10T14:30:16.050036+0000 mgr.gibba032.whopeo (mgr.24625) 2013 : cluster [DBG] pgmap v1134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:16 gibba032 bash[24004]: audit 2021-03-10T14:30:16.417734+0000 mon.c (mon.1) 40 : audit [DBG] from='client.? 172.21.2.111:0/1391193495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[30791]: audit 2021-03-10T14:30:15.885769+0000 mgr.gibba032.whopeo (mgr.24625) 2012 : audit [DBG] from='client.55390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[30791]: cluster 2021-03-10T14:30:16.050036+0000 mgr.gibba032.whopeo (mgr.24625) 2013 : cluster [DBG] pgmap v1134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[30791]: audit 2021-03-10T14:30:16.417734+0000 mon.c (mon.1) 40 : audit [DBG] from='client.? 172.21.2.111:0/1391193495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[31774]: audit 2021-03-10T14:30:15.885769+0000 mgr.gibba032.whopeo (mgr.24625) 2012 : audit [DBG] from='client.55390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[31774]: cluster 2021-03-10T14:30:16.050036+0000 mgr.gibba032.whopeo (mgr.24625) 2013 : cluster [DBG] pgmap v1134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:16 gibba011 bash[31774]: audit 2021-03-10T14:30:16.417734+0000 mon.c (mon.1) 40 : audit [DBG] from='client.? 172.21.2.111:0/1391193495' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:17 gibba032 bash[24004]: audit 2021-03-10T14:30:16.110735+0000 mgr.gibba032.whopeo (mgr.24625) 2014 : audit [DBG] from='client.55396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:17 gibba032 bash[24004]: cephadm 2021-03-10T14:30:17.014309+0000 mgr.gibba032.whopeo (mgr.24625) 2015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:17 gibba011 bash[30791]: audit 2021-03-10T14:30:16.110735+0000 mgr.gibba032.whopeo (mgr.24625) 2014 : audit [DBG] from='client.55396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:17 gibba011 bash[30791]: cephadm 2021-03-10T14:30:17.014309+0000 mgr.gibba032.whopeo (mgr.24625) 2015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:18.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:17 gibba011 bash[31774]: audit 2021-03-10T14:30:16.110735+0000 mgr.gibba032.whopeo (mgr.24625) 2014 : audit [DBG] from='client.55396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:17 gibba011 bash[31774]: cephadm 2021-03-10T14:30:17.014309+0000 mgr.gibba032.whopeo (mgr.24625) 2015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:18 gibba032 bash[24004]: cluster 2021-03-10T14:30:18.050849+0000 mgr.gibba032.whopeo (mgr.24625) 2016 : cluster [DBG] pgmap v1135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:18 gibba011 bash[30791]: cluster 2021-03-10T14:30:18.050849+0000 mgr.gibba032.whopeo (mgr.24625) 2016 : cluster [DBG] pgmap v1135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:18 gibba011 bash[31774]: cluster 2021-03-10T14:30:18.050849+0000 mgr.gibba032.whopeo (mgr.24625) 2016 : cluster [DBG] pgmap v1135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:20 gibba032 bash[24004]: cluster 2021-03-10T14:30:20.051576+0000 mgr.gibba032.whopeo (mgr.24625) 2017 : cluster [DBG] pgmap v1136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:20 gibba011 bash[30791]: cluster 2021-03-10T14:30:20.051576+0000 mgr.gibba032.whopeo (mgr.24625) 2017 : cluster [DBG] pgmap v1136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:20 gibba011 bash[31774]: cluster 2021-03-10T14:30:20.051576+0000 mgr.gibba032.whopeo (mgr.24625) 2017 : cluster [DBG] pgmap v1136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:22 gibba032 bash[24004]: cluster 2021-03-10T14:30:22.053144+0000 mgr.gibba032.whopeo (mgr.24625) 2018 : cluster [DBG] pgmap v1137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:22 gibba011 bash[30791]: cluster 2021-03-10T14:30:22.053144+0000 mgr.gibba032.whopeo (mgr.24625) 2018 : cluster [DBG] pgmap v1137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:22 gibba011 bash[31774]: cluster 2021-03-10T14:30:22.053144+0000 mgr.gibba032.whopeo (mgr.24625) 2018 : cluster [DBG] pgmap v1137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:24 gibba032 bash[24004]: cluster 2021-03-10T14:30:24.054717+0000 mgr.gibba032.whopeo (mgr.24625) 2019 : cluster [DBG] pgmap v1138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:24 gibba011 bash[30791]: cluster 2021-03-10T14:30:24.054717+0000 mgr.gibba032.whopeo (mgr.24625) 2019 : cluster [DBG] pgmap v1138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:24 gibba011 bash[31774]: cluster 2021-03-10T14:30:24.054717+0000 mgr.gibba032.whopeo (mgr.24625) 2019 : cluster [DBG] pgmap v1138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:26 gibba032 bash[24004]: cluster 2021-03-10T14:30:26.055371+0000 mgr.gibba032.whopeo (mgr.24625) 2020 : cluster [DBG] pgmap v1139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:26 gibba011 bash[30791]: cluster 2021-03-10T14:30:26.055371+0000 mgr.gibba032.whopeo (mgr.24625) 2020 : cluster [DBG] pgmap v1139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:26 gibba011 bash[31774]: cluster 2021-03-10T14:30:26.055371+0000 mgr.gibba032.whopeo (mgr.24625) 2020 : cluster [DBG] pgmap v1139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:28 gibba032 bash[24004]: cluster 2021-03-10T14:30:28.056070+0000 mgr.gibba032.whopeo (mgr.24625) 2021 : cluster [DBG] pgmap v1140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:28 gibba011 bash[30791]: cluster 2021-03-10T14:30:28.056070+0000 mgr.gibba032.whopeo (mgr.24625) 2021 : cluster [DBG] pgmap v1140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:28 gibba011 bash[31774]: cluster 2021-03-10T14:30:28.056070+0000 mgr.gibba032.whopeo (mgr.24625) 2021 : cluster [DBG] pgmap v1140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:30 gibba032 bash[24004]: cluster 2021-03-10T14:30:30.056655+0000 mgr.gibba032.whopeo (mgr.24625) 2022 : cluster [DBG] pgmap v1141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:30 gibba011 bash[30791]: cluster 2021-03-10T14:30:30.056655+0000 mgr.gibba032.whopeo (mgr.24625) 2022 : cluster [DBG] pgmap v1141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:30 gibba011 bash[31774]: cluster 2021-03-10T14:30:30.056655+0000 mgr.gibba032.whopeo (mgr.24625) 2022 : cluster [DBG] pgmap v1141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:32 gibba032 bash[24004]: cephadm 2021-03-10T14:30:32.030362+0000 mgr.gibba032.whopeo (mgr.24625) 2023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:32 gibba032 bash[24004]: cluster 2021-03-10T14:30:32.058081+0000 mgr.gibba032.whopeo (mgr.24625) 2024 : cluster [DBG] pgmap v1142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:32 gibba011 bash[30791]: cephadm 2021-03-10T14:30:32.030362+0000 mgr.gibba032.whopeo (mgr.24625) 2023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:32 gibba011 bash[30791]: cluster 2021-03-10T14:30:32.058081+0000 mgr.gibba032.whopeo (mgr.24625) 2024 : cluster [DBG] pgmap v1142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:32 gibba011 bash[31774]: cephadm 2021-03-10T14:30:32.030362+0000 mgr.gibba032.whopeo (mgr.24625) 2023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:32 gibba011 bash[31774]: cluster 2021-03-10T14:30:32.058081+0000 mgr.gibba032.whopeo (mgr.24625) 2024 : cluster [DBG] pgmap v1142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:34 gibba032 bash[24004]: cluster 2021-03-10T14:30:34.058907+0000 mgr.gibba032.whopeo (mgr.24625) 2025 : cluster [DBG] pgmap v1143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:34 gibba011 bash[30791]: cluster 2021-03-10T14:30:34.058907+0000 mgr.gibba032.whopeo (mgr.24625) 2025 : cluster [DBG] pgmap v1143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:34 gibba011 bash[31774]: cluster 2021-03-10T14:30:34.058907+0000 mgr.gibba032.whopeo (mgr.24625) 2025 : cluster [DBG] pgmap v1143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:36 gibba032 bash[24004]: cluster 2021-03-10T14:30:36.060168+0000 mgr.gibba032.whopeo (mgr.24625) 2026 : cluster [DBG] pgmap v1144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:36 gibba011 bash[30791]: cluster 2021-03-10T14:30:36.060168+0000 mgr.gibba032.whopeo (mgr.24625) 2026 : cluster [DBG] pgmap v1144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:36 gibba011 bash[31774]: cluster 2021-03-10T14:30:36.060168+0000 mgr.gibba032.whopeo (mgr.24625) 2026 : cluster [DBG] pgmap v1144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:38 gibba032 bash[24004]: cluster 2021-03-10T14:30:38.060772+0000 mgr.gibba032.whopeo (mgr.24625) 2027 : cluster [DBG] pgmap v1145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:38 gibba011 bash[30791]: cluster 2021-03-10T14:30:38.060772+0000 mgr.gibba032.whopeo (mgr.24625) 2027 : cluster [DBG] pgmap v1145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:38 gibba011 bash[31774]: cluster 2021-03-10T14:30:38.060772+0000 mgr.gibba032.whopeo (mgr.24625) 2027 : cluster [DBG] pgmap v1145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:40 gibba032 bash[24004]: cluster 2021-03-10T14:30:40.061426+0000 mgr.gibba032.whopeo (mgr.24625) 2028 : cluster [DBG] pgmap v1146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:40 gibba011 bash[30791]: cluster 2021-03-10T14:30:40.061426+0000 mgr.gibba032.whopeo (mgr.24625) 2028 : cluster [DBG] pgmap v1146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:40 gibba011 bash[31774]: cluster 2021-03-10T14:30:40.061426+0000 mgr.gibba032.whopeo (mgr.24625) 2028 : cluster [DBG] pgmap v1146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:42 gibba032 bash[24004]: cluster 2021-03-10T14:30:42.062174+0000 mgr.gibba032.whopeo (mgr.24625) 2029 : cluster [DBG] pgmap v1147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:42 gibba011 bash[30791]: cluster 2021-03-10T14:30:42.062174+0000 mgr.gibba032.whopeo (mgr.24625) 2029 : cluster [DBG] pgmap v1147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:42 gibba011 bash[31774]: cluster 2021-03-10T14:30:42.062174+0000 mgr.gibba032.whopeo (mgr.24625) 2029 : cluster [DBG] pgmap v1147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:44 gibba032 bash[24004]: cluster 2021-03-10T14:30:44.063584+0000 mgr.gibba032.whopeo (mgr.24625) 2030 : cluster [DBG] pgmap v1148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:44 gibba011 bash[30791]: cluster 2021-03-10T14:30:44.063584+0000 mgr.gibba032.whopeo (mgr.24625) 2030 : cluster [DBG] pgmap v1148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:44 gibba011 bash[31774]: cluster 2021-03-10T14:30:44.063584+0000 mgr.gibba032.whopeo (mgr.24625) 2030 : cluster [DBG] pgmap v1148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:46.660 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:30:46.890 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:30:46.890 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (37m) 3m ago 42m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:30:46.890 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (42m) 2m ago 42m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:30:46.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (37m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:30:46.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (38m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:30:46.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (37m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:30:46.891 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (36m) 3m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:30:46.891 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (36m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (36m) 3m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (42m) 3m ago 42m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (42m) 2m ago 42m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (35m) 3m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (35m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:30:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (34m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:30:46.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (34m) 3m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:30:46.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (34m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:30:46.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (33m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:30:46.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (33m) 2m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:30:46.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (32m) 2m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:30:46.894 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (42m) 2m ago 42m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:30:46.894 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 40m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:30:47.146 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:30:47.146 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:30:47.146 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:30:47.147 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:47.147 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:30:47.147 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:30:47.147 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:47.147 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:30:47.148 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:30:47.148 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:30:47.148 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:30:47.148 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:30:47.149 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:30:47.149 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:30:47.149 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:30:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:46 gibba032 bash[24004]: cluster 2021-03-10T14:30:46.064946+0000 mgr.gibba032.whopeo (mgr.24625) 2031 : cluster [DBG] pgmap v1149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:47.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:46 gibba011 bash[30791]: cluster 2021-03-10T14:30:46.064946+0000 mgr.gibba032.whopeo (mgr.24625) 2031 : cluster [DBG] pgmap v1149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:46 gibba011 bash[31774]: cluster 2021-03-10T14:30:46.064946+0000 mgr.gibba032.whopeo (mgr.24625) 2031 : cluster [DBG] pgmap v1149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:46.647961+0000 mgr.gibba032.whopeo (mgr.24625) 2032 : audit [DBG] from='client.55408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:46.874217+0000 mgr.gibba032.whopeo (mgr.24625) 2033 : audit [DBG] from='client.45429 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.047048+0000 mgr.gibba032.whopeo (mgr.24625) 2034 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:30:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.048684+0000 mgr.gibba032.whopeo (mgr.24625) 2035 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:30:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.145887+0000 mon.c (mon.1) 41 : audit [DBG] from='client.? 172.21.2.111:0/1581182026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.246909+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:30:48.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.250071+0000 mon.a (mon.0) 1141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:30:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.251443+0000 mon.a (mon.0) 1142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.252867+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.254192+0000 mon.a (mon.0) 1144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.256606+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:47 gibba032 bash[24004]: audit 2021-03-10T14:30:47.257984+0000 mon.a (mon.0) 1146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:46.647961+0000 mgr.gibba032.whopeo (mgr.24625) 2032 : audit [DBG] from='client.55408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:46.874217+0000 mgr.gibba032.whopeo (mgr.24625) 2033 : audit [DBG] from='client.45429 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.047048+0000 mgr.gibba032.whopeo (mgr.24625) 2034 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:30:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.048684+0000 mgr.gibba032.whopeo (mgr.24625) 2035 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:30:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.145887+0000 mon.c (mon.1) 41 : audit [DBG] from='client.? 172.21.2.111:0/1581182026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.246909+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:30:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.250071+0000 mon.a (mon.0) 1141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:30:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.251443+0000 mon.a (mon.0) 1142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.252867+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.254192+0000 mon.a (mon.0) 1144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.256606+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[30791]: audit 2021-03-10T14:30:47.257984+0000 mon.a (mon.0) 1146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:46.647961+0000 mgr.gibba032.whopeo (mgr.24625) 2032 : audit [DBG] from='client.55408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:46.874217+0000 mgr.gibba032.whopeo (mgr.24625) 2033 : audit [DBG] from='client.45429 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:30:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.047048+0000 mgr.gibba032.whopeo (mgr.24625) 2034 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:30:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.048684+0000 mgr.gibba032.whopeo (mgr.24625) 2035 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:30:48.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.145887+0000 mon.c (mon.1) 41 : audit [DBG] from='client.? 172.21.2.111:0/1581182026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.246909+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:30:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.250071+0000 mon.a (mon.0) 1141 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:30:48.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.251443+0000 mon.a (mon.0) 1142 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.252867+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.254192+0000 mon.a (mon.0) 1144 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.256606+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:48.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:47 gibba011 bash[31774]: audit 2021-03-10T14:30:47.257984+0000 mon.a (mon.0) 1146 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:30:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.246216+0000 mgr.gibba032.whopeo (mgr.24625) 2036 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:30:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.247845+0000 mgr.gibba032.whopeo (mgr.24625) 2037 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:30:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.249174+0000 mgr.gibba032.whopeo (mgr.24625) 2038 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:30:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.249264+0000 mgr.gibba032.whopeo (mgr.24625) 2039 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:30:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.250606+0000 mgr.gibba032.whopeo (mgr.24625) 2040 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:30:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.250727+0000 mgr.gibba032.whopeo (mgr.24625) 2041 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:30:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.251877+0000 mgr.gibba032.whopeo (mgr.24625) 2042 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:30:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.252971+0000 mgr.gibba032.whopeo (mgr.24625) 2043 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:30:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.253008+0000 mgr.gibba032.whopeo (mgr.24625) 2044 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:30:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.254233+0000 mgr.gibba032.whopeo (mgr.24625) 2045 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:30:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.254548+0000 mgr.gibba032.whopeo (mgr.24625) 2046 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:30:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.255599+0000 mgr.gibba032.whopeo (mgr.24625) 2047 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:30:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.255638+0000 mgr.gibba032.whopeo (mgr.24625) 2048 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:30:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cephadm 2021-03-10T14:30:47.255742+0000 mgr.gibba032.whopeo (mgr.24625) 2049 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:48 gibba032 bash[24004]: cluster 2021-03-10T14:30:48.065838+0000 mgr.gibba032.whopeo (mgr.24625) 2050 : cluster [DBG] pgmap v1150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.246216+0000 mgr.gibba032.whopeo (mgr.24625) 2036 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:30:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.247845+0000 mgr.gibba032.whopeo (mgr.24625) 2037 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:30:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.249174+0000 mgr.gibba032.whopeo (mgr.24625) 2038 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:30:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.249264+0000 mgr.gibba032.whopeo (mgr.24625) 2039 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:30:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.250606+0000 mgr.gibba032.whopeo (mgr.24625) 2040 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:30:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.250727+0000 mgr.gibba032.whopeo (mgr.24625) 2041 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:30:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.251877+0000 mgr.gibba032.whopeo (mgr.24625) 2042 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:30:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.252971+0000 mgr.gibba032.whopeo (mgr.24625) 2043 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:30:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.253008+0000 mgr.gibba032.whopeo (mgr.24625) 2044 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:30:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.254233+0000 mgr.gibba032.whopeo (mgr.24625) 2045 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:30:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.254548+0000 mgr.gibba032.whopeo (mgr.24625) 2046 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:30:49.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.255599+0000 mgr.gibba032.whopeo (mgr.24625) 2047 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:30:49.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.255638+0000 mgr.gibba032.whopeo (mgr.24625) 2048 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:30:49.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cephadm 2021-03-10T14:30:47.255742+0000 mgr.gibba032.whopeo (mgr.24625) 2049 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:49.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[30791]: cluster 2021-03-10T14:30:48.065838+0000 mgr.gibba032.whopeo (mgr.24625) 2050 : cluster [DBG] pgmap v1150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:49.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.246216+0000 mgr.gibba032.whopeo (mgr.24625) 2036 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:30:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.247845+0000 mgr.gibba032.whopeo (mgr.24625) 2037 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:30:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.249174+0000 mgr.gibba032.whopeo (mgr.24625) 2038 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:30:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.249264+0000 mgr.gibba032.whopeo (mgr.24625) 2039 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:30:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.250606+0000 mgr.gibba032.whopeo (mgr.24625) 2040 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:30:49.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.250727+0000 mgr.gibba032.whopeo (mgr.24625) 2041 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:30:49.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.251877+0000 mgr.gibba032.whopeo (mgr.24625) 2042 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:30:49.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.252971+0000 mgr.gibba032.whopeo (mgr.24625) 2043 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:30:49.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.253008+0000 mgr.gibba032.whopeo (mgr.24625) 2044 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:30:49.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.254233+0000 mgr.gibba032.whopeo (mgr.24625) 2045 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:30:49.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.254548+0000 mgr.gibba032.whopeo (mgr.24625) 2046 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:30:49.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.255599+0000 mgr.gibba032.whopeo (mgr.24625) 2047 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:30:49.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.255638+0000 mgr.gibba032.whopeo (mgr.24625) 2048 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:30:49.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cephadm 2021-03-10T14:30:47.255742+0000 mgr.gibba032.whopeo (mgr.24625) 2049 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:30:49.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:48 gibba011 bash[31774]: cluster 2021-03-10T14:30:48.065838+0000 mgr.gibba032.whopeo (mgr.24625) 2050 : cluster [DBG] pgmap v1150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:50 gibba032 bash[24004]: cluster 2021-03-10T14:30:50.066592+0000 mgr.gibba032.whopeo (mgr.24625) 2051 : cluster [DBG] pgmap v1151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:50 gibba011 bash[30791]: cluster 2021-03-10T14:30:50.066592+0000 mgr.gibba032.whopeo (mgr.24625) 2051 : cluster [DBG] pgmap v1151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:50 gibba011 bash[31774]: cluster 2021-03-10T14:30:50.066592+0000 mgr.gibba032.whopeo (mgr.24625) 2051 : cluster [DBG] pgmap v1151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:52 gibba032 bash[24004]: cluster 2021-03-10T14:30:52.068217+0000 mgr.gibba032.whopeo (mgr.24625) 2052 : cluster [DBG] pgmap v1152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:52 gibba011 bash[30791]: cluster 2021-03-10T14:30:52.068217+0000 mgr.gibba032.whopeo (mgr.24625) 2052 : cluster [DBG] pgmap v1152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:52 gibba011 bash[31774]: cluster 2021-03-10T14:30:52.068217+0000 mgr.gibba032.whopeo (mgr.24625) 2052 : cluster [DBG] pgmap v1152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:54 gibba032 bash[24004]: cluster 2021-03-10T14:30:54.069588+0000 mgr.gibba032.whopeo (mgr.24625) 2053 : cluster [DBG] pgmap v1153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:54 gibba011 bash[30791]: cluster 2021-03-10T14:30:54.069588+0000 mgr.gibba032.whopeo (mgr.24625) 2053 : cluster [DBG] pgmap v1153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:54 gibba011 bash[31774]: cluster 2021-03-10T14:30:54.069588+0000 mgr.gibba032.whopeo (mgr.24625) 2053 : cluster [DBG] pgmap v1153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: cluster 2021-03-10T14:30:56.071011+0000 mgr.gibba032.whopeo (mgr.24625) 2054 : cluster [DBG] pgmap v1154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.222305+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:30:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.223108+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:30:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.223838+0000 mon.a (mon.0) 1149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.224524+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.229540+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:30:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:56 gibba032 bash[24004]: audit 2021-03-10T14:30:56.230183+0000 mon.a (mon.0) 1152 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:30:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: cluster 2021-03-10T14:30:56.071011+0000 mgr.gibba032.whopeo (mgr.24625) 2054 : cluster [DBG] pgmap v1154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.222305+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:30:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.223108+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:30:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.223838+0000 mon.a (mon.0) 1149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.224524+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.229540+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:30:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[30791]: audit 2021-03-10T14:30:56.230183+0000 mon.a (mon.0) 1152 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:30:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: cluster 2021-03-10T14:30:56.071011+0000 mgr.gibba032.whopeo (mgr.24625) 2054 : cluster [DBG] pgmap v1154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.222305+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:30:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.223108+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:30:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.223838+0000 mon.a (mon.0) 1149 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.224524+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:30:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.229540+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:30:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:56 gibba011 bash[31774]: audit 2021-03-10T14:30:56.230183+0000 mon.a (mon.0) 1152 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:30:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:30:58 gibba032 bash[24004]: cluster 2021-03-10T14:30:58.072702+0000 mgr.gibba032.whopeo (mgr.24625) 2055 : cluster [DBG] pgmap v1155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:30:58 gibba011 bash[30791]: cluster 2021-03-10T14:30:58.072702+0000 mgr.gibba032.whopeo (mgr.24625) 2055 : cluster [DBG] pgmap v1155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:30:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:30:58 gibba011 bash[31774]: cluster 2021-03-10T14:30:58.072702+0000 mgr.gibba032.whopeo (mgr.24625) 2055 : cluster [DBG] pgmap v1155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:00 gibba032 bash[24004]: cluster 2021-03-10T14:31:00.073497+0000 mgr.gibba032.whopeo (mgr.24625) 2056 : cluster [DBG] pgmap v1156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:00 gibba011 bash[30791]: cluster 2021-03-10T14:31:00.073497+0000 mgr.gibba032.whopeo (mgr.24625) 2056 : cluster [DBG] pgmap v1156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:00 gibba011 bash[31774]: cluster 2021-03-10T14:31:00.073497+0000 mgr.gibba032.whopeo (mgr.24625) 2056 : cluster [DBG] pgmap v1156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:02 gibba032 bash[24004]: cluster 2021-03-10T14:31:02.075090+0000 mgr.gibba032.whopeo (mgr.24625) 2057 : cluster [DBG] pgmap v1157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:02 gibba011 bash[30791]: cluster 2021-03-10T14:31:02.075090+0000 mgr.gibba032.whopeo (mgr.24625) 2057 : cluster [DBG] pgmap v1157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:02 gibba011 bash[31774]: cluster 2021-03-10T14:31:02.075090+0000 mgr.gibba032.whopeo (mgr.24625) 2057 : cluster [DBG] pgmap v1157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:03 gibba032 bash[24004]: cephadm 2021-03-10T14:31:02.271756+0000 mgr.gibba032.whopeo (mgr.24625) 2058 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:03 gibba011 bash[30791]: cephadm 2021-03-10T14:31:02.271756+0000 mgr.gibba032.whopeo (mgr.24625) 2058 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:03.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:03 gibba011 bash[31774]: cephadm 2021-03-10T14:31:02.271756+0000 mgr.gibba032.whopeo (mgr.24625) 2058 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:04 gibba032 bash[24004]: cluster 2021-03-10T14:31:04.076449+0000 mgr.gibba032.whopeo (mgr.24625) 2059 : cluster [DBG] pgmap v1158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:04 gibba011 bash[30791]: cluster 2021-03-10T14:31:04.076449+0000 mgr.gibba032.whopeo (mgr.24625) 2059 : cluster [DBG] pgmap v1158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:04 gibba011 bash[31774]: cluster 2021-03-10T14:31:04.076449+0000 mgr.gibba032.whopeo (mgr.24625) 2059 : cluster [DBG] pgmap v1158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:06 gibba032 bash[24004]: cluster 2021-03-10T14:31:06.077222+0000 mgr.gibba032.whopeo (mgr.24625) 2060 : cluster [DBG] pgmap v1159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:06 gibba011 bash[30791]: cluster 2021-03-10T14:31:06.077222+0000 mgr.gibba032.whopeo (mgr.24625) 2060 : cluster [DBG] pgmap v1159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:06 gibba011 bash[31774]: cluster 2021-03-10T14:31:06.077222+0000 mgr.gibba032.whopeo (mgr.24625) 2060 : cluster [DBG] pgmap v1159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:08 gibba032 bash[24004]: cluster 2021-03-10T14:31:08.077959+0000 mgr.gibba032.whopeo (mgr.24625) 2061 : cluster [DBG] pgmap v1160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:08 gibba011 bash[30791]: cluster 2021-03-10T14:31:08.077959+0000 mgr.gibba032.whopeo (mgr.24625) 2061 : cluster [DBG] pgmap v1160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:08 gibba011 bash[31774]: cluster 2021-03-10T14:31:08.077959+0000 mgr.gibba032.whopeo (mgr.24625) 2061 : cluster [DBG] pgmap v1160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:10 gibba032 bash[24004]: cluster 2021-03-10T14:31:10.078880+0000 mgr.gibba032.whopeo (mgr.24625) 2062 : cluster [DBG] pgmap v1161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:10 gibba011 bash[30791]: cluster 2021-03-10T14:31:10.078880+0000 mgr.gibba032.whopeo (mgr.24625) 2062 : cluster [DBG] pgmap v1161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:10 gibba011 bash[31774]: cluster 2021-03-10T14:31:10.078880+0000 mgr.gibba032.whopeo (mgr.24625) 2062 : cluster [DBG] pgmap v1161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:12 gibba032 bash[24004]: cluster 2021-03-10T14:31:12.080378+0000 mgr.gibba032.whopeo (mgr.24625) 2063 : cluster [DBG] pgmap v1162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:12 gibba011 bash[30791]: cluster 2021-03-10T14:31:12.080378+0000 mgr.gibba032.whopeo (mgr.24625) 2063 : cluster [DBG] pgmap v1162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:12 gibba011 bash[31774]: cluster 2021-03-10T14:31:12.080378+0000 mgr.gibba032.whopeo (mgr.24625) 2063 : cluster [DBG] pgmap v1162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:14 gibba032 bash[24004]: cluster 2021-03-10T14:31:14.081034+0000 mgr.gibba032.whopeo (mgr.24625) 2064 : cluster [DBG] pgmap v1163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:14 gibba011 bash[30791]: cluster 2021-03-10T14:31:14.081034+0000 mgr.gibba032.whopeo (mgr.24625) 2064 : cluster [DBG] pgmap v1163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:14.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:14 gibba011 bash[31774]: cluster 2021-03-10T14:31:14.081034+0000 mgr.gibba032.whopeo (mgr.24625) 2064 : cluster [DBG] pgmap v1163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:16 gibba032 bash[24004]: cluster 2021-03-10T14:31:16.082300+0000 mgr.gibba032.whopeo (mgr.24625) 2065 : cluster [DBG] pgmap v1164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:16 gibba011 bash[30791]: cluster 2021-03-10T14:31:16.082300+0000 mgr.gibba032.whopeo (mgr.24625) 2065 : cluster [DBG] pgmap v1164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:16 gibba011 bash[31774]: cluster 2021-03-10T14:31:16.082300+0000 mgr.gibba032.whopeo (mgr.24625) 2065 : cluster [DBG] pgmap v1164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:17.386 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:31:17.611 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:31:17.611 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (37m) 3m ago 43m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:31:17.612 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (43m) 2m ago 43m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:31:17.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (38m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:31:17.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (38m) 2m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:31:17.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (37m) 3m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:31:17.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (37m) 3m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:31:17.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (37m) 2m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:31:17.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (37m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:31:17.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (43m) 3m ago 43m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:31:17.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (43m) 2m ago 43m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:31:17.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (36m) 3m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:31:17.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (35m) 3m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:31:17.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (35m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:31:17.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (34m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:31:17.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (34m) 2m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:31:17.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (34m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:31:17.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (33m) 2m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:31:17.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (33m) 2m ago 43m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:31:17.615 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (42m) 2m ago 43m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:31:17.615 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 41m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:31:17.860 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:31:17.860 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:31:17.861 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:31:17.861 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:17.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:31:17.861 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:31:17.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:17.862 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:31:17.862 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:31:17.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:17.863 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:31:17.863 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:31:17.863 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:31:17.863 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:31:17.863 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:31:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:17 gibba032 bash[24004]: audit 2021-03-10T14:31:17.859910+0000 mon.c (mon.1) 42 : audit [DBG] from='client.? 172.21.2.111:0/2980387357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:17 gibba011 bash[30791]: audit 2021-03-10T14:31:17.859910+0000 mon.c (mon.1) 42 : audit [DBG] from='client.? 172.21.2.111:0/2980387357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:17 gibba011 bash[31774]: audit 2021-03-10T14:31:17.859910+0000 mon.c (mon.1) 42 : audit [DBG] from='client.? 172.21.2.111:0/2980387357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:18 gibba032 bash[24004]: cephadm 2021-03-10T14:31:17.287678+0000 mgr.gibba032.whopeo (mgr.24625) 2066 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:18 gibba032 bash[24004]: audit 2021-03-10T14:31:17.373975+0000 mgr.gibba032.whopeo (mgr.24625) 2067 : audit [DBG] from='client.45441 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:18 gibba032 bash[24004]: audit 2021-03-10T14:31:17.595884+0000 mgr.gibba032.whopeo (mgr.24625) 2068 : audit [DBG] from='client.45447 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:18 gibba032 bash[24004]: cluster 2021-03-10T14:31:18.084092+0000 mgr.gibba032.whopeo (mgr.24625) 2069 : cluster [DBG] pgmap v1165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[30791]: cephadm 2021-03-10T14:31:17.287678+0000 mgr.gibba032.whopeo (mgr.24625) 2066 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[30791]: audit 2021-03-10T14:31:17.373975+0000 mgr.gibba032.whopeo (mgr.24625) 2067 : audit [DBG] from='client.45441 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[30791]: audit 2021-03-10T14:31:17.595884+0000 mgr.gibba032.whopeo (mgr.24625) 2068 : audit [DBG] from='client.45447 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[30791]: cluster 2021-03-10T14:31:18.084092+0000 mgr.gibba032.whopeo (mgr.24625) 2069 : cluster [DBG] pgmap v1165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[31774]: cephadm 2021-03-10T14:31:17.287678+0000 mgr.gibba032.whopeo (mgr.24625) 2066 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[31774]: audit 2021-03-10T14:31:17.373975+0000 mgr.gibba032.whopeo (mgr.24625) 2067 : audit [DBG] from='client.45441 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[31774]: audit 2021-03-10T14:31:17.595884+0000 mgr.gibba032.whopeo (mgr.24625) 2068 : audit [DBG] from='client.45447 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:18 gibba011 bash[31774]: cluster 2021-03-10T14:31:18.084092+0000 mgr.gibba032.whopeo (mgr.24625) 2069 : cluster [DBG] pgmap v1165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:20 gibba032 bash[24004]: cluster 2021-03-10T14:31:20.084954+0000 mgr.gibba032.whopeo (mgr.24625) 2070 : cluster [DBG] pgmap v1166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:20 gibba011 bash[30791]: cluster 2021-03-10T14:31:20.084954+0000 mgr.gibba032.whopeo (mgr.24625) 2070 : cluster [DBG] pgmap v1166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:20 gibba011 bash[31774]: cluster 2021-03-10T14:31:20.084954+0000 mgr.gibba032.whopeo (mgr.24625) 2070 : cluster [DBG] pgmap v1166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:22 gibba032 bash[24004]: cluster 2021-03-10T14:31:22.085708+0000 mgr.gibba032.whopeo (mgr.24625) 2071 : cluster [DBG] pgmap v1167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:22 gibba011 bash[30791]: cluster 2021-03-10T14:31:22.085708+0000 mgr.gibba032.whopeo (mgr.24625) 2071 : cluster [DBG] pgmap v1167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:22 gibba011 bash[31774]: cluster 2021-03-10T14:31:22.085708+0000 mgr.gibba032.whopeo (mgr.24625) 2071 : cluster [DBG] pgmap v1167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:24 gibba032 bash[24004]: cluster 2021-03-10T14:31:24.087047+0000 mgr.gibba032.whopeo (mgr.24625) 2072 : cluster [DBG] pgmap v1168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:24 gibba011 bash[30791]: cluster 2021-03-10T14:31:24.087047+0000 mgr.gibba032.whopeo (mgr.24625) 2072 : cluster [DBG] pgmap v1168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:24 gibba011 bash[31774]: cluster 2021-03-10T14:31:24.087047+0000 mgr.gibba032.whopeo (mgr.24625) 2072 : cluster [DBG] pgmap v1168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:26 gibba032 bash[24004]: cluster 2021-03-10T14:31:26.088413+0000 mgr.gibba032.whopeo (mgr.24625) 2073 : cluster [DBG] pgmap v1169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:26 gibba011 bash[30791]: cluster 2021-03-10T14:31:26.088413+0000 mgr.gibba032.whopeo (mgr.24625) 2073 : cluster [DBG] pgmap v1169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:26 gibba011 bash[31774]: cluster 2021-03-10T14:31:26.088413+0000 mgr.gibba032.whopeo (mgr.24625) 2073 : cluster [DBG] pgmap v1169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:28 gibba032 bash[24004]: cluster 2021-03-10T14:31:28.089238+0000 mgr.gibba032.whopeo (mgr.24625) 2074 : cluster [DBG] pgmap v1170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:28 gibba011 bash[30791]: cluster 2021-03-10T14:31:28.089238+0000 mgr.gibba032.whopeo (mgr.24625) 2074 : cluster [DBG] pgmap v1170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:28 gibba011 bash[31774]: cluster 2021-03-10T14:31:28.089238+0000 mgr.gibba032.whopeo (mgr.24625) 2074 : cluster [DBG] pgmap v1170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:30 gibba032 bash[24004]: cluster 2021-03-10T14:31:30.090495+0000 mgr.gibba032.whopeo (mgr.24625) 2075 : cluster [DBG] pgmap v1171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:30 gibba011 bash[30791]: cluster 2021-03-10T14:31:30.090495+0000 mgr.gibba032.whopeo (mgr.24625) 2075 : cluster [DBG] pgmap v1171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:30 gibba011 bash[31774]: cluster 2021-03-10T14:31:30.090495+0000 mgr.gibba032.whopeo (mgr.24625) 2075 : cluster [DBG] pgmap v1171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:32 gibba032 bash[24004]: cluster 2021-03-10T14:31:32.092151+0000 mgr.gibba032.whopeo (mgr.24625) 2076 : cluster [DBG] pgmap v1172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:32 gibba011 bash[30791]: cluster 2021-03-10T14:31:32.092151+0000 mgr.gibba032.whopeo (mgr.24625) 2076 : cluster [DBG] pgmap v1172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:32 gibba011 bash[31774]: cluster 2021-03-10T14:31:32.092151+0000 mgr.gibba032.whopeo (mgr.24625) 2076 : cluster [DBG] pgmap v1172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:33 gibba032 bash[24004]: cephadm 2021-03-10T14:31:32.296840+0000 mgr.gibba032.whopeo (mgr.24625) 2077 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:33.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:33 gibba011 bash[30791]: cephadm 2021-03-10T14:31:32.296840+0000 mgr.gibba032.whopeo (mgr.24625) 2077 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:33.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:33 gibba011 bash[31774]: cephadm 2021-03-10T14:31:32.296840+0000 mgr.gibba032.whopeo (mgr.24625) 2077 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:34 gibba032 bash[24004]: cluster 2021-03-10T14:31:34.093406+0000 mgr.gibba032.whopeo (mgr.24625) 2078 : cluster [DBG] pgmap v1173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:34 gibba011 bash[30791]: cluster 2021-03-10T14:31:34.093406+0000 mgr.gibba032.whopeo (mgr.24625) 2078 : cluster [DBG] pgmap v1173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:34 gibba011 bash[31774]: cluster 2021-03-10T14:31:34.093406+0000 mgr.gibba032.whopeo (mgr.24625) 2078 : cluster [DBG] pgmap v1173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:36 gibba032 bash[24004]: cluster 2021-03-10T14:31:36.094874+0000 mgr.gibba032.whopeo (mgr.24625) 2079 : cluster [DBG] pgmap v1174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:36 gibba011 bash[30791]: cluster 2021-03-10T14:31:36.094874+0000 mgr.gibba032.whopeo (mgr.24625) 2079 : cluster [DBG] pgmap v1174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:36 gibba011 bash[31774]: cluster 2021-03-10T14:31:36.094874+0000 mgr.gibba032.whopeo (mgr.24625) 2079 : cluster [DBG] pgmap v1174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:38 gibba032 bash[24004]: cluster 2021-03-10T14:31:38.096196+0000 mgr.gibba032.whopeo (mgr.24625) 2080 : cluster [DBG] pgmap v1175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:38 gibba011 bash[30791]: cluster 2021-03-10T14:31:38.096196+0000 mgr.gibba032.whopeo (mgr.24625) 2080 : cluster [DBG] pgmap v1175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:38 gibba011 bash[31774]: cluster 2021-03-10T14:31:38.096196+0000 mgr.gibba032.whopeo (mgr.24625) 2080 : cluster [DBG] pgmap v1175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:40 gibba032 bash[24004]: cluster 2021-03-10T14:31:40.097341+0000 mgr.gibba032.whopeo (mgr.24625) 2081 : cluster [DBG] pgmap v1176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:40 gibba011 bash[30791]: cluster 2021-03-10T14:31:40.097341+0000 mgr.gibba032.whopeo (mgr.24625) 2081 : cluster [DBG] pgmap v1176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:40 gibba011 bash[31774]: cluster 2021-03-10T14:31:40.097341+0000 mgr.gibba032.whopeo (mgr.24625) 2081 : cluster [DBG] pgmap v1176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:42 gibba032 bash[24004]: cluster 2021-03-10T14:31:42.098944+0000 mgr.gibba032.whopeo (mgr.24625) 2082 : cluster [DBG] pgmap v1177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:42 gibba011 bash[30791]: cluster 2021-03-10T14:31:42.098944+0000 mgr.gibba032.whopeo (mgr.24625) 2082 : cluster [DBG] pgmap v1177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:42 gibba011 bash[31774]: cluster 2021-03-10T14:31:42.098944+0000 mgr.gibba032.whopeo (mgr.24625) 2082 : cluster [DBG] pgmap v1177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:44 gibba032 bash[24004]: cluster 2021-03-10T14:31:44.100380+0000 mgr.gibba032.whopeo (mgr.24625) 2083 : cluster [DBG] pgmap v1178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:44 gibba011 bash[30791]: cluster 2021-03-10T14:31:44.100380+0000 mgr.gibba032.whopeo (mgr.24625) 2083 : cluster [DBG] pgmap v1178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:44 gibba011 bash[31774]: cluster 2021-03-10T14:31:44.100380+0000 mgr.gibba032.whopeo (mgr.24625) 2083 : cluster [DBG] pgmap v1178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:46 gibba032 bash[24004]: cluster 2021-03-10T14:31:46.101362+0000 mgr.gibba032.whopeo (mgr.24625) 2084 : cluster [DBG] pgmap v1179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:46 gibba011 bash[30791]: cluster 2021-03-10T14:31:46.101362+0000 mgr.gibba032.whopeo (mgr.24625) 2084 : cluster [DBG] pgmap v1179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:46 gibba011 bash[31774]: cluster 2021-03-10T14:31:46.101362+0000 mgr.gibba032.whopeo (mgr.24625) 2084 : cluster [DBG] pgmap v1179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:48.117 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:31:48.390 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:31:48.390 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (38m) 4m ago 43m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:31:48.390 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (43m) 3m ago 43m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:31:48.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (38m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:31:48.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (39m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:31:48.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (38m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:31:48.391 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (37m) 4m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:31:48.392 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (37m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:31:48.392 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (37m) 4m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:31:48.392 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (43m) 4m ago 43m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:31:48.392 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (43m) 3m ago 43m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:31:48.392 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (36m) 4m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:31:48.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (36m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:31:48.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (35m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:31:48.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (35m) 4m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:31:48.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (35m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:31:48.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (34m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:31:48.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (34m) 3m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:31:48.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (33m) 3m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:31:48.394 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (43m) 3m ago 43m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:31:48.394 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 41m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:31:48.650 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:31:48.650 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:31:48.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:31:48.651 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:48.651 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:31:48.651 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:31:48.651 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:48.651 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:31:48.652 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:31:48.652 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:31:48.652 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:31:48.652 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:31:48.652 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:31:48.653 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:31:48.653 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:31:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.313418+0000 mgr.gibba032.whopeo (mgr.24625) 2085 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:31:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.315011+0000 mgr.gibba032.whopeo (mgr.24625) 2086 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:31:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.521644+0000 mgr.gibba032.whopeo (mgr.24625) 2087 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:31:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.522088+0000 mon.a (mon.0) 1153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:31:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.523469+0000 mgr.gibba032.whopeo (mgr.24625) 2088 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:31:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.524784+0000 mgr.gibba032.whopeo (mgr.24625) 2089 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:31:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.524819+0000 mgr.gibba032.whopeo (mgr.24625) 2090 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:31:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.525461+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:31:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.526240+0000 mgr.gibba032.whopeo (mgr.24625) 2091 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:31:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.526277+0000 mgr.gibba032.whopeo (mgr.24625) 2092 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:31:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.526919+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.527311+0000 mgr.gibba032.whopeo (mgr.24625) 2093 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:31:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.528211+0000 mgr.gibba032.whopeo (mgr.24625) 2094 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:31:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.528277+0000 mgr.gibba032.whopeo (mgr.24625) 2095 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:31:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.528310+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.529317+0000 mgr.gibba032.whopeo (mgr.24625) 2096 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:31:48.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.529627+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.529631+0000 mgr.gibba032.whopeo (mgr.24625) 2097 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:31:48.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.530582+0000 mgr.gibba032.whopeo (mgr.24625) 2098 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:31:48.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.530642+0000 mgr.gibba032.whopeo (mgr.24625) 2099 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:31:48.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cephadm 2021-03-10T14:31:47.530818+0000 mgr.gibba032.whopeo (mgr.24625) 2100 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.313418+0000 mgr.gibba032.whopeo (mgr.24625) 2085 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:31:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.315011+0000 mgr.gibba032.whopeo (mgr.24625) 2086 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:31:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.521644+0000 mgr.gibba032.whopeo (mgr.24625) 2087 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:31:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.522088+0000 mon.a (mon.0) 1153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:31:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.523469+0000 mgr.gibba032.whopeo (mgr.24625) 2088 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:31:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.524784+0000 mgr.gibba032.whopeo (mgr.24625) 2089 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:31:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.524819+0000 mgr.gibba032.whopeo (mgr.24625) 2090 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:31:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.525461+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:31:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.526240+0000 mgr.gibba032.whopeo (mgr.24625) 2091 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:31:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.526277+0000 mgr.gibba032.whopeo (mgr.24625) 2092 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:31:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.526919+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.527311+0000 mgr.gibba032.whopeo (mgr.24625) 2093 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:31:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.528211+0000 mgr.gibba032.whopeo (mgr.24625) 2094 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:31:48.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.528277+0000 mgr.gibba032.whopeo (mgr.24625) 2095 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:31:48.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.528310+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.529317+0000 mgr.gibba032.whopeo (mgr.24625) 2096 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:31:48.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.529627+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.529631+0000 mgr.gibba032.whopeo (mgr.24625) 2097 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:31:48.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.530582+0000 mgr.gibba032.whopeo (mgr.24625) 2098 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:31:48.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.530642+0000 mgr.gibba032.whopeo (mgr.24625) 2099 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:31:48.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cephadm 2021-03-10T14:31:47.530818+0000 mgr.gibba032.whopeo (mgr.24625) 2100 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:48.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.531703+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:47.532987+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: audit 2021-03-10T14:31:48.096922+0000 mgr.gibba032.whopeo (mgr.24625) 2101 : audit [DBG] from='client.45459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:48.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[30791]: cluster 2021-03-10T14:31:48.102976+0000 mgr.gibba032.whopeo (mgr.24625) 2102 : cluster [DBG] pgmap v1180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:48.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.531703+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:47.532987+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:48.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: audit 2021-03-10T14:31:48.096922+0000 mgr.gibba032.whopeo (mgr.24625) 2101 : audit [DBG] from='client.45459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:48.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:48 gibba011 bash[31774]: cluster 2021-03-10T14:31:48.102976+0000 mgr.gibba032.whopeo (mgr.24625) 2102 : cluster [DBG] pgmap v1180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.313418+0000 mgr.gibba032.whopeo (mgr.24625) 2085 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:31:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.315011+0000 mgr.gibba032.whopeo (mgr.24625) 2086 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:31:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.521644+0000 mgr.gibba032.whopeo (mgr.24625) 2087 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:31:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.522088+0000 mon.a (mon.0) 1153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:31:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.523469+0000 mgr.gibba032.whopeo (mgr.24625) 2088 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:31:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.524784+0000 mgr.gibba032.whopeo (mgr.24625) 2089 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:31:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.524819+0000 mgr.gibba032.whopeo (mgr.24625) 2090 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:31:49.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.525461+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:31:49.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.526240+0000 mgr.gibba032.whopeo (mgr.24625) 2091 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:31:49.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.526277+0000 mgr.gibba032.whopeo (mgr.24625) 2092 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:31:49.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.526919+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.527311+0000 mgr.gibba032.whopeo (mgr.24625) 2093 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.528211+0000 mgr.gibba032.whopeo (mgr.24625) 2094 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.528277+0000 mgr.gibba032.whopeo (mgr.24625) 2095 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.528310+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.529317+0000 mgr.gibba032.whopeo (mgr.24625) 2096 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.529627+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.529631+0000 mgr.gibba032.whopeo (mgr.24625) 2097 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:31:49.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.530582+0000 mgr.gibba032.whopeo (mgr.24625) 2098 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:31:49.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.530642+0000 mgr.gibba032.whopeo (mgr.24625) 2099 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:31:49.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cephadm 2021-03-10T14:31:47.530818+0000 mgr.gibba032.whopeo (mgr.24625) 2100 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:31:49.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.531703+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:47.532987+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: audit 2021-03-10T14:31:48.096922+0000 mgr.gibba032.whopeo (mgr.24625) 2101 : audit [DBG] from='client.45459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:49.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:48 gibba032 bash[24004]: cluster 2021-03-10T14:31:48.102976+0000 mgr.gibba032.whopeo (mgr.24625) 2102 : cluster [DBG] pgmap v1180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:49.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:49 gibba011 bash[30791]: audit 2021-03-10T14:31:48.346022+0000 mgr.gibba032.whopeo (mgr.24625) 2103 : audit [DBG] from='client.55450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:49.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:49 gibba011 bash[30791]: audit 2021-03-10T14:31:48.649513+0000 mon.c (mon.1) 43 : audit [DBG] from='client.? 172.21.2.111:0/311670075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:49 gibba011 bash[31774]: audit 2021-03-10T14:31:48.346022+0000 mgr.gibba032.whopeo (mgr.24625) 2103 : audit [DBG] from='client.55450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:49 gibba011 bash[31774]: audit 2021-03-10T14:31:48.649513+0000 mon.c (mon.1) 43 : audit [DBG] from='client.? 172.21.2.111:0/311670075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:49 gibba032 bash[24004]: audit 2021-03-10T14:31:48.346022+0000 mgr.gibba032.whopeo (mgr.24625) 2103 : audit [DBG] from='client.55450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:31:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:49 gibba032 bash[24004]: audit 2021-03-10T14:31:48.649513+0000 mon.c (mon.1) 43 : audit [DBG] from='client.? 172.21.2.111:0/311670075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:31:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:50 gibba011 bash[30791]: cluster 2021-03-10T14:31:50.104494+0000 mgr.gibba032.whopeo (mgr.24625) 2104 : cluster [DBG] pgmap v1181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:50 gibba011 bash[31774]: cluster 2021-03-10T14:31:50.104494+0000 mgr.gibba032.whopeo (mgr.24625) 2104 : cluster [DBG] pgmap v1181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:50 gibba032 bash[24004]: cluster 2021-03-10T14:31:50.104494+0000 mgr.gibba032.whopeo (mgr.24625) 2104 : cluster [DBG] pgmap v1181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:52 gibba032 bash[24004]: cluster 2021-03-10T14:31:52.106213+0000 mgr.gibba032.whopeo (mgr.24625) 2105 : cluster [DBG] pgmap v1182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:52 gibba011 bash[30791]: cluster 2021-03-10T14:31:52.106213+0000 mgr.gibba032.whopeo (mgr.24625) 2105 : cluster [DBG] pgmap v1182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:52 gibba011 bash[31774]: cluster 2021-03-10T14:31:52.106213+0000 mgr.gibba032.whopeo (mgr.24625) 2105 : cluster [DBG] pgmap v1182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:54 gibba032 bash[24004]: cluster 2021-03-10T14:31:54.107193+0000 mgr.gibba032.whopeo (mgr.24625) 2106 : cluster [DBG] pgmap v1183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:54 gibba011 bash[30791]: cluster 2021-03-10T14:31:54.107193+0000 mgr.gibba032.whopeo (mgr.24625) 2106 : cluster [DBG] pgmap v1183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:54.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:54 gibba011 bash[31774]: cluster 2021-03-10T14:31:54.107193+0000 mgr.gibba032.whopeo (mgr.24625) 2106 : cluster [DBG] pgmap v1183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: cluster 2021-03-10T14:31:56.108469+0000 mgr.gibba032.whopeo (mgr.24625) 2107 : cluster [DBG] pgmap v1184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.242464+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:31:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.243148+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:31:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.243761+0000 mon.a (mon.0) 1162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:31:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.244468+0000 mon.a (mon.0) 1163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:31:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.245256+0000 mon.a (mon.0) 1164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:31:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:56 gibba032 bash[24004]: audit 2021-03-10T14:31:56.245951+0000 mon.a (mon.0) 1165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:31:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: cluster 2021-03-10T14:31:56.108469+0000 mgr.gibba032.whopeo (mgr.24625) 2107 : cluster [DBG] pgmap v1184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.242464+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:31:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.243148+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:31:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.243761+0000 mon.a (mon.0) 1162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:31:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.244468+0000 mon.a (mon.0) 1163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:31:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.245256+0000 mon.a (mon.0) 1164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:31:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[30791]: audit 2021-03-10T14:31:56.245951+0000 mon.a (mon.0) 1165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:31:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: cluster 2021-03-10T14:31:56.108469+0000 mgr.gibba032.whopeo (mgr.24625) 2107 : cluster [DBG] pgmap v1184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.242464+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:31:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.243148+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:31:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.243761+0000 mon.a (mon.0) 1162 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:31:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.244468+0000 mon.a (mon.0) 1163 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:31:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.245256+0000 mon.a (mon.0) 1164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:31:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:56 gibba011 bash[31774]: audit 2021-03-10T14:31:56.245951+0000 mon.a (mon.0) 1165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:31:58 gibba032 bash[24004]: cluster 2021-03-10T14:31:58.110132+0000 mgr.gibba032.whopeo (mgr.24625) 2108 : cluster [DBG] pgmap v1185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:31:58 gibba011 bash[30791]: cluster 2021-03-10T14:31:58.110132+0000 mgr.gibba032.whopeo (mgr.24625) 2108 : cluster [DBG] pgmap v1185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:31:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:31:58 gibba011 bash[31774]: cluster 2021-03-10T14:31:58.110132+0000 mgr.gibba032.whopeo (mgr.24625) 2108 : cluster [DBG] pgmap v1185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:00 gibba032 bash[24004]: cluster 2021-03-10T14:32:00.111687+0000 mgr.gibba032.whopeo (mgr.24625) 2109 : cluster [DBG] pgmap v1186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:00 gibba011 bash[30791]: cluster 2021-03-10T14:32:00.111687+0000 mgr.gibba032.whopeo (mgr.24625) 2109 : cluster [DBG] pgmap v1186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:00 gibba011 bash[31774]: cluster 2021-03-10T14:32:00.111687+0000 mgr.gibba032.whopeo (mgr.24625) 2109 : cluster [DBG] pgmap v1186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:02 gibba032 bash[24004]: cluster 2021-03-10T14:32:02.113109+0000 mgr.gibba032.whopeo (mgr.24625) 2110 : cluster [DBG] pgmap v1187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:02 gibba011 bash[30791]: cluster 2021-03-10T14:32:02.113109+0000 mgr.gibba032.whopeo (mgr.24625) 2110 : cluster [DBG] pgmap v1187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:02 gibba011 bash[31774]: cluster 2021-03-10T14:32:02.113109+0000 mgr.gibba032.whopeo (mgr.24625) 2110 : cluster [DBG] pgmap v1187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:03 gibba032 bash[24004]: cephadm 2021-03-10T14:32:02.546816+0000 mgr.gibba032.whopeo (mgr.24625) 2111 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:03 gibba011 bash[30791]: cephadm 2021-03-10T14:32:02.546816+0000 mgr.gibba032.whopeo (mgr.24625) 2111 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:03.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:03 gibba011 bash[31774]: cephadm 2021-03-10T14:32:02.546816+0000 mgr.gibba032.whopeo (mgr.24625) 2111 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:04 gibba032 bash[24004]: cluster 2021-03-10T14:32:04.114473+0000 mgr.gibba032.whopeo (mgr.24625) 2112 : cluster [DBG] pgmap v1188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:04 gibba011 bash[30791]: cluster 2021-03-10T14:32:04.114473+0000 mgr.gibba032.whopeo (mgr.24625) 2112 : cluster [DBG] pgmap v1188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:04 gibba011 bash[31774]: cluster 2021-03-10T14:32:04.114473+0000 mgr.gibba032.whopeo (mgr.24625) 2112 : cluster [DBG] pgmap v1188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:06 gibba032 bash[24004]: cluster 2021-03-10T14:32:06.116106+0000 mgr.gibba032.whopeo (mgr.24625) 2113 : cluster [DBG] pgmap v1189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:06 gibba011 bash[30791]: cluster 2021-03-10T14:32:06.116106+0000 mgr.gibba032.whopeo (mgr.24625) 2113 : cluster [DBG] pgmap v1189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:06 gibba011 bash[31774]: cluster 2021-03-10T14:32:06.116106+0000 mgr.gibba032.whopeo (mgr.24625) 2113 : cluster [DBG] pgmap v1189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:08 gibba032 bash[24004]: cluster 2021-03-10T14:32:08.117117+0000 mgr.gibba032.whopeo (mgr.24625) 2114 : cluster [DBG] pgmap v1190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:08 gibba011 bash[30791]: cluster 2021-03-10T14:32:08.117117+0000 mgr.gibba032.whopeo (mgr.24625) 2114 : cluster [DBG] pgmap v1190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:08 gibba011 bash[31774]: cluster 2021-03-10T14:32:08.117117+0000 mgr.gibba032.whopeo (mgr.24625) 2114 : cluster [DBG] pgmap v1190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:10 gibba032 bash[24004]: cluster 2021-03-10T14:32:10.118532+0000 mgr.gibba032.whopeo (mgr.24625) 2115 : cluster [DBG] pgmap v1191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:10 gibba011 bash[30791]: cluster 2021-03-10T14:32:10.118532+0000 mgr.gibba032.whopeo (mgr.24625) 2115 : cluster [DBG] pgmap v1191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:10 gibba011 bash[31774]: cluster 2021-03-10T14:32:10.118532+0000 mgr.gibba032.whopeo (mgr.24625) 2115 : cluster [DBG] pgmap v1191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:12 gibba032 bash[24004]: cluster 2021-03-10T14:32:12.120190+0000 mgr.gibba032.whopeo (mgr.24625) 2116 : cluster [DBG] pgmap v1192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:12 gibba011 bash[30791]: cluster 2021-03-10T14:32:12.120190+0000 mgr.gibba032.whopeo (mgr.24625) 2116 : cluster [DBG] pgmap v1192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:12 gibba011 bash[31774]: cluster 2021-03-10T14:32:12.120190+0000 mgr.gibba032.whopeo (mgr.24625) 2116 : cluster [DBG] pgmap v1192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:14 gibba032 bash[24004]: cluster 2021-03-10T14:32:14.121408+0000 mgr.gibba032.whopeo (mgr.24625) 2117 : cluster [DBG] pgmap v1193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:14 gibba011 bash[30791]: cluster 2021-03-10T14:32:14.121408+0000 mgr.gibba032.whopeo (mgr.24625) 2117 : cluster [DBG] pgmap v1193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:14 gibba011 bash[31774]: cluster 2021-03-10T14:32:14.121408+0000 mgr.gibba032.whopeo (mgr.24625) 2117 : cluster [DBG] pgmap v1193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:16 gibba032 bash[24004]: cluster 2021-03-10T14:32:16.122985+0000 mgr.gibba032.whopeo (mgr.24625) 2118 : cluster [DBG] pgmap v1194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:16 gibba011 bash[30791]: cluster 2021-03-10T14:32:16.122985+0000 mgr.gibba032.whopeo (mgr.24625) 2118 : cluster [DBG] pgmap v1194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:16 gibba011 bash[31774]: cluster 2021-03-10T14:32:16.122985+0000 mgr.gibba032.whopeo (mgr.24625) 2118 : cluster [DBG] pgmap v1194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:18 gibba032 bash[24004]: cephadm 2021-03-10T14:32:17.562830+0000 mgr.gibba032.whopeo (mgr.24625) 2119 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:18 gibba032 bash[24004]: cluster 2021-03-10T14:32:18.124791+0000 mgr.gibba032.whopeo (mgr.24625) 2120 : cluster [DBG] pgmap v1195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:18 gibba011 bash[30791]: cephadm 2021-03-10T14:32:17.562830+0000 mgr.gibba032.whopeo (mgr.24625) 2119 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:18 gibba011 bash[30791]: cluster 2021-03-10T14:32:18.124791+0000 mgr.gibba032.whopeo (mgr.24625) 2120 : cluster [DBG] pgmap v1195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:18 gibba011 bash[31774]: cephadm 2021-03-10T14:32:17.562830+0000 mgr.gibba032.whopeo (mgr.24625) 2119 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:18 gibba011 bash[31774]: cluster 2021-03-10T14:32:18.124791+0000 mgr.gibba032.whopeo (mgr.24625) 2120 : cluster [DBG] pgmap v1195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:18.895 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:32:19.120 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:32:19.120 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (38m) 4m ago 44m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:32:19.121 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (44m) 3m ago 44m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:32:19.121 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (39m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:32:19.121 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (39m) 3m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:32:19.121 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (38m) 4m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:32:19.121 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (38m) 4m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:32:19.122 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (38m) 3m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:32:19.122 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (38m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:32:19.122 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (44m) 4m ago 44m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:32:19.122 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (44m) 3m ago 44m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:32:19.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (37m) 4m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:32:19.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (36m) 4m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:32:19.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (36m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:32:19.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (35m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:32:19.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (35m) 3m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:32:19.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (35m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:32:19.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (34m) 3m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:32:19.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (34m) 3m ago 44m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:32:19.124 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (43m) 3m ago 44m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:32:19.124 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 42m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:32:19.372 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:32:19.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:32:19.373 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:32:19.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:19.373 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:32:19.373 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:32:19.374 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:19.374 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:32:19.374 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:32:19.374 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:19.374 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:32:19.375 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:32:19.375 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:32:19.375 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:32:19.375 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:32:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:19 gibba032 bash[24004]: audit 2021-03-10T14:32:18.882377+0000 mgr.gibba032.whopeo (mgr.24625) 2121 : audit [DBG] from='client.55462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:19 gibba032 bash[24004]: audit 2021-03-10T14:32:19.104962+0000 mgr.gibba032.whopeo (mgr.24625) 2122 : audit [DBG] from='client.45483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:19.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:19 gibba011 bash[30791]: audit 2021-03-10T14:32:18.882377+0000 mgr.gibba032.whopeo (mgr.24625) 2121 : audit [DBG] from='client.55462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:19 gibba011 bash[30791]: audit 2021-03-10T14:32:19.104962+0000 mgr.gibba032.whopeo (mgr.24625) 2122 : audit [DBG] from='client.45483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:19.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:19 gibba011 bash[31774]: audit 2021-03-10T14:32:18.882377+0000 mgr.gibba032.whopeo (mgr.24625) 2121 : audit [DBG] from='client.55462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:19.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:19 gibba011 bash[31774]: audit 2021-03-10T14:32:19.104962+0000 mgr.gibba032.whopeo (mgr.24625) 2122 : audit [DBG] from='client.45483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:20 gibba032 bash[24004]: audit 2021-03-10T14:32:19.371931+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? 172.21.2.111:0/247546841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:20 gibba032 bash[24004]: cluster 2021-03-10T14:32:20.126304+0000 mgr.gibba032.whopeo (mgr.24625) 2123 : cluster [DBG] pgmap v1196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:20 gibba011 bash[30791]: audit 2021-03-10T14:32:19.371931+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? 172.21.2.111:0/247546841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:20 gibba011 bash[30791]: cluster 2021-03-10T14:32:20.126304+0000 mgr.gibba032.whopeo (mgr.24625) 2123 : cluster [DBG] pgmap v1196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:20 gibba011 bash[31774]: audit 2021-03-10T14:32:19.371931+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? 172.21.2.111:0/247546841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:20 gibba011 bash[31774]: cluster 2021-03-10T14:32:20.126304+0000 mgr.gibba032.whopeo (mgr.24625) 2123 : cluster [DBG] pgmap v1196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:22 gibba032 bash[24004]: cluster 2021-03-10T14:32:22.127802+0000 mgr.gibba032.whopeo (mgr.24625) 2124 : cluster [DBG] pgmap v1197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:22 gibba011 bash[30791]: cluster 2021-03-10T14:32:22.127802+0000 mgr.gibba032.whopeo (mgr.24625) 2124 : cluster [DBG] pgmap v1197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:22 gibba011 bash[31774]: cluster 2021-03-10T14:32:22.127802+0000 mgr.gibba032.whopeo (mgr.24625) 2124 : cluster [DBG] pgmap v1197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:24 gibba032 bash[24004]: cluster 2021-03-10T14:32:24.129238+0000 mgr.gibba032.whopeo (mgr.24625) 2125 : cluster [DBG] pgmap v1198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:24 gibba011 bash[30791]: cluster 2021-03-10T14:32:24.129238+0000 mgr.gibba032.whopeo (mgr.24625) 2125 : cluster [DBG] pgmap v1198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:24 gibba011 bash[31774]: cluster 2021-03-10T14:32:24.129238+0000 mgr.gibba032.whopeo (mgr.24625) 2125 : cluster [DBG] pgmap v1198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:27.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:26 gibba032 bash[24004]: cluster 2021-03-10T14:32:26.130250+0000 mgr.gibba032.whopeo (mgr.24625) 2126 : cluster [DBG] pgmap v1199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:26 gibba011 bash[30791]: cluster 2021-03-10T14:32:26.130250+0000 mgr.gibba032.whopeo (mgr.24625) 2126 : cluster [DBG] pgmap v1199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:26 gibba011 bash[31774]: cluster 2021-03-10T14:32:26.130250+0000 mgr.gibba032.whopeo (mgr.24625) 2126 : cluster [DBG] pgmap v1199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:28 gibba032 bash[24004]: cluster 2021-03-10T14:32:28.131898+0000 mgr.gibba032.whopeo (mgr.24625) 2127 : cluster [DBG] pgmap v1200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:28 gibba011 bash[30791]: cluster 2021-03-10T14:32:28.131898+0000 mgr.gibba032.whopeo (mgr.24625) 2127 : cluster [DBG] pgmap v1200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:28 gibba011 bash[31774]: cluster 2021-03-10T14:32:28.131898+0000 mgr.gibba032.whopeo (mgr.24625) 2127 : cluster [DBG] pgmap v1200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:30 gibba032 bash[24004]: cluster 2021-03-10T14:32:30.133442+0000 mgr.gibba032.whopeo (mgr.24625) 2128 : cluster [DBG] pgmap v1201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:30 gibba011 bash[30791]: cluster 2021-03-10T14:32:30.133442+0000 mgr.gibba032.whopeo (mgr.24625) 2128 : cluster [DBG] pgmap v1201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:30 gibba011 bash[31774]: cluster 2021-03-10T14:32:30.133442+0000 mgr.gibba032.whopeo (mgr.24625) 2128 : cluster [DBG] pgmap v1201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:32 gibba032 bash[24004]: cluster 2021-03-10T14:32:32.134919+0000 mgr.gibba032.whopeo (mgr.24625) 2129 : cluster [DBG] pgmap v1202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:32 gibba011 bash[30791]: cluster 2021-03-10T14:32:32.134919+0000 mgr.gibba032.whopeo (mgr.24625) 2129 : cluster [DBG] pgmap v1202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:32 gibba011 bash[31774]: cluster 2021-03-10T14:32:32.134919+0000 mgr.gibba032.whopeo (mgr.24625) 2129 : cluster [DBG] pgmap v1202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:33.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:33 gibba032 bash[24004]: cephadm 2021-03-10T14:32:32.578945+0000 mgr.gibba032.whopeo (mgr.24625) 2130 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:33.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:33 gibba011 bash[30791]: cephadm 2021-03-10T14:32:32.578945+0000 mgr.gibba032.whopeo (mgr.24625) 2130 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:33.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:33 gibba011 bash[31774]: cephadm 2021-03-10T14:32:32.578945+0000 mgr.gibba032.whopeo (mgr.24625) 2130 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:34 gibba032 bash[24004]: cluster 2021-03-10T14:32:34.136349+0000 mgr.gibba032.whopeo (mgr.24625) 2131 : cluster [DBG] pgmap v1203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:34 gibba011 bash[30791]: cluster 2021-03-10T14:32:34.136349+0000 mgr.gibba032.whopeo (mgr.24625) 2131 : cluster [DBG] pgmap v1203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:34 gibba011 bash[31774]: cluster 2021-03-10T14:32:34.136349+0000 mgr.gibba032.whopeo (mgr.24625) 2131 : cluster [DBG] pgmap v1203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:36 gibba032 bash[24004]: cluster 2021-03-10T14:32:36.137786+0000 mgr.gibba032.whopeo (mgr.24625) 2132 : cluster [DBG] pgmap v1204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:37 gibba011 bash[30791]: cluster 2021-03-10T14:32:36.137786+0000 mgr.gibba032.whopeo (mgr.24625) 2132 : cluster [DBG] pgmap v1204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:37 gibba011 bash[31774]: cluster 2021-03-10T14:32:36.137786+0000 mgr.gibba032.whopeo (mgr.24625) 2132 : cluster [DBG] pgmap v1204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:38 gibba032 bash[24004]: cluster 2021-03-10T14:32:38.139404+0000 mgr.gibba032.whopeo (mgr.24625) 2133 : cluster [DBG] pgmap v1205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:38 gibba011 bash[30791]: cluster 2021-03-10T14:32:38.139404+0000 mgr.gibba032.whopeo (mgr.24625) 2133 : cluster [DBG] pgmap v1205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:38 gibba011 bash[31774]: cluster 2021-03-10T14:32:38.139404+0000 mgr.gibba032.whopeo (mgr.24625) 2133 : cluster [DBG] pgmap v1205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:40 gibba032 bash[24004]: cluster 2021-03-10T14:32:40.140941+0000 mgr.gibba032.whopeo (mgr.24625) 2134 : cluster [DBG] pgmap v1206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:40 gibba011 bash[30791]: cluster 2021-03-10T14:32:40.140941+0000 mgr.gibba032.whopeo (mgr.24625) 2134 : cluster [DBG] pgmap v1206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:40 gibba011 bash[31774]: cluster 2021-03-10T14:32:40.140941+0000 mgr.gibba032.whopeo (mgr.24625) 2134 : cluster [DBG] pgmap v1206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:42 gibba032 bash[24004]: cluster 2021-03-10T14:32:42.142672+0000 mgr.gibba032.whopeo (mgr.24625) 2135 : cluster [DBG] pgmap v1207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:42 gibba011 bash[30791]: cluster 2021-03-10T14:32:42.142672+0000 mgr.gibba032.whopeo (mgr.24625) 2135 : cluster [DBG] pgmap v1207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:42 gibba011 bash[31774]: cluster 2021-03-10T14:32:42.142672+0000 mgr.gibba032.whopeo (mgr.24625) 2135 : cluster [DBG] pgmap v1207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:44 gibba032 bash[24004]: cluster 2021-03-10T14:32:44.144217+0000 mgr.gibba032.whopeo (mgr.24625) 2136 : cluster [DBG] pgmap v1208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:44 gibba011 bash[30791]: cluster 2021-03-10T14:32:44.144217+0000 mgr.gibba032.whopeo (mgr.24625) 2136 : cluster [DBG] pgmap v1208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:44 gibba011 bash[31774]: cluster 2021-03-10T14:32:44.144217+0000 mgr.gibba032.whopeo (mgr.24625) 2136 : cluster [DBG] pgmap v1208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:47.006 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:47 gibba011 bash[30791]: cluster 2021-03-10T14:32:46.145735+0000 mgr.gibba032.whopeo (mgr.24625) 2137 : cluster [DBG] pgmap v1209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:47 gibba032 bash[24004]: cluster 2021-03-10T14:32:46.145735+0000 mgr.gibba032.whopeo (mgr.24625) 2137 : cluster [DBG] pgmap v1209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:47 gibba011 bash[31774]: cluster 2021-03-10T14:32:46.145735+0000 mgr.gibba032.whopeo (mgr.24625) 2137 : cluster [DBG] pgmap v1209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.595708+0000 mgr.gibba032.whopeo (mgr.24625) 2138 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:32:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.597254+0000 mgr.gibba032.whopeo (mgr.24625) 2139 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:32:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.821746+0000 mgr.gibba032.whopeo (mgr.24625) 2140 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:32:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.822162+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:32:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.823587+0000 mgr.gibba032.whopeo (mgr.24625) 2141 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:32:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.825098+0000 mgr.gibba032.whopeo (mgr.24625) 2142 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:32:49.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.825134+0000 mgr.gibba032.whopeo (mgr.24625) 2143 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:32:49.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.825464+0000 mon.a (mon.0) 1168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:32:49.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.826499+0000 mgr.gibba032.whopeo (mgr.24625) 2144 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:32:49.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.826537+0000 mgr.gibba032.whopeo (mgr.24625) 2145 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:32:49.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.826932+0000 mon.a (mon.0) 1169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.827514+0000 mgr.gibba032.whopeo (mgr.24625) 2146 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:32:49.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.828506+0000 mgr.gibba032.whopeo (mgr.24625) 2147 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:32:49.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.828541+0000 mgr.gibba032.whopeo (mgr.24625) 2148 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:32:49.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.828576+0000 mon.a (mon.0) 1170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.829784+0000 mon.a (mon.0) 1171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.829958+0000 mgr.gibba032.whopeo (mgr.24625) 2149 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:32:49.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.830269+0000 mgr.gibba032.whopeo (mgr.24625) 2150 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:32:49.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.831341+0000 mgr.gibba032.whopeo (mgr.24625) 2151 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:32:49.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.831380+0000 mgr.gibba032.whopeo (mgr.24625) 2152 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:32:49.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cephadm 2021-03-10T14:32:47.831482+0000 mgr.gibba032.whopeo (mgr.24625) 2153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:49.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.595708+0000 mgr.gibba032.whopeo (mgr.24625) 2138 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:32:49.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.597254+0000 mgr.gibba032.whopeo (mgr.24625) 2139 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:32:49.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.821746+0000 mgr.gibba032.whopeo (mgr.24625) 2140 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:32:49.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.822162+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:32:49.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.823587+0000 mgr.gibba032.whopeo (mgr.24625) 2141 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:32:49.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.825098+0000 mgr.gibba032.whopeo (mgr.24625) 2142 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:32:49.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.825134+0000 mgr.gibba032.whopeo (mgr.24625) 2143 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:32:49.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.825464+0000 mon.a (mon.0) 1168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:32:49.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.826499+0000 mgr.gibba032.whopeo (mgr.24625) 2144 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:32:49.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.826537+0000 mgr.gibba032.whopeo (mgr.24625) 2145 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:32:49.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.826932+0000 mon.a (mon.0) 1169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.827514+0000 mgr.gibba032.whopeo (mgr.24625) 2146 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:32:49.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.828506+0000 mgr.gibba032.whopeo (mgr.24625) 2147 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:32:49.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.828541+0000 mgr.gibba032.whopeo (mgr.24625) 2148 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:32:49.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.828576+0000 mon.a (mon.0) 1170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.829784+0000 mon.a (mon.0) 1171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.829958+0000 mgr.gibba032.whopeo (mgr.24625) 2149 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:32:49.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.830269+0000 mgr.gibba032.whopeo (mgr.24625) 2150 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:32:49.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.831341+0000 mgr.gibba032.whopeo (mgr.24625) 2151 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:32:49.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.831380+0000 mgr.gibba032.whopeo (mgr.24625) 2152 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:32:49.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cephadm 2021-03-10T14:32:47.831482+0000 mgr.gibba032.whopeo (mgr.24625) 2153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:49.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.831990+0000 mon.a (mon.0) 1172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: audit 2021-03-10T14:32:47.833579+0000 mon.a (mon.0) 1173 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[31774]: cluster 2021-03-10T14:32:48.147516+0000 mgr.gibba032.whopeo (mgr.24625) 2154 : cluster [DBG] pgmap v1210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:49.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.831990+0000 mon.a (mon.0) 1172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: audit 2021-03-10T14:32:47.833579+0000 mon.a (mon.0) 1173 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:48 gibba011 bash[30791]: cluster 2021-03-10T14:32:48.147516+0000 mgr.gibba032.whopeo (mgr.24625) 2154 : cluster [DBG] pgmap v1210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.595708+0000 mgr.gibba032.whopeo (mgr.24625) 2138 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:32:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.597254+0000 mgr.gibba032.whopeo (mgr.24625) 2139 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:32:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.821746+0000 mgr.gibba032.whopeo (mgr.24625) 2140 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:32:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.822162+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:32:49.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.823587+0000 mgr.gibba032.whopeo (mgr.24625) 2141 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:32:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.825098+0000 mgr.gibba032.whopeo (mgr.24625) 2142 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:32:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.825134+0000 mgr.gibba032.whopeo (mgr.24625) 2143 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:32:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.825464+0000 mon.a (mon.0) 1168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:32:49.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.826499+0000 mgr.gibba032.whopeo (mgr.24625) 2144 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:32:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.826537+0000 mgr.gibba032.whopeo (mgr.24625) 2145 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:32:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.826932+0000 mon.a (mon.0) 1169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.827514+0000 mgr.gibba032.whopeo (mgr.24625) 2146 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:32:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.828506+0000 mgr.gibba032.whopeo (mgr.24625) 2147 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:32:49.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.828541+0000 mgr.gibba032.whopeo (mgr.24625) 2148 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:32:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.828576+0000 mon.a (mon.0) 1170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.829784+0000 mon.a (mon.0) 1171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.829958+0000 mgr.gibba032.whopeo (mgr.24625) 2149 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:32:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.830269+0000 mgr.gibba032.whopeo (mgr.24625) 2150 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:32:49.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.831341+0000 mgr.gibba032.whopeo (mgr.24625) 2151 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:32:49.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.831380+0000 mgr.gibba032.whopeo (mgr.24625) 2152 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:32:49.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cephadm 2021-03-10T14:32:47.831482+0000 mgr.gibba032.whopeo (mgr.24625) 2153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:32:49.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.831990+0000 mon.a (mon.0) 1172 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: audit 2021-03-10T14:32:47.833579+0000 mon.a (mon.0) 1173 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:49.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:48 gibba032 bash[24004]: cluster 2021-03-10T14:32:48.147516+0000 mgr.gibba032.whopeo (mgr.24625) 2154 : cluster [DBG] pgmap v1210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:49.618 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:32:49.845 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:32:49.846 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (39m) 5m ago 44m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:32:49.846 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (44m) 4m ago 44m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:32:49.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (39m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:32:49.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (40m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:32:49.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (39m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:32:49.847 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (39m) 5m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:32:49.847 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (38m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:32:49.847 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (38m) 5m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:32:49.847 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (44m) 5m ago 45m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:32:49.847 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (44m) 4m ago 44m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:32:49.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (37m) 5m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:32:49.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (37m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:32:49.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (36m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:32:49.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (36m) 5m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:32:49.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (36m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:32:49.849 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (35m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:32:49.849 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (35m) 4m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:32:49.849 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (34m) 4m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:32:49.849 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (44m) 4m ago 45m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:32:49.849 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 42m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:32:50.095 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:32:50.096 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:32:50.096 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:32:50.096 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:50.096 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:32:50.097 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:32:50.097 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:50.097 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:32:50.097 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:32:50.098 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:32:50.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:32:50.098 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:32:50.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:32:50.099 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:32:50.099 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:32:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:50 gibba032 bash[24004]: audit 2021-03-10T14:32:50.095202+0000 mon.c (mon.1) 44 : audit [DBG] from='client.? 172.21.2.111:0/2835411383' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:50 gibba011 bash[30791]: audit 2021-03-10T14:32:50.095202+0000 mon.c (mon.1) 44 : audit [DBG] from='client.? 172.21.2.111:0/2835411383' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:50 gibba011 bash[31774]: audit 2021-03-10T14:32:50.095202+0000 mon.c (mon.1) 44 : audit [DBG] from='client.? 172.21.2.111:0/2835411383' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:32:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:51 gibba032 bash[24004]: audit 2021-03-10T14:32:49.605900+0000 mgr.gibba032.whopeo (mgr.24625) 2155 : audit [DBG] from='client.45495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:51 gibba032 bash[24004]: audit 2021-03-10T14:32:49.829449+0000 mgr.gibba032.whopeo (mgr.24625) 2156 : audit [DBG] from='client.55486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:51 gibba032 bash[24004]: cluster 2021-03-10T14:32:50.148841+0000 mgr.gibba032.whopeo (mgr.24625) 2157 : cluster [DBG] pgmap v1211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:51.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[30791]: audit 2021-03-10T14:32:49.605900+0000 mgr.gibba032.whopeo (mgr.24625) 2155 : audit [DBG] from='client.45495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[30791]: audit 2021-03-10T14:32:49.829449+0000 mgr.gibba032.whopeo (mgr.24625) 2156 : audit [DBG] from='client.55486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[30791]: cluster 2021-03-10T14:32:50.148841+0000 mgr.gibba032.whopeo (mgr.24625) 2157 : cluster [DBG] pgmap v1211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[31774]: audit 2021-03-10T14:32:49.605900+0000 mgr.gibba032.whopeo (mgr.24625) 2155 : audit [DBG] from='client.45495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[31774]: audit 2021-03-10T14:32:49.829449+0000 mgr.gibba032.whopeo (mgr.24625) 2156 : audit [DBG] from='client.55486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:32:51.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:51 gibba011 bash[31774]: cluster 2021-03-10T14:32:50.148841+0000 mgr.gibba032.whopeo (mgr.24625) 2157 : cluster [DBG] pgmap v1211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:52 gibba032 bash[24004]: cluster 2021-03-10T14:32:52.150560+0000 mgr.gibba032.whopeo (mgr.24625) 2158 : cluster [DBG] pgmap v1212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:52 gibba011 bash[30791]: cluster 2021-03-10T14:32:52.150560+0000 mgr.gibba032.whopeo (mgr.24625) 2158 : cluster [DBG] pgmap v1212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:52 gibba011 bash[31774]: cluster 2021-03-10T14:32:52.150560+0000 mgr.gibba032.whopeo (mgr.24625) 2158 : cluster [DBG] pgmap v1212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:54 gibba032 bash[24004]: cluster 2021-03-10T14:32:54.152058+0000 mgr.gibba032.whopeo (mgr.24625) 2159 : cluster [DBG] pgmap v1213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:54 gibba011 bash[30791]: cluster 2021-03-10T14:32:54.152058+0000 mgr.gibba032.whopeo (mgr.24625) 2159 : cluster [DBG] pgmap v1213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:54 gibba011 bash[31774]: cluster 2021-03-10T14:32:54.152058+0000 mgr.gibba032.whopeo (mgr.24625) 2159 : cluster [DBG] pgmap v1213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: cluster 2021-03-10T14:32:56.153519+0000 mgr.gibba032.whopeo (mgr.24625) 2160 : cluster [DBG] pgmap v1214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.259515+0000 mon.a (mon.0) 1174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:32:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.260185+0000 mon.a (mon.0) 1175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:32:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.270901+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:32:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.271759+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:32:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.272480+0000 mon.a (mon.0) 1178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:32:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[30791]: audit 2021-03-10T14:32:56.273183+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:32:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: cluster 2021-03-10T14:32:56.153519+0000 mgr.gibba032.whopeo (mgr.24625) 2160 : cluster [DBG] pgmap v1214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.259515+0000 mon.a (mon.0) 1174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:32:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.260185+0000 mon.a (mon.0) 1175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:32:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.270901+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:32:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.271759+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:32:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.272480+0000 mon.a (mon.0) 1178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:32:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:57 gibba011 bash[31774]: audit 2021-03-10T14:32:56.273183+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:32:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: cluster 2021-03-10T14:32:56.153519+0000 mgr.gibba032.whopeo (mgr.24625) 2160 : cluster [DBG] pgmap v1214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.259515+0000 mon.a (mon.0) 1174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.260185+0000 mon.a (mon.0) 1175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.270901+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.271759+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.272480+0000 mon.a (mon.0) 1178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:57 gibba032 bash[24004]: audit 2021-03-10T14:32:56.273183+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:32:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:32:58 gibba032 bash[24004]: cluster 2021-03-10T14:32:58.155239+0000 mgr.gibba032.whopeo (mgr.24625) 2161 : cluster [DBG] pgmap v1215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:32:58 gibba011 bash[30791]: cluster 2021-03-10T14:32:58.155239+0000 mgr.gibba032.whopeo (mgr.24625) 2161 : cluster [DBG] pgmap v1215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:32:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:32:58 gibba011 bash[31774]: cluster 2021-03-10T14:32:58.155239+0000 mgr.gibba032.whopeo (mgr.24625) 2161 : cluster [DBG] pgmap v1215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:00 gibba032 bash[24004]: cluster 2021-03-10T14:33:00.155988+0000 mgr.gibba032.whopeo (mgr.24625) 2162 : cluster [DBG] pgmap v1216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:00 gibba011 bash[30791]: cluster 2021-03-10T14:33:00.155988+0000 mgr.gibba032.whopeo (mgr.24625) 2162 : cluster [DBG] pgmap v1216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:00 gibba011 bash[31774]: cluster 2021-03-10T14:33:00.155988+0000 mgr.gibba032.whopeo (mgr.24625) 2162 : cluster [DBG] pgmap v1216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:02 gibba032 bash[24004]: cluster 2021-03-10T14:33:02.157620+0000 mgr.gibba032.whopeo (mgr.24625) 2163 : cluster [DBG] pgmap v1217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:02 gibba011 bash[30791]: cluster 2021-03-10T14:33:02.157620+0000 mgr.gibba032.whopeo (mgr.24625) 2163 : cluster [DBG] pgmap v1217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:02 gibba011 bash[31774]: cluster 2021-03-10T14:33:02.157620+0000 mgr.gibba032.whopeo (mgr.24625) 2163 : cluster [DBG] pgmap v1217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:03 gibba032 bash[24004]: cephadm 2021-03-10T14:33:02.844937+0000 mgr.gibba032.whopeo (mgr.24625) 2164 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:03 gibba011 bash[30791]: cephadm 2021-03-10T14:33:02.844937+0000 mgr.gibba032.whopeo (mgr.24625) 2164 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:03.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:03 gibba011 bash[31774]: cephadm 2021-03-10T14:33:02.844937+0000 mgr.gibba032.whopeo (mgr.24625) 2164 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:04 gibba032 bash[24004]: cluster 2021-03-10T14:33:04.158827+0000 mgr.gibba032.whopeo (mgr.24625) 2165 : cluster [DBG] pgmap v1218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:04 gibba011 bash[30791]: cluster 2021-03-10T14:33:04.158827+0000 mgr.gibba032.whopeo (mgr.24625) 2165 : cluster [DBG] pgmap v1218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:04 gibba011 bash[31774]: cluster 2021-03-10T14:33:04.158827+0000 mgr.gibba032.whopeo (mgr.24625) 2165 : cluster [DBG] pgmap v1218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:07 gibba011 bash[30791]: cluster 2021-03-10T14:33:06.160327+0000 mgr.gibba032.whopeo (mgr.24625) 2166 : cluster [DBG] pgmap v1219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:07 gibba011 bash[31774]: cluster 2021-03-10T14:33:06.160327+0000 mgr.gibba032.whopeo (mgr.24625) 2166 : cluster [DBG] pgmap v1219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:07 gibba032 bash[24004]: cluster 2021-03-10T14:33:06.160327+0000 mgr.gibba032.whopeo (mgr.24625) 2166 : cluster [DBG] pgmap v1219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:08 gibba032 bash[24004]: cluster 2021-03-10T14:33:08.162061+0000 mgr.gibba032.whopeo (mgr.24625) 2167 : cluster [DBG] pgmap v1220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:08 gibba011 bash[30791]: cluster 2021-03-10T14:33:08.162061+0000 mgr.gibba032.whopeo (mgr.24625) 2167 : cluster [DBG] pgmap v1220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:08 gibba011 bash[31774]: cluster 2021-03-10T14:33:08.162061+0000 mgr.gibba032.whopeo (mgr.24625) 2167 : cluster [DBG] pgmap v1220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:10 gibba032 bash[24004]: cluster 2021-03-10T14:33:10.163647+0000 mgr.gibba032.whopeo (mgr.24625) 2168 : cluster [DBG] pgmap v1221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:10 gibba011 bash[30791]: cluster 2021-03-10T14:33:10.163647+0000 mgr.gibba032.whopeo (mgr.24625) 2168 : cluster [DBG] pgmap v1221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:10 gibba011 bash[31774]: cluster 2021-03-10T14:33:10.163647+0000 mgr.gibba032.whopeo (mgr.24625) 2168 : cluster [DBG] pgmap v1221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:12 gibba032 bash[24004]: cluster 2021-03-10T14:33:12.165298+0000 mgr.gibba032.whopeo (mgr.24625) 2169 : cluster [DBG] pgmap v1222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:12 gibba011 bash[30791]: cluster 2021-03-10T14:33:12.165298+0000 mgr.gibba032.whopeo (mgr.24625) 2169 : cluster [DBG] pgmap v1222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:12 gibba011 bash[31774]: cluster 2021-03-10T14:33:12.165298+0000 mgr.gibba032.whopeo (mgr.24625) 2169 : cluster [DBG] pgmap v1222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:15 gibba032 bash[24004]: cluster 2021-03-10T14:33:14.166787+0000 mgr.gibba032.whopeo (mgr.24625) 2170 : cluster [DBG] pgmap v1223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:15.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:15 gibba011 bash[30791]: cluster 2021-03-10T14:33:14.166787+0000 mgr.gibba032.whopeo (mgr.24625) 2170 : cluster [DBG] pgmap v1223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:15.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:15 gibba011 bash[31774]: cluster 2021-03-10T14:33:14.166787+0000 mgr.gibba032.whopeo (mgr.24625) 2170 : cluster [DBG] pgmap v1223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:18 gibba032 bash[24004]: cluster 2021-03-10T14:33:16.168299+0000 mgr.gibba032.whopeo (mgr.24625) 2171 : cluster [DBG] pgmap v1224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:18 gibba011 bash[30791]: cluster 2021-03-10T14:33:16.168299+0000 mgr.gibba032.whopeo (mgr.24625) 2171 : cluster [DBG] pgmap v1224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:18 gibba011 bash[31774]: cluster 2021-03-10T14:33:16.168299+0000 mgr.gibba032.whopeo (mgr.24625) 2171 : cluster [DBG] pgmap v1224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:19 gibba011 bash[30791]: cephadm 2021-03-10T14:33:17.860961+0000 mgr.gibba032.whopeo (mgr.24625) 2172 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:19 gibba011 bash[31774]: cephadm 2021-03-10T14:33:17.860961+0000 mgr.gibba032.whopeo (mgr.24625) 2172 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:19 gibba032 bash[24004]: cephadm 2021-03-10T14:33:17.860961+0000 mgr.gibba032.whopeo (mgr.24625) 2172 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:20 gibba011 bash[30791]: cluster 2021-03-10T14:33:18.169799+0000 mgr.gibba032.whopeo (mgr.24625) 2173 : cluster [DBG] pgmap v1225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:20 gibba011 bash[31774]: cluster 2021-03-10T14:33:18.169799+0000 mgr.gibba032.whopeo (mgr.24625) 2173 : cluster [DBG] pgmap v1225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:20.339 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:33:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:20 gibba032 bash[24004]: cluster 2021-03-10T14:33:18.169799+0000 mgr.gibba032.whopeo (mgr.24625) 2173 : cluster [DBG] pgmap v1225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:20.569 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:33:20.570 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (39m) 5m ago 45m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:33:20.570 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (45m) 4m ago 45m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:33:20.570 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (40m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:33:20.571 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (40m) 4m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:33:20.571 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (39m) 5m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:33:20.571 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (39m) 5m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:33:20.571 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (39m) 4m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:33:20.572 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (39m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:33:20.572 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (45m) 5m ago 45m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:33:20.572 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (45m) 4m ago 45m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:33:20.572 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (38m) 5m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:33:20.572 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (37m) 5m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:33:20.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (37m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:33:20.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (36m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:33:20.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (36m) 4m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:33:20.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (36m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:33:20.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (35m) 4m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:33:20.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (35m) 4m ago 45m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:33:20.574 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (44m) 4m ago 45m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:33:20.574 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 43m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:33:20.818 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:33:20.819 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:33:20.819 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:33:20.819 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:20.819 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:33:20.819 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:33:20.820 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:20.820 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:33:20.820 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:33:20.820 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:20.820 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:33:20.821 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:33:20.821 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:33:20.821 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:33:20.821 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:33:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:21 gibba011 bash[30791]: audit 2021-03-10T14:33:20.818143+0000 mon.a (mon.0) 1180 : audit [DBG] from='client.? 172.21.2.111:0/2949768112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:21 gibba011 bash[31774]: audit 2021-03-10T14:33:20.818143+0000 mon.a (mon.0) 1180 : audit [DBG] from='client.? 172.21.2.111:0/2949768112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:21 gibba032 bash[24004]: audit 2021-03-10T14:33:20.818143+0000 mon.a (mon.0) 1180 : audit [DBG] from='client.? 172.21.2.111:0/2949768112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[30791]: cluster 2021-03-10T14:33:20.171084+0000 mgr.gibba032.whopeo (mgr.24625) 2174 : cluster [DBG] pgmap v1226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[30791]: audit 2021-03-10T14:33:20.328431+0000 mgr.gibba032.whopeo (mgr.24625) 2175 : audit [DBG] from='client.55498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[30791]: audit 2021-03-10T14:33:20.553791+0000 mgr.gibba032.whopeo (mgr.24625) 2176 : audit [DBG] from='client.55504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[31774]: cluster 2021-03-10T14:33:20.171084+0000 mgr.gibba032.whopeo (mgr.24625) 2174 : cluster [DBG] pgmap v1226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[31774]: audit 2021-03-10T14:33:20.328431+0000 mgr.gibba032.whopeo (mgr.24625) 2175 : audit [DBG] from='client.55498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:22 gibba011 bash[31774]: audit 2021-03-10T14:33:20.553791+0000 mgr.gibba032.whopeo (mgr.24625) 2176 : audit [DBG] from='client.55504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:22 gibba032 bash[24004]: cluster 2021-03-10T14:33:20.171084+0000 mgr.gibba032.whopeo (mgr.24625) 2174 : cluster [DBG] pgmap v1226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:22 gibba032 bash[24004]: audit 2021-03-10T14:33:20.328431+0000 mgr.gibba032.whopeo (mgr.24625) 2175 : audit [DBG] from='client.55498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:22 gibba032 bash[24004]: audit 2021-03-10T14:33:20.553791+0000 mgr.gibba032.whopeo (mgr.24625) 2176 : audit [DBG] from='client.55504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:24 gibba011 bash[30791]: cluster 2021-03-10T14:33:22.172763+0000 mgr.gibba032.whopeo (mgr.24625) 2177 : cluster [DBG] pgmap v1227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:24.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:24 gibba011 bash[31774]: cluster 2021-03-10T14:33:22.172763+0000 mgr.gibba032.whopeo (mgr.24625) 2177 : cluster [DBG] pgmap v1227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:24 gibba032 bash[24004]: cluster 2021-03-10T14:33:22.172763+0000 mgr.gibba032.whopeo (mgr.24625) 2177 : cluster [DBG] pgmap v1227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:26 gibba011 bash[30791]: cluster 2021-03-10T14:33:24.174205+0000 mgr.gibba032.whopeo (mgr.24625) 2178 : cluster [DBG] pgmap v1228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:26 gibba011 bash[31774]: cluster 2021-03-10T14:33:24.174205+0000 mgr.gibba032.whopeo (mgr.24625) 2178 : cluster [DBG] pgmap v1228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:26 gibba032 bash[24004]: cluster 2021-03-10T14:33:24.174205+0000 mgr.gibba032.whopeo (mgr.24625) 2178 : cluster [DBG] pgmap v1228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:28 gibba011 bash[30791]: cluster 2021-03-10T14:33:26.175650+0000 mgr.gibba032.whopeo (mgr.24625) 2179 : cluster [DBG] pgmap v1229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:28 gibba011 bash[31774]: cluster 2021-03-10T14:33:26.175650+0000 mgr.gibba032.whopeo (mgr.24625) 2179 : cluster [DBG] pgmap v1229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:28.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:28 gibba032 bash[24004]: cluster 2021-03-10T14:33:26.175650+0000 mgr.gibba032.whopeo (mgr.24625) 2179 : cluster [DBG] pgmap v1229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:30 gibba011 bash[30791]: cluster 2021-03-10T14:33:28.177354+0000 mgr.gibba032.whopeo (mgr.24625) 2180 : cluster [DBG] pgmap v1230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:30.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:30 gibba011 bash[31774]: cluster 2021-03-10T14:33:28.177354+0000 mgr.gibba032.whopeo (mgr.24625) 2180 : cluster [DBG] pgmap v1230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:30 gibba032 bash[24004]: cluster 2021-03-10T14:33:28.177354+0000 mgr.gibba032.whopeo (mgr.24625) 2180 : cluster [DBG] pgmap v1230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:32 gibba032 bash[24004]: cluster 2021-03-10T14:33:30.178881+0000 mgr.gibba032.whopeo (mgr.24625) 2181 : cluster [DBG] pgmap v1231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:32 gibba011 bash[30791]: cluster 2021-03-10T14:33:30.178881+0000 mgr.gibba032.whopeo (mgr.24625) 2181 : cluster [DBG] pgmap v1231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:32 gibba011 bash[31774]: cluster 2021-03-10T14:33:30.178881+0000 mgr.gibba032.whopeo (mgr.24625) 2181 : cluster [DBG] pgmap v1231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:34 gibba032 bash[24004]: cluster 2021-03-10T14:33:32.180658+0000 mgr.gibba032.whopeo (mgr.24625) 2182 : cluster [DBG] pgmap v1232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:34 gibba032 bash[24004]: cephadm 2021-03-10T14:33:32.876909+0000 mgr.gibba032.whopeo (mgr.24625) 2183 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:34 gibba011 bash[30791]: cluster 2021-03-10T14:33:32.180658+0000 mgr.gibba032.whopeo (mgr.24625) 2182 : cluster [DBG] pgmap v1232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:34 gibba011 bash[30791]: cephadm 2021-03-10T14:33:32.876909+0000 mgr.gibba032.whopeo (mgr.24625) 2183 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:34 gibba011 bash[31774]: cluster 2021-03-10T14:33:32.180658+0000 mgr.gibba032.whopeo (mgr.24625) 2182 : cluster [DBG] pgmap v1232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:34 gibba011 bash[31774]: cephadm 2021-03-10T14:33:32.876909+0000 mgr.gibba032.whopeo (mgr.24625) 2183 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:36 gibba032 bash[24004]: cluster 2021-03-10T14:33:34.181766+0000 mgr.gibba032.whopeo (mgr.24625) 2184 : cluster [DBG] pgmap v1233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:36 gibba011 bash[30791]: cluster 2021-03-10T14:33:34.181766+0000 mgr.gibba032.whopeo (mgr.24625) 2184 : cluster [DBG] pgmap v1233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:36 gibba011 bash[31774]: cluster 2021-03-10T14:33:34.181766+0000 mgr.gibba032.whopeo (mgr.24625) 2184 : cluster [DBG] pgmap v1233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:38 gibba032 bash[24004]: cluster 2021-03-10T14:33:36.183257+0000 mgr.gibba032.whopeo (mgr.24625) 2185 : cluster [DBG] pgmap v1234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:38 gibba011 bash[30791]: cluster 2021-03-10T14:33:36.183257+0000 mgr.gibba032.whopeo (mgr.24625) 2185 : cluster [DBG] pgmap v1234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:38 gibba011 bash[31774]: cluster 2021-03-10T14:33:36.183257+0000 mgr.gibba032.whopeo (mgr.24625) 2185 : cluster [DBG] pgmap v1234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:40 gibba032 bash[24004]: cluster 2021-03-10T14:33:38.184259+0000 mgr.gibba032.whopeo (mgr.24625) 2186 : cluster [DBG] pgmap v1235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:40 gibba011 bash[30791]: cluster 2021-03-10T14:33:38.184259+0000 mgr.gibba032.whopeo (mgr.24625) 2186 : cluster [DBG] pgmap v1235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:40 gibba011 bash[31774]: cluster 2021-03-10T14:33:38.184259+0000 mgr.gibba032.whopeo (mgr.24625) 2186 : cluster [DBG] pgmap v1235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:42 gibba032 bash[24004]: cluster 2021-03-10T14:33:40.185545+0000 mgr.gibba032.whopeo (mgr.24625) 2187 : cluster [DBG] pgmap v1236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:42 gibba011 bash[30791]: cluster 2021-03-10T14:33:40.185545+0000 mgr.gibba032.whopeo (mgr.24625) 2187 : cluster [DBG] pgmap v1236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:42 gibba011 bash[31774]: cluster 2021-03-10T14:33:40.185545+0000 mgr.gibba032.whopeo (mgr.24625) 2187 : cluster [DBG] pgmap v1236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:44 gibba032 bash[24004]: cluster 2021-03-10T14:33:42.186616+0000 mgr.gibba032.whopeo (mgr.24625) 2188 : cluster [DBG] pgmap v1237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:44 gibba011 bash[30791]: cluster 2021-03-10T14:33:42.186616+0000 mgr.gibba032.whopeo (mgr.24625) 2188 : cluster [DBG] pgmap v1237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:44 gibba011 bash[31774]: cluster 2021-03-10T14:33:42.186616+0000 mgr.gibba032.whopeo (mgr.24625) 2188 : cluster [DBG] pgmap v1237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:46 gibba032 bash[24004]: cluster 2021-03-10T14:33:44.187233+0000 mgr.gibba032.whopeo (mgr.24625) 2189 : cluster [DBG] pgmap v1238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:46 gibba011 bash[30791]: cluster 2021-03-10T14:33:44.187233+0000 mgr.gibba032.whopeo (mgr.24625) 2189 : cluster [DBG] pgmap v1238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:46 gibba011 bash[31774]: cluster 2021-03-10T14:33:44.187233+0000 mgr.gibba032.whopeo (mgr.24625) 2189 : cluster [DBG] pgmap v1238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: cluster 2021-03-10T14:33:46.187854+0000 mgr.gibba032.whopeo (mgr.24625) 2190 : cluster [DBG] pgmap v1239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.135378+0000 mon.a (mon.0) 1181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:33:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.138715+0000 mon.a (mon.0) 1182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:33:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.140360+0000 mon.a (mon.0) 1183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.141904+0000 mon.a (mon.0) 1184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.143468+0000 mon.a (mon.0) 1185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.147072+0000 mon.a (mon.0) 1186 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:48 gibba032 bash[24004]: audit 2021-03-10T14:33:48.148871+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: cluster 2021-03-10T14:33:46.187854+0000 mgr.gibba032.whopeo (mgr.24625) 2190 : cluster [DBG] pgmap v1239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.135378+0000 mon.a (mon.0) 1181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.138715+0000 mon.a (mon.0) 1182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.140360+0000 mon.a (mon.0) 1183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.141904+0000 mon.a (mon.0) 1184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.143468+0000 mon.a (mon.0) 1185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.147072+0000 mon.a (mon.0) 1186 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[30791]: audit 2021-03-10T14:33:48.148871+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: cluster 2021-03-10T14:33:46.187854+0000 mgr.gibba032.whopeo (mgr.24625) 2190 : cluster [DBG] pgmap v1239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.135378+0000 mon.a (mon.0) 1181 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:33:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.138715+0000 mon.a (mon.0) 1182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:33:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.140360+0000 mon.a (mon.0) 1183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.141904+0000 mon.a (mon.0) 1184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.143468+0000 mon.a (mon.0) 1185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.147072+0000 mon.a (mon.0) 1186 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:48 gibba011 bash[31774]: audit 2021-03-10T14:33:48.148871+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:47.893708+0000 mgr.gibba032.whopeo (mgr.24625) 2191 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:33:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:47.895258+0000 mgr.gibba032.whopeo (mgr.24625) 2192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:33:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.135014+0000 mgr.gibba032.whopeo (mgr.24625) 2193 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:33:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.137045+0000 mgr.gibba032.whopeo (mgr.24625) 2194 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:33:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.138374+0000 mgr.gibba032.whopeo (mgr.24625) 2195 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:33:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.138411+0000 mgr.gibba032.whopeo (mgr.24625) 2196 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:33:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.140169+0000 mgr.gibba032.whopeo (mgr.24625) 2197 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:33:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.140205+0000 mgr.gibba032.whopeo (mgr.24625) 2198 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.141740+0000 mgr.gibba032.whopeo (mgr.24625) 2199 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.143628+0000 mgr.gibba032.whopeo (mgr.24625) 2200 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.143665+0000 mgr.gibba032.whopeo (mgr.24625) 2201 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.145261+0000 mgr.gibba032.whopeo (mgr.24625) 2202 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.145580+0000 mgr.gibba032.whopeo (mgr.24625) 2203 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:33:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.147039+0000 mgr.gibba032.whopeo (mgr.24625) 2204 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:33:49.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.147124+0000 mgr.gibba032.whopeo (mgr.24625) 2205 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:33:49.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:49 gibba032 bash[24004]: cephadm 2021-03-10T14:33:48.147383+0000 mgr.gibba032.whopeo (mgr.24625) 2206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:47.893708+0000 mgr.gibba032.whopeo (mgr.24625) 2191 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:33:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:47.895258+0000 mgr.gibba032.whopeo (mgr.24625) 2192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:33:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.135014+0000 mgr.gibba032.whopeo (mgr.24625) 2193 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:33:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.137045+0000 mgr.gibba032.whopeo (mgr.24625) 2194 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:33:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.138374+0000 mgr.gibba032.whopeo (mgr.24625) 2195 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:33:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.138411+0000 mgr.gibba032.whopeo (mgr.24625) 2196 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:33:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.140169+0000 mgr.gibba032.whopeo (mgr.24625) 2197 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:33:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.140205+0000 mgr.gibba032.whopeo (mgr.24625) 2198 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:33:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.141740+0000 mgr.gibba032.whopeo (mgr.24625) 2199 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:33:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.143628+0000 mgr.gibba032.whopeo (mgr.24625) 2200 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:33:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.143665+0000 mgr.gibba032.whopeo (mgr.24625) 2201 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:33:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.145261+0000 mgr.gibba032.whopeo (mgr.24625) 2202 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:33:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.145580+0000 mgr.gibba032.whopeo (mgr.24625) 2203 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:33:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.147039+0000 mgr.gibba032.whopeo (mgr.24625) 2204 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:33:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.147124+0000 mgr.gibba032.whopeo (mgr.24625) 2205 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:33:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[30791]: cephadm 2021-03-10T14:33:48.147383+0000 mgr.gibba032.whopeo (mgr.24625) 2206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:47.893708+0000 mgr.gibba032.whopeo (mgr.24625) 2191 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:33:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:47.895258+0000 mgr.gibba032.whopeo (mgr.24625) 2192 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:33:49.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.135014+0000 mgr.gibba032.whopeo (mgr.24625) 2193 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:33:49.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.137045+0000 mgr.gibba032.whopeo (mgr.24625) 2194 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:33:49.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.138374+0000 mgr.gibba032.whopeo (mgr.24625) 2195 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:33:49.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.138411+0000 mgr.gibba032.whopeo (mgr.24625) 2196 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.140169+0000 mgr.gibba032.whopeo (mgr.24625) 2197 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.140205+0000 mgr.gibba032.whopeo (mgr.24625) 2198 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.141740+0000 mgr.gibba032.whopeo (mgr.24625) 2199 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.143628+0000 mgr.gibba032.whopeo (mgr.24625) 2200 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.143665+0000 mgr.gibba032.whopeo (mgr.24625) 2201 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:33:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.145261+0000 mgr.gibba032.whopeo (mgr.24625) 2202 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:33:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.145580+0000 mgr.gibba032.whopeo (mgr.24625) 2203 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:33:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.147039+0000 mgr.gibba032.whopeo (mgr.24625) 2204 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:33:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.147124+0000 mgr.gibba032.whopeo (mgr.24625) 2205 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:33:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:49 gibba011 bash[31774]: cephadm 2021-03-10T14:33:48.147383+0000 mgr.gibba032.whopeo (mgr.24625) 2206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:33:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:50 gibba032 bash[24004]: cluster 2021-03-10T14:33:48.188589+0000 mgr.gibba032.whopeo (mgr.24625) 2207 : cluster [DBG] pgmap v1240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:50 gibba011 bash[30791]: cluster 2021-03-10T14:33:48.188589+0000 mgr.gibba032.whopeo (mgr.24625) 2207 : cluster [DBG] pgmap v1240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:50 gibba011 bash[31774]: cluster 2021-03-10T14:33:48.188589+0000 mgr.gibba032.whopeo (mgr.24625) 2207 : cluster [DBG] pgmap v1240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:51.059 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:33:51.318 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:33:51.318 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (40m) 6m ago 45m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:33:51.318 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (45m) 5m ago 45m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:33:51.319 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (40m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:33:51.319 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (41m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:33:51.319 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (40m) 6m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:33:51.320 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (40m) 6m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:33:51.320 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (39m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:33:51.320 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (39m) 6m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:33:51.320 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (45m) 6m ago 46m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:33:51.320 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (45m) 5m ago 46m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:33:51.321 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (38m) 6m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:33:51.321 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (38m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:33:51.321 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (37m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:33:51.321 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (37m) 6m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:33:51.322 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (37m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:33:51.322 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (36m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:33:51.322 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (36m) 5m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:33:51.322 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (35m) 5m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:33:51.323 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (45m) 5m ago 46m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:33:51.323 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 43m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:33:51.578 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:33:51.578 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:33:51.579 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:33:51.579 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:51.579 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:33:51.579 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:33:51.579 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:51.580 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:33:51.580 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:33:51.580 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:33:51.580 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:33:51.581 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:33:51.581 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:33:51.581 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:33:51.581 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:33:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:52 gibba032 bash[24004]: cluster 2021-03-10T14:33:50.189189+0000 mgr.gibba032.whopeo (mgr.24625) 2208 : cluster [DBG] pgmap v1241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:52 gibba032 bash[24004]: audit 2021-03-10T14:33:51.048006+0000 mgr.gibba032.whopeo (mgr.24625) 2209 : audit [DBG] from='client.45531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:52 gibba032 bash[24004]: audit 2021-03-10T14:33:51.577528+0000 mon.a (mon.0) 1188 : audit [DBG] from='client.? 172.21.2.111:0/1042690225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[30791]: cluster 2021-03-10T14:33:50.189189+0000 mgr.gibba032.whopeo (mgr.24625) 2208 : cluster [DBG] pgmap v1241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[30791]: audit 2021-03-10T14:33:51.048006+0000 mgr.gibba032.whopeo (mgr.24625) 2209 : audit [DBG] from='client.45531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[30791]: audit 2021-03-10T14:33:51.577528+0000 mon.a (mon.0) 1188 : audit [DBG] from='client.? 172.21.2.111:0/1042690225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[31774]: cluster 2021-03-10T14:33:50.189189+0000 mgr.gibba032.whopeo (mgr.24625) 2208 : cluster [DBG] pgmap v1241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[31774]: audit 2021-03-10T14:33:51.048006+0000 mgr.gibba032.whopeo (mgr.24625) 2209 : audit [DBG] from='client.45531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:52 gibba011 bash[31774]: audit 2021-03-10T14:33:51.577528+0000 mon.a (mon.0) 1188 : audit [DBG] from='client.? 172.21.2.111:0/1042690225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:33:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:53 gibba032 bash[24004]: audit 2021-03-10T14:33:51.274000+0000 mgr.gibba032.whopeo (mgr.24625) 2210 : audit [DBG] from='client.55522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:53.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:53 gibba011 bash[30791]: audit 2021-03-10T14:33:51.274000+0000 mgr.gibba032.whopeo (mgr.24625) 2210 : audit [DBG] from='client.55522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:53.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:53 gibba011 bash[31774]: audit 2021-03-10T14:33:51.274000+0000 mgr.gibba032.whopeo (mgr.24625) 2210 : audit [DBG] from='client.55522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:33:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:54 gibba032 bash[24004]: cluster 2021-03-10T14:33:52.189879+0000 mgr.gibba032.whopeo (mgr.24625) 2211 : cluster [DBG] pgmap v1242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:54 gibba011 bash[30791]: cluster 2021-03-10T14:33:52.189879+0000 mgr.gibba032.whopeo (mgr.24625) 2211 : cluster [DBG] pgmap v1242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:54 gibba011 bash[31774]: cluster 2021-03-10T14:33:52.189879+0000 mgr.gibba032.whopeo (mgr.24625) 2211 : cluster [DBG] pgmap v1242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:55 gibba032 bash[24004]: cluster 2021-03-10T14:33:54.190860+0000 mgr.gibba032.whopeo (mgr.24625) 2212 : cluster [DBG] pgmap v1243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:55.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:55 gibba011 bash[30791]: cluster 2021-03-10T14:33:54.190860+0000 mgr.gibba032.whopeo (mgr.24625) 2212 : cluster [DBG] pgmap v1243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:55 gibba011 bash[31774]: cluster 2021-03-10T14:33:54.190860+0000 mgr.gibba032.whopeo (mgr.24625) 2212 : cluster [DBG] pgmap v1243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.265043+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:33:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.268627+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:33:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.277397+0000 mon.a (mon.0) 1191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:33:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.278119+0000 mon.a (mon.0) 1192 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:33:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.278800+0000 mon.a (mon.0) 1193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:33:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[30791]: audit 2021-03-10T14:33:56.279511+0000 mon.a (mon.0) 1194 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:33:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.265043+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:33:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.268627+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:33:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.277397+0000 mon.a (mon.0) 1191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:33:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.278119+0000 mon.a (mon.0) 1192 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:33:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.278800+0000 mon.a (mon.0) 1193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:33:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:57 gibba011 bash[31774]: audit 2021-03-10T14:33:56.279511+0000 mon.a (mon.0) 1194 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:33:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.265043+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:33:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.268627+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.277397+0000 mon.a (mon.0) 1191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.278119+0000 mon.a (mon.0) 1192 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.278800+0000 mon.a (mon.0) 1193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:57 gibba032 bash[24004]: audit 2021-03-10T14:33:56.279511+0000 mon.a (mon.0) 1194 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:33:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:33:58 gibba011 bash[30791]: cluster 2021-03-10T14:33:56.192348+0000 mgr.gibba032.whopeo (mgr.24625) 2213 : cluster [DBG] pgmap v1244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:33:58 gibba011 bash[31774]: cluster 2021-03-10T14:33:56.192348+0000 mgr.gibba032.whopeo (mgr.24625) 2213 : cluster [DBG] pgmap v1244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:33:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:33:58 gibba032 bash[24004]: cluster 2021-03-10T14:33:56.192348+0000 mgr.gibba032.whopeo (mgr.24625) 2213 : cluster [DBG] pgmap v1244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:00 gibba011 bash[30791]: cluster 2021-03-10T14:33:58.194106+0000 mgr.gibba032.whopeo (mgr.24625) 2214 : cluster [DBG] pgmap v1245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:00 gibba011 bash[31774]: cluster 2021-03-10T14:33:58.194106+0000 mgr.gibba032.whopeo (mgr.24625) 2214 : cluster [DBG] pgmap v1245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:00 gibba032 bash[24004]: cluster 2021-03-10T14:33:58.194106+0000 mgr.gibba032.whopeo (mgr.24625) 2214 : cluster [DBG] pgmap v1245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:02 gibba011 bash[30791]: cluster 2021-03-10T14:34:00.195423+0000 mgr.gibba032.whopeo (mgr.24625) 2215 : cluster [DBG] pgmap v1246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:02.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:02 gibba011 bash[31774]: cluster 2021-03-10T14:34:00.195423+0000 mgr.gibba032.whopeo (mgr.24625) 2215 : cluster [DBG] pgmap v1246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:02 gibba032 bash[24004]: cluster 2021-03-10T14:34:00.195423+0000 mgr.gibba032.whopeo (mgr.24625) 2215 : cluster [DBG] pgmap v1246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:04.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:04 gibba011 bash[30791]: cluster 2021-03-10T14:34:02.197252+0000 mgr.gibba032.whopeo (mgr.24625) 2216 : cluster [DBG] pgmap v1247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:04 gibba011 bash[30791]: cephadm 2021-03-10T14:34:03.163467+0000 mgr.gibba032.whopeo (mgr.24625) 2217 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:04 gibba011 bash[31774]: cluster 2021-03-10T14:34:02.197252+0000 mgr.gibba032.whopeo (mgr.24625) 2216 : cluster [DBG] pgmap v1247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:04.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:04 gibba011 bash[31774]: cephadm 2021-03-10T14:34:03.163467+0000 mgr.gibba032.whopeo (mgr.24625) 2217 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:04 gibba032 bash[24004]: cluster 2021-03-10T14:34:02.197252+0000 mgr.gibba032.whopeo (mgr.24625) 2216 : cluster [DBG] pgmap v1247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:04 gibba032 bash[24004]: cephadm 2021-03-10T14:34:03.163467+0000 mgr.gibba032.whopeo (mgr.24625) 2217 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:06 gibba011 bash[30791]: cluster 2021-03-10T14:34:04.198475+0000 mgr.gibba032.whopeo (mgr.24625) 2218 : cluster [DBG] pgmap v1248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:06.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:06 gibba011 bash[31774]: cluster 2021-03-10T14:34:04.198475+0000 mgr.gibba032.whopeo (mgr.24625) 2218 : cluster [DBG] pgmap v1248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:06 gibba032 bash[24004]: cluster 2021-03-10T14:34:04.198475+0000 mgr.gibba032.whopeo (mgr.24625) 2218 : cluster [DBG] pgmap v1248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:08.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:08 gibba011 bash[30791]: cluster 2021-03-10T14:34:06.200122+0000 mgr.gibba032.whopeo (mgr.24625) 2219 : cluster [DBG] pgmap v1249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:08.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:08 gibba011 bash[31774]: cluster 2021-03-10T14:34:06.200122+0000 mgr.gibba032.whopeo (mgr.24625) 2219 : cluster [DBG] pgmap v1249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:08 gibba032 bash[24004]: cluster 2021-03-10T14:34:06.200122+0000 mgr.gibba032.whopeo (mgr.24625) 2219 : cluster [DBG] pgmap v1249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:10 gibba032 bash[24004]: cluster 2021-03-10T14:34:08.201628+0000 mgr.gibba032.whopeo (mgr.24625) 2220 : cluster [DBG] pgmap v1250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:10 gibba011 bash[30791]: cluster 2021-03-10T14:34:08.201628+0000 mgr.gibba032.whopeo (mgr.24625) 2220 : cluster [DBG] pgmap v1250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:10 gibba011 bash[31774]: cluster 2021-03-10T14:34:08.201628+0000 mgr.gibba032.whopeo (mgr.24625) 2220 : cluster [DBG] pgmap v1250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:12 gibba032 bash[24004]: cluster 2021-03-10T14:34:10.203225+0000 mgr.gibba032.whopeo (mgr.24625) 2221 : cluster [DBG] pgmap v1251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:12 gibba011 bash[30791]: cluster 2021-03-10T14:34:10.203225+0000 mgr.gibba032.whopeo (mgr.24625) 2221 : cluster [DBG] pgmap v1251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:12 gibba011 bash[31774]: cluster 2021-03-10T14:34:10.203225+0000 mgr.gibba032.whopeo (mgr.24625) 2221 : cluster [DBG] pgmap v1251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:14 gibba032 bash[24004]: cluster 2021-03-10T14:34:12.204891+0000 mgr.gibba032.whopeo (mgr.24625) 2222 : cluster [DBG] pgmap v1252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:14 gibba011 bash[30791]: cluster 2021-03-10T14:34:12.204891+0000 mgr.gibba032.whopeo (mgr.24625) 2222 : cluster [DBG] pgmap v1252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:14 gibba011 bash[31774]: cluster 2021-03-10T14:34:12.204891+0000 mgr.gibba032.whopeo (mgr.24625) 2222 : cluster [DBG] pgmap v1252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:16 gibba032 bash[24004]: cluster 2021-03-10T14:34:14.206134+0000 mgr.gibba032.whopeo (mgr.24625) 2223 : cluster [DBG] pgmap v1253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:16 gibba011 bash[30791]: cluster 2021-03-10T14:34:14.206134+0000 mgr.gibba032.whopeo (mgr.24625) 2223 : cluster [DBG] pgmap v1253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:16 gibba011 bash[31774]: cluster 2021-03-10T14:34:14.206134+0000 mgr.gibba032.whopeo (mgr.24625) 2223 : cluster [DBG] pgmap v1253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:18 gibba032 bash[24004]: cluster 2021-03-10T14:34:16.207749+0000 mgr.gibba032.whopeo (mgr.24625) 2224 : cluster [DBG] pgmap v1254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:18 gibba011 bash[30791]: cluster 2021-03-10T14:34:16.207749+0000 mgr.gibba032.whopeo (mgr.24625) 2224 : cluster [DBG] pgmap v1254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:18 gibba011 bash[31774]: cluster 2021-03-10T14:34:16.207749+0000 mgr.gibba032.whopeo (mgr.24625) 2224 : cluster [DBG] pgmap v1254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:19 gibba032 bash[24004]: cephadm 2021-03-10T14:34:18.171984+0000 mgr.gibba032.whopeo (mgr.24625) 2225 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:19.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:19 gibba011 bash[30791]: cephadm 2021-03-10T14:34:18.171984+0000 mgr.gibba032.whopeo (mgr.24625) 2225 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:19.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:19 gibba011 bash[31774]: cephadm 2021-03-10T14:34:18.171984+0000 mgr.gibba032.whopeo (mgr.24625) 2225 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:20 gibba032 bash[24004]: cluster 2021-03-10T14:34:18.209493+0000 mgr.gibba032.whopeo (mgr.24625) 2226 : cluster [DBG] pgmap v1255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:20 gibba011 bash[30791]: cluster 2021-03-10T14:34:18.209493+0000 mgr.gibba032.whopeo (mgr.24625) 2226 : cluster [DBG] pgmap v1255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:20 gibba011 bash[31774]: cluster 2021-03-10T14:34:18.209493+0000 mgr.gibba032.whopeo (mgr.24625) 2226 : cluster [DBG] pgmap v1255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:21.820 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:34:22.047 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:34:22.047 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (41m) 6m ago 46m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:34:22.048 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (46m) 5m ago 46m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:34:22.048 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (41m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:34:22.048 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (41m) 5m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:34:22.048 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (40m) 6m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:34:22.049 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (40m) 6m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:34:22.049 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (40m) 5m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:34:22.049 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (40m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:34:22.049 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (46m) 6m ago 46m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:34:22.049 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (46m) 5m ago 46m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:34:22.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (39m) 6m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:34:22.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (38m) 6m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:34:22.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (38m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:34:22.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (38m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:34:22.050 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (37m) 5m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:34:22.051 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (37m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:34:22.051 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (36m) 5m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:34:22.051 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (36m) 5m ago 46m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:34:22.051 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (45m) 5m ago 46m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:34:22.051 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 44m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:34:22.301 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:34:22.301 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:34:22.302 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:34:22.302 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:22.302 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:34:22.302 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:34:22.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:22.303 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:34:22.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:34:22.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:22.303 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:34:22.304 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:34:22.304 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:34:22.304 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:34:22.304 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:34:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:22 gibba032 bash[24004]: cluster 2021-03-10T14:34:20.210493+0000 mgr.gibba032.whopeo (mgr.24625) 2227 : cluster [DBG] pgmap v1256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:22 gibba011 bash[30791]: cluster 2021-03-10T14:34:20.210493+0000 mgr.gibba032.whopeo (mgr.24625) 2227 : cluster [DBG] pgmap v1256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:22 gibba011 bash[31774]: cluster 2021-03-10T14:34:20.210493+0000 mgr.gibba032.whopeo (mgr.24625) 2227 : cluster [DBG] pgmap v1256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:23 gibba032 bash[24004]: audit 2021-03-10T14:34:21.808134+0000 mgr.gibba032.whopeo (mgr.24625) 2228 : audit [DBG] from='client.45549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:23 gibba032 bash[24004]: audit 2021-03-10T14:34:22.031632+0000 mgr.gibba032.whopeo (mgr.24625) 2229 : audit [DBG] from='client.45555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:23 gibba032 bash[24004]: audit 2021-03-10T14:34:22.301030+0000 mon.c (mon.1) 45 : audit [DBG] from='client.? 172.21.2.111:0/840417493' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[30791]: audit 2021-03-10T14:34:21.808134+0000 mgr.gibba032.whopeo (mgr.24625) 2228 : audit [DBG] from='client.45549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[30791]: audit 2021-03-10T14:34:22.031632+0000 mgr.gibba032.whopeo (mgr.24625) 2229 : audit [DBG] from='client.45555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[30791]: audit 2021-03-10T14:34:22.301030+0000 mon.c (mon.1) 45 : audit [DBG] from='client.? 172.21.2.111:0/840417493' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[31774]: audit 2021-03-10T14:34:21.808134+0000 mgr.gibba032.whopeo (mgr.24625) 2228 : audit [DBG] from='client.45549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[31774]: audit 2021-03-10T14:34:22.031632+0000 mgr.gibba032.whopeo (mgr.24625) 2229 : audit [DBG] from='client.45555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:23.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:23 gibba011 bash[31774]: audit 2021-03-10T14:34:22.301030+0000 mon.c (mon.1) 45 : audit [DBG] from='client.? 172.21.2.111:0/840417493' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:24 gibba032 bash[24004]: cluster 2021-03-10T14:34:22.211427+0000 mgr.gibba032.whopeo (mgr.24625) 2230 : cluster [DBG] pgmap v1257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:24 gibba011 bash[30791]: cluster 2021-03-10T14:34:22.211427+0000 mgr.gibba032.whopeo (mgr.24625) 2230 : cluster [DBG] pgmap v1257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:24 gibba011 bash[31774]: cluster 2021-03-10T14:34:22.211427+0000 mgr.gibba032.whopeo (mgr.24625) 2230 : cluster [DBG] pgmap v1257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:26 gibba032 bash[24004]: cluster 2021-03-10T14:34:24.212070+0000 mgr.gibba032.whopeo (mgr.24625) 2231 : cluster [DBG] pgmap v1258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:26 gibba011 bash[30791]: cluster 2021-03-10T14:34:24.212070+0000 mgr.gibba032.whopeo (mgr.24625) 2231 : cluster [DBG] pgmap v1258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:26 gibba011 bash[31774]: cluster 2021-03-10T14:34:24.212070+0000 mgr.gibba032.whopeo (mgr.24625) 2231 : cluster [DBG] pgmap v1258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:28.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:28 gibba032 bash[24004]: cluster 2021-03-10T14:34:26.213602+0000 mgr.gibba032.whopeo (mgr.24625) 2232 : cluster [DBG] pgmap v1259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:28 gibba011 bash[30791]: cluster 2021-03-10T14:34:26.213602+0000 mgr.gibba032.whopeo (mgr.24625) 2232 : cluster [DBG] pgmap v1259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:28 gibba011 bash[31774]: cluster 2021-03-10T14:34:26.213602+0000 mgr.gibba032.whopeo (mgr.24625) 2232 : cluster [DBG] pgmap v1259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:30 gibba032 bash[24004]: cluster 2021-03-10T14:34:28.215215+0000 mgr.gibba032.whopeo (mgr.24625) 2233 : cluster [DBG] pgmap v1260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:30 gibba011 bash[30791]: cluster 2021-03-10T14:34:28.215215+0000 mgr.gibba032.whopeo (mgr.24625) 2233 : cluster [DBG] pgmap v1260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:30 gibba011 bash[31774]: cluster 2021-03-10T14:34:28.215215+0000 mgr.gibba032.whopeo (mgr.24625) 2233 : cluster [DBG] pgmap v1260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:32 gibba032 bash[24004]: cluster 2021-03-10T14:34:30.216171+0000 mgr.gibba032.whopeo (mgr.24625) 2234 : cluster [DBG] pgmap v1261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:32 gibba011 bash[30791]: cluster 2021-03-10T14:34:30.216171+0000 mgr.gibba032.whopeo (mgr.24625) 2234 : cluster [DBG] pgmap v1261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:32 gibba011 bash[31774]: cluster 2021-03-10T14:34:30.216171+0000 mgr.gibba032.whopeo (mgr.24625) 2234 : cluster [DBG] pgmap v1261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:34 gibba032 bash[24004]: cluster 2021-03-10T14:34:32.216653+0000 mgr.gibba032.whopeo (mgr.24625) 2235 : cluster [DBG] pgmap v1262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:34 gibba032 bash[24004]: cephadm 2021-03-10T14:34:33.186363+0000 mgr.gibba032.whopeo (mgr.24625) 2236 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:34 gibba011 bash[30791]: cluster 2021-03-10T14:34:32.216653+0000 mgr.gibba032.whopeo (mgr.24625) 2235 : cluster [DBG] pgmap v1262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:34 gibba011 bash[30791]: cephadm 2021-03-10T14:34:33.186363+0000 mgr.gibba032.whopeo (mgr.24625) 2236 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:34 gibba011 bash[31774]: cluster 2021-03-10T14:34:32.216653+0000 mgr.gibba032.whopeo (mgr.24625) 2235 : cluster [DBG] pgmap v1262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:34 gibba011 bash[31774]: cephadm 2021-03-10T14:34:33.186363+0000 mgr.gibba032.whopeo (mgr.24625) 2236 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:35 gibba032 bash[24004]: cluster 2021-03-10T14:34:34.217662+0000 mgr.gibba032.whopeo (mgr.24625) 2237 : cluster [DBG] pgmap v1263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:35 gibba011 bash[30791]: cluster 2021-03-10T14:34:34.217662+0000 mgr.gibba032.whopeo (mgr.24625) 2237 : cluster [DBG] pgmap v1263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:35 gibba011 bash[31774]: cluster 2021-03-10T14:34:34.217662+0000 mgr.gibba032.whopeo (mgr.24625) 2237 : cluster [DBG] pgmap v1263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:38.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:38 gibba011 bash[30791]: cluster 2021-03-10T14:34:36.219251+0000 mgr.gibba032.whopeo (mgr.24625) 2238 : cluster [DBG] pgmap v1264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:38 gibba011 bash[31774]: cluster 2021-03-10T14:34:36.219251+0000 mgr.gibba032.whopeo (mgr.24625) 2238 : cluster [DBG] pgmap v1264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:38 gibba032 bash[24004]: cluster 2021-03-10T14:34:36.219251+0000 mgr.gibba032.whopeo (mgr.24625) 2238 : cluster [DBG] pgmap v1264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:40 gibba011 bash[30791]: cluster 2021-03-10T14:34:38.221024+0000 mgr.gibba032.whopeo (mgr.24625) 2239 : cluster [DBG] pgmap v1265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:40.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:40 gibba011 bash[31774]: cluster 2021-03-10T14:34:38.221024+0000 mgr.gibba032.whopeo (mgr.24625) 2239 : cluster [DBG] pgmap v1265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:40 gibba032 bash[24004]: cluster 2021-03-10T14:34:38.221024+0000 mgr.gibba032.whopeo (mgr.24625) 2239 : cluster [DBG] pgmap v1265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:42 gibba011 bash[30791]: cluster 2021-03-10T14:34:40.222647+0000 mgr.gibba032.whopeo (mgr.24625) 2240 : cluster [DBG] pgmap v1266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:42 gibba011 bash[31774]: cluster 2021-03-10T14:34:40.222647+0000 mgr.gibba032.whopeo (mgr.24625) 2240 : cluster [DBG] pgmap v1266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:42 gibba032 bash[24004]: cluster 2021-03-10T14:34:40.222647+0000 mgr.gibba032.whopeo (mgr.24625) 2240 : cluster [DBG] pgmap v1266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:44 gibba011 bash[30791]: cluster 2021-03-10T14:34:42.224474+0000 mgr.gibba032.whopeo (mgr.24625) 2241 : cluster [DBG] pgmap v1267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:44.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:44 gibba011 bash[31774]: cluster 2021-03-10T14:34:42.224474+0000 mgr.gibba032.whopeo (mgr.24625) 2241 : cluster [DBG] pgmap v1267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:44 gibba032 bash[24004]: cluster 2021-03-10T14:34:42.224474+0000 mgr.gibba032.whopeo (mgr.24625) 2241 : cluster [DBG] pgmap v1267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:46 gibba032 bash[24004]: cluster 2021-03-10T14:34:44.225815+0000 mgr.gibba032.whopeo (mgr.24625) 2242 : cluster [DBG] pgmap v1268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:46 gibba011 bash[30791]: cluster 2021-03-10T14:34:44.225815+0000 mgr.gibba032.whopeo (mgr.24625) 2242 : cluster [DBG] pgmap v1268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:46 gibba011 bash[31774]: cluster 2021-03-10T14:34:44.225815+0000 mgr.gibba032.whopeo (mgr.24625) 2242 : cluster [DBG] pgmap v1268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:48 gibba032 bash[24004]: cluster 2021-03-10T14:34:46.227432+0000 mgr.gibba032.whopeo (mgr.24625) 2243 : cluster [DBG] pgmap v1269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:48 gibba011 bash[30791]: cluster 2021-03-10T14:34:46.227432+0000 mgr.gibba032.whopeo (mgr.24625) 2243 : cluster [DBG] pgmap v1269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:48 gibba011 bash[31774]: cluster 2021-03-10T14:34:46.227432+0000 mgr.gibba032.whopeo (mgr.24625) 2243 : cluster [DBG] pgmap v1269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:49.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cluster 2021-03-10T14:34:48.227933+0000 mgr.gibba032.whopeo (mgr.24625) 2244 : cluster [DBG] pgmap v1270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:49.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.400893+0000 mgr.gibba032.whopeo (mgr.24625) 2245 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:34:49.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.403136+0000 mon.a (mon.0) 1195 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.415092+0000 mgr.gibba032.whopeo (mgr.24625) 2246 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:34:49.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.417367+0000 mon.a (mon.0) 1196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.605863+0000 mon.a (mon.0) 1197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.605867+0000 mgr.gibba032.whopeo (mgr.24625) 2247 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:34:49.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.607553+0000 mgr.gibba032.whopeo (mgr.24625) 2248 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:34:49.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.608819+0000 mgr.gibba032.whopeo (mgr.24625) 2249 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:34:49.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.608858+0000 mgr.gibba032.whopeo (mgr.24625) 2250 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:34:49.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.609469+0000 mon.a (mon.0) 1198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:34:49.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.610201+0000 mgr.gibba032.whopeo (mgr.24625) 2251 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:34:49.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.610244+0000 mgr.gibba032.whopeo (mgr.24625) 2252 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:34:49.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.610644+0000 mon.a (mon.0) 1199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.611446+0000 mgr.gibba032.whopeo (mgr.24625) 2253 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:34:49.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.612138+0000 mon.a (mon.0) 1200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.612696+0000 mgr.gibba032.whopeo (mgr.24625) 2254 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:34:49.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.612734+0000 mgr.gibba032.whopeo (mgr.24625) 2255 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:34:49.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.613402+0000 mon.a (mon.0) 1201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.614149+0000 mgr.gibba032.whopeo (mgr.24625) 2256 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:34:49.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.614470+0000 mgr.gibba032.whopeo (mgr.24625) 2257 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:34:49.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.615866+0000 mgr.gibba032.whopeo (mgr.24625) 2258 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:34:49.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.615906+0000 mgr.gibba032.whopeo (mgr.24625) 2259 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:34:49.769 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: cephadm 2021-03-10T14:34:48.616012+0000 mgr.gibba032.whopeo (mgr.24625) 2260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:49.769 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.616020+0000 mon.a (mon.0) 1202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.770 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:49 gibba032 bash[24004]: audit 2021-03-10T14:34:48.617593+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cluster 2021-03-10T14:34:48.227933+0000 mgr.gibba032.whopeo (mgr.24625) 2244 : cluster [DBG] pgmap v1270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:49.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.400893+0000 mgr.gibba032.whopeo (mgr.24625) 2245 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:34:49.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.403136+0000 mon.a (mon.0) 1195 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.415092+0000 mgr.gibba032.whopeo (mgr.24625) 2246 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:34:49.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.417367+0000 mon.a (mon.0) 1196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.605863+0000 mon.a (mon.0) 1197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.605867+0000 mgr.gibba032.whopeo (mgr.24625) 2247 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:34:49.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.607553+0000 mgr.gibba032.whopeo (mgr.24625) 2248 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:34:49.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.608819+0000 mgr.gibba032.whopeo (mgr.24625) 2249 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:34:49.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.608858+0000 mgr.gibba032.whopeo (mgr.24625) 2250 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:34:49.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.609469+0000 mon.a (mon.0) 1198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:34:49.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.610201+0000 mgr.gibba032.whopeo (mgr.24625) 2251 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:34:49.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.610244+0000 mgr.gibba032.whopeo (mgr.24625) 2252 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:34:49.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.610644+0000 mon.a (mon.0) 1199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.611446+0000 mgr.gibba032.whopeo (mgr.24625) 2253 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:34:49.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.612138+0000 mon.a (mon.0) 1200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.612696+0000 mgr.gibba032.whopeo (mgr.24625) 2254 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:34:49.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.612734+0000 mgr.gibba032.whopeo (mgr.24625) 2255 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:34:49.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.613402+0000 mon.a (mon.0) 1201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.614149+0000 mgr.gibba032.whopeo (mgr.24625) 2256 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:34:49.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.614470+0000 mgr.gibba032.whopeo (mgr.24625) 2257 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:34:49.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cluster 2021-03-10T14:34:48.227933+0000 mgr.gibba032.whopeo (mgr.24625) 2244 : cluster [DBG] pgmap v1270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:49.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.400893+0000 mgr.gibba032.whopeo (mgr.24625) 2245 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:34:49.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.403136+0000 mon.a (mon.0) 1195 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.415092+0000 mgr.gibba032.whopeo (mgr.24625) 2246 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:34:49.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.417367+0000 mon.a (mon.0) 1196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.605863+0000 mon.a (mon.0) 1197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:34:49.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.605867+0000 mgr.gibba032.whopeo (mgr.24625) 2247 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:34:49.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.607553+0000 mgr.gibba032.whopeo (mgr.24625) 2248 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:34:49.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.608819+0000 mgr.gibba032.whopeo (mgr.24625) 2249 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:34:49.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.608858+0000 mgr.gibba032.whopeo (mgr.24625) 2250 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:34:49.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.609469+0000 mon.a (mon.0) 1198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:34:49.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.610201+0000 mgr.gibba032.whopeo (mgr.24625) 2251 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:34:49.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.610244+0000 mgr.gibba032.whopeo (mgr.24625) 2252 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:34:49.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.610644+0000 mon.a (mon.0) 1199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.611446+0000 mgr.gibba032.whopeo (mgr.24625) 2253 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:34:49.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.612138+0000 mon.a (mon.0) 1200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.612696+0000 mgr.gibba032.whopeo (mgr.24625) 2254 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:34:49.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.612734+0000 mgr.gibba032.whopeo (mgr.24625) 2255 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:34:49.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.613402+0000 mon.a (mon.0) 1201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.614149+0000 mgr.gibba032.whopeo (mgr.24625) 2256 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:34:49.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.614470+0000 mgr.gibba032.whopeo (mgr.24625) 2257 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:34:49.848 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.615866+0000 mgr.gibba032.whopeo (mgr.24625) 2258 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:34:49.849 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.615906+0000 mgr.gibba032.whopeo (mgr.24625) 2259 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:34:49.849 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: cephadm 2021-03-10T14:34:48.616012+0000 mgr.gibba032.whopeo (mgr.24625) 2260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:49.850 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.616020+0000 mon.a (mon.0) 1202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.850 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[30791]: audit 2021-03-10T14:34:48.617593+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.615866+0000 mgr.gibba032.whopeo (mgr.24625) 2258 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:34:49.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.615906+0000 mgr.gibba032.whopeo (mgr.24625) 2259 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:34:49.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: cephadm 2021-03-10T14:34:48.616012+0000 mgr.gibba032.whopeo (mgr.24625) 2260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:34:49.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.616020+0000 mon.a (mon.0) 1202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:49.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:49 gibba011 bash[31774]: audit 2021-03-10T14:34:48.617593+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:52 gibba011 bash[30791]: cluster 2021-03-10T14:34:50.228720+0000 mgr.gibba032.whopeo (mgr.24625) 2261 : cluster [DBG] pgmap v1271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:52 gibba011 bash[31774]: cluster 2021-03-10T14:34:50.228720+0000 mgr.gibba032.whopeo (mgr.24625) 2261 : cluster [DBG] pgmap v1271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:52 gibba032 bash[24004]: cluster 2021-03-10T14:34:50.228720+0000 mgr.gibba032.whopeo (mgr.24625) 2261 : cluster [DBG] pgmap v1271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:52.545 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:34:52.773 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:34:52.774 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (41m) 7m ago 46m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:34:52.774 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (46m) 6m ago 46m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:34:52.774 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (41m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:34:52.774 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (42m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:34:52.774 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (41m) 7m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:34:52.775 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (41m) 7m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:34:52.775 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (40m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:34:52.775 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (40m) 7m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:34:52.775 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (47m) 7m ago 47m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:34:52.775 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (47m) 6m ago 47m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:34:52.776 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (39m) 7m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:34:52.776 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (39m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:34:52.776 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (38m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:34:52.776 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (38m) 7m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:34:52.776 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (38m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:34:52.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (37m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:34:52.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (37m) 6m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:34:52.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (36m) 6m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:34:52.777 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (46m) 6m ago 47m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:34:52.777 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 44m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:34:53.023 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:34:53.023 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:34:53.023 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:34:53.024 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:53.024 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:34:53.024 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:34:53.024 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:53.024 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:34:53.025 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:34:53.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:34:53.025 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:34:53.025 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:34:53.025 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:34:53.026 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:34:53.026 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:34:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:53 gibba011 bash[30791]: audit 2021-03-10T14:34:53.022642+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.2.111:0/2431891674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:53 gibba011 bash[31774]: audit 2021-03-10T14:34:53.022642+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.2.111:0/2431891674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:53 gibba032 bash[24004]: audit 2021-03-10T14:34:53.022642+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.2.111:0/2431891674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:34:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:54 gibba032 bash[24004]: cluster 2021-03-10T14:34:52.230101+0000 mgr.gibba032.whopeo (mgr.24625) 2262 : cluster [DBG] pgmap v1272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:54 gibba032 bash[24004]: audit 2021-03-10T14:34:52.533495+0000 mgr.gibba032.whopeo (mgr.24625) 2263 : audit [DBG] from='client.45567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:54 gibba032 bash[24004]: audit 2021-03-10T14:34:52.758140+0000 mgr.gibba032.whopeo (mgr.24625) 2264 : audit [DBG] from='client.45573 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[30791]: cluster 2021-03-10T14:34:52.230101+0000 mgr.gibba032.whopeo (mgr.24625) 2262 : cluster [DBG] pgmap v1272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[30791]: audit 2021-03-10T14:34:52.533495+0000 mgr.gibba032.whopeo (mgr.24625) 2263 : audit [DBG] from='client.45567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[30791]: audit 2021-03-10T14:34:52.758140+0000 mgr.gibba032.whopeo (mgr.24625) 2264 : audit [DBG] from='client.45573 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[31774]: cluster 2021-03-10T14:34:52.230101+0000 mgr.gibba032.whopeo (mgr.24625) 2262 : cluster [DBG] pgmap v1272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[31774]: audit 2021-03-10T14:34:52.533495+0000 mgr.gibba032.whopeo (mgr.24625) 2263 : audit [DBG] from='client.45567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:54 gibba011 bash[31774]: audit 2021-03-10T14:34:52.758140+0000 mgr.gibba032.whopeo (mgr.24625) 2264 : audit [DBG] from='client.45573 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:34:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:56 gibba032 bash[24004]: cluster 2021-03-10T14:34:54.231442+0000 mgr.gibba032.whopeo (mgr.24625) 2265 : cluster [DBG] pgmap v1273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:56 gibba011 bash[30791]: cluster 2021-03-10T14:34:54.231442+0000 mgr.gibba032.whopeo (mgr.24625) 2265 : cluster [DBG] pgmap v1273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:56 gibba011 bash[31774]: cluster 2021-03-10T14:34:54.231442+0000 mgr.gibba032.whopeo (mgr.24625) 2265 : cluster [DBG] pgmap v1273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.270879+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:34:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.271577+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:34:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.282997+0000 mon.a (mon.0) 1207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:34:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.283763+0000 mon.a (mon.0) 1208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:34:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.284430+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:34:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:57 gibba032 bash[24004]: audit 2021-03-10T14:34:56.285086+0000 mon.a (mon.0) 1210 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:34:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.270879+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:34:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.271577+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:34:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.282997+0000 mon.a (mon.0) 1207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:34:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.283763+0000 mon.a (mon.0) 1208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:34:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.284430+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:34:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[30791]: audit 2021-03-10T14:34:56.285086+0000 mon.a (mon.0) 1210 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:34:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.270879+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:34:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.271577+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:34:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.282997+0000 mon.a (mon.0) 1207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:34:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.283763+0000 mon.a (mon.0) 1208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:34:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.284430+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:34:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:57 gibba011 bash[31774]: audit 2021-03-10T14:34:56.285086+0000 mon.a (mon.0) 1210 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:34:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:34:58 gibba032 bash[24004]: cluster 2021-03-10T14:34:56.233096+0000 mgr.gibba032.whopeo (mgr.24625) 2266 : cluster [DBG] pgmap v1274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:34:58 gibba011 bash[30791]: cluster 2021-03-10T14:34:56.233096+0000 mgr.gibba032.whopeo (mgr.24625) 2266 : cluster [DBG] pgmap v1274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:34:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:34:58 gibba011 bash[31774]: cluster 2021-03-10T14:34:56.233096+0000 mgr.gibba032.whopeo (mgr.24625) 2266 : cluster [DBG] pgmap v1274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:00 gibba032 bash[24004]: cluster 2021-03-10T14:34:58.234594+0000 mgr.gibba032.whopeo (mgr.24625) 2267 : cluster [DBG] pgmap v1275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:00 gibba011 bash[30791]: cluster 2021-03-10T14:34:58.234594+0000 mgr.gibba032.whopeo (mgr.24625) 2267 : cluster [DBG] pgmap v1275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:00 gibba011 bash[31774]: cluster 2021-03-10T14:34:58.234594+0000 mgr.gibba032.whopeo (mgr.24625) 2267 : cluster [DBG] pgmap v1275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:02 gibba032 bash[24004]: cluster 2021-03-10T14:35:00.235948+0000 mgr.gibba032.whopeo (mgr.24625) 2268 : cluster [DBG] pgmap v1276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:02 gibba011 bash[30791]: cluster 2021-03-10T14:35:00.235948+0000 mgr.gibba032.whopeo (mgr.24625) 2268 : cluster [DBG] pgmap v1276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:02 gibba011 bash[31774]: cluster 2021-03-10T14:35:00.235948+0000 mgr.gibba032.whopeo (mgr.24625) 2268 : cluster [DBG] pgmap v1276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:04 gibba032 bash[24004]: cluster 2021-03-10T14:35:02.237491+0000 mgr.gibba032.whopeo (mgr.24625) 2269 : cluster [DBG] pgmap v1277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:04 gibba011 bash[30791]: cluster 2021-03-10T14:35:02.237491+0000 mgr.gibba032.whopeo (mgr.24625) 2269 : cluster [DBG] pgmap v1277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:04 gibba011 bash[31774]: cluster 2021-03-10T14:35:02.237491+0000 mgr.gibba032.whopeo (mgr.24625) 2269 : cluster [DBG] pgmap v1277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:05 gibba032 bash[24004]: cephadm 2021-03-10T14:35:03.631933+0000 mgr.gibba032.whopeo (mgr.24625) 2270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:05 gibba011 bash[30791]: cephadm 2021-03-10T14:35:03.631933+0000 mgr.gibba032.whopeo (mgr.24625) 2270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:05 gibba011 bash[31774]: cephadm 2021-03-10T14:35:03.631933+0000 mgr.gibba032.whopeo (mgr.24625) 2270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:06 gibba032 bash[24004]: cluster 2021-03-10T14:35:04.238709+0000 mgr.gibba032.whopeo (mgr.24625) 2271 : cluster [DBG] pgmap v1278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:06 gibba011 bash[30791]: cluster 2021-03-10T14:35:04.238709+0000 mgr.gibba032.whopeo (mgr.24625) 2271 : cluster [DBG] pgmap v1278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:06 gibba011 bash[31774]: cluster 2021-03-10T14:35:04.238709+0000 mgr.gibba032.whopeo (mgr.24625) 2271 : cluster [DBG] pgmap v1278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:08 gibba032 bash[24004]: cluster 2021-03-10T14:35:06.240373+0000 mgr.gibba032.whopeo (mgr.24625) 2272 : cluster [DBG] pgmap v1279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:08 gibba011 bash[30791]: cluster 2021-03-10T14:35:06.240373+0000 mgr.gibba032.whopeo (mgr.24625) 2272 : cluster [DBG] pgmap v1279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:08 gibba011 bash[31774]: cluster 2021-03-10T14:35:06.240373+0000 mgr.gibba032.whopeo (mgr.24625) 2272 : cluster [DBG] pgmap v1279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:10 gibba032 bash[24004]: cluster 2021-03-10T14:35:08.241982+0000 mgr.gibba032.whopeo (mgr.24625) 2273 : cluster [DBG] pgmap v1280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:10 gibba011 bash[30791]: cluster 2021-03-10T14:35:08.241982+0000 mgr.gibba032.whopeo (mgr.24625) 2273 : cluster [DBG] pgmap v1280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:10 gibba011 bash[31774]: cluster 2021-03-10T14:35:08.241982+0000 mgr.gibba032.whopeo (mgr.24625) 2273 : cluster [DBG] pgmap v1280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:12 gibba032 bash[24004]: cluster 2021-03-10T14:35:10.243444+0000 mgr.gibba032.whopeo (mgr.24625) 2274 : cluster [DBG] pgmap v1281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:12 gibba011 bash[30791]: cluster 2021-03-10T14:35:10.243444+0000 mgr.gibba032.whopeo (mgr.24625) 2274 : cluster [DBG] pgmap v1281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:12 gibba011 bash[31774]: cluster 2021-03-10T14:35:10.243444+0000 mgr.gibba032.whopeo (mgr.24625) 2274 : cluster [DBG] pgmap v1281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:14 gibba032 bash[24004]: cluster 2021-03-10T14:35:12.244994+0000 mgr.gibba032.whopeo (mgr.24625) 2275 : cluster [DBG] pgmap v1282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:14 gibba011 bash[30791]: cluster 2021-03-10T14:35:12.244994+0000 mgr.gibba032.whopeo (mgr.24625) 2275 : cluster [DBG] pgmap v1282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:14 gibba011 bash[31774]: cluster 2021-03-10T14:35:12.244994+0000 mgr.gibba032.whopeo (mgr.24625) 2275 : cluster [DBG] pgmap v1282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:16.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:16 gibba032 bash[24004]: cluster 2021-03-10T14:35:14.246238+0000 mgr.gibba032.whopeo (mgr.24625) 2276 : cluster [DBG] pgmap v1283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:16 gibba011 bash[30791]: cluster 2021-03-10T14:35:14.246238+0000 mgr.gibba032.whopeo (mgr.24625) 2276 : cluster [DBG] pgmap v1283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:16 gibba011 bash[31774]: cluster 2021-03-10T14:35:14.246238+0000 mgr.gibba032.whopeo (mgr.24625) 2276 : cluster [DBG] pgmap v1283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:18 gibba032 bash[24004]: cluster 2021-03-10T14:35:16.247987+0000 mgr.gibba032.whopeo (mgr.24625) 2277 : cluster [DBG] pgmap v1284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:18 gibba011 bash[30791]: cluster 2021-03-10T14:35:16.247987+0000 mgr.gibba032.whopeo (mgr.24625) 2277 : cluster [DBG] pgmap v1284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:18 gibba011 bash[31774]: cluster 2021-03-10T14:35:16.247987+0000 mgr.gibba032.whopeo (mgr.24625) 2277 : cluster [DBG] pgmap v1284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:20 gibba032 bash[24004]: cluster 2021-03-10T14:35:18.248773+0000 mgr.gibba032.whopeo (mgr.24625) 2278 : cluster [DBG] pgmap v1285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:20 gibba032 bash[24004]: cephadm 2021-03-10T14:35:18.647919+0000 mgr.gibba032.whopeo (mgr.24625) 2279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:20 gibba011 bash[30791]: cluster 2021-03-10T14:35:18.248773+0000 mgr.gibba032.whopeo (mgr.24625) 2278 : cluster [DBG] pgmap v1285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:20 gibba011 bash[30791]: cephadm 2021-03-10T14:35:18.647919+0000 mgr.gibba032.whopeo (mgr.24625) 2279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:20 gibba011 bash[31774]: cluster 2021-03-10T14:35:18.248773+0000 mgr.gibba032.whopeo (mgr.24625) 2278 : cluster [DBG] pgmap v1285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:20 gibba011 bash[31774]: cephadm 2021-03-10T14:35:18.647919+0000 mgr.gibba032.whopeo (mgr.24625) 2279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:22 gibba032 bash[24004]: cluster 2021-03-10T14:35:20.249424+0000 mgr.gibba032.whopeo (mgr.24625) 2280 : cluster [DBG] pgmap v1286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:22 gibba011 bash[30791]: cluster 2021-03-10T14:35:20.249424+0000 mgr.gibba032.whopeo (mgr.24625) 2280 : cluster [DBG] pgmap v1286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:22 gibba011 bash[31774]: cluster 2021-03-10T14:35:20.249424+0000 mgr.gibba032.whopeo (mgr.24625) 2280 : cluster [DBG] pgmap v1286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:23.262 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:35:23.487 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:35:23.488 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (42m) 7m ago 47m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:35:23.488 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (47m) 6m ago 47m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:35:23.488 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (42m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:35:23.488 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (42m) 6m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:35:23.488 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (41m) 7m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:35:23.489 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (41m) 7m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:35:23.489 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (41m) 6m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:35:23.489 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (41m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:35:23.489 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (47m) 7m ago 47m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:35:23.489 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (47m) 6m ago 47m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:35:23.490 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (40m) 7m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:35:23.490 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (39m) 7m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:35:23.490 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (39m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:35:23.490 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (39m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:35:23.491 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (38m) 6m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:35:23.491 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (38m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:35:23.491 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (37m) 6m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:35:23.491 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (37m) 6m ago 47m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:35:23.491 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (46m) 6m ago 47m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:35:23.492 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 45m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:35:23.737 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:35:23.737 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:35:23.738 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:35:23.738 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:23.738 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:35:23.738 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:35:23.738 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:23.739 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:35:23.739 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:35:23.739 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:23.739 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:35:23.739 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:35:23.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:35:23.740 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:35:23.740 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:35:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:24 gibba032 bash[24004]: cluster 2021-03-10T14:35:22.250125+0000 mgr.gibba032.whopeo (mgr.24625) 2281 : cluster [DBG] pgmap v1287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:24 gibba032 bash[24004]: audit 2021-03-10T14:35:23.736680+0000 mon.c (mon.1) 46 : audit [DBG] from='client.? 172.21.2.111:0/3405349116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:24 gibba011 bash[30791]: cluster 2021-03-10T14:35:22.250125+0000 mgr.gibba032.whopeo (mgr.24625) 2281 : cluster [DBG] pgmap v1287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:24 gibba011 bash[30791]: audit 2021-03-10T14:35:23.736680+0000 mon.c (mon.1) 46 : audit [DBG] from='client.? 172.21.2.111:0/3405349116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:24 gibba011 bash[31774]: cluster 2021-03-10T14:35:22.250125+0000 mgr.gibba032.whopeo (mgr.24625) 2281 : cluster [DBG] pgmap v1287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:24 gibba011 bash[31774]: audit 2021-03-10T14:35:23.736680+0000 mon.c (mon.1) 46 : audit [DBG] from='client.? 172.21.2.111:0/3405349116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:25 gibba032 bash[24004]: audit 2021-03-10T14:35:23.250656+0000 mgr.gibba032.whopeo (mgr.24625) 2282 : audit [DBG] from='client.55570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:25 gibba032 bash[24004]: audit 2021-03-10T14:35:23.472282+0000 mgr.gibba032.whopeo (mgr.24625) 2283 : audit [DBG] from='client.55576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:25.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:25 gibba011 bash[30791]: audit 2021-03-10T14:35:23.250656+0000 mgr.gibba032.whopeo (mgr.24625) 2282 : audit [DBG] from='client.55570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:25.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:25 gibba011 bash[30791]: audit 2021-03-10T14:35:23.472282+0000 mgr.gibba032.whopeo (mgr.24625) 2283 : audit [DBG] from='client.55576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:25.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:25 gibba011 bash[31774]: audit 2021-03-10T14:35:23.250656+0000 mgr.gibba032.whopeo (mgr.24625) 2282 : audit [DBG] from='client.55570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:25.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:25 gibba011 bash[31774]: audit 2021-03-10T14:35:23.472282+0000 mgr.gibba032.whopeo (mgr.24625) 2283 : audit [DBG] from='client.55576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:26 gibba032 bash[24004]: cluster 2021-03-10T14:35:24.250719+0000 mgr.gibba032.whopeo (mgr.24625) 2284 : cluster [DBG] pgmap v1288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:26 gibba011 bash[30791]: cluster 2021-03-10T14:35:24.250719+0000 mgr.gibba032.whopeo (mgr.24625) 2284 : cluster [DBG] pgmap v1288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:26 gibba011 bash[31774]: cluster 2021-03-10T14:35:24.250719+0000 mgr.gibba032.whopeo (mgr.24625) 2284 : cluster [DBG] pgmap v1288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:27.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:27 gibba032 bash[24004]: cluster 2021-03-10T14:35:26.251507+0000 mgr.gibba032.whopeo (mgr.24625) 2285 : cluster [DBG] pgmap v1289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:27.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:27 gibba011 bash[30791]: cluster 2021-03-10T14:35:26.251507+0000 mgr.gibba032.whopeo (mgr.24625) 2285 : cluster [DBG] pgmap v1289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:27.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:27 gibba011 bash[31774]: cluster 2021-03-10T14:35:26.251507+0000 mgr.gibba032.whopeo (mgr.24625) 2285 : cluster [DBG] pgmap v1289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:29 gibba011 bash[30791]: cluster 2021-03-10T14:35:28.253096+0000 mgr.gibba032.whopeo (mgr.24625) 2286 : cluster [DBG] pgmap v1290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:29.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:29 gibba011 bash[31774]: cluster 2021-03-10T14:35:28.253096+0000 mgr.gibba032.whopeo (mgr.24625) 2286 : cluster [DBG] pgmap v1290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:29 gibba032 bash[24004]: cluster 2021-03-10T14:35:28.253096+0000 mgr.gibba032.whopeo (mgr.24625) 2286 : cluster [DBG] pgmap v1290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:32 gibba032 bash[24004]: cluster 2021-03-10T14:35:30.254609+0000 mgr.gibba032.whopeo (mgr.24625) 2287 : cluster [DBG] pgmap v1291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:32 gibba011 bash[30791]: cluster 2021-03-10T14:35:30.254609+0000 mgr.gibba032.whopeo (mgr.24625) 2287 : cluster [DBG] pgmap v1291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:32 gibba011 bash[31774]: cluster 2021-03-10T14:35:30.254609+0000 mgr.gibba032.whopeo (mgr.24625) 2287 : cluster [DBG] pgmap v1291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:34 gibba032 bash[24004]: cluster 2021-03-10T14:35:32.256489+0000 mgr.gibba032.whopeo (mgr.24625) 2288 : cluster [DBG] pgmap v1292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:34 gibba011 bash[30791]: cluster 2021-03-10T14:35:32.256489+0000 mgr.gibba032.whopeo (mgr.24625) 2288 : cluster [DBG] pgmap v1292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:34 gibba011 bash[31774]: cluster 2021-03-10T14:35:32.256489+0000 mgr.gibba032.whopeo (mgr.24625) 2288 : cluster [DBG] pgmap v1292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:35 gibba032 bash[24004]: cephadm 2021-03-10T14:35:33.660243+0000 mgr.gibba032.whopeo (mgr.24625) 2289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:35 gibba011 bash[30791]: cephadm 2021-03-10T14:35:33.660243+0000 mgr.gibba032.whopeo (mgr.24625) 2289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:35 gibba011 bash[31774]: cephadm 2021-03-10T14:35:33.660243+0000 mgr.gibba032.whopeo (mgr.24625) 2289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:36 gibba032 bash[24004]: cluster 2021-03-10T14:35:34.257616+0000 mgr.gibba032.whopeo (mgr.24625) 2290 : cluster [DBG] pgmap v1293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:36 gibba011 bash[30791]: cluster 2021-03-10T14:35:34.257616+0000 mgr.gibba032.whopeo (mgr.24625) 2290 : cluster [DBG] pgmap v1293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:36 gibba011 bash[31774]: cluster 2021-03-10T14:35:34.257616+0000 mgr.gibba032.whopeo (mgr.24625) 2290 : cluster [DBG] pgmap v1293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:38 gibba032 bash[24004]: cluster 2021-03-10T14:35:36.259320+0000 mgr.gibba032.whopeo (mgr.24625) 2291 : cluster [DBG] pgmap v1294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:38 gibba011 bash[30791]: cluster 2021-03-10T14:35:36.259320+0000 mgr.gibba032.whopeo (mgr.24625) 2291 : cluster [DBG] pgmap v1294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:38 gibba011 bash[31774]: cluster 2021-03-10T14:35:36.259320+0000 mgr.gibba032.whopeo (mgr.24625) 2291 : cluster [DBG] pgmap v1294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:40 gibba032 bash[24004]: cluster 2021-03-10T14:35:38.261148+0000 mgr.gibba032.whopeo (mgr.24625) 2292 : cluster [DBG] pgmap v1295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:40 gibba011 bash[30791]: cluster 2021-03-10T14:35:38.261148+0000 mgr.gibba032.whopeo (mgr.24625) 2292 : cluster [DBG] pgmap v1295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:40 gibba011 bash[31774]: cluster 2021-03-10T14:35:38.261148+0000 mgr.gibba032.whopeo (mgr.24625) 2292 : cluster [DBG] pgmap v1295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:42 gibba032 bash[24004]: cluster 2021-03-10T14:35:40.262756+0000 mgr.gibba032.whopeo (mgr.24625) 2293 : cluster [DBG] pgmap v1296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:42 gibba011 bash[30791]: cluster 2021-03-10T14:35:40.262756+0000 mgr.gibba032.whopeo (mgr.24625) 2293 : cluster [DBG] pgmap v1296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:42 gibba011 bash[31774]: cluster 2021-03-10T14:35:40.262756+0000 mgr.gibba032.whopeo (mgr.24625) 2293 : cluster [DBG] pgmap v1296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:44 gibba032 bash[24004]: cluster 2021-03-10T14:35:42.264559+0000 mgr.gibba032.whopeo (mgr.24625) 2294 : cluster [DBG] pgmap v1297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:44 gibba011 bash[30791]: cluster 2021-03-10T14:35:42.264559+0000 mgr.gibba032.whopeo (mgr.24625) 2294 : cluster [DBG] pgmap v1297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:44 gibba011 bash[31774]: cluster 2021-03-10T14:35:42.264559+0000 mgr.gibba032.whopeo (mgr.24625) 2294 : cluster [DBG] pgmap v1297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:46 gibba032 bash[24004]: cluster 2021-03-10T14:35:44.265685+0000 mgr.gibba032.whopeo (mgr.24625) 2295 : cluster [DBG] pgmap v1298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:46 gibba011 bash[30791]: cluster 2021-03-10T14:35:44.265685+0000 mgr.gibba032.whopeo (mgr.24625) 2295 : cluster [DBG] pgmap v1298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:46 gibba011 bash[31774]: cluster 2021-03-10T14:35:44.265685+0000 mgr.gibba032.whopeo (mgr.24625) 2295 : cluster [DBG] pgmap v1298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:48 gibba032 bash[24004]: cluster 2021-03-10T14:35:46.267166+0000 mgr.gibba032.whopeo (mgr.24625) 2296 : cluster [DBG] pgmap v1299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:48 gibba011 bash[30791]: cluster 2021-03-10T14:35:46.267166+0000 mgr.gibba032.whopeo (mgr.24625) 2296 : cluster [DBG] pgmap v1299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:48 gibba011 bash[31774]: cluster 2021-03-10T14:35:46.267166+0000 mgr.gibba032.whopeo (mgr.24625) 2296 : cluster [DBG] pgmap v1299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cluster 2021-03-10T14:35:48.268986+0000 mgr.gibba032.whopeo (mgr.24625) 2297 : cluster [DBG] pgmap v1300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.676948+0000 mgr.gibba032.whopeo (mgr.24625) 2298 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:35:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.678512+0000 mgr.gibba032.whopeo (mgr.24625) 2299 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:35:50.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.896776+0000 mgr.gibba032.whopeo (mgr.24625) 2300 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:35:50.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.896867+0000 mon.a (mon.0) 1211 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:35:50.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.898465+0000 mgr.gibba032.whopeo (mgr.24625) 2301 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:35:50.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.899633+0000 mgr.gibba032.whopeo (mgr.24625) 2302 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:35:50.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.899723+0000 mgr.gibba032.whopeo (mgr.24625) 2303 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:35:50.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.900122+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:35:50.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.900867+0000 mgr.gibba032.whopeo (mgr.24625) 2304 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:35:50.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.900904+0000 mgr.gibba032.whopeo (mgr.24625) 2305 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:35:50.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.901437+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.901937+0000 mgr.gibba032.whopeo (mgr.24625) 2306 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:35:50.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.902690+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.902881+0000 mgr.gibba032.whopeo (mgr.24625) 2307 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:35:50.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.902919+0000 mgr.gibba032.whopeo (mgr.24625) 2308 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:35:50.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.903789+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.904081+0000 mgr.gibba032.whopeo (mgr.24625) 2309 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:35:50.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.904436+0000 mgr.gibba032.whopeo (mgr.24625) 2310 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:35:50.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.905856+0000 mgr.gibba032.whopeo (mgr.24625) 2311 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:35:50.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.905894+0000 mgr.gibba032.whopeo (mgr.24625) 2312 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:35:50.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.905953+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: cephadm 2021-03-10T14:35:48.905998+0000 mgr.gibba032.whopeo (mgr.24625) 2313 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:50.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:49 gibba032 bash[24004]: audit 2021-03-10T14:35:48.907401+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cluster 2021-03-10T14:35:48.268986+0000 mgr.gibba032.whopeo (mgr.24625) 2297 : cluster [DBG] pgmap v1300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.676948+0000 mgr.gibba032.whopeo (mgr.24625) 2298 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:35:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.678512+0000 mgr.gibba032.whopeo (mgr.24625) 2299 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:35:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.896776+0000 mgr.gibba032.whopeo (mgr.24625) 2300 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:35:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.896867+0000 mon.a (mon.0) 1211 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:35:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.898465+0000 mgr.gibba032.whopeo (mgr.24625) 2301 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:35:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.899633+0000 mgr.gibba032.whopeo (mgr.24625) 2302 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:35:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.899723+0000 mgr.gibba032.whopeo (mgr.24625) 2303 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:35:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.900122+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:35:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.900867+0000 mgr.gibba032.whopeo (mgr.24625) 2304 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:35:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.900904+0000 mgr.gibba032.whopeo (mgr.24625) 2305 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:35:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.901437+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.901937+0000 mgr.gibba032.whopeo (mgr.24625) 2306 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:35:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.902690+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.902881+0000 mgr.gibba032.whopeo (mgr.24625) 2307 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:35:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.902919+0000 mgr.gibba032.whopeo (mgr.24625) 2308 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:35:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.903789+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.904081+0000 mgr.gibba032.whopeo (mgr.24625) 2309 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:35:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.904436+0000 mgr.gibba032.whopeo (mgr.24625) 2310 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:35:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.905856+0000 mgr.gibba032.whopeo (mgr.24625) 2311 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:35:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.905894+0000 mgr.gibba032.whopeo (mgr.24625) 2312 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:35:50.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cluster 2021-03-10T14:35:48.268986+0000 mgr.gibba032.whopeo (mgr.24625) 2297 : cluster [DBG] pgmap v1300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:50.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.676948+0000 mgr.gibba032.whopeo (mgr.24625) 2298 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:35:50.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.678512+0000 mgr.gibba032.whopeo (mgr.24625) 2299 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:35:50.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.896776+0000 mgr.gibba032.whopeo (mgr.24625) 2300 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:35:50.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.896867+0000 mon.a (mon.0) 1211 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:35:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.898465+0000 mgr.gibba032.whopeo (mgr.24625) 2301 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:35:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.899633+0000 mgr.gibba032.whopeo (mgr.24625) 2302 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:35:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.899723+0000 mgr.gibba032.whopeo (mgr.24625) 2303 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:35:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.900122+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:35:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.900867+0000 mgr.gibba032.whopeo (mgr.24625) 2304 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:35:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.900904+0000 mgr.gibba032.whopeo (mgr.24625) 2305 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:35:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.901437+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.901937+0000 mgr.gibba032.whopeo (mgr.24625) 2306 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:35:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.902690+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.902881+0000 mgr.gibba032.whopeo (mgr.24625) 2307 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:35:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.902919+0000 mgr.gibba032.whopeo (mgr.24625) 2308 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:35:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.903789+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.904081+0000 mgr.gibba032.whopeo (mgr.24625) 2309 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:35:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.904436+0000 mgr.gibba032.whopeo (mgr.24625) 2310 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:35:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.905856+0000 mgr.gibba032.whopeo (mgr.24625) 2311 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:35:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.905894+0000 mgr.gibba032.whopeo (mgr.24625) 2312 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:35:50.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.905953+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: cephadm 2021-03-10T14:35:48.905998+0000 mgr.gibba032.whopeo (mgr.24625) 2313 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:50.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[30791]: audit 2021-03-10T14:35:48.907401+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.905953+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: cephadm 2021-03-10T14:35:48.905998+0000 mgr.gibba032.whopeo (mgr.24625) 2313 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:35:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:49 gibba011 bash[31774]: audit 2021-03-10T14:35:48.907401+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:52 gibba032 bash[24004]: cluster 2021-03-10T14:35:50.270363+0000 mgr.gibba032.whopeo (mgr.24625) 2314 : cluster [DBG] pgmap v1301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:52 gibba011 bash[30791]: cluster 2021-03-10T14:35:50.270363+0000 mgr.gibba032.whopeo (mgr.24625) 2314 : cluster [DBG] pgmap v1301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:52 gibba011 bash[31774]: cluster 2021-03-10T14:35:50.270363+0000 mgr.gibba032.whopeo (mgr.24625) 2314 : cluster [DBG] pgmap v1301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:53.977 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:35:54.239 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:35:54.239 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (42m) 8m ago 47m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:35:54.239 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (47m) 7m ago 47m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (42m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (43m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (42m) 8m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (42m) 8m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (41m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:35:54.240 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (41m) 8m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:35:54.241 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (48m) 8m ago 48m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:35:54.241 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (48m) 7m ago 48m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:35:54.241 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (40m) 8m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:35:54.241 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (40m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:35:54.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (39m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:35:54.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (39m) 8m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:35:54.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (39m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:35:54.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (38m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:35:54.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (38m) 7m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:35:54.243 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (37m) 7m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:35:54.243 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (47m) 7m ago 48m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:35:54.243 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 45m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:35:54.494 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:35:54.495 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:35:54.495 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:35:54.495 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:54.495 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:35:54.496 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:35:54.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:54.496 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:35:54.496 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:35:54.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:35:54.497 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:35:54.497 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:35:54.497 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:35:54.497 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:35:54.497 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:35:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:54 gibba032 bash[24004]: cluster 2021-03-10T14:35:52.272160+0000 mgr.gibba032.whopeo (mgr.24625) 2315 : cluster [DBG] pgmap v1302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:54 gibba011 bash[30791]: cluster 2021-03-10T14:35:52.272160+0000 mgr.gibba032.whopeo (mgr.24625) 2315 : cluster [DBG] pgmap v1302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:54 gibba011 bash[31774]: cluster 2021-03-10T14:35:52.272160+0000 mgr.gibba032.whopeo (mgr.24625) 2315 : cluster [DBG] pgmap v1302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:55 gibba032 bash[24004]: audit 2021-03-10T14:35:53.964899+0000 mgr.gibba032.whopeo (mgr.24625) 2316 : audit [DBG] from='client.55588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:55 gibba032 bash[24004]: audit 2021-03-10T14:35:54.194066+0000 mgr.gibba032.whopeo (mgr.24625) 2317 : audit [DBG] from='client.55594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:55 gibba032 bash[24004]: audit 2021-03-10T14:35:54.493747+0000 mon.a (mon.0) 1218 : audit [DBG] from='client.? 172.21.2.111:0/3399988170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[30791]: audit 2021-03-10T14:35:53.964899+0000 mgr.gibba032.whopeo (mgr.24625) 2316 : audit [DBG] from='client.55588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[30791]: audit 2021-03-10T14:35:54.194066+0000 mgr.gibba032.whopeo (mgr.24625) 2317 : audit [DBG] from='client.55594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[30791]: audit 2021-03-10T14:35:54.493747+0000 mon.a (mon.0) 1218 : audit [DBG] from='client.? 172.21.2.111:0/3399988170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[31774]: audit 2021-03-10T14:35:53.964899+0000 mgr.gibba032.whopeo (mgr.24625) 2316 : audit [DBG] from='client.55588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[31774]: audit 2021-03-10T14:35:54.194066+0000 mgr.gibba032.whopeo (mgr.24625) 2317 : audit [DBG] from='client.55594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:35:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:55 gibba011 bash[31774]: audit 2021-03-10T14:35:54.493747+0000 mon.a (mon.0) 1218 : audit [DBG] from='client.? 172.21.2.111:0/3399988170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:35:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:56 gibba032 bash[24004]: cluster 2021-03-10T14:35:54.273533+0000 mgr.gibba032.whopeo (mgr.24625) 2318 : cluster [DBG] pgmap v1303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:56 gibba011 bash[30791]: cluster 2021-03-10T14:35:54.273533+0000 mgr.gibba032.whopeo (mgr.24625) 2318 : cluster [DBG] pgmap v1303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:56 gibba011 bash[31774]: cluster 2021-03-10T14:35:54.273533+0000 mgr.gibba032.whopeo (mgr.24625) 2318 : cluster [DBG] pgmap v1303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.279630+0000 mon.a (mon.0) 1219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:35:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.280267+0000 mon.a (mon.0) 1220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:35:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.288522+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:35:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.289296+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:35:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.289972+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:35:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:57 gibba032 bash[24004]: audit 2021-03-10T14:35:56.290647+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:35:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.279630+0000 mon.a (mon.0) 1219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:35:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.280267+0000 mon.a (mon.0) 1220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:35:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.288522+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:35:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.289296+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:35:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.289972+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:35:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[30791]: audit 2021-03-10T14:35:56.290647+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:35:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.279630+0000 mon.a (mon.0) 1219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:35:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.280267+0000 mon.a (mon.0) 1220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:35:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.288522+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:35:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.289296+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:35:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.289972+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:35:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:57 gibba011 bash[31774]: audit 2021-03-10T14:35:56.290647+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:35:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:35:58 gibba032 bash[24004]: cluster 2021-03-10T14:35:56.274081+0000 mgr.gibba032.whopeo (mgr.24625) 2319 : cluster [DBG] pgmap v1304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:35:58 gibba011 bash[30791]: cluster 2021-03-10T14:35:56.274081+0000 mgr.gibba032.whopeo (mgr.24625) 2319 : cluster [DBG] pgmap v1304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:35:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:35:58 gibba011 bash[31774]: cluster 2021-03-10T14:35:56.274081+0000 mgr.gibba032.whopeo (mgr.24625) 2319 : cluster [DBG] pgmap v1304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:00 gibba032 bash[24004]: cluster 2021-03-10T14:35:58.275742+0000 mgr.gibba032.whopeo (mgr.24625) 2320 : cluster [DBG] pgmap v1305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:00 gibba011 bash[30791]: cluster 2021-03-10T14:35:58.275742+0000 mgr.gibba032.whopeo (mgr.24625) 2320 : cluster [DBG] pgmap v1305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:00 gibba011 bash[31774]: cluster 2021-03-10T14:35:58.275742+0000 mgr.gibba032.whopeo (mgr.24625) 2320 : cluster [DBG] pgmap v1305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:02 gibba032 bash[24004]: cluster 2021-03-10T14:36:00.277353+0000 mgr.gibba032.whopeo (mgr.24625) 2321 : cluster [DBG] pgmap v1306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:02 gibba011 bash[30791]: cluster 2021-03-10T14:36:00.277353+0000 mgr.gibba032.whopeo (mgr.24625) 2321 : cluster [DBG] pgmap v1306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:02 gibba011 bash[31774]: cluster 2021-03-10T14:36:00.277353+0000 mgr.gibba032.whopeo (mgr.24625) 2321 : cluster [DBG] pgmap v1306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:04.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:04 gibba032 bash[24004]: cluster 2021-03-10T14:36:02.279004+0000 mgr.gibba032.whopeo (mgr.24625) 2322 : cluster [DBG] pgmap v1307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:04 gibba011 bash[30791]: cluster 2021-03-10T14:36:02.279004+0000 mgr.gibba032.whopeo (mgr.24625) 2322 : cluster [DBG] pgmap v1307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:04 gibba011 bash[31774]: cluster 2021-03-10T14:36:02.279004+0000 mgr.gibba032.whopeo (mgr.24625) 2322 : cluster [DBG] pgmap v1307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:05 gibba032 bash[24004]: cephadm 2021-03-10T14:36:03.915484+0000 mgr.gibba032.whopeo (mgr.24625) 2323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:05 gibba011 bash[30791]: cephadm 2021-03-10T14:36:03.915484+0000 mgr.gibba032.whopeo (mgr.24625) 2323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:05 gibba011 bash[31774]: cephadm 2021-03-10T14:36:03.915484+0000 mgr.gibba032.whopeo (mgr.24625) 2323 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:06 gibba032 bash[24004]: cluster 2021-03-10T14:36:04.280275+0000 mgr.gibba032.whopeo (mgr.24625) 2324 : cluster [DBG] pgmap v1308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:06 gibba011 bash[30791]: cluster 2021-03-10T14:36:04.280275+0000 mgr.gibba032.whopeo (mgr.24625) 2324 : cluster [DBG] pgmap v1308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:06 gibba011 bash[31774]: cluster 2021-03-10T14:36:04.280275+0000 mgr.gibba032.whopeo (mgr.24625) 2324 : cluster [DBG] pgmap v1308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:08 gibba032 bash[24004]: cluster 2021-03-10T14:36:06.282033+0000 mgr.gibba032.whopeo (mgr.24625) 2325 : cluster [DBG] pgmap v1309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:08 gibba011 bash[30791]: cluster 2021-03-10T14:36:06.282033+0000 mgr.gibba032.whopeo (mgr.24625) 2325 : cluster [DBG] pgmap v1309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:08 gibba011 bash[31774]: cluster 2021-03-10T14:36:06.282033+0000 mgr.gibba032.whopeo (mgr.24625) 2325 : cluster [DBG] pgmap v1309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:10 gibba032 bash[24004]: cluster 2021-03-10T14:36:08.283811+0000 mgr.gibba032.whopeo (mgr.24625) 2326 : cluster [DBG] pgmap v1310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:10 gibba011 bash[30791]: cluster 2021-03-10T14:36:08.283811+0000 mgr.gibba032.whopeo (mgr.24625) 2326 : cluster [DBG] pgmap v1310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:10.716 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:10 gibba011 bash[31774]: cluster 2021-03-10T14:36:08.283811+0000 mgr.gibba032.whopeo (mgr.24625) 2326 : cluster [DBG] pgmap v1310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:12 gibba032 bash[24004]: cluster 2021-03-10T14:36:10.285399+0000 mgr.gibba032.whopeo (mgr.24625) 2327 : cluster [DBG] pgmap v1311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:12 gibba011 bash[30791]: cluster 2021-03-10T14:36:10.285399+0000 mgr.gibba032.whopeo (mgr.24625) 2327 : cluster [DBG] pgmap v1311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:12 gibba011 bash[31774]: cluster 2021-03-10T14:36:10.285399+0000 mgr.gibba032.whopeo (mgr.24625) 2327 : cluster [DBG] pgmap v1311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:14 gibba032 bash[24004]: cluster 2021-03-10T14:36:12.286656+0000 mgr.gibba032.whopeo (mgr.24625) 2328 : cluster [DBG] pgmap v1312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:14 gibba011 bash[30791]: cluster 2021-03-10T14:36:12.286656+0000 mgr.gibba032.whopeo (mgr.24625) 2328 : cluster [DBG] pgmap v1312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:14 gibba011 bash[31774]: cluster 2021-03-10T14:36:12.286656+0000 mgr.gibba032.whopeo (mgr.24625) 2328 : cluster [DBG] pgmap v1312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:16 gibba032 bash[24004]: cluster 2021-03-10T14:36:14.287659+0000 mgr.gibba032.whopeo (mgr.24625) 2329 : cluster [DBG] pgmap v1313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:16 gibba011 bash[30791]: cluster 2021-03-10T14:36:14.287659+0000 mgr.gibba032.whopeo (mgr.24625) 2329 : cluster [DBG] pgmap v1313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:16 gibba011 bash[31774]: cluster 2021-03-10T14:36:14.287659+0000 mgr.gibba032.whopeo (mgr.24625) 2329 : cluster [DBG] pgmap v1313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:18 gibba032 bash[24004]: cluster 2021-03-10T14:36:16.289275+0000 mgr.gibba032.whopeo (mgr.24625) 2330 : cluster [DBG] pgmap v1314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:18 gibba011 bash[30791]: cluster 2021-03-10T14:36:16.289275+0000 mgr.gibba032.whopeo (mgr.24625) 2330 : cluster [DBG] pgmap v1314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:18 gibba011 bash[31774]: cluster 2021-03-10T14:36:16.289275+0000 mgr.gibba032.whopeo (mgr.24625) 2330 : cluster [DBG] pgmap v1314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:20 gibba032 bash[24004]: cluster 2021-03-10T14:36:18.290403+0000 mgr.gibba032.whopeo (mgr.24625) 2331 : cluster [DBG] pgmap v1315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:20 gibba032 bash[24004]: cephadm 2021-03-10T14:36:18.931486+0000 mgr.gibba032.whopeo (mgr.24625) 2332 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:20 gibba011 bash[30791]: cluster 2021-03-10T14:36:18.290403+0000 mgr.gibba032.whopeo (mgr.24625) 2331 : cluster [DBG] pgmap v1315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:20 gibba011 bash[30791]: cephadm 2021-03-10T14:36:18.931486+0000 mgr.gibba032.whopeo (mgr.24625) 2332 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:20 gibba011 bash[31774]: cluster 2021-03-10T14:36:18.290403+0000 mgr.gibba032.whopeo (mgr.24625) 2331 : cluster [DBG] pgmap v1315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:20 gibba011 bash[31774]: cephadm 2021-03-10T14:36:18.931486+0000 mgr.gibba032.whopeo (mgr.24625) 2332 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:22 gibba032 bash[24004]: cluster 2021-03-10T14:36:20.291246+0000 mgr.gibba032.whopeo (mgr.24625) 2333 : cluster [DBG] pgmap v1316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:22 gibba011 bash[30791]: cluster 2021-03-10T14:36:20.291246+0000 mgr.gibba032.whopeo (mgr.24625) 2333 : cluster [DBG] pgmap v1316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:22 gibba011 bash[31774]: cluster 2021-03-10T14:36:20.291246+0000 mgr.gibba032.whopeo (mgr.24625) 2333 : cluster [DBG] pgmap v1316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:24 gibba032 bash[24004]: cluster 2021-03-10T14:36:22.293015+0000 mgr.gibba032.whopeo (mgr.24625) 2334 : cluster [DBG] pgmap v1317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:24 gibba011 bash[30791]: cluster 2021-03-10T14:36:22.293015+0000 mgr.gibba032.whopeo (mgr.24625) 2334 : cluster [DBG] pgmap v1317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:24 gibba011 bash[31774]: cluster 2021-03-10T14:36:22.293015+0000 mgr.gibba032.whopeo (mgr.24625) 2334 : cluster [DBG] pgmap v1317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:24.738 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:36:24.966 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:36:24.966 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (43m) 8m ago 48m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:36:24.966 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (48m) 7m ago 48m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:36:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (43m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:36:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (43m) 7m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:36:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (42m) 8m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:36:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (42m) 8m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:36:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (42m) 7m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:36:24.968 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (42m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:36:24.968 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (48m) 8m ago 48m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:36:24.968 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (48m) 7m ago 48m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:36:24.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (41m) 8m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:36:24.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (40m) 8m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:36:24.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (40m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:36:24.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (40m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:36:24.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (39m) 7m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:36:24.969 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (39m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:36:24.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (38m) 7m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:36:24.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (38m) 7m ago 48m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:36:24.970 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (47m) 7m ago 48m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:36:24.970 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 46m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:36:25.216 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:36:25.217 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:36:25.217 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:36:25.217 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:25.217 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:36:25.217 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:36:25.218 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:25.218 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:36:25.218 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:36:25.218 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:25.218 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:36:25.219 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:36:25.219 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:36:25.219 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:36:25.219 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:36:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:26 gibba032 bash[24004]: cluster 2021-03-10T14:36:24.294399+0000 mgr.gibba032.whopeo (mgr.24625) 2335 : cluster [DBG] pgmap v1318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:26 gibba032 bash[24004]: audit 2021-03-10T14:36:24.725962+0000 mgr.gibba032.whopeo (mgr.24625) 2336 : audit [DBG] from='client.55606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:26 gibba032 bash[24004]: audit 2021-03-10T14:36:24.950131+0000 mgr.gibba032.whopeo (mgr.24625) 2337 : audit [DBG] from='client.45627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:26 gibba032 bash[24004]: audit 2021-03-10T14:36:25.215847+0000 mon.a (mon.0) 1225 : audit [DBG] from='client.? 172.21.2.111:0/2266313854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[30791]: cluster 2021-03-10T14:36:24.294399+0000 mgr.gibba032.whopeo (mgr.24625) 2335 : cluster [DBG] pgmap v1318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[30791]: audit 2021-03-10T14:36:24.725962+0000 mgr.gibba032.whopeo (mgr.24625) 2336 : audit [DBG] from='client.55606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[30791]: audit 2021-03-10T14:36:24.950131+0000 mgr.gibba032.whopeo (mgr.24625) 2337 : audit [DBG] from='client.45627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[30791]: audit 2021-03-10T14:36:25.215847+0000 mon.a (mon.0) 1225 : audit [DBG] from='client.? 172.21.2.111:0/2266313854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[31774]: cluster 2021-03-10T14:36:24.294399+0000 mgr.gibba032.whopeo (mgr.24625) 2335 : cluster [DBG] pgmap v1318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[31774]: audit 2021-03-10T14:36:24.725962+0000 mgr.gibba032.whopeo (mgr.24625) 2336 : audit [DBG] from='client.55606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[31774]: audit 2021-03-10T14:36:24.950131+0000 mgr.gibba032.whopeo (mgr.24625) 2337 : audit [DBG] from='client.45627 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:26 gibba011 bash[31774]: audit 2021-03-10T14:36:25.215847+0000 mon.a (mon.0) 1225 : audit [DBG] from='client.? 172.21.2.111:0/2266313854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:28 gibba032 bash[24004]: cluster 2021-03-10T14:36:26.295891+0000 mgr.gibba032.whopeo (mgr.24625) 2338 : cluster [DBG] pgmap v1319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:28 gibba011 bash[30791]: cluster 2021-03-10T14:36:26.295891+0000 mgr.gibba032.whopeo (mgr.24625) 2338 : cluster [DBG] pgmap v1319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:28 gibba011 bash[31774]: cluster 2021-03-10T14:36:26.295891+0000 mgr.gibba032.whopeo (mgr.24625) 2338 : cluster [DBG] pgmap v1319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:30 gibba032 bash[24004]: cluster 2021-03-10T14:36:28.297597+0000 mgr.gibba032.whopeo (mgr.24625) 2339 : cluster [DBG] pgmap v1320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:30 gibba011 bash[30791]: cluster 2021-03-10T14:36:28.297597+0000 mgr.gibba032.whopeo (mgr.24625) 2339 : cluster [DBG] pgmap v1320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:30 gibba011 bash[31774]: cluster 2021-03-10T14:36:28.297597+0000 mgr.gibba032.whopeo (mgr.24625) 2339 : cluster [DBG] pgmap v1320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:32 gibba032 bash[24004]: cluster 2021-03-10T14:36:30.299057+0000 mgr.gibba032.whopeo (mgr.24625) 2340 : cluster [DBG] pgmap v1321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:32 gibba011 bash[30791]: cluster 2021-03-10T14:36:30.299057+0000 mgr.gibba032.whopeo (mgr.24625) 2340 : cluster [DBG] pgmap v1321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:32 gibba011 bash[31774]: cluster 2021-03-10T14:36:30.299057+0000 mgr.gibba032.whopeo (mgr.24625) 2340 : cluster [DBG] pgmap v1321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:34 gibba032 bash[24004]: cluster 2021-03-10T14:36:32.300868+0000 mgr.gibba032.whopeo (mgr.24625) 2341 : cluster [DBG] pgmap v1322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:34 gibba011 bash[30791]: cluster 2021-03-10T14:36:32.300868+0000 mgr.gibba032.whopeo (mgr.24625) 2341 : cluster [DBG] pgmap v1322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:34 gibba011 bash[31774]: cluster 2021-03-10T14:36:32.300868+0000 mgr.gibba032.whopeo (mgr.24625) 2341 : cluster [DBG] pgmap v1322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:35 gibba032 bash[24004]: cephadm 2021-03-10T14:36:33.947476+0000 mgr.gibba032.whopeo (mgr.24625) 2342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:35 gibba011 bash[30791]: cephadm 2021-03-10T14:36:33.947476+0000 mgr.gibba032.whopeo (mgr.24625) 2342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:35 gibba011 bash[31774]: cephadm 2021-03-10T14:36:33.947476+0000 mgr.gibba032.whopeo (mgr.24625) 2342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:36 gibba032 bash[24004]: cluster 2021-03-10T14:36:34.302013+0000 mgr.gibba032.whopeo (mgr.24625) 2343 : cluster [DBG] pgmap v1323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:36 gibba011 bash[30791]: cluster 2021-03-10T14:36:34.302013+0000 mgr.gibba032.whopeo (mgr.24625) 2343 : cluster [DBG] pgmap v1323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:36 gibba011 bash[31774]: cluster 2021-03-10T14:36:34.302013+0000 mgr.gibba032.whopeo (mgr.24625) 2343 : cluster [DBG] pgmap v1323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:38 gibba032 bash[24004]: cluster 2021-03-10T14:36:36.303617+0000 mgr.gibba032.whopeo (mgr.24625) 2344 : cluster [DBG] pgmap v1324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:38 gibba011 bash[30791]: cluster 2021-03-10T14:36:36.303617+0000 mgr.gibba032.whopeo (mgr.24625) 2344 : cluster [DBG] pgmap v1324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:38 gibba011 bash[31774]: cluster 2021-03-10T14:36:36.303617+0000 mgr.gibba032.whopeo (mgr.24625) 2344 : cluster [DBG] pgmap v1324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:40 gibba032 bash[24004]: cluster 2021-03-10T14:36:38.305149+0000 mgr.gibba032.whopeo (mgr.24625) 2345 : cluster [DBG] pgmap v1325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:40 gibba011 bash[30791]: cluster 2021-03-10T14:36:38.305149+0000 mgr.gibba032.whopeo (mgr.24625) 2345 : cluster [DBG] pgmap v1325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:40 gibba011 bash[31774]: cluster 2021-03-10T14:36:38.305149+0000 mgr.gibba032.whopeo (mgr.24625) 2345 : cluster [DBG] pgmap v1325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:42 gibba032 bash[24004]: cluster 2021-03-10T14:36:40.306666+0000 mgr.gibba032.whopeo (mgr.24625) 2346 : cluster [DBG] pgmap v1326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:42 gibba011 bash[30791]: cluster 2021-03-10T14:36:40.306666+0000 mgr.gibba032.whopeo (mgr.24625) 2346 : cluster [DBG] pgmap v1326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:42 gibba011 bash[31774]: cluster 2021-03-10T14:36:40.306666+0000 mgr.gibba032.whopeo (mgr.24625) 2346 : cluster [DBG] pgmap v1326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:44 gibba032 bash[24004]: cluster 2021-03-10T14:36:42.308263+0000 mgr.gibba032.whopeo (mgr.24625) 2347 : cluster [DBG] pgmap v1327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:44 gibba011 bash[30791]: cluster 2021-03-10T14:36:42.308263+0000 mgr.gibba032.whopeo (mgr.24625) 2347 : cluster [DBG] pgmap v1327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:44 gibba011 bash[31774]: cluster 2021-03-10T14:36:42.308263+0000 mgr.gibba032.whopeo (mgr.24625) 2347 : cluster [DBG] pgmap v1327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:46 gibba032 bash[24004]: cluster 2021-03-10T14:36:44.309481+0000 mgr.gibba032.whopeo (mgr.24625) 2348 : cluster [DBG] pgmap v1328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:46 gibba011 bash[30791]: cluster 2021-03-10T14:36:44.309481+0000 mgr.gibba032.whopeo (mgr.24625) 2348 : cluster [DBG] pgmap v1328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:46 gibba011 bash[31774]: cluster 2021-03-10T14:36:44.309481+0000 mgr.gibba032.whopeo (mgr.24625) 2348 : cluster [DBG] pgmap v1328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:48 gibba032 bash[24004]: cluster 2021-03-10T14:36:46.311158+0000 mgr.gibba032.whopeo (mgr.24625) 2349 : cluster [DBG] pgmap v1329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:48 gibba011 bash[30791]: cluster 2021-03-10T14:36:46.311158+0000 mgr.gibba032.whopeo (mgr.24625) 2349 : cluster [DBG] pgmap v1329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:48 gibba011 bash[31774]: cluster 2021-03-10T14:36:46.311158+0000 mgr.gibba032.whopeo (mgr.24625) 2349 : cluster [DBG] pgmap v1329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cluster 2021-03-10T14:36:48.312793+0000 mgr.gibba032.whopeo (mgr.24625) 2350 : cluster [DBG] pgmap v1330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:48.953711+0000 mgr.gibba032.whopeo (mgr.24625) 2351 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:36:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:48.955277+0000 mgr.gibba032.whopeo (mgr.24625) 2352 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:36:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.179032+0000 mgr.gibba032.whopeo (mgr.24625) 2353 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:36:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.179192+0000 mon.a (mon.0) 1226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:36:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.180815+0000 mgr.gibba032.whopeo (mgr.24625) 2354 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:36:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.182221+0000 mgr.gibba032.whopeo (mgr.24625) 2355 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:36:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.182260+0000 mgr.gibba032.whopeo (mgr.24625) 2356 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:36:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.182327+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:36:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.183681+0000 mgr.gibba032.whopeo (mgr.24625) 2357 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:36:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.183720+0000 mgr.gibba032.whopeo (mgr.24625) 2358 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:36:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.183854+0000 mon.a (mon.0) 1228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.185079+0000 mgr.gibba032.whopeo (mgr.24625) 2359 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:36:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.185310+0000 mon.a (mon.0) 1229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.186330+0000 mgr.gibba032.whopeo (mgr.24625) 2360 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:36:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.186365+0000 mgr.gibba032.whopeo (mgr.24625) 2361 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:36:50.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.186713+0000 mon.a (mon.0) 1230 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.187819+0000 mgr.gibba032.whopeo (mgr.24625) 2362 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:36:50.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.188193+0000 mgr.gibba032.whopeo (mgr.24625) 2363 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:36:50.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.189204+0000 mgr.gibba032.whopeo (mgr.24625) 2364 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:36:50.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.189242+0000 mgr.gibba032.whopeo (mgr.24625) 2365 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:36:50.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: cephadm 2021-03-10T14:36:49.189346+0000 mgr.gibba032.whopeo (mgr.24625) 2366 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:50.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.189476+0000 mon.a (mon.0) 1231 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:50 gibba032 bash[24004]: audit 2021-03-10T14:36:49.191009+0000 mon.a (mon.0) 1232 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cluster 2021-03-10T14:36:48.312793+0000 mgr.gibba032.whopeo (mgr.24625) 2350 : cluster [DBG] pgmap v1330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:48.953711+0000 mgr.gibba032.whopeo (mgr.24625) 2351 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:48.955277+0000 mgr.gibba032.whopeo (mgr.24625) 2352 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.179032+0000 mgr.gibba032.whopeo (mgr.24625) 2353 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.179192+0000 mon.a (mon.0) 1226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.180815+0000 mgr.gibba032.whopeo (mgr.24625) 2354 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:36:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.182221+0000 mgr.gibba032.whopeo (mgr.24625) 2355 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:36:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.182260+0000 mgr.gibba032.whopeo (mgr.24625) 2356 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:36:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.182327+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:36:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.183681+0000 mgr.gibba032.whopeo (mgr.24625) 2357 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:36:50.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.183720+0000 mgr.gibba032.whopeo (mgr.24625) 2358 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:36:50.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.183854+0000 mon.a (mon.0) 1228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.185079+0000 mgr.gibba032.whopeo (mgr.24625) 2359 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:36:50.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.185310+0000 mon.a (mon.0) 1229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.186330+0000 mgr.gibba032.whopeo (mgr.24625) 2360 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:36:50.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.186365+0000 mgr.gibba032.whopeo (mgr.24625) 2361 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:36:50.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.186713+0000 mon.a (mon.0) 1230 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.187819+0000 mgr.gibba032.whopeo (mgr.24625) 2362 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:36:50.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.188193+0000 mgr.gibba032.whopeo (mgr.24625) 2363 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:36:50.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.189204+0000 mgr.gibba032.whopeo (mgr.24625) 2364 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:36:50.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.189242+0000 mgr.gibba032.whopeo (mgr.24625) 2365 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:36:50.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: cephadm 2021-03-10T14:36:49.189346+0000 mgr.gibba032.whopeo (mgr.24625) 2366 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:50.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.189476+0000 mon.a (mon.0) 1231 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[30791]: audit 2021-03-10T14:36:49.191009+0000 mon.a (mon.0) 1232 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cluster 2021-03-10T14:36:48.312793+0000 mgr.gibba032.whopeo (mgr.24625) 2350 : cluster [DBG] pgmap v1330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:48.953711+0000 mgr.gibba032.whopeo (mgr.24625) 2351 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:36:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:48.955277+0000 mgr.gibba032.whopeo (mgr.24625) 2352 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:36:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.179032+0000 mgr.gibba032.whopeo (mgr.24625) 2353 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:36:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.179192+0000 mon.a (mon.0) 1226 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:36:50.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.180815+0000 mgr.gibba032.whopeo (mgr.24625) 2354 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:36:50.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.182221+0000 mgr.gibba032.whopeo (mgr.24625) 2355 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:36:50.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.182260+0000 mgr.gibba032.whopeo (mgr.24625) 2356 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:36:50.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.182327+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:36:50.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.183681+0000 mgr.gibba032.whopeo (mgr.24625) 2357 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:36:50.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.183720+0000 mgr.gibba032.whopeo (mgr.24625) 2358 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:36:50.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.183854+0000 mon.a (mon.0) 1228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.185079+0000 mgr.gibba032.whopeo (mgr.24625) 2359 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:36:50.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.185310+0000 mon.a (mon.0) 1229 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.186330+0000 mgr.gibba032.whopeo (mgr.24625) 2360 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:36:50.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.186365+0000 mgr.gibba032.whopeo (mgr.24625) 2361 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:36:50.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.186713+0000 mon.a (mon.0) 1230 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.187819+0000 mgr.gibba032.whopeo (mgr.24625) 2362 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:36:50.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.188193+0000 mgr.gibba032.whopeo (mgr.24625) 2363 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:36:50.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.189204+0000 mgr.gibba032.whopeo (mgr.24625) 2364 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:36:50.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.189242+0000 mgr.gibba032.whopeo (mgr.24625) 2365 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:36:50.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: cephadm 2021-03-10T14:36:49.189346+0000 mgr.gibba032.whopeo (mgr.24625) 2366 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:36:50.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.189476+0000 mon.a (mon.0) 1231 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:50.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:50 gibba011 bash[31774]: audit 2021-03-10T14:36:49.191009+0000 mon.a (mon.0) 1232 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:52 gibba032 bash[24004]: cluster 2021-03-10T14:36:50.314186+0000 mgr.gibba032.whopeo (mgr.24625) 2367 : cluster [DBG] pgmap v1331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:52 gibba011 bash[30791]: cluster 2021-03-10T14:36:50.314186+0000 mgr.gibba032.whopeo (mgr.24625) 2367 : cluster [DBG] pgmap v1331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:52 gibba011 bash[31774]: cluster 2021-03-10T14:36:50.314186+0000 mgr.gibba032.whopeo (mgr.24625) 2367 : cluster [DBG] pgmap v1331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:54 gibba032 bash[24004]: cluster 2021-03-10T14:36:52.315709+0000 mgr.gibba032.whopeo (mgr.24625) 2368 : cluster [DBG] pgmap v1332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:54 gibba011 bash[30791]: cluster 2021-03-10T14:36:52.315709+0000 mgr.gibba032.whopeo (mgr.24625) 2368 : cluster [DBG] pgmap v1332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:54 gibba011 bash[31774]: cluster 2021-03-10T14:36:52.315709+0000 mgr.gibba032.whopeo (mgr.24625) 2368 : cluster [DBG] pgmap v1332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:55.456 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:36:55.678 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:36:55.678 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (43m) 9m ago 48m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:36:55.678 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (48m) 8m ago 48m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:36:55.679 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (43m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:36:55.679 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (44m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:36:55.679 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (43m) 9m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:36:55.679 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (43m) 9m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:36:55.680 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (42m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:36:55.680 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (42m) 9m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:36:55.680 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (49m) 9m ago 49m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:36:55.680 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (49m) 8m ago 49m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:36:55.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (41m) 9m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:36:55.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:36:55.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:36:55.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (40m) 9m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:36:55.682 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (40m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:36:55.682 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (39m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:36:55.682 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (39m) 8m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:36:55.682 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (38m) 8m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:36:55.683 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (48m) 8m ago 49m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:36:55.683 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 47m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:36:55.923 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:36:55.924 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:36:55.924 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:36:55.924 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:55.924 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:36:55.924 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:36:55.925 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:55.925 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:36:55.925 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:36:55.925 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:36:55.925 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:36:55.926 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:36:55.926 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:36:55.926 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:36:55.926 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:36:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:56 gibba032 bash[24004]: cluster 2021-03-10T14:36:54.316932+0000 mgr.gibba032.whopeo (mgr.24625) 2369 : cluster [DBG] pgmap v1333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:56 gibba032 bash[24004]: audit 2021-03-10T14:36:55.922882+0000 mon.a (mon.0) 1233 : audit [DBG] from='client.? 172.21.2.111:0/1978531329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:56 gibba011 bash[30791]: cluster 2021-03-10T14:36:54.316932+0000 mgr.gibba032.whopeo (mgr.24625) 2369 : cluster [DBG] pgmap v1333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:56 gibba011 bash[30791]: audit 2021-03-10T14:36:55.922882+0000 mon.a (mon.0) 1233 : audit [DBG] from='client.? 172.21.2.111:0/1978531329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:56 gibba011 bash[31774]: cluster 2021-03-10T14:36:54.316932+0000 mgr.gibba032.whopeo (mgr.24625) 2369 : cluster [DBG] pgmap v1333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:56 gibba011 bash[31774]: audit 2021-03-10T14:36:55.922882+0000 mon.a (mon.0) 1233 : audit [DBG] from='client.? 172.21.2.111:0/1978531329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:36:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:55.443849+0000 mgr.gibba032.whopeo (mgr.24625) 2370 : audit [DBG] from='client.45639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:55.665765+0000 mgr.gibba032.whopeo (mgr.24625) 2371 : audit [DBG] from='client.55630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.284389+0000 mon.a (mon.0) 1234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:36:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.285112+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:36:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.294135+0000 mon.a (mon.0) 1236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:36:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.294846+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.295539+0000 mon.a (mon.0) 1238 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:57 gibba032 bash[24004]: audit 2021-03-10T14:36:56.296206+0000 mon.a (mon.0) 1239 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:36:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:55.443849+0000 mgr.gibba032.whopeo (mgr.24625) 2370 : audit [DBG] from='client.45639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:55.665765+0000 mgr.gibba032.whopeo (mgr.24625) 2371 : audit [DBG] from='client.55630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.284389+0000 mon.a (mon.0) 1234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:36:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.285112+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:36:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.294135+0000 mon.a (mon.0) 1236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:36:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.294846+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.295539+0000 mon.a (mon.0) 1238 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[30791]: audit 2021-03-10T14:36:56.296206+0000 mon.a (mon.0) 1239 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:36:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:55.443849+0000 mgr.gibba032.whopeo (mgr.24625) 2370 : audit [DBG] from='client.45639 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:55.665765+0000 mgr.gibba032.whopeo (mgr.24625) 2371 : audit [DBG] from='client.55630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:36:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.284389+0000 mon.a (mon.0) 1234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:36:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.285112+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:36:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.294135+0000 mon.a (mon.0) 1236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:36:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.294846+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.295539+0000 mon.a (mon.0) 1238 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:36:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:57 gibba011 bash[31774]: audit 2021-03-10T14:36:56.296206+0000 mon.a (mon.0) 1239 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:36:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:36:58 gibba032 bash[24004]: cluster 2021-03-10T14:36:56.317711+0000 mgr.gibba032.whopeo (mgr.24625) 2372 : cluster [DBG] pgmap v1334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:36:58 gibba011 bash[30791]: cluster 2021-03-10T14:36:56.317711+0000 mgr.gibba032.whopeo (mgr.24625) 2372 : cluster [DBG] pgmap v1334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:36:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:36:58 gibba011 bash[31774]: cluster 2021-03-10T14:36:56.317711+0000 mgr.gibba032.whopeo (mgr.24625) 2372 : cluster [DBG] pgmap v1334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:00 gibba032 bash[24004]: cluster 2021-03-10T14:36:58.319115+0000 mgr.gibba032.whopeo (mgr.24625) 2373 : cluster [DBG] pgmap v1335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:00 gibba011 bash[30791]: cluster 2021-03-10T14:36:58.319115+0000 mgr.gibba032.whopeo (mgr.24625) 2373 : cluster [DBG] pgmap v1335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:00 gibba011 bash[31774]: cluster 2021-03-10T14:36:58.319115+0000 mgr.gibba032.whopeo (mgr.24625) 2373 : cluster [DBG] pgmap v1335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:02 gibba032 bash[24004]: cluster 2021-03-10T14:37:00.320245+0000 mgr.gibba032.whopeo (mgr.24625) 2374 : cluster [DBG] pgmap v1336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:02 gibba011 bash[30791]: cluster 2021-03-10T14:37:00.320245+0000 mgr.gibba032.whopeo (mgr.24625) 2374 : cluster [DBG] pgmap v1336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:02 gibba011 bash[31774]: cluster 2021-03-10T14:37:00.320245+0000 mgr.gibba032.whopeo (mgr.24625) 2374 : cluster [DBG] pgmap v1336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:04 gibba032 bash[24004]: cluster 2021-03-10T14:37:02.322029+0000 mgr.gibba032.whopeo (mgr.24625) 2375 : cluster [DBG] pgmap v1337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:04 gibba011 bash[30791]: cluster 2021-03-10T14:37:02.322029+0000 mgr.gibba032.whopeo (mgr.24625) 2375 : cluster [DBG] pgmap v1337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:04 gibba011 bash[31774]: cluster 2021-03-10T14:37:02.322029+0000 mgr.gibba032.whopeo (mgr.24625) 2375 : cluster [DBG] pgmap v1337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:05 gibba032 bash[24004]: cephadm 2021-03-10T14:37:04.205210+0000 mgr.gibba032.whopeo (mgr.24625) 2376 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:05 gibba011 bash[30791]: cephadm 2021-03-10T14:37:04.205210+0000 mgr.gibba032.whopeo (mgr.24625) 2376 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:05 gibba011 bash[31774]: cephadm 2021-03-10T14:37:04.205210+0000 mgr.gibba032.whopeo (mgr.24625) 2376 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:06 gibba011 bash[30791]: cluster 2021-03-10T14:37:04.323113+0000 mgr.gibba032.whopeo (mgr.24625) 2377 : cluster [DBG] pgmap v1338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:06 gibba011 bash[31774]: cluster 2021-03-10T14:37:04.323113+0000 mgr.gibba032.whopeo (mgr.24625) 2377 : cluster [DBG] pgmap v1338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:06 gibba032 bash[24004]: cluster 2021-03-10T14:37:04.323113+0000 mgr.gibba032.whopeo (mgr.24625) 2377 : cluster [DBG] pgmap v1338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:07 gibba011 bash[30791]: cluster 2021-03-10T14:37:06.324775+0000 mgr.gibba032.whopeo (mgr.24625) 2378 : cluster [DBG] pgmap v1339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:07 gibba011 bash[31774]: cluster 2021-03-10T14:37:06.324775+0000 mgr.gibba032.whopeo (mgr.24625) 2378 : cluster [DBG] pgmap v1339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:07 gibba032 bash[24004]: cluster 2021-03-10T14:37:06.324775+0000 mgr.gibba032.whopeo (mgr.24625) 2378 : cluster [DBG] pgmap v1339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:09.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:09 gibba032 bash[24004]: cluster 2021-03-10T14:37:08.326568+0000 mgr.gibba032.whopeo (mgr.24625) 2379 : cluster [DBG] pgmap v1340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:09 gibba011 bash[30791]: cluster 2021-03-10T14:37:08.326568+0000 mgr.gibba032.whopeo (mgr.24625) 2379 : cluster [DBG] pgmap v1340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:09.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:09 gibba011 bash[31774]: cluster 2021-03-10T14:37:08.326568+0000 mgr.gibba032.whopeo (mgr.24625) 2379 : cluster [DBG] pgmap v1340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:12 gibba032 bash[24004]: cluster 2021-03-10T14:37:10.328193+0000 mgr.gibba032.whopeo (mgr.24625) 2380 : cluster [DBG] pgmap v1341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:12 gibba011 bash[30791]: cluster 2021-03-10T14:37:10.328193+0000 mgr.gibba032.whopeo (mgr.24625) 2380 : cluster [DBG] pgmap v1341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:12 gibba011 bash[31774]: cluster 2021-03-10T14:37:10.328193+0000 mgr.gibba032.whopeo (mgr.24625) 2380 : cluster [DBG] pgmap v1341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:14 gibba032 bash[24004]: cluster 2021-03-10T14:37:12.330125+0000 mgr.gibba032.whopeo (mgr.24625) 2381 : cluster [DBG] pgmap v1342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:14 gibba011 bash[30791]: cluster 2021-03-10T14:37:12.330125+0000 mgr.gibba032.whopeo (mgr.24625) 2381 : cluster [DBG] pgmap v1342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:14 gibba011 bash[31774]: cluster 2021-03-10T14:37:12.330125+0000 mgr.gibba032.whopeo (mgr.24625) 2381 : cluster [DBG] pgmap v1342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:16 gibba032 bash[24004]: cluster 2021-03-10T14:37:14.331385+0000 mgr.gibba032.whopeo (mgr.24625) 2382 : cluster [DBG] pgmap v1343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:16 gibba011 bash[30791]: cluster 2021-03-10T14:37:14.331385+0000 mgr.gibba032.whopeo (mgr.24625) 2382 : cluster [DBG] pgmap v1343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:16 gibba011 bash[31774]: cluster 2021-03-10T14:37:14.331385+0000 mgr.gibba032.whopeo (mgr.24625) 2382 : cluster [DBG] pgmap v1343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:18 gibba032 bash[24004]: cluster 2021-03-10T14:37:16.332851+0000 mgr.gibba032.whopeo (mgr.24625) 2383 : cluster [DBG] pgmap v1344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:18 gibba011 bash[30791]: cluster 2021-03-10T14:37:16.332851+0000 mgr.gibba032.whopeo (mgr.24625) 2383 : cluster [DBG] pgmap v1344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:18 gibba011 bash[31774]: cluster 2021-03-10T14:37:16.332851+0000 mgr.gibba032.whopeo (mgr.24625) 2383 : cluster [DBG] pgmap v1344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:20 gibba032 bash[24004]: cluster 2021-03-10T14:37:18.334570+0000 mgr.gibba032.whopeo (mgr.24625) 2384 : cluster [DBG] pgmap v1345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:20 gibba032 bash[24004]: cephadm 2021-03-10T14:37:19.220425+0000 mgr.gibba032.whopeo (mgr.24625) 2385 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:20 gibba011 bash[30791]: cluster 2021-03-10T14:37:18.334570+0000 mgr.gibba032.whopeo (mgr.24625) 2384 : cluster [DBG] pgmap v1345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:20 gibba011 bash[30791]: cephadm 2021-03-10T14:37:19.220425+0000 mgr.gibba032.whopeo (mgr.24625) 2385 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:20 gibba011 bash[31774]: cluster 2021-03-10T14:37:18.334570+0000 mgr.gibba032.whopeo (mgr.24625) 2384 : cluster [DBG] pgmap v1345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:20 gibba011 bash[31774]: cephadm 2021-03-10T14:37:19.220425+0000 mgr.gibba032.whopeo (mgr.24625) 2385 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:22 gibba032 bash[24004]: cluster 2021-03-10T14:37:20.335032+0000 mgr.gibba032.whopeo (mgr.24625) 2386 : cluster [DBG] pgmap v1346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:22 gibba011 bash[30791]: cluster 2021-03-10T14:37:20.335032+0000 mgr.gibba032.whopeo (mgr.24625) 2386 : cluster [DBG] pgmap v1346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:22 gibba011 bash[31774]: cluster 2021-03-10T14:37:20.335032+0000 mgr.gibba032.whopeo (mgr.24625) 2386 : cluster [DBG] pgmap v1346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:24 gibba032 bash[24004]: cluster 2021-03-10T14:37:22.336584+0000 mgr.gibba032.whopeo (mgr.24625) 2387 : cluster [DBG] pgmap v1347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:24 gibba011 bash[30791]: cluster 2021-03-10T14:37:22.336584+0000 mgr.gibba032.whopeo (mgr.24625) 2387 : cluster [DBG] pgmap v1347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:24 gibba011 bash[31774]: cluster 2021-03-10T14:37:22.336584+0000 mgr.gibba032.whopeo (mgr.24625) 2387 : cluster [DBG] pgmap v1347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:26.166 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:37:26.396 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:37:26.397 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (44m) 9m ago 49m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:37:26.397 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (49m) 8m ago 49m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:37:26.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (44m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:37:26.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (44m) 8m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:37:26.398 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (43m) 9m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:37:26.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (43m) 9m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:37:26.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (43m) 8m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:37:26.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (43m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:37:26.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (49m) 9m ago 49m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:37:26.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (49m) 8m ago 49m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:37:26.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (42m) 9m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:37:26.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (41m) 9m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:37:26.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:37:26.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:37:26.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (40m) 8m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:37:26.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (40m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:37:26.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (39m) 8m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:37:26.401 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (39m) 8m ago 49m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:37:26.401 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (49m) 8m ago 49m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:37:26.401 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 47m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:37:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:26 gibba032 bash[24004]: cluster 2021-03-10T14:37:24.337795+0000 mgr.gibba032.whopeo (mgr.24625) 2388 : cluster [DBG] pgmap v1348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:26 gibba011 bash[30791]: cluster 2021-03-10T14:37:24.337795+0000 mgr.gibba032.whopeo (mgr.24625) 2388 : cluster [DBG] pgmap v1348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:26 gibba011 bash[31774]: cluster 2021-03-10T14:37:24.337795+0000 mgr.gibba032.whopeo (mgr.24625) 2388 : cluster [DBG] pgmap v1348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:26.643 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:37:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:37:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:37:26.643 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:37:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:37:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:37:26.644 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:37:26.645 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:26.645 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:37:26.645 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:37:26.645 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:37:26.645 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:37:26.646 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:37:27.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:27 gibba032 bash[24004]: audit 2021-03-10T14:37:26.154641+0000 mgr.gibba032.whopeo (mgr.24625) 2389 : audit [DBG] from='client.55642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:27.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:27 gibba032 bash[24004]: audit 2021-03-10T14:37:26.642004+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.2.111:0/3437018346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:27 gibba011 bash[30791]: audit 2021-03-10T14:37:26.154641+0000 mgr.gibba032.whopeo (mgr.24625) 2389 : audit [DBG] from='client.55642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:27 gibba011 bash[30791]: audit 2021-03-10T14:37:26.642004+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.2.111:0/3437018346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:27.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:27 gibba011 bash[31774]: audit 2021-03-10T14:37:26.154641+0000 mgr.gibba032.whopeo (mgr.24625) 2389 : audit [DBG] from='client.55642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:27.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:27 gibba011 bash[31774]: audit 2021-03-10T14:37:26.642004+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.2.111:0/3437018346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:28 gibba032 bash[24004]: cluster 2021-03-10T14:37:26.339392+0000 mgr.gibba032.whopeo (mgr.24625) 2390 : cluster [DBG] pgmap v1349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:28 gibba032 bash[24004]: audit 2021-03-10T14:37:26.380885+0000 mgr.gibba032.whopeo (mgr.24625) 2391 : audit [DBG] from='client.55648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:28 gibba011 bash[30791]: cluster 2021-03-10T14:37:26.339392+0000 mgr.gibba032.whopeo (mgr.24625) 2390 : cluster [DBG] pgmap v1349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:28 gibba011 bash[30791]: audit 2021-03-10T14:37:26.380885+0000 mgr.gibba032.whopeo (mgr.24625) 2391 : audit [DBG] from='client.55648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:28 gibba011 bash[31774]: cluster 2021-03-10T14:37:26.339392+0000 mgr.gibba032.whopeo (mgr.24625) 2390 : cluster [DBG] pgmap v1349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:28 gibba011 bash[31774]: audit 2021-03-10T14:37:26.380885+0000 mgr.gibba032.whopeo (mgr.24625) 2391 : audit [DBG] from='client.55648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:30 gibba032 bash[24004]: cluster 2021-03-10T14:37:28.339906+0000 mgr.gibba032.whopeo (mgr.24625) 2392 : cluster [DBG] pgmap v1350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:30 gibba011 bash[30791]: cluster 2021-03-10T14:37:28.339906+0000 mgr.gibba032.whopeo (mgr.24625) 2392 : cluster [DBG] pgmap v1350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:30 gibba011 bash[31774]: cluster 2021-03-10T14:37:28.339906+0000 mgr.gibba032.whopeo (mgr.24625) 2392 : cluster [DBG] pgmap v1350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:32 gibba032 bash[24004]: cluster 2021-03-10T14:37:30.341251+0000 mgr.gibba032.whopeo (mgr.24625) 2393 : cluster [DBG] pgmap v1351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:32 gibba011 bash[30791]: cluster 2021-03-10T14:37:30.341251+0000 mgr.gibba032.whopeo (mgr.24625) 2393 : cluster [DBG] pgmap v1351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:32 gibba011 bash[31774]: cluster 2021-03-10T14:37:30.341251+0000 mgr.gibba032.whopeo (mgr.24625) 2393 : cluster [DBG] pgmap v1351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:34 gibba032 bash[24004]: cluster 2021-03-10T14:37:32.343019+0000 mgr.gibba032.whopeo (mgr.24625) 2394 : cluster [DBG] pgmap v1352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:34 gibba011 bash[30791]: cluster 2021-03-10T14:37:32.343019+0000 mgr.gibba032.whopeo (mgr.24625) 2394 : cluster [DBG] pgmap v1352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:34 gibba011 bash[31774]: cluster 2021-03-10T14:37:32.343019+0000 mgr.gibba032.whopeo (mgr.24625) 2394 : cluster [DBG] pgmap v1352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:35 gibba032 bash[24004]: cephadm 2021-03-10T14:37:34.235641+0000 mgr.gibba032.whopeo (mgr.24625) 2395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:35 gibba011 bash[30791]: cephadm 2021-03-10T14:37:34.235641+0000 mgr.gibba032.whopeo (mgr.24625) 2395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:35 gibba011 bash[31774]: cephadm 2021-03-10T14:37:34.235641+0000 mgr.gibba032.whopeo (mgr.24625) 2395 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:36 gibba032 bash[24004]: cluster 2021-03-10T14:37:34.344172+0000 mgr.gibba032.whopeo (mgr.24625) 2396 : cluster [DBG] pgmap v1353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:36 gibba011 bash[30791]: cluster 2021-03-10T14:37:34.344172+0000 mgr.gibba032.whopeo (mgr.24625) 2396 : cluster [DBG] pgmap v1353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:36 gibba011 bash[31774]: cluster 2021-03-10T14:37:34.344172+0000 mgr.gibba032.whopeo (mgr.24625) 2396 : cluster [DBG] pgmap v1353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:38 gibba032 bash[24004]: cluster 2021-03-10T14:37:36.344705+0000 mgr.gibba032.whopeo (mgr.24625) 2397 : cluster [DBG] pgmap v1354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:38 gibba011 bash[30791]: cluster 2021-03-10T14:37:36.344705+0000 mgr.gibba032.whopeo (mgr.24625) 2397 : cluster [DBG] pgmap v1354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:38 gibba011 bash[31774]: cluster 2021-03-10T14:37:36.344705+0000 mgr.gibba032.whopeo (mgr.24625) 2397 : cluster [DBG] pgmap v1354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:40 gibba011 bash[30791]: cluster 2021-03-10T14:37:38.346281+0000 mgr.gibba032.whopeo (mgr.24625) 2398 : cluster [DBG] pgmap v1355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:40 gibba011 bash[31774]: cluster 2021-03-10T14:37:38.346281+0000 mgr.gibba032.whopeo (mgr.24625) 2398 : cluster [DBG] pgmap v1355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:40 gibba032 bash[24004]: cluster 2021-03-10T14:37:38.346281+0000 mgr.gibba032.whopeo (mgr.24625) 2398 : cluster [DBG] pgmap v1355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:42 gibba011 bash[30791]: cluster 2021-03-10T14:37:40.347756+0000 mgr.gibba032.whopeo (mgr.24625) 2399 : cluster [DBG] pgmap v1356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:42 gibba011 bash[31774]: cluster 2021-03-10T14:37:40.347756+0000 mgr.gibba032.whopeo (mgr.24625) 2399 : cluster [DBG] pgmap v1356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:42 gibba032 bash[24004]: cluster 2021-03-10T14:37:40.347756+0000 mgr.gibba032.whopeo (mgr.24625) 2399 : cluster [DBG] pgmap v1356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:43 gibba011 bash[30791]: cluster 2021-03-10T14:37:42.349576+0000 mgr.gibba032.whopeo (mgr.24625) 2400 : cluster [DBG] pgmap v1357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:43.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:43 gibba011 bash[31774]: cluster 2021-03-10T14:37:42.349576+0000 mgr.gibba032.whopeo (mgr.24625) 2400 : cluster [DBG] pgmap v1357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:43 gibba032 bash[24004]: cluster 2021-03-10T14:37:42.349576+0000 mgr.gibba032.whopeo (mgr.24625) 2400 : cluster [DBG] pgmap v1357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:45.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:45 gibba032 bash[24004]: cluster 2021-03-10T14:37:44.350660+0000 mgr.gibba032.whopeo (mgr.24625) 2401 : cluster [DBG] pgmap v1358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:45.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:45 gibba011 bash[30791]: cluster 2021-03-10T14:37:44.350660+0000 mgr.gibba032.whopeo (mgr.24625) 2401 : cluster [DBG] pgmap v1358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:45.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:45 gibba011 bash[31774]: cluster 2021-03-10T14:37:44.350660+0000 mgr.gibba032.whopeo (mgr.24625) 2401 : cluster [DBG] pgmap v1358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:48 gibba032 bash[24004]: cluster 2021-03-10T14:37:46.352370+0000 mgr.gibba032.whopeo (mgr.24625) 2402 : cluster [DBG] pgmap v1359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:48 gibba011 bash[30791]: cluster 2021-03-10T14:37:46.352370+0000 mgr.gibba032.whopeo (mgr.24625) 2402 : cluster [DBG] pgmap v1359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:48 gibba011 bash[31774]: cluster 2021-03-10T14:37:46.352370+0000 mgr.gibba032.whopeo (mgr.24625) 2402 : cluster [DBG] pgmap v1359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:50 gibba032 bash[24004]: cluster 2021-03-10T14:37:48.354148+0000 mgr.gibba032.whopeo (mgr.24625) 2403 : cluster [DBG] pgmap v1360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:50 gibba032 bash[24004]: cephadm 2021-03-10T14:37:49.253308+0000 mgr.gibba032.whopeo (mgr.24625) 2404 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:37:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:50 gibba032 bash[24004]: audit 2021-03-10T14:37:49.256223+0000 mon.a (mon.0) 1241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:37:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[30791]: cluster 2021-03-10T14:37:48.354148+0000 mgr.gibba032.whopeo (mgr.24625) 2403 : cluster [DBG] pgmap v1360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[30791]: cephadm 2021-03-10T14:37:49.253308+0000 mgr.gibba032.whopeo (mgr.24625) 2404 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:37:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[30791]: audit 2021-03-10T14:37:49.256223+0000 mon.a (mon.0) 1241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:37:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[31774]: cluster 2021-03-10T14:37:48.354148+0000 mgr.gibba032.whopeo (mgr.24625) 2403 : cluster [DBG] pgmap v1360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[31774]: cephadm 2021-03-10T14:37:49.253308+0000 mgr.gibba032.whopeo (mgr.24625) 2404 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:37:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:50 gibba011 bash[31774]: audit 2021-03-10T14:37:49.256223+0000 mon.a (mon.0) 1241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:37:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:52 gibba032 bash[24004]: cluster 2021-03-10T14:37:50.355695+0000 mgr.gibba032.whopeo (mgr.24625) 2405 : cluster [DBG] pgmap v1361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:52 gibba011 bash[30791]: cluster 2021-03-10T14:37:50.355695+0000 mgr.gibba032.whopeo (mgr.24625) 2405 : cluster [DBG] pgmap v1361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:52 gibba011 bash[31774]: cluster 2021-03-10T14:37:50.355695+0000 mgr.gibba032.whopeo (mgr.24625) 2405 : cluster [DBG] pgmap v1361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cluster 2021-03-10T14:37:52.356503+0000 mgr.gibba032.whopeo (mgr.24625) 2406 : cluster [DBG] pgmap v1362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:52.898760+0000 mgr.gibba032.whopeo (mgr.24625) 2407 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:52.900773+0000 mon.a (mon.0) 1242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.096450+0000 mon.a (mon.0) 1243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.106740+0000 mgr.gibba032.whopeo (mgr.24625) 2408 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.111905+0000 mon.a (mon.0) 1244 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.112599+0000 mgr.gibba032.whopeo (mgr.24625) 2409 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.116471+0000 mgr.gibba032.whopeo (mgr.24625) 2410 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.116483+0000 mon.a (mon.0) 1245 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.117021+0000 mgr.gibba032.whopeo (mgr.24625) 2411 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.119990+0000 mgr.gibba032.whopeo (mgr.24625) 2412 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.120287+0000 mgr.gibba032.whopeo (mgr.24625) 2413 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.120500+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.122597+0000 mgr.gibba032.whopeo (mgr.24625) 2414 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.123337+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.125213+0000 mgr.gibba032.whopeo (mgr.24625) 2415 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.125584+0000 mgr.gibba032.whopeo (mgr.24625) 2416 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.128954+0000 mgr.gibba032.whopeo (mgr.24625) 2417 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.129638+0000 mon.a (mon.0) 1248 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.131074+0000 mgr.gibba032.whopeo (mgr.24625) 2418 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.133895+0000 mgr.gibba032.whopeo (mgr.24625) 2419 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.134177+0000 mgr.gibba032.whopeo (mgr.24625) 2420 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: audit 2021-03-10T14:37:53.134416+0000 mon.a (mon.0) 1249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:53 gibba032 bash[24004]: cephadm 2021-03-10T14:37:53.134754+0000 mgr.gibba032.whopeo (mgr.24625) 2421 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cluster 2021-03-10T14:37:52.356503+0000 mgr.gibba032.whopeo (mgr.24625) 2406 : cluster [DBG] pgmap v1362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:52.898760+0000 mgr.gibba032.whopeo (mgr.24625) 2407 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:37:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:52.900773+0000 mon.a (mon.0) 1242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.096450+0000 mon.a (mon.0) 1243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.106740+0000 mgr.gibba032.whopeo (mgr.24625) 2408 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.111905+0000 mon.a (mon.0) 1244 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.112599+0000 mgr.gibba032.whopeo (mgr.24625) 2409 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.116471+0000 mgr.gibba032.whopeo (mgr.24625) 2410 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.116483+0000 mon.a (mon.0) 1245 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.117021+0000 mgr.gibba032.whopeo (mgr.24625) 2411 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.119990+0000 mgr.gibba032.whopeo (mgr.24625) 2412 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.120287+0000 mgr.gibba032.whopeo (mgr.24625) 2413 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.120500+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.122597+0000 mgr.gibba032.whopeo (mgr.24625) 2414 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.123337+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.125213+0000 mgr.gibba032.whopeo (mgr.24625) 2415 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.125584+0000 mgr.gibba032.whopeo (mgr.24625) 2416 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.128954+0000 mgr.gibba032.whopeo (mgr.24625) 2417 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.129638+0000 mon.a (mon.0) 1248 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.131074+0000 mgr.gibba032.whopeo (mgr.24625) 2418 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:37:54.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.133895+0000 mgr.gibba032.whopeo (mgr.24625) 2419 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:37:54.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cluster 2021-03-10T14:37:52.356503+0000 mgr.gibba032.whopeo (mgr.24625) 2406 : cluster [DBG] pgmap v1362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:54.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:52.898760+0000 mgr.gibba032.whopeo (mgr.24625) 2407 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:37:54.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:52.900773+0000 mon.a (mon.0) 1242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.096450+0000 mon.a (mon.0) 1243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:37:54.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.106740+0000 mgr.gibba032.whopeo (mgr.24625) 2408 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:37:54.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.111905+0000 mon.a (mon.0) 1244 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:37:54.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.112599+0000 mgr.gibba032.whopeo (mgr.24625) 2409 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:37:54.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.116471+0000 mgr.gibba032.whopeo (mgr.24625) 2410 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:37:54.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.116483+0000 mon.a (mon.0) 1245 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.117021+0000 mgr.gibba032.whopeo (mgr.24625) 2411 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:37:54.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.119990+0000 mgr.gibba032.whopeo (mgr.24625) 2412 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:37:54.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.120287+0000 mgr.gibba032.whopeo (mgr.24625) 2413 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:37:54.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.120500+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.122597+0000 mgr.gibba032.whopeo (mgr.24625) 2414 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:37:54.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.123337+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.125213+0000 mgr.gibba032.whopeo (mgr.24625) 2415 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:37:54.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.125584+0000 mgr.gibba032.whopeo (mgr.24625) 2416 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:37:54.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.128954+0000 mgr.gibba032.whopeo (mgr.24625) 2417 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:37:54.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.129638+0000 mon.a (mon.0) 1248 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.131074+0000 mgr.gibba032.whopeo (mgr.24625) 2418 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:37:54.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.133895+0000 mgr.gibba032.whopeo (mgr.24625) 2419 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:37:54.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.134177+0000 mgr.gibba032.whopeo (mgr.24625) 2420 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:37:54.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: audit 2021-03-10T14:37:53.134416+0000 mon.a (mon.0) 1249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[30791]: cephadm 2021-03-10T14:37:53.134754+0000 mgr.gibba032.whopeo (mgr.24625) 2421 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:54.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.134177+0000 mgr.gibba032.whopeo (mgr.24625) 2420 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:37:54.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: audit 2021-03-10T14:37:53.134416+0000 mon.a (mon.0) 1249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:54.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:53 gibba011 bash[31774]: cephadm 2021-03-10T14:37:53.134754+0000 mgr.gibba032.whopeo (mgr.24625) 2421 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:37:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:55 gibba032 bash[24004]: cluster 2021-03-10T14:37:54.357611+0000 mgr.gibba032.whopeo (mgr.24625) 2422 : cluster [DBG] pgmap v1363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:55 gibba011 bash[30791]: cluster 2021-03-10T14:37:54.357611+0000 mgr.gibba032.whopeo (mgr.24625) 2422 : cluster [DBG] pgmap v1363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:55 gibba011 bash[31774]: cluster 2021-03-10T14:37:54.357611+0000 mgr.gibba032.whopeo (mgr.24625) 2422 : cluster [DBG] pgmap v1363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:56.885 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:37:57.115 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:37:57.116 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (44m) 4s ago 49m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:37:57.116 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (49m) 9m ago 49m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:37:57.116 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (44m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:37:57.116 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (45m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:37:57.116 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (44m) 4s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:37:57.117 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (44m) 4s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:37:57.117 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (43m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:37:57.117 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (43m) 4s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:37:57.117 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (50m) 4s ago 50m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:37:57.117 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (50m) 9m ago 50m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:37:57.118 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (42m) 4s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:37:57.118 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (42m) 4s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:37:57.118 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (42m) 4s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:37:57.118 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (41m) 4s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:37:57.118 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:37:57.119 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (40m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:37:57.119 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (40m) 9m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:37:57.119 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (39m) 9m ago 50m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:37:57.119 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (49m) 9m ago 50m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:37:57.119 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4s ago 48m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:37:57.371 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:37:57.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:37:57.372 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:37:57.372 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:57.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:37:57.373 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:37:57.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:57.373 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:37:57.373 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:37:57.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:37:57.374 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:37:57.374 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:37:57.374 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:37:57.374 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:37:57.374 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:37:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.302477+0000 mon.a (mon.0) 1250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:37:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.303095+0000 mon.a (mon.0) 1251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:37:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.313361+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:37:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.314137+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.314786+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:57 gibba032 bash[24004]: audit 2021-03-10T14:37:56.315492+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.302477+0000 mon.a (mon.0) 1250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:37:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.303095+0000 mon.a (mon.0) 1251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:37:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.313361+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:37:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.314137+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.314786+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[30791]: audit 2021-03-10T14:37:56.315492+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.302477+0000 mon.a (mon.0) 1250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:37:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.303095+0000 mon.a (mon.0) 1251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:37:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.313361+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:37:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.314137+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.314786+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:37:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:57 gibba011 bash[31774]: audit 2021-03-10T14:37:56.315492+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:37:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:58 gibba032 bash[24004]: cluster 2021-03-10T14:37:56.359015+0000 mgr.gibba032.whopeo (mgr.24625) 2423 : cluster [DBG] pgmap v1364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:58 gibba032 bash[24004]: audit 2021-03-10T14:37:56.872530+0000 mgr.gibba032.whopeo (mgr.24625) 2424 : audit [DBG] from='client.55660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:58 gibba032 bash[24004]: audit 2021-03-10T14:37:57.099600+0000 mgr.gibba032.whopeo (mgr.24625) 2425 : audit [DBG] from='client.55666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:37:58 gibba032 bash[24004]: audit 2021-03-10T14:37:57.370585+0000 mon.c (mon.1) 47 : audit [DBG] from='client.? 172.21.2.111:0/3303921570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[30791]: cluster 2021-03-10T14:37:56.359015+0000 mgr.gibba032.whopeo (mgr.24625) 2423 : cluster [DBG] pgmap v1364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[30791]: audit 2021-03-10T14:37:56.872530+0000 mgr.gibba032.whopeo (mgr.24625) 2424 : audit [DBG] from='client.55660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[30791]: audit 2021-03-10T14:37:57.099600+0000 mgr.gibba032.whopeo (mgr.24625) 2425 : audit [DBG] from='client.55666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[30791]: audit 2021-03-10T14:37:57.370585+0000 mon.c (mon.1) 47 : audit [DBG] from='client.? 172.21.2.111:0/3303921570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:37:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[31774]: cluster 2021-03-10T14:37:56.359015+0000 mgr.gibba032.whopeo (mgr.24625) 2423 : cluster [DBG] pgmap v1364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:37:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[31774]: audit 2021-03-10T14:37:56.872530+0000 mgr.gibba032.whopeo (mgr.24625) 2424 : audit [DBG] from='client.55660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[31774]: audit 2021-03-10T14:37:57.099600+0000 mgr.gibba032.whopeo (mgr.24625) 2425 : audit [DBG] from='client.55666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:37:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:37:58 gibba011 bash[31774]: audit 2021-03-10T14:37:57.370585+0000 mon.c (mon.1) 47 : audit [DBG] from='client.? 172.21.2.111:0/3303921570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:00 gibba032 bash[24004]: cluster 2021-03-10T14:37:58.360536+0000 mgr.gibba032.whopeo (mgr.24625) 2426 : cluster [DBG] pgmap v1365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:00 gibba011 bash[30791]: cluster 2021-03-10T14:37:58.360536+0000 mgr.gibba032.whopeo (mgr.24625) 2426 : cluster [DBG] pgmap v1365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:00 gibba011 bash[31774]: cluster 2021-03-10T14:37:58.360536+0000 mgr.gibba032.whopeo (mgr.24625) 2426 : cluster [DBG] pgmap v1365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:02 gibba032 bash[24004]: cluster 2021-03-10T14:38:00.362116+0000 mgr.gibba032.whopeo (mgr.24625) 2427 : cluster [DBG] pgmap v1366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:02 gibba011 bash[30791]: cluster 2021-03-10T14:38:00.362116+0000 mgr.gibba032.whopeo (mgr.24625) 2427 : cluster [DBG] pgmap v1366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:02 gibba011 bash[31774]: cluster 2021-03-10T14:38:00.362116+0000 mgr.gibba032.whopeo (mgr.24625) 2427 : cluster [DBG] pgmap v1366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:04 gibba032 bash[24004]: cluster 2021-03-10T14:38:02.363856+0000 mgr.gibba032.whopeo (mgr.24625) 2428 : cluster [DBG] pgmap v1367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:04 gibba011 bash[30791]: cluster 2021-03-10T14:38:02.363856+0000 mgr.gibba032.whopeo (mgr.24625) 2428 : cluster [DBG] pgmap v1367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:04 gibba011 bash[31774]: cluster 2021-03-10T14:38:02.363856+0000 mgr.gibba032.whopeo (mgr.24625) 2428 : cluster [DBG] pgmap v1367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:06 gibba032 bash[24004]: cluster 2021-03-10T14:38:04.365150+0000 mgr.gibba032.whopeo (mgr.24625) 2429 : cluster [DBG] pgmap v1368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:06 gibba011 bash[30791]: cluster 2021-03-10T14:38:04.365150+0000 mgr.gibba032.whopeo (mgr.24625) 2429 : cluster [DBG] pgmap v1368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:06 gibba011 bash[31774]: cluster 2021-03-10T14:38:04.365150+0000 mgr.gibba032.whopeo (mgr.24625) 2429 : cluster [DBG] pgmap v1368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:08 gibba032 bash[24004]: cluster 2021-03-10T14:38:06.367070+0000 mgr.gibba032.whopeo (mgr.24625) 2430 : cluster [DBG] pgmap v1369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:08 gibba011 bash[30791]: cluster 2021-03-10T14:38:06.367070+0000 mgr.gibba032.whopeo (mgr.24625) 2430 : cluster [DBG] pgmap v1369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:08 gibba011 bash[31774]: cluster 2021-03-10T14:38:06.367070+0000 mgr.gibba032.whopeo (mgr.24625) 2430 : cluster [DBG] pgmap v1369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:09.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:09 gibba032 bash[24004]: cephadm 2021-03-10T14:38:08.151061+0000 mgr.gibba032.whopeo (mgr.24625) 2431 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:09.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:09 gibba011 bash[30791]: cephadm 2021-03-10T14:38:08.151061+0000 mgr.gibba032.whopeo (mgr.24625) 2431 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:09.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:09 gibba011 bash[31774]: cephadm 2021-03-10T14:38:08.151061+0000 mgr.gibba032.whopeo (mgr.24625) 2431 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:10 gibba032 bash[24004]: cluster 2021-03-10T14:38:08.367944+0000 mgr.gibba032.whopeo (mgr.24625) 2432 : cluster [DBG] pgmap v1370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:10 gibba011 bash[30791]: cluster 2021-03-10T14:38:08.367944+0000 mgr.gibba032.whopeo (mgr.24625) 2432 : cluster [DBG] pgmap v1370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:10 gibba011 bash[31774]: cluster 2021-03-10T14:38:08.367944+0000 mgr.gibba032.whopeo (mgr.24625) 2432 : cluster [DBG] pgmap v1370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:12 gibba032 bash[24004]: cluster 2021-03-10T14:38:10.369444+0000 mgr.gibba032.whopeo (mgr.24625) 2433 : cluster [DBG] pgmap v1371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:12 gibba011 bash[30791]: cluster 2021-03-10T14:38:10.369444+0000 mgr.gibba032.whopeo (mgr.24625) 2433 : cluster [DBG] pgmap v1371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:12 gibba011 bash[31774]: cluster 2021-03-10T14:38:10.369444+0000 mgr.gibba032.whopeo (mgr.24625) 2433 : cluster [DBG] pgmap v1371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:14 gibba032 bash[24004]: cluster 2021-03-10T14:38:12.371243+0000 mgr.gibba032.whopeo (mgr.24625) 2434 : cluster [DBG] pgmap v1372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:14 gibba011 bash[30791]: cluster 2021-03-10T14:38:12.371243+0000 mgr.gibba032.whopeo (mgr.24625) 2434 : cluster [DBG] pgmap v1372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:14 gibba011 bash[31774]: cluster 2021-03-10T14:38:12.371243+0000 mgr.gibba032.whopeo (mgr.24625) 2434 : cluster [DBG] pgmap v1372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:16 gibba032 bash[24004]: cluster 2021-03-10T14:38:14.372787+0000 mgr.gibba032.whopeo (mgr.24625) 2435 : cluster [DBG] pgmap v1373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:16 gibba011 bash[30791]: cluster 2021-03-10T14:38:14.372787+0000 mgr.gibba032.whopeo (mgr.24625) 2435 : cluster [DBG] pgmap v1373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:16 gibba011 bash[31774]: cluster 2021-03-10T14:38:14.372787+0000 mgr.gibba032.whopeo (mgr.24625) 2435 : cluster [DBG] pgmap v1373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:18 gibba032 bash[24004]: cluster 2021-03-10T14:38:16.373791+0000 mgr.gibba032.whopeo (mgr.24625) 2436 : cluster [DBG] pgmap v1374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:18 gibba011 bash[30791]: cluster 2021-03-10T14:38:16.373791+0000 mgr.gibba032.whopeo (mgr.24625) 2436 : cluster [DBG] pgmap v1374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:18 gibba011 bash[31774]: cluster 2021-03-10T14:38:16.373791+0000 mgr.gibba032.whopeo (mgr.24625) 2436 : cluster [DBG] pgmap v1374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:20 gibba011 bash[30791]: cluster 2021-03-10T14:38:18.375172+0000 mgr.gibba032.whopeo (mgr.24625) 2437 : cluster [DBG] pgmap v1375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:20 gibba011 bash[31774]: cluster 2021-03-10T14:38:18.375172+0000 mgr.gibba032.whopeo (mgr.24625) 2437 : cluster [DBG] pgmap v1375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:20 gibba032 bash[24004]: cluster 2021-03-10T14:38:18.375172+0000 mgr.gibba032.whopeo (mgr.24625) 2437 : cluster [DBG] pgmap v1375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:22 gibba011 bash[30791]: cluster 2021-03-10T14:38:20.376842+0000 mgr.gibba032.whopeo (mgr.24625) 2438 : cluster [DBG] pgmap v1376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:22 gibba011 bash[31774]: cluster 2021-03-10T14:38:20.376842+0000 mgr.gibba032.whopeo (mgr.24625) 2438 : cluster [DBG] pgmap v1376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:22 gibba032 bash[24004]: cluster 2021-03-10T14:38:20.376842+0000 mgr.gibba032.whopeo (mgr.24625) 2438 : cluster [DBG] pgmap v1376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:24 gibba011 bash[30791]: cluster 2021-03-10T14:38:22.378864+0000 mgr.gibba032.whopeo (mgr.24625) 2439 : cluster [DBG] pgmap v1377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:24 gibba011 bash[30791]: cephadm 2021-03-10T14:38:23.167101+0000 mgr.gibba032.whopeo (mgr.24625) 2440 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:24 gibba011 bash[31774]: cluster 2021-03-10T14:38:22.378864+0000 mgr.gibba032.whopeo (mgr.24625) 2439 : cluster [DBG] pgmap v1377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:24 gibba011 bash[31774]: cephadm 2021-03-10T14:38:23.167101+0000 mgr.gibba032.whopeo (mgr.24625) 2440 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:24 gibba032 bash[24004]: cluster 2021-03-10T14:38:22.378864+0000 mgr.gibba032.whopeo (mgr.24625) 2439 : cluster [DBG] pgmap v1377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:24 gibba032 bash[24004]: cephadm 2021-03-10T14:38:23.167101+0000 mgr.gibba032.whopeo (mgr.24625) 2440 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:26 gibba011 bash[30791]: cluster 2021-03-10T14:38:24.380114+0000 mgr.gibba032.whopeo (mgr.24625) 2441 : cluster [DBG] pgmap v1378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:26 gibba011 bash[31774]: cluster 2021-03-10T14:38:24.380114+0000 mgr.gibba032.whopeo (mgr.24625) 2441 : cluster [DBG] pgmap v1378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:26 gibba032 bash[24004]: cluster 2021-03-10T14:38:24.380114+0000 mgr.gibba032.whopeo (mgr.24625) 2441 : cluster [DBG] pgmap v1378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:27.613 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:38:27.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:27 gibba032 bash[24004]: cluster 2021-03-10T14:38:26.381655+0000 mgr.gibba032.whopeo (mgr.24625) 2442 : cluster [DBG] pgmap v1379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:27.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:27 gibba011 bash[30791]: cluster 2021-03-10T14:38:26.381655+0000 mgr.gibba032.whopeo (mgr.24625) 2442 : cluster [DBG] pgmap v1379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:27.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:27 gibba011 bash[31774]: cluster 2021-03-10T14:38:26.381655+0000 mgr.gibba032.whopeo (mgr.24625) 2442 : cluster [DBG] pgmap v1379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:27.839 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:38:27.840 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (45m) 34s ago 50m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:38:27.840 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (50m) 9m ago 50m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:38:27.840 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (45m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:38:27.840 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (45m) 9m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:38:27.840 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (44m) 34s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:38:27.841 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (44m) 34s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:38:27.841 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (44m) 9m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:38:27.841 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (44m) 34s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:38:27.841 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (50m) 34s ago 50m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:38:27.842 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (50m) 9m ago 50m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:38:27.842 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (43m) 34s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:38:27.842 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (42m) 34s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:38:27.842 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (42m) 34s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:38:27.842 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (42m) 34s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:38:27.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (41m) 9m ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:38:27.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (41m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:38:27.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (40m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:38:27.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (40m) 9m ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:38:27.844 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (50m) 9m ago 50m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:38:27.844 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 34s ago 48m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:38:28.097 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:38:28.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:38:28.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:38:28.098 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:28.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:38:28.099 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:38:28.099 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:28.099 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:38:28.099 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:38:28.099 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:28.100 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:38:28.100 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:38:28.100 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:38:28.100 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:38:28.100 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:38:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:28 gibba032 bash[24004]: audit 2021-03-10T14:38:27.600657+0000 mgr.gibba032.whopeo (mgr.24625) 2443 : audit [DBG] from='client.55678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:28 gibba032 bash[24004]: audit 2021-03-10T14:38:27.824273+0000 mgr.gibba032.whopeo (mgr.24625) 2444 : audit [DBG] from='client.45699 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:28 gibba032 bash[24004]: audit 2021-03-10T14:38:28.096767+0000 mon.c (mon.1) 48 : audit [DBG] from='client.? 172.21.2.111:0/2086212218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[30791]: audit 2021-03-10T14:38:27.600657+0000 mgr.gibba032.whopeo (mgr.24625) 2443 : audit [DBG] from='client.55678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[30791]: audit 2021-03-10T14:38:27.824273+0000 mgr.gibba032.whopeo (mgr.24625) 2444 : audit [DBG] from='client.45699 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[30791]: audit 2021-03-10T14:38:28.096767+0000 mon.c (mon.1) 48 : audit [DBG] from='client.? 172.21.2.111:0/2086212218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[31774]: audit 2021-03-10T14:38:27.600657+0000 mgr.gibba032.whopeo (mgr.24625) 2443 : audit [DBG] from='client.55678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[31774]: audit 2021-03-10T14:38:27.824273+0000 mgr.gibba032.whopeo (mgr.24625) 2444 : audit [DBG] from='client.45699 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:38:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:28 gibba011 bash[31774]: audit 2021-03-10T14:38:28.096767+0000 mon.c (mon.1) 48 : audit [DBG] from='client.? 172.21.2.111:0/2086212218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:29 gibba032 bash[24004]: cluster 2021-03-10T14:38:28.382520+0000 mgr.gibba032.whopeo (mgr.24625) 2445 : cluster [DBG] pgmap v1380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:29.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:29 gibba011 bash[30791]: cluster 2021-03-10T14:38:28.382520+0000 mgr.gibba032.whopeo (mgr.24625) 2445 : cluster [DBG] pgmap v1380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:29.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:29 gibba011 bash[31774]: cluster 2021-03-10T14:38:28.382520+0000 mgr.gibba032.whopeo (mgr.24625) 2445 : cluster [DBG] pgmap v1380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:32 gibba032 bash[24004]: cluster 2021-03-10T14:38:30.383892+0000 mgr.gibba032.whopeo (mgr.24625) 2446 : cluster [DBG] pgmap v1381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:32 gibba011 bash[30791]: cluster 2021-03-10T14:38:30.383892+0000 mgr.gibba032.whopeo (mgr.24625) 2446 : cluster [DBG] pgmap v1381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:32 gibba011 bash[31774]: cluster 2021-03-10T14:38:30.383892+0000 mgr.gibba032.whopeo (mgr.24625) 2446 : cluster [DBG] pgmap v1381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:34 gibba032 bash[24004]: cluster 2021-03-10T14:38:32.384701+0000 mgr.gibba032.whopeo (mgr.24625) 2447 : cluster [DBG] pgmap v1382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:34 gibba011 bash[30791]: cluster 2021-03-10T14:38:32.384701+0000 mgr.gibba032.whopeo (mgr.24625) 2447 : cluster [DBG] pgmap v1382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:34 gibba011 bash[31774]: cluster 2021-03-10T14:38:32.384701+0000 mgr.gibba032.whopeo (mgr.24625) 2447 : cluster [DBG] pgmap v1382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:36 gibba032 bash[24004]: cluster 2021-03-10T14:38:34.385798+0000 mgr.gibba032.whopeo (mgr.24625) 2448 : cluster [DBG] pgmap v1383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:36 gibba011 bash[30791]: cluster 2021-03-10T14:38:34.385798+0000 mgr.gibba032.whopeo (mgr.24625) 2448 : cluster [DBG] pgmap v1383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:36 gibba011 bash[31774]: cluster 2021-03-10T14:38:34.385798+0000 mgr.gibba032.whopeo (mgr.24625) 2448 : cluster [DBG] pgmap v1383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:38 gibba032 bash[24004]: cluster 2021-03-10T14:38:36.387647+0000 mgr.gibba032.whopeo (mgr.24625) 2449 : cluster [DBG] pgmap v1384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:38 gibba011 bash[30791]: cluster 2021-03-10T14:38:36.387647+0000 mgr.gibba032.whopeo (mgr.24625) 2449 : cluster [DBG] pgmap v1384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:38 gibba011 bash[31774]: cluster 2021-03-10T14:38:36.387647+0000 mgr.gibba032.whopeo (mgr.24625) 2449 : cluster [DBG] pgmap v1384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:39.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:39 gibba032 bash[24004]: cephadm 2021-03-10T14:38:38.183141+0000 mgr.gibba032.whopeo (mgr.24625) 2450 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:39.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:39 gibba011 bash[30791]: cephadm 2021-03-10T14:38:38.183141+0000 mgr.gibba032.whopeo (mgr.24625) 2450 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:39.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:39 gibba011 bash[31774]: cephadm 2021-03-10T14:38:38.183141+0000 mgr.gibba032.whopeo (mgr.24625) 2450 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:40 gibba032 bash[24004]: cluster 2021-03-10T14:38:38.389234+0000 mgr.gibba032.whopeo (mgr.24625) 2451 : cluster [DBG] pgmap v1385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:40 gibba011 bash[30791]: cluster 2021-03-10T14:38:38.389234+0000 mgr.gibba032.whopeo (mgr.24625) 2451 : cluster [DBG] pgmap v1385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:40 gibba011 bash[31774]: cluster 2021-03-10T14:38:38.389234+0000 mgr.gibba032.whopeo (mgr.24625) 2451 : cluster [DBG] pgmap v1385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:42 gibba032 bash[24004]: cluster 2021-03-10T14:38:40.390823+0000 mgr.gibba032.whopeo (mgr.24625) 2452 : cluster [DBG] pgmap v1386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:42 gibba011 bash[30791]: cluster 2021-03-10T14:38:40.390823+0000 mgr.gibba032.whopeo (mgr.24625) 2452 : cluster [DBG] pgmap v1386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:42 gibba011 bash[31774]: cluster 2021-03-10T14:38:40.390823+0000 mgr.gibba032.whopeo (mgr.24625) 2452 : cluster [DBG] pgmap v1386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:44 gibba032 bash[24004]: cluster 2021-03-10T14:38:42.392640+0000 mgr.gibba032.whopeo (mgr.24625) 2453 : cluster [DBG] pgmap v1387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:44 gibba011 bash[30791]: cluster 2021-03-10T14:38:42.392640+0000 mgr.gibba032.whopeo (mgr.24625) 2453 : cluster [DBG] pgmap v1387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:44 gibba011 bash[31774]: cluster 2021-03-10T14:38:42.392640+0000 mgr.gibba032.whopeo (mgr.24625) 2453 : cluster [DBG] pgmap v1387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:46 gibba032 bash[24004]: cluster 2021-03-10T14:38:44.393835+0000 mgr.gibba032.whopeo (mgr.24625) 2454 : cluster [DBG] pgmap v1388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:46 gibba011 bash[30791]: cluster 2021-03-10T14:38:44.393835+0000 mgr.gibba032.whopeo (mgr.24625) 2454 : cluster [DBG] pgmap v1388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:46 gibba011 bash[31774]: cluster 2021-03-10T14:38:44.393835+0000 mgr.gibba032.whopeo (mgr.24625) 2454 : cluster [DBG] pgmap v1388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:48 gibba032 bash[24004]: cluster 2021-03-10T14:38:46.395618+0000 mgr.gibba032.whopeo (mgr.24625) 2455 : cluster [DBG] pgmap v1389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:48 gibba011 bash[30791]: cluster 2021-03-10T14:38:46.395618+0000 mgr.gibba032.whopeo (mgr.24625) 2455 : cluster [DBG] pgmap v1389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:48 gibba011 bash[31774]: cluster 2021-03-10T14:38:46.395618+0000 mgr.gibba032.whopeo (mgr.24625) 2455 : cluster [DBG] pgmap v1389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:50 gibba032 bash[24004]: cluster 2021-03-10T14:38:48.397145+0000 mgr.gibba032.whopeo (mgr.24625) 2456 : cluster [DBG] pgmap v1390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:50 gibba011 bash[30791]: cluster 2021-03-10T14:38:48.397145+0000 mgr.gibba032.whopeo (mgr.24625) 2456 : cluster [DBG] pgmap v1390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:50 gibba011 bash[31774]: cluster 2021-03-10T14:38:48.397145+0000 mgr.gibba032.whopeo (mgr.24625) 2456 : cluster [DBG] pgmap v1390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:52 gibba011 bash[30791]: cluster 2021-03-10T14:38:50.398613+0000 mgr.gibba032.whopeo (mgr.24625) 2457 : cluster [DBG] pgmap v1391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:52 gibba011 bash[31774]: cluster 2021-03-10T14:38:50.398613+0000 mgr.gibba032.whopeo (mgr.24625) 2457 : cluster [DBG] pgmap v1391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:52 gibba032 bash[24004]: cluster 2021-03-10T14:38:50.398613+0000 mgr.gibba032.whopeo (mgr.24625) 2457 : cluster [DBG] pgmap v1391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:53 gibba011 bash[30791]: audit 2021-03-10T14:38:53.193514+0000 mon.a (mon.0) 1256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:38:53.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:53 gibba011 bash[31774]: audit 2021-03-10T14:38:53.193514+0000 mon.a (mon.0) 1256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:38:53.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:53 gibba032 bash[24004]: audit 2021-03-10T14:38:53.193514+0000 mon.a (mon.0) 1256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:38:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:54 gibba011 bash[30791]: cluster 2021-03-10T14:38:52.400328+0000 mgr.gibba032.whopeo (mgr.24625) 2458 : cluster [DBG] pgmap v1392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:54 gibba011 bash[30791]: cephadm 2021-03-10T14:38:53.189383+0000 mgr.gibba032.whopeo (mgr.24625) 2459 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:38:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:54 gibba011 bash[31774]: cluster 2021-03-10T14:38:52.400328+0000 mgr.gibba032.whopeo (mgr.24625) 2458 : cluster [DBG] pgmap v1392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:54 gibba011 bash[31774]: cephadm 2021-03-10T14:38:53.189383+0000 mgr.gibba032.whopeo (mgr.24625) 2459 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:38:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:54 gibba032 bash[24004]: cluster 2021-03-10T14:38:52.400328+0000 mgr.gibba032.whopeo (mgr.24625) 2458 : cluster [DBG] pgmap v1392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:54 gibba032 bash[24004]: cephadm 2021-03-10T14:38:53.189383+0000 mgr.gibba032.whopeo (mgr.24625) 2459 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:38:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:56 gibba011 bash[30791]: cluster 2021-03-10T14:38:54.401532+0000 mgr.gibba032.whopeo (mgr.24625) 2460 : cluster [DBG] pgmap v1393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:56 gibba011 bash[31774]: cluster 2021-03-10T14:38:54.401532+0000 mgr.gibba032.whopeo (mgr.24625) 2460 : cluster [DBG] pgmap v1393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:56 gibba032 bash[24004]: cluster 2021-03-10T14:38:54.401532+0000 mgr.gibba032.whopeo (mgr.24625) 2460 : cluster [DBG] pgmap v1393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.308689+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:38:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.310938+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:38:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.319146+0000 mon.a (mon.0) 1259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:38:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.319434+0000 mon.a (mon.0) 1260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.319727+0000 mon.a (mon.0) 1261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.319945+0000 mon.a (mon.0) 1262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:56.844692+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.044311+0000 mon.a (mon.0) 1264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.048014+0000 mon.a (mon.0) 1265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.051232+0000 mon.a (mon.0) 1266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.054146+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.057225+0000 mon.a (mon.0) 1268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.062913+0000 mon.a (mon.0) 1269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:57 gibba032 bash[24004]: audit 2021-03-10T14:38:57.066081+0000 mon.a (mon.0) 1270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.308689+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:38:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.310938+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:38:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.319146+0000 mon.a (mon.0) 1259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:38:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.319434+0000 mon.a (mon.0) 1260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:38:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.319727+0000 mon.a (mon.0) 1261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.319945+0000 mon.a (mon.0) 1262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:56.844692+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.044311+0000 mon.a (mon.0) 1264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.048014+0000 mon.a (mon.0) 1265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:38:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.051232+0000 mon.a (mon.0) 1266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.054146+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.057225+0000 mon.a (mon.0) 1268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.062913+0000 mon.a (mon.0) 1269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[30791]: audit 2021-03-10T14:38:57.066081+0000 mon.a (mon.0) 1270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.308689+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:38:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.310938+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:38:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.319146+0000 mon.a (mon.0) 1259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:38:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.319434+0000 mon.a (mon.0) 1260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:38:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.319727+0000 mon.a (mon.0) 1261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.319945+0000 mon.a (mon.0) 1262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:38:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:56.844692+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.044311+0000 mon.a (mon.0) 1264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:38:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.048014+0000 mon.a (mon.0) 1265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:38:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.051232+0000 mon.a (mon.0) 1266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.054146+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.057225+0000 mon.a (mon.0) 1268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.062913+0000 mon.a (mon.0) 1269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:57 gibba011 bash[31774]: audit 2021-03-10T14:38:57.066081+0000 mon.a (mon.0) 1270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:58.345 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:38:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cluster 2021-03-10T14:38:56.402156+0000 mgr.gibba032.whopeo (mgr.24625) 2461 : cluster [DBG] pgmap v1394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:56.842941+0000 mgr.gibba032.whopeo (mgr.24625) 2462 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.044520+0000 mgr.gibba032.whopeo (mgr.24625) 2463 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.047814+0000 mgr.gibba032.whopeo (mgr.24625) 2464 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.050858+0000 mgr.gibba032.whopeo (mgr.24625) 2465 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.050895+0000 mgr.gibba032.whopeo (mgr.24625) 2466 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.053802+0000 mgr.gibba032.whopeo (mgr.24625) 2467 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.053837+0000 mgr.gibba032.whopeo (mgr.24625) 2468 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.056921+0000 mgr.gibba032.whopeo (mgr.24625) 2469 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.059577+0000 mgr.gibba032.whopeo (mgr.24625) 2470 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:38:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.059613+0000 mgr.gibba032.whopeo (mgr.24625) 2471 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:38:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.062407+0000 mgr.gibba032.whopeo (mgr.24625) 2472 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:38:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.062716+0000 mgr.gibba032.whopeo (mgr.24625) 2473 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:38:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.065771+0000 mgr.gibba032.whopeo (mgr.24625) 2474 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:38:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.065807+0000 mgr.gibba032.whopeo (mgr.24625) 2475 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:38:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[30791]: cephadm 2021-03-10T14:38:57.065910+0000 mgr.gibba032.whopeo (mgr.24625) 2476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cluster 2021-03-10T14:38:56.402156+0000 mgr.gibba032.whopeo (mgr.24625) 2461 : cluster [DBG] pgmap v1394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:56.842941+0000 mgr.gibba032.whopeo (mgr.24625) 2462 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:38:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.044520+0000 mgr.gibba032.whopeo (mgr.24625) 2463 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:38:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.047814+0000 mgr.gibba032.whopeo (mgr.24625) 2464 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:38:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.050858+0000 mgr.gibba032.whopeo (mgr.24625) 2465 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:38:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.050895+0000 mgr.gibba032.whopeo (mgr.24625) 2466 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:38:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.053802+0000 mgr.gibba032.whopeo (mgr.24625) 2467 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:38:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.053837+0000 mgr.gibba032.whopeo (mgr.24625) 2468 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:38:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.056921+0000 mgr.gibba032.whopeo (mgr.24625) 2469 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:38:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.059577+0000 mgr.gibba032.whopeo (mgr.24625) 2470 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:38:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.059613+0000 mgr.gibba032.whopeo (mgr.24625) 2471 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:38:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.062407+0000 mgr.gibba032.whopeo (mgr.24625) 2472 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:38:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.062716+0000 mgr.gibba032.whopeo (mgr.24625) 2473 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:38:58.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.065771+0000 mgr.gibba032.whopeo (mgr.24625) 2474 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:38:58.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.065807+0000 mgr.gibba032.whopeo (mgr.24625) 2475 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:38:58.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:58 gibba011 bash[31774]: cephadm 2021-03-10T14:38:57.065910+0000 mgr.gibba032.whopeo (mgr.24625) 2476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:58.600 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:38:58.600 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (45m) 65s ago 50m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:38:58.601 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (50m) 1s ago 50m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:38:58.601 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (45m) 1s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:38:58.601 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (46m) 1s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:38:58.602 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (45m) 65s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:38:58.602 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (45m) 65s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:38:58.602 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (44m) 1s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:38:58.603 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (44m) 65s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:38:58.603 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (51m) 65s ago 51m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:38:58.603 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (51m) 1s ago 51m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:38:58.603 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (43m) 65s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:38:58.604 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (43m) 65s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:38:58.604 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (43m) 65s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:38:58.604 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (42m) 65s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:38:58.604 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (42m) 1s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:38:58.604 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (41m) 1s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:38:58.605 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (41m) 1s ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:38:58.605 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (40m) 1s ago 51m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:38:58.605 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (50m) 1s ago 51m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:38:58.606 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 65s ago 49m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:38:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cluster 2021-03-10T14:38:56.402156+0000 mgr.gibba032.whopeo (mgr.24625) 2461 : cluster [DBG] pgmap v1394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:38:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:56.842941+0000 mgr.gibba032.whopeo (mgr.24625) 2462 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:38:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.044520+0000 mgr.gibba032.whopeo (mgr.24625) 2463 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:38:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.047814+0000 mgr.gibba032.whopeo (mgr.24625) 2464 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:38:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.050858+0000 mgr.gibba032.whopeo (mgr.24625) 2465 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:38:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.050895+0000 mgr.gibba032.whopeo (mgr.24625) 2466 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:38:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.053802+0000 mgr.gibba032.whopeo (mgr.24625) 2467 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:38:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.053837+0000 mgr.gibba032.whopeo (mgr.24625) 2468 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:38:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.056921+0000 mgr.gibba032.whopeo (mgr.24625) 2469 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:38:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.059577+0000 mgr.gibba032.whopeo (mgr.24625) 2470 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:38:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.059613+0000 mgr.gibba032.whopeo (mgr.24625) 2471 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:38:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.062407+0000 mgr.gibba032.whopeo (mgr.24625) 2472 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:38:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.062716+0000 mgr.gibba032.whopeo (mgr.24625) 2473 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:38:58.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.065771+0000 mgr.gibba032.whopeo (mgr.24625) 2474 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:38:58.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.065807+0000 mgr.gibba032.whopeo (mgr.24625) 2475 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:38:58.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:58 gibba032 bash[24004]: cephadm 2021-03-10T14:38:57.065910+0000 mgr.gibba032.whopeo (mgr.24625) 2476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:38:58.846 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:38:58.847 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:38:58.847 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:38:58.847 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:58.847 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:38:58.848 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:38:58.848 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:58.848 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:38:58.848 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:38:58.849 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:38:58.849 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:38:58.849 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:38:58.849 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:38:58.849 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:38:58.850 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:38:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:38:59 gibba011 bash[30791]: audit 2021-03-10T14:38:58.845580+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.2.111:0/1688021673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:59.710 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:38:59 gibba011 bash[31774]: audit 2021-03-10T14:38:58.845580+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.2.111:0/1688021673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:38:59.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:38:59 gibba032 bash[24004]: audit 2021-03-10T14:38:58.845580+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.2.111:0/1688021673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[30791]: audit 2021-03-10T14:38:58.333448+0000 mgr.gibba032.whopeo (mgr.24625) 2477 : audit [DBG] from='client.55696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[30791]: cluster 2021-03-10T14:38:58.403543+0000 mgr.gibba032.whopeo (mgr.24625) 2478 : cluster [DBG] pgmap v1395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[30791]: audit 2021-03-10T14:38:58.557222+0000 mgr.gibba032.whopeo (mgr.24625) 2479 : audit [DBG] from='client.55702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[31774]: audit 2021-03-10T14:38:58.333448+0000 mgr.gibba032.whopeo (mgr.24625) 2477 : audit [DBG] from='client.55696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[31774]: cluster 2021-03-10T14:38:58.403543+0000 mgr.gibba032.whopeo (mgr.24625) 2478 : cluster [DBG] pgmap v1395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:00 gibba011 bash[31774]: audit 2021-03-10T14:38:58.557222+0000 mgr.gibba032.whopeo (mgr.24625) 2479 : audit [DBG] from='client.55702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:00 gibba032 bash[24004]: audit 2021-03-10T14:38:58.333448+0000 mgr.gibba032.whopeo (mgr.24625) 2477 : audit [DBG] from='client.55696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:00 gibba032 bash[24004]: cluster 2021-03-10T14:38:58.403543+0000 mgr.gibba032.whopeo (mgr.24625) 2478 : cluster [DBG] pgmap v1395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:00 gibba032 bash[24004]: audit 2021-03-10T14:38:58.557222+0000 mgr.gibba032.whopeo (mgr.24625) 2479 : audit [DBG] from='client.55702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:02 gibba032 bash[24004]: cluster 2021-03-10T14:39:00.404930+0000 mgr.gibba032.whopeo (mgr.24625) 2480 : cluster [DBG] pgmap v1396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:02 gibba011 bash[30791]: cluster 2021-03-10T14:39:00.404930+0000 mgr.gibba032.whopeo (mgr.24625) 2480 : cluster [DBG] pgmap v1396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:02 gibba011 bash[31774]: cluster 2021-03-10T14:39:00.404930+0000 mgr.gibba032.whopeo (mgr.24625) 2480 : cluster [DBG] pgmap v1396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:04 gibba032 bash[24004]: cluster 2021-03-10T14:39:02.406483+0000 mgr.gibba032.whopeo (mgr.24625) 2481 : cluster [DBG] pgmap v1397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:04 gibba011 bash[30791]: cluster 2021-03-10T14:39:02.406483+0000 mgr.gibba032.whopeo (mgr.24625) 2481 : cluster [DBG] pgmap v1397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:04 gibba011 bash[31774]: cluster 2021-03-10T14:39:02.406483+0000 mgr.gibba032.whopeo (mgr.24625) 2481 : cluster [DBG] pgmap v1397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:06 gibba032 bash[24004]: cluster 2021-03-10T14:39:04.407449+0000 mgr.gibba032.whopeo (mgr.24625) 2482 : cluster [DBG] pgmap v1398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:06 gibba011 bash[30791]: cluster 2021-03-10T14:39:04.407449+0000 mgr.gibba032.whopeo (mgr.24625) 2482 : cluster [DBG] pgmap v1398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:06 gibba011 bash[31774]: cluster 2021-03-10T14:39:04.407449+0000 mgr.gibba032.whopeo (mgr.24625) 2482 : cluster [DBG] pgmap v1398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:08 gibba032 bash[24004]: cluster 2021-03-10T14:39:06.409100+0000 mgr.gibba032.whopeo (mgr.24625) 2483 : cluster [DBG] pgmap v1399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:08 gibba011 bash[30791]: cluster 2021-03-10T14:39:06.409100+0000 mgr.gibba032.whopeo (mgr.24625) 2483 : cluster [DBG] pgmap v1399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:08 gibba011 bash[31774]: cluster 2021-03-10T14:39:06.409100+0000 mgr.gibba032.whopeo (mgr.24625) 2483 : cluster [DBG] pgmap v1399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:10 gibba032 bash[24004]: cluster 2021-03-10T14:39:08.410589+0000 mgr.gibba032.whopeo (mgr.24625) 2484 : cluster [DBG] pgmap v1400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:10 gibba011 bash[30791]: cluster 2021-03-10T14:39:08.410589+0000 mgr.gibba032.whopeo (mgr.24625) 2484 : cluster [DBG] pgmap v1400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:10 gibba011 bash[31774]: cluster 2021-03-10T14:39:08.410589+0000 mgr.gibba032.whopeo (mgr.24625) 2484 : cluster [DBG] pgmap v1400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:12 gibba032 bash[24004]: cluster 2021-03-10T14:39:10.412124+0000 mgr.gibba032.whopeo (mgr.24625) 2485 : cluster [DBG] pgmap v1401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:12 gibba011 bash[30791]: cluster 2021-03-10T14:39:10.412124+0000 mgr.gibba032.whopeo (mgr.24625) 2485 : cluster [DBG] pgmap v1401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:12 gibba011 bash[31774]: cluster 2021-03-10T14:39:10.412124+0000 mgr.gibba032.whopeo (mgr.24625) 2485 : cluster [DBG] pgmap v1401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:13 gibba032 bash[24004]: cephadm 2021-03-10T14:39:12.081883+0000 mgr.gibba032.whopeo (mgr.24625) 2486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:13 gibba011 bash[30791]: cephadm 2021-03-10T14:39:12.081883+0000 mgr.gibba032.whopeo (mgr.24625) 2486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:13.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:13 gibba011 bash[31774]: cephadm 2021-03-10T14:39:12.081883+0000 mgr.gibba032.whopeo (mgr.24625) 2486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:14 gibba032 bash[24004]: cluster 2021-03-10T14:39:12.413508+0000 mgr.gibba032.whopeo (mgr.24625) 2487 : cluster [DBG] pgmap v1402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:14 gibba011 bash[30791]: cluster 2021-03-10T14:39:12.413508+0000 mgr.gibba032.whopeo (mgr.24625) 2487 : cluster [DBG] pgmap v1402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:14 gibba011 bash[31774]: cluster 2021-03-10T14:39:12.413508+0000 mgr.gibba032.whopeo (mgr.24625) 2487 : cluster [DBG] pgmap v1402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:16 gibba032 bash[24004]: cluster 2021-03-10T14:39:14.414992+0000 mgr.gibba032.whopeo (mgr.24625) 2488 : cluster [DBG] pgmap v1403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:16 gibba011 bash[30791]: cluster 2021-03-10T14:39:14.414992+0000 mgr.gibba032.whopeo (mgr.24625) 2488 : cluster [DBG] pgmap v1403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:16 gibba011 bash[31774]: cluster 2021-03-10T14:39:14.414992+0000 mgr.gibba032.whopeo (mgr.24625) 2488 : cluster [DBG] pgmap v1403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:18 gibba011 bash[30791]: cluster 2021-03-10T14:39:16.416814+0000 mgr.gibba032.whopeo (mgr.24625) 2489 : cluster [DBG] pgmap v1404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:18 gibba011 bash[31774]: cluster 2021-03-10T14:39:16.416814+0000 mgr.gibba032.whopeo (mgr.24625) 2489 : cluster [DBG] pgmap v1404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:18 gibba032 bash[24004]: cluster 2021-03-10T14:39:16.416814+0000 mgr.gibba032.whopeo (mgr.24625) 2489 : cluster [DBG] pgmap v1404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:20 gibba011 bash[30791]: cluster 2021-03-10T14:39:18.418161+0000 mgr.gibba032.whopeo (mgr.24625) 2490 : cluster [DBG] pgmap v1405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:20 gibba011 bash[31774]: cluster 2021-03-10T14:39:18.418161+0000 mgr.gibba032.whopeo (mgr.24625) 2490 : cluster [DBG] pgmap v1405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:20 gibba032 bash[24004]: cluster 2021-03-10T14:39:18.418161+0000 mgr.gibba032.whopeo (mgr.24625) 2490 : cluster [DBG] pgmap v1405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:22 gibba011 bash[30791]: cluster 2021-03-10T14:39:20.419748+0000 mgr.gibba032.whopeo (mgr.24625) 2491 : cluster [DBG] pgmap v1406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:22 gibba011 bash[31774]: cluster 2021-03-10T14:39:20.419748+0000 mgr.gibba032.whopeo (mgr.24625) 2491 : cluster [DBG] pgmap v1406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:22 gibba032 bash[24004]: cluster 2021-03-10T14:39:20.419748+0000 mgr.gibba032.whopeo (mgr.24625) 2491 : cluster [DBG] pgmap v1406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:24 gibba011 bash[30791]: cluster 2021-03-10T14:39:22.421422+0000 mgr.gibba032.whopeo (mgr.24625) 2492 : cluster [DBG] pgmap v1407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:24 gibba011 bash[31774]: cluster 2021-03-10T14:39:22.421422+0000 mgr.gibba032.whopeo (mgr.24625) 2492 : cluster [DBG] pgmap v1407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:24 gibba032 bash[24004]: cluster 2021-03-10T14:39:22.421422+0000 mgr.gibba032.whopeo (mgr.24625) 2492 : cluster [DBG] pgmap v1407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:26 gibba011 bash[30791]: cluster 2021-03-10T14:39:24.422950+0000 mgr.gibba032.whopeo (mgr.24625) 2493 : cluster [DBG] pgmap v1408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:26 gibba011 bash[31774]: cluster 2021-03-10T14:39:24.422950+0000 mgr.gibba032.whopeo (mgr.24625) 2493 : cluster [DBG] pgmap v1408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:26 gibba032 bash[24004]: cluster 2021-03-10T14:39:24.422950+0000 mgr.gibba032.whopeo (mgr.24625) 2493 : cluster [DBG] pgmap v1408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:28 gibba011 bash[30791]: cluster 2021-03-10T14:39:26.424728+0000 mgr.gibba032.whopeo (mgr.24625) 2494 : cluster [DBG] pgmap v1409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:28 gibba011 bash[30791]: cephadm 2021-03-10T14:39:27.097665+0000 mgr.gibba032.whopeo (mgr.24625) 2495 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:28 gibba011 bash[31774]: cluster 2021-03-10T14:39:26.424728+0000 mgr.gibba032.whopeo (mgr.24625) 2494 : cluster [DBG] pgmap v1409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:28 gibba011 bash[31774]: cephadm 2021-03-10T14:39:27.097665+0000 mgr.gibba032.whopeo (mgr.24625) 2495 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:28 gibba032 bash[24004]: cluster 2021-03-10T14:39:26.424728+0000 mgr.gibba032.whopeo (mgr.24625) 2494 : cluster [DBG] pgmap v1409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:28 gibba032 bash[24004]: cephadm 2021-03-10T14:39:27.097665+0000 mgr.gibba032.whopeo (mgr.24625) 2495 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:29.090 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:39:29.316 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:39:29.316 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (46m) 96s ago 51m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:39:29.316 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (51m) 32s ago 51m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:39:29.317 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (46m) 32s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:39:29.317 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (46m) 32s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:39:29.317 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (45m) 96s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:39:29.317 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (45m) 96s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:39:29.317 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (45m) 32s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:39:29.318 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (45m) 96s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:39:29.318 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (51m) 96s ago 51m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:39:29.318 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (51m) 32s ago 51m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:39:29.318 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (44m) 96s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:39:29.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (44m) 96s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:39:29.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (43m) 96s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:39:29.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (43m) 96s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:39:29.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (42m) 32s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:39:29.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (42m) 32s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:39:29.320 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (41m) 32s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:39:29.320 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (41m) 32s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:39:29.320 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (51m) 32s ago 51m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:39:29.320 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 96s ago 49m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:39:29.569 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:39:29.569 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:39:29.570 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:39:29.570 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:39:29.570 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:39:29.570 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:39:29.571 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:39:29.571 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:39:29.571 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:39:29.571 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:39:29.571 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:39:29.572 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:39:29.572 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:39:29.572 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:39:29.572 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:39:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[30791]: cluster 2021-03-10T14:39:28.426114+0000 mgr.gibba032.whopeo (mgr.24625) 2496 : cluster [DBG] pgmap v1410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[30791]: audit 2021-03-10T14:39:29.077575+0000 mgr.gibba032.whopeo (mgr.24625) 2497 : audit [DBG] from='client.45729 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[30791]: audit 2021-03-10T14:39:29.302714+0000 mgr.gibba032.whopeo (mgr.24625) 2498 : audit [DBG] from='client.45735 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[30791]: audit 2021-03-10T14:39:29.567887+0000 mon.c (mon.1) 49 : audit [DBG] from='client.? 172.21.2.111:0/3755349791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[31774]: cluster 2021-03-10T14:39:28.426114+0000 mgr.gibba032.whopeo (mgr.24625) 2496 : cluster [DBG] pgmap v1410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[31774]: audit 2021-03-10T14:39:29.077575+0000 mgr.gibba032.whopeo (mgr.24625) 2497 : audit [DBG] from='client.45729 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[31774]: audit 2021-03-10T14:39:29.302714+0000 mgr.gibba032.whopeo (mgr.24625) 2498 : audit [DBG] from='client.45735 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:30 gibba011 bash[31774]: audit 2021-03-10T14:39:29.567887+0000 mon.c (mon.1) 49 : audit [DBG] from='client.? 172.21.2.111:0/3755349791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:30 gibba032 bash[24004]: cluster 2021-03-10T14:39:28.426114+0000 mgr.gibba032.whopeo (mgr.24625) 2496 : cluster [DBG] pgmap v1410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:30 gibba032 bash[24004]: audit 2021-03-10T14:39:29.077575+0000 mgr.gibba032.whopeo (mgr.24625) 2497 : audit [DBG] from='client.45729 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:30 gibba032 bash[24004]: audit 2021-03-10T14:39:29.302714+0000 mgr.gibba032.whopeo (mgr.24625) 2498 : audit [DBG] from='client.45735 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:39:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:30 gibba032 bash[24004]: audit 2021-03-10T14:39:29.567887+0000 mon.c (mon.1) 49 : audit [DBG] from='client.? 172.21.2.111:0/3755349791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:32 gibba032 bash[24004]: cluster 2021-03-10T14:39:30.427788+0000 mgr.gibba032.whopeo (mgr.24625) 2499 : cluster [DBG] pgmap v1411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:32 gibba011 bash[30791]: cluster 2021-03-10T14:39:30.427788+0000 mgr.gibba032.whopeo (mgr.24625) 2499 : cluster [DBG] pgmap v1411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:32 gibba011 bash[31774]: cluster 2021-03-10T14:39:30.427788+0000 mgr.gibba032.whopeo (mgr.24625) 2499 : cluster [DBG] pgmap v1411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:34 gibba032 bash[24004]: cluster 2021-03-10T14:39:32.429299+0000 mgr.gibba032.whopeo (mgr.24625) 2500 : cluster [DBG] pgmap v1412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:34 gibba011 bash[30791]: cluster 2021-03-10T14:39:32.429299+0000 mgr.gibba032.whopeo (mgr.24625) 2500 : cluster [DBG] pgmap v1412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:34 gibba011 bash[31774]: cluster 2021-03-10T14:39:32.429299+0000 mgr.gibba032.whopeo (mgr.24625) 2500 : cluster [DBG] pgmap v1412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:36 gibba032 bash[24004]: cluster 2021-03-10T14:39:34.430831+0000 mgr.gibba032.whopeo (mgr.24625) 2501 : cluster [DBG] pgmap v1413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:36 gibba011 bash[30791]: cluster 2021-03-10T14:39:34.430831+0000 mgr.gibba032.whopeo (mgr.24625) 2501 : cluster [DBG] pgmap v1413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:36 gibba011 bash[31774]: cluster 2021-03-10T14:39:34.430831+0000 mgr.gibba032.whopeo (mgr.24625) 2501 : cluster [DBG] pgmap v1413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:38 gibba032 bash[24004]: cluster 2021-03-10T14:39:36.432695+0000 mgr.gibba032.whopeo (mgr.24625) 2502 : cluster [DBG] pgmap v1414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:38 gibba011 bash[30791]: cluster 2021-03-10T14:39:36.432695+0000 mgr.gibba032.whopeo (mgr.24625) 2502 : cluster [DBG] pgmap v1414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:38 gibba011 bash[31774]: cluster 2021-03-10T14:39:36.432695+0000 mgr.gibba032.whopeo (mgr.24625) 2502 : cluster [DBG] pgmap v1414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:40 gibba032 bash[24004]: cluster 2021-03-10T14:39:38.434293+0000 mgr.gibba032.whopeo (mgr.24625) 2503 : cluster [DBG] pgmap v1415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:40 gibba011 bash[30791]: cluster 2021-03-10T14:39:38.434293+0000 mgr.gibba032.whopeo (mgr.24625) 2503 : cluster [DBG] pgmap v1415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:40 gibba011 bash[31774]: cluster 2021-03-10T14:39:38.434293+0000 mgr.gibba032.whopeo (mgr.24625) 2503 : cluster [DBG] pgmap v1415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:42 gibba032 bash[24004]: cluster 2021-03-10T14:39:40.435727+0000 mgr.gibba032.whopeo (mgr.24625) 2504 : cluster [DBG] pgmap v1416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:42 gibba011 bash[30791]: cluster 2021-03-10T14:39:40.435727+0000 mgr.gibba032.whopeo (mgr.24625) 2504 : cluster [DBG] pgmap v1416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:42 gibba011 bash[31774]: cluster 2021-03-10T14:39:40.435727+0000 mgr.gibba032.whopeo (mgr.24625) 2504 : cluster [DBG] pgmap v1416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:43 gibba032 bash[24004]: cephadm 2021-03-10T14:39:42.113645+0000 mgr.gibba032.whopeo (mgr.24625) 2505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:43 gibba011 bash[30791]: cephadm 2021-03-10T14:39:42.113645+0000 mgr.gibba032.whopeo (mgr.24625) 2505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:43.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:43 gibba011 bash[31774]: cephadm 2021-03-10T14:39:42.113645+0000 mgr.gibba032.whopeo (mgr.24625) 2505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:44 gibba011 bash[30791]: cluster 2021-03-10T14:39:42.437367+0000 mgr.gibba032.whopeo (mgr.24625) 2506 : cluster [DBG] pgmap v1417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:44 gibba011 bash[31774]: cluster 2021-03-10T14:39:42.437367+0000 mgr.gibba032.whopeo (mgr.24625) 2506 : cluster [DBG] pgmap v1417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:44 gibba032 bash[24004]: cluster 2021-03-10T14:39:42.437367+0000 mgr.gibba032.whopeo (mgr.24625) 2506 : cluster [DBG] pgmap v1417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:46 gibba011 bash[30791]: cluster 2021-03-10T14:39:44.438734+0000 mgr.gibba032.whopeo (mgr.24625) 2507 : cluster [DBG] pgmap v1418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:46 gibba011 bash[31774]: cluster 2021-03-10T14:39:44.438734+0000 mgr.gibba032.whopeo (mgr.24625) 2507 : cluster [DBG] pgmap v1418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:46 gibba032 bash[24004]: cluster 2021-03-10T14:39:44.438734+0000 mgr.gibba032.whopeo (mgr.24625) 2507 : cluster [DBG] pgmap v1418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:48 gibba011 bash[30791]: cluster 2021-03-10T14:39:46.440408+0000 mgr.gibba032.whopeo (mgr.24625) 2508 : cluster [DBG] pgmap v1419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:48 gibba011 bash[31774]: cluster 2021-03-10T14:39:46.440408+0000 mgr.gibba032.whopeo (mgr.24625) 2508 : cluster [DBG] pgmap v1419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:48 gibba032 bash[24004]: cluster 2021-03-10T14:39:46.440408+0000 mgr.gibba032.whopeo (mgr.24625) 2508 : cluster [DBG] pgmap v1419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:50 gibba011 bash[30791]: cluster 2021-03-10T14:39:48.441880+0000 mgr.gibba032.whopeo (mgr.24625) 2509 : cluster [DBG] pgmap v1420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:50 gibba011 bash[31774]: cluster 2021-03-10T14:39:48.441880+0000 mgr.gibba032.whopeo (mgr.24625) 2509 : cluster [DBG] pgmap v1420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:50 gibba032 bash[24004]: cluster 2021-03-10T14:39:48.441880+0000 mgr.gibba032.whopeo (mgr.24625) 2509 : cluster [DBG] pgmap v1420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:52 gibba011 bash[30791]: cluster 2021-03-10T14:39:50.443434+0000 mgr.gibba032.whopeo (mgr.24625) 2510 : cluster [DBG] pgmap v1421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:52 gibba011 bash[31774]: cluster 2021-03-10T14:39:50.443434+0000 mgr.gibba032.whopeo (mgr.24625) 2510 : cluster [DBG] pgmap v1421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:52 gibba032 bash[24004]: cluster 2021-03-10T14:39:50.443434+0000 mgr.gibba032.whopeo (mgr.24625) 2510 : cluster [DBG] pgmap v1421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:54 gibba011 bash[30791]: cluster 2021-03-10T14:39:52.445256+0000 mgr.gibba032.whopeo (mgr.24625) 2511 : cluster [DBG] pgmap v1422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:54 gibba011 bash[31774]: cluster 2021-03-10T14:39:52.445256+0000 mgr.gibba032.whopeo (mgr.24625) 2511 : cluster [DBG] pgmap v1422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:54 gibba032 bash[24004]: cluster 2021-03-10T14:39:52.445256+0000 mgr.gibba032.whopeo (mgr.24625) 2511 : cluster [DBG] pgmap v1422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:56 gibba011 bash[30791]: cluster 2021-03-10T14:39:54.446808+0000 mgr.gibba032.whopeo (mgr.24625) 2512 : cluster [DBG] pgmap v1423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:56 gibba011 bash[31774]: cluster 2021-03-10T14:39:54.446808+0000 mgr.gibba032.whopeo (mgr.24625) 2512 : cluster [DBG] pgmap v1423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:56 gibba032 bash[24004]: cluster 2021-03-10T14:39:54.446808+0000 mgr.gibba032.whopeo (mgr.24625) 2512 : cluster [DBG] pgmap v1423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.311856+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:39:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.312488+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:39:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.326700+0000 mon.a (mon.0) 1274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:39:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.326849+0000 mon.a (mon.0) 1275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.327032+0000 mon.a (mon.0) 1276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[30791]: audit 2021-03-10T14:39:56.327176+0000 mon.a (mon.0) 1277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:39:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.311856+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:39:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.312488+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:39:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.326700+0000 mon.a (mon.0) 1274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:39:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.326849+0000 mon.a (mon.0) 1275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.327032+0000 mon.a (mon.0) 1276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:57 gibba011 bash[31774]: audit 2021-03-10T14:39:56.327176+0000 mon.a (mon.0) 1277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:39:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.311856+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:39:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.312488+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:39:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.326700+0000 mon.a (mon.0) 1274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T14:39:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.326849+0000 mon.a (mon.0) 1275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.327032+0000 mon.a (mon.0) 1276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:39:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:57 gibba032 bash[24004]: audit 2021-03-10T14:39:56.327176+0000 mon.a (mon.0) 1277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:39:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: cluster 2021-03-10T14:39:56.448671+0000 mgr.gibba032.whopeo (mgr.24625) 2513 : cluster [DBG] pgmap v1424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.130140+0000 mgr.gibba032.whopeo (mgr.24625) 2514 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:39:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.131660+0000 mgr.gibba032.whopeo (mgr.24625) 2515 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:39:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.330904+0000 mon.a (mon.0) 1278 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:39:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.334087+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:39:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.335562+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.336870+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.338121+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.341243+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[30791]: audit 2021-03-10T14:39:57.343399+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: cluster 2021-03-10T14:39:56.448671+0000 mgr.gibba032.whopeo (mgr.24625) 2513 : cluster [DBG] pgmap v1424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.130140+0000 mgr.gibba032.whopeo (mgr.24625) 2514 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:39:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.131660+0000 mgr.gibba032.whopeo (mgr.24625) 2515 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:39:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.330904+0000 mon.a (mon.0) 1278 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:39:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.334087+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:39:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.335562+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.336870+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.338121+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.341243+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:58 gibba011 bash[31774]: audit 2021-03-10T14:39:57.343399+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: cluster 2021-03-10T14:39:56.448671+0000 mgr.gibba032.whopeo (mgr.24625) 2513 : cluster [DBG] pgmap v1424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.130140+0000 mgr.gibba032.whopeo (mgr.24625) 2514 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:39:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.131660+0000 mgr.gibba032.whopeo (mgr.24625) 2515 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:39:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.330904+0000 mon.a (mon.0) 1278 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:39:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.334087+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:39:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.335562+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.336870+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.338121+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.341243+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:58 gibba032 bash[24004]: audit 2021-03-10T14:39:57.343399+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:39:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.330900+0000 mgr.gibba032.whopeo (mgr.24625) 2516 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:39:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.332717+0000 mgr.gibba032.whopeo (mgr.24625) 2517 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:39:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.333998+0000 mgr.gibba032.whopeo (mgr.24625) 2518 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:39:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.334034+0000 mgr.gibba032.whopeo (mgr.24625) 2519 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:39:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.335303+0000 mgr.gibba032.whopeo (mgr.24625) 2520 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:39:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.335358+0000 mgr.gibba032.whopeo (mgr.24625) 2521 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:39:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.336413+0000 mgr.gibba032.whopeo (mgr.24625) 2522 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:39:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.337996+0000 mgr.gibba032.whopeo (mgr.24625) 2523 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:39:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.338033+0000 mgr.gibba032.whopeo (mgr.24625) 2524 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:39:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.339902+0000 mgr.gibba032.whopeo (mgr.24625) 2525 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:39:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.340298+0000 mgr.gibba032.whopeo (mgr.24625) 2526 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:39:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.342268+0000 mgr.gibba032.whopeo (mgr.24625) 2527 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:39:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.342303+0000 mgr.gibba032.whopeo (mgr.24625) 2528 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:39:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cephadm 2021-03-10T14:39:57.342408+0000 mgr.gibba032.whopeo (mgr.24625) 2529 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[30791]: cluster 2021-03-10T14:39:58.450330+0000 mgr.gibba032.whopeo (mgr.24625) 2530 : cluster [DBG] pgmap v1425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:59.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.330900+0000 mgr.gibba032.whopeo (mgr.24625) 2516 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:39:59.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.332717+0000 mgr.gibba032.whopeo (mgr.24625) 2517 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:39:59.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.333998+0000 mgr.gibba032.whopeo (mgr.24625) 2518 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:39:59.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.334034+0000 mgr.gibba032.whopeo (mgr.24625) 2519 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:39:59.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.335303+0000 mgr.gibba032.whopeo (mgr.24625) 2520 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:39:59.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.335358+0000 mgr.gibba032.whopeo (mgr.24625) 2521 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:39:59.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.336413+0000 mgr.gibba032.whopeo (mgr.24625) 2522 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:39:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.337996+0000 mgr.gibba032.whopeo (mgr.24625) 2523 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:39:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.338033+0000 mgr.gibba032.whopeo (mgr.24625) 2524 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:39:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.339902+0000 mgr.gibba032.whopeo (mgr.24625) 2525 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:39:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.340298+0000 mgr.gibba032.whopeo (mgr.24625) 2526 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:39:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.342268+0000 mgr.gibba032.whopeo (mgr.24625) 2527 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:39:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.342303+0000 mgr.gibba032.whopeo (mgr.24625) 2528 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:39:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cephadm 2021-03-10T14:39:57.342408+0000 mgr.gibba032.whopeo (mgr.24625) 2529 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:39:59 gibba011 bash[31774]: cluster 2021-03-10T14:39:58.450330+0000 mgr.gibba032.whopeo (mgr.24625) 2530 : cluster [DBG] pgmap v1425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.330900+0000 mgr.gibba032.whopeo (mgr.24625) 2516 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:39:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.332717+0000 mgr.gibba032.whopeo (mgr.24625) 2517 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:39:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.333998+0000 mgr.gibba032.whopeo (mgr.24625) 2518 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:39:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.334034+0000 mgr.gibba032.whopeo (mgr.24625) 2519 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:39:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.335303+0000 mgr.gibba032.whopeo (mgr.24625) 2520 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:39:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.335358+0000 mgr.gibba032.whopeo (mgr.24625) 2521 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:39:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.336413+0000 mgr.gibba032.whopeo (mgr.24625) 2522 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:39:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.337996+0000 mgr.gibba032.whopeo (mgr.24625) 2523 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:39:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.338033+0000 mgr.gibba032.whopeo (mgr.24625) 2524 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:39:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.339902+0000 mgr.gibba032.whopeo (mgr.24625) 2525 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:39:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.340298+0000 mgr.gibba032.whopeo (mgr.24625) 2526 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:39:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.342268+0000 mgr.gibba032.whopeo (mgr.24625) 2527 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:39:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.342303+0000 mgr.gibba032.whopeo (mgr.24625) 2528 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:39:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cephadm 2021-03-10T14:39:57.342408+0000 mgr.gibba032.whopeo (mgr.24625) 2529 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:39:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:39:59 gibba032 bash[24004]: cluster 2021-03-10T14:39:58.450330+0000 mgr.gibba032.whopeo (mgr.24625) 2530 : cluster [DBG] pgmap v1425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:39:59.811 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:40:00.031 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:40:00.032 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (46m) 2m ago 52m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:40:00.032 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (51m) 63s ago 51m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:40:00.032 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (46m) 63s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:40:00.033 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (47m) 63s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:40:00.033 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (46m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:40:00.033 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (46m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:40:00.033 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (45m) 63s ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:40:00.033 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (45m) 2m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:40:00.034 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (52m) 2m ago 52m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:40:00.034 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (52m) 63s ago 52m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:40:00.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (44m) 2m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:40:00.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:40:00.035 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:40:00.035 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (43m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:40:00.035 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (43m) 63s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:40:00.035 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (42m) 63s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:40:00.035 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (42m) 63s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:40:00.036 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (41m) 63s ago 52m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:40:00.036 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (51m) 63s ago 52m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:40:00.036 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 50m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:40:00.281 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:40:00.281 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:40:00.281 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:40:00.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:00.282 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:40:00.282 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:40:00.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:00.283 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:40:00.283 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:40:00.283 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:00.283 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:40:00.283 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:40:00.284 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:40:00.284 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:40:00.284 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:40:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[30791]: debug 2021-03-10T14:39:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:40:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:00 gibba032 bash[24004]: audit 2021-03-10T14:39:59.799106+0000 mgr.gibba032.whopeo (mgr.24625) 2531 : audit [DBG] from='client.45747 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:00 gibba032 bash[24004]: cluster 2021-03-10T14:40:00.000120+0000 mon.a (mon.0) 1285 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:40:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:00 gibba032 bash[24004]: audit 2021-03-10T14:40:00.020382+0000 mgr.gibba032.whopeo (mgr.24625) 2532 : audit [DBG] from='client.55738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:00 gibba032 bash[24004]: audit 2021-03-10T14:40:00.279977+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.2.111:0/1053776273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[30791]: audit 2021-03-10T14:39:59.799106+0000 mgr.gibba032.whopeo (mgr.24625) 2531 : audit [DBG] from='client.45747 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[30791]: cluster 2021-03-10T14:40:00.000120+0000 mon.a (mon.0) 1285 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:40:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[30791]: audit 2021-03-10T14:40:00.020382+0000 mgr.gibba032.whopeo (mgr.24625) 2532 : audit [DBG] from='client.55738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[30791]: audit 2021-03-10T14:40:00.279977+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.2.111:0/1053776273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[31774]: audit 2021-03-10T14:39:59.799106+0000 mgr.gibba032.whopeo (mgr.24625) 2531 : audit [DBG] from='client.45747 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[31774]: cluster 2021-03-10T14:40:00.000120+0000 mon.a (mon.0) 1285 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:40:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[31774]: audit 2021-03-10T14:40:00.020382+0000 mgr.gibba032.whopeo (mgr.24625) 2532 : audit [DBG] from='client.55738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:00 gibba011 bash[31774]: audit 2021-03-10T14:40:00.279977+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.2.111:0/1053776273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:02 gibba032 bash[24004]: cluster 2021-03-10T14:40:00.451803+0000 mgr.gibba032.whopeo (mgr.24625) 2533 : cluster [DBG] pgmap v1426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:02 gibba011 bash[30791]: cluster 2021-03-10T14:40:00.451803+0000 mgr.gibba032.whopeo (mgr.24625) 2533 : cluster [DBG] pgmap v1426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:02 gibba011 bash[31774]: cluster 2021-03-10T14:40:00.451803+0000 mgr.gibba032.whopeo (mgr.24625) 2533 : cluster [DBG] pgmap v1426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:04 gibba032 bash[24004]: cluster 2021-03-10T14:40:02.453465+0000 mgr.gibba032.whopeo (mgr.24625) 2534 : cluster [DBG] pgmap v1427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:04 gibba011 bash[30791]: cluster 2021-03-10T14:40:02.453465+0000 mgr.gibba032.whopeo (mgr.24625) 2534 : cluster [DBG] pgmap v1427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:04 gibba011 bash[31774]: cluster 2021-03-10T14:40:02.453465+0000 mgr.gibba032.whopeo (mgr.24625) 2534 : cluster [DBG] pgmap v1427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:06 gibba032 bash[24004]: cluster 2021-03-10T14:40:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 2535 : cluster [DBG] pgmap v1428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:06 gibba011 bash[30791]: cluster 2021-03-10T14:40:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 2535 : cluster [DBG] pgmap v1428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:06 gibba011 bash[31774]: cluster 2021-03-10T14:40:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 2535 : cluster [DBG] pgmap v1428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:08 gibba032 bash[24004]: cluster 2021-03-10T14:40:06.456832+0000 mgr.gibba032.whopeo (mgr.24625) 2536 : cluster [DBG] pgmap v1429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:08 gibba011 bash[30791]: cluster 2021-03-10T14:40:06.456832+0000 mgr.gibba032.whopeo (mgr.24625) 2536 : cluster [DBG] pgmap v1429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:08.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:08 gibba011 bash[31774]: cluster 2021-03-10T14:40:06.456832+0000 mgr.gibba032.whopeo (mgr.24625) 2536 : cluster [DBG] pgmap v1429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:10 gibba032 bash[24004]: cluster 2021-03-10T14:40:08.458408+0000 mgr.gibba032.whopeo (mgr.24625) 2537 : cluster [DBG] pgmap v1430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:10 gibba011 bash[30791]: cluster 2021-03-10T14:40:08.458408+0000 mgr.gibba032.whopeo (mgr.24625) 2537 : cluster [DBG] pgmap v1430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:10 gibba011 bash[31774]: cluster 2021-03-10T14:40:08.458408+0000 mgr.gibba032.whopeo (mgr.24625) 2537 : cluster [DBG] pgmap v1430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:12 gibba011 bash[30791]: cluster 2021-03-10T14:40:10.459692+0000 mgr.gibba032.whopeo (mgr.24625) 2538 : cluster [DBG] pgmap v1431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:12 gibba011 bash[31774]: cluster 2021-03-10T14:40:10.459692+0000 mgr.gibba032.whopeo (mgr.24625) 2538 : cluster [DBG] pgmap v1431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:12 gibba032 bash[24004]: cluster 2021-03-10T14:40:10.459692+0000 mgr.gibba032.whopeo (mgr.24625) 2538 : cluster [DBG] pgmap v1431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:14 gibba011 bash[30791]: cephadm 2021-03-10T14:40:12.358456+0000 mgr.gibba032.whopeo (mgr.24625) 2539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:14 gibba011 bash[30791]: cluster 2021-03-10T14:40:12.461469+0000 mgr.gibba032.whopeo (mgr.24625) 2540 : cluster [DBG] pgmap v1432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:14 gibba011 bash[31774]: cephadm 2021-03-10T14:40:12.358456+0000 mgr.gibba032.whopeo (mgr.24625) 2539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:14 gibba011 bash[31774]: cluster 2021-03-10T14:40:12.461469+0000 mgr.gibba032.whopeo (mgr.24625) 2540 : cluster [DBG] pgmap v1432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:14 gibba032 bash[24004]: cephadm 2021-03-10T14:40:12.358456+0000 mgr.gibba032.whopeo (mgr.24625) 2539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:14 gibba032 bash[24004]: cluster 2021-03-10T14:40:12.461469+0000 mgr.gibba032.whopeo (mgr.24625) 2540 : cluster [DBG] pgmap v1432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:16 gibba011 bash[30791]: cluster 2021-03-10T14:40:14.462951+0000 mgr.gibba032.whopeo (mgr.24625) 2541 : cluster [DBG] pgmap v1433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:16 gibba011 bash[31774]: cluster 2021-03-10T14:40:14.462951+0000 mgr.gibba032.whopeo (mgr.24625) 2541 : cluster [DBG] pgmap v1433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:16 gibba032 bash[24004]: cluster 2021-03-10T14:40:14.462951+0000 mgr.gibba032.whopeo (mgr.24625) 2541 : cluster [DBG] pgmap v1433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:18 gibba011 bash[30791]: cluster 2021-03-10T14:40:16.464626+0000 mgr.gibba032.whopeo (mgr.24625) 2542 : cluster [DBG] pgmap v1434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:18 gibba011 bash[31774]: cluster 2021-03-10T14:40:16.464626+0000 mgr.gibba032.whopeo (mgr.24625) 2542 : cluster [DBG] pgmap v1434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:18 gibba032 bash[24004]: cluster 2021-03-10T14:40:16.464626+0000 mgr.gibba032.whopeo (mgr.24625) 2542 : cluster [DBG] pgmap v1434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:20 gibba011 bash[30791]: cluster 2021-03-10T14:40:18.466242+0000 mgr.gibba032.whopeo (mgr.24625) 2543 : cluster [DBG] pgmap v1435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:20 gibba011 bash[31774]: cluster 2021-03-10T14:40:18.466242+0000 mgr.gibba032.whopeo (mgr.24625) 2543 : cluster [DBG] pgmap v1435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:20 gibba032 bash[24004]: cluster 2021-03-10T14:40:18.466242+0000 mgr.gibba032.whopeo (mgr.24625) 2543 : cluster [DBG] pgmap v1435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:22 gibba011 bash[30791]: cluster 2021-03-10T14:40:20.467707+0000 mgr.gibba032.whopeo (mgr.24625) 2544 : cluster [DBG] pgmap v1436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:22 gibba011 bash[31774]: cluster 2021-03-10T14:40:20.467707+0000 mgr.gibba032.whopeo (mgr.24625) 2544 : cluster [DBG] pgmap v1436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:22 gibba032 bash[24004]: cluster 2021-03-10T14:40:20.467707+0000 mgr.gibba032.whopeo (mgr.24625) 2544 : cluster [DBG] pgmap v1436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:24 gibba011 bash[30791]: cluster 2021-03-10T14:40:22.469237+0000 mgr.gibba032.whopeo (mgr.24625) 2545 : cluster [DBG] pgmap v1437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:24 gibba011 bash[31774]: cluster 2021-03-10T14:40:22.469237+0000 mgr.gibba032.whopeo (mgr.24625) 2545 : cluster [DBG] pgmap v1437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:24 gibba032 bash[24004]: cluster 2021-03-10T14:40:22.469237+0000 mgr.gibba032.whopeo (mgr.24625) 2545 : cluster [DBG] pgmap v1437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:26 gibba011 bash[30791]: cluster 2021-03-10T14:40:24.470754+0000 mgr.gibba032.whopeo (mgr.24625) 2546 : cluster [DBG] pgmap v1438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:26 gibba011 bash[31774]: cluster 2021-03-10T14:40:24.470754+0000 mgr.gibba032.whopeo (mgr.24625) 2546 : cluster [DBG] pgmap v1438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:26 gibba032 bash[24004]: cluster 2021-03-10T14:40:24.470754+0000 mgr.gibba032.whopeo (mgr.24625) 2546 : cluster [DBG] pgmap v1438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:28 gibba011 bash[30791]: cluster 2021-03-10T14:40:26.472605+0000 mgr.gibba032.whopeo (mgr.24625) 2547 : cluster [DBG] pgmap v1439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:28 gibba011 bash[31774]: cluster 2021-03-10T14:40:26.472605+0000 mgr.gibba032.whopeo (mgr.24625) 2547 : cluster [DBG] pgmap v1439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:28 gibba032 bash[24004]: cluster 2021-03-10T14:40:26.472605+0000 mgr.gibba032.whopeo (mgr.24625) 2547 : cluster [DBG] pgmap v1439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:29 gibba032 bash[24004]: cephadm 2021-03-10T14:40:27.374101+0000 mgr.gibba032.whopeo (mgr.24625) 2548 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:29.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:29 gibba011 bash[30791]: cephadm 2021-03-10T14:40:27.374101+0000 mgr.gibba032.whopeo (mgr.24625) 2548 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:29.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:29 gibba011 bash[31774]: cephadm 2021-03-10T14:40:27.374101+0000 mgr.gibba032.whopeo (mgr.24625) 2548 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:30.526 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:40:30.752 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:40:30.752 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (47m) 2m ago 52m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:40:30.752 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (52m) 93s ago 52m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:40:30.753 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (47m) 93s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:40:30.753 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (47m) 93s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:40:30.753 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (46m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:40:30.753 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (46m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:40:30.754 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (46m) 93s ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:40:30.754 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (46m) 2m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:40:30.754 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (52m) 2m ago 52m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:40:30.754 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (52m) 93s ago 52m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:40:30.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (45m) 2m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:40:30.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (45m) 2m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:40:30.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:40:30.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:40:30.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (43m) 93s ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:40:30.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (43m) 93s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:40:30.756 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (42m) 93s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:40:30.756 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (42m) 93s ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:40:30.756 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (52m) 93s ago 52m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:40:30.756 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 50m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:40:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:30 gibba032 bash[24004]: cluster 2021-03-10T14:40:28.473965+0000 mgr.gibba032.whopeo (mgr.24625) 2549 : cluster [DBG] pgmap v1440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:30 gibba011 bash[30791]: cluster 2021-03-10T14:40:28.473965+0000 mgr.gibba032.whopeo (mgr.24625) 2549 : cluster [DBG] pgmap v1440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:30 gibba011 bash[31774]: cluster 2021-03-10T14:40:28.473965+0000 mgr.gibba032.whopeo (mgr.24625) 2549 : cluster [DBG] pgmap v1440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:31.001 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:40:31.002 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:40:31.002 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:40:31.002 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:31.002 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:40:31.002 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:40:31.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:31.003 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:40:31.003 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:40:31.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:40:31.003 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:40:31.004 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:40:31.004 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:40:31.004 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:40:31.004 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:40:31.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:31 gibba032 bash[24004]: cluster 2021-03-10T14:40:30.475410+0000 mgr.gibba032.whopeo (mgr.24625) 2550 : cluster [DBG] pgmap v1441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:31.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:31 gibba032 bash[24004]: audit 2021-03-10T14:40:30.513353+0000 mgr.gibba032.whopeo (mgr.24625) 2551 : audit [DBG] from='client.45765 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:31 gibba032 bash[24004]: audit 2021-03-10T14:40:30.735659+0000 mgr.gibba032.whopeo (mgr.24625) 2552 : audit [DBG] from='client.55756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:31 gibba032 bash[24004]: audit 2021-03-10T14:40:31.000197+0000 mon.c (mon.1) 50 : audit [DBG] from='client.? 172.21.2.111:0/504986775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[30791]: cluster 2021-03-10T14:40:30.475410+0000 mgr.gibba032.whopeo (mgr.24625) 2550 : cluster [DBG] pgmap v1441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[30791]: audit 2021-03-10T14:40:30.513353+0000 mgr.gibba032.whopeo (mgr.24625) 2551 : audit [DBG] from='client.45765 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[30791]: audit 2021-03-10T14:40:30.735659+0000 mgr.gibba032.whopeo (mgr.24625) 2552 : audit [DBG] from='client.55756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[30791]: audit 2021-03-10T14:40:31.000197+0000 mon.c (mon.1) 50 : audit [DBG] from='client.? 172.21.2.111:0/504986775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[31774]: cluster 2021-03-10T14:40:30.475410+0000 mgr.gibba032.whopeo (mgr.24625) 2550 : cluster [DBG] pgmap v1441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[31774]: audit 2021-03-10T14:40:30.513353+0000 mgr.gibba032.whopeo (mgr.24625) 2551 : audit [DBG] from='client.45765 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[31774]: audit 2021-03-10T14:40:30.735659+0000 mgr.gibba032.whopeo (mgr.24625) 2552 : audit [DBG] from='client.55756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:40:31.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:31 gibba011 bash[31774]: audit 2021-03-10T14:40:31.000197+0000 mon.c (mon.1) 50 : audit [DBG] from='client.? 172.21.2.111:0/504986775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:33 gibba011 bash[30791]: cluster 2021-03-10T14:40:32.476968+0000 mgr.gibba032.whopeo (mgr.24625) 2553 : cluster [DBG] pgmap v1442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:33 gibba011 bash[31774]: cluster 2021-03-10T14:40:32.476968+0000 mgr.gibba032.whopeo (mgr.24625) 2553 : cluster [DBG] pgmap v1442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:34.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:33 gibba032 bash[24004]: cluster 2021-03-10T14:40:32.476968+0000 mgr.gibba032.whopeo (mgr.24625) 2553 : cluster [DBG] pgmap v1442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:35 gibba011 bash[30791]: cluster 2021-03-10T14:40:34.478255+0000 mgr.gibba032.whopeo (mgr.24625) 2554 : cluster [DBG] pgmap v1443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:35.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:35 gibba011 bash[31774]: cluster 2021-03-10T14:40:34.478255+0000 mgr.gibba032.whopeo (mgr.24625) 2554 : cluster [DBG] pgmap v1443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:36.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:35 gibba032 bash[24004]: cluster 2021-03-10T14:40:34.478255+0000 mgr.gibba032.whopeo (mgr.24625) 2554 : cluster [DBG] pgmap v1443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:38 gibba032 bash[24004]: cluster 2021-03-10T14:40:36.480086+0000 mgr.gibba032.whopeo (mgr.24625) 2555 : cluster [DBG] pgmap v1444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:38 gibba011 bash[30791]: cluster 2021-03-10T14:40:36.480086+0000 mgr.gibba032.whopeo (mgr.24625) 2555 : cluster [DBG] pgmap v1444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:38 gibba011 bash[31774]: cluster 2021-03-10T14:40:36.480086+0000 mgr.gibba032.whopeo (mgr.24625) 2555 : cluster [DBG] pgmap v1444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:40 gibba032 bash[24004]: cluster 2021-03-10T14:40:38.481695+0000 mgr.gibba032.whopeo (mgr.24625) 2556 : cluster [DBG] pgmap v1445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:40 gibba011 bash[30791]: cluster 2021-03-10T14:40:38.481695+0000 mgr.gibba032.whopeo (mgr.24625) 2556 : cluster [DBG] pgmap v1445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:40 gibba011 bash[31774]: cluster 2021-03-10T14:40:38.481695+0000 mgr.gibba032.whopeo (mgr.24625) 2556 : cluster [DBG] pgmap v1445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:42 gibba011 bash[30791]: cluster 2021-03-10T14:40:40.483031+0000 mgr.gibba032.whopeo (mgr.24625) 2557 : cluster [DBG] pgmap v1446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:42 gibba011 bash[31774]: cluster 2021-03-10T14:40:40.483031+0000 mgr.gibba032.whopeo (mgr.24625) 2557 : cluster [DBG] pgmap v1446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:42 gibba032 bash[24004]: cluster 2021-03-10T14:40:40.483031+0000 mgr.gibba032.whopeo (mgr.24625) 2557 : cluster [DBG] pgmap v1446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:44 gibba011 bash[30791]: cephadm 2021-03-10T14:40:42.382490+0000 mgr.gibba032.whopeo (mgr.24625) 2558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:44 gibba011 bash[30791]: cluster 2021-03-10T14:40:42.484876+0000 mgr.gibba032.whopeo (mgr.24625) 2559 : cluster [DBG] pgmap v1447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:44 gibba011 bash[31774]: cephadm 2021-03-10T14:40:42.382490+0000 mgr.gibba032.whopeo (mgr.24625) 2558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:44 gibba011 bash[31774]: cluster 2021-03-10T14:40:42.484876+0000 mgr.gibba032.whopeo (mgr.24625) 2559 : cluster [DBG] pgmap v1447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:44 gibba032 bash[24004]: cephadm 2021-03-10T14:40:42.382490+0000 mgr.gibba032.whopeo (mgr.24625) 2558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:44 gibba032 bash[24004]: cluster 2021-03-10T14:40:42.484876+0000 mgr.gibba032.whopeo (mgr.24625) 2559 : cluster [DBG] pgmap v1447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:46 gibba011 bash[30791]: cluster 2021-03-10T14:40:44.486371+0000 mgr.gibba032.whopeo (mgr.24625) 2560 : cluster [DBG] pgmap v1448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:46 gibba011 bash[31774]: cluster 2021-03-10T14:40:44.486371+0000 mgr.gibba032.whopeo (mgr.24625) 2560 : cluster [DBG] pgmap v1448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:46 gibba032 bash[24004]: cluster 2021-03-10T14:40:44.486371+0000 mgr.gibba032.whopeo (mgr.24625) 2560 : cluster [DBG] pgmap v1448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:48 gibba011 bash[30791]: cluster 2021-03-10T14:40:46.486915+0000 mgr.gibba032.whopeo (mgr.24625) 2561 : cluster [DBG] pgmap v1449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:48 gibba011 bash[31774]: cluster 2021-03-10T14:40:46.486915+0000 mgr.gibba032.whopeo (mgr.24625) 2561 : cluster [DBG] pgmap v1449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:48 gibba032 bash[24004]: cluster 2021-03-10T14:40:46.486915+0000 mgr.gibba032.whopeo (mgr.24625) 2561 : cluster [DBG] pgmap v1449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:50 gibba011 bash[30791]: cluster 2021-03-10T14:40:48.488425+0000 mgr.gibba032.whopeo (mgr.24625) 2562 : cluster [DBG] pgmap v1450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:50 gibba011 bash[31774]: cluster 2021-03-10T14:40:48.488425+0000 mgr.gibba032.whopeo (mgr.24625) 2562 : cluster [DBG] pgmap v1450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:50 gibba032 bash[24004]: cluster 2021-03-10T14:40:48.488425+0000 mgr.gibba032.whopeo (mgr.24625) 2562 : cluster [DBG] pgmap v1450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:52 gibba011 bash[30791]: cluster 2021-03-10T14:40:50.489843+0000 mgr.gibba032.whopeo (mgr.24625) 2563 : cluster [DBG] pgmap v1451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:52 gibba011 bash[31774]: cluster 2021-03-10T14:40:50.489843+0000 mgr.gibba032.whopeo (mgr.24625) 2563 : cluster [DBG] pgmap v1451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:52 gibba032 bash[24004]: cluster 2021-03-10T14:40:50.489843+0000 mgr.gibba032.whopeo (mgr.24625) 2563 : cluster [DBG] pgmap v1451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:54 gibba011 bash[30791]: cluster 2021-03-10T14:40:52.490624+0000 mgr.gibba032.whopeo (mgr.24625) 2564 : cluster [DBG] pgmap v1452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:54 gibba011 bash[31774]: cluster 2021-03-10T14:40:52.490624+0000 mgr.gibba032.whopeo (mgr.24625) 2564 : cluster [DBG] pgmap v1452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:54 gibba032 bash[24004]: cluster 2021-03-10T14:40:52.490624+0000 mgr.gibba032.whopeo (mgr.24625) 2564 : cluster [DBG] pgmap v1452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:56 gibba011 bash[30791]: cluster 2021-03-10T14:40:54.491997+0000 mgr.gibba032.whopeo (mgr.24625) 2565 : cluster [DBG] pgmap v1453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:56 gibba011 bash[31774]: cluster 2021-03-10T14:40:54.491997+0000 mgr.gibba032.whopeo (mgr.24625) 2565 : cluster [DBG] pgmap v1453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:56 gibba032 bash[24004]: cluster 2021-03-10T14:40:54.491997+0000 mgr.gibba032.whopeo (mgr.24625) 2565 : cluster [DBG] pgmap v1453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.316545+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:40:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.317058+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:40:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.334842+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:40:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.335610+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.336266+0000 mon.a (mon.0) 1291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[30791]: audit 2021-03-10T14:40:56.336926+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:40:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.316545+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:40:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.317058+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:40:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.334842+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:40:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.335610+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.336266+0000 mon.a (mon.0) 1291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:57 gibba011 bash[31774]: audit 2021-03-10T14:40:56.336926+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:40:57.624 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.316545+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:40:57.625 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.317058+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:40:57.625 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.334842+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T14:40:57.625 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.335610+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.625 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.336266+0000 mon.a (mon.0) 1291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:40:57.626 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:57 gibba032 bash[24004]: audit 2021-03-10T14:40:56.336926+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:40:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: cluster 2021-03-10T14:40:56.493759+0000 mgr.gibba032.whopeo (mgr.24625) 2566 : cluster [DBG] pgmap v1454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.612324+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:40:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.615276+0000 mon.a (mon.0) 1294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:40:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.616606+0000 mon.a (mon.0) 1295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.617707+0000 mon.a (mon.0) 1296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.618845+0000 mon.a (mon.0) 1297 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.621177+0000 mon.a (mon.0) 1298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:58 gibba032 bash[24004]: audit 2021-03-10T14:40:57.622667+0000 mon.a (mon.0) 1299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: cluster 2021-03-10T14:40:56.493759+0000 mgr.gibba032.whopeo (mgr.24625) 2566 : cluster [DBG] pgmap v1454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.612324+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:40:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.615276+0000 mon.a (mon.0) 1294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:40:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.616606+0000 mon.a (mon.0) 1295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.617707+0000 mon.a (mon.0) 1296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.618845+0000 mon.a (mon.0) 1297 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.621177+0000 mon.a (mon.0) 1298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[30791]: audit 2021-03-10T14:40:57.622667+0000 mon.a (mon.0) 1299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: cluster 2021-03-10T14:40:56.493759+0000 mgr.gibba032.whopeo (mgr.24625) 2566 : cluster [DBG] pgmap v1454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:40:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.612324+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:40:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.615276+0000 mon.a (mon.0) 1294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:40:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.616606+0000 mon.a (mon.0) 1295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.617707+0000 mon.a (mon.0) 1296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.618845+0000 mon.a (mon.0) 1297 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.621177+0000 mon.a (mon.0) 1298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:58 gibba011 bash[31774]: audit 2021-03-10T14:40:57.622667+0000 mon.a (mon.0) 1299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:40:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.398994+0000 mgr.gibba032.whopeo (mgr.24625) 2567 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:40:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.400180+0000 mgr.gibba032.whopeo (mgr.24625) 2568 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:40:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.612145+0000 mgr.gibba032.whopeo (mgr.24625) 2569 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:40:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.613697+0000 mgr.gibba032.whopeo (mgr.24625) 2570 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:40:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.614830+0000 mgr.gibba032.whopeo (mgr.24625) 2571 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:40:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.614869+0000 mgr.gibba032.whopeo (mgr.24625) 2572 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:40:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.615953+0000 mgr.gibba032.whopeo (mgr.24625) 2573 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:40:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.615998+0000 mgr.gibba032.whopeo (mgr.24625) 2574 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:40:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.616984+0000 mgr.gibba032.whopeo (mgr.24625) 2575 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:40:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.618109+0000 mgr.gibba032.whopeo (mgr.24625) 2576 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:40:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.618148+0000 mgr.gibba032.whopeo (mgr.24625) 2577 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:40:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.619440+0000 mgr.gibba032.whopeo (mgr.24625) 2578 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:40:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.619765+0000 mgr.gibba032.whopeo (mgr.24625) 2579 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:40:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.621023+0000 mgr.gibba032.whopeo (mgr.24625) 2580 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:40:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.621065+0000 mgr.gibba032.whopeo (mgr.24625) 2581 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:40:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:40:59 gibba032 bash[24004]: cephadm 2021-03-10T14:40:57.621170+0000 mgr.gibba032.whopeo (mgr.24625) 2582 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.398994+0000 mgr.gibba032.whopeo (mgr.24625) 2567 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:40:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.400180+0000 mgr.gibba032.whopeo (mgr.24625) 2568 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:40:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.612145+0000 mgr.gibba032.whopeo (mgr.24625) 2569 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:40:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.613697+0000 mgr.gibba032.whopeo (mgr.24625) 2570 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:40:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.614830+0000 mgr.gibba032.whopeo (mgr.24625) 2571 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:40:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.614869+0000 mgr.gibba032.whopeo (mgr.24625) 2572 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:40:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.615953+0000 mgr.gibba032.whopeo (mgr.24625) 2573 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:40:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.615998+0000 mgr.gibba032.whopeo (mgr.24625) 2574 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:40:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.616984+0000 mgr.gibba032.whopeo (mgr.24625) 2575 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:40:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.618109+0000 mgr.gibba032.whopeo (mgr.24625) 2576 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:40:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.618148+0000 mgr.gibba032.whopeo (mgr.24625) 2577 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:40:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.619440+0000 mgr.gibba032.whopeo (mgr.24625) 2578 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:40:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.619765+0000 mgr.gibba032.whopeo (mgr.24625) 2579 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:40:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.621023+0000 mgr.gibba032.whopeo (mgr.24625) 2580 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:40:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.621065+0000 mgr.gibba032.whopeo (mgr.24625) 2581 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:40:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[30791]: cephadm 2021-03-10T14:40:57.621170+0000 mgr.gibba032.whopeo (mgr.24625) 2582 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:40:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.398994+0000 mgr.gibba032.whopeo (mgr.24625) 2567 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:40:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.400180+0000 mgr.gibba032.whopeo (mgr.24625) 2568 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:40:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.612145+0000 mgr.gibba032.whopeo (mgr.24625) 2569 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:40:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.613697+0000 mgr.gibba032.whopeo (mgr.24625) 2570 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:40:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.614830+0000 mgr.gibba032.whopeo (mgr.24625) 2571 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:40:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.614869+0000 mgr.gibba032.whopeo (mgr.24625) 2572 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:40:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.615953+0000 mgr.gibba032.whopeo (mgr.24625) 2573 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:40:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.615998+0000 mgr.gibba032.whopeo (mgr.24625) 2574 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:40:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.616984+0000 mgr.gibba032.whopeo (mgr.24625) 2575 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:40:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.618109+0000 mgr.gibba032.whopeo (mgr.24625) 2576 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:40:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.618148+0000 mgr.gibba032.whopeo (mgr.24625) 2577 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:40:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.619440+0000 mgr.gibba032.whopeo (mgr.24625) 2578 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:40:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.619765+0000 mgr.gibba032.whopeo (mgr.24625) 2579 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:40:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.621023+0000 mgr.gibba032.whopeo (mgr.24625) 2580 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:40:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.621065+0000 mgr.gibba032.whopeo (mgr.24625) 2581 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:40:59.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:40:59 gibba011 bash[31774]: cephadm 2021-03-10T14:40:57.621170+0000 mgr.gibba032.whopeo (mgr.24625) 2582 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:00 gibba032 bash[24004]: cluster 2021-03-10T14:40:58.494303+0000 mgr.gibba032.whopeo (mgr.24625) 2583 : cluster [DBG] pgmap v1455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:00 gibba011 bash[30791]: cluster 2021-03-10T14:40:58.494303+0000 mgr.gibba032.whopeo (mgr.24625) 2583 : cluster [DBG] pgmap v1455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:00 gibba011 bash[31774]: cluster 2021-03-10T14:40:58.494303+0000 mgr.gibba032.whopeo (mgr.24625) 2583 : cluster [DBG] pgmap v1455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:01.243 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:41:01.491 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:41:01.491 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (47m) 3m ago 53m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:41:01.491 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (52m) 2m ago 52m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:41:01.492 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (48m) 2m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:41:01.492 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (48m) 2m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:41:01.492 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (47m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:41:01.492 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (47m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:41:01.492 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (46m) 2m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:41:01.493 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (46m) 3m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:41:01.493 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (53m) 3m ago 53m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:41:01.493 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (53m) 2m ago 53m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:41:01.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (45m) 3m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:41:01.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:41:01.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:41:01.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (44m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:41:01.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:41:01.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (43m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:41:01.495 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (43m) 2m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:41:01.495 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (42m) 2m ago 53m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:41:01.495 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (52m) 2m ago 53m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:41:01.495 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 51m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:41:01.740 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:41:01.740 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:41:01.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:41:01.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:01.741 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:41:01.741 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:41:01.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:01.742 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:41:01.742 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:41:01.742 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:01.742 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:41:01.742 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:41:01.743 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:41:01.743 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:41:01.743 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:41:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:02 gibba032 bash[24004]: cluster 2021-03-10T14:41:00.495719+0000 mgr.gibba032.whopeo (mgr.24625) 2584 : cluster [DBG] pgmap v1456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:02 gibba032 bash[24004]: audit 2021-03-10T14:41:01.230037+0000 mgr.gibba032.whopeo (mgr.24625) 2585 : audit [DBG] from='client.45783 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:02 gibba032 bash[24004]: audit 2021-03-10T14:41:01.738629+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.2.111:0/312298277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[30791]: cluster 2021-03-10T14:41:00.495719+0000 mgr.gibba032.whopeo (mgr.24625) 2584 : cluster [DBG] pgmap v1456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[30791]: audit 2021-03-10T14:41:01.230037+0000 mgr.gibba032.whopeo (mgr.24625) 2585 : audit [DBG] from='client.45783 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[30791]: audit 2021-03-10T14:41:01.738629+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.2.111:0/312298277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[31774]: cluster 2021-03-10T14:41:00.495719+0000 mgr.gibba032.whopeo (mgr.24625) 2584 : cluster [DBG] pgmap v1456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[31774]: audit 2021-03-10T14:41:01.230037+0000 mgr.gibba032.whopeo (mgr.24625) 2585 : audit [DBG] from='client.45783 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:02 gibba011 bash[31774]: audit 2021-03-10T14:41:01.738629+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.2.111:0/312298277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:03 gibba011 bash[30791]: audit 2021-03-10T14:41:01.458961+0000 mgr.gibba032.whopeo (mgr.24625) 2586 : audit [DBG] from='client.55774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:03 gibba011 bash[31774]: audit 2021-03-10T14:41:01.458961+0000 mgr.gibba032.whopeo (mgr.24625) 2586 : audit [DBG] from='client.55774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:03 gibba032 bash[24004]: audit 2021-03-10T14:41:01.458961+0000 mgr.gibba032.whopeo (mgr.24625) 2586 : audit [DBG] from='client.55774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:04 gibba011 bash[30791]: cluster 2021-03-10T14:41:02.497440+0000 mgr.gibba032.whopeo (mgr.24625) 2587 : cluster [DBG] pgmap v1457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:04 gibba011 bash[31774]: cluster 2021-03-10T14:41:02.497440+0000 mgr.gibba032.whopeo (mgr.24625) 2587 : cluster [DBG] pgmap v1457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:04 gibba032 bash[24004]: cluster 2021-03-10T14:41:02.497440+0000 mgr.gibba032.whopeo (mgr.24625) 2587 : cluster [DBG] pgmap v1457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:06 gibba011 bash[30791]: cluster 2021-03-10T14:41:04.498182+0000 mgr.gibba032.whopeo (mgr.24625) 2588 : cluster [DBG] pgmap v1458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:06 gibba011 bash[31774]: cluster 2021-03-10T14:41:04.498182+0000 mgr.gibba032.whopeo (mgr.24625) 2588 : cluster [DBG] pgmap v1458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:06 gibba032 bash[24004]: cluster 2021-03-10T14:41:04.498182+0000 mgr.gibba032.whopeo (mgr.24625) 2588 : cluster [DBG] pgmap v1458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:08.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:08 gibba011 bash[30791]: cluster 2021-03-10T14:41:06.499677+0000 mgr.gibba032.whopeo (mgr.24625) 2589 : cluster [DBG] pgmap v1459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:08 gibba011 bash[31774]: cluster 2021-03-10T14:41:06.499677+0000 mgr.gibba032.whopeo (mgr.24625) 2589 : cluster [DBG] pgmap v1459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:08 gibba032 bash[24004]: cluster 2021-03-10T14:41:06.499677+0000 mgr.gibba032.whopeo (mgr.24625) 2589 : cluster [DBG] pgmap v1459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:10 gibba011 bash[30791]: cluster 2021-03-10T14:41:08.501348+0000 mgr.gibba032.whopeo (mgr.24625) 2590 : cluster [DBG] pgmap v1460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:10 gibba011 bash[31774]: cluster 2021-03-10T14:41:08.501348+0000 mgr.gibba032.whopeo (mgr.24625) 2590 : cluster [DBG] pgmap v1460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:10 gibba032 bash[24004]: cluster 2021-03-10T14:41:08.501348+0000 mgr.gibba032.whopeo (mgr.24625) 2590 : cluster [DBG] pgmap v1460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:12 gibba011 bash[30791]: cluster 2021-03-10T14:41:10.502098+0000 mgr.gibba032.whopeo (mgr.24625) 2591 : cluster [DBG] pgmap v1461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:12 gibba011 bash[31774]: cluster 2021-03-10T14:41:10.502098+0000 mgr.gibba032.whopeo (mgr.24625) 2591 : cluster [DBG] pgmap v1461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:12.629 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:12 gibba032 bash[24004]: cluster 2021-03-10T14:41:10.502098+0000 mgr.gibba032.whopeo (mgr.24625) 2591 : cluster [DBG] pgmap v1461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:14 gibba011 bash[30791]: cluster 2021-03-10T14:41:12.503715+0000 mgr.gibba032.whopeo (mgr.24625) 2592 : cluster [DBG] pgmap v1462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:14 gibba011 bash[30791]: cephadm 2021-03-10T14:41:12.624960+0000 mgr.gibba032.whopeo (mgr.24625) 2593 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:14 gibba011 bash[31774]: cluster 2021-03-10T14:41:12.503715+0000 mgr.gibba032.whopeo (mgr.24625) 2592 : cluster [DBG] pgmap v1462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:14 gibba011 bash[31774]: cephadm 2021-03-10T14:41:12.624960+0000 mgr.gibba032.whopeo (mgr.24625) 2593 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:14 gibba032 bash[24004]: cluster 2021-03-10T14:41:12.503715+0000 mgr.gibba032.whopeo (mgr.24625) 2592 : cluster [DBG] pgmap v1462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:14 gibba032 bash[24004]: cephadm 2021-03-10T14:41:12.624960+0000 mgr.gibba032.whopeo (mgr.24625) 2593 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:16 gibba011 bash[30791]: cluster 2021-03-10T14:41:14.505133+0000 mgr.gibba032.whopeo (mgr.24625) 2594 : cluster [DBG] pgmap v1463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:16 gibba011 bash[31774]: cluster 2021-03-10T14:41:14.505133+0000 mgr.gibba032.whopeo (mgr.24625) 2594 : cluster [DBG] pgmap v1463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:16 gibba032 bash[24004]: cluster 2021-03-10T14:41:14.505133+0000 mgr.gibba032.whopeo (mgr.24625) 2594 : cluster [DBG] pgmap v1463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:18 gibba011 bash[30791]: cluster 2021-03-10T14:41:16.506741+0000 mgr.gibba032.whopeo (mgr.24625) 2595 : cluster [DBG] pgmap v1464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:18 gibba011 bash[31774]: cluster 2021-03-10T14:41:16.506741+0000 mgr.gibba032.whopeo (mgr.24625) 2595 : cluster [DBG] pgmap v1464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:18 gibba032 bash[24004]: cluster 2021-03-10T14:41:16.506741+0000 mgr.gibba032.whopeo (mgr.24625) 2595 : cluster [DBG] pgmap v1464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:20 gibba032 bash[24004]: cluster 2021-03-10T14:41:18.508283+0000 mgr.gibba032.whopeo (mgr.24625) 2596 : cluster [DBG] pgmap v1465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:20 gibba011 bash[30791]: cluster 2021-03-10T14:41:18.508283+0000 mgr.gibba032.whopeo (mgr.24625) 2596 : cluster [DBG] pgmap v1465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:20 gibba011 bash[31774]: cluster 2021-03-10T14:41:18.508283+0000 mgr.gibba032.whopeo (mgr.24625) 2596 : cluster [DBG] pgmap v1465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:22 gibba032 bash[24004]: cluster 2021-03-10T14:41:20.509827+0000 mgr.gibba032.whopeo (mgr.24625) 2597 : cluster [DBG] pgmap v1466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:22 gibba011 bash[30791]: cluster 2021-03-10T14:41:20.509827+0000 mgr.gibba032.whopeo (mgr.24625) 2597 : cluster [DBG] pgmap v1466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:22 gibba011 bash[31774]: cluster 2021-03-10T14:41:20.509827+0000 mgr.gibba032.whopeo (mgr.24625) 2597 : cluster [DBG] pgmap v1466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:24 gibba032 bash[24004]: cluster 2021-03-10T14:41:22.511470+0000 mgr.gibba032.whopeo (mgr.24625) 2598 : cluster [DBG] pgmap v1467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:24 gibba011 bash[30791]: cluster 2021-03-10T14:41:22.511470+0000 mgr.gibba032.whopeo (mgr.24625) 2598 : cluster [DBG] pgmap v1467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:24 gibba011 bash[31774]: cluster 2021-03-10T14:41:22.511470+0000 mgr.gibba032.whopeo (mgr.24625) 2598 : cluster [DBG] pgmap v1467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:25.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:25 gibba032 bash[24004]: cluster 2021-03-10T14:41:24.512921+0000 mgr.gibba032.whopeo (mgr.24625) 2599 : cluster [DBG] pgmap v1468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:25.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:25 gibba011 bash[30791]: cluster 2021-03-10T14:41:24.512921+0000 mgr.gibba032.whopeo (mgr.24625) 2599 : cluster [DBG] pgmap v1468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:25.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:25 gibba011 bash[31774]: cluster 2021-03-10T14:41:24.512921+0000 mgr.gibba032.whopeo (mgr.24625) 2599 : cluster [DBG] pgmap v1468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:28 gibba011 bash[30791]: cluster 2021-03-10T14:41:26.514652+0000 mgr.gibba032.whopeo (mgr.24625) 2600 : cluster [DBG] pgmap v1469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:28 gibba011 bash[31774]: cluster 2021-03-10T14:41:26.514652+0000 mgr.gibba032.whopeo (mgr.24625) 2600 : cluster [DBG] pgmap v1469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:28 gibba032 bash[24004]: cluster 2021-03-10T14:41:26.514652+0000 mgr.gibba032.whopeo (mgr.24625) 2600 : cluster [DBG] pgmap v1469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:29 gibba011 bash[30791]: cephadm 2021-03-10T14:41:27.641041+0000 mgr.gibba032.whopeo (mgr.24625) 2601 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:29.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:29 gibba011 bash[31774]: cephadm 2021-03-10T14:41:27.641041+0000 mgr.gibba032.whopeo (mgr.24625) 2601 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:29 gibba032 bash[24004]: cephadm 2021-03-10T14:41:27.641041+0000 mgr.gibba032.whopeo (mgr.24625) 2601 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:30 gibba011 bash[30791]: cluster 2021-03-10T14:41:28.515445+0000 mgr.gibba032.whopeo (mgr.24625) 2602 : cluster [DBG] pgmap v1470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:30 gibba011 bash[31774]: cluster 2021-03-10T14:41:28.515445+0000 mgr.gibba032.whopeo (mgr.24625) 2602 : cluster [DBG] pgmap v1470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:30 gibba032 bash[24004]: cluster 2021-03-10T14:41:28.515445+0000 mgr.gibba032.whopeo (mgr.24625) 2602 : cluster [DBG] pgmap v1470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:31.979 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:41:32.203 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:41:32.204 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (48m) 3m ago 53m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:41:32.204 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (53m) 2m ago 53m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:41:32.204 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (48m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:41:32.205 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (48m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:41:32.205 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (47m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:41:32.205 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (47m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:41:32.205 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (47m) 2m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:41:32.205 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (47m) 3m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:41:32.206 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (53m) 3m ago 53m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:41:32.206 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (53m) 2m ago 53m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:41:32.206 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (46m) 3m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:41:32.206 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (46m) 3m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:41:32.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:41:32.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:41:32.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (44m) 2m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:41:32.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (44m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:41:32.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (43m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:41:32.208 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (43m) 2m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:41:32.208 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (53m) 2m ago 53m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:41:32.208 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 51m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:41:32.454 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:41:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:41:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:41:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:41:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:41:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:41:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:41:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:41:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:41:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:41:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:41:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:41:32.458 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:41:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:32 gibba011 bash[30791]: cluster 2021-03-10T14:41:30.516961+0000 mgr.gibba032.whopeo (mgr.24625) 2603 : cluster [DBG] pgmap v1471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:32 gibba011 bash[31774]: cluster 2021-03-10T14:41:30.516961+0000 mgr.gibba032.whopeo (mgr.24625) 2603 : cluster [DBG] pgmap v1471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:32 gibba032 bash[24004]: cluster 2021-03-10T14:41:30.516961+0000 mgr.gibba032.whopeo (mgr.24625) 2603 : cluster [DBG] pgmap v1471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:33.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[30791]: audit 2021-03-10T14:41:31.966918+0000 mgr.gibba032.whopeo (mgr.24625) 2604 : audit [DBG] from='client.45801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[30791]: audit 2021-03-10T14:41:32.188327+0000 mgr.gibba032.whopeo (mgr.24625) 2605 : audit [DBG] from='client.45807 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[30791]: audit 2021-03-10T14:41:32.453362+0000 mon.a (mon.0) 1301 : audit [DBG] from='client.? 172.21.2.111:0/3100152294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[31774]: audit 2021-03-10T14:41:31.966918+0000 mgr.gibba032.whopeo (mgr.24625) 2604 : audit [DBG] from='client.45801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[31774]: audit 2021-03-10T14:41:32.188327+0000 mgr.gibba032.whopeo (mgr.24625) 2605 : audit [DBG] from='client.45807 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:33 gibba011 bash[31774]: audit 2021-03-10T14:41:32.453362+0000 mon.a (mon.0) 1301 : audit [DBG] from='client.? 172.21.2.111:0/3100152294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:33 gibba032 bash[24004]: audit 2021-03-10T14:41:31.966918+0000 mgr.gibba032.whopeo (mgr.24625) 2604 : audit [DBG] from='client.45801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:33 gibba032 bash[24004]: audit 2021-03-10T14:41:32.188327+0000 mgr.gibba032.whopeo (mgr.24625) 2605 : audit [DBG] from='client.45807 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:41:33.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:33 gibba032 bash[24004]: audit 2021-03-10T14:41:32.453362+0000 mon.a (mon.0) 1301 : audit [DBG] from='client.? 172.21.2.111:0/3100152294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:34 gibba011 bash[30791]: cluster 2021-03-10T14:41:32.518718+0000 mgr.gibba032.whopeo (mgr.24625) 2606 : cluster [DBG] pgmap v1472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:34 gibba011 bash[31774]: cluster 2021-03-10T14:41:32.518718+0000 mgr.gibba032.whopeo (mgr.24625) 2606 : cluster [DBG] pgmap v1472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:34 gibba032 bash[24004]: cluster 2021-03-10T14:41:32.518718+0000 mgr.gibba032.whopeo (mgr.24625) 2606 : cluster [DBG] pgmap v1472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:36 gibba011 bash[30791]: cluster 2021-03-10T14:41:34.519632+0000 mgr.gibba032.whopeo (mgr.24625) 2607 : cluster [DBG] pgmap v1473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:36 gibba011 bash[31774]: cluster 2021-03-10T14:41:34.519632+0000 mgr.gibba032.whopeo (mgr.24625) 2607 : cluster [DBG] pgmap v1473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:36 gibba032 bash[24004]: cluster 2021-03-10T14:41:34.519632+0000 mgr.gibba032.whopeo (mgr.24625) 2607 : cluster [DBG] pgmap v1473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:38 gibba011 bash[30791]: cluster 2021-03-10T14:41:36.521167+0000 mgr.gibba032.whopeo (mgr.24625) 2608 : cluster [DBG] pgmap v1474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:38 gibba011 bash[31774]: cluster 2021-03-10T14:41:36.521167+0000 mgr.gibba032.whopeo (mgr.24625) 2608 : cluster [DBG] pgmap v1474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:38 gibba032 bash[24004]: cluster 2021-03-10T14:41:36.521167+0000 mgr.gibba032.whopeo (mgr.24625) 2608 : cluster [DBG] pgmap v1474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:40 gibba011 bash[30791]: cluster 2021-03-10T14:41:38.522643+0000 mgr.gibba032.whopeo (mgr.24625) 2609 : cluster [DBG] pgmap v1475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:40 gibba011 bash[31774]: cluster 2021-03-10T14:41:38.522643+0000 mgr.gibba032.whopeo (mgr.24625) 2609 : cluster [DBG] pgmap v1475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:40 gibba032 bash[24004]: cluster 2021-03-10T14:41:38.522643+0000 mgr.gibba032.whopeo (mgr.24625) 2609 : cluster [DBG] pgmap v1475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:42.661 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:42 gibba032 bash[24004]: cluster 2021-03-10T14:41:40.524107+0000 mgr.gibba032.whopeo (mgr.24625) 2610 : cluster [DBG] pgmap v1476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:42 gibba011 bash[30791]: cluster 2021-03-10T14:41:40.524107+0000 mgr.gibba032.whopeo (mgr.24625) 2610 : cluster [DBG] pgmap v1476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:42 gibba011 bash[31774]: cluster 2021-03-10T14:41:40.524107+0000 mgr.gibba032.whopeo (mgr.24625) 2610 : cluster [DBG] pgmap v1476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:44 gibba032 bash[24004]: cluster 2021-03-10T14:41:42.525870+0000 mgr.gibba032.whopeo (mgr.24625) 2611 : cluster [DBG] pgmap v1477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:44 gibba032 bash[24004]: cephadm 2021-03-10T14:41:42.657136+0000 mgr.gibba032.whopeo (mgr.24625) 2612 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:44 gibba011 bash[30791]: cluster 2021-03-10T14:41:42.525870+0000 mgr.gibba032.whopeo (mgr.24625) 2611 : cluster [DBG] pgmap v1477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:44 gibba011 bash[30791]: cephadm 2021-03-10T14:41:42.657136+0000 mgr.gibba032.whopeo (mgr.24625) 2612 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:44 gibba011 bash[31774]: cluster 2021-03-10T14:41:42.525870+0000 mgr.gibba032.whopeo (mgr.24625) 2611 : cluster [DBG] pgmap v1477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:44 gibba011 bash[31774]: cephadm 2021-03-10T14:41:42.657136+0000 mgr.gibba032.whopeo (mgr.24625) 2612 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:46 gibba032 bash[24004]: cluster 2021-03-10T14:41:44.527339+0000 mgr.gibba032.whopeo (mgr.24625) 2613 : cluster [DBG] pgmap v1478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:46 gibba011 bash[30791]: cluster 2021-03-10T14:41:44.527339+0000 mgr.gibba032.whopeo (mgr.24625) 2613 : cluster [DBG] pgmap v1478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:46 gibba011 bash[31774]: cluster 2021-03-10T14:41:44.527339+0000 mgr.gibba032.whopeo (mgr.24625) 2613 : cluster [DBG] pgmap v1478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:48 gibba032 bash[24004]: cluster 2021-03-10T14:41:46.528371+0000 mgr.gibba032.whopeo (mgr.24625) 2614 : cluster [DBG] pgmap v1479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:48 gibba011 bash[30791]: cluster 2021-03-10T14:41:46.528371+0000 mgr.gibba032.whopeo (mgr.24625) 2614 : cluster [DBG] pgmap v1479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:48 gibba011 bash[31774]: cluster 2021-03-10T14:41:46.528371+0000 mgr.gibba032.whopeo (mgr.24625) 2614 : cluster [DBG] pgmap v1479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:49.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:49 gibba032 bash[24004]: cluster 2021-03-10T14:41:48.529794+0000 mgr.gibba032.whopeo (mgr.24625) 2615 : cluster [DBG] pgmap v1480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:49.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:49 gibba011 bash[30791]: cluster 2021-03-10T14:41:48.529794+0000 mgr.gibba032.whopeo (mgr.24625) 2615 : cluster [DBG] pgmap v1480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:49.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:49 gibba011 bash[31774]: cluster 2021-03-10T14:41:48.529794+0000 mgr.gibba032.whopeo (mgr.24625) 2615 : cluster [DBG] pgmap v1480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:52 gibba011 bash[30791]: cluster 2021-03-10T14:41:50.531600+0000 mgr.gibba032.whopeo (mgr.24625) 2616 : cluster [DBG] pgmap v1481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:52 gibba011 bash[31774]: cluster 2021-03-10T14:41:50.531600+0000 mgr.gibba032.whopeo (mgr.24625) 2616 : cluster [DBG] pgmap v1481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:52 gibba032 bash[24004]: cluster 2021-03-10T14:41:50.531600+0000 mgr.gibba032.whopeo (mgr.24625) 2616 : cluster [DBG] pgmap v1481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:54 gibba011 bash[30791]: cluster 2021-03-10T14:41:52.532465+0000 mgr.gibba032.whopeo (mgr.24625) 2617 : cluster [DBG] pgmap v1482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:54 gibba011 bash[31774]: cluster 2021-03-10T14:41:52.532465+0000 mgr.gibba032.whopeo (mgr.24625) 2617 : cluster [DBG] pgmap v1482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:54 gibba032 bash[24004]: cluster 2021-03-10T14:41:52.532465+0000 mgr.gibba032.whopeo (mgr.24625) 2617 : cluster [DBG] pgmap v1482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:56 gibba011 bash[30791]: cluster 2021-03-10T14:41:54.534023+0000 mgr.gibba032.whopeo (mgr.24625) 2618 : cluster [DBG] pgmap v1483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:56 gibba011 bash[31774]: cluster 2021-03-10T14:41:54.534023+0000 mgr.gibba032.whopeo (mgr.24625) 2618 : cluster [DBG] pgmap v1483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:56 gibba032 bash[24004]: cluster 2021-03-10T14:41:54.534023+0000 mgr.gibba032.whopeo (mgr.24625) 2618 : cluster [DBG] pgmap v1483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.321939+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:41:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.322172+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:41:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.344371+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:41:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.344846+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:41:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.345298+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:41:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[30791]: audit 2021-03-10T14:41:56.345811+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:41:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.321939+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:41:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.322172+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:41:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.344371+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:41:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.344846+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:41:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.345298+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:41:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:57 gibba011 bash[31774]: audit 2021-03-10T14:41:56.345811+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:41:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.321939+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:41:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.322172+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.344371+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.344846+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.345298+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:57 gibba032 bash[24004]: audit 2021-03-10T14:41:56.345811+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:41:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: cluster 2021-03-10T14:41:56.535571+0000 mgr.gibba032.whopeo (mgr.24625) 2619 : cluster [DBG] pgmap v1484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.885280+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.888448+0000 mon.a (mon.0) 1309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.889782+0000 mon.a (mon.0) 1310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.891279+0000 mon.a (mon.0) 1311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.892770+0000 mon.a (mon.0) 1312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.895647+0000 mon.a (mon.0) 1313 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[30791]: audit 2021-03-10T14:41:57.897206+0000 mon.a (mon.0) 1314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: cluster 2021-03-10T14:41:56.535571+0000 mgr.gibba032.whopeo (mgr.24625) 2619 : cluster [DBG] pgmap v1484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.885280+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:41:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.888448+0000 mon.a (mon.0) 1309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:41:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.889782+0000 mon.a (mon.0) 1310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.891279+0000 mon.a (mon.0) 1311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.892770+0000 mon.a (mon.0) 1312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.895647+0000 mon.a (mon.0) 1313 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:58 gibba011 bash[31774]: audit 2021-03-10T14:41:57.897206+0000 mon.a (mon.0) 1314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: cluster 2021-03-10T14:41:56.535571+0000 mgr.gibba032.whopeo (mgr.24625) 2619 : cluster [DBG] pgmap v1484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.885280+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.888448+0000 mon.a (mon.0) 1309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.889782+0000 mon.a (mon.0) 1310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.891279+0000 mon.a (mon.0) 1311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.892770+0000 mon.a (mon.0) 1312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.895647+0000 mon.a (mon.0) 1313 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:58 gibba032 bash[24004]: audit 2021-03-10T14:41:57.897206+0000 mon.a (mon.0) 1314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:41:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.673817+0000 mgr.gibba032.whopeo (mgr.24625) 2620 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.675445+0000 mgr.gibba032.whopeo (mgr.24625) 2621 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.885248+0000 mgr.gibba032.whopeo (mgr.24625) 2622 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.886774+0000 mgr.gibba032.whopeo (mgr.24625) 2623 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.888209+0000 mgr.gibba032.whopeo (mgr.24625) 2624 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:41:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.888248+0000 mgr.gibba032.whopeo (mgr.24625) 2625 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:41:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.889774+0000 mgr.gibba032.whopeo (mgr.24625) 2626 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:41:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.889812+0000 mgr.gibba032.whopeo (mgr.24625) 2627 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:41:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.891197+0000 mgr.gibba032.whopeo (mgr.24625) 2628 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:41:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.892465+0000 mgr.gibba032.whopeo (mgr.24625) 2629 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:41:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.892499+0000 mgr.gibba032.whopeo (mgr.24625) 2630 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:41:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.893960+0000 mgr.gibba032.whopeo (mgr.24625) 2631 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:41:59.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.894273+0000 mgr.gibba032.whopeo (mgr.24625) 2632 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:41:59.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.895457+0000 mgr.gibba032.whopeo (mgr.24625) 2633 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:41:59.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.895495+0000 mgr.gibba032.whopeo (mgr.24625) 2634 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:41:59.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[30791]: cephadm 2021-03-10T14:41:57.895600+0000 mgr.gibba032.whopeo (mgr.24625) 2635 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.673817+0000 mgr.gibba032.whopeo (mgr.24625) 2620 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:41:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.675445+0000 mgr.gibba032.whopeo (mgr.24625) 2621 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:41:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.885248+0000 mgr.gibba032.whopeo (mgr.24625) 2622 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:41:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.886774+0000 mgr.gibba032.whopeo (mgr.24625) 2623 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:41:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.888209+0000 mgr.gibba032.whopeo (mgr.24625) 2624 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:41:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.888248+0000 mgr.gibba032.whopeo (mgr.24625) 2625 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:41:59.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.889774+0000 mgr.gibba032.whopeo (mgr.24625) 2626 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:41:59.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.889812+0000 mgr.gibba032.whopeo (mgr.24625) 2627 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:41:59.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.891197+0000 mgr.gibba032.whopeo (mgr.24625) 2628 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:41:59.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.892465+0000 mgr.gibba032.whopeo (mgr.24625) 2629 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:41:59.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.892499+0000 mgr.gibba032.whopeo (mgr.24625) 2630 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:41:59.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.893960+0000 mgr.gibba032.whopeo (mgr.24625) 2631 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:41:59.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.894273+0000 mgr.gibba032.whopeo (mgr.24625) 2632 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:41:59.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.895457+0000 mgr.gibba032.whopeo (mgr.24625) 2633 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:41:59.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.895495+0000 mgr.gibba032.whopeo (mgr.24625) 2634 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:41:59.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:41:59 gibba011 bash[31774]: cephadm 2021-03-10T14:41:57.895600+0000 mgr.gibba032.whopeo (mgr.24625) 2635 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.673817+0000 mgr.gibba032.whopeo (mgr.24625) 2620 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.675445+0000 mgr.gibba032.whopeo (mgr.24625) 2621 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.885248+0000 mgr.gibba032.whopeo (mgr.24625) 2622 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.886774+0000 mgr.gibba032.whopeo (mgr.24625) 2623 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.888209+0000 mgr.gibba032.whopeo (mgr.24625) 2624 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.888248+0000 mgr.gibba032.whopeo (mgr.24625) 2625 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.889774+0000 mgr.gibba032.whopeo (mgr.24625) 2626 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.889812+0000 mgr.gibba032.whopeo (mgr.24625) 2627 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.891197+0000 mgr.gibba032.whopeo (mgr.24625) 2628 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.892465+0000 mgr.gibba032.whopeo (mgr.24625) 2629 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.892499+0000 mgr.gibba032.whopeo (mgr.24625) 2630 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.893960+0000 mgr.gibba032.whopeo (mgr.24625) 2631 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.894273+0000 mgr.gibba032.whopeo (mgr.24625) 2632 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:41:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.895457+0000 mgr.gibba032.whopeo (mgr.24625) 2633 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:41:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.895495+0000 mgr.gibba032.whopeo (mgr.24625) 2634 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:41:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:41:59 gibba032 bash[24004]: cephadm 2021-03-10T14:41:57.895600+0000 mgr.gibba032.whopeo (mgr.24625) 2635 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:00 gibba011 bash[30791]: cluster 2021-03-10T14:41:58.536480+0000 mgr.gibba032.whopeo (mgr.24625) 2636 : cluster [DBG] pgmap v1485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:00 gibba011 bash[31774]: cluster 2021-03-10T14:41:58.536480+0000 mgr.gibba032.whopeo (mgr.24625) 2636 : cluster [DBG] pgmap v1485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:00 gibba032 bash[24004]: cluster 2021-03-10T14:41:58.536480+0000 mgr.gibba032.whopeo (mgr.24625) 2636 : cluster [DBG] pgmap v1485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:02 gibba011 bash[30791]: cluster 2021-03-10T14:42:00.538040+0000 mgr.gibba032.whopeo (mgr.24625) 2637 : cluster [DBG] pgmap v1486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:02 gibba011 bash[31774]: cluster 2021-03-10T14:42:00.538040+0000 mgr.gibba032.whopeo (mgr.24625) 2637 : cluster [DBG] pgmap v1486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:02.696 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:42:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:02 gibba032 bash[24004]: cluster 2021-03-10T14:42:00.538040+0000 mgr.gibba032.whopeo (mgr.24625) 2637 : cluster [DBG] pgmap v1486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:02.922 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:42:02.923 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (48m) 4m ago 54m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:42:02.923 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (53m) 3m ago 53m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:42:02.923 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (49m) 3m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:42:02.923 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (49m) 3m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:42:02.924 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (48m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:42:02.924 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (48m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:42:02.924 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (47m) 3m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:42:02.924 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (47m) 4m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:42:02.924 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (54m) 4m ago 54m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:42:02.925 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (54m) 3m ago 54m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:42:02.925 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (46m) 4m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:42:02.925 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (46m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:42:02.925 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (46m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:42:02.925 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (45m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:42:02.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:42:02.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (44m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:42:02.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (44m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:42:02.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (44m) 3m ago 54m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:42:02.927 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (53m) 3m ago 54m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:42:02.927 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 52m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:42:03.177 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:42:03.177 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:42:03.177 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:42:03.178 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:03.178 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:42:03.178 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:42:03.178 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:03.178 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:42:03.179 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:42:03.179 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:03.179 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:42:03.179 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:42:03.179 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:42:03.180 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:42:03.180 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:42:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:03 gibba032 bash[24004]: audit 2021-03-10T14:42:03.176015+0000 mon.c (mon.1) 51 : audit [DBG] from='client.? 172.21.2.111:0/3506853074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:03 gibba011 bash[30791]: audit 2021-03-10T14:42:03.176015+0000 mon.c (mon.1) 51 : audit [DBG] from='client.? 172.21.2.111:0/3506853074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:03 gibba011 bash[31774]: audit 2021-03-10T14:42:03.176015+0000 mon.c (mon.1) 51 : audit [DBG] from='client.? 172.21.2.111:0/3506853074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:04 gibba032 bash[24004]: cluster 2021-03-10T14:42:02.539578+0000 mgr.gibba032.whopeo (mgr.24625) 2638 : cluster [DBG] pgmap v1487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:04 gibba032 bash[24004]: audit 2021-03-10T14:42:02.683602+0000 mgr.gibba032.whopeo (mgr.24625) 2639 : audit [DBG] from='client.55804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:04 gibba032 bash[24004]: audit 2021-03-10T14:42:02.906978+0000 mgr.gibba032.whopeo (mgr.24625) 2640 : audit [DBG] from='client.45825 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[30791]: cluster 2021-03-10T14:42:02.539578+0000 mgr.gibba032.whopeo (mgr.24625) 2638 : cluster [DBG] pgmap v1487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[30791]: audit 2021-03-10T14:42:02.683602+0000 mgr.gibba032.whopeo (mgr.24625) 2639 : audit [DBG] from='client.55804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[30791]: audit 2021-03-10T14:42:02.906978+0000 mgr.gibba032.whopeo (mgr.24625) 2640 : audit [DBG] from='client.45825 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[31774]: cluster 2021-03-10T14:42:02.539578+0000 mgr.gibba032.whopeo (mgr.24625) 2638 : cluster [DBG] pgmap v1487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[31774]: audit 2021-03-10T14:42:02.683602+0000 mgr.gibba032.whopeo (mgr.24625) 2639 : audit [DBG] from='client.55804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:04 gibba011 bash[31774]: audit 2021-03-10T14:42:02.906978+0000 mgr.gibba032.whopeo (mgr.24625) 2640 : audit [DBG] from='client.45825 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:06 gibba032 bash[24004]: cluster 2021-03-10T14:42:04.541180+0000 mgr.gibba032.whopeo (mgr.24625) 2641 : cluster [DBG] pgmap v1488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:06 gibba011 bash[30791]: cluster 2021-03-10T14:42:04.541180+0000 mgr.gibba032.whopeo (mgr.24625) 2641 : cluster [DBG] pgmap v1488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:06 gibba011 bash[31774]: cluster 2021-03-10T14:42:04.541180+0000 mgr.gibba032.whopeo (mgr.24625) 2641 : cluster [DBG] pgmap v1488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:08 gibba032 bash[24004]: cluster 2021-03-10T14:42:06.542837+0000 mgr.gibba032.whopeo (mgr.24625) 2642 : cluster [DBG] pgmap v1489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:08 gibba011 bash[30791]: cluster 2021-03-10T14:42:06.542837+0000 mgr.gibba032.whopeo (mgr.24625) 2642 : cluster [DBG] pgmap v1489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:08 gibba011 bash[31774]: cluster 2021-03-10T14:42:06.542837+0000 mgr.gibba032.whopeo (mgr.24625) 2642 : cluster [DBG] pgmap v1489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:10 gibba032 bash[24004]: cluster 2021-03-10T14:42:08.544419+0000 mgr.gibba032.whopeo (mgr.24625) 2643 : cluster [DBG] pgmap v1490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:10 gibba011 bash[30791]: cluster 2021-03-10T14:42:08.544419+0000 mgr.gibba032.whopeo (mgr.24625) 2643 : cluster [DBG] pgmap v1490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:10 gibba011 bash[31774]: cluster 2021-03-10T14:42:08.544419+0000 mgr.gibba032.whopeo (mgr.24625) 2643 : cluster [DBG] pgmap v1490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:12 gibba011 bash[30791]: cluster 2021-03-10T14:42:10.545845+0000 mgr.gibba032.whopeo (mgr.24625) 2644 : cluster [DBG] pgmap v1491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:12 gibba011 bash[31774]: cluster 2021-03-10T14:42:10.545845+0000 mgr.gibba032.whopeo (mgr.24625) 2644 : cluster [DBG] pgmap v1491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:12 gibba032 bash[24004]: cluster 2021-03-10T14:42:10.545845+0000 mgr.gibba032.whopeo (mgr.24625) 2644 : cluster [DBG] pgmap v1491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:14 gibba011 bash[30791]: cluster 2021-03-10T14:42:12.547596+0000 mgr.gibba032.whopeo (mgr.24625) 2645 : cluster [DBG] pgmap v1492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:14 gibba011 bash[30791]: cephadm 2021-03-10T14:42:12.902341+0000 mgr.gibba032.whopeo (mgr.24625) 2646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:14 gibba011 bash[31774]: cluster 2021-03-10T14:42:12.547596+0000 mgr.gibba032.whopeo (mgr.24625) 2645 : cluster [DBG] pgmap v1492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:14 gibba011 bash[31774]: cephadm 2021-03-10T14:42:12.902341+0000 mgr.gibba032.whopeo (mgr.24625) 2646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:14 gibba032 bash[24004]: cluster 2021-03-10T14:42:12.547596+0000 mgr.gibba032.whopeo (mgr.24625) 2645 : cluster [DBG] pgmap v1492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:14 gibba032 bash[24004]: cephadm 2021-03-10T14:42:12.902341+0000 mgr.gibba032.whopeo (mgr.24625) 2646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:16 gibba011 bash[30791]: cluster 2021-03-10T14:42:14.548873+0000 mgr.gibba032.whopeo (mgr.24625) 2647 : cluster [DBG] pgmap v1493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:16 gibba011 bash[31774]: cluster 2021-03-10T14:42:14.548873+0000 mgr.gibba032.whopeo (mgr.24625) 2647 : cluster [DBG] pgmap v1493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:16 gibba032 bash[24004]: cluster 2021-03-10T14:42:14.548873+0000 mgr.gibba032.whopeo (mgr.24625) 2647 : cluster [DBG] pgmap v1493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:18 gibba011 bash[30791]: cluster 2021-03-10T14:42:16.549919+0000 mgr.gibba032.whopeo (mgr.24625) 2648 : cluster [DBG] pgmap v1494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:18 gibba011 bash[31774]: cluster 2021-03-10T14:42:16.549919+0000 mgr.gibba032.whopeo (mgr.24625) 2648 : cluster [DBG] pgmap v1494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:18 gibba032 bash[24004]: cluster 2021-03-10T14:42:16.549919+0000 mgr.gibba032.whopeo (mgr.24625) 2648 : cluster [DBG] pgmap v1494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:20 gibba011 bash[30791]: cluster 2021-03-10T14:42:18.551431+0000 mgr.gibba032.whopeo (mgr.24625) 2649 : cluster [DBG] pgmap v1495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:20 gibba011 bash[31774]: cluster 2021-03-10T14:42:18.551431+0000 mgr.gibba032.whopeo (mgr.24625) 2649 : cluster [DBG] pgmap v1495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:20 gibba032 bash[24004]: cluster 2021-03-10T14:42:18.551431+0000 mgr.gibba032.whopeo (mgr.24625) 2649 : cluster [DBG] pgmap v1495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:22 gibba011 bash[30791]: cluster 2021-03-10T14:42:20.552999+0000 mgr.gibba032.whopeo (mgr.24625) 2650 : cluster [DBG] pgmap v1496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:22 gibba011 bash[31774]: cluster 2021-03-10T14:42:20.552999+0000 mgr.gibba032.whopeo (mgr.24625) 2650 : cluster [DBG] pgmap v1496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:22 gibba032 bash[24004]: cluster 2021-03-10T14:42:20.552999+0000 mgr.gibba032.whopeo (mgr.24625) 2650 : cluster [DBG] pgmap v1496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:24 gibba032 bash[24004]: cluster 2021-03-10T14:42:22.554294+0000 mgr.gibba032.whopeo (mgr.24625) 2651 : cluster [DBG] pgmap v1497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:24 gibba011 bash[30791]: cluster 2021-03-10T14:42:22.554294+0000 mgr.gibba032.whopeo (mgr.24625) 2651 : cluster [DBG] pgmap v1497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:24 gibba011 bash[31774]: cluster 2021-03-10T14:42:22.554294+0000 mgr.gibba032.whopeo (mgr.24625) 2651 : cluster [DBG] pgmap v1497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:26 gibba032 bash[24004]: cluster 2021-03-10T14:42:24.555601+0000 mgr.gibba032.whopeo (mgr.24625) 2652 : cluster [DBG] pgmap v1498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:26 gibba011 bash[30791]: cluster 2021-03-10T14:42:24.555601+0000 mgr.gibba032.whopeo (mgr.24625) 2652 : cluster [DBG] pgmap v1498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:26 gibba011 bash[31774]: cluster 2021-03-10T14:42:24.555601+0000 mgr.gibba032.whopeo (mgr.24625) 2652 : cluster [DBG] pgmap v1498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:28 gibba032 bash[24004]: cluster 2021-03-10T14:42:26.557408+0000 mgr.gibba032.whopeo (mgr.24625) 2653 : cluster [DBG] pgmap v1499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:28 gibba011 bash[30791]: cluster 2021-03-10T14:42:26.557408+0000 mgr.gibba032.whopeo (mgr.24625) 2653 : cluster [DBG] pgmap v1499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:28 gibba011 bash[31774]: cluster 2021-03-10T14:42:26.557408+0000 mgr.gibba032.whopeo (mgr.24625) 2653 : cluster [DBG] pgmap v1499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:29 gibba032 bash[24004]: cephadm 2021-03-10T14:42:27.918386+0000 mgr.gibba032.whopeo (mgr.24625) 2654 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:29.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:29 gibba011 bash[30791]: cephadm 2021-03-10T14:42:27.918386+0000 mgr.gibba032.whopeo (mgr.24625) 2654 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:29.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:29 gibba011 bash[31774]: cephadm 2021-03-10T14:42:27.918386+0000 mgr.gibba032.whopeo (mgr.24625) 2654 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:30 gibba032 bash[24004]: cluster 2021-03-10T14:42:28.558848+0000 mgr.gibba032.whopeo (mgr.24625) 2655 : cluster [DBG] pgmap v1500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:30 gibba011 bash[30791]: cluster 2021-03-10T14:42:28.558848+0000 mgr.gibba032.whopeo (mgr.24625) 2655 : cluster [DBG] pgmap v1500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:30 gibba011 bash[31774]: cluster 2021-03-10T14:42:28.558848+0000 mgr.gibba032.whopeo (mgr.24625) 2655 : cluster [DBG] pgmap v1500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:32 gibba032 bash[24004]: cluster 2021-03-10T14:42:30.560409+0000 mgr.gibba032.whopeo (mgr.24625) 2656 : cluster [DBG] pgmap v1501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:32 gibba011 bash[30791]: cluster 2021-03-10T14:42:30.560409+0000 mgr.gibba032.whopeo (mgr.24625) 2656 : cluster [DBG] pgmap v1501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:32 gibba011 bash[31774]: cluster 2021-03-10T14:42:30.560409+0000 mgr.gibba032.whopeo (mgr.24625) 2656 : cluster [DBG] pgmap v1501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:33.421 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:42:33.649 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:42:33.649 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (49m) 4m ago 54m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:42:33.649 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (54m) 3m ago 54m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:42:33.650 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (49m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:42:33.650 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (49m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:42:33.650 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (48m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:42:33.650 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (48m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:42:33.651 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (48m) 3m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:42:33.651 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (48m) 4m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:42:33.651 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (54m) 4m ago 54m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:42:33.651 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (54m) 3m ago 54m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:42:33.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (47m) 4m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (47m) 4m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (46m) 4m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (46m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (45m) 3m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (45m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:42:33.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (44m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:42:33.653 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (44m) 3m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:42:33.653 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (54m) 3m ago 54m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:42:33.653 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 52m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:42:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:33 gibba032 bash[24004]: cluster 2021-03-10T14:42:32.561935+0000 mgr.gibba032.whopeo (mgr.24625) 2657 : cluster [DBG] pgmap v1502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:33.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:33 gibba011 bash[30791]: cluster 2021-03-10T14:42:32.561935+0000 mgr.gibba032.whopeo (mgr.24625) 2657 : cluster [DBG] pgmap v1502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:33.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:33 gibba011 bash[31774]: cluster 2021-03-10T14:42:32.561935+0000 mgr.gibba032.whopeo (mgr.24625) 2657 : cluster [DBG] pgmap v1502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:33.898 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:42:33.899 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:42:33.899 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:42:33.899 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:33.900 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:42:33.900 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:42:33.900 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:33.900 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:42:33.900 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:42:33.901 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:42:33.901 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:42:33.901 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:42:33.901 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:42:33.901 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:42:33.902 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:42:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:34 gibba032 bash[24004]: audit 2021-03-10T14:42:33.409459+0000 mgr.gibba032.whopeo (mgr.24625) 2658 : audit [DBG] from='client.45837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:34 gibba032 bash[24004]: audit 2021-03-10T14:42:33.633534+0000 mgr.gibba032.whopeo (mgr.24625) 2659 : audit [DBG] from='client.45843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:34 gibba032 bash[24004]: audit 2021-03-10T14:42:33.897141+0000 mon.a (mon.0) 1315 : audit [DBG] from='client.? 172.21.2.111:0/2172792722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[30791]: audit 2021-03-10T14:42:33.409459+0000 mgr.gibba032.whopeo (mgr.24625) 2658 : audit [DBG] from='client.45837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[30791]: audit 2021-03-10T14:42:33.633534+0000 mgr.gibba032.whopeo (mgr.24625) 2659 : audit [DBG] from='client.45843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[30791]: audit 2021-03-10T14:42:33.897141+0000 mon.a (mon.0) 1315 : audit [DBG] from='client.? 172.21.2.111:0/2172792722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[31774]: audit 2021-03-10T14:42:33.409459+0000 mgr.gibba032.whopeo (mgr.24625) 2658 : audit [DBG] from='client.45837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[31774]: audit 2021-03-10T14:42:33.633534+0000 mgr.gibba032.whopeo (mgr.24625) 2659 : audit [DBG] from='client.45843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:42:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:34 gibba011 bash[31774]: audit 2021-03-10T14:42:33.897141+0000 mon.a (mon.0) 1315 : audit [DBG] from='client.? 172.21.2.111:0/2172792722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:35.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:35 gibba032 bash[24004]: cluster 2021-03-10T14:42:34.563279+0000 mgr.gibba032.whopeo (mgr.24625) 2660 : cluster [DBG] pgmap v1503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:35 gibba011 bash[30791]: cluster 2021-03-10T14:42:34.563279+0000 mgr.gibba032.whopeo (mgr.24625) 2660 : cluster [DBG] pgmap v1503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:35.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:35 gibba011 bash[31774]: cluster 2021-03-10T14:42:34.563279+0000 mgr.gibba032.whopeo (mgr.24625) 2660 : cluster [DBG] pgmap v1503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:38 gibba011 bash[30791]: cluster 2021-03-10T14:42:36.564890+0000 mgr.gibba032.whopeo (mgr.24625) 2661 : cluster [DBG] pgmap v1504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:38 gibba011 bash[31774]: cluster 2021-03-10T14:42:36.564890+0000 mgr.gibba032.whopeo (mgr.24625) 2661 : cluster [DBG] pgmap v1504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:38 gibba032 bash[24004]: cluster 2021-03-10T14:42:36.564890+0000 mgr.gibba032.whopeo (mgr.24625) 2661 : cluster [DBG] pgmap v1504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:40 gibba011 bash[30791]: cluster 2021-03-10T14:42:38.566254+0000 mgr.gibba032.whopeo (mgr.24625) 2662 : cluster [DBG] pgmap v1505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:40 gibba011 bash[31774]: cluster 2021-03-10T14:42:38.566254+0000 mgr.gibba032.whopeo (mgr.24625) 2662 : cluster [DBG] pgmap v1505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:40 gibba032 bash[24004]: cluster 2021-03-10T14:42:38.566254+0000 mgr.gibba032.whopeo (mgr.24625) 2662 : cluster [DBG] pgmap v1505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:42 gibba011 bash[30791]: cluster 2021-03-10T14:42:40.567221+0000 mgr.gibba032.whopeo (mgr.24625) 2663 : cluster [DBG] pgmap v1506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:42 gibba011 bash[31774]: cluster 2021-03-10T14:42:40.567221+0000 mgr.gibba032.whopeo (mgr.24625) 2663 : cluster [DBG] pgmap v1506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:42 gibba032 bash[24004]: cluster 2021-03-10T14:42:40.567221+0000 mgr.gibba032.whopeo (mgr.24625) 2663 : cluster [DBG] pgmap v1506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:44 gibba011 bash[30791]: cluster 2021-03-10T14:42:42.568724+0000 mgr.gibba032.whopeo (mgr.24625) 2664 : cluster [DBG] pgmap v1507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:44 gibba011 bash[30791]: cephadm 2021-03-10T14:42:42.934432+0000 mgr.gibba032.whopeo (mgr.24625) 2665 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:44 gibba011 bash[31774]: cluster 2021-03-10T14:42:42.568724+0000 mgr.gibba032.whopeo (mgr.24625) 2664 : cluster [DBG] pgmap v1507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:44 gibba011 bash[31774]: cephadm 2021-03-10T14:42:42.934432+0000 mgr.gibba032.whopeo (mgr.24625) 2665 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:44 gibba032 bash[24004]: cluster 2021-03-10T14:42:42.568724+0000 mgr.gibba032.whopeo (mgr.24625) 2664 : cluster [DBG] pgmap v1507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:44 gibba032 bash[24004]: cephadm 2021-03-10T14:42:42.934432+0000 mgr.gibba032.whopeo (mgr.24625) 2665 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:46 gibba032 bash[24004]: cluster 2021-03-10T14:42:44.569924+0000 mgr.gibba032.whopeo (mgr.24625) 2666 : cluster [DBG] pgmap v1508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:46 gibba011 bash[30791]: cluster 2021-03-10T14:42:44.569924+0000 mgr.gibba032.whopeo (mgr.24625) 2666 : cluster [DBG] pgmap v1508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:46 gibba011 bash[31774]: cluster 2021-03-10T14:42:44.569924+0000 mgr.gibba032.whopeo (mgr.24625) 2666 : cluster [DBG] pgmap v1508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:48 gibba032 bash[24004]: cluster 2021-03-10T14:42:46.571558+0000 mgr.gibba032.whopeo (mgr.24625) 2667 : cluster [DBG] pgmap v1509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:48 gibba011 bash[30791]: cluster 2021-03-10T14:42:46.571558+0000 mgr.gibba032.whopeo (mgr.24625) 2667 : cluster [DBG] pgmap v1509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:48 gibba011 bash[31774]: cluster 2021-03-10T14:42:46.571558+0000 mgr.gibba032.whopeo (mgr.24625) 2667 : cluster [DBG] pgmap v1509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:50 gibba032 bash[24004]: cluster 2021-03-10T14:42:48.573070+0000 mgr.gibba032.whopeo (mgr.24625) 2668 : cluster [DBG] pgmap v1510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:50.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:50 gibba011 bash[30791]: cluster 2021-03-10T14:42:48.573070+0000 mgr.gibba032.whopeo (mgr.24625) 2668 : cluster [DBG] pgmap v1510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:50 gibba011 bash[31774]: cluster 2021-03-10T14:42:48.573070+0000 mgr.gibba032.whopeo (mgr.24625) 2668 : cluster [DBG] pgmap v1510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:52 gibba032 bash[24004]: cluster 2021-03-10T14:42:50.574536+0000 mgr.gibba032.whopeo (mgr.24625) 2669 : cluster [DBG] pgmap v1511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:52 gibba011 bash[30791]: cluster 2021-03-10T14:42:50.574536+0000 mgr.gibba032.whopeo (mgr.24625) 2669 : cluster [DBG] pgmap v1511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:52 gibba011 bash[31774]: cluster 2021-03-10T14:42:50.574536+0000 mgr.gibba032.whopeo (mgr.24625) 2669 : cluster [DBG] pgmap v1511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:54 gibba032 bash[24004]: cluster 2021-03-10T14:42:52.576384+0000 mgr.gibba032.whopeo (mgr.24625) 2670 : cluster [DBG] pgmap v1512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:54 gibba011 bash[30791]: cluster 2021-03-10T14:42:52.576384+0000 mgr.gibba032.whopeo (mgr.24625) 2670 : cluster [DBG] pgmap v1512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:54 gibba011 bash[31774]: cluster 2021-03-10T14:42:52.576384+0000 mgr.gibba032.whopeo (mgr.24625) 2670 : cluster [DBG] pgmap v1512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:55 gibba032 bash[24004]: cluster 2021-03-10T14:42:54.577672+0000 mgr.gibba032.whopeo (mgr.24625) 2671 : cluster [DBG] pgmap v1513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:55.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:55 gibba011 bash[30791]: cluster 2021-03-10T14:42:54.577672+0000 mgr.gibba032.whopeo (mgr.24625) 2671 : cluster [DBG] pgmap v1513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:55.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:55 gibba011 bash[31774]: cluster 2021-03-10T14:42:54.577672+0000 mgr.gibba032.whopeo (mgr.24625) 2671 : cluster [DBG] pgmap v1513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.326162+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:42:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.326327+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:42:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.352998+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:42:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.353202+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:42:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.353384+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:42:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[30791]: audit 2021-03-10T14:42:56.353559+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:42:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.326162+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:42:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.326327+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:42:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.352998+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:42:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.353202+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:42:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.353384+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:42:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:57 gibba011 bash[31774]: audit 2021-03-10T14:42:56.353559+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:42:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.326162+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:42:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.326327+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.352998+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.353202+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.353384+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:57 gibba032 bash[24004]: audit 2021-03-10T14:42:56.353559+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T14:42:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: cluster 2021-03-10T14:42:56.579279+0000 mgr.gibba032.whopeo (mgr.24625) 2672 : cluster [DBG] pgmap v1514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.166997+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:42:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.178385+0000 mon.a (mon.0) 1323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:42:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.181543+0000 mon.a (mon.0) 1324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.184494+0000 mon.a (mon.0) 1325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.187014+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.192139+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:58 gibba032 bash[24004]: audit 2021-03-10T14:42:58.195601+0000 mon.a (mon.0) 1328 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: cluster 2021-03-10T14:42:56.579279+0000 mgr.gibba032.whopeo (mgr.24625) 2672 : cluster [DBG] pgmap v1514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.166997+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:42:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.178385+0000 mon.a (mon.0) 1323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:42:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.181543+0000 mon.a (mon.0) 1324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.184494+0000 mon.a (mon.0) 1325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.187014+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.192139+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[30791]: audit 2021-03-10T14:42:58.195601+0000 mon.a (mon.0) 1328 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: cluster 2021-03-10T14:42:56.579279+0000 mgr.gibba032.whopeo (mgr.24625) 2672 : cluster [DBG] pgmap v1514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:42:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.166997+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:42:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.178385+0000 mon.a (mon.0) 1323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:42:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.181543+0000 mon.a (mon.0) 1324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.184494+0000 mon.a (mon.0) 1325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.187014+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.192139+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:58 gibba011 bash[31774]: audit 2021-03-10T14:42:58.195601+0000 mon.a (mon.0) 1328 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:42:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:57.950978+0000 mgr.gibba032.whopeo (mgr.24625) 2673 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:42:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:57.952513+0000 mgr.gibba032.whopeo (mgr.24625) 2674 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:42:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.173892+0000 mgr.gibba032.whopeo (mgr.24625) 2675 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:42:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.178052+0000 mgr.gibba032.whopeo (mgr.24625) 2676 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:42:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.181055+0000 mgr.gibba032.whopeo (mgr.24625) 2677 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:42:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.181208+0000 mgr.gibba032.whopeo (mgr.24625) 2678 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:42:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.183400+0000 mgr.gibba032.whopeo (mgr.24625) 2679 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:42:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.183555+0000 mgr.gibba032.whopeo (mgr.24625) 2680 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:42:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.186152+0000 mgr.gibba032.whopeo (mgr.24625) 2681 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:42:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.188182+0000 mgr.gibba032.whopeo (mgr.24625) 2682 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:42:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.188540+0000 mgr.gibba032.whopeo (mgr.24625) 2683 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:42:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.190999+0000 mgr.gibba032.whopeo (mgr.24625) 2684 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:42:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.192317+0000 mgr.gibba032.whopeo (mgr.24625) 2685 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:42:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.194739+0000 mgr.gibba032.whopeo (mgr.24625) 2686 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:42:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.195153+0000 mgr.gibba032.whopeo (mgr.24625) 2687 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:42:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:42:59 gibba032 bash[24004]: cephadm 2021-03-10T14:42:58.195578+0000 mgr.gibba032.whopeo (mgr.24625) 2688 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:57.950978+0000 mgr.gibba032.whopeo (mgr.24625) 2673 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:42:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:57.952513+0000 mgr.gibba032.whopeo (mgr.24625) 2674 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:42:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.173892+0000 mgr.gibba032.whopeo (mgr.24625) 2675 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:42:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.178052+0000 mgr.gibba032.whopeo (mgr.24625) 2676 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:42:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.181055+0000 mgr.gibba032.whopeo (mgr.24625) 2677 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:42:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.181208+0000 mgr.gibba032.whopeo (mgr.24625) 2678 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:42:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.183400+0000 mgr.gibba032.whopeo (mgr.24625) 2679 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:42:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.183555+0000 mgr.gibba032.whopeo (mgr.24625) 2680 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:42:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.186152+0000 mgr.gibba032.whopeo (mgr.24625) 2681 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:42:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.188182+0000 mgr.gibba032.whopeo (mgr.24625) 2682 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:42:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.188540+0000 mgr.gibba032.whopeo (mgr.24625) 2683 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:42:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.190999+0000 mgr.gibba032.whopeo (mgr.24625) 2684 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:42:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.192317+0000 mgr.gibba032.whopeo (mgr.24625) 2685 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:42:59.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.194739+0000 mgr.gibba032.whopeo (mgr.24625) 2686 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:42:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.195153+0000 mgr.gibba032.whopeo (mgr.24625) 2687 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:42:59.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[30791]: cephadm 2021-03-10T14:42:58.195578+0000 mgr.gibba032.whopeo (mgr.24625) 2688 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:42:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:57.950978+0000 mgr.gibba032.whopeo (mgr.24625) 2673 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:42:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:57.952513+0000 mgr.gibba032.whopeo (mgr.24625) 2674 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:42:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.173892+0000 mgr.gibba032.whopeo (mgr.24625) 2675 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:42:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.178052+0000 mgr.gibba032.whopeo (mgr.24625) 2676 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:42:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.181055+0000 mgr.gibba032.whopeo (mgr.24625) 2677 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:42:59.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.181208+0000 mgr.gibba032.whopeo (mgr.24625) 2678 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:42:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.183400+0000 mgr.gibba032.whopeo (mgr.24625) 2679 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:42:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.183555+0000 mgr.gibba032.whopeo (mgr.24625) 2680 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:42:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.186152+0000 mgr.gibba032.whopeo (mgr.24625) 2681 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:42:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.188182+0000 mgr.gibba032.whopeo (mgr.24625) 2682 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:42:59.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.188540+0000 mgr.gibba032.whopeo (mgr.24625) 2683 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:42:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.190999+0000 mgr.gibba032.whopeo (mgr.24625) 2684 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:42:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.192317+0000 mgr.gibba032.whopeo (mgr.24625) 2685 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:42:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.194739+0000 mgr.gibba032.whopeo (mgr.24625) 2686 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:42:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.195153+0000 mgr.gibba032.whopeo (mgr.24625) 2687 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:42:59.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:42:59 gibba011 bash[31774]: cephadm 2021-03-10T14:42:58.195578+0000 mgr.gibba032.whopeo (mgr.24625) 2688 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:00 gibba032 bash[24004]: cluster 2021-03-10T14:42:58.580818+0000 mgr.gibba032.whopeo (mgr.24625) 2689 : cluster [DBG] pgmap v1515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:00 gibba011 bash[30791]: cluster 2021-03-10T14:42:58.580818+0000 mgr.gibba032.whopeo (mgr.24625) 2689 : cluster [DBG] pgmap v1515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:00 gibba011 bash[31774]: cluster 2021-03-10T14:42:58.580818+0000 mgr.gibba032.whopeo (mgr.24625) 2689 : cluster [DBG] pgmap v1515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:02 gibba032 bash[24004]: cluster 2021-03-10T14:43:00.582181+0000 mgr.gibba032.whopeo (mgr.24625) 2690 : cluster [DBG] pgmap v1516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:02 gibba011 bash[30791]: cluster 2021-03-10T14:43:00.582181+0000 mgr.gibba032.whopeo (mgr.24625) 2690 : cluster [DBG] pgmap v1516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:02 gibba011 bash[31774]: cluster 2021-03-10T14:43:00.582181+0000 mgr.gibba032.whopeo (mgr.24625) 2690 : cluster [DBG] pgmap v1516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:04.142 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:43:04.368 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:43:04.368 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (49m) 5m ago 55m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:43:04.368 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (54m) 4m ago 54m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:43:04.368 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (50m) 4m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:43:04.369 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (50m) 4m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:43:04.369 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (49m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:43:04.369 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (49m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:43:04.369 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (48m) 4m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:43:04.369 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (48m) 5m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:43:04.370 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (55m) 5m ago 55m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:43:04.370 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (55m) 4m ago 55m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:43:04.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (48m) 5m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:43:04.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (47m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:43:04.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (47m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:43:04.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (46m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:43:04.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (46m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:43:04.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (45m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:43:04.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (45m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:43:04.372 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (45m) 4m ago 55m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:43:04.372 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (54m) 4m ago 55m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:43:04.372 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 53m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:43:04.621 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:43:04.621 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:43:04.621 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:43:04.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:04.622 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:43:04.622 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:43:04.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:04.623 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:43:04.623 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:43:04.623 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:04.623 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:43:04.623 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:43:04.624 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:43:04.624 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:43:04.624 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:43:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:04 gibba032 bash[24004]: cluster 2021-03-10T14:43:02.583809+0000 mgr.gibba032.whopeo (mgr.24625) 2691 : cluster [DBG] pgmap v1517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:04 gibba011 bash[30791]: cluster 2021-03-10T14:43:02.583809+0000 mgr.gibba032.whopeo (mgr.24625) 2691 : cluster [DBG] pgmap v1517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:04 gibba011 bash[31774]: cluster 2021-03-10T14:43:02.583809+0000 mgr.gibba032.whopeo (mgr.24625) 2691 : cluster [DBG] pgmap v1517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:05.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:05 gibba032 bash[24004]: audit 2021-03-10T14:43:04.128318+0000 mgr.gibba032.whopeo (mgr.24625) 2692 : audit [DBG] from='client.55840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:05 gibba032 bash[24004]: audit 2021-03-10T14:43:04.352407+0000 mgr.gibba032.whopeo (mgr.24625) 2693 : audit [DBG] from='client.55846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:05 gibba032 bash[24004]: audit 2021-03-10T14:43:04.620130+0000 mon.c (mon.1) 52 : audit [DBG] from='client.? 172.21.2.111:0/1407442680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:05.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[30791]: audit 2021-03-10T14:43:04.128318+0000 mgr.gibba032.whopeo (mgr.24625) 2692 : audit [DBG] from='client.55840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[30791]: audit 2021-03-10T14:43:04.352407+0000 mgr.gibba032.whopeo (mgr.24625) 2693 : audit [DBG] from='client.55846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[30791]: audit 2021-03-10T14:43:04.620130+0000 mon.c (mon.1) 52 : audit [DBG] from='client.? 172.21.2.111:0/1407442680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[31774]: audit 2021-03-10T14:43:04.128318+0000 mgr.gibba032.whopeo (mgr.24625) 2692 : audit [DBG] from='client.55840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[31774]: audit 2021-03-10T14:43:04.352407+0000 mgr.gibba032.whopeo (mgr.24625) 2693 : audit [DBG] from='client.55846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:05.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:05 gibba011 bash[31774]: audit 2021-03-10T14:43:04.620130+0000 mon.c (mon.1) 52 : audit [DBG] from='client.? 172.21.2.111:0/1407442680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:06 gibba032 bash[24004]: cluster 2021-03-10T14:43:04.585185+0000 mgr.gibba032.whopeo (mgr.24625) 2694 : cluster [DBG] pgmap v1518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:06 gibba011 bash[30791]: cluster 2021-03-10T14:43:04.585185+0000 mgr.gibba032.whopeo (mgr.24625) 2694 : cluster [DBG] pgmap v1518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:06 gibba011 bash[31774]: cluster 2021-03-10T14:43:04.585185+0000 mgr.gibba032.whopeo (mgr.24625) 2694 : cluster [DBG] pgmap v1518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:08 gibba032 bash[24004]: cluster 2021-03-10T14:43:06.586785+0000 mgr.gibba032.whopeo (mgr.24625) 2695 : cluster [DBG] pgmap v1519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:08 gibba011 bash[30791]: cluster 2021-03-10T14:43:06.586785+0000 mgr.gibba032.whopeo (mgr.24625) 2695 : cluster [DBG] pgmap v1519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:08 gibba011 bash[31774]: cluster 2021-03-10T14:43:06.586785+0000 mgr.gibba032.whopeo (mgr.24625) 2695 : cluster [DBG] pgmap v1519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:10 gibba032 bash[24004]: cluster 2021-03-10T14:43:08.588300+0000 mgr.gibba032.whopeo (mgr.24625) 2696 : cluster [DBG] pgmap v1520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:10 gibba011 bash[30791]: cluster 2021-03-10T14:43:08.588300+0000 mgr.gibba032.whopeo (mgr.24625) 2696 : cluster [DBG] pgmap v1520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:10 gibba011 bash[31774]: cluster 2021-03-10T14:43:08.588300+0000 mgr.gibba032.whopeo (mgr.24625) 2696 : cluster [DBG] pgmap v1520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:12 gibba011 bash[30791]: cluster 2021-03-10T14:43:10.589666+0000 mgr.gibba032.whopeo (mgr.24625) 2697 : cluster [DBG] pgmap v1521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:12 gibba011 bash[31774]: cluster 2021-03-10T14:43:10.589666+0000 mgr.gibba032.whopeo (mgr.24625) 2697 : cluster [DBG] pgmap v1521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:12 gibba032 bash[24004]: cluster 2021-03-10T14:43:10.589666+0000 mgr.gibba032.whopeo (mgr.24625) 2697 : cluster [DBG] pgmap v1521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:14 gibba011 bash[30791]: cluster 2021-03-10T14:43:12.591322+0000 mgr.gibba032.whopeo (mgr.24625) 2698 : cluster [DBG] pgmap v1522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:14 gibba011 bash[30791]: cephadm 2021-03-10T14:43:13.211388+0000 mgr.gibba032.whopeo (mgr.24625) 2699 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:14 gibba011 bash[31774]: cluster 2021-03-10T14:43:12.591322+0000 mgr.gibba032.whopeo (mgr.24625) 2698 : cluster [DBG] pgmap v1522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:14 gibba011 bash[31774]: cephadm 2021-03-10T14:43:13.211388+0000 mgr.gibba032.whopeo (mgr.24625) 2699 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:14 gibba032 bash[24004]: cluster 2021-03-10T14:43:12.591322+0000 mgr.gibba032.whopeo (mgr.24625) 2698 : cluster [DBG] pgmap v1522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:14 gibba032 bash[24004]: cephadm 2021-03-10T14:43:13.211388+0000 mgr.gibba032.whopeo (mgr.24625) 2699 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:16 gibba032 bash[24004]: cluster 2021-03-10T14:43:14.592683+0000 mgr.gibba032.whopeo (mgr.24625) 2700 : cluster [DBG] pgmap v1523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:16 gibba011 bash[30791]: cluster 2021-03-10T14:43:14.592683+0000 mgr.gibba032.whopeo (mgr.24625) 2700 : cluster [DBG] pgmap v1523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:16 gibba011 bash[31774]: cluster 2021-03-10T14:43:14.592683+0000 mgr.gibba032.whopeo (mgr.24625) 2700 : cluster [DBG] pgmap v1523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:18 gibba032 bash[24004]: cluster 2021-03-10T14:43:16.594461+0000 mgr.gibba032.whopeo (mgr.24625) 2701 : cluster [DBG] pgmap v1524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:18 gibba011 bash[30791]: cluster 2021-03-10T14:43:16.594461+0000 mgr.gibba032.whopeo (mgr.24625) 2701 : cluster [DBG] pgmap v1524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:18 gibba011 bash[31774]: cluster 2021-03-10T14:43:16.594461+0000 mgr.gibba032.whopeo (mgr.24625) 2701 : cluster [DBG] pgmap v1524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:20 gibba032 bash[24004]: cluster 2021-03-10T14:43:18.595972+0000 mgr.gibba032.whopeo (mgr.24625) 2702 : cluster [DBG] pgmap v1525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:20 gibba011 bash[30791]: cluster 2021-03-10T14:43:18.595972+0000 mgr.gibba032.whopeo (mgr.24625) 2702 : cluster [DBG] pgmap v1525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:20 gibba011 bash[31774]: cluster 2021-03-10T14:43:18.595972+0000 mgr.gibba032.whopeo (mgr.24625) 2702 : cluster [DBG] pgmap v1525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:22 gibba032 bash[24004]: cluster 2021-03-10T14:43:20.597431+0000 mgr.gibba032.whopeo (mgr.24625) 2703 : cluster [DBG] pgmap v1526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:22 gibba011 bash[30791]: cluster 2021-03-10T14:43:20.597431+0000 mgr.gibba032.whopeo (mgr.24625) 2703 : cluster [DBG] pgmap v1526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:22 gibba011 bash[31774]: cluster 2021-03-10T14:43:20.597431+0000 mgr.gibba032.whopeo (mgr.24625) 2703 : cluster [DBG] pgmap v1526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:24 gibba032 bash[24004]: cluster 2021-03-10T14:43:22.599116+0000 mgr.gibba032.whopeo (mgr.24625) 2704 : cluster [DBG] pgmap v1527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:24 gibba011 bash[30791]: cluster 2021-03-10T14:43:22.599116+0000 mgr.gibba032.whopeo (mgr.24625) 2704 : cluster [DBG] pgmap v1527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:24 gibba011 bash[31774]: cluster 2021-03-10T14:43:22.599116+0000 mgr.gibba032.whopeo (mgr.24625) 2704 : cluster [DBG] pgmap v1527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:26 gibba032 bash[24004]: cluster 2021-03-10T14:43:24.600447+0000 mgr.gibba032.whopeo (mgr.24625) 2705 : cluster [DBG] pgmap v1528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:26 gibba011 bash[30791]: cluster 2021-03-10T14:43:24.600447+0000 mgr.gibba032.whopeo (mgr.24625) 2705 : cluster [DBG] pgmap v1528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:26 gibba011 bash[31774]: cluster 2021-03-10T14:43:24.600447+0000 mgr.gibba032.whopeo (mgr.24625) 2705 : cluster [DBG] pgmap v1528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:28 gibba032 bash[24004]: cluster 2021-03-10T14:43:26.601971+0000 mgr.gibba032.whopeo (mgr.24625) 2706 : cluster [DBG] pgmap v1529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:28 gibba011 bash[30791]: cluster 2021-03-10T14:43:26.601971+0000 mgr.gibba032.whopeo (mgr.24625) 2706 : cluster [DBG] pgmap v1529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:28 gibba011 bash[31774]: cluster 2021-03-10T14:43:26.601971+0000 mgr.gibba032.whopeo (mgr.24625) 2706 : cluster [DBG] pgmap v1529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:29.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:29 gibba032 bash[24004]: cephadm 2021-03-10T14:43:28.224470+0000 mgr.gibba032.whopeo (mgr.24625) 2707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:29.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:29 gibba011 bash[30791]: cephadm 2021-03-10T14:43:28.224470+0000 mgr.gibba032.whopeo (mgr.24625) 2707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:29.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:29 gibba011 bash[31774]: cephadm 2021-03-10T14:43:28.224470+0000 mgr.gibba032.whopeo (mgr.24625) 2707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:30 gibba032 bash[24004]: cluster 2021-03-10T14:43:28.603575+0000 mgr.gibba032.whopeo (mgr.24625) 2708 : cluster [DBG] pgmap v1530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:30 gibba011 bash[30791]: cluster 2021-03-10T14:43:28.603575+0000 mgr.gibba032.whopeo (mgr.24625) 2708 : cluster [DBG] pgmap v1530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:30.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:30 gibba011 bash[31774]: cluster 2021-03-10T14:43:28.603575+0000 mgr.gibba032.whopeo (mgr.24625) 2708 : cluster [DBG] pgmap v1530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:32 gibba011 bash[30791]: cluster 2021-03-10T14:43:30.605047+0000 mgr.gibba032.whopeo (mgr.24625) 2709 : cluster [DBG] pgmap v1531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:32 gibba011 bash[31774]: cluster 2021-03-10T14:43:30.605047+0000 mgr.gibba032.whopeo (mgr.24625) 2709 : cluster [DBG] pgmap v1531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:32 gibba032 bash[24004]: cluster 2021-03-10T14:43:30.605047+0000 mgr.gibba032.whopeo (mgr.24625) 2709 : cluster [DBG] pgmap v1531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:34 gibba032 bash[24004]: cluster 2021-03-10T14:43:32.606808+0000 mgr.gibba032.whopeo (mgr.24625) 2710 : cluster [DBG] pgmap v1532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:34 gibba011 bash[30791]: cluster 2021-03-10T14:43:32.606808+0000 mgr.gibba032.whopeo (mgr.24625) 2710 : cluster [DBG] pgmap v1532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:34 gibba011 bash[31774]: cluster 2021-03-10T14:43:32.606808+0000 mgr.gibba032.whopeo (mgr.24625) 2710 : cluster [DBG] pgmap v1532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:34.863 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:43:35.090 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:43:35.090 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (50m) 5m ago 55m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:43:35.091 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (55m) 4m ago 55m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:43:35.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (50m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:43:35.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (50m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:43:35.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (49m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:43:35.091 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (49m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:43:35.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (49m) 4m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:43:35.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (49m) 5m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:43:35.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (55m) 5m ago 55m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:43:35.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (55m) 4m ago 55m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:43:35.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (48m) 5m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:43:35.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (48m) 5m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:43:35.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (47m) 5m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:43:35.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (47m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:43:35.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (46m) 4m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:43:35.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (46m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:43:35.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (45m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:43:35.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (45m) 4m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:43:35.094 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (55m) 4m ago 55m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:43:35.094 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 53m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:43:35.340 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:43:35.340 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:43:35.340 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:43:35.341 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:35.341 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:43:35.341 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:43:35.341 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:35.341 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:43:35.342 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:43:35.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:43:35.342 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:43:35.342 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:43:35.342 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:43:35.343 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:43:35.343 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:43:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:36 gibba032 bash[24004]: cluster 2021-03-10T14:43:34.608236+0000 mgr.gibba032.whopeo (mgr.24625) 2711 : cluster [DBG] pgmap v1533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:36 gibba032 bash[24004]: audit 2021-03-10T14:43:34.850035+0000 mgr.gibba032.whopeo (mgr.24625) 2712 : audit [DBG] from='client.45873 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:36 gibba032 bash[24004]: audit 2021-03-10T14:43:35.074062+0000 mgr.gibba032.whopeo (mgr.24625) 2713 : audit [DBG] from='client.55864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:36 gibba032 bash[24004]: audit 2021-03-10T14:43:35.338605+0000 mon.c (mon.1) 53 : audit [DBG] from='client.? 172.21.2.111:0/28540688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[30791]: cluster 2021-03-10T14:43:34.608236+0000 mgr.gibba032.whopeo (mgr.24625) 2711 : cluster [DBG] pgmap v1533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[30791]: audit 2021-03-10T14:43:34.850035+0000 mgr.gibba032.whopeo (mgr.24625) 2712 : audit [DBG] from='client.45873 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[30791]: audit 2021-03-10T14:43:35.074062+0000 mgr.gibba032.whopeo (mgr.24625) 2713 : audit [DBG] from='client.55864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[30791]: audit 2021-03-10T14:43:35.338605+0000 mon.c (mon.1) 53 : audit [DBG] from='client.? 172.21.2.111:0/28540688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[31774]: cluster 2021-03-10T14:43:34.608236+0000 mgr.gibba032.whopeo (mgr.24625) 2711 : cluster [DBG] pgmap v1533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[31774]: audit 2021-03-10T14:43:34.850035+0000 mgr.gibba032.whopeo (mgr.24625) 2712 : audit [DBG] from='client.45873 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[31774]: audit 2021-03-10T14:43:35.074062+0000 mgr.gibba032.whopeo (mgr.24625) 2713 : audit [DBG] from='client.55864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:43:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:36 gibba011 bash[31774]: audit 2021-03-10T14:43:35.338605+0000 mon.c (mon.1) 53 : audit [DBG] from='client.? 172.21.2.111:0/28540688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:38 gibba032 bash[24004]: cluster 2021-03-10T14:43:36.609702+0000 mgr.gibba032.whopeo (mgr.24625) 2714 : cluster [DBG] pgmap v1534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:38 gibba011 bash[30791]: cluster 2021-03-10T14:43:36.609702+0000 mgr.gibba032.whopeo (mgr.24625) 2714 : cluster [DBG] pgmap v1534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:38 gibba011 bash[31774]: cluster 2021-03-10T14:43:36.609702+0000 mgr.gibba032.whopeo (mgr.24625) 2714 : cluster [DBG] pgmap v1534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:40 gibba032 bash[24004]: cluster 2021-03-10T14:43:38.611012+0000 mgr.gibba032.whopeo (mgr.24625) 2715 : cluster [DBG] pgmap v1535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:40 gibba011 bash[30791]: cluster 2021-03-10T14:43:38.611012+0000 mgr.gibba032.whopeo (mgr.24625) 2715 : cluster [DBG] pgmap v1535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:40 gibba011 bash[31774]: cluster 2021-03-10T14:43:38.611012+0000 mgr.gibba032.whopeo (mgr.24625) 2715 : cluster [DBG] pgmap v1535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:42 gibba032 bash[24004]: cluster 2021-03-10T14:43:40.612703+0000 mgr.gibba032.whopeo (mgr.24625) 2716 : cluster [DBG] pgmap v1536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:42 gibba011 bash[30791]: cluster 2021-03-10T14:43:40.612703+0000 mgr.gibba032.whopeo (mgr.24625) 2716 : cluster [DBG] pgmap v1536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:42 gibba011 bash[31774]: cluster 2021-03-10T14:43:40.612703+0000 mgr.gibba032.whopeo (mgr.24625) 2716 : cluster [DBG] pgmap v1536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:44 gibba032 bash[24004]: cluster 2021-03-10T14:43:42.614439+0000 mgr.gibba032.whopeo (mgr.24625) 2717 : cluster [DBG] pgmap v1537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:44 gibba032 bash[24004]: cephadm 2021-03-10T14:43:43.240643+0000 mgr.gibba032.whopeo (mgr.24625) 2718 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:44 gibba011 bash[30791]: cluster 2021-03-10T14:43:42.614439+0000 mgr.gibba032.whopeo (mgr.24625) 2717 : cluster [DBG] pgmap v1537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:44 gibba011 bash[30791]: cephadm 2021-03-10T14:43:43.240643+0000 mgr.gibba032.whopeo (mgr.24625) 2718 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:44 gibba011 bash[31774]: cluster 2021-03-10T14:43:42.614439+0000 mgr.gibba032.whopeo (mgr.24625) 2717 : cluster [DBG] pgmap v1537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:44 gibba011 bash[31774]: cephadm 2021-03-10T14:43:43.240643+0000 mgr.gibba032.whopeo (mgr.24625) 2718 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:43:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:46 gibba032 bash[24004]: cluster 2021-03-10T14:43:44.615793+0000 mgr.gibba032.whopeo (mgr.24625) 2719 : cluster [DBG] pgmap v1538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:46 gibba011 bash[30791]: cluster 2021-03-10T14:43:44.615793+0000 mgr.gibba032.whopeo (mgr.24625) 2719 : cluster [DBG] pgmap v1538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:46 gibba011 bash[31774]: cluster 2021-03-10T14:43:44.615793+0000 mgr.gibba032.whopeo (mgr.24625) 2719 : cluster [DBG] pgmap v1538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:48 gibba032 bash[24004]: cluster 2021-03-10T14:43:46.617562+0000 mgr.gibba032.whopeo (mgr.24625) 2720 : cluster [DBG] pgmap v1539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:48 gibba011 bash[30791]: cluster 2021-03-10T14:43:46.617562+0000 mgr.gibba032.whopeo (mgr.24625) 2720 : cluster [DBG] pgmap v1539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:48 gibba011 bash[31774]: cluster 2021-03-10T14:43:46.617562+0000 mgr.gibba032.whopeo (mgr.24625) 2720 : cluster [DBG] pgmap v1539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:50 gibba032 bash[24004]: cluster 2021-03-10T14:43:48.618990+0000 mgr.gibba032.whopeo (mgr.24625) 2721 : cluster [DBG] pgmap v1540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:50.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:50 gibba011 bash[30791]: cluster 2021-03-10T14:43:48.618990+0000 mgr.gibba032.whopeo (mgr.24625) 2721 : cluster [DBG] pgmap v1540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:50 gibba011 bash[31774]: cluster 2021-03-10T14:43:48.618990+0000 mgr.gibba032.whopeo (mgr.24625) 2721 : cluster [DBG] pgmap v1540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:52 gibba032 bash[24004]: cluster 2021-03-10T14:43:50.620558+0000 mgr.gibba032.whopeo (mgr.24625) 2722 : cluster [DBG] pgmap v1541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:52 gibba011 bash[30791]: cluster 2021-03-10T14:43:50.620558+0000 mgr.gibba032.whopeo (mgr.24625) 2722 : cluster [DBG] pgmap v1541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:52 gibba011 bash[31774]: cluster 2021-03-10T14:43:50.620558+0000 mgr.gibba032.whopeo (mgr.24625) 2722 : cluster [DBG] pgmap v1541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:53 gibba032 bash[24004]: cluster 2021-03-10T14:43:52.622098+0000 mgr.gibba032.whopeo (mgr.24625) 2723 : cluster [DBG] pgmap v1542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:53.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:53 gibba011 bash[30791]: cluster 2021-03-10T14:43:52.622098+0000 mgr.gibba032.whopeo (mgr.24625) 2723 : cluster [DBG] pgmap v1542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:53.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:53 gibba011 bash[31774]: cluster 2021-03-10T14:43:52.622098+0000 mgr.gibba032.whopeo (mgr.24625) 2723 : cluster [DBG] pgmap v1542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:55 gibba032 bash[24004]: cluster 2021-03-10T14:43:54.623462+0000 mgr.gibba032.whopeo (mgr.24625) 2724 : cluster [DBG] pgmap v1543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:55 gibba011 bash[30791]: cluster 2021-03-10T14:43:54.623462+0000 mgr.gibba032.whopeo (mgr.24625) 2724 : cluster [DBG] pgmap v1543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:55 gibba011 bash[31774]: cluster 2021-03-10T14:43:54.623462+0000 mgr.gibba032.whopeo (mgr.24625) 2724 : cluster [DBG] pgmap v1543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.332077+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:43:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.338146+0000 mon.a (mon.0) 1330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.357443+0000 mon.a (mon.0) 1331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.358155+0000 mon.a (mon.0) 1332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.358900+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:57 gibba032 bash[24004]: audit 2021-03-10T14:43:56.359631+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:43:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.332077+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:43:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.338146+0000 mon.a (mon.0) 1330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:43:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.357443+0000 mon.a (mon.0) 1331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:43:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.358155+0000 mon.a (mon.0) 1332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:43:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.358900+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:43:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[30791]: audit 2021-03-10T14:43:56.359631+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:43:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.332077+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:43:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.338146+0000 mon.a (mon.0) 1330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:43:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.357443+0000 mon.a (mon.0) 1331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T14:43:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.358155+0000 mon.a (mon.0) 1332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:43:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.358900+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:43:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:57 gibba011 bash[31774]: audit 2021-03-10T14:43:56.359631+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:43:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:58 gibba032 bash[24004]: cluster 2021-03-10T14:43:56.625252+0000 mgr.gibba032.whopeo (mgr.24625) 2725 : cluster [DBG] pgmap v1544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:58 gibba011 bash[30791]: cluster 2021-03-10T14:43:56.625252+0000 mgr.gibba032.whopeo (mgr.24625) 2725 : cluster [DBG] pgmap v1544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:58 gibba011 bash[31774]: cluster 2021-03-10T14:43:56.625252+0000 mgr.gibba032.whopeo (mgr.24625) 2725 : cluster [DBG] pgmap v1544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:43:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.253752+0000 mgr.gibba032.whopeo (mgr.24625) 2726 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:43:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.255295+0000 mgr.gibba032.whopeo (mgr.24625) 2727 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:43:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.477456+0000 mon.a (mon.0) 1335 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:43:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.480678+0000 mon.a (mon.0) 1336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:43:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.482117+0000 mon.a (mon.0) 1337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.483600+0000 mon.a (mon.0) 1338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.484905+0000 mon.a (mon.0) 1339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.486588+0000 mon.a (mon.0) 1340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:43:59 gibba032 bash[24004]: audit 2021-03-10T14:43:58.488238+0000 mon.a (mon.0) 1341 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.253752+0000 mgr.gibba032.whopeo (mgr.24625) 2726 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:43:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.255295+0000 mgr.gibba032.whopeo (mgr.24625) 2727 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:43:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.477456+0000 mon.a (mon.0) 1335 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:43:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.480678+0000 mon.a (mon.0) 1336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:43:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.482117+0000 mon.a (mon.0) 1337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.483600+0000 mon.a (mon.0) 1338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.484905+0000 mon.a (mon.0) 1339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.486588+0000 mon.a (mon.0) 1340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[30791]: audit 2021-03-10T14:43:58.488238+0000 mon.a (mon.0) 1341 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.253752+0000 mgr.gibba032.whopeo (mgr.24625) 2726 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:43:59.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.255295+0000 mgr.gibba032.whopeo (mgr.24625) 2727 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:43:59.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.477456+0000 mon.a (mon.0) 1335 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:43:59.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.480678+0000 mon.a (mon.0) 1336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:43:59.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.482117+0000 mon.a (mon.0) 1337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.483600+0000 mon.a (mon.0) 1338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.484905+0000 mon.a (mon.0) 1339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.486588+0000 mon.a (mon.0) 1340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:43:59.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:43:59 gibba011 bash[31774]: audit 2021-03-10T14:43:58.488238+0000 mon.a (mon.0) 1341 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.477288+0000 mgr.gibba032.whopeo (mgr.24625) 2728 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:44:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.479140+0000 mgr.gibba032.whopeo (mgr.24625) 2729 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:44:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.480289+0000 mgr.gibba032.whopeo (mgr.24625) 2730 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:44:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.480366+0000 mgr.gibba032.whopeo (mgr.24625) 2731 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:44:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.481752+0000 mgr.gibba032.whopeo (mgr.24625) 2732 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:44:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.481847+0000 mgr.gibba032.whopeo (mgr.24625) 2733 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:44:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.482916+0000 mgr.gibba032.whopeo (mgr.24625) 2734 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:44:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.483576+0000 mgr.gibba032.whopeo (mgr.24625) 2735 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:44:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.483613+0000 mgr.gibba032.whopeo (mgr.24625) 2736 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:44:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.485069+0000 mgr.gibba032.whopeo (mgr.24625) 2737 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:44:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.485394+0000 mgr.gibba032.whopeo (mgr.24625) 2738 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:44:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.486306+0000 mgr.gibba032.whopeo (mgr.24625) 2739 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:44:00.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.486368+0000 mgr.gibba032.whopeo (mgr.24625) 2740 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:44:00.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cephadm 2021-03-10T14:43:58.486482+0000 mgr.gibba032.whopeo (mgr.24625) 2741 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:00.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:00 gibba032 bash[24004]: cluster 2021-03-10T14:43:58.627000+0000 mgr.gibba032.whopeo (mgr.24625) 2742 : cluster [DBG] pgmap v1545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.477288+0000 mgr.gibba032.whopeo (mgr.24625) 2728 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:44:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.479140+0000 mgr.gibba032.whopeo (mgr.24625) 2729 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:44:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.480289+0000 mgr.gibba032.whopeo (mgr.24625) 2730 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:44:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.480366+0000 mgr.gibba032.whopeo (mgr.24625) 2731 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:44:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.481752+0000 mgr.gibba032.whopeo (mgr.24625) 2732 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:44:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.481847+0000 mgr.gibba032.whopeo (mgr.24625) 2733 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:44:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.482916+0000 mgr.gibba032.whopeo (mgr.24625) 2734 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:44:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.483576+0000 mgr.gibba032.whopeo (mgr.24625) 2735 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:44:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.483613+0000 mgr.gibba032.whopeo (mgr.24625) 2736 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:44:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.485069+0000 mgr.gibba032.whopeo (mgr.24625) 2737 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:44:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.485394+0000 mgr.gibba032.whopeo (mgr.24625) 2738 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:44:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.486306+0000 mgr.gibba032.whopeo (mgr.24625) 2739 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:44:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.486368+0000 mgr.gibba032.whopeo (mgr.24625) 2740 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:44:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cephadm 2021-03-10T14:43:58.486482+0000 mgr.gibba032.whopeo (mgr.24625) 2741 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[30791]: cluster 2021-03-10T14:43:58.627000+0000 mgr.gibba032.whopeo (mgr.24625) 2742 : cluster [DBG] pgmap v1545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:00.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.477288+0000 mgr.gibba032.whopeo (mgr.24625) 2728 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:44:00.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.479140+0000 mgr.gibba032.whopeo (mgr.24625) 2729 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:44:00.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.480289+0000 mgr.gibba032.whopeo (mgr.24625) 2730 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:44:00.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.480366+0000 mgr.gibba032.whopeo (mgr.24625) 2731 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:44:00.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.481752+0000 mgr.gibba032.whopeo (mgr.24625) 2732 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:44:00.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.481847+0000 mgr.gibba032.whopeo (mgr.24625) 2733 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:44:00.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.482916+0000 mgr.gibba032.whopeo (mgr.24625) 2734 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:44:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.483576+0000 mgr.gibba032.whopeo (mgr.24625) 2735 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:44:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.483613+0000 mgr.gibba032.whopeo (mgr.24625) 2736 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:44:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.485069+0000 mgr.gibba032.whopeo (mgr.24625) 2737 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:44:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.485394+0000 mgr.gibba032.whopeo (mgr.24625) 2738 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:44:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.486306+0000 mgr.gibba032.whopeo (mgr.24625) 2739 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:44:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.486368+0000 mgr.gibba032.whopeo (mgr.24625) 2740 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:44:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cephadm 2021-03-10T14:43:58.486482+0000 mgr.gibba032.whopeo (mgr.24625) 2741 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:00 gibba011 bash[31774]: cluster 2021-03-10T14:43:58.627000+0000 mgr.gibba032.whopeo (mgr.24625) 2742 : cluster [DBG] pgmap v1545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:02 gibba032 bash[24004]: cluster 2021-03-10T14:44:00.628690+0000 mgr.gibba032.whopeo (mgr.24625) 2743 : cluster [DBG] pgmap v1546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:02 gibba011 bash[30791]: cluster 2021-03-10T14:44:00.628690+0000 mgr.gibba032.whopeo (mgr.24625) 2743 : cluster [DBG] pgmap v1546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:02 gibba011 bash[31774]: cluster 2021-03-10T14:44:00.628690+0000 mgr.gibba032.whopeo (mgr.24625) 2743 : cluster [DBG] pgmap v1546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:04 gibba032 bash[24004]: cluster 2021-03-10T14:44:02.630287+0000 mgr.gibba032.whopeo (mgr.24625) 2744 : cluster [DBG] pgmap v1547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:04 gibba011 bash[30791]: cluster 2021-03-10T14:44:02.630287+0000 mgr.gibba032.whopeo (mgr.24625) 2744 : cluster [DBG] pgmap v1547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:04 gibba011 bash[31774]: cluster 2021-03-10T14:44:02.630287+0000 mgr.gibba032.whopeo (mgr.24625) 2744 : cluster [DBG] pgmap v1547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:05.584 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:44:05.811 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:44:05.811 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (50m) 6m ago 56m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:44:05.811 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (55m) 5m ago 55m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:44:05.811 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (51m) 5m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:44:05.812 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (51m) 5m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:44:05.812 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (50m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:44:05.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (50m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:44:05.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (49m) 5m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:44:05.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (50m) 6m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:44:05.813 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (56m) 6m ago 56m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:44:05.813 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (56m) 5m ago 56m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:44:05.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (49m) 6m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:44:05.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:44:05.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:44:05.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (47m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:44:05.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (47m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:44:05.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (46m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:44:05.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (46m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:44:05.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (46m) 5m ago 56m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:44:05.815 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (55m) 5m ago 56m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:44:05.815 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 54m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:44:06.059 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:44:06.060 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:44:06.060 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:44:06.060 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:06.060 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:06.061 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:44:06.062 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:44:06.062 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:44:06.062 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:44:06.062 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:44:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:06 gibba032 bash[24004]: cluster 2021-03-10T14:44:04.631835+0000 mgr.gibba032.whopeo (mgr.24625) 2745 : cluster [DBG] pgmap v1548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:06 gibba032 bash[24004]: audit 2021-03-10T14:44:06.058362+0000 mon.a (mon.0) 1342 : audit [DBG] from='client.? 172.21.2.111:0/2469043683' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:06 gibba011 bash[30791]: cluster 2021-03-10T14:44:04.631835+0000 mgr.gibba032.whopeo (mgr.24625) 2745 : cluster [DBG] pgmap v1548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:06 gibba011 bash[30791]: audit 2021-03-10T14:44:06.058362+0000 mon.a (mon.0) 1342 : audit [DBG] from='client.? 172.21.2.111:0/2469043683' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:06 gibba011 bash[31774]: cluster 2021-03-10T14:44:04.631835+0000 mgr.gibba032.whopeo (mgr.24625) 2745 : cluster [DBG] pgmap v1548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:06 gibba011 bash[31774]: audit 2021-03-10T14:44:06.058362+0000 mon.a (mon.0) 1342 : audit [DBG] from='client.? 172.21.2.111:0/2469043683' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:07 gibba032 bash[24004]: audit 2021-03-10T14:44:05.570880+0000 mgr.gibba032.whopeo (mgr.24625) 2746 : audit [DBG] from='client.55876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:07 gibba032 bash[24004]: audit 2021-03-10T14:44:05.794065+0000 mgr.gibba032.whopeo (mgr.24625) 2747 : audit [DBG] from='client.55882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:07 gibba011 bash[30791]: audit 2021-03-10T14:44:05.570880+0000 mgr.gibba032.whopeo (mgr.24625) 2746 : audit [DBG] from='client.55876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:07 gibba011 bash[30791]: audit 2021-03-10T14:44:05.794065+0000 mgr.gibba032.whopeo (mgr.24625) 2747 : audit [DBG] from='client.55882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:07 gibba011 bash[31774]: audit 2021-03-10T14:44:05.570880+0000 mgr.gibba032.whopeo (mgr.24625) 2746 : audit [DBG] from='client.55876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:07 gibba011 bash[31774]: audit 2021-03-10T14:44:05.794065+0000 mgr.gibba032.whopeo (mgr.24625) 2747 : audit [DBG] from='client.55882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:08 gibba032 bash[24004]: cluster 2021-03-10T14:44:06.633622+0000 mgr.gibba032.whopeo (mgr.24625) 2748 : cluster [DBG] pgmap v1549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:08 gibba011 bash[30791]: cluster 2021-03-10T14:44:06.633622+0000 mgr.gibba032.whopeo (mgr.24625) 2748 : cluster [DBG] pgmap v1549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:08 gibba011 bash[31774]: cluster 2021-03-10T14:44:06.633622+0000 mgr.gibba032.whopeo (mgr.24625) 2748 : cluster [DBG] pgmap v1549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:10 gibba032 bash[24004]: cluster 2021-03-10T14:44:08.635251+0000 mgr.gibba032.whopeo (mgr.24625) 2749 : cluster [DBG] pgmap v1550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:10 gibba011 bash[30791]: cluster 2021-03-10T14:44:08.635251+0000 mgr.gibba032.whopeo (mgr.24625) 2749 : cluster [DBG] pgmap v1550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:10 gibba011 bash[31774]: cluster 2021-03-10T14:44:08.635251+0000 mgr.gibba032.whopeo (mgr.24625) 2749 : cluster [DBG] pgmap v1550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:12 gibba032 bash[24004]: cluster 2021-03-10T14:44:10.636920+0000 mgr.gibba032.whopeo (mgr.24625) 2750 : cluster [DBG] pgmap v1551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:12 gibba011 bash[30791]: cluster 2021-03-10T14:44:10.636920+0000 mgr.gibba032.whopeo (mgr.24625) 2750 : cluster [DBG] pgmap v1551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:12 gibba011 bash[31774]: cluster 2021-03-10T14:44:10.636920+0000 mgr.gibba032.whopeo (mgr.24625) 2750 : cluster [DBG] pgmap v1551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:13 gibba032 bash[24004]: cluster 2021-03-10T14:44:12.638601+0000 mgr.gibba032.whopeo (mgr.24625) 2751 : cluster [DBG] pgmap v1552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:13 gibba011 bash[30791]: cluster 2021-03-10T14:44:12.638601+0000 mgr.gibba032.whopeo (mgr.24625) 2751 : cluster [DBG] pgmap v1552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:13.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:13 gibba011 bash[31774]: cluster 2021-03-10T14:44:12.638601+0000 mgr.gibba032.whopeo (mgr.24625) 2751 : cluster [DBG] pgmap v1552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:14 gibba032 bash[24004]: cephadm 2021-03-10T14:44:13.500537+0000 mgr.gibba032.whopeo (mgr.24625) 2752 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:14 gibba011 bash[30791]: cephadm 2021-03-10T14:44:13.500537+0000 mgr.gibba032.whopeo (mgr.24625) 2752 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:14 gibba011 bash[31774]: cephadm 2021-03-10T14:44:13.500537+0000 mgr.gibba032.whopeo (mgr.24625) 2752 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:15.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:15 gibba032 bash[24004]: cluster 2021-03-10T14:44:14.639956+0000 mgr.gibba032.whopeo (mgr.24625) 2753 : cluster [DBG] pgmap v1553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:15.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:15 gibba011 bash[30791]: cluster 2021-03-10T14:44:14.639956+0000 mgr.gibba032.whopeo (mgr.24625) 2753 : cluster [DBG] pgmap v1553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:15.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:15 gibba011 bash[31774]: cluster 2021-03-10T14:44:14.639956+0000 mgr.gibba032.whopeo (mgr.24625) 2753 : cluster [DBG] pgmap v1553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:18 gibba032 bash[24004]: cluster 2021-03-10T14:44:16.641518+0000 mgr.gibba032.whopeo (mgr.24625) 2754 : cluster [DBG] pgmap v1554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:18 gibba011 bash[30791]: cluster 2021-03-10T14:44:16.641518+0000 mgr.gibba032.whopeo (mgr.24625) 2754 : cluster [DBG] pgmap v1554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:18 gibba011 bash[31774]: cluster 2021-03-10T14:44:16.641518+0000 mgr.gibba032.whopeo (mgr.24625) 2754 : cluster [DBG] pgmap v1554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:20 gibba032 bash[24004]: cluster 2021-03-10T14:44:18.643122+0000 mgr.gibba032.whopeo (mgr.24625) 2755 : cluster [DBG] pgmap v1555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:20 gibba011 bash[30791]: cluster 2021-03-10T14:44:18.643122+0000 mgr.gibba032.whopeo (mgr.24625) 2755 : cluster [DBG] pgmap v1555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:20 gibba011 bash[31774]: cluster 2021-03-10T14:44:18.643122+0000 mgr.gibba032.whopeo (mgr.24625) 2755 : cluster [DBG] pgmap v1555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:22 gibba032 bash[24004]: cluster 2021-03-10T14:44:20.644596+0000 mgr.gibba032.whopeo (mgr.24625) 2756 : cluster [DBG] pgmap v1556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:22 gibba011 bash[30791]: cluster 2021-03-10T14:44:20.644596+0000 mgr.gibba032.whopeo (mgr.24625) 2756 : cluster [DBG] pgmap v1556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:22 gibba011 bash[31774]: cluster 2021-03-10T14:44:20.644596+0000 mgr.gibba032.whopeo (mgr.24625) 2756 : cluster [DBG] pgmap v1556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:24 gibba032 bash[24004]: cluster 2021-03-10T14:44:22.646208+0000 mgr.gibba032.whopeo (mgr.24625) 2757 : cluster [DBG] pgmap v1557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:24 gibba011 bash[30791]: cluster 2021-03-10T14:44:22.646208+0000 mgr.gibba032.whopeo (mgr.24625) 2757 : cluster [DBG] pgmap v1557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:24 gibba011 bash[31774]: cluster 2021-03-10T14:44:22.646208+0000 mgr.gibba032.whopeo (mgr.24625) 2757 : cluster [DBG] pgmap v1557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:26 gibba032 bash[24004]: cluster 2021-03-10T14:44:24.647457+0000 mgr.gibba032.whopeo (mgr.24625) 2758 : cluster [DBG] pgmap v1558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:26 gibba011 bash[30791]: cluster 2021-03-10T14:44:24.647457+0000 mgr.gibba032.whopeo (mgr.24625) 2758 : cluster [DBG] pgmap v1558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:26 gibba011 bash[31774]: cluster 2021-03-10T14:44:24.647457+0000 mgr.gibba032.whopeo (mgr.24625) 2758 : cluster [DBG] pgmap v1558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:28 gibba032 bash[24004]: cluster 2021-03-10T14:44:26.649270+0000 mgr.gibba032.whopeo (mgr.24625) 2759 : cluster [DBG] pgmap v1559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:28 gibba011 bash[30791]: cluster 2021-03-10T14:44:26.649270+0000 mgr.gibba032.whopeo (mgr.24625) 2759 : cluster [DBG] pgmap v1559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:28 gibba011 bash[31774]: cluster 2021-03-10T14:44:26.649270+0000 mgr.gibba032.whopeo (mgr.24625) 2759 : cluster [DBG] pgmap v1559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:30 gibba032 bash[24004]: cephadm 2021-03-10T14:44:28.516487+0000 mgr.gibba032.whopeo (mgr.24625) 2760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:30 gibba032 bash[24004]: cluster 2021-03-10T14:44:28.650872+0000 mgr.gibba032.whopeo (mgr.24625) 2761 : cluster [DBG] pgmap v1560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:30 gibba011 bash[30791]: cephadm 2021-03-10T14:44:28.516487+0000 mgr.gibba032.whopeo (mgr.24625) 2760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:30 gibba011 bash[30791]: cluster 2021-03-10T14:44:28.650872+0000 mgr.gibba032.whopeo (mgr.24625) 2761 : cluster [DBG] pgmap v1560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:30 gibba011 bash[31774]: cephadm 2021-03-10T14:44:28.516487+0000 mgr.gibba032.whopeo (mgr.24625) 2760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:30 gibba011 bash[31774]: cluster 2021-03-10T14:44:28.650872+0000 mgr.gibba032.whopeo (mgr.24625) 2761 : cluster [DBG] pgmap v1560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:32 gibba032 bash[24004]: cluster 2021-03-10T14:44:30.652469+0000 mgr.gibba032.whopeo (mgr.24625) 2762 : cluster [DBG] pgmap v1561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:32 gibba011 bash[30791]: cluster 2021-03-10T14:44:30.652469+0000 mgr.gibba032.whopeo (mgr.24625) 2762 : cluster [DBG] pgmap v1561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:32 gibba011 bash[31774]: cluster 2021-03-10T14:44:30.652469+0000 mgr.gibba032.whopeo (mgr.24625) 2762 : cluster [DBG] pgmap v1561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:34 gibba032 bash[24004]: cluster 2021-03-10T14:44:32.654133+0000 mgr.gibba032.whopeo (mgr.24625) 2763 : cluster [DBG] pgmap v1562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:34 gibba011 bash[30791]: cluster 2021-03-10T14:44:32.654133+0000 mgr.gibba032.whopeo (mgr.24625) 2763 : cluster [DBG] pgmap v1562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:34 gibba011 bash[31774]: cluster 2021-03-10T14:44:32.654133+0000 mgr.gibba032.whopeo (mgr.24625) 2763 : cluster [DBG] pgmap v1562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:35 gibba032 bash[24004]: cluster 2021-03-10T14:44:34.655638+0000 mgr.gibba032.whopeo (mgr.24625) 2764 : cluster [DBG] pgmap v1563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:35 gibba011 bash[30791]: cluster 2021-03-10T14:44:34.655638+0000 mgr.gibba032.whopeo (mgr.24625) 2764 : cluster [DBG] pgmap v1563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:35.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:35 gibba011 bash[31774]: cluster 2021-03-10T14:44:34.655638+0000 mgr.gibba032.whopeo (mgr.24625) 2764 : cluster [DBG] pgmap v1563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:36.304 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:44:36.560 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:44:36.560 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (51m) 6m ago 56m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:44:36.560 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (56m) 5m ago 56m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:44:36.561 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (51m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:44:36.561 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (51m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:44:36.562 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (50m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:44:36.562 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (50m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:44:36.563 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (50m) 5m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:44:36.563 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (50m) 6m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:44:36.563 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (56m) 6m ago 56m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:44:36.564 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (56m) 5m ago 56m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:44:36.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (49m) 6m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:44:36.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (49m) 6m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:44:36.565 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (48m) 6m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:44:36.565 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:44:36.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (47m) 5m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:44:36.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (47m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:44:36.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (46m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:44:36.567 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (46m) 5m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:44:36.567 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (56m) 5m ago 56m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:44:36.568 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 54m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:44:36.809 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:44:36.810 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:44:36.810 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:44:36.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:36.811 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:44:36.811 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:44:36.812 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:36.812 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:44:36.812 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:44:36.813 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:44:36.813 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:44:36.813 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:44:36.814 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:44:36.814 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:44:36.815 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:44:37.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:37 gibba032 bash[24004]: audit 2021-03-10T14:44:36.290690+0000 mgr.gibba032.whopeo (mgr.24625) 2765 : audit [DBG] from='client.45909 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:37.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:37 gibba032 bash[24004]: audit 2021-03-10T14:44:36.807953+0000 mon.a (mon.0) 1343 : audit [DBG] from='client.? 172.21.2.111:0/3506883369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:37 gibba011 bash[30791]: audit 2021-03-10T14:44:36.290690+0000 mgr.gibba032.whopeo (mgr.24625) 2765 : audit [DBG] from='client.45909 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:37 gibba011 bash[30791]: audit 2021-03-10T14:44:36.807953+0000 mon.a (mon.0) 1343 : audit [DBG] from='client.? 172.21.2.111:0/3506883369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:37 gibba011 bash[31774]: audit 2021-03-10T14:44:36.290690+0000 mgr.gibba032.whopeo (mgr.24625) 2765 : audit [DBG] from='client.45909 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:37 gibba011 bash[31774]: audit 2021-03-10T14:44:36.807953+0000 mon.a (mon.0) 1343 : audit [DBG] from='client.? 172.21.2.111:0/3506883369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:44:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:38 gibba032 bash[24004]: audit 2021-03-10T14:44:36.516138+0000 mgr.gibba032.whopeo (mgr.24625) 2766 : audit [DBG] from='client.45915 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:38 gibba032 bash[24004]: cluster 2021-03-10T14:44:36.657269+0000 mgr.gibba032.whopeo (mgr.24625) 2767 : cluster [DBG] pgmap v1564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:38 gibba011 bash[30791]: audit 2021-03-10T14:44:36.516138+0000 mgr.gibba032.whopeo (mgr.24625) 2766 : audit [DBG] from='client.45915 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:38 gibba011 bash[30791]: cluster 2021-03-10T14:44:36.657269+0000 mgr.gibba032.whopeo (mgr.24625) 2767 : cluster [DBG] pgmap v1564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:38 gibba011 bash[31774]: audit 2021-03-10T14:44:36.516138+0000 mgr.gibba032.whopeo (mgr.24625) 2766 : audit [DBG] from='client.45915 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:44:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:38 gibba011 bash[31774]: cluster 2021-03-10T14:44:36.657269+0000 mgr.gibba032.whopeo (mgr.24625) 2767 : cluster [DBG] pgmap v1564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:40 gibba032 bash[24004]: cluster 2021-03-10T14:44:38.658866+0000 mgr.gibba032.whopeo (mgr.24625) 2768 : cluster [DBG] pgmap v1565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:40 gibba011 bash[30791]: cluster 2021-03-10T14:44:38.658866+0000 mgr.gibba032.whopeo (mgr.24625) 2768 : cluster [DBG] pgmap v1565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:40 gibba011 bash[31774]: cluster 2021-03-10T14:44:38.658866+0000 mgr.gibba032.whopeo (mgr.24625) 2768 : cluster [DBG] pgmap v1565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:42 gibba032 bash[24004]: cluster 2021-03-10T14:44:40.659638+0000 mgr.gibba032.whopeo (mgr.24625) 2769 : cluster [DBG] pgmap v1566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:42 gibba011 bash[30791]: cluster 2021-03-10T14:44:40.659638+0000 mgr.gibba032.whopeo (mgr.24625) 2769 : cluster [DBG] pgmap v1566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:42 gibba011 bash[31774]: cluster 2021-03-10T14:44:40.659638+0000 mgr.gibba032.whopeo (mgr.24625) 2769 : cluster [DBG] pgmap v1566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:44 gibba032 bash[24004]: cluster 2021-03-10T14:44:42.661238+0000 mgr.gibba032.whopeo (mgr.24625) 2770 : cluster [DBG] pgmap v1567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:44 gibba011 bash[30791]: cluster 2021-03-10T14:44:42.661238+0000 mgr.gibba032.whopeo (mgr.24625) 2770 : cluster [DBG] pgmap v1567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:44 gibba011 bash[31774]: cluster 2021-03-10T14:44:42.661238+0000 mgr.gibba032.whopeo (mgr.24625) 2770 : cluster [DBG] pgmap v1567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:45.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:45 gibba032 bash[24004]: cephadm 2021-03-10T14:44:43.532080+0000 mgr.gibba032.whopeo (mgr.24625) 2771 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:45.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:45 gibba011 bash[30791]: cephadm 2021-03-10T14:44:43.532080+0000 mgr.gibba032.whopeo (mgr.24625) 2771 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:45.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:45 gibba011 bash[31774]: cephadm 2021-03-10T14:44:43.532080+0000 mgr.gibba032.whopeo (mgr.24625) 2771 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:44:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:46 gibba032 bash[24004]: cluster 2021-03-10T14:44:44.662734+0000 mgr.gibba032.whopeo (mgr.24625) 2772 : cluster [DBG] pgmap v1568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:46 gibba011 bash[30791]: cluster 2021-03-10T14:44:44.662734+0000 mgr.gibba032.whopeo (mgr.24625) 2772 : cluster [DBG] pgmap v1568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:46 gibba011 bash[31774]: cluster 2021-03-10T14:44:44.662734+0000 mgr.gibba032.whopeo (mgr.24625) 2772 : cluster [DBG] pgmap v1568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:48 gibba032 bash[24004]: cluster 2021-03-10T14:44:46.664249+0000 mgr.gibba032.whopeo (mgr.24625) 2773 : cluster [DBG] pgmap v1569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:48 gibba011 bash[30791]: cluster 2021-03-10T14:44:46.664249+0000 mgr.gibba032.whopeo (mgr.24625) 2773 : cluster [DBG] pgmap v1569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:48 gibba011 bash[31774]: cluster 2021-03-10T14:44:46.664249+0000 mgr.gibba032.whopeo (mgr.24625) 2773 : cluster [DBG] pgmap v1569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:50 gibba032 bash[24004]: cluster 2021-03-10T14:44:48.665909+0000 mgr.gibba032.whopeo (mgr.24625) 2774 : cluster [DBG] pgmap v1570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:50.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:50 gibba011 bash[30791]: cluster 2021-03-10T14:44:48.665909+0000 mgr.gibba032.whopeo (mgr.24625) 2774 : cluster [DBG] pgmap v1570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:50 gibba011 bash[31774]: cluster 2021-03-10T14:44:48.665909+0000 mgr.gibba032.whopeo (mgr.24625) 2774 : cluster [DBG] pgmap v1570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:52 gibba032 bash[24004]: cluster 2021-03-10T14:44:50.666469+0000 mgr.gibba032.whopeo (mgr.24625) 2775 : cluster [DBG] pgmap v1571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:52 gibba011 bash[30791]: cluster 2021-03-10T14:44:50.666469+0000 mgr.gibba032.whopeo (mgr.24625) 2775 : cluster [DBG] pgmap v1571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:52 gibba011 bash[31774]: cluster 2021-03-10T14:44:50.666469+0000 mgr.gibba032.whopeo (mgr.24625) 2775 : cluster [DBG] pgmap v1571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:54 gibba032 bash[24004]: cluster 2021-03-10T14:44:52.667917+0000 mgr.gibba032.whopeo (mgr.24625) 2776 : cluster [DBG] pgmap v1572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:54 gibba011 bash[30791]: cluster 2021-03-10T14:44:52.667917+0000 mgr.gibba032.whopeo (mgr.24625) 2776 : cluster [DBG] pgmap v1572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:54 gibba011 bash[31774]: cluster 2021-03-10T14:44:52.667917+0000 mgr.gibba032.whopeo (mgr.24625) 2776 : cluster [DBG] pgmap v1572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: cluster 2021-03-10T14:44:54.669520+0000 mgr.gibba032.whopeo (mgr.24625) 2777 : cluster [DBG] pgmap v1573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.338362+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:44:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.339982+0000 mon.a (mon.0) 1345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:44:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.367135+0000 mon.a (mon.0) 1346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:44:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.367302+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:44:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.367455+0000 mon.a (mon.0) 1348 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:44:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:56 gibba032 bash[24004]: audit 2021-03-10T14:44:56.367631+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:44:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: cluster 2021-03-10T14:44:54.669520+0000 mgr.gibba032.whopeo (mgr.24625) 2777 : cluster [DBG] pgmap v1573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.338362+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:44:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.339982+0000 mon.a (mon.0) 1345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:44:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.367135+0000 mon.a (mon.0) 1346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:44:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.367302+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:44:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.367455+0000 mon.a (mon.0) 1348 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:44:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[30791]: audit 2021-03-10T14:44:56.367631+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:44:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: cluster 2021-03-10T14:44:54.669520+0000 mgr.gibba032.whopeo (mgr.24625) 2777 : cluster [DBG] pgmap v1573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.338362+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:44:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.339982+0000 mon.a (mon.0) 1345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:44:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.367135+0000 mon.a (mon.0) 1346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T14:44:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.367302+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:44:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.367455+0000 mon.a (mon.0) 1348 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:44:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:56 gibba011 bash[31774]: audit 2021-03-10T14:44:56.367631+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:44:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:58 gibba032 bash[24004]: cluster 2021-03-10T14:44:56.671324+0000 mgr.gibba032.whopeo (mgr.24625) 2778 : cluster [DBG] pgmap v1574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:58 gibba011 bash[30791]: cluster 2021-03-10T14:44:56.671324+0000 mgr.gibba032.whopeo (mgr.24625) 2778 : cluster [DBG] pgmap v1574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:44:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:58 gibba011 bash[31774]: cluster 2021-03-10T14:44:56.671324+0000 mgr.gibba032.whopeo (mgr.24625) 2778 : cluster [DBG] pgmap v1574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:00.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cluster 2021-03-10T14:44:58.671861+0000 mgr.gibba032.whopeo (mgr.24625) 2779 : cluster [DBG] pgmap v1575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.740649+0000 mgr.gibba032.whopeo (mgr.24625) 2780 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:45:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.742718+0000 mon.a (mon.0) 1350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.761363+0000 mgr.gibba032.whopeo (mgr.24625) 2781 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:45:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.763448+0000 mon.a (mon.0) 1351 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.952071+0000 mgr.gibba032.whopeo (mgr.24625) 2782 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:45:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.952130+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.954356+0000 mgr.gibba032.whopeo (mgr.24625) 2783 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:45:00.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.955572+0000 mon.a (mon.0) 1353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:45:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.955614+0000 mgr.gibba032.whopeo (mgr.24625) 2784 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:45:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.955679+0000 mgr.gibba032.whopeo (mgr.24625) 2785 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:45:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.956700+0000 mgr.gibba032.whopeo (mgr.24625) 2786 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:45:00.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.956762+0000 mgr.gibba032.whopeo (mgr.24625) 2787 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:45:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.957525+0000 mon.a (mon.0) 1354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.957880+0000 mgr.gibba032.whopeo (mgr.24625) 2788 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:45:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.958660+0000 mon.a (mon.0) 1355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.958835+0000 mgr.gibba032.whopeo (mgr.24625) 2789 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:45:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.958926+0000 mgr.gibba032.whopeo (mgr.24625) 2790 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:45:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.959834+0000 mon.a (mon.0) 1356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.960038+0000 mgr.gibba032.whopeo (mgr.24625) 2791 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:45:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.960441+0000 mgr.gibba032.whopeo (mgr.24625) 2792 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:45:00.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.961524+0000 mgr.gibba032.whopeo (mgr.24625) 2793 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:45:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.961607+0000 mgr.gibba032.whopeo (mgr.24625) 2794 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:45:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: cephadm 2021-03-10T14:44:58.961734+0000 mgr.gibba032.whopeo (mgr.24625) 2795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.961980+0000 mon.a (mon.0) 1357 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:44:59 gibba032 bash[24004]: audit 2021-03-10T14:44:58.963394+0000 mon.a (mon.0) 1358 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cluster 2021-03-10T14:44:58.671861+0000 mgr.gibba032.whopeo (mgr.24625) 2779 : cluster [DBG] pgmap v1575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:00.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.740649+0000 mgr.gibba032.whopeo (mgr.24625) 2780 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:45:00.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.742718+0000 mon.a (mon.0) 1350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.761363+0000 mgr.gibba032.whopeo (mgr.24625) 2781 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:45:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.763448+0000 mon.a (mon.0) 1351 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.952071+0000 mgr.gibba032.whopeo (mgr.24625) 2782 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:45:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.952130+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.954356+0000 mgr.gibba032.whopeo (mgr.24625) 2783 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:45:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.955572+0000 mon.a (mon.0) 1353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:45:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.955614+0000 mgr.gibba032.whopeo (mgr.24625) 2784 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:45:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.955679+0000 mgr.gibba032.whopeo (mgr.24625) 2785 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:45:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.956700+0000 mgr.gibba032.whopeo (mgr.24625) 2786 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:45:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.956762+0000 mgr.gibba032.whopeo (mgr.24625) 2787 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:45:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.957525+0000 mon.a (mon.0) 1354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.957880+0000 mgr.gibba032.whopeo (mgr.24625) 2788 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:45:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.958660+0000 mon.a (mon.0) 1355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.958835+0000 mgr.gibba032.whopeo (mgr.24625) 2789 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:45:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.958926+0000 mgr.gibba032.whopeo (mgr.24625) 2790 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:45:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.959834+0000 mon.a (mon.0) 1356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.960038+0000 mgr.gibba032.whopeo (mgr.24625) 2791 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:45:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.960441+0000 mgr.gibba032.whopeo (mgr.24625) 2792 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:45:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.961524+0000 mgr.gibba032.whopeo (mgr.24625) 2793 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:45:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.961607+0000 mgr.gibba032.whopeo (mgr.24625) 2794 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:45:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: cephadm 2021-03-10T14:44:58.961734+0000 mgr.gibba032.whopeo (mgr.24625) 2795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.961980+0000 mon.a (mon.0) 1357 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[30791]: audit 2021-03-10T14:44:58.963394+0000 mon.a (mon.0) 1358 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cluster 2021-03-10T14:44:58.671861+0000 mgr.gibba032.whopeo (mgr.24625) 2779 : cluster [DBG] pgmap v1575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.740649+0000 mgr.gibba032.whopeo (mgr.24625) 2780 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:45:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.742718+0000 mon.a (mon.0) 1350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.761363+0000 mgr.gibba032.whopeo (mgr.24625) 2781 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:45:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.763448+0000 mon.a (mon.0) 1351 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.952071+0000 mgr.gibba032.whopeo (mgr.24625) 2782 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:45:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.952130+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:45:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.954356+0000 mgr.gibba032.whopeo (mgr.24625) 2783 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:45:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.955572+0000 mon.a (mon.0) 1353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:45:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.955614+0000 mgr.gibba032.whopeo (mgr.24625) 2784 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:45:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.955679+0000 mgr.gibba032.whopeo (mgr.24625) 2785 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:45:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.956700+0000 mgr.gibba032.whopeo (mgr.24625) 2786 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:45:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.956762+0000 mgr.gibba032.whopeo (mgr.24625) 2787 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:45:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.957525+0000 mon.a (mon.0) 1354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.957880+0000 mgr.gibba032.whopeo (mgr.24625) 2788 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:45:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.958660+0000 mon.a (mon.0) 1355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.958835+0000 mgr.gibba032.whopeo (mgr.24625) 2789 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:45:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.958926+0000 mgr.gibba032.whopeo (mgr.24625) 2790 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:45:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.959834+0000 mon.a (mon.0) 1356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.960038+0000 mgr.gibba032.whopeo (mgr.24625) 2791 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:45:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.960441+0000 mgr.gibba032.whopeo (mgr.24625) 2792 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:45:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.961524+0000 mgr.gibba032.whopeo (mgr.24625) 2793 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:45:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.961607+0000 mgr.gibba032.whopeo (mgr.24625) 2794 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:45:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: cephadm 2021-03-10T14:44:58.961734+0000 mgr.gibba032.whopeo (mgr.24625) 2795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.961980+0000 mon.a (mon.0) 1357 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:00.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:44:59 gibba011 bash[31774]: audit 2021-03-10T14:44:58.963394+0000 mon.a (mon.0) 1358 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:02 gibba032 bash[24004]: cluster 2021-03-10T14:45:00.672395+0000 mgr.gibba032.whopeo (mgr.24625) 2796 : cluster [DBG] pgmap v1576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:02 gibba011 bash[30791]: cluster 2021-03-10T14:45:00.672395+0000 mgr.gibba032.whopeo (mgr.24625) 2796 : cluster [DBG] pgmap v1576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:02 gibba011 bash[31774]: cluster 2021-03-10T14:45:00.672395+0000 mgr.gibba032.whopeo (mgr.24625) 2796 : cluster [DBG] pgmap v1576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:04 gibba032 bash[24004]: cluster 2021-03-10T14:45:02.673981+0000 mgr.gibba032.whopeo (mgr.24625) 2797 : cluster [DBG] pgmap v1577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:04 gibba011 bash[30791]: cluster 2021-03-10T14:45:02.673981+0000 mgr.gibba032.whopeo (mgr.24625) 2797 : cluster [DBG] pgmap v1577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:04 gibba011 bash[31774]: cluster 2021-03-10T14:45:02.673981+0000 mgr.gibba032.whopeo (mgr.24625) 2797 : cluster [DBG] pgmap v1577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:06 gibba032 bash[24004]: cluster 2021-03-10T14:45:04.675357+0000 mgr.gibba032.whopeo (mgr.24625) 2798 : cluster [DBG] pgmap v1578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:06 gibba011 bash[30791]: cluster 2021-03-10T14:45:04.675357+0000 mgr.gibba032.whopeo (mgr.24625) 2798 : cluster [DBG] pgmap v1578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:06 gibba011 bash[31774]: cluster 2021-03-10T14:45:04.675357+0000 mgr.gibba032.whopeo (mgr.24625) 2798 : cluster [DBG] pgmap v1578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:07.051 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:45:07.277 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:45:07.278 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (51m) 7m ago 57m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:45:07.278 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (56m) 6m ago 56m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:45:07.278 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (52m) 6m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:45:07.278 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (52m) 6m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:45:07.279 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (51m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:45:07.279 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (51m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:45:07.279 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (50m) 6m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:45:07.279 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (51m) 7m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:45:07.279 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (57m) 7m ago 57m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:45:07.280 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (57m) 6m ago 57m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:45:07.280 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (50m) 7m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:45:07.280 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:45:07.280 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:45:07.280 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (48m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:45:07.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:45:07.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (47m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:45:07.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (47m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:45:07.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (47m) 6m ago 57m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:45:07.281 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (56m) 6m ago 57m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:45:07.282 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 55m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:45:07.525 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:45:07.525 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:45:07.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:45:07.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:07.526 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:45:07.526 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:45:07.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:45:07.527 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:45:07.528 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:45:07.528 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:45:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:08 gibba032 bash[24004]: cluster 2021-03-10T14:45:06.677137+0000 mgr.gibba032.whopeo (mgr.24625) 2799 : cluster [DBG] pgmap v1579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:08 gibba032 bash[24004]: audit 2021-03-10T14:45:07.038219+0000 mgr.gibba032.whopeo (mgr.24625) 2800 : audit [DBG] from='client.45927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:08 gibba032 bash[24004]: audit 2021-03-10T14:45:07.261764+0000 mgr.gibba032.whopeo (mgr.24625) 2801 : audit [DBG] from='client.45933 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:08 gibba032 bash[24004]: audit 2021-03-10T14:45:07.524024+0000 mon.a (mon.0) 1359 : audit [DBG] from='client.? 172.21.2.111:0/1385313410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[30791]: cluster 2021-03-10T14:45:06.677137+0000 mgr.gibba032.whopeo (mgr.24625) 2799 : cluster [DBG] pgmap v1579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[30791]: audit 2021-03-10T14:45:07.038219+0000 mgr.gibba032.whopeo (mgr.24625) 2800 : audit [DBG] from='client.45927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[30791]: audit 2021-03-10T14:45:07.261764+0000 mgr.gibba032.whopeo (mgr.24625) 2801 : audit [DBG] from='client.45933 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[30791]: audit 2021-03-10T14:45:07.524024+0000 mon.a (mon.0) 1359 : audit [DBG] from='client.? 172.21.2.111:0/1385313410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[31774]: cluster 2021-03-10T14:45:06.677137+0000 mgr.gibba032.whopeo (mgr.24625) 2799 : cluster [DBG] pgmap v1579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[31774]: audit 2021-03-10T14:45:07.038219+0000 mgr.gibba032.whopeo (mgr.24625) 2800 : audit [DBG] from='client.45927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[31774]: audit 2021-03-10T14:45:07.261764+0000 mgr.gibba032.whopeo (mgr.24625) 2801 : audit [DBG] from='client.45933 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:08 gibba011 bash[31774]: audit 2021-03-10T14:45:07.524024+0000 mon.a (mon.0) 1359 : audit [DBG] from='client.? 172.21.2.111:0/1385313410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:10 gibba032 bash[24004]: cluster 2021-03-10T14:45:08.678719+0000 mgr.gibba032.whopeo (mgr.24625) 2802 : cluster [DBG] pgmap v1580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:10 gibba011 bash[30791]: cluster 2021-03-10T14:45:08.678719+0000 mgr.gibba032.whopeo (mgr.24625) 2802 : cluster [DBG] pgmap v1580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:10 gibba011 bash[31774]: cluster 2021-03-10T14:45:08.678719+0000 mgr.gibba032.whopeo (mgr.24625) 2802 : cluster [DBG] pgmap v1580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:12 gibba032 bash[24004]: cluster 2021-03-10T14:45:10.679186+0000 mgr.gibba032.whopeo (mgr.24625) 2803 : cluster [DBG] pgmap v1581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:12 gibba011 bash[30791]: cluster 2021-03-10T14:45:10.679186+0000 mgr.gibba032.whopeo (mgr.24625) 2803 : cluster [DBG] pgmap v1581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:12 gibba011 bash[31774]: cluster 2021-03-10T14:45:10.679186+0000 mgr.gibba032.whopeo (mgr.24625) 2803 : cluster [DBG] pgmap v1581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:14 gibba032 bash[24004]: cluster 2021-03-10T14:45:12.680877+0000 mgr.gibba032.whopeo (mgr.24625) 2804 : cluster [DBG] pgmap v1582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:14 gibba011 bash[30791]: cluster 2021-03-10T14:45:12.680877+0000 mgr.gibba032.whopeo (mgr.24625) 2804 : cluster [DBG] pgmap v1582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:14 gibba011 bash[31774]: cluster 2021-03-10T14:45:12.680877+0000 mgr.gibba032.whopeo (mgr.24625) 2804 : cluster [DBG] pgmap v1582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:15.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:15 gibba032 bash[24004]: cephadm 2021-03-10T14:45:13.977783+0000 mgr.gibba032.whopeo (mgr.24625) 2805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:15.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:15 gibba011 bash[30791]: cephadm 2021-03-10T14:45:13.977783+0000 mgr.gibba032.whopeo (mgr.24625) 2805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:15.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:15 gibba011 bash[31774]: cephadm 2021-03-10T14:45:13.977783+0000 mgr.gibba032.whopeo (mgr.24625) 2805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:16 gibba032 bash[24004]: cluster 2021-03-10T14:45:14.682238+0000 mgr.gibba032.whopeo (mgr.24625) 2806 : cluster [DBG] pgmap v1583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:16 gibba011 bash[30791]: cluster 2021-03-10T14:45:14.682238+0000 mgr.gibba032.whopeo (mgr.24625) 2806 : cluster [DBG] pgmap v1583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:16 gibba011 bash[31774]: cluster 2021-03-10T14:45:14.682238+0000 mgr.gibba032.whopeo (mgr.24625) 2806 : cluster [DBG] pgmap v1583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:17 gibba032 bash[24004]: cluster 2021-03-10T14:45:16.683948+0000 mgr.gibba032.whopeo (mgr.24625) 2807 : cluster [DBG] pgmap v1584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:17 gibba011 bash[30791]: cluster 2021-03-10T14:45:16.683948+0000 mgr.gibba032.whopeo (mgr.24625) 2807 : cluster [DBG] pgmap v1584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:17.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:17 gibba011 bash[31774]: cluster 2021-03-10T14:45:16.683948+0000 mgr.gibba032.whopeo (mgr.24625) 2807 : cluster [DBG] pgmap v1584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:19 gibba032 bash[24004]: cluster 2021-03-10T14:45:18.685587+0000 mgr.gibba032.whopeo (mgr.24625) 2808 : cluster [DBG] pgmap v1585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:19 gibba011 bash[30791]: cluster 2021-03-10T14:45:18.685587+0000 mgr.gibba032.whopeo (mgr.24625) 2808 : cluster [DBG] pgmap v1585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:20.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:19 gibba011 bash[31774]: cluster 2021-03-10T14:45:18.685587+0000 mgr.gibba032.whopeo (mgr.24625) 2808 : cluster [DBG] pgmap v1585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:22 gibba032 bash[24004]: cluster 2021-03-10T14:45:20.686229+0000 mgr.gibba032.whopeo (mgr.24625) 2809 : cluster [DBG] pgmap v1586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:22 gibba011 bash[30791]: cluster 2021-03-10T14:45:20.686229+0000 mgr.gibba032.whopeo (mgr.24625) 2809 : cluster [DBG] pgmap v1586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:22 gibba011 bash[31774]: cluster 2021-03-10T14:45:20.686229+0000 mgr.gibba032.whopeo (mgr.24625) 2809 : cluster [DBG] pgmap v1586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:24 gibba032 bash[24004]: cluster 2021-03-10T14:45:22.687619+0000 mgr.gibba032.whopeo (mgr.24625) 2810 : cluster [DBG] pgmap v1587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:24 gibba011 bash[30791]: cluster 2021-03-10T14:45:22.687619+0000 mgr.gibba032.whopeo (mgr.24625) 2810 : cluster [DBG] pgmap v1587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:24 gibba011 bash[31774]: cluster 2021-03-10T14:45:22.687619+0000 mgr.gibba032.whopeo (mgr.24625) 2810 : cluster [DBG] pgmap v1587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:26 gibba032 bash[24004]: cluster 2021-03-10T14:45:24.689125+0000 mgr.gibba032.whopeo (mgr.24625) 2811 : cluster [DBG] pgmap v1588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:26 gibba011 bash[30791]: cluster 2021-03-10T14:45:24.689125+0000 mgr.gibba032.whopeo (mgr.24625) 2811 : cluster [DBG] pgmap v1588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:26 gibba011 bash[31774]: cluster 2021-03-10T14:45:24.689125+0000 mgr.gibba032.whopeo (mgr.24625) 2811 : cluster [DBG] pgmap v1588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:28 gibba032 bash[24004]: cluster 2021-03-10T14:45:26.690736+0000 mgr.gibba032.whopeo (mgr.24625) 2812 : cluster [DBG] pgmap v1589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:28 gibba011 bash[30791]: cluster 2021-03-10T14:45:26.690736+0000 mgr.gibba032.whopeo (mgr.24625) 2812 : cluster [DBG] pgmap v1589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:28 gibba011 bash[31774]: cluster 2021-03-10T14:45:26.690736+0000 mgr.gibba032.whopeo (mgr.24625) 2812 : cluster [DBG] pgmap v1589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:30 gibba032 bash[24004]: cluster 2021-03-10T14:45:28.692435+0000 mgr.gibba032.whopeo (mgr.24625) 2813 : cluster [DBG] pgmap v1590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:30 gibba032 bash[24004]: cephadm 2021-03-10T14:45:28.993910+0000 mgr.gibba032.whopeo (mgr.24625) 2814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:30 gibba011 bash[30791]: cluster 2021-03-10T14:45:28.692435+0000 mgr.gibba032.whopeo (mgr.24625) 2813 : cluster [DBG] pgmap v1590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:30 gibba011 bash[30791]: cephadm 2021-03-10T14:45:28.993910+0000 mgr.gibba032.whopeo (mgr.24625) 2814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:30 gibba011 bash[31774]: cluster 2021-03-10T14:45:28.692435+0000 mgr.gibba032.whopeo (mgr.24625) 2813 : cluster [DBG] pgmap v1590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:30 gibba011 bash[31774]: cephadm 2021-03-10T14:45:28.993910+0000 mgr.gibba032.whopeo (mgr.24625) 2814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:32 gibba032 bash[24004]: cluster 2021-03-10T14:45:30.693240+0000 mgr.gibba032.whopeo (mgr.24625) 2815 : cluster [DBG] pgmap v1591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:32 gibba011 bash[30791]: cluster 2021-03-10T14:45:30.693240+0000 mgr.gibba032.whopeo (mgr.24625) 2815 : cluster [DBG] pgmap v1591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:32 gibba011 bash[31774]: cluster 2021-03-10T14:45:30.693240+0000 mgr.gibba032.whopeo (mgr.24625) 2815 : cluster [DBG] pgmap v1591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:34 gibba032 bash[24004]: cluster 2021-03-10T14:45:32.694786+0000 mgr.gibba032.whopeo (mgr.24625) 2816 : cluster [DBG] pgmap v1592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:34 gibba011 bash[30791]: cluster 2021-03-10T14:45:32.694786+0000 mgr.gibba032.whopeo (mgr.24625) 2816 : cluster [DBG] pgmap v1592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:34 gibba011 bash[31774]: cluster 2021-03-10T14:45:32.694786+0000 mgr.gibba032.whopeo (mgr.24625) 2816 : cluster [DBG] pgmap v1592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:36 gibba032 bash[24004]: cluster 2021-03-10T14:45:34.696372+0000 mgr.gibba032.whopeo (mgr.24625) 2817 : cluster [DBG] pgmap v1593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:36 gibba011 bash[30791]: cluster 2021-03-10T14:45:34.696372+0000 mgr.gibba032.whopeo (mgr.24625) 2817 : cluster [DBG] pgmap v1593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:36 gibba011 bash[31774]: cluster 2021-03-10T14:45:34.696372+0000 mgr.gibba032.whopeo (mgr.24625) 2817 : cluster [DBG] pgmap v1593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:37.767 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:45:37.992 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:45:37.993 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (52m) 7m ago 57m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:45:37.993 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (57m) 6m ago 57m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:45:37.993 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (52m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:45:37.993 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (52m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:45:37.993 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (51m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:45:37.994 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (51m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:45:37.994 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (51m) 6m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:45:37.994 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (51m) 7m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:45:37.994 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (57m) 7m ago 57m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:45:37.995 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (57m) 6m ago 57m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:45:37.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (50m) 7m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:45:37.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (50m) 7m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:45:37.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (49m) 7m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:45:37.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:45:37.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (48m) 6m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:45:37.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:45:37.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (48m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:45:37.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (47m) 6m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:45:37.997 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (57m) 6m ago 57m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:45:37.997 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 55m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:45:38.251 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:45:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:45:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:45:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:45:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:45:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:45:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:45:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:45:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:45:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:45:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:45:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:45:38.254 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:45:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:38 gibba032 bash[24004]: cluster 2021-03-10T14:45:36.697915+0000 mgr.gibba032.whopeo (mgr.24625) 2818 : cluster [DBG] pgmap v1594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:38 gibba032 bash[24004]: audit 2021-03-10T14:45:38.249964+0000 mon.c (mon.1) 54 : audit [DBG] from='client.? 172.21.2.111:0/1076915733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:38 gibba011 bash[30791]: cluster 2021-03-10T14:45:36.697915+0000 mgr.gibba032.whopeo (mgr.24625) 2818 : cluster [DBG] pgmap v1594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:38 gibba011 bash[30791]: audit 2021-03-10T14:45:38.249964+0000 mon.c (mon.1) 54 : audit [DBG] from='client.? 172.21.2.111:0/1076915733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:38 gibba011 bash[31774]: cluster 2021-03-10T14:45:36.697915+0000 mgr.gibba032.whopeo (mgr.24625) 2818 : cluster [DBG] pgmap v1594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:38 gibba011 bash[31774]: audit 2021-03-10T14:45:38.249964+0000 mon.c (mon.1) 54 : audit [DBG] from='client.? 172.21.2.111:0/1076915733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:45:39.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:39 gibba032 bash[24004]: audit 2021-03-10T14:45:37.753950+0000 mgr.gibba032.whopeo (mgr.24625) 2819 : audit [DBG] from='client.45945 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:39 gibba032 bash[24004]: audit 2021-03-10T14:45:37.976015+0000 mgr.gibba032.whopeo (mgr.24625) 2820 : audit [DBG] from='client.55936 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:39 gibba032 bash[24004]: cluster 2021-03-10T14:45:38.699501+0000 mgr.gibba032.whopeo (mgr.24625) 2821 : cluster [DBG] pgmap v1595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[30791]: audit 2021-03-10T14:45:37.753950+0000 mgr.gibba032.whopeo (mgr.24625) 2819 : audit [DBG] from='client.45945 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[30791]: audit 2021-03-10T14:45:37.976015+0000 mgr.gibba032.whopeo (mgr.24625) 2820 : audit [DBG] from='client.55936 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[30791]: cluster 2021-03-10T14:45:38.699501+0000 mgr.gibba032.whopeo (mgr.24625) 2821 : cluster [DBG] pgmap v1595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[31774]: audit 2021-03-10T14:45:37.753950+0000 mgr.gibba032.whopeo (mgr.24625) 2819 : audit [DBG] from='client.45945 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[31774]: audit 2021-03-10T14:45:37.976015+0000 mgr.gibba032.whopeo (mgr.24625) 2820 : audit [DBG] from='client.55936 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:45:39.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:39 gibba011 bash[31774]: cluster 2021-03-10T14:45:38.699501+0000 mgr.gibba032.whopeo (mgr.24625) 2821 : cluster [DBG] pgmap v1595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:42 gibba032 bash[24004]: cluster 2021-03-10T14:45:40.700102+0000 mgr.gibba032.whopeo (mgr.24625) 2822 : cluster [DBG] pgmap v1596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:42 gibba011 bash[30791]: cluster 2021-03-10T14:45:40.700102+0000 mgr.gibba032.whopeo (mgr.24625) 2822 : cluster [DBG] pgmap v1596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:42 gibba011 bash[31774]: cluster 2021-03-10T14:45:40.700102+0000 mgr.gibba032.whopeo (mgr.24625) 2822 : cluster [DBG] pgmap v1596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:44 gibba032 bash[24004]: cluster 2021-03-10T14:45:42.701583+0000 mgr.gibba032.whopeo (mgr.24625) 2823 : cluster [DBG] pgmap v1597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:44 gibba011 bash[30791]: cluster 2021-03-10T14:45:42.701583+0000 mgr.gibba032.whopeo (mgr.24625) 2823 : cluster [DBG] pgmap v1597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:44 gibba011 bash[31774]: cluster 2021-03-10T14:45:42.701583+0000 mgr.gibba032.whopeo (mgr.24625) 2823 : cluster [DBG] pgmap v1597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:45.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:45 gibba032 bash[24004]: cephadm 2021-03-10T14:45:44.009693+0000 mgr.gibba032.whopeo (mgr.24625) 2824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:45.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:45 gibba011 bash[30791]: cephadm 2021-03-10T14:45:44.009693+0000 mgr.gibba032.whopeo (mgr.24625) 2824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:45.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:45 gibba011 bash[31774]: cephadm 2021-03-10T14:45:44.009693+0000 mgr.gibba032.whopeo (mgr.24625) 2824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:45:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:46 gibba032 bash[24004]: cluster 2021-03-10T14:45:44.702948+0000 mgr.gibba032.whopeo (mgr.24625) 2825 : cluster [DBG] pgmap v1598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:46 gibba011 bash[30791]: cluster 2021-03-10T14:45:44.702948+0000 mgr.gibba032.whopeo (mgr.24625) 2825 : cluster [DBG] pgmap v1598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:46 gibba011 bash[31774]: cluster 2021-03-10T14:45:44.702948+0000 mgr.gibba032.whopeo (mgr.24625) 2825 : cluster [DBG] pgmap v1598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:48 gibba032 bash[24004]: cluster 2021-03-10T14:45:46.704651+0000 mgr.gibba032.whopeo (mgr.24625) 2826 : cluster [DBG] pgmap v1599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:48 gibba011 bash[30791]: cluster 2021-03-10T14:45:46.704651+0000 mgr.gibba032.whopeo (mgr.24625) 2826 : cluster [DBG] pgmap v1599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:48 gibba011 bash[31774]: cluster 2021-03-10T14:45:46.704651+0000 mgr.gibba032.whopeo (mgr.24625) 2826 : cluster [DBG] pgmap v1599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:50 gibba032 bash[24004]: cluster 2021-03-10T14:45:48.706300+0000 mgr.gibba032.whopeo (mgr.24625) 2827 : cluster [DBG] pgmap v1600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:50.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:50 gibba011 bash[30791]: cluster 2021-03-10T14:45:48.706300+0000 mgr.gibba032.whopeo (mgr.24625) 2827 : cluster [DBG] pgmap v1600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:50 gibba011 bash[31774]: cluster 2021-03-10T14:45:48.706300+0000 mgr.gibba032.whopeo (mgr.24625) 2827 : cluster [DBG] pgmap v1600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:52 gibba032 bash[24004]: cluster 2021-03-10T14:45:50.707644+0000 mgr.gibba032.whopeo (mgr.24625) 2828 : cluster [DBG] pgmap v1601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:52 gibba011 bash[30791]: cluster 2021-03-10T14:45:50.707644+0000 mgr.gibba032.whopeo (mgr.24625) 2828 : cluster [DBG] pgmap v1601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:52 gibba011 bash[31774]: cluster 2021-03-10T14:45:50.707644+0000 mgr.gibba032.whopeo (mgr.24625) 2828 : cluster [DBG] pgmap v1601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:53 gibba032 bash[24004]: cluster 2021-03-10T14:45:52.709304+0000 mgr.gibba032.whopeo (mgr.24625) 2829 : cluster [DBG] pgmap v1602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:53.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:53 gibba011 bash[30791]: cluster 2021-03-10T14:45:52.709304+0000 mgr.gibba032.whopeo (mgr.24625) 2829 : cluster [DBG] pgmap v1602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:53.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:53 gibba011 bash[31774]: cluster 2021-03-10T14:45:52.709304+0000 mgr.gibba032.whopeo (mgr.24625) 2829 : cluster [DBG] pgmap v1602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:55 gibba011 bash[30791]: cluster 2021-03-10T14:45:54.710624+0000 mgr.gibba032.whopeo (mgr.24625) 2830 : cluster [DBG] pgmap v1603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:55 gibba011 bash[31774]: cluster 2021-03-10T14:45:54.710624+0000 mgr.gibba032.whopeo (mgr.24625) 2830 : cluster [DBG] pgmap v1603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:55 gibba032 bash[24004]: cluster 2021-03-10T14:45:54.710624+0000 mgr.gibba032.whopeo (mgr.24625) 2830 : cluster [DBG] pgmap v1603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.345792+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:45:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.346435+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:45:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.374978+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:45:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.375762+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.376424+0000 mon.a (mon.0) 1364 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[30791]: audit 2021-03-10T14:45:56.377075+0000 mon.a (mon.0) 1365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:45:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.345792+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:45:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.346435+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:45:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.374978+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:45:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.375762+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.376424+0000 mon.a (mon.0) 1364 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:56 gibba011 bash[31774]: audit 2021-03-10T14:45:56.377075+0000 mon.a (mon.0) 1365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:45:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.345792+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:45:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.346435+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:45:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.374978+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:45:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.375762+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.376424+0000 mon.a (mon.0) 1364 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:45:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:56 gibba032 bash[24004]: audit 2021-03-10T14:45:56.377075+0000 mon.a (mon.0) 1365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T14:45:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:45:58 gibba032 bash[24004]: cluster 2021-03-10T14:45:56.712387+0000 mgr.gibba032.whopeo (mgr.24625) 2831 : cluster [DBG] pgmap v1604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:45:58 gibba011 bash[30791]: cluster 2021-03-10T14:45:56.712387+0000 mgr.gibba032.whopeo (mgr.24625) 2831 : cluster [DBG] pgmap v1604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:45:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:45:58 gibba011 bash[31774]: cluster 2021-03-10T14:45:56.712387+0000 mgr.gibba032.whopeo (mgr.24625) 2831 : cluster [DBG] pgmap v1604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cluster 2021-03-10T14:45:58.713980+0000 mgr.gibba032.whopeo (mgr.24625) 2832 : cluster [DBG] pgmap v1605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.017837+0000 mgr.gibba032.whopeo (mgr.24625) 2833 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:46:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.019461+0000 mgr.gibba032.whopeo (mgr.24625) 2834 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:46:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.238774+0000 mgr.gibba032.whopeo (mgr.24625) 2835 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:46:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.239103+0000 mon.a (mon.0) 1366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:46:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.240478+0000 mgr.gibba032.whopeo (mgr.24625) 2836 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:46:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.241797+0000 mgr.gibba032.whopeo (mgr.24625) 2837 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:46:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.241832+0000 mgr.gibba032.whopeo (mgr.24625) 2838 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:46:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.242161+0000 mon.a (mon.0) 1367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:46:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.243027+0000 mgr.gibba032.whopeo (mgr.24625) 2839 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:46:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.243064+0000 mgr.gibba032.whopeo (mgr.24625) 2840 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:46:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.243518+0000 mon.a (mon.0) 1368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.244196+0000 mgr.gibba032.whopeo (mgr.24625) 2841 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:46:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.244853+0000 mon.a (mon.0) 1369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.245176+0000 mgr.gibba032.whopeo (mgr.24625) 2842 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:46:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.245213+0000 mgr.gibba032.whopeo (mgr.24625) 2843 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:46:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.246050+0000 mon.a (mon.0) 1370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.246388+0000 mgr.gibba032.whopeo (mgr.24625) 2844 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:46:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.246724+0000 mgr.gibba032.whopeo (mgr.24625) 2845 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:46:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.247766+0000 mgr.gibba032.whopeo (mgr.24625) 2846 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:46:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.247838+0000 mgr.gibba032.whopeo (mgr.24625) 2847 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:46:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: cephadm 2021-03-10T14:45:59.247946+0000 mgr.gibba032.whopeo (mgr.24625) 2848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.248312+0000 mon.a (mon.0) 1371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:00 gibba032 bash[24004]: audit 2021-03-10T14:45:59.249687+0000 mon.a (mon.0) 1372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cluster 2021-03-10T14:45:58.713980+0000 mgr.gibba032.whopeo (mgr.24625) 2832 : cluster [DBG] pgmap v1605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.017837+0000 mgr.gibba032.whopeo (mgr.24625) 2833 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:46:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.019461+0000 mgr.gibba032.whopeo (mgr.24625) 2834 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:46:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.238774+0000 mgr.gibba032.whopeo (mgr.24625) 2835 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:46:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.239103+0000 mon.a (mon.0) 1366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:46:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.240478+0000 mgr.gibba032.whopeo (mgr.24625) 2836 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:46:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.241797+0000 mgr.gibba032.whopeo (mgr.24625) 2837 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:46:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.241832+0000 mgr.gibba032.whopeo (mgr.24625) 2838 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:46:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.242161+0000 mon.a (mon.0) 1367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:46:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.243027+0000 mgr.gibba032.whopeo (mgr.24625) 2839 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:46:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.243064+0000 mgr.gibba032.whopeo (mgr.24625) 2840 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:46:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.243518+0000 mon.a (mon.0) 1368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.244196+0000 mgr.gibba032.whopeo (mgr.24625) 2841 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:46:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.244853+0000 mon.a (mon.0) 1369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.245176+0000 mgr.gibba032.whopeo (mgr.24625) 2842 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:46:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.245213+0000 mgr.gibba032.whopeo (mgr.24625) 2843 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:46:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.246050+0000 mon.a (mon.0) 1370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.246388+0000 mgr.gibba032.whopeo (mgr.24625) 2844 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:46:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.246724+0000 mgr.gibba032.whopeo (mgr.24625) 2845 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:46:00.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.247766+0000 mgr.gibba032.whopeo (mgr.24625) 2846 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:46:00.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.247838+0000 mgr.gibba032.whopeo (mgr.24625) 2847 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:46:00.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cluster 2021-03-10T14:45:58.713980+0000 mgr.gibba032.whopeo (mgr.24625) 2832 : cluster [DBG] pgmap v1605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:00.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.017837+0000 mgr.gibba032.whopeo (mgr.24625) 2833 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:46:00.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.019461+0000 mgr.gibba032.whopeo (mgr.24625) 2834 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:46:00.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.238774+0000 mgr.gibba032.whopeo (mgr.24625) 2835 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:46:00.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.239103+0000 mon.a (mon.0) 1366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:46:00.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.240478+0000 mgr.gibba032.whopeo (mgr.24625) 2836 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:46:00.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.241797+0000 mgr.gibba032.whopeo (mgr.24625) 2837 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:46:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.241832+0000 mgr.gibba032.whopeo (mgr.24625) 2838 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:46:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.242161+0000 mon.a (mon.0) 1367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:46:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.243027+0000 mgr.gibba032.whopeo (mgr.24625) 2839 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:46:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.243064+0000 mgr.gibba032.whopeo (mgr.24625) 2840 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:46:00.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.243518+0000 mon.a (mon.0) 1368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.244196+0000 mgr.gibba032.whopeo (mgr.24625) 2841 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:46:00.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.244853+0000 mon.a (mon.0) 1369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.245176+0000 mgr.gibba032.whopeo (mgr.24625) 2842 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:46:00.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.245213+0000 mgr.gibba032.whopeo (mgr.24625) 2843 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:46:00.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.246050+0000 mon.a (mon.0) 1370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.246388+0000 mgr.gibba032.whopeo (mgr.24625) 2844 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:46:00.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.246724+0000 mgr.gibba032.whopeo (mgr.24625) 2845 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:46:00.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.247766+0000 mgr.gibba032.whopeo (mgr.24625) 2846 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:46:00.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.247838+0000 mgr.gibba032.whopeo (mgr.24625) 2847 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:46:00.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: cephadm 2021-03-10T14:45:59.247946+0000 mgr.gibba032.whopeo (mgr.24625) 2848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:00.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.248312+0000 mon.a (mon.0) 1371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[31774]: audit 2021-03-10T14:45:59.249687+0000 mon.a (mon.0) 1372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: cephadm 2021-03-10T14:45:59.247946+0000 mgr.gibba032.whopeo (mgr.24625) 2848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:00.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.248312+0000 mon.a (mon.0) 1371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:00.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:00 gibba011 bash[30791]: audit 2021-03-10T14:45:59.249687+0000 mon.a (mon.0) 1372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:02 gibba032 bash[24004]: cluster 2021-03-10T14:46:00.715348+0000 mgr.gibba032.whopeo (mgr.24625) 2849 : cluster [DBG] pgmap v1606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:02 gibba011 bash[30791]: cluster 2021-03-10T14:46:00.715348+0000 mgr.gibba032.whopeo (mgr.24625) 2849 : cluster [DBG] pgmap v1606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:02 gibba011 bash[31774]: cluster 2021-03-10T14:46:00.715348+0000 mgr.gibba032.whopeo (mgr.24625) 2849 : cluster [DBG] pgmap v1606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:04 gibba032 bash[24004]: cluster 2021-03-10T14:46:02.717186+0000 mgr.gibba032.whopeo (mgr.24625) 2850 : cluster [DBG] pgmap v1607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:04 gibba011 bash[30791]: cluster 2021-03-10T14:46:02.717186+0000 mgr.gibba032.whopeo (mgr.24625) 2850 : cluster [DBG] pgmap v1607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:04 gibba011 bash[31774]: cluster 2021-03-10T14:46:02.717186+0000 mgr.gibba032.whopeo (mgr.24625) 2850 : cluster [DBG] pgmap v1607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:06 gibba032 bash[24004]: cluster 2021-03-10T14:46:04.718441+0000 mgr.gibba032.whopeo (mgr.24625) 2851 : cluster [DBG] pgmap v1608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:06 gibba011 bash[30791]: cluster 2021-03-10T14:46:04.718441+0000 mgr.gibba032.whopeo (mgr.24625) 2851 : cluster [DBG] pgmap v1608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:06 gibba011 bash[31774]: cluster 2021-03-10T14:46:04.718441+0000 mgr.gibba032.whopeo (mgr.24625) 2851 : cluster [DBG] pgmap v1608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:08.501 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:46:08.732 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:46:08.732 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (52m) 8m ago 58m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:46:08.732 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (57m) 7m ago 57m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:46:08.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (53m) 7m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:46:08.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (53m) 7m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:46:08.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (52m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:46:08.733 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (52m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:46:08.734 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (51m) 7m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:46:08.734 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (52m) 8m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:46:08.734 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (58m) 8m ago 58m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:46:08.734 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (58m) 7m ago 58m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:46:08.734 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (51m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:46:08.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (50m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:46:08.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (50m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:46:08.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (49m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:46:08.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:46:08.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (48m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:46:08.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (48m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:46:08.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (48m) 7m ago 58m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:46:08.736 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (57m) 7m ago 58m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:46:08.736 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 56m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:46:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:08 gibba032 bash[24004]: cluster 2021-03-10T14:46:06.720122+0000 mgr.gibba032.whopeo (mgr.24625) 2852 : cluster [DBG] pgmap v1609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:08 gibba011 bash[30791]: cluster 2021-03-10T14:46:06.720122+0000 mgr.gibba032.whopeo (mgr.24625) 2852 : cluster [DBG] pgmap v1609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:08 gibba011 bash[31774]: cluster 2021-03-10T14:46:06.720122+0000 mgr.gibba032.whopeo (mgr.24625) 2852 : cluster [DBG] pgmap v1609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:08.982 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:46:08.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:46:08.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:46:08.983 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:08.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:46:08.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:46:08.984 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:08.984 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:46:08.984 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:46:08.984 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:08.984 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:46:08.985 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:46:08.985 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:46:08.985 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:46:08.985 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:46:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:09 gibba032 bash[24004]: audit 2021-03-10T14:46:08.980801+0000 mon.a (mon.0) 1373 : audit [DBG] from='client.? 172.21.2.111:0/2640930354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:09 gibba011 bash[30791]: audit 2021-03-10T14:46:08.980801+0000 mon.a (mon.0) 1373 : audit [DBG] from='client.? 172.21.2.111:0/2640930354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:09.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:09 gibba011 bash[31774]: audit 2021-03-10T14:46:08.980801+0000 mon.a (mon.0) 1373 : audit [DBG] from='client.? 172.21.2.111:0/2640930354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:10 gibba032 bash[24004]: audit 2021-03-10T14:46:08.488589+0000 mgr.gibba032.whopeo (mgr.24625) 2853 : audit [DBG] from='client.45963 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:10 gibba032 bash[24004]: audit 2021-03-10T14:46:08.715190+0000 mgr.gibba032.whopeo (mgr.24625) 2854 : audit [DBG] from='client.55954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:10 gibba032 bash[24004]: cluster 2021-03-10T14:46:08.721235+0000 mgr.gibba032.whopeo (mgr.24625) 2855 : cluster [DBG] pgmap v1610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[30791]: audit 2021-03-10T14:46:08.488589+0000 mgr.gibba032.whopeo (mgr.24625) 2853 : audit [DBG] from='client.45963 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[30791]: audit 2021-03-10T14:46:08.715190+0000 mgr.gibba032.whopeo (mgr.24625) 2854 : audit [DBG] from='client.55954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[30791]: cluster 2021-03-10T14:46:08.721235+0000 mgr.gibba032.whopeo (mgr.24625) 2855 : cluster [DBG] pgmap v1610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[31774]: audit 2021-03-10T14:46:08.488589+0000 mgr.gibba032.whopeo (mgr.24625) 2853 : audit [DBG] from='client.45963 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[31774]: audit 2021-03-10T14:46:08.715190+0000 mgr.gibba032.whopeo (mgr.24625) 2854 : audit [DBG] from='client.55954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:10 gibba011 bash[31774]: cluster 2021-03-10T14:46:08.721235+0000 mgr.gibba032.whopeo (mgr.24625) 2855 : cluster [DBG] pgmap v1610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:12 gibba032 bash[24004]: cluster 2021-03-10T14:46:10.722316+0000 mgr.gibba032.whopeo (mgr.24625) 2856 : cluster [DBG] pgmap v1611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:12 gibba011 bash[30791]: cluster 2021-03-10T14:46:10.722316+0000 mgr.gibba032.whopeo (mgr.24625) 2856 : cluster [DBG] pgmap v1611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:12 gibba011 bash[31774]: cluster 2021-03-10T14:46:10.722316+0000 mgr.gibba032.whopeo (mgr.24625) 2856 : cluster [DBG] pgmap v1611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:14 gibba032 bash[24004]: cluster 2021-03-10T14:46:12.724148+0000 mgr.gibba032.whopeo (mgr.24625) 2857 : cluster [DBG] pgmap v1612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:14 gibba011 bash[30791]: cluster 2021-03-10T14:46:12.724148+0000 mgr.gibba032.whopeo (mgr.24625) 2857 : cluster [DBG] pgmap v1612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:14 gibba011 bash[31774]: cluster 2021-03-10T14:46:12.724148+0000 mgr.gibba032.whopeo (mgr.24625) 2857 : cluster [DBG] pgmap v1612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:15.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:15 gibba032 bash[24004]: cephadm 2021-03-10T14:46:14.263970+0000 mgr.gibba032.whopeo (mgr.24625) 2858 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:15.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:15 gibba011 bash[30791]: cephadm 2021-03-10T14:46:14.263970+0000 mgr.gibba032.whopeo (mgr.24625) 2858 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:15.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:15 gibba011 bash[31774]: cephadm 2021-03-10T14:46:14.263970+0000 mgr.gibba032.whopeo (mgr.24625) 2858 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:16 gibba032 bash[24004]: cluster 2021-03-10T14:46:14.725344+0000 mgr.gibba032.whopeo (mgr.24625) 2859 : cluster [DBG] pgmap v1613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:16 gibba011 bash[30791]: cluster 2021-03-10T14:46:14.725344+0000 mgr.gibba032.whopeo (mgr.24625) 2859 : cluster [DBG] pgmap v1613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:16 gibba011 bash[31774]: cluster 2021-03-10T14:46:14.725344+0000 mgr.gibba032.whopeo (mgr.24625) 2859 : cluster [DBG] pgmap v1613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:17 gibba032 bash[24004]: cluster 2021-03-10T14:46:16.726954+0000 mgr.gibba032.whopeo (mgr.24625) 2860 : cluster [DBG] pgmap v1614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:17 gibba011 bash[30791]: cluster 2021-03-10T14:46:16.726954+0000 mgr.gibba032.whopeo (mgr.24625) 2860 : cluster [DBG] pgmap v1614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:17.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:17 gibba011 bash[31774]: cluster 2021-03-10T14:46:16.726954+0000 mgr.gibba032.whopeo (mgr.24625) 2860 : cluster [DBG] pgmap v1614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:19 gibba011 bash[30791]: cluster 2021-03-10T14:46:18.728498+0000 mgr.gibba032.whopeo (mgr.24625) 2861 : cluster [DBG] pgmap v1615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:20.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:19 gibba011 bash[31774]: cluster 2021-03-10T14:46:18.728498+0000 mgr.gibba032.whopeo (mgr.24625) 2861 : cluster [DBG] pgmap v1615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:19 gibba032 bash[24004]: cluster 2021-03-10T14:46:18.728498+0000 mgr.gibba032.whopeo (mgr.24625) 2861 : cluster [DBG] pgmap v1615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:22 gibba032 bash[24004]: cluster 2021-03-10T14:46:20.729824+0000 mgr.gibba032.whopeo (mgr.24625) 2862 : cluster [DBG] pgmap v1616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:22 gibba011 bash[30791]: cluster 2021-03-10T14:46:20.729824+0000 mgr.gibba032.whopeo (mgr.24625) 2862 : cluster [DBG] pgmap v1616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:22 gibba011 bash[31774]: cluster 2021-03-10T14:46:20.729824+0000 mgr.gibba032.whopeo (mgr.24625) 2862 : cluster [DBG] pgmap v1616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:24 gibba032 bash[24004]: cluster 2021-03-10T14:46:22.731567+0000 mgr.gibba032.whopeo (mgr.24625) 2863 : cluster [DBG] pgmap v1617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:24 gibba011 bash[30791]: cluster 2021-03-10T14:46:22.731567+0000 mgr.gibba032.whopeo (mgr.24625) 2863 : cluster [DBG] pgmap v1617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:24 gibba011 bash[31774]: cluster 2021-03-10T14:46:22.731567+0000 mgr.gibba032.whopeo (mgr.24625) 2863 : cluster [DBG] pgmap v1617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:26 gibba032 bash[24004]: cluster 2021-03-10T14:46:24.732974+0000 mgr.gibba032.whopeo (mgr.24625) 2864 : cluster [DBG] pgmap v1618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:26 gibba011 bash[30791]: cluster 2021-03-10T14:46:24.732974+0000 mgr.gibba032.whopeo (mgr.24625) 2864 : cluster [DBG] pgmap v1618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:26 gibba011 bash[31774]: cluster 2021-03-10T14:46:24.732974+0000 mgr.gibba032.whopeo (mgr.24625) 2864 : cluster [DBG] pgmap v1618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:28 gibba032 bash[24004]: cluster 2021-03-10T14:46:26.734657+0000 mgr.gibba032.whopeo (mgr.24625) 2865 : cluster [DBG] pgmap v1619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:28 gibba011 bash[30791]: cluster 2021-03-10T14:46:26.734657+0000 mgr.gibba032.whopeo (mgr.24625) 2865 : cluster [DBG] pgmap v1619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:28 gibba011 bash[31774]: cluster 2021-03-10T14:46:26.734657+0000 mgr.gibba032.whopeo (mgr.24625) 2865 : cluster [DBG] pgmap v1619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:30 gibba032 bash[24004]: cluster 2021-03-10T14:46:28.736274+0000 mgr.gibba032.whopeo (mgr.24625) 2866 : cluster [DBG] pgmap v1620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:30 gibba032 bash[24004]: cephadm 2021-03-10T14:46:29.280059+0000 mgr.gibba032.whopeo (mgr.24625) 2867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:30 gibba011 bash[30791]: cluster 2021-03-10T14:46:28.736274+0000 mgr.gibba032.whopeo (mgr.24625) 2866 : cluster [DBG] pgmap v1620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:30 gibba011 bash[30791]: cephadm 2021-03-10T14:46:29.280059+0000 mgr.gibba032.whopeo (mgr.24625) 2867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:30 gibba011 bash[31774]: cluster 2021-03-10T14:46:28.736274+0000 mgr.gibba032.whopeo (mgr.24625) 2866 : cluster [DBG] pgmap v1620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:30 gibba011 bash[31774]: cephadm 2021-03-10T14:46:29.280059+0000 mgr.gibba032.whopeo (mgr.24625) 2867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:32 gibba032 bash[24004]: cluster 2021-03-10T14:46:30.737615+0000 mgr.gibba032.whopeo (mgr.24625) 2868 : cluster [DBG] pgmap v1621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:32 gibba011 bash[30791]: cluster 2021-03-10T14:46:30.737615+0000 mgr.gibba032.whopeo (mgr.24625) 2868 : cluster [DBG] pgmap v1621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:32 gibba011 bash[31774]: cluster 2021-03-10T14:46:30.737615+0000 mgr.gibba032.whopeo (mgr.24625) 2868 : cluster [DBG] pgmap v1621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:34 gibba032 bash[24004]: cluster 2021-03-10T14:46:32.739440+0000 mgr.gibba032.whopeo (mgr.24625) 2869 : cluster [DBG] pgmap v1622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:34 gibba011 bash[30791]: cluster 2021-03-10T14:46:32.739440+0000 mgr.gibba032.whopeo (mgr.24625) 2869 : cluster [DBG] pgmap v1622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:34 gibba011 bash[31774]: cluster 2021-03-10T14:46:32.739440+0000 mgr.gibba032.whopeo (mgr.24625) 2869 : cluster [DBG] pgmap v1622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:35 gibba032 bash[24004]: cluster 2021-03-10T14:46:34.740766+0000 mgr.gibba032.whopeo (mgr.24625) 2870 : cluster [DBG] pgmap v1623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:35.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:35 gibba011 bash[30791]: cluster 2021-03-10T14:46:34.740766+0000 mgr.gibba032.whopeo (mgr.24625) 2870 : cluster [DBG] pgmap v1623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:35.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:35 gibba011 bash[31774]: cluster 2021-03-10T14:46:34.740766+0000 mgr.gibba032.whopeo (mgr.24625) 2870 : cluster [DBG] pgmap v1623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:38 gibba032 bash[24004]: cluster 2021-03-10T14:46:36.742369+0000 mgr.gibba032.whopeo (mgr.24625) 2871 : cluster [DBG] pgmap v1624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:38 gibba011 bash[30791]: cluster 2021-03-10T14:46:36.742369+0000 mgr.gibba032.whopeo (mgr.24625) 2871 : cluster [DBG] pgmap v1624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:38 gibba011 bash[31774]: cluster 2021-03-10T14:46:36.742369+0000 mgr.gibba032.whopeo (mgr.24625) 2871 : cluster [DBG] pgmap v1624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:39.225 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:46:39.448 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:46:39.448 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (53m) 8m ago 58m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:46:39.448 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (58m) 7m ago 58m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:46:39.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (53m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:46:39.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (53m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:46:39.449 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (52m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:46:39.449 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (52m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:46:39.449 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (52m) 7m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:46:39.450 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (52m) 8m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:46:39.450 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (58m) 8m ago 58m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:46:39.450 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (58m) 7m ago 58m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:46:39.450 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (51m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:46:39.450 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (51m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:46:39.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (50m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:46:39.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (50m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:46:39.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (49m) 7m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:46:39.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:46:39.451 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (49m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:46:39.452 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (48m) 7m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:46:39.452 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (58m) 7m ago 58m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:46:39.452 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 56m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:46:39.700 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:46:39.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:46:39.701 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:46:39.701 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:39.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:46:39.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:46:39.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:39.702 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:46:39.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:46:39.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:46:39.703 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:46:39.703 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:46:39.703 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:46:39.703 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:46:39.703 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:46:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:40 gibba032 bash[24004]: cluster 2021-03-10T14:46:38.743997+0000 mgr.gibba032.whopeo (mgr.24625) 2872 : cluster [DBG] pgmap v1625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:40 gibba032 bash[24004]: audit 2021-03-10T14:46:39.212997+0000 mgr.gibba032.whopeo (mgr.24625) 2873 : audit [DBG] from='client.55966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:40 gibba032 bash[24004]: audit 2021-03-10T14:46:39.435682+0000 mgr.gibba032.whopeo (mgr.24625) 2874 : audit [DBG] from='client.55972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:40 gibba032 bash[24004]: audit 2021-03-10T14:46:39.699211+0000 mon.a (mon.0) 1374 : audit [DBG] from='client.? 172.21.2.111:0/3892148010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[30791]: cluster 2021-03-10T14:46:38.743997+0000 mgr.gibba032.whopeo (mgr.24625) 2872 : cluster [DBG] pgmap v1625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[30791]: audit 2021-03-10T14:46:39.212997+0000 mgr.gibba032.whopeo (mgr.24625) 2873 : audit [DBG] from='client.55966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[30791]: audit 2021-03-10T14:46:39.435682+0000 mgr.gibba032.whopeo (mgr.24625) 2874 : audit [DBG] from='client.55972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[30791]: audit 2021-03-10T14:46:39.699211+0000 mon.a (mon.0) 1374 : audit [DBG] from='client.? 172.21.2.111:0/3892148010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[31774]: cluster 2021-03-10T14:46:38.743997+0000 mgr.gibba032.whopeo (mgr.24625) 2872 : cluster [DBG] pgmap v1625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[31774]: audit 2021-03-10T14:46:39.212997+0000 mgr.gibba032.whopeo (mgr.24625) 2873 : audit [DBG] from='client.55966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[31774]: audit 2021-03-10T14:46:39.435682+0000 mgr.gibba032.whopeo (mgr.24625) 2874 : audit [DBG] from='client.55972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:46:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:40 gibba011 bash[31774]: audit 2021-03-10T14:46:39.699211+0000 mon.a (mon.0) 1374 : audit [DBG] from='client.? 172.21.2.111:0/3892148010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:46:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:42 gibba032 bash[24004]: cluster 2021-03-10T14:46:40.745326+0000 mgr.gibba032.whopeo (mgr.24625) 2875 : cluster [DBG] pgmap v1626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:42 gibba011 bash[30791]: cluster 2021-03-10T14:46:40.745326+0000 mgr.gibba032.whopeo (mgr.24625) 2875 : cluster [DBG] pgmap v1626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:42 gibba011 bash[31774]: cluster 2021-03-10T14:46:40.745326+0000 mgr.gibba032.whopeo (mgr.24625) 2875 : cluster [DBG] pgmap v1626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:44 gibba032 bash[24004]: cluster 2021-03-10T14:46:42.747063+0000 mgr.gibba032.whopeo (mgr.24625) 2876 : cluster [DBG] pgmap v1627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:44 gibba011 bash[30791]: cluster 2021-03-10T14:46:42.747063+0000 mgr.gibba032.whopeo (mgr.24625) 2876 : cluster [DBG] pgmap v1627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:44 gibba011 bash[31774]: cluster 2021-03-10T14:46:42.747063+0000 mgr.gibba032.whopeo (mgr.24625) 2876 : cluster [DBG] pgmap v1627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:45.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:45 gibba032 bash[24004]: cephadm 2021-03-10T14:46:44.296194+0000 mgr.gibba032.whopeo (mgr.24625) 2877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:45 gibba011 bash[30791]: cephadm 2021-03-10T14:46:44.296194+0000 mgr.gibba032.whopeo (mgr.24625) 2877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:45.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:45 gibba011 bash[31774]: cephadm 2021-03-10T14:46:44.296194+0000 mgr.gibba032.whopeo (mgr.24625) 2877 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:46:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:46 gibba032 bash[24004]: cluster 2021-03-10T14:46:44.748615+0000 mgr.gibba032.whopeo (mgr.24625) 2878 : cluster [DBG] pgmap v1628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:46 gibba011 bash[30791]: cluster 2021-03-10T14:46:44.748615+0000 mgr.gibba032.whopeo (mgr.24625) 2878 : cluster [DBG] pgmap v1628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:46 gibba011 bash[31774]: cluster 2021-03-10T14:46:44.748615+0000 mgr.gibba032.whopeo (mgr.24625) 2878 : cluster [DBG] pgmap v1628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:47.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:47 gibba011 bash[30791]: cluster 2021-03-10T14:46:46.750437+0000 mgr.gibba032.whopeo (mgr.24625) 2879 : cluster [DBG] pgmap v1629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:47.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:47 gibba011 bash[31774]: cluster 2021-03-10T14:46:46.750437+0000 mgr.gibba032.whopeo (mgr.24625) 2879 : cluster [DBG] pgmap v1629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:47 gibba032 bash[24004]: cluster 2021-03-10T14:46:46.750437+0000 mgr.gibba032.whopeo (mgr.24625) 2879 : cluster [DBG] pgmap v1629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:49 gibba011 bash[30791]: cluster 2021-03-10T14:46:48.752106+0000 mgr.gibba032.whopeo (mgr.24625) 2880 : cluster [DBG] pgmap v1630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:49 gibba011 bash[31774]: cluster 2021-03-10T14:46:48.752106+0000 mgr.gibba032.whopeo (mgr.24625) 2880 : cluster [DBG] pgmap v1630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:49 gibba032 bash[24004]: cluster 2021-03-10T14:46:48.752106+0000 mgr.gibba032.whopeo (mgr.24625) 2880 : cluster [DBG] pgmap v1630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:52 gibba032 bash[24004]: cluster 2021-03-10T14:46:50.753464+0000 mgr.gibba032.whopeo (mgr.24625) 2881 : cluster [DBG] pgmap v1631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:52 gibba011 bash[30791]: cluster 2021-03-10T14:46:50.753464+0000 mgr.gibba032.whopeo (mgr.24625) 2881 : cluster [DBG] pgmap v1631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:52 gibba011 bash[31774]: cluster 2021-03-10T14:46:50.753464+0000 mgr.gibba032.whopeo (mgr.24625) 2881 : cluster [DBG] pgmap v1631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:54 gibba032 bash[24004]: cluster 2021-03-10T14:46:52.755226+0000 mgr.gibba032.whopeo (mgr.24625) 2882 : cluster [DBG] pgmap v1632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:54 gibba011 bash[30791]: cluster 2021-03-10T14:46:52.755226+0000 mgr.gibba032.whopeo (mgr.24625) 2882 : cluster [DBG] pgmap v1632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:54 gibba011 bash[31774]: cluster 2021-03-10T14:46:52.755226+0000 mgr.gibba032.whopeo (mgr.24625) 2882 : cluster [DBG] pgmap v1632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: cluster 2021-03-10T14:46:54.756743+0000 mgr.gibba032.whopeo (mgr.24625) 2883 : cluster [DBG] pgmap v1633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.352588+0000 mon.a (mon.0) 1375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:46:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.353201+0000 mon.a (mon.0) 1376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:46:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.380631+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:46:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.381373+0000 mon.a (mon.0) 1378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:46:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.382058+0000 mon.a (mon.0) 1379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:46:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:56 gibba032 bash[24004]: audit 2021-03-10T14:46:56.382729+0000 mon.a (mon.0) 1380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:46:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: cluster 2021-03-10T14:46:54.756743+0000 mgr.gibba032.whopeo (mgr.24625) 2883 : cluster [DBG] pgmap v1633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.352588+0000 mon.a (mon.0) 1375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:46:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.353201+0000 mon.a (mon.0) 1376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:46:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.380631+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:46:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.381373+0000 mon.a (mon.0) 1378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:46:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.382058+0000 mon.a (mon.0) 1379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:46:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[30791]: audit 2021-03-10T14:46:56.382729+0000 mon.a (mon.0) 1380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:46:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: cluster 2021-03-10T14:46:54.756743+0000 mgr.gibba032.whopeo (mgr.24625) 2883 : cluster [DBG] pgmap v1633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.352588+0000 mon.a (mon.0) 1375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:46:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.353201+0000 mon.a (mon.0) 1376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:46:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.380631+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T14:46:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.381373+0000 mon.a (mon.0) 1378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T14:46:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.382058+0000 mon.a (mon.0) 1379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:46:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:56 gibba011 bash[31774]: audit 2021-03-10T14:46:56.382729+0000 mon.a (mon.0) 1380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:46:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:58 gibba032 bash[24004]: cluster 2021-03-10T14:46:56.758214+0000 mgr.gibba032.whopeo (mgr.24625) 2884 : cluster [DBG] pgmap v1634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:58 gibba011 bash[30791]: cluster 2021-03-10T14:46:56.758214+0000 mgr.gibba032.whopeo (mgr.24625) 2884 : cluster [DBG] pgmap v1634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:58 gibba011 bash[31774]: cluster 2021-03-10T14:46:56.758214+0000 mgr.gibba032.whopeo (mgr.24625) 2884 : cluster [DBG] pgmap v1634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:46:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:46:59 gibba032 bash[24004]: audit 2021-03-10T14:46:59.316441+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:46:59.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:46:59 gibba011 bash[30791]: audit 2021-03-10T14:46:59.316441+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:46:59.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:46:59 gibba011 bash[31774]: audit 2021-03-10T14:46:59.316441+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:47:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:00 gibba032 bash[24004]: cluster 2021-03-10T14:46:58.759637+0000 mgr.gibba032.whopeo (mgr.24625) 2885 : cluster [DBG] pgmap v1635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:00 gibba032 bash[24004]: cephadm 2021-03-10T14:46:59.312858+0000 mgr.gibba032.whopeo (mgr.24625) 2886 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:47:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:00 gibba011 bash[30791]: cluster 2021-03-10T14:46:58.759637+0000 mgr.gibba032.whopeo (mgr.24625) 2885 : cluster [DBG] pgmap v1635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:00 gibba011 bash[30791]: cephadm 2021-03-10T14:46:59.312858+0000 mgr.gibba032.whopeo (mgr.24625) 2886 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:47:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:00 gibba011 bash[31774]: cluster 2021-03-10T14:46:58.759637+0000 mgr.gibba032.whopeo (mgr.24625) 2885 : cluster [DBG] pgmap v1635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:00 gibba011 bash[31774]: cephadm 2021-03-10T14:46:59.312858+0000 mgr.gibba032.whopeo (mgr.24625) 2886 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:47:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:02 gibba032 bash[24004]: cluster 2021-03-10T14:47:00.761028+0000 mgr.gibba032.whopeo (mgr.24625) 2887 : cluster [DBG] pgmap v1636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:02 gibba011 bash[30791]: cluster 2021-03-10T14:47:00.761028+0000 mgr.gibba032.whopeo (mgr.24625) 2887 : cluster [DBG] pgmap v1636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:02 gibba011 bash[31774]: cluster 2021-03-10T14:47:00.761028+0000 mgr.gibba032.whopeo (mgr.24625) 2887 : cluster [DBG] pgmap v1636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:03 gibba011 bash[30791]: cluster 2021-03-10T14:47:02.762554+0000 mgr.gibba032.whopeo (mgr.24625) 2888 : cluster [DBG] pgmap v1637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:03.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:03 gibba011 bash[31774]: cluster 2021-03-10T14:47:02.762554+0000 mgr.gibba032.whopeo (mgr.24625) 2888 : cluster [DBG] pgmap v1637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:03 gibba032 bash[24004]: cluster 2021-03-10T14:47:02.762554+0000 mgr.gibba032.whopeo (mgr.24625) 2888 : cluster [DBG] pgmap v1637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:04.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:03.932665+0000 mon.a (mon.0) 1382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:47:04.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.106303+0000 mgr.gibba032.whopeo (mgr.24625) 2889 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:47:04.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.107642+0000 mon.a (mon.0) 1383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.300752+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.305900+0000 mgr.gibba032.whopeo (mgr.24625) 2890 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:47:04.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.308150+0000 mgr.gibba032.whopeo (mgr.24625) 2891 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:47:04.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.309685+0000 mon.a (mon.0) 1385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:47:04.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.310117+0000 mgr.gibba032.whopeo (mgr.24625) 2892 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:47:04.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.310159+0000 mgr.gibba032.whopeo (mgr.24625) 2893 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:47:04.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.311347+0000 mon.a (mon.0) 1386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.311826+0000 mgr.gibba032.whopeo (mgr.24625) 2894 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:47:04.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.311866+0000 mgr.gibba032.whopeo (mgr.24625) 2895 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:47:04.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.313097+0000 mgr.gibba032.whopeo (mgr.24625) 2896 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:47:04.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.313597+0000 mon.a (mon.0) 1387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.314364+0000 mgr.gibba032.whopeo (mgr.24625) 2897 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:47:04.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.314406+0000 mgr.gibba032.whopeo (mgr.24625) 2898 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:47:04.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.315039+0000 mon.a (mon.0) 1388 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.315998+0000 mgr.gibba032.whopeo (mgr.24625) 2899 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:47:04.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.316445+0000 mgr.gibba032.whopeo (mgr.24625) 2900 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:47:04.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.317789+0000 mgr.gibba032.whopeo (mgr.24625) 2901 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:47:04.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.317798+0000 mon.a (mon.0) 1389 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.317832+0000 mgr.gibba032.whopeo (mgr.24625) 2902 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:47:04.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: cephadm 2021-03-10T14:47:04.317962+0000 mgr.gibba032.whopeo (mgr.24625) 2903 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:04.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:04 gibba032 bash[24004]: audit 2021-03-10T14:47:04.319472+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:03.932665+0000 mon.a (mon.0) 1382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:47:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.106303+0000 mgr.gibba032.whopeo (mgr.24625) 2889 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:47:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.107642+0000 mon.a (mon.0) 1383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.300752+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.305900+0000 mgr.gibba032.whopeo (mgr.24625) 2890 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:47:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.308150+0000 mgr.gibba032.whopeo (mgr.24625) 2891 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:47:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.309685+0000 mon.a (mon.0) 1385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:47:04.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.310117+0000 mgr.gibba032.whopeo (mgr.24625) 2892 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:47:04.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.310159+0000 mgr.gibba032.whopeo (mgr.24625) 2893 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:47:04.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.311347+0000 mon.a (mon.0) 1386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.311826+0000 mgr.gibba032.whopeo (mgr.24625) 2894 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:47:04.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.311866+0000 mgr.gibba032.whopeo (mgr.24625) 2895 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:47:04.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.313097+0000 mgr.gibba032.whopeo (mgr.24625) 2896 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:47:04.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.313597+0000 mon.a (mon.0) 1387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.314364+0000 mgr.gibba032.whopeo (mgr.24625) 2897 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:47:04.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.314406+0000 mgr.gibba032.whopeo (mgr.24625) 2898 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:47:04.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.315039+0000 mon.a (mon.0) 1388 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.315998+0000 mgr.gibba032.whopeo (mgr.24625) 2899 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:47:04.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.316445+0000 mgr.gibba032.whopeo (mgr.24625) 2900 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:47:04.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.317789+0000 mgr.gibba032.whopeo (mgr.24625) 2901 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:47:04.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:03.932665+0000 mon.a (mon.0) 1382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:47:04.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.106303+0000 mgr.gibba032.whopeo (mgr.24625) 2889 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:47:04.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.107642+0000 mon.a (mon.0) 1383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.300752+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:47:04.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.305900+0000 mgr.gibba032.whopeo (mgr.24625) 2890 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:47:04.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.308150+0000 mgr.gibba032.whopeo (mgr.24625) 2891 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:47:04.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.309685+0000 mon.a (mon.0) 1385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:47:04.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.310117+0000 mgr.gibba032.whopeo (mgr.24625) 2892 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:47:04.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.310159+0000 mgr.gibba032.whopeo (mgr.24625) 2893 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:47:04.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.311347+0000 mon.a (mon.0) 1386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.311826+0000 mgr.gibba032.whopeo (mgr.24625) 2894 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:47:04.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.311866+0000 mgr.gibba032.whopeo (mgr.24625) 2895 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:47:04.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.313097+0000 mgr.gibba032.whopeo (mgr.24625) 2896 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:47:04.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.313597+0000 mon.a (mon.0) 1387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.314364+0000 mgr.gibba032.whopeo (mgr.24625) 2897 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:47:04.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.314406+0000 mgr.gibba032.whopeo (mgr.24625) 2898 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:47:04.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.315039+0000 mon.a (mon.0) 1388 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.315998+0000 mgr.gibba032.whopeo (mgr.24625) 2899 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:47:04.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.316445+0000 mgr.gibba032.whopeo (mgr.24625) 2900 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:47:04.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.317789+0000 mgr.gibba032.whopeo (mgr.24625) 2901 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:47:04.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.317798+0000 mon.a (mon.0) 1389 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.317832+0000 mgr.gibba032.whopeo (mgr.24625) 2902 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:47:04.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: cephadm 2021-03-10T14:47:04.317962+0000 mgr.gibba032.whopeo (mgr.24625) 2903 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:04.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[30791]: audit 2021-03-10T14:47:04.319472+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.317798+0000 mon.a (mon.0) 1389 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:04.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.317832+0000 mgr.gibba032.whopeo (mgr.24625) 2902 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:47:04.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: cephadm 2021-03-10T14:47:04.317962+0000 mgr.gibba032.whopeo (mgr.24625) 2903 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:04.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:04 gibba011 bash[31774]: audit 2021-03-10T14:47:04.319472+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:05 gibba011 bash[30791]: cluster 2021-03-10T14:47:04.763977+0000 mgr.gibba032.whopeo (mgr.24625) 2904 : cluster [DBG] pgmap v1638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:05 gibba011 bash[31774]: cluster 2021-03-10T14:47:04.763977+0000 mgr.gibba032.whopeo (mgr.24625) 2904 : cluster [DBG] pgmap v1638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:05 gibba032 bash[24004]: cluster 2021-03-10T14:47:04.763977+0000 mgr.gibba032.whopeo (mgr.24625) 2904 : cluster [DBG] pgmap v1638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:08 gibba032 bash[24004]: cluster 2021-03-10T14:47:06.765545+0000 mgr.gibba032.whopeo (mgr.24625) 2905 : cluster [DBG] pgmap v1639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:08 gibba011 bash[30791]: cluster 2021-03-10T14:47:06.765545+0000 mgr.gibba032.whopeo (mgr.24625) 2905 : cluster [DBG] pgmap v1639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:08 gibba011 bash[31774]: cluster 2021-03-10T14:47:06.765545+0000 mgr.gibba032.whopeo (mgr.24625) 2905 : cluster [DBG] pgmap v1639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:09.943 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:47:10.171 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:47:10.172 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (53m) 9m ago 59m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:47:10.172 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (59m) 8m ago 59m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:47:10.172 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (54m) 8m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:47:10.172 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (54m) 8m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:47:10.172 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (53m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:47:10.173 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (53m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:47:10.173 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (52m) 8m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:47:10.173 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (53m) 9m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:47:10.173 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (59m) 9m ago 59m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:47:10.173 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (59m) 8m ago 59m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:47:10.174 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (52m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:47:10.174 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (51m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:47:10.174 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (51m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:47:10.174 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (50m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:47:10.174 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (50m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:47:10.175 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (49m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:47:10.175 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (49m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:47:10.175 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (49m) 8m ago 59m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:47:10.175 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (58m) 8m ago 59m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:47:10.175 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 57m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:47:10.418 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:47:10.418 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:47:10.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:47:10.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:10.419 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:47:10.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:47:10.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:10.420 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:47:10.420 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:47:10.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:10.420 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:47:10.421 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:47:10.421 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:47:10.421 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:47:10.421 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:47:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:10 gibba032 bash[24004]: cluster 2021-03-10T14:47:08.766934+0000 mgr.gibba032.whopeo (mgr.24625) 2906 : cluster [DBG] pgmap v1640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:10 gibba032 bash[24004]: audit 2021-03-10T14:47:10.416668+0000 mon.a (mon.0) 1391 : audit [DBG] from='client.? 172.21.2.111:0/1495860779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:10 gibba011 bash[30791]: cluster 2021-03-10T14:47:08.766934+0000 mgr.gibba032.whopeo (mgr.24625) 2906 : cluster [DBG] pgmap v1640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:10 gibba011 bash[30791]: audit 2021-03-10T14:47:10.416668+0000 mon.a (mon.0) 1391 : audit [DBG] from='client.? 172.21.2.111:0/1495860779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:10 gibba011 bash[31774]: cluster 2021-03-10T14:47:08.766934+0000 mgr.gibba032.whopeo (mgr.24625) 2906 : cluster [DBG] pgmap v1640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:10 gibba011 bash[31774]: audit 2021-03-10T14:47:10.416668+0000 mon.a (mon.0) 1391 : audit [DBG] from='client.? 172.21.2.111:0/1495860779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:11 gibba032 bash[24004]: audit 2021-03-10T14:47:09.929910+0000 mgr.gibba032.whopeo (mgr.24625) 2907 : audit [DBG] from='client.45999 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:11 gibba032 bash[24004]: audit 2021-03-10T14:47:10.155462+0000 mgr.gibba032.whopeo (mgr.24625) 2908 : audit [DBG] from='client.46005 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:11 gibba011 bash[30791]: audit 2021-03-10T14:47:09.929910+0000 mgr.gibba032.whopeo (mgr.24625) 2907 : audit [DBG] from='client.45999 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:11 gibba011 bash[30791]: audit 2021-03-10T14:47:10.155462+0000 mgr.gibba032.whopeo (mgr.24625) 2908 : audit [DBG] from='client.46005 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:11.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:11 gibba011 bash[31774]: audit 2021-03-10T14:47:09.929910+0000 mgr.gibba032.whopeo (mgr.24625) 2907 : audit [DBG] from='client.45999 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:11.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:11 gibba011 bash[31774]: audit 2021-03-10T14:47:10.155462+0000 mgr.gibba032.whopeo (mgr.24625) 2908 : audit [DBG] from='client.46005 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:12 gibba032 bash[24004]: cluster 2021-03-10T14:47:10.768362+0000 mgr.gibba032.whopeo (mgr.24625) 2909 : cluster [DBG] pgmap v1641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:12 gibba011 bash[30791]: cluster 2021-03-10T14:47:10.768362+0000 mgr.gibba032.whopeo (mgr.24625) 2909 : cluster [DBG] pgmap v1641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:12 gibba011 bash[31774]: cluster 2021-03-10T14:47:10.768362+0000 mgr.gibba032.whopeo (mgr.24625) 2909 : cluster [DBG] pgmap v1641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:14 gibba032 bash[24004]: cluster 2021-03-10T14:47:12.769918+0000 mgr.gibba032.whopeo (mgr.24625) 2910 : cluster [DBG] pgmap v1642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:14 gibba011 bash[30791]: cluster 2021-03-10T14:47:12.769918+0000 mgr.gibba032.whopeo (mgr.24625) 2910 : cluster [DBG] pgmap v1642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:14 gibba011 bash[31774]: cluster 2021-03-10T14:47:12.769918+0000 mgr.gibba032.whopeo (mgr.24625) 2910 : cluster [DBG] pgmap v1642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:16 gibba032 bash[24004]: cluster 2021-03-10T14:47:14.771323+0000 mgr.gibba032.whopeo (mgr.24625) 2911 : cluster [DBG] pgmap v1643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:16 gibba011 bash[30791]: cluster 2021-03-10T14:47:14.771323+0000 mgr.gibba032.whopeo (mgr.24625) 2911 : cluster [DBG] pgmap v1643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:16 gibba011 bash[31774]: cluster 2021-03-10T14:47:14.771323+0000 mgr.gibba032.whopeo (mgr.24625) 2911 : cluster [DBG] pgmap v1643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:17 gibba011 bash[30791]: cluster 2021-03-10T14:47:16.773056+0000 mgr.gibba032.whopeo (mgr.24625) 2912 : cluster [DBG] pgmap v1644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:17.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:17 gibba011 bash[31774]: cluster 2021-03-10T14:47:16.773056+0000 mgr.gibba032.whopeo (mgr.24625) 2912 : cluster [DBG] pgmap v1644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:17 gibba032 bash[24004]: cluster 2021-03-10T14:47:16.773056+0000 mgr.gibba032.whopeo (mgr.24625) 2912 : cluster [DBG] pgmap v1644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:19 gibba032 bash[24004]: cluster 2021-03-10T14:47:18.774442+0000 mgr.gibba032.whopeo (mgr.24625) 2913 : cluster [DBG] pgmap v1645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:19 gibba032 bash[24004]: cephadm 2021-03-10T14:47:19.320953+0000 mgr.gibba032.whopeo (mgr.24625) 2914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:19 gibba011 bash[30791]: cluster 2021-03-10T14:47:18.774442+0000 mgr.gibba032.whopeo (mgr.24625) 2913 : cluster [DBG] pgmap v1645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:19 gibba011 bash[30791]: cephadm 2021-03-10T14:47:19.320953+0000 mgr.gibba032.whopeo (mgr.24625) 2914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:19 gibba011 bash[31774]: cluster 2021-03-10T14:47:18.774442+0000 mgr.gibba032.whopeo (mgr.24625) 2913 : cluster [DBG] pgmap v1645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:19 gibba011 bash[31774]: cephadm 2021-03-10T14:47:19.320953+0000 mgr.gibba032.whopeo (mgr.24625) 2914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:22 gibba032 bash[24004]: cluster 2021-03-10T14:47:20.775794+0000 mgr.gibba032.whopeo (mgr.24625) 2915 : cluster [DBG] pgmap v1646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:22 gibba011 bash[30791]: cluster 2021-03-10T14:47:20.775794+0000 mgr.gibba032.whopeo (mgr.24625) 2915 : cluster [DBG] pgmap v1646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:22 gibba011 bash[31774]: cluster 2021-03-10T14:47:20.775794+0000 mgr.gibba032.whopeo (mgr.24625) 2915 : cluster [DBG] pgmap v1646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:24 gibba032 bash[24004]: cluster 2021-03-10T14:47:22.777478+0000 mgr.gibba032.whopeo (mgr.24625) 2916 : cluster [DBG] pgmap v1647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:24 gibba011 bash[30791]: cluster 2021-03-10T14:47:22.777478+0000 mgr.gibba032.whopeo (mgr.24625) 2916 : cluster [DBG] pgmap v1647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:24 gibba011 bash[31774]: cluster 2021-03-10T14:47:22.777478+0000 mgr.gibba032.whopeo (mgr.24625) 2916 : cluster [DBG] pgmap v1647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:26 gibba032 bash[24004]: cluster 2021-03-10T14:47:24.778883+0000 mgr.gibba032.whopeo (mgr.24625) 2917 : cluster [DBG] pgmap v1648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:26 gibba011 bash[30791]: cluster 2021-03-10T14:47:24.778883+0000 mgr.gibba032.whopeo (mgr.24625) 2917 : cluster [DBG] pgmap v1648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:26 gibba011 bash[31774]: cluster 2021-03-10T14:47:24.778883+0000 mgr.gibba032.whopeo (mgr.24625) 2917 : cluster [DBG] pgmap v1648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:28 gibba032 bash[24004]: cluster 2021-03-10T14:47:26.780624+0000 mgr.gibba032.whopeo (mgr.24625) 2918 : cluster [DBG] pgmap v1649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:28 gibba011 bash[30791]: cluster 2021-03-10T14:47:26.780624+0000 mgr.gibba032.whopeo (mgr.24625) 2918 : cluster [DBG] pgmap v1649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:28 gibba011 bash[31774]: cluster 2021-03-10T14:47:26.780624+0000 mgr.gibba032.whopeo (mgr.24625) 2918 : cluster [DBG] pgmap v1649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:30 gibba032 bash[24004]: cluster 2021-03-10T14:47:28.782103+0000 mgr.gibba032.whopeo (mgr.24625) 2919 : cluster [DBG] pgmap v1650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:30 gibba011 bash[30791]: cluster 2021-03-10T14:47:28.782103+0000 mgr.gibba032.whopeo (mgr.24625) 2919 : cluster [DBG] pgmap v1650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:30 gibba011 bash[31774]: cluster 2021-03-10T14:47:28.782103+0000 mgr.gibba032.whopeo (mgr.24625) 2919 : cluster [DBG] pgmap v1650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:32 gibba032 bash[24004]: cluster 2021-03-10T14:47:30.783561+0000 mgr.gibba032.whopeo (mgr.24625) 2920 : cluster [DBG] pgmap v1651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:32 gibba011 bash[30791]: cluster 2021-03-10T14:47:30.783561+0000 mgr.gibba032.whopeo (mgr.24625) 2920 : cluster [DBG] pgmap v1651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:32 gibba011 bash[31774]: cluster 2021-03-10T14:47:30.783561+0000 mgr.gibba032.whopeo (mgr.24625) 2920 : cluster [DBG] pgmap v1651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:33.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:33 gibba032 bash[24004]: cluster 2021-03-10T14:47:32.785278+0000 mgr.gibba032.whopeo (mgr.24625) 2921 : cluster [DBG] pgmap v1652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:33 gibba011 bash[30791]: cluster 2021-03-10T14:47:32.785278+0000 mgr.gibba032.whopeo (mgr.24625) 2921 : cluster [DBG] pgmap v1652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:33 gibba011 bash[31774]: cluster 2021-03-10T14:47:32.785278+0000 mgr.gibba032.whopeo (mgr.24625) 2921 : cluster [DBG] pgmap v1652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:34 gibba032 bash[24004]: cephadm 2021-03-10T14:47:34.336768+0000 mgr.gibba032.whopeo (mgr.24625) 2922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:34 gibba011 bash[30791]: cephadm 2021-03-10T14:47:34.336768+0000 mgr.gibba032.whopeo (mgr.24625) 2922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:34 gibba011 bash[31774]: cephadm 2021-03-10T14:47:34.336768+0000 mgr.gibba032.whopeo (mgr.24625) 2922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:35 gibba011 bash[30791]: cluster 2021-03-10T14:47:34.786677+0000 mgr.gibba032.whopeo (mgr.24625) 2923 : cluster [DBG] pgmap v1653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:35.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:35 gibba011 bash[31774]: cluster 2021-03-10T14:47:34.786677+0000 mgr.gibba032.whopeo (mgr.24625) 2923 : cluster [DBG] pgmap v1653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:36.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:35 gibba032 bash[24004]: cluster 2021-03-10T14:47:34.786677+0000 mgr.gibba032.whopeo (mgr.24625) 2923 : cluster [DBG] pgmap v1653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:38 gibba032 bash[24004]: cluster 2021-03-10T14:47:36.788513+0000 mgr.gibba032.whopeo (mgr.24625) 2924 : cluster [DBG] pgmap v1654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:38 gibba011 bash[30791]: cluster 2021-03-10T14:47:36.788513+0000 mgr.gibba032.whopeo (mgr.24625) 2924 : cluster [DBG] pgmap v1654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:38 gibba011 bash[31774]: cluster 2021-03-10T14:47:36.788513+0000 mgr.gibba032.whopeo (mgr.24625) 2924 : cluster [DBG] pgmap v1654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:40.665 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:47:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:40 gibba032 bash[24004]: cluster 2021-03-10T14:47:38.790016+0000 mgr.gibba032.whopeo (mgr.24625) 2925 : cluster [DBG] pgmap v1655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:40 gibba011 bash[30791]: cluster 2021-03-10T14:47:38.790016+0000 mgr.gibba032.whopeo (mgr.24625) 2925 : cluster [DBG] pgmap v1655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:40 gibba011 bash[31774]: cluster 2021-03-10T14:47:38.790016+0000 mgr.gibba032.whopeo (mgr.24625) 2925 : cluster [DBG] pgmap v1655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:40.890 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:47:40.890 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (54m) 9m ago 59m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:47:40.891 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (59m) 8m ago 59m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:47:40.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (54m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:47:40.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (54m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:47:40.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (54m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:47:40.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (53m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:47:40.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (53m) 8m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:47:40.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (53m) 9m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:47:40.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (59m) 9m ago 59m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:47:40.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (59m) 8m ago 59m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:47:40.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (52m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:47:40.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (52m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:47:40.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (51m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:47:40.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (51m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:47:40.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (50m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:47:40.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (50m) 8m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:47:40.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (50m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:47:40.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (49m) 8m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:47:40.894 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (59m) 8m ago 59m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:47:40.894 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 57m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:47:41.141 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:47:41.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:47:41.141 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:47:41.141 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:41.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:47:41.142 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:47:41.142 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:41.142 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:47:41.142 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:47:41.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:47:41.143 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:47:41.143 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:47:41.143 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:47:41.143 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:47:41.144 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:47:41.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:41 gibba032 bash[24004]: audit 2021-03-10T14:47:41.139093+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? 172.21.2.111:0/2305075021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:41.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:41 gibba011 bash[30791]: audit 2021-03-10T14:47:41.139093+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? 172.21.2.111:0/2305075021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:41.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:41 gibba011 bash[31774]: audit 2021-03-10T14:47:41.139093+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? 172.21.2.111:0/2305075021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:47:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:42 gibba032 bash[24004]: audit 2021-03-10T14:47:40.651431+0000 mgr.gibba032.whopeo (mgr.24625) 2926 : audit [DBG] from='client.56002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:42.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:42 gibba032 bash[24004]: cluster 2021-03-10T14:47:40.791391+0000 mgr.gibba032.whopeo (mgr.24625) 2927 : cluster [DBG] pgmap v1656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:42.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:42 gibba032 bash[24004]: audit 2021-03-10T14:47:40.874958+0000 mgr.gibba032.whopeo (mgr.24625) 2928 : audit [DBG] from='client.46023 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[30791]: audit 2021-03-10T14:47:40.651431+0000 mgr.gibba032.whopeo (mgr.24625) 2926 : audit [DBG] from='client.56002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[30791]: cluster 2021-03-10T14:47:40.791391+0000 mgr.gibba032.whopeo (mgr.24625) 2927 : cluster [DBG] pgmap v1656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[30791]: audit 2021-03-10T14:47:40.874958+0000 mgr.gibba032.whopeo (mgr.24625) 2928 : audit [DBG] from='client.46023 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[31774]: audit 2021-03-10T14:47:40.651431+0000 mgr.gibba032.whopeo (mgr.24625) 2926 : audit [DBG] from='client.56002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[31774]: cluster 2021-03-10T14:47:40.791391+0000 mgr.gibba032.whopeo (mgr.24625) 2927 : cluster [DBG] pgmap v1656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:42 gibba011 bash[31774]: audit 2021-03-10T14:47:40.874958+0000 mgr.gibba032.whopeo (mgr.24625) 2928 : audit [DBG] from='client.46023 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:47:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:44 gibba032 bash[24004]: cluster 2021-03-10T14:47:42.793088+0000 mgr.gibba032.whopeo (mgr.24625) 2929 : cluster [DBG] pgmap v1657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:44 gibba011 bash[30791]: cluster 2021-03-10T14:47:42.793088+0000 mgr.gibba032.whopeo (mgr.24625) 2929 : cluster [DBG] pgmap v1657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:44 gibba011 bash[31774]: cluster 2021-03-10T14:47:42.793088+0000 mgr.gibba032.whopeo (mgr.24625) 2929 : cluster [DBG] pgmap v1657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:45 gibba011 bash[30791]: cluster 2021-03-10T14:47:44.794560+0000 mgr.gibba032.whopeo (mgr.24625) 2930 : cluster [DBG] pgmap v1658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:45.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:45 gibba011 bash[31774]: cluster 2021-03-10T14:47:44.794560+0000 mgr.gibba032.whopeo (mgr.24625) 2930 : cluster [DBG] pgmap v1658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:45 gibba032 bash[24004]: cluster 2021-03-10T14:47:44.794560+0000 mgr.gibba032.whopeo (mgr.24625) 2930 : cluster [DBG] pgmap v1658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:48 gibba032 bash[24004]: cluster 2021-03-10T14:47:46.796249+0000 mgr.gibba032.whopeo (mgr.24625) 2931 : cluster [DBG] pgmap v1659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:48 gibba011 bash[30791]: cluster 2021-03-10T14:47:46.796249+0000 mgr.gibba032.whopeo (mgr.24625) 2931 : cluster [DBG] pgmap v1659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:48 gibba011 bash[31774]: cluster 2021-03-10T14:47:46.796249+0000 mgr.gibba032.whopeo (mgr.24625) 2931 : cluster [DBG] pgmap v1659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:50 gibba032 bash[24004]: cluster 2021-03-10T14:47:48.797644+0000 mgr.gibba032.whopeo (mgr.24625) 2932 : cluster [DBG] pgmap v1660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:50 gibba032 bash[24004]: cephadm 2021-03-10T14:47:49.352836+0000 mgr.gibba032.whopeo (mgr.24625) 2933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:50 gibba011 bash[30791]: cluster 2021-03-10T14:47:48.797644+0000 mgr.gibba032.whopeo (mgr.24625) 2932 : cluster [DBG] pgmap v1660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:50 gibba011 bash[30791]: cephadm 2021-03-10T14:47:49.352836+0000 mgr.gibba032.whopeo (mgr.24625) 2933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:50 gibba011 bash[31774]: cluster 2021-03-10T14:47:48.797644+0000 mgr.gibba032.whopeo (mgr.24625) 2932 : cluster [DBG] pgmap v1660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:50 gibba011 bash[31774]: cephadm 2021-03-10T14:47:49.352836+0000 mgr.gibba032.whopeo (mgr.24625) 2933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:47:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:52 gibba032 bash[24004]: cluster 2021-03-10T14:47:50.799128+0000 mgr.gibba032.whopeo (mgr.24625) 2934 : cluster [DBG] pgmap v1661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:52 gibba011 bash[30791]: cluster 2021-03-10T14:47:50.799128+0000 mgr.gibba032.whopeo (mgr.24625) 2934 : cluster [DBG] pgmap v1661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:52 gibba011 bash[31774]: cluster 2021-03-10T14:47:50.799128+0000 mgr.gibba032.whopeo (mgr.24625) 2934 : cluster [DBG] pgmap v1661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:54 gibba032 bash[24004]: cluster 2021-03-10T14:47:52.801026+0000 mgr.gibba032.whopeo (mgr.24625) 2935 : cluster [DBG] pgmap v1662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:54 gibba011 bash[30791]: cluster 2021-03-10T14:47:52.801026+0000 mgr.gibba032.whopeo (mgr.24625) 2935 : cluster [DBG] pgmap v1662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:54 gibba011 bash[31774]: cluster 2021-03-10T14:47:52.801026+0000 mgr.gibba032.whopeo (mgr.24625) 2935 : cluster [DBG] pgmap v1662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: cluster 2021-03-10T14:47:54.802366+0000 mgr.gibba032.whopeo (mgr.24625) 2936 : cluster [DBG] pgmap v1663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.360996+0000 mon.a (mon.0) 1393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:47:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.361952+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:47:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.386757+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:47:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.387901+0000 mon.a (mon.0) 1396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:47:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.388985+0000 mon.a (mon.0) 1397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:47:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:56 gibba032 bash[24004]: audit 2021-03-10T14:47:56.390160+0000 mon.a (mon.0) 1398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:47:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: cluster 2021-03-10T14:47:54.802366+0000 mgr.gibba032.whopeo (mgr.24625) 2936 : cluster [DBG] pgmap v1663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.360996+0000 mon.a (mon.0) 1393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:47:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.361952+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:47:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.386757+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:47:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.387901+0000 mon.a (mon.0) 1396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:47:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.388985+0000 mon.a (mon.0) 1397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:47:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[30791]: audit 2021-03-10T14:47:56.390160+0000 mon.a (mon.0) 1398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:47:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: cluster 2021-03-10T14:47:54.802366+0000 mgr.gibba032.whopeo (mgr.24625) 2936 : cluster [DBG] pgmap v1663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.360996+0000 mon.a (mon.0) 1393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:47:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.361952+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:47:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.386757+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:47:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.387901+0000 mon.a (mon.0) 1396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:47:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.388985+0000 mon.a (mon.0) 1397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:47:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:56 gibba011 bash[31774]: audit 2021-03-10T14:47:56.390160+0000 mon.a (mon.0) 1398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:47:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:57 gibba032 bash[24004]: cluster 2021-03-10T14:47:56.804292+0000 mgr.gibba032.whopeo (mgr.24625) 2937 : cluster [DBG] pgmap v1664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:57 gibba011 bash[30791]: cluster 2021-03-10T14:47:56.804292+0000 mgr.gibba032.whopeo (mgr.24625) 2937 : cluster [DBG] pgmap v1664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:47:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:57 gibba011 bash[31774]: cluster 2021-03-10T14:47:56.804292+0000 mgr.gibba032.whopeo (mgr.24625) 2937 : cluster [DBG] pgmap v1664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:47:59 gibba032 bash[24004]: cluster 2021-03-10T14:47:58.805961+0000 mgr.gibba032.whopeo (mgr.24625) 2938 : cluster [DBG] pgmap v1665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:47:59 gibba011 bash[30791]: cluster 2021-03-10T14:47:58.805961+0000 mgr.gibba032.whopeo (mgr.24625) 2938 : cluster [DBG] pgmap v1665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:47:59 gibba011 bash[31774]: cluster 2021-03-10T14:47:58.805961+0000 mgr.gibba032.whopeo (mgr.24625) 2938 : cluster [DBG] pgmap v1665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:02 gibba032 bash[24004]: cluster 2021-03-10T14:48:00.807358+0000 mgr.gibba032.whopeo (mgr.24625) 2939 : cluster [DBG] pgmap v1666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:02 gibba011 bash[30791]: cluster 2021-03-10T14:48:00.807358+0000 mgr.gibba032.whopeo (mgr.24625) 2939 : cluster [DBG] pgmap v1666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:02 gibba011 bash[31774]: cluster 2021-03-10T14:48:00.807358+0000 mgr.gibba032.whopeo (mgr.24625) 2939 : cluster [DBG] pgmap v1666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:04 gibba032 bash[24004]: cluster 2021-03-10T14:48:02.809064+0000 mgr.gibba032.whopeo (mgr.24625) 2940 : cluster [DBG] pgmap v1667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:04.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:04 gibba032 bash[24004]: audit 2021-03-10T14:48:04.374047+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:48:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:04 gibba011 bash[30791]: cluster 2021-03-10T14:48:02.809064+0000 mgr.gibba032.whopeo (mgr.24625) 2940 : cluster [DBG] pgmap v1667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:04 gibba011 bash[30791]: audit 2021-03-10T14:48:04.374047+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:48:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:04 gibba011 bash[31774]: cluster 2021-03-10T14:48:02.809064+0000 mgr.gibba032.whopeo (mgr.24625) 2940 : cluster [DBG] pgmap v1667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:04 gibba011 bash[31774]: audit 2021-03-10T14:48:04.374047+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:48:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:05 gibba032 bash[24004]: cephadm 2021-03-10T14:48:04.370451+0000 mgr.gibba032.whopeo (mgr.24625) 2941 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:48:05.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:05 gibba011 bash[30791]: cephadm 2021-03-10T14:48:04.370451+0000 mgr.gibba032.whopeo (mgr.24625) 2941 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:48:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:05 gibba011 bash[31774]: cephadm 2021-03-10T14:48:04.370451+0000 mgr.gibba032.whopeo (mgr.24625) 2941 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:48:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:06 gibba032 bash[24004]: cluster 2021-03-10T14:48:04.810525+0000 mgr.gibba032.whopeo (mgr.24625) 2942 : cluster [DBG] pgmap v1668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:06 gibba011 bash[30791]: cluster 2021-03-10T14:48:04.810525+0000 mgr.gibba032.whopeo (mgr.24625) 2942 : cluster [DBG] pgmap v1668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:06 gibba011 bash[31774]: cluster 2021-03-10T14:48:04.810525+0000 mgr.gibba032.whopeo (mgr.24625) 2942 : cluster [DBG] pgmap v1668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: cluster 2021-03-10T14:48:06.812318+0000 mgr.gibba032.whopeo (mgr.24625) 2943 : cluster [DBG] pgmap v1669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:07.907039+0000 mon.a (mon.0) 1400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.107057+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.114443+0000 mon.a (mon.0) 1402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:48:08.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.116214+0000 mon.a (mon.0) 1403 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.117544+0000 mon.a (mon.0) 1404 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.118681+0000 mon.a (mon.0) 1405 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.121445+0000 mon.a (mon.0) 1406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:08 gibba032 bash[24004]: audit 2021-03-10T14:48:08.122844+0000 mon.a (mon.0) 1407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: cluster 2021-03-10T14:48:06.812318+0000 mgr.gibba032.whopeo (mgr.24625) 2943 : cluster [DBG] pgmap v1669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:07.907039+0000 mon.a (mon.0) 1400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.107057+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.114443+0000 mon.a (mon.0) 1402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:48:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.116214+0000 mon.a (mon.0) 1403 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.117544+0000 mon.a (mon.0) 1404 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.118681+0000 mon.a (mon.0) 1405 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.121445+0000 mon.a (mon.0) 1406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[30791]: audit 2021-03-10T14:48:08.122844+0000 mon.a (mon.0) 1407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: cluster 2021-03-10T14:48:06.812318+0000 mgr.gibba032.whopeo (mgr.24625) 2943 : cluster [DBG] pgmap v1669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:07.907039+0000 mon.a (mon.0) 1400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.107057+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:48:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.114443+0000 mon.a (mon.0) 1402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:48:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.116214+0000 mon.a (mon.0) 1403 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.117544+0000 mon.a (mon.0) 1404 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.118681+0000 mon.a (mon.0) 1405 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.121445+0000 mon.a (mon.0) 1406 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:08.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:08 gibba011 bash[31774]: audit 2021-03-10T14:48:08.122844+0000 mon.a (mon.0) 1407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:07.904846+0000 mgr.gibba032.whopeo (mgr.24625) 2944 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:48:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.110975+0000 mgr.gibba032.whopeo (mgr.24625) 2945 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:48:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.112879+0000 mgr.gibba032.whopeo (mgr.24625) 2946 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:48:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.114350+0000 mgr.gibba032.whopeo (mgr.24625) 2947 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:48:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.114410+0000 mgr.gibba032.whopeo (mgr.24625) 2948 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:48:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.115535+0000 mgr.gibba032.whopeo (mgr.24625) 2949 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:48:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.115596+0000 mgr.gibba032.whopeo (mgr.24625) 2950 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:48:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.116822+0000 mgr.gibba032.whopeo (mgr.24625) 2951 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:48:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.118162+0000 mgr.gibba032.whopeo (mgr.24625) 2952 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:48:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.118225+0000 mgr.gibba032.whopeo (mgr.24625) 2953 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:48:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.119417+0000 mgr.gibba032.whopeo (mgr.24625) 2954 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:48:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.119779+0000 mgr.gibba032.whopeo (mgr.24625) 2955 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:48:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.120744+0000 mgr.gibba032.whopeo (mgr.24625) 2956 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:48:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.120824+0000 mgr.gibba032.whopeo (mgr.24625) 2957 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:48:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:09 gibba032 bash[24004]: cephadm 2021-03-10T14:48:08.120951+0000 mgr.gibba032.whopeo (mgr.24625) 2958 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:07.904846+0000 mgr.gibba032.whopeo (mgr.24625) 2944 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:48:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.110975+0000 mgr.gibba032.whopeo (mgr.24625) 2945 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:48:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.112879+0000 mgr.gibba032.whopeo (mgr.24625) 2946 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:48:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.114350+0000 mgr.gibba032.whopeo (mgr.24625) 2947 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:48:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.114410+0000 mgr.gibba032.whopeo (mgr.24625) 2948 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:48:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.115535+0000 mgr.gibba032.whopeo (mgr.24625) 2949 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:48:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.115596+0000 mgr.gibba032.whopeo (mgr.24625) 2950 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:48:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.116822+0000 mgr.gibba032.whopeo (mgr.24625) 2951 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:48:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.118162+0000 mgr.gibba032.whopeo (mgr.24625) 2952 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:48:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.118225+0000 mgr.gibba032.whopeo (mgr.24625) 2953 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:48:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.119417+0000 mgr.gibba032.whopeo (mgr.24625) 2954 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:48:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.119779+0000 mgr.gibba032.whopeo (mgr.24625) 2955 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:48:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.120744+0000 mgr.gibba032.whopeo (mgr.24625) 2956 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:48:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.120824+0000 mgr.gibba032.whopeo (mgr.24625) 2957 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:48:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[30791]: cephadm 2021-03-10T14:48:08.120951+0000 mgr.gibba032.whopeo (mgr.24625) 2958 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:07.904846+0000 mgr.gibba032.whopeo (mgr.24625) 2944 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:48:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.110975+0000 mgr.gibba032.whopeo (mgr.24625) 2945 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:48:09.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.112879+0000 mgr.gibba032.whopeo (mgr.24625) 2946 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:48:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.114350+0000 mgr.gibba032.whopeo (mgr.24625) 2947 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:48:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.114410+0000 mgr.gibba032.whopeo (mgr.24625) 2948 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:48:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.115535+0000 mgr.gibba032.whopeo (mgr.24625) 2949 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:48:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.115596+0000 mgr.gibba032.whopeo (mgr.24625) 2950 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:48:09.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.116822+0000 mgr.gibba032.whopeo (mgr.24625) 2951 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:48:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.118162+0000 mgr.gibba032.whopeo (mgr.24625) 2952 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:48:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.118225+0000 mgr.gibba032.whopeo (mgr.24625) 2953 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:48:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.119417+0000 mgr.gibba032.whopeo (mgr.24625) 2954 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:48:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.119779+0000 mgr.gibba032.whopeo (mgr.24625) 2955 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:48:09.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.120744+0000 mgr.gibba032.whopeo (mgr.24625) 2956 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:48:09.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.120824+0000 mgr.gibba032.whopeo (mgr.24625) 2957 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:48:09.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:09 gibba011 bash[31774]: cephadm 2021-03-10T14:48:08.120951+0000 mgr.gibba032.whopeo (mgr.24625) 2958 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:10 gibba032 bash[24004]: cluster 2021-03-10T14:48:08.813678+0000 mgr.gibba032.whopeo (mgr.24625) 2959 : cluster [DBG] pgmap v1670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:10 gibba011 bash[30791]: cluster 2021-03-10T14:48:08.813678+0000 mgr.gibba032.whopeo (mgr.24625) 2959 : cluster [DBG] pgmap v1670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:10 gibba011 bash[31774]: cluster 2021-03-10T14:48:08.813678+0000 mgr.gibba032.whopeo (mgr.24625) 2959 : cluster [DBG] pgmap v1670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:11.386 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:48:11.610 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:48:11.611 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (54m) 3s ago 60m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:48:11.611 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (60m) 9m ago 60m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:48:11.611 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (55m) 9m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:48:11.611 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (55m) 9m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:48:11.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (54m) 3s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:48:11.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (54m) 3s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:48:11.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (53m) 9m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:48:11.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (54m) 3s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:48:11.612 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (60m) 3s ago 60m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:48:11.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (60m) 9m ago 60m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:48:11.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (53m) 3s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:48:11.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (52m) 3s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:48:11.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (52m) 3s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:48:11.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (51m) 3s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:48:11.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (51m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:48:11.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (51m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:48:11.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (50m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:48:11.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (50m) 9m ago 60m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:48:11.615 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (59m) 9m ago 60m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:48:11.615 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3s ago 58m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:48:11.860 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:48:11.860 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:48:11.860 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:48:11.861 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:11.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:48:11.861 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:48:11.861 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:11.861 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:48:11.862 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:48:11.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:11.862 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:48:11.862 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:48:11.862 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:48:11.863 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:48:11.863 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:48:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:12 gibba032 bash[24004]: cluster 2021-03-10T14:48:10.814998+0000 mgr.gibba032.whopeo (mgr.24625) 2960 : cluster [DBG] pgmap v1671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:12 gibba032 bash[24004]: audit 2021-03-10T14:48:11.372641+0000 mgr.gibba032.whopeo (mgr.24625) 2961 : audit [DBG] from='client.46035 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:12 gibba032 bash[24004]: audit 2021-03-10T14:48:11.858220+0000 mon.c (mon.1) 55 : audit [DBG] from='client.? 172.21.2.111:0/1239807291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[30791]: cluster 2021-03-10T14:48:10.814998+0000 mgr.gibba032.whopeo (mgr.24625) 2960 : cluster [DBG] pgmap v1671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[30791]: audit 2021-03-10T14:48:11.372641+0000 mgr.gibba032.whopeo (mgr.24625) 2961 : audit [DBG] from='client.46035 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[30791]: audit 2021-03-10T14:48:11.858220+0000 mon.c (mon.1) 55 : audit [DBG] from='client.? 172.21.2.111:0/1239807291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[31774]: cluster 2021-03-10T14:48:10.814998+0000 mgr.gibba032.whopeo (mgr.24625) 2960 : cluster [DBG] pgmap v1671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[31774]: audit 2021-03-10T14:48:11.372641+0000 mgr.gibba032.whopeo (mgr.24625) 2961 : audit [DBG] from='client.46035 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:12 gibba011 bash[31774]: audit 2021-03-10T14:48:11.858220+0000 mon.c (mon.1) 55 : audit [DBG] from='client.? 172.21.2.111:0/1239807291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:13 gibba032 bash[24004]: audit 2021-03-10T14:48:11.598453+0000 mgr.gibba032.whopeo (mgr.24625) 2962 : audit [DBG] from='client.56026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:13 gibba011 bash[30791]: audit 2021-03-10T14:48:11.598453+0000 mgr.gibba032.whopeo (mgr.24625) 2962 : audit [DBG] from='client.56026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:13.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:13 gibba011 bash[31774]: audit 2021-03-10T14:48:11.598453+0000 mgr.gibba032.whopeo (mgr.24625) 2962 : audit [DBG] from='client.56026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:14 gibba032 bash[24004]: cluster 2021-03-10T14:48:12.816700+0000 mgr.gibba032.whopeo (mgr.24625) 2963 : cluster [DBG] pgmap v1672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:14 gibba011 bash[30791]: cluster 2021-03-10T14:48:12.816700+0000 mgr.gibba032.whopeo (mgr.24625) 2963 : cluster [DBG] pgmap v1672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:14 gibba011 bash[31774]: cluster 2021-03-10T14:48:12.816700+0000 mgr.gibba032.whopeo (mgr.24625) 2963 : cluster [DBG] pgmap v1672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:16 gibba032 bash[24004]: cluster 2021-03-10T14:48:14.817955+0000 mgr.gibba032.whopeo (mgr.24625) 2964 : cluster [DBG] pgmap v1673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:16 gibba011 bash[30791]: cluster 2021-03-10T14:48:14.817955+0000 mgr.gibba032.whopeo (mgr.24625) 2964 : cluster [DBG] pgmap v1673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:16 gibba011 bash[31774]: cluster 2021-03-10T14:48:14.817955+0000 mgr.gibba032.whopeo (mgr.24625) 2964 : cluster [DBG] pgmap v1673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:18 gibba032 bash[24004]: cluster 2021-03-10T14:48:16.819574+0000 mgr.gibba032.whopeo (mgr.24625) 2965 : cluster [DBG] pgmap v1674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:18 gibba011 bash[30791]: cluster 2021-03-10T14:48:16.819574+0000 mgr.gibba032.whopeo (mgr.24625) 2965 : cluster [DBG] pgmap v1674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:18 gibba011 bash[31774]: cluster 2021-03-10T14:48:16.819574+0000 mgr.gibba032.whopeo (mgr.24625) 2965 : cluster [DBG] pgmap v1674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:19 gibba011 bash[30791]: cluster 2021-03-10T14:48:18.820970+0000 mgr.gibba032.whopeo (mgr.24625) 2966 : cluster [DBG] pgmap v1675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:19 gibba011 bash[31774]: cluster 2021-03-10T14:48:18.820970+0000 mgr.gibba032.whopeo (mgr.24625) 2966 : cluster [DBG] pgmap v1675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:19 gibba032 bash[24004]: cluster 2021-03-10T14:48:18.820970+0000 mgr.gibba032.whopeo (mgr.24625) 2966 : cluster [DBG] pgmap v1675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:22 gibba032 bash[24004]: cluster 2021-03-10T14:48:20.822241+0000 mgr.gibba032.whopeo (mgr.24625) 2967 : cluster [DBG] pgmap v1676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:22 gibba011 bash[30791]: cluster 2021-03-10T14:48:20.822241+0000 mgr.gibba032.whopeo (mgr.24625) 2967 : cluster [DBG] pgmap v1676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:22 gibba011 bash[31774]: cluster 2021-03-10T14:48:20.822241+0000 mgr.gibba032.whopeo (mgr.24625) 2967 : cluster [DBG] pgmap v1676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:24 gibba032 bash[24004]: cluster 2021-03-10T14:48:22.824085+0000 mgr.gibba032.whopeo (mgr.24625) 2968 : cluster [DBG] pgmap v1677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:24.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:24 gibba032 bash[24004]: cephadm 2021-03-10T14:48:23.136891+0000 mgr.gibba032.whopeo (mgr.24625) 2969 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:24 gibba011 bash[30791]: cluster 2021-03-10T14:48:22.824085+0000 mgr.gibba032.whopeo (mgr.24625) 2968 : cluster [DBG] pgmap v1677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:24 gibba011 bash[30791]: cephadm 2021-03-10T14:48:23.136891+0000 mgr.gibba032.whopeo (mgr.24625) 2969 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:24 gibba011 bash[31774]: cluster 2021-03-10T14:48:22.824085+0000 mgr.gibba032.whopeo (mgr.24625) 2968 : cluster [DBG] pgmap v1677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:24 gibba011 bash[31774]: cephadm 2021-03-10T14:48:23.136891+0000 mgr.gibba032.whopeo (mgr.24625) 2969 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:26 gibba032 bash[24004]: cluster 2021-03-10T14:48:24.825670+0000 mgr.gibba032.whopeo (mgr.24625) 2970 : cluster [DBG] pgmap v1678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:26 gibba011 bash[30791]: cluster 2021-03-10T14:48:24.825670+0000 mgr.gibba032.whopeo (mgr.24625) 2970 : cluster [DBG] pgmap v1678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:26 gibba011 bash[31774]: cluster 2021-03-10T14:48:24.825670+0000 mgr.gibba032.whopeo (mgr.24625) 2970 : cluster [DBG] pgmap v1678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:28 gibba032 bash[24004]: cluster 2021-03-10T14:48:26.827289+0000 mgr.gibba032.whopeo (mgr.24625) 2971 : cluster [DBG] pgmap v1679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:28 gibba011 bash[30791]: cluster 2021-03-10T14:48:26.827289+0000 mgr.gibba032.whopeo (mgr.24625) 2971 : cluster [DBG] pgmap v1679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:28 gibba011 bash[31774]: cluster 2021-03-10T14:48:26.827289+0000 mgr.gibba032.whopeo (mgr.24625) 2971 : cluster [DBG] pgmap v1679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:30 gibba032 bash[24004]: cluster 2021-03-10T14:48:28.828927+0000 mgr.gibba032.whopeo (mgr.24625) 2972 : cluster [DBG] pgmap v1680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:30 gibba011 bash[30791]: cluster 2021-03-10T14:48:28.828927+0000 mgr.gibba032.whopeo (mgr.24625) 2972 : cluster [DBG] pgmap v1680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:30 gibba011 bash[31774]: cluster 2021-03-10T14:48:28.828927+0000 mgr.gibba032.whopeo (mgr.24625) 2972 : cluster [DBG] pgmap v1680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:32 gibba032 bash[24004]: cluster 2021-03-10T14:48:30.830289+0000 mgr.gibba032.whopeo (mgr.24625) 2973 : cluster [DBG] pgmap v1681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:32 gibba011 bash[30791]: cluster 2021-03-10T14:48:30.830289+0000 mgr.gibba032.whopeo (mgr.24625) 2973 : cluster [DBG] pgmap v1681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:32 gibba011 bash[31774]: cluster 2021-03-10T14:48:30.830289+0000 mgr.gibba032.whopeo (mgr.24625) 2973 : cluster [DBG] pgmap v1681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:34 gibba032 bash[24004]: cluster 2021-03-10T14:48:32.831917+0000 mgr.gibba032.whopeo (mgr.24625) 2974 : cluster [DBG] pgmap v1682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:34 gibba011 bash[30791]: cluster 2021-03-10T14:48:32.831917+0000 mgr.gibba032.whopeo (mgr.24625) 2974 : cluster [DBG] pgmap v1682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:34 gibba011 bash[31774]: cluster 2021-03-10T14:48:32.831917+0000 mgr.gibba032.whopeo (mgr.24625) 2974 : cluster [DBG] pgmap v1682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:36 gibba032 bash[24004]: cluster 2021-03-10T14:48:34.833428+0000 mgr.gibba032.whopeo (mgr.24625) 2975 : cluster [DBG] pgmap v1683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:36 gibba011 bash[30791]: cluster 2021-03-10T14:48:34.833428+0000 mgr.gibba032.whopeo (mgr.24625) 2975 : cluster [DBG] pgmap v1683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:36 gibba011 bash[31774]: cluster 2021-03-10T14:48:34.833428+0000 mgr.gibba032.whopeo (mgr.24625) 2975 : cluster [DBG] pgmap v1683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:38 gibba032 bash[24004]: cluster 2021-03-10T14:48:36.834969+0000 mgr.gibba032.whopeo (mgr.24625) 2976 : cluster [DBG] pgmap v1684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:38 gibba011 bash[30791]: cluster 2021-03-10T14:48:36.834969+0000 mgr.gibba032.whopeo (mgr.24625) 2976 : cluster [DBG] pgmap v1684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:38 gibba011 bash[31774]: cluster 2021-03-10T14:48:36.834969+0000 mgr.gibba032.whopeo (mgr.24625) 2976 : cluster [DBG] pgmap v1684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:39.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:39 gibba032 bash[24004]: cephadm 2021-03-10T14:48:38.143304+0000 mgr.gibba032.whopeo (mgr.24625) 2977 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:39 gibba011 bash[30791]: cephadm 2021-03-10T14:48:38.143304+0000 mgr.gibba032.whopeo (mgr.24625) 2977 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:39.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:39 gibba011 bash[31774]: cephadm 2021-03-10T14:48:38.143304+0000 mgr.gibba032.whopeo (mgr.24625) 2977 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:40 gibba011 bash[30791]: cluster 2021-03-10T14:48:38.836474+0000 mgr.gibba032.whopeo (mgr.24625) 2978 : cluster [DBG] pgmap v1685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:40 gibba011 bash[31774]: cluster 2021-03-10T14:48:38.836474+0000 mgr.gibba032.whopeo (mgr.24625) 2978 : cluster [DBG] pgmap v1685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:40 gibba032 bash[24004]: cluster 2021-03-10T14:48:38.836474+0000 mgr.gibba032.whopeo (mgr.24625) 2978 : cluster [DBG] pgmap v1685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:42.098 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:48:42.330 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:48:42.331 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (55m) 34s ago 60m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:48:42.331 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (60m) 9m ago 60m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:48:42.331 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (55m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:48:42.332 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (55m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:48:42.332 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (55m) 34s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:48:42.332 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (54m) 34s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:48:42.332 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (54m) 9m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:48:42.332 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (54m) 34s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:48:42.333 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (60m) 34s ago 60m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:48:42.333 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (60m) 9m ago 60m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:48:42.333 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (53m) 34s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:48:42.333 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (53m) 34s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:48:42.333 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (52m) 34s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:48:42.334 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (52m) 34s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:48:42.334 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (51m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:48:42.334 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (51m) 9m ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:48:42.334 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (51m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:48:42.334 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (50m) 9m ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:48:42.335 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (60m) 9m ago 60m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:48:42.335 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 34s ago 58m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:48:42.581 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:48:42.582 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:48:42.582 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:48:42.582 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:42.582 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:48:42.583 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:48:42.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:42.583 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:48:42.583 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:48:42.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:48:42.584 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:48:42.584 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:48:42.584 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:48:42.584 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:48:42.584 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:48:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:42 gibba032 bash[24004]: cluster 2021-03-10T14:48:40.837789+0000 mgr.gibba032.whopeo (mgr.24625) 2979 : cluster [DBG] pgmap v1686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:42 gibba011 bash[30791]: cluster 2021-03-10T14:48:40.837789+0000 mgr.gibba032.whopeo (mgr.24625) 2979 : cluster [DBG] pgmap v1686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:42 gibba011 bash[31774]: cluster 2021-03-10T14:48:40.837789+0000 mgr.gibba032.whopeo (mgr.24625) 2979 : cluster [DBG] pgmap v1686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:43 gibba032 bash[24004]: audit 2021-03-10T14:48:42.085288+0000 mgr.gibba032.whopeo (mgr.24625) 2980 : audit [DBG] from='client.46053 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:43 gibba032 bash[24004]: audit 2021-03-10T14:48:42.313545+0000 mgr.gibba032.whopeo (mgr.24625) 2981 : audit [DBG] from='client.46059 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:43 gibba032 bash[24004]: audit 2021-03-10T14:48:42.580076+0000 mon.c (mon.1) 56 : audit [DBG] from='client.? 172.21.2.111:0/3563957131' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[30791]: audit 2021-03-10T14:48:42.085288+0000 mgr.gibba032.whopeo (mgr.24625) 2980 : audit [DBG] from='client.46053 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[30791]: audit 2021-03-10T14:48:42.313545+0000 mgr.gibba032.whopeo (mgr.24625) 2981 : audit [DBG] from='client.46059 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[30791]: audit 2021-03-10T14:48:42.580076+0000 mon.c (mon.1) 56 : audit [DBG] from='client.? 172.21.2.111:0/3563957131' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[31774]: audit 2021-03-10T14:48:42.085288+0000 mgr.gibba032.whopeo (mgr.24625) 2980 : audit [DBG] from='client.46053 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[31774]: audit 2021-03-10T14:48:42.313545+0000 mgr.gibba032.whopeo (mgr.24625) 2981 : audit [DBG] from='client.46059 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:48:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:43 gibba011 bash[31774]: audit 2021-03-10T14:48:42.580076+0000 mon.c (mon.1) 56 : audit [DBG] from='client.? 172.21.2.111:0/3563957131' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:48:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:44 gibba032 bash[24004]: cluster 2021-03-10T14:48:42.839554+0000 mgr.gibba032.whopeo (mgr.24625) 2982 : cluster [DBG] pgmap v1687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:44 gibba011 bash[30791]: cluster 2021-03-10T14:48:42.839554+0000 mgr.gibba032.whopeo (mgr.24625) 2982 : cluster [DBG] pgmap v1687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:44 gibba011 bash[31774]: cluster 2021-03-10T14:48:42.839554+0000 mgr.gibba032.whopeo (mgr.24625) 2982 : cluster [DBG] pgmap v1687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:46 gibba032 bash[24004]: cluster 2021-03-10T14:48:44.841002+0000 mgr.gibba032.whopeo (mgr.24625) 2983 : cluster [DBG] pgmap v1688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:46 gibba011 bash[30791]: cluster 2021-03-10T14:48:44.841002+0000 mgr.gibba032.whopeo (mgr.24625) 2983 : cluster [DBG] pgmap v1688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:46 gibba011 bash[31774]: cluster 2021-03-10T14:48:44.841002+0000 mgr.gibba032.whopeo (mgr.24625) 2983 : cluster [DBG] pgmap v1688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:47.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:47 gibba011 bash[30791]: cluster 2021-03-10T14:48:46.842653+0000 mgr.gibba032.whopeo (mgr.24625) 2984 : cluster [DBG] pgmap v1689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:47.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:47 gibba011 bash[31774]: cluster 2021-03-10T14:48:46.842653+0000 mgr.gibba032.whopeo (mgr.24625) 2984 : cluster [DBG] pgmap v1689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:47 gibba032 bash[24004]: cluster 2021-03-10T14:48:46.842653+0000 mgr.gibba032.whopeo (mgr.24625) 2984 : cluster [DBG] pgmap v1689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:49 gibba032 bash[24004]: cluster 2021-03-10T14:48:48.844337+0000 mgr.gibba032.whopeo (mgr.24625) 2985 : cluster [DBG] pgmap v1690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:49 gibba011 bash[30791]: cluster 2021-03-10T14:48:48.844337+0000 mgr.gibba032.whopeo (mgr.24625) 2985 : cluster [DBG] pgmap v1690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:49 gibba011 bash[31774]: cluster 2021-03-10T14:48:48.844337+0000 mgr.gibba032.whopeo (mgr.24625) 2985 : cluster [DBG] pgmap v1690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:52 gibba032 bash[24004]: cluster 2021-03-10T14:48:50.845672+0000 mgr.gibba032.whopeo (mgr.24625) 2986 : cluster [DBG] pgmap v1691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:52 gibba011 bash[30791]: cluster 2021-03-10T14:48:50.845672+0000 mgr.gibba032.whopeo (mgr.24625) 2986 : cluster [DBG] pgmap v1691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:52 gibba011 bash[31774]: cluster 2021-03-10T14:48:50.845672+0000 mgr.gibba032.whopeo (mgr.24625) 2986 : cluster [DBG] pgmap v1691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:54 gibba032 bash[24004]: cluster 2021-03-10T14:48:52.847412+0000 mgr.gibba032.whopeo (mgr.24625) 2987 : cluster [DBG] pgmap v1692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:54 gibba032 bash[24004]: cephadm 2021-03-10T14:48:53.156728+0000 mgr.gibba032.whopeo (mgr.24625) 2988 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:54 gibba011 bash[30791]: cluster 2021-03-10T14:48:52.847412+0000 mgr.gibba032.whopeo (mgr.24625) 2987 : cluster [DBG] pgmap v1692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:54 gibba011 bash[30791]: cephadm 2021-03-10T14:48:53.156728+0000 mgr.gibba032.whopeo (mgr.24625) 2988 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:54 gibba011 bash[31774]: cluster 2021-03-10T14:48:52.847412+0000 mgr.gibba032.whopeo (mgr.24625) 2987 : cluster [DBG] pgmap v1692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:54 gibba011 bash[31774]: cephadm 2021-03-10T14:48:53.156728+0000 mgr.gibba032.whopeo (mgr.24625) 2988 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:48:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: cluster 2021-03-10T14:48:54.848709+0000 mgr.gibba032.whopeo (mgr.24625) 2989 : cluster [DBG] pgmap v1693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.368636+0000 mon.a (mon.0) 1408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:48:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.369252+0000 mon.a (mon.0) 1409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:48:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.393889+0000 mon.a (mon.0) 1410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:48:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.394037+0000 mon.a (mon.0) 1411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.394164+0000 mon.a (mon.0) 1412 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:56 gibba032 bash[24004]: audit 2021-03-10T14:48:56.394307+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:48:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: cluster 2021-03-10T14:48:54.848709+0000 mgr.gibba032.whopeo (mgr.24625) 2989 : cluster [DBG] pgmap v1693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.368636+0000 mon.a (mon.0) 1408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:48:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.369252+0000 mon.a (mon.0) 1409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:48:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.393889+0000 mon.a (mon.0) 1410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:48:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.394037+0000 mon.a (mon.0) 1411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.394164+0000 mon.a (mon.0) 1412 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[30791]: audit 2021-03-10T14:48:56.394307+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:48:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: cluster 2021-03-10T14:48:54.848709+0000 mgr.gibba032.whopeo (mgr.24625) 2989 : cluster [DBG] pgmap v1693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.368636+0000 mon.a (mon.0) 1408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:48:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.369252+0000 mon.a (mon.0) 1409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:48:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.393889+0000 mon.a (mon.0) 1410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:48:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.394037+0000 mon.a (mon.0) 1411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.394164+0000 mon.a (mon.0) 1412 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T14:48:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:56 gibba011 bash[31774]: audit 2021-03-10T14:48:56.394307+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:48:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:58 gibba011 bash[30791]: cluster 2021-03-10T14:48:56.850271+0000 mgr.gibba032.whopeo (mgr.24625) 2990 : cluster [DBG] pgmap v1694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:58 gibba011 bash[31774]: cluster 2021-03-10T14:48:56.850271+0000 mgr.gibba032.whopeo (mgr.24625) 2990 : cluster [DBG] pgmap v1694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:58 gibba032 bash[24004]: cluster 2021-03-10T14:48:56.850271+0000 mgr.gibba032.whopeo (mgr.24625) 2990 : cluster [DBG] pgmap v1694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:48:59 gibba011 bash[30791]: cluster 2021-03-10T14:48:58.851871+0000 mgr.gibba032.whopeo (mgr.24625) 2991 : cluster [DBG] pgmap v1695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:48:59.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:48:59 gibba011 bash[31774]: cluster 2021-03-10T14:48:58.851871+0000 mgr.gibba032.whopeo (mgr.24625) 2991 : cluster [DBG] pgmap v1695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:48:59 gibba032 bash[24004]: cluster 2021-03-10T14:48:58.851871+0000 mgr.gibba032.whopeo (mgr.24625) 2991 : cluster [DBG] pgmap v1695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:02 gibba032 bash[24004]: cluster 2021-03-10T14:49:00.853294+0000 mgr.gibba032.whopeo (mgr.24625) 2992 : cluster [DBG] pgmap v1696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:02 gibba011 bash[30791]: cluster 2021-03-10T14:49:00.853294+0000 mgr.gibba032.whopeo (mgr.24625) 2992 : cluster [DBG] pgmap v1696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:02 gibba011 bash[31774]: cluster 2021-03-10T14:49:00.853294+0000 mgr.gibba032.whopeo (mgr.24625) 2992 : cluster [DBG] pgmap v1696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:04 gibba032 bash[24004]: cluster 2021-03-10T14:49:02.855041+0000 mgr.gibba032.whopeo (mgr.24625) 2993 : cluster [DBG] pgmap v1697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:04 gibba011 bash[30791]: cluster 2021-03-10T14:49:02.855041+0000 mgr.gibba032.whopeo (mgr.24625) 2993 : cluster [DBG] pgmap v1697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:04 gibba011 bash[31774]: cluster 2021-03-10T14:49:02.855041+0000 mgr.gibba032.whopeo (mgr.24625) 2993 : cluster [DBG] pgmap v1697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:06 gibba011 bash[30791]: cluster 2021-03-10T14:49:04.856584+0000 mgr.gibba032.whopeo (mgr.24625) 2994 : cluster [DBG] pgmap v1698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:06 gibba011 bash[31774]: cluster 2021-03-10T14:49:04.856584+0000 mgr.gibba032.whopeo (mgr.24625) 2994 : cluster [DBG] pgmap v1698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:06 gibba032 bash[24004]: cluster 2021-03-10T14:49:04.856584+0000 mgr.gibba032.whopeo (mgr.24625) 2994 : cluster [DBG] pgmap v1698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:07 gibba011 bash[30791]: cluster 2021-03-10T14:49:06.858327+0000 mgr.gibba032.whopeo (mgr.24625) 2995 : cluster [DBG] pgmap v1699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:07 gibba011 bash[31774]: cluster 2021-03-10T14:49:06.858327+0000 mgr.gibba032.whopeo (mgr.24625) 2995 : cluster [DBG] pgmap v1699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:07 gibba032 bash[24004]: cluster 2021-03-10T14:49:06.858327+0000 mgr.gibba032.whopeo (mgr.24625) 2995 : cluster [DBG] pgmap v1699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:08 gibba011 bash[30791]: cephadm 2021-03-10T14:49:08.173500+0000 mgr.gibba032.whopeo (mgr.24625) 2996 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:49:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:08 gibba011 bash[30791]: audit 2021-03-10T14:49:08.178146+0000 mon.a (mon.0) 1414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:08 gibba011 bash[31774]: cephadm 2021-03-10T14:49:08.173500+0000 mgr.gibba032.whopeo (mgr.24625) 2996 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:49:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:08 gibba011 bash[31774]: audit 2021-03-10T14:49:08.178146+0000 mon.a (mon.0) 1414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:09.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:08 gibba032 bash[24004]: cephadm 2021-03-10T14:49:08.173500+0000 mgr.gibba032.whopeo (mgr.24625) 2996 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:49:09.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:08 gibba032 bash[24004]: audit 2021-03-10T14:49:08.178146+0000 mon.a (mon.0) 1414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:09 gibba032 bash[24004]: cluster 2021-03-10T14:49:08.859879+0000 mgr.gibba032.whopeo (mgr.24625) 2997 : cluster [DBG] pgmap v1700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:09 gibba011 bash[30791]: cluster 2021-03-10T14:49:08.859879+0000 mgr.gibba032.whopeo (mgr.24625) 2997 : cluster [DBG] pgmap v1700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:09 gibba011 bash[31774]: cluster 2021-03-10T14:49:08.859879+0000 mgr.gibba032.whopeo (mgr.24625) 2997 : cluster [DBG] pgmap v1700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:12 gibba032 bash[24004]: cluster 2021-03-10T14:49:10.861272+0000 mgr.gibba032.whopeo (mgr.24625) 2998 : cluster [DBG] pgmap v1701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:12 gibba032 bash[24004]: audit 2021-03-10T14:49:11.845255+0000 mon.a (mon.0) 1415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:12 gibba032 bash[24004]: audit 2021-03-10T14:49:11.846367+0000 mon.a (mon.0) 1416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:49:12.828 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:49:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[30791]: cluster 2021-03-10T14:49:10.861272+0000 mgr.gibba032.whopeo (mgr.24625) 2998 : cluster [DBG] pgmap v1701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[30791]: audit 2021-03-10T14:49:11.845255+0000 mon.a (mon.0) 1415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[30791]: audit 2021-03-10T14:49:11.846367+0000 mon.a (mon.0) 1416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:49:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[31774]: cluster 2021-03-10T14:49:10.861272+0000 mgr.gibba032.whopeo (mgr.24625) 2998 : cluster [DBG] pgmap v1701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[31774]: audit 2021-03-10T14:49:11.845255+0000 mon.a (mon.0) 1415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:12 gibba011 bash[31774]: audit 2021-03-10T14:49:11.846367+0000 mon.a (mon.0) 1416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T14:49:13.053 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:49:13.054 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (55m) 65s ago 61m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:49:13.054 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (61m) 1s ago 61m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:49:13.054 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (56m) 1s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:49:13.054 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (56m) 1s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:49:13.055 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (55m) 65s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:49:13.055 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (55m) 65s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:49:13.055 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (54m) 1s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:49:13.055 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (55m) 65s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:49:13.055 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (61m) 65s ago 61m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:49:13.056 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (61m) 1s ago 61m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:49:13.056 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (54m) 65s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:49:13.056 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (53m) 65s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:49:13.056 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (53m) 65s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:49:13.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (52m) 65s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:49:13.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (52m) 1s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:49:13.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (52m) 1s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:49:13.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (51m) 1s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:49:13.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (51m) 1s ago 61m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:49:13.058 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (60m) 1s ago 61m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:49:13.058 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 65s ago 59m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:49:13.306 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:49:13.307 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:49:13.307 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:49:13.307 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:13.307 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:49:13.307 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:49:13.308 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:13.308 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:49:13.308 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:49:13.308 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:13.308 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:49:13.309 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:49:13.309 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:49:13.309 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:49:13.309 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:49:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:13 gibba032 bash[24004]: audit 2021-03-10T14:49:13.304564+0000 mon.a (mon.0) 1417 : audit [DBG] from='client.? 172.21.2.111:0/1587620859' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:13 gibba011 bash[30791]: audit 2021-03-10T14:49:13.304564+0000 mon.a (mon.0) 1417 : audit [DBG] from='client.? 172.21.2.111:0/1587620859' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:13.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:13 gibba011 bash[31774]: audit 2021-03-10T14:49:13.304564+0000 mon.a (mon.0) 1417 : audit [DBG] from='client.? 172.21.2.111:0/1587620859' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:14.794 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:14 gibba032 bash[24004]: audit 2021-03-10T14:49:12.814113+0000 mgr.gibba032.whopeo (mgr.24625) 2999 : audit [DBG] from='client.56056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:14.794 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:14 gibba032 bash[24004]: cluster 2021-03-10T14:49:12.862726+0000 mgr.gibba032.whopeo (mgr.24625) 3000 : cluster [DBG] pgmap v1702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:14.795 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:14 gibba032 bash[24004]: audit 2021-03-10T14:49:13.040817+0000 mgr.gibba032.whopeo (mgr.24625) 3001 : audit [DBG] from='client.46077 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[30791]: audit 2021-03-10T14:49:12.814113+0000 mgr.gibba032.whopeo (mgr.24625) 2999 : audit [DBG] from='client.56056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[30791]: cluster 2021-03-10T14:49:12.862726+0000 mgr.gibba032.whopeo (mgr.24625) 3000 : cluster [DBG] pgmap v1702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[30791]: audit 2021-03-10T14:49:13.040817+0000 mgr.gibba032.whopeo (mgr.24625) 3001 : audit [DBG] from='client.46077 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[31774]: audit 2021-03-10T14:49:12.814113+0000 mgr.gibba032.whopeo (mgr.24625) 2999 : audit [DBG] from='client.56056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[31774]: cluster 2021-03-10T14:49:12.862726+0000 mgr.gibba032.whopeo (mgr.24625) 3000 : cluster [DBG] pgmap v1702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:14 gibba011 bash[31774]: audit 2021-03-10T14:49:13.040817+0000 mgr.gibba032.whopeo (mgr.24625) 3001 : audit [DBG] from='client.46077 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:15.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:15 gibba011 bash[30791]: cluster 2021-03-10T14:49:14.863226+0000 mgr.gibba032.whopeo (mgr.24625) 3002 : cluster [DBG] pgmap v1703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:15.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:15 gibba011 bash[31774]: cluster 2021-03-10T14:49:14.863226+0000 mgr.gibba032.whopeo (mgr.24625) 3002 : cluster [DBG] pgmap v1703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:15 gibba032 bash[24004]: cluster 2021-03-10T14:49:14.863226+0000 mgr.gibba032.whopeo (mgr.24625) 3002 : cluster [DBG] pgmap v1703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:16.667604+0000 mon.a (mon.0) 1418 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:16.838749+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.027611+0000 mon.a (mon.0) 1420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.031252+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:49:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.032812+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.034303+0000 mon.a (mon.0) 1423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.035585+0000 mon.a (mon.0) 1424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.038208+0000 mon.a (mon.0) 1425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:17 gibba032 bash[24004]: audit 2021-03-10T14:49:17.039790+0000 mon.a (mon.0) 1426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:16.667604+0000 mon.a (mon.0) 1418 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:16.838749+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.027611+0000 mon.a (mon.0) 1420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.031252+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:49:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.032812+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.034303+0000 mon.a (mon.0) 1423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.035585+0000 mon.a (mon.0) 1424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.038208+0000 mon.a (mon.0) 1425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[30791]: audit 2021-03-10T14:49:17.039790+0000 mon.a (mon.0) 1426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:16.667604+0000 mon.a (mon.0) 1418 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:49:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:16.838749+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.027611+0000 mon.a (mon.0) 1420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:49:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.031252+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:49:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.032812+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.034303+0000 mon.a (mon.0) 1423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.035585+0000 mon.a (mon.0) 1424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.038208+0000 mon.a (mon.0) 1425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:17.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:17 gibba011 bash[31774]: audit 2021-03-10T14:49:17.039790+0000 mon.a (mon.0) 1426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:16.836633+0000 mgr.gibba032.whopeo (mgr.24625) 3003 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:49:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cluster 2021-03-10T14:49:16.863628+0000 mgr.gibba032.whopeo (mgr.24625) 3004 : cluster [DBG] pgmap v1704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.027629+0000 mgr.gibba032.whopeo (mgr.24625) 3005 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:49:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.029467+0000 mgr.gibba032.whopeo (mgr.24625) 3006 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:49:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.030938+0000 mgr.gibba032.whopeo (mgr.24625) 3007 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:49:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.030977+0000 mgr.gibba032.whopeo (mgr.24625) 3008 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:49:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.032396+0000 mgr.gibba032.whopeo (mgr.24625) 3009 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:49:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.032437+0000 mgr.gibba032.whopeo (mgr.24625) 3010 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:49:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.033680+0000 mgr.gibba032.whopeo (mgr.24625) 3011 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:49:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.034810+0000 mgr.gibba032.whopeo (mgr.24625) 3012 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:49:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.034852+0000 mgr.gibba032.whopeo (mgr.24625) 3013 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:49:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.036179+0000 mgr.gibba032.whopeo (mgr.24625) 3014 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:49:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.036559+0000 mgr.gibba032.whopeo (mgr.24625) 3015 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:49:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.038000+0000 mgr.gibba032.whopeo (mgr.24625) 3016 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:49:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.038046+0000 mgr.gibba032.whopeo (mgr.24625) 3017 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:49:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:18 gibba032 bash[24004]: cephadm 2021-03-10T14:49:17.038172+0000 mgr.gibba032.whopeo (mgr.24625) 3018 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:16.836633+0000 mgr.gibba032.whopeo (mgr.24625) 3003 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:49:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cluster 2021-03-10T14:49:16.863628+0000 mgr.gibba032.whopeo (mgr.24625) 3004 : cluster [DBG] pgmap v1704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.027629+0000 mgr.gibba032.whopeo (mgr.24625) 3005 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:49:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.029467+0000 mgr.gibba032.whopeo (mgr.24625) 3006 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:49:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.030938+0000 mgr.gibba032.whopeo (mgr.24625) 3007 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:49:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.030977+0000 mgr.gibba032.whopeo (mgr.24625) 3008 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:49:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.032396+0000 mgr.gibba032.whopeo (mgr.24625) 3009 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:49:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.032437+0000 mgr.gibba032.whopeo (mgr.24625) 3010 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:49:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.033680+0000 mgr.gibba032.whopeo (mgr.24625) 3011 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:49:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.034810+0000 mgr.gibba032.whopeo (mgr.24625) 3012 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:49:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.034852+0000 mgr.gibba032.whopeo (mgr.24625) 3013 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:49:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.036179+0000 mgr.gibba032.whopeo (mgr.24625) 3014 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:49:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.036559+0000 mgr.gibba032.whopeo (mgr.24625) 3015 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:49:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.038000+0000 mgr.gibba032.whopeo (mgr.24625) 3016 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:49:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.038046+0000 mgr.gibba032.whopeo (mgr.24625) 3017 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:49:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[30791]: cephadm 2021-03-10T14:49:17.038172+0000 mgr.gibba032.whopeo (mgr.24625) 3018 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:18.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:16.836633+0000 mgr.gibba032.whopeo (mgr.24625) 3003 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:49:18.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cluster 2021-03-10T14:49:16.863628+0000 mgr.gibba032.whopeo (mgr.24625) 3004 : cluster [DBG] pgmap v1704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:18.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.027629+0000 mgr.gibba032.whopeo (mgr.24625) 3005 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:49:18.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.029467+0000 mgr.gibba032.whopeo (mgr.24625) 3006 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:49:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.030938+0000 mgr.gibba032.whopeo (mgr.24625) 3007 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:49:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.030977+0000 mgr.gibba032.whopeo (mgr.24625) 3008 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:49:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.032396+0000 mgr.gibba032.whopeo (mgr.24625) 3009 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:49:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.032437+0000 mgr.gibba032.whopeo (mgr.24625) 3010 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:49:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.033680+0000 mgr.gibba032.whopeo (mgr.24625) 3011 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:49:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.034810+0000 mgr.gibba032.whopeo (mgr.24625) 3012 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:49:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.034852+0000 mgr.gibba032.whopeo (mgr.24625) 3013 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:49:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.036179+0000 mgr.gibba032.whopeo (mgr.24625) 3014 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:49:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.036559+0000 mgr.gibba032.whopeo (mgr.24625) 3015 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:49:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.038000+0000 mgr.gibba032.whopeo (mgr.24625) 3016 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:49:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.038046+0000 mgr.gibba032.whopeo (mgr.24625) 3017 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:49:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:18 gibba011 bash[31774]: cephadm 2021-03-10T14:49:17.038172+0000 mgr.gibba032.whopeo (mgr.24625) 3018 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:20 gibba011 bash[30791]: cluster 2021-03-10T14:49:18.865106+0000 mgr.gibba032.whopeo (mgr.24625) 3019 : cluster [DBG] pgmap v1705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:20 gibba011 bash[31774]: cluster 2021-03-10T14:49:18.865106+0000 mgr.gibba032.whopeo (mgr.24625) 3019 : cluster [DBG] pgmap v1705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:20 gibba032 bash[24004]: cluster 2021-03-10T14:49:18.865106+0000 mgr.gibba032.whopeo (mgr.24625) 3019 : cluster [DBG] pgmap v1705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:22 gibba032 bash[24004]: cluster 2021-03-10T14:49:20.866553+0000 mgr.gibba032.whopeo (mgr.24625) 3020 : cluster [DBG] pgmap v1706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:22 gibba011 bash[30791]: cluster 2021-03-10T14:49:20.866553+0000 mgr.gibba032.whopeo (mgr.24625) 3020 : cluster [DBG] pgmap v1706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:22 gibba011 bash[31774]: cluster 2021-03-10T14:49:20.866553+0000 mgr.gibba032.whopeo (mgr.24625) 3020 : cluster [DBG] pgmap v1706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:24 gibba011 bash[30791]: cluster 2021-03-10T14:49:22.868211+0000 mgr.gibba032.whopeo (mgr.24625) 3021 : cluster [DBG] pgmap v1707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:24 gibba011 bash[31774]: cluster 2021-03-10T14:49:22.868211+0000 mgr.gibba032.whopeo (mgr.24625) 3021 : cluster [DBG] pgmap v1707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:24 gibba032 bash[24004]: cluster 2021-03-10T14:49:22.868211+0000 mgr.gibba032.whopeo (mgr.24625) 3021 : cluster [DBG] pgmap v1707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:25 gibba011 bash[30791]: cluster 2021-03-10T14:49:24.869644+0000 mgr.gibba032.whopeo (mgr.24625) 3022 : cluster [DBG] pgmap v1708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:25.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:25 gibba011 bash[31774]: cluster 2021-03-10T14:49:24.869644+0000 mgr.gibba032.whopeo (mgr.24625) 3022 : cluster [DBG] pgmap v1708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:25 gibba032 bash[24004]: cluster 2021-03-10T14:49:24.869644+0000 mgr.gibba032.whopeo (mgr.24625) 3022 : cluster [DBG] pgmap v1708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:28 gibba011 bash[30791]: cluster 2021-03-10T14:49:26.871255+0000 mgr.gibba032.whopeo (mgr.24625) 3023 : cluster [DBG] pgmap v1709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:28 gibba011 bash[31774]: cluster 2021-03-10T14:49:26.871255+0000 mgr.gibba032.whopeo (mgr.24625) 3023 : cluster [DBG] pgmap v1709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:29.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:28 gibba032 bash[24004]: cluster 2021-03-10T14:49:26.871255+0000 mgr.gibba032.whopeo (mgr.24625) 3023 : cluster [DBG] pgmap v1709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:30 gibba011 bash[30791]: cluster 2021-03-10T14:49:28.872964+0000 mgr.gibba032.whopeo (mgr.24625) 3024 : cluster [DBG] pgmap v1710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:30 gibba011 bash[31774]: cluster 2021-03-10T14:49:28.872964+0000 mgr.gibba032.whopeo (mgr.24625) 3024 : cluster [DBG] pgmap v1710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:30 gibba032 bash[24004]: cluster 2021-03-10T14:49:28.872964+0000 mgr.gibba032.whopeo (mgr.24625) 3024 : cluster [DBG] pgmap v1710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:32 gibba011 bash[30791]: cluster 2021-03-10T14:49:30.874279+0000 mgr.gibba032.whopeo (mgr.24625) 3025 : cluster [DBG] pgmap v1711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:32 gibba011 bash[31774]: cluster 2021-03-10T14:49:30.874279+0000 mgr.gibba032.whopeo (mgr.24625) 3025 : cluster [DBG] pgmap v1711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:32 gibba032 bash[24004]: cluster 2021-03-10T14:49:30.874279+0000 mgr.gibba032.whopeo (mgr.24625) 3025 : cluster [DBG] pgmap v1711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:33 gibba011 bash[30791]: cephadm 2021-03-10T14:49:32.054151+0000 mgr.gibba032.whopeo (mgr.24625) 3026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:33 gibba011 bash[31774]: cephadm 2021-03-10T14:49:32.054151+0000 mgr.gibba032.whopeo (mgr.24625) 3026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:34.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:33 gibba032 bash[24004]: cephadm 2021-03-10T14:49:32.054151+0000 mgr.gibba032.whopeo (mgr.24625) 3026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:34 gibba011 bash[30791]: cluster 2021-03-10T14:49:32.875992+0000 mgr.gibba032.whopeo (mgr.24625) 3027 : cluster [DBG] pgmap v1712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:34 gibba011 bash[31774]: cluster 2021-03-10T14:49:32.875992+0000 mgr.gibba032.whopeo (mgr.24625) 3027 : cluster [DBG] pgmap v1712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:35.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:34 gibba032 bash[24004]: cluster 2021-03-10T14:49:32.875992+0000 mgr.gibba032.whopeo (mgr.24625) 3027 : cluster [DBG] pgmap v1712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:35.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:35 gibba011 bash[30791]: cluster 2021-03-10T14:49:34.877621+0000 mgr.gibba032.whopeo (mgr.24625) 3028 : cluster [DBG] pgmap v1713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:35.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:35 gibba011 bash[31774]: cluster 2021-03-10T14:49:34.877621+0000 mgr.gibba032.whopeo (mgr.24625) 3028 : cluster [DBG] pgmap v1713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:36.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:35 gibba032 bash[24004]: cluster 2021-03-10T14:49:34.877621+0000 mgr.gibba032.whopeo (mgr.24625) 3028 : cluster [DBG] pgmap v1713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:38 gibba011 bash[30791]: cluster 2021-03-10T14:49:36.879091+0000 mgr.gibba032.whopeo (mgr.24625) 3029 : cluster [DBG] pgmap v1714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:38 gibba011 bash[31774]: cluster 2021-03-10T14:49:36.879091+0000 mgr.gibba032.whopeo (mgr.24625) 3029 : cluster [DBG] pgmap v1714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:38 gibba032 bash[24004]: cluster 2021-03-10T14:49:36.879091+0000 mgr.gibba032.whopeo (mgr.24625) 3029 : cluster [DBG] pgmap v1714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:40 gibba011 bash[30791]: cluster 2021-03-10T14:49:38.880780+0000 mgr.gibba032.whopeo (mgr.24625) 3030 : cluster [DBG] pgmap v1715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:40 gibba011 bash[31774]: cluster 2021-03-10T14:49:38.880780+0000 mgr.gibba032.whopeo (mgr.24625) 3030 : cluster [DBG] pgmap v1715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:40 gibba032 bash[24004]: cluster 2021-03-10T14:49:38.880780+0000 mgr.gibba032.whopeo (mgr.24625) 3030 : cluster [DBG] pgmap v1715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:42 gibba011 bash[30791]: cluster 2021-03-10T14:49:40.882195+0000 mgr.gibba032.whopeo (mgr.24625) 3031 : cluster [DBG] pgmap v1716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:42 gibba011 bash[31774]: cluster 2021-03-10T14:49:40.882195+0000 mgr.gibba032.whopeo (mgr.24625) 3031 : cluster [DBG] pgmap v1716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:42 gibba032 bash[24004]: cluster 2021-03-10T14:49:40.882195+0000 mgr.gibba032.whopeo (mgr.24625) 3031 : cluster [DBG] pgmap v1716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:43.548 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:49:43.780 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:49:43.781 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (56m) 95s ago 61m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:49:43.781 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (61m) 31s ago 61m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:49:43.781 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (56m) 31s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:49:43.781 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (57m) 31s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:49:43.781 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (56m) 95s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:49:43.782 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (55m) 95s ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:49:43.782 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (55m) 31s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:49:43.782 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (55m) 95s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:49:43.782 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (61m) 95s ago 61m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:49:43.783 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (61m) 31s ago 61m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:49:43.783 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (54m) 95s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:49:43.783 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (54m) 95s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:49:43.783 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (53m) 95s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:49:43.783 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (53m) 95s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:49:43.784 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (52m) 31s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:49:43.784 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (52m) 31s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:49:43.784 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (52m) 31s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:49:43.784 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (51m) 31s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:49:43.785 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (61m) 31s ago 61m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:49:43.785 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 95s ago 59m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:49:44.063 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:49:44.063 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:49:44.063 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:49:44.063 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:44.064 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:49:44.064 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:49:44.064 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:44.064 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:49:44.064 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:49:44.065 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:49:44.065 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:49:44.065 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:49:44.065 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:49:44.066 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:49:44.066 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:49:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:44 gibba011 bash[30791]: cluster 2021-03-10T14:49:42.883921+0000 mgr.gibba032.whopeo (mgr.24625) 3032 : cluster [DBG] pgmap v1717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:44 gibba011 bash[30791]: audit 2021-03-10T14:49:44.061043+0000 mon.a (mon.0) 1427 : audit [DBG] from='client.? 172.21.2.111:0/2660104222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:44 gibba011 bash[31774]: cluster 2021-03-10T14:49:42.883921+0000 mgr.gibba032.whopeo (mgr.24625) 3032 : cluster [DBG] pgmap v1717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:44 gibba011 bash[31774]: audit 2021-03-10T14:49:44.061043+0000 mon.a (mon.0) 1427 : audit [DBG] from='client.? 172.21.2.111:0/2660104222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:44 gibba032 bash[24004]: cluster 2021-03-10T14:49:42.883921+0000 mgr.gibba032.whopeo (mgr.24625) 3032 : cluster [DBG] pgmap v1717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:44 gibba032 bash[24004]: audit 2021-03-10T14:49:44.061043+0000 mon.a (mon.0) 1427 : audit [DBG] from='client.? 172.21.2.111:0/2660104222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:49:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[30791]: audit 2021-03-10T14:49:43.535525+0000 mgr.gibba032.whopeo (mgr.24625) 3033 : audit [DBG] from='client.46089 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[30791]: audit 2021-03-10T14:49:43.763808+0000 mgr.gibba032.whopeo (mgr.24625) 3034 : audit [DBG] from='client.46095 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:45.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[30791]: cluster 2021-03-10T14:49:44.885492+0000 mgr.gibba032.whopeo (mgr.24625) 3035 : cluster [DBG] pgmap v1718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[31774]: audit 2021-03-10T14:49:43.535525+0000 mgr.gibba032.whopeo (mgr.24625) 3033 : audit [DBG] from='client.46089 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[31774]: audit 2021-03-10T14:49:43.763808+0000 mgr.gibba032.whopeo (mgr.24625) 3034 : audit [DBG] from='client.46095 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:45 gibba011 bash[31774]: cluster 2021-03-10T14:49:44.885492+0000 mgr.gibba032.whopeo (mgr.24625) 3035 : cluster [DBG] pgmap v1718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:45 gibba032 bash[24004]: audit 2021-03-10T14:49:43.535525+0000 mgr.gibba032.whopeo (mgr.24625) 3033 : audit [DBG] from='client.46089 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:45 gibba032 bash[24004]: audit 2021-03-10T14:49:43.763808+0000 mgr.gibba032.whopeo (mgr.24625) 3034 : audit [DBG] from='client.46095 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:49:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:45 gibba032 bash[24004]: cluster 2021-03-10T14:49:44.885492+0000 mgr.gibba032.whopeo (mgr.24625) 3035 : cluster [DBG] pgmap v1718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:48 gibba011 bash[30791]: cluster 2021-03-10T14:49:46.886919+0000 mgr.gibba032.whopeo (mgr.24625) 3036 : cluster [DBG] pgmap v1719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:48 gibba011 bash[30791]: cephadm 2021-03-10T14:49:47.069942+0000 mgr.gibba032.whopeo (mgr.24625) 3037 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:48 gibba011 bash[31774]: cluster 2021-03-10T14:49:46.886919+0000 mgr.gibba032.whopeo (mgr.24625) 3036 : cluster [DBG] pgmap v1719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:48 gibba011 bash[31774]: cephadm 2021-03-10T14:49:47.069942+0000 mgr.gibba032.whopeo (mgr.24625) 3037 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:48 gibba032 bash[24004]: cluster 2021-03-10T14:49:46.886919+0000 mgr.gibba032.whopeo (mgr.24625) 3036 : cluster [DBG] pgmap v1719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:48 gibba032 bash[24004]: cephadm 2021-03-10T14:49:47.069942+0000 mgr.gibba032.whopeo (mgr.24625) 3037 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:49:49.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:49 gibba011 bash[30791]: cluster 2021-03-10T14:49:48.888579+0000 mgr.gibba032.whopeo (mgr.24625) 3038 : cluster [DBG] pgmap v1720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:49.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:49 gibba011 bash[31774]: cluster 2021-03-10T14:49:48.888579+0000 mgr.gibba032.whopeo (mgr.24625) 3038 : cluster [DBG] pgmap v1720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:49 gibba032 bash[24004]: cluster 2021-03-10T14:49:48.888579+0000 mgr.gibba032.whopeo (mgr.24625) 3038 : cluster [DBG] pgmap v1720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:52 gibba011 bash[30791]: cluster 2021-03-10T14:49:50.889871+0000 mgr.gibba032.whopeo (mgr.24625) 3039 : cluster [DBG] pgmap v1721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:52 gibba011 bash[31774]: cluster 2021-03-10T14:49:50.889871+0000 mgr.gibba032.whopeo (mgr.24625) 3039 : cluster [DBG] pgmap v1721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:52 gibba032 bash[24004]: cluster 2021-03-10T14:49:50.889871+0000 mgr.gibba032.whopeo (mgr.24625) 3039 : cluster [DBG] pgmap v1721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:54 gibba011 bash[30791]: cluster 2021-03-10T14:49:52.891587+0000 mgr.gibba032.whopeo (mgr.24625) 3040 : cluster [DBG] pgmap v1722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:54 gibba011 bash[31774]: cluster 2021-03-10T14:49:52.891587+0000 mgr.gibba032.whopeo (mgr.24625) 3040 : cluster [DBG] pgmap v1722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:54 gibba032 bash[24004]: cluster 2021-03-10T14:49:52.891587+0000 mgr.gibba032.whopeo (mgr.24625) 3040 : cluster [DBG] pgmap v1722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:55.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:55 gibba011 bash[30791]: cluster 2021-03-10T14:49:54.893099+0000 mgr.gibba032.whopeo (mgr.24625) 3041 : cluster [DBG] pgmap v1723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:55.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:55 gibba011 bash[31774]: cluster 2021-03-10T14:49:54.893099+0000 mgr.gibba032.whopeo (mgr.24625) 3041 : cluster [DBG] pgmap v1723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:55 gibba032 bash[24004]: cluster 2021-03-10T14:49:54.893099+0000 mgr.gibba032.whopeo (mgr.24625) 3041 : cluster [DBG] pgmap v1723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.376008+0000 mon.a (mon.0) 1428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:49:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.376954+0000 mon.a (mon.0) 1429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:49:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.397503+0000 mon.a (mon.0) 1430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:49:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.398566+0000 mon.a (mon.0) 1431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:49:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.399661+0000 mon.a (mon.0) 1432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:49:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[30791]: audit 2021-03-10T14:49:56.400654+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:49:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.376008+0000 mon.a (mon.0) 1428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:49:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.376954+0000 mon.a (mon.0) 1429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:49:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.397503+0000 mon.a (mon.0) 1430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:49:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.398566+0000 mon.a (mon.0) 1431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:49:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.399661+0000 mon.a (mon.0) 1432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:49:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:56 gibba011 bash[31774]: audit 2021-03-10T14:49:56.400654+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:49:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.376008+0000 mon.a (mon.0) 1428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:49:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.376954+0000 mon.a (mon.0) 1429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:49:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.397503+0000 mon.a (mon.0) 1430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:49:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.398566+0000 mon.a (mon.0) 1431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:49:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.399661+0000 mon.a (mon.0) 1432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:49:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:56 gibba032 bash[24004]: audit 2021-03-10T14:49:56.400654+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T14:49:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:57 gibba011 bash[30791]: cluster 2021-03-10T14:49:56.894930+0000 mgr.gibba032.whopeo (mgr.24625) 3042 : cluster [DBG] pgmap v1724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:57 gibba011 bash[31774]: cluster 2021-03-10T14:49:56.894930+0000 mgr.gibba032.whopeo (mgr.24625) 3042 : cluster [DBG] pgmap v1724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:49:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:57 gibba032 bash[24004]: cluster 2021-03-10T14:49:56.894930+0000 mgr.gibba032.whopeo (mgr.24625) 3042 : cluster [DBG] pgmap v1724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:49:59 gibba032 bash[24004]: cluster 2021-03-10T14:49:58.896432+0000 mgr.gibba032.whopeo (mgr.24625) 3043 : cluster [DBG] pgmap v1725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:49:59 gibba011 bash[30791]: cluster 2021-03-10T14:49:58.896432+0000 mgr.gibba032.whopeo (mgr.24625) 3043 : cluster [DBG] pgmap v1725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:00 gibba011 bash[30791]: debug 2021-03-10T14:49:59.999+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:50:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:49:59 gibba011 bash[31774]: cluster 2021-03-10T14:49:58.896432+0000 mgr.gibba032.whopeo (mgr.24625) 3043 : cluster [DBG] pgmap v1725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:00 gibba032 bash[24004]: cluster 2021-03-10T14:50:00.000460+0000 mon.a (mon.0) 1434 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:50:01.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:00 gibba011 bash[30791]: cluster 2021-03-10T14:50:00.000460+0000 mon.a (mon.0) 1434 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:50:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:00 gibba011 bash[31774]: cluster 2021-03-10T14:50:00.000460+0000 mon.a (mon.0) 1434 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T14:50:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:02 gibba011 bash[30791]: cluster 2021-03-10T14:50:00.897923+0000 mgr.gibba032.whopeo (mgr.24625) 3044 : cluster [DBG] pgmap v1726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:02 gibba011 bash[31774]: cluster 2021-03-10T14:50:00.897923+0000 mgr.gibba032.whopeo (mgr.24625) 3044 : cluster [DBG] pgmap v1726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:02 gibba032 bash[24004]: cluster 2021-03-10T14:50:00.897923+0000 mgr.gibba032.whopeo (mgr.24625) 3044 : cluster [DBG] pgmap v1726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:03 gibba011 bash[30791]: cephadm 2021-03-10T14:50:02.085927+0000 mgr.gibba032.whopeo (mgr.24625) 3045 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:03 gibba011 bash[31774]: cephadm 2021-03-10T14:50:02.085927+0000 mgr.gibba032.whopeo (mgr.24625) 3045 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:03 gibba032 bash[24004]: cephadm 2021-03-10T14:50:02.085927+0000 mgr.gibba032.whopeo (mgr.24625) 3045 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:04 gibba011 bash[30791]: cluster 2021-03-10T14:50:02.899682+0000 mgr.gibba032.whopeo (mgr.24625) 3046 : cluster [DBG] pgmap v1727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:04 gibba011 bash[31774]: cluster 2021-03-10T14:50:02.899682+0000 mgr.gibba032.whopeo (mgr.24625) 3046 : cluster [DBG] pgmap v1727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:04 gibba032 bash[24004]: cluster 2021-03-10T14:50:02.899682+0000 mgr.gibba032.whopeo (mgr.24625) 3046 : cluster [DBG] pgmap v1727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:05 gibba011 bash[30791]: cluster 2021-03-10T14:50:04.901174+0000 mgr.gibba032.whopeo (mgr.24625) 3047 : cluster [DBG] pgmap v1728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:05 gibba011 bash[31774]: cluster 2021-03-10T14:50:04.901174+0000 mgr.gibba032.whopeo (mgr.24625) 3047 : cluster [DBG] pgmap v1728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:05 gibba032 bash[24004]: cluster 2021-03-10T14:50:04.901174+0000 mgr.gibba032.whopeo (mgr.24625) 3047 : cluster [DBG] pgmap v1728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:08 gibba011 bash[30791]: cluster 2021-03-10T14:50:06.902806+0000 mgr.gibba032.whopeo (mgr.24625) 3048 : cluster [DBG] pgmap v1729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:08 gibba011 bash[31774]: cluster 2021-03-10T14:50:06.902806+0000 mgr.gibba032.whopeo (mgr.24625) 3048 : cluster [DBG] pgmap v1729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:09.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:08 gibba032 bash[24004]: cluster 2021-03-10T14:50:06.902806+0000 mgr.gibba032.whopeo (mgr.24625) 3048 : cluster [DBG] pgmap v1729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:09 gibba011 bash[30791]: cluster 2021-03-10T14:50:08.904421+0000 mgr.gibba032.whopeo (mgr.24625) 3049 : cluster [DBG] pgmap v1730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:09.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:09 gibba011 bash[31774]: cluster 2021-03-10T14:50:08.904421+0000 mgr.gibba032.whopeo (mgr.24625) 3049 : cluster [DBG] pgmap v1730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:10.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:09 gibba032 bash[24004]: cluster 2021-03-10T14:50:08.904421+0000 mgr.gibba032.whopeo (mgr.24625) 3049 : cluster [DBG] pgmap v1730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:12 gibba011 bash[30791]: cluster 2021-03-10T14:50:10.905791+0000 mgr.gibba032.whopeo (mgr.24625) 3050 : cluster [DBG] pgmap v1731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:12 gibba011 bash[31774]: cluster 2021-03-10T14:50:10.905791+0000 mgr.gibba032.whopeo (mgr.24625) 3050 : cluster [DBG] pgmap v1731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:12 gibba032 bash[24004]: cluster 2021-03-10T14:50:10.905791+0000 mgr.gibba032.whopeo (mgr.24625) 3050 : cluster [DBG] pgmap v1731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:14.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:13 gibba032 bash[24004]: cluster 2021-03-10T14:50:12.907296+0000 mgr.gibba032.whopeo (mgr.24625) 3051 : cluster [DBG] pgmap v1732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:13 gibba011 bash[30791]: cluster 2021-03-10T14:50:12.907296+0000 mgr.gibba032.whopeo (mgr.24625) 3051 : cluster [DBG] pgmap v1732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:14.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:13 gibba011 bash[31774]: cluster 2021-03-10T14:50:12.907296+0000 mgr.gibba032.whopeo (mgr.24625) 3051 : cluster [DBG] pgmap v1732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:14.305 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:50:14.534 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:50:14.535 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (56m) 2m ago 62m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:50:14.535 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (62m) 62s ago 62m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:50:14.535 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (57m) 62s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:50:14.536 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (57m) 62s ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:50:14.536 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (56m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:50:14.536 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (56m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:50:14.536 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (55m) 62s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:50:14.537 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (56m) 2m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:50:14.537 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (62m) 2m ago 62m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:50:14.537 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (62m) 62s ago 62m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:50:14.537 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (55m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:50:14.537 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (54m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:50:14.538 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (54m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:50:14.538 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (53m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:50:14.538 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (53m) 62s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:50:14.538 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (53m) 62s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:50:14.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (52m) 62s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:50:14.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (52m) 62s ago 62m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:50:14.539 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (61m) 62s ago 62m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:50:14.539 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 60m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:50:14.787 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:50:14.788 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:50:14.788 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:50:14.788 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:14.788 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:50:14.789 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:50:14.789 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:14.789 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:50:14.789 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:50:14.790 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:14.790 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:50:14.790 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:50:14.790 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:50:14.791 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:50:14.791 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:50:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:14 gibba032 bash[24004]: audit 2021-03-10T14:50:14.291360+0000 mgr.gibba032.whopeo (mgr.24625) 3052 : audit [DBG] from='client.56092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:15.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:14 gibba032 bash[24004]: audit 2021-03-10T14:50:14.517778+0000 mgr.gibba032.whopeo (mgr.24625) 3053 : audit [DBG] from='client.56098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:14 gibba011 bash[30791]: audit 2021-03-10T14:50:14.291360+0000 mgr.gibba032.whopeo (mgr.24625) 3052 : audit [DBG] from='client.56092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:14 gibba011 bash[30791]: audit 2021-03-10T14:50:14.517778+0000 mgr.gibba032.whopeo (mgr.24625) 3053 : audit [DBG] from='client.56098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:15.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:14 gibba011 bash[31774]: audit 2021-03-10T14:50:14.291360+0000 mgr.gibba032.whopeo (mgr.24625) 3052 : audit [DBG] from='client.56092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:14 gibba011 bash[31774]: audit 2021-03-10T14:50:14.517778+0000 mgr.gibba032.whopeo (mgr.24625) 3053 : audit [DBG] from='client.56098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:15 gibba032 bash[24004]: audit 2021-03-10T14:50:14.785856+0000 mon.c (mon.1) 57 : audit [DBG] from='client.? 172.21.2.111:0/101232721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:16.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:15 gibba032 bash[24004]: cluster 2021-03-10T14:50:14.908595+0000 mgr.gibba032.whopeo (mgr.24625) 3054 : cluster [DBG] pgmap v1733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:16.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:15 gibba011 bash[30791]: audit 2021-03-10T14:50:14.785856+0000 mon.c (mon.1) 57 : audit [DBG] from='client.? 172.21.2.111:0/101232721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:16.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:15 gibba011 bash[30791]: cluster 2021-03-10T14:50:14.908595+0000 mgr.gibba032.whopeo (mgr.24625) 3054 : cluster [DBG] pgmap v1733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:16.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:15 gibba011 bash[31774]: audit 2021-03-10T14:50:14.785856+0000 mon.c (mon.1) 57 : audit [DBG] from='client.? 172.21.2.111:0/101232721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:16.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:15 gibba011 bash[31774]: cluster 2021-03-10T14:50:14.908595+0000 mgr.gibba032.whopeo (mgr.24625) 3054 : cluster [DBG] pgmap v1733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:18.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cluster 2021-03-10T14:50:16.910259+0000 mgr.gibba032.whopeo (mgr.24625) 3055 : cluster [DBG] pgmap v1734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.102756+0000 mgr.gibba032.whopeo (mgr.24625) 3056 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:50:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.104353+0000 mgr.gibba032.whopeo (mgr.24625) 3057 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:50:18.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.334846+0000 mon.a (mon.0) 1435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:50:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.338499+0000 mgr.gibba032.whopeo (mgr.24625) 3058 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:50:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.340531+0000 mgr.gibba032.whopeo (mgr.24625) 3059 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:50:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.342208+0000 mgr.gibba032.whopeo (mgr.24625) 3060 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:50:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.342248+0000 mgr.gibba032.whopeo (mgr.24625) 3061 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:50:18.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.342332+0000 mon.a (mon.0) 1436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:50:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.343862+0000 mgr.gibba032.whopeo (mgr.24625) 3062 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:50:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.343905+0000 mgr.gibba032.whopeo (mgr.24625) 3063 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:50:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.344122+0000 mon.a (mon.0) 1437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.345178+0000 mgr.gibba032.whopeo (mgr.24625) 3064 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:50:18.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.345786+0000 mon.a (mon.0) 1438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.346283+0000 mgr.gibba032.whopeo (mgr.24625) 3065 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:50:18.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.346325+0000 mgr.gibba032.whopeo (mgr.24625) 3066 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:50:18.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.347346+0000 mon.a (mon.0) 1439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.347635+0000 mgr.gibba032.whopeo (mgr.24625) 3067 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:50:18.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.348003+0000 mgr.gibba032.whopeo (mgr.24625) 3068 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:50:18.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.349686+0000 mgr.gibba032.whopeo (mgr.24625) 3069 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:50:18.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.349732+0000 mgr.gibba032.whopeo (mgr.24625) 3070 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:50:18.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.349758+0000 mon.a (mon.0) 1440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: cephadm 2021-03-10T14:50:17.349858+0000 mgr.gibba032.whopeo (mgr.24625) 3071 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:18.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:18 gibba032 bash[24004]: audit 2021-03-10T14:50:17.351459+0000 mon.a (mon.0) 1441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cluster 2021-03-10T14:50:16.910259+0000 mgr.gibba032.whopeo (mgr.24625) 3055 : cluster [DBG] pgmap v1734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.102756+0000 mgr.gibba032.whopeo (mgr.24625) 3056 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:50:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.104353+0000 mgr.gibba032.whopeo (mgr.24625) 3057 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:50:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.334846+0000 mon.a (mon.0) 1435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:50:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.338499+0000 mgr.gibba032.whopeo (mgr.24625) 3058 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:50:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.340531+0000 mgr.gibba032.whopeo (mgr.24625) 3059 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:50:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.342208+0000 mgr.gibba032.whopeo (mgr.24625) 3060 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:50:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.342248+0000 mgr.gibba032.whopeo (mgr.24625) 3061 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:50:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.342332+0000 mon.a (mon.0) 1436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:50:18.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.343862+0000 mgr.gibba032.whopeo (mgr.24625) 3062 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:50:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.343905+0000 mgr.gibba032.whopeo (mgr.24625) 3063 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:50:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.344122+0000 mon.a (mon.0) 1437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.345178+0000 mgr.gibba032.whopeo (mgr.24625) 3064 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:50:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.345786+0000 mon.a (mon.0) 1438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.346283+0000 mgr.gibba032.whopeo (mgr.24625) 3065 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:50:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.346325+0000 mgr.gibba032.whopeo (mgr.24625) 3066 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:50:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.347346+0000 mon.a (mon.0) 1439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.347635+0000 mgr.gibba032.whopeo (mgr.24625) 3067 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:50:18.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.348003+0000 mgr.gibba032.whopeo (mgr.24625) 3068 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:50:18.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.349686+0000 mgr.gibba032.whopeo (mgr.24625) 3069 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:50:18.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.349732+0000 mgr.gibba032.whopeo (mgr.24625) 3070 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:50:18.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cluster 2021-03-10T14:50:16.910259+0000 mgr.gibba032.whopeo (mgr.24625) 3055 : cluster [DBG] pgmap v1734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.102756+0000 mgr.gibba032.whopeo (mgr.24625) 3056 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:50:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.104353+0000 mgr.gibba032.whopeo (mgr.24625) 3057 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:50:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.334846+0000 mon.a (mon.0) 1435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:50:18.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.338499+0000 mgr.gibba032.whopeo (mgr.24625) 3058 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:50:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.340531+0000 mgr.gibba032.whopeo (mgr.24625) 3059 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:50:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.342208+0000 mgr.gibba032.whopeo (mgr.24625) 3060 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:50:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.342248+0000 mgr.gibba032.whopeo (mgr.24625) 3061 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:50:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.342332+0000 mon.a (mon.0) 1436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:50:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.343862+0000 mgr.gibba032.whopeo (mgr.24625) 3062 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:50:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.343905+0000 mgr.gibba032.whopeo (mgr.24625) 3063 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:50:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.344122+0000 mon.a (mon.0) 1437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.345178+0000 mgr.gibba032.whopeo (mgr.24625) 3064 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:50:18.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.345786+0000 mon.a (mon.0) 1438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.346283+0000 mgr.gibba032.whopeo (mgr.24625) 3065 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:50:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.346325+0000 mgr.gibba032.whopeo (mgr.24625) 3066 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:50:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.347346+0000 mon.a (mon.0) 1439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.347635+0000 mgr.gibba032.whopeo (mgr.24625) 3067 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:50:18.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.348003+0000 mgr.gibba032.whopeo (mgr.24625) 3068 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:50:18.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.349686+0000 mgr.gibba032.whopeo (mgr.24625) 3069 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:50:18.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.349732+0000 mgr.gibba032.whopeo (mgr.24625) 3070 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:50:18.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.349758+0000 mon.a (mon.0) 1440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: cephadm 2021-03-10T14:50:17.349858+0000 mgr.gibba032.whopeo (mgr.24625) 3071 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:18.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[31774]: audit 2021-03-10T14:50:17.351459+0000 mon.a (mon.0) 1441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.349758+0000 mon.a (mon.0) 1440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:18.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: cephadm 2021-03-10T14:50:17.349858+0000 mgr.gibba032.whopeo (mgr.24625) 3071 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:18.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:18 gibba011 bash[30791]: audit 2021-03-10T14:50:17.351459+0000 mon.a (mon.0) 1441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:20 gibba032 bash[24004]: cluster 2021-03-10T14:50:18.911863+0000 mgr.gibba032.whopeo (mgr.24625) 3072 : cluster [DBG] pgmap v1735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:20 gibba011 bash[30791]: cluster 2021-03-10T14:50:18.911863+0000 mgr.gibba032.whopeo (mgr.24625) 3072 : cluster [DBG] pgmap v1735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:20 gibba011 bash[31774]: cluster 2021-03-10T14:50:18.911863+0000 mgr.gibba032.whopeo (mgr.24625) 3072 : cluster [DBG] pgmap v1735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:22 gibba011 bash[30791]: cluster 2021-03-10T14:50:20.913328+0000 mgr.gibba032.whopeo (mgr.24625) 3073 : cluster [DBG] pgmap v1736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:22 gibba011 bash[31774]: cluster 2021-03-10T14:50:20.913328+0000 mgr.gibba032.whopeo (mgr.24625) 3073 : cluster [DBG] pgmap v1736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:22 gibba032 bash[24004]: cluster 2021-03-10T14:50:20.913328+0000 mgr.gibba032.whopeo (mgr.24625) 3073 : cluster [DBG] pgmap v1736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:23 gibba032 bash[24004]: cluster 2021-03-10T14:50:22.915092+0000 mgr.gibba032.whopeo (mgr.24625) 3074 : cluster [DBG] pgmap v1737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:23 gibba011 bash[30791]: cluster 2021-03-10T14:50:22.915092+0000 mgr.gibba032.whopeo (mgr.24625) 3074 : cluster [DBG] pgmap v1737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:23 gibba011 bash[31774]: cluster 2021-03-10T14:50:22.915092+0000 mgr.gibba032.whopeo (mgr.24625) 3074 : cluster [DBG] pgmap v1737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:25 gibba032 bash[24004]: cluster 2021-03-10T14:50:24.916493+0000 mgr.gibba032.whopeo (mgr.24625) 3075 : cluster [DBG] pgmap v1738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:26.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:25 gibba011 bash[30791]: cluster 2021-03-10T14:50:24.916493+0000 mgr.gibba032.whopeo (mgr.24625) 3075 : cluster [DBG] pgmap v1738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:25 gibba011 bash[31774]: cluster 2021-03-10T14:50:24.916493+0000 mgr.gibba032.whopeo (mgr.24625) 3075 : cluster [DBG] pgmap v1738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:27 gibba032 bash[24004]: cluster 2021-03-10T14:50:26.917959+0000 mgr.gibba032.whopeo (mgr.24625) 3076 : cluster [DBG] pgmap v1739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:27 gibba011 bash[30791]: cluster 2021-03-10T14:50:26.917959+0000 mgr.gibba032.whopeo (mgr.24625) 3076 : cluster [DBG] pgmap v1739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:27 gibba011 bash[31774]: cluster 2021-03-10T14:50:26.917959+0000 mgr.gibba032.whopeo (mgr.24625) 3076 : cluster [DBG] pgmap v1739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:30.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:29 gibba032 bash[24004]: cluster 2021-03-10T14:50:28.919610+0000 mgr.gibba032.whopeo (mgr.24625) 3077 : cluster [DBG] pgmap v1740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:29 gibba011 bash[30791]: cluster 2021-03-10T14:50:28.919610+0000 mgr.gibba032.whopeo (mgr.24625) 3077 : cluster [DBG] pgmap v1740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:29 gibba011 bash[31774]: cluster 2021-03-10T14:50:28.919610+0000 mgr.gibba032.whopeo (mgr.24625) 3077 : cluster [DBG] pgmap v1740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:32 gibba032 bash[24004]: cluster 2021-03-10T14:50:30.921027+0000 mgr.gibba032.whopeo (mgr.24625) 3078 : cluster [DBG] pgmap v1741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:32 gibba011 bash[30791]: cluster 2021-03-10T14:50:30.921027+0000 mgr.gibba032.whopeo (mgr.24625) 3078 : cluster [DBG] pgmap v1741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:32 gibba011 bash[31774]: cluster 2021-03-10T14:50:30.921027+0000 mgr.gibba032.whopeo (mgr.24625) 3078 : cluster [DBG] pgmap v1741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:33 gibba011 bash[30791]: cephadm 2021-03-10T14:50:32.365945+0000 mgr.gibba032.whopeo (mgr.24625) 3079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:33 gibba011 bash[30791]: cluster 2021-03-10T14:50:32.922769+0000 mgr.gibba032.whopeo (mgr.24625) 3080 : cluster [DBG] pgmap v1742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:33.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:33 gibba011 bash[31774]: cephadm 2021-03-10T14:50:32.365945+0000 mgr.gibba032.whopeo (mgr.24625) 3079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:33.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:33 gibba011 bash[31774]: cluster 2021-03-10T14:50:32.922769+0000 mgr.gibba032.whopeo (mgr.24625) 3080 : cluster [DBG] pgmap v1742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:34.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:33 gibba032 bash[24004]: cephadm 2021-03-10T14:50:32.365945+0000 mgr.gibba032.whopeo (mgr.24625) 3079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:34.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:33 gibba032 bash[24004]: cluster 2021-03-10T14:50:32.922769+0000 mgr.gibba032.whopeo (mgr.24625) 3080 : cluster [DBG] pgmap v1742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:35 gibba032 bash[24004]: cluster 2021-03-10T14:50:34.924176+0000 mgr.gibba032.whopeo (mgr.24625) 3081 : cluster [DBG] pgmap v1743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:36.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:35 gibba011 bash[30791]: cluster 2021-03-10T14:50:34.924176+0000 mgr.gibba032.whopeo (mgr.24625) 3081 : cluster [DBG] pgmap v1743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:35 gibba011 bash[31774]: cluster 2021-03-10T14:50:34.924176+0000 mgr.gibba032.whopeo (mgr.24625) 3081 : cluster [DBG] pgmap v1743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:37 gibba032 bash[24004]: cluster 2021-03-10T14:50:36.925891+0000 mgr.gibba032.whopeo (mgr.24625) 3082 : cluster [DBG] pgmap v1744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:37 gibba011 bash[30791]: cluster 2021-03-10T14:50:36.925891+0000 mgr.gibba032.whopeo (mgr.24625) 3082 : cluster [DBG] pgmap v1744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:37 gibba011 bash[31774]: cluster 2021-03-10T14:50:36.925891+0000 mgr.gibba032.whopeo (mgr.24625) 3082 : cluster [DBG] pgmap v1744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:39 gibba032 bash[24004]: cluster 2021-03-10T14:50:38.927442+0000 mgr.gibba032.whopeo (mgr.24625) 3083 : cluster [DBG] pgmap v1745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:39 gibba011 bash[30791]: cluster 2021-03-10T14:50:38.927442+0000 mgr.gibba032.whopeo (mgr.24625) 3083 : cluster [DBG] pgmap v1745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:39 gibba011 bash[31774]: cluster 2021-03-10T14:50:38.927442+0000 mgr.gibba032.whopeo (mgr.24625) 3083 : cluster [DBG] pgmap v1745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:42 gibba011 bash[30791]: cluster 2021-03-10T14:50:40.928830+0000 mgr.gibba032.whopeo (mgr.24625) 3084 : cluster [DBG] pgmap v1746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:42 gibba011 bash[31774]: cluster 2021-03-10T14:50:40.928830+0000 mgr.gibba032.whopeo (mgr.24625) 3084 : cluster [DBG] pgmap v1746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:42 gibba032 bash[24004]: cluster 2021-03-10T14:50:40.928830+0000 mgr.gibba032.whopeo (mgr.24625) 3084 : cluster [DBG] pgmap v1746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:43 gibba011 bash[30791]: cluster 2021-03-10T14:50:42.930723+0000 mgr.gibba032.whopeo (mgr.24625) 3085 : cluster [DBG] pgmap v1747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:43.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:43 gibba011 bash[31774]: cluster 2021-03-10T14:50:42.930723+0000 mgr.gibba032.whopeo (mgr.24625) 3085 : cluster [DBG] pgmap v1747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:43 gibba032 bash[24004]: cluster 2021-03-10T14:50:42.930723+0000 mgr.gibba032.whopeo (mgr.24625) 3085 : cluster [DBG] pgmap v1747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:45.028 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:50:45.255 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:50:45.255 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (57m) 2m ago 62m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:50:45.255 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (62m) 93s ago 62m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:50:45.255 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (57m) 93s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:50:45.256 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (58m) 93s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:50:45.256 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (57m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:50:45.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (56m) 2m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:50:45.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (56m) 93s ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:50:45.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (56m) 2m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:50:45.257 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (62m) 2m ago 62m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:50:45.257 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (62m) 93s ago 62m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:50:45.257 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (55m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:50:45.257 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (55m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:50:45.257 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (54m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:50:45.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (54m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:50:45.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (53m) 93s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:50:45.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (53m) 93s ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:50:45.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (53m) 93s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:50:45.259 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (52m) 93s ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:50:45.259 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (62m) 93s ago 62m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:50:45.259 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 60m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:50:45.501 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:50:45.501 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:50:45.501 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:50:45.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:45.502 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:50:45.502 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:50:45.502 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:45.502 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:50:45.503 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:50:45.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:50:45.503 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:50:45.503 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:50:45.503 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:50:45.504 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:50:45.504 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:50:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:45 gibba032 bash[24004]: cluster 2021-03-10T14:50:44.932203+0000 mgr.gibba032.whopeo (mgr.24625) 3086 : cluster [DBG] pgmap v1748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:46.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:45 gibba032 bash[24004]: audit 2021-03-10T14:50:45.015037+0000 mgr.gibba032.whopeo (mgr.24625) 3087 : audit [DBG] from='client.46125 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:45 gibba032 bash[24004]: audit 2021-03-10T14:50:45.239468+0000 mgr.gibba032.whopeo (mgr.24625) 3088 : audit [DBG] from='client.56116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:45 gibba032 bash[24004]: audit 2021-03-10T14:50:45.499128+0000 mon.a (mon.0) 1442 : audit [DBG] from='client.? 172.21.2.111:0/3843383682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[30791]: cluster 2021-03-10T14:50:44.932203+0000 mgr.gibba032.whopeo (mgr.24625) 3086 : cluster [DBG] pgmap v1748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[30791]: audit 2021-03-10T14:50:45.015037+0000 mgr.gibba032.whopeo (mgr.24625) 3087 : audit [DBG] from='client.46125 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[30791]: audit 2021-03-10T14:50:45.239468+0000 mgr.gibba032.whopeo (mgr.24625) 3088 : audit [DBG] from='client.56116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[30791]: audit 2021-03-10T14:50:45.499128+0000 mon.a (mon.0) 1442 : audit [DBG] from='client.? 172.21.2.111:0/3843383682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[31774]: cluster 2021-03-10T14:50:44.932203+0000 mgr.gibba032.whopeo (mgr.24625) 3086 : cluster [DBG] pgmap v1748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[31774]: audit 2021-03-10T14:50:45.015037+0000 mgr.gibba032.whopeo (mgr.24625) 3087 : audit [DBG] from='client.46125 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[31774]: audit 2021-03-10T14:50:45.239468+0000 mgr.gibba032.whopeo (mgr.24625) 3088 : audit [DBG] from='client.56116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:50:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:45 gibba011 bash[31774]: audit 2021-03-10T14:50:45.499128+0000 mon.a (mon.0) 1442 : audit [DBG] from='client.? 172.21.2.111:0/3843383682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:50:47.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:47 gibba032 bash[24004]: cluster 2021-03-10T14:50:46.933702+0000 mgr.gibba032.whopeo (mgr.24625) 3089 : cluster [DBG] pgmap v1749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:47.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:47 gibba032 bash[24004]: cephadm 2021-03-10T14:50:47.381966+0000 mgr.gibba032.whopeo (mgr.24625) 3090 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:47 gibba011 bash[30791]: cluster 2021-03-10T14:50:46.933702+0000 mgr.gibba032.whopeo (mgr.24625) 3089 : cluster [DBG] pgmap v1749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:47 gibba011 bash[30791]: cephadm 2021-03-10T14:50:47.381966+0000 mgr.gibba032.whopeo (mgr.24625) 3090 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:47 gibba011 bash[31774]: cluster 2021-03-10T14:50:46.933702+0000 mgr.gibba032.whopeo (mgr.24625) 3089 : cluster [DBG] pgmap v1749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:47 gibba011 bash[31774]: cephadm 2021-03-10T14:50:47.381966+0000 mgr.gibba032.whopeo (mgr.24625) 3090 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:50:50.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:49 gibba032 bash[24004]: cluster 2021-03-10T14:50:48.935352+0000 mgr.gibba032.whopeo (mgr.24625) 3091 : cluster [DBG] pgmap v1750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:50 gibba011 bash[30791]: cluster 2021-03-10T14:50:48.935352+0000 mgr.gibba032.whopeo (mgr.24625) 3091 : cluster [DBG] pgmap v1750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:50 gibba011 bash[31774]: cluster 2021-03-10T14:50:48.935352+0000 mgr.gibba032.whopeo (mgr.24625) 3091 : cluster [DBG] pgmap v1750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:52 gibba011 bash[30791]: cluster 2021-03-10T14:50:50.936928+0000 mgr.gibba032.whopeo (mgr.24625) 3092 : cluster [DBG] pgmap v1751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:52 gibba011 bash[31774]: cluster 2021-03-10T14:50:50.936928+0000 mgr.gibba032.whopeo (mgr.24625) 3092 : cluster [DBG] pgmap v1751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:52 gibba032 bash[24004]: cluster 2021-03-10T14:50:50.936928+0000 mgr.gibba032.whopeo (mgr.24625) 3092 : cluster [DBG] pgmap v1751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:54 gibba011 bash[30791]: cluster 2021-03-10T14:50:52.938671+0000 mgr.gibba032.whopeo (mgr.24625) 3093 : cluster [DBG] pgmap v1752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:54.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:54 gibba011 bash[31774]: cluster 2021-03-10T14:50:52.938671+0000 mgr.gibba032.whopeo (mgr.24625) 3093 : cluster [DBG] pgmap v1752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:54 gibba032 bash[24004]: cluster 2021-03-10T14:50:52.938671+0000 mgr.gibba032.whopeo (mgr.24625) 3093 : cluster [DBG] pgmap v1752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:56 gibba011 bash[30791]: cluster 2021-03-10T14:50:54.940073+0000 mgr.gibba032.whopeo (mgr.24625) 3094 : cluster [DBG] pgmap v1753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:56.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:56 gibba011 bash[31774]: cluster 2021-03-10T14:50:54.940073+0000 mgr.gibba032.whopeo (mgr.24625) 3094 : cluster [DBG] pgmap v1753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:56.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:56 gibba032 bash[24004]: cluster 2021-03-10T14:50:54.940073+0000 mgr.gibba032.whopeo (mgr.24625) 3094 : cluster [DBG] pgmap v1753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.383462+0000 mon.a (mon.0) 1443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:50:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.384157+0000 mon.a (mon.0) 1444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:50:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.404604+0000 mon.a (mon.0) 1445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:50:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.405363+0000 mon.a (mon.0) 1446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:50:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.406071+0000 mon.a (mon.0) 1447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T14:50:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[30791]: audit 2021-03-10T14:50:56.406789+0000 mon.a (mon.0) 1448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:50:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.383462+0000 mon.a (mon.0) 1443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:50:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.384157+0000 mon.a (mon.0) 1444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:50:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.404604+0000 mon.a (mon.0) 1445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:50:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.405363+0000 mon.a (mon.0) 1446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:50:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.406071+0000 mon.a (mon.0) 1447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T14:50:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:57 gibba011 bash[31774]: audit 2021-03-10T14:50:56.406789+0000 mon.a (mon.0) 1448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:50:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.383462+0000 mon.a (mon.0) 1443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:50:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.384157+0000 mon.a (mon.0) 1444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:50:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.404604+0000 mon.a (mon.0) 1445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T14:50:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.405363+0000 mon.a (mon.0) 1446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T14:50:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.406071+0000 mon.a (mon.0) 1447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T14:50:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:57 gibba032 bash[24004]: audit 2021-03-10T14:50:56.406789+0000 mon.a (mon.0) 1448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:50:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:50:58 gibba011 bash[30791]: cluster 2021-03-10T14:50:56.941818+0000 mgr.gibba032.whopeo (mgr.24625) 3095 : cluster [DBG] pgmap v1754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:50:58 gibba011 bash[31774]: cluster 2021-03-10T14:50:56.941818+0000 mgr.gibba032.whopeo (mgr.24625) 3095 : cluster [DBG] pgmap v1754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:50:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:50:58 gibba032 bash[24004]: cluster 2021-03-10T14:50:56.941818+0000 mgr.gibba032.whopeo (mgr.24625) 3095 : cluster [DBG] pgmap v1754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:00 gibba011 bash[30791]: cluster 2021-03-10T14:50:58.943428+0000 mgr.gibba032.whopeo (mgr.24625) 3096 : cluster [DBG] pgmap v1755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:00 gibba011 bash[31774]: cluster 2021-03-10T14:50:58.943428+0000 mgr.gibba032.whopeo (mgr.24625) 3096 : cluster [DBG] pgmap v1755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:00 gibba032 bash[24004]: cluster 2021-03-10T14:50:58.943428+0000 mgr.gibba032.whopeo (mgr.24625) 3096 : cluster [DBG] pgmap v1755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:02 gibba011 bash[30791]: cluster 2021-03-10T14:51:00.944890+0000 mgr.gibba032.whopeo (mgr.24625) 3097 : cluster [DBG] pgmap v1756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:02.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:02 gibba011 bash[31774]: cluster 2021-03-10T14:51:00.944890+0000 mgr.gibba032.whopeo (mgr.24625) 3097 : cluster [DBG] pgmap v1756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:02.403 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:02 gibba032 bash[24004]: cluster 2021-03-10T14:51:00.944890+0000 mgr.gibba032.whopeo (mgr.24625) 3097 : cluster [DBG] pgmap v1756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:03.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:03 gibba011 bash[30791]: cephadm 2021-03-10T14:51:02.397999+0000 mgr.gibba032.whopeo (mgr.24625) 3098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:03.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:03 gibba011 bash[31774]: cephadm 2021-03-10T14:51:02.397999+0000 mgr.gibba032.whopeo (mgr.24625) 3098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:03 gibba032 bash[24004]: cephadm 2021-03-10T14:51:02.397999+0000 mgr.gibba032.whopeo (mgr.24625) 3098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:04.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:04 gibba011 bash[30791]: cluster 2021-03-10T14:51:02.946509+0000 mgr.gibba032.whopeo (mgr.24625) 3099 : cluster [DBG] pgmap v1757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:04 gibba011 bash[31774]: cluster 2021-03-10T14:51:02.946509+0000 mgr.gibba032.whopeo (mgr.24625) 3099 : cluster [DBG] pgmap v1757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:04 gibba032 bash[24004]: cluster 2021-03-10T14:51:02.946509+0000 mgr.gibba032.whopeo (mgr.24625) 3099 : cluster [DBG] pgmap v1757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:06 gibba011 bash[30791]: cluster 2021-03-10T14:51:04.948004+0000 mgr.gibba032.whopeo (mgr.24625) 3100 : cluster [DBG] pgmap v1758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:06.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:06 gibba011 bash[31774]: cluster 2021-03-10T14:51:04.948004+0000 mgr.gibba032.whopeo (mgr.24625) 3100 : cluster [DBG] pgmap v1758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:06.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:06 gibba032 bash[24004]: cluster 2021-03-10T14:51:04.948004+0000 mgr.gibba032.whopeo (mgr.24625) 3100 : cluster [DBG] pgmap v1758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:07 gibba011 bash[30791]: cluster 2021-03-10T14:51:06.949596+0000 mgr.gibba032.whopeo (mgr.24625) 3101 : cluster [DBG] pgmap v1759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:07 gibba011 bash[31774]: cluster 2021-03-10T14:51:06.949596+0000 mgr.gibba032.whopeo (mgr.24625) 3101 : cluster [DBG] pgmap v1759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:07 gibba032 bash[24004]: cluster 2021-03-10T14:51:06.949596+0000 mgr.gibba032.whopeo (mgr.24625) 3101 : cluster [DBG] pgmap v1759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:10 gibba011 bash[30791]: cluster 2021-03-10T14:51:08.951159+0000 mgr.gibba032.whopeo (mgr.24625) 3102 : cluster [DBG] pgmap v1760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:10 gibba011 bash[31774]: cluster 2021-03-10T14:51:08.951159+0000 mgr.gibba032.whopeo (mgr.24625) 3102 : cluster [DBG] pgmap v1760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:10 gibba032 bash[24004]: cluster 2021-03-10T14:51:08.951159+0000 mgr.gibba032.whopeo (mgr.24625) 3102 : cluster [DBG] pgmap v1760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:12 gibba011 bash[30791]: cluster 2021-03-10T14:51:10.952522+0000 mgr.gibba032.whopeo (mgr.24625) 3103 : cluster [DBG] pgmap v1761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:12 gibba011 bash[31774]: cluster 2021-03-10T14:51:10.952522+0000 mgr.gibba032.whopeo (mgr.24625) 3103 : cluster [DBG] pgmap v1761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:12 gibba032 bash[24004]: cluster 2021-03-10T14:51:10.952522+0000 mgr.gibba032.whopeo (mgr.24625) 3103 : cluster [DBG] pgmap v1761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:14 gibba011 bash[30791]: cluster 2021-03-10T14:51:12.954234+0000 mgr.gibba032.whopeo (mgr.24625) 3104 : cluster [DBG] pgmap v1762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:14 gibba011 bash[31774]: cluster 2021-03-10T14:51:12.954234+0000 mgr.gibba032.whopeo (mgr.24625) 3104 : cluster [DBG] pgmap v1762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:14.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:14 gibba032 bash[24004]: cluster 2021-03-10T14:51:12.954234+0000 mgr.gibba032.whopeo (mgr.24625) 3104 : cluster [DBG] pgmap v1762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:15.744 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:51:15.971 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:51:15.972 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (57m) 3m ago 63m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:51:15.972 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (63m) 2m ago 63m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:51:15.972 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (58m) 2m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:51:15.972 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (58m) 2m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:51:15.973 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (57m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:51:15.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (57m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:51:15.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (56m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:51:15.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (57m) 3m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:51:15.973 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (63m) 3m ago 63m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:51:15.974 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (63m) 2m ago 63m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:51:15.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (56m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:51:15.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (55m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:51:15.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (55m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:51:15.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (54m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:51:15.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (54m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:51:15.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (54m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:51:15.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (53m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:51:15.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (53m) 2m ago 63m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:51:15.976 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (62m) 2m ago 63m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:51:15.976 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 61m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:51:16.226 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:51:16.226 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:51:16.226 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:51:16.227 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:16.227 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:51:16.227 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:51:16.227 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:16.227 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:51:16.228 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:51:16.228 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:16.228 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:51:16.228 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:51:16.228 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:51:16.229 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:51:16.229 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:51:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:16 gibba011 bash[30791]: cluster 2021-03-10T14:51:14.955615+0000 mgr.gibba032.whopeo (mgr.24625) 3105 : cluster [DBG] pgmap v1763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:16 gibba011 bash[31774]: cluster 2021-03-10T14:51:14.955615+0000 mgr.gibba032.whopeo (mgr.24625) 3105 : cluster [DBG] pgmap v1763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:16 gibba032 bash[24004]: cluster 2021-03-10T14:51:14.955615+0000 mgr.gibba032.whopeo (mgr.24625) 3105 : cluster [DBG] pgmap v1763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[30791]: audit 2021-03-10T14:51:15.729777+0000 mgr.gibba032.whopeo (mgr.24625) 3106 : audit [DBG] from='client.46143 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[30791]: audit 2021-03-10T14:51:15.955481+0000 mgr.gibba032.whopeo (mgr.24625) 3107 : audit [DBG] from='client.46149 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[30791]: audit 2021-03-10T14:51:16.224313+0000 mon.c (mon.1) 58 : audit [DBG] from='client.? 172.21.2.111:0/3649468782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[31774]: audit 2021-03-10T14:51:15.729777+0000 mgr.gibba032.whopeo (mgr.24625) 3106 : audit [DBG] from='client.46143 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[31774]: audit 2021-03-10T14:51:15.955481+0000 mgr.gibba032.whopeo (mgr.24625) 3107 : audit [DBG] from='client.46149 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:17 gibba011 bash[31774]: audit 2021-03-10T14:51:16.224313+0000 mon.c (mon.1) 58 : audit [DBG] from='client.? 172.21.2.111:0/3649468782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:17 gibba032 bash[24004]: audit 2021-03-10T14:51:15.729777+0000 mgr.gibba032.whopeo (mgr.24625) 3106 : audit [DBG] from='client.46143 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:17 gibba032 bash[24004]: audit 2021-03-10T14:51:15.955481+0000 mgr.gibba032.whopeo (mgr.24625) 3107 : audit [DBG] from='client.46149 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:17 gibba032 bash[24004]: audit 2021-03-10T14:51:16.224313+0000 mon.c (mon.1) 58 : audit [DBG] from='client.? 172.21.2.111:0/3649468782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: cluster 2021-03-10T14:51:16.957301+0000 mgr.gibba032.whopeo (mgr.24625) 3108 : cluster [DBG] pgmap v1764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.414575+0000 mgr.gibba032.whopeo (mgr.24625) 3109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:51:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.416145+0000 mgr.gibba032.whopeo (mgr.24625) 3110 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:51:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.637700+0000 mon.a (mon.0) 1449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:51:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.641137+0000 mon.a (mon.0) 1450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:51:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.642918+0000 mon.a (mon.0) 1451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.644659+0000 mon.a (mon.0) 1452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.646053+0000 mon.a (mon.0) 1453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.648900+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[30791]: audit 2021-03-10T14:51:17.650552+0000 mon.a (mon.0) 1455 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: cluster 2021-03-10T14:51:16.957301+0000 mgr.gibba032.whopeo (mgr.24625) 3108 : cluster [DBG] pgmap v1764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:18.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.414575+0000 mgr.gibba032.whopeo (mgr.24625) 3109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:51:18.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.416145+0000 mgr.gibba032.whopeo (mgr.24625) 3110 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:51:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.637700+0000 mon.a (mon.0) 1449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:51:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.641137+0000 mon.a (mon.0) 1450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:51:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.642918+0000 mon.a (mon.0) 1451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.644659+0000 mon.a (mon.0) 1452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.646053+0000 mon.a (mon.0) 1453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.648900+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:18 gibba011 bash[31774]: audit 2021-03-10T14:51:17.650552+0000 mon.a (mon.0) 1455 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: cluster 2021-03-10T14:51:16.957301+0000 mgr.gibba032.whopeo (mgr.24625) 3108 : cluster [DBG] pgmap v1764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.414575+0000 mgr.gibba032.whopeo (mgr.24625) 3109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:51:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.416145+0000 mgr.gibba032.whopeo (mgr.24625) 3110 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:51:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.637700+0000 mon.a (mon.0) 1449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:51:18.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.641137+0000 mon.a (mon.0) 1450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:51:18.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.642918+0000 mon.a (mon.0) 1451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.644659+0000 mon.a (mon.0) 1452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.646053+0000 mon.a (mon.0) 1453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.648900+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:18.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:18 gibba032 bash[24004]: audit 2021-03-10T14:51:17.650552+0000 mon.a (mon.0) 1455 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.637432+0000 mgr.gibba032.whopeo (mgr.24625) 3111 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:51:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.639499+0000 mgr.gibba032.whopeo (mgr.24625) 3112 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:51:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.641124+0000 mgr.gibba032.whopeo (mgr.24625) 3113 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:51:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.641160+0000 mgr.gibba032.whopeo (mgr.24625) 3114 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:51:19.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.642676+0000 mgr.gibba032.whopeo (mgr.24625) 3115 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:51:19.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.642714+0000 mgr.gibba032.whopeo (mgr.24625) 3116 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:51:19.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.644013+0000 mgr.gibba032.whopeo (mgr.24625) 3117 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:51:19.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.645364+0000 mgr.gibba032.whopeo (mgr.24625) 3118 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:51:19.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.645403+0000 mgr.gibba032.whopeo (mgr.24625) 3119 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:51:19.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.646875+0000 mgr.gibba032.whopeo (mgr.24625) 3120 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:51:19.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.647208+0000 mgr.gibba032.whopeo (mgr.24625) 3121 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:51:19.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.648628+0000 mgr.gibba032.whopeo (mgr.24625) 3122 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:51:19.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.648670+0000 mgr.gibba032.whopeo (mgr.24625) 3123 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:51:19.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[30791]: cephadm 2021-03-10T14:51:17.648779+0000 mgr.gibba032.whopeo (mgr.24625) 3124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:19.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.637432+0000 mgr.gibba032.whopeo (mgr.24625) 3111 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:51:19.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.639499+0000 mgr.gibba032.whopeo (mgr.24625) 3112 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:51:19.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.641124+0000 mgr.gibba032.whopeo (mgr.24625) 3113 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:51:19.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.641160+0000 mgr.gibba032.whopeo (mgr.24625) 3114 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:51:19.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.642676+0000 mgr.gibba032.whopeo (mgr.24625) 3115 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:51:19.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.642714+0000 mgr.gibba032.whopeo (mgr.24625) 3116 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:51:19.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.644013+0000 mgr.gibba032.whopeo (mgr.24625) 3117 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:51:19.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.645364+0000 mgr.gibba032.whopeo (mgr.24625) 3118 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:51:19.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.645403+0000 mgr.gibba032.whopeo (mgr.24625) 3119 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:51:19.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.646875+0000 mgr.gibba032.whopeo (mgr.24625) 3120 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:51:19.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.647208+0000 mgr.gibba032.whopeo (mgr.24625) 3121 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:51:19.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.648628+0000 mgr.gibba032.whopeo (mgr.24625) 3122 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:51:19.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.648670+0000 mgr.gibba032.whopeo (mgr.24625) 3123 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:51:19.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:19 gibba011 bash[31774]: cephadm 2021-03-10T14:51:17.648779+0000 mgr.gibba032.whopeo (mgr.24625) 3124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.637432+0000 mgr.gibba032.whopeo (mgr.24625) 3111 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:51:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.639499+0000 mgr.gibba032.whopeo (mgr.24625) 3112 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:51:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.641124+0000 mgr.gibba032.whopeo (mgr.24625) 3113 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:51:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.641160+0000 mgr.gibba032.whopeo (mgr.24625) 3114 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:51:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.642676+0000 mgr.gibba032.whopeo (mgr.24625) 3115 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:51:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.642714+0000 mgr.gibba032.whopeo (mgr.24625) 3116 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:51:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.644013+0000 mgr.gibba032.whopeo (mgr.24625) 3117 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:51:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.645364+0000 mgr.gibba032.whopeo (mgr.24625) 3118 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:51:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.645403+0000 mgr.gibba032.whopeo (mgr.24625) 3119 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:51:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.646875+0000 mgr.gibba032.whopeo (mgr.24625) 3120 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:51:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.647208+0000 mgr.gibba032.whopeo (mgr.24625) 3121 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:51:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.648628+0000 mgr.gibba032.whopeo (mgr.24625) 3122 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:51:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.648670+0000 mgr.gibba032.whopeo (mgr.24625) 3123 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:51:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:19 gibba032 bash[24004]: cephadm 2021-03-10T14:51:17.648779+0000 mgr.gibba032.whopeo (mgr.24625) 3124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:20 gibba011 bash[30791]: cluster 2021-03-10T14:51:18.958877+0000 mgr.gibba032.whopeo (mgr.24625) 3125 : cluster [DBG] pgmap v1765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:20 gibba011 bash[31774]: cluster 2021-03-10T14:51:18.958877+0000 mgr.gibba032.whopeo (mgr.24625) 3125 : cluster [DBG] pgmap v1765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:20 gibba032 bash[24004]: cluster 2021-03-10T14:51:18.958877+0000 mgr.gibba032.whopeo (mgr.24625) 3125 : cluster [DBG] pgmap v1765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:22 gibba011 bash[30791]: cluster 2021-03-10T14:51:20.960354+0000 mgr.gibba032.whopeo (mgr.24625) 3126 : cluster [DBG] pgmap v1766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:22 gibba011 bash[31774]: cluster 2021-03-10T14:51:20.960354+0000 mgr.gibba032.whopeo (mgr.24625) 3126 : cluster [DBG] pgmap v1766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:22 gibba032 bash[24004]: cluster 2021-03-10T14:51:20.960354+0000 mgr.gibba032.whopeo (mgr.24625) 3126 : cluster [DBG] pgmap v1766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:24 gibba011 bash[30791]: cluster 2021-03-10T14:51:22.961995+0000 mgr.gibba032.whopeo (mgr.24625) 3127 : cluster [DBG] pgmap v1767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:24 gibba011 bash[31774]: cluster 2021-03-10T14:51:22.961995+0000 mgr.gibba032.whopeo (mgr.24625) 3127 : cluster [DBG] pgmap v1767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:24 gibba032 bash[24004]: cluster 2021-03-10T14:51:22.961995+0000 mgr.gibba032.whopeo (mgr.24625) 3127 : cluster [DBG] pgmap v1767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:26 gibba032 bash[24004]: cluster 2021-03-10T14:51:24.963360+0000 mgr.gibba032.whopeo (mgr.24625) 3128 : cluster [DBG] pgmap v1768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:26 gibba011 bash[30791]: cluster 2021-03-10T14:51:24.963360+0000 mgr.gibba032.whopeo (mgr.24625) 3128 : cluster [DBG] pgmap v1768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:26 gibba011 bash[31774]: cluster 2021-03-10T14:51:24.963360+0000 mgr.gibba032.whopeo (mgr.24625) 3128 : cluster [DBG] pgmap v1768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:27 gibba032 bash[24004]: cluster 2021-03-10T14:51:26.965164+0000 mgr.gibba032.whopeo (mgr.24625) 3129 : cluster [DBG] pgmap v1769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:27 gibba011 bash[30791]: cluster 2021-03-10T14:51:26.965164+0000 mgr.gibba032.whopeo (mgr.24625) 3129 : cluster [DBG] pgmap v1769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:27 gibba011 bash[31774]: cluster 2021-03-10T14:51:26.965164+0000 mgr.gibba032.whopeo (mgr.24625) 3129 : cluster [DBG] pgmap v1769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:30 gibba011 bash[30791]: cluster 2021-03-10T14:51:28.966637+0000 mgr.gibba032.whopeo (mgr.24625) 3130 : cluster [DBG] pgmap v1770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:30 gibba011 bash[31774]: cluster 2021-03-10T14:51:28.966637+0000 mgr.gibba032.whopeo (mgr.24625) 3130 : cluster [DBG] pgmap v1770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:30 gibba032 bash[24004]: cluster 2021-03-10T14:51:28.966637+0000 mgr.gibba032.whopeo (mgr.24625) 3130 : cluster [DBG] pgmap v1770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:32 gibba011 bash[30791]: cluster 2021-03-10T14:51:30.967995+0000 mgr.gibba032.whopeo (mgr.24625) 3131 : cluster [DBG] pgmap v1771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:32 gibba011 bash[31774]: cluster 2021-03-10T14:51:30.967995+0000 mgr.gibba032.whopeo (mgr.24625) 3131 : cluster [DBG] pgmap v1771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:32 gibba032 bash[24004]: cluster 2021-03-10T14:51:30.967995+0000 mgr.gibba032.whopeo (mgr.24625) 3131 : cluster [DBG] pgmap v1771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:34 gibba011 bash[30791]: cephadm 2021-03-10T14:51:32.652500+0000 mgr.gibba032.whopeo (mgr.24625) 3132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:34 gibba011 bash[30791]: cluster 2021-03-10T14:51:32.969686+0000 mgr.gibba032.whopeo (mgr.24625) 3133 : cluster [DBG] pgmap v1772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:34 gibba011 bash[31774]: cephadm 2021-03-10T14:51:32.652500+0000 mgr.gibba032.whopeo (mgr.24625) 3132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:34 gibba011 bash[31774]: cluster 2021-03-10T14:51:32.969686+0000 mgr.gibba032.whopeo (mgr.24625) 3133 : cluster [DBG] pgmap v1772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:34 gibba032 bash[24004]: cephadm 2021-03-10T14:51:32.652500+0000 mgr.gibba032.whopeo (mgr.24625) 3132 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:34 gibba032 bash[24004]: cluster 2021-03-10T14:51:32.969686+0000 mgr.gibba032.whopeo (mgr.24625) 3133 : cluster [DBG] pgmap v1772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:36.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:36 gibba011 bash[30791]: cluster 2021-03-10T14:51:34.970915+0000 mgr.gibba032.whopeo (mgr.24625) 3134 : cluster [DBG] pgmap v1773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:36 gibba011 bash[31774]: cluster 2021-03-10T14:51:34.970915+0000 mgr.gibba032.whopeo (mgr.24625) 3134 : cluster [DBG] pgmap v1773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:36.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:36 gibba032 bash[24004]: cluster 2021-03-10T14:51:34.970915+0000 mgr.gibba032.whopeo (mgr.24625) 3134 : cluster [DBG] pgmap v1773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:37 gibba032 bash[24004]: cluster 2021-03-10T14:51:36.972708+0000 mgr.gibba032.whopeo (mgr.24625) 3135 : cluster [DBG] pgmap v1774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:37 gibba011 bash[30791]: cluster 2021-03-10T14:51:36.972708+0000 mgr.gibba032.whopeo (mgr.24625) 3135 : cluster [DBG] pgmap v1774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:37 gibba011 bash[31774]: cluster 2021-03-10T14:51:36.972708+0000 mgr.gibba032.whopeo (mgr.24625) 3135 : cluster [DBG] pgmap v1774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:40 gibba011 bash[30791]: cluster 2021-03-10T14:51:38.974087+0000 mgr.gibba032.whopeo (mgr.24625) 3136 : cluster [DBG] pgmap v1775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:40 gibba011 bash[31774]: cluster 2021-03-10T14:51:38.974087+0000 mgr.gibba032.whopeo (mgr.24625) 3136 : cluster [DBG] pgmap v1775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:40 gibba032 bash[24004]: cluster 2021-03-10T14:51:38.974087+0000 mgr.gibba032.whopeo (mgr.24625) 3136 : cluster [DBG] pgmap v1775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:42 gibba011 bash[30791]: cluster 2021-03-10T14:51:40.975583+0000 mgr.gibba032.whopeo (mgr.24625) 3137 : cluster [DBG] pgmap v1776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:42 gibba011 bash[31774]: cluster 2021-03-10T14:51:40.975583+0000 mgr.gibba032.whopeo (mgr.24625) 3137 : cluster [DBG] pgmap v1776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:42 gibba032 bash[24004]: cluster 2021-03-10T14:51:40.975583+0000 mgr.gibba032.whopeo (mgr.24625) 3137 : cluster [DBG] pgmap v1776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:44 gibba011 bash[30791]: cluster 2021-03-10T14:51:42.977425+0000 mgr.gibba032.whopeo (mgr.24625) 3138 : cluster [DBG] pgmap v1777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:44 gibba011 bash[31774]: cluster 2021-03-10T14:51:42.977425+0000 mgr.gibba032.whopeo (mgr.24625) 3138 : cluster [DBG] pgmap v1777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:44.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:44 gibba032 bash[24004]: cluster 2021-03-10T14:51:42.977425+0000 mgr.gibba032.whopeo (mgr.24625) 3138 : cluster [DBG] pgmap v1777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:46 gibba011 bash[30791]: cluster 2021-03-10T14:51:44.978703+0000 mgr.gibba032.whopeo (mgr.24625) 3139 : cluster [DBG] pgmap v1778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:46 gibba011 bash[31774]: cluster 2021-03-10T14:51:44.978703+0000 mgr.gibba032.whopeo (mgr.24625) 3139 : cluster [DBG] pgmap v1778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:46.473 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:51:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:46 gibba032 bash[24004]: cluster 2021-03-10T14:51:44.978703+0000 mgr.gibba032.whopeo (mgr.24625) 3139 : cluster [DBG] pgmap v1778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:46.728 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:51:46.729 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (58m) 3m ago 63m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:51:46.729 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (63m) 2m ago 63m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:51:46.729 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (58m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:51:46.729 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (59m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:51:46.729 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (58m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:51:46.730 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (57m) 3m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:51:46.730 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (57m) 2m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:51:46.730 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (57m) 3m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:51:46.730 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (63m) 3m ago 63m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:51:46.730 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (63m) 2m ago 63m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:51:46.731 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (56m) 3m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:51:46.731 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (56m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:51:46.731 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (55m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:51:46.731 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (55m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:51:46.731 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (55m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:51:46.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (54m) 2m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:51:46.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (54m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:51:46.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (53m) 2m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:51:46.732 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (63m) 2m ago 63m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:51:46.732 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 61m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:51:46.978 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:51:46.978 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:51:46.978 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:51:46.979 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:46.979 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:51:46.979 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:51:46.979 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:46.979 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:51:46.980 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:51:46.980 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:51:46.980 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:51:46.980 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:51:46.980 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:51:46.981 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:51:46.981 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:51:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:47 gibba011 bash[30791]: audit 2021-03-10T14:51:46.459526+0000 mgr.gibba032.whopeo (mgr.24625) 3140 : audit [DBG] from='client.46161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:47 gibba011 bash[30791]: audit 2021-03-10T14:51:46.976170+0000 mon.c (mon.1) 59 : audit [DBG] from='client.? 172.21.2.111:0/2445153953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:47.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:47 gibba011 bash[31774]: audit 2021-03-10T14:51:46.459526+0000 mgr.gibba032.whopeo (mgr.24625) 3140 : audit [DBG] from='client.46161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:47 gibba011 bash[31774]: audit 2021-03-10T14:51:46.976170+0000 mon.c (mon.1) 59 : audit [DBG] from='client.? 172.21.2.111:0/2445153953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:47 gibba032 bash[24004]: audit 2021-03-10T14:51:46.459526+0000 mgr.gibba032.whopeo (mgr.24625) 3140 : audit [DBG] from='client.46161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:47.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:47 gibba032 bash[24004]: audit 2021-03-10T14:51:46.976170+0000 mon.c (mon.1) 59 : audit [DBG] from='client.? 172.21.2.111:0/2445153953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:51:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:48 gibba011 bash[30791]: audit 2021-03-10T14:51:46.684062+0000 mgr.gibba032.whopeo (mgr.24625) 3141 : audit [DBG] from='client.56152 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:48 gibba011 bash[30791]: cluster 2021-03-10T14:51:46.980378+0000 mgr.gibba032.whopeo (mgr.24625) 3142 : cluster [DBG] pgmap v1779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:48 gibba011 bash[31774]: audit 2021-03-10T14:51:46.684062+0000 mgr.gibba032.whopeo (mgr.24625) 3141 : audit [DBG] from='client.56152 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:48 gibba011 bash[31774]: cluster 2021-03-10T14:51:46.980378+0000 mgr.gibba032.whopeo (mgr.24625) 3142 : cluster [DBG] pgmap v1779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:48 gibba032 bash[24004]: audit 2021-03-10T14:51:46.684062+0000 mgr.gibba032.whopeo (mgr.24625) 3141 : audit [DBG] from='client.56152 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:51:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:48 gibba032 bash[24004]: cluster 2021-03-10T14:51:46.980378+0000 mgr.gibba032.whopeo (mgr.24625) 3142 : cluster [DBG] pgmap v1779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:49 gibba032 bash[24004]: cephadm 2021-03-10T14:51:47.668266+0000 mgr.gibba032.whopeo (mgr.24625) 3143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:49 gibba011 bash[30791]: cephadm 2021-03-10T14:51:47.668266+0000 mgr.gibba032.whopeo (mgr.24625) 3143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:49 gibba011 bash[31774]: cephadm 2021-03-10T14:51:47.668266+0000 mgr.gibba032.whopeo (mgr.24625) 3143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:51:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:50 gibba032 bash[24004]: cluster 2021-03-10T14:51:48.981876+0000 mgr.gibba032.whopeo (mgr.24625) 3144 : cluster [DBG] pgmap v1780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:50 gibba011 bash[30791]: cluster 2021-03-10T14:51:48.981876+0000 mgr.gibba032.whopeo (mgr.24625) 3144 : cluster [DBG] pgmap v1780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:50 gibba011 bash[31774]: cluster 2021-03-10T14:51:48.981876+0000 mgr.gibba032.whopeo (mgr.24625) 3144 : cluster [DBG] pgmap v1780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:52 gibba032 bash[24004]: cluster 2021-03-10T14:51:50.983346+0000 mgr.gibba032.whopeo (mgr.24625) 3145 : cluster [DBG] pgmap v1781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:52 gibba011 bash[30791]: cluster 2021-03-10T14:51:50.983346+0000 mgr.gibba032.whopeo (mgr.24625) 3145 : cluster [DBG] pgmap v1781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:52 gibba011 bash[31774]: cluster 2021-03-10T14:51:50.983346+0000 mgr.gibba032.whopeo (mgr.24625) 3145 : cluster [DBG] pgmap v1781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:54 gibba032 bash[24004]: cluster 2021-03-10T14:51:52.985286+0000 mgr.gibba032.whopeo (mgr.24625) 3146 : cluster [DBG] pgmap v1782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:54 gibba011 bash[30791]: cluster 2021-03-10T14:51:52.985286+0000 mgr.gibba032.whopeo (mgr.24625) 3146 : cluster [DBG] pgmap v1782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:54 gibba011 bash[31774]: cluster 2021-03-10T14:51:52.985286+0000 mgr.gibba032.whopeo (mgr.24625) 3146 : cluster [DBG] pgmap v1782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:56 gibba032 bash[24004]: cluster 2021-03-10T14:51:54.986484+0000 mgr.gibba032.whopeo (mgr.24625) 3147 : cluster [DBG] pgmap v1783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:56 gibba011 bash[30791]: cluster 2021-03-10T14:51:54.986484+0000 mgr.gibba032.whopeo (mgr.24625) 3147 : cluster [DBG] pgmap v1783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:56 gibba011 bash[31774]: cluster 2021-03-10T14:51:54.986484+0000 mgr.gibba032.whopeo (mgr.24625) 3147 : cluster [DBG] pgmap v1783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.390735+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:51:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.391426+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:51:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.410519+0000 mon.a (mon.0) 1458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:51:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.411304+0000 mon.a (mon.0) 1459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:51:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.412043+0000 mon.a (mon.0) 1460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:51:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:57 gibba032 bash[24004]: audit 2021-03-10T14:51:56.412753+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:51:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.390735+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:51:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.391426+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:51:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.410519+0000 mon.a (mon.0) 1458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:51:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.411304+0000 mon.a (mon.0) 1459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:51:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.412043+0000 mon.a (mon.0) 1460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:51:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[30791]: audit 2021-03-10T14:51:56.412753+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:51:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.390735+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:51:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.391426+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:51:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.410519+0000 mon.a (mon.0) 1458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T14:51:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.411304+0000 mon.a (mon.0) 1459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:51:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.412043+0000 mon.a (mon.0) 1460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T14:51:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:57 gibba011 bash[31774]: audit 2021-03-10T14:51:56.412753+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:51:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:51:58 gibba032 bash[24004]: cluster 2021-03-10T14:51:56.988144+0000 mgr.gibba032.whopeo (mgr.24625) 3148 : cluster [DBG] pgmap v1784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:51:58 gibba011 bash[30791]: cluster 2021-03-10T14:51:56.988144+0000 mgr.gibba032.whopeo (mgr.24625) 3148 : cluster [DBG] pgmap v1784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:51:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:51:58 gibba011 bash[31774]: cluster 2021-03-10T14:51:56.988144+0000 mgr.gibba032.whopeo (mgr.24625) 3148 : cluster [DBG] pgmap v1784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:00 gibba032 bash[24004]: cluster 2021-03-10T14:51:58.989422+0000 mgr.gibba032.whopeo (mgr.24625) 3149 : cluster [DBG] pgmap v1785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:00 gibba011 bash[30791]: cluster 2021-03-10T14:51:58.989422+0000 mgr.gibba032.whopeo (mgr.24625) 3149 : cluster [DBG] pgmap v1785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:00 gibba011 bash[31774]: cluster 2021-03-10T14:51:58.989422+0000 mgr.gibba032.whopeo (mgr.24625) 3149 : cluster [DBG] pgmap v1785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:02 gibba032 bash[24004]: cluster 2021-03-10T14:52:00.990758+0000 mgr.gibba032.whopeo (mgr.24625) 3150 : cluster [DBG] pgmap v1786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:02 gibba011 bash[30791]: cluster 2021-03-10T14:52:00.990758+0000 mgr.gibba032.whopeo (mgr.24625) 3150 : cluster [DBG] pgmap v1786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:02 gibba011 bash[31774]: cluster 2021-03-10T14:52:00.990758+0000 mgr.gibba032.whopeo (mgr.24625) 3150 : cluster [DBG] pgmap v1786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:04 gibba032 bash[24004]: cephadm 2021-03-10T14:52:02.672355+0000 mgr.gibba032.whopeo (mgr.24625) 3151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:04.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:04 gibba032 bash[24004]: cluster 2021-03-10T14:52:02.992559+0000 mgr.gibba032.whopeo (mgr.24625) 3152 : cluster [DBG] pgmap v1787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:04 gibba011 bash[30791]: cephadm 2021-03-10T14:52:02.672355+0000 mgr.gibba032.whopeo (mgr.24625) 3151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:04 gibba011 bash[30791]: cluster 2021-03-10T14:52:02.992559+0000 mgr.gibba032.whopeo (mgr.24625) 3152 : cluster [DBG] pgmap v1787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:04 gibba011 bash[31774]: cephadm 2021-03-10T14:52:02.672355+0000 mgr.gibba032.whopeo (mgr.24625) 3151 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:04 gibba011 bash[31774]: cluster 2021-03-10T14:52:02.992559+0000 mgr.gibba032.whopeo (mgr.24625) 3152 : cluster [DBG] pgmap v1787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:06.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:06 gibba032 bash[24004]: cluster 2021-03-10T14:52:04.993658+0000 mgr.gibba032.whopeo (mgr.24625) 3153 : cluster [DBG] pgmap v1788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:06 gibba011 bash[30791]: cluster 2021-03-10T14:52:04.993658+0000 mgr.gibba032.whopeo (mgr.24625) 3153 : cluster [DBG] pgmap v1788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:06 gibba011 bash[31774]: cluster 2021-03-10T14:52:04.993658+0000 mgr.gibba032.whopeo (mgr.24625) 3153 : cluster [DBG] pgmap v1788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:07 gibba032 bash[24004]: cluster 2021-03-10T14:52:06.995103+0000 mgr.gibba032.whopeo (mgr.24625) 3154 : cluster [DBG] pgmap v1789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:07 gibba011 bash[30791]: cluster 2021-03-10T14:52:06.995103+0000 mgr.gibba032.whopeo (mgr.24625) 3154 : cluster [DBG] pgmap v1789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:07 gibba011 bash[31774]: cluster 2021-03-10T14:52:06.995103+0000 mgr.gibba032.whopeo (mgr.24625) 3154 : cluster [DBG] pgmap v1789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:10 gibba011 bash[30791]: cluster 2021-03-10T14:52:08.996413+0000 mgr.gibba032.whopeo (mgr.24625) 3155 : cluster [DBG] pgmap v1790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:10 gibba011 bash[31774]: cluster 2021-03-10T14:52:08.996413+0000 mgr.gibba032.whopeo (mgr.24625) 3155 : cluster [DBG] pgmap v1790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:10 gibba032 bash[24004]: cluster 2021-03-10T14:52:08.996413+0000 mgr.gibba032.whopeo (mgr.24625) 3155 : cluster [DBG] pgmap v1790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:12 gibba011 bash[30791]: cluster 2021-03-10T14:52:10.997695+0000 mgr.gibba032.whopeo (mgr.24625) 3156 : cluster [DBG] pgmap v1791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:12.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:12 gibba011 bash[31774]: cluster 2021-03-10T14:52:10.997695+0000 mgr.gibba032.whopeo (mgr.24625) 3156 : cluster [DBG] pgmap v1791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:12 gibba032 bash[24004]: cluster 2021-03-10T14:52:10.997695+0000 mgr.gibba032.whopeo (mgr.24625) 3156 : cluster [DBG] pgmap v1791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:14 gibba011 bash[30791]: cluster 2021-03-10T14:52:12.999209+0000 mgr.gibba032.whopeo (mgr.24625) 3157 : cluster [DBG] pgmap v1792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:14 gibba011 bash[31774]: cluster 2021-03-10T14:52:12.999209+0000 mgr.gibba032.whopeo (mgr.24625) 3157 : cluster [DBG] pgmap v1792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:14 gibba032 bash[24004]: cluster 2021-03-10T14:52:12.999209+0000 mgr.gibba032.whopeo (mgr.24625) 3157 : cluster [DBG] pgmap v1792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:16 gibba032 bash[24004]: cluster 2021-03-10T14:52:15.000325+0000 mgr.gibba032.whopeo (mgr.24625) 3158 : cluster [DBG] pgmap v1793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:16 gibba011 bash[30791]: cluster 2021-03-10T14:52:15.000325+0000 mgr.gibba032.whopeo (mgr.24625) 3158 : cluster [DBG] pgmap v1793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:16 gibba011 bash[31774]: cluster 2021-03-10T14:52:15.000325+0000 mgr.gibba032.whopeo (mgr.24625) 3158 : cluster [DBG] pgmap v1793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:17.223 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:52:17.451 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:52:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (58m) 4m ago 64m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:52:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (64m) 3m ago 64m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:52:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (59m) 3m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:52:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (59m) 3m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:52:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (58m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:52:17.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (58m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:52:17.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (58m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:52:17.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (58m) 4m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:52:17.453 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (64m) 4m ago 64m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:52:17.454 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (64m) 3m ago 64m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:52:17.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (57m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:52:17.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (56m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:52:17.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (56m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:52:17.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (55m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:52:17.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (55m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:52:17.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (55m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:52:17.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (54m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:52:17.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (54m) 3m ago 64m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:52:17.456 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (63m) 3m ago 64m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:52:17.456 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 62m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:52:17.703 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:52:17.704 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:52:17.704 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:52:17.704 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:17.704 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:52:17.705 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:52:17.705 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:17.705 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:52:17.705 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:52:17.705 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:17.706 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:52:17.706 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:52:17.706 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:52:17.706 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:52:17.707 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:52:17.921 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:17 gibba032 bash[24004]: cluster 2021-03-10T14:52:17.001763+0000 mgr.gibba032.whopeo (mgr.24625) 3159 : cluster [DBG] pgmap v1794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:17.922 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:17 gibba032 bash[24004]: audit 2021-03-10T14:52:17.210046+0000 mgr.gibba032.whopeo (mgr.24625) 3160 : audit [DBG] from='client.56164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:17.922 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:17 gibba032 bash[24004]: audit 2021-03-10T14:52:17.434584+0000 mgr.gibba032.whopeo (mgr.24625) 3161 : audit [DBG] from='client.56170 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[30791]: cluster 2021-03-10T14:52:17.001763+0000 mgr.gibba032.whopeo (mgr.24625) 3159 : cluster [DBG] pgmap v1794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[30791]: audit 2021-03-10T14:52:17.210046+0000 mgr.gibba032.whopeo (mgr.24625) 3160 : audit [DBG] from='client.56164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[30791]: audit 2021-03-10T14:52:17.434584+0000 mgr.gibba032.whopeo (mgr.24625) 3161 : audit [DBG] from='client.56170 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[31774]: cluster 2021-03-10T14:52:17.001763+0000 mgr.gibba032.whopeo (mgr.24625) 3159 : cluster [DBG] pgmap v1794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[31774]: audit 2021-03-10T14:52:17.210046+0000 mgr.gibba032.whopeo (mgr.24625) 3160 : audit [DBG] from='client.56164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:17 gibba011 bash[31774]: audit 2021-03-10T14:52:17.434584+0000 mgr.gibba032.whopeo (mgr.24625) 3161 : audit [DBG] from='client.56170 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:19.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.682424+0000 mgr.gibba032.whopeo (mgr.24625) 3162 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:52:19.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.684011+0000 mgr.gibba032.whopeo (mgr.24625) 3163 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:52:19.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.701539+0000 mon.c (mon.1) 60 : audit [DBG] from='client.? 172.21.2.111:0/4238182586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.905618+0000 mgr.gibba032.whopeo (mgr.24625) 3164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:52:19.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.905806+0000 mon.a (mon.0) 1462 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:52:19.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.907975+0000 mgr.gibba032.whopeo (mgr.24625) 3165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:52:19.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.909216+0000 mgr.gibba032.whopeo (mgr.24625) 3166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:52:19.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.909262+0000 mgr.gibba032.whopeo (mgr.24625) 3167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:52:19.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.909477+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:52:19.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.910844+0000 mgr.gibba032.whopeo (mgr.24625) 3168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:52:19.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.910887+0000 mgr.gibba032.whopeo (mgr.24625) 3169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:52:19.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.911137+0000 mon.a (mon.0) 1464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.912138+0000 mgr.gibba032.whopeo (mgr.24625) 3170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:52:19.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.912695+0000 mon.a (mon.0) 1465 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.018 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.913398+0000 mgr.gibba032.whopeo (mgr.24625) 3171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:52:19.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.913447+0000 mgr.gibba032.whopeo (mgr.24625) 3172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:52:19.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.914074+0000 mon.a (mon.0) 1466 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.914880+0000 mgr.gibba032.whopeo (mgr.24625) 3173 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:52:19.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.915297+0000 mgr.gibba032.whopeo (mgr.24625) 3174 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:52:19.019 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.916789+0000 mgr.gibba032.whopeo (mgr.24625) 3175 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:52:19.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.916839+0000 mon.a (mon.0) 1467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.916844+0000 mgr.gibba032.whopeo (mgr.24625) 3176 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:52:19.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: cephadm 2021-03-10T14:52:17.916990+0000 mgr.gibba032.whopeo (mgr.24625) 3177 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:19.020 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:18 gibba032 bash[24004]: audit 2021-03-10T14:52:17.918477+0000 mon.a (mon.0) 1468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.682424+0000 mgr.gibba032.whopeo (mgr.24625) 3162 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:52:19.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.684011+0000 mgr.gibba032.whopeo (mgr.24625) 3163 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:52:19.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.701539+0000 mon.c (mon.1) 60 : audit [DBG] from='client.? 172.21.2.111:0/4238182586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.905618+0000 mgr.gibba032.whopeo (mgr.24625) 3164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:52:19.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.905806+0000 mon.a (mon.0) 1462 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:52:19.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.907975+0000 mgr.gibba032.whopeo (mgr.24625) 3165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:52:19.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.909216+0000 mgr.gibba032.whopeo (mgr.24625) 3166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:52:19.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.909262+0000 mgr.gibba032.whopeo (mgr.24625) 3167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:52:19.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.909477+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:52:19.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.910844+0000 mgr.gibba032.whopeo (mgr.24625) 3168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:52:19.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.910887+0000 mgr.gibba032.whopeo (mgr.24625) 3169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:52:19.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.911137+0000 mon.a (mon.0) 1464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.912138+0000 mgr.gibba032.whopeo (mgr.24625) 3170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:52:19.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.912695+0000 mon.a (mon.0) 1465 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.913398+0000 mgr.gibba032.whopeo (mgr.24625) 3171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:52:19.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.913447+0000 mgr.gibba032.whopeo (mgr.24625) 3172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:52:19.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.914074+0000 mon.a (mon.0) 1466 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.914880+0000 mgr.gibba032.whopeo (mgr.24625) 3173 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:52:19.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.915297+0000 mgr.gibba032.whopeo (mgr.24625) 3174 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:52:19.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.916789+0000 mgr.gibba032.whopeo (mgr.24625) 3175 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:52:19.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.916839+0000 mon.a (mon.0) 1467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.682424+0000 mgr.gibba032.whopeo (mgr.24625) 3162 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:52:19.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.684011+0000 mgr.gibba032.whopeo (mgr.24625) 3163 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:52:19.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.701539+0000 mon.c (mon.1) 60 : audit [DBG] from='client.? 172.21.2.111:0/4238182586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.905618+0000 mgr.gibba032.whopeo (mgr.24625) 3164 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:52:19.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.905806+0000 mon.a (mon.0) 1462 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:52:19.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.907975+0000 mgr.gibba032.whopeo (mgr.24625) 3165 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:52:19.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.909216+0000 mgr.gibba032.whopeo (mgr.24625) 3166 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:52:19.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.909262+0000 mgr.gibba032.whopeo (mgr.24625) 3167 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:52:19.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.909477+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:52:19.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.910844+0000 mgr.gibba032.whopeo (mgr.24625) 3168 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:52:19.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.910887+0000 mgr.gibba032.whopeo (mgr.24625) 3169 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:52:19.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.911137+0000 mon.a (mon.0) 1464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.912138+0000 mgr.gibba032.whopeo (mgr.24625) 3170 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:52:19.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.912695+0000 mon.a (mon.0) 1465 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.913398+0000 mgr.gibba032.whopeo (mgr.24625) 3171 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:52:19.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.913447+0000 mgr.gibba032.whopeo (mgr.24625) 3172 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:52:19.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.914074+0000 mon.a (mon.0) 1466 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.914880+0000 mgr.gibba032.whopeo (mgr.24625) 3173 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:52:19.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.915297+0000 mgr.gibba032.whopeo (mgr.24625) 3174 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:52:19.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.916789+0000 mgr.gibba032.whopeo (mgr.24625) 3175 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:52:19.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.916839+0000 mon.a (mon.0) 1467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.916844+0000 mgr.gibba032.whopeo (mgr.24625) 3176 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:52:19.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: cephadm 2021-03-10T14:52:17.916990+0000 mgr.gibba032.whopeo (mgr.24625) 3177 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:19.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[30791]: audit 2021-03-10T14:52:17.918477+0000 mon.a (mon.0) 1468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:19.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.916844+0000 mgr.gibba032.whopeo (mgr.24625) 3176 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:52:19.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: cephadm 2021-03-10T14:52:17.916990+0000 mgr.gibba032.whopeo (mgr.24625) 3177 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:19.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:18 gibba011 bash[31774]: audit 2021-03-10T14:52:17.918477+0000 mon.a (mon.0) 1468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:19 gibba032 bash[24004]: cluster 2021-03-10T14:52:19.003239+0000 mgr.gibba032.whopeo (mgr.24625) 3178 : cluster [DBG] pgmap v1795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:20.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:19 gibba011 bash[30791]: cluster 2021-03-10T14:52:19.003239+0000 mgr.gibba032.whopeo (mgr.24625) 3178 : cluster [DBG] pgmap v1795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:20.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:19 gibba011 bash[31774]: cluster 2021-03-10T14:52:19.003239+0000 mgr.gibba032.whopeo (mgr.24625) 3178 : cluster [DBG] pgmap v1795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:22 gibba011 bash[30791]: cluster 2021-03-10T14:52:21.004583+0000 mgr.gibba032.whopeo (mgr.24625) 3179 : cluster [DBG] pgmap v1796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:22 gibba011 bash[31774]: cluster 2021-03-10T14:52:21.004583+0000 mgr.gibba032.whopeo (mgr.24625) 3179 : cluster [DBG] pgmap v1796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:22 gibba032 bash[24004]: cluster 2021-03-10T14:52:21.004583+0000 mgr.gibba032.whopeo (mgr.24625) 3179 : cluster [DBG] pgmap v1796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:24 gibba011 bash[30791]: cluster 2021-03-10T14:52:23.006242+0000 mgr.gibba032.whopeo (mgr.24625) 3180 : cluster [DBG] pgmap v1797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:24 gibba011 bash[31774]: cluster 2021-03-10T14:52:23.006242+0000 mgr.gibba032.whopeo (mgr.24625) 3180 : cluster [DBG] pgmap v1797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:24 gibba032 bash[24004]: cluster 2021-03-10T14:52:23.006242+0000 mgr.gibba032.whopeo (mgr.24625) 3180 : cluster [DBG] pgmap v1797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:26 gibba032 bash[24004]: cluster 2021-03-10T14:52:25.007561+0000 mgr.gibba032.whopeo (mgr.24625) 3181 : cluster [DBG] pgmap v1798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:26 gibba011 bash[30791]: cluster 2021-03-10T14:52:25.007561+0000 mgr.gibba032.whopeo (mgr.24625) 3181 : cluster [DBG] pgmap v1798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:26 gibba011 bash[31774]: cluster 2021-03-10T14:52:25.007561+0000 mgr.gibba032.whopeo (mgr.24625) 3181 : cluster [DBG] pgmap v1798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:28.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:27 gibba032 bash[24004]: cluster 2021-03-10T14:52:27.009298+0000 mgr.gibba032.whopeo (mgr.24625) 3182 : cluster [DBG] pgmap v1799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:27 gibba011 bash[30791]: cluster 2021-03-10T14:52:27.009298+0000 mgr.gibba032.whopeo (mgr.24625) 3182 : cluster [DBG] pgmap v1799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:27 gibba011 bash[31774]: cluster 2021-03-10T14:52:27.009298+0000 mgr.gibba032.whopeo (mgr.24625) 3182 : cluster [DBG] pgmap v1799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:30 gibba011 bash[30791]: cluster 2021-03-10T14:52:29.010874+0000 mgr.gibba032.whopeo (mgr.24625) 3183 : cluster [DBG] pgmap v1800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:30 gibba011 bash[31774]: cluster 2021-03-10T14:52:29.010874+0000 mgr.gibba032.whopeo (mgr.24625) 3183 : cluster [DBG] pgmap v1800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:30 gibba032 bash[24004]: cluster 2021-03-10T14:52:29.010874+0000 mgr.gibba032.whopeo (mgr.24625) 3183 : cluster [DBG] pgmap v1800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:32 gibba032 bash[24004]: cluster 2021-03-10T14:52:31.012227+0000 mgr.gibba032.whopeo (mgr.24625) 3184 : cluster [DBG] pgmap v1801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:32 gibba011 bash[30791]: cluster 2021-03-10T14:52:31.012227+0000 mgr.gibba032.whopeo (mgr.24625) 3184 : cluster [DBG] pgmap v1801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:32 gibba011 bash[31774]: cluster 2021-03-10T14:52:31.012227+0000 mgr.gibba032.whopeo (mgr.24625) 3184 : cluster [DBG] pgmap v1801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:34 gibba032 bash[24004]: cephadm 2021-03-10T14:52:32.933001+0000 mgr.gibba032.whopeo (mgr.24625) 3185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:34 gibba032 bash[24004]: cluster 2021-03-10T14:52:33.014047+0000 mgr.gibba032.whopeo (mgr.24625) 3186 : cluster [DBG] pgmap v1802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:34 gibba011 bash[30791]: cephadm 2021-03-10T14:52:32.933001+0000 mgr.gibba032.whopeo (mgr.24625) 3185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:34 gibba011 bash[30791]: cluster 2021-03-10T14:52:33.014047+0000 mgr.gibba032.whopeo (mgr.24625) 3186 : cluster [DBG] pgmap v1802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:34 gibba011 bash[31774]: cephadm 2021-03-10T14:52:32.933001+0000 mgr.gibba032.whopeo (mgr.24625) 3185 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:34 gibba011 bash[31774]: cluster 2021-03-10T14:52:33.014047+0000 mgr.gibba032.whopeo (mgr.24625) 3186 : cluster [DBG] pgmap v1802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:36 gibba032 bash[24004]: cluster 2021-03-10T14:52:35.015242+0000 mgr.gibba032.whopeo (mgr.24625) 3187 : cluster [DBG] pgmap v1803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:36 gibba011 bash[30791]: cluster 2021-03-10T14:52:35.015242+0000 mgr.gibba032.whopeo (mgr.24625) 3187 : cluster [DBG] pgmap v1803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:36 gibba011 bash[31774]: cluster 2021-03-10T14:52:35.015242+0000 mgr.gibba032.whopeo (mgr.24625) 3187 : cluster [DBG] pgmap v1803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:37 gibba032 bash[24004]: cluster 2021-03-10T14:52:37.016890+0000 mgr.gibba032.whopeo (mgr.24625) 3188 : cluster [DBG] pgmap v1804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:37 gibba011 bash[30791]: cluster 2021-03-10T14:52:37.016890+0000 mgr.gibba032.whopeo (mgr.24625) 3188 : cluster [DBG] pgmap v1804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:37 gibba011 bash[31774]: cluster 2021-03-10T14:52:37.016890+0000 mgr.gibba032.whopeo (mgr.24625) 3188 : cluster [DBG] pgmap v1804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:40 gibba032 bash[24004]: cluster 2021-03-10T14:52:39.018234+0000 mgr.gibba032.whopeo (mgr.24625) 3189 : cluster [DBG] pgmap v1805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:40 gibba011 bash[30791]: cluster 2021-03-10T14:52:39.018234+0000 mgr.gibba032.whopeo (mgr.24625) 3189 : cluster [DBG] pgmap v1805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:40 gibba011 bash[31774]: cluster 2021-03-10T14:52:39.018234+0000 mgr.gibba032.whopeo (mgr.24625) 3189 : cluster [DBG] pgmap v1805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:42 gibba032 bash[24004]: cluster 2021-03-10T14:52:41.019560+0000 mgr.gibba032.whopeo (mgr.24625) 3190 : cluster [DBG] pgmap v1806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:42 gibba011 bash[30791]: cluster 2021-03-10T14:52:41.019560+0000 mgr.gibba032.whopeo (mgr.24625) 3190 : cluster [DBG] pgmap v1806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:42 gibba011 bash[31774]: cluster 2021-03-10T14:52:41.019560+0000 mgr.gibba032.whopeo (mgr.24625) 3190 : cluster [DBG] pgmap v1806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:44 gibba032 bash[24004]: cluster 2021-03-10T14:52:43.020286+0000 mgr.gibba032.whopeo (mgr.24625) 3191 : cluster [DBG] pgmap v1807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:44 gibba011 bash[30791]: cluster 2021-03-10T14:52:43.020286+0000 mgr.gibba032.whopeo (mgr.24625) 3191 : cluster [DBG] pgmap v1807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:44 gibba011 bash[31774]: cluster 2021-03-10T14:52:43.020286+0000 mgr.gibba032.whopeo (mgr.24625) 3191 : cluster [DBG] pgmap v1807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:46 gibba032 bash[24004]: cluster 2021-03-10T14:52:45.021559+0000 mgr.gibba032.whopeo (mgr.24625) 3192 : cluster [DBG] pgmap v1808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:46 gibba011 bash[30791]: cluster 2021-03-10T14:52:45.021559+0000 mgr.gibba032.whopeo (mgr.24625) 3192 : cluster [DBG] pgmap v1808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:46 gibba011 bash[31774]: cluster 2021-03-10T14:52:45.021559+0000 mgr.gibba032.whopeo (mgr.24625) 3192 : cluster [DBG] pgmap v1808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:47.946 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:52:47.954 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:47 gibba032 bash[24004]: cluster 2021-03-10T14:52:47.023279+0000 mgr.gibba032.whopeo (mgr.24625) 3193 : cluster [DBG] pgmap v1809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:47 gibba011 bash[30791]: cluster 2021-03-10T14:52:47.023279+0000 mgr.gibba032.whopeo (mgr.24625) 3193 : cluster [DBG] pgmap v1809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:47 gibba011 bash[31774]: cluster 2021-03-10T14:52:47.023279+0000 mgr.gibba032.whopeo (mgr.24625) 3193 : cluster [DBG] pgmap v1809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:48.209 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:52:48.210 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (59m) 4m ago 64m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:52:48.210 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (64m) 3m ago 64m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:52:48.210 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (59m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:52:48.210 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (60m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:52:48.211 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (59m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:52:48.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (58m) 4m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:52:48.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (58m) 3m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:52:48.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (58m) 4m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:52:48.211 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (64m) 4m ago 64m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:52:48.212 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (64m) 3m ago 64m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:52:48.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (57m) 4m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:52:48.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (57m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:52:48.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (56m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:52:48.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (56m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:52:48.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (56m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:52:48.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (55m) 3m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:52:48.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (55m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:52:48.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (54m) 3m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:52:48.214 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (64m) 3m ago 64m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:52:48.214 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 62m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:52:48.459 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:52:48.460 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:52:48.460 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:52:48.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:48.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:52:48.461 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:52:48.461 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:48.461 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:52:48.461 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:52:48.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:52:48.462 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:52:48.462 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:52:48.462 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:52:48.463 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:52:48.463 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:52:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:48 gibba032 bash[24004]: audit 2021-03-10T14:52:47.932490+0000 mgr.gibba032.whopeo (mgr.24625) 3194 : audit [DBG] from='client.56182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:48 gibba032 bash[24004]: cephadm 2021-03-10T14:52:47.948912+0000 mgr.gibba032.whopeo (mgr.24625) 3195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:48 gibba032 bash[24004]: audit 2021-03-10T14:52:48.165868+0000 mgr.gibba032.whopeo (mgr.24625) 3196 : audit [DBG] from='client.46203 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:48 gibba032 bash[24004]: audit 2021-03-10T14:52:48.457710+0000 mon.c (mon.1) 61 : audit [DBG] from='client.? 172.21.2.111:0/314466112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[30791]: audit 2021-03-10T14:52:47.932490+0000 mgr.gibba032.whopeo (mgr.24625) 3194 : audit [DBG] from='client.56182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[30791]: cephadm 2021-03-10T14:52:47.948912+0000 mgr.gibba032.whopeo (mgr.24625) 3195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[30791]: audit 2021-03-10T14:52:48.165868+0000 mgr.gibba032.whopeo (mgr.24625) 3196 : audit [DBG] from='client.46203 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[30791]: audit 2021-03-10T14:52:48.457710+0000 mon.c (mon.1) 61 : audit [DBG] from='client.? 172.21.2.111:0/314466112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:49.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[31774]: audit 2021-03-10T14:52:47.932490+0000 mgr.gibba032.whopeo (mgr.24625) 3194 : audit [DBG] from='client.56182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[31774]: cephadm 2021-03-10T14:52:47.948912+0000 mgr.gibba032.whopeo (mgr.24625) 3195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:52:49.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[31774]: audit 2021-03-10T14:52:48.165868+0000 mgr.gibba032.whopeo (mgr.24625) 3196 : audit [DBG] from='client.46203 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:52:49.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:48 gibba011 bash[31774]: audit 2021-03-10T14:52:48.457710+0000 mon.c (mon.1) 61 : audit [DBG] from='client.? 172.21.2.111:0/314466112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:52:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:49 gibba032 bash[24004]: cluster 2021-03-10T14:52:49.024828+0000 mgr.gibba032.whopeo (mgr.24625) 3197 : cluster [DBG] pgmap v1810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:49 gibba011 bash[30791]: cluster 2021-03-10T14:52:49.024828+0000 mgr.gibba032.whopeo (mgr.24625) 3197 : cluster [DBG] pgmap v1810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:49 gibba011 bash[31774]: cluster 2021-03-10T14:52:49.024828+0000 mgr.gibba032.whopeo (mgr.24625) 3197 : cluster [DBG] pgmap v1810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:52 gibba032 bash[24004]: cluster 2021-03-10T14:52:51.026163+0000 mgr.gibba032.whopeo (mgr.24625) 3198 : cluster [DBG] pgmap v1811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:52 gibba011 bash[30791]: cluster 2021-03-10T14:52:51.026163+0000 mgr.gibba032.whopeo (mgr.24625) 3198 : cluster [DBG] pgmap v1811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:52 gibba011 bash[31774]: cluster 2021-03-10T14:52:51.026163+0000 mgr.gibba032.whopeo (mgr.24625) 3198 : cluster [DBG] pgmap v1811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:54 gibba032 bash[24004]: cluster 2021-03-10T14:52:53.028005+0000 mgr.gibba032.whopeo (mgr.24625) 3199 : cluster [DBG] pgmap v1812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:54 gibba011 bash[30791]: cluster 2021-03-10T14:52:53.028005+0000 mgr.gibba032.whopeo (mgr.24625) 3199 : cluster [DBG] pgmap v1812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:54 gibba011 bash[31774]: cluster 2021-03-10T14:52:53.028005+0000 mgr.gibba032.whopeo (mgr.24625) 3199 : cluster [DBG] pgmap v1812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:56 gibba032 bash[24004]: cluster 2021-03-10T14:52:55.029089+0000 mgr.gibba032.whopeo (mgr.24625) 3200 : cluster [DBG] pgmap v1813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:56 gibba011 bash[30791]: cluster 2021-03-10T14:52:55.029089+0000 mgr.gibba032.whopeo (mgr.24625) 3200 : cluster [DBG] pgmap v1813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:56 gibba011 bash[31774]: cluster 2021-03-10T14:52:55.029089+0000 mgr.gibba032.whopeo (mgr.24625) 3200 : cluster [DBG] pgmap v1813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.398535+0000 mon.a (mon.0) 1469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:52:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.399216+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:52:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.416167+0000 mon.a (mon.0) 1471 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:52:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.416932+0000 mon.a (mon.0) 1472 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:52:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.417615+0000 mon.a (mon.0) 1473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:52:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:57 gibba032 bash[24004]: audit 2021-03-10T14:52:56.418282+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:52:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.398535+0000 mon.a (mon.0) 1469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:52:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.399216+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:52:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.416167+0000 mon.a (mon.0) 1471 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:52:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.416932+0000 mon.a (mon.0) 1472 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:52:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.417615+0000 mon.a (mon.0) 1473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:52:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[30791]: audit 2021-03-10T14:52:56.418282+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:52:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.398535+0000 mon.a (mon.0) 1469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:52:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.399216+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:52:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.416167+0000 mon.a (mon.0) 1471 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:52:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.416932+0000 mon.a (mon.0) 1472 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:52:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.417615+0000 mon.a (mon.0) 1473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T14:52:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:57 gibba011 bash[31774]: audit 2021-03-10T14:52:56.418282+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:52:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:52:58 gibba032 bash[24004]: cluster 2021-03-10T14:52:57.030794+0000 mgr.gibba032.whopeo (mgr.24625) 3201 : cluster [DBG] pgmap v1814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:52:58 gibba011 bash[30791]: cluster 2021-03-10T14:52:57.030794+0000 mgr.gibba032.whopeo (mgr.24625) 3201 : cluster [DBG] pgmap v1814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:52:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:52:58 gibba011 bash[31774]: cluster 2021-03-10T14:52:57.030794+0000 mgr.gibba032.whopeo (mgr.24625) 3201 : cluster [DBG] pgmap v1814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:00 gibba032 bash[24004]: cluster 2021-03-10T14:52:59.032383+0000 mgr.gibba032.whopeo (mgr.24625) 3202 : cluster [DBG] pgmap v1815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:00 gibba011 bash[30791]: cluster 2021-03-10T14:52:59.032383+0000 mgr.gibba032.whopeo (mgr.24625) 3202 : cluster [DBG] pgmap v1815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:00 gibba011 bash[31774]: cluster 2021-03-10T14:52:59.032383+0000 mgr.gibba032.whopeo (mgr.24625) 3202 : cluster [DBG] pgmap v1815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:02 gibba032 bash[24004]: cluster 2021-03-10T14:53:01.033733+0000 mgr.gibba032.whopeo (mgr.24625) 3203 : cluster [DBG] pgmap v1816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:02 gibba011 bash[30791]: cluster 2021-03-10T14:53:01.033733+0000 mgr.gibba032.whopeo (mgr.24625) 3203 : cluster [DBG] pgmap v1816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:02 gibba011 bash[31774]: cluster 2021-03-10T14:53:01.033733+0000 mgr.gibba032.whopeo (mgr.24625) 3203 : cluster [DBG] pgmap v1816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:04 gibba032 bash[24004]: cephadm 2021-03-10T14:53:02.964966+0000 mgr.gibba032.whopeo (mgr.24625) 3204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:04 gibba032 bash[24004]: cluster 2021-03-10T14:53:03.035373+0000 mgr.gibba032.whopeo (mgr.24625) 3205 : cluster [DBG] pgmap v1817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:04 gibba011 bash[30791]: cephadm 2021-03-10T14:53:02.964966+0000 mgr.gibba032.whopeo (mgr.24625) 3204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:04 gibba011 bash[30791]: cluster 2021-03-10T14:53:03.035373+0000 mgr.gibba032.whopeo (mgr.24625) 3205 : cluster [DBG] pgmap v1817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:04 gibba011 bash[31774]: cephadm 2021-03-10T14:53:02.964966+0000 mgr.gibba032.whopeo (mgr.24625) 3204 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:04 gibba011 bash[31774]: cluster 2021-03-10T14:53:03.035373+0000 mgr.gibba032.whopeo (mgr.24625) 3205 : cluster [DBG] pgmap v1817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:06 gibba032 bash[24004]: cluster 2021-03-10T14:53:05.036562+0000 mgr.gibba032.whopeo (mgr.24625) 3206 : cluster [DBG] pgmap v1818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:06 gibba011 bash[30791]: cluster 2021-03-10T14:53:05.036562+0000 mgr.gibba032.whopeo (mgr.24625) 3206 : cluster [DBG] pgmap v1818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:06 gibba011 bash[31774]: cluster 2021-03-10T14:53:05.036562+0000 mgr.gibba032.whopeo (mgr.24625) 3206 : cluster [DBG] pgmap v1818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:07 gibba032 bash[24004]: cluster 2021-03-10T14:53:07.038309+0000 mgr.gibba032.whopeo (mgr.24625) 3207 : cluster [DBG] pgmap v1819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:07 gibba011 bash[30791]: cluster 2021-03-10T14:53:07.038309+0000 mgr.gibba032.whopeo (mgr.24625) 3207 : cluster [DBG] pgmap v1819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:07 gibba011 bash[31774]: cluster 2021-03-10T14:53:07.038309+0000 mgr.gibba032.whopeo (mgr.24625) 3207 : cluster [DBG] pgmap v1819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:10 gibba032 bash[24004]: cluster 2021-03-10T14:53:09.039874+0000 mgr.gibba032.whopeo (mgr.24625) 3208 : cluster [DBG] pgmap v1820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:10 gibba011 bash[30791]: cluster 2021-03-10T14:53:09.039874+0000 mgr.gibba032.whopeo (mgr.24625) 3208 : cluster [DBG] pgmap v1820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:10 gibba011 bash[31774]: cluster 2021-03-10T14:53:09.039874+0000 mgr.gibba032.whopeo (mgr.24625) 3208 : cluster [DBG] pgmap v1820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:12 gibba032 bash[24004]: cluster 2021-03-10T14:53:11.041279+0000 mgr.gibba032.whopeo (mgr.24625) 3209 : cluster [DBG] pgmap v1821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:12 gibba011 bash[30791]: cluster 2021-03-10T14:53:11.041279+0000 mgr.gibba032.whopeo (mgr.24625) 3209 : cluster [DBG] pgmap v1821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:12 gibba011 bash[31774]: cluster 2021-03-10T14:53:11.041279+0000 mgr.gibba032.whopeo (mgr.24625) 3209 : cluster [DBG] pgmap v1821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:14 gibba032 bash[24004]: cluster 2021-03-10T14:53:13.042014+0000 mgr.gibba032.whopeo (mgr.24625) 3210 : cluster [DBG] pgmap v1822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:14 gibba011 bash[30791]: cluster 2021-03-10T14:53:13.042014+0000 mgr.gibba032.whopeo (mgr.24625) 3210 : cluster [DBG] pgmap v1822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:14.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:14 gibba011 bash[31774]: cluster 2021-03-10T14:53:13.042014+0000 mgr.gibba032.whopeo (mgr.24625) 3210 : cluster [DBG] pgmap v1822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:16 gibba032 bash[24004]: cluster 2021-03-10T14:53:15.043016+0000 mgr.gibba032.whopeo (mgr.24625) 3211 : cluster [DBG] pgmap v1823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:16 gibba011 bash[30791]: cluster 2021-03-10T14:53:15.043016+0000 mgr.gibba032.whopeo (mgr.24625) 3211 : cluster [DBG] pgmap v1823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:16 gibba011 bash[31774]: cluster 2021-03-10T14:53:15.043016+0000 mgr.gibba032.whopeo (mgr.24625) 3211 : cluster [DBG] pgmap v1823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:17 gibba032 bash[24004]: cluster 2021-03-10T14:53:17.044784+0000 mgr.gibba032.whopeo (mgr.24625) 3212 : cluster [DBG] pgmap v1824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:17 gibba011 bash[30791]: cluster 2021-03-10T14:53:17.044784+0000 mgr.gibba032.whopeo (mgr.24625) 3212 : cluster [DBG] pgmap v1824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:17 gibba011 bash[31774]: cluster 2021-03-10T14:53:17.044784+0000 mgr.gibba032.whopeo (mgr.24625) 3212 : cluster [DBG] pgmap v1824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:18.702 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:53:18.927 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:53:18.927 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (59m) 5m ago 65m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:53:18.927 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (65m) 4m ago 65m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:53:18.928 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (60m) 4m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:53:18.928 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (60m) 4m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:53:18.928 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (59m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:53:18.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (59m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:53:18.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (59m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:53:18.929 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (59m) 5m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:53:18.929 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (65m) 5m ago 65m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:53:18.929 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (65m) 4m ago 65m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:53:18.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (58m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:53:18.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (57m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:53:18.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (57m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:53:18.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (56m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:53:18.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (56m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:53:18.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (56m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:53:18.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (55m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:53:18.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (55m) 4m ago 65m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:53:18.931 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (64m) 4m ago 65m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:53:18.931 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 63m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:53:19.175 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:53:19.176 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:53:19.176 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:53:19.176 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:19.176 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:19.177 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:53:19.178 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:53:19.178 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:53:19.178 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:53:19.178 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:53:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:17.974805+0000 mgr.gibba032.whopeo (mgr.24625) 3213 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:53:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:17.976351+0000 mgr.gibba032.whopeo (mgr.24625) 3214 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:53:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.185476+0000 mgr.gibba032.whopeo (mgr.24625) 3215 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:53:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.185654+0000 mon.a (mon.0) 1475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:53:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.187528+0000 mgr.gibba032.whopeo (mgr.24625) 3216 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:53:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.188941+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:53:19.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.189227+0000 mgr.gibba032.whopeo (mgr.24625) 3217 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:53:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.189267+0000 mgr.gibba032.whopeo (mgr.24625) 3218 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:53:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.190625+0000 mon.a (mon.0) 1477 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.190660+0000 mgr.gibba032.whopeo (mgr.24625) 3219 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:53:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.190695+0000 mgr.gibba032.whopeo (mgr.24625) 3220 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:53:19.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.192163+0000 mgr.gibba032.whopeo (mgr.24625) 3221 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:53:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.192407+0000 mon.a (mon.0) 1478 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.193758+0000 mon.a (mon.0) 1479 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.193851+0000 mgr.gibba032.whopeo (mgr.24625) 3222 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:53:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.193885+0000 mgr.gibba032.whopeo (mgr.24625) 3223 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:53:19.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.195315+0000 mgr.gibba032.whopeo (mgr.24625) 3224 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:53:19.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.195703+0000 mgr.gibba032.whopeo (mgr.24625) 3225 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:53:19.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.197077+0000 mon.a (mon.0) 1480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.197380+0000 mgr.gibba032.whopeo (mgr.24625) 3226 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:53:19.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.197418+0000 mgr.gibba032.whopeo (mgr.24625) 3227 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:53:19.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: cephadm 2021-03-10T14:53:18.197569+0000 mgr.gibba032.whopeo (mgr.24625) 3228 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:19.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:18.198730+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:19 gibba032 bash[24004]: audit 2021-03-10T14:53:19.173547+0000 mon.c (mon.1) 62 : audit [DBG] from='client.? 172.21.2.111:0/3587582942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:17.974805+0000 mgr.gibba032.whopeo (mgr.24625) 3213 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:53:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:17.976351+0000 mgr.gibba032.whopeo (mgr.24625) 3214 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:53:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.185476+0000 mgr.gibba032.whopeo (mgr.24625) 3215 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:53:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.185654+0000 mon.a (mon.0) 1475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:53:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.187528+0000 mgr.gibba032.whopeo (mgr.24625) 3216 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:53:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.188941+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:53:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.189227+0000 mgr.gibba032.whopeo (mgr.24625) 3217 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:53:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.189267+0000 mgr.gibba032.whopeo (mgr.24625) 3218 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:53:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.190625+0000 mon.a (mon.0) 1477 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.190660+0000 mgr.gibba032.whopeo (mgr.24625) 3219 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:53:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.190695+0000 mgr.gibba032.whopeo (mgr.24625) 3220 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:53:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.192163+0000 mgr.gibba032.whopeo (mgr.24625) 3221 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:53:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.192407+0000 mon.a (mon.0) 1478 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.193758+0000 mon.a (mon.0) 1479 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.193851+0000 mgr.gibba032.whopeo (mgr.24625) 3222 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:53:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.193885+0000 mgr.gibba032.whopeo (mgr.24625) 3223 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:53:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.195315+0000 mgr.gibba032.whopeo (mgr.24625) 3224 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:53:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.195703+0000 mgr.gibba032.whopeo (mgr.24625) 3225 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:53:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.197077+0000 mon.a (mon.0) 1480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.197380+0000 mgr.gibba032.whopeo (mgr.24625) 3226 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:53:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.197418+0000 mgr.gibba032.whopeo (mgr.24625) 3227 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:53:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: cephadm 2021-03-10T14:53:18.197569+0000 mgr.gibba032.whopeo (mgr.24625) 3228 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:18.198730+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[30791]: audit 2021-03-10T14:53:19.173547+0000 mon.c (mon.1) 62 : audit [DBG] from='client.? 172.21.2.111:0/3587582942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:17.974805+0000 mgr.gibba032.whopeo (mgr.24625) 3213 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:53:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:17.976351+0000 mgr.gibba032.whopeo (mgr.24625) 3214 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:53:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.185476+0000 mgr.gibba032.whopeo (mgr.24625) 3215 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:53:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.185654+0000 mon.a (mon.0) 1475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:53:19.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.187528+0000 mgr.gibba032.whopeo (mgr.24625) 3216 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:53:19.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.188941+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:53:19.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.189227+0000 mgr.gibba032.whopeo (mgr.24625) 3217 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:53:19.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.189267+0000 mgr.gibba032.whopeo (mgr.24625) 3218 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:53:19.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.190625+0000 mon.a (mon.0) 1477 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.190660+0000 mgr.gibba032.whopeo (mgr.24625) 3219 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:53:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.190695+0000 mgr.gibba032.whopeo (mgr.24625) 3220 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:53:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.192163+0000 mgr.gibba032.whopeo (mgr.24625) 3221 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:53:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.192407+0000 mon.a (mon.0) 1478 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.193758+0000 mon.a (mon.0) 1479 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.193851+0000 mgr.gibba032.whopeo (mgr.24625) 3222 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:53:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.193885+0000 mgr.gibba032.whopeo (mgr.24625) 3223 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:53:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.195315+0000 mgr.gibba032.whopeo (mgr.24625) 3224 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:53:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.195703+0000 mgr.gibba032.whopeo (mgr.24625) 3225 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:53:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.197077+0000 mon.a (mon.0) 1480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.197380+0000 mgr.gibba032.whopeo (mgr.24625) 3226 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:53:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.197418+0000 mgr.gibba032.whopeo (mgr.24625) 3227 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:53:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: cephadm 2021-03-10T14:53:18.197569+0000 mgr.gibba032.whopeo (mgr.24625) 3228 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:18.198730+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:19 gibba011 bash[31774]: audit 2021-03-10T14:53:19.173547+0000 mon.c (mon.1) 62 : audit [DBG] from='client.? 172.21.2.111:0/3587582942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:20 gibba032 bash[24004]: audit 2021-03-10T14:53:18.688003+0000 mgr.gibba032.whopeo (mgr.24625) 3229 : audit [DBG] from='client.56200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:20 gibba032 bash[24004]: audit 2021-03-10T14:53:18.911177+0000 mgr.gibba032.whopeo (mgr.24625) 3230 : audit [DBG] from='client.56206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:20 gibba032 bash[24004]: cluster 2021-03-10T14:53:19.046394+0000 mgr.gibba032.whopeo (mgr.24625) 3231 : cluster [DBG] pgmap v1825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[30791]: audit 2021-03-10T14:53:18.688003+0000 mgr.gibba032.whopeo (mgr.24625) 3229 : audit [DBG] from='client.56200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[30791]: audit 2021-03-10T14:53:18.911177+0000 mgr.gibba032.whopeo (mgr.24625) 3230 : audit [DBG] from='client.56206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[30791]: cluster 2021-03-10T14:53:19.046394+0000 mgr.gibba032.whopeo (mgr.24625) 3231 : cluster [DBG] pgmap v1825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[31774]: audit 2021-03-10T14:53:18.688003+0000 mgr.gibba032.whopeo (mgr.24625) 3229 : audit [DBG] from='client.56200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[31774]: audit 2021-03-10T14:53:18.911177+0000 mgr.gibba032.whopeo (mgr.24625) 3230 : audit [DBG] from='client.56206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:20 gibba011 bash[31774]: cluster 2021-03-10T14:53:19.046394+0000 mgr.gibba032.whopeo (mgr.24625) 3231 : cluster [DBG] pgmap v1825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:22 gibba032 bash[24004]: cluster 2021-03-10T14:53:21.047723+0000 mgr.gibba032.whopeo (mgr.24625) 3232 : cluster [DBG] pgmap v1826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:22 gibba011 bash[30791]: cluster 2021-03-10T14:53:21.047723+0000 mgr.gibba032.whopeo (mgr.24625) 3232 : cluster [DBG] pgmap v1826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:22 gibba011 bash[31774]: cluster 2021-03-10T14:53:21.047723+0000 mgr.gibba032.whopeo (mgr.24625) 3232 : cluster [DBG] pgmap v1826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:24 gibba032 bash[24004]: cluster 2021-03-10T14:53:23.048710+0000 mgr.gibba032.whopeo (mgr.24625) 3233 : cluster [DBG] pgmap v1827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:24 gibba011 bash[30791]: cluster 2021-03-10T14:53:23.048710+0000 mgr.gibba032.whopeo (mgr.24625) 3233 : cluster [DBG] pgmap v1827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:24 gibba011 bash[31774]: cluster 2021-03-10T14:53:23.048710+0000 mgr.gibba032.whopeo (mgr.24625) 3233 : cluster [DBG] pgmap v1827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:26 gibba032 bash[24004]: cluster 2021-03-10T14:53:25.049977+0000 mgr.gibba032.whopeo (mgr.24625) 3234 : cluster [DBG] pgmap v1828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:26 gibba011 bash[30791]: cluster 2021-03-10T14:53:25.049977+0000 mgr.gibba032.whopeo (mgr.24625) 3234 : cluster [DBG] pgmap v1828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:26 gibba011 bash[31774]: cluster 2021-03-10T14:53:25.049977+0000 mgr.gibba032.whopeo (mgr.24625) 3234 : cluster [DBG] pgmap v1828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:27 gibba032 bash[24004]: cluster 2021-03-10T14:53:27.051696+0000 mgr.gibba032.whopeo (mgr.24625) 3235 : cluster [DBG] pgmap v1829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:27 gibba011 bash[30791]: cluster 2021-03-10T14:53:27.051696+0000 mgr.gibba032.whopeo (mgr.24625) 3235 : cluster [DBG] pgmap v1829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:27 gibba011 bash[31774]: cluster 2021-03-10T14:53:27.051696+0000 mgr.gibba032.whopeo (mgr.24625) 3235 : cluster [DBG] pgmap v1829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:30 gibba032 bash[24004]: cluster 2021-03-10T14:53:29.053284+0000 mgr.gibba032.whopeo (mgr.24625) 3236 : cluster [DBG] pgmap v1830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:30 gibba011 bash[30791]: cluster 2021-03-10T14:53:29.053284+0000 mgr.gibba032.whopeo (mgr.24625) 3236 : cluster [DBG] pgmap v1830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:30 gibba011 bash[31774]: cluster 2021-03-10T14:53:29.053284+0000 mgr.gibba032.whopeo (mgr.24625) 3236 : cluster [DBG] pgmap v1830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:32 gibba032 bash[24004]: cluster 2021-03-10T14:53:31.054907+0000 mgr.gibba032.whopeo (mgr.24625) 3237 : cluster [DBG] pgmap v1831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:32 gibba011 bash[30791]: cluster 2021-03-10T14:53:31.054907+0000 mgr.gibba032.whopeo (mgr.24625) 3237 : cluster [DBG] pgmap v1831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:32 gibba011 bash[31774]: cluster 2021-03-10T14:53:31.054907+0000 mgr.gibba032.whopeo (mgr.24625) 3237 : cluster [DBG] pgmap v1831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:34 gibba032 bash[24004]: cluster 2021-03-10T14:53:33.055847+0000 mgr.gibba032.whopeo (mgr.24625) 3238 : cluster [DBG] pgmap v1832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:34 gibba032 bash[24004]: cephadm 2021-03-10T14:53:33.198900+0000 mgr.gibba032.whopeo (mgr.24625) 3239 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:34 gibba011 bash[30791]: cluster 2021-03-10T14:53:33.055847+0000 mgr.gibba032.whopeo (mgr.24625) 3238 : cluster [DBG] pgmap v1832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:34 gibba011 bash[30791]: cephadm 2021-03-10T14:53:33.198900+0000 mgr.gibba032.whopeo (mgr.24625) 3239 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:34 gibba011 bash[31774]: cluster 2021-03-10T14:53:33.055847+0000 mgr.gibba032.whopeo (mgr.24625) 3238 : cluster [DBG] pgmap v1832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:34 gibba011 bash[31774]: cephadm 2021-03-10T14:53:33.198900+0000 mgr.gibba032.whopeo (mgr.24625) 3239 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:36 gibba032 bash[24004]: cluster 2021-03-10T14:53:35.057081+0000 mgr.gibba032.whopeo (mgr.24625) 3240 : cluster [DBG] pgmap v1833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:36 gibba011 bash[30791]: cluster 2021-03-10T14:53:35.057081+0000 mgr.gibba032.whopeo (mgr.24625) 3240 : cluster [DBG] pgmap v1833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:36 gibba011 bash[31774]: cluster 2021-03-10T14:53:35.057081+0000 mgr.gibba032.whopeo (mgr.24625) 3240 : cluster [DBG] pgmap v1833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:37 gibba032 bash[24004]: cluster 2021-03-10T14:53:37.058835+0000 mgr.gibba032.whopeo (mgr.24625) 3241 : cluster [DBG] pgmap v1834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:37 gibba011 bash[30791]: cluster 2021-03-10T14:53:37.058835+0000 mgr.gibba032.whopeo (mgr.24625) 3241 : cluster [DBG] pgmap v1834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:37 gibba011 bash[31774]: cluster 2021-03-10T14:53:37.058835+0000 mgr.gibba032.whopeo (mgr.24625) 3241 : cluster [DBG] pgmap v1834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:40 gibba032 bash[24004]: cluster 2021-03-10T14:53:39.060499+0000 mgr.gibba032.whopeo (mgr.24625) 3242 : cluster [DBG] pgmap v1835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:40 gibba011 bash[30791]: cluster 2021-03-10T14:53:39.060499+0000 mgr.gibba032.whopeo (mgr.24625) 3242 : cluster [DBG] pgmap v1835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:40 gibba011 bash[31774]: cluster 2021-03-10T14:53:39.060499+0000 mgr.gibba032.whopeo (mgr.24625) 3242 : cluster [DBG] pgmap v1835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:42 gibba032 bash[24004]: cluster 2021-03-10T14:53:41.061925+0000 mgr.gibba032.whopeo (mgr.24625) 3243 : cluster [DBG] pgmap v1836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:42 gibba011 bash[30791]: cluster 2021-03-10T14:53:41.061925+0000 mgr.gibba032.whopeo (mgr.24625) 3243 : cluster [DBG] pgmap v1836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:42 gibba011 bash[31774]: cluster 2021-03-10T14:53:41.061925+0000 mgr.gibba032.whopeo (mgr.24625) 3243 : cluster [DBG] pgmap v1836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:44 gibba032 bash[24004]: cluster 2021-03-10T14:53:43.063391+0000 mgr.gibba032.whopeo (mgr.24625) 3244 : cluster [DBG] pgmap v1837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:44 gibba011 bash[30791]: cluster 2021-03-10T14:53:43.063391+0000 mgr.gibba032.whopeo (mgr.24625) 3244 : cluster [DBG] pgmap v1837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:44 gibba011 bash[31774]: cluster 2021-03-10T14:53:43.063391+0000 mgr.gibba032.whopeo (mgr.24625) 3244 : cluster [DBG] pgmap v1837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:46 gibba032 bash[24004]: cluster 2021-03-10T14:53:45.064742+0000 mgr.gibba032.whopeo (mgr.24625) 3245 : cluster [DBG] pgmap v1838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:46 gibba011 bash[30791]: cluster 2021-03-10T14:53:45.064742+0000 mgr.gibba032.whopeo (mgr.24625) 3245 : cluster [DBG] pgmap v1838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:46 gibba011 bash[31774]: cluster 2021-03-10T14:53:45.064742+0000 mgr.gibba032.whopeo (mgr.24625) 3245 : cluster [DBG] pgmap v1838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:47 gibba032 bash[24004]: cluster 2021-03-10T14:53:47.066211+0000 mgr.gibba032.whopeo (mgr.24625) 3246 : cluster [DBG] pgmap v1839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:47 gibba011 bash[30791]: cluster 2021-03-10T14:53:47.066211+0000 mgr.gibba032.whopeo (mgr.24625) 3246 : cluster [DBG] pgmap v1839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:47 gibba011 bash[31774]: cluster 2021-03-10T14:53:47.066211+0000 mgr.gibba032.whopeo (mgr.24625) 3246 : cluster [DBG] pgmap v1839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:48 gibba032 bash[24004]: cephadm 2021-03-10T14:53:48.212791+0000 mgr.gibba032.whopeo (mgr.24625) 3247 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:48 gibba011 bash[30791]: cephadm 2021-03-10T14:53:48.212791+0000 mgr.gibba032.whopeo (mgr.24625) 3247 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:49.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:48 gibba011 bash[31774]: cephadm 2021-03-10T14:53:48.212791+0000 mgr.gibba032.whopeo (mgr.24625) 3247 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:53:49.420 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:53:49.646 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:53:49.647 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (60m) 5m ago 65m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:53:49.647 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (65m) 4m ago 65m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:53:49.647 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (60m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:53:49.647 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (61m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:53:49.647 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (60m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:53:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (60m) 5m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:53:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (59m) 4m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:53:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (59m) 5m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:53:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (65m) 5m ago 65m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:53:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (65m) 4m ago 65m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:53:49.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (58m) 5m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:53:49.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (58m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:53:49.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (57m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:53:49.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (57m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:53:49.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (57m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:53:49.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (56m) 4m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:53:49.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (56m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:53:49.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (55m) 4m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:53:49.650 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (65m) 4m ago 66m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:53:49.650 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 63m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:53:49.899 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:53:49.899 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:53:49.899 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:53:49.900 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:49.900 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:53:49.900 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:53:49.900 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:49.901 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:53:49.901 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:53:49.901 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:53:49.901 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:53:49.901 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:53:49.902 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:53:49.902 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:53:49.902 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:53:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:49 gibba032 bash[24004]: cluster 2021-03-10T14:53:49.067685+0000 mgr.gibba032.whopeo (mgr.24625) 3248 : cluster [DBG] pgmap v1840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:49 gibba032 bash[24004]: audit 2021-03-10T14:53:49.406470+0000 mgr.gibba032.whopeo (mgr.24625) 3249 : audit [DBG] from='client.46233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:49 gibba011 bash[30791]: cluster 2021-03-10T14:53:49.067685+0000 mgr.gibba032.whopeo (mgr.24625) 3248 : cluster [DBG] pgmap v1840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:49 gibba011 bash[30791]: audit 2021-03-10T14:53:49.406470+0000 mgr.gibba032.whopeo (mgr.24625) 3249 : audit [DBG] from='client.46233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:49 gibba011 bash[31774]: cluster 2021-03-10T14:53:49.067685+0000 mgr.gibba032.whopeo (mgr.24625) 3248 : cluster [DBG] pgmap v1840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:49 gibba011 bash[31774]: audit 2021-03-10T14:53:49.406470+0000 mgr.gibba032.whopeo (mgr.24625) 3249 : audit [DBG] from='client.46233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:50 gibba032 bash[24004]: audit 2021-03-10T14:53:49.630893+0000 mgr.gibba032.whopeo (mgr.24625) 3250 : audit [DBG] from='client.46239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:50 gibba032 bash[24004]: audit 2021-03-10T14:53:49.896964+0000 mon.a (mon.0) 1482 : audit [DBG] from='client.? 172.21.2.111:0/1794405965' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:50 gibba011 bash[30791]: audit 2021-03-10T14:53:49.630893+0000 mgr.gibba032.whopeo (mgr.24625) 3250 : audit [DBG] from='client.46239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:50 gibba011 bash[30791]: audit 2021-03-10T14:53:49.896964+0000 mon.a (mon.0) 1482 : audit [DBG] from='client.? 172.21.2.111:0/1794405965' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:51.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:50 gibba011 bash[31774]: audit 2021-03-10T14:53:49.630893+0000 mgr.gibba032.whopeo (mgr.24625) 3250 : audit [DBG] from='client.46239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:53:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:50 gibba011 bash[31774]: audit 2021-03-10T14:53:49.896964+0000 mon.a (mon.0) 1482 : audit [DBG] from='client.? 172.21.2.111:0/1794405965' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:53:52.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:51 gibba032 bash[24004]: cluster 2021-03-10T14:53:51.069207+0000 mgr.gibba032.whopeo (mgr.24625) 3251 : cluster [DBG] pgmap v1841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:52.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:51 gibba011 bash[30791]: cluster 2021-03-10T14:53:51.069207+0000 mgr.gibba032.whopeo (mgr.24625) 3251 : cluster [DBG] pgmap v1841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:52.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:51 gibba011 bash[31774]: cluster 2021-03-10T14:53:51.069207+0000 mgr.gibba032.whopeo (mgr.24625) 3251 : cluster [DBG] pgmap v1841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:54 gibba032 bash[24004]: cluster 2021-03-10T14:53:53.070177+0000 mgr.gibba032.whopeo (mgr.24625) 3252 : cluster [DBG] pgmap v1842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:54 gibba011 bash[30791]: cluster 2021-03-10T14:53:53.070177+0000 mgr.gibba032.whopeo (mgr.24625) 3252 : cluster [DBG] pgmap v1842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:54 gibba011 bash[31774]: cluster 2021-03-10T14:53:53.070177+0000 mgr.gibba032.whopeo (mgr.24625) 3252 : cluster [DBG] pgmap v1842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:56 gibba032 bash[24004]: cluster 2021-03-10T14:53:55.071393+0000 mgr.gibba032.whopeo (mgr.24625) 3253 : cluster [DBG] pgmap v1843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:56 gibba011 bash[30791]: cluster 2021-03-10T14:53:55.071393+0000 mgr.gibba032.whopeo (mgr.24625) 3253 : cluster [DBG] pgmap v1843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:56 gibba011 bash[31774]: cluster 2021-03-10T14:53:55.071393+0000 mgr.gibba032.whopeo (mgr.24625) 3253 : cluster [DBG] pgmap v1843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.404511+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:53:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.405139+0000 mon.a (mon.0) 1484 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:53:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.422227+0000 mon.a (mon.0) 1485 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:53:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.422979+0000 mon.a (mon.0) 1486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.423750+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:57 gibba032 bash[24004]: audit 2021-03-10T14:53:56.424416+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:53:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.404511+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:53:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.405139+0000 mon.a (mon.0) 1484 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:53:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.422227+0000 mon.a (mon.0) 1485 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:53:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.422979+0000 mon.a (mon.0) 1486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.423750+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[30791]: audit 2021-03-10T14:53:56.424416+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:53:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.404511+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:53:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.405139+0000 mon.a (mon.0) 1484 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:53:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.422227+0000 mon.a (mon.0) 1485 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T14:53:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.422979+0000 mon.a (mon.0) 1486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.423750+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T14:53:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:57 gibba011 bash[31774]: audit 2021-03-10T14:53:56.424416+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:53:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:53:58 gibba032 bash[24004]: cluster 2021-03-10T14:53:57.073045+0000 mgr.gibba032.whopeo (mgr.24625) 3254 : cluster [DBG] pgmap v1844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:53:58 gibba011 bash[30791]: cluster 2021-03-10T14:53:57.073045+0000 mgr.gibba032.whopeo (mgr.24625) 3254 : cluster [DBG] pgmap v1844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:53:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:53:58 gibba011 bash[31774]: cluster 2021-03-10T14:53:57.073045+0000 mgr.gibba032.whopeo (mgr.24625) 3254 : cluster [DBG] pgmap v1844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:00 gibba032 bash[24004]: cluster 2021-03-10T14:53:59.074651+0000 mgr.gibba032.whopeo (mgr.24625) 3255 : cluster [DBG] pgmap v1845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:00 gibba011 bash[30791]: cluster 2021-03-10T14:53:59.074651+0000 mgr.gibba032.whopeo (mgr.24625) 3255 : cluster [DBG] pgmap v1845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:00 gibba011 bash[31774]: cluster 2021-03-10T14:53:59.074651+0000 mgr.gibba032.whopeo (mgr.24625) 3255 : cluster [DBG] pgmap v1845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:02 gibba032 bash[24004]: cluster 2021-03-10T14:54:01.076106+0000 mgr.gibba032.whopeo (mgr.24625) 3256 : cluster [DBG] pgmap v1846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:02 gibba011 bash[30791]: cluster 2021-03-10T14:54:01.076106+0000 mgr.gibba032.whopeo (mgr.24625) 3256 : cluster [DBG] pgmap v1846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:02 gibba011 bash[31774]: cluster 2021-03-10T14:54:01.076106+0000 mgr.gibba032.whopeo (mgr.24625) 3256 : cluster [DBG] pgmap v1846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:04 gibba032 bash[24004]: cluster 2021-03-10T14:54:03.077049+0000 mgr.gibba032.whopeo (mgr.24625) 3257 : cluster [DBG] pgmap v1847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:04.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:04 gibba032 bash[24004]: cephadm 2021-03-10T14:54:03.228646+0000 mgr.gibba032.whopeo (mgr.24625) 3258 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:04 gibba011 bash[30791]: cluster 2021-03-10T14:54:03.077049+0000 mgr.gibba032.whopeo (mgr.24625) 3257 : cluster [DBG] pgmap v1847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:04 gibba011 bash[30791]: cephadm 2021-03-10T14:54:03.228646+0000 mgr.gibba032.whopeo (mgr.24625) 3258 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:04 gibba011 bash[31774]: cluster 2021-03-10T14:54:03.077049+0000 mgr.gibba032.whopeo (mgr.24625) 3257 : cluster [DBG] pgmap v1847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:04 gibba011 bash[31774]: cephadm 2021-03-10T14:54:03.228646+0000 mgr.gibba032.whopeo (mgr.24625) 3258 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:06 gibba032 bash[24004]: cluster 2021-03-10T14:54:05.078179+0000 mgr.gibba032.whopeo (mgr.24625) 3259 : cluster [DBG] pgmap v1848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:06 gibba011 bash[30791]: cluster 2021-03-10T14:54:05.078179+0000 mgr.gibba032.whopeo (mgr.24625) 3259 : cluster [DBG] pgmap v1848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:06 gibba011 bash[31774]: cluster 2021-03-10T14:54:05.078179+0000 mgr.gibba032.whopeo (mgr.24625) 3259 : cluster [DBG] pgmap v1848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:07 gibba032 bash[24004]: cluster 2021-03-10T14:54:07.079815+0000 mgr.gibba032.whopeo (mgr.24625) 3260 : cluster [DBG] pgmap v1849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:07 gibba011 bash[30791]: cluster 2021-03-10T14:54:07.079815+0000 mgr.gibba032.whopeo (mgr.24625) 3260 : cluster [DBG] pgmap v1849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:07 gibba011 bash[31774]: cluster 2021-03-10T14:54:07.079815+0000 mgr.gibba032.whopeo (mgr.24625) 3260 : cluster [DBG] pgmap v1849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:10 gibba032 bash[24004]: cluster 2021-03-10T14:54:09.081409+0000 mgr.gibba032.whopeo (mgr.24625) 3261 : cluster [DBG] pgmap v1850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:10 gibba011 bash[30791]: cluster 2021-03-10T14:54:09.081409+0000 mgr.gibba032.whopeo (mgr.24625) 3261 : cluster [DBG] pgmap v1850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:10 gibba011 bash[31774]: cluster 2021-03-10T14:54:09.081409+0000 mgr.gibba032.whopeo (mgr.24625) 3261 : cluster [DBG] pgmap v1850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:12 gibba032 bash[24004]: cluster 2021-03-10T14:54:11.082910+0000 mgr.gibba032.whopeo (mgr.24625) 3262 : cluster [DBG] pgmap v1851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:12 gibba011 bash[30791]: cluster 2021-03-10T14:54:11.082910+0000 mgr.gibba032.whopeo (mgr.24625) 3262 : cluster [DBG] pgmap v1851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:12 gibba011 bash[31774]: cluster 2021-03-10T14:54:11.082910+0000 mgr.gibba032.whopeo (mgr.24625) 3262 : cluster [DBG] pgmap v1851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:14 gibba032 bash[24004]: cluster 2021-03-10T14:54:13.083840+0000 mgr.gibba032.whopeo (mgr.24625) 3263 : cluster [DBG] pgmap v1852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:14 gibba011 bash[30791]: cluster 2021-03-10T14:54:13.083840+0000 mgr.gibba032.whopeo (mgr.24625) 3263 : cluster [DBG] pgmap v1852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:14 gibba011 bash[31774]: cluster 2021-03-10T14:54:13.083840+0000 mgr.gibba032.whopeo (mgr.24625) 3263 : cluster [DBG] pgmap v1852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:16 gibba032 bash[24004]: cluster 2021-03-10T14:54:15.085051+0000 mgr.gibba032.whopeo (mgr.24625) 3264 : cluster [DBG] pgmap v1853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:16 gibba011 bash[30791]: cluster 2021-03-10T14:54:15.085051+0000 mgr.gibba032.whopeo (mgr.24625) 3264 : cluster [DBG] pgmap v1853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:16 gibba011 bash[31774]: cluster 2021-03-10T14:54:15.085051+0000 mgr.gibba032.whopeo (mgr.24625) 3264 : cluster [DBG] pgmap v1853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:17 gibba032 bash[24004]: cluster 2021-03-10T14:54:17.086746+0000 mgr.gibba032.whopeo (mgr.24625) 3265 : cluster [DBG] pgmap v1854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:17 gibba011 bash[30791]: cluster 2021-03-10T14:54:17.086746+0000 mgr.gibba032.whopeo (mgr.24625) 3265 : cluster [DBG] pgmap v1854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:17 gibba011 bash[31774]: cluster 2021-03-10T14:54:17.086746+0000 mgr.gibba032.whopeo (mgr.24625) 3265 : cluster [DBG] pgmap v1854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.237737+0000 mgr.gibba032.whopeo (mgr.24625) 3266 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:54:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.239296+0000 mgr.gibba032.whopeo (mgr.24625) 3267 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:54:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.494207+0000 mon.a (mon.0) 1489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:54:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.495217+0000 mgr.gibba032.whopeo (mgr.24625) 3268 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:54:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.498877+0000 mon.a (mon.0) 1490 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:54:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.499233+0000 mgr.gibba032.whopeo (mgr.24625) 3269 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:54:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.502475+0000 mgr.gibba032.whopeo (mgr.24625) 3270 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:54:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.502690+0000 mgr.gibba032.whopeo (mgr.24625) 3271 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:54:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.502700+0000 mon.a (mon.0) 1491 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.505349+0000 mgr.gibba032.whopeo (mgr.24625) 3272 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:54:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.505416+0000 mgr.gibba032.whopeo (mgr.24625) 3273 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:54:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.506026+0000 mon.a (mon.0) 1492 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.507851+0000 mgr.gibba032.whopeo (mgr.24625) 3274 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:54:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.508573+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.510096+0000 mgr.gibba032.whopeo (mgr.24625) 3275 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:54:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.510200+0000 mgr.gibba032.whopeo (mgr.24625) 3276 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:54:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.512786+0000 mgr.gibba032.whopeo (mgr.24625) 3277 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:54:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.513258+0000 mgr.gibba032.whopeo (mgr.24625) 3278 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:54:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.513689+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.515316+0000 mgr.gibba032.whopeo (mgr.24625) 3279 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:54:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.515394+0000 mgr.gibba032.whopeo (mgr.24625) 3280 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:54:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: cephadm 2021-03-10T14:54:18.515553+0000 mgr.gibba032.whopeo (mgr.24625) 3281 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:19 gibba032 bash[24004]: audit 2021-03-10T14:54:18.516312+0000 mon.a (mon.0) 1495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.237737+0000 mgr.gibba032.whopeo (mgr.24625) 3266 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:54:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.239296+0000 mgr.gibba032.whopeo (mgr.24625) 3267 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:54:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.494207+0000 mon.a (mon.0) 1489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:54:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.495217+0000 mgr.gibba032.whopeo (mgr.24625) 3268 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:54:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.498877+0000 mon.a (mon.0) 1490 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:54:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.499233+0000 mgr.gibba032.whopeo (mgr.24625) 3269 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:54:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.502475+0000 mgr.gibba032.whopeo (mgr.24625) 3270 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:54:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.502690+0000 mgr.gibba032.whopeo (mgr.24625) 3271 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:54:19.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.502700+0000 mon.a (mon.0) 1491 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.505349+0000 mgr.gibba032.whopeo (mgr.24625) 3272 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:54:19.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.505416+0000 mgr.gibba032.whopeo (mgr.24625) 3273 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:54:19.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.506026+0000 mon.a (mon.0) 1492 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.507851+0000 mgr.gibba032.whopeo (mgr.24625) 3274 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:54:19.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.508573+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.510096+0000 mgr.gibba032.whopeo (mgr.24625) 3275 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:54:19.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.510200+0000 mgr.gibba032.whopeo (mgr.24625) 3276 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:54:19.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.512786+0000 mgr.gibba032.whopeo (mgr.24625) 3277 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:54:19.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.513258+0000 mgr.gibba032.whopeo (mgr.24625) 3278 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:54:19.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.513689+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.515316+0000 mgr.gibba032.whopeo (mgr.24625) 3279 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:54:19.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.515394+0000 mgr.gibba032.whopeo (mgr.24625) 3280 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:54:19.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: cephadm 2021-03-10T14:54:18.515553+0000 mgr.gibba032.whopeo (mgr.24625) 3281 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:19.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[30791]: audit 2021-03-10T14:54:18.516312+0000 mon.a (mon.0) 1495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.237737+0000 mgr.gibba032.whopeo (mgr.24625) 3266 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:54:19.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.239296+0000 mgr.gibba032.whopeo (mgr.24625) 3267 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:54:19.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.494207+0000 mon.a (mon.0) 1489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:54:19.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.495217+0000 mgr.gibba032.whopeo (mgr.24625) 3268 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:54:19.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.498877+0000 mon.a (mon.0) 1490 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:54:19.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.499233+0000 mgr.gibba032.whopeo (mgr.24625) 3269 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:54:19.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.502475+0000 mgr.gibba032.whopeo (mgr.24625) 3270 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:54:19.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.502690+0000 mgr.gibba032.whopeo (mgr.24625) 3271 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:54:19.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.502700+0000 mon.a (mon.0) 1491 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.505349+0000 mgr.gibba032.whopeo (mgr.24625) 3272 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:54:19.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.505416+0000 mgr.gibba032.whopeo (mgr.24625) 3273 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:54:19.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.506026+0000 mon.a (mon.0) 1492 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.507851+0000 mgr.gibba032.whopeo (mgr.24625) 3274 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:54:19.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.508573+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.510096+0000 mgr.gibba032.whopeo (mgr.24625) 3275 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.510200+0000 mgr.gibba032.whopeo (mgr.24625) 3276 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.512786+0000 mgr.gibba032.whopeo (mgr.24625) 3277 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.513258+0000 mgr.gibba032.whopeo (mgr.24625) 3278 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.513689+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.515316+0000 mgr.gibba032.whopeo (mgr.24625) 3279 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:54:19.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.515394+0000 mgr.gibba032.whopeo (mgr.24625) 3280 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:54:19.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: cephadm 2021-03-10T14:54:18.515553+0000 mgr.gibba032.whopeo (mgr.24625) 3281 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:19.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:19 gibba011 bash[31774]: audit 2021-03-10T14:54:18.516312+0000 mon.a (mon.0) 1495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:20.140 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:54:20.366 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:54:20.366 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (60m) 6m ago 66m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:54:20.367 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (66m) 5m ago 66m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:54:20.367 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (61m) 5m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:54:20.367 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (61m) 5m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:54:20.367 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (60m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:54:20.367 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (60m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:54:20.368 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (60m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:54:20.368 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (60m) 6m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:54:20.368 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (66m) 6m ago 66m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:54:20.368 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (66m) 5m ago 66m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:54:20.368 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (59m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:54:20.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (58m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:54:20.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (58m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:54:20.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (57m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:54:20.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (57m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:54:20.369 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (57m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:54:20.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (56m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:54:20.370 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (56m) 5m ago 66m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:54:20.370 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (65m) 5m ago 66m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:54:20.370 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 64m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:54:20.618 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:54:20.618 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:54:20.618 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:54:20.619 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:20.619 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:54:20.619 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:54:20.619 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:20.619 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:54:20.620 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:54:20.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:20.620 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:54:20.620 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:54:20.620 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:54:20.621 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:54:20.621 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:54:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:20 gibba032 bash[24004]: cluster 2021-03-10T14:54:19.088287+0000 mgr.gibba032.whopeo (mgr.24625) 3282 : cluster [DBG] pgmap v1855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:20 gibba011 bash[30791]: cluster 2021-03-10T14:54:19.088287+0000 mgr.gibba032.whopeo (mgr.24625) 3282 : cluster [DBG] pgmap v1855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:20 gibba011 bash[31774]: cluster 2021-03-10T14:54:19.088287+0000 mgr.gibba032.whopeo (mgr.24625) 3282 : cluster [DBG] pgmap v1855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[30791]: audit 2021-03-10T14:54:20.126851+0000 mgr.gibba032.whopeo (mgr.24625) 3283 : audit [DBG] from='client.46251 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[30791]: audit 2021-03-10T14:54:20.349817+0000 mgr.gibba032.whopeo (mgr.24625) 3284 : audit [DBG] from='client.56242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[30791]: audit 2021-03-10T14:54:20.615728+0000 mon.a (mon.0) 1496 : audit [DBG] from='client.? 172.21.2.111:0/2632213377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[31774]: audit 2021-03-10T14:54:20.126851+0000 mgr.gibba032.whopeo (mgr.24625) 3283 : audit [DBG] from='client.46251 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[31774]: audit 2021-03-10T14:54:20.349817+0000 mgr.gibba032.whopeo (mgr.24625) 3284 : audit [DBG] from='client.56242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:21 gibba011 bash[31774]: audit 2021-03-10T14:54:20.615728+0000 mon.a (mon.0) 1496 : audit [DBG] from='client.? 172.21.2.111:0/2632213377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:21 gibba032 bash[24004]: audit 2021-03-10T14:54:20.126851+0000 mgr.gibba032.whopeo (mgr.24625) 3283 : audit [DBG] from='client.46251 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:21 gibba032 bash[24004]: audit 2021-03-10T14:54:20.349817+0000 mgr.gibba032.whopeo (mgr.24625) 3284 : audit [DBG] from='client.56242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:21 gibba032 bash[24004]: audit 2021-03-10T14:54:20.615728+0000 mon.a (mon.0) 1496 : audit [DBG] from='client.? 172.21.2.111:0/2632213377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:22 gibba011 bash[30791]: cluster 2021-03-10T14:54:21.089777+0000 mgr.gibba032.whopeo (mgr.24625) 3285 : cluster [DBG] pgmap v1856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:22 gibba011 bash[31774]: cluster 2021-03-10T14:54:21.089777+0000 mgr.gibba032.whopeo (mgr.24625) 3285 : cluster [DBG] pgmap v1856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:22 gibba032 bash[24004]: cluster 2021-03-10T14:54:21.089777+0000 mgr.gibba032.whopeo (mgr.24625) 3285 : cluster [DBG] pgmap v1856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:24 gibba011 bash[30791]: cluster 2021-03-10T14:54:23.090892+0000 mgr.gibba032.whopeo (mgr.24625) 3286 : cluster [DBG] pgmap v1857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:24 gibba011 bash[31774]: cluster 2021-03-10T14:54:23.090892+0000 mgr.gibba032.whopeo (mgr.24625) 3286 : cluster [DBG] pgmap v1857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:24 gibba032 bash[24004]: cluster 2021-03-10T14:54:23.090892+0000 mgr.gibba032.whopeo (mgr.24625) 3286 : cluster [DBG] pgmap v1857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:26 gibba011 bash[30791]: cluster 2021-03-10T14:54:25.092231+0000 mgr.gibba032.whopeo (mgr.24625) 3287 : cluster [DBG] pgmap v1858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:26 gibba011 bash[31774]: cluster 2021-03-10T14:54:25.092231+0000 mgr.gibba032.whopeo (mgr.24625) 3287 : cluster [DBG] pgmap v1858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:26 gibba032 bash[24004]: cluster 2021-03-10T14:54:25.092231+0000 mgr.gibba032.whopeo (mgr.24625) 3287 : cluster [DBG] pgmap v1858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:27 gibba032 bash[24004]: cluster 2021-03-10T14:54:27.093620+0000 mgr.gibba032.whopeo (mgr.24625) 3288 : cluster [DBG] pgmap v1859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:27 gibba011 bash[30791]: cluster 2021-03-10T14:54:27.093620+0000 mgr.gibba032.whopeo (mgr.24625) 3288 : cluster [DBG] pgmap v1859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:27 gibba011 bash[31774]: cluster 2021-03-10T14:54:27.093620+0000 mgr.gibba032.whopeo (mgr.24625) 3288 : cluster [DBG] pgmap v1859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:30 gibba032 bash[24004]: cluster 2021-03-10T14:54:29.095229+0000 mgr.gibba032.whopeo (mgr.24625) 3289 : cluster [DBG] pgmap v1860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:30 gibba011 bash[30791]: cluster 2021-03-10T14:54:29.095229+0000 mgr.gibba032.whopeo (mgr.24625) 3289 : cluster [DBG] pgmap v1860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:30 gibba011 bash[31774]: cluster 2021-03-10T14:54:29.095229+0000 mgr.gibba032.whopeo (mgr.24625) 3289 : cluster [DBG] pgmap v1860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:32 gibba032 bash[24004]: cluster 2021-03-10T14:54:31.096752+0000 mgr.gibba032.whopeo (mgr.24625) 3290 : cluster [DBG] pgmap v1861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:32 gibba011 bash[30791]: cluster 2021-03-10T14:54:31.096752+0000 mgr.gibba032.whopeo (mgr.24625) 3290 : cluster [DBG] pgmap v1861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:32 gibba011 bash[31774]: cluster 2021-03-10T14:54:31.096752+0000 mgr.gibba032.whopeo (mgr.24625) 3290 : cluster [DBG] pgmap v1861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:34 gibba032 bash[24004]: cluster 2021-03-10T14:54:33.097656+0000 mgr.gibba032.whopeo (mgr.24625) 3291 : cluster [DBG] pgmap v1862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:34 gibba032 bash[24004]: cephadm 2021-03-10T14:54:33.531681+0000 mgr.gibba032.whopeo (mgr.24625) 3292 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:34 gibba011 bash[30791]: cluster 2021-03-10T14:54:33.097656+0000 mgr.gibba032.whopeo (mgr.24625) 3291 : cluster [DBG] pgmap v1862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:34 gibba011 bash[30791]: cephadm 2021-03-10T14:54:33.531681+0000 mgr.gibba032.whopeo (mgr.24625) 3292 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:34 gibba011 bash[31774]: cluster 2021-03-10T14:54:33.097656+0000 mgr.gibba032.whopeo (mgr.24625) 3291 : cluster [DBG] pgmap v1862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:34 gibba011 bash[31774]: cephadm 2021-03-10T14:54:33.531681+0000 mgr.gibba032.whopeo (mgr.24625) 3292 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:36 gibba032 bash[24004]: cluster 2021-03-10T14:54:35.099018+0000 mgr.gibba032.whopeo (mgr.24625) 3293 : cluster [DBG] pgmap v1863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:36 gibba011 bash[30791]: cluster 2021-03-10T14:54:35.099018+0000 mgr.gibba032.whopeo (mgr.24625) 3293 : cluster [DBG] pgmap v1863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:36 gibba011 bash[31774]: cluster 2021-03-10T14:54:35.099018+0000 mgr.gibba032.whopeo (mgr.24625) 3293 : cluster [DBG] pgmap v1863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:37 gibba032 bash[24004]: cluster 2021-03-10T14:54:37.100599+0000 mgr.gibba032.whopeo (mgr.24625) 3294 : cluster [DBG] pgmap v1864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:37 gibba011 bash[30791]: cluster 2021-03-10T14:54:37.100599+0000 mgr.gibba032.whopeo (mgr.24625) 3294 : cluster [DBG] pgmap v1864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:37 gibba011 bash[31774]: cluster 2021-03-10T14:54:37.100599+0000 mgr.gibba032.whopeo (mgr.24625) 3294 : cluster [DBG] pgmap v1864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:40 gibba032 bash[24004]: cluster 2021-03-10T14:54:39.102197+0000 mgr.gibba032.whopeo (mgr.24625) 3295 : cluster [DBG] pgmap v1865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:40 gibba011 bash[30791]: cluster 2021-03-10T14:54:39.102197+0000 mgr.gibba032.whopeo (mgr.24625) 3295 : cluster [DBG] pgmap v1865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:40 gibba011 bash[31774]: cluster 2021-03-10T14:54:39.102197+0000 mgr.gibba032.whopeo (mgr.24625) 3295 : cluster [DBG] pgmap v1865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:42 gibba032 bash[24004]: cluster 2021-03-10T14:54:41.103726+0000 mgr.gibba032.whopeo (mgr.24625) 3296 : cluster [DBG] pgmap v1866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:42 gibba011 bash[30791]: cluster 2021-03-10T14:54:41.103726+0000 mgr.gibba032.whopeo (mgr.24625) 3296 : cluster [DBG] pgmap v1866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:42 gibba011 bash[31774]: cluster 2021-03-10T14:54:41.103726+0000 mgr.gibba032.whopeo (mgr.24625) 3296 : cluster [DBG] pgmap v1866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:44 gibba032 bash[24004]: cluster 2021-03-10T14:54:43.105018+0000 mgr.gibba032.whopeo (mgr.24625) 3297 : cluster [DBG] pgmap v1867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:44 gibba011 bash[30791]: cluster 2021-03-10T14:54:43.105018+0000 mgr.gibba032.whopeo (mgr.24625) 3297 : cluster [DBG] pgmap v1867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:44 gibba011 bash[31774]: cluster 2021-03-10T14:54:43.105018+0000 mgr.gibba032.whopeo (mgr.24625) 3297 : cluster [DBG] pgmap v1867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:46 gibba032 bash[24004]: cluster 2021-03-10T14:54:45.106253+0000 mgr.gibba032.whopeo (mgr.24625) 3298 : cluster [DBG] pgmap v1868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:46 gibba011 bash[30791]: cluster 2021-03-10T14:54:45.106253+0000 mgr.gibba032.whopeo (mgr.24625) 3298 : cluster [DBG] pgmap v1868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:46 gibba011 bash[31774]: cluster 2021-03-10T14:54:45.106253+0000 mgr.gibba032.whopeo (mgr.24625) 3298 : cluster [DBG] pgmap v1868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:47 gibba032 bash[24004]: cluster 2021-03-10T14:54:47.107810+0000 mgr.gibba032.whopeo (mgr.24625) 3299 : cluster [DBG] pgmap v1869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:47 gibba011 bash[30791]: cluster 2021-03-10T14:54:47.107810+0000 mgr.gibba032.whopeo (mgr.24625) 3299 : cluster [DBG] pgmap v1869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:47 gibba011 bash[31774]: cluster 2021-03-10T14:54:47.107810+0000 mgr.gibba032.whopeo (mgr.24625) 3299 : cluster [DBG] pgmap v1869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:48 gibba032 bash[24004]: cephadm 2021-03-10T14:54:48.541140+0000 mgr.gibba032.whopeo (mgr.24625) 3300 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:48 gibba011 bash[30791]: cephadm 2021-03-10T14:54:48.541140+0000 mgr.gibba032.whopeo (mgr.24625) 3300 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:49.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:48 gibba011 bash[31774]: cephadm 2021-03-10T14:54:48.541140+0000 mgr.gibba032.whopeo (mgr.24625) 3300 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:54:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:49 gibba032 bash[24004]: cluster 2021-03-10T14:54:49.109422+0000 mgr.gibba032.whopeo (mgr.24625) 3301 : cluster [DBG] pgmap v1870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:49 gibba011 bash[30791]: cluster 2021-03-10T14:54:49.109422+0000 mgr.gibba032.whopeo (mgr.24625) 3301 : cluster [DBG] pgmap v1870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:49 gibba011 bash[31774]: cluster 2021-03-10T14:54:49.109422+0000 mgr.gibba032.whopeo (mgr.24625) 3301 : cluster [DBG] pgmap v1870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:50.859 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:54:51.088 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:54:51.088 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (61m) 6m ago 66m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:54:51.088 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (66m) 5m ago 66m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:54:51.089 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (61m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:54:51.089 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (62m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:54:51.089 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (61m) 6m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:54:51.089 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (61m) 6m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:54:51.090 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (60m) 5m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:54:51.090 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (60m) 6m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:54:51.090 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (66m) 6m ago 67m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:54:51.090 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (66m) 5m ago 67m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:54:51.090 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (59m) 6m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:54:51.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (59m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:54:51.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (58m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:54:51.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (58m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:54:51.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (58m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:54:51.091 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (57m) 5m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:54:51.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (57m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:54:51.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (56m) 5m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:54:51.092 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (66m) 5m ago 67m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:54:51.092 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 64m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:54:51.335 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:54:51.336 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:54:51.336 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:54:51.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:51.336 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:54:51.337 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:54:51.337 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:51.337 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:54:51.337 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:54:51.337 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:54:51.338 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:54:51.338 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:54:51.338 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:54:51.338 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:54:51.339 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:54:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:52 gibba032 bash[24004]: audit 2021-03-10T14:54:50.846097+0000 mgr.gibba032.whopeo (mgr.24625) 3302 : audit [DBG] from='client.56254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:52 gibba032 bash[24004]: audit 2021-03-10T14:54:51.071231+0000 mgr.gibba032.whopeo (mgr.24625) 3303 : audit [DBG] from='client.46275 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:52 gibba032 bash[24004]: cluster 2021-03-10T14:54:51.110004+0000 mgr.gibba032.whopeo (mgr.24625) 3304 : cluster [DBG] pgmap v1871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:52.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:52 gibba032 bash[24004]: audit 2021-03-10T14:54:51.333345+0000 mon.c (mon.1) 63 : audit [DBG] from='client.? 172.21.2.111:0/1191476275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[30791]: audit 2021-03-10T14:54:50.846097+0000 mgr.gibba032.whopeo (mgr.24625) 3302 : audit [DBG] from='client.56254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[30791]: audit 2021-03-10T14:54:51.071231+0000 mgr.gibba032.whopeo (mgr.24625) 3303 : audit [DBG] from='client.46275 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[30791]: cluster 2021-03-10T14:54:51.110004+0000 mgr.gibba032.whopeo (mgr.24625) 3304 : cluster [DBG] pgmap v1871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[30791]: audit 2021-03-10T14:54:51.333345+0000 mon.c (mon.1) 63 : audit [DBG] from='client.? 172.21.2.111:0/1191476275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[31774]: audit 2021-03-10T14:54:50.846097+0000 mgr.gibba032.whopeo (mgr.24625) 3302 : audit [DBG] from='client.56254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[31774]: audit 2021-03-10T14:54:51.071231+0000 mgr.gibba032.whopeo (mgr.24625) 3303 : audit [DBG] from='client.46275 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:54:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[31774]: cluster 2021-03-10T14:54:51.110004+0000 mgr.gibba032.whopeo (mgr.24625) 3304 : cluster [DBG] pgmap v1871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:52 gibba011 bash[31774]: audit 2021-03-10T14:54:51.333345+0000 mon.c (mon.1) 63 : audit [DBG] from='client.? 172.21.2.111:0/1191476275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:54:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:54 gibba032 bash[24004]: cluster 2021-03-10T14:54:53.111246+0000 mgr.gibba032.whopeo (mgr.24625) 3305 : cluster [DBG] pgmap v1872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:54 gibba011 bash[30791]: cluster 2021-03-10T14:54:53.111246+0000 mgr.gibba032.whopeo (mgr.24625) 3305 : cluster [DBG] pgmap v1872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:54 gibba011 bash[31774]: cluster 2021-03-10T14:54:53.111246+0000 mgr.gibba032.whopeo (mgr.24625) 3305 : cluster [DBG] pgmap v1872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:56 gibba032 bash[24004]: cluster 2021-03-10T14:54:55.112558+0000 mgr.gibba032.whopeo (mgr.24625) 3306 : cluster [DBG] pgmap v1873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:56 gibba011 bash[30791]: cluster 2021-03-10T14:54:55.112558+0000 mgr.gibba032.whopeo (mgr.24625) 3306 : cluster [DBG] pgmap v1873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:56 gibba011 bash[31774]: cluster 2021-03-10T14:54:55.112558+0000 mgr.gibba032.whopeo (mgr.24625) 3306 : cluster [DBG] pgmap v1873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.411427+0000 mon.a (mon.0) 1497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:54:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.412074+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:54:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.427398+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:54:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.428205+0000 mon.a (mon.0) 1500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T14:54:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.428902+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:54:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:57 gibba032 bash[24004]: audit 2021-03-10T14:54:56.429584+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:54:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.411427+0000 mon.a (mon.0) 1497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:54:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.412074+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:54:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.427398+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:54:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.428205+0000 mon.a (mon.0) 1500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T14:54:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.428902+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:54:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[30791]: audit 2021-03-10T14:54:56.429584+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:54:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.411427+0000 mon.a (mon.0) 1497 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:54:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.412074+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:54:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.427398+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T14:54:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.428205+0000 mon.a (mon.0) 1500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T14:54:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.428902+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T14:54:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:57 gibba011 bash[31774]: audit 2021-03-10T14:54:56.429584+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T14:54:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:54:58 gibba032 bash[24004]: cluster 2021-03-10T14:54:57.114137+0000 mgr.gibba032.whopeo (mgr.24625) 3307 : cluster [DBG] pgmap v1874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:54:58 gibba011 bash[30791]: cluster 2021-03-10T14:54:57.114137+0000 mgr.gibba032.whopeo (mgr.24625) 3307 : cluster [DBG] pgmap v1874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:54:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:54:58 gibba011 bash[31774]: cluster 2021-03-10T14:54:57.114137+0000 mgr.gibba032.whopeo (mgr.24625) 3307 : cluster [DBG] pgmap v1874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:00 gibba032 bash[24004]: cluster 2021-03-10T14:54:59.115759+0000 mgr.gibba032.whopeo (mgr.24625) 3308 : cluster [DBG] pgmap v1875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:00 gibba011 bash[30791]: cluster 2021-03-10T14:54:59.115759+0000 mgr.gibba032.whopeo (mgr.24625) 3308 : cluster [DBG] pgmap v1875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:00 gibba011 bash[31774]: cluster 2021-03-10T14:54:59.115759+0000 mgr.gibba032.whopeo (mgr.24625) 3308 : cluster [DBG] pgmap v1875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:02 gibba032 bash[24004]: cluster 2021-03-10T14:55:01.117273+0000 mgr.gibba032.whopeo (mgr.24625) 3309 : cluster [DBG] pgmap v1876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:02 gibba011 bash[30791]: cluster 2021-03-10T14:55:01.117273+0000 mgr.gibba032.whopeo (mgr.24625) 3309 : cluster [DBG] pgmap v1876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:02 gibba011 bash[31774]: cluster 2021-03-10T14:55:01.117273+0000 mgr.gibba032.whopeo (mgr.24625) 3309 : cluster [DBG] pgmap v1876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:04 gibba032 bash[24004]: cluster 2021-03-10T14:55:03.118964+0000 mgr.gibba032.whopeo (mgr.24625) 3310 : cluster [DBG] pgmap v1877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:04 gibba032 bash[24004]: cephadm 2021-03-10T14:55:03.556934+0000 mgr.gibba032.whopeo (mgr.24625) 3311 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:04 gibba011 bash[30791]: cluster 2021-03-10T14:55:03.118964+0000 mgr.gibba032.whopeo (mgr.24625) 3310 : cluster [DBG] pgmap v1877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:04 gibba011 bash[30791]: cephadm 2021-03-10T14:55:03.556934+0000 mgr.gibba032.whopeo (mgr.24625) 3311 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:04 gibba011 bash[31774]: cluster 2021-03-10T14:55:03.118964+0000 mgr.gibba032.whopeo (mgr.24625) 3310 : cluster [DBG] pgmap v1877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:04 gibba011 bash[31774]: cephadm 2021-03-10T14:55:03.556934+0000 mgr.gibba032.whopeo (mgr.24625) 3311 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:06 gibba032 bash[24004]: cluster 2021-03-10T14:55:05.120321+0000 mgr.gibba032.whopeo (mgr.24625) 3312 : cluster [DBG] pgmap v1878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:06 gibba011 bash[30791]: cluster 2021-03-10T14:55:05.120321+0000 mgr.gibba032.whopeo (mgr.24625) 3312 : cluster [DBG] pgmap v1878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:06 gibba011 bash[31774]: cluster 2021-03-10T14:55:05.120321+0000 mgr.gibba032.whopeo (mgr.24625) 3312 : cluster [DBG] pgmap v1878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:07 gibba032 bash[24004]: cluster 2021-03-10T14:55:07.122016+0000 mgr.gibba032.whopeo (mgr.24625) 3313 : cluster [DBG] pgmap v1879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:07 gibba011 bash[30791]: cluster 2021-03-10T14:55:07.122016+0000 mgr.gibba032.whopeo (mgr.24625) 3313 : cluster [DBG] pgmap v1879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:07 gibba011 bash[31774]: cluster 2021-03-10T14:55:07.122016+0000 mgr.gibba032.whopeo (mgr.24625) 3313 : cluster [DBG] pgmap v1879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:10 gibba032 bash[24004]: cluster 2021-03-10T14:55:09.123589+0000 mgr.gibba032.whopeo (mgr.24625) 3314 : cluster [DBG] pgmap v1880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:10 gibba011 bash[30791]: cluster 2021-03-10T14:55:09.123589+0000 mgr.gibba032.whopeo (mgr.24625) 3314 : cluster [DBG] pgmap v1880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:10 gibba011 bash[31774]: cluster 2021-03-10T14:55:09.123589+0000 mgr.gibba032.whopeo (mgr.24625) 3314 : cluster [DBG] pgmap v1880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:12 gibba032 bash[24004]: cluster 2021-03-10T14:55:11.125118+0000 mgr.gibba032.whopeo (mgr.24625) 3315 : cluster [DBG] pgmap v1881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:12 gibba011 bash[30791]: cluster 2021-03-10T14:55:11.125118+0000 mgr.gibba032.whopeo (mgr.24625) 3315 : cluster [DBG] pgmap v1881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:12 gibba011 bash[31774]: cluster 2021-03-10T14:55:11.125118+0000 mgr.gibba032.whopeo (mgr.24625) 3315 : cluster [DBG] pgmap v1881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:14 gibba032 bash[24004]: cluster 2021-03-10T14:55:13.126600+0000 mgr.gibba032.whopeo (mgr.24625) 3316 : cluster [DBG] pgmap v1882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:14 gibba011 bash[30791]: cluster 2021-03-10T14:55:13.126600+0000 mgr.gibba032.whopeo (mgr.24625) 3316 : cluster [DBG] pgmap v1882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:14 gibba011 bash[31774]: cluster 2021-03-10T14:55:13.126600+0000 mgr.gibba032.whopeo (mgr.24625) 3316 : cluster [DBG] pgmap v1882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:16 gibba032 bash[24004]: cluster 2021-03-10T14:55:15.127868+0000 mgr.gibba032.whopeo (mgr.24625) 3317 : cluster [DBG] pgmap v1883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:16 gibba011 bash[30791]: cluster 2021-03-10T14:55:15.127868+0000 mgr.gibba032.whopeo (mgr.24625) 3317 : cluster [DBG] pgmap v1883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:16 gibba011 bash[31774]: cluster 2021-03-10T14:55:15.127868+0000 mgr.gibba032.whopeo (mgr.24625) 3317 : cluster [DBG] pgmap v1883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:17 gibba032 bash[24004]: cluster 2021-03-10T14:55:17.129401+0000 mgr.gibba032.whopeo (mgr.24625) 3318 : cluster [DBG] pgmap v1884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:17 gibba011 bash[30791]: cluster 2021-03-10T14:55:17.129401+0000 mgr.gibba032.whopeo (mgr.24625) 3318 : cluster [DBG] pgmap v1884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:17 gibba011 bash[31774]: cluster 2021-03-10T14:55:17.129401+0000 mgr.gibba032.whopeo (mgr.24625) 3318 : cluster [DBG] pgmap v1884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.776659+0000 mgr.gibba032.whopeo (mgr.24625) 3319 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:55:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.778511+0000 mon.a (mon.0) 1503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.779709+0000 mgr.gibba032.whopeo (mgr.24625) 3320 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:55:20.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.781356+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.985352+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.985477+0000 mgr.gibba032.whopeo (mgr.24625) 3321 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:55:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.987407+0000 mgr.gibba032.whopeo (mgr.24625) 3322 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:55:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.988751+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:55:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.988824+0000 mgr.gibba032.whopeo (mgr.24625) 3323 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:55:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.988898+0000 mgr.gibba032.whopeo (mgr.24625) 3324 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:55:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.990375+0000 mon.a (mon.0) 1507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.990485+0000 mgr.gibba032.whopeo (mgr.24625) 3325 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:55:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.990638+0000 mgr.gibba032.whopeo (mgr.24625) 3326 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:55:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.991891+0000 mgr.gibba032.whopeo (mgr.24625) 3327 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:55:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.991968+0000 mon.a (mon.0) 1508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.993090+0000 mgr.gibba032.whopeo (mgr.24625) 3328 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:55:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.993127+0000 mgr.gibba032.whopeo (mgr.24625) 3329 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:55:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.993477+0000 mon.a (mon.0) 1509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.994590+0000 mgr.gibba032.whopeo (mgr.24625) 3330 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:55:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.994914+0000 mgr.gibba032.whopeo (mgr.24625) 3331 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:55:20.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.996103+0000 mon.a (mon.0) 1510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.776659+0000 mgr.gibba032.whopeo (mgr.24625) 3319 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:55:20.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.778511+0000 mon.a (mon.0) 1503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.779709+0000 mgr.gibba032.whopeo (mgr.24625) 3320 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:55:20.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.781356+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.985352+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.985477+0000 mgr.gibba032.whopeo (mgr.24625) 3321 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:55:20.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.987407+0000 mgr.gibba032.whopeo (mgr.24625) 3322 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:55:20.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.988751+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:55:20.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.988824+0000 mgr.gibba032.whopeo (mgr.24625) 3323 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:55:20.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.988898+0000 mgr.gibba032.whopeo (mgr.24625) 3324 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:55:20.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.990375+0000 mon.a (mon.0) 1507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.990485+0000 mgr.gibba032.whopeo (mgr.24625) 3325 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:55:20.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.990638+0000 mgr.gibba032.whopeo (mgr.24625) 3326 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:55:20.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.991891+0000 mgr.gibba032.whopeo (mgr.24625) 3327 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:55:20.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.991968+0000 mon.a (mon.0) 1508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.993090+0000 mgr.gibba032.whopeo (mgr.24625) 3328 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:55:20.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.993127+0000 mgr.gibba032.whopeo (mgr.24625) 3329 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:55:20.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.993477+0000 mon.a (mon.0) 1509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.994590+0000 mgr.gibba032.whopeo (mgr.24625) 3330 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:55:20.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.994914+0000 mgr.gibba032.whopeo (mgr.24625) 3331 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:55:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.996103+0000 mon.a (mon.0) 1510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.996179+0000 mgr.gibba032.whopeo (mgr.24625) 3332 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:55:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.996218+0000 mgr.gibba032.whopeo (mgr.24625) 3333 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:55:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cephadm 2021-03-10T14:55:18.996322+0000 mgr.gibba032.whopeo (mgr.24625) 3334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: audit 2021-03-10T14:55:18.997733+0000 mon.a (mon.0) 1511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[31774]: cluster 2021-03-10T14:55:19.131048+0000 mgr.gibba032.whopeo (mgr.24625) 3335 : cluster [DBG] pgmap v1885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:20.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.996179+0000 mgr.gibba032.whopeo (mgr.24625) 3332 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:55:20.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.996218+0000 mgr.gibba032.whopeo (mgr.24625) 3333 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:55:20.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cephadm 2021-03-10T14:55:18.996322+0000 mgr.gibba032.whopeo (mgr.24625) 3334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:20.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: audit 2021-03-10T14:55:18.997733+0000 mon.a (mon.0) 1511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.094 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:19 gibba011 bash[30791]: cluster 2021-03-10T14:55:19.131048+0000 mgr.gibba032.whopeo (mgr.24625) 3335 : cluster [DBG] pgmap v1885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:20.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.776659+0000 mgr.gibba032.whopeo (mgr.24625) 3319 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:55:20.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.778511+0000 mon.a (mon.0) 1503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.779709+0000 mgr.gibba032.whopeo (mgr.24625) 3320 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:55:20.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.781356+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.985352+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:55:20.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.985477+0000 mgr.gibba032.whopeo (mgr.24625) 3321 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:55:20.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.987407+0000 mgr.gibba032.whopeo (mgr.24625) 3322 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:55:20.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.988751+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:55:20.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.988824+0000 mgr.gibba032.whopeo (mgr.24625) 3323 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:55:20.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.988898+0000 mgr.gibba032.whopeo (mgr.24625) 3324 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:55:20.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.990375+0000 mon.a (mon.0) 1507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.990485+0000 mgr.gibba032.whopeo (mgr.24625) 3325 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:55:20.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.990638+0000 mgr.gibba032.whopeo (mgr.24625) 3326 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:55:20.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.991891+0000 mgr.gibba032.whopeo (mgr.24625) 3327 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:55:20.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.991968+0000 mon.a (mon.0) 1508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.993090+0000 mgr.gibba032.whopeo (mgr.24625) 3328 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:55:20.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.993127+0000 mgr.gibba032.whopeo (mgr.24625) 3329 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:55:20.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.993477+0000 mon.a (mon.0) 1509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.994590+0000 mgr.gibba032.whopeo (mgr.24625) 3330 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:55:20.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.994914+0000 mgr.gibba032.whopeo (mgr.24625) 3331 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:55:20.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.996103+0000 mon.a (mon.0) 1510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.996179+0000 mgr.gibba032.whopeo (mgr.24625) 3332 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:55:20.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.996218+0000 mgr.gibba032.whopeo (mgr.24625) 3333 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:55:20.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cephadm 2021-03-10T14:55:18.996322+0000 mgr.gibba032.whopeo (mgr.24625) 3334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:20.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: audit 2021-03-10T14:55:18.997733+0000 mon.a (mon.0) 1511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:20.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:19 gibba032 bash[24004]: cluster 2021-03-10T14:55:19.131048+0000 mgr.gibba032.whopeo (mgr.24625) 3335 : cluster [DBG] pgmap v1885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:21.578 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:55:21.832 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:55:21.832 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (62m) 7m ago 67m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:55:21.832 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (67m) 6m ago 67m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:55:21.833 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (62m) 6m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:55:21.833 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (62m) 6m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:55:21.833 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (61m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:55:21.833 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (61m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:55:21.833 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (61m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:55:21.834 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (61m) 7m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:55:21.834 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (67m) 7m ago 67m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:55:21.834 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (67m) 6m ago 67m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:55:21.834 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (60m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:55:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (59m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:55:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (59m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:55:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (59m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:55:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (58m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:55:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (58m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:55:21.836 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (57m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:55:21.836 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (57m) 6m ago 67m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:55:21.836 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (66m) 6m ago 67m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:55:21.836 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 65m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:55:22.080 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:55:22.080 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:55:22.080 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:55:22.081 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:22.081 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:55:22.081 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:55:22.081 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:22.081 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:55:22.082 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:55:22.082 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:22.082 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:55:22.082 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:55:22.082 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:55:22.083 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:55:22.083 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:55:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:22 gibba032 bash[24004]: cluster 2021-03-10T14:55:21.132555+0000 mgr.gibba032.whopeo (mgr.24625) 3336 : cluster [DBG] pgmap v1886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:22 gibba032 bash[24004]: audit 2021-03-10T14:55:21.565369+0000 mgr.gibba032.whopeo (mgr.24625) 3337 : audit [DBG] from='client.56272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:22 gibba032 bash[24004]: audit 2021-03-10T14:55:22.077737+0000 mon.c (mon.1) 64 : audit [DBG] from='client.? 172.21.2.111:0/4183205073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[30791]: cluster 2021-03-10T14:55:21.132555+0000 mgr.gibba032.whopeo (mgr.24625) 3336 : cluster [DBG] pgmap v1886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[30791]: audit 2021-03-10T14:55:21.565369+0000 mgr.gibba032.whopeo (mgr.24625) 3337 : audit [DBG] from='client.56272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:22.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[30791]: audit 2021-03-10T14:55:22.077737+0000 mon.c (mon.1) 64 : audit [DBG] from='client.? 172.21.2.111:0/4183205073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[31774]: cluster 2021-03-10T14:55:21.132555+0000 mgr.gibba032.whopeo (mgr.24625) 3336 : cluster [DBG] pgmap v1886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[31774]: audit 2021-03-10T14:55:21.565369+0000 mgr.gibba032.whopeo (mgr.24625) 3337 : audit [DBG] from='client.56272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:22 gibba011 bash[31774]: audit 2021-03-10T14:55:22.077737+0000 mon.c (mon.1) 64 : audit [DBG] from='client.? 172.21.2.111:0/4183205073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:23 gibba032 bash[24004]: audit 2021-03-10T14:55:21.788491+0000 mgr.gibba032.whopeo (mgr.24625) 3338 : audit [DBG] from='client.56278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:23 gibba011 bash[30791]: audit 2021-03-10T14:55:21.788491+0000 mgr.gibba032.whopeo (mgr.24625) 3338 : audit [DBG] from='client.56278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:23.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:23 gibba011 bash[31774]: audit 2021-03-10T14:55:21.788491+0000 mgr.gibba032.whopeo (mgr.24625) 3338 : audit [DBG] from='client.56278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:24 gibba032 bash[24004]: cluster 2021-03-10T14:55:23.134057+0000 mgr.gibba032.whopeo (mgr.24625) 3339 : cluster [DBG] pgmap v1887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:24 gibba011 bash[30791]: cluster 2021-03-10T14:55:23.134057+0000 mgr.gibba032.whopeo (mgr.24625) 3339 : cluster [DBG] pgmap v1887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:24 gibba011 bash[31774]: cluster 2021-03-10T14:55:23.134057+0000 mgr.gibba032.whopeo (mgr.24625) 3339 : cluster [DBG] pgmap v1887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:26 gibba032 bash[24004]: cluster 2021-03-10T14:55:25.135368+0000 mgr.gibba032.whopeo (mgr.24625) 3340 : cluster [DBG] pgmap v1888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:26 gibba011 bash[30791]: cluster 2021-03-10T14:55:25.135368+0000 mgr.gibba032.whopeo (mgr.24625) 3340 : cluster [DBG] pgmap v1888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:26 gibba011 bash[31774]: cluster 2021-03-10T14:55:25.135368+0000 mgr.gibba032.whopeo (mgr.24625) 3340 : cluster [DBG] pgmap v1888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:27 gibba032 bash[24004]: cluster 2021-03-10T14:55:27.137048+0000 mgr.gibba032.whopeo (mgr.24625) 3341 : cluster [DBG] pgmap v1889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:27 gibba011 bash[30791]: cluster 2021-03-10T14:55:27.137048+0000 mgr.gibba032.whopeo (mgr.24625) 3341 : cluster [DBG] pgmap v1889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:27 gibba011 bash[31774]: cluster 2021-03-10T14:55:27.137048+0000 mgr.gibba032.whopeo (mgr.24625) 3341 : cluster [DBG] pgmap v1889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:30 gibba032 bash[24004]: cluster 2021-03-10T14:55:29.138388+0000 mgr.gibba032.whopeo (mgr.24625) 3342 : cluster [DBG] pgmap v1890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:30 gibba011 bash[30791]: cluster 2021-03-10T14:55:29.138388+0000 mgr.gibba032.whopeo (mgr.24625) 3342 : cluster [DBG] pgmap v1890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:30 gibba011 bash[31774]: cluster 2021-03-10T14:55:29.138388+0000 mgr.gibba032.whopeo (mgr.24625) 3342 : cluster [DBG] pgmap v1890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:32 gibba032 bash[24004]: cluster 2021-03-10T14:55:31.139845+0000 mgr.gibba032.whopeo (mgr.24625) 3343 : cluster [DBG] pgmap v1891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:32 gibba011 bash[30791]: cluster 2021-03-10T14:55:31.139845+0000 mgr.gibba032.whopeo (mgr.24625) 3343 : cluster [DBG] pgmap v1891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:32 gibba011 bash[31774]: cluster 2021-03-10T14:55:31.139845+0000 mgr.gibba032.whopeo (mgr.24625) 3343 : cluster [DBG] pgmap v1891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:34 gibba032 bash[24004]: cluster 2021-03-10T14:55:33.141605+0000 mgr.gibba032.whopeo (mgr.24625) 3344 : cluster [DBG] pgmap v1892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:34 gibba011 bash[30791]: cluster 2021-03-10T14:55:33.141605+0000 mgr.gibba032.whopeo (mgr.24625) 3344 : cluster [DBG] pgmap v1892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:34 gibba011 bash[31774]: cluster 2021-03-10T14:55:33.141605+0000 mgr.gibba032.whopeo (mgr.24625) 3344 : cluster [DBG] pgmap v1892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:35 gibba032 bash[24004]: cephadm 2021-03-10T14:55:34.012434+0000 mgr.gibba032.whopeo (mgr.24625) 3345 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:35 gibba011 bash[30791]: cephadm 2021-03-10T14:55:34.012434+0000 mgr.gibba032.whopeo (mgr.24625) 3345 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:35.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:35 gibba011 bash[31774]: cephadm 2021-03-10T14:55:34.012434+0000 mgr.gibba032.whopeo (mgr.24625) 3345 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:36 gibba032 bash[24004]: cluster 2021-03-10T14:55:35.143062+0000 mgr.gibba032.whopeo (mgr.24625) 3346 : cluster [DBG] pgmap v1893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:36 gibba011 bash[30791]: cluster 2021-03-10T14:55:35.143062+0000 mgr.gibba032.whopeo (mgr.24625) 3346 : cluster [DBG] pgmap v1893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:36 gibba011 bash[31774]: cluster 2021-03-10T14:55:35.143062+0000 mgr.gibba032.whopeo (mgr.24625) 3346 : cluster [DBG] pgmap v1893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:37 gibba032 bash[24004]: cluster 2021-03-10T14:55:37.144326+0000 mgr.gibba032.whopeo (mgr.24625) 3347 : cluster [DBG] pgmap v1894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:37 gibba011 bash[30791]: cluster 2021-03-10T14:55:37.144326+0000 mgr.gibba032.whopeo (mgr.24625) 3347 : cluster [DBG] pgmap v1894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:37 gibba011 bash[31774]: cluster 2021-03-10T14:55:37.144326+0000 mgr.gibba032.whopeo (mgr.24625) 3347 : cluster [DBG] pgmap v1894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:40 gibba032 bash[24004]: cluster 2021-03-10T14:55:39.145953+0000 mgr.gibba032.whopeo (mgr.24625) 3348 : cluster [DBG] pgmap v1895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:40 gibba011 bash[30791]: cluster 2021-03-10T14:55:39.145953+0000 mgr.gibba032.whopeo (mgr.24625) 3348 : cluster [DBG] pgmap v1895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:40 gibba011 bash[31774]: cluster 2021-03-10T14:55:39.145953+0000 mgr.gibba032.whopeo (mgr.24625) 3348 : cluster [DBG] pgmap v1895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:42 gibba032 bash[24004]: cluster 2021-03-10T14:55:41.147510+0000 mgr.gibba032.whopeo (mgr.24625) 3349 : cluster [DBG] pgmap v1896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:42 gibba011 bash[30791]: cluster 2021-03-10T14:55:41.147510+0000 mgr.gibba032.whopeo (mgr.24625) 3349 : cluster [DBG] pgmap v1896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:42 gibba011 bash[31774]: cluster 2021-03-10T14:55:41.147510+0000 mgr.gibba032.whopeo (mgr.24625) 3349 : cluster [DBG] pgmap v1896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:44 gibba032 bash[24004]: cluster 2021-03-10T14:55:43.149277+0000 mgr.gibba032.whopeo (mgr.24625) 3350 : cluster [DBG] pgmap v1897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:44 gibba011 bash[30791]: cluster 2021-03-10T14:55:43.149277+0000 mgr.gibba032.whopeo (mgr.24625) 3350 : cluster [DBG] pgmap v1897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:44 gibba011 bash[31774]: cluster 2021-03-10T14:55:43.149277+0000 mgr.gibba032.whopeo (mgr.24625) 3350 : cluster [DBG] pgmap v1897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:46 gibba032 bash[24004]: cluster 2021-03-10T14:55:45.150725+0000 mgr.gibba032.whopeo (mgr.24625) 3351 : cluster [DBG] pgmap v1898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:46 gibba011 bash[30791]: cluster 2021-03-10T14:55:45.150725+0000 mgr.gibba032.whopeo (mgr.24625) 3351 : cluster [DBG] pgmap v1898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:46 gibba011 bash[31774]: cluster 2021-03-10T14:55:45.150725+0000 mgr.gibba032.whopeo (mgr.24625) 3351 : cluster [DBG] pgmap v1898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:47 gibba032 bash[24004]: cluster 2021-03-10T14:55:47.152271+0000 mgr.gibba032.whopeo (mgr.24625) 3352 : cluster [DBG] pgmap v1899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:47 gibba011 bash[30791]: cluster 2021-03-10T14:55:47.152271+0000 mgr.gibba032.whopeo (mgr.24625) 3352 : cluster [DBG] pgmap v1899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:47 gibba011 bash[31774]: cluster 2021-03-10T14:55:47.152271+0000 mgr.gibba032.whopeo (mgr.24625) 3352 : cluster [DBG] pgmap v1899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:50 gibba032 bash[24004]: cephadm 2021-03-10T14:55:49.013745+0000 mgr.gibba032.whopeo (mgr.24625) 3353 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:50 gibba032 bash[24004]: cluster 2021-03-10T14:55:49.153859+0000 mgr.gibba032.whopeo (mgr.24625) 3354 : cluster [DBG] pgmap v1900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:50 gibba011 bash[30791]: cephadm 2021-03-10T14:55:49.013745+0000 mgr.gibba032.whopeo (mgr.24625) 3353 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:50 gibba011 bash[30791]: cluster 2021-03-10T14:55:49.153859+0000 mgr.gibba032.whopeo (mgr.24625) 3354 : cluster [DBG] pgmap v1900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:50 gibba011 bash[31774]: cephadm 2021-03-10T14:55:49.013745+0000 mgr.gibba032.whopeo (mgr.24625) 3353 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:55:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:50 gibba011 bash[31774]: cluster 2021-03-10T14:55:49.153859+0000 mgr.gibba032.whopeo (mgr.24625) 3354 : cluster [DBG] pgmap v1900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:52.323 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:55:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:52 gibba032 bash[24004]: cluster 2021-03-10T14:55:51.155559+0000 mgr.gibba032.whopeo (mgr.24625) 3355 : cluster [DBG] pgmap v1901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:52.552 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:55:52.553 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (62m) 7m ago 67m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:55:52.553 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (67m) 6m ago 67m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:55:52.553 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (62m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:55:52.553 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (63m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:55:52.553 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (62m) 7m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:55:52.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (62m) 7m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:55:52.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (61m) 6m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:55:52.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (61m) 7m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:55:52.554 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (68m) 7m ago 68m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:55:52.555 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (68m) 6m ago 68m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:55:52.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (60m) 7m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:55:52.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (60m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:55:52.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (59m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:55:52.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (59m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:55:52.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (59m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:55:52.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (58m) 6m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:55:52.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (58m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:55:52.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (57m) 6m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:55:52.556 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (67m) 6m ago 68m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:55:52.557 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 65m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:55:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:52 gibba011 bash[30791]: cluster 2021-03-10T14:55:51.155559+0000 mgr.gibba032.whopeo (mgr.24625) 3355 : cluster [DBG] pgmap v1901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:52 gibba011 bash[31774]: cluster 2021-03-10T14:55:51.155559+0000 mgr.gibba032.whopeo (mgr.24625) 3355 : cluster [DBG] pgmap v1901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:52.802 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:55:52.803 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:55:52.803 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:55:52.803 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:52.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:55:52.804 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:55:52.804 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:52.804 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:55:52.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:55:52.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:55:52.805 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:55:52.805 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:55:52.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:55:52.806 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:55:52.806 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:55:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:53 gibba032 bash[24004]: audit 2021-03-10T14:55:52.309990+0000 mgr.gibba032.whopeo (mgr.24625) 3356 : audit [DBG] from='client.46305 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:53 gibba032 bash[24004]: audit 2021-03-10T14:55:52.537088+0000 mgr.gibba032.whopeo (mgr.24625) 3357 : audit [DBG] from='client.46311 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:53 gibba032 bash[24004]: audit 2021-03-10T14:55:52.800631+0000 mon.a (mon.0) 1512 : audit [DBG] from='client.? 172.21.2.111:0/2307507727' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[30791]: audit 2021-03-10T14:55:52.309990+0000 mgr.gibba032.whopeo (mgr.24625) 3356 : audit [DBG] from='client.46305 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[30791]: audit 2021-03-10T14:55:52.537088+0000 mgr.gibba032.whopeo (mgr.24625) 3357 : audit [DBG] from='client.46311 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[30791]: audit 2021-03-10T14:55:52.800631+0000 mon.a (mon.0) 1512 : audit [DBG] from='client.? 172.21.2.111:0/2307507727' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[31774]: audit 2021-03-10T14:55:52.309990+0000 mgr.gibba032.whopeo (mgr.24625) 3356 : audit [DBG] from='client.46305 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[31774]: audit 2021-03-10T14:55:52.537088+0000 mgr.gibba032.whopeo (mgr.24625) 3357 : audit [DBG] from='client.46311 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:55:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:53 gibba011 bash[31774]: audit 2021-03-10T14:55:52.800631+0000 mon.a (mon.0) 1512 : audit [DBG] from='client.? 172.21.2.111:0/2307507727' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:55:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:54 gibba032 bash[24004]: cluster 2021-03-10T14:55:53.157263+0000 mgr.gibba032.whopeo (mgr.24625) 3358 : cluster [DBG] pgmap v1902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:54 gibba011 bash[30791]: cluster 2021-03-10T14:55:53.157263+0000 mgr.gibba032.whopeo (mgr.24625) 3358 : cluster [DBG] pgmap v1902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:54 gibba011 bash[31774]: cluster 2021-03-10T14:55:53.157263+0000 mgr.gibba032.whopeo (mgr.24625) 3358 : cluster [DBG] pgmap v1902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:56 gibba032 bash[24004]: cluster 2021-03-10T14:55:55.158811+0000 mgr.gibba032.whopeo (mgr.24625) 3359 : cluster [DBG] pgmap v1903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:56 gibba011 bash[30791]: cluster 2021-03-10T14:55:55.158811+0000 mgr.gibba032.whopeo (mgr.24625) 3359 : cluster [DBG] pgmap v1903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:56 gibba011 bash[31774]: cluster 2021-03-10T14:55:55.158811+0000 mgr.gibba032.whopeo (mgr.24625) 3359 : cluster [DBG] pgmap v1903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.418684+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:55:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.419404+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:55:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.432831+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:55:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.433610+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.434308+0000 mon.a (mon.0) 1517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[30791]: audit 2021-03-10T14:55:56.435069+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.418684+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:55:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.419404+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:55:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.432831+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:55:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.433610+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.434308+0000 mon.a (mon.0) 1517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:57 gibba011 bash[31774]: audit 2021-03-10T14:55:56.435069+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.418684+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:55:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.419404+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:55:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.432831+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T14:55:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.433610+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.434308+0000 mon.a (mon.0) 1517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:55:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:57 gibba032 bash[24004]: audit 2021-03-10T14:55:56.435069+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T14:55:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:55:58 gibba011 bash[30791]: cluster 2021-03-10T14:55:57.160051+0000 mgr.gibba032.whopeo (mgr.24625) 3360 : cluster [DBG] pgmap v1904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:55:58 gibba011 bash[31774]: cluster 2021-03-10T14:55:57.160051+0000 mgr.gibba032.whopeo (mgr.24625) 3360 : cluster [DBG] pgmap v1904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:55:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:55:58 gibba032 bash[24004]: cluster 2021-03-10T14:55:57.160051+0000 mgr.gibba032.whopeo (mgr.24625) 3360 : cluster [DBG] pgmap v1904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:00 gibba011 bash[30791]: cluster 2021-03-10T14:55:59.161618+0000 mgr.gibba032.whopeo (mgr.24625) 3361 : cluster [DBG] pgmap v1905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:00 gibba011 bash[31774]: cluster 2021-03-10T14:55:59.161618+0000 mgr.gibba032.whopeo (mgr.24625) 3361 : cluster [DBG] pgmap v1905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:00 gibba032 bash[24004]: cluster 2021-03-10T14:55:59.161618+0000 mgr.gibba032.whopeo (mgr.24625) 3361 : cluster [DBG] pgmap v1905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:02 gibba011 bash[30791]: cluster 2021-03-10T14:56:01.163037+0000 mgr.gibba032.whopeo (mgr.24625) 3362 : cluster [DBG] pgmap v1906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:02 gibba011 bash[31774]: cluster 2021-03-10T14:56:01.163037+0000 mgr.gibba032.whopeo (mgr.24625) 3362 : cluster [DBG] pgmap v1906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:02 gibba032 bash[24004]: cluster 2021-03-10T14:56:01.163037+0000 mgr.gibba032.whopeo (mgr.24625) 3362 : cluster [DBG] pgmap v1906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:04 gibba011 bash[30791]: cluster 2021-03-10T14:56:03.164829+0000 mgr.gibba032.whopeo (mgr.24625) 3363 : cluster [DBG] pgmap v1907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:04 gibba011 bash[31774]: cluster 2021-03-10T14:56:03.164829+0000 mgr.gibba032.whopeo (mgr.24625) 3363 : cluster [DBG] pgmap v1907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:04 gibba032 bash[24004]: cluster 2021-03-10T14:56:03.164829+0000 mgr.gibba032.whopeo (mgr.24625) 3363 : cluster [DBG] pgmap v1907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:05 gibba011 bash[30791]: cephadm 2021-03-10T14:56:04.017611+0000 mgr.gibba032.whopeo (mgr.24625) 3364 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:05 gibba011 bash[31774]: cephadm 2021-03-10T14:56:04.017611+0000 mgr.gibba032.whopeo (mgr.24625) 3364 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:05 gibba032 bash[24004]: cephadm 2021-03-10T14:56:04.017611+0000 mgr.gibba032.whopeo (mgr.24625) 3364 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:06 gibba011 bash[30791]: cluster 2021-03-10T14:56:05.166375+0000 mgr.gibba032.whopeo (mgr.24625) 3365 : cluster [DBG] pgmap v1908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:06 gibba011 bash[31774]: cluster 2021-03-10T14:56:05.166375+0000 mgr.gibba032.whopeo (mgr.24625) 3365 : cluster [DBG] pgmap v1908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:06 gibba032 bash[24004]: cluster 2021-03-10T14:56:05.166375+0000 mgr.gibba032.whopeo (mgr.24625) 3365 : cluster [DBG] pgmap v1908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:07 gibba032 bash[24004]: cluster 2021-03-10T14:56:07.167913+0000 mgr.gibba032.whopeo (mgr.24625) 3366 : cluster [DBG] pgmap v1909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:07 gibba011 bash[30791]: cluster 2021-03-10T14:56:07.167913+0000 mgr.gibba032.whopeo (mgr.24625) 3366 : cluster [DBG] pgmap v1909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:07 gibba011 bash[31774]: cluster 2021-03-10T14:56:07.167913+0000 mgr.gibba032.whopeo (mgr.24625) 3366 : cluster [DBG] pgmap v1909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:10 gibba032 bash[24004]: cluster 2021-03-10T14:56:09.169437+0000 mgr.gibba032.whopeo (mgr.24625) 3367 : cluster [DBG] pgmap v1910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:10 gibba011 bash[30791]: cluster 2021-03-10T14:56:09.169437+0000 mgr.gibba032.whopeo (mgr.24625) 3367 : cluster [DBG] pgmap v1910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:10 gibba011 bash[31774]: cluster 2021-03-10T14:56:09.169437+0000 mgr.gibba032.whopeo (mgr.24625) 3367 : cluster [DBG] pgmap v1910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:12 gibba032 bash[24004]: cluster 2021-03-10T14:56:11.171185+0000 mgr.gibba032.whopeo (mgr.24625) 3368 : cluster [DBG] pgmap v1911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:12 gibba011 bash[30791]: cluster 2021-03-10T14:56:11.171185+0000 mgr.gibba032.whopeo (mgr.24625) 3368 : cluster [DBG] pgmap v1911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:12 gibba011 bash[31774]: cluster 2021-03-10T14:56:11.171185+0000 mgr.gibba032.whopeo (mgr.24625) 3368 : cluster [DBG] pgmap v1911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:14 gibba032 bash[24004]: cluster 2021-03-10T14:56:13.173002+0000 mgr.gibba032.whopeo (mgr.24625) 3369 : cluster [DBG] pgmap v1912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:14 gibba011 bash[30791]: cluster 2021-03-10T14:56:13.173002+0000 mgr.gibba032.whopeo (mgr.24625) 3369 : cluster [DBG] pgmap v1912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:14 gibba011 bash[31774]: cluster 2021-03-10T14:56:13.173002+0000 mgr.gibba032.whopeo (mgr.24625) 3369 : cluster [DBG] pgmap v1912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:16 gibba011 bash[30791]: cluster 2021-03-10T14:56:15.174473+0000 mgr.gibba032.whopeo (mgr.24625) 3370 : cluster [DBG] pgmap v1913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:16 gibba011 bash[31774]: cluster 2021-03-10T14:56:15.174473+0000 mgr.gibba032.whopeo (mgr.24625) 3370 : cluster [DBG] pgmap v1913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:16 gibba032 bash[24004]: cluster 2021-03-10T14:56:15.174473+0000 mgr.gibba032.whopeo (mgr.24625) 3370 : cluster [DBG] pgmap v1913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:17 gibba032 bash[24004]: cluster 2021-03-10T14:56:17.175998+0000 mgr.gibba032.whopeo (mgr.24625) 3371 : cluster [DBG] pgmap v1914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:17 gibba011 bash[30791]: cluster 2021-03-10T14:56:17.175998+0000 mgr.gibba032.whopeo (mgr.24625) 3371 : cluster [DBG] pgmap v1914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:17 gibba011 bash[31774]: cluster 2021-03-10T14:56:17.175998+0000 mgr.gibba032.whopeo (mgr.24625) 3371 : cluster [DBG] pgmap v1914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.034251+0000 mgr.gibba032.whopeo (mgr.24625) 3372 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:56:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.035828+0000 mgr.gibba032.whopeo (mgr.24625) 3373 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:56:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cluster 2021-03-10T14:56:19.176517+0000 mgr.gibba032.whopeo (mgr.24625) 3374 : cluster [DBG] pgmap v1915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.250129+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:56:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.250628+0000 mgr.gibba032.whopeo (mgr.24625) 3375 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:56:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.252469+0000 mgr.gibba032.whopeo (mgr.24625) 3376 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:56:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.253655+0000 mon.a (mon.0) 1520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:56:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.254177+0000 mgr.gibba032.whopeo (mgr.24625) 3377 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:56:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.254284+0000 mgr.gibba032.whopeo (mgr.24625) 3378 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:56:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.255373+0000 mon.a (mon.0) 1521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.255821+0000 mgr.gibba032.whopeo (mgr.24625) 3379 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:56:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.255893+0000 mgr.gibba032.whopeo (mgr.24625) 3380 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:56:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.257128+0000 mon.a (mon.0) 1522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.257277+0000 mgr.gibba032.whopeo (mgr.24625) 3381 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:56:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.258349+0000 mgr.gibba032.whopeo (mgr.24625) 3382 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:56:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.258386+0000 mgr.gibba032.whopeo (mgr.24625) 3383 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:56:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.258599+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.259555+0000 mgr.gibba032.whopeo (mgr.24625) 3384 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:56:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.259868+0000 mgr.gibba032.whopeo (mgr.24625) 3385 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:56:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.260979+0000 mgr.gibba032.whopeo (mgr.24625) 3386 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:56:20.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.261067+0000 mgr.gibba032.whopeo (mgr.24625) 3387 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:56:20.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.261123+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: cephadm 2021-03-10T14:56:19.261241+0000 mgr.gibba032.whopeo (mgr.24625) 3388 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:20.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:20 gibba032 bash[24004]: audit 2021-03-10T14:56:19.262498+0000 mon.a (mon.0) 1525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.034251+0000 mgr.gibba032.whopeo (mgr.24625) 3372 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:56:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.035828+0000 mgr.gibba032.whopeo (mgr.24625) 3373 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:56:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cluster 2021-03-10T14:56:19.176517+0000 mgr.gibba032.whopeo (mgr.24625) 3374 : cluster [DBG] pgmap v1915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.250129+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:56:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.250628+0000 mgr.gibba032.whopeo (mgr.24625) 3375 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:56:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.252469+0000 mgr.gibba032.whopeo (mgr.24625) 3376 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:56:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.253655+0000 mon.a (mon.0) 1520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:56:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.254177+0000 mgr.gibba032.whopeo (mgr.24625) 3377 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:56:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.254284+0000 mgr.gibba032.whopeo (mgr.24625) 3378 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:56:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.255373+0000 mon.a (mon.0) 1521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.255821+0000 mgr.gibba032.whopeo (mgr.24625) 3379 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:56:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.255893+0000 mgr.gibba032.whopeo (mgr.24625) 3380 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:56:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.257128+0000 mon.a (mon.0) 1522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.257277+0000 mgr.gibba032.whopeo (mgr.24625) 3381 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:56:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.258349+0000 mgr.gibba032.whopeo (mgr.24625) 3382 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:56:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.258386+0000 mgr.gibba032.whopeo (mgr.24625) 3383 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:56:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.258599+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.259555+0000 mgr.gibba032.whopeo (mgr.24625) 3384 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:56:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.259868+0000 mgr.gibba032.whopeo (mgr.24625) 3385 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:56:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.260979+0000 mgr.gibba032.whopeo (mgr.24625) 3386 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:56:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.261067+0000 mgr.gibba032.whopeo (mgr.24625) 3387 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:56:20.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.034251+0000 mgr.gibba032.whopeo (mgr.24625) 3372 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:56:20.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.035828+0000 mgr.gibba032.whopeo (mgr.24625) 3373 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:56:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cluster 2021-03-10T14:56:19.176517+0000 mgr.gibba032.whopeo (mgr.24625) 3374 : cluster [DBG] pgmap v1915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.250129+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:56:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.250628+0000 mgr.gibba032.whopeo (mgr.24625) 3375 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:56:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.252469+0000 mgr.gibba032.whopeo (mgr.24625) 3376 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.253655+0000 mon.a (mon.0) 1520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.254177+0000 mgr.gibba032.whopeo (mgr.24625) 3377 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.254284+0000 mgr.gibba032.whopeo (mgr.24625) 3378 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.255373+0000 mon.a (mon.0) 1521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.255821+0000 mgr.gibba032.whopeo (mgr.24625) 3379 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:56:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.255893+0000 mgr.gibba032.whopeo (mgr.24625) 3380 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:56:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.257128+0000 mon.a (mon.0) 1522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.257277+0000 mgr.gibba032.whopeo (mgr.24625) 3381 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:56:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.258349+0000 mgr.gibba032.whopeo (mgr.24625) 3382 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:56:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.258386+0000 mgr.gibba032.whopeo (mgr.24625) 3383 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:56:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.258599+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.259555+0000 mgr.gibba032.whopeo (mgr.24625) 3384 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:56:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.259868+0000 mgr.gibba032.whopeo (mgr.24625) 3385 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:56:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.260979+0000 mgr.gibba032.whopeo (mgr.24625) 3386 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:56:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.261067+0000 mgr.gibba032.whopeo (mgr.24625) 3387 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:56:20.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.261123+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: cephadm 2021-03-10T14:56:19.261241+0000 mgr.gibba032.whopeo (mgr.24625) 3388 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:20.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[30791]: audit 2021-03-10T14:56:19.262498+0000 mon.a (mon.0) 1525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.261123+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: cephadm 2021-03-10T14:56:19.261241+0000 mgr.gibba032.whopeo (mgr.24625) 3388 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:20 gibba011 bash[31774]: audit 2021-03-10T14:56:19.262498+0000 mon.a (mon.0) 1525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:22 gibba011 bash[30791]: cluster 2021-03-10T14:56:21.177882+0000 mgr.gibba032.whopeo (mgr.24625) 3389 : cluster [DBG] pgmap v1916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:22 gibba011 bash[31774]: cluster 2021-03-10T14:56:21.177882+0000 mgr.gibba032.whopeo (mgr.24625) 3389 : cluster [DBG] pgmap v1916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:22 gibba032 bash[24004]: cluster 2021-03-10T14:56:21.177882+0000 mgr.gibba032.whopeo (mgr.24625) 3389 : cluster [DBG] pgmap v1916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:23.045 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:56:23.272 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:56:23.273 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (63m) 8m ago 68m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:56:23.273 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (68m) 7m ago 68m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:56:23.273 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (63m) 7m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:56:23.273 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (63m) 7m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:56:23.274 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (62m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:56:23.274 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (62m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:56:23.274 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (62m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:56:23.274 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (62m) 8m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:56:23.275 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (68m) 8m ago 68m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:56:23.275 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (68m) 7m ago 68m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:56:23.275 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (61m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:56:23.275 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (60m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:56:23.275 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (60m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:56:23.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (60m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:56:23.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (59m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:56:23.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (59m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:56:23.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (58m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:56:23.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (58m) 7m ago 68m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:56:23.277 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (67m) 7m ago 68m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:56:23.277 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 66m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:56:23.521 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:56:23.522 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:56:23.522 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:56:23.522 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:23.522 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:56:23.523 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:56:23.523 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:23.523 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:56:23.523 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:56:23.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:23.524 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:56:23.524 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:56:23.524 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:56:23.524 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:56:23.525 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[30791]: audit 2021-03-10T14:56:23.033326+0000 mgr.gibba032.whopeo (mgr.24625) 3390 : audit [DBG] from='client.46323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[30791]: cluster 2021-03-10T14:56:23.179634+0000 mgr.gibba032.whopeo (mgr.24625) 3391 : cluster [DBG] pgmap v1917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[30791]: audit 2021-03-10T14:56:23.256251+0000 mgr.gibba032.whopeo (mgr.24625) 3392 : audit [DBG] from='client.46329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[30791]: audit 2021-03-10T14:56:23.519219+0000 mon.a (mon.0) 1526 : audit [DBG] from='client.? 172.21.2.111:0/2812706525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[31774]: audit 2021-03-10T14:56:23.033326+0000 mgr.gibba032.whopeo (mgr.24625) 3390 : audit [DBG] from='client.46323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[31774]: cluster 2021-03-10T14:56:23.179634+0000 mgr.gibba032.whopeo (mgr.24625) 3391 : cluster [DBG] pgmap v1917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[31774]: audit 2021-03-10T14:56:23.256251+0000 mgr.gibba032.whopeo (mgr.24625) 3392 : audit [DBG] from='client.46329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:24 gibba011 bash[31774]: audit 2021-03-10T14:56:23.519219+0000 mon.a (mon.0) 1526 : audit [DBG] from='client.? 172.21.2.111:0/2812706525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:24 gibba032 bash[24004]: audit 2021-03-10T14:56:23.033326+0000 mgr.gibba032.whopeo (mgr.24625) 3390 : audit [DBG] from='client.46323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:24 gibba032 bash[24004]: cluster 2021-03-10T14:56:23.179634+0000 mgr.gibba032.whopeo (mgr.24625) 3391 : cluster [DBG] pgmap v1917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:24.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:24 gibba032 bash[24004]: audit 2021-03-10T14:56:23.256251+0000 mgr.gibba032.whopeo (mgr.24625) 3392 : audit [DBG] from='client.46329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:24.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:24 gibba032 bash[24004]: audit 2021-03-10T14:56:23.519219+0000 mon.a (mon.0) 1526 : audit [DBG] from='client.? 172.21.2.111:0/2812706525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:26 gibba011 bash[30791]: cluster 2021-03-10T14:56:25.181213+0000 mgr.gibba032.whopeo (mgr.24625) 3393 : cluster [DBG] pgmap v1918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:26 gibba011 bash[31774]: cluster 2021-03-10T14:56:25.181213+0000 mgr.gibba032.whopeo (mgr.24625) 3393 : cluster [DBG] pgmap v1918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:26 gibba032 bash[24004]: cluster 2021-03-10T14:56:25.181213+0000 mgr.gibba032.whopeo (mgr.24625) 3393 : cluster [DBG] pgmap v1918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:27 gibba032 bash[24004]: cluster 2021-03-10T14:56:27.182186+0000 mgr.gibba032.whopeo (mgr.24625) 3394 : cluster [DBG] pgmap v1919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:27 gibba011 bash[30791]: cluster 2021-03-10T14:56:27.182186+0000 mgr.gibba032.whopeo (mgr.24625) 3394 : cluster [DBG] pgmap v1919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:27 gibba011 bash[31774]: cluster 2021-03-10T14:56:27.182186+0000 mgr.gibba032.whopeo (mgr.24625) 3394 : cluster [DBG] pgmap v1919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:30 gibba032 bash[24004]: cluster 2021-03-10T14:56:29.183431+0000 mgr.gibba032.whopeo (mgr.24625) 3395 : cluster [DBG] pgmap v1920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:30 gibba011 bash[30791]: cluster 2021-03-10T14:56:29.183431+0000 mgr.gibba032.whopeo (mgr.24625) 3395 : cluster [DBG] pgmap v1920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:30 gibba011 bash[31774]: cluster 2021-03-10T14:56:29.183431+0000 mgr.gibba032.whopeo (mgr.24625) 3395 : cluster [DBG] pgmap v1920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:32 gibba011 bash[30791]: cluster 2021-03-10T14:56:31.184913+0000 mgr.gibba032.whopeo (mgr.24625) 3396 : cluster [DBG] pgmap v1921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:32 gibba011 bash[31774]: cluster 2021-03-10T14:56:31.184913+0000 mgr.gibba032.whopeo (mgr.24625) 3396 : cluster [DBG] pgmap v1921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:32 gibba032 bash[24004]: cluster 2021-03-10T14:56:31.184913+0000 mgr.gibba032.whopeo (mgr.24625) 3396 : cluster [DBG] pgmap v1921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:34 gibba011 bash[30791]: cluster 2021-03-10T14:56:33.186665+0000 mgr.gibba032.whopeo (mgr.24625) 3397 : cluster [DBG] pgmap v1922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:34 gibba011 bash[31774]: cluster 2021-03-10T14:56:33.186665+0000 mgr.gibba032.whopeo (mgr.24625) 3397 : cluster [DBG] pgmap v1922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:34 gibba032 bash[24004]: cluster 2021-03-10T14:56:33.186665+0000 mgr.gibba032.whopeo (mgr.24625) 3397 : cluster [DBG] pgmap v1922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:35 gibba011 bash[30791]: cephadm 2021-03-10T14:56:34.276766+0000 mgr.gibba032.whopeo (mgr.24625) 3398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:35 gibba011 bash[31774]: cephadm 2021-03-10T14:56:34.276766+0000 mgr.gibba032.whopeo (mgr.24625) 3398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:35 gibba032 bash[24004]: cephadm 2021-03-10T14:56:34.276766+0000 mgr.gibba032.whopeo (mgr.24625) 3398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:36 gibba011 bash[30791]: cluster 2021-03-10T14:56:35.188188+0000 mgr.gibba032.whopeo (mgr.24625) 3399 : cluster [DBG] pgmap v1923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:36 gibba011 bash[31774]: cluster 2021-03-10T14:56:35.188188+0000 mgr.gibba032.whopeo (mgr.24625) 3399 : cluster [DBG] pgmap v1923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:36 gibba032 bash[24004]: cluster 2021-03-10T14:56:35.188188+0000 mgr.gibba032.whopeo (mgr.24625) 3399 : cluster [DBG] pgmap v1923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:37 gibba032 bash[24004]: cluster 2021-03-10T14:56:37.189690+0000 mgr.gibba032.whopeo (mgr.24625) 3400 : cluster [DBG] pgmap v1924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:37 gibba011 bash[30791]: cluster 2021-03-10T14:56:37.189690+0000 mgr.gibba032.whopeo (mgr.24625) 3400 : cluster [DBG] pgmap v1924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:37 gibba011 bash[31774]: cluster 2021-03-10T14:56:37.189690+0000 mgr.gibba032.whopeo (mgr.24625) 3400 : cluster [DBG] pgmap v1924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:40 gibba032 bash[24004]: cluster 2021-03-10T14:56:39.191160+0000 mgr.gibba032.whopeo (mgr.24625) 3401 : cluster [DBG] pgmap v1925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:40 gibba011 bash[30791]: cluster 2021-03-10T14:56:39.191160+0000 mgr.gibba032.whopeo (mgr.24625) 3401 : cluster [DBG] pgmap v1925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:40 gibba011 bash[31774]: cluster 2021-03-10T14:56:39.191160+0000 mgr.gibba032.whopeo (mgr.24625) 3401 : cluster [DBG] pgmap v1925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:42 gibba011 bash[30791]: cluster 2021-03-10T14:56:41.192693+0000 mgr.gibba032.whopeo (mgr.24625) 3402 : cluster [DBG] pgmap v1926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:42 gibba011 bash[31774]: cluster 2021-03-10T14:56:41.192693+0000 mgr.gibba032.whopeo (mgr.24625) 3402 : cluster [DBG] pgmap v1926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:42 gibba032 bash[24004]: cluster 2021-03-10T14:56:41.192693+0000 mgr.gibba032.whopeo (mgr.24625) 3402 : cluster [DBG] pgmap v1926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:44 gibba011 bash[31774]: cluster 2021-03-10T14:56:43.194150+0000 mgr.gibba032.whopeo (mgr.24625) 3403 : cluster [DBG] pgmap v1927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:44 gibba011 bash[30791]: cluster 2021-03-10T14:56:43.194150+0000 mgr.gibba032.whopeo (mgr.24625) 3403 : cluster [DBG] pgmap v1927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:44 gibba032 bash[24004]: cluster 2021-03-10T14:56:43.194150+0000 mgr.gibba032.whopeo (mgr.24625) 3403 : cluster [DBG] pgmap v1927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:46 gibba011 bash[30791]: cluster 2021-03-10T14:56:45.195652+0000 mgr.gibba032.whopeo (mgr.24625) 3404 : cluster [DBG] pgmap v1928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:46 gibba011 bash[31774]: cluster 2021-03-10T14:56:45.195652+0000 mgr.gibba032.whopeo (mgr.24625) 3404 : cluster [DBG] pgmap v1928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:46 gibba032 bash[24004]: cluster 2021-03-10T14:56:45.195652+0000 mgr.gibba032.whopeo (mgr.24625) 3404 : cluster [DBG] pgmap v1928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:47 gibba032 bash[24004]: cluster 2021-03-10T14:56:47.197233+0000 mgr.gibba032.whopeo (mgr.24625) 3405 : cluster [DBG] pgmap v1929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:47 gibba011 bash[30791]: cluster 2021-03-10T14:56:47.197233+0000 mgr.gibba032.whopeo (mgr.24625) 3405 : cluster [DBG] pgmap v1929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:47 gibba011 bash[31774]: cluster 2021-03-10T14:56:47.197233+0000 mgr.gibba032.whopeo (mgr.24625) 3405 : cluster [DBG] pgmap v1929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:50 gibba011 bash[30791]: cluster 2021-03-10T14:56:49.198578+0000 mgr.gibba032.whopeo (mgr.24625) 3406 : cluster [DBG] pgmap v1930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:50 gibba011 bash[30791]: cephadm 2021-03-10T14:56:49.292552+0000 mgr.gibba032.whopeo (mgr.24625) 3407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:50 gibba011 bash[31774]: cluster 2021-03-10T14:56:49.198578+0000 mgr.gibba032.whopeo (mgr.24625) 3406 : cluster [DBG] pgmap v1930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:50 gibba011 bash[31774]: cephadm 2021-03-10T14:56:49.292552+0000 mgr.gibba032.whopeo (mgr.24625) 3407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:50 gibba032 bash[24004]: cluster 2021-03-10T14:56:49.198578+0000 mgr.gibba032.whopeo (mgr.24625) 3406 : cluster [DBG] pgmap v1930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:50 gibba032 bash[24004]: cephadm 2021-03-10T14:56:49.292552+0000 mgr.gibba032.whopeo (mgr.24625) 3407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:56:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:52 gibba011 bash[30791]: cluster 2021-03-10T14:56:51.200081+0000 mgr.gibba032.whopeo (mgr.24625) 3408 : cluster [DBG] pgmap v1931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:52 gibba011 bash[31774]: cluster 2021-03-10T14:56:51.200081+0000 mgr.gibba032.whopeo (mgr.24625) 3408 : cluster [DBG] pgmap v1931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:52 gibba032 bash[24004]: cluster 2021-03-10T14:56:51.200081+0000 mgr.gibba032.whopeo (mgr.24625) 3408 : cluster [DBG] pgmap v1931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:53.765 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:56:53.991 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:56:53.992 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (63m) 8m ago 68m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:56:53.992 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (68m) 7m ago 68m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:56:53.992 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (63m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:56:53.992 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (64m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:56:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (63m) 8m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:56:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (63m) 8m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:56:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (62m) 7m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:56:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (62m) 8m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:56:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (69m) 8m ago 69m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:56:53.994 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (69m) 7m ago 69m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:56:53.994 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (61m) 8m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:56:53.994 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (61m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:56:53.994 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (60m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:56:53.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (60m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:56:53.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (60m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:56:53.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (59m) 7m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:56:53.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (59m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:56:53.995 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (58m) 7m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:56:53.996 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (68m) 7m ago 69m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:56:53.996 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 66m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:56:54.241 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:56:54.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:56:54.242 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:56:54.242 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:54.242 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:56:54.242 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:56:54.242 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:54.243 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:56:54.243 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:56:54.243 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:56:54.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:56:54.243 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:56:54.244 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:56:54.244 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:56:54.244 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:56:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:54 gibba011 bash[30791]: cluster 2021-03-10T14:56:53.201617+0000 mgr.gibba032.whopeo (mgr.24625) 3409 : cluster [DBG] pgmap v1932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:54 gibba011 bash[30791]: audit 2021-03-10T14:56:54.239113+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.2.111:0/2960736933' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:54 gibba011 bash[31774]: cluster 2021-03-10T14:56:53.201617+0000 mgr.gibba032.whopeo (mgr.24625) 3409 : cluster [DBG] pgmap v1932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:54 gibba011 bash[31774]: audit 2021-03-10T14:56:54.239113+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.2.111:0/2960736933' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:54 gibba032 bash[24004]: cluster 2021-03-10T14:56:53.201617+0000 mgr.gibba032.whopeo (mgr.24625) 3409 : cluster [DBG] pgmap v1932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:54 gibba032 bash[24004]: audit 2021-03-10T14:56:54.239113+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.2.111:0/2960736933' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:56:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:55 gibba011 bash[30791]: audit 2021-03-10T14:56:53.752279+0000 mgr.gibba032.whopeo (mgr.24625) 3410 : audit [DBG] from='client.46341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:55 gibba011 bash[30791]: audit 2021-03-10T14:56:53.975344+0000 mgr.gibba032.whopeo (mgr.24625) 3411 : audit [DBG] from='client.46347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:55 gibba011 bash[31774]: audit 2021-03-10T14:56:53.752279+0000 mgr.gibba032.whopeo (mgr.24625) 3410 : audit [DBG] from='client.46341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:55 gibba011 bash[31774]: audit 2021-03-10T14:56:53.975344+0000 mgr.gibba032.whopeo (mgr.24625) 3411 : audit [DBG] from='client.46347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:55.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:55 gibba032 bash[24004]: audit 2021-03-10T14:56:53.752279+0000 mgr.gibba032.whopeo (mgr.24625) 3410 : audit [DBG] from='client.46341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:55.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:55 gibba032 bash[24004]: audit 2021-03-10T14:56:53.975344+0000 mgr.gibba032.whopeo (mgr.24625) 3411 : audit [DBG] from='client.46347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:56:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:56 gibba011 bash[30791]: cluster 2021-03-10T14:56:55.203076+0000 mgr.gibba032.whopeo (mgr.24625) 3412 : cluster [DBG] pgmap v1933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:56 gibba011 bash[31774]: cluster 2021-03-10T14:56:55.203076+0000 mgr.gibba032.whopeo (mgr.24625) 3412 : cluster [DBG] pgmap v1933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:56 gibba032 bash[24004]: cluster 2021-03-10T14:56:55.203076+0000 mgr.gibba032.whopeo (mgr.24625) 3412 : cluster [DBG] pgmap v1933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.425696+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:56:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.426908+0000 mon.a (mon.0) 1529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:56:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.438555+0000 mon.a (mon.0) 1530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:56:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.439296+0000 mon.a (mon.0) 1531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.440256+0000 mon.a (mon.0) 1532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[30791]: audit 2021-03-10T14:56:56.440948+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:56:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.425696+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:56:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.426908+0000 mon.a (mon.0) 1529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:56:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.438555+0000 mon.a (mon.0) 1530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:56:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.439296+0000 mon.a (mon.0) 1531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.440256+0000 mon.a (mon.0) 1532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:57 gibba011 bash[31774]: audit 2021-03-10T14:56:56.440948+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:56:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.425696+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:56:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.426908+0000 mon.a (mon.0) 1529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:56:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.438555+0000 mon.a (mon.0) 1530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T14:56:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.439296+0000 mon.a (mon.0) 1531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.440256+0000 mon.a (mon.0) 1532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:56:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:57 gibba032 bash[24004]: audit 2021-03-10T14:56:56.440948+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T14:56:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:56:58 gibba011 bash[30791]: cluster 2021-03-10T14:56:57.204669+0000 mgr.gibba032.whopeo (mgr.24625) 3413 : cluster [DBG] pgmap v1934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:56:58 gibba011 bash[31774]: cluster 2021-03-10T14:56:57.204669+0000 mgr.gibba032.whopeo (mgr.24625) 3413 : cluster [DBG] pgmap v1934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:56:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:56:58 gibba032 bash[24004]: cluster 2021-03-10T14:56:57.204669+0000 mgr.gibba032.whopeo (mgr.24625) 3413 : cluster [DBG] pgmap v1934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:00 gibba011 bash[30791]: cluster 2021-03-10T14:56:59.206136+0000 mgr.gibba032.whopeo (mgr.24625) 3414 : cluster [DBG] pgmap v1935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:00 gibba011 bash[31774]: cluster 2021-03-10T14:56:59.206136+0000 mgr.gibba032.whopeo (mgr.24625) 3414 : cluster [DBG] pgmap v1935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:00 gibba032 bash[24004]: cluster 2021-03-10T14:56:59.206136+0000 mgr.gibba032.whopeo (mgr.24625) 3414 : cluster [DBG] pgmap v1935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:02 gibba011 bash[30791]: cluster 2021-03-10T14:57:01.207616+0000 mgr.gibba032.whopeo (mgr.24625) 3415 : cluster [DBG] pgmap v1936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:02 gibba011 bash[31774]: cluster 2021-03-10T14:57:01.207616+0000 mgr.gibba032.whopeo (mgr.24625) 3415 : cluster [DBG] pgmap v1936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:02 gibba032 bash[24004]: cluster 2021-03-10T14:57:01.207616+0000 mgr.gibba032.whopeo (mgr.24625) 3415 : cluster [DBG] pgmap v1936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:04 gibba032 bash[24004]: cluster 2021-03-10T14:57:03.209348+0000 mgr.gibba032.whopeo (mgr.24625) 3416 : cluster [DBG] pgmap v1937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:04 gibba011 bash[30791]: cluster 2021-03-10T14:57:03.209348+0000 mgr.gibba032.whopeo (mgr.24625) 3416 : cluster [DBG] pgmap v1937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:04 gibba011 bash[31774]: cluster 2021-03-10T14:57:03.209348+0000 mgr.gibba032.whopeo (mgr.24625) 3416 : cluster [DBG] pgmap v1937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:05 gibba032 bash[24004]: cephadm 2021-03-10T14:57:04.298400+0000 mgr.gibba032.whopeo (mgr.24625) 3417 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:05 gibba011 bash[30791]: cephadm 2021-03-10T14:57:04.298400+0000 mgr.gibba032.whopeo (mgr.24625) 3417 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:05 gibba011 bash[31774]: cephadm 2021-03-10T14:57:04.298400+0000 mgr.gibba032.whopeo (mgr.24625) 3417 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:06 gibba032 bash[24004]: cluster 2021-03-10T14:57:05.210804+0000 mgr.gibba032.whopeo (mgr.24625) 3418 : cluster [DBG] pgmap v1938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:06 gibba011 bash[30791]: cluster 2021-03-10T14:57:05.210804+0000 mgr.gibba032.whopeo (mgr.24625) 3418 : cluster [DBG] pgmap v1938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:06 gibba011 bash[31774]: cluster 2021-03-10T14:57:05.210804+0000 mgr.gibba032.whopeo (mgr.24625) 3418 : cluster [DBG] pgmap v1938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:07 gibba032 bash[24004]: cluster 2021-03-10T14:57:07.212344+0000 mgr.gibba032.whopeo (mgr.24625) 3419 : cluster [DBG] pgmap v1939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:07 gibba011 bash[30791]: cluster 2021-03-10T14:57:07.212344+0000 mgr.gibba032.whopeo (mgr.24625) 3419 : cluster [DBG] pgmap v1939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:07 gibba011 bash[31774]: cluster 2021-03-10T14:57:07.212344+0000 mgr.gibba032.whopeo (mgr.24625) 3419 : cluster [DBG] pgmap v1939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:10 gibba011 bash[30791]: cluster 2021-03-10T14:57:09.213760+0000 mgr.gibba032.whopeo (mgr.24625) 3420 : cluster [DBG] pgmap v1940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:10 gibba011 bash[31774]: cluster 2021-03-10T14:57:09.213760+0000 mgr.gibba032.whopeo (mgr.24625) 3420 : cluster [DBG] pgmap v1940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:10 gibba032 bash[24004]: cluster 2021-03-10T14:57:09.213760+0000 mgr.gibba032.whopeo (mgr.24625) 3420 : cluster [DBG] pgmap v1940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:12 gibba011 bash[30791]: cluster 2021-03-10T14:57:11.215257+0000 mgr.gibba032.whopeo (mgr.24625) 3421 : cluster [DBG] pgmap v1941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:12 gibba011 bash[31774]: cluster 2021-03-10T14:57:11.215257+0000 mgr.gibba032.whopeo (mgr.24625) 3421 : cluster [DBG] pgmap v1941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:12 gibba032 bash[24004]: cluster 2021-03-10T14:57:11.215257+0000 mgr.gibba032.whopeo (mgr.24625) 3421 : cluster [DBG] pgmap v1941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:14 gibba011 bash[30791]: cluster 2021-03-10T14:57:13.217059+0000 mgr.gibba032.whopeo (mgr.24625) 3422 : cluster [DBG] pgmap v1942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:14 gibba011 bash[31774]: cluster 2021-03-10T14:57:13.217059+0000 mgr.gibba032.whopeo (mgr.24625) 3422 : cluster [DBG] pgmap v1942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:14 gibba032 bash[24004]: cluster 2021-03-10T14:57:13.217059+0000 mgr.gibba032.whopeo (mgr.24625) 3422 : cluster [DBG] pgmap v1942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:16 gibba011 bash[30791]: cluster 2021-03-10T14:57:15.218599+0000 mgr.gibba032.whopeo (mgr.24625) 3423 : cluster [DBG] pgmap v1943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:16 gibba011 bash[31774]: cluster 2021-03-10T14:57:15.218599+0000 mgr.gibba032.whopeo (mgr.24625) 3423 : cluster [DBG] pgmap v1943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:16 gibba032 bash[24004]: cluster 2021-03-10T14:57:15.218599+0000 mgr.gibba032.whopeo (mgr.24625) 3423 : cluster [DBG] pgmap v1943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:17 gibba032 bash[24004]: cluster 2021-03-10T14:57:17.220159+0000 mgr.gibba032.whopeo (mgr.24625) 3424 : cluster [DBG] pgmap v1944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:17 gibba011 bash[30791]: cluster 2021-03-10T14:57:17.220159+0000 mgr.gibba032.whopeo (mgr.24625) 3424 : cluster [DBG] pgmap v1944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:17 gibba011 bash[31774]: cluster 2021-03-10T14:57:17.220159+0000 mgr.gibba032.whopeo (mgr.24625) 3424 : cluster [DBG] pgmap v1944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cluster 2021-03-10T14:57:19.221654+0000 mgr.gibba032.whopeo (mgr.24625) 3425 : cluster [DBG] pgmap v1945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.313759+0000 mgr.gibba032.whopeo (mgr.24625) 3426 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:57:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.315209+0000 mgr.gibba032.whopeo (mgr.24625) 3427 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:57:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.508343+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:57:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.508496+0000 mgr.gibba032.whopeo (mgr.24625) 3428 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:57:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.510224+0000 mgr.gibba032.whopeo (mgr.24625) 3429 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:57:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.511462+0000 mon.a (mon.0) 1535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:57:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.511466+0000 mgr.gibba032.whopeo (mgr.24625) 3430 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:57:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.511504+0000 mgr.gibba032.whopeo (mgr.24625) 3431 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:57:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.512839+0000 mon.a (mon.0) 1536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.512950+0000 mgr.gibba032.whopeo (mgr.24625) 3432 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:57:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.512987+0000 mgr.gibba032.whopeo (mgr.24625) 3433 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:57:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.514189+0000 mon.a (mon.0) 1537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.514358+0000 mgr.gibba032.whopeo (mgr.24625) 3434 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:57:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.515590+0000 mgr.gibba032.whopeo (mgr.24625) 3435 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:57:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.515626+0000 mgr.gibba032.whopeo (mgr.24625) 3436 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:57:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.515654+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.517207+0000 mgr.gibba032.whopeo (mgr.24625) 3437 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:57:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.517516+0000 mgr.gibba032.whopeo (mgr.24625) 3438 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:57:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.518413+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.519054+0000 mgr.gibba032.whopeo (mgr.24625) 3439 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:57:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cluster 2021-03-10T14:57:19.221654+0000 mgr.gibba032.whopeo (mgr.24625) 3425 : cluster [DBG] pgmap v1945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.313759+0000 mgr.gibba032.whopeo (mgr.24625) 3426 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:57:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.315209+0000 mgr.gibba032.whopeo (mgr.24625) 3427 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:57:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.508343+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:57:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.508496+0000 mgr.gibba032.whopeo (mgr.24625) 3428 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:57:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.510224+0000 mgr.gibba032.whopeo (mgr.24625) 3429 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:57:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.511462+0000 mon.a (mon.0) 1535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:57:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.511466+0000 mgr.gibba032.whopeo (mgr.24625) 3430 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:57:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.511504+0000 mgr.gibba032.whopeo (mgr.24625) 3431 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:57:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.512839+0000 mon.a (mon.0) 1536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.512950+0000 mgr.gibba032.whopeo (mgr.24625) 3432 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:57:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.512987+0000 mgr.gibba032.whopeo (mgr.24625) 3433 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:57:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.514189+0000 mon.a (mon.0) 1537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.514358+0000 mgr.gibba032.whopeo (mgr.24625) 3434 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:57:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.515590+0000 mgr.gibba032.whopeo (mgr.24625) 3435 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:57:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.515626+0000 mgr.gibba032.whopeo (mgr.24625) 3436 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:57:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.515654+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.517207+0000 mgr.gibba032.whopeo (mgr.24625) 3437 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:57:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.517516+0000 mgr.gibba032.whopeo (mgr.24625) 3438 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:57:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.518413+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.519054+0000 mgr.gibba032.whopeo (mgr.24625) 3439 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:57:20.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.519090+0000 mgr.gibba032.whopeo (mgr.24625) 3440 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:57:20.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: cephadm 2021-03-10T14:57:19.519193+0000 mgr.gibba032.whopeo (mgr.24625) 3441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:20.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[30791]: audit 2021-03-10T14:57:19.520194+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:20.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.519090+0000 mgr.gibba032.whopeo (mgr.24625) 3440 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:57:20.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: cephadm 2021-03-10T14:57:19.519193+0000 mgr.gibba032.whopeo (mgr.24625) 3441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:20 gibba011 bash[31774]: audit 2021-03-10T14:57:19.520194+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cluster 2021-03-10T14:57:19.221654+0000 mgr.gibba032.whopeo (mgr.24625) 3425 : cluster [DBG] pgmap v1945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.313759+0000 mgr.gibba032.whopeo (mgr.24625) 3426 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:57:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.315209+0000 mgr.gibba032.whopeo (mgr.24625) 3427 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:57:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.508343+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:57:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.508496+0000 mgr.gibba032.whopeo (mgr.24625) 3428 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:57:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.510224+0000 mgr.gibba032.whopeo (mgr.24625) 3429 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:57:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.511462+0000 mon.a (mon.0) 1535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:57:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.511466+0000 mgr.gibba032.whopeo (mgr.24625) 3430 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:57:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.511504+0000 mgr.gibba032.whopeo (mgr.24625) 3431 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:57:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.512839+0000 mon.a (mon.0) 1536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.512950+0000 mgr.gibba032.whopeo (mgr.24625) 3432 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:57:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.512987+0000 mgr.gibba032.whopeo (mgr.24625) 3433 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:57:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.514189+0000 mon.a (mon.0) 1537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.514358+0000 mgr.gibba032.whopeo (mgr.24625) 3434 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:57:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.515590+0000 mgr.gibba032.whopeo (mgr.24625) 3435 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:57:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.515626+0000 mgr.gibba032.whopeo (mgr.24625) 3436 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:57:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.515654+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.517207+0000 mgr.gibba032.whopeo (mgr.24625) 3437 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:57:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.517516+0000 mgr.gibba032.whopeo (mgr.24625) 3438 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:57:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.518413+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:21.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.519054+0000 mgr.gibba032.whopeo (mgr.24625) 3439 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:57:21.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.519090+0000 mgr.gibba032.whopeo (mgr.24625) 3440 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:57:21.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: cephadm 2021-03-10T14:57:19.519193+0000 mgr.gibba032.whopeo (mgr.24625) 3441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:21.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:20 gibba032 bash[24004]: audit 2021-03-10T14:57:19.520194+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:22 gibba011 bash[30791]: cluster 2021-03-10T14:57:21.223243+0000 mgr.gibba032.whopeo (mgr.24625) 3442 : cluster [DBG] pgmap v1946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:22.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:22 gibba011 bash[31774]: cluster 2021-03-10T14:57:21.223243+0000 mgr.gibba032.whopeo (mgr.24625) 3442 : cluster [DBG] pgmap v1946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:22 gibba032 bash[24004]: cluster 2021-03-10T14:57:21.223243+0000 mgr.gibba032.whopeo (mgr.24625) 3442 : cluster [DBG] pgmap v1946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:24.483 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:57:24.712 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:57:24.713 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (64m) 9m ago 69m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:57:24.713 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (69m) 8m ago 69m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:57:24.713 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (64m) 8m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:57:24.713 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (64m) 8m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:57:24.714 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (63m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:57:24.714 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (63m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:57:24.714 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (63m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:57:24.714 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (63m) 9m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:57:24.714 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (69m) 9m ago 69m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:57:24.715 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (69m) 8m ago 69m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:57:24.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (62m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:57:24.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (61m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:57:24.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (61m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:57:24.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (61m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:57:24.716 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (60m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:57:24.716 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (60m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:57:24.716 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (59m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:57:24.716 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (59m) 8m ago 69m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:57:24.716 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (68m) 8m ago 69m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:57:24.717 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 67m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:57:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:24 gibba011 bash[30791]: cluster 2021-03-10T14:57:23.225111+0000 mgr.gibba032.whopeo (mgr.24625) 3443 : cluster [DBG] pgmap v1947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:24 gibba011 bash[31774]: cluster 2021-03-10T14:57:23.225111+0000 mgr.gibba032.whopeo (mgr.24625) 3443 : cluster [DBG] pgmap v1947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:24.964 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:57:24.965 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:57:24.965 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:57:24.965 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:24.965 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:57:24.966 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:57:24.966 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:24.966 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:57:24.966 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:57:24.966 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:24.967 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:57:24.967 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:57:24.967 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:57:24.967 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:57:24.967 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:57:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:24 gibba032 bash[24004]: cluster 2021-03-10T14:57:23.225111+0000 mgr.gibba032.whopeo (mgr.24625) 3443 : cluster [DBG] pgmap v1947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[30791]: audit 2021-03-10T14:57:24.469235+0000 mgr.gibba032.whopeo (mgr.24625) 3444 : audit [DBG] from='client.56344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[30791]: audit 2021-03-10T14:57:24.695851+0000 mgr.gibba032.whopeo (mgr.24625) 3445 : audit [DBG] from='client.46365 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[30791]: audit 2021-03-10T14:57:24.962372+0000 mon.a (mon.0) 1541 : audit [DBG] from='client.? 172.21.2.111:0/89886118' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[31774]: audit 2021-03-10T14:57:24.469235+0000 mgr.gibba032.whopeo (mgr.24625) 3444 : audit [DBG] from='client.56344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:25.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[31774]: audit 2021-03-10T14:57:24.695851+0000 mgr.gibba032.whopeo (mgr.24625) 3445 : audit [DBG] from='client.46365 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:25.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:25 gibba011 bash[31774]: audit 2021-03-10T14:57:24.962372+0000 mon.a (mon.0) 1541 : audit [DBG] from='client.? 172.21.2.111:0/89886118' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:25 gibba032 bash[24004]: audit 2021-03-10T14:57:24.469235+0000 mgr.gibba032.whopeo (mgr.24625) 3444 : audit [DBG] from='client.56344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:25 gibba032 bash[24004]: audit 2021-03-10T14:57:24.695851+0000 mgr.gibba032.whopeo (mgr.24625) 3445 : audit [DBG] from='client.46365 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:25 gibba032 bash[24004]: audit 2021-03-10T14:57:24.962372+0000 mon.a (mon.0) 1541 : audit [DBG] from='client.? 172.21.2.111:0/89886118' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:26 gibba011 bash[30791]: cluster 2021-03-10T14:57:25.226443+0000 mgr.gibba032.whopeo (mgr.24625) 3446 : cluster [DBG] pgmap v1948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:26 gibba011 bash[31774]: cluster 2021-03-10T14:57:25.226443+0000 mgr.gibba032.whopeo (mgr.24625) 3446 : cluster [DBG] pgmap v1948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:26 gibba032 bash[24004]: cluster 2021-03-10T14:57:25.226443+0000 mgr.gibba032.whopeo (mgr.24625) 3446 : cluster [DBG] pgmap v1948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:27 gibba011 bash[30791]: cluster 2021-03-10T14:57:27.227897+0000 mgr.gibba032.whopeo (mgr.24625) 3447 : cluster [DBG] pgmap v1949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:27 gibba011 bash[31774]: cluster 2021-03-10T14:57:27.227897+0000 mgr.gibba032.whopeo (mgr.24625) 3447 : cluster [DBG] pgmap v1949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:27 gibba032 bash[24004]: cluster 2021-03-10T14:57:27.227897+0000 mgr.gibba032.whopeo (mgr.24625) 3447 : cluster [DBG] pgmap v1949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:30 gibba011 bash[30791]: cluster 2021-03-10T14:57:29.229438+0000 mgr.gibba032.whopeo (mgr.24625) 3448 : cluster [DBG] pgmap v1950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:30 gibba011 bash[31774]: cluster 2021-03-10T14:57:29.229438+0000 mgr.gibba032.whopeo (mgr.24625) 3448 : cluster [DBG] pgmap v1950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:30 gibba032 bash[24004]: cluster 2021-03-10T14:57:29.229438+0000 mgr.gibba032.whopeo (mgr.24625) 3448 : cluster [DBG] pgmap v1950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:32 gibba011 bash[30791]: cluster 2021-03-10T14:57:31.231034+0000 mgr.gibba032.whopeo (mgr.24625) 3449 : cluster [DBG] pgmap v1951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:32 gibba011 bash[31774]: cluster 2021-03-10T14:57:31.231034+0000 mgr.gibba032.whopeo (mgr.24625) 3449 : cluster [DBG] pgmap v1951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:32 gibba032 bash[24004]: cluster 2021-03-10T14:57:31.231034+0000 mgr.gibba032.whopeo (mgr.24625) 3449 : cluster [DBG] pgmap v1951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:34.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:34 gibba011 bash[30791]: cluster 2021-03-10T14:57:33.232712+0000 mgr.gibba032.whopeo (mgr.24625) 3450 : cluster [DBG] pgmap v1952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:34 gibba011 bash[31774]: cluster 2021-03-10T14:57:33.232712+0000 mgr.gibba032.whopeo (mgr.24625) 3450 : cluster [DBG] pgmap v1952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:34 gibba032 bash[24004]: cluster 2021-03-10T14:57:33.232712+0000 mgr.gibba032.whopeo (mgr.24625) 3450 : cluster [DBG] pgmap v1952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:35 gibba011 bash[30791]: cephadm 2021-03-10T14:57:34.534814+0000 mgr.gibba032.whopeo (mgr.24625) 3451 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:35 gibba011 bash[31774]: cephadm 2021-03-10T14:57:34.534814+0000 mgr.gibba032.whopeo (mgr.24625) 3451 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:35 gibba032 bash[24004]: cephadm 2021-03-10T14:57:34.534814+0000 mgr.gibba032.whopeo (mgr.24625) 3451 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:36 gibba011 bash[30791]: cluster 2021-03-10T14:57:35.234069+0000 mgr.gibba032.whopeo (mgr.24625) 3452 : cluster [DBG] pgmap v1953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:36 gibba011 bash[31774]: cluster 2021-03-10T14:57:35.234069+0000 mgr.gibba032.whopeo (mgr.24625) 3452 : cluster [DBG] pgmap v1953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:36 gibba032 bash[24004]: cluster 2021-03-10T14:57:35.234069+0000 mgr.gibba032.whopeo (mgr.24625) 3452 : cluster [DBG] pgmap v1953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:37 gibba011 bash[30791]: cluster 2021-03-10T14:57:37.235577+0000 mgr.gibba032.whopeo (mgr.24625) 3453 : cluster [DBG] pgmap v1954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:37 gibba011 bash[31774]: cluster 2021-03-10T14:57:37.235577+0000 mgr.gibba032.whopeo (mgr.24625) 3453 : cluster [DBG] pgmap v1954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:37 gibba032 bash[24004]: cluster 2021-03-10T14:57:37.235577+0000 mgr.gibba032.whopeo (mgr.24625) 3453 : cluster [DBG] pgmap v1954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:40 gibba011 bash[30791]: cluster 2021-03-10T14:57:39.237052+0000 mgr.gibba032.whopeo (mgr.24625) 3454 : cluster [DBG] pgmap v1955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:40 gibba011 bash[31774]: cluster 2021-03-10T14:57:39.237052+0000 mgr.gibba032.whopeo (mgr.24625) 3454 : cluster [DBG] pgmap v1955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:40 gibba032 bash[24004]: cluster 2021-03-10T14:57:39.237052+0000 mgr.gibba032.whopeo (mgr.24625) 3454 : cluster [DBG] pgmap v1955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:42 gibba011 bash[30791]: cluster 2021-03-10T14:57:41.238543+0000 mgr.gibba032.whopeo (mgr.24625) 3455 : cluster [DBG] pgmap v1956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:42 gibba011 bash[31774]: cluster 2021-03-10T14:57:41.238543+0000 mgr.gibba032.whopeo (mgr.24625) 3455 : cluster [DBG] pgmap v1956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:42 gibba032 bash[24004]: cluster 2021-03-10T14:57:41.238543+0000 mgr.gibba032.whopeo (mgr.24625) 3455 : cluster [DBG] pgmap v1956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:44 gibba011 bash[30791]: cluster 2021-03-10T14:57:43.240350+0000 mgr.gibba032.whopeo (mgr.24625) 3456 : cluster [DBG] pgmap v1957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:44 gibba011 bash[31774]: cluster 2021-03-10T14:57:43.240350+0000 mgr.gibba032.whopeo (mgr.24625) 3456 : cluster [DBG] pgmap v1957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:44 gibba032 bash[24004]: cluster 2021-03-10T14:57:43.240350+0000 mgr.gibba032.whopeo (mgr.24625) 3456 : cluster [DBG] pgmap v1957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:46 gibba011 bash[30791]: cluster 2021-03-10T14:57:45.241896+0000 mgr.gibba032.whopeo (mgr.24625) 3457 : cluster [DBG] pgmap v1958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:46 gibba011 bash[31774]: cluster 2021-03-10T14:57:45.241896+0000 mgr.gibba032.whopeo (mgr.24625) 3457 : cluster [DBG] pgmap v1958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:46 gibba032 bash[24004]: cluster 2021-03-10T14:57:45.241896+0000 mgr.gibba032.whopeo (mgr.24625) 3457 : cluster [DBG] pgmap v1958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:47 gibba011 bash[30791]: cluster 2021-03-10T14:57:47.243657+0000 mgr.gibba032.whopeo (mgr.24625) 3458 : cluster [DBG] pgmap v1959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:47 gibba011 bash[31774]: cluster 2021-03-10T14:57:47.243657+0000 mgr.gibba032.whopeo (mgr.24625) 3458 : cluster [DBG] pgmap v1959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:47 gibba032 bash[24004]: cluster 2021-03-10T14:57:47.243657+0000 mgr.gibba032.whopeo (mgr.24625) 3458 : cluster [DBG] pgmap v1959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:50 gibba011 bash[30791]: cluster 2021-03-10T14:57:49.245211+0000 mgr.gibba032.whopeo (mgr.24625) 3459 : cluster [DBG] pgmap v1960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:50 gibba011 bash[30791]: cephadm 2021-03-10T14:57:49.550290+0000 mgr.gibba032.whopeo (mgr.24625) 3460 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:50 gibba011 bash[31774]: cluster 2021-03-10T14:57:49.245211+0000 mgr.gibba032.whopeo (mgr.24625) 3459 : cluster [DBG] pgmap v1960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:50 gibba011 bash[31774]: cephadm 2021-03-10T14:57:49.550290+0000 mgr.gibba032.whopeo (mgr.24625) 3460 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:50 gibba032 bash[24004]: cluster 2021-03-10T14:57:49.245211+0000 mgr.gibba032.whopeo (mgr.24625) 3459 : cluster [DBG] pgmap v1960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:50 gibba032 bash[24004]: cephadm 2021-03-10T14:57:49.550290+0000 mgr.gibba032.whopeo (mgr.24625) 3460 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:57:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:52 gibba011 bash[30791]: cluster 2021-03-10T14:57:51.246987+0000 mgr.gibba032.whopeo (mgr.24625) 3461 : cluster [DBG] pgmap v1961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:52 gibba011 bash[31774]: cluster 2021-03-10T14:57:51.246987+0000 mgr.gibba032.whopeo (mgr.24625) 3461 : cluster [DBG] pgmap v1961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:52 gibba032 bash[24004]: cluster 2021-03-10T14:57:51.246987+0000 mgr.gibba032.whopeo (mgr.24625) 3461 : cluster [DBG] pgmap v1961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:54 gibba011 bash[30791]: cluster 2021-03-10T14:57:53.248902+0000 mgr.gibba032.whopeo (mgr.24625) 3462 : cluster [DBG] pgmap v1962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:54 gibba011 bash[31774]: cluster 2021-03-10T14:57:53.248902+0000 mgr.gibba032.whopeo (mgr.24625) 3462 : cluster [DBG] pgmap v1962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:54 gibba032 bash[24004]: cluster 2021-03-10T14:57:53.248902+0000 mgr.gibba032.whopeo (mgr.24625) 3462 : cluster [DBG] pgmap v1962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:55.205 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:57:55.430 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:57:55.431 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (64m) 9m ago 69m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:57:55.431 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (69m) 8m ago 69m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:57:55.431 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (64m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:57:55.431 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (65m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:57:55.432 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (64m) 9m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:57:55.432 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (64m) 9m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:57:55.432 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (63m) 8m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:57:55.432 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (63m) 9m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:57:55.433 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (70m) 9m ago 70m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:57:55.433 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (70m) 8m ago 70m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:57:55.433 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (62m) 9m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:57:55.433 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (62m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:57:55.433 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (62m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:57:55.434 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (61m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:57:55.434 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (61m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:57:55.434 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (60m) 8m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:57:55.434 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (60m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:57:55.435 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (59m) 8m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:57:55.435 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (69m) 8m ago 70m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:57:55.435 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 67m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:57:55.678 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:57:55.678 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:57:55.679 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:57:55.679 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:55.679 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:57:55.679 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:57:55.679 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:57:55.680 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:57:55.681 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:57:55.681 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:57:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:56 gibba032 bash[24004]: audit 2021-03-10T14:57:55.192228+0000 mgr.gibba032.whopeo (mgr.24625) 3463 : audit [DBG] from='client.46377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:56 gibba032 bash[24004]: cluster 2021-03-10T14:57:55.250600+0000 mgr.gibba032.whopeo (mgr.24625) 3464 : cluster [DBG] pgmap v1963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:56 gibba032 bash[24004]: audit 2021-03-10T14:57:55.414262+0000 mgr.gibba032.whopeo (mgr.24625) 3465 : audit [DBG] from='client.46383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:56 gibba032 bash[24004]: audit 2021-03-10T14:57:55.675975+0000 mon.a (mon.0) 1542 : audit [DBG] from='client.? 172.21.2.111:0/887787037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[30791]: audit 2021-03-10T14:57:55.192228+0000 mgr.gibba032.whopeo (mgr.24625) 3463 : audit [DBG] from='client.46377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[30791]: cluster 2021-03-10T14:57:55.250600+0000 mgr.gibba032.whopeo (mgr.24625) 3464 : cluster [DBG] pgmap v1963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[30791]: audit 2021-03-10T14:57:55.414262+0000 mgr.gibba032.whopeo (mgr.24625) 3465 : audit [DBG] from='client.46383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[30791]: audit 2021-03-10T14:57:55.675975+0000 mon.a (mon.0) 1542 : audit [DBG] from='client.? 172.21.2.111:0/887787037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[31774]: audit 2021-03-10T14:57:55.192228+0000 mgr.gibba032.whopeo (mgr.24625) 3463 : audit [DBG] from='client.46377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[31774]: cluster 2021-03-10T14:57:55.250600+0000 mgr.gibba032.whopeo (mgr.24625) 3464 : cluster [DBG] pgmap v1963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[31774]: audit 2021-03-10T14:57:55.414262+0000 mgr.gibba032.whopeo (mgr.24625) 3465 : audit [DBG] from='client.46383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:57:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:56 gibba011 bash[31774]: audit 2021-03-10T14:57:55.675975+0000 mon.a (mon.0) 1542 : audit [DBG] from='client.? 172.21.2.111:0/887787037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:57:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.431227+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:57:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.431989+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:57:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.444759+0000 mon.a (mon.0) 1545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:57:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.445658+0000 mon.a (mon.0) 1546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:57:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.446393+0000 mon.a (mon.0) 1547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:57:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:57 gibba032 bash[24004]: audit 2021-03-10T14:57:56.447119+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:57:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.431227+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:57:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.431989+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:57:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.444759+0000 mon.a (mon.0) 1545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:57:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.445658+0000 mon.a (mon.0) 1546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:57:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.446393+0000 mon.a (mon.0) 1547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:57:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[30791]: audit 2021-03-10T14:57:56.447119+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:57:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.431227+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:57:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.431989+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:57:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.444759+0000 mon.a (mon.0) 1545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T14:57:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.445658+0000 mon.a (mon.0) 1546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T14:57:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.446393+0000 mon.a (mon.0) 1547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T14:57:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:57 gibba011 bash[31774]: audit 2021-03-10T14:57:56.447119+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T14:57:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:57:58 gibba032 bash[24004]: cluster 2021-03-10T14:57:57.252476+0000 mgr.gibba032.whopeo (mgr.24625) 3466 : cluster [DBG] pgmap v1964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:57:58 gibba011 bash[30791]: cluster 2021-03-10T14:57:57.252476+0000 mgr.gibba032.whopeo (mgr.24625) 3466 : cluster [DBG] pgmap v1964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:57:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:57:58 gibba011 bash[31774]: cluster 2021-03-10T14:57:57.252476+0000 mgr.gibba032.whopeo (mgr.24625) 3466 : cluster [DBG] pgmap v1964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:00 gibba032 bash[24004]: cluster 2021-03-10T14:57:59.253572+0000 mgr.gibba032.whopeo (mgr.24625) 3467 : cluster [DBG] pgmap v1965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:00 gibba011 bash[30791]: cluster 2021-03-10T14:57:59.253572+0000 mgr.gibba032.whopeo (mgr.24625) 3467 : cluster [DBG] pgmap v1965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:00 gibba011 bash[31774]: cluster 2021-03-10T14:57:59.253572+0000 mgr.gibba032.whopeo (mgr.24625) 3467 : cluster [DBG] pgmap v1965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:02 gibba032 bash[24004]: cluster 2021-03-10T14:58:01.255105+0000 mgr.gibba032.whopeo (mgr.24625) 3468 : cluster [DBG] pgmap v1966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:02 gibba011 bash[30791]: cluster 2021-03-10T14:58:01.255105+0000 mgr.gibba032.whopeo (mgr.24625) 3468 : cluster [DBG] pgmap v1966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:02 gibba011 bash[31774]: cluster 2021-03-10T14:58:01.255105+0000 mgr.gibba032.whopeo (mgr.24625) 3468 : cluster [DBG] pgmap v1966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:04 gibba032 bash[24004]: cluster 2021-03-10T14:58:03.256952+0000 mgr.gibba032.whopeo (mgr.24625) 3469 : cluster [DBG] pgmap v1967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:04 gibba011 bash[31774]: cluster 2021-03-10T14:58:03.256952+0000 mgr.gibba032.whopeo (mgr.24625) 3469 : cluster [DBG] pgmap v1967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:04 gibba011 bash[30791]: cluster 2021-03-10T14:58:03.256952+0000 mgr.gibba032.whopeo (mgr.24625) 3469 : cluster [DBG] pgmap v1967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:05.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:05 gibba032 bash[24004]: cephadm 2021-03-10T14:58:04.566389+0000 mgr.gibba032.whopeo (mgr.24625) 3470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:05 gibba011 bash[30791]: cephadm 2021-03-10T14:58:04.566389+0000 mgr.gibba032.whopeo (mgr.24625) 3470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:05 gibba011 bash[31774]: cephadm 2021-03-10T14:58:04.566389+0000 mgr.gibba032.whopeo (mgr.24625) 3470 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:06 gibba032 bash[24004]: cluster 2021-03-10T14:58:05.258422+0000 mgr.gibba032.whopeo (mgr.24625) 3471 : cluster [DBG] pgmap v1968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:06 gibba011 bash[30791]: cluster 2021-03-10T14:58:05.258422+0000 mgr.gibba032.whopeo (mgr.24625) 3471 : cluster [DBG] pgmap v1968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:06 gibba011 bash[31774]: cluster 2021-03-10T14:58:05.258422+0000 mgr.gibba032.whopeo (mgr.24625) 3471 : cluster [DBG] pgmap v1968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:07 gibba011 bash[30791]: cluster 2021-03-10T14:58:07.260169+0000 mgr.gibba032.whopeo (mgr.24625) 3472 : cluster [DBG] pgmap v1969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:07 gibba011 bash[31774]: cluster 2021-03-10T14:58:07.260169+0000 mgr.gibba032.whopeo (mgr.24625) 3472 : cluster [DBG] pgmap v1969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:07 gibba032 bash[24004]: cluster 2021-03-10T14:58:07.260169+0000 mgr.gibba032.whopeo (mgr.24625) 3472 : cluster [DBG] pgmap v1969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:10 gibba011 bash[30791]: cluster 2021-03-10T14:58:09.261520+0000 mgr.gibba032.whopeo (mgr.24625) 3473 : cluster [DBG] pgmap v1970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:10 gibba011 bash[31774]: cluster 2021-03-10T14:58:09.261520+0000 mgr.gibba032.whopeo (mgr.24625) 3473 : cluster [DBG] pgmap v1970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:10 gibba032 bash[24004]: cluster 2021-03-10T14:58:09.261520+0000 mgr.gibba032.whopeo (mgr.24625) 3473 : cluster [DBG] pgmap v1970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:12 gibba032 bash[24004]: cluster 2021-03-10T14:58:11.263124+0000 mgr.gibba032.whopeo (mgr.24625) 3474 : cluster [DBG] pgmap v1971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:12 gibba011 bash[30791]: cluster 2021-03-10T14:58:11.263124+0000 mgr.gibba032.whopeo (mgr.24625) 3474 : cluster [DBG] pgmap v1971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:12 gibba011 bash[31774]: cluster 2021-03-10T14:58:11.263124+0000 mgr.gibba032.whopeo (mgr.24625) 3474 : cluster [DBG] pgmap v1971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:14 gibba032 bash[24004]: cluster 2021-03-10T14:58:13.264501+0000 mgr.gibba032.whopeo (mgr.24625) 3475 : cluster [DBG] pgmap v1972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:14 gibba011 bash[30791]: cluster 2021-03-10T14:58:13.264501+0000 mgr.gibba032.whopeo (mgr.24625) 3475 : cluster [DBG] pgmap v1972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:14 gibba011 bash[31774]: cluster 2021-03-10T14:58:13.264501+0000 mgr.gibba032.whopeo (mgr.24625) 3475 : cluster [DBG] pgmap v1972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:16 gibba032 bash[24004]: cluster 2021-03-10T14:58:15.265948+0000 mgr.gibba032.whopeo (mgr.24625) 3476 : cluster [DBG] pgmap v1973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:16 gibba011 bash[30791]: cluster 2021-03-10T14:58:15.265948+0000 mgr.gibba032.whopeo (mgr.24625) 3476 : cluster [DBG] pgmap v1973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:16 gibba011 bash[31774]: cluster 2021-03-10T14:58:15.265948+0000 mgr.gibba032.whopeo (mgr.24625) 3476 : cluster [DBG] pgmap v1973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:17 gibba011 bash[30791]: cluster 2021-03-10T14:58:17.267711+0000 mgr.gibba032.whopeo (mgr.24625) 3477 : cluster [DBG] pgmap v1974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:17 gibba011 bash[31774]: cluster 2021-03-10T14:58:17.267711+0000 mgr.gibba032.whopeo (mgr.24625) 3477 : cluster [DBG] pgmap v1974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:17 gibba032 bash[24004]: cluster 2021-03-10T14:58:17.267711+0000 mgr.gibba032.whopeo (mgr.24625) 3477 : cluster [DBG] pgmap v1974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:20 gibba032 bash[24004]: cluster 2021-03-10T14:58:19.269069+0000 mgr.gibba032.whopeo (mgr.24625) 3478 : cluster [DBG] pgmap v1975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:20 gibba032 bash[24004]: cephadm 2021-03-10T14:58:19.584133+0000 mgr.gibba032.whopeo (mgr.24625) 3479 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:58:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:20 gibba032 bash[24004]: audit 2021-03-10T14:58:19.586999+0000 mon.a (mon.0) 1549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:58:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[30791]: cluster 2021-03-10T14:58:19.269069+0000 mgr.gibba032.whopeo (mgr.24625) 3478 : cluster [DBG] pgmap v1975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[30791]: cephadm 2021-03-10T14:58:19.584133+0000 mgr.gibba032.whopeo (mgr.24625) 3479 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:58:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[30791]: audit 2021-03-10T14:58:19.586999+0000 mon.a (mon.0) 1549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:58:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[31774]: cluster 2021-03-10T14:58:19.269069+0000 mgr.gibba032.whopeo (mgr.24625) 3478 : cluster [DBG] pgmap v1975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[31774]: cephadm 2021-03-10T14:58:19.584133+0000 mgr.gibba032.whopeo (mgr.24625) 3479 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:58:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:20 gibba011 bash[31774]: audit 2021-03-10T14:58:19.586999+0000 mon.a (mon.0) 1549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:58:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:22 gibba032 bash[24004]: cluster 2021-03-10T14:58:21.270653+0000 mgr.gibba032.whopeo (mgr.24625) 3480 : cluster [DBG] pgmap v1976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:22 gibba011 bash[30791]: cluster 2021-03-10T14:58:21.270653+0000 mgr.gibba032.whopeo (mgr.24625) 3480 : cluster [DBG] pgmap v1976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:22 gibba011 bash[31774]: cluster 2021-03-10T14:58:21.270653+0000 mgr.gibba032.whopeo (mgr.24625) 3480 : cluster [DBG] pgmap v1976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.122230+0000 mgr.gibba032.whopeo (mgr.24625) 3481 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:58:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.123852+0000 mon.a (mon.0) 1550 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cluster 2021-03-10T14:58:23.272270+0000 mgr.gibba032.whopeo (mgr.24625) 3482 : cluster [DBG] pgmap v1977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.326281+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.330834+0000 mgr.gibba032.whopeo (mgr.24625) 3483 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:58:24.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.333241+0000 mgr.gibba032.whopeo (mgr.24625) 3484 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:58:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.333777+0000 mon.a (mon.0) 1552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:58:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.334955+0000 mgr.gibba032.whopeo (mgr.24625) 3485 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:58:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.334992+0000 mgr.gibba032.whopeo (mgr.24625) 3486 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:58:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.336115+0000 mon.a (mon.0) 1553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.336437+0000 mgr.gibba032.whopeo (mgr.24625) 3487 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:58:24.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.336473+0000 mgr.gibba032.whopeo (mgr.24625) 3488 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:58:24.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.337642+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.337769+0000 mgr.gibba032.whopeo (mgr.24625) 3489 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:58:24.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.339043+0000 mon.a (mon.0) 1555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.339110+0000 mgr.gibba032.whopeo (mgr.24625) 3490 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:58:24.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.339150+0000 mgr.gibba032.whopeo (mgr.24625) 3491 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:58:24.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.340729+0000 mgr.gibba032.whopeo (mgr.24625) 3492 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:58:24.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.341143+0000 mgr.gibba032.whopeo (mgr.24625) 3493 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:58:24.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.341890+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.342896+0000 mgr.gibba032.whopeo (mgr.24625) 3494 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:58:24.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.342934+0000 mgr.gibba032.whopeo (mgr.24625) 3495 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:58:24.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: cephadm 2021-03-10T14:58:23.343057+0000 mgr.gibba032.whopeo (mgr.24625) 3496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:24.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:24 gibba032 bash[24004]: audit 2021-03-10T14:58:23.343680+0000 mon.a (mon.0) 1557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.122230+0000 mgr.gibba032.whopeo (mgr.24625) 3481 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:58:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.123852+0000 mon.a (mon.0) 1550 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cluster 2021-03-10T14:58:23.272270+0000 mgr.gibba032.whopeo (mgr.24625) 3482 : cluster [DBG] pgmap v1977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.326281+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.330834+0000 mgr.gibba032.whopeo (mgr.24625) 3483 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:58:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.333241+0000 mgr.gibba032.whopeo (mgr.24625) 3484 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:58:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.333777+0000 mon.a (mon.0) 1552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:58:24.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.334955+0000 mgr.gibba032.whopeo (mgr.24625) 3485 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:58:24.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.334992+0000 mgr.gibba032.whopeo (mgr.24625) 3486 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:58:24.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.336115+0000 mon.a (mon.0) 1553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.336437+0000 mgr.gibba032.whopeo (mgr.24625) 3487 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:58:24.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.336473+0000 mgr.gibba032.whopeo (mgr.24625) 3488 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:58:24.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.337642+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.337769+0000 mgr.gibba032.whopeo (mgr.24625) 3489 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:58:24.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.339043+0000 mon.a (mon.0) 1555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.339110+0000 mgr.gibba032.whopeo (mgr.24625) 3490 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:58:24.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.339150+0000 mgr.gibba032.whopeo (mgr.24625) 3491 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:58:24.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.340729+0000 mgr.gibba032.whopeo (mgr.24625) 3492 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:58:24.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.341143+0000 mgr.gibba032.whopeo (mgr.24625) 3493 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:58:24.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.341890+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.342896+0000 mgr.gibba032.whopeo (mgr.24625) 3494 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:58:24.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.122230+0000 mgr.gibba032.whopeo (mgr.24625) 3481 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:58:24.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.123852+0000 mon.a (mon.0) 1550 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cluster 2021-03-10T14:58:23.272270+0000 mgr.gibba032.whopeo (mgr.24625) 3482 : cluster [DBG] pgmap v1977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:24.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.326281+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:58:24.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.330834+0000 mgr.gibba032.whopeo (mgr.24625) 3483 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:58:24.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.333241+0000 mgr.gibba032.whopeo (mgr.24625) 3484 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:58:24.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.333777+0000 mon.a (mon.0) 1552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:58:24.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.334955+0000 mgr.gibba032.whopeo (mgr.24625) 3485 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:58:24.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.334992+0000 mgr.gibba032.whopeo (mgr.24625) 3486 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:58:24.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.336115+0000 mon.a (mon.0) 1553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.336437+0000 mgr.gibba032.whopeo (mgr.24625) 3487 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:58:24.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.336473+0000 mgr.gibba032.whopeo (mgr.24625) 3488 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:58:24.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.337642+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.337769+0000 mgr.gibba032.whopeo (mgr.24625) 3489 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:58:24.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.339043+0000 mon.a (mon.0) 1555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.339110+0000 mgr.gibba032.whopeo (mgr.24625) 3490 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:58:24.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.339150+0000 mgr.gibba032.whopeo (mgr.24625) 3491 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:58:24.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.340729+0000 mgr.gibba032.whopeo (mgr.24625) 3492 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:58:24.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.341143+0000 mgr.gibba032.whopeo (mgr.24625) 3493 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:58:24.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.341890+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.342896+0000 mgr.gibba032.whopeo (mgr.24625) 3494 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:58:24.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.342934+0000 mgr.gibba032.whopeo (mgr.24625) 3495 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:58:24.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: cephadm 2021-03-10T14:58:23.343057+0000 mgr.gibba032.whopeo (mgr.24625) 3496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:24.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[30791]: audit 2021-03-10T14:58:23.343680+0000 mon.a (mon.0) 1557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:24.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.342934+0000 mgr.gibba032.whopeo (mgr.24625) 3495 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:58:24.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: cephadm 2021-03-10T14:58:23.343057+0000 mgr.gibba032.whopeo (mgr.24625) 3496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:24.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:24 gibba011 bash[31774]: audit 2021-03-10T14:58:23.343680+0000 mon.a (mon.0) 1557 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:25.921 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:58:26.150 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:58:26.150 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (65m) 3s ago 70m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:58:26.151 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (70m) 9m ago 70m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:58:26.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (65m) 9m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:58:26.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (65m) 9m ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:58:26.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (64m) 3s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:58:26.151 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (64m) 3s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:58:26.152 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (64m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:58:26.152 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (64m) 3s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:58:26.152 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (70m) 3s ago 70m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:58:26.152 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (70m) 9m ago 70m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:58:26.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (63m) 3s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:58:26.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (62m) 3s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:58:26.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (62m) 3s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:58:26.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (62m) 3s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:58:26.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (61m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:58:26.154 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (61m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:58:26.154 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (60m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:58:26.154 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (60m) 9m ago 70m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:58:26.154 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (70m) 9m ago 70m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:58:26.154 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3s ago 68m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:58:26.404 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:58:26.404 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:58:26.404 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:58:26.405 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:26.405 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:58:26.405 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:58:26.405 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:26.405 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:58:26.406 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:58:26.406 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:26.406 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:58:26.406 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:58:26.407 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:58:26.407 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:58:26.407 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:58:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:26 gibba032 bash[24004]: cluster 2021-03-10T14:58:25.273908+0000 mgr.gibba032.whopeo (mgr.24625) 3497 : cluster [DBG] pgmap v1978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:26 gibba011 bash[30791]: cluster 2021-03-10T14:58:25.273908+0000 mgr.gibba032.whopeo (mgr.24625) 3497 : cluster [DBG] pgmap v1978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:26 gibba011 bash[31774]: cluster 2021-03-10T14:58:25.273908+0000 mgr.gibba032.whopeo (mgr.24625) 3497 : cluster [DBG] pgmap v1978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:27.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:27 gibba032 bash[24004]: audit 2021-03-10T14:58:25.907690+0000 mgr.gibba032.whopeo (mgr.24625) 3498 : audit [DBG] from='client.56380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:27 gibba032 bash[24004]: audit 2021-03-10T14:58:26.133664+0000 mgr.gibba032.whopeo (mgr.24625) 3499 : audit [DBG] from='client.46401 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:27 gibba032 bash[24004]: audit 2021-03-10T14:58:26.401656+0000 mon.c (mon.1) 65 : audit [DBG] from='client.? 172.21.2.111:0/4160455851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[30791]: audit 2021-03-10T14:58:25.907690+0000 mgr.gibba032.whopeo (mgr.24625) 3498 : audit [DBG] from='client.56380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[30791]: audit 2021-03-10T14:58:26.133664+0000 mgr.gibba032.whopeo (mgr.24625) 3499 : audit [DBG] from='client.46401 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[30791]: audit 2021-03-10T14:58:26.401656+0000 mon.c (mon.1) 65 : audit [DBG] from='client.? 172.21.2.111:0/4160455851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:27.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[31774]: audit 2021-03-10T14:58:25.907690+0000 mgr.gibba032.whopeo (mgr.24625) 3498 : audit [DBG] from='client.56380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[31774]: audit 2021-03-10T14:58:26.133664+0000 mgr.gibba032.whopeo (mgr.24625) 3499 : audit [DBG] from='client.46401 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:27.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:27 gibba011 bash[31774]: audit 2021-03-10T14:58:26.401656+0000 mon.c (mon.1) 65 : audit [DBG] from='client.? 172.21.2.111:0/4160455851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:28 gibba032 bash[24004]: cluster 2021-03-10T14:58:27.275681+0000 mgr.gibba032.whopeo (mgr.24625) 3500 : cluster [DBG] pgmap v1979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:28 gibba011 bash[30791]: cluster 2021-03-10T14:58:27.275681+0000 mgr.gibba032.whopeo (mgr.24625) 3500 : cluster [DBG] pgmap v1979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:28 gibba011 bash[31774]: cluster 2021-03-10T14:58:27.275681+0000 mgr.gibba032.whopeo (mgr.24625) 3500 : cluster [DBG] pgmap v1979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:30 gibba032 bash[24004]: cluster 2021-03-10T14:58:29.276941+0000 mgr.gibba032.whopeo (mgr.24625) 3501 : cluster [DBG] pgmap v1980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:30 gibba011 bash[30791]: cluster 2021-03-10T14:58:29.276941+0000 mgr.gibba032.whopeo (mgr.24625) 3501 : cluster [DBG] pgmap v1980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:30 gibba011 bash[31774]: cluster 2021-03-10T14:58:29.276941+0000 mgr.gibba032.whopeo (mgr.24625) 3501 : cluster [DBG] pgmap v1980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:32 gibba032 bash[24004]: cluster 2021-03-10T14:58:31.278766+0000 mgr.gibba032.whopeo (mgr.24625) 3502 : cluster [DBG] pgmap v1981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:32 gibba011 bash[30791]: cluster 2021-03-10T14:58:31.278766+0000 mgr.gibba032.whopeo (mgr.24625) 3502 : cluster [DBG] pgmap v1981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:32 gibba011 bash[31774]: cluster 2021-03-10T14:58:31.278766+0000 mgr.gibba032.whopeo (mgr.24625) 3502 : cluster [DBG] pgmap v1981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:34 gibba032 bash[24004]: cluster 2021-03-10T14:58:33.280403+0000 mgr.gibba032.whopeo (mgr.24625) 3503 : cluster [DBG] pgmap v1982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:34 gibba011 bash[30791]: cluster 2021-03-10T14:58:33.280403+0000 mgr.gibba032.whopeo (mgr.24625) 3503 : cluster [DBG] pgmap v1982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:34 gibba011 bash[31774]: cluster 2021-03-10T14:58:33.280403+0000 mgr.gibba032.whopeo (mgr.24625) 3503 : cluster [DBG] pgmap v1982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:36 gibba032 bash[24004]: cluster 2021-03-10T14:58:35.281782+0000 mgr.gibba032.whopeo (mgr.24625) 3504 : cluster [DBG] pgmap v1983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:36 gibba011 bash[30791]: cluster 2021-03-10T14:58:35.281782+0000 mgr.gibba032.whopeo (mgr.24625) 3504 : cluster [DBG] pgmap v1983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:36 gibba011 bash[31774]: cluster 2021-03-10T14:58:35.281782+0000 mgr.gibba032.whopeo (mgr.24625) 3504 : cluster [DBG] pgmap v1983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:37 gibba011 bash[30791]: cluster 2021-03-10T14:58:37.283254+0000 mgr.gibba032.whopeo (mgr.24625) 3505 : cluster [DBG] pgmap v1984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:37 gibba011 bash[31774]: cluster 2021-03-10T14:58:37.283254+0000 mgr.gibba032.whopeo (mgr.24625) 3505 : cluster [DBG] pgmap v1984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:37 gibba032 bash[24004]: cluster 2021-03-10T14:58:37.283254+0000 mgr.gibba032.whopeo (mgr.24625) 3505 : cluster [DBG] pgmap v1984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:38 gibba011 bash[30791]: cephadm 2021-03-10T14:58:38.358870+0000 mgr.gibba032.whopeo (mgr.24625) 3506 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:39.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:38 gibba011 bash[31774]: cephadm 2021-03-10T14:58:38.358870+0000 mgr.gibba032.whopeo (mgr.24625) 3506 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:38 gibba032 bash[24004]: cephadm 2021-03-10T14:58:38.358870+0000 mgr.gibba032.whopeo (mgr.24625) 3506 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:39 gibba011 bash[30791]: cluster 2021-03-10T14:58:39.284389+0000 mgr.gibba032.whopeo (mgr.24625) 3507 : cluster [DBG] pgmap v1985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:39 gibba011 bash[31774]: cluster 2021-03-10T14:58:39.284389+0000 mgr.gibba032.whopeo (mgr.24625) 3507 : cluster [DBG] pgmap v1985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:39 gibba032 bash[24004]: cluster 2021-03-10T14:58:39.284389+0000 mgr.gibba032.whopeo (mgr.24625) 3507 : cluster [DBG] pgmap v1985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:42 gibba032 bash[24004]: cluster 2021-03-10T14:58:41.286023+0000 mgr.gibba032.whopeo (mgr.24625) 3508 : cluster [DBG] pgmap v1986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:42 gibba011 bash[30791]: cluster 2021-03-10T14:58:41.286023+0000 mgr.gibba032.whopeo (mgr.24625) 3508 : cluster [DBG] pgmap v1986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:42 gibba011 bash[31774]: cluster 2021-03-10T14:58:41.286023+0000 mgr.gibba032.whopeo (mgr.24625) 3508 : cluster [DBG] pgmap v1986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:44 gibba032 bash[24004]: cluster 2021-03-10T14:58:43.287687+0000 mgr.gibba032.whopeo (mgr.24625) 3509 : cluster [DBG] pgmap v1987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:44 gibba011 bash[30791]: cluster 2021-03-10T14:58:43.287687+0000 mgr.gibba032.whopeo (mgr.24625) 3509 : cluster [DBG] pgmap v1987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:44 gibba011 bash[31774]: cluster 2021-03-10T14:58:43.287687+0000 mgr.gibba032.whopeo (mgr.24625) 3509 : cluster [DBG] pgmap v1987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:46 gibba032 bash[24004]: cluster 2021-03-10T14:58:45.289090+0000 mgr.gibba032.whopeo (mgr.24625) 3510 : cluster [DBG] pgmap v1988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:46 gibba011 bash[30791]: cluster 2021-03-10T14:58:45.289090+0000 mgr.gibba032.whopeo (mgr.24625) 3510 : cluster [DBG] pgmap v1988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:46 gibba011 bash[31774]: cluster 2021-03-10T14:58:45.289090+0000 mgr.gibba032.whopeo (mgr.24625) 3510 : cluster [DBG] pgmap v1988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:47 gibba011 bash[30791]: cluster 2021-03-10T14:58:47.290906+0000 mgr.gibba032.whopeo (mgr.24625) 3511 : cluster [DBG] pgmap v1989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:47 gibba011 bash[31774]: cluster 2021-03-10T14:58:47.290906+0000 mgr.gibba032.whopeo (mgr.24625) 3511 : cluster [DBG] pgmap v1989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:47 gibba032 bash[24004]: cluster 2021-03-10T14:58:47.290906+0000 mgr.gibba032.whopeo (mgr.24625) 3511 : cluster [DBG] pgmap v1989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:50 gibba032 bash[24004]: cluster 2021-03-10T14:58:49.292355+0000 mgr.gibba032.whopeo (mgr.24625) 3512 : cluster [DBG] pgmap v1990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:50 gibba011 bash[30791]: cluster 2021-03-10T14:58:49.292355+0000 mgr.gibba032.whopeo (mgr.24625) 3512 : cluster [DBG] pgmap v1990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:50 gibba011 bash[31774]: cluster 2021-03-10T14:58:49.292355+0000 mgr.gibba032.whopeo (mgr.24625) 3512 : cluster [DBG] pgmap v1990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:52 gibba032 bash[24004]: cluster 2021-03-10T14:58:51.294256+0000 mgr.gibba032.whopeo (mgr.24625) 3513 : cluster [DBG] pgmap v1991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:52 gibba011 bash[30791]: cluster 2021-03-10T14:58:51.294256+0000 mgr.gibba032.whopeo (mgr.24625) 3513 : cluster [DBG] pgmap v1991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:52 gibba011 bash[31774]: cluster 2021-03-10T14:58:51.294256+0000 mgr.gibba032.whopeo (mgr.24625) 3513 : cluster [DBG] pgmap v1991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:54 gibba032 bash[24004]: cluster 2021-03-10T14:58:53.295829+0000 mgr.gibba032.whopeo (mgr.24625) 3514 : cluster [DBG] pgmap v1992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:54 gibba032 bash[24004]: cephadm 2021-03-10T14:58:53.372809+0000 mgr.gibba032.whopeo (mgr.24625) 3515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:54 gibba011 bash[30791]: cluster 2021-03-10T14:58:53.295829+0000 mgr.gibba032.whopeo (mgr.24625) 3514 : cluster [DBG] pgmap v1992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:54 gibba011 bash[30791]: cephadm 2021-03-10T14:58:53.372809+0000 mgr.gibba032.whopeo (mgr.24625) 3515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:54 gibba011 bash[31774]: cluster 2021-03-10T14:58:53.295829+0000 mgr.gibba032.whopeo (mgr.24625) 3514 : cluster [DBG] pgmap v1992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:54 gibba011 bash[31774]: cephadm 2021-03-10T14:58:53.372809+0000 mgr.gibba032.whopeo (mgr.24625) 3515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:58:56.647 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:58:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:56 gibba032 bash[24004]: cluster 2021-03-10T14:58:55.297413+0000 mgr.gibba032.whopeo (mgr.24625) 3516 : cluster [DBG] pgmap v1993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:56 gibba011 bash[30791]: cluster 2021-03-10T14:58:55.297413+0000 mgr.gibba032.whopeo (mgr.24625) 3516 : cluster [DBG] pgmap v1993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:56 gibba011 bash[31774]: cluster 2021-03-10T14:58:55.297413+0000 mgr.gibba032.whopeo (mgr.24625) 3516 : cluster [DBG] pgmap v1993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:56.903 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:58:56.904 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (65m) 33s ago 70m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:58:56.904 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (70m) 9m ago 70m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:58:56.904 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (65m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:58:56.905 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (66m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:58:56.905 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (65m) 33s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:58:56.905 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (65m) 33s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:58:56.905 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (64m) 9m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:58:56.905 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (64m) 33s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:58:56.906 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (71m) 33s ago 71m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:58:56.906 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (71m) 9m ago 71m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:58:56.906 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (63m) 33s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:58:56.906 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (63m) 33s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:58:56.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (63m) 33s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:58:56.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (62m) 33s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:58:56.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (62m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:58:56.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (61m) 9m ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:58:56.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (61m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:58:56.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (60m) 9m ago 71m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:58:56.908 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (70m) 9m ago 71m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:58:56.908 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 33s ago 69m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:58:57.211 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:58:57.211 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:58:57.211 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:58:57.212 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:57.212 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:58:57.212 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:58:57.212 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:57.213 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:58:57.213 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:58:57.213 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:58:57.213 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:58:57.213 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:58:57.214 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:58:57.214 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:58:57.214 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:58:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.434662+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:58:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.434809+0000 mon.a (mon.0) 1559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:58:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.450523+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:58:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.450662+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:58:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.450782+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.450912+0000 mon.a (mon.0) 1563 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:56.633646+0000 mgr.gibba032.whopeo (mgr.24625) 3517 : audit [DBG] from='client.56398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:57 gibba032 bash[24004]: audit 2021-03-10T14:58:57.208838+0000 mon.a (mon.0) 1564 : audit [DBG] from='client.? 172.21.2.111:0/2399758581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.434662+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:58:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.434809+0000 mon.a (mon.0) 1559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:58:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.450523+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:58:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.450662+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:58:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.450782+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.450912+0000 mon.a (mon.0) 1563 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:56.633646+0000 mgr.gibba032.whopeo (mgr.24625) 3517 : audit [DBG] from='client.56398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[30791]: audit 2021-03-10T14:58:57.208838+0000 mon.a (mon.0) 1564 : audit [DBG] from='client.? 172.21.2.111:0/2399758581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.434662+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:58:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.434809+0000 mon.a (mon.0) 1559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:58:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.450523+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T14:58:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.450662+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T14:58:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.450782+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.450912+0000 mon.a (mon.0) 1563 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T14:58:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:56.633646+0000 mgr.gibba032.whopeo (mgr.24625) 3517 : audit [DBG] from='client.56398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:57 gibba011 bash[31774]: audit 2021-03-10T14:58:57.208838+0000 mon.a (mon.0) 1564 : audit [DBG] from='client.? 172.21.2.111:0/2399758581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:58:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:58 gibba032 bash[24004]: audit 2021-03-10T14:58:56.857482+0000 mgr.gibba032.whopeo (mgr.24625) 3518 : audit [DBG] from='client.56404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:58:58 gibba032 bash[24004]: cluster 2021-03-10T14:58:57.299228+0000 mgr.gibba032.whopeo (mgr.24625) 3519 : cluster [DBG] pgmap v1994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:58 gibba011 bash[30791]: audit 2021-03-10T14:58:56.857482+0000 mgr.gibba032.whopeo (mgr.24625) 3518 : audit [DBG] from='client.56404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:58:58 gibba011 bash[30791]: cluster 2021-03-10T14:58:57.299228+0000 mgr.gibba032.whopeo (mgr.24625) 3519 : cluster [DBG] pgmap v1994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:58:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:58 gibba011 bash[31774]: audit 2021-03-10T14:58:56.857482+0000 mgr.gibba032.whopeo (mgr.24625) 3518 : audit [DBG] from='client.56404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:58:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:58:58 gibba011 bash[31774]: cluster 2021-03-10T14:58:57.299228+0000 mgr.gibba032.whopeo (mgr.24625) 3519 : cluster [DBG] pgmap v1994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:00 gibba032 bash[24004]: cluster 2021-03-10T14:58:59.300576+0000 mgr.gibba032.whopeo (mgr.24625) 3520 : cluster [DBG] pgmap v1995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:00 gibba011 bash[30791]: cluster 2021-03-10T14:58:59.300576+0000 mgr.gibba032.whopeo (mgr.24625) 3520 : cluster [DBG] pgmap v1995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:00 gibba011 bash[31774]: cluster 2021-03-10T14:58:59.300576+0000 mgr.gibba032.whopeo (mgr.24625) 3520 : cluster [DBG] pgmap v1995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:02 gibba032 bash[24004]: cluster 2021-03-10T14:59:01.302133+0000 mgr.gibba032.whopeo (mgr.24625) 3521 : cluster [DBG] pgmap v1996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:02 gibba011 bash[30791]: cluster 2021-03-10T14:59:01.302133+0000 mgr.gibba032.whopeo (mgr.24625) 3521 : cluster [DBG] pgmap v1996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:02 gibba011 bash[31774]: cluster 2021-03-10T14:59:01.302133+0000 mgr.gibba032.whopeo (mgr.24625) 3521 : cluster [DBG] pgmap v1996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:04 gibba032 bash[24004]: cluster 2021-03-10T14:59:03.303736+0000 mgr.gibba032.whopeo (mgr.24625) 3522 : cluster [DBG] pgmap v1997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:04 gibba011 bash[30791]: cluster 2021-03-10T14:59:03.303736+0000 mgr.gibba032.whopeo (mgr.24625) 3522 : cluster [DBG] pgmap v1997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:04 gibba011 bash[31774]: cluster 2021-03-10T14:59:03.303736+0000 mgr.gibba032.whopeo (mgr.24625) 3522 : cluster [DBG] pgmap v1997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:06 gibba032 bash[24004]: cluster 2021-03-10T14:59:05.305248+0000 mgr.gibba032.whopeo (mgr.24625) 3523 : cluster [DBG] pgmap v1998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:06 gibba011 bash[30791]: cluster 2021-03-10T14:59:05.305248+0000 mgr.gibba032.whopeo (mgr.24625) 3523 : cluster [DBG] pgmap v1998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:06 gibba011 bash[31774]: cluster 2021-03-10T14:59:05.305248+0000 mgr.gibba032.whopeo (mgr.24625) 3523 : cluster [DBG] pgmap v1998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:07 gibba011 bash[30791]: cluster 2021-03-10T14:59:07.306983+0000 mgr.gibba032.whopeo (mgr.24625) 3524 : cluster [DBG] pgmap v1999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:07 gibba011 bash[31774]: cluster 2021-03-10T14:59:07.306983+0000 mgr.gibba032.whopeo (mgr.24625) 3524 : cluster [DBG] pgmap v1999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:07 gibba032 bash[24004]: cluster 2021-03-10T14:59:07.306983+0000 mgr.gibba032.whopeo (mgr.24625) 3524 : cluster [DBG] pgmap v1999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:09.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:08 gibba011 bash[30791]: cephadm 2021-03-10T14:59:08.388886+0000 mgr.gibba032.whopeo (mgr.24625) 3525 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:09.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:08 gibba011 bash[31774]: cephadm 2021-03-10T14:59:08.388886+0000 mgr.gibba032.whopeo (mgr.24625) 3525 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:09.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:08 gibba032 bash[24004]: cephadm 2021-03-10T14:59:08.388886+0000 mgr.gibba032.whopeo (mgr.24625) 3525 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:09 gibba032 bash[24004]: cluster 2021-03-10T14:59:09.308323+0000 mgr.gibba032.whopeo (mgr.24625) 3526 : cluster [DBG] pgmap v2000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:09 gibba011 bash[30791]: cluster 2021-03-10T14:59:09.308323+0000 mgr.gibba032.whopeo (mgr.24625) 3526 : cluster [DBG] pgmap v2000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:09 gibba011 bash[31774]: cluster 2021-03-10T14:59:09.308323+0000 mgr.gibba032.whopeo (mgr.24625) 3526 : cluster [DBG] pgmap v2000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:12 gibba032 bash[24004]: cluster 2021-03-10T14:59:11.310036+0000 mgr.gibba032.whopeo (mgr.24625) 3527 : cluster [DBG] pgmap v2001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:12 gibba011 bash[30791]: cluster 2021-03-10T14:59:11.310036+0000 mgr.gibba032.whopeo (mgr.24625) 3527 : cluster [DBG] pgmap v2001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:12 gibba011 bash[31774]: cluster 2021-03-10T14:59:11.310036+0000 mgr.gibba032.whopeo (mgr.24625) 3527 : cluster [DBG] pgmap v2001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:14 gibba032 bash[24004]: cluster 2021-03-10T14:59:13.311619+0000 mgr.gibba032.whopeo (mgr.24625) 3528 : cluster [DBG] pgmap v2002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:14 gibba011 bash[30791]: cluster 2021-03-10T14:59:13.311619+0000 mgr.gibba032.whopeo (mgr.24625) 3528 : cluster [DBG] pgmap v2002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:14 gibba011 bash[31774]: cluster 2021-03-10T14:59:13.311619+0000 mgr.gibba032.whopeo (mgr.24625) 3528 : cluster [DBG] pgmap v2002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:16 gibba032 bash[24004]: cluster 2021-03-10T14:59:15.313041+0000 mgr.gibba032.whopeo (mgr.24625) 3529 : cluster [DBG] pgmap v2003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:16 gibba011 bash[30791]: cluster 2021-03-10T14:59:15.313041+0000 mgr.gibba032.whopeo (mgr.24625) 3529 : cluster [DBG] pgmap v2003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:16 gibba011 bash[31774]: cluster 2021-03-10T14:59:15.313041+0000 mgr.gibba032.whopeo (mgr.24625) 3529 : cluster [DBG] pgmap v2003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:17 gibba011 bash[30791]: cluster 2021-03-10T14:59:17.314665+0000 mgr.gibba032.whopeo (mgr.24625) 3530 : cluster [DBG] pgmap v2004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:17 gibba011 bash[31774]: cluster 2021-03-10T14:59:17.314665+0000 mgr.gibba032.whopeo (mgr.24625) 3530 : cluster [DBG] pgmap v2004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:17 gibba032 bash[24004]: cluster 2021-03-10T14:59:17.314665+0000 mgr.gibba032.whopeo (mgr.24625) 3530 : cluster [DBG] pgmap v2004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:20 gibba032 bash[24004]: cluster 2021-03-10T14:59:19.316071+0000 mgr.gibba032.whopeo (mgr.24625) 3531 : cluster [DBG] pgmap v2005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:20 gibba011 bash[30791]: cluster 2021-03-10T14:59:19.316071+0000 mgr.gibba032.whopeo (mgr.24625) 3531 : cluster [DBG] pgmap v2005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:20 gibba011 bash[31774]: cluster 2021-03-10T14:59:19.316071+0000 mgr.gibba032.whopeo (mgr.24625) 3531 : cluster [DBG] pgmap v2005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:22 gibba032 bash[24004]: cluster 2021-03-10T14:59:21.317701+0000 mgr.gibba032.whopeo (mgr.24625) 3532 : cluster [DBG] pgmap v2006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:22 gibba011 bash[30791]: cluster 2021-03-10T14:59:21.317701+0000 mgr.gibba032.whopeo (mgr.24625) 3532 : cluster [DBG] pgmap v2006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:22 gibba011 bash[31774]: cluster 2021-03-10T14:59:21.317701+0000 mgr.gibba032.whopeo (mgr.24625) 3532 : cluster [DBG] pgmap v2006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:24 gibba032 bash[24004]: cluster 2021-03-10T14:59:23.319039+0000 mgr.gibba032.whopeo (mgr.24625) 3533 : cluster [DBG] pgmap v2007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:24 gibba032 bash[24004]: cephadm 2021-03-10T14:59:23.405484+0000 mgr.gibba032.whopeo (mgr.24625) 3534 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:59:24.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:24 gibba032 bash[24004]: audit 2021-03-10T14:59:23.409199+0000 mon.a (mon.0) 1565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:59:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[30791]: cluster 2021-03-10T14:59:23.319039+0000 mgr.gibba032.whopeo (mgr.24625) 3533 : cluster [DBG] pgmap v2007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[30791]: cephadm 2021-03-10T14:59:23.405484+0000 mgr.gibba032.whopeo (mgr.24625) 3534 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:59:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[30791]: audit 2021-03-10T14:59:23.409199+0000 mon.a (mon.0) 1565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:59:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[31774]: cluster 2021-03-10T14:59:23.319039+0000 mgr.gibba032.whopeo (mgr.24625) 3533 : cluster [DBG] pgmap v2007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[31774]: cephadm 2021-03-10T14:59:23.405484+0000 mgr.gibba032.whopeo (mgr.24625) 3534 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T14:59:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:24 gibba011 bash[31774]: audit 2021-03-10T14:59:23.409199+0000 mon.a (mon.0) 1565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T14:59:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:26 gibba032 bash[24004]: cluster 2021-03-10T14:59:25.320348+0000 mgr.gibba032.whopeo (mgr.24625) 3535 : cluster [DBG] pgmap v2008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:26 gibba011 bash[30791]: cluster 2021-03-10T14:59:25.320348+0000 mgr.gibba032.whopeo (mgr.24625) 3535 : cluster [DBG] pgmap v2008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:26 gibba011 bash[31774]: cluster 2021-03-10T14:59:25.320348+0000 mgr.gibba032.whopeo (mgr.24625) 3535 : cluster [DBG] pgmap v2008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:27.460 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:59:27.688 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:59:27.688 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (66m) 64s ago 71m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:59:27.688 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (71m) 0s ago 71m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:59:27.689 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (66m) 0s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:59:27.689 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (66m) 0s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:59:27.689 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (65m) 64s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:59:27.689 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (65m) 64s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:59:27.689 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (65m) 0s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:59:27.690 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (65m) 64s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:59:27.690 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (71m) 64s ago 71m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:59:27.690 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (71m) 0s ago 71m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:59:27.690 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (64m) 64s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:59:27.690 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (63m) 64s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:59:27.691 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (63m) 64s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:59:27.691 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (63m) 64s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:59:27.691 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (62m) 0s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:59:27.691 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (62m) 0s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:59:27.691 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (61m) 0s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:59:27.692 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (61m) 0s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:59:27.692 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (71m) 0s ago 71m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:59:27.692 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 64s ago 69m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:59:27.943 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:59:27.944 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:59:27.944 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:59:27.944 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:27.944 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:59:27.944 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:59:27.945 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:27.945 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:59:27.945 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:59:27.945 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:27.945 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:59:27.946 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:59:27.946 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:59:27.946 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:59:27.946 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:59:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.119394+0000 mgr.gibba032.whopeo (mgr.24625) 3536 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:59:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.120366+0000 mon.a (mon.0) 1566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.313023+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.313612+0000 mgr.gibba032.whopeo (mgr.24625) 3537 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:59:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.315193+0000 mgr.gibba032.whopeo (mgr.24625) 3538 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:59:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.316302+0000 mgr.gibba032.whopeo (mgr.24625) 3539 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:59:28.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.316367+0000 mgr.gibba032.whopeo (mgr.24625) 3540 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:59:28.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.316450+0000 mon.a (mon.0) 1568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:59:28.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.317317+0000 mgr.gibba032.whopeo (mgr.24625) 3541 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:59:28.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.317407+0000 mgr.gibba032.whopeo (mgr.24625) 3542 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:59:28.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.317684+0000 mon.a (mon.0) 1569 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.318406+0000 mgr.gibba032.whopeo (mgr.24625) 3543 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:59:28.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.318834+0000 mon.a (mon.0) 1570 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.319270+0000 mgr.gibba032.whopeo (mgr.24625) 3544 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:59:28.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.319329+0000 mgr.gibba032.whopeo (mgr.24625) 3545 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:59:28.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.319813+0000 mon.a (mon.0) 1571 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.320797+0000 mgr.gibba032.whopeo (mgr.24625) 3546 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:59:28.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cluster 2021-03-10T14:59:27.320978+0000 mgr.gibba032.whopeo (mgr.24625) 3547 : cluster [DBG] pgmap v2009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:28.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.321248+0000 mgr.gibba032.whopeo (mgr.24625) 3548 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:59:28.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.321893+0000 mon.a (mon.0) 1572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.322316+0000 mgr.gibba032.whopeo (mgr.24625) 3549 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:59:28.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.322460+0000 mgr.gibba032.whopeo (mgr.24625) 3550 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:59:28.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: cephadm 2021-03-10T14:59:27.322656+0000 mgr.gibba032.whopeo (mgr.24625) 3551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:28.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.323737+0000 mon.a (mon.0) 1573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.446918+0000 mgr.gibba032.whopeo (mgr.24625) 3552 : audit [DBG] from='client.56416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.671565+0000 mgr.gibba032.whopeo (mgr.24625) 3553 : audit [DBG] from='client.46437 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:28 gibba032 bash[24004]: audit 2021-03-10T14:59:27.941419+0000 mon.c (mon.1) 66 : audit [DBG] from='client.? 172.21.2.111:0/804962663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.119394+0000 mgr.gibba032.whopeo (mgr.24625) 3536 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:59:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.120366+0000 mon.a (mon.0) 1566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.313023+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.313612+0000 mgr.gibba032.whopeo (mgr.24625) 3537 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:59:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.315193+0000 mgr.gibba032.whopeo (mgr.24625) 3538 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:59:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.316302+0000 mgr.gibba032.whopeo (mgr.24625) 3539 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:59:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.316367+0000 mgr.gibba032.whopeo (mgr.24625) 3540 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:59:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.316450+0000 mon.a (mon.0) 1568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:59:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.317317+0000 mgr.gibba032.whopeo (mgr.24625) 3541 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:59:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.317407+0000 mgr.gibba032.whopeo (mgr.24625) 3542 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:59:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.317684+0000 mon.a (mon.0) 1569 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.318406+0000 mgr.gibba032.whopeo (mgr.24625) 3543 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:59:28.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.318834+0000 mon.a (mon.0) 1570 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.319270+0000 mgr.gibba032.whopeo (mgr.24625) 3544 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:59:28.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.319329+0000 mgr.gibba032.whopeo (mgr.24625) 3545 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:59:28.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.319813+0000 mon.a (mon.0) 1571 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.320797+0000 mgr.gibba032.whopeo (mgr.24625) 3546 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:59:28.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cluster 2021-03-10T14:59:27.320978+0000 mgr.gibba032.whopeo (mgr.24625) 3547 : cluster [DBG] pgmap v2009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:28.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.321248+0000 mgr.gibba032.whopeo (mgr.24625) 3548 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:59:28.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.321893+0000 mon.a (mon.0) 1572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.322316+0000 mgr.gibba032.whopeo (mgr.24625) 3549 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:59:28.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.119394+0000 mgr.gibba032.whopeo (mgr.24625) 3536 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T14:59:28.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.120366+0000 mon.a (mon.0) 1566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.313023+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T14:59:28.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.313612+0000 mgr.gibba032.whopeo (mgr.24625) 3537 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T14:59:28.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.315193+0000 mgr.gibba032.whopeo (mgr.24625) 3538 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T14:59:28.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.316302+0000 mgr.gibba032.whopeo (mgr.24625) 3539 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T14:59:28.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.316367+0000 mgr.gibba032.whopeo (mgr.24625) 3540 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T14:59:28.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.316450+0000 mon.a (mon.0) 1568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T14:59:28.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.317317+0000 mgr.gibba032.whopeo (mgr.24625) 3541 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T14:59:28.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.317407+0000 mgr.gibba032.whopeo (mgr.24625) 3542 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T14:59:28.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.317684+0000 mon.a (mon.0) 1569 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.318406+0000 mgr.gibba032.whopeo (mgr.24625) 3543 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T14:59:28.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.318834+0000 mon.a (mon.0) 1570 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.319270+0000 mgr.gibba032.whopeo (mgr.24625) 3544 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T14:59:28.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.319329+0000 mgr.gibba032.whopeo (mgr.24625) 3545 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T14:59:28.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.319813+0000 mon.a (mon.0) 1571 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.320797+0000 mgr.gibba032.whopeo (mgr.24625) 3546 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T14:59:28.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cluster 2021-03-10T14:59:27.320978+0000 mgr.gibba032.whopeo (mgr.24625) 3547 : cluster [DBG] pgmap v2009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:28.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.321248+0000 mgr.gibba032.whopeo (mgr.24625) 3548 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T14:59:28.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.321893+0000 mon.a (mon.0) 1572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.322316+0000 mgr.gibba032.whopeo (mgr.24625) 3549 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T14:59:28.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.322460+0000 mgr.gibba032.whopeo (mgr.24625) 3550 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:59:28.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: cephadm 2021-03-10T14:59:27.322656+0000 mgr.gibba032.whopeo (mgr.24625) 3551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:28.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.323737+0000 mon.a (mon.0) 1573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.446918+0000 mgr.gibba032.whopeo (mgr.24625) 3552 : audit [DBG] from='client.56416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.671565+0000 mgr.gibba032.whopeo (mgr.24625) 3553 : audit [DBG] from='client.46437 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[30791]: audit 2021-03-10T14:59:27.941419+0000 mon.c (mon.1) 66 : audit [DBG] from='client.? 172.21.2.111:0/804962663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.322460+0000 mgr.gibba032.whopeo (mgr.24625) 3550 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T14:59:28.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: cephadm 2021-03-10T14:59:27.322656+0000 mgr.gibba032.whopeo (mgr.24625) 3551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:28.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.323737+0000 mon.a (mon.0) 1573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:28.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.446918+0000 mgr.gibba032.whopeo (mgr.24625) 3552 : audit [DBG] from='client.56416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.671565+0000 mgr.gibba032.whopeo (mgr.24625) 3553 : audit [DBG] from='client.46437 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:28.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:28 gibba011 bash[31774]: audit 2021-03-10T14:59:27.941419+0000 mon.c (mon.1) 66 : audit [DBG] from='client.? 172.21.2.111:0/804962663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:30 gibba032 bash[24004]: cluster 2021-03-10T14:59:29.321936+0000 mgr.gibba032.whopeo (mgr.24625) 3554 : cluster [DBG] pgmap v2010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:30 gibba011 bash[30791]: cluster 2021-03-10T14:59:29.321936+0000 mgr.gibba032.whopeo (mgr.24625) 3554 : cluster [DBG] pgmap v2010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:30 gibba011 bash[31774]: cluster 2021-03-10T14:59:29.321936+0000 mgr.gibba032.whopeo (mgr.24625) 3554 : cluster [DBG] pgmap v2010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:32 gibba032 bash[24004]: cluster 2021-03-10T14:59:31.323559+0000 mgr.gibba032.whopeo (mgr.24625) 3555 : cluster [DBG] pgmap v2011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:32 gibba011 bash[30791]: cluster 2021-03-10T14:59:31.323559+0000 mgr.gibba032.whopeo (mgr.24625) 3555 : cluster [DBG] pgmap v2011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:32 gibba011 bash[31774]: cluster 2021-03-10T14:59:31.323559+0000 mgr.gibba032.whopeo (mgr.24625) 3555 : cluster [DBG] pgmap v2011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:34 gibba032 bash[24004]: cluster 2021-03-10T14:59:33.325092+0000 mgr.gibba032.whopeo (mgr.24625) 3556 : cluster [DBG] pgmap v2012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:34 gibba011 bash[30791]: cluster 2021-03-10T14:59:33.325092+0000 mgr.gibba032.whopeo (mgr.24625) 3556 : cluster [DBG] pgmap v2012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:34 gibba011 bash[31774]: cluster 2021-03-10T14:59:33.325092+0000 mgr.gibba032.whopeo (mgr.24625) 3556 : cluster [DBG] pgmap v2012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:36 gibba032 bash[24004]: cluster 2021-03-10T14:59:35.326380+0000 mgr.gibba032.whopeo (mgr.24625) 3557 : cluster [DBG] pgmap v2013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:36 gibba011 bash[30791]: cluster 2021-03-10T14:59:35.326380+0000 mgr.gibba032.whopeo (mgr.24625) 3557 : cluster [DBG] pgmap v2013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:36 gibba011 bash[31774]: cluster 2021-03-10T14:59:35.326380+0000 mgr.gibba032.whopeo (mgr.24625) 3557 : cluster [DBG] pgmap v2013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:37 gibba032 bash[24004]: cluster 2021-03-10T14:59:37.328070+0000 mgr.gibba032.whopeo (mgr.24625) 3558 : cluster [DBG] pgmap v2014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:37 gibba011 bash[30791]: cluster 2021-03-10T14:59:37.328070+0000 mgr.gibba032.whopeo (mgr.24625) 3558 : cluster [DBG] pgmap v2014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:37 gibba011 bash[31774]: cluster 2021-03-10T14:59:37.328070+0000 mgr.gibba032.whopeo (mgr.24625) 3558 : cluster [DBG] pgmap v2014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:40 gibba032 bash[24004]: cluster 2021-03-10T14:59:39.329098+0000 mgr.gibba032.whopeo (mgr.24625) 3559 : cluster [DBG] pgmap v2015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:40 gibba011 bash[30791]: cluster 2021-03-10T14:59:39.329098+0000 mgr.gibba032.whopeo (mgr.24625) 3559 : cluster [DBG] pgmap v2015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:40 gibba011 bash[31774]: cluster 2021-03-10T14:59:39.329098+0000 mgr.gibba032.whopeo (mgr.24625) 3559 : cluster [DBG] pgmap v2015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:42 gibba032 bash[24004]: cluster 2021-03-10T14:59:41.330722+0000 mgr.gibba032.whopeo (mgr.24625) 3560 : cluster [DBG] pgmap v2016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:42 gibba011 bash[30791]: cluster 2021-03-10T14:59:41.330722+0000 mgr.gibba032.whopeo (mgr.24625) 3560 : cluster [DBG] pgmap v2016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:42 gibba011 bash[31774]: cluster 2021-03-10T14:59:41.330722+0000 mgr.gibba032.whopeo (mgr.24625) 3560 : cluster [DBG] pgmap v2016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:43 gibba032 bash[24004]: cephadm 2021-03-10T14:59:42.338663+0000 mgr.gibba032.whopeo (mgr.24625) 3561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:43 gibba011 bash[30791]: cephadm 2021-03-10T14:59:42.338663+0000 mgr.gibba032.whopeo (mgr.24625) 3561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:43 gibba011 bash[31774]: cephadm 2021-03-10T14:59:42.338663+0000 mgr.gibba032.whopeo (mgr.24625) 3561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:44 gibba032 bash[24004]: cluster 2021-03-10T14:59:43.332117+0000 mgr.gibba032.whopeo (mgr.24625) 3562 : cluster [DBG] pgmap v2017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:44 gibba011 bash[30791]: cluster 2021-03-10T14:59:43.332117+0000 mgr.gibba032.whopeo (mgr.24625) 3562 : cluster [DBG] pgmap v2017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:44 gibba011 bash[31774]: cluster 2021-03-10T14:59:43.332117+0000 mgr.gibba032.whopeo (mgr.24625) 3562 : cluster [DBG] pgmap v2017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:46 gibba032 bash[24004]: cluster 2021-03-10T14:59:45.333689+0000 mgr.gibba032.whopeo (mgr.24625) 3563 : cluster [DBG] pgmap v2018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:46 gibba011 bash[30791]: cluster 2021-03-10T14:59:45.333689+0000 mgr.gibba032.whopeo (mgr.24625) 3563 : cluster [DBG] pgmap v2018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:46 gibba011 bash[31774]: cluster 2021-03-10T14:59:45.333689+0000 mgr.gibba032.whopeo (mgr.24625) 3563 : cluster [DBG] pgmap v2018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:47 gibba032 bash[24004]: cluster 2021-03-10T14:59:47.335257+0000 mgr.gibba032.whopeo (mgr.24625) 3564 : cluster [DBG] pgmap v2019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:47 gibba011 bash[30791]: cluster 2021-03-10T14:59:47.335257+0000 mgr.gibba032.whopeo (mgr.24625) 3564 : cluster [DBG] pgmap v2019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:48.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:47 gibba011 bash[31774]: cluster 2021-03-10T14:59:47.335257+0000 mgr.gibba032.whopeo (mgr.24625) 3564 : cluster [DBG] pgmap v2019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:50 gibba032 bash[24004]: cluster 2021-03-10T14:59:49.336632+0000 mgr.gibba032.whopeo (mgr.24625) 3565 : cluster [DBG] pgmap v2020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:50 gibba011 bash[30791]: cluster 2021-03-10T14:59:49.336632+0000 mgr.gibba032.whopeo (mgr.24625) 3565 : cluster [DBG] pgmap v2020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:50 gibba011 bash[31774]: cluster 2021-03-10T14:59:49.336632+0000 mgr.gibba032.whopeo (mgr.24625) 3565 : cluster [DBG] pgmap v2020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:52 gibba032 bash[24004]: cluster 2021-03-10T14:59:51.338462+0000 mgr.gibba032.whopeo (mgr.24625) 3566 : cluster [DBG] pgmap v2021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:52 gibba011 bash[30791]: cluster 2021-03-10T14:59:51.338462+0000 mgr.gibba032.whopeo (mgr.24625) 3566 : cluster [DBG] pgmap v2021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:52 gibba011 bash[31774]: cluster 2021-03-10T14:59:51.338462+0000 mgr.gibba032.whopeo (mgr.24625) 3566 : cluster [DBG] pgmap v2021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:54 gibba032 bash[24004]: cluster 2021-03-10T14:59:53.339886+0000 mgr.gibba032.whopeo (mgr.24625) 3567 : cluster [DBG] pgmap v2022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:54 gibba011 bash[30791]: cluster 2021-03-10T14:59:53.339886+0000 mgr.gibba032.whopeo (mgr.24625) 3567 : cluster [DBG] pgmap v2022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:54 gibba011 bash[31774]: cluster 2021-03-10T14:59:53.339886+0000 mgr.gibba032.whopeo (mgr.24625) 3567 : cluster [DBG] pgmap v2022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:56 gibba032 bash[24004]: cluster 2021-03-10T14:59:55.341287+0000 mgr.gibba032.whopeo (mgr.24625) 3568 : cluster [DBG] pgmap v2023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:56 gibba011 bash[30791]: cluster 2021-03-10T14:59:55.341287+0000 mgr.gibba032.whopeo (mgr.24625) 3568 : cluster [DBG] pgmap v2023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:56 gibba011 bash[31774]: cluster 2021-03-10T14:59:55.341287+0000 mgr.gibba032.whopeo (mgr.24625) 3568 : cluster [DBG] pgmap v2023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.438055+0000 mon.a (mon.0) 1574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:59:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.439018+0000 mon.a (mon.0) 1575 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:59:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.454348+0000 mon.a (mon.0) 1576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:59:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.455489+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.456555+0000 mon.a (mon.0) 1578 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:57 gibba032 bash[24004]: audit 2021-03-10T14:59:56.457630+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:59:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.438055+0000 mon.a (mon.0) 1574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:59:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.439018+0000 mon.a (mon.0) 1575 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:59:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.454348+0000 mon.a (mon.0) 1576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:59:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.455489+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.456555+0000 mon.a (mon.0) 1578 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[30791]: audit 2021-03-10T14:59:56.457630+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:59:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.438055+0000 mon.a (mon.0) 1574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T14:59:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.439018+0000 mon.a (mon.0) 1575 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T14:59:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.454348+0000 mon.a (mon.0) 1576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T14:59:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.455489+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.456555+0000 mon.a (mon.0) 1578 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T14:59:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:57 gibba011 bash[31774]: audit 2021-03-10T14:59:56.457630+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T14:59:58.192 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T14:59:58.419 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T14:59:58.420 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (66m) 95s ago 71m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T14:59:58.420 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (71m) 31s ago 71m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T14:59:58.420 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (66m) 31s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T14:59:58.420 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (67m) 31s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T14:59:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (66m) 95s ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T14:59:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (66m) 95s ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T14:59:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (65m) 31s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T14:59:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (65m) 95s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T14:59:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (72m) 95s ago 72m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T14:59:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (72m) 31s ago 72m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T14:59:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (64m) 95s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T14:59:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (64m) 95s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T14:59:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (64m) 95s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T14:59:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (63m) 95s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T14:59:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (63m) 31s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T14:59:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (62m) 31s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T14:59:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (62m) 31s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T14:59:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (61m) 31s ago 72m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T14:59:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (71m) 31s ago 72m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T14:59:58.424 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 95s ago 70m docker.io/ceph/ceph:v15.2.0 2021-03-10T14:59:58.669 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T14:59:58.669 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T14:59:58.670 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:59:58.670 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:58.670 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T14:59:58.670 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T14:59:58.671 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:58.671 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T14:59:58.671 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T14:59:58.671 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T14:59:58.671 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T14:59:58.672 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T14:59:58.672 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T14:59:58.672 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T14:59:58.672 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T14:59:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:58 gibba032 bash[24004]: cluster 2021-03-10T14:59:57.342915+0000 mgr.gibba032.whopeo (mgr.24625) 3569 : cluster [DBG] pgmap v2024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:58 gibba032 bash[24004]: cephadm 2021-03-10T14:59:57.354485+0000 mgr.gibba032.whopeo (mgr.24625) 3570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:58 gibba011 bash[30791]: cluster 2021-03-10T14:59:57.342915+0000 mgr.gibba032.whopeo (mgr.24625) 3569 : cluster [DBG] pgmap v2024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:58 gibba011 bash[30791]: cephadm 2021-03-10T14:59:57.354485+0000 mgr.gibba032.whopeo (mgr.24625) 3570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:58 gibba011 bash[31774]: cluster 2021-03-10T14:59:57.342915+0000 mgr.gibba032.whopeo (mgr.24625) 3569 : cluster [DBG] pgmap v2024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T14:59:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:58 gibba011 bash[31774]: cephadm 2021-03-10T14:59:57.354485+0000 mgr.gibba032.whopeo (mgr.24625) 3570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T14:59:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:59 gibba032 bash[24004]: audit 2021-03-10T14:59:58.178176+0000 mgr.gibba032.whopeo (mgr.24625) 3571 : audit [DBG] from='client.46449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:59 gibba032 bash[24004]: audit 2021-03-10T14:59:58.403567+0000 mgr.gibba032.whopeo (mgr.24625) 3572 : audit [DBG] from='client.56440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 14:59:59 gibba032 bash[24004]: audit 2021-03-10T14:59:58.667003+0000 mon.c (mon.1) 67 : audit [DBG] from='client.? 172.21.2.111:0/2584717911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[30791]: audit 2021-03-10T14:59:58.178176+0000 mgr.gibba032.whopeo (mgr.24625) 3571 : audit [DBG] from='client.46449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[30791]: audit 2021-03-10T14:59:58.403567+0000 mgr.gibba032.whopeo (mgr.24625) 3572 : audit [DBG] from='client.56440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[30791]: audit 2021-03-10T14:59:58.667003+0000 mon.c (mon.1) 67 : audit [DBG] from='client.? 172.21.2.111:0/2584717911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T14:59:59.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[31774]: audit 2021-03-10T14:59:58.178176+0000 mgr.gibba032.whopeo (mgr.24625) 3571 : audit [DBG] from='client.46449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[31774]: audit 2021-03-10T14:59:58.403567+0000 mgr.gibba032.whopeo (mgr.24625) 3572 : audit [DBG] from='client.56440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T14:59:59.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 14:59:59 gibba011 bash[31774]: audit 2021-03-10T14:59:58.667003+0000 mon.c (mon.1) 67 : audit [DBG] from='client.? 172.21.2.111:0/2584717911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:00 gibba011 bash[30791]: debug 2021-03-10T14:59:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:00:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:00 gibba032 bash[24004]: cluster 2021-03-10T14:59:59.344013+0000 mgr.gibba032.whopeo (mgr.24625) 3573 : cluster [DBG] pgmap v2025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:00 gibba032 bash[24004]: cluster 2021-03-10T15:00:00.000441+0000 mon.a (mon.0) 1580 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:00:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:00 gibba011 bash[30791]: cluster 2021-03-10T14:59:59.344013+0000 mgr.gibba032.whopeo (mgr.24625) 3573 : cluster [DBG] pgmap v2025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:00 gibba011 bash[30791]: cluster 2021-03-10T15:00:00.000441+0000 mon.a (mon.0) 1580 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:00:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:00 gibba011 bash[31774]: cluster 2021-03-10T14:59:59.344013+0000 mgr.gibba032.whopeo (mgr.24625) 3573 : cluster [DBG] pgmap v2025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:00 gibba011 bash[31774]: cluster 2021-03-10T15:00:00.000441+0000 mon.a (mon.0) 1580 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:00:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:02 gibba032 bash[24004]: cluster 2021-03-10T15:00:01.345912+0000 mgr.gibba032.whopeo (mgr.24625) 3574 : cluster [DBG] pgmap v2026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:02 gibba011 bash[30791]: cluster 2021-03-10T15:00:01.345912+0000 mgr.gibba032.whopeo (mgr.24625) 3574 : cluster [DBG] pgmap v2026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:02 gibba011 bash[31774]: cluster 2021-03-10T15:00:01.345912+0000 mgr.gibba032.whopeo (mgr.24625) 3574 : cluster [DBG] pgmap v2026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:04 gibba032 bash[24004]: cluster 2021-03-10T15:00:03.347385+0000 mgr.gibba032.whopeo (mgr.24625) 3575 : cluster [DBG] pgmap v2027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:04 gibba011 bash[30791]: cluster 2021-03-10T15:00:03.347385+0000 mgr.gibba032.whopeo (mgr.24625) 3575 : cluster [DBG] pgmap v2027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:04 gibba011 bash[31774]: cluster 2021-03-10T15:00:03.347385+0000 mgr.gibba032.whopeo (mgr.24625) 3575 : cluster [DBG] pgmap v2027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:06 gibba032 bash[24004]: cluster 2021-03-10T15:00:05.348891+0000 mgr.gibba032.whopeo (mgr.24625) 3576 : cluster [DBG] pgmap v2028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:06 gibba011 bash[30791]: cluster 2021-03-10T15:00:05.348891+0000 mgr.gibba032.whopeo (mgr.24625) 3576 : cluster [DBG] pgmap v2028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:06 gibba011 bash[31774]: cluster 2021-03-10T15:00:05.348891+0000 mgr.gibba032.whopeo (mgr.24625) 3576 : cluster [DBG] pgmap v2028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:07 gibba032 bash[24004]: cluster 2021-03-10T15:00:07.350792+0000 mgr.gibba032.whopeo (mgr.24625) 3577 : cluster [DBG] pgmap v2029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:07 gibba011 bash[30791]: cluster 2021-03-10T15:00:07.350792+0000 mgr.gibba032.whopeo (mgr.24625) 3577 : cluster [DBG] pgmap v2029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:07 gibba011 bash[31774]: cluster 2021-03-10T15:00:07.350792+0000 mgr.gibba032.whopeo (mgr.24625) 3577 : cluster [DBG] pgmap v2029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:10 gibba032 bash[24004]: cluster 2021-03-10T15:00:09.352373+0000 mgr.gibba032.whopeo (mgr.24625) 3578 : cluster [DBG] pgmap v2030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:10 gibba011 bash[30791]: cluster 2021-03-10T15:00:09.352373+0000 mgr.gibba032.whopeo (mgr.24625) 3578 : cluster [DBG] pgmap v2030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:10 gibba011 bash[31774]: cluster 2021-03-10T15:00:09.352373+0000 mgr.gibba032.whopeo (mgr.24625) 3578 : cluster [DBG] pgmap v2030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:12 gibba032 bash[24004]: cluster 2021-03-10T15:00:11.354339+0000 mgr.gibba032.whopeo (mgr.24625) 3579 : cluster [DBG] pgmap v2031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:12 gibba011 bash[30791]: cluster 2021-03-10T15:00:11.354339+0000 mgr.gibba032.whopeo (mgr.24625) 3579 : cluster [DBG] pgmap v2031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:12 gibba011 bash[31774]: cluster 2021-03-10T15:00:11.354339+0000 mgr.gibba032.whopeo (mgr.24625) 3579 : cluster [DBG] pgmap v2031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:13 gibba032 bash[24004]: cephadm 2021-03-10T15:00:12.361223+0000 mgr.gibba032.whopeo (mgr.24625) 3580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:13 gibba011 bash[30791]: cephadm 2021-03-10T15:00:12.361223+0000 mgr.gibba032.whopeo (mgr.24625) 3580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:13.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:13 gibba011 bash[31774]: cephadm 2021-03-10T15:00:12.361223+0000 mgr.gibba032.whopeo (mgr.24625) 3580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:14 gibba032 bash[24004]: cluster 2021-03-10T15:00:13.356148+0000 mgr.gibba032.whopeo (mgr.24625) 3581 : cluster [DBG] pgmap v2032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:14 gibba011 bash[30791]: cluster 2021-03-10T15:00:13.356148+0000 mgr.gibba032.whopeo (mgr.24625) 3581 : cluster [DBG] pgmap v2032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:14 gibba011 bash[31774]: cluster 2021-03-10T15:00:13.356148+0000 mgr.gibba032.whopeo (mgr.24625) 3581 : cluster [DBG] pgmap v2032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:16 gibba032 bash[24004]: cluster 2021-03-10T15:00:15.357686+0000 mgr.gibba032.whopeo (mgr.24625) 3582 : cluster [DBG] pgmap v2033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:16 gibba011 bash[30791]: cluster 2021-03-10T15:00:15.357686+0000 mgr.gibba032.whopeo (mgr.24625) 3582 : cluster [DBG] pgmap v2033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:16 gibba011 bash[31774]: cluster 2021-03-10T15:00:15.357686+0000 mgr.gibba032.whopeo (mgr.24625) 3582 : cluster [DBG] pgmap v2033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:17 gibba032 bash[24004]: cluster 2021-03-10T15:00:17.359354+0000 mgr.gibba032.whopeo (mgr.24625) 3583 : cluster [DBG] pgmap v2034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:17 gibba011 bash[30791]: cluster 2021-03-10T15:00:17.359354+0000 mgr.gibba032.whopeo (mgr.24625) 3583 : cluster [DBG] pgmap v2034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:17 gibba011 bash[31774]: cluster 2021-03-10T15:00:17.359354+0000 mgr.gibba032.whopeo (mgr.24625) 3583 : cluster [DBG] pgmap v2034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:20 gibba032 bash[24004]: cluster 2021-03-10T15:00:19.360750+0000 mgr.gibba032.whopeo (mgr.24625) 3584 : cluster [DBG] pgmap v2035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:20 gibba011 bash[30791]: cluster 2021-03-10T15:00:19.360750+0000 mgr.gibba032.whopeo (mgr.24625) 3584 : cluster [DBG] pgmap v2035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:20 gibba011 bash[31774]: cluster 2021-03-10T15:00:19.360750+0000 mgr.gibba032.whopeo (mgr.24625) 3584 : cluster [DBG] pgmap v2035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:22 gibba032 bash[24004]: cluster 2021-03-10T15:00:21.362469+0000 mgr.gibba032.whopeo (mgr.24625) 3585 : cluster [DBG] pgmap v2036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:22 gibba011 bash[30791]: cluster 2021-03-10T15:00:21.362469+0000 mgr.gibba032.whopeo (mgr.24625) 3585 : cluster [DBG] pgmap v2036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:22.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:22 gibba011 bash[31774]: cluster 2021-03-10T15:00:21.362469+0000 mgr.gibba032.whopeo (mgr.24625) 3585 : cluster [DBG] pgmap v2036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:24 gibba032 bash[24004]: cluster 2021-03-10T15:00:23.363990+0000 mgr.gibba032.whopeo (mgr.24625) 3586 : cluster [DBG] pgmap v2037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:24 gibba011 bash[30791]: cluster 2021-03-10T15:00:23.363990+0000 mgr.gibba032.whopeo (mgr.24625) 3586 : cluster [DBG] pgmap v2037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:24 gibba011 bash[31774]: cluster 2021-03-10T15:00:23.363990+0000 mgr.gibba032.whopeo (mgr.24625) 3586 : cluster [DBG] pgmap v2037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:26 gibba032 bash[24004]: cluster 2021-03-10T15:00:25.365309+0000 mgr.gibba032.whopeo (mgr.24625) 3587 : cluster [DBG] pgmap v2038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:26 gibba011 bash[30791]: cluster 2021-03-10T15:00:25.365309+0000 mgr.gibba032.whopeo (mgr.24625) 3587 : cluster [DBG] pgmap v2038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:26 gibba011 bash[31774]: cluster 2021-03-10T15:00:25.365309+0000 mgr.gibba032.whopeo (mgr.24625) 3587 : cluster [DBG] pgmap v2038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:28.910 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:00:29.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cluster 2021-03-10T15:00:27.366924+0000 mgr.gibba032.whopeo (mgr.24625) 3588 : cluster [DBG] pgmap v2039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:29.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.377861+0000 mgr.gibba032.whopeo (mgr.24625) 3589 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:00:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.379461+0000 mgr.gibba032.whopeo (mgr.24625) 3590 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:00:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.605445+0000 mon.a (mon.0) 1581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:00:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.605913+0000 mgr.gibba032.whopeo (mgr.24625) 3591 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:00:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.608431+0000 mgr.gibba032.whopeo (mgr.24625) 3592 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:00:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.609121+0000 mon.a (mon.0) 1582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:00:29.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.609882+0000 mgr.gibba032.whopeo (mgr.24625) 3593 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:00:29.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.609921+0000 mgr.gibba032.whopeo (mgr.24625) 3594 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:00:29.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.611122+0000 mon.a (mon.0) 1583 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.611173+0000 mgr.gibba032.whopeo (mgr.24625) 3595 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:00:29.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.611214+0000 mgr.gibba032.whopeo (mgr.24625) 3596 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:00:29.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.612320+0000 mgr.gibba032.whopeo (mgr.24625) 3597 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:00:29.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.612349+0000 mon.a (mon.0) 1584 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.613246+0000 mgr.gibba032.whopeo (mgr.24625) 3598 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:00:29.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.613284+0000 mgr.gibba032.whopeo (mgr.24625) 3599 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:00:29.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.613751+0000 mon.a (mon.0) 1585 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.614709+0000 mgr.gibba032.whopeo (mgr.24625) 3600 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:00:29.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.615041+0000 mgr.gibba032.whopeo (mgr.24625) 3601 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:00:29.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.615972+0000 mon.a (mon.0) 1586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.616252+0000 mgr.gibba032.whopeo (mgr.24625) 3602 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:00:29.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.616293+0000 mgr.gibba032.whopeo (mgr.24625) 3603 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:00:29.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: cephadm 2021-03-10T15:00:27.616409+0000 mgr.gibba032.whopeo (mgr.24625) 3604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:29.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:28 gibba032 bash[24004]: audit 2021-03-10T15:00:27.617412+0000 mon.a (mon.0) 1587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cluster 2021-03-10T15:00:27.366924+0000 mgr.gibba032.whopeo (mgr.24625) 3588 : cluster [DBG] pgmap v2039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.377861+0000 mgr.gibba032.whopeo (mgr.24625) 3589 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:00:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.379461+0000 mgr.gibba032.whopeo (mgr.24625) 3590 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:00:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.605445+0000 mon.a (mon.0) 1581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:00:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.605913+0000 mgr.gibba032.whopeo (mgr.24625) 3591 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:00:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.608431+0000 mgr.gibba032.whopeo (mgr.24625) 3592 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:00:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.609121+0000 mon.a (mon.0) 1582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:00:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.609882+0000 mgr.gibba032.whopeo (mgr.24625) 3593 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:00:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.609921+0000 mgr.gibba032.whopeo (mgr.24625) 3594 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:00:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.611122+0000 mon.a (mon.0) 1583 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.611173+0000 mgr.gibba032.whopeo (mgr.24625) 3595 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:00:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.611214+0000 mgr.gibba032.whopeo (mgr.24625) 3596 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:00:29.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.612320+0000 mgr.gibba032.whopeo (mgr.24625) 3597 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:00:29.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.612349+0000 mon.a (mon.0) 1584 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.613246+0000 mgr.gibba032.whopeo (mgr.24625) 3598 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:00:29.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.613284+0000 mgr.gibba032.whopeo (mgr.24625) 3599 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:00:29.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.613751+0000 mon.a (mon.0) 1585 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.614709+0000 mgr.gibba032.whopeo (mgr.24625) 3600 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:00:29.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.615041+0000 mgr.gibba032.whopeo (mgr.24625) 3601 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:00:29.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.615972+0000 mon.a (mon.0) 1586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.616252+0000 mgr.gibba032.whopeo (mgr.24625) 3602 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:00:29.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.616293+0000 mgr.gibba032.whopeo (mgr.24625) 3603 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:00:29.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: cephadm 2021-03-10T15:00:27.616409+0000 mgr.gibba032.whopeo (mgr.24625) 3604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:29.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[30791]: audit 2021-03-10T15:00:27.617412+0000 mon.a (mon.0) 1587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cluster 2021-03-10T15:00:27.366924+0000 mgr.gibba032.whopeo (mgr.24625) 3588 : cluster [DBG] pgmap v2039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:29.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.377861+0000 mgr.gibba032.whopeo (mgr.24625) 3589 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:00:29.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.379461+0000 mgr.gibba032.whopeo (mgr.24625) 3590 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:00:29.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.605445+0000 mon.a (mon.0) 1581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:00:29.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.605913+0000 mgr.gibba032.whopeo (mgr.24625) 3591 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:00:29.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.608431+0000 mgr.gibba032.whopeo (mgr.24625) 3592 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:00:29.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.609121+0000 mon.a (mon.0) 1582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:00:29.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.609882+0000 mgr.gibba032.whopeo (mgr.24625) 3593 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:00:29.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.609921+0000 mgr.gibba032.whopeo (mgr.24625) 3594 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:00:29.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.611122+0000 mon.a (mon.0) 1583 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.611173+0000 mgr.gibba032.whopeo (mgr.24625) 3595 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:00:29.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.611214+0000 mgr.gibba032.whopeo (mgr.24625) 3596 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:00:29.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.612320+0000 mgr.gibba032.whopeo (mgr.24625) 3597 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:00:29.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.612349+0000 mon.a (mon.0) 1584 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.613246+0000 mgr.gibba032.whopeo (mgr.24625) 3598 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:00:29.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.613284+0000 mgr.gibba032.whopeo (mgr.24625) 3599 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:00:29.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.613751+0000 mon.a (mon.0) 1585 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.614709+0000 mgr.gibba032.whopeo (mgr.24625) 3600 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:00:29.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.615041+0000 mgr.gibba032.whopeo (mgr.24625) 3601 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:00:29.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.615972+0000 mon.a (mon.0) 1586 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.616252+0000 mgr.gibba032.whopeo (mgr.24625) 3602 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:00:29.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.616293+0000 mgr.gibba032.whopeo (mgr.24625) 3603 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:00:29.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: cephadm 2021-03-10T15:00:27.616409+0000 mgr.gibba032.whopeo (mgr.24625) 3604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:29.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:28 gibba011 bash[31774]: audit 2021-03-10T15:00:27.617412+0000 mon.a (mon.0) 1587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:29.139 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:00:29.139 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (67m) 2m ago 72m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:00:29.139 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (72m) 62s ago 72m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:00:29.139 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (67m) 62s ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:00:29.140 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (67m) 62s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:00:29.140 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (66m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:00:29.140 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (66m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:00:29.140 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (66m) 62s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:00:29.141 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (66m) 2m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:00:29.141 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (72m) 2m ago 72m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:00:29.141 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (72m) 62s ago 72m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:00:29.141 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (65m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:00:29.141 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (64m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:00:29.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (64m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:00:29.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (64m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:00:29.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (63m) 62s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:00:29.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (63m) 62s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:00:29.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (62m) 62s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:00:29.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (62m) 62s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:00:29.143 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (72m) 62s ago 72m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:00:29.143 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 70m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:00:29.388 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:00:29.389 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:00:29.389 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:00:29.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:00:29.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:00:29.390 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:00:29.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:00:29.390 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:00:29.391 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:00:29.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:00:29.391 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:00:29.391 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:00:29.392 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:00:29.392 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:00:29.392 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:00:30.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:29 gibba032 bash[24004]: audit 2021-03-10T15:00:29.386388+0000 mon.c (mon.1) 68 : audit [DBG] from='client.? 172.21.2.111:0/532940814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:30.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:29 gibba011 bash[30791]: audit 2021-03-10T15:00:29.386388+0000 mon.c (mon.1) 68 : audit [DBG] from='client.? 172.21.2.111:0/532940814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:30.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:29 gibba011 bash[31774]: audit 2021-03-10T15:00:29.386388+0000 mon.c (mon.1) 68 : audit [DBG] from='client.? 172.21.2.111:0/532940814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:00:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:30 gibba032 bash[24004]: audit 2021-03-10T15:00:28.896564+0000 mgr.gibba032.whopeo (mgr.24625) 3605 : audit [DBG] from='client.46467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:30 gibba032 bash[24004]: audit 2021-03-10T15:00:29.122073+0000 mgr.gibba032.whopeo (mgr.24625) 3606 : audit [DBG] from='client.46473 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:30 gibba032 bash[24004]: cluster 2021-03-10T15:00:29.368165+0000 mgr.gibba032.whopeo (mgr.24625) 3607 : cluster [DBG] pgmap v2040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[30791]: audit 2021-03-10T15:00:28.896564+0000 mgr.gibba032.whopeo (mgr.24625) 3605 : audit [DBG] from='client.46467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[30791]: audit 2021-03-10T15:00:29.122073+0000 mgr.gibba032.whopeo (mgr.24625) 3606 : audit [DBG] from='client.46473 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[30791]: cluster 2021-03-10T15:00:29.368165+0000 mgr.gibba032.whopeo (mgr.24625) 3607 : cluster [DBG] pgmap v2040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[31774]: audit 2021-03-10T15:00:28.896564+0000 mgr.gibba032.whopeo (mgr.24625) 3605 : audit [DBG] from='client.46467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[31774]: audit 2021-03-10T15:00:29.122073+0000 mgr.gibba032.whopeo (mgr.24625) 3606 : audit [DBG] from='client.46473 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:00:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:30 gibba011 bash[31774]: cluster 2021-03-10T15:00:29.368165+0000 mgr.gibba032.whopeo (mgr.24625) 3607 : cluster [DBG] pgmap v2040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:32 gibba032 bash[24004]: cluster 2021-03-10T15:00:31.369790+0000 mgr.gibba032.whopeo (mgr.24625) 3608 : cluster [DBG] pgmap v2041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:33.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:32 gibba011 bash[30791]: cluster 2021-03-10T15:00:31.369790+0000 mgr.gibba032.whopeo (mgr.24625) 3608 : cluster [DBG] pgmap v2041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:33.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:32 gibba011 bash[31774]: cluster 2021-03-10T15:00:31.369790+0000 mgr.gibba032.whopeo (mgr.24625) 3608 : cluster [DBG] pgmap v2041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:35.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:34 gibba032 bash[24004]: cluster 2021-03-10T15:00:33.371247+0000 mgr.gibba032.whopeo (mgr.24625) 3609 : cluster [DBG] pgmap v2042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:35.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:34 gibba011 bash[30791]: cluster 2021-03-10T15:00:33.371247+0000 mgr.gibba032.whopeo (mgr.24625) 3609 : cluster [DBG] pgmap v2042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:35.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:34 gibba011 bash[31774]: cluster 2021-03-10T15:00:33.371247+0000 mgr.gibba032.whopeo (mgr.24625) 3609 : cluster [DBG] pgmap v2042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:36 gibba032 bash[24004]: cluster 2021-03-10T15:00:35.372762+0000 mgr.gibba032.whopeo (mgr.24625) 3610 : cluster [DBG] pgmap v2043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:36 gibba011 bash[30791]: cluster 2021-03-10T15:00:35.372762+0000 mgr.gibba032.whopeo (mgr.24625) 3610 : cluster [DBG] pgmap v2043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:36 gibba011 bash[31774]: cluster 2021-03-10T15:00:35.372762+0000 mgr.gibba032.whopeo (mgr.24625) 3610 : cluster [DBG] pgmap v2043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:37 gibba032 bash[24004]: cluster 2021-03-10T15:00:37.374402+0000 mgr.gibba032.whopeo (mgr.24625) 3611 : cluster [DBG] pgmap v2044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:37 gibba011 bash[30791]: cluster 2021-03-10T15:00:37.374402+0000 mgr.gibba032.whopeo (mgr.24625) 3611 : cluster [DBG] pgmap v2044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:37 gibba011 bash[31774]: cluster 2021-03-10T15:00:37.374402+0000 mgr.gibba032.whopeo (mgr.24625) 3611 : cluster [DBG] pgmap v2044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:40 gibba032 bash[24004]: cluster 2021-03-10T15:00:39.375617+0000 mgr.gibba032.whopeo (mgr.24625) 3612 : cluster [DBG] pgmap v2045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:40 gibba011 bash[30791]: cluster 2021-03-10T15:00:39.375617+0000 mgr.gibba032.whopeo (mgr.24625) 3612 : cluster [DBG] pgmap v2045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:40 gibba011 bash[31774]: cluster 2021-03-10T15:00:39.375617+0000 mgr.gibba032.whopeo (mgr.24625) 3612 : cluster [DBG] pgmap v2045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:42 gibba032 bash[24004]: cluster 2021-03-10T15:00:41.377431+0000 mgr.gibba032.whopeo (mgr.24625) 3613 : cluster [DBG] pgmap v2046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:42 gibba011 bash[30791]: cluster 2021-03-10T15:00:41.377431+0000 mgr.gibba032.whopeo (mgr.24625) 3613 : cluster [DBG] pgmap v2046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:42 gibba011 bash[31774]: cluster 2021-03-10T15:00:41.377431+0000 mgr.gibba032.whopeo (mgr.24625) 3613 : cluster [DBG] pgmap v2046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:43 gibba032 bash[24004]: cephadm 2021-03-10T15:00:42.632254+0000 mgr.gibba032.whopeo (mgr.24625) 3614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:43 gibba011 bash[30791]: cephadm 2021-03-10T15:00:42.632254+0000 mgr.gibba032.whopeo (mgr.24625) 3614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:43 gibba011 bash[31774]: cephadm 2021-03-10T15:00:42.632254+0000 mgr.gibba032.whopeo (mgr.24625) 3614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:44 gibba032 bash[24004]: cluster 2021-03-10T15:00:43.379467+0000 mgr.gibba032.whopeo (mgr.24625) 3615 : cluster [DBG] pgmap v2047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:44 gibba011 bash[30791]: cluster 2021-03-10T15:00:43.379467+0000 mgr.gibba032.whopeo (mgr.24625) 3615 : cluster [DBG] pgmap v2047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:44 gibba011 bash[31774]: cluster 2021-03-10T15:00:43.379467+0000 mgr.gibba032.whopeo (mgr.24625) 3615 : cluster [DBG] pgmap v2047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:46 gibba032 bash[24004]: cluster 2021-03-10T15:00:45.380897+0000 mgr.gibba032.whopeo (mgr.24625) 3616 : cluster [DBG] pgmap v2048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:46.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:46 gibba011 bash[30791]: cluster 2021-03-10T15:00:45.380897+0000 mgr.gibba032.whopeo (mgr.24625) 3616 : cluster [DBG] pgmap v2048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:46.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:46 gibba011 bash[31774]: cluster 2021-03-10T15:00:45.380897+0000 mgr.gibba032.whopeo (mgr.24625) 3616 : cluster [DBG] pgmap v2048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:47 gibba032 bash[24004]: cluster 2021-03-10T15:00:47.382558+0000 mgr.gibba032.whopeo (mgr.24625) 3617 : cluster [DBG] pgmap v2049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:47 gibba011 bash[30791]: cluster 2021-03-10T15:00:47.382558+0000 mgr.gibba032.whopeo (mgr.24625) 3617 : cluster [DBG] pgmap v2049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:48.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:47 gibba011 bash[31774]: cluster 2021-03-10T15:00:47.382558+0000 mgr.gibba032.whopeo (mgr.24625) 3617 : cluster [DBG] pgmap v2049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:50 gibba032 bash[24004]: cluster 2021-03-10T15:00:49.383877+0000 mgr.gibba032.whopeo (mgr.24625) 3618 : cluster [DBG] pgmap v2050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:50 gibba011 bash[30791]: cluster 2021-03-10T15:00:49.383877+0000 mgr.gibba032.whopeo (mgr.24625) 3618 : cluster [DBG] pgmap v2050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:50 gibba011 bash[31774]: cluster 2021-03-10T15:00:49.383877+0000 mgr.gibba032.whopeo (mgr.24625) 3618 : cluster [DBG] pgmap v2050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:52 gibba032 bash[24004]: cluster 2021-03-10T15:00:51.385577+0000 mgr.gibba032.whopeo (mgr.24625) 3619 : cluster [DBG] pgmap v2051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:52 gibba011 bash[30791]: cluster 2021-03-10T15:00:51.385577+0000 mgr.gibba032.whopeo (mgr.24625) 3619 : cluster [DBG] pgmap v2051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:52 gibba011 bash[31774]: cluster 2021-03-10T15:00:51.385577+0000 mgr.gibba032.whopeo (mgr.24625) 3619 : cluster [DBG] pgmap v2051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:54 gibba032 bash[24004]: cluster 2021-03-10T15:00:53.386995+0000 mgr.gibba032.whopeo (mgr.24625) 3620 : cluster [DBG] pgmap v2052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:54 gibba011 bash[30791]: cluster 2021-03-10T15:00:53.386995+0000 mgr.gibba032.whopeo (mgr.24625) 3620 : cluster [DBG] pgmap v2052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:54 gibba011 bash[31774]: cluster 2021-03-10T15:00:53.386995+0000 mgr.gibba032.whopeo (mgr.24625) 3620 : cluster [DBG] pgmap v2052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: cluster 2021-03-10T15:00:55.388475+0000 mgr.gibba032.whopeo (mgr.24625) 3621 : cluster [DBG] pgmap v2053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.443856+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:00:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.445388+0000 mon.a (mon.0) 1589 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:00:56.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.461961+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T15:00:56.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.462734+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:00:56.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.463476+0000 mon.a (mon.0) 1592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:00:56.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:56 gibba032 bash[24004]: audit 2021-03-10T15:00:56.464179+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:00:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: cluster 2021-03-10T15:00:55.388475+0000 mgr.gibba032.whopeo (mgr.24625) 3621 : cluster [DBG] pgmap v2053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.443856+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:00:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.445388+0000 mon.a (mon.0) 1589 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:00:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.461961+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T15:00:56.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.462734+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:00:56.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.463476+0000 mon.a (mon.0) 1592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:00:56.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[30791]: audit 2021-03-10T15:00:56.464179+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:00:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: cluster 2021-03-10T15:00:55.388475+0000 mgr.gibba032.whopeo (mgr.24625) 3621 : cluster [DBG] pgmap v2053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:56.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.443856+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:00:56.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.445388+0000 mon.a (mon.0) 1589 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:00:56.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.461961+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T15:00:56.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.462734+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:00:56.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.463476+0000 mon.a (mon.0) 1592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:00:56.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:56 gibba011 bash[31774]: audit 2021-03-10T15:00:56.464179+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:00:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:57 gibba032 bash[24004]: cluster 2021-03-10T15:00:57.390021+0000 mgr.gibba032.whopeo (mgr.24625) 3622 : cluster [DBG] pgmap v2054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:00:57 gibba032 bash[24004]: cephadm 2021-03-10T15:00:57.648151+0000 mgr.gibba032.whopeo (mgr.24625) 3623 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:57 gibba011 bash[30791]: cluster 2021-03-10T15:00:57.390021+0000 mgr.gibba032.whopeo (mgr.24625) 3622 : cluster [DBG] pgmap v2054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:00:57 gibba011 bash[30791]: cephadm 2021-03-10T15:00:57.648151+0000 mgr.gibba032.whopeo (mgr.24625) 3623 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:57 gibba011 bash[31774]: cluster 2021-03-10T15:00:57.390021+0000 mgr.gibba032.whopeo (mgr.24625) 3622 : cluster [DBG] pgmap v2054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:00:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:00:57 gibba011 bash[31774]: cephadm 2021-03-10T15:00:57.648151+0000 mgr.gibba032.whopeo (mgr.24625) 3623 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:00:59.632 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:00:59.857 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:00:59.858 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (67m) 2m ago 73m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:00:59.858 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (72m) 92s ago 72m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:00:59.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (67m) 92s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:00:59.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (68m) 92s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:00:59.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (67m) 2m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:00:59.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (67m) 2m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:00:59.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (66m) 92s ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:00:59.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (66m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:00:59.859 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (73m) 2m ago 73m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:00:59.860 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (73m) 92s ago 73m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:00:59.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (65m) 2m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:00:59.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (65m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:00:59.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (65m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:00:59.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (64m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:00:59.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (64m) 92s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:00:59.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (63m) 92s ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:00:59.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (63m) 92s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:00:59.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (62m) 92s ago 73m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:00:59.862 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (72m) 92s ago 73m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:00:59.862 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 71m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:01:00.105 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:01:00.105 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:01:00.105 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:01:00.106 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:00.106 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:01:00.106 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:01:00.106 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:00.107 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:01:00.107 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:01:00.107 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:00.107 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:01:00.107 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:01:00.108 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:01:00.108 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:01:00.108 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:01:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:00 gibba032 bash[24004]: cluster 2021-03-10T15:00:59.391344+0000 mgr.gibba032.whopeo (mgr.24625) 3624 : cluster [DBG] pgmap v2055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:00 gibba032 bash[24004]: audit 2021-03-10T15:00:59.618528+0000 mgr.gibba032.whopeo (mgr.24625) 3625 : audit [DBG] from='client.46485 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:00 gibba032 bash[24004]: audit 2021-03-10T15:01:00.102640+0000 mon.c (mon.1) 69 : audit [DBG] from='client.? 172.21.2.111:0/3359346146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[30791]: cluster 2021-03-10T15:00:59.391344+0000 mgr.gibba032.whopeo (mgr.24625) 3624 : cluster [DBG] pgmap v2055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[30791]: audit 2021-03-10T15:00:59.618528+0000 mgr.gibba032.whopeo (mgr.24625) 3625 : audit [DBG] from='client.46485 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[30791]: audit 2021-03-10T15:01:00.102640+0000 mon.c (mon.1) 69 : audit [DBG] from='client.? 172.21.2.111:0/3359346146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[31774]: cluster 2021-03-10T15:00:59.391344+0000 mgr.gibba032.whopeo (mgr.24625) 3624 : cluster [DBG] pgmap v2055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[31774]: audit 2021-03-10T15:00:59.618528+0000 mgr.gibba032.whopeo (mgr.24625) 3625 : audit [DBG] from='client.46485 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:00 gibba011 bash[31774]: audit 2021-03-10T15:01:00.102640+0000 mon.c (mon.1) 69 : audit [DBG] from='client.? 172.21.2.111:0/3359346146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:01 gibba032 bash[24004]: audit 2021-03-10T15:00:59.841549+0000 mgr.gibba032.whopeo (mgr.24625) 3626 : audit [DBG] from='client.46491 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:01 gibba011 bash[30791]: audit 2021-03-10T15:00:59.841549+0000 mgr.gibba032.whopeo (mgr.24625) 3626 : audit [DBG] from='client.46491 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:01 gibba011 bash[31774]: audit 2021-03-10T15:00:59.841549+0000 mgr.gibba032.whopeo (mgr.24625) 3626 : audit [DBG] from='client.46491 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:02 gibba032 bash[24004]: cluster 2021-03-10T15:01:01.393260+0000 mgr.gibba032.whopeo (mgr.24625) 3627 : cluster [DBG] pgmap v2056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:02 gibba011 bash[30791]: cluster 2021-03-10T15:01:01.393260+0000 mgr.gibba032.whopeo (mgr.24625) 3627 : cluster [DBG] pgmap v2056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:02 gibba011 bash[31774]: cluster 2021-03-10T15:01:01.393260+0000 mgr.gibba032.whopeo (mgr.24625) 3627 : cluster [DBG] pgmap v2056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:04 gibba032 bash[24004]: cluster 2021-03-10T15:01:03.394737+0000 mgr.gibba032.whopeo (mgr.24625) 3628 : cluster [DBG] pgmap v2057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:04 gibba011 bash[30791]: cluster 2021-03-10T15:01:03.394737+0000 mgr.gibba032.whopeo (mgr.24625) 3628 : cluster [DBG] pgmap v2057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:04 gibba011 bash[31774]: cluster 2021-03-10T15:01:03.394737+0000 mgr.gibba032.whopeo (mgr.24625) 3628 : cluster [DBG] pgmap v2057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:06 gibba032 bash[24004]: cluster 2021-03-10T15:01:05.396267+0000 mgr.gibba032.whopeo (mgr.24625) 3629 : cluster [DBG] pgmap v2058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:06 gibba011 bash[30791]: cluster 2021-03-10T15:01:05.396267+0000 mgr.gibba032.whopeo (mgr.24625) 3629 : cluster [DBG] pgmap v2058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:06 gibba011 bash[31774]: cluster 2021-03-10T15:01:05.396267+0000 mgr.gibba032.whopeo (mgr.24625) 3629 : cluster [DBG] pgmap v2058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:07 gibba032 bash[24004]: cluster 2021-03-10T15:01:07.398055+0000 mgr.gibba032.whopeo (mgr.24625) 3630 : cluster [DBG] pgmap v2059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:07 gibba011 bash[30791]: cluster 2021-03-10T15:01:07.398055+0000 mgr.gibba032.whopeo (mgr.24625) 3630 : cluster [DBG] pgmap v2059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:07 gibba011 bash[31774]: cluster 2021-03-10T15:01:07.398055+0000 mgr.gibba032.whopeo (mgr.24625) 3630 : cluster [DBG] pgmap v2059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:10 gibba032 bash[24004]: cluster 2021-03-10T15:01:09.399395+0000 mgr.gibba032.whopeo (mgr.24625) 3631 : cluster [DBG] pgmap v2060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:10 gibba011 bash[30791]: cluster 2021-03-10T15:01:09.399395+0000 mgr.gibba032.whopeo (mgr.24625) 3631 : cluster [DBG] pgmap v2060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:10 gibba011 bash[31774]: cluster 2021-03-10T15:01:09.399395+0000 mgr.gibba032.whopeo (mgr.24625) 3631 : cluster [DBG] pgmap v2060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:12 gibba032 bash[24004]: cluster 2021-03-10T15:01:11.401089+0000 mgr.gibba032.whopeo (mgr.24625) 3632 : cluster [DBG] pgmap v2061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:12 gibba011 bash[30791]: cluster 2021-03-10T15:01:11.401089+0000 mgr.gibba032.whopeo (mgr.24625) 3632 : cluster [DBG] pgmap v2061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:12 gibba011 bash[31774]: cluster 2021-03-10T15:01:11.401089+0000 mgr.gibba032.whopeo (mgr.24625) 3632 : cluster [DBG] pgmap v2061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:13 gibba032 bash[24004]: cephadm 2021-03-10T15:01:12.653022+0000 mgr.gibba032.whopeo (mgr.24625) 3633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:13 gibba011 bash[30791]: cephadm 2021-03-10T15:01:12.653022+0000 mgr.gibba032.whopeo (mgr.24625) 3633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:13.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:13 gibba011 bash[31774]: cephadm 2021-03-10T15:01:12.653022+0000 mgr.gibba032.whopeo (mgr.24625) 3633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:14 gibba032 bash[24004]: cluster 2021-03-10T15:01:13.402643+0000 mgr.gibba032.whopeo (mgr.24625) 3634 : cluster [DBG] pgmap v2062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:14 gibba011 bash[30791]: cluster 2021-03-10T15:01:13.402643+0000 mgr.gibba032.whopeo (mgr.24625) 3634 : cluster [DBG] pgmap v2062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:14 gibba011 bash[31774]: cluster 2021-03-10T15:01:13.402643+0000 mgr.gibba032.whopeo (mgr.24625) 3634 : cluster [DBG] pgmap v2062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:16 gibba032 bash[24004]: cluster 2021-03-10T15:01:15.404137+0000 mgr.gibba032.whopeo (mgr.24625) 3635 : cluster [DBG] pgmap v2063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:16 gibba011 bash[30791]: cluster 2021-03-10T15:01:15.404137+0000 mgr.gibba032.whopeo (mgr.24625) 3635 : cluster [DBG] pgmap v2063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:16 gibba011 bash[31774]: cluster 2021-03-10T15:01:15.404137+0000 mgr.gibba032.whopeo (mgr.24625) 3635 : cluster [DBG] pgmap v2063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:17 gibba032 bash[24004]: cluster 2021-03-10T15:01:17.405629+0000 mgr.gibba032.whopeo (mgr.24625) 3636 : cluster [DBG] pgmap v2064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:17 gibba011 bash[30791]: cluster 2021-03-10T15:01:17.405629+0000 mgr.gibba032.whopeo (mgr.24625) 3636 : cluster [DBG] pgmap v2064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:17 gibba011 bash[31774]: cluster 2021-03-10T15:01:17.405629+0000 mgr.gibba032.whopeo (mgr.24625) 3636 : cluster [DBG] pgmap v2064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:20 gibba032 bash[24004]: cluster 2021-03-10T15:01:19.406851+0000 mgr.gibba032.whopeo (mgr.24625) 3637 : cluster [DBG] pgmap v2065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:20 gibba011 bash[30791]: cluster 2021-03-10T15:01:19.406851+0000 mgr.gibba032.whopeo (mgr.24625) 3637 : cluster [DBG] pgmap v2065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:20 gibba011 bash[31774]: cluster 2021-03-10T15:01:19.406851+0000 mgr.gibba032.whopeo (mgr.24625) 3637 : cluster [DBG] pgmap v2065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:22 gibba032 bash[24004]: cluster 2021-03-10T15:01:21.408761+0000 mgr.gibba032.whopeo (mgr.24625) 3638 : cluster [DBG] pgmap v2066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:22 gibba011 bash[30791]: cluster 2021-03-10T15:01:21.408761+0000 mgr.gibba032.whopeo (mgr.24625) 3638 : cluster [DBG] pgmap v2066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:22.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:22 gibba011 bash[31774]: cluster 2021-03-10T15:01:21.408761+0000 mgr.gibba032.whopeo (mgr.24625) 3638 : cluster [DBG] pgmap v2066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:24 gibba032 bash[24004]: cluster 2021-03-10T15:01:23.410403+0000 mgr.gibba032.whopeo (mgr.24625) 3639 : cluster [DBG] pgmap v2067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:24 gibba011 bash[30791]: cluster 2021-03-10T15:01:23.410403+0000 mgr.gibba032.whopeo (mgr.24625) 3639 : cluster [DBG] pgmap v2067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:24 gibba011 bash[31774]: cluster 2021-03-10T15:01:23.410403+0000 mgr.gibba032.whopeo (mgr.24625) 3639 : cluster [DBG] pgmap v2067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:26 gibba032 bash[24004]: cluster 2021-03-10T15:01:25.412008+0000 mgr.gibba032.whopeo (mgr.24625) 3640 : cluster [DBG] pgmap v2068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:26 gibba011 bash[30791]: cluster 2021-03-10T15:01:25.412008+0000 mgr.gibba032.whopeo (mgr.24625) 3640 : cluster [DBG] pgmap v2068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:26 gibba011 bash[31774]: cluster 2021-03-10T15:01:25.412008+0000 mgr.gibba032.whopeo (mgr.24625) 3640 : cluster [DBG] pgmap v2068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:27 gibba032 bash[24004]: cluster 2021-03-10T15:01:27.413633+0000 mgr.gibba032.whopeo (mgr.24625) 3641 : cluster [DBG] pgmap v2069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:27 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.669847+0000 mgr.gibba032.whopeo (mgr.24625) 3642 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:01:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:27 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.671188+0000 mgr.gibba032.whopeo (mgr.24625) 3643 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:01:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[30791]: cluster 2021-03-10T15:01:27.413633+0000 mgr.gibba032.whopeo (mgr.24625) 3641 : cluster [DBG] pgmap v2069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.669847+0000 mgr.gibba032.whopeo (mgr.24625) 3642 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:01:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.671188+0000 mgr.gibba032.whopeo (mgr.24625) 3643 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:01:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[31774]: cluster 2021-03-10T15:01:27.413633+0000 mgr.gibba032.whopeo (mgr.24625) 3641 : cluster [DBG] pgmap v2069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.669847+0000 mgr.gibba032.whopeo (mgr.24625) 3642 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:01:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:27 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.671188+0000 mgr.gibba032.whopeo (mgr.24625) 3643 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:01:29.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.897511+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:01:29.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.898054+0000 mgr.gibba032.whopeo (mgr.24625) 3644 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:01:29.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.900501+0000 mgr.gibba032.whopeo (mgr.24625) 3645 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:01:29.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.901227+0000 mon.a (mon.0) 1595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:01:29.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.902506+0000 mgr.gibba032.whopeo (mgr.24625) 3646 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:01:29.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.902544+0000 mgr.gibba032.whopeo (mgr.24625) 3647 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:01:29.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.903320+0000 mon.a (mon.0) 1596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.904433+0000 mgr.gibba032.whopeo (mgr.24625) 3648 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:01:29.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.904469+0000 mgr.gibba032.whopeo (mgr.24625) 3649 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:01:29.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.905347+0000 mon.a (mon.0) 1597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.906118+0000 mgr.gibba032.whopeo (mgr.24625) 3650 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:01:29.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.907020+0000 mgr.gibba032.whopeo (mgr.24625) 3651 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:01:29.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.907065+0000 mon.a (mon.0) 1598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.907093+0000 mgr.gibba032.whopeo (mgr.24625) 3652 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:01:29.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.908425+0000 mgr.gibba032.whopeo (mgr.24625) 3653 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:01:29.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.908757+0000 mgr.gibba032.whopeo (mgr.24625) 3654 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:01:29.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.909711+0000 mon.a (mon.0) 1599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.910082+0000 mgr.gibba032.whopeo (mgr.24625) 3655 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:01:29.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.910140+0000 mgr.gibba032.whopeo (mgr.24625) 3656 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:01:29.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: cephadm 2021-03-10T15:01:27.910243+0000 mgr.gibba032.whopeo (mgr.24625) 3657 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:29.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:28 gibba032 bash[24004]: audit 2021-03-10T15:01:27.911346+0000 mon.a (mon.0) 1600 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.897511+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:01:29.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.898054+0000 mgr.gibba032.whopeo (mgr.24625) 3644 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:01:29.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.900501+0000 mgr.gibba032.whopeo (mgr.24625) 3645 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:01:29.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.901227+0000 mon.a (mon.0) 1595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:01:29.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.902506+0000 mgr.gibba032.whopeo (mgr.24625) 3646 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:01:29.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.902544+0000 mgr.gibba032.whopeo (mgr.24625) 3647 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:01:29.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.903320+0000 mon.a (mon.0) 1596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.904433+0000 mgr.gibba032.whopeo (mgr.24625) 3648 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:01:29.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.904469+0000 mgr.gibba032.whopeo (mgr.24625) 3649 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:01:29.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.905347+0000 mon.a (mon.0) 1597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.906118+0000 mgr.gibba032.whopeo (mgr.24625) 3650 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:01:29.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.907020+0000 mgr.gibba032.whopeo (mgr.24625) 3651 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:01:29.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.907065+0000 mon.a (mon.0) 1598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.907093+0000 mgr.gibba032.whopeo (mgr.24625) 3652 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:01:29.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.908425+0000 mgr.gibba032.whopeo (mgr.24625) 3653 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:01:29.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.908757+0000 mgr.gibba032.whopeo (mgr.24625) 3654 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:01:29.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.909711+0000 mon.a (mon.0) 1599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.910082+0000 mgr.gibba032.whopeo (mgr.24625) 3655 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:01:29.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.910140+0000 mgr.gibba032.whopeo (mgr.24625) 3656 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:01:29.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: cephadm 2021-03-10T15:01:27.910243+0000 mgr.gibba032.whopeo (mgr.24625) 3657 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:29.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[30791]: audit 2021-03-10T15:01:27.911346+0000 mon.a (mon.0) 1600 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.897511+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:01:29.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.898054+0000 mgr.gibba032.whopeo (mgr.24625) 3644 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:01:29.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.900501+0000 mgr.gibba032.whopeo (mgr.24625) 3645 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:01:29.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.901227+0000 mon.a (mon.0) 1595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:01:29.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.902506+0000 mgr.gibba032.whopeo (mgr.24625) 3646 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:01:29.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.902544+0000 mgr.gibba032.whopeo (mgr.24625) 3647 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:01:29.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.903320+0000 mon.a (mon.0) 1596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.904433+0000 mgr.gibba032.whopeo (mgr.24625) 3648 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:01:29.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.904469+0000 mgr.gibba032.whopeo (mgr.24625) 3649 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:01:29.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.905347+0000 mon.a (mon.0) 1597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.906118+0000 mgr.gibba032.whopeo (mgr.24625) 3650 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:01:29.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.907020+0000 mgr.gibba032.whopeo (mgr.24625) 3651 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:01:29.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.907065+0000 mon.a (mon.0) 1598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.907093+0000 mgr.gibba032.whopeo (mgr.24625) 3652 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:01:29.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.908425+0000 mgr.gibba032.whopeo (mgr.24625) 3653 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:01:29.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.908757+0000 mgr.gibba032.whopeo (mgr.24625) 3654 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:01:29.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.909711+0000 mon.a (mon.0) 1599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:29.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.910082+0000 mgr.gibba032.whopeo (mgr.24625) 3655 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:01:29.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.910140+0000 mgr.gibba032.whopeo (mgr.24625) 3656 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:01:29.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: cephadm 2021-03-10T15:01:27.910243+0000 mgr.gibba032.whopeo (mgr.24625) 3657 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:29.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:28 gibba011 bash[31774]: audit 2021-03-10T15:01:27.911346+0000 mon.a (mon.0) 1600 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:29 gibba032 bash[24004]: cluster 2021-03-10T15:01:29.414963+0000 mgr.gibba032.whopeo (mgr.24625) 3658 : cluster [DBG] pgmap v2070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:29 gibba011 bash[30791]: cluster 2021-03-10T15:01:29.414963+0000 mgr.gibba032.whopeo (mgr.24625) 3658 : cluster [DBG] pgmap v2070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:29 gibba011 bash[31774]: cluster 2021-03-10T15:01:29.414963+0000 mgr.gibba032.whopeo (mgr.24625) 3658 : cluster [DBG] pgmap v2070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:30.352 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:01:30.579 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:01:30.580 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (68m) 3m ago 73m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:01:30.580 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (73m) 2m ago 73m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:01:30.580 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (68m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:01:30.580 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (68m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:01:30.581 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (67m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:01:30.581 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (67m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:01:30.581 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (67m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:01:30.581 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (67m) 3m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:01:30.582 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (73m) 3m ago 73m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:01:30.582 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (73m) 2m ago 73m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:01:30.582 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (66m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:01:30.582 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (66m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:01:30.583 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (65m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:01:30.583 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (65m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:01:30.583 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (64m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:01:30.583 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (64m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:01:30.583 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (63m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:01:30.584 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (63m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:01:30.584 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (73m) 2m ago 73m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:01:30.584 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 71m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:01:30.830 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:01:30.830 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:01:30.831 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:01:30.831 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:30.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:01:30.831 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:01:30.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:30.832 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:01:30.832 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:01:30.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:01:30.833 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:01:30.833 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:01:30.833 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:01:30.833 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:01:30.833 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:01:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:30 gibba032 bash[24004]: audit 2021-03-10T15:01:30.339144+0000 mgr.gibba032.whopeo (mgr.24625) 3659 : audit [DBG] from='client.46503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:30 gibba032 bash[24004]: audit 2021-03-10T15:01:30.562380+0000 mgr.gibba032.whopeo (mgr.24625) 3660 : audit [DBG] from='client.46509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:30 gibba032 bash[24004]: audit 2021-03-10T15:01:30.827875+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.2.111:0/1026845458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[30791]: audit 2021-03-10T15:01:30.339144+0000 mgr.gibba032.whopeo (mgr.24625) 3659 : audit [DBG] from='client.46503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[30791]: audit 2021-03-10T15:01:30.562380+0000 mgr.gibba032.whopeo (mgr.24625) 3660 : audit [DBG] from='client.46509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[30791]: audit 2021-03-10T15:01:30.827875+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.2.111:0/1026845458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[31774]: audit 2021-03-10T15:01:30.339144+0000 mgr.gibba032.whopeo (mgr.24625) 3659 : audit [DBG] from='client.46503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[31774]: audit 2021-03-10T15:01:30.562380+0000 mgr.gibba032.whopeo (mgr.24625) 3660 : audit [DBG] from='client.46509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:01:31.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:30 gibba011 bash[31774]: audit 2021-03-10T15:01:30.827875+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.2.111:0/1026845458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:01:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:31 gibba032 bash[24004]: cluster 2021-03-10T15:01:31.416744+0000 mgr.gibba032.whopeo (mgr.24625) 3661 : cluster [DBG] pgmap v2071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:31 gibba011 bash[30791]: cluster 2021-03-10T15:01:31.416744+0000 mgr.gibba032.whopeo (mgr.24625) 3661 : cluster [DBG] pgmap v2071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:31 gibba011 bash[31774]: cluster 2021-03-10T15:01:31.416744+0000 mgr.gibba032.whopeo (mgr.24625) 3661 : cluster [DBG] pgmap v2071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:34 gibba032 bash[24004]: cluster 2021-03-10T15:01:33.418177+0000 mgr.gibba032.whopeo (mgr.24625) 3662 : cluster [DBG] pgmap v2072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:34 gibba011 bash[30791]: cluster 2021-03-10T15:01:33.418177+0000 mgr.gibba032.whopeo (mgr.24625) 3662 : cluster [DBG] pgmap v2072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:34.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:34 gibba011 bash[31774]: cluster 2021-03-10T15:01:33.418177+0000 mgr.gibba032.whopeo (mgr.24625) 3662 : cluster [DBG] pgmap v2072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:36 gibba032 bash[24004]: cluster 2021-03-10T15:01:35.419599+0000 mgr.gibba032.whopeo (mgr.24625) 3663 : cluster [DBG] pgmap v2073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:36 gibba011 bash[30791]: cluster 2021-03-10T15:01:35.419599+0000 mgr.gibba032.whopeo (mgr.24625) 3663 : cluster [DBG] pgmap v2073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:36 gibba011 bash[31774]: cluster 2021-03-10T15:01:35.419599+0000 mgr.gibba032.whopeo (mgr.24625) 3663 : cluster [DBG] pgmap v2073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:37 gibba032 bash[24004]: cluster 2021-03-10T15:01:37.421330+0000 mgr.gibba032.whopeo (mgr.24625) 3664 : cluster [DBG] pgmap v2074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:37 gibba011 bash[30791]: cluster 2021-03-10T15:01:37.421330+0000 mgr.gibba032.whopeo (mgr.24625) 3664 : cluster [DBG] pgmap v2074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:37 gibba011 bash[31774]: cluster 2021-03-10T15:01:37.421330+0000 mgr.gibba032.whopeo (mgr.24625) 3664 : cluster [DBG] pgmap v2074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:40 gibba032 bash[24004]: cluster 2021-03-10T15:01:39.422743+0000 mgr.gibba032.whopeo (mgr.24625) 3665 : cluster [DBG] pgmap v2075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:40 gibba011 bash[30791]: cluster 2021-03-10T15:01:39.422743+0000 mgr.gibba032.whopeo (mgr.24625) 3665 : cluster [DBG] pgmap v2075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:40 gibba011 bash[31774]: cluster 2021-03-10T15:01:39.422743+0000 mgr.gibba032.whopeo (mgr.24625) 3665 : cluster [DBG] pgmap v2075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:42 gibba032 bash[24004]: cluster 2021-03-10T15:01:41.424635+0000 mgr.gibba032.whopeo (mgr.24625) 3666 : cluster [DBG] pgmap v2076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:42 gibba011 bash[30791]: cluster 2021-03-10T15:01:41.424635+0000 mgr.gibba032.whopeo (mgr.24625) 3666 : cluster [DBG] pgmap v2076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:42 gibba011 bash[31774]: cluster 2021-03-10T15:01:41.424635+0000 mgr.gibba032.whopeo (mgr.24625) 3666 : cluster [DBG] pgmap v2076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:44 gibba032 bash[24004]: cephadm 2021-03-10T15:01:42.924963+0000 mgr.gibba032.whopeo (mgr.24625) 3667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:44.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:44 gibba032 bash[24004]: cluster 2021-03-10T15:01:43.426065+0000 mgr.gibba032.whopeo (mgr.24625) 3668 : cluster [DBG] pgmap v2077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:44 gibba011 bash[30791]: cephadm 2021-03-10T15:01:42.924963+0000 mgr.gibba032.whopeo (mgr.24625) 3667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:44 gibba011 bash[30791]: cluster 2021-03-10T15:01:43.426065+0000 mgr.gibba032.whopeo (mgr.24625) 3668 : cluster [DBG] pgmap v2077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:44 gibba011 bash[31774]: cephadm 2021-03-10T15:01:42.924963+0000 mgr.gibba032.whopeo (mgr.24625) 3667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:44.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:44 gibba011 bash[31774]: cluster 2021-03-10T15:01:43.426065+0000 mgr.gibba032.whopeo (mgr.24625) 3668 : cluster [DBG] pgmap v2077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:46 gibba011 bash[30791]: cluster 2021-03-10T15:01:45.427538+0000 mgr.gibba032.whopeo (mgr.24625) 3669 : cluster [DBG] pgmap v2078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:46.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:46 gibba011 bash[31774]: cluster 2021-03-10T15:01:45.427538+0000 mgr.gibba032.whopeo (mgr.24625) 3669 : cluster [DBG] pgmap v2078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:46 gibba032 bash[24004]: cluster 2021-03-10T15:01:45.427538+0000 mgr.gibba032.whopeo (mgr.24625) 3669 : cluster [DBG] pgmap v2078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:47 gibba032 bash[24004]: cluster 2021-03-10T15:01:47.429199+0000 mgr.gibba032.whopeo (mgr.24625) 3670 : cluster [DBG] pgmap v2079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:47 gibba011 bash[30791]: cluster 2021-03-10T15:01:47.429199+0000 mgr.gibba032.whopeo (mgr.24625) 3670 : cluster [DBG] pgmap v2079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:47 gibba011 bash[31774]: cluster 2021-03-10T15:01:47.429199+0000 mgr.gibba032.whopeo (mgr.24625) 3670 : cluster [DBG] pgmap v2079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:50 gibba032 bash[24004]: cluster 2021-03-10T15:01:49.430506+0000 mgr.gibba032.whopeo (mgr.24625) 3671 : cluster [DBG] pgmap v2080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:50 gibba011 bash[30791]: cluster 2021-03-10T15:01:49.430506+0000 mgr.gibba032.whopeo (mgr.24625) 3671 : cluster [DBG] pgmap v2080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:50 gibba011 bash[31774]: cluster 2021-03-10T15:01:49.430506+0000 mgr.gibba032.whopeo (mgr.24625) 3671 : cluster [DBG] pgmap v2080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:52 gibba011 bash[30791]: cluster 2021-03-10T15:01:51.432291+0000 mgr.gibba032.whopeo (mgr.24625) 3672 : cluster [DBG] pgmap v2081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:52 gibba011 bash[31774]: cluster 2021-03-10T15:01:51.432291+0000 mgr.gibba032.whopeo (mgr.24625) 3672 : cluster [DBG] pgmap v2081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:52 gibba032 bash[24004]: cluster 2021-03-10T15:01:51.432291+0000 mgr.gibba032.whopeo (mgr.24625) 3672 : cluster [DBG] pgmap v2081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:54 gibba011 bash[30791]: cluster 2021-03-10T15:01:53.433698+0000 mgr.gibba032.whopeo (mgr.24625) 3673 : cluster [DBG] pgmap v2082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:54 gibba011 bash[31774]: cluster 2021-03-10T15:01:53.433698+0000 mgr.gibba032.whopeo (mgr.24625) 3673 : cluster [DBG] pgmap v2082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:54 gibba032 bash[24004]: cluster 2021-03-10T15:01:53.433698+0000 mgr.gibba032.whopeo (mgr.24625) 3673 : cluster [DBG] pgmap v2082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: cluster 2021-03-10T15:01:55.435286+0000 mgr.gibba032.whopeo (mgr.24625) 3674 : cluster [DBG] pgmap v2083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.451611+0000 mon.a (mon.0) 1602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:01:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.452281+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:01:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.468107+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T15:01:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.468906+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.469602+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[30791]: audit 2021-03-10T15:01:56.470279+0000 mon.a (mon.0) 1607 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:01:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: cluster 2021-03-10T15:01:55.435286+0000 mgr.gibba032.whopeo (mgr.24625) 3674 : cluster [DBG] pgmap v2083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.451611+0000 mon.a (mon.0) 1602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:01:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.452281+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:01:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.468107+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T15:01:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.468906+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.469602+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:56 gibba011 bash[31774]: audit 2021-03-10T15:01:56.470279+0000 mon.a (mon.0) 1607 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:01:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: cluster 2021-03-10T15:01:55.435286+0000 mgr.gibba032.whopeo (mgr.24625) 3674 : cluster [DBG] pgmap v2083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.451611+0000 mon.a (mon.0) 1602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:01:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.452281+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:01:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.468107+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T15:01:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.468906+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.469602+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:01:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:56 gibba032 bash[24004]: audit 2021-03-10T15:01:56.470279+0000 mon.a (mon.0) 1607 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:01:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:57 gibba032 bash[24004]: cluster 2021-03-10T15:01:57.437062+0000 mgr.gibba032.whopeo (mgr.24625) 3675 : cluster [DBG] pgmap v2084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:57 gibba011 bash[30791]: cluster 2021-03-10T15:01:57.437062+0000 mgr.gibba032.whopeo (mgr.24625) 3675 : cluster [DBG] pgmap v2084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:57 gibba011 bash[31774]: cluster 2021-03-10T15:01:57.437062+0000 mgr.gibba032.whopeo (mgr.24625) 3675 : cluster [DBG] pgmap v2084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:01:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:58 gibba032 bash[24004]: cephadm 2021-03-10T15:01:57.936466+0000 mgr.gibba032.whopeo (mgr.24625) 3676 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:59.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:58 gibba011 bash[30791]: cephadm 2021-03-10T15:01:57.936466+0000 mgr.gibba032.whopeo (mgr.24625) 3676 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:01:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:58 gibba011 bash[31774]: cephadm 2021-03-10T15:01:57.936466+0000 mgr.gibba032.whopeo (mgr.24625) 3676 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:01:59 gibba032 bash[24004]: cluster 2021-03-10T15:01:59.438435+0000 mgr.gibba032.whopeo (mgr.24625) 3677 : cluster [DBG] pgmap v2085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:01:59 gibba011 bash[30791]: cluster 2021-03-10T15:01:59.438435+0000 mgr.gibba032.whopeo (mgr.24625) 3677 : cluster [DBG] pgmap v2085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:01:59 gibba011 bash[31774]: cluster 2021-03-10T15:01:59.438435+0000 mgr.gibba032.whopeo (mgr.24625) 3677 : cluster [DBG] pgmap v2085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:01.073 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:02:01.300 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:02:01.300 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (68m) 3m ago 74m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:02:01.301 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (73m) 2m ago 73m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:02:01.301 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (69m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:02:01.301 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (69m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:02:01.301 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (68m) 3m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:02:01.301 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (68m) 3m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:02:01.302 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (67m) 2m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:02:01.302 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (67m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:02:01.302 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (74m) 3m ago 74m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:02:01.302 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (74m) 2m ago 74m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:02:01.302 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (66m) 3m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:02:01.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (66m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:02:01.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (66m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:02:01.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (65m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:02:01.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (65m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:02:01.303 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (64m) 2m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:02:01.304 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (64m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:02:01.304 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (63m) 2m ago 74m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:02:01.304 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (73m) 2m ago 74m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:02:01.304 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 72m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:02:01.548 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:02:01.549 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:02:01.549 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:02:01.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:01.549 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:02:01.550 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:02:01.550 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:01.550 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:02:01.550 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:02:01.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:01.551 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:02:01.551 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:02:01.551 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:02:01.551 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:02:01.552 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:02:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:02 gibba032 bash[24004]: audit 2021-03-10T15:02:01.058936+0000 mgr.gibba032.whopeo (mgr.24625) 3678 : audit [DBG] from='client.56506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:02 gibba032 bash[24004]: audit 2021-03-10T15:02:01.286156+0000 mgr.gibba032.whopeo (mgr.24625) 3679 : audit [DBG] from='client.56512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:02 gibba032 bash[24004]: cluster 2021-03-10T15:02:01.440225+0000 mgr.gibba032.whopeo (mgr.24625) 3680 : cluster [DBG] pgmap v2086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:02 gibba032 bash[24004]: audit 2021-03-10T15:02:01.545874+0000 mon.a (mon.0) 1608 : audit [DBG] from='client.? 172.21.2.111:0/627119699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[30791]: audit 2021-03-10T15:02:01.058936+0000 mgr.gibba032.whopeo (mgr.24625) 3678 : audit [DBG] from='client.56506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[30791]: audit 2021-03-10T15:02:01.286156+0000 mgr.gibba032.whopeo (mgr.24625) 3679 : audit [DBG] from='client.56512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[30791]: cluster 2021-03-10T15:02:01.440225+0000 mgr.gibba032.whopeo (mgr.24625) 3680 : cluster [DBG] pgmap v2086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[30791]: audit 2021-03-10T15:02:01.545874+0000 mon.a (mon.0) 1608 : audit [DBG] from='client.? 172.21.2.111:0/627119699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[31774]: audit 2021-03-10T15:02:01.058936+0000 mgr.gibba032.whopeo (mgr.24625) 3678 : audit [DBG] from='client.56506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[31774]: audit 2021-03-10T15:02:01.286156+0000 mgr.gibba032.whopeo (mgr.24625) 3679 : audit [DBG] from='client.56512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[31774]: cluster 2021-03-10T15:02:01.440225+0000 mgr.gibba032.whopeo (mgr.24625) 3680 : cluster [DBG] pgmap v2086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:02 gibba011 bash[31774]: audit 2021-03-10T15:02:01.545874+0000 mon.a (mon.0) 1608 : audit [DBG] from='client.? 172.21.2.111:0/627119699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:04 gibba011 bash[30791]: cluster 2021-03-10T15:02:03.441583+0000 mgr.gibba032.whopeo (mgr.24625) 3681 : cluster [DBG] pgmap v2087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:04 gibba011 bash[31774]: cluster 2021-03-10T15:02:03.441583+0000 mgr.gibba032.whopeo (mgr.24625) 3681 : cluster [DBG] pgmap v2087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:04 gibba032 bash[24004]: cluster 2021-03-10T15:02:03.441583+0000 mgr.gibba032.whopeo (mgr.24625) 3681 : cluster [DBG] pgmap v2087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:06 gibba011 bash[30791]: cluster 2021-03-10T15:02:05.443004+0000 mgr.gibba032.whopeo (mgr.24625) 3682 : cluster [DBG] pgmap v2088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:06 gibba011 bash[31774]: cluster 2021-03-10T15:02:05.443004+0000 mgr.gibba032.whopeo (mgr.24625) 3682 : cluster [DBG] pgmap v2088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:06 gibba032 bash[24004]: cluster 2021-03-10T15:02:05.443004+0000 mgr.gibba032.whopeo (mgr.24625) 3682 : cluster [DBG] pgmap v2088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:07 gibba032 bash[24004]: cluster 2021-03-10T15:02:07.444828+0000 mgr.gibba032.whopeo (mgr.24625) 3683 : cluster [DBG] pgmap v2089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:07 gibba011 bash[30791]: cluster 2021-03-10T15:02:07.444828+0000 mgr.gibba032.whopeo (mgr.24625) 3683 : cluster [DBG] pgmap v2089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:07 gibba011 bash[31774]: cluster 2021-03-10T15:02:07.444828+0000 mgr.gibba032.whopeo (mgr.24625) 3683 : cluster [DBG] pgmap v2089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:10 gibba011 bash[30791]: cluster 2021-03-10T15:02:09.446145+0000 mgr.gibba032.whopeo (mgr.24625) 3684 : cluster [DBG] pgmap v2090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:10 gibba011 bash[31774]: cluster 2021-03-10T15:02:09.446145+0000 mgr.gibba032.whopeo (mgr.24625) 3684 : cluster [DBG] pgmap v2090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:10 gibba032 bash[24004]: cluster 2021-03-10T15:02:09.446145+0000 mgr.gibba032.whopeo (mgr.24625) 3684 : cluster [DBG] pgmap v2090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:12 gibba011 bash[30791]: cluster 2021-03-10T15:02:11.447800+0000 mgr.gibba032.whopeo (mgr.24625) 3685 : cluster [DBG] pgmap v2091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:12 gibba011 bash[31774]: cluster 2021-03-10T15:02:11.447800+0000 mgr.gibba032.whopeo (mgr.24625) 3685 : cluster [DBG] pgmap v2091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:12.957 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:12 gibba032 bash[24004]: cluster 2021-03-10T15:02:11.447800+0000 mgr.gibba032.whopeo (mgr.24625) 3685 : cluster [DBG] pgmap v2091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:14 gibba011 bash[30791]: cephadm 2021-03-10T15:02:12.952528+0000 mgr.gibba032.whopeo (mgr.24625) 3686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:14 gibba011 bash[30791]: cluster 2021-03-10T15:02:13.449339+0000 mgr.gibba032.whopeo (mgr.24625) 3687 : cluster [DBG] pgmap v2092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:14 gibba011 bash[31774]: cephadm 2021-03-10T15:02:12.952528+0000 mgr.gibba032.whopeo (mgr.24625) 3686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:14.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:14 gibba011 bash[31774]: cluster 2021-03-10T15:02:13.449339+0000 mgr.gibba032.whopeo (mgr.24625) 3687 : cluster [DBG] pgmap v2092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:14 gibba032 bash[24004]: cephadm 2021-03-10T15:02:12.952528+0000 mgr.gibba032.whopeo (mgr.24625) 3686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:15.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:14 gibba032 bash[24004]: cluster 2021-03-10T15:02:13.449339+0000 mgr.gibba032.whopeo (mgr.24625) 3687 : cluster [DBG] pgmap v2092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:16 gibba011 bash[30791]: cluster 2021-03-10T15:02:15.450960+0000 mgr.gibba032.whopeo (mgr.24625) 3688 : cluster [DBG] pgmap v2093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:16 gibba011 bash[31774]: cluster 2021-03-10T15:02:15.450960+0000 mgr.gibba032.whopeo (mgr.24625) 3688 : cluster [DBG] pgmap v2093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:16 gibba032 bash[24004]: cluster 2021-03-10T15:02:15.450960+0000 mgr.gibba032.whopeo (mgr.24625) 3688 : cluster [DBG] pgmap v2093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:17 gibba032 bash[24004]: cluster 2021-03-10T15:02:17.452579+0000 mgr.gibba032.whopeo (mgr.24625) 3689 : cluster [DBG] pgmap v2094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:17 gibba011 bash[30791]: cluster 2021-03-10T15:02:17.452579+0000 mgr.gibba032.whopeo (mgr.24625) 3689 : cluster [DBG] pgmap v2094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:17 gibba011 bash[31774]: cluster 2021-03-10T15:02:17.452579+0000 mgr.gibba032.whopeo (mgr.24625) 3689 : cluster [DBG] pgmap v2094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:20 gibba011 bash[30791]: cluster 2021-03-10T15:02:19.453941+0000 mgr.gibba032.whopeo (mgr.24625) 3690 : cluster [DBG] pgmap v2095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:20 gibba011 bash[31774]: cluster 2021-03-10T15:02:19.453941+0000 mgr.gibba032.whopeo (mgr.24625) 3690 : cluster [DBG] pgmap v2095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:20 gibba032 bash[24004]: cluster 2021-03-10T15:02:19.453941+0000 mgr.gibba032.whopeo (mgr.24625) 3690 : cluster [DBG] pgmap v2095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:22 gibba011 bash[30791]: cluster 2021-03-10T15:02:21.455623+0000 mgr.gibba032.whopeo (mgr.24625) 3691 : cluster [DBG] pgmap v2096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:22 gibba011 bash[31774]: cluster 2021-03-10T15:02:21.455623+0000 mgr.gibba032.whopeo (mgr.24625) 3691 : cluster [DBG] pgmap v2096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:22 gibba032 bash[24004]: cluster 2021-03-10T15:02:21.455623+0000 mgr.gibba032.whopeo (mgr.24625) 3691 : cluster [DBG] pgmap v2096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:24 gibba011 bash[30791]: cluster 2021-03-10T15:02:23.457228+0000 mgr.gibba032.whopeo (mgr.24625) 3692 : cluster [DBG] pgmap v2097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:24 gibba011 bash[31774]: cluster 2021-03-10T15:02:23.457228+0000 mgr.gibba032.whopeo (mgr.24625) 3692 : cluster [DBG] pgmap v2097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:24 gibba032 bash[24004]: cluster 2021-03-10T15:02:23.457228+0000 mgr.gibba032.whopeo (mgr.24625) 3692 : cluster [DBG] pgmap v2097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:26 gibba011 bash[30791]: cluster 2021-03-10T15:02:25.458615+0000 mgr.gibba032.whopeo (mgr.24625) 3693 : cluster [DBG] pgmap v2098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:26 gibba011 bash[31774]: cluster 2021-03-10T15:02:25.458615+0000 mgr.gibba032.whopeo (mgr.24625) 3693 : cluster [DBG] pgmap v2098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:26 gibba032 bash[24004]: cluster 2021-03-10T15:02:25.458615+0000 mgr.gibba032.whopeo (mgr.24625) 3693 : cluster [DBG] pgmap v2098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:28.201 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:27 gibba032 bash[24004]: cluster 2021-03-10T15:02:27.460389+0000 mgr.gibba032.whopeo (mgr.24625) 3694 : cluster [DBG] pgmap v2099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:27 gibba011 bash[30791]: cluster 2021-03-10T15:02:27.460389+0000 mgr.gibba032.whopeo (mgr.24625) 3694 : cluster [DBG] pgmap v2099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:27 gibba011 bash[31774]: cluster 2021-03-10T15:02:27.460389+0000 mgr.gibba032.whopeo (mgr.24625) 3694 : cluster [DBG] pgmap v2099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:29.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:27.968509+0000 mgr.gibba032.whopeo (mgr.24625) 3695 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:02:29.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:27.969041+0000 mgr.gibba032.whopeo (mgr.24625) 3696 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:02:29.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.184906+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:02:29.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.185452+0000 mgr.gibba032.whopeo (mgr.24625) 3697 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:02:29.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.187439+0000 mgr.gibba032.whopeo (mgr.24625) 3698 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:02:29.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.188461+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:02:29.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.189094+0000 mgr.gibba032.whopeo (mgr.24625) 3699 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:02:29.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.189133+0000 mgr.gibba032.whopeo (mgr.24625) 3700 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:02:29.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.190081+0000 mon.a (mon.0) 1611 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.190616+0000 mgr.gibba032.whopeo (mgr.24625) 3701 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:02:29.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.190655+0000 mgr.gibba032.whopeo (mgr.24625) 3702 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:02:29.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.191820+0000 mon.a (mon.0) 1612 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.192000+0000 mgr.gibba032.whopeo (mgr.24625) 3703 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:02:29.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.193200+0000 mon.a (mon.0) 1613 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.193472+0000 mgr.gibba032.whopeo (mgr.24625) 3704 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:02:29.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.193510+0000 mgr.gibba032.whopeo (mgr.24625) 3705 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:02:29.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.194862+0000 mgr.gibba032.whopeo (mgr.24625) 3706 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:02:29.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.195178+0000 mgr.gibba032.whopeo (mgr.24625) 3707 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:02:29.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.196209+0000 mon.a (mon.0) 1614 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.196407+0000 mgr.gibba032.whopeo (mgr.24625) 3708 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:02:29.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.196446+0000 mgr.gibba032.whopeo (mgr.24625) 3709 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:02:29.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: cephadm 2021-03-10T15:02:28.196580+0000 mgr.gibba032.whopeo (mgr.24625) 3710 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:29.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:29 gibba032 bash[24004]: audit 2021-03-10T15:02:28.197730+0000 mon.a (mon.0) 1615 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:27.968509+0000 mgr.gibba032.whopeo (mgr.24625) 3695 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:02:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:27.969041+0000 mgr.gibba032.whopeo (mgr.24625) 3696 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:02:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.184906+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:02:29.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.185452+0000 mgr.gibba032.whopeo (mgr.24625) 3697 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:02:29.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.187439+0000 mgr.gibba032.whopeo (mgr.24625) 3698 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:02:29.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.188461+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:02:29.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.189094+0000 mgr.gibba032.whopeo (mgr.24625) 3699 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:02:29.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.189133+0000 mgr.gibba032.whopeo (mgr.24625) 3700 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:02:29.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.190081+0000 mon.a (mon.0) 1611 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.190616+0000 mgr.gibba032.whopeo (mgr.24625) 3701 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:02:29.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.190655+0000 mgr.gibba032.whopeo (mgr.24625) 3702 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:02:29.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.191820+0000 mon.a (mon.0) 1612 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.192000+0000 mgr.gibba032.whopeo (mgr.24625) 3703 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:02:29.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.193200+0000 mon.a (mon.0) 1613 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.193472+0000 mgr.gibba032.whopeo (mgr.24625) 3704 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:02:29.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.193510+0000 mgr.gibba032.whopeo (mgr.24625) 3705 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:02:29.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.194862+0000 mgr.gibba032.whopeo (mgr.24625) 3706 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:02:29.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.195178+0000 mgr.gibba032.whopeo (mgr.24625) 3707 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:02:29.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.196209+0000 mon.a (mon.0) 1614 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.196407+0000 mgr.gibba032.whopeo (mgr.24625) 3708 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:02:29.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.196446+0000 mgr.gibba032.whopeo (mgr.24625) 3709 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:02:29.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: cephadm 2021-03-10T15:02:28.196580+0000 mgr.gibba032.whopeo (mgr.24625) 3710 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:29.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[30791]: audit 2021-03-10T15:02:28.197730+0000 mon.a (mon.0) 1615 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:27.968509+0000 mgr.gibba032.whopeo (mgr.24625) 3695 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:02:29.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:27.969041+0000 mgr.gibba032.whopeo (mgr.24625) 3696 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:02:29.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.184906+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:02:29.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.185452+0000 mgr.gibba032.whopeo (mgr.24625) 3697 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:02:29.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.187439+0000 mgr.gibba032.whopeo (mgr.24625) 3698 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:02:29.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.188461+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:02:29.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.189094+0000 mgr.gibba032.whopeo (mgr.24625) 3699 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:02:29.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.189133+0000 mgr.gibba032.whopeo (mgr.24625) 3700 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:02:29.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.190081+0000 mon.a (mon.0) 1611 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.190616+0000 mgr.gibba032.whopeo (mgr.24625) 3701 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:02:29.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.190655+0000 mgr.gibba032.whopeo (mgr.24625) 3702 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:02:29.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.191820+0000 mon.a (mon.0) 1612 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.192000+0000 mgr.gibba032.whopeo (mgr.24625) 3703 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:02:29.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.193200+0000 mon.a (mon.0) 1613 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.193472+0000 mgr.gibba032.whopeo (mgr.24625) 3704 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:02:29.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.193510+0000 mgr.gibba032.whopeo (mgr.24625) 3705 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:02:29.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.194862+0000 mgr.gibba032.whopeo (mgr.24625) 3706 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:02:29.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.195178+0000 mgr.gibba032.whopeo (mgr.24625) 3707 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:02:29.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.196209+0000 mon.a (mon.0) 1614 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:29.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.196407+0000 mgr.gibba032.whopeo (mgr.24625) 3708 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:02:29.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.196446+0000 mgr.gibba032.whopeo (mgr.24625) 3709 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:02:29.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: cephadm 2021-03-10T15:02:28.196580+0000 mgr.gibba032.whopeo (mgr.24625) 3710 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:29.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:29 gibba011 bash[31774]: audit 2021-03-10T15:02:28.197730+0000 mon.a (mon.0) 1615 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:30 gibba032 bash[24004]: cluster 2021-03-10T15:02:29.461718+0000 mgr.gibba032.whopeo (mgr.24625) 3711 : cluster [DBG] pgmap v2100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:30 gibba011 bash[30791]: cluster 2021-03-10T15:02:29.461718+0000 mgr.gibba032.whopeo (mgr.24625) 3711 : cluster [DBG] pgmap v2100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:30 gibba011 bash[31774]: cluster 2021-03-10T15:02:29.461718+0000 mgr.gibba032.whopeo (mgr.24625) 3711 : cluster [DBG] pgmap v2100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:31.789 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:02:32.043 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:02:32.044 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (69m) 4m ago 74m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:02:32.044 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (74m) 3m ago 74m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:02:32.044 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (69m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:02:32.044 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (69m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:02:32.045 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (68m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:02:32.045 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (68m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:02:32.045 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (68m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:02:32.045 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (68m) 4m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:02:32.045 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (74m) 4m ago 74m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:02:32.046 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (74m) 3m ago 74m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:02:32.046 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (67m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:02:32.046 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (67m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:02:32.046 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (66m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:02:32.047 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (66m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:02:32.047 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (65m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:02:32.047 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (65m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:02:32.047 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (64m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:02:32.047 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (64m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:02:32.048 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (74m) 3m ago 74m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:02:32.048 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 72m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:02:32.297 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:02:32.297 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:02:32.297 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:02:32.297 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:32.298 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:02:32.298 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:02:32.298 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:32.298 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:02:32.298 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:02:32.299 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:02:32.299 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:02:32.299 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:02:32.299 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:02:32.300 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:02:32.300 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:02:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[30791]: cluster 2021-03-10T15:02:31.463584+0000 mgr.gibba032.whopeo (mgr.24625) 3712 : cluster [DBG] pgmap v2101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[30791]: audit 2021-03-10T15:02:31.776131+0000 mgr.gibba032.whopeo (mgr.24625) 3713 : audit [DBG] from='client.46539 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[30791]: audit 2021-03-10T15:02:32.294277+0000 mon.c (mon.1) 70 : audit [DBG] from='client.? 172.21.2.111:0/1745612863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[31774]: cluster 2021-03-10T15:02:31.463584+0000 mgr.gibba032.whopeo (mgr.24625) 3712 : cluster [DBG] pgmap v2101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[31774]: audit 2021-03-10T15:02:31.776131+0000 mgr.gibba032.whopeo (mgr.24625) 3713 : audit [DBG] from='client.46539 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:32 gibba011 bash[31774]: audit 2021-03-10T15:02:32.294277+0000 mon.c (mon.1) 70 : audit [DBG] from='client.? 172.21.2.111:0/1745612863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:32 gibba032 bash[24004]: cluster 2021-03-10T15:02:31.463584+0000 mgr.gibba032.whopeo (mgr.24625) 3712 : cluster [DBG] pgmap v2101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:32 gibba032 bash[24004]: audit 2021-03-10T15:02:31.776131+0000 mgr.gibba032.whopeo (mgr.24625) 3713 : audit [DBG] from='client.46539 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:33.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:32 gibba032 bash[24004]: audit 2021-03-10T15:02:32.294277+0000 mon.c (mon.1) 70 : audit [DBG] from='client.? 172.21.2.111:0/1745612863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:02:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:33 gibba011 bash[30791]: audit 2021-03-10T15:02:31.999099+0000 mgr.gibba032.whopeo (mgr.24625) 3714 : audit [DBG] from='client.46545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:33.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:33 gibba011 bash[31774]: audit 2021-03-10T15:02:31.999099+0000 mgr.gibba032.whopeo (mgr.24625) 3714 : audit [DBG] from='client.46545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:34.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:33 gibba032 bash[24004]: audit 2021-03-10T15:02:31.999099+0000 mgr.gibba032.whopeo (mgr.24625) 3714 : audit [DBG] from='client.46545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:02:34.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:34 gibba011 bash[30791]: cluster 2021-03-10T15:02:33.465131+0000 mgr.gibba032.whopeo (mgr.24625) 3715 : cluster [DBG] pgmap v2102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:34.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:34 gibba011 bash[31774]: cluster 2021-03-10T15:02:33.465131+0000 mgr.gibba032.whopeo (mgr.24625) 3715 : cluster [DBG] pgmap v2102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:35.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:34 gibba032 bash[24004]: cluster 2021-03-10T15:02:33.465131+0000 mgr.gibba032.whopeo (mgr.24625) 3715 : cluster [DBG] pgmap v2102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:36 gibba011 bash[30791]: cluster 2021-03-10T15:02:35.466631+0000 mgr.gibba032.whopeo (mgr.24625) 3716 : cluster [DBG] pgmap v2103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:36 gibba011 bash[31774]: cluster 2021-03-10T15:02:35.466631+0000 mgr.gibba032.whopeo (mgr.24625) 3716 : cluster [DBG] pgmap v2103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:36 gibba032 bash[24004]: cluster 2021-03-10T15:02:35.466631+0000 mgr.gibba032.whopeo (mgr.24625) 3716 : cluster [DBG] pgmap v2103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:37 gibba032 bash[24004]: cluster 2021-03-10T15:02:37.468479+0000 mgr.gibba032.whopeo (mgr.24625) 3717 : cluster [DBG] pgmap v2104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:37 gibba011 bash[30791]: cluster 2021-03-10T15:02:37.468479+0000 mgr.gibba032.whopeo (mgr.24625) 3717 : cluster [DBG] pgmap v2104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:37 gibba011 bash[31774]: cluster 2021-03-10T15:02:37.468479+0000 mgr.gibba032.whopeo (mgr.24625) 3717 : cluster [DBG] pgmap v2104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:40 gibba011 bash[30791]: cluster 2021-03-10T15:02:39.469720+0000 mgr.gibba032.whopeo (mgr.24625) 3718 : cluster [DBG] pgmap v2105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:40 gibba011 bash[31774]: cluster 2021-03-10T15:02:39.469720+0000 mgr.gibba032.whopeo (mgr.24625) 3718 : cluster [DBG] pgmap v2105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:40 gibba032 bash[24004]: cluster 2021-03-10T15:02:39.469720+0000 mgr.gibba032.whopeo (mgr.24625) 3718 : cluster [DBG] pgmap v2105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:42 gibba011 bash[30791]: cluster 2021-03-10T15:02:41.471460+0000 mgr.gibba032.whopeo (mgr.24625) 3719 : cluster [DBG] pgmap v2106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:42 gibba011 bash[31774]: cluster 2021-03-10T15:02:41.471460+0000 mgr.gibba032.whopeo (mgr.24625) 3719 : cluster [DBG] pgmap v2106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:42 gibba032 bash[24004]: cluster 2021-03-10T15:02:41.471460+0000 mgr.gibba032.whopeo (mgr.24625) 3719 : cluster [DBG] pgmap v2106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:44 gibba011 bash[30791]: cephadm 2021-03-10T15:02:43.208977+0000 mgr.gibba032.whopeo (mgr.24625) 3720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:44 gibba011 bash[30791]: cluster 2021-03-10T15:02:43.473076+0000 mgr.gibba032.whopeo (mgr.24625) 3721 : cluster [DBG] pgmap v2107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:44 gibba011 bash[31774]: cephadm 2021-03-10T15:02:43.208977+0000 mgr.gibba032.whopeo (mgr.24625) 3720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:44 gibba011 bash[31774]: cluster 2021-03-10T15:02:43.473076+0000 mgr.gibba032.whopeo (mgr.24625) 3721 : cluster [DBG] pgmap v2107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:44 gibba032 bash[24004]: cephadm 2021-03-10T15:02:43.208977+0000 mgr.gibba032.whopeo (mgr.24625) 3720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:44 gibba032 bash[24004]: cluster 2021-03-10T15:02:43.473076+0000 mgr.gibba032.whopeo (mgr.24625) 3721 : cluster [DBG] pgmap v2107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:46 gibba011 bash[30791]: cluster 2021-03-10T15:02:45.474552+0000 mgr.gibba032.whopeo (mgr.24625) 3722 : cluster [DBG] pgmap v2108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:46.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:46 gibba011 bash[31774]: cluster 2021-03-10T15:02:45.474552+0000 mgr.gibba032.whopeo (mgr.24625) 3722 : cluster [DBG] pgmap v2108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:46 gibba032 bash[24004]: cluster 2021-03-10T15:02:45.474552+0000 mgr.gibba032.whopeo (mgr.24625) 3722 : cluster [DBG] pgmap v2108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:47 gibba032 bash[24004]: cluster 2021-03-10T15:02:47.476181+0000 mgr.gibba032.whopeo (mgr.24625) 3723 : cluster [DBG] pgmap v2109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:47 gibba011 bash[30791]: cluster 2021-03-10T15:02:47.476181+0000 mgr.gibba032.whopeo (mgr.24625) 3723 : cluster [DBG] pgmap v2109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:47 gibba011 bash[31774]: cluster 2021-03-10T15:02:47.476181+0000 mgr.gibba032.whopeo (mgr.24625) 3723 : cluster [DBG] pgmap v2109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:50 gibba011 bash[30791]: cluster 2021-03-10T15:02:49.477592+0000 mgr.gibba032.whopeo (mgr.24625) 3724 : cluster [DBG] pgmap v2110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:50 gibba011 bash[31774]: cluster 2021-03-10T15:02:49.477592+0000 mgr.gibba032.whopeo (mgr.24625) 3724 : cluster [DBG] pgmap v2110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:50 gibba032 bash[24004]: cluster 2021-03-10T15:02:49.477592+0000 mgr.gibba032.whopeo (mgr.24625) 3724 : cluster [DBG] pgmap v2110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:52 gibba011 bash[30791]: cluster 2021-03-10T15:02:51.479336+0000 mgr.gibba032.whopeo (mgr.24625) 3725 : cluster [DBG] pgmap v2111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:52 gibba011 bash[31774]: cluster 2021-03-10T15:02:51.479336+0000 mgr.gibba032.whopeo (mgr.24625) 3725 : cluster [DBG] pgmap v2111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:52 gibba032 bash[24004]: cluster 2021-03-10T15:02:51.479336+0000 mgr.gibba032.whopeo (mgr.24625) 3725 : cluster [DBG] pgmap v2111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:54 gibba011 bash[30791]: cluster 2021-03-10T15:02:53.480727+0000 mgr.gibba032.whopeo (mgr.24625) 3726 : cluster [DBG] pgmap v2112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:54 gibba011 bash[31774]: cluster 2021-03-10T15:02:53.480727+0000 mgr.gibba032.whopeo (mgr.24625) 3726 : cluster [DBG] pgmap v2112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:54 gibba032 bash[24004]: cluster 2021-03-10T15:02:53.480727+0000 mgr.gibba032.whopeo (mgr.24625) 3726 : cluster [DBG] pgmap v2112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: cluster 2021-03-10T15:02:55.481936+0000 mgr.gibba032.whopeo (mgr.24625) 3727 : cluster [DBG] pgmap v2113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.459413+0000 mon.a (mon.0) 1616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:02:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.460325+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:02:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.473927+0000 mon.a (mon.0) 1618 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:02:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.474954+0000 mon.a (mon.0) 1619 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:02:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.476006+0000 mon.a (mon.0) 1620 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:02:56.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[30791]: audit 2021-03-10T15:02:56.477011+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:02:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: cluster 2021-03-10T15:02:55.481936+0000 mgr.gibba032.whopeo (mgr.24625) 3727 : cluster [DBG] pgmap v2113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.459413+0000 mon.a (mon.0) 1616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:02:56.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.460325+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:02:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.473927+0000 mon.a (mon.0) 1618 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:02:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.474954+0000 mon.a (mon.0) 1619 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:02:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.476006+0000 mon.a (mon.0) 1620 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:02:56.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:56 gibba011 bash[31774]: audit 2021-03-10T15:02:56.477011+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:02:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: cluster 2021-03-10T15:02:55.481936+0000 mgr.gibba032.whopeo (mgr.24625) 3727 : cluster [DBG] pgmap v2113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.459413+0000 mon.a (mon.0) 1616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:02:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.460325+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:02:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.473927+0000 mon.a (mon.0) 1618 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:02:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.474954+0000 mon.a (mon.0) 1619 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:02:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.476006+0000 mon.a (mon.0) 1620 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:02:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:56 gibba032 bash[24004]: audit 2021-03-10T15:02:56.477011+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:02:58.229 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:57 gibba032 bash[24004]: cluster 2021-03-10T15:02:57.483348+0000 mgr.gibba032.whopeo (mgr.24625) 3728 : cluster [DBG] pgmap v2114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:57 gibba011 bash[30791]: cluster 2021-03-10T15:02:57.483348+0000 mgr.gibba032.whopeo (mgr.24625) 3728 : cluster [DBG] pgmap v2114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:57 gibba011 bash[31774]: cluster 2021-03-10T15:02:57.483348+0000 mgr.gibba032.whopeo (mgr.24625) 3728 : cluster [DBG] pgmap v2114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:02:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:58 gibba032 bash[24004]: cephadm 2021-03-10T15:02:58.224654+0000 mgr.gibba032.whopeo (mgr.24625) 3729 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:59.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:58 gibba011 bash[30791]: cephadm 2021-03-10T15:02:58.224654+0000 mgr.gibba032.whopeo (mgr.24625) 3729 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:02:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:58 gibba011 bash[31774]: cephadm 2021-03-10T15:02:58.224654+0000 mgr.gibba032.whopeo (mgr.24625) 3729 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:02:59 gibba032 bash[24004]: cluster 2021-03-10T15:02:59.484486+0000 mgr.gibba032.whopeo (mgr.24625) 3730 : cluster [DBG] pgmap v2115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:02:59 gibba011 bash[30791]: cluster 2021-03-10T15:02:59.484486+0000 mgr.gibba032.whopeo (mgr.24625) 3730 : cluster [DBG] pgmap v2115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:02:59 gibba011 bash[31774]: cluster 2021-03-10T15:02:59.484486+0000 mgr.gibba032.whopeo (mgr.24625) 3730 : cluster [DBG] pgmap v2115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:02.540 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:03:02.771 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:03:02.771 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (69m) 4m ago 75m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:03:02.771 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (74m) 3m ago 74m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:03:02.772 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (70m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:03:02.772 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (70m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:03:02.772 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (69m) 4m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:03:02.772 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (69m) 4m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:03:02.773 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (68m) 3m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:03:02.773 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (68m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:03:02.773 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (75m) 4m ago 75m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:03:02.773 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (75m) 3m ago 75m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:03:02.773 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (67m) 4m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:03:02.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (67m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:03:02.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (67m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:03:02.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (66m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:03:02.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (66m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:03:02.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (65m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:03:02.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (65m) 3m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:03:02.775 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (65m) 3m ago 75m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:03:02.775 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (74m) 3m ago 75m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:03:02.776 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 73m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:03:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:02 gibba011 bash[30791]: cluster 2021-03-10T15:03:01.485153+0000 mgr.gibba032.whopeo (mgr.24625) 3731 : cluster [DBG] pgmap v2116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:02 gibba011 bash[31774]: cluster 2021-03-10T15:03:01.485153+0000 mgr.gibba032.whopeo (mgr.24625) 3731 : cluster [DBG] pgmap v2116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:02 gibba032 bash[24004]: cluster 2021-03-10T15:03:01.485153+0000 mgr.gibba032.whopeo (mgr.24625) 3731 : cluster [DBG] pgmap v2116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:03.020 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:03:03.021 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:03:03.021 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:03:03.021 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:03.021 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:03:03.022 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:03:03.022 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:03.022 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:03:03.022 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:03:03.023 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:03.023 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:03:03.023 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:03:03.023 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:03:03.024 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:03:03.024 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:03:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[30791]: audit 2021-03-10T15:03:02.526332+0000 mgr.gibba032.whopeo (mgr.24625) 3732 : audit [DBG] from='client.56542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[30791]: audit 2021-03-10T15:03:02.753660+0000 mgr.gibba032.whopeo (mgr.24625) 3733 : audit [DBG] from='client.46563 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[30791]: audit 2021-03-10T15:03:03.018360+0000 mon.c (mon.1) 71 : audit [DBG] from='client.? 172.21.2.111:0/748123756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:03.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[31774]: audit 2021-03-10T15:03:02.526332+0000 mgr.gibba032.whopeo (mgr.24625) 3732 : audit [DBG] from='client.56542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:03.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[31774]: audit 2021-03-10T15:03:02.753660+0000 mgr.gibba032.whopeo (mgr.24625) 3733 : audit [DBG] from='client.46563 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:03.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:03 gibba011 bash[31774]: audit 2021-03-10T15:03:03.018360+0000 mon.c (mon.1) 71 : audit [DBG] from='client.? 172.21.2.111:0/748123756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:03 gibba032 bash[24004]: audit 2021-03-10T15:03:02.526332+0000 mgr.gibba032.whopeo (mgr.24625) 3732 : audit [DBG] from='client.56542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:03 gibba032 bash[24004]: audit 2021-03-10T15:03:02.753660+0000 mgr.gibba032.whopeo (mgr.24625) 3733 : audit [DBG] from='client.46563 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:04.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:03 gibba032 bash[24004]: audit 2021-03-10T15:03:03.018360+0000 mon.c (mon.1) 71 : audit [DBG] from='client.? 172.21.2.111:0/748123756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:04 gibba011 bash[30791]: cluster 2021-03-10T15:03:03.486536+0000 mgr.gibba032.whopeo (mgr.24625) 3734 : cluster [DBG] pgmap v2117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:04 gibba011 bash[31774]: cluster 2021-03-10T15:03:03.486536+0000 mgr.gibba032.whopeo (mgr.24625) 3734 : cluster [DBG] pgmap v2117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:04 gibba032 bash[24004]: cluster 2021-03-10T15:03:03.486536+0000 mgr.gibba032.whopeo (mgr.24625) 3734 : cluster [DBG] pgmap v2117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:06 gibba011 bash[30791]: cluster 2021-03-10T15:03:05.488236+0000 mgr.gibba032.whopeo (mgr.24625) 3735 : cluster [DBG] pgmap v2118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:06 gibba011 bash[31774]: cluster 2021-03-10T15:03:05.488236+0000 mgr.gibba032.whopeo (mgr.24625) 3735 : cluster [DBG] pgmap v2118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:06 gibba032 bash[24004]: cluster 2021-03-10T15:03:05.488236+0000 mgr.gibba032.whopeo (mgr.24625) 3735 : cluster [DBG] pgmap v2118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:07 gibba032 bash[24004]: cluster 2021-03-10T15:03:07.488910+0000 mgr.gibba032.whopeo (mgr.24625) 3736 : cluster [DBG] pgmap v2119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:07 gibba011 bash[30791]: cluster 2021-03-10T15:03:07.488910+0000 mgr.gibba032.whopeo (mgr.24625) 3736 : cluster [DBG] pgmap v2119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:07 gibba011 bash[31774]: cluster 2021-03-10T15:03:07.488910+0000 mgr.gibba032.whopeo (mgr.24625) 3736 : cluster [DBG] pgmap v2119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:10 gibba011 bash[30791]: cluster 2021-03-10T15:03:09.490105+0000 mgr.gibba032.whopeo (mgr.24625) 3737 : cluster [DBG] pgmap v2120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:10 gibba011 bash[31774]: cluster 2021-03-10T15:03:09.490105+0000 mgr.gibba032.whopeo (mgr.24625) 3737 : cluster [DBG] pgmap v2120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:10 gibba032 bash[24004]: cluster 2021-03-10T15:03:09.490105+0000 mgr.gibba032.whopeo (mgr.24625) 3737 : cluster [DBG] pgmap v2120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:12 gibba011 bash[30791]: cluster 2021-03-10T15:03:11.491854+0000 mgr.gibba032.whopeo (mgr.24625) 3738 : cluster [DBG] pgmap v2121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:12 gibba011 bash[31774]: cluster 2021-03-10T15:03:11.491854+0000 mgr.gibba032.whopeo (mgr.24625) 3738 : cluster [DBG] pgmap v2121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:12 gibba032 bash[24004]: cluster 2021-03-10T15:03:11.491854+0000 mgr.gibba032.whopeo (mgr.24625) 3738 : cluster [DBG] pgmap v2121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:14 gibba011 bash[30791]: cephadm 2021-03-10T15:03:13.232638+0000 mgr.gibba032.whopeo (mgr.24625) 3739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:14 gibba011 bash[30791]: cluster 2021-03-10T15:03:13.492401+0000 mgr.gibba032.whopeo (mgr.24625) 3740 : cluster [DBG] pgmap v2122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:14 gibba011 bash[31774]: cephadm 2021-03-10T15:03:13.232638+0000 mgr.gibba032.whopeo (mgr.24625) 3739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:14.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:14 gibba011 bash[31774]: cluster 2021-03-10T15:03:13.492401+0000 mgr.gibba032.whopeo (mgr.24625) 3740 : cluster [DBG] pgmap v2122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:14 gibba032 bash[24004]: cephadm 2021-03-10T15:03:13.232638+0000 mgr.gibba032.whopeo (mgr.24625) 3739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:15.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:14 gibba032 bash[24004]: cluster 2021-03-10T15:03:13.492401+0000 mgr.gibba032.whopeo (mgr.24625) 3740 : cluster [DBG] pgmap v2122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:16 gibba032 bash[24004]: cluster 2021-03-10T15:03:15.493637+0000 mgr.gibba032.whopeo (mgr.24625) 3741 : cluster [DBG] pgmap v2123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:16 gibba011 bash[30791]: cluster 2021-03-10T15:03:15.493637+0000 mgr.gibba032.whopeo (mgr.24625) 3741 : cluster [DBG] pgmap v2123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:16 gibba011 bash[31774]: cluster 2021-03-10T15:03:15.493637+0000 mgr.gibba032.whopeo (mgr.24625) 3741 : cluster [DBG] pgmap v2123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:17 gibba032 bash[24004]: cluster 2021-03-10T15:03:17.495265+0000 mgr.gibba032.whopeo (mgr.24625) 3742 : cluster [DBG] pgmap v2124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:17 gibba011 bash[30791]: cluster 2021-03-10T15:03:17.495265+0000 mgr.gibba032.whopeo (mgr.24625) 3742 : cluster [DBG] pgmap v2124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:17 gibba011 bash[31774]: cluster 2021-03-10T15:03:17.495265+0000 mgr.gibba032.whopeo (mgr.24625) 3742 : cluster [DBG] pgmap v2124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:20 gibba011 bash[30791]: cluster 2021-03-10T15:03:19.495945+0000 mgr.gibba032.whopeo (mgr.24625) 3743 : cluster [DBG] pgmap v2125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:20 gibba011 bash[31774]: cluster 2021-03-10T15:03:19.495945+0000 mgr.gibba032.whopeo (mgr.24625) 3743 : cluster [DBG] pgmap v2125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:20 gibba032 bash[24004]: cluster 2021-03-10T15:03:19.495945+0000 mgr.gibba032.whopeo (mgr.24625) 3743 : cluster [DBG] pgmap v2125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:22 gibba011 bash[30791]: cluster 2021-03-10T15:03:21.497349+0000 mgr.gibba032.whopeo (mgr.24625) 3744 : cluster [DBG] pgmap v2126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:22.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:22 gibba011 bash[31774]: cluster 2021-03-10T15:03:21.497349+0000 mgr.gibba032.whopeo (mgr.24625) 3744 : cluster [DBG] pgmap v2126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:22 gibba032 bash[24004]: cluster 2021-03-10T15:03:21.497349+0000 mgr.gibba032.whopeo (mgr.24625) 3744 : cluster [DBG] pgmap v2126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:24 gibba011 bash[30791]: cluster 2021-03-10T15:03:23.498955+0000 mgr.gibba032.whopeo (mgr.24625) 3745 : cluster [DBG] pgmap v2127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:24 gibba011 bash[31774]: cluster 2021-03-10T15:03:23.498955+0000 mgr.gibba032.whopeo (mgr.24625) 3745 : cluster [DBG] pgmap v2127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:24 gibba032 bash[24004]: cluster 2021-03-10T15:03:23.498955+0000 mgr.gibba032.whopeo (mgr.24625) 3745 : cluster [DBG] pgmap v2127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:26 gibba032 bash[24004]: cluster 2021-03-10T15:03:25.499725+0000 mgr.gibba032.whopeo (mgr.24625) 3746 : cluster [DBG] pgmap v2128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:26 gibba011 bash[30791]: cluster 2021-03-10T15:03:25.499725+0000 mgr.gibba032.whopeo (mgr.24625) 3746 : cluster [DBG] pgmap v2128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:27.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:26 gibba011 bash[31774]: cluster 2021-03-10T15:03:25.499725+0000 mgr.gibba032.whopeo (mgr.24625) 3746 : cluster [DBG] pgmap v2128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:27 gibba032 bash[24004]: cluster 2021-03-10T15:03:27.501151+0000 mgr.gibba032.whopeo (mgr.24625) 3747 : cluster [DBG] pgmap v2129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:27 gibba011 bash[30791]: cluster 2021-03-10T15:03:27.501151+0000 mgr.gibba032.whopeo (mgr.24625) 3747 : cluster [DBG] pgmap v2129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:27 gibba011 bash[31774]: cluster 2021-03-10T15:03:27.501151+0000 mgr.gibba032.whopeo (mgr.24625) 3747 : cluster [DBG] pgmap v2129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:29.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.249471+0000 mgr.gibba032.whopeo (mgr.24625) 3748 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:03:29.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.251005+0000 mgr.gibba032.whopeo (mgr.24625) 3749 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:03:29.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.482328+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:03:29.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.483331+0000 mgr.gibba032.whopeo (mgr.24625) 3750 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:03:29.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.486373+0000 mgr.gibba032.whopeo (mgr.24625) 3751 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:03:29.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.486561+0000 mon.a (mon.0) 1623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:03:29.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.489133+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.489806+0000 mgr.gibba032.whopeo (mgr.24625) 3752 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:03:29.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.490127+0000 mgr.gibba032.whopeo (mgr.24625) 3753 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:03:29.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.493831+0000 mon.a (mon.0) 1625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.494593+0000 mgr.gibba032.whopeo (mgr.24625) 3754 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:03:29.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.494844+0000 mgr.gibba032.whopeo (mgr.24625) 3755 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:03:29.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.498181+0000 mon.a (mon.0) 1626 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.498782+0000 mgr.gibba032.whopeo (mgr.24625) 3756 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:03:29.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.502168+0000 mgr.gibba032.whopeo (mgr.24625) 3757 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:03:29.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.502752+0000 mgr.gibba032.whopeo (mgr.24625) 3758 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:03:29.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.506792+0000 mon.a (mon.0) 1627 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.507577+0000 mgr.gibba032.whopeo (mgr.24625) 3759 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:03:29.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.509761+0000 mgr.gibba032.whopeo (mgr.24625) 3760 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:03:29.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: audit 2021-03-10T15:03:28.512812+0000 mon.a (mon.0) 1628 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.513681+0000 mgr.gibba032.whopeo (mgr.24625) 3761 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:03:29.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.514183+0000 mgr.gibba032.whopeo (mgr.24625) 3762 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:03:29.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:29 gibba032 bash[24004]: cephadm 2021-03-10T15:03:28.514828+0000 mgr.gibba032.whopeo (mgr.24625) 3763 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:29.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.249471+0000 mgr.gibba032.whopeo (mgr.24625) 3748 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:03:29.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.251005+0000 mgr.gibba032.whopeo (mgr.24625) 3749 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:03:29.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.482328+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:03:29.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.483331+0000 mgr.gibba032.whopeo (mgr.24625) 3750 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:03:29.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.486373+0000 mgr.gibba032.whopeo (mgr.24625) 3751 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:03:29.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.486561+0000 mon.a (mon.0) 1623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:03:29.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.489133+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.489806+0000 mgr.gibba032.whopeo (mgr.24625) 3752 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:03:29.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.490127+0000 mgr.gibba032.whopeo (mgr.24625) 3753 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:03:29.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.493831+0000 mon.a (mon.0) 1625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.494593+0000 mgr.gibba032.whopeo (mgr.24625) 3754 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:03:29.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.494844+0000 mgr.gibba032.whopeo (mgr.24625) 3755 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:03:29.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.498181+0000 mon.a (mon.0) 1626 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.498782+0000 mgr.gibba032.whopeo (mgr.24625) 3756 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:03:29.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.502168+0000 mgr.gibba032.whopeo (mgr.24625) 3757 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:03:29.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.502752+0000 mgr.gibba032.whopeo (mgr.24625) 3758 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:03:29.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.506792+0000 mon.a (mon.0) 1627 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.507577+0000 mgr.gibba032.whopeo (mgr.24625) 3759 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:03:29.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.509761+0000 mgr.gibba032.whopeo (mgr.24625) 3760 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:03:29.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: audit 2021-03-10T15:03:28.512812+0000 mon.a (mon.0) 1628 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.513681+0000 mgr.gibba032.whopeo (mgr.24625) 3761 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:03:29.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.249471+0000 mgr.gibba032.whopeo (mgr.24625) 3748 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:03:29.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.251005+0000 mgr.gibba032.whopeo (mgr.24625) 3749 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:03:29.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.482328+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:03:29.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.483331+0000 mgr.gibba032.whopeo (mgr.24625) 3750 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:03:29.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.486373+0000 mgr.gibba032.whopeo (mgr.24625) 3751 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:03:29.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.486561+0000 mon.a (mon.0) 1623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:03:29.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.489133+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.489806+0000 mgr.gibba032.whopeo (mgr.24625) 3752 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:03:29.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.490127+0000 mgr.gibba032.whopeo (mgr.24625) 3753 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:03:29.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.493831+0000 mon.a (mon.0) 1625 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.494593+0000 mgr.gibba032.whopeo (mgr.24625) 3754 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:03:29.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.494844+0000 mgr.gibba032.whopeo (mgr.24625) 3755 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:03:29.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.498181+0000 mon.a (mon.0) 1626 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.498782+0000 mgr.gibba032.whopeo (mgr.24625) 3756 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:03:29.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.502168+0000 mgr.gibba032.whopeo (mgr.24625) 3757 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:03:29.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.502752+0000 mgr.gibba032.whopeo (mgr.24625) 3758 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:03:29.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.506792+0000 mon.a (mon.0) 1627 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.507577+0000 mgr.gibba032.whopeo (mgr.24625) 3759 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:03:29.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.509761+0000 mgr.gibba032.whopeo (mgr.24625) 3760 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:03:29.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: audit 2021-03-10T15:03:28.512812+0000 mon.a (mon.0) 1628 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:29.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.513681+0000 mgr.gibba032.whopeo (mgr.24625) 3761 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:03:29.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.514183+0000 mgr.gibba032.whopeo (mgr.24625) 3762 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:03:29.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[31774]: cephadm 2021-03-10T15:03:28.514828+0000 mgr.gibba032.whopeo (mgr.24625) 3763 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:29.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.514183+0000 mgr.gibba032.whopeo (mgr.24625) 3762 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:03:29.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:29 gibba011 bash[30791]: cephadm 2021-03-10T15:03:28.514828+0000 mgr.gibba032.whopeo (mgr.24625) 3763 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:30 gibba032 bash[24004]: cluster 2021-03-10T15:03:29.501886+0000 mgr.gibba032.whopeo (mgr.24625) 3764 : cluster [DBG] pgmap v2130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:30 gibba011 bash[30791]: cluster 2021-03-10T15:03:29.501886+0000 mgr.gibba032.whopeo (mgr.24625) 3764 : cluster [DBG] pgmap v2130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:30 gibba011 bash[31774]: cluster 2021-03-10T15:03:29.501886+0000 mgr.gibba032.whopeo (mgr.24625) 3764 : cluster [DBG] pgmap v2130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:32 gibba011 bash[30791]: cluster 2021-03-10T15:03:31.503398+0000 mgr.gibba032.whopeo (mgr.24625) 3765 : cluster [DBG] pgmap v2131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:32 gibba011 bash[31774]: cluster 2021-03-10T15:03:31.503398+0000 mgr.gibba032.whopeo (mgr.24625) 3765 : cluster [DBG] pgmap v2131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:32 gibba032 bash[24004]: cluster 2021-03-10T15:03:31.503398+0000 mgr.gibba032.whopeo (mgr.24625) 3765 : cluster [DBG] pgmap v2131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:33.263 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:03:33.494 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:03:33.494 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (70m) 5m ago 75m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:03:33.494 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (75m) 4m ago 75m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:03:33.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (70m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:03:33.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (70m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:03:33.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (69m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:03:33.495 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (69m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:03:33.495 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (69m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:03:33.496 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (69m) 5m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:03:33.496 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (75m) 5m ago 75m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:03:33.496 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (75m) 4m ago 75m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:03:33.496 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (68m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:03:33.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (68m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:03:33.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (67m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:03:33.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (67m) 5m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:03:33.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (66m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:03:33.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (66m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:03:33.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (65m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:03:33.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (65m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:03:33.498 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (75m) 4m ago 75m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:03:33.498 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 73m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:03:33.745 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:03:33.746 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:03:33.746 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:03:33.746 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:33.746 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:03:33.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:03:33.747 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:33.747 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:03:33.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:03:33.748 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:03:33.748 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:03:33.748 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:03:33.748 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:03:33.748 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:03:33.749 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:03:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[30791]: audit 2021-03-10T15:03:33.249518+0000 mgr.gibba032.whopeo (mgr.24625) 3766 : audit [DBG] from='client.56560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[30791]: audit 2021-03-10T15:03:33.476617+0000 mgr.gibba032.whopeo (mgr.24625) 3767 : audit [DBG] from='client.46581 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[30791]: cluster 2021-03-10T15:03:33.504162+0000 mgr.gibba032.whopeo (mgr.24625) 3768 : cluster [DBG] pgmap v2132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:34.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[30791]: audit 2021-03-10T15:03:33.742636+0000 mon.c (mon.1) 72 : audit [DBG] from='client.? 172.21.2.111:0/2189280945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[31774]: audit 2021-03-10T15:03:33.249518+0000 mgr.gibba032.whopeo (mgr.24625) 3766 : audit [DBG] from='client.56560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[31774]: audit 2021-03-10T15:03:33.476617+0000 mgr.gibba032.whopeo (mgr.24625) 3767 : audit [DBG] from='client.46581 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[31774]: cluster 2021-03-10T15:03:33.504162+0000 mgr.gibba032.whopeo (mgr.24625) 3768 : cluster [DBG] pgmap v2132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:34.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:34 gibba011 bash[31774]: audit 2021-03-10T15:03:33.742636+0000 mon.c (mon.1) 72 : audit [DBG] from='client.? 172.21.2.111:0/2189280945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:35.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:34 gibba032 bash[24004]: audit 2021-03-10T15:03:33.249518+0000 mgr.gibba032.whopeo (mgr.24625) 3766 : audit [DBG] from='client.56560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:34 gibba032 bash[24004]: audit 2021-03-10T15:03:33.476617+0000 mgr.gibba032.whopeo (mgr.24625) 3767 : audit [DBG] from='client.46581 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:03:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:34 gibba032 bash[24004]: cluster 2021-03-10T15:03:33.504162+0000 mgr.gibba032.whopeo (mgr.24625) 3768 : cluster [DBG] pgmap v2132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:34 gibba032 bash[24004]: audit 2021-03-10T15:03:33.742636+0000 mon.c (mon.1) 72 : audit [DBG] from='client.? 172.21.2.111:0/2189280945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:03:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:36 gibba032 bash[24004]: cluster 2021-03-10T15:03:35.505393+0000 mgr.gibba032.whopeo (mgr.24625) 3769 : cluster [DBG] pgmap v2133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:36 gibba011 bash[30791]: cluster 2021-03-10T15:03:35.505393+0000 mgr.gibba032.whopeo (mgr.24625) 3769 : cluster [DBG] pgmap v2133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:36 gibba011 bash[31774]: cluster 2021-03-10T15:03:35.505393+0000 mgr.gibba032.whopeo (mgr.24625) 3769 : cluster [DBG] pgmap v2133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:37 gibba032 bash[24004]: cluster 2021-03-10T15:03:37.506292+0000 mgr.gibba032.whopeo (mgr.24625) 3770 : cluster [DBG] pgmap v2134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:37 gibba011 bash[30791]: cluster 2021-03-10T15:03:37.506292+0000 mgr.gibba032.whopeo (mgr.24625) 3770 : cluster [DBG] pgmap v2134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:37 gibba011 bash[31774]: cluster 2021-03-10T15:03:37.506292+0000 mgr.gibba032.whopeo (mgr.24625) 3770 : cluster [DBG] pgmap v2134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:40 gibba011 bash[30791]: cluster 2021-03-10T15:03:39.507677+0000 mgr.gibba032.whopeo (mgr.24625) 3771 : cluster [DBG] pgmap v2135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:40 gibba011 bash[31774]: cluster 2021-03-10T15:03:39.507677+0000 mgr.gibba032.whopeo (mgr.24625) 3771 : cluster [DBG] pgmap v2135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:40 gibba032 bash[24004]: cluster 2021-03-10T15:03:39.507677+0000 mgr.gibba032.whopeo (mgr.24625) 3771 : cluster [DBG] pgmap v2135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:42 gibba032 bash[24004]: cluster 2021-03-10T15:03:41.509211+0000 mgr.gibba032.whopeo (mgr.24625) 3772 : cluster [DBG] pgmap v2136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:42 gibba011 bash[30791]: cluster 2021-03-10T15:03:41.509211+0000 mgr.gibba032.whopeo (mgr.24625) 3772 : cluster [DBG] pgmap v2136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:43.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:42 gibba011 bash[31774]: cluster 2021-03-10T15:03:41.509211+0000 mgr.gibba032.whopeo (mgr.24625) 3772 : cluster [DBG] pgmap v2136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:44 gibba032 bash[24004]: cluster 2021-03-10T15:03:43.509954+0000 mgr.gibba032.whopeo (mgr.24625) 3773 : cluster [DBG] pgmap v2137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:44 gibba032 bash[24004]: cephadm 2021-03-10T15:03:43.518810+0000 mgr.gibba032.whopeo (mgr.24625) 3774 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:45.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:44 gibba011 bash[30791]: cluster 2021-03-10T15:03:43.509954+0000 mgr.gibba032.whopeo (mgr.24625) 3773 : cluster [DBG] pgmap v2137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:45.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:44 gibba011 bash[30791]: cephadm 2021-03-10T15:03:43.518810+0000 mgr.gibba032.whopeo (mgr.24625) 3774 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:45.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:44 gibba011 bash[31774]: cluster 2021-03-10T15:03:43.509954+0000 mgr.gibba032.whopeo (mgr.24625) 3773 : cluster [DBG] pgmap v2137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:45.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:44 gibba011 bash[31774]: cephadm 2021-03-10T15:03:43.518810+0000 mgr.gibba032.whopeo (mgr.24625) 3774 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:47.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:46 gibba032 bash[24004]: cluster 2021-03-10T15:03:45.511400+0000 mgr.gibba032.whopeo (mgr.24625) 3775 : cluster [DBG] pgmap v2138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:47.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:46 gibba011 bash[30791]: cluster 2021-03-10T15:03:45.511400+0000 mgr.gibba032.whopeo (mgr.24625) 3775 : cluster [DBG] pgmap v2138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:46 gibba011 bash[31774]: cluster 2021-03-10T15:03:45.511400+0000 mgr.gibba032.whopeo (mgr.24625) 3775 : cluster [DBG] pgmap v2138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:47 gibba032 bash[24004]: cluster 2021-03-10T15:03:47.513183+0000 mgr.gibba032.whopeo (mgr.24625) 3776 : cluster [DBG] pgmap v2139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:47 gibba011 bash[30791]: cluster 2021-03-10T15:03:47.513183+0000 mgr.gibba032.whopeo (mgr.24625) 3776 : cluster [DBG] pgmap v2139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:47 gibba011 bash[31774]: cluster 2021-03-10T15:03:47.513183+0000 mgr.gibba032.whopeo (mgr.24625) 3776 : cluster [DBG] pgmap v2139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:50 gibba011 bash[30791]: cluster 2021-03-10T15:03:49.513929+0000 mgr.gibba032.whopeo (mgr.24625) 3777 : cluster [DBG] pgmap v2140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:50 gibba011 bash[31774]: cluster 2021-03-10T15:03:49.513929+0000 mgr.gibba032.whopeo (mgr.24625) 3777 : cluster [DBG] pgmap v2140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:50 gibba032 bash[24004]: cluster 2021-03-10T15:03:49.513929+0000 mgr.gibba032.whopeo (mgr.24625) 3777 : cluster [DBG] pgmap v2140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:52 gibba032 bash[24004]: cluster 2021-03-10T15:03:51.515543+0000 mgr.gibba032.whopeo (mgr.24625) 3778 : cluster [DBG] pgmap v2141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:52 gibba011 bash[30791]: cluster 2021-03-10T15:03:51.515543+0000 mgr.gibba032.whopeo (mgr.24625) 3778 : cluster [DBG] pgmap v2141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:53.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:52 gibba011 bash[31774]: cluster 2021-03-10T15:03:51.515543+0000 mgr.gibba032.whopeo (mgr.24625) 3778 : cluster [DBG] pgmap v2141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:54 gibba032 bash[24004]: cluster 2021-03-10T15:03:53.517059+0000 mgr.gibba032.whopeo (mgr.24625) 3779 : cluster [DBG] pgmap v2142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:54 gibba011 bash[30791]: cluster 2021-03-10T15:03:53.517059+0000 mgr.gibba032.whopeo (mgr.24625) 3779 : cluster [DBG] pgmap v2142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:54 gibba011 bash[31774]: cluster 2021-03-10T15:03:53.517059+0000 mgr.gibba032.whopeo (mgr.24625) 3779 : cluster [DBG] pgmap v2142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: cluster 2021-03-10T15:03:55.517723+0000 mgr.gibba032.whopeo (mgr.24625) 3780 : cluster [DBG] pgmap v2143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.466062+0000 mon.a (mon.0) 1629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:03:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.466748+0000 mon.a (mon.0) 1630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:03:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.481053+0000 mon.a (mon.0) 1631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:03:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.481811+0000 mon.a (mon.0) 1632 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:03:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.482512+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:03:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:56 gibba032 bash[24004]: audit 2021-03-10T15:03:56.483195+0000 mon.a (mon.0) 1634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:03:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: cluster 2021-03-10T15:03:55.517723+0000 mgr.gibba032.whopeo (mgr.24625) 3780 : cluster [DBG] pgmap v2143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.466062+0000 mon.a (mon.0) 1629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:03:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.466748+0000 mon.a (mon.0) 1630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:03:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.481053+0000 mon.a (mon.0) 1631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:03:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.481811+0000 mon.a (mon.0) 1632 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:03:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.482512+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:03:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[30791]: audit 2021-03-10T15:03:56.483195+0000 mon.a (mon.0) 1634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:03:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: cluster 2021-03-10T15:03:55.517723+0000 mgr.gibba032.whopeo (mgr.24625) 3780 : cluster [DBG] pgmap v2143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.466062+0000 mon.a (mon.0) 1629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:03:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.466748+0000 mon.a (mon.0) 1630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:03:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.481053+0000 mon.a (mon.0) 1631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:03:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.481811+0000 mon.a (mon.0) 1632 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:03:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.482512+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:03:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:56 gibba011 bash[31774]: audit 2021-03-10T15:03:56.483195+0000 mon.a (mon.0) 1634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:03:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:57 gibba032 bash[24004]: cluster 2021-03-10T15:03:57.519352+0000 mgr.gibba032.whopeo (mgr.24625) 3781 : cluster [DBG] pgmap v2144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:57 gibba011 bash[30791]: cluster 2021-03-10T15:03:57.519352+0000 mgr.gibba032.whopeo (mgr.24625) 3781 : cluster [DBG] pgmap v2144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:57 gibba011 bash[31774]: cluster 2021-03-10T15:03:57.519352+0000 mgr.gibba032.whopeo (mgr.24625) 3781 : cluster [DBG] pgmap v2144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:03:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:58 gibba032 bash[24004]: cephadm 2021-03-10T15:03:58.534505+0000 mgr.gibba032.whopeo (mgr.24625) 3782 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:59.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:58 gibba011 bash[30791]: cephadm 2021-03-10T15:03:58.534505+0000 mgr.gibba032.whopeo (mgr.24625) 3782 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:03:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:58 gibba011 bash[31774]: cephadm 2021-03-10T15:03:58.534505+0000 mgr.gibba032.whopeo (mgr.24625) 3782 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:03:59 gibba032 bash[24004]: cluster 2021-03-10T15:03:59.520908+0000 mgr.gibba032.whopeo (mgr.24625) 3783 : cluster [DBG] pgmap v2145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:03:59 gibba011 bash[30791]: cluster 2021-03-10T15:03:59.520908+0000 mgr.gibba032.whopeo (mgr.24625) 3783 : cluster [DBG] pgmap v2145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:03:59 gibba011 bash[31774]: cluster 2021-03-10T15:03:59.520908+0000 mgr.gibba032.whopeo (mgr.24625) 3783 : cluster [DBG] pgmap v2145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:02 gibba032 bash[24004]: cluster 2021-03-10T15:04:01.521970+0000 mgr.gibba032.whopeo (mgr.24625) 3784 : cluster [DBG] pgmap v2146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:02 gibba011 bash[30791]: cluster 2021-03-10T15:04:01.521970+0000 mgr.gibba032.whopeo (mgr.24625) 3784 : cluster [DBG] pgmap v2146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:03.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:02 gibba011 bash[31774]: cluster 2021-03-10T15:04:01.521970+0000 mgr.gibba032.whopeo (mgr.24625) 3784 : cluster [DBG] pgmap v2146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:03.991 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:04:04.218 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:04:04.218 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (70m) 5m ago 76m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:04:04.218 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (75m) 4m ago 75m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:04:04.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (71m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:04:04.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (71m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:04:04.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (70m) 5m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:04:04.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (70m) 5m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:04:04.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (69m) 4m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:04:04.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (69m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:04:04.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (76m) 5m ago 76m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:04:04.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (76m) 4m ago 76m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:04:04.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (69m) 5m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:04:04.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (68m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:04:04.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (68m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:04:04.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (67m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:04:04.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (67m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:04:04.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (66m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:04:04.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (66m) 4m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:04:04.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (66m) 4m ago 76m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:04:04.222 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (75m) 4m ago 76m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:04:04.222 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 74m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:04:04.467 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:04:04.468 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:04:04.468 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:04:04.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:04.468 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:04:04.468 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:04:04.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:04.469 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:04:04.469 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:04:04.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:04.470 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:04:04.470 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:04:04.470 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:04:04.470 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:04:04.470 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:04:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:04 gibba032 bash[24004]: cluster 2021-03-10T15:04:03.523614+0000 mgr.gibba032.whopeo (mgr.24625) 3785 : cluster [DBG] pgmap v2147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:05.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:04 gibba032 bash[24004]: audit 2021-03-10T15:04:04.465020+0000 mon.c (mon.1) 73 : audit [DBG] from='client.? 172.21.2.111:0/2549499031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:05.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:04 gibba011 bash[30791]: cluster 2021-03-10T15:04:03.523614+0000 mgr.gibba032.whopeo (mgr.24625) 3785 : cluster [DBG] pgmap v2147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:05.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:04 gibba011 bash[30791]: audit 2021-03-10T15:04:04.465020+0000 mon.c (mon.1) 73 : audit [DBG] from='client.? 172.21.2.111:0/2549499031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:04 gibba011 bash[31774]: cluster 2021-03-10T15:04:03.523614+0000 mgr.gibba032.whopeo (mgr.24625) 3785 : cluster [DBG] pgmap v2147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:04 gibba011 bash[31774]: audit 2021-03-10T15:04:04.465020+0000 mon.c (mon.1) 73 : audit [DBG] from='client.? 172.21.2.111:0/2549499031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:05 gibba032 bash[24004]: audit 2021-03-10T15:04:03.977028+0000 mgr.gibba032.whopeo (mgr.24625) 3786 : audit [DBG] from='client.46593 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:06.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:05 gibba032 bash[24004]: audit 2021-03-10T15:04:04.201211+0000 mgr.gibba032.whopeo (mgr.24625) 3787 : audit [DBG] from='client.56584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:05 gibba011 bash[30791]: audit 2021-03-10T15:04:03.977028+0000 mgr.gibba032.whopeo (mgr.24625) 3786 : audit [DBG] from='client.46593 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:05 gibba011 bash[30791]: audit 2021-03-10T15:04:04.201211+0000 mgr.gibba032.whopeo (mgr.24625) 3787 : audit [DBG] from='client.56584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:06.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:05 gibba011 bash[31774]: audit 2021-03-10T15:04:03.977028+0000 mgr.gibba032.whopeo (mgr.24625) 3786 : audit [DBG] from='client.46593 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:06.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:05 gibba011 bash[31774]: audit 2021-03-10T15:04:04.201211+0000 mgr.gibba032.whopeo (mgr.24625) 3787 : audit [DBG] from='client.56584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:06 gibba032 bash[24004]: cluster 2021-03-10T15:04:05.525380+0000 mgr.gibba032.whopeo (mgr.24625) 3788 : cluster [DBG] pgmap v2148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:07.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:06 gibba011 bash[30791]: cluster 2021-03-10T15:04:05.525380+0000 mgr.gibba032.whopeo (mgr.24625) 3788 : cluster [DBG] pgmap v2148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:07.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:06 gibba011 bash[31774]: cluster 2021-03-10T15:04:05.525380+0000 mgr.gibba032.whopeo (mgr.24625) 3788 : cluster [DBG] pgmap v2148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:07 gibba032 bash[24004]: cluster 2021-03-10T15:04:07.526417+0000 mgr.gibba032.whopeo (mgr.24625) 3789 : cluster [DBG] pgmap v2149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:07 gibba011 bash[30791]: cluster 2021-03-10T15:04:07.526417+0000 mgr.gibba032.whopeo (mgr.24625) 3789 : cluster [DBG] pgmap v2149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:07 gibba011 bash[31774]: cluster 2021-03-10T15:04:07.526417+0000 mgr.gibba032.whopeo (mgr.24625) 3789 : cluster [DBG] pgmap v2149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:10 gibba032 bash[24004]: cluster 2021-03-10T15:04:09.527840+0000 mgr.gibba032.whopeo (mgr.24625) 3790 : cluster [DBG] pgmap v2150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:11.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:10 gibba011 bash[30791]: cluster 2021-03-10T15:04:09.527840+0000 mgr.gibba032.whopeo (mgr.24625) 3790 : cluster [DBG] pgmap v2150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:11.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:10 gibba011 bash[31774]: cluster 2021-03-10T15:04:09.527840+0000 mgr.gibba032.whopeo (mgr.24625) 3790 : cluster [DBG] pgmap v2150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:12 gibba032 bash[24004]: cluster 2021-03-10T15:04:11.529405+0000 mgr.gibba032.whopeo (mgr.24625) 3791 : cluster [DBG] pgmap v2151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:12 gibba011 bash[30791]: cluster 2021-03-10T15:04:11.529405+0000 mgr.gibba032.whopeo (mgr.24625) 3791 : cluster [DBG] pgmap v2151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:12 gibba011 bash[31774]: cluster 2021-03-10T15:04:11.529405+0000 mgr.gibba032.whopeo (mgr.24625) 3791 : cluster [DBG] pgmap v2151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:14 gibba032 bash[24004]: cluster 2021-03-10T15:04:13.530414+0000 mgr.gibba032.whopeo (mgr.24625) 3792 : cluster [DBG] pgmap v2152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:15.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:14 gibba032 bash[24004]: cephadm 2021-03-10T15:04:13.548572+0000 mgr.gibba032.whopeo (mgr.24625) 3793 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:14 gibba011 bash[30791]: cluster 2021-03-10T15:04:13.530414+0000 mgr.gibba032.whopeo (mgr.24625) 3792 : cluster [DBG] pgmap v2152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:15.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:14 gibba011 bash[30791]: cephadm 2021-03-10T15:04:13.548572+0000 mgr.gibba032.whopeo (mgr.24625) 3793 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:14 gibba011 bash[31774]: cluster 2021-03-10T15:04:13.530414+0000 mgr.gibba032.whopeo (mgr.24625) 3792 : cluster [DBG] pgmap v2152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:14 gibba011 bash[31774]: cephadm 2021-03-10T15:04:13.548572+0000 mgr.gibba032.whopeo (mgr.24625) 3793 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:16 gibba032 bash[24004]: cluster 2021-03-10T15:04:15.531884+0000 mgr.gibba032.whopeo (mgr.24625) 3794 : cluster [DBG] pgmap v2153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:16 gibba011 bash[30791]: cluster 2021-03-10T15:04:15.531884+0000 mgr.gibba032.whopeo (mgr.24625) 3794 : cluster [DBG] pgmap v2153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:16 gibba011 bash[31774]: cluster 2021-03-10T15:04:15.531884+0000 mgr.gibba032.whopeo (mgr.24625) 3794 : cluster [DBG] pgmap v2153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:17 gibba032 bash[24004]: cluster 2021-03-10T15:04:17.533674+0000 mgr.gibba032.whopeo (mgr.24625) 3795 : cluster [DBG] pgmap v2154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:17 gibba011 bash[30791]: cluster 2021-03-10T15:04:17.533674+0000 mgr.gibba032.whopeo (mgr.24625) 3795 : cluster [DBG] pgmap v2154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:17 gibba011 bash[31774]: cluster 2021-03-10T15:04:17.533674+0000 mgr.gibba032.whopeo (mgr.24625) 3795 : cluster [DBG] pgmap v2154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:20 gibba032 bash[24004]: cluster 2021-03-10T15:04:19.534643+0000 mgr.gibba032.whopeo (mgr.24625) 3796 : cluster [DBG] pgmap v2155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:21.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:20 gibba011 bash[30791]: cluster 2021-03-10T15:04:19.534643+0000 mgr.gibba032.whopeo (mgr.24625) 3796 : cluster [DBG] pgmap v2155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:21.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:20 gibba011 bash[31774]: cluster 2021-03-10T15:04:19.534643+0000 mgr.gibba032.whopeo (mgr.24625) 3796 : cluster [DBG] pgmap v2155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:22 gibba032 bash[24004]: cluster 2021-03-10T15:04:21.536262+0000 mgr.gibba032.whopeo (mgr.24625) 3797 : cluster [DBG] pgmap v2156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:22 gibba011 bash[30791]: cluster 2021-03-10T15:04:21.536262+0000 mgr.gibba032.whopeo (mgr.24625) 3797 : cluster [DBG] pgmap v2156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:23.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:22 gibba011 bash[31774]: cluster 2021-03-10T15:04:21.536262+0000 mgr.gibba032.whopeo (mgr.24625) 3797 : cluster [DBG] pgmap v2156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:24 gibba032 bash[24004]: cluster 2021-03-10T15:04:23.537876+0000 mgr.gibba032.whopeo (mgr.24625) 3798 : cluster [DBG] pgmap v2157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:24 gibba011 bash[30791]: cluster 2021-03-10T15:04:23.537876+0000 mgr.gibba032.whopeo (mgr.24625) 3798 : cluster [DBG] pgmap v2157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:25.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:24 gibba011 bash[31774]: cluster 2021-03-10T15:04:23.537876+0000 mgr.gibba032.whopeo (mgr.24625) 3798 : cluster [DBG] pgmap v2157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:26 gibba032 bash[24004]: cluster 2021-03-10T15:04:25.538806+0000 mgr.gibba032.whopeo (mgr.24625) 3799 : cluster [DBG] pgmap v2158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:26 gibba011 bash[30791]: cluster 2021-03-10T15:04:25.538806+0000 mgr.gibba032.whopeo (mgr.24625) 3799 : cluster [DBG] pgmap v2158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:27.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:26 gibba011 bash[31774]: cluster 2021-03-10T15:04:25.538806+0000 mgr.gibba032.whopeo (mgr.24625) 3799 : cluster [DBG] pgmap v2158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:27 gibba032 bash[24004]: cluster 2021-03-10T15:04:27.540452+0000 mgr.gibba032.whopeo (mgr.24625) 3800 : cluster [DBG] pgmap v2159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:27 gibba011 bash[30791]: cluster 2021-03-10T15:04:27.540452+0000 mgr.gibba032.whopeo (mgr.24625) 3800 : cluster [DBG] pgmap v2159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:27 gibba011 bash[31774]: cluster 2021-03-10T15:04:27.540452+0000 mgr.gibba032.whopeo (mgr.24625) 3800 : cluster [DBG] pgmap v2159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:30.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.564941+0000 mgr.gibba032.whopeo (mgr.24625) 3801 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:04:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.565933+0000 mgr.gibba032.whopeo (mgr.24625) 3802 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:04:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.785156+0000 mon.a (mon.0) 1635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:04:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.785438+0000 mgr.gibba032.whopeo (mgr.24625) 3803 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:04:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.787334+0000 mgr.gibba032.whopeo (mgr.24625) 3804 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:04:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.788346+0000 mon.a (mon.0) 1636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:04:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.788719+0000 mgr.gibba032.whopeo (mgr.24625) 3805 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:04:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.788762+0000 mgr.gibba032.whopeo (mgr.24625) 3806 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:04:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.789866+0000 mon.a (mon.0) 1637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.790231+0000 mgr.gibba032.whopeo (mgr.24625) 3807 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:04:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.790268+0000 mgr.gibba032.whopeo (mgr.24625) 3808 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:04:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.791382+0000 mon.a (mon.0) 1638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.791538+0000 mgr.gibba032.whopeo (mgr.24625) 3809 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:04:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.792851+0000 mon.a (mon.0) 1639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.793033+0000 mgr.gibba032.whopeo (mgr.24625) 3810 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:04:30.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.793069+0000 mgr.gibba032.whopeo (mgr.24625) 3811 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:04:30.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.794447+0000 mgr.gibba032.whopeo (mgr.24625) 3812 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:04:30.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.794775+0000 mgr.gibba032.whopeo (mgr.24625) 3813 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:04:30.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.795681+0000 mon.a (mon.0) 1640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.796240+0000 mgr.gibba032.whopeo (mgr.24625) 3814 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:04:30.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.796279+0000 mgr.gibba032.whopeo (mgr.24625) 3815 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:04:30.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: cephadm 2021-03-10T15:04:28.796386+0000 mgr.gibba032.whopeo (mgr.24625) 3816 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:30.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[30791]: audit 2021-03-10T15:04:28.797362+0000 mon.a (mon.0) 1641 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.564941+0000 mgr.gibba032.whopeo (mgr.24625) 3801 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:04:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.565933+0000 mgr.gibba032.whopeo (mgr.24625) 3802 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:04:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.785156+0000 mon.a (mon.0) 1635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:04:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.785438+0000 mgr.gibba032.whopeo (mgr.24625) 3803 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:04:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.787334+0000 mgr.gibba032.whopeo (mgr.24625) 3804 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:04:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.788346+0000 mon.a (mon.0) 1636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:04:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.788719+0000 mgr.gibba032.whopeo (mgr.24625) 3805 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:04:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.788762+0000 mgr.gibba032.whopeo (mgr.24625) 3806 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:04:30.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.789866+0000 mon.a (mon.0) 1637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.790231+0000 mgr.gibba032.whopeo (mgr.24625) 3807 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:04:30.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.790268+0000 mgr.gibba032.whopeo (mgr.24625) 3808 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:04:30.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.791382+0000 mon.a (mon.0) 1638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.791538+0000 mgr.gibba032.whopeo (mgr.24625) 3809 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:04:30.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.792851+0000 mon.a (mon.0) 1639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.793033+0000 mgr.gibba032.whopeo (mgr.24625) 3810 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:04:30.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.793069+0000 mgr.gibba032.whopeo (mgr.24625) 3811 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:04:30.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.794447+0000 mgr.gibba032.whopeo (mgr.24625) 3812 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:04:30.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.794775+0000 mgr.gibba032.whopeo (mgr.24625) 3813 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:04:30.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.795681+0000 mon.a (mon.0) 1640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.796240+0000 mgr.gibba032.whopeo (mgr.24625) 3814 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:04:30.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.796279+0000 mgr.gibba032.whopeo (mgr.24625) 3815 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:04:30.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: cephadm 2021-03-10T15:04:28.796386+0000 mgr.gibba032.whopeo (mgr.24625) 3816 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:30.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:29 gibba011 bash[31774]: audit 2021-03-10T15:04:28.797362+0000 mon.a (mon.0) 1641 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.564941+0000 mgr.gibba032.whopeo (mgr.24625) 3801 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:04:30.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.565933+0000 mgr.gibba032.whopeo (mgr.24625) 3802 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:04:30.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.785156+0000 mon.a (mon.0) 1635 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:04:30.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.785438+0000 mgr.gibba032.whopeo (mgr.24625) 3803 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:04:30.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.787334+0000 mgr.gibba032.whopeo (mgr.24625) 3804 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:04:30.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.788346+0000 mon.a (mon.0) 1636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:04:30.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.788719+0000 mgr.gibba032.whopeo (mgr.24625) 3805 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:04:30.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.788762+0000 mgr.gibba032.whopeo (mgr.24625) 3806 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:04:30.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.789866+0000 mon.a (mon.0) 1637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.790231+0000 mgr.gibba032.whopeo (mgr.24625) 3807 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:04:30.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.790268+0000 mgr.gibba032.whopeo (mgr.24625) 3808 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:04:30.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.791382+0000 mon.a (mon.0) 1638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.791538+0000 mgr.gibba032.whopeo (mgr.24625) 3809 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:04:30.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.792851+0000 mon.a (mon.0) 1639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.793033+0000 mgr.gibba032.whopeo (mgr.24625) 3810 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:04:30.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.793069+0000 mgr.gibba032.whopeo (mgr.24625) 3811 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:04:30.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.794447+0000 mgr.gibba032.whopeo (mgr.24625) 3812 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:04:30.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.794775+0000 mgr.gibba032.whopeo (mgr.24625) 3813 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:04:30.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.795681+0000 mon.a (mon.0) 1640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:30.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.796240+0000 mgr.gibba032.whopeo (mgr.24625) 3814 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:04:30.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.796279+0000 mgr.gibba032.whopeo (mgr.24625) 3815 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:04:30.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: cephadm 2021-03-10T15:04:28.796386+0000 mgr.gibba032.whopeo (mgr.24625) 3816 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:30.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:29 gibba032 bash[24004]: audit 2021-03-10T15:04:28.797362+0000 mon.a (mon.0) 1641 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:30 gibba011 bash[30791]: cluster 2021-03-10T15:04:29.541904+0000 mgr.gibba032.whopeo (mgr.24625) 3817 : cluster [DBG] pgmap v2160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:31.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:30 gibba011 bash[31774]: cluster 2021-03-10T15:04:29.541904+0000 mgr.gibba032.whopeo (mgr.24625) 3817 : cluster [DBG] pgmap v2160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:30 gibba032 bash[24004]: cluster 2021-03-10T15:04:29.541904+0000 mgr.gibba032.whopeo (mgr.24625) 3817 : cluster [DBG] pgmap v2160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:33.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:32 gibba011 bash[30791]: cluster 2021-03-10T15:04:31.543025+0000 mgr.gibba032.whopeo (mgr.24625) 3818 : cluster [DBG] pgmap v2161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:33.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:32 gibba011 bash[31774]: cluster 2021-03-10T15:04:31.543025+0000 mgr.gibba032.whopeo (mgr.24625) 3818 : cluster [DBG] pgmap v2161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:32 gibba032 bash[24004]: cluster 2021-03-10T15:04:31.543025+0000 mgr.gibba032.whopeo (mgr.24625) 3818 : cluster [DBG] pgmap v2161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:34.710 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:04:34.941 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:04:34.941 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (71m) 6m ago 76m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:04:34.942 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (76m) 5m ago 76m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:04:34.942 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (71m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:04:34.942 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (71m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:04:34.942 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (70m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:04:34.943 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (70m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:04:34.943 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (70m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:04:34.943 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (70m) 6m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:04:34.943 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (76m) 6m ago 76m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:04:34.944 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (76m) 5m ago 76m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:04:34.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (69m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:04:34.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (69m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:04:34.945 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (68m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:04:34.945 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (68m) 6m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:04:34.945 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (67m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:04:34.945 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (67m) 5m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:04:34.946 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (66m) 5m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:04:34.946 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (66m) 5m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:04:34.946 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (76m) 5m ago 76m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:04:34.946 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 74m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:04:35.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:34 gibba011 bash[30791]: cluster 2021-03-10T15:04:33.544320+0000 mgr.gibba032.whopeo (mgr.24625) 3819 : cluster [DBG] pgmap v2162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:35.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:34 gibba011 bash[31774]: cluster 2021-03-10T15:04:33.544320+0000 mgr.gibba032.whopeo (mgr.24625) 3819 : cluster [DBG] pgmap v2162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:35.187 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:04:35.187 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:04:35.187 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:04:35.188 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:35.188 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:04:35.188 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:04:35.188 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:35.189 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:04:35.189 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:04:35.189 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:04:35.189 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:04:35.190 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:04:35.190 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:04:35.190 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:04:35.190 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:04:35.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:34 gibba032 bash[24004]: cluster 2021-03-10T15:04:33.544320+0000 mgr.gibba032.whopeo (mgr.24625) 3819 : cluster [DBG] pgmap v2162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:36.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:35 gibba011 bash[30791]: audit 2021-03-10T15:04:34.696534+0000 mgr.gibba032.whopeo (mgr.24625) 3820 : audit [DBG] from='client.46611 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:36.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:35 gibba011 bash[30791]: audit 2021-03-10T15:04:35.184867+0000 mon.a (mon.0) 1642 : audit [DBG] from='client.? 172.21.2.111:0/863745344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:36.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:35 gibba011 bash[31774]: audit 2021-03-10T15:04:34.696534+0000 mgr.gibba032.whopeo (mgr.24625) 3820 : audit [DBG] from='client.46611 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:36.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:35 gibba011 bash[31774]: audit 2021-03-10T15:04:35.184867+0000 mon.a (mon.0) 1642 : audit [DBG] from='client.? 172.21.2.111:0/863745344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:35 gibba032 bash[24004]: audit 2021-03-10T15:04:34.696534+0000 mgr.gibba032.whopeo (mgr.24625) 3820 : audit [DBG] from='client.46611 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:36.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:35 gibba032 bash[24004]: audit 2021-03-10T15:04:35.184867+0000 mon.a (mon.0) 1642 : audit [DBG] from='client.? 172.21.2.111:0/863745344' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:04:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:36 gibba032 bash[24004]: audit 2021-03-10T15:04:34.925256+0000 mgr.gibba032.whopeo (mgr.24625) 3821 : audit [DBG] from='client.56602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:37.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:36 gibba032 bash[24004]: cluster 2021-03-10T15:04:35.545677+0000 mgr.gibba032.whopeo (mgr.24625) 3822 : cluster [DBG] pgmap v2163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:36 gibba011 bash[30791]: audit 2021-03-10T15:04:34.925256+0000 mgr.gibba032.whopeo (mgr.24625) 3821 : audit [DBG] from='client.56602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:37.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:36 gibba011 bash[30791]: cluster 2021-03-10T15:04:35.545677+0000 mgr.gibba032.whopeo (mgr.24625) 3822 : cluster [DBG] pgmap v2163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:36 gibba011 bash[31774]: audit 2021-03-10T15:04:34.925256+0000 mgr.gibba032.whopeo (mgr.24625) 3821 : audit [DBG] from='client.56602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:04:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:36 gibba011 bash[31774]: cluster 2021-03-10T15:04:35.545677+0000 mgr.gibba032.whopeo (mgr.24625) 3822 : cluster [DBG] pgmap v2163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:37 gibba032 bash[24004]: cluster 2021-03-10T15:04:37.546399+0000 mgr.gibba032.whopeo (mgr.24625) 3823 : cluster [DBG] pgmap v2164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:37 gibba011 bash[30791]: cluster 2021-03-10T15:04:37.546399+0000 mgr.gibba032.whopeo (mgr.24625) 3823 : cluster [DBG] pgmap v2164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:37 gibba011 bash[31774]: cluster 2021-03-10T15:04:37.546399+0000 mgr.gibba032.whopeo (mgr.24625) 3823 : cluster [DBG] pgmap v2164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:40 gibba032 bash[24004]: cluster 2021-03-10T15:04:39.547830+0000 mgr.gibba032.whopeo (mgr.24625) 3824 : cluster [DBG] pgmap v2165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:40 gibba011 bash[30791]: cluster 2021-03-10T15:04:39.547830+0000 mgr.gibba032.whopeo (mgr.24625) 3824 : cluster [DBG] pgmap v2165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:40 gibba011 bash[31774]: cluster 2021-03-10T15:04:39.547830+0000 mgr.gibba032.whopeo (mgr.24625) 3824 : cluster [DBG] pgmap v2165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:42 gibba032 bash[24004]: cluster 2021-03-10T15:04:41.549620+0000 mgr.gibba032.whopeo (mgr.24625) 3825 : cluster [DBG] pgmap v2166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:42 gibba011 bash[30791]: cluster 2021-03-10T15:04:41.549620+0000 mgr.gibba032.whopeo (mgr.24625) 3825 : cluster [DBG] pgmap v2166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:42 gibba011 bash[31774]: cluster 2021-03-10T15:04:41.549620+0000 mgr.gibba032.whopeo (mgr.24625) 3825 : cluster [DBG] pgmap v2166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:44 gibba032 bash[24004]: cluster 2021-03-10T15:04:43.550547+0000 mgr.gibba032.whopeo (mgr.24625) 3826 : cluster [DBG] pgmap v2167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:44 gibba032 bash[24004]: cephadm 2021-03-10T15:04:43.812220+0000 mgr.gibba032.whopeo (mgr.24625) 3827 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:45.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:44 gibba011 bash[30791]: cluster 2021-03-10T15:04:43.550547+0000 mgr.gibba032.whopeo (mgr.24625) 3826 : cluster [DBG] pgmap v2167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:45.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:44 gibba011 bash[30791]: cephadm 2021-03-10T15:04:43.812220+0000 mgr.gibba032.whopeo (mgr.24625) 3827 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:44 gibba011 bash[31774]: cluster 2021-03-10T15:04:43.550547+0000 mgr.gibba032.whopeo (mgr.24625) 3826 : cluster [DBG] pgmap v2167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:44 gibba011 bash[31774]: cephadm 2021-03-10T15:04:43.812220+0000 mgr.gibba032.whopeo (mgr.24625) 3827 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:46 gibba032 bash[24004]: cluster 2021-03-10T15:04:45.552081+0000 mgr.gibba032.whopeo (mgr.24625) 3828 : cluster [DBG] pgmap v2168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:47.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:46 gibba011 bash[30791]: cluster 2021-03-10T15:04:45.552081+0000 mgr.gibba032.whopeo (mgr.24625) 3828 : cluster [DBG] pgmap v2168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:47.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:46 gibba011 bash[31774]: cluster 2021-03-10T15:04:45.552081+0000 mgr.gibba032.whopeo (mgr.24625) 3828 : cluster [DBG] pgmap v2168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:47 gibba032 bash[24004]: cluster 2021-03-10T15:04:47.553656+0000 mgr.gibba032.whopeo (mgr.24625) 3829 : cluster [DBG] pgmap v2169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:47 gibba011 bash[30791]: cluster 2021-03-10T15:04:47.553656+0000 mgr.gibba032.whopeo (mgr.24625) 3829 : cluster [DBG] pgmap v2169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:47 gibba011 bash[31774]: cluster 2021-03-10T15:04:47.553656+0000 mgr.gibba032.whopeo (mgr.24625) 3829 : cluster [DBG] pgmap v2169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:50 gibba032 bash[24004]: cluster 2021-03-10T15:04:49.554685+0000 mgr.gibba032.whopeo (mgr.24625) 3830 : cluster [DBG] pgmap v2170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:50 gibba011 bash[30791]: cluster 2021-03-10T15:04:49.554685+0000 mgr.gibba032.whopeo (mgr.24625) 3830 : cluster [DBG] pgmap v2170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:50 gibba011 bash[31774]: cluster 2021-03-10T15:04:49.554685+0000 mgr.gibba032.whopeo (mgr.24625) 3830 : cluster [DBG] pgmap v2170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:52 gibba032 bash[24004]: cluster 2021-03-10T15:04:51.556400+0000 mgr.gibba032.whopeo (mgr.24625) 3831 : cluster [DBG] pgmap v2171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:52 gibba011 bash[30791]: cluster 2021-03-10T15:04:51.556400+0000 mgr.gibba032.whopeo (mgr.24625) 3831 : cluster [DBG] pgmap v2171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:53.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:52 gibba011 bash[31774]: cluster 2021-03-10T15:04:51.556400+0000 mgr.gibba032.whopeo (mgr.24625) 3831 : cluster [DBG] pgmap v2171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:54 gibba032 bash[24004]: cluster 2021-03-10T15:04:53.557749+0000 mgr.gibba032.whopeo (mgr.24625) 3832 : cluster [DBG] pgmap v2172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:54 gibba011 bash[30791]: cluster 2021-03-10T15:04:53.557749+0000 mgr.gibba032.whopeo (mgr.24625) 3832 : cluster [DBG] pgmap v2172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:54 gibba011 bash[31774]: cluster 2021-03-10T15:04:53.557749+0000 mgr.gibba032.whopeo (mgr.24625) 3832 : cluster [DBG] pgmap v2172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: cluster 2021-03-10T15:04:55.558721+0000 mgr.gibba032.whopeo (mgr.24625) 3833 : cluster [DBG] pgmap v2173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.471040+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:04:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.471792+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:04:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.487006+0000 mon.a (mon.0) 1645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:04:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.487858+0000 mon.a (mon.0) 1646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:04:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.488538+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:04:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:56 gibba032 bash[24004]: audit 2021-03-10T15:04:56.489208+0000 mon.a (mon.0) 1648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:04:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: cluster 2021-03-10T15:04:55.558721+0000 mgr.gibba032.whopeo (mgr.24625) 3833 : cluster [DBG] pgmap v2173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.471040+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:04:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.471792+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:04:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.487006+0000 mon.a (mon.0) 1645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:04:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.487858+0000 mon.a (mon.0) 1646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:04:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.488538+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:04:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[30791]: audit 2021-03-10T15:04:56.489208+0000 mon.a (mon.0) 1648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:04:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: cluster 2021-03-10T15:04:55.558721+0000 mgr.gibba032.whopeo (mgr.24625) 3833 : cluster [DBG] pgmap v2173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.471040+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:04:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.471792+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:04:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.487006+0000 mon.a (mon.0) 1645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:04:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.487858+0000 mon.a (mon.0) 1646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:04:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.488538+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:04:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:56 gibba011 bash[31774]: audit 2021-03-10T15:04:56.489208+0000 mon.a (mon.0) 1648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:04:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:57 gibba032 bash[24004]: cluster 2021-03-10T15:04:57.560418+0000 mgr.gibba032.whopeo (mgr.24625) 3834 : cluster [DBG] pgmap v2174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:57 gibba011 bash[30791]: cluster 2021-03-10T15:04:57.560418+0000 mgr.gibba032.whopeo (mgr.24625) 3834 : cluster [DBG] pgmap v2174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:57 gibba011 bash[31774]: cluster 2021-03-10T15:04:57.560418+0000 mgr.gibba032.whopeo (mgr.24625) 3834 : cluster [DBG] pgmap v2174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:04:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:04:59 gibba032 bash[24004]: cephadm 2021-03-10T15:04:58.814502+0000 mgr.gibba032.whopeo (mgr.24625) 3835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:59.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:04:59 gibba011 bash[30791]: cephadm 2021-03-10T15:04:58.814502+0000 mgr.gibba032.whopeo (mgr.24625) 3835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:04:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:04:59 gibba011 bash[31774]: cephadm 2021-03-10T15:04:58.814502+0000 mgr.gibba032.whopeo (mgr.24625) 3835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:00 gibba011 bash[30791]: cluster 2021-03-10T15:04:59.561885+0000 mgr.gibba032.whopeo (mgr.24625) 3836 : cluster [DBG] pgmap v2175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:00 gibba011 bash[31774]: cluster 2021-03-10T15:04:59.561885+0000 mgr.gibba032.whopeo (mgr.24625) 3836 : cluster [DBG] pgmap v2175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:00 gibba032 bash[24004]: cluster 2021-03-10T15:04:59.561885+0000 mgr.gibba032.whopeo (mgr.24625) 3836 : cluster [DBG] pgmap v2175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:02 gibba032 bash[24004]: cluster 2021-03-10T15:05:01.563593+0000 mgr.gibba032.whopeo (mgr.24625) 3837 : cluster [DBG] pgmap v2176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:02 gibba011 bash[30791]: cluster 2021-03-10T15:05:01.563593+0000 mgr.gibba032.whopeo (mgr.24625) 3837 : cluster [DBG] pgmap v2176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:03.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:02 gibba011 bash[31774]: cluster 2021-03-10T15:05:01.563593+0000 mgr.gibba032.whopeo (mgr.24625) 3837 : cluster [DBG] pgmap v2176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:04 gibba032 bash[24004]: cluster 2021-03-10T15:05:03.565155+0000 mgr.gibba032.whopeo (mgr.24625) 3838 : cluster [DBG] pgmap v2177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:05.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:04 gibba011 bash[30791]: cluster 2021-03-10T15:05:03.565155+0000 mgr.gibba032.whopeo (mgr.24625) 3838 : cluster [DBG] pgmap v2177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:04 gibba011 bash[31774]: cluster 2021-03-10T15:05:03.565155+0000 mgr.gibba032.whopeo (mgr.24625) 3838 : cluster [DBG] pgmap v2177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:05.430 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:05:05.657 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:05:05.657 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (71m) 6m ago 77m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:05:05.657 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (76m) 5m ago 76m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:05:05.657 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (72m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:05:05.658 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (72m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:05:05.658 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (71m) 6m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:05:05.658 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (71m) 6m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:05:05.658 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (70m) 5m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:05:05.659 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (70m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:05:05.659 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (77m) 6m ago 77m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:05:05.659 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (77m) 5m ago 77m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:05:05.659 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (70m) 6m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:05:05.660 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (69m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:05:05.660 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (69m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:05:05.660 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (68m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:05:05.660 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (68m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:05:05.660 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (67m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:05:05.661 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (67m) 5m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:05:05.661 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (67m) 5m ago 77m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:05:05.661 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (76m) 5m ago 77m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:05:05.661 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 75m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:05:05.905 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:05:05.906 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:05:05.906 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:05:05.906 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:05.906 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:05:05.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:05:05.907 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:05.907 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:05:05.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:05:05.907 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:05.908 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:05:05.908 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:05:05.908 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:05:05.908 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:05:05.909 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:05:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:06 gibba032 bash[24004]: audit 2021-03-10T15:05:05.416504+0000 mgr.gibba032.whopeo (mgr.24625) 3839 : audit [DBG] from='client.46629 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:06 gibba032 bash[24004]: cluster 2021-03-10T15:05:05.566725+0000 mgr.gibba032.whopeo (mgr.24625) 3840 : cluster [DBG] pgmap v2178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:07.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:06 gibba032 bash[24004]: audit 2021-03-10T15:05:05.641427+0000 mgr.gibba032.whopeo (mgr.24625) 3841 : audit [DBG] from='client.56620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:06 gibba032 bash[24004]: audit 2021-03-10T15:05:05.902972+0000 mon.a (mon.0) 1649 : audit [DBG] from='client.? 172.21.2.111:0/2887696631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:07.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[30791]: audit 2021-03-10T15:05:05.416504+0000 mgr.gibba032.whopeo (mgr.24625) 3839 : audit [DBG] from='client.46629 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[30791]: cluster 2021-03-10T15:05:05.566725+0000 mgr.gibba032.whopeo (mgr.24625) 3840 : cluster [DBG] pgmap v2178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:07.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[30791]: audit 2021-03-10T15:05:05.641427+0000 mgr.gibba032.whopeo (mgr.24625) 3841 : audit [DBG] from='client.56620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[30791]: audit 2021-03-10T15:05:05.902972+0000 mon.a (mon.0) 1649 : audit [DBG] from='client.? 172.21.2.111:0/2887696631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[31774]: audit 2021-03-10T15:05:05.416504+0000 mgr.gibba032.whopeo (mgr.24625) 3839 : audit [DBG] from='client.46629 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[31774]: cluster 2021-03-10T15:05:05.566725+0000 mgr.gibba032.whopeo (mgr.24625) 3840 : cluster [DBG] pgmap v2178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[31774]: audit 2021-03-10T15:05:05.641427+0000 mgr.gibba032.whopeo (mgr.24625) 3841 : audit [DBG] from='client.56620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:06 gibba011 bash[31774]: audit 2021-03-10T15:05:05.902972+0000 mon.a (mon.0) 1649 : audit [DBG] from='client.? 172.21.2.111:0/2887696631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:08 gibba032 bash[24004]: cluster 2021-03-10T15:05:07.568262+0000 mgr.gibba032.whopeo (mgr.24625) 3842 : cluster [DBG] pgmap v2179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:08 gibba011 bash[30791]: cluster 2021-03-10T15:05:07.568262+0000 mgr.gibba032.whopeo (mgr.24625) 3842 : cluster [DBG] pgmap v2179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:08 gibba011 bash[31774]: cluster 2021-03-10T15:05:07.568262+0000 mgr.gibba032.whopeo (mgr.24625) 3842 : cluster [DBG] pgmap v2179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:10 gibba032 bash[24004]: cluster 2021-03-10T15:05:09.569701+0000 mgr.gibba032.whopeo (mgr.24625) 3843 : cluster [DBG] pgmap v2180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:11.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:10 gibba011 bash[30791]: cluster 2021-03-10T15:05:09.569701+0000 mgr.gibba032.whopeo (mgr.24625) 3843 : cluster [DBG] pgmap v2180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:11.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:10 gibba011 bash[31774]: cluster 2021-03-10T15:05:09.569701+0000 mgr.gibba032.whopeo (mgr.24625) 3843 : cluster [DBG] pgmap v2180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:12 gibba032 bash[24004]: cluster 2021-03-10T15:05:11.571345+0000 mgr.gibba032.whopeo (mgr.24625) 3844 : cluster [DBG] pgmap v2181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:12 gibba011 bash[30791]: cluster 2021-03-10T15:05:11.571345+0000 mgr.gibba032.whopeo (mgr.24625) 3844 : cluster [DBG] pgmap v2181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:12 gibba011 bash[31774]: cluster 2021-03-10T15:05:11.571345+0000 mgr.gibba032.whopeo (mgr.24625) 3844 : cluster [DBG] pgmap v2181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:14 gibba032 bash[24004]: cluster 2021-03-10T15:05:13.572780+0000 mgr.gibba032.whopeo (mgr.24625) 3845 : cluster [DBG] pgmap v2182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:15.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:14 gibba032 bash[24004]: cephadm 2021-03-10T15:05:13.830521+0000 mgr.gibba032.whopeo (mgr.24625) 3846 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:14 gibba011 bash[30791]: cluster 2021-03-10T15:05:13.572780+0000 mgr.gibba032.whopeo (mgr.24625) 3845 : cluster [DBG] pgmap v2182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:15.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:14 gibba011 bash[30791]: cephadm 2021-03-10T15:05:13.830521+0000 mgr.gibba032.whopeo (mgr.24625) 3846 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:14 gibba011 bash[31774]: cluster 2021-03-10T15:05:13.572780+0000 mgr.gibba032.whopeo (mgr.24625) 3845 : cluster [DBG] pgmap v2182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:14 gibba011 bash[31774]: cephadm 2021-03-10T15:05:13.830521+0000 mgr.gibba032.whopeo (mgr.24625) 3846 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:16 gibba032 bash[24004]: cluster 2021-03-10T15:05:15.574320+0000 mgr.gibba032.whopeo (mgr.24625) 3847 : cluster [DBG] pgmap v2183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:16 gibba011 bash[30791]: cluster 2021-03-10T15:05:15.574320+0000 mgr.gibba032.whopeo (mgr.24625) 3847 : cluster [DBG] pgmap v2183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:16 gibba011 bash[31774]: cluster 2021-03-10T15:05:15.574320+0000 mgr.gibba032.whopeo (mgr.24625) 3847 : cluster [DBG] pgmap v2183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:18 gibba011 bash[30791]: cluster 2021-03-10T15:05:17.576093+0000 mgr.gibba032.whopeo (mgr.24625) 3848 : cluster [DBG] pgmap v2184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:18 gibba011 bash[31774]: cluster 2021-03-10T15:05:17.576093+0000 mgr.gibba032.whopeo (mgr.24625) 3848 : cluster [DBG] pgmap v2184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:18 gibba032 bash[24004]: cluster 2021-03-10T15:05:17.576093+0000 mgr.gibba032.whopeo (mgr.24625) 3848 : cluster [DBG] pgmap v2184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:20 gibba032 bash[24004]: cluster 2021-03-10T15:05:19.577301+0000 mgr.gibba032.whopeo (mgr.24625) 3849 : cluster [DBG] pgmap v2185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:21.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:20 gibba011 bash[30791]: cluster 2021-03-10T15:05:19.577301+0000 mgr.gibba032.whopeo (mgr.24625) 3849 : cluster [DBG] pgmap v2185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:21.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:20 gibba011 bash[31774]: cluster 2021-03-10T15:05:19.577301+0000 mgr.gibba032.whopeo (mgr.24625) 3849 : cluster [DBG] pgmap v2185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:22 gibba032 bash[24004]: cluster 2021-03-10T15:05:21.579088+0000 mgr.gibba032.whopeo (mgr.24625) 3850 : cluster [DBG] pgmap v2186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:22 gibba011 bash[30791]: cluster 2021-03-10T15:05:21.579088+0000 mgr.gibba032.whopeo (mgr.24625) 3850 : cluster [DBG] pgmap v2186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:23.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:22 gibba011 bash[31774]: cluster 2021-03-10T15:05:21.579088+0000 mgr.gibba032.whopeo (mgr.24625) 3850 : cluster [DBG] pgmap v2186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:24 gibba032 bash[24004]: cluster 2021-03-10T15:05:23.580587+0000 mgr.gibba032.whopeo (mgr.24625) 3851 : cluster [DBG] pgmap v2187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:24 gibba011 bash[30791]: cluster 2021-03-10T15:05:23.580587+0000 mgr.gibba032.whopeo (mgr.24625) 3851 : cluster [DBG] pgmap v2187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:25.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:24 gibba011 bash[31774]: cluster 2021-03-10T15:05:23.580587+0000 mgr.gibba032.whopeo (mgr.24625) 3851 : cluster [DBG] pgmap v2187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:26 gibba032 bash[24004]: cluster 2021-03-10T15:05:25.581901+0000 mgr.gibba032.whopeo (mgr.24625) 3852 : cluster [DBG] pgmap v2188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:26 gibba011 bash[30791]: cluster 2021-03-10T15:05:25.581901+0000 mgr.gibba032.whopeo (mgr.24625) 3852 : cluster [DBG] pgmap v2188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:27.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:26 gibba011 bash[31774]: cluster 2021-03-10T15:05:25.581901+0000 mgr.gibba032.whopeo (mgr.24625) 3852 : cluster [DBG] pgmap v2188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:28 gibba011 bash[30791]: cluster 2021-03-10T15:05:27.583524+0000 mgr.gibba032.whopeo (mgr.24625) 3853 : cluster [DBG] pgmap v2189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:28 gibba011 bash[31774]: cluster 2021-03-10T15:05:27.583524+0000 mgr.gibba032.whopeo (mgr.24625) 3853 : cluster [DBG] pgmap v2189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:28 gibba032 bash[24004]: cluster 2021-03-10T15:05:27.583524+0000 mgr.gibba032.whopeo (mgr.24625) 3853 : cluster [DBG] pgmap v2189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.044430+0000 mgr.gibba032.whopeo (mgr.24625) 3854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:05:30.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.046089+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.047612+0000 mgr.gibba032.whopeo (mgr.24625) 3855 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:05:30.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.048866+0000 mon.a (mon.0) 1651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.258439+0000 mon.a (mon.0) 1652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.258706+0000 mgr.gibba032.whopeo (mgr.24625) 3856 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:05:30.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.260429+0000 mgr.gibba032.whopeo (mgr.24625) 3857 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:05:30.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.261585+0000 mgr.gibba032.whopeo (mgr.24625) 3858 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:05:30.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.261625+0000 mgr.gibba032.whopeo (mgr.24625) 3859 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:05:30.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.261663+0000 mon.a (mon.0) 1653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:05:30.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.262998+0000 mon.a (mon.0) 1654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.263131+0000 mgr.gibba032.whopeo (mgr.24625) 3860 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:05:30.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.263168+0000 mgr.gibba032.whopeo (mgr.24625) 3861 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:05:30.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.264324+0000 mgr.gibba032.whopeo (mgr.24625) 3862 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:05:30.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.264367+0000 mon.a (mon.0) 1655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.265839+0000 mon.a (mon.0) 1656 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.267309+0000 mgr.gibba032.whopeo (mgr.24625) 3863 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:05:30.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.267353+0000 mgr.gibba032.whopeo (mgr.24625) 3864 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:05:30.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.270338+0000 mgr.gibba032.whopeo (mgr.24625) 3865 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:05:30.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.270453+0000 mon.a (mon.0) 1657 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.270839+0000 mgr.gibba032.whopeo (mgr.24625) 3866 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:05:30.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.044430+0000 mgr.gibba032.whopeo (mgr.24625) 3854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:05:30.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.046089+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.047612+0000 mgr.gibba032.whopeo (mgr.24625) 3855 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:05:30.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.048866+0000 mon.a (mon.0) 1651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.258439+0000 mon.a (mon.0) 1652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.258706+0000 mgr.gibba032.whopeo (mgr.24625) 3856 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:05:30.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.260429+0000 mgr.gibba032.whopeo (mgr.24625) 3857 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:05:30.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.261585+0000 mgr.gibba032.whopeo (mgr.24625) 3858 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:05:30.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.261625+0000 mgr.gibba032.whopeo (mgr.24625) 3859 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:05:30.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.261663+0000 mon.a (mon.0) 1653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:05:30.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.262998+0000 mon.a (mon.0) 1654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.263131+0000 mgr.gibba032.whopeo (mgr.24625) 3860 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:05:30.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.263168+0000 mgr.gibba032.whopeo (mgr.24625) 3861 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:05:30.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.264324+0000 mgr.gibba032.whopeo (mgr.24625) 3862 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:05:30.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.264367+0000 mon.a (mon.0) 1655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.265839+0000 mon.a (mon.0) 1656 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.267309+0000 mgr.gibba032.whopeo (mgr.24625) 3863 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:05:30.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.267353+0000 mgr.gibba032.whopeo (mgr.24625) 3864 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:05:30.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.270338+0000 mgr.gibba032.whopeo (mgr.24625) 3865 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:05:30.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.270453+0000 mon.a (mon.0) 1657 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.270839+0000 mgr.gibba032.whopeo (mgr.24625) 3866 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:05:30.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: audit 2021-03-10T15:05:29.273981+0000 mon.a (mon.0) 1658 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.274186+0000 mgr.gibba032.whopeo (mgr.24625) 3867 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:05:30.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.274233+0000 mgr.gibba032.whopeo (mgr.24625) 3868 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:05:30.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cephadm 2021-03-10T15:05:29.274374+0000 mgr.gibba032.whopeo (mgr.24625) 3869 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:30.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[30791]: cluster 2021-03-10T15:05:29.584914+0000 mgr.gibba032.whopeo (mgr.24625) 3870 : cluster [DBG] pgmap v2190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:30.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: audit 2021-03-10T15:05:29.273981+0000 mon.a (mon.0) 1658 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.274186+0000 mgr.gibba032.whopeo (mgr.24625) 3867 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:05:30.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.274233+0000 mgr.gibba032.whopeo (mgr.24625) 3868 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:05:30.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cephadm 2021-03-10T15:05:29.274374+0000 mgr.gibba032.whopeo (mgr.24625) 3869 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:30.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:30 gibba011 bash[31774]: cluster 2021-03-10T15:05:29.584914+0000 mgr.gibba032.whopeo (mgr.24625) 3870 : cluster [DBG] pgmap v2190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:30.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.044430+0000 mgr.gibba032.whopeo (mgr.24625) 3854 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:05:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.046089+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.047612+0000 mgr.gibba032.whopeo (mgr.24625) 3855 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:05:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.048866+0000 mon.a (mon.0) 1651 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.258439+0000 mon.a (mon.0) 1652 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:05:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.258706+0000 mgr.gibba032.whopeo (mgr.24625) 3856 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:05:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.260429+0000 mgr.gibba032.whopeo (mgr.24625) 3857 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:05:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.261585+0000 mgr.gibba032.whopeo (mgr.24625) 3858 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:05:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.261625+0000 mgr.gibba032.whopeo (mgr.24625) 3859 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:05:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.261663+0000 mon.a (mon.0) 1653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:05:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.262998+0000 mon.a (mon.0) 1654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.263131+0000 mgr.gibba032.whopeo (mgr.24625) 3860 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:05:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.263168+0000 mgr.gibba032.whopeo (mgr.24625) 3861 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:05:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.264324+0000 mgr.gibba032.whopeo (mgr.24625) 3862 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:05:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.264367+0000 mon.a (mon.0) 1655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.265839+0000 mon.a (mon.0) 1656 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.267309+0000 mgr.gibba032.whopeo (mgr.24625) 3863 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:05:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.267353+0000 mgr.gibba032.whopeo (mgr.24625) 3864 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:05:30.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.270338+0000 mgr.gibba032.whopeo (mgr.24625) 3865 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:05:30.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.270453+0000 mon.a (mon.0) 1657 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.270839+0000 mgr.gibba032.whopeo (mgr.24625) 3866 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:05:30.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: audit 2021-03-10T15:05:29.273981+0000 mon.a (mon.0) 1658 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:30.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.274186+0000 mgr.gibba032.whopeo (mgr.24625) 3867 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:05:30.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.274233+0000 mgr.gibba032.whopeo (mgr.24625) 3868 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:05:30.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cephadm 2021-03-10T15:05:29.274374+0000 mgr.gibba032.whopeo (mgr.24625) 3869 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:30.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:30 gibba032 bash[24004]: cluster 2021-03-10T15:05:29.584914+0000 mgr.gibba032.whopeo (mgr.24625) 3870 : cluster [DBG] pgmap v2190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:32 gibba032 bash[24004]: cluster 2021-03-10T15:05:31.586658+0000 mgr.gibba032.whopeo (mgr.24625) 3871 : cluster [DBG] pgmap v2191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:33.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:32 gibba011 bash[30791]: cluster 2021-03-10T15:05:31.586658+0000 mgr.gibba032.whopeo (mgr.24625) 3871 : cluster [DBG] pgmap v2191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:33.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:32 gibba011 bash[31774]: cluster 2021-03-10T15:05:31.586658+0000 mgr.gibba032.whopeo (mgr.24625) 3871 : cluster [DBG] pgmap v2191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:34 gibba032 bash[24004]: cluster 2021-03-10T15:05:33.587989+0000 mgr.gibba032.whopeo (mgr.24625) 3872 : cluster [DBG] pgmap v2192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:35.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:34 gibba011 bash[30791]: cluster 2021-03-10T15:05:33.587989+0000 mgr.gibba032.whopeo (mgr.24625) 3872 : cluster [DBG] pgmap v2192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:35.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:34 gibba011 bash[31774]: cluster 2021-03-10T15:05:33.587989+0000 mgr.gibba032.whopeo (mgr.24625) 3872 : cluster [DBG] pgmap v2192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:36.150 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:05:36.378 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:05:36.378 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (72m) 7m ago 77m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:05:36.379 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (77m) 6m ago 77m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:05:36.379 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (72m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:05:36.379 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (72m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:05:36.379 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (71m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:05:36.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (71m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:05:36.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (71m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:05:36.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (71m) 7m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:05:36.380 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (77m) 7m ago 77m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:05:36.381 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (77m) 6m ago 77m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:05:36.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (70m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:05:36.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (70m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:05:36.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (69m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:05:36.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (69m) 7m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:05:36.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (68m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:05:36.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (68m) 6m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:05:36.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (67m) 6m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:05:36.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (67m) 6m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:05:36.383 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (77m) 6m ago 77m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:05:36.383 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 75m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:05:36.629 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:05:36.629 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:05:36.630 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:05:36.630 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:36.630 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:05:36.630 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:05:36.631 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:36.631 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:05:36.631 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:05:36.631 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:05:36.632 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:05:36.632 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:05:36.632 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:05:36.632 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:05:36.633 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:05:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:36 gibba032 bash[24004]: cluster 2021-03-10T15:05:35.589379+0000 mgr.gibba032.whopeo (mgr.24625) 3873 : cluster [DBG] pgmap v2193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:36 gibba032 bash[24004]: audit 2021-03-10T15:05:36.626724+0000 mon.a (mon.0) 1659 : audit [DBG] from='client.? 172.21.2.111:0/3966150373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:36 gibba011 bash[30791]: cluster 2021-03-10T15:05:35.589379+0000 mgr.gibba032.whopeo (mgr.24625) 3873 : cluster [DBG] pgmap v2193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:37.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:36 gibba011 bash[30791]: audit 2021-03-10T15:05:36.626724+0000 mon.a (mon.0) 1659 : audit [DBG] from='client.? 172.21.2.111:0/3966150373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:36 gibba011 bash[31774]: cluster 2021-03-10T15:05:35.589379+0000 mgr.gibba032.whopeo (mgr.24625) 3873 : cluster [DBG] pgmap v2193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:36 gibba011 bash[31774]: audit 2021-03-10T15:05:36.626724+0000 mon.a (mon.0) 1659 : audit [DBG] from='client.? 172.21.2.111:0/3966150373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:05:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:37 gibba032 bash[24004]: audit 2021-03-10T15:05:36.136433+0000 mgr.gibba032.whopeo (mgr.24625) 3874 : audit [DBG] from='client.56632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:38.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:37 gibba032 bash[24004]: audit 2021-03-10T15:05:36.361055+0000 mgr.gibba032.whopeo (mgr.24625) 3875 : audit [DBG] from='client.46653 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:37 gibba011 bash[30791]: audit 2021-03-10T15:05:36.136433+0000 mgr.gibba032.whopeo (mgr.24625) 3874 : audit [DBG] from='client.56632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:37 gibba011 bash[30791]: audit 2021-03-10T15:05:36.361055+0000 mgr.gibba032.whopeo (mgr.24625) 3875 : audit [DBG] from='client.46653 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:37 gibba011 bash[31774]: audit 2021-03-10T15:05:36.136433+0000 mgr.gibba032.whopeo (mgr.24625) 3874 : audit [DBG] from='client.56632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:37 gibba011 bash[31774]: audit 2021-03-10T15:05:36.361055+0000 mgr.gibba032.whopeo (mgr.24625) 3875 : audit [DBG] from='client.46653 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:05:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:38 gibba032 bash[24004]: cluster 2021-03-10T15:05:37.591100+0000 mgr.gibba032.whopeo (mgr.24625) 3876 : cluster [DBG] pgmap v2194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:38 gibba011 bash[30791]: cluster 2021-03-10T15:05:37.591100+0000 mgr.gibba032.whopeo (mgr.24625) 3876 : cluster [DBG] pgmap v2194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:39.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:38 gibba011 bash[31774]: cluster 2021-03-10T15:05:37.591100+0000 mgr.gibba032.whopeo (mgr.24625) 3876 : cluster [DBG] pgmap v2194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:40 gibba032 bash[24004]: cluster 2021-03-10T15:05:39.592626+0000 mgr.gibba032.whopeo (mgr.24625) 3877 : cluster [DBG] pgmap v2195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:40 gibba011 bash[30791]: cluster 2021-03-10T15:05:39.592626+0000 mgr.gibba032.whopeo (mgr.24625) 3877 : cluster [DBG] pgmap v2195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:40 gibba011 bash[31774]: cluster 2021-03-10T15:05:39.592626+0000 mgr.gibba032.whopeo (mgr.24625) 3877 : cluster [DBG] pgmap v2195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:42 gibba032 bash[24004]: cluster 2021-03-10T15:05:41.594117+0000 mgr.gibba032.whopeo (mgr.24625) 3878 : cluster [DBG] pgmap v2196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:42 gibba011 bash[30791]: cluster 2021-03-10T15:05:41.594117+0000 mgr.gibba032.whopeo (mgr.24625) 3878 : cluster [DBG] pgmap v2196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:42 gibba011 bash[31774]: cluster 2021-03-10T15:05:41.594117+0000 mgr.gibba032.whopeo (mgr.24625) 3878 : cluster [DBG] pgmap v2196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:44 gibba032 bash[24004]: cluster 2021-03-10T15:05:43.595697+0000 mgr.gibba032.whopeo (mgr.24625) 3879 : cluster [DBG] pgmap v2197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:45.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:44 gibba011 bash[30791]: cluster 2021-03-10T15:05:43.595697+0000 mgr.gibba032.whopeo (mgr.24625) 3879 : cluster [DBG] pgmap v2197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:44 gibba011 bash[31774]: cluster 2021-03-10T15:05:43.595697+0000 mgr.gibba032.whopeo (mgr.24625) 3879 : cluster [DBG] pgmap v2197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:45 gibba032 bash[24004]: cephadm 2021-03-10T15:05:44.290437+0000 mgr.gibba032.whopeo (mgr.24625) 3880 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:45 gibba011 bash[30791]: cephadm 2021-03-10T15:05:44.290437+0000 mgr.gibba032.whopeo (mgr.24625) 3880 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:46.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:45 gibba011 bash[31774]: cephadm 2021-03-10T15:05:44.290437+0000 mgr.gibba032.whopeo (mgr.24625) 3880 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:05:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:46 gibba032 bash[24004]: cluster 2021-03-10T15:05:45.597171+0000 mgr.gibba032.whopeo (mgr.24625) 3881 : cluster [DBG] pgmap v2198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:46 gibba011 bash[30791]: cluster 2021-03-10T15:05:45.597171+0000 mgr.gibba032.whopeo (mgr.24625) 3881 : cluster [DBG] pgmap v2198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:46 gibba011 bash[31774]: cluster 2021-03-10T15:05:45.597171+0000 mgr.gibba032.whopeo (mgr.24625) 3881 : cluster [DBG] pgmap v2198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:48 gibba011 bash[30791]: cluster 2021-03-10T15:05:47.598667+0000 mgr.gibba032.whopeo (mgr.24625) 3882 : cluster [DBG] pgmap v2199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:48 gibba011 bash[31774]: cluster 2021-03-10T15:05:47.598667+0000 mgr.gibba032.whopeo (mgr.24625) 3882 : cluster [DBG] pgmap v2199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:48 gibba032 bash[24004]: cluster 2021-03-10T15:05:47.598667+0000 mgr.gibba032.whopeo (mgr.24625) 3882 : cluster [DBG] pgmap v2199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:50 gibba032 bash[24004]: cluster 2021-03-10T15:05:49.600127+0000 mgr.gibba032.whopeo (mgr.24625) 3883 : cluster [DBG] pgmap v2200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:50 gibba011 bash[30791]: cluster 2021-03-10T15:05:49.600127+0000 mgr.gibba032.whopeo (mgr.24625) 3883 : cluster [DBG] pgmap v2200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:50 gibba011 bash[31774]: cluster 2021-03-10T15:05:49.600127+0000 mgr.gibba032.whopeo (mgr.24625) 3883 : cluster [DBG] pgmap v2200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:52 gibba032 bash[24004]: cluster 2021-03-10T15:05:51.601883+0000 mgr.gibba032.whopeo (mgr.24625) 3884 : cluster [DBG] pgmap v2201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:52 gibba011 bash[30791]: cluster 2021-03-10T15:05:51.601883+0000 mgr.gibba032.whopeo (mgr.24625) 3884 : cluster [DBG] pgmap v2201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:53.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:52 gibba011 bash[31774]: cluster 2021-03-10T15:05:51.601883+0000 mgr.gibba032.whopeo (mgr.24625) 3884 : cluster [DBG] pgmap v2201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:54 gibba032 bash[24004]: cluster 2021-03-10T15:05:53.603483+0000 mgr.gibba032.whopeo (mgr.24625) 3885 : cluster [DBG] pgmap v2202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:54 gibba011 bash[30791]: cluster 2021-03-10T15:05:53.603483+0000 mgr.gibba032.whopeo (mgr.24625) 3885 : cluster [DBG] pgmap v2202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:54 gibba011 bash[31774]: cluster 2021-03-10T15:05:53.603483+0000 mgr.gibba032.whopeo (mgr.24625) 3885 : cluster [DBG] pgmap v2202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: cluster 2021-03-10T15:05:55.605108+0000 mgr.gibba032.whopeo (mgr.24625) 3886 : cluster [DBG] pgmap v2203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.479028+0000 mon.a (mon.0) 1660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:05:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.479977+0000 mon.a (mon.0) 1661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:05:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.492702+0000 mon.a (mon.0) 1662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:05:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.493721+0000 mon.a (mon.0) 1663 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:05:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.494790+0000 mon.a (mon.0) 1664 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:05:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:56 gibba032 bash[24004]: audit 2021-03-10T15:05:56.495850+0000 mon.a (mon.0) 1665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:05:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: cluster 2021-03-10T15:05:55.605108+0000 mgr.gibba032.whopeo (mgr.24625) 3886 : cluster [DBG] pgmap v2203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.479028+0000 mon.a (mon.0) 1660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:05:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.479977+0000 mon.a (mon.0) 1661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:05:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.492702+0000 mon.a (mon.0) 1662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:05:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.493721+0000 mon.a (mon.0) 1663 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:05:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.494790+0000 mon.a (mon.0) 1664 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:05:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[30791]: audit 2021-03-10T15:05:56.495850+0000 mon.a (mon.0) 1665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:05:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: cluster 2021-03-10T15:05:55.605108+0000 mgr.gibba032.whopeo (mgr.24625) 3886 : cluster [DBG] pgmap v2203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.479028+0000 mon.a (mon.0) 1660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:05:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.479977+0000 mon.a (mon.0) 1661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:05:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.492702+0000 mon.a (mon.0) 1662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:05:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.493721+0000 mon.a (mon.0) 1663 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:05:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.494790+0000 mon.a (mon.0) 1664 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:05:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:56 gibba011 bash[31774]: audit 2021-03-10T15:05:56.495850+0000 mon.a (mon.0) 1665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:05:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:05:58 gibba011 bash[30791]: cluster 2021-03-10T15:05:57.606837+0000 mgr.gibba032.whopeo (mgr.24625) 3887 : cluster [DBG] pgmap v2204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:05:58 gibba011 bash[31774]: cluster 2021-03-10T15:05:57.606837+0000 mgr.gibba032.whopeo (mgr.24625) 3887 : cluster [DBG] pgmap v2204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:05:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:05:58 gibba032 bash[24004]: cluster 2021-03-10T15:05:57.606837+0000 mgr.gibba032.whopeo (mgr.24625) 3887 : cluster [DBG] pgmap v2204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:01.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:00 gibba032 bash[24004]: cephadm 2021-03-10T15:05:59.306347+0000 mgr.gibba032.whopeo (mgr.24625) 3888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:01.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:00 gibba032 bash[24004]: cluster 2021-03-10T15:05:59.608356+0000 mgr.gibba032.whopeo (mgr.24625) 3889 : cluster [DBG] pgmap v2205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:01.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:00 gibba011 bash[30791]: cephadm 2021-03-10T15:05:59.306347+0000 mgr.gibba032.whopeo (mgr.24625) 3888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:01.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:00 gibba011 bash[30791]: cluster 2021-03-10T15:05:59.608356+0000 mgr.gibba032.whopeo (mgr.24625) 3889 : cluster [DBG] pgmap v2205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:00 gibba011 bash[31774]: cephadm 2021-03-10T15:05:59.306347+0000 mgr.gibba032.whopeo (mgr.24625) 3888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:00 gibba011 bash[31774]: cluster 2021-03-10T15:05:59.608356+0000 mgr.gibba032.whopeo (mgr.24625) 3889 : cluster [DBG] pgmap v2205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:02 gibba032 bash[24004]: cluster 2021-03-10T15:06:01.609854+0000 mgr.gibba032.whopeo (mgr.24625) 3890 : cluster [DBG] pgmap v2206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:02 gibba011 bash[30791]: cluster 2021-03-10T15:06:01.609854+0000 mgr.gibba032.whopeo (mgr.24625) 3890 : cluster [DBG] pgmap v2206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:03.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:02 gibba011 bash[31774]: cluster 2021-03-10T15:06:01.609854+0000 mgr.gibba032.whopeo (mgr.24625) 3890 : cluster [DBG] pgmap v2206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:04 gibba032 bash[24004]: cluster 2021-03-10T15:06:03.611508+0000 mgr.gibba032.whopeo (mgr.24625) 3891 : cluster [DBG] pgmap v2207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:05.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:04 gibba011 bash[30791]: cluster 2021-03-10T15:06:03.611508+0000 mgr.gibba032.whopeo (mgr.24625) 3891 : cluster [DBG] pgmap v2207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:04 gibba011 bash[31774]: cluster 2021-03-10T15:06:03.611508+0000 mgr.gibba032.whopeo (mgr.24625) 3891 : cluster [DBG] pgmap v2207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:06.875 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:06:07.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:06 gibba032 bash[24004]: cluster 2021-03-10T15:06:05.613099+0000 mgr.gibba032.whopeo (mgr.24625) 3892 : cluster [DBG] pgmap v2208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:07.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:06 gibba011 bash[30791]: cluster 2021-03-10T15:06:05.613099+0000 mgr.gibba032.whopeo (mgr.24625) 3892 : cluster [DBG] pgmap v2208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:07.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:06 gibba011 bash[31774]: cluster 2021-03-10T15:06:05.613099+0000 mgr.gibba032.whopeo (mgr.24625) 3892 : cluster [DBG] pgmap v2208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:07.113 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:06:07.113 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (72m) 7m ago 78m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:06:07.114 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (77m) 6m ago 77m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:06:07.114 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (73m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:06:07.114 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (73m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:06:07.114 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (72m) 7m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:06:07.115 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (72m) 7m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:06:07.115 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (71m) 6m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:06:07.115 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (72m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:06:07.115 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (78m) 7m ago 78m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:06:07.115 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (78m) 6m ago 78m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:06:07.116 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (71m) 7m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:06:07.116 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (70m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:06:07.116 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (70m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:06:07.116 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (69m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:06:07.117 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (69m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:06:07.117 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (68m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:06:07.117 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (68m) 6m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:06:07.117 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (68m) 6m ago 78m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:06:07.118 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (77m) 6m ago 78m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:06:07.118 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 76m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:06:07.369 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:06:07.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:06:07.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:06:07.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:07.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:06:07.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:06:07.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:07.371 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:06:07.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:06:07.372 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:07.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:06:07.372 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:06:07.372 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:06:07.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:06:07.373 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:06:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:07 gibba032 bash[24004]: audit 2021-03-10T15:06:06.861110+0000 mgr.gibba032.whopeo (mgr.24625) 3893 : audit [DBG] from='client.56650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:08.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:07 gibba032 bash[24004]: audit 2021-03-10T15:06:07.367041+0000 mon.c (mon.1) 74 : audit [DBG] from='client.? 172.21.2.111:0/3102349960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:07 gibba011 bash[30791]: audit 2021-03-10T15:06:06.861110+0000 mgr.gibba032.whopeo (mgr.24625) 3893 : audit [DBG] from='client.56650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:08.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:07 gibba011 bash[30791]: audit 2021-03-10T15:06:07.367041+0000 mon.c (mon.1) 74 : audit [DBG] from='client.? 172.21.2.111:0/3102349960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:07 gibba011 bash[31774]: audit 2021-03-10T15:06:06.861110+0000 mgr.gibba032.whopeo (mgr.24625) 3893 : audit [DBG] from='client.56650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:08.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:07 gibba011 bash[31774]: audit 2021-03-10T15:06:07.367041+0000 mon.c (mon.1) 74 : audit [DBG] from='client.? 172.21.2.111:0/3102349960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:09.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:08 gibba032 bash[24004]: audit 2021-03-10T15:06:07.084044+0000 mgr.gibba032.whopeo (mgr.24625) 3894 : audit [DBG] from='client.46671 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:09.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:08 gibba032 bash[24004]: cluster 2021-03-10T15:06:07.614634+0000 mgr.gibba032.whopeo (mgr.24625) 3895 : cluster [DBG] pgmap v2209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:09.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:08 gibba011 bash[30791]: audit 2021-03-10T15:06:07.084044+0000 mgr.gibba032.whopeo (mgr.24625) 3894 : audit [DBG] from='client.46671 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:09.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:08 gibba011 bash[30791]: cluster 2021-03-10T15:06:07.614634+0000 mgr.gibba032.whopeo (mgr.24625) 3895 : cluster [DBG] pgmap v2209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:09.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:08 gibba011 bash[31774]: audit 2021-03-10T15:06:07.084044+0000 mgr.gibba032.whopeo (mgr.24625) 3894 : audit [DBG] from='client.46671 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:09.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:08 gibba011 bash[31774]: cluster 2021-03-10T15:06:07.614634+0000 mgr.gibba032.whopeo (mgr.24625) 3895 : cluster [DBG] pgmap v2209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:10 gibba032 bash[24004]: cluster 2021-03-10T15:06:09.616156+0000 mgr.gibba032.whopeo (mgr.24625) 3896 : cluster [DBG] pgmap v2210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:11.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:10 gibba011 bash[30791]: cluster 2021-03-10T15:06:09.616156+0000 mgr.gibba032.whopeo (mgr.24625) 3896 : cluster [DBG] pgmap v2210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:11.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:10 gibba011 bash[31774]: cluster 2021-03-10T15:06:09.616156+0000 mgr.gibba032.whopeo (mgr.24625) 3896 : cluster [DBG] pgmap v2210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:12 gibba032 bash[24004]: cluster 2021-03-10T15:06:11.617845+0000 mgr.gibba032.whopeo (mgr.24625) 3897 : cluster [DBG] pgmap v2211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:12 gibba011 bash[30791]: cluster 2021-03-10T15:06:11.617845+0000 mgr.gibba032.whopeo (mgr.24625) 3897 : cluster [DBG] pgmap v2211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:12 gibba011 bash[31774]: cluster 2021-03-10T15:06:11.617845+0000 mgr.gibba032.whopeo (mgr.24625) 3897 : cluster [DBG] pgmap v2211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:14 gibba032 bash[24004]: cluster 2021-03-10T15:06:13.619416+0000 mgr.gibba032.whopeo (mgr.24625) 3898 : cluster [DBG] pgmap v2212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:14 gibba011 bash[30791]: cluster 2021-03-10T15:06:13.619416+0000 mgr.gibba032.whopeo (mgr.24625) 3898 : cluster [DBG] pgmap v2212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:14 gibba011 bash[31774]: cluster 2021-03-10T15:06:13.619416+0000 mgr.gibba032.whopeo (mgr.24625) 3898 : cluster [DBG] pgmap v2212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:15 gibba032 bash[24004]: cephadm 2021-03-10T15:06:14.322389+0000 mgr.gibba032.whopeo (mgr.24625) 3899 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:16.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:15 gibba011 bash[30791]: cephadm 2021-03-10T15:06:14.322389+0000 mgr.gibba032.whopeo (mgr.24625) 3899 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:16.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:15 gibba011 bash[31774]: cephadm 2021-03-10T15:06:14.322389+0000 mgr.gibba032.whopeo (mgr.24625) 3899 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:16 gibba032 bash[24004]: cluster 2021-03-10T15:06:15.621037+0000 mgr.gibba032.whopeo (mgr.24625) 3900 : cluster [DBG] pgmap v2213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:16 gibba011 bash[30791]: cluster 2021-03-10T15:06:15.621037+0000 mgr.gibba032.whopeo (mgr.24625) 3900 : cluster [DBG] pgmap v2213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:16 gibba011 bash[31774]: cluster 2021-03-10T15:06:15.621037+0000 mgr.gibba032.whopeo (mgr.24625) 3900 : cluster [DBG] pgmap v2213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:18 gibba011 bash[30791]: cluster 2021-03-10T15:06:17.622814+0000 mgr.gibba032.whopeo (mgr.24625) 3901 : cluster [DBG] pgmap v2214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:18 gibba011 bash[31774]: cluster 2021-03-10T15:06:17.622814+0000 mgr.gibba032.whopeo (mgr.24625) 3901 : cluster [DBG] pgmap v2214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:18 gibba032 bash[24004]: cluster 2021-03-10T15:06:17.622814+0000 mgr.gibba032.whopeo (mgr.24625) 3901 : cluster [DBG] pgmap v2214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:20 gibba032 bash[24004]: cluster 2021-03-10T15:06:19.624220+0000 mgr.gibba032.whopeo (mgr.24625) 3902 : cluster [DBG] pgmap v2215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:21.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:20 gibba011 bash[30791]: cluster 2021-03-10T15:06:19.624220+0000 mgr.gibba032.whopeo (mgr.24625) 3902 : cluster [DBG] pgmap v2215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:21.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:20 gibba011 bash[31774]: cluster 2021-03-10T15:06:19.624220+0000 mgr.gibba032.whopeo (mgr.24625) 3902 : cluster [DBG] pgmap v2215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:22 gibba032 bash[24004]: cluster 2021-03-10T15:06:21.625947+0000 mgr.gibba032.whopeo (mgr.24625) 3903 : cluster [DBG] pgmap v2216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:22 gibba011 bash[30791]: cluster 2021-03-10T15:06:21.625947+0000 mgr.gibba032.whopeo (mgr.24625) 3903 : cluster [DBG] pgmap v2216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:23.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:22 gibba011 bash[31774]: cluster 2021-03-10T15:06:21.625947+0000 mgr.gibba032.whopeo (mgr.24625) 3903 : cluster [DBG] pgmap v2216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:24 gibba032 bash[24004]: cluster 2021-03-10T15:06:23.627539+0000 mgr.gibba032.whopeo (mgr.24625) 3904 : cluster [DBG] pgmap v2217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:24 gibba011 bash[30791]: cluster 2021-03-10T15:06:23.627539+0000 mgr.gibba032.whopeo (mgr.24625) 3904 : cluster [DBG] pgmap v2217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:25.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:24 gibba011 bash[31774]: cluster 2021-03-10T15:06:23.627539+0000 mgr.gibba032.whopeo (mgr.24625) 3904 : cluster [DBG] pgmap v2217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:26 gibba032 bash[24004]: cluster 2021-03-10T15:06:25.629169+0000 mgr.gibba032.whopeo (mgr.24625) 3905 : cluster [DBG] pgmap v2218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:26 gibba011 bash[30791]: cluster 2021-03-10T15:06:25.629169+0000 mgr.gibba032.whopeo (mgr.24625) 3905 : cluster [DBG] pgmap v2218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:27.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:26 gibba011 bash[31774]: cluster 2021-03-10T15:06:25.629169+0000 mgr.gibba032.whopeo (mgr.24625) 3905 : cluster [DBG] pgmap v2218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:28 gibba011 bash[30791]: cluster 2021-03-10T15:06:27.630897+0000 mgr.gibba032.whopeo (mgr.24625) 3906 : cluster [DBG] pgmap v2219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:28 gibba011 bash[31774]: cluster 2021-03-10T15:06:27.630897+0000 mgr.gibba032.whopeo (mgr.24625) 3906 : cluster [DBG] pgmap v2219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:28 gibba032 bash[24004]: cluster 2021-03-10T15:06:27.630897+0000 mgr.gibba032.whopeo (mgr.24625) 3906 : cluster [DBG] pgmap v2219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.333890+0000 mgr.gibba032.whopeo (mgr.24625) 3907 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:06:31.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.335410+0000 mgr.gibba032.whopeo (mgr.24625) 3908 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:06:31.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.613463+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:06:31.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.614555+0000 mgr.gibba032.whopeo (mgr.24625) 3909 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:06:31.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.616719+0000 mgr.gibba032.whopeo (mgr.24625) 3910 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:06:31.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.617891+0000 mon.a (mon.0) 1667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:06:31.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.618211+0000 mgr.gibba032.whopeo (mgr.24625) 3911 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:06:31.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.618268+0000 mgr.gibba032.whopeo (mgr.24625) 3912 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:06:31.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.619566+0000 mon.a (mon.0) 1668 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.619797+0000 mgr.gibba032.whopeo (mgr.24625) 3913 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:06:31.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.619860+0000 mgr.gibba032.whopeo (mgr.24625) 3914 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:06:31.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.621125+0000 mon.a (mon.0) 1669 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.621272+0000 mgr.gibba032.whopeo (mgr.24625) 3915 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:06:31.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.622598+0000 mon.a (mon.0) 1670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.622775+0000 mgr.gibba032.whopeo (mgr.24625) 3916 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:06:31.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.622839+0000 mgr.gibba032.whopeo (mgr.24625) 3917 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:06:31.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.624711+0000 mgr.gibba032.whopeo (mgr.24625) 3918 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:06:31.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.625356+0000 mgr.gibba032.whopeo (mgr.24625) 3919 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:06:31.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.625827+0000 mon.a (mon.0) 1671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.627565+0000 mgr.gibba032.whopeo (mgr.24625) 3920 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:06:31.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.627810+0000 mgr.gibba032.whopeo (mgr.24625) 3921 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:06:31.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cephadm 2021-03-10T15:06:29.628048+0000 mgr.gibba032.whopeo (mgr.24625) 3922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:31.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: audit 2021-03-10T15:06:29.628083+0000 mon.a (mon.0) 1672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:30 gibba032 bash[24004]: cluster 2021-03-10T15:06:29.631768+0000 mgr.gibba032.whopeo (mgr.24625) 3923 : cluster [DBG] pgmap v2220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.333890+0000 mgr.gibba032.whopeo (mgr.24625) 3907 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:06:31.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.335410+0000 mgr.gibba032.whopeo (mgr.24625) 3908 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:06:31.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.613463+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:06:31.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.614555+0000 mgr.gibba032.whopeo (mgr.24625) 3909 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:06:31.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.616719+0000 mgr.gibba032.whopeo (mgr.24625) 3910 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:06:31.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.617891+0000 mon.a (mon.0) 1667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:06:31.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.618211+0000 mgr.gibba032.whopeo (mgr.24625) 3911 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:06:31.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.618268+0000 mgr.gibba032.whopeo (mgr.24625) 3912 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:06:31.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.619566+0000 mon.a (mon.0) 1668 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.619797+0000 mgr.gibba032.whopeo (mgr.24625) 3913 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:06:31.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.619860+0000 mgr.gibba032.whopeo (mgr.24625) 3914 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:06:31.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.621125+0000 mon.a (mon.0) 1669 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.621272+0000 mgr.gibba032.whopeo (mgr.24625) 3915 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:06:31.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.622598+0000 mon.a (mon.0) 1670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.622775+0000 mgr.gibba032.whopeo (mgr.24625) 3916 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:06:31.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.622839+0000 mgr.gibba032.whopeo (mgr.24625) 3917 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:06:31.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.624711+0000 mgr.gibba032.whopeo (mgr.24625) 3918 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:06:31.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.625356+0000 mgr.gibba032.whopeo (mgr.24625) 3919 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:06:31.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.625827+0000 mon.a (mon.0) 1671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.627565+0000 mgr.gibba032.whopeo (mgr.24625) 3920 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:06:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.627810+0000 mgr.gibba032.whopeo (mgr.24625) 3921 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:06:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cephadm 2021-03-10T15:06:29.628048+0000 mgr.gibba032.whopeo (mgr.24625) 3922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: audit 2021-03-10T15:06:29.628083+0000 mon.a (mon.0) 1672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[30791]: cluster 2021-03-10T15:06:29.631768+0000 mgr.gibba032.whopeo (mgr.24625) 3923 : cluster [DBG] pgmap v2220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.333890+0000 mgr.gibba032.whopeo (mgr.24625) 3907 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:06:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.335410+0000 mgr.gibba032.whopeo (mgr.24625) 3908 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:06:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.613463+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:06:31.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.614555+0000 mgr.gibba032.whopeo (mgr.24625) 3909 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:06:31.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.616719+0000 mgr.gibba032.whopeo (mgr.24625) 3910 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:06:31.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.617891+0000 mon.a (mon.0) 1667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:06:31.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.618211+0000 mgr.gibba032.whopeo (mgr.24625) 3911 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:06:31.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.618268+0000 mgr.gibba032.whopeo (mgr.24625) 3912 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:06:31.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.619566+0000 mon.a (mon.0) 1668 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.619797+0000 mgr.gibba032.whopeo (mgr.24625) 3913 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:06:31.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.619860+0000 mgr.gibba032.whopeo (mgr.24625) 3914 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:06:31.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.621125+0000 mon.a (mon.0) 1669 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.621272+0000 mgr.gibba032.whopeo (mgr.24625) 3915 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:06:31.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.622598+0000 mon.a (mon.0) 1670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.622775+0000 mgr.gibba032.whopeo (mgr.24625) 3916 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:06:31.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.622839+0000 mgr.gibba032.whopeo (mgr.24625) 3917 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:06:31.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.624711+0000 mgr.gibba032.whopeo (mgr.24625) 3918 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:06:31.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.625356+0000 mgr.gibba032.whopeo (mgr.24625) 3919 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:06:31.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.625827+0000 mon.a (mon.0) 1671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.627565+0000 mgr.gibba032.whopeo (mgr.24625) 3920 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:06:31.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.627810+0000 mgr.gibba032.whopeo (mgr.24625) 3921 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:06:31.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cephadm 2021-03-10T15:06:29.628048+0000 mgr.gibba032.whopeo (mgr.24625) 3922 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:31.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: audit 2021-03-10T15:06:29.628083+0000 mon.a (mon.0) 1672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:31.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:30 gibba011 bash[31774]: cluster 2021-03-10T15:06:29.631768+0000 mgr.gibba032.whopeo (mgr.24625) 3923 : cluster [DBG] pgmap v2220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:32 gibba032 bash[24004]: cluster 2021-03-10T15:06:31.633443+0000 mgr.gibba032.whopeo (mgr.24625) 3924 : cluster [DBG] pgmap v2221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:33.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:32 gibba011 bash[30791]: cluster 2021-03-10T15:06:31.633443+0000 mgr.gibba032.whopeo (mgr.24625) 3924 : cluster [DBG] pgmap v2221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:33.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:32 gibba011 bash[31774]: cluster 2021-03-10T15:06:31.633443+0000 mgr.gibba032.whopeo (mgr.24625) 3924 : cluster [DBG] pgmap v2221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:34 gibba032 bash[24004]: cluster 2021-03-10T15:06:33.635077+0000 mgr.gibba032.whopeo (mgr.24625) 3925 : cluster [DBG] pgmap v2222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:35.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:34 gibba011 bash[30791]: cluster 2021-03-10T15:06:33.635077+0000 mgr.gibba032.whopeo (mgr.24625) 3925 : cluster [DBG] pgmap v2222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:35.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:34 gibba011 bash[31774]: cluster 2021-03-10T15:06:33.635077+0000 mgr.gibba032.whopeo (mgr.24625) 3925 : cluster [DBG] pgmap v2222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:36 gibba032 bash[24004]: cluster 2021-03-10T15:06:35.636661+0000 mgr.gibba032.whopeo (mgr.24625) 3926 : cluster [DBG] pgmap v2223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:36 gibba011 bash[30791]: cluster 2021-03-10T15:06:35.636661+0000 mgr.gibba032.whopeo (mgr.24625) 3926 : cluster [DBG] pgmap v2223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:36 gibba011 bash[31774]: cluster 2021-03-10T15:06:35.636661+0000 mgr.gibba032.whopeo (mgr.24625) 3926 : cluster [DBG] pgmap v2223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:37.613 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:06:37.840 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:06:37.841 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (73m) 8m ago 78m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:06:37.841 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (78m) 7m ago 78m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:06:37.841 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (73m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:06:37.842 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (73m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:06:37.842 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (72m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:06:37.842 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (72m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:06:37.842 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (72m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:06:37.843 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (72m) 8m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:06:37.843 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (78m) 8m ago 78m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:06:37.843 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (78m) 7m ago 78m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:06:37.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (71m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:06:37.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (71m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:06:37.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (70m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:06:37.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (70m) 8m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:06:37.845 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (69m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:06:37.845 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (69m) 7m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:06:37.845 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (69m) 7m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:06:37.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (68m) 7m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:06:37.846 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (78m) 7m ago 78m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:06:37.846 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 76m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:06:38.095 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:06:38.096 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:06:38.096 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:06:38.097 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:38.097 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:06:38.097 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:06:38.097 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:38.098 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:06:38.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:06:38.098 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:06:38.098 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:06:38.099 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:06:38.099 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:06:38.099 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:06:38.099 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:06:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[30791]: audit 2021-03-10T15:06:37.599505+0000 mgr.gibba032.whopeo (mgr.24625) 3927 : audit [DBG] from='client.56668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[30791]: cluster 2021-03-10T15:06:37.638405+0000 mgr.gibba032.whopeo (mgr.24625) 3928 : cluster [DBG] pgmap v2224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[30791]: audit 2021-03-10T15:06:37.823592+0000 mgr.gibba032.whopeo (mgr.24625) 3929 : audit [DBG] from='client.56674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[31774]: audit 2021-03-10T15:06:37.599505+0000 mgr.gibba032.whopeo (mgr.24625) 3927 : audit [DBG] from='client.56668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:38.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[31774]: cluster 2021-03-10T15:06:37.638405+0000 mgr.gibba032.whopeo (mgr.24625) 3928 : cluster [DBG] pgmap v2224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:38.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:38 gibba011 bash[31774]: audit 2021-03-10T15:06:37.823592+0000 mgr.gibba032.whopeo (mgr.24625) 3929 : audit [DBG] from='client.56674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:38 gibba032 bash[24004]: audit 2021-03-10T15:06:37.599505+0000 mgr.gibba032.whopeo (mgr.24625) 3927 : audit [DBG] from='client.56668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:38 gibba032 bash[24004]: cluster 2021-03-10T15:06:37.638405+0000 mgr.gibba032.whopeo (mgr.24625) 3928 : cluster [DBG] pgmap v2224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:38 gibba032 bash[24004]: audit 2021-03-10T15:06:37.823592+0000 mgr.gibba032.whopeo (mgr.24625) 3929 : audit [DBG] from='client.56674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:06:39.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:39 gibba011 bash[30791]: audit 2021-03-10T15:06:38.093139+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? 172.21.2.111:0/2735609220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:39 gibba011 bash[31774]: audit 2021-03-10T15:06:38.093139+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? 172.21.2.111:0/2735609220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:39.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:39 gibba032 bash[24004]: audit 2021-03-10T15:06:38.093139+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? 172.21.2.111:0/2735609220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:06:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:40 gibba011 bash[30791]: cluster 2021-03-10T15:06:39.639886+0000 mgr.gibba032.whopeo (mgr.24625) 3930 : cluster [DBG] pgmap v2225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:40 gibba011 bash[31774]: cluster 2021-03-10T15:06:39.639886+0000 mgr.gibba032.whopeo (mgr.24625) 3930 : cluster [DBG] pgmap v2225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:40 gibba032 bash[24004]: cluster 2021-03-10T15:06:39.639886+0000 mgr.gibba032.whopeo (mgr.24625) 3930 : cluster [DBG] pgmap v2225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:42 gibba032 bash[24004]: cluster 2021-03-10T15:06:41.641649+0000 mgr.gibba032.whopeo (mgr.24625) 3931 : cluster [DBG] pgmap v2226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:42 gibba011 bash[30791]: cluster 2021-03-10T15:06:41.641649+0000 mgr.gibba032.whopeo (mgr.24625) 3931 : cluster [DBG] pgmap v2226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:42 gibba011 bash[31774]: cluster 2021-03-10T15:06:41.641649+0000 mgr.gibba032.whopeo (mgr.24625) 3931 : cluster [DBG] pgmap v2226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:44 gibba032 bash[24004]: cluster 2021-03-10T15:06:43.643141+0000 mgr.gibba032.whopeo (mgr.24625) 3932 : cluster [DBG] pgmap v2227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:45.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:44 gibba011 bash[30791]: cluster 2021-03-10T15:06:43.643141+0000 mgr.gibba032.whopeo (mgr.24625) 3932 : cluster [DBG] pgmap v2227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:44 gibba011 bash[31774]: cluster 2021-03-10T15:06:43.643141+0000 mgr.gibba032.whopeo (mgr.24625) 3932 : cluster [DBG] pgmap v2227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:45 gibba032 bash[24004]: cephadm 2021-03-10T15:06:44.644089+0000 mgr.gibba032.whopeo (mgr.24625) 3933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:45 gibba011 bash[30791]: cephadm 2021-03-10T15:06:44.644089+0000 mgr.gibba032.whopeo (mgr.24625) 3933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:46.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:45 gibba011 bash[31774]: cephadm 2021-03-10T15:06:44.644089+0000 mgr.gibba032.whopeo (mgr.24625) 3933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:06:47.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:46 gibba032 bash[24004]: cluster 2021-03-10T15:06:45.644762+0000 mgr.gibba032.whopeo (mgr.24625) 3934 : cluster [DBG] pgmap v2228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:46 gibba011 bash[30791]: cluster 2021-03-10T15:06:45.644762+0000 mgr.gibba032.whopeo (mgr.24625) 3934 : cluster [DBG] pgmap v2228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:46 gibba011 bash[31774]: cluster 2021-03-10T15:06:45.644762+0000 mgr.gibba032.whopeo (mgr.24625) 3934 : cluster [DBG] pgmap v2228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:48 gibba011 bash[30791]: cluster 2021-03-10T15:06:47.646566+0000 mgr.gibba032.whopeo (mgr.24625) 3935 : cluster [DBG] pgmap v2229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:48 gibba011 bash[31774]: cluster 2021-03-10T15:06:47.646566+0000 mgr.gibba032.whopeo (mgr.24625) 3935 : cluster [DBG] pgmap v2229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:48 gibba032 bash[24004]: cluster 2021-03-10T15:06:47.646566+0000 mgr.gibba032.whopeo (mgr.24625) 3935 : cluster [DBG] pgmap v2229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:50 gibba032 bash[24004]: cluster 2021-03-10T15:06:49.647940+0000 mgr.gibba032.whopeo (mgr.24625) 3936 : cluster [DBG] pgmap v2230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:50 gibba011 bash[30791]: cluster 2021-03-10T15:06:49.647940+0000 mgr.gibba032.whopeo (mgr.24625) 3936 : cluster [DBG] pgmap v2230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:50 gibba011 bash[31774]: cluster 2021-03-10T15:06:49.647940+0000 mgr.gibba032.whopeo (mgr.24625) 3936 : cluster [DBG] pgmap v2230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:52 gibba032 bash[24004]: cluster 2021-03-10T15:06:51.649871+0000 mgr.gibba032.whopeo (mgr.24625) 3937 : cluster [DBG] pgmap v2231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:52 gibba011 bash[30791]: cluster 2021-03-10T15:06:51.649871+0000 mgr.gibba032.whopeo (mgr.24625) 3937 : cluster [DBG] pgmap v2231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:53.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:52 gibba011 bash[31774]: cluster 2021-03-10T15:06:51.649871+0000 mgr.gibba032.whopeo (mgr.24625) 3937 : cluster [DBG] pgmap v2231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:54 gibba032 bash[24004]: cluster 2021-03-10T15:06:53.651485+0000 mgr.gibba032.whopeo (mgr.24625) 3938 : cluster [DBG] pgmap v2232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:54 gibba011 bash[30791]: cluster 2021-03-10T15:06:53.651485+0000 mgr.gibba032.whopeo (mgr.24625) 3938 : cluster [DBG] pgmap v2232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:54 gibba011 bash[31774]: cluster 2021-03-10T15:06:53.651485+0000 mgr.gibba032.whopeo (mgr.24625) 3938 : cluster [DBG] pgmap v2232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: cluster 2021-03-10T15:06:55.653181+0000 mgr.gibba032.whopeo (mgr.24625) 3939 : cluster [DBG] pgmap v2233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.482386+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:06:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.482647+0000 mon.a (mon.0) 1675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:06:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.499789+0000 mon.a (mon.0) 1676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:06:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.499968+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:06:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.500145+0000 mon.a (mon.0) 1678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:06:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:56 gibba032 bash[24004]: audit 2021-03-10T15:06:56.500312+0000 mon.a (mon.0) 1679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:06:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: cluster 2021-03-10T15:06:55.653181+0000 mgr.gibba032.whopeo (mgr.24625) 3939 : cluster [DBG] pgmap v2233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:57.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.482386+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:06:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.482647+0000 mon.a (mon.0) 1675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:06:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.499789+0000 mon.a (mon.0) 1676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:06:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.499968+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:06:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.500145+0000 mon.a (mon.0) 1678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:06:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[30791]: audit 2021-03-10T15:06:56.500312+0000 mon.a (mon.0) 1679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:06:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: cluster 2021-03-10T15:06:55.653181+0000 mgr.gibba032.whopeo (mgr.24625) 3939 : cluster [DBG] pgmap v2233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.482386+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:06:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.482647+0000 mon.a (mon.0) 1675 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:06:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.499789+0000 mon.a (mon.0) 1676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:06:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.499968+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:06:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.500145+0000 mon.a (mon.0) 1678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:06:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:56 gibba011 bash[31774]: audit 2021-03-10T15:06:56.500312+0000 mon.a (mon.0) 1679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:06:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:06:58 gibba011 bash[30791]: cluster 2021-03-10T15:06:57.654940+0000 mgr.gibba032.whopeo (mgr.24625) 3940 : cluster [DBG] pgmap v2234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:06:58 gibba011 bash[31774]: cluster 2021-03-10T15:06:57.654940+0000 mgr.gibba032.whopeo (mgr.24625) 3940 : cluster [DBG] pgmap v2234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:06:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:06:58 gibba032 bash[24004]: cluster 2021-03-10T15:06:57.654940+0000 mgr.gibba032.whopeo (mgr.24625) 3940 : cluster [DBG] pgmap v2234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:01.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:00 gibba032 bash[24004]: cluster 2021-03-10T15:06:59.656519+0000 mgr.gibba032.whopeo (mgr.24625) 3941 : cluster [DBG] pgmap v2235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:01.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:00 gibba032 bash[24004]: cephadm 2021-03-10T15:06:59.660012+0000 mgr.gibba032.whopeo (mgr.24625) 3942 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:01.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:00 gibba011 bash[30791]: cluster 2021-03-10T15:06:59.656519+0000 mgr.gibba032.whopeo (mgr.24625) 3941 : cluster [DBG] pgmap v2235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:01.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:00 gibba011 bash[30791]: cephadm 2021-03-10T15:06:59.660012+0000 mgr.gibba032.whopeo (mgr.24625) 3942 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:00 gibba011 bash[31774]: cluster 2021-03-10T15:06:59.656519+0000 mgr.gibba032.whopeo (mgr.24625) 3941 : cluster [DBG] pgmap v2235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:00 gibba011 bash[31774]: cephadm 2021-03-10T15:06:59.660012+0000 mgr.gibba032.whopeo (mgr.24625) 3942 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:02 gibba032 bash[24004]: cluster 2021-03-10T15:07:01.658240+0000 mgr.gibba032.whopeo (mgr.24625) 3943 : cluster [DBG] pgmap v2236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:02 gibba011 bash[30791]: cluster 2021-03-10T15:07:01.658240+0000 mgr.gibba032.whopeo (mgr.24625) 3943 : cluster [DBG] pgmap v2236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:02 gibba011 bash[31774]: cluster 2021-03-10T15:07:01.658240+0000 mgr.gibba032.whopeo (mgr.24625) 3943 : cluster [DBG] pgmap v2236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:05.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:04 gibba032 bash[24004]: cluster 2021-03-10T15:07:03.659817+0000 mgr.gibba032.whopeo (mgr.24625) 3944 : cluster [DBG] pgmap v2237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:05.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:04 gibba011 bash[30791]: cluster 2021-03-10T15:07:03.659817+0000 mgr.gibba032.whopeo (mgr.24625) 3944 : cluster [DBG] pgmap v2237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:05.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:04 gibba011 bash[31774]: cluster 2021-03-10T15:07:03.659817+0000 mgr.gibba032.whopeo (mgr.24625) 3944 : cluster [DBG] pgmap v2237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:07.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:06 gibba032 bash[24004]: cluster 2021-03-10T15:07:05.660289+0000 mgr.gibba032.whopeo (mgr.24625) 3945 : cluster [DBG] pgmap v2238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:07.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:06 gibba011 bash[30791]: cluster 2021-03-10T15:07:05.660289+0000 mgr.gibba032.whopeo (mgr.24625) 3945 : cluster [DBG] pgmap v2238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:07.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:06 gibba011 bash[31774]: cluster 2021-03-10T15:07:05.660289+0000 mgr.gibba032.whopeo (mgr.24625) 3945 : cluster [DBG] pgmap v2238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:08 gibba011 bash[30791]: cluster 2021-03-10T15:07:07.661933+0000 mgr.gibba032.whopeo (mgr.24625) 3946 : cluster [DBG] pgmap v2239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:08 gibba011 bash[31774]: cluster 2021-03-10T15:07:07.661933+0000 mgr.gibba032.whopeo (mgr.24625) 3946 : cluster [DBG] pgmap v2239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:08.348 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:07:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:08 gibba032 bash[24004]: cluster 2021-03-10T15:07:07.661933+0000 mgr.gibba032.whopeo (mgr.24625) 3946 : cluster [DBG] pgmap v2239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:08.572 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:07:08.573 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (73m) 8m ago 79m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:07:08.573 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (78m) 7m ago 78m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:07:08.573 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (74m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:07:08.573 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (74m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:07:08.574 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (73m) 8m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:07:08.574 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (73m) 8m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:07:08.574 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (72m) 7m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:07:08.574 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (73m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:07:08.575 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (79m) 8m ago 79m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:07:08.575 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (79m) 7m ago 79m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:07:08.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (72m) 8m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:07:08.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (71m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:07:08.575 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (71m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:07:08.576 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (70m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:07:08.576 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (70m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:07:08.576 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (69m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:07:08.576 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (69m) 7m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:07:08.577 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (69m) 7m ago 79m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:07:08.577 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (78m) 7m ago 79m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:07:08.577 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 77m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:07:08.821 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:07:08.822 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:07:08.822 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:07:08.822 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:08.823 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:07:08.823 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:07:08.823 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:08.823 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:07:08.824 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:07:08.824 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:08.824 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:07:08.824 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:07:08.825 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:07:08.825 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:07:08.825 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:07:09.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[30791]: audit 2021-03-10T15:07:08.334610+0000 mgr.gibba032.whopeo (mgr.24625) 3947 : audit [DBG] from='client.56686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[30791]: audit 2021-03-10T15:07:08.555647+0000 mgr.gibba032.whopeo (mgr.24625) 3948 : audit [DBG] from='client.56692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[30791]: audit 2021-03-10T15:07:08.818708+0000 mon.c (mon.1) 75 : audit [DBG] from='client.? 172.21.2.111:0/1342768676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[31774]: audit 2021-03-10T15:07:08.334610+0000 mgr.gibba032.whopeo (mgr.24625) 3947 : audit [DBG] from='client.56686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[31774]: audit 2021-03-10T15:07:08.555647+0000 mgr.gibba032.whopeo (mgr.24625) 3948 : audit [DBG] from='client.56692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:09 gibba011 bash[31774]: audit 2021-03-10T15:07:08.818708+0000 mon.c (mon.1) 75 : audit [DBG] from='client.? 172.21.2.111:0/1342768676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:09 gibba032 bash[24004]: audit 2021-03-10T15:07:08.334610+0000 mgr.gibba032.whopeo (mgr.24625) 3947 : audit [DBG] from='client.56686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:09 gibba032 bash[24004]: audit 2021-03-10T15:07:08.555647+0000 mgr.gibba032.whopeo (mgr.24625) 3948 : audit [DBG] from='client.56692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:09 gibba032 bash[24004]: audit 2021-03-10T15:07:08.818708+0000 mon.c (mon.1) 75 : audit [DBG] from='client.? 172.21.2.111:0/1342768676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:10 gibba011 bash[30791]: cluster 2021-03-10T15:07:09.663411+0000 mgr.gibba032.whopeo (mgr.24625) 3949 : cluster [DBG] pgmap v2240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:10 gibba011 bash[31774]: cluster 2021-03-10T15:07:09.663411+0000 mgr.gibba032.whopeo (mgr.24625) 3949 : cluster [DBG] pgmap v2240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:10 gibba032 bash[24004]: cluster 2021-03-10T15:07:09.663411+0000 mgr.gibba032.whopeo (mgr.24625) 3949 : cluster [DBG] pgmap v2240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:12 gibba032 bash[24004]: cluster 2021-03-10T15:07:11.665076+0000 mgr.gibba032.whopeo (mgr.24625) 3950 : cluster [DBG] pgmap v2241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:12 gibba011 bash[30791]: cluster 2021-03-10T15:07:11.665076+0000 mgr.gibba032.whopeo (mgr.24625) 3950 : cluster [DBG] pgmap v2241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:12 gibba011 bash[31774]: cluster 2021-03-10T15:07:11.665076+0000 mgr.gibba032.whopeo (mgr.24625) 3950 : cluster [DBG] pgmap v2241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:15.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:14 gibba032 bash[24004]: cluster 2021-03-10T15:07:13.666523+0000 mgr.gibba032.whopeo (mgr.24625) 3951 : cluster [DBG] pgmap v2242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:14 gibba011 bash[30791]: cluster 2021-03-10T15:07:13.666523+0000 mgr.gibba032.whopeo (mgr.24625) 3951 : cluster [DBG] pgmap v2242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:14 gibba011 bash[31774]: cluster 2021-03-10T15:07:13.666523+0000 mgr.gibba032.whopeo (mgr.24625) 3951 : cluster [DBG] pgmap v2242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:15 gibba032 bash[24004]: cephadm 2021-03-10T15:07:14.676139+0000 mgr.gibba032.whopeo (mgr.24625) 3952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:16.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:15 gibba011 bash[30791]: cephadm 2021-03-10T15:07:14.676139+0000 mgr.gibba032.whopeo (mgr.24625) 3952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:16.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:15 gibba011 bash[31774]: cephadm 2021-03-10T15:07:14.676139+0000 mgr.gibba032.whopeo (mgr.24625) 3952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:16 gibba032 bash[24004]: cluster 2021-03-10T15:07:15.667012+0000 mgr.gibba032.whopeo (mgr.24625) 3953 : cluster [DBG] pgmap v2243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:16 gibba011 bash[30791]: cluster 2021-03-10T15:07:15.667012+0000 mgr.gibba032.whopeo (mgr.24625) 3953 : cluster [DBG] pgmap v2243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:16 gibba011 bash[31774]: cluster 2021-03-10T15:07:15.667012+0000 mgr.gibba032.whopeo (mgr.24625) 3953 : cluster [DBG] pgmap v2243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:18 gibba011 bash[30791]: cluster 2021-03-10T15:07:17.668575+0000 mgr.gibba032.whopeo (mgr.24625) 3954 : cluster [DBG] pgmap v2244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:18 gibba011 bash[31774]: cluster 2021-03-10T15:07:17.668575+0000 mgr.gibba032.whopeo (mgr.24625) 3954 : cluster [DBG] pgmap v2244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:18 gibba032 bash[24004]: cluster 2021-03-10T15:07:17.668575+0000 mgr.gibba032.whopeo (mgr.24625) 3954 : cluster [DBG] pgmap v2244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:20 gibba032 bash[24004]: cluster 2021-03-10T15:07:19.670041+0000 mgr.gibba032.whopeo (mgr.24625) 3955 : cluster [DBG] pgmap v2245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:21.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:20 gibba011 bash[30791]: cluster 2021-03-10T15:07:19.670041+0000 mgr.gibba032.whopeo (mgr.24625) 3955 : cluster [DBG] pgmap v2245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:21.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:20 gibba011 bash[31774]: cluster 2021-03-10T15:07:19.670041+0000 mgr.gibba032.whopeo (mgr.24625) 3955 : cluster [DBG] pgmap v2245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:22 gibba032 bash[24004]: cluster 2021-03-10T15:07:21.671668+0000 mgr.gibba032.whopeo (mgr.24625) 3956 : cluster [DBG] pgmap v2246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:22 gibba011 bash[30791]: cluster 2021-03-10T15:07:21.671668+0000 mgr.gibba032.whopeo (mgr.24625) 3956 : cluster [DBG] pgmap v2246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:23.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:22 gibba011 bash[31774]: cluster 2021-03-10T15:07:21.671668+0000 mgr.gibba032.whopeo (mgr.24625) 3956 : cluster [DBG] pgmap v2246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:25.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:24 gibba032 bash[24004]: cluster 2021-03-10T15:07:23.673201+0000 mgr.gibba032.whopeo (mgr.24625) 3957 : cluster [DBG] pgmap v2247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:25.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:24 gibba011 bash[30791]: cluster 2021-03-10T15:07:23.673201+0000 mgr.gibba032.whopeo (mgr.24625) 3957 : cluster [DBG] pgmap v2247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:25.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:24 gibba011 bash[31774]: cluster 2021-03-10T15:07:23.673201+0000 mgr.gibba032.whopeo (mgr.24625) 3957 : cluster [DBG] pgmap v2247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:26 gibba011 bash[30791]: cluster 2021-03-10T15:07:25.674515+0000 mgr.gibba032.whopeo (mgr.24625) 3958 : cluster [DBG] pgmap v2248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:27.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:26 gibba011 bash[31774]: cluster 2021-03-10T15:07:25.674515+0000 mgr.gibba032.whopeo (mgr.24625) 3958 : cluster [DBG] pgmap v2248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:27.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:26 gibba032 bash[24004]: cluster 2021-03-10T15:07:25.674515+0000 mgr.gibba032.whopeo (mgr.24625) 3958 : cluster [DBG] pgmap v2248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:28 gibba011 bash[30791]: cluster 2021-03-10T15:07:27.675953+0000 mgr.gibba032.whopeo (mgr.24625) 3959 : cluster [DBG] pgmap v2249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:28 gibba011 bash[31774]: cluster 2021-03-10T15:07:27.675953+0000 mgr.gibba032.whopeo (mgr.24625) 3959 : cluster [DBG] pgmap v2249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:28 gibba032 bash[24004]: cluster 2021-03-10T15:07:27.675953+0000 mgr.gibba032.whopeo (mgr.24625) 3959 : cluster [DBG] pgmap v2249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cluster 2021-03-10T15:07:29.677510+0000 mgr.gibba032.whopeo (mgr.24625) 3960 : cluster [DBG] pgmap v2250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:31.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.692841+0000 mgr.gibba032.whopeo (mgr.24625) 3961 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:07:31.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.694370+0000 mgr.gibba032.whopeo (mgr.24625) 3962 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:07:31.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.921893+0000 mon.a (mon.0) 1680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:07:31.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.921930+0000 mgr.gibba032.whopeo (mgr.24625) 3963 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:07:31.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.923856+0000 mgr.gibba032.whopeo (mgr.24625) 3964 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:07:31.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.925058+0000 mon.a (mon.0) 1681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:07:31.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.925459+0000 mgr.gibba032.whopeo (mgr.24625) 3965 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:07:31.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.925517+0000 mgr.gibba032.whopeo (mgr.24625) 3966 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:07:31.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.926795+0000 mon.a (mon.0) 1682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.926798+0000 mgr.gibba032.whopeo (mgr.24625) 3967 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:07:31.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.926859+0000 mgr.gibba032.whopeo (mgr.24625) 3968 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:07:31.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.927914+0000 mgr.gibba032.whopeo (mgr.24625) 3969 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:07:31.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.928362+0000 mon.a (mon.0) 1683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.928894+0000 mgr.gibba032.whopeo (mgr.24625) 3970 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:07:31.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.928979+0000 mgr.gibba032.whopeo (mgr.24625) 3971 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:07:31.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.929439+0000 mon.a (mon.0) 1684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.930337+0000 mgr.gibba032.whopeo (mgr.24625) 3972 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:07:31.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.930694+0000 mgr.gibba032.whopeo (mgr.24625) 3973 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:07:31.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.931648+0000 mgr.gibba032.whopeo (mgr.24625) 3974 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:07:31.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.931706+0000 mgr.gibba032.whopeo (mgr.24625) 3975 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:07:31.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: cephadm 2021-03-10T15:07:29.931846+0000 mgr.gibba032.whopeo (mgr.24625) 3976 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:31.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.931959+0000 mon.a (mon.0) 1685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:30 gibba032 bash[24004]: audit 2021-03-10T15:07:29.933256+0000 mon.a (mon.0) 1686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cluster 2021-03-10T15:07:29.677510+0000 mgr.gibba032.whopeo (mgr.24625) 3960 : cluster [DBG] pgmap v2250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.692841+0000 mgr.gibba032.whopeo (mgr.24625) 3961 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:07:31.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.694370+0000 mgr.gibba032.whopeo (mgr.24625) 3962 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:07:31.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.921893+0000 mon.a (mon.0) 1680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:07:31.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.921930+0000 mgr.gibba032.whopeo (mgr.24625) 3963 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:07:31.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.923856+0000 mgr.gibba032.whopeo (mgr.24625) 3964 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:07:31.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.925058+0000 mon.a (mon.0) 1681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:07:31.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.925459+0000 mgr.gibba032.whopeo (mgr.24625) 3965 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:07:31.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.925517+0000 mgr.gibba032.whopeo (mgr.24625) 3966 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:07:31.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.926795+0000 mon.a (mon.0) 1682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.926798+0000 mgr.gibba032.whopeo (mgr.24625) 3967 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:07:31.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.926859+0000 mgr.gibba032.whopeo (mgr.24625) 3968 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:07:31.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.927914+0000 mgr.gibba032.whopeo (mgr.24625) 3969 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:07:31.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.928362+0000 mon.a (mon.0) 1683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.928894+0000 mgr.gibba032.whopeo (mgr.24625) 3970 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:07:31.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.928979+0000 mgr.gibba032.whopeo (mgr.24625) 3971 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:07:31.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.929439+0000 mon.a (mon.0) 1684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.930337+0000 mgr.gibba032.whopeo (mgr.24625) 3972 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:07:31.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.930694+0000 mgr.gibba032.whopeo (mgr.24625) 3973 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:07:31.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.931648+0000 mgr.gibba032.whopeo (mgr.24625) 3974 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:07:31.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.931706+0000 mgr.gibba032.whopeo (mgr.24625) 3975 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:07:31.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: cephadm 2021-03-10T15:07:29.931846+0000 mgr.gibba032.whopeo (mgr.24625) 3976 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:31.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.931959+0000 mon.a (mon.0) 1685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[30791]: audit 2021-03-10T15:07:29.933256+0000 mon.a (mon.0) 1686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cluster 2021-03-10T15:07:29.677510+0000 mgr.gibba032.whopeo (mgr.24625) 3960 : cluster [DBG] pgmap v2250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:31.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.692841+0000 mgr.gibba032.whopeo (mgr.24625) 3961 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:07:31.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.694370+0000 mgr.gibba032.whopeo (mgr.24625) 3962 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:07:31.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.921893+0000 mon.a (mon.0) 1680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:07:31.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.921930+0000 mgr.gibba032.whopeo (mgr.24625) 3963 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:07:31.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.923856+0000 mgr.gibba032.whopeo (mgr.24625) 3964 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:07:31.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.925058+0000 mon.a (mon.0) 1681 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:07:31.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.925459+0000 mgr.gibba032.whopeo (mgr.24625) 3965 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:07:31.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.925517+0000 mgr.gibba032.whopeo (mgr.24625) 3966 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:07:31.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.926795+0000 mon.a (mon.0) 1682 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.926798+0000 mgr.gibba032.whopeo (mgr.24625) 3967 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:07:31.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.926859+0000 mgr.gibba032.whopeo (mgr.24625) 3968 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:07:31.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.927914+0000 mgr.gibba032.whopeo (mgr.24625) 3969 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:07:31.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.928362+0000 mon.a (mon.0) 1683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.928894+0000 mgr.gibba032.whopeo (mgr.24625) 3970 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:07:31.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.928979+0000 mgr.gibba032.whopeo (mgr.24625) 3971 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:07:31.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.929439+0000 mon.a (mon.0) 1684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.930337+0000 mgr.gibba032.whopeo (mgr.24625) 3972 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:07:31.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.930694+0000 mgr.gibba032.whopeo (mgr.24625) 3973 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:07:31.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.931648+0000 mgr.gibba032.whopeo (mgr.24625) 3974 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:07:31.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.931706+0000 mgr.gibba032.whopeo (mgr.24625) 3975 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:07:31.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: cephadm 2021-03-10T15:07:29.931846+0000 mgr.gibba032.whopeo (mgr.24625) 3976 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:31.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.931959+0000 mon.a (mon.0) 1685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:31.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:30 gibba011 bash[31774]: audit 2021-03-10T15:07:29.933256+0000 mon.a (mon.0) 1686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:32 gibba032 bash[24004]: cluster 2021-03-10T15:07:31.679049+0000 mgr.gibba032.whopeo (mgr.24625) 3977 : cluster [DBG] pgmap v2251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:32 gibba011 bash[30791]: cluster 2021-03-10T15:07:31.679049+0000 mgr.gibba032.whopeo (mgr.24625) 3977 : cluster [DBG] pgmap v2251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:32 gibba011 bash[31774]: cluster 2021-03-10T15:07:31.679049+0000 mgr.gibba032.whopeo (mgr.24625) 3977 : cluster [DBG] pgmap v2251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:33 gibba032 bash[24004]: cluster 2021-03-10T15:07:33.680650+0000 mgr.gibba032.whopeo (mgr.24625) 3978 : cluster [DBG] pgmap v2252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:33 gibba011 bash[30791]: cluster 2021-03-10T15:07:33.680650+0000 mgr.gibba032.whopeo (mgr.24625) 3978 : cluster [DBG] pgmap v2252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:33 gibba011 bash[31774]: cluster 2021-03-10T15:07:33.680650+0000 mgr.gibba032.whopeo (mgr.24625) 3978 : cluster [DBG] pgmap v2252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:36 gibba032 bash[24004]: cluster 2021-03-10T15:07:35.681964+0000 mgr.gibba032.whopeo (mgr.24625) 3979 : cluster [DBG] pgmap v2253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:36 gibba011 bash[30791]: cluster 2021-03-10T15:07:35.681964+0000 mgr.gibba032.whopeo (mgr.24625) 3979 : cluster [DBG] pgmap v2253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:37.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:36 gibba011 bash[31774]: cluster 2021-03-10T15:07:35.681964+0000 mgr.gibba032.whopeo (mgr.24625) 3979 : cluster [DBG] pgmap v2253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:38 gibba011 bash[30791]: cluster 2021-03-10T15:07:37.683518+0000 mgr.gibba032.whopeo (mgr.24625) 3980 : cluster [DBG] pgmap v2254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:38 gibba011 bash[31774]: cluster 2021-03-10T15:07:37.683518+0000 mgr.gibba032.whopeo (mgr.24625) 3980 : cluster [DBG] pgmap v2254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:38 gibba032 bash[24004]: cluster 2021-03-10T15:07:37.683518+0000 mgr.gibba032.whopeo (mgr.24625) 3980 : cluster [DBG] pgmap v2254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:39.063 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:07:39.290 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:07:39.290 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (74m) 9m ago 79m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:07:39.290 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (79m) 8m ago 79m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:07:39.291 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (74m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:07:39.291 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (74m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:07:39.291 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (73m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:07:39.291 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (73m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:07:39.292 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (73m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:07:39.292 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (73m) 9m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:07:39.292 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (79m) 9m ago 79m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:07:39.292 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (79m) 8m ago 79m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:07:39.293 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (72m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:07:39.293 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (72m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:07:39.293 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (71m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:07:39.293 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (71m) 9m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:07:39.293 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (70m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:07:39.294 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (70m) 8m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:07:39.294 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (70m) 8m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:07:39.294 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (69m) 8m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:07:39.294 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (79m) 8m ago 79m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:07:39.294 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 77m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:07:39.540 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:07:39.540 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:07:39.540 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:07:39.540 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:39.541 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:07:39.541 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:07:39.541 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:39.541 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:07:39.542 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:07:39.542 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:07:39.542 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:07:39.542 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:07:39.542 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:07:39.543 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:07:39.543 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:07:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:39 gibba032 bash[24004]: audit 2021-03-10T15:07:39.537297+0000 mon.c (mon.1) 76 : audit [DBG] from='client.? 172.21.2.111:0/75562368' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:39 gibba011 bash[30791]: audit 2021-03-10T15:07:39.537297+0000 mon.c (mon.1) 76 : audit [DBG] from='client.? 172.21.2.111:0/75562368' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:39 gibba011 bash[31774]: audit 2021-03-10T15:07:39.537297+0000 mon.c (mon.1) 76 : audit [DBG] from='client.? 172.21.2.111:0/75562368' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:07:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:40 gibba032 bash[24004]: audit 2021-03-10T15:07:39.049667+0000 mgr.gibba032.whopeo (mgr.24625) 3981 : audit [DBG] from='client.46719 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:40 gibba032 bash[24004]: audit 2021-03-10T15:07:39.272605+0000 mgr.gibba032.whopeo (mgr.24625) 3982 : audit [DBG] from='client.46725 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:40 gibba032 bash[24004]: cluster 2021-03-10T15:07:39.684700+0000 mgr.gibba032.whopeo (mgr.24625) 3983 : cluster [DBG] pgmap v2255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[30791]: audit 2021-03-10T15:07:39.049667+0000 mgr.gibba032.whopeo (mgr.24625) 3981 : audit [DBG] from='client.46719 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[30791]: audit 2021-03-10T15:07:39.272605+0000 mgr.gibba032.whopeo (mgr.24625) 3982 : audit [DBG] from='client.46725 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[30791]: cluster 2021-03-10T15:07:39.684700+0000 mgr.gibba032.whopeo (mgr.24625) 3983 : cluster [DBG] pgmap v2255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[31774]: audit 2021-03-10T15:07:39.049667+0000 mgr.gibba032.whopeo (mgr.24625) 3981 : audit [DBG] from='client.46719 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[31774]: audit 2021-03-10T15:07:39.272605+0000 mgr.gibba032.whopeo (mgr.24625) 3982 : audit [DBG] from='client.46725 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:07:41.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:40 gibba011 bash[31774]: cluster 2021-03-10T15:07:39.684700+0000 mgr.gibba032.whopeo (mgr.24625) 3983 : cluster [DBG] pgmap v2255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:42 gibba032 bash[24004]: cluster 2021-03-10T15:07:41.686159+0000 mgr.gibba032.whopeo (mgr.24625) 3984 : cluster [DBG] pgmap v2256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:42 gibba011 bash[30791]: cluster 2021-03-10T15:07:41.686159+0000 mgr.gibba032.whopeo (mgr.24625) 3984 : cluster [DBG] pgmap v2256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:42 gibba011 bash[31774]: cluster 2021-03-10T15:07:41.686159+0000 mgr.gibba032.whopeo (mgr.24625) 3984 : cluster [DBG] pgmap v2256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:45.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:44 gibba011 bash[30791]: cluster 2021-03-10T15:07:43.687704+0000 mgr.gibba032.whopeo (mgr.24625) 3985 : cluster [DBG] pgmap v2257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:44 gibba011 bash[31774]: cluster 2021-03-10T15:07:43.687704+0000 mgr.gibba032.whopeo (mgr.24625) 3985 : cluster [DBG] pgmap v2257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:44 gibba032 bash[24004]: cluster 2021-03-10T15:07:43.687704+0000 mgr.gibba032.whopeo (mgr.24625) 3985 : cluster [DBG] pgmap v2257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:46 gibba011 bash[30791]: cephadm 2021-03-10T15:07:44.947892+0000 mgr.gibba032.whopeo (mgr.24625) 3986 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:47.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:46 gibba011 bash[30791]: cluster 2021-03-10T15:07:45.689057+0000 mgr.gibba032.whopeo (mgr.24625) 3987 : cluster [DBG] pgmap v2258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:46 gibba011 bash[31774]: cephadm 2021-03-10T15:07:44.947892+0000 mgr.gibba032.whopeo (mgr.24625) 3986 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:47.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:46 gibba011 bash[31774]: cluster 2021-03-10T15:07:45.689057+0000 mgr.gibba032.whopeo (mgr.24625) 3987 : cluster [DBG] pgmap v2258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:47.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:46 gibba032 bash[24004]: cephadm 2021-03-10T15:07:44.947892+0000 mgr.gibba032.whopeo (mgr.24625) 3986 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:07:47.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:46 gibba032 bash[24004]: cluster 2021-03-10T15:07:45.689057+0000 mgr.gibba032.whopeo (mgr.24625) 3987 : cluster [DBG] pgmap v2258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:48 gibba032 bash[24004]: cluster 2021-03-10T15:07:47.690786+0000 mgr.gibba032.whopeo (mgr.24625) 3988 : cluster [DBG] pgmap v2259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:48 gibba011 bash[30791]: cluster 2021-03-10T15:07:47.690786+0000 mgr.gibba032.whopeo (mgr.24625) 3988 : cluster [DBG] pgmap v2259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:48 gibba011 bash[31774]: cluster 2021-03-10T15:07:47.690786+0000 mgr.gibba032.whopeo (mgr.24625) 3988 : cluster [DBG] pgmap v2259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:50 gibba032 bash[24004]: cluster 2021-03-10T15:07:49.692187+0000 mgr.gibba032.whopeo (mgr.24625) 3989 : cluster [DBG] pgmap v2260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:50 gibba011 bash[30791]: cluster 2021-03-10T15:07:49.692187+0000 mgr.gibba032.whopeo (mgr.24625) 3989 : cluster [DBG] pgmap v2260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:50 gibba011 bash[31774]: cluster 2021-03-10T15:07:49.692187+0000 mgr.gibba032.whopeo (mgr.24625) 3989 : cluster [DBG] pgmap v2260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:52 gibba011 bash[30791]: cluster 2021-03-10T15:07:51.693909+0000 mgr.gibba032.whopeo (mgr.24625) 3990 : cluster [DBG] pgmap v2261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:53.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:52 gibba011 bash[31774]: cluster 2021-03-10T15:07:51.693909+0000 mgr.gibba032.whopeo (mgr.24625) 3990 : cluster [DBG] pgmap v2261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:52 gibba032 bash[24004]: cluster 2021-03-10T15:07:51.693909+0000 mgr.gibba032.whopeo (mgr.24625) 3990 : cluster [DBG] pgmap v2261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:54 gibba011 bash[30791]: cluster 2021-03-10T15:07:53.695437+0000 mgr.gibba032.whopeo (mgr.24625) 3991 : cluster [DBG] pgmap v2262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:54 gibba011 bash[31774]: cluster 2021-03-10T15:07:53.695437+0000 mgr.gibba032.whopeo (mgr.24625) 3991 : cluster [DBG] pgmap v2262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:54 gibba032 bash[24004]: cluster 2021-03-10T15:07:53.695437+0000 mgr.gibba032.whopeo (mgr.24625) 3991 : cluster [DBG] pgmap v2262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: cluster 2021-03-10T15:07:55.696116+0000 mgr.gibba032.whopeo (mgr.24625) 3992 : cluster [DBG] pgmap v2263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.489220+0000 mon.a (mon.0) 1687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:07:57.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.489908+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:07:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.503766+0000 mon.a (mon.0) 1689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:07:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.504555+0000 mon.a (mon.0) 1690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:07:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.505266+0000 mon.a (mon.0) 1691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:07:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[31774]: audit 2021-03-10T15:07:56.505960+0000 mon.a (mon.0) 1692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:07:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: cluster 2021-03-10T15:07:55.696116+0000 mgr.gibba032.whopeo (mgr.24625) 3992 : cluster [DBG] pgmap v2263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.489220+0000 mon.a (mon.0) 1687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:07:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.489908+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:07:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.503766+0000 mon.a (mon.0) 1689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:07:57.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.504555+0000 mon.a (mon.0) 1690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:07:57.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.505266+0000 mon.a (mon.0) 1691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:07:57.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:56 gibba011 bash[30791]: audit 2021-03-10T15:07:56.505960+0000 mon.a (mon.0) 1692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:07:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: cluster 2021-03-10T15:07:55.696116+0000 mgr.gibba032.whopeo (mgr.24625) 3992 : cluster [DBG] pgmap v2263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.489220+0000 mon.a (mon.0) 1687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:07:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.489908+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:07:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.503766+0000 mon.a (mon.0) 1689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:07:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.504555+0000 mon.a (mon.0) 1690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:07:57.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.505266+0000 mon.a (mon.0) 1691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:07:57.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:56 gibba032 bash[24004]: audit 2021-03-10T15:07:56.505960+0000 mon.a (mon.0) 1692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:07:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:07:58 gibba032 bash[24004]: cluster 2021-03-10T15:07:57.697533+0000 mgr.gibba032.whopeo (mgr.24625) 3993 : cluster [DBG] pgmap v2264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:07:58 gibba011 bash[30791]: cluster 2021-03-10T15:07:57.697533+0000 mgr.gibba032.whopeo (mgr.24625) 3993 : cluster [DBG] pgmap v2264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:07:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:07:58 gibba011 bash[31774]: cluster 2021-03-10T15:07:57.697533+0000 mgr.gibba032.whopeo (mgr.24625) 3993 : cluster [DBG] pgmap v2264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:01.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:00 gibba011 bash[30791]: cluster 2021-03-10T15:07:59.699058+0000 mgr.gibba032.whopeo (mgr.24625) 3994 : cluster [DBG] pgmap v2265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:00 gibba011 bash[31774]: cluster 2021-03-10T15:07:59.699058+0000 mgr.gibba032.whopeo (mgr.24625) 3994 : cluster [DBG] pgmap v2265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:00 gibba032 bash[24004]: cluster 2021-03-10T15:07:59.699058+0000 mgr.gibba032.whopeo (mgr.24625) 3994 : cluster [DBG] pgmap v2265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:01 gibba011 bash[30791]: cephadm 2021-03-10T15:07:59.963915+0000 mgr.gibba032.whopeo (mgr.24625) 3995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:02.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:01 gibba011 bash[31774]: cephadm 2021-03-10T15:07:59.963915+0000 mgr.gibba032.whopeo (mgr.24625) 3995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:01 gibba032 bash[24004]: cephadm 2021-03-10T15:07:59.963915+0000 mgr.gibba032.whopeo (mgr.24625) 3995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:02 gibba011 bash[30791]: cluster 2021-03-10T15:08:01.700859+0000 mgr.gibba032.whopeo (mgr.24625) 3996 : cluster [DBG] pgmap v2266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:02 gibba011 bash[31774]: cluster 2021-03-10T15:08:01.700859+0000 mgr.gibba032.whopeo (mgr.24625) 3996 : cluster [DBG] pgmap v2266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:02 gibba032 bash[24004]: cluster 2021-03-10T15:08:01.700859+0000 mgr.gibba032.whopeo (mgr.24625) 3996 : cluster [DBG] pgmap v2266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:05.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:04 gibba011 bash[30791]: cluster 2021-03-10T15:08:03.702473+0000 mgr.gibba032.whopeo (mgr.24625) 3997 : cluster [DBG] pgmap v2267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:04 gibba011 bash[31774]: cluster 2021-03-10T15:08:03.702473+0000 mgr.gibba032.whopeo (mgr.24625) 3997 : cluster [DBG] pgmap v2267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:04 gibba032 bash[24004]: cluster 2021-03-10T15:08:03.702473+0000 mgr.gibba032.whopeo (mgr.24625) 3997 : cluster [DBG] pgmap v2267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:07.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:06 gibba011 bash[30791]: cluster 2021-03-10T15:08:05.703316+0000 mgr.gibba032.whopeo (mgr.24625) 3998 : cluster [DBG] pgmap v2268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:07.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:06 gibba011 bash[31774]: cluster 2021-03-10T15:08:05.703316+0000 mgr.gibba032.whopeo (mgr.24625) 3998 : cluster [DBG] pgmap v2268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:07.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:06 gibba032 bash[24004]: cluster 2021-03-10T15:08:05.703316+0000 mgr.gibba032.whopeo (mgr.24625) 3998 : cluster [DBG] pgmap v2268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:08 gibba032 bash[24004]: cluster 2021-03-10T15:08:07.705061+0000 mgr.gibba032.whopeo (mgr.24625) 3999 : cluster [DBG] pgmap v2269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:08 gibba011 bash[30791]: cluster 2021-03-10T15:08:07.705061+0000 mgr.gibba032.whopeo (mgr.24625) 3999 : cluster [DBG] pgmap v2269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:08 gibba011 bash[31774]: cluster 2021-03-10T15:08:07.705061+0000 mgr.gibba032.whopeo (mgr.24625) 3999 : cluster [DBG] pgmap v2269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:09.782 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:08:10.012 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:08:10.012 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (74m) 9m ago 80m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:08:10.012 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (80m) 8m ago 80m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:08:10.013 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (75m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:08:10.013 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (75m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:08:10.013 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (74m) 9m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:08:10.013 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (74m) 9m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:08:10.014 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (73m) 8m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:08:10.014 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (74m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:08:10.014 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (80m) 9m ago 80m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:08:10.014 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (80m) 8m ago 80m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:08:10.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (73m) 9m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:08:10.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (72m) 9m ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:08:10.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (72m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:08:10.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (71m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:08:10.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (71m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:08:10.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (70m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:08:10.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (70m) 8m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:08:10.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (70m) 8m ago 80m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:08:10.016 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (79m) 8m ago 80m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:08:10.017 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 78m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:08:10.266 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:08:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:08:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:08:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:08:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:08:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:08:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:08:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:08:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:08:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:08:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:08:10.269 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:08:11.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[30791]: cluster 2021-03-10T15:08:09.706550+0000 mgr.gibba032.whopeo (mgr.24625) 4000 : cluster [DBG] pgmap v2270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:11.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[30791]: audit 2021-03-10T15:08:09.768354+0000 mgr.gibba032.whopeo (mgr.24625) 4001 : audit [DBG] from='client.56722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:11.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[30791]: audit 2021-03-10T15:08:10.263576+0000 mon.c (mon.1) 77 : audit [DBG] from='client.? 172.21.2.111:0/4114800366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:11.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[31774]: cluster 2021-03-10T15:08:09.706550+0000 mgr.gibba032.whopeo (mgr.24625) 4000 : cluster [DBG] pgmap v2270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:11.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[31774]: audit 2021-03-10T15:08:09.768354+0000 mgr.gibba032.whopeo (mgr.24625) 4001 : audit [DBG] from='client.56722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:11.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:10 gibba011 bash[31774]: audit 2021-03-10T15:08:10.263576+0000 mon.c (mon.1) 77 : audit [DBG] from='client.? 172.21.2.111:0/4114800366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:11.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:10 gibba032 bash[24004]: cluster 2021-03-10T15:08:09.706550+0000 mgr.gibba032.whopeo (mgr.24625) 4000 : cluster [DBG] pgmap v2270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:11.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:10 gibba032 bash[24004]: audit 2021-03-10T15:08:09.768354+0000 mgr.gibba032.whopeo (mgr.24625) 4001 : audit [DBG] from='client.56722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:11.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:10 gibba032 bash[24004]: audit 2021-03-10T15:08:10.263576+0000 mon.c (mon.1) 77 : audit [DBG] from='client.? 172.21.2.111:0/4114800366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:12.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:11 gibba011 bash[30791]: audit 2021-03-10T15:08:09.994646+0000 mgr.gibba032.whopeo (mgr.24625) 4002 : audit [DBG] from='client.46743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:12.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:11 gibba011 bash[31774]: audit 2021-03-10T15:08:09.994646+0000 mgr.gibba032.whopeo (mgr.24625) 4002 : audit [DBG] from='client.46743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:12.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:11 gibba032 bash[24004]: audit 2021-03-10T15:08:09.994646+0000 mgr.gibba032.whopeo (mgr.24625) 4002 : audit [DBG] from='client.46743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:12 gibba011 bash[30791]: cluster 2021-03-10T15:08:11.708389+0000 mgr.gibba032.whopeo (mgr.24625) 4003 : cluster [DBG] pgmap v2271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:12 gibba011 bash[31774]: cluster 2021-03-10T15:08:11.708389+0000 mgr.gibba032.whopeo (mgr.24625) 4003 : cluster [DBG] pgmap v2271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:12 gibba032 bash[24004]: cluster 2021-03-10T15:08:11.708389+0000 mgr.gibba032.whopeo (mgr.24625) 4003 : cluster [DBG] pgmap v2271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:15.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:14 gibba011 bash[30791]: cluster 2021-03-10T15:08:13.709975+0000 mgr.gibba032.whopeo (mgr.24625) 4004 : cluster [DBG] pgmap v2272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:15.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:14 gibba011 bash[31774]: cluster 2021-03-10T15:08:13.709975+0000 mgr.gibba032.whopeo (mgr.24625) 4004 : cluster [DBG] pgmap v2272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:15.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:14 gibba032 bash[24004]: cluster 2021-03-10T15:08:13.709975+0000 mgr.gibba032.whopeo (mgr.24625) 4004 : cluster [DBG] pgmap v2272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:17.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:16 gibba011 bash[30791]: cephadm 2021-03-10T15:08:14.979721+0000 mgr.gibba032.whopeo (mgr.24625) 4005 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:17.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:16 gibba011 bash[30791]: cluster 2021-03-10T15:08:15.710677+0000 mgr.gibba032.whopeo (mgr.24625) 4006 : cluster [DBG] pgmap v2273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:17.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:16 gibba011 bash[31774]: cephadm 2021-03-10T15:08:14.979721+0000 mgr.gibba032.whopeo (mgr.24625) 4005 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:17.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:16 gibba011 bash[31774]: cluster 2021-03-10T15:08:15.710677+0000 mgr.gibba032.whopeo (mgr.24625) 4006 : cluster [DBG] pgmap v2273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:17.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:16 gibba032 bash[24004]: cephadm 2021-03-10T15:08:14.979721+0000 mgr.gibba032.whopeo (mgr.24625) 4005 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:17.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:16 gibba032 bash[24004]: cluster 2021-03-10T15:08:15.710677+0000 mgr.gibba032.whopeo (mgr.24625) 4006 : cluster [DBG] pgmap v2273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:18 gibba032 bash[24004]: cluster 2021-03-10T15:08:17.712406+0000 mgr.gibba032.whopeo (mgr.24625) 4007 : cluster [DBG] pgmap v2274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:18 gibba011 bash[30791]: cluster 2021-03-10T15:08:17.712406+0000 mgr.gibba032.whopeo (mgr.24625) 4007 : cluster [DBG] pgmap v2274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:18 gibba011 bash[31774]: cluster 2021-03-10T15:08:17.712406+0000 mgr.gibba032.whopeo (mgr.24625) 4007 : cluster [DBG] pgmap v2274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:21.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:20 gibba011 bash[30791]: cluster 2021-03-10T15:08:19.713870+0000 mgr.gibba032.whopeo (mgr.24625) 4008 : cluster [DBG] pgmap v2275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:21.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:20 gibba011 bash[31774]: cluster 2021-03-10T15:08:19.713870+0000 mgr.gibba032.whopeo (mgr.24625) 4008 : cluster [DBG] pgmap v2275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:21.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:20 gibba032 bash[24004]: cluster 2021-03-10T15:08:19.713870+0000 mgr.gibba032.whopeo (mgr.24625) 4008 : cluster [DBG] pgmap v2275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:22 gibba011 bash[30791]: cluster 2021-03-10T15:08:21.715631+0000 mgr.gibba032.whopeo (mgr.24625) 4009 : cluster [DBG] pgmap v2276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:23.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:22 gibba011 bash[31774]: cluster 2021-03-10T15:08:21.715631+0000 mgr.gibba032.whopeo (mgr.24625) 4009 : cluster [DBG] pgmap v2276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:22 gibba032 bash[24004]: cluster 2021-03-10T15:08:21.715631+0000 mgr.gibba032.whopeo (mgr.24625) 4009 : cluster [DBG] pgmap v2276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:25.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:24 gibba011 bash[30791]: cluster 2021-03-10T15:08:23.717239+0000 mgr.gibba032.whopeo (mgr.24625) 4010 : cluster [DBG] pgmap v2277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:25.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:24 gibba011 bash[31774]: cluster 2021-03-10T15:08:23.717239+0000 mgr.gibba032.whopeo (mgr.24625) 4010 : cluster [DBG] pgmap v2277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:24 gibba032 bash[24004]: cluster 2021-03-10T15:08:23.717239+0000 mgr.gibba032.whopeo (mgr.24625) 4010 : cluster [DBG] pgmap v2277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:26 gibba011 bash[30791]: cluster 2021-03-10T15:08:25.718562+0000 mgr.gibba032.whopeo (mgr.24625) 4011 : cluster [DBG] pgmap v2278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:27.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:26 gibba011 bash[31774]: cluster 2021-03-10T15:08:25.718562+0000 mgr.gibba032.whopeo (mgr.24625) 4011 : cluster [DBG] pgmap v2278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:27.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:26 gibba032 bash[24004]: cluster 2021-03-10T15:08:25.718562+0000 mgr.gibba032.whopeo (mgr.24625) 4011 : cluster [DBG] pgmap v2278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:28 gibba032 bash[24004]: cluster 2021-03-10T15:08:27.720351+0000 mgr.gibba032.whopeo (mgr.24625) 4012 : cluster [DBG] pgmap v2279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:28 gibba011 bash[30791]: cluster 2021-03-10T15:08:27.720351+0000 mgr.gibba032.whopeo (mgr.24625) 4012 : cluster [DBG] pgmap v2279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:28 gibba011 bash[31774]: cluster 2021-03-10T15:08:27.720351+0000 mgr.gibba032.whopeo (mgr.24625) 4012 : cluster [DBG] pgmap v2279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:30 gibba011 bash[30791]: cluster 2021-03-10T15:08:29.721841+0000 mgr.gibba032.whopeo (mgr.24625) 4013 : cluster [DBG] pgmap v2280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:31.191 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:30 gibba011 bash[30791]: audit 2021-03-10T15:08:30.000222+0000 mon.a (mon.0) 1693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:08:31.192 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:30 gibba011 bash[31774]: cluster 2021-03-10T15:08:29.721841+0000 mgr.gibba032.whopeo (mgr.24625) 4013 : cluster [DBG] pgmap v2280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:31.192 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:30 gibba011 bash[31774]: audit 2021-03-10T15:08:30.000222+0000 mon.a (mon.0) 1693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:08:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:30 gibba032 bash[24004]: cluster 2021-03-10T15:08:29.721841+0000 mgr.gibba032.whopeo (mgr.24625) 4013 : cluster [DBG] pgmap v2280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:30 gibba032 bash[24004]: audit 2021-03-10T15:08:30.000222+0000 mon.a (mon.0) 1693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:08:32.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:31 gibba011 bash[30791]: cephadm 2021-03-10T15:08:29.997351+0000 mgr.gibba032.whopeo (mgr.24625) 4014 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:08:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:31 gibba011 bash[31774]: cephadm 2021-03-10T15:08:29.997351+0000 mgr.gibba032.whopeo (mgr.24625) 4014 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:08:32.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:31 gibba032 bash[24004]: cephadm 2021-03-10T15:08:29.997351+0000 mgr.gibba032.whopeo (mgr.24625) 4014 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:08:33.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:32 gibba011 bash[30791]: cluster 2021-03-10T15:08:31.723398+0000 mgr.gibba032.whopeo (mgr.24625) 4015 : cluster [DBG] pgmap v2281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:33.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:32 gibba011 bash[31774]: cluster 2021-03-10T15:08:31.723398+0000 mgr.gibba032.whopeo (mgr.24625) 4015 : cluster [DBG] pgmap v2281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:32 gibba032 bash[24004]: cluster 2021-03-10T15:08:31.723398+0000 mgr.gibba032.whopeo (mgr.24625) 4015 : cluster [DBG] pgmap v2281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:35.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.601456+0000 mgr.gibba032.whopeo (mgr.24625) 4016 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:08:35.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.603332+0000 mon.a (mon.0) 1694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cluster 2021-03-10T15:08:33.725083+0000 mgr.gibba032.whopeo (mgr.24625) 4017 : cluster [DBG] pgmap v2282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:35.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.812409+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.816642+0000 mgr.gibba032.whopeo (mgr.24625) 4018 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:08:35.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.819879+0000 mgr.gibba032.whopeo (mgr.24625) 4019 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:08:35.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.820153+0000 mon.a (mon.0) 1696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:08:35.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.822535+0000 mgr.gibba032.whopeo (mgr.24625) 4020 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:08:35.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.822593+0000 mgr.gibba032.whopeo (mgr.24625) 4021 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:08:35.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.823021+0000 mon.a (mon.0) 1697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.825543+0000 mgr.gibba032.whopeo (mgr.24625) 4022 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:08:35.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.825660+0000 mgr.gibba032.whopeo (mgr.24625) 4023 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:08:35.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.825759+0000 mon.a (mon.0) 1698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.828475+0000 mgr.gibba032.whopeo (mgr.24625) 4024 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:08:35.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.828901+0000 mon.a (mon.0) 1699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.830863+0000 mgr.gibba032.whopeo (mgr.24625) 4025 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:08:35.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.830929+0000 mgr.gibba032.whopeo (mgr.24625) 4026 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:08:35.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.833933+0000 mgr.gibba032.whopeo (mgr.24625) 4027 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:08:35.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.834171+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.834482+0000 mgr.gibba032.whopeo (mgr.24625) 4028 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:08:35.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.836906+0000 mgr.gibba032.whopeo (mgr.24625) 4029 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:08:35.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.836969+0000 mgr.gibba032.whopeo (mgr.24625) 4030 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:08:35.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: cephadm 2021-03-10T15:08:33.837103+0000 mgr.gibba032.whopeo (mgr.24625) 4031 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:35.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:34 gibba032 bash[24004]: audit 2021-03-10T15:08:33.837547+0000 mon.a (mon.0) 1701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.601456+0000 mgr.gibba032.whopeo (mgr.24625) 4016 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:08:35.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.603332+0000 mon.a (mon.0) 1694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cluster 2021-03-10T15:08:33.725083+0000 mgr.gibba032.whopeo (mgr.24625) 4017 : cluster [DBG] pgmap v2282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:35.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.812409+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.816642+0000 mgr.gibba032.whopeo (mgr.24625) 4018 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:08:35.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.819879+0000 mgr.gibba032.whopeo (mgr.24625) 4019 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:08:35.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.820153+0000 mon.a (mon.0) 1696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:08:35.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.822535+0000 mgr.gibba032.whopeo (mgr.24625) 4020 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:08:35.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.822593+0000 mgr.gibba032.whopeo (mgr.24625) 4021 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:08:35.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.823021+0000 mon.a (mon.0) 1697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.825543+0000 mgr.gibba032.whopeo (mgr.24625) 4022 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:08:35.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.825660+0000 mgr.gibba032.whopeo (mgr.24625) 4023 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:08:35.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.825759+0000 mon.a (mon.0) 1698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.828475+0000 mgr.gibba032.whopeo (mgr.24625) 4024 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:08:35.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.828901+0000 mon.a (mon.0) 1699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.830863+0000 mgr.gibba032.whopeo (mgr.24625) 4025 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:08:35.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.830929+0000 mgr.gibba032.whopeo (mgr.24625) 4026 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:08:35.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.833933+0000 mgr.gibba032.whopeo (mgr.24625) 4027 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:08:35.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.834171+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.834482+0000 mgr.gibba032.whopeo (mgr.24625) 4028 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:08:35.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.836906+0000 mgr.gibba032.whopeo (mgr.24625) 4029 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:08:35.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.601456+0000 mgr.gibba032.whopeo (mgr.24625) 4016 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:08:35.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.603332+0000 mon.a (mon.0) 1694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cluster 2021-03-10T15:08:33.725083+0000 mgr.gibba032.whopeo (mgr.24625) 4017 : cluster [DBG] pgmap v2282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:35.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.812409+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:08:35.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.816642+0000 mgr.gibba032.whopeo (mgr.24625) 4018 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:08:35.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.819879+0000 mgr.gibba032.whopeo (mgr.24625) 4019 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:08:35.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.820153+0000 mon.a (mon.0) 1696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:08:35.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.822535+0000 mgr.gibba032.whopeo (mgr.24625) 4020 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:08:35.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.822593+0000 mgr.gibba032.whopeo (mgr.24625) 4021 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:08:35.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.823021+0000 mon.a (mon.0) 1697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.825543+0000 mgr.gibba032.whopeo (mgr.24625) 4022 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:08:35.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.825660+0000 mgr.gibba032.whopeo (mgr.24625) 4023 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:08:35.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.825759+0000 mon.a (mon.0) 1698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.828475+0000 mgr.gibba032.whopeo (mgr.24625) 4024 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:08:35.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.828901+0000 mon.a (mon.0) 1699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.830863+0000 mgr.gibba032.whopeo (mgr.24625) 4025 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:08:35.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.830929+0000 mgr.gibba032.whopeo (mgr.24625) 4026 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:08:35.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.833933+0000 mgr.gibba032.whopeo (mgr.24625) 4027 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:08:35.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.834171+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.834482+0000 mgr.gibba032.whopeo (mgr.24625) 4028 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:08:35.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.836906+0000 mgr.gibba032.whopeo (mgr.24625) 4029 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:08:35.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.836969+0000 mgr.gibba032.whopeo (mgr.24625) 4030 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:08:35.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: cephadm 2021-03-10T15:08:33.837103+0000 mgr.gibba032.whopeo (mgr.24625) 4031 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:35.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[31774]: audit 2021-03-10T15:08:33.837547+0000 mon.a (mon.0) 1701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:35.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.836969+0000 mgr.gibba032.whopeo (mgr.24625) 4030 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:08:35.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: cephadm 2021-03-10T15:08:33.837103+0000 mgr.gibba032.whopeo (mgr.24625) 4031 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:35.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:34 gibba011 bash[30791]: audit 2021-03-10T15:08:33.837547+0000 mon.a (mon.0) 1701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:37.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:36 gibba011 bash[30791]: cluster 2021-03-10T15:08:35.725836+0000 mgr.gibba032.whopeo (mgr.24625) 4032 : cluster [DBG] pgmap v2283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:37.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:36 gibba011 bash[31774]: cluster 2021-03-10T15:08:35.725836+0000 mgr.gibba032.whopeo (mgr.24625) 4032 : cluster [DBG] pgmap v2283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:37.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:36 gibba032 bash[24004]: cluster 2021-03-10T15:08:35.725836+0000 mgr.gibba032.whopeo (mgr.24625) 4032 : cluster [DBG] pgmap v2283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:38 gibba032 bash[24004]: cluster 2021-03-10T15:08:37.727390+0000 mgr.gibba032.whopeo (mgr.24625) 4033 : cluster [DBG] pgmap v2284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:38 gibba011 bash[30791]: cluster 2021-03-10T15:08:37.727390+0000 mgr.gibba032.whopeo (mgr.24625) 4033 : cluster [DBG] pgmap v2284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:38 gibba011 bash[31774]: cluster 2021-03-10T15:08:37.727390+0000 mgr.gibba032.whopeo (mgr.24625) 4033 : cluster [DBG] pgmap v2284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:40.507 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:08:40.747 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:08:40.747 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (75m) 7s ago 80m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:08:40.747 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (80m) 9m ago 80m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:08:40.748 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (75m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:08:40.748 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (75m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:08:40.748 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (75m) 7s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:08:40.748 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (74m) 7s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:08:40.749 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (74m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:08:40.749 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (74m) 7s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:08:40.749 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (80m) 7s ago 80m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:08:40.749 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (80m) 9m ago 80m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:08:40.750 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (73m) 7s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:08:40.750 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (73m) 7s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:08:40.750 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (72m) 7s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:08:40.750 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (72m) 7s ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:08:40.750 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (71m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:08:40.751 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (71m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:08:40.751 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (71m) 9m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:08:40.751 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (70m) 9m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:08:40.751 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (80m) 9m ago 80m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:08:40.751 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7s ago 78m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:08:41.002 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:08:41.002 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:08:41.002 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:08:41.002 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:41.003 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:08:41.003 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:08:41.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:41.003 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:08:41.003 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:08:41.004 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:08:41.004 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:08:41.004 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:08:41.004 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:08:41.004 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:08:41.005 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:08:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:40 gibba011 bash[30791]: cluster 2021-03-10T15:08:39.728818+0000 mgr.gibba032.whopeo (mgr.24625) 4034 : cluster [DBG] pgmap v2285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:41.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:40 gibba011 bash[31774]: cluster 2021-03-10T15:08:39.728818+0000 mgr.gibba032.whopeo (mgr.24625) 4034 : cluster [DBG] pgmap v2285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:40 gibba032 bash[24004]: cluster 2021-03-10T15:08:39.728818+0000 mgr.gibba032.whopeo (mgr.24625) 4034 : cluster [DBG] pgmap v2285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[30791]: audit 2021-03-10T15:08:40.494186+0000 mgr.gibba032.whopeo (mgr.24625) 4035 : audit [DBG] from='client.46755 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[30791]: audit 2021-03-10T15:08:40.720900+0000 mgr.gibba032.whopeo (mgr.24625) 4036 : audit [DBG] from='client.46761 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[30791]: audit 2021-03-10T15:08:40.999245+0000 mon.c (mon.1) 78 : audit [DBG] from='client.? 172.21.2.111:0/2612903688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[31774]: audit 2021-03-10T15:08:40.494186+0000 mgr.gibba032.whopeo (mgr.24625) 4035 : audit [DBG] from='client.46755 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[31774]: audit 2021-03-10T15:08:40.720900+0000 mgr.gibba032.whopeo (mgr.24625) 4036 : audit [DBG] from='client.46761 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:41 gibba011 bash[31774]: audit 2021-03-10T15:08:40.999245+0000 mon.c (mon.1) 78 : audit [DBG] from='client.? 172.21.2.111:0/2612903688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:41 gibba032 bash[24004]: audit 2021-03-10T15:08:40.494186+0000 mgr.gibba032.whopeo (mgr.24625) 4035 : audit [DBG] from='client.46755 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:41 gibba032 bash[24004]: audit 2021-03-10T15:08:40.720900+0000 mgr.gibba032.whopeo (mgr.24625) 4036 : audit [DBG] from='client.46761 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:08:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:41 gibba032 bash[24004]: audit 2021-03-10T15:08:40.999245+0000 mon.c (mon.1) 78 : audit [DBG] from='client.? 172.21.2.111:0/2612903688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:08:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:42 gibba011 bash[30791]: cluster 2021-03-10T15:08:41.730575+0000 mgr.gibba032.whopeo (mgr.24625) 4037 : cluster [DBG] pgmap v2286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:43.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:42 gibba011 bash[31774]: cluster 2021-03-10T15:08:41.730575+0000 mgr.gibba032.whopeo (mgr.24625) 4037 : cluster [DBG] pgmap v2286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:42 gibba032 bash[24004]: cluster 2021-03-10T15:08:41.730575+0000 mgr.gibba032.whopeo (mgr.24625) 4037 : cluster [DBG] pgmap v2286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:45.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:44 gibba011 bash[30791]: cluster 2021-03-10T15:08:43.732201+0000 mgr.gibba032.whopeo (mgr.24625) 4038 : cluster [DBG] pgmap v2287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:45.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:44 gibba011 bash[31774]: cluster 2021-03-10T15:08:43.732201+0000 mgr.gibba032.whopeo (mgr.24625) 4038 : cluster [DBG] pgmap v2287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:44 gibba032 bash[24004]: cluster 2021-03-10T15:08:43.732201+0000 mgr.gibba032.whopeo (mgr.24625) 4038 : cluster [DBG] pgmap v2287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:46 gibba011 bash[30791]: cluster 2021-03-10T15:08:45.733538+0000 mgr.gibba032.whopeo (mgr.24625) 4039 : cluster [DBG] pgmap v2288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:46 gibba011 bash[31774]: cluster 2021-03-10T15:08:45.733538+0000 mgr.gibba032.whopeo (mgr.24625) 4039 : cluster [DBG] pgmap v2288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:46 gibba032 bash[24004]: cluster 2021-03-10T15:08:45.733538+0000 mgr.gibba032.whopeo (mgr.24625) 4039 : cluster [DBG] pgmap v2288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:48 gibba032 bash[24004]: cluster 2021-03-10T15:08:47.735096+0000 mgr.gibba032.whopeo (mgr.24625) 4040 : cluster [DBG] pgmap v2289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:48 gibba011 bash[30791]: cluster 2021-03-10T15:08:47.735096+0000 mgr.gibba032.whopeo (mgr.24625) 4040 : cluster [DBG] pgmap v2289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:48 gibba011 bash[31774]: cluster 2021-03-10T15:08:47.735096+0000 mgr.gibba032.whopeo (mgr.24625) 4040 : cluster [DBG] pgmap v2289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:49 gibba032 bash[24004]: cephadm 2021-03-10T15:08:48.853065+0000 mgr.gibba032.whopeo (mgr.24625) 4041 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:49 gibba011 bash[30791]: cephadm 2021-03-10T15:08:48.853065+0000 mgr.gibba032.whopeo (mgr.24625) 4041 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:49 gibba011 bash[31774]: cephadm 2021-03-10T15:08:48.853065+0000 mgr.gibba032.whopeo (mgr.24625) 4041 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:08:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:50 gibba032 bash[24004]: cluster 2021-03-10T15:08:49.736667+0000 mgr.gibba032.whopeo (mgr.24625) 4042 : cluster [DBG] pgmap v2290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:50 gibba011 bash[30791]: cluster 2021-03-10T15:08:49.736667+0000 mgr.gibba032.whopeo (mgr.24625) 4042 : cluster [DBG] pgmap v2290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:50 gibba011 bash[31774]: cluster 2021-03-10T15:08:49.736667+0000 mgr.gibba032.whopeo (mgr.24625) 4042 : cluster [DBG] pgmap v2290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:52 gibba011 bash[30791]: cluster 2021-03-10T15:08:51.738157+0000 mgr.gibba032.whopeo (mgr.24625) 4043 : cluster [DBG] pgmap v2291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:53.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:52 gibba011 bash[31774]: cluster 2021-03-10T15:08:51.738157+0000 mgr.gibba032.whopeo (mgr.24625) 4043 : cluster [DBG] pgmap v2291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:52 gibba032 bash[24004]: cluster 2021-03-10T15:08:51.738157+0000 mgr.gibba032.whopeo (mgr.24625) 4043 : cluster [DBG] pgmap v2291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:54 gibba011 bash[30791]: cluster 2021-03-10T15:08:53.739898+0000 mgr.gibba032.whopeo (mgr.24625) 4044 : cluster [DBG] pgmap v2292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:55.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:54 gibba011 bash[31774]: cluster 2021-03-10T15:08:53.739898+0000 mgr.gibba032.whopeo (mgr.24625) 4044 : cluster [DBG] pgmap v2292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:54 gibba032 bash[24004]: cluster 2021-03-10T15:08:53.739898+0000 mgr.gibba032.whopeo (mgr.24625) 4044 : cluster [DBG] pgmap v2292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: cluster 2021-03-10T15:08:55.741218+0000 mgr.gibba032.whopeo (mgr.24625) 4045 : cluster [DBG] pgmap v2293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.495896+0000 mon.a (mon.0) 1702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:08:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.496040+0000 mon.a (mon.0) 1703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:08:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.509557+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:08:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.509743+0000 mon.a (mon.0) 1705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:08:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.509908+0000 mon.a (mon.0) 1706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:08:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[30791]: audit 2021-03-10T15:08:56.510072+0000 mon.a (mon.0) 1707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:08:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: cluster 2021-03-10T15:08:55.741218+0000 mgr.gibba032.whopeo (mgr.24625) 4045 : cluster [DBG] pgmap v2293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.495896+0000 mon.a (mon.0) 1702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:08:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.496040+0000 mon.a (mon.0) 1703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:08:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.509557+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:08:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.509743+0000 mon.a (mon.0) 1705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:08:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.509908+0000 mon.a (mon.0) 1706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:08:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:56 gibba011 bash[31774]: audit 2021-03-10T15:08:56.510072+0000 mon.a (mon.0) 1707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:08:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: cluster 2021-03-10T15:08:55.741218+0000 mgr.gibba032.whopeo (mgr.24625) 4045 : cluster [DBG] pgmap v2293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.495896+0000 mon.a (mon.0) 1702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:08:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.496040+0000 mon.a (mon.0) 1703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:08:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.509557+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:08:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.509743+0000 mon.a (mon.0) 1705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:08:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.509908+0000 mon.a (mon.0) 1706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:08:57.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:56 gibba032 bash[24004]: audit 2021-03-10T15:08:56.510072+0000 mon.a (mon.0) 1707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:08:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:08:58 gibba032 bash[24004]: cluster 2021-03-10T15:08:57.742978+0000 mgr.gibba032.whopeo (mgr.24625) 4046 : cluster [DBG] pgmap v2294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:08:58 gibba011 bash[30791]: cluster 2021-03-10T15:08:57.742978+0000 mgr.gibba032.whopeo (mgr.24625) 4046 : cluster [DBG] pgmap v2294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:08:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:08:58 gibba011 bash[31774]: cluster 2021-03-10T15:08:57.742978+0000 mgr.gibba032.whopeo (mgr.24625) 4046 : cluster [DBG] pgmap v2294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:01.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:00 gibba011 bash[30791]: cluster 2021-03-10T15:08:59.744572+0000 mgr.gibba032.whopeo (mgr.24625) 4047 : cluster [DBG] pgmap v2295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:01.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:00 gibba011 bash[31774]: cluster 2021-03-10T15:08:59.744572+0000 mgr.gibba032.whopeo (mgr.24625) 4047 : cluster [DBG] pgmap v2295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:00 gibba032 bash[24004]: cluster 2021-03-10T15:08:59.744572+0000 mgr.gibba032.whopeo (mgr.24625) 4047 : cluster [DBG] pgmap v2295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:02 gibba011 bash[30791]: cluster 2021-03-10T15:09:01.746336+0000 mgr.gibba032.whopeo (mgr.24625) 4048 : cluster [DBG] pgmap v2296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:03.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:02 gibba011 bash[31774]: cluster 2021-03-10T15:09:01.746336+0000 mgr.gibba032.whopeo (mgr.24625) 4048 : cluster [DBG] pgmap v2296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:02 gibba032 bash[24004]: cluster 2021-03-10T15:09:01.746336+0000 mgr.gibba032.whopeo (mgr.24625) 4048 : cluster [DBG] pgmap v2296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:05.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:04 gibba011 bash[30791]: cluster 2021-03-10T15:09:03.747963+0000 mgr.gibba032.whopeo (mgr.24625) 4049 : cluster [DBG] pgmap v2297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:05.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:04 gibba011 bash[30791]: cephadm 2021-03-10T15:09:03.868670+0000 mgr.gibba032.whopeo (mgr.24625) 4050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:05.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:04 gibba011 bash[31774]: cluster 2021-03-10T15:09:03.747963+0000 mgr.gibba032.whopeo (mgr.24625) 4049 : cluster [DBG] pgmap v2297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:05.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:04 gibba011 bash[31774]: cephadm 2021-03-10T15:09:03.868670+0000 mgr.gibba032.whopeo (mgr.24625) 4050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:04 gibba032 bash[24004]: cluster 2021-03-10T15:09:03.747963+0000 mgr.gibba032.whopeo (mgr.24625) 4049 : cluster [DBG] pgmap v2297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:04 gibba032 bash[24004]: cephadm 2021-03-10T15:09:03.868670+0000 mgr.gibba032.whopeo (mgr.24625) 4050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:07.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:06 gibba032 bash[24004]: cluster 2021-03-10T15:09:05.749325+0000 mgr.gibba032.whopeo (mgr.24625) 4051 : cluster [DBG] pgmap v2298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:06 gibba011 bash[30791]: cluster 2021-03-10T15:09:05.749325+0000 mgr.gibba032.whopeo (mgr.24625) 4051 : cluster [DBG] pgmap v2298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:06 gibba011 bash[31774]: cluster 2021-03-10T15:09:05.749325+0000 mgr.gibba032.whopeo (mgr.24625) 4051 : cluster [DBG] pgmap v2298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:08 gibba032 bash[24004]: cluster 2021-03-10T15:09:07.751093+0000 mgr.gibba032.whopeo (mgr.24625) 4052 : cluster [DBG] pgmap v2299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:08 gibba011 bash[30791]: cluster 2021-03-10T15:09:07.751093+0000 mgr.gibba032.whopeo (mgr.24625) 4052 : cluster [DBG] pgmap v2299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:08 gibba011 bash[31774]: cluster 2021-03-10T15:09:07.751093+0000 mgr.gibba032.whopeo (mgr.24625) 4052 : cluster [DBG] pgmap v2299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:11.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:10 gibba011 bash[30791]: cluster 2021-03-10T15:09:09.752601+0000 mgr.gibba032.whopeo (mgr.24625) 4053 : cluster [DBG] pgmap v2300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:11.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:10 gibba011 bash[31774]: cluster 2021-03-10T15:09:09.752601+0000 mgr.gibba032.whopeo (mgr.24625) 4053 : cluster [DBG] pgmap v2300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:11.245 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:09:11.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:10 gibba032 bash[24004]: cluster 2021-03-10T15:09:09.752601+0000 mgr.gibba032.whopeo (mgr.24625) 4053 : cluster [DBG] pgmap v2300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:11.475 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:09:11.475 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (75m) 37s ago 81m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:09:11.475 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (81m) 9m ago 81m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:09:11.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (76m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:09:11.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (76m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:09:11.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (75m) 37s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:09:11.476 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (75m) 37s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:09:11.476 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (74m) 9m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:09:11.477 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (75m) 37s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:09:11.477 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (81m) 37s ago 81m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:09:11.477 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (81m) 9m ago 81m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:09:11.477 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (74m) 37s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:09:11.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (73m) 37s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:09:11.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (73m) 37s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:09:11.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (72m) 37s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:09:11.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (72m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:09:11.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (72m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:09:11.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (71m) 9m ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:09:11.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (71m) 9m ago 81m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:09:11.479 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (80m) 9m ago 81m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:09:11.479 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 37s ago 79m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:09:11.725 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:09:11.725 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:09:11.725 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:09:11.725 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:11.726 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:09:11.726 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:09:11.726 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:11.726 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:09:11.727 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:09:11.727 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:11.727 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:09:11.727 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:09:11.728 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:09:11.728 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:09:11.728 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:09:12.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:11 gibba011 bash[30791]: audit 2021-03-10T15:09:11.722188+0000 mon.c (mon.1) 79 : audit [DBG] from='client.? 172.21.2.111:0/1679946249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:12.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:11 gibba011 bash[31774]: audit 2021-03-10T15:09:11.722188+0000 mon.c (mon.1) 79 : audit [DBG] from='client.? 172.21.2.111:0/1679946249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:11 gibba032 bash[24004]: audit 2021-03-10T15:09:11.722188+0000 mon.c (mon.1) 79 : audit [DBG] from='client.? 172.21.2.111:0/1679946249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:12 gibba032 bash[24004]: audit 2021-03-10T15:09:11.230968+0000 mgr.gibba032.whopeo (mgr.24625) 4054 : audit [DBG] from='client.46773 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:12 gibba032 bash[24004]: audit 2021-03-10T15:09:11.457499+0000 mgr.gibba032.whopeo (mgr.24625) 4055 : audit [DBG] from='client.46779 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:12 gibba032 bash[24004]: cluster 2021-03-10T15:09:11.754266+0000 mgr.gibba032.whopeo (mgr.24625) 4056 : cluster [DBG] pgmap v2301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[30791]: audit 2021-03-10T15:09:11.230968+0000 mgr.gibba032.whopeo (mgr.24625) 4054 : audit [DBG] from='client.46773 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[30791]: audit 2021-03-10T15:09:11.457499+0000 mgr.gibba032.whopeo (mgr.24625) 4055 : audit [DBG] from='client.46779 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[30791]: cluster 2021-03-10T15:09:11.754266+0000 mgr.gibba032.whopeo (mgr.24625) 4056 : cluster [DBG] pgmap v2301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[31774]: audit 2021-03-10T15:09:11.230968+0000 mgr.gibba032.whopeo (mgr.24625) 4054 : audit [DBG] from='client.46773 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[31774]: audit 2021-03-10T15:09:11.457499+0000 mgr.gibba032.whopeo (mgr.24625) 4055 : audit [DBG] from='client.46779 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:13.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:12 gibba011 bash[31774]: cluster 2021-03-10T15:09:11.754266+0000 mgr.gibba032.whopeo (mgr.24625) 4056 : cluster [DBG] pgmap v2301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:15.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:14 gibba032 bash[24004]: cluster 2021-03-10T15:09:13.755882+0000 mgr.gibba032.whopeo (mgr.24625) 4057 : cluster [DBG] pgmap v2302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:14 gibba011 bash[30791]: cluster 2021-03-10T15:09:13.755882+0000 mgr.gibba032.whopeo (mgr.24625) 4057 : cluster [DBG] pgmap v2302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:14 gibba011 bash[31774]: cluster 2021-03-10T15:09:13.755882+0000 mgr.gibba032.whopeo (mgr.24625) 4057 : cluster [DBG] pgmap v2302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:17.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:16 gibba032 bash[24004]: cluster 2021-03-10T15:09:15.757335+0000 mgr.gibba032.whopeo (mgr.24625) 4058 : cluster [DBG] pgmap v2303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:16 gibba011 bash[30791]: cluster 2021-03-10T15:09:15.757335+0000 mgr.gibba032.whopeo (mgr.24625) 4058 : cluster [DBG] pgmap v2303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:17.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:16 gibba011 bash[31774]: cluster 2021-03-10T15:09:15.757335+0000 mgr.gibba032.whopeo (mgr.24625) 4058 : cluster [DBG] pgmap v2303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:18 gibba032 bash[24004]: cluster 2021-03-10T15:09:17.759106+0000 mgr.gibba032.whopeo (mgr.24625) 4059 : cluster [DBG] pgmap v2304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:18 gibba011 bash[30791]: cluster 2021-03-10T15:09:17.759106+0000 mgr.gibba032.whopeo (mgr.24625) 4059 : cluster [DBG] pgmap v2304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:18 gibba011 bash[31774]: cluster 2021-03-10T15:09:17.759106+0000 mgr.gibba032.whopeo (mgr.24625) 4059 : cluster [DBG] pgmap v2304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:19 gibba032 bash[24004]: cephadm 2021-03-10T15:09:18.884492+0000 mgr.gibba032.whopeo (mgr.24625) 4060 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:19.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:19 gibba011 bash[30791]: cephadm 2021-03-10T15:09:18.884492+0000 mgr.gibba032.whopeo (mgr.24625) 4060 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:19.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:19 gibba011 bash[31774]: cephadm 2021-03-10T15:09:18.884492+0000 mgr.gibba032.whopeo (mgr.24625) 4060 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:20 gibba032 bash[24004]: cluster 2021-03-10T15:09:19.760557+0000 mgr.gibba032.whopeo (mgr.24625) 4061 : cluster [DBG] pgmap v2305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:20 gibba011 bash[30791]: cluster 2021-03-10T15:09:19.760557+0000 mgr.gibba032.whopeo (mgr.24625) 4061 : cluster [DBG] pgmap v2305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:20 gibba011 bash[31774]: cluster 2021-03-10T15:09:19.760557+0000 mgr.gibba032.whopeo (mgr.24625) 4061 : cluster [DBG] pgmap v2305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:23.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:22 gibba011 bash[30791]: cluster 2021-03-10T15:09:21.762297+0000 mgr.gibba032.whopeo (mgr.24625) 4062 : cluster [DBG] pgmap v2306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:23.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:22 gibba011 bash[31774]: cluster 2021-03-10T15:09:21.762297+0000 mgr.gibba032.whopeo (mgr.24625) 4062 : cluster [DBG] pgmap v2306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:22 gibba032 bash[24004]: cluster 2021-03-10T15:09:21.762297+0000 mgr.gibba032.whopeo (mgr.24625) 4062 : cluster [DBG] pgmap v2306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:24 gibba032 bash[24004]: cluster 2021-03-10T15:09:23.763925+0000 mgr.gibba032.whopeo (mgr.24625) 4063 : cluster [DBG] pgmap v2307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:24 gibba011 bash[30791]: cluster 2021-03-10T15:09:23.763925+0000 mgr.gibba032.whopeo (mgr.24625) 4063 : cluster [DBG] pgmap v2307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:25.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:24 gibba011 bash[31774]: cluster 2021-03-10T15:09:23.763925+0000 mgr.gibba032.whopeo (mgr.24625) 4063 : cluster [DBG] pgmap v2307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:27.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:26 gibba032 bash[24004]: cluster 2021-03-10T15:09:25.765250+0000 mgr.gibba032.whopeo (mgr.24625) 4064 : cluster [DBG] pgmap v2308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:27.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:26 gibba011 bash[30791]: cluster 2021-03-10T15:09:25.765250+0000 mgr.gibba032.whopeo (mgr.24625) 4064 : cluster [DBG] pgmap v2308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:27.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:26 gibba011 bash[31774]: cluster 2021-03-10T15:09:25.765250+0000 mgr.gibba032.whopeo (mgr.24625) 4064 : cluster [DBG] pgmap v2308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:28 gibba032 bash[24004]: cluster 2021-03-10T15:09:27.767026+0000 mgr.gibba032.whopeo (mgr.24625) 4065 : cluster [DBG] pgmap v2309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:28 gibba011 bash[30791]: cluster 2021-03-10T15:09:27.767026+0000 mgr.gibba032.whopeo (mgr.24625) 4065 : cluster [DBG] pgmap v2309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:28 gibba011 bash[31774]: cluster 2021-03-10T15:09:27.767026+0000 mgr.gibba032.whopeo (mgr.24625) 4065 : cluster [DBG] pgmap v2309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:30 gibba032 bash[24004]: cluster 2021-03-10T15:09:29.768651+0000 mgr.gibba032.whopeo (mgr.24625) 4066 : cluster [DBG] pgmap v2310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:30 gibba011 bash[30791]: cluster 2021-03-10T15:09:29.768651+0000 mgr.gibba032.whopeo (mgr.24625) 4066 : cluster [DBG] pgmap v2310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:30 gibba011 bash[31774]: cluster 2021-03-10T15:09:29.768651+0000 mgr.gibba032.whopeo (mgr.24625) 4066 : cluster [DBG] pgmap v2310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:32 gibba032 bash[24004]: cluster 2021-03-10T15:09:31.770325+0000 mgr.gibba032.whopeo (mgr.24625) 4067 : cluster [DBG] pgmap v2311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:32 gibba011 bash[30791]: cluster 2021-03-10T15:09:31.770325+0000 mgr.gibba032.whopeo (mgr.24625) 4067 : cluster [DBG] pgmap v2311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:32 gibba011 bash[31774]: cluster 2021-03-10T15:09:31.770325+0000 mgr.gibba032.whopeo (mgr.24625) 4067 : cluster [DBG] pgmap v2311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:35.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:34 gibba032 bash[24004]: cluster 2021-03-10T15:09:33.771798+0000 mgr.gibba032.whopeo (mgr.24625) 4068 : cluster [DBG] pgmap v2312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:35.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:34 gibba032 bash[24004]: cephadm 2021-03-10T15:09:33.901276+0000 mgr.gibba032.whopeo (mgr.24625) 4069 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:09:35.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:34 gibba032 bash[24004]: audit 2021-03-10T15:09:33.905270+0000 mon.a (mon.0) 1708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:09:35.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[30791]: cluster 2021-03-10T15:09:33.771798+0000 mgr.gibba032.whopeo (mgr.24625) 4068 : cluster [DBG] pgmap v2312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:35.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[30791]: cephadm 2021-03-10T15:09:33.901276+0000 mgr.gibba032.whopeo (mgr.24625) 4069 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:09:35.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[30791]: audit 2021-03-10T15:09:33.905270+0000 mon.a (mon.0) 1708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:09:35.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[31774]: cluster 2021-03-10T15:09:33.771798+0000 mgr.gibba032.whopeo (mgr.24625) 4068 : cluster [DBG] pgmap v2312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:35.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[31774]: cephadm 2021-03-10T15:09:33.901276+0000 mgr.gibba032.whopeo (mgr.24625) 4069 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:09:35.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:34 gibba011 bash[31774]: audit 2021-03-10T15:09:33.905270+0000 mon.a (mon.0) 1708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:09:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:36 gibba032 bash[24004]: cluster 2021-03-10T15:09:35.773358+0000 mgr.gibba032.whopeo (mgr.24625) 4070 : cluster [DBG] pgmap v2313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:36 gibba011 bash[30791]: cluster 2021-03-10T15:09:35.773358+0000 mgr.gibba032.whopeo (mgr.24625) 4070 : cluster [DBG] pgmap v2313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:36 gibba011 bash[31774]: cluster 2021-03-10T15:09:35.773358+0000 mgr.gibba032.whopeo (mgr.24625) 4070 : cluster [DBG] pgmap v2313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.572983+0000 mgr.gibba032.whopeo (mgr.24625) 4071 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:09:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.574691+0000 mon.a (mon.0) 1709 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:38.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cluster 2021-03-10T15:09:37.774025+0000 mgr.gibba032.whopeo (mgr.24625) 4072 : cluster [DBG] pgmap v2314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.789628+0000 mon.a (mon.0) 1710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.790110+0000 mgr.gibba032.whopeo (mgr.24625) 4073 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:09:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.793166+0000 mgr.gibba032.whopeo (mgr.24625) 4074 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:09:38.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.793361+0000 mon.a (mon.0) 1711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:09:38.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.796068+0000 mon.a (mon.0) 1712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.796694+0000 mgr.gibba032.whopeo (mgr.24625) 4075 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:09:38.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.797204+0000 mgr.gibba032.whopeo (mgr.24625) 4076 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:09:38.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.800863+0000 mon.a (mon.0) 1713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.801336+0000 mgr.gibba032.whopeo (mgr.24625) 4077 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:09:38.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.801553+0000 mgr.gibba032.whopeo (mgr.24625) 4078 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:09:38.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.804660+0000 mon.a (mon.0) 1714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.805030+0000 mgr.gibba032.whopeo (mgr.24625) 4079 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:09:38.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.808853+0000 mgr.gibba032.whopeo (mgr.24625) 4080 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:09:38.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.809126+0000 mgr.gibba032.whopeo (mgr.24625) 4081 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:09:38.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.813425+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.814203+0000 mgr.gibba032.whopeo (mgr.24625) 4082 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:09:38.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.816249+0000 mgr.gibba032.whopeo (mgr.24625) 4083 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:09:38.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.572983+0000 mgr.gibba032.whopeo (mgr.24625) 4071 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:09:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.574691+0000 mon.a (mon.0) 1709 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cluster 2021-03-10T15:09:37.774025+0000 mgr.gibba032.whopeo (mgr.24625) 4072 : cluster [DBG] pgmap v2314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.789628+0000 mon.a (mon.0) 1710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:38.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.790110+0000 mgr.gibba032.whopeo (mgr.24625) 4073 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:09:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.793166+0000 mgr.gibba032.whopeo (mgr.24625) 4074 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:09:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.793361+0000 mon.a (mon.0) 1711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:09:38.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.796068+0000 mon.a (mon.0) 1712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.796694+0000 mgr.gibba032.whopeo (mgr.24625) 4075 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:09:38.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.797204+0000 mgr.gibba032.whopeo (mgr.24625) 4076 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:09:38.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.800863+0000 mon.a (mon.0) 1713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.801336+0000 mgr.gibba032.whopeo (mgr.24625) 4077 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:09:38.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.801553+0000 mgr.gibba032.whopeo (mgr.24625) 4078 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:09:38.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.804660+0000 mon.a (mon.0) 1714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.805030+0000 mgr.gibba032.whopeo (mgr.24625) 4079 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:09:38.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.808853+0000 mgr.gibba032.whopeo (mgr.24625) 4080 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:09:38.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.809126+0000 mgr.gibba032.whopeo (mgr.24625) 4081 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:09:38.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.813425+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.814203+0000 mgr.gibba032.whopeo (mgr.24625) 4082 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:09:38.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.816249+0000 mgr.gibba032.whopeo (mgr.24625) 4083 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:09:38.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: audit 2021-03-10T15:09:37.820133+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.844 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.820672+0000 mgr.gibba032.whopeo (mgr.24625) 4084 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:09:38.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.820909+0000 mgr.gibba032.whopeo (mgr.24625) 4085 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:09:38.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[30791]: cephadm 2021-03-10T15:09:37.821506+0000 mgr.gibba032.whopeo (mgr.24625) 4086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:38.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: audit 2021-03-10T15:09:37.820133+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:38.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.820672+0000 mgr.gibba032.whopeo (mgr.24625) 4084 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:09:38.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.820909+0000 mgr.gibba032.whopeo (mgr.24625) 4085 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:09:38.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:38 gibba011 bash[31774]: cephadm 2021-03-10T15:09:37.821506+0000 mgr.gibba032.whopeo (mgr.24625) 4086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.572983+0000 mgr.gibba032.whopeo (mgr.24625) 4071 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:09:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.574691+0000 mon.a (mon.0) 1709 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:39.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cluster 2021-03-10T15:09:37.774025+0000 mgr.gibba032.whopeo (mgr.24625) 4072 : cluster [DBG] pgmap v2314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.789628+0000 mon.a (mon.0) 1710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:09:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.790110+0000 mgr.gibba032.whopeo (mgr.24625) 4073 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:09:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.793166+0000 mgr.gibba032.whopeo (mgr.24625) 4074 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:09:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.793361+0000 mon.a (mon.0) 1711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:09:39.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.796068+0000 mon.a (mon.0) 1712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.796694+0000 mgr.gibba032.whopeo (mgr.24625) 4075 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:09:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.797204+0000 mgr.gibba032.whopeo (mgr.24625) 4076 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:09:39.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.800863+0000 mon.a (mon.0) 1713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:39.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.801336+0000 mgr.gibba032.whopeo (mgr.24625) 4077 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:09:39.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.801553+0000 mgr.gibba032.whopeo (mgr.24625) 4078 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:09:39.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.804660+0000 mon.a (mon.0) 1714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:39.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.805030+0000 mgr.gibba032.whopeo (mgr.24625) 4079 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:09:39.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.808853+0000 mgr.gibba032.whopeo (mgr.24625) 4080 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:09:39.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.809126+0000 mgr.gibba032.whopeo (mgr.24625) 4081 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:09:39.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.813425+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:39.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.814203+0000 mgr.gibba032.whopeo (mgr.24625) 4082 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:09:39.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.816249+0000 mgr.gibba032.whopeo (mgr.24625) 4083 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:09:39.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: audit 2021-03-10T15:09:37.820133+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:39.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.820672+0000 mgr.gibba032.whopeo (mgr.24625) 4084 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:09:39.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.820909+0000 mgr.gibba032.whopeo (mgr.24625) 4085 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:09:39.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:38 gibba032 bash[24004]: cephadm 2021-03-10T15:09:37.821506+0000 mgr.gibba032.whopeo (mgr.24625) 4086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:40 gibba032 bash[24004]: cluster 2021-03-10T15:09:39.775470+0000 mgr.gibba032.whopeo (mgr.24625) 4087 : cluster [DBG] pgmap v2315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:40 gibba011 bash[30791]: cluster 2021-03-10T15:09:39.775470+0000 mgr.gibba032.whopeo (mgr.24625) 4087 : cluster [DBG] pgmap v2315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:40 gibba011 bash[31774]: cluster 2021-03-10T15:09:39.775470+0000 mgr.gibba032.whopeo (mgr.24625) 4087 : cluster [DBG] pgmap v2315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:41.967 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:09:42.194 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:09:42.194 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (76m) 68s ago 81m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:09:42.195 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (81m) 4s ago 81m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:09:42.195 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (76m) 4s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:09:42.195 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (76m) 4s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:09:42.195 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (76m) 68s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:09:42.196 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (75m) 68s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:09:42.196 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (75m) 4s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:09:42.196 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (75m) 68s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:09:42.196 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (81m) 68s ago 81m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:09:42.196 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (81m) 4s ago 81m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:09:42.197 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (74m) 68s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:09:42.197 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (74m) 68s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:09:42.197 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (73m) 68s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:09:42.197 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (73m) 68s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:09:42.198 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (72m) 4s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:09:42.198 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (72m) 4s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:09:42.198 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (72m) 4s ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:09:42.198 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (71m) 4s ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:09:42.198 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (81m) 4s ago 81m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:09:42.199 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 68s ago 79m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:09:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:09:42.443 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:09:42.443 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:09:42.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:42.443 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:09:42.444 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:09:42.444 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:42.444 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:09:42.444 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:09:42.445 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:09:42.445 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:09:42.445 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:09:42.445 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:09:42.446 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:09:42.446 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:09:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:42 gibba032 bash[24004]: cluster 2021-03-10T15:09:41.777211+0000 mgr.gibba032.whopeo (mgr.24625) 4088 : cluster [DBG] pgmap v2316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:42 gibba032 bash[24004]: audit 2021-03-10T15:09:41.952871+0000 mgr.gibba032.whopeo (mgr.24625) 4089 : audit [DBG] from='client.46791 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:42 gibba032 bash[24004]: audit 2021-03-10T15:09:42.439857+0000 mon.a (mon.0) 1717 : audit [DBG] from='client.? 172.21.2.111:0/3064731760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[30791]: cluster 2021-03-10T15:09:41.777211+0000 mgr.gibba032.whopeo (mgr.24625) 4088 : cluster [DBG] pgmap v2316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[30791]: audit 2021-03-10T15:09:41.952871+0000 mgr.gibba032.whopeo (mgr.24625) 4089 : audit [DBG] from='client.46791 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[30791]: audit 2021-03-10T15:09:42.439857+0000 mon.a (mon.0) 1717 : audit [DBG] from='client.? 172.21.2.111:0/3064731760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[31774]: cluster 2021-03-10T15:09:41.777211+0000 mgr.gibba032.whopeo (mgr.24625) 4088 : cluster [DBG] pgmap v2316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[31774]: audit 2021-03-10T15:09:41.952871+0000 mgr.gibba032.whopeo (mgr.24625) 4089 : audit [DBG] from='client.46791 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:42 gibba011 bash[31774]: audit 2021-03-10T15:09:42.439857+0000 mon.a (mon.0) 1717 : audit [DBG] from='client.? 172.21.2.111:0/3064731760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:09:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:43 gibba032 bash[24004]: audit 2021-03-10T15:09:42.179253+0000 mgr.gibba032.whopeo (mgr.24625) 4090 : audit [DBG] from='client.46797 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:43 gibba011 bash[30791]: audit 2021-03-10T15:09:42.179253+0000 mgr.gibba032.whopeo (mgr.24625) 4090 : audit [DBG] from='client.46797 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:43 gibba011 bash[31774]: audit 2021-03-10T15:09:42.179253+0000 mgr.gibba032.whopeo (mgr.24625) 4090 : audit [DBG] from='client.46797 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:09:45.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:44 gibba032 bash[24004]: cluster 2021-03-10T15:09:43.779017+0000 mgr.gibba032.whopeo (mgr.24625) 4091 : cluster [DBG] pgmap v2317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:44 gibba011 bash[30791]: cluster 2021-03-10T15:09:43.779017+0000 mgr.gibba032.whopeo (mgr.24625) 4091 : cluster [DBG] pgmap v2317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:44 gibba011 bash[31774]: cluster 2021-03-10T15:09:43.779017+0000 mgr.gibba032.whopeo (mgr.24625) 4091 : cluster [DBG] pgmap v2317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:46 gibba032 bash[24004]: cluster 2021-03-10T15:09:45.780419+0000 mgr.gibba032.whopeo (mgr.24625) 4092 : cluster [DBG] pgmap v2318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:46 gibba011 bash[30791]: cluster 2021-03-10T15:09:45.780419+0000 mgr.gibba032.whopeo (mgr.24625) 4092 : cluster [DBG] pgmap v2318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:46 gibba011 bash[31774]: cluster 2021-03-10T15:09:45.780419+0000 mgr.gibba032.whopeo (mgr.24625) 4092 : cluster [DBG] pgmap v2318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:48 gibba032 bash[24004]: cluster 2021-03-10T15:09:47.781937+0000 mgr.gibba032.whopeo (mgr.24625) 4093 : cluster [DBG] pgmap v2319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:48 gibba011 bash[30791]: cluster 2021-03-10T15:09:47.781937+0000 mgr.gibba032.whopeo (mgr.24625) 4093 : cluster [DBG] pgmap v2319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:48 gibba011 bash[31774]: cluster 2021-03-10T15:09:47.781937+0000 mgr.gibba032.whopeo (mgr.24625) 4093 : cluster [DBG] pgmap v2319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:50 gibba032 bash[24004]: cluster 2021-03-10T15:09:49.783391+0000 mgr.gibba032.whopeo (mgr.24625) 4094 : cluster [DBG] pgmap v2320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:50 gibba011 bash[30791]: cluster 2021-03-10T15:09:49.783391+0000 mgr.gibba032.whopeo (mgr.24625) 4094 : cluster [DBG] pgmap v2320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:51.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:50 gibba011 bash[31774]: cluster 2021-03-10T15:09:49.783391+0000 mgr.gibba032.whopeo (mgr.24625) 4094 : cluster [DBG] pgmap v2320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:52 gibba032 bash[24004]: cluster 2021-03-10T15:09:51.785344+0000 mgr.gibba032.whopeo (mgr.24625) 4095 : cluster [DBG] pgmap v2321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:52 gibba011 bash[30791]: cluster 2021-03-10T15:09:51.785344+0000 mgr.gibba032.whopeo (mgr.24625) 4095 : cluster [DBG] pgmap v2321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:52 gibba011 bash[31774]: cluster 2021-03-10T15:09:51.785344+0000 mgr.gibba032.whopeo (mgr.24625) 4095 : cluster [DBG] pgmap v2321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:53 gibba032 bash[24004]: cephadm 2021-03-10T15:09:52.837937+0000 mgr.gibba032.whopeo (mgr.24625) 4096 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:53 gibba011 bash[30791]: cephadm 2021-03-10T15:09:52.837937+0000 mgr.gibba032.whopeo (mgr.24625) 4096 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:53 gibba011 bash[31774]: cephadm 2021-03-10T15:09:52.837937+0000 mgr.gibba032.whopeo (mgr.24625) 4096 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:09:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:54 gibba032 bash[24004]: cluster 2021-03-10T15:09:53.786840+0000 mgr.gibba032.whopeo (mgr.24625) 4097 : cluster [DBG] pgmap v2322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:54 gibba011 bash[30791]: cluster 2021-03-10T15:09:53.786840+0000 mgr.gibba032.whopeo (mgr.24625) 4097 : cluster [DBG] pgmap v2322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:54 gibba011 bash[31774]: cluster 2021-03-10T15:09:53.786840+0000 mgr.gibba032.whopeo (mgr.24625) 4097 : cluster [DBG] pgmap v2322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: cluster 2021-03-10T15:09:55.788312+0000 mgr.gibba032.whopeo (mgr.24625) 4098 : cluster [DBG] pgmap v2323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.503207+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:09:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.503946+0000 mon.a (mon.0) 1719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:09:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.512565+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:09:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.513309+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:09:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.513979+0000 mon.a (mon.0) 1722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:09:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:56 gibba032 bash[24004]: audit 2021-03-10T15:09:56.514658+0000 mon.a (mon.0) 1723 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:09:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: cluster 2021-03-10T15:09:55.788312+0000 mgr.gibba032.whopeo (mgr.24625) 4098 : cluster [DBG] pgmap v2323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.503207+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:09:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.503946+0000 mon.a (mon.0) 1719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:09:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.512565+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:09:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.513309+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:09:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.513979+0000 mon.a (mon.0) 1722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:09:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[30791]: audit 2021-03-10T15:09:56.514658+0000 mon.a (mon.0) 1723 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:09:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: cluster 2021-03-10T15:09:55.788312+0000 mgr.gibba032.whopeo (mgr.24625) 4098 : cluster [DBG] pgmap v2323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.503207+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:09:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.503946+0000 mon.a (mon.0) 1719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:09:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.512565+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:09:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.513309+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:09:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.513979+0000 mon.a (mon.0) 1722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:09:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:56 gibba011 bash[31774]: audit 2021-03-10T15:09:56.514658+0000 mon.a (mon.0) 1723 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:09:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:09:58 gibba032 bash[24004]: cluster 2021-03-10T15:09:57.790050+0000 mgr.gibba032.whopeo (mgr.24625) 4099 : cluster [DBG] pgmap v2324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:09:58 gibba011 bash[30791]: cluster 2021-03-10T15:09:57.790050+0000 mgr.gibba032.whopeo (mgr.24625) 4099 : cluster [DBG] pgmap v2324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:09:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:09:58 gibba011 bash[31774]: cluster 2021-03-10T15:09:57.790050+0000 mgr.gibba032.whopeo (mgr.24625) 4099 : cluster [DBG] pgmap v2324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:00 gibba011 bash[30791]: debug 2021-03-10T15:09:59.996+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:10:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:00 gibba032 bash[24004]: cluster 2021-03-10T15:09:59.791777+0000 mgr.gibba032.whopeo (mgr.24625) 4100 : cluster [DBG] pgmap v2325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:01.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:00 gibba032 bash[24004]: cluster 2021-03-10T15:10:00.000285+0000 mon.a (mon.0) 1724 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:10:01.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:00 gibba011 bash[30791]: cluster 2021-03-10T15:09:59.791777+0000 mgr.gibba032.whopeo (mgr.24625) 4100 : cluster [DBG] pgmap v2325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:01.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:00 gibba011 bash[30791]: cluster 2021-03-10T15:10:00.000285+0000 mon.a (mon.0) 1724 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:10:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:00 gibba011 bash[31774]: cluster 2021-03-10T15:09:59.791777+0000 mgr.gibba032.whopeo (mgr.24625) 4100 : cluster [DBG] pgmap v2325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:00 gibba011 bash[31774]: cluster 2021-03-10T15:10:00.000285+0000 mon.a (mon.0) 1724 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:10:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:02 gibba032 bash[24004]: cluster 2021-03-10T15:10:01.793627+0000 mgr.gibba032.whopeo (mgr.24625) 4101 : cluster [DBG] pgmap v2326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:02 gibba011 bash[30791]: cluster 2021-03-10T15:10:01.793627+0000 mgr.gibba032.whopeo (mgr.24625) 4101 : cluster [DBG] pgmap v2326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:03.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:02 gibba011 bash[31774]: cluster 2021-03-10T15:10:01.793627+0000 mgr.gibba032.whopeo (mgr.24625) 4101 : cluster [DBG] pgmap v2326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:04 gibba032 bash[24004]: cluster 2021-03-10T15:10:03.795126+0000 mgr.gibba032.whopeo (mgr.24625) 4102 : cluster [DBG] pgmap v2327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:05.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:04 gibba011 bash[31774]: cluster 2021-03-10T15:10:03.795126+0000 mgr.gibba032.whopeo (mgr.24625) 4102 : cluster [DBG] pgmap v2327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:05.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:04 gibba011 bash[30791]: cluster 2021-03-10T15:10:03.795126+0000 mgr.gibba032.whopeo (mgr.24625) 4102 : cluster [DBG] pgmap v2327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:07.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:06 gibba032 bash[24004]: cluster 2021-03-10T15:10:05.796598+0000 mgr.gibba032.whopeo (mgr.24625) 4103 : cluster [DBG] pgmap v2328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:06 gibba011 bash[30791]: cluster 2021-03-10T15:10:05.796598+0000 mgr.gibba032.whopeo (mgr.24625) 4103 : cluster [DBG] pgmap v2328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:06 gibba011 bash[31774]: cluster 2021-03-10T15:10:05.796598+0000 mgr.gibba032.whopeo (mgr.24625) 4103 : cluster [DBG] pgmap v2328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:08 gibba032 bash[24004]: cluster 2021-03-10T15:10:07.798378+0000 mgr.gibba032.whopeo (mgr.24625) 4104 : cluster [DBG] pgmap v2329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:08 gibba032 bash[24004]: cephadm 2021-03-10T15:10:07.853919+0000 mgr.gibba032.whopeo (mgr.24625) 4105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:08 gibba011 bash[30791]: cluster 2021-03-10T15:10:07.798378+0000 mgr.gibba032.whopeo (mgr.24625) 4104 : cluster [DBG] pgmap v2329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:08 gibba011 bash[30791]: cephadm 2021-03-10T15:10:07.853919+0000 mgr.gibba032.whopeo (mgr.24625) 4105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:08 gibba011 bash[31774]: cluster 2021-03-10T15:10:07.798378+0000 mgr.gibba032.whopeo (mgr.24625) 4104 : cluster [DBG] pgmap v2329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:08 gibba011 bash[31774]: cephadm 2021-03-10T15:10:07.853919+0000 mgr.gibba032.whopeo (mgr.24625) 4105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:11.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:10 gibba032 bash[24004]: cluster 2021-03-10T15:10:09.799923+0000 mgr.gibba032.whopeo (mgr.24625) 4106 : cluster [DBG] pgmap v2330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:10 gibba011 bash[30791]: cluster 2021-03-10T15:10:09.799923+0000 mgr.gibba032.whopeo (mgr.24625) 4106 : cluster [DBG] pgmap v2330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:10 gibba011 bash[31774]: cluster 2021-03-10T15:10:09.799923+0000 mgr.gibba032.whopeo (mgr.24625) 4106 : cluster [DBG] pgmap v2330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:12.685 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:10:12.906 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:10:12.907 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (76m) 99s ago 82m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:10:12.907 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (82m) 35s ago 82m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:10:12.907 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (77m) 35s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:10:12.908 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (77m) 35s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:10:12.908 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (76m) 99s ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:10:12.908 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (76m) 99s ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:10:12.908 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (75m) 35s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:10:12.909 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (76m) 99s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:10:12.909 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (82m) 99s ago 82m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:10:12.909 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (82m) 35s ago 82m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:10:12.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (75m) 99s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:10:12.910 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (74m) 99s ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:10:12.910 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (74m) 99s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:10:12.910 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (73m) 99s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:10:12.910 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (73m) 35s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:10:12.911 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (73m) 35s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:10:12.911 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (72m) 35s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:10:12.911 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (72m) 35s ago 82m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:10:12.911 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (81m) 35s ago 82m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:10:12.912 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 99s ago 80m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:10:13.163 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:10:13.163 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:10:13.163 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:10:13.164 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:13.164 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:10:13.164 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:10:13.164 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:13.165 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:10:13.165 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:10:13.165 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:13.165 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:10:13.166 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:10:13.166 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:10:13.166 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:10:13.166 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:10:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:12 gibba032 bash[24004]: cluster 2021-03-10T15:10:11.801708+0000 mgr.gibba032.whopeo (mgr.24625) 4107 : cluster [DBG] pgmap v2331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:12 gibba011 bash[30791]: cluster 2021-03-10T15:10:11.801708+0000 mgr.gibba032.whopeo (mgr.24625) 4107 : cluster [DBG] pgmap v2331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:13.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:12 gibba011 bash[31774]: cluster 2021-03-10T15:10:11.801708+0000 mgr.gibba032.whopeo (mgr.24625) 4107 : cluster [DBG] pgmap v2331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:13 gibba032 bash[24004]: audit 2021-03-10T15:10:12.671182+0000 mgr.gibba032.whopeo (mgr.24625) 4108 : audit [DBG] from='client.46809 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:13 gibba032 bash[24004]: audit 2021-03-10T15:10:12.892734+0000 mgr.gibba032.whopeo (mgr.24625) 4109 : audit [DBG] from='client.46815 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:13 gibba032 bash[24004]: audit 2021-03-10T15:10:13.160466+0000 mon.c (mon.1) 80 : audit [DBG] from='client.? 172.21.2.111:0/3884990089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[30791]: audit 2021-03-10T15:10:12.671182+0000 mgr.gibba032.whopeo (mgr.24625) 4108 : audit [DBG] from='client.46809 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[30791]: audit 2021-03-10T15:10:12.892734+0000 mgr.gibba032.whopeo (mgr.24625) 4109 : audit [DBG] from='client.46815 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[30791]: audit 2021-03-10T15:10:13.160466+0000 mon.c (mon.1) 80 : audit [DBG] from='client.? 172.21.2.111:0/3884990089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[31774]: audit 2021-03-10T15:10:12.671182+0000 mgr.gibba032.whopeo (mgr.24625) 4108 : audit [DBG] from='client.46809 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[31774]: audit 2021-03-10T15:10:12.892734+0000 mgr.gibba032.whopeo (mgr.24625) 4109 : audit [DBG] from='client.46815 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:13 gibba011 bash[31774]: audit 2021-03-10T15:10:13.160466+0000 mon.c (mon.1) 80 : audit [DBG] from='client.? 172.21.2.111:0/3884990089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:15.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:14 gibba032 bash[24004]: cluster 2021-03-10T15:10:13.803200+0000 mgr.gibba032.whopeo (mgr.24625) 4110 : cluster [DBG] pgmap v2332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:14 gibba011 bash[30791]: cluster 2021-03-10T15:10:13.803200+0000 mgr.gibba032.whopeo (mgr.24625) 4110 : cluster [DBG] pgmap v2332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:14 gibba011 bash[31774]: cluster 2021-03-10T15:10:13.803200+0000 mgr.gibba032.whopeo (mgr.24625) 4110 : cluster [DBG] pgmap v2332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:16 gibba032 bash[24004]: cluster 2021-03-10T15:10:15.804604+0000 mgr.gibba032.whopeo (mgr.24625) 4111 : cluster [DBG] pgmap v2333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:16 gibba011 bash[30791]: cluster 2021-03-10T15:10:15.804604+0000 mgr.gibba032.whopeo (mgr.24625) 4111 : cluster [DBG] pgmap v2333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:17.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:16 gibba011 bash[31774]: cluster 2021-03-10T15:10:15.804604+0000 mgr.gibba032.whopeo (mgr.24625) 4111 : cluster [DBG] pgmap v2333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:18 gibba032 bash[24004]: cluster 2021-03-10T15:10:17.806244+0000 mgr.gibba032.whopeo (mgr.24625) 4112 : cluster [DBG] pgmap v2334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:18 gibba011 bash[30791]: cluster 2021-03-10T15:10:17.806244+0000 mgr.gibba032.whopeo (mgr.24625) 4112 : cluster [DBG] pgmap v2334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:18.655 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:18 gibba011 bash[31774]: cluster 2021-03-10T15:10:17.806244+0000 mgr.gibba032.whopeo (mgr.24625) 4112 : cluster [DBG] pgmap v2334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:21.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:20 gibba032 bash[24004]: cluster 2021-03-10T15:10:19.807736+0000 mgr.gibba032.whopeo (mgr.24625) 4113 : cluster [DBG] pgmap v2335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:21.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:20 gibba011 bash[30791]: cluster 2021-03-10T15:10:19.807736+0000 mgr.gibba032.whopeo (mgr.24625) 4113 : cluster [DBG] pgmap v2335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:21.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:20 gibba011 bash[31774]: cluster 2021-03-10T15:10:19.807736+0000 mgr.gibba032.whopeo (mgr.24625) 4113 : cluster [DBG] pgmap v2335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:22 gibba032 bash[24004]: cluster 2021-03-10T15:10:21.809436+0000 mgr.gibba032.whopeo (mgr.24625) 4114 : cluster [DBG] pgmap v2336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:22 gibba011 bash[30791]: cluster 2021-03-10T15:10:21.809436+0000 mgr.gibba032.whopeo (mgr.24625) 4114 : cluster [DBG] pgmap v2336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:23.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:22 gibba011 bash[31774]: cluster 2021-03-10T15:10:21.809436+0000 mgr.gibba032.whopeo (mgr.24625) 4114 : cluster [DBG] pgmap v2336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:23 gibba032 bash[24004]: cephadm 2021-03-10T15:10:22.869902+0000 mgr.gibba032.whopeo (mgr.24625) 4115 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:23 gibba011 bash[30791]: cephadm 2021-03-10T15:10:22.869902+0000 mgr.gibba032.whopeo (mgr.24625) 4115 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:23 gibba011 bash[31774]: cephadm 2021-03-10T15:10:22.869902+0000 mgr.gibba032.whopeo (mgr.24625) 4115 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:24 gibba032 bash[24004]: cluster 2021-03-10T15:10:23.810800+0000 mgr.gibba032.whopeo (mgr.24625) 4116 : cluster [DBG] pgmap v2337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:24 gibba011 bash[30791]: cluster 2021-03-10T15:10:23.810800+0000 mgr.gibba032.whopeo (mgr.24625) 4116 : cluster [DBG] pgmap v2337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:24 gibba011 bash[31774]: cluster 2021-03-10T15:10:23.810800+0000 mgr.gibba032.whopeo (mgr.24625) 4116 : cluster [DBG] pgmap v2337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:27.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:26 gibba032 bash[24004]: cluster 2021-03-10T15:10:25.812171+0000 mgr.gibba032.whopeo (mgr.24625) 4117 : cluster [DBG] pgmap v2338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:27.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:26 gibba011 bash[30791]: cluster 2021-03-10T15:10:25.812171+0000 mgr.gibba032.whopeo (mgr.24625) 4117 : cluster [DBG] pgmap v2338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:27.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:26 gibba011 bash[31774]: cluster 2021-03-10T15:10:25.812171+0000 mgr.gibba032.whopeo (mgr.24625) 4117 : cluster [DBG] pgmap v2338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:28 gibba032 bash[24004]: cluster 2021-03-10T15:10:27.813685+0000 mgr.gibba032.whopeo (mgr.24625) 4118 : cluster [DBG] pgmap v2339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:28 gibba011 bash[30791]: cluster 2021-03-10T15:10:27.813685+0000 mgr.gibba032.whopeo (mgr.24625) 4118 : cluster [DBG] pgmap v2339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:28 gibba011 bash[31774]: cluster 2021-03-10T15:10:27.813685+0000 mgr.gibba032.whopeo (mgr.24625) 4118 : cluster [DBG] pgmap v2339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:30 gibba032 bash[24004]: cluster 2021-03-10T15:10:29.815197+0000 mgr.gibba032.whopeo (mgr.24625) 4119 : cluster [DBG] pgmap v2340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:30 gibba011 bash[30791]: cluster 2021-03-10T15:10:29.815197+0000 mgr.gibba032.whopeo (mgr.24625) 4119 : cluster [DBG] pgmap v2340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:30 gibba011 bash[31774]: cluster 2021-03-10T15:10:29.815197+0000 mgr.gibba032.whopeo (mgr.24625) 4119 : cluster [DBG] pgmap v2340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:32 gibba032 bash[24004]: cluster 2021-03-10T15:10:31.816940+0000 mgr.gibba032.whopeo (mgr.24625) 4120 : cluster [DBG] pgmap v2341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:32 gibba011 bash[30791]: cluster 2021-03-10T15:10:31.816940+0000 mgr.gibba032.whopeo (mgr.24625) 4120 : cluster [DBG] pgmap v2341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:32 gibba011 bash[31774]: cluster 2021-03-10T15:10:31.816940+0000 mgr.gibba032.whopeo (mgr.24625) 4120 : cluster [DBG] pgmap v2341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:35.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:34 gibba032 bash[24004]: cluster 2021-03-10T15:10:33.818549+0000 mgr.gibba032.whopeo (mgr.24625) 4121 : cluster [DBG] pgmap v2342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:35.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:34 gibba011 bash[30791]: cluster 2021-03-10T15:10:33.818549+0000 mgr.gibba032.whopeo (mgr.24625) 4121 : cluster [DBG] pgmap v2342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:35.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:34 gibba011 bash[31774]: cluster 2021-03-10T15:10:33.818549+0000 mgr.gibba032.whopeo (mgr.24625) 4121 : cluster [DBG] pgmap v2342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:36 gibba032 bash[24004]: cluster 2021-03-10T15:10:35.819888+0000 mgr.gibba032.whopeo (mgr.24625) 4122 : cluster [DBG] pgmap v2343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:36 gibba011 bash[30791]: cluster 2021-03-10T15:10:35.819888+0000 mgr.gibba032.whopeo (mgr.24625) 4122 : cluster [DBG] pgmap v2343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:36 gibba011 bash[31774]: cluster 2021-03-10T15:10:35.819888+0000 mgr.gibba032.whopeo (mgr.24625) 4122 : cluster [DBG] pgmap v2343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: cluster 2021-03-10T15:10:37.821481+0000 mgr.gibba032.whopeo (mgr.24625) 4123 : cluster [DBG] pgmap v2344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: cephadm 2021-03-10T15:10:37.881775+0000 mgr.gibba032.whopeo (mgr.24625) 4124 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:10:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: cephadm 2021-03-10T15:10:37.883368+0000 mgr.gibba032.whopeo (mgr.24625) 4125 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:10:38.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.093019+0000 mon.a (mon.0) 1725 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:10:38.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.096572+0000 mon.a (mon.0) 1726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:10:38.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.097834+0000 mon.a (mon.0) 1727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.099270+0000 mon.a (mon.0) 1728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.100353+0000 mon.a (mon.0) 1729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.102702+0000 mon.a (mon.0) 1730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:38 gibba032 bash[24004]: audit 2021-03-10T15:10:38.104009+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: cluster 2021-03-10T15:10:37.821481+0000 mgr.gibba032.whopeo (mgr.24625) 4123 : cluster [DBG] pgmap v2344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: cephadm 2021-03-10T15:10:37.881775+0000 mgr.gibba032.whopeo (mgr.24625) 4124 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:10:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: cephadm 2021-03-10T15:10:37.883368+0000 mgr.gibba032.whopeo (mgr.24625) 4125 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:10:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.093019+0000 mon.a (mon.0) 1725 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:10:38.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.096572+0000 mon.a (mon.0) 1726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:10:38.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.097834+0000 mon.a (mon.0) 1727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.099270+0000 mon.a (mon.0) 1728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.100353+0000 mon.a (mon.0) 1729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.102702+0000 mon.a (mon.0) 1730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[30791]: audit 2021-03-10T15:10:38.104009+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: cluster 2021-03-10T15:10:37.821481+0000 mgr.gibba032.whopeo (mgr.24625) 4123 : cluster [DBG] pgmap v2344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:38.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: cephadm 2021-03-10T15:10:37.881775+0000 mgr.gibba032.whopeo (mgr.24625) 4124 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:10:38.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: cephadm 2021-03-10T15:10:37.883368+0000 mgr.gibba032.whopeo (mgr.24625) 4125 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:10:38.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.093019+0000 mon.a (mon.0) 1725 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:10:38.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.096572+0000 mon.a (mon.0) 1726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:10:38.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.097834+0000 mon.a (mon.0) 1727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.099270+0000 mon.a (mon.0) 1728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.100353+0000 mon.a (mon.0) 1729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.102702+0000 mon.a (mon.0) 1730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:38.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:38 gibba011 bash[31774]: audit 2021-03-10T15:10:38.104009+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.093729+0000 mgr.gibba032.whopeo (mgr.24625) 4126 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:10:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.095333+0000 mgr.gibba032.whopeo (mgr.24625) 4127 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:10:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.096598+0000 mgr.gibba032.whopeo (mgr.24625) 4128 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:10:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.096642+0000 mgr.gibba032.whopeo (mgr.24625) 4129 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:10:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.097934+0000 mgr.gibba032.whopeo (mgr.24625) 4130 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:10:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.097976+0000 mgr.gibba032.whopeo (mgr.24625) 4131 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:10:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.099027+0000 mgr.gibba032.whopeo (mgr.24625) 4132 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:10:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.100081+0000 mgr.gibba032.whopeo (mgr.24625) 4133 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:10:39.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.100140+0000 mgr.gibba032.whopeo (mgr.24625) 4134 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:10:39.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.101268+0000 mgr.gibba032.whopeo (mgr.24625) 4135 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:10:39.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.101579+0000 mgr.gibba032.whopeo (mgr.24625) 4136 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:10:39.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.102609+0000 mgr.gibba032.whopeo (mgr.24625) 4137 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:10:39.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.102652+0000 mgr.gibba032.whopeo (mgr.24625) 4138 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:10:39.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:39 gibba032 bash[24004]: cephadm 2021-03-10T15:10:38.102765+0000 mgr.gibba032.whopeo (mgr.24625) 4139 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:39.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.093729+0000 mgr.gibba032.whopeo (mgr.24625) 4126 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:10:39.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.095333+0000 mgr.gibba032.whopeo (mgr.24625) 4127 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:10:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.096598+0000 mgr.gibba032.whopeo (mgr.24625) 4128 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:10:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.096642+0000 mgr.gibba032.whopeo (mgr.24625) 4129 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:10:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.097934+0000 mgr.gibba032.whopeo (mgr.24625) 4130 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:10:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.097976+0000 mgr.gibba032.whopeo (mgr.24625) 4131 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:10:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.099027+0000 mgr.gibba032.whopeo (mgr.24625) 4132 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:10:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.100081+0000 mgr.gibba032.whopeo (mgr.24625) 4133 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:10:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.100140+0000 mgr.gibba032.whopeo (mgr.24625) 4134 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:10:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.101268+0000 mgr.gibba032.whopeo (mgr.24625) 4135 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:10:39.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.101579+0000 mgr.gibba032.whopeo (mgr.24625) 4136 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:10:39.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.102609+0000 mgr.gibba032.whopeo (mgr.24625) 4137 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:10:39.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.102652+0000 mgr.gibba032.whopeo (mgr.24625) 4138 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:10:39.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[30791]: cephadm 2021-03-10T15:10:38.102765+0000 mgr.gibba032.whopeo (mgr.24625) 4139 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:39.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.093729+0000 mgr.gibba032.whopeo (mgr.24625) 4126 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:10:39.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.095333+0000 mgr.gibba032.whopeo (mgr.24625) 4127 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:10:39.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.096598+0000 mgr.gibba032.whopeo (mgr.24625) 4128 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:10:39.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.096642+0000 mgr.gibba032.whopeo (mgr.24625) 4129 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:10:39.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.097934+0000 mgr.gibba032.whopeo (mgr.24625) 4130 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:10:39.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.097976+0000 mgr.gibba032.whopeo (mgr.24625) 4131 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:10:39.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.099027+0000 mgr.gibba032.whopeo (mgr.24625) 4132 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:10:39.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.100081+0000 mgr.gibba032.whopeo (mgr.24625) 4133 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:10:39.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.100140+0000 mgr.gibba032.whopeo (mgr.24625) 4134 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:10:39.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.101268+0000 mgr.gibba032.whopeo (mgr.24625) 4135 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:10:39.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.101579+0000 mgr.gibba032.whopeo (mgr.24625) 4136 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:10:39.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.102609+0000 mgr.gibba032.whopeo (mgr.24625) 4137 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:10:39.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.102652+0000 mgr.gibba032.whopeo (mgr.24625) 4138 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:10:39.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:39 gibba011 bash[31774]: cephadm 2021-03-10T15:10:38.102765+0000 mgr.gibba032.whopeo (mgr.24625) 4139 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:40 gibba032 bash[24004]: cluster 2021-03-10T15:10:39.822887+0000 mgr.gibba032.whopeo (mgr.24625) 4140 : cluster [DBG] pgmap v2345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:40 gibba011 bash[30791]: cluster 2021-03-10T15:10:39.822887+0000 mgr.gibba032.whopeo (mgr.24625) 4140 : cluster [DBG] pgmap v2345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:40 gibba011 bash[31774]: cluster 2021-03-10T15:10:39.822887+0000 mgr.gibba032.whopeo (mgr.24625) 4140 : cluster [DBG] pgmap v2345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:42 gibba032 bash[24004]: cluster 2021-03-10T15:10:41.824695+0000 mgr.gibba032.whopeo (mgr.24625) 4141 : cluster [DBG] pgmap v2346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:42 gibba011 bash[30791]: cluster 2021-03-10T15:10:41.824695+0000 mgr.gibba032.whopeo (mgr.24625) 4141 : cluster [DBG] pgmap v2346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:43.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:42 gibba011 bash[31774]: cluster 2021-03-10T15:10:41.824695+0000 mgr.gibba032.whopeo (mgr.24625) 4141 : cluster [DBG] pgmap v2346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:43.407 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:10:43.635 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:10:43.635 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (77m) 2m ago 82m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:10:43.635 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (82m) 66s ago 82m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:10:43.636 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (77m) 66s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:10:43.636 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (78m) 66s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:10:43.636 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (77m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:10:43.636 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (76m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:10:43.636 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (76m) 66s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:10:43.637 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (76m) 2m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:10:43.637 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (82m) 2m ago 82m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:10:43.637 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (82m) 66s ago 82m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:10:43.637 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (75m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:10:43.638 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (75m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:10:43.638 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (74m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:10:43.638 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (74m) 2m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:10:43.638 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (73m) 66s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:10:43.639 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (73m) 66s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:10:43.639 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (73m) 66s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:10:43.639 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (72m) 66s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:10:43.639 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (82m) 66s ago 82m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:10:43.639 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 80m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:10:43.887 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:10:43.887 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:10:43.888 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:10:43.888 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:43.888 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:10:43.888 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:10:43.889 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:43.889 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:10:43.889 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:10:43.889 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:10:43.889 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:10:43.890 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:10:43.890 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:10:43.890 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:10:43.890 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:10:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:43 gibba032 bash[24004]: audit 2021-03-10T15:10:43.884140+0000 mon.a (mon.0) 1732 : audit [DBG] from='client.? 172.21.2.111:0/557382098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:43 gibba011 bash[30791]: audit 2021-03-10T15:10:43.884140+0000 mon.a (mon.0) 1732 : audit [DBG] from='client.? 172.21.2.111:0/557382098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:43 gibba011 bash[31774]: audit 2021-03-10T15:10:43.884140+0000 mon.a (mon.0) 1732 : audit [DBG] from='client.? 172.21.2.111:0/557382098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:10:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:44 gibba032 bash[24004]: audit 2021-03-10T15:10:43.393695+0000 mgr.gibba032.whopeo (mgr.24625) 4142 : audit [DBG] from='client.56812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:44 gibba032 bash[24004]: audit 2021-03-10T15:10:43.617561+0000 mgr.gibba032.whopeo (mgr.24625) 4143 : audit [DBG] from='client.56818 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:44 gibba032 bash[24004]: cluster 2021-03-10T15:10:43.826253+0000 mgr.gibba032.whopeo (mgr.24625) 4144 : cluster [DBG] pgmap v2347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[30791]: audit 2021-03-10T15:10:43.393695+0000 mgr.gibba032.whopeo (mgr.24625) 4142 : audit [DBG] from='client.56812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[30791]: audit 2021-03-10T15:10:43.617561+0000 mgr.gibba032.whopeo (mgr.24625) 4143 : audit [DBG] from='client.56818 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[30791]: cluster 2021-03-10T15:10:43.826253+0000 mgr.gibba032.whopeo (mgr.24625) 4144 : cluster [DBG] pgmap v2347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[31774]: audit 2021-03-10T15:10:43.393695+0000 mgr.gibba032.whopeo (mgr.24625) 4142 : audit [DBG] from='client.56812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[31774]: audit 2021-03-10T15:10:43.617561+0000 mgr.gibba032.whopeo (mgr.24625) 4143 : audit [DBG] from='client.56818 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:10:45.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:44 gibba011 bash[31774]: cluster 2021-03-10T15:10:43.826253+0000 mgr.gibba032.whopeo (mgr.24625) 4144 : cluster [DBG] pgmap v2347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:46 gibba032 bash[24004]: cluster 2021-03-10T15:10:45.827615+0000 mgr.gibba032.whopeo (mgr.24625) 4145 : cluster [DBG] pgmap v2348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:46 gibba011 bash[30791]: cluster 2021-03-10T15:10:45.827615+0000 mgr.gibba032.whopeo (mgr.24625) 4145 : cluster [DBG] pgmap v2348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:46 gibba011 bash[31774]: cluster 2021-03-10T15:10:45.827615+0000 mgr.gibba032.whopeo (mgr.24625) 4145 : cluster [DBG] pgmap v2348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:48 gibba032 bash[24004]: cluster 2021-03-10T15:10:47.829376+0000 mgr.gibba032.whopeo (mgr.24625) 4146 : cluster [DBG] pgmap v2349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:48 gibba011 bash[30791]: cluster 2021-03-10T15:10:47.829376+0000 mgr.gibba032.whopeo (mgr.24625) 4146 : cluster [DBG] pgmap v2349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:48 gibba011 bash[31774]: cluster 2021-03-10T15:10:47.829376+0000 mgr.gibba032.whopeo (mgr.24625) 4146 : cluster [DBG] pgmap v2349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:50 gibba032 bash[24004]: cluster 2021-03-10T15:10:49.830856+0000 mgr.gibba032.whopeo (mgr.24625) 4147 : cluster [DBG] pgmap v2350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:50 gibba011 bash[30791]: cluster 2021-03-10T15:10:49.830856+0000 mgr.gibba032.whopeo (mgr.24625) 4147 : cluster [DBG] pgmap v2350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:51.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:50 gibba011 bash[31774]: cluster 2021-03-10T15:10:49.830856+0000 mgr.gibba032.whopeo (mgr.24625) 4147 : cluster [DBG] pgmap v2350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:52 gibba032 bash[24004]: cluster 2021-03-10T15:10:51.832650+0000 mgr.gibba032.whopeo (mgr.24625) 4148 : cluster [DBG] pgmap v2351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:52 gibba011 bash[30791]: cluster 2021-03-10T15:10:51.832650+0000 mgr.gibba032.whopeo (mgr.24625) 4148 : cluster [DBG] pgmap v2351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:52 gibba011 bash[31774]: cluster 2021-03-10T15:10:51.832650+0000 mgr.gibba032.whopeo (mgr.24625) 4148 : cluster [DBG] pgmap v2351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:54 gibba032 bash[24004]: cephadm 2021-03-10T15:10:53.118420+0000 mgr.gibba032.whopeo (mgr.24625) 4149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:54 gibba032 bash[24004]: cluster 2021-03-10T15:10:53.834191+0000 mgr.gibba032.whopeo (mgr.24625) 4150 : cluster [DBG] pgmap v2352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:54 gibba011 bash[30791]: cephadm 2021-03-10T15:10:53.118420+0000 mgr.gibba032.whopeo (mgr.24625) 4149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:54 gibba011 bash[30791]: cluster 2021-03-10T15:10:53.834191+0000 mgr.gibba032.whopeo (mgr.24625) 4150 : cluster [DBG] pgmap v2352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:54 gibba011 bash[31774]: cephadm 2021-03-10T15:10:53.118420+0000 mgr.gibba032.whopeo (mgr.24625) 4149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:10:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:54 gibba011 bash[31774]: cluster 2021-03-10T15:10:53.834191+0000 mgr.gibba032.whopeo (mgr.24625) 4150 : cluster [DBG] pgmap v2352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: cluster 2021-03-10T15:10:55.835574+0000 mgr.gibba032.whopeo (mgr.24625) 4151 : cluster [DBG] pgmap v2353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.505582+0000 mon.a (mon.0) 1733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:10:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.506224+0000 mon.a (mon.0) 1734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:10:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.517971+0000 mon.a (mon.0) 1735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:10:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.518764+0000 mon.a (mon.0) 1736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:10:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.519604+0000 mon.a (mon.0) 1737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:10:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:56 gibba032 bash[24004]: audit 2021-03-10T15:10:56.520314+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:10:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: cluster 2021-03-10T15:10:55.835574+0000 mgr.gibba032.whopeo (mgr.24625) 4151 : cluster [DBG] pgmap v2353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.505582+0000 mon.a (mon.0) 1733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:10:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.506224+0000 mon.a (mon.0) 1734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:10:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.517971+0000 mon.a (mon.0) 1735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:10:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.518764+0000 mon.a (mon.0) 1736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:10:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.519604+0000 mon.a (mon.0) 1737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:10:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[30791]: audit 2021-03-10T15:10:56.520314+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:10:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: cluster 2021-03-10T15:10:55.835574+0000 mgr.gibba032.whopeo (mgr.24625) 4151 : cluster [DBG] pgmap v2353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.505582+0000 mon.a (mon.0) 1733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:10:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.506224+0000 mon.a (mon.0) 1734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:10:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.517971+0000 mon.a (mon.0) 1735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:10:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.518764+0000 mon.a (mon.0) 1736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:10:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.519604+0000 mon.a (mon.0) 1737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:10:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:56 gibba011 bash[31774]: audit 2021-03-10T15:10:56.520314+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:10:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:10:58 gibba032 bash[24004]: cluster 2021-03-10T15:10:57.837283+0000 mgr.gibba032.whopeo (mgr.24625) 4152 : cluster [DBG] pgmap v2354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:10:58 gibba011 bash[30791]: cluster 2021-03-10T15:10:57.837283+0000 mgr.gibba032.whopeo (mgr.24625) 4152 : cluster [DBG] pgmap v2354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:10:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:10:58 gibba011 bash[31774]: cluster 2021-03-10T15:10:57.837283+0000 mgr.gibba032.whopeo (mgr.24625) 4152 : cluster [DBG] pgmap v2354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:00 gibba032 bash[24004]: cluster 2021-03-10T15:10:59.838646+0000 mgr.gibba032.whopeo (mgr.24625) 4153 : cluster [DBG] pgmap v2355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:01.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:00 gibba011 bash[30791]: cluster 2021-03-10T15:10:59.838646+0000 mgr.gibba032.whopeo (mgr.24625) 4153 : cluster [DBG] pgmap v2355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:00 gibba011 bash[31774]: cluster 2021-03-10T15:10:59.838646+0000 mgr.gibba032.whopeo (mgr.24625) 4153 : cluster [DBG] pgmap v2355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:02 gibba032 bash[24004]: cluster 2021-03-10T15:11:01.840375+0000 mgr.gibba032.whopeo (mgr.24625) 4154 : cluster [DBG] pgmap v2356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:02 gibba011 bash[30791]: cluster 2021-03-10T15:11:01.840375+0000 mgr.gibba032.whopeo (mgr.24625) 4154 : cluster [DBG] pgmap v2356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:03.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:02 gibba011 bash[31774]: cluster 2021-03-10T15:11:01.840375+0000 mgr.gibba032.whopeo (mgr.24625) 4154 : cluster [DBG] pgmap v2356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:05.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:04 gibba032 bash[24004]: cluster 2021-03-10T15:11:03.841829+0000 mgr.gibba032.whopeo (mgr.24625) 4155 : cluster [DBG] pgmap v2357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:05.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:04 gibba011 bash[30791]: cluster 2021-03-10T15:11:03.841829+0000 mgr.gibba032.whopeo (mgr.24625) 4155 : cluster [DBG] pgmap v2357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:05.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:04 gibba011 bash[31774]: cluster 2021-03-10T15:11:03.841829+0000 mgr.gibba032.whopeo (mgr.24625) 4155 : cluster [DBG] pgmap v2357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:07.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:06 gibba032 bash[24004]: cluster 2021-03-10T15:11:05.843272+0000 mgr.gibba032.whopeo (mgr.24625) 4156 : cluster [DBG] pgmap v2358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:06 gibba011 bash[30791]: cluster 2021-03-10T15:11:05.843272+0000 mgr.gibba032.whopeo (mgr.24625) 4156 : cluster [DBG] pgmap v2358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:06 gibba011 bash[31774]: cluster 2021-03-10T15:11:05.843272+0000 mgr.gibba032.whopeo (mgr.24625) 4156 : cluster [DBG] pgmap v2358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:08 gibba032 bash[24004]: cluster 2021-03-10T15:11:07.845142+0000 mgr.gibba032.whopeo (mgr.24625) 4157 : cluster [DBG] pgmap v2359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:08 gibba011 bash[30791]: cluster 2021-03-10T15:11:07.845142+0000 mgr.gibba032.whopeo (mgr.24625) 4157 : cluster [DBG] pgmap v2359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:08 gibba011 bash[31774]: cluster 2021-03-10T15:11:07.845142+0000 mgr.gibba032.whopeo (mgr.24625) 4157 : cluster [DBG] pgmap v2359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:09 gibba032 bash[24004]: cephadm 2021-03-10T15:11:08.122559+0000 mgr.gibba032.whopeo (mgr.24625) 4158 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:09 gibba011 bash[30791]: cephadm 2021-03-10T15:11:08.122559+0000 mgr.gibba032.whopeo (mgr.24625) 4158 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:09 gibba011 bash[31774]: cephadm 2021-03-10T15:11:08.122559+0000 mgr.gibba032.whopeo (mgr.24625) 4158 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:10 gibba032 bash[24004]: cluster 2021-03-10T15:11:09.846596+0000 mgr.gibba032.whopeo (mgr.24625) 4159 : cluster [DBG] pgmap v2360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:10 gibba011 bash[30791]: cluster 2021-03-10T15:11:09.846596+0000 mgr.gibba032.whopeo (mgr.24625) 4159 : cluster [DBG] pgmap v2360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:10 gibba011 bash[31774]: cluster 2021-03-10T15:11:09.846596+0000 mgr.gibba032.whopeo (mgr.24625) 4159 : cluster [DBG] pgmap v2360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:12 gibba032 bash[24004]: cluster 2021-03-10T15:11:11.848198+0000 mgr.gibba032.whopeo (mgr.24625) 4160 : cluster [DBG] pgmap v2361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:12 gibba011 bash[30791]: cluster 2021-03-10T15:11:11.848198+0000 mgr.gibba032.whopeo (mgr.24625) 4160 : cluster [DBG] pgmap v2361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:12 gibba011 bash[31774]: cluster 2021-03-10T15:11:11.848198+0000 mgr.gibba032.whopeo (mgr.24625) 4160 : cluster [DBG] pgmap v2361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:14.131 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:11:14.390 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:11:14.390 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (77m) 2m ago 83m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:11:14.391 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (83m) 96s ago 83m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:11:14.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (78m) 96s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:11:14.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (78m) 96s ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:11:14.391 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (77m) 2m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:11:14.392 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (77m) 2m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:11:14.392 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (76m) 96s ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:11:14.392 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (77m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:11:14.392 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (83m) 2m ago 83m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:11:14.392 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (83m) 96s ago 83m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:11:14.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (76m) 2m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:11:14.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (75m) 2m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:11:14.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (75m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:11:14.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (74m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:11:14.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (74m) 96s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:11:14.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (74m) 96s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:11:14.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (73m) 96s ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:11:14.394 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (73m) 96s ago 83m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:11:14.395 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (82m) 96s ago 83m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:11:14.395 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 81m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:11:14.648 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:11:14.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:11:14.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:11:14.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:14.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:11:14.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:11:14.650 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:14.650 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:11:14.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:11:14.651 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:14.651 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:11:14.651 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:11:14.651 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:11:14.651 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:11:14.652 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:11:15.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:14 gibba032 bash[24004]: cluster 2021-03-10T15:11:13.849827+0000 mgr.gibba032.whopeo (mgr.24625) 4161 : cluster [DBG] pgmap v2362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:15.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:14 gibba032 bash[24004]: audit 2021-03-10T15:11:14.646252+0000 mon.a (mon.0) 1739 : audit [DBG] from='client.? 172.21.2.111:0/2102498250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:14 gibba011 bash[30791]: cluster 2021-03-10T15:11:13.849827+0000 mgr.gibba032.whopeo (mgr.24625) 4161 : cluster [DBG] pgmap v2362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:14 gibba011 bash[30791]: audit 2021-03-10T15:11:14.646252+0000 mon.a (mon.0) 1739 : audit [DBG] from='client.? 172.21.2.111:0/2102498250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:14 gibba011 bash[31774]: cluster 2021-03-10T15:11:13.849827+0000 mgr.gibba032.whopeo (mgr.24625) 4161 : cluster [DBG] pgmap v2362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:14 gibba011 bash[31774]: audit 2021-03-10T15:11:14.646252+0000 mon.a (mon.0) 1739 : audit [DBG] from='client.? 172.21.2.111:0/2102498250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:15 gibba032 bash[24004]: audit 2021-03-10T15:11:14.117927+0000 mgr.gibba032.whopeo (mgr.24625) 4162 : audit [DBG] from='client.46845 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:16.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:15 gibba032 bash[24004]: audit 2021-03-10T15:11:14.344798+0000 mgr.gibba032.whopeo (mgr.24625) 4163 : audit [DBG] from='client.56836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:16.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:15 gibba011 bash[30791]: audit 2021-03-10T15:11:14.117927+0000 mgr.gibba032.whopeo (mgr.24625) 4162 : audit [DBG] from='client.46845 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:15 gibba011 bash[30791]: audit 2021-03-10T15:11:14.344798+0000 mgr.gibba032.whopeo (mgr.24625) 4163 : audit [DBG] from='client.56836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:15 gibba011 bash[31774]: audit 2021-03-10T15:11:14.117927+0000 mgr.gibba032.whopeo (mgr.24625) 4162 : audit [DBG] from='client.46845 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:15 gibba011 bash[31774]: audit 2021-03-10T15:11:14.344798+0000 mgr.gibba032.whopeo (mgr.24625) 4163 : audit [DBG] from='client.56836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:16 gibba032 bash[24004]: cluster 2021-03-10T15:11:15.851193+0000 mgr.gibba032.whopeo (mgr.24625) 4164 : cluster [DBG] pgmap v2363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:16 gibba011 bash[30791]: cluster 2021-03-10T15:11:15.851193+0000 mgr.gibba032.whopeo (mgr.24625) 4164 : cluster [DBG] pgmap v2363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:17.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:16 gibba011 bash[31774]: cluster 2021-03-10T15:11:15.851193+0000 mgr.gibba032.whopeo (mgr.24625) 4164 : cluster [DBG] pgmap v2363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:18 gibba032 bash[24004]: cluster 2021-03-10T15:11:17.852911+0000 mgr.gibba032.whopeo (mgr.24625) 4165 : cluster [DBG] pgmap v2364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:18 gibba011 bash[30791]: cluster 2021-03-10T15:11:17.852911+0000 mgr.gibba032.whopeo (mgr.24625) 4165 : cluster [DBG] pgmap v2364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:18 gibba011 bash[31774]: cluster 2021-03-10T15:11:17.852911+0000 mgr.gibba032.whopeo (mgr.24625) 4165 : cluster [DBG] pgmap v2364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:21.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:20 gibba032 bash[24004]: cluster 2021-03-10T15:11:19.854455+0000 mgr.gibba032.whopeo (mgr.24625) 4166 : cluster [DBG] pgmap v2365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:21.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:20 gibba011 bash[30791]: cluster 2021-03-10T15:11:19.854455+0000 mgr.gibba032.whopeo (mgr.24625) 4166 : cluster [DBG] pgmap v2365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:21.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:20 gibba011 bash[31774]: cluster 2021-03-10T15:11:19.854455+0000 mgr.gibba032.whopeo (mgr.24625) 4166 : cluster [DBG] pgmap v2365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:22 gibba032 bash[24004]: cluster 2021-03-10T15:11:21.855999+0000 mgr.gibba032.whopeo (mgr.24625) 4167 : cluster [DBG] pgmap v2366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:22 gibba011 bash[30791]: cluster 2021-03-10T15:11:21.855999+0000 mgr.gibba032.whopeo (mgr.24625) 4167 : cluster [DBG] pgmap v2366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:23.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:22 gibba011 bash[31774]: cluster 2021-03-10T15:11:21.855999+0000 mgr.gibba032.whopeo (mgr.24625) 4167 : cluster [DBG] pgmap v2366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:24 gibba032 bash[24004]: cephadm 2021-03-10T15:11:23.138504+0000 mgr.gibba032.whopeo (mgr.24625) 4168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:24 gibba032 bash[24004]: cluster 2021-03-10T15:11:23.857418+0000 mgr.gibba032.whopeo (mgr.24625) 4169 : cluster [DBG] pgmap v2367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:24 gibba011 bash[30791]: cephadm 2021-03-10T15:11:23.138504+0000 mgr.gibba032.whopeo (mgr.24625) 4168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:24 gibba011 bash[30791]: cluster 2021-03-10T15:11:23.857418+0000 mgr.gibba032.whopeo (mgr.24625) 4169 : cluster [DBG] pgmap v2367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:24 gibba011 bash[31774]: cephadm 2021-03-10T15:11:23.138504+0000 mgr.gibba032.whopeo (mgr.24625) 4168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:24 gibba011 bash[31774]: cluster 2021-03-10T15:11:23.857418+0000 mgr.gibba032.whopeo (mgr.24625) 4169 : cluster [DBG] pgmap v2367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:27.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:26 gibba032 bash[24004]: cluster 2021-03-10T15:11:25.858762+0000 mgr.gibba032.whopeo (mgr.24625) 4170 : cluster [DBG] pgmap v2368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:27.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:26 gibba011 bash[30791]: cluster 2021-03-10T15:11:25.858762+0000 mgr.gibba032.whopeo (mgr.24625) 4170 : cluster [DBG] pgmap v2368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:27.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:26 gibba011 bash[31774]: cluster 2021-03-10T15:11:25.858762+0000 mgr.gibba032.whopeo (mgr.24625) 4170 : cluster [DBG] pgmap v2368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:28 gibba032 bash[24004]: cluster 2021-03-10T15:11:27.860624+0000 mgr.gibba032.whopeo (mgr.24625) 4171 : cluster [DBG] pgmap v2369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:28 gibba011 bash[30791]: cluster 2021-03-10T15:11:27.860624+0000 mgr.gibba032.whopeo (mgr.24625) 4171 : cluster [DBG] pgmap v2369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:28 gibba011 bash[31774]: cluster 2021-03-10T15:11:27.860624+0000 mgr.gibba032.whopeo (mgr.24625) 4171 : cluster [DBG] pgmap v2369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:30 gibba032 bash[24004]: cluster 2021-03-10T15:11:29.862004+0000 mgr.gibba032.whopeo (mgr.24625) 4172 : cluster [DBG] pgmap v2370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:30 gibba011 bash[30791]: cluster 2021-03-10T15:11:29.862004+0000 mgr.gibba032.whopeo (mgr.24625) 4172 : cluster [DBG] pgmap v2370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:30 gibba011 bash[31774]: cluster 2021-03-10T15:11:29.862004+0000 mgr.gibba032.whopeo (mgr.24625) 4172 : cluster [DBG] pgmap v2370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:32 gibba032 bash[24004]: cluster 2021-03-10T15:11:31.863817+0000 mgr.gibba032.whopeo (mgr.24625) 4173 : cluster [DBG] pgmap v2371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:32 gibba011 bash[30791]: cluster 2021-03-10T15:11:31.863817+0000 mgr.gibba032.whopeo (mgr.24625) 4173 : cluster [DBG] pgmap v2371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:32 gibba011 bash[31774]: cluster 2021-03-10T15:11:31.863817+0000 mgr.gibba032.whopeo (mgr.24625) 4173 : cluster [DBG] pgmap v2371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:35.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:34 gibba032 bash[24004]: cluster 2021-03-10T15:11:33.865435+0000 mgr.gibba032.whopeo (mgr.24625) 4174 : cluster [DBG] pgmap v2372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:35.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:34 gibba011 bash[30791]: cluster 2021-03-10T15:11:33.865435+0000 mgr.gibba032.whopeo (mgr.24625) 4174 : cluster [DBG] pgmap v2372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:35.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:34 gibba011 bash[31774]: cluster 2021-03-10T15:11:33.865435+0000 mgr.gibba032.whopeo (mgr.24625) 4174 : cluster [DBG] pgmap v2372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:36 gibba032 bash[24004]: cluster 2021-03-10T15:11:35.866811+0000 mgr.gibba032.whopeo (mgr.24625) 4175 : cluster [DBG] pgmap v2373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:36 gibba011 bash[30791]: cluster 2021-03-10T15:11:35.866811+0000 mgr.gibba032.whopeo (mgr.24625) 4175 : cluster [DBG] pgmap v2373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:36 gibba011 bash[31774]: cluster 2021-03-10T15:11:35.866811+0000 mgr.gibba032.whopeo (mgr.24625) 4175 : cluster [DBG] pgmap v2373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:38 gibba032 bash[24004]: cluster 2021-03-10T15:11:37.868508+0000 mgr.gibba032.whopeo (mgr.24625) 4176 : cluster [DBG] pgmap v2374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:38 gibba011 bash[30791]: cluster 2021-03-10T15:11:37.868508+0000 mgr.gibba032.whopeo (mgr.24625) 4176 : cluster [DBG] pgmap v2374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:38 gibba011 bash[31774]: cluster 2021-03-10T15:11:37.868508+0000 mgr.gibba032.whopeo (mgr.24625) 4176 : cluster [DBG] pgmap v2374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:39.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.153551+0000 mgr.gibba032.whopeo (mgr.24625) 4177 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:11:39.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.155092+0000 mgr.gibba032.whopeo (mgr.24625) 4178 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:11:39.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.379285+0000 mon.a (mon.0) 1740 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:11:39.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.379514+0000 mgr.gibba032.whopeo (mgr.24625) 4179 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:11:39.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.381369+0000 mgr.gibba032.whopeo (mgr.24625) 4180 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:11:39.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.382210+0000 mon.a (mon.0) 1741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:11:39.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.382657+0000 mgr.gibba032.whopeo (mgr.24625) 4181 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:11:39.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.382695+0000 mgr.gibba032.whopeo (mgr.24625) 4182 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:11:39.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.383718+0000 mon.a (mon.0) 1742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.383899+0000 mgr.gibba032.whopeo (mgr.24625) 4183 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:11:39.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.383968+0000 mgr.gibba032.whopeo (mgr.24625) 4184 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:11:39.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.385180+0000 mon.a (mon.0) 1743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.385180+0000 mgr.gibba032.whopeo (mgr.24625) 4185 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:11:39.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.386412+0000 mon.a (mon.0) 1744 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.386491+0000 mgr.gibba032.whopeo (mgr.24625) 4186 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:11:39.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.386528+0000 mgr.gibba032.whopeo (mgr.24625) 4187 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:11:39.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.387857+0000 mgr.gibba032.whopeo (mgr.24625) 4188 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:11:39.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.388294+0000 mgr.gibba032.whopeo (mgr.24625) 4189 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:11:39.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.389088+0000 mon.a (mon.0) 1745 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.389532+0000 mgr.gibba032.whopeo (mgr.24625) 4190 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:11:39.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.389571+0000 mgr.gibba032.whopeo (mgr.24625) 4191 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:11:39.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: cephadm 2021-03-10T15:11:38.389744+0000 mgr.gibba032.whopeo (mgr.24625) 4192 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:39.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:39 gibba032 bash[24004]: audit 2021-03-10T15:11:38.390591+0000 mon.a (mon.0) 1746 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.153551+0000 mgr.gibba032.whopeo (mgr.24625) 4177 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:11:39.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.155092+0000 mgr.gibba032.whopeo (mgr.24625) 4178 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:11:39.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.379285+0000 mon.a (mon.0) 1740 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:11:39.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.379514+0000 mgr.gibba032.whopeo (mgr.24625) 4179 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:11:39.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.381369+0000 mgr.gibba032.whopeo (mgr.24625) 4180 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:11:39.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.382210+0000 mon.a (mon.0) 1741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:11:39.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.382657+0000 mgr.gibba032.whopeo (mgr.24625) 4181 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:11:39.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.382695+0000 mgr.gibba032.whopeo (mgr.24625) 4182 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:11:39.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.383718+0000 mon.a (mon.0) 1742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.383899+0000 mgr.gibba032.whopeo (mgr.24625) 4183 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:11:39.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.383968+0000 mgr.gibba032.whopeo (mgr.24625) 4184 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:11:39.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.385180+0000 mon.a (mon.0) 1743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.385180+0000 mgr.gibba032.whopeo (mgr.24625) 4185 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:11:39.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.386412+0000 mon.a (mon.0) 1744 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.386491+0000 mgr.gibba032.whopeo (mgr.24625) 4186 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:11:39.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.386528+0000 mgr.gibba032.whopeo (mgr.24625) 4187 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:11:39.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.387857+0000 mgr.gibba032.whopeo (mgr.24625) 4188 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:11:39.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.388294+0000 mgr.gibba032.whopeo (mgr.24625) 4189 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:11:39.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.389088+0000 mon.a (mon.0) 1745 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.389532+0000 mgr.gibba032.whopeo (mgr.24625) 4190 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:11:39.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.389571+0000 mgr.gibba032.whopeo (mgr.24625) 4191 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:11:39.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: cephadm 2021-03-10T15:11:38.389744+0000 mgr.gibba032.whopeo (mgr.24625) 4192 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:39.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[30791]: audit 2021-03-10T15:11:38.390591+0000 mon.a (mon.0) 1746 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.153551+0000 mgr.gibba032.whopeo (mgr.24625) 4177 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:11:39.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.155092+0000 mgr.gibba032.whopeo (mgr.24625) 4178 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:11:39.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.379285+0000 mon.a (mon.0) 1740 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:11:39.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.379514+0000 mgr.gibba032.whopeo (mgr.24625) 4179 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:11:39.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.381369+0000 mgr.gibba032.whopeo (mgr.24625) 4180 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:11:39.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.382210+0000 mon.a (mon.0) 1741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:11:39.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.382657+0000 mgr.gibba032.whopeo (mgr.24625) 4181 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:11:39.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.382695+0000 mgr.gibba032.whopeo (mgr.24625) 4182 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:11:39.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.383718+0000 mon.a (mon.0) 1742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.383899+0000 mgr.gibba032.whopeo (mgr.24625) 4183 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:11:39.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.383968+0000 mgr.gibba032.whopeo (mgr.24625) 4184 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:11:39.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.385180+0000 mon.a (mon.0) 1743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.385180+0000 mgr.gibba032.whopeo (mgr.24625) 4185 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:11:39.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.386412+0000 mon.a (mon.0) 1744 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.386491+0000 mgr.gibba032.whopeo (mgr.24625) 4186 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:11:39.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.386528+0000 mgr.gibba032.whopeo (mgr.24625) 4187 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:11:39.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.387857+0000 mgr.gibba032.whopeo (mgr.24625) 4188 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:11:39.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.388294+0000 mgr.gibba032.whopeo (mgr.24625) 4189 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:11:39.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.389088+0000 mon.a (mon.0) 1745 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:39.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.389532+0000 mgr.gibba032.whopeo (mgr.24625) 4190 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:11:39.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.389571+0000 mgr.gibba032.whopeo (mgr.24625) 4191 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:11:39.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: cephadm 2021-03-10T15:11:38.389744+0000 mgr.gibba032.whopeo (mgr.24625) 4192 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:39.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:39 gibba011 bash[31774]: audit 2021-03-10T15:11:38.390591+0000 mon.a (mon.0) 1746 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:40 gibba032 bash[24004]: cluster 2021-03-10T15:11:39.869972+0000 mgr.gibba032.whopeo (mgr.24625) 4193 : cluster [DBG] pgmap v2375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:40 gibba011 bash[30791]: cluster 2021-03-10T15:11:39.869972+0000 mgr.gibba032.whopeo (mgr.24625) 4193 : cluster [DBG] pgmap v2375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:40 gibba011 bash[31774]: cluster 2021-03-10T15:11:39.869972+0000 mgr.gibba032.whopeo (mgr.24625) 4193 : cluster [DBG] pgmap v2375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:42 gibba032 bash[24004]: cluster 2021-03-10T15:11:41.871667+0000 mgr.gibba032.whopeo (mgr.24625) 4194 : cluster [DBG] pgmap v2376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:42 gibba011 bash[30791]: cluster 2021-03-10T15:11:41.871667+0000 mgr.gibba032.whopeo (mgr.24625) 4194 : cluster [DBG] pgmap v2376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:42 gibba011 bash[31774]: cluster 2021-03-10T15:11:41.871667+0000 mgr.gibba032.whopeo (mgr.24625) 4194 : cluster [DBG] pgmap v2376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:44.890 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:11:45.118 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:11:45.119 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (78m) 3m ago 83m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:11:45.119 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (83m) 2m ago 83m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:11:45.119 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (78m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:11:45.119 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (79m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:11:45.120 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (78m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:11:45.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (77m) 3m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:11:45.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (77m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:11:45.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (77m) 3m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:11:45.121 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (83m) 3m ago 83m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:11:45.121 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (83m) 2m ago 83m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:11:45.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (76m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:11:45.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (76m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:11:45.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (75m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:11:45.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (75m) 3m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:11:45.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (74m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:11:45.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (74m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:11:45.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (74m) 2m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:11:45.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (73m) 2m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:11:45.123 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (83m) 2m ago 83m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:11:45.123 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 81m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:11:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:44 gibba032 bash[24004]: cluster 2021-03-10T15:11:43.873323+0000 mgr.gibba032.whopeo (mgr.24625) 4195 : cluster [DBG] pgmap v2377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:44 gibba011 bash[30791]: cluster 2021-03-10T15:11:43.873323+0000 mgr.gibba032.whopeo (mgr.24625) 4195 : cluster [DBG] pgmap v2377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:45.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:44 gibba011 bash[31774]: cluster 2021-03-10T15:11:43.873323+0000 mgr.gibba032.whopeo (mgr.24625) 4195 : cluster [DBG] pgmap v2377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:45.369 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:11:45.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:11:45.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:11:45.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:45.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:11:45.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:11:45.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:45.371 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:11:45.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:11:45.372 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:11:45.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:11:45.372 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:11:45.372 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:11:45.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:11:45.373 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:11:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:45 gibba032 bash[24004]: audit 2021-03-10T15:11:44.876161+0000 mgr.gibba032.whopeo (mgr.24625) 4196 : audit [DBG] from='client.56848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:45 gibba032 bash[24004]: audit 2021-03-10T15:11:45.366869+0000 mon.a (mon.0) 1747 : audit [DBG] from='client.? 172.21.2.111:0/852116652' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:45 gibba011 bash[30791]: audit 2021-03-10T15:11:44.876161+0000 mgr.gibba032.whopeo (mgr.24625) 4196 : audit [DBG] from='client.56848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:45 gibba011 bash[30791]: audit 2021-03-10T15:11:45.366869+0000 mon.a (mon.0) 1747 : audit [DBG] from='client.? 172.21.2.111:0/852116652' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:45 gibba011 bash[31774]: audit 2021-03-10T15:11:44.876161+0000 mgr.gibba032.whopeo (mgr.24625) 4196 : audit [DBG] from='client.56848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:45 gibba011 bash[31774]: audit 2021-03-10T15:11:45.366869+0000 mon.a (mon.0) 1747 : audit [DBG] from='client.? 172.21.2.111:0/852116652' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:11:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:46 gibba032 bash[24004]: audit 2021-03-10T15:11:45.101430+0000 mgr.gibba032.whopeo (mgr.24625) 4197 : audit [DBG] from='client.46869 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:46 gibba032 bash[24004]: cluster 2021-03-10T15:11:45.874781+0000 mgr.gibba032.whopeo (mgr.24625) 4198 : cluster [DBG] pgmap v2378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:47.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:46 gibba011 bash[30791]: audit 2021-03-10T15:11:45.101430+0000 mgr.gibba032.whopeo (mgr.24625) 4197 : audit [DBG] from='client.46869 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:47.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:46 gibba011 bash[30791]: cluster 2021-03-10T15:11:45.874781+0000 mgr.gibba032.whopeo (mgr.24625) 4198 : cluster [DBG] pgmap v2378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:46 gibba011 bash[31774]: audit 2021-03-10T15:11:45.101430+0000 mgr.gibba032.whopeo (mgr.24625) 4197 : audit [DBG] from='client.46869 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:11:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:46 gibba011 bash[31774]: cluster 2021-03-10T15:11:45.874781+0000 mgr.gibba032.whopeo (mgr.24625) 4198 : cluster [DBG] pgmap v2378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:48 gibba032 bash[24004]: cluster 2021-03-10T15:11:47.876583+0000 mgr.gibba032.whopeo (mgr.24625) 4199 : cluster [DBG] pgmap v2379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:48 gibba011 bash[30791]: cluster 2021-03-10T15:11:47.876583+0000 mgr.gibba032.whopeo (mgr.24625) 4199 : cluster [DBG] pgmap v2379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:48 gibba011 bash[31774]: cluster 2021-03-10T15:11:47.876583+0000 mgr.gibba032.whopeo (mgr.24625) 4199 : cluster [DBG] pgmap v2379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:50 gibba032 bash[24004]: cluster 2021-03-10T15:11:49.878022+0000 mgr.gibba032.whopeo (mgr.24625) 4200 : cluster [DBG] pgmap v2380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:50 gibba011 bash[30791]: cluster 2021-03-10T15:11:49.878022+0000 mgr.gibba032.whopeo (mgr.24625) 4200 : cluster [DBG] pgmap v2380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:51.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:50 gibba011 bash[31774]: cluster 2021-03-10T15:11:49.878022+0000 mgr.gibba032.whopeo (mgr.24625) 4200 : cluster [DBG] pgmap v2380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:52 gibba032 bash[24004]: cluster 2021-03-10T15:11:51.879660+0000 mgr.gibba032.whopeo (mgr.24625) 4201 : cluster [DBG] pgmap v2381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:52 gibba011 bash[30791]: cluster 2021-03-10T15:11:51.879660+0000 mgr.gibba032.whopeo (mgr.24625) 4201 : cluster [DBG] pgmap v2381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:52 gibba011 bash[31774]: cluster 2021-03-10T15:11:51.879660+0000 mgr.gibba032.whopeo (mgr.24625) 4201 : cluster [DBG] pgmap v2381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:54 gibba032 bash[24004]: cephadm 2021-03-10T15:11:53.395966+0000 mgr.gibba032.whopeo (mgr.24625) 4202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:54 gibba032 bash[24004]: cluster 2021-03-10T15:11:53.881400+0000 mgr.gibba032.whopeo (mgr.24625) 4203 : cluster [DBG] pgmap v2382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:54 gibba011 bash[30791]: cephadm 2021-03-10T15:11:53.395966+0000 mgr.gibba032.whopeo (mgr.24625) 4202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:54 gibba011 bash[30791]: cluster 2021-03-10T15:11:53.881400+0000 mgr.gibba032.whopeo (mgr.24625) 4203 : cluster [DBG] pgmap v2382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:54 gibba011 bash[31774]: cephadm 2021-03-10T15:11:53.395966+0000 mgr.gibba032.whopeo (mgr.24625) 4202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:11:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:54 gibba011 bash[31774]: cluster 2021-03-10T15:11:53.881400+0000 mgr.gibba032.whopeo (mgr.24625) 4203 : cluster [DBG] pgmap v2382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: cluster 2021-03-10T15:11:55.882781+0000 mgr.gibba032.whopeo (mgr.24625) 4204 : cluster [DBG] pgmap v2383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.513525+0000 mon.a (mon.0) 1748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:11:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.514153+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:11:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.524094+0000 mon.a (mon.0) 1750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:11:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.524825+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:11:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.525475+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:11:57.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:56 gibba032 bash[24004]: audit 2021-03-10T15:11:56.526143+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:11:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: cluster 2021-03-10T15:11:55.882781+0000 mgr.gibba032.whopeo (mgr.24625) 4204 : cluster [DBG] pgmap v2383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.513525+0000 mon.a (mon.0) 1748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:11:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.514153+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:11:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.524094+0000 mon.a (mon.0) 1750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:11:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.524825+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:11:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.525475+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:11:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[30791]: audit 2021-03-10T15:11:56.526143+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:11:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: cluster 2021-03-10T15:11:55.882781+0000 mgr.gibba032.whopeo (mgr.24625) 4204 : cluster [DBG] pgmap v2383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.513525+0000 mon.a (mon.0) 1748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:11:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.514153+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:11:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.524094+0000 mon.a (mon.0) 1750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:11:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.524825+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:11:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.525475+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:11:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:56 gibba011 bash[31774]: audit 2021-03-10T15:11:56.526143+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:11:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:11:58 gibba032 bash[24004]: cluster 2021-03-10T15:11:57.884722+0000 mgr.gibba032.whopeo (mgr.24625) 4205 : cluster [DBG] pgmap v2384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:11:58 gibba011 bash[30791]: cluster 2021-03-10T15:11:57.884722+0000 mgr.gibba032.whopeo (mgr.24625) 4205 : cluster [DBG] pgmap v2384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:11:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:11:58 gibba011 bash[31774]: cluster 2021-03-10T15:11:57.884722+0000 mgr.gibba032.whopeo (mgr.24625) 4205 : cluster [DBG] pgmap v2384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:00 gibba032 bash[24004]: cluster 2021-03-10T15:11:59.886356+0000 mgr.gibba032.whopeo (mgr.24625) 4206 : cluster [DBG] pgmap v2385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:01.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:00 gibba011 bash[30791]: cluster 2021-03-10T15:11:59.886356+0000 mgr.gibba032.whopeo (mgr.24625) 4206 : cluster [DBG] pgmap v2385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:00 gibba011 bash[31774]: cluster 2021-03-10T15:11:59.886356+0000 mgr.gibba032.whopeo (mgr.24625) 4206 : cluster [DBG] pgmap v2385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:02 gibba032 bash[24004]: cluster 2021-03-10T15:12:01.888364+0000 mgr.gibba032.whopeo (mgr.24625) 4207 : cluster [DBG] pgmap v2386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:02 gibba011 bash[30791]: cluster 2021-03-10T15:12:01.888364+0000 mgr.gibba032.whopeo (mgr.24625) 4207 : cluster [DBG] pgmap v2386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:03.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:02 gibba011 bash[31774]: cluster 2021-03-10T15:12:01.888364+0000 mgr.gibba032.whopeo (mgr.24625) 4207 : cluster [DBG] pgmap v2386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:05.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:04 gibba032 bash[24004]: cluster 2021-03-10T15:12:03.890094+0000 mgr.gibba032.whopeo (mgr.24625) 4208 : cluster [DBG] pgmap v2387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:05.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:04 gibba011 bash[30791]: cluster 2021-03-10T15:12:03.890094+0000 mgr.gibba032.whopeo (mgr.24625) 4208 : cluster [DBG] pgmap v2387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:05.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:04 gibba011 bash[31774]: cluster 2021-03-10T15:12:03.890094+0000 mgr.gibba032.whopeo (mgr.24625) 4208 : cluster [DBG] pgmap v2387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:07.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:06 gibba032 bash[24004]: cluster 2021-03-10T15:12:05.891615+0000 mgr.gibba032.whopeo (mgr.24625) 4209 : cluster [DBG] pgmap v2388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:06 gibba011 bash[30791]: cluster 2021-03-10T15:12:05.891615+0000 mgr.gibba032.whopeo (mgr.24625) 4209 : cluster [DBG] pgmap v2388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:06 gibba011 bash[31774]: cluster 2021-03-10T15:12:05.891615+0000 mgr.gibba032.whopeo (mgr.24625) 4209 : cluster [DBG] pgmap v2388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:08 gibba032 bash[24004]: cluster 2021-03-10T15:12:07.893506+0000 mgr.gibba032.whopeo (mgr.24625) 4210 : cluster [DBG] pgmap v2389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:08 gibba011 bash[30791]: cluster 2021-03-10T15:12:07.893506+0000 mgr.gibba032.whopeo (mgr.24625) 4210 : cluster [DBG] pgmap v2389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:08 gibba011 bash[31774]: cluster 2021-03-10T15:12:07.893506+0000 mgr.gibba032.whopeo (mgr.24625) 4210 : cluster [DBG] pgmap v2389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:09 gibba032 bash[24004]: cephadm 2021-03-10T15:12:08.401067+0000 mgr.gibba032.whopeo (mgr.24625) 4211 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:09.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:09 gibba011 bash[30791]: cephadm 2021-03-10T15:12:08.401067+0000 mgr.gibba032.whopeo (mgr.24625) 4211 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:09 gibba011 bash[31774]: cephadm 2021-03-10T15:12:08.401067+0000 mgr.gibba032.whopeo (mgr.24625) 4211 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:10 gibba032 bash[24004]: cluster 2021-03-10T15:12:09.895184+0000 mgr.gibba032.whopeo (mgr.24625) 4212 : cluster [DBG] pgmap v2390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:10 gibba011 bash[30791]: cluster 2021-03-10T15:12:09.895184+0000 mgr.gibba032.whopeo (mgr.24625) 4212 : cluster [DBG] pgmap v2390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:10 gibba011 bash[31774]: cluster 2021-03-10T15:12:09.895184+0000 mgr.gibba032.whopeo (mgr.24625) 4212 : cluster [DBG] pgmap v2390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:12 gibba032 bash[24004]: cluster 2021-03-10T15:12:11.896934+0000 mgr.gibba032.whopeo (mgr.24625) 4213 : cluster [DBG] pgmap v2391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:12 gibba011 bash[30791]: cluster 2021-03-10T15:12:11.896934+0000 mgr.gibba032.whopeo (mgr.24625) 4213 : cluster [DBG] pgmap v2391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:12 gibba011 bash[31774]: cluster 2021-03-10T15:12:11.896934+0000 mgr.gibba032.whopeo (mgr.24625) 4213 : cluster [DBG] pgmap v2391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:15.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:14 gibba032 bash[24004]: cluster 2021-03-10T15:12:13.898477+0000 mgr.gibba032.whopeo (mgr.24625) 4214 : cluster [DBG] pgmap v2392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:14 gibba011 bash[30791]: cluster 2021-03-10T15:12:13.898477+0000 mgr.gibba032.whopeo (mgr.24625) 4214 : cluster [DBG] pgmap v2392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:14 gibba011 bash[31774]: cluster 2021-03-10T15:12:13.898477+0000 mgr.gibba032.whopeo (mgr.24625) 4214 : cluster [DBG] pgmap v2392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:15.613 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:12:15.843 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:12:15.843 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (78m) 3m ago 84m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:12:15.844 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (84m) 2m ago 84m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:12:15.844 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (79m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:12:15.844 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (79m) 2m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:12:15.844 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (78m) 3m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:12:15.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (78m) 3m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:12:15.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (77m) 2m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:12:15.845 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (78m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:12:15.845 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (84m) 3m ago 84m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:12:15.846 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (84m) 2m ago 84m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:12:15.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (77m) 3m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:12:15.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (76m) 3m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:12:15.846 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (76m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:12:15.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (75m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:12:15.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (75m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:12:15.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (75m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:12:15.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (74m) 2m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:12:15.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (74m) 2m ago 84m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:12:15.848 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (83m) 2m ago 84m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:12:15.848 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 82m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:12:16.091 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:12:16.092 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:12:16.092 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:12:16.092 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:16.092 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:12:16.093 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:12:16.093 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:16.093 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:12:16.093 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:12:16.093 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:16.094 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:12:16.094 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:12:16.094 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:12:16.094 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:12:16.095 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:12:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:16 gibba032 bash[24004]: audit 2021-03-10T15:12:15.600474+0000 mgr.gibba032.whopeo (mgr.24625) 4215 : audit [DBG] from='client.56866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:16 gibba032 bash[24004]: audit 2021-03-10T15:12:15.827740+0000 mgr.gibba032.whopeo (mgr.24625) 4216 : audit [DBG] from='client.46887 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:16 gibba032 bash[24004]: cluster 2021-03-10T15:12:15.899509+0000 mgr.gibba032.whopeo (mgr.24625) 4217 : cluster [DBG] pgmap v2393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:17.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:16 gibba032 bash[24004]: audit 2021-03-10T15:12:16.088944+0000 mon.a (mon.0) 1754 : audit [DBG] from='client.? 172.21.2.111:0/2983715905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[30791]: audit 2021-03-10T15:12:15.600474+0000 mgr.gibba032.whopeo (mgr.24625) 4215 : audit [DBG] from='client.56866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[30791]: audit 2021-03-10T15:12:15.827740+0000 mgr.gibba032.whopeo (mgr.24625) 4216 : audit [DBG] from='client.46887 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[30791]: cluster 2021-03-10T15:12:15.899509+0000 mgr.gibba032.whopeo (mgr.24625) 4217 : cluster [DBG] pgmap v2393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[30791]: audit 2021-03-10T15:12:16.088944+0000 mon.a (mon.0) 1754 : audit [DBG] from='client.? 172.21.2.111:0/2983715905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[31774]: audit 2021-03-10T15:12:15.600474+0000 mgr.gibba032.whopeo (mgr.24625) 4215 : audit [DBG] from='client.56866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[31774]: audit 2021-03-10T15:12:15.827740+0000 mgr.gibba032.whopeo (mgr.24625) 4216 : audit [DBG] from='client.46887 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[31774]: cluster 2021-03-10T15:12:15.899509+0000 mgr.gibba032.whopeo (mgr.24625) 4217 : cluster [DBG] pgmap v2393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:16 gibba011 bash[31774]: audit 2021-03-10T15:12:16.088944+0000 mon.a (mon.0) 1754 : audit [DBG] from='client.? 172.21.2.111:0/2983715905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:18 gibba032 bash[24004]: cluster 2021-03-10T15:12:17.901283+0000 mgr.gibba032.whopeo (mgr.24625) 4218 : cluster [DBG] pgmap v2394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:18 gibba011 bash[30791]: cluster 2021-03-10T15:12:17.901283+0000 mgr.gibba032.whopeo (mgr.24625) 4218 : cluster [DBG] pgmap v2394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:18 gibba011 bash[31774]: cluster 2021-03-10T15:12:17.901283+0000 mgr.gibba032.whopeo (mgr.24625) 4218 : cluster [DBG] pgmap v2394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:21.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:20 gibba032 bash[24004]: cluster 2021-03-10T15:12:19.902669+0000 mgr.gibba032.whopeo (mgr.24625) 4219 : cluster [DBG] pgmap v2395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:21.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:20 gibba011 bash[30791]: cluster 2021-03-10T15:12:19.902669+0000 mgr.gibba032.whopeo (mgr.24625) 4219 : cluster [DBG] pgmap v2395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:21.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:20 gibba011 bash[31774]: cluster 2021-03-10T15:12:19.902669+0000 mgr.gibba032.whopeo (mgr.24625) 4219 : cluster [DBG] pgmap v2395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:22 gibba032 bash[24004]: cluster 2021-03-10T15:12:21.904350+0000 mgr.gibba032.whopeo (mgr.24625) 4220 : cluster [DBG] pgmap v2396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:22 gibba011 bash[30791]: cluster 2021-03-10T15:12:21.904350+0000 mgr.gibba032.whopeo (mgr.24625) 4220 : cluster [DBG] pgmap v2396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:23.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:22 gibba011 bash[31774]: cluster 2021-03-10T15:12:21.904350+0000 mgr.gibba032.whopeo (mgr.24625) 4220 : cluster [DBG] pgmap v2396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:24 gibba032 bash[24004]: cephadm 2021-03-10T15:12:23.417026+0000 mgr.gibba032.whopeo (mgr.24625) 4221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:24 gibba032 bash[24004]: cluster 2021-03-10T15:12:23.905649+0000 mgr.gibba032.whopeo (mgr.24625) 4222 : cluster [DBG] pgmap v2397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:24 gibba011 bash[30791]: cephadm 2021-03-10T15:12:23.417026+0000 mgr.gibba032.whopeo (mgr.24625) 4221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:24 gibba011 bash[30791]: cluster 2021-03-10T15:12:23.905649+0000 mgr.gibba032.whopeo (mgr.24625) 4222 : cluster [DBG] pgmap v2397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:24 gibba011 bash[31774]: cephadm 2021-03-10T15:12:23.417026+0000 mgr.gibba032.whopeo (mgr.24625) 4221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:24 gibba011 bash[31774]: cluster 2021-03-10T15:12:23.905649+0000 mgr.gibba032.whopeo (mgr.24625) 4222 : cluster [DBG] pgmap v2397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:27.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:26 gibba032 bash[24004]: cluster 2021-03-10T15:12:25.907003+0000 mgr.gibba032.whopeo (mgr.24625) 4223 : cluster [DBG] pgmap v2398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:27.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:26 gibba011 bash[30791]: cluster 2021-03-10T15:12:25.907003+0000 mgr.gibba032.whopeo (mgr.24625) 4223 : cluster [DBG] pgmap v2398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:27.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:26 gibba011 bash[31774]: cluster 2021-03-10T15:12:25.907003+0000 mgr.gibba032.whopeo (mgr.24625) 4223 : cluster [DBG] pgmap v2398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:28 gibba032 bash[24004]: cluster 2021-03-10T15:12:27.908817+0000 mgr.gibba032.whopeo (mgr.24625) 4224 : cluster [DBG] pgmap v2399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:28 gibba011 bash[30791]: cluster 2021-03-10T15:12:27.908817+0000 mgr.gibba032.whopeo (mgr.24625) 4224 : cluster [DBG] pgmap v2399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:28 gibba011 bash[31774]: cluster 2021-03-10T15:12:27.908817+0000 mgr.gibba032.whopeo (mgr.24625) 4224 : cluster [DBG] pgmap v2399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:31.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:30 gibba032 bash[24004]: cluster 2021-03-10T15:12:29.910124+0000 mgr.gibba032.whopeo (mgr.24625) 4225 : cluster [DBG] pgmap v2400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:31.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:30 gibba011 bash[30791]: cluster 2021-03-10T15:12:29.910124+0000 mgr.gibba032.whopeo (mgr.24625) 4225 : cluster [DBG] pgmap v2400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:30 gibba011 bash[31774]: cluster 2021-03-10T15:12:29.910124+0000 mgr.gibba032.whopeo (mgr.24625) 4225 : cluster [DBG] pgmap v2400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:32 gibba032 bash[24004]: cluster 2021-03-10T15:12:31.911910+0000 mgr.gibba032.whopeo (mgr.24625) 4226 : cluster [DBG] pgmap v2401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:32 gibba011 bash[30791]: cluster 2021-03-10T15:12:31.911910+0000 mgr.gibba032.whopeo (mgr.24625) 4226 : cluster [DBG] pgmap v2401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:32 gibba011 bash[31774]: cluster 2021-03-10T15:12:31.911910+0000 mgr.gibba032.whopeo (mgr.24625) 4226 : cluster [DBG] pgmap v2401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:35.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:34 gibba032 bash[24004]: cluster 2021-03-10T15:12:33.913521+0000 mgr.gibba032.whopeo (mgr.24625) 4227 : cluster [DBG] pgmap v2402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:35.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:34 gibba011 bash[30791]: cluster 2021-03-10T15:12:33.913521+0000 mgr.gibba032.whopeo (mgr.24625) 4227 : cluster [DBG] pgmap v2402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:35.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:34 gibba011 bash[31774]: cluster 2021-03-10T15:12:33.913521+0000 mgr.gibba032.whopeo (mgr.24625) 4227 : cluster [DBG] pgmap v2402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:37 gibba032 bash[24004]: cluster 2021-03-10T15:12:35.914844+0000 mgr.gibba032.whopeo (mgr.24625) 4228 : cluster [DBG] pgmap v2403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:37 gibba011 bash[30791]: cluster 2021-03-10T15:12:35.914844+0000 mgr.gibba032.whopeo (mgr.24625) 4228 : cluster [DBG] pgmap v2403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:37 gibba011 bash[31774]: cluster 2021-03-10T15:12:35.914844+0000 mgr.gibba032.whopeo (mgr.24625) 4228 : cluster [DBG] pgmap v2403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:38 gibba032 bash[24004]: cluster 2021-03-10T15:12:37.916567+0000 mgr.gibba032.whopeo (mgr.24625) 4229 : cluster [DBG] pgmap v2404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:38 gibba011 bash[30791]: cluster 2021-03-10T15:12:37.916567+0000 mgr.gibba032.whopeo (mgr.24625) 4229 : cluster [DBG] pgmap v2404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:38 gibba011 bash[31774]: cluster 2021-03-10T15:12:37.916567+0000 mgr.gibba032.whopeo (mgr.24625) 4229 : cluster [DBG] pgmap v2404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.429662+0000 mgr.gibba032.whopeo (mgr.24625) 4230 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:12:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.431207+0000 mgr.gibba032.whopeo (mgr.24625) 4231 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:12:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.639973+0000 mon.a (mon.0) 1755 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:12:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.640665+0000 mgr.gibba032.whopeo (mgr.24625) 4232 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:12:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.642636+0000 mgr.gibba032.whopeo (mgr.24625) 4233 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:12:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.643462+0000 mon.a (mon.0) 1756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:12:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.643940+0000 mgr.gibba032.whopeo (mgr.24625) 4234 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:12:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.643980+0000 mgr.gibba032.whopeo (mgr.24625) 4235 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:12:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.644978+0000 mon.a (mon.0) 1757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.645295+0000 mgr.gibba032.whopeo (mgr.24625) 4236 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:12:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.645331+0000 mgr.gibba032.whopeo (mgr.24625) 4237 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:12:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.646411+0000 mon.a (mon.0) 1758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.646544+0000 mgr.gibba032.whopeo (mgr.24625) 4238 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:12:40.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.647597+0000 mgr.gibba032.whopeo (mgr.24625) 4239 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:12:40.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.647633+0000 mgr.gibba032.whopeo (mgr.24625) 4240 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:12:40.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.647635+0000 mon.a (mon.0) 1759 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.649147+0000 mgr.gibba032.whopeo (mgr.24625) 4241 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:12:40.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.649456+0000 mgr.gibba032.whopeo (mgr.24625) 4242 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:12:40.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.650144+0000 mon.a (mon.0) 1760 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.650548+0000 mgr.gibba032.whopeo (mgr.24625) 4243 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:12:40.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.650586+0000 mgr.gibba032.whopeo (mgr.24625) 4244 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:12:40.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: cephadm 2021-03-10T15:12:38.650689+0000 mgr.gibba032.whopeo (mgr.24625) 4245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:40.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:39 gibba032 bash[24004]: audit 2021-03-10T15:12:38.651725+0000 mon.a (mon.0) 1761 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.429662+0000 mgr.gibba032.whopeo (mgr.24625) 4230 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:12:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.431207+0000 mgr.gibba032.whopeo (mgr.24625) 4231 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:12:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.639973+0000 mon.a (mon.0) 1755 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:12:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.640665+0000 mgr.gibba032.whopeo (mgr.24625) 4232 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:12:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.642636+0000 mgr.gibba032.whopeo (mgr.24625) 4233 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:12:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.643462+0000 mon.a (mon.0) 1756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:12:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.643940+0000 mgr.gibba032.whopeo (mgr.24625) 4234 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:12:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.643980+0000 mgr.gibba032.whopeo (mgr.24625) 4235 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:12:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.644978+0000 mon.a (mon.0) 1757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.645295+0000 mgr.gibba032.whopeo (mgr.24625) 4236 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:12:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.645331+0000 mgr.gibba032.whopeo (mgr.24625) 4237 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:12:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.646411+0000 mon.a (mon.0) 1758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.646544+0000 mgr.gibba032.whopeo (mgr.24625) 4238 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:12:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.647597+0000 mgr.gibba032.whopeo (mgr.24625) 4239 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:12:40.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.647633+0000 mgr.gibba032.whopeo (mgr.24625) 4240 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:12:40.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.647635+0000 mon.a (mon.0) 1759 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.649147+0000 mgr.gibba032.whopeo (mgr.24625) 4241 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:12:40.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.649456+0000 mgr.gibba032.whopeo (mgr.24625) 4242 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:12:40.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.650144+0000 mon.a (mon.0) 1760 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.650548+0000 mgr.gibba032.whopeo (mgr.24625) 4243 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:12:40.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.650586+0000 mgr.gibba032.whopeo (mgr.24625) 4244 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:12:40.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: cephadm 2021-03-10T15:12:38.650689+0000 mgr.gibba032.whopeo (mgr.24625) 4245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:40.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[30791]: audit 2021-03-10T15:12:38.651725+0000 mon.a (mon.0) 1761 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.429662+0000 mgr.gibba032.whopeo (mgr.24625) 4230 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:12:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.431207+0000 mgr.gibba032.whopeo (mgr.24625) 4231 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:12:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.639973+0000 mon.a (mon.0) 1755 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:12:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.640665+0000 mgr.gibba032.whopeo (mgr.24625) 4232 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:12:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.642636+0000 mgr.gibba032.whopeo (mgr.24625) 4233 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:12:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.643462+0000 mon.a (mon.0) 1756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:12:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.643940+0000 mgr.gibba032.whopeo (mgr.24625) 4234 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:12:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.643980+0000 mgr.gibba032.whopeo (mgr.24625) 4235 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:12:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.644978+0000 mon.a (mon.0) 1757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.645295+0000 mgr.gibba032.whopeo (mgr.24625) 4236 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:12:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.645331+0000 mgr.gibba032.whopeo (mgr.24625) 4237 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:12:40.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.646411+0000 mon.a (mon.0) 1758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.646544+0000 mgr.gibba032.whopeo (mgr.24625) 4238 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:12:40.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.647597+0000 mgr.gibba032.whopeo (mgr.24625) 4239 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:12:40.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.647633+0000 mgr.gibba032.whopeo (mgr.24625) 4240 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:12:40.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.647635+0000 mon.a (mon.0) 1759 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.649147+0000 mgr.gibba032.whopeo (mgr.24625) 4241 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:12:40.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.649456+0000 mgr.gibba032.whopeo (mgr.24625) 4242 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:12:40.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.650144+0000 mon.a (mon.0) 1760 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:40.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.650548+0000 mgr.gibba032.whopeo (mgr.24625) 4243 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:12:40.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.650586+0000 mgr.gibba032.whopeo (mgr.24625) 4244 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:12:40.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: cephadm 2021-03-10T15:12:38.650689+0000 mgr.gibba032.whopeo (mgr.24625) 4245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:40.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:39 gibba011 bash[31774]: audit 2021-03-10T15:12:38.651725+0000 mon.a (mon.0) 1761 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:40 gibba032 bash[24004]: cluster 2021-03-10T15:12:39.917857+0000 mgr.gibba032.whopeo (mgr.24625) 4246 : cluster [DBG] pgmap v2405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:40 gibba011 bash[30791]: cluster 2021-03-10T15:12:39.917857+0000 mgr.gibba032.whopeo (mgr.24625) 4246 : cluster [DBG] pgmap v2405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:40 gibba011 bash[31774]: cluster 2021-03-10T15:12:39.917857+0000 mgr.gibba032.whopeo (mgr.24625) 4246 : cluster [DBG] pgmap v2405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:42 gibba032 bash[24004]: cluster 2021-03-10T15:12:41.919651+0000 mgr.gibba032.whopeo (mgr.24625) 4247 : cluster [DBG] pgmap v2406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:42 gibba011 bash[30791]: cluster 2021-03-10T15:12:41.919651+0000 mgr.gibba032.whopeo (mgr.24625) 4247 : cluster [DBG] pgmap v2406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:42 gibba011 bash[31774]: cluster 2021-03-10T15:12:41.919651+0000 mgr.gibba032.whopeo (mgr.24625) 4247 : cluster [DBG] pgmap v2406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:44 gibba032 bash[24004]: cluster 2021-03-10T15:12:43.921160+0000 mgr.gibba032.whopeo (mgr.24625) 4248 : cluster [DBG] pgmap v2407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:44 gibba011 bash[30791]: cluster 2021-03-10T15:12:43.921160+0000 mgr.gibba032.whopeo (mgr.24625) 4248 : cluster [DBG] pgmap v2407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:44 gibba011 bash[31774]: cluster 2021-03-10T15:12:43.921160+0000 mgr.gibba032.whopeo (mgr.24625) 4248 : cluster [DBG] pgmap v2407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:46.334 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:12:46.560 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:12:46.560 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (79m) 4m ago 84m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:12:46.560 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (84m) 3m ago 84m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:12:46.561 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (79m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:12:46.561 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (80m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:12:46.561 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (79m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:12:46.561 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (78m) 4m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:12:46.562 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (78m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:12:46.562 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (78m) 4m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:12:46.562 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (84m) 4m ago 84m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:12:46.562 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (84m) 3m ago 84m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:12:46.563 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (77m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:12:46.563 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (77m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:12:46.563 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (76m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:12:46.563 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (76m) 4m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:12:46.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (76m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:12:46.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (75m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:12:46.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (75m) 3m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:12:46.564 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (74m) 3m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:12:46.564 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (84m) 3m ago 84m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:12:46.565 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 82m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:12:46.809 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:12:46.809 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:12:46.809 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:12:46.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:46.810 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:12:46.810 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:12:46.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:46.811 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:12:46.811 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:12:46.811 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:12:46.811 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:12:46.811 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:12:46.812 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:12:46.812 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:12:46.812 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:12:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:47 gibba032 bash[24004]: cluster 2021-03-10T15:12:45.922457+0000 mgr.gibba032.whopeo (mgr.24625) 4249 : cluster [DBG] pgmap v2408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:47.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:47 gibba032 bash[24004]: audit 2021-03-10T15:12:46.806695+0000 mon.a (mon.0) 1762 : audit [DBG] from='client.? 172.21.2.111:0/2149116497' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:47.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:47 gibba011 bash[30791]: cluster 2021-03-10T15:12:45.922457+0000 mgr.gibba032.whopeo (mgr.24625) 4249 : cluster [DBG] pgmap v2408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:47.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:47 gibba011 bash[30791]: audit 2021-03-10T15:12:46.806695+0000 mon.a (mon.0) 1762 : audit [DBG] from='client.? 172.21.2.111:0/2149116497' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:47.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:47 gibba011 bash[31774]: cluster 2021-03-10T15:12:45.922457+0000 mgr.gibba032.whopeo (mgr.24625) 4249 : cluster [DBG] pgmap v2408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:47.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:47 gibba011 bash[31774]: audit 2021-03-10T15:12:46.806695+0000 mon.a (mon.0) 1762 : audit [DBG] from='client.? 172.21.2.111:0/2149116497' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:12:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:48 gibba011 bash[30791]: audit 2021-03-10T15:12:46.321183+0000 mgr.gibba032.whopeo (mgr.24625) 4250 : audit [DBG] from='client.46899 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:48 gibba011 bash[30791]: audit 2021-03-10T15:12:46.543187+0000 mgr.gibba032.whopeo (mgr.24625) 4251 : audit [DBG] from='client.46905 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:48 gibba011 bash[31774]: audit 2021-03-10T15:12:46.321183+0000 mgr.gibba032.whopeo (mgr.24625) 4250 : audit [DBG] from='client.46899 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:48 gibba011 bash[31774]: audit 2021-03-10T15:12:46.543187+0000 mgr.gibba032.whopeo (mgr.24625) 4251 : audit [DBG] from='client.46905 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:48 gibba032 bash[24004]: audit 2021-03-10T15:12:46.321183+0000 mgr.gibba032.whopeo (mgr.24625) 4250 : audit [DBG] from='client.46899 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:48.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:48 gibba032 bash[24004]: audit 2021-03-10T15:12:46.543187+0000 mgr.gibba032.whopeo (mgr.24625) 4251 : audit [DBG] from='client.46905 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:12:49.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:49 gibba011 bash[30791]: cluster 2021-03-10T15:12:47.924305+0000 mgr.gibba032.whopeo (mgr.24625) 4252 : cluster [DBG] pgmap v2409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:49.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:49 gibba011 bash[31774]: cluster 2021-03-10T15:12:47.924305+0000 mgr.gibba032.whopeo (mgr.24625) 4252 : cluster [DBG] pgmap v2409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:49 gibba032 bash[24004]: cluster 2021-03-10T15:12:47.924305+0000 mgr.gibba032.whopeo (mgr.24625) 4252 : cluster [DBG] pgmap v2409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:51 gibba011 bash[30791]: cluster 2021-03-10T15:12:49.925795+0000 mgr.gibba032.whopeo (mgr.24625) 4253 : cluster [DBG] pgmap v2410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:51.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:51 gibba011 bash[31774]: cluster 2021-03-10T15:12:49.925795+0000 mgr.gibba032.whopeo (mgr.24625) 4253 : cluster [DBG] pgmap v2410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:51 gibba032 bash[24004]: cluster 2021-03-10T15:12:49.925795+0000 mgr.gibba032.whopeo (mgr.24625) 4253 : cluster [DBG] pgmap v2410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:53 gibba011 bash[30791]: cluster 2021-03-10T15:12:51.927544+0000 mgr.gibba032.whopeo (mgr.24625) 4254 : cluster [DBG] pgmap v2411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:53 gibba011 bash[31774]: cluster 2021-03-10T15:12:51.927544+0000 mgr.gibba032.whopeo (mgr.24625) 4254 : cluster [DBG] pgmap v2411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:53 gibba032 bash[24004]: cluster 2021-03-10T15:12:51.927544+0000 mgr.gibba032.whopeo (mgr.24625) 4254 : cluster [DBG] pgmap v2411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:55 gibba011 bash[30791]: cephadm 2021-03-10T15:12:53.666686+0000 mgr.gibba032.whopeo (mgr.24625) 4255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:55 gibba011 bash[30791]: cluster 2021-03-10T15:12:53.929135+0000 mgr.gibba032.whopeo (mgr.24625) 4256 : cluster [DBG] pgmap v2412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:55 gibba011 bash[31774]: cephadm 2021-03-10T15:12:53.666686+0000 mgr.gibba032.whopeo (mgr.24625) 4255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:55.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:55 gibba011 bash[31774]: cluster 2021-03-10T15:12:53.929135+0000 mgr.gibba032.whopeo (mgr.24625) 4256 : cluster [DBG] pgmap v2412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:55 gibba032 bash[24004]: cephadm 2021-03-10T15:12:53.666686+0000 mgr.gibba032.whopeo (mgr.24625) 4255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:12:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:55 gibba032 bash[24004]: cluster 2021-03-10T15:12:53.929135+0000 mgr.gibba032.whopeo (mgr.24625) 4256 : cluster [DBG] pgmap v2412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: cluster 2021-03-10T15:12:55.930467+0000 mgr.gibba032.whopeo (mgr.24625) 4257 : cluster [DBG] pgmap v2413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:57.362 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.520200+0000 mon.a (mon.0) 1763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:12:57.363 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.520808+0000 mon.a (mon.0) 1764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:12:57.363 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.529442+0000 mon.a (mon.0) 1765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:12:57.363 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.530174+0000 mon.a (mon.0) 1766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:12:57.363 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.530864+0000 mon.a (mon.0) 1767 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:12:57.364 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[30791]: audit 2021-03-10T15:12:56.531691+0000 mon.a (mon.0) 1768 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:12:57.364 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: cluster 2021-03-10T15:12:55.930467+0000 mgr.gibba032.whopeo (mgr.24625) 4257 : cluster [DBG] pgmap v2413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:57.364 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.520200+0000 mon.a (mon.0) 1763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:12:57.364 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.520808+0000 mon.a (mon.0) 1764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:12:57.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.529442+0000 mon.a (mon.0) 1765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:12:57.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.530174+0000 mon.a (mon.0) 1766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:12:57.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.530864+0000 mon.a (mon.0) 1767 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:12:57.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:57 gibba011 bash[31774]: audit 2021-03-10T15:12:56.531691+0000 mon.a (mon.0) 1768 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:12:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: cluster 2021-03-10T15:12:55.930467+0000 mgr.gibba032.whopeo (mgr.24625) 4257 : cluster [DBG] pgmap v2413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.520200+0000 mon.a (mon.0) 1763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:12:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.520808+0000 mon.a (mon.0) 1764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:12:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.529442+0000 mon.a (mon.0) 1765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:12:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.530174+0000 mon.a (mon.0) 1766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:12:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.530864+0000 mon.a (mon.0) 1767 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:12:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:57 gibba032 bash[24004]: audit 2021-03-10T15:12:56.531691+0000 mon.a (mon.0) 1768 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:12:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:12:58 gibba011 bash[30791]: cluster 2021-03-10T15:12:57.932271+0000 mgr.gibba032.whopeo (mgr.24625) 4258 : cluster [DBG] pgmap v2414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:12:58 gibba011 bash[31774]: cluster 2021-03-10T15:12:57.932271+0000 mgr.gibba032.whopeo (mgr.24625) 4258 : cluster [DBG] pgmap v2414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:12:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:12:58 gibba032 bash[24004]: cluster 2021-03-10T15:12:57.932271+0000 mgr.gibba032.whopeo (mgr.24625) 4258 : cluster [DBG] pgmap v2414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:00 gibba032 bash[24004]: cluster 2021-03-10T15:12:59.933608+0000 mgr.gibba032.whopeo (mgr.24625) 4259 : cluster [DBG] pgmap v2415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:01.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:00 gibba011 bash[30791]: cluster 2021-03-10T15:12:59.933608+0000 mgr.gibba032.whopeo (mgr.24625) 4259 : cluster [DBG] pgmap v2415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:00 gibba011 bash[31774]: cluster 2021-03-10T15:12:59.933608+0000 mgr.gibba032.whopeo (mgr.24625) 4259 : cluster [DBG] pgmap v2415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:03 gibba011 bash[30791]: cluster 2021-03-10T15:13:01.935314+0000 mgr.gibba032.whopeo (mgr.24625) 4260 : cluster [DBG] pgmap v2416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:03.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:03 gibba011 bash[31774]: cluster 2021-03-10T15:13:01.935314+0000 mgr.gibba032.whopeo (mgr.24625) 4260 : cluster [DBG] pgmap v2416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:03 gibba032 bash[24004]: cluster 2021-03-10T15:13:01.935314+0000 mgr.gibba032.whopeo (mgr.24625) 4260 : cluster [DBG] pgmap v2416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:05.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:05 gibba011 bash[30791]: cluster 2021-03-10T15:13:03.936879+0000 mgr.gibba032.whopeo (mgr.24625) 4261 : cluster [DBG] pgmap v2417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:05.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:05 gibba011 bash[31774]: cluster 2021-03-10T15:13:03.936879+0000 mgr.gibba032.whopeo (mgr.24625) 4261 : cluster [DBG] pgmap v2417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:05 gibba032 bash[24004]: cluster 2021-03-10T15:13:03.936879+0000 mgr.gibba032.whopeo (mgr.24625) 4261 : cluster [DBG] pgmap v2417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:07 gibba011 bash[30791]: cluster 2021-03-10T15:13:05.938233+0000 mgr.gibba032.whopeo (mgr.24625) 4262 : cluster [DBG] pgmap v2418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:07 gibba011 bash[31774]: cluster 2021-03-10T15:13:05.938233+0000 mgr.gibba032.whopeo (mgr.24625) 4262 : cluster [DBG] pgmap v2418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:07 gibba032 bash[24004]: cluster 2021-03-10T15:13:05.938233+0000 mgr.gibba032.whopeo (mgr.24625) 4262 : cluster [DBG] pgmap v2418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:08 gibba011 bash[30791]: cluster 2021-03-10T15:13:07.939990+0000 mgr.gibba032.whopeo (mgr.24625) 4263 : cluster [DBG] pgmap v2419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:08 gibba011 bash[31774]: cluster 2021-03-10T15:13:07.939990+0000 mgr.gibba032.whopeo (mgr.24625) 4263 : cluster [DBG] pgmap v2419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:08.678 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:08 gibba032 bash[24004]: cluster 2021-03-10T15:13:07.939990+0000 mgr.gibba032.whopeo (mgr.24625) 4263 : cluster [DBG] pgmap v2419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:09.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:09 gibba011 bash[30791]: cephadm 2021-03-10T15:13:08.673159+0000 mgr.gibba032.whopeo (mgr.24625) 4264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:09 gibba011 bash[31774]: cephadm 2021-03-10T15:13:08.673159+0000 mgr.gibba032.whopeo (mgr.24625) 4264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:09 gibba032 bash[24004]: cephadm 2021-03-10T15:13:08.673159+0000 mgr.gibba032.whopeo (mgr.24625) 4264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:10 gibba011 bash[30791]: cluster 2021-03-10T15:13:09.941450+0000 mgr.gibba032.whopeo (mgr.24625) 4265 : cluster [DBG] pgmap v2420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:10 gibba011 bash[31774]: cluster 2021-03-10T15:13:09.941450+0000 mgr.gibba032.whopeo (mgr.24625) 4265 : cluster [DBG] pgmap v2420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:10 gibba032 bash[24004]: cluster 2021-03-10T15:13:09.941450+0000 mgr.gibba032.whopeo (mgr.24625) 4265 : cluster [DBG] pgmap v2420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:13 gibba011 bash[30791]: cluster 2021-03-10T15:13:11.943251+0000 mgr.gibba032.whopeo (mgr.24625) 4266 : cluster [DBG] pgmap v2421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:13 gibba011 bash[31774]: cluster 2021-03-10T15:13:11.943251+0000 mgr.gibba032.whopeo (mgr.24625) 4266 : cluster [DBG] pgmap v2421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:13 gibba032 bash[24004]: cluster 2021-03-10T15:13:11.943251+0000 mgr.gibba032.whopeo (mgr.24625) 4266 : cluster [DBG] pgmap v2421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:15 gibba011 bash[30791]: cluster 2021-03-10T15:13:13.944899+0000 mgr.gibba032.whopeo (mgr.24625) 4267 : cluster [DBG] pgmap v2422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:15 gibba011 bash[31774]: cluster 2021-03-10T15:13:13.944899+0000 mgr.gibba032.whopeo (mgr.24625) 4267 : cluster [DBG] pgmap v2422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:15 gibba032 bash[24004]: cluster 2021-03-10T15:13:13.944899+0000 mgr.gibba032.whopeo (mgr.24625) 4267 : cluster [DBG] pgmap v2422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:17.049 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:13:17.275 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:13:17.275 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (79m) 4m ago 85m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:13:17.275 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (85m) 3m ago 85m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:13:17.275 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (80m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:13:17.276 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (80m) 3m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:13:17.276 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (79m) 4m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:13:17.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (79m) 4m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:13:17.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (79m) 3m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:13:17.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (79m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:13:17.277 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (85m) 4m ago 85m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:13:17.277 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (85m) 3m ago 85m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:13:17.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (78m) 4m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:13:17.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (77m) 4m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:13:17.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (77m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:13:17.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (76m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:13:17.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (76m) 3m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:13:17.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (76m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:13:17.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (75m) 3m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:13:17.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (75m) 3m ago 85m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:13:17.279 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (84m) 3m ago 85m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:13:17.279 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 83m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:13:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:17 gibba011 bash[30791]: cluster 2021-03-10T15:13:15.946210+0000 mgr.gibba032.whopeo (mgr.24625) 4268 : cluster [DBG] pgmap v2423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:17.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:17 gibba011 bash[31774]: cluster 2021-03-10T15:13:15.946210+0000 mgr.gibba032.whopeo (mgr.24625) 4268 : cluster [DBG] pgmap v2423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:17 gibba032 bash[24004]: cluster 2021-03-10T15:13:15.946210+0000 mgr.gibba032.whopeo (mgr.24625) 4268 : cluster [DBG] pgmap v2423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:17.524 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:13:17.524 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:13:17.524 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:13:17.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:17.525 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:13:17.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:13:17.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:17.525 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:13:17.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:13:17.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:17.526 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:13:17.526 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:13:17.526 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:13:17.526 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:13:17.527 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:13:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:18 gibba011 bash[30791]: audit 2021-03-10T15:13:17.036253+0000 mgr.gibba032.whopeo (mgr.24625) 4269 : audit [DBG] from='client.56902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:18 gibba011 bash[30791]: audit 2021-03-10T15:13:17.521401+0000 mon.c (mon.1) 81 : audit [DBG] from='client.? 172.21.2.111:0/1530670146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:18 gibba011 bash[31774]: audit 2021-03-10T15:13:17.036253+0000 mgr.gibba032.whopeo (mgr.24625) 4269 : audit [DBG] from='client.56902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:18 gibba011 bash[31774]: audit 2021-03-10T15:13:17.521401+0000 mon.c (mon.1) 81 : audit [DBG] from='client.? 172.21.2.111:0/1530670146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:18 gibba032 bash[24004]: audit 2021-03-10T15:13:17.036253+0000 mgr.gibba032.whopeo (mgr.24625) 4269 : audit [DBG] from='client.56902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:18 gibba032 bash[24004]: audit 2021-03-10T15:13:17.521401+0000 mon.c (mon.1) 81 : audit [DBG] from='client.? 172.21.2.111:0/1530670146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:19 gibba011 bash[30791]: audit 2021-03-10T15:13:17.257888+0000 mgr.gibba032.whopeo (mgr.24625) 4270 : audit [DBG] from='client.46923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:19.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:19 gibba011 bash[30791]: cluster 2021-03-10T15:13:17.948003+0000 mgr.gibba032.whopeo (mgr.24625) 4271 : cluster [DBG] pgmap v2424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:19.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:19 gibba011 bash[31774]: audit 2021-03-10T15:13:17.257888+0000 mgr.gibba032.whopeo (mgr.24625) 4270 : audit [DBG] from='client.46923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:19.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:19 gibba011 bash[31774]: cluster 2021-03-10T15:13:17.948003+0000 mgr.gibba032.whopeo (mgr.24625) 4271 : cluster [DBG] pgmap v2424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:19 gibba032 bash[24004]: audit 2021-03-10T15:13:17.257888+0000 mgr.gibba032.whopeo (mgr.24625) 4270 : audit [DBG] from='client.46923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:19.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:19 gibba032 bash[24004]: cluster 2021-03-10T15:13:17.948003+0000 mgr.gibba032.whopeo (mgr.24625) 4271 : cluster [DBG] pgmap v2424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:21.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:21 gibba011 bash[30791]: cluster 2021-03-10T15:13:19.949551+0000 mgr.gibba032.whopeo (mgr.24625) 4272 : cluster [DBG] pgmap v2425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:21.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:21 gibba011 bash[31774]: cluster 2021-03-10T15:13:19.949551+0000 mgr.gibba032.whopeo (mgr.24625) 4272 : cluster [DBG] pgmap v2425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:21 gibba032 bash[24004]: cluster 2021-03-10T15:13:19.949551+0000 mgr.gibba032.whopeo (mgr.24625) 4272 : cluster [DBG] pgmap v2425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:23 gibba011 bash[30791]: cluster 2021-03-10T15:13:21.951178+0000 mgr.gibba032.whopeo (mgr.24625) 4273 : cluster [DBG] pgmap v2426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:23.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:23 gibba011 bash[31774]: cluster 2021-03-10T15:13:21.951178+0000 mgr.gibba032.whopeo (mgr.24625) 4273 : cluster [DBG] pgmap v2426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:23 gibba032 bash[24004]: cluster 2021-03-10T15:13:21.951178+0000 mgr.gibba032.whopeo (mgr.24625) 4273 : cluster [DBG] pgmap v2426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:25 gibba011 bash[30791]: cephadm 2021-03-10T15:13:23.688935+0000 mgr.gibba032.whopeo (mgr.24625) 4274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:25 gibba011 bash[30791]: cluster 2021-03-10T15:13:23.952616+0000 mgr.gibba032.whopeo (mgr.24625) 4275 : cluster [DBG] pgmap v2427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:25 gibba011 bash[31774]: cephadm 2021-03-10T15:13:23.688935+0000 mgr.gibba032.whopeo (mgr.24625) 4274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:25.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:25 gibba011 bash[31774]: cluster 2021-03-10T15:13:23.952616+0000 mgr.gibba032.whopeo (mgr.24625) 4275 : cluster [DBG] pgmap v2427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:25 gibba032 bash[24004]: cephadm 2021-03-10T15:13:23.688935+0000 mgr.gibba032.whopeo (mgr.24625) 4274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:25 gibba032 bash[24004]: cluster 2021-03-10T15:13:23.952616+0000 mgr.gibba032.whopeo (mgr.24625) 4275 : cluster [DBG] pgmap v2427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:27.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:27 gibba011 bash[30791]: cluster 2021-03-10T15:13:25.953980+0000 mgr.gibba032.whopeo (mgr.24625) 4276 : cluster [DBG] pgmap v2428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:27.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:27 gibba011 bash[31774]: cluster 2021-03-10T15:13:25.953980+0000 mgr.gibba032.whopeo (mgr.24625) 4276 : cluster [DBG] pgmap v2428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:27.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:27 gibba032 bash[24004]: cluster 2021-03-10T15:13:25.953980+0000 mgr.gibba032.whopeo (mgr.24625) 4276 : cluster [DBG] pgmap v2428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:28 gibba011 bash[30791]: cluster 2021-03-10T15:13:27.955742+0000 mgr.gibba032.whopeo (mgr.24625) 4277 : cluster [DBG] pgmap v2429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:28 gibba011 bash[31774]: cluster 2021-03-10T15:13:27.955742+0000 mgr.gibba032.whopeo (mgr.24625) 4277 : cluster [DBG] pgmap v2429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:28 gibba032 bash[24004]: cluster 2021-03-10T15:13:27.955742+0000 mgr.gibba032.whopeo (mgr.24625) 4277 : cluster [DBG] pgmap v2429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:31 gibba011 bash[30791]: cluster 2021-03-10T15:13:29.957190+0000 mgr.gibba032.whopeo (mgr.24625) 4278 : cluster [DBG] pgmap v2430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:31 gibba011 bash[31774]: cluster 2021-03-10T15:13:29.957190+0000 mgr.gibba032.whopeo (mgr.24625) 4278 : cluster [DBG] pgmap v2430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:31.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:31 gibba032 bash[24004]: cluster 2021-03-10T15:13:29.957190+0000 mgr.gibba032.whopeo (mgr.24625) 4278 : cluster [DBG] pgmap v2430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:33 gibba011 bash[30791]: cluster 2021-03-10T15:13:31.958893+0000 mgr.gibba032.whopeo (mgr.24625) 4279 : cluster [DBG] pgmap v2431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:33 gibba011 bash[31774]: cluster 2021-03-10T15:13:31.958893+0000 mgr.gibba032.whopeo (mgr.24625) 4279 : cluster [DBG] pgmap v2431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:33 gibba032 bash[24004]: cluster 2021-03-10T15:13:31.958893+0000 mgr.gibba032.whopeo (mgr.24625) 4279 : cluster [DBG] pgmap v2431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:35.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:35 gibba011 bash[30791]: cluster 2021-03-10T15:13:33.960412+0000 mgr.gibba032.whopeo (mgr.24625) 4280 : cluster [DBG] pgmap v2432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:35.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:35 gibba011 bash[31774]: cluster 2021-03-10T15:13:33.960412+0000 mgr.gibba032.whopeo (mgr.24625) 4280 : cluster [DBG] pgmap v2432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:35 gibba032 bash[24004]: cluster 2021-03-10T15:13:33.960412+0000 mgr.gibba032.whopeo (mgr.24625) 4280 : cluster [DBG] pgmap v2432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:37 gibba011 bash[30791]: cluster 2021-03-10T15:13:35.961783+0000 mgr.gibba032.whopeo (mgr.24625) 4281 : cluster [DBG] pgmap v2433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:37 gibba011 bash[31774]: cluster 2021-03-10T15:13:35.961783+0000 mgr.gibba032.whopeo (mgr.24625) 4281 : cluster [DBG] pgmap v2433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:37.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:37 gibba032 bash[24004]: cluster 2021-03-10T15:13:35.961783+0000 mgr.gibba032.whopeo (mgr.24625) 4281 : cluster [DBG] pgmap v2433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:38 gibba011 bash[30791]: cluster 2021-03-10T15:13:37.963484+0000 mgr.gibba032.whopeo (mgr.24625) 4282 : cluster [DBG] pgmap v2434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:38 gibba011 bash[31774]: cluster 2021-03-10T15:13:37.963484+0000 mgr.gibba032.whopeo (mgr.24625) 4282 : cluster [DBG] pgmap v2434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:38 gibba032 bash[24004]: cluster 2021-03-10T15:13:37.963484+0000 mgr.gibba032.whopeo (mgr.24625) 4282 : cluster [DBG] pgmap v2434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.705754+0000 mgr.gibba032.whopeo (mgr.24625) 4283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:13:40.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.707423+0000 mgr.gibba032.whopeo (mgr.24625) 4284 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:13:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.920490+0000 mon.a (mon.0) 1769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:13:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.921288+0000 mgr.gibba032.whopeo (mgr.24625) 4285 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:13:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.922975+0000 mgr.gibba032.whopeo (mgr.24625) 4286 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:13:40.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.924019+0000 mon.a (mon.0) 1770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:13:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.924231+0000 mgr.gibba032.whopeo (mgr.24625) 4287 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:13:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.924269+0000 mgr.gibba032.whopeo (mgr.24625) 4288 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:13:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.925226+0000 mon.a (mon.0) 1771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.925841+0000 mgr.gibba032.whopeo (mgr.24625) 4289 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:13:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.925880+0000 mgr.gibba032.whopeo (mgr.24625) 4290 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:13:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.926772+0000 mon.a (mon.0) 1772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.927438+0000 mgr.gibba032.whopeo (mgr.24625) 4291 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:13:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.928311+0000 mon.a (mon.0) 1773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.928871+0000 mgr.gibba032.whopeo (mgr.24625) 4292 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:13:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.928906+0000 mgr.gibba032.whopeo (mgr.24625) 4293 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:13:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.930383+0000 mgr.gibba032.whopeo (mgr.24625) 4294 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:13:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.930695+0000 mgr.gibba032.whopeo (mgr.24625) 4295 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:13:40.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.931311+0000 mon.a (mon.0) 1774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.931898+0000 mgr.gibba032.whopeo (mgr.24625) 4296 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:13:40.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.931936+0000 mgr.gibba032.whopeo (mgr.24625) 4297 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:13:40.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: cephadm 2021-03-10T15:13:38.932155+0000 mgr.gibba032.whopeo (mgr.24625) 4298 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:40.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:39 gibba032 bash[24004]: audit 2021-03-10T15:13:38.932947+0000 mon.a (mon.0) 1775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.705754+0000 mgr.gibba032.whopeo (mgr.24625) 4283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:13:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.707423+0000 mgr.gibba032.whopeo (mgr.24625) 4284 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:13:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.920490+0000 mon.a (mon.0) 1769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:13:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.921288+0000 mgr.gibba032.whopeo (mgr.24625) 4285 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:13:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.922975+0000 mgr.gibba032.whopeo (mgr.24625) 4286 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:13:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.924019+0000 mon.a (mon.0) 1770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:13:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.924231+0000 mgr.gibba032.whopeo (mgr.24625) 4287 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:13:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.924269+0000 mgr.gibba032.whopeo (mgr.24625) 4288 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:13:40.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.925226+0000 mon.a (mon.0) 1771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.925841+0000 mgr.gibba032.whopeo (mgr.24625) 4289 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:13:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.925880+0000 mgr.gibba032.whopeo (mgr.24625) 4290 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:13:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.926772+0000 mon.a (mon.0) 1772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.927438+0000 mgr.gibba032.whopeo (mgr.24625) 4291 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:13:40.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.928311+0000 mon.a (mon.0) 1773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.928871+0000 mgr.gibba032.whopeo (mgr.24625) 4292 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:13:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.928906+0000 mgr.gibba032.whopeo (mgr.24625) 4293 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:13:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.930383+0000 mgr.gibba032.whopeo (mgr.24625) 4294 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:13:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.930695+0000 mgr.gibba032.whopeo (mgr.24625) 4295 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:13:40.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.931311+0000 mon.a (mon.0) 1774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.931898+0000 mgr.gibba032.whopeo (mgr.24625) 4296 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:13:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.931936+0000 mgr.gibba032.whopeo (mgr.24625) 4297 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:13:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: cephadm 2021-03-10T15:13:38.932155+0000 mgr.gibba032.whopeo (mgr.24625) 4298 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:40.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[30791]: audit 2021-03-10T15:13:38.932947+0000 mon.a (mon.0) 1775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.705754+0000 mgr.gibba032.whopeo (mgr.24625) 4283 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:13:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.707423+0000 mgr.gibba032.whopeo (mgr.24625) 4284 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:13:40.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.920490+0000 mon.a (mon.0) 1769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:13:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.921288+0000 mgr.gibba032.whopeo (mgr.24625) 4285 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:13:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.922975+0000 mgr.gibba032.whopeo (mgr.24625) 4286 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:13:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.924019+0000 mon.a (mon.0) 1770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:13:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.924231+0000 mgr.gibba032.whopeo (mgr.24625) 4287 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:13:40.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.924269+0000 mgr.gibba032.whopeo (mgr.24625) 4288 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:13:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.925226+0000 mon.a (mon.0) 1771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.925841+0000 mgr.gibba032.whopeo (mgr.24625) 4289 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:13:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.925880+0000 mgr.gibba032.whopeo (mgr.24625) 4290 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:13:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.926772+0000 mon.a (mon.0) 1772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.927438+0000 mgr.gibba032.whopeo (mgr.24625) 4291 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:13:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.928311+0000 mon.a (mon.0) 1773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.928871+0000 mgr.gibba032.whopeo (mgr.24625) 4292 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:13:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.928906+0000 mgr.gibba032.whopeo (mgr.24625) 4293 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:13:40.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.930383+0000 mgr.gibba032.whopeo (mgr.24625) 4294 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:13:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.930695+0000 mgr.gibba032.whopeo (mgr.24625) 4295 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:13:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.931311+0000 mon.a (mon.0) 1774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.931898+0000 mgr.gibba032.whopeo (mgr.24625) 4296 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:13:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.931936+0000 mgr.gibba032.whopeo (mgr.24625) 4297 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:13:40.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: cephadm 2021-03-10T15:13:38.932155+0000 mgr.gibba032.whopeo (mgr.24625) 4298 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:40.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:39 gibba011 bash[31774]: audit 2021-03-10T15:13:38.932947+0000 mon.a (mon.0) 1775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:40 gibba032 bash[24004]: cluster 2021-03-10T15:13:39.964801+0000 mgr.gibba032.whopeo (mgr.24625) 4299 : cluster [DBG] pgmap v2435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:40 gibba011 bash[30791]: cluster 2021-03-10T15:13:39.964801+0000 mgr.gibba032.whopeo (mgr.24625) 4299 : cluster [DBG] pgmap v2435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:40 gibba011 bash[31774]: cluster 2021-03-10T15:13:39.964801+0000 mgr.gibba032.whopeo (mgr.24625) 4299 : cluster [DBG] pgmap v2435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:43 gibba011 bash[30791]: cluster 2021-03-10T15:13:41.966525+0000 mgr.gibba032.whopeo (mgr.24625) 4300 : cluster [DBG] pgmap v2436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:43 gibba011 bash[31774]: cluster 2021-03-10T15:13:41.966525+0000 mgr.gibba032.whopeo (mgr.24625) 4300 : cluster [DBG] pgmap v2436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:43 gibba032 bash[24004]: cluster 2021-03-10T15:13:41.966525+0000 mgr.gibba032.whopeo (mgr.24625) 4300 : cluster [DBG] pgmap v2436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:45 gibba011 bash[30791]: cluster 2021-03-10T15:13:43.968224+0000 mgr.gibba032.whopeo (mgr.24625) 4301 : cluster [DBG] pgmap v2437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:45 gibba011 bash[31774]: cluster 2021-03-10T15:13:43.968224+0000 mgr.gibba032.whopeo (mgr.24625) 4301 : cluster [DBG] pgmap v2437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:45 gibba032 bash[24004]: cluster 2021-03-10T15:13:43.968224+0000 mgr.gibba032.whopeo (mgr.24625) 4301 : cluster [DBG] pgmap v2437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:47.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:47 gibba011 bash[30791]: cluster 2021-03-10T15:13:45.969640+0000 mgr.gibba032.whopeo (mgr.24625) 4302 : cluster [DBG] pgmap v2438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:47.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:47 gibba011 bash[31774]: cluster 2021-03-10T15:13:45.969640+0000 mgr.gibba032.whopeo (mgr.24625) 4302 : cluster [DBG] pgmap v2438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:47 gibba032 bash[24004]: cluster 2021-03-10T15:13:45.969640+0000 mgr.gibba032.whopeo (mgr.24625) 4302 : cluster [DBG] pgmap v2438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:47.764 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:13:47.993 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:13:47.994 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (80m) 5m ago 85m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:13:47.994 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (85m) 4m ago 85m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:13:47.994 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (80m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:13:47.995 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (81m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:13:47.995 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (80m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:13:47.995 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (79m) 5m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:13:47.995 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (79m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:13:47.996 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (79m) 5m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:13:47.996 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (85m) 5m ago 85m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:13:47.996 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (85m) 4m ago 85m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:13:47.996 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (78m) 5m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:13:47.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (78m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:13:47.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (77m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:13:47.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (77m) 5m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:13:47.997 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (77m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:13:47.998 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (76m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:13:47.998 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (76m) 4m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:13:47.998 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (75m) 4m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:13:47.998 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (85m) 4m ago 85m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:13:47.999 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 83m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:13:48.248 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:13:48.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:13:48.249 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:13:48.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:48.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:13:48.250 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:13:48.250 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:48.250 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:13:48.250 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:13:48.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:13:48.251 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:13:48.251 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:13:48.251 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:13:48.251 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:13:48.252 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:13:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[30791]: audit 2021-03-10T15:13:47.751044+0000 mgr.gibba032.whopeo (mgr.24625) 4303 : audit [DBG] from='client.46935 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[30791]: cluster 2021-03-10T15:13:47.971288+0000 mgr.gibba032.whopeo (mgr.24625) 4304 : cluster [DBG] pgmap v2439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:48.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[30791]: audit 2021-03-10T15:13:47.976775+0000 mgr.gibba032.whopeo (mgr.24625) 4305 : audit [DBG] from='client.46941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[30791]: audit 2021-03-10T15:13:48.245620+0000 mon.c (mon.1) 82 : audit [DBG] from='client.? 172.21.2.111:0/996646029' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[31774]: audit 2021-03-10T15:13:47.751044+0000 mgr.gibba032.whopeo (mgr.24625) 4303 : audit [DBG] from='client.46935 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[31774]: cluster 2021-03-10T15:13:47.971288+0000 mgr.gibba032.whopeo (mgr.24625) 4304 : cluster [DBG] pgmap v2439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:48.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[31774]: audit 2021-03-10T15:13:47.976775+0000 mgr.gibba032.whopeo (mgr.24625) 4305 : audit [DBG] from='client.46941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:48 gibba011 bash[31774]: audit 2021-03-10T15:13:48.245620+0000 mon.c (mon.1) 82 : audit [DBG] from='client.? 172.21.2.111:0/996646029' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:48 gibba032 bash[24004]: audit 2021-03-10T15:13:47.751044+0000 mgr.gibba032.whopeo (mgr.24625) 4303 : audit [DBG] from='client.46935 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:48 gibba032 bash[24004]: cluster 2021-03-10T15:13:47.971288+0000 mgr.gibba032.whopeo (mgr.24625) 4304 : cluster [DBG] pgmap v2439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:48.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:48 gibba032 bash[24004]: audit 2021-03-10T15:13:47.976775+0000 mgr.gibba032.whopeo (mgr.24625) 4305 : audit [DBG] from='client.46941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:13:48.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:48 gibba032 bash[24004]: audit 2021-03-10T15:13:48.245620+0000 mon.c (mon.1) 82 : audit [DBG] from='client.? 172.21.2.111:0/996646029' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:13:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:51 gibba011 bash[30791]: cluster 2021-03-10T15:13:49.972842+0000 mgr.gibba032.whopeo (mgr.24625) 4306 : cluster [DBG] pgmap v2440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:51.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:51 gibba011 bash[31774]: cluster 2021-03-10T15:13:49.972842+0000 mgr.gibba032.whopeo (mgr.24625) 4306 : cluster [DBG] pgmap v2440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:51 gibba032 bash[24004]: cluster 2021-03-10T15:13:49.972842+0000 mgr.gibba032.whopeo (mgr.24625) 4306 : cluster [DBG] pgmap v2440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:53 gibba011 bash[30791]: cluster 2021-03-10T15:13:51.974608+0000 mgr.gibba032.whopeo (mgr.24625) 4307 : cluster [DBG] pgmap v2441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:53 gibba011 bash[31774]: cluster 2021-03-10T15:13:51.974608+0000 mgr.gibba032.whopeo (mgr.24625) 4307 : cluster [DBG] pgmap v2441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:53 gibba032 bash[24004]: cluster 2021-03-10T15:13:51.974608+0000 mgr.gibba032.whopeo (mgr.24625) 4307 : cluster [DBG] pgmap v2441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:55 gibba011 bash[30791]: cephadm 2021-03-10T15:13:53.948097+0000 mgr.gibba032.whopeo (mgr.24625) 4308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:55 gibba011 bash[30791]: cluster 2021-03-10T15:13:53.975923+0000 mgr.gibba032.whopeo (mgr.24625) 4309 : cluster [DBG] pgmap v2442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:55 gibba011 bash[31774]: cephadm 2021-03-10T15:13:53.948097+0000 mgr.gibba032.whopeo (mgr.24625) 4308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:55.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:55 gibba011 bash[31774]: cluster 2021-03-10T15:13:53.975923+0000 mgr.gibba032.whopeo (mgr.24625) 4309 : cluster [DBG] pgmap v2442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:55 gibba032 bash[24004]: cephadm 2021-03-10T15:13:53.948097+0000 mgr.gibba032.whopeo (mgr.24625) 4308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:13:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:55 gibba032 bash[24004]: cluster 2021-03-10T15:13:53.975923+0000 mgr.gibba032.whopeo (mgr.24625) 4309 : cluster [DBG] pgmap v2442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: cluster 2021-03-10T15:13:55.977367+0000 mgr.gibba032.whopeo (mgr.24625) 4310 : cluster [DBG] pgmap v2443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.526955+0000 mon.a (mon.0) 1776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:13:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.527632+0000 mon.a (mon.0) 1777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:13:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.535251+0000 mon.a (mon.0) 1778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:13:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.536018+0000 mon.a (mon.0) 1779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:13:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.536712+0000 mon.a (mon.0) 1780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:13:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[30791]: audit 2021-03-10T15:13:56.537382+0000 mon.a (mon.0) 1781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:13:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: cluster 2021-03-10T15:13:55.977367+0000 mgr.gibba032.whopeo (mgr.24625) 4310 : cluster [DBG] pgmap v2443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.526955+0000 mon.a (mon.0) 1776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:13:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.527632+0000 mon.a (mon.0) 1777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:13:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.535251+0000 mon.a (mon.0) 1778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:13:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.536018+0000 mon.a (mon.0) 1779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:13:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.536712+0000 mon.a (mon.0) 1780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:13:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:57 gibba011 bash[31774]: audit 2021-03-10T15:13:56.537382+0000 mon.a (mon.0) 1781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:13:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: cluster 2021-03-10T15:13:55.977367+0000 mgr.gibba032.whopeo (mgr.24625) 4310 : cluster [DBG] pgmap v2443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.526955+0000 mon.a (mon.0) 1776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:13:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.527632+0000 mon.a (mon.0) 1777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:13:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.535251+0000 mon.a (mon.0) 1778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:13:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.536018+0000 mon.a (mon.0) 1779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:13:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.536712+0000 mon.a (mon.0) 1780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:13:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:57 gibba032 bash[24004]: audit 2021-03-10T15:13:56.537382+0000 mon.a (mon.0) 1781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:13:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:13:58 gibba011 bash[30791]: cluster 2021-03-10T15:13:57.978966+0000 mgr.gibba032.whopeo (mgr.24625) 4311 : cluster [DBG] pgmap v2444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:13:58 gibba011 bash[31774]: cluster 2021-03-10T15:13:57.978966+0000 mgr.gibba032.whopeo (mgr.24625) 4311 : cluster [DBG] pgmap v2444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:13:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:13:58 gibba032 bash[24004]: cluster 2021-03-10T15:13:57.978966+0000 mgr.gibba032.whopeo (mgr.24625) 4311 : cluster [DBG] pgmap v2444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:01.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:01 gibba011 bash[30791]: cluster 2021-03-10T15:13:59.980276+0000 mgr.gibba032.whopeo (mgr.24625) 4312 : cluster [DBG] pgmap v2445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:01 gibba011 bash[31774]: cluster 2021-03-10T15:13:59.980276+0000 mgr.gibba032.whopeo (mgr.24625) 4312 : cluster [DBG] pgmap v2445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:01 gibba032 bash[24004]: cluster 2021-03-10T15:13:59.980276+0000 mgr.gibba032.whopeo (mgr.24625) 4312 : cluster [DBG] pgmap v2445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:03 gibba011 bash[30791]: cluster 2021-03-10T15:14:01.981961+0000 mgr.gibba032.whopeo (mgr.24625) 4313 : cluster [DBG] pgmap v2446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:03.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:03 gibba011 bash[31774]: cluster 2021-03-10T15:14:01.981961+0000 mgr.gibba032.whopeo (mgr.24625) 4313 : cluster [DBG] pgmap v2446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:03 gibba032 bash[24004]: cluster 2021-03-10T15:14:01.981961+0000 mgr.gibba032.whopeo (mgr.24625) 4313 : cluster [DBG] pgmap v2446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:05.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:05 gibba011 bash[30791]: cluster 2021-03-10T15:14:03.983575+0000 mgr.gibba032.whopeo (mgr.24625) 4314 : cluster [DBG] pgmap v2447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:05.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:05 gibba011 bash[31774]: cluster 2021-03-10T15:14:03.983575+0000 mgr.gibba032.whopeo (mgr.24625) 4314 : cluster [DBG] pgmap v2447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:05 gibba032 bash[24004]: cluster 2021-03-10T15:14:03.983575+0000 mgr.gibba032.whopeo (mgr.24625) 4314 : cluster [DBG] pgmap v2447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:07 gibba011 bash[30791]: cluster 2021-03-10T15:14:05.985116+0000 mgr.gibba032.whopeo (mgr.24625) 4315 : cluster [DBG] pgmap v2448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:07.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:07 gibba011 bash[31774]: cluster 2021-03-10T15:14:05.985116+0000 mgr.gibba032.whopeo (mgr.24625) 4315 : cluster [DBG] pgmap v2448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:07 gibba032 bash[24004]: cluster 2021-03-10T15:14:05.985116+0000 mgr.gibba032.whopeo (mgr.24625) 4315 : cluster [DBG] pgmap v2448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:08 gibba011 bash[30791]: cluster 2021-03-10T15:14:07.986734+0000 mgr.gibba032.whopeo (mgr.24625) 4316 : cluster [DBG] pgmap v2449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:08 gibba011 bash[31774]: cluster 2021-03-10T15:14:07.986734+0000 mgr.gibba032.whopeo (mgr.24625) 4316 : cluster [DBG] pgmap v2449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:08 gibba032 bash[24004]: cluster 2021-03-10T15:14:07.986734+0000 mgr.gibba032.whopeo (mgr.24625) 4316 : cluster [DBG] pgmap v2449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:09 gibba011 bash[30791]: cephadm 2021-03-10T15:14:08.964229+0000 mgr.gibba032.whopeo (mgr.24625) 4317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:09 gibba011 bash[31774]: cephadm 2021-03-10T15:14:08.964229+0000 mgr.gibba032.whopeo (mgr.24625) 4317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:09 gibba032 bash[24004]: cephadm 2021-03-10T15:14:08.964229+0000 mgr.gibba032.whopeo (mgr.24625) 4317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:10 gibba011 bash[30791]: cluster 2021-03-10T15:14:09.988309+0000 mgr.gibba032.whopeo (mgr.24625) 4318 : cluster [DBG] pgmap v2450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:10 gibba011 bash[31774]: cluster 2021-03-10T15:14:09.988309+0000 mgr.gibba032.whopeo (mgr.24625) 4318 : cluster [DBG] pgmap v2450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:10 gibba032 bash[24004]: cluster 2021-03-10T15:14:09.988309+0000 mgr.gibba032.whopeo (mgr.24625) 4318 : cluster [DBG] pgmap v2450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:13 gibba011 bash[30791]: cluster 2021-03-10T15:14:11.990032+0000 mgr.gibba032.whopeo (mgr.24625) 4319 : cluster [DBG] pgmap v2451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:13.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:13 gibba011 bash[31774]: cluster 2021-03-10T15:14:11.990032+0000 mgr.gibba032.whopeo (mgr.24625) 4319 : cluster [DBG] pgmap v2451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:13 gibba032 bash[24004]: cluster 2021-03-10T15:14:11.990032+0000 mgr.gibba032.whopeo (mgr.24625) 4319 : cluster [DBG] pgmap v2451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:15 gibba011 bash[30791]: cluster 2021-03-10T15:14:13.991673+0000 mgr.gibba032.whopeo (mgr.24625) 4320 : cluster [DBG] pgmap v2452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:15 gibba011 bash[31774]: cluster 2021-03-10T15:14:13.991673+0000 mgr.gibba032.whopeo (mgr.24625) 4320 : cluster [DBG] pgmap v2452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:15 gibba032 bash[24004]: cluster 2021-03-10T15:14:13.991673+0000 mgr.gibba032.whopeo (mgr.24625) 4320 : cluster [DBG] pgmap v2452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:17.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:17 gibba011 bash[30791]: cluster 2021-03-10T15:14:15.993138+0000 mgr.gibba032.whopeo (mgr.24625) 4321 : cluster [DBG] pgmap v2453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:17.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:17 gibba011 bash[31774]: cluster 2021-03-10T15:14:15.993138+0000 mgr.gibba032.whopeo (mgr.24625) 4321 : cluster [DBG] pgmap v2453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:17 gibba032 bash[24004]: cluster 2021-03-10T15:14:15.993138+0000 mgr.gibba032.whopeo (mgr.24625) 4321 : cluster [DBG] pgmap v2453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:18.495 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:14:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:18 gibba011 bash[30791]: cluster 2021-03-10T15:14:17.994611+0000 mgr.gibba032.whopeo (mgr.24625) 4322 : cluster [DBG] pgmap v2454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:18 gibba011 bash[31774]: cluster 2021-03-10T15:14:17.994611+0000 mgr.gibba032.whopeo (mgr.24625) 4322 : cluster [DBG] pgmap v2454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:18.722 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:14:18.722 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (80m) 5m ago 86m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:14:18.723 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (86m) 4m ago 86m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:14:18.723 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (81m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:14:18.723 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (81m) 4m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:14:18.723 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (80m) 5m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:14:18.724 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (80m) 5m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:14:18.724 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (80m) 4m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:14:18.724 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (80m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:14:18.724 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (86m) 5m ago 86m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:14:18.725 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (86m) 4m ago 86m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:14:18.725 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (79m) 5m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:14:18.725 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (78m) 5m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:14:18.725 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (78m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:14:18.725 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (77m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:14:18.726 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (77m) 4m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:14:18.726 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (77m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:14:18.726 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (76m) 4m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:14:18.726 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (76m) 4m ago 86m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:14:18.727 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (85m) 4m ago 86m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:14:18.727 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 84m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:14:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:18 gibba032 bash[24004]: cluster 2021-03-10T15:14:17.994611+0000 mgr.gibba032.whopeo (mgr.24625) 4322 : cluster [DBG] pgmap v2454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:18.972 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:14:18.972 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:14:18.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:14:18.973 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:18.973 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:14:18.973 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:14:18.973 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:18.974 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:14:18.974 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:14:18.974 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:18.975 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:14:18.975 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:14:18.975 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:14:18.975 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:14:18.975 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:14:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[30791]: audit 2021-03-10T15:14:18.482255+0000 mgr.gibba032.whopeo (mgr.24625) 4323 : audit [DBG] from='client.46953 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[30791]: audit 2021-03-10T15:14:18.707094+0000 mgr.gibba032.whopeo (mgr.24625) 4324 : audit [DBG] from='client.46959 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[30791]: audit 2021-03-10T15:14:18.969487+0000 mon.c (mon.1) 83 : audit [DBG] from='client.? 172.21.2.111:0/951817167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:19.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[31774]: audit 2021-03-10T15:14:18.482255+0000 mgr.gibba032.whopeo (mgr.24625) 4323 : audit [DBG] from='client.46953 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[31774]: audit 2021-03-10T15:14:18.707094+0000 mgr.gibba032.whopeo (mgr.24625) 4324 : audit [DBG] from='client.46959 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:19 gibba011 bash[31774]: audit 2021-03-10T15:14:18.969487+0000 mon.c (mon.1) 83 : audit [DBG] from='client.? 172.21.2.111:0/951817167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:19 gibba032 bash[24004]: audit 2021-03-10T15:14:18.482255+0000 mgr.gibba032.whopeo (mgr.24625) 4323 : audit [DBG] from='client.46953 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:19 gibba032 bash[24004]: audit 2021-03-10T15:14:18.707094+0000 mgr.gibba032.whopeo (mgr.24625) 4324 : audit [DBG] from='client.46959 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:19 gibba032 bash[24004]: audit 2021-03-10T15:14:18.969487+0000 mon.c (mon.1) 83 : audit [DBG] from='client.? 172.21.2.111:0/951817167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:20 gibba011 bash[30791]: cluster 2021-03-10T15:14:19.996152+0000 mgr.gibba032.whopeo (mgr.24625) 4325 : cluster [DBG] pgmap v2455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:20 gibba011 bash[31774]: cluster 2021-03-10T15:14:19.996152+0000 mgr.gibba032.whopeo (mgr.24625) 4325 : cluster [DBG] pgmap v2455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:20 gibba032 bash[24004]: cluster 2021-03-10T15:14:19.996152+0000 mgr.gibba032.whopeo (mgr.24625) 4325 : cluster [DBG] pgmap v2455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:23 gibba011 bash[30791]: cluster 2021-03-10T15:14:21.997893+0000 mgr.gibba032.whopeo (mgr.24625) 4326 : cluster [DBG] pgmap v2456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:23.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:23 gibba011 bash[31774]: cluster 2021-03-10T15:14:21.997893+0000 mgr.gibba032.whopeo (mgr.24625) 4326 : cluster [DBG] pgmap v2456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:23 gibba032 bash[24004]: cluster 2021-03-10T15:14:21.997893+0000 mgr.gibba032.whopeo (mgr.24625) 4326 : cluster [DBG] pgmap v2456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:25 gibba011 bash[30791]: cephadm 2021-03-10T15:14:23.968793+0000 mgr.gibba032.whopeo (mgr.24625) 4327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:25 gibba011 bash[30791]: cluster 2021-03-10T15:14:23.999197+0000 mgr.gibba032.whopeo (mgr.24625) 4328 : cluster [DBG] pgmap v2457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:25 gibba011 bash[31774]: cephadm 2021-03-10T15:14:23.968793+0000 mgr.gibba032.whopeo (mgr.24625) 4327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:25.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:25 gibba011 bash[31774]: cluster 2021-03-10T15:14:23.999197+0000 mgr.gibba032.whopeo (mgr.24625) 4328 : cluster [DBG] pgmap v2457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:25 gibba032 bash[24004]: cephadm 2021-03-10T15:14:23.968793+0000 mgr.gibba032.whopeo (mgr.24625) 4327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:25 gibba032 bash[24004]: cluster 2021-03-10T15:14:23.999197+0000 mgr.gibba032.whopeo (mgr.24625) 4328 : cluster [DBG] pgmap v2457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:27.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:27 gibba032 bash[24004]: cluster 2021-03-10T15:14:26.000731+0000 mgr.gibba032.whopeo (mgr.24625) 4329 : cluster [DBG] pgmap v2458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:27.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:27 gibba011 bash[30791]: cluster 2021-03-10T15:14:26.000731+0000 mgr.gibba032.whopeo (mgr.24625) 4329 : cluster [DBG] pgmap v2458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:27.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:27 gibba011 bash[31774]: cluster 2021-03-10T15:14:26.000731+0000 mgr.gibba032.whopeo (mgr.24625) 4329 : cluster [DBG] pgmap v2458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:28 gibba011 bash[30791]: cluster 2021-03-10T15:14:28.002045+0000 mgr.gibba032.whopeo (mgr.24625) 4330 : cluster [DBG] pgmap v2459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:28 gibba011 bash[31774]: cluster 2021-03-10T15:14:28.002045+0000 mgr.gibba032.whopeo (mgr.24625) 4330 : cluster [DBG] pgmap v2459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:28 gibba032 bash[24004]: cluster 2021-03-10T15:14:28.002045+0000 mgr.gibba032.whopeo (mgr.24625) 4330 : cluster [DBG] pgmap v2459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:31 gibba011 bash[30791]: cluster 2021-03-10T15:14:30.003327+0000 mgr.gibba032.whopeo (mgr.24625) 4331 : cluster [DBG] pgmap v2460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:31.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:31 gibba011 bash[31774]: cluster 2021-03-10T15:14:30.003327+0000 mgr.gibba032.whopeo (mgr.24625) 4331 : cluster [DBG] pgmap v2460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:31.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:31 gibba032 bash[24004]: cluster 2021-03-10T15:14:30.003327+0000 mgr.gibba032.whopeo (mgr.24625) 4331 : cluster [DBG] pgmap v2460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:33 gibba011 bash[30791]: cluster 2021-03-10T15:14:32.005026+0000 mgr.gibba032.whopeo (mgr.24625) 4332 : cluster [DBG] pgmap v2461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:33 gibba011 bash[31774]: cluster 2021-03-10T15:14:32.005026+0000 mgr.gibba032.whopeo (mgr.24625) 4332 : cluster [DBG] pgmap v2461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:33 gibba032 bash[24004]: cluster 2021-03-10T15:14:32.005026+0000 mgr.gibba032.whopeo (mgr.24625) 4332 : cluster [DBG] pgmap v2461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:35 gibba032 bash[24004]: cluster 2021-03-10T15:14:34.006577+0000 mgr.gibba032.whopeo (mgr.24625) 4333 : cluster [DBG] pgmap v2462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:35 gibba011 bash[30791]: cluster 2021-03-10T15:14:34.006577+0000 mgr.gibba032.whopeo (mgr.24625) 4333 : cluster [DBG] pgmap v2462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:35 gibba011 bash[31774]: cluster 2021-03-10T15:14:34.006577+0000 mgr.gibba032.whopeo (mgr.24625) 4333 : cluster [DBG] pgmap v2462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:37.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:37 gibba032 bash[24004]: cluster 2021-03-10T15:14:36.008011+0000 mgr.gibba032.whopeo (mgr.24625) 4334 : cluster [DBG] pgmap v2463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:37.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:37 gibba011 bash[30791]: cluster 2021-03-10T15:14:36.008011+0000 mgr.gibba032.whopeo (mgr.24625) 4334 : cluster [DBG] pgmap v2463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:37.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:37 gibba011 bash[31774]: cluster 2021-03-10T15:14:36.008011+0000 mgr.gibba032.whopeo (mgr.24625) 4334 : cluster [DBG] pgmap v2463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:38 gibba011 bash[30791]: cluster 2021-03-10T15:14:38.009662+0000 mgr.gibba032.whopeo (mgr.24625) 4335 : cluster [DBG] pgmap v2464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:38 gibba011 bash[31774]: cluster 2021-03-10T15:14:38.009662+0000 mgr.gibba032.whopeo (mgr.24625) 4335 : cluster [DBG] pgmap v2464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:38 gibba032 bash[24004]: cluster 2021-03-10T15:14:38.009662+0000 mgr.gibba032.whopeo (mgr.24625) 4335 : cluster [DBG] pgmap v2464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:39.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: cephadm 2021-03-10T15:14:38.985843+0000 mgr.gibba032.whopeo (mgr.24625) 4336 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:14:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: cephadm 2021-03-10T15:14:38.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4337 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:14:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.207124+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:14:39.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.210693+0000 mon.a (mon.0) 1783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:14:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.212105+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.213531+0000 mon.a (mon.0) 1785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.214819+0000 mon.a (mon.0) 1786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.216934+0000 mon.a (mon.0) 1787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:39 gibba032 bash[24004]: audit 2021-03-10T15:14:39.218268+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: cephadm 2021-03-10T15:14:38.985843+0000 mgr.gibba032.whopeo (mgr.24625) 4336 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:14:39.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: cephadm 2021-03-10T15:14:38.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4337 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:14:39.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.207124+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:14:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.210693+0000 mon.a (mon.0) 1783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:14:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.212105+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.213531+0000 mon.a (mon.0) 1785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.214819+0000 mon.a (mon.0) 1786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.216934+0000 mon.a (mon.0) 1787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[30791]: audit 2021-03-10T15:14:39.218268+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: cephadm 2021-03-10T15:14:38.985843+0000 mgr.gibba032.whopeo (mgr.24625) 4336 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:14:39.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: cephadm 2021-03-10T15:14:38.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4337 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:14:39.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.207124+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:14:39.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.210693+0000 mon.a (mon.0) 1783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:14:39.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.212105+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.213531+0000 mon.a (mon.0) 1785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.214819+0000 mon.a (mon.0) 1786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.216934+0000 mon.a (mon.0) 1787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:39.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:39 gibba011 bash[31774]: audit 2021-03-10T15:14:39.218268+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.208003+0000 mgr.gibba032.whopeo (mgr.24625) 4338 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:14:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.209758+0000 mgr.gibba032.whopeo (mgr.24625) 4339 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:14:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.211219+0000 mgr.gibba032.whopeo (mgr.24625) 4340 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:14:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.211300+0000 mgr.gibba032.whopeo (mgr.24625) 4341 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:14:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.212620+0000 mgr.gibba032.whopeo (mgr.24625) 4342 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:14:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.212686+0000 mgr.gibba032.whopeo (mgr.24625) 4343 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:14:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.213689+0000 mgr.gibba032.whopeo (mgr.24625) 4344 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:14:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.214593+0000 mgr.gibba032.whopeo (mgr.24625) 4345 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:14:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.214659+0000 mgr.gibba032.whopeo (mgr.24625) 4346 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:14:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.215768+0000 mgr.gibba032.whopeo (mgr.24625) 4347 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:14:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.216147+0000 mgr.gibba032.whopeo (mgr.24625) 4348 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:14:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.217091+0000 mgr.gibba032.whopeo (mgr.24625) 4349 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:14:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.217202+0000 mgr.gibba032.whopeo (mgr.24625) 4350 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:14:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cephadm 2021-03-10T15:14:39.217328+0000 mgr.gibba032.whopeo (mgr.24625) 4351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:40 gibba032 bash[24004]: cluster 2021-03-10T15:14:40.010293+0000 mgr.gibba032.whopeo (mgr.24625) 4352 : cluster [DBG] pgmap v2465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.208003+0000 mgr.gibba032.whopeo (mgr.24625) 4338 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:14:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.209758+0000 mgr.gibba032.whopeo (mgr.24625) 4339 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:14:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.211219+0000 mgr.gibba032.whopeo (mgr.24625) 4340 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:14:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.211300+0000 mgr.gibba032.whopeo (mgr.24625) 4341 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:14:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.212620+0000 mgr.gibba032.whopeo (mgr.24625) 4342 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:14:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.212686+0000 mgr.gibba032.whopeo (mgr.24625) 4343 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:14:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.213689+0000 mgr.gibba032.whopeo (mgr.24625) 4344 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:14:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.214593+0000 mgr.gibba032.whopeo (mgr.24625) 4345 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:14:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.214659+0000 mgr.gibba032.whopeo (mgr.24625) 4346 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:14:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.215768+0000 mgr.gibba032.whopeo (mgr.24625) 4347 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:14:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.216147+0000 mgr.gibba032.whopeo (mgr.24625) 4348 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:14:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.217091+0000 mgr.gibba032.whopeo (mgr.24625) 4349 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:14:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.217202+0000 mgr.gibba032.whopeo (mgr.24625) 4350 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:14:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cephadm 2021-03-10T15:14:39.217328+0000 mgr.gibba032.whopeo (mgr.24625) 4351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[30791]: cluster 2021-03-10T15:14:40.010293+0000 mgr.gibba032.whopeo (mgr.24625) 4352 : cluster [DBG] pgmap v2465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.208003+0000 mgr.gibba032.whopeo (mgr.24625) 4338 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:14:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.209758+0000 mgr.gibba032.whopeo (mgr.24625) 4339 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:14:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.211219+0000 mgr.gibba032.whopeo (mgr.24625) 4340 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:14:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.211300+0000 mgr.gibba032.whopeo (mgr.24625) 4341 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:14:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.212620+0000 mgr.gibba032.whopeo (mgr.24625) 4342 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:14:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.212686+0000 mgr.gibba032.whopeo (mgr.24625) 4343 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:14:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.213689+0000 mgr.gibba032.whopeo (mgr.24625) 4344 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:14:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.214593+0000 mgr.gibba032.whopeo (mgr.24625) 4345 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:14:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.214659+0000 mgr.gibba032.whopeo (mgr.24625) 4346 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:14:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.215768+0000 mgr.gibba032.whopeo (mgr.24625) 4347 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:14:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.216147+0000 mgr.gibba032.whopeo (mgr.24625) 4348 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:14:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.217091+0000 mgr.gibba032.whopeo (mgr.24625) 4349 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:14:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.217202+0000 mgr.gibba032.whopeo (mgr.24625) 4350 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:14:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cephadm 2021-03-10T15:14:39.217328+0000 mgr.gibba032.whopeo (mgr.24625) 4351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:40 gibba011 bash[31774]: cluster 2021-03-10T15:14:40.010293+0000 mgr.gibba032.whopeo (mgr.24625) 4352 : cluster [DBG] pgmap v2465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:43 gibba011 bash[30791]: cluster 2021-03-10T15:14:42.011857+0000 mgr.gibba032.whopeo (mgr.24625) 4353 : cluster [DBG] pgmap v2466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:43.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:43 gibba011 bash[31774]: cluster 2021-03-10T15:14:42.011857+0000 mgr.gibba032.whopeo (mgr.24625) 4353 : cluster [DBG] pgmap v2466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:43 gibba032 bash[24004]: cluster 2021-03-10T15:14:42.011857+0000 mgr.gibba032.whopeo (mgr.24625) 4353 : cluster [DBG] pgmap v2466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:45 gibba032 bash[24004]: cluster 2021-03-10T15:14:44.013466+0000 mgr.gibba032.whopeo (mgr.24625) 4354 : cluster [DBG] pgmap v2467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:45 gibba011 bash[30791]: cluster 2021-03-10T15:14:44.013466+0000 mgr.gibba032.whopeo (mgr.24625) 4354 : cluster [DBG] pgmap v2467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:45 gibba011 bash[31774]: cluster 2021-03-10T15:14:44.013466+0000 mgr.gibba032.whopeo (mgr.24625) 4354 : cluster [DBG] pgmap v2467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:47 gibba032 bash[24004]: cluster 2021-03-10T15:14:46.015243+0000 mgr.gibba032.whopeo (mgr.24625) 4355 : cluster [DBG] pgmap v2468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:47.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:47 gibba011 bash[30791]: cluster 2021-03-10T15:14:46.015243+0000 mgr.gibba032.whopeo (mgr.24625) 4355 : cluster [DBG] pgmap v2468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:47.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:47 gibba011 bash[31774]: cluster 2021-03-10T15:14:46.015243+0000 mgr.gibba032.whopeo (mgr.24625) 4355 : cluster [DBG] pgmap v2468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:48 gibba011 bash[30791]: cluster 2021-03-10T15:14:48.016620+0000 mgr.gibba032.whopeo (mgr.24625) 4356 : cluster [DBG] pgmap v2469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:48 gibba011 bash[31774]: cluster 2021-03-10T15:14:48.016620+0000 mgr.gibba032.whopeo (mgr.24625) 4356 : cluster [DBG] pgmap v2469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:48 gibba032 bash[24004]: cluster 2021-03-10T15:14:48.016620+0000 mgr.gibba032.whopeo (mgr.24625) 4356 : cluster [DBG] pgmap v2469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:49.216 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:14:49.445 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:14:49.445 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (81m) 6m ago 86m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:14:49.445 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (86m) 5m ago 86m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:14:49.445 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (81m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:14:49.446 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (82m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:14:49.446 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (81m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:14:49.446 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (81m) 6m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:14:49.446 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (80m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:14:49.447 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (80m) 6m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:14:49.447 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (86m) 6m ago 86m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:14:49.447 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (86m) 5m ago 86m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:14:49.447 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (79m) 6m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:14:49.447 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (79m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:14:49.448 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (78m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:14:49.448 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (78m) 6m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:14:49.448 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (78m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:14:49.448 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (77m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:14:49.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (77m) 5m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:14:49.449 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (76m) 5m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:14:49.449 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (86m) 5m ago 87m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:14:49.449 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 84m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:14:49.694 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:14:49.695 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:14:49.695 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:14:49.695 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:49.695 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:14:49.696 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:14:49.696 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:49.696 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:14:49.696 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:14:49.697 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:14:49.697 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:14:49.697 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:14:49.697 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:14:49.697 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:14:49.698 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:14:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:49 gibba032 bash[24004]: audit 2021-03-10T15:14:49.691993+0000 mon.c (mon.1) 84 : audit [DBG] from='client.? 172.21.2.111:0/314006672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:49 gibba011 bash[30791]: audit 2021-03-10T15:14:49.691993+0000 mon.c (mon.1) 84 : audit [DBG] from='client.? 172.21.2.111:0/314006672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:49 gibba011 bash[31774]: audit 2021-03-10T15:14:49.691993+0000 mon.c (mon.1) 84 : audit [DBG] from='client.? 172.21.2.111:0/314006672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:14:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[30791]: audit 2021-03-10T15:14:49.202739+0000 mgr.gibba032.whopeo (mgr.24625) 4357 : audit [DBG] from='client.46971 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[30791]: audit 2021-03-10T15:14:49.427836+0000 mgr.gibba032.whopeo (mgr.24625) 4358 : audit [DBG] from='client.46977 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[30791]: cluster 2021-03-10T15:14:50.017333+0000 mgr.gibba032.whopeo (mgr.24625) 4359 : cluster [DBG] pgmap v2470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[31774]: audit 2021-03-10T15:14:49.202739+0000 mgr.gibba032.whopeo (mgr.24625) 4357 : audit [DBG] from='client.46971 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[31774]: audit 2021-03-10T15:14:49.427836+0000 mgr.gibba032.whopeo (mgr.24625) 4358 : audit [DBG] from='client.46977 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:50 gibba011 bash[31774]: cluster 2021-03-10T15:14:50.017333+0000 mgr.gibba032.whopeo (mgr.24625) 4359 : cluster [DBG] pgmap v2470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:50 gibba032 bash[24004]: audit 2021-03-10T15:14:49.202739+0000 mgr.gibba032.whopeo (mgr.24625) 4357 : audit [DBG] from='client.46971 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:50 gibba032 bash[24004]: audit 2021-03-10T15:14:49.427836+0000 mgr.gibba032.whopeo (mgr.24625) 4358 : audit [DBG] from='client.46977 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:14:51.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:50 gibba032 bash[24004]: cluster 2021-03-10T15:14:50.017333+0000 mgr.gibba032.whopeo (mgr.24625) 4359 : cluster [DBG] pgmap v2470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:53 gibba032 bash[24004]: cluster 2021-03-10T15:14:52.018878+0000 mgr.gibba032.whopeo (mgr.24625) 4360 : cluster [DBG] pgmap v2471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:53 gibba011 bash[30791]: cluster 2021-03-10T15:14:52.018878+0000 mgr.gibba032.whopeo (mgr.24625) 4360 : cluster [DBG] pgmap v2471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:53 gibba011 bash[31774]: cluster 2021-03-10T15:14:52.018878+0000 mgr.gibba032.whopeo (mgr.24625) 4360 : cluster [DBG] pgmap v2471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:55 gibba032 bash[24004]: cluster 2021-03-10T15:14:54.020479+0000 mgr.gibba032.whopeo (mgr.24625) 4361 : cluster [DBG] pgmap v2472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:55 gibba011 bash[30791]: cluster 2021-03-10T15:14:54.020479+0000 mgr.gibba032.whopeo (mgr.24625) 4361 : cluster [DBG] pgmap v2472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:55 gibba011 bash[31774]: cluster 2021-03-10T15:14:54.020479+0000 mgr.gibba032.whopeo (mgr.24625) 4361 : cluster [DBG] pgmap v2472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:56 gibba032 bash[24004]: cephadm 2021-03-10T15:14:54.225027+0000 mgr.gibba032.whopeo (mgr.24625) 4362 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:56 gibba011 bash[30791]: cephadm 2021-03-10T15:14:54.225027+0000 mgr.gibba032.whopeo (mgr.24625) 4362 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:56 gibba011 bash[31774]: cephadm 2021-03-10T15:14:54.225027+0000 mgr.gibba032.whopeo (mgr.24625) 4362 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:14:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: cluster 2021-03-10T15:14:56.022049+0000 mgr.gibba032.whopeo (mgr.24625) 4363 : cluster [DBG] pgmap v2473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.529270+0000 mon.a (mon.0) 1789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:14:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.529403+0000 mon.a (mon.0) 1790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:14:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.540701+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:14:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.540882+0000 mon.a (mon.0) 1792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:14:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.541030+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:14:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:57 gibba032 bash[24004]: audit 2021-03-10T15:14:56.541171+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:14:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: cluster 2021-03-10T15:14:56.022049+0000 mgr.gibba032.whopeo (mgr.24625) 4363 : cluster [DBG] pgmap v2473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.529270+0000 mon.a (mon.0) 1789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:14:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.529403+0000 mon.a (mon.0) 1790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:14:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.540701+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:14:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.540882+0000 mon.a (mon.0) 1792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:14:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.541030+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:14:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[30791]: audit 2021-03-10T15:14:56.541171+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:14:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: cluster 2021-03-10T15:14:56.022049+0000 mgr.gibba032.whopeo (mgr.24625) 4363 : cluster [DBG] pgmap v2473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.529270+0000 mon.a (mon.0) 1789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:14:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.529403+0000 mon.a (mon.0) 1790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:14:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.540701+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:14:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.540882+0000 mon.a (mon.0) 1792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T15:14:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.541030+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:14:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:57 gibba011 bash[31774]: audit 2021-03-10T15:14:56.541171+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:14:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:14:58 gibba011 bash[30791]: cluster 2021-03-10T15:14:58.022532+0000 mgr.gibba032.whopeo (mgr.24625) 4364 : cluster [DBG] pgmap v2474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:14:58 gibba011 bash[31774]: cluster 2021-03-10T15:14:58.022532+0000 mgr.gibba032.whopeo (mgr.24625) 4364 : cluster [DBG] pgmap v2474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:14:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:14:58 gibba032 bash[24004]: cluster 2021-03-10T15:14:58.022532+0000 mgr.gibba032.whopeo (mgr.24625) 4364 : cluster [DBG] pgmap v2474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:01.473 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:01 gibba011 bash[30791]: cluster 2021-03-10T15:15:00.022893+0000 mgr.gibba032.whopeo (mgr.24625) 4365 : cluster [DBG] pgmap v2475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:01.473 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:01 gibba011 bash[31774]: cluster 2021-03-10T15:15:00.022893+0000 mgr.gibba032.whopeo (mgr.24625) 4365 : cluster [DBG] pgmap v2475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:01 gibba032 bash[24004]: cluster 2021-03-10T15:15:00.022893+0000 mgr.gibba032.whopeo (mgr.24625) 4365 : cluster [DBG] pgmap v2475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:03 gibba032 bash[24004]: cluster 2021-03-10T15:15:02.023558+0000 mgr.gibba032.whopeo (mgr.24625) 4366 : cluster [DBG] pgmap v2476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:03 gibba011 bash[30791]: cluster 2021-03-10T15:15:02.023558+0000 mgr.gibba032.whopeo (mgr.24625) 4366 : cluster [DBG] pgmap v2476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:03 gibba011 bash[31774]: cluster 2021-03-10T15:15:02.023558+0000 mgr.gibba032.whopeo (mgr.24625) 4366 : cluster [DBG] pgmap v2476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:05 gibba032 bash[24004]: cluster 2021-03-10T15:15:04.025128+0000 mgr.gibba032.whopeo (mgr.24625) 4367 : cluster [DBG] pgmap v2477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:05 gibba011 bash[30791]: cluster 2021-03-10T15:15:04.025128+0000 mgr.gibba032.whopeo (mgr.24625) 4367 : cluster [DBG] pgmap v2477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:05.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:05 gibba011 bash[31774]: cluster 2021-03-10T15:15:04.025128+0000 mgr.gibba032.whopeo (mgr.24625) 4367 : cluster [DBG] pgmap v2477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:07 gibba032 bash[24004]: cluster 2021-03-10T15:15:06.026550+0000 mgr.gibba032.whopeo (mgr.24625) 4368 : cluster [DBG] pgmap v2478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:07.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:07 gibba011 bash[30791]: cluster 2021-03-10T15:15:06.026550+0000 mgr.gibba032.whopeo (mgr.24625) 4368 : cluster [DBG] pgmap v2478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:07.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:07 gibba011 bash[31774]: cluster 2021-03-10T15:15:06.026550+0000 mgr.gibba032.whopeo (mgr.24625) 4368 : cluster [DBG] pgmap v2478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:08 gibba032 bash[24004]: cluster 2021-03-10T15:15:08.027348+0000 mgr.gibba032.whopeo (mgr.24625) 4369 : cluster [DBG] pgmap v2479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:08 gibba011 bash[30791]: cluster 2021-03-10T15:15:08.027348+0000 mgr.gibba032.whopeo (mgr.24625) 4369 : cluster [DBG] pgmap v2479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:08 gibba011 bash[31774]: cluster 2021-03-10T15:15:08.027348+0000 mgr.gibba032.whopeo (mgr.24625) 4369 : cluster [DBG] pgmap v2479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:11.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:11 gibba032 bash[24004]: cephadm 2021-03-10T15:15:09.240850+0000 mgr.gibba032.whopeo (mgr.24625) 4370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:11 gibba032 bash[24004]: cluster 2021-03-10T15:15:10.028039+0000 mgr.gibba032.whopeo (mgr.24625) 4371 : cluster [DBG] pgmap v2480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:11 gibba011 bash[30791]: cephadm 2021-03-10T15:15:09.240850+0000 mgr.gibba032.whopeo (mgr.24625) 4370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:11 gibba011 bash[30791]: cluster 2021-03-10T15:15:10.028039+0000 mgr.gibba032.whopeo (mgr.24625) 4371 : cluster [DBG] pgmap v2480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:11 gibba011 bash[31774]: cephadm 2021-03-10T15:15:09.240850+0000 mgr.gibba032.whopeo (mgr.24625) 4370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:11 gibba011 bash[31774]: cluster 2021-03-10T15:15:10.028039+0000 mgr.gibba032.whopeo (mgr.24625) 4371 : cluster [DBG] pgmap v2480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:13.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:13 gibba032 bash[24004]: cluster 2021-03-10T15:15:12.029425+0000 mgr.gibba032.whopeo (mgr.24625) 4372 : cluster [DBG] pgmap v2481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:13.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:13 gibba011 bash[30791]: cluster 2021-03-10T15:15:12.029425+0000 mgr.gibba032.whopeo (mgr.24625) 4372 : cluster [DBG] pgmap v2481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:13.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:13 gibba011 bash[31774]: cluster 2021-03-10T15:15:12.029425+0000 mgr.gibba032.whopeo (mgr.24625) 4372 : cluster [DBG] pgmap v2481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:15 gibba032 bash[24004]: cluster 2021-03-10T15:15:14.030974+0000 mgr.gibba032.whopeo (mgr.24625) 4373 : cluster [DBG] pgmap v2482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:15.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:15 gibba011 bash[30791]: cluster 2021-03-10T15:15:14.030974+0000 mgr.gibba032.whopeo (mgr.24625) 4373 : cluster [DBG] pgmap v2482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:15 gibba011 bash[31774]: cluster 2021-03-10T15:15:14.030974+0000 mgr.gibba032.whopeo (mgr.24625) 4373 : cluster [DBG] pgmap v2482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:17 gibba032 bash[24004]: cluster 2021-03-10T15:15:16.032550+0000 mgr.gibba032.whopeo (mgr.24625) 4374 : cluster [DBG] pgmap v2483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:17 gibba011 bash[30791]: cluster 2021-03-10T15:15:16.032550+0000 mgr.gibba032.whopeo (mgr.24625) 4374 : cluster [DBG] pgmap v2483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:17 gibba011 bash[31774]: cluster 2021-03-10T15:15:16.032550+0000 mgr.gibba032.whopeo (mgr.24625) 4374 : cluster [DBG] pgmap v2483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:18 gibba032 bash[24004]: cluster 2021-03-10T15:15:18.033276+0000 mgr.gibba032.whopeo (mgr.24625) 4375 : cluster [DBG] pgmap v2484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:18 gibba011 bash[30791]: cluster 2021-03-10T15:15:18.033276+0000 mgr.gibba032.whopeo (mgr.24625) 4375 : cluster [DBG] pgmap v2484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:18 gibba011 bash[31774]: cluster 2021-03-10T15:15:18.033276+0000 mgr.gibba032.whopeo (mgr.24625) 4375 : cluster [DBG] pgmap v2484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:19.938 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:15:20.168 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:15:20.169 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (81m) 6m ago 87m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:15:20.169 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (87m) 5m ago 87m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:15:20.169 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (82m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:15:20.169 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (82m) 5m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:15:20.170 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (81m) 6m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:15:20.170 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (81m) 6m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:15:20.170 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (81m) 5m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:15:20.170 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (81m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:15:20.171 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (87m) 6m ago 87m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:15:20.171 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (87m) 5m ago 87m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:15:20.171 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (80m) 6m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:15:20.172 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (79m) 6m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:15:20.172 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (79m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:15:20.172 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (78m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:15:20.172 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (78m) 5m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:15:20.173 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (78m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:15:20.173 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (77m) 5m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:15:20.173 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (77m) 5m ago 87m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:15:20.173 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (86m) 5m ago 87m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:15:20.173 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 85m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:15:20.419 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:15:20.419 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:15:20.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:15:20.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:20.420 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:15:20.420 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:15:20.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:20.420 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:15:20.421 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:15:20.421 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:20.421 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:15:20.421 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:15:20.421 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:15:20.422 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:15:20.422 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:15:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:21 gibba032 bash[24004]: audit 2021-03-10T15:15:19.924772+0000 mgr.gibba032.whopeo (mgr.24625) 4376 : audit [DBG] from='client.56974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:21.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:21 gibba032 bash[24004]: cluster 2021-03-10T15:15:20.033976+0000 mgr.gibba032.whopeo (mgr.24625) 4377 : cluster [DBG] pgmap v2485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:21.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:21 gibba032 bash[24004]: audit 2021-03-10T15:15:20.416388+0000 mon.a (mon.0) 1795 : audit [DBG] from='client.? 172.21.2.111:0/4049599269' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:21.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[30791]: audit 2021-03-10T15:15:19.924772+0000 mgr.gibba032.whopeo (mgr.24625) 4376 : audit [DBG] from='client.56974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:21.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[30791]: cluster 2021-03-10T15:15:20.033976+0000 mgr.gibba032.whopeo (mgr.24625) 4377 : cluster [DBG] pgmap v2485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:21.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[30791]: audit 2021-03-10T15:15:20.416388+0000 mon.a (mon.0) 1795 : audit [DBG] from='client.? 172.21.2.111:0/4049599269' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:21.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[31774]: audit 2021-03-10T15:15:19.924772+0000 mgr.gibba032.whopeo (mgr.24625) 4376 : audit [DBG] from='client.56974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:21.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[31774]: cluster 2021-03-10T15:15:20.033976+0000 mgr.gibba032.whopeo (mgr.24625) 4377 : cluster [DBG] pgmap v2485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:21.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:21 gibba011 bash[31774]: audit 2021-03-10T15:15:20.416388+0000 mon.a (mon.0) 1795 : audit [DBG] from='client.? 172.21.2.111:0/4049599269' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:22 gibba032 bash[24004]: audit 2021-03-10T15:15:20.149516+0000 mgr.gibba032.whopeo (mgr.24625) 4378 : audit [DBG] from='client.46995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:22 gibba011 bash[30791]: audit 2021-03-10T15:15:20.149516+0000 mgr.gibba032.whopeo (mgr.24625) 4378 : audit [DBG] from='client.46995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:22 gibba011 bash[31774]: audit 2021-03-10T15:15:20.149516+0000 mgr.gibba032.whopeo (mgr.24625) 4378 : audit [DBG] from='client.46995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:23 gibba032 bash[24004]: cluster 2021-03-10T15:15:22.035404+0000 mgr.gibba032.whopeo (mgr.24625) 4379 : cluster [DBG] pgmap v2486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:23 gibba011 bash[30791]: cluster 2021-03-10T15:15:22.035404+0000 mgr.gibba032.whopeo (mgr.24625) 4379 : cluster [DBG] pgmap v2486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:23 gibba011 bash[31774]: cluster 2021-03-10T15:15:22.035404+0000 mgr.gibba032.whopeo (mgr.24625) 4379 : cluster [DBG] pgmap v2486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:25 gibba032 bash[24004]: cluster 2021-03-10T15:15:24.037054+0000 mgr.gibba032.whopeo (mgr.24625) 4380 : cluster [DBG] pgmap v2487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:25.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:25 gibba011 bash[30791]: cluster 2021-03-10T15:15:24.037054+0000 mgr.gibba032.whopeo (mgr.24625) 4380 : cluster [DBG] pgmap v2487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:25.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:25 gibba011 bash[31774]: cluster 2021-03-10T15:15:24.037054+0000 mgr.gibba032.whopeo (mgr.24625) 4380 : cluster [DBG] pgmap v2487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:26 gibba032 bash[24004]: cephadm 2021-03-10T15:15:24.254520+0000 mgr.gibba032.whopeo (mgr.24625) 4381 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:26 gibba011 bash[30791]: cephadm 2021-03-10T15:15:24.254520+0000 mgr.gibba032.whopeo (mgr.24625) 4381 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:26 gibba011 bash[31774]: cephadm 2021-03-10T15:15:24.254520+0000 mgr.gibba032.whopeo (mgr.24625) 4381 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:27.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:27 gibba032 bash[24004]: cluster 2021-03-10T15:15:26.038498+0000 mgr.gibba032.whopeo (mgr.24625) 4382 : cluster [DBG] pgmap v2488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:27.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:27 gibba011 bash[30791]: cluster 2021-03-10T15:15:26.038498+0000 mgr.gibba032.whopeo (mgr.24625) 4382 : cluster [DBG] pgmap v2488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:27.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:27 gibba011 bash[31774]: cluster 2021-03-10T15:15:26.038498+0000 mgr.gibba032.whopeo (mgr.24625) 4382 : cluster [DBG] pgmap v2488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:28 gibba032 bash[24004]: cluster 2021-03-10T15:15:28.039396+0000 mgr.gibba032.whopeo (mgr.24625) 4383 : cluster [DBG] pgmap v2489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:28.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:28 gibba011 bash[30791]: cluster 2021-03-10T15:15:28.039396+0000 mgr.gibba032.whopeo (mgr.24625) 4383 : cluster [DBG] pgmap v2489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:28 gibba011 bash[31774]: cluster 2021-03-10T15:15:28.039396+0000 mgr.gibba032.whopeo (mgr.24625) 4383 : cluster [DBG] pgmap v2489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:31.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:31 gibba032 bash[24004]: cluster 2021-03-10T15:15:30.040141+0000 mgr.gibba032.whopeo (mgr.24625) 4384 : cluster [DBG] pgmap v2490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:31.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:31 gibba011 bash[30791]: cluster 2021-03-10T15:15:30.040141+0000 mgr.gibba032.whopeo (mgr.24625) 4384 : cluster [DBG] pgmap v2490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:31 gibba011 bash[31774]: cluster 2021-03-10T15:15:30.040141+0000 mgr.gibba032.whopeo (mgr.24625) 4384 : cluster [DBG] pgmap v2490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:33.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:33 gibba032 bash[24004]: cluster 2021-03-10T15:15:32.041533+0000 mgr.gibba032.whopeo (mgr.24625) 4385 : cluster [DBG] pgmap v2491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:33 gibba011 bash[30791]: cluster 2021-03-10T15:15:32.041533+0000 mgr.gibba032.whopeo (mgr.24625) 4385 : cluster [DBG] pgmap v2491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:33.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:33 gibba011 bash[31774]: cluster 2021-03-10T15:15:32.041533+0000 mgr.gibba032.whopeo (mgr.24625) 4385 : cluster [DBG] pgmap v2491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:35 gibba032 bash[24004]: cluster 2021-03-10T15:15:34.043081+0000 mgr.gibba032.whopeo (mgr.24625) 4386 : cluster [DBG] pgmap v2492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:35 gibba011 bash[30791]: cluster 2021-03-10T15:15:34.043081+0000 mgr.gibba032.whopeo (mgr.24625) 4386 : cluster [DBG] pgmap v2492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:35 gibba011 bash[31774]: cluster 2021-03-10T15:15:34.043081+0000 mgr.gibba032.whopeo (mgr.24625) 4386 : cluster [DBG] pgmap v2492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:36 gibba032 bash[24004]: cluster 2021-03-10T15:15:36.044568+0000 mgr.gibba032.whopeo (mgr.24625) 4387 : cluster [DBG] pgmap v2493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:36 gibba011 bash[30791]: cluster 2021-03-10T15:15:36.044568+0000 mgr.gibba032.whopeo (mgr.24625) 4387 : cluster [DBG] pgmap v2493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:36 gibba011 bash[31774]: cluster 2021-03-10T15:15:36.044568+0000 mgr.gibba032.whopeo (mgr.24625) 4387 : cluster [DBG] pgmap v2493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:38 gibba032 bash[24004]: cluster 2021-03-10T15:15:38.045330+0000 mgr.gibba032.whopeo (mgr.24625) 4388 : cluster [DBG] pgmap v2494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:38 gibba011 bash[30791]: cluster 2021-03-10T15:15:38.045330+0000 mgr.gibba032.whopeo (mgr.24625) 4388 : cluster [DBG] pgmap v2494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:38 gibba011 bash[31774]: cluster 2021-03-10T15:15:38.045330+0000 mgr.gibba032.whopeo (mgr.24625) 4388 : cluster [DBG] pgmap v2494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.472680+0000 mgr.gibba032.whopeo (mgr.24625) 4389 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:15:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.473821+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.487135+0000 mgr.gibba032.whopeo (mgr.24625) 4390 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:15:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.487819+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.694023+0000 mon.a (mon.0) 1798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.695006+0000 mgr.gibba032.whopeo (mgr.24625) 4391 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:15:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.696976+0000 mgr.gibba032.whopeo (mgr.24625) 4392 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:15:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.697517+0000 mon.a (mon.0) 1799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:15:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.698423+0000 mgr.gibba032.whopeo (mgr.24625) 4393 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:15:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.698474+0000 mgr.gibba032.whopeo (mgr.24625) 4394 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:15:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.699095+0000 mon.a (mon.0) 1800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.700294+0000 mgr.gibba032.whopeo (mgr.24625) 4395 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:15:40.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.700345+0000 mgr.gibba032.whopeo (mgr.24625) 4396 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:15:40.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.700877+0000 mon.a (mon.0) 1801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.701678+0000 mgr.gibba032.whopeo (mgr.24625) 4397 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:15:40.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.702467+0000 mon.a (mon.0) 1802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.703272+0000 mgr.gibba032.whopeo (mgr.24625) 4398 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:15:40.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.703322+0000 mgr.gibba032.whopeo (mgr.24625) 4399 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:15:40.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.705161+0000 mgr.gibba032.whopeo (mgr.24625) 4400 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:15:40.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.705687+0000 mgr.gibba032.whopeo (mgr.24625) 4401 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:15:40.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.705698+0000 mon.a (mon.0) 1803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.707253+0000 mgr.gibba032.whopeo (mgr.24625) 4402 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:15:40.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.707309+0000 mgr.gibba032.whopeo (mgr.24625) 4403 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:15:40.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cephadm 2021-03-10T15:15:39.707462+0000 mgr.gibba032.whopeo (mgr.24625) 4404 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:40.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: audit 2021-03-10T15:15:39.707877+0000 mon.a (mon.0) 1804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:40 gibba032 bash[24004]: cluster 2021-03-10T15:15:40.046032+0000 mgr.gibba032.whopeo (mgr.24625) 4405 : cluster [DBG] pgmap v2495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.472680+0000 mgr.gibba032.whopeo (mgr.24625) 4389 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:15:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.473821+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.487135+0000 mgr.gibba032.whopeo (mgr.24625) 4390 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:15:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.487819+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.694023+0000 mon.a (mon.0) 1798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.695006+0000 mgr.gibba032.whopeo (mgr.24625) 4391 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:15:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.696976+0000 mgr.gibba032.whopeo (mgr.24625) 4392 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:15:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.697517+0000 mon.a (mon.0) 1799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:15:40.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.698423+0000 mgr.gibba032.whopeo (mgr.24625) 4393 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:15:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.698474+0000 mgr.gibba032.whopeo (mgr.24625) 4394 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:15:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.699095+0000 mon.a (mon.0) 1800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.700294+0000 mgr.gibba032.whopeo (mgr.24625) 4395 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:15:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.700345+0000 mgr.gibba032.whopeo (mgr.24625) 4396 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:15:40.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.700877+0000 mon.a (mon.0) 1801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.701678+0000 mgr.gibba032.whopeo (mgr.24625) 4397 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:15:40.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.702467+0000 mon.a (mon.0) 1802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.703272+0000 mgr.gibba032.whopeo (mgr.24625) 4398 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:15:40.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.703322+0000 mgr.gibba032.whopeo (mgr.24625) 4399 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:15:40.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.705161+0000 mgr.gibba032.whopeo (mgr.24625) 4400 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:15:40.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.705687+0000 mgr.gibba032.whopeo (mgr.24625) 4401 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:15:40.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.705698+0000 mon.a (mon.0) 1803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.472680+0000 mgr.gibba032.whopeo (mgr.24625) 4389 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:15:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.473821+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.487135+0000 mgr.gibba032.whopeo (mgr.24625) 4390 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:15:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.487819+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.694023+0000 mon.a (mon.0) 1798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:15:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.695006+0000 mgr.gibba032.whopeo (mgr.24625) 4391 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:15:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.696976+0000 mgr.gibba032.whopeo (mgr.24625) 4392 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:15:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.697517+0000 mon.a (mon.0) 1799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:15:40.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.698423+0000 mgr.gibba032.whopeo (mgr.24625) 4393 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:15:40.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.698474+0000 mgr.gibba032.whopeo (mgr.24625) 4394 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:15:40.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.699095+0000 mon.a (mon.0) 1800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.700294+0000 mgr.gibba032.whopeo (mgr.24625) 4395 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:15:40.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.700345+0000 mgr.gibba032.whopeo (mgr.24625) 4396 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:15:40.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.700877+0000 mon.a (mon.0) 1801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.701678+0000 mgr.gibba032.whopeo (mgr.24625) 4397 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:15:40.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.702467+0000 mon.a (mon.0) 1802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.703272+0000 mgr.gibba032.whopeo (mgr.24625) 4398 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:15:40.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.703322+0000 mgr.gibba032.whopeo (mgr.24625) 4399 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:15:40.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.705161+0000 mgr.gibba032.whopeo (mgr.24625) 4400 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:15:40.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.705687+0000 mgr.gibba032.whopeo (mgr.24625) 4401 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:15:40.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.705698+0000 mon.a (mon.0) 1803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.707253+0000 mgr.gibba032.whopeo (mgr.24625) 4402 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:15:40.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.707309+0000 mgr.gibba032.whopeo (mgr.24625) 4403 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:15:40.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cephadm 2021-03-10T15:15:39.707462+0000 mgr.gibba032.whopeo (mgr.24625) 4404 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:40.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: audit 2021-03-10T15:15:39.707877+0000 mon.a (mon.0) 1804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[30791]: cluster 2021-03-10T15:15:40.046032+0000 mgr.gibba032.whopeo (mgr.24625) 4405 : cluster [DBG] pgmap v2495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:40.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.707253+0000 mgr.gibba032.whopeo (mgr.24625) 4402 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:15:40.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.707309+0000 mgr.gibba032.whopeo (mgr.24625) 4403 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:15:40.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cephadm 2021-03-10T15:15:39.707462+0000 mgr.gibba032.whopeo (mgr.24625) 4404 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:40.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: audit 2021-03-10T15:15:39.707877+0000 mon.a (mon.0) 1804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:40.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:40 gibba011 bash[31774]: cluster 2021-03-10T15:15:40.046032+0000 mgr.gibba032.whopeo (mgr.24625) 4405 : cluster [DBG] pgmap v2495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:43 gibba032 bash[24004]: cluster 2021-03-10T15:15:42.047536+0000 mgr.gibba032.whopeo (mgr.24625) 4406 : cluster [DBG] pgmap v2496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:43.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:43 gibba011 bash[30791]: cluster 2021-03-10T15:15:42.047536+0000 mgr.gibba032.whopeo (mgr.24625) 4406 : cluster [DBG] pgmap v2496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:43.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:43 gibba011 bash[31774]: cluster 2021-03-10T15:15:42.047536+0000 mgr.gibba032.whopeo (mgr.24625) 4406 : cluster [DBG] pgmap v2496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:45 gibba032 bash[24004]: cluster 2021-03-10T15:15:44.049286+0000 mgr.gibba032.whopeo (mgr.24625) 4407 : cluster [DBG] pgmap v2497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:45.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:45 gibba011 bash[30791]: cluster 2021-03-10T15:15:44.049286+0000 mgr.gibba032.whopeo (mgr.24625) 4407 : cluster [DBG] pgmap v2497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:45 gibba011 bash[31774]: cluster 2021-03-10T15:15:44.049286+0000 mgr.gibba032.whopeo (mgr.24625) 4407 : cluster [DBG] pgmap v2497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:47 gibba032 bash[24004]: cluster 2021-03-10T15:15:46.050936+0000 mgr.gibba032.whopeo (mgr.24625) 4408 : cluster [DBG] pgmap v2498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:47.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:47 gibba011 bash[30791]: cluster 2021-03-10T15:15:46.050936+0000 mgr.gibba032.whopeo (mgr.24625) 4408 : cluster [DBG] pgmap v2498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:47.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:47 gibba011 bash[31774]: cluster 2021-03-10T15:15:46.050936+0000 mgr.gibba032.whopeo (mgr.24625) 4408 : cluster [DBG] pgmap v2498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:48 gibba032 bash[24004]: cluster 2021-03-10T15:15:48.051793+0000 mgr.gibba032.whopeo (mgr.24625) 4409 : cluster [DBG] pgmap v2499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:48 gibba011 bash[30791]: cluster 2021-03-10T15:15:48.051793+0000 mgr.gibba032.whopeo (mgr.24625) 4409 : cluster [DBG] pgmap v2499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:48 gibba011 bash[31774]: cluster 2021-03-10T15:15:48.051793+0000 mgr.gibba032.whopeo (mgr.24625) 4409 : cluster [DBG] pgmap v2499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:50.666 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:15:50.892 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:15:50.892 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (82m) 7m ago 87m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:15:50.892 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (87m) 6m ago 87m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:15:50.893 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (82m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:15:50.893 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (83m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:15:50.893 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (82m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:15:50.893 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (82m) 7m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:15:50.894 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (81m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:15:50.894 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (81m) 7m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:15:50.894 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (87m) 7m ago 88m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:15:50.894 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (87m) 6m ago 88m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:15:50.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (80m) 7m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:15:50.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (80m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:15:50.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (79m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:15:50.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (79m) 7m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:15:50.896 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (79m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:15:50.896 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (78m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:15:50.896 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (78m) 6m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:15:50.896 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (77m) 6m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:15:50.896 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (87m) 6m ago 88m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:15:50.897 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 85m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:15:51.142 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:15:51.142 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:15:51.142 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:15:51.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:51.143 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:15:51.143 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:15:51.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:51.143 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:15:51.144 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:15:51.144 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:15:51.144 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:15:51.144 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:15:51.145 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:15:51.145 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:15:51.145 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:15:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:51 gibba032 bash[24004]: cluster 2021-03-10T15:15:50.052717+0000 mgr.gibba032.whopeo (mgr.24625) 4410 : cluster [DBG] pgmap v2500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:51 gibba011 bash[30791]: cluster 2021-03-10T15:15:50.052717+0000 mgr.gibba032.whopeo (mgr.24625) 4410 : cluster [DBG] pgmap v2500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:51.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:51 gibba011 bash[31774]: cluster 2021-03-10T15:15:50.052717+0000 mgr.gibba032.whopeo (mgr.24625) 4410 : cluster [DBG] pgmap v2500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:52 gibba032 bash[24004]: audit 2021-03-10T15:15:50.651251+0000 mgr.gibba032.whopeo (mgr.24625) 4411 : audit [DBG] from='client.47007 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:52 gibba032 bash[24004]: audit 2021-03-10T15:15:50.875201+0000 mgr.gibba032.whopeo (mgr.24625) 4412 : audit [DBG] from='client.47013 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:52 gibba032 bash[24004]: audit 2021-03-10T15:15:51.139410+0000 mon.c (mon.1) 85 : audit [DBG] from='client.? 172.21.2.111:0/3894307995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[30791]: audit 2021-03-10T15:15:50.651251+0000 mgr.gibba032.whopeo (mgr.24625) 4411 : audit [DBG] from='client.47007 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[30791]: audit 2021-03-10T15:15:50.875201+0000 mgr.gibba032.whopeo (mgr.24625) 4412 : audit [DBG] from='client.47013 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[30791]: audit 2021-03-10T15:15:51.139410+0000 mon.c (mon.1) 85 : audit [DBG] from='client.? 172.21.2.111:0/3894307995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[31774]: audit 2021-03-10T15:15:50.651251+0000 mgr.gibba032.whopeo (mgr.24625) 4411 : audit [DBG] from='client.47007 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[31774]: audit 2021-03-10T15:15:50.875201+0000 mgr.gibba032.whopeo (mgr.24625) 4412 : audit [DBG] from='client.47013 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:15:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:52 gibba011 bash[31774]: audit 2021-03-10T15:15:51.139410+0000 mon.c (mon.1) 85 : audit [DBG] from='client.? 172.21.2.111:0/3894307995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:15:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:53 gibba032 bash[24004]: cluster 2021-03-10T15:15:52.054119+0000 mgr.gibba032.whopeo (mgr.24625) 4413 : cluster [DBG] pgmap v2501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:53 gibba011 bash[30791]: cluster 2021-03-10T15:15:52.054119+0000 mgr.gibba032.whopeo (mgr.24625) 4413 : cluster [DBG] pgmap v2501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:53.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:53 gibba011 bash[31774]: cluster 2021-03-10T15:15:52.054119+0000 mgr.gibba032.whopeo (mgr.24625) 4413 : cluster [DBG] pgmap v2501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:54 gibba032 bash[24004]: cluster 2021-03-10T15:15:54.055859+0000 mgr.gibba032.whopeo (mgr.24625) 4414 : cluster [DBG] pgmap v2502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:54 gibba011 bash[30791]: cluster 2021-03-10T15:15:54.055859+0000 mgr.gibba032.whopeo (mgr.24625) 4414 : cluster [DBG] pgmap v2502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:54 gibba011 bash[31774]: cluster 2021-03-10T15:15:54.055859+0000 mgr.gibba032.whopeo (mgr.24625) 4414 : cluster [DBG] pgmap v2502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:55 gibba032 bash[24004]: cephadm 2021-03-10T15:15:54.723541+0000 mgr.gibba032.whopeo (mgr.24625) 4415 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:55 gibba011 bash[30791]: cephadm 2021-03-10T15:15:54.723541+0000 mgr.gibba032.whopeo (mgr.24625) 4415 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:55 gibba011 bash[31774]: cephadm 2021-03-10T15:15:54.723541+0000 mgr.gibba032.whopeo (mgr.24625) 4415 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:15:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:56 gibba032 bash[24004]: cluster 2021-03-10T15:15:56.057346+0000 mgr.gibba032.whopeo (mgr.24625) 4416 : cluster [DBG] pgmap v2503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:56 gibba011 bash[30791]: cluster 2021-03-10T15:15:56.057346+0000 mgr.gibba032.whopeo (mgr.24625) 4416 : cluster [DBG] pgmap v2503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:56 gibba011 bash[31774]: cluster 2021-03-10T15:15:56.057346+0000 mgr.gibba032.whopeo (mgr.24625) 4416 : cluster [DBG] pgmap v2503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.535426+0000 mon.a (mon.0) 1805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:15:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.536060+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:15:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.544059+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:15:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.544773+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:15:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.545446+0000 mon.a (mon.0) 1809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:15:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:57 gibba032 bash[24004]: audit 2021-03-10T15:15:56.546119+0000 mon.a (mon.0) 1810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:15:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.535426+0000 mon.a (mon.0) 1805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:15:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.536060+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:15:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.544059+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:15:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.544773+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:15:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.545446+0000 mon.a (mon.0) 1809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:15:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[30791]: audit 2021-03-10T15:15:56.546119+0000 mon.a (mon.0) 1810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:15:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.535426+0000 mon.a (mon.0) 1805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:15:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.536060+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:15:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.544059+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:15:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.544773+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:15:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.545446+0000 mon.a (mon.0) 1809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:15:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:57 gibba011 bash[31774]: audit 2021-03-10T15:15:56.546119+0000 mon.a (mon.0) 1810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:15:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:15:58 gibba032 bash[24004]: cluster 2021-03-10T15:15:58.058078+0000 mgr.gibba032.whopeo (mgr.24625) 4417 : cluster [DBG] pgmap v2504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:15:58 gibba011 bash[30791]: cluster 2021-03-10T15:15:58.058078+0000 mgr.gibba032.whopeo (mgr.24625) 4417 : cluster [DBG] pgmap v2504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:15:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:15:58 gibba011 bash[31774]: cluster 2021-03-10T15:15:58.058078+0000 mgr.gibba032.whopeo (mgr.24625) 4417 : cluster [DBG] pgmap v2504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:01 gibba032 bash[24004]: cluster 2021-03-10T15:16:00.058427+0000 mgr.gibba032.whopeo (mgr.24625) 4418 : cluster [DBG] pgmap v2505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:01 gibba011 bash[30791]: cluster 2021-03-10T15:16:00.058427+0000 mgr.gibba032.whopeo (mgr.24625) 4418 : cluster [DBG] pgmap v2505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:01 gibba011 bash[31774]: cluster 2021-03-10T15:16:00.058427+0000 mgr.gibba032.whopeo (mgr.24625) 4418 : cluster [DBG] pgmap v2505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:03 gibba032 bash[24004]: cluster 2021-03-10T15:16:02.059972+0000 mgr.gibba032.whopeo (mgr.24625) 4419 : cluster [DBG] pgmap v2506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:03 gibba011 bash[30791]: cluster 2021-03-10T15:16:02.059972+0000 mgr.gibba032.whopeo (mgr.24625) 4419 : cluster [DBG] pgmap v2506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:03.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:03 gibba011 bash[31774]: cluster 2021-03-10T15:16:02.059972+0000 mgr.gibba032.whopeo (mgr.24625) 4419 : cluster [DBG] pgmap v2506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:05 gibba032 bash[24004]: cluster 2021-03-10T15:16:04.061518+0000 mgr.gibba032.whopeo (mgr.24625) 4420 : cluster [DBG] pgmap v2507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:05 gibba011 bash[30791]: cluster 2021-03-10T15:16:04.061518+0000 mgr.gibba032.whopeo (mgr.24625) 4420 : cluster [DBG] pgmap v2507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:05.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:05 gibba011 bash[31774]: cluster 2021-03-10T15:16:04.061518+0000 mgr.gibba032.whopeo (mgr.24625) 4420 : cluster [DBG] pgmap v2507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:06 gibba032 bash[24004]: cluster 2021-03-10T15:16:06.063036+0000 mgr.gibba032.whopeo (mgr.24625) 4421 : cluster [DBG] pgmap v2508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:06 gibba011 bash[30791]: cluster 2021-03-10T15:16:06.063036+0000 mgr.gibba032.whopeo (mgr.24625) 4421 : cluster [DBG] pgmap v2508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:06 gibba011 bash[31774]: cluster 2021-03-10T15:16:06.063036+0000 mgr.gibba032.whopeo (mgr.24625) 4421 : cluster [DBG] pgmap v2508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:08 gibba032 bash[24004]: cluster 2021-03-10T15:16:08.063810+0000 mgr.gibba032.whopeo (mgr.24625) 4422 : cluster [DBG] pgmap v2509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:08 gibba011 bash[30791]: cluster 2021-03-10T15:16:08.063810+0000 mgr.gibba032.whopeo (mgr.24625) 4422 : cluster [DBG] pgmap v2509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:08 gibba011 bash[31774]: cluster 2021-03-10T15:16:08.063810+0000 mgr.gibba032.whopeo (mgr.24625) 4422 : cluster [DBG] pgmap v2509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:11.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:11 gibba032 bash[24004]: cephadm 2021-03-10T15:16:09.739437+0000 mgr.gibba032.whopeo (mgr.24625) 4423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:11 gibba032 bash[24004]: cluster 2021-03-10T15:16:10.064703+0000 mgr.gibba032.whopeo (mgr.24625) 4424 : cluster [DBG] pgmap v2510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:11 gibba011 bash[30791]: cephadm 2021-03-10T15:16:09.739437+0000 mgr.gibba032.whopeo (mgr.24625) 4423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:11 gibba011 bash[30791]: cluster 2021-03-10T15:16:10.064703+0000 mgr.gibba032.whopeo (mgr.24625) 4424 : cluster [DBG] pgmap v2510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:11.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:11 gibba011 bash[31774]: cephadm 2021-03-10T15:16:09.739437+0000 mgr.gibba032.whopeo (mgr.24625) 4423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:11 gibba011 bash[31774]: cluster 2021-03-10T15:16:10.064703+0000 mgr.gibba032.whopeo (mgr.24625) 4424 : cluster [DBG] pgmap v2510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:13.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:13 gibba032 bash[24004]: cluster 2021-03-10T15:16:12.066279+0000 mgr.gibba032.whopeo (mgr.24625) 4425 : cluster [DBG] pgmap v2511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:13.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:13 gibba011 bash[30791]: cluster 2021-03-10T15:16:12.066279+0000 mgr.gibba032.whopeo (mgr.24625) 4425 : cluster [DBG] pgmap v2511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:13.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:13 gibba011 bash[31774]: cluster 2021-03-10T15:16:12.066279+0000 mgr.gibba032.whopeo (mgr.24625) 4425 : cluster [DBG] pgmap v2511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:14 gibba032 bash[24004]: cluster 2021-03-10T15:16:14.067709+0000 mgr.gibba032.whopeo (mgr.24625) 4426 : cluster [DBG] pgmap v2512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:14 gibba011 bash[30791]: cluster 2021-03-10T15:16:14.067709+0000 mgr.gibba032.whopeo (mgr.24625) 4426 : cluster [DBG] pgmap v2512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:14 gibba011 bash[31774]: cluster 2021-03-10T15:16:14.067709+0000 mgr.gibba032.whopeo (mgr.24625) 4426 : cluster [DBG] pgmap v2512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:17.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:17 gibba032 bash[24004]: cluster 2021-03-10T15:16:16.069209+0000 mgr.gibba032.whopeo (mgr.24625) 4427 : cluster [DBG] pgmap v2513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:17.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:17 gibba011 bash[30791]: cluster 2021-03-10T15:16:16.069209+0000 mgr.gibba032.whopeo (mgr.24625) 4427 : cluster [DBG] pgmap v2513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:17.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:17 gibba011 bash[31774]: cluster 2021-03-10T15:16:16.069209+0000 mgr.gibba032.whopeo (mgr.24625) 4427 : cluster [DBG] pgmap v2513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:18 gibba032 bash[24004]: cluster 2021-03-10T15:16:18.070147+0000 mgr.gibba032.whopeo (mgr.24625) 4428 : cluster [DBG] pgmap v2514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:18 gibba011 bash[30791]: cluster 2021-03-10T15:16:18.070147+0000 mgr.gibba032.whopeo (mgr.24625) 4428 : cluster [DBG] pgmap v2514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:18 gibba011 bash[31774]: cluster 2021-03-10T15:16:18.070147+0000 mgr.gibba032.whopeo (mgr.24625) 4428 : cluster [DBG] pgmap v2514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:21.382 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:16:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:21 gibba032 bash[24004]: cluster 2021-03-10T15:16:20.071002+0000 mgr.gibba032.whopeo (mgr.24625) 4429 : cluster [DBG] pgmap v2515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:21.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:21 gibba011 bash[30791]: cluster 2021-03-10T15:16:20.071002+0000 mgr.gibba032.whopeo (mgr.24625) 4429 : cluster [DBG] pgmap v2515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:21.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:21 gibba011 bash[31774]: cluster 2021-03-10T15:16:20.071002+0000 mgr.gibba032.whopeo (mgr.24625) 4429 : cluster [DBG] pgmap v2515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:21.615 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:16:21.615 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (83m) 7m ago 88m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:16:21.616 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (88m) 6m ago 88m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:16:21.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (83m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:16:21.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (83m) 6m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:16:21.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (82m) 7m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:16:21.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (82m) 7m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:16:21.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (82m) 6m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:16:21.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (82m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:16:21.617 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (88m) 7m ago 88m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:16:21.618 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (88m) 6m ago 88m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:16:21.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (81m) 7m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:16:21.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (80m) 7m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:16:21.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (80m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:16:21.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (80m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:16:21.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (79m) 6m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:16:21.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (79m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:16:21.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (78m) 6m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:16:21.620 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (78m) 6m ago 88m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:16:21.620 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (87m) 6m ago 88m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:16:21.620 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 86m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:16:21.867 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:16:21.867 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:16:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:16:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:16:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:16:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:16:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:16:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:16:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:16:21.870 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:16:21.870 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:16:21.870 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:16:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:22 gibba032 bash[24004]: audit 2021-03-10T15:16:21.864320+0000 mon.a (mon.0) 1811 : audit [DBG] from='client.? 172.21.2.111:0/2037747621' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:22 gibba011 bash[30791]: audit 2021-03-10T15:16:21.864320+0000 mon.a (mon.0) 1811 : audit [DBG] from='client.? 172.21.2.111:0/2037747621' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:22 gibba011 bash[31774]: audit 2021-03-10T15:16:21.864320+0000 mon.a (mon.0) 1811 : audit [DBG] from='client.? 172.21.2.111:0/2037747621' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:23 gibba032 bash[24004]: audit 2021-03-10T15:16:21.369335+0000 mgr.gibba032.whopeo (mgr.24625) 4430 : audit [DBG] from='client.47025 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:23 gibba032 bash[24004]: audit 2021-03-10T15:16:21.599128+0000 mgr.gibba032.whopeo (mgr.24625) 4431 : audit [DBG] from='client.57016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:23 gibba032 bash[24004]: cluster 2021-03-10T15:16:22.072543+0000 mgr.gibba032.whopeo (mgr.24625) 4432 : cluster [DBG] pgmap v2516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[30791]: audit 2021-03-10T15:16:21.369335+0000 mgr.gibba032.whopeo (mgr.24625) 4430 : audit [DBG] from='client.47025 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[30791]: audit 2021-03-10T15:16:21.599128+0000 mgr.gibba032.whopeo (mgr.24625) 4431 : audit [DBG] from='client.57016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[30791]: cluster 2021-03-10T15:16:22.072543+0000 mgr.gibba032.whopeo (mgr.24625) 4432 : cluster [DBG] pgmap v2516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[31774]: audit 2021-03-10T15:16:21.369335+0000 mgr.gibba032.whopeo (mgr.24625) 4430 : audit [DBG] from='client.47025 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[31774]: audit 2021-03-10T15:16:21.599128+0000 mgr.gibba032.whopeo (mgr.24625) 4431 : audit [DBG] from='client.57016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:23 gibba011 bash[31774]: cluster 2021-03-10T15:16:22.072543+0000 mgr.gibba032.whopeo (mgr.24625) 4432 : cluster [DBG] pgmap v2516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:24 gibba032 bash[24004]: cluster 2021-03-10T15:16:24.073868+0000 mgr.gibba032.whopeo (mgr.24625) 4433 : cluster [DBG] pgmap v2517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:24 gibba011 bash[30791]: cluster 2021-03-10T15:16:24.073868+0000 mgr.gibba032.whopeo (mgr.24625) 4433 : cluster [DBG] pgmap v2517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:24 gibba011 bash[31774]: cluster 2021-03-10T15:16:24.073868+0000 mgr.gibba032.whopeo (mgr.24625) 4433 : cluster [DBG] pgmap v2517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:25 gibba032 bash[24004]: cephadm 2021-03-10T15:16:24.755495+0000 mgr.gibba032.whopeo (mgr.24625) 4434 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:25.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:25 gibba011 bash[30791]: cephadm 2021-03-10T15:16:24.755495+0000 mgr.gibba032.whopeo (mgr.24625) 4434 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:25.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:25 gibba011 bash[31774]: cephadm 2021-03-10T15:16:24.755495+0000 mgr.gibba032.whopeo (mgr.24625) 4434 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:26 gibba032 bash[24004]: cluster 2021-03-10T15:16:26.075285+0000 mgr.gibba032.whopeo (mgr.24625) 4435 : cluster [DBG] pgmap v2518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:26 gibba011 bash[30791]: cluster 2021-03-10T15:16:26.075285+0000 mgr.gibba032.whopeo (mgr.24625) 4435 : cluster [DBG] pgmap v2518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:26 gibba011 bash[31774]: cluster 2021-03-10T15:16:26.075285+0000 mgr.gibba032.whopeo (mgr.24625) 4435 : cluster [DBG] pgmap v2518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:28 gibba032 bash[24004]: cluster 2021-03-10T15:16:28.076176+0000 mgr.gibba032.whopeo (mgr.24625) 4436 : cluster [DBG] pgmap v2519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:28 gibba011 bash[30791]: cluster 2021-03-10T15:16:28.076176+0000 mgr.gibba032.whopeo (mgr.24625) 4436 : cluster [DBG] pgmap v2519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:28 gibba011 bash[31774]: cluster 2021-03-10T15:16:28.076176+0000 mgr.gibba032.whopeo (mgr.24625) 4436 : cluster [DBG] pgmap v2519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:31.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:31 gibba032 bash[24004]: cluster 2021-03-10T15:16:30.076877+0000 mgr.gibba032.whopeo (mgr.24625) 4437 : cluster [DBG] pgmap v2520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:31.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:31 gibba011 bash[30791]: cluster 2021-03-10T15:16:30.076877+0000 mgr.gibba032.whopeo (mgr.24625) 4437 : cluster [DBG] pgmap v2520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:31 gibba011 bash[31774]: cluster 2021-03-10T15:16:30.076877+0000 mgr.gibba032.whopeo (mgr.24625) 4437 : cluster [DBG] pgmap v2520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:32 gibba032 bash[24004]: cluster 2021-03-10T15:16:32.078557+0000 mgr.gibba032.whopeo (mgr.24625) 4438 : cluster [DBG] pgmap v2521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:32 gibba011 bash[30791]: cluster 2021-03-10T15:16:32.078557+0000 mgr.gibba032.whopeo (mgr.24625) 4438 : cluster [DBG] pgmap v2521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:32 gibba011 bash[31774]: cluster 2021-03-10T15:16:32.078557+0000 mgr.gibba032.whopeo (mgr.24625) 4438 : cluster [DBG] pgmap v2521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:35 gibba032 bash[24004]: cluster 2021-03-10T15:16:34.080098+0000 mgr.gibba032.whopeo (mgr.24625) 4439 : cluster [DBG] pgmap v2522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:35.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:35 gibba011 bash[30791]: cluster 2021-03-10T15:16:34.080098+0000 mgr.gibba032.whopeo (mgr.24625) 4439 : cluster [DBG] pgmap v2522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:35 gibba011 bash[31774]: cluster 2021-03-10T15:16:34.080098+0000 mgr.gibba032.whopeo (mgr.24625) 4439 : cluster [DBG] pgmap v2522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:36 gibba032 bash[24004]: cluster 2021-03-10T15:16:36.081881+0000 mgr.gibba032.whopeo (mgr.24625) 4440 : cluster [DBG] pgmap v2523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:36 gibba011 bash[30791]: cluster 2021-03-10T15:16:36.081881+0000 mgr.gibba032.whopeo (mgr.24625) 4440 : cluster [DBG] pgmap v2523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:36 gibba011 bash[31774]: cluster 2021-03-10T15:16:36.081881+0000 mgr.gibba032.whopeo (mgr.24625) 4440 : cluster [DBG] pgmap v2523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:38 gibba032 bash[24004]: cluster 2021-03-10T15:16:38.082862+0000 mgr.gibba032.whopeo (mgr.24625) 4441 : cluster [DBG] pgmap v2524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:38 gibba011 bash[30791]: cluster 2021-03-10T15:16:38.082862+0000 mgr.gibba032.whopeo (mgr.24625) 4441 : cluster [DBG] pgmap v2524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:38 gibba011 bash[31774]: cluster 2021-03-10T15:16:38.082862+0000 mgr.gibba032.whopeo (mgr.24625) 4441 : cluster [DBG] pgmap v2524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.772273+0000 mgr.gibba032.whopeo (mgr.24625) 4442 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:16:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.773871+0000 mgr.gibba032.whopeo (mgr.24625) 4443 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:16:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.986629+0000 mon.a (mon.0) 1812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:16:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4444 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:16:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.989240+0000 mgr.gibba032.whopeo (mgr.24625) 4445 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:16:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.989909+0000 mon.a (mon.0) 1813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:16:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.990316+0000 mgr.gibba032.whopeo (mgr.24625) 4446 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:16:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.990354+0000 mgr.gibba032.whopeo (mgr.24625) 4447 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:16:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.991140+0000 mon.a (mon.0) 1814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.991503+0000 mgr.gibba032.whopeo (mgr.24625) 4448 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:16:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.991543+0000 mgr.gibba032.whopeo (mgr.24625) 4449 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:16:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.992340+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.992467+0000 mgr.gibba032.whopeo (mgr.24625) 4450 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:16:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.993328+0000 mgr.gibba032.whopeo (mgr.24625) 4451 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:16:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.993366+0000 mgr.gibba032.whopeo (mgr.24625) 4452 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:16:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.993370+0000 mon.a (mon.0) 1816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.994787+0000 mgr.gibba032.whopeo (mgr.24625) 4453 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:16:41.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.995098+0000 mgr.gibba032.whopeo (mgr.24625) 4454 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:16:41.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.995289+0000 mon.a (mon.0) 1817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.996166+0000 mgr.gibba032.whopeo (mgr.24625) 4455 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:16:41.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.996227+0000 mgr.gibba032.whopeo (mgr.24625) 4456 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:16:41.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cephadm 2021-03-10T15:16:39.996337+0000 mgr.gibba032.whopeo (mgr.24625) 4457 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:41.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: audit 2021-03-10T15:16:39.996998+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:40 gibba032 bash[24004]: cluster 2021-03-10T15:16:40.084093+0000 mgr.gibba032.whopeo (mgr.24625) 4458 : cluster [DBG] pgmap v2525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.772273+0000 mgr.gibba032.whopeo (mgr.24625) 4442 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:16:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.773871+0000 mgr.gibba032.whopeo (mgr.24625) 4443 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:16:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.986629+0000 mon.a (mon.0) 1812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:16:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4444 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:16:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.989240+0000 mgr.gibba032.whopeo (mgr.24625) 4445 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:16:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.989909+0000 mon.a (mon.0) 1813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:16:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.990316+0000 mgr.gibba032.whopeo (mgr.24625) 4446 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:16:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.990354+0000 mgr.gibba032.whopeo (mgr.24625) 4447 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:16:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.991140+0000 mon.a (mon.0) 1814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.991503+0000 mgr.gibba032.whopeo (mgr.24625) 4448 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:16:41.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.991543+0000 mgr.gibba032.whopeo (mgr.24625) 4449 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:16:41.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.992340+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.992467+0000 mgr.gibba032.whopeo (mgr.24625) 4450 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:16:41.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.993328+0000 mgr.gibba032.whopeo (mgr.24625) 4451 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:16:41.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.993366+0000 mgr.gibba032.whopeo (mgr.24625) 4452 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:16:41.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.993370+0000 mon.a (mon.0) 1816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.994787+0000 mgr.gibba032.whopeo (mgr.24625) 4453 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:16:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.995098+0000 mgr.gibba032.whopeo (mgr.24625) 4454 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:16:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.995289+0000 mon.a (mon.0) 1817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.996166+0000 mgr.gibba032.whopeo (mgr.24625) 4455 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:16:41.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.996227+0000 mgr.gibba032.whopeo (mgr.24625) 4456 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:16:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.772273+0000 mgr.gibba032.whopeo (mgr.24625) 4442 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:16:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.773871+0000 mgr.gibba032.whopeo (mgr.24625) 4443 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:16:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.986629+0000 mon.a (mon.0) 1812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:16:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.987413+0000 mgr.gibba032.whopeo (mgr.24625) 4444 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:16:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.989240+0000 mgr.gibba032.whopeo (mgr.24625) 4445 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:16:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.989909+0000 mon.a (mon.0) 1813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:16:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.990316+0000 mgr.gibba032.whopeo (mgr.24625) 4446 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:16:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.990354+0000 mgr.gibba032.whopeo (mgr.24625) 4447 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:16:41.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.991140+0000 mon.a (mon.0) 1814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.991503+0000 mgr.gibba032.whopeo (mgr.24625) 4448 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:16:41.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.991543+0000 mgr.gibba032.whopeo (mgr.24625) 4449 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:16:41.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.992340+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.992467+0000 mgr.gibba032.whopeo (mgr.24625) 4450 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:16:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.993328+0000 mgr.gibba032.whopeo (mgr.24625) 4451 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:16:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.993366+0000 mgr.gibba032.whopeo (mgr.24625) 4452 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:16:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.993370+0000 mon.a (mon.0) 1816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.994787+0000 mgr.gibba032.whopeo (mgr.24625) 4453 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:16:41.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.995098+0000 mgr.gibba032.whopeo (mgr.24625) 4454 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:16:41.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.995289+0000 mon.a (mon.0) 1817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.996166+0000 mgr.gibba032.whopeo (mgr.24625) 4455 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:16:41.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.996227+0000 mgr.gibba032.whopeo (mgr.24625) 4456 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:16:41.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cephadm 2021-03-10T15:16:39.996337+0000 mgr.gibba032.whopeo (mgr.24625) 4457 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:41.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: audit 2021-03-10T15:16:39.996998+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[30791]: cluster 2021-03-10T15:16:40.084093+0000 mgr.gibba032.whopeo (mgr.24625) 4458 : cluster [DBG] pgmap v2525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:41.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cephadm 2021-03-10T15:16:39.996337+0000 mgr.gibba032.whopeo (mgr.24625) 4457 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:41.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: audit 2021-03-10T15:16:39.996998+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:41.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:40 gibba011 bash[31774]: cluster 2021-03-10T15:16:40.084093+0000 mgr.gibba032.whopeo (mgr.24625) 4458 : cluster [DBG] pgmap v2525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:43 gibba032 bash[24004]: cluster 2021-03-10T15:16:42.085797+0000 mgr.gibba032.whopeo (mgr.24625) 4459 : cluster [DBG] pgmap v2526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:43 gibba011 bash[30791]: cluster 2021-03-10T15:16:42.085797+0000 mgr.gibba032.whopeo (mgr.24625) 4459 : cluster [DBG] pgmap v2526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:43.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:43 gibba011 bash[31774]: cluster 2021-03-10T15:16:42.085797+0000 mgr.gibba032.whopeo (mgr.24625) 4459 : cluster [DBG] pgmap v2526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:44 gibba032 bash[24004]: cluster 2021-03-10T15:16:44.087272+0000 mgr.gibba032.whopeo (mgr.24625) 4460 : cluster [DBG] pgmap v2527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:44 gibba011 bash[30791]: cluster 2021-03-10T15:16:44.087272+0000 mgr.gibba032.whopeo (mgr.24625) 4460 : cluster [DBG] pgmap v2527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:44 gibba011 bash[31774]: cluster 2021-03-10T15:16:44.087272+0000 mgr.gibba032.whopeo (mgr.24625) 4460 : cluster [DBG] pgmap v2527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:46 gibba032 bash[24004]: cluster 2021-03-10T15:16:46.088785+0000 mgr.gibba032.whopeo (mgr.24625) 4461 : cluster [DBG] pgmap v2528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:46 gibba011 bash[30791]: cluster 2021-03-10T15:16:46.088785+0000 mgr.gibba032.whopeo (mgr.24625) 4461 : cluster [DBG] pgmap v2528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:46 gibba011 bash[31774]: cluster 2021-03-10T15:16:46.088785+0000 mgr.gibba032.whopeo (mgr.24625) 4461 : cluster [DBG] pgmap v2528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:48 gibba032 bash[24004]: cluster 2021-03-10T15:16:48.090551+0000 mgr.gibba032.whopeo (mgr.24625) 4462 : cluster [DBG] pgmap v2529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:48 gibba011 bash[30791]: cluster 2021-03-10T15:16:48.090551+0000 mgr.gibba032.whopeo (mgr.24625) 4462 : cluster [DBG] pgmap v2529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:48 gibba011 bash[31774]: cluster 2021-03-10T15:16:48.090551+0000 mgr.gibba032.whopeo (mgr.24625) 4462 : cluster [DBG] pgmap v2529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:50 gibba032 bash[24004]: cluster 2021-03-10T15:16:50.091997+0000 mgr.gibba032.whopeo (mgr.24625) 4463 : cluster [DBG] pgmap v2530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:50 gibba011 bash[30791]: cluster 2021-03-10T15:16:50.091997+0000 mgr.gibba032.whopeo (mgr.24625) 4463 : cluster [DBG] pgmap v2530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:50 gibba011 bash[31774]: cluster 2021-03-10T15:16:50.091997+0000 mgr.gibba032.whopeo (mgr.24625) 4463 : cluster [DBG] pgmap v2530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:52.110 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:16:52.344 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:16:52.345 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (83m) 8m ago 88m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:16:52.345 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (88m) 7m ago 88m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:16:52.345 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (83m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:16:52.346 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (84m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:16:52.346 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (83m) 8m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:16:52.346 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (83m) 8m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:16:52.346 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (82m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:16:52.347 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (82m) 8m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:16:52.347 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (88m) 8m ago 89m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:16:52.347 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (89m) 7m ago 89m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:16:52.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (81m) 8m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:16:52.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (81m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:16:52.348 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (80m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:16:52.348 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (80m) 8m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:16:52.348 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (80m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:16:52.348 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (79m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:16:52.349 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (79m) 7m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:16:52.349 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (78m) 7m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:16:52.349 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (88m) 7m ago 89m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:16:52.349 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 86m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:16:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:52 gibba032 bash[24004]: cluster 2021-03-10T15:16:52.093588+0000 mgr.gibba032.whopeo (mgr.24625) 4464 : cluster [DBG] pgmap v2531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:52 gibba032 bash[24004]: audit 2021-03-10T15:16:52.096838+0000 mgr.gibba032.whopeo (mgr.24625) 4465 : audit [DBG] from='client.57028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:52 gibba011 bash[30791]: cluster 2021-03-10T15:16:52.093588+0000 mgr.gibba032.whopeo (mgr.24625) 4464 : cluster [DBG] pgmap v2531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:52 gibba011 bash[30791]: audit 2021-03-10T15:16:52.096838+0000 mgr.gibba032.whopeo (mgr.24625) 4465 : audit [DBG] from='client.57028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:52 gibba011 bash[31774]: cluster 2021-03-10T15:16:52.093588+0000 mgr.gibba032.whopeo (mgr.24625) 4464 : cluster [DBG] pgmap v2531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:52 gibba011 bash[31774]: audit 2021-03-10T15:16:52.096838+0000 mgr.gibba032.whopeo (mgr.24625) 4465 : audit [DBG] from='client.57028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:52.597 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:16:52.598 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:16:52.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:16:52.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:52.599 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:16:52.600 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:16:52.600 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:52.601 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:16:52.601 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:16:52.601 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:16:52.602 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:16:52.602 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:16:52.603 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:16:52.603 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:16:52.603 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:16:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:53 gibba032 bash[24004]: audit 2021-03-10T15:16:52.328115+0000 mgr.gibba032.whopeo (mgr.24625) 4466 : audit [DBG] from='client.47049 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:53 gibba032 bash[24004]: audit 2021-03-10T15:16:52.594738+0000 mon.a (mon.0) 1819 : audit [DBG] from='client.? 172.21.2.111:0/1359454285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:53.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:53 gibba011 bash[30791]: audit 2021-03-10T15:16:52.328115+0000 mgr.gibba032.whopeo (mgr.24625) 4466 : audit [DBG] from='client.47049 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:53.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:53 gibba011 bash[30791]: audit 2021-03-10T15:16:52.594738+0000 mon.a (mon.0) 1819 : audit [DBG] from='client.? 172.21.2.111:0/1359454285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:53.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:53 gibba011 bash[31774]: audit 2021-03-10T15:16:52.328115+0000 mgr.gibba032.whopeo (mgr.24625) 4466 : audit [DBG] from='client.47049 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:16:53.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:53 gibba011 bash[31774]: audit 2021-03-10T15:16:52.594738+0000 mon.a (mon.0) 1819 : audit [DBG] from='client.? 172.21.2.111:0/1359454285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:16:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:54 gibba032 bash[24004]: cluster 2021-03-10T15:16:54.094873+0000 mgr.gibba032.whopeo (mgr.24625) 4467 : cluster [DBG] pgmap v2532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:54 gibba011 bash[30791]: cluster 2021-03-10T15:16:54.094873+0000 mgr.gibba032.whopeo (mgr.24625) 4467 : cluster [DBG] pgmap v2532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:54 gibba011 bash[31774]: cluster 2021-03-10T15:16:54.094873+0000 mgr.gibba032.whopeo (mgr.24625) 4467 : cluster [DBG] pgmap v2532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:55.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:55 gibba032 bash[24004]: cephadm 2021-03-10T15:16:55.012338+0000 mgr.gibba032.whopeo (mgr.24625) 4468 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:55.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:55 gibba011 bash[30791]: cephadm 2021-03-10T15:16:55.012338+0000 mgr.gibba032.whopeo (mgr.24625) 4468 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:55.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:55 gibba011 bash[31774]: cephadm 2021-03-10T15:16:55.012338+0000 mgr.gibba032.whopeo (mgr.24625) 4468 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:16:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:56 gibba032 bash[24004]: cluster 2021-03-10T15:16:56.096566+0000 mgr.gibba032.whopeo (mgr.24625) 4469 : cluster [DBG] pgmap v2533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:56 gibba011 bash[30791]: cluster 2021-03-10T15:16:56.096566+0000 mgr.gibba032.whopeo (mgr.24625) 4469 : cluster [DBG] pgmap v2533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:56 gibba011 bash[31774]: cluster 2021-03-10T15:16:56.096566+0000 mgr.gibba032.whopeo (mgr.24625) 4469 : cluster [DBG] pgmap v2533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.538418+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:16:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.539074+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:16:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.549724+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:16:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.550485+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:16:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.551188+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:16:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:57 gibba032 bash[24004]: audit 2021-03-10T15:16:56.551937+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:16:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.538418+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:16:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.539074+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:16:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.549724+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:16:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.550485+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:16:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.551188+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:16:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[30791]: audit 2021-03-10T15:16:56.551937+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:16:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.538418+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:16:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.539074+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:16:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.549724+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:16:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.550485+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:16:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.551188+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:16:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:57 gibba011 bash[31774]: audit 2021-03-10T15:16:56.551937+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:16:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:16:58 gibba032 bash[24004]: cluster 2021-03-10T15:16:58.097439+0000 mgr.gibba032.whopeo (mgr.24625) 4470 : cluster [DBG] pgmap v2534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:16:58 gibba011 bash[30791]: cluster 2021-03-10T15:16:58.097439+0000 mgr.gibba032.whopeo (mgr.24625) 4470 : cluster [DBG] pgmap v2534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:16:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:16:58 gibba011 bash[31774]: cluster 2021-03-10T15:16:58.097439+0000 mgr.gibba032.whopeo (mgr.24625) 4470 : cluster [DBG] pgmap v2534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:00 gibba032 bash[24004]: cluster 2021-03-10T15:17:00.098089+0000 mgr.gibba032.whopeo (mgr.24625) 4471 : cluster [DBG] pgmap v2535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:00 gibba011 bash[30791]: cluster 2021-03-10T15:17:00.098089+0000 mgr.gibba032.whopeo (mgr.24625) 4471 : cluster [DBG] pgmap v2535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:00 gibba011 bash[31774]: cluster 2021-03-10T15:17:00.098089+0000 mgr.gibba032.whopeo (mgr.24625) 4471 : cluster [DBG] pgmap v2535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:02 gibba032 bash[24004]: cluster 2021-03-10T15:17:02.099393+0000 mgr.gibba032.whopeo (mgr.24625) 4472 : cluster [DBG] pgmap v2536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:02 gibba011 bash[30791]: cluster 2021-03-10T15:17:02.099393+0000 mgr.gibba032.whopeo (mgr.24625) 4472 : cluster [DBG] pgmap v2536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:02 gibba011 bash[31774]: cluster 2021-03-10T15:17:02.099393+0000 mgr.gibba032.whopeo (mgr.24625) 4472 : cluster [DBG] pgmap v2536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:04 gibba032 bash[24004]: cluster 2021-03-10T15:17:04.100695+0000 mgr.gibba032.whopeo (mgr.24625) 4473 : cluster [DBG] pgmap v2537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:04 gibba011 bash[30791]: cluster 2021-03-10T15:17:04.100695+0000 mgr.gibba032.whopeo (mgr.24625) 4473 : cluster [DBG] pgmap v2537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:04 gibba011 bash[31774]: cluster 2021-03-10T15:17:04.100695+0000 mgr.gibba032.whopeo (mgr.24625) 4473 : cluster [DBG] pgmap v2537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:06 gibba032 bash[24004]: cluster 2021-03-10T15:17:06.102208+0000 mgr.gibba032.whopeo (mgr.24625) 4474 : cluster [DBG] pgmap v2538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:06 gibba011 bash[30791]: cluster 2021-03-10T15:17:06.102208+0000 mgr.gibba032.whopeo (mgr.24625) 4474 : cluster [DBG] pgmap v2538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:06 gibba011 bash[31774]: cluster 2021-03-10T15:17:06.102208+0000 mgr.gibba032.whopeo (mgr.24625) 4474 : cluster [DBG] pgmap v2538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:08 gibba032 bash[24004]: cluster 2021-03-10T15:17:08.103003+0000 mgr.gibba032.whopeo (mgr.24625) 4475 : cluster [DBG] pgmap v2539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:08 gibba011 bash[30791]: cluster 2021-03-10T15:17:08.103003+0000 mgr.gibba032.whopeo (mgr.24625) 4475 : cluster [DBG] pgmap v2539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:08 gibba011 bash[31774]: cluster 2021-03-10T15:17:08.103003+0000 mgr.gibba032.whopeo (mgr.24625) 4475 : cluster [DBG] pgmap v2539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:10 gibba032 bash[24004]: cephadm 2021-03-10T15:17:10.027942+0000 mgr.gibba032.whopeo (mgr.24625) 4476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:10 gibba032 bash[24004]: cluster 2021-03-10T15:17:10.104432+0000 mgr.gibba032.whopeo (mgr.24625) 4477 : cluster [DBG] pgmap v2540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:10 gibba011 bash[30791]: cephadm 2021-03-10T15:17:10.027942+0000 mgr.gibba032.whopeo (mgr.24625) 4476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:10 gibba011 bash[30791]: cluster 2021-03-10T15:17:10.104432+0000 mgr.gibba032.whopeo (mgr.24625) 4477 : cluster [DBG] pgmap v2540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:10 gibba011 bash[31774]: cephadm 2021-03-10T15:17:10.027942+0000 mgr.gibba032.whopeo (mgr.24625) 4476 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:10 gibba011 bash[31774]: cluster 2021-03-10T15:17:10.104432+0000 mgr.gibba032.whopeo (mgr.24625) 4477 : cluster [DBG] pgmap v2540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:12 gibba032 bash[24004]: cluster 2021-03-10T15:17:12.105995+0000 mgr.gibba032.whopeo (mgr.24625) 4478 : cluster [DBG] pgmap v2541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:12 gibba011 bash[30791]: cluster 2021-03-10T15:17:12.105995+0000 mgr.gibba032.whopeo (mgr.24625) 4478 : cluster [DBG] pgmap v2541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:12 gibba011 bash[31774]: cluster 2021-03-10T15:17:12.105995+0000 mgr.gibba032.whopeo (mgr.24625) 4478 : cluster [DBG] pgmap v2541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:14 gibba032 bash[24004]: cluster 2021-03-10T15:17:14.107340+0000 mgr.gibba032.whopeo (mgr.24625) 4479 : cluster [DBG] pgmap v2542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:14 gibba011 bash[30791]: cluster 2021-03-10T15:17:14.107340+0000 mgr.gibba032.whopeo (mgr.24625) 4479 : cluster [DBG] pgmap v2542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:14 gibba011 bash[31774]: cluster 2021-03-10T15:17:14.107340+0000 mgr.gibba032.whopeo (mgr.24625) 4479 : cluster [DBG] pgmap v2542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:16 gibba032 bash[24004]: cluster 2021-03-10T15:17:16.108896+0000 mgr.gibba032.whopeo (mgr.24625) 4480 : cluster [DBG] pgmap v2543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:16 gibba011 bash[30791]: cluster 2021-03-10T15:17:16.108896+0000 mgr.gibba032.whopeo (mgr.24625) 4480 : cluster [DBG] pgmap v2543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:16 gibba011 bash[31774]: cluster 2021-03-10T15:17:16.108896+0000 mgr.gibba032.whopeo (mgr.24625) 4480 : cluster [DBG] pgmap v2543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:18 gibba032 bash[24004]: cluster 2021-03-10T15:17:18.109823+0000 mgr.gibba032.whopeo (mgr.24625) 4481 : cluster [DBG] pgmap v2544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:18 gibba011 bash[30791]: cluster 2021-03-10T15:17:18.109823+0000 mgr.gibba032.whopeo (mgr.24625) 4481 : cluster [DBG] pgmap v2544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:18 gibba011 bash[31774]: cluster 2021-03-10T15:17:18.109823+0000 mgr.gibba032.whopeo (mgr.24625) 4481 : cluster [DBG] pgmap v2544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:20 gibba032 bash[24004]: cluster 2021-03-10T15:17:20.111319+0000 mgr.gibba032.whopeo (mgr.24625) 4482 : cluster [DBG] pgmap v2545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:20 gibba011 bash[30791]: cluster 2021-03-10T15:17:20.111319+0000 mgr.gibba032.whopeo (mgr.24625) 4482 : cluster [DBG] pgmap v2545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:20 gibba011 bash[31774]: cluster 2021-03-10T15:17:20.111319+0000 mgr.gibba032.whopeo (mgr.24625) 4482 : cluster [DBG] pgmap v2545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:22 gibba032 bash[24004]: cluster 2021-03-10T15:17:22.112888+0000 mgr.gibba032.whopeo (mgr.24625) 4483 : cluster [DBG] pgmap v2546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:22 gibba011 bash[30791]: cluster 2021-03-10T15:17:22.112888+0000 mgr.gibba032.whopeo (mgr.24625) 4483 : cluster [DBG] pgmap v2546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:22 gibba011 bash[31774]: cluster 2021-03-10T15:17:22.112888+0000 mgr.gibba032.whopeo (mgr.24625) 4483 : cluster [DBG] pgmap v2546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:22.842 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:17:23.071 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:17:23.071 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (84m) 8m ago 89m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:17:23.071 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (89m) 7m ago 89m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:17:23.072 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (84m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:17:23.072 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (84m) 7m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:17:23.072 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (83m) 8m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:17:23.072 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (83m) 8m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:17:23.073 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (83m) 7m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:17:23.073 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (83m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:17:23.073 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (89m) 8m ago 89m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:17:23.073 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (89m) 7m ago 89m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:17:23.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (82m) 8m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:17:23.074 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (81m) 8m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:17:23.074 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (81m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:17:23.074 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (81m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:17:23.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (80m) 7m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:17:23.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (80m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:17:23.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (79m) 7m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:17:23.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (79m) 7m ago 89m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:17:23.075 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (88m) 7m ago 89m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:17:23.076 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 87m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:17:23.325 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:17:23.325 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:17:23.325 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:17:23.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:23.326 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:17:23.326 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:17:23.326 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:23.326 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:17:23.327 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:17:23.327 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:23.327 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:17:23.327 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:17:23.327 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:17:23.328 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:17:23.328 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:17:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:23 gibba032 bash[24004]: audit 2021-03-10T15:17:22.829064+0000 mgr.gibba032.whopeo (mgr.24625) 4484 : audit [DBG] from='client.57046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:23 gibba032 bash[24004]: audit 2021-03-10T15:17:23.054836+0000 mgr.gibba032.whopeo (mgr.24625) 4485 : audit [DBG] from='client.57052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:23 gibba032 bash[24004]: audit 2021-03-10T15:17:23.322138+0000 mon.a (mon.0) 1826 : audit [DBG] from='client.? 172.21.2.111:0/4091247165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[30791]: audit 2021-03-10T15:17:22.829064+0000 mgr.gibba032.whopeo (mgr.24625) 4484 : audit [DBG] from='client.57046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[30791]: audit 2021-03-10T15:17:23.054836+0000 mgr.gibba032.whopeo (mgr.24625) 4485 : audit [DBG] from='client.57052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[30791]: audit 2021-03-10T15:17:23.322138+0000 mon.a (mon.0) 1826 : audit [DBG] from='client.? 172.21.2.111:0/4091247165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:23.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[31774]: audit 2021-03-10T15:17:22.829064+0000 mgr.gibba032.whopeo (mgr.24625) 4484 : audit [DBG] from='client.57046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[31774]: audit 2021-03-10T15:17:23.054836+0000 mgr.gibba032.whopeo (mgr.24625) 4485 : audit [DBG] from='client.57052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:23.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:23 gibba011 bash[31774]: audit 2021-03-10T15:17:23.322138+0000 mon.a (mon.0) 1826 : audit [DBG] from='client.? 172.21.2.111:0/4091247165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:24 gibba032 bash[24004]: cluster 2021-03-10T15:17:24.114379+0000 mgr.gibba032.whopeo (mgr.24625) 4486 : cluster [DBG] pgmap v2547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:24.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:24 gibba011 bash[30791]: cluster 2021-03-10T15:17:24.114379+0000 mgr.gibba032.whopeo (mgr.24625) 4486 : cluster [DBG] pgmap v2547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:24 gibba011 bash[31774]: cluster 2021-03-10T15:17:24.114379+0000 mgr.gibba032.whopeo (mgr.24625) 4486 : cluster [DBG] pgmap v2547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:25.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:25 gibba032 bash[24004]: cephadm 2021-03-10T15:17:25.043426+0000 mgr.gibba032.whopeo (mgr.24625) 4487 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:25 gibba011 bash[30791]: cephadm 2021-03-10T15:17:25.043426+0000 mgr.gibba032.whopeo (mgr.24625) 4487 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:25 gibba011 bash[31774]: cephadm 2021-03-10T15:17:25.043426+0000 mgr.gibba032.whopeo (mgr.24625) 4487 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:26 gibba032 bash[24004]: cluster 2021-03-10T15:17:26.115971+0000 mgr.gibba032.whopeo (mgr.24625) 4488 : cluster [DBG] pgmap v2548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:26 gibba011 bash[30791]: cluster 2021-03-10T15:17:26.115971+0000 mgr.gibba032.whopeo (mgr.24625) 4488 : cluster [DBG] pgmap v2548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:26 gibba011 bash[31774]: cluster 2021-03-10T15:17:26.115971+0000 mgr.gibba032.whopeo (mgr.24625) 4488 : cluster [DBG] pgmap v2548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:28 gibba032 bash[24004]: cluster 2021-03-10T15:17:28.116955+0000 mgr.gibba032.whopeo (mgr.24625) 4489 : cluster [DBG] pgmap v2549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:28.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:28 gibba011 bash[30791]: cluster 2021-03-10T15:17:28.116955+0000 mgr.gibba032.whopeo (mgr.24625) 4489 : cluster [DBG] pgmap v2549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:28.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:28 gibba011 bash[31774]: cluster 2021-03-10T15:17:28.116955+0000 mgr.gibba032.whopeo (mgr.24625) 4489 : cluster [DBG] pgmap v2549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:30 gibba032 bash[24004]: cluster 2021-03-10T15:17:30.118484+0000 mgr.gibba032.whopeo (mgr.24625) 4490 : cluster [DBG] pgmap v2550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:30 gibba011 bash[30791]: cluster 2021-03-10T15:17:30.118484+0000 mgr.gibba032.whopeo (mgr.24625) 4490 : cluster [DBG] pgmap v2550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:30 gibba011 bash[31774]: cluster 2021-03-10T15:17:30.118484+0000 mgr.gibba032.whopeo (mgr.24625) 4490 : cluster [DBG] pgmap v2550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:32 gibba032 bash[24004]: cluster 2021-03-10T15:17:32.120238+0000 mgr.gibba032.whopeo (mgr.24625) 4491 : cluster [DBG] pgmap v2551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:32 gibba011 bash[30791]: cluster 2021-03-10T15:17:32.120238+0000 mgr.gibba032.whopeo (mgr.24625) 4491 : cluster [DBG] pgmap v2551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:32 gibba011 bash[31774]: cluster 2021-03-10T15:17:32.120238+0000 mgr.gibba032.whopeo (mgr.24625) 4491 : cluster [DBG] pgmap v2551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:34 gibba032 bash[24004]: cluster 2021-03-10T15:17:34.121767+0000 mgr.gibba032.whopeo (mgr.24625) 4492 : cluster [DBG] pgmap v2552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:34 gibba011 bash[30791]: cluster 2021-03-10T15:17:34.121767+0000 mgr.gibba032.whopeo (mgr.24625) 4492 : cluster [DBG] pgmap v2552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:34 gibba011 bash[31774]: cluster 2021-03-10T15:17:34.121767+0000 mgr.gibba032.whopeo (mgr.24625) 4492 : cluster [DBG] pgmap v2552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:36 gibba032 bash[24004]: cluster 2021-03-10T15:17:36.123369+0000 mgr.gibba032.whopeo (mgr.24625) 4493 : cluster [DBG] pgmap v2553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:36 gibba011 bash[30791]: cluster 2021-03-10T15:17:36.123369+0000 mgr.gibba032.whopeo (mgr.24625) 4493 : cluster [DBG] pgmap v2553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:36 gibba011 bash[31774]: cluster 2021-03-10T15:17:36.123369+0000 mgr.gibba032.whopeo (mgr.24625) 4493 : cluster [DBG] pgmap v2553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:38 gibba032 bash[24004]: cluster 2021-03-10T15:17:38.125025+0000 mgr.gibba032.whopeo (mgr.24625) 4494 : cluster [DBG] pgmap v2554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:38 gibba011 bash[30791]: cluster 2021-03-10T15:17:38.125025+0000 mgr.gibba032.whopeo (mgr.24625) 4494 : cluster [DBG] pgmap v2554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:38 gibba011 bash[31774]: cluster 2021-03-10T15:17:38.125025+0000 mgr.gibba032.whopeo (mgr.24625) 4494 : cluster [DBG] pgmap v2554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:40.370 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:40 gibba011 bash[30791]: audit 2021-03-10T15:17:40.061879+0000 mon.a (mon.0) 1827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:17:40.371 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:40 gibba011 bash[31774]: audit 2021-03-10T15:17:40.061879+0000 mon.a (mon.0) 1827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:17:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:40 gibba032 bash[24004]: audit 2021-03-10T15:17:40.061879+0000 mon.a (mon.0) 1827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:17:41.393 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:41 gibba011 bash[30791]: cephadm 2021-03-10T15:17:40.059789+0000 mgr.gibba032.whopeo (mgr.24625) 4495 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:17:41.394 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:41 gibba011 bash[30791]: cluster 2021-03-10T15:17:40.125610+0000 mgr.gibba032.whopeo (mgr.24625) 4496 : cluster [DBG] pgmap v2555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:41.394 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:41 gibba011 bash[31774]: cephadm 2021-03-10T15:17:40.059789+0000 mgr.gibba032.whopeo (mgr.24625) 4495 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:17:41.394 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:41 gibba011 bash[31774]: cluster 2021-03-10T15:17:40.125610+0000 mgr.gibba032.whopeo (mgr.24625) 4496 : cluster [DBG] pgmap v2555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:41 gibba032 bash[24004]: cephadm 2021-03-10T15:17:40.059789+0000 mgr.gibba032.whopeo (mgr.24625) 4495 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:17:41.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:41 gibba032 bash[24004]: cluster 2021-03-10T15:17:40.125610+0000 mgr.gibba032.whopeo (mgr.24625) 4496 : cluster [DBG] pgmap v2555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:42 gibba011 bash[30791]: cluster 2021-03-10T15:17:42.127120+0000 mgr.gibba032.whopeo (mgr.24625) 4497 : cluster [DBG] pgmap v2556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:42 gibba011 bash[31774]: cluster 2021-03-10T15:17:42.127120+0000 mgr.gibba032.whopeo (mgr.24625) 4497 : cluster [DBG] pgmap v2556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:42 gibba032 bash[24004]: cluster 2021-03-10T15:17:42.127120+0000 mgr.gibba032.whopeo (mgr.24625) 4497 : cluster [DBG] pgmap v2556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:44 gibba011 bash[30791]: cluster 2021-03-10T15:17:44.128598+0000 mgr.gibba032.whopeo (mgr.24625) 4498 : cluster [DBG] pgmap v2557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:44 gibba011 bash[31774]: cluster 2021-03-10T15:17:44.128598+0000 mgr.gibba032.whopeo (mgr.24625) 4498 : cluster [DBG] pgmap v2557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:44 gibba032 bash[24004]: cluster 2021-03-10T15:17:44.128598+0000 mgr.gibba032.whopeo (mgr.24625) 4498 : cluster [DBG] pgmap v2557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:45.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:44.659108+0000 mon.a (mon.0) 1828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:17:45.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:44.837688+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:44.837837+0000 mgr.gibba032.whopeo (mgr.24625) 4499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:17:45.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.046689+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.052781+0000 mgr.gibba032.whopeo (mgr.24625) 4500 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:17:45.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.054896+0000 mgr.gibba032.whopeo (mgr.24625) 4501 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:17:45.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.055101+0000 mon.a (mon.0) 1831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:17:45.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.057006+0000 mon.a (mon.0) 1832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.057296+0000 mgr.gibba032.whopeo (mgr.24625) 4502 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:17:45.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.057336+0000 mgr.gibba032.whopeo (mgr.24625) 4503 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:17:45.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.058783+0000 mgr.gibba032.whopeo (mgr.24625) 4504 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:17:45.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.058821+0000 mgr.gibba032.whopeo (mgr.24625) 4505 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:17:45.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.059221+0000 mon.a (mon.0) 1833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.060843+0000 mon.a (mon.0) 1834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.060945+0000 mgr.gibba032.whopeo (mgr.24625) 4506 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:17:45.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.062345+0000 mgr.gibba032.whopeo (mgr.24625) 4507 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:17:45.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.062384+0000 mgr.gibba032.whopeo (mgr.24625) 4508 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:17:45.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.063972+0000 mgr.gibba032.whopeo (mgr.24625) 4509 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:17:45.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.064424+0000 mgr.gibba032.whopeo (mgr.24625) 4510 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:17:45.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.064489+0000 mon.a (mon.0) 1835 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.065708+0000 mgr.gibba032.whopeo (mgr.24625) 4511 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:17:45.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.065752+0000 mgr.gibba032.whopeo (mgr.24625) 4512 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:17:45.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: cephadm 2021-03-10T15:17:45.065874+0000 mgr.gibba032.whopeo (mgr.24625) 4513 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:45.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:45 gibba032 bash[24004]: audit 2021-03-10T15:17:45.066181+0000 mon.a (mon.0) 1836 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:44.659108+0000 mon.a (mon.0) 1828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:17:45.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:44.837688+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:44.837837+0000 mgr.gibba032.whopeo (mgr.24625) 4499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:17:45.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.046689+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.052781+0000 mgr.gibba032.whopeo (mgr.24625) 4500 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:17:45.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.054896+0000 mgr.gibba032.whopeo (mgr.24625) 4501 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:17:45.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.055101+0000 mon.a (mon.0) 1831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:17:45.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.057006+0000 mon.a (mon.0) 1832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.057296+0000 mgr.gibba032.whopeo (mgr.24625) 4502 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:17:45.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.057336+0000 mgr.gibba032.whopeo (mgr.24625) 4503 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:17:45.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.058783+0000 mgr.gibba032.whopeo (mgr.24625) 4504 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:17:45.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.058821+0000 mgr.gibba032.whopeo (mgr.24625) 4505 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:17:45.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.059221+0000 mon.a (mon.0) 1833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.060843+0000 mon.a (mon.0) 1834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.060945+0000 mgr.gibba032.whopeo (mgr.24625) 4506 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:17:45.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.062345+0000 mgr.gibba032.whopeo (mgr.24625) 4507 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:17:45.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.062384+0000 mgr.gibba032.whopeo (mgr.24625) 4508 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:17:45.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.063972+0000 mgr.gibba032.whopeo (mgr.24625) 4509 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:17:45.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.064424+0000 mgr.gibba032.whopeo (mgr.24625) 4510 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:17:45.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.064489+0000 mon.a (mon.0) 1835 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:44.659108+0000 mon.a (mon.0) 1828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:17:45.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:44.837688+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:44.837837+0000 mgr.gibba032.whopeo (mgr.24625) 4499 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:17:45.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.046689+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:17:45.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.052781+0000 mgr.gibba032.whopeo (mgr.24625) 4500 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:17:45.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.054896+0000 mgr.gibba032.whopeo (mgr.24625) 4501 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:17:45.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.055101+0000 mon.a (mon.0) 1831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:17:45.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.057006+0000 mon.a (mon.0) 1832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.057296+0000 mgr.gibba032.whopeo (mgr.24625) 4502 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:17:45.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.057336+0000 mgr.gibba032.whopeo (mgr.24625) 4503 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:17:45.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.058783+0000 mgr.gibba032.whopeo (mgr.24625) 4504 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:17:45.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.058821+0000 mgr.gibba032.whopeo (mgr.24625) 4505 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:17:45.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.059221+0000 mon.a (mon.0) 1833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.060843+0000 mon.a (mon.0) 1834 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.060945+0000 mgr.gibba032.whopeo (mgr.24625) 4506 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:17:45.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.062345+0000 mgr.gibba032.whopeo (mgr.24625) 4507 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:17:45.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.062384+0000 mgr.gibba032.whopeo (mgr.24625) 4508 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:17:45.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.063972+0000 mgr.gibba032.whopeo (mgr.24625) 4509 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:17:45.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.064424+0000 mgr.gibba032.whopeo (mgr.24625) 4510 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:17:45.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.064489+0000 mon.a (mon.0) 1835 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.065708+0000 mgr.gibba032.whopeo (mgr.24625) 4511 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:17:45.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.065752+0000 mgr.gibba032.whopeo (mgr.24625) 4512 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:17:45.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: cephadm 2021-03-10T15:17:45.065874+0000 mgr.gibba032.whopeo (mgr.24625) 4513 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:45.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[30791]: audit 2021-03-10T15:17:45.066181+0000 mon.a (mon.0) 1836 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:45.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.065708+0000 mgr.gibba032.whopeo (mgr.24625) 4511 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:17:45.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.065752+0000 mgr.gibba032.whopeo (mgr.24625) 4512 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:17:45.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: cephadm 2021-03-10T15:17:45.065874+0000 mgr.gibba032.whopeo (mgr.24625) 4513 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:17:45.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:45 gibba011 bash[31774]: audit 2021-03-10T15:17:45.066181+0000 mon.a (mon.0) 1836 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:46 gibba032 bash[24004]: cluster 2021-03-10T15:17:46.130088+0000 mgr.gibba032.whopeo (mgr.24625) 4514 : cluster [DBG] pgmap v2558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:46 gibba011 bash[30791]: cluster 2021-03-10T15:17:46.130088+0000 mgr.gibba032.whopeo (mgr.24625) 4514 : cluster [DBG] pgmap v2558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:46 gibba011 bash[31774]: cluster 2021-03-10T15:17:46.130088+0000 mgr.gibba032.whopeo (mgr.24625) 4514 : cluster [DBG] pgmap v2558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:48 gibba032 bash[24004]: cluster 2021-03-10T15:17:48.131844+0000 mgr.gibba032.whopeo (mgr.24625) 4515 : cluster [DBG] pgmap v2559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:48 gibba011 bash[30791]: cluster 2021-03-10T15:17:48.131844+0000 mgr.gibba032.whopeo (mgr.24625) 4515 : cluster [DBG] pgmap v2559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:48 gibba011 bash[31774]: cluster 2021-03-10T15:17:48.131844+0000 mgr.gibba032.whopeo (mgr.24625) 4515 : cluster [DBG] pgmap v2559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:50 gibba032 bash[24004]: cluster 2021-03-10T15:17:50.133403+0000 mgr.gibba032.whopeo (mgr.24625) 4516 : cluster [DBG] pgmap v2560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:50 gibba011 bash[30791]: cluster 2021-03-10T15:17:50.133403+0000 mgr.gibba032.whopeo (mgr.24625) 4516 : cluster [DBG] pgmap v2560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:50 gibba011 bash[31774]: cluster 2021-03-10T15:17:50.133403+0000 mgr.gibba032.whopeo (mgr.24625) 4516 : cluster [DBG] pgmap v2560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:52 gibba032 bash[24004]: cluster 2021-03-10T15:17:52.134974+0000 mgr.gibba032.whopeo (mgr.24625) 4517 : cluster [DBG] pgmap v2561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:52 gibba011 bash[30791]: cluster 2021-03-10T15:17:52.134974+0000 mgr.gibba032.whopeo (mgr.24625) 4517 : cluster [DBG] pgmap v2561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:52 gibba011 bash[31774]: cluster 2021-03-10T15:17:52.134974+0000 mgr.gibba032.whopeo (mgr.24625) 4517 : cluster [DBG] pgmap v2561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:53.576 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:17:53.803 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:17:53.803 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (84m) 9m ago 89m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:17:53.804 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (89m) 8m ago 89m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:17:53.804 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (84m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:17:53.804 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (85m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:17:53.804 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (84m) 9m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:17:53.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (84m) 9m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:17:53.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (83m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:17:53.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (83m) 9m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:17:53.805 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (90m) 9m ago 90m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:17:53.806 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (90m) 8m ago 90m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:17:53.806 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (82m) 9m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:17:53.806 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (82m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:17:53.806 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (81m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:17:53.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (81m) 9m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:17:53.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (81m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:17:53.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (80m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:17:53.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (80m) 8m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:17:53.808 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (79m) 8m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:17:53.808 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (89m) 8m ago 90m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:17:53.808 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 87m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:17:54.054 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:17:54.055 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:17:54.055 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:17:54.055 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:54.055 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:17:54.056 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:17:54.056 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:54.056 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:17:54.056 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:17:54.057 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:17:54.057 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:17:54.057 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:17:54.058 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:17:54.058 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:17:54.058 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:17:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:54 gibba032 bash[24004]: audit 2021-03-10T15:17:54.051552+0000 mon.c (mon.1) 86 : audit [DBG] from='client.? 172.21.2.111:0/1341220135' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:54 gibba011 bash[30791]: audit 2021-03-10T15:17:54.051552+0000 mon.c (mon.1) 86 : audit [DBG] from='client.? 172.21.2.111:0/1341220135' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:54 gibba011 bash[31774]: audit 2021-03-10T15:17:54.051552+0000 mon.c (mon.1) 86 : audit [DBG] from='client.? 172.21.2.111:0/1341220135' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:17:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:55 gibba032 bash[24004]: audit 2021-03-10T15:17:53.562749+0000 mgr.gibba032.whopeo (mgr.24625) 4518 : audit [DBG] from='client.47079 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:55 gibba032 bash[24004]: audit 2021-03-10T15:17:53.786509+0000 mgr.gibba032.whopeo (mgr.24625) 4519 : audit [DBG] from='client.57070 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:55 gibba032 bash[24004]: cluster 2021-03-10T15:17:54.135743+0000 mgr.gibba032.whopeo (mgr.24625) 4520 : cluster [DBG] pgmap v2562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[30791]: audit 2021-03-10T15:17:53.562749+0000 mgr.gibba032.whopeo (mgr.24625) 4518 : audit [DBG] from='client.47079 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[30791]: audit 2021-03-10T15:17:53.786509+0000 mgr.gibba032.whopeo (mgr.24625) 4519 : audit [DBG] from='client.57070 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[30791]: cluster 2021-03-10T15:17:54.135743+0000 mgr.gibba032.whopeo (mgr.24625) 4520 : cluster [DBG] pgmap v2562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[31774]: audit 2021-03-10T15:17:53.562749+0000 mgr.gibba032.whopeo (mgr.24625) 4518 : audit [DBG] from='client.47079 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[31774]: audit 2021-03-10T15:17:53.786509+0000 mgr.gibba032.whopeo (mgr.24625) 4519 : audit [DBG] from='client.57070 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:17:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:55 gibba011 bash[31774]: cluster 2021-03-10T15:17:54.135743+0000 mgr.gibba032.whopeo (mgr.24625) 4520 : cluster [DBG] pgmap v2562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:56 gibba032 bash[24004]: cluster 2021-03-10T15:17:56.137179+0000 mgr.gibba032.whopeo (mgr.24625) 4521 : cluster [DBG] pgmap v2563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:56 gibba011 bash[30791]: cluster 2021-03-10T15:17:56.137179+0000 mgr.gibba032.whopeo (mgr.24625) 4521 : cluster [DBG] pgmap v2563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:56 gibba011 bash[31774]: cluster 2021-03-10T15:17:56.137179+0000 mgr.gibba032.whopeo (mgr.24625) 4521 : cluster [DBG] pgmap v2563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.545773+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:17:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.546537+0000 mon.a (mon.0) 1838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:17:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.554997+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:17:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.555814+0000 mon.a (mon.0) 1840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:17:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.556692+0000 mon.a (mon.0) 1841 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:17:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:57 gibba032 bash[24004]: audit 2021-03-10T15:17:56.557488+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:17:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.545773+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:17:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.546537+0000 mon.a (mon.0) 1838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:17:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.554997+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:17:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.555814+0000 mon.a (mon.0) 1840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:17:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.556692+0000 mon.a (mon.0) 1841 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:17:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[31774]: audit 2021-03-10T15:17:56.557488+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:17:57.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.545773+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:17:57.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.546537+0000 mon.a (mon.0) 1838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:17:57.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.554997+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:17:57.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.555814+0000 mon.a (mon.0) 1840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:17:57.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.556692+0000 mon.a (mon.0) 1841 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T15:17:57.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:57 gibba011 bash[30791]: audit 2021-03-10T15:17:56.557488+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:17:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:17:58 gibba032 bash[24004]: cluster 2021-03-10T15:17:58.139014+0000 mgr.gibba032.whopeo (mgr.24625) 4522 : cluster [DBG] pgmap v2564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:17:58 gibba011 bash[30791]: cluster 2021-03-10T15:17:58.139014+0000 mgr.gibba032.whopeo (mgr.24625) 4522 : cluster [DBG] pgmap v2564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:17:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:17:58 gibba011 bash[31774]: cluster 2021-03-10T15:17:58.139014+0000 mgr.gibba032.whopeo (mgr.24625) 4522 : cluster [DBG] pgmap v2564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:00 gibba032 bash[24004]: cephadm 2021-03-10T15:18:00.070560+0000 mgr.gibba032.whopeo (mgr.24625) 4523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:00 gibba032 bash[24004]: cluster 2021-03-10T15:18:00.139860+0000 mgr.gibba032.whopeo (mgr.24625) 4524 : cluster [DBG] pgmap v2565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:00 gibba011 bash[30791]: cephadm 2021-03-10T15:18:00.070560+0000 mgr.gibba032.whopeo (mgr.24625) 4523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:00 gibba011 bash[30791]: cluster 2021-03-10T15:18:00.139860+0000 mgr.gibba032.whopeo (mgr.24625) 4524 : cluster [DBG] pgmap v2565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:00 gibba011 bash[31774]: cephadm 2021-03-10T15:18:00.070560+0000 mgr.gibba032.whopeo (mgr.24625) 4523 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:00 gibba011 bash[31774]: cluster 2021-03-10T15:18:00.139860+0000 mgr.gibba032.whopeo (mgr.24625) 4524 : cluster [DBG] pgmap v2565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:02 gibba032 bash[24004]: cluster 2021-03-10T15:18:02.141199+0000 mgr.gibba032.whopeo (mgr.24625) 4525 : cluster [DBG] pgmap v2566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:02 gibba011 bash[30791]: cluster 2021-03-10T15:18:02.141199+0000 mgr.gibba032.whopeo (mgr.24625) 4525 : cluster [DBG] pgmap v2566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:02 gibba011 bash[31774]: cluster 2021-03-10T15:18:02.141199+0000 mgr.gibba032.whopeo (mgr.24625) 4525 : cluster [DBG] pgmap v2566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:04 gibba032 bash[24004]: cluster 2021-03-10T15:18:04.142654+0000 mgr.gibba032.whopeo (mgr.24625) 4526 : cluster [DBG] pgmap v2567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:04 gibba011 bash[30791]: cluster 2021-03-10T15:18:04.142654+0000 mgr.gibba032.whopeo (mgr.24625) 4526 : cluster [DBG] pgmap v2567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:04 gibba011 bash[31774]: cluster 2021-03-10T15:18:04.142654+0000 mgr.gibba032.whopeo (mgr.24625) 4526 : cluster [DBG] pgmap v2567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:06 gibba032 bash[24004]: cluster 2021-03-10T15:18:06.144215+0000 mgr.gibba032.whopeo (mgr.24625) 4527 : cluster [DBG] pgmap v2568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:06 gibba011 bash[30791]: cluster 2021-03-10T15:18:06.144215+0000 mgr.gibba032.whopeo (mgr.24625) 4527 : cluster [DBG] pgmap v2568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:06 gibba011 bash[31774]: cluster 2021-03-10T15:18:06.144215+0000 mgr.gibba032.whopeo (mgr.24625) 4527 : cluster [DBG] pgmap v2568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:08 gibba032 bash[24004]: cluster 2021-03-10T15:18:08.145918+0000 mgr.gibba032.whopeo (mgr.24625) 4528 : cluster [DBG] pgmap v2569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:08 gibba011 bash[30791]: cluster 2021-03-10T15:18:08.145918+0000 mgr.gibba032.whopeo (mgr.24625) 4528 : cluster [DBG] pgmap v2569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:08 gibba011 bash[31774]: cluster 2021-03-10T15:18:08.145918+0000 mgr.gibba032.whopeo (mgr.24625) 4528 : cluster [DBG] pgmap v2569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:10 gibba032 bash[24004]: cluster 2021-03-10T15:18:10.147419+0000 mgr.gibba032.whopeo (mgr.24625) 4529 : cluster [DBG] pgmap v2570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:10 gibba011 bash[30791]: cluster 2021-03-10T15:18:10.147419+0000 mgr.gibba032.whopeo (mgr.24625) 4529 : cluster [DBG] pgmap v2570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:10 gibba011 bash[31774]: cluster 2021-03-10T15:18:10.147419+0000 mgr.gibba032.whopeo (mgr.24625) 4529 : cluster [DBG] pgmap v2570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:12 gibba032 bash[24004]: cluster 2021-03-10T15:18:12.149169+0000 mgr.gibba032.whopeo (mgr.24625) 4530 : cluster [DBG] pgmap v2571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:12 gibba011 bash[30791]: cluster 2021-03-10T15:18:12.149169+0000 mgr.gibba032.whopeo (mgr.24625) 4530 : cluster [DBG] pgmap v2571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:12 gibba011 bash[31774]: cluster 2021-03-10T15:18:12.149169+0000 mgr.gibba032.whopeo (mgr.24625) 4530 : cluster [DBG] pgmap v2571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:14 gibba032 bash[24004]: cluster 2021-03-10T15:18:14.150706+0000 mgr.gibba032.whopeo (mgr.24625) 4531 : cluster [DBG] pgmap v2572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:14 gibba011 bash[30791]: cluster 2021-03-10T15:18:14.150706+0000 mgr.gibba032.whopeo (mgr.24625) 4531 : cluster [DBG] pgmap v2572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:14 gibba011 bash[31774]: cluster 2021-03-10T15:18:14.150706+0000 mgr.gibba032.whopeo (mgr.24625) 4531 : cluster [DBG] pgmap v2572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:15 gibba032 bash[24004]: cephadm 2021-03-10T15:18:15.086580+0000 mgr.gibba032.whopeo (mgr.24625) 4532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:15.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:15 gibba011 bash[30791]: cephadm 2021-03-10T15:18:15.086580+0000 mgr.gibba032.whopeo (mgr.24625) 4532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:15 gibba011 bash[31774]: cephadm 2021-03-10T15:18:15.086580+0000 mgr.gibba032.whopeo (mgr.24625) 4532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:16 gibba032 bash[24004]: cluster 2021-03-10T15:18:16.152261+0000 mgr.gibba032.whopeo (mgr.24625) 4533 : cluster [DBG] pgmap v2573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:16 gibba011 bash[30791]: cluster 2021-03-10T15:18:16.152261+0000 mgr.gibba032.whopeo (mgr.24625) 4533 : cluster [DBG] pgmap v2573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:16 gibba011 bash[31774]: cluster 2021-03-10T15:18:16.152261+0000 mgr.gibba032.whopeo (mgr.24625) 4533 : cluster [DBG] pgmap v2573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:18 gibba032 bash[24004]: cluster 2021-03-10T15:18:18.154098+0000 mgr.gibba032.whopeo (mgr.24625) 4534 : cluster [DBG] pgmap v2574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:18 gibba011 bash[30791]: cluster 2021-03-10T15:18:18.154098+0000 mgr.gibba032.whopeo (mgr.24625) 4534 : cluster [DBG] pgmap v2574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:18 gibba011 bash[31774]: cluster 2021-03-10T15:18:18.154098+0000 mgr.gibba032.whopeo (mgr.24625) 4534 : cluster [DBG] pgmap v2574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:20 gibba032 bash[24004]: cluster 2021-03-10T15:18:20.155728+0000 mgr.gibba032.whopeo (mgr.24625) 4535 : cluster [DBG] pgmap v2575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:20 gibba011 bash[30791]: cluster 2021-03-10T15:18:20.155728+0000 mgr.gibba032.whopeo (mgr.24625) 4535 : cluster [DBG] pgmap v2575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:20 gibba011 bash[31774]: cluster 2021-03-10T15:18:20.155728+0000 mgr.gibba032.whopeo (mgr.24625) 4535 : cluster [DBG] pgmap v2575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:22 gibba032 bash[24004]: cluster 2021-03-10T15:18:22.157291+0000 mgr.gibba032.whopeo (mgr.24625) 4536 : cluster [DBG] pgmap v2576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:22 gibba011 bash[30791]: cluster 2021-03-10T15:18:22.157291+0000 mgr.gibba032.whopeo (mgr.24625) 4536 : cluster [DBG] pgmap v2576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:22 gibba011 bash[31774]: cluster 2021-03-10T15:18:22.157291+0000 mgr.gibba032.whopeo (mgr.24625) 4536 : cluster [DBG] pgmap v2576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:24.300 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:18:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:24 gibba032 bash[24004]: cluster 2021-03-10T15:18:24.158796+0000 mgr.gibba032.whopeo (mgr.24625) 4537 : cluster [DBG] pgmap v2577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:24.527 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:18:24.527 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (85m) 9m ago 90m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:18:24.527 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (90m) 8m ago 90m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:18:24.527 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (85m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:18:24.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (85m) 8m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:18:24.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (84m) 9m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:18:24.528 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (84m) 9m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:18:24.528 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (84m) 8m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:18:24.528 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (84m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:18:24.529 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (90m) 9m ago 90m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:18:24.529 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (90m) 8m ago 90m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:18:24.529 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (83m) 9m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:18:24.529 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (82m) 9m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:18:24.529 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (82m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:18:24.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (82m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:18:24.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (81m) 8m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:18:24.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (81m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:18:24.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (80m) 8m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:18:24.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (80m) 8m ago 90m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:18:24.531 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (89m) 8m ago 90m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:18:24.531 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 88m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:18:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:24 gibba011 bash[30791]: cluster 2021-03-10T15:18:24.158796+0000 mgr.gibba032.whopeo (mgr.24625) 4537 : cluster [DBG] pgmap v2577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:24 gibba011 bash[31774]: cluster 2021-03-10T15:18:24.158796+0000 mgr.gibba032.whopeo (mgr.24625) 4537 : cluster [DBG] pgmap v2577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:24.780 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:18:24.780 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:18:24.780 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:18:24.780 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:24.781 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:18:24.781 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:18:24.781 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:24.781 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:18:24.782 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:18:24.782 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:24.782 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:18:24.782 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:18:24.783 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:18:24.783 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:18:24.783 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:18:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:25 gibba032 bash[24004]: audit 2021-03-10T15:18:24.286718+0000 mgr.gibba032.whopeo (mgr.24625) 4538 : audit [DBG] from='client.47097 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:25 gibba032 bash[24004]: audit 2021-03-10T15:18:24.510656+0000 mgr.gibba032.whopeo (mgr.24625) 4539 : audit [DBG] from='client.47103 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:25 gibba032 bash[24004]: audit 2021-03-10T15:18:24.777136+0000 mon.a (mon.0) 1843 : audit [DBG] from='client.? 172.21.2.111:0/1153579074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:25.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[30791]: audit 2021-03-10T15:18:24.286718+0000 mgr.gibba032.whopeo (mgr.24625) 4538 : audit [DBG] from='client.47097 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[30791]: audit 2021-03-10T15:18:24.510656+0000 mgr.gibba032.whopeo (mgr.24625) 4539 : audit [DBG] from='client.47103 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[30791]: audit 2021-03-10T15:18:24.777136+0000 mon.a (mon.0) 1843 : audit [DBG] from='client.? 172.21.2.111:0/1153579074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:25.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[31774]: audit 2021-03-10T15:18:24.286718+0000 mgr.gibba032.whopeo (mgr.24625) 4538 : audit [DBG] from='client.47097 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[31774]: audit 2021-03-10T15:18:24.510656+0000 mgr.gibba032.whopeo (mgr.24625) 4539 : audit [DBG] from='client.47103 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:25.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:25 gibba011 bash[31774]: audit 2021-03-10T15:18:24.777136+0000 mon.a (mon.0) 1843 : audit [DBG] from='client.? 172.21.2.111:0/1153579074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:26 gibba032 bash[24004]: cluster 2021-03-10T15:18:26.160257+0000 mgr.gibba032.whopeo (mgr.24625) 4540 : cluster [DBG] pgmap v2578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:26 gibba011 bash[30791]: cluster 2021-03-10T15:18:26.160257+0000 mgr.gibba032.whopeo (mgr.24625) 4540 : cluster [DBG] pgmap v2578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:26 gibba011 bash[31774]: cluster 2021-03-10T15:18:26.160257+0000 mgr.gibba032.whopeo (mgr.24625) 4540 : cluster [DBG] pgmap v2578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:28 gibba032 bash[24004]: cluster 2021-03-10T15:18:28.161867+0000 mgr.gibba032.whopeo (mgr.24625) 4541 : cluster [DBG] pgmap v2579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:28 gibba011 bash[30791]: cluster 2021-03-10T15:18:28.161867+0000 mgr.gibba032.whopeo (mgr.24625) 4541 : cluster [DBG] pgmap v2579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:28 gibba011 bash[31774]: cluster 2021-03-10T15:18:28.161867+0000 mgr.gibba032.whopeo (mgr.24625) 4541 : cluster [DBG] pgmap v2579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:30 gibba032 bash[24004]: cephadm 2021-03-10T15:18:30.102643+0000 mgr.gibba032.whopeo (mgr.24625) 4542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:30 gibba032 bash[24004]: cluster 2021-03-10T15:18:30.163341+0000 mgr.gibba032.whopeo (mgr.24625) 4543 : cluster [DBG] pgmap v2580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:30 gibba011 bash[30791]: cephadm 2021-03-10T15:18:30.102643+0000 mgr.gibba032.whopeo (mgr.24625) 4542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:30 gibba011 bash[30791]: cluster 2021-03-10T15:18:30.163341+0000 mgr.gibba032.whopeo (mgr.24625) 4543 : cluster [DBG] pgmap v2580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:30 gibba011 bash[31774]: cephadm 2021-03-10T15:18:30.102643+0000 mgr.gibba032.whopeo (mgr.24625) 4542 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:30 gibba011 bash[31774]: cluster 2021-03-10T15:18:30.163341+0000 mgr.gibba032.whopeo (mgr.24625) 4543 : cluster [DBG] pgmap v2580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:32 gibba032 bash[24004]: cluster 2021-03-10T15:18:32.164839+0000 mgr.gibba032.whopeo (mgr.24625) 4544 : cluster [DBG] pgmap v2581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:32 gibba011 bash[30791]: cluster 2021-03-10T15:18:32.164839+0000 mgr.gibba032.whopeo (mgr.24625) 4544 : cluster [DBG] pgmap v2581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:32 gibba011 bash[31774]: cluster 2021-03-10T15:18:32.164839+0000 mgr.gibba032.whopeo (mgr.24625) 4544 : cluster [DBG] pgmap v2581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:34 gibba032 bash[24004]: cluster 2021-03-10T15:18:34.166193+0000 mgr.gibba032.whopeo (mgr.24625) 4545 : cluster [DBG] pgmap v2582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:34 gibba011 bash[30791]: cluster 2021-03-10T15:18:34.166193+0000 mgr.gibba032.whopeo (mgr.24625) 4545 : cluster [DBG] pgmap v2582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:34 gibba011 bash[31774]: cluster 2021-03-10T15:18:34.166193+0000 mgr.gibba032.whopeo (mgr.24625) 4545 : cluster [DBG] pgmap v2582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:36 gibba032 bash[24004]: cluster 2021-03-10T15:18:36.167673+0000 mgr.gibba032.whopeo (mgr.24625) 4546 : cluster [DBG] pgmap v2583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:36 gibba011 bash[30791]: cluster 2021-03-10T15:18:36.167673+0000 mgr.gibba032.whopeo (mgr.24625) 4546 : cluster [DBG] pgmap v2583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:36 gibba011 bash[31774]: cluster 2021-03-10T15:18:36.167673+0000 mgr.gibba032.whopeo (mgr.24625) 4546 : cluster [DBG] pgmap v2583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:38 gibba032 bash[24004]: cluster 2021-03-10T15:18:38.169447+0000 mgr.gibba032.whopeo (mgr.24625) 4547 : cluster [DBG] pgmap v2584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:38 gibba011 bash[30791]: cluster 2021-03-10T15:18:38.169447+0000 mgr.gibba032.whopeo (mgr.24625) 4547 : cluster [DBG] pgmap v2584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:38 gibba011 bash[31774]: cluster 2021-03-10T15:18:38.169447+0000 mgr.gibba032.whopeo (mgr.24625) 4547 : cluster [DBG] pgmap v2584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:40 gibba032 bash[24004]: cluster 2021-03-10T15:18:40.171081+0000 mgr.gibba032.whopeo (mgr.24625) 4548 : cluster [DBG] pgmap v2585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:40 gibba011 bash[30791]: cluster 2021-03-10T15:18:40.171081+0000 mgr.gibba032.whopeo (mgr.24625) 4548 : cluster [DBG] pgmap v2585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:40 gibba011 bash[31774]: cluster 2021-03-10T15:18:40.171081+0000 mgr.gibba032.whopeo (mgr.24625) 4548 : cluster [DBG] pgmap v2585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:42 gibba032 bash[24004]: cluster 2021-03-10T15:18:42.172799+0000 mgr.gibba032.whopeo (mgr.24625) 4549 : cluster [DBG] pgmap v2586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:42 gibba011 bash[30791]: cluster 2021-03-10T15:18:42.172799+0000 mgr.gibba032.whopeo (mgr.24625) 4549 : cluster [DBG] pgmap v2586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:42 gibba011 bash[31774]: cluster 2021-03-10T15:18:42.172799+0000 mgr.gibba032.whopeo (mgr.24625) 4549 : cluster [DBG] pgmap v2586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:44 gibba032 bash[24004]: cluster 2021-03-10T15:18:44.174255+0000 mgr.gibba032.whopeo (mgr.24625) 4550 : cluster [DBG] pgmap v2587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:44 gibba011 bash[30791]: cluster 2021-03-10T15:18:44.174255+0000 mgr.gibba032.whopeo (mgr.24625) 4550 : cluster [DBG] pgmap v2587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:44 gibba011 bash[31774]: cluster 2021-03-10T15:18:44.174255+0000 mgr.gibba032.whopeo (mgr.24625) 4550 : cluster [DBG] pgmap v2587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:45 gibba032 bash[24004]: cephadm 2021-03-10T15:18:45.120396+0000 mgr.gibba032.whopeo (mgr.24625) 4551 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:18:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:45 gibba032 bash[24004]: audit 2021-03-10T15:18:45.122467+0000 mon.a (mon.0) 1844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:18:45.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:45 gibba011 bash[30791]: cephadm 2021-03-10T15:18:45.120396+0000 mgr.gibba032.whopeo (mgr.24625) 4551 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:18:45.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:45 gibba011 bash[30791]: audit 2021-03-10T15:18:45.122467+0000 mon.a (mon.0) 1844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:18:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:45 gibba011 bash[31774]: cephadm 2021-03-10T15:18:45.120396+0000 mgr.gibba032.whopeo (mgr.24625) 4551 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:18:45.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:45 gibba011 bash[31774]: audit 2021-03-10T15:18:45.122467+0000 mon.a (mon.0) 1844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:18:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:46 gibba011 bash[30791]: cluster 2021-03-10T15:18:46.175770+0000 mgr.gibba032.whopeo (mgr.24625) 4552 : cluster [DBG] pgmap v2588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:46 gibba011 bash[31774]: cluster 2021-03-10T15:18:46.175770+0000 mgr.gibba032.whopeo (mgr.24625) 4552 : cluster [DBG] pgmap v2588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:46 gibba032 bash[24004]: cluster 2021-03-10T15:18:46.175770+0000 mgr.gibba032.whopeo (mgr.24625) 4552 : cluster [DBG] pgmap v2588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:48 gibba032 bash[24004]: cluster 2021-03-10T15:18:48.177466+0000 mgr.gibba032.whopeo (mgr.24625) 4553 : cluster [DBG] pgmap v2589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:48 gibba011 bash[30791]: cluster 2021-03-10T15:18:48.177466+0000 mgr.gibba032.whopeo (mgr.24625) 4553 : cluster [DBG] pgmap v2589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:48 gibba011 bash[31774]: cluster 2021-03-10T15:18:48.177466+0000 mgr.gibba032.whopeo (mgr.24625) 4553 : cluster [DBG] pgmap v2589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.657616+0000 mgr.gibba032.whopeo (mgr.24625) 4554 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:18:50.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.658391+0000 mon.a (mon.0) 1845 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.867852+0000 mon.a (mon.0) 1846 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.879462+0000 mgr.gibba032.whopeo (mgr.24625) 4555 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:18:50.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.883873+0000 mon.a (mon.0) 1847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:18:50.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.885782+0000 mgr.gibba032.whopeo (mgr.24625) 4556 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:18:50.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.888676+0000 mon.a (mon.0) 1848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.890154+0000 mgr.gibba032.whopeo (mgr.24625) 4557 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:18:50.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.890651+0000 mgr.gibba032.whopeo (mgr.24625) 4558 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:18:50.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.893600+0000 mon.a (mon.0) 1849 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.894469+0000 mgr.gibba032.whopeo (mgr.24625) 4559 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:18:50.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.894853+0000 mgr.gibba032.whopeo (mgr.24625) 4560 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:18:50.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.896934+0000 mon.a (mon.0) 1850 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.897361+0000 mgr.gibba032.whopeo (mgr.24625) 4561 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:18:50.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.899865+0000 mgr.gibba032.whopeo (mgr.24625) 4562 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:18:50.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.900107+0000 mgr.gibba032.whopeo (mgr.24625) 4563 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:18:50.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.903189+0000 mon.a (mon.0) 1851 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.903734+0000 mgr.gibba032.whopeo (mgr.24625) 4564 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:18:50.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.906121+0000 mgr.gibba032.whopeo (mgr.24625) 4565 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:18:50.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: audit 2021-03-10T15:18:48.908442+0000 mon.a (mon.0) 1852 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.909130+0000 mgr.gibba032.whopeo (mgr.24625) 4566 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:18:50.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.909629+0000 mgr.gibba032.whopeo (mgr.24625) 4567 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:18:50.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:49 gibba032 bash[24004]: cephadm 2021-03-10T15:18:48.910461+0000 mgr.gibba032.whopeo (mgr.24625) 4568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.657616+0000 mgr.gibba032.whopeo (mgr.24625) 4554 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:18:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.658391+0000 mon.a (mon.0) 1845 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.867852+0000 mon.a (mon.0) 1846 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.879462+0000 mgr.gibba032.whopeo (mgr.24625) 4555 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:18:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.883873+0000 mon.a (mon.0) 1847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:18:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.885782+0000 mgr.gibba032.whopeo (mgr.24625) 4556 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:18:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.888676+0000 mon.a (mon.0) 1848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.890154+0000 mgr.gibba032.whopeo (mgr.24625) 4557 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:18:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.890651+0000 mgr.gibba032.whopeo (mgr.24625) 4558 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:18:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.893600+0000 mon.a (mon.0) 1849 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.894469+0000 mgr.gibba032.whopeo (mgr.24625) 4559 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:18:50.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.894853+0000 mgr.gibba032.whopeo (mgr.24625) 4560 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:18:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.896934+0000 mon.a (mon.0) 1850 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.897361+0000 mgr.gibba032.whopeo (mgr.24625) 4561 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:18:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.899865+0000 mgr.gibba032.whopeo (mgr.24625) 4562 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:18:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.900107+0000 mgr.gibba032.whopeo (mgr.24625) 4563 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:18:50.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.903189+0000 mon.a (mon.0) 1851 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.903734+0000 mgr.gibba032.whopeo (mgr.24625) 4564 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:18:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.906121+0000 mgr.gibba032.whopeo (mgr.24625) 4565 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:18:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: audit 2021-03-10T15:18:48.908442+0000 mon.a (mon.0) 1852 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.909130+0000 mgr.gibba032.whopeo (mgr.24625) 4566 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:18:50.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.909629+0000 mgr.gibba032.whopeo (mgr.24625) 4567 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:18:50.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[30791]: cephadm 2021-03-10T15:18:48.910461+0000 mgr.gibba032.whopeo (mgr.24625) 4568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:50.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.657616+0000 mgr.gibba032.whopeo (mgr.24625) 4554 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:18:50.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.658391+0000 mon.a (mon.0) 1845 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.867852+0000 mon.a (mon.0) 1846 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:18:50.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.879462+0000 mgr.gibba032.whopeo (mgr.24625) 4555 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:18:50.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.883873+0000 mon.a (mon.0) 1847 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:18:50.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.885782+0000 mgr.gibba032.whopeo (mgr.24625) 4556 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:18:50.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.888676+0000 mon.a (mon.0) 1848 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.890154+0000 mgr.gibba032.whopeo (mgr.24625) 4557 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:18:50.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.890651+0000 mgr.gibba032.whopeo (mgr.24625) 4558 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:18:50.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.893600+0000 mon.a (mon.0) 1849 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.894469+0000 mgr.gibba032.whopeo (mgr.24625) 4559 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:18:50.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.894853+0000 mgr.gibba032.whopeo (mgr.24625) 4560 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:18:50.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.896934+0000 mon.a (mon.0) 1850 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.897361+0000 mgr.gibba032.whopeo (mgr.24625) 4561 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:18:50.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.899865+0000 mgr.gibba032.whopeo (mgr.24625) 4562 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:18:50.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.900107+0000 mgr.gibba032.whopeo (mgr.24625) 4563 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:18:50.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.903189+0000 mon.a (mon.0) 1851 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.903734+0000 mgr.gibba032.whopeo (mgr.24625) 4564 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:18:50.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.906121+0000 mgr.gibba032.whopeo (mgr.24625) 4565 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:18:50.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: audit 2021-03-10T15:18:48.908442+0000 mon.a (mon.0) 1852 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:50.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.909130+0000 mgr.gibba032.whopeo (mgr.24625) 4566 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:18:50.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.909629+0000 mgr.gibba032.whopeo (mgr.24625) 4567 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:18:50.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:49 gibba011 bash[31774]: cephadm 2021-03-10T15:18:48.910461+0000 mgr.gibba032.whopeo (mgr.24625) 4568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:18:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:50 gibba032 bash[24004]: cluster 2021-03-10T15:18:50.179099+0000 mgr.gibba032.whopeo (mgr.24625) 4569 : cluster [DBG] pgmap v2590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:50 gibba011 bash[30791]: cluster 2021-03-10T15:18:50.179099+0000 mgr.gibba032.whopeo (mgr.24625) 4569 : cluster [DBG] pgmap v2590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:50 gibba011 bash[31774]: cluster 2021-03-10T15:18:50.179099+0000 mgr.gibba032.whopeo (mgr.24625) 4569 : cluster [DBG] pgmap v2590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:52 gibba032 bash[24004]: cluster 2021-03-10T15:18:52.180528+0000 mgr.gibba032.whopeo (mgr.24625) 4570 : cluster [DBG] pgmap v2591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:52 gibba011 bash[30791]: cluster 2021-03-10T15:18:52.180528+0000 mgr.gibba032.whopeo (mgr.24625) 4570 : cluster [DBG] pgmap v2591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:52 gibba011 bash[31774]: cluster 2021-03-10T15:18:52.180528+0000 mgr.gibba032.whopeo (mgr.24625) 4570 : cluster [DBG] pgmap v2591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:54 gibba032 bash[24004]: cluster 2021-03-10T15:18:54.181945+0000 mgr.gibba032.whopeo (mgr.24625) 4571 : cluster [DBG] pgmap v2592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:54 gibba011 bash[30791]: cluster 2021-03-10T15:18:54.181945+0000 mgr.gibba032.whopeo (mgr.24625) 4571 : cluster [DBG] pgmap v2592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:54 gibba011 bash[31774]: cluster 2021-03-10T15:18:54.181945+0000 mgr.gibba032.whopeo (mgr.24625) 4571 : cluster [DBG] pgmap v2592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:55.023 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:18:55.251 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:18:55.252 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (85m) 6s ago 90m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:18:55.252 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (90m) 9m ago 90m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:18:55.252 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (85m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:18:55.252 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (86m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:18:55.252 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (85m) 6s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:18:55.253 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (85m) 6s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:18:55.253 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (84m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:18:55.253 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (84m) 6s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:18:55.253 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (91m) 6s ago 91m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:18:55.254 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (91m) 9m ago 91m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:18:55.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (83m) 6s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:18:55.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (83m) 6s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:18:55.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (83m) 6s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:18:55.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (82m) 6s ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:18:55.255 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (82m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:18:55.255 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (81m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:18:55.255 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (81m) 9m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:18:55.255 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (80m) 9m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:18:55.256 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (90m) 9m ago 91m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:18:55.256 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6s ago 88m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:18:55.505 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:18:55.506 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:18:55.506 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:18:55.506 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:55.507 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:18:55.507 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:18:55.507 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:55.507 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:18:55.508 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:18:55.508 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:18:55.508 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:18:55.508 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:18:55.508 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:18:55.509 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:18:55.509 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:18:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:55 gibba032 bash[24004]: audit 2021-03-10T15:18:55.011022+0000 mgr.gibba032.whopeo (mgr.24625) 4572 : audit [DBG] from='client.57100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:55 gibba011 bash[30791]: audit 2021-03-10T15:18:55.011022+0000 mgr.gibba032.whopeo (mgr.24625) 4572 : audit [DBG] from='client.57100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:55 gibba011 bash[31774]: audit 2021-03-10T15:18:55.011022+0000 mgr.gibba032.whopeo (mgr.24625) 4572 : audit [DBG] from='client.57100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[30791]: audit 2021-03-10T15:18:55.235330+0000 mgr.gibba032.whopeo (mgr.24625) 4573 : audit [DBG] from='client.57106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[30791]: audit 2021-03-10T15:18:55.502998+0000 mon.a (mon.0) 1853 : audit [DBG] from='client.? 172.21.2.111:0/3961646851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[30791]: cluster 2021-03-10T15:18:56.183466+0000 mgr.gibba032.whopeo (mgr.24625) 4574 : cluster [DBG] pgmap v2593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[31774]: audit 2021-03-10T15:18:55.235330+0000 mgr.gibba032.whopeo (mgr.24625) 4573 : audit [DBG] from='client.57106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[31774]: audit 2021-03-10T15:18:55.502998+0000 mon.a (mon.0) 1853 : audit [DBG] from='client.? 172.21.2.111:0/3961646851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:56 gibba011 bash[31774]: cluster 2021-03-10T15:18:56.183466+0000 mgr.gibba032.whopeo (mgr.24625) 4574 : cluster [DBG] pgmap v2593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:56 gibba032 bash[24004]: audit 2021-03-10T15:18:55.235330+0000 mgr.gibba032.whopeo (mgr.24625) 4573 : audit [DBG] from='client.57106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:18:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:56 gibba032 bash[24004]: audit 2021-03-10T15:18:55.502998+0000 mon.a (mon.0) 1853 : audit [DBG] from='client.? 172.21.2.111:0/3961646851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:18:56.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:56 gibba032 bash[24004]: cluster 2021-03-10T15:18:56.183466+0000 mgr.gibba032.whopeo (mgr.24625) 4574 : cluster [DBG] pgmap v2593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.548354+0000 mon.a (mon.0) 1854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:18:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.548980+0000 mon.a (mon.0) 1855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:18:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.561086+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:18:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.561828+0000 mon.a (mon.0) 1857 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.562558+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[30791]: audit 2021-03-10T15:18:56.563229+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:18:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.548354+0000 mon.a (mon.0) 1854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:18:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.548980+0000 mon.a (mon.0) 1855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:18:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.561086+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:18:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.561828+0000 mon.a (mon.0) 1857 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.562558+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:57 gibba011 bash[31774]: audit 2021-03-10T15:18:56.563229+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:18:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.548354+0000 mon.a (mon.0) 1854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:18:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.548980+0000 mon.a (mon.0) 1855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:18:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.561086+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:18:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.561828+0000 mon.a (mon.0) 1857 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.562558+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:18:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:57 gibba032 bash[24004]: audit 2021-03-10T15:18:56.563229+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:18:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:18:58 gibba032 bash[24004]: cluster 2021-03-10T15:18:58.185271+0000 mgr.gibba032.whopeo (mgr.24625) 4575 : cluster [DBG] pgmap v2594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:18:58 gibba011 bash[30791]: cluster 2021-03-10T15:18:58.185271+0000 mgr.gibba032.whopeo (mgr.24625) 4575 : cluster [DBG] pgmap v2594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:18:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:18:58 gibba011 bash[31774]: cluster 2021-03-10T15:18:58.185271+0000 mgr.gibba032.whopeo (mgr.24625) 4575 : cluster [DBG] pgmap v2594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:00 gibba032 bash[24004]: cluster 2021-03-10T15:19:00.186761+0000 mgr.gibba032.whopeo (mgr.24625) 4576 : cluster [DBG] pgmap v2595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:00 gibba011 bash[30791]: cluster 2021-03-10T15:19:00.186761+0000 mgr.gibba032.whopeo (mgr.24625) 4576 : cluster [DBG] pgmap v2595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:00 gibba011 bash[31774]: cluster 2021-03-10T15:19:00.186761+0000 mgr.gibba032.whopeo (mgr.24625) 4576 : cluster [DBG] pgmap v2595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:02 gibba032 bash[24004]: cluster 2021-03-10T15:19:02.188546+0000 mgr.gibba032.whopeo (mgr.24625) 4577 : cluster [DBG] pgmap v2596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:02 gibba011 bash[30791]: cluster 2021-03-10T15:19:02.188546+0000 mgr.gibba032.whopeo (mgr.24625) 4577 : cluster [DBG] pgmap v2596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:02 gibba011 bash[31774]: cluster 2021-03-10T15:19:02.188546+0000 mgr.gibba032.whopeo (mgr.24625) 4577 : cluster [DBG] pgmap v2596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:04 gibba011 bash[30791]: cephadm 2021-03-10T15:19:03.916876+0000 mgr.gibba032.whopeo (mgr.24625) 4578 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:04 gibba011 bash[30791]: cluster 2021-03-10T15:19:04.189895+0000 mgr.gibba032.whopeo (mgr.24625) 4579 : cluster [DBG] pgmap v2597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:04 gibba011 bash[31774]: cephadm 2021-03-10T15:19:03.916876+0000 mgr.gibba032.whopeo (mgr.24625) 4578 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:04 gibba011 bash[31774]: cluster 2021-03-10T15:19:04.189895+0000 mgr.gibba032.whopeo (mgr.24625) 4579 : cluster [DBG] pgmap v2597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:04 gibba032 bash[24004]: cephadm 2021-03-10T15:19:03.916876+0000 mgr.gibba032.whopeo (mgr.24625) 4578 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:04 gibba032 bash[24004]: cluster 2021-03-10T15:19:04.189895+0000 mgr.gibba032.whopeo (mgr.24625) 4579 : cluster [DBG] pgmap v2597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:06 gibba011 bash[30791]: cluster 2021-03-10T15:19:06.191353+0000 mgr.gibba032.whopeo (mgr.24625) 4580 : cluster [DBG] pgmap v2598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:06 gibba011 bash[31774]: cluster 2021-03-10T15:19:06.191353+0000 mgr.gibba032.whopeo (mgr.24625) 4580 : cluster [DBG] pgmap v2598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:06 gibba032 bash[24004]: cluster 2021-03-10T15:19:06.191353+0000 mgr.gibba032.whopeo (mgr.24625) 4580 : cluster [DBG] pgmap v2598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:08 gibba032 bash[24004]: cluster 2021-03-10T15:19:08.192916+0000 mgr.gibba032.whopeo (mgr.24625) 4581 : cluster [DBG] pgmap v2599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:08 gibba011 bash[30791]: cluster 2021-03-10T15:19:08.192916+0000 mgr.gibba032.whopeo (mgr.24625) 4581 : cluster [DBG] pgmap v2599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:08 gibba011 bash[31774]: cluster 2021-03-10T15:19:08.192916+0000 mgr.gibba032.whopeo (mgr.24625) 4581 : cluster [DBG] pgmap v2599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:10 gibba011 bash[30791]: cluster 2021-03-10T15:19:10.194485+0000 mgr.gibba032.whopeo (mgr.24625) 4582 : cluster [DBG] pgmap v2600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:10 gibba011 bash[31774]: cluster 2021-03-10T15:19:10.194485+0000 mgr.gibba032.whopeo (mgr.24625) 4582 : cluster [DBG] pgmap v2600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:10 gibba032 bash[24004]: cluster 2021-03-10T15:19:10.194485+0000 mgr.gibba032.whopeo (mgr.24625) 4582 : cluster [DBG] pgmap v2600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:12 gibba011 bash[30791]: cluster 2021-03-10T15:19:12.196185+0000 mgr.gibba032.whopeo (mgr.24625) 4583 : cluster [DBG] pgmap v2601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:12 gibba011 bash[31774]: cluster 2021-03-10T15:19:12.196185+0000 mgr.gibba032.whopeo (mgr.24625) 4583 : cluster [DBG] pgmap v2601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:12 gibba032 bash[24004]: cluster 2021-03-10T15:19:12.196185+0000 mgr.gibba032.whopeo (mgr.24625) 4583 : cluster [DBG] pgmap v2601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:14 gibba011 bash[30791]: cluster 2021-03-10T15:19:14.197252+0000 mgr.gibba032.whopeo (mgr.24625) 4584 : cluster [DBG] pgmap v2602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:14 gibba011 bash[31774]: cluster 2021-03-10T15:19:14.197252+0000 mgr.gibba032.whopeo (mgr.24625) 4584 : cluster [DBG] pgmap v2602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:14 gibba032 bash[24004]: cluster 2021-03-10T15:19:14.197252+0000 mgr.gibba032.whopeo (mgr.24625) 4584 : cluster [DBG] pgmap v2602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:16 gibba011 bash[30791]: cluster 2021-03-10T15:19:16.198786+0000 mgr.gibba032.whopeo (mgr.24625) 4585 : cluster [DBG] pgmap v2603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:16 gibba011 bash[31774]: cluster 2021-03-10T15:19:16.198786+0000 mgr.gibba032.whopeo (mgr.24625) 4585 : cluster [DBG] pgmap v2603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:16 gibba032 bash[24004]: cluster 2021-03-10T15:19:16.198786+0000 mgr.gibba032.whopeo (mgr.24625) 4585 : cluster [DBG] pgmap v2603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:18 gibba032 bash[24004]: cluster 2021-03-10T15:19:18.200585+0000 mgr.gibba032.whopeo (mgr.24625) 4586 : cluster [DBG] pgmap v2604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:18 gibba011 bash[30791]: cluster 2021-03-10T15:19:18.200585+0000 mgr.gibba032.whopeo (mgr.24625) 4586 : cluster [DBG] pgmap v2604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:18 gibba011 bash[31774]: cluster 2021-03-10T15:19:18.200585+0000 mgr.gibba032.whopeo (mgr.24625) 4586 : cluster [DBG] pgmap v2604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:19 gibba032 bash[24004]: cephadm 2021-03-10T15:19:18.932899+0000 mgr.gibba032.whopeo (mgr.24625) 4587 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:19.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:19 gibba011 bash[30791]: cephadm 2021-03-10T15:19:18.932899+0000 mgr.gibba032.whopeo (mgr.24625) 4587 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:19.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:19 gibba011 bash[31774]: cephadm 2021-03-10T15:19:18.932899+0000 mgr.gibba032.whopeo (mgr.24625) 4587 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:20 gibba032 bash[24004]: cluster 2021-03-10T15:19:20.202223+0000 mgr.gibba032.whopeo (mgr.24625) 4588 : cluster [DBG] pgmap v2605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:20 gibba011 bash[30791]: cluster 2021-03-10T15:19:20.202223+0000 mgr.gibba032.whopeo (mgr.24625) 4588 : cluster [DBG] pgmap v2605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:20 gibba011 bash[31774]: cluster 2021-03-10T15:19:20.202223+0000 mgr.gibba032.whopeo (mgr.24625) 4588 : cluster [DBG] pgmap v2605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:22 gibba011 bash[30791]: cluster 2021-03-10T15:19:22.203804+0000 mgr.gibba032.whopeo (mgr.24625) 4589 : cluster [DBG] pgmap v2606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:22 gibba011 bash[31774]: cluster 2021-03-10T15:19:22.203804+0000 mgr.gibba032.whopeo (mgr.24625) 4589 : cluster [DBG] pgmap v2606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:22 gibba032 bash[24004]: cluster 2021-03-10T15:19:22.203804+0000 mgr.gibba032.whopeo (mgr.24625) 4589 : cluster [DBG] pgmap v2606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:24 gibba011 bash[30791]: cluster 2021-03-10T15:19:24.205052+0000 mgr.gibba032.whopeo (mgr.24625) 4590 : cluster [DBG] pgmap v2607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:24 gibba011 bash[31774]: cluster 2021-03-10T15:19:24.205052+0000 mgr.gibba032.whopeo (mgr.24625) 4590 : cluster [DBG] pgmap v2607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:24 gibba032 bash[24004]: cluster 2021-03-10T15:19:24.205052+0000 mgr.gibba032.whopeo (mgr.24625) 4590 : cluster [DBG] pgmap v2607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:25.751 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:19:26.004 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:19:26.005 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (86m) 37s ago 91m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:19:26.005 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (91m) 9m ago 91m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:19:26.005 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (86m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:19:26.005 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (86m) 9m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:19:26.006 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (85m) 37s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:19:26.006 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (85m) 37s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:19:26.006 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (85m) 9m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:19:26.006 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (85m) 37s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:19:26.007 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (91m) 37s ago 91m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:19:26.007 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (91m) 9m ago 91m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:19:26.007 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (84m) 37s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:19:26.007 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (83m) 37s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:19:26.008 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (83m) 37s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:19:26.008 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (83m) 37s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:19:26.008 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (82m) 9m ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:19:26.009 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (82m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:19:26.009 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (81m) 9m ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:19:26.009 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (81m) 9m ago 91m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:19:26.009 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (91m) 9m ago 91m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:19:26.010 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 37s ago 89m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:19:26.258 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:19:26.259 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:19:26.259 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:19:26.259 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:26.259 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:19:26.260 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:19:26.260 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:26.260 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:19:26.260 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:19:26.260 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:26.261 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:19:26.261 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:19:26.261 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:19:26.261 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:19:26.261 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:19:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[30791]: audit 2021-03-10T15:19:25.737455+0000 mgr.gibba032.whopeo (mgr.24625) 4591 : audit [DBG] from='client.47133 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[30791]: audit 2021-03-10T15:19:25.961426+0000 mgr.gibba032.whopeo (mgr.24625) 4592 : audit [DBG] from='client.57124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[30791]: cluster 2021-03-10T15:19:26.206543+0000 mgr.gibba032.whopeo (mgr.24625) 4593 : cluster [DBG] pgmap v2608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:26.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[30791]: audit 2021-03-10T15:19:26.255559+0000 mon.c (mon.1) 87 : audit [DBG] from='client.? 172.21.2.111:0/2517316839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[31774]: audit 2021-03-10T15:19:25.737455+0000 mgr.gibba032.whopeo (mgr.24625) 4591 : audit [DBG] from='client.47133 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[31774]: audit 2021-03-10T15:19:25.961426+0000 mgr.gibba032.whopeo (mgr.24625) 4592 : audit [DBG] from='client.57124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[31774]: cluster 2021-03-10T15:19:26.206543+0000 mgr.gibba032.whopeo (mgr.24625) 4593 : cluster [DBG] pgmap v2608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:26 gibba011 bash[31774]: audit 2021-03-10T15:19:26.255559+0000 mon.c (mon.1) 87 : audit [DBG] from='client.? 172.21.2.111:0/2517316839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:26 gibba032 bash[24004]: audit 2021-03-10T15:19:25.737455+0000 mgr.gibba032.whopeo (mgr.24625) 4591 : audit [DBG] from='client.47133 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:26 gibba032 bash[24004]: audit 2021-03-10T15:19:25.961426+0000 mgr.gibba032.whopeo (mgr.24625) 4592 : audit [DBG] from='client.57124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:26.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:26 gibba032 bash[24004]: cluster 2021-03-10T15:19:26.206543+0000 mgr.gibba032.whopeo (mgr.24625) 4593 : cluster [DBG] pgmap v2608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:26.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:26 gibba032 bash[24004]: audit 2021-03-10T15:19:26.255559+0000 mon.c (mon.1) 87 : audit [DBG] from='client.? 172.21.2.111:0/2517316839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:28 gibba032 bash[24004]: cluster 2021-03-10T15:19:28.208305+0000 mgr.gibba032.whopeo (mgr.24625) 4594 : cluster [DBG] pgmap v2609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:28.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:28 gibba011 bash[30791]: cluster 2021-03-10T15:19:28.208305+0000 mgr.gibba032.whopeo (mgr.24625) 4594 : cluster [DBG] pgmap v2609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:28 gibba011 bash[31774]: cluster 2021-03-10T15:19:28.208305+0000 mgr.gibba032.whopeo (mgr.24625) 4594 : cluster [DBG] pgmap v2609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:30 gibba011 bash[30791]: cluster 2021-03-10T15:19:30.209856+0000 mgr.gibba032.whopeo (mgr.24625) 4595 : cluster [DBG] pgmap v2610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:30 gibba011 bash[31774]: cluster 2021-03-10T15:19:30.209856+0000 mgr.gibba032.whopeo (mgr.24625) 4595 : cluster [DBG] pgmap v2610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:30 gibba032 bash[24004]: cluster 2021-03-10T15:19:30.209856+0000 mgr.gibba032.whopeo (mgr.24625) 4595 : cluster [DBG] pgmap v2610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:33.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:33 gibba032 bash[24004]: cluster 2021-03-10T15:19:32.211585+0000 mgr.gibba032.whopeo (mgr.24625) 4596 : cluster [DBG] pgmap v2611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:33.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:33 gibba011 bash[30791]: cluster 2021-03-10T15:19:32.211585+0000 mgr.gibba032.whopeo (mgr.24625) 4596 : cluster [DBG] pgmap v2611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:33.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:33 gibba011 bash[31774]: cluster 2021-03-10T15:19:32.211585+0000 mgr.gibba032.whopeo (mgr.24625) 4596 : cluster [DBG] pgmap v2611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:34 gibba032 bash[24004]: cephadm 2021-03-10T15:19:33.948930+0000 mgr.gibba032.whopeo (mgr.24625) 4597 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:34 gibba011 bash[30791]: cephadm 2021-03-10T15:19:33.948930+0000 mgr.gibba032.whopeo (mgr.24625) 4597 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:34.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:34 gibba011 bash[31774]: cephadm 2021-03-10T15:19:33.948930+0000 mgr.gibba032.whopeo (mgr.24625) 4597 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:35.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:35 gibba032 bash[24004]: cluster 2021-03-10T15:19:34.212928+0000 mgr.gibba032.whopeo (mgr.24625) 4598 : cluster [DBG] pgmap v2612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:35.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:35 gibba011 bash[30791]: cluster 2021-03-10T15:19:34.212928+0000 mgr.gibba032.whopeo (mgr.24625) 4598 : cluster [DBG] pgmap v2612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:35.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:35 gibba011 bash[31774]: cluster 2021-03-10T15:19:34.212928+0000 mgr.gibba032.whopeo (mgr.24625) 4598 : cluster [DBG] pgmap v2612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:37.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:37 gibba032 bash[24004]: cluster 2021-03-10T15:19:36.214367+0000 mgr.gibba032.whopeo (mgr.24625) 4599 : cluster [DBG] pgmap v2613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:37.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:37 gibba011 bash[30791]: cluster 2021-03-10T15:19:36.214367+0000 mgr.gibba032.whopeo (mgr.24625) 4599 : cluster [DBG] pgmap v2613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:37 gibba011 bash[31774]: cluster 2021-03-10T15:19:36.214367+0000 mgr.gibba032.whopeo (mgr.24625) 4599 : cluster [DBG] pgmap v2613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:39.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:39 gibba032 bash[24004]: cluster 2021-03-10T15:19:38.216125+0000 mgr.gibba032.whopeo (mgr.24625) 4600 : cluster [DBG] pgmap v2614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:39.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:39 gibba011 bash[30791]: cluster 2021-03-10T15:19:38.216125+0000 mgr.gibba032.whopeo (mgr.24625) 4600 : cluster [DBG] pgmap v2614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:39 gibba011 bash[31774]: cluster 2021-03-10T15:19:38.216125+0000 mgr.gibba032.whopeo (mgr.24625) 4600 : cluster [DBG] pgmap v2614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:41.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:41 gibba032 bash[24004]: cluster 2021-03-10T15:19:40.217793+0000 mgr.gibba032.whopeo (mgr.24625) 4601 : cluster [DBG] pgmap v2615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:41.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:41 gibba011 bash[30791]: cluster 2021-03-10T15:19:40.217793+0000 mgr.gibba032.whopeo (mgr.24625) 4601 : cluster [DBG] pgmap v2615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:41.875 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:41 gibba011 bash[31774]: cluster 2021-03-10T15:19:40.217793+0000 mgr.gibba032.whopeo (mgr.24625) 4601 : cluster [DBG] pgmap v2615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:43 gibba011 bash[30791]: cluster 2021-03-10T15:19:42.219708+0000 mgr.gibba032.whopeo (mgr.24625) 4602 : cluster [DBG] pgmap v2616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:43.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:43 gibba011 bash[31774]: cluster 2021-03-10T15:19:42.219708+0000 mgr.gibba032.whopeo (mgr.24625) 4602 : cluster [DBG] pgmap v2616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:43 gibba032 bash[24004]: cluster 2021-03-10T15:19:42.219708+0000 mgr.gibba032.whopeo (mgr.24625) 4602 : cluster [DBG] pgmap v2616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:45.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:45 gibba011 bash[30791]: cluster 2021-03-10T15:19:44.221121+0000 mgr.gibba032.whopeo (mgr.24625) 4603 : cluster [DBG] pgmap v2617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:45.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:45 gibba011 bash[31774]: cluster 2021-03-10T15:19:44.221121+0000 mgr.gibba032.whopeo (mgr.24625) 4603 : cluster [DBG] pgmap v2617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:45 gibba032 bash[24004]: cluster 2021-03-10T15:19:44.221121+0000 mgr.gibba032.whopeo (mgr.24625) 4603 : cluster [DBG] pgmap v2617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:47.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:47 gibba011 bash[30791]: cluster 2021-03-10T15:19:46.222773+0000 mgr.gibba032.whopeo (mgr.24625) 4604 : cluster [DBG] pgmap v2618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:47.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:47 gibba011 bash[31774]: cluster 2021-03-10T15:19:46.222773+0000 mgr.gibba032.whopeo (mgr.24625) 4604 : cluster [DBG] pgmap v2618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:47 gibba032 bash[24004]: cluster 2021-03-10T15:19:46.222773+0000 mgr.gibba032.whopeo (mgr.24625) 4604 : cluster [DBG] pgmap v2618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:49.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[30791]: cluster 2021-03-10T15:19:48.224726+0000 mgr.gibba032.whopeo (mgr.24625) 4605 : cluster [DBG] pgmap v2619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:49.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[30791]: cephadm 2021-03-10T15:19:48.965841+0000 mgr.gibba032.whopeo (mgr.24625) 4606 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:19:49.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[30791]: audit 2021-03-10T15:19:48.969175+0000 mon.a (mon.0) 1860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:49.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[31774]: cluster 2021-03-10T15:19:48.224726+0000 mgr.gibba032.whopeo (mgr.24625) 4605 : cluster [DBG] pgmap v2619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:49.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[31774]: cephadm 2021-03-10T15:19:48.965841+0000 mgr.gibba032.whopeo (mgr.24625) 4606 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:19:49.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:49 gibba011 bash[31774]: audit 2021-03-10T15:19:48.969175+0000 mon.a (mon.0) 1860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:49 gibba032 bash[24004]: cluster 2021-03-10T15:19:48.224726+0000 mgr.gibba032.whopeo (mgr.24625) 4605 : cluster [DBG] pgmap v2619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:49 gibba032 bash[24004]: cephadm 2021-03-10T15:19:48.965841+0000 mgr.gibba032.whopeo (mgr.24625) 4606 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:19:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:49 gibba032 bash[24004]: audit 2021-03-10T15:19:48.969175+0000 mon.a (mon.0) 1860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:51.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:51 gibba011 bash[30791]: cluster 2021-03-10T15:19:50.226112+0000 mgr.gibba032.whopeo (mgr.24625) 4607 : cluster [DBG] pgmap v2620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:51.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:51 gibba011 bash[31774]: cluster 2021-03-10T15:19:50.226112+0000 mgr.gibba032.whopeo (mgr.24625) 4607 : cluster [DBG] pgmap v2620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:52.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:51 gibba032 bash[24004]: cluster 2021-03-10T15:19:50.226112+0000 mgr.gibba032.whopeo (mgr.24625) 4607 : cluster [DBG] pgmap v2620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:54.070 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:53 gibba032 bash[24004]: cluster 2021-03-10T15:19:52.226620+0000 mgr.gibba032.whopeo (mgr.24625) 4608 : cluster [DBG] pgmap v2621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:54.071 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:53 gibba032 bash[24004]: audit 2021-03-10T15:19:52.785863+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:54.071 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:53 gibba032 bash[24004]: audit 2021-03-10T15:19:52.786952+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:19:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[30791]: cluster 2021-03-10T15:19:52.226620+0000 mgr.gibba032.whopeo (mgr.24625) 4608 : cluster [DBG] pgmap v2621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[30791]: audit 2021-03-10T15:19:52.785863+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[30791]: audit 2021-03-10T15:19:52.786952+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:19:54.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[31774]: cluster 2021-03-10T15:19:52.226620+0000 mgr.gibba032.whopeo (mgr.24625) 4608 : cluster [DBG] pgmap v2621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:54.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[31774]: audit 2021-03-10T15:19:52.785863+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:54.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:53 gibba011 bash[31774]: audit 2021-03-10T15:19:52.786952+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:19:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:55 gibba011 bash[30791]: cluster 2021-03-10T15:19:54.227214+0000 mgr.gibba032.whopeo (mgr.24625) 4609 : cluster [DBG] pgmap v2622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:56.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:55 gibba011 bash[31774]: cluster 2021-03-10T15:19:54.227214+0000 mgr.gibba032.whopeo (mgr.24625) 4609 : cluster [DBG] pgmap v2622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:55 gibba032 bash[24004]: cluster 2021-03-10T15:19:54.227214+0000 mgr.gibba032.whopeo (mgr.24625) 4609 : cluster [DBG] pgmap v2622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:56.501 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:19:56.729 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:19:56.729 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (86m) 68s ago 91m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:19:56.730 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (91m) 3s ago 91m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:19:56.730 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (86m) 3s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:19:56.730 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (87m) 3s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:19:56.730 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (86m) 68s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:19:56.731 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (86m) 68s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:19:56.731 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (85m) 3s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:19:56.731 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (85m) 68s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:19:56.731 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (92m) 68s ago 92m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:19:56.732 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (92m) 3s ago 92m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:19:56.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (84m) 68s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:19:56.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (84m) 68s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:19:56.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (84m) 68s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:19:56.732 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (83m) 68s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:19:56.733 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (83m) 3s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:19:56.733 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (82m) 3s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:19:56.733 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (82m) 3s ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:19:56.733 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (81m) 3s ago 92m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:19:56.734 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (91m) 3s ago 92m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:19:56.734 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 68s ago 90m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:19:56.982 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:19:56.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:19:56.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:19:56.983 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:56.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:19:56.984 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:19:56.984 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:56.984 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:19:56.984 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:19:56.985 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:19:56.985 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:19:56.985 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:19:56.985 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:19:56.985 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:19:56.986 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.554294+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.554590+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:19:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.566337+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:19:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.566462+0000 mon.a (mon.0) 1866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T15:19:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.566574+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:19:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:56 gibba032 bash[24004]: audit 2021-03-10T15:19:56.566706+0000 mon.a (mon.0) 1868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:19:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.554294+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:19:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.554590+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:19:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.566337+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:19:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.566462+0000 mon.a (mon.0) 1866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T15:19:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.566574+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:19:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[30791]: audit 2021-03-10T15:19:56.566706+0000 mon.a (mon.0) 1868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:19:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.554294+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:19:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.554590+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:19:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.566337+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:19:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.566462+0000 mon.a (mon.0) 1866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T15:19:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.566574+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:19:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:56 gibba011 bash[31774]: audit 2021-03-10T15:19:56.566706+0000 mon.a (mon.0) 1868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:19:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: cluster 2021-03-10T15:19:56.228469+0000 mgr.gibba032.whopeo (mgr.24625) 4610 : cluster [DBG] pgmap v2623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:56.488540+0000 mgr.gibba032.whopeo (mgr.24625) 4611 : audit [DBG] from='client.47151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:56.713778+0000 mgr.gibba032.whopeo (mgr.24625) 4612 : audit [DBG] from='client.57142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:56.979757+0000 mon.a (mon.0) 1869 : audit [DBG] from='client.? 172.21.2.111:0/3513220426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.263939+0000 mon.a (mon.0) 1870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:58.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.430862+0000 mon.a (mon.0) 1871 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.619264+0000 mon.a (mon.0) 1872 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.627785+0000 mon.a (mon.0) 1873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:19:58.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.629472+0000 mon.a (mon.0) 1874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.631072+0000 mon.a (mon.0) 1875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.632268+0000 mon.a (mon.0) 1876 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.634593+0000 mon.a (mon.0) 1877 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:57 gibba032 bash[24004]: audit 2021-03-10T15:19:57.636247+0000 mon.a (mon.0) 1878 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: cluster 2021-03-10T15:19:56.228469+0000 mgr.gibba032.whopeo (mgr.24625) 4610 : cluster [DBG] pgmap v2623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:58.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:56.488540+0000 mgr.gibba032.whopeo (mgr.24625) 4611 : audit [DBG] from='client.47151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:56.713778+0000 mgr.gibba032.whopeo (mgr.24625) 4612 : audit [DBG] from='client.57142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:56.979757+0000 mon.a (mon.0) 1869 : audit [DBG] from='client.? 172.21.2.111:0/3513220426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.263939+0000 mon.a (mon.0) 1870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.430862+0000 mon.a (mon.0) 1871 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.619264+0000 mon.a (mon.0) 1872 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.627785+0000 mon.a (mon.0) 1873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:19:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.629472+0000 mon.a (mon.0) 1874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.631072+0000 mon.a (mon.0) 1875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.632268+0000 mon.a (mon.0) 1876 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.634593+0000 mon.a (mon.0) 1877 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[30791]: audit 2021-03-10T15:19:57.636247+0000 mon.a (mon.0) 1878 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: cluster 2021-03-10T15:19:56.228469+0000 mgr.gibba032.whopeo (mgr.24625) 4610 : cluster [DBG] pgmap v2623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:19:58.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:56.488540+0000 mgr.gibba032.whopeo (mgr.24625) 4611 : audit [DBG] from='client.47151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:56.713778+0000 mgr.gibba032.whopeo (mgr.24625) 4612 : audit [DBG] from='client.57142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:19:58.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:56.979757+0000 mon.a (mon.0) 1869 : audit [DBG] from='client.? 172.21.2.111:0/3513220426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.263939+0000 mon.a (mon.0) 1870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:19:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.430862+0000 mon.a (mon.0) 1871 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.619264+0000 mon.a (mon.0) 1872 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:19:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.627785+0000 mon.a (mon.0) 1873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:19:58.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.629472+0000 mon.a (mon.0) 1874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.631072+0000 mon.a (mon.0) 1875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.632268+0000 mon.a (mon.0) 1876 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.634593+0000 mon.a (mon.0) 1877 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:58.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:57 gibba011 bash[31774]: audit 2021-03-10T15:19:57.636247+0000 mon.a (mon.0) 1878 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:19:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.430211+0000 mgr.gibba032.whopeo (mgr.24625) 4613 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:19:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.625549+0000 mgr.gibba032.whopeo (mgr.24625) 4614 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:19:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.627600+0000 mgr.gibba032.whopeo (mgr.24625) 4615 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:19:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.629170+0000 mgr.gibba032.whopeo (mgr.24625) 4616 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:19:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.629205+0000 mgr.gibba032.whopeo (mgr.24625) 4617 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:19:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.630646+0000 mgr.gibba032.whopeo (mgr.24625) 4618 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:19:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.630683+0000 mgr.gibba032.whopeo (mgr.24625) 4619 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:19:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.631936+0000 mgr.gibba032.whopeo (mgr.24625) 4620 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:19:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.632829+0000 mgr.gibba032.whopeo (mgr.24625) 4621 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:19:59.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.632871+0000 mgr.gibba032.whopeo (mgr.24625) 4622 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:19:59.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.634144+0000 mgr.gibba032.whopeo (mgr.24625) 4623 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:19:59.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.634527+0000 mgr.gibba032.whopeo (mgr.24625) 4624 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:19:59.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.635669+0000 mgr.gibba032.whopeo (mgr.24625) 4625 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:19:59.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.635707+0000 mgr.gibba032.whopeo (mgr.24625) 4626 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:19:59.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[30791]: cephadm 2021-03-10T15:19:57.635816+0000 mgr.gibba032.whopeo (mgr.24625) 4627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.430211+0000 mgr.gibba032.whopeo (mgr.24625) 4613 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:19:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.625549+0000 mgr.gibba032.whopeo (mgr.24625) 4614 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:19:59.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.627600+0000 mgr.gibba032.whopeo (mgr.24625) 4615 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:19:59.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.629170+0000 mgr.gibba032.whopeo (mgr.24625) 4616 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:19:59.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.629205+0000 mgr.gibba032.whopeo (mgr.24625) 4617 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:19:59.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.630646+0000 mgr.gibba032.whopeo (mgr.24625) 4618 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:19:59.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.630683+0000 mgr.gibba032.whopeo (mgr.24625) 4619 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:19:59.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.631936+0000 mgr.gibba032.whopeo (mgr.24625) 4620 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:19:59.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.632829+0000 mgr.gibba032.whopeo (mgr.24625) 4621 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:19:59.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.632871+0000 mgr.gibba032.whopeo (mgr.24625) 4622 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:19:59.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.634144+0000 mgr.gibba032.whopeo (mgr.24625) 4623 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:19:59.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.634527+0000 mgr.gibba032.whopeo (mgr.24625) 4624 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:19:59.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.635669+0000 mgr.gibba032.whopeo (mgr.24625) 4625 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:19:59.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.635707+0000 mgr.gibba032.whopeo (mgr.24625) 4626 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:19:59.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:58 gibba011 bash[31774]: cephadm 2021-03-10T15:19:57.635816+0000 mgr.gibba032.whopeo (mgr.24625) 4627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:19:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.430211+0000 mgr.gibba032.whopeo (mgr.24625) 4613 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:19:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.625549+0000 mgr.gibba032.whopeo (mgr.24625) 4614 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:19:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.627600+0000 mgr.gibba032.whopeo (mgr.24625) 4615 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:19:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.629170+0000 mgr.gibba032.whopeo (mgr.24625) 4616 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:19:59.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.629205+0000 mgr.gibba032.whopeo (mgr.24625) 4617 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:19:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.630646+0000 mgr.gibba032.whopeo (mgr.24625) 4618 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:19:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.630683+0000 mgr.gibba032.whopeo (mgr.24625) 4619 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:19:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.631936+0000 mgr.gibba032.whopeo (mgr.24625) 4620 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:19:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.632829+0000 mgr.gibba032.whopeo (mgr.24625) 4621 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:19:59.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.632871+0000 mgr.gibba032.whopeo (mgr.24625) 4622 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:19:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.634144+0000 mgr.gibba032.whopeo (mgr.24625) 4623 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:19:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.634527+0000 mgr.gibba032.whopeo (mgr.24625) 4624 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:19:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.635669+0000 mgr.gibba032.whopeo (mgr.24625) 4625 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:19:59.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.635707+0000 mgr.gibba032.whopeo (mgr.24625) 4626 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:19:59.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:58 gibba032 bash[24004]: cephadm 2021-03-10T15:19:57.635816+0000 mgr.gibba032.whopeo (mgr.24625) 4627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:19:59 gibba032 bash[24004]: cluster 2021-03-10T15:19:58.230097+0000 mgr.gibba032.whopeo (mgr.24625) 4628 : cluster [DBG] pgmap v2624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:19:59 gibba011 bash[30791]: cluster 2021-03-10T15:19:58.230097+0000 mgr.gibba032.whopeo (mgr.24625) 4628 : cluster [DBG] pgmap v2624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:00 gibba011 bash[30791]: debug 2021-03-10T15:19:59.998+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:20:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:19:59 gibba011 bash[31774]: cluster 2021-03-10T15:19:58.230097+0000 mgr.gibba032.whopeo (mgr.24625) 4628 : cluster [DBG] pgmap v2624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:00 gibba032 bash[24004]: cluster 2021-03-10T15:20:00.000235+0000 mon.a (mon.0) 1879 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:20:01.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:00 gibba011 bash[30791]: cluster 2021-03-10T15:20:00.000235+0000 mon.a (mon.0) 1879 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:20:01.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:00 gibba011 bash[31774]: cluster 2021-03-10T15:20:00.000235+0000 mon.a (mon.0) 1879 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:20:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:01 gibba032 bash[24004]: cluster 2021-03-10T15:20:00.231696+0000 mgr.gibba032.whopeo (mgr.24625) 4629 : cluster [DBG] pgmap v2625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:01 gibba011 bash[30791]: cluster 2021-03-10T15:20:00.231696+0000 mgr.gibba032.whopeo (mgr.24625) 4629 : cluster [DBG] pgmap v2625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:01 gibba011 bash[31774]: cluster 2021-03-10T15:20:00.231696+0000 mgr.gibba032.whopeo (mgr.24625) 4629 : cluster [DBG] pgmap v2625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:03 gibba032 bash[24004]: cluster 2021-03-10T15:20:02.233462+0000 mgr.gibba032.whopeo (mgr.24625) 4630 : cluster [DBG] pgmap v2626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:03 gibba011 bash[30791]: cluster 2021-03-10T15:20:02.233462+0000 mgr.gibba032.whopeo (mgr.24625) 4630 : cluster [DBG] pgmap v2626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:04.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:03 gibba011 bash[31774]: cluster 2021-03-10T15:20:02.233462+0000 mgr.gibba032.whopeo (mgr.24625) 4630 : cluster [DBG] pgmap v2626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:06.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:05 gibba032 bash[24004]: cluster 2021-03-10T15:20:04.234759+0000 mgr.gibba032.whopeo (mgr.24625) 4631 : cluster [DBG] pgmap v2627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:06.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:05 gibba011 bash[30791]: cluster 2021-03-10T15:20:04.234759+0000 mgr.gibba032.whopeo (mgr.24625) 4631 : cluster [DBG] pgmap v2627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:06.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:05 gibba011 bash[31774]: cluster 2021-03-10T15:20:04.234759+0000 mgr.gibba032.whopeo (mgr.24625) 4631 : cluster [DBG] pgmap v2627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:07 gibba032 bash[24004]: cluster 2021-03-10T15:20:06.236272+0000 mgr.gibba032.whopeo (mgr.24625) 4632 : cluster [DBG] pgmap v2628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:07 gibba011 bash[30791]: cluster 2021-03-10T15:20:06.236272+0000 mgr.gibba032.whopeo (mgr.24625) 4632 : cluster [DBG] pgmap v2628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:07 gibba011 bash[31774]: cluster 2021-03-10T15:20:06.236272+0000 mgr.gibba032.whopeo (mgr.24625) 4632 : cluster [DBG] pgmap v2628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:09 gibba032 bash[24004]: cluster 2021-03-10T15:20:08.237935+0000 mgr.gibba032.whopeo (mgr.24625) 4633 : cluster [DBG] pgmap v2629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:09 gibba011 bash[30791]: cluster 2021-03-10T15:20:08.237935+0000 mgr.gibba032.whopeo (mgr.24625) 4633 : cluster [DBG] pgmap v2629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:09 gibba011 bash[31774]: cluster 2021-03-10T15:20:08.237935+0000 mgr.gibba032.whopeo (mgr.24625) 4633 : cluster [DBG] pgmap v2629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:11 gibba032 bash[24004]: cluster 2021-03-10T15:20:10.239600+0000 mgr.gibba032.whopeo (mgr.24625) 4634 : cluster [DBG] pgmap v2630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:11 gibba011 bash[30791]: cluster 2021-03-10T15:20:10.239600+0000 mgr.gibba032.whopeo (mgr.24625) 4634 : cluster [DBG] pgmap v2630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:12.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:11 gibba011 bash[31774]: cluster 2021-03-10T15:20:10.239600+0000 mgr.gibba032.whopeo (mgr.24625) 4634 : cluster [DBG] pgmap v2630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:13 gibba032 bash[24004]: cluster 2021-03-10T15:20:12.241138+0000 mgr.gibba032.whopeo (mgr.24625) 4635 : cluster [DBG] pgmap v2631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:13 gibba032 bash[24004]: cephadm 2021-03-10T15:20:12.640807+0000 mgr.gibba032.whopeo (mgr.24625) 4636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:13 gibba011 bash[30791]: cluster 2021-03-10T15:20:12.241138+0000 mgr.gibba032.whopeo (mgr.24625) 4635 : cluster [DBG] pgmap v2631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:13 gibba011 bash[30791]: cephadm 2021-03-10T15:20:12.640807+0000 mgr.gibba032.whopeo (mgr.24625) 4636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:13 gibba011 bash[31774]: cluster 2021-03-10T15:20:12.241138+0000 mgr.gibba032.whopeo (mgr.24625) 4635 : cluster [DBG] pgmap v2631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:13 gibba011 bash[31774]: cephadm 2021-03-10T15:20:12.640807+0000 mgr.gibba032.whopeo (mgr.24625) 4636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:15 gibba032 bash[24004]: cluster 2021-03-10T15:20:14.242253+0000 mgr.gibba032.whopeo (mgr.24625) 4637 : cluster [DBG] pgmap v2632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:15 gibba011 bash[30791]: cluster 2021-03-10T15:20:14.242253+0000 mgr.gibba032.whopeo (mgr.24625) 4637 : cluster [DBG] pgmap v2632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:15 gibba011 bash[31774]: cluster 2021-03-10T15:20:14.242253+0000 mgr.gibba032.whopeo (mgr.24625) 4637 : cluster [DBG] pgmap v2632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:17 gibba032 bash[24004]: cluster 2021-03-10T15:20:16.243706+0000 mgr.gibba032.whopeo (mgr.24625) 4638 : cluster [DBG] pgmap v2633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:17 gibba011 bash[30791]: cluster 2021-03-10T15:20:16.243706+0000 mgr.gibba032.whopeo (mgr.24625) 4638 : cluster [DBG] pgmap v2633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:17 gibba011 bash[31774]: cluster 2021-03-10T15:20:16.243706+0000 mgr.gibba032.whopeo (mgr.24625) 4638 : cluster [DBG] pgmap v2633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:19 gibba032 bash[24004]: cluster 2021-03-10T15:20:18.245465+0000 mgr.gibba032.whopeo (mgr.24625) 4639 : cluster [DBG] pgmap v2634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:19 gibba011 bash[30791]: cluster 2021-03-10T15:20:18.245465+0000 mgr.gibba032.whopeo (mgr.24625) 4639 : cluster [DBG] pgmap v2634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:19 gibba011 bash[31774]: cluster 2021-03-10T15:20:18.245465+0000 mgr.gibba032.whopeo (mgr.24625) 4639 : cluster [DBG] pgmap v2634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:21 gibba032 bash[24004]: cluster 2021-03-10T15:20:20.247004+0000 mgr.gibba032.whopeo (mgr.24625) 4640 : cluster [DBG] pgmap v2635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:21 gibba011 bash[30791]: cluster 2021-03-10T15:20:20.247004+0000 mgr.gibba032.whopeo (mgr.24625) 4640 : cluster [DBG] pgmap v2635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:21 gibba011 bash[31774]: cluster 2021-03-10T15:20:20.247004+0000 mgr.gibba032.whopeo (mgr.24625) 4640 : cluster [DBG] pgmap v2635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:23 gibba032 bash[24004]: cluster 2021-03-10T15:20:22.248785+0000 mgr.gibba032.whopeo (mgr.24625) 4641 : cluster [DBG] pgmap v2636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:23 gibba011 bash[30791]: cluster 2021-03-10T15:20:22.248785+0000 mgr.gibba032.whopeo (mgr.24625) 4641 : cluster [DBG] pgmap v2636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:23 gibba011 bash[31774]: cluster 2021-03-10T15:20:22.248785+0000 mgr.gibba032.whopeo (mgr.24625) 4641 : cluster [DBG] pgmap v2636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:25 gibba032 bash[24004]: cluster 2021-03-10T15:20:24.250220+0000 mgr.gibba032.whopeo (mgr.24625) 4642 : cluster [DBG] pgmap v2637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:26.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:25 gibba011 bash[30791]: cluster 2021-03-10T15:20:24.250220+0000 mgr.gibba032.whopeo (mgr.24625) 4642 : cluster [DBG] pgmap v2637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:25 gibba011 bash[31774]: cluster 2021-03-10T15:20:24.250220+0000 mgr.gibba032.whopeo (mgr.24625) 4642 : cluster [DBG] pgmap v2637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:27.221 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:20:27.451 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:20:27.452 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (87m) 98s ago 92m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:20:27.452 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (92m) 34s ago 92m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:20:27.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (87m) 34s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:20:27.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (87m) 34s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:20:27.453 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (86m) 98s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:20:27.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (86m) 98s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:20:27.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (86m) 34s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:20:27.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (86m) 98s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:20:27.454 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (92m) 98s ago 92m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:20:27.454 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (92m) 34s ago 92m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:20:27.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (85m) 98s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:20:27.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (84m) 98s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:20:27.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (84m) 98s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:20:27.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (84m) 98s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:20:27.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (83m) 34s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:20:27.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (83m) 34s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:20:27.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (82m) 34s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:20:27.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (82m) 34s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:20:27.456 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (92m) 34s ago 92m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:20:27.456 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 98s ago 90m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:20:27.705 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:20:27.706 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:20:27.706 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:20:27.707 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:27.707 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:20:27.708 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:20:27.708 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:27.709 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:20:27.709 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:20:27.709 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:27.710 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:20:27.710 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:20:27.711 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:20:27.711 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:20:27.712 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:20:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:27 gibba032 bash[24004]: cluster 2021-03-10T15:20:26.251898+0000 mgr.gibba032.whopeo (mgr.24625) 4643 : cluster [DBG] pgmap v2638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:27 gibba032 bash[24004]: audit 2021-03-10T15:20:27.208185+0000 mgr.gibba032.whopeo (mgr.24625) 4644 : audit [DBG] from='client.57154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:28.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:27 gibba032 bash[24004]: audit 2021-03-10T15:20:27.702371+0000 mon.c (mon.1) 88 : audit [DBG] from='client.? 172.21.2.111:0/460813954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[30791]: cluster 2021-03-10T15:20:26.251898+0000 mgr.gibba032.whopeo (mgr.24625) 4643 : cluster [DBG] pgmap v2638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[30791]: audit 2021-03-10T15:20:27.208185+0000 mgr.gibba032.whopeo (mgr.24625) 4644 : audit [DBG] from='client.57154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:28.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[30791]: audit 2021-03-10T15:20:27.702371+0000 mon.c (mon.1) 88 : audit [DBG] from='client.? 172.21.2.111:0/460813954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[31774]: cluster 2021-03-10T15:20:26.251898+0000 mgr.gibba032.whopeo (mgr.24625) 4643 : cluster [DBG] pgmap v2638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:28.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[31774]: audit 2021-03-10T15:20:27.208185+0000 mgr.gibba032.whopeo (mgr.24625) 4644 : audit [DBG] from='client.57154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:28.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:27 gibba011 bash[31774]: audit 2021-03-10T15:20:27.702371+0000 mon.c (mon.1) 88 : audit [DBG] from='client.? 172.21.2.111:0/460813954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:29.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:28 gibba032 bash[24004]: audit 2021-03-10T15:20:27.434647+0000 mgr.gibba032.whopeo (mgr.24625) 4645 : audit [DBG] from='client.47175 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:29.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:28 gibba032 bash[24004]: cephadm 2021-03-10T15:20:27.656831+0000 mgr.gibba032.whopeo (mgr.24625) 4646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:29.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:28 gibba011 bash[30791]: audit 2021-03-10T15:20:27.434647+0000 mgr.gibba032.whopeo (mgr.24625) 4645 : audit [DBG] from='client.47175 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:29.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:28 gibba011 bash[30791]: cephadm 2021-03-10T15:20:27.656831+0000 mgr.gibba032.whopeo (mgr.24625) 4646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:29.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:28 gibba011 bash[31774]: audit 2021-03-10T15:20:27.434647+0000 mgr.gibba032.whopeo (mgr.24625) 4645 : audit [DBG] from='client.47175 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:29.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:28 gibba011 bash[31774]: cephadm 2021-03-10T15:20:27.656831+0000 mgr.gibba032.whopeo (mgr.24625) 4646 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:29 gibba032 bash[24004]: cluster 2021-03-10T15:20:28.253637+0000 mgr.gibba032.whopeo (mgr.24625) 4647 : cluster [DBG] pgmap v2639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:29 gibba011 bash[30791]: cluster 2021-03-10T15:20:28.253637+0000 mgr.gibba032.whopeo (mgr.24625) 4647 : cluster [DBG] pgmap v2639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:29 gibba011 bash[31774]: cluster 2021-03-10T15:20:28.253637+0000 mgr.gibba032.whopeo (mgr.24625) 4647 : cluster [DBG] pgmap v2639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:31 gibba032 bash[24004]: cluster 2021-03-10T15:20:30.255183+0000 mgr.gibba032.whopeo (mgr.24625) 4648 : cluster [DBG] pgmap v2640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:31 gibba011 bash[30791]: cluster 2021-03-10T15:20:30.255183+0000 mgr.gibba032.whopeo (mgr.24625) 4648 : cluster [DBG] pgmap v2640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:31 gibba011 bash[31774]: cluster 2021-03-10T15:20:30.255183+0000 mgr.gibba032.whopeo (mgr.24625) 4648 : cluster [DBG] pgmap v2640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:33 gibba032 bash[24004]: cluster 2021-03-10T15:20:32.257058+0000 mgr.gibba032.whopeo (mgr.24625) 4649 : cluster [DBG] pgmap v2641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:33 gibba011 bash[30791]: cluster 2021-03-10T15:20:32.257058+0000 mgr.gibba032.whopeo (mgr.24625) 4649 : cluster [DBG] pgmap v2641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:33 gibba011 bash[31774]: cluster 2021-03-10T15:20:32.257058+0000 mgr.gibba032.whopeo (mgr.24625) 4649 : cluster [DBG] pgmap v2641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:36 gibba011 bash[30791]: cluster 2021-03-10T15:20:34.258245+0000 mgr.gibba032.whopeo (mgr.24625) 4650 : cluster [DBG] pgmap v2642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:36.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:36 gibba011 bash[31774]: cluster 2021-03-10T15:20:34.258245+0000 mgr.gibba032.whopeo (mgr.24625) 4650 : cluster [DBG] pgmap v2642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:36 gibba032 bash[24004]: cluster 2021-03-10T15:20:34.258245+0000 mgr.gibba032.whopeo (mgr.24625) 4650 : cluster [DBG] pgmap v2642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:38 gibba011 bash[30791]: cluster 2021-03-10T15:20:36.259978+0000 mgr.gibba032.whopeo (mgr.24625) 4651 : cluster [DBG] pgmap v2643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:38.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:38 gibba011 bash[31774]: cluster 2021-03-10T15:20:36.259978+0000 mgr.gibba032.whopeo (mgr.24625) 4651 : cluster [DBG] pgmap v2643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:38 gibba032 bash[24004]: cluster 2021-03-10T15:20:36.259978+0000 mgr.gibba032.whopeo (mgr.24625) 4651 : cluster [DBG] pgmap v2643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:40 gibba011 bash[30791]: cluster 2021-03-10T15:20:38.261835+0000 mgr.gibba032.whopeo (mgr.24625) 4652 : cluster [DBG] pgmap v2644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:40 gibba011 bash[31774]: cluster 2021-03-10T15:20:38.261835+0000 mgr.gibba032.whopeo (mgr.24625) 4652 : cluster [DBG] pgmap v2644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:40 gibba032 bash[24004]: cluster 2021-03-10T15:20:38.261835+0000 mgr.gibba032.whopeo (mgr.24625) 4652 : cluster [DBG] pgmap v2644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:42 gibba011 bash[30791]: cluster 2021-03-10T15:20:40.263471+0000 mgr.gibba032.whopeo (mgr.24625) 4653 : cluster [DBG] pgmap v2645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:42 gibba011 bash[31774]: cluster 2021-03-10T15:20:40.263471+0000 mgr.gibba032.whopeo (mgr.24625) 4653 : cluster [DBG] pgmap v2645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:42 gibba032 bash[24004]: cluster 2021-03-10T15:20:40.263471+0000 mgr.gibba032.whopeo (mgr.24625) 4653 : cluster [DBG] pgmap v2645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:44 gibba011 bash[30791]: cluster 2021-03-10T15:20:42.265302+0000 mgr.gibba032.whopeo (mgr.24625) 4654 : cluster [DBG] pgmap v2646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:44 gibba011 bash[30791]: cephadm 2021-03-10T15:20:42.672883+0000 mgr.gibba032.whopeo (mgr.24625) 4655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:44 gibba011 bash[31774]: cluster 2021-03-10T15:20:42.265302+0000 mgr.gibba032.whopeo (mgr.24625) 4654 : cluster [DBG] pgmap v2646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:44 gibba011 bash[31774]: cephadm 2021-03-10T15:20:42.672883+0000 mgr.gibba032.whopeo (mgr.24625) 4655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:44 gibba032 bash[24004]: cluster 2021-03-10T15:20:42.265302+0000 mgr.gibba032.whopeo (mgr.24625) 4654 : cluster [DBG] pgmap v2646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:44 gibba032 bash[24004]: cephadm 2021-03-10T15:20:42.672883+0000 mgr.gibba032.whopeo (mgr.24625) 4655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:46 gibba011 bash[30791]: cluster 2021-03-10T15:20:44.266358+0000 mgr.gibba032.whopeo (mgr.24625) 4656 : cluster [DBG] pgmap v2647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:46 gibba011 bash[31774]: cluster 2021-03-10T15:20:44.266358+0000 mgr.gibba032.whopeo (mgr.24625) 4656 : cluster [DBG] pgmap v2647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:46 gibba032 bash[24004]: cluster 2021-03-10T15:20:44.266358+0000 mgr.gibba032.whopeo (mgr.24625) 4656 : cluster [DBG] pgmap v2647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:48 gibba011 bash[30791]: cluster 2021-03-10T15:20:46.268058+0000 mgr.gibba032.whopeo (mgr.24625) 4657 : cluster [DBG] pgmap v2648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:48 gibba011 bash[31774]: cluster 2021-03-10T15:20:46.268058+0000 mgr.gibba032.whopeo (mgr.24625) 4657 : cluster [DBG] pgmap v2648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:48 gibba032 bash[24004]: cluster 2021-03-10T15:20:46.268058+0000 mgr.gibba032.whopeo (mgr.24625) 4657 : cluster [DBG] pgmap v2648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:50 gibba011 bash[30791]: cluster 2021-03-10T15:20:48.269504+0000 mgr.gibba032.whopeo (mgr.24625) 4658 : cluster [DBG] pgmap v2649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:50 gibba011 bash[31774]: cluster 2021-03-10T15:20:48.269504+0000 mgr.gibba032.whopeo (mgr.24625) 4658 : cluster [DBG] pgmap v2649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:50 gibba032 bash[24004]: cluster 2021-03-10T15:20:48.269504+0000 mgr.gibba032.whopeo (mgr.24625) 4658 : cluster [DBG] pgmap v2649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:52 gibba011 bash[30791]: cluster 2021-03-10T15:20:50.271061+0000 mgr.gibba032.whopeo (mgr.24625) 4659 : cluster [DBG] pgmap v2650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:52 gibba011 bash[31774]: cluster 2021-03-10T15:20:50.271061+0000 mgr.gibba032.whopeo (mgr.24625) 4659 : cluster [DBG] pgmap v2650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:52 gibba032 bash[24004]: cluster 2021-03-10T15:20:50.271061+0000 mgr.gibba032.whopeo (mgr.24625) 4659 : cluster [DBG] pgmap v2650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:54 gibba011 bash[30791]: cluster 2021-03-10T15:20:52.272711+0000 mgr.gibba032.whopeo (mgr.24625) 4660 : cluster [DBG] pgmap v2651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:54 gibba011 bash[31774]: cluster 2021-03-10T15:20:52.272711+0000 mgr.gibba032.whopeo (mgr.24625) 4660 : cluster [DBG] pgmap v2651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:54 gibba032 bash[24004]: cluster 2021-03-10T15:20:52.272711+0000 mgr.gibba032.whopeo (mgr.24625) 4660 : cluster [DBG] pgmap v2651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:56 gibba032 bash[24004]: cluster 2021-03-10T15:20:54.273983+0000 mgr.gibba032.whopeo (mgr.24625) 4661 : cluster [DBG] pgmap v2652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:56 gibba011 bash[30791]: cluster 2021-03-10T15:20:54.273983+0000 mgr.gibba032.whopeo (mgr.24625) 4661 : cluster [DBG] pgmap v2652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:56 gibba011 bash[31774]: cluster 2021-03-10T15:20:54.273983+0000 mgr.gibba032.whopeo (mgr.24625) 4661 : cluster [DBG] pgmap v2652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.557100+0000 mon.a (mon.0) 1880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:20:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.557788+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:20:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.569809+0000 mon.a (mon.0) 1882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:20:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.570556+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.571245+0000 mon.a (mon.0) 1884 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:57 gibba032 bash[24004]: audit 2021-03-10T15:20:56.571983+0000 mon.a (mon.0) 1885 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:20:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.557100+0000 mon.a (mon.0) 1880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:20:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.557788+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:20:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.569809+0000 mon.a (mon.0) 1882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:20:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.570556+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.571245+0000 mon.a (mon.0) 1884 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[30791]: audit 2021-03-10T15:20:56.571983+0000 mon.a (mon.0) 1885 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:20:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.557100+0000 mon.a (mon.0) 1880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:20:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.557788+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:20:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.569809+0000 mon.a (mon.0) 1882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:20:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.570556+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.571245+0000 mon.a (mon.0) 1884 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:20:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:57 gibba011 bash[31774]: audit 2021-03-10T15:20:56.571983+0000 mon.a (mon.0) 1885 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:20:57.951 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:20:58.212 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:20:58.212 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (87m) 2m ago 92m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:20:58.212 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (92m) 65s ago 92m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:20:58.213 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (87m) 65s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:20:58.213 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (88m) 65s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:20:58.213 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (87m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:20:58.213 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (87m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:20:58.214 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (86m) 65s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:20:58.214 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (86m) 2m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:20:58.214 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (93m) 2m ago 93m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:20:58.214 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (93m) 65s ago 93m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:20:58.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (85m) 2m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:20:58.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:20:58.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:20:58.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (84m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:20:58.215 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (84m) 65s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:20:58.216 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (83m) 65s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:20:58.216 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (83m) 65s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:20:58.216 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (82m) 65s ago 93m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:20:58.216 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (92m) 65s ago 93m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:20:58.217 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 91m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:20:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: cluster 2021-03-10T15:20:56.275685+0000 mgr.gibba032.whopeo (mgr.24625) 4662 : cluster [DBG] pgmap v2653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.917894+0000 mon.a (mon.0) 1886 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:20:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.921188+0000 mon.a (mon.0) 1887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:20:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.922651+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.923907+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.925051+0000 mon.a (mon.0) 1890 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.927488+0000 mon.a (mon.0) 1891 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:58 gibba032 bash[24004]: audit 2021-03-10T15:20:57.928982+0000 mon.a (mon.0) 1892 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.466 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:20:58.466 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:20:58.466 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:20:58.466 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:58.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:20:58.467 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:20:58.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:58.467 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:20:58.468 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:20:58.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:20:58.470 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:20:58.470 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:20:58.470 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:20:58.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:20:58.471 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:20:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: cluster 2021-03-10T15:20:56.275685+0000 mgr.gibba032.whopeo (mgr.24625) 4662 : cluster [DBG] pgmap v2653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.917894+0000 mon.a (mon.0) 1886 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:20:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.921188+0000 mon.a (mon.0) 1887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:20:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.922651+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.923907+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.925051+0000 mon.a (mon.0) 1890 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.927488+0000 mon.a (mon.0) 1891 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[30791]: audit 2021-03-10T15:20:57.928982+0000 mon.a (mon.0) 1892 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: cluster 2021-03-10T15:20:56.275685+0000 mgr.gibba032.whopeo (mgr.24625) 4662 : cluster [DBG] pgmap v2653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:20:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.917894+0000 mon.a (mon.0) 1886 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:20:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.921188+0000 mon.a (mon.0) 1887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:20:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.922651+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.923907+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.925051+0000 mon.a (mon.0) 1890 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.927488+0000 mon.a (mon.0) 1891 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:58 gibba011 bash[31774]: audit 2021-03-10T15:20:57.928982+0000 mon.a (mon.0) 1892 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:59.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.689651+0000 mgr.gibba032.whopeo (mgr.24625) 4663 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:20:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.691198+0000 mgr.gibba032.whopeo (mgr.24625) 4664 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:20:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.919205+0000 mgr.gibba032.whopeo (mgr.24625) 4665 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:20:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.921082+0000 mgr.gibba032.whopeo (mgr.24625) 4666 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:20:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.922305+0000 mgr.gibba032.whopeo (mgr.24625) 4667 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:20:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.922343+0000 mgr.gibba032.whopeo (mgr.24625) 4668 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:20:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.923524+0000 mgr.gibba032.whopeo (mgr.24625) 4669 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:20:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.923562+0000 mgr.gibba032.whopeo (mgr.24625) 4670 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:20:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.924650+0000 mgr.gibba032.whopeo (mgr.24625) 4671 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:20:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.925816+0000 mgr.gibba032.whopeo (mgr.24625) 4672 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:20:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.925856+0000 mgr.gibba032.whopeo (mgr.24625) 4673 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:20:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.927125+0000 mgr.gibba032.whopeo (mgr.24625) 4674 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:20:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.927510+0000 mgr.gibba032.whopeo (mgr.24625) 4675 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:20:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.928725+0000 mgr.gibba032.whopeo (mgr.24625) 4676 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:20:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.928769+0000 mgr.gibba032.whopeo (mgr.24625) 4677 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:20:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: cephadm 2021-03-10T15:20:57.928887+0000 mgr.gibba032.whopeo (mgr.24625) 4678 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: audit 2021-03-10T15:20:57.939377+0000 mgr.gibba032.whopeo (mgr.24625) 4679 : audit [DBG] from='client.57172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: audit 2021-03-10T15:20:58.168103+0000 mgr.gibba032.whopeo (mgr.24625) 4680 : audit [DBG] from='client.47193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:20:59 gibba032 bash[24004]: audit 2021-03-10T15:20:58.463050+0000 mon.c (mon.1) 89 : audit [DBG] from='client.? 172.21.2.111:0/1037936219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.689651+0000 mgr.gibba032.whopeo (mgr.24625) 4663 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:20:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.691198+0000 mgr.gibba032.whopeo (mgr.24625) 4664 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:20:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.919205+0000 mgr.gibba032.whopeo (mgr.24625) 4665 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:20:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.921082+0000 mgr.gibba032.whopeo (mgr.24625) 4666 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:20:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.922305+0000 mgr.gibba032.whopeo (mgr.24625) 4667 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:20:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.922343+0000 mgr.gibba032.whopeo (mgr.24625) 4668 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:20:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.923524+0000 mgr.gibba032.whopeo (mgr.24625) 4669 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:20:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.923562+0000 mgr.gibba032.whopeo (mgr.24625) 4670 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:20:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.924650+0000 mgr.gibba032.whopeo (mgr.24625) 4671 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:20:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.925816+0000 mgr.gibba032.whopeo (mgr.24625) 4672 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:20:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.925856+0000 mgr.gibba032.whopeo (mgr.24625) 4673 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:20:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.927125+0000 mgr.gibba032.whopeo (mgr.24625) 4674 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:20:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.927510+0000 mgr.gibba032.whopeo (mgr.24625) 4675 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:20:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.928725+0000 mgr.gibba032.whopeo (mgr.24625) 4676 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:20:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.928769+0000 mgr.gibba032.whopeo (mgr.24625) 4677 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:20:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: cephadm 2021-03-10T15:20:57.928887+0000 mgr.gibba032.whopeo (mgr.24625) 4678 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: audit 2021-03-10T15:20:57.939377+0000 mgr.gibba032.whopeo (mgr.24625) 4679 : audit [DBG] from='client.57172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: audit 2021-03-10T15:20:58.168103+0000 mgr.gibba032.whopeo (mgr.24625) 4680 : audit [DBG] from='client.47193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[30791]: audit 2021-03-10T15:20:58.463050+0000 mon.c (mon.1) 89 : audit [DBG] from='client.? 172.21.2.111:0/1037936219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:20:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.689651+0000 mgr.gibba032.whopeo (mgr.24625) 4663 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:20:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.691198+0000 mgr.gibba032.whopeo (mgr.24625) 4664 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:20:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.919205+0000 mgr.gibba032.whopeo (mgr.24625) 4665 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:20:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.921082+0000 mgr.gibba032.whopeo (mgr.24625) 4666 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:20:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.922305+0000 mgr.gibba032.whopeo (mgr.24625) 4667 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:20:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.922343+0000 mgr.gibba032.whopeo (mgr.24625) 4668 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:20:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.923524+0000 mgr.gibba032.whopeo (mgr.24625) 4669 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:20:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.923562+0000 mgr.gibba032.whopeo (mgr.24625) 4670 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:20:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.924650+0000 mgr.gibba032.whopeo (mgr.24625) 4671 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:20:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.925816+0000 mgr.gibba032.whopeo (mgr.24625) 4672 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:20:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.925856+0000 mgr.gibba032.whopeo (mgr.24625) 4673 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:20:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.927125+0000 mgr.gibba032.whopeo (mgr.24625) 4674 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:20:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.927510+0000 mgr.gibba032.whopeo (mgr.24625) 4675 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:20:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.928725+0000 mgr.gibba032.whopeo (mgr.24625) 4676 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:20:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.928769+0000 mgr.gibba032.whopeo (mgr.24625) 4677 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:20:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: cephadm 2021-03-10T15:20:57.928887+0000 mgr.gibba032.whopeo (mgr.24625) 4678 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:20:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: audit 2021-03-10T15:20:57.939377+0000 mgr.gibba032.whopeo (mgr.24625) 4679 : audit [DBG] from='client.57172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: audit 2021-03-10T15:20:58.168103+0000 mgr.gibba032.whopeo (mgr.24625) 4680 : audit [DBG] from='client.47193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:20:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:20:59 gibba011 bash[31774]: audit 2021-03-10T15:20:58.463050+0000 mon.c (mon.1) 89 : audit [DBG] from='client.? 172.21.2.111:0/1037936219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:00 gibba032 bash[24004]: cluster 2021-03-10T15:20:58.277271+0000 mgr.gibba032.whopeo (mgr.24625) 4681 : cluster [DBG] pgmap v2654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:00 gibba011 bash[30791]: cluster 2021-03-10T15:20:58.277271+0000 mgr.gibba032.whopeo (mgr.24625) 4681 : cluster [DBG] pgmap v2654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:00 gibba011 bash[31774]: cluster 2021-03-10T15:20:58.277271+0000 mgr.gibba032.whopeo (mgr.24625) 4681 : cluster [DBG] pgmap v2654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:02 gibba032 bash[24004]: cluster 2021-03-10T15:21:00.278867+0000 mgr.gibba032.whopeo (mgr.24625) 4682 : cluster [DBG] pgmap v2655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:02 gibba011 bash[30791]: cluster 2021-03-10T15:21:00.278867+0000 mgr.gibba032.whopeo (mgr.24625) 4682 : cluster [DBG] pgmap v2655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:02 gibba011 bash[31774]: cluster 2021-03-10T15:21:00.278867+0000 mgr.gibba032.whopeo (mgr.24625) 4682 : cluster [DBG] pgmap v2655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:04 gibba032 bash[24004]: cluster 2021-03-10T15:21:02.280520+0000 mgr.gibba032.whopeo (mgr.24625) 4683 : cluster [DBG] pgmap v2656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:04 gibba011 bash[30791]: cluster 2021-03-10T15:21:02.280520+0000 mgr.gibba032.whopeo (mgr.24625) 4683 : cluster [DBG] pgmap v2656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:04 gibba011 bash[31774]: cluster 2021-03-10T15:21:02.280520+0000 mgr.gibba032.whopeo (mgr.24625) 4683 : cluster [DBG] pgmap v2656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:06 gibba032 bash[24004]: cluster 2021-03-10T15:21:04.281923+0000 mgr.gibba032.whopeo (mgr.24625) 4684 : cluster [DBG] pgmap v2657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:06 gibba011 bash[30791]: cluster 2021-03-10T15:21:04.281923+0000 mgr.gibba032.whopeo (mgr.24625) 4684 : cluster [DBG] pgmap v2657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:06 gibba011 bash[31774]: cluster 2021-03-10T15:21:04.281923+0000 mgr.gibba032.whopeo (mgr.24625) 4684 : cluster [DBG] pgmap v2657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:08 gibba032 bash[24004]: cluster 2021-03-10T15:21:06.284300+0000 mgr.gibba032.whopeo (mgr.24625) 4685 : cluster [DBG] pgmap v2658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:08 gibba011 bash[30791]: cluster 2021-03-10T15:21:06.284300+0000 mgr.gibba032.whopeo (mgr.24625) 4685 : cluster [DBG] pgmap v2658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:08 gibba011 bash[31774]: cluster 2021-03-10T15:21:06.284300+0000 mgr.gibba032.whopeo (mgr.24625) 4685 : cluster [DBG] pgmap v2658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:10 gibba032 bash[24004]: cluster 2021-03-10T15:21:08.286076+0000 mgr.gibba032.whopeo (mgr.24625) 4686 : cluster [DBG] pgmap v2659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:10 gibba011 bash[30791]: cluster 2021-03-10T15:21:08.286076+0000 mgr.gibba032.whopeo (mgr.24625) 4686 : cluster [DBG] pgmap v2659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:10 gibba011 bash[31774]: cluster 2021-03-10T15:21:08.286076+0000 mgr.gibba032.whopeo (mgr.24625) 4686 : cluster [DBG] pgmap v2659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:12 gibba032 bash[24004]: cluster 2021-03-10T15:21:10.287405+0000 mgr.gibba032.whopeo (mgr.24625) 4687 : cluster [DBG] pgmap v2660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:12 gibba011 bash[30791]: cluster 2021-03-10T15:21:10.287405+0000 mgr.gibba032.whopeo (mgr.24625) 4687 : cluster [DBG] pgmap v2660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:12 gibba011 bash[31774]: cluster 2021-03-10T15:21:10.287405+0000 mgr.gibba032.whopeo (mgr.24625) 4687 : cluster [DBG] pgmap v2660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:14 gibba032 bash[24004]: cluster 2021-03-10T15:21:12.288539+0000 mgr.gibba032.whopeo (mgr.24625) 4688 : cluster [DBG] pgmap v2661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:14 gibba032 bash[24004]: cephadm 2021-03-10T15:21:12.944885+0000 mgr.gibba032.whopeo (mgr.24625) 4689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:14 gibba011 bash[30791]: cluster 2021-03-10T15:21:12.288539+0000 mgr.gibba032.whopeo (mgr.24625) 4688 : cluster [DBG] pgmap v2661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:14 gibba011 bash[30791]: cephadm 2021-03-10T15:21:12.944885+0000 mgr.gibba032.whopeo (mgr.24625) 4689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:14 gibba011 bash[31774]: cluster 2021-03-10T15:21:12.288539+0000 mgr.gibba032.whopeo (mgr.24625) 4688 : cluster [DBG] pgmap v2661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:14 gibba011 bash[31774]: cephadm 2021-03-10T15:21:12.944885+0000 mgr.gibba032.whopeo (mgr.24625) 4689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:16 gibba032 bash[24004]: cluster 2021-03-10T15:21:14.288865+0000 mgr.gibba032.whopeo (mgr.24625) 4690 : cluster [DBG] pgmap v2662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:16 gibba011 bash[30791]: cluster 2021-03-10T15:21:14.288865+0000 mgr.gibba032.whopeo (mgr.24625) 4690 : cluster [DBG] pgmap v2662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:16 gibba011 bash[31774]: cluster 2021-03-10T15:21:14.288865+0000 mgr.gibba032.whopeo (mgr.24625) 4690 : cluster [DBG] pgmap v2662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:18 gibba032 bash[24004]: cluster 2021-03-10T15:21:16.289879+0000 mgr.gibba032.whopeo (mgr.24625) 4691 : cluster [DBG] pgmap v2663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:18 gibba011 bash[30791]: cluster 2021-03-10T15:21:16.289879+0000 mgr.gibba032.whopeo (mgr.24625) 4691 : cluster [DBG] pgmap v2663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:18 gibba011 bash[31774]: cluster 2021-03-10T15:21:16.289879+0000 mgr.gibba032.whopeo (mgr.24625) 4691 : cluster [DBG] pgmap v2663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:20 gibba032 bash[24004]: cluster 2021-03-10T15:21:18.290758+0000 mgr.gibba032.whopeo (mgr.24625) 4692 : cluster [DBG] pgmap v2664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:20 gibba011 bash[30791]: cluster 2021-03-10T15:21:18.290758+0000 mgr.gibba032.whopeo (mgr.24625) 4692 : cluster [DBG] pgmap v2664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:20 gibba011 bash[31774]: cluster 2021-03-10T15:21:18.290758+0000 mgr.gibba032.whopeo (mgr.24625) 4692 : cluster [DBG] pgmap v2664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:22 gibba032 bash[24004]: cluster 2021-03-10T15:21:20.291404+0000 mgr.gibba032.whopeo (mgr.24625) 4693 : cluster [DBG] pgmap v2665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:22 gibba011 bash[30791]: cluster 2021-03-10T15:21:20.291404+0000 mgr.gibba032.whopeo (mgr.24625) 4693 : cluster [DBG] pgmap v2665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:22 gibba011 bash[31774]: cluster 2021-03-10T15:21:20.291404+0000 mgr.gibba032.whopeo (mgr.24625) 4693 : cluster [DBG] pgmap v2665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:24 gibba032 bash[24004]: cluster 2021-03-10T15:21:22.292106+0000 mgr.gibba032.whopeo (mgr.24625) 4694 : cluster [DBG] pgmap v2666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:24 gibba011 bash[30791]: cluster 2021-03-10T15:21:22.292106+0000 mgr.gibba032.whopeo (mgr.24625) 4694 : cluster [DBG] pgmap v2666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:24 gibba011 bash[31774]: cluster 2021-03-10T15:21:22.292106+0000 mgr.gibba032.whopeo (mgr.24625) 4694 : cluster [DBG] pgmap v2666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:26 gibba032 bash[24004]: cluster 2021-03-10T15:21:24.292923+0000 mgr.gibba032.whopeo (mgr.24625) 4695 : cluster [DBG] pgmap v2667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:26 gibba011 bash[30791]: cluster 2021-03-10T15:21:24.292923+0000 mgr.gibba032.whopeo (mgr.24625) 4695 : cluster [DBG] pgmap v2667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:26 gibba011 bash[31774]: cluster 2021-03-10T15:21:24.292923+0000 mgr.gibba032.whopeo (mgr.24625) 4695 : cluster [DBG] pgmap v2667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:28 gibba032 bash[24004]: cluster 2021-03-10T15:21:26.293822+0000 mgr.gibba032.whopeo (mgr.24625) 4696 : cluster [DBG] pgmap v2668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:28 gibba011 bash[30791]: cluster 2021-03-10T15:21:26.293822+0000 mgr.gibba032.whopeo (mgr.24625) 4696 : cluster [DBG] pgmap v2668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:28 gibba011 bash[31774]: cluster 2021-03-10T15:21:26.293822+0000 mgr.gibba032.whopeo (mgr.24625) 4696 : cluster [DBG] pgmap v2668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:28.715 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:21:28.946 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:21:28.946 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (88m) 2m ago 93m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:21:28.946 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (93m) 96s ago 93m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:21:28.946 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (88m) 96s ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:21:28.947 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (88m) 96s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:21:28.947 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (87m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:21:28.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (87m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:21:28.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (87m) 96s ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:21:28.948 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (87m) 2m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:21:28.948 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (93m) 2m ago 93m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:21:28.948 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (93m) 96s ago 93m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:21:28.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (86m) 2m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:21:28.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (85m) 2m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:21:28.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:21:28.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:21:28.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (84m) 96s ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:21:28.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (84m) 96s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:21:28.950 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (83m) 96s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:21:28.950 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (83m) 96s ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:21:28.950 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (93m) 96s ago 93m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:21:28.950 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 91m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:21:29.197 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:21:29.197 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:21:29.198 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:21:29.198 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:29.198 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:21:29.198 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:21:29.199 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:29.199 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:21:29.199 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:21:29.199 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:29.199 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:21:29.200 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:21:29.200 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:21:29.200 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:21:29.200 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:21:29.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:29 gibba032 bash[24004]: cephadm 2021-03-10T15:21:27.960887+0000 mgr.gibba032.whopeo (mgr.24625) 4697 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:29.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:29 gibba032 bash[24004]: audit 2021-03-10T15:21:29.194473+0000 mon.a (mon.0) 1893 : audit [DBG] from='client.? 172.21.2.111:0/4081810320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:29 gibba011 bash[30791]: cephadm 2021-03-10T15:21:27.960887+0000 mgr.gibba032.whopeo (mgr.24625) 4697 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:29 gibba011 bash[30791]: audit 2021-03-10T15:21:29.194473+0000 mon.a (mon.0) 1893 : audit [DBG] from='client.? 172.21.2.111:0/4081810320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:29.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:29 gibba011 bash[31774]: cephadm 2021-03-10T15:21:27.960887+0000 mgr.gibba032.whopeo (mgr.24625) 4697 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:29.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:29 gibba011 bash[31774]: audit 2021-03-10T15:21:29.194473+0000 mon.a (mon.0) 1893 : audit [DBG] from='client.? 172.21.2.111:0/4081810320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[30791]: cluster 2021-03-10T15:21:28.295496+0000 mgr.gibba032.whopeo (mgr.24625) 4698 : cluster [DBG] pgmap v2669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:30.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[30791]: audit 2021-03-10T15:21:28.702134+0000 mgr.gibba032.whopeo (mgr.24625) 4699 : audit [DBG] from='client.47205 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:30.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[30791]: audit 2021-03-10T15:21:28.929615+0000 mgr.gibba032.whopeo (mgr.24625) 4700 : audit [DBG] from='client.57196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[31774]: cluster 2021-03-10T15:21:28.295496+0000 mgr.gibba032.whopeo (mgr.24625) 4698 : cluster [DBG] pgmap v2669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[31774]: audit 2021-03-10T15:21:28.702134+0000 mgr.gibba032.whopeo (mgr.24625) 4699 : audit [DBG] from='client.47205 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:30 gibba011 bash[31774]: audit 2021-03-10T15:21:28.929615+0000 mgr.gibba032.whopeo (mgr.24625) 4700 : audit [DBG] from='client.57196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:30 gibba032 bash[24004]: cluster 2021-03-10T15:21:28.295496+0000 mgr.gibba032.whopeo (mgr.24625) 4698 : cluster [DBG] pgmap v2669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:30 gibba032 bash[24004]: audit 2021-03-10T15:21:28.702134+0000 mgr.gibba032.whopeo (mgr.24625) 4699 : audit [DBG] from='client.47205 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:30 gibba032 bash[24004]: audit 2021-03-10T15:21:28.929615+0000 mgr.gibba032.whopeo (mgr.24625) 4700 : audit [DBG] from='client.57196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:21:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:31 gibba011 bash[30791]: cluster 2021-03-10T15:21:30.296061+0000 mgr.gibba032.whopeo (mgr.24625) 4701 : cluster [DBG] pgmap v2670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:31.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:31 gibba011 bash[31774]: cluster 2021-03-10T15:21:30.296061+0000 mgr.gibba032.whopeo (mgr.24625) 4701 : cluster [DBG] pgmap v2670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:31.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:31 gibba032 bash[24004]: cluster 2021-03-10T15:21:30.296061+0000 mgr.gibba032.whopeo (mgr.24625) 4701 : cluster [DBG] pgmap v2670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:33.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:33 gibba011 bash[30791]: cluster 2021-03-10T15:21:32.297709+0000 mgr.gibba032.whopeo (mgr.24625) 4702 : cluster [DBG] pgmap v2671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:33.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:33 gibba011 bash[31774]: cluster 2021-03-10T15:21:32.297709+0000 mgr.gibba032.whopeo (mgr.24625) 4702 : cluster [DBG] pgmap v2671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:34.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:33 gibba032 bash[24004]: cluster 2021-03-10T15:21:32.297709+0000 mgr.gibba032.whopeo (mgr.24625) 4702 : cluster [DBG] pgmap v2671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:35.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:35 gibba011 bash[30791]: cluster 2021-03-10T15:21:34.299056+0000 mgr.gibba032.whopeo (mgr.24625) 4703 : cluster [DBG] pgmap v2672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:35.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:35 gibba011 bash[31774]: cluster 2021-03-10T15:21:34.299056+0000 mgr.gibba032.whopeo (mgr.24625) 4703 : cluster [DBG] pgmap v2672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:36.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:35 gibba032 bash[24004]: cluster 2021-03-10T15:21:34.299056+0000 mgr.gibba032.whopeo (mgr.24625) 4703 : cluster [DBG] pgmap v2672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:37.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:37 gibba011 bash[30791]: cluster 2021-03-10T15:21:36.300674+0000 mgr.gibba032.whopeo (mgr.24625) 4704 : cluster [DBG] pgmap v2673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:37 gibba011 bash[31774]: cluster 2021-03-10T15:21:36.300674+0000 mgr.gibba032.whopeo (mgr.24625) 4704 : cluster [DBG] pgmap v2673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:37 gibba032 bash[24004]: cluster 2021-03-10T15:21:36.300674+0000 mgr.gibba032.whopeo (mgr.24625) 4704 : cluster [DBG] pgmap v2673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:39.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:39 gibba011 bash[30791]: cluster 2021-03-10T15:21:38.301383+0000 mgr.gibba032.whopeo (mgr.24625) 4705 : cluster [DBG] pgmap v2674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:39 gibba011 bash[31774]: cluster 2021-03-10T15:21:38.301383+0000 mgr.gibba032.whopeo (mgr.24625) 4705 : cluster [DBG] pgmap v2674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:39 gibba032 bash[24004]: cluster 2021-03-10T15:21:38.301383+0000 mgr.gibba032.whopeo (mgr.24625) 4705 : cluster [DBG] pgmap v2674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:41.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:41 gibba011 bash[30791]: cluster 2021-03-10T15:21:40.302579+0000 mgr.gibba032.whopeo (mgr.24625) 4706 : cluster [DBG] pgmap v2675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:41.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:41 gibba011 bash[31774]: cluster 2021-03-10T15:21:40.302579+0000 mgr.gibba032.whopeo (mgr.24625) 4706 : cluster [DBG] pgmap v2675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:42.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:41 gibba032 bash[24004]: cluster 2021-03-10T15:21:40.302579+0000 mgr.gibba032.whopeo (mgr.24625) 4706 : cluster [DBG] pgmap v2675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:43 gibba011 bash[30791]: cluster 2021-03-10T15:21:42.304150+0000 mgr.gibba032.whopeo (mgr.24625) 4707 : cluster [DBG] pgmap v2676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:43 gibba011 bash[30791]: cephadm 2021-03-10T15:21:42.976725+0000 mgr.gibba032.whopeo (mgr.24625) 4708 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:43 gibba011 bash[31774]: cluster 2021-03-10T15:21:42.304150+0000 mgr.gibba032.whopeo (mgr.24625) 4707 : cluster [DBG] pgmap v2676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:43 gibba011 bash[31774]: cephadm 2021-03-10T15:21:42.976725+0000 mgr.gibba032.whopeo (mgr.24625) 4708 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:43 gibba032 bash[24004]: cluster 2021-03-10T15:21:42.304150+0000 mgr.gibba032.whopeo (mgr.24625) 4707 : cluster [DBG] pgmap v2676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:43 gibba032 bash[24004]: cephadm 2021-03-10T15:21:42.976725+0000 mgr.gibba032.whopeo (mgr.24625) 4708 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:45.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:45 gibba011 bash[30791]: cluster 2021-03-10T15:21:44.305435+0000 mgr.gibba032.whopeo (mgr.24625) 4709 : cluster [DBG] pgmap v2677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:45 gibba011 bash[31774]: cluster 2021-03-10T15:21:44.305435+0000 mgr.gibba032.whopeo (mgr.24625) 4709 : cluster [DBG] pgmap v2677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:45 gibba032 bash[24004]: cluster 2021-03-10T15:21:44.305435+0000 mgr.gibba032.whopeo (mgr.24625) 4709 : cluster [DBG] pgmap v2677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:47.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:47 gibba011 bash[30791]: cluster 2021-03-10T15:21:46.306045+0000 mgr.gibba032.whopeo (mgr.24625) 4710 : cluster [DBG] pgmap v2678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:47.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:47 gibba011 bash[31774]: cluster 2021-03-10T15:21:46.306045+0000 mgr.gibba032.whopeo (mgr.24625) 4710 : cluster [DBG] pgmap v2678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:47 gibba032 bash[24004]: cluster 2021-03-10T15:21:46.306045+0000 mgr.gibba032.whopeo (mgr.24625) 4710 : cluster [DBG] pgmap v2678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:49 gibba032 bash[24004]: cluster 2021-03-10T15:21:48.307678+0000 mgr.gibba032.whopeo (mgr.24625) 4711 : cluster [DBG] pgmap v2679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:49 gibba011 bash[30791]: cluster 2021-03-10T15:21:48.307678+0000 mgr.gibba032.whopeo (mgr.24625) 4711 : cluster [DBG] pgmap v2679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:49 gibba011 bash[31774]: cluster 2021-03-10T15:21:48.307678+0000 mgr.gibba032.whopeo (mgr.24625) 4711 : cluster [DBG] pgmap v2679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:52.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:51 gibba032 bash[24004]: cluster 2021-03-10T15:21:50.309187+0000 mgr.gibba032.whopeo (mgr.24625) 4712 : cluster [DBG] pgmap v2680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:52.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:51 gibba011 bash[30791]: cluster 2021-03-10T15:21:50.309187+0000 mgr.gibba032.whopeo (mgr.24625) 4712 : cluster [DBG] pgmap v2680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:52.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:51 gibba011 bash[31774]: cluster 2021-03-10T15:21:50.309187+0000 mgr.gibba032.whopeo (mgr.24625) 4712 : cluster [DBG] pgmap v2680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:53 gibba032 bash[24004]: cluster 2021-03-10T15:21:52.310824+0000 mgr.gibba032.whopeo (mgr.24625) 4713 : cluster [DBG] pgmap v2681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:53 gibba011 bash[30791]: cluster 2021-03-10T15:21:52.310824+0000 mgr.gibba032.whopeo (mgr.24625) 4713 : cluster [DBG] pgmap v2681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:54.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:53 gibba011 bash[31774]: cluster 2021-03-10T15:21:52.310824+0000 mgr.gibba032.whopeo (mgr.24625) 4713 : cluster [DBG] pgmap v2681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:55 gibba032 bash[24004]: cluster 2021-03-10T15:21:54.311520+0000 mgr.gibba032.whopeo (mgr.24625) 4714 : cluster [DBG] pgmap v2682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:56.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:55 gibba011 bash[30791]: cluster 2021-03-10T15:21:54.311520+0000 mgr.gibba032.whopeo (mgr.24625) 4714 : cluster [DBG] pgmap v2682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:56.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:55 gibba011 bash[31774]: cluster 2021-03-10T15:21:54.311520+0000 mgr.gibba032.whopeo (mgr.24625) 4714 : cluster [DBG] pgmap v2682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.564635+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:21:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.565305+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:21:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.575642+0000 mon.a (mon.0) 1896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:21:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.576396+0000 mon.a (mon.0) 1897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:21:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.577226+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:21:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:56 gibba032 bash[24004]: audit 2021-03-10T15:21:56.577919+0000 mon.a (mon.0) 1899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:21:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.564635+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:21:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.565305+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:21:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.575642+0000 mon.a (mon.0) 1896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:21:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.576396+0000 mon.a (mon.0) 1897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:21:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.577226+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:21:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[30791]: audit 2021-03-10T15:21:56.577919+0000 mon.a (mon.0) 1899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:21:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.564635+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:21:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.565305+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:21:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.575642+0000 mon.a (mon.0) 1896 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:21:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.576396+0000 mon.a (mon.0) 1897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:21:57.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.577226+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:21:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:56 gibba011 bash[31774]: audit 2021-03-10T15:21:56.577919+0000 mon.a (mon.0) 1899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:21:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:57 gibba032 bash[24004]: cluster 2021-03-10T15:21:56.312857+0000 mgr.gibba032.whopeo (mgr.24625) 4715 : cluster [DBG] pgmap v2683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:57 gibba011 bash[30791]: cluster 2021-03-10T15:21:56.312857+0000 mgr.gibba032.whopeo (mgr.24625) 4715 : cluster [DBG] pgmap v2683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:57 gibba011 bash[31774]: cluster 2021-03-10T15:21:56.312857+0000 mgr.gibba032.whopeo (mgr.24625) 4715 : cluster [DBG] pgmap v2683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:21:59.440 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:21:59.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:57.993543+0000 mgr.gibba032.whopeo (mgr.24625) 4716 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:21:59.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:57.995099+0000 mgr.gibba032.whopeo (mgr.24625) 4717 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:21:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.197696+0000 mon.a (mon.0) 1900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:21:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.199104+0000 mgr.gibba032.whopeo (mgr.24625) 4718 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:21:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.201128+0000 mon.a (mon.0) 1901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:21:59.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.201140+0000 mgr.gibba032.whopeo (mgr.24625) 4719 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:21:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.203016+0000 mon.a (mon.0) 1902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.204319+0000 mgr.gibba032.whopeo (mgr.24625) 4720 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:21:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.204816+0000 mgr.gibba032.whopeo (mgr.24625) 4721 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:21:59.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.207617+0000 mon.a (mon.0) 1903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.208936+0000 mgr.gibba032.whopeo (mgr.24625) 4722 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:21:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.209303+0000 mgr.gibba032.whopeo (mgr.24625) 4723 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:21:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.211307+0000 mon.a (mon.0) 1904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.212228+0000 mgr.gibba032.whopeo (mgr.24625) 4724 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:21:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.215244+0000 mgr.gibba032.whopeo (mgr.24625) 4725 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:21:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.215628+0000 mgr.gibba032.whopeo (mgr.24625) 4726 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:21:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.218799+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.219791+0000 mgr.gibba032.whopeo (mgr.24625) 4727 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:21:59.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.221725+0000 mgr.gibba032.whopeo (mgr.24625) 4728 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:21:59.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: audit 2021-03-10T15:21:58.223684+0000 mon.a (mon.0) 1906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.224895+0000 mgr.gibba032.whopeo (mgr.24625) 4729 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:21:59.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.225298+0000 mgr.gibba032.whopeo (mgr.24625) 4730 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:21:59.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:21:59 gibba032 bash[24004]: cephadm 2021-03-10T15:21:58.225943+0000 mgr.gibba032.whopeo (mgr.24625) 4731 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:57.993543+0000 mgr.gibba032.whopeo (mgr.24625) 4716 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:21:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:57.995099+0000 mgr.gibba032.whopeo (mgr.24625) 4717 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:21:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.197696+0000 mon.a (mon.0) 1900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:21:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.199104+0000 mgr.gibba032.whopeo (mgr.24625) 4718 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:21:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.201128+0000 mon.a (mon.0) 1901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:21:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.201140+0000 mgr.gibba032.whopeo (mgr.24625) 4719 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:21:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.203016+0000 mon.a (mon.0) 1902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.204319+0000 mgr.gibba032.whopeo (mgr.24625) 4720 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:21:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.204816+0000 mgr.gibba032.whopeo (mgr.24625) 4721 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:21:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.207617+0000 mon.a (mon.0) 1903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.208936+0000 mgr.gibba032.whopeo (mgr.24625) 4722 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:21:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.209303+0000 mgr.gibba032.whopeo (mgr.24625) 4723 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:21:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.211307+0000 mon.a (mon.0) 1904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.212228+0000 mgr.gibba032.whopeo (mgr.24625) 4724 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:21:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.215244+0000 mgr.gibba032.whopeo (mgr.24625) 4725 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:21:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.215628+0000 mgr.gibba032.whopeo (mgr.24625) 4726 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:21:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.218799+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.219791+0000 mgr.gibba032.whopeo (mgr.24625) 4727 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:21:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.221725+0000 mgr.gibba032.whopeo (mgr.24625) 4728 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:21:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: audit 2021-03-10T15:21:58.223684+0000 mon.a (mon.0) 1906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.224895+0000 mgr.gibba032.whopeo (mgr.24625) 4729 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:21:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.225298+0000 mgr.gibba032.whopeo (mgr.24625) 4730 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:21:59.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[30791]: cephadm 2021-03-10T15:21:58.225943+0000 mgr.gibba032.whopeo (mgr.24625) 4731 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:57.993543+0000 mgr.gibba032.whopeo (mgr.24625) 4716 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:21:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:57.995099+0000 mgr.gibba032.whopeo (mgr.24625) 4717 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:21:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.197696+0000 mon.a (mon.0) 1900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:21:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.199104+0000 mgr.gibba032.whopeo (mgr.24625) 4718 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:21:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.201128+0000 mon.a (mon.0) 1901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:21:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.201140+0000 mgr.gibba032.whopeo (mgr.24625) 4719 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:21:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.203016+0000 mon.a (mon.0) 1902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.204319+0000 mgr.gibba032.whopeo (mgr.24625) 4720 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:21:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.204816+0000 mgr.gibba032.whopeo (mgr.24625) 4721 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:21:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.207617+0000 mon.a (mon.0) 1903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.208936+0000 mgr.gibba032.whopeo (mgr.24625) 4722 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:21:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.209303+0000 mgr.gibba032.whopeo (mgr.24625) 4723 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:21:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.211307+0000 mon.a (mon.0) 1904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.212228+0000 mgr.gibba032.whopeo (mgr.24625) 4724 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:21:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.215244+0000 mgr.gibba032.whopeo (mgr.24625) 4725 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:21:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.215628+0000 mgr.gibba032.whopeo (mgr.24625) 4726 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:21:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.218799+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.219791+0000 mgr.gibba032.whopeo (mgr.24625) 4727 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:21:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.221725+0000 mgr.gibba032.whopeo (mgr.24625) 4728 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:21:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: audit 2021-03-10T15:21:58.223684+0000 mon.a (mon.0) 1906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:21:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.224895+0000 mgr.gibba032.whopeo (mgr.24625) 4729 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:21:59.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.225298+0000 mgr.gibba032.whopeo (mgr.24625) 4730 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:21:59.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:21:59 gibba011 bash[31774]: cephadm 2021-03-10T15:21:58.225943+0000 mgr.gibba032.whopeo (mgr.24625) 4731 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:21:59.670 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:21:59.671 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (88m) 3m ago 94m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:21:59.671 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (93m) 2m ago 93m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:21:59.671 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (88m) 2m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:21:59.672 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (89m) 2m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:21:59.672 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (88m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:21:59.672 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (88m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:21:59.672 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (87m) 2m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:21:59.673 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (87m) 3m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:21:59.673 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (94m) 3m ago 94m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:21:59.673 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (94m) 2m ago 94m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:21:59.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (86m) 3m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:21:59.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:21:59.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:21:59.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (85m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:21:59.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:21:59.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (84m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:21:59.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (84m) 2m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:21:59.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (83m) 2m ago 94m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:21:59.675 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (93m) 2m ago 94m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:21:59.675 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 92m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:21:59.918 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:21:59.919 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:21:59.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:21:59.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:59.919 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:21:59.920 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:21:59.920 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:59.920 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:21:59.920 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:21:59.921 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:21:59.921 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:21:59.921 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:21:59.921 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:21:59.922 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:21:59.922 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:22:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:00 gibba032 bash[24004]: cluster 2021-03-10T15:21:58.314548+0000 mgr.gibba032.whopeo (mgr.24625) 4732 : cluster [DBG] pgmap v2684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:00 gibba032 bash[24004]: audit 2021-03-10T15:21:59.915710+0000 mon.a (mon.0) 1907 : audit [DBG] from='client.? 172.21.2.111:0/3769996381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:00 gibba011 bash[30791]: cluster 2021-03-10T15:21:58.314548+0000 mgr.gibba032.whopeo (mgr.24625) 4732 : cluster [DBG] pgmap v2684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:00 gibba011 bash[30791]: audit 2021-03-10T15:21:59.915710+0000 mon.a (mon.0) 1907 : audit [DBG] from='client.? 172.21.2.111:0/3769996381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:00 gibba011 bash[31774]: cluster 2021-03-10T15:21:58.314548+0000 mgr.gibba032.whopeo (mgr.24625) 4732 : cluster [DBG] pgmap v2684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:00 gibba011 bash[31774]: audit 2021-03-10T15:21:59.915710+0000 mon.a (mon.0) 1907 : audit [DBG] from='client.? 172.21.2.111:0/3769996381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:01 gibba032 bash[24004]: audit 2021-03-10T15:21:59.427040+0000 mgr.gibba032.whopeo (mgr.24625) 4733 : audit [DBG] from='client.57208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:01 gibba032 bash[24004]: audit 2021-03-10T15:21:59.654624+0000 mgr.gibba032.whopeo (mgr.24625) 4734 : audit [DBG] from='client.47229 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:01 gibba011 bash[30791]: audit 2021-03-10T15:21:59.427040+0000 mgr.gibba032.whopeo (mgr.24625) 4733 : audit [DBG] from='client.57208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:01 gibba011 bash[30791]: audit 2021-03-10T15:21:59.654624+0000 mgr.gibba032.whopeo (mgr.24625) 4734 : audit [DBG] from='client.47229 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:01 gibba011 bash[31774]: audit 2021-03-10T15:21:59.427040+0000 mgr.gibba032.whopeo (mgr.24625) 4733 : audit [DBG] from='client.57208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:01 gibba011 bash[31774]: audit 2021-03-10T15:21:59.654624+0000 mgr.gibba032.whopeo (mgr.24625) 4734 : audit [DBG] from='client.47229 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:02 gibba032 bash[24004]: cluster 2021-03-10T15:22:00.315505+0000 mgr.gibba032.whopeo (mgr.24625) 4735 : cluster [DBG] pgmap v2685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:02 gibba011 bash[30791]: cluster 2021-03-10T15:22:00.315505+0000 mgr.gibba032.whopeo (mgr.24625) 4735 : cluster [DBG] pgmap v2685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:02 gibba011 bash[31774]: cluster 2021-03-10T15:22:00.315505+0000 mgr.gibba032.whopeo (mgr.24625) 4735 : cluster [DBG] pgmap v2685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:04 gibba032 bash[24004]: cluster 2021-03-10T15:22:02.316999+0000 mgr.gibba032.whopeo (mgr.24625) 4736 : cluster [DBG] pgmap v2686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:04 gibba011 bash[30791]: cluster 2021-03-10T15:22:02.316999+0000 mgr.gibba032.whopeo (mgr.24625) 4736 : cluster [DBG] pgmap v2686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:04 gibba011 bash[31774]: cluster 2021-03-10T15:22:02.316999+0000 mgr.gibba032.whopeo (mgr.24625) 4736 : cluster [DBG] pgmap v2686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:06 gibba032 bash[24004]: cluster 2021-03-10T15:22:04.317807+0000 mgr.gibba032.whopeo (mgr.24625) 4737 : cluster [DBG] pgmap v2687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:06 gibba011 bash[30791]: cluster 2021-03-10T15:22:04.317807+0000 mgr.gibba032.whopeo (mgr.24625) 4737 : cluster [DBG] pgmap v2687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:06 gibba011 bash[31774]: cluster 2021-03-10T15:22:04.317807+0000 mgr.gibba032.whopeo (mgr.24625) 4737 : cluster [DBG] pgmap v2687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:08 gibba011 bash[30791]: cluster 2021-03-10T15:22:06.318915+0000 mgr.gibba032.whopeo (mgr.24625) 4738 : cluster [DBG] pgmap v2688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:08 gibba011 bash[31774]: cluster 2021-03-10T15:22:06.318915+0000 mgr.gibba032.whopeo (mgr.24625) 4738 : cluster [DBG] pgmap v2688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:08 gibba032 bash[24004]: cluster 2021-03-10T15:22:06.318915+0000 mgr.gibba032.whopeo (mgr.24625) 4738 : cluster [DBG] pgmap v2688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:10 gibba011 bash[30791]: cluster 2021-03-10T15:22:08.319907+0000 mgr.gibba032.whopeo (mgr.24625) 4739 : cluster [DBG] pgmap v2689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:10 gibba011 bash[31774]: cluster 2021-03-10T15:22:08.319907+0000 mgr.gibba032.whopeo (mgr.24625) 4739 : cluster [DBG] pgmap v2689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:10 gibba032 bash[24004]: cluster 2021-03-10T15:22:08.319907+0000 mgr.gibba032.whopeo (mgr.24625) 4739 : cluster [DBG] pgmap v2689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:12 gibba011 bash[30791]: cluster 2021-03-10T15:22:10.320646+0000 mgr.gibba032.whopeo (mgr.24625) 4740 : cluster [DBG] pgmap v2690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:12 gibba011 bash[31774]: cluster 2021-03-10T15:22:10.320646+0000 mgr.gibba032.whopeo (mgr.24625) 4740 : cluster [DBG] pgmap v2690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:12 gibba032 bash[24004]: cluster 2021-03-10T15:22:10.320646+0000 mgr.gibba032.whopeo (mgr.24625) 4740 : cluster [DBG] pgmap v2690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:13 gibba011 bash[30791]: cluster 2021-03-10T15:22:12.321506+0000 mgr.gibba032.whopeo (mgr.24625) 4741 : cluster [DBG] pgmap v2691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:13 gibba011 bash[30791]: cephadm 2021-03-10T15:22:13.241962+0000 mgr.gibba032.whopeo (mgr.24625) 4742 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:13.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:13 gibba011 bash[31774]: cluster 2021-03-10T15:22:12.321506+0000 mgr.gibba032.whopeo (mgr.24625) 4741 : cluster [DBG] pgmap v2691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:13.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:13 gibba011 bash[31774]: cephadm 2021-03-10T15:22:13.241962+0000 mgr.gibba032.whopeo (mgr.24625) 4742 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:14.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:13 gibba032 bash[24004]: cluster 2021-03-10T15:22:12.321506+0000 mgr.gibba032.whopeo (mgr.24625) 4741 : cluster [DBG] pgmap v2691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:14.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:13 gibba032 bash[24004]: cephadm 2021-03-10T15:22:13.241962+0000 mgr.gibba032.whopeo (mgr.24625) 4742 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:15.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:15 gibba011 bash[30791]: cluster 2021-03-10T15:22:14.322605+0000 mgr.gibba032.whopeo (mgr.24625) 4743 : cluster [DBG] pgmap v2692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:15.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:15 gibba011 bash[31774]: cluster 2021-03-10T15:22:14.322605+0000 mgr.gibba032.whopeo (mgr.24625) 4743 : cluster [DBG] pgmap v2692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:16.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:15 gibba032 bash[24004]: cluster 2021-03-10T15:22:14.322605+0000 mgr.gibba032.whopeo (mgr.24625) 4743 : cluster [DBG] pgmap v2692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:17 gibba011 bash[30791]: cluster 2021-03-10T15:22:16.324135+0000 mgr.gibba032.whopeo (mgr.24625) 4744 : cluster [DBG] pgmap v2693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:17.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:17 gibba011 bash[31774]: cluster 2021-03-10T15:22:16.324135+0000 mgr.gibba032.whopeo (mgr.24625) 4744 : cluster [DBG] pgmap v2693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:17 gibba032 bash[24004]: cluster 2021-03-10T15:22:16.324135+0000 mgr.gibba032.whopeo (mgr.24625) 4744 : cluster [DBG] pgmap v2693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:19 gibba011 bash[30791]: cluster 2021-03-10T15:22:18.325090+0000 mgr.gibba032.whopeo (mgr.24625) 4745 : cluster [DBG] pgmap v2694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:19.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:19 gibba011 bash[31774]: cluster 2021-03-10T15:22:18.325090+0000 mgr.gibba032.whopeo (mgr.24625) 4745 : cluster [DBG] pgmap v2694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:19 gibba032 bash[24004]: cluster 2021-03-10T15:22:18.325090+0000 mgr.gibba032.whopeo (mgr.24625) 4745 : cluster [DBG] pgmap v2694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:21 gibba011 bash[30791]: cluster 2021-03-10T15:22:20.326522+0000 mgr.gibba032.whopeo (mgr.24625) 4746 : cluster [DBG] pgmap v2695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:21 gibba011 bash[31774]: cluster 2021-03-10T15:22:20.326522+0000 mgr.gibba032.whopeo (mgr.24625) 4746 : cluster [DBG] pgmap v2695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:22.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:21 gibba032 bash[24004]: cluster 2021-03-10T15:22:20.326522+0000 mgr.gibba032.whopeo (mgr.24625) 4746 : cluster [DBG] pgmap v2695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:23 gibba032 bash[24004]: cluster 2021-03-10T15:22:22.328202+0000 mgr.gibba032.whopeo (mgr.24625) 4747 : cluster [DBG] pgmap v2696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:23 gibba011 bash[30791]: cluster 2021-03-10T15:22:22.328202+0000 mgr.gibba032.whopeo (mgr.24625) 4747 : cluster [DBG] pgmap v2696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:23 gibba011 bash[31774]: cluster 2021-03-10T15:22:22.328202+0000 mgr.gibba032.whopeo (mgr.24625) 4747 : cluster [DBG] pgmap v2696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:25 gibba032 bash[24004]: cluster 2021-03-10T15:22:24.329320+0000 mgr.gibba032.whopeo (mgr.24625) 4748 : cluster [DBG] pgmap v2697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:26.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:25 gibba011 bash[30791]: cluster 2021-03-10T15:22:24.329320+0000 mgr.gibba032.whopeo (mgr.24625) 4748 : cluster [DBG] pgmap v2697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:26.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:25 gibba011 bash[31774]: cluster 2021-03-10T15:22:24.329320+0000 mgr.gibba032.whopeo (mgr.24625) 4748 : cluster [DBG] pgmap v2697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:27 gibba032 bash[24004]: cluster 2021-03-10T15:22:26.330160+0000 mgr.gibba032.whopeo (mgr.24625) 4749 : cluster [DBG] pgmap v2698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:27 gibba011 bash[30791]: cluster 2021-03-10T15:22:26.330160+0000 mgr.gibba032.whopeo (mgr.24625) 4749 : cluster [DBG] pgmap v2698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:27 gibba011 bash[31774]: cluster 2021-03-10T15:22:26.330160+0000 mgr.gibba032.whopeo (mgr.24625) 4749 : cluster [DBG] pgmap v2698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:29.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:28 gibba032 bash[24004]: cephadm 2021-03-10T15:22:28.257896+0000 mgr.gibba032.whopeo (mgr.24625) 4750 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:28 gibba011 bash[30791]: cephadm 2021-03-10T15:22:28.257896+0000 mgr.gibba032.whopeo (mgr.24625) 4750 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:29.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:28 gibba011 bash[31774]: cephadm 2021-03-10T15:22:28.257896+0000 mgr.gibba032.whopeo (mgr.24625) 4750 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:29 gibba032 bash[24004]: cluster 2021-03-10T15:22:28.331785+0000 mgr.gibba032.whopeo (mgr.24625) 4751 : cluster [DBG] pgmap v2699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:30.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:29 gibba011 bash[30791]: cluster 2021-03-10T15:22:28.331785+0000 mgr.gibba032.whopeo (mgr.24625) 4751 : cluster [DBG] pgmap v2699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:30.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:29 gibba011 bash[31774]: cluster 2021-03-10T15:22:28.331785+0000 mgr.gibba032.whopeo (mgr.24625) 4751 : cluster [DBG] pgmap v2699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:30.162 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:22:30.394 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:22:30.394 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (89m) 3m ago 94m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:22:30.394 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (94m) 2m ago 94m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:22:30.394 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (89m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:22:30.395 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (89m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:22:30.395 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (88m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:22:30.395 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (88m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:22:30.395 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (88m) 2m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:22:30.396 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (88m) 3m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:22:30.396 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (94m) 3m ago 94m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:22:30.396 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (94m) 2m ago 94m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:22:30.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (87m) 3m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:22:30.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (87m) 3m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:22:30.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:22:30.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:22:30.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (85m) 2m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:22:30.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (85m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:22:30.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (84m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:22:30.398 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (84m) 2m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:22:30.398 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (94m) 2m ago 94m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:22:30.398 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 92m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:22:30.645 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:22:30.646 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:22:30.646 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:22:30.646 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:22:30.646 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:22:30.647 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:22:30.647 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:22:30.647 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:22:30.647 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:22:30.647 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:22:30.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:22:30.648 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:22:30.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:22:30.648 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:22:30.649 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:22:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:30 gibba032 bash[24004]: audit 2021-03-10T15:22:30.149285+0000 mgr.gibba032.whopeo (mgr.24625) 4752 : audit [DBG] from='client.47241 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:30 gibba011 bash[30791]: audit 2021-03-10T15:22:30.149285+0000 mgr.gibba032.whopeo (mgr.24625) 4752 : audit [DBG] from='client.47241 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:31.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:30 gibba011 bash[31774]: audit 2021-03-10T15:22:30.149285+0000 mgr.gibba032.whopeo (mgr.24625) 4752 : audit [DBG] from='client.47241 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:32.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:31 gibba032 bash[24004]: cluster 2021-03-10T15:22:30.333383+0000 mgr.gibba032.whopeo (mgr.24625) 4753 : cluster [DBG] pgmap v2700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:32.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:31 gibba032 bash[24004]: audit 2021-03-10T15:22:30.377853+0000 mgr.gibba032.whopeo (mgr.24625) 4754 : audit [DBG] from='client.47247 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:32.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:31 gibba032 bash[24004]: audit 2021-03-10T15:22:30.642876+0000 mon.a (mon.0) 1908 : audit [DBG] from='client.? 172.21.2.111:0/1816397426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:32.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[30791]: cluster 2021-03-10T15:22:30.333383+0000 mgr.gibba032.whopeo (mgr.24625) 4753 : cluster [DBG] pgmap v2700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:32.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[30791]: audit 2021-03-10T15:22:30.377853+0000 mgr.gibba032.whopeo (mgr.24625) 4754 : audit [DBG] from='client.47247 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:32.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[30791]: audit 2021-03-10T15:22:30.642876+0000 mon.a (mon.0) 1908 : audit [DBG] from='client.? 172.21.2.111:0/1816397426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[31774]: cluster 2021-03-10T15:22:30.333383+0000 mgr.gibba032.whopeo (mgr.24625) 4753 : cluster [DBG] pgmap v2700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[31774]: audit 2021-03-10T15:22:30.377853+0000 mgr.gibba032.whopeo (mgr.24625) 4754 : audit [DBG] from='client.47247 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:22:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:31 gibba011 bash[31774]: audit 2021-03-10T15:22:30.642876+0000 mon.a (mon.0) 1908 : audit [DBG] from='client.? 172.21.2.111:0/1816397426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:34.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:33 gibba032 bash[24004]: cluster 2021-03-10T15:22:32.335125+0000 mgr.gibba032.whopeo (mgr.24625) 4755 : cluster [DBG] pgmap v2701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:34.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:33 gibba011 bash[30791]: cluster 2021-03-10T15:22:32.335125+0000 mgr.gibba032.whopeo (mgr.24625) 4755 : cluster [DBG] pgmap v2701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:34.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:33 gibba011 bash[31774]: cluster 2021-03-10T15:22:32.335125+0000 mgr.gibba032.whopeo (mgr.24625) 4755 : cluster [DBG] pgmap v2701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:36.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:35 gibba032 bash[24004]: cluster 2021-03-10T15:22:34.335793+0000 mgr.gibba032.whopeo (mgr.24625) 4756 : cluster [DBG] pgmap v2702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:36.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:35 gibba011 bash[30791]: cluster 2021-03-10T15:22:34.335793+0000 mgr.gibba032.whopeo (mgr.24625) 4756 : cluster [DBG] pgmap v2702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:36.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:35 gibba011 bash[31774]: cluster 2021-03-10T15:22:34.335793+0000 mgr.gibba032.whopeo (mgr.24625) 4756 : cluster [DBG] pgmap v2702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:38.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:37 gibba032 bash[24004]: cluster 2021-03-10T15:22:36.337204+0000 mgr.gibba032.whopeo (mgr.24625) 4757 : cluster [DBG] pgmap v2703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:38.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:37 gibba011 bash[30791]: cluster 2021-03-10T15:22:36.337204+0000 mgr.gibba032.whopeo (mgr.24625) 4757 : cluster [DBG] pgmap v2703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:37 gibba011 bash[31774]: cluster 2021-03-10T15:22:36.337204+0000 mgr.gibba032.whopeo (mgr.24625) 4757 : cluster [DBG] pgmap v2703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:39 gibba032 bash[24004]: cluster 2021-03-10T15:22:38.338973+0000 mgr.gibba032.whopeo (mgr.24625) 4758 : cluster [DBG] pgmap v2704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:39 gibba011 bash[30791]: cluster 2021-03-10T15:22:38.338973+0000 mgr.gibba032.whopeo (mgr.24625) 4758 : cluster [DBG] pgmap v2704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:39 gibba011 bash[31774]: cluster 2021-03-10T15:22:38.338973+0000 mgr.gibba032.whopeo (mgr.24625) 4758 : cluster [DBG] pgmap v2704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:42.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:41 gibba032 bash[24004]: cluster 2021-03-10T15:22:40.340571+0000 mgr.gibba032.whopeo (mgr.24625) 4759 : cluster [DBG] pgmap v2705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:41 gibba011 bash[30791]: cluster 2021-03-10T15:22:40.340571+0000 mgr.gibba032.whopeo (mgr.24625) 4759 : cluster [DBG] pgmap v2705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:41 gibba011 bash[31774]: cluster 2021-03-10T15:22:40.340571+0000 mgr.gibba032.whopeo (mgr.24625) 4759 : cluster [DBG] pgmap v2705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:43 gibba032 bash[24004]: cluster 2021-03-10T15:22:42.341603+0000 mgr.gibba032.whopeo (mgr.24625) 4760 : cluster [DBG] pgmap v2706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:43 gibba032 bash[24004]: cephadm 2021-03-10T15:22:43.273893+0000 mgr.gibba032.whopeo (mgr.24625) 4761 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:44.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:43 gibba011 bash[30791]: cluster 2021-03-10T15:22:42.341603+0000 mgr.gibba032.whopeo (mgr.24625) 4760 : cluster [DBG] pgmap v2706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:44.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:43 gibba011 bash[30791]: cephadm 2021-03-10T15:22:43.273893+0000 mgr.gibba032.whopeo (mgr.24625) 4761 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:43 gibba011 bash[31774]: cluster 2021-03-10T15:22:42.341603+0000 mgr.gibba032.whopeo (mgr.24625) 4760 : cluster [DBG] pgmap v2706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:43 gibba011 bash[31774]: cephadm 2021-03-10T15:22:43.273893+0000 mgr.gibba032.whopeo (mgr.24625) 4761 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:22:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:45 gibba032 bash[24004]: cluster 2021-03-10T15:22:44.342646+0000 mgr.gibba032.whopeo (mgr.24625) 4762 : cluster [DBG] pgmap v2707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:45 gibba011 bash[30791]: cluster 2021-03-10T15:22:44.342646+0000 mgr.gibba032.whopeo (mgr.24625) 4762 : cluster [DBG] pgmap v2707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:46.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:45 gibba011 bash[31774]: cluster 2021-03-10T15:22:44.342646+0000 mgr.gibba032.whopeo (mgr.24625) 4762 : cluster [DBG] pgmap v2707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:48.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:47 gibba032 bash[24004]: cluster 2021-03-10T15:22:46.344395+0000 mgr.gibba032.whopeo (mgr.24625) 4763 : cluster [DBG] pgmap v2708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:48.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:47 gibba011 bash[30791]: cluster 2021-03-10T15:22:46.344395+0000 mgr.gibba032.whopeo (mgr.24625) 4763 : cluster [DBG] pgmap v2708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:48.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:47 gibba011 bash[31774]: cluster 2021-03-10T15:22:46.344395+0000 mgr.gibba032.whopeo (mgr.24625) 4763 : cluster [DBG] pgmap v2708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:49 gibba032 bash[24004]: cluster 2021-03-10T15:22:48.345911+0000 mgr.gibba032.whopeo (mgr.24625) 4764 : cluster [DBG] pgmap v2709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:50.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:49 gibba011 bash[30791]: cluster 2021-03-10T15:22:48.345911+0000 mgr.gibba032.whopeo (mgr.24625) 4764 : cluster [DBG] pgmap v2709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:49 gibba011 bash[31774]: cluster 2021-03-10T15:22:48.345911+0000 mgr.gibba032.whopeo (mgr.24625) 4764 : cluster [DBG] pgmap v2709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:52.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:51 gibba032 bash[24004]: cluster 2021-03-10T15:22:50.346906+0000 mgr.gibba032.whopeo (mgr.24625) 4765 : cluster [DBG] pgmap v2710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:52.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:51 gibba011 bash[30791]: cluster 2021-03-10T15:22:50.346906+0000 mgr.gibba032.whopeo (mgr.24625) 4765 : cluster [DBG] pgmap v2710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:52.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:51 gibba011 bash[31774]: cluster 2021-03-10T15:22:50.346906+0000 mgr.gibba032.whopeo (mgr.24625) 4765 : cluster [DBG] pgmap v2710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:53 gibba032 bash[24004]: cluster 2021-03-10T15:22:52.348619+0000 mgr.gibba032.whopeo (mgr.24625) 4766 : cluster [DBG] pgmap v2711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:53 gibba011 bash[30791]: cluster 2021-03-10T15:22:52.348619+0000 mgr.gibba032.whopeo (mgr.24625) 4766 : cluster [DBG] pgmap v2711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:54.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:53 gibba011 bash[31774]: cluster 2021-03-10T15:22:52.348619+0000 mgr.gibba032.whopeo (mgr.24625) 4766 : cluster [DBG] pgmap v2711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:55 gibba032 bash[24004]: cluster 2021-03-10T15:22:54.349959+0000 mgr.gibba032.whopeo (mgr.24625) 4767 : cluster [DBG] pgmap v2712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:55 gibba011 bash[30791]: cluster 2021-03-10T15:22:54.349959+0000 mgr.gibba032.whopeo (mgr.24625) 4767 : cluster [DBG] pgmap v2712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:56.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:55 gibba011 bash[31774]: cluster 2021-03-10T15:22:54.349959+0000 mgr.gibba032.whopeo (mgr.24625) 4767 : cluster [DBG] pgmap v2712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.567089+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:22:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.567776+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:22:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.582090+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:22:57.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.582821+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:22:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.583593+0000 mon.a (mon.0) 1913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:22:57.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:56 gibba032 bash[24004]: audit 2021-03-10T15:22:56.584287+0000 mon.a (mon.0) 1914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:22:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.567089+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:22:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.567776+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:22:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.582090+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:22:57.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.582821+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:22:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.583593+0000 mon.a (mon.0) 1913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:22:57.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[30791]: audit 2021-03-10T15:22:56.584287+0000 mon.a (mon.0) 1914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:22:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.567089+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:22:57.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.567776+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:22:57.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.582090+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:22:57.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.582821+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:22:57.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.583593+0000 mon.a (mon.0) 1913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:22:57.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:56 gibba011 bash[31774]: audit 2021-03-10T15:22:56.584287+0000 mon.a (mon.0) 1914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:22:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:57 gibba032 bash[24004]: cluster 2021-03-10T15:22:56.351659+0000 mgr.gibba032.whopeo (mgr.24625) 4768 : cluster [DBG] pgmap v2713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:57 gibba011 bash[30791]: cluster 2021-03-10T15:22:56.351659+0000 mgr.gibba032.whopeo (mgr.24625) 4768 : cluster [DBG] pgmap v2713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:57 gibba011 bash[31774]: cluster 2021-03-10T15:22:56.351659+0000 mgr.gibba032.whopeo (mgr.24625) 4768 : cluster [DBG] pgmap v2713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:22:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.290535+0000 mgr.gibba032.whopeo (mgr.24625) 4769 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:22:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.292360+0000 mgr.gibba032.whopeo (mgr.24625) 4770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:22:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.499496+0000 mon.a (mon.0) 1915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:22:59.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.515977+0000 mon.a (mon.0) 1916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:22:59.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.520105+0000 mon.a (mon.0) 1917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.523967+0000 mon.a (mon.0) 1918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.527322+0000 mon.a (mon.0) 1919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.535570+0000 mon.a (mon.0) 1920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:58 gibba032 bash[24004]: audit 2021-03-10T15:22:58.541374+0000 mon.a (mon.0) 1921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.290535+0000 mgr.gibba032.whopeo (mgr.24625) 4769 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:22:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.292360+0000 mgr.gibba032.whopeo (mgr.24625) 4770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:22:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.499496+0000 mon.a (mon.0) 1915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:22:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.515977+0000 mon.a (mon.0) 1916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:22:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.520105+0000 mon.a (mon.0) 1917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.523967+0000 mon.a (mon.0) 1918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.527322+0000 mon.a (mon.0) 1919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.535570+0000 mon.a (mon.0) 1920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[30791]: audit 2021-03-10T15:22:58.541374+0000 mon.a (mon.0) 1921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.290535+0000 mgr.gibba032.whopeo (mgr.24625) 4769 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:22:59.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.292360+0000 mgr.gibba032.whopeo (mgr.24625) 4770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:22:59.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.499496+0000 mon.a (mon.0) 1915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:22:59.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.515977+0000 mon.a (mon.0) 1916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:22:59.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.520105+0000 mon.a (mon.0) 1917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.523967+0000 mon.a (mon.0) 1918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.527322+0000 mon.a (mon.0) 1919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.535570+0000 mon.a (mon.0) 1920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:22:59.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:58 gibba011 bash[31774]: audit 2021-03-10T15:22:58.541374+0000 mon.a (mon.0) 1921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:00.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cluster 2021-03-10T15:22:58.352263+0000 mgr.gibba032.whopeo (mgr.24625) 4771 : cluster [DBG] pgmap v2714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.511921+0000 mgr.gibba032.whopeo (mgr.24625) 4772 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:23:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.517721+0000 mgr.gibba032.whopeo (mgr.24625) 4773 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:23:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.521200+0000 mgr.gibba032.whopeo (mgr.24625) 4774 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:23:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.521429+0000 mgr.gibba032.whopeo (mgr.24625) 4775 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:23:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.524957+0000 mgr.gibba032.whopeo (mgr.24625) 4776 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:23:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.525277+0000 mgr.gibba032.whopeo (mgr.24625) 4777 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:23:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.528687+0000 mgr.gibba032.whopeo (mgr.24625) 4778 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:23:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.532145+0000 mgr.gibba032.whopeo (mgr.24625) 4779 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:23:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.532393+0000 mgr.gibba032.whopeo (mgr.24625) 4780 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:23:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.536927+0000 mgr.gibba032.whopeo (mgr.24625) 4781 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:23:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.539050+0000 mgr.gibba032.whopeo (mgr.24625) 4782 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:23:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.542805+0000 mgr.gibba032.whopeo (mgr.24625) 4783 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:23:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.543039+0000 mgr.gibba032.whopeo (mgr.24625) 4784 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:23:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[30791]: cephadm 2021-03-10T15:22:58.543936+0000 mgr.gibba032.whopeo (mgr.24625) 4785 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:00.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cluster 2021-03-10T15:22:58.352263+0000 mgr.gibba032.whopeo (mgr.24625) 4771 : cluster [DBG] pgmap v2714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:00.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.511921+0000 mgr.gibba032.whopeo (mgr.24625) 4772 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:23:00.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.517721+0000 mgr.gibba032.whopeo (mgr.24625) 4773 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:23:00.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.521200+0000 mgr.gibba032.whopeo (mgr.24625) 4774 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:23:00.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.521429+0000 mgr.gibba032.whopeo (mgr.24625) 4775 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:23:00.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.524957+0000 mgr.gibba032.whopeo (mgr.24625) 4776 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:23:00.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.525277+0000 mgr.gibba032.whopeo (mgr.24625) 4777 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:23:00.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.528687+0000 mgr.gibba032.whopeo (mgr.24625) 4778 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:23:00.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.532145+0000 mgr.gibba032.whopeo (mgr.24625) 4779 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:23:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.532393+0000 mgr.gibba032.whopeo (mgr.24625) 4780 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:23:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.536927+0000 mgr.gibba032.whopeo (mgr.24625) 4781 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:23:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.539050+0000 mgr.gibba032.whopeo (mgr.24625) 4782 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:23:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.542805+0000 mgr.gibba032.whopeo (mgr.24625) 4783 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:23:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.543039+0000 mgr.gibba032.whopeo (mgr.24625) 4784 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:23:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:22:59 gibba011 bash[31774]: cephadm 2021-03-10T15:22:58.543936+0000 mgr.gibba032.whopeo (mgr.24625) 4785 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cluster 2021-03-10T15:22:58.352263+0000 mgr.gibba032.whopeo (mgr.24625) 4771 : cluster [DBG] pgmap v2714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.511921+0000 mgr.gibba032.whopeo (mgr.24625) 4772 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:23:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.517721+0000 mgr.gibba032.whopeo (mgr.24625) 4773 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:23:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.521200+0000 mgr.gibba032.whopeo (mgr.24625) 4774 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:23:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.521429+0000 mgr.gibba032.whopeo (mgr.24625) 4775 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:23:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.524957+0000 mgr.gibba032.whopeo (mgr.24625) 4776 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:23:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.525277+0000 mgr.gibba032.whopeo (mgr.24625) 4777 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:23:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.528687+0000 mgr.gibba032.whopeo (mgr.24625) 4778 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:23:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.532145+0000 mgr.gibba032.whopeo (mgr.24625) 4779 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:23:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.532393+0000 mgr.gibba032.whopeo (mgr.24625) 4780 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:23:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.536927+0000 mgr.gibba032.whopeo (mgr.24625) 4781 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:23:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.539050+0000 mgr.gibba032.whopeo (mgr.24625) 4782 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:23:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.542805+0000 mgr.gibba032.whopeo (mgr.24625) 4783 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:23:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.543039+0000 mgr.gibba032.whopeo (mgr.24625) 4784 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:23:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:22:59 gibba032 bash[24004]: cephadm 2021-03-10T15:22:58.543936+0000 mgr.gibba032.whopeo (mgr.24625) 4785 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:00.893 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:23:01.121 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:23:01.122 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (89m) 4m ago 95m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:23:01.122 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (94m) 3m ago 94m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:23:01.122 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (90m) 3m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:23:01.122 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (90m) 3m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:23:01.123 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (89m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:23:01.123 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (89m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:23:01.123 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (88m) 3m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:23:01.123 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (88m) 4m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:23:01.124 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (95m) 4m ago 95m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:23:01.124 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (95m) 3m ago 95m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:23:01.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (87m) 4m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:23:01.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:23:01.124 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:23:01.125 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (86m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:23:01.125 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:23:01.125 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (85m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:23:01.125 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (85m) 3m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:23:01.126 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (84m) 3m ago 95m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:23:01.126 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (94m) 3m ago 95m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:23:01.126 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 93m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:23:01.369 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:23:01.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:23:01.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:23:01.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:01.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:23:01.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:23:01.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:01.371 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:23:01.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:23:01.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:01.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:23:01.372 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:23:01.372 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:23:01.372 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:23:01.373 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:23:02.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[30791]: cluster 2021-03-10T15:23:00.353430+0000 mgr.gibba032.whopeo (mgr.24625) 4786 : cluster [DBG] pgmap v2715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:02.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[30791]: audit 2021-03-10T15:23:00.879901+0000 mgr.gibba032.whopeo (mgr.24625) 4787 : audit [DBG] from='client.47259 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[30791]: audit 2021-03-10T15:23:01.104739+0000 mgr.gibba032.whopeo (mgr.24625) 4788 : audit [DBG] from='client.47265 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[30791]: audit 2021-03-10T15:23:01.366481+0000 mon.a (mon.0) 1922 : audit [DBG] from='client.? 172.21.2.111:0/3688335429' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:02.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[31774]: cluster 2021-03-10T15:23:00.353430+0000 mgr.gibba032.whopeo (mgr.24625) 4786 : cluster [DBG] pgmap v2715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:02.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[31774]: audit 2021-03-10T15:23:00.879901+0000 mgr.gibba032.whopeo (mgr.24625) 4787 : audit [DBG] from='client.47259 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[31774]: audit 2021-03-10T15:23:01.104739+0000 mgr.gibba032.whopeo (mgr.24625) 4788 : audit [DBG] from='client.47265 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:01 gibba011 bash[31774]: audit 2021-03-10T15:23:01.366481+0000 mon.a (mon.0) 1922 : audit [DBG] from='client.? 172.21.2.111:0/3688335429' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:01 gibba032 bash[24004]: cluster 2021-03-10T15:23:00.353430+0000 mgr.gibba032.whopeo (mgr.24625) 4786 : cluster [DBG] pgmap v2715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:01 gibba032 bash[24004]: audit 2021-03-10T15:23:00.879901+0000 mgr.gibba032.whopeo (mgr.24625) 4787 : audit [DBG] from='client.47259 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:01 gibba032 bash[24004]: audit 2021-03-10T15:23:01.104739+0000 mgr.gibba032.whopeo (mgr.24625) 4788 : audit [DBG] from='client.47265 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:01 gibba032 bash[24004]: audit 2021-03-10T15:23:01.366481+0000 mon.a (mon.0) 1922 : audit [DBG] from='client.? 172.21.2.111:0/3688335429' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:04.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:03 gibba011 bash[30791]: cluster 2021-03-10T15:23:02.355050+0000 mgr.gibba032.whopeo (mgr.24625) 4789 : cluster [DBG] pgmap v2716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:04.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:03 gibba011 bash[31774]: cluster 2021-03-10T15:23:02.355050+0000 mgr.gibba032.whopeo (mgr.24625) 4789 : cluster [DBG] pgmap v2716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:03 gibba032 bash[24004]: cluster 2021-03-10T15:23:02.355050+0000 mgr.gibba032.whopeo (mgr.24625) 4789 : cluster [DBG] pgmap v2716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:06.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:05 gibba011 bash[30791]: cluster 2021-03-10T15:23:04.356435+0000 mgr.gibba032.whopeo (mgr.24625) 4790 : cluster [DBG] pgmap v2717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:06.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:05 gibba011 bash[31774]: cluster 2021-03-10T15:23:04.356435+0000 mgr.gibba032.whopeo (mgr.24625) 4790 : cluster [DBG] pgmap v2717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:06.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:05 gibba032 bash[24004]: cluster 2021-03-10T15:23:04.356435+0000 mgr.gibba032.whopeo (mgr.24625) 4790 : cluster [DBG] pgmap v2717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:07 gibba011 bash[30791]: cluster 2021-03-10T15:23:06.357910+0000 mgr.gibba032.whopeo (mgr.24625) 4791 : cluster [DBG] pgmap v2718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:08.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:07 gibba011 bash[31774]: cluster 2021-03-10T15:23:06.357910+0000 mgr.gibba032.whopeo (mgr.24625) 4791 : cluster [DBG] pgmap v2718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:07 gibba032 bash[24004]: cluster 2021-03-10T15:23:06.357910+0000 mgr.gibba032.whopeo (mgr.24625) 4791 : cluster [DBG] pgmap v2718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:10.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:09 gibba011 bash[30791]: cluster 2021-03-10T15:23:08.359674+0000 mgr.gibba032.whopeo (mgr.24625) 4792 : cluster [DBG] pgmap v2719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:10.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:09 gibba011 bash[31774]: cluster 2021-03-10T15:23:08.359674+0000 mgr.gibba032.whopeo (mgr.24625) 4792 : cluster [DBG] pgmap v2719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:09 gibba032 bash[24004]: cluster 2021-03-10T15:23:08.359674+0000 mgr.gibba032.whopeo (mgr.24625) 4792 : cluster [DBG] pgmap v2719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:12.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:11 gibba011 bash[30791]: cluster 2021-03-10T15:23:10.361138+0000 mgr.gibba032.whopeo (mgr.24625) 4793 : cluster [DBG] pgmap v2720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:12.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:11 gibba011 bash[31774]: cluster 2021-03-10T15:23:10.361138+0000 mgr.gibba032.whopeo (mgr.24625) 4793 : cluster [DBG] pgmap v2720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:11 gibba032 bash[24004]: cluster 2021-03-10T15:23:10.361138+0000 mgr.gibba032.whopeo (mgr.24625) 4793 : cluster [DBG] pgmap v2720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:13 gibba032 bash[24004]: cluster 2021-03-10T15:23:12.362871+0000 mgr.gibba032.whopeo (mgr.24625) 4794 : cluster [DBG] pgmap v2721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:13 gibba011 bash[30791]: cluster 2021-03-10T15:23:12.362871+0000 mgr.gibba032.whopeo (mgr.24625) 4794 : cluster [DBG] pgmap v2721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:13 gibba011 bash[31774]: cluster 2021-03-10T15:23:12.362871+0000 mgr.gibba032.whopeo (mgr.24625) 4794 : cluster [DBG] pgmap v2721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:15.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:14 gibba032 bash[24004]: cephadm 2021-03-10T15:23:13.559898+0000 mgr.gibba032.whopeo (mgr.24625) 4795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:14 gibba011 bash[30791]: cephadm 2021-03-10T15:23:13.559898+0000 mgr.gibba032.whopeo (mgr.24625) 4795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:14 gibba011 bash[31774]: cephadm 2021-03-10T15:23:13.559898+0000 mgr.gibba032.whopeo (mgr.24625) 4795 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:16.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:15 gibba032 bash[24004]: cluster 2021-03-10T15:23:14.363859+0000 mgr.gibba032.whopeo (mgr.24625) 4796 : cluster [DBG] pgmap v2722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:15 gibba011 bash[30791]: cluster 2021-03-10T15:23:14.363859+0000 mgr.gibba032.whopeo (mgr.24625) 4796 : cluster [DBG] pgmap v2722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:15 gibba011 bash[31774]: cluster 2021-03-10T15:23:14.363859+0000 mgr.gibba032.whopeo (mgr.24625) 4796 : cluster [DBG] pgmap v2722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:17 gibba032 bash[24004]: cluster 2021-03-10T15:23:16.365252+0000 mgr.gibba032.whopeo (mgr.24625) 4797 : cluster [DBG] pgmap v2723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:17 gibba011 bash[30791]: cluster 2021-03-10T15:23:16.365252+0000 mgr.gibba032.whopeo (mgr.24625) 4797 : cluster [DBG] pgmap v2723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:17 gibba011 bash[31774]: cluster 2021-03-10T15:23:16.365252+0000 mgr.gibba032.whopeo (mgr.24625) 4797 : cluster [DBG] pgmap v2723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:19 gibba032 bash[24004]: cluster 2021-03-10T15:23:18.366890+0000 mgr.gibba032.whopeo (mgr.24625) 4798 : cluster [DBG] pgmap v2724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:19 gibba011 bash[30791]: cluster 2021-03-10T15:23:18.366890+0000 mgr.gibba032.whopeo (mgr.24625) 4798 : cluster [DBG] pgmap v2724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:19 gibba011 bash[31774]: cluster 2021-03-10T15:23:18.366890+0000 mgr.gibba032.whopeo (mgr.24625) 4798 : cluster [DBG] pgmap v2724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:21 gibba032 bash[24004]: cluster 2021-03-10T15:23:20.368397+0000 mgr.gibba032.whopeo (mgr.24625) 4799 : cluster [DBG] pgmap v2725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:21 gibba011 bash[30791]: cluster 2021-03-10T15:23:20.368397+0000 mgr.gibba032.whopeo (mgr.24625) 4799 : cluster [DBG] pgmap v2725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:21 gibba011 bash[31774]: cluster 2021-03-10T15:23:20.368397+0000 mgr.gibba032.whopeo (mgr.24625) 4799 : cluster [DBG] pgmap v2725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:23 gibba032 bash[24004]: cluster 2021-03-10T15:23:22.369438+0000 mgr.gibba032.whopeo (mgr.24625) 4800 : cluster [DBG] pgmap v2726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:23 gibba011 bash[30791]: cluster 2021-03-10T15:23:22.369438+0000 mgr.gibba032.whopeo (mgr.24625) 4800 : cluster [DBG] pgmap v2726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:23 gibba011 bash[31774]: cluster 2021-03-10T15:23:22.369438+0000 mgr.gibba032.whopeo (mgr.24625) 4800 : cluster [DBG] pgmap v2726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:25 gibba032 bash[24004]: cluster 2021-03-10T15:23:24.370628+0000 mgr.gibba032.whopeo (mgr.24625) 4801 : cluster [DBG] pgmap v2727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:25 gibba011 bash[30791]: cluster 2021-03-10T15:23:24.370628+0000 mgr.gibba032.whopeo (mgr.24625) 4801 : cluster [DBG] pgmap v2727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:25 gibba011 bash[31774]: cluster 2021-03-10T15:23:24.370628+0000 mgr.gibba032.whopeo (mgr.24625) 4801 : cluster [DBG] pgmap v2727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:27 gibba032 bash[24004]: cluster 2021-03-10T15:23:26.372141+0000 mgr.gibba032.whopeo (mgr.24625) 4802 : cluster [DBG] pgmap v2728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:27 gibba011 bash[30791]: cluster 2021-03-10T15:23:26.372141+0000 mgr.gibba032.whopeo (mgr.24625) 4802 : cluster [DBG] pgmap v2728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:27 gibba011 bash[31774]: cluster 2021-03-10T15:23:26.372141+0000 mgr.gibba032.whopeo (mgr.24625) 4802 : cluster [DBG] pgmap v2728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:29 gibba032 bash[24004]: cluster 2021-03-10T15:23:28.373976+0000 mgr.gibba032.whopeo (mgr.24625) 4803 : cluster [DBG] pgmap v2729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:29 gibba032 bash[24004]: cephadm 2021-03-10T15:23:28.575799+0000 mgr.gibba032.whopeo (mgr.24625) 4804 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:29 gibba011 bash[30791]: cluster 2021-03-10T15:23:28.373976+0000 mgr.gibba032.whopeo (mgr.24625) 4803 : cluster [DBG] pgmap v2729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:29 gibba011 bash[30791]: cephadm 2021-03-10T15:23:28.575799+0000 mgr.gibba032.whopeo (mgr.24625) 4804 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:29 gibba011 bash[31774]: cluster 2021-03-10T15:23:28.373976+0000 mgr.gibba032.whopeo (mgr.24625) 4803 : cluster [DBG] pgmap v2729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:29 gibba011 bash[31774]: cephadm 2021-03-10T15:23:28.575799+0000 mgr.gibba032.whopeo (mgr.24625) 4804 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:31.612 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:23:31.840 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:23:31.840 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (90m) 4m ago 95m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:23:31.841 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (95m) 3m ago 95m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:23:31.841 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (90m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:23:31.841 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (90m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:23:31.841 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (89m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:23:31.842 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (89m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:23:31.842 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (89m) 3m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:23:31.842 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (89m) 4m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:23:31.842 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (95m) 4m ago 95m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:23:31.843 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (95m) 3m ago 95m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:23:31.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (88m) 4m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:23:31.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (88m) 4m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:23:31.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:23:31.843 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:23:31.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (86m) 3m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:23:31.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (86m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:23:31.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (85m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:23:31.844 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (85m) 3m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:23:31.845 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (95m) 3m ago 95m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:23:31.845 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 93m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:23:32.095 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:23:32.095 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:23:32.095 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:23:32.096 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:32.096 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:23:32.096 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:23:32.096 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:32.097 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:23:32.097 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:23:32.097 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:23:32.097 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:23:32.098 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:23:32.098 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:23:32.098 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:23:32.098 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:23:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:31 gibba032 bash[24004]: cluster 2021-03-10T15:23:30.375223+0000 mgr.gibba032.whopeo (mgr.24625) 4805 : cluster [DBG] pgmap v2730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:31 gibba011 bash[30791]: cluster 2021-03-10T15:23:30.375223+0000 mgr.gibba032.whopeo (mgr.24625) 4805 : cluster [DBG] pgmap v2730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:31 gibba011 bash[31774]: cluster 2021-03-10T15:23:30.375223+0000 mgr.gibba032.whopeo (mgr.24625) 4805 : cluster [DBG] pgmap v2730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:32 gibba032 bash[24004]: audit 2021-03-10T15:23:31.599056+0000 mgr.gibba032.whopeo (mgr.24625) 4806 : audit [DBG] from='client.57262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:32 gibba032 bash[24004]: audit 2021-03-10T15:23:31.824386+0000 mgr.gibba032.whopeo (mgr.24625) 4807 : audit [DBG] from='client.47283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:32 gibba032 bash[24004]: audit 2021-03-10T15:23:32.091824+0000 mon.a (mon.0) 1923 : audit [DBG] from='client.? 172.21.2.111:0/1445759507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:33.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[30791]: audit 2021-03-10T15:23:31.599056+0000 mgr.gibba032.whopeo (mgr.24625) 4806 : audit [DBG] from='client.57262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[30791]: audit 2021-03-10T15:23:31.824386+0000 mgr.gibba032.whopeo (mgr.24625) 4807 : audit [DBG] from='client.47283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[30791]: audit 2021-03-10T15:23:32.091824+0000 mon.a (mon.0) 1923 : audit [DBG] from='client.? 172.21.2.111:0/1445759507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[31774]: audit 2021-03-10T15:23:31.599056+0000 mgr.gibba032.whopeo (mgr.24625) 4806 : audit [DBG] from='client.57262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[31774]: audit 2021-03-10T15:23:31.824386+0000 mgr.gibba032.whopeo (mgr.24625) 4807 : audit [DBG] from='client.47283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:23:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:32 gibba011 bash[31774]: audit 2021-03-10T15:23:32.091824+0000 mon.a (mon.0) 1923 : audit [DBG] from='client.? 172.21.2.111:0/1445759507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:23:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:33 gibba032 bash[24004]: cluster 2021-03-10T15:23:32.376780+0000 mgr.gibba032.whopeo (mgr.24625) 4808 : cluster [DBG] pgmap v2731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:33 gibba011 bash[30791]: cluster 2021-03-10T15:23:32.376780+0000 mgr.gibba032.whopeo (mgr.24625) 4808 : cluster [DBG] pgmap v2731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:33 gibba011 bash[31774]: cluster 2021-03-10T15:23:32.376780+0000 mgr.gibba032.whopeo (mgr.24625) 4808 : cluster [DBG] pgmap v2731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:35 gibba032 bash[24004]: cluster 2021-03-10T15:23:34.378100+0000 mgr.gibba032.whopeo (mgr.24625) 4809 : cluster [DBG] pgmap v2732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:35 gibba011 bash[30791]: cluster 2021-03-10T15:23:34.378100+0000 mgr.gibba032.whopeo (mgr.24625) 4809 : cluster [DBG] pgmap v2732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:35 gibba011 bash[31774]: cluster 2021-03-10T15:23:34.378100+0000 mgr.gibba032.whopeo (mgr.24625) 4809 : cluster [DBG] pgmap v2732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:37 gibba032 bash[24004]: cluster 2021-03-10T15:23:36.379667+0000 mgr.gibba032.whopeo (mgr.24625) 4810 : cluster [DBG] pgmap v2733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:37 gibba011 bash[30791]: cluster 2021-03-10T15:23:36.379667+0000 mgr.gibba032.whopeo (mgr.24625) 4810 : cluster [DBG] pgmap v2733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:37 gibba011 bash[31774]: cluster 2021-03-10T15:23:36.379667+0000 mgr.gibba032.whopeo (mgr.24625) 4810 : cluster [DBG] pgmap v2733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:39 gibba032 bash[24004]: cluster 2021-03-10T15:23:38.381204+0000 mgr.gibba032.whopeo (mgr.24625) 4811 : cluster [DBG] pgmap v2734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:39 gibba011 bash[30791]: cluster 2021-03-10T15:23:38.381204+0000 mgr.gibba032.whopeo (mgr.24625) 4811 : cluster [DBG] pgmap v2734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:39 gibba011 bash[31774]: cluster 2021-03-10T15:23:38.381204+0000 mgr.gibba032.whopeo (mgr.24625) 4811 : cluster [DBG] pgmap v2734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:41 gibba032 bash[24004]: cluster 2021-03-10T15:23:40.382725+0000 mgr.gibba032.whopeo (mgr.24625) 4812 : cluster [DBG] pgmap v2735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:41 gibba011 bash[30791]: cluster 2021-03-10T15:23:40.382725+0000 mgr.gibba032.whopeo (mgr.24625) 4812 : cluster [DBG] pgmap v2735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:41 gibba011 bash[31774]: cluster 2021-03-10T15:23:40.382725+0000 mgr.gibba032.whopeo (mgr.24625) 4812 : cluster [DBG] pgmap v2735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:43 gibba032 bash[24004]: cluster 2021-03-10T15:23:42.384406+0000 mgr.gibba032.whopeo (mgr.24625) 4813 : cluster [DBG] pgmap v2736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:43 gibba011 bash[30791]: cluster 2021-03-10T15:23:42.384406+0000 mgr.gibba032.whopeo (mgr.24625) 4813 : cluster [DBG] pgmap v2736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:43 gibba011 bash[31774]: cluster 2021-03-10T15:23:42.384406+0000 mgr.gibba032.whopeo (mgr.24625) 4813 : cluster [DBG] pgmap v2736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:44 gibba032 bash[24004]: cephadm 2021-03-10T15:23:43.580065+0000 mgr.gibba032.whopeo (mgr.24625) 4814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:44 gibba011 bash[30791]: cephadm 2021-03-10T15:23:43.580065+0000 mgr.gibba032.whopeo (mgr.24625) 4814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:44 gibba011 bash[31774]: cephadm 2021-03-10T15:23:43.580065+0000 mgr.gibba032.whopeo (mgr.24625) 4814 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:23:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:45 gibba032 bash[24004]: cluster 2021-03-10T15:23:44.385527+0000 mgr.gibba032.whopeo (mgr.24625) 4815 : cluster [DBG] pgmap v2737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:45 gibba011 bash[30791]: cluster 2021-03-10T15:23:44.385527+0000 mgr.gibba032.whopeo (mgr.24625) 4815 : cluster [DBG] pgmap v2737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:45 gibba011 bash[31774]: cluster 2021-03-10T15:23:44.385527+0000 mgr.gibba032.whopeo (mgr.24625) 4815 : cluster [DBG] pgmap v2737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:48 gibba032 bash[24004]: cluster 2021-03-10T15:23:46.387245+0000 mgr.gibba032.whopeo (mgr.24625) 4816 : cluster [DBG] pgmap v2738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:48 gibba011 bash[30791]: cluster 2021-03-10T15:23:46.387245+0000 mgr.gibba032.whopeo (mgr.24625) 4816 : cluster [DBG] pgmap v2738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:48 gibba011 bash[31774]: cluster 2021-03-10T15:23:46.387245+0000 mgr.gibba032.whopeo (mgr.24625) 4816 : cluster [DBG] pgmap v2738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:50 gibba011 bash[30791]: cluster 2021-03-10T15:23:48.389110+0000 mgr.gibba032.whopeo (mgr.24625) 4817 : cluster [DBG] pgmap v2739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:50 gibba011 bash[31774]: cluster 2021-03-10T15:23:48.389110+0000 mgr.gibba032.whopeo (mgr.24625) 4817 : cluster [DBG] pgmap v2739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:50 gibba032 bash[24004]: cluster 2021-03-10T15:23:48.389110+0000 mgr.gibba032.whopeo (mgr.24625) 4817 : cluster [DBG] pgmap v2739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:52 gibba011 bash[30791]: cluster 2021-03-10T15:23:50.390715+0000 mgr.gibba032.whopeo (mgr.24625) 4818 : cluster [DBG] pgmap v2740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:52 gibba011 bash[31774]: cluster 2021-03-10T15:23:50.390715+0000 mgr.gibba032.whopeo (mgr.24625) 4818 : cluster [DBG] pgmap v2740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:52 gibba032 bash[24004]: cluster 2021-03-10T15:23:50.390715+0000 mgr.gibba032.whopeo (mgr.24625) 4818 : cluster [DBG] pgmap v2740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:54 gibba011 bash[30791]: cluster 2021-03-10T15:23:52.392520+0000 mgr.gibba032.whopeo (mgr.24625) 4819 : cluster [DBG] pgmap v2741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:54 gibba011 bash[31774]: cluster 2021-03-10T15:23:52.392520+0000 mgr.gibba032.whopeo (mgr.24625) 4819 : cluster [DBG] pgmap v2741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:54 gibba032 bash[24004]: cluster 2021-03-10T15:23:52.392520+0000 mgr.gibba032.whopeo (mgr.24625) 4819 : cluster [DBG] pgmap v2741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:56 gibba011 bash[30791]: cluster 2021-03-10T15:23:54.393849+0000 mgr.gibba032.whopeo (mgr.24625) 4820 : cluster [DBG] pgmap v2742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:56 gibba011 bash[31774]: cluster 2021-03-10T15:23:54.393849+0000 mgr.gibba032.whopeo (mgr.24625) 4820 : cluster [DBG] pgmap v2742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:56 gibba032 bash[24004]: cluster 2021-03-10T15:23:54.393849+0000 mgr.gibba032.whopeo (mgr.24625) 4820 : cluster [DBG] pgmap v2742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.574815+0000 mon.a (mon.0) 1924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:23:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.575507+0000 mon.a (mon.0) 1925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:23:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.587635+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:23:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.588371+0000 mon.a (mon.0) 1927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T15:23:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.589077+0000 mon.a (mon.0) 1928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[30791]: audit 2021-03-10T15:23:56.589755+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.574815+0000 mon.a (mon.0) 1924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:23:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.575507+0000 mon.a (mon.0) 1925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:23:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.587635+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:23:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.588371+0000 mon.a (mon.0) 1927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T15:23:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.589077+0000 mon.a (mon.0) 1928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:57 gibba011 bash[31774]: audit 2021-03-10T15:23:56.589755+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.574815+0000 mon.a (mon.0) 1924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:23:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.575507+0000 mon.a (mon.0) 1925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:23:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.587635+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:23:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.588371+0000 mon.a (mon.0) 1927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T15:23:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.589077+0000 mon.a (mon.0) 1928 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:57 gibba032 bash[24004]: audit 2021-03-10T15:23:56.589755+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:23:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:58 gibba011 bash[30791]: cluster 2021-03-10T15:23:56.395349+0000 mgr.gibba032.whopeo (mgr.24625) 4821 : cluster [DBG] pgmap v2743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:58 gibba011 bash[31774]: cluster 2021-03-10T15:23:56.395349+0000 mgr.gibba032.whopeo (mgr.24625) 4821 : cluster [DBG] pgmap v2743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:23:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:58 gibba032 bash[24004]: cluster 2021-03-10T15:23:56.395349+0000 mgr.gibba032.whopeo (mgr.24625) 4821 : cluster [DBG] pgmap v2743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cluster 2021-03-10T15:23:58.397143+0000 mgr.gibba032.whopeo (mgr.24625) 4822 : cluster [DBG] pgmap v2744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.596956+0000 mgr.gibba032.whopeo (mgr.24625) 4823 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:24:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.598523+0000 mgr.gibba032.whopeo (mgr.24625) 4824 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:24:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.790418+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:24:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.791785+0000 mgr.gibba032.whopeo (mgr.24625) 4825 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:24:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.794105+0000 mon.a (mon.0) 1931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:24:00.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.794351+0000 mgr.gibba032.whopeo (mgr.24625) 4826 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:24:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.795827+0000 mgr.gibba032.whopeo (mgr.24625) 4827 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:24:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.795867+0000 mgr.gibba032.whopeo (mgr.24625) 4828 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:24:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.795981+0000 mon.a (mon.0) 1932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.797513+0000 mon.a (mon.0) 1933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.797555+0000 mgr.gibba032.whopeo (mgr.24625) 4829 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:24:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.797592+0000 mgr.gibba032.whopeo (mgr.24625) 4830 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:24:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.799173+0000 mon.a (mon.0) 1934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.799288+0000 mgr.gibba032.whopeo (mgr.24625) 4831 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:24:00.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.800534+0000 mgr.gibba032.whopeo (mgr.24625) 4832 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:24:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.800573+0000 mgr.gibba032.whopeo (mgr.24625) 4833 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:24:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.802138+0000 mgr.gibba032.whopeo (mgr.24625) 4834 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:24:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.802286+0000 mon.a (mon.0) 1935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.802494+0000 mgr.gibba032.whopeo (mgr.24625) 4835 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:24:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.803868+0000 mgr.gibba032.whopeo (mgr.24625) 4836 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:24:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.803911+0000 mgr.gibba032.whopeo (mgr.24625) 4837 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:24:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: cephadm 2021-03-10T15:23:58.804031+0000 mgr.gibba032.whopeo (mgr.24625) 4838 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:00.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[30791]: audit 2021-03-10T15:23:58.804045+0000 mon.a (mon.0) 1936 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cluster 2021-03-10T15:23:58.397143+0000 mgr.gibba032.whopeo (mgr.24625) 4822 : cluster [DBG] pgmap v2744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.596956+0000 mgr.gibba032.whopeo (mgr.24625) 4823 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:24:00.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.598523+0000 mgr.gibba032.whopeo (mgr.24625) 4824 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:24:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.790418+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:24:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.791785+0000 mgr.gibba032.whopeo (mgr.24625) 4825 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:24:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.794105+0000 mon.a (mon.0) 1931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:24:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.794351+0000 mgr.gibba032.whopeo (mgr.24625) 4826 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:24:00.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.795827+0000 mgr.gibba032.whopeo (mgr.24625) 4827 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:24:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.795867+0000 mgr.gibba032.whopeo (mgr.24625) 4828 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:24:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.795981+0000 mon.a (mon.0) 1932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.797513+0000 mon.a (mon.0) 1933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.797555+0000 mgr.gibba032.whopeo (mgr.24625) 4829 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:24:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.797592+0000 mgr.gibba032.whopeo (mgr.24625) 4830 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:24:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.799173+0000 mon.a (mon.0) 1934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.799288+0000 mgr.gibba032.whopeo (mgr.24625) 4831 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:24:00.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.800534+0000 mgr.gibba032.whopeo (mgr.24625) 4832 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:24:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.800573+0000 mgr.gibba032.whopeo (mgr.24625) 4833 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:24:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.802138+0000 mgr.gibba032.whopeo (mgr.24625) 4834 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:24:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.802286+0000 mon.a (mon.0) 1935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.802494+0000 mgr.gibba032.whopeo (mgr.24625) 4835 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:24:00.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.803868+0000 mgr.gibba032.whopeo (mgr.24625) 4836 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:24:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.803911+0000 mgr.gibba032.whopeo (mgr.24625) 4837 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:24:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: cephadm 2021-03-10T15:23:58.804031+0000 mgr.gibba032.whopeo (mgr.24625) 4838 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:00.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:23:59 gibba011 bash[31774]: audit 2021-03-10T15:23:58.804045+0000 mon.a (mon.0) 1936 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cluster 2021-03-10T15:23:58.397143+0000 mgr.gibba032.whopeo (mgr.24625) 4822 : cluster [DBG] pgmap v2744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.596956+0000 mgr.gibba032.whopeo (mgr.24625) 4823 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:24:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.598523+0000 mgr.gibba032.whopeo (mgr.24625) 4824 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:24:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.790418+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:24:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.791785+0000 mgr.gibba032.whopeo (mgr.24625) 4825 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:24:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.794105+0000 mon.a (mon.0) 1931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:24:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.794351+0000 mgr.gibba032.whopeo (mgr.24625) 4826 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:24:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.795827+0000 mgr.gibba032.whopeo (mgr.24625) 4827 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:24:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.795867+0000 mgr.gibba032.whopeo (mgr.24625) 4828 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:24:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.795981+0000 mon.a (mon.0) 1932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.797513+0000 mon.a (mon.0) 1933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.797555+0000 mgr.gibba032.whopeo (mgr.24625) 4829 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:24:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.797592+0000 mgr.gibba032.whopeo (mgr.24625) 4830 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:24:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.799173+0000 mon.a (mon.0) 1934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.799288+0000 mgr.gibba032.whopeo (mgr.24625) 4831 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:24:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.800534+0000 mgr.gibba032.whopeo (mgr.24625) 4832 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:24:00.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.800573+0000 mgr.gibba032.whopeo (mgr.24625) 4833 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:24:00.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.802138+0000 mgr.gibba032.whopeo (mgr.24625) 4834 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:24:00.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.802286+0000 mon.a (mon.0) 1935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:00.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.802494+0000 mgr.gibba032.whopeo (mgr.24625) 4835 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:24:00.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.803868+0000 mgr.gibba032.whopeo (mgr.24625) 4836 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:24:00.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.803911+0000 mgr.gibba032.whopeo (mgr.24625) 4837 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:24:00.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: cephadm 2021-03-10T15:23:58.804031+0000 mgr.gibba032.whopeo (mgr.24625) 4838 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:00.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:23:59 gibba032 bash[24004]: audit 2021-03-10T15:23:58.804045+0000 mon.a (mon.0) 1936 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:02.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:01 gibba011 bash[30791]: cluster 2021-03-10T15:24:00.398795+0000 mgr.gibba032.whopeo (mgr.24625) 4839 : cluster [DBG] pgmap v2745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:02.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:01 gibba011 bash[31774]: cluster 2021-03-10T15:24:00.398795+0000 mgr.gibba032.whopeo (mgr.24625) 4839 : cluster [DBG] pgmap v2745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:01 gibba032 bash[24004]: cluster 2021-03-10T15:24:00.398795+0000 mgr.gibba032.whopeo (mgr.24625) 4839 : cluster [DBG] pgmap v2745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:02.338 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:24:02.597 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:24:02.598 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (90m) 5m ago 96m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:24:02.598 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (95m) 4m ago 95m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:24:02.599 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (91m) 4m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:24:02.599 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (91m) 4m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:24:02.599 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (90m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:24:02.599 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (90m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:24:02.600 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (89m) 4m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:24:02.600 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (89m) 5m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:24:02.600 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (96m) 5m ago 96m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:24:02.600 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (96m) 4m ago 96m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:24:02.601 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (88m) 5m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:24:02.601 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (88m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:24:02.601 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (88m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:24:02.601 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (87m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:24:02.601 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:24:02.602 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (86m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:24:02.602 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (86m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:24:02.602 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (85m) 4m ago 96m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:24:02.602 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (95m) 4m ago 96m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:24:02.603 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 94m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:24:02.855 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:24:02.855 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:24:02.856 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:24:02.856 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:02.856 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:24:02.856 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:24:02.857 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:02.857 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:24:02.857 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:24:02.857 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:02.858 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:24:02.858 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:24:02.858 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:24:02.858 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:24:02.858 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:24:04.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[30791]: audit 2021-03-10T15:24:02.326376+0000 mgr.gibba032.whopeo (mgr.24625) 4840 : audit [DBG] from='client.57280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[30791]: cluster 2021-03-10T15:24:02.400328+0000 mgr.gibba032.whopeo (mgr.24625) 4841 : cluster [DBG] pgmap v2746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:04.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[30791]: audit 2021-03-10T15:24:02.554593+0000 mgr.gibba032.whopeo (mgr.24625) 4842 : audit [DBG] from='client.47301 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[30791]: audit 2021-03-10T15:24:02.852324+0000 mon.c (mon.1) 90 : audit [DBG] from='client.? 172.21.2.111:0/2880566277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:04.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[31774]: audit 2021-03-10T15:24:02.326376+0000 mgr.gibba032.whopeo (mgr.24625) 4840 : audit [DBG] from='client.57280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[31774]: cluster 2021-03-10T15:24:02.400328+0000 mgr.gibba032.whopeo (mgr.24625) 4841 : cluster [DBG] pgmap v2746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:04.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[31774]: audit 2021-03-10T15:24:02.554593+0000 mgr.gibba032.whopeo (mgr.24625) 4842 : audit [DBG] from='client.47301 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:03 gibba011 bash[31774]: audit 2021-03-10T15:24:02.852324+0000 mon.c (mon.1) 90 : audit [DBG] from='client.? 172.21.2.111:0/2880566277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:03 gibba032 bash[24004]: audit 2021-03-10T15:24:02.326376+0000 mgr.gibba032.whopeo (mgr.24625) 4840 : audit [DBG] from='client.57280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:03 gibba032 bash[24004]: cluster 2021-03-10T15:24:02.400328+0000 mgr.gibba032.whopeo (mgr.24625) 4841 : cluster [DBG] pgmap v2746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:03 gibba032 bash[24004]: audit 2021-03-10T15:24:02.554593+0000 mgr.gibba032.whopeo (mgr.24625) 4842 : audit [DBG] from='client.47301 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:04.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:03 gibba032 bash[24004]: audit 2021-03-10T15:24:02.852324+0000 mon.c (mon.1) 90 : audit [DBG] from='client.? 172.21.2.111:0/2880566277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:06.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:05 gibba011 bash[30791]: cluster 2021-03-10T15:24:04.401766+0000 mgr.gibba032.whopeo (mgr.24625) 4843 : cluster [DBG] pgmap v2747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:06.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:05 gibba011 bash[31774]: cluster 2021-03-10T15:24:04.401766+0000 mgr.gibba032.whopeo (mgr.24625) 4843 : cluster [DBG] pgmap v2747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:06.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:05 gibba032 bash[24004]: cluster 2021-03-10T15:24:04.401766+0000 mgr.gibba032.whopeo (mgr.24625) 4843 : cluster [DBG] pgmap v2747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:08.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:07 gibba011 bash[30791]: cluster 2021-03-10T15:24:06.403413+0000 mgr.gibba032.whopeo (mgr.24625) 4844 : cluster [DBG] pgmap v2748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:08.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:07 gibba011 bash[31774]: cluster 2021-03-10T15:24:06.403413+0000 mgr.gibba032.whopeo (mgr.24625) 4844 : cluster [DBG] pgmap v2748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:07 gibba032 bash[24004]: cluster 2021-03-10T15:24:06.403413+0000 mgr.gibba032.whopeo (mgr.24625) 4844 : cluster [DBG] pgmap v2748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:09 gibba032 bash[24004]: cluster 2021-03-10T15:24:08.405098+0000 mgr.gibba032.whopeo (mgr.24625) 4845 : cluster [DBG] pgmap v2749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:09 gibba011 bash[30791]: cluster 2021-03-10T15:24:08.405098+0000 mgr.gibba032.whopeo (mgr.24625) 4845 : cluster [DBG] pgmap v2749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:09 gibba011 bash[31774]: cluster 2021-03-10T15:24:08.405098+0000 mgr.gibba032.whopeo (mgr.24625) 4845 : cluster [DBG] pgmap v2749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:11 gibba032 bash[24004]: cluster 2021-03-10T15:24:10.406672+0000 mgr.gibba032.whopeo (mgr.24625) 4846 : cluster [DBG] pgmap v2750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:11 gibba011 bash[30791]: cluster 2021-03-10T15:24:10.406672+0000 mgr.gibba032.whopeo (mgr.24625) 4846 : cluster [DBG] pgmap v2750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:12.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:11 gibba011 bash[31774]: cluster 2021-03-10T15:24:10.406672+0000 mgr.gibba032.whopeo (mgr.24625) 4846 : cluster [DBG] pgmap v2750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:13 gibba032 bash[24004]: cluster 2021-03-10T15:24:12.408480+0000 mgr.gibba032.whopeo (mgr.24625) 4847 : cluster [DBG] pgmap v2751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:13 gibba011 bash[30791]: cluster 2021-03-10T15:24:12.408480+0000 mgr.gibba032.whopeo (mgr.24625) 4847 : cluster [DBG] pgmap v2751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:13 gibba011 bash[31774]: cluster 2021-03-10T15:24:12.408480+0000 mgr.gibba032.whopeo (mgr.24625) 4847 : cluster [DBG] pgmap v2751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:15.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:14 gibba032 bash[24004]: cephadm 2021-03-10T15:24:13.820061+0000 mgr.gibba032.whopeo (mgr.24625) 4848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:14 gibba011 bash[30791]: cephadm 2021-03-10T15:24:13.820061+0000 mgr.gibba032.whopeo (mgr.24625) 4848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:15.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:14 gibba011 bash[31774]: cephadm 2021-03-10T15:24:13.820061+0000 mgr.gibba032.whopeo (mgr.24625) 4848 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:15 gibba032 bash[24004]: cluster 2021-03-10T15:24:14.409929+0000 mgr.gibba032.whopeo (mgr.24625) 4849 : cluster [DBG] pgmap v2752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:15 gibba011 bash[30791]: cluster 2021-03-10T15:24:14.409929+0000 mgr.gibba032.whopeo (mgr.24625) 4849 : cluster [DBG] pgmap v2752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:15 gibba011 bash[31774]: cluster 2021-03-10T15:24:14.409929+0000 mgr.gibba032.whopeo (mgr.24625) 4849 : cluster [DBG] pgmap v2752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:17 gibba032 bash[24004]: cluster 2021-03-10T15:24:16.411368+0000 mgr.gibba032.whopeo (mgr.24625) 4850 : cluster [DBG] pgmap v2753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:17 gibba011 bash[30791]: cluster 2021-03-10T15:24:16.411368+0000 mgr.gibba032.whopeo (mgr.24625) 4850 : cluster [DBG] pgmap v2753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:17 gibba011 bash[31774]: cluster 2021-03-10T15:24:16.411368+0000 mgr.gibba032.whopeo (mgr.24625) 4850 : cluster [DBG] pgmap v2753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:19 gibba032 bash[24004]: cluster 2021-03-10T15:24:18.413038+0000 mgr.gibba032.whopeo (mgr.24625) 4851 : cluster [DBG] pgmap v2754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:19 gibba011 bash[30791]: cluster 2021-03-10T15:24:18.413038+0000 mgr.gibba032.whopeo (mgr.24625) 4851 : cluster [DBG] pgmap v2754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:19 gibba011 bash[31774]: cluster 2021-03-10T15:24:18.413038+0000 mgr.gibba032.whopeo (mgr.24625) 4851 : cluster [DBG] pgmap v2754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:21 gibba032 bash[24004]: cluster 2021-03-10T15:24:20.414620+0000 mgr.gibba032.whopeo (mgr.24625) 4852 : cluster [DBG] pgmap v2755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:21 gibba011 bash[30791]: cluster 2021-03-10T15:24:20.414620+0000 mgr.gibba032.whopeo (mgr.24625) 4852 : cluster [DBG] pgmap v2755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:21 gibba011 bash[31774]: cluster 2021-03-10T15:24:20.414620+0000 mgr.gibba032.whopeo (mgr.24625) 4852 : cluster [DBG] pgmap v2755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:23 gibba032 bash[24004]: cluster 2021-03-10T15:24:22.416166+0000 mgr.gibba032.whopeo (mgr.24625) 4853 : cluster [DBG] pgmap v2756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:23 gibba011 bash[30791]: cluster 2021-03-10T15:24:22.416166+0000 mgr.gibba032.whopeo (mgr.24625) 4853 : cluster [DBG] pgmap v2756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:23 gibba011 bash[31774]: cluster 2021-03-10T15:24:22.416166+0000 mgr.gibba032.whopeo (mgr.24625) 4853 : cluster [DBG] pgmap v2756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:25 gibba032 bash[24004]: cluster 2021-03-10T15:24:24.417419+0000 mgr.gibba032.whopeo (mgr.24625) 4854 : cluster [DBG] pgmap v2757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:26.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:25 gibba011 bash[30791]: cluster 2021-03-10T15:24:24.417419+0000 mgr.gibba032.whopeo (mgr.24625) 4854 : cluster [DBG] pgmap v2757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:25 gibba011 bash[31774]: cluster 2021-03-10T15:24:24.417419+0000 mgr.gibba032.whopeo (mgr.24625) 4854 : cluster [DBG] pgmap v2757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:27 gibba032 bash[24004]: cluster 2021-03-10T15:24:26.418790+0000 mgr.gibba032.whopeo (mgr.24625) 4855 : cluster [DBG] pgmap v2758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:28.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:27 gibba011 bash[30791]: cluster 2021-03-10T15:24:26.418790+0000 mgr.gibba032.whopeo (mgr.24625) 4855 : cluster [DBG] pgmap v2758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:27 gibba011 bash[31774]: cluster 2021-03-10T15:24:26.418790+0000 mgr.gibba032.whopeo (mgr.24625) 4855 : cluster [DBG] pgmap v2758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:29 gibba032 bash[24004]: cluster 2021-03-10T15:24:28.420488+0000 mgr.gibba032.whopeo (mgr.24625) 4856 : cluster [DBG] pgmap v2759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:29 gibba032 bash[24004]: cephadm 2021-03-10T15:24:28.836182+0000 mgr.gibba032.whopeo (mgr.24625) 4857 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:29 gibba011 bash[30791]: cluster 2021-03-10T15:24:28.420488+0000 mgr.gibba032.whopeo (mgr.24625) 4856 : cluster [DBG] pgmap v2759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:29 gibba011 bash[30791]: cephadm 2021-03-10T15:24:28.836182+0000 mgr.gibba032.whopeo (mgr.24625) 4857 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:29 gibba011 bash[31774]: cluster 2021-03-10T15:24:28.420488+0000 mgr.gibba032.whopeo (mgr.24625) 4856 : cluster [DBG] pgmap v2759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:30.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:29 gibba011 bash[31774]: cephadm 2021-03-10T15:24:28.836182+0000 mgr.gibba032.whopeo (mgr.24625) 4857 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:31 gibba032 bash[24004]: cluster 2021-03-10T15:24:30.421917+0000 mgr.gibba032.whopeo (mgr.24625) 4858 : cluster [DBG] pgmap v2760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:31 gibba011 bash[30791]: cluster 2021-03-10T15:24:30.421917+0000 mgr.gibba032.whopeo (mgr.24625) 4858 : cluster [DBG] pgmap v2760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:31 gibba011 bash[31774]: cluster 2021-03-10T15:24:30.421917+0000 mgr.gibba032.whopeo (mgr.24625) 4858 : cluster [DBG] pgmap v2760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:33.104 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:24:33.335 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:24:33.336 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (91m) 5m ago 96m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:24:33.336 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (96m) 4m ago 96m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:24:33.336 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (91m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:24:33.337 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (91m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:24:33.337 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (90m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:24:33.337 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (90m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:24:33.337 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (90m) 4m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:24:33.338 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (90m) 5m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:24:33.338 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (96m) 5m ago 96m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:24:33.338 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (96m) 4m ago 96m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:24:33.338 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (89m) 5m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:24:33.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (89m) 5m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:24:33.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (88m) 5m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:24:33.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (88m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:24:33.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (87m) 4m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:24:33.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (87m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:24:33.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (86m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:24:33.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (86m) 4m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:24:33.340 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (96m) 4m ago 96m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:24:33.341 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 94m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:24:33.584 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:24:33.584 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:24:33.585 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:24:33.585 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:33.585 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:24:33.586 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:24:33.586 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:33.587 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:24:33.587 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:24:33.588 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:24:33.588 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:24:33.588 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:24:33.588 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:24:33.588 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:24:33.589 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:24:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:33 gibba032 bash[24004]: cluster 2021-03-10T15:24:32.423644+0000 mgr.gibba032.whopeo (mgr.24625) 4859 : cluster [DBG] pgmap v2761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:33 gibba032 bash[24004]: audit 2021-03-10T15:24:33.091296+0000 mgr.gibba032.whopeo (mgr.24625) 4860 : audit [DBG] from='client.57298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:33 gibba032 bash[24004]: audit 2021-03-10T15:24:33.318806+0000 mgr.gibba032.whopeo (mgr.24625) 4861 : audit [DBG] from='client.57304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:33 gibba032 bash[24004]: audit 2021-03-10T15:24:33.580963+0000 mon.a (mon.0) 1937 : audit [DBG] from='client.? 172.21.2.111:0/2011724350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[30791]: cluster 2021-03-10T15:24:32.423644+0000 mgr.gibba032.whopeo (mgr.24625) 4859 : cluster [DBG] pgmap v2761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[30791]: audit 2021-03-10T15:24:33.091296+0000 mgr.gibba032.whopeo (mgr.24625) 4860 : audit [DBG] from='client.57298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[30791]: audit 2021-03-10T15:24:33.318806+0000 mgr.gibba032.whopeo (mgr.24625) 4861 : audit [DBG] from='client.57304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[30791]: audit 2021-03-10T15:24:33.580963+0000 mon.a (mon.0) 1937 : audit [DBG] from='client.? 172.21.2.111:0/2011724350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[31774]: cluster 2021-03-10T15:24:32.423644+0000 mgr.gibba032.whopeo (mgr.24625) 4859 : cluster [DBG] pgmap v2761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[31774]: audit 2021-03-10T15:24:33.091296+0000 mgr.gibba032.whopeo (mgr.24625) 4860 : audit [DBG] from='client.57298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[31774]: audit 2021-03-10T15:24:33.318806+0000 mgr.gibba032.whopeo (mgr.24625) 4861 : audit [DBG] from='client.57304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:24:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:33 gibba011 bash[31774]: audit 2021-03-10T15:24:33.580963+0000 mon.a (mon.0) 1937 : audit [DBG] from='client.? 172.21.2.111:0/2011724350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:24:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:35 gibba032 bash[24004]: cluster 2021-03-10T15:24:34.425194+0000 mgr.gibba032.whopeo (mgr.24625) 4862 : cluster [DBG] pgmap v2762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:35 gibba011 bash[30791]: cluster 2021-03-10T15:24:34.425194+0000 mgr.gibba032.whopeo (mgr.24625) 4862 : cluster [DBG] pgmap v2762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:35 gibba011 bash[31774]: cluster 2021-03-10T15:24:34.425194+0000 mgr.gibba032.whopeo (mgr.24625) 4862 : cluster [DBG] pgmap v2762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:37 gibba032 bash[24004]: cluster 2021-03-10T15:24:36.426678+0000 mgr.gibba032.whopeo (mgr.24625) 4863 : cluster [DBG] pgmap v2763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:37 gibba011 bash[30791]: cluster 2021-03-10T15:24:36.426678+0000 mgr.gibba032.whopeo (mgr.24625) 4863 : cluster [DBG] pgmap v2763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:37 gibba011 bash[31774]: cluster 2021-03-10T15:24:36.426678+0000 mgr.gibba032.whopeo (mgr.24625) 4863 : cluster [DBG] pgmap v2763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:39 gibba032 bash[24004]: cluster 2021-03-10T15:24:38.428481+0000 mgr.gibba032.whopeo (mgr.24625) 4864 : cluster [DBG] pgmap v2764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:39 gibba011 bash[30791]: cluster 2021-03-10T15:24:38.428481+0000 mgr.gibba032.whopeo (mgr.24625) 4864 : cluster [DBG] pgmap v2764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:39 gibba011 bash[31774]: cluster 2021-03-10T15:24:38.428481+0000 mgr.gibba032.whopeo (mgr.24625) 4864 : cluster [DBG] pgmap v2764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:41 gibba032 bash[24004]: cluster 2021-03-10T15:24:40.430118+0000 mgr.gibba032.whopeo (mgr.24625) 4865 : cluster [DBG] pgmap v2765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:41 gibba011 bash[30791]: cluster 2021-03-10T15:24:40.430118+0000 mgr.gibba032.whopeo (mgr.24625) 4865 : cluster [DBG] pgmap v2765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:41 gibba011 bash[31774]: cluster 2021-03-10T15:24:40.430118+0000 mgr.gibba032.whopeo (mgr.24625) 4865 : cluster [DBG] pgmap v2765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:44 gibba032 bash[24004]: cluster 2021-03-10T15:24:42.431944+0000 mgr.gibba032.whopeo (mgr.24625) 4866 : cluster [DBG] pgmap v2766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:44 gibba011 bash[30791]: cluster 2021-03-10T15:24:42.431944+0000 mgr.gibba032.whopeo (mgr.24625) 4866 : cluster [DBG] pgmap v2766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:44 gibba011 bash[31774]: cluster 2021-03-10T15:24:42.431944+0000 mgr.gibba032.whopeo (mgr.24625) 4866 : cluster [DBG] pgmap v2766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:45 gibba011 bash[30791]: cephadm 2021-03-10T15:24:43.851958+0000 mgr.gibba032.whopeo (mgr.24625) 4867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:45 gibba011 bash[31774]: cephadm 2021-03-10T15:24:43.851958+0000 mgr.gibba032.whopeo (mgr.24625) 4867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:45 gibba032 bash[24004]: cephadm 2021-03-10T15:24:43.851958+0000 mgr.gibba032.whopeo (mgr.24625) 4867 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:24:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:46 gibba011 bash[30791]: cluster 2021-03-10T15:24:44.433601+0000 mgr.gibba032.whopeo (mgr.24625) 4868 : cluster [DBG] pgmap v2767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:46 gibba011 bash[31774]: cluster 2021-03-10T15:24:44.433601+0000 mgr.gibba032.whopeo (mgr.24625) 4868 : cluster [DBG] pgmap v2767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:46 gibba032 bash[24004]: cluster 2021-03-10T15:24:44.433601+0000 mgr.gibba032.whopeo (mgr.24625) 4868 : cluster [DBG] pgmap v2767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:48 gibba011 bash[30791]: cluster 2021-03-10T15:24:46.435198+0000 mgr.gibba032.whopeo (mgr.24625) 4869 : cluster [DBG] pgmap v2768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:48 gibba011 bash[31774]: cluster 2021-03-10T15:24:46.435198+0000 mgr.gibba032.whopeo (mgr.24625) 4869 : cluster [DBG] pgmap v2768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:48 gibba032 bash[24004]: cluster 2021-03-10T15:24:46.435198+0000 mgr.gibba032.whopeo (mgr.24625) 4869 : cluster [DBG] pgmap v2768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:50 gibba011 bash[30791]: cluster 2021-03-10T15:24:48.436940+0000 mgr.gibba032.whopeo (mgr.24625) 4870 : cluster [DBG] pgmap v2769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:50 gibba011 bash[31774]: cluster 2021-03-10T15:24:48.436940+0000 mgr.gibba032.whopeo (mgr.24625) 4870 : cluster [DBG] pgmap v2769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:50 gibba032 bash[24004]: cluster 2021-03-10T15:24:48.436940+0000 mgr.gibba032.whopeo (mgr.24625) 4870 : cluster [DBG] pgmap v2769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:52 gibba011 bash[30791]: cluster 2021-03-10T15:24:50.438504+0000 mgr.gibba032.whopeo (mgr.24625) 4871 : cluster [DBG] pgmap v2770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:52 gibba011 bash[31774]: cluster 2021-03-10T15:24:50.438504+0000 mgr.gibba032.whopeo (mgr.24625) 4871 : cluster [DBG] pgmap v2770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:52 gibba032 bash[24004]: cluster 2021-03-10T15:24:50.438504+0000 mgr.gibba032.whopeo (mgr.24625) 4871 : cluster [DBG] pgmap v2770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:54 gibba011 bash[30791]: cluster 2021-03-10T15:24:52.440286+0000 mgr.gibba032.whopeo (mgr.24625) 4872 : cluster [DBG] pgmap v2771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:54 gibba011 bash[31774]: cluster 2021-03-10T15:24:52.440286+0000 mgr.gibba032.whopeo (mgr.24625) 4872 : cluster [DBG] pgmap v2771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:54 gibba032 bash[24004]: cluster 2021-03-10T15:24:52.440286+0000 mgr.gibba032.whopeo (mgr.24625) 4872 : cluster [DBG] pgmap v2771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:56 gibba011 bash[30791]: cluster 2021-03-10T15:24:54.441544+0000 mgr.gibba032.whopeo (mgr.24625) 4873 : cluster [DBG] pgmap v2772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:56 gibba011 bash[31774]: cluster 2021-03-10T15:24:54.441544+0000 mgr.gibba032.whopeo (mgr.24625) 4873 : cluster [DBG] pgmap v2772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:56 gibba032 bash[24004]: cluster 2021-03-10T15:24:54.441544+0000 mgr.gibba032.whopeo (mgr.24625) 4873 : cluster [DBG] pgmap v2772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.576979+0000 mon.a (mon.0) 1938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:24:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.577595+0000 mon.a (mon.0) 1939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:24:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.593872+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:24:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.594581+0000 mon.a (mon.0) 1941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:24:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.595252+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:24:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[30791]: audit 2021-03-10T15:24:56.595975+0000 mon.a (mon.0) 1943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:24:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.576979+0000 mon.a (mon.0) 1938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:24:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.577595+0000 mon.a (mon.0) 1939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:24:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.593872+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:24:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.594581+0000 mon.a (mon.0) 1941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:24:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.595252+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:24:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:57 gibba011 bash[31774]: audit 2021-03-10T15:24:56.595975+0000 mon.a (mon.0) 1943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:24:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.576979+0000 mon.a (mon.0) 1938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:24:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.577595+0000 mon.a (mon.0) 1939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:24:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.593872+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:24:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.594581+0000 mon.a (mon.0) 1941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:24:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.595252+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:24:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:57 gibba032 bash[24004]: audit 2021-03-10T15:24:56.595975+0000 mon.a (mon.0) 1943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:24:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:24:58 gibba011 bash[30791]: cluster 2021-03-10T15:24:56.443153+0000 mgr.gibba032.whopeo (mgr.24625) 4874 : cluster [DBG] pgmap v2773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:24:58 gibba011 bash[31774]: cluster 2021-03-10T15:24:56.443153+0000 mgr.gibba032.whopeo (mgr.24625) 4874 : cluster [DBG] pgmap v2773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:24:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:24:58 gibba032 bash[24004]: cluster 2021-03-10T15:24:56.443153+0000 mgr.gibba032.whopeo (mgr.24625) 4874 : cluster [DBG] pgmap v2773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cluster 2021-03-10T15:24:58.445042+0000 mgr.gibba032.whopeo (mgr.24625) 4875 : cluster [DBG] pgmap v2774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:58.868811+0000 mgr.gibba032.whopeo (mgr.24625) 4876 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:25:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:58.870167+0000 mgr.gibba032.whopeo (mgr.24625) 4877 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:25:00.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.069036+0000 mon.a (mon.0) 1944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:25:00.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.070319+0000 mgr.gibba032.whopeo (mgr.24625) 4878 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:25:00.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.072113+0000 mon.a (mon.0) 1945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:25:00.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.072183+0000 mgr.gibba032.whopeo (mgr.24625) 4879 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:25:00.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.073657+0000 mgr.gibba032.whopeo (mgr.24625) 4880 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:25:00.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.073664+0000 mon.a (mon.0) 1946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.073758+0000 mgr.gibba032.whopeo (mgr.24625) 4881 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:25:00.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.075503+0000 mon.a (mon.0) 1947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.076606+0000 mgr.gibba032.whopeo (mgr.24625) 4882 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:25:00.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.076643+0000 mgr.gibba032.whopeo (mgr.24625) 4883 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:25:00.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.077533+0000 mgr.gibba032.whopeo (mgr.24625) 4884 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:25:00.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.077958+0000 mon.a (mon.0) 1948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.078269+0000 mgr.gibba032.whopeo (mgr.24625) 4885 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:25:00.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.078308+0000 mgr.gibba032.whopeo (mgr.24625) 4886 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:25:00.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.079254+0000 mgr.gibba032.whopeo (mgr.24625) 4887 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:25:00.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.079573+0000 mgr.gibba032.whopeo (mgr.24625) 4888 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:25:00.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.079761+0000 mon.a (mon.0) 1949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.342 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.080569+0000 mgr.gibba032.whopeo (mgr.24625) 4889 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:25:00.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.080607+0000 mgr.gibba032.whopeo (mgr.24625) 4890 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:25:00.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: cephadm 2021-03-10T15:24:59.080711+0000 mgr.gibba032.whopeo (mgr.24625) 4891 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:00.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[30791]: audit 2021-03-10T15:24:59.080843+0000 mon.a (mon.0) 1950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cluster 2021-03-10T15:24:58.445042+0000 mgr.gibba032.whopeo (mgr.24625) 4875 : cluster [DBG] pgmap v2774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:00.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:58.868811+0000 mgr.gibba032.whopeo (mgr.24625) 4876 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:25:00.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:58.870167+0000 mgr.gibba032.whopeo (mgr.24625) 4877 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:25:00.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.069036+0000 mon.a (mon.0) 1944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:25:00.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.070319+0000 mgr.gibba032.whopeo (mgr.24625) 4878 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:25:00.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.072113+0000 mon.a (mon.0) 1945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:25:00.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.072183+0000 mgr.gibba032.whopeo (mgr.24625) 4879 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:25:00.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.073657+0000 mgr.gibba032.whopeo (mgr.24625) 4880 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:25:00.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.073664+0000 mon.a (mon.0) 1946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.073758+0000 mgr.gibba032.whopeo (mgr.24625) 4881 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:25:00.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.075503+0000 mon.a (mon.0) 1947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.076606+0000 mgr.gibba032.whopeo (mgr.24625) 4882 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:25:00.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.076643+0000 mgr.gibba032.whopeo (mgr.24625) 4883 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:25:00.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.077533+0000 mgr.gibba032.whopeo (mgr.24625) 4884 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:25:00.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.077958+0000 mon.a (mon.0) 1948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.351 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.078269+0000 mgr.gibba032.whopeo (mgr.24625) 4885 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:25:00.351 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.078308+0000 mgr.gibba032.whopeo (mgr.24625) 4886 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:25:00.351 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.079254+0000 mgr.gibba032.whopeo (mgr.24625) 4887 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:25:00.352 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.079573+0000 mgr.gibba032.whopeo (mgr.24625) 4888 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:25:00.352 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.079761+0000 mon.a (mon.0) 1949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.353 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.080569+0000 mgr.gibba032.whopeo (mgr.24625) 4889 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:25:00.353 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.080607+0000 mgr.gibba032.whopeo (mgr.24625) 4890 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:25:00.354 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: cephadm 2021-03-10T15:24:59.080711+0000 mgr.gibba032.whopeo (mgr.24625) 4891 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:00.354 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:00 gibba011 bash[31774]: audit 2021-03-10T15:24:59.080843+0000 mon.a (mon.0) 1950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cluster 2021-03-10T15:24:58.445042+0000 mgr.gibba032.whopeo (mgr.24625) 4875 : cluster [DBG] pgmap v2774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:58.868811+0000 mgr.gibba032.whopeo (mgr.24625) 4876 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:25:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:58.870167+0000 mgr.gibba032.whopeo (mgr.24625) 4877 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:25:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.069036+0000 mon.a (mon.0) 1944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:25:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.070319+0000 mgr.gibba032.whopeo (mgr.24625) 4878 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:25:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.072113+0000 mon.a (mon.0) 1945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:25:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.072183+0000 mgr.gibba032.whopeo (mgr.24625) 4879 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:25:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.073657+0000 mgr.gibba032.whopeo (mgr.24625) 4880 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:25:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.073664+0000 mon.a (mon.0) 1946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.073758+0000 mgr.gibba032.whopeo (mgr.24625) 4881 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:25:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.075503+0000 mon.a (mon.0) 1947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.076606+0000 mgr.gibba032.whopeo (mgr.24625) 4882 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:25:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.076643+0000 mgr.gibba032.whopeo (mgr.24625) 4883 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:25:00.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.077533+0000 mgr.gibba032.whopeo (mgr.24625) 4884 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:25:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.077958+0000 mon.a (mon.0) 1948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.078269+0000 mgr.gibba032.whopeo (mgr.24625) 4885 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:25:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.078308+0000 mgr.gibba032.whopeo (mgr.24625) 4886 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:25:00.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.079254+0000 mgr.gibba032.whopeo (mgr.24625) 4887 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:25:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.079573+0000 mgr.gibba032.whopeo (mgr.24625) 4888 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:25:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.079761+0000 mon.a (mon.0) 1949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.080569+0000 mgr.gibba032.whopeo (mgr.24625) 4889 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:25:00.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.080607+0000 mgr.gibba032.whopeo (mgr.24625) 4890 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:25:00.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: cephadm 2021-03-10T15:24:59.080711+0000 mgr.gibba032.whopeo (mgr.24625) 4891 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:00.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:00 gibba032 bash[24004]: audit 2021-03-10T15:24:59.080843+0000 mon.a (mon.0) 1950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:02 gibba032 bash[24004]: cluster 2021-03-10T15:25:00.446765+0000 mgr.gibba032.whopeo (mgr.24625) 4892 : cluster [DBG] pgmap v2775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:02 gibba011 bash[30791]: cluster 2021-03-10T15:25:00.446765+0000 mgr.gibba032.whopeo (mgr.24625) 4892 : cluster [DBG] pgmap v2775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:02 gibba011 bash[31774]: cluster 2021-03-10T15:25:00.446765+0000 mgr.gibba032.whopeo (mgr.24625) 4892 : cluster [DBG] pgmap v2775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:03.828 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:25:04.056 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:25:04.056 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (91m) 6m ago 97m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:25:04.056 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (96m) 5m ago 96m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:25:04.056 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (92m) 5m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:25:04.057 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (92m) 5m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:25:04.057 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (91m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:25:04.057 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (91m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:25:04.057 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (90m) 5m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:25:04.057 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (90m) 6m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:25:04.058 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (97m) 6m ago 97m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:25:04.058 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (97m) 5m ago 97m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:25:04.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (90m) 6m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:25:04.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (89m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:25:04.059 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (89m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:25:04.059 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (88m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:25:04.059 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (88m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:25:04.059 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (87m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:25:04.060 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (87m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:25:04.060 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (87m) 5m ago 97m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:25:04.060 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (96m) 5m ago 97m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:25:04.060 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 95m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:25:04.310 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:25:04.310 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:25:04.311 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:25:04.311 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:04.311 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:25:04.311 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:25:04.312 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:04.312 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:25:04.312 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:25:04.312 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:04.313 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:25:04.313 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:25:04.313 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:25:04.313 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:25:04.313 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:25:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:04 gibba032 bash[24004]: cluster 2021-03-10T15:25:02.448581+0000 mgr.gibba032.whopeo (mgr.24625) 4893 : cluster [DBG] pgmap v2776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:04 gibba011 bash[30791]: cluster 2021-03-10T15:25:02.448581+0000 mgr.gibba032.whopeo (mgr.24625) 4893 : cluster [DBG] pgmap v2776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:04 gibba011 bash[31774]: cluster 2021-03-10T15:25:02.448581+0000 mgr.gibba032.whopeo (mgr.24625) 4893 : cluster [DBG] pgmap v2776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:05 gibba032 bash[24004]: audit 2021-03-10T15:25:03.815909+0000 mgr.gibba032.whopeo (mgr.24625) 4894 : audit [DBG] from='client.47331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:05 gibba032 bash[24004]: audit 2021-03-10T15:25:04.039310+0000 mgr.gibba032.whopeo (mgr.24625) 4895 : audit [DBG] from='client.57322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:05 gibba032 bash[24004]: audit 2021-03-10T15:25:04.307287+0000 mon.c (mon.1) 91 : audit [DBG] from='client.? 172.21.2.111:0/2446675370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[30791]: audit 2021-03-10T15:25:03.815909+0000 mgr.gibba032.whopeo (mgr.24625) 4894 : audit [DBG] from='client.47331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[30791]: audit 2021-03-10T15:25:04.039310+0000 mgr.gibba032.whopeo (mgr.24625) 4895 : audit [DBG] from='client.57322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[30791]: audit 2021-03-10T15:25:04.307287+0000 mon.c (mon.1) 91 : audit [DBG] from='client.? 172.21.2.111:0/2446675370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[31774]: audit 2021-03-10T15:25:03.815909+0000 mgr.gibba032.whopeo (mgr.24625) 4894 : audit [DBG] from='client.47331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[31774]: audit 2021-03-10T15:25:04.039310+0000 mgr.gibba032.whopeo (mgr.24625) 4895 : audit [DBG] from='client.57322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:05.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:05 gibba011 bash[31774]: audit 2021-03-10T15:25:04.307287+0000 mon.c (mon.1) 91 : audit [DBG] from='client.? 172.21.2.111:0/2446675370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:06 gibba032 bash[24004]: cluster 2021-03-10T15:25:04.450140+0000 mgr.gibba032.whopeo (mgr.24625) 4896 : cluster [DBG] pgmap v2777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:06 gibba011 bash[30791]: cluster 2021-03-10T15:25:04.450140+0000 mgr.gibba032.whopeo (mgr.24625) 4896 : cluster [DBG] pgmap v2777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:06 gibba011 bash[31774]: cluster 2021-03-10T15:25:04.450140+0000 mgr.gibba032.whopeo (mgr.24625) 4896 : cluster [DBG] pgmap v2777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:08 gibba032 bash[24004]: cluster 2021-03-10T15:25:06.451694+0000 mgr.gibba032.whopeo (mgr.24625) 4897 : cluster [DBG] pgmap v2778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:08 gibba011 bash[30791]: cluster 2021-03-10T15:25:06.451694+0000 mgr.gibba032.whopeo (mgr.24625) 4897 : cluster [DBG] pgmap v2778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:08 gibba011 bash[31774]: cluster 2021-03-10T15:25:06.451694+0000 mgr.gibba032.whopeo (mgr.24625) 4897 : cluster [DBG] pgmap v2778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:10 gibba032 bash[24004]: cluster 2021-03-10T15:25:08.453478+0000 mgr.gibba032.whopeo (mgr.24625) 4898 : cluster [DBG] pgmap v2779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:10 gibba011 bash[30791]: cluster 2021-03-10T15:25:08.453478+0000 mgr.gibba032.whopeo (mgr.24625) 4898 : cluster [DBG] pgmap v2779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:10 gibba011 bash[31774]: cluster 2021-03-10T15:25:08.453478+0000 mgr.gibba032.whopeo (mgr.24625) 4898 : cluster [DBG] pgmap v2779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:12 gibba032 bash[24004]: cluster 2021-03-10T15:25:10.455057+0000 mgr.gibba032.whopeo (mgr.24625) 4899 : cluster [DBG] pgmap v2780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:12 gibba011 bash[30791]: cluster 2021-03-10T15:25:10.455057+0000 mgr.gibba032.whopeo (mgr.24625) 4899 : cluster [DBG] pgmap v2780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:12 gibba011 bash[31774]: cluster 2021-03-10T15:25:10.455057+0000 mgr.gibba032.whopeo (mgr.24625) 4899 : cluster [DBG] pgmap v2780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:14 gibba032 bash[24004]: cluster 2021-03-10T15:25:12.456588+0000 mgr.gibba032.whopeo (mgr.24625) 4900 : cluster [DBG] pgmap v2781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:14 gibba011 bash[30791]: cluster 2021-03-10T15:25:12.456588+0000 mgr.gibba032.whopeo (mgr.24625) 4900 : cluster [DBG] pgmap v2781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:14 gibba011 bash[31774]: cluster 2021-03-10T15:25:12.456588+0000 mgr.gibba032.whopeo (mgr.24625) 4900 : cluster [DBG] pgmap v2781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:15 gibba032 bash[24004]: cephadm 2021-03-10T15:25:14.087269+0000 mgr.gibba032.whopeo (mgr.24625) 4901 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:15.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:15 gibba011 bash[30791]: cephadm 2021-03-10T15:25:14.087269+0000 mgr.gibba032.whopeo (mgr.24625) 4901 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:15.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:15 gibba011 bash[31774]: cephadm 2021-03-10T15:25:14.087269+0000 mgr.gibba032.whopeo (mgr.24625) 4901 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:16 gibba032 bash[24004]: cluster 2021-03-10T15:25:14.458107+0000 mgr.gibba032.whopeo (mgr.24625) 4902 : cluster [DBG] pgmap v2782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:16 gibba011 bash[30791]: cluster 2021-03-10T15:25:14.458107+0000 mgr.gibba032.whopeo (mgr.24625) 4902 : cluster [DBG] pgmap v2782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:16 gibba011 bash[31774]: cluster 2021-03-10T15:25:14.458107+0000 mgr.gibba032.whopeo (mgr.24625) 4902 : cluster [DBG] pgmap v2782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:18 gibba032 bash[24004]: cluster 2021-03-10T15:25:16.459698+0000 mgr.gibba032.whopeo (mgr.24625) 4903 : cluster [DBG] pgmap v2783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:18 gibba011 bash[30791]: cluster 2021-03-10T15:25:16.459698+0000 mgr.gibba032.whopeo (mgr.24625) 4903 : cluster [DBG] pgmap v2783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:18 gibba011 bash[31774]: cluster 2021-03-10T15:25:16.459698+0000 mgr.gibba032.whopeo (mgr.24625) 4903 : cluster [DBG] pgmap v2783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:20 gibba032 bash[24004]: cluster 2021-03-10T15:25:18.461285+0000 mgr.gibba032.whopeo (mgr.24625) 4904 : cluster [DBG] pgmap v2784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:20 gibba011 bash[30791]: cluster 2021-03-10T15:25:18.461285+0000 mgr.gibba032.whopeo (mgr.24625) 4904 : cluster [DBG] pgmap v2784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:20 gibba011 bash[31774]: cluster 2021-03-10T15:25:18.461285+0000 mgr.gibba032.whopeo (mgr.24625) 4904 : cluster [DBG] pgmap v2784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:22 gibba032 bash[24004]: cluster 2021-03-10T15:25:20.462861+0000 mgr.gibba032.whopeo (mgr.24625) 4905 : cluster [DBG] pgmap v2785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:22 gibba011 bash[30791]: cluster 2021-03-10T15:25:20.462861+0000 mgr.gibba032.whopeo (mgr.24625) 4905 : cluster [DBG] pgmap v2785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:22 gibba011 bash[31774]: cluster 2021-03-10T15:25:20.462861+0000 mgr.gibba032.whopeo (mgr.24625) 4905 : cluster [DBG] pgmap v2785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:24 gibba032 bash[24004]: cluster 2021-03-10T15:25:22.464381+0000 mgr.gibba032.whopeo (mgr.24625) 4906 : cluster [DBG] pgmap v2786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:24 gibba011 bash[30791]: cluster 2021-03-10T15:25:22.464381+0000 mgr.gibba032.whopeo (mgr.24625) 4906 : cluster [DBG] pgmap v2786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:24 gibba011 bash[31774]: cluster 2021-03-10T15:25:22.464381+0000 mgr.gibba032.whopeo (mgr.24625) 4906 : cluster [DBG] pgmap v2786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:26 gibba032 bash[24004]: cluster 2021-03-10T15:25:24.465903+0000 mgr.gibba032.whopeo (mgr.24625) 4907 : cluster [DBG] pgmap v2787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:26 gibba011 bash[30791]: cluster 2021-03-10T15:25:24.465903+0000 mgr.gibba032.whopeo (mgr.24625) 4907 : cluster [DBG] pgmap v2787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:26 gibba011 bash[31774]: cluster 2021-03-10T15:25:24.465903+0000 mgr.gibba032.whopeo (mgr.24625) 4907 : cluster [DBG] pgmap v2787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:28 gibba032 bash[24004]: cluster 2021-03-10T15:25:26.467244+0000 mgr.gibba032.whopeo (mgr.24625) 4908 : cluster [DBG] pgmap v2788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:28 gibba011 bash[30791]: cluster 2021-03-10T15:25:26.467244+0000 mgr.gibba032.whopeo (mgr.24625) 4908 : cluster [DBG] pgmap v2788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:28 gibba011 bash[31774]: cluster 2021-03-10T15:25:26.467244+0000 mgr.gibba032.whopeo (mgr.24625) 4908 : cluster [DBG] pgmap v2788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:30 gibba032 bash[24004]: cluster 2021-03-10T15:25:28.468764+0000 mgr.gibba032.whopeo (mgr.24625) 4909 : cluster [DBG] pgmap v2789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:30 gibba032 bash[24004]: cephadm 2021-03-10T15:25:29.098939+0000 mgr.gibba032.whopeo (mgr.24625) 4910 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:30 gibba011 bash[30791]: cluster 2021-03-10T15:25:28.468764+0000 mgr.gibba032.whopeo (mgr.24625) 4909 : cluster [DBG] pgmap v2789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:30 gibba011 bash[30791]: cephadm 2021-03-10T15:25:29.098939+0000 mgr.gibba032.whopeo (mgr.24625) 4910 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:30 gibba011 bash[31774]: cluster 2021-03-10T15:25:28.468764+0000 mgr.gibba032.whopeo (mgr.24625) 4909 : cluster [DBG] pgmap v2789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:30 gibba011 bash[31774]: cephadm 2021-03-10T15:25:29.098939+0000 mgr.gibba032.whopeo (mgr.24625) 4910 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:32 gibba032 bash[24004]: cluster 2021-03-10T15:25:30.470212+0000 mgr.gibba032.whopeo (mgr.24625) 4911 : cluster [DBG] pgmap v2790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:32 gibba011 bash[30791]: cluster 2021-03-10T15:25:30.470212+0000 mgr.gibba032.whopeo (mgr.24625) 4911 : cluster [DBG] pgmap v2790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:32 gibba011 bash[31774]: cluster 2021-03-10T15:25:30.470212+0000 mgr.gibba032.whopeo (mgr.24625) 4911 : cluster [DBG] pgmap v2790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:34 gibba032 bash[24004]: cluster 2021-03-10T15:25:32.471961+0000 mgr.gibba032.whopeo (mgr.24625) 4912 : cluster [DBG] pgmap v2791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:34.553 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:25:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:34 gibba011 bash[30791]: cluster 2021-03-10T15:25:32.471961+0000 mgr.gibba032.whopeo (mgr.24625) 4912 : cluster [DBG] pgmap v2791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:34 gibba011 bash[31774]: cluster 2021-03-10T15:25:32.471961+0000 mgr.gibba032.whopeo (mgr.24625) 4912 : cluster [DBG] pgmap v2791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:34.810 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:25:34.810 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (92m) 6m ago 97m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:25:34.811 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (97m) 5m ago 97m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:25:34.811 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (92m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:25:34.811 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (92m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:25:34.811 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (91m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:25:34.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (91m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:25:34.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (91m) 5m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:25:34.812 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (91m) 6m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:25:34.812 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (97m) 6m ago 97m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:25:34.813 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (97m) 5m ago 97m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:25:34.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (90m) 6m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:25:34.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (90m) 6m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:25:34.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (89m) 6m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:25:34.813 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (89m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:25:34.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (88m) 5m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:25:34.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (88m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:25:34.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (87m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:25:34.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (87m) 5m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:25:34.815 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (97m) 5m ago 97m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:25:34.815 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 95m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:25:35.061 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:25:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:25:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:25:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:25:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:25:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:25:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:25:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:25:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:25:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:25:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:25:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:25:35.064 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:25:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:35 gibba032 bash[24004]: audit 2021-03-10T15:25:35.058066+0000 mon.a (mon.0) 1951 : audit [DBG] from='client.? 172.21.2.111:0/3179860925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:35.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:35 gibba011 bash[30791]: audit 2021-03-10T15:25:35.058066+0000 mon.a (mon.0) 1951 : audit [DBG] from='client.? 172.21.2.111:0/3179860925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:35 gibba011 bash[31774]: audit 2021-03-10T15:25:35.058066+0000 mon.a (mon.0) 1951 : audit [DBG] from='client.? 172.21.2.111:0/3179860925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:25:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:36 gibba032 bash[24004]: cluster 2021-03-10T15:25:34.473260+0000 mgr.gibba032.whopeo (mgr.24625) 4913 : cluster [DBG] pgmap v2792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:36 gibba032 bash[24004]: audit 2021-03-10T15:25:34.540395+0000 mgr.gibba032.whopeo (mgr.24625) 4914 : audit [DBG] from='client.57334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:36 gibba032 bash[24004]: audit 2021-03-10T15:25:34.766751+0000 mgr.gibba032.whopeo (mgr.24625) 4915 : audit [DBG] from='client.47355 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[30791]: cluster 2021-03-10T15:25:34.473260+0000 mgr.gibba032.whopeo (mgr.24625) 4913 : cluster [DBG] pgmap v2792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[30791]: audit 2021-03-10T15:25:34.540395+0000 mgr.gibba032.whopeo (mgr.24625) 4914 : audit [DBG] from='client.57334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[30791]: audit 2021-03-10T15:25:34.766751+0000 mgr.gibba032.whopeo (mgr.24625) 4915 : audit [DBG] from='client.47355 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[31774]: cluster 2021-03-10T15:25:34.473260+0000 mgr.gibba032.whopeo (mgr.24625) 4913 : cluster [DBG] pgmap v2792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[31774]: audit 2021-03-10T15:25:34.540395+0000 mgr.gibba032.whopeo (mgr.24625) 4914 : audit [DBG] from='client.57334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:36 gibba011 bash[31774]: audit 2021-03-10T15:25:34.766751+0000 mgr.gibba032.whopeo (mgr.24625) 4915 : audit [DBG] from='client.47355 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:25:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:38 gibba011 bash[30791]: cluster 2021-03-10T15:25:36.474855+0000 mgr.gibba032.whopeo (mgr.24625) 4916 : cluster [DBG] pgmap v2793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:38 gibba011 bash[31774]: cluster 2021-03-10T15:25:36.474855+0000 mgr.gibba032.whopeo (mgr.24625) 4916 : cluster [DBG] pgmap v2793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:38 gibba032 bash[24004]: cluster 2021-03-10T15:25:36.474855+0000 mgr.gibba032.whopeo (mgr.24625) 4916 : cluster [DBG] pgmap v2793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:40 gibba011 bash[30791]: cluster 2021-03-10T15:25:38.476709+0000 mgr.gibba032.whopeo (mgr.24625) 4917 : cluster [DBG] pgmap v2794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:40 gibba011 bash[31774]: cluster 2021-03-10T15:25:38.476709+0000 mgr.gibba032.whopeo (mgr.24625) 4917 : cluster [DBG] pgmap v2794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:40 gibba032 bash[24004]: cluster 2021-03-10T15:25:38.476709+0000 mgr.gibba032.whopeo (mgr.24625) 4917 : cluster [DBG] pgmap v2794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:42 gibba011 bash[30791]: cluster 2021-03-10T15:25:40.478058+0000 mgr.gibba032.whopeo (mgr.24625) 4918 : cluster [DBG] pgmap v2795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:42 gibba011 bash[31774]: cluster 2021-03-10T15:25:40.478058+0000 mgr.gibba032.whopeo (mgr.24625) 4918 : cluster [DBG] pgmap v2795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:42 gibba032 bash[24004]: cluster 2021-03-10T15:25:40.478058+0000 mgr.gibba032.whopeo (mgr.24625) 4918 : cluster [DBG] pgmap v2795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:44 gibba032 bash[24004]: cluster 2021-03-10T15:25:42.479838+0000 mgr.gibba032.whopeo (mgr.24625) 4919 : cluster [DBG] pgmap v2796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:44 gibba011 bash[30791]: cluster 2021-03-10T15:25:42.479838+0000 mgr.gibba032.whopeo (mgr.24625) 4919 : cluster [DBG] pgmap v2796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:44 gibba011 bash[31774]: cluster 2021-03-10T15:25:42.479838+0000 mgr.gibba032.whopeo (mgr.24625) 4919 : cluster [DBG] pgmap v2796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:45.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:45 gibba011 bash[30791]: cephadm 2021-03-10T15:25:44.114875+0000 mgr.gibba032.whopeo (mgr.24625) 4920 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:45 gibba011 bash[31774]: cephadm 2021-03-10T15:25:44.114875+0000 mgr.gibba032.whopeo (mgr.24625) 4920 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:45.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:45 gibba032 bash[24004]: cephadm 2021-03-10T15:25:44.114875+0000 mgr.gibba032.whopeo (mgr.24625) 4920 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:25:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:46 gibba011 bash[30791]: cluster 2021-03-10T15:25:44.481287+0000 mgr.gibba032.whopeo (mgr.24625) 4921 : cluster [DBG] pgmap v2797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:46 gibba011 bash[31774]: cluster 2021-03-10T15:25:44.481287+0000 mgr.gibba032.whopeo (mgr.24625) 4921 : cluster [DBG] pgmap v2797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:46 gibba032 bash[24004]: cluster 2021-03-10T15:25:44.481287+0000 mgr.gibba032.whopeo (mgr.24625) 4921 : cluster [DBG] pgmap v2797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:48 gibba011 bash[30791]: cluster 2021-03-10T15:25:46.481784+0000 mgr.gibba032.whopeo (mgr.24625) 4922 : cluster [DBG] pgmap v2798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:48 gibba011 bash[31774]: cluster 2021-03-10T15:25:46.481784+0000 mgr.gibba032.whopeo (mgr.24625) 4922 : cluster [DBG] pgmap v2798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:48 gibba032 bash[24004]: cluster 2021-03-10T15:25:46.481784+0000 mgr.gibba032.whopeo (mgr.24625) 4922 : cluster [DBG] pgmap v2798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:50 gibba011 bash[30791]: cluster 2021-03-10T15:25:48.483419+0000 mgr.gibba032.whopeo (mgr.24625) 4923 : cluster [DBG] pgmap v2799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:50 gibba011 bash[31774]: cluster 2021-03-10T15:25:48.483419+0000 mgr.gibba032.whopeo (mgr.24625) 4923 : cluster [DBG] pgmap v2799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:50 gibba032 bash[24004]: cluster 2021-03-10T15:25:48.483419+0000 mgr.gibba032.whopeo (mgr.24625) 4923 : cluster [DBG] pgmap v2799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:52 gibba011 bash[30791]: cluster 2021-03-10T15:25:50.484827+0000 mgr.gibba032.whopeo (mgr.24625) 4924 : cluster [DBG] pgmap v2800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:52 gibba011 bash[31774]: cluster 2021-03-10T15:25:50.484827+0000 mgr.gibba032.whopeo (mgr.24625) 4924 : cluster [DBG] pgmap v2800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:52 gibba032 bash[24004]: cluster 2021-03-10T15:25:50.484827+0000 mgr.gibba032.whopeo (mgr.24625) 4924 : cluster [DBG] pgmap v2800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:54 gibba032 bash[24004]: cluster 2021-03-10T15:25:52.485337+0000 mgr.gibba032.whopeo (mgr.24625) 4925 : cluster [DBG] pgmap v2801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:54.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:54 gibba011 bash[30791]: cluster 2021-03-10T15:25:52.485337+0000 mgr.gibba032.whopeo (mgr.24625) 4925 : cluster [DBG] pgmap v2801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:54 gibba011 bash[31774]: cluster 2021-03-10T15:25:52.485337+0000 mgr.gibba032.whopeo (mgr.24625) 4925 : cluster [DBG] pgmap v2801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:56 gibba032 bash[24004]: cluster 2021-03-10T15:25:54.486671+0000 mgr.gibba032.whopeo (mgr.24625) 4926 : cluster [DBG] pgmap v2802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:56 gibba011 bash[30791]: cluster 2021-03-10T15:25:54.486671+0000 mgr.gibba032.whopeo (mgr.24625) 4926 : cluster [DBG] pgmap v2802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:56 gibba011 bash[31774]: cluster 2021-03-10T15:25:54.486671+0000 mgr.gibba032.whopeo (mgr.24625) 4926 : cluster [DBG] pgmap v2802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.594335+0000 mon.a (mon.0) 1952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:25:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.594977+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:25:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.607526+0000 mon.a (mon.0) 1954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:25:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.608288+0000 mon.a (mon.0) 1955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.609003+0000 mon.a (mon.0) 1956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:57 gibba032 bash[24004]: audit 2021-03-10T15:25:56.609688+0000 mon.a (mon.0) 1957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T15:25:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.594335+0000 mon.a (mon.0) 1952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:25:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.594977+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:25:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.607526+0000 mon.a (mon.0) 1954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:25:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.608288+0000 mon.a (mon.0) 1955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.609003+0000 mon.a (mon.0) 1956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[30791]: audit 2021-03-10T15:25:56.609688+0000 mon.a (mon.0) 1957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T15:25:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.594335+0000 mon.a (mon.0) 1952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:25:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.594977+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:25:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.607526+0000 mon.a (mon.0) 1954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:25:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.608288+0000 mon.a (mon.0) 1955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.609003+0000 mon.a (mon.0) 1956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:25:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:57 gibba011 bash[31774]: audit 2021-03-10T15:25:56.609688+0000 mon.a (mon.0) 1957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T15:25:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:25:58 gibba032 bash[24004]: cluster 2021-03-10T15:25:56.488226+0000 mgr.gibba032.whopeo (mgr.24625) 4927 : cluster [DBG] pgmap v2803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:25:58 gibba011 bash[30791]: cluster 2021-03-10T15:25:56.488226+0000 mgr.gibba032.whopeo (mgr.24625) 4927 : cluster [DBG] pgmap v2803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:25:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:25:58 gibba011 bash[31774]: cluster 2021-03-10T15:25:56.488226+0000 mgr.gibba032.whopeo (mgr.24625) 4927 : cluster [DBG] pgmap v2803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: cluster 2021-03-10T15:25:58.489748+0000 mgr.gibba032.whopeo (mgr.24625) 4928 : cluster [DBG] pgmap v2804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.330146+0000 mgr.gibba032.whopeo (mgr.24625) 4929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:26:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.330388+0000 mon.a (mon.0) 1958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.333103+0000 mgr.gibba032.whopeo (mgr.24625) 4930 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:26:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.333324+0000 mon.a (mon.0) 1959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.521695+0000 mon.a (mon.0) 1960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.524938+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:26:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.526601+0000 mon.a (mon.0) 1962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.528063+0000 mon.a (mon.0) 1963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.529557+0000 mon.a (mon.0) 1964 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.532487+0000 mon.a (mon.0) 1965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:00 gibba032 bash[24004]: audit 2021-03-10T15:25:59.534182+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: cluster 2021-03-10T15:25:58.489748+0000 mgr.gibba032.whopeo (mgr.24625) 4928 : cluster [DBG] pgmap v2804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:00.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.330146+0000 mgr.gibba032.whopeo (mgr.24625) 4929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:26:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.330388+0000 mon.a (mon.0) 1958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.333103+0000 mgr.gibba032.whopeo (mgr.24625) 4930 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:26:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.333324+0000 mon.a (mon.0) 1959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.521695+0000 mon.a (mon.0) 1960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.524938+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:26:00.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.526601+0000 mon.a (mon.0) 1962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.528063+0000 mon.a (mon.0) 1963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.529557+0000 mon.a (mon.0) 1964 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.532487+0000 mon.a (mon.0) 1965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[30791]: audit 2021-03-10T15:25:59.534182+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: cluster 2021-03-10T15:25:58.489748+0000 mgr.gibba032.whopeo (mgr.24625) 4928 : cluster [DBG] pgmap v2804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.330146+0000 mgr.gibba032.whopeo (mgr.24625) 4929 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:26:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.330388+0000 mon.a (mon.0) 1958 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.333103+0000 mgr.gibba032.whopeo (mgr.24625) 4930 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:26:00.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.333324+0000 mon.a (mon.0) 1959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.521695+0000 mon.a (mon.0) 1960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:26:00.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.524938+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:26:00.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.526601+0000 mon.a (mon.0) 1962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.528063+0000 mon.a (mon.0) 1963 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.529557+0000 mon.a (mon.0) 1964 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.532487+0000 mon.a (mon.0) 1965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:00.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:00 gibba011 bash[31774]: audit 2021-03-10T15:25:59.534182+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.523002+0000 mgr.gibba032.whopeo (mgr.24625) 4931 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:26:01.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.525036+0000 mgr.gibba032.whopeo (mgr.24625) 4932 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:26:01.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.526397+0000 mgr.gibba032.whopeo (mgr.24625) 4933 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:26:01.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.526435+0000 mgr.gibba032.whopeo (mgr.24625) 4934 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:26:01.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.527944+0000 mgr.gibba032.whopeo (mgr.24625) 4935 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:26:01.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.527999+0000 mgr.gibba032.whopeo (mgr.24625) 4936 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:26:01.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.529383+0000 mgr.gibba032.whopeo (mgr.24625) 4937 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:26:01.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.530666+0000 mgr.gibba032.whopeo (mgr.24625) 4938 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:26:01.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.530703+0000 mgr.gibba032.whopeo (mgr.24625) 4939 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:26:01.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.532358+0000 mgr.gibba032.whopeo (mgr.24625) 4940 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:26:01.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.532682+0000 mgr.gibba032.whopeo (mgr.24625) 4941 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:26:01.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.534083+0000 mgr.gibba032.whopeo (mgr.24625) 4942 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:26:01.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.534123+0000 mgr.gibba032.whopeo (mgr.24625) 4943 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:26:01.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:01 gibba032 bash[24004]: cephadm 2021-03-10T15:25:59.534229+0000 mgr.gibba032.whopeo (mgr.24625) 4944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:01.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.523002+0000 mgr.gibba032.whopeo (mgr.24625) 4931 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:26:01.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.525036+0000 mgr.gibba032.whopeo (mgr.24625) 4932 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:26:01.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.526397+0000 mgr.gibba032.whopeo (mgr.24625) 4933 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:26:01.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.526435+0000 mgr.gibba032.whopeo (mgr.24625) 4934 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:26:01.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.527944+0000 mgr.gibba032.whopeo (mgr.24625) 4935 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:26:01.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.527999+0000 mgr.gibba032.whopeo (mgr.24625) 4936 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:26:01.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.529383+0000 mgr.gibba032.whopeo (mgr.24625) 4937 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:26:01.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.530666+0000 mgr.gibba032.whopeo (mgr.24625) 4938 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:26:01.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.530703+0000 mgr.gibba032.whopeo (mgr.24625) 4939 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:26:01.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.532358+0000 mgr.gibba032.whopeo (mgr.24625) 4940 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:26:01.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.532682+0000 mgr.gibba032.whopeo (mgr.24625) 4941 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:26:01.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.534083+0000 mgr.gibba032.whopeo (mgr.24625) 4942 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:26:01.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.534123+0000 mgr.gibba032.whopeo (mgr.24625) 4943 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:26:01.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[30791]: cephadm 2021-03-10T15:25:59.534229+0000 mgr.gibba032.whopeo (mgr.24625) 4944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.523002+0000 mgr.gibba032.whopeo (mgr.24625) 4931 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:26:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.525036+0000 mgr.gibba032.whopeo (mgr.24625) 4932 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:26:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.526397+0000 mgr.gibba032.whopeo (mgr.24625) 4933 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:26:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.526435+0000 mgr.gibba032.whopeo (mgr.24625) 4934 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:26:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.527944+0000 mgr.gibba032.whopeo (mgr.24625) 4935 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:26:01.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.527999+0000 mgr.gibba032.whopeo (mgr.24625) 4936 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:26:01.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.529383+0000 mgr.gibba032.whopeo (mgr.24625) 4937 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:26:01.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.530666+0000 mgr.gibba032.whopeo (mgr.24625) 4938 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:26:01.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.530703+0000 mgr.gibba032.whopeo (mgr.24625) 4939 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:26:01.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.532358+0000 mgr.gibba032.whopeo (mgr.24625) 4940 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:26:01.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.532682+0000 mgr.gibba032.whopeo (mgr.24625) 4941 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:26:01.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.534083+0000 mgr.gibba032.whopeo (mgr.24625) 4942 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:26:01.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.534123+0000 mgr.gibba032.whopeo (mgr.24625) 4943 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:26:01.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:01 gibba011 bash[31774]: cephadm 2021-03-10T15:25:59.534229+0000 mgr.gibba032.whopeo (mgr.24625) 4944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:02 gibba032 bash[24004]: cluster 2021-03-10T15:26:00.491298+0000 mgr.gibba032.whopeo (mgr.24625) 4945 : cluster [DBG] pgmap v2805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:02 gibba011 bash[30791]: cluster 2021-03-10T15:26:00.491298+0000 mgr.gibba032.whopeo (mgr.24625) 4945 : cluster [DBG] pgmap v2805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:02 gibba011 bash[31774]: cluster 2021-03-10T15:26:00.491298+0000 mgr.gibba032.whopeo (mgr.24625) 4945 : cluster [DBG] pgmap v2805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:04 gibba032 bash[24004]: cluster 2021-03-10T15:26:02.493091+0000 mgr.gibba032.whopeo (mgr.24625) 4946 : cluster [DBG] pgmap v2806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:04 gibba011 bash[30791]: cluster 2021-03-10T15:26:02.493091+0000 mgr.gibba032.whopeo (mgr.24625) 4946 : cluster [DBG] pgmap v2806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:04.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:04 gibba011 bash[31774]: cluster 2021-03-10T15:26:02.493091+0000 mgr.gibba032.whopeo (mgr.24625) 4946 : cluster [DBG] pgmap v2806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:05.303 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:26:05.532 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:26:05.533 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (92m) 7m ago 98m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:26:05.533 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (97m) 6m ago 97m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:26:05.533 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (93m) 6m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:26:05.534 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (93m) 6m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:26:05.534 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (92m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:26:05.534 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (92m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:26:05.534 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (91m) 6m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:26:05.535 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (91m) 7m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:26:05.535 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (98m) 7m ago 98m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:26:05.535 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (98m) 6m ago 98m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:26:05.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (91m) 7m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:26:05.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:26:05.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:26:05.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (89m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:26:05.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (89m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:26:05.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (88m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:26:05.537 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (88m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:26:05.537 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (88m) 6m ago 98m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:26:05.537 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (97m) 6m ago 98m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:26:05.537 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 96m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:26:05.782 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:26:05.783 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:26:05.783 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:26:05.783 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:05.783 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:26:05.784 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:26:05.784 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:05.784 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:26:05.784 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:26:05.785 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:05.785 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:26:05.785 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:26:05.785 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:26:05.786 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:26:05.786 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:26:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:06 gibba032 bash[24004]: cluster 2021-03-10T15:26:04.493788+0000 mgr.gibba032.whopeo (mgr.24625) 4947 : cluster [DBG] pgmap v2807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:06 gibba032 bash[24004]: audit 2021-03-10T15:26:05.290504+0000 mgr.gibba032.whopeo (mgr.24625) 4948 : audit [DBG] from='client.47367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:06 gibba032 bash[24004]: audit 2021-03-10T15:26:05.779722+0000 mon.c (mon.1) 92 : audit [DBG] from='client.? 172.21.2.111:0/436478431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:06.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[30791]: cluster 2021-03-10T15:26:04.493788+0000 mgr.gibba032.whopeo (mgr.24625) 4947 : cluster [DBG] pgmap v2807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:06.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[30791]: audit 2021-03-10T15:26:05.290504+0000 mgr.gibba032.whopeo (mgr.24625) 4948 : audit [DBG] from='client.47367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:06.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[30791]: audit 2021-03-10T15:26:05.779722+0000 mon.c (mon.1) 92 : audit [DBG] from='client.? 172.21.2.111:0/436478431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[31774]: cluster 2021-03-10T15:26:04.493788+0000 mgr.gibba032.whopeo (mgr.24625) 4947 : cluster [DBG] pgmap v2807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[31774]: audit 2021-03-10T15:26:05.290504+0000 mgr.gibba032.whopeo (mgr.24625) 4948 : audit [DBG] from='client.47367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:06.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:06 gibba011 bash[31774]: audit 2021-03-10T15:26:05.779722+0000 mon.c (mon.1) 92 : audit [DBG] from='client.? 172.21.2.111:0/436478431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:07 gibba032 bash[24004]: audit 2021-03-10T15:26:05.516576+0000 mgr.gibba032.whopeo (mgr.24625) 4949 : audit [DBG] from='client.47373 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:07 gibba011 bash[30791]: audit 2021-03-10T15:26:05.516576+0000 mgr.gibba032.whopeo (mgr.24625) 4949 : audit [DBG] from='client.47373 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:07.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:07 gibba011 bash[31774]: audit 2021-03-10T15:26:05.516576+0000 mgr.gibba032.whopeo (mgr.24625) 4949 : audit [DBG] from='client.47373 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:08 gibba032 bash[24004]: cluster 2021-03-10T15:26:06.495253+0000 mgr.gibba032.whopeo (mgr.24625) 4950 : cluster [DBG] pgmap v2808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:08 gibba011 bash[30791]: cluster 2021-03-10T15:26:06.495253+0000 mgr.gibba032.whopeo (mgr.24625) 4950 : cluster [DBG] pgmap v2808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:08 gibba011 bash[31774]: cluster 2021-03-10T15:26:06.495253+0000 mgr.gibba032.whopeo (mgr.24625) 4950 : cluster [DBG] pgmap v2808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:09.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:09 gibba032 bash[24004]: cluster 2021-03-10T15:26:08.497012+0000 mgr.gibba032.whopeo (mgr.24625) 4951 : cluster [DBG] pgmap v2809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:09.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:09 gibba011 bash[30791]: cluster 2021-03-10T15:26:08.497012+0000 mgr.gibba032.whopeo (mgr.24625) 4951 : cluster [DBG] pgmap v2809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:09.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:09 gibba011 bash[31774]: cluster 2021-03-10T15:26:08.497012+0000 mgr.gibba032.whopeo (mgr.24625) 4951 : cluster [DBG] pgmap v2809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:11 gibba011 bash[30791]: cluster 2021-03-10T15:26:10.497736+0000 mgr.gibba032.whopeo (mgr.24625) 4952 : cluster [DBG] pgmap v2810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:11.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:11 gibba011 bash[31774]: cluster 2021-03-10T15:26:10.497736+0000 mgr.gibba032.whopeo (mgr.24625) 4952 : cluster [DBG] pgmap v2810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:12.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:11 gibba032 bash[24004]: cluster 2021-03-10T15:26:10.497736+0000 mgr.gibba032.whopeo (mgr.24625) 4952 : cluster [DBG] pgmap v2810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:14.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:13 gibba032 bash[24004]: cluster 2021-03-10T15:26:12.499378+0000 mgr.gibba032.whopeo (mgr.24625) 4953 : cluster [DBG] pgmap v2811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:13 gibba011 bash[30791]: cluster 2021-03-10T15:26:12.499378+0000 mgr.gibba032.whopeo (mgr.24625) 4953 : cluster [DBG] pgmap v2811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:14.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:13 gibba011 bash[31774]: cluster 2021-03-10T15:26:12.499378+0000 mgr.gibba032.whopeo (mgr.24625) 4953 : cluster [DBG] pgmap v2811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:15 gibba032 bash[24004]: cluster 2021-03-10T15:26:14.500916+0000 mgr.gibba032.whopeo (mgr.24625) 4954 : cluster [DBG] pgmap v2812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:15 gibba032 bash[24004]: cephadm 2021-03-10T15:26:14.540597+0000 mgr.gibba032.whopeo (mgr.24625) 4955 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:16.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:15 gibba011 bash[30791]: cluster 2021-03-10T15:26:14.500916+0000 mgr.gibba032.whopeo (mgr.24625) 4954 : cluster [DBG] pgmap v2812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:16.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:15 gibba011 bash[30791]: cephadm 2021-03-10T15:26:14.540597+0000 mgr.gibba032.whopeo (mgr.24625) 4955 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:16.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:15 gibba011 bash[31774]: cluster 2021-03-10T15:26:14.500916+0000 mgr.gibba032.whopeo (mgr.24625) 4954 : cluster [DBG] pgmap v2812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:16.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:15 gibba011 bash[31774]: cephadm 2021-03-10T15:26:14.540597+0000 mgr.gibba032.whopeo (mgr.24625) 4955 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:17 gibba032 bash[24004]: cluster 2021-03-10T15:26:16.501557+0000 mgr.gibba032.whopeo (mgr.24625) 4956 : cluster [DBG] pgmap v2813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:18.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:17 gibba011 bash[30791]: cluster 2021-03-10T15:26:16.501557+0000 mgr.gibba032.whopeo (mgr.24625) 4956 : cluster [DBG] pgmap v2813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:18.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:17 gibba011 bash[31774]: cluster 2021-03-10T15:26:16.501557+0000 mgr.gibba032.whopeo (mgr.24625) 4956 : cluster [DBG] pgmap v2813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:19 gibba032 bash[24004]: cluster 2021-03-10T15:26:18.503069+0000 mgr.gibba032.whopeo (mgr.24625) 4957 : cluster [DBG] pgmap v2814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:20.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:19 gibba011 bash[30791]: cluster 2021-03-10T15:26:18.503069+0000 mgr.gibba032.whopeo (mgr.24625) 4957 : cluster [DBG] pgmap v2814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:20.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:19 gibba011 bash[31774]: cluster 2021-03-10T15:26:18.503069+0000 mgr.gibba032.whopeo (mgr.24625) 4957 : cluster [DBG] pgmap v2814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:21 gibba032 bash[24004]: cluster 2021-03-10T15:26:20.504660+0000 mgr.gibba032.whopeo (mgr.24625) 4958 : cluster [DBG] pgmap v2815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:21 gibba011 bash[30791]: cluster 2021-03-10T15:26:20.504660+0000 mgr.gibba032.whopeo (mgr.24625) 4958 : cluster [DBG] pgmap v2815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:22.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:21 gibba011 bash[31774]: cluster 2021-03-10T15:26:20.504660+0000 mgr.gibba032.whopeo (mgr.24625) 4958 : cluster [DBG] pgmap v2815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:24.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:23 gibba032 bash[24004]: cluster 2021-03-10T15:26:22.505326+0000 mgr.gibba032.whopeo (mgr.24625) 4959 : cluster [DBG] pgmap v2816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:24.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:23 gibba011 bash[30791]: cluster 2021-03-10T15:26:22.505326+0000 mgr.gibba032.whopeo (mgr.24625) 4959 : cluster [DBG] pgmap v2816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:24.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:23 gibba011 bash[31774]: cluster 2021-03-10T15:26:22.505326+0000 mgr.gibba032.whopeo (mgr.24625) 4959 : cluster [DBG] pgmap v2816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:25 gibba032 bash[24004]: cluster 2021-03-10T15:26:24.506583+0000 mgr.gibba032.whopeo (mgr.24625) 4960 : cluster [DBG] pgmap v2817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:26.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:25 gibba011 bash[30791]: cluster 2021-03-10T15:26:24.506583+0000 mgr.gibba032.whopeo (mgr.24625) 4960 : cluster [DBG] pgmap v2817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:26.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:25 gibba011 bash[31774]: cluster 2021-03-10T15:26:24.506583+0000 mgr.gibba032.whopeo (mgr.24625) 4960 : cluster [DBG] pgmap v2817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:27 gibba032 bash[24004]: cluster 2021-03-10T15:26:26.508175+0000 mgr.gibba032.whopeo (mgr.24625) 4961 : cluster [DBG] pgmap v2818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:27 gibba011 bash[30791]: cluster 2021-03-10T15:26:26.508175+0000 mgr.gibba032.whopeo (mgr.24625) 4961 : cluster [DBG] pgmap v2818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:28.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:27 gibba011 bash[31774]: cluster 2021-03-10T15:26:26.508175+0000 mgr.gibba032.whopeo (mgr.24625) 4961 : cluster [DBG] pgmap v2818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:29 gibba032 bash[24004]: cluster 2021-03-10T15:26:28.508882+0000 mgr.gibba032.whopeo (mgr.24625) 4962 : cluster [DBG] pgmap v2819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:30.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:29 gibba011 bash[30791]: cluster 2021-03-10T15:26:28.508882+0000 mgr.gibba032.whopeo (mgr.24625) 4962 : cluster [DBG] pgmap v2819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:30.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:29 gibba011 bash[31774]: cluster 2021-03-10T15:26:28.508882+0000 mgr.gibba032.whopeo (mgr.24625) 4962 : cluster [DBG] pgmap v2819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:30 gibba032 bash[24004]: cephadm 2021-03-10T15:26:29.556681+0000 mgr.gibba032.whopeo (mgr.24625) 4963 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:30 gibba011 bash[30791]: cephadm 2021-03-10T15:26:29.556681+0000 mgr.gibba032.whopeo (mgr.24625) 4963 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:31.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:30 gibba011 bash[31774]: cephadm 2021-03-10T15:26:29.556681+0000 mgr.gibba032.whopeo (mgr.24625) 4963 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:32.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:31 gibba032 bash[24004]: cluster 2021-03-10T15:26:30.510371+0000 mgr.gibba032.whopeo (mgr.24625) 4964 : cluster [DBG] pgmap v2820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:32.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:31 gibba011 bash[30791]: cluster 2021-03-10T15:26:30.510371+0000 mgr.gibba032.whopeo (mgr.24625) 4964 : cluster [DBG] pgmap v2820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:31 gibba011 bash[31774]: cluster 2021-03-10T15:26:30.510371+0000 mgr.gibba032.whopeo (mgr.24625) 4964 : cluster [DBG] pgmap v2820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:34.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:33 gibba011 bash[30791]: cluster 2021-03-10T15:26:32.512190+0000 mgr.gibba032.whopeo (mgr.24625) 4965 : cluster [DBG] pgmap v2821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:34.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:33 gibba011 bash[31774]: cluster 2021-03-10T15:26:32.512190+0000 mgr.gibba032.whopeo (mgr.24625) 4965 : cluster [DBG] pgmap v2821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:33 gibba032 bash[24004]: cluster 2021-03-10T15:26:32.512190+0000 mgr.gibba032.whopeo (mgr.24625) 4965 : cluster [DBG] pgmap v2821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:36.021 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:26:36.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:35 gibba011 bash[30791]: cluster 2021-03-10T15:26:34.513392+0000 mgr.gibba032.whopeo (mgr.24625) 4966 : cluster [DBG] pgmap v2822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:36.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:35 gibba011 bash[31774]: cluster 2021-03-10T15:26:34.513392+0000 mgr.gibba032.whopeo (mgr.24625) 4966 : cluster [DBG] pgmap v2822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:36.250 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:26:36.250 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (93m) 7m ago 98m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:26:36.250 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (98m) 6m ago 98m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:26:36.250 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (93m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:26:36.251 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (93m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:26:36.251 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (92m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:26:36.251 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (92m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:26:36.251 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (92m) 6m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:26:36.252 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (92m) 7m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:26:36.252 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (98m) 7m ago 98m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:26:36.252 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (98m) 6m ago 98m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:26:36.252 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (91m) 7m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:26:36.253 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (91m) 7m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:26:36.253 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (90m) 7m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:26:36.253 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:26:36.253 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (89m) 6m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:26:36.253 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (89m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:26:36.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (88m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:26:36.254 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (88m) 6m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:26:36.254 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (98m) 6m ago 98m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:26:36.254 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 96m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:26:36.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:35 gibba032 bash[24004]: cluster 2021-03-10T15:26:34.513392+0000 mgr.gibba032.whopeo (mgr.24625) 4966 : cluster [DBG] pgmap v2822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:36.497 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:26:36.498 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:26:36.499 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:26:36.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:36.499 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:26:36.499 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:26:36.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:36.500 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:26:36.500 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:26:36.500 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:26:36.500 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:26:36.501 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:26:36.501 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:26:36.501 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:26:36.501 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:26:37.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[30791]: audit 2021-03-10T15:26:36.008473+0000 mgr.gibba032.whopeo (mgr.24625) 4967 : audit [DBG] from='client.57370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[30791]: audit 2021-03-10T15:26:36.232316+0000 mgr.gibba032.whopeo (mgr.24625) 4968 : audit [DBG] from='client.47391 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[30791]: audit 2021-03-10T15:26:36.494478+0000 mon.a (mon.0) 1967 : audit [DBG] from='client.? 172.21.2.111:0/2670727906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[31774]: audit 2021-03-10T15:26:36.008473+0000 mgr.gibba032.whopeo (mgr.24625) 4967 : audit [DBG] from='client.57370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[31774]: audit 2021-03-10T15:26:36.232316+0000 mgr.gibba032.whopeo (mgr.24625) 4968 : audit [DBG] from='client.47391 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:36 gibba011 bash[31774]: audit 2021-03-10T15:26:36.494478+0000 mon.a (mon.0) 1967 : audit [DBG] from='client.? 172.21.2.111:0/2670727906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:36 gibba032 bash[24004]: audit 2021-03-10T15:26:36.008473+0000 mgr.gibba032.whopeo (mgr.24625) 4967 : audit [DBG] from='client.57370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:36 gibba032 bash[24004]: audit 2021-03-10T15:26:36.232316+0000 mgr.gibba032.whopeo (mgr.24625) 4968 : audit [DBG] from='client.47391 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:26:37.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:36 gibba032 bash[24004]: audit 2021-03-10T15:26:36.494478+0000 mon.a (mon.0) 1967 : audit [DBG] from='client.? 172.21.2.111:0/2670727906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:26:38.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:37 gibba011 bash[30791]: cluster 2021-03-10T15:26:36.515054+0000 mgr.gibba032.whopeo (mgr.24625) 4969 : cluster [DBG] pgmap v2823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:38.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:37 gibba011 bash[31774]: cluster 2021-03-10T15:26:36.515054+0000 mgr.gibba032.whopeo (mgr.24625) 4969 : cluster [DBG] pgmap v2823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:37 gibba032 bash[24004]: cluster 2021-03-10T15:26:36.515054+0000 mgr.gibba032.whopeo (mgr.24625) 4969 : cluster [DBG] pgmap v2823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:39 gibba011 bash[30791]: cluster 2021-03-10T15:26:38.516578+0000 mgr.gibba032.whopeo (mgr.24625) 4970 : cluster [DBG] pgmap v2824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:39 gibba011 bash[31774]: cluster 2021-03-10T15:26:38.516578+0000 mgr.gibba032.whopeo (mgr.24625) 4970 : cluster [DBG] pgmap v2824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:39 gibba032 bash[24004]: cluster 2021-03-10T15:26:38.516578+0000 mgr.gibba032.whopeo (mgr.24625) 4970 : cluster [DBG] pgmap v2824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:42.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:41 gibba011 bash[30791]: cluster 2021-03-10T15:26:40.517493+0000 mgr.gibba032.whopeo (mgr.24625) 4971 : cluster [DBG] pgmap v2825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:41 gibba011 bash[31774]: cluster 2021-03-10T15:26:40.517493+0000 mgr.gibba032.whopeo (mgr.24625) 4971 : cluster [DBG] pgmap v2825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:41 gibba032 bash[24004]: cluster 2021-03-10T15:26:40.517493+0000 mgr.gibba032.whopeo (mgr.24625) 4971 : cluster [DBG] pgmap v2825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:44.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:43 gibba011 bash[30791]: cluster 2021-03-10T15:26:42.519125+0000 mgr.gibba032.whopeo (mgr.24625) 4972 : cluster [DBG] pgmap v2826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:43 gibba011 bash[31774]: cluster 2021-03-10T15:26:42.519125+0000 mgr.gibba032.whopeo (mgr.24625) 4972 : cluster [DBG] pgmap v2826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:43 gibba032 bash[24004]: cluster 2021-03-10T15:26:42.519125+0000 mgr.gibba032.whopeo (mgr.24625) 4972 : cluster [DBG] pgmap v2826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:46.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:45 gibba011 bash[30791]: cluster 2021-03-10T15:26:44.520714+0000 mgr.gibba032.whopeo (mgr.24625) 4973 : cluster [DBG] pgmap v2827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:46.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:45 gibba011 bash[30791]: cephadm 2021-03-10T15:26:44.572506+0000 mgr.gibba032.whopeo (mgr.24625) 4974 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:46.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:45 gibba011 bash[31774]: cluster 2021-03-10T15:26:44.520714+0000 mgr.gibba032.whopeo (mgr.24625) 4973 : cluster [DBG] pgmap v2827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:46.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:45 gibba011 bash[31774]: cephadm 2021-03-10T15:26:44.572506+0000 mgr.gibba032.whopeo (mgr.24625) 4974 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:45 gibba032 bash[24004]: cluster 2021-03-10T15:26:44.520714+0000 mgr.gibba032.whopeo (mgr.24625) 4973 : cluster [DBG] pgmap v2827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:45 gibba032 bash[24004]: cephadm 2021-03-10T15:26:44.572506+0000 mgr.gibba032.whopeo (mgr.24625) 4974 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:26:48.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:47 gibba011 bash[30791]: cluster 2021-03-10T15:26:46.521450+0000 mgr.gibba032.whopeo (mgr.24625) 4975 : cluster [DBG] pgmap v2828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:48.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:47 gibba011 bash[31774]: cluster 2021-03-10T15:26:46.521450+0000 mgr.gibba032.whopeo (mgr.24625) 4975 : cluster [DBG] pgmap v2828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:47 gibba032 bash[24004]: cluster 2021-03-10T15:26:46.521450+0000 mgr.gibba032.whopeo (mgr.24625) 4975 : cluster [DBG] pgmap v2828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:49 gibba032 bash[24004]: cluster 2021-03-10T15:26:48.522879+0000 mgr.gibba032.whopeo (mgr.24625) 4976 : cluster [DBG] pgmap v2829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:49 gibba011 bash[30791]: cluster 2021-03-10T15:26:48.522879+0000 mgr.gibba032.whopeo (mgr.24625) 4976 : cluster [DBG] pgmap v2829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:49 gibba011 bash[31774]: cluster 2021-03-10T15:26:48.522879+0000 mgr.gibba032.whopeo (mgr.24625) 4976 : cluster [DBG] pgmap v2829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:51 gibba032 bash[24004]: cluster 2021-03-10T15:26:50.524371+0000 mgr.gibba032.whopeo (mgr.24625) 4977 : cluster [DBG] pgmap v2830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:51 gibba011 bash[30791]: cluster 2021-03-10T15:26:50.524371+0000 mgr.gibba032.whopeo (mgr.24625) 4977 : cluster [DBG] pgmap v2830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:52.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:51 gibba011 bash[31774]: cluster 2021-03-10T15:26:50.524371+0000 mgr.gibba032.whopeo (mgr.24625) 4977 : cluster [DBG] pgmap v2830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:53 gibba032 bash[24004]: cluster 2021-03-10T15:26:52.525250+0000 mgr.gibba032.whopeo (mgr.24625) 4978 : cluster [DBG] pgmap v2831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:53 gibba011 bash[30791]: cluster 2021-03-10T15:26:52.525250+0000 mgr.gibba032.whopeo (mgr.24625) 4978 : cluster [DBG] pgmap v2831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:54.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:53 gibba011 bash[31774]: cluster 2021-03-10T15:26:52.525250+0000 mgr.gibba032.whopeo (mgr.24625) 4978 : cluster [DBG] pgmap v2831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:55 gibba032 bash[24004]: cluster 2021-03-10T15:26:54.526623+0000 mgr.gibba032.whopeo (mgr.24625) 4979 : cluster [DBG] pgmap v2832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:55 gibba011 bash[30791]: cluster 2021-03-10T15:26:54.526623+0000 mgr.gibba032.whopeo (mgr.24625) 4979 : cluster [DBG] pgmap v2832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:55 gibba011 bash[31774]: cluster 2021-03-10T15:26:54.526623+0000 mgr.gibba032.whopeo (mgr.24625) 4979 : cluster [DBG] pgmap v2832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.598292+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:26:57.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.601442+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:26:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.614099+0000 mon.a (mon.0) 1970 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:26:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.614839+0000 mon.a (mon.0) 1971 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.615556+0000 mon.a (mon.0) 1972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:56 gibba032 bash[24004]: audit 2021-03-10T15:26:56.616247+0000 mon.a (mon.0) 1973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:26:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.598292+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:26:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.601442+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:26:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.614099+0000 mon.a (mon.0) 1970 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:26:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.614839+0000 mon.a (mon.0) 1971 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.615556+0000 mon.a (mon.0) 1972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[30791]: audit 2021-03-10T15:26:56.616247+0000 mon.a (mon.0) 1973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:26:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.598292+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:26:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.601442+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:26:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.614099+0000 mon.a (mon.0) 1970 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:26:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.614839+0000 mon.a (mon.0) 1971 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.615556+0000 mon.a (mon.0) 1972 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:26:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:56 gibba011 bash[31774]: audit 2021-03-10T15:26:56.616247+0000 mon.a (mon.0) 1973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:26:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:57 gibba032 bash[24004]: cluster 2021-03-10T15:26:56.528459+0000 mgr.gibba032.whopeo (mgr.24625) 4980 : cluster [DBG] pgmap v2833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:57 gibba011 bash[30791]: cluster 2021-03-10T15:26:56.528459+0000 mgr.gibba032.whopeo (mgr.24625) 4980 : cluster [DBG] pgmap v2833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:57 gibba011 bash[31774]: cluster 2021-03-10T15:26:56.528459+0000 mgr.gibba032.whopeo (mgr.24625) 4980 : cluster [DBG] pgmap v2833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: cluster 2021-03-10T15:26:58.529826+0000 mgr.gibba032.whopeo (mgr.24625) 4981 : cluster [DBG] pgmap v2834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.787121+0000 mon.a (mon.0) 1974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:27:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.790485+0000 mon.a (mon.0) 1975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:27:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.792826+0000 mon.a (mon.0) 1976 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.795079+0000 mon.a (mon.0) 1977 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.797395+0000 mon.a (mon.0) 1978 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.801911+0000 mon.a (mon.0) 1979 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:26:59 gibba032 bash[24004]: audit 2021-03-10T15:26:59.804604+0000 mon.a (mon.0) 1980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: cluster 2021-03-10T15:26:58.529826+0000 mgr.gibba032.whopeo (mgr.24625) 4981 : cluster [DBG] pgmap v2834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.787121+0000 mon.a (mon.0) 1974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:27:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.790485+0000 mon.a (mon.0) 1975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:27:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.792826+0000 mon.a (mon.0) 1976 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.795079+0000 mon.a (mon.0) 1977 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.797395+0000 mon.a (mon.0) 1978 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.801911+0000 mon.a (mon.0) 1979 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[30791]: audit 2021-03-10T15:26:59.804604+0000 mon.a (mon.0) 1980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: cluster 2021-03-10T15:26:58.529826+0000 mgr.gibba032.whopeo (mgr.24625) 4981 : cluster [DBG] pgmap v2834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.787121+0000 mon.a (mon.0) 1974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:27:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.790485+0000 mon.a (mon.0) 1975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:27:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.792826+0000 mon.a (mon.0) 1976 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.795079+0000 mon.a (mon.0) 1977 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.797395+0000 mon.a (mon.0) 1978 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.801911+0000 mon.a (mon.0) 1979 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:26:59 gibba011 bash[31774]: audit 2021-03-10T15:26:59.804604+0000 mon.a (mon.0) 1980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:01.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.577082+0000 mgr.gibba032.whopeo (mgr.24625) 4982 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:27:01.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.578415+0000 mgr.gibba032.whopeo (mgr.24625) 4983 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:27:01.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.788498+0000 mgr.gibba032.whopeo (mgr.24625) 4984 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:27:01.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.791224+0000 mgr.gibba032.whopeo (mgr.24625) 4985 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:27:01.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.793402+0000 mgr.gibba032.whopeo (mgr.24625) 4986 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:27:01.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.793438+0000 mgr.gibba032.whopeo (mgr.24625) 4987 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:27:01.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.795740+0000 mgr.gibba032.whopeo (mgr.24625) 4988 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:27:01.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.795776+0000 mgr.gibba032.whopeo (mgr.24625) 4989 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:27:01.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.798054+0000 mgr.gibba032.whopeo (mgr.24625) 4990 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:27:01.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.800077+0000 mgr.gibba032.whopeo (mgr.24625) 4991 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:27:01.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.800133+0000 mgr.gibba032.whopeo (mgr.24625) 4992 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:27:01.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.802718+0000 mgr.gibba032.whopeo (mgr.24625) 4993 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:27:01.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.803027+0000 mgr.gibba032.whopeo (mgr.24625) 4994 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:27:01.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.805273+0000 mgr.gibba032.whopeo (mgr.24625) 4995 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:27:01.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.805309+0000 mgr.gibba032.whopeo (mgr.24625) 4996 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:27:01.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:00 gibba032 bash[24004]: cephadm 2021-03-10T15:26:59.805455+0000 mgr.gibba032.whopeo (mgr.24625) 4997 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:01.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.577082+0000 mgr.gibba032.whopeo (mgr.24625) 4982 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:27:01.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.578415+0000 mgr.gibba032.whopeo (mgr.24625) 4983 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:27:01.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.788498+0000 mgr.gibba032.whopeo (mgr.24625) 4984 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:27:01.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.791224+0000 mgr.gibba032.whopeo (mgr.24625) 4985 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:27:01.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.793402+0000 mgr.gibba032.whopeo (mgr.24625) 4986 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:27:01.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.793438+0000 mgr.gibba032.whopeo (mgr.24625) 4987 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:27:01.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.795740+0000 mgr.gibba032.whopeo (mgr.24625) 4988 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:27:01.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.795776+0000 mgr.gibba032.whopeo (mgr.24625) 4989 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:27:01.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.798054+0000 mgr.gibba032.whopeo (mgr.24625) 4990 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:27:01.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.800077+0000 mgr.gibba032.whopeo (mgr.24625) 4991 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:27:01.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.800133+0000 mgr.gibba032.whopeo (mgr.24625) 4992 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:27:01.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.802718+0000 mgr.gibba032.whopeo (mgr.24625) 4993 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:27:01.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.803027+0000 mgr.gibba032.whopeo (mgr.24625) 4994 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:27:01.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.805273+0000 mgr.gibba032.whopeo (mgr.24625) 4995 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:27:01.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.805309+0000 mgr.gibba032.whopeo (mgr.24625) 4996 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:27:01.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[30791]: cephadm 2021-03-10T15:26:59.805455+0000 mgr.gibba032.whopeo (mgr.24625) 4997 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:01.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.577082+0000 mgr.gibba032.whopeo (mgr.24625) 4982 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:27:01.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.578415+0000 mgr.gibba032.whopeo (mgr.24625) 4983 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:27:01.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.788498+0000 mgr.gibba032.whopeo (mgr.24625) 4984 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:27:01.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.791224+0000 mgr.gibba032.whopeo (mgr.24625) 4985 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:27:01.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.793402+0000 mgr.gibba032.whopeo (mgr.24625) 4986 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:27:01.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.793438+0000 mgr.gibba032.whopeo (mgr.24625) 4987 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:27:01.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.795740+0000 mgr.gibba032.whopeo (mgr.24625) 4988 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:27:01.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.795776+0000 mgr.gibba032.whopeo (mgr.24625) 4989 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:27:01.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.798054+0000 mgr.gibba032.whopeo (mgr.24625) 4990 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:27:01.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.800077+0000 mgr.gibba032.whopeo (mgr.24625) 4991 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:27:01.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.800133+0000 mgr.gibba032.whopeo (mgr.24625) 4992 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:27:01.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.802718+0000 mgr.gibba032.whopeo (mgr.24625) 4993 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:27:01.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.803027+0000 mgr.gibba032.whopeo (mgr.24625) 4994 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:27:01.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.805273+0000 mgr.gibba032.whopeo (mgr.24625) 4995 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:27:01.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.805309+0000 mgr.gibba032.whopeo (mgr.24625) 4996 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:27:01.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:00 gibba011 bash[31774]: cephadm 2021-03-10T15:26:59.805455+0000 mgr.gibba032.whopeo (mgr.24625) 4997 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:01 gibba032 bash[24004]: cluster 2021-03-10T15:27:00.531472+0000 mgr.gibba032.whopeo (mgr.24625) 4998 : cluster [DBG] pgmap v2835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:01 gibba011 bash[30791]: cluster 2021-03-10T15:27:00.531472+0000 mgr.gibba032.whopeo (mgr.24625) 4998 : cluster [DBG] pgmap v2835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:01 gibba011 bash[31774]: cluster 2021-03-10T15:27:00.531472+0000 mgr.gibba032.whopeo (mgr.24625) 4998 : cluster [DBG] pgmap v2835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:03 gibba032 bash[24004]: cluster 2021-03-10T15:27:02.533296+0000 mgr.gibba032.whopeo (mgr.24625) 4999 : cluster [DBG] pgmap v2836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:03 gibba011 bash[30791]: cluster 2021-03-10T15:27:02.533296+0000 mgr.gibba032.whopeo (mgr.24625) 4999 : cluster [DBG] pgmap v2836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:04.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:03 gibba011 bash[31774]: cluster 2021-03-10T15:27:02.533296+0000 mgr.gibba032.whopeo (mgr.24625) 4999 : cluster [DBG] pgmap v2836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:06.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:05 gibba032 bash[24004]: cluster 2021-03-10T15:27:04.534205+0000 mgr.gibba032.whopeo (mgr.24625) 5000 : cluster [DBG] pgmap v2837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:05 gibba011 bash[30791]: cluster 2021-03-10T15:27:04.534205+0000 mgr.gibba032.whopeo (mgr.24625) 5000 : cluster [DBG] pgmap v2837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:06.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:05 gibba011 bash[31774]: cluster 2021-03-10T15:27:04.534205+0000 mgr.gibba032.whopeo (mgr.24625) 5000 : cluster [DBG] pgmap v2837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:06.742 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:27:06.967 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:27:06.967 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (93m) 8m ago 99m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:27:06.968 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (98m) 7m ago 98m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:27:06.968 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (94m) 7m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:27:06.968 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (94m) 7m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:27:06.968 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (93m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:27:06.969 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (93m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:27:06.969 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (92m) 7m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:27:06.969 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (93m) 8m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:27:06.969 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (99m) 8m ago 99m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:27:06.969 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (99m) 7m ago 99m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:27:06.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (92m) 8m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:27:06.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:27:06.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:27:06.970 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (90m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:27:06.971 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:27:06.971 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (89m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:27:06.971 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (89m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:27:06.971 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (89m) 7m ago 99m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:27:06.972 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (98m) 7m ago 99m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:27:06.972 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 97m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:27:07.220 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:27:07.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:27:07.220 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:27:07.220 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:07.221 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:27:07.221 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:27:07.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:07.221 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:27:07.222 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:27:07.222 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:07.222 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:27:07.222 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:27:07.223 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:27:07.223 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:27:07.223 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:27:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:07 gibba032 bash[24004]: cluster 2021-03-10T15:27:06.535820+0000 mgr.gibba032.whopeo (mgr.24625) 5001 : cluster [DBG] pgmap v2838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:07 gibba032 bash[24004]: audit 2021-03-10T15:27:06.728937+0000 mgr.gibba032.whopeo (mgr.24625) 5002 : audit [DBG] from='client.57388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:07 gibba032 bash[24004]: audit 2021-03-10T15:27:06.954448+0000 mgr.gibba032.whopeo (mgr.24625) 5003 : audit [DBG] from='client.57394 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:07 gibba032 bash[24004]: audit 2021-03-10T15:27:07.216640+0000 mon.a (mon.0) 1981 : audit [DBG] from='client.? 172.21.2.111:0/2866706768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[30791]: cluster 2021-03-10T15:27:06.535820+0000 mgr.gibba032.whopeo (mgr.24625) 5001 : cluster [DBG] pgmap v2838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[30791]: audit 2021-03-10T15:27:06.728937+0000 mgr.gibba032.whopeo (mgr.24625) 5002 : audit [DBG] from='client.57388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[30791]: audit 2021-03-10T15:27:06.954448+0000 mgr.gibba032.whopeo (mgr.24625) 5003 : audit [DBG] from='client.57394 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[30791]: audit 2021-03-10T15:27:07.216640+0000 mon.a (mon.0) 1981 : audit [DBG] from='client.? 172.21.2.111:0/2866706768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:08.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[31774]: cluster 2021-03-10T15:27:06.535820+0000 mgr.gibba032.whopeo (mgr.24625) 5001 : cluster [DBG] pgmap v2838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:08.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[31774]: audit 2021-03-10T15:27:06.728937+0000 mgr.gibba032.whopeo (mgr.24625) 5002 : audit [DBG] from='client.57388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[31774]: audit 2021-03-10T15:27:06.954448+0000 mgr.gibba032.whopeo (mgr.24625) 5003 : audit [DBG] from='client.57394 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:08.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:07 gibba011 bash[31774]: audit 2021-03-10T15:27:07.216640+0000 mon.a (mon.0) 1981 : audit [DBG] from='client.? 172.21.2.111:0/2866706768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:09 gibba032 bash[24004]: cluster 2021-03-10T15:27:08.537445+0000 mgr.gibba032.whopeo (mgr.24625) 5004 : cluster [DBG] pgmap v2839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:09 gibba011 bash[30791]: cluster 2021-03-10T15:27:08.537445+0000 mgr.gibba032.whopeo (mgr.24625) 5004 : cluster [DBG] pgmap v2839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:09 gibba011 bash[31774]: cluster 2021-03-10T15:27:08.537445+0000 mgr.gibba032.whopeo (mgr.24625) 5004 : cluster [DBG] pgmap v2839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:11 gibba032 bash[24004]: cluster 2021-03-10T15:27:10.538443+0000 mgr.gibba032.whopeo (mgr.24625) 5005 : cluster [DBG] pgmap v2840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:11 gibba011 bash[30791]: cluster 2021-03-10T15:27:10.538443+0000 mgr.gibba032.whopeo (mgr.24625) 5005 : cluster [DBG] pgmap v2840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:11 gibba011 bash[31774]: cluster 2021-03-10T15:27:10.538443+0000 mgr.gibba032.whopeo (mgr.24625) 5005 : cluster [DBG] pgmap v2840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:13 gibba032 bash[24004]: cluster 2021-03-10T15:27:12.540275+0000 mgr.gibba032.whopeo (mgr.24625) 5006 : cluster [DBG] pgmap v2841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:13 gibba011 bash[30791]: cluster 2021-03-10T15:27:12.540275+0000 mgr.gibba032.whopeo (mgr.24625) 5006 : cluster [DBG] pgmap v2841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:13 gibba011 bash[31774]: cluster 2021-03-10T15:27:12.540275+0000 mgr.gibba032.whopeo (mgr.24625) 5006 : cluster [DBG] pgmap v2841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:16.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:15 gibba032 bash[24004]: cluster 2021-03-10T15:27:14.541528+0000 mgr.gibba032.whopeo (mgr.24625) 5007 : cluster [DBG] pgmap v2842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:15 gibba032 bash[24004]: cephadm 2021-03-10T15:27:14.821475+0000 mgr.gibba032.whopeo (mgr.24625) 5008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:15 gibba011 bash[30791]: cluster 2021-03-10T15:27:14.541528+0000 mgr.gibba032.whopeo (mgr.24625) 5007 : cluster [DBG] pgmap v2842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:16.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:15 gibba011 bash[30791]: cephadm 2021-03-10T15:27:14.821475+0000 mgr.gibba032.whopeo (mgr.24625) 5008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:15 gibba011 bash[31774]: cluster 2021-03-10T15:27:14.541528+0000 mgr.gibba032.whopeo (mgr.24625) 5007 : cluster [DBG] pgmap v2842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:15 gibba011 bash[31774]: cephadm 2021-03-10T15:27:14.821475+0000 mgr.gibba032.whopeo (mgr.24625) 5008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:17 gibba032 bash[24004]: cluster 2021-03-10T15:27:16.543268+0000 mgr.gibba032.whopeo (mgr.24625) 5009 : cluster [DBG] pgmap v2843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:17 gibba011 bash[30791]: cluster 2021-03-10T15:27:16.543268+0000 mgr.gibba032.whopeo (mgr.24625) 5009 : cluster [DBG] pgmap v2843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:17 gibba011 bash[31774]: cluster 2021-03-10T15:27:16.543268+0000 mgr.gibba032.whopeo (mgr.24625) 5009 : cluster [DBG] pgmap v2843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:19 gibba032 bash[24004]: cluster 2021-03-10T15:27:18.544923+0000 mgr.gibba032.whopeo (mgr.24625) 5010 : cluster [DBG] pgmap v2844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:19 gibba011 bash[30791]: cluster 2021-03-10T15:27:18.544923+0000 mgr.gibba032.whopeo (mgr.24625) 5010 : cluster [DBG] pgmap v2844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:20.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:19 gibba011 bash[31774]: cluster 2021-03-10T15:27:18.544923+0000 mgr.gibba032.whopeo (mgr.24625) 5010 : cluster [DBG] pgmap v2844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:21 gibba032 bash[24004]: cluster 2021-03-10T15:27:20.546278+0000 mgr.gibba032.whopeo (mgr.24625) 5011 : cluster [DBG] pgmap v2845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:21 gibba011 bash[31774]: cluster 2021-03-10T15:27:20.546278+0000 mgr.gibba032.whopeo (mgr.24625) 5011 : cluster [DBG] pgmap v2845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:21 gibba011 bash[30791]: cluster 2021-03-10T15:27:20.546278+0000 mgr.gibba032.whopeo (mgr.24625) 5011 : cluster [DBG] pgmap v2845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:24 gibba032 bash[24004]: cluster 2021-03-10T15:27:22.547776+0000 mgr.gibba032.whopeo (mgr.24625) 5012 : cluster [DBG] pgmap v2846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:24 gibba011 bash[30791]: cluster 2021-03-10T15:27:22.547776+0000 mgr.gibba032.whopeo (mgr.24625) 5012 : cluster [DBG] pgmap v2846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:24 gibba011 bash[31774]: cluster 2021-03-10T15:27:22.547776+0000 mgr.gibba032.whopeo (mgr.24625) 5012 : cluster [DBG] pgmap v2846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:26 gibba011 bash[30791]: cluster 2021-03-10T15:27:24.549191+0000 mgr.gibba032.whopeo (mgr.24625) 5013 : cluster [DBG] pgmap v2847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:26 gibba011 bash[31774]: cluster 2021-03-10T15:27:24.549191+0000 mgr.gibba032.whopeo (mgr.24625) 5013 : cluster [DBG] pgmap v2847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:26 gibba032 bash[24004]: cluster 2021-03-10T15:27:24.549191+0000 mgr.gibba032.whopeo (mgr.24625) 5013 : cluster [DBG] pgmap v2847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:28 gibba011 bash[30791]: cluster 2021-03-10T15:27:26.551017+0000 mgr.gibba032.whopeo (mgr.24625) 5014 : cluster [DBG] pgmap v2848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:28 gibba011 bash[31774]: cluster 2021-03-10T15:27:26.551017+0000 mgr.gibba032.whopeo (mgr.24625) 5014 : cluster [DBG] pgmap v2848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:28 gibba032 bash[24004]: cluster 2021-03-10T15:27:26.551017+0000 mgr.gibba032.whopeo (mgr.24625) 5014 : cluster [DBG] pgmap v2848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:30 gibba032 bash[24004]: cluster 2021-03-10T15:27:28.551947+0000 mgr.gibba032.whopeo (mgr.24625) 5015 : cluster [DBG] pgmap v2849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:30 gibba011 bash[30791]: cluster 2021-03-10T15:27:28.551947+0000 mgr.gibba032.whopeo (mgr.24625) 5015 : cluster [DBG] pgmap v2849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:30 gibba011 bash[31774]: cluster 2021-03-10T15:27:28.551947+0000 mgr.gibba032.whopeo (mgr.24625) 5015 : cluster [DBG] pgmap v2849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:31.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:31 gibba011 bash[30791]: cephadm 2021-03-10T15:27:29.837270+0000 mgr.gibba032.whopeo (mgr.24625) 5016 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:31.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:31 gibba011 bash[31774]: cephadm 2021-03-10T15:27:29.837270+0000 mgr.gibba032.whopeo (mgr.24625) 5016 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:31.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:31 gibba032 bash[24004]: cephadm 2021-03-10T15:27:29.837270+0000 mgr.gibba032.whopeo (mgr.24625) 5016 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:32 gibba011 bash[30791]: cluster 2021-03-10T15:27:30.553306+0000 mgr.gibba032.whopeo (mgr.24625) 5017 : cluster [DBG] pgmap v2850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:32 gibba011 bash[31774]: cluster 2021-03-10T15:27:30.553306+0000 mgr.gibba032.whopeo (mgr.24625) 5017 : cluster [DBG] pgmap v2850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:32 gibba032 bash[24004]: cluster 2021-03-10T15:27:30.553306+0000 mgr.gibba032.whopeo (mgr.24625) 5017 : cluster [DBG] pgmap v2850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:34 gibba011 bash[30791]: cluster 2021-03-10T15:27:32.554957+0000 mgr.gibba032.whopeo (mgr.24625) 5018 : cluster [DBG] pgmap v2851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:34 gibba011 bash[31774]: cluster 2021-03-10T15:27:32.554957+0000 mgr.gibba032.whopeo (mgr.24625) 5018 : cluster [DBG] pgmap v2851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:34 gibba032 bash[24004]: cluster 2021-03-10T15:27:32.554957+0000 mgr.gibba032.whopeo (mgr.24625) 5018 : cluster [DBG] pgmap v2851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:36 gibba011 bash[30791]: cluster 2021-03-10T15:27:34.555873+0000 mgr.gibba032.whopeo (mgr.24625) 5019 : cluster [DBG] pgmap v2852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:36.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:36 gibba011 bash[31774]: cluster 2021-03-10T15:27:34.555873+0000 mgr.gibba032.whopeo (mgr.24625) 5019 : cluster [DBG] pgmap v2852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:36 gibba032 bash[24004]: cluster 2021-03-10T15:27:34.555873+0000 mgr.gibba032.whopeo (mgr.24625) 5019 : cluster [DBG] pgmap v2852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:37.465 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:27:37.695 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:27:37.695 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (94m) 8m ago 99m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:27:37.695 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (99m) 7m ago 99m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:27:37.696 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (94m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:27:37.696 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (94m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:27:37.696 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (93m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:27:37.696 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (93m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:27:37.696 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (93m) 7m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:27:37.697 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (93m) 8m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:27:37.697 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (99m) 8m ago 99m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:27:37.697 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (99m) 7m ago 99m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:27:37.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (92m) 8m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:27:37.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (92m) 8m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:27:37.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (91m) 8m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:27:37.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:27:37.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (90m) 7m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:27:37.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:27:37.699 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (90m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:27:37.699 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (89m) 7m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:27:37.699 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (99m) 7m ago 99m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:27:37.699 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 97m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:27:37.951 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:27:37.951 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:27:37.951 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:27:37.952 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:37.952 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:27:37.952 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:27:37.952 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:37.952 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:27:37.953 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:27:37.953 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:27:37.953 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:27:37.953 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:27:37.954 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:27:37.954 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:27:37.954 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:27:38.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:38 gibba011 bash[30791]: cluster 2021-03-10T15:27:36.557496+0000 mgr.gibba032.whopeo (mgr.24625) 5020 : cluster [DBG] pgmap v2853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:38 gibba011 bash[30791]: audit 2021-03-10T15:27:37.947900+0000 mon.c (mon.1) 93 : audit [DBG] from='client.? 172.21.2.111:0/3921626704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:38 gibba011 bash[31774]: cluster 2021-03-10T15:27:36.557496+0000 mgr.gibba032.whopeo (mgr.24625) 5020 : cluster [DBG] pgmap v2853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:38 gibba011 bash[31774]: audit 2021-03-10T15:27:37.947900+0000 mon.c (mon.1) 93 : audit [DBG] from='client.? 172.21.2.111:0/3921626704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:38 gibba032 bash[24004]: cluster 2021-03-10T15:27:36.557496+0000 mgr.gibba032.whopeo (mgr.24625) 5020 : cluster [DBG] pgmap v2853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:38 gibba032 bash[24004]: audit 2021-03-10T15:27:37.947900+0000 mon.c (mon.1) 93 : audit [DBG] from='client.? 172.21.2.111:0/3921626704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:27:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:39 gibba011 bash[30791]: audit 2021-03-10T15:27:37.452136+0000 mgr.gibba032.whopeo (mgr.24625) 5021 : audit [DBG] from='client.57406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:39 gibba011 bash[30791]: audit 2021-03-10T15:27:37.678923+0000 mgr.gibba032.whopeo (mgr.24625) 5022 : audit [DBG] from='client.57412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:39 gibba011 bash[31774]: audit 2021-03-10T15:27:37.452136+0000 mgr.gibba032.whopeo (mgr.24625) 5021 : audit [DBG] from='client.57406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:39 gibba011 bash[31774]: audit 2021-03-10T15:27:37.678923+0000 mgr.gibba032.whopeo (mgr.24625) 5022 : audit [DBG] from='client.57412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:39.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:39 gibba032 bash[24004]: audit 2021-03-10T15:27:37.452136+0000 mgr.gibba032.whopeo (mgr.24625) 5021 : audit [DBG] from='client.57406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:39.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:39 gibba032 bash[24004]: audit 2021-03-10T15:27:37.678923+0000 mgr.gibba032.whopeo (mgr.24625) 5022 : audit [DBG] from='client.57412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:27:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:40 gibba032 bash[24004]: cluster 2021-03-10T15:27:38.559015+0000 mgr.gibba032.whopeo (mgr.24625) 5023 : cluster [DBG] pgmap v2854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:40 gibba011 bash[30791]: cluster 2021-03-10T15:27:38.559015+0000 mgr.gibba032.whopeo (mgr.24625) 5023 : cluster [DBG] pgmap v2854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:40 gibba011 bash[31774]: cluster 2021-03-10T15:27:38.559015+0000 mgr.gibba032.whopeo (mgr.24625) 5023 : cluster [DBG] pgmap v2854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:42 gibba032 bash[24004]: cluster 2021-03-10T15:27:40.560624+0000 mgr.gibba032.whopeo (mgr.24625) 5024 : cluster [DBG] pgmap v2855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:42 gibba011 bash[30791]: cluster 2021-03-10T15:27:40.560624+0000 mgr.gibba032.whopeo (mgr.24625) 5024 : cluster [DBG] pgmap v2855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:42 gibba011 bash[31774]: cluster 2021-03-10T15:27:40.560624+0000 mgr.gibba032.whopeo (mgr.24625) 5024 : cluster [DBG] pgmap v2855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:44 gibba032 bash[24004]: cluster 2021-03-10T15:27:42.562247+0000 mgr.gibba032.whopeo (mgr.24625) 5025 : cluster [DBG] pgmap v2856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:44 gibba011 bash[30791]: cluster 2021-03-10T15:27:42.562247+0000 mgr.gibba032.whopeo (mgr.24625) 5025 : cluster [DBG] pgmap v2856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:44 gibba011 bash[31774]: cluster 2021-03-10T15:27:42.562247+0000 mgr.gibba032.whopeo (mgr.24625) 5025 : cluster [DBG] pgmap v2856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:46 gibba032 bash[24004]: cluster 2021-03-10T15:27:44.563630+0000 mgr.gibba032.whopeo (mgr.24625) 5026 : cluster [DBG] pgmap v2857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:46 gibba032 bash[24004]: cephadm 2021-03-10T15:27:44.847515+0000 mgr.gibba032.whopeo (mgr.24625) 5027 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:46 gibba011 bash[30791]: cluster 2021-03-10T15:27:44.563630+0000 mgr.gibba032.whopeo (mgr.24625) 5026 : cluster [DBG] pgmap v2857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:46 gibba011 bash[30791]: cephadm 2021-03-10T15:27:44.847515+0000 mgr.gibba032.whopeo (mgr.24625) 5027 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:46 gibba011 bash[31774]: cluster 2021-03-10T15:27:44.563630+0000 mgr.gibba032.whopeo (mgr.24625) 5026 : cluster [DBG] pgmap v2857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:46 gibba011 bash[31774]: cephadm 2021-03-10T15:27:44.847515+0000 mgr.gibba032.whopeo (mgr.24625) 5027 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:27:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:48 gibba032 bash[24004]: cluster 2021-03-10T15:27:46.565358+0000 mgr.gibba032.whopeo (mgr.24625) 5028 : cluster [DBG] pgmap v2858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:48 gibba011 bash[30791]: cluster 2021-03-10T15:27:46.565358+0000 mgr.gibba032.whopeo (mgr.24625) 5028 : cluster [DBG] pgmap v2858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:48 gibba011 bash[31774]: cluster 2021-03-10T15:27:46.565358+0000 mgr.gibba032.whopeo (mgr.24625) 5028 : cluster [DBG] pgmap v2858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:50 gibba032 bash[24004]: cluster 2021-03-10T15:27:48.566834+0000 mgr.gibba032.whopeo (mgr.24625) 5029 : cluster [DBG] pgmap v2859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:50 gibba011 bash[30791]: cluster 2021-03-10T15:27:48.566834+0000 mgr.gibba032.whopeo (mgr.24625) 5029 : cluster [DBG] pgmap v2859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:50 gibba011 bash[31774]: cluster 2021-03-10T15:27:48.566834+0000 mgr.gibba032.whopeo (mgr.24625) 5029 : cluster [DBG] pgmap v2859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:52 gibba032 bash[24004]: cluster 2021-03-10T15:27:50.568325+0000 mgr.gibba032.whopeo (mgr.24625) 5030 : cluster [DBG] pgmap v2860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:52 gibba011 bash[30791]: cluster 2021-03-10T15:27:50.568325+0000 mgr.gibba032.whopeo (mgr.24625) 5030 : cluster [DBG] pgmap v2860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:52 gibba011 bash[31774]: cluster 2021-03-10T15:27:50.568325+0000 mgr.gibba032.whopeo (mgr.24625) 5030 : cluster [DBG] pgmap v2860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:54 gibba032 bash[24004]: cluster 2021-03-10T15:27:52.569967+0000 mgr.gibba032.whopeo (mgr.24625) 5031 : cluster [DBG] pgmap v2861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:54 gibba011 bash[30791]: cluster 2021-03-10T15:27:52.569967+0000 mgr.gibba032.whopeo (mgr.24625) 5031 : cluster [DBG] pgmap v2861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:54.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:54 gibba011 bash[31774]: cluster 2021-03-10T15:27:52.569967+0000 mgr.gibba032.whopeo (mgr.24625) 5031 : cluster [DBG] pgmap v2861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:56 gibba032 bash[24004]: cluster 2021-03-10T15:27:54.571445+0000 mgr.gibba032.whopeo (mgr.24625) 5032 : cluster [DBG] pgmap v2862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:56 gibba011 bash[30791]: cluster 2021-03-10T15:27:54.571445+0000 mgr.gibba032.whopeo (mgr.24625) 5032 : cluster [DBG] pgmap v2862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:56 gibba011 bash[31774]: cluster 2021-03-10T15:27:54.571445+0000 mgr.gibba032.whopeo (mgr.24625) 5032 : cluster [DBG] pgmap v2862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.603266+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:27:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.603956+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:27:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.620037+0000 mon.a (mon.0) 1984 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:27:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.620779+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:27:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.621473+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:27:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:57 gibba032 bash[24004]: audit 2021-03-10T15:27:56.622159+0000 mon.a (mon.0) 1987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:27:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.603266+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:27:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.603956+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:27:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.620037+0000 mon.a (mon.0) 1984 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:27:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.620779+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:27:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.621473+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:27:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[30791]: audit 2021-03-10T15:27:56.622159+0000 mon.a (mon.0) 1987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:27:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.603266+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:27:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.603956+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:27:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.620037+0000 mon.a (mon.0) 1984 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:27:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.620779+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:27:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.621473+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:27:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:57 gibba011 bash[31774]: audit 2021-03-10T15:27:56.622159+0000 mon.a (mon.0) 1987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:27:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:27:58 gibba032 bash[24004]: cluster 2021-03-10T15:27:56.573285+0000 mgr.gibba032.whopeo (mgr.24625) 5033 : cluster [DBG] pgmap v2863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:27:58 gibba011 bash[30791]: cluster 2021-03-10T15:27:56.573285+0000 mgr.gibba032.whopeo (mgr.24625) 5033 : cluster [DBG] pgmap v2863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:27:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:27:58 gibba011 bash[31774]: cluster 2021-03-10T15:27:56.573285+0000 mgr.gibba032.whopeo (mgr.24625) 5033 : cluster [DBG] pgmap v2863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: cluster 2021-03-10T15:27:58.574778+0000 mgr.gibba032.whopeo (mgr.24625) 5034 : cluster [DBG] pgmap v2864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.075212+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:28:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.078399+0000 mon.a (mon.0) 1989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:28:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.080016+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.081586+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.083118+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.086315+0000 mon.a (mon.0) 1993 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:00 gibba032 bash[24004]: audit 2021-03-10T15:28:00.088150+0000 mon.a (mon.0) 1994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: cluster 2021-03-10T15:27:58.574778+0000 mgr.gibba032.whopeo (mgr.24625) 5034 : cluster [DBG] pgmap v2864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.075212+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:28:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.078399+0000 mon.a (mon.0) 1989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:28:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.080016+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.081586+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.083118+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.086315+0000 mon.a (mon.0) 1993 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[30791]: audit 2021-03-10T15:28:00.088150+0000 mon.a (mon.0) 1994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: cluster 2021-03-10T15:27:58.574778+0000 mgr.gibba032.whopeo (mgr.24625) 5034 : cluster [DBG] pgmap v2864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:00.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.075212+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:28:00.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.078399+0000 mon.a (mon.0) 1989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:28:00.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.080016+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.081586+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.083118+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.086315+0000 mon.a (mon.0) 1993 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:00.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:00 gibba011 bash[31774]: audit 2021-03-10T15:28:00.088150+0000 mon.a (mon.0) 1994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:27:59.861569+0000 mgr.gibba032.whopeo (mgr.24625) 5035 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:28:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:27:59.863126+0000 mgr.gibba032.whopeo (mgr.24625) 5036 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:28:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.076635+0000 mgr.gibba032.whopeo (mgr.24625) 5037 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:28:01.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.078488+0000 mgr.gibba032.whopeo (mgr.24625) 5038 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:28:01.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.080003+0000 mgr.gibba032.whopeo (mgr.24625) 5039 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:28:01.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.080097+0000 mgr.gibba032.whopeo (mgr.24625) 5040 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:28:01.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.081603+0000 mgr.gibba032.whopeo (mgr.24625) 5041 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:28:01.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.081639+0000 mgr.gibba032.whopeo (mgr.24625) 5042 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:28:01.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.083205+0000 mgr.gibba032.whopeo (mgr.24625) 5043 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:28:01.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.084698+0000 mgr.gibba032.whopeo (mgr.24625) 5044 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:28:01.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.084735+0000 mgr.gibba032.whopeo (mgr.24625) 5045 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:28:01.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.086265+0000 mgr.gibba032.whopeo (mgr.24625) 5046 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:28:01.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.086751+0000 mgr.gibba032.whopeo (mgr.24625) 5047 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:28:01.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.087956+0000 mgr.gibba032.whopeo (mgr.24625) 5048 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:28:01.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.087994+0000 mgr.gibba032.whopeo (mgr.24625) 5049 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:28:01.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:01 gibba032 bash[24004]: cephadm 2021-03-10T15:28:00.088153+0000 mgr.gibba032.whopeo (mgr.24625) 5050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:27:59.861569+0000 mgr.gibba032.whopeo (mgr.24625) 5035 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:28:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:27:59.863126+0000 mgr.gibba032.whopeo (mgr.24625) 5036 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:28:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.076635+0000 mgr.gibba032.whopeo (mgr.24625) 5037 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:28:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.078488+0000 mgr.gibba032.whopeo (mgr.24625) 5038 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:28:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.080003+0000 mgr.gibba032.whopeo (mgr.24625) 5039 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:28:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.080097+0000 mgr.gibba032.whopeo (mgr.24625) 5040 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:28:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.081603+0000 mgr.gibba032.whopeo (mgr.24625) 5041 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:28:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.081639+0000 mgr.gibba032.whopeo (mgr.24625) 5042 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:28:01.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.083205+0000 mgr.gibba032.whopeo (mgr.24625) 5043 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:28:01.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.084698+0000 mgr.gibba032.whopeo (mgr.24625) 5044 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:28:01.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.084735+0000 mgr.gibba032.whopeo (mgr.24625) 5045 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:28:01.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.086265+0000 mgr.gibba032.whopeo (mgr.24625) 5046 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:28:01.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.086751+0000 mgr.gibba032.whopeo (mgr.24625) 5047 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:28:01.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.087956+0000 mgr.gibba032.whopeo (mgr.24625) 5048 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:28:01.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.087994+0000 mgr.gibba032.whopeo (mgr.24625) 5049 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:28:01.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[30791]: cephadm 2021-03-10T15:28:00.088153+0000 mgr.gibba032.whopeo (mgr.24625) 5050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:01.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:27:59.861569+0000 mgr.gibba032.whopeo (mgr.24625) 5035 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:28:01.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:27:59.863126+0000 mgr.gibba032.whopeo (mgr.24625) 5036 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:28:01.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.076635+0000 mgr.gibba032.whopeo (mgr.24625) 5037 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:28:01.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.078488+0000 mgr.gibba032.whopeo (mgr.24625) 5038 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:28:01.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.080003+0000 mgr.gibba032.whopeo (mgr.24625) 5039 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:28:01.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.080097+0000 mgr.gibba032.whopeo (mgr.24625) 5040 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:28:01.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.081603+0000 mgr.gibba032.whopeo (mgr.24625) 5041 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:28:01.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.081639+0000 mgr.gibba032.whopeo (mgr.24625) 5042 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:28:01.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.083205+0000 mgr.gibba032.whopeo (mgr.24625) 5043 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:28:01.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.084698+0000 mgr.gibba032.whopeo (mgr.24625) 5044 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:28:01.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.084735+0000 mgr.gibba032.whopeo (mgr.24625) 5045 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:28:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.086265+0000 mgr.gibba032.whopeo (mgr.24625) 5046 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:28:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.086751+0000 mgr.gibba032.whopeo (mgr.24625) 5047 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:28:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.087956+0000 mgr.gibba032.whopeo (mgr.24625) 5048 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:28:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.087994+0000 mgr.gibba032.whopeo (mgr.24625) 5049 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:28:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:01 gibba011 bash[31774]: cephadm 2021-03-10T15:28:00.088153+0000 mgr.gibba032.whopeo (mgr.24625) 5050 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:02 gibba032 bash[24004]: cluster 2021-03-10T15:28:00.576436+0000 mgr.gibba032.whopeo (mgr.24625) 5051 : cluster [DBG] pgmap v2865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:02 gibba011 bash[30791]: cluster 2021-03-10T15:28:00.576436+0000 mgr.gibba032.whopeo (mgr.24625) 5051 : cluster [DBG] pgmap v2865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:02 gibba011 bash[31774]: cluster 2021-03-10T15:28:00.576436+0000 mgr.gibba032.whopeo (mgr.24625) 5051 : cluster [DBG] pgmap v2865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:04 gibba032 bash[24004]: cluster 2021-03-10T15:28:02.578057+0000 mgr.gibba032.whopeo (mgr.24625) 5052 : cluster [DBG] pgmap v2866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:04 gibba011 bash[30791]: cluster 2021-03-10T15:28:02.578057+0000 mgr.gibba032.whopeo (mgr.24625) 5052 : cluster [DBG] pgmap v2866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:04 gibba011 bash[31774]: cluster 2021-03-10T15:28:02.578057+0000 mgr.gibba032.whopeo (mgr.24625) 5052 : cluster [DBG] pgmap v2866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:06 gibba032 bash[24004]: cluster 2021-03-10T15:28:04.579564+0000 mgr.gibba032.whopeo (mgr.24625) 5053 : cluster [DBG] pgmap v2867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:06 gibba011 bash[30791]: cluster 2021-03-10T15:28:04.579564+0000 mgr.gibba032.whopeo (mgr.24625) 5053 : cluster [DBG] pgmap v2867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:06 gibba011 bash[31774]: cluster 2021-03-10T15:28:04.579564+0000 mgr.gibba032.whopeo (mgr.24625) 5053 : cluster [DBG] pgmap v2867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:08.203 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:28:08.432 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:28:08.432 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (94m) 9m ago 100m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:28:08.433 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (99m) 8m ago 99m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:28:08.433 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (95m) 8m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:28:08.433 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (95m) 8m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:28:08.433 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (94m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:28:08.434 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (94m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:28:08.434 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (93m) 8m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:28:08.434 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (94m) 9m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:28:08.434 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (100m) 9m ago 100m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:28:08.435 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (100m) 8m ago 100m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:28:08.435 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (93m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:28:08.435 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (92m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:28:08.435 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (92m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:28:08.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (91m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:28:08.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:28:08.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (90m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:28:08.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (90m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:28:08.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (90m) 8m ago 100m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:28:08.437 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (99m) 8m ago 100m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:28:08.437 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 98m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:28:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:08 gibba032 bash[24004]: cluster 2021-03-10T15:28:06.581317+0000 mgr.gibba032.whopeo (mgr.24625) 5054 : cluster [DBG] pgmap v2868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:08 gibba011 bash[30791]: cluster 2021-03-10T15:28:06.581317+0000 mgr.gibba032.whopeo (mgr.24625) 5054 : cluster [DBG] pgmap v2868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:08 gibba011 bash[31774]: cluster 2021-03-10T15:28:06.581317+0000 mgr.gibba032.whopeo (mgr.24625) 5054 : cluster [DBG] pgmap v2868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:08.684 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:28:08.685 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:28:08.685 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:28:08.685 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:08.685 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:28:08.686 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:28:08.686 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:08.686 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:28:08.686 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:28:08.687 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:08.687 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:28:08.687 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:28:08.687 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:28:08.688 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:28:08.688 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:28:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:09 gibba032 bash[24004]: audit 2021-03-10T15:28:08.190272+0000 mgr.gibba032.whopeo (mgr.24625) 5055 : audit [DBG] from='client.57424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:09 gibba032 bash[24004]: audit 2021-03-10T15:28:08.416362+0000 mgr.gibba032.whopeo (mgr.24625) 5056 : audit [DBG] from='client.57430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:09 gibba032 bash[24004]: audit 2021-03-10T15:28:08.681778+0000 mon.c (mon.1) 94 : audit [DBG] from='client.? 172.21.2.111:0/1696088997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:09.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[30791]: audit 2021-03-10T15:28:08.190272+0000 mgr.gibba032.whopeo (mgr.24625) 5055 : audit [DBG] from='client.57424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[30791]: audit 2021-03-10T15:28:08.416362+0000 mgr.gibba032.whopeo (mgr.24625) 5056 : audit [DBG] from='client.57430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[30791]: audit 2021-03-10T15:28:08.681778+0000 mon.c (mon.1) 94 : audit [DBG] from='client.? 172.21.2.111:0/1696088997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[31774]: audit 2021-03-10T15:28:08.190272+0000 mgr.gibba032.whopeo (mgr.24625) 5055 : audit [DBG] from='client.57424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[31774]: audit 2021-03-10T15:28:08.416362+0000 mgr.gibba032.whopeo (mgr.24625) 5056 : audit [DBG] from='client.57430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:09.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:09 gibba011 bash[31774]: audit 2021-03-10T15:28:08.681778+0000 mon.c (mon.1) 94 : audit [DBG] from='client.? 172.21.2.111:0/1696088997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:10 gibba032 bash[24004]: cluster 2021-03-10T15:28:08.583307+0000 mgr.gibba032.whopeo (mgr.24625) 5057 : cluster [DBG] pgmap v2869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:10 gibba011 bash[30791]: cluster 2021-03-10T15:28:08.583307+0000 mgr.gibba032.whopeo (mgr.24625) 5057 : cluster [DBG] pgmap v2869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:10 gibba011 bash[31774]: cluster 2021-03-10T15:28:08.583307+0000 mgr.gibba032.whopeo (mgr.24625) 5057 : cluster [DBG] pgmap v2869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:12 gibba032 bash[24004]: cluster 2021-03-10T15:28:10.584995+0000 mgr.gibba032.whopeo (mgr.24625) 5058 : cluster [DBG] pgmap v2870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:12 gibba011 bash[30791]: cluster 2021-03-10T15:28:10.584995+0000 mgr.gibba032.whopeo (mgr.24625) 5058 : cluster [DBG] pgmap v2870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:12 gibba011 bash[31774]: cluster 2021-03-10T15:28:10.584995+0000 mgr.gibba032.whopeo (mgr.24625) 5058 : cluster [DBG] pgmap v2870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:14 gibba032 bash[24004]: cluster 2021-03-10T15:28:12.586749+0000 mgr.gibba032.whopeo (mgr.24625) 5059 : cluster [DBG] pgmap v2871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:14 gibba011 bash[30791]: cluster 2021-03-10T15:28:12.586749+0000 mgr.gibba032.whopeo (mgr.24625) 5059 : cluster [DBG] pgmap v2871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:14 gibba011 bash[31774]: cluster 2021-03-10T15:28:12.586749+0000 mgr.gibba032.whopeo (mgr.24625) 5059 : cluster [DBG] pgmap v2871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:16 gibba032 bash[24004]: cluster 2021-03-10T15:28:14.588284+0000 mgr.gibba032.whopeo (mgr.24625) 5060 : cluster [DBG] pgmap v2872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:16 gibba032 bash[24004]: cephadm 2021-03-10T15:28:15.104210+0000 mgr.gibba032.whopeo (mgr.24625) 5061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:16 gibba011 bash[30791]: cluster 2021-03-10T15:28:14.588284+0000 mgr.gibba032.whopeo (mgr.24625) 5060 : cluster [DBG] pgmap v2872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:16 gibba011 bash[30791]: cephadm 2021-03-10T15:28:15.104210+0000 mgr.gibba032.whopeo (mgr.24625) 5061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:16 gibba011 bash[31774]: cluster 2021-03-10T15:28:14.588284+0000 mgr.gibba032.whopeo (mgr.24625) 5060 : cluster [DBG] pgmap v2872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:16 gibba011 bash[31774]: cephadm 2021-03-10T15:28:15.104210+0000 mgr.gibba032.whopeo (mgr.24625) 5061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:18 gibba011 bash[30791]: cluster 2021-03-10T15:28:16.588780+0000 mgr.gibba032.whopeo (mgr.24625) 5062 : cluster [DBG] pgmap v2873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:18 gibba011 bash[31774]: cluster 2021-03-10T15:28:16.588780+0000 mgr.gibba032.whopeo (mgr.24625) 5062 : cluster [DBG] pgmap v2873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:18 gibba032 bash[24004]: cluster 2021-03-10T15:28:16.588780+0000 mgr.gibba032.whopeo (mgr.24625) 5062 : cluster [DBG] pgmap v2873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:20 gibba011 bash[30791]: cluster 2021-03-10T15:28:18.590061+0000 mgr.gibba032.whopeo (mgr.24625) 5063 : cluster [DBG] pgmap v2874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:20 gibba011 bash[31774]: cluster 2021-03-10T15:28:18.590061+0000 mgr.gibba032.whopeo (mgr.24625) 5063 : cluster [DBG] pgmap v2874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:20 gibba032 bash[24004]: cluster 2021-03-10T15:28:18.590061+0000 mgr.gibba032.whopeo (mgr.24625) 5063 : cluster [DBG] pgmap v2874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:22 gibba011 bash[30791]: cluster 2021-03-10T15:28:20.591627+0000 mgr.gibba032.whopeo (mgr.24625) 5064 : cluster [DBG] pgmap v2875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:22 gibba011 bash[31774]: cluster 2021-03-10T15:28:20.591627+0000 mgr.gibba032.whopeo (mgr.24625) 5064 : cluster [DBG] pgmap v2875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:22 gibba032 bash[24004]: cluster 2021-03-10T15:28:20.591627+0000 mgr.gibba032.whopeo (mgr.24625) 5064 : cluster [DBG] pgmap v2875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:24 gibba011 bash[30791]: cluster 2021-03-10T15:28:22.593377+0000 mgr.gibba032.whopeo (mgr.24625) 5065 : cluster [DBG] pgmap v2876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:24 gibba011 bash[31774]: cluster 2021-03-10T15:28:22.593377+0000 mgr.gibba032.whopeo (mgr.24625) 5065 : cluster [DBG] pgmap v2876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:24 gibba032 bash[24004]: cluster 2021-03-10T15:28:22.593377+0000 mgr.gibba032.whopeo (mgr.24625) 5065 : cluster [DBG] pgmap v2876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:26 gibba011 bash[30791]: cluster 2021-03-10T15:28:24.593904+0000 mgr.gibba032.whopeo (mgr.24625) 5066 : cluster [DBG] pgmap v2877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:26 gibba011 bash[31774]: cluster 2021-03-10T15:28:24.593904+0000 mgr.gibba032.whopeo (mgr.24625) 5066 : cluster [DBG] pgmap v2877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:26 gibba032 bash[24004]: cluster 2021-03-10T15:28:24.593904+0000 mgr.gibba032.whopeo (mgr.24625) 5066 : cluster [DBG] pgmap v2877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:28 gibba011 bash[30791]: cluster 2021-03-10T15:28:26.595404+0000 mgr.gibba032.whopeo (mgr.24625) 5067 : cluster [DBG] pgmap v2878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:28 gibba011 bash[31774]: cluster 2021-03-10T15:28:26.595404+0000 mgr.gibba032.whopeo (mgr.24625) 5067 : cluster [DBG] pgmap v2878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:28 gibba032 bash[24004]: cluster 2021-03-10T15:28:26.595404+0000 mgr.gibba032.whopeo (mgr.24625) 5067 : cluster [DBG] pgmap v2878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:30 gibba032 bash[24004]: cluster 2021-03-10T15:28:28.596927+0000 mgr.gibba032.whopeo (mgr.24625) 5068 : cluster [DBG] pgmap v2879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:30 gibba011 bash[30791]: cluster 2021-03-10T15:28:28.596927+0000 mgr.gibba032.whopeo (mgr.24625) 5068 : cluster [DBG] pgmap v2879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:30 gibba011 bash[31774]: cluster 2021-03-10T15:28:28.596927+0000 mgr.gibba032.whopeo (mgr.24625) 5068 : cluster [DBG] pgmap v2879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:31.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:31 gibba011 bash[30791]: cephadm 2021-03-10T15:28:30.116951+0000 mgr.gibba032.whopeo (mgr.24625) 5069 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:31.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:31 gibba011 bash[31774]: cephadm 2021-03-10T15:28:30.116951+0000 mgr.gibba032.whopeo (mgr.24625) 5069 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:31.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:31 gibba032 bash[24004]: cephadm 2021-03-10T15:28:30.116951+0000 mgr.gibba032.whopeo (mgr.24625) 5069 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:32 gibba032 bash[24004]: cluster 2021-03-10T15:28:30.598271+0000 mgr.gibba032.whopeo (mgr.24625) 5070 : cluster [DBG] pgmap v2880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:32 gibba011 bash[30791]: cluster 2021-03-10T15:28:30.598271+0000 mgr.gibba032.whopeo (mgr.24625) 5070 : cluster [DBG] pgmap v2880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:32 gibba011 bash[31774]: cluster 2021-03-10T15:28:30.598271+0000 mgr.gibba032.whopeo (mgr.24625) 5070 : cluster [DBG] pgmap v2880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:34 gibba032 bash[24004]: cluster 2021-03-10T15:28:32.598817+0000 mgr.gibba032.whopeo (mgr.24625) 5071 : cluster [DBG] pgmap v2881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:34 gibba011 bash[30791]: cluster 2021-03-10T15:28:32.598817+0000 mgr.gibba032.whopeo (mgr.24625) 5071 : cluster [DBG] pgmap v2881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:34.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:34 gibba011 bash[31774]: cluster 2021-03-10T15:28:32.598817+0000 mgr.gibba032.whopeo (mgr.24625) 5071 : cluster [DBG] pgmap v2881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:36 gibba032 bash[24004]: cluster 2021-03-10T15:28:34.600288+0000 mgr.gibba032.whopeo (mgr.24625) 5072 : cluster [DBG] pgmap v2882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:36.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:36 gibba011 bash[30791]: cluster 2021-03-10T15:28:34.600288+0000 mgr.gibba032.whopeo (mgr.24625) 5072 : cluster [DBG] pgmap v2882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:36 gibba011 bash[31774]: cluster 2021-03-10T15:28:34.600288+0000 mgr.gibba032.whopeo (mgr.24625) 5072 : cluster [DBG] pgmap v2882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:38 gibba032 bash[24004]: cluster 2021-03-10T15:28:36.602058+0000 mgr.gibba032.whopeo (mgr.24625) 5073 : cluster [DBG] pgmap v2883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:38 gibba011 bash[30791]: cluster 2021-03-10T15:28:36.602058+0000 mgr.gibba032.whopeo (mgr.24625) 5073 : cluster [DBG] pgmap v2883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:38 gibba011 bash[31774]: cluster 2021-03-10T15:28:36.602058+0000 mgr.gibba032.whopeo (mgr.24625) 5073 : cluster [DBG] pgmap v2883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:38.928 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:28:39.157 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:28:39.158 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (95m) 9m ago 100m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:28:39.158 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (100m) 8m ago 100m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:28:39.158 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (95m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:28:39.159 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (95m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:28:39.159 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (94m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:28:39.159 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (94m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:28:39.159 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (94m) 8m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:28:39.160 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (94m) 9m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:28:39.160 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (100m) 9m ago 100m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:28:39.160 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (100m) 8m ago 100m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:28:39.160 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (93m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:28:39.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (93m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:28:39.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (92m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:28:39.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (92m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:28:39.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (91m) 8m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:28:39.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:28:39.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (91m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:28:39.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (90m) 8m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:28:39.163 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (100m) 8m ago 100m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:28:39.163 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 98m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:28:39.409 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:28:39.409 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:28:39.410 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:28:39.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:39.410 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:28:39.410 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:28:39.411 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:39.411 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:28:39.411 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:28:39.411 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:28:39.411 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:28:39.412 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:28:39.412 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:28:39.412 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:28:39.412 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:28:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:40 gibba032 bash[24004]: cluster 2021-03-10T15:28:38.603571+0000 mgr.gibba032.whopeo (mgr.24625) 5074 : cluster [DBG] pgmap v2884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:40 gibba032 bash[24004]: audit 2021-03-10T15:28:38.915453+0000 mgr.gibba032.whopeo (mgr.24625) 5075 : audit [DBG] from='client.57442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:40 gibba032 bash[24004]: audit 2021-03-10T15:28:39.141308+0000 mgr.gibba032.whopeo (mgr.24625) 5076 : audit [DBG] from='client.57448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:40 gibba032 bash[24004]: audit 2021-03-10T15:28:39.406085+0000 mon.c (mon.1) 95 : audit [DBG] from='client.? 172.21.2.111:0/1515206329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[30791]: cluster 2021-03-10T15:28:38.603571+0000 mgr.gibba032.whopeo (mgr.24625) 5074 : cluster [DBG] pgmap v2884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[30791]: audit 2021-03-10T15:28:38.915453+0000 mgr.gibba032.whopeo (mgr.24625) 5075 : audit [DBG] from='client.57442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[30791]: audit 2021-03-10T15:28:39.141308+0000 mgr.gibba032.whopeo (mgr.24625) 5076 : audit [DBG] from='client.57448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[30791]: audit 2021-03-10T15:28:39.406085+0000 mon.c (mon.1) 95 : audit [DBG] from='client.? 172.21.2.111:0/1515206329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[31774]: cluster 2021-03-10T15:28:38.603571+0000 mgr.gibba032.whopeo (mgr.24625) 5074 : cluster [DBG] pgmap v2884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[31774]: audit 2021-03-10T15:28:38.915453+0000 mgr.gibba032.whopeo (mgr.24625) 5075 : audit [DBG] from='client.57442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[31774]: audit 2021-03-10T15:28:39.141308+0000 mgr.gibba032.whopeo (mgr.24625) 5076 : audit [DBG] from='client.57448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:28:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:40 gibba011 bash[31774]: audit 2021-03-10T15:28:39.406085+0000 mon.c (mon.1) 95 : audit [DBG] from='client.? 172.21.2.111:0/1515206329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:28:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:42 gibba032 bash[24004]: cluster 2021-03-10T15:28:40.604362+0000 mgr.gibba032.whopeo (mgr.24625) 5077 : cluster [DBG] pgmap v2885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:42 gibba011 bash[30791]: cluster 2021-03-10T15:28:40.604362+0000 mgr.gibba032.whopeo (mgr.24625) 5077 : cluster [DBG] pgmap v2885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:42 gibba011 bash[31774]: cluster 2021-03-10T15:28:40.604362+0000 mgr.gibba032.whopeo (mgr.24625) 5077 : cluster [DBG] pgmap v2885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:44 gibba032 bash[24004]: cluster 2021-03-10T15:28:42.605766+0000 mgr.gibba032.whopeo (mgr.24625) 5078 : cluster [DBG] pgmap v2886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:44 gibba011 bash[30791]: cluster 2021-03-10T15:28:42.605766+0000 mgr.gibba032.whopeo (mgr.24625) 5078 : cluster [DBG] pgmap v2886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:44.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:44 gibba011 bash[31774]: cluster 2021-03-10T15:28:42.605766+0000 mgr.gibba032.whopeo (mgr.24625) 5078 : cluster [DBG] pgmap v2886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:46 gibba032 bash[24004]: cluster 2021-03-10T15:28:44.607147+0000 mgr.gibba032.whopeo (mgr.24625) 5079 : cluster [DBG] pgmap v2887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:46 gibba032 bash[24004]: cephadm 2021-03-10T15:28:45.128974+0000 mgr.gibba032.whopeo (mgr.24625) 5080 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:46.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:46 gibba011 bash[30791]: cluster 2021-03-10T15:28:44.607147+0000 mgr.gibba032.whopeo (mgr.24625) 5079 : cluster [DBG] pgmap v2887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:46.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:46 gibba011 bash[30791]: cephadm 2021-03-10T15:28:45.128974+0000 mgr.gibba032.whopeo (mgr.24625) 5080 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:46.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:46 gibba011 bash[31774]: cluster 2021-03-10T15:28:44.607147+0000 mgr.gibba032.whopeo (mgr.24625) 5079 : cluster [DBG] pgmap v2887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:46.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:46 gibba011 bash[31774]: cephadm 2021-03-10T15:28:45.128974+0000 mgr.gibba032.whopeo (mgr.24625) 5080 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:28:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:48 gibba032 bash[24004]: cluster 2021-03-10T15:28:46.608988+0000 mgr.gibba032.whopeo (mgr.24625) 5081 : cluster [DBG] pgmap v2888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:48 gibba011 bash[30791]: cluster 2021-03-10T15:28:46.608988+0000 mgr.gibba032.whopeo (mgr.24625) 5081 : cluster [DBG] pgmap v2888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:48 gibba011 bash[31774]: cluster 2021-03-10T15:28:46.608988+0000 mgr.gibba032.whopeo (mgr.24625) 5081 : cluster [DBG] pgmap v2888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:50 gibba032 bash[24004]: cluster 2021-03-10T15:28:48.610354+0000 mgr.gibba032.whopeo (mgr.24625) 5082 : cluster [DBG] pgmap v2889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:50 gibba011 bash[30791]: cluster 2021-03-10T15:28:48.610354+0000 mgr.gibba032.whopeo (mgr.24625) 5082 : cluster [DBG] pgmap v2889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:50 gibba011 bash[31774]: cluster 2021-03-10T15:28:48.610354+0000 mgr.gibba032.whopeo (mgr.24625) 5082 : cluster [DBG] pgmap v2889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:52 gibba032 bash[24004]: cluster 2021-03-10T15:28:50.611684+0000 mgr.gibba032.whopeo (mgr.24625) 5083 : cluster [DBG] pgmap v2890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:52.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:52 gibba011 bash[30791]: cluster 2021-03-10T15:28:50.611684+0000 mgr.gibba032.whopeo (mgr.24625) 5083 : cluster [DBG] pgmap v2890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:52 gibba011 bash[31774]: cluster 2021-03-10T15:28:50.611684+0000 mgr.gibba032.whopeo (mgr.24625) 5083 : cluster [DBG] pgmap v2890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:54 gibba032 bash[24004]: cluster 2021-03-10T15:28:52.613367+0000 mgr.gibba032.whopeo (mgr.24625) 5084 : cluster [DBG] pgmap v2891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:54.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:54 gibba011 bash[30791]: cluster 2021-03-10T15:28:52.613367+0000 mgr.gibba032.whopeo (mgr.24625) 5084 : cluster [DBG] pgmap v2891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:54 gibba011 bash[31774]: cluster 2021-03-10T15:28:52.613367+0000 mgr.gibba032.whopeo (mgr.24625) 5084 : cluster [DBG] pgmap v2891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:56 gibba032 bash[24004]: cluster 2021-03-10T15:28:54.614826+0000 mgr.gibba032.whopeo (mgr.24625) 5085 : cluster [DBG] pgmap v2892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:56 gibba011 bash[30791]: cluster 2021-03-10T15:28:54.614826+0000 mgr.gibba032.whopeo (mgr.24625) 5085 : cluster [DBG] pgmap v2892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:56.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:56 gibba011 bash[31774]: cluster 2021-03-10T15:28:54.614826+0000 mgr.gibba032.whopeo (mgr.24625) 5085 : cluster [DBG] pgmap v2892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.606471+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:28:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.607130+0000 mon.a (mon.0) 1996 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:28:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.627006+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:28:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.627841+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:28:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.628602+0000 mon.a (mon.0) 1999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:28:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:57 gibba032 bash[24004]: audit 2021-03-10T15:28:56.629349+0000 mon.a (mon.0) 2000 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:28:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.606471+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:28:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.607130+0000 mon.a (mon.0) 1996 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:28:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.627006+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:28:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.627841+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:28:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.628602+0000 mon.a (mon.0) 1999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:28:57.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[30791]: audit 2021-03-10T15:28:56.629349+0000 mon.a (mon.0) 2000 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:28:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.606471+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:28:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.607130+0000 mon.a (mon.0) 1996 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:28:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.627006+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T15:28:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.627841+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:28:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.628602+0000 mon.a (mon.0) 1999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:28:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:57 gibba011 bash[31774]: audit 2021-03-10T15:28:56.629349+0000 mon.a (mon.0) 2000 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:28:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:58 gibba032 bash[24004]: cluster 2021-03-10T15:28:56.615413+0000 mgr.gibba032.whopeo (mgr.24625) 5086 : cluster [DBG] pgmap v2893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:58 gibba011 bash[30791]: cluster 2021-03-10T15:28:56.615413+0000 mgr.gibba032.whopeo (mgr.24625) 5086 : cluster [DBG] pgmap v2893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:58 gibba011 bash[31774]: cluster 2021-03-10T15:28:56.615413+0000 mgr.gibba032.whopeo (mgr.24625) 5086 : cluster [DBG] pgmap v2893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:28:59 gibba032 bash[24004]: cluster 2021-03-10T15:28:58.616702+0000 mgr.gibba032.whopeo (mgr.24625) 5087 : cluster [DBG] pgmap v2894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:59.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:28:59 gibba011 bash[30791]: cluster 2021-03-10T15:28:58.616702+0000 mgr.gibba032.whopeo (mgr.24625) 5087 : cluster [DBG] pgmap v2894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:28:59.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:28:59 gibba011 bash[31774]: cluster 2021-03-10T15:28:58.616702+0000 mgr.gibba032.whopeo (mgr.24625) 5087 : cluster [DBG] pgmap v2894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:00 gibba032 bash[24004]: cephadm 2021-03-10T15:29:00.133169+0000 mgr.gibba032.whopeo (mgr.24625) 5088 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:29:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:00 gibba032 bash[24004]: audit 2021-03-10T15:29:00.134627+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:29:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:00 gibba011 bash[30791]: cephadm 2021-03-10T15:29:00.133169+0000 mgr.gibba032.whopeo (mgr.24625) 5088 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:29:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:00 gibba011 bash[30791]: audit 2021-03-10T15:29:00.134627+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:29:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:00 gibba011 bash[31774]: cephadm 2021-03-10T15:29:00.133169+0000 mgr.gibba032.whopeo (mgr.24625) 5088 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:29:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:00 gibba011 bash[31774]: audit 2021-03-10T15:29:00.134627+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:29:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:01 gibba032 bash[24004]: cluster 2021-03-10T15:29:00.618062+0000 mgr.gibba032.whopeo (mgr.24625) 5089 : cluster [DBG] pgmap v2895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:01 gibba011 bash[30791]: cluster 2021-03-10T15:29:00.618062+0000 mgr.gibba032.whopeo (mgr.24625) 5089 : cluster [DBG] pgmap v2895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:01 gibba011 bash[31774]: cluster 2021-03-10T15:29:00.618062+0000 mgr.gibba032.whopeo (mgr.24625) 5089 : cluster [DBG] pgmap v2895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:03 gibba032 bash[24004]: cluster 2021-03-10T15:29:02.619786+0000 mgr.gibba032.whopeo (mgr.24625) 5090 : cluster [DBG] pgmap v2896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:04.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:03 gibba032 bash[24004]: audit 2021-03-10T15:29:03.694623+0000 mon.a (mon.0) 2002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:04.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:03 gibba011 bash[30791]: cluster 2021-03-10T15:29:02.619786+0000 mgr.gibba032.whopeo (mgr.24625) 5090 : cluster [DBG] pgmap v2896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:04.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:03 gibba011 bash[30791]: audit 2021-03-10T15:29:03.694623+0000 mon.a (mon.0) 2002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:04.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:03 gibba011 bash[31774]: cluster 2021-03-10T15:29:02.619786+0000 mgr.gibba032.whopeo (mgr.24625) 5090 : cluster [DBG] pgmap v2896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:04.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:03 gibba011 bash[31774]: audit 2021-03-10T15:29:03.694623+0000 mon.a (mon.0) 2002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:05.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.694162+0000 mgr.gibba032.whopeo (mgr.24625) 5091 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:29:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.890523+0000 mon.a (mon.0) 2003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.896384+0000 mgr.gibba032.whopeo (mgr.24625) 5092 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:29:05.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.898249+0000 mon.a (mon.0) 2004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:29:05.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.898620+0000 mgr.gibba032.whopeo (mgr.24625) 5093 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:29:05.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.900247+0000 mon.a (mon.0) 2005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.900573+0000 mgr.gibba032.whopeo (mgr.24625) 5094 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:29:05.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.900632+0000 mgr.gibba032.whopeo (mgr.24625) 5095 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:29:05.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.902306+0000 mon.a (mon.0) 2006 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.902338+0000 mgr.gibba032.whopeo (mgr.24625) 5096 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:29:05.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.902398+0000 mgr.gibba032.whopeo (mgr.24625) 5097 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:29:05.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.903654+0000 mgr.gibba032.whopeo (mgr.24625) 5098 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:29:05.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.903696+0000 mon.a (mon.0) 2007 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.905178+0000 mgr.gibba032.whopeo (mgr.24625) 5099 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:29:05.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.905242+0000 mgr.gibba032.whopeo (mgr.24625) 5100 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:29:05.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.906629+0000 mgr.gibba032.whopeo (mgr.24625) 5101 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:29:05.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.906823+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.906995+0000 mgr.gibba032.whopeo (mgr.24625) 5102 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:29:05.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: audit 2021-03-10T15:29:03.908418+0000 mon.a (mon.0) 2009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.908540+0000 mgr.gibba032.whopeo (mgr.24625) 5103 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:29:05.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.908608+0000 mgr.gibba032.whopeo (mgr.24625) 5104 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:29:05.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:04 gibba032 bash[24004]: cephadm 2021-03-10T15:29:03.908761+0000 mgr.gibba032.whopeo (mgr.24625) 5105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:05.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.694162+0000 mgr.gibba032.whopeo (mgr.24625) 5091 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:29:05.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.890523+0000 mon.a (mon.0) 2003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:05.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.896384+0000 mgr.gibba032.whopeo (mgr.24625) 5092 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:29:05.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.898249+0000 mon.a (mon.0) 2004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:29:05.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.898620+0000 mgr.gibba032.whopeo (mgr.24625) 5093 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:29:05.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.900247+0000 mon.a (mon.0) 2005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.900573+0000 mgr.gibba032.whopeo (mgr.24625) 5094 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:29:05.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.900632+0000 mgr.gibba032.whopeo (mgr.24625) 5095 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:29:05.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.902306+0000 mon.a (mon.0) 2006 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.902338+0000 mgr.gibba032.whopeo (mgr.24625) 5096 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:29:05.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.902398+0000 mgr.gibba032.whopeo (mgr.24625) 5097 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:29:05.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.903654+0000 mgr.gibba032.whopeo (mgr.24625) 5098 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:29:05.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.903696+0000 mon.a (mon.0) 2007 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.905178+0000 mgr.gibba032.whopeo (mgr.24625) 5099 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:29:05.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.905242+0000 mgr.gibba032.whopeo (mgr.24625) 5100 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:29:05.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.906629+0000 mgr.gibba032.whopeo (mgr.24625) 5101 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:29:05.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.906823+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.906995+0000 mgr.gibba032.whopeo (mgr.24625) 5102 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:29:05.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: audit 2021-03-10T15:29:03.908418+0000 mon.a (mon.0) 2009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.908540+0000 mgr.gibba032.whopeo (mgr.24625) 5103 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:29:05.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.908608+0000 mgr.gibba032.whopeo (mgr.24625) 5104 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:29:05.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[30791]: cephadm 2021-03-10T15:29:03.908761+0000 mgr.gibba032.whopeo (mgr.24625) 5105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:05.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.694162+0000 mgr.gibba032.whopeo (mgr.24625) 5091 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:29:05.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.890523+0000 mon.a (mon.0) 2003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:29:05.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.896384+0000 mgr.gibba032.whopeo (mgr.24625) 5092 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:29:05.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.898249+0000 mon.a (mon.0) 2004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:29:05.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.898620+0000 mgr.gibba032.whopeo (mgr.24625) 5093 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:29:05.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.900247+0000 mon.a (mon.0) 2005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.900573+0000 mgr.gibba032.whopeo (mgr.24625) 5094 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:29:05.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.900632+0000 mgr.gibba032.whopeo (mgr.24625) 5095 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:29:05.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.902306+0000 mon.a (mon.0) 2006 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.902338+0000 mgr.gibba032.whopeo (mgr.24625) 5096 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:29:05.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.902398+0000 mgr.gibba032.whopeo (mgr.24625) 5097 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:29:05.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.903654+0000 mgr.gibba032.whopeo (mgr.24625) 5098 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:29:05.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.903696+0000 mon.a (mon.0) 2007 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.905178+0000 mgr.gibba032.whopeo (mgr.24625) 5099 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:29:05.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.905242+0000 mgr.gibba032.whopeo (mgr.24625) 5100 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:29:05.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.906629+0000 mgr.gibba032.whopeo (mgr.24625) 5101 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:29:05.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.906823+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.906995+0000 mgr.gibba032.whopeo (mgr.24625) 5102 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:29:05.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: audit 2021-03-10T15:29:03.908418+0000 mon.a (mon.0) 2009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:05.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.908540+0000 mgr.gibba032.whopeo (mgr.24625) 5103 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:29:05.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.908608+0000 mgr.gibba032.whopeo (mgr.24625) 5104 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:29:05.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:04 gibba011 bash[31774]: cephadm 2021-03-10T15:29:03.908761+0000 mgr.gibba032.whopeo (mgr.24625) 5105 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:06.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:05 gibba032 bash[24004]: cluster 2021-03-10T15:29:04.620656+0000 mgr.gibba032.whopeo (mgr.24625) 5106 : cluster [DBG] pgmap v2897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:06.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:05 gibba011 bash[30791]: cluster 2021-03-10T15:29:04.620656+0000 mgr.gibba032.whopeo (mgr.24625) 5106 : cluster [DBG] pgmap v2897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:06.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:05 gibba011 bash[31774]: cluster 2021-03-10T15:29:04.620656+0000 mgr.gibba032.whopeo (mgr.24625) 5106 : cluster [DBG] pgmap v2897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:08.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:07 gibba032 bash[24004]: cluster 2021-03-10T15:29:06.622288+0000 mgr.gibba032.whopeo (mgr.24625) 5107 : cluster [DBG] pgmap v2898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:07 gibba011 bash[30791]: cluster 2021-03-10T15:29:06.622288+0000 mgr.gibba032.whopeo (mgr.24625) 5107 : cluster [DBG] pgmap v2898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:07 gibba011 bash[31774]: cluster 2021-03-10T15:29:06.622288+0000 mgr.gibba032.whopeo (mgr.24625) 5107 : cluster [DBG] pgmap v2898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:09.655 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:29:09.885 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:29:09.885 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (95m) 6s ago 101m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:29:09.885 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (101m) 9m ago 101m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:29:09.886 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (96m) 9m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:29:09.886 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (96m) 9m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:29:09.886 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (95m) 6s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:29:09.886 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (95m) 6s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:29:09.887 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (94m) 9m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:29:09.887 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (95m) 6s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:29:09.887 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (101m) 6s ago 101m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:29:09.887 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (101m) 9m ago 101m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:29:09.887 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (94m) 6s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:29:09.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (93m) 6s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:29:09.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (93m) 6s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:29:09.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (92m) 6s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:29:09.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (92m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:29:09.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (91m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:29:09.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (91m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:29:09.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (91m) 9m ago 101m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:29:09.889 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (100m) 9m ago 101m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:29:09.889 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6s ago 99m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:29:10.139 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:29:10.139 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:29:10.140 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:29:10.140 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:10.140 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:29:10.141 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:29:10.141 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:10.141 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:29:10.141 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:29:10.142 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:10.142 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:29:10.143 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:29:10.143 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:29:10.143 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:29:10.144 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:29:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:09 gibba032 bash[24004]: cluster 2021-03-10T15:29:08.623881+0000 mgr.gibba032.whopeo (mgr.24625) 5108 : cluster [DBG] pgmap v2899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:09 gibba011 bash[30791]: cluster 2021-03-10T15:29:08.623881+0000 mgr.gibba032.whopeo (mgr.24625) 5108 : cluster [DBG] pgmap v2899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:09 gibba011 bash[31774]: cluster 2021-03-10T15:29:08.623881+0000 mgr.gibba032.whopeo (mgr.24625) 5108 : cluster [DBG] pgmap v2899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:11.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:10 gibba032 bash[24004]: audit 2021-03-10T15:29:09.641882+0000 mgr.gibba032.whopeo (mgr.24625) 5109 : audit [DBG] from='client.47475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:10 gibba032 bash[24004]: audit 2021-03-10T15:29:09.868929+0000 mgr.gibba032.whopeo (mgr.24625) 5110 : audit [DBG] from='client.47481 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:10 gibba032 bash[24004]: audit 2021-03-10T15:29:10.135445+0000 mon.a (mon.0) 2010 : audit [DBG] from='client.? 172.21.2.111:0/521880725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[30791]: audit 2021-03-10T15:29:09.641882+0000 mgr.gibba032.whopeo (mgr.24625) 5109 : audit [DBG] from='client.47475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[30791]: audit 2021-03-10T15:29:09.868929+0000 mgr.gibba032.whopeo (mgr.24625) 5110 : audit [DBG] from='client.47481 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[30791]: audit 2021-03-10T15:29:10.135445+0000 mon.a (mon.0) 2010 : audit [DBG] from='client.? 172.21.2.111:0/521880725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[31774]: audit 2021-03-10T15:29:09.641882+0000 mgr.gibba032.whopeo (mgr.24625) 5109 : audit [DBG] from='client.47475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[31774]: audit 2021-03-10T15:29:09.868929+0000 mgr.gibba032.whopeo (mgr.24625) 5110 : audit [DBG] from='client.47481 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:10 gibba011 bash[31774]: audit 2021-03-10T15:29:10.135445+0000 mon.a (mon.0) 2010 : audit [DBG] from='client.? 172.21.2.111:0/521880725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:11 gibba032 bash[24004]: cluster 2021-03-10T15:29:10.625289+0000 mgr.gibba032.whopeo (mgr.24625) 5111 : cluster [DBG] pgmap v2900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:11 gibba011 bash[30791]: cluster 2021-03-10T15:29:10.625289+0000 mgr.gibba032.whopeo (mgr.24625) 5111 : cluster [DBG] pgmap v2900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:12.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:11 gibba011 bash[31774]: cluster 2021-03-10T15:29:10.625289+0000 mgr.gibba032.whopeo (mgr.24625) 5111 : cluster [DBG] pgmap v2900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:13 gibba032 bash[24004]: cluster 2021-03-10T15:29:12.626024+0000 mgr.gibba032.whopeo (mgr.24625) 5112 : cluster [DBG] pgmap v2901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:13 gibba011 bash[31774]: cluster 2021-03-10T15:29:12.626024+0000 mgr.gibba032.whopeo (mgr.24625) 5112 : cluster [DBG] pgmap v2901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:13 gibba011 bash[30791]: cluster 2021-03-10T15:29:12.626024+0000 mgr.gibba032.whopeo (mgr.24625) 5112 : cluster [DBG] pgmap v2901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:15 gibba032 bash[24004]: cluster 2021-03-10T15:29:14.627177+0000 mgr.gibba032.whopeo (mgr.24625) 5113 : cluster [DBG] pgmap v2902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:15 gibba011 bash[30791]: cluster 2021-03-10T15:29:14.627177+0000 mgr.gibba032.whopeo (mgr.24625) 5113 : cluster [DBG] pgmap v2902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:16.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:15 gibba011 bash[31774]: cluster 2021-03-10T15:29:14.627177+0000 mgr.gibba032.whopeo (mgr.24625) 5113 : cluster [DBG] pgmap v2902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:17 gibba032 bash[24004]: cluster 2021-03-10T15:29:16.628949+0000 mgr.gibba032.whopeo (mgr.24625) 5114 : cluster [DBG] pgmap v2903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:17 gibba011 bash[30791]: cluster 2021-03-10T15:29:16.628949+0000 mgr.gibba032.whopeo (mgr.24625) 5114 : cluster [DBG] pgmap v2903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:17 gibba011 bash[31774]: cluster 2021-03-10T15:29:16.628949+0000 mgr.gibba032.whopeo (mgr.24625) 5114 : cluster [DBG] pgmap v2903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:20.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:19 gibba032 bash[24004]: cluster 2021-03-10T15:29:18.630381+0000 mgr.gibba032.whopeo (mgr.24625) 5115 : cluster [DBG] pgmap v2904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:19 gibba032 bash[24004]: cephadm 2021-03-10T15:29:18.920985+0000 mgr.gibba032.whopeo (mgr.24625) 5116 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:19 gibba011 bash[30791]: cluster 2021-03-10T15:29:18.630381+0000 mgr.gibba032.whopeo (mgr.24625) 5115 : cluster [DBG] pgmap v2904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:20.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:19 gibba011 bash[30791]: cephadm 2021-03-10T15:29:18.920985+0000 mgr.gibba032.whopeo (mgr.24625) 5116 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:20.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:19 gibba011 bash[31774]: cluster 2021-03-10T15:29:18.630381+0000 mgr.gibba032.whopeo (mgr.24625) 5115 : cluster [DBG] pgmap v2904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:20.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:19 gibba011 bash[31774]: cephadm 2021-03-10T15:29:18.920985+0000 mgr.gibba032.whopeo (mgr.24625) 5116 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:21 gibba032 bash[24004]: cluster 2021-03-10T15:29:20.630942+0000 mgr.gibba032.whopeo (mgr.24625) 5117 : cluster [DBG] pgmap v2905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:21 gibba011 bash[30791]: cluster 2021-03-10T15:29:20.630942+0000 mgr.gibba032.whopeo (mgr.24625) 5117 : cluster [DBG] pgmap v2905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:21 gibba011 bash[31774]: cluster 2021-03-10T15:29:20.630942+0000 mgr.gibba032.whopeo (mgr.24625) 5117 : cluster [DBG] pgmap v2905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:24 gibba032 bash[24004]: cluster 2021-03-10T15:29:22.632639+0000 mgr.gibba032.whopeo (mgr.24625) 5118 : cluster [DBG] pgmap v2906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:24 gibba011 bash[30791]: cluster 2021-03-10T15:29:22.632639+0000 mgr.gibba032.whopeo (mgr.24625) 5118 : cluster [DBG] pgmap v2906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:24 gibba011 bash[31774]: cluster 2021-03-10T15:29:22.632639+0000 mgr.gibba032.whopeo (mgr.24625) 5118 : cluster [DBG] pgmap v2906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:26 gibba011 bash[30791]: cluster 2021-03-10T15:29:24.634104+0000 mgr.gibba032.whopeo (mgr.24625) 5119 : cluster [DBG] pgmap v2907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:26 gibba011 bash[31774]: cluster 2021-03-10T15:29:24.634104+0000 mgr.gibba032.whopeo (mgr.24625) 5119 : cluster [DBG] pgmap v2907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:26 gibba032 bash[24004]: cluster 2021-03-10T15:29:24.634104+0000 mgr.gibba032.whopeo (mgr.24625) 5119 : cluster [DBG] pgmap v2907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:28 gibba011 bash[30791]: cluster 2021-03-10T15:29:26.635842+0000 mgr.gibba032.whopeo (mgr.24625) 5120 : cluster [DBG] pgmap v2908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:28 gibba011 bash[31774]: cluster 2021-03-10T15:29:26.635842+0000 mgr.gibba032.whopeo (mgr.24625) 5120 : cluster [DBG] pgmap v2908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:28 gibba032 bash[24004]: cluster 2021-03-10T15:29:26.635842+0000 mgr.gibba032.whopeo (mgr.24625) 5120 : cluster [DBG] pgmap v2908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:30 gibba011 bash[30791]: cluster 2021-03-10T15:29:28.637179+0000 mgr.gibba032.whopeo (mgr.24625) 5121 : cluster [DBG] pgmap v2909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:30 gibba011 bash[31774]: cluster 2021-03-10T15:29:28.637179+0000 mgr.gibba032.whopeo (mgr.24625) 5121 : cluster [DBG] pgmap v2909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:30 gibba032 bash[24004]: cluster 2021-03-10T15:29:28.637179+0000 mgr.gibba032.whopeo (mgr.24625) 5121 : cluster [DBG] pgmap v2909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:32 gibba011 bash[30791]: cluster 2021-03-10T15:29:30.638611+0000 mgr.gibba032.whopeo (mgr.24625) 5122 : cluster [DBG] pgmap v2910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:32 gibba011 bash[31774]: cluster 2021-03-10T15:29:30.638611+0000 mgr.gibba032.whopeo (mgr.24625) 5122 : cluster [DBG] pgmap v2910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:32 gibba032 bash[24004]: cluster 2021-03-10T15:29:30.638611+0000 mgr.gibba032.whopeo (mgr.24625) 5122 : cluster [DBG] pgmap v2910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:34 gibba032 bash[24004]: cluster 2021-03-10T15:29:32.640307+0000 mgr.gibba032.whopeo (mgr.24625) 5123 : cluster [DBG] pgmap v2911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:34 gibba011 bash[30791]: cluster 2021-03-10T15:29:32.640307+0000 mgr.gibba032.whopeo (mgr.24625) 5123 : cluster [DBG] pgmap v2911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:34 gibba011 bash[31774]: cluster 2021-03-10T15:29:32.640307+0000 mgr.gibba032.whopeo (mgr.24625) 5123 : cluster [DBG] pgmap v2911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:35.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:35 gibba011 bash[30791]: cephadm 2021-03-10T15:29:33.937095+0000 mgr.gibba032.whopeo (mgr.24625) 5124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:35.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:35 gibba011 bash[31774]: cephadm 2021-03-10T15:29:33.937095+0000 mgr.gibba032.whopeo (mgr.24625) 5124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:35 gibba032 bash[24004]: cephadm 2021-03-10T15:29:33.937095+0000 mgr.gibba032.whopeo (mgr.24625) 5124 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:36 gibba011 bash[30791]: cluster 2021-03-10T15:29:34.641630+0000 mgr.gibba032.whopeo (mgr.24625) 5125 : cluster [DBG] pgmap v2912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:36.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:36 gibba011 bash[31774]: cluster 2021-03-10T15:29:34.641630+0000 mgr.gibba032.whopeo (mgr.24625) 5125 : cluster [DBG] pgmap v2912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:36 gibba032 bash[24004]: cluster 2021-03-10T15:29:34.641630+0000 mgr.gibba032.whopeo (mgr.24625) 5125 : cluster [DBG] pgmap v2912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:38 gibba011 bash[30791]: cluster 2021-03-10T15:29:36.642675+0000 mgr.gibba032.whopeo (mgr.24625) 5126 : cluster [DBG] pgmap v2913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:38 gibba011 bash[31774]: cluster 2021-03-10T15:29:36.642675+0000 mgr.gibba032.whopeo (mgr.24625) 5126 : cluster [DBG] pgmap v2913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:38 gibba032 bash[24004]: cluster 2021-03-10T15:29:36.642675+0000 mgr.gibba032.whopeo (mgr.24625) 5126 : cluster [DBG] pgmap v2913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:40 gibba011 bash[30791]: cluster 2021-03-10T15:29:38.643926+0000 mgr.gibba032.whopeo (mgr.24625) 5127 : cluster [DBG] pgmap v2914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:40 gibba011 bash[31774]: cluster 2021-03-10T15:29:38.643926+0000 mgr.gibba032.whopeo (mgr.24625) 5127 : cluster [DBG] pgmap v2914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:40.385 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:29:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:40 gibba032 bash[24004]: cluster 2021-03-10T15:29:38.643926+0000 mgr.gibba032.whopeo (mgr.24625) 5127 : cluster [DBG] pgmap v2914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:40.613 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:29:40.613 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (96m) 36s ago 101m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:29:40.613 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (101m) 9m ago 101m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:29:40.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (96m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:29:40.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (96m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:29:40.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (96m) 36s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:29:40.614 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (95m) 36s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:29:40.615 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (95m) 9m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:29:40.615 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (95m) 36s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:29:40.615 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (101m) 36s ago 101m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:29:40.615 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (101m) 9m ago 101m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:29:40.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (94m) 36s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:29:40.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (94m) 36s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:29:40.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (93m) 36s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:29:40.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (93m) 36s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:29:40.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (92m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:29:40.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (92m) 9m ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:29:40.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (92m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:29:40.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (91m) 9m ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:29:40.617 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (101m) 9m ago 101m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:29:40.617 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 36s ago 99m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:29:40.862 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:29:40.862 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:29:40.862 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:29:40.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:40.863 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:29:40.863 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:29:40.863 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:40.863 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:29:40.864 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:29:40.864 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:29:40.864 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:29:40.864 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:29:40.864 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:29:40.865 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:29:40.865 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:29:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:41 gibba011 bash[30791]: audit 2021-03-10T15:29:40.372272+0000 mgr.gibba032.whopeo (mgr.24625) 5128 : audit [DBG] from='client.57478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:41 gibba011 bash[30791]: audit 2021-03-10T15:29:40.858808+0000 mon.c (mon.1) 96 : audit [DBG] from='client.? 172.21.2.111:0/157867372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:41 gibba011 bash[31774]: audit 2021-03-10T15:29:40.372272+0000 mgr.gibba032.whopeo (mgr.24625) 5128 : audit [DBG] from='client.57478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:41 gibba011 bash[31774]: audit 2021-03-10T15:29:40.858808+0000 mon.c (mon.1) 96 : audit [DBG] from='client.? 172.21.2.111:0/157867372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:41 gibba032 bash[24004]: audit 2021-03-10T15:29:40.372272+0000 mgr.gibba032.whopeo (mgr.24625) 5128 : audit [DBG] from='client.57478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:41 gibba032 bash[24004]: audit 2021-03-10T15:29:40.858808+0000 mon.c (mon.1) 96 : audit [DBG] from='client.? 172.21.2.111:0/157867372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:29:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:42 gibba032 bash[24004]: audit 2021-03-10T15:29:40.596940+0000 mgr.gibba032.whopeo (mgr.24625) 5129 : audit [DBG] from='client.47499 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:42 gibba032 bash[24004]: cluster 2021-03-10T15:29:40.644636+0000 mgr.gibba032.whopeo (mgr.24625) 5130 : cluster [DBG] pgmap v2915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:42 gibba011 bash[30791]: audit 2021-03-10T15:29:40.596940+0000 mgr.gibba032.whopeo (mgr.24625) 5129 : audit [DBG] from='client.47499 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:42 gibba011 bash[30791]: cluster 2021-03-10T15:29:40.644636+0000 mgr.gibba032.whopeo (mgr.24625) 5130 : cluster [DBG] pgmap v2915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:42 gibba011 bash[31774]: audit 2021-03-10T15:29:40.596940+0000 mgr.gibba032.whopeo (mgr.24625) 5129 : audit [DBG] from='client.47499 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:29:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:42 gibba011 bash[31774]: cluster 2021-03-10T15:29:40.644636+0000 mgr.gibba032.whopeo (mgr.24625) 5130 : cluster [DBG] pgmap v2915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:44 gibba032 bash[24004]: cluster 2021-03-10T15:29:42.646138+0000 mgr.gibba032.whopeo (mgr.24625) 5131 : cluster [DBG] pgmap v2916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:44 gibba011 bash[30791]: cluster 2021-03-10T15:29:42.646138+0000 mgr.gibba032.whopeo (mgr.24625) 5131 : cluster [DBG] pgmap v2916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:44 gibba011 bash[31774]: cluster 2021-03-10T15:29:42.646138+0000 mgr.gibba032.whopeo (mgr.24625) 5131 : cluster [DBG] pgmap v2916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:46 gibba032 bash[24004]: cluster 2021-03-10T15:29:44.646992+0000 mgr.gibba032.whopeo (mgr.24625) 5132 : cluster [DBG] pgmap v2917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:46 gibba011 bash[30791]: cluster 2021-03-10T15:29:44.646992+0000 mgr.gibba032.whopeo (mgr.24625) 5132 : cluster [DBG] pgmap v2917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:46.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:46 gibba011 bash[31774]: cluster 2021-03-10T15:29:44.646992+0000 mgr.gibba032.whopeo (mgr.24625) 5132 : cluster [DBG] pgmap v2917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:48 gibba032 bash[24004]: cluster 2021-03-10T15:29:46.648558+0000 mgr.gibba032.whopeo (mgr.24625) 5133 : cluster [DBG] pgmap v2918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:48 gibba011 bash[30791]: cluster 2021-03-10T15:29:46.648558+0000 mgr.gibba032.whopeo (mgr.24625) 5133 : cluster [DBG] pgmap v2918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:48 gibba011 bash[31774]: cluster 2021-03-10T15:29:46.648558+0000 mgr.gibba032.whopeo (mgr.24625) 5133 : cluster [DBG] pgmap v2918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:50 gibba032 bash[24004]: cluster 2021-03-10T15:29:48.649934+0000 mgr.gibba032.whopeo (mgr.24625) 5134 : cluster [DBG] pgmap v2919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:50 gibba032 bash[24004]: cephadm 2021-03-10T15:29:48.948840+0000 mgr.gibba032.whopeo (mgr.24625) 5135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:50 gibba011 bash[30791]: cluster 2021-03-10T15:29:48.649934+0000 mgr.gibba032.whopeo (mgr.24625) 5134 : cluster [DBG] pgmap v2919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:50 gibba011 bash[30791]: cephadm 2021-03-10T15:29:48.948840+0000 mgr.gibba032.whopeo (mgr.24625) 5135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:50 gibba011 bash[31774]: cluster 2021-03-10T15:29:48.649934+0000 mgr.gibba032.whopeo (mgr.24625) 5134 : cluster [DBG] pgmap v2919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:50 gibba011 bash[31774]: cephadm 2021-03-10T15:29:48.948840+0000 mgr.gibba032.whopeo (mgr.24625) 5135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:29:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:52 gibba032 bash[24004]: cluster 2021-03-10T15:29:50.651465+0000 mgr.gibba032.whopeo (mgr.24625) 5136 : cluster [DBG] pgmap v2920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:52 gibba011 bash[30791]: cluster 2021-03-10T15:29:50.651465+0000 mgr.gibba032.whopeo (mgr.24625) 5136 : cluster [DBG] pgmap v2920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:52 gibba011 bash[31774]: cluster 2021-03-10T15:29:50.651465+0000 mgr.gibba032.whopeo (mgr.24625) 5136 : cluster [DBG] pgmap v2920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:54 gibba032 bash[24004]: cluster 2021-03-10T15:29:52.652679+0000 mgr.gibba032.whopeo (mgr.24625) 5137 : cluster [DBG] pgmap v2921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:54 gibba011 bash[30791]: cluster 2021-03-10T15:29:52.652679+0000 mgr.gibba032.whopeo (mgr.24625) 5137 : cluster [DBG] pgmap v2921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:54 gibba011 bash[31774]: cluster 2021-03-10T15:29:52.652679+0000 mgr.gibba032.whopeo (mgr.24625) 5137 : cluster [DBG] pgmap v2921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:56 gibba032 bash[24004]: cluster 2021-03-10T15:29:54.654112+0000 mgr.gibba032.whopeo (mgr.24625) 5138 : cluster [DBG] pgmap v2922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:56 gibba011 bash[30791]: cluster 2021-03-10T15:29:54.654112+0000 mgr.gibba032.whopeo (mgr.24625) 5138 : cluster [DBG] pgmap v2922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:56 gibba011 bash[31774]: cluster 2021-03-10T15:29:54.654112+0000 mgr.gibba032.whopeo (mgr.24625) 5138 : cluster [DBG] pgmap v2922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.613865+0000 mon.a (mon.0) 2011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:29:57.529 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.614508+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:29:57.529 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.633452+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:29:57.529 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.634213+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:29:57.529 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.634885+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:29:57.530 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:57 gibba032 bash[24004]: audit 2021-03-10T15:29:56.635596+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:29:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.613865+0000 mon.a (mon.0) 2011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:29:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.614508+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:29:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.633452+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:29:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.634213+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:29:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.634885+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:29:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[30791]: audit 2021-03-10T15:29:56.635596+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:29:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.613865+0000 mon.a (mon.0) 2011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:29:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.614508+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:29:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.633452+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:29:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.634213+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:29:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.634885+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:29:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:57 gibba011 bash[31774]: audit 2021-03-10T15:29:56.635596+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:29:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:29:58 gibba032 bash[24004]: cluster 2021-03-10T15:29:56.654804+0000 mgr.gibba032.whopeo (mgr.24625) 5139 : cluster [DBG] pgmap v2923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:29:58 gibba011 bash[30791]: cluster 2021-03-10T15:29:56.654804+0000 mgr.gibba032.whopeo (mgr.24625) 5139 : cluster [DBG] pgmap v2923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:29:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:29:58 gibba011 bash[31774]: cluster 2021-03-10T15:29:56.654804+0000 mgr.gibba032.whopeo (mgr.24625) 5139 : cluster [DBG] pgmap v2923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:00 gibba011 bash[30791]: debug 2021-03-10T15:29:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:30:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:00 gibba011 bash[30791]: cluster 2021-03-10T15:29:58.656137+0000 mgr.gibba032.whopeo (mgr.24625) 5140 : cluster [DBG] pgmap v2924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:00 gibba011 bash[30791]: cluster 2021-03-10T15:30:00.000481+0000 mon.a (mon.0) 2017 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:30:00.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:00 gibba011 bash[31774]: cluster 2021-03-10T15:29:58.656137+0000 mgr.gibba032.whopeo (mgr.24625) 5140 : cluster [DBG] pgmap v2924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:00.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:00 gibba011 bash[31774]: cluster 2021-03-10T15:30:00.000481+0000 mon.a (mon.0) 2017 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:30:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:00 gibba032 bash[24004]: cluster 2021-03-10T15:29:58.656137+0000 mgr.gibba032.whopeo (mgr.24625) 5140 : cluster [DBG] pgmap v2924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:00 gibba032 bash[24004]: cluster 2021-03-10T15:30:00.000481+0000 mon.a (mon.0) 2017 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:30:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:02 gibba032 bash[24004]: cluster 2021-03-10T15:30:00.657321+0000 mgr.gibba032.whopeo (mgr.24625) 5141 : cluster [DBG] pgmap v2925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:02 gibba011 bash[30791]: cluster 2021-03-10T15:30:00.657321+0000 mgr.gibba032.whopeo (mgr.24625) 5141 : cluster [DBG] pgmap v2925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:02 gibba011 bash[31774]: cluster 2021-03-10T15:30:00.657321+0000 mgr.gibba032.whopeo (mgr.24625) 5141 : cluster [DBG] pgmap v2925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:04 gibba032 bash[24004]: cluster 2021-03-10T15:30:02.659069+0000 mgr.gibba032.whopeo (mgr.24625) 5142 : cluster [DBG] pgmap v2926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:04 gibba032 bash[24004]: audit 2021-03-10T15:30:03.955497+0000 mon.a (mon.0) 2018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:30:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:04 gibba011 bash[30791]: cluster 2021-03-10T15:30:02.659069+0000 mgr.gibba032.whopeo (mgr.24625) 5142 : cluster [DBG] pgmap v2926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:04 gibba011 bash[30791]: audit 2021-03-10T15:30:03.955497+0000 mon.a (mon.0) 2018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:30:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:04 gibba011 bash[31774]: cluster 2021-03-10T15:30:02.659069+0000 mgr.gibba032.whopeo (mgr.24625) 5142 : cluster [DBG] pgmap v2926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:04 gibba011 bash[31774]: audit 2021-03-10T15:30:03.955497+0000 mon.a (mon.0) 2018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:30:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:05 gibba032 bash[24004]: cephadm 2021-03-10T15:30:03.952972+0000 mgr.gibba032.whopeo (mgr.24625) 5143 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:30:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:05 gibba011 bash[30791]: cephadm 2021-03-10T15:30:03.952972+0000 mgr.gibba032.whopeo (mgr.24625) 5143 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:30:05.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:05 gibba011 bash[31774]: cephadm 2021-03-10T15:30:03.952972+0000 mgr.gibba032.whopeo (mgr.24625) 5143 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:30:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:06 gibba032 bash[24004]: cluster 2021-03-10T15:30:04.660554+0000 mgr.gibba032.whopeo (mgr.24625) 5144 : cluster [DBG] pgmap v2927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:06 gibba011 bash[30791]: cluster 2021-03-10T15:30:04.660554+0000 mgr.gibba032.whopeo (mgr.24625) 5144 : cluster [DBG] pgmap v2927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:06 gibba011 bash[31774]: cluster 2021-03-10T15:30:04.660554+0000 mgr.gibba032.whopeo (mgr.24625) 5144 : cluster [DBG] pgmap v2927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: cluster 2021-03-10T15:30:06.662087+0000 mgr.gibba032.whopeo (mgr.24625) 5145 : cluster [DBG] pgmap v2928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.695457+0000 mon.a (mon.0) 2019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.889482+0000 mon.a (mon.0) 2020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.893079+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:30:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.894924+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.896590+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.898256+0000 mon.a (mon.0) 2024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.905910+0000 mon.a (mon.0) 2025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:08 gibba032 bash[24004]: audit 2021-03-10T15:30:07.911469+0000 mon.a (mon.0) 2026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: cluster 2021-03-10T15:30:06.662087+0000 mgr.gibba032.whopeo (mgr.24625) 5145 : cluster [DBG] pgmap v2928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.695457+0000 mon.a (mon.0) 2019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.889482+0000 mon.a (mon.0) 2020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.893079+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:30:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.894924+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.896590+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.898256+0000 mon.a (mon.0) 2024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.905910+0000 mon.a (mon.0) 2025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[30791]: audit 2021-03-10T15:30:07.911469+0000 mon.a (mon.0) 2026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: cluster 2021-03-10T15:30:06.662087+0000 mgr.gibba032.whopeo (mgr.24625) 5145 : cluster [DBG] pgmap v2928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.695457+0000 mon.a (mon.0) 2019 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.889482+0000 mon.a (mon.0) 2020 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:30:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.893079+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:30:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.894924+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.896590+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.898256+0000 mon.a (mon.0) 2024 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.905910+0000 mon.a (mon.0) 2025 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:08.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:08 gibba011 bash[31774]: audit 2021-03-10T15:30:07.911469+0000 mon.a (mon.0) 2026 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.695130+0000 mgr.gibba032.whopeo (mgr.24625) 5146 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:30:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.891081+0000 mgr.gibba032.whopeo (mgr.24625) 5147 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:30:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.893565+0000 mgr.gibba032.whopeo (mgr.24625) 5148 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:30:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.894997+0000 mgr.gibba032.whopeo (mgr.24625) 5149 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:30:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.895035+0000 mgr.gibba032.whopeo (mgr.24625) 5150 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:30:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.896726+0000 mgr.gibba032.whopeo (mgr.24625) 5151 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:30:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.896762+0000 mgr.gibba032.whopeo (mgr.24625) 5152 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:30:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.899360+0000 mgr.gibba032.whopeo (mgr.24625) 5153 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:30:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.902809+0000 mgr.gibba032.whopeo (mgr.24625) 5154 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:30:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.903277+0000 mgr.gibba032.whopeo (mgr.24625) 5155 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:30:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.907285+0000 mgr.gibba032.whopeo (mgr.24625) 5156 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:30:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.909409+0000 mgr.gibba032.whopeo (mgr.24625) 5157 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:30:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.912958+0000 mgr.gibba032.whopeo (mgr.24625) 5158 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:30:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.913318+0000 mgr.gibba032.whopeo (mgr.24625) 5159 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:30:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:09 gibba032 bash[24004]: cephadm 2021-03-10T15:30:07.914104+0000 mgr.gibba032.whopeo (mgr.24625) 5160 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.695130+0000 mgr.gibba032.whopeo (mgr.24625) 5146 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:30:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.891081+0000 mgr.gibba032.whopeo (mgr.24625) 5147 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:30:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.893565+0000 mgr.gibba032.whopeo (mgr.24625) 5148 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:30:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.894997+0000 mgr.gibba032.whopeo (mgr.24625) 5149 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:30:09.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.895035+0000 mgr.gibba032.whopeo (mgr.24625) 5150 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:30:09.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.896726+0000 mgr.gibba032.whopeo (mgr.24625) 5151 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:30:09.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.896762+0000 mgr.gibba032.whopeo (mgr.24625) 5152 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:30:09.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.899360+0000 mgr.gibba032.whopeo (mgr.24625) 5153 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:30:09.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.902809+0000 mgr.gibba032.whopeo (mgr.24625) 5154 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:30:09.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.903277+0000 mgr.gibba032.whopeo (mgr.24625) 5155 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:30:09.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.907285+0000 mgr.gibba032.whopeo (mgr.24625) 5156 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:30:09.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.909409+0000 mgr.gibba032.whopeo (mgr.24625) 5157 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:30:09.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.912958+0000 mgr.gibba032.whopeo (mgr.24625) 5158 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:30:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.913318+0000 mgr.gibba032.whopeo (mgr.24625) 5159 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:30:09.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[31774]: cephadm 2021-03-10T15:30:07.914104+0000 mgr.gibba032.whopeo (mgr.24625) 5160 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.695130+0000 mgr.gibba032.whopeo (mgr.24625) 5146 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:30:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.891081+0000 mgr.gibba032.whopeo (mgr.24625) 5147 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:30:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.893565+0000 mgr.gibba032.whopeo (mgr.24625) 5148 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:30:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.894997+0000 mgr.gibba032.whopeo (mgr.24625) 5149 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:30:09.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.895035+0000 mgr.gibba032.whopeo (mgr.24625) 5150 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:30:09.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.896726+0000 mgr.gibba032.whopeo (mgr.24625) 5151 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:30:09.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.896762+0000 mgr.gibba032.whopeo (mgr.24625) 5152 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:30:09.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.899360+0000 mgr.gibba032.whopeo (mgr.24625) 5153 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:30:09.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.902809+0000 mgr.gibba032.whopeo (mgr.24625) 5154 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:30:09.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.903277+0000 mgr.gibba032.whopeo (mgr.24625) 5155 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:30:09.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.907285+0000 mgr.gibba032.whopeo (mgr.24625) 5156 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:30:09.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.909409+0000 mgr.gibba032.whopeo (mgr.24625) 5157 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:30:09.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.912958+0000 mgr.gibba032.whopeo (mgr.24625) 5158 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:30:09.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.913318+0000 mgr.gibba032.whopeo (mgr.24625) 5159 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:30:09.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:09 gibba011 bash[30791]: cephadm 2021-03-10T15:30:07.914104+0000 mgr.gibba032.whopeo (mgr.24625) 5160 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:10 gibba032 bash[24004]: cluster 2021-03-10T15:30:08.663088+0000 mgr.gibba032.whopeo (mgr.24625) 5161 : cluster [DBG] pgmap v2929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:10 gibba011 bash[30791]: cluster 2021-03-10T15:30:08.663088+0000 mgr.gibba032.whopeo (mgr.24625) 5161 : cluster [DBG] pgmap v2929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:10 gibba011 bash[31774]: cluster 2021-03-10T15:30:08.663088+0000 mgr.gibba032.whopeo (mgr.24625) 5161 : cluster [DBG] pgmap v2929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:11.106 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:30:11.363 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:30:11.364 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (96m) 67s ago 102m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:30:11.364 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (102m) 3s ago 102m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:30:11.364 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (97m) 3s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:30:11.364 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (97m) 3s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:30:11.365 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (96m) 67s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:30:11.365 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (96m) 67s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:30:11.365 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (95m) 3s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:30:11.365 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (96m) 67s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:30:11.366 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (102m) 67s ago 102m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:30:11.366 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (102m) 3s ago 102m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:30:11.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (95m) 67s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:30:11.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (94m) 67s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:30:11.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (94m) 67s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:30:11.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (93m) 67s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:30:11.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (93m) 3s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:30:11.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (93m) 3s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:30:11.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (92m) 3s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:30:11.368 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (92m) 3s ago 102m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:30:11.368 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (101m) 3s ago 102m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:30:11.368 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 67s ago 100m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:30:11.614 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:30:11.614 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:30:11.614 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:30:11.615 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:11.615 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:30:11.615 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:30:11.615 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:11.615 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:30:11.616 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:30:11.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:11.616 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:30:11.616 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:30:11.617 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:30:11.617 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:30:11.617 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:30:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:12 gibba032 bash[24004]: cluster 2021-03-10T15:30:10.664396+0000 mgr.gibba032.whopeo (mgr.24625) 5162 : cluster [DBG] pgmap v2930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:12 gibba032 bash[24004]: audit 2021-03-10T15:30:11.093493+0000 mgr.gibba032.whopeo (mgr.24625) 5163 : audit [DBG] from='client.57496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:12 gibba032 bash[24004]: audit 2021-03-10T15:30:11.318874+0000 mgr.gibba032.whopeo (mgr.24625) 5164 : audit [DBG] from='client.57502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:12 gibba032 bash[24004]: audit 2021-03-10T15:30:11.610831+0000 mon.c (mon.1) 97 : audit [DBG] from='client.? 172.21.2.111:0/276573222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[30791]: cluster 2021-03-10T15:30:10.664396+0000 mgr.gibba032.whopeo (mgr.24625) 5162 : cluster [DBG] pgmap v2930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:12.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[30791]: audit 2021-03-10T15:30:11.093493+0000 mgr.gibba032.whopeo (mgr.24625) 5163 : audit [DBG] from='client.57496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[30791]: audit 2021-03-10T15:30:11.318874+0000 mgr.gibba032.whopeo (mgr.24625) 5164 : audit [DBG] from='client.57502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[30791]: audit 2021-03-10T15:30:11.610831+0000 mon.c (mon.1) 97 : audit [DBG] from='client.? 172.21.2.111:0/276573222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:12.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[31774]: cluster 2021-03-10T15:30:10.664396+0000 mgr.gibba032.whopeo (mgr.24625) 5162 : cluster [DBG] pgmap v2930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:12.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[31774]: audit 2021-03-10T15:30:11.093493+0000 mgr.gibba032.whopeo (mgr.24625) 5163 : audit [DBG] from='client.57496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[31774]: audit 2021-03-10T15:30:11.318874+0000 mgr.gibba032.whopeo (mgr.24625) 5164 : audit [DBG] from='client.57502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:12.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:12 gibba011 bash[31774]: audit 2021-03-10T15:30:11.610831+0000 mon.c (mon.1) 97 : audit [DBG] from='client.? 172.21.2.111:0/276573222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:14 gibba032 bash[24004]: cluster 2021-03-10T15:30:12.666180+0000 mgr.gibba032.whopeo (mgr.24625) 5165 : cluster [DBG] pgmap v2931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:14 gibba011 bash[30791]: cluster 2021-03-10T15:30:12.666180+0000 mgr.gibba032.whopeo (mgr.24625) 5165 : cluster [DBG] pgmap v2931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:14 gibba011 bash[31774]: cluster 2021-03-10T15:30:12.666180+0000 mgr.gibba032.whopeo (mgr.24625) 5165 : cluster [DBG] pgmap v2931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:16 gibba032 bash[24004]: cluster 2021-03-10T15:30:14.667570+0000 mgr.gibba032.whopeo (mgr.24625) 5166 : cluster [DBG] pgmap v2932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:16 gibba011 bash[30791]: cluster 2021-03-10T15:30:14.667570+0000 mgr.gibba032.whopeo (mgr.24625) 5166 : cluster [DBG] pgmap v2932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:16 gibba011 bash[31774]: cluster 2021-03-10T15:30:14.667570+0000 mgr.gibba032.whopeo (mgr.24625) 5166 : cluster [DBG] pgmap v2932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:18 gibba011 bash[30791]: cluster 2021-03-10T15:30:16.669337+0000 mgr.gibba032.whopeo (mgr.24625) 5167 : cluster [DBG] pgmap v2933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:18 gibba011 bash[31774]: cluster 2021-03-10T15:30:16.669337+0000 mgr.gibba032.whopeo (mgr.24625) 5167 : cluster [DBG] pgmap v2933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:18 gibba032 bash[24004]: cluster 2021-03-10T15:30:16.669337+0000 mgr.gibba032.whopeo (mgr.24625) 5167 : cluster [DBG] pgmap v2933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:20 gibba011 bash[30791]: cluster 2021-03-10T15:30:18.670901+0000 mgr.gibba032.whopeo (mgr.24625) 5168 : cluster [DBG] pgmap v2934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:20 gibba011 bash[31774]: cluster 2021-03-10T15:30:18.670901+0000 mgr.gibba032.whopeo (mgr.24625) 5168 : cluster [DBG] pgmap v2934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:20 gibba032 bash[24004]: cluster 2021-03-10T15:30:18.670901+0000 mgr.gibba032.whopeo (mgr.24625) 5168 : cluster [DBG] pgmap v2934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:22 gibba011 bash[30791]: cluster 2021-03-10T15:30:20.671563+0000 mgr.gibba032.whopeo (mgr.24625) 5169 : cluster [DBG] pgmap v2935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:22 gibba011 bash[31774]: cluster 2021-03-10T15:30:20.671563+0000 mgr.gibba032.whopeo (mgr.24625) 5169 : cluster [DBG] pgmap v2935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:22 gibba032 bash[24004]: cluster 2021-03-10T15:30:20.671563+0000 mgr.gibba032.whopeo (mgr.24625) 5169 : cluster [DBG] pgmap v2935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:24 gibba011 bash[30791]: cluster 2021-03-10T15:30:22.673257+0000 mgr.gibba032.whopeo (mgr.24625) 5170 : cluster [DBG] pgmap v2936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:24 gibba011 bash[30791]: cephadm 2021-03-10T15:30:22.930124+0000 mgr.gibba032.whopeo (mgr.24625) 5171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:24 gibba011 bash[31774]: cluster 2021-03-10T15:30:22.673257+0000 mgr.gibba032.whopeo (mgr.24625) 5170 : cluster [DBG] pgmap v2936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:24 gibba011 bash[31774]: cephadm 2021-03-10T15:30:22.930124+0000 mgr.gibba032.whopeo (mgr.24625) 5171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:24 gibba032 bash[24004]: cluster 2021-03-10T15:30:22.673257+0000 mgr.gibba032.whopeo (mgr.24625) 5170 : cluster [DBG] pgmap v2936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:24 gibba032 bash[24004]: cephadm 2021-03-10T15:30:22.930124+0000 mgr.gibba032.whopeo (mgr.24625) 5171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:26 gibba011 bash[30791]: cluster 2021-03-10T15:30:24.674648+0000 mgr.gibba032.whopeo (mgr.24625) 5172 : cluster [DBG] pgmap v2937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:26 gibba011 bash[31774]: cluster 2021-03-10T15:30:24.674648+0000 mgr.gibba032.whopeo (mgr.24625) 5172 : cluster [DBG] pgmap v2937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:26 gibba032 bash[24004]: cluster 2021-03-10T15:30:24.674648+0000 mgr.gibba032.whopeo (mgr.24625) 5172 : cluster [DBG] pgmap v2937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:28 gibba011 bash[30791]: cluster 2021-03-10T15:30:26.676323+0000 mgr.gibba032.whopeo (mgr.24625) 5173 : cluster [DBG] pgmap v2938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:28 gibba011 bash[31774]: cluster 2021-03-10T15:30:26.676323+0000 mgr.gibba032.whopeo (mgr.24625) 5173 : cluster [DBG] pgmap v2938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:28 gibba032 bash[24004]: cluster 2021-03-10T15:30:26.676323+0000 mgr.gibba032.whopeo (mgr.24625) 5173 : cluster [DBG] pgmap v2938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:30 gibba011 bash[30791]: cluster 2021-03-10T15:30:28.677901+0000 mgr.gibba032.whopeo (mgr.24625) 5174 : cluster [DBG] pgmap v2939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:30 gibba011 bash[31774]: cluster 2021-03-10T15:30:28.677901+0000 mgr.gibba032.whopeo (mgr.24625) 5174 : cluster [DBG] pgmap v2939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:30 gibba032 bash[24004]: cluster 2021-03-10T15:30:28.677901+0000 mgr.gibba032.whopeo (mgr.24625) 5174 : cluster [DBG] pgmap v2939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:32 gibba011 bash[30791]: cluster 2021-03-10T15:30:30.679246+0000 mgr.gibba032.whopeo (mgr.24625) 5175 : cluster [DBG] pgmap v2940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:32 gibba011 bash[31774]: cluster 2021-03-10T15:30:30.679246+0000 mgr.gibba032.whopeo (mgr.24625) 5175 : cluster [DBG] pgmap v2940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:32 gibba032 bash[24004]: cluster 2021-03-10T15:30:30.679246+0000 mgr.gibba032.whopeo (mgr.24625) 5175 : cluster [DBG] pgmap v2940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:34 gibba032 bash[24004]: cluster 2021-03-10T15:30:32.681041+0000 mgr.gibba032.whopeo (mgr.24625) 5176 : cluster [DBG] pgmap v2941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:34 gibba011 bash[30791]: cluster 2021-03-10T15:30:32.681041+0000 mgr.gibba032.whopeo (mgr.24625) 5176 : cluster [DBG] pgmap v2941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:34.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:34 gibba011 bash[31774]: cluster 2021-03-10T15:30:32.681041+0000 mgr.gibba032.whopeo (mgr.24625) 5176 : cluster [DBG] pgmap v2941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:36 gibba032 bash[24004]: cluster 2021-03-10T15:30:34.682327+0000 mgr.gibba032.whopeo (mgr.24625) 5177 : cluster [DBG] pgmap v2942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:36.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:36 gibba011 bash[30791]: cluster 2021-03-10T15:30:34.682327+0000 mgr.gibba032.whopeo (mgr.24625) 5177 : cluster [DBG] pgmap v2942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:36 gibba011 bash[31774]: cluster 2021-03-10T15:30:34.682327+0000 mgr.gibba032.whopeo (mgr.24625) 5177 : cluster [DBG] pgmap v2942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:38 gibba032 bash[24004]: cluster 2021-03-10T15:30:36.684114+0000 mgr.gibba032.whopeo (mgr.24625) 5178 : cluster [DBG] pgmap v2943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:38 gibba011 bash[30791]: cluster 2021-03-10T15:30:36.684114+0000 mgr.gibba032.whopeo (mgr.24625) 5178 : cluster [DBG] pgmap v2943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:38 gibba011 bash[31774]: cluster 2021-03-10T15:30:36.684114+0000 mgr.gibba032.whopeo (mgr.24625) 5178 : cluster [DBG] pgmap v2943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:39.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:39 gibba032 bash[24004]: cephadm 2021-03-10T15:30:37.946161+0000 mgr.gibba032.whopeo (mgr.24625) 5179 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:39.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:39 gibba011 bash[30791]: cephadm 2021-03-10T15:30:37.946161+0000 mgr.gibba032.whopeo (mgr.24625) 5179 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:39 gibba011 bash[31774]: cephadm 2021-03-10T15:30:37.946161+0000 mgr.gibba032.whopeo (mgr.24625) 5179 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:40 gibba032 bash[24004]: cluster 2021-03-10T15:30:38.685701+0000 mgr.gibba032.whopeo (mgr.24625) 5180 : cluster [DBG] pgmap v2944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:40 gibba011 bash[30791]: cluster 2021-03-10T15:30:38.685701+0000 mgr.gibba032.whopeo (mgr.24625) 5180 : cluster [DBG] pgmap v2944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:40 gibba011 bash[31774]: cluster 2021-03-10T15:30:38.685701+0000 mgr.gibba032.whopeo (mgr.24625) 5180 : cluster [DBG] pgmap v2944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:41.861 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:30:42.090 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:30:42.090 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (97m) 98s ago 102m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:30:42.090 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (102m) 34s ago 102m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:30:42.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (97m) 34s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:30:42.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (97m) 34s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:30:42.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (97m) 98s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:30:42.091 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (96m) 98s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:30:42.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (96m) 34s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:30:42.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (96m) 98s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:30:42.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (102m) 98s ago 102m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:30:42.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (102m) 34s ago 102m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:30:42.092 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (95m) 98s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:30:42.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (95m) 98s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:30:42.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (94m) 98s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:30:42.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (94m) 98s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:30:42.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (93m) 34s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:30:42.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (93m) 34s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:30:42.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (93m) 34s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:30:42.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (92m) 34s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:30:42.094 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (102m) 34s ago 102m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:30:42.095 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 98s ago 100m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:30:42.342 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:30:42.342 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:30:42.342 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:30:42.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:42.343 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:30:42.343 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:30:42.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:42.344 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:30:42.344 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:30:42.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:30:42.344 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:30:42.345 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:30:42.345 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:30:42.345 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:30:42.345 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:30:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:42 gibba032 bash[24004]: cluster 2021-03-10T15:30:40.686368+0000 mgr.gibba032.whopeo (mgr.24625) 5181 : cluster [DBG] pgmap v2945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:42 gibba032 bash[24004]: audit 2021-03-10T15:30:42.337724+0000 mon.a (mon.0) 2027 : audit [DBG] from='client.? 172.21.2.111:0/1128343197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:42 gibba011 bash[30791]: cluster 2021-03-10T15:30:40.686368+0000 mgr.gibba032.whopeo (mgr.24625) 5181 : cluster [DBG] pgmap v2945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:42 gibba011 bash[30791]: audit 2021-03-10T15:30:42.337724+0000 mon.a (mon.0) 2027 : audit [DBG] from='client.? 172.21.2.111:0/1128343197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:42 gibba011 bash[31774]: cluster 2021-03-10T15:30:40.686368+0000 mgr.gibba032.whopeo (mgr.24625) 5181 : cluster [DBG] pgmap v2945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:42.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:42 gibba011 bash[31774]: audit 2021-03-10T15:30:42.337724+0000 mon.a (mon.0) 2027 : audit [DBG] from='client.? 172.21.2.111:0/1128343197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:30:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:43 gibba032 bash[24004]: audit 2021-03-10T15:30:41.847119+0000 mgr.gibba032.whopeo (mgr.24625) 5182 : audit [DBG] from='client.47529 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:43 gibba032 bash[24004]: audit 2021-03-10T15:30:42.073376+0000 mgr.gibba032.whopeo (mgr.24625) 5183 : audit [DBG] from='client.57520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:43 gibba011 bash[30791]: audit 2021-03-10T15:30:41.847119+0000 mgr.gibba032.whopeo (mgr.24625) 5182 : audit [DBG] from='client.47529 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:43 gibba011 bash[30791]: audit 2021-03-10T15:30:42.073376+0000 mgr.gibba032.whopeo (mgr.24625) 5183 : audit [DBG] from='client.57520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:43 gibba011 bash[31774]: audit 2021-03-10T15:30:41.847119+0000 mgr.gibba032.whopeo (mgr.24625) 5182 : audit [DBG] from='client.47529 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:43.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:43 gibba011 bash[31774]: audit 2021-03-10T15:30:42.073376+0000 mgr.gibba032.whopeo (mgr.24625) 5183 : audit [DBG] from='client.57520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:30:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:44 gibba032 bash[24004]: cluster 2021-03-10T15:30:42.687995+0000 mgr.gibba032.whopeo (mgr.24625) 5184 : cluster [DBG] pgmap v2946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:44 gibba011 bash[30791]: cluster 2021-03-10T15:30:42.687995+0000 mgr.gibba032.whopeo (mgr.24625) 5184 : cluster [DBG] pgmap v2946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:44 gibba011 bash[31774]: cluster 2021-03-10T15:30:42.687995+0000 mgr.gibba032.whopeo (mgr.24625) 5184 : cluster [DBG] pgmap v2946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:46 gibba032 bash[24004]: cluster 2021-03-10T15:30:44.689385+0000 mgr.gibba032.whopeo (mgr.24625) 5185 : cluster [DBG] pgmap v2947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:46.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:46 gibba011 bash[30791]: cluster 2021-03-10T15:30:44.689385+0000 mgr.gibba032.whopeo (mgr.24625) 5185 : cluster [DBG] pgmap v2947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:46.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:46 gibba011 bash[31774]: cluster 2021-03-10T15:30:44.689385+0000 mgr.gibba032.whopeo (mgr.24625) 5185 : cluster [DBG] pgmap v2947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:48 gibba032 bash[24004]: cluster 2021-03-10T15:30:46.691121+0000 mgr.gibba032.whopeo (mgr.24625) 5186 : cluster [DBG] pgmap v2948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:48 gibba011 bash[30791]: cluster 2021-03-10T15:30:46.691121+0000 mgr.gibba032.whopeo (mgr.24625) 5186 : cluster [DBG] pgmap v2948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:48 gibba011 bash[31774]: cluster 2021-03-10T15:30:46.691121+0000 mgr.gibba032.whopeo (mgr.24625) 5186 : cluster [DBG] pgmap v2948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:50 gibba032 bash[24004]: cluster 2021-03-10T15:30:48.692506+0000 mgr.gibba032.whopeo (mgr.24625) 5187 : cluster [DBG] pgmap v2949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:50 gibba011 bash[30791]: cluster 2021-03-10T15:30:48.692506+0000 mgr.gibba032.whopeo (mgr.24625) 5187 : cluster [DBG] pgmap v2949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:50.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:50 gibba011 bash[31774]: cluster 2021-03-10T15:30:48.692506+0000 mgr.gibba032.whopeo (mgr.24625) 5187 : cluster [DBG] pgmap v2949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:52 gibba032 bash[24004]: cluster 2021-03-10T15:30:50.693850+0000 mgr.gibba032.whopeo (mgr.24625) 5188 : cluster [DBG] pgmap v2950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:52.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:52 gibba011 bash[30791]: cluster 2021-03-10T15:30:50.693850+0000 mgr.gibba032.whopeo (mgr.24625) 5188 : cluster [DBG] pgmap v2950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:52 gibba011 bash[31774]: cluster 2021-03-10T15:30:50.693850+0000 mgr.gibba032.whopeo (mgr.24625) 5188 : cluster [DBG] pgmap v2950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:54 gibba032 bash[24004]: cluster 2021-03-10T15:30:52.695601+0000 mgr.gibba032.whopeo (mgr.24625) 5189 : cluster [DBG] pgmap v2951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:54 gibba032 bash[24004]: cephadm 2021-03-10T15:30:52.962264+0000 mgr.gibba032.whopeo (mgr.24625) 5190 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:54.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:54 gibba011 bash[30791]: cluster 2021-03-10T15:30:52.695601+0000 mgr.gibba032.whopeo (mgr.24625) 5189 : cluster [DBG] pgmap v2951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:54.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:54 gibba011 bash[30791]: cephadm 2021-03-10T15:30:52.962264+0000 mgr.gibba032.whopeo (mgr.24625) 5190 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:54.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:54 gibba011 bash[31774]: cluster 2021-03-10T15:30:52.695601+0000 mgr.gibba032.whopeo (mgr.24625) 5189 : cluster [DBG] pgmap v2951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:54.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:54 gibba011 bash[31774]: cephadm 2021-03-10T15:30:52.962264+0000 mgr.gibba032.whopeo (mgr.24625) 5190 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:30:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:56 gibba032 bash[24004]: cluster 2021-03-10T15:30:54.697142+0000 mgr.gibba032.whopeo (mgr.24625) 5191 : cluster [DBG] pgmap v2952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:56.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:56 gibba011 bash[30791]: cluster 2021-03-10T15:30:54.697142+0000 mgr.gibba032.whopeo (mgr.24625) 5191 : cluster [DBG] pgmap v2952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:56 gibba011 bash[31774]: cluster 2021-03-10T15:30:54.697142+0000 mgr.gibba032.whopeo (mgr.24625) 5191 : cluster [DBG] pgmap v2952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.621125+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:30:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.621295+0000 mon.a (mon.0) 2029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:30:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.638922+0000 mon.a (mon.0) 2030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:30:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.639170+0000 mon.a (mon.0) 2031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:30:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.639414+0000 mon.a (mon.0) 2032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:30:57.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:57 gibba032 bash[24004]: audit 2021-03-10T15:30:56.639656+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:30:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.621125+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:30:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.621295+0000 mon.a (mon.0) 2029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:30:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.638922+0000 mon.a (mon.0) 2030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:30:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.639170+0000 mon.a (mon.0) 2031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:30:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.639414+0000 mon.a (mon.0) 2032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:30:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[30791]: audit 2021-03-10T15:30:56.639656+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:30:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.621125+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:30:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.621295+0000 mon.a (mon.0) 2029 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:30:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.638922+0000 mon.a (mon.0) 2030 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T15:30:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.639170+0000 mon.a (mon.0) 2031 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:30:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.639414+0000 mon.a (mon.0) 2032 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:30:57.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:57 gibba011 bash[31774]: audit 2021-03-10T15:30:56.639656+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:30:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:30:58 gibba032 bash[24004]: cluster 2021-03-10T15:30:56.698938+0000 mgr.gibba032.whopeo (mgr.24625) 5192 : cluster [DBG] pgmap v2953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:30:58 gibba011 bash[30791]: cluster 2021-03-10T15:30:56.698938+0000 mgr.gibba032.whopeo (mgr.24625) 5192 : cluster [DBG] pgmap v2953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:30:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:30:58 gibba011 bash[31774]: cluster 2021-03-10T15:30:56.698938+0000 mgr.gibba032.whopeo (mgr.24625) 5192 : cluster [DBG] pgmap v2953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:00 gibba032 bash[24004]: cluster 2021-03-10T15:30:58.700441+0000 mgr.gibba032.whopeo (mgr.24625) 5193 : cluster [DBG] pgmap v2954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:00 gibba011 bash[30791]: cluster 2021-03-10T15:30:58.700441+0000 mgr.gibba032.whopeo (mgr.24625) 5193 : cluster [DBG] pgmap v2954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:00.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:00 gibba011 bash[31774]: cluster 2021-03-10T15:30:58.700441+0000 mgr.gibba032.whopeo (mgr.24625) 5193 : cluster [DBG] pgmap v2954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:02 gibba032 bash[24004]: cluster 2021-03-10T15:31:00.701188+0000 mgr.gibba032.whopeo (mgr.24625) 5194 : cluster [DBG] pgmap v2955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:02 gibba011 bash[30791]: cluster 2021-03-10T15:31:00.701188+0000 mgr.gibba032.whopeo (mgr.24625) 5194 : cluster [DBG] pgmap v2955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:02 gibba011 bash[31774]: cluster 2021-03-10T15:31:00.701188+0000 mgr.gibba032.whopeo (mgr.24625) 5194 : cluster [DBG] pgmap v2955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:04 gibba032 bash[24004]: cluster 2021-03-10T15:31:02.702862+0000 mgr.gibba032.whopeo (mgr.24625) 5195 : cluster [DBG] pgmap v2956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:04 gibba011 bash[30791]: cluster 2021-03-10T15:31:02.702862+0000 mgr.gibba032.whopeo (mgr.24625) 5195 : cluster [DBG] pgmap v2956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:04 gibba011 bash[31774]: cluster 2021-03-10T15:31:02.702862+0000 mgr.gibba032.whopeo (mgr.24625) 5195 : cluster [DBG] pgmap v2956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:06 gibba032 bash[24004]: cluster 2021-03-10T15:31:04.704378+0000 mgr.gibba032.whopeo (mgr.24625) 5196 : cluster [DBG] pgmap v2957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:06.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:06 gibba011 bash[30791]: cluster 2021-03-10T15:31:04.704378+0000 mgr.gibba032.whopeo (mgr.24625) 5196 : cluster [DBG] pgmap v2957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:06 gibba011 bash[31774]: cluster 2021-03-10T15:31:04.704378+0000 mgr.gibba032.whopeo (mgr.24625) 5196 : cluster [DBG] pgmap v2957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:07 gibba011 bash[30791]: cluster 2021-03-10T15:31:06.706105+0000 mgr.gibba032.whopeo (mgr.24625) 5197 : cluster [DBG] pgmap v2958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:07.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:07 gibba011 bash[31774]: cluster 2021-03-10T15:31:06.706105+0000 mgr.gibba032.whopeo (mgr.24625) 5197 : cluster [DBG] pgmap v2958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:07 gibba032 bash[24004]: cluster 2021-03-10T15:31:06.706105+0000 mgr.gibba032.whopeo (mgr.24625) 5197 : cluster [DBG] pgmap v2958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:07.977586+0000 mgr.gibba032.whopeo (mgr.24625) 5198 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:31:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:07.979129+0000 mgr.gibba032.whopeo (mgr.24625) 5199 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:31:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.190045+0000 mon.a (mon.0) 2034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:31:09.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.191627+0000 mgr.gibba032.whopeo (mgr.24625) 5200 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:31:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.193654+0000 mon.a (mon.0) 2035 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:31:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.194892+0000 mgr.gibba032.whopeo (mgr.24625) 5201 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:31:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.196442+0000 mon.a (mon.0) 2036 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.197626+0000 mgr.gibba032.whopeo (mgr.24625) 5202 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:31:09.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.197667+0000 mgr.gibba032.whopeo (mgr.24625) 5203 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:31:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.199255+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.200060+0000 mgr.gibba032.whopeo (mgr.24625) 5204 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:31:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.200114+0000 mgr.gibba032.whopeo (mgr.24625) 5205 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:31:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.201608+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.202336+0000 mgr.gibba032.whopeo (mgr.24625) 5206 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:31:09.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.204340+0000 mgr.gibba032.whopeo (mgr.24625) 5207 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:31:09.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.204379+0000 mgr.gibba032.whopeo (mgr.24625) 5208 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:31:09.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.205977+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.206697+0000 mgr.gibba032.whopeo (mgr.24625) 5209 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:31:09.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.207106+0000 mgr.gibba032.whopeo (mgr.24625) 5210 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:31:09.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: audit 2021-03-10T15:31:08.208678+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.209852+0000 mgr.gibba032.whopeo (mgr.24625) 5211 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:31:09.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.209892+0000 mgr.gibba032.whopeo (mgr.24625) 5212 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:31:09.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:09 gibba032 bash[24004]: cephadm 2021-03-10T15:31:08.210010+0000 mgr.gibba032.whopeo (mgr.24625) 5213 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:07.977586+0000 mgr.gibba032.whopeo (mgr.24625) 5198 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:31:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:07.979129+0000 mgr.gibba032.whopeo (mgr.24625) 5199 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:31:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.190045+0000 mon.a (mon.0) 2034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:31:09.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.191627+0000 mgr.gibba032.whopeo (mgr.24625) 5200 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:31:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.193654+0000 mon.a (mon.0) 2035 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:31:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.194892+0000 mgr.gibba032.whopeo (mgr.24625) 5201 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:31:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.196442+0000 mon.a (mon.0) 2036 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.197626+0000 mgr.gibba032.whopeo (mgr.24625) 5202 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:31:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.197667+0000 mgr.gibba032.whopeo (mgr.24625) 5203 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:31:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.199255+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.200060+0000 mgr.gibba032.whopeo (mgr.24625) 5204 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:31:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.200114+0000 mgr.gibba032.whopeo (mgr.24625) 5205 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:31:09.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.201608+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.202336+0000 mgr.gibba032.whopeo (mgr.24625) 5206 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:31:09.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.204340+0000 mgr.gibba032.whopeo (mgr.24625) 5207 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:31:09.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.204379+0000 mgr.gibba032.whopeo (mgr.24625) 5208 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:31:09.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.205977+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.206697+0000 mgr.gibba032.whopeo (mgr.24625) 5209 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:31:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.207106+0000 mgr.gibba032.whopeo (mgr.24625) 5210 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:31:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: audit 2021-03-10T15:31:08.208678+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.209852+0000 mgr.gibba032.whopeo (mgr.24625) 5211 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:31:09.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.209892+0000 mgr.gibba032.whopeo (mgr.24625) 5212 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:31:09.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[30791]: cephadm 2021-03-10T15:31:08.210010+0000 mgr.gibba032.whopeo (mgr.24625) 5213 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:07.977586+0000 mgr.gibba032.whopeo (mgr.24625) 5198 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:31:09.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:07.979129+0000 mgr.gibba032.whopeo (mgr.24625) 5199 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:31:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.190045+0000 mon.a (mon.0) 2034 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:31:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.191627+0000 mgr.gibba032.whopeo (mgr.24625) 5200 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:31:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.193654+0000 mon.a (mon.0) 2035 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:31:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.194892+0000 mgr.gibba032.whopeo (mgr.24625) 5201 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:31:09.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.196442+0000 mon.a (mon.0) 2036 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.197626+0000 mgr.gibba032.whopeo (mgr.24625) 5202 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:31:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.197667+0000 mgr.gibba032.whopeo (mgr.24625) 5203 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:31:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.199255+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.200060+0000 mgr.gibba032.whopeo (mgr.24625) 5204 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:31:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.200114+0000 mgr.gibba032.whopeo (mgr.24625) 5205 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:31:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.201608+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.202336+0000 mgr.gibba032.whopeo (mgr.24625) 5206 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:31:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.204340+0000 mgr.gibba032.whopeo (mgr.24625) 5207 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:31:09.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.204379+0000 mgr.gibba032.whopeo (mgr.24625) 5208 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:31:09.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.205977+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.206697+0000 mgr.gibba032.whopeo (mgr.24625) 5209 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:31:09.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.207106+0000 mgr.gibba032.whopeo (mgr.24625) 5210 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:31:09.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: audit 2021-03-10T15:31:08.208678+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:09.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.209852+0000 mgr.gibba032.whopeo (mgr.24625) 5211 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:31:09.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.209892+0000 mgr.gibba032.whopeo (mgr.24625) 5212 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:31:09.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:09 gibba011 bash[31774]: cephadm 2021-03-10T15:31:08.210010+0000 mgr.gibba032.whopeo (mgr.24625) 5213 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:10 gibba032 bash[24004]: cluster 2021-03-10T15:31:08.707684+0000 mgr.gibba032.whopeo (mgr.24625) 5214 : cluster [DBG] pgmap v2959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:10 gibba011 bash[30791]: cluster 2021-03-10T15:31:08.707684+0000 mgr.gibba032.whopeo (mgr.24625) 5214 : cluster [DBG] pgmap v2959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:10 gibba011 bash[31774]: cluster 2021-03-10T15:31:08.707684+0000 mgr.gibba032.whopeo (mgr.24625) 5214 : cluster [DBG] pgmap v2959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:12 gibba032 bash[24004]: cluster 2021-03-10T15:31:10.708379+0000 mgr.gibba032.whopeo (mgr.24625) 5215 : cluster [DBG] pgmap v2960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:12 gibba011 bash[30791]: cluster 2021-03-10T15:31:10.708379+0000 mgr.gibba032.whopeo (mgr.24625) 5215 : cluster [DBG] pgmap v2960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:12 gibba011 bash[31774]: cluster 2021-03-10T15:31:10.708379+0000 mgr.gibba032.whopeo (mgr.24625) 5215 : cluster [DBG] pgmap v2960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:12.586 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:31:12.814 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:31:12.815 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (97m) 2m ago 103m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:31:12.815 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (103m) 65s ago 103m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:31:12.815 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (98m) 65s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:31:12.815 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (98m) 65s ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:31:12.816 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (97m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:31:12.816 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (97m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:31:12.816 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (96m) 65s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:31:12.816 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (97m) 2m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:31:12.816 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (103m) 2m ago 103m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:31:12.817 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (103m) 65s ago 103m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:31:12.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (96m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:31:12.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (95m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:31:12.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (95m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:31:12.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (94m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:31:12.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (94m) 65s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:31:12.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (94m) 65s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:31:12.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (93m) 65s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:31:12.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (93m) 65s ago 103m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:31:12.819 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (102m) 65s ago 103m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:31:12.819 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 101m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:31:13.068 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:31:13.069 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:31:13.069 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:31:13.069 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:13.070 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:31:13.070 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:31:13.070 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:13.070 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:31:13.071 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:31:13.071 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:13.071 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:31:13.071 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:31:13.071 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:31:13.072 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:31:13.072 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:31:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[30791]: audit 2021-03-10T15:31:12.573028+0000 mgr.gibba032.whopeo (mgr.24625) 5216 : audit [DBG] from='client.57532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[30791]: cluster 2021-03-10T15:31:12.710062+0000 mgr.gibba032.whopeo (mgr.24625) 5217 : cluster [DBG] pgmap v2961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:14.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[30791]: audit 2021-03-10T15:31:12.798134+0000 mgr.gibba032.whopeo (mgr.24625) 5218 : audit [DBG] from='client.47553 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[30791]: audit 2021-03-10T15:31:13.065639+0000 mon.a (mon.0) 2041 : audit [DBG] from='client.? 172.21.2.111:0/1293342021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[31774]: audit 2021-03-10T15:31:12.573028+0000 mgr.gibba032.whopeo (mgr.24625) 5216 : audit [DBG] from='client.57532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[31774]: cluster 2021-03-10T15:31:12.710062+0000 mgr.gibba032.whopeo (mgr.24625) 5217 : cluster [DBG] pgmap v2961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[31774]: audit 2021-03-10T15:31:12.798134+0000 mgr.gibba032.whopeo (mgr.24625) 5218 : audit [DBG] from='client.47553 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:13 gibba011 bash[31774]: audit 2021-03-10T15:31:13.065639+0000 mon.a (mon.0) 2041 : audit [DBG] from='client.? 172.21.2.111:0/1293342021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:13 gibba032 bash[24004]: audit 2021-03-10T15:31:12.573028+0000 mgr.gibba032.whopeo (mgr.24625) 5216 : audit [DBG] from='client.57532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:13 gibba032 bash[24004]: cluster 2021-03-10T15:31:12.710062+0000 mgr.gibba032.whopeo (mgr.24625) 5217 : cluster [DBG] pgmap v2961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:13 gibba032 bash[24004]: audit 2021-03-10T15:31:12.798134+0000 mgr.gibba032.whopeo (mgr.24625) 5218 : audit [DBG] from='client.47553 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:13 gibba032 bash[24004]: audit 2021-03-10T15:31:13.065639+0000 mon.a (mon.0) 2041 : audit [DBG] from='client.? 172.21.2.111:0/1293342021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:16.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:15 gibba011 bash[30791]: cluster 2021-03-10T15:31:14.711457+0000 mgr.gibba032.whopeo (mgr.24625) 5219 : cluster [DBG] pgmap v2962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:16.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:15 gibba011 bash[31774]: cluster 2021-03-10T15:31:14.711457+0000 mgr.gibba032.whopeo (mgr.24625) 5219 : cluster [DBG] pgmap v2962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:15 gibba032 bash[24004]: cluster 2021-03-10T15:31:14.711457+0000 mgr.gibba032.whopeo (mgr.24625) 5219 : cluster [DBG] pgmap v2962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:17 gibba032 bash[24004]: cluster 2021-03-10T15:31:16.712999+0000 mgr.gibba032.whopeo (mgr.24625) 5220 : cluster [DBG] pgmap v2963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:17 gibba011 bash[30791]: cluster 2021-03-10T15:31:16.712999+0000 mgr.gibba032.whopeo (mgr.24625) 5220 : cluster [DBG] pgmap v2963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:17 gibba011 bash[31774]: cluster 2021-03-10T15:31:16.712999+0000 mgr.gibba032.whopeo (mgr.24625) 5220 : cluster [DBG] pgmap v2963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:19 gibba032 bash[24004]: cluster 2021-03-10T15:31:18.714583+0000 mgr.gibba032.whopeo (mgr.24625) 5221 : cluster [DBG] pgmap v2964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:19 gibba011 bash[30791]: cluster 2021-03-10T15:31:18.714583+0000 mgr.gibba032.whopeo (mgr.24625) 5221 : cluster [DBG] pgmap v2964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:19 gibba011 bash[31774]: cluster 2021-03-10T15:31:18.714583+0000 mgr.gibba032.whopeo (mgr.24625) 5221 : cluster [DBG] pgmap v2964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:21 gibba032 bash[24004]: cluster 2021-03-10T15:31:20.715186+0000 mgr.gibba032.whopeo (mgr.24625) 5222 : cluster [DBG] pgmap v2965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:21 gibba011 bash[30791]: cluster 2021-03-10T15:31:20.715186+0000 mgr.gibba032.whopeo (mgr.24625) 5222 : cluster [DBG] pgmap v2965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:22.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:21 gibba011 bash[31774]: cluster 2021-03-10T15:31:20.715186+0000 mgr.gibba032.whopeo (mgr.24625) 5222 : cluster [DBG] pgmap v2965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:23 gibba032 bash[24004]: cluster 2021-03-10T15:31:22.716947+0000 mgr.gibba032.whopeo (mgr.24625) 5223 : cluster [DBG] pgmap v2966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:23 gibba032 bash[24004]: cephadm 2021-03-10T15:31:23.225986+0000 mgr.gibba032.whopeo (mgr.24625) 5224 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:23 gibba011 bash[30791]: cluster 2021-03-10T15:31:22.716947+0000 mgr.gibba032.whopeo (mgr.24625) 5223 : cluster [DBG] pgmap v2966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:23 gibba011 bash[30791]: cephadm 2021-03-10T15:31:23.225986+0000 mgr.gibba032.whopeo (mgr.24625) 5224 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:23 gibba011 bash[31774]: cluster 2021-03-10T15:31:22.716947+0000 mgr.gibba032.whopeo (mgr.24625) 5223 : cluster [DBG] pgmap v2966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:23 gibba011 bash[31774]: cephadm 2021-03-10T15:31:23.225986+0000 mgr.gibba032.whopeo (mgr.24625) 5224 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:25 gibba032 bash[24004]: cluster 2021-03-10T15:31:24.718455+0000 mgr.gibba032.whopeo (mgr.24625) 5225 : cluster [DBG] pgmap v2967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:25 gibba011 bash[30791]: cluster 2021-03-10T15:31:24.718455+0000 mgr.gibba032.whopeo (mgr.24625) 5225 : cluster [DBG] pgmap v2967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:25 gibba011 bash[31774]: cluster 2021-03-10T15:31:24.718455+0000 mgr.gibba032.whopeo (mgr.24625) 5225 : cluster [DBG] pgmap v2967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:27 gibba032 bash[24004]: cluster 2021-03-10T15:31:26.720306+0000 mgr.gibba032.whopeo (mgr.24625) 5226 : cluster [DBG] pgmap v2968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:27 gibba011 bash[30791]: cluster 2021-03-10T15:31:26.720306+0000 mgr.gibba032.whopeo (mgr.24625) 5226 : cluster [DBG] pgmap v2968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:27 gibba011 bash[31774]: cluster 2021-03-10T15:31:26.720306+0000 mgr.gibba032.whopeo (mgr.24625) 5226 : cluster [DBG] pgmap v2968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:29 gibba032 bash[24004]: cluster 2021-03-10T15:31:28.721693+0000 mgr.gibba032.whopeo (mgr.24625) 5227 : cluster [DBG] pgmap v2969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:29 gibba011 bash[30791]: cluster 2021-03-10T15:31:28.721693+0000 mgr.gibba032.whopeo (mgr.24625) 5227 : cluster [DBG] pgmap v2969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:30.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:29 gibba011 bash[31774]: cluster 2021-03-10T15:31:28.721693+0000 mgr.gibba032.whopeo (mgr.24625) 5227 : cluster [DBG] pgmap v2969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:31 gibba032 bash[24004]: cluster 2021-03-10T15:31:30.723046+0000 mgr.gibba032.whopeo (mgr.24625) 5228 : cluster [DBG] pgmap v2970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:31 gibba011 bash[30791]: cluster 2021-03-10T15:31:30.723046+0000 mgr.gibba032.whopeo (mgr.24625) 5228 : cluster [DBG] pgmap v2970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:31 gibba011 bash[31774]: cluster 2021-03-10T15:31:30.723046+0000 mgr.gibba032.whopeo (mgr.24625) 5228 : cluster [DBG] pgmap v2970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:33 gibba032 bash[24004]: cluster 2021-03-10T15:31:32.724834+0000 mgr.gibba032.whopeo (mgr.24625) 5229 : cluster [DBG] pgmap v2971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:33 gibba011 bash[30791]: cluster 2021-03-10T15:31:32.724834+0000 mgr.gibba032.whopeo (mgr.24625) 5229 : cluster [DBG] pgmap v2971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:33 gibba011 bash[31774]: cluster 2021-03-10T15:31:32.724834+0000 mgr.gibba032.whopeo (mgr.24625) 5229 : cluster [DBG] pgmap v2971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:35 gibba032 bash[24004]: cluster 2021-03-10T15:31:34.726340+0000 mgr.gibba032.whopeo (mgr.24625) 5230 : cluster [DBG] pgmap v2972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:35 gibba011 bash[30791]: cluster 2021-03-10T15:31:34.726340+0000 mgr.gibba032.whopeo (mgr.24625) 5230 : cluster [DBG] pgmap v2972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:36.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:35 gibba011 bash[31774]: cluster 2021-03-10T15:31:34.726340+0000 mgr.gibba032.whopeo (mgr.24625) 5230 : cluster [DBG] pgmap v2972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:38.237 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:37 gibba032 bash[24004]: cluster 2021-03-10T15:31:36.727974+0000 mgr.gibba032.whopeo (mgr.24625) 5231 : cluster [DBG] pgmap v2973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:37 gibba011 bash[30791]: cluster 2021-03-10T15:31:36.727974+0000 mgr.gibba032.whopeo (mgr.24625) 5231 : cluster [DBG] pgmap v2973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:37 gibba011 bash[31774]: cluster 2021-03-10T15:31:36.727974+0000 mgr.gibba032.whopeo (mgr.24625) 5231 : cluster [DBG] pgmap v2973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:38 gibba032 bash[24004]: cephadm 2021-03-10T15:31:38.232791+0000 mgr.gibba032.whopeo (mgr.24625) 5232 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:38 gibba011 bash[30791]: cephadm 2021-03-10T15:31:38.232791+0000 mgr.gibba032.whopeo (mgr.24625) 5232 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:38 gibba011 bash[31774]: cephadm 2021-03-10T15:31:38.232791+0000 mgr.gibba032.whopeo (mgr.24625) 5232 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:39 gibba032 bash[24004]: cluster 2021-03-10T15:31:38.729644+0000 mgr.gibba032.whopeo (mgr.24625) 5233 : cluster [DBG] pgmap v2974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:39 gibba011 bash[30791]: cluster 2021-03-10T15:31:38.729644+0000 mgr.gibba032.whopeo (mgr.24625) 5233 : cluster [DBG] pgmap v2974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:39 gibba011 bash[31774]: cluster 2021-03-10T15:31:38.729644+0000 mgr.gibba032.whopeo (mgr.24625) 5233 : cluster [DBG] pgmap v2974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:41 gibba032 bash[24004]: cluster 2021-03-10T15:31:40.731121+0000 mgr.gibba032.whopeo (mgr.24625) 5234 : cluster [DBG] pgmap v2975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:41 gibba011 bash[30791]: cluster 2021-03-10T15:31:40.731121+0000 mgr.gibba032.whopeo (mgr.24625) 5234 : cluster [DBG] pgmap v2975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:41 gibba011 bash[31774]: cluster 2021-03-10T15:31:40.731121+0000 mgr.gibba032.whopeo (mgr.24625) 5234 : cluster [DBG] pgmap v2975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:43.349 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:31:43.576 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:31:43.576 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (98m) 2m ago 103m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:31:43.577 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (103m) 95s ago 103m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:31:43.577 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (98m) 95s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:31:43.577 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (99m) 95s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:31:43.577 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (98m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:31:43.577 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (97m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:31:43.578 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (97m) 95s ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:31:43.578 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (97m) 2m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:31:43.578 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (103m) 2m ago 103m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:31:43.578 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (103m) 95s ago 103m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:31:43.579 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (96m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:31:43.579 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (96m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:31:43.579 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (95m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:31:43.579 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (95m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:31:43.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (94m) 95s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:31:43.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (94m) 95s ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:31:43.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (94m) 95s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:31:43.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (93m) 95s ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:31:43.581 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (103m) 95s ago 103m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:31:43.581 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 101m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:31:43.826 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:31:43.826 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:31:43.826 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:31:43.827 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:43.827 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:31:43.827 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:31:43.827 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:43.827 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:31:43.828 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:31:43.828 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:31:43.828 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:31:43.828 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:31:43.829 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:31:43.829 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:31:43.829 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:31:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:43 gibba032 bash[24004]: cluster 2021-03-10T15:31:42.733002+0000 mgr.gibba032.whopeo (mgr.24625) 5235 : cluster [DBG] pgmap v2976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:43 gibba032 bash[24004]: audit 2021-03-10T15:31:43.335517+0000 mgr.gibba032.whopeo (mgr.24625) 5236 : audit [DBG] from='client.47565 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:44.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:43 gibba032 bash[24004]: audit 2021-03-10T15:31:43.822634+0000 mon.c (mon.1) 98 : audit [DBG] from='client.? 172.21.2.111:0/4211174005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[30791]: cluster 2021-03-10T15:31:42.733002+0000 mgr.gibba032.whopeo (mgr.24625) 5235 : cluster [DBG] pgmap v2976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[30791]: audit 2021-03-10T15:31:43.335517+0000 mgr.gibba032.whopeo (mgr.24625) 5236 : audit [DBG] from='client.47565 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:44.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[30791]: audit 2021-03-10T15:31:43.822634+0000 mon.c (mon.1) 98 : audit [DBG] from='client.? 172.21.2.111:0/4211174005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[31774]: cluster 2021-03-10T15:31:42.733002+0000 mgr.gibba032.whopeo (mgr.24625) 5235 : cluster [DBG] pgmap v2976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[31774]: audit 2021-03-10T15:31:43.335517+0000 mgr.gibba032.whopeo (mgr.24625) 5236 : audit [DBG] from='client.47565 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:44.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:43 gibba011 bash[31774]: audit 2021-03-10T15:31:43.822634+0000 mon.c (mon.1) 98 : audit [DBG] from='client.? 172.21.2.111:0/4211174005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:31:45.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:44 gibba032 bash[24004]: audit 2021-03-10T15:31:43.559930+0000 mgr.gibba032.whopeo (mgr.24625) 5237 : audit [DBG] from='client.57556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:44 gibba011 bash[30791]: audit 2021-03-10T15:31:43.559930+0000 mgr.gibba032.whopeo (mgr.24625) 5237 : audit [DBG] from='client.57556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:44 gibba011 bash[31774]: audit 2021-03-10T15:31:43.559930+0000 mgr.gibba032.whopeo (mgr.24625) 5237 : audit [DBG] from='client.57556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:31:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:45 gibba032 bash[24004]: cluster 2021-03-10T15:31:44.734561+0000 mgr.gibba032.whopeo (mgr.24625) 5238 : cluster [DBG] pgmap v2977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:45 gibba011 bash[30791]: cluster 2021-03-10T15:31:44.734561+0000 mgr.gibba032.whopeo (mgr.24625) 5238 : cluster [DBG] pgmap v2977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:45 gibba011 bash[31774]: cluster 2021-03-10T15:31:44.734561+0000 mgr.gibba032.whopeo (mgr.24625) 5238 : cluster [DBG] pgmap v2977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:47 gibba032 bash[24004]: cluster 2021-03-10T15:31:46.736231+0000 mgr.gibba032.whopeo (mgr.24625) 5239 : cluster [DBG] pgmap v2978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:47 gibba011 bash[30791]: cluster 2021-03-10T15:31:46.736231+0000 mgr.gibba032.whopeo (mgr.24625) 5239 : cluster [DBG] pgmap v2978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:47 gibba011 bash[31774]: cluster 2021-03-10T15:31:46.736231+0000 mgr.gibba032.whopeo (mgr.24625) 5239 : cluster [DBG] pgmap v2978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:49 gibba032 bash[24004]: cluster 2021-03-10T15:31:48.737575+0000 mgr.gibba032.whopeo (mgr.24625) 5240 : cluster [DBG] pgmap v2979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:49 gibba011 bash[30791]: cluster 2021-03-10T15:31:48.737575+0000 mgr.gibba032.whopeo (mgr.24625) 5240 : cluster [DBG] pgmap v2979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:50.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:49 gibba011 bash[31774]: cluster 2021-03-10T15:31:48.737575+0000 mgr.gibba032.whopeo (mgr.24625) 5240 : cluster [DBG] pgmap v2979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:51 gibba032 bash[24004]: cluster 2021-03-10T15:31:50.738906+0000 mgr.gibba032.whopeo (mgr.24625) 5241 : cluster [DBG] pgmap v2980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:51 gibba011 bash[30791]: cluster 2021-03-10T15:31:50.738906+0000 mgr.gibba032.whopeo (mgr.24625) 5241 : cluster [DBG] pgmap v2980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:51 gibba011 bash[31774]: cluster 2021-03-10T15:31:50.738906+0000 mgr.gibba032.whopeo (mgr.24625) 5241 : cluster [DBG] pgmap v2980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:54 gibba032 bash[24004]: cluster 2021-03-10T15:31:52.740692+0000 mgr.gibba032.whopeo (mgr.24625) 5242 : cluster [DBG] pgmap v2981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:54 gibba032 bash[24004]: cephadm 2021-03-10T15:31:53.248824+0000 mgr.gibba032.whopeo (mgr.24625) 5243 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:54 gibba011 bash[30791]: cluster 2021-03-10T15:31:52.740692+0000 mgr.gibba032.whopeo (mgr.24625) 5242 : cluster [DBG] pgmap v2981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:54 gibba011 bash[30791]: cephadm 2021-03-10T15:31:53.248824+0000 mgr.gibba032.whopeo (mgr.24625) 5243 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:54 gibba011 bash[31774]: cluster 2021-03-10T15:31:52.740692+0000 mgr.gibba032.whopeo (mgr.24625) 5242 : cluster [DBG] pgmap v2981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:54 gibba011 bash[31774]: cephadm 2021-03-10T15:31:53.248824+0000 mgr.gibba032.whopeo (mgr.24625) 5243 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:31:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:56 gibba011 bash[30791]: cluster 2021-03-10T15:31:54.742257+0000 mgr.gibba032.whopeo (mgr.24625) 5244 : cluster [DBG] pgmap v2982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:56 gibba011 bash[31774]: cluster 2021-03-10T15:31:54.742257+0000 mgr.gibba032.whopeo (mgr.24625) 5244 : cluster [DBG] pgmap v2982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:56 gibba032 bash[24004]: cluster 2021-03-10T15:31:54.742257+0000 mgr.gibba032.whopeo (mgr.24625) 5244 : cluster [DBG] pgmap v2982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.629509+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:31:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.630124+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:31:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.642583+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:31:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.643291+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.644023+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[30791]: audit 2021-03-10T15:31:56.644688+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:31:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.629509+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:31:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.630124+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:31:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.642583+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:31:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.643291+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.644023+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:57 gibba011 bash[31774]: audit 2021-03-10T15:31:56.644688+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:31:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.629509+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:31:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.630124+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:31:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.642583+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:31:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.643291+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.644023+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:31:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:57 gibba032 bash[24004]: audit 2021-03-10T15:31:56.644688+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:31:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:31:58 gibba011 bash[30791]: cluster 2021-03-10T15:31:56.743870+0000 mgr.gibba032.whopeo (mgr.24625) 5245 : cluster [DBG] pgmap v2983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:31:58 gibba011 bash[31774]: cluster 2021-03-10T15:31:56.743870+0000 mgr.gibba032.whopeo (mgr.24625) 5245 : cluster [DBG] pgmap v2983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:31:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:31:58 gibba032 bash[24004]: cluster 2021-03-10T15:31:56.743870+0000 mgr.gibba032.whopeo (mgr.24625) 5245 : cluster [DBG] pgmap v2983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:00 gibba011 bash[30791]: cluster 2021-03-10T15:31:58.745422+0000 mgr.gibba032.whopeo (mgr.24625) 5246 : cluster [DBG] pgmap v2984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:00 gibba011 bash[31774]: cluster 2021-03-10T15:31:58.745422+0000 mgr.gibba032.whopeo (mgr.24625) 5246 : cluster [DBG] pgmap v2984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:00 gibba032 bash[24004]: cluster 2021-03-10T15:31:58.745422+0000 mgr.gibba032.whopeo (mgr.24625) 5246 : cluster [DBG] pgmap v2984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:02 gibba011 bash[30791]: cluster 2021-03-10T15:32:00.746765+0000 mgr.gibba032.whopeo (mgr.24625) 5247 : cluster [DBG] pgmap v2985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:02 gibba011 bash[31774]: cluster 2021-03-10T15:32:00.746765+0000 mgr.gibba032.whopeo (mgr.24625) 5247 : cluster [DBG] pgmap v2985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:02 gibba032 bash[24004]: cluster 2021-03-10T15:32:00.746765+0000 mgr.gibba032.whopeo (mgr.24625) 5247 : cluster [DBG] pgmap v2985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:04 gibba011 bash[30791]: cluster 2021-03-10T15:32:02.748548+0000 mgr.gibba032.whopeo (mgr.24625) 5248 : cluster [DBG] pgmap v2986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:04 gibba011 bash[31774]: cluster 2021-03-10T15:32:02.748548+0000 mgr.gibba032.whopeo (mgr.24625) 5248 : cluster [DBG] pgmap v2986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:04 gibba032 bash[24004]: cluster 2021-03-10T15:32:02.748548+0000 mgr.gibba032.whopeo (mgr.24625) 5248 : cluster [DBG] pgmap v2986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:06.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:06 gibba011 bash[30791]: cluster 2021-03-10T15:32:04.749820+0000 mgr.gibba032.whopeo (mgr.24625) 5249 : cluster [DBG] pgmap v2987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:06.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:06 gibba011 bash[31774]: cluster 2021-03-10T15:32:04.749820+0000 mgr.gibba032.whopeo (mgr.24625) 5249 : cluster [DBG] pgmap v2987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:06 gibba032 bash[24004]: cluster 2021-03-10T15:32:04.749820+0000 mgr.gibba032.whopeo (mgr.24625) 5249 : cluster [DBG] pgmap v2987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:08 gibba011 bash[30791]: cluster 2021-03-10T15:32:06.751463+0000 mgr.gibba032.whopeo (mgr.24625) 5250 : cluster [DBG] pgmap v2988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:08 gibba011 bash[31774]: cluster 2021-03-10T15:32:06.751463+0000 mgr.gibba032.whopeo (mgr.24625) 5250 : cluster [DBG] pgmap v2988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:08.486 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:08 gibba032 bash[24004]: cluster 2021-03-10T15:32:06.751463+0000 mgr.gibba032.whopeo (mgr.24625) 5250 : cluster [DBG] pgmap v2988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.261693+0000 mgr.gibba032.whopeo (mgr.24625) 5251 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:32:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.263188+0000 mgr.gibba032.whopeo (mgr.24625) 5252 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:32:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.469506+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:32:09.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.471235+0000 mgr.gibba032.whopeo (mgr.24625) 5253 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:32:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.473022+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:32:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.473092+0000 mgr.gibba032.whopeo (mgr.24625) 5254 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:32:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.474672+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.474783+0000 mgr.gibba032.whopeo (mgr.24625) 5255 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:32:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.474821+0000 mgr.gibba032.whopeo (mgr.24625) 5256 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:32:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.476402+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.476622+0000 mgr.gibba032.whopeo (mgr.24625) 5257 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:32:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.476659+0000 mgr.gibba032.whopeo (mgr.24625) 5258 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:32:09.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.478013+0000 mgr.gibba032.whopeo (mgr.24625) 5259 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:32:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.478015+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.479294+0000 mgr.gibba032.whopeo (mgr.24625) 5260 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:32:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.479332+0000 mgr.gibba032.whopeo (mgr.24625) 5261 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:32:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.480764+0000 mgr.gibba032.whopeo (mgr.24625) 5262 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:32:09.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.480857+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.481075+0000 mgr.gibba032.whopeo (mgr.24625) 5263 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:32:09.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.482348+0000 mgr.gibba032.whopeo (mgr.24625) 5264 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:32:09.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.482387+0000 mgr.gibba032.whopeo (mgr.24625) 5265 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:32:09.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: audit 2021-03-10T15:32:08.482419+0000 mon.a (mon.0) 2054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:09 gibba032 bash[24004]: cephadm 2021-03-10T15:32:08.482492+0000 mgr.gibba032.whopeo (mgr.24625) 5266 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.261693+0000 mgr.gibba032.whopeo (mgr.24625) 5251 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:32:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.263188+0000 mgr.gibba032.whopeo (mgr.24625) 5252 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:32:09.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.469506+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:32:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.471235+0000 mgr.gibba032.whopeo (mgr.24625) 5253 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:32:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.473022+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:32:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.473092+0000 mgr.gibba032.whopeo (mgr.24625) 5254 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:32:09.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.474672+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.474783+0000 mgr.gibba032.whopeo (mgr.24625) 5255 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:32:09.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.474821+0000 mgr.gibba032.whopeo (mgr.24625) 5256 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:32:09.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.476402+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.476622+0000 mgr.gibba032.whopeo (mgr.24625) 5257 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:32:09.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.476659+0000 mgr.gibba032.whopeo (mgr.24625) 5258 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:32:09.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.478013+0000 mgr.gibba032.whopeo (mgr.24625) 5259 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:32:09.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.478015+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.479294+0000 mgr.gibba032.whopeo (mgr.24625) 5260 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:32:09.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.479332+0000 mgr.gibba032.whopeo (mgr.24625) 5261 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:32:09.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.480764+0000 mgr.gibba032.whopeo (mgr.24625) 5262 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:32:09.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.480857+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.481075+0000 mgr.gibba032.whopeo (mgr.24625) 5263 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:32:09.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.482348+0000 mgr.gibba032.whopeo (mgr.24625) 5264 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:32:09.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.482387+0000 mgr.gibba032.whopeo (mgr.24625) 5265 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:32:09.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: audit 2021-03-10T15:32:08.482419+0000 mon.a (mon.0) 2054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[30791]: cephadm 2021-03-10T15:32:08.482492+0000 mgr.gibba032.whopeo (mgr.24625) 5266 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:09.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.261693+0000 mgr.gibba032.whopeo (mgr.24625) 5251 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:32:09.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.263188+0000 mgr.gibba032.whopeo (mgr.24625) 5252 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:32:09.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.469506+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:32:09.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.471235+0000 mgr.gibba032.whopeo (mgr.24625) 5253 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:32:09.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.473022+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:32:09.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.473092+0000 mgr.gibba032.whopeo (mgr.24625) 5254 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:32:09.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.474672+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.474783+0000 mgr.gibba032.whopeo (mgr.24625) 5255 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:32:09.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.474821+0000 mgr.gibba032.whopeo (mgr.24625) 5256 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:32:09.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.476402+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.476622+0000 mgr.gibba032.whopeo (mgr.24625) 5257 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:32:09.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.476659+0000 mgr.gibba032.whopeo (mgr.24625) 5258 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:32:09.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.478013+0000 mgr.gibba032.whopeo (mgr.24625) 5259 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:32:09.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.478015+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.479294+0000 mgr.gibba032.whopeo (mgr.24625) 5260 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:32:09.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.479332+0000 mgr.gibba032.whopeo (mgr.24625) 5261 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:32:09.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.480764+0000 mgr.gibba032.whopeo (mgr.24625) 5262 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:32:09.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.480857+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.481075+0000 mgr.gibba032.whopeo (mgr.24625) 5263 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:32:09.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.482348+0000 mgr.gibba032.whopeo (mgr.24625) 5264 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:32:09.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.482387+0000 mgr.gibba032.whopeo (mgr.24625) 5265 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:32:09.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: audit 2021-03-10T15:32:08.482419+0000 mon.a (mon.0) 2054 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:09.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:09 gibba011 bash[31774]: cephadm 2021-03-10T15:32:08.482492+0000 mgr.gibba032.whopeo (mgr.24625) 5266 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:10 gibba032 bash[24004]: cluster 2021-03-10T15:32:08.753109+0000 mgr.gibba032.whopeo (mgr.24625) 5267 : cluster [DBG] pgmap v2989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:10.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:10 gibba011 bash[30791]: cluster 2021-03-10T15:32:08.753109+0000 mgr.gibba032.whopeo (mgr.24625) 5267 : cluster [DBG] pgmap v2989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:10.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:10 gibba011 bash[31774]: cluster 2021-03-10T15:32:08.753109+0000 mgr.gibba032.whopeo (mgr.24625) 5267 : cluster [DBG] pgmap v2989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:12 gibba032 bash[24004]: cluster 2021-03-10T15:32:10.754475+0000 mgr.gibba032.whopeo (mgr.24625) 5268 : cluster [DBG] pgmap v2990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:12 gibba011 bash[30791]: cluster 2021-03-10T15:32:10.754475+0000 mgr.gibba032.whopeo (mgr.24625) 5268 : cluster [DBG] pgmap v2990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:12.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:12 gibba011 bash[31774]: cluster 2021-03-10T15:32:10.754475+0000 mgr.gibba032.whopeo (mgr.24625) 5268 : cluster [DBG] pgmap v2990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:14.067 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:32:14.295 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:32:14.295 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (98m) 3m ago 104m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:32:14.295 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (104m) 2m ago 104m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:32:14.295 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (99m) 2m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:32:14.296 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (99m) 2m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:32:14.296 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (98m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:32:14.296 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (98m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:32:14.296 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (97m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:32:14.297 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (98m) 3m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:32:14.297 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (104m) 3m ago 104m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:32:14.297 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (104m) 2m ago 104m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:32:14.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (97m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:32:14.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (96m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:32:14.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (96m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:32:14.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (95m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:32:14.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (95m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:32:14.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (95m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:32:14.299 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (94m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:32:14.299 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (94m) 2m ago 104m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:32:14.299 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (103m) 2m ago 104m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:32:14.299 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 102m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:32:14.543 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:32:14.543 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:32:14.543 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:32:14.544 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:14.544 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:32:14.544 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:32:14.544 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:14.545 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:32:14.545 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:32:14.545 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:14.545 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:32:14.545 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:32:14.546 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:32:14.546 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:32:14.546 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:32:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:14 gibba032 bash[24004]: cluster 2021-03-10T15:32:12.756386+0000 mgr.gibba032.whopeo (mgr.24625) 5269 : cluster [DBG] pgmap v2991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:14 gibba011 bash[30791]: cluster 2021-03-10T15:32:12.756386+0000 mgr.gibba032.whopeo (mgr.24625) 5269 : cluster [DBG] pgmap v2991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:14.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:14 gibba011 bash[31774]: cluster 2021-03-10T15:32:12.756386+0000 mgr.gibba032.whopeo (mgr.24625) 5269 : cluster [DBG] pgmap v2991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:15.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[30791]: audit 2021-03-10T15:32:14.053955+0000 mgr.gibba032.whopeo (mgr.24625) 5270 : audit [DBG] from='client.47583 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:15.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[30791]: audit 2021-03-10T15:32:14.278742+0000 mgr.gibba032.whopeo (mgr.24625) 5271 : audit [DBG] from='client.47589 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:15.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[30791]: audit 2021-03-10T15:32:14.539960+0000 mon.c (mon.1) 99 : audit [DBG] from='client.? 172.21.2.111:0/1815478608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:15.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[31774]: audit 2021-03-10T15:32:14.053955+0000 mgr.gibba032.whopeo (mgr.24625) 5270 : audit [DBG] from='client.47583 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:15.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[31774]: audit 2021-03-10T15:32:14.278742+0000 mgr.gibba032.whopeo (mgr.24625) 5271 : audit [DBG] from='client.47589 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:15.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:15 gibba011 bash[31774]: audit 2021-03-10T15:32:14.539960+0000 mon.c (mon.1) 99 : audit [DBG] from='client.? 172.21.2.111:0/1815478608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:15 gibba032 bash[24004]: audit 2021-03-10T15:32:14.053955+0000 mgr.gibba032.whopeo (mgr.24625) 5270 : audit [DBG] from='client.47583 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:15 gibba032 bash[24004]: audit 2021-03-10T15:32:14.278742+0000 mgr.gibba032.whopeo (mgr.24625) 5271 : audit [DBG] from='client.47589 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:16.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:15 gibba032 bash[24004]: audit 2021-03-10T15:32:14.539960+0000 mon.c (mon.1) 99 : audit [DBG] from='client.? 172.21.2.111:0/1815478608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:16 gibba011 bash[30791]: cluster 2021-03-10T15:32:14.757662+0000 mgr.gibba032.whopeo (mgr.24625) 5272 : cluster [DBG] pgmap v2992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:16 gibba011 bash[31774]: cluster 2021-03-10T15:32:14.757662+0000 mgr.gibba032.whopeo (mgr.24625) 5272 : cluster [DBG] pgmap v2992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:17.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:16 gibba032 bash[24004]: cluster 2021-03-10T15:32:14.757662+0000 mgr.gibba032.whopeo (mgr.24625) 5272 : cluster [DBG] pgmap v2992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:17 gibba011 bash[30791]: cluster 2021-03-10T15:32:16.759253+0000 mgr.gibba032.whopeo (mgr.24625) 5273 : cluster [DBG] pgmap v2993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:17.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:17 gibba011 bash[31774]: cluster 2021-03-10T15:32:16.759253+0000 mgr.gibba032.whopeo (mgr.24625) 5273 : cluster [DBG] pgmap v2993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:18.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:17 gibba032 bash[24004]: cluster 2021-03-10T15:32:16.759253+0000 mgr.gibba032.whopeo (mgr.24625) 5273 : cluster [DBG] pgmap v2993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:19 gibba032 bash[24004]: cluster 2021-03-10T15:32:18.760927+0000 mgr.gibba032.whopeo (mgr.24625) 5274 : cluster [DBG] pgmap v2994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:19 gibba011 bash[30791]: cluster 2021-03-10T15:32:18.760927+0000 mgr.gibba032.whopeo (mgr.24625) 5274 : cluster [DBG] pgmap v2994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:20.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:19 gibba011 bash[31774]: cluster 2021-03-10T15:32:18.760927+0000 mgr.gibba032.whopeo (mgr.24625) 5274 : cluster [DBG] pgmap v2994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:21 gibba032 bash[24004]: cluster 2021-03-10T15:32:20.762284+0000 mgr.gibba032.whopeo (mgr.24625) 5275 : cluster [DBG] pgmap v2995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:21 gibba011 bash[30791]: cluster 2021-03-10T15:32:20.762284+0000 mgr.gibba032.whopeo (mgr.24625) 5275 : cluster [DBG] pgmap v2995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:22.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:21 gibba011 bash[31774]: cluster 2021-03-10T15:32:20.762284+0000 mgr.gibba032.whopeo (mgr.24625) 5275 : cluster [DBG] pgmap v2995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:23 gibba032 bash[24004]: cluster 2021-03-10T15:32:22.764074+0000 mgr.gibba032.whopeo (mgr.24625) 5276 : cluster [DBG] pgmap v2996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:23 gibba032 bash[24004]: cephadm 2021-03-10T15:32:23.496900+0000 mgr.gibba032.whopeo (mgr.24625) 5277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:23 gibba011 bash[30791]: cluster 2021-03-10T15:32:22.764074+0000 mgr.gibba032.whopeo (mgr.24625) 5276 : cluster [DBG] pgmap v2996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:23 gibba011 bash[30791]: cephadm 2021-03-10T15:32:23.496900+0000 mgr.gibba032.whopeo (mgr.24625) 5277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:23 gibba011 bash[31774]: cluster 2021-03-10T15:32:22.764074+0000 mgr.gibba032.whopeo (mgr.24625) 5276 : cluster [DBG] pgmap v2996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:23 gibba011 bash[31774]: cephadm 2021-03-10T15:32:23.496900+0000 mgr.gibba032.whopeo (mgr.24625) 5277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:25 gibba032 bash[24004]: cluster 2021-03-10T15:32:24.765357+0000 mgr.gibba032.whopeo (mgr.24625) 5278 : cluster [DBG] pgmap v2997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:25 gibba011 bash[30791]: cluster 2021-03-10T15:32:24.765357+0000 mgr.gibba032.whopeo (mgr.24625) 5278 : cluster [DBG] pgmap v2997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:25 gibba011 bash[31774]: cluster 2021-03-10T15:32:24.765357+0000 mgr.gibba032.whopeo (mgr.24625) 5278 : cluster [DBG] pgmap v2997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:27 gibba032 bash[24004]: cluster 2021-03-10T15:32:26.766890+0000 mgr.gibba032.whopeo (mgr.24625) 5279 : cluster [DBG] pgmap v2998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:27 gibba011 bash[30791]: cluster 2021-03-10T15:32:26.766890+0000 mgr.gibba032.whopeo (mgr.24625) 5279 : cluster [DBG] pgmap v2998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:27 gibba011 bash[31774]: cluster 2021-03-10T15:32:26.766890+0000 mgr.gibba032.whopeo (mgr.24625) 5279 : cluster [DBG] pgmap v2998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:29 gibba032 bash[24004]: cluster 2021-03-10T15:32:28.768565+0000 mgr.gibba032.whopeo (mgr.24625) 5280 : cluster [DBG] pgmap v2999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:29 gibba011 bash[30791]: cluster 2021-03-10T15:32:28.768565+0000 mgr.gibba032.whopeo (mgr.24625) 5280 : cluster [DBG] pgmap v2999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:30.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:29 gibba011 bash[31774]: cluster 2021-03-10T15:32:28.768565+0000 mgr.gibba032.whopeo (mgr.24625) 5280 : cluster [DBG] pgmap v2999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:31 gibba032 bash[24004]: cluster 2021-03-10T15:32:30.769943+0000 mgr.gibba032.whopeo (mgr.24625) 5281 : cluster [DBG] pgmap v3000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:31 gibba011 bash[30791]: cluster 2021-03-10T15:32:30.769943+0000 mgr.gibba032.whopeo (mgr.24625) 5281 : cluster [DBG] pgmap v3000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:31 gibba011 bash[31774]: cluster 2021-03-10T15:32:30.769943+0000 mgr.gibba032.whopeo (mgr.24625) 5281 : cluster [DBG] pgmap v3000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:33 gibba032 bash[24004]: cluster 2021-03-10T15:32:32.771714+0000 mgr.gibba032.whopeo (mgr.24625) 5282 : cluster [DBG] pgmap v3001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:33 gibba011 bash[30791]: cluster 2021-03-10T15:32:32.771714+0000 mgr.gibba032.whopeo (mgr.24625) 5282 : cluster [DBG] pgmap v3001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:33 gibba011 bash[31774]: cluster 2021-03-10T15:32:32.771714+0000 mgr.gibba032.whopeo (mgr.24625) 5282 : cluster [DBG] pgmap v3001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:35 gibba032 bash[24004]: cluster 2021-03-10T15:32:34.773251+0000 mgr.gibba032.whopeo (mgr.24625) 5283 : cluster [DBG] pgmap v3002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:35 gibba011 bash[30791]: cluster 2021-03-10T15:32:34.773251+0000 mgr.gibba032.whopeo (mgr.24625) 5283 : cluster [DBG] pgmap v3002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:35 gibba011 bash[31774]: cluster 2021-03-10T15:32:34.773251+0000 mgr.gibba032.whopeo (mgr.24625) 5283 : cluster [DBG] pgmap v3002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:37 gibba032 bash[24004]: cluster 2021-03-10T15:32:36.774950+0000 mgr.gibba032.whopeo (mgr.24625) 5284 : cluster [DBG] pgmap v3003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:37 gibba011 bash[30791]: cluster 2021-03-10T15:32:36.774950+0000 mgr.gibba032.whopeo (mgr.24625) 5284 : cluster [DBG] pgmap v3003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:37 gibba011 bash[31774]: cluster 2021-03-10T15:32:36.774950+0000 mgr.gibba032.whopeo (mgr.24625) 5284 : cluster [DBG] pgmap v3003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:39.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:38 gibba032 bash[24004]: cephadm 2021-03-10T15:32:38.512857+0000 mgr.gibba032.whopeo (mgr.24625) 5285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:38 gibba011 bash[30791]: cephadm 2021-03-10T15:32:38.512857+0000 mgr.gibba032.whopeo (mgr.24625) 5285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:39.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:38 gibba011 bash[31774]: cephadm 2021-03-10T15:32:38.512857+0000 mgr.gibba032.whopeo (mgr.24625) 5285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:39 gibba032 bash[24004]: cluster 2021-03-10T15:32:38.776619+0000 mgr.gibba032.whopeo (mgr.24625) 5286 : cluster [DBG] pgmap v3004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:39 gibba011 bash[30791]: cluster 2021-03-10T15:32:38.776619+0000 mgr.gibba032.whopeo (mgr.24625) 5286 : cluster [DBG] pgmap v3004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:40.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:39 gibba011 bash[31774]: cluster 2021-03-10T15:32:38.776619+0000 mgr.gibba032.whopeo (mgr.24625) 5286 : cluster [DBG] pgmap v3004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:41 gibba032 bash[24004]: cluster 2021-03-10T15:32:40.777926+0000 mgr.gibba032.whopeo (mgr.24625) 5287 : cluster [DBG] pgmap v3005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:41 gibba011 bash[30791]: cluster 2021-03-10T15:32:40.777926+0000 mgr.gibba032.whopeo (mgr.24625) 5287 : cluster [DBG] pgmap v3005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:41 gibba011 bash[31774]: cluster 2021-03-10T15:32:40.777926+0000 mgr.gibba032.whopeo (mgr.24625) 5287 : cluster [DBG] pgmap v3005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:43 gibba032 bash[24004]: cluster 2021-03-10T15:32:42.779569+0000 mgr.gibba032.whopeo (mgr.24625) 5288 : cluster [DBG] pgmap v3006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:43 gibba011 bash[30791]: cluster 2021-03-10T15:32:42.779569+0000 mgr.gibba032.whopeo (mgr.24625) 5288 : cluster [DBG] pgmap v3006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:43 gibba011 bash[31774]: cluster 2021-03-10T15:32:42.779569+0000 mgr.gibba032.whopeo (mgr.24625) 5288 : cluster [DBG] pgmap v3006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:44.787 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:32:45.014 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:32:45.015 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (99m) 3m ago 104m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:32:45.015 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (104m) 2m ago 104m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:32:45.015 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (99m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:32:45.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (100m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:32:45.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (99m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:32:45.016 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (98m) 3m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:32:45.016 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (98m) 2m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:32:45.016 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (98m) 3m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:32:45.017 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (104m) 3m ago 104m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:32:45.017 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (104m) 2m ago 104m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:32:45.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (97m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:32:45.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (97m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:32:45.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (96m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:32:45.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (96m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:32:45.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (95m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:32:45.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (95m) 2m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:32:45.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (95m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:32:45.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (94m) 2m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:32:45.019 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (104m) 2m ago 104m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:32:45.019 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 102m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:32:45.263 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:32:45.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:32:45.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:32:45.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:45.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:32:45.264 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:32:45.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:45.265 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:32:45.265 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:32:45.265 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:32:45.265 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:32:45.265 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:32:45.266 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:32:45.266 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:32:45.266 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:32:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:45 gibba032 bash[24004]: audit 2021-03-10T15:32:44.773984+0000 mgr.gibba032.whopeo (mgr.24625) 5289 : audit [DBG] from='client.47601 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:45 gibba032 bash[24004]: cluster 2021-03-10T15:32:44.780974+0000 mgr.gibba032.whopeo (mgr.24625) 5290 : cluster [DBG] pgmap v3007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:46.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:45 gibba032 bash[24004]: audit 2021-03-10T15:32:44.998123+0000 mgr.gibba032.whopeo (mgr.24625) 5291 : audit [DBG] from='client.47607 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:45 gibba032 bash[24004]: audit 2021-03-10T15:32:45.259902+0000 mon.a (mon.0) 2055 : audit [DBG] from='client.? 172.21.2.111:0/2561693090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[30791]: audit 2021-03-10T15:32:44.773984+0000 mgr.gibba032.whopeo (mgr.24625) 5289 : audit [DBG] from='client.47601 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[30791]: cluster 2021-03-10T15:32:44.780974+0000 mgr.gibba032.whopeo (mgr.24625) 5290 : cluster [DBG] pgmap v3007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:46.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[30791]: audit 2021-03-10T15:32:44.998123+0000 mgr.gibba032.whopeo (mgr.24625) 5291 : audit [DBG] from='client.47607 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[30791]: audit 2021-03-10T15:32:45.259902+0000 mon.a (mon.0) 2055 : audit [DBG] from='client.? 172.21.2.111:0/2561693090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[31774]: audit 2021-03-10T15:32:44.773984+0000 mgr.gibba032.whopeo (mgr.24625) 5289 : audit [DBG] from='client.47601 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[31774]: cluster 2021-03-10T15:32:44.780974+0000 mgr.gibba032.whopeo (mgr.24625) 5290 : cluster [DBG] pgmap v3007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[31774]: audit 2021-03-10T15:32:44.998123+0000 mgr.gibba032.whopeo (mgr.24625) 5291 : audit [DBG] from='client.47607 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:32:46.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:45 gibba011 bash[31774]: audit 2021-03-10T15:32:45.259902+0000 mon.a (mon.0) 2055 : audit [DBG] from='client.? 172.21.2.111:0/2561693090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:32:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:47 gibba032 bash[24004]: cluster 2021-03-10T15:32:46.782496+0000 mgr.gibba032.whopeo (mgr.24625) 5292 : cluster [DBG] pgmap v3008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:47 gibba011 bash[30791]: cluster 2021-03-10T15:32:46.782496+0000 mgr.gibba032.whopeo (mgr.24625) 5292 : cluster [DBG] pgmap v3008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:47 gibba011 bash[31774]: cluster 2021-03-10T15:32:46.782496+0000 mgr.gibba032.whopeo (mgr.24625) 5292 : cluster [DBG] pgmap v3008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:50.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:49 gibba032 bash[24004]: cluster 2021-03-10T15:32:48.783853+0000 mgr.gibba032.whopeo (mgr.24625) 5293 : cluster [DBG] pgmap v3009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:49 gibba011 bash[30791]: cluster 2021-03-10T15:32:48.783853+0000 mgr.gibba032.whopeo (mgr.24625) 5293 : cluster [DBG] pgmap v3009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:50.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:49 gibba011 bash[31774]: cluster 2021-03-10T15:32:48.783853+0000 mgr.gibba032.whopeo (mgr.24625) 5293 : cluster [DBG] pgmap v3009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:51 gibba032 bash[24004]: cluster 2021-03-10T15:32:50.785181+0000 mgr.gibba032.whopeo (mgr.24625) 5294 : cluster [DBG] pgmap v3010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:51 gibba011 bash[30791]: cluster 2021-03-10T15:32:50.785181+0000 mgr.gibba032.whopeo (mgr.24625) 5294 : cluster [DBG] pgmap v3010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:52.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:51 gibba011 bash[31774]: cluster 2021-03-10T15:32:50.785181+0000 mgr.gibba032.whopeo (mgr.24625) 5294 : cluster [DBG] pgmap v3010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:54 gibba032 bash[24004]: cluster 2021-03-10T15:32:52.786809+0000 mgr.gibba032.whopeo (mgr.24625) 5295 : cluster [DBG] pgmap v3011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:54 gibba032 bash[24004]: cephadm 2021-03-10T15:32:53.528911+0000 mgr.gibba032.whopeo (mgr.24625) 5296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:54 gibba011 bash[30791]: cluster 2021-03-10T15:32:52.786809+0000 mgr.gibba032.whopeo (mgr.24625) 5295 : cluster [DBG] pgmap v3011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:54 gibba011 bash[30791]: cephadm 2021-03-10T15:32:53.528911+0000 mgr.gibba032.whopeo (mgr.24625) 5296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:54 gibba011 bash[31774]: cluster 2021-03-10T15:32:52.786809+0000 mgr.gibba032.whopeo (mgr.24625) 5295 : cluster [DBG] pgmap v3011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:54 gibba011 bash[31774]: cephadm 2021-03-10T15:32:53.528911+0000 mgr.gibba032.whopeo (mgr.24625) 5296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:32:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:56 gibba011 bash[30791]: cluster 2021-03-10T15:32:54.788243+0000 mgr.gibba032.whopeo (mgr.24625) 5297 : cluster [DBG] pgmap v3012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:56 gibba011 bash[31774]: cluster 2021-03-10T15:32:54.788243+0000 mgr.gibba032.whopeo (mgr.24625) 5297 : cluster [DBG] pgmap v3012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:56 gibba032 bash[24004]: cluster 2021-03-10T15:32:54.788243+0000 mgr.gibba032.whopeo (mgr.24625) 5297 : cluster [DBG] pgmap v3012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.632874+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:32:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.633515+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:32:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.649595+0000 mon.a (mon.0) 2058 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:32:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.650355+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.651085+0000 mon.a (mon.0) 2060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[30791]: audit 2021-03-10T15:32:56.651871+0000 mon.a (mon.0) 2061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.632874+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:32:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.633515+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:32:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.649595+0000 mon.a (mon.0) 2058 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:32:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.650355+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.651085+0000 mon.a (mon.0) 2060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:57 gibba011 bash[31774]: audit 2021-03-10T15:32:56.651871+0000 mon.a (mon.0) 2061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.632874+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:32:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.633515+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:32:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.649595+0000 mon.a (mon.0) 2058 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T15:32:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.650355+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.651085+0000 mon.a (mon.0) 2060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:32:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:57 gibba032 bash[24004]: audit 2021-03-10T15:32:56.651871+0000 mon.a (mon.0) 2061 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:32:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:32:58 gibba011 bash[30791]: cluster 2021-03-10T15:32:56.789753+0000 mgr.gibba032.whopeo (mgr.24625) 5298 : cluster [DBG] pgmap v3013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:32:58 gibba011 bash[31774]: cluster 2021-03-10T15:32:56.789753+0000 mgr.gibba032.whopeo (mgr.24625) 5298 : cluster [DBG] pgmap v3013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:32:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:32:58 gibba032 bash[24004]: cluster 2021-03-10T15:32:56.789753+0000 mgr.gibba032.whopeo (mgr.24625) 5298 : cluster [DBG] pgmap v3013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:00 gibba011 bash[30791]: cluster 2021-03-10T15:32:58.791275+0000 mgr.gibba032.whopeo (mgr.24625) 5299 : cluster [DBG] pgmap v3014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:00 gibba011 bash[31774]: cluster 2021-03-10T15:32:58.791275+0000 mgr.gibba032.whopeo (mgr.24625) 5299 : cluster [DBG] pgmap v3014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:00 gibba032 bash[24004]: cluster 2021-03-10T15:32:58.791275+0000 mgr.gibba032.whopeo (mgr.24625) 5299 : cluster [DBG] pgmap v3014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:02 gibba011 bash[30791]: cluster 2021-03-10T15:33:00.792787+0000 mgr.gibba032.whopeo (mgr.24625) 5300 : cluster [DBG] pgmap v3015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:02 gibba011 bash[31774]: cluster 2021-03-10T15:33:00.792787+0000 mgr.gibba032.whopeo (mgr.24625) 5300 : cluster [DBG] pgmap v3015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:02 gibba032 bash[24004]: cluster 2021-03-10T15:33:00.792787+0000 mgr.gibba032.whopeo (mgr.24625) 5300 : cluster [DBG] pgmap v3015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:04 gibba011 bash[30791]: cluster 2021-03-10T15:33:02.794341+0000 mgr.gibba032.whopeo (mgr.24625) 5301 : cluster [DBG] pgmap v3016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:04 gibba011 bash[31774]: cluster 2021-03-10T15:33:02.794341+0000 mgr.gibba032.whopeo (mgr.24625) 5301 : cluster [DBG] pgmap v3016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:04 gibba032 bash[24004]: cluster 2021-03-10T15:33:02.794341+0000 mgr.gibba032.whopeo (mgr.24625) 5301 : cluster [DBG] pgmap v3016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:06.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:06 gibba011 bash[30791]: cluster 2021-03-10T15:33:04.795855+0000 mgr.gibba032.whopeo (mgr.24625) 5302 : cluster [DBG] pgmap v3017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:06.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:06 gibba011 bash[31774]: cluster 2021-03-10T15:33:04.795855+0000 mgr.gibba032.whopeo (mgr.24625) 5302 : cluster [DBG] pgmap v3017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:06 gibba032 bash[24004]: cluster 2021-03-10T15:33:04.795855+0000 mgr.gibba032.whopeo (mgr.24625) 5302 : cluster [DBG] pgmap v3017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:08 gibba011 bash[30791]: cluster 2021-03-10T15:33:06.797481+0000 mgr.gibba032.whopeo (mgr.24625) 5303 : cluster [DBG] pgmap v3018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:08 gibba011 bash[31774]: cluster 2021-03-10T15:33:06.797481+0000 mgr.gibba032.whopeo (mgr.24625) 5303 : cluster [DBG] pgmap v3018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:08 gibba032 bash[24004]: cluster 2021-03-10T15:33:06.797481+0000 mgr.gibba032.whopeo (mgr.24625) 5303 : cluster [DBG] pgmap v3018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:10.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.545708+0000 mgr.gibba032.whopeo (mgr.24625) 5304 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:33:10.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.547291+0000 mgr.gibba032.whopeo (mgr.24625) 5305 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:33:10.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.756577+0000 mon.a (mon.0) 2062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:33:10.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.758437+0000 mgr.gibba032.whopeo (mgr.24625) 5306 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:33:10.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.760231+0000 mon.a (mon.0) 2063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:33:10.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.760333+0000 mgr.gibba032.whopeo (mgr.24625) 5307 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:33:10.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.761753+0000 mgr.gibba032.whopeo (mgr.24625) 5308 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:33:10.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.761799+0000 mgr.gibba032.whopeo (mgr.24625) 5309 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:33:10.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.761801+0000 mon.a (mon.0) 2064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.763224+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.763260+0000 mgr.gibba032.whopeo (mgr.24625) 5310 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:33:10.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.763306+0000 mgr.gibba032.whopeo (mgr.24625) 5311 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:33:10.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.764535+0000 mgr.gibba032.whopeo (mgr.24625) 5312 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:33:10.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.764653+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.765743+0000 mgr.gibba032.whopeo (mgr.24625) 5313 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:33:10.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.765788+0000 mgr.gibba032.whopeo (mgr.24625) 5314 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:33:10.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.767270+0000 mgr.gibba032.whopeo (mgr.24625) 5315 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:33:10.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.767347+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.767605+0000 mgr.gibba032.whopeo (mgr.24625) 5316 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:33:10.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: audit 2021-03-10T15:33:08.768967+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.769058+0000 mgr.gibba032.whopeo (mgr.24625) 5317 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:33:10.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.769106+0000 mgr.gibba032.whopeo (mgr.24625) 5318 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:33:10.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cephadm 2021-03-10T15:33:08.769230+0000 mgr.gibba032.whopeo (mgr.24625) 5319 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:10.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[30791]: cluster 2021-03-10T15:33:08.798019+0000 mgr.gibba032.whopeo (mgr.24625) 5320 : cluster [DBG] pgmap v3019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:10.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.545708+0000 mgr.gibba032.whopeo (mgr.24625) 5304 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:33:10.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.547291+0000 mgr.gibba032.whopeo (mgr.24625) 5305 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:33:10.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.756577+0000 mon.a (mon.0) 2062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:33:10.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.758437+0000 mgr.gibba032.whopeo (mgr.24625) 5306 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:33:10.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.760231+0000 mon.a (mon.0) 2063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:33:10.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.760333+0000 mgr.gibba032.whopeo (mgr.24625) 5307 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:33:10.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.761753+0000 mgr.gibba032.whopeo (mgr.24625) 5308 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:33:10.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.761799+0000 mgr.gibba032.whopeo (mgr.24625) 5309 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:33:10.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.761801+0000 mon.a (mon.0) 2064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.763224+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.763260+0000 mgr.gibba032.whopeo (mgr.24625) 5310 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:33:10.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.763306+0000 mgr.gibba032.whopeo (mgr.24625) 5311 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:33:10.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.764535+0000 mgr.gibba032.whopeo (mgr.24625) 5312 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:33:10.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.764653+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.765743+0000 mgr.gibba032.whopeo (mgr.24625) 5313 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:33:10.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.765788+0000 mgr.gibba032.whopeo (mgr.24625) 5314 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:33:10.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.767270+0000 mgr.gibba032.whopeo (mgr.24625) 5315 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:33:10.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.767347+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.767605+0000 mgr.gibba032.whopeo (mgr.24625) 5316 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:33:10.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: audit 2021-03-10T15:33:08.768967+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.769058+0000 mgr.gibba032.whopeo (mgr.24625) 5317 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:33:10.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.769106+0000 mgr.gibba032.whopeo (mgr.24625) 5318 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:33:10.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cephadm 2021-03-10T15:33:08.769230+0000 mgr.gibba032.whopeo (mgr.24625) 5319 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:10.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:09 gibba011 bash[31774]: cluster 2021-03-10T15:33:08.798019+0000 mgr.gibba032.whopeo (mgr.24625) 5320 : cluster [DBG] pgmap v3019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:10.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.545708+0000 mgr.gibba032.whopeo (mgr.24625) 5304 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:33:10.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.547291+0000 mgr.gibba032.whopeo (mgr.24625) 5305 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:33:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.756577+0000 mon.a (mon.0) 2062 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:33:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.758437+0000 mgr.gibba032.whopeo (mgr.24625) 5306 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:33:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.760231+0000 mon.a (mon.0) 2063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:33:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.760333+0000 mgr.gibba032.whopeo (mgr.24625) 5307 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:33:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.761753+0000 mgr.gibba032.whopeo (mgr.24625) 5308 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:33:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.761799+0000 mgr.gibba032.whopeo (mgr.24625) 5309 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:33:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.761801+0000 mon.a (mon.0) 2064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.763224+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.763260+0000 mgr.gibba032.whopeo (mgr.24625) 5310 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:33:10.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.763306+0000 mgr.gibba032.whopeo (mgr.24625) 5311 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:33:10.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.764535+0000 mgr.gibba032.whopeo (mgr.24625) 5312 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:33:10.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.764653+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.765743+0000 mgr.gibba032.whopeo (mgr.24625) 5313 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:33:10.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.765788+0000 mgr.gibba032.whopeo (mgr.24625) 5314 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:33:10.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.767270+0000 mgr.gibba032.whopeo (mgr.24625) 5315 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:33:10.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.767347+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.767605+0000 mgr.gibba032.whopeo (mgr.24625) 5316 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:33:10.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: audit 2021-03-10T15:33:08.768967+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:10.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.769058+0000 mgr.gibba032.whopeo (mgr.24625) 5317 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:33:10.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.769106+0000 mgr.gibba032.whopeo (mgr.24625) 5318 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:33:10.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cephadm 2021-03-10T15:33:08.769230+0000 mgr.gibba032.whopeo (mgr.24625) 5319 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:10.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:09 gibba032 bash[24004]: cluster 2021-03-10T15:33:08.798019+0000 mgr.gibba032.whopeo (mgr.24625) 5320 : cluster [DBG] pgmap v3019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:11 gibba032 bash[24004]: cluster 2021-03-10T15:33:10.799407+0000 mgr.gibba032.whopeo (mgr.24625) 5321 : cluster [DBG] pgmap v3020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:11 gibba011 bash[30791]: cluster 2021-03-10T15:33:10.799407+0000 mgr.gibba032.whopeo (mgr.24625) 5321 : cluster [DBG] pgmap v3020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:12.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:11 gibba011 bash[31774]: cluster 2021-03-10T15:33:10.799407+0000 mgr.gibba032.whopeo (mgr.24625) 5321 : cluster [DBG] pgmap v3020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:13 gibba032 bash[24004]: cluster 2021-03-10T15:33:12.801099+0000 mgr.gibba032.whopeo (mgr.24625) 5322 : cluster [DBG] pgmap v3021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:13 gibba011 bash[30791]: cluster 2021-03-10T15:33:12.801099+0000 mgr.gibba032.whopeo (mgr.24625) 5322 : cluster [DBG] pgmap v3021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:13 gibba011 bash[31774]: cluster 2021-03-10T15:33:12.801099+0000 mgr.gibba032.whopeo (mgr.24625) 5322 : cluster [DBG] pgmap v3021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:15.508 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:33:15.737 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:33:15.738 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (99m) 4m ago 105m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:33:15.738 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (105m) 3m ago 105m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:33:15.738 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (100m) 3m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:33:15.739 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (100m) 3m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:33:15.739 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (99m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:33:15.739 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (99m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:33:15.739 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (98m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:33:15.740 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (99m) 4m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:33:15.740 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (105m) 4m ago 105m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:33:15.740 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (105m) 3m ago 105m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:33:15.740 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (98m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:33:15.740 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (97m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:33:15.741 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (97m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:33:15.741 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (96m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:33:15.741 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (96m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:33:15.741 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (96m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:33:15.742 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (95m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:33:15.742 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (95m) 3m ago 105m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:33:15.742 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (104m) 3m ago 105m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:33:15.743 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 103m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:33:15.991 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:33:15.992 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:33:15.992 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:33:15.992 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:15.993 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:33:15.993 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:33:15.993 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:15.993 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:33:15.994 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:33:15.994 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:15.994 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:33:15.995 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:33:15.995 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:33:15.995 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:33:15.995 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:33:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:15 gibba032 bash[24004]: cluster 2021-03-10T15:33:14.802386+0000 mgr.gibba032.whopeo (mgr.24625) 5323 : cluster [DBG] pgmap v3022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:16.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:15 gibba032 bash[24004]: audit 2021-03-10T15:33:15.494823+0000 mgr.gibba032.whopeo (mgr.24625) 5324 : audit [DBG] from='client.47619 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:15 gibba011 bash[30791]: cluster 2021-03-10T15:33:14.802386+0000 mgr.gibba032.whopeo (mgr.24625) 5323 : cluster [DBG] pgmap v3022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:16.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:15 gibba011 bash[30791]: audit 2021-03-10T15:33:15.494823+0000 mgr.gibba032.whopeo (mgr.24625) 5324 : audit [DBG] from='client.47619 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:15 gibba011 bash[31774]: cluster 2021-03-10T15:33:14.802386+0000 mgr.gibba032.whopeo (mgr.24625) 5323 : cluster [DBG] pgmap v3022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:15 gibba011 bash[31774]: audit 2021-03-10T15:33:15.494823+0000 mgr.gibba032.whopeo (mgr.24625) 5324 : audit [DBG] from='client.47619 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:16 gibba032 bash[24004]: audit 2021-03-10T15:33:15.720752+0000 mgr.gibba032.whopeo (mgr.24625) 5325 : audit [DBG] from='client.57610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:17.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:16 gibba032 bash[24004]: audit 2021-03-10T15:33:15.988363+0000 mon.c (mon.1) 100 : audit [DBG] from='client.? 172.21.2.111:0/2184187535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:16 gibba011 bash[30791]: audit 2021-03-10T15:33:15.720752+0000 mgr.gibba032.whopeo (mgr.24625) 5325 : audit [DBG] from='client.57610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:17.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:16 gibba011 bash[30791]: audit 2021-03-10T15:33:15.988363+0000 mon.c (mon.1) 100 : audit [DBG] from='client.? 172.21.2.111:0/2184187535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:16 gibba011 bash[31774]: audit 2021-03-10T15:33:15.720752+0000 mgr.gibba032.whopeo (mgr.24625) 5325 : audit [DBG] from='client.57610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:17.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:16 gibba011 bash[31774]: audit 2021-03-10T15:33:15.988363+0000 mon.c (mon.1) 100 : audit [DBG] from='client.? 172.21.2.111:0/2184187535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:18.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:17 gibba032 bash[24004]: cluster 2021-03-10T15:33:16.803912+0000 mgr.gibba032.whopeo (mgr.24625) 5326 : cluster [DBG] pgmap v3023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:17 gibba011 bash[30791]: cluster 2021-03-10T15:33:16.803912+0000 mgr.gibba032.whopeo (mgr.24625) 5326 : cluster [DBG] pgmap v3023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:17 gibba011 bash[31774]: cluster 2021-03-10T15:33:16.803912+0000 mgr.gibba032.whopeo (mgr.24625) 5326 : cluster [DBG] pgmap v3023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:20.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:19 gibba032 bash[24004]: cluster 2021-03-10T15:33:18.805265+0000 mgr.gibba032.whopeo (mgr.24625) 5327 : cluster [DBG] pgmap v3024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:19 gibba011 bash[30791]: cluster 2021-03-10T15:33:18.805265+0000 mgr.gibba032.whopeo (mgr.24625) 5327 : cluster [DBG] pgmap v3024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:20.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:19 gibba011 bash[31774]: cluster 2021-03-10T15:33:18.805265+0000 mgr.gibba032.whopeo (mgr.24625) 5327 : cluster [DBG] pgmap v3024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:21 gibba032 bash[24004]: cluster 2021-03-10T15:33:20.806661+0000 mgr.gibba032.whopeo (mgr.24625) 5328 : cluster [DBG] pgmap v3025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:21 gibba011 bash[30791]: cluster 2021-03-10T15:33:20.806661+0000 mgr.gibba032.whopeo (mgr.24625) 5328 : cluster [DBG] pgmap v3025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:22.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:21 gibba011 bash[31774]: cluster 2021-03-10T15:33:20.806661+0000 mgr.gibba032.whopeo (mgr.24625) 5328 : cluster [DBG] pgmap v3025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:23 gibba032 bash[24004]: cluster 2021-03-10T15:33:22.808387+0000 mgr.gibba032.whopeo (mgr.24625) 5329 : cluster [DBG] pgmap v3026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:23 gibba011 bash[30791]: cluster 2021-03-10T15:33:22.808387+0000 mgr.gibba032.whopeo (mgr.24625) 5329 : cluster [DBG] pgmap v3026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:23 gibba011 bash[31774]: cluster 2021-03-10T15:33:22.808387+0000 mgr.gibba032.whopeo (mgr.24625) 5329 : cluster [DBG] pgmap v3026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:24 gibba032 bash[24004]: cephadm 2021-03-10T15:33:23.784971+0000 mgr.gibba032.whopeo (mgr.24625) 5330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:24 gibba011 bash[30791]: cephadm 2021-03-10T15:33:23.784971+0000 mgr.gibba032.whopeo (mgr.24625) 5330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:24 gibba011 bash[31774]: cephadm 2021-03-10T15:33:23.784971+0000 mgr.gibba032.whopeo (mgr.24625) 5330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:26.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:25 gibba032 bash[24004]: cluster 2021-03-10T15:33:24.809642+0000 mgr.gibba032.whopeo (mgr.24625) 5331 : cluster [DBG] pgmap v3027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:25 gibba011 bash[30791]: cluster 2021-03-10T15:33:24.809642+0000 mgr.gibba032.whopeo (mgr.24625) 5331 : cluster [DBG] pgmap v3027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:25 gibba011 bash[31774]: cluster 2021-03-10T15:33:24.809642+0000 mgr.gibba032.whopeo (mgr.24625) 5331 : cluster [DBG] pgmap v3027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:28.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:27 gibba032 bash[24004]: cluster 2021-03-10T15:33:26.811146+0000 mgr.gibba032.whopeo (mgr.24625) 5332 : cluster [DBG] pgmap v3028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:27 gibba011 bash[30791]: cluster 2021-03-10T15:33:26.811146+0000 mgr.gibba032.whopeo (mgr.24625) 5332 : cluster [DBG] pgmap v3028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:28.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:27 gibba011 bash[31774]: cluster 2021-03-10T15:33:26.811146+0000 mgr.gibba032.whopeo (mgr.24625) 5332 : cluster [DBG] pgmap v3028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:30.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:29 gibba032 bash[24004]: cluster 2021-03-10T15:33:28.812653+0000 mgr.gibba032.whopeo (mgr.24625) 5333 : cluster [DBG] pgmap v3029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:29 gibba011 bash[30791]: cluster 2021-03-10T15:33:28.812653+0000 mgr.gibba032.whopeo (mgr.24625) 5333 : cluster [DBG] pgmap v3029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:29 gibba011 bash[31774]: cluster 2021-03-10T15:33:28.812653+0000 mgr.gibba032.whopeo (mgr.24625) 5333 : cluster [DBG] pgmap v3029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:32.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:31 gibba032 bash[24004]: cluster 2021-03-10T15:33:30.814010+0000 mgr.gibba032.whopeo (mgr.24625) 5334 : cluster [DBG] pgmap v3030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:31 gibba011 bash[30791]: cluster 2021-03-10T15:33:30.814010+0000 mgr.gibba032.whopeo (mgr.24625) 5334 : cluster [DBG] pgmap v3030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:31 gibba011 bash[31774]: cluster 2021-03-10T15:33:30.814010+0000 mgr.gibba032.whopeo (mgr.24625) 5334 : cluster [DBG] pgmap v3030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:33 gibba032 bash[24004]: cluster 2021-03-10T15:33:32.815676+0000 mgr.gibba032.whopeo (mgr.24625) 5335 : cluster [DBG] pgmap v3031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:33 gibba011 bash[30791]: cluster 2021-03-10T15:33:32.815676+0000 mgr.gibba032.whopeo (mgr.24625) 5335 : cluster [DBG] pgmap v3031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:34.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:33 gibba011 bash[31774]: cluster 2021-03-10T15:33:32.815676+0000 mgr.gibba032.whopeo (mgr.24625) 5335 : cluster [DBG] pgmap v3031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:36.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:35 gibba032 bash[24004]: cluster 2021-03-10T15:33:34.817274+0000 mgr.gibba032.whopeo (mgr.24625) 5336 : cluster [DBG] pgmap v3032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:35 gibba011 bash[30791]: cluster 2021-03-10T15:33:34.817274+0000 mgr.gibba032.whopeo (mgr.24625) 5336 : cluster [DBG] pgmap v3032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:36.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:35 gibba011 bash[31774]: cluster 2021-03-10T15:33:34.817274+0000 mgr.gibba032.whopeo (mgr.24625) 5336 : cluster [DBG] pgmap v3032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:38.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:37 gibba032 bash[24004]: cluster 2021-03-10T15:33:36.818969+0000 mgr.gibba032.whopeo (mgr.24625) 5337 : cluster [DBG] pgmap v3033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:37 gibba011 bash[30791]: cluster 2021-03-10T15:33:36.818969+0000 mgr.gibba032.whopeo (mgr.24625) 5337 : cluster [DBG] pgmap v3033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:38.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:37 gibba011 bash[31774]: cluster 2021-03-10T15:33:36.818969+0000 mgr.gibba032.whopeo (mgr.24625) 5337 : cluster [DBG] pgmap v3033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:40 gibba032 bash[24004]: cephadm 2021-03-10T15:33:38.794060+0000 mgr.gibba032.whopeo (mgr.24625) 5338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:40 gibba032 bash[24004]: cluster 2021-03-10T15:33:38.820665+0000 mgr.gibba032.whopeo (mgr.24625) 5339 : cluster [DBG] pgmap v3034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:40.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:40 gibba011 bash[30791]: cephadm 2021-03-10T15:33:38.794060+0000 mgr.gibba032.whopeo (mgr.24625) 5338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:40.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:40 gibba011 bash[30791]: cluster 2021-03-10T15:33:38.820665+0000 mgr.gibba032.whopeo (mgr.24625) 5339 : cluster [DBG] pgmap v3034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:40 gibba011 bash[31774]: cephadm 2021-03-10T15:33:38.794060+0000 mgr.gibba032.whopeo (mgr.24625) 5338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:40.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:40 gibba011 bash[31774]: cluster 2021-03-10T15:33:38.820665+0000 mgr.gibba032.whopeo (mgr.24625) 5339 : cluster [DBG] pgmap v3034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:42 gibba011 bash[30791]: cluster 2021-03-10T15:33:40.822050+0000 mgr.gibba032.whopeo (mgr.24625) 5340 : cluster [DBG] pgmap v3035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:42 gibba011 bash[31774]: cluster 2021-03-10T15:33:40.822050+0000 mgr.gibba032.whopeo (mgr.24625) 5340 : cluster [DBG] pgmap v3035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:42 gibba032 bash[24004]: cluster 2021-03-10T15:33:40.822050+0000 mgr.gibba032.whopeo (mgr.24625) 5340 : cluster [DBG] pgmap v3035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:44 gibba011 bash[30791]: cluster 2021-03-10T15:33:42.823741+0000 mgr.gibba032.whopeo (mgr.24625) 5341 : cluster [DBG] pgmap v3036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:44 gibba011 bash[31774]: cluster 2021-03-10T15:33:42.823741+0000 mgr.gibba032.whopeo (mgr.24625) 5341 : cluster [DBG] pgmap v3036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:44 gibba032 bash[24004]: cluster 2021-03-10T15:33:42.823741+0000 mgr.gibba032.whopeo (mgr.24625) 5341 : cluster [DBG] pgmap v3036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:46.239 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:33:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:46 gibba011 bash[30791]: cluster 2021-03-10T15:33:44.825137+0000 mgr.gibba032.whopeo (mgr.24625) 5342 : cluster [DBG] pgmap v3037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:46.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:46 gibba011 bash[31774]: cluster 2021-03-10T15:33:44.825137+0000 mgr.gibba032.whopeo (mgr.24625) 5342 : cluster [DBG] pgmap v3037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:46.498 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:33:46.498 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (100m) 4m ago 105m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:33:46.498 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (105m) 3m ago 105m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:33:46.498 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (100m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:33:46.499 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (101m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:33:46.499 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (100m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:33:46.499 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (99m) 4m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:33:46.499 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (99m) 3m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:33:46.500 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (99m) 4m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:33:46.500 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (105m) 4m ago 105m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:33:46.500 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (105m) 3m ago 105m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:33:46.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (98m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:33:46.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (98m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:33:46.501 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (97m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:33:46.501 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (97m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:33:46.501 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (97m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:33:46.501 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (96m) 3m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:33:46.501 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (96m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:33:46.502 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (95m) 3m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:33:46.502 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (105m) 3m ago 105m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:33:46.502 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 103m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:33:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:46 gibba032 bash[24004]: cluster 2021-03-10T15:33:44.825137+0000 mgr.gibba032.whopeo (mgr.24625) 5342 : cluster [DBG] pgmap v3037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:46.758 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:33:46.759 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:33:46.759 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:33:46.759 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:46.759 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:33:46.759 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:33:46.760 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:46.760 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:33:46.760 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:33:46.760 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:33:46.761 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:33:46.761 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:33:46.761 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:33:46.761 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:33:46.761 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:33:47.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[30791]: audit 2021-03-10T15:33:46.225955+0000 mgr.gibba032.whopeo (mgr.24625) 5343 : audit [DBG] from='client.57622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[30791]: audit 2021-03-10T15:33:46.453836+0000 mgr.gibba032.whopeo (mgr.24625) 5344 : audit [DBG] from='client.47643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[30791]: audit 2021-03-10T15:33:46.755243+0000 mon.c (mon.1) 101 : audit [DBG] from='client.? 172.21.2.111:0/3010861345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[31774]: audit 2021-03-10T15:33:46.225955+0000 mgr.gibba032.whopeo (mgr.24625) 5343 : audit [DBG] from='client.57622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[31774]: audit 2021-03-10T15:33:46.453836+0000 mgr.gibba032.whopeo (mgr.24625) 5344 : audit [DBG] from='client.47643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:47 gibba011 bash[31774]: audit 2021-03-10T15:33:46.755243+0000 mon.c (mon.1) 101 : audit [DBG] from='client.? 172.21.2.111:0/3010861345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:47 gibba032 bash[24004]: audit 2021-03-10T15:33:46.225955+0000 mgr.gibba032.whopeo (mgr.24625) 5343 : audit [DBG] from='client.57622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:47 gibba032 bash[24004]: audit 2021-03-10T15:33:46.453836+0000 mgr.gibba032.whopeo (mgr.24625) 5344 : audit [DBG] from='client.47643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:33:47.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:47 gibba032 bash[24004]: audit 2021-03-10T15:33:46.755243+0000 mon.c (mon.1) 101 : audit [DBG] from='client.? 172.21.2.111:0/3010861345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:33:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:48 gibba011 bash[30791]: cluster 2021-03-10T15:33:46.826866+0000 mgr.gibba032.whopeo (mgr.24625) 5345 : cluster [DBG] pgmap v3038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:48 gibba011 bash[31774]: cluster 2021-03-10T15:33:46.826866+0000 mgr.gibba032.whopeo (mgr.24625) 5345 : cluster [DBG] pgmap v3038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:48 gibba032 bash[24004]: cluster 2021-03-10T15:33:46.826866+0000 mgr.gibba032.whopeo (mgr.24625) 5345 : cluster [DBG] pgmap v3038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:50 gibba011 bash[30791]: cluster 2021-03-10T15:33:48.828437+0000 mgr.gibba032.whopeo (mgr.24625) 5346 : cluster [DBG] pgmap v3039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:50.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:50 gibba011 bash[31774]: cluster 2021-03-10T15:33:48.828437+0000 mgr.gibba032.whopeo (mgr.24625) 5346 : cluster [DBG] pgmap v3039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:50 gibba032 bash[24004]: cluster 2021-03-10T15:33:48.828437+0000 mgr.gibba032.whopeo (mgr.24625) 5346 : cluster [DBG] pgmap v3039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:52 gibba011 bash[30791]: cluster 2021-03-10T15:33:50.829803+0000 mgr.gibba032.whopeo (mgr.24625) 5347 : cluster [DBG] pgmap v3040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:52.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:52 gibba011 bash[31774]: cluster 2021-03-10T15:33:50.829803+0000 mgr.gibba032.whopeo (mgr.24625) 5347 : cluster [DBG] pgmap v3040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:52 gibba032 bash[24004]: cluster 2021-03-10T15:33:50.829803+0000 mgr.gibba032.whopeo (mgr.24625) 5347 : cluster [DBG] pgmap v3040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:54 gibba011 bash[30791]: cluster 2021-03-10T15:33:52.831501+0000 mgr.gibba032.whopeo (mgr.24625) 5348 : cluster [DBG] pgmap v3041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:54 gibba011 bash[31774]: cluster 2021-03-10T15:33:52.831501+0000 mgr.gibba032.whopeo (mgr.24625) 5348 : cluster [DBG] pgmap v3041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:54 gibba032 bash[24004]: cluster 2021-03-10T15:33:52.831501+0000 mgr.gibba032.whopeo (mgr.24625) 5348 : cluster [DBG] pgmap v3041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:55 gibba011 bash[30791]: cephadm 2021-03-10T15:33:53.803329+0000 mgr.gibba032.whopeo (mgr.24625) 5349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:55 gibba011 bash[31774]: cephadm 2021-03-10T15:33:53.803329+0000 mgr.gibba032.whopeo (mgr.24625) 5349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:55 gibba032 bash[24004]: cephadm 2021-03-10T15:33:53.803329+0000 mgr.gibba032.whopeo (mgr.24625) 5349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:33:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:56 gibba032 bash[24004]: cluster 2021-03-10T15:33:54.833051+0000 mgr.gibba032.whopeo (mgr.24625) 5350 : cluster [DBG] pgmap v3042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:56 gibba011 bash[30791]: cluster 2021-03-10T15:33:54.833051+0000 mgr.gibba032.whopeo (mgr.24625) 5350 : cluster [DBG] pgmap v3042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:56 gibba011 bash[31774]: cluster 2021-03-10T15:33:54.833051+0000 mgr.gibba032.whopeo (mgr.24625) 5350 : cluster [DBG] pgmap v3042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.638884+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:33:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.639594+0000 mon.a (mon.0) 2070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:33:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.655702+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T15:33:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.656445+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:33:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.657136+0000 mon.a (mon.0) 2073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:33:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:57 gibba032 bash[24004]: audit 2021-03-10T15:33:56.657824+0000 mon.a (mon.0) 2074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:33:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.638884+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:33:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.639594+0000 mon.a (mon.0) 2070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:33:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.655702+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T15:33:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.656445+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:33:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.657136+0000 mon.a (mon.0) 2073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:33:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[30791]: audit 2021-03-10T15:33:56.657824+0000 mon.a (mon.0) 2074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:33:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.638884+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:33:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.639594+0000 mon.a (mon.0) 2070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:33:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.655702+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T15:33:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.656445+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:33:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.657136+0000 mon.a (mon.0) 2073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:33:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:57 gibba011 bash[31774]: audit 2021-03-10T15:33:56.657824+0000 mon.a (mon.0) 2074 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:33:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:33:58 gibba032 bash[24004]: cluster 2021-03-10T15:33:56.834947+0000 mgr.gibba032.whopeo (mgr.24625) 5351 : cluster [DBG] pgmap v3043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:33:58 gibba011 bash[30791]: cluster 2021-03-10T15:33:56.834947+0000 mgr.gibba032.whopeo (mgr.24625) 5351 : cluster [DBG] pgmap v3043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:33:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:33:58 gibba011 bash[31774]: cluster 2021-03-10T15:33:56.834947+0000 mgr.gibba032.whopeo (mgr.24625) 5351 : cluster [DBG] pgmap v3043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:00 gibba032 bash[24004]: cluster 2021-03-10T15:33:58.836524+0000 mgr.gibba032.whopeo (mgr.24625) 5352 : cluster [DBG] pgmap v3044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:00 gibba011 bash[30791]: cluster 2021-03-10T15:33:58.836524+0000 mgr.gibba032.whopeo (mgr.24625) 5352 : cluster [DBG] pgmap v3044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:00 gibba011 bash[31774]: cluster 2021-03-10T15:33:58.836524+0000 mgr.gibba032.whopeo (mgr.24625) 5352 : cluster [DBG] pgmap v3044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:02 gibba032 bash[24004]: cluster 2021-03-10T15:34:00.837986+0000 mgr.gibba032.whopeo (mgr.24625) 5353 : cluster [DBG] pgmap v3045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:02 gibba011 bash[30791]: cluster 2021-03-10T15:34:00.837986+0000 mgr.gibba032.whopeo (mgr.24625) 5353 : cluster [DBG] pgmap v3045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:02 gibba011 bash[31774]: cluster 2021-03-10T15:34:00.837986+0000 mgr.gibba032.whopeo (mgr.24625) 5353 : cluster [DBG] pgmap v3045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:04 gibba032 bash[24004]: cluster 2021-03-10T15:34:02.839769+0000 mgr.gibba032.whopeo (mgr.24625) 5354 : cluster [DBG] pgmap v3046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:04 gibba011 bash[30791]: cluster 2021-03-10T15:34:02.839769+0000 mgr.gibba032.whopeo (mgr.24625) 5354 : cluster [DBG] pgmap v3046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:04 gibba011 bash[31774]: cluster 2021-03-10T15:34:02.839769+0000 mgr.gibba032.whopeo (mgr.24625) 5354 : cluster [DBG] pgmap v3046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:06 gibba032 bash[24004]: cluster 2021-03-10T15:34:04.841329+0000 mgr.gibba032.whopeo (mgr.24625) 5355 : cluster [DBG] pgmap v3047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:06 gibba011 bash[30791]: cluster 2021-03-10T15:34:04.841329+0000 mgr.gibba032.whopeo (mgr.24625) 5355 : cluster [DBG] pgmap v3047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:06 gibba011 bash[31774]: cluster 2021-03-10T15:34:04.841329+0000 mgr.gibba032.whopeo (mgr.24625) 5355 : cluster [DBG] pgmap v3047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:08 gibba032 bash[24004]: cluster 2021-03-10T15:34:06.843152+0000 mgr.gibba032.whopeo (mgr.24625) 5356 : cluster [DBG] pgmap v3048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:08 gibba011 bash[30791]: cluster 2021-03-10T15:34:06.843152+0000 mgr.gibba032.whopeo (mgr.24625) 5356 : cluster [DBG] pgmap v3048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:08 gibba011 bash[31774]: cluster 2021-03-10T15:34:06.843152+0000 mgr.gibba032.whopeo (mgr.24625) 5356 : cluster [DBG] pgmap v3048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:08.809740+0000 mgr.gibba032.whopeo (mgr.24625) 5357 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:34:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:08.811308+0000 mgr.gibba032.whopeo (mgr.24625) 5358 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:34:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cluster 2021-03-10T15:34:08.844502+0000 mgr.gibba032.whopeo (mgr.24625) 5359 : cluster [DBG] pgmap v3049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.074067+0000 mon.a (mon.0) 2075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:34:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.075482+0000 mgr.gibba032.whopeo (mgr.24625) 5360 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:34:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.077278+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:34:10.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.077601+0000 mgr.gibba032.whopeo (mgr.24625) 5361 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:34:10.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.078897+0000 mgr.gibba032.whopeo (mgr.24625) 5362 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:34:10.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.078935+0000 mgr.gibba032.whopeo (mgr.24625) 5363 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:34:10.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.079003+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.080273+0000 mgr.gibba032.whopeo (mgr.24625) 5364 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:34:10.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.080311+0000 mgr.gibba032.whopeo (mgr.24625) 5365 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:34:10.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.080326+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.081621+0000 mgr.gibba032.whopeo (mgr.24625) 5366 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:34:10.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.081645+0000 mon.a (mon.0) 2079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.082897+0000 mgr.gibba032.whopeo (mgr.24625) 5367 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:34:10.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.082935+0000 mgr.gibba032.whopeo (mgr.24625) 5368 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:34:10.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.084387+0000 mgr.gibba032.whopeo (mgr.24625) 5369 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:34:10.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.084418+0000 mon.a (mon.0) 2080 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.084740+0000 mgr.gibba032.whopeo (mgr.24625) 5370 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:34:10.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:08.809740+0000 mgr.gibba032.whopeo (mgr.24625) 5357 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:34:10.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:08.811308+0000 mgr.gibba032.whopeo (mgr.24625) 5358 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:34:10.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cluster 2021-03-10T15:34:08.844502+0000 mgr.gibba032.whopeo (mgr.24625) 5359 : cluster [DBG] pgmap v3049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:10.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.074067+0000 mon.a (mon.0) 2075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:34:10.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.075482+0000 mgr.gibba032.whopeo (mgr.24625) 5360 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:34:10.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.077278+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:34:10.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.077601+0000 mgr.gibba032.whopeo (mgr.24625) 5361 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:34:10.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.078897+0000 mgr.gibba032.whopeo (mgr.24625) 5362 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:34:10.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.078935+0000 mgr.gibba032.whopeo (mgr.24625) 5363 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:34:10.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.079003+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.080273+0000 mgr.gibba032.whopeo (mgr.24625) 5364 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:34:10.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.080311+0000 mgr.gibba032.whopeo (mgr.24625) 5365 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:34:10.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.080326+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.081621+0000 mgr.gibba032.whopeo (mgr.24625) 5366 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:34:10.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.081645+0000 mon.a (mon.0) 2079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.082897+0000 mgr.gibba032.whopeo (mgr.24625) 5367 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:34:10.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.082935+0000 mgr.gibba032.whopeo (mgr.24625) 5368 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:34:10.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.084387+0000 mgr.gibba032.whopeo (mgr.24625) 5369 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:34:10.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.084418+0000 mon.a (mon.0) 2080 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.084740+0000 mgr.gibba032.whopeo (mgr.24625) 5370 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:34:10.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: audit 2021-03-10T15:34:09.086098+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.086323+0000 mgr.gibba032.whopeo (mgr.24625) 5371 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:34:10.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.086365+0000 mgr.gibba032.whopeo (mgr.24625) 5372 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:34:10.343 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[30791]: cephadm 2021-03-10T15:34:09.086478+0000 mgr.gibba032.whopeo (mgr.24625) 5373 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:10.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: audit 2021-03-10T15:34:09.086098+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.086323+0000 mgr.gibba032.whopeo (mgr.24625) 5371 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:34:10.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.086365+0000 mgr.gibba032.whopeo (mgr.24625) 5372 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:34:10.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:10 gibba011 bash[31774]: cephadm 2021-03-10T15:34:09.086478+0000 mgr.gibba032.whopeo (mgr.24625) 5373 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:08.809740+0000 mgr.gibba032.whopeo (mgr.24625) 5357 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:34:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:08.811308+0000 mgr.gibba032.whopeo (mgr.24625) 5358 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:34:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cluster 2021-03-10T15:34:08.844502+0000 mgr.gibba032.whopeo (mgr.24625) 5359 : cluster [DBG] pgmap v3049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.074067+0000 mon.a (mon.0) 2075 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:34:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.075482+0000 mgr.gibba032.whopeo (mgr.24625) 5360 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:34:10.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.077278+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:34:10.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.077601+0000 mgr.gibba032.whopeo (mgr.24625) 5361 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:34:10.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.078897+0000 mgr.gibba032.whopeo (mgr.24625) 5362 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:34:10.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.078935+0000 mgr.gibba032.whopeo (mgr.24625) 5363 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:34:10.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.079003+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.080273+0000 mgr.gibba032.whopeo (mgr.24625) 5364 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:34:10.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.080311+0000 mgr.gibba032.whopeo (mgr.24625) 5365 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:34:10.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.080326+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.081621+0000 mgr.gibba032.whopeo (mgr.24625) 5366 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:34:10.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.081645+0000 mon.a (mon.0) 2079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.082897+0000 mgr.gibba032.whopeo (mgr.24625) 5367 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:34:10.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.082935+0000 mgr.gibba032.whopeo (mgr.24625) 5368 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:34:10.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.084387+0000 mgr.gibba032.whopeo (mgr.24625) 5369 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:34:10.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.084418+0000 mon.a (mon.0) 2080 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.084740+0000 mgr.gibba032.whopeo (mgr.24625) 5370 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:34:10.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: audit 2021-03-10T15:34:09.086098+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:10.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.086323+0000 mgr.gibba032.whopeo (mgr.24625) 5371 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:34:10.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.086365+0000 mgr.gibba032.whopeo (mgr.24625) 5372 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:34:10.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:10 gibba032 bash[24004]: cephadm 2021-03-10T15:34:09.086478+0000 mgr.gibba032.whopeo (mgr.24625) 5373 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:12 gibba032 bash[24004]: cluster 2021-03-10T15:34:10.845867+0000 mgr.gibba032.whopeo (mgr.24625) 5374 : cluster [DBG] pgmap v3050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:12 gibba011 bash[30791]: cluster 2021-03-10T15:34:10.845867+0000 mgr.gibba032.whopeo (mgr.24625) 5374 : cluster [DBG] pgmap v3050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:12 gibba011 bash[31774]: cluster 2021-03-10T15:34:10.845867+0000 mgr.gibba032.whopeo (mgr.24625) 5374 : cluster [DBG] pgmap v3050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:14 gibba032 bash[24004]: cluster 2021-03-10T15:34:12.847362+0000 mgr.gibba032.whopeo (mgr.24625) 5375 : cluster [DBG] pgmap v3051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:14 gibba011 bash[30791]: cluster 2021-03-10T15:34:12.847362+0000 mgr.gibba032.whopeo (mgr.24625) 5375 : cluster [DBG] pgmap v3051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:14 gibba011 bash[31774]: cluster 2021-03-10T15:34:12.847362+0000 mgr.gibba032.whopeo (mgr.24625) 5375 : cluster [DBG] pgmap v3051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:16 gibba032 bash[24004]: cluster 2021-03-10T15:34:14.848946+0000 mgr.gibba032.whopeo (mgr.24625) 5376 : cluster [DBG] pgmap v3052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:16 gibba011 bash[30791]: cluster 2021-03-10T15:34:14.848946+0000 mgr.gibba032.whopeo (mgr.24625) 5376 : cluster [DBG] pgmap v3052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:16 gibba011 bash[31774]: cluster 2021-03-10T15:34:14.848946+0000 mgr.gibba032.whopeo (mgr.24625) 5376 : cluster [DBG] pgmap v3052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:17.003 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:34:17.234 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:34:17.235 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (100m) 5m ago 106m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:34:17.235 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (106m) 4m ago 106m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:34:17.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (101m) 4m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:34:17.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (101m) 4m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:34:17.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (100m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:34:17.236 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (100m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:34:17.236 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (100m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:34:17.236 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (100m) 5m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:34:17.236 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (106m) 5m ago 106m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:34:17.237 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (106m) 4m ago 106m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:34:17.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (99m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:34:17.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (98m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:34:17.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (98m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:34:17.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (97m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:34:17.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (97m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:34:17.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (97m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:34:17.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (96m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:34:17.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (96m) 4m ago 106m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:34:17.239 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (105m) 4m ago 106m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:34:17.239 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 104m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:34:17.484 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:34:17.485 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:34:17.485 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:34:17.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:17.485 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:34:17.486 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:34:17.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:17.486 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:34:17.486 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:34:17.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:17.487 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:34:17.487 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:34:17.487 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:34:17.487 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:34:17.488 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:34:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:18 gibba032 bash[24004]: cluster 2021-03-10T15:34:16.850449+0000 mgr.gibba032.whopeo (mgr.24625) 5377 : cluster [DBG] pgmap v3053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:18 gibba032 bash[24004]: audit 2021-03-10T15:34:16.990356+0000 mgr.gibba032.whopeo (mgr.24625) 5378 : audit [DBG] from='client.57640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:18 gibba032 bash[24004]: audit 2021-03-10T15:34:17.218065+0000 mgr.gibba032.whopeo (mgr.24625) 5379 : audit [DBG] from='client.57646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:18 gibba032 bash[24004]: audit 2021-03-10T15:34:17.481281+0000 mon.c (mon.1) 102 : audit [DBG] from='client.? 172.21.2.111:0/3732478843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[30791]: cluster 2021-03-10T15:34:16.850449+0000 mgr.gibba032.whopeo (mgr.24625) 5377 : cluster [DBG] pgmap v3053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:18.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[30791]: audit 2021-03-10T15:34:16.990356+0000 mgr.gibba032.whopeo (mgr.24625) 5378 : audit [DBG] from='client.57640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[30791]: audit 2021-03-10T15:34:17.218065+0000 mgr.gibba032.whopeo (mgr.24625) 5379 : audit [DBG] from='client.57646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[30791]: audit 2021-03-10T15:34:17.481281+0000 mon.c (mon.1) 102 : audit [DBG] from='client.? 172.21.2.111:0/3732478843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:18.619 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[31774]: cluster 2021-03-10T15:34:16.850449+0000 mgr.gibba032.whopeo (mgr.24625) 5377 : cluster [DBG] pgmap v3053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:18.620 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[31774]: audit 2021-03-10T15:34:16.990356+0000 mgr.gibba032.whopeo (mgr.24625) 5378 : audit [DBG] from='client.57640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.620 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[31774]: audit 2021-03-10T15:34:17.218065+0000 mgr.gibba032.whopeo (mgr.24625) 5379 : audit [DBG] from='client.57646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:18.620 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:18 gibba011 bash[31774]: audit 2021-03-10T15:34:17.481281+0000 mon.c (mon.1) 102 : audit [DBG] from='client.? 172.21.2.111:0/3732478843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:20 gibba032 bash[24004]: cluster 2021-03-10T15:34:18.851953+0000 mgr.gibba032.whopeo (mgr.24625) 5380 : cluster [DBG] pgmap v3054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:20 gibba011 bash[30791]: cluster 2021-03-10T15:34:18.851953+0000 mgr.gibba032.whopeo (mgr.24625) 5380 : cluster [DBG] pgmap v3054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:20 gibba011 bash[31774]: cluster 2021-03-10T15:34:18.851953+0000 mgr.gibba032.whopeo (mgr.24625) 5380 : cluster [DBG] pgmap v3054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:22 gibba032 bash[24004]: cluster 2021-03-10T15:34:20.853338+0000 mgr.gibba032.whopeo (mgr.24625) 5381 : cluster [DBG] pgmap v3055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:22 gibba011 bash[30791]: cluster 2021-03-10T15:34:20.853338+0000 mgr.gibba032.whopeo (mgr.24625) 5381 : cluster [DBG] pgmap v3055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:22 gibba011 bash[31774]: cluster 2021-03-10T15:34:20.853338+0000 mgr.gibba032.whopeo (mgr.24625) 5381 : cluster [DBG] pgmap v3055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:24 gibba032 bash[24004]: cluster 2021-03-10T15:34:22.854996+0000 mgr.gibba032.whopeo (mgr.24625) 5382 : cluster [DBG] pgmap v3056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:24 gibba011 bash[30791]: cluster 2021-03-10T15:34:22.854996+0000 mgr.gibba032.whopeo (mgr.24625) 5382 : cluster [DBG] pgmap v3056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:24 gibba011 bash[31774]: cluster 2021-03-10T15:34:22.854996+0000 mgr.gibba032.whopeo (mgr.24625) 5382 : cluster [DBG] pgmap v3056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:25 gibba032 bash[24004]: cephadm 2021-03-10T15:34:24.102366+0000 mgr.gibba032.whopeo (mgr.24625) 5383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:25.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:25 gibba011 bash[30791]: cephadm 2021-03-10T15:34:24.102366+0000 mgr.gibba032.whopeo (mgr.24625) 5383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:25.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:25 gibba011 bash[31774]: cephadm 2021-03-10T15:34:24.102366+0000 mgr.gibba032.whopeo (mgr.24625) 5383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:26 gibba032 bash[24004]: cluster 2021-03-10T15:34:24.856549+0000 mgr.gibba032.whopeo (mgr.24625) 5384 : cluster [DBG] pgmap v3057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:26 gibba011 bash[30791]: cluster 2021-03-10T15:34:24.856549+0000 mgr.gibba032.whopeo (mgr.24625) 5384 : cluster [DBG] pgmap v3057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:26 gibba011 bash[31774]: cluster 2021-03-10T15:34:24.856549+0000 mgr.gibba032.whopeo (mgr.24625) 5384 : cluster [DBG] pgmap v3057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:28 gibba032 bash[24004]: cluster 2021-03-10T15:34:26.858182+0000 mgr.gibba032.whopeo (mgr.24625) 5385 : cluster [DBG] pgmap v3058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:28 gibba011 bash[30791]: cluster 2021-03-10T15:34:26.858182+0000 mgr.gibba032.whopeo (mgr.24625) 5385 : cluster [DBG] pgmap v3058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:28 gibba011 bash[31774]: cluster 2021-03-10T15:34:26.858182+0000 mgr.gibba032.whopeo (mgr.24625) 5385 : cluster [DBG] pgmap v3058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:30 gibba032 bash[24004]: cluster 2021-03-10T15:34:28.859760+0000 mgr.gibba032.whopeo (mgr.24625) 5386 : cluster [DBG] pgmap v3059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:30 gibba011 bash[30791]: cluster 2021-03-10T15:34:28.859760+0000 mgr.gibba032.whopeo (mgr.24625) 5386 : cluster [DBG] pgmap v3059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:30 gibba011 bash[31774]: cluster 2021-03-10T15:34:28.859760+0000 mgr.gibba032.whopeo (mgr.24625) 5386 : cluster [DBG] pgmap v3059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:32 gibba032 bash[24004]: cluster 2021-03-10T15:34:30.861141+0000 mgr.gibba032.whopeo (mgr.24625) 5387 : cluster [DBG] pgmap v3060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:32 gibba011 bash[30791]: cluster 2021-03-10T15:34:30.861141+0000 mgr.gibba032.whopeo (mgr.24625) 5387 : cluster [DBG] pgmap v3060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:32.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:32 gibba011 bash[31774]: cluster 2021-03-10T15:34:30.861141+0000 mgr.gibba032.whopeo (mgr.24625) 5387 : cluster [DBG] pgmap v3060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:34 gibba032 bash[24004]: cluster 2021-03-10T15:34:32.862650+0000 mgr.gibba032.whopeo (mgr.24625) 5388 : cluster [DBG] pgmap v3061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:34 gibba011 bash[30791]: cluster 2021-03-10T15:34:32.862650+0000 mgr.gibba032.whopeo (mgr.24625) 5388 : cluster [DBG] pgmap v3061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:34 gibba011 bash[31774]: cluster 2021-03-10T15:34:32.862650+0000 mgr.gibba032.whopeo (mgr.24625) 5388 : cluster [DBG] pgmap v3061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:36 gibba032 bash[24004]: cluster 2021-03-10T15:34:34.864262+0000 mgr.gibba032.whopeo (mgr.24625) 5389 : cluster [DBG] pgmap v3062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:36 gibba011 bash[30791]: cluster 2021-03-10T15:34:34.864262+0000 mgr.gibba032.whopeo (mgr.24625) 5389 : cluster [DBG] pgmap v3062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:36 gibba011 bash[31774]: cluster 2021-03-10T15:34:34.864262+0000 mgr.gibba032.whopeo (mgr.24625) 5389 : cluster [DBG] pgmap v3062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:38 gibba032 bash[24004]: cluster 2021-03-10T15:34:36.865798+0000 mgr.gibba032.whopeo (mgr.24625) 5390 : cluster [DBG] pgmap v3063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:38 gibba011 bash[30791]: cluster 2021-03-10T15:34:36.865798+0000 mgr.gibba032.whopeo (mgr.24625) 5390 : cluster [DBG] pgmap v3063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:38 gibba011 bash[31774]: cluster 2021-03-10T15:34:36.865798+0000 mgr.gibba032.whopeo (mgr.24625) 5390 : cluster [DBG] pgmap v3063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:40 gibba032 bash[24004]: cluster 2021-03-10T15:34:38.867456+0000 mgr.gibba032.whopeo (mgr.24625) 5391 : cluster [DBG] pgmap v3064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:40 gibba032 bash[24004]: cephadm 2021-03-10T15:34:39.118172+0000 mgr.gibba032.whopeo (mgr.24625) 5392 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:40 gibba011 bash[30791]: cluster 2021-03-10T15:34:38.867456+0000 mgr.gibba032.whopeo (mgr.24625) 5391 : cluster [DBG] pgmap v3064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:40 gibba011 bash[30791]: cephadm 2021-03-10T15:34:39.118172+0000 mgr.gibba032.whopeo (mgr.24625) 5392 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:40 gibba011 bash[31774]: cluster 2021-03-10T15:34:38.867456+0000 mgr.gibba032.whopeo (mgr.24625) 5391 : cluster [DBG] pgmap v3064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:40 gibba011 bash[31774]: cephadm 2021-03-10T15:34:39.118172+0000 mgr.gibba032.whopeo (mgr.24625) 5392 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:42 gibba032 bash[24004]: cluster 2021-03-10T15:34:40.868978+0000 mgr.gibba032.whopeo (mgr.24625) 5393 : cluster [DBG] pgmap v3065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:42 gibba011 bash[30791]: cluster 2021-03-10T15:34:40.868978+0000 mgr.gibba032.whopeo (mgr.24625) 5393 : cluster [DBG] pgmap v3065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:42 gibba011 bash[31774]: cluster 2021-03-10T15:34:40.868978+0000 mgr.gibba032.whopeo (mgr.24625) 5393 : cluster [DBG] pgmap v3065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:44 gibba032 bash[24004]: cluster 2021-03-10T15:34:42.870695+0000 mgr.gibba032.whopeo (mgr.24625) 5394 : cluster [DBG] pgmap v3066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:44 gibba011 bash[30791]: cluster 2021-03-10T15:34:42.870695+0000 mgr.gibba032.whopeo (mgr.24625) 5394 : cluster [DBG] pgmap v3066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:44.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:44 gibba011 bash[31774]: cluster 2021-03-10T15:34:42.870695+0000 mgr.gibba032.whopeo (mgr.24625) 5394 : cluster [DBG] pgmap v3066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:46 gibba032 bash[24004]: cluster 2021-03-10T15:34:44.872120+0000 mgr.gibba032.whopeo (mgr.24625) 5395 : cluster [DBG] pgmap v3067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:46 gibba011 bash[30791]: cluster 2021-03-10T15:34:44.872120+0000 mgr.gibba032.whopeo (mgr.24625) 5395 : cluster [DBG] pgmap v3067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:46 gibba011 bash[31774]: cluster 2021-03-10T15:34:44.872120+0000 mgr.gibba032.whopeo (mgr.24625) 5395 : cluster [DBG] pgmap v3067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:47.730 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:34:47.951 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:34:47.952 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (101m) 5m ago 106m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:34:47.952 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (106m) 4m ago 106m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:34:47.952 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (101m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:34:47.953 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (102m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:34:47.953 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (101m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:34:47.953 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (100m) 5m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:34:47.953 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (100m) 4m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:34:47.954 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (100m) 5m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:34:47.954 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (106m) 5m ago 106m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:34:47.954 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (106m) 4m ago 106m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:34:47.954 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (99m) 5m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:34:47.954 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (99m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:34:47.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (98m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:34:47.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (98m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:34:47.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (98m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:34:47.955 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (97m) 4m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:34:47.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (97m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:34:47.956 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (96m) 4m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:34:47.956 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (106m) 4m ago 106m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:34:47.956 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 104m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:34:48.206 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:34:48.206 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:34:48.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:34:48.207 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:48.207 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:34:48.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:34:48.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:48.208 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:34:48.208 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:34:48.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:34:48.208 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:34:48.209 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:34:48.209 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:34:48.209 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:34:48.209 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:34:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:48 gibba032 bash[24004]: cluster 2021-03-10T15:34:46.873624+0000 mgr.gibba032.whopeo (mgr.24625) 5396 : cluster [DBG] pgmap v3068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:48 gibba032 bash[24004]: audit 2021-03-10T15:34:48.202976+0000 mon.c (mon.1) 103 : audit [DBG] from='client.? 172.21.2.111:0/380898675' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:48 gibba011 bash[30791]: cluster 2021-03-10T15:34:46.873624+0000 mgr.gibba032.whopeo (mgr.24625) 5396 : cluster [DBG] pgmap v3068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:48 gibba011 bash[30791]: audit 2021-03-10T15:34:48.202976+0000 mon.c (mon.1) 103 : audit [DBG] from='client.? 172.21.2.111:0/380898675' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:48 gibba011 bash[31774]: cluster 2021-03-10T15:34:46.873624+0000 mgr.gibba032.whopeo (mgr.24625) 5396 : cluster [DBG] pgmap v3068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:48 gibba011 bash[31774]: audit 2021-03-10T15:34:48.202976+0000 mon.c (mon.1) 103 : audit [DBG] from='client.? 172.21.2.111:0/380898675' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:34:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:49 gibba032 bash[24004]: audit 2021-03-10T15:34:47.717603+0000 mgr.gibba032.whopeo (mgr.24625) 5397 : audit [DBG] from='client.57658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:49 gibba032 bash[24004]: audit 2021-03-10T15:34:47.938656+0000 mgr.gibba032.whopeo (mgr.24625) 5398 : audit [DBG] from='client.47679 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:49 gibba011 bash[30791]: audit 2021-03-10T15:34:47.717603+0000 mgr.gibba032.whopeo (mgr.24625) 5397 : audit [DBG] from='client.57658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:49 gibba011 bash[30791]: audit 2021-03-10T15:34:47.938656+0000 mgr.gibba032.whopeo (mgr.24625) 5398 : audit [DBG] from='client.47679 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:49 gibba011 bash[31774]: audit 2021-03-10T15:34:47.717603+0000 mgr.gibba032.whopeo (mgr.24625) 5397 : audit [DBG] from='client.57658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:49 gibba011 bash[31774]: audit 2021-03-10T15:34:47.938656+0000 mgr.gibba032.whopeo (mgr.24625) 5398 : audit [DBG] from='client.47679 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:34:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:50 gibba011 bash[30791]: cluster 2021-03-10T15:34:48.875135+0000 mgr.gibba032.whopeo (mgr.24625) 5399 : cluster [DBG] pgmap v3069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:50 gibba011 bash[31774]: cluster 2021-03-10T15:34:48.875135+0000 mgr.gibba032.whopeo (mgr.24625) 5399 : cluster [DBG] pgmap v3069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:50 gibba032 bash[24004]: cluster 2021-03-10T15:34:48.875135+0000 mgr.gibba032.whopeo (mgr.24625) 5399 : cluster [DBG] pgmap v3069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:52 gibba011 bash[30791]: cluster 2021-03-10T15:34:50.876504+0000 mgr.gibba032.whopeo (mgr.24625) 5400 : cluster [DBG] pgmap v3070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:52 gibba011 bash[31774]: cluster 2021-03-10T15:34:50.876504+0000 mgr.gibba032.whopeo (mgr.24625) 5400 : cluster [DBG] pgmap v3070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:52 gibba032 bash[24004]: cluster 2021-03-10T15:34:50.876504+0000 mgr.gibba032.whopeo (mgr.24625) 5400 : cluster [DBG] pgmap v3070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:54 gibba032 bash[24004]: cluster 2021-03-10T15:34:52.878131+0000 mgr.gibba032.whopeo (mgr.24625) 5401 : cluster [DBG] pgmap v3071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:54 gibba011 bash[30791]: cluster 2021-03-10T15:34:52.878131+0000 mgr.gibba032.whopeo (mgr.24625) 5401 : cluster [DBG] pgmap v3071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:54.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:54 gibba011 bash[31774]: cluster 2021-03-10T15:34:52.878131+0000 mgr.gibba032.whopeo (mgr.24625) 5401 : cluster [DBG] pgmap v3071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:55 gibba011 bash[30791]: cephadm 2021-03-10T15:34:54.134244+0000 mgr.gibba032.whopeo (mgr.24625) 5402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:55.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:55 gibba011 bash[31774]: cephadm 2021-03-10T15:34:54.134244+0000 mgr.gibba032.whopeo (mgr.24625) 5402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:55.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:55 gibba032 bash[24004]: cephadm 2021-03-10T15:34:54.134244+0000 mgr.gibba032.whopeo (mgr.24625) 5402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:34:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:56 gibba011 bash[30791]: cluster 2021-03-10T15:34:54.879487+0000 mgr.gibba032.whopeo (mgr.24625) 5403 : cluster [DBG] pgmap v3072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:56 gibba011 bash[31774]: cluster 2021-03-10T15:34:54.879487+0000 mgr.gibba032.whopeo (mgr.24625) 5403 : cluster [DBG] pgmap v3072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:56 gibba032 bash[24004]: cluster 2021-03-10T15:34:54.879487+0000 mgr.gibba032.whopeo (mgr.24625) 5403 : cluster [DBG] pgmap v3072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.646000+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:34:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.646641+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:34:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.661888+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:34:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.662639+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:34:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.663455+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:34:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[30791]: audit 2021-03-10T15:34:56.664166+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:34:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.646000+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:34:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.646641+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:34:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.661888+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:34:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.662639+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:34:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.663455+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:34:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:57 gibba011 bash[31774]: audit 2021-03-10T15:34:56.664166+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:34:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.646000+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:34:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.646641+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:34:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.661888+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T15:34:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.662639+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:34:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.663455+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T15:34:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:57 gibba032 bash[24004]: audit 2021-03-10T15:34:56.664166+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:34:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:34:58 gibba011 bash[30791]: cluster 2021-03-10T15:34:56.881171+0000 mgr.gibba032.whopeo (mgr.24625) 5404 : cluster [DBG] pgmap v3073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:34:58 gibba011 bash[31774]: cluster 2021-03-10T15:34:56.881171+0000 mgr.gibba032.whopeo (mgr.24625) 5404 : cluster [DBG] pgmap v3073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:34:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:34:58 gibba032 bash[24004]: cluster 2021-03-10T15:34:56.881171+0000 mgr.gibba032.whopeo (mgr.24625) 5404 : cluster [DBG] pgmap v3073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:00 gibba011 bash[30791]: cluster 2021-03-10T15:34:58.882777+0000 mgr.gibba032.whopeo (mgr.24625) 5405 : cluster [DBG] pgmap v3074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:00 gibba011 bash[31774]: cluster 2021-03-10T15:34:58.882777+0000 mgr.gibba032.whopeo (mgr.24625) 5405 : cluster [DBG] pgmap v3074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:00 gibba032 bash[24004]: cluster 2021-03-10T15:34:58.882777+0000 mgr.gibba032.whopeo (mgr.24625) 5405 : cluster [DBG] pgmap v3074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:02 gibba011 bash[30791]: cluster 2021-03-10T15:35:00.884127+0000 mgr.gibba032.whopeo (mgr.24625) 5406 : cluster [DBG] pgmap v3075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:02 gibba011 bash[31774]: cluster 2021-03-10T15:35:00.884127+0000 mgr.gibba032.whopeo (mgr.24625) 5406 : cluster [DBG] pgmap v3075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:02 gibba032 bash[24004]: cluster 2021-03-10T15:35:00.884127+0000 mgr.gibba032.whopeo (mgr.24625) 5406 : cluster [DBG] pgmap v3075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:04 gibba032 bash[24004]: cluster 2021-03-10T15:35:02.885921+0000 mgr.gibba032.whopeo (mgr.24625) 5407 : cluster [DBG] pgmap v3076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:04.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:04 gibba011 bash[30791]: cluster 2021-03-10T15:35:02.885921+0000 mgr.gibba032.whopeo (mgr.24625) 5407 : cluster [DBG] pgmap v3076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:04.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:04 gibba011 bash[31774]: cluster 2021-03-10T15:35:02.885921+0000 mgr.gibba032.whopeo (mgr.24625) 5407 : cluster [DBG] pgmap v3076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:06 gibba032 bash[24004]: cluster 2021-03-10T15:35:04.887432+0000 mgr.gibba032.whopeo (mgr.24625) 5408 : cluster [DBG] pgmap v3077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:06.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:06 gibba011 bash[30791]: cluster 2021-03-10T15:35:04.887432+0000 mgr.gibba032.whopeo (mgr.24625) 5408 : cluster [DBG] pgmap v3077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:06 gibba011 bash[31774]: cluster 2021-03-10T15:35:04.887432+0000 mgr.gibba032.whopeo (mgr.24625) 5408 : cluster [DBG] pgmap v3077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:08 gibba032 bash[24004]: cluster 2021-03-10T15:35:06.889131+0000 mgr.gibba032.whopeo (mgr.24625) 5409 : cluster [DBG] pgmap v3078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:08 gibba011 bash[30791]: cluster 2021-03-10T15:35:06.889131+0000 mgr.gibba032.whopeo (mgr.24625) 5409 : cluster [DBG] pgmap v3078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:08.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:08 gibba011 bash[31774]: cluster 2021-03-10T15:35:06.889131+0000 mgr.gibba032.whopeo (mgr.24625) 5409 : cluster [DBG] pgmap v3078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:10.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cluster 2021-03-10T15:35:08.890808+0000 mgr.gibba032.whopeo (mgr.24625) 5410 : cluster [DBG] pgmap v3079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:10.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.150971+0000 mgr.gibba032.whopeo (mgr.24625) 5411 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:35:10.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.152603+0000 mgr.gibba032.whopeo (mgr.24625) 5412 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:35:10.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.380486+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:35:10.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.392678+0000 mgr.gibba032.whopeo (mgr.24625) 5413 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:35:10.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.396509+0000 mon.a (mon.0) 2089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:35:10.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.398691+0000 mgr.gibba032.whopeo (mgr.24625) 5414 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:35:10.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.401230+0000 mon.a (mon.0) 2090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.402322+0000 mgr.gibba032.whopeo (mgr.24625) 5415 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:35:10.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.402606+0000 mgr.gibba032.whopeo (mgr.24625) 5416 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:35:10.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.404871+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.406089+0000 mgr.gibba032.whopeo (mgr.24625) 5417 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:35:10.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.406677+0000 mgr.gibba032.whopeo (mgr.24625) 5418 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:35:10.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.408425+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.409383+0000 mgr.gibba032.whopeo (mgr.24625) 5419 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:35:10.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.412105+0000 mgr.gibba032.whopeo (mgr.24625) 5420 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:35:10.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.412413+0000 mgr.gibba032.whopeo (mgr.24625) 5421 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:35:10.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.415407+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.417241+0000 mgr.gibba032.whopeo (mgr.24625) 5422 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:35:10.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.419465+0000 mgr.gibba032.whopeo (mgr.24625) 5423 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:35:10.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: audit 2021-03-10T15:35:09.421670+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.423652+0000 mgr.gibba032.whopeo (mgr.24625) 5424 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:35:10.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.423878+0000 mgr.gibba032.whopeo (mgr.24625) 5425 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:35:10.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:10 gibba032 bash[24004]: cephadm 2021-03-10T15:35:09.424693+0000 mgr.gibba032.whopeo (mgr.24625) 5426 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:10.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cluster 2021-03-10T15:35:08.890808+0000 mgr.gibba032.whopeo (mgr.24625) 5410 : cluster [DBG] pgmap v3079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:10.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.150971+0000 mgr.gibba032.whopeo (mgr.24625) 5411 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:35:10.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.152603+0000 mgr.gibba032.whopeo (mgr.24625) 5412 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:35:10.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.380486+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:35:10.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.392678+0000 mgr.gibba032.whopeo (mgr.24625) 5413 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:35:10.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.396509+0000 mon.a (mon.0) 2089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:35:10.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.398691+0000 mgr.gibba032.whopeo (mgr.24625) 5414 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:35:10.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.401230+0000 mon.a (mon.0) 2090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.402322+0000 mgr.gibba032.whopeo (mgr.24625) 5415 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:35:10.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.402606+0000 mgr.gibba032.whopeo (mgr.24625) 5416 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:35:10.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.404871+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.406089+0000 mgr.gibba032.whopeo (mgr.24625) 5417 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:35:10.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.406677+0000 mgr.gibba032.whopeo (mgr.24625) 5418 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:35:10.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.408425+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.409383+0000 mgr.gibba032.whopeo (mgr.24625) 5419 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:35:10.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.412105+0000 mgr.gibba032.whopeo (mgr.24625) 5420 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:35:10.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.412413+0000 mgr.gibba032.whopeo (mgr.24625) 5421 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:35:10.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.415407+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.417241+0000 mgr.gibba032.whopeo (mgr.24625) 5422 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:35:10.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.419465+0000 mgr.gibba032.whopeo (mgr.24625) 5423 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:35:10.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cluster 2021-03-10T15:35:08.890808+0000 mgr.gibba032.whopeo (mgr.24625) 5410 : cluster [DBG] pgmap v3079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:10.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.150971+0000 mgr.gibba032.whopeo (mgr.24625) 5411 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:35:10.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.152603+0000 mgr.gibba032.whopeo (mgr.24625) 5412 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:35:10.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.380486+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:35:10.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.392678+0000 mgr.gibba032.whopeo (mgr.24625) 5413 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:35:10.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.396509+0000 mon.a (mon.0) 2089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:35:10.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.398691+0000 mgr.gibba032.whopeo (mgr.24625) 5414 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:35:10.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.401230+0000 mon.a (mon.0) 2090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.402322+0000 mgr.gibba032.whopeo (mgr.24625) 5415 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:35:10.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.402606+0000 mgr.gibba032.whopeo (mgr.24625) 5416 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:35:10.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.404871+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.406089+0000 mgr.gibba032.whopeo (mgr.24625) 5417 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:35:10.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.406677+0000 mgr.gibba032.whopeo (mgr.24625) 5418 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:35:10.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.408425+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.409383+0000 mgr.gibba032.whopeo (mgr.24625) 5419 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:35:10.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.412105+0000 mgr.gibba032.whopeo (mgr.24625) 5420 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:35:10.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.412413+0000 mgr.gibba032.whopeo (mgr.24625) 5421 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:35:10.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.415407+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.417241+0000 mgr.gibba032.whopeo (mgr.24625) 5422 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:35:10.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.419465+0000 mgr.gibba032.whopeo (mgr.24625) 5423 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:35:10.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: audit 2021-03-10T15:35:09.421670+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.423652+0000 mgr.gibba032.whopeo (mgr.24625) 5424 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:35:10.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.423878+0000 mgr.gibba032.whopeo (mgr.24625) 5425 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:35:10.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[30791]: cephadm 2021-03-10T15:35:09.424693+0000 mgr.gibba032.whopeo (mgr.24625) 5426 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:10.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: audit 2021-03-10T15:35:09.421670+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:10.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.423652+0000 mgr.gibba032.whopeo (mgr.24625) 5424 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:35:10.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.423878+0000 mgr.gibba032.whopeo (mgr.24625) 5425 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:35:10.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:10 gibba011 bash[31774]: cephadm 2021-03-10T15:35:09.424693+0000 mgr.gibba032.whopeo (mgr.24625) 5426 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:12 gibba032 bash[24004]: cluster 2021-03-10T15:35:10.892220+0000 mgr.gibba032.whopeo (mgr.24625) 5427 : cluster [DBG] pgmap v3080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:12 gibba011 bash[30791]: cluster 2021-03-10T15:35:10.892220+0000 mgr.gibba032.whopeo (mgr.24625) 5427 : cluster [DBG] pgmap v3080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:12.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:12 gibba011 bash[31774]: cluster 2021-03-10T15:35:10.892220+0000 mgr.gibba032.whopeo (mgr.24625) 5427 : cluster [DBG] pgmap v3080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:14 gibba032 bash[24004]: cluster 2021-03-10T15:35:12.894031+0000 mgr.gibba032.whopeo (mgr.24625) 5428 : cluster [DBG] pgmap v3081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:14 gibba011 bash[30791]: cluster 2021-03-10T15:35:12.894031+0000 mgr.gibba032.whopeo (mgr.24625) 5428 : cluster [DBG] pgmap v3081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:14.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:14 gibba011 bash[31774]: cluster 2021-03-10T15:35:12.894031+0000 mgr.gibba032.whopeo (mgr.24625) 5428 : cluster [DBG] pgmap v3081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:16 gibba032 bash[24004]: cluster 2021-03-10T15:35:14.895265+0000 mgr.gibba032.whopeo (mgr.24625) 5429 : cluster [DBG] pgmap v3082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:16 gibba011 bash[30791]: cluster 2021-03-10T15:35:14.895265+0000 mgr.gibba032.whopeo (mgr.24625) 5429 : cluster [DBG] pgmap v3082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:16 gibba011 bash[31774]: cluster 2021-03-10T15:35:14.895265+0000 mgr.gibba032.whopeo (mgr.24625) 5429 : cluster [DBG] pgmap v3082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:18.448 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:35:18.676 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:35:18.677 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (101m) 6m ago 107m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:35:18.677 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (107m) 5m ago 107m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:35:18.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (102m) 5m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:35:18.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (102m) 5m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:35:18.678 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (101m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:35:18.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (101m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:35:18.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (101m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:35:18.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (101m) 6m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:35:18.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (107m) 6m ago 107m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:35:18.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (107m) 5m ago 107m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:35:18.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (100m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:35:18.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (99m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:35:18.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (99m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:35:18.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (98m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:35:18.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (98m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:35:18.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (98m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:35:18.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (97m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:35:18.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (97m) 5m ago 107m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:35:18.681 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (106m) 5m ago 107m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:35:18.681 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 105m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:35:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:18 gibba032 bash[24004]: cluster 2021-03-10T15:35:16.897111+0000 mgr.gibba032.whopeo (mgr.24625) 5430 : cluster [DBG] pgmap v3083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:18 gibba011 bash[30791]: cluster 2021-03-10T15:35:16.897111+0000 mgr.gibba032.whopeo (mgr.24625) 5430 : cluster [DBG] pgmap v3083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:18 gibba011 bash[31774]: cluster 2021-03-10T15:35:16.897111+0000 mgr.gibba032.whopeo (mgr.24625) 5430 : cluster [DBG] pgmap v3083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:18.926 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:35:18.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:35:18.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:35:18.927 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:18.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:35:18.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:35:18.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:18.928 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:35:18.928 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:35:18.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:18.928 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:35:18.929 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:35:18.929 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:35:18.929 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:35:18.929 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:35:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:19 gibba032 bash[24004]: audit 2021-03-10T15:35:18.436181+0000 mgr.gibba032.whopeo (mgr.24625) 5431 : audit [DBG] from='client.47691 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:19 gibba032 bash[24004]: audit 2021-03-10T15:35:18.923083+0000 mon.c (mon.1) 104 : audit [DBG] from='client.? 172.21.2.111:0/1086125494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:19 gibba011 bash[30791]: audit 2021-03-10T15:35:18.436181+0000 mgr.gibba032.whopeo (mgr.24625) 5431 : audit [DBG] from='client.47691 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:19.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:19 gibba011 bash[30791]: audit 2021-03-10T15:35:18.923083+0000 mon.c (mon.1) 104 : audit [DBG] from='client.? 172.21.2.111:0/1086125494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:19.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:19 gibba011 bash[31774]: audit 2021-03-10T15:35:18.436181+0000 mgr.gibba032.whopeo (mgr.24625) 5431 : audit [DBG] from='client.47691 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:19.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:19 gibba011 bash[31774]: audit 2021-03-10T15:35:18.923083+0000 mon.c (mon.1) 104 : audit [DBG] from='client.? 172.21.2.111:0/1086125494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:20 gibba032 bash[24004]: audit 2021-03-10T15:35:18.660996+0000 mgr.gibba032.whopeo (mgr.24625) 5432 : audit [DBG] from='client.47697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:20 gibba032 bash[24004]: cluster 2021-03-10T15:35:18.898701+0000 mgr.gibba032.whopeo (mgr.24625) 5433 : cluster [DBG] pgmap v3084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:20 gibba011 bash[30791]: audit 2021-03-10T15:35:18.660996+0000 mgr.gibba032.whopeo (mgr.24625) 5432 : audit [DBG] from='client.47697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:20 gibba011 bash[30791]: cluster 2021-03-10T15:35:18.898701+0000 mgr.gibba032.whopeo (mgr.24625) 5433 : cluster [DBG] pgmap v3084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:20 gibba011 bash[31774]: audit 2021-03-10T15:35:18.660996+0000 mgr.gibba032.whopeo (mgr.24625) 5432 : audit [DBG] from='client.47697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:20 gibba011 bash[31774]: cluster 2021-03-10T15:35:18.898701+0000 mgr.gibba032.whopeo (mgr.24625) 5433 : cluster [DBG] pgmap v3084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:22 gibba032 bash[24004]: cluster 2021-03-10T15:35:20.900076+0000 mgr.gibba032.whopeo (mgr.24625) 5434 : cluster [DBG] pgmap v3085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:22 gibba011 bash[30791]: cluster 2021-03-10T15:35:20.900076+0000 mgr.gibba032.whopeo (mgr.24625) 5434 : cluster [DBG] pgmap v3085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:22 gibba011 bash[31774]: cluster 2021-03-10T15:35:20.900076+0000 mgr.gibba032.whopeo (mgr.24625) 5434 : cluster [DBG] pgmap v3085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:24.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:24 gibba032 bash[24004]: cluster 2021-03-10T15:35:22.901608+0000 mgr.gibba032.whopeo (mgr.24625) 5435 : cluster [DBG] pgmap v3086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:24.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:24 gibba011 bash[30791]: cluster 2021-03-10T15:35:22.901608+0000 mgr.gibba032.whopeo (mgr.24625) 5435 : cluster [DBG] pgmap v3086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:24.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:24 gibba011 bash[31774]: cluster 2021-03-10T15:35:22.901608+0000 mgr.gibba032.whopeo (mgr.24625) 5435 : cluster [DBG] pgmap v3086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:25.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:25 gibba032 bash[24004]: cephadm 2021-03-10T15:35:24.429440+0000 mgr.gibba032.whopeo (mgr.24625) 5436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:25 gibba011 bash[30791]: cephadm 2021-03-10T15:35:24.429440+0000 mgr.gibba032.whopeo (mgr.24625) 5436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:25 gibba011 bash[31774]: cephadm 2021-03-10T15:35:24.429440+0000 mgr.gibba032.whopeo (mgr.24625) 5436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:26 gibba032 bash[24004]: cluster 2021-03-10T15:35:24.903151+0000 mgr.gibba032.whopeo (mgr.24625) 5437 : cluster [DBG] pgmap v3087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:26 gibba011 bash[30791]: cluster 2021-03-10T15:35:24.903151+0000 mgr.gibba032.whopeo (mgr.24625) 5437 : cluster [DBG] pgmap v3087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:26 gibba011 bash[31774]: cluster 2021-03-10T15:35:24.903151+0000 mgr.gibba032.whopeo (mgr.24625) 5437 : cluster [DBG] pgmap v3087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:28 gibba032 bash[24004]: cluster 2021-03-10T15:35:26.904881+0000 mgr.gibba032.whopeo (mgr.24625) 5438 : cluster [DBG] pgmap v3088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:28.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:28 gibba011 bash[30791]: cluster 2021-03-10T15:35:26.904881+0000 mgr.gibba032.whopeo (mgr.24625) 5438 : cluster [DBG] pgmap v3088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:28.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:28 gibba011 bash[31774]: cluster 2021-03-10T15:35:26.904881+0000 mgr.gibba032.whopeo (mgr.24625) 5438 : cluster [DBG] pgmap v3088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:30 gibba032 bash[24004]: cluster 2021-03-10T15:35:28.906479+0000 mgr.gibba032.whopeo (mgr.24625) 5439 : cluster [DBG] pgmap v3089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:30 gibba011 bash[30791]: cluster 2021-03-10T15:35:28.906479+0000 mgr.gibba032.whopeo (mgr.24625) 5439 : cluster [DBG] pgmap v3089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:30 gibba011 bash[31774]: cluster 2021-03-10T15:35:28.906479+0000 mgr.gibba032.whopeo (mgr.24625) 5439 : cluster [DBG] pgmap v3089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:32 gibba032 bash[24004]: cluster 2021-03-10T15:35:30.907844+0000 mgr.gibba032.whopeo (mgr.24625) 5440 : cluster [DBG] pgmap v3090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:32 gibba011 bash[30791]: cluster 2021-03-10T15:35:30.907844+0000 mgr.gibba032.whopeo (mgr.24625) 5440 : cluster [DBG] pgmap v3090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:32 gibba011 bash[31774]: cluster 2021-03-10T15:35:30.907844+0000 mgr.gibba032.whopeo (mgr.24625) 5440 : cluster [DBG] pgmap v3090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:34.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:34 gibba032 bash[24004]: cluster 2021-03-10T15:35:32.909644+0000 mgr.gibba032.whopeo (mgr.24625) 5441 : cluster [DBG] pgmap v3091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:34.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:34 gibba011 bash[30791]: cluster 2021-03-10T15:35:32.909644+0000 mgr.gibba032.whopeo (mgr.24625) 5441 : cluster [DBG] pgmap v3091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:34.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:34 gibba011 bash[31774]: cluster 2021-03-10T15:35:32.909644+0000 mgr.gibba032.whopeo (mgr.24625) 5441 : cluster [DBG] pgmap v3091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:36.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:36 gibba011 bash[30791]: cluster 2021-03-10T15:35:34.911062+0000 mgr.gibba032.whopeo (mgr.24625) 5442 : cluster [DBG] pgmap v3092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:36 gibba011 bash[31774]: cluster 2021-03-10T15:35:34.911062+0000 mgr.gibba032.whopeo (mgr.24625) 5442 : cluster [DBG] pgmap v3092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:36 gibba032 bash[24004]: cluster 2021-03-10T15:35:34.911062+0000 mgr.gibba032.whopeo (mgr.24625) 5442 : cluster [DBG] pgmap v3092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:38.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:38 gibba011 bash[30791]: cluster 2021-03-10T15:35:36.912844+0000 mgr.gibba032.whopeo (mgr.24625) 5443 : cluster [DBG] pgmap v3093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:38.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:38 gibba011 bash[31774]: cluster 2021-03-10T15:35:36.912844+0000 mgr.gibba032.whopeo (mgr.24625) 5443 : cluster [DBG] pgmap v3093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:39.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:38 gibba032 bash[24004]: cluster 2021-03-10T15:35:36.912844+0000 mgr.gibba032.whopeo (mgr.24625) 5443 : cluster [DBG] pgmap v3093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:40 gibba011 bash[30791]: cluster 2021-03-10T15:35:38.914441+0000 mgr.gibba032.whopeo (mgr.24625) 5444 : cluster [DBG] pgmap v3094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:40 gibba011 bash[30791]: cephadm 2021-03-10T15:35:39.445105+0000 mgr.gibba032.whopeo (mgr.24625) 5445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:40 gibba011 bash[31774]: cluster 2021-03-10T15:35:38.914441+0000 mgr.gibba032.whopeo (mgr.24625) 5444 : cluster [DBG] pgmap v3094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:40 gibba011 bash[31774]: cephadm 2021-03-10T15:35:39.445105+0000 mgr.gibba032.whopeo (mgr.24625) 5445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:40 gibba032 bash[24004]: cluster 2021-03-10T15:35:38.914441+0000 mgr.gibba032.whopeo (mgr.24625) 5444 : cluster [DBG] pgmap v3094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:40 gibba032 bash[24004]: cephadm 2021-03-10T15:35:39.445105+0000 mgr.gibba032.whopeo (mgr.24625) 5445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:42 gibba011 bash[30791]: cluster 2021-03-10T15:35:40.915937+0000 mgr.gibba032.whopeo (mgr.24625) 5446 : cluster [DBG] pgmap v3095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:42.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:42 gibba011 bash[31774]: cluster 2021-03-10T15:35:40.915937+0000 mgr.gibba032.whopeo (mgr.24625) 5446 : cluster [DBG] pgmap v3095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:42 gibba032 bash[24004]: cluster 2021-03-10T15:35:40.915937+0000 mgr.gibba032.whopeo (mgr.24625) 5446 : cluster [DBG] pgmap v3095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:44 gibba011 bash[30791]: cluster 2021-03-10T15:35:42.917564+0000 mgr.gibba032.whopeo (mgr.24625) 5447 : cluster [DBG] pgmap v3096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:44.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:44 gibba011 bash[31774]: cluster 2021-03-10T15:35:42.917564+0000 mgr.gibba032.whopeo (mgr.24625) 5447 : cluster [DBG] pgmap v3096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:44 gibba032 bash[24004]: cluster 2021-03-10T15:35:42.917564+0000 mgr.gibba032.whopeo (mgr.24625) 5447 : cluster [DBG] pgmap v3096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:46.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:46 gibba011 bash[30791]: cluster 2021-03-10T15:35:44.919132+0000 mgr.gibba032.whopeo (mgr.24625) 5448 : cluster [DBG] pgmap v3097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:46.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:46 gibba011 bash[31774]: cluster 2021-03-10T15:35:44.919132+0000 mgr.gibba032.whopeo (mgr.24625) 5448 : cluster [DBG] pgmap v3097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:47.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:46 gibba032 bash[24004]: cluster 2021-03-10T15:35:44.919132+0000 mgr.gibba032.whopeo (mgr.24625) 5448 : cluster [DBG] pgmap v3097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:48 gibba011 bash[30791]: cluster 2021-03-10T15:35:46.920808+0000 mgr.gibba032.whopeo (mgr.24625) 5449 : cluster [DBG] pgmap v3098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:48 gibba011 bash[31774]: cluster 2021-03-10T15:35:46.920808+0000 mgr.gibba032.whopeo (mgr.24625) 5449 : cluster [DBG] pgmap v3098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:49.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:48 gibba032 bash[24004]: cluster 2021-03-10T15:35:46.920808+0000 mgr.gibba032.whopeo (mgr.24625) 5449 : cluster [DBG] pgmap v3098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:49.170 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:35:49.396 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:35:49.396 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (102m) 6m ago 107m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:35:49.397 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (107m) 5m ago 107m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:35:49.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (102m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:35:49.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (103m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:35:49.398 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (102m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:35:49.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (102m) 6m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:35:49.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (101m) 5m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:35:49.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (101m) 6m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:35:49.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (107m) 6m ago 107m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:35:49.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (107m) 5m ago 107m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:35:49.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (100m) 6m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:35:49.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (100m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:35:49.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (99m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:35:49.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (99m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:35:49.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (99m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:35:49.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (98m) 5m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:35:49.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (98m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:35:49.401 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (97m) 5m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:35:49.401 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (107m) 5m ago 108m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:35:49.401 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 105m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:35:49.644 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:35:49.645 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:35:49.645 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:35:49.645 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:49.646 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:35:49.646 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:35:49.646 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:49.646 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:35:49.647 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:35:49.647 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:35:49.647 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:35:49.647 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:35:49.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:35:49.648 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:35:49.648 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:35:50.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[30791]: cluster 2021-03-10T15:35:48.922388+0000 mgr.gibba032.whopeo (mgr.24625) 5450 : cluster [DBG] pgmap v3099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:50.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[30791]: audit 2021-03-10T15:35:49.157281+0000 mgr.gibba032.whopeo (mgr.24625) 5451 : audit [DBG] from='client.57694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:50.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[30791]: audit 2021-03-10T15:35:49.379988+0000 mgr.gibba032.whopeo (mgr.24625) 5452 : audit [DBG] from='client.47715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:50.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[30791]: audit 2021-03-10T15:35:49.641346+0000 mon.a (mon.0) 2095 : audit [DBG] from='client.? 172.21.2.111:0/3920815018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:50.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[31774]: cluster 2021-03-10T15:35:48.922388+0000 mgr.gibba032.whopeo (mgr.24625) 5450 : cluster [DBG] pgmap v3099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:50.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[31774]: audit 2021-03-10T15:35:49.157281+0000 mgr.gibba032.whopeo (mgr.24625) 5451 : audit [DBG] from='client.57694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:50.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[31774]: audit 2021-03-10T15:35:49.379988+0000 mgr.gibba032.whopeo (mgr.24625) 5452 : audit [DBG] from='client.47715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:50.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:50 gibba011 bash[31774]: audit 2021-03-10T15:35:49.641346+0000 mon.a (mon.0) 2095 : audit [DBG] from='client.? 172.21.2.111:0/3920815018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:50 gibba032 bash[24004]: cluster 2021-03-10T15:35:48.922388+0000 mgr.gibba032.whopeo (mgr.24625) 5450 : cluster [DBG] pgmap v3099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:50 gibba032 bash[24004]: audit 2021-03-10T15:35:49.157281+0000 mgr.gibba032.whopeo (mgr.24625) 5451 : audit [DBG] from='client.57694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:50 gibba032 bash[24004]: audit 2021-03-10T15:35:49.379988+0000 mgr.gibba032.whopeo (mgr.24625) 5452 : audit [DBG] from='client.47715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:35:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:50 gibba032 bash[24004]: audit 2021-03-10T15:35:49.641346+0000 mon.a (mon.0) 2095 : audit [DBG] from='client.? 172.21.2.111:0/3920815018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:35:52.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:52 gibba011 bash[30791]: cluster 2021-03-10T15:35:50.923848+0000 mgr.gibba032.whopeo (mgr.24625) 5453 : cluster [DBG] pgmap v3100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:52.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:52 gibba011 bash[31774]: cluster 2021-03-10T15:35:50.923848+0000 mgr.gibba032.whopeo (mgr.24625) 5453 : cluster [DBG] pgmap v3100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:52 gibba032 bash[24004]: cluster 2021-03-10T15:35:50.923848+0000 mgr.gibba032.whopeo (mgr.24625) 5453 : cluster [DBG] pgmap v3100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:54 gibba032 bash[24004]: cluster 2021-03-10T15:35:52.925487+0000 mgr.gibba032.whopeo (mgr.24625) 5454 : cluster [DBG] pgmap v3101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:54 gibba011 bash[30791]: cluster 2021-03-10T15:35:52.925487+0000 mgr.gibba032.whopeo (mgr.24625) 5454 : cluster [DBG] pgmap v3101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:55.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:54 gibba011 bash[31774]: cluster 2021-03-10T15:35:52.925487+0000 mgr.gibba032.whopeo (mgr.24625) 5454 : cluster [DBG] pgmap v3101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:56.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:55 gibba032 bash[24004]: cephadm 2021-03-10T15:35:54.460141+0000 mgr.gibba032.whopeo (mgr.24625) 5455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:56.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:55 gibba011 bash[30791]: cephadm 2021-03-10T15:35:54.460141+0000 mgr.gibba032.whopeo (mgr.24625) 5455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:56.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:55 gibba011 bash[31774]: cephadm 2021-03-10T15:35:54.460141+0000 mgr.gibba032.whopeo (mgr.24625) 5455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:35:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:56 gibba032 bash[24004]: cluster 2021-03-10T15:35:54.926986+0000 mgr.gibba032.whopeo (mgr.24625) 5456 : cluster [DBG] pgmap v3102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:57.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:56 gibba011 bash[30791]: cluster 2021-03-10T15:35:54.926986+0000 mgr.gibba032.whopeo (mgr.24625) 5456 : cluster [DBG] pgmap v3102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:57.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:56 gibba011 bash[31774]: cluster 2021-03-10T15:35:54.926986+0000 mgr.gibba032.whopeo (mgr.24625) 5456 : cluster [DBG] pgmap v3102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.649616+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:35:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.649782+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:35:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.668207+0000 mon.a (mon.0) 2098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:35:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.668479+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.668714+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:57 gibba032 bash[24004]: audit 2021-03-10T15:35:56.668960+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.649616+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:35:58.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.649782+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:35:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.668207+0000 mon.a (mon.0) 2098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:35:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.668479+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.668714+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[30791]: audit 2021-03-10T15:35:56.668960+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.649616+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:35:58.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.649782+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:35:58.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.668207+0000 mon.a (mon.0) 2098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:35:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.668479+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.668714+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:35:58.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:57 gibba011 bash[31774]: audit 2021-03-10T15:35:56.668960+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:35:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:58 gibba032 bash[24004]: cluster 2021-03-10T15:35:56.928808+0000 mgr.gibba032.whopeo (mgr.24625) 5457 : cluster [DBG] pgmap v3103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:58 gibba011 bash[30791]: cluster 2021-03-10T15:35:56.928808+0000 mgr.gibba032.whopeo (mgr.24625) 5457 : cluster [DBG] pgmap v3103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:35:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:58 gibba011 bash[31774]: cluster 2021-03-10T15:35:56.928808+0000 mgr.gibba032.whopeo (mgr.24625) 5457 : cluster [DBG] pgmap v3103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:00.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:35:59 gibba032 bash[24004]: cluster 2021-03-10T15:35:58.930458+0000 mgr.gibba032.whopeo (mgr.24625) 5458 : cluster [DBG] pgmap v3104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:00.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:35:59 gibba011 bash[30791]: cluster 2021-03-10T15:35:58.930458+0000 mgr.gibba032.whopeo (mgr.24625) 5458 : cluster [DBG] pgmap v3104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:00.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:35:59 gibba011 bash[31774]: cluster 2021-03-10T15:35:58.930458+0000 mgr.gibba032.whopeo (mgr.24625) 5458 : cluster [DBG] pgmap v3104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:01 gibba032 bash[24004]: cluster 2021-03-10T15:36:00.931833+0000 mgr.gibba032.whopeo (mgr.24625) 5459 : cluster [DBG] pgmap v3105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:01 gibba011 bash[30791]: cluster 2021-03-10T15:36:00.931833+0000 mgr.gibba032.whopeo (mgr.24625) 5459 : cluster [DBG] pgmap v3105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:01 gibba011 bash[31774]: cluster 2021-03-10T15:36:00.931833+0000 mgr.gibba032.whopeo (mgr.24625) 5459 : cluster [DBG] pgmap v3105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:04 gibba032 bash[24004]: cluster 2021-03-10T15:36:02.933795+0000 mgr.gibba032.whopeo (mgr.24625) 5460 : cluster [DBG] pgmap v3106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:04 gibba011 bash[30791]: cluster 2021-03-10T15:36:02.933795+0000 mgr.gibba032.whopeo (mgr.24625) 5460 : cluster [DBG] pgmap v3106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:04 gibba011 bash[31774]: cluster 2021-03-10T15:36:02.933795+0000 mgr.gibba032.whopeo (mgr.24625) 5460 : cluster [DBG] pgmap v3106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:06.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:06 gibba011 bash[30791]: cluster 2021-03-10T15:36:04.935424+0000 mgr.gibba032.whopeo (mgr.24625) 5461 : cluster [DBG] pgmap v3107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:06.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:06 gibba011 bash[31774]: cluster 2021-03-10T15:36:04.935424+0000 mgr.gibba032.whopeo (mgr.24625) 5461 : cluster [DBG] pgmap v3107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:06 gibba032 bash[24004]: cluster 2021-03-10T15:36:04.935424+0000 mgr.gibba032.whopeo (mgr.24625) 5461 : cluster [DBG] pgmap v3107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:08.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:08 gibba011 bash[30791]: cluster 2021-03-10T15:36:06.937214+0000 mgr.gibba032.whopeo (mgr.24625) 5462 : cluster [DBG] pgmap v3108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:08 gibba011 bash[31774]: cluster 2021-03-10T15:36:06.937214+0000 mgr.gibba032.whopeo (mgr.24625) 5462 : cluster [DBG] pgmap v3108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:08 gibba032 bash[24004]: cluster 2021-03-10T15:36:06.937214+0000 mgr.gibba032.whopeo (mgr.24625) 5462 : cluster [DBG] pgmap v3108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:10.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: cluster 2021-03-10T15:36:08.939027+0000 mgr.gibba032.whopeo (mgr.24625) 5463 : cluster [DBG] pgmap v3109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:10.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.679373+0000 mon.a (mon.0) 2102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.683774+0000 mon.a (mon.0) 2103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.886210+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.889616+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:36:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.891018+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.892277+0000 mon.a (mon.0) 2107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.893450+0000 mon.a (mon.0) 2108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.895511+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:10 gibba032 bash[24004]: audit 2021-03-10T15:36:09.896984+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: cluster 2021-03-10T15:36:08.939027+0000 mgr.gibba032.whopeo (mgr.24625) 5463 : cluster [DBG] pgmap v3109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.679373+0000 mon.a (mon.0) 2102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.683774+0000 mon.a (mon.0) 2103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.886210+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.889616+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:36:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.891018+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.892277+0000 mon.a (mon.0) 2107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.893450+0000 mon.a (mon.0) 2108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.895511+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[30791]: audit 2021-03-10T15:36:09.896984+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: cluster 2021-03-10T15:36:08.939027+0000 mgr.gibba032.whopeo (mgr.24625) 5463 : cluster [DBG] pgmap v3109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:10.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.679373+0000 mon.a (mon.0) 2102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.683774+0000 mon.a (mon.0) 2103 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.886210+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:36:10.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.889616+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:36:10.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.891018+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.892277+0000 mon.a (mon.0) 2107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.893450+0000 mon.a (mon.0) 2108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.895511+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:10.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:10 gibba011 bash[31774]: audit 2021-03-10T15:36:09.896984+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.679581+0000 mgr.gibba032.whopeo (mgr.24625) 5464 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:36:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.683382+0000 mgr.gibba032.whopeo (mgr.24625) 5465 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:36:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.887778+0000 mgr.gibba032.whopeo (mgr.24625) 5466 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:36:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.889619+0000 mgr.gibba032.whopeo (mgr.24625) 5467 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:36:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.890842+0000 mgr.gibba032.whopeo (mgr.24625) 5468 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:36:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.890882+0000 mgr.gibba032.whopeo (mgr.24625) 5469 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:36:11.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.892087+0000 mgr.gibba032.whopeo (mgr.24625) 5470 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:36:11.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.892128+0000 mgr.gibba032.whopeo (mgr.24625) 5471 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:36:11.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.893055+0000 mgr.gibba032.whopeo (mgr.24625) 5472 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:36:11.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.894113+0000 mgr.gibba032.whopeo (mgr.24625) 5473 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:36:11.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.894156+0000 mgr.gibba032.whopeo (mgr.24625) 5474 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:36:11.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.895309+0000 mgr.gibba032.whopeo (mgr.24625) 5475 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:36:11.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.895653+0000 mgr.gibba032.whopeo (mgr.24625) 5476 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:36:11.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.896779+0000 mgr.gibba032.whopeo (mgr.24625) 5477 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:36:11.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.896818+0000 mgr.gibba032.whopeo (mgr.24625) 5478 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:36:11.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[30791]: cephadm 2021-03-10T15:36:09.896932+0000 mgr.gibba032.whopeo (mgr.24625) 5479 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.679581+0000 mgr.gibba032.whopeo (mgr.24625) 5464 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:36:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.683382+0000 mgr.gibba032.whopeo (mgr.24625) 5465 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:36:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.887778+0000 mgr.gibba032.whopeo (mgr.24625) 5466 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:36:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.889619+0000 mgr.gibba032.whopeo (mgr.24625) 5467 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:36:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.890842+0000 mgr.gibba032.whopeo (mgr.24625) 5468 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:36:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.890882+0000 mgr.gibba032.whopeo (mgr.24625) 5469 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:36:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.892087+0000 mgr.gibba032.whopeo (mgr.24625) 5470 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:36:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.892128+0000 mgr.gibba032.whopeo (mgr.24625) 5471 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:36:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.893055+0000 mgr.gibba032.whopeo (mgr.24625) 5472 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:36:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.894113+0000 mgr.gibba032.whopeo (mgr.24625) 5473 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:36:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.894156+0000 mgr.gibba032.whopeo (mgr.24625) 5474 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:36:11.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.895309+0000 mgr.gibba032.whopeo (mgr.24625) 5475 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:36:11.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.895653+0000 mgr.gibba032.whopeo (mgr.24625) 5476 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:36:11.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.896779+0000 mgr.gibba032.whopeo (mgr.24625) 5477 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:36:11.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.896818+0000 mgr.gibba032.whopeo (mgr.24625) 5478 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:36:11.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:11 gibba011 bash[31774]: cephadm 2021-03-10T15:36:09.896932+0000 mgr.gibba032.whopeo (mgr.24625) 5479 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:11.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.679581+0000 mgr.gibba032.whopeo (mgr.24625) 5464 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:36:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.683382+0000 mgr.gibba032.whopeo (mgr.24625) 5465 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:36:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.887778+0000 mgr.gibba032.whopeo (mgr.24625) 5466 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:36:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.889619+0000 mgr.gibba032.whopeo (mgr.24625) 5467 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:36:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.890842+0000 mgr.gibba032.whopeo (mgr.24625) 5468 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:36:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.890882+0000 mgr.gibba032.whopeo (mgr.24625) 5469 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:36:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.892087+0000 mgr.gibba032.whopeo (mgr.24625) 5470 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:36:11.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.892128+0000 mgr.gibba032.whopeo (mgr.24625) 5471 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:36:11.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.893055+0000 mgr.gibba032.whopeo (mgr.24625) 5472 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:36:11.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.894113+0000 mgr.gibba032.whopeo (mgr.24625) 5473 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:36:11.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.894156+0000 mgr.gibba032.whopeo (mgr.24625) 5474 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:36:11.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.895309+0000 mgr.gibba032.whopeo (mgr.24625) 5475 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:36:11.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.895653+0000 mgr.gibba032.whopeo (mgr.24625) 5476 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:36:11.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.896779+0000 mgr.gibba032.whopeo (mgr.24625) 5477 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:36:11.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.896818+0000 mgr.gibba032.whopeo (mgr.24625) 5478 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:36:11.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:11 gibba032 bash[24004]: cephadm 2021-03-10T15:36:09.896932+0000 mgr.gibba032.whopeo (mgr.24625) 5479 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:12 gibba011 bash[30791]: cluster 2021-03-10T15:36:10.940493+0000 mgr.gibba032.whopeo (mgr.24625) 5480 : cluster [DBG] pgmap v3110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:12 gibba011 bash[31774]: cluster 2021-03-10T15:36:10.940493+0000 mgr.gibba032.whopeo (mgr.24625) 5480 : cluster [DBG] pgmap v3110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:12 gibba032 bash[24004]: cluster 2021-03-10T15:36:10.940493+0000 mgr.gibba032.whopeo (mgr.24625) 5480 : cluster [DBG] pgmap v3110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:14 gibba011 bash[30791]: cluster 2021-03-10T15:36:12.942047+0000 mgr.gibba032.whopeo (mgr.24625) 5481 : cluster [DBG] pgmap v3111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:14 gibba011 bash[31774]: cluster 2021-03-10T15:36:12.942047+0000 mgr.gibba032.whopeo (mgr.24625) 5481 : cluster [DBG] pgmap v3111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:14 gibba032 bash[24004]: cluster 2021-03-10T15:36:12.942047+0000 mgr.gibba032.whopeo (mgr.24625) 5481 : cluster [DBG] pgmap v3111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:16 gibba011 bash[30791]: cluster 2021-03-10T15:36:14.943587+0000 mgr.gibba032.whopeo (mgr.24625) 5482 : cluster [DBG] pgmap v3112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:16 gibba011 bash[31774]: cluster 2021-03-10T15:36:14.943587+0000 mgr.gibba032.whopeo (mgr.24625) 5482 : cluster [DBG] pgmap v3112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:16 gibba032 bash[24004]: cluster 2021-03-10T15:36:14.943587+0000 mgr.gibba032.whopeo (mgr.24625) 5482 : cluster [DBG] pgmap v3112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:18.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:18 gibba011 bash[30791]: cluster 2021-03-10T15:36:16.945067+0000 mgr.gibba032.whopeo (mgr.24625) 5483 : cluster [DBG] pgmap v3113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:18.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:18 gibba011 bash[31774]: cluster 2021-03-10T15:36:16.945067+0000 mgr.gibba032.whopeo (mgr.24625) 5483 : cluster [DBG] pgmap v3113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:18 gibba032 bash[24004]: cluster 2021-03-10T15:36:16.945067+0000 mgr.gibba032.whopeo (mgr.24625) 5483 : cluster [DBG] pgmap v3113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:19.888 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:36:20.117 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:36:20.117 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (102m) 7m ago 108m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:36:20.117 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (108m) 6m ago 108m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:36:20.118 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (103m) 6m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:36:20.118 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (103m) 6m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:36:20.118 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (102m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:36:20.118 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (102m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:36:20.118 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (102m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:36:20.119 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (102m) 7m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:36:20.119 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (108m) 7m ago 108m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:36:20.119 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (108m) 6m ago 108m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:36:20.119 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (101m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:36:20.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (100m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:36:20.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (100m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:36:20.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (99m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:36:20.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (99m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:36:20.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (99m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:36:20.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (98m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:36:20.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (98m) 6m ago 108m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:36:20.121 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (107m) 6m ago 108m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:36:20.121 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 106m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:36:20.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:20 gibba011 bash[30791]: cluster 2021-03-10T15:36:18.946690+0000 mgr.gibba032.whopeo (mgr.24625) 5484 : cluster [DBG] pgmap v3114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:20.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:20 gibba011 bash[31774]: cluster 2021-03-10T15:36:18.946690+0000 mgr.gibba032.whopeo (mgr.24625) 5484 : cluster [DBG] pgmap v3114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:20.366 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:36:20.366 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:36:20.366 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:36:20.367 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:20.367 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:36:20.367 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:36:20.367 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:20.368 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:36:20.368 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:36:20.368 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:20.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:36:20.369 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:36:20.369 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:36:20.369 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:36:20.370 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:36:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:20 gibba032 bash[24004]: cluster 2021-03-10T15:36:18.946690+0000 mgr.gibba032.whopeo (mgr.24625) 5484 : cluster [DBG] pgmap v3114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:21.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[30791]: audit 2021-03-10T15:36:19.874663+0000 mgr.gibba032.whopeo (mgr.24625) 5485 : audit [DBG] from='client.47727 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[30791]: audit 2021-03-10T15:36:20.100961+0000 mgr.gibba032.whopeo (mgr.24625) 5486 : audit [DBG] from='client.57718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[30791]: audit 2021-03-10T15:36:20.362605+0000 mon.a (mon.0) 2111 : audit [DBG] from='client.? 172.21.2.111:0/1533422686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:21.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[31774]: audit 2021-03-10T15:36:19.874663+0000 mgr.gibba032.whopeo (mgr.24625) 5485 : audit [DBG] from='client.47727 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[31774]: audit 2021-03-10T15:36:20.100961+0000 mgr.gibba032.whopeo (mgr.24625) 5486 : audit [DBG] from='client.57718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:21 gibba011 bash[31774]: audit 2021-03-10T15:36:20.362605+0000 mon.a (mon.0) 2111 : audit [DBG] from='client.? 172.21.2.111:0/1533422686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:21 gibba032 bash[24004]: audit 2021-03-10T15:36:19.874663+0000 mgr.gibba032.whopeo (mgr.24625) 5485 : audit [DBG] from='client.47727 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:21 gibba032 bash[24004]: audit 2021-03-10T15:36:20.100961+0000 mgr.gibba032.whopeo (mgr.24625) 5486 : audit [DBG] from='client.57718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:21.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:21 gibba032 bash[24004]: audit 2021-03-10T15:36:20.362605+0000 mon.a (mon.0) 2111 : audit [DBG] from='client.? 172.21.2.111:0/1533422686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:22 gibba032 bash[24004]: cluster 2021-03-10T15:36:20.948114+0000 mgr.gibba032.whopeo (mgr.24625) 5487 : cluster [DBG] pgmap v3115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:22 gibba011 bash[30791]: cluster 2021-03-10T15:36:20.948114+0000 mgr.gibba032.whopeo (mgr.24625) 5487 : cluster [DBG] pgmap v3115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:22 gibba011 bash[31774]: cluster 2021-03-10T15:36:20.948114+0000 mgr.gibba032.whopeo (mgr.24625) 5487 : cluster [DBG] pgmap v3115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:24 gibba032 bash[24004]: cluster 2021-03-10T15:36:22.949888+0000 mgr.gibba032.whopeo (mgr.24625) 5488 : cluster [DBG] pgmap v3116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:24 gibba011 bash[30791]: cluster 2021-03-10T15:36:22.949888+0000 mgr.gibba032.whopeo (mgr.24625) 5488 : cluster [DBG] pgmap v3116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:24 gibba011 bash[31774]: cluster 2021-03-10T15:36:22.949888+0000 mgr.gibba032.whopeo (mgr.24625) 5488 : cluster [DBG] pgmap v3116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:26 gibba032 bash[24004]: cephadm 2021-03-10T15:36:24.912960+0000 mgr.gibba032.whopeo (mgr.24625) 5489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:26 gibba032 bash[24004]: cluster 2021-03-10T15:36:24.951382+0000 mgr.gibba032.whopeo (mgr.24625) 5490 : cluster [DBG] pgmap v3117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:26 gibba011 bash[30791]: cephadm 2021-03-10T15:36:24.912960+0000 mgr.gibba032.whopeo (mgr.24625) 5489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:26.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:26 gibba011 bash[30791]: cluster 2021-03-10T15:36:24.951382+0000 mgr.gibba032.whopeo (mgr.24625) 5490 : cluster [DBG] pgmap v3117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:26 gibba011 bash[31774]: cephadm 2021-03-10T15:36:24.912960+0000 mgr.gibba032.whopeo (mgr.24625) 5489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:26 gibba011 bash[31774]: cluster 2021-03-10T15:36:24.951382+0000 mgr.gibba032.whopeo (mgr.24625) 5490 : cluster [DBG] pgmap v3117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:28 gibba032 bash[24004]: cluster 2021-03-10T15:36:26.953105+0000 mgr.gibba032.whopeo (mgr.24625) 5491 : cluster [DBG] pgmap v3118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:28 gibba011 bash[30791]: cluster 2021-03-10T15:36:26.953105+0000 mgr.gibba032.whopeo (mgr.24625) 5491 : cluster [DBG] pgmap v3118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:28 gibba011 bash[31774]: cluster 2021-03-10T15:36:26.953105+0000 mgr.gibba032.whopeo (mgr.24625) 5491 : cluster [DBG] pgmap v3118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:30 gibba032 bash[24004]: cluster 2021-03-10T15:36:28.954719+0000 mgr.gibba032.whopeo (mgr.24625) 5492 : cluster [DBG] pgmap v3119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:30 gibba011 bash[30791]: cluster 2021-03-10T15:36:28.954719+0000 mgr.gibba032.whopeo (mgr.24625) 5492 : cluster [DBG] pgmap v3119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:30 gibba011 bash[31774]: cluster 2021-03-10T15:36:28.954719+0000 mgr.gibba032.whopeo (mgr.24625) 5492 : cluster [DBG] pgmap v3119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:32 gibba032 bash[24004]: cluster 2021-03-10T15:36:30.956112+0000 mgr.gibba032.whopeo (mgr.24625) 5493 : cluster [DBG] pgmap v3120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:32 gibba011 bash[30791]: cluster 2021-03-10T15:36:30.956112+0000 mgr.gibba032.whopeo (mgr.24625) 5493 : cluster [DBG] pgmap v3120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:32 gibba011 bash[31774]: cluster 2021-03-10T15:36:30.956112+0000 mgr.gibba032.whopeo (mgr.24625) 5493 : cluster [DBG] pgmap v3120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:34 gibba032 bash[24004]: cluster 2021-03-10T15:36:32.957849+0000 mgr.gibba032.whopeo (mgr.24625) 5494 : cluster [DBG] pgmap v3121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:34 gibba011 bash[30791]: cluster 2021-03-10T15:36:32.957849+0000 mgr.gibba032.whopeo (mgr.24625) 5494 : cluster [DBG] pgmap v3121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:34 gibba011 bash[31774]: cluster 2021-03-10T15:36:32.957849+0000 mgr.gibba032.whopeo (mgr.24625) 5494 : cluster [DBG] pgmap v3121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:36 gibba032 bash[24004]: cluster 2021-03-10T15:36:34.958966+0000 mgr.gibba032.whopeo (mgr.24625) 5495 : cluster [DBG] pgmap v3122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:36 gibba011 bash[30791]: cluster 2021-03-10T15:36:34.958966+0000 mgr.gibba032.whopeo (mgr.24625) 5495 : cluster [DBG] pgmap v3122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:36 gibba011 bash[31774]: cluster 2021-03-10T15:36:34.958966+0000 mgr.gibba032.whopeo (mgr.24625) 5495 : cluster [DBG] pgmap v3122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:38 gibba032 bash[24004]: cluster 2021-03-10T15:36:36.960490+0000 mgr.gibba032.whopeo (mgr.24625) 5496 : cluster [DBG] pgmap v3123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:38 gibba011 bash[30791]: cluster 2021-03-10T15:36:36.960490+0000 mgr.gibba032.whopeo (mgr.24625) 5496 : cluster [DBG] pgmap v3123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:38 gibba011 bash[31774]: cluster 2021-03-10T15:36:36.960490+0000 mgr.gibba032.whopeo (mgr.24625) 5496 : cluster [DBG] pgmap v3123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:40.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:40 gibba032 bash[24004]: cluster 2021-03-10T15:36:38.962088+0000 mgr.gibba032.whopeo (mgr.24625) 5497 : cluster [DBG] pgmap v3124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:40 gibba011 bash[30791]: cluster 2021-03-10T15:36:38.962088+0000 mgr.gibba032.whopeo (mgr.24625) 5497 : cluster [DBG] pgmap v3124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:40 gibba011 bash[31774]: cluster 2021-03-10T15:36:38.962088+0000 mgr.gibba032.whopeo (mgr.24625) 5497 : cluster [DBG] pgmap v3124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:41 gibba032 bash[24004]: cephadm 2021-03-10T15:36:39.929002+0000 mgr.gibba032.whopeo (mgr.24625) 5498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:41.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:41 gibba011 bash[30791]: cephadm 2021-03-10T15:36:39.929002+0000 mgr.gibba032.whopeo (mgr.24625) 5498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:41.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:41 gibba011 bash[31774]: cephadm 2021-03-10T15:36:39.929002+0000 mgr.gibba032.whopeo (mgr.24625) 5498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:42 gibba032 bash[24004]: cluster 2021-03-10T15:36:40.963477+0000 mgr.gibba032.whopeo (mgr.24625) 5499 : cluster [DBG] pgmap v3125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:42 gibba011 bash[30791]: cluster 2021-03-10T15:36:40.963477+0000 mgr.gibba032.whopeo (mgr.24625) 5499 : cluster [DBG] pgmap v3125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:42 gibba011 bash[31774]: cluster 2021-03-10T15:36:40.963477+0000 mgr.gibba032.whopeo (mgr.24625) 5499 : cluster [DBG] pgmap v3125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:44 gibba032 bash[24004]: cluster 2021-03-10T15:36:42.965505+0000 mgr.gibba032.whopeo (mgr.24625) 5500 : cluster [DBG] pgmap v3126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:44 gibba011 bash[30791]: cluster 2021-03-10T15:36:42.965505+0000 mgr.gibba032.whopeo (mgr.24625) 5500 : cluster [DBG] pgmap v3126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:44.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:44 gibba011 bash[31774]: cluster 2021-03-10T15:36:42.965505+0000 mgr.gibba032.whopeo (mgr.24625) 5500 : cluster [DBG] pgmap v3126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:46 gibba032 bash[24004]: cluster 2021-03-10T15:36:44.966794+0000 mgr.gibba032.whopeo (mgr.24625) 5501 : cluster [DBG] pgmap v3127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:46 gibba011 bash[30791]: cluster 2021-03-10T15:36:44.966794+0000 mgr.gibba032.whopeo (mgr.24625) 5501 : cluster [DBG] pgmap v3127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:46 gibba011 bash[31774]: cluster 2021-03-10T15:36:44.966794+0000 mgr.gibba032.whopeo (mgr.24625) 5501 : cluster [DBG] pgmap v3127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:48 gibba032 bash[24004]: cluster 2021-03-10T15:36:46.968541+0000 mgr.gibba032.whopeo (mgr.24625) 5502 : cluster [DBG] pgmap v3128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:48 gibba011 bash[30791]: cluster 2021-03-10T15:36:46.968541+0000 mgr.gibba032.whopeo (mgr.24625) 5502 : cluster [DBG] pgmap v3128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:48 gibba011 bash[31774]: cluster 2021-03-10T15:36:46.968541+0000 mgr.gibba032.whopeo (mgr.24625) 5502 : cluster [DBG] pgmap v3128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:50 gibba032 bash[24004]: cluster 2021-03-10T15:36:48.970304+0000 mgr.gibba032.whopeo (mgr.24625) 5503 : cluster [DBG] pgmap v3129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:50 gibba011 bash[30791]: cluster 2021-03-10T15:36:48.970304+0000 mgr.gibba032.whopeo (mgr.24625) 5503 : cluster [DBG] pgmap v3129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:50 gibba011 bash[31774]: cluster 2021-03-10T15:36:48.970304+0000 mgr.gibba032.whopeo (mgr.24625) 5503 : cluster [DBG] pgmap v3129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:50.616 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:36:50.858 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:36:50.859 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (103m) 7m ago 108m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:36:50.859 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (108m) 6m ago 108m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:36:50.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (103m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:36:50.860 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (104m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:36:50.860 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (103m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:36:50.860 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (103m) 7m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:36:50.860 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (102m) 6m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:36:50.860 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (102m) 7m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:36:50.861 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (108m) 7m ago 109m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:36:50.861 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (108m) 6m ago 109m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:36:50.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (101m) 7m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:36:50.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (101m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:36:50.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (100m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:36:50.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (100m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:36:50.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (100m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:36:50.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (99m) 6m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:36:50.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (99m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:36:50.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (98m) 6m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:36:50.863 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (108m) 6m ago 109m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:36:50.863 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 106m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:36:51.107 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:36:51.107 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:36:51.108 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:36:51.108 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:51.108 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:36:51.109 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:36:51.109 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:51.109 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:36:51.110 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:36:51.110 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:36:51.110 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:36:51.111 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:36:51.111 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:36:51.111 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:36:51.111 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:36:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:51 gibba032 bash[24004]: audit 2021-03-10T15:36:50.593903+0000 mgr.gibba032.whopeo (mgr.24625) 5504 : audit [DBG] from='client.47745 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:51 gibba032 bash[24004]: audit 2021-03-10T15:36:51.103677+0000 mon.a (mon.0) 2112 : audit [DBG] from='client.? 172.21.2.111:0/1493513080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:51 gibba011 bash[30791]: audit 2021-03-10T15:36:50.593903+0000 mgr.gibba032.whopeo (mgr.24625) 5504 : audit [DBG] from='client.47745 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:51 gibba011 bash[30791]: audit 2021-03-10T15:36:51.103677+0000 mon.a (mon.0) 2112 : audit [DBG] from='client.? 172.21.2.111:0/1493513080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:51.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:51 gibba011 bash[31774]: audit 2021-03-10T15:36:50.593903+0000 mgr.gibba032.whopeo (mgr.24625) 5504 : audit [DBG] from='client.47745 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:51.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:51 gibba011 bash[31774]: audit 2021-03-10T15:36:51.103677+0000 mon.a (mon.0) 2112 : audit [DBG] from='client.? 172.21.2.111:0/1493513080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:36:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:52 gibba032 bash[24004]: audit 2021-03-10T15:36:50.842097+0000 mgr.gibba032.whopeo (mgr.24625) 5505 : audit [DBG] from='client.57736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:52 gibba032 bash[24004]: cluster 2021-03-10T15:36:50.971736+0000 mgr.gibba032.whopeo (mgr.24625) 5506 : cluster [DBG] pgmap v3130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:52 gibba011 bash[30791]: audit 2021-03-10T15:36:50.842097+0000 mgr.gibba032.whopeo (mgr.24625) 5505 : audit [DBG] from='client.57736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:52 gibba011 bash[30791]: cluster 2021-03-10T15:36:50.971736+0000 mgr.gibba032.whopeo (mgr.24625) 5506 : cluster [DBG] pgmap v3130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:52 gibba011 bash[31774]: audit 2021-03-10T15:36:50.842097+0000 mgr.gibba032.whopeo (mgr.24625) 5505 : audit [DBG] from='client.57736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:36:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:52 gibba011 bash[31774]: cluster 2021-03-10T15:36:50.971736+0000 mgr.gibba032.whopeo (mgr.24625) 5506 : cluster [DBG] pgmap v3130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:54 gibba032 bash[24004]: cluster 2021-03-10T15:36:52.973644+0000 mgr.gibba032.whopeo (mgr.24625) 5507 : cluster [DBG] pgmap v3131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:54 gibba011 bash[30791]: cluster 2021-03-10T15:36:52.973644+0000 mgr.gibba032.whopeo (mgr.24625) 5507 : cluster [DBG] pgmap v3131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:54.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:54 gibba011 bash[31774]: cluster 2021-03-10T15:36:52.973644+0000 mgr.gibba032.whopeo (mgr.24625) 5507 : cluster [DBG] pgmap v3131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:56 gibba011 bash[30791]: cephadm 2021-03-10T15:36:54.944931+0000 mgr.gibba032.whopeo (mgr.24625) 5508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:56.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:56 gibba011 bash[30791]: cluster 2021-03-10T15:36:54.975030+0000 mgr.gibba032.whopeo (mgr.24625) 5509 : cluster [DBG] pgmap v3132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:56 gibba011 bash[31774]: cephadm 2021-03-10T15:36:54.944931+0000 mgr.gibba032.whopeo (mgr.24625) 5508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:56 gibba011 bash[31774]: cluster 2021-03-10T15:36:54.975030+0000 mgr.gibba032.whopeo (mgr.24625) 5509 : cluster [DBG] pgmap v3132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:56 gibba032 bash[24004]: cephadm 2021-03-10T15:36:54.944931+0000 mgr.gibba032.whopeo (mgr.24625) 5508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:36:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:56 gibba032 bash[24004]: cluster 2021-03-10T15:36:54.975030+0000 mgr.gibba032.whopeo (mgr.24625) 5509 : cluster [DBG] pgmap v3132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.653721+0000 mon.a (mon.0) 2113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:36:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.654388+0000 mon.a (mon.0) 2114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:36:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.673242+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:36:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.674033+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:36:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.674703+0000 mon.a (mon.0) 2117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:36:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[30791]: audit 2021-03-10T15:36:56.675435+0000 mon.a (mon.0) 2118 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:36:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.653721+0000 mon.a (mon.0) 2113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:36:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.654388+0000 mon.a (mon.0) 2114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:36:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.673242+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:36:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.674033+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:36:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.674703+0000 mon.a (mon.0) 2117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:36:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:57 gibba011 bash[31774]: audit 2021-03-10T15:36:56.675435+0000 mon.a (mon.0) 2118 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:36:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.653721+0000 mon.a (mon.0) 2113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:36:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.654388+0000 mon.a (mon.0) 2114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:36:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.673242+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:36:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.674033+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:36:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.674703+0000 mon.a (mon.0) 2117 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:36:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:57 gibba032 bash[24004]: audit 2021-03-10T15:36:56.675435+0000 mon.a (mon.0) 2118 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:36:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:36:58 gibba011 bash[30791]: cluster 2021-03-10T15:36:56.976684+0000 mgr.gibba032.whopeo (mgr.24625) 5510 : cluster [DBG] pgmap v3133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:36:58 gibba011 bash[31774]: cluster 2021-03-10T15:36:56.976684+0000 mgr.gibba032.whopeo (mgr.24625) 5510 : cluster [DBG] pgmap v3133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:36:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:36:58 gibba032 bash[24004]: cluster 2021-03-10T15:36:56.976684+0000 mgr.gibba032.whopeo (mgr.24625) 5510 : cluster [DBG] pgmap v3133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:00 gibba011 bash[30791]: cluster 2021-03-10T15:36:58.978294+0000 mgr.gibba032.whopeo (mgr.24625) 5511 : cluster [DBG] pgmap v3134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:00 gibba011 bash[31774]: cluster 2021-03-10T15:36:58.978294+0000 mgr.gibba032.whopeo (mgr.24625) 5511 : cluster [DBG] pgmap v3134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:00 gibba032 bash[24004]: cluster 2021-03-10T15:36:58.978294+0000 mgr.gibba032.whopeo (mgr.24625) 5511 : cluster [DBG] pgmap v3134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:02 gibba011 bash[30791]: cluster 2021-03-10T15:37:00.979626+0000 mgr.gibba032.whopeo (mgr.24625) 5512 : cluster [DBG] pgmap v3135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:02 gibba011 bash[31774]: cluster 2021-03-10T15:37:00.979626+0000 mgr.gibba032.whopeo (mgr.24625) 5512 : cluster [DBG] pgmap v3135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:02 gibba032 bash[24004]: cluster 2021-03-10T15:37:00.979626+0000 mgr.gibba032.whopeo (mgr.24625) 5512 : cluster [DBG] pgmap v3135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:04 gibba011 bash[30791]: cluster 2021-03-10T15:37:02.981526+0000 mgr.gibba032.whopeo (mgr.24625) 5513 : cluster [DBG] pgmap v3136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:04 gibba011 bash[31774]: cluster 2021-03-10T15:37:02.981526+0000 mgr.gibba032.whopeo (mgr.24625) 5513 : cluster [DBG] pgmap v3136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:04.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:04 gibba032 bash[24004]: cluster 2021-03-10T15:37:02.981526+0000 mgr.gibba032.whopeo (mgr.24625) 5513 : cluster [DBG] pgmap v3136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:06 gibba011 bash[30791]: cluster 2021-03-10T15:37:04.982883+0000 mgr.gibba032.whopeo (mgr.24625) 5514 : cluster [DBG] pgmap v3137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:06 gibba011 bash[31774]: cluster 2021-03-10T15:37:04.982883+0000 mgr.gibba032.whopeo (mgr.24625) 5514 : cluster [DBG] pgmap v3137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:06 gibba032 bash[24004]: cluster 2021-03-10T15:37:04.982883+0000 mgr.gibba032.whopeo (mgr.24625) 5514 : cluster [DBG] pgmap v3137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:08 gibba011 bash[30791]: cluster 2021-03-10T15:37:06.984654+0000 mgr.gibba032.whopeo (mgr.24625) 5515 : cluster [DBG] pgmap v3138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:08 gibba011 bash[31774]: cluster 2021-03-10T15:37:06.984654+0000 mgr.gibba032.whopeo (mgr.24625) 5515 : cluster [DBG] pgmap v3138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:08 gibba032 bash[24004]: cluster 2021-03-10T15:37:06.984654+0000 mgr.gibba032.whopeo (mgr.24625) 5515 : cluster [DBG] pgmap v3138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: cluster 2021-03-10T15:37:08.986107+0000 mgr.gibba032.whopeo (mgr.24625) 5516 : cluster [DBG] pgmap v3139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.173712+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:37:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.176771+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:37:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.178431+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.180315+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.181853+0000 mon.a (mon.0) 2123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.184710+0000 mon.a (mon.0) 2124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:10 gibba032 bash[24004]: audit 2021-03-10T15:37:10.186388+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: cluster 2021-03-10T15:37:08.986107+0000 mgr.gibba032.whopeo (mgr.24625) 5516 : cluster [DBG] pgmap v3139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:10.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.173712+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:37:10.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.176771+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:37:10.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.178431+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.180315+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.181853+0000 mon.a (mon.0) 2123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.184710+0000 mon.a (mon.0) 2124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[30791]: audit 2021-03-10T15:37:10.186388+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: cluster 2021-03-10T15:37:08.986107+0000 mgr.gibba032.whopeo (mgr.24625) 5516 : cluster [DBG] pgmap v3139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:10.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.173712+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:37:10.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.176771+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:37:10.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.178431+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.180315+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.181853+0000 mon.a (mon.0) 2123 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.184710+0000 mon.a (mon.0) 2124 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:10.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:10 gibba011 bash[31774]: audit 2021-03-10T15:37:10.186388+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:09.961744+0000 mgr.gibba032.whopeo (mgr.24625) 5517 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:37:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:09.963320+0000 mgr.gibba032.whopeo (mgr.24625) 5518 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:37:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.175059+0000 mgr.gibba032.whopeo (mgr.24625) 5519 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:37:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.176965+0000 mgr.gibba032.whopeo (mgr.24625) 5520 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:37:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.178797+0000 mgr.gibba032.whopeo (mgr.24625) 5521 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:37:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.178834+0000 mgr.gibba032.whopeo (mgr.24625) 5522 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:37:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.180351+0000 mgr.gibba032.whopeo (mgr.24625) 5523 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:37:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.180387+0000 mgr.gibba032.whopeo (mgr.24625) 5524 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:37:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.181810+0000 mgr.gibba032.whopeo (mgr.24625) 5525 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:37:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.183111+0000 mgr.gibba032.whopeo (mgr.24625) 5526 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:37:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.183148+0000 mgr.gibba032.whopeo (mgr.24625) 5527 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:37:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.184667+0000 mgr.gibba032.whopeo (mgr.24625) 5528 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:37:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.184987+0000 mgr.gibba032.whopeo (mgr.24625) 5529 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:37:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.186780+0000 mgr.gibba032.whopeo (mgr.24625) 5530 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:37:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.186819+0000 mgr.gibba032.whopeo (mgr.24625) 5531 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:37:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:11 gibba032 bash[24004]: cephadm 2021-03-10T15:37:10.186922+0000 mgr.gibba032.whopeo (mgr.24625) 5532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:09.961744+0000 mgr.gibba032.whopeo (mgr.24625) 5517 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:37:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:09.963320+0000 mgr.gibba032.whopeo (mgr.24625) 5518 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:37:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.175059+0000 mgr.gibba032.whopeo (mgr.24625) 5519 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:37:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.176965+0000 mgr.gibba032.whopeo (mgr.24625) 5520 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:37:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.178797+0000 mgr.gibba032.whopeo (mgr.24625) 5521 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:37:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.178834+0000 mgr.gibba032.whopeo (mgr.24625) 5522 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:37:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.180351+0000 mgr.gibba032.whopeo (mgr.24625) 5523 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:37:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.180387+0000 mgr.gibba032.whopeo (mgr.24625) 5524 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:37:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.181810+0000 mgr.gibba032.whopeo (mgr.24625) 5525 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:37:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.183111+0000 mgr.gibba032.whopeo (mgr.24625) 5526 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:37:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.183148+0000 mgr.gibba032.whopeo (mgr.24625) 5527 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:37:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.184667+0000 mgr.gibba032.whopeo (mgr.24625) 5528 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:37:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.184987+0000 mgr.gibba032.whopeo (mgr.24625) 5529 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:37:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.186780+0000 mgr.gibba032.whopeo (mgr.24625) 5530 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:37:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.186819+0000 mgr.gibba032.whopeo (mgr.24625) 5531 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:37:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[30791]: cephadm 2021-03-10T15:37:10.186922+0000 mgr.gibba032.whopeo (mgr.24625) 5532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:09.961744+0000 mgr.gibba032.whopeo (mgr.24625) 5517 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:37:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:09.963320+0000 mgr.gibba032.whopeo (mgr.24625) 5518 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:37:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.175059+0000 mgr.gibba032.whopeo (mgr.24625) 5519 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:37:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.176965+0000 mgr.gibba032.whopeo (mgr.24625) 5520 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:37:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.178797+0000 mgr.gibba032.whopeo (mgr.24625) 5521 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:37:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.178834+0000 mgr.gibba032.whopeo (mgr.24625) 5522 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:37:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.180351+0000 mgr.gibba032.whopeo (mgr.24625) 5523 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:37:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.180387+0000 mgr.gibba032.whopeo (mgr.24625) 5524 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:37:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.181810+0000 mgr.gibba032.whopeo (mgr.24625) 5525 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:37:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.183111+0000 mgr.gibba032.whopeo (mgr.24625) 5526 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:37:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.183148+0000 mgr.gibba032.whopeo (mgr.24625) 5527 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:37:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.184667+0000 mgr.gibba032.whopeo (mgr.24625) 5528 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:37:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.184987+0000 mgr.gibba032.whopeo (mgr.24625) 5529 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:37:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.186780+0000 mgr.gibba032.whopeo (mgr.24625) 5530 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:37:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.186819+0000 mgr.gibba032.whopeo (mgr.24625) 5531 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:37:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:11 gibba011 bash[31774]: cephadm 2021-03-10T15:37:10.186922+0000 mgr.gibba032.whopeo (mgr.24625) 5532 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:12 gibba032 bash[24004]: cluster 2021-03-10T15:37:10.987492+0000 mgr.gibba032.whopeo (mgr.24625) 5533 : cluster [DBG] pgmap v3140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:12 gibba011 bash[30791]: cluster 2021-03-10T15:37:10.987492+0000 mgr.gibba032.whopeo (mgr.24625) 5533 : cluster [DBG] pgmap v3140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:12.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:12 gibba011 bash[31774]: cluster 2021-03-10T15:37:10.987492+0000 mgr.gibba032.whopeo (mgr.24625) 5533 : cluster [DBG] pgmap v3140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:14 gibba032 bash[24004]: cluster 2021-03-10T15:37:12.989528+0000 mgr.gibba032.whopeo (mgr.24625) 5534 : cluster [DBG] pgmap v3141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:14.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:14 gibba011 bash[30791]: cluster 2021-03-10T15:37:12.989528+0000 mgr.gibba032.whopeo (mgr.24625) 5534 : cluster [DBG] pgmap v3141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:14.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:14 gibba011 bash[31774]: cluster 2021-03-10T15:37:12.989528+0000 mgr.gibba032.whopeo (mgr.24625) 5534 : cluster [DBG] pgmap v3141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:16 gibba032 bash[24004]: cluster 2021-03-10T15:37:14.990876+0000 mgr.gibba032.whopeo (mgr.24625) 5535 : cluster [DBG] pgmap v3142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:16 gibba011 bash[30791]: cluster 2021-03-10T15:37:14.990876+0000 mgr.gibba032.whopeo (mgr.24625) 5535 : cluster [DBG] pgmap v3142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:16 gibba011 bash[31774]: cluster 2021-03-10T15:37:14.990876+0000 mgr.gibba032.whopeo (mgr.24625) 5535 : cluster [DBG] pgmap v3142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:18 gibba032 bash[24004]: cluster 2021-03-10T15:37:16.992472+0000 mgr.gibba032.whopeo (mgr.24625) 5536 : cluster [DBG] pgmap v3143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:18 gibba011 bash[30791]: cluster 2021-03-10T15:37:16.992472+0000 mgr.gibba032.whopeo (mgr.24625) 5536 : cluster [DBG] pgmap v3143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:18 gibba011 bash[31774]: cluster 2021-03-10T15:37:16.992472+0000 mgr.gibba032.whopeo (mgr.24625) 5536 : cluster [DBG] pgmap v3143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:20 gibba032 bash[24004]: cluster 2021-03-10T15:37:18.993860+0000 mgr.gibba032.whopeo (mgr.24625) 5537 : cluster [DBG] pgmap v3144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:20 gibba011 bash[30791]: cluster 2021-03-10T15:37:18.993860+0000 mgr.gibba032.whopeo (mgr.24625) 5537 : cluster [DBG] pgmap v3144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:20 gibba011 bash[31774]: cluster 2021-03-10T15:37:18.993860+0000 mgr.gibba032.whopeo (mgr.24625) 5537 : cluster [DBG] pgmap v3144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:21.351 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:37:21.579 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:37:21.580 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (104m) 8m ago 109m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:37:21.580 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (109m) 7m ago 109m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:37:21.581 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (104m) 7m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:37:21.581 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (104m) 7m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:37:21.582 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (103m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:37:21.582 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (103m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:37:21.583 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (103m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:37:21.583 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (103m) 8m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:37:21.584 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (109m) 8m ago 109m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:37:21.584 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (109m) 7m ago 109m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:37:21.584 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (102m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:37:21.585 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (101m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:37:21.585 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (101m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:37:21.586 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (101m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:37:21.587 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (100m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:37:21.587 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (100m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:37:21.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (99m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:37:21.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (99m) 7m ago 109m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:37:21.588 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (108m) 7m ago 109m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:37:21.589 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 107m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:37:21.831 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:37:21.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:37:21.832 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:37:21.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:21.832 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:37:21.833 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:37:21.833 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:21.833 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:37:21.833 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:37:21.834 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:21.834 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:37:21.834 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:37:21.834 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:37:21.835 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:37:21.835 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:37:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:22 gibba032 bash[24004]: cluster 2021-03-10T15:37:20.995216+0000 mgr.gibba032.whopeo (mgr.24625) 5538 : cluster [DBG] pgmap v3145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:22 gibba032 bash[24004]: audit 2021-03-10T15:37:21.337654+0000 mgr.gibba032.whopeo (mgr.24625) 5539 : audit [DBG] from='client.47763 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:22 gibba032 bash[24004]: audit 2021-03-10T15:37:21.563112+0000 mgr.gibba032.whopeo (mgr.24625) 5540 : audit [DBG] from='client.47769 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:22 gibba032 bash[24004]: audit 2021-03-10T15:37:21.827732+0000 mon.c (mon.1) 105 : audit [DBG] from='client.? 172.21.2.111:0/3904749489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[30791]: cluster 2021-03-10T15:37:20.995216+0000 mgr.gibba032.whopeo (mgr.24625) 5538 : cluster [DBG] pgmap v3145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[30791]: audit 2021-03-10T15:37:21.337654+0000 mgr.gibba032.whopeo (mgr.24625) 5539 : audit [DBG] from='client.47763 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[30791]: audit 2021-03-10T15:37:21.563112+0000 mgr.gibba032.whopeo (mgr.24625) 5540 : audit [DBG] from='client.47769 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[30791]: audit 2021-03-10T15:37:21.827732+0000 mon.c (mon.1) 105 : audit [DBG] from='client.? 172.21.2.111:0/3904749489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[31774]: cluster 2021-03-10T15:37:20.995216+0000 mgr.gibba032.whopeo (mgr.24625) 5538 : cluster [DBG] pgmap v3145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[31774]: audit 2021-03-10T15:37:21.337654+0000 mgr.gibba032.whopeo (mgr.24625) 5539 : audit [DBG] from='client.47763 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[31774]: audit 2021-03-10T15:37:21.563112+0000 mgr.gibba032.whopeo (mgr.24625) 5540 : audit [DBG] from='client.47769 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:22 gibba011 bash[31774]: audit 2021-03-10T15:37:21.827732+0000 mon.c (mon.1) 105 : audit [DBG] from='client.? 172.21.2.111:0/3904749489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:24 gibba011 bash[30791]: cluster 2021-03-10T15:37:22.996936+0000 mgr.gibba032.whopeo (mgr.24625) 5541 : cluster [DBG] pgmap v3146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:24 gibba011 bash[31774]: cluster 2021-03-10T15:37:22.996936+0000 mgr.gibba032.whopeo (mgr.24625) 5541 : cluster [DBG] pgmap v3146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:24 gibba032 bash[24004]: cluster 2021-03-10T15:37:22.996936+0000 mgr.gibba032.whopeo (mgr.24625) 5541 : cluster [DBG] pgmap v3146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:26 gibba011 bash[30791]: cluster 2021-03-10T15:37:24.998297+0000 mgr.gibba032.whopeo (mgr.24625) 5542 : cluster [DBG] pgmap v3147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:26.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:26 gibba011 bash[30791]: cephadm 2021-03-10T15:37:25.202903+0000 mgr.gibba032.whopeo (mgr.24625) 5543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:26 gibba011 bash[31774]: cluster 2021-03-10T15:37:24.998297+0000 mgr.gibba032.whopeo (mgr.24625) 5542 : cluster [DBG] pgmap v3147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:26.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:26 gibba011 bash[31774]: cephadm 2021-03-10T15:37:25.202903+0000 mgr.gibba032.whopeo (mgr.24625) 5543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:26 gibba032 bash[24004]: cluster 2021-03-10T15:37:24.998297+0000 mgr.gibba032.whopeo (mgr.24625) 5542 : cluster [DBG] pgmap v3147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:26 gibba032 bash[24004]: cephadm 2021-03-10T15:37:25.202903+0000 mgr.gibba032.whopeo (mgr.24625) 5543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:28.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:28 gibba011 bash[30791]: cluster 2021-03-10T15:37:26.999936+0000 mgr.gibba032.whopeo (mgr.24625) 5544 : cluster [DBG] pgmap v3148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:28.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:28 gibba011 bash[31774]: cluster 2021-03-10T15:37:26.999936+0000 mgr.gibba032.whopeo (mgr.24625) 5544 : cluster [DBG] pgmap v3148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:28 gibba032 bash[24004]: cluster 2021-03-10T15:37:26.999936+0000 mgr.gibba032.whopeo (mgr.24625) 5544 : cluster [DBG] pgmap v3148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:30.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:30 gibba011 bash[30791]: cluster 2021-03-10T15:37:29.001342+0000 mgr.gibba032.whopeo (mgr.24625) 5545 : cluster [DBG] pgmap v3149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:30.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:30 gibba011 bash[31774]: cluster 2021-03-10T15:37:29.001342+0000 mgr.gibba032.whopeo (mgr.24625) 5545 : cluster [DBG] pgmap v3149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:30 gibba032 bash[24004]: cluster 2021-03-10T15:37:29.001342+0000 mgr.gibba032.whopeo (mgr.24625) 5545 : cluster [DBG] pgmap v3149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:32 gibba011 bash[30791]: cluster 2021-03-10T15:37:31.002669+0000 mgr.gibba032.whopeo (mgr.24625) 5546 : cluster [DBG] pgmap v3150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:32 gibba011 bash[31774]: cluster 2021-03-10T15:37:31.002669+0000 mgr.gibba032.whopeo (mgr.24625) 5546 : cluster [DBG] pgmap v3150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:32 gibba032 bash[24004]: cluster 2021-03-10T15:37:31.002669+0000 mgr.gibba032.whopeo (mgr.24625) 5546 : cluster [DBG] pgmap v3150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:34 gibba011 bash[30791]: cluster 2021-03-10T15:37:33.004362+0000 mgr.gibba032.whopeo (mgr.24625) 5547 : cluster [DBG] pgmap v3151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:34 gibba011 bash[31774]: cluster 2021-03-10T15:37:33.004362+0000 mgr.gibba032.whopeo (mgr.24625) 5547 : cluster [DBG] pgmap v3151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:34 gibba032 bash[24004]: cluster 2021-03-10T15:37:33.004362+0000 mgr.gibba032.whopeo (mgr.24625) 5547 : cluster [DBG] pgmap v3151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:36.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:36 gibba011 bash[30791]: cluster 2021-03-10T15:37:35.005473+0000 mgr.gibba032.whopeo (mgr.24625) 5548 : cluster [DBG] pgmap v3152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:36.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:36 gibba011 bash[31774]: cluster 2021-03-10T15:37:35.005473+0000 mgr.gibba032.whopeo (mgr.24625) 5548 : cluster [DBG] pgmap v3152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:36 gibba032 bash[24004]: cluster 2021-03-10T15:37:35.005473+0000 mgr.gibba032.whopeo (mgr.24625) 5548 : cluster [DBG] pgmap v3152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:38.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:38 gibba011 bash[30791]: cluster 2021-03-10T15:37:37.007455+0000 mgr.gibba032.whopeo (mgr.24625) 5549 : cluster [DBG] pgmap v3153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:38.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:38 gibba011 bash[31774]: cluster 2021-03-10T15:37:37.007455+0000 mgr.gibba032.whopeo (mgr.24625) 5549 : cluster [DBG] pgmap v3153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:38 gibba032 bash[24004]: cluster 2021-03-10T15:37:37.007455+0000 mgr.gibba032.whopeo (mgr.24625) 5549 : cluster [DBG] pgmap v3153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:40 gibba032 bash[24004]: cluster 2021-03-10T15:37:39.009085+0000 mgr.gibba032.whopeo (mgr.24625) 5550 : cluster [DBG] pgmap v3154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:40 gibba011 bash[30791]: cluster 2021-03-10T15:37:39.009085+0000 mgr.gibba032.whopeo (mgr.24625) 5550 : cluster [DBG] pgmap v3154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:40 gibba011 bash[31774]: cluster 2021-03-10T15:37:39.009085+0000 mgr.gibba032.whopeo (mgr.24625) 5550 : cluster [DBG] pgmap v3154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:41 gibba032 bash[24004]: cephadm 2021-03-10T15:37:40.218959+0000 mgr.gibba032.whopeo (mgr.24625) 5551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:41.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:41 gibba011 bash[30791]: cephadm 2021-03-10T15:37:40.218959+0000 mgr.gibba032.whopeo (mgr.24625) 5551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:41.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:41 gibba011 bash[31774]: cephadm 2021-03-10T15:37:40.218959+0000 mgr.gibba032.whopeo (mgr.24625) 5551 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:42 gibba032 bash[24004]: cluster 2021-03-10T15:37:41.010573+0000 mgr.gibba032.whopeo (mgr.24625) 5552 : cluster [DBG] pgmap v3155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:42 gibba011 bash[30791]: cluster 2021-03-10T15:37:41.010573+0000 mgr.gibba032.whopeo (mgr.24625) 5552 : cluster [DBG] pgmap v3155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:42 gibba011 bash[31774]: cluster 2021-03-10T15:37:41.010573+0000 mgr.gibba032.whopeo (mgr.24625) 5552 : cluster [DBG] pgmap v3155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:44 gibba011 bash[30791]: cluster 2021-03-10T15:37:43.012208+0000 mgr.gibba032.whopeo (mgr.24625) 5553 : cluster [DBG] pgmap v3156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:44 gibba011 bash[31774]: cluster 2021-03-10T15:37:43.012208+0000 mgr.gibba032.whopeo (mgr.24625) 5553 : cluster [DBG] pgmap v3156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:44 gibba032 bash[24004]: cluster 2021-03-10T15:37:43.012208+0000 mgr.gibba032.whopeo (mgr.24625) 5553 : cluster [DBG] pgmap v3156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:46 gibba011 bash[30791]: cluster 2021-03-10T15:37:45.013973+0000 mgr.gibba032.whopeo (mgr.24625) 5554 : cluster [DBG] pgmap v3157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:46 gibba011 bash[31774]: cluster 2021-03-10T15:37:45.013973+0000 mgr.gibba032.whopeo (mgr.24625) 5554 : cluster [DBG] pgmap v3157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:46 gibba032 bash[24004]: cluster 2021-03-10T15:37:45.013973+0000 mgr.gibba032.whopeo (mgr.24625) 5554 : cluster [DBG] pgmap v3157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:48.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:48 gibba011 bash[30791]: cluster 2021-03-10T15:37:47.015679+0000 mgr.gibba032.whopeo (mgr.24625) 5555 : cluster [DBG] pgmap v3158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:48 gibba011 bash[31774]: cluster 2021-03-10T15:37:47.015679+0000 mgr.gibba032.whopeo (mgr.24625) 5555 : cluster [DBG] pgmap v3158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:48 gibba032 bash[24004]: cluster 2021-03-10T15:37:47.015679+0000 mgr.gibba032.whopeo (mgr.24625) 5555 : cluster [DBG] pgmap v3158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:50 gibba011 bash[30791]: cluster 2021-03-10T15:37:49.017172+0000 mgr.gibba032.whopeo (mgr.24625) 5556 : cluster [DBG] pgmap v3159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:50.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:50 gibba011 bash[31774]: cluster 2021-03-10T15:37:49.017172+0000 mgr.gibba032.whopeo (mgr.24625) 5556 : cluster [DBG] pgmap v3159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:50 gibba032 bash[24004]: cluster 2021-03-10T15:37:49.017172+0000 mgr.gibba032.whopeo (mgr.24625) 5556 : cluster [DBG] pgmap v3159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:52.074 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:37:52.327 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:37:52.327 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (104m) 8m ago 109m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:37:52.328 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (109m) 7m ago 109m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:37:52.328 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (104m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:37:52.328 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (105m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:37:52.329 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (104m) 8m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:37:52.329 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (104m) 8m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:37:52.329 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (103m) 7m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:37:52.329 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (103m) 8m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:37:52.330 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (109m) 8m ago 110m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:37:52.330 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (110m) 7m ago 110m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:37:52.330 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (102m) 8m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:37:52.330 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (102m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:37:52.331 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (101m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:37:52.331 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (101m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:37:52.331 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (101m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:37:52.331 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (100m) 7m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:37:52.332 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (100m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:37:52.332 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (99m) 7m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:37:52.332 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (109m) 7m ago 110m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:37:52.332 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 107m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:37:52.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:52 gibba011 bash[30791]: cluster 2021-03-10T15:37:51.018427+0000 mgr.gibba032.whopeo (mgr.24625) 5557 : cluster [DBG] pgmap v3160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:52.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:52 gibba011 bash[31774]: cluster 2021-03-10T15:37:51.018427+0000 mgr.gibba032.whopeo (mgr.24625) 5557 : cluster [DBG] pgmap v3160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:52 gibba032 bash[24004]: cluster 2021-03-10T15:37:51.018427+0000 mgr.gibba032.whopeo (mgr.24625) 5557 : cluster [DBG] pgmap v3160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:52.580 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:37:52.581 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:37:52.581 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:37:52.581 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:52.582 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:37:52.582 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:37:52.582 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:52.583 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:37:52.583 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:37:52.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:37:52.583 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:37:52.584 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:37:52.584 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:37:52.584 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:37:52.584 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:37:53.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[30791]: audit 2021-03-10T15:37:52.061430+0000 mgr.gibba032.whopeo (mgr.24625) 5558 : audit [DBG] from='client.57766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[30791]: audit 2021-03-10T15:37:52.283516+0000 mgr.gibba032.whopeo (mgr.24625) 5559 : audit [DBG] from='client.57772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[30791]: audit 2021-03-10T15:37:52.577169+0000 mon.c (mon.1) 106 : audit [DBG] from='client.? 172.21.2.111:0/3266311747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:53.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[31774]: audit 2021-03-10T15:37:52.061430+0000 mgr.gibba032.whopeo (mgr.24625) 5558 : audit [DBG] from='client.57766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[31774]: audit 2021-03-10T15:37:52.283516+0000 mgr.gibba032.whopeo (mgr.24625) 5559 : audit [DBG] from='client.57772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:53 gibba011 bash[31774]: audit 2021-03-10T15:37:52.577169+0000 mon.c (mon.1) 106 : audit [DBG] from='client.? 172.21.2.111:0/3266311747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:53 gibba032 bash[24004]: audit 2021-03-10T15:37:52.061430+0000 mgr.gibba032.whopeo (mgr.24625) 5558 : audit [DBG] from='client.57766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:53 gibba032 bash[24004]: audit 2021-03-10T15:37:52.283516+0000 mgr.gibba032.whopeo (mgr.24625) 5559 : audit [DBG] from='client.57772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:37:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:53 gibba032 bash[24004]: audit 2021-03-10T15:37:52.577169+0000 mon.c (mon.1) 106 : audit [DBG] from='client.? 172.21.2.111:0/3266311747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:54 gibba011 bash[30791]: cluster 2021-03-10T15:37:53.020664+0000 mgr.gibba032.whopeo (mgr.24625) 5560 : cluster [DBG] pgmap v3161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:54 gibba011 bash[31774]: cluster 2021-03-10T15:37:53.020664+0000 mgr.gibba032.whopeo (mgr.24625) 5560 : cluster [DBG] pgmap v3161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:54 gibba032 bash[24004]: cluster 2021-03-10T15:37:53.020664+0000 mgr.gibba032.whopeo (mgr.24625) 5560 : cluster [DBG] pgmap v3161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:56 gibba011 bash[30791]: cluster 2021-03-10T15:37:55.021945+0000 mgr.gibba032.whopeo (mgr.24625) 5561 : cluster [DBG] pgmap v3162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:56.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:56 gibba011 bash[30791]: cephadm 2021-03-10T15:37:55.234987+0000 mgr.gibba032.whopeo (mgr.24625) 5562 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:56 gibba011 bash[31774]: cluster 2021-03-10T15:37:55.021945+0000 mgr.gibba032.whopeo (mgr.24625) 5561 : cluster [DBG] pgmap v3162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:56 gibba011 bash[31774]: cephadm 2021-03-10T15:37:55.234987+0000 mgr.gibba032.whopeo (mgr.24625) 5562 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:56 gibba032 bash[24004]: cluster 2021-03-10T15:37:55.021945+0000 mgr.gibba032.whopeo (mgr.24625) 5561 : cluster [DBG] pgmap v3162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:56 gibba032 bash[24004]: cephadm 2021-03-10T15:37:55.234987+0000 mgr.gibba032.whopeo (mgr.24625) 5562 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:37:57.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.661067+0000 mon.a (mon.0) 2126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:37:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.661709+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:37:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.679295+0000 mon.a (mon.0) 2128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:37:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.680138+0000 mon.a (mon.0) 2129 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.680787+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[30791]: audit 2021-03-10T15:37:56.681462+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.661067+0000 mon.a (mon.0) 2126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:37:57.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.661709+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:37:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.679295+0000 mon.a (mon.0) 2128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:37:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.680138+0000 mon.a (mon.0) 2129 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.680787+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:57 gibba011 bash[31774]: audit 2021-03-10T15:37:56.681462+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.661067+0000 mon.a (mon.0) 2126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:37:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.661709+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:37:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.679295+0000 mon.a (mon.0) 2128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T15:37:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.680138+0000 mon.a (mon.0) 2129 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.680787+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:37:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:57 gibba032 bash[24004]: audit 2021-03-10T15:37:56.681462+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:37:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:37:58 gibba032 bash[24004]: cluster 2021-03-10T15:37:57.023646+0000 mgr.gibba032.whopeo (mgr.24625) 5563 : cluster [DBG] pgmap v3163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:37:58 gibba011 bash[30791]: cluster 2021-03-10T15:37:57.023646+0000 mgr.gibba032.whopeo (mgr.24625) 5563 : cluster [DBG] pgmap v3163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:37:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:37:58 gibba011 bash[31774]: cluster 2021-03-10T15:37:57.023646+0000 mgr.gibba032.whopeo (mgr.24625) 5563 : cluster [DBG] pgmap v3163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:00 gibba032 bash[24004]: cluster 2021-03-10T15:37:59.025144+0000 mgr.gibba032.whopeo (mgr.24625) 5564 : cluster [DBG] pgmap v3164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:00 gibba011 bash[30791]: cluster 2021-03-10T15:37:59.025144+0000 mgr.gibba032.whopeo (mgr.24625) 5564 : cluster [DBG] pgmap v3164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:00 gibba011 bash[31774]: cluster 2021-03-10T15:37:59.025144+0000 mgr.gibba032.whopeo (mgr.24625) 5564 : cluster [DBG] pgmap v3164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:02 gibba032 bash[24004]: cluster 2021-03-10T15:38:01.026504+0000 mgr.gibba032.whopeo (mgr.24625) 5565 : cluster [DBG] pgmap v3165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:02 gibba011 bash[30791]: cluster 2021-03-10T15:38:01.026504+0000 mgr.gibba032.whopeo (mgr.24625) 5565 : cluster [DBG] pgmap v3165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:02 gibba011 bash[31774]: cluster 2021-03-10T15:38:01.026504+0000 mgr.gibba032.whopeo (mgr.24625) 5565 : cluster [DBG] pgmap v3165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:04 gibba011 bash[30791]: cluster 2021-03-10T15:38:03.027091+0000 mgr.gibba032.whopeo (mgr.24625) 5566 : cluster [DBG] pgmap v3166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:04 gibba011 bash[31774]: cluster 2021-03-10T15:38:03.027091+0000 mgr.gibba032.whopeo (mgr.24625) 5566 : cluster [DBG] pgmap v3166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:04 gibba032 bash[24004]: cluster 2021-03-10T15:38:03.027091+0000 mgr.gibba032.whopeo (mgr.24625) 5566 : cluster [DBG] pgmap v3166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:06 gibba032 bash[24004]: cluster 2021-03-10T15:38:05.028327+0000 mgr.gibba032.whopeo (mgr.24625) 5567 : cluster [DBG] pgmap v3167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:06 gibba011 bash[30791]: cluster 2021-03-10T15:38:05.028327+0000 mgr.gibba032.whopeo (mgr.24625) 5567 : cluster [DBG] pgmap v3167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:06 gibba011 bash[31774]: cluster 2021-03-10T15:38:05.028327+0000 mgr.gibba032.whopeo (mgr.24625) 5567 : cluster [DBG] pgmap v3167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:08 gibba032 bash[24004]: cluster 2021-03-10T15:38:07.029995+0000 mgr.gibba032.whopeo (mgr.24625) 5568 : cluster [DBG] pgmap v3168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:08.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:08 gibba011 bash[30791]: cluster 2021-03-10T15:38:07.029995+0000 mgr.gibba032.whopeo (mgr.24625) 5568 : cluster [DBG] pgmap v3168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:08 gibba011 bash[31774]: cluster 2021-03-10T15:38:07.029995+0000 mgr.gibba032.whopeo (mgr.24625) 5568 : cluster [DBG] pgmap v3168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:10.500 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:10 gibba032 bash[24004]: cluster 2021-03-10T15:38:09.031605+0000 mgr.gibba032.whopeo (mgr.24625) 5569 : cluster [DBG] pgmap v3169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:10 gibba011 bash[30791]: cluster 2021-03-10T15:38:09.031605+0000 mgr.gibba032.whopeo (mgr.24625) 5569 : cluster [DBG] pgmap v3169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:10 gibba011 bash[31774]: cluster 2021-03-10T15:38:09.031605+0000 mgr.gibba032.whopeo (mgr.24625) 5569 : cluster [DBG] pgmap v3169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.251665+0000 mgr.gibba032.whopeo (mgr.24625) 5570 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:38:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.253267+0000 mgr.gibba032.whopeo (mgr.24625) 5571 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:38:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.485761+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:38:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.487410+0000 mgr.gibba032.whopeo (mgr.24625) 5572 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:38:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.489067+0000 mgr.gibba032.whopeo (mgr.24625) 5573 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:38:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.489139+0000 mon.a (mon.0) 2133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:38:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.490246+0000 mgr.gibba032.whopeo (mgr.24625) 5574 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:38:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.490288+0000 mgr.gibba032.whopeo (mgr.24625) 5575 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:38:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.490399+0000 mon.a (mon.0) 2134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.491462+0000 mgr.gibba032.whopeo (mgr.24625) 5576 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:38:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.491503+0000 mgr.gibba032.whopeo (mgr.24625) 5577 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:38:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.491659+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.492707+0000 mgr.gibba032.whopeo (mgr.24625) 5578 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:38:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.492873+0000 mon.a (mon.0) 2136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.493755+0000 mgr.gibba032.whopeo (mgr.24625) 5579 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:38:11.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.493798+0000 mgr.gibba032.whopeo (mgr.24625) 5580 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:38:11.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.495120+0000 mgr.gibba032.whopeo (mgr.24625) 5581 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:38:11.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.495269+0000 mon.a (mon.0) 2137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.495492+0000 mgr.gibba032.whopeo (mgr.24625) 5582 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:38:11.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.496599+0000 mgr.gibba032.whopeo (mgr.24625) 5583 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:38:11.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.496645+0000 mgr.gibba032.whopeo (mgr.24625) 5584 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:38:11.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: cephadm 2021-03-10T15:38:10.496767+0000 mgr.gibba032.whopeo (mgr.24625) 5585 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:11.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:11 gibba032 bash[24004]: audit 2021-03-10T15:38:10.496792+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.251665+0000 mgr.gibba032.whopeo (mgr.24625) 5570 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:38:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.253267+0000 mgr.gibba032.whopeo (mgr.24625) 5571 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:38:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.485761+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:38:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.487410+0000 mgr.gibba032.whopeo (mgr.24625) 5572 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:38:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.489067+0000 mgr.gibba032.whopeo (mgr.24625) 5573 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:38:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.489139+0000 mon.a (mon.0) 2133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:38:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.490246+0000 mgr.gibba032.whopeo (mgr.24625) 5574 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:38:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.490288+0000 mgr.gibba032.whopeo (mgr.24625) 5575 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:38:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.490399+0000 mon.a (mon.0) 2134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.491462+0000 mgr.gibba032.whopeo (mgr.24625) 5576 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:38:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.491503+0000 mgr.gibba032.whopeo (mgr.24625) 5577 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:38:11.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.491659+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.492707+0000 mgr.gibba032.whopeo (mgr.24625) 5578 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:38:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.492873+0000 mon.a (mon.0) 2136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.493755+0000 mgr.gibba032.whopeo (mgr.24625) 5579 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:38:11.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.493798+0000 mgr.gibba032.whopeo (mgr.24625) 5580 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:38:11.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.495120+0000 mgr.gibba032.whopeo (mgr.24625) 5581 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:38:11.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.495269+0000 mon.a (mon.0) 2137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.495492+0000 mgr.gibba032.whopeo (mgr.24625) 5582 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:38:11.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.496599+0000 mgr.gibba032.whopeo (mgr.24625) 5583 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:38:11.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.496645+0000 mgr.gibba032.whopeo (mgr.24625) 5584 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:38:11.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: cephadm 2021-03-10T15:38:10.496767+0000 mgr.gibba032.whopeo (mgr.24625) 5585 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:11.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[30791]: audit 2021-03-10T15:38:10.496792+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.251665+0000 mgr.gibba032.whopeo (mgr.24625) 5570 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:38:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.253267+0000 mgr.gibba032.whopeo (mgr.24625) 5571 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:38:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.485761+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:38:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.487410+0000 mgr.gibba032.whopeo (mgr.24625) 5572 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:38:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.489067+0000 mgr.gibba032.whopeo (mgr.24625) 5573 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:38:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.489139+0000 mon.a (mon.0) 2133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:38:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.490246+0000 mgr.gibba032.whopeo (mgr.24625) 5574 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:38:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.490288+0000 mgr.gibba032.whopeo (mgr.24625) 5575 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:38:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.490399+0000 mon.a (mon.0) 2134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.491462+0000 mgr.gibba032.whopeo (mgr.24625) 5576 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:38:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.491503+0000 mgr.gibba032.whopeo (mgr.24625) 5577 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:38:11.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.491659+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.492707+0000 mgr.gibba032.whopeo (mgr.24625) 5578 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:38:11.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.492873+0000 mon.a (mon.0) 2136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.493755+0000 mgr.gibba032.whopeo (mgr.24625) 5579 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:38:11.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.493798+0000 mgr.gibba032.whopeo (mgr.24625) 5580 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:38:11.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.495120+0000 mgr.gibba032.whopeo (mgr.24625) 5581 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:38:11.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.495269+0000 mon.a (mon.0) 2137 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:11.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.495492+0000 mgr.gibba032.whopeo (mgr.24625) 5582 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:38:11.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.496599+0000 mgr.gibba032.whopeo (mgr.24625) 5583 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:38:11.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.496645+0000 mgr.gibba032.whopeo (mgr.24625) 5584 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:38:11.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: cephadm 2021-03-10T15:38:10.496767+0000 mgr.gibba032.whopeo (mgr.24625) 5585 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:11.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:11 gibba011 bash[31774]: audit 2021-03-10T15:38:10.496792+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:12 gibba032 bash[24004]: cluster 2021-03-10T15:38:11.033035+0000 mgr.gibba032.whopeo (mgr.24625) 5586 : cluster [DBG] pgmap v3170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:12 gibba011 bash[30791]: cluster 2021-03-10T15:38:11.033035+0000 mgr.gibba032.whopeo (mgr.24625) 5586 : cluster [DBG] pgmap v3170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:12.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:12 gibba011 bash[31774]: cluster 2021-03-10T15:38:11.033035+0000 mgr.gibba032.whopeo (mgr.24625) 5586 : cluster [DBG] pgmap v3170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:14 gibba011 bash[30791]: cluster 2021-03-10T15:38:13.033768+0000 mgr.gibba032.whopeo (mgr.24625) 5587 : cluster [DBG] pgmap v3171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:14 gibba011 bash[31774]: cluster 2021-03-10T15:38:13.033768+0000 mgr.gibba032.whopeo (mgr.24625) 5587 : cluster [DBG] pgmap v3171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:14 gibba032 bash[24004]: cluster 2021-03-10T15:38:13.033768+0000 mgr.gibba032.whopeo (mgr.24625) 5587 : cluster [DBG] pgmap v3171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:16 gibba032 bash[24004]: cluster 2021-03-10T15:38:15.034954+0000 mgr.gibba032.whopeo (mgr.24625) 5588 : cluster [DBG] pgmap v3172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:16 gibba011 bash[30791]: cluster 2021-03-10T15:38:15.034954+0000 mgr.gibba032.whopeo (mgr.24625) 5588 : cluster [DBG] pgmap v3172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:16 gibba011 bash[31774]: cluster 2021-03-10T15:38:15.034954+0000 mgr.gibba032.whopeo (mgr.24625) 5588 : cluster [DBG] pgmap v3172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:18 gibba032 bash[24004]: cluster 2021-03-10T15:38:17.036717+0000 mgr.gibba032.whopeo (mgr.24625) 5589 : cluster [DBG] pgmap v3173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:18 gibba011 bash[30791]: cluster 2021-03-10T15:38:17.036717+0000 mgr.gibba032.whopeo (mgr.24625) 5589 : cluster [DBG] pgmap v3173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:18 gibba011 bash[31774]: cluster 2021-03-10T15:38:17.036717+0000 mgr.gibba032.whopeo (mgr.24625) 5589 : cluster [DBG] pgmap v3173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:20 gibba032 bash[24004]: cluster 2021-03-10T15:38:19.038288+0000 mgr.gibba032.whopeo (mgr.24625) 5590 : cluster [DBG] pgmap v3174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:20 gibba011 bash[30791]: cluster 2021-03-10T15:38:19.038288+0000 mgr.gibba032.whopeo (mgr.24625) 5590 : cluster [DBG] pgmap v3174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:20 gibba011 bash[31774]: cluster 2021-03-10T15:38:19.038288+0000 mgr.gibba032.whopeo (mgr.24625) 5590 : cluster [DBG] pgmap v3174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:22 gibba032 bash[24004]: cluster 2021-03-10T15:38:21.039660+0000 mgr.gibba032.whopeo (mgr.24625) 5591 : cluster [DBG] pgmap v3175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:22 gibba011 bash[30791]: cluster 2021-03-10T15:38:21.039660+0000 mgr.gibba032.whopeo (mgr.24625) 5591 : cluster [DBG] pgmap v3175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:22 gibba011 bash[31774]: cluster 2021-03-10T15:38:21.039660+0000 mgr.gibba032.whopeo (mgr.24625) 5591 : cluster [DBG] pgmap v3175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:22.825 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:38:23.054 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:38:23.054 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (105m) 9m ago 110m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:38:23.054 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (110m) 8m ago 110m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:38:23.054 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (105m) 8m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:38:23.055 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (105m) 8m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:38:23.055 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (104m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:38:23.055 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (104m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:38:23.055 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (104m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:38:23.056 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (104m) 9m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:38:23.056 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (110m) 9m ago 110m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:38:23.056 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (110m) 8m ago 110m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:38:23.056 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (103m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:38:23.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (102m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:38:23.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (102m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:38:23.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (102m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:38:23.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (101m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:38:23.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (101m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:38:23.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (100m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:38:23.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (100m) 8m ago 110m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:38:23.058 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (109m) 8m ago 110m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:38:23.058 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 108m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:38:23.302 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:38:23.303 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:38:23.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:38:23.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:23.303 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:38:23.304 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:38:23.304 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:23.304 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:38:23.304 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:38:23.305 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:23.305 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:38:23.305 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:38:23.305 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:38:23.305 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:38:23.306 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:38:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[30791]: audit 2021-03-10T15:38:22.812725+0000 mgr.gibba032.whopeo (mgr.24625) 5592 : audit [DBG] from='client.47799 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[30791]: audit 2021-03-10T15:38:23.037904+0000 mgr.gibba032.whopeo (mgr.24625) 5593 : audit [DBG] from='client.57790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[30791]: cluster 2021-03-10T15:38:23.041285+0000 mgr.gibba032.whopeo (mgr.24625) 5594 : cluster [DBG] pgmap v3176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:24.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[30791]: audit 2021-03-10T15:38:23.299131+0000 mon.c (mon.1) 107 : audit [DBG] from='client.? 172.21.2.111:0/4097779377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[31774]: audit 2021-03-10T15:38:22.812725+0000 mgr.gibba032.whopeo (mgr.24625) 5592 : audit [DBG] from='client.47799 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[31774]: audit 2021-03-10T15:38:23.037904+0000 mgr.gibba032.whopeo (mgr.24625) 5593 : audit [DBG] from='client.57790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[31774]: cluster 2021-03-10T15:38:23.041285+0000 mgr.gibba032.whopeo (mgr.24625) 5594 : cluster [DBG] pgmap v3176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:24 gibba011 bash[31774]: audit 2021-03-10T15:38:23.299131+0000 mon.c (mon.1) 107 : audit [DBG] from='client.? 172.21.2.111:0/4097779377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:24 gibba032 bash[24004]: audit 2021-03-10T15:38:22.812725+0000 mgr.gibba032.whopeo (mgr.24625) 5592 : audit [DBG] from='client.47799 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:24 gibba032 bash[24004]: audit 2021-03-10T15:38:23.037904+0000 mgr.gibba032.whopeo (mgr.24625) 5593 : audit [DBG] from='client.57790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:24 gibba032 bash[24004]: cluster 2021-03-10T15:38:23.041285+0000 mgr.gibba032.whopeo (mgr.24625) 5594 : cluster [DBG] pgmap v3176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:24.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:24 gibba032 bash[24004]: audit 2021-03-10T15:38:23.299131+0000 mon.c (mon.1) 107 : audit [DBG] from='client.? 172.21.2.111:0/4097779377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:26 gibba032 bash[24004]: cluster 2021-03-10T15:38:25.042614+0000 mgr.gibba032.whopeo (mgr.24625) 5595 : cluster [DBG] pgmap v3177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:26 gibba032 bash[24004]: cephadm 2021-03-10T15:38:25.508799+0000 mgr.gibba032.whopeo (mgr.24625) 5596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:26 gibba011 bash[30791]: cluster 2021-03-10T15:38:25.042614+0000 mgr.gibba032.whopeo (mgr.24625) 5595 : cluster [DBG] pgmap v3177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:26 gibba011 bash[30791]: cephadm 2021-03-10T15:38:25.508799+0000 mgr.gibba032.whopeo (mgr.24625) 5596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:26 gibba011 bash[31774]: cluster 2021-03-10T15:38:25.042614+0000 mgr.gibba032.whopeo (mgr.24625) 5595 : cluster [DBG] pgmap v3177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:26 gibba011 bash[31774]: cephadm 2021-03-10T15:38:25.508799+0000 mgr.gibba032.whopeo (mgr.24625) 5596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:28 gibba032 bash[24004]: cluster 2021-03-10T15:38:27.044352+0000 mgr.gibba032.whopeo (mgr.24625) 5597 : cluster [DBG] pgmap v3178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:28 gibba011 bash[30791]: cluster 2021-03-10T15:38:27.044352+0000 mgr.gibba032.whopeo (mgr.24625) 5597 : cluster [DBG] pgmap v3178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:28 gibba011 bash[31774]: cluster 2021-03-10T15:38:27.044352+0000 mgr.gibba032.whopeo (mgr.24625) 5597 : cluster [DBG] pgmap v3178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:30 gibba032 bash[24004]: cluster 2021-03-10T15:38:29.046065+0000 mgr.gibba032.whopeo (mgr.24625) 5598 : cluster [DBG] pgmap v3179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:30 gibba011 bash[30791]: cluster 2021-03-10T15:38:29.046065+0000 mgr.gibba032.whopeo (mgr.24625) 5598 : cluster [DBG] pgmap v3179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:30 gibba011 bash[31774]: cluster 2021-03-10T15:38:29.046065+0000 mgr.gibba032.whopeo (mgr.24625) 5598 : cluster [DBG] pgmap v3179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:32 gibba032 bash[24004]: cluster 2021-03-10T15:38:31.047582+0000 mgr.gibba032.whopeo (mgr.24625) 5599 : cluster [DBG] pgmap v3180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:32 gibba011 bash[30791]: cluster 2021-03-10T15:38:31.047582+0000 mgr.gibba032.whopeo (mgr.24625) 5599 : cluster [DBG] pgmap v3180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:32 gibba011 bash[31774]: cluster 2021-03-10T15:38:31.047582+0000 mgr.gibba032.whopeo (mgr.24625) 5599 : cluster [DBG] pgmap v3180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:34 gibba011 bash[30791]: cluster 2021-03-10T15:38:33.049348+0000 mgr.gibba032.whopeo (mgr.24625) 5600 : cluster [DBG] pgmap v3181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:34 gibba011 bash[31774]: cluster 2021-03-10T15:38:33.049348+0000 mgr.gibba032.whopeo (mgr.24625) 5600 : cluster [DBG] pgmap v3181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:34 gibba032 bash[24004]: cluster 2021-03-10T15:38:33.049348+0000 mgr.gibba032.whopeo (mgr.24625) 5600 : cluster [DBG] pgmap v3181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:36 gibba032 bash[24004]: cluster 2021-03-10T15:38:35.050739+0000 mgr.gibba032.whopeo (mgr.24625) 5601 : cluster [DBG] pgmap v3182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:36 gibba011 bash[30791]: cluster 2021-03-10T15:38:35.050739+0000 mgr.gibba032.whopeo (mgr.24625) 5601 : cluster [DBG] pgmap v3182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:36 gibba011 bash[31774]: cluster 2021-03-10T15:38:35.050739+0000 mgr.gibba032.whopeo (mgr.24625) 5601 : cluster [DBG] pgmap v3182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:38 gibba032 bash[24004]: cluster 2021-03-10T15:38:37.052619+0000 mgr.gibba032.whopeo (mgr.24625) 5602 : cluster [DBG] pgmap v3183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:38 gibba011 bash[30791]: cluster 2021-03-10T15:38:37.052619+0000 mgr.gibba032.whopeo (mgr.24625) 5602 : cluster [DBG] pgmap v3183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:38 gibba011 bash[31774]: cluster 2021-03-10T15:38:37.052619+0000 mgr.gibba032.whopeo (mgr.24625) 5602 : cluster [DBG] pgmap v3183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:40 gibba032 bash[24004]: cluster 2021-03-10T15:38:39.053980+0000 mgr.gibba032.whopeo (mgr.24625) 5603 : cluster [DBG] pgmap v3184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:40 gibba011 bash[30791]: cluster 2021-03-10T15:38:39.053980+0000 mgr.gibba032.whopeo (mgr.24625) 5603 : cluster [DBG] pgmap v3184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:40 gibba011 bash[31774]: cluster 2021-03-10T15:38:39.053980+0000 mgr.gibba032.whopeo (mgr.24625) 5603 : cluster [DBG] pgmap v3184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:41.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:41 gibba032 bash[24004]: cephadm 2021-03-10T15:38:40.520274+0000 mgr.gibba032.whopeo (mgr.24625) 5604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:41.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:41 gibba011 bash[30791]: cephadm 2021-03-10T15:38:40.520274+0000 mgr.gibba032.whopeo (mgr.24625) 5604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:41.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:41 gibba011 bash[31774]: cephadm 2021-03-10T15:38:40.520274+0000 mgr.gibba032.whopeo (mgr.24625) 5604 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:42 gibba032 bash[24004]: cluster 2021-03-10T15:38:41.055293+0000 mgr.gibba032.whopeo (mgr.24625) 5605 : cluster [DBG] pgmap v3185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:42 gibba011 bash[30791]: cluster 2021-03-10T15:38:41.055293+0000 mgr.gibba032.whopeo (mgr.24625) 5605 : cluster [DBG] pgmap v3185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:42 gibba011 bash[31774]: cluster 2021-03-10T15:38:41.055293+0000 mgr.gibba032.whopeo (mgr.24625) 5605 : cluster [DBG] pgmap v3185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:44 gibba011 bash[30791]: cluster 2021-03-10T15:38:43.057092+0000 mgr.gibba032.whopeo (mgr.24625) 5606 : cluster [DBG] pgmap v3186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:44 gibba011 bash[31774]: cluster 2021-03-10T15:38:43.057092+0000 mgr.gibba032.whopeo (mgr.24625) 5606 : cluster [DBG] pgmap v3186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:44 gibba032 bash[24004]: cluster 2021-03-10T15:38:43.057092+0000 mgr.gibba032.whopeo (mgr.24625) 5606 : cluster [DBG] pgmap v3186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:46 gibba032 bash[24004]: cluster 2021-03-10T15:38:45.058484+0000 mgr.gibba032.whopeo (mgr.24625) 5607 : cluster [DBG] pgmap v3187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:46 gibba011 bash[30791]: cluster 2021-03-10T15:38:45.058484+0000 mgr.gibba032.whopeo (mgr.24625) 5607 : cluster [DBG] pgmap v3187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:46 gibba011 bash[31774]: cluster 2021-03-10T15:38:45.058484+0000 mgr.gibba032.whopeo (mgr.24625) 5607 : cluster [DBG] pgmap v3187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:48 gibba032 bash[24004]: cluster 2021-03-10T15:38:47.060314+0000 mgr.gibba032.whopeo (mgr.24625) 5608 : cluster [DBG] pgmap v3188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:48 gibba011 bash[30791]: cluster 2021-03-10T15:38:47.060314+0000 mgr.gibba032.whopeo (mgr.24625) 5608 : cluster [DBG] pgmap v3188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:48 gibba011 bash[31774]: cluster 2021-03-10T15:38:47.060314+0000 mgr.gibba032.whopeo (mgr.24625) 5608 : cluster [DBG] pgmap v3188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:50 gibba032 bash[24004]: cluster 2021-03-10T15:38:49.061753+0000 mgr.gibba032.whopeo (mgr.24625) 5609 : cluster [DBG] pgmap v3189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:50 gibba011 bash[30791]: cluster 2021-03-10T15:38:49.061753+0000 mgr.gibba032.whopeo (mgr.24625) 5609 : cluster [DBG] pgmap v3189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:50 gibba011 bash[31774]: cluster 2021-03-10T15:38:49.061753+0000 mgr.gibba032.whopeo (mgr.24625) 5609 : cluster [DBG] pgmap v3189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:52 gibba032 bash[24004]: cluster 2021-03-10T15:38:51.063227+0000 mgr.gibba032.whopeo (mgr.24625) 5610 : cluster [DBG] pgmap v3190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:52 gibba011 bash[30791]: cluster 2021-03-10T15:38:51.063227+0000 mgr.gibba032.whopeo (mgr.24625) 5610 : cluster [DBG] pgmap v3190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:52 gibba011 bash[31774]: cluster 2021-03-10T15:38:51.063227+0000 mgr.gibba032.whopeo (mgr.24625) 5610 : cluster [DBG] pgmap v3190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:53.547 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:38:53.775 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:38:53.775 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (105m) 9m ago 110m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:38:53.775 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (110m) 8m ago 110m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:38:53.775 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (105m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:38:53.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (106m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:38:53.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (105m) 9m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:38:53.776 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (105m) 9m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:38:53.776 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (104m) 8m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:38:53.777 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (104m) 9m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:38:53.777 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (111m) 9m ago 111m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:38:53.777 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (111m) 8m ago 111m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:38:53.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (103m) 9m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:38:53.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (103m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:38:53.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (102m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:38:53.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (102m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:38:53.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (102m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:38:53.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (101m) 8m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:38:53.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (101m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:38:53.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (100m) 8m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:38:53.779 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (110m) 8m ago 111m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:38:53.779 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 108m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:38:54.024 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:38:54.024 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:38:54.024 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:38:54.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:54.025 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:38:54.025 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:38:54.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:54.026 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:38:54.026 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:38:54.026 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:38:54.026 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:38:54.026 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:38:54.027 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:38:54.027 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:38:54.027 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:38:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[30791]: cluster 2021-03-10T15:38:53.064730+0000 mgr.gibba032.whopeo (mgr.24625) 5611 : cluster [DBG] pgmap v3191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[30791]: audit 2021-03-10T15:38:53.534223+0000 mgr.gibba032.whopeo (mgr.24625) 5612 : audit [DBG] from='client.47817 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[30791]: audit 2021-03-10T15:38:54.020600+0000 mon.c (mon.1) 108 : audit [DBG] from='client.? 172.21.2.111:0/1752915571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[31774]: cluster 2021-03-10T15:38:53.064730+0000 mgr.gibba032.whopeo (mgr.24625) 5611 : cluster [DBG] pgmap v3191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[31774]: audit 2021-03-10T15:38:53.534223+0000 mgr.gibba032.whopeo (mgr.24625) 5612 : audit [DBG] from='client.47817 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:54 gibba011 bash[31774]: audit 2021-03-10T15:38:54.020600+0000 mon.c (mon.1) 108 : audit [DBG] from='client.? 172.21.2.111:0/1752915571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:54 gibba032 bash[24004]: cluster 2021-03-10T15:38:53.064730+0000 mgr.gibba032.whopeo (mgr.24625) 5611 : cluster [DBG] pgmap v3191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:54 gibba032 bash[24004]: audit 2021-03-10T15:38:53.534223+0000 mgr.gibba032.whopeo (mgr.24625) 5612 : audit [DBG] from='client.47817 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:54 gibba032 bash[24004]: audit 2021-03-10T15:38:54.020600+0000 mon.c (mon.1) 108 : audit [DBG] from='client.? 172.21.2.111:0/1752915571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:38:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:55 gibba011 bash[30791]: audit 2021-03-10T15:38:53.758189+0000 mgr.gibba032.whopeo (mgr.24625) 5613 : audit [DBG] from='client.47823 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:55.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:55 gibba011 bash[31774]: audit 2021-03-10T15:38:53.758189+0000 mgr.gibba032.whopeo (mgr.24625) 5613 : audit [DBG] from='client.47823 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:55 gibba032 bash[24004]: audit 2021-03-10T15:38:53.758189+0000 mgr.gibba032.whopeo (mgr.24625) 5613 : audit [DBG] from='client.47823 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:38:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:56 gibba011 bash[30791]: cluster 2021-03-10T15:38:55.065319+0000 mgr.gibba032.whopeo (mgr.24625) 5614 : cluster [DBG] pgmap v3192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:56 gibba011 bash[30791]: cephadm 2021-03-10T15:38:55.532433+0000 mgr.gibba032.whopeo (mgr.24625) 5615 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:56 gibba011 bash[31774]: cluster 2021-03-10T15:38:55.065319+0000 mgr.gibba032.whopeo (mgr.24625) 5614 : cluster [DBG] pgmap v3192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:56.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:56 gibba011 bash[31774]: cephadm 2021-03-10T15:38:55.532433+0000 mgr.gibba032.whopeo (mgr.24625) 5615 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:56 gibba032 bash[24004]: cluster 2021-03-10T15:38:55.065319+0000 mgr.gibba032.whopeo (mgr.24625) 5614 : cluster [DBG] pgmap v3192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:56 gibba032 bash[24004]: cephadm 2021-03-10T15:38:55.532433+0000 mgr.gibba032.whopeo (mgr.24625) 5615 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:38:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.664362+0000 mon.a (mon.0) 2139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:38:57.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.665101+0000 mon.a (mon.0) 2140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:38:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.686093+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:38:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.686871+0000 mon.a (mon.0) 2142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:38:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.687657+0000 mon.a (mon.0) 2143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:38:57.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[30791]: audit 2021-03-10T15:38:56.688395+0000 mon.a (mon.0) 2144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:38:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.664362+0000 mon.a (mon.0) 2139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:38:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.665101+0000 mon.a (mon.0) 2140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:38:57.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.686093+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:38:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.686871+0000 mon.a (mon.0) 2142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:38:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.687657+0000 mon.a (mon.0) 2143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:38:57.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:57 gibba011 bash[31774]: audit 2021-03-10T15:38:56.688395+0000 mon.a (mon.0) 2144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:38:57.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.664362+0000 mon.a (mon.0) 2139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.665101+0000 mon.a (mon.0) 2140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.686093+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:38:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.686871+0000 mon.a (mon.0) 2142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.687657+0000 mon.a (mon.0) 2143 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:38:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:57 gibba032 bash[24004]: audit 2021-03-10T15:38:56.688395+0000 mon.a (mon.0) 2144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:38:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:38:58 gibba011 bash[30791]: cluster 2021-03-10T15:38:57.065919+0000 mgr.gibba032.whopeo (mgr.24625) 5616 : cluster [DBG] pgmap v3193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:38:58 gibba011 bash[31774]: cluster 2021-03-10T15:38:57.065919+0000 mgr.gibba032.whopeo (mgr.24625) 5616 : cluster [DBG] pgmap v3193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:38:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:38:58 gibba032 bash[24004]: cluster 2021-03-10T15:38:57.065919+0000 mgr.gibba032.whopeo (mgr.24625) 5616 : cluster [DBG] pgmap v3193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:00 gibba032 bash[24004]: cluster 2021-03-10T15:38:59.067422+0000 mgr.gibba032.whopeo (mgr.24625) 5617 : cluster [DBG] pgmap v3194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:00 gibba011 bash[30791]: cluster 2021-03-10T15:38:59.067422+0000 mgr.gibba032.whopeo (mgr.24625) 5617 : cluster [DBG] pgmap v3194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:00 gibba011 bash[31774]: cluster 2021-03-10T15:38:59.067422+0000 mgr.gibba032.whopeo (mgr.24625) 5617 : cluster [DBG] pgmap v3194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:02 gibba032 bash[24004]: cluster 2021-03-10T15:39:01.068984+0000 mgr.gibba032.whopeo (mgr.24625) 5618 : cluster [DBG] pgmap v3195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:02 gibba011 bash[30791]: cluster 2021-03-10T15:39:01.068984+0000 mgr.gibba032.whopeo (mgr.24625) 5618 : cluster [DBG] pgmap v3195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:02 gibba011 bash[31774]: cluster 2021-03-10T15:39:01.068984+0000 mgr.gibba032.whopeo (mgr.24625) 5618 : cluster [DBG] pgmap v3195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:04 gibba011 bash[30791]: cluster 2021-03-10T15:39:03.070484+0000 mgr.gibba032.whopeo (mgr.24625) 5619 : cluster [DBG] pgmap v3196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:04 gibba011 bash[31774]: cluster 2021-03-10T15:39:03.070484+0000 mgr.gibba032.whopeo (mgr.24625) 5619 : cluster [DBG] pgmap v3196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:04 gibba032 bash[24004]: cluster 2021-03-10T15:39:03.070484+0000 mgr.gibba032.whopeo (mgr.24625) 5619 : cluster [DBG] pgmap v3196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:06 gibba032 bash[24004]: cluster 2021-03-10T15:39:05.071811+0000 mgr.gibba032.whopeo (mgr.24625) 5620 : cluster [DBG] pgmap v3197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:06 gibba011 bash[30791]: cluster 2021-03-10T15:39:05.071811+0000 mgr.gibba032.whopeo (mgr.24625) 5620 : cluster [DBG] pgmap v3197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:06 gibba011 bash[31774]: cluster 2021-03-10T15:39:05.071811+0000 mgr.gibba032.whopeo (mgr.24625) 5620 : cluster [DBG] pgmap v3197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:08 gibba032 bash[24004]: cluster 2021-03-10T15:39:07.073566+0000 mgr.gibba032.whopeo (mgr.24625) 5621 : cluster [DBG] pgmap v3198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:08 gibba011 bash[30791]: cluster 2021-03-10T15:39:07.073566+0000 mgr.gibba032.whopeo (mgr.24625) 5621 : cluster [DBG] pgmap v3198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:08 gibba011 bash[31774]: cluster 2021-03-10T15:39:07.073566+0000 mgr.gibba032.whopeo (mgr.24625) 5621 : cluster [DBG] pgmap v3198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:10 gibba032 bash[24004]: cluster 2021-03-10T15:39:09.075171+0000 mgr.gibba032.whopeo (mgr.24625) 5622 : cluster [DBG] pgmap v3199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:10 gibba011 bash[30791]: cluster 2021-03-10T15:39:09.075171+0000 mgr.gibba032.whopeo (mgr.24625) 5622 : cluster [DBG] pgmap v3199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:10 gibba011 bash[31774]: cluster 2021-03-10T15:39:09.075171+0000 mgr.gibba032.whopeo (mgr.24625) 5622 : cluster [DBG] pgmap v3199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:11.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:11 gibba032 bash[24004]: cephadm 2021-03-10T15:39:10.546564+0000 mgr.gibba032.whopeo (mgr.24625) 5623 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:39:11.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:11 gibba032 bash[24004]: audit 2021-03-10T15:39:10.548361+0000 mon.a (mon.0) 2145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:39:11.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:11 gibba011 bash[30791]: cephadm 2021-03-10T15:39:10.546564+0000 mgr.gibba032.whopeo (mgr.24625) 5623 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:39:11.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:11 gibba011 bash[30791]: audit 2021-03-10T15:39:10.548361+0000 mon.a (mon.0) 2145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:39:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:11 gibba011 bash[31774]: cephadm 2021-03-10T15:39:10.546564+0000 mgr.gibba032.whopeo (mgr.24625) 5623 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:39:11.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:11 gibba011 bash[31774]: audit 2021-03-10T15:39:10.548361+0000 mon.a (mon.0) 2145 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:39:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:12 gibba032 bash[24004]: cluster 2021-03-10T15:39:11.076805+0000 mgr.gibba032.whopeo (mgr.24625) 5624 : cluster [DBG] pgmap v3200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:12 gibba011 bash[30791]: cluster 2021-03-10T15:39:11.076805+0000 mgr.gibba032.whopeo (mgr.24625) 5624 : cluster [DBG] pgmap v3200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:12 gibba011 bash[31774]: cluster 2021-03-10T15:39:11.076805+0000 mgr.gibba032.whopeo (mgr.24625) 5624 : cluster [DBG] pgmap v3200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:14 gibba011 bash[30791]: cluster 2021-03-10T15:39:13.078351+0000 mgr.gibba032.whopeo (mgr.24625) 5625 : cluster [DBG] pgmap v3201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:14 gibba011 bash[31774]: cluster 2021-03-10T15:39:13.078351+0000 mgr.gibba032.whopeo (mgr.24625) 5625 : cluster [DBG] pgmap v3201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:14.353 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:14 gibba032 bash[24004]: cluster 2021-03-10T15:39:13.078351+0000 mgr.gibba032.whopeo (mgr.24625) 5625 : cluster [DBG] pgmap v3201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:15.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.115169+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.115171+0000 mgr.gibba032.whopeo (mgr.24625) 5626 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:39:15.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.332078+0000 mon.a (mon.0) 2147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.337344+0000 mgr.gibba032.whopeo (mgr.24625) 5627 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:39:15.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.339224+0000 mgr.gibba032.whopeo (mgr.24625) 5628 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:39:15.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.339297+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:39:15.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.340426+0000 mgr.gibba032.whopeo (mgr.24625) 5629 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:39:15.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.340463+0000 mgr.gibba032.whopeo (mgr.24625) 5630 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:39:15.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.340603+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.341988+0000 mgr.gibba032.whopeo (mgr.24625) 5631 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:39:15.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.342013+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.342027+0000 mgr.gibba032.whopeo (mgr.24625) 5632 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:39:15.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.343547+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.343777+0000 mgr.gibba032.whopeo (mgr.24625) 5633 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:39:15.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.345285+0000 mgr.gibba032.whopeo (mgr.24625) 5634 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:39:15.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.345324+0000 mgr.gibba032.whopeo (mgr.24625) 5635 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:39:15.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.346926+0000 mgr.gibba032.whopeo (mgr.24625) 5636 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:39:15.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.346935+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.347298+0000 mgr.gibba032.whopeo (mgr.24625) 5637 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:39:15.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: audit 2021-03-10T15:39:14.348776+0000 mon.a (mon.0) 2153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.348895+0000 mgr.gibba032.whopeo (mgr.24625) 5638 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:39:15.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.348939+0000 mgr.gibba032.whopeo (mgr.24625) 5639 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:39:15.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:15 gibba032 bash[24004]: cephadm 2021-03-10T15:39:14.349061+0000 mgr.gibba032.whopeo (mgr.24625) 5640 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:15.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.115169+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.615 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.115171+0000 mgr.gibba032.whopeo (mgr.24625) 5626 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:39:15.616 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.332078+0000 mon.a (mon.0) 2147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.616 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.337344+0000 mgr.gibba032.whopeo (mgr.24625) 5627 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:39:15.616 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.339224+0000 mgr.gibba032.whopeo (mgr.24625) 5628 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:39:15.616 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.339297+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:39:15.617 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.340426+0000 mgr.gibba032.whopeo (mgr.24625) 5629 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:39:15.617 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.340463+0000 mgr.gibba032.whopeo (mgr.24625) 5630 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:39:15.617 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.340603+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.617 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.341988+0000 mgr.gibba032.whopeo (mgr.24625) 5631 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:39:15.618 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.342013+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.618 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.342027+0000 mgr.gibba032.whopeo (mgr.24625) 5632 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:39:15.618 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.343547+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.618 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.343777+0000 mgr.gibba032.whopeo (mgr.24625) 5633 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:39:15.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.345285+0000 mgr.gibba032.whopeo (mgr.24625) 5634 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:39:15.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.345324+0000 mgr.gibba032.whopeo (mgr.24625) 5635 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:39:15.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.346926+0000 mgr.gibba032.whopeo (mgr.24625) 5636 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:39:15.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.346935+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.619 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.347298+0000 mgr.gibba032.whopeo (mgr.24625) 5637 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:39:15.620 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: audit 2021-03-10T15:39:14.348776+0000 mon.a (mon.0) 2153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.620 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.348895+0000 mgr.gibba032.whopeo (mgr.24625) 5638 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:39:15.620 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.348939+0000 mgr.gibba032.whopeo (mgr.24625) 5639 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:39:15.620 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[30791]: cephadm 2021-03-10T15:39:14.349061+0000 mgr.gibba032.whopeo (mgr.24625) 5640 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:15.621 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.115169+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.621 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.115171+0000 mgr.gibba032.whopeo (mgr.24625) 5626 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:39:15.621 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.332078+0000 mon.a (mon.0) 2147 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:39:15.621 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.337344+0000 mgr.gibba032.whopeo (mgr.24625) 5627 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:39:15.622 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.339224+0000 mgr.gibba032.whopeo (mgr.24625) 5628 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:39:15.622 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.339297+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:39:15.622 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.340426+0000 mgr.gibba032.whopeo (mgr.24625) 5629 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:39:15.622 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.340463+0000 mgr.gibba032.whopeo (mgr.24625) 5630 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:39:15.623 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.340603+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.623 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.341988+0000 mgr.gibba032.whopeo (mgr.24625) 5631 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:39:15.623 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.342013+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.623 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.342027+0000 mgr.gibba032.whopeo (mgr.24625) 5632 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:39:15.623 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.343547+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.624 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.343777+0000 mgr.gibba032.whopeo (mgr.24625) 5633 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:39:15.624 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.345285+0000 mgr.gibba032.whopeo (mgr.24625) 5634 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:39:15.624 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.345324+0000 mgr.gibba032.whopeo (mgr.24625) 5635 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:39:15.624 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.346926+0000 mgr.gibba032.whopeo (mgr.24625) 5636 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:39:15.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.346935+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.347298+0000 mgr.gibba032.whopeo (mgr.24625) 5637 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:39:15.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: audit 2021-03-10T15:39:14.348776+0000 mon.a (mon.0) 2153 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:15.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.348895+0000 mgr.gibba032.whopeo (mgr.24625) 5638 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:39:15.625 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.348939+0000 mgr.gibba032.whopeo (mgr.24625) 5639 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:39:15.626 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:15 gibba011 bash[31774]: cephadm 2021-03-10T15:39:14.349061+0000 mgr.gibba032.whopeo (mgr.24625) 5640 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:16 gibba032 bash[24004]: cluster 2021-03-10T15:39:15.079676+0000 mgr.gibba032.whopeo (mgr.24625) 5641 : cluster [DBG] pgmap v3202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:16.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:16 gibba011 bash[30791]: cluster 2021-03-10T15:39:15.079676+0000 mgr.gibba032.whopeo (mgr.24625) 5641 : cluster [DBG] pgmap v3202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:16.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:16 gibba011 bash[31774]: cluster 2021-03-10T15:39:15.079676+0000 mgr.gibba032.whopeo (mgr.24625) 5641 : cluster [DBG] pgmap v3202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:18 gibba032 bash[24004]: cluster 2021-03-10T15:39:17.081587+0000 mgr.gibba032.whopeo (mgr.24625) 5642 : cluster [DBG] pgmap v3203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:18 gibba011 bash[30791]: cluster 2021-03-10T15:39:17.081587+0000 mgr.gibba032.whopeo (mgr.24625) 5642 : cluster [DBG] pgmap v3203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:18.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:18 gibba011 bash[31774]: cluster 2021-03-10T15:39:17.081587+0000 mgr.gibba032.whopeo (mgr.24625) 5642 : cluster [DBG] pgmap v3203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:20 gibba032 bash[24004]: cluster 2021-03-10T15:39:19.083109+0000 mgr.gibba032.whopeo (mgr.24625) 5643 : cluster [DBG] pgmap v3204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:20 gibba011 bash[30791]: cluster 2021-03-10T15:39:19.083109+0000 mgr.gibba032.whopeo (mgr.24625) 5643 : cluster [DBG] pgmap v3204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:20 gibba011 bash[31774]: cluster 2021-03-10T15:39:19.083109+0000 mgr.gibba032.whopeo (mgr.24625) 5643 : cluster [DBG] pgmap v3204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:22 gibba032 bash[24004]: cluster 2021-03-10T15:39:21.084695+0000 mgr.gibba032.whopeo (mgr.24625) 5644 : cluster [DBG] pgmap v3205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:22 gibba011 bash[30791]: cluster 2021-03-10T15:39:21.084695+0000 mgr.gibba032.whopeo (mgr.24625) 5644 : cluster [DBG] pgmap v3205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:22 gibba011 bash[31774]: cluster 2021-03-10T15:39:21.084695+0000 mgr.gibba032.whopeo (mgr.24625) 5644 : cluster [DBG] pgmap v3205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:24.268 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:39:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:24 gibba011 bash[30791]: cluster 2021-03-10T15:39:23.086325+0000 mgr.gibba032.whopeo (mgr.24625) 5645 : cluster [DBG] pgmap v3206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:24 gibba011 bash[31774]: cluster 2021-03-10T15:39:23.086325+0000 mgr.gibba032.whopeo (mgr.24625) 5645 : cluster [DBG] pgmap v3206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:24.495 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:39:24.496 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (106m) 10s ago 111m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:39:24.496 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (111m) 9m ago 111m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:39:24.496 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (106m) 9m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:39:24.496 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (106m) 9m ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:39:24.496 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (105m) 10s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:39:24.497 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (105m) 10s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:39:24.497 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (105m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:39:24.497 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (105m) 10s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:39:24.497 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (111m) 10s ago 111m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:39:24.498 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (111m) 9m ago 111m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:39:24.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (104m) 10s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:39:24.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (103m) 10s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:39:24.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (103m) 10s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:39:24.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (103m) 10s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:39:24.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (102m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:39:24.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (102m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:39:24.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (101m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:39:24.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (101m) 9m ago 111m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:39:24.499 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (110m) 9m ago 111m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:39:24.500 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10s ago 109m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:39:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:24 gibba032 bash[24004]: cluster 2021-03-10T15:39:23.086325+0000 mgr.gibba032.whopeo (mgr.24625) 5645 : cluster [DBG] pgmap v3206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:24.742 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:39:24.743 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:39:24.743 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:39:24.744 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:24.744 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:39:24.744 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:39:24.745 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:24.745 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:39:24.746 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:39:24.746 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:24.747 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:39:24.747 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:39:24.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:39:24.748 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:39:24.748 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:39:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[30791]: audit 2021-03-10T15:39:24.254932+0000 mgr.gibba032.whopeo (mgr.24625) 5646 : audit [DBG] from='client.47835 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[30791]: audit 2021-03-10T15:39:24.479020+0000 mgr.gibba032.whopeo (mgr.24625) 5647 : audit [DBG] from='client.57826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[30791]: audit 2021-03-10T15:39:24.738704+0000 mon.a (mon.0) 2154 : audit [DBG] from='client.? 172.21.2.111:0/3177099816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:25.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[31774]: audit 2021-03-10T15:39:24.254932+0000 mgr.gibba032.whopeo (mgr.24625) 5646 : audit [DBG] from='client.47835 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[31774]: audit 2021-03-10T15:39:24.479020+0000 mgr.gibba032.whopeo (mgr.24625) 5647 : audit [DBG] from='client.57826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:25 gibba011 bash[31774]: audit 2021-03-10T15:39:24.738704+0000 mon.a (mon.0) 2154 : audit [DBG] from='client.? 172.21.2.111:0/3177099816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:25.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:25 gibba032 bash[24004]: audit 2021-03-10T15:39:24.254932+0000 mgr.gibba032.whopeo (mgr.24625) 5646 : audit [DBG] from='client.47835 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:25 gibba032 bash[24004]: audit 2021-03-10T15:39:24.479020+0000 mgr.gibba032.whopeo (mgr.24625) 5647 : audit [DBG] from='client.57826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:25.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:25 gibba032 bash[24004]: audit 2021-03-10T15:39:24.738704+0000 mon.a (mon.0) 2154 : audit [DBG] from='client.? 172.21.2.111:0/3177099816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:26.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:26 gibba011 bash[30791]: cluster 2021-03-10T15:39:25.086889+0000 mgr.gibba032.whopeo (mgr.24625) 5648 : cluster [DBG] pgmap v3207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:26.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:26 gibba011 bash[31774]: cluster 2021-03-10T15:39:25.086889+0000 mgr.gibba032.whopeo (mgr.24625) 5648 : cluster [DBG] pgmap v3207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:26 gibba032 bash[24004]: cluster 2021-03-10T15:39:25.086889+0000 mgr.gibba032.whopeo (mgr.24625) 5648 : cluster [DBG] pgmap v3207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:28 gibba032 bash[24004]: cluster 2021-03-10T15:39:27.088440+0000 mgr.gibba032.whopeo (mgr.24625) 5649 : cluster [DBG] pgmap v3208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:28 gibba011 bash[30791]: cluster 2021-03-10T15:39:27.088440+0000 mgr.gibba032.whopeo (mgr.24625) 5649 : cluster [DBG] pgmap v3208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:28 gibba011 bash[31774]: cluster 2021-03-10T15:39:27.088440+0000 mgr.gibba032.whopeo (mgr.24625) 5649 : cluster [DBG] pgmap v3208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:30 gibba032 bash[24004]: cluster 2021-03-10T15:39:29.089795+0000 mgr.gibba032.whopeo (mgr.24625) 5650 : cluster [DBG] pgmap v3209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:30 gibba032 bash[24004]: cephadm 2021-03-10T15:39:29.364730+0000 mgr.gibba032.whopeo (mgr.24625) 5651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:30 gibba011 bash[30791]: cluster 2021-03-10T15:39:29.089795+0000 mgr.gibba032.whopeo (mgr.24625) 5650 : cluster [DBG] pgmap v3209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:30 gibba011 bash[30791]: cephadm 2021-03-10T15:39:29.364730+0000 mgr.gibba032.whopeo (mgr.24625) 5651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:30 gibba011 bash[31774]: cluster 2021-03-10T15:39:29.089795+0000 mgr.gibba032.whopeo (mgr.24625) 5650 : cluster [DBG] pgmap v3209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:30 gibba011 bash[31774]: cephadm 2021-03-10T15:39:29.364730+0000 mgr.gibba032.whopeo (mgr.24625) 5651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:32 gibba032 bash[24004]: cluster 2021-03-10T15:39:31.091275+0000 mgr.gibba032.whopeo (mgr.24625) 5652 : cluster [DBG] pgmap v3210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:32 gibba011 bash[30791]: cluster 2021-03-10T15:39:31.091275+0000 mgr.gibba032.whopeo (mgr.24625) 5652 : cluster [DBG] pgmap v3210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:32 gibba011 bash[31774]: cluster 2021-03-10T15:39:31.091275+0000 mgr.gibba032.whopeo (mgr.24625) 5652 : cluster [DBG] pgmap v3210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:34 gibba011 bash[30791]: cluster 2021-03-10T15:39:33.093004+0000 mgr.gibba032.whopeo (mgr.24625) 5653 : cluster [DBG] pgmap v3211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:34 gibba011 bash[31774]: cluster 2021-03-10T15:39:33.093004+0000 mgr.gibba032.whopeo (mgr.24625) 5653 : cluster [DBG] pgmap v3211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:34 gibba032 bash[24004]: cluster 2021-03-10T15:39:33.093004+0000 mgr.gibba032.whopeo (mgr.24625) 5653 : cluster [DBG] pgmap v3211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:36 gibba032 bash[24004]: cluster 2021-03-10T15:39:35.094445+0000 mgr.gibba032.whopeo (mgr.24625) 5654 : cluster [DBG] pgmap v3212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:36 gibba011 bash[30791]: cluster 2021-03-10T15:39:35.094445+0000 mgr.gibba032.whopeo (mgr.24625) 5654 : cluster [DBG] pgmap v3212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:36 gibba011 bash[31774]: cluster 2021-03-10T15:39:35.094445+0000 mgr.gibba032.whopeo (mgr.24625) 5654 : cluster [DBG] pgmap v3212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:38 gibba032 bash[24004]: cluster 2021-03-10T15:39:37.096006+0000 mgr.gibba032.whopeo (mgr.24625) 5655 : cluster [DBG] pgmap v3213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:38 gibba011 bash[30791]: cluster 2021-03-10T15:39:37.096006+0000 mgr.gibba032.whopeo (mgr.24625) 5655 : cluster [DBG] pgmap v3213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:38 gibba011 bash[31774]: cluster 2021-03-10T15:39:37.096006+0000 mgr.gibba032.whopeo (mgr.24625) 5655 : cluster [DBG] pgmap v3213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:40 gibba032 bash[24004]: cluster 2021-03-10T15:39:39.097443+0000 mgr.gibba032.whopeo (mgr.24625) 5656 : cluster [DBG] pgmap v3214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:40 gibba011 bash[30791]: cluster 2021-03-10T15:39:39.097443+0000 mgr.gibba032.whopeo (mgr.24625) 5656 : cluster [DBG] pgmap v3214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:40 gibba011 bash[31774]: cluster 2021-03-10T15:39:39.097443+0000 mgr.gibba032.whopeo (mgr.24625) 5656 : cluster [DBG] pgmap v3214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:42 gibba032 bash[24004]: cluster 2021-03-10T15:39:41.098931+0000 mgr.gibba032.whopeo (mgr.24625) 5657 : cluster [DBG] pgmap v3215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:42 gibba011 bash[30791]: cluster 2021-03-10T15:39:41.098931+0000 mgr.gibba032.whopeo (mgr.24625) 5657 : cluster [DBG] pgmap v3215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:42 gibba011 bash[31774]: cluster 2021-03-10T15:39:41.098931+0000 mgr.gibba032.whopeo (mgr.24625) 5657 : cluster [DBG] pgmap v3215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:44 gibba011 bash[30791]: cluster 2021-03-10T15:39:43.100006+0000 mgr.gibba032.whopeo (mgr.24625) 5658 : cluster [DBG] pgmap v3216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:44 gibba011 bash[31774]: cluster 2021-03-10T15:39:43.100006+0000 mgr.gibba032.whopeo (mgr.24625) 5658 : cluster [DBG] pgmap v3216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:44.385 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:44 gibba032 bash[24004]: cluster 2021-03-10T15:39:43.100006+0000 mgr.gibba032.whopeo (mgr.24625) 5658 : cluster [DBG] pgmap v3216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:45.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:45 gibba011 bash[30791]: cephadm 2021-03-10T15:39:44.380738+0000 mgr.gibba032.whopeo (mgr.24625) 5659 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:45.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:45 gibba011 bash[31774]: cephadm 2021-03-10T15:39:44.380738+0000 mgr.gibba032.whopeo (mgr.24625) 5659 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:45 gibba032 bash[24004]: cephadm 2021-03-10T15:39:44.380738+0000 mgr.gibba032.whopeo (mgr.24625) 5659 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:39:46.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:46 gibba011 bash[30791]: cluster 2021-03-10T15:39:45.100489+0000 mgr.gibba032.whopeo (mgr.24625) 5660 : cluster [DBG] pgmap v3217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:46.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:46 gibba011 bash[31774]: cluster 2021-03-10T15:39:45.100489+0000 mgr.gibba032.whopeo (mgr.24625) 5660 : cluster [DBG] pgmap v3217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:46 gibba032 bash[24004]: cluster 2021-03-10T15:39:45.100489+0000 mgr.gibba032.whopeo (mgr.24625) 5660 : cluster [DBG] pgmap v3217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:48 gibba032 bash[24004]: cluster 2021-03-10T15:39:47.101820+0000 mgr.gibba032.whopeo (mgr.24625) 5661 : cluster [DBG] pgmap v3218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:48 gibba011 bash[30791]: cluster 2021-03-10T15:39:47.101820+0000 mgr.gibba032.whopeo (mgr.24625) 5661 : cluster [DBG] pgmap v3218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:48.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:48 gibba011 bash[31774]: cluster 2021-03-10T15:39:47.101820+0000 mgr.gibba032.whopeo (mgr.24625) 5661 : cluster [DBG] pgmap v3218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:50 gibba032 bash[24004]: cluster 2021-03-10T15:39:49.103403+0000 mgr.gibba032.whopeo (mgr.24625) 5662 : cluster [DBG] pgmap v3219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:50 gibba011 bash[30791]: cluster 2021-03-10T15:39:49.103403+0000 mgr.gibba032.whopeo (mgr.24625) 5662 : cluster [DBG] pgmap v3219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:50 gibba011 bash[31774]: cluster 2021-03-10T15:39:49.103403+0000 mgr.gibba032.whopeo (mgr.24625) 5662 : cluster [DBG] pgmap v3219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:52 gibba032 bash[24004]: cluster 2021-03-10T15:39:51.105032+0000 mgr.gibba032.whopeo (mgr.24625) 5663 : cluster [DBG] pgmap v3220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:52 gibba011 bash[30791]: cluster 2021-03-10T15:39:51.105032+0000 mgr.gibba032.whopeo (mgr.24625) 5663 : cluster [DBG] pgmap v3220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:52 gibba011 bash[31774]: cluster 2021-03-10T15:39:51.105032+0000 mgr.gibba032.whopeo (mgr.24625) 5663 : cluster [DBG] pgmap v3220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:54 gibba011 bash[30791]: cluster 2021-03-10T15:39:53.106224+0000 mgr.gibba032.whopeo (mgr.24625) 5664 : cluster [DBG] pgmap v3221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:54 gibba011 bash[31774]: cluster 2021-03-10T15:39:53.106224+0000 mgr.gibba032.whopeo (mgr.24625) 5664 : cluster [DBG] pgmap v3221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:54 gibba032 bash[24004]: cluster 2021-03-10T15:39:53.106224+0000 mgr.gibba032.whopeo (mgr.24625) 5664 : cluster [DBG] pgmap v3221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:54.988 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:39:55.209 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:39:55.210 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (106m) 41s ago 111m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:39:55.210 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (111m) 9m ago 111m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:39:55.210 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (106m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:39:55.210 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (107m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:39:55.211 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (106m) 41s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:39:55.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (106m) 41s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:39:55.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (105m) 9m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:39:55.211 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (105m) 41s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:39:55.211 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (112m) 41s ago 112m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:39:55.212 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (112m) 9m ago 112m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:39:55.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (104m) 41s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:39:55.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (104m) 41s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:39:55.212 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (104m) 41s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:39:55.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (103m) 41s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:39:55.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (103m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:39:55.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (102m) 9m ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:39:55.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (102m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:39:55.213 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (101m) 9m ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:39:55.214 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (111m) 9m ago 112m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:39:55.214 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 41s ago 109m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:39:55.456 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:39:55.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:39:55.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:39:55.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:55.457 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:39:55.457 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:39:55.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:55.458 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:39:55.458 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:39:55.458 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:39:55.458 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:39:55.459 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:39:55.459 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:39:55.459 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:39:55.459 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:39:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:56 gibba032 bash[24004]: audit 2021-03-10T15:39:54.974568+0000 mgr.gibba032.whopeo (mgr.24625) 5665 : audit [DBG] from='client.57838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:56 gibba032 bash[24004]: cluster 2021-03-10T15:39:55.107465+0000 mgr.gibba032.whopeo (mgr.24625) 5666 : cluster [DBG] pgmap v3222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:56.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:56 gibba032 bash[24004]: audit 2021-03-10T15:39:55.196139+0000 mgr.gibba032.whopeo (mgr.24625) 5667 : audit [DBG] from='client.47859 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:56 gibba032 bash[24004]: audit 2021-03-10T15:39:55.452465+0000 mon.a (mon.0) 2155 : audit [DBG] from='client.? 172.21.2.111:0/408843762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[30791]: audit 2021-03-10T15:39:54.974568+0000 mgr.gibba032.whopeo (mgr.24625) 5665 : audit [DBG] from='client.57838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[30791]: cluster 2021-03-10T15:39:55.107465+0000 mgr.gibba032.whopeo (mgr.24625) 5666 : cluster [DBG] pgmap v3222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:56.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[30791]: audit 2021-03-10T15:39:55.196139+0000 mgr.gibba032.whopeo (mgr.24625) 5667 : audit [DBG] from='client.47859 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[30791]: audit 2021-03-10T15:39:55.452465+0000 mon.a (mon.0) 2155 : audit [DBG] from='client.? 172.21.2.111:0/408843762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:56.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[31774]: audit 2021-03-10T15:39:54.974568+0000 mgr.gibba032.whopeo (mgr.24625) 5665 : audit [DBG] from='client.57838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[31774]: cluster 2021-03-10T15:39:55.107465+0000 mgr.gibba032.whopeo (mgr.24625) 5666 : cluster [DBG] pgmap v3222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:56.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[31774]: audit 2021-03-10T15:39:55.196139+0000 mgr.gibba032.whopeo (mgr.24625) 5667 : audit [DBG] from='client.47859 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:39:56.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:56 gibba011 bash[31774]: audit 2021-03-10T15:39:55.452465+0000 mon.a (mon.0) 2155 : audit [DBG] from='client.? 172.21.2.111:0/408843762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:39:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.671885+0000 mon.a (mon.0) 2156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:39:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.672525+0000 mon.a (mon.0) 2157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:39:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.692941+0000 mon.a (mon.0) 2158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:39:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.693703+0000 mon.a (mon.0) 2159 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:39:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.694446+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:39:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:57 gibba032 bash[24004]: audit 2021-03-10T15:39:56.695122+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:39:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.671885+0000 mon.a (mon.0) 2156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:39:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.672525+0000 mon.a (mon.0) 2157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:39:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.692941+0000 mon.a (mon.0) 2158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:39:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.693703+0000 mon.a (mon.0) 2159 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:39:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.694446+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:39:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[30791]: audit 2021-03-10T15:39:56.695122+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:39:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.671885+0000 mon.a (mon.0) 2156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:39:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.672525+0000 mon.a (mon.0) 2157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:39:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.692941+0000 mon.a (mon.0) 2158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:39:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.693703+0000 mon.a (mon.0) 2159 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:39:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.694446+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:39:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:57 gibba011 bash[31774]: audit 2021-03-10T15:39:56.695122+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:39:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:39:58 gibba032 bash[24004]: cluster 2021-03-10T15:39:57.109029+0000 mgr.gibba032.whopeo (mgr.24625) 5668 : cluster [DBG] pgmap v3223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:39:58 gibba011 bash[30791]: cluster 2021-03-10T15:39:57.109029+0000 mgr.gibba032.whopeo (mgr.24625) 5668 : cluster [DBG] pgmap v3223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:39:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:39:58 gibba011 bash[31774]: cluster 2021-03-10T15:39:57.109029+0000 mgr.gibba032.whopeo (mgr.24625) 5668 : cluster [DBG] pgmap v3223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[30791]: debug 2021-03-10T15:39:59.999+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:40:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[30791]: cluster 2021-03-10T15:39:59.110638+0000 mgr.gibba032.whopeo (mgr.24625) 5669 : cluster [DBG] pgmap v3224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[30791]: cephadm 2021-03-10T15:39:59.396792+0000 mgr.gibba032.whopeo (mgr.24625) 5670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[30791]: cluster 2021-03-10T15:40:00.000308+0000 mon.a (mon.0) 2162 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:40:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[31774]: cluster 2021-03-10T15:39:59.110638+0000 mgr.gibba032.whopeo (mgr.24625) 5669 : cluster [DBG] pgmap v3224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[31774]: cephadm 2021-03-10T15:39:59.396792+0000 mgr.gibba032.whopeo (mgr.24625) 5670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:00 gibba011 bash[31774]: cluster 2021-03-10T15:40:00.000308+0000 mon.a (mon.0) 2162 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:40:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:00 gibba032 bash[24004]: cluster 2021-03-10T15:39:59.110638+0000 mgr.gibba032.whopeo (mgr.24625) 5669 : cluster [DBG] pgmap v3224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:00 gibba032 bash[24004]: cephadm 2021-03-10T15:39:59.396792+0000 mgr.gibba032.whopeo (mgr.24625) 5670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:00.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:00 gibba032 bash[24004]: cluster 2021-03-10T15:40:00.000308+0000 mon.a (mon.0) 2162 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:40:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:02 gibba032 bash[24004]: cluster 2021-03-10T15:40:01.112293+0000 mgr.gibba032.whopeo (mgr.24625) 5671 : cluster [DBG] pgmap v3225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:02 gibba011 bash[30791]: cluster 2021-03-10T15:40:01.112293+0000 mgr.gibba032.whopeo (mgr.24625) 5671 : cluster [DBG] pgmap v3225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:02 gibba011 bash[31774]: cluster 2021-03-10T15:40:01.112293+0000 mgr.gibba032.whopeo (mgr.24625) 5671 : cluster [DBG] pgmap v3225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:04 gibba011 bash[30791]: cluster 2021-03-10T15:40:03.113259+0000 mgr.gibba032.whopeo (mgr.24625) 5672 : cluster [DBG] pgmap v3226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:04 gibba011 bash[31774]: cluster 2021-03-10T15:40:03.113259+0000 mgr.gibba032.whopeo (mgr.24625) 5672 : cluster [DBG] pgmap v3226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:04 gibba032 bash[24004]: cluster 2021-03-10T15:40:03.113259+0000 mgr.gibba032.whopeo (mgr.24625) 5672 : cluster [DBG] pgmap v3226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:06 gibba032 bash[24004]: cluster 2021-03-10T15:40:05.114758+0000 mgr.gibba032.whopeo (mgr.24625) 5673 : cluster [DBG] pgmap v3227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:06 gibba011 bash[30791]: cluster 2021-03-10T15:40:05.114758+0000 mgr.gibba032.whopeo (mgr.24625) 5673 : cluster [DBG] pgmap v3227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:06 gibba011 bash[31774]: cluster 2021-03-10T15:40:05.114758+0000 mgr.gibba032.whopeo (mgr.24625) 5673 : cluster [DBG] pgmap v3227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:08 gibba032 bash[24004]: cluster 2021-03-10T15:40:07.116279+0000 mgr.gibba032.whopeo (mgr.24625) 5674 : cluster [DBG] pgmap v3228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:08 gibba011 bash[30791]: cluster 2021-03-10T15:40:07.116279+0000 mgr.gibba032.whopeo (mgr.24625) 5674 : cluster [DBG] pgmap v3228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:08 gibba011 bash[31774]: cluster 2021-03-10T15:40:07.116279+0000 mgr.gibba032.whopeo (mgr.24625) 5674 : cluster [DBG] pgmap v3228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:10 gibba032 bash[24004]: cluster 2021-03-10T15:40:09.117883+0000 mgr.gibba032.whopeo (mgr.24625) 5675 : cluster [DBG] pgmap v3229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:10 gibba011 bash[30791]: cluster 2021-03-10T15:40:09.117883+0000 mgr.gibba032.whopeo (mgr.24625) 5675 : cluster [DBG] pgmap v3229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:10 gibba011 bash[31774]: cluster 2021-03-10T15:40:09.117883+0000 mgr.gibba032.whopeo (mgr.24625) 5675 : cluster [DBG] pgmap v3229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:12 gibba032 bash[24004]: cluster 2021-03-10T15:40:11.119323+0000 mgr.gibba032.whopeo (mgr.24625) 5676 : cluster [DBG] pgmap v3230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:12 gibba011 bash[30791]: cluster 2021-03-10T15:40:11.119323+0000 mgr.gibba032.whopeo (mgr.24625) 5676 : cluster [DBG] pgmap v3230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:12 gibba011 bash[31774]: cluster 2021-03-10T15:40:11.119323+0000 mgr.gibba032.whopeo (mgr.24625) 5676 : cluster [DBG] pgmap v3230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:14 gibba011 bash[30791]: cluster 2021-03-10T15:40:13.121124+0000 mgr.gibba032.whopeo (mgr.24625) 5677 : cluster [DBG] pgmap v3231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:14.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:14 gibba011 bash[31774]: cluster 2021-03-10T15:40:13.121124+0000 mgr.gibba032.whopeo (mgr.24625) 5677 : cluster [DBG] pgmap v3231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:14 gibba032 bash[24004]: cluster 2021-03-10T15:40:13.121124+0000 mgr.gibba032.whopeo (mgr.24625) 5677 : cluster [DBG] pgmap v3231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:15.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:15 gibba011 bash[30791]: cephadm 2021-03-10T15:40:14.413641+0000 mgr.gibba032.whopeo (mgr.24625) 5678 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:40:15.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:15 gibba011 bash[30791]: audit 2021-03-10T15:40:14.416811+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:40:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:15 gibba011 bash[31774]: cephadm 2021-03-10T15:40:14.413641+0000 mgr.gibba032.whopeo (mgr.24625) 5678 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:40:15.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:15 gibba011 bash[31774]: audit 2021-03-10T15:40:14.416811+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:40:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:15 gibba032 bash[24004]: cephadm 2021-03-10T15:40:14.413641+0000 mgr.gibba032.whopeo (mgr.24625) 5678 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:40:15.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:15 gibba032 bash[24004]: audit 2021-03-10T15:40:14.416811+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:40:16.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:16 gibba011 bash[30791]: cluster 2021-03-10T15:40:15.121943+0000 mgr.gibba032.whopeo (mgr.24625) 5679 : cluster [DBG] pgmap v3232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:16.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:16 gibba011 bash[31774]: cluster 2021-03-10T15:40:15.121943+0000 mgr.gibba032.whopeo (mgr.24625) 5679 : cluster [DBG] pgmap v3232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:16 gibba032 bash[24004]: cluster 2021-03-10T15:40:15.121943+0000 mgr.gibba032.whopeo (mgr.24625) 5679 : cluster [DBG] pgmap v3232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:18 gibba032 bash[24004]: cluster 2021-03-10T15:40:17.123231+0000 mgr.gibba032.whopeo (mgr.24625) 5680 : cluster [DBG] pgmap v3233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:18 gibba032 bash[24004]: audit 2021-03-10T15:40:18.086157+0000 mon.a (mon.0) 2164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:18 gibba011 bash[30791]: cluster 2021-03-10T15:40:17.123231+0000 mgr.gibba032.whopeo (mgr.24625) 5680 : cluster [DBG] pgmap v3233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:18 gibba011 bash[30791]: audit 2021-03-10T15:40:18.086157+0000 mon.a (mon.0) 2164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:18 gibba011 bash[31774]: cluster 2021-03-10T15:40:17.123231+0000 mgr.gibba032.whopeo (mgr.24625) 5680 : cluster [DBG] pgmap v3233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:18 gibba011 bash[31774]: audit 2021-03-10T15:40:18.086157+0000 mon.a (mon.0) 2164 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.086126+0000 mgr.gibba032.whopeo (mgr.24625) 5681 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:40:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.288449+0000 mon.a (mon.0) 2165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.294376+0000 mgr.gibba032.whopeo (mgr.24625) 5682 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:40:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.296924+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:40:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.297607+0000 mgr.gibba032.whopeo (mgr.24625) 5683 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:40:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.299457+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.299567+0000 mgr.gibba032.whopeo (mgr.24625) 5684 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:40:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.299729+0000 mgr.gibba032.whopeo (mgr.24625) 5685 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:40:19.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.301511+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.301676+0000 mgr.gibba032.whopeo (mgr.24625) 5686 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:40:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.301781+0000 mgr.gibba032.whopeo (mgr.24625) 5687 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:40:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.303280+0000 mon.a (mon.0) 2169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.303437+0000 mgr.gibba032.whopeo (mgr.24625) 5688 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:40:19.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.305276+0000 mgr.gibba032.whopeo (mgr.24625) 5689 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:40:19.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.305517+0000 mgr.gibba032.whopeo (mgr.24625) 5690 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:40:19.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.307555+0000 mon.a (mon.0) 2170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.307943+0000 mgr.gibba032.whopeo (mgr.24625) 5691 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:40:19.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.309016+0000 mgr.gibba032.whopeo (mgr.24625) 5692 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:40:19.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: audit 2021-03-10T15:40:18.310609+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.310900+0000 mgr.gibba032.whopeo (mgr.24625) 5693 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:40:19.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.311009+0000 mgr.gibba032.whopeo (mgr.24625) 5694 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:40:19.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[30791]: cephadm 2021-03-10T15:40:18.311326+0000 mgr.gibba032.whopeo (mgr.24625) 5695 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.086126+0000 mgr.gibba032.whopeo (mgr.24625) 5681 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:40:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.288449+0000 mon.a (mon.0) 2165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:19.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.294376+0000 mgr.gibba032.whopeo (mgr.24625) 5682 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:40:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.296924+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:40:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.297607+0000 mgr.gibba032.whopeo (mgr.24625) 5683 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:40:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.299457+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.299567+0000 mgr.gibba032.whopeo (mgr.24625) 5684 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:40:19.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.299729+0000 mgr.gibba032.whopeo (mgr.24625) 5685 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:40:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.301511+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.301676+0000 mgr.gibba032.whopeo (mgr.24625) 5686 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:40:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.301781+0000 mgr.gibba032.whopeo (mgr.24625) 5687 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:40:19.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.303280+0000 mon.a (mon.0) 2169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.303437+0000 mgr.gibba032.whopeo (mgr.24625) 5688 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:40:19.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.305276+0000 mgr.gibba032.whopeo (mgr.24625) 5689 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:40:19.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.305517+0000 mgr.gibba032.whopeo (mgr.24625) 5690 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:40:19.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.307555+0000 mon.a (mon.0) 2170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.307943+0000 mgr.gibba032.whopeo (mgr.24625) 5691 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:40:19.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.309016+0000 mgr.gibba032.whopeo (mgr.24625) 5692 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:40:19.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: audit 2021-03-10T15:40:18.310609+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.310900+0000 mgr.gibba032.whopeo (mgr.24625) 5693 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:40:19.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.311009+0000 mgr.gibba032.whopeo (mgr.24625) 5694 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:40:19.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:19 gibba011 bash[31774]: cephadm 2021-03-10T15:40:18.311326+0000 mgr.gibba032.whopeo (mgr.24625) 5695 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.086126+0000 mgr.gibba032.whopeo (mgr.24625) 5681 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:40:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.288449+0000 mon.a (mon.0) 2165 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:40:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.294376+0000 mgr.gibba032.whopeo (mgr.24625) 5682 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:40:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.296924+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:40:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.297607+0000 mgr.gibba032.whopeo (mgr.24625) 5683 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:40:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.299457+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.299567+0000 mgr.gibba032.whopeo (mgr.24625) 5684 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:40:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.299729+0000 mgr.gibba032.whopeo (mgr.24625) 5685 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:40:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.301511+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.301676+0000 mgr.gibba032.whopeo (mgr.24625) 5686 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:40:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.301781+0000 mgr.gibba032.whopeo (mgr.24625) 5687 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:40:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.303280+0000 mon.a (mon.0) 2169 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.303437+0000 mgr.gibba032.whopeo (mgr.24625) 5688 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:40:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.305276+0000 mgr.gibba032.whopeo (mgr.24625) 5689 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:40:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.305517+0000 mgr.gibba032.whopeo (mgr.24625) 5690 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:40:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.307555+0000 mon.a (mon.0) 2170 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.307943+0000 mgr.gibba032.whopeo (mgr.24625) 5691 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:40:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.309016+0000 mgr.gibba032.whopeo (mgr.24625) 5692 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:40:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: audit 2021-03-10T15:40:18.310609+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.310900+0000 mgr.gibba032.whopeo (mgr.24625) 5693 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:40:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.311009+0000 mgr.gibba032.whopeo (mgr.24625) 5694 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:40:19.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:19 gibba032 bash[24004]: cephadm 2021-03-10T15:40:18.311326+0000 mgr.gibba032.whopeo (mgr.24625) 5695 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:20 gibba011 bash[30791]: cluster 2021-03-10T15:40:19.124801+0000 mgr.gibba032.whopeo (mgr.24625) 5696 : cluster [DBG] pgmap v3234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:20 gibba011 bash[31774]: cluster 2021-03-10T15:40:19.124801+0000 mgr.gibba032.whopeo (mgr.24625) 5696 : cluster [DBG] pgmap v3234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:20 gibba032 bash[24004]: cluster 2021-03-10T15:40:19.124801+0000 mgr.gibba032.whopeo (mgr.24625) 5696 : cluster [DBG] pgmap v3234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:22 gibba011 bash[30791]: cluster 2021-03-10T15:40:21.126207+0000 mgr.gibba032.whopeo (mgr.24625) 5697 : cluster [DBG] pgmap v3235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:22 gibba011 bash[31774]: cluster 2021-03-10T15:40:21.126207+0000 mgr.gibba032.whopeo (mgr.24625) 5697 : cluster [DBG] pgmap v3235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:22 gibba032 bash[24004]: cluster 2021-03-10T15:40:21.126207+0000 mgr.gibba032.whopeo (mgr.24625) 5697 : cluster [DBG] pgmap v3235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:24 gibba011 bash[30791]: cluster 2021-03-10T15:40:23.127375+0000 mgr.gibba032.whopeo (mgr.24625) 5698 : cluster [DBG] pgmap v3236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:24 gibba011 bash[31774]: cluster 2021-03-10T15:40:23.127375+0000 mgr.gibba032.whopeo (mgr.24625) 5698 : cluster [DBG] pgmap v3236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:24 gibba032 bash[24004]: cluster 2021-03-10T15:40:23.127375+0000 mgr.gibba032.whopeo (mgr.24625) 5698 : cluster [DBG] pgmap v3236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:25.696 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:40:25.923 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:40:25.923 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (107m) 71s ago 112m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:40:25.924 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (112m) 7s ago 112m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:40:25.924 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (107m) 7s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:40:25.924 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (107m) 7s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:40:25.925 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (106m) 71s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:40:25.925 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (106m) 71s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:40:25.925 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (106m) 7s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:40:25.926 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (106m) 71s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:40:25.926 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (112m) 71s ago 112m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:40:25.926 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (112m) 7s ago 112m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:40:25.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (105m) 71s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:40:25.926 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (104m) 71s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:40:25.927 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (104m) 71s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:40:25.927 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (104m) 71s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:40:25.927 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (103m) 7s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:40:25.927 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (103m) 7s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:40:25.927 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (102m) 7s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:40:25.928 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (102m) 7s ago 112m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:40:25.928 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (112m) 7s ago 112m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:40:25.928 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 71s ago 110m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:40:26.169 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:40:26.169 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:40:26.169 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:40:26.170 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:26.170 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:40:26.170 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:40:26.170 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:26.171 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:40:26.171 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:40:26.171 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:26.171 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:40:26.172 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:40:26.172 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:40:26.172 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:40:26.172 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:40:26.193 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[30791]: cluster 2021-03-10T15:40:25.128832+0000 mgr.gibba032.whopeo (mgr.24625) 5699 : cluster [DBG] pgmap v3237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:26.194 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[30791]: audit 2021-03-10T15:40:25.683932+0000 mgr.gibba032.whopeo (mgr.24625) 5700 : audit [DBG] from='client.47871 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:26.194 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[30791]: audit 2021-03-10T15:40:26.165692+0000 mon.c (mon.1) 109 : audit [DBG] from='client.? 172.21.2.111:0/3571005672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:26.194 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[31774]: cluster 2021-03-10T15:40:25.128832+0000 mgr.gibba032.whopeo (mgr.24625) 5699 : cluster [DBG] pgmap v3237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:26.195 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[31774]: audit 2021-03-10T15:40:25.683932+0000 mgr.gibba032.whopeo (mgr.24625) 5700 : audit [DBG] from='client.47871 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:26.195 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:26 gibba011 bash[31774]: audit 2021-03-10T15:40:26.165692+0000 mon.c (mon.1) 109 : audit [DBG] from='client.? 172.21.2.111:0/3571005672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:26 gibba032 bash[24004]: cluster 2021-03-10T15:40:25.128832+0000 mgr.gibba032.whopeo (mgr.24625) 5699 : cluster [DBG] pgmap v3237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:26 gibba032 bash[24004]: audit 2021-03-10T15:40:25.683932+0000 mgr.gibba032.whopeo (mgr.24625) 5700 : audit [DBG] from='client.47871 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:26 gibba032 bash[24004]: audit 2021-03-10T15:40:26.165692+0000 mon.c (mon.1) 109 : audit [DBG] from='client.? 172.21.2.111:0/3571005672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:27.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:27 gibba032 bash[24004]: audit 2021-03-10T15:40:25.906228+0000 mgr.gibba032.whopeo (mgr.24625) 5701 : audit [DBG] from='client.47877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:27.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:27 gibba011 bash[30791]: audit 2021-03-10T15:40:25.906228+0000 mgr.gibba032.whopeo (mgr.24625) 5701 : audit [DBG] from='client.47877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:27.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:27 gibba011 bash[31774]: audit 2021-03-10T15:40:25.906228+0000 mgr.gibba032.whopeo (mgr.24625) 5701 : audit [DBG] from='client.47877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:28 gibba032 bash[24004]: cluster 2021-03-10T15:40:27.130336+0000 mgr.gibba032.whopeo (mgr.24625) 5702 : cluster [DBG] pgmap v3238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:28 gibba011 bash[30791]: cluster 2021-03-10T15:40:27.130336+0000 mgr.gibba032.whopeo (mgr.24625) 5702 : cluster [DBG] pgmap v3238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:28.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:28 gibba011 bash[31774]: cluster 2021-03-10T15:40:27.130336+0000 mgr.gibba032.whopeo (mgr.24625) 5702 : cluster [DBG] pgmap v3238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:30 gibba032 bash[24004]: cluster 2021-03-10T15:40:29.131946+0000 mgr.gibba032.whopeo (mgr.24625) 5703 : cluster [DBG] pgmap v3239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:30 gibba011 bash[30791]: cluster 2021-03-10T15:40:29.131946+0000 mgr.gibba032.whopeo (mgr.24625) 5703 : cluster [DBG] pgmap v3239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:30 gibba011 bash[31774]: cluster 2021-03-10T15:40:29.131946+0000 mgr.gibba032.whopeo (mgr.24625) 5703 : cluster [DBG] pgmap v3239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:32 gibba032 bash[24004]: cluster 2021-03-10T15:40:31.133467+0000 mgr.gibba032.whopeo (mgr.24625) 5704 : cluster [DBG] pgmap v3240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:32 gibba011 bash[30791]: cluster 2021-03-10T15:40:31.133467+0000 mgr.gibba032.whopeo (mgr.24625) 5704 : cluster [DBG] pgmap v3240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:32 gibba011 bash[31774]: cluster 2021-03-10T15:40:31.133467+0000 mgr.gibba032.whopeo (mgr.24625) 5704 : cluster [DBG] pgmap v3240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:34 gibba011 bash[30791]: cluster 2021-03-10T15:40:33.134689+0000 mgr.gibba032.whopeo (mgr.24625) 5705 : cluster [DBG] pgmap v3241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:34.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:34 gibba011 bash[30791]: cephadm 2021-03-10T15:40:33.327313+0000 mgr.gibba032.whopeo (mgr.24625) 5706 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:34 gibba011 bash[31774]: cluster 2021-03-10T15:40:33.134689+0000 mgr.gibba032.whopeo (mgr.24625) 5705 : cluster [DBG] pgmap v3241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:34.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:34 gibba011 bash[31774]: cephadm 2021-03-10T15:40:33.327313+0000 mgr.gibba032.whopeo (mgr.24625) 5706 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:34 gibba032 bash[24004]: cluster 2021-03-10T15:40:33.134689+0000 mgr.gibba032.whopeo (mgr.24625) 5705 : cluster [DBG] pgmap v3241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:34 gibba032 bash[24004]: cephadm 2021-03-10T15:40:33.327313+0000 mgr.gibba032.whopeo (mgr.24625) 5706 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:36 gibba032 bash[24004]: cluster 2021-03-10T15:40:35.136103+0000 mgr.gibba032.whopeo (mgr.24625) 5707 : cluster [DBG] pgmap v3242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:36 gibba011 bash[30791]: cluster 2021-03-10T15:40:35.136103+0000 mgr.gibba032.whopeo (mgr.24625) 5707 : cluster [DBG] pgmap v3242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:36 gibba011 bash[31774]: cluster 2021-03-10T15:40:35.136103+0000 mgr.gibba032.whopeo (mgr.24625) 5707 : cluster [DBG] pgmap v3242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:38.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:38 gibba032 bash[24004]: cluster 2021-03-10T15:40:37.137649+0000 mgr.gibba032.whopeo (mgr.24625) 5708 : cluster [DBG] pgmap v3243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:38.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:38 gibba011 bash[30791]: cluster 2021-03-10T15:40:37.137649+0000 mgr.gibba032.whopeo (mgr.24625) 5708 : cluster [DBG] pgmap v3243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:38 gibba011 bash[31774]: cluster 2021-03-10T15:40:37.137649+0000 mgr.gibba032.whopeo (mgr.24625) 5708 : cluster [DBG] pgmap v3243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:40.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:40 gibba032 bash[24004]: cluster 2021-03-10T15:40:39.139198+0000 mgr.gibba032.whopeo (mgr.24625) 5709 : cluster [DBG] pgmap v3244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:40 gibba011 bash[30791]: cluster 2021-03-10T15:40:39.139198+0000 mgr.gibba032.whopeo (mgr.24625) 5709 : cluster [DBG] pgmap v3244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:40 gibba011 bash[31774]: cluster 2021-03-10T15:40:39.139198+0000 mgr.gibba032.whopeo (mgr.24625) 5709 : cluster [DBG] pgmap v3244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:42 gibba032 bash[24004]: cluster 2021-03-10T15:40:41.140711+0000 mgr.gibba032.whopeo (mgr.24625) 5710 : cluster [DBG] pgmap v3245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:42 gibba011 bash[30791]: cluster 2021-03-10T15:40:41.140711+0000 mgr.gibba032.whopeo (mgr.24625) 5710 : cluster [DBG] pgmap v3245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:42.676 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:42 gibba011 bash[31774]: cluster 2021-03-10T15:40:41.140711+0000 mgr.gibba032.whopeo (mgr.24625) 5710 : cluster [DBG] pgmap v3245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:44.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:44 gibba011 bash[30791]: cluster 2021-03-10T15:40:43.142514+0000 mgr.gibba032.whopeo (mgr.24625) 5711 : cluster [DBG] pgmap v3246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:44.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:44 gibba011 bash[31774]: cluster 2021-03-10T15:40:43.142514+0000 mgr.gibba032.whopeo (mgr.24625) 5711 : cluster [DBG] pgmap v3246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:44 gibba032 bash[24004]: cluster 2021-03-10T15:40:43.142514+0000 mgr.gibba032.whopeo (mgr.24625) 5711 : cluster [DBG] pgmap v3246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:46.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:46 gibba032 bash[24004]: cluster 2021-03-10T15:40:45.143993+0000 mgr.gibba032.whopeo (mgr.24625) 5712 : cluster [DBG] pgmap v3247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:46 gibba011 bash[30791]: cluster 2021-03-10T15:40:45.143993+0000 mgr.gibba032.whopeo (mgr.24625) 5712 : cluster [DBG] pgmap v3247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:46 gibba011 bash[31774]: cluster 2021-03-10T15:40:45.143993+0000 mgr.gibba032.whopeo (mgr.24625) 5712 : cluster [DBG] pgmap v3247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:48 gibba032 bash[24004]: cluster 2021-03-10T15:40:47.145521+0000 mgr.gibba032.whopeo (mgr.24625) 5713 : cluster [DBG] pgmap v3248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:48 gibba011 bash[30791]: cluster 2021-03-10T15:40:47.145521+0000 mgr.gibba032.whopeo (mgr.24625) 5713 : cluster [DBG] pgmap v3248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:48 gibba011 bash[31774]: cluster 2021-03-10T15:40:47.145521+0000 mgr.gibba032.whopeo (mgr.24625) 5713 : cluster [DBG] pgmap v3248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:49 gibba032 bash[24004]: cephadm 2021-03-10T15:40:48.343169+0000 mgr.gibba032.whopeo (mgr.24625) 5714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:49 gibba011 bash[30791]: cephadm 2021-03-10T15:40:48.343169+0000 mgr.gibba032.whopeo (mgr.24625) 5714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:49 gibba011 bash[31774]: cephadm 2021-03-10T15:40:48.343169+0000 mgr.gibba032.whopeo (mgr.24625) 5714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:40:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:50 gibba032 bash[24004]: cluster 2021-03-10T15:40:49.147010+0000 mgr.gibba032.whopeo (mgr.24625) 5715 : cluster [DBG] pgmap v3249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:50 gibba011 bash[30791]: cluster 2021-03-10T15:40:49.147010+0000 mgr.gibba032.whopeo (mgr.24625) 5715 : cluster [DBG] pgmap v3249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:50 gibba011 bash[31774]: cluster 2021-03-10T15:40:49.147010+0000 mgr.gibba032.whopeo (mgr.24625) 5715 : cluster [DBG] pgmap v3249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:52 gibba032 bash[24004]: cluster 2021-03-10T15:40:51.148527+0000 mgr.gibba032.whopeo (mgr.24625) 5716 : cluster [DBG] pgmap v3250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:52 gibba011 bash[30791]: cluster 2021-03-10T15:40:51.148527+0000 mgr.gibba032.whopeo (mgr.24625) 5716 : cluster [DBG] pgmap v3250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:52 gibba011 bash[31774]: cluster 2021-03-10T15:40:51.148527+0000 mgr.gibba032.whopeo (mgr.24625) 5716 : cluster [DBG] pgmap v3250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:54 gibba011 bash[30791]: cluster 2021-03-10T15:40:53.149842+0000 mgr.gibba032.whopeo (mgr.24625) 5717 : cluster [DBG] pgmap v3251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:54 gibba011 bash[31774]: cluster 2021-03-10T15:40:53.149842+0000 mgr.gibba032.whopeo (mgr.24625) 5717 : cluster [DBG] pgmap v3251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:54 gibba032 bash[24004]: cluster 2021-03-10T15:40:53.149842+0000 mgr.gibba032.whopeo (mgr.24625) 5717 : cluster [DBG] pgmap v3251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:56.422 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:40:56.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:56 gibba032 bash[24004]: cluster 2021-03-10T15:40:55.151401+0000 mgr.gibba032.whopeo (mgr.24625) 5718 : cluster [DBG] pgmap v3252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:56 gibba011 bash[30791]: cluster 2021-03-10T15:40:55.151401+0000 mgr.gibba032.whopeo (mgr.24625) 5718 : cluster [DBG] pgmap v3252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:56 gibba011 bash[31774]: cluster 2021-03-10T15:40:55.151401+0000 mgr.gibba032.whopeo (mgr.24625) 5718 : cluster [DBG] pgmap v3252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:56.651 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:40:56.652 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (107m) 102s ago 112m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:40:56.652 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (112m) 38s ago 112m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:40:56.652 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (107m) 38s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:40:56.652 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (108m) 38s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:40:56.653 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (107m) 102s ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:40:56.653 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (107m) 102s ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:40:56.653 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (106m) 38s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:40:56.653 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (106m) 102s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:40:56.654 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (113m) 102s ago 113m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:40:56.654 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (113m) 38s ago 113m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:40:56.654 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (105m) 102s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:40:56.654 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (105m) 102s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:40:56.655 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (105m) 102s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:40:56.655 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (104m) 102s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:40:56.655 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (104m) 38s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:40:56.655 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (103m) 38s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:40:56.656 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (103m) 38s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:40:56.656 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (102m) 38s ago 113m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:40:56.656 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (112m) 38s ago 113m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:40:56.656 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 102s ago 111m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:40:56.906 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:40:56.906 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:40:56.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:40:56.907 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:56.907 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:40:56.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:40:56.908 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:56.908 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:40:56.908 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:40:56.908 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:40:56.909 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:40:56.909 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:40:56.909 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:40:56.909 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:40:56.910 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:40:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.399877+0000 mgr.gibba032.whopeo (mgr.24625) 5719 : audit [DBG] from='client.57874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.635201+0000 mgr.gibba032.whopeo (mgr.24625) 5720 : audit [DBG] from='client.47895 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.673577+0000 mon.a (mon.0) 2172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:40:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.675099+0000 mon.a (mon.0) 2173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:40:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.699689+0000 mon.a (mon.0) 2174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:40:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.699913+0000 mon.a (mon.0) 2175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.700082+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:40:57.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.700257+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:57 gibba032 bash[24004]: audit 2021-03-10T15:40:56.902712+0000 mon.c (mon.1) 110 : audit [DBG] from='client.? 172.21.2.111:0/3502778341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.399877+0000 mgr.gibba032.whopeo (mgr.24625) 5719 : audit [DBG] from='client.57874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.635201+0000 mgr.gibba032.whopeo (mgr.24625) 5720 : audit [DBG] from='client.47895 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.673577+0000 mon.a (mon.0) 2172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:40:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.675099+0000 mon.a (mon.0) 2173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:40:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.699689+0000 mon.a (mon.0) 2174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:40:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.699913+0000 mon.a (mon.0) 2175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.700082+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:40:57.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.700257+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[30791]: audit 2021-03-10T15:40:56.902712+0000 mon.c (mon.1) 110 : audit [DBG] from='client.? 172.21.2.111:0/3502778341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:57.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.399877+0000 mgr.gibba032.whopeo (mgr.24625) 5719 : audit [DBG] from='client.57874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.635201+0000 mgr.gibba032.whopeo (mgr.24625) 5720 : audit [DBG] from='client.47895 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:40:57.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.673577+0000 mon.a (mon.0) 2172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:40:57.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.675099+0000 mon.a (mon.0) 2173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:40:57.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.699689+0000 mon.a (mon.0) 2174 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:40:57.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.699913+0000 mon.a (mon.0) 2175 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.700082+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:40:57.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.700257+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:40:57.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:57 gibba011 bash[31774]: audit 2021-03-10T15:40:56.902712+0000 mon.c (mon.1) 110 : audit [DBG] from='client.? 172.21.2.111:0/3502778341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:40:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:40:58 gibba032 bash[24004]: cluster 2021-03-10T15:40:57.152910+0000 mgr.gibba032.whopeo (mgr.24625) 5721 : cluster [DBG] pgmap v3253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:40:58 gibba011 bash[30791]: cluster 2021-03-10T15:40:57.152910+0000 mgr.gibba032.whopeo (mgr.24625) 5721 : cluster [DBG] pgmap v3253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:40:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:40:58 gibba011 bash[31774]: cluster 2021-03-10T15:40:57.152910+0000 mgr.gibba032.whopeo (mgr.24625) 5721 : cluster [DBG] pgmap v3253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:00.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:00 gibba032 bash[24004]: cluster 2021-03-10T15:40:59.154648+0000 mgr.gibba032.whopeo (mgr.24625) 5722 : cluster [DBG] pgmap v3254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:00 gibba011 bash[30791]: cluster 2021-03-10T15:40:59.154648+0000 mgr.gibba032.whopeo (mgr.24625) 5722 : cluster [DBG] pgmap v3254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:00 gibba011 bash[31774]: cluster 2021-03-10T15:40:59.154648+0000 mgr.gibba032.whopeo (mgr.24625) 5722 : cluster [DBG] pgmap v3254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:02 gibba032 bash[24004]: cluster 2021-03-10T15:41:01.156356+0000 mgr.gibba032.whopeo (mgr.24625) 5723 : cluster [DBG] pgmap v3255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:02 gibba011 bash[30791]: cluster 2021-03-10T15:41:01.156356+0000 mgr.gibba032.whopeo (mgr.24625) 5723 : cluster [DBG] pgmap v3255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:02 gibba011 bash[31774]: cluster 2021-03-10T15:41:01.156356+0000 mgr.gibba032.whopeo (mgr.24625) 5723 : cluster [DBG] pgmap v3255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:04 gibba011 bash[30791]: cluster 2021-03-10T15:41:03.158305+0000 mgr.gibba032.whopeo (mgr.24625) 5724 : cluster [DBG] pgmap v3256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:04.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:04 gibba011 bash[30791]: cephadm 2021-03-10T15:41:03.359170+0000 mgr.gibba032.whopeo (mgr.24625) 5725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:04 gibba011 bash[31774]: cluster 2021-03-10T15:41:03.158305+0000 mgr.gibba032.whopeo (mgr.24625) 5724 : cluster [DBG] pgmap v3256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:04.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:04 gibba011 bash[31774]: cephadm 2021-03-10T15:41:03.359170+0000 mgr.gibba032.whopeo (mgr.24625) 5725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:04 gibba032 bash[24004]: cluster 2021-03-10T15:41:03.158305+0000 mgr.gibba032.whopeo (mgr.24625) 5724 : cluster [DBG] pgmap v3256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:04 gibba032 bash[24004]: cephadm 2021-03-10T15:41:03.359170+0000 mgr.gibba032.whopeo (mgr.24625) 5725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:06 gibba032 bash[24004]: cluster 2021-03-10T15:41:05.160088+0000 mgr.gibba032.whopeo (mgr.24625) 5726 : cluster [DBG] pgmap v3257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:06 gibba011 bash[30791]: cluster 2021-03-10T15:41:05.160088+0000 mgr.gibba032.whopeo (mgr.24625) 5726 : cluster [DBG] pgmap v3257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:06 gibba011 bash[31774]: cluster 2021-03-10T15:41:05.160088+0000 mgr.gibba032.whopeo (mgr.24625) 5726 : cluster [DBG] pgmap v3257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:08.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:08 gibba032 bash[24004]: cluster 2021-03-10T15:41:07.161549+0000 mgr.gibba032.whopeo (mgr.24625) 5727 : cluster [DBG] pgmap v3258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:08 gibba011 bash[30791]: cluster 2021-03-10T15:41:07.161549+0000 mgr.gibba032.whopeo (mgr.24625) 5727 : cluster [DBG] pgmap v3258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:08 gibba011 bash[31774]: cluster 2021-03-10T15:41:07.161549+0000 mgr.gibba032.whopeo (mgr.24625) 5727 : cluster [DBG] pgmap v3258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:10.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:10 gibba032 bash[24004]: cluster 2021-03-10T15:41:09.163224+0000 mgr.gibba032.whopeo (mgr.24625) 5728 : cluster [DBG] pgmap v3259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:10 gibba011 bash[30791]: cluster 2021-03-10T15:41:09.163224+0000 mgr.gibba032.whopeo (mgr.24625) 5728 : cluster [DBG] pgmap v3259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:10 gibba011 bash[31774]: cluster 2021-03-10T15:41:09.163224+0000 mgr.gibba032.whopeo (mgr.24625) 5728 : cluster [DBG] pgmap v3259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:12 gibba032 bash[24004]: cluster 2021-03-10T15:41:11.164913+0000 mgr.gibba032.whopeo (mgr.24625) 5729 : cluster [DBG] pgmap v3260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:12 gibba011 bash[30791]: cluster 2021-03-10T15:41:11.164913+0000 mgr.gibba032.whopeo (mgr.24625) 5729 : cluster [DBG] pgmap v3260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:12 gibba011 bash[31774]: cluster 2021-03-10T15:41:11.164913+0000 mgr.gibba032.whopeo (mgr.24625) 5729 : cluster [DBG] pgmap v3260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:14 gibba032 bash[24004]: cluster 2021-03-10T15:41:13.166852+0000 mgr.gibba032.whopeo (mgr.24625) 5730 : cluster [DBG] pgmap v3261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:14 gibba011 bash[30791]: cluster 2021-03-10T15:41:13.166852+0000 mgr.gibba032.whopeo (mgr.24625) 5730 : cluster [DBG] pgmap v3261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:14 gibba011 bash[31774]: cluster 2021-03-10T15:41:13.166852+0000 mgr.gibba032.whopeo (mgr.24625) 5730 : cluster [DBG] pgmap v3261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:16.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:16 gibba032 bash[24004]: cluster 2021-03-10T15:41:15.168652+0000 mgr.gibba032.whopeo (mgr.24625) 5731 : cluster [DBG] pgmap v3262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:16 gibba011 bash[30791]: cluster 2021-03-10T15:41:15.168652+0000 mgr.gibba032.whopeo (mgr.24625) 5731 : cluster [DBG] pgmap v3262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:16 gibba011 bash[31774]: cluster 2021-03-10T15:41:15.168652+0000 mgr.gibba032.whopeo (mgr.24625) 5731 : cluster [DBG] pgmap v3262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:18.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:18 gibba032 bash[24004]: cluster 2021-03-10T15:41:17.170441+0000 mgr.gibba032.whopeo (mgr.24625) 5732 : cluster [DBG] pgmap v3263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:18 gibba011 bash[30791]: cluster 2021-03-10T15:41:17.170441+0000 mgr.gibba032.whopeo (mgr.24625) 5732 : cluster [DBG] pgmap v3263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:18 gibba011 bash[31774]: cluster 2021-03-10T15:41:17.170441+0000 mgr.gibba032.whopeo (mgr.24625) 5732 : cluster [DBG] pgmap v3263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:20.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.373672+0000 mgr.gibba032.whopeo (mgr.24625) 5733 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:41:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.375251+0000 mgr.gibba032.whopeo (mgr.24625) 5734 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:41:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.598229+0000 mon.a (mon.0) 2178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:41:20.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.599896+0000 mgr.gibba032.whopeo (mgr.24625) 5735 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:41:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.601812+0000 mon.a (mon.0) 2179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:41:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.601867+0000 mgr.gibba032.whopeo (mgr.24625) 5736 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:41:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.603510+0000 mon.a (mon.0) 2180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.603622+0000 mgr.gibba032.whopeo (mgr.24625) 5737 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:41:20.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.603658+0000 mgr.gibba032.whopeo (mgr.24625) 5738 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:41:20.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.605253+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.605404+0000 mgr.gibba032.whopeo (mgr.24625) 5739 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:41:20.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.605443+0000 mgr.gibba032.whopeo (mgr.24625) 5740 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:41:20.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.606727+0000 mgr.gibba032.whopeo (mgr.24625) 5741 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:41:20.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.606958+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.609175+0000 mgr.gibba032.whopeo (mgr.24625) 5742 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:41:20.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.609400+0000 mgr.gibba032.whopeo (mgr.24625) 5743 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:41:20.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.612384+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.613434+0000 mgr.gibba032.whopeo (mgr.24625) 5744 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:41:20.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.615520+0000 mgr.gibba032.whopeo (mgr.24625) 5745 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:41:20.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: audit 2021-03-10T15:41:18.617744+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.618872+0000 mgr.gibba032.whopeo (mgr.24625) 5746 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:41:20.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.619166+0000 mgr.gibba032.whopeo (mgr.24625) 5747 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:41:20.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:19 gibba032 bash[24004]: cephadm 2021-03-10T15:41:18.619920+0000 mgr.gibba032.whopeo (mgr.24625) 5748 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:20.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.373672+0000 mgr.gibba032.whopeo (mgr.24625) 5733 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:41:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.375251+0000 mgr.gibba032.whopeo (mgr.24625) 5734 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:41:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.598229+0000 mon.a (mon.0) 2178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:41:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.599896+0000 mgr.gibba032.whopeo (mgr.24625) 5735 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:41:20.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.601812+0000 mon.a (mon.0) 2179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:41:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.601867+0000 mgr.gibba032.whopeo (mgr.24625) 5736 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:41:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.603510+0000 mon.a (mon.0) 2180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.603622+0000 mgr.gibba032.whopeo (mgr.24625) 5737 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:41:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.603658+0000 mgr.gibba032.whopeo (mgr.24625) 5738 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:41:20.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.605253+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.605404+0000 mgr.gibba032.whopeo (mgr.24625) 5739 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:41:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.605443+0000 mgr.gibba032.whopeo (mgr.24625) 5740 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:41:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.606727+0000 mgr.gibba032.whopeo (mgr.24625) 5741 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:41:20.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.606958+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.609175+0000 mgr.gibba032.whopeo (mgr.24625) 5742 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:41:20.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.609400+0000 mgr.gibba032.whopeo (mgr.24625) 5743 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:41:20.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.612384+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.613434+0000 mgr.gibba032.whopeo (mgr.24625) 5744 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:41:20.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.615520+0000 mgr.gibba032.whopeo (mgr.24625) 5745 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:41:20.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: audit 2021-03-10T15:41:18.617744+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.618872+0000 mgr.gibba032.whopeo (mgr.24625) 5746 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:41:20.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.619166+0000 mgr.gibba032.whopeo (mgr.24625) 5747 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:41:20.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[30791]: cephadm 2021-03-10T15:41:18.619920+0000 mgr.gibba032.whopeo (mgr.24625) 5748 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:20.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.373672+0000 mgr.gibba032.whopeo (mgr.24625) 5733 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:41:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.375251+0000 mgr.gibba032.whopeo (mgr.24625) 5734 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:41:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.598229+0000 mon.a (mon.0) 2178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:41:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.599896+0000 mgr.gibba032.whopeo (mgr.24625) 5735 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:41:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.601812+0000 mon.a (mon.0) 2179 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:41:20.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.601867+0000 mgr.gibba032.whopeo (mgr.24625) 5736 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:41:20.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.603510+0000 mon.a (mon.0) 2180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.603622+0000 mgr.gibba032.whopeo (mgr.24625) 5737 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:41:20.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.603658+0000 mgr.gibba032.whopeo (mgr.24625) 5738 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:41:20.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.605253+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.605404+0000 mgr.gibba032.whopeo (mgr.24625) 5739 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:41:20.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.605443+0000 mgr.gibba032.whopeo (mgr.24625) 5740 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:41:20.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.606727+0000 mgr.gibba032.whopeo (mgr.24625) 5741 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:41:20.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.606958+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.609175+0000 mgr.gibba032.whopeo (mgr.24625) 5742 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:41:20.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.609400+0000 mgr.gibba032.whopeo (mgr.24625) 5743 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:41:20.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.612384+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.613434+0000 mgr.gibba032.whopeo (mgr.24625) 5744 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:41:20.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.615520+0000 mgr.gibba032.whopeo (mgr.24625) 5745 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:41:20.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: audit 2021-03-10T15:41:18.617744+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:20.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.618872+0000 mgr.gibba032.whopeo (mgr.24625) 5746 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:41:20.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.619166+0000 mgr.gibba032.whopeo (mgr.24625) 5747 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:41:20.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:19 gibba011 bash[31774]: cephadm 2021-03-10T15:41:18.619920+0000 mgr.gibba032.whopeo (mgr.24625) 5748 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:20 gibba032 bash[24004]: cluster 2021-03-10T15:41:19.171938+0000 mgr.gibba032.whopeo (mgr.24625) 5749 : cluster [DBG] pgmap v3264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:21.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:20 gibba011 bash[30791]: cluster 2021-03-10T15:41:19.171938+0000 mgr.gibba032.whopeo (mgr.24625) 5749 : cluster [DBG] pgmap v3264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:21.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:20 gibba011 bash[31774]: cluster 2021-03-10T15:41:19.171938+0000 mgr.gibba032.whopeo (mgr.24625) 5749 : cluster [DBG] pgmap v3264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:22 gibba032 bash[24004]: cluster 2021-03-10T15:41:21.173406+0000 mgr.gibba032.whopeo (mgr.24625) 5750 : cluster [DBG] pgmap v3265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:23.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:22 gibba011 bash[30791]: cluster 2021-03-10T15:41:21.173406+0000 mgr.gibba032.whopeo (mgr.24625) 5750 : cluster [DBG] pgmap v3265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:23.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:22 gibba011 bash[31774]: cluster 2021-03-10T15:41:21.173406+0000 mgr.gibba032.whopeo (mgr.24625) 5750 : cluster [DBG] pgmap v3265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:24 gibba032 bash[24004]: cluster 2021-03-10T15:41:23.175195+0000 mgr.gibba032.whopeo (mgr.24625) 5751 : cluster [DBG] pgmap v3266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:24 gibba011 bash[30791]: cluster 2021-03-10T15:41:23.175195+0000 mgr.gibba032.whopeo (mgr.24625) 5751 : cluster [DBG] pgmap v3266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:24 gibba011 bash[31774]: cluster 2021-03-10T15:41:23.175195+0000 mgr.gibba032.whopeo (mgr.24625) 5751 : cluster [DBG] pgmap v3266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:26 gibba032 bash[24004]: cluster 2021-03-10T15:41:25.176682+0000 mgr.gibba032.whopeo (mgr.24625) 5752 : cluster [DBG] pgmap v3267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:26 gibba011 bash[30791]: cluster 2021-03-10T15:41:25.176682+0000 mgr.gibba032.whopeo (mgr.24625) 5752 : cluster [DBG] pgmap v3267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:26 gibba011 bash[31774]: cluster 2021-03-10T15:41:25.176682+0000 mgr.gibba032.whopeo (mgr.24625) 5752 : cluster [DBG] pgmap v3267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:27.152 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:41:27.380 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:41:27.381 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (108m) 2m ago 113m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:41:27.381 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (113m) 69s ago 113m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:41:27.381 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (108m) 69s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:41:27.381 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (108m) 69s ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:41:27.382 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (107m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:41:27.382 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (107m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:41:27.382 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (107m) 69s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:41:27.382 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (107m) 2m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:41:27.383 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (113m) 2m ago 113m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:41:27.383 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (113m) 69s ago 113m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:41:27.383 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (106m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:41:27.383 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (105m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:41:27.383 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (105m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:41:27.384 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (105m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:41:27.384 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (104m) 69s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:41:27.384 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (104m) 69s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:41:27.384 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (103m) 69s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:41:27.384 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (103m) 69s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:41:27.385 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (113m) 69s ago 113m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:41:27.385 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 111m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:41:27.632 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:41:27.632 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:41:27.633 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:41:27.633 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:27.633 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:41:27.634 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:41:27.634 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:27.634 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:41:27.634 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:41:27.634 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:27.635 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:41:27.635 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:41:27.635 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:41:27.635 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:41:27.636 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:41:28.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:28 gibba032 bash[24004]: audit 2021-03-10T15:41:27.137858+0000 mgr.gibba032.whopeo (mgr.24625) 5753 : audit [DBG] from='client.57892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:28 gibba032 bash[24004]: cluster 2021-03-10T15:41:27.178199+0000 mgr.gibba032.whopeo (mgr.24625) 5754 : cluster [DBG] pgmap v3268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:28 gibba032 bash[24004]: audit 2021-03-10T15:41:27.364221+0000 mgr.gibba032.whopeo (mgr.24625) 5755 : audit [DBG] from='client.47913 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:28 gibba032 bash[24004]: audit 2021-03-10T15:41:27.628887+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.2.111:0/2734789154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[30791]: audit 2021-03-10T15:41:27.137858+0000 mgr.gibba032.whopeo (mgr.24625) 5753 : audit [DBG] from='client.57892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[30791]: cluster 2021-03-10T15:41:27.178199+0000 mgr.gibba032.whopeo (mgr.24625) 5754 : cluster [DBG] pgmap v3268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:28.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[30791]: audit 2021-03-10T15:41:27.364221+0000 mgr.gibba032.whopeo (mgr.24625) 5755 : audit [DBG] from='client.47913 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[30791]: audit 2021-03-10T15:41:27.628887+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.2.111:0/2734789154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:28.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[31774]: audit 2021-03-10T15:41:27.137858+0000 mgr.gibba032.whopeo (mgr.24625) 5753 : audit [DBG] from='client.57892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[31774]: cluster 2021-03-10T15:41:27.178199+0000 mgr.gibba032.whopeo (mgr.24625) 5754 : cluster [DBG] pgmap v3268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:28.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[31774]: audit 2021-03-10T15:41:27.364221+0000 mgr.gibba032.whopeo (mgr.24625) 5755 : audit [DBG] from='client.47913 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:28.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:28 gibba011 bash[31774]: audit 2021-03-10T15:41:27.628887+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.2.111:0/2734789154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:30 gibba032 bash[24004]: cluster 2021-03-10T15:41:29.179537+0000 mgr.gibba032.whopeo (mgr.24625) 5756 : cluster [DBG] pgmap v3269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:30 gibba011 bash[30791]: cluster 2021-03-10T15:41:29.179537+0000 mgr.gibba032.whopeo (mgr.24625) 5756 : cluster [DBG] pgmap v3269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:30 gibba011 bash[31774]: cluster 2021-03-10T15:41:29.179537+0000 mgr.gibba032.whopeo (mgr.24625) 5756 : cluster [DBG] pgmap v3269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:32 gibba011 bash[30791]: cluster 2021-03-10T15:41:31.181227+0000 mgr.gibba032.whopeo (mgr.24625) 5757 : cluster [DBG] pgmap v3270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:32 gibba011 bash[31774]: cluster 2021-03-10T15:41:31.181227+0000 mgr.gibba032.whopeo (mgr.24625) 5757 : cluster [DBG] pgmap v3270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:32 gibba032 bash[24004]: cluster 2021-03-10T15:41:31.181227+0000 mgr.gibba032.whopeo (mgr.24625) 5757 : cluster [DBG] pgmap v3270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:34 gibba032 bash[24004]: cluster 2021-03-10T15:41:33.183208+0000 mgr.gibba032.whopeo (mgr.24625) 5758 : cluster [DBG] pgmap v3271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:34.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:34 gibba032 bash[24004]: cephadm 2021-03-10T15:41:33.622111+0000 mgr.gibba032.whopeo (mgr.24625) 5759 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:34 gibba011 bash[30791]: cluster 2021-03-10T15:41:33.183208+0000 mgr.gibba032.whopeo (mgr.24625) 5758 : cluster [DBG] pgmap v3271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:34.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:34 gibba011 bash[30791]: cephadm 2021-03-10T15:41:33.622111+0000 mgr.gibba032.whopeo (mgr.24625) 5759 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:34 gibba011 bash[31774]: cluster 2021-03-10T15:41:33.183208+0000 mgr.gibba032.whopeo (mgr.24625) 5758 : cluster [DBG] pgmap v3271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:34.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:34 gibba011 bash[31774]: cephadm 2021-03-10T15:41:33.622111+0000 mgr.gibba032.whopeo (mgr.24625) 5759 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:36 gibba032 bash[24004]: cluster 2021-03-10T15:41:35.184927+0000 mgr.gibba032.whopeo (mgr.24625) 5760 : cluster [DBG] pgmap v3272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:36 gibba011 bash[30791]: cluster 2021-03-10T15:41:35.184927+0000 mgr.gibba032.whopeo (mgr.24625) 5760 : cluster [DBG] pgmap v3272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:36.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:36 gibba011 bash[31774]: cluster 2021-03-10T15:41:35.184927+0000 mgr.gibba032.whopeo (mgr.24625) 5760 : cluster [DBG] pgmap v3272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:38 gibba011 bash[30791]: cluster 2021-03-10T15:41:37.186714+0000 mgr.gibba032.whopeo (mgr.24625) 5761 : cluster [DBG] pgmap v3273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:38 gibba011 bash[31774]: cluster 2021-03-10T15:41:37.186714+0000 mgr.gibba032.whopeo (mgr.24625) 5761 : cluster [DBG] pgmap v3273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:38 gibba032 bash[24004]: cluster 2021-03-10T15:41:37.186714+0000 mgr.gibba032.whopeo (mgr.24625) 5761 : cluster [DBG] pgmap v3273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:40 gibba011 bash[30791]: cluster 2021-03-10T15:41:39.188323+0000 mgr.gibba032.whopeo (mgr.24625) 5762 : cluster [DBG] pgmap v3274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:40 gibba011 bash[31774]: cluster 2021-03-10T15:41:39.188323+0000 mgr.gibba032.whopeo (mgr.24625) 5762 : cluster [DBG] pgmap v3274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:40 gibba032 bash[24004]: cluster 2021-03-10T15:41:39.188323+0000 mgr.gibba032.whopeo (mgr.24625) 5762 : cluster [DBG] pgmap v3274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:42 gibba011 bash[30791]: cluster 2021-03-10T15:41:41.189773+0000 mgr.gibba032.whopeo (mgr.24625) 5763 : cluster [DBG] pgmap v3275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:42 gibba011 bash[31774]: cluster 2021-03-10T15:41:41.189773+0000 mgr.gibba032.whopeo (mgr.24625) 5763 : cluster [DBG] pgmap v3275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:42 gibba032 bash[24004]: cluster 2021-03-10T15:41:41.189773+0000 mgr.gibba032.whopeo (mgr.24625) 5763 : cluster [DBG] pgmap v3275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:44 gibba032 bash[24004]: cluster 2021-03-10T15:41:43.191453+0000 mgr.gibba032.whopeo (mgr.24625) 5764 : cluster [DBG] pgmap v3276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:44 gibba011 bash[30791]: cluster 2021-03-10T15:41:43.191453+0000 mgr.gibba032.whopeo (mgr.24625) 5764 : cluster [DBG] pgmap v3276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:44.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:44 gibba011 bash[31774]: cluster 2021-03-10T15:41:43.191453+0000 mgr.gibba032.whopeo (mgr.24625) 5764 : cluster [DBG] pgmap v3276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:46 gibba011 bash[30791]: cluster 2021-03-10T15:41:45.192845+0000 mgr.gibba032.whopeo (mgr.24625) 5765 : cluster [DBG] pgmap v3277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:46 gibba011 bash[31774]: cluster 2021-03-10T15:41:45.192845+0000 mgr.gibba032.whopeo (mgr.24625) 5765 : cluster [DBG] pgmap v3277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:46.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:46 gibba032 bash[24004]: cluster 2021-03-10T15:41:45.192845+0000 mgr.gibba032.whopeo (mgr.24625) 5765 : cluster [DBG] pgmap v3277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:48 gibba011 bash[30791]: cluster 2021-03-10T15:41:47.194218+0000 mgr.gibba032.whopeo (mgr.24625) 5766 : cluster [DBG] pgmap v3278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:48 gibba011 bash[31774]: cluster 2021-03-10T15:41:47.194218+0000 mgr.gibba032.whopeo (mgr.24625) 5766 : cluster [DBG] pgmap v3278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:48.643 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:48 gibba032 bash[24004]: cluster 2021-03-10T15:41:47.194218+0000 mgr.gibba032.whopeo (mgr.24625) 5766 : cluster [DBG] pgmap v3278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:49 gibba011 bash[30791]: cephadm 2021-03-10T15:41:48.638011+0000 mgr.gibba032.whopeo (mgr.24625) 5767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:49 gibba011 bash[31774]: cephadm 2021-03-10T15:41:48.638011+0000 mgr.gibba032.whopeo (mgr.24625) 5767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:49.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:49 gibba032 bash[24004]: cephadm 2021-03-10T15:41:48.638011+0000 mgr.gibba032.whopeo (mgr.24625) 5767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:41:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:50 gibba011 bash[30791]: cluster 2021-03-10T15:41:49.195684+0000 mgr.gibba032.whopeo (mgr.24625) 5768 : cluster [DBG] pgmap v3279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:50 gibba011 bash[31774]: cluster 2021-03-10T15:41:49.195684+0000 mgr.gibba032.whopeo (mgr.24625) 5768 : cluster [DBG] pgmap v3279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:50 gibba032 bash[24004]: cluster 2021-03-10T15:41:49.195684+0000 mgr.gibba032.whopeo (mgr.24625) 5768 : cluster [DBG] pgmap v3279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:52 gibba011 bash[30791]: cluster 2021-03-10T15:41:51.197349+0000 mgr.gibba032.whopeo (mgr.24625) 5769 : cluster [DBG] pgmap v3280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:52 gibba011 bash[31774]: cluster 2021-03-10T15:41:51.197349+0000 mgr.gibba032.whopeo (mgr.24625) 5769 : cluster [DBG] pgmap v3280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:52 gibba032 bash[24004]: cluster 2021-03-10T15:41:51.197349+0000 mgr.gibba032.whopeo (mgr.24625) 5769 : cluster [DBG] pgmap v3280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:54 gibba032 bash[24004]: cluster 2021-03-10T15:41:53.199125+0000 mgr.gibba032.whopeo (mgr.24625) 5770 : cluster [DBG] pgmap v3281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:54 gibba011 bash[30791]: cluster 2021-03-10T15:41:53.199125+0000 mgr.gibba032.whopeo (mgr.24625) 5770 : cluster [DBG] pgmap v3281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:54.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:54 gibba011 bash[31774]: cluster 2021-03-10T15:41:53.199125+0000 mgr.gibba032.whopeo (mgr.24625) 5770 : cluster [DBG] pgmap v3281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:56 gibba011 bash[30791]: cluster 2021-03-10T15:41:55.200600+0000 mgr.gibba032.whopeo (mgr.24625) 5771 : cluster [DBG] pgmap v3282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:56 gibba011 bash[31774]: cluster 2021-03-10T15:41:55.200600+0000 mgr.gibba032.whopeo (mgr.24625) 5771 : cluster [DBG] pgmap v3282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:56.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:56 gibba032 bash[24004]: cluster 2021-03-10T15:41:55.200600+0000 mgr.gibba032.whopeo (mgr.24625) 5771 : cluster [DBG] pgmap v3282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.677925+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:41:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.683199+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:41:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.703736+0000 mon.a (mon.0) 2188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:41:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.704504+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.705227+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[30791]: audit 2021-03-10T15:41:56.705901+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:41:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.677925+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:41:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.683199+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:41:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.703736+0000 mon.a (mon.0) 2188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:41:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.704504+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.705227+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:57 gibba011 bash[31774]: audit 2021-03-10T15:41:56.705901+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.677925+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:41:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.683199+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:41:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.703736+0000 mon.a (mon.0) 2188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:41:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.704504+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.705227+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:41:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:57 gibba032 bash[24004]: audit 2021-03-10T15:41:56.705901+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:41:57.876 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:41:58.104 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:41:58.105 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (108m) 2m ago 113m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:41:58.105 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (113m) 100s ago 113m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:41:58.105 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (108m) 100s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:41:58.105 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (109m) 100s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:41:58.105 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (108m) 2m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:41:58.106 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (108m) 2m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:41:58.106 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (107m) 100s ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:41:58.106 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (107m) 2m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:41:58.106 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (114m) 2m ago 114m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:41:58.107 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (114m) 100s ago 114m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:41:58.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (106m) 2m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:41:58.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (106m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:41:58.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (106m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:41:58.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (105m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:41:58.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (105m) 100s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:41:58.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (104m) 100s ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:41:58.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (104m) 100s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:41:58.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (103m) 100s ago 114m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:41:58.108 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (113m) 100s ago 114m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:41:58.109 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 112m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:41:58.356 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:41:58.356 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:41:58.356 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:41:58.357 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:58.357 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:41:58.357 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:41:58.357 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:58.358 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:41:58.358 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:41:58.358 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:41:58.358 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:41:58.359 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:41:58.359 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:41:58.359 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:41:58.359 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:58 gibba011 bash[30791]: cluster 2021-03-10T15:41:57.202276+0000 mgr.gibba032.whopeo (mgr.24625) 5772 : cluster [DBG] pgmap v3283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:58 gibba011 bash[31774]: cluster 2021-03-10T15:41:57.202276+0000 mgr.gibba032.whopeo (mgr.24625) 5772 : cluster [DBG] pgmap v3283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:58 gibba032 bash[24004]: cluster 2021-03-10T15:41:57.202276+0000 mgr.gibba032.whopeo (mgr.24625) 5772 : cluster [DBG] pgmap v3283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[30791]: audit 2021-03-10T15:41:57.862640+0000 mgr.gibba032.whopeo (mgr.24625) 5773 : audit [DBG] from='client.57910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[30791]: audit 2021-03-10T15:41:58.092811+0000 mgr.gibba032.whopeo (mgr.24625) 5774 : audit [DBG] from='client.57916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[30791]: audit 2021-03-10T15:41:58.353169+0000 mon.a (mon.0) 2192 : audit [DBG] from='client.? 172.21.2.111:0/304310352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:59.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[31774]: audit 2021-03-10T15:41:57.862640+0000 mgr.gibba032.whopeo (mgr.24625) 5773 : audit [DBG] from='client.57910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[31774]: audit 2021-03-10T15:41:58.092811+0000 mgr.gibba032.whopeo (mgr.24625) 5774 : audit [DBG] from='client.57916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:41:59 gibba011 bash[31774]: audit 2021-03-10T15:41:58.353169+0000 mon.a (mon.0) 2192 : audit [DBG] from='client.? 172.21.2.111:0/304310352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:59 gibba032 bash[24004]: audit 2021-03-10T15:41:57.862640+0000 mgr.gibba032.whopeo (mgr.24625) 5773 : audit [DBG] from='client.57910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:59 gibba032 bash[24004]: audit 2021-03-10T15:41:58.092811+0000 mgr.gibba032.whopeo (mgr.24625) 5774 : audit [DBG] from='client.57916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:41:59 gibba032 bash[24004]: audit 2021-03-10T15:41:58.353169+0000 mon.a (mon.0) 2192 : audit [DBG] from='client.? 172.21.2.111:0/304310352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:00.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:00 gibba011 bash[30791]: cluster 2021-03-10T15:41:59.203753+0000 mgr.gibba032.whopeo (mgr.24625) 5775 : cluster [DBG] pgmap v3284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:00 gibba011 bash[31774]: cluster 2021-03-10T15:41:59.203753+0000 mgr.gibba032.whopeo (mgr.24625) 5775 : cluster [DBG] pgmap v3284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:00 gibba032 bash[24004]: cluster 2021-03-10T15:41:59.203753+0000 mgr.gibba032.whopeo (mgr.24625) 5775 : cluster [DBG] pgmap v3284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:02 gibba011 bash[30791]: cluster 2021-03-10T15:42:01.205475+0000 mgr.gibba032.whopeo (mgr.24625) 5776 : cluster [DBG] pgmap v3285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:02 gibba011 bash[31774]: cluster 2021-03-10T15:42:01.205475+0000 mgr.gibba032.whopeo (mgr.24625) 5776 : cluster [DBG] pgmap v3285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:02 gibba032 bash[24004]: cluster 2021-03-10T15:42:01.205475+0000 mgr.gibba032.whopeo (mgr.24625) 5776 : cluster [DBG] pgmap v3285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:04.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:04 gibba032 bash[24004]: cluster 2021-03-10T15:42:03.207224+0000 mgr.gibba032.whopeo (mgr.24625) 5777 : cluster [DBG] pgmap v3286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:04.590 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:04 gibba032 bash[24004]: cephadm 2021-03-10T15:42:03.654169+0000 mgr.gibba032.whopeo (mgr.24625) 5778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:04.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:04 gibba011 bash[30791]: cluster 2021-03-10T15:42:03.207224+0000 mgr.gibba032.whopeo (mgr.24625) 5777 : cluster [DBG] pgmap v3286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:04.592 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:04 gibba011 bash[30791]: cephadm 2021-03-10T15:42:03.654169+0000 mgr.gibba032.whopeo (mgr.24625) 5778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:04.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:04 gibba011 bash[31774]: cluster 2021-03-10T15:42:03.207224+0000 mgr.gibba032.whopeo (mgr.24625) 5777 : cluster [DBG] pgmap v3286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:04.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:04 gibba011 bash[31774]: cephadm 2021-03-10T15:42:03.654169+0000 mgr.gibba032.whopeo (mgr.24625) 5778 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:06 gibba011 bash[30791]: cluster 2021-03-10T15:42:05.208628+0000 mgr.gibba032.whopeo (mgr.24625) 5779 : cluster [DBG] pgmap v3287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:06.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:06 gibba011 bash[31774]: cluster 2021-03-10T15:42:05.208628+0000 mgr.gibba032.whopeo (mgr.24625) 5779 : cluster [DBG] pgmap v3287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:06 gibba032 bash[24004]: cluster 2021-03-10T15:42:05.208628+0000 mgr.gibba032.whopeo (mgr.24625) 5779 : cluster [DBG] pgmap v3287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:08 gibba011 bash[30791]: cluster 2021-03-10T15:42:07.210224+0000 mgr.gibba032.whopeo (mgr.24625) 5780 : cluster [DBG] pgmap v3288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:08.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:08 gibba011 bash[31774]: cluster 2021-03-10T15:42:07.210224+0000 mgr.gibba032.whopeo (mgr.24625) 5780 : cluster [DBG] pgmap v3288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:08 gibba032 bash[24004]: cluster 2021-03-10T15:42:07.210224+0000 mgr.gibba032.whopeo (mgr.24625) 5780 : cluster [DBG] pgmap v3288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:10 gibba011 bash[30791]: cluster 2021-03-10T15:42:09.211649+0000 mgr.gibba032.whopeo (mgr.24625) 5781 : cluster [DBG] pgmap v3289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:10 gibba011 bash[31774]: cluster 2021-03-10T15:42:09.211649+0000 mgr.gibba032.whopeo (mgr.24625) 5781 : cluster [DBG] pgmap v3289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:10.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:10 gibba032 bash[24004]: cluster 2021-03-10T15:42:09.211649+0000 mgr.gibba032.whopeo (mgr.24625) 5781 : cluster [DBG] pgmap v3289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:12.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:12 gibba011 bash[30791]: cluster 2021-03-10T15:42:11.213215+0000 mgr.gibba032.whopeo (mgr.24625) 5782 : cluster [DBG] pgmap v3290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:12 gibba011 bash[31774]: cluster 2021-03-10T15:42:11.213215+0000 mgr.gibba032.whopeo (mgr.24625) 5782 : cluster [DBG] pgmap v3290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:12 gibba032 bash[24004]: cluster 2021-03-10T15:42:11.213215+0000 mgr.gibba032.whopeo (mgr.24625) 5782 : cluster [DBG] pgmap v3290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:14.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:14 gibba032 bash[24004]: cluster 2021-03-10T15:42:13.214983+0000 mgr.gibba032.whopeo (mgr.24625) 5783 : cluster [DBG] pgmap v3291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:14 gibba011 bash[30791]: cluster 2021-03-10T15:42:13.214983+0000 mgr.gibba032.whopeo (mgr.24625) 5783 : cluster [DBG] pgmap v3291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:14.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:14 gibba011 bash[31774]: cluster 2021-03-10T15:42:13.214983+0000 mgr.gibba032.whopeo (mgr.24625) 5783 : cluster [DBG] pgmap v3291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:16 gibba011 bash[30791]: cluster 2021-03-10T15:42:15.216481+0000 mgr.gibba032.whopeo (mgr.24625) 5784 : cluster [DBG] pgmap v3292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:16 gibba011 bash[31774]: cluster 2021-03-10T15:42:15.216481+0000 mgr.gibba032.whopeo (mgr.24625) 5784 : cluster [DBG] pgmap v3292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:16 gibba032 bash[24004]: cluster 2021-03-10T15:42:15.216481+0000 mgr.gibba032.whopeo (mgr.24625) 5784 : cluster [DBG] pgmap v3292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:18.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:18 gibba011 bash[30791]: cluster 2021-03-10T15:42:17.218111+0000 mgr.gibba032.whopeo (mgr.24625) 5785 : cluster [DBG] pgmap v3293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:18.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:18 gibba011 bash[31774]: cluster 2021-03-10T15:42:17.218111+0000 mgr.gibba032.whopeo (mgr.24625) 5785 : cluster [DBG] pgmap v3293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:18 gibba032 bash[24004]: cluster 2021-03-10T15:42:17.218111+0000 mgr.gibba032.whopeo (mgr.24625) 5785 : cluster [DBG] pgmap v3293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:19.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.670764+0000 mgr.gibba032.whopeo (mgr.24625) 5786 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:42:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.672372+0000 mgr.gibba032.whopeo (mgr.24625) 5787 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:42:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.921721+0000 mon.a (mon.0) 2193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:42:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.925380+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:42:19.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.926941+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.928470+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.929667+0000 mon.a (mon.0) 2197 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.933461+0000 mon.a (mon.0) 2198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[30791]: audit 2021-03-10T15:42:18.936027+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.670764+0000 mgr.gibba032.whopeo (mgr.24625) 5786 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:42:19.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.672372+0000 mgr.gibba032.whopeo (mgr.24625) 5787 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:42:19.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.921721+0000 mon.a (mon.0) 2193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:42:19.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.925380+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:42:19.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.926941+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.928470+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.929667+0000 mon.a (mon.0) 2197 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.933461+0000 mon.a (mon.0) 2198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:19 gibba011 bash[31774]: audit 2021-03-10T15:42:18.936027+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.670764+0000 mgr.gibba032.whopeo (mgr.24625) 5786 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:42:19.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.672372+0000 mgr.gibba032.whopeo (mgr.24625) 5787 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:42:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.921721+0000 mon.a (mon.0) 2193 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:42:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.925380+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:42:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.926941+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.928470+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.929667+0000 mon.a (mon.0) 2197 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.933461+0000 mon.a (mon.0) 2198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:19.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:19 gibba032 bash[24004]: audit 2021-03-10T15:42:18.936027+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.923190+0000 mgr.gibba032.whopeo (mgr.24625) 5788 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:42:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.925419+0000 mgr.gibba032.whopeo (mgr.24625) 5789 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:42:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.926844+0000 mgr.gibba032.whopeo (mgr.24625) 5790 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:42:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.926880+0000 mgr.gibba032.whopeo (mgr.24625) 5791 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:42:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.928210+0000 mgr.gibba032.whopeo (mgr.24625) 5792 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:42:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.928247+0000 mgr.gibba032.whopeo (mgr.24625) 5793 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:42:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.929357+0000 mgr.gibba032.whopeo (mgr.24625) 5794 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:42:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.931537+0000 mgr.gibba032.whopeo (mgr.24625) 5795 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:42:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.931575+0000 mgr.gibba032.whopeo (mgr.24625) 5796 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:42:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.933994+0000 mgr.gibba032.whopeo (mgr.24625) 5797 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:42:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.934326+0000 mgr.gibba032.whopeo (mgr.24625) 5798 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:42:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.936620+0000 mgr.gibba032.whopeo (mgr.24625) 5799 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:42:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.936658+0000 mgr.gibba032.whopeo (mgr.24625) 5800 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:42:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cephadm 2021-03-10T15:42:18.936768+0000 mgr.gibba032.whopeo (mgr.24625) 5801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[30791]: cluster 2021-03-10T15:42:19.219545+0000 mgr.gibba032.whopeo (mgr.24625) 5802 : cluster [DBG] pgmap v3294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:20.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.923190+0000 mgr.gibba032.whopeo (mgr.24625) 5788 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:42:20.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.925419+0000 mgr.gibba032.whopeo (mgr.24625) 5789 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:42:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.926844+0000 mgr.gibba032.whopeo (mgr.24625) 5790 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:42:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.926880+0000 mgr.gibba032.whopeo (mgr.24625) 5791 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:42:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.928210+0000 mgr.gibba032.whopeo (mgr.24625) 5792 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:42:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.928247+0000 mgr.gibba032.whopeo (mgr.24625) 5793 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:42:20.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.929357+0000 mgr.gibba032.whopeo (mgr.24625) 5794 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:42:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.931537+0000 mgr.gibba032.whopeo (mgr.24625) 5795 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:42:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.931575+0000 mgr.gibba032.whopeo (mgr.24625) 5796 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:42:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.933994+0000 mgr.gibba032.whopeo (mgr.24625) 5797 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:42:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.934326+0000 mgr.gibba032.whopeo (mgr.24625) 5798 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:42:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.936620+0000 mgr.gibba032.whopeo (mgr.24625) 5799 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:42:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.936658+0000 mgr.gibba032.whopeo (mgr.24625) 5800 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:42:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cephadm 2021-03-10T15:42:18.936768+0000 mgr.gibba032.whopeo (mgr.24625) 5801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:20 gibba011 bash[31774]: cluster 2021-03-10T15:42:19.219545+0000 mgr.gibba032.whopeo (mgr.24625) 5802 : cluster [DBG] pgmap v3294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.923190+0000 mgr.gibba032.whopeo (mgr.24625) 5788 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:42:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.925419+0000 mgr.gibba032.whopeo (mgr.24625) 5789 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:42:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.926844+0000 mgr.gibba032.whopeo (mgr.24625) 5790 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:42:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.926880+0000 mgr.gibba032.whopeo (mgr.24625) 5791 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:42:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.928210+0000 mgr.gibba032.whopeo (mgr.24625) 5792 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:42:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.928247+0000 mgr.gibba032.whopeo (mgr.24625) 5793 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:42:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.929357+0000 mgr.gibba032.whopeo (mgr.24625) 5794 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:42:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.931537+0000 mgr.gibba032.whopeo (mgr.24625) 5795 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:42:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.931575+0000 mgr.gibba032.whopeo (mgr.24625) 5796 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:42:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.933994+0000 mgr.gibba032.whopeo (mgr.24625) 5797 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:42:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.934326+0000 mgr.gibba032.whopeo (mgr.24625) 5798 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:42:20.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.936620+0000 mgr.gibba032.whopeo (mgr.24625) 5799 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:42:20.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.936658+0000 mgr.gibba032.whopeo (mgr.24625) 5800 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:42:20.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cephadm 2021-03-10T15:42:18.936768+0000 mgr.gibba032.whopeo (mgr.24625) 5801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:20.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:20 gibba032 bash[24004]: cluster 2021-03-10T15:42:19.219545+0000 mgr.gibba032.whopeo (mgr.24625) 5802 : cluster [DBG] pgmap v3294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:22 gibba011 bash[30791]: cluster 2021-03-10T15:42:21.221179+0000 mgr.gibba032.whopeo (mgr.24625) 5803 : cluster [DBG] pgmap v3295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:22.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:22 gibba011 bash[31774]: cluster 2021-03-10T15:42:21.221179+0000 mgr.gibba032.whopeo (mgr.24625) 5803 : cluster [DBG] pgmap v3295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:22 gibba032 bash[24004]: cluster 2021-03-10T15:42:21.221179+0000 mgr.gibba032.whopeo (mgr.24625) 5803 : cluster [DBG] pgmap v3295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:24 gibba032 bash[24004]: cluster 2021-03-10T15:42:23.222905+0000 mgr.gibba032.whopeo (mgr.24625) 5804 : cluster [DBG] pgmap v3296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:24 gibba011 bash[30791]: cluster 2021-03-10T15:42:23.222905+0000 mgr.gibba032.whopeo (mgr.24625) 5804 : cluster [DBG] pgmap v3296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:24 gibba011 bash[31774]: cluster 2021-03-10T15:42:23.222905+0000 mgr.gibba032.whopeo (mgr.24625) 5804 : cluster [DBG] pgmap v3296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:26 gibba011 bash[30791]: cluster 2021-03-10T15:42:25.224327+0000 mgr.gibba032.whopeo (mgr.24625) 5805 : cluster [DBG] pgmap v3297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:26 gibba011 bash[31774]: cluster 2021-03-10T15:42:25.224327+0000 mgr.gibba032.whopeo (mgr.24625) 5805 : cluster [DBG] pgmap v3297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:26 gibba032 bash[24004]: cluster 2021-03-10T15:42:25.224327+0000 mgr.gibba032.whopeo (mgr.24625) 5805 : cluster [DBG] pgmap v3297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:28 gibba011 bash[31774]: cluster 2021-03-10T15:42:27.226020+0000 mgr.gibba032.whopeo (mgr.24625) 5806 : cluster [DBG] pgmap v3298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:28.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:28 gibba011 bash[30791]: cluster 2021-03-10T15:42:27.226020+0000 mgr.gibba032.whopeo (mgr.24625) 5806 : cluster [DBG] pgmap v3298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:28.635 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:42:28.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:28 gibba032 bash[24004]: cluster 2021-03-10T15:42:27.226020+0000 mgr.gibba032.whopeo (mgr.24625) 5806 : cluster [DBG] pgmap v3298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:28.864 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:42:28.864 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (109m) 3m ago 114m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:42:28.865 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (114m) 2m ago 114m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:42:28.865 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (109m) 2m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:42:28.865 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (109m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:42:28.866 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (108m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:42:28.866 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (108m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:42:28.866 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (108m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:42:28.866 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (108m) 3m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:42:28.867 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (114m) 3m ago 114m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:42:28.867 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (114m) 2m ago 114m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:42:28.867 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (107m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:42:28.867 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (106m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:42:28.868 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (106m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:42:28.868 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (106m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:42:28.868 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (105m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:42:28.868 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (105m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:42:28.869 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (104m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:42:28.869 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (104m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:42:28.869 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (114m) 2m ago 114m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:42:28.869 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 112m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:42:29.120 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:42:29.121 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:42:29.121 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:42:29.121 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:29.122 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:42:29.122 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:42:29.122 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:29.122 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:42:29.122 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:42:29.123 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:29.123 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:42:29.123 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:42:29.123 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:42:29.124 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:42:29.124 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:42:29.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:29 gibba011 bash[30791]: audit 2021-03-10T15:42:28.623062+0000 mgr.gibba032.whopeo (mgr.24625) 5807 : audit [DBG] from='client.47943 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:29.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:29 gibba011 bash[30791]: audit 2021-03-10T15:42:29.117710+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? 172.21.2.111:0/2372743735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:29.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:29 gibba011 bash[31774]: audit 2021-03-10T15:42:28.623062+0000 mgr.gibba032.whopeo (mgr.24625) 5807 : audit [DBG] from='client.47943 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:29.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:29 gibba011 bash[31774]: audit 2021-03-10T15:42:29.117710+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? 172.21.2.111:0/2372743735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:29.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:29 gibba032 bash[24004]: audit 2021-03-10T15:42:28.623062+0000 mgr.gibba032.whopeo (mgr.24625) 5807 : audit [DBG] from='client.47943 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:29.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:29 gibba032 bash[24004]: audit 2021-03-10T15:42:29.117710+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? 172.21.2.111:0/2372743735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:42:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:30 gibba011 bash[30791]: audit 2021-03-10T15:42:28.849750+0000 mgr.gibba032.whopeo (mgr.24625) 5808 : audit [DBG] from='client.57934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:30 gibba011 bash[30791]: cluster 2021-03-10T15:42:29.227502+0000 mgr.gibba032.whopeo (mgr.24625) 5809 : cluster [DBG] pgmap v3299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:30 gibba011 bash[31774]: audit 2021-03-10T15:42:28.849750+0000 mgr.gibba032.whopeo (mgr.24625) 5808 : audit [DBG] from='client.57934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:30 gibba011 bash[31774]: cluster 2021-03-10T15:42:29.227502+0000 mgr.gibba032.whopeo (mgr.24625) 5809 : cluster [DBG] pgmap v3299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:30 gibba032 bash[24004]: audit 2021-03-10T15:42:28.849750+0000 mgr.gibba032.whopeo (mgr.24625) 5808 : audit [DBG] from='client.57934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:42:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:30 gibba032 bash[24004]: cluster 2021-03-10T15:42:29.227502+0000 mgr.gibba032.whopeo (mgr.24625) 5809 : cluster [DBG] pgmap v3299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:32.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:32 gibba011 bash[30791]: cluster 2021-03-10T15:42:31.229053+0000 mgr.gibba032.whopeo (mgr.24625) 5810 : cluster [DBG] pgmap v3300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:32.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:32 gibba011 bash[31774]: cluster 2021-03-10T15:42:31.229053+0000 mgr.gibba032.whopeo (mgr.24625) 5810 : cluster [DBG] pgmap v3300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:32 gibba032 bash[24004]: cluster 2021-03-10T15:42:31.229053+0000 mgr.gibba032.whopeo (mgr.24625) 5810 : cluster [DBG] pgmap v3300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:34 gibba032 bash[24004]: cluster 2021-03-10T15:42:33.230765+0000 mgr.gibba032.whopeo (mgr.24625) 5811 : cluster [DBG] pgmap v3301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:34 gibba011 bash[30791]: cluster 2021-03-10T15:42:33.230765+0000 mgr.gibba032.whopeo (mgr.24625) 5811 : cluster [DBG] pgmap v3301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:34 gibba011 bash[31774]: cluster 2021-03-10T15:42:33.230765+0000 mgr.gibba032.whopeo (mgr.24625) 5811 : cluster [DBG] pgmap v3301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:35 gibba032 bash[24004]: cephadm 2021-03-10T15:42:33.952586+0000 mgr.gibba032.whopeo (mgr.24625) 5812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:35.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:35 gibba011 bash[30791]: cephadm 2021-03-10T15:42:33.952586+0000 mgr.gibba032.whopeo (mgr.24625) 5812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:35 gibba011 bash[31774]: cephadm 2021-03-10T15:42:33.952586+0000 mgr.gibba032.whopeo (mgr.24625) 5812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:36.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:36 gibba032 bash[24004]: cluster 2021-03-10T15:42:35.232426+0000 mgr.gibba032.whopeo (mgr.24625) 5813 : cluster [DBG] pgmap v3302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:36.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:36 gibba011 bash[30791]: cluster 2021-03-10T15:42:35.232426+0000 mgr.gibba032.whopeo (mgr.24625) 5813 : cluster [DBG] pgmap v3302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:36 gibba011 bash[31774]: cluster 2021-03-10T15:42:35.232426+0000 mgr.gibba032.whopeo (mgr.24625) 5813 : cluster [DBG] pgmap v3302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:38 gibba011 bash[30791]: cluster 2021-03-10T15:42:37.234052+0000 mgr.gibba032.whopeo (mgr.24625) 5814 : cluster [DBG] pgmap v3303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:38.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:38 gibba011 bash[31774]: cluster 2021-03-10T15:42:37.234052+0000 mgr.gibba032.whopeo (mgr.24625) 5814 : cluster [DBG] pgmap v3303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:38 gibba032 bash[24004]: cluster 2021-03-10T15:42:37.234052+0000 mgr.gibba032.whopeo (mgr.24625) 5814 : cluster [DBG] pgmap v3303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:40 gibba011 bash[30791]: cluster 2021-03-10T15:42:39.235337+0000 mgr.gibba032.whopeo (mgr.24625) 5815 : cluster [DBG] pgmap v3304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:40.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:40 gibba011 bash[31774]: cluster 2021-03-10T15:42:39.235337+0000 mgr.gibba032.whopeo (mgr.24625) 5815 : cluster [DBG] pgmap v3304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:40 gibba032 bash[24004]: cluster 2021-03-10T15:42:39.235337+0000 mgr.gibba032.whopeo (mgr.24625) 5815 : cluster [DBG] pgmap v3304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:42 gibba011 bash[30791]: cluster 2021-03-10T15:42:41.236857+0000 mgr.gibba032.whopeo (mgr.24625) 5816 : cluster [DBG] pgmap v3305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:42 gibba011 bash[31774]: cluster 2021-03-10T15:42:41.236857+0000 mgr.gibba032.whopeo (mgr.24625) 5816 : cluster [DBG] pgmap v3305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:42 gibba032 bash[24004]: cluster 2021-03-10T15:42:41.236857+0000 mgr.gibba032.whopeo (mgr.24625) 5816 : cluster [DBG] pgmap v3305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:44.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:44 gibba032 bash[24004]: cluster 2021-03-10T15:42:43.238447+0000 mgr.gibba032.whopeo (mgr.24625) 5817 : cluster [DBG] pgmap v3306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:44.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:44 gibba011 bash[30791]: cluster 2021-03-10T15:42:43.238447+0000 mgr.gibba032.whopeo (mgr.24625) 5817 : cluster [DBG] pgmap v3306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:44 gibba011 bash[31774]: cluster 2021-03-10T15:42:43.238447+0000 mgr.gibba032.whopeo (mgr.24625) 5817 : cluster [DBG] pgmap v3306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:46.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:46 gibba011 bash[30791]: cluster 2021-03-10T15:42:45.239748+0000 mgr.gibba032.whopeo (mgr.24625) 5818 : cluster [DBG] pgmap v3307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:46.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:46 gibba011 bash[31774]: cluster 2021-03-10T15:42:45.239748+0000 mgr.gibba032.whopeo (mgr.24625) 5818 : cluster [DBG] pgmap v3307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:46 gibba032 bash[24004]: cluster 2021-03-10T15:42:45.239748+0000 mgr.gibba032.whopeo (mgr.24625) 5818 : cluster [DBG] pgmap v3307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:48.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:48 gibba011 bash[30791]: cluster 2021-03-10T15:42:47.241422+0000 mgr.gibba032.whopeo (mgr.24625) 5819 : cluster [DBG] pgmap v3308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:48.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:48 gibba011 bash[31774]: cluster 2021-03-10T15:42:47.241422+0000 mgr.gibba032.whopeo (mgr.24625) 5819 : cluster [DBG] pgmap v3308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:48 gibba032 bash[24004]: cluster 2021-03-10T15:42:47.241422+0000 mgr.gibba032.whopeo (mgr.24625) 5819 : cluster [DBG] pgmap v3308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:50 gibba011 bash[30791]: cephadm 2021-03-10T15:42:48.963338+0000 mgr.gibba032.whopeo (mgr.24625) 5820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:50 gibba011 bash[30791]: cluster 2021-03-10T15:42:49.242766+0000 mgr.gibba032.whopeo (mgr.24625) 5821 : cluster [DBG] pgmap v3309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:50 gibba011 bash[31774]: cephadm 2021-03-10T15:42:48.963338+0000 mgr.gibba032.whopeo (mgr.24625) 5820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:50 gibba011 bash[31774]: cluster 2021-03-10T15:42:49.242766+0000 mgr.gibba032.whopeo (mgr.24625) 5821 : cluster [DBG] pgmap v3309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:50 gibba032 bash[24004]: cephadm 2021-03-10T15:42:48.963338+0000 mgr.gibba032.whopeo (mgr.24625) 5820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:42:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:50 gibba032 bash[24004]: cluster 2021-03-10T15:42:49.242766+0000 mgr.gibba032.whopeo (mgr.24625) 5821 : cluster [DBG] pgmap v3309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:52.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:52 gibba032 bash[24004]: cluster 2021-03-10T15:42:51.244317+0000 mgr.gibba032.whopeo (mgr.24625) 5822 : cluster [DBG] pgmap v3310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:52.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:52 gibba011 bash[30791]: cluster 2021-03-10T15:42:51.244317+0000 mgr.gibba032.whopeo (mgr.24625) 5822 : cluster [DBG] pgmap v3310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:52 gibba011 bash[31774]: cluster 2021-03-10T15:42:51.244317+0000 mgr.gibba032.whopeo (mgr.24625) 5822 : cluster [DBG] pgmap v3310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:54 gibba032 bash[24004]: cluster 2021-03-10T15:42:53.246065+0000 mgr.gibba032.whopeo (mgr.24625) 5823 : cluster [DBG] pgmap v3311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:54 gibba011 bash[30791]: cluster 2021-03-10T15:42:53.246065+0000 mgr.gibba032.whopeo (mgr.24625) 5823 : cluster [DBG] pgmap v3311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:54 gibba011 bash[31774]: cluster 2021-03-10T15:42:53.246065+0000 mgr.gibba032.whopeo (mgr.24625) 5823 : cluster [DBG] pgmap v3311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:56.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:56 gibba011 bash[30791]: cluster 2021-03-10T15:42:55.247461+0000 mgr.gibba032.whopeo (mgr.24625) 5824 : cluster [DBG] pgmap v3312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:56.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:56 gibba011 bash[31774]: cluster 2021-03-10T15:42:55.247461+0000 mgr.gibba032.whopeo (mgr.24625) 5824 : cluster [DBG] pgmap v3312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:56 gibba032 bash[24004]: cluster 2021-03-10T15:42:55.247461+0000 mgr.gibba032.whopeo (mgr.24625) 5824 : cluster [DBG] pgmap v3312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.683824+0000 mon.a (mon.0) 2201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:42:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.688951+0000 mon.a (mon.0) 2202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:42:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.710444+0000 mon.a (mon.0) 2203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:42:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.711280+0000 mon.a (mon.0) 2204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.712108+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[30791]: audit 2021-03-10T15:42:56.712865+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:42:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.683824+0000 mon.a (mon.0) 2201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:42:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.688951+0000 mon.a (mon.0) 2202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:42:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.710444+0000 mon.a (mon.0) 2203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:42:57.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.711280+0000 mon.a (mon.0) 2204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.712108+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:57 gibba011 bash[31774]: audit 2021-03-10T15:42:56.712865+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:42:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.683824+0000 mon.a (mon.0) 2201 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:42:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.688951+0000 mon.a (mon.0) 2202 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:42:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.710444+0000 mon.a (mon.0) 2203 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T15:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.711280+0000 mon.a (mon.0) 2204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.712108+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:42:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:57 gibba032 bash[24004]: audit 2021-03-10T15:42:56.712865+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:42:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:42:58 gibba032 bash[24004]: cluster 2021-03-10T15:42:57.249199+0000 mgr.gibba032.whopeo (mgr.24625) 5825 : cluster [DBG] pgmap v3313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:42:58 gibba011 bash[30791]: cluster 2021-03-10T15:42:57.249199+0000 mgr.gibba032.whopeo (mgr.24625) 5825 : cluster [DBG] pgmap v3313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:42:58 gibba011 bash[31774]: cluster 2021-03-10T15:42:57.249199+0000 mgr.gibba032.whopeo (mgr.24625) 5825 : cluster [DBG] pgmap v3313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:42:59.362 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:42:59.591 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:42:59.591 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (109m) 3m ago 115m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:42:59.591 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (114m) 2m ago 114m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:42:59.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (109m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:42:59.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (110m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:42:59.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (109m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:42:59.592 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (109m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:42:59.593 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (108m) 2m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:42:59.593 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (108m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:42:59.593 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (115m) 3m ago 115m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:42:59.593 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (115m) 2m ago 115m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:42:59.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (107m) 3m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:42:59.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (107m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:42:59.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (107m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:42:59.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (106m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:42:59.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (106m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:42:59.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (105m) 2m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:42:59.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (105m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:42:59.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (104m) 2m ago 115m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:42:59.596 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (114m) 2m ago 115m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:42:59.596 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 113m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:42:59.842 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:42:59.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:42:59.842 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:42:59.842 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:59.843 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:42:59.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:42:59.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:59.843 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:42:59.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:42:59.844 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:42:59.844 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:42:59.844 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:42:59.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:42:59.845 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:42:59.845 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:43:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:00 gibba032 bash[24004]: cluster 2021-03-10T15:42:59.250569+0000 mgr.gibba032.whopeo (mgr.24625) 5826 : cluster [DBG] pgmap v3314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:00 gibba032 bash[24004]: audit 2021-03-10T15:42:59.350126+0000 mgr.gibba032.whopeo (mgr.24625) 5827 : audit [DBG] from='client.57946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:00 gibba032 bash[24004]: audit 2021-03-10T15:42:59.574829+0000 mgr.gibba032.whopeo (mgr.24625) 5828 : audit [DBG] from='client.57952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:00 gibba032 bash[24004]: audit 2021-03-10T15:42:59.838905+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.2.111:0/3826027414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[30791]: cluster 2021-03-10T15:42:59.250569+0000 mgr.gibba032.whopeo (mgr.24625) 5826 : cluster [DBG] pgmap v3314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[30791]: audit 2021-03-10T15:42:59.350126+0000 mgr.gibba032.whopeo (mgr.24625) 5827 : audit [DBG] from='client.57946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[30791]: audit 2021-03-10T15:42:59.574829+0000 mgr.gibba032.whopeo (mgr.24625) 5828 : audit [DBG] from='client.57952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[30791]: audit 2021-03-10T15:42:59.838905+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.2.111:0/3826027414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[31774]: cluster 2021-03-10T15:42:59.250569+0000 mgr.gibba032.whopeo (mgr.24625) 5826 : cluster [DBG] pgmap v3314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[31774]: audit 2021-03-10T15:42:59.350126+0000 mgr.gibba032.whopeo (mgr.24625) 5827 : audit [DBG] from='client.57946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[31774]: audit 2021-03-10T15:42:59.574829+0000 mgr.gibba032.whopeo (mgr.24625) 5828 : audit [DBG] from='client.57952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:00.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:00 gibba011 bash[31774]: audit 2021-03-10T15:42:59.838905+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.2.111:0/3826027414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:02 gibba032 bash[24004]: cluster 2021-03-10T15:43:01.252103+0000 mgr.gibba032.whopeo (mgr.24625) 5829 : cluster [DBG] pgmap v3315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:02 gibba011 bash[30791]: cluster 2021-03-10T15:43:01.252103+0000 mgr.gibba032.whopeo (mgr.24625) 5829 : cluster [DBG] pgmap v3315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:02 gibba011 bash[31774]: cluster 2021-03-10T15:43:01.252103+0000 mgr.gibba032.whopeo (mgr.24625) 5829 : cluster [DBG] pgmap v3315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:04 gibba032 bash[24004]: cluster 2021-03-10T15:43:03.253550+0000 mgr.gibba032.whopeo (mgr.24625) 5830 : cluster [DBG] pgmap v3316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:04.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:04 gibba011 bash[30791]: cluster 2021-03-10T15:43:03.253550+0000 mgr.gibba032.whopeo (mgr.24625) 5830 : cluster [DBG] pgmap v3316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:04 gibba011 bash[31774]: cluster 2021-03-10T15:43:03.253550+0000 mgr.gibba032.whopeo (mgr.24625) 5830 : cluster [DBG] pgmap v3316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:05.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:05 gibba032 bash[24004]: cephadm 2021-03-10T15:43:03.979121+0000 mgr.gibba032.whopeo (mgr.24625) 5831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:05.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:05 gibba011 bash[30791]: cephadm 2021-03-10T15:43:03.979121+0000 mgr.gibba032.whopeo (mgr.24625) 5831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:05.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:05 gibba011 bash[31774]: cephadm 2021-03-10T15:43:03.979121+0000 mgr.gibba032.whopeo (mgr.24625) 5831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:06.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:06 gibba032 bash[24004]: cluster 2021-03-10T15:43:05.255148+0000 mgr.gibba032.whopeo (mgr.24625) 5832 : cluster [DBG] pgmap v3317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:06.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:06 gibba011 bash[30791]: cluster 2021-03-10T15:43:05.255148+0000 mgr.gibba032.whopeo (mgr.24625) 5832 : cluster [DBG] pgmap v3317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:06 gibba011 bash[31774]: cluster 2021-03-10T15:43:05.255148+0000 mgr.gibba032.whopeo (mgr.24625) 5832 : cluster [DBG] pgmap v3317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:08.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:08 gibba011 bash[30791]: cluster 2021-03-10T15:43:07.257104+0000 mgr.gibba032.whopeo (mgr.24625) 5833 : cluster [DBG] pgmap v3318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:08.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:08 gibba011 bash[31774]: cluster 2021-03-10T15:43:07.257104+0000 mgr.gibba032.whopeo (mgr.24625) 5833 : cluster [DBG] pgmap v3318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:08 gibba032 bash[24004]: cluster 2021-03-10T15:43:07.257104+0000 mgr.gibba032.whopeo (mgr.24625) 5833 : cluster [DBG] pgmap v3318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:10.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:10 gibba011 bash[30791]: cluster 2021-03-10T15:43:09.258314+0000 mgr.gibba032.whopeo (mgr.24625) 5834 : cluster [DBG] pgmap v3319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:10.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:10 gibba011 bash[31774]: cluster 2021-03-10T15:43:09.258314+0000 mgr.gibba032.whopeo (mgr.24625) 5834 : cluster [DBG] pgmap v3319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:10 gibba032 bash[24004]: cluster 2021-03-10T15:43:09.258314+0000 mgr.gibba032.whopeo (mgr.24625) 5834 : cluster [DBG] pgmap v3319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:12 gibba032 bash[24004]: cluster 2021-03-10T15:43:11.260067+0000 mgr.gibba032.whopeo (mgr.24625) 5835 : cluster [DBG] pgmap v3320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:12 gibba011 bash[30791]: cluster 2021-03-10T15:43:11.260067+0000 mgr.gibba032.whopeo (mgr.24625) 5835 : cluster [DBG] pgmap v3320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:12 gibba011 bash[31774]: cluster 2021-03-10T15:43:11.260067+0000 mgr.gibba032.whopeo (mgr.24625) 5835 : cluster [DBG] pgmap v3320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:14 gibba032 bash[24004]: cluster 2021-03-10T15:43:13.261822+0000 mgr.gibba032.whopeo (mgr.24625) 5836 : cluster [DBG] pgmap v3321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:14.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:14 gibba011 bash[30791]: cluster 2021-03-10T15:43:13.261822+0000 mgr.gibba032.whopeo (mgr.24625) 5836 : cluster [DBG] pgmap v3321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:14 gibba011 bash[31774]: cluster 2021-03-10T15:43:13.261822+0000 mgr.gibba032.whopeo (mgr.24625) 5836 : cluster [DBG] pgmap v3321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:16 gibba011 bash[30791]: cluster 2021-03-10T15:43:15.263406+0000 mgr.gibba032.whopeo (mgr.24625) 5837 : cluster [DBG] pgmap v3322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:16 gibba011 bash[31774]: cluster 2021-03-10T15:43:15.263406+0000 mgr.gibba032.whopeo (mgr.24625) 5837 : cluster [DBG] pgmap v3322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:16 gibba032 bash[24004]: cluster 2021-03-10T15:43:15.263406+0000 mgr.gibba032.whopeo (mgr.24625) 5837 : cluster [DBG] pgmap v3322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:18 gibba032 bash[24004]: cluster 2021-03-10T15:43:17.265254+0000 mgr.gibba032.whopeo (mgr.24625) 5838 : cluster [DBG] pgmap v3323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:18.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:18 gibba011 bash[30791]: cluster 2021-03-10T15:43:17.265254+0000 mgr.gibba032.whopeo (mgr.24625) 5838 : cluster [DBG] pgmap v3323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:18 gibba011 bash[31774]: cluster 2021-03-10T15:43:17.265254+0000 mgr.gibba032.whopeo (mgr.24625) 5838 : cluster [DBG] pgmap v3323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:18.993485+0000 mgr.gibba032.whopeo (mgr.24625) 5839 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:43:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:18.995095+0000 mgr.gibba032.whopeo (mgr.24625) 5840 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:43:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.219444+0000 mon.a (mon.0) 2208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:43:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.220721+0000 mgr.gibba032.whopeo (mgr.24625) 5841 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:43:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.222837+0000 mgr.gibba032.whopeo (mgr.24625) 5842 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:43:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.222990+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:43:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.224065+0000 mgr.gibba032.whopeo (mgr.24625) 5843 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:43:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.224108+0000 mgr.gibba032.whopeo (mgr.24625) 5844 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:43:20.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.224452+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.225441+0000 mgr.gibba032.whopeo (mgr.24625) 5845 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:43:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.225483+0000 mgr.gibba032.whopeo (mgr.24625) 5846 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:43:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.225764+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.226572+0000 mgr.gibba032.whopeo (mgr.24625) 5847 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:43:20.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.226908+0000 mon.a (mon.0) 2212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.227560+0000 mgr.gibba032.whopeo (mgr.24625) 5848 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:43:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.227601+0000 mgr.gibba032.whopeo (mgr.24625) 5849 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:43:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.228730+0000 mgr.gibba032.whopeo (mgr.24625) 5850 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:43:20.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.229093+0000 mgr.gibba032.whopeo (mgr.24625) 5851 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:43:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.229143+0000 mon.a (mon.0) 2213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.230262+0000 mgr.gibba032.whopeo (mgr.24625) 5852 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:43:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.230307+0000 mgr.gibba032.whopeo (mgr.24625) 5853 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:43:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cephadm 2021-03-10T15:43:19.230425+0000 mgr.gibba032.whopeo (mgr.24625) 5854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:20.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: audit 2021-03-10T15:43:19.230614+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:20 gibba032 bash[24004]: cluster 2021-03-10T15:43:19.266346+0000 mgr.gibba032.whopeo (mgr.24625) 5855 : cluster [DBG] pgmap v3324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:18.993485+0000 mgr.gibba032.whopeo (mgr.24625) 5839 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:43:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:18.995095+0000 mgr.gibba032.whopeo (mgr.24625) 5840 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:43:20.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.219444+0000 mon.a (mon.0) 2208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:43:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.220721+0000 mgr.gibba032.whopeo (mgr.24625) 5841 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:43:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.222837+0000 mgr.gibba032.whopeo (mgr.24625) 5842 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:43:20.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.222990+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:43:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.224065+0000 mgr.gibba032.whopeo (mgr.24625) 5843 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:43:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.224108+0000 mgr.gibba032.whopeo (mgr.24625) 5844 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:43:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.224452+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.225441+0000 mgr.gibba032.whopeo (mgr.24625) 5845 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:43:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.225483+0000 mgr.gibba032.whopeo (mgr.24625) 5846 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:43:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.225764+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.226572+0000 mgr.gibba032.whopeo (mgr.24625) 5847 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:43:20.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.226908+0000 mon.a (mon.0) 2212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.227560+0000 mgr.gibba032.whopeo (mgr.24625) 5848 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:43:20.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.227601+0000 mgr.gibba032.whopeo (mgr.24625) 5849 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:43:20.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.228730+0000 mgr.gibba032.whopeo (mgr.24625) 5850 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:43:20.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.229093+0000 mgr.gibba032.whopeo (mgr.24625) 5851 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:43:20.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.229143+0000 mon.a (mon.0) 2213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.230262+0000 mgr.gibba032.whopeo (mgr.24625) 5852 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:43:20.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.230307+0000 mgr.gibba032.whopeo (mgr.24625) 5853 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:43:20.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:18.993485+0000 mgr.gibba032.whopeo (mgr.24625) 5839 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:43:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:18.995095+0000 mgr.gibba032.whopeo (mgr.24625) 5840 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:43:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.219444+0000 mon.a (mon.0) 2208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:43:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.220721+0000 mgr.gibba032.whopeo (mgr.24625) 5841 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:43:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.222837+0000 mgr.gibba032.whopeo (mgr.24625) 5842 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:43:20.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.222990+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:43:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.224065+0000 mgr.gibba032.whopeo (mgr.24625) 5843 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:43:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.224108+0000 mgr.gibba032.whopeo (mgr.24625) 5844 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:43:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.224452+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.225441+0000 mgr.gibba032.whopeo (mgr.24625) 5845 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:43:20.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.225483+0000 mgr.gibba032.whopeo (mgr.24625) 5846 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:43:20.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.225764+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.226572+0000 mgr.gibba032.whopeo (mgr.24625) 5847 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:43:20.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.226908+0000 mon.a (mon.0) 2212 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.227560+0000 mgr.gibba032.whopeo (mgr.24625) 5848 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:43:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.227601+0000 mgr.gibba032.whopeo (mgr.24625) 5849 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:43:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.228730+0000 mgr.gibba032.whopeo (mgr.24625) 5850 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:43:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.229093+0000 mgr.gibba032.whopeo (mgr.24625) 5851 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:43:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.229143+0000 mon.a (mon.0) 2213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.230262+0000 mgr.gibba032.whopeo (mgr.24625) 5852 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:43:20.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.230307+0000 mgr.gibba032.whopeo (mgr.24625) 5853 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:43:20.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cephadm 2021-03-10T15:43:19.230425+0000 mgr.gibba032.whopeo (mgr.24625) 5854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:20.593 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: audit 2021-03-10T15:43:19.230614+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[30791]: cluster 2021-03-10T15:43:19.266346+0000 mgr.gibba032.whopeo (mgr.24625) 5855 : cluster [DBG] pgmap v3324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:20.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cephadm 2021-03-10T15:43:19.230425+0000 mgr.gibba032.whopeo (mgr.24625) 5854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:20.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: audit 2021-03-10T15:43:19.230614+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:20.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:20 gibba011 bash[31774]: cluster 2021-03-10T15:43:19.266346+0000 mgr.gibba032.whopeo (mgr.24625) 5855 : cluster [DBG] pgmap v3324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:22 gibba032 bash[24004]: cluster 2021-03-10T15:43:21.268210+0000 mgr.gibba032.whopeo (mgr.24625) 5856 : cluster [DBG] pgmap v3325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:22 gibba011 bash[30791]: cluster 2021-03-10T15:43:21.268210+0000 mgr.gibba032.whopeo (mgr.24625) 5856 : cluster [DBG] pgmap v3325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:22.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:22 gibba011 bash[31774]: cluster 2021-03-10T15:43:21.268210+0000 mgr.gibba032.whopeo (mgr.24625) 5856 : cluster [DBG] pgmap v3325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:24 gibba032 bash[24004]: cluster 2021-03-10T15:43:23.269697+0000 mgr.gibba032.whopeo (mgr.24625) 5857 : cluster [DBG] pgmap v3326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:24.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:24 gibba011 bash[30791]: cluster 2021-03-10T15:43:23.269697+0000 mgr.gibba032.whopeo (mgr.24625) 5857 : cluster [DBG] pgmap v3326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:24.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:24 gibba011 bash[31774]: cluster 2021-03-10T15:43:23.269697+0000 mgr.gibba032.whopeo (mgr.24625) 5857 : cluster [DBG] pgmap v3326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:26.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:26 gibba032 bash[24004]: cluster 2021-03-10T15:43:25.271431+0000 mgr.gibba032.whopeo (mgr.24625) 5858 : cluster [DBG] pgmap v3327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:26.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:26 gibba011 bash[30791]: cluster 2021-03-10T15:43:25.271431+0000 mgr.gibba032.whopeo (mgr.24625) 5858 : cluster [DBG] pgmap v3327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:26 gibba011 bash[31774]: cluster 2021-03-10T15:43:25.271431+0000 mgr.gibba032.whopeo (mgr.24625) 5858 : cluster [DBG] pgmap v3327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:28 gibba032 bash[24004]: cluster 2021-03-10T15:43:27.273264+0000 mgr.gibba032.whopeo (mgr.24625) 5859 : cluster [DBG] pgmap v3328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:28.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:28 gibba011 bash[30791]: cluster 2021-03-10T15:43:27.273264+0000 mgr.gibba032.whopeo (mgr.24625) 5859 : cluster [DBG] pgmap v3328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:28 gibba011 bash[31774]: cluster 2021-03-10T15:43:27.273264+0000 mgr.gibba032.whopeo (mgr.24625) 5859 : cluster [DBG] pgmap v3328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:30.083 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:43:30.310 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:43:30.310 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (110m) 4m ago 115m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:43:30.311 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (115m) 3m ago 115m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:43:30.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (110m) 3m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:43:30.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (110m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:43:30.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (109m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:43:30.312 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (109m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:43:30.312 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (109m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:43:30.312 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (109m) 4m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:43:30.312 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (115m) 4m ago 115m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:43:30.313 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (115m) 3m ago 115m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:43:30.313 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (108m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:43:30.313 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (108m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:43:30.313 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (107m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:43:30.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (107m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:43:30.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (106m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:43:30.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (106m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:43:30.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (105m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:43:30.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (105m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:43:30.315 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (115m) 3m ago 115m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:43:30.315 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 113m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:43:30.565 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:43:30.565 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:43:30.566 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:43:30.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:43:30.566 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:43:30.566 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:43:30.567 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:43:30.567 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:43:30.567 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:43:30.567 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:43:30.568 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:43:30.568 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:43:30.568 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:43:30.568 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:43:30.569 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:43:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:30 gibba032 bash[24004]: cluster 2021-03-10T15:43:29.274689+0000 mgr.gibba032.whopeo (mgr.24625) 5860 : cluster [DBG] pgmap v3329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:30 gibba011 bash[30791]: cluster 2021-03-10T15:43:29.274689+0000 mgr.gibba032.whopeo (mgr.24625) 5860 : cluster [DBG] pgmap v3329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:30 gibba011 bash[31774]: cluster 2021-03-10T15:43:29.274689+0000 mgr.gibba032.whopeo (mgr.24625) 5860 : cluster [DBG] pgmap v3329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:31.359 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:31 gibba032 bash[24004]: audit 2021-03-10T15:43:30.071289+0000 mgr.gibba032.whopeo (mgr.24625) 5861 : audit [DBG] from='client.57964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.359 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:31 gibba032 bash[24004]: audit 2021-03-10T15:43:30.294139+0000 mgr.gibba032.whopeo (mgr.24625) 5862 : audit [DBG] from='client.57970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[30791]: audit 2021-03-10T15:43:30.071289+0000 mgr.gibba032.whopeo (mgr.24625) 5861 : audit [DBG] from='client.57964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[30791]: audit 2021-03-10T15:43:30.294139+0000 mgr.gibba032.whopeo (mgr.24625) 5862 : audit [DBG] from='client.57970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[30791]: audit 2021-03-10T15:43:30.562694+0000 mon.a (mon.0) 2215 : audit [DBG] from='client.? 172.21.2.111:0/2975551350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:31.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[31774]: audit 2021-03-10T15:43:30.071289+0000 mgr.gibba032.whopeo (mgr.24625) 5861 : audit [DBG] from='client.57964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[31774]: audit 2021-03-10T15:43:30.294139+0000 mgr.gibba032.whopeo (mgr.24625) 5862 : audit [DBG] from='client.57970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:43:31.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:31 gibba011 bash[31774]: audit 2021-03-10T15:43:30.562694+0000 mon.a (mon.0) 2215 : audit [DBG] from='client.? 172.21.2.111:0/2975551350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:31 gibba032 bash[24004]: audit 2021-03-10T15:43:30.562694+0000 mon.a (mon.0) 2215 : audit [DBG] from='client.? 172.21.2.111:0/2975551350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:43:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:32 gibba032 bash[24004]: cluster 2021-03-10T15:43:31.276401+0000 mgr.gibba032.whopeo (mgr.24625) 5863 : cluster [DBG] pgmap v3330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:32 gibba011 bash[30791]: cluster 2021-03-10T15:43:31.276401+0000 mgr.gibba032.whopeo (mgr.24625) 5863 : cluster [DBG] pgmap v3330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:32 gibba011 bash[31774]: cluster 2021-03-10T15:43:31.276401+0000 mgr.gibba032.whopeo (mgr.24625) 5863 : cluster [DBG] pgmap v3330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:34.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:34 gibba032 bash[24004]: cluster 2021-03-10T15:43:33.277849+0000 mgr.gibba032.whopeo (mgr.24625) 5864 : cluster [DBG] pgmap v3331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:34 gibba011 bash[30791]: cluster 2021-03-10T15:43:33.277849+0000 mgr.gibba032.whopeo (mgr.24625) 5864 : cluster [DBG] pgmap v3331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:34 gibba011 bash[31774]: cluster 2021-03-10T15:43:33.277849+0000 mgr.gibba032.whopeo (mgr.24625) 5864 : cluster [DBG] pgmap v3331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:35.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:35 gibba032 bash[24004]: cephadm 2021-03-10T15:43:34.246315+0000 mgr.gibba032.whopeo (mgr.24625) 5865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:35 gibba011 bash[30791]: cephadm 2021-03-10T15:43:34.246315+0000 mgr.gibba032.whopeo (mgr.24625) 5865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:35 gibba011 bash[31774]: cephadm 2021-03-10T15:43:34.246315+0000 mgr.gibba032.whopeo (mgr.24625) 5865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:36 gibba032 bash[24004]: cluster 2021-03-10T15:43:35.279329+0000 mgr.gibba032.whopeo (mgr.24625) 5866 : cluster [DBG] pgmap v3332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:36 gibba011 bash[30791]: cluster 2021-03-10T15:43:35.279329+0000 mgr.gibba032.whopeo (mgr.24625) 5866 : cluster [DBG] pgmap v3332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:36.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:36 gibba011 bash[31774]: cluster 2021-03-10T15:43:35.279329+0000 mgr.gibba032.whopeo (mgr.24625) 5866 : cluster [DBG] pgmap v3332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:38 gibba032 bash[24004]: cluster 2021-03-10T15:43:37.281156+0000 mgr.gibba032.whopeo (mgr.24625) 5867 : cluster [DBG] pgmap v3333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:38 gibba011 bash[30791]: cluster 2021-03-10T15:43:37.281156+0000 mgr.gibba032.whopeo (mgr.24625) 5867 : cluster [DBG] pgmap v3333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:38 gibba011 bash[31774]: cluster 2021-03-10T15:43:37.281156+0000 mgr.gibba032.whopeo (mgr.24625) 5867 : cluster [DBG] pgmap v3333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:40 gibba032 bash[24004]: cluster 2021-03-10T15:43:39.282511+0000 mgr.gibba032.whopeo (mgr.24625) 5868 : cluster [DBG] pgmap v3334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:40 gibba011 bash[30791]: cluster 2021-03-10T15:43:39.282511+0000 mgr.gibba032.whopeo (mgr.24625) 5868 : cluster [DBG] pgmap v3334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:40 gibba011 bash[31774]: cluster 2021-03-10T15:43:39.282511+0000 mgr.gibba032.whopeo (mgr.24625) 5868 : cluster [DBG] pgmap v3334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:42 gibba032 bash[24004]: cluster 2021-03-10T15:43:41.284306+0000 mgr.gibba032.whopeo (mgr.24625) 5869 : cluster [DBG] pgmap v3335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:42 gibba011 bash[30791]: cluster 2021-03-10T15:43:41.284306+0000 mgr.gibba032.whopeo (mgr.24625) 5869 : cluster [DBG] pgmap v3335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:42 gibba011 bash[31774]: cluster 2021-03-10T15:43:41.284306+0000 mgr.gibba032.whopeo (mgr.24625) 5869 : cluster [DBG] pgmap v3335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:44 gibba032 bash[24004]: cluster 2021-03-10T15:43:43.286024+0000 mgr.gibba032.whopeo (mgr.24625) 5870 : cluster [DBG] pgmap v3336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:44 gibba011 bash[30791]: cluster 2021-03-10T15:43:43.286024+0000 mgr.gibba032.whopeo (mgr.24625) 5870 : cluster [DBG] pgmap v3336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:44 gibba011 bash[31774]: cluster 2021-03-10T15:43:43.286024+0000 mgr.gibba032.whopeo (mgr.24625) 5870 : cluster [DBG] pgmap v3336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:46 gibba032 bash[24004]: cluster 2021-03-10T15:43:45.287299+0000 mgr.gibba032.whopeo (mgr.24625) 5871 : cluster [DBG] pgmap v3337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:46 gibba011 bash[30791]: cluster 2021-03-10T15:43:45.287299+0000 mgr.gibba032.whopeo (mgr.24625) 5871 : cluster [DBG] pgmap v3337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:46.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:46 gibba011 bash[31774]: cluster 2021-03-10T15:43:45.287299+0000 mgr.gibba032.whopeo (mgr.24625) 5871 : cluster [DBG] pgmap v3337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:48 gibba032 bash[24004]: cluster 2021-03-10T15:43:47.289103+0000 mgr.gibba032.whopeo (mgr.24625) 5872 : cluster [DBG] pgmap v3338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:48 gibba011 bash[30791]: cluster 2021-03-10T15:43:47.289103+0000 mgr.gibba032.whopeo (mgr.24625) 5872 : cluster [DBG] pgmap v3338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:48 gibba011 bash[31774]: cluster 2021-03-10T15:43:47.289103+0000 mgr.gibba032.whopeo (mgr.24625) 5872 : cluster [DBG] pgmap v3338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:50 gibba032 bash[24004]: cephadm 2021-03-10T15:43:49.257587+0000 mgr.gibba032.whopeo (mgr.24625) 5873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:50.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:50 gibba032 bash[24004]: cluster 2021-03-10T15:43:49.290408+0000 mgr.gibba032.whopeo (mgr.24625) 5874 : cluster [DBG] pgmap v3339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:50 gibba011 bash[30791]: cephadm 2021-03-10T15:43:49.257587+0000 mgr.gibba032.whopeo (mgr.24625) 5873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:50.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:50 gibba011 bash[30791]: cluster 2021-03-10T15:43:49.290408+0000 mgr.gibba032.whopeo (mgr.24625) 5874 : cluster [DBG] pgmap v3339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:50 gibba011 bash[31774]: cephadm 2021-03-10T15:43:49.257587+0000 mgr.gibba032.whopeo (mgr.24625) 5873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:43:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:50 gibba011 bash[31774]: cluster 2021-03-10T15:43:49.290408+0000 mgr.gibba032.whopeo (mgr.24625) 5874 : cluster [DBG] pgmap v3339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:52 gibba032 bash[24004]: cluster 2021-03-10T15:43:51.292217+0000 mgr.gibba032.whopeo (mgr.24625) 5875 : cluster [DBG] pgmap v3340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:52 gibba011 bash[30791]: cluster 2021-03-10T15:43:51.292217+0000 mgr.gibba032.whopeo (mgr.24625) 5875 : cluster [DBG] pgmap v3340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:52.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:52 gibba011 bash[31774]: cluster 2021-03-10T15:43:51.292217+0000 mgr.gibba032.whopeo (mgr.24625) 5875 : cluster [DBG] pgmap v3340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:54 gibba032 bash[24004]: cluster 2021-03-10T15:43:53.293831+0000 mgr.gibba032.whopeo (mgr.24625) 5876 : cluster [DBG] pgmap v3341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:54 gibba011 bash[30791]: cluster 2021-03-10T15:43:53.293831+0000 mgr.gibba032.whopeo (mgr.24625) 5876 : cluster [DBG] pgmap v3341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:54 gibba011 bash[31774]: cluster 2021-03-10T15:43:53.293831+0000 mgr.gibba032.whopeo (mgr.24625) 5876 : cluster [DBG] pgmap v3341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:56 gibba032 bash[24004]: cluster 2021-03-10T15:43:55.295179+0000 mgr.gibba032.whopeo (mgr.24625) 5877 : cluster [DBG] pgmap v3342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:56 gibba011 bash[30791]: cluster 2021-03-10T15:43:55.295179+0000 mgr.gibba032.whopeo (mgr.24625) 5877 : cluster [DBG] pgmap v3342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:56.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:56 gibba011 bash[31774]: cluster 2021-03-10T15:43:55.295179+0000 mgr.gibba032.whopeo (mgr.24625) 5877 : cluster [DBG] pgmap v3342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.696148+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:43:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.696809+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:43:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.717309+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:43:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.718097+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.718814+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:43:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:57 gibba032 bash[24004]: audit 2021-03-10T15:43:56.719536+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:43:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.696148+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:43:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.696809+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:43:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.717309+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:43:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.718097+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:43:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.718814+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:43:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[30791]: audit 2021-03-10T15:43:56.719536+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:43:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.696148+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:43:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.696809+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:43:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.717309+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:43:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.718097+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:43:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.718814+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T15:43:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:57 gibba011 bash[31774]: audit 2021-03-10T15:43:56.719536+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:43:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:43:58 gibba032 bash[24004]: cluster 2021-03-10T15:43:57.296942+0000 mgr.gibba032.whopeo (mgr.24625) 5878 : cluster [DBG] pgmap v3343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:43:58 gibba011 bash[30791]: cluster 2021-03-10T15:43:57.296942+0000 mgr.gibba032.whopeo (mgr.24625) 5878 : cluster [DBG] pgmap v3343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:43:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:43:58 gibba011 bash[31774]: cluster 2021-03-10T15:43:57.296942+0000 mgr.gibba032.whopeo (mgr.24625) 5878 : cluster [DBG] pgmap v3343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:00 gibba032 bash[24004]: cluster 2021-03-10T15:43:59.298042+0000 mgr.gibba032.whopeo (mgr.24625) 5879 : cluster [DBG] pgmap v3344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:00.809 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:44:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:00 gibba011 bash[30791]: cluster 2021-03-10T15:43:59.298042+0000 mgr.gibba032.whopeo (mgr.24625) 5879 : cluster [DBG] pgmap v3344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:00 gibba011 bash[31774]: cluster 2021-03-10T15:43:59.298042+0000 mgr.gibba032.whopeo (mgr.24625) 5879 : cluster [DBG] pgmap v3344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:01.039 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:44:01.039 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (110m) 4m ago 116m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:44:01.039 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (115m) 3m ago 115m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:44:01.040 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (111m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:44:01.040 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (111m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:44:01.040 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (110m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:44:01.040 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (110m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:44:01.041 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (109m) 3m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:44:01.041 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (109m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:44:01.041 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (116m) 4m ago 116m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:44:01.041 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (116m) 3m ago 116m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:44:01.041 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (108m) 4m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:44:01.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (108m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:44:01.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (108m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:44:01.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (107m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:44:01.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (107m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:44:01.043 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (106m) 3m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:44:01.043 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (106m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:44:01.043 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (105m) 3m ago 116m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:44:01.043 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (115m) 3m ago 116m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:44:01.044 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 114m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:44:01.290 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:44:01.291 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:44:01.291 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:44:01.291 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:01.292 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:44:01.292 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:44:01.292 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:01.292 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:44:01.292 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:44:01.293 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:01.293 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:44:01.293 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:44:01.293 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:44:01.294 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:44:01.294 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:44:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:01 gibba032 bash[24004]: audit 2021-03-10T15:44:01.288153+0000 mon.c (mon.1) 111 : audit [DBG] from='client.? 172.21.2.111:0/243985430' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:01 gibba011 bash[30791]: audit 2021-03-10T15:44:01.288153+0000 mon.c (mon.1) 111 : audit [DBG] from='client.? 172.21.2.111:0/243985430' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:01 gibba011 bash[31774]: audit 2021-03-10T15:44:01.288153+0000 mon.c (mon.1) 111 : audit [DBG] from='client.? 172.21.2.111:0/243985430' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:02 gibba032 bash[24004]: audit 2021-03-10T15:44:00.796426+0000 mgr.gibba032.whopeo (mgr.24625) 5880 : audit [DBG] from='client.47997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:02 gibba032 bash[24004]: audit 2021-03-10T15:44:01.023288+0000 mgr.gibba032.whopeo (mgr.24625) 5881 : audit [DBG] from='client.57988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:02 gibba032 bash[24004]: cluster 2021-03-10T15:44:01.299797+0000 mgr.gibba032.whopeo (mgr.24625) 5882 : cluster [DBG] pgmap v3345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[30791]: audit 2021-03-10T15:44:00.796426+0000 mgr.gibba032.whopeo (mgr.24625) 5880 : audit [DBG] from='client.47997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[30791]: audit 2021-03-10T15:44:01.023288+0000 mgr.gibba032.whopeo (mgr.24625) 5881 : audit [DBG] from='client.57988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[30791]: cluster 2021-03-10T15:44:01.299797+0000 mgr.gibba032.whopeo (mgr.24625) 5882 : cluster [DBG] pgmap v3345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[31774]: audit 2021-03-10T15:44:00.796426+0000 mgr.gibba032.whopeo (mgr.24625) 5880 : audit [DBG] from='client.47997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[31774]: audit 2021-03-10T15:44:01.023288+0000 mgr.gibba032.whopeo (mgr.24625) 5881 : audit [DBG] from='client.57988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:02 gibba011 bash[31774]: cluster 2021-03-10T15:44:01.299797+0000 mgr.gibba032.whopeo (mgr.24625) 5882 : cluster [DBG] pgmap v3345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:04 gibba032 bash[24004]: cluster 2021-03-10T15:44:03.301274+0000 mgr.gibba032.whopeo (mgr.24625) 5883 : cluster [DBG] pgmap v3346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:04 gibba011 bash[30791]: cluster 2021-03-10T15:44:03.301274+0000 mgr.gibba032.whopeo (mgr.24625) 5883 : cluster [DBG] pgmap v3346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:04 gibba011 bash[31774]: cluster 2021-03-10T15:44:03.301274+0000 mgr.gibba032.whopeo (mgr.24625) 5883 : cluster [DBG] pgmap v3346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:05 gibba032 bash[24004]: cephadm 2021-03-10T15:44:04.273470+0000 mgr.gibba032.whopeo (mgr.24625) 5884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:05.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:05 gibba011 bash[30791]: cephadm 2021-03-10T15:44:04.273470+0000 mgr.gibba032.whopeo (mgr.24625) 5884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:05.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:05 gibba011 bash[31774]: cephadm 2021-03-10T15:44:04.273470+0000 mgr.gibba032.whopeo (mgr.24625) 5884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:06 gibba032 bash[24004]: cluster 2021-03-10T15:44:05.302732+0000 mgr.gibba032.whopeo (mgr.24625) 5885 : cluster [DBG] pgmap v3347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:06 gibba011 bash[30791]: cluster 2021-03-10T15:44:05.302732+0000 mgr.gibba032.whopeo (mgr.24625) 5885 : cluster [DBG] pgmap v3347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:06 gibba011 bash[31774]: cluster 2021-03-10T15:44:05.302732+0000 mgr.gibba032.whopeo (mgr.24625) 5885 : cluster [DBG] pgmap v3347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:08 gibba032 bash[24004]: cluster 2021-03-10T15:44:07.304524+0000 mgr.gibba032.whopeo (mgr.24625) 5886 : cluster [DBG] pgmap v3348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:08.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:08 gibba011 bash[30791]: cluster 2021-03-10T15:44:07.304524+0000 mgr.gibba032.whopeo (mgr.24625) 5886 : cluster [DBG] pgmap v3348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:08 gibba011 bash[31774]: cluster 2021-03-10T15:44:07.304524+0000 mgr.gibba032.whopeo (mgr.24625) 5886 : cluster [DBG] pgmap v3348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:10 gibba032 bash[24004]: cluster 2021-03-10T15:44:09.305840+0000 mgr.gibba032.whopeo (mgr.24625) 5887 : cluster [DBG] pgmap v3349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:10 gibba011 bash[30791]: cluster 2021-03-10T15:44:09.305840+0000 mgr.gibba032.whopeo (mgr.24625) 5887 : cluster [DBG] pgmap v3349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:10 gibba011 bash[31774]: cluster 2021-03-10T15:44:09.305840+0000 mgr.gibba032.whopeo (mgr.24625) 5887 : cluster [DBG] pgmap v3349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:12 gibba032 bash[24004]: cluster 2021-03-10T15:44:11.307401+0000 mgr.gibba032.whopeo (mgr.24625) 5888 : cluster [DBG] pgmap v3350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:12 gibba011 bash[30791]: cluster 2021-03-10T15:44:11.307401+0000 mgr.gibba032.whopeo (mgr.24625) 5888 : cluster [DBG] pgmap v3350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:12 gibba011 bash[31774]: cluster 2021-03-10T15:44:11.307401+0000 mgr.gibba032.whopeo (mgr.24625) 5888 : cluster [DBG] pgmap v3350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:14 gibba032 bash[24004]: cluster 2021-03-10T15:44:13.309077+0000 mgr.gibba032.whopeo (mgr.24625) 5889 : cluster [DBG] pgmap v3351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:14 gibba011 bash[30791]: cluster 2021-03-10T15:44:13.309077+0000 mgr.gibba032.whopeo (mgr.24625) 5889 : cluster [DBG] pgmap v3351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:14.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:14 gibba011 bash[31774]: cluster 2021-03-10T15:44:13.309077+0000 mgr.gibba032.whopeo (mgr.24625) 5889 : cluster [DBG] pgmap v3351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:16 gibba032 bash[24004]: cluster 2021-03-10T15:44:15.310641+0000 mgr.gibba032.whopeo (mgr.24625) 5890 : cluster [DBG] pgmap v3352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:16 gibba011 bash[30791]: cluster 2021-03-10T15:44:15.310641+0000 mgr.gibba032.whopeo (mgr.24625) 5890 : cluster [DBG] pgmap v3352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:16.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:16 gibba011 bash[31774]: cluster 2021-03-10T15:44:15.310641+0000 mgr.gibba032.whopeo (mgr.24625) 5890 : cluster [DBG] pgmap v3352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:18 gibba032 bash[24004]: cluster 2021-03-10T15:44:17.312454+0000 mgr.gibba032.whopeo (mgr.24625) 5891 : cluster [DBG] pgmap v3353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:18 gibba011 bash[30791]: cluster 2021-03-10T15:44:17.312454+0000 mgr.gibba032.whopeo (mgr.24625) 5891 : cluster [DBG] pgmap v3353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:18 gibba011 bash[31774]: cluster 2021-03-10T15:44:17.312454+0000 mgr.gibba032.whopeo (mgr.24625) 5891 : cluster [DBG] pgmap v3353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.278972+0000 mgr.gibba032.whopeo (mgr.24625) 5892 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:44:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.280617+0000 mgr.gibba032.whopeo (mgr.24625) 5893 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:44:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cluster 2021-03-10T15:44:19.312907+0000 mgr.gibba032.whopeo (mgr.24625) 5894 : cluster [DBG] pgmap v3354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.535230+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:44:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.547195+0000 mgr.gibba032.whopeo (mgr.24625) 5895 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:44:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.551426+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:44:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.553481+0000 mgr.gibba032.whopeo (mgr.24625) 5896 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:44:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.556141+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.557849+0000 mgr.gibba032.whopeo (mgr.24625) 5897 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:44:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.558421+0000 mgr.gibba032.whopeo (mgr.24625) 5898 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:44:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.560909+0000 mon.a (mon.0) 2225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.561925+0000 mgr.gibba032.whopeo (mgr.24625) 5899 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:44:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.562339+0000 mgr.gibba032.whopeo (mgr.24625) 5900 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:44:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.564501+0000 mon.a (mon.0) 2226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.565447+0000 mgr.gibba032.whopeo (mgr.24625) 5901 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:44:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.568388+0000 mgr.gibba032.whopeo (mgr.24625) 5902 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:44:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.568664+0000 mgr.gibba032.whopeo (mgr.24625) 5903 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:44:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.571564+0000 mon.a (mon.0) 2227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.572595+0000 mgr.gibba032.whopeo (mgr.24625) 5904 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:44:20.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.574648+0000 mgr.gibba032.whopeo (mgr.24625) 5905 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:44:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.278972+0000 mgr.gibba032.whopeo (mgr.24625) 5892 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:44:20.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.280617+0000 mgr.gibba032.whopeo (mgr.24625) 5893 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:44:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cluster 2021-03-10T15:44:19.312907+0000 mgr.gibba032.whopeo (mgr.24625) 5894 : cluster [DBG] pgmap v3354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.535230+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:44:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.547195+0000 mgr.gibba032.whopeo (mgr.24625) 5895 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:44:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.551426+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:44:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.553481+0000 mgr.gibba032.whopeo (mgr.24625) 5896 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:44:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.556141+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.557849+0000 mgr.gibba032.whopeo (mgr.24625) 5897 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:44:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.558421+0000 mgr.gibba032.whopeo (mgr.24625) 5898 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:44:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.560909+0000 mon.a (mon.0) 2225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.561925+0000 mgr.gibba032.whopeo (mgr.24625) 5899 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:44:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.562339+0000 mgr.gibba032.whopeo (mgr.24625) 5900 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:44:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.564501+0000 mon.a (mon.0) 2226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.565447+0000 mgr.gibba032.whopeo (mgr.24625) 5901 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:44:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.568388+0000 mgr.gibba032.whopeo (mgr.24625) 5902 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:44:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.568664+0000 mgr.gibba032.whopeo (mgr.24625) 5903 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:44:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.571564+0000 mon.a (mon.0) 2227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.572595+0000 mgr.gibba032.whopeo (mgr.24625) 5904 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:44:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.574648+0000 mgr.gibba032.whopeo (mgr.24625) 5905 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:44:20.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: audit 2021-03-10T15:44:19.576651+0000 mon.a (mon.0) 2228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.842 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.577591+0000 mgr.gibba032.whopeo (mgr.24625) 5906 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:44:20.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.577853+0000 mgr.gibba032.whopeo (mgr.24625) 5907 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:44:20.843 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[30791]: cephadm 2021-03-10T15:44:19.578610+0000 mgr.gibba032.whopeo (mgr.24625) 5908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: audit 2021-03-10T15:44:19.576651+0000 mon.a (mon.0) 2228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.577591+0000 mgr.gibba032.whopeo (mgr.24625) 5906 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:44:20.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.577853+0000 mgr.gibba032.whopeo (mgr.24625) 5907 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:44:20.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:20 gibba011 bash[31774]: cephadm 2021-03-10T15:44:19.578610+0000 mgr.gibba032.whopeo (mgr.24625) 5908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.278972+0000 mgr.gibba032.whopeo (mgr.24625) 5892 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:44:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.280617+0000 mgr.gibba032.whopeo (mgr.24625) 5893 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:44:21.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cluster 2021-03-10T15:44:19.312907+0000 mgr.gibba032.whopeo (mgr.24625) 5894 : cluster [DBG] pgmap v3354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.535230+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:44:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.547195+0000 mgr.gibba032.whopeo (mgr.24625) 5895 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:44:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.551426+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:44:21.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.553481+0000 mgr.gibba032.whopeo (mgr.24625) 5896 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:44:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.556141+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.557849+0000 mgr.gibba032.whopeo (mgr.24625) 5897 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:44:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.558421+0000 mgr.gibba032.whopeo (mgr.24625) 5898 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:44:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.560909+0000 mon.a (mon.0) 2225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:21.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.561925+0000 mgr.gibba032.whopeo (mgr.24625) 5899 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:44:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.562339+0000 mgr.gibba032.whopeo (mgr.24625) 5900 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:44:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.564501+0000 mon.a (mon.0) 2226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.565447+0000 mgr.gibba032.whopeo (mgr.24625) 5901 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:44:21.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.568388+0000 mgr.gibba032.whopeo (mgr.24625) 5902 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:44:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.568664+0000 mgr.gibba032.whopeo (mgr.24625) 5903 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:44:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.571564+0000 mon.a (mon.0) 2227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.572595+0000 mgr.gibba032.whopeo (mgr.24625) 5904 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:44:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.574648+0000 mgr.gibba032.whopeo (mgr.24625) 5905 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:44:21.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: audit 2021-03-10T15:44:19.576651+0000 mon.a (mon.0) 2228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.577591+0000 mgr.gibba032.whopeo (mgr.24625) 5906 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:44:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.577853+0000 mgr.gibba032.whopeo (mgr.24625) 5907 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:44:21.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:20 gibba032 bash[24004]: cephadm 2021-03-10T15:44:19.578610+0000 mgr.gibba032.whopeo (mgr.24625) 5908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:22 gibba011 bash[30791]: cluster 2021-03-10T15:44:21.314305+0000 mgr.gibba032.whopeo (mgr.24625) 5909 : cluster [DBG] pgmap v3355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:22.866 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:22 gibba011 bash[31774]: cluster 2021-03-10T15:44:21.314305+0000 mgr.gibba032.whopeo (mgr.24625) 5909 : cluster [DBG] pgmap v3355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:22 gibba032 bash[24004]: cluster 2021-03-10T15:44:21.314305+0000 mgr.gibba032.whopeo (mgr.24625) 5909 : cluster [DBG] pgmap v3355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:24 gibba032 bash[24004]: cluster 2021-03-10T15:44:23.315885+0000 mgr.gibba032.whopeo (mgr.24625) 5910 : cluster [DBG] pgmap v3356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:24 gibba011 bash[30791]: cluster 2021-03-10T15:44:23.315885+0000 mgr.gibba032.whopeo (mgr.24625) 5910 : cluster [DBG] pgmap v3356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:24 gibba011 bash[31774]: cluster 2021-03-10T15:44:23.315885+0000 mgr.gibba032.whopeo (mgr.24625) 5910 : cluster [DBG] pgmap v3356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:26 gibba032 bash[24004]: cluster 2021-03-10T15:44:25.317491+0000 mgr.gibba032.whopeo (mgr.24625) 5911 : cluster [DBG] pgmap v3357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:26 gibba011 bash[30791]: cluster 2021-03-10T15:44:25.317491+0000 mgr.gibba032.whopeo (mgr.24625) 5911 : cluster [DBG] pgmap v3357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:26 gibba011 bash[31774]: cluster 2021-03-10T15:44:25.317491+0000 mgr.gibba032.whopeo (mgr.24625) 5911 : cluster [DBG] pgmap v3357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:28 gibba032 bash[24004]: cluster 2021-03-10T15:44:27.319123+0000 mgr.gibba032.whopeo (mgr.24625) 5912 : cluster [DBG] pgmap v3358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:28 gibba011 bash[30791]: cluster 2021-03-10T15:44:27.319123+0000 mgr.gibba032.whopeo (mgr.24625) 5912 : cluster [DBG] pgmap v3358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:28.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:28 gibba011 bash[31774]: cluster 2021-03-10T15:44:27.319123+0000 mgr.gibba032.whopeo (mgr.24625) 5912 : cluster [DBG] pgmap v3358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:30 gibba032 bash[24004]: cluster 2021-03-10T15:44:29.320650+0000 mgr.gibba032.whopeo (mgr.24625) 5913 : cluster [DBG] pgmap v3359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:30 gibba011 bash[30791]: cluster 2021-03-10T15:44:29.320650+0000 mgr.gibba032.whopeo (mgr.24625) 5913 : cluster [DBG] pgmap v3359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:30.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:30 gibba011 bash[31774]: cluster 2021-03-10T15:44:29.320650+0000 mgr.gibba032.whopeo (mgr.24625) 5913 : cluster [DBG] pgmap v3359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:31.538 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:44:31.769 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:44:31.769 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (111m) 5m ago 116m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:44:31.770 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (116m) 4m ago 116m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:44:31.770 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (111m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:44:31.770 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (111m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:44:31.771 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (110m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:44:31.771 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (110m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:44:31.771 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (110m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:44:31.771 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (110m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:44:31.771 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (116m) 5m ago 116m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:44:31.772 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (116m) 4m ago 116m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:44:31.772 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (109m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:44:31.772 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (109m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:44:31.772 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (108m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:44:31.773 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (108m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:44:31.773 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (107m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:44:31.773 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (107m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:44:31.773 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (106m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:44:31.774 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (106m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:44:31.774 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (116m) 4m ago 116m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:44:31.774 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 114m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:44:32.019 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:44:32.019 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:44:32.019 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:44:32.020 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:32.020 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:44:32.020 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:44:32.020 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:32.020 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:44:32.021 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:44:32.021 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:44:32.021 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:44:32.021 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:44:32.022 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:44:32.022 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:44:32.022 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:44:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:32 gibba032 bash[24004]: cluster 2021-03-10T15:44:31.322146+0000 mgr.gibba032.whopeo (mgr.24625) 5914 : cluster [DBG] pgmap v3360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:32 gibba032 bash[24004]: audit 2021-03-10T15:44:31.525665+0000 mgr.gibba032.whopeo (mgr.24625) 5915 : audit [DBG] from='client.58000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:32 gibba032 bash[24004]: audit 2021-03-10T15:44:31.753263+0000 mgr.gibba032.whopeo (mgr.24625) 5916 : audit [DBG] from='client.58006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:32 gibba032 bash[24004]: audit 2021-03-10T15:44:32.016779+0000 mon.a (mon.0) 2229 : audit [DBG] from='client.? 172.21.2.111:0/2947046058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[30791]: cluster 2021-03-10T15:44:31.322146+0000 mgr.gibba032.whopeo (mgr.24625) 5914 : cluster [DBG] pgmap v3360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[30791]: audit 2021-03-10T15:44:31.525665+0000 mgr.gibba032.whopeo (mgr.24625) 5915 : audit [DBG] from='client.58000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[30791]: audit 2021-03-10T15:44:31.753263+0000 mgr.gibba032.whopeo (mgr.24625) 5916 : audit [DBG] from='client.58006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[30791]: audit 2021-03-10T15:44:32.016779+0000 mon.a (mon.0) 2229 : audit [DBG] from='client.? 172.21.2.111:0/2947046058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[31774]: cluster 2021-03-10T15:44:31.322146+0000 mgr.gibba032.whopeo (mgr.24625) 5914 : cluster [DBG] pgmap v3360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[31774]: audit 2021-03-10T15:44:31.525665+0000 mgr.gibba032.whopeo (mgr.24625) 5915 : audit [DBG] from='client.58000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[31774]: audit 2021-03-10T15:44:31.753263+0000 mgr.gibba032.whopeo (mgr.24625) 5916 : audit [DBG] from='client.58006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:44:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:32 gibba011 bash[31774]: audit 2021-03-10T15:44:32.016779+0000 mon.a (mon.0) 2229 : audit [DBG] from='client.? 172.21.2.111:0/2947046058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:44:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:34 gibba032 bash[24004]: cluster 2021-03-10T15:44:33.323584+0000 mgr.gibba032.whopeo (mgr.24625) 5917 : cluster [DBG] pgmap v3361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:34 gibba011 bash[30791]: cluster 2021-03-10T15:44:33.323584+0000 mgr.gibba032.whopeo (mgr.24625) 5917 : cluster [DBG] pgmap v3361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:34 gibba011 bash[31774]: cluster 2021-03-10T15:44:33.323584+0000 mgr.gibba032.whopeo (mgr.24625) 5917 : cluster [DBG] pgmap v3361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:35 gibba032 bash[24004]: cephadm 2021-03-10T15:44:34.594843+0000 mgr.gibba032.whopeo (mgr.24625) 5918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:35 gibba011 bash[30791]: cephadm 2021-03-10T15:44:34.594843+0000 mgr.gibba032.whopeo (mgr.24625) 5918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:35 gibba011 bash[31774]: cephadm 2021-03-10T15:44:34.594843+0000 mgr.gibba032.whopeo (mgr.24625) 5918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:36 gibba032 bash[24004]: cluster 2021-03-10T15:44:35.324978+0000 mgr.gibba032.whopeo (mgr.24625) 5919 : cluster [DBG] pgmap v3362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:36 gibba011 bash[30791]: cluster 2021-03-10T15:44:35.324978+0000 mgr.gibba032.whopeo (mgr.24625) 5919 : cluster [DBG] pgmap v3362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:36 gibba011 bash[31774]: cluster 2021-03-10T15:44:35.324978+0000 mgr.gibba032.whopeo (mgr.24625) 5919 : cluster [DBG] pgmap v3362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:38 gibba032 bash[24004]: cluster 2021-03-10T15:44:37.326698+0000 mgr.gibba032.whopeo (mgr.24625) 5920 : cluster [DBG] pgmap v3363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:38 gibba011 bash[30791]: cluster 2021-03-10T15:44:37.326698+0000 mgr.gibba032.whopeo (mgr.24625) 5920 : cluster [DBG] pgmap v3363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:38.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:38 gibba011 bash[31774]: cluster 2021-03-10T15:44:37.326698+0000 mgr.gibba032.whopeo (mgr.24625) 5920 : cluster [DBG] pgmap v3363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:40 gibba032 bash[24004]: cluster 2021-03-10T15:44:39.328188+0000 mgr.gibba032.whopeo (mgr.24625) 5921 : cluster [DBG] pgmap v3364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:40 gibba011 bash[30791]: cluster 2021-03-10T15:44:39.328188+0000 mgr.gibba032.whopeo (mgr.24625) 5921 : cluster [DBG] pgmap v3364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:40 gibba011 bash[31774]: cluster 2021-03-10T15:44:39.328188+0000 mgr.gibba032.whopeo (mgr.24625) 5921 : cluster [DBG] pgmap v3364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:42 gibba032 bash[24004]: cluster 2021-03-10T15:44:41.329697+0000 mgr.gibba032.whopeo (mgr.24625) 5922 : cluster [DBG] pgmap v3365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:42 gibba011 bash[30791]: cluster 2021-03-10T15:44:41.329697+0000 mgr.gibba032.whopeo (mgr.24625) 5922 : cluster [DBG] pgmap v3365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:42 gibba011 bash[31774]: cluster 2021-03-10T15:44:41.329697+0000 mgr.gibba032.whopeo (mgr.24625) 5922 : cluster [DBG] pgmap v3365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:44 gibba032 bash[24004]: cluster 2021-03-10T15:44:43.331283+0000 mgr.gibba032.whopeo (mgr.24625) 5923 : cluster [DBG] pgmap v3366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:44 gibba011 bash[30791]: cluster 2021-03-10T15:44:43.331283+0000 mgr.gibba032.whopeo (mgr.24625) 5923 : cluster [DBG] pgmap v3366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:44.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:44 gibba011 bash[31774]: cluster 2021-03-10T15:44:43.331283+0000 mgr.gibba032.whopeo (mgr.24625) 5923 : cluster [DBG] pgmap v3366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:46 gibba032 bash[24004]: cluster 2021-03-10T15:44:45.332905+0000 mgr.gibba032.whopeo (mgr.24625) 5924 : cluster [DBG] pgmap v3367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:46 gibba011 bash[30791]: cluster 2021-03-10T15:44:45.332905+0000 mgr.gibba032.whopeo (mgr.24625) 5924 : cluster [DBG] pgmap v3367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:46 gibba011 bash[31774]: cluster 2021-03-10T15:44:45.332905+0000 mgr.gibba032.whopeo (mgr.24625) 5924 : cluster [DBG] pgmap v3367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:48 gibba032 bash[24004]: cluster 2021-03-10T15:44:47.334638+0000 mgr.gibba032.whopeo (mgr.24625) 5925 : cluster [DBG] pgmap v3368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:48 gibba011 bash[30791]: cluster 2021-03-10T15:44:47.334638+0000 mgr.gibba032.whopeo (mgr.24625) 5925 : cluster [DBG] pgmap v3368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:48 gibba011 bash[31774]: cluster 2021-03-10T15:44:47.334638+0000 mgr.gibba032.whopeo (mgr.24625) 5925 : cluster [DBG] pgmap v3368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:50 gibba032 bash[24004]: cluster 2021-03-10T15:44:49.336135+0000 mgr.gibba032.whopeo (mgr.24625) 5926 : cluster [DBG] pgmap v3369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:50 gibba032 bash[24004]: cephadm 2021-03-10T15:44:49.610848+0000 mgr.gibba032.whopeo (mgr.24625) 5927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:50 gibba011 bash[30791]: cluster 2021-03-10T15:44:49.336135+0000 mgr.gibba032.whopeo (mgr.24625) 5926 : cluster [DBG] pgmap v3369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:50 gibba011 bash[30791]: cephadm 2021-03-10T15:44:49.610848+0000 mgr.gibba032.whopeo (mgr.24625) 5927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:50 gibba011 bash[31774]: cluster 2021-03-10T15:44:49.336135+0000 mgr.gibba032.whopeo (mgr.24625) 5926 : cluster [DBG] pgmap v3369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:50.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:50 gibba011 bash[31774]: cephadm 2021-03-10T15:44:49.610848+0000 mgr.gibba032.whopeo (mgr.24625) 5927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:44:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:52 gibba032 bash[24004]: cluster 2021-03-10T15:44:51.337556+0000 mgr.gibba032.whopeo (mgr.24625) 5928 : cluster [DBG] pgmap v3370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:52 gibba011 bash[30791]: cluster 2021-03-10T15:44:51.337556+0000 mgr.gibba032.whopeo (mgr.24625) 5928 : cluster [DBG] pgmap v3370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:52 gibba011 bash[31774]: cluster 2021-03-10T15:44:51.337556+0000 mgr.gibba032.whopeo (mgr.24625) 5928 : cluster [DBG] pgmap v3370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:54 gibba032 bash[24004]: cluster 2021-03-10T15:44:53.339019+0000 mgr.gibba032.whopeo (mgr.24625) 5929 : cluster [DBG] pgmap v3371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:54 gibba011 bash[30791]: cluster 2021-03-10T15:44:53.339019+0000 mgr.gibba032.whopeo (mgr.24625) 5929 : cluster [DBG] pgmap v3371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:54 gibba011 bash[31774]: cluster 2021-03-10T15:44:53.339019+0000 mgr.gibba032.whopeo (mgr.24625) 5929 : cluster [DBG] pgmap v3371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:56 gibba032 bash[24004]: cluster 2021-03-10T15:44:55.340414+0000 mgr.gibba032.whopeo (mgr.24625) 5930 : cluster [DBG] pgmap v3372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:56.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:56 gibba011 bash[30791]: cluster 2021-03-10T15:44:55.340414+0000 mgr.gibba032.whopeo (mgr.24625) 5930 : cluster [DBG] pgmap v3372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:56 gibba011 bash[31774]: cluster 2021-03-10T15:44:55.340414+0000 mgr.gibba032.whopeo (mgr.24625) 5930 : cluster [DBG] pgmap v3372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.699362+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:44:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.700066+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:44:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.724250+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:44:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.725054+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:44:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.725851+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:44:57.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:57 gibba032 bash[24004]: audit 2021-03-10T15:44:56.726665+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:44:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.699362+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:44:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.700066+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:44:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.724250+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:44:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.725054+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:44:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.725851+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:44:57.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[30791]: audit 2021-03-10T15:44:56.726665+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:44:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.699362+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:44:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.700066+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:44:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.724250+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T15:44:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.725054+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:44:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.725851+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:44:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:57 gibba011 bash[31774]: audit 2021-03-10T15:44:56.726665+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:44:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:44:58 gibba032 bash[24004]: cluster 2021-03-10T15:44:57.342183+0000 mgr.gibba032.whopeo (mgr.24625) 5931 : cluster [DBG] pgmap v3373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:44:58 gibba011 bash[30791]: cluster 2021-03-10T15:44:57.342183+0000 mgr.gibba032.whopeo (mgr.24625) 5931 : cluster [DBG] pgmap v3373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:44:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:44:58 gibba011 bash[31774]: cluster 2021-03-10T15:44:57.342183+0000 mgr.gibba032.whopeo (mgr.24625) 5931 : cluster [DBG] pgmap v3373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:00 gibba032 bash[24004]: cluster 2021-03-10T15:44:59.343538+0000 mgr.gibba032.whopeo (mgr.24625) 5932 : cluster [DBG] pgmap v3374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:00 gibba011 bash[30791]: cluster 2021-03-10T15:44:59.343538+0000 mgr.gibba032.whopeo (mgr.24625) 5932 : cluster [DBG] pgmap v3374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:00 gibba011 bash[31774]: cluster 2021-03-10T15:44:59.343538+0000 mgr.gibba032.whopeo (mgr.24625) 5932 : cluster [DBG] pgmap v3374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:02.272 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:45:02.500 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:45:02.500 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (111m) 5m ago 117m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:45:02.500 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (116m) 4m ago 116m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:45:02.501 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (112m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:45:02.501 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (112m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:45:02.501 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (111m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:45:02.501 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (111m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:45:02.501 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (110m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:45:02.502 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (110m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:45:02.502 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (117m) 5m ago 117m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:45:02.502 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (117m) 4m ago 117m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:45:02.502 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (109m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:45:02.503 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (109m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:45:02.503 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (109m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:45:02.503 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (108m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:45:02.503 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (108m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:45:02.504 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (107m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:45:02.504 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (107m) 4m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:45:02.504 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (106m) 4m ago 117m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:45:02.504 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (116m) 4m ago 117m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:45:02.504 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 115m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:45:02.747 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:45:02.747 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:45:02.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:45:02.748 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:02.748 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:45:02.748 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:45:02.748 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:02.748 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:45:02.749 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:45:02.749 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:02.749 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:45:02.749 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:45:02.750 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:45:02.750 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:45:02.750 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:45:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:02 gibba032 bash[24004]: cluster 2021-03-10T15:45:01.345078+0000 mgr.gibba032.whopeo (mgr.24625) 5933 : cluster [DBG] pgmap v3375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:02 gibba011 bash[30791]: cluster 2021-03-10T15:45:01.345078+0000 mgr.gibba032.whopeo (mgr.24625) 5933 : cluster [DBG] pgmap v3375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:02 gibba011 bash[31774]: cluster 2021-03-10T15:45:01.345078+0000 mgr.gibba032.whopeo (mgr.24625) 5933 : cluster [DBG] pgmap v3375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:03 gibba032 bash[24004]: audit 2021-03-10T15:45:02.251354+0000 mgr.gibba032.whopeo (mgr.24625) 5934 : audit [DBG] from='client.48033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:03 gibba032 bash[24004]: audit 2021-03-10T15:45:02.486867+0000 mgr.gibba032.whopeo (mgr.24625) 5935 : audit [DBG] from='client.58024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:03 gibba032 bash[24004]: audit 2021-03-10T15:45:02.744719+0000 mon.c (mon.1) 112 : audit [DBG] from='client.? 172.21.2.111:0/382539241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[30791]: audit 2021-03-10T15:45:02.251354+0000 mgr.gibba032.whopeo (mgr.24625) 5934 : audit [DBG] from='client.48033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[30791]: audit 2021-03-10T15:45:02.486867+0000 mgr.gibba032.whopeo (mgr.24625) 5935 : audit [DBG] from='client.58024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[30791]: audit 2021-03-10T15:45:02.744719+0000 mon.c (mon.1) 112 : audit [DBG] from='client.? 172.21.2.111:0/382539241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:03.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[31774]: audit 2021-03-10T15:45:02.251354+0000 mgr.gibba032.whopeo (mgr.24625) 5934 : audit [DBG] from='client.48033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[31774]: audit 2021-03-10T15:45:02.486867+0000 mgr.gibba032.whopeo (mgr.24625) 5935 : audit [DBG] from='client.58024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:03.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:03 gibba011 bash[31774]: audit 2021-03-10T15:45:02.744719+0000 mon.c (mon.1) 112 : audit [DBG] from='client.? 172.21.2.111:0/382539241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:04 gibba032 bash[24004]: cluster 2021-03-10T15:45:03.346643+0000 mgr.gibba032.whopeo (mgr.24625) 5936 : cluster [DBG] pgmap v3376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:04 gibba011 bash[30791]: cluster 2021-03-10T15:45:03.346643+0000 mgr.gibba032.whopeo (mgr.24625) 5936 : cluster [DBG] pgmap v3376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:04 gibba011 bash[31774]: cluster 2021-03-10T15:45:03.346643+0000 mgr.gibba032.whopeo (mgr.24625) 5936 : cluster [DBG] pgmap v3376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:05.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:05 gibba032 bash[24004]: cephadm 2021-03-10T15:45:04.616903+0000 mgr.gibba032.whopeo (mgr.24625) 5937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:05 gibba011 bash[30791]: cephadm 2021-03-10T15:45:04.616903+0000 mgr.gibba032.whopeo (mgr.24625) 5937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:05 gibba011 bash[31774]: cephadm 2021-03-10T15:45:04.616903+0000 mgr.gibba032.whopeo (mgr.24625) 5937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:06 gibba032 bash[24004]: cluster 2021-03-10T15:45:05.348141+0000 mgr.gibba032.whopeo (mgr.24625) 5938 : cluster [DBG] pgmap v3377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:06 gibba011 bash[30791]: cluster 2021-03-10T15:45:05.348141+0000 mgr.gibba032.whopeo (mgr.24625) 5938 : cluster [DBG] pgmap v3377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:06 gibba011 bash[31774]: cluster 2021-03-10T15:45:05.348141+0000 mgr.gibba032.whopeo (mgr.24625) 5938 : cluster [DBG] pgmap v3377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:08 gibba032 bash[24004]: cluster 2021-03-10T15:45:07.349966+0000 mgr.gibba032.whopeo (mgr.24625) 5939 : cluster [DBG] pgmap v3378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:08 gibba011 bash[30791]: cluster 2021-03-10T15:45:07.349966+0000 mgr.gibba032.whopeo (mgr.24625) 5939 : cluster [DBG] pgmap v3378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:08 gibba011 bash[31774]: cluster 2021-03-10T15:45:07.349966+0000 mgr.gibba032.whopeo (mgr.24625) 5939 : cluster [DBG] pgmap v3378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:10 gibba032 bash[24004]: cluster 2021-03-10T15:45:09.351461+0000 mgr.gibba032.whopeo (mgr.24625) 5940 : cluster [DBG] pgmap v3379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:10 gibba011 bash[30791]: cluster 2021-03-10T15:45:09.351461+0000 mgr.gibba032.whopeo (mgr.24625) 5940 : cluster [DBG] pgmap v3379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:10 gibba011 bash[31774]: cluster 2021-03-10T15:45:09.351461+0000 mgr.gibba032.whopeo (mgr.24625) 5940 : cluster [DBG] pgmap v3379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:12 gibba032 bash[24004]: cluster 2021-03-10T15:45:11.353190+0000 mgr.gibba032.whopeo (mgr.24625) 5941 : cluster [DBG] pgmap v3380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:12 gibba011 bash[30791]: cluster 2021-03-10T15:45:11.353190+0000 mgr.gibba032.whopeo (mgr.24625) 5941 : cluster [DBG] pgmap v3380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:12 gibba011 bash[31774]: cluster 2021-03-10T15:45:11.353190+0000 mgr.gibba032.whopeo (mgr.24625) 5941 : cluster [DBG] pgmap v3380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:14 gibba032 bash[24004]: cluster 2021-03-10T15:45:13.354541+0000 mgr.gibba032.whopeo (mgr.24625) 5942 : cluster [DBG] pgmap v3381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:14 gibba011 bash[30791]: cluster 2021-03-10T15:45:13.354541+0000 mgr.gibba032.whopeo (mgr.24625) 5942 : cluster [DBG] pgmap v3381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:14 gibba011 bash[31774]: cluster 2021-03-10T15:45:13.354541+0000 mgr.gibba032.whopeo (mgr.24625) 5942 : cluster [DBG] pgmap v3381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:16 gibba032 bash[24004]: cluster 2021-03-10T15:45:15.355926+0000 mgr.gibba032.whopeo (mgr.24625) 5943 : cluster [DBG] pgmap v3382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:16 gibba011 bash[30791]: cluster 2021-03-10T15:45:15.355926+0000 mgr.gibba032.whopeo (mgr.24625) 5943 : cluster [DBG] pgmap v3382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:16 gibba011 bash[31774]: cluster 2021-03-10T15:45:15.355926+0000 mgr.gibba032.whopeo (mgr.24625) 5943 : cluster [DBG] pgmap v3382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:18 gibba032 bash[24004]: cluster 2021-03-10T15:45:17.357541+0000 mgr.gibba032.whopeo (mgr.24625) 5944 : cluster [DBG] pgmap v3383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:18 gibba011 bash[30791]: cluster 2021-03-10T15:45:17.357541+0000 mgr.gibba032.whopeo (mgr.24625) 5944 : cluster [DBG] pgmap v3383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:18 gibba011 bash[31774]: cluster 2021-03-10T15:45:17.357541+0000 mgr.gibba032.whopeo (mgr.24625) 5944 : cluster [DBG] pgmap v3383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: cluster 2021-03-10T15:45:19.359046+0000 mgr.gibba032.whopeo (mgr.24625) 5945 : cluster [DBG] pgmap v3384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.629792+0000 mgr.gibba032.whopeo (mgr.24625) 5946 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:45:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.631336+0000 mgr.gibba032.whopeo (mgr.24625) 5947 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:45:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.858025+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:45:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.861442+0000 mon.a (mon.0) 2237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:45:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.863212+0000 mon.a (mon.0) 2238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.867040+0000 mon.a (mon.0) 2239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.870569+0000 mon.a (mon.0) 2240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.877880+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:20 gibba032 bash[24004]: audit 2021-03-10T15:45:19.883049+0000 mon.a (mon.0) 2242 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: cluster 2021-03-10T15:45:19.359046+0000 mgr.gibba032.whopeo (mgr.24625) 5945 : cluster [DBG] pgmap v3384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.629792+0000 mgr.gibba032.whopeo (mgr.24625) 5946 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:45:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.631336+0000 mgr.gibba032.whopeo (mgr.24625) 5947 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:45:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.858025+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:45:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.861442+0000 mon.a (mon.0) 2237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:45:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.863212+0000 mon.a (mon.0) 2238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.867040+0000 mon.a (mon.0) 2239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.870569+0000 mon.a (mon.0) 2240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.877880+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[30791]: audit 2021-03-10T15:45:19.883049+0000 mon.a (mon.0) 2242 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: cluster 2021-03-10T15:45:19.359046+0000 mgr.gibba032.whopeo (mgr.24625) 5945 : cluster [DBG] pgmap v3384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.629792+0000 mgr.gibba032.whopeo (mgr.24625) 5946 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:45:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.631336+0000 mgr.gibba032.whopeo (mgr.24625) 5947 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:45:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.858025+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:45:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.861442+0000 mon.a (mon.0) 2237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:45:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.863212+0000 mon.a (mon.0) 2238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.867040+0000 mon.a (mon.0) 2239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.870569+0000 mon.a (mon.0) 2240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.877880+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:20.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:20 gibba011 bash[31774]: audit 2021-03-10T15:45:19.883049+0000 mon.a (mon.0) 2242 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.859345+0000 mgr.gibba032.whopeo (mgr.24625) 5948 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:45:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.861477+0000 mgr.gibba032.whopeo (mgr.24625) 5949 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:45:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.864133+0000 mgr.gibba032.whopeo (mgr.24625) 5950 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:45:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.864711+0000 mgr.gibba032.whopeo (mgr.24625) 5951 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:45:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.868120+0000 mgr.gibba032.whopeo (mgr.24625) 5952 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:45:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.868366+0000 mgr.gibba032.whopeo (mgr.24625) 5953 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:45:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.871425+0000 mgr.gibba032.whopeo (mgr.24625) 5954 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:45:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.874361+0000 mgr.gibba032.whopeo (mgr.24625) 5955 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:45:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.874634+0000 mgr.gibba032.whopeo (mgr.24625) 5956 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:45:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.878686+0000 mgr.gibba032.whopeo (mgr.24625) 5957 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:45:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.880807+0000 mgr.gibba032.whopeo (mgr.24625) 5958 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:45:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.884082+0000 mgr.gibba032.whopeo (mgr.24625) 5959 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:45:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.884513+0000 mgr.gibba032.whopeo (mgr.24625) 5960 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:45:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:21 gibba032 bash[24004]: cephadm 2021-03-10T15:45:19.885089+0000 mgr.gibba032.whopeo (mgr.24625) 5961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.859345+0000 mgr.gibba032.whopeo (mgr.24625) 5948 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:45:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.861477+0000 mgr.gibba032.whopeo (mgr.24625) 5949 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:45:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.864133+0000 mgr.gibba032.whopeo (mgr.24625) 5950 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:45:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.864711+0000 mgr.gibba032.whopeo (mgr.24625) 5951 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:45:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.868120+0000 mgr.gibba032.whopeo (mgr.24625) 5952 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:45:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.868366+0000 mgr.gibba032.whopeo (mgr.24625) 5953 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:45:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.871425+0000 mgr.gibba032.whopeo (mgr.24625) 5954 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:45:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.874361+0000 mgr.gibba032.whopeo (mgr.24625) 5955 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:45:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.874634+0000 mgr.gibba032.whopeo (mgr.24625) 5956 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:45:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.878686+0000 mgr.gibba032.whopeo (mgr.24625) 5957 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:45:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.880807+0000 mgr.gibba032.whopeo (mgr.24625) 5958 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:45:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.884082+0000 mgr.gibba032.whopeo (mgr.24625) 5959 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:45:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.884513+0000 mgr.gibba032.whopeo (mgr.24625) 5960 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:45:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[30791]: cephadm 2021-03-10T15:45:19.885089+0000 mgr.gibba032.whopeo (mgr.24625) 5961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.859345+0000 mgr.gibba032.whopeo (mgr.24625) 5948 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:45:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.861477+0000 mgr.gibba032.whopeo (mgr.24625) 5949 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:45:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.864133+0000 mgr.gibba032.whopeo (mgr.24625) 5950 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:45:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.864711+0000 mgr.gibba032.whopeo (mgr.24625) 5951 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:45:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.868120+0000 mgr.gibba032.whopeo (mgr.24625) 5952 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:45:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.868366+0000 mgr.gibba032.whopeo (mgr.24625) 5953 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:45:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.871425+0000 mgr.gibba032.whopeo (mgr.24625) 5954 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:45:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.874361+0000 mgr.gibba032.whopeo (mgr.24625) 5955 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:45:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.874634+0000 mgr.gibba032.whopeo (mgr.24625) 5956 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:45:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.878686+0000 mgr.gibba032.whopeo (mgr.24625) 5957 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:45:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.880807+0000 mgr.gibba032.whopeo (mgr.24625) 5958 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:45:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.884082+0000 mgr.gibba032.whopeo (mgr.24625) 5959 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:45:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.884513+0000 mgr.gibba032.whopeo (mgr.24625) 5960 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:45:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:21 gibba011 bash[31774]: cephadm 2021-03-10T15:45:19.885089+0000 mgr.gibba032.whopeo (mgr.24625) 5961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:22 gibba032 bash[24004]: cluster 2021-03-10T15:45:21.360712+0000 mgr.gibba032.whopeo (mgr.24625) 5962 : cluster [DBG] pgmap v3385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:22 gibba011 bash[30791]: cluster 2021-03-10T15:45:21.360712+0000 mgr.gibba032.whopeo (mgr.24625) 5962 : cluster [DBG] pgmap v3385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:22 gibba011 bash[31774]: cluster 2021-03-10T15:45:21.360712+0000 mgr.gibba032.whopeo (mgr.24625) 5962 : cluster [DBG] pgmap v3385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:24 gibba032 bash[24004]: cluster 2021-03-10T15:45:23.362076+0000 mgr.gibba032.whopeo (mgr.24625) 5963 : cluster [DBG] pgmap v3386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:24 gibba011 bash[30791]: cluster 2021-03-10T15:45:23.362076+0000 mgr.gibba032.whopeo (mgr.24625) 5963 : cluster [DBG] pgmap v3386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:24 gibba011 bash[31774]: cluster 2021-03-10T15:45:23.362076+0000 mgr.gibba032.whopeo (mgr.24625) 5963 : cluster [DBG] pgmap v3386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:26 gibba032 bash[24004]: cluster 2021-03-10T15:45:25.363345+0000 mgr.gibba032.whopeo (mgr.24625) 5964 : cluster [DBG] pgmap v3387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:26 gibba011 bash[30791]: cluster 2021-03-10T15:45:25.363345+0000 mgr.gibba032.whopeo (mgr.24625) 5964 : cluster [DBG] pgmap v3387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:26 gibba011 bash[31774]: cluster 2021-03-10T15:45:25.363345+0000 mgr.gibba032.whopeo (mgr.24625) 5964 : cluster [DBG] pgmap v3387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:28.820 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:28 gibba032 bash[24004]: cluster 2021-03-10T15:45:27.365145+0000 mgr.gibba032.whopeo (mgr.24625) 5965 : cluster [DBG] pgmap v3388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:28 gibba011 bash[30791]: cluster 2021-03-10T15:45:27.365145+0000 mgr.gibba032.whopeo (mgr.24625) 5965 : cluster [DBG] pgmap v3388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:28 gibba011 bash[31774]: cluster 2021-03-10T15:45:27.365145+0000 mgr.gibba032.whopeo (mgr.24625) 5965 : cluster [DBG] pgmap v3388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:30 gibba032 bash[24004]: cluster 2021-03-10T15:45:29.366510+0000 mgr.gibba032.whopeo (mgr.24625) 5966 : cluster [DBG] pgmap v3389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:30 gibba011 bash[30791]: cluster 2021-03-10T15:45:29.366510+0000 mgr.gibba032.whopeo (mgr.24625) 5966 : cluster [DBG] pgmap v3389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:30 gibba011 bash[31774]: cluster 2021-03-10T15:45:29.366510+0000 mgr.gibba032.whopeo (mgr.24625) 5966 : cluster [DBG] pgmap v3389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:32 gibba032 bash[24004]: cluster 2021-03-10T15:45:31.367969+0000 mgr.gibba032.whopeo (mgr.24625) 5967 : cluster [DBG] pgmap v3390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:32 gibba011 bash[30791]: cluster 2021-03-10T15:45:31.367969+0000 mgr.gibba032.whopeo (mgr.24625) 5967 : cluster [DBG] pgmap v3390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:32 gibba011 bash[31774]: cluster 2021-03-10T15:45:31.367969+0000 mgr.gibba032.whopeo (mgr.24625) 5967 : cluster [DBG] pgmap v3390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:32.990 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:45:33.217 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:45:33.218 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (112m) 6m ago 117m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:45:33.218 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (117m) 5m ago 117m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:45:33.218 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (112m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:45:33.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (112m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:45:33.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (111m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:45:33.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (111m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:45:33.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (111m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:45:33.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (111m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:45:33.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (117m) 6m ago 117m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:45:33.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (117m) 5m ago 117m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:45:33.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (110m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:45:33.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (110m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:45:33.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (109m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:45:33.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (109m) 6m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:45:33.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (108m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:45:33.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (108m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:45:33.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (107m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:45:33.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (107m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:45:33.222 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (117m) 5m ago 117m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:45:33.222 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 115m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:45:33.466 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:45:33.466 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:45:33.466 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:45:33.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:33.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:45:33.467 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:45:33.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:33.468 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:45:33.468 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:45:33.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:45:33.468 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:45:33.468 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:45:33.469 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:45:33.469 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:45:33.469 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:45:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:34 gibba032 bash[24004]: audit 2021-03-10T15:45:32.977965+0000 mgr.gibba032.whopeo (mgr.24625) 5968 : audit [DBG] from='client.58036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:34 gibba032 bash[24004]: audit 2021-03-10T15:45:33.205537+0000 mgr.gibba032.whopeo (mgr.24625) 5969 : audit [DBG] from='client.48057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:34 gibba032 bash[24004]: cluster 2021-03-10T15:45:33.369308+0000 mgr.gibba032.whopeo (mgr.24625) 5970 : cluster [DBG] pgmap v3391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:34 gibba032 bash[24004]: audit 2021-03-10T15:45:33.464204+0000 mon.c (mon.1) 113 : audit [DBG] from='client.? 172.21.2.111:0/4028899409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[30791]: audit 2021-03-10T15:45:32.977965+0000 mgr.gibba032.whopeo (mgr.24625) 5968 : audit [DBG] from='client.58036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[30791]: audit 2021-03-10T15:45:33.205537+0000 mgr.gibba032.whopeo (mgr.24625) 5969 : audit [DBG] from='client.48057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[30791]: cluster 2021-03-10T15:45:33.369308+0000 mgr.gibba032.whopeo (mgr.24625) 5970 : cluster [DBG] pgmap v3391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[30791]: audit 2021-03-10T15:45:33.464204+0000 mon.c (mon.1) 113 : audit [DBG] from='client.? 172.21.2.111:0/4028899409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[31774]: audit 2021-03-10T15:45:32.977965+0000 mgr.gibba032.whopeo (mgr.24625) 5968 : audit [DBG] from='client.58036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[31774]: audit 2021-03-10T15:45:33.205537+0000 mgr.gibba032.whopeo (mgr.24625) 5969 : audit [DBG] from='client.48057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:45:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[31774]: cluster 2021-03-10T15:45:33.369308+0000 mgr.gibba032.whopeo (mgr.24625) 5970 : cluster [DBG] pgmap v3391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:34 gibba011 bash[31774]: audit 2021-03-10T15:45:33.464204+0000 mon.c (mon.1) 113 : audit [DBG] from='client.? 172.21.2.111:0/4028899409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:45:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:36 gibba032 bash[24004]: cephadm 2021-03-10T15:45:34.895809+0000 mgr.gibba032.whopeo (mgr.24625) 5971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:36.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:36 gibba032 bash[24004]: cluster 2021-03-10T15:45:35.370829+0000 mgr.gibba032.whopeo (mgr.24625) 5972 : cluster [DBG] pgmap v3392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:36 gibba011 bash[30791]: cephadm 2021-03-10T15:45:34.895809+0000 mgr.gibba032.whopeo (mgr.24625) 5971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:36 gibba011 bash[30791]: cluster 2021-03-10T15:45:35.370829+0000 mgr.gibba032.whopeo (mgr.24625) 5972 : cluster [DBG] pgmap v3392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:36 gibba011 bash[31774]: cephadm 2021-03-10T15:45:34.895809+0000 mgr.gibba032.whopeo (mgr.24625) 5971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:36.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:36 gibba011 bash[31774]: cluster 2021-03-10T15:45:35.370829+0000 mgr.gibba032.whopeo (mgr.24625) 5972 : cluster [DBG] pgmap v3392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:38 gibba032 bash[24004]: cluster 2021-03-10T15:45:37.372523+0000 mgr.gibba032.whopeo (mgr.24625) 5973 : cluster [DBG] pgmap v3393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:38 gibba011 bash[30791]: cluster 2021-03-10T15:45:37.372523+0000 mgr.gibba032.whopeo (mgr.24625) 5973 : cluster [DBG] pgmap v3393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:38 gibba011 bash[31774]: cluster 2021-03-10T15:45:37.372523+0000 mgr.gibba032.whopeo (mgr.24625) 5973 : cluster [DBG] pgmap v3393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:40 gibba032 bash[24004]: cluster 2021-03-10T15:45:39.373791+0000 mgr.gibba032.whopeo (mgr.24625) 5974 : cluster [DBG] pgmap v3394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:40 gibba011 bash[30791]: cluster 2021-03-10T15:45:39.373791+0000 mgr.gibba032.whopeo (mgr.24625) 5974 : cluster [DBG] pgmap v3394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:40 gibba011 bash[31774]: cluster 2021-03-10T15:45:39.373791+0000 mgr.gibba032.whopeo (mgr.24625) 5974 : cluster [DBG] pgmap v3394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:42 gibba032 bash[24004]: cluster 2021-03-10T15:45:41.375534+0000 mgr.gibba032.whopeo (mgr.24625) 5975 : cluster [DBG] pgmap v3395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:42 gibba011 bash[30791]: cluster 2021-03-10T15:45:41.375534+0000 mgr.gibba032.whopeo (mgr.24625) 5975 : cluster [DBG] pgmap v3395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:42 gibba011 bash[31774]: cluster 2021-03-10T15:45:41.375534+0000 mgr.gibba032.whopeo (mgr.24625) 5975 : cluster [DBG] pgmap v3395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:44 gibba032 bash[24004]: cluster 2021-03-10T15:45:43.376881+0000 mgr.gibba032.whopeo (mgr.24625) 5976 : cluster [DBG] pgmap v3396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:44 gibba011 bash[30791]: cluster 2021-03-10T15:45:43.376881+0000 mgr.gibba032.whopeo (mgr.24625) 5976 : cluster [DBG] pgmap v3396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:44 gibba011 bash[31774]: cluster 2021-03-10T15:45:43.376881+0000 mgr.gibba032.whopeo (mgr.24625) 5976 : cluster [DBG] pgmap v3396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:46 gibba032 bash[24004]: cluster 2021-03-10T15:45:45.378405+0000 mgr.gibba032.whopeo (mgr.24625) 5977 : cluster [DBG] pgmap v3397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:46 gibba011 bash[30791]: cluster 2021-03-10T15:45:45.378405+0000 mgr.gibba032.whopeo (mgr.24625) 5977 : cluster [DBG] pgmap v3397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:46 gibba011 bash[31774]: cluster 2021-03-10T15:45:45.378405+0000 mgr.gibba032.whopeo (mgr.24625) 5977 : cluster [DBG] pgmap v3397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:48 gibba032 bash[24004]: cluster 2021-03-10T15:45:47.380275+0000 mgr.gibba032.whopeo (mgr.24625) 5978 : cluster [DBG] pgmap v3398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:48 gibba011 bash[30791]: cluster 2021-03-10T15:45:47.380275+0000 mgr.gibba032.whopeo (mgr.24625) 5978 : cluster [DBG] pgmap v3398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:48 gibba011 bash[31774]: cluster 2021-03-10T15:45:47.380275+0000 mgr.gibba032.whopeo (mgr.24625) 5978 : cluster [DBG] pgmap v3398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:50 gibba032 bash[24004]: cluster 2021-03-10T15:45:49.381742+0000 mgr.gibba032.whopeo (mgr.24625) 5979 : cluster [DBG] pgmap v3399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:50 gibba011 bash[30791]: cluster 2021-03-10T15:45:49.381742+0000 mgr.gibba032.whopeo (mgr.24625) 5979 : cluster [DBG] pgmap v3399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:50 gibba011 bash[31774]: cluster 2021-03-10T15:45:49.381742+0000 mgr.gibba032.whopeo (mgr.24625) 5979 : cluster [DBG] pgmap v3399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:51 gibba032 bash[24004]: cephadm 2021-03-10T15:45:49.911815+0000 mgr.gibba032.whopeo (mgr.24625) 5980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:51 gibba011 bash[30791]: cephadm 2021-03-10T15:45:49.911815+0000 mgr.gibba032.whopeo (mgr.24625) 5980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:51.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:51 gibba011 bash[31774]: cephadm 2021-03-10T15:45:49.911815+0000 mgr.gibba032.whopeo (mgr.24625) 5980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:45:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:52 gibba032 bash[24004]: cluster 2021-03-10T15:45:51.383537+0000 mgr.gibba032.whopeo (mgr.24625) 5981 : cluster [DBG] pgmap v3400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:52.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:52 gibba011 bash[30791]: cluster 2021-03-10T15:45:51.383537+0000 mgr.gibba032.whopeo (mgr.24625) 5981 : cluster [DBG] pgmap v3400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:52 gibba011 bash[31774]: cluster 2021-03-10T15:45:51.383537+0000 mgr.gibba032.whopeo (mgr.24625) 5981 : cluster [DBG] pgmap v3400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:54 gibba032 bash[24004]: cluster 2021-03-10T15:45:53.385146+0000 mgr.gibba032.whopeo (mgr.24625) 5982 : cluster [DBG] pgmap v3401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:54 gibba011 bash[30791]: cluster 2021-03-10T15:45:53.385146+0000 mgr.gibba032.whopeo (mgr.24625) 5982 : cluster [DBG] pgmap v3401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:54 gibba011 bash[31774]: cluster 2021-03-10T15:45:53.385146+0000 mgr.gibba032.whopeo (mgr.24625) 5982 : cluster [DBG] pgmap v3401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:56 gibba032 bash[24004]: cluster 2021-03-10T15:45:55.386878+0000 mgr.gibba032.whopeo (mgr.24625) 5983 : cluster [DBG] pgmap v3402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:56 gibba011 bash[30791]: cluster 2021-03-10T15:45:55.386878+0000 mgr.gibba032.whopeo (mgr.24625) 5983 : cluster [DBG] pgmap v3402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:56 gibba011 bash[31774]: cluster 2021-03-10T15:45:55.386878+0000 mgr.gibba032.whopeo (mgr.24625) 5983 : cluster [DBG] pgmap v3402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.706600+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:45:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.707252+0000 mon.a (mon.0) 2244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:45:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.730709+0000 mon.a (mon.0) 2245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:45:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.731561+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.732220+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T15:45:57.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:57 gibba032 bash[24004]: audit 2021-03-10T15:45:56.732902+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.706600+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:45:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.707252+0000 mon.a (mon.0) 2244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:45:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.730709+0000 mon.a (mon.0) 2245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:45:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.731561+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.732220+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T15:45:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[30791]: audit 2021-03-10T15:45:56.732902+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.706600+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:45:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.707252+0000 mon.a (mon.0) 2244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:45:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.730709+0000 mon.a (mon.0) 2245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T15:45:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.731561+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.732220+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T15:45:57.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:57 gibba011 bash[31774]: audit 2021-03-10T15:45:56.732902+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:45:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:45:58 gibba032 bash[24004]: cluster 2021-03-10T15:45:57.388870+0000 mgr.gibba032.whopeo (mgr.24625) 5984 : cluster [DBG] pgmap v3403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:45:58 gibba011 bash[30791]: cluster 2021-03-10T15:45:57.388870+0000 mgr.gibba032.whopeo (mgr.24625) 5984 : cluster [DBG] pgmap v3403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:45:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:45:58 gibba011 bash[31774]: cluster 2021-03-10T15:45:57.388870+0000 mgr.gibba032.whopeo (mgr.24625) 5984 : cluster [DBG] pgmap v3403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:00 gibba032 bash[24004]: cluster 2021-03-10T15:45:59.390565+0000 mgr.gibba032.whopeo (mgr.24625) 5985 : cluster [DBG] pgmap v3404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:00 gibba011 bash[30791]: cluster 2021-03-10T15:45:59.390565+0000 mgr.gibba032.whopeo (mgr.24625) 5985 : cluster [DBG] pgmap v3404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:00 gibba011 bash[31774]: cluster 2021-03-10T15:45:59.390565+0000 mgr.gibba032.whopeo (mgr.24625) 5985 : cluster [DBG] pgmap v3404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:02 gibba032 bash[24004]: cluster 2021-03-10T15:46:01.392398+0000 mgr.gibba032.whopeo (mgr.24625) 5986 : cluster [DBG] pgmap v3405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:02 gibba011 bash[30791]: cluster 2021-03-10T15:46:01.392398+0000 mgr.gibba032.whopeo (mgr.24625) 5986 : cluster [DBG] pgmap v3405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:02 gibba011 bash[31774]: cluster 2021-03-10T15:46:01.392398+0000 mgr.gibba032.whopeo (mgr.24625) 5986 : cluster [DBG] pgmap v3405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:03.712 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:46:03.940 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:46:03.940 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (112m) 6m ago 118m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:46:03.941 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (117m) 5m ago 117m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:46:03.941 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (113m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:46:03.941 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (113m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:46:03.941 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (112m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:46:03.942 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (112m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:46:03.942 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (111m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:46:03.942 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (111m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:46:03.942 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (118m) 6m ago 118m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:46:03.943 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (118m) 5m ago 118m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:46:03.943 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (111m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:46:03.943 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (110m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:46:03.943 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (110m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:46:03.943 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (109m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:46:03.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (109m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:46:03.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (108m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:46:03.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (108m) 5m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:46:03.944 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (108m) 5m ago 118m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:46:03.944 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (117m) 5m ago 118m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:46:03.945 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 116m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:46:04.189 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:46:04.189 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:46:04.190 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:46:04.190 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:04.190 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:46:04.190 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:46:04.191 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:04.191 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:46:04.191 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:46:04.191 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:04.191 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:46:04.192 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:46:04.192 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:46:04.192 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:46:04.192 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:46:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:04 gibba032 bash[24004]: cluster 2021-03-10T15:46:03.394052+0000 mgr.gibba032.whopeo (mgr.24625) 5987 : cluster [DBG] pgmap v3406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:04 gibba032 bash[24004]: audit 2021-03-10T15:46:03.700676+0000 mgr.gibba032.whopeo (mgr.24625) 5988 : audit [DBG] from='client.48069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:04 gibba032 bash[24004]: audit 2021-03-10T15:46:04.187533+0000 mon.a (mon.0) 2249 : audit [DBG] from='client.? 172.21.2.111:0/1076626491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[30791]: cluster 2021-03-10T15:46:03.394052+0000 mgr.gibba032.whopeo (mgr.24625) 5987 : cluster [DBG] pgmap v3406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[30791]: audit 2021-03-10T15:46:03.700676+0000 mgr.gibba032.whopeo (mgr.24625) 5988 : audit [DBG] from='client.48069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[30791]: audit 2021-03-10T15:46:04.187533+0000 mon.a (mon.0) 2249 : audit [DBG] from='client.? 172.21.2.111:0/1076626491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[31774]: cluster 2021-03-10T15:46:03.394052+0000 mgr.gibba032.whopeo (mgr.24625) 5987 : cluster [DBG] pgmap v3406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[31774]: audit 2021-03-10T15:46:03.700676+0000 mgr.gibba032.whopeo (mgr.24625) 5988 : audit [DBG] from='client.48069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:04 gibba011 bash[31774]: audit 2021-03-10T15:46:04.187533+0000 mon.a (mon.0) 2249 : audit [DBG] from='client.? 172.21.2.111:0/1076626491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:05.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:05 gibba032 bash[24004]: audit 2021-03-10T15:46:03.925034+0000 mgr.gibba032.whopeo (mgr.24625) 5989 : audit [DBG] from='client.58060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:05 gibba011 bash[30791]: audit 2021-03-10T15:46:03.925034+0000 mgr.gibba032.whopeo (mgr.24625) 5989 : audit [DBG] from='client.58060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:05.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:05 gibba011 bash[31774]: audit 2021-03-10T15:46:03.925034+0000 mgr.gibba032.whopeo (mgr.24625) 5989 : audit [DBG] from='client.58060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:06 gibba032 bash[24004]: cephadm 2021-03-10T15:46:04.927839+0000 mgr.gibba032.whopeo (mgr.24625) 5990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:06 gibba032 bash[24004]: cluster 2021-03-10T15:46:05.395640+0000 mgr.gibba032.whopeo (mgr.24625) 5991 : cluster [DBG] pgmap v3407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:06 gibba011 bash[30791]: cephadm 2021-03-10T15:46:04.927839+0000 mgr.gibba032.whopeo (mgr.24625) 5990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:06 gibba011 bash[30791]: cluster 2021-03-10T15:46:05.395640+0000 mgr.gibba032.whopeo (mgr.24625) 5991 : cluster [DBG] pgmap v3407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:06 gibba011 bash[31774]: cephadm 2021-03-10T15:46:04.927839+0000 mgr.gibba032.whopeo (mgr.24625) 5990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:06.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:06 gibba011 bash[31774]: cluster 2021-03-10T15:46:05.395640+0000 mgr.gibba032.whopeo (mgr.24625) 5991 : cluster [DBG] pgmap v3407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:08 gibba032 bash[24004]: cluster 2021-03-10T15:46:07.397386+0000 mgr.gibba032.whopeo (mgr.24625) 5992 : cluster [DBG] pgmap v3408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:08 gibba011 bash[30791]: cluster 2021-03-10T15:46:07.397386+0000 mgr.gibba032.whopeo (mgr.24625) 5992 : cluster [DBG] pgmap v3408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:08 gibba011 bash[31774]: cluster 2021-03-10T15:46:07.397386+0000 mgr.gibba032.whopeo (mgr.24625) 5992 : cluster [DBG] pgmap v3408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:10 gibba032 bash[24004]: cluster 2021-03-10T15:46:09.398873+0000 mgr.gibba032.whopeo (mgr.24625) 5993 : cluster [DBG] pgmap v3409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:10 gibba011 bash[30791]: cluster 2021-03-10T15:46:09.398873+0000 mgr.gibba032.whopeo (mgr.24625) 5993 : cluster [DBG] pgmap v3409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:10 gibba011 bash[31774]: cluster 2021-03-10T15:46:09.398873+0000 mgr.gibba032.whopeo (mgr.24625) 5993 : cluster [DBG] pgmap v3409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:12 gibba032 bash[24004]: cluster 2021-03-10T15:46:11.400631+0000 mgr.gibba032.whopeo (mgr.24625) 5994 : cluster [DBG] pgmap v3410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:12 gibba011 bash[30791]: cluster 2021-03-10T15:46:11.400631+0000 mgr.gibba032.whopeo (mgr.24625) 5994 : cluster [DBG] pgmap v3410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:12 gibba011 bash[31774]: cluster 2021-03-10T15:46:11.400631+0000 mgr.gibba032.whopeo (mgr.24625) 5994 : cluster [DBG] pgmap v3410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:14 gibba032 bash[24004]: cluster 2021-03-10T15:46:13.402237+0000 mgr.gibba032.whopeo (mgr.24625) 5995 : cluster [DBG] pgmap v3411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:14 gibba011 bash[30791]: cluster 2021-03-10T15:46:13.402237+0000 mgr.gibba032.whopeo (mgr.24625) 5995 : cluster [DBG] pgmap v3411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:14 gibba011 bash[31774]: cluster 2021-03-10T15:46:13.402237+0000 mgr.gibba032.whopeo (mgr.24625) 5995 : cluster [DBG] pgmap v3411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:16 gibba032 bash[24004]: cluster 2021-03-10T15:46:15.403671+0000 mgr.gibba032.whopeo (mgr.24625) 5996 : cluster [DBG] pgmap v3412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:16 gibba011 bash[30791]: cluster 2021-03-10T15:46:15.403671+0000 mgr.gibba032.whopeo (mgr.24625) 5996 : cluster [DBG] pgmap v3412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:16 gibba011 bash[31774]: cluster 2021-03-10T15:46:15.403671+0000 mgr.gibba032.whopeo (mgr.24625) 5996 : cluster [DBG] pgmap v3412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:18 gibba032 bash[24004]: cluster 2021-03-10T15:46:17.405389+0000 mgr.gibba032.whopeo (mgr.24625) 5997 : cluster [DBG] pgmap v3413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:18 gibba011 bash[30791]: cluster 2021-03-10T15:46:17.405389+0000 mgr.gibba032.whopeo (mgr.24625) 5997 : cluster [DBG] pgmap v3413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:18 gibba011 bash[31774]: cluster 2021-03-10T15:46:17.405389+0000 mgr.gibba032.whopeo (mgr.24625) 5997 : cluster [DBG] pgmap v3413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: cluster 2021-03-10T15:46:19.406845+0000 mgr.gibba032.whopeo (mgr.24625) 5998 : cluster [DBG] pgmap v3414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.148268+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.151038+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.359245+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.366272+0000 mon.a (mon.0) 2253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:46:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.369419+0000 mon.a (mon.0) 2254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.372320+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.374853+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.380071+0000 mon.a (mon.0) 2257 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:20 gibba032 bash[24004]: audit 2021-03-10T15:46:20.382962+0000 mon.a (mon.0) 2258 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: cluster 2021-03-10T15:46:19.406845+0000 mgr.gibba032.whopeo (mgr.24625) 5998 : cluster [DBG] pgmap v3414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.148268+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.151038+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.359245+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.366272+0000 mon.a (mon.0) 2253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:46:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.369419+0000 mon.a (mon.0) 2254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.372320+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.374853+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.380071+0000 mon.a (mon.0) 2257 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[30791]: audit 2021-03-10T15:46:20.382962+0000 mon.a (mon.0) 2258 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: cluster 2021-03-10T15:46:19.406845+0000 mgr.gibba032.whopeo (mgr.24625) 5998 : cluster [DBG] pgmap v3414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.148268+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.151038+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.359245+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:46:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.366272+0000 mon.a (mon.0) 2253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:46:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.369419+0000 mon.a (mon.0) 2254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.372320+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.374853+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.380071+0000 mon.a (mon.0) 2257 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:20 gibba011 bash[31774]: audit 2021-03-10T15:46:20.382962+0000 mon.a (mon.0) 2258 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.147573+0000 mgr.gibba032.whopeo (mgr.24625) 5999 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:46:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.150726+0000 mgr.gibba032.whopeo (mgr.24625) 6000 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:46:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.364069+0000 mgr.gibba032.whopeo (mgr.24625) 6001 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:46:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.367257+0000 mgr.gibba032.whopeo (mgr.24625) 6002 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:46:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.370157+0000 mgr.gibba032.whopeo (mgr.24625) 6003 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:46:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.370199+0000 mgr.gibba032.whopeo (mgr.24625) 6004 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:46:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.372982+0000 mgr.gibba032.whopeo (mgr.24625) 6005 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:46:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.373023+0000 mgr.gibba032.whopeo (mgr.24625) 6006 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:46:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.375986+0000 mgr.gibba032.whopeo (mgr.24625) 6007 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:46:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.378072+0000 mgr.gibba032.whopeo (mgr.24625) 6008 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:46:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.378119+0000 mgr.gibba032.whopeo (mgr.24625) 6009 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:46:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.380593+0000 mgr.gibba032.whopeo (mgr.24625) 6010 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:46:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.381105+0000 mgr.gibba032.whopeo (mgr.24625) 6011 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:46:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.383998+0000 mgr.gibba032.whopeo (mgr.24625) 6012 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:46:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.384057+0000 mgr.gibba032.whopeo (mgr.24625) 6013 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:46:21.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:21 gibba032 bash[24004]: cephadm 2021-03-10T15:46:20.384303+0000 mgr.gibba032.whopeo (mgr.24625) 6014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.147573+0000 mgr.gibba032.whopeo (mgr.24625) 5999 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:46:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.150726+0000 mgr.gibba032.whopeo (mgr.24625) 6000 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:46:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.364069+0000 mgr.gibba032.whopeo (mgr.24625) 6001 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:46:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.367257+0000 mgr.gibba032.whopeo (mgr.24625) 6002 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:46:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.370157+0000 mgr.gibba032.whopeo (mgr.24625) 6003 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:46:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.370199+0000 mgr.gibba032.whopeo (mgr.24625) 6004 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:46:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.372982+0000 mgr.gibba032.whopeo (mgr.24625) 6005 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:46:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.373023+0000 mgr.gibba032.whopeo (mgr.24625) 6006 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:46:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.375986+0000 mgr.gibba032.whopeo (mgr.24625) 6007 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:46:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.378072+0000 mgr.gibba032.whopeo (mgr.24625) 6008 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:46:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.378119+0000 mgr.gibba032.whopeo (mgr.24625) 6009 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:46:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.380593+0000 mgr.gibba032.whopeo (mgr.24625) 6010 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:46:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.381105+0000 mgr.gibba032.whopeo (mgr.24625) 6011 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:46:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.383998+0000 mgr.gibba032.whopeo (mgr.24625) 6012 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:46:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.384057+0000 mgr.gibba032.whopeo (mgr.24625) 6013 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:46:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[30791]: cephadm 2021-03-10T15:46:20.384303+0000 mgr.gibba032.whopeo (mgr.24625) 6014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.147573+0000 mgr.gibba032.whopeo (mgr.24625) 5999 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:46:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.150726+0000 mgr.gibba032.whopeo (mgr.24625) 6000 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:46:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.364069+0000 mgr.gibba032.whopeo (mgr.24625) 6001 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:46:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.367257+0000 mgr.gibba032.whopeo (mgr.24625) 6002 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:46:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.370157+0000 mgr.gibba032.whopeo (mgr.24625) 6003 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:46:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.370199+0000 mgr.gibba032.whopeo (mgr.24625) 6004 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:46:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.372982+0000 mgr.gibba032.whopeo (mgr.24625) 6005 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:46:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.373023+0000 mgr.gibba032.whopeo (mgr.24625) 6006 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:46:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.375986+0000 mgr.gibba032.whopeo (mgr.24625) 6007 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:46:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.378072+0000 mgr.gibba032.whopeo (mgr.24625) 6008 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:46:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.378119+0000 mgr.gibba032.whopeo (mgr.24625) 6009 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:46:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.380593+0000 mgr.gibba032.whopeo (mgr.24625) 6010 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:46:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.381105+0000 mgr.gibba032.whopeo (mgr.24625) 6011 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:46:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.383998+0000 mgr.gibba032.whopeo (mgr.24625) 6012 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:46:21.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.384057+0000 mgr.gibba032.whopeo (mgr.24625) 6013 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:46:21.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:21 gibba011 bash[31774]: cephadm 2021-03-10T15:46:20.384303+0000 mgr.gibba032.whopeo (mgr.24625) 6014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:22 gibba032 bash[24004]: cluster 2021-03-10T15:46:21.408306+0000 mgr.gibba032.whopeo (mgr.24625) 6015 : cluster [DBG] pgmap v3415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:22 gibba011 bash[30791]: cluster 2021-03-10T15:46:21.408306+0000 mgr.gibba032.whopeo (mgr.24625) 6015 : cluster [DBG] pgmap v3415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:22 gibba011 bash[31774]: cluster 2021-03-10T15:46:21.408306+0000 mgr.gibba032.whopeo (mgr.24625) 6015 : cluster [DBG] pgmap v3415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:24 gibba032 bash[24004]: cluster 2021-03-10T15:46:23.409928+0000 mgr.gibba032.whopeo (mgr.24625) 6016 : cluster [DBG] pgmap v3416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:24 gibba011 bash[30791]: cluster 2021-03-10T15:46:23.409928+0000 mgr.gibba032.whopeo (mgr.24625) 6016 : cluster [DBG] pgmap v3416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:24 gibba011 bash[31774]: cluster 2021-03-10T15:46:23.409928+0000 mgr.gibba032.whopeo (mgr.24625) 6016 : cluster [DBG] pgmap v3416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:26 gibba032 bash[24004]: cluster 2021-03-10T15:46:25.411565+0000 mgr.gibba032.whopeo (mgr.24625) 6017 : cluster [DBG] pgmap v3417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:26 gibba011 bash[30791]: cluster 2021-03-10T15:46:25.411565+0000 mgr.gibba032.whopeo (mgr.24625) 6017 : cluster [DBG] pgmap v3417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:26.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:26 gibba011 bash[31774]: cluster 2021-03-10T15:46:25.411565+0000 mgr.gibba032.whopeo (mgr.24625) 6017 : cluster [DBG] pgmap v3417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:28 gibba032 bash[24004]: cluster 2021-03-10T15:46:27.413313+0000 mgr.gibba032.whopeo (mgr.24625) 6018 : cluster [DBG] pgmap v3418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:28 gibba011 bash[30791]: cluster 2021-03-10T15:46:27.413313+0000 mgr.gibba032.whopeo (mgr.24625) 6018 : cluster [DBG] pgmap v3418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:28 gibba011 bash[31774]: cluster 2021-03-10T15:46:27.413313+0000 mgr.gibba032.whopeo (mgr.24625) 6018 : cluster [DBG] pgmap v3418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:30 gibba032 bash[24004]: cluster 2021-03-10T15:46:29.414686+0000 mgr.gibba032.whopeo (mgr.24625) 6019 : cluster [DBG] pgmap v3419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:30.944 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:30 gibba011 bash[30791]: cluster 2021-03-10T15:46:29.414686+0000 mgr.gibba032.whopeo (mgr.24625) 6019 : cluster [DBG] pgmap v3419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:30.944 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:30 gibba011 bash[31774]: cluster 2021-03-10T15:46:29.414686+0000 mgr.gibba032.whopeo (mgr.24625) 6019 : cluster [DBG] pgmap v3419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:32 gibba032 bash[24004]: cluster 2021-03-10T15:46:31.416462+0000 mgr.gibba032.whopeo (mgr.24625) 6020 : cluster [DBG] pgmap v3420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:32 gibba011 bash[30791]: cluster 2021-03-10T15:46:31.416462+0000 mgr.gibba032.whopeo (mgr.24625) 6020 : cluster [DBG] pgmap v3420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:32 gibba011 bash[31774]: cluster 2021-03-10T15:46:31.416462+0000 mgr.gibba032.whopeo (mgr.24625) 6020 : cluster [DBG] pgmap v3420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:34.432 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:46:34.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:34 gibba032 bash[24004]: cluster 2021-03-10T15:46:33.417992+0000 mgr.gibba032.whopeo (mgr.24625) 6021 : cluster [DBG] pgmap v3421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:34 gibba011 bash[30791]: cluster 2021-03-10T15:46:33.417992+0000 mgr.gibba032.whopeo (mgr.24625) 6021 : cluster [DBG] pgmap v3421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:34.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:34 gibba011 bash[31774]: cluster 2021-03-10T15:46:33.417992+0000 mgr.gibba032.whopeo (mgr.24625) 6021 : cluster [DBG] pgmap v3421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:34.693 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:46:34.693 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (113m) 7m ago 118m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:46:34.693 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (118m) 6m ago 118m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:46:34.693 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (113m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:46:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (113m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:46:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (112m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:46:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (112m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:46:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (112m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:46:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (112m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:46:34.695 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (118m) 7m ago 118m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:46:34.695 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (118m) 6m ago 118m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:46:34.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (111m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:46:34.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (111m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:46:34.695 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:46:34.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:46:34.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (109m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:46:34.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (109m) 6m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:46:34.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (108m) 6m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:46:34.696 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (108m) 6m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:46:34.697 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (118m) 6m ago 118m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:46:34.697 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 116m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:46:34.970 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:46:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:46:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:46:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:46:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:46:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:46:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:46:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:46:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:46:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:46:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:46:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:46:34.973 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:46:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:35 gibba032 bash[24004]: audit 2021-03-10T15:46:34.421174+0000 mgr.gibba032.whopeo (mgr.24625) 6022 : audit [DBG] from='client.58072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:35 gibba032 bash[24004]: audit 2021-03-10T15:46:34.649622+0000 mgr.gibba032.whopeo (mgr.24625) 6023 : audit [DBG] from='client.48093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:35 gibba032 bash[24004]: audit 2021-03-10T15:46:34.968900+0000 mon.c (mon.1) 114 : audit [DBG] from='client.? 172.21.2.111:0/665209930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[30791]: audit 2021-03-10T15:46:34.421174+0000 mgr.gibba032.whopeo (mgr.24625) 6022 : audit [DBG] from='client.58072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[30791]: audit 2021-03-10T15:46:34.649622+0000 mgr.gibba032.whopeo (mgr.24625) 6023 : audit [DBG] from='client.48093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[30791]: audit 2021-03-10T15:46:34.968900+0000 mon.c (mon.1) 114 : audit [DBG] from='client.? 172.21.2.111:0/665209930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:35.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[31774]: audit 2021-03-10T15:46:34.421174+0000 mgr.gibba032.whopeo (mgr.24625) 6022 : audit [DBG] from='client.58072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[31774]: audit 2021-03-10T15:46:34.649622+0000 mgr.gibba032.whopeo (mgr.24625) 6023 : audit [DBG] from='client.48093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:46:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:35 gibba011 bash[31774]: audit 2021-03-10T15:46:34.968900+0000 mon.c (mon.1) 114 : audit [DBG] from='client.? 172.21.2.111:0/665209930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:46:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:36 gibba032 bash[24004]: cephadm 2021-03-10T15:46:35.394332+0000 mgr.gibba032.whopeo (mgr.24625) 6024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:36 gibba032 bash[24004]: cluster 2021-03-10T15:46:35.419464+0000 mgr.gibba032.whopeo (mgr.24625) 6025 : cluster [DBG] pgmap v3422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:36 gibba011 bash[30791]: cephadm 2021-03-10T15:46:35.394332+0000 mgr.gibba032.whopeo (mgr.24625) 6024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:36 gibba011 bash[30791]: cluster 2021-03-10T15:46:35.419464+0000 mgr.gibba032.whopeo (mgr.24625) 6025 : cluster [DBG] pgmap v3422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:36 gibba011 bash[31774]: cephadm 2021-03-10T15:46:35.394332+0000 mgr.gibba032.whopeo (mgr.24625) 6024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:36 gibba011 bash[31774]: cluster 2021-03-10T15:46:35.419464+0000 mgr.gibba032.whopeo (mgr.24625) 6025 : cluster [DBG] pgmap v3422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:38 gibba032 bash[24004]: cluster 2021-03-10T15:46:37.421222+0000 mgr.gibba032.whopeo (mgr.24625) 6026 : cluster [DBG] pgmap v3423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:38.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:38 gibba011 bash[30791]: cluster 2021-03-10T15:46:37.421222+0000 mgr.gibba032.whopeo (mgr.24625) 6026 : cluster [DBG] pgmap v3423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:38 gibba011 bash[31774]: cluster 2021-03-10T15:46:37.421222+0000 mgr.gibba032.whopeo (mgr.24625) 6026 : cluster [DBG] pgmap v3423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:40 gibba032 bash[24004]: cluster 2021-03-10T15:46:39.422760+0000 mgr.gibba032.whopeo (mgr.24625) 6027 : cluster [DBG] pgmap v3424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:40 gibba011 bash[30791]: cluster 2021-03-10T15:46:39.422760+0000 mgr.gibba032.whopeo (mgr.24625) 6027 : cluster [DBG] pgmap v3424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:40 gibba011 bash[31774]: cluster 2021-03-10T15:46:39.422760+0000 mgr.gibba032.whopeo (mgr.24625) 6027 : cluster [DBG] pgmap v3424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:42 gibba032 bash[24004]: cluster 2021-03-10T15:46:41.424302+0000 mgr.gibba032.whopeo (mgr.24625) 6028 : cluster [DBG] pgmap v3425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:42 gibba011 bash[30791]: cluster 2021-03-10T15:46:41.424302+0000 mgr.gibba032.whopeo (mgr.24625) 6028 : cluster [DBG] pgmap v3425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:42.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:42 gibba011 bash[31774]: cluster 2021-03-10T15:46:41.424302+0000 mgr.gibba032.whopeo (mgr.24625) 6028 : cluster [DBG] pgmap v3425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:44.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:44 gibba032 bash[24004]: cluster 2021-03-10T15:46:43.425925+0000 mgr.gibba032.whopeo (mgr.24625) 6029 : cluster [DBG] pgmap v3426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:44 gibba011 bash[30791]: cluster 2021-03-10T15:46:43.425925+0000 mgr.gibba032.whopeo (mgr.24625) 6029 : cluster [DBG] pgmap v3426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:44 gibba011 bash[31774]: cluster 2021-03-10T15:46:43.425925+0000 mgr.gibba032.whopeo (mgr.24625) 6029 : cluster [DBG] pgmap v3426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:46 gibba032 bash[24004]: cluster 2021-03-10T15:46:45.427468+0000 mgr.gibba032.whopeo (mgr.24625) 6030 : cluster [DBG] pgmap v3427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:46 gibba011 bash[30791]: cluster 2021-03-10T15:46:45.427468+0000 mgr.gibba032.whopeo (mgr.24625) 6030 : cluster [DBG] pgmap v3427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:46 gibba011 bash[31774]: cluster 2021-03-10T15:46:45.427468+0000 mgr.gibba032.whopeo (mgr.24625) 6030 : cluster [DBG] pgmap v3427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:48 gibba032 bash[24004]: cluster 2021-03-10T15:46:47.428949+0000 mgr.gibba032.whopeo (mgr.24625) 6031 : cluster [DBG] pgmap v3428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:48 gibba011 bash[30791]: cluster 2021-03-10T15:46:47.428949+0000 mgr.gibba032.whopeo (mgr.24625) 6031 : cluster [DBG] pgmap v3428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:48 gibba011 bash[31774]: cluster 2021-03-10T15:46:47.428949+0000 mgr.gibba032.whopeo (mgr.24625) 6031 : cluster [DBG] pgmap v3428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:50 gibba032 bash[24004]: cluster 2021-03-10T15:46:49.430406+0000 mgr.gibba032.whopeo (mgr.24625) 6032 : cluster [DBG] pgmap v3429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:50 gibba011 bash[30791]: cluster 2021-03-10T15:46:49.430406+0000 mgr.gibba032.whopeo (mgr.24625) 6032 : cluster [DBG] pgmap v3429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:50 gibba011 bash[31774]: cluster 2021-03-10T15:46:49.430406+0000 mgr.gibba032.whopeo (mgr.24625) 6032 : cluster [DBG] pgmap v3429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:51 gibba011 bash[30791]: cephadm 2021-03-10T15:46:50.410320+0000 mgr.gibba032.whopeo (mgr.24625) 6033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:51.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:51 gibba011 bash[31774]: cephadm 2021-03-10T15:46:50.410320+0000 mgr.gibba032.whopeo (mgr.24625) 6033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:52.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:51 gibba032 bash[24004]: cephadm 2021-03-10T15:46:50.410320+0000 mgr.gibba032.whopeo (mgr.24625) 6033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:46:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:52 gibba011 bash[30791]: cluster 2021-03-10T15:46:51.432228+0000 mgr.gibba032.whopeo (mgr.24625) 6034 : cluster [DBG] pgmap v3430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:52 gibba011 bash[31774]: cluster 2021-03-10T15:46:51.432228+0000 mgr.gibba032.whopeo (mgr.24625) 6034 : cluster [DBG] pgmap v3430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:52 gibba032 bash[24004]: cluster 2021-03-10T15:46:51.432228+0000 mgr.gibba032.whopeo (mgr.24625) 6034 : cluster [DBG] pgmap v3430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:54 gibba032 bash[24004]: cluster 2021-03-10T15:46:53.433813+0000 mgr.gibba032.whopeo (mgr.24625) 6035 : cluster [DBG] pgmap v3431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:54 gibba011 bash[31774]: cluster 2021-03-10T15:46:53.433813+0000 mgr.gibba032.whopeo (mgr.24625) 6035 : cluster [DBG] pgmap v3431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:54 gibba011 bash[30791]: cluster 2021-03-10T15:46:53.433813+0000 mgr.gibba032.whopeo (mgr.24625) 6035 : cluster [DBG] pgmap v3431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:56 gibba032 bash[24004]: cluster 2021-03-10T15:46:55.435443+0000 mgr.gibba032.whopeo (mgr.24625) 6036 : cluster [DBG] pgmap v3432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:56 gibba011 bash[30791]: cluster 2021-03-10T15:46:55.435443+0000 mgr.gibba032.whopeo (mgr.24625) 6036 : cluster [DBG] pgmap v3432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:56 gibba011 bash[31774]: cluster 2021-03-10T15:46:55.435443+0000 mgr.gibba032.whopeo (mgr.24625) 6036 : cluster [DBG] pgmap v3432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.709953+0000 mon.a (mon.0) 2259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:46:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.710623+0000 mon.a (mon.0) 2260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:46:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.737438+0000 mon.a (mon.0) 2261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:46:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.738195+0000 mon.a (mon.0) 2262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:46:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.738903+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:46:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[30791]: audit 2021-03-10T15:46:56.739661+0000 mon.a (mon.0) 2264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:46:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.709953+0000 mon.a (mon.0) 2259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:46:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.710623+0000 mon.a (mon.0) 2260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:46:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.737438+0000 mon.a (mon.0) 2261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:46:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.738195+0000 mon.a (mon.0) 2262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:46:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.738903+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:46:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:57 gibba011 bash[31774]: audit 2021-03-10T15:46:56.739661+0000 mon.a (mon.0) 2264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:46:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.709953+0000 mon.a (mon.0) 2259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:46:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.710623+0000 mon.a (mon.0) 2260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:46:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.737438+0000 mon.a (mon.0) 2261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T15:46:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.738195+0000 mon.a (mon.0) 2262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:46:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.738903+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T15:46:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:57 gibba032 bash[24004]: audit 2021-03-10T15:46:56.739661+0000 mon.a (mon.0) 2264 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:46:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:46:58 gibba011 bash[30791]: cluster 2021-03-10T15:46:57.437035+0000 mgr.gibba032.whopeo (mgr.24625) 6037 : cluster [DBG] pgmap v3433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:46:58 gibba011 bash[31774]: cluster 2021-03-10T15:46:57.437035+0000 mgr.gibba032.whopeo (mgr.24625) 6037 : cluster [DBG] pgmap v3433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:46:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:46:58 gibba032 bash[24004]: cluster 2021-03-10T15:46:57.437035+0000 mgr.gibba032.whopeo (mgr.24625) 6037 : cluster [DBG] pgmap v3433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:00 gibba011 bash[30791]: cluster 2021-03-10T15:46:59.438318+0000 mgr.gibba032.whopeo (mgr.24625) 6038 : cluster [DBG] pgmap v3434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:00 gibba011 bash[31774]: cluster 2021-03-10T15:46:59.438318+0000 mgr.gibba032.whopeo (mgr.24625) 6038 : cluster [DBG] pgmap v3434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:00 gibba032 bash[24004]: cluster 2021-03-10T15:46:59.438318+0000 mgr.gibba032.whopeo (mgr.24625) 6038 : cluster [DBG] pgmap v3434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:02 gibba011 bash[30791]: cluster 2021-03-10T15:47:01.439935+0000 mgr.gibba032.whopeo (mgr.24625) 6039 : cluster [DBG] pgmap v3435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:02 gibba011 bash[31774]: cluster 2021-03-10T15:47:01.439935+0000 mgr.gibba032.whopeo (mgr.24625) 6039 : cluster [DBG] pgmap v3435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:02 gibba032 bash[24004]: cluster 2021-03-10T15:47:01.439935+0000 mgr.gibba032.whopeo (mgr.24625) 6039 : cluster [DBG] pgmap v3435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:04 gibba032 bash[24004]: cluster 2021-03-10T15:47:03.441558+0000 mgr.gibba032.whopeo (mgr.24625) 6040 : cluster [DBG] pgmap v3436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:04 gibba011 bash[30791]: cluster 2021-03-10T15:47:03.441558+0000 mgr.gibba032.whopeo (mgr.24625) 6040 : cluster [DBG] pgmap v3436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:04 gibba011 bash[31774]: cluster 2021-03-10T15:47:03.441558+0000 mgr.gibba032.whopeo (mgr.24625) 6040 : cluster [DBG] pgmap v3436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:05.213 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:47:05.438 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:47:05.438 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (113m) 7m ago 119m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:47:05.438 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (118m) 6m ago 118m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:47:05.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (114m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:47:05.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (114m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:47:05.439 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (113m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:47:05.439 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (113m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:47:05.440 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (112m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:47:05.440 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (112m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:47:05.440 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (119m) 7m ago 119m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:47:05.440 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (119m) 6m ago 119m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:47:05.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (112m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:47:05.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (111m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:47:05.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (111m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:47:05.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:47:05.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (110m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:47:05.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (109m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:47:05.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (109m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:47:05.442 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (109m) 6m ago 119m 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:47:05.442 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (118m) 6m ago 119m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:47:05.442 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 117m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:47:05.688 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:47:05.688 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:47:05.688 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:47:05.689 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:05.689 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:47:05.689 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:47:05.689 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:05.690 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:47:05.690 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:47:05.690 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:05.690 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:47:05.690 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:47:05.691 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:47:05.691 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:47:05.691 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:47:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:06 gibba032 bash[24004]: audit 2021-03-10T15:47:05.201355+0000 mgr.gibba032.whopeo (mgr.24625) 6041 : audit [DBG] from='client.48105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:06 gibba032 bash[24004]: audit 2021-03-10T15:47:05.422857+0000 mgr.gibba032.whopeo (mgr.24625) 6042 : audit [DBG] from='client.58096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:06 gibba032 bash[24004]: cephadm 2021-03-10T15:47:05.434634+0000 mgr.gibba032.whopeo (mgr.24625) 6043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:06 gibba032 bash[24004]: cluster 2021-03-10T15:47:05.442340+0000 mgr.gibba032.whopeo (mgr.24625) 6044 : cluster [DBG] pgmap v3437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:06 gibba032 bash[24004]: audit 2021-03-10T15:47:05.686348+0000 mon.c (mon.1) 115 : audit [DBG] from='client.? 172.21.2.111:0/3673612931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[30791]: audit 2021-03-10T15:47:05.201355+0000 mgr.gibba032.whopeo (mgr.24625) 6041 : audit [DBG] from='client.48105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[30791]: audit 2021-03-10T15:47:05.422857+0000 mgr.gibba032.whopeo (mgr.24625) 6042 : audit [DBG] from='client.58096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[30791]: cephadm 2021-03-10T15:47:05.434634+0000 mgr.gibba032.whopeo (mgr.24625) 6043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[30791]: cluster 2021-03-10T15:47:05.442340+0000 mgr.gibba032.whopeo (mgr.24625) 6044 : cluster [DBG] pgmap v3437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:06.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[30791]: audit 2021-03-10T15:47:05.686348+0000 mon.c (mon.1) 115 : audit [DBG] from='client.? 172.21.2.111:0/3673612931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[31774]: audit 2021-03-10T15:47:05.201355+0000 mgr.gibba032.whopeo (mgr.24625) 6041 : audit [DBG] from='client.48105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[31774]: audit 2021-03-10T15:47:05.422857+0000 mgr.gibba032.whopeo (mgr.24625) 6042 : audit [DBG] from='client.58096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:06.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[31774]: cephadm 2021-03-10T15:47:05.434634+0000 mgr.gibba032.whopeo (mgr.24625) 6043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[31774]: cluster 2021-03-10T15:47:05.442340+0000 mgr.gibba032.whopeo (mgr.24625) 6044 : cluster [DBG] pgmap v3437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:06.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:06 gibba011 bash[31774]: audit 2021-03-10T15:47:05.686348+0000 mon.c (mon.1) 115 : audit [DBG] from='client.? 172.21.2.111:0/3673612931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:08 gibba011 bash[30791]: cluster 2021-03-10T15:47:07.443727+0000 mgr.gibba032.whopeo (mgr.24625) 6045 : cluster [DBG] pgmap v3438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:08 gibba011 bash[31774]: cluster 2021-03-10T15:47:07.443727+0000 mgr.gibba032.whopeo (mgr.24625) 6045 : cluster [DBG] pgmap v3438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:08 gibba032 bash[24004]: cluster 2021-03-10T15:47:07.443727+0000 mgr.gibba032.whopeo (mgr.24625) 6045 : cluster [DBG] pgmap v3438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:10 gibba011 bash[30791]: cluster 2021-03-10T15:47:09.445118+0000 mgr.gibba032.whopeo (mgr.24625) 6046 : cluster [DBG] pgmap v3439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:10 gibba011 bash[31774]: cluster 2021-03-10T15:47:09.445118+0000 mgr.gibba032.whopeo (mgr.24625) 6046 : cluster [DBG] pgmap v3439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:10 gibba032 bash[24004]: cluster 2021-03-10T15:47:09.445118+0000 mgr.gibba032.whopeo (mgr.24625) 6046 : cluster [DBG] pgmap v3439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:12 gibba011 bash[30791]: cluster 2021-03-10T15:47:11.446835+0000 mgr.gibba032.whopeo (mgr.24625) 6047 : cluster [DBG] pgmap v3440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:12 gibba011 bash[31774]: cluster 2021-03-10T15:47:11.446835+0000 mgr.gibba032.whopeo (mgr.24625) 6047 : cluster [DBG] pgmap v3440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:12 gibba032 bash[24004]: cluster 2021-03-10T15:47:11.446835+0000 mgr.gibba032.whopeo (mgr.24625) 6047 : cluster [DBG] pgmap v3440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:14 gibba032 bash[24004]: cluster 2021-03-10T15:47:13.448353+0000 mgr.gibba032.whopeo (mgr.24625) 6048 : cluster [DBG] pgmap v3441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:14 gibba011 bash[30791]: cluster 2021-03-10T15:47:13.448353+0000 mgr.gibba032.whopeo (mgr.24625) 6048 : cluster [DBG] pgmap v3441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:14 gibba011 bash[31774]: cluster 2021-03-10T15:47:13.448353+0000 mgr.gibba032.whopeo (mgr.24625) 6048 : cluster [DBG] pgmap v3441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:16 gibba011 bash[30791]: cluster 2021-03-10T15:47:15.449940+0000 mgr.gibba032.whopeo (mgr.24625) 6049 : cluster [DBG] pgmap v3442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:16 gibba011 bash[31774]: cluster 2021-03-10T15:47:15.449940+0000 mgr.gibba032.whopeo (mgr.24625) 6049 : cluster [DBG] pgmap v3442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:16 gibba032 bash[24004]: cluster 2021-03-10T15:47:15.449940+0000 mgr.gibba032.whopeo (mgr.24625) 6049 : cluster [DBG] pgmap v3442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:18 gibba011 bash[30791]: cluster 2021-03-10T15:47:17.451576+0000 mgr.gibba032.whopeo (mgr.24625) 6050 : cluster [DBG] pgmap v3443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:18 gibba011 bash[31774]: cluster 2021-03-10T15:47:17.451576+0000 mgr.gibba032.whopeo (mgr.24625) 6050 : cluster [DBG] pgmap v3443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:19.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:18 gibba032 bash[24004]: cluster 2021-03-10T15:47:17.451576+0000 mgr.gibba032.whopeo (mgr.24625) 6050 : cluster [DBG] pgmap v3443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:20 gibba011 bash[30791]: cluster 2021-03-10T15:47:19.452965+0000 mgr.gibba032.whopeo (mgr.24625) 6051 : cluster [DBG] pgmap v3444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:20 gibba011 bash[31774]: cluster 2021-03-10T15:47:19.452965+0000 mgr.gibba032.whopeo (mgr.24625) 6051 : cluster [DBG] pgmap v3444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:20 gibba032 bash[24004]: cluster 2021-03-10T15:47:19.452965+0000 mgr.gibba032.whopeo (mgr.24625) 6051 : cluster [DBG] pgmap v3444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:22.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.437652+0000 mgr.gibba032.whopeo (mgr.24625) 6052 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:47:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.439221+0000 mgr.gibba032.whopeo (mgr.24625) 6053 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:47:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.665894+0000 mon.a (mon.0) 2265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:47:22.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.666947+0000 mgr.gibba032.whopeo (mgr.24625) 6054 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:47:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.668746+0000 mgr.gibba032.whopeo (mgr.24625) 6055 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:47:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.669106+0000 mon.a (mon.0) 2266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:47:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.670032+0000 mgr.gibba032.whopeo (mgr.24625) 6056 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:47:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.670094+0000 mgr.gibba032.whopeo (mgr.24625) 6057 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:47:22.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.670482+0000 mon.a (mon.0) 2267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.671465+0000 mgr.gibba032.whopeo (mgr.24625) 6058 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:47:22.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.671525+0000 mgr.gibba032.whopeo (mgr.24625) 6059 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:47:22.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.671863+0000 mon.a (mon.0) 2268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.672682+0000 mgr.gibba032.whopeo (mgr.24625) 6060 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:47:22.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.673245+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.673973+0000 mgr.gibba032.whopeo (mgr.24625) 6061 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:47:22.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.674034+0000 mgr.gibba032.whopeo (mgr.24625) 6062 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:47:22.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.675437+0000 mgr.gibba032.whopeo (mgr.24625) 6063 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:47:22.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.675785+0000 mgr.gibba032.whopeo (mgr.24625) 6064 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:47:22.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.675910+0000 mon.a (mon.0) 2270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.676944+0000 mgr.gibba032.whopeo (mgr.24625) 6065 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:47:22.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.677019+0000 mgr.gibba032.whopeo (mgr.24625) 6066 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:47:22.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: cephadm 2021-03-10T15:47:20.677141+0000 mgr.gibba032.whopeo (mgr.24625) 6067 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:22.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:21 gibba032 bash[24004]: audit 2021-03-10T15:47:20.677528+0000 mon.a (mon.0) 2271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.437652+0000 mgr.gibba032.whopeo (mgr.24625) 6052 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:47:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.439221+0000 mgr.gibba032.whopeo (mgr.24625) 6053 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:47:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.665894+0000 mon.a (mon.0) 2265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:47:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.666947+0000 mgr.gibba032.whopeo (mgr.24625) 6054 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:47:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.668746+0000 mgr.gibba032.whopeo (mgr.24625) 6055 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:47:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.669106+0000 mon.a (mon.0) 2266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:47:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.670032+0000 mgr.gibba032.whopeo (mgr.24625) 6056 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:47:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.670094+0000 mgr.gibba032.whopeo (mgr.24625) 6057 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:47:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.670482+0000 mon.a (mon.0) 2267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.671465+0000 mgr.gibba032.whopeo (mgr.24625) 6058 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:47:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.671525+0000 mgr.gibba032.whopeo (mgr.24625) 6059 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:47:22.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.671863+0000 mon.a (mon.0) 2268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.672682+0000 mgr.gibba032.whopeo (mgr.24625) 6060 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:47:22.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.673245+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.673973+0000 mgr.gibba032.whopeo (mgr.24625) 6061 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:47:22.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.674034+0000 mgr.gibba032.whopeo (mgr.24625) 6062 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:47:22.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.675437+0000 mgr.gibba032.whopeo (mgr.24625) 6063 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:47:22.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.675785+0000 mgr.gibba032.whopeo (mgr.24625) 6064 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:47:22.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.675910+0000 mon.a (mon.0) 2270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.676944+0000 mgr.gibba032.whopeo (mgr.24625) 6065 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:47:22.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.677019+0000 mgr.gibba032.whopeo (mgr.24625) 6066 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:47:22.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: cephadm 2021-03-10T15:47:20.677141+0000 mgr.gibba032.whopeo (mgr.24625) 6067 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:22.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[30791]: audit 2021-03-10T15:47:20.677528+0000 mon.a (mon.0) 2271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.437652+0000 mgr.gibba032.whopeo (mgr.24625) 6052 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:47:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.439221+0000 mgr.gibba032.whopeo (mgr.24625) 6053 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:47:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.665894+0000 mon.a (mon.0) 2265 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:47:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.666947+0000 mgr.gibba032.whopeo (mgr.24625) 6054 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:47:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.668746+0000 mgr.gibba032.whopeo (mgr.24625) 6055 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:47:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.669106+0000 mon.a (mon.0) 2266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:47:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.670032+0000 mgr.gibba032.whopeo (mgr.24625) 6056 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:47:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.670094+0000 mgr.gibba032.whopeo (mgr.24625) 6057 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:47:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.670482+0000 mon.a (mon.0) 2267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.671465+0000 mgr.gibba032.whopeo (mgr.24625) 6058 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:47:22.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.671525+0000 mgr.gibba032.whopeo (mgr.24625) 6059 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:47:22.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.671863+0000 mon.a (mon.0) 2268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.672682+0000 mgr.gibba032.whopeo (mgr.24625) 6060 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:47:22.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.673245+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.673973+0000 mgr.gibba032.whopeo (mgr.24625) 6061 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:47:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.674034+0000 mgr.gibba032.whopeo (mgr.24625) 6062 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:47:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.675437+0000 mgr.gibba032.whopeo (mgr.24625) 6063 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:47:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.675785+0000 mgr.gibba032.whopeo (mgr.24625) 6064 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:47:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.675910+0000 mon.a (mon.0) 2270 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.676944+0000 mgr.gibba032.whopeo (mgr.24625) 6065 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:47:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.677019+0000 mgr.gibba032.whopeo (mgr.24625) 6066 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:47:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: cephadm 2021-03-10T15:47:20.677141+0000 mgr.gibba032.whopeo (mgr.24625) 6067 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:21 gibba011 bash[31774]: audit 2021-03-10T15:47:20.677528+0000 mon.a (mon.0) 2271 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:22 gibba032 bash[24004]: cluster 2021-03-10T15:47:21.454594+0000 mgr.gibba032.whopeo (mgr.24625) 6068 : cluster [DBG] pgmap v3445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:23.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:22 gibba011 bash[30791]: cluster 2021-03-10T15:47:21.454594+0000 mgr.gibba032.whopeo (mgr.24625) 6068 : cluster [DBG] pgmap v3445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:23.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:22 gibba011 bash[31774]: cluster 2021-03-10T15:47:21.454594+0000 mgr.gibba032.whopeo (mgr.24625) 6068 : cluster [DBG] pgmap v3445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:24 gibba032 bash[24004]: cluster 2021-03-10T15:47:23.456223+0000 mgr.gibba032.whopeo (mgr.24625) 6069 : cluster [DBG] pgmap v3446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:24 gibba011 bash[30791]: cluster 2021-03-10T15:47:23.456223+0000 mgr.gibba032.whopeo (mgr.24625) 6069 : cluster [DBG] pgmap v3446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:24 gibba011 bash[31774]: cluster 2021-03-10T15:47:23.456223+0000 mgr.gibba032.whopeo (mgr.24625) 6069 : cluster [DBG] pgmap v3446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:26 gibba011 bash[30791]: cluster 2021-03-10T15:47:25.457750+0000 mgr.gibba032.whopeo (mgr.24625) 6070 : cluster [DBG] pgmap v3447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:26 gibba011 bash[31774]: cluster 2021-03-10T15:47:25.457750+0000 mgr.gibba032.whopeo (mgr.24625) 6070 : cluster [DBG] pgmap v3447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:26 gibba032 bash[24004]: cluster 2021-03-10T15:47:25.457750+0000 mgr.gibba032.whopeo (mgr.24625) 6070 : cluster [DBG] pgmap v3447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:28.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:28 gibba011 bash[30791]: cluster 2021-03-10T15:47:27.459368+0000 mgr.gibba032.whopeo (mgr.24625) 6071 : cluster [DBG] pgmap v3448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:28.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:28 gibba011 bash[31774]: cluster 2021-03-10T15:47:27.459368+0000 mgr.gibba032.whopeo (mgr.24625) 6071 : cluster [DBG] pgmap v3448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:28 gibba032 bash[24004]: cluster 2021-03-10T15:47:27.459368+0000 mgr.gibba032.whopeo (mgr.24625) 6071 : cluster [DBG] pgmap v3448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:30 gibba011 bash[30791]: cluster 2021-03-10T15:47:29.460968+0000 mgr.gibba032.whopeo (mgr.24625) 6072 : cluster [DBG] pgmap v3449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:30 gibba011 bash[31774]: cluster 2021-03-10T15:47:29.460968+0000 mgr.gibba032.whopeo (mgr.24625) 6072 : cluster [DBG] pgmap v3449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:30 gibba032 bash[24004]: cluster 2021-03-10T15:47:29.460968+0000 mgr.gibba032.whopeo (mgr.24625) 6072 : cluster [DBG] pgmap v3449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:32 gibba011 bash[30791]: cluster 2021-03-10T15:47:31.462417+0000 mgr.gibba032.whopeo (mgr.24625) 6073 : cluster [DBG] pgmap v3450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:32 gibba011 bash[31774]: cluster 2021-03-10T15:47:31.462417+0000 mgr.gibba032.whopeo (mgr.24625) 6073 : cluster [DBG] pgmap v3450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:32 gibba032 bash[24004]: cluster 2021-03-10T15:47:31.462417+0000 mgr.gibba032.whopeo (mgr.24625) 6073 : cluster [DBG] pgmap v3450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:34 gibba011 bash[30791]: cluster 2021-03-10T15:47:33.463971+0000 mgr.gibba032.whopeo (mgr.24625) 6074 : cluster [DBG] pgmap v3451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:34 gibba011 bash[31774]: cluster 2021-03-10T15:47:33.463971+0000 mgr.gibba032.whopeo (mgr.24625) 6074 : cluster [DBG] pgmap v3451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:34 gibba032 bash[24004]: cluster 2021-03-10T15:47:33.463971+0000 mgr.gibba032.whopeo (mgr.24625) 6074 : cluster [DBG] pgmap v3451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:35.932 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:47:36.158 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:47:36.158 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (114m) 8m ago 119m 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:47:36.158 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (119m) 7m ago 119m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:47:36.158 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (114m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:47:36.159 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (114m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:47:36.159 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (113m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:47:36.159 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (113m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:47:36.159 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (113m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:47:36.160 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (113m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:47:36.160 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (119m) 8m ago 119m 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:47:36.160 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (119m) 7m ago 119m 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:47:36.160 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (112m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:47:36.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (112m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:47:36.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:47:36.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:47:36.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:47:36.161 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:47:36.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (109m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:47:36.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (109m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:47:36.162 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (119m) 7m ago 119m 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:47:36.162 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 117m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:47:36.406 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:47:36.406 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:47:36.407 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:47:36.407 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:36.407 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:47:36.407 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:47:36.408 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:36.408 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:47:36.408 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:47:36.408 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:47:36.409 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:47:36.409 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:47:36.409 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:47:36.409 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:47:36.409 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:47:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[30791]: cluster 2021-03-10T15:47:35.465612+0000 mgr.gibba032.whopeo (mgr.24625) 6075 : cluster [DBG] pgmap v3452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[30791]: cephadm 2021-03-10T15:47:35.692782+0000 mgr.gibba032.whopeo (mgr.24625) 6076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[30791]: audit 2021-03-10T15:47:36.404938+0000 mon.c (mon.1) 116 : audit [DBG] from='client.? 172.21.2.111:0/4115049373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[31774]: cluster 2021-03-10T15:47:35.465612+0000 mgr.gibba032.whopeo (mgr.24625) 6075 : cluster [DBG] pgmap v3452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[31774]: cephadm 2021-03-10T15:47:35.692782+0000 mgr.gibba032.whopeo (mgr.24625) 6076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:36.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:36 gibba011 bash[31774]: audit 2021-03-10T15:47:36.404938+0000 mon.c (mon.1) 116 : audit [DBG] from='client.? 172.21.2.111:0/4115049373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:36 gibba032 bash[24004]: cluster 2021-03-10T15:47:35.465612+0000 mgr.gibba032.whopeo (mgr.24625) 6075 : cluster [DBG] pgmap v3452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:36 gibba032 bash[24004]: cephadm 2021-03-10T15:47:35.692782+0000 mgr.gibba032.whopeo (mgr.24625) 6076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:36 gibba032 bash[24004]: audit 2021-03-10T15:47:36.404938+0000 mon.c (mon.1) 116 : audit [DBG] from='client.? 172.21.2.111:0/4115049373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:47:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:37 gibba011 bash[30791]: audit 2021-03-10T15:47:35.919932+0000 mgr.gibba032.whopeo (mgr.24625) 6077 : audit [DBG] from='client.48123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:37 gibba011 bash[30791]: audit 2021-03-10T15:47:36.143290+0000 mgr.gibba032.whopeo (mgr.24625) 6078 : audit [DBG] from='client.48129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:37 gibba011 bash[31774]: audit 2021-03-10T15:47:35.919932+0000 mgr.gibba032.whopeo (mgr.24625) 6077 : audit [DBG] from='client.48123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:37 gibba011 bash[31774]: audit 2021-03-10T15:47:36.143290+0000 mgr.gibba032.whopeo (mgr.24625) 6078 : audit [DBG] from='client.48129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:38.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:37 gibba032 bash[24004]: audit 2021-03-10T15:47:35.919932+0000 mgr.gibba032.whopeo (mgr.24625) 6077 : audit [DBG] from='client.48123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:38.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:37 gibba032 bash[24004]: audit 2021-03-10T15:47:36.143290+0000 mgr.gibba032.whopeo (mgr.24625) 6078 : audit [DBG] from='client.48129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:47:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:38 gibba011 bash[30791]: cluster 2021-03-10T15:47:37.467266+0000 mgr.gibba032.whopeo (mgr.24625) 6079 : cluster [DBG] pgmap v3453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:38 gibba011 bash[31774]: cluster 2021-03-10T15:47:37.467266+0000 mgr.gibba032.whopeo (mgr.24625) 6079 : cluster [DBG] pgmap v3453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:38 gibba032 bash[24004]: cluster 2021-03-10T15:47:37.467266+0000 mgr.gibba032.whopeo (mgr.24625) 6079 : cluster [DBG] pgmap v3453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:40 gibba011 bash[30791]: cluster 2021-03-10T15:47:39.468788+0000 mgr.gibba032.whopeo (mgr.24625) 6080 : cluster [DBG] pgmap v3454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:40 gibba011 bash[31774]: cluster 2021-03-10T15:47:39.468788+0000 mgr.gibba032.whopeo (mgr.24625) 6080 : cluster [DBG] pgmap v3454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:40 gibba032 bash[24004]: cluster 2021-03-10T15:47:39.468788+0000 mgr.gibba032.whopeo (mgr.24625) 6080 : cluster [DBG] pgmap v3454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:42 gibba011 bash[30791]: cluster 2021-03-10T15:47:41.470279+0000 mgr.gibba032.whopeo (mgr.24625) 6081 : cluster [DBG] pgmap v3455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:42 gibba011 bash[31774]: cluster 2021-03-10T15:47:41.470279+0000 mgr.gibba032.whopeo (mgr.24625) 6081 : cluster [DBG] pgmap v3455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:42 gibba032 bash[24004]: cluster 2021-03-10T15:47:41.470279+0000 mgr.gibba032.whopeo (mgr.24625) 6081 : cluster [DBG] pgmap v3455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:44 gibba011 bash[30791]: cluster 2021-03-10T15:47:43.471748+0000 mgr.gibba032.whopeo (mgr.24625) 6082 : cluster [DBG] pgmap v3456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:44 gibba011 bash[31774]: cluster 2021-03-10T15:47:43.471748+0000 mgr.gibba032.whopeo (mgr.24625) 6082 : cluster [DBG] pgmap v3456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:44 gibba032 bash[24004]: cluster 2021-03-10T15:47:43.471748+0000 mgr.gibba032.whopeo (mgr.24625) 6082 : cluster [DBG] pgmap v3456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:46 gibba011 bash[30791]: cluster 2021-03-10T15:47:45.473227+0000 mgr.gibba032.whopeo (mgr.24625) 6083 : cluster [DBG] pgmap v3457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:46 gibba011 bash[31774]: cluster 2021-03-10T15:47:45.473227+0000 mgr.gibba032.whopeo (mgr.24625) 6083 : cluster [DBG] pgmap v3457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:46 gibba032 bash[24004]: cluster 2021-03-10T15:47:45.473227+0000 mgr.gibba032.whopeo (mgr.24625) 6083 : cluster [DBG] pgmap v3457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:48 gibba011 bash[30791]: cluster 2021-03-10T15:47:47.474970+0000 mgr.gibba032.whopeo (mgr.24625) 6084 : cluster [DBG] pgmap v3458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:48 gibba011 bash[31774]: cluster 2021-03-10T15:47:47.474970+0000 mgr.gibba032.whopeo (mgr.24625) 6084 : cluster [DBG] pgmap v3458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:48 gibba032 bash[24004]: cluster 2021-03-10T15:47:47.474970+0000 mgr.gibba032.whopeo (mgr.24625) 6084 : cluster [DBG] pgmap v3458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:50.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:50 gibba011 bash[30791]: cluster 2021-03-10T15:47:49.476507+0000 mgr.gibba032.whopeo (mgr.24625) 6085 : cluster [DBG] pgmap v3459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:50 gibba011 bash[31774]: cluster 2021-03-10T15:47:49.476507+0000 mgr.gibba032.whopeo (mgr.24625) 6085 : cluster [DBG] pgmap v3459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:50 gibba032 bash[24004]: cluster 2021-03-10T15:47:49.476507+0000 mgr.gibba032.whopeo (mgr.24625) 6085 : cluster [DBG] pgmap v3459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:51 gibba011 bash[30791]: cephadm 2021-03-10T15:47:50.708111+0000 mgr.gibba032.whopeo (mgr.24625) 6086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:51.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:51 gibba011 bash[31774]: cephadm 2021-03-10T15:47:50.708111+0000 mgr.gibba032.whopeo (mgr.24625) 6086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:52.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:51 gibba032 bash[24004]: cephadm 2021-03-10T15:47:50.708111+0000 mgr.gibba032.whopeo (mgr.24625) 6086 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:47:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:52 gibba011 bash[30791]: cluster 2021-03-10T15:47:51.478145+0000 mgr.gibba032.whopeo (mgr.24625) 6087 : cluster [DBG] pgmap v3460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:52 gibba011 bash[31774]: cluster 2021-03-10T15:47:51.478145+0000 mgr.gibba032.whopeo (mgr.24625) 6087 : cluster [DBG] pgmap v3460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:52 gibba032 bash[24004]: cluster 2021-03-10T15:47:51.478145+0000 mgr.gibba032.whopeo (mgr.24625) 6087 : cluster [DBG] pgmap v3460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:54 gibba011 bash[30791]: cluster 2021-03-10T15:47:53.479718+0000 mgr.gibba032.whopeo (mgr.24625) 6088 : cluster [DBG] pgmap v3461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:54 gibba011 bash[31774]: cluster 2021-03-10T15:47:53.479718+0000 mgr.gibba032.whopeo (mgr.24625) 6088 : cluster [DBG] pgmap v3461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:54 gibba032 bash[24004]: cluster 2021-03-10T15:47:53.479718+0000 mgr.gibba032.whopeo (mgr.24625) 6088 : cluster [DBG] pgmap v3461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:56 gibba011 bash[30791]: cluster 2021-03-10T15:47:55.481066+0000 mgr.gibba032.whopeo (mgr.24625) 6089 : cluster [DBG] pgmap v3462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:56 gibba011 bash[31774]: cluster 2021-03-10T15:47:55.481066+0000 mgr.gibba032.whopeo (mgr.24625) 6089 : cluster [DBG] pgmap v3462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:56 gibba032 bash[24004]: cluster 2021-03-10T15:47:55.481066+0000 mgr.gibba032.whopeo (mgr.24625) 6089 : cluster [DBG] pgmap v3462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.718264+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:47:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.718924+0000 mon.a (mon.0) 2273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:47:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.743806+0000 mon.a (mon.0) 2274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:47:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.744556+0000 mon.a (mon.0) 2275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:47:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.745229+0000 mon.a (mon.0) 2276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:47:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[30791]: audit 2021-03-10T15:47:56.745909+0000 mon.a (mon.0) 2277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:47:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.718264+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:47:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.718924+0000 mon.a (mon.0) 2273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:47:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.743806+0000 mon.a (mon.0) 2274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:47:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.744556+0000 mon.a (mon.0) 2275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:47:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.745229+0000 mon.a (mon.0) 2276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:47:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:57 gibba011 bash[31774]: audit 2021-03-10T15:47:56.745909+0000 mon.a (mon.0) 2277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:47:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.718264+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:47:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.718924+0000 mon.a (mon.0) 2273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:47:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.743806+0000 mon.a (mon.0) 2274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:47:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.744556+0000 mon.a (mon.0) 2275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:47:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.745229+0000 mon.a (mon.0) 2276 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T15:47:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:57 gibba032 bash[24004]: audit 2021-03-10T15:47:56.745909+0000 mon.a (mon.0) 2277 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T15:47:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:47:58 gibba011 bash[30791]: cluster 2021-03-10T15:47:57.482789+0000 mgr.gibba032.whopeo (mgr.24625) 6090 : cluster [DBG] pgmap v3463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:47:58 gibba011 bash[31774]: cluster 2021-03-10T15:47:57.482789+0000 mgr.gibba032.whopeo (mgr.24625) 6090 : cluster [DBG] pgmap v3463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:47:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:47:58 gibba032 bash[24004]: cluster 2021-03-10T15:47:57.482789+0000 mgr.gibba032.whopeo (mgr.24625) 6090 : cluster [DBG] pgmap v3463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:00.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:00 gibba011 bash[30791]: cluster 2021-03-10T15:47:59.484347+0000 mgr.gibba032.whopeo (mgr.24625) 6091 : cluster [DBG] pgmap v3464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:00 gibba011 bash[31774]: cluster 2021-03-10T15:47:59.484347+0000 mgr.gibba032.whopeo (mgr.24625) 6091 : cluster [DBG] pgmap v3464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:00 gibba032 bash[24004]: cluster 2021-03-10T15:47:59.484347+0000 mgr.gibba032.whopeo (mgr.24625) 6091 : cluster [DBG] pgmap v3464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:02 gibba032 bash[24004]: cluster 2021-03-10T15:48:01.484886+0000 mgr.gibba032.whopeo (mgr.24625) 6092 : cluster [DBG] pgmap v3465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:02 gibba011 bash[30791]: cluster 2021-03-10T15:48:01.484886+0000 mgr.gibba032.whopeo (mgr.24625) 6092 : cluster [DBG] pgmap v3465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:02 gibba011 bash[31774]: cluster 2021-03-10T15:48:01.484886+0000 mgr.gibba032.whopeo (mgr.24625) 6092 : cluster [DBG] pgmap v3465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:04 gibba011 bash[30791]: cluster 2021-03-10T15:48:03.486302+0000 mgr.gibba032.whopeo (mgr.24625) 6093 : cluster [DBG] pgmap v3466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:04 gibba011 bash[31774]: cluster 2021-03-10T15:48:03.486302+0000 mgr.gibba032.whopeo (mgr.24625) 6093 : cluster [DBG] pgmap v3466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:04 gibba032 bash[24004]: cluster 2021-03-10T15:48:03.486302+0000 mgr.gibba032.whopeo (mgr.24625) 6093 : cluster [DBG] pgmap v3466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:06.648 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:48:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:06 gibba011 bash[30791]: cluster 2021-03-10T15:48:05.487962+0000 mgr.gibba032.whopeo (mgr.24625) 6094 : cluster [DBG] pgmap v3467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:06 gibba011 bash[30791]: cephadm 2021-03-10T15:48:05.719877+0000 mgr.gibba032.whopeo (mgr.24625) 6095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:06 gibba011 bash[31774]: cluster 2021-03-10T15:48:05.487962+0000 mgr.gibba032.whopeo (mgr.24625) 6094 : cluster [DBG] pgmap v3467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:06 gibba011 bash[31774]: cephadm 2021-03-10T15:48:05.719877+0000 mgr.gibba032.whopeo (mgr.24625) 6095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:06.874 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:48:06.874 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (114m) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:48:06.875 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (119m) 7m ago 119m 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:48:06.875 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (115m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:48:06.875 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (115m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:48:06.875 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (114m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:48:06.876 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (114m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:48:06.876 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (113m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:48:06.876 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (114m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:48:06.876 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:48:06.876 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:48:06.877 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (113m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:48:06.877 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (112m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:48:06.877 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (112m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:48:06.877 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:48:06.877 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (111m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:48:06.878 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:48:06.878 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:48:06.878 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (110m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:48:06.878 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (119m) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:48:06.879 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 118m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:48:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:06 gibba032 bash[24004]: cluster 2021-03-10T15:48:05.487962+0000 mgr.gibba032.whopeo (mgr.24625) 6094 : cluster [DBG] pgmap v3467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:07.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:06 gibba032 bash[24004]: cephadm 2021-03-10T15:48:05.719877+0000 mgr.gibba032.whopeo (mgr.24625) 6095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:07.123 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:48:07.123 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:48:07.124 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:48:07.124 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:07.124 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:48:07.124 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:48:07.125 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:07.125 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:48:07.125 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:48:07.125 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:07.125 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:48:07.126 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:48:07.126 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:48:07.126 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:48:07.126 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:48:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[30791]: audit 2021-03-10T15:48:06.636525+0000 mgr.gibba032.whopeo (mgr.24625) 6096 : audit [DBG] from='client.48141 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[30791]: audit 2021-03-10T15:48:06.859006+0000 mgr.gibba032.whopeo (mgr.24625) 6097 : audit [DBG] from='client.48147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[30791]: audit 2021-03-10T15:48:07.121835+0000 mon.a (mon.0) 2278 : audit [DBG] from='client.? 172.21.2.111:0/4155564100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:07.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[31774]: audit 2021-03-10T15:48:06.636525+0000 mgr.gibba032.whopeo (mgr.24625) 6096 : audit [DBG] from='client.48141 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:07.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[31774]: audit 2021-03-10T15:48:06.859006+0000 mgr.gibba032.whopeo (mgr.24625) 6097 : audit [DBG] from='client.48147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:07.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:07 gibba011 bash[31774]: audit 2021-03-10T15:48:07.121835+0000 mon.a (mon.0) 2278 : audit [DBG] from='client.? 172.21.2.111:0/4155564100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:07 gibba032 bash[24004]: audit 2021-03-10T15:48:06.636525+0000 mgr.gibba032.whopeo (mgr.24625) 6096 : audit [DBG] from='client.48141 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:08.130 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:07 gibba032 bash[24004]: audit 2021-03-10T15:48:06.859006+0000 mgr.gibba032.whopeo (mgr.24625) 6097 : audit [DBG] from='client.48147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:08.131 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:07 gibba032 bash[24004]: audit 2021-03-10T15:48:07.121835+0000 mon.a (mon.0) 2278 : audit [DBG] from='client.? 172.21.2.111:0/4155564100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:08 gibba011 bash[30791]: cluster 2021-03-10T15:48:07.488673+0000 mgr.gibba032.whopeo (mgr.24625) 6098 : cluster [DBG] pgmap v3468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:08 gibba011 bash[31774]: cluster 2021-03-10T15:48:07.488673+0000 mgr.gibba032.whopeo (mgr.24625) 6098 : cluster [DBG] pgmap v3468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:08 gibba032 bash[24004]: cluster 2021-03-10T15:48:07.488673+0000 mgr.gibba032.whopeo (mgr.24625) 6098 : cluster [DBG] pgmap v3468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:10 gibba011 bash[30791]: cluster 2021-03-10T15:48:09.490011+0000 mgr.gibba032.whopeo (mgr.24625) 6099 : cluster [DBG] pgmap v3469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:10 gibba011 bash[31774]: cluster 2021-03-10T15:48:09.490011+0000 mgr.gibba032.whopeo (mgr.24625) 6099 : cluster [DBG] pgmap v3469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:10 gibba032 bash[24004]: cluster 2021-03-10T15:48:09.490011+0000 mgr.gibba032.whopeo (mgr.24625) 6099 : cluster [DBG] pgmap v3469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:12 gibba032 bash[24004]: cluster 2021-03-10T15:48:11.491538+0000 mgr.gibba032.whopeo (mgr.24625) 6100 : cluster [DBG] pgmap v3470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:12 gibba011 bash[30791]: cluster 2021-03-10T15:48:11.491538+0000 mgr.gibba032.whopeo (mgr.24625) 6100 : cluster [DBG] pgmap v3470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:12 gibba011 bash[31774]: cluster 2021-03-10T15:48:11.491538+0000 mgr.gibba032.whopeo (mgr.24625) 6100 : cluster [DBG] pgmap v3470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:14 gibba011 bash[30791]: cluster 2021-03-10T15:48:13.492910+0000 mgr.gibba032.whopeo (mgr.24625) 6101 : cluster [DBG] pgmap v3471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:14 gibba011 bash[31774]: cluster 2021-03-10T15:48:13.492910+0000 mgr.gibba032.whopeo (mgr.24625) 6101 : cluster [DBG] pgmap v3471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:14 gibba032 bash[24004]: cluster 2021-03-10T15:48:13.492910+0000 mgr.gibba032.whopeo (mgr.24625) 6101 : cluster [DBG] pgmap v3471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:16 gibba011 bash[30791]: cluster 2021-03-10T15:48:15.494501+0000 mgr.gibba032.whopeo (mgr.24625) 6102 : cluster [DBG] pgmap v3472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:16 gibba011 bash[31774]: cluster 2021-03-10T15:48:15.494501+0000 mgr.gibba032.whopeo (mgr.24625) 6102 : cluster [DBG] pgmap v3472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:16 gibba032 bash[24004]: cluster 2021-03-10T15:48:15.494501+0000 mgr.gibba032.whopeo (mgr.24625) 6102 : cluster [DBG] pgmap v3472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:18.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:18 gibba011 bash[30791]: cluster 2021-03-10T15:48:17.496255+0000 mgr.gibba032.whopeo (mgr.24625) 6103 : cluster [DBG] pgmap v3473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:18 gibba011 bash[31774]: cluster 2021-03-10T15:48:17.496255+0000 mgr.gibba032.whopeo (mgr.24625) 6103 : cluster [DBG] pgmap v3473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:18 gibba032 bash[24004]: cluster 2021-03-10T15:48:17.496255+0000 mgr.gibba032.whopeo (mgr.24625) 6103 : cluster [DBG] pgmap v3473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:20 gibba011 bash[30791]: cluster 2021-03-10T15:48:19.496716+0000 mgr.gibba032.whopeo (mgr.24625) 6104 : cluster [DBG] pgmap v3474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:20 gibba011 bash[31774]: cluster 2021-03-10T15:48:19.496716+0000 mgr.gibba032.whopeo (mgr.24625) 6104 : cluster [DBG] pgmap v3474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:20 gibba032 bash[24004]: cluster 2021-03-10T15:48:19.496716+0000 mgr.gibba032.whopeo (mgr.24625) 6104 : cluster [DBG] pgmap v3474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:21.851 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:21 gibba011 bash[30791]: cephadm 2021-03-10T15:48:20.734370+0000 mgr.gibba032.whopeo (mgr.24625) 6105 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:48:21.852 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:21 gibba011 bash[30791]: audit 2021-03-10T15:48:20.736516+0000 mon.a (mon.0) 2279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:48:21.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:21 gibba011 bash[31774]: cephadm 2021-03-10T15:48:20.734370+0000 mgr.gibba032.whopeo (mgr.24625) 6105 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:48:21.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:21 gibba011 bash[31774]: audit 2021-03-10T15:48:20.736516+0000 mon.a (mon.0) 2279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:48:22.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:21 gibba032 bash[24004]: cephadm 2021-03-10T15:48:20.734370+0000 mgr.gibba032.whopeo (mgr.24625) 6105 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:48:22.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:21 gibba032 bash[24004]: audit 2021-03-10T15:48:20.736516+0000 mon.a (mon.0) 2279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:48:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:22 gibba032 bash[24004]: cluster 2021-03-10T15:48:21.498086+0000 mgr.gibba032.whopeo (mgr.24625) 6106 : cluster [DBG] pgmap v3475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:23.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:22 gibba011 bash[30791]: cluster 2021-03-10T15:48:21.498086+0000 mgr.gibba032.whopeo (mgr.24625) 6106 : cluster [DBG] pgmap v3475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:23.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:22 gibba011 bash[31774]: cluster 2021-03-10T15:48:21.498086+0000 mgr.gibba032.whopeo (mgr.24625) 6106 : cluster [DBG] pgmap v3475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:24 gibba011 bash[30791]: cluster 2021-03-10T15:48:23.499639+0000 mgr.gibba032.whopeo (mgr.24625) 6107 : cluster [DBG] pgmap v3476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:24 gibba011 bash[31774]: cluster 2021-03-10T15:48:23.499639+0000 mgr.gibba032.whopeo (mgr.24625) 6107 : cluster [DBG] pgmap v3476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:24 gibba032 bash[24004]: cluster 2021-03-10T15:48:23.499639+0000 mgr.gibba032.whopeo (mgr.24625) 6107 : cluster [DBG] pgmap v3476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:25.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:25 gibba032 bash[24004]: audit 2021-03-10T15:48:25.446037+0000 mon.a (mon.0) 2280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:48:25.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:25 gibba011 bash[30791]: audit 2021-03-10T15:48:25.446037+0000 mon.a (mon.0) 2280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:48:25.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:25 gibba011 bash[31774]: audit 2021-03-10T15:48:25.446037+0000 mon.a (mon.0) 2280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:48:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: cluster 2021-03-10T15:48:25.500273+0000 mgr.gibba032.whopeo (mgr.24625) 6108 : cluster [DBG] pgmap v3477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.625565+0000 mon.a (mon.0) 2281 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.625753+0000 mgr.gibba032.whopeo (mgr.24625) 6109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:48:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.858882+0000 mon.a (mon.0) 2282 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.870402+0000 mgr.gibba032.whopeo (mgr.24625) 6110 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:48:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.874465+0000 mon.a (mon.0) 2283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:48:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.878638+0000 mon.a (mon.0) 2284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.882287+0000 mon.a (mon.0) 2285 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.885675+0000 mon.a (mon.0) 2286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.892450+0000 mon.a (mon.0) 2287 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:26 gibba032 bash[24004]: audit 2021-03-10T15:48:25.897965+0000 mon.a (mon.0) 2288 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: cluster 2021-03-10T15:48:25.500273+0000 mgr.gibba032.whopeo (mgr.24625) 6108 : cluster [DBG] pgmap v3477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.625565+0000 mon.a (mon.0) 2281 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.625753+0000 mgr.gibba032.whopeo (mgr.24625) 6109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:48:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.858882+0000 mon.a (mon.0) 2282 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.870402+0000 mgr.gibba032.whopeo (mgr.24625) 6110 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:48:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.874465+0000 mon.a (mon.0) 2283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:48:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.878638+0000 mon.a (mon.0) 2284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.882287+0000 mon.a (mon.0) 2285 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.885675+0000 mon.a (mon.0) 2286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.892450+0000 mon.a (mon.0) 2287 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[30791]: audit 2021-03-10T15:48:25.897965+0000 mon.a (mon.0) 2288 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: cluster 2021-03-10T15:48:25.500273+0000 mgr.gibba032.whopeo (mgr.24625) 6108 : cluster [DBG] pgmap v3477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:27.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.625565+0000 mon.a (mon.0) 2281 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.625753+0000 mgr.gibba032.whopeo (mgr.24625) 6109 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:48:27.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.858882+0000 mon.a (mon.0) 2282 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:48:27.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.870402+0000 mgr.gibba032.whopeo (mgr.24625) 6110 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:48:27.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.874465+0000 mon.a (mon.0) 2283 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:48:27.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.878638+0000 mon.a (mon.0) 2284 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.882287+0000 mon.a (mon.0) 2285 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.885675+0000 mon.a (mon.0) 2286 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.892450+0000 mon.a (mon.0) 2287 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:27.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:26 gibba011 bash[31774]: audit 2021-03-10T15:48:25.897965+0000 mon.a (mon.0) 2288 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:28.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.875706+0000 mgr.gibba032.whopeo (mgr.24625) 6111 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:48:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.879155+0000 mgr.gibba032.whopeo (mgr.24625) 6112 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:48:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.879699+0000 mgr.gibba032.whopeo (mgr.24625) 6113 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:48:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.883208+0000 mgr.gibba032.whopeo (mgr.24625) 6114 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:48:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.883437+0000 mgr.gibba032.whopeo (mgr.24625) 6115 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:48:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.886357+0000 mgr.gibba032.whopeo (mgr.24625) 6116 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:48:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.889241+0000 mgr.gibba032.whopeo (mgr.24625) 6117 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:48:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.889447+0000 mgr.gibba032.whopeo (mgr.24625) 6118 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:48:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.893555+0000 mgr.gibba032.whopeo (mgr.24625) 6119 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:48:28.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.895467+0000 mgr.gibba032.whopeo (mgr.24625) 6120 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:48:28.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.898715+0000 mgr.gibba032.whopeo (mgr.24625) 6121 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:48:28.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.899182+0000 mgr.gibba032.whopeo (mgr.24625) 6122 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:48:28.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:27 gibba032 bash[24004]: cephadm 2021-03-10T15:48:25.899890+0000 mgr.gibba032.whopeo (mgr.24625) 6123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.875706+0000 mgr.gibba032.whopeo (mgr.24625) 6111 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:48:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.879155+0000 mgr.gibba032.whopeo (mgr.24625) 6112 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:48:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.879699+0000 mgr.gibba032.whopeo (mgr.24625) 6113 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:48:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.883208+0000 mgr.gibba032.whopeo (mgr.24625) 6114 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:48:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.883437+0000 mgr.gibba032.whopeo (mgr.24625) 6115 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:48:28.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.886357+0000 mgr.gibba032.whopeo (mgr.24625) 6116 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:48:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.889241+0000 mgr.gibba032.whopeo (mgr.24625) 6117 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:48:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.889447+0000 mgr.gibba032.whopeo (mgr.24625) 6118 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:48:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.893555+0000 mgr.gibba032.whopeo (mgr.24625) 6119 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:48:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.895467+0000 mgr.gibba032.whopeo (mgr.24625) 6120 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:48:28.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.898715+0000 mgr.gibba032.whopeo (mgr.24625) 6121 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:48:28.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.899182+0000 mgr.gibba032.whopeo (mgr.24625) 6122 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:48:28.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[30791]: cephadm 2021-03-10T15:48:25.899890+0000 mgr.gibba032.whopeo (mgr.24625) 6123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:28.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.875706+0000 mgr.gibba032.whopeo (mgr.24625) 6111 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:48:28.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.879155+0000 mgr.gibba032.whopeo (mgr.24625) 6112 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.879699+0000 mgr.gibba032.whopeo (mgr.24625) 6113 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.883208+0000 mgr.gibba032.whopeo (mgr.24625) 6114 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.883437+0000 mgr.gibba032.whopeo (mgr.24625) 6115 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.886357+0000 mgr.gibba032.whopeo (mgr.24625) 6116 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.889241+0000 mgr.gibba032.whopeo (mgr.24625) 6117 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:48:28.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.889447+0000 mgr.gibba032.whopeo (mgr.24625) 6118 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:48:28.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.893555+0000 mgr.gibba032.whopeo (mgr.24625) 6119 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:48:28.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.895467+0000 mgr.gibba032.whopeo (mgr.24625) 6120 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:48:28.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.898715+0000 mgr.gibba032.whopeo (mgr.24625) 6121 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:48:28.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.899182+0000 mgr.gibba032.whopeo (mgr.24625) 6122 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:48:28.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:27 gibba011 bash[31774]: cephadm 2021-03-10T15:48:25.899890+0000 mgr.gibba032.whopeo (mgr.24625) 6123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:28 gibba032 bash[24004]: cluster 2021-03-10T15:48:27.501613+0000 mgr.gibba032.whopeo (mgr.24625) 6124 : cluster [DBG] pgmap v3478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:29.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:28 gibba011 bash[30791]: cluster 2021-03-10T15:48:27.501613+0000 mgr.gibba032.whopeo (mgr.24625) 6124 : cluster [DBG] pgmap v3478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:28 gibba011 bash[31774]: cluster 2021-03-10T15:48:27.501613+0000 mgr.gibba032.whopeo (mgr.24625) 6124 : cluster [DBG] pgmap v3478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:30 gibba032 bash[24004]: cluster 2021-03-10T15:48:29.503086+0000 mgr.gibba032.whopeo (mgr.24625) 6125 : cluster [DBG] pgmap v3479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:30 gibba011 bash[30791]: cluster 2021-03-10T15:48:29.503086+0000 mgr.gibba032.whopeo (mgr.24625) 6125 : cluster [DBG] pgmap v3479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:30 gibba011 bash[31774]: cluster 2021-03-10T15:48:29.503086+0000 mgr.gibba032.whopeo (mgr.24625) 6125 : cluster [DBG] pgmap v3479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:32 gibba032 bash[24004]: cluster 2021-03-10T15:48:31.503904+0000 mgr.gibba032.whopeo (mgr.24625) 6126 : cluster [DBG] pgmap v3480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:33.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:32 gibba011 bash[30791]: cluster 2021-03-10T15:48:31.503904+0000 mgr.gibba032.whopeo (mgr.24625) 6126 : cluster [DBG] pgmap v3480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:32 gibba011 bash[31774]: cluster 2021-03-10T15:48:31.503904+0000 mgr.gibba032.whopeo (mgr.24625) 6126 : cluster [DBG] pgmap v3480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:34 gibba011 bash[30791]: cluster 2021-03-10T15:48:33.505294+0000 mgr.gibba032.whopeo (mgr.24625) 6127 : cluster [DBG] pgmap v3481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:34 gibba011 bash[31774]: cluster 2021-03-10T15:48:33.505294+0000 mgr.gibba032.whopeo (mgr.24625) 6127 : cluster [DBG] pgmap v3481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:34 gibba032 bash[24004]: cluster 2021-03-10T15:48:33.505294+0000 mgr.gibba032.whopeo (mgr.24625) 6127 : cluster [DBG] pgmap v3481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:36 gibba011 bash[30791]: cluster 2021-03-10T15:48:35.506935+0000 mgr.gibba032.whopeo (mgr.24625) 6128 : cluster [DBG] pgmap v3482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:36 gibba011 bash[31774]: cluster 2021-03-10T15:48:35.506935+0000 mgr.gibba032.whopeo (mgr.24625) 6128 : cluster [DBG] pgmap v3482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:36 gibba032 bash[24004]: cluster 2021-03-10T15:48:35.506935+0000 mgr.gibba032.whopeo (mgr.24625) 6128 : cluster [DBG] pgmap v3482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:37.366 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:48:37.588 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:48:37.588 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (115m) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:48:37.588 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:48:37.588 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (115m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:48:37.589 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (115m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:48:37.589 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (114m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:48:37.589 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (114m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:48:37.589 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (114m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:48:37.589 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (114m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:48:37.590 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:48:37.590 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:48:37.590 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (113m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:48:37.590 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (113m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:48:37.590 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:48:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:48:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:48:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:48:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:48:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (110m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:48:37.592 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:48:37.592 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 118m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:48:37.837 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:48:37.837 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:48:37.837 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:48:37.837 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:37.838 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:48:37.838 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:48:37.838 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:37.838 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:48:37.839 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:48:37.840 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:48:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[30791]: audit 2021-03-10T15:48:37.355313+0000 mgr.gibba032.whopeo (mgr.24625) 6129 : audit [DBG] from='client.48159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:38.892 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[30791]: cluster 2021-03-10T15:48:37.508433+0000 mgr.gibba032.whopeo (mgr.24625) 6130 : cluster [DBG] pgmap v3483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:38.892 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[30791]: audit 2021-03-10T15:48:37.577564+0000 mgr.gibba032.whopeo (mgr.24625) 6131 : audit [DBG] from='client.58147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:38.893 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[30791]: audit 2021-03-10T15:48:37.835692+0000 mon.c (mon.1) 117 : audit [DBG] from='client.? 172.21.2.111:0/1182446302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:38.893 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[31774]: audit 2021-03-10T15:48:37.355313+0000 mgr.gibba032.whopeo (mgr.24625) 6129 : audit [DBG] from='client.48159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:38.893 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[31774]: cluster 2021-03-10T15:48:37.508433+0000 mgr.gibba032.whopeo (mgr.24625) 6130 : cluster [DBG] pgmap v3483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:38.893 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[31774]: audit 2021-03-10T15:48:37.577564+0000 mgr.gibba032.whopeo (mgr.24625) 6131 : audit [DBG] from='client.58147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:38.894 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:38 gibba011 bash[31774]: audit 2021-03-10T15:48:37.835692+0000 mon.c (mon.1) 117 : audit [DBG] from='client.? 172.21.2.111:0/1182446302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:38 gibba032 bash[24004]: audit 2021-03-10T15:48:37.355313+0000 mgr.gibba032.whopeo (mgr.24625) 6129 : audit [DBG] from='client.48159 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:38 gibba032 bash[24004]: cluster 2021-03-10T15:48:37.508433+0000 mgr.gibba032.whopeo (mgr.24625) 6130 : cluster [DBG] pgmap v3483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:38 gibba032 bash[24004]: audit 2021-03-10T15:48:37.577564+0000 mgr.gibba032.whopeo (mgr.24625) 6131 : audit [DBG] from='client.58147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:48:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:38 gibba032 bash[24004]: audit 2021-03-10T15:48:37.835692+0000 mon.c (mon.1) 117 : audit [DBG] from='client.? 172.21.2.111:0/1182446302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:48:40.916 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:40 gibba032 bash[24004]: cluster 2021-03-10T15:48:39.509973+0000 mgr.gibba032.whopeo (mgr.24625) 6132 : cluster [DBG] pgmap v3484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:40 gibba011 bash[30791]: cluster 2021-03-10T15:48:39.509973+0000 mgr.gibba032.whopeo (mgr.24625) 6132 : cluster [DBG] pgmap v3484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:40 gibba011 bash[31774]: cluster 2021-03-10T15:48:39.509973+0000 mgr.gibba032.whopeo (mgr.24625) 6132 : cluster [DBG] pgmap v3484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:42 gibba032 bash[24004]: cephadm 2021-03-10T15:48:40.913298+0000 mgr.gibba032.whopeo (mgr.24625) 6133 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:42 gibba032 bash[24004]: cluster 2021-03-10T15:48:41.511605+0000 mgr.gibba032.whopeo (mgr.24625) 6134 : cluster [DBG] pgmap v3485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:42 gibba011 bash[30791]: cephadm 2021-03-10T15:48:40.913298+0000 mgr.gibba032.whopeo (mgr.24625) 6133 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:42 gibba011 bash[30791]: cluster 2021-03-10T15:48:41.511605+0000 mgr.gibba032.whopeo (mgr.24625) 6134 : cluster [DBG] pgmap v3485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:42 gibba011 bash[31774]: cephadm 2021-03-10T15:48:40.913298+0000 mgr.gibba032.whopeo (mgr.24625) 6133 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:42 gibba011 bash[31774]: cluster 2021-03-10T15:48:41.511605+0000 mgr.gibba032.whopeo (mgr.24625) 6134 : cluster [DBG] pgmap v3485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:44 gibba011 bash[30791]: cluster 2021-03-10T15:48:43.512333+0000 mgr.gibba032.whopeo (mgr.24625) 6135 : cluster [DBG] pgmap v3486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:44 gibba011 bash[31774]: cluster 2021-03-10T15:48:43.512333+0000 mgr.gibba032.whopeo (mgr.24625) 6135 : cluster [DBG] pgmap v3486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:44 gibba032 bash[24004]: cluster 2021-03-10T15:48:43.512333+0000 mgr.gibba032.whopeo (mgr.24625) 6135 : cluster [DBG] pgmap v3486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:46 gibba011 bash[30791]: cluster 2021-03-10T15:48:45.513620+0000 mgr.gibba032.whopeo (mgr.24625) 6136 : cluster [DBG] pgmap v3487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:46 gibba011 bash[31774]: cluster 2021-03-10T15:48:45.513620+0000 mgr.gibba032.whopeo (mgr.24625) 6136 : cluster [DBG] pgmap v3487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:46 gibba032 bash[24004]: cluster 2021-03-10T15:48:45.513620+0000 mgr.gibba032.whopeo (mgr.24625) 6136 : cluster [DBG] pgmap v3487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:48 gibba032 bash[24004]: cluster 2021-03-10T15:48:47.515433+0000 mgr.gibba032.whopeo (mgr.24625) 6137 : cluster [DBG] pgmap v3488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:48 gibba011 bash[30791]: cluster 2021-03-10T15:48:47.515433+0000 mgr.gibba032.whopeo (mgr.24625) 6137 : cluster [DBG] pgmap v3488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:48 gibba011 bash[31774]: cluster 2021-03-10T15:48:47.515433+0000 mgr.gibba032.whopeo (mgr.24625) 6137 : cluster [DBG] pgmap v3488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:50 gibba032 bash[24004]: cluster 2021-03-10T15:48:49.516197+0000 mgr.gibba032.whopeo (mgr.24625) 6138 : cluster [DBG] pgmap v3489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:50 gibba011 bash[30791]: cluster 2021-03-10T15:48:49.516197+0000 mgr.gibba032.whopeo (mgr.24625) 6138 : cluster [DBG] pgmap v3489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:50 gibba011 bash[31774]: cluster 2021-03-10T15:48:49.516197+0000 mgr.gibba032.whopeo (mgr.24625) 6138 : cluster [DBG] pgmap v3489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:52 gibba032 bash[24004]: cluster 2021-03-10T15:48:51.517872+0000 mgr.gibba032.whopeo (mgr.24625) 6139 : cluster [DBG] pgmap v3490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:52 gibba011 bash[31774]: cluster 2021-03-10T15:48:51.517872+0000 mgr.gibba032.whopeo (mgr.24625) 6139 : cluster [DBG] pgmap v3490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:53.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:52 gibba011 bash[30791]: cluster 2021-03-10T15:48:51.517872+0000 mgr.gibba032.whopeo (mgr.24625) 6139 : cluster [DBG] pgmap v3490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:54 gibba011 bash[30791]: cluster 2021-03-10T15:48:53.519348+0000 mgr.gibba032.whopeo (mgr.24625) 6140 : cluster [DBG] pgmap v3491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:54 gibba011 bash[31774]: cluster 2021-03-10T15:48:53.519348+0000 mgr.gibba032.whopeo (mgr.24625) 6140 : cluster [DBG] pgmap v3491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:54 gibba032 bash[24004]: cluster 2021-03-10T15:48:53.519348+0000 mgr.gibba032.whopeo (mgr.24625) 6140 : cluster [DBG] pgmap v3491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:56 gibba032 bash[24004]: cluster 2021-03-10T15:48:55.520311+0000 mgr.gibba032.whopeo (mgr.24625) 6141 : cluster [DBG] pgmap v3492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:56 gibba011 bash[30791]: cluster 2021-03-10T15:48:55.520311+0000 mgr.gibba032.whopeo (mgr.24625) 6141 : cluster [DBG] pgmap v3492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:56 gibba011 bash[31774]: cluster 2021-03-10T15:48:55.520311+0000 mgr.gibba032.whopeo (mgr.24625) 6141 : cluster [DBG] pgmap v3492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: cephadm 2021-03-10T15:48:55.928837+0000 mgr.gibba032.whopeo (mgr.24625) 6142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.722133+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:48:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.722809+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:48:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.751371+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:48:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.752196+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:48:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.752949+0000 mon.a (mon.0) 2293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:48:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:57 gibba032 bash[24004]: audit 2021-03-10T15:48:56.753717+0000 mon.a (mon.0) 2294 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:48:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: cephadm 2021-03-10T15:48:55.928837+0000 mgr.gibba032.whopeo (mgr.24625) 6142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.722133+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:48:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.722809+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:48:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.751371+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:48:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.752196+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:48:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.752949+0000 mon.a (mon.0) 2293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:48:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[30791]: audit 2021-03-10T15:48:56.753717+0000 mon.a (mon.0) 2294 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:48:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: cephadm 2021-03-10T15:48:55.928837+0000 mgr.gibba032.whopeo (mgr.24625) 6142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:48:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.722133+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:48:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.722809+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:48:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.751371+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:48:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.752196+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:48:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.752949+0000 mon.a (mon.0) 2293 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:48:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:57 gibba011 bash[31774]: audit 2021-03-10T15:48:56.753717+0000 mon.a (mon.0) 2294 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:48:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:48:58 gibba032 bash[24004]: cluster 2021-03-10T15:48:57.521954+0000 mgr.gibba032.whopeo (mgr.24625) 6143 : cluster [DBG] pgmap v3493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:48:58 gibba011 bash[30791]: cluster 2021-03-10T15:48:57.521954+0000 mgr.gibba032.whopeo (mgr.24625) 6143 : cluster [DBG] pgmap v3493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:48:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:48:58 gibba011 bash[31774]: cluster 2021-03-10T15:48:57.521954+0000 mgr.gibba032.whopeo (mgr.24625) 6143 : cluster [DBG] pgmap v3493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:00 gibba032 bash[24004]: cluster 2021-03-10T15:48:59.523325+0000 mgr.gibba032.whopeo (mgr.24625) 6144 : cluster [DBG] pgmap v3494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:00 gibba011 bash[30791]: cluster 2021-03-10T15:48:59.523325+0000 mgr.gibba032.whopeo (mgr.24625) 6144 : cluster [DBG] pgmap v3494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:00 gibba011 bash[31774]: cluster 2021-03-10T15:48:59.523325+0000 mgr.gibba032.whopeo (mgr.24625) 6144 : cluster [DBG] pgmap v3494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:02 gibba032 bash[24004]: cluster 2021-03-10T15:49:01.523952+0000 mgr.gibba032.whopeo (mgr.24625) 6145 : cluster [DBG] pgmap v3495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:02 gibba011 bash[30791]: cluster 2021-03-10T15:49:01.523952+0000 mgr.gibba032.whopeo (mgr.24625) 6145 : cluster [DBG] pgmap v3495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:02 gibba011 bash[31774]: cluster 2021-03-10T15:49:01.523952+0000 mgr.gibba032.whopeo (mgr.24625) 6145 : cluster [DBG] pgmap v3495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:04 gibba011 bash[30791]: cluster 2021-03-10T15:49:03.525303+0000 mgr.gibba032.whopeo (mgr.24625) 6146 : cluster [DBG] pgmap v3496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:04 gibba011 bash[31774]: cluster 2021-03-10T15:49:03.525303+0000 mgr.gibba032.whopeo (mgr.24625) 6146 : cluster [DBG] pgmap v3496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:04 gibba032 bash[24004]: cluster 2021-03-10T15:49:03.525303+0000 mgr.gibba032.whopeo (mgr.24625) 6146 : cluster [DBG] pgmap v3496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:06 gibba032 bash[24004]: cluster 2021-03-10T15:49:05.526882+0000 mgr.gibba032.whopeo (mgr.24625) 6147 : cluster [DBG] pgmap v3497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:06 gibba011 bash[30791]: cluster 2021-03-10T15:49:05.526882+0000 mgr.gibba032.whopeo (mgr.24625) 6147 : cluster [DBG] pgmap v3497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:06 gibba011 bash[31774]: cluster 2021-03-10T15:49:05.526882+0000 mgr.gibba032.whopeo (mgr.24625) 6147 : cluster [DBG] pgmap v3497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:08.079 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:49:08.304 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:49:08.305 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (115m) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:49:08.305 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:49:08.305 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (116m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:49:08.305 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (116m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:49:08.305 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (115m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:49:08.306 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (115m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:49:08.306 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (114m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:49:08.306 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (115m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:49:08.306 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:49:08.306 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:49:08.307 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (114m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:49:08.307 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (113m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:49:08.307 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (113m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:49:08.307 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:49:08.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (112m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:49:08.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:49:08.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:49:08.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (111m) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:49:08.308 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:49:08.309 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 119m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:49:08.558 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:49:08.558 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:49:08.559 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:49:08.559 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:08.559 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:49:08.559 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:49:08.559 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:08.560 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:49:08.560 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:49:08.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:08.560 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:49:08.561 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:49:08.561 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:49:08.561 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:49:08.561 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:49:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:08 gibba032 bash[24004]: cluster 2021-03-10T15:49:07.527851+0000 mgr.gibba032.whopeo (mgr.24625) 6148 : cluster [DBG] pgmap v3498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:08 gibba032 bash[24004]: audit 2021-03-10T15:49:08.557448+0000 mon.c (mon.1) 118 : audit [DBG] from='client.? 172.21.2.111:0/3446405049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:08 gibba011 bash[30791]: cluster 2021-03-10T15:49:07.527851+0000 mgr.gibba032.whopeo (mgr.24625) 6148 : cluster [DBG] pgmap v3498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:08 gibba011 bash[30791]: audit 2021-03-10T15:49:08.557448+0000 mon.c (mon.1) 118 : audit [DBG] from='client.? 172.21.2.111:0/3446405049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:09.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:08 gibba011 bash[31774]: cluster 2021-03-10T15:49:07.527851+0000 mgr.gibba032.whopeo (mgr.24625) 6148 : cluster [DBG] pgmap v3498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:09.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:08 gibba011 bash[31774]: audit 2021-03-10T15:49:08.557448+0000 mon.c (mon.1) 118 : audit [DBG] from='client.? 172.21.2.111:0/3446405049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:09 gibba032 bash[24004]: audit 2021-03-10T15:49:08.068592+0000 mgr.gibba032.whopeo (mgr.24625) 6149 : audit [DBG] from='client.48177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:09 gibba032 bash[24004]: audit 2021-03-10T15:49:08.292421+0000 mgr.gibba032.whopeo (mgr.24625) 6150 : audit [DBG] from='client.58165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:09 gibba011 bash[30791]: audit 2021-03-10T15:49:08.068592+0000 mgr.gibba032.whopeo (mgr.24625) 6149 : audit [DBG] from='client.48177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:09 gibba011 bash[30791]: audit 2021-03-10T15:49:08.292421+0000 mgr.gibba032.whopeo (mgr.24625) 6150 : audit [DBG] from='client.58165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:09 gibba011 bash[31774]: audit 2021-03-10T15:49:08.068592+0000 mgr.gibba032.whopeo (mgr.24625) 6149 : audit [DBG] from='client.48177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:09 gibba011 bash[31774]: audit 2021-03-10T15:49:08.292421+0000 mgr.gibba032.whopeo (mgr.24625) 6150 : audit [DBG] from='client.58165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:10.947 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:10 gibba032 bash[24004]: cluster 2021-03-10T15:49:09.529134+0000 mgr.gibba032.whopeo (mgr.24625) 6151 : cluster [DBG] pgmap v3499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:10 gibba011 bash[30791]: cluster 2021-03-10T15:49:09.529134+0000 mgr.gibba032.whopeo (mgr.24625) 6151 : cluster [DBG] pgmap v3499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:10 gibba011 bash[31774]: cluster 2021-03-10T15:49:09.529134+0000 mgr.gibba032.whopeo (mgr.24625) 6151 : cluster [DBG] pgmap v3499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:12 gibba032 bash[24004]: cephadm 2021-03-10T15:49:10.944690+0000 mgr.gibba032.whopeo (mgr.24625) 6152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:12 gibba032 bash[24004]: cluster 2021-03-10T15:49:11.530783+0000 mgr.gibba032.whopeo (mgr.24625) 6153 : cluster [DBG] pgmap v3500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:12 gibba011 bash[30791]: cephadm 2021-03-10T15:49:10.944690+0000 mgr.gibba032.whopeo (mgr.24625) 6152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:12 gibba011 bash[30791]: cluster 2021-03-10T15:49:11.530783+0000 mgr.gibba032.whopeo (mgr.24625) 6153 : cluster [DBG] pgmap v3500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:12 gibba011 bash[31774]: cephadm 2021-03-10T15:49:10.944690+0000 mgr.gibba032.whopeo (mgr.24625) 6152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:12 gibba011 bash[31774]: cluster 2021-03-10T15:49:11.530783+0000 mgr.gibba032.whopeo (mgr.24625) 6153 : cluster [DBG] pgmap v3500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:14 gibba011 bash[30791]: cluster 2021-03-10T15:49:13.531800+0000 mgr.gibba032.whopeo (mgr.24625) 6154 : cluster [DBG] pgmap v3501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:14 gibba011 bash[31774]: cluster 2021-03-10T15:49:13.531800+0000 mgr.gibba032.whopeo (mgr.24625) 6154 : cluster [DBG] pgmap v3501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:14 gibba032 bash[24004]: cluster 2021-03-10T15:49:13.531800+0000 mgr.gibba032.whopeo (mgr.24625) 6154 : cluster [DBG] pgmap v3501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:16 gibba032 bash[24004]: cluster 2021-03-10T15:49:15.533436+0000 mgr.gibba032.whopeo (mgr.24625) 6155 : cluster [DBG] pgmap v3502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:16 gibba011 bash[30791]: cluster 2021-03-10T15:49:15.533436+0000 mgr.gibba032.whopeo (mgr.24625) 6155 : cluster [DBG] pgmap v3502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:16 gibba011 bash[31774]: cluster 2021-03-10T15:49:15.533436+0000 mgr.gibba032.whopeo (mgr.24625) 6155 : cluster [DBG] pgmap v3502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:18 gibba032 bash[24004]: cluster 2021-03-10T15:49:17.534923+0000 mgr.gibba032.whopeo (mgr.24625) 6156 : cluster [DBG] pgmap v3503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:18 gibba011 bash[30791]: cluster 2021-03-10T15:49:17.534923+0000 mgr.gibba032.whopeo (mgr.24625) 6156 : cluster [DBG] pgmap v3503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:18 gibba011 bash[31774]: cluster 2021-03-10T15:49:17.534923+0000 mgr.gibba032.whopeo (mgr.24625) 6156 : cluster [DBG] pgmap v3503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:20 gibba032 bash[24004]: cluster 2021-03-10T15:49:19.535786+0000 mgr.gibba032.whopeo (mgr.24625) 6157 : cluster [DBG] pgmap v3504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:21.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:20 gibba011 bash[30791]: cluster 2021-03-10T15:49:19.535786+0000 mgr.gibba032.whopeo (mgr.24625) 6157 : cluster [DBG] pgmap v3504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:20 gibba011 bash[31774]: cluster 2021-03-10T15:49:19.535786+0000 mgr.gibba032.whopeo (mgr.24625) 6157 : cluster [DBG] pgmap v3504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:22 gibba032 bash[24004]: cluster 2021-03-10T15:49:21.537195+0000 mgr.gibba032.whopeo (mgr.24625) 6158 : cluster [DBG] pgmap v3505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:22 gibba011 bash[30791]: cluster 2021-03-10T15:49:21.537195+0000 mgr.gibba032.whopeo (mgr.24625) 6158 : cluster [DBG] pgmap v3505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:23.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:22 gibba011 bash[31774]: cluster 2021-03-10T15:49:21.537195+0000 mgr.gibba032.whopeo (mgr.24625) 6158 : cluster [DBG] pgmap v3505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:24 gibba011 bash[30791]: cluster 2021-03-10T15:49:23.538788+0000 mgr.gibba032.whopeo (mgr.24625) 6159 : cluster [DBG] pgmap v3506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:24 gibba011 bash[31774]: cluster 2021-03-10T15:49:23.538788+0000 mgr.gibba032.whopeo (mgr.24625) 6159 : cluster [DBG] pgmap v3506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:24 gibba032 bash[24004]: cluster 2021-03-10T15:49:23.538788+0000 mgr.gibba032.whopeo (mgr.24625) 6159 : cluster [DBG] pgmap v3506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:26 gibba032 bash[24004]: cluster 2021-03-10T15:49:25.539671+0000 mgr.gibba032.whopeo (mgr.24625) 6160 : cluster [DBG] pgmap v3507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:26 gibba032 bash[24004]: audit 2021-03-10T15:49:25.952601+0000 mon.a (mon.0) 2295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:49:27.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:26 gibba011 bash[30791]: cluster 2021-03-10T15:49:25.539671+0000 mgr.gibba032.whopeo (mgr.24625) 6160 : cluster [DBG] pgmap v3507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:27.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:26 gibba011 bash[30791]: audit 2021-03-10T15:49:25.952601+0000 mon.a (mon.0) 2295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:49:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:26 gibba011 bash[31774]: cluster 2021-03-10T15:49:25.539671+0000 mgr.gibba032.whopeo (mgr.24625) 6160 : cluster [DBG] pgmap v3507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:27.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:26 gibba011 bash[31774]: audit 2021-03-10T15:49:25.952601+0000 mon.a (mon.0) 2295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:49:28.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:27 gibba032 bash[24004]: cephadm 2021-03-10T15:49:25.950359+0000 mgr.gibba032.whopeo (mgr.24625) 6161 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:49:28.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:27 gibba011 bash[30791]: cephadm 2021-03-10T15:49:25.950359+0000 mgr.gibba032.whopeo (mgr.24625) 6161 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:49:28.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:27 gibba011 bash[31774]: cephadm 2021-03-10T15:49:25.950359+0000 mgr.gibba032.whopeo (mgr.24625) 6161 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:49:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:28 gibba032 bash[24004]: cluster 2021-03-10T15:49:27.541323+0000 mgr.gibba032.whopeo (mgr.24625) 6162 : cluster [DBG] pgmap v3508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:28 gibba011 bash[30791]: cluster 2021-03-10T15:49:27.541323+0000 mgr.gibba032.whopeo (mgr.24625) 6162 : cluster [DBG] pgmap v3508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:28 gibba011 bash[31774]: cluster 2021-03-10T15:49:27.541323+0000 mgr.gibba032.whopeo (mgr.24625) 6162 : cluster [DBG] pgmap v3508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.427313+0000 mgr.gibba032.whopeo (mgr.24625) 6163 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:49:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.428155+0000 mon.a (mon.0) 2296 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cluster 2021-03-10T15:49:29.542596+0000 mgr.gibba032.whopeo (mgr.24625) 6164 : cluster [DBG] pgmap v3509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.646757+0000 mon.a (mon.0) 2297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.651998+0000 mgr.gibba032.whopeo (mgr.24625) 6165 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:49:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.654426+0000 mgr.gibba032.whopeo (mgr.24625) 6166 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:49:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.654540+0000 mon.a (mon.0) 2298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:49:30.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.656224+0000 mgr.gibba032.whopeo (mgr.24625) 6167 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:49:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.656264+0000 mgr.gibba032.whopeo (mgr.24625) 6168 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:49:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.656524+0000 mon.a (mon.0) 2299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.657662+0000 mgr.gibba032.whopeo (mgr.24625) 6169 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:49:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.657702+0000 mgr.gibba032.whopeo (mgr.24625) 6170 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:49:30.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.658158+0000 mon.a (mon.0) 2300 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.659582+0000 mon.a (mon.0) 2301 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.659628+0000 mgr.gibba032.whopeo (mgr.24625) 6171 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:49:30.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.661429+0000 mgr.gibba032.whopeo (mgr.24625) 6172 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:49:30.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.661475+0000 mgr.gibba032.whopeo (mgr.24625) 6173 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:49:30.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.663082+0000 mgr.gibba032.whopeo (mgr.24625) 6174 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:49:30.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.663511+0000 mon.a (mon.0) 2302 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.663533+0000 mgr.gibba032.whopeo (mgr.24625) 6175 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:49:30.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.664858+0000 mgr.gibba032.whopeo (mgr.24625) 6176 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:49:30.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.664904+0000 mgr.gibba032.whopeo (mgr.24625) 6177 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:49:30.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: cephadm 2021-03-10T15:49:29.665032+0000 mgr.gibba032.whopeo (mgr.24625) 6178 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:30.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:30 gibba032 bash[24004]: audit 2021-03-10T15:49:29.665336+0000 mon.a (mon.0) 2303 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.427313+0000 mgr.gibba032.whopeo (mgr.24625) 6163 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.428155+0000 mon.a (mon.0) 2296 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cluster 2021-03-10T15:49:29.542596+0000 mgr.gibba032.whopeo (mgr.24625) 6164 : cluster [DBG] pgmap v3509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.646757+0000 mon.a (mon.0) 2297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.651998+0000 mgr.gibba032.whopeo (mgr.24625) 6165 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:49:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.654426+0000 mgr.gibba032.whopeo (mgr.24625) 6166 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:49:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.654540+0000 mon.a (mon.0) 2298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:49:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.656224+0000 mgr.gibba032.whopeo (mgr.24625) 6167 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:49:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.656264+0000 mgr.gibba032.whopeo (mgr.24625) 6168 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:49:30.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.656524+0000 mon.a (mon.0) 2299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.657662+0000 mgr.gibba032.whopeo (mgr.24625) 6169 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:49:30.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.657702+0000 mgr.gibba032.whopeo (mgr.24625) 6170 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:49:30.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.658158+0000 mon.a (mon.0) 2300 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.659582+0000 mon.a (mon.0) 2301 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.659628+0000 mgr.gibba032.whopeo (mgr.24625) 6171 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:49:30.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.661429+0000 mgr.gibba032.whopeo (mgr.24625) 6172 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:49:30.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.661475+0000 mgr.gibba032.whopeo (mgr.24625) 6173 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:49:30.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.663082+0000 mgr.gibba032.whopeo (mgr.24625) 6174 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:49:30.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.663511+0000 mon.a (mon.0) 2302 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.663533+0000 mgr.gibba032.whopeo (mgr.24625) 6175 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:49:30.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.664858+0000 mgr.gibba032.whopeo (mgr.24625) 6176 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:49:30.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.664904+0000 mgr.gibba032.whopeo (mgr.24625) 6177 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:49:30.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: cephadm 2021-03-10T15:49:29.665032+0000 mgr.gibba032.whopeo (mgr.24625) 6178 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:30.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[30791]: audit 2021-03-10T15:49:29.665336+0000 mon.a (mon.0) 2303 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.427313+0000 mgr.gibba032.whopeo (mgr.24625) 6163 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:49:30.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.428155+0000 mon.a (mon.0) 2296 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cluster 2021-03-10T15:49:29.542596+0000 mgr.gibba032.whopeo (mgr.24625) 6164 : cluster [DBG] pgmap v3509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:30.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.646757+0000 mon.a (mon.0) 2297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:49:30.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.651998+0000 mgr.gibba032.whopeo (mgr.24625) 6165 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:49:30.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.654426+0000 mgr.gibba032.whopeo (mgr.24625) 6166 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:49:30.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.654540+0000 mon.a (mon.0) 2298 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:49:30.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.656224+0000 mgr.gibba032.whopeo (mgr.24625) 6167 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:49:30.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.656264+0000 mgr.gibba032.whopeo (mgr.24625) 6168 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:49:30.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.656524+0000 mon.a (mon.0) 2299 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.657662+0000 mgr.gibba032.whopeo (mgr.24625) 6169 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:49:30.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.657702+0000 mgr.gibba032.whopeo (mgr.24625) 6170 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:49:30.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.658158+0000 mon.a (mon.0) 2300 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.659582+0000 mon.a (mon.0) 2301 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.659628+0000 mgr.gibba032.whopeo (mgr.24625) 6171 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:49:30.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.661429+0000 mgr.gibba032.whopeo (mgr.24625) 6172 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:49:30.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.661475+0000 mgr.gibba032.whopeo (mgr.24625) 6173 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:49:30.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.663082+0000 mgr.gibba032.whopeo (mgr.24625) 6174 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:49:30.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.663511+0000 mon.a (mon.0) 2302 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:30.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.663533+0000 mgr.gibba032.whopeo (mgr.24625) 6175 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:49:30.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.664858+0000 mgr.gibba032.whopeo (mgr.24625) 6176 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:49:30.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.664904+0000 mgr.gibba032.whopeo (mgr.24625) 6177 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:49:30.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: cephadm 2021-03-10T15:49:29.665032+0000 mgr.gibba032.whopeo (mgr.24625) 6178 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:30.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:30 gibba011 bash[31774]: audit 2021-03-10T15:49:29.665336+0000 mon.a (mon.0) 2303 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:32 gibba032 bash[24004]: cluster 2021-03-10T15:49:31.543663+0000 mgr.gibba032.whopeo (mgr.24625) 6179 : cluster [DBG] pgmap v3510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:33.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:32 gibba011 bash[30791]: cluster 2021-03-10T15:49:31.543663+0000 mgr.gibba032.whopeo (mgr.24625) 6179 : cluster [DBG] pgmap v3510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:32 gibba011 bash[31774]: cluster 2021-03-10T15:49:31.543663+0000 mgr.gibba032.whopeo (mgr.24625) 6179 : cluster [DBG] pgmap v3510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:34 gibba011 bash[30791]: cluster 2021-03-10T15:49:33.544936+0000 mgr.gibba032.whopeo (mgr.24625) 6180 : cluster [DBG] pgmap v3511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:34 gibba011 bash[31774]: cluster 2021-03-10T15:49:33.544936+0000 mgr.gibba032.whopeo (mgr.24625) 6180 : cluster [DBG] pgmap v3511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:34 gibba032 bash[24004]: cluster 2021-03-10T15:49:33.544936+0000 mgr.gibba032.whopeo (mgr.24625) 6180 : cluster [DBG] pgmap v3511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:36 gibba032 bash[24004]: cluster 2021-03-10T15:49:35.546535+0000 mgr.gibba032.whopeo (mgr.24625) 6181 : cluster [DBG] pgmap v3512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:36 gibba011 bash[30791]: cluster 2021-03-10T15:49:35.546535+0000 mgr.gibba032.whopeo (mgr.24625) 6181 : cluster [DBG] pgmap v3512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:36 gibba011 bash[31774]: cluster 2021-03-10T15:49:35.546535+0000 mgr.gibba032.whopeo (mgr.24625) 6181 : cluster [DBG] pgmap v3512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:38.804 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:49:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:38 gibba032 bash[24004]: cluster 2021-03-10T15:49:37.547704+0000 mgr.gibba032.whopeo (mgr.24625) 6182 : cluster [DBG] pgmap v3513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:39.030 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:49:39.031 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (116m) 9s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:49:39.031 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:49:39.031 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (116m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:49:39.031 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (116m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:49:39.032 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (115m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:49:39.032 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (115m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:49:39.032 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (115m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:49:39.032 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (115m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:49:39.032 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:49:39.033 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:49:39.033 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (114m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:49:39.033 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (114m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:49:39.033 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (113m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:49:39.033 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (113m) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:49:39.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:49:39.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:49:39.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:49:39.034 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (111m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:49:39.035 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:49:39.035 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9s ago 119m docker.io/ceph/ceph:v15.2.0 2021-03-10T15:49:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:38 gibba011 bash[30791]: cluster 2021-03-10T15:49:37.547704+0000 mgr.gibba032.whopeo (mgr.24625) 6182 : cluster [DBG] pgmap v3513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:38 gibba011 bash[31774]: cluster 2021-03-10T15:49:37.547704+0000 mgr.gibba032.whopeo (mgr.24625) 6182 : cluster [DBG] pgmap v3513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:39.280 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:49:39.280 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:49:39.280 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:49:39.280 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:49:39.281 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:49:39.282 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:49:39.282 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:49:39.282 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:49:39.282 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:49:39.282 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:49:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:39 gibba032 bash[24004]: audit 2021-03-10T15:49:38.792684+0000 mgr.gibba032.whopeo (mgr.24625) 6183 : audit [DBG] from='client.58177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:39 gibba032 bash[24004]: audit 2021-03-10T15:49:39.279069+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.? 172.21.2.111:0/166572839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:39 gibba011 bash[30791]: audit 2021-03-10T15:49:38.792684+0000 mgr.gibba032.whopeo (mgr.24625) 6183 : audit [DBG] from='client.58177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:39 gibba011 bash[30791]: audit 2021-03-10T15:49:39.279069+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.? 172.21.2.111:0/166572839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:40.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:39 gibba011 bash[31774]: audit 2021-03-10T15:49:38.792684+0000 mgr.gibba032.whopeo (mgr.24625) 6183 : audit [DBG] from='client.58177 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:40.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:39 gibba011 bash[31774]: audit 2021-03-10T15:49:39.279069+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.? 172.21.2.111:0/166572839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:49:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:40 gibba032 bash[24004]: audit 2021-03-10T15:49:39.016315+0000 mgr.gibba032.whopeo (mgr.24625) 6184 : audit [DBG] from='client.58183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:40 gibba032 bash[24004]: cluster 2021-03-10T15:49:39.549097+0000 mgr.gibba032.whopeo (mgr.24625) 6185 : cluster [DBG] pgmap v3514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:40 gibba011 bash[30791]: audit 2021-03-10T15:49:39.016315+0000 mgr.gibba032.whopeo (mgr.24625) 6184 : audit [DBG] from='client.58183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:40 gibba011 bash[30791]: cluster 2021-03-10T15:49:39.549097+0000 mgr.gibba032.whopeo (mgr.24625) 6185 : cluster [DBG] pgmap v3514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:40 gibba011 bash[31774]: audit 2021-03-10T15:49:39.016315+0000 mgr.gibba032.whopeo (mgr.24625) 6184 : audit [DBG] from='client.58183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:49:41.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:40 gibba011 bash[31774]: cluster 2021-03-10T15:49:39.549097+0000 mgr.gibba032.whopeo (mgr.24625) 6185 : cluster [DBG] pgmap v3514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:42 gibba032 bash[24004]: cluster 2021-03-10T15:49:41.550829+0000 mgr.gibba032.whopeo (mgr.24625) 6186 : cluster [DBG] pgmap v3515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:42 gibba011 bash[30791]: cluster 2021-03-10T15:49:41.550829+0000 mgr.gibba032.whopeo (mgr.24625) 6186 : cluster [DBG] pgmap v3515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:42 gibba011 bash[31774]: cluster 2021-03-10T15:49:41.550829+0000 mgr.gibba032.whopeo (mgr.24625) 6186 : cluster [DBG] pgmap v3515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:44 gibba011 bash[30791]: cluster 2021-03-10T15:49:43.551876+0000 mgr.gibba032.whopeo (mgr.24625) 6187 : cluster [DBG] pgmap v3516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:44 gibba011 bash[31774]: cluster 2021-03-10T15:49:43.551876+0000 mgr.gibba032.whopeo (mgr.24625) 6187 : cluster [DBG] pgmap v3516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:44.683 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:44 gibba032 bash[24004]: cluster 2021-03-10T15:49:43.551876+0000 mgr.gibba032.whopeo (mgr.24625) 6187 : cluster [DBG] pgmap v3516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:45.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:45 gibba011 bash[30791]: cephadm 2021-03-10T15:49:44.681025+0000 mgr.gibba032.whopeo (mgr.24625) 6188 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:45.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:45 gibba011 bash[31774]: cephadm 2021-03-10T15:49:44.681025+0000 mgr.gibba032.whopeo (mgr.24625) 6188 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:45.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:45 gibba032 bash[24004]: cephadm 2021-03-10T15:49:44.681025+0000 mgr.gibba032.whopeo (mgr.24625) 6188 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:49:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:46 gibba032 bash[24004]: cluster 2021-03-10T15:49:45.553346+0000 mgr.gibba032.whopeo (mgr.24625) 6189 : cluster [DBG] pgmap v3517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:46 gibba011 bash[30791]: cluster 2021-03-10T15:49:45.553346+0000 mgr.gibba032.whopeo (mgr.24625) 6189 : cluster [DBG] pgmap v3517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:46 gibba011 bash[31774]: cluster 2021-03-10T15:49:45.553346+0000 mgr.gibba032.whopeo (mgr.24625) 6189 : cluster [DBG] pgmap v3517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:48 gibba032 bash[24004]: cluster 2021-03-10T15:49:47.555096+0000 mgr.gibba032.whopeo (mgr.24625) 6190 : cluster [DBG] pgmap v3518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:48 gibba011 bash[30791]: cluster 2021-03-10T15:49:47.555096+0000 mgr.gibba032.whopeo (mgr.24625) 6190 : cluster [DBG] pgmap v3518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:48 gibba011 bash[31774]: cluster 2021-03-10T15:49:47.555096+0000 mgr.gibba032.whopeo (mgr.24625) 6190 : cluster [DBG] pgmap v3518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:50 gibba032 bash[24004]: cluster 2021-03-10T15:49:49.556146+0000 mgr.gibba032.whopeo (mgr.24625) 6191 : cluster [DBG] pgmap v3519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:50 gibba011 bash[30791]: cluster 2021-03-10T15:49:49.556146+0000 mgr.gibba032.whopeo (mgr.24625) 6191 : cluster [DBG] pgmap v3519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:50 gibba011 bash[31774]: cluster 2021-03-10T15:49:49.556146+0000 mgr.gibba032.whopeo (mgr.24625) 6191 : cluster [DBG] pgmap v3519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:52 gibba032 bash[24004]: cluster 2021-03-10T15:49:51.557528+0000 mgr.gibba032.whopeo (mgr.24625) 6192 : cluster [DBG] pgmap v3520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:53.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:52 gibba011 bash[30791]: cluster 2021-03-10T15:49:51.557528+0000 mgr.gibba032.whopeo (mgr.24625) 6192 : cluster [DBG] pgmap v3520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:53.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:52 gibba011 bash[31774]: cluster 2021-03-10T15:49:51.557528+0000 mgr.gibba032.whopeo (mgr.24625) 6192 : cluster [DBG] pgmap v3520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:54 gibba011 bash[30791]: cluster 2021-03-10T15:49:53.559121+0000 mgr.gibba032.whopeo (mgr.24625) 6193 : cluster [DBG] pgmap v3521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:54 gibba011 bash[31774]: cluster 2021-03-10T15:49:53.559121+0000 mgr.gibba032.whopeo (mgr.24625) 6193 : cluster [DBG] pgmap v3521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:54 gibba032 bash[24004]: cluster 2021-03-10T15:49:53.559121+0000 mgr.gibba032.whopeo (mgr.24625) 6193 : cluster [DBG] pgmap v3521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:56 gibba032 bash[24004]: cluster 2021-03-10T15:49:55.560285+0000 mgr.gibba032.whopeo (mgr.24625) 6194 : cluster [DBG] pgmap v3522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:56 gibba011 bash[30791]: cluster 2021-03-10T15:49:55.560285+0000 mgr.gibba032.whopeo (mgr.24625) 6194 : cluster [DBG] pgmap v3522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:56 gibba011 bash[31774]: cluster 2021-03-10T15:49:55.560285+0000 mgr.gibba032.whopeo (mgr.24625) 6194 : cluster [DBG] pgmap v3522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.728250+0000 mon.a (mon.0) 2305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:49:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.728987+0000 mon.a (mon.0) 2306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:49:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.758374+0000 mon.a (mon.0) 2307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:49:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.759115+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:49:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.759904+0000 mon.a (mon.0) 2309 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:49:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:57 gibba032 bash[24004]: audit 2021-03-10T15:49:56.760590+0000 mon.a (mon.0) 2310 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:49:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.728250+0000 mon.a (mon.0) 2305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:49:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.728987+0000 mon.a (mon.0) 2306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:49:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.758374+0000 mon.a (mon.0) 2307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:49:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.759115+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:49:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.759904+0000 mon.a (mon.0) 2309 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:49:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[30791]: audit 2021-03-10T15:49:56.760590+0000 mon.a (mon.0) 2310 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:49:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.728250+0000 mon.a (mon.0) 2305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:49:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.728987+0000 mon.a (mon.0) 2306 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:49:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.758374+0000 mon.a (mon.0) 2307 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:49:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.759115+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:49:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.759904+0000 mon.a (mon.0) 2309 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:49:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:57 gibba011 bash[31774]: audit 2021-03-10T15:49:56.760590+0000 mon.a (mon.0) 2310 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:49:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:49:58 gibba032 bash[24004]: cluster 2021-03-10T15:49:57.562007+0000 mgr.gibba032.whopeo (mgr.24625) 6195 : cluster [DBG] pgmap v3523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:49:58 gibba011 bash[30791]: cluster 2021-03-10T15:49:57.562007+0000 mgr.gibba032.whopeo (mgr.24625) 6195 : cluster [DBG] pgmap v3523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:49:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:49:58 gibba011 bash[31774]: cluster 2021-03-10T15:49:57.562007+0000 mgr.gibba032.whopeo (mgr.24625) 6195 : cluster [DBG] pgmap v3523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[30791]: debug 2021-03-10T15:49:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:50:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:00 gibba032 bash[24004]: cluster 2021-03-10T15:49:59.563473+0000 mgr.gibba032.whopeo (mgr.24625) 6196 : cluster [DBG] pgmap v3524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:00 gibba032 bash[24004]: cephadm 2021-03-10T15:49:59.693678+0000 mgr.gibba032.whopeo (mgr.24625) 6197 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:00 gibba032 bash[24004]: cluster 2021-03-10T15:50:00.000282+0000 mon.a (mon.0) 2311 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:50:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[30791]: cluster 2021-03-10T15:49:59.563473+0000 mgr.gibba032.whopeo (mgr.24625) 6196 : cluster [DBG] pgmap v3524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[30791]: cephadm 2021-03-10T15:49:59.693678+0000 mgr.gibba032.whopeo (mgr.24625) 6197 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:01.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[30791]: cluster 2021-03-10T15:50:00.000282+0000 mon.a (mon.0) 2311 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:50:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[31774]: cluster 2021-03-10T15:49:59.563473+0000 mgr.gibba032.whopeo (mgr.24625) 6196 : cluster [DBG] pgmap v3524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[31774]: cephadm 2021-03-10T15:49:59.693678+0000 mgr.gibba032.whopeo (mgr.24625) 6197 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:00 gibba011 bash[31774]: cluster 2021-03-10T15:50:00.000282+0000 mon.a (mon.0) 2311 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T15:50:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:02 gibba032 bash[24004]: cluster 2021-03-10T15:50:01.565140+0000 mgr.gibba032.whopeo (mgr.24625) 6198 : cluster [DBG] pgmap v3525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:02 gibba011 bash[30791]: cluster 2021-03-10T15:50:01.565140+0000 mgr.gibba032.whopeo (mgr.24625) 6198 : cluster [DBG] pgmap v3525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:02 gibba011 bash[31774]: cluster 2021-03-10T15:50:01.565140+0000 mgr.gibba032.whopeo (mgr.24625) 6198 : cluster [DBG] pgmap v3525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:04 gibba032 bash[24004]: cluster 2021-03-10T15:50:03.566456+0000 mgr.gibba032.whopeo (mgr.24625) 6199 : cluster [DBG] pgmap v3526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:04 gibba011 bash[30791]: cluster 2021-03-10T15:50:03.566456+0000 mgr.gibba032.whopeo (mgr.24625) 6199 : cluster [DBG] pgmap v3526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:04 gibba011 bash[31774]: cluster 2021-03-10T15:50:03.566456+0000 mgr.gibba032.whopeo (mgr.24625) 6199 : cluster [DBG] pgmap v3526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:06 gibba032 bash[24004]: cluster 2021-03-10T15:50:05.567971+0000 mgr.gibba032.whopeo (mgr.24625) 6200 : cluster [DBG] pgmap v3527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:07.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:06 gibba011 bash[30791]: cluster 2021-03-10T15:50:05.567971+0000 mgr.gibba032.whopeo (mgr.24625) 6200 : cluster [DBG] pgmap v3527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:06 gibba011 bash[31774]: cluster 2021-03-10T15:50:05.567971+0000 mgr.gibba032.whopeo (mgr.24625) 6200 : cluster [DBG] pgmap v3527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:08 gibba032 bash[24004]: cluster 2021-03-10T15:50:07.569783+0000 mgr.gibba032.whopeo (mgr.24625) 6201 : cluster [DBG] pgmap v3528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:08 gibba011 bash[30791]: cluster 2021-03-10T15:50:07.569783+0000 mgr.gibba032.whopeo (mgr.24625) 6201 : cluster [DBG] pgmap v3528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:08 gibba011 bash[31774]: cluster 2021-03-10T15:50:07.569783+0000 mgr.gibba032.whopeo (mgr.24625) 6201 : cluster [DBG] pgmap v3528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:09.525 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:50:09.751 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:50:09.751 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (116m) 40s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:50:09.751 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:50:09.752 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (117m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:50:09.752 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (117m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:50:09.752 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (116m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:50:09.752 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (116m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:50:09.753 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (115m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:50:09.753 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (116m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:50:09.753 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 40s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:50:09.753 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:50:09.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (115m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:50:09.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (114m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:50:09.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (114m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:50:09.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (113m) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:50:09.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (113m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:50:09.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:50:09.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:50:09.755 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (112m) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:50:09.755 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:50:09.756 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 40s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:50:10.000 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:50:10.001 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:50:10.001 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:50:10.001 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:10.001 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:50:10.001 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:50:10.002 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:10.002 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:50:10.002 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:50:10.002 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:10.003 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:50:10.003 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:50:10.003 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:50:10.003 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:50:10.003 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:50:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:10 gibba032 bash[24004]: audit 2021-03-10T15:50:09.513649+0000 mgr.gibba032.whopeo (mgr.24625) 6202 : audit [DBG] from='client.48213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:10 gibba032 bash[24004]: cluster 2021-03-10T15:50:09.571277+0000 mgr.gibba032.whopeo (mgr.24625) 6203 : cluster [DBG] pgmap v3529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:10 gibba032 bash[24004]: audit 2021-03-10T15:50:09.737213+0000 mgr.gibba032.whopeo (mgr.24625) 6204 : audit [DBG] from='client.58201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:10 gibba032 bash[24004]: audit 2021-03-10T15:50:09.999519+0000 mon.c (mon.1) 119 : audit [DBG] from='client.? 172.21.2.111:0/3546179162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[30791]: audit 2021-03-10T15:50:09.513649+0000 mgr.gibba032.whopeo (mgr.24625) 6202 : audit [DBG] from='client.48213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[30791]: cluster 2021-03-10T15:50:09.571277+0000 mgr.gibba032.whopeo (mgr.24625) 6203 : cluster [DBG] pgmap v3529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[30791]: audit 2021-03-10T15:50:09.737213+0000 mgr.gibba032.whopeo (mgr.24625) 6204 : audit [DBG] from='client.58201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[30791]: audit 2021-03-10T15:50:09.999519+0000 mon.c (mon.1) 119 : audit [DBG] from='client.? 172.21.2.111:0/3546179162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:11.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[31774]: audit 2021-03-10T15:50:09.513649+0000 mgr.gibba032.whopeo (mgr.24625) 6202 : audit [DBG] from='client.48213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[31774]: cluster 2021-03-10T15:50:09.571277+0000 mgr.gibba032.whopeo (mgr.24625) 6203 : cluster [DBG] pgmap v3529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:11.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[31774]: audit 2021-03-10T15:50:09.737213+0000 mgr.gibba032.whopeo (mgr.24625) 6204 : audit [DBG] from='client.58201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:11.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:10 gibba011 bash[31774]: audit 2021-03-10T15:50:09.999519+0000 mon.c (mon.1) 119 : audit [DBG] from='client.? 172.21.2.111:0/3546179162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:12 gibba032 bash[24004]: cluster 2021-03-10T15:50:11.572913+0000 mgr.gibba032.whopeo (mgr.24625) 6205 : cluster [DBG] pgmap v3530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:12 gibba011 bash[30791]: cluster 2021-03-10T15:50:11.572913+0000 mgr.gibba032.whopeo (mgr.24625) 6205 : cluster [DBG] pgmap v3530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:12 gibba011 bash[31774]: cluster 2021-03-10T15:50:11.572913+0000 mgr.gibba032.whopeo (mgr.24625) 6205 : cluster [DBG] pgmap v3530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:14.712 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:14 gibba032 bash[24004]: cluster 2021-03-10T15:50:13.574284+0000 mgr.gibba032.whopeo (mgr.24625) 6206 : cluster [DBG] pgmap v3531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:14 gibba011 bash[30791]: cluster 2021-03-10T15:50:13.574284+0000 mgr.gibba032.whopeo (mgr.24625) 6206 : cluster [DBG] pgmap v3531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:14 gibba011 bash[31774]: cluster 2021-03-10T15:50:13.574284+0000 mgr.gibba032.whopeo (mgr.24625) 6206 : cluster [DBG] pgmap v3531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:15.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:15 gibba032 bash[24004]: cephadm 2021-03-10T15:50:14.709707+0000 mgr.gibba032.whopeo (mgr.24625) 6207 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:15.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:15 gibba011 bash[30791]: cephadm 2021-03-10T15:50:14.709707+0000 mgr.gibba032.whopeo (mgr.24625) 6207 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:15.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:15 gibba011 bash[31774]: cephadm 2021-03-10T15:50:14.709707+0000 mgr.gibba032.whopeo (mgr.24625) 6207 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:16 gibba032 bash[24004]: cluster 2021-03-10T15:50:15.575759+0000 mgr.gibba032.whopeo (mgr.24625) 6208 : cluster [DBG] pgmap v3532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:16 gibba011 bash[30791]: cluster 2021-03-10T15:50:15.575759+0000 mgr.gibba032.whopeo (mgr.24625) 6208 : cluster [DBG] pgmap v3532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:16 gibba011 bash[31774]: cluster 2021-03-10T15:50:15.575759+0000 mgr.gibba032.whopeo (mgr.24625) 6208 : cluster [DBG] pgmap v3532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:18 gibba032 bash[24004]: cluster 2021-03-10T15:50:17.577604+0000 mgr.gibba032.whopeo (mgr.24625) 6209 : cluster [DBG] pgmap v3533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:18 gibba011 bash[30791]: cluster 2021-03-10T15:50:17.577604+0000 mgr.gibba032.whopeo (mgr.24625) 6209 : cluster [DBG] pgmap v3533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:18 gibba011 bash[31774]: cluster 2021-03-10T15:50:17.577604+0000 mgr.gibba032.whopeo (mgr.24625) 6209 : cluster [DBG] pgmap v3533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:20 gibba032 bash[24004]: cluster 2021-03-10T15:50:19.578979+0000 mgr.gibba032.whopeo (mgr.24625) 6210 : cluster [DBG] pgmap v3534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:20 gibba011 bash[30791]: cluster 2021-03-10T15:50:19.578979+0000 mgr.gibba032.whopeo (mgr.24625) 6210 : cluster [DBG] pgmap v3534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:20 gibba011 bash[31774]: cluster 2021-03-10T15:50:19.578979+0000 mgr.gibba032.whopeo (mgr.24625) 6210 : cluster [DBG] pgmap v3534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:22 gibba032 bash[24004]: cluster 2021-03-10T15:50:21.580669+0000 mgr.gibba032.whopeo (mgr.24625) 6211 : cluster [DBG] pgmap v3535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:22 gibba011 bash[30791]: cluster 2021-03-10T15:50:21.580669+0000 mgr.gibba032.whopeo (mgr.24625) 6211 : cluster [DBG] pgmap v3535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:23.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:22 gibba011 bash[31774]: cluster 2021-03-10T15:50:21.580669+0000 mgr.gibba032.whopeo (mgr.24625) 6211 : cluster [DBG] pgmap v3535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:24 gibba032 bash[24004]: cluster 2021-03-10T15:50:23.581936+0000 mgr.gibba032.whopeo (mgr.24625) 6212 : cluster [DBG] pgmap v3536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:24 gibba011 bash[30791]: cluster 2021-03-10T15:50:23.581936+0000 mgr.gibba032.whopeo (mgr.24625) 6212 : cluster [DBG] pgmap v3536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:24 gibba011 bash[31774]: cluster 2021-03-10T15:50:23.581936+0000 mgr.gibba032.whopeo (mgr.24625) 6212 : cluster [DBG] pgmap v3536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:26 gibba032 bash[24004]: cluster 2021-03-10T15:50:25.583049+0000 mgr.gibba032.whopeo (mgr.24625) 6213 : cluster [DBG] pgmap v3537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:27.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:26 gibba011 bash[30791]: cluster 2021-03-10T15:50:25.583049+0000 mgr.gibba032.whopeo (mgr.24625) 6213 : cluster [DBG] pgmap v3537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:26 gibba011 bash[31774]: cluster 2021-03-10T15:50:25.583049+0000 mgr.gibba032.whopeo (mgr.24625) 6213 : cluster [DBG] pgmap v3537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:28 gibba032 bash[24004]: cluster 2021-03-10T15:50:27.584666+0000 mgr.gibba032.whopeo (mgr.24625) 6214 : cluster [DBG] pgmap v3538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:29.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:28 gibba011 bash[30791]: cluster 2021-03-10T15:50:27.584666+0000 mgr.gibba032.whopeo (mgr.24625) 6214 : cluster [DBG] pgmap v3538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:28 gibba011 bash[31774]: cluster 2021-03-10T15:50:27.584666+0000 mgr.gibba032.whopeo (mgr.24625) 6214 : cluster [DBG] pgmap v3538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:30 gibba032 bash[24004]: cluster 2021-03-10T15:50:29.585949+0000 mgr.gibba032.whopeo (mgr.24625) 6215 : cluster [DBG] pgmap v3539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:30 gibba032 bash[24004]: cephadm 2021-03-10T15:50:29.725761+0000 mgr.gibba032.whopeo (mgr.24625) 6216 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:50:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:30 gibba032 bash[24004]: audit 2021-03-10T15:50:29.728257+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[30791]: cluster 2021-03-10T15:50:29.585949+0000 mgr.gibba032.whopeo (mgr.24625) 6215 : cluster [DBG] pgmap v3539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[30791]: cephadm 2021-03-10T15:50:29.725761+0000 mgr.gibba032.whopeo (mgr.24625) 6216 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:50:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[30791]: audit 2021-03-10T15:50:29.728257+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[31774]: cluster 2021-03-10T15:50:29.585949+0000 mgr.gibba032.whopeo (mgr.24625) 6215 : cluster [DBG] pgmap v3539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[31774]: cephadm 2021-03-10T15:50:29.725761+0000 mgr.gibba032.whopeo (mgr.24625) 6216 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:50:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:30 gibba011 bash[31774]: audit 2021-03-10T15:50:29.728257+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:32 gibba032 bash[24004]: cluster 2021-03-10T15:50:31.587437+0000 mgr.gibba032.whopeo (mgr.24625) 6217 : cluster [DBG] pgmap v3540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:32 gibba011 bash[30791]: cluster 2021-03-10T15:50:31.587437+0000 mgr.gibba032.whopeo (mgr.24625) 6217 : cluster [DBG] pgmap v3540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:32 gibba011 bash[31774]: cluster 2021-03-10T15:50:31.587437+0000 mgr.gibba032.whopeo (mgr.24625) 6217 : cluster [DBG] pgmap v3540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:34 gibba032 bash[24004]: audit 2021-03-10T15:50:33.427906+0000 mon.a (mon.0) 2313 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:34 gibba032 bash[24004]: audit 2021-03-10T15:50:33.428943+0000 mon.a (mon.0) 2314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:50:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:34 gibba032 bash[24004]: cluster 2021-03-10T15:50:33.588131+0000 mgr.gibba032.whopeo (mgr.24625) 6218 : cluster [DBG] pgmap v3541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[30791]: audit 2021-03-10T15:50:33.427906+0000 mon.a (mon.0) 2313 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[30791]: audit 2021-03-10T15:50:33.428943+0000 mon.a (mon.0) 2314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:50:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[30791]: cluster 2021-03-10T15:50:33.588131+0000 mgr.gibba032.whopeo (mgr.24625) 6218 : cluster [DBG] pgmap v3541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[31774]: audit 2021-03-10T15:50:33.427906+0000 mon.a (mon.0) 2313 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[31774]: audit 2021-03-10T15:50:33.428943+0000 mon.a (mon.0) 2314 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T15:50:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:34 gibba011 bash[31774]: cluster 2021-03-10T15:50:33.588131+0000 mgr.gibba032.whopeo (mgr.24625) 6218 : cluster [DBG] pgmap v3541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:36 gibba032 bash[24004]: cluster 2021-03-10T15:50:35.589215+0000 mgr.gibba032.whopeo (mgr.24625) 6219 : cluster [DBG] pgmap v3542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:36 gibba011 bash[30791]: cluster 2021-03-10T15:50:35.589215+0000 mgr.gibba032.whopeo (mgr.24625) 6219 : cluster [DBG] pgmap v3542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:36 gibba011 bash[31774]: cluster 2021-03-10T15:50:35.589215+0000 mgr.gibba032.whopeo (mgr.24625) 6219 : cluster [DBG] pgmap v3542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: cluster 2021-03-10T15:50:37.589967+0000 mgr.gibba032.whopeo (mgr.24625) 6220 : cluster [DBG] pgmap v3543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.096911+0000 mon.a (mon.0) 2315 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.265112+0000 mon.a (mon.0) 2316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.454921+0000 mon.a (mon.0) 2317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.457994+0000 mon.a (mon.0) 2318 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:50:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.459544+0000 mon.a (mon.0) 2319 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.460852+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.462070+0000 mon.a (mon.0) 2321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.464527+0000 mon.a (mon.0) 2322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:38.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:38 gibba032 bash[24004]: audit 2021-03-10T15:50:38.466012+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: cluster 2021-03-10T15:50:37.589967+0000 mgr.gibba032.whopeo (mgr.24625) 6220 : cluster [DBG] pgmap v3543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.096911+0000 mon.a (mon.0) 2315 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.265112+0000 mon.a (mon.0) 2316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.454921+0000 mon.a (mon.0) 2317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.457994+0000 mon.a (mon.0) 2318 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:50:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.459544+0000 mon.a (mon.0) 2319 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.460852+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.462070+0000 mon.a (mon.0) 2321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.464527+0000 mon.a (mon.0) 2322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[30791]: audit 2021-03-10T15:50:38.466012+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: cluster 2021-03-10T15:50:37.589967+0000 mgr.gibba032.whopeo (mgr.24625) 6220 : cluster [DBG] pgmap v3543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:39.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.096911+0000 mon.a (mon.0) 2315 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:50:39.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.265112+0000 mon.a (mon.0) 2316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:39.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.454921+0000 mon.a (mon.0) 2317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:50:39.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.457994+0000 mon.a (mon.0) 2318 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:50:39.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.459544+0000 mon.a (mon.0) 2319 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.460852+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.462070+0000 mon.a (mon.0) 2321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.464527+0000 mon.a (mon.0) 2322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:39.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:38 gibba011 bash[31774]: audit 2021-03-10T15:50:38.466012+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.264994+0000 mgr.gibba032.whopeo (mgr.24625) 6221 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:50:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.455800+0000 mgr.gibba032.whopeo (mgr.24625) 6222 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:50:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.457677+0000 mgr.gibba032.whopeo (mgr.24625) 6223 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:50:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.458886+0000 mgr.gibba032.whopeo (mgr.24625) 6224 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:50:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.458923+0000 mgr.gibba032.whopeo (mgr.24625) 6225 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:50:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.460177+0000 mgr.gibba032.whopeo (mgr.24625) 6226 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:50:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.460214+0000 mgr.gibba032.whopeo (mgr.24625) 6227 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:50:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.461364+0000 mgr.gibba032.whopeo (mgr.24625) 6228 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:50:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.462584+0000 mgr.gibba032.whopeo (mgr.24625) 6229 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:50:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.462621+0000 mgr.gibba032.whopeo (mgr.24625) 6230 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:50:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.463844+0000 mgr.gibba032.whopeo (mgr.24625) 6231 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:50:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.464226+0000 mgr.gibba032.whopeo (mgr.24625) 6232 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:50:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.465546+0000 mgr.gibba032.whopeo (mgr.24625) 6233 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:50:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.465583+0000 mgr.gibba032.whopeo (mgr.24625) 6234 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:50:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:39 gibba032 bash[24004]: cephadm 2021-03-10T15:50:38.465686+0000 mgr.gibba032.whopeo (mgr.24625) 6235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.264994+0000 mgr.gibba032.whopeo (mgr.24625) 6221 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:50:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.455800+0000 mgr.gibba032.whopeo (mgr.24625) 6222 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:50:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.457677+0000 mgr.gibba032.whopeo (mgr.24625) 6223 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:50:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.458886+0000 mgr.gibba032.whopeo (mgr.24625) 6224 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:50:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.458923+0000 mgr.gibba032.whopeo (mgr.24625) 6225 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:50:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.460177+0000 mgr.gibba032.whopeo (mgr.24625) 6226 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:50:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.460214+0000 mgr.gibba032.whopeo (mgr.24625) 6227 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:50:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.461364+0000 mgr.gibba032.whopeo (mgr.24625) 6228 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:50:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.462584+0000 mgr.gibba032.whopeo (mgr.24625) 6229 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:50:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.462621+0000 mgr.gibba032.whopeo (mgr.24625) 6230 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:50:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.463844+0000 mgr.gibba032.whopeo (mgr.24625) 6231 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:50:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.464226+0000 mgr.gibba032.whopeo (mgr.24625) 6232 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:50:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.465546+0000 mgr.gibba032.whopeo (mgr.24625) 6233 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:50:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.465583+0000 mgr.gibba032.whopeo (mgr.24625) 6234 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:50:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[30791]: cephadm 2021-03-10T15:50:38.465686+0000 mgr.gibba032.whopeo (mgr.24625) 6235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.264994+0000 mgr.gibba032.whopeo (mgr.24625) 6221 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:50:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.455800+0000 mgr.gibba032.whopeo (mgr.24625) 6222 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:50:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.457677+0000 mgr.gibba032.whopeo (mgr.24625) 6223 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:50:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.458886+0000 mgr.gibba032.whopeo (mgr.24625) 6224 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:50:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.458923+0000 mgr.gibba032.whopeo (mgr.24625) 6225 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:50:40.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.460177+0000 mgr.gibba032.whopeo (mgr.24625) 6226 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:50:40.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.460214+0000 mgr.gibba032.whopeo (mgr.24625) 6227 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:50:40.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.461364+0000 mgr.gibba032.whopeo (mgr.24625) 6228 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:50:40.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.462584+0000 mgr.gibba032.whopeo (mgr.24625) 6229 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:50:40.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.462621+0000 mgr.gibba032.whopeo (mgr.24625) 6230 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:50:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.463844+0000 mgr.gibba032.whopeo (mgr.24625) 6231 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:50:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.464226+0000 mgr.gibba032.whopeo (mgr.24625) 6232 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:50:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.465546+0000 mgr.gibba032.whopeo (mgr.24625) 6233 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:50:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.465583+0000 mgr.gibba032.whopeo (mgr.24625) 6234 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:50:40.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:39 gibba011 bash[31774]: cephadm 2021-03-10T15:50:38.465686+0000 mgr.gibba032.whopeo (mgr.24625) 6235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:40.244 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:50:40.472 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:50:40.472 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (117m) 71s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:50:40.472 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:50:40.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (117m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:50:40.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (117m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:50:40.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (117m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:50:40.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (116m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:50:40.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (116m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:50:40.474 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (116m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:50:40.474 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 71s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:50:40.474 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:50:40.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (115m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:50:40.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (115m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:50:40.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (114m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:50:40.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (114m) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:50:40.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (113m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:50:40.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (113m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:50:40.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (113m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:50:40.476 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (112m) 7s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:50:40.476 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:50:40.476 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 71s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:50:40.739 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:50:40.739 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:50:40.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:50:40.740 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:40.740 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:50:40.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:50:40.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:40.741 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:50:40.741 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:50:40.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:50:40.741 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:50:40.742 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:50:40.742 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:50:40.742 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:50:40.742 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:50:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:40 gibba032 bash[24004]: cluster 2021-03-10T15:50:39.591231+0000 mgr.gibba032.whopeo (mgr.24625) 6236 : cluster [DBG] pgmap v3544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:40 gibba011 bash[30791]: cluster 2021-03-10T15:50:39.591231+0000 mgr.gibba032.whopeo (mgr.24625) 6236 : cluster [DBG] pgmap v3544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:40 gibba011 bash[31774]: cluster 2021-03-10T15:50:39.591231+0000 mgr.gibba032.whopeo (mgr.24625) 6236 : cluster [DBG] pgmap v3544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:41 gibba032 bash[24004]: audit 2021-03-10T15:50:40.233443+0000 mgr.gibba032.whopeo (mgr.24625) 6237 : audit [DBG] from='client.58213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:41 gibba032 bash[24004]: audit 2021-03-10T15:50:40.458242+0000 mgr.gibba032.whopeo (mgr.24625) 6238 : audit [DBG] from='client.58219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:41 gibba032 bash[24004]: audit 2021-03-10T15:50:40.739044+0000 mon.c (mon.1) 120 : audit [DBG] from='client.? 172.21.2.111:0/2198129043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[30791]: audit 2021-03-10T15:50:40.233443+0000 mgr.gibba032.whopeo (mgr.24625) 6237 : audit [DBG] from='client.58213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[30791]: audit 2021-03-10T15:50:40.458242+0000 mgr.gibba032.whopeo (mgr.24625) 6238 : audit [DBG] from='client.58219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[30791]: audit 2021-03-10T15:50:40.739044+0000 mon.c (mon.1) 120 : audit [DBG] from='client.? 172.21.2.111:0/2198129043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[31774]: audit 2021-03-10T15:50:40.233443+0000 mgr.gibba032.whopeo (mgr.24625) 6237 : audit [DBG] from='client.58213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[31774]: audit 2021-03-10T15:50:40.458242+0000 mgr.gibba032.whopeo (mgr.24625) 6238 : audit [DBG] from='client.58219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:50:42.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:41 gibba011 bash[31774]: audit 2021-03-10T15:50:40.739044+0000 mon.c (mon.1) 120 : audit [DBG] from='client.? 172.21.2.111:0/2198129043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:50:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:42 gibba032 bash[24004]: cluster 2021-03-10T15:50:41.592918+0000 mgr.gibba032.whopeo (mgr.24625) 6239 : cluster [DBG] pgmap v3545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:42 gibba011 bash[30791]: cluster 2021-03-10T15:50:41.592918+0000 mgr.gibba032.whopeo (mgr.24625) 6239 : cluster [DBG] pgmap v3545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:42 gibba011 bash[31774]: cluster 2021-03-10T15:50:41.592918+0000 mgr.gibba032.whopeo (mgr.24625) 6239 : cluster [DBG] pgmap v3545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:44 gibba032 bash[24004]: cluster 2021-03-10T15:50:43.594299+0000 mgr.gibba032.whopeo (mgr.24625) 6240 : cluster [DBG] pgmap v3546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:44 gibba011 bash[30791]: cluster 2021-03-10T15:50:43.594299+0000 mgr.gibba032.whopeo (mgr.24625) 6240 : cluster [DBG] pgmap v3546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:44 gibba011 bash[31774]: cluster 2021-03-10T15:50:43.594299+0000 mgr.gibba032.whopeo (mgr.24625) 6240 : cluster [DBG] pgmap v3546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:46 gibba032 bash[24004]: cluster 2021-03-10T15:50:45.595762+0000 mgr.gibba032.whopeo (mgr.24625) 6241 : cluster [DBG] pgmap v3547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:46 gibba011 bash[30791]: cluster 2021-03-10T15:50:45.595762+0000 mgr.gibba032.whopeo (mgr.24625) 6241 : cluster [DBG] pgmap v3547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:46 gibba011 bash[31774]: cluster 2021-03-10T15:50:45.595762+0000 mgr.gibba032.whopeo (mgr.24625) 6241 : cluster [DBG] pgmap v3547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:48 gibba032 bash[24004]: cluster 2021-03-10T15:50:47.596383+0000 mgr.gibba032.whopeo (mgr.24625) 6242 : cluster [DBG] pgmap v3548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:48 gibba011 bash[30791]: cluster 2021-03-10T15:50:47.596383+0000 mgr.gibba032.whopeo (mgr.24625) 6242 : cluster [DBG] pgmap v3548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:48 gibba011 bash[31774]: cluster 2021-03-10T15:50:47.596383+0000 mgr.gibba032.whopeo (mgr.24625) 6242 : cluster [DBG] pgmap v3548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:50 gibba032 bash[24004]: cluster 2021-03-10T15:50:49.597741+0000 mgr.gibba032.whopeo (mgr.24625) 6243 : cluster [DBG] pgmap v3549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:50 gibba011 bash[30791]: cluster 2021-03-10T15:50:49.597741+0000 mgr.gibba032.whopeo (mgr.24625) 6243 : cluster [DBG] pgmap v3549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:50 gibba011 bash[31774]: cluster 2021-03-10T15:50:49.597741+0000 mgr.gibba032.whopeo (mgr.24625) 6243 : cluster [DBG] pgmap v3549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:52 gibba032 bash[24004]: cluster 2021-03-10T15:50:51.599503+0000 mgr.gibba032.whopeo (mgr.24625) 6244 : cluster [DBG] pgmap v3550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:53.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:52 gibba011 bash[30791]: cluster 2021-03-10T15:50:51.599503+0000 mgr.gibba032.whopeo (mgr.24625) 6244 : cluster [DBG] pgmap v3550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:53.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:52 gibba011 bash[31774]: cluster 2021-03-10T15:50:51.599503+0000 mgr.gibba032.whopeo (mgr.24625) 6244 : cluster [DBG] pgmap v3550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:54 gibba032 bash[24004]: cephadm 2021-03-10T15:50:53.473055+0000 mgr.gibba032.whopeo (mgr.24625) 6245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:54 gibba032 bash[24004]: cluster 2021-03-10T15:50:53.601064+0000 mgr.gibba032.whopeo (mgr.24625) 6246 : cluster [DBG] pgmap v3551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:54 gibba011 bash[30791]: cephadm 2021-03-10T15:50:53.473055+0000 mgr.gibba032.whopeo (mgr.24625) 6245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:54 gibba011 bash[30791]: cluster 2021-03-10T15:50:53.601064+0000 mgr.gibba032.whopeo (mgr.24625) 6246 : cluster [DBG] pgmap v3551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:54 gibba011 bash[31774]: cephadm 2021-03-10T15:50:53.473055+0000 mgr.gibba032.whopeo (mgr.24625) 6245 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:50:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:54 gibba011 bash[31774]: cluster 2021-03-10T15:50:53.601064+0000 mgr.gibba032.whopeo (mgr.24625) 6246 : cluster [DBG] pgmap v3551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:56 gibba032 bash[24004]: cluster 2021-03-10T15:50:55.602438+0000 mgr.gibba032.whopeo (mgr.24625) 6247 : cluster [DBG] pgmap v3552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:56 gibba011 bash[30791]: cluster 2021-03-10T15:50:55.602438+0000 mgr.gibba032.whopeo (mgr.24625) 6247 : cluster [DBG] pgmap v3552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:56 gibba011 bash[31774]: cluster 2021-03-10T15:50:55.602438+0000 mgr.gibba032.whopeo (mgr.24625) 6247 : cluster [DBG] pgmap v3552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.732999+0000 mon.a (mon.0) 2324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:50:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.733696+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:50:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.768285+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:50:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.769097+0000 mon.a (mon.0) 2327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:50:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.769890+0000 mon.a (mon.0) 2328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:50:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:57 gibba032 bash[24004]: audit 2021-03-10T15:50:56.770727+0000 mon.a (mon.0) 2329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:50:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.732999+0000 mon.a (mon.0) 2324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:50:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.733696+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:50:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.768285+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:50:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.769097+0000 mon.a (mon.0) 2327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:50:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.769890+0000 mon.a (mon.0) 2328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:50:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[30791]: audit 2021-03-10T15:50:56.770727+0000 mon.a (mon.0) 2329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:50:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.732999+0000 mon.a (mon.0) 2324 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:50:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.733696+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:50:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.768285+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:50:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.769097+0000 mon.a (mon.0) 2327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:50:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.769890+0000 mon.a (mon.0) 2328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:50:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:57 gibba011 bash[31774]: audit 2021-03-10T15:50:56.770727+0000 mon.a (mon.0) 2329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:50:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:50:58 gibba032 bash[24004]: cluster 2021-03-10T15:50:57.604315+0000 mgr.gibba032.whopeo (mgr.24625) 6248 : cluster [DBG] pgmap v3553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:50:58 gibba011 bash[30791]: cluster 2021-03-10T15:50:57.604315+0000 mgr.gibba032.whopeo (mgr.24625) 6248 : cluster [DBG] pgmap v3553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:50:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:50:58 gibba011 bash[31774]: cluster 2021-03-10T15:50:57.604315+0000 mgr.gibba032.whopeo (mgr.24625) 6248 : cluster [DBG] pgmap v3553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:00 gibba032 bash[24004]: cluster 2021-03-10T15:50:59.605693+0000 mgr.gibba032.whopeo (mgr.24625) 6249 : cluster [DBG] pgmap v3554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:00 gibba011 bash[30791]: cluster 2021-03-10T15:50:59.605693+0000 mgr.gibba032.whopeo (mgr.24625) 6249 : cluster [DBG] pgmap v3554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:00 gibba011 bash[31774]: cluster 2021-03-10T15:50:59.605693+0000 mgr.gibba032.whopeo (mgr.24625) 6249 : cluster [DBG] pgmap v3554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:02 gibba032 bash[24004]: cluster 2021-03-10T15:51:01.607387+0000 mgr.gibba032.whopeo (mgr.24625) 6250 : cluster [DBG] pgmap v3555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:02 gibba011 bash[30791]: cluster 2021-03-10T15:51:01.607387+0000 mgr.gibba032.whopeo (mgr.24625) 6250 : cluster [DBG] pgmap v3555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:02 gibba011 bash[31774]: cluster 2021-03-10T15:51:01.607387+0000 mgr.gibba032.whopeo (mgr.24625) 6250 : cluster [DBG] pgmap v3555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:04 gibba032 bash[24004]: cluster 2021-03-10T15:51:03.608950+0000 mgr.gibba032.whopeo (mgr.24625) 6251 : cluster [DBG] pgmap v3556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:04 gibba011 bash[30791]: cluster 2021-03-10T15:51:03.608950+0000 mgr.gibba032.whopeo (mgr.24625) 6251 : cluster [DBG] pgmap v3556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:04 gibba011 bash[31774]: cluster 2021-03-10T15:51:03.608950+0000 mgr.gibba032.whopeo (mgr.24625) 6251 : cluster [DBG] pgmap v3556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:06 gibba032 bash[24004]: cluster 2021-03-10T15:51:05.610436+0000 mgr.gibba032.whopeo (mgr.24625) 6252 : cluster [DBG] pgmap v3557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:06 gibba011 bash[30791]: cluster 2021-03-10T15:51:05.610436+0000 mgr.gibba032.whopeo (mgr.24625) 6252 : cluster [DBG] pgmap v3557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:06 gibba011 bash[31774]: cluster 2021-03-10T15:51:05.610436+0000 mgr.gibba032.whopeo (mgr.24625) 6252 : cluster [DBG] pgmap v3557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:08 gibba032 bash[24004]: cluster 2021-03-10T15:51:07.612274+0000 mgr.gibba032.whopeo (mgr.24625) 6253 : cluster [DBG] pgmap v3558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:08 gibba011 bash[30791]: cluster 2021-03-10T15:51:07.612274+0000 mgr.gibba032.whopeo (mgr.24625) 6253 : cluster [DBG] pgmap v3558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:08 gibba011 bash[31774]: cluster 2021-03-10T15:51:07.612274+0000 mgr.gibba032.whopeo (mgr.24625) 6253 : cluster [DBG] pgmap v3558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:09 gibba032 bash[24004]: cephadm 2021-03-10T15:51:08.489132+0000 mgr.gibba032.whopeo (mgr.24625) 6254 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:09 gibba011 bash[30791]: cephadm 2021-03-10T15:51:08.489132+0000 mgr.gibba032.whopeo (mgr.24625) 6254 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:09 gibba011 bash[31774]: cephadm 2021-03-10T15:51:08.489132+0000 mgr.gibba032.whopeo (mgr.24625) 6254 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:10.979 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:51:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:10 gibba032 bash[24004]: cluster 2021-03-10T15:51:09.613782+0000 mgr.gibba032.whopeo (mgr.24625) 6255 : cluster [DBG] pgmap v3559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:10 gibba011 bash[30791]: cluster 2021-03-10T15:51:09.613782+0000 mgr.gibba032.whopeo (mgr.24625) 6255 : cluster [DBG] pgmap v3559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:10 gibba011 bash[31774]: cluster 2021-03-10T15:51:09.613782+0000 mgr.gibba032.whopeo (mgr.24625) 6255 : cluster [DBG] pgmap v3559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:11.207 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:51:11.207 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (117m) 101s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:51:11.208 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 37s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:51:11.208 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (118m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:51:11.208 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (118m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:51:11.208 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (117m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:51:11.208 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (117m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:51:11.209 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (116m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:51:11.209 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (117m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:51:11.209 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 101s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:51:11.209 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 37s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:51:11.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (116m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:51:11.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (115m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:51:11.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (115m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:51:11.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (114m) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:51:11.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (114m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:51:11.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (113m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:51:11.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (113m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:51:11.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (113m) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:51:11.212 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 37s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:51:11.212 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 101s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:51:11.454 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:51:11.455 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:51:11.455 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:51:11.455 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:11.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:51:11.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:51:11.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:11.456 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:51:11.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:51:11.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:11.457 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:51:11.457 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:51:11.457 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:51:11.458 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:51:11.458 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:51:12.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:11 gibba032 bash[24004]: audit 2021-03-10T15:51:11.454007+0000 mon.a (mon.0) 2330 : audit [DBG] from='client.? 172.21.2.111:0/2759264036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:11 gibba011 bash[30791]: audit 2021-03-10T15:51:11.454007+0000 mon.a (mon.0) 2330 : audit [DBG] from='client.? 172.21.2.111:0/2759264036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:12.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:11 gibba011 bash[31774]: audit 2021-03-10T15:51:11.454007+0000 mon.a (mon.0) 2330 : audit [DBG] from='client.? 172.21.2.111:0/2759264036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:12 gibba032 bash[24004]: audit 2021-03-10T15:51:10.967374+0000 mgr.gibba032.whopeo (mgr.24625) 6256 : audit [DBG] from='client.58231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:12 gibba032 bash[24004]: audit 2021-03-10T15:51:11.193060+0000 mgr.gibba032.whopeo (mgr.24625) 6257 : audit [DBG] from='client.48255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:12 gibba032 bash[24004]: cluster 2021-03-10T15:51:11.615470+0000 mgr.gibba032.whopeo (mgr.24625) 6258 : cluster [DBG] pgmap v3560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[30791]: audit 2021-03-10T15:51:10.967374+0000 mgr.gibba032.whopeo (mgr.24625) 6256 : audit [DBG] from='client.58231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[30791]: audit 2021-03-10T15:51:11.193060+0000 mgr.gibba032.whopeo (mgr.24625) 6257 : audit [DBG] from='client.48255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[30791]: cluster 2021-03-10T15:51:11.615470+0000 mgr.gibba032.whopeo (mgr.24625) 6258 : cluster [DBG] pgmap v3560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[31774]: audit 2021-03-10T15:51:10.967374+0000 mgr.gibba032.whopeo (mgr.24625) 6256 : audit [DBG] from='client.58231 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[31774]: audit 2021-03-10T15:51:11.193060+0000 mgr.gibba032.whopeo (mgr.24625) 6257 : audit [DBG] from='client.48255 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:12 gibba011 bash[31774]: cluster 2021-03-10T15:51:11.615470+0000 mgr.gibba032.whopeo (mgr.24625) 6258 : cluster [DBG] pgmap v3560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:14 gibba032 bash[24004]: cluster 2021-03-10T15:51:13.616884+0000 mgr.gibba032.whopeo (mgr.24625) 6259 : cluster [DBG] pgmap v3561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:14 gibba011 bash[30791]: cluster 2021-03-10T15:51:13.616884+0000 mgr.gibba032.whopeo (mgr.24625) 6259 : cluster [DBG] pgmap v3561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:14 gibba011 bash[31774]: cluster 2021-03-10T15:51:13.616884+0000 mgr.gibba032.whopeo (mgr.24625) 6259 : cluster [DBG] pgmap v3561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:16 gibba032 bash[24004]: cluster 2021-03-10T15:51:15.618365+0000 mgr.gibba032.whopeo (mgr.24625) 6260 : cluster [DBG] pgmap v3562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:16 gibba011 bash[30791]: cluster 2021-03-10T15:51:15.618365+0000 mgr.gibba032.whopeo (mgr.24625) 6260 : cluster [DBG] pgmap v3562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:16 gibba011 bash[31774]: cluster 2021-03-10T15:51:15.618365+0000 mgr.gibba032.whopeo (mgr.24625) 6260 : cluster [DBG] pgmap v3562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:18 gibba032 bash[24004]: cluster 2021-03-10T15:51:17.619848+0000 mgr.gibba032.whopeo (mgr.24625) 6261 : cluster [DBG] pgmap v3563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:18 gibba011 bash[30791]: cluster 2021-03-10T15:51:17.619848+0000 mgr.gibba032.whopeo (mgr.24625) 6261 : cluster [DBG] pgmap v3563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:18 gibba011 bash[31774]: cluster 2021-03-10T15:51:17.619848+0000 mgr.gibba032.whopeo (mgr.24625) 6261 : cluster [DBG] pgmap v3563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:20 gibba032 bash[24004]: cluster 2021-03-10T15:51:19.621198+0000 mgr.gibba032.whopeo (mgr.24625) 6262 : cluster [DBG] pgmap v3564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:20 gibba011 bash[30791]: cluster 2021-03-10T15:51:19.621198+0000 mgr.gibba032.whopeo (mgr.24625) 6262 : cluster [DBG] pgmap v3564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:20 gibba011 bash[31774]: cluster 2021-03-10T15:51:19.621198+0000 mgr.gibba032.whopeo (mgr.24625) 6262 : cluster [DBG] pgmap v3564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:22 gibba032 bash[24004]: cluster 2021-03-10T15:51:21.622821+0000 mgr.gibba032.whopeo (mgr.24625) 6263 : cluster [DBG] pgmap v3565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:22 gibba011 bash[30791]: cluster 2021-03-10T15:51:21.622821+0000 mgr.gibba032.whopeo (mgr.24625) 6263 : cluster [DBG] pgmap v3565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:23.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:22 gibba011 bash[31774]: cluster 2021-03-10T15:51:21.622821+0000 mgr.gibba032.whopeo (mgr.24625) 6263 : cluster [DBG] pgmap v3565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:24 gibba032 bash[24004]: cephadm 2021-03-10T15:51:23.505170+0000 mgr.gibba032.whopeo (mgr.24625) 6264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:24 gibba032 bash[24004]: cluster 2021-03-10T15:51:23.624227+0000 mgr.gibba032.whopeo (mgr.24625) 6265 : cluster [DBG] pgmap v3566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:24 gibba011 bash[30791]: cephadm 2021-03-10T15:51:23.505170+0000 mgr.gibba032.whopeo (mgr.24625) 6264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:24 gibba011 bash[30791]: cluster 2021-03-10T15:51:23.624227+0000 mgr.gibba032.whopeo (mgr.24625) 6265 : cluster [DBG] pgmap v3566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:24 gibba011 bash[31774]: cephadm 2021-03-10T15:51:23.505170+0000 mgr.gibba032.whopeo (mgr.24625) 6264 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:24.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:24 gibba011 bash[31774]: cluster 2021-03-10T15:51:23.624227+0000 mgr.gibba032.whopeo (mgr.24625) 6265 : cluster [DBG] pgmap v3566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:26 gibba032 bash[24004]: cluster 2021-03-10T15:51:25.625534+0000 mgr.gibba032.whopeo (mgr.24625) 6266 : cluster [DBG] pgmap v3567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:26 gibba011 bash[30791]: cluster 2021-03-10T15:51:25.625534+0000 mgr.gibba032.whopeo (mgr.24625) 6266 : cluster [DBG] pgmap v3567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:26 gibba011 bash[31774]: cluster 2021-03-10T15:51:25.625534+0000 mgr.gibba032.whopeo (mgr.24625) 6266 : cluster [DBG] pgmap v3567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:28 gibba032 bash[24004]: cluster 2021-03-10T15:51:27.627191+0000 mgr.gibba032.whopeo (mgr.24625) 6267 : cluster [DBG] pgmap v3568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:28 gibba011 bash[30791]: cluster 2021-03-10T15:51:27.627191+0000 mgr.gibba032.whopeo (mgr.24625) 6267 : cluster [DBG] pgmap v3568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:28 gibba011 bash[31774]: cluster 2021-03-10T15:51:27.627191+0000 mgr.gibba032.whopeo (mgr.24625) 6267 : cluster [DBG] pgmap v3568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:30 gibba032 bash[24004]: cluster 2021-03-10T15:51:29.628752+0000 mgr.gibba032.whopeo (mgr.24625) 6268 : cluster [DBG] pgmap v3569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:30 gibba011 bash[30791]: cluster 2021-03-10T15:51:29.628752+0000 mgr.gibba032.whopeo (mgr.24625) 6268 : cluster [DBG] pgmap v3569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:30 gibba011 bash[31774]: cluster 2021-03-10T15:51:29.628752+0000 mgr.gibba032.whopeo (mgr.24625) 6268 : cluster [DBG] pgmap v3569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:32 gibba032 bash[24004]: cluster 2021-03-10T15:51:31.630520+0000 mgr.gibba032.whopeo (mgr.24625) 6269 : cluster [DBG] pgmap v3570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:32 gibba011 bash[30791]: cluster 2021-03-10T15:51:31.630520+0000 mgr.gibba032.whopeo (mgr.24625) 6269 : cluster [DBG] pgmap v3570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:33.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:32 gibba011 bash[31774]: cluster 2021-03-10T15:51:31.630520+0000 mgr.gibba032.whopeo (mgr.24625) 6269 : cluster [DBG] pgmap v3570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:34 gibba032 bash[24004]: cluster 2021-03-10T15:51:33.631890+0000 mgr.gibba032.whopeo (mgr.24625) 6270 : cluster [DBG] pgmap v3571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:34 gibba011 bash[30791]: cluster 2021-03-10T15:51:33.631890+0000 mgr.gibba032.whopeo (mgr.24625) 6270 : cluster [DBG] pgmap v3571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:34 gibba011 bash[31774]: cluster 2021-03-10T15:51:33.631890+0000 mgr.gibba032.whopeo (mgr.24625) 6270 : cluster [DBG] pgmap v3571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:36 gibba032 bash[24004]: cluster 2021-03-10T15:51:35.633173+0000 mgr.gibba032.whopeo (mgr.24625) 6271 : cluster [DBG] pgmap v3572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:36 gibba011 bash[30791]: cluster 2021-03-10T15:51:35.633173+0000 mgr.gibba032.whopeo (mgr.24625) 6271 : cluster [DBG] pgmap v3572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:36 gibba011 bash[31774]: cluster 2021-03-10T15:51:35.633173+0000 mgr.gibba032.whopeo (mgr.24625) 6271 : cluster [DBG] pgmap v3572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:38 gibba032 bash[24004]: cluster 2021-03-10T15:51:37.634838+0000 mgr.gibba032.whopeo (mgr.24625) 6272 : cluster [DBG] pgmap v3573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:38 gibba011 bash[30791]: cluster 2021-03-10T15:51:37.634838+0000 mgr.gibba032.whopeo (mgr.24625) 6272 : cluster [DBG] pgmap v3573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:38 gibba011 bash[31774]: cluster 2021-03-10T15:51:37.634838+0000 mgr.gibba032.whopeo (mgr.24625) 6272 : cluster [DBG] pgmap v3573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.513911+0000 mgr.gibba032.whopeo (mgr.24625) 6273 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:51:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.515703+0000 mgr.gibba032.whopeo (mgr.24625) 6274 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:51:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.742549+0000 mon.a (mon.0) 2331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:51:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.743620+0000 mgr.gibba032.whopeo (mgr.24625) 6275 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:51:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.745786+0000 mgr.gibba032.whopeo (mgr.24625) 6276 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:51:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.746043+0000 mon.a (mon.0) 2332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:51:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.747118+0000 mgr.gibba032.whopeo (mgr.24625) 6277 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:51:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.747158+0000 mgr.gibba032.whopeo (mgr.24625) 6278 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:51:40.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.747626+0000 mon.a (mon.0) 2333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.748613+0000 mgr.gibba032.whopeo (mgr.24625) 6279 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:51:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.748651+0000 mgr.gibba032.whopeo (mgr.24625) 6280 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:51:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.749191+0000 mon.a (mon.0) 2334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.749801+0000 mgr.gibba032.whopeo (mgr.24625) 6281 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:51:40.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.750577+0000 mon.a (mon.0) 2335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.750986+0000 mgr.gibba032.whopeo (mgr.24625) 6282 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:51:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.751023+0000 mgr.gibba032.whopeo (mgr.24625) 6283 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:51:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.752243+0000 mgr.gibba032.whopeo (mgr.24625) 6284 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:51:40.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.752562+0000 mgr.gibba032.whopeo (mgr.24625) 6285 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:51:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.753030+0000 mon.a (mon.0) 2336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.753929+0000 mgr.gibba032.whopeo (mgr.24625) 6286 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:51:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.753966+0000 mgr.gibba032.whopeo (mgr.24625) 6287 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:51:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: cephadm 2021-03-10T15:51:38.754073+0000 mgr.gibba032.whopeo (mgr.24625) 6288 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:40.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:39 gibba032 bash[24004]: audit 2021-03-10T15:51:38.754400+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.513911+0000 mgr.gibba032.whopeo (mgr.24625) 6273 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:51:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.515703+0000 mgr.gibba032.whopeo (mgr.24625) 6274 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:51:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.742549+0000 mon.a (mon.0) 2331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:51:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.743620+0000 mgr.gibba032.whopeo (mgr.24625) 6275 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:51:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.745786+0000 mgr.gibba032.whopeo (mgr.24625) 6276 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:51:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.746043+0000 mon.a (mon.0) 2332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:51:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.747118+0000 mgr.gibba032.whopeo (mgr.24625) 6277 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:51:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.747158+0000 mgr.gibba032.whopeo (mgr.24625) 6278 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:51:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.747626+0000 mon.a (mon.0) 2333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.748613+0000 mgr.gibba032.whopeo (mgr.24625) 6279 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:51:40.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.748651+0000 mgr.gibba032.whopeo (mgr.24625) 6280 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:51:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.749191+0000 mon.a (mon.0) 2334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.749801+0000 mgr.gibba032.whopeo (mgr.24625) 6281 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:51:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.750577+0000 mon.a (mon.0) 2335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.750986+0000 mgr.gibba032.whopeo (mgr.24625) 6282 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:51:40.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.751023+0000 mgr.gibba032.whopeo (mgr.24625) 6283 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:51:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.752243+0000 mgr.gibba032.whopeo (mgr.24625) 6284 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:51:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.752562+0000 mgr.gibba032.whopeo (mgr.24625) 6285 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:51:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.753030+0000 mon.a (mon.0) 2336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.753929+0000 mgr.gibba032.whopeo (mgr.24625) 6286 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:51:40.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.753966+0000 mgr.gibba032.whopeo (mgr.24625) 6287 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:51:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: cephadm 2021-03-10T15:51:38.754073+0000 mgr.gibba032.whopeo (mgr.24625) 6288 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:40.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[30791]: audit 2021-03-10T15:51:38.754400+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.513911+0000 mgr.gibba032.whopeo (mgr.24625) 6273 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:51:40.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.515703+0000 mgr.gibba032.whopeo (mgr.24625) 6274 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:51:40.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.742549+0000 mon.a (mon.0) 2331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:51:40.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.743620+0000 mgr.gibba032.whopeo (mgr.24625) 6275 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:51:40.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.745786+0000 mgr.gibba032.whopeo (mgr.24625) 6276 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:51:40.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.746043+0000 mon.a (mon.0) 2332 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:51:40.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.747118+0000 mgr.gibba032.whopeo (mgr.24625) 6277 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:51:40.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.747158+0000 mgr.gibba032.whopeo (mgr.24625) 6278 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:51:40.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.747626+0000 mon.a (mon.0) 2333 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.748613+0000 mgr.gibba032.whopeo (mgr.24625) 6279 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:51:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.748651+0000 mgr.gibba032.whopeo (mgr.24625) 6280 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:51:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.749191+0000 mon.a (mon.0) 2334 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.749801+0000 mgr.gibba032.whopeo (mgr.24625) 6281 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:51:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.750577+0000 mon.a (mon.0) 2335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.750986+0000 mgr.gibba032.whopeo (mgr.24625) 6282 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:51:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.751023+0000 mgr.gibba032.whopeo (mgr.24625) 6283 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:51:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.752243+0000 mgr.gibba032.whopeo (mgr.24625) 6284 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:51:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.752562+0000 mgr.gibba032.whopeo (mgr.24625) 6285 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:51:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.753030+0000 mon.a (mon.0) 2336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:40.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.753929+0000 mgr.gibba032.whopeo (mgr.24625) 6286 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:51:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.753966+0000 mgr.gibba032.whopeo (mgr.24625) 6287 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:51:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: cephadm 2021-03-10T15:51:38.754073+0000 mgr.gibba032.whopeo (mgr.24625) 6288 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:40.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:39 gibba011 bash[31774]: audit 2021-03-10T15:51:38.754400+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:40 gibba011 bash[30791]: cluster 2021-03-10T15:51:39.636237+0000 mgr.gibba032.whopeo (mgr.24625) 6289 : cluster [DBG] pgmap v3574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:41.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:40 gibba011 bash[31774]: cluster 2021-03-10T15:51:39.636237+0000 mgr.gibba032.whopeo (mgr.24625) 6289 : cluster [DBG] pgmap v3574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:40 gibba032 bash[24004]: cluster 2021-03-10T15:51:39.636237+0000 mgr.gibba032.whopeo (mgr.24625) 6289 : cluster [DBG] pgmap v3574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:41.697 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:51:41.926 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:51:41.926 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (118m) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:51:41.927 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 68s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:51:41.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (118m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:51:41.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (118m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:51:41.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:51:41.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (117m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:51:41.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (117m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:51:41.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (117m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:51:41.928 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:51:41.928 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 68s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:51:41.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:51:41.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:51:41.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:51:41.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:51:41.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (114m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:51:41.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (114m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:51:41.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (114m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:51:41.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (113m) 68s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:51:41.931 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 68s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:51:41.931 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:51:42.208 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:51:42.208 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:51:42.208 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:51:42.209 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:42.209 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:51:42.209 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:51:42.209 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:42.210 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:51:42.210 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:51:42.210 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:51:42.210 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:51:42.210 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:51:42.211 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:51:42.211 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:51:42.211 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:51:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[30791]: cluster 2021-03-10T15:51:41.637768+0000 mgr.gibba032.whopeo (mgr.24625) 6290 : cluster [DBG] pgmap v3575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[30791]: audit 2021-03-10T15:51:41.686275+0000 mgr.gibba032.whopeo (mgr.24625) 6291 : audit [DBG] from='client.58249 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[30791]: audit 2021-03-10T15:51:41.911670+0000 mgr.gibba032.whopeo (mgr.24625) 6292 : audit [DBG] from='client.48273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[30791]: audit 2021-03-10T15:51:42.207275+0000 mon.c (mon.1) 121 : audit [DBG] from='client.? 172.21.2.111:0/539542375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[31774]: cluster 2021-03-10T15:51:41.637768+0000 mgr.gibba032.whopeo (mgr.24625) 6290 : cluster [DBG] pgmap v3575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[31774]: audit 2021-03-10T15:51:41.686275+0000 mgr.gibba032.whopeo (mgr.24625) 6291 : audit [DBG] from='client.58249 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[31774]: audit 2021-03-10T15:51:41.911670+0000 mgr.gibba032.whopeo (mgr.24625) 6292 : audit [DBG] from='client.48273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:42 gibba011 bash[31774]: audit 2021-03-10T15:51:42.207275+0000 mon.c (mon.1) 121 : audit [DBG] from='client.? 172.21.2.111:0/539542375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:42 gibba032 bash[24004]: cluster 2021-03-10T15:51:41.637768+0000 mgr.gibba032.whopeo (mgr.24625) 6290 : cluster [DBG] pgmap v3575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:42 gibba032 bash[24004]: audit 2021-03-10T15:51:41.686275+0000 mgr.gibba032.whopeo (mgr.24625) 6291 : audit [DBG] from='client.58249 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:42 gibba032 bash[24004]: audit 2021-03-10T15:51:41.911670+0000 mgr.gibba032.whopeo (mgr.24625) 6292 : audit [DBG] from='client.48273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:51:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:42 gibba032 bash[24004]: audit 2021-03-10T15:51:42.207275+0000 mon.c (mon.1) 121 : audit [DBG] from='client.? 172.21.2.111:0/539542375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:51:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:44 gibba032 bash[24004]: cluster 2021-03-10T15:51:43.639126+0000 mgr.gibba032.whopeo (mgr.24625) 6293 : cluster [DBG] pgmap v3576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:44 gibba011 bash[30791]: cluster 2021-03-10T15:51:43.639126+0000 mgr.gibba032.whopeo (mgr.24625) 6293 : cluster [DBG] pgmap v3576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:44 gibba011 bash[31774]: cluster 2021-03-10T15:51:43.639126+0000 mgr.gibba032.whopeo (mgr.24625) 6293 : cluster [DBG] pgmap v3576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:46 gibba032 bash[24004]: cluster 2021-03-10T15:51:45.640677+0000 mgr.gibba032.whopeo (mgr.24625) 6294 : cluster [DBG] pgmap v3577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:46 gibba011 bash[30791]: cluster 2021-03-10T15:51:45.640677+0000 mgr.gibba032.whopeo (mgr.24625) 6294 : cluster [DBG] pgmap v3577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:46 gibba011 bash[31774]: cluster 2021-03-10T15:51:45.640677+0000 mgr.gibba032.whopeo (mgr.24625) 6294 : cluster [DBG] pgmap v3577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:48 gibba032 bash[24004]: cluster 2021-03-10T15:51:47.642322+0000 mgr.gibba032.whopeo (mgr.24625) 6295 : cluster [DBG] pgmap v3578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:48 gibba011 bash[30791]: cluster 2021-03-10T15:51:47.642322+0000 mgr.gibba032.whopeo (mgr.24625) 6295 : cluster [DBG] pgmap v3578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:48 gibba011 bash[31774]: cluster 2021-03-10T15:51:47.642322+0000 mgr.gibba032.whopeo (mgr.24625) 6295 : cluster [DBG] pgmap v3578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:50 gibba032 bash[24004]: cluster 2021-03-10T15:51:49.643750+0000 mgr.gibba032.whopeo (mgr.24625) 6296 : cluster [DBG] pgmap v3579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:50 gibba011 bash[30791]: cluster 2021-03-10T15:51:49.643750+0000 mgr.gibba032.whopeo (mgr.24625) 6296 : cluster [DBG] pgmap v3579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:50 gibba011 bash[31774]: cluster 2021-03-10T15:51:49.643750+0000 mgr.gibba032.whopeo (mgr.24625) 6296 : cluster [DBG] pgmap v3579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:53.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:52 gibba032 bash[24004]: cluster 2021-03-10T15:51:51.645275+0000 mgr.gibba032.whopeo (mgr.24625) 6297 : cluster [DBG] pgmap v3580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:53.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:52 gibba011 bash[30791]: cluster 2021-03-10T15:51:51.645275+0000 mgr.gibba032.whopeo (mgr.24625) 6297 : cluster [DBG] pgmap v3580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:53.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:52 gibba011 bash[31774]: cluster 2021-03-10T15:51:51.645275+0000 mgr.gibba032.whopeo (mgr.24625) 6297 : cluster [DBG] pgmap v3580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:54 gibba032 bash[24004]: cluster 2021-03-10T15:51:53.646892+0000 mgr.gibba032.whopeo (mgr.24625) 6298 : cluster [DBG] pgmap v3581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:54 gibba032 bash[24004]: cephadm 2021-03-10T15:51:53.770054+0000 mgr.gibba032.whopeo (mgr.24625) 6299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:54 gibba011 bash[30791]: cluster 2021-03-10T15:51:53.646892+0000 mgr.gibba032.whopeo (mgr.24625) 6298 : cluster [DBG] pgmap v3581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:54.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:54 gibba011 bash[30791]: cephadm 2021-03-10T15:51:53.770054+0000 mgr.gibba032.whopeo (mgr.24625) 6299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:54 gibba011 bash[31774]: cluster 2021-03-10T15:51:53.646892+0000 mgr.gibba032.whopeo (mgr.24625) 6298 : cluster [DBG] pgmap v3581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:54 gibba011 bash[31774]: cephadm 2021-03-10T15:51:53.770054+0000 mgr.gibba032.whopeo (mgr.24625) 6299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:51:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:56 gibba032 bash[24004]: cluster 2021-03-10T15:51:55.648623+0000 mgr.gibba032.whopeo (mgr.24625) 6300 : cluster [DBG] pgmap v3582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:56 gibba011 bash[30791]: cluster 2021-03-10T15:51:55.648623+0000 mgr.gibba032.whopeo (mgr.24625) 6300 : cluster [DBG] pgmap v3582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:56 gibba011 bash[31774]: cluster 2021-03-10T15:51:55.648623+0000 mgr.gibba032.whopeo (mgr.24625) 6300 : cluster [DBG] pgmap v3582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.736854+0000 mon.a (mon.0) 2338 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:51:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.737035+0000 mon.a (mon.0) 2339 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:51:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.775827+0000 mon.a (mon.0) 2340 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:51:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.776650+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:51:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.777483+0000 mon.a (mon.0) 2342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:51:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:57 gibba032 bash[24004]: audit 2021-03-10T15:51:56.778295+0000 mon.a (mon.0) 2343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:51:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.736854+0000 mon.a (mon.0) 2338 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:51:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.737035+0000 mon.a (mon.0) 2339 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:51:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.775827+0000 mon.a (mon.0) 2340 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:51:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.776650+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:51:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.777483+0000 mon.a (mon.0) 2342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:51:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[30791]: audit 2021-03-10T15:51:56.778295+0000 mon.a (mon.0) 2343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:51:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.736854+0000 mon.a (mon.0) 2338 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:51:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.737035+0000 mon.a (mon.0) 2339 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:51:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.775827+0000 mon.a (mon.0) 2340 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:51:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.776650+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T15:51:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.777483+0000 mon.a (mon.0) 2342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:51:58.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:57 gibba011 bash[31774]: audit 2021-03-10T15:51:56.778295+0000 mon.a (mon.0) 2343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T15:51:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:51:58 gibba032 bash[24004]: cluster 2021-03-10T15:51:57.650342+0000 mgr.gibba032.whopeo (mgr.24625) 6301 : cluster [DBG] pgmap v3583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:51:58 gibba011 bash[30791]: cluster 2021-03-10T15:51:57.650342+0000 mgr.gibba032.whopeo (mgr.24625) 6301 : cluster [DBG] pgmap v3583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:51:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:51:58 gibba011 bash[31774]: cluster 2021-03-10T15:51:57.650342+0000 mgr.gibba032.whopeo (mgr.24625) 6301 : cluster [DBG] pgmap v3583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:00 gibba032 bash[24004]: cluster 2021-03-10T15:51:59.651982+0000 mgr.gibba032.whopeo (mgr.24625) 6302 : cluster [DBG] pgmap v3584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:00 gibba011 bash[30791]: cluster 2021-03-10T15:51:59.651982+0000 mgr.gibba032.whopeo (mgr.24625) 6302 : cluster [DBG] pgmap v3584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:00 gibba011 bash[31774]: cluster 2021-03-10T15:51:59.651982+0000 mgr.gibba032.whopeo (mgr.24625) 6302 : cluster [DBG] pgmap v3584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:03.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:02 gibba032 bash[24004]: cluster 2021-03-10T15:52:01.653812+0000 mgr.gibba032.whopeo (mgr.24625) 6303 : cluster [DBG] pgmap v3585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:02 gibba011 bash[30791]: cluster 2021-03-10T15:52:01.653812+0000 mgr.gibba032.whopeo (mgr.24625) 6303 : cluster [DBG] pgmap v3585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:02 gibba011 bash[31774]: cluster 2021-03-10T15:52:01.653812+0000 mgr.gibba032.whopeo (mgr.24625) 6303 : cluster [DBG] pgmap v3585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:04 gibba032 bash[24004]: cluster 2021-03-10T15:52:03.655425+0000 mgr.gibba032.whopeo (mgr.24625) 6304 : cluster [DBG] pgmap v3586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:04 gibba011 bash[30791]: cluster 2021-03-10T15:52:03.655425+0000 mgr.gibba032.whopeo (mgr.24625) 6304 : cluster [DBG] pgmap v3586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:04 gibba011 bash[31774]: cluster 2021-03-10T15:52:03.655425+0000 mgr.gibba032.whopeo (mgr.24625) 6304 : cluster [DBG] pgmap v3586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:06 gibba032 bash[24004]: cluster 2021-03-10T15:52:05.655956+0000 mgr.gibba032.whopeo (mgr.24625) 6305 : cluster [DBG] pgmap v3587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:06 gibba011 bash[30791]: cluster 2021-03-10T15:52:05.655956+0000 mgr.gibba032.whopeo (mgr.24625) 6305 : cluster [DBG] pgmap v3587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:06 gibba011 bash[31774]: cluster 2021-03-10T15:52:05.655956+0000 mgr.gibba032.whopeo (mgr.24625) 6305 : cluster [DBG] pgmap v3587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:08 gibba032 bash[24004]: cluster 2021-03-10T15:52:07.657656+0000 mgr.gibba032.whopeo (mgr.24625) 6306 : cluster [DBG] pgmap v3588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:08 gibba011 bash[30791]: cluster 2021-03-10T15:52:07.657656+0000 mgr.gibba032.whopeo (mgr.24625) 6306 : cluster [DBG] pgmap v3588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:08 gibba011 bash[31774]: cluster 2021-03-10T15:52:07.657656+0000 mgr.gibba032.whopeo (mgr.24625) 6306 : cluster [DBG] pgmap v3588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:09 gibba032 bash[24004]: cephadm 2021-03-10T15:52:08.776154+0000 mgr.gibba032.whopeo (mgr.24625) 6307 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:10.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:09 gibba011 bash[30791]: cephadm 2021-03-10T15:52:08.776154+0000 mgr.gibba032.whopeo (mgr.24625) 6307 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:09 gibba011 bash[31774]: cephadm 2021-03-10T15:52:08.776154+0000 mgr.gibba032.whopeo (mgr.24625) 6307 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:10 gibba032 bash[24004]: cluster 2021-03-10T15:52:09.659235+0000 mgr.gibba032.whopeo (mgr.24625) 6308 : cluster [DBG] pgmap v3589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:10 gibba011 bash[30791]: cluster 2021-03-10T15:52:09.659235+0000 mgr.gibba032.whopeo (mgr.24625) 6308 : cluster [DBG] pgmap v3589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:10 gibba011 bash[31774]: cluster 2021-03-10T15:52:09.659235+0000 mgr.gibba032.whopeo (mgr.24625) 6308 : cluster [DBG] pgmap v3589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:12.451 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:52:12.676 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:52:12.677 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (118m) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:52:12.677 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 99s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:52:12.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (119m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:52:12.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (119m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:52:12.678 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:52:12.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:52:12.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (117m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:52:12.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:52:12.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:52:12.679 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 99s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:52:12.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (117m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:52:12.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:52:12.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:52:12.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:52:12.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (115m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:52:12.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (115m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:52:12.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (114m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:52:12.681 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (114m) 99s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:52:12.681 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 99s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:52:12.681 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:52:12.928 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:52:12.929 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:52:12.929 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:52:12.929 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:12.929 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:52:12.930 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:52:12.930 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:12.930 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:52:12.930 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:52:12.931 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:12.931 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:52:12.931 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:52:12.931 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:52:12.932 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:52:12.932 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:52:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:12 gibba032 bash[24004]: cluster 2021-03-10T15:52:11.660913+0000 mgr.gibba032.whopeo (mgr.24625) 6309 : cluster [DBG] pgmap v3590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:12 gibba011 bash[30791]: cluster 2021-03-10T15:52:11.660913+0000 mgr.gibba032.whopeo (mgr.24625) 6309 : cluster [DBG] pgmap v3590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:12 gibba011 bash[31774]: cluster 2021-03-10T15:52:11.660913+0000 mgr.gibba032.whopeo (mgr.24625) 6309 : cluster [DBG] pgmap v3590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[30791]: audit 2021-03-10T15:52:12.439811+0000 mgr.gibba032.whopeo (mgr.24625) 6310 : audit [DBG] from='client.58267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[30791]: audit 2021-03-10T15:52:12.662552+0000 mgr.gibba032.whopeo (mgr.24625) 6311 : audit [DBG] from='client.58273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[30791]: audit 2021-03-10T15:52:12.927973+0000 mon.a (mon.0) 2344 : audit [DBG] from='client.? 172.21.2.111:0/4120496040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:14.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[31774]: audit 2021-03-10T15:52:12.439811+0000 mgr.gibba032.whopeo (mgr.24625) 6310 : audit [DBG] from='client.58267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[31774]: audit 2021-03-10T15:52:12.662552+0000 mgr.gibba032.whopeo (mgr.24625) 6311 : audit [DBG] from='client.58273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:13 gibba011 bash[31774]: audit 2021-03-10T15:52:12.927973+0000 mon.a (mon.0) 2344 : audit [DBG] from='client.? 172.21.2.111:0/4120496040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:13 gibba032 bash[24004]: audit 2021-03-10T15:52:12.439811+0000 mgr.gibba032.whopeo (mgr.24625) 6310 : audit [DBG] from='client.58267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:13 gibba032 bash[24004]: audit 2021-03-10T15:52:12.662552+0000 mgr.gibba032.whopeo (mgr.24625) 6311 : audit [DBG] from='client.58273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:13 gibba032 bash[24004]: audit 2021-03-10T15:52:12.927973+0000 mon.a (mon.0) 2344 : audit [DBG] from='client.? 172.21.2.111:0/4120496040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:14 gibba011 bash[30791]: cluster 2021-03-10T15:52:13.662446+0000 mgr.gibba032.whopeo (mgr.24625) 6312 : cluster [DBG] pgmap v3591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:14 gibba011 bash[31774]: cluster 2021-03-10T15:52:13.662446+0000 mgr.gibba032.whopeo (mgr.24625) 6312 : cluster [DBG] pgmap v3591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:15.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:14 gibba032 bash[24004]: cluster 2021-03-10T15:52:13.662446+0000 mgr.gibba032.whopeo (mgr.24625) 6312 : cluster [DBG] pgmap v3591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:16 gibba011 bash[30791]: cluster 2021-03-10T15:52:15.663751+0000 mgr.gibba032.whopeo (mgr.24625) 6313 : cluster [DBG] pgmap v3592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:16 gibba011 bash[31774]: cluster 2021-03-10T15:52:15.663751+0000 mgr.gibba032.whopeo (mgr.24625) 6313 : cluster [DBG] pgmap v3592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:16 gibba032 bash[24004]: cluster 2021-03-10T15:52:15.663751+0000 mgr.gibba032.whopeo (mgr.24625) 6313 : cluster [DBG] pgmap v3592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:18 gibba011 bash[30791]: cluster 2021-03-10T15:52:17.665599+0000 mgr.gibba032.whopeo (mgr.24625) 6314 : cluster [DBG] pgmap v3593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:18 gibba011 bash[31774]: cluster 2021-03-10T15:52:17.665599+0000 mgr.gibba032.whopeo (mgr.24625) 6314 : cluster [DBG] pgmap v3593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:18 gibba032 bash[24004]: cluster 2021-03-10T15:52:17.665599+0000 mgr.gibba032.whopeo (mgr.24625) 6314 : cluster [DBG] pgmap v3593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:20 gibba011 bash[30791]: cluster 2021-03-10T15:52:19.667084+0000 mgr.gibba032.whopeo (mgr.24625) 6315 : cluster [DBG] pgmap v3594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:20 gibba011 bash[31774]: cluster 2021-03-10T15:52:19.667084+0000 mgr.gibba032.whopeo (mgr.24625) 6315 : cluster [DBG] pgmap v3594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:20 gibba032 bash[24004]: cluster 2021-03-10T15:52:19.667084+0000 mgr.gibba032.whopeo (mgr.24625) 6315 : cluster [DBG] pgmap v3594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:22 gibba011 bash[30791]: cluster 2021-03-10T15:52:21.668759+0000 mgr.gibba032.whopeo (mgr.24625) 6316 : cluster [DBG] pgmap v3595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:22 gibba011 bash[31774]: cluster 2021-03-10T15:52:21.668759+0000 mgr.gibba032.whopeo (mgr.24625) 6316 : cluster [DBG] pgmap v3595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:22 gibba032 bash[24004]: cluster 2021-03-10T15:52:21.668759+0000 mgr.gibba032.whopeo (mgr.24625) 6316 : cluster [DBG] pgmap v3595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:24 gibba032 bash[24004]: cluster 2021-03-10T15:52:23.670248+0000 mgr.gibba032.whopeo (mgr.24625) 6317 : cluster [DBG] pgmap v3596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:24 gibba032 bash[24004]: cephadm 2021-03-10T15:52:23.792236+0000 mgr.gibba032.whopeo (mgr.24625) 6318 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:24 gibba011 bash[30791]: cluster 2021-03-10T15:52:23.670248+0000 mgr.gibba032.whopeo (mgr.24625) 6317 : cluster [DBG] pgmap v3596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:24.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:24 gibba011 bash[30791]: cephadm 2021-03-10T15:52:23.792236+0000 mgr.gibba032.whopeo (mgr.24625) 6318 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:24 gibba011 bash[31774]: cluster 2021-03-10T15:52:23.670248+0000 mgr.gibba032.whopeo (mgr.24625) 6317 : cluster [DBG] pgmap v3596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:24 gibba011 bash[31774]: cephadm 2021-03-10T15:52:23.792236+0000 mgr.gibba032.whopeo (mgr.24625) 6318 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:26 gibba032 bash[24004]: cluster 2021-03-10T15:52:25.670786+0000 mgr.gibba032.whopeo (mgr.24625) 6319 : cluster [DBG] pgmap v3597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:26 gibba011 bash[30791]: cluster 2021-03-10T15:52:25.670786+0000 mgr.gibba032.whopeo (mgr.24625) 6319 : cluster [DBG] pgmap v3597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:26 gibba011 bash[31774]: cluster 2021-03-10T15:52:25.670786+0000 mgr.gibba032.whopeo (mgr.24625) 6319 : cluster [DBG] pgmap v3597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:28 gibba032 bash[24004]: cluster 2021-03-10T15:52:27.672534+0000 mgr.gibba032.whopeo (mgr.24625) 6320 : cluster [DBG] pgmap v3598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:28 gibba011 bash[30791]: cluster 2021-03-10T15:52:27.672534+0000 mgr.gibba032.whopeo (mgr.24625) 6320 : cluster [DBG] pgmap v3598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:28 gibba011 bash[31774]: cluster 2021-03-10T15:52:27.672534+0000 mgr.gibba032.whopeo (mgr.24625) 6320 : cluster [DBG] pgmap v3598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:30 gibba032 bash[24004]: cluster 2021-03-10T15:52:29.674112+0000 mgr.gibba032.whopeo (mgr.24625) 6321 : cluster [DBG] pgmap v3599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:30 gibba011 bash[30791]: cluster 2021-03-10T15:52:29.674112+0000 mgr.gibba032.whopeo (mgr.24625) 6321 : cluster [DBG] pgmap v3599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:30 gibba011 bash[31774]: cluster 2021-03-10T15:52:29.674112+0000 mgr.gibba032.whopeo (mgr.24625) 6321 : cluster [DBG] pgmap v3599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:32 gibba011 bash[30791]: cluster 2021-03-10T15:52:31.675892+0000 mgr.gibba032.whopeo (mgr.24625) 6322 : cluster [DBG] pgmap v3600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:32 gibba011 bash[31774]: cluster 2021-03-10T15:52:31.675892+0000 mgr.gibba032.whopeo (mgr.24625) 6322 : cluster [DBG] pgmap v3600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:32 gibba032 bash[24004]: cluster 2021-03-10T15:52:31.675892+0000 mgr.gibba032.whopeo (mgr.24625) 6322 : cluster [DBG] pgmap v3600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:34 gibba032 bash[24004]: cluster 2021-03-10T15:52:33.677531+0000 mgr.gibba032.whopeo (mgr.24625) 6323 : cluster [DBG] pgmap v3601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:34 gibba011 bash[30791]: cluster 2021-03-10T15:52:33.677531+0000 mgr.gibba032.whopeo (mgr.24625) 6323 : cluster [DBG] pgmap v3601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:34 gibba011 bash[31774]: cluster 2021-03-10T15:52:33.677531+0000 mgr.gibba032.whopeo (mgr.24625) 6323 : cluster [DBG] pgmap v3601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:36 gibba032 bash[24004]: cluster 2021-03-10T15:52:35.678083+0000 mgr.gibba032.whopeo (mgr.24625) 6324 : cluster [DBG] pgmap v3602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:36 gibba011 bash[30791]: cluster 2021-03-10T15:52:35.678083+0000 mgr.gibba032.whopeo (mgr.24625) 6324 : cluster [DBG] pgmap v3602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:36 gibba011 bash[31774]: cluster 2021-03-10T15:52:35.678083+0000 mgr.gibba032.whopeo (mgr.24625) 6324 : cluster [DBG] pgmap v3602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:39.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:38 gibba032 bash[24004]: cluster 2021-03-10T15:52:37.679726+0000 mgr.gibba032.whopeo (mgr.24625) 6325 : cluster [DBG] pgmap v3603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:38 gibba011 bash[30791]: cluster 2021-03-10T15:52:37.679726+0000 mgr.gibba032.whopeo (mgr.24625) 6325 : cluster [DBG] pgmap v3603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:38 gibba011 bash[31774]: cluster 2021-03-10T15:52:37.679726+0000 mgr.gibba032.whopeo (mgr.24625) 6325 : cluster [DBG] pgmap v3603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:40.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.798383+0000 mgr.gibba032.whopeo (mgr.24625) 6326 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:52:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.798921+0000 mgr.gibba032.whopeo (mgr.24625) 6327 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:52:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:38.995572+0000 mon.a (mon.0) 2345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:52:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:38.998909+0000 mon.a (mon.0) 2346 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:52:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:39.000209+0000 mon.a (mon.0) 2347 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:39.001639+0000 mon.a (mon.0) 2348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:39.003014+0000 mon.a (mon.0) 2349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:39.005295+0000 mon.a (mon.0) 2350 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:39 gibba032 bash[24004]: audit 2021-03-10T15:52:39.006768+0000 mon.a (mon.0) 2351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.798383+0000 mgr.gibba032.whopeo (mgr.24625) 6326 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:52:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.798921+0000 mgr.gibba032.whopeo (mgr.24625) 6327 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:52:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:38.995572+0000 mon.a (mon.0) 2345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:52:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:38.998909+0000 mon.a (mon.0) 2346 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:52:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:39.000209+0000 mon.a (mon.0) 2347 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:39.001639+0000 mon.a (mon.0) 2348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:39.003014+0000 mon.a (mon.0) 2349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:39.005295+0000 mon.a (mon.0) 2350 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[30791]: audit 2021-03-10T15:52:39.006768+0000 mon.a (mon.0) 2351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.798383+0000 mgr.gibba032.whopeo (mgr.24625) 6326 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:52:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.798921+0000 mgr.gibba032.whopeo (mgr.24625) 6327 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:52:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:38.995572+0000 mon.a (mon.0) 2345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:52:40.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:38.998909+0000 mon.a (mon.0) 2346 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:52:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:39.000209+0000 mon.a (mon.0) 2347 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:39.001639+0000 mon.a (mon.0) 2348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:39.003014+0000 mon.a (mon.0) 2349 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:39.005295+0000 mon.a (mon.0) 2350 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:40.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:39 gibba011 bash[31774]: audit 2021-03-10T15:52:39.006768+0000 mon.a (mon.0) 2351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.996613+0000 mgr.gibba032.whopeo (mgr.24625) 6328 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:52:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.998336+0000 mgr.gibba032.whopeo (mgr.24625) 6329 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:52:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.999739+0000 mgr.gibba032.whopeo (mgr.24625) 6330 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:52:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:38.999776+0000 mgr.gibba032.whopeo (mgr.24625) 6331 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:52:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.001112+0000 mgr.gibba032.whopeo (mgr.24625) 6332 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:52:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.001150+0000 mgr.gibba032.whopeo (mgr.24625) 6333 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:52:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.002282+0000 mgr.gibba032.whopeo (mgr.24625) 6334 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:52:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.003316+0000 mgr.gibba032.whopeo (mgr.24625) 6335 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:52:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.003352+0000 mgr.gibba032.whopeo (mgr.24625) 6336 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:52:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.004543+0000 mgr.gibba032.whopeo (mgr.24625) 6337 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:52:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.004946+0000 mgr.gibba032.whopeo (mgr.24625) 6338 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:52:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.006201+0000 mgr.gibba032.whopeo (mgr.24625) 6339 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:52:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.006247+0000 mgr.gibba032.whopeo (mgr.24625) 6340 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:52:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cephadm 2021-03-10T15:52:39.006366+0000 mgr.gibba032.whopeo (mgr.24625) 6341 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[30791]: cluster 2021-03-10T15:52:39.681275+0000 mgr.gibba032.whopeo (mgr.24625) 6342 : cluster [DBG] pgmap v3604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:41.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.996613+0000 mgr.gibba032.whopeo (mgr.24625) 6328 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:52:41.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.998336+0000 mgr.gibba032.whopeo (mgr.24625) 6329 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:52:41.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.999739+0000 mgr.gibba032.whopeo (mgr.24625) 6330 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:52:41.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:38.999776+0000 mgr.gibba032.whopeo (mgr.24625) 6331 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:52:41.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.001112+0000 mgr.gibba032.whopeo (mgr.24625) 6332 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:52:41.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.001150+0000 mgr.gibba032.whopeo (mgr.24625) 6333 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:52:41.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.002282+0000 mgr.gibba032.whopeo (mgr.24625) 6334 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:52:41.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.003316+0000 mgr.gibba032.whopeo (mgr.24625) 6335 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:52:41.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.003352+0000 mgr.gibba032.whopeo (mgr.24625) 6336 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:52:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.004543+0000 mgr.gibba032.whopeo (mgr.24625) 6337 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:52:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.004946+0000 mgr.gibba032.whopeo (mgr.24625) 6338 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:52:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.006201+0000 mgr.gibba032.whopeo (mgr.24625) 6339 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:52:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.006247+0000 mgr.gibba032.whopeo (mgr.24625) 6340 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:52:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cephadm 2021-03-10T15:52:39.006366+0000 mgr.gibba032.whopeo (mgr.24625) 6341 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:41.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:40 gibba011 bash[31774]: cluster 2021-03-10T15:52:39.681275+0000 mgr.gibba032.whopeo (mgr.24625) 6342 : cluster [DBG] pgmap v3604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.996613+0000 mgr.gibba032.whopeo (mgr.24625) 6328 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:52:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.998336+0000 mgr.gibba032.whopeo (mgr.24625) 6329 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:52:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.999739+0000 mgr.gibba032.whopeo (mgr.24625) 6330 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:52:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:38.999776+0000 mgr.gibba032.whopeo (mgr.24625) 6331 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:52:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.001112+0000 mgr.gibba032.whopeo (mgr.24625) 6332 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:52:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.001150+0000 mgr.gibba032.whopeo (mgr.24625) 6333 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:52:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.002282+0000 mgr.gibba032.whopeo (mgr.24625) 6334 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:52:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.003316+0000 mgr.gibba032.whopeo (mgr.24625) 6335 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:52:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.003352+0000 mgr.gibba032.whopeo (mgr.24625) 6336 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:52:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.004543+0000 mgr.gibba032.whopeo (mgr.24625) 6337 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:52:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.004946+0000 mgr.gibba032.whopeo (mgr.24625) 6338 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:52:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.006201+0000 mgr.gibba032.whopeo (mgr.24625) 6339 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:52:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.006247+0000 mgr.gibba032.whopeo (mgr.24625) 6340 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:52:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cephadm 2021-03-10T15:52:39.006366+0000 mgr.gibba032.whopeo (mgr.24625) 6341 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:40 gibba032 bash[24004]: cluster 2021-03-10T15:52:39.681275+0000 mgr.gibba032.whopeo (mgr.24625) 6342 : cluster [DBG] pgmap v3604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:42 gibba011 bash[31774]: cluster 2021-03-10T15:52:41.682870+0000 mgr.gibba032.whopeo (mgr.24625) 6343 : cluster [DBG] pgmap v3605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:42 gibba011 bash[30791]: cluster 2021-03-10T15:52:41.682870+0000 mgr.gibba032.whopeo (mgr.24625) 6343 : cluster [DBG] pgmap v3605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:43.178 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:52:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:42 gibba032 bash[24004]: cluster 2021-03-10T15:52:41.682870+0000 mgr.gibba032.whopeo (mgr.24625) 6343 : cluster [DBG] pgmap v3605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:43.413 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:52:43.413 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (119m) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:52:43.414 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:52:43.414 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (119m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:52:43.414 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:52:43.415 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:52:43.415 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (118m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:52:43.415 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:52:43.415 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (118m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:52:43.415 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:52:43.416 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:52:43.416 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:52:43.416 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:52:43.416 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:52:43.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:52:43.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:52:43.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:52:43.417 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:52:43.418 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (114m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:52:43.418 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:52:43.418 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:52:43.663 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:52:43.663 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:52:43.663 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:52:43.663 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:43.664 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:52:43.664 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:52:43.664 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:43.664 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:52:43.665 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:52:43.665 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:52:43.665 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:52:43.665 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:52:43.666 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:52:43.666 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:52:43.666 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:52:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:44 gibba032 bash[24004]: audit 2021-03-10T15:52:43.167587+0000 mgr.gibba032.whopeo (mgr.24625) 6344 : audit [DBG] from='client.58285 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:44 gibba032 bash[24004]: audit 2021-03-10T15:52:43.394295+0000 mgr.gibba032.whopeo (mgr.24625) 6345 : audit [DBG] from='client.58291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:44 gibba032 bash[24004]: audit 2021-03-10T15:52:43.662488+0000 mon.a (mon.0) 2352 : audit [DBG] from='client.? 172.21.2.111:0/115297844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:44 gibba032 bash[24004]: cluster 2021-03-10T15:52:43.684461+0000 mgr.gibba032.whopeo (mgr.24625) 6346 : cluster [DBG] pgmap v3606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[30791]: audit 2021-03-10T15:52:43.167587+0000 mgr.gibba032.whopeo (mgr.24625) 6344 : audit [DBG] from='client.58285 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[30791]: audit 2021-03-10T15:52:43.394295+0000 mgr.gibba032.whopeo (mgr.24625) 6345 : audit [DBG] from='client.58291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[30791]: audit 2021-03-10T15:52:43.662488+0000 mon.a (mon.0) 2352 : audit [DBG] from='client.? 172.21.2.111:0/115297844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[30791]: cluster 2021-03-10T15:52:43.684461+0000 mgr.gibba032.whopeo (mgr.24625) 6346 : cluster [DBG] pgmap v3606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[31774]: audit 2021-03-10T15:52:43.167587+0000 mgr.gibba032.whopeo (mgr.24625) 6344 : audit [DBG] from='client.58285 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[31774]: audit 2021-03-10T15:52:43.394295+0000 mgr.gibba032.whopeo (mgr.24625) 6345 : audit [DBG] from='client.58291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:52:44.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[31774]: audit 2021-03-10T15:52:43.662488+0000 mon.a (mon.0) 2352 : audit [DBG] from='client.? 172.21.2.111:0/115297844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:52:44.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:44 gibba011 bash[31774]: cluster 2021-03-10T15:52:43.684461+0000 mgr.gibba032.whopeo (mgr.24625) 6346 : cluster [DBG] pgmap v3606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:46 gibba032 bash[24004]: cluster 2021-03-10T15:52:45.685118+0000 mgr.gibba032.whopeo (mgr.24625) 6347 : cluster [DBG] pgmap v3607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:46 gibba011 bash[30791]: cluster 2021-03-10T15:52:45.685118+0000 mgr.gibba032.whopeo (mgr.24625) 6347 : cluster [DBG] pgmap v3607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:46 gibba011 bash[31774]: cluster 2021-03-10T15:52:45.685118+0000 mgr.gibba032.whopeo (mgr.24625) 6347 : cluster [DBG] pgmap v3607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:48 gibba011 bash[30791]: cluster 2021-03-10T15:52:47.686747+0000 mgr.gibba032.whopeo (mgr.24625) 6348 : cluster [DBG] pgmap v3608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:48 gibba011 bash[31774]: cluster 2021-03-10T15:52:47.686747+0000 mgr.gibba032.whopeo (mgr.24625) 6348 : cluster [DBG] pgmap v3608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:48 gibba032 bash[24004]: cluster 2021-03-10T15:52:47.686747+0000 mgr.gibba032.whopeo (mgr.24625) 6348 : cluster [DBG] pgmap v3608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:50 gibba011 bash[30791]: cluster 2021-03-10T15:52:49.688246+0000 mgr.gibba032.whopeo (mgr.24625) 6349 : cluster [DBG] pgmap v3609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:50 gibba011 bash[31774]: cluster 2021-03-10T15:52:49.688246+0000 mgr.gibba032.whopeo (mgr.24625) 6349 : cluster [DBG] pgmap v3609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:50 gibba032 bash[24004]: cluster 2021-03-10T15:52:49.688246+0000 mgr.gibba032.whopeo (mgr.24625) 6349 : cluster [DBG] pgmap v3609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:52 gibba011 bash[30791]: cluster 2021-03-10T15:52:51.689824+0000 mgr.gibba032.whopeo (mgr.24625) 6350 : cluster [DBG] pgmap v3610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:53.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:52 gibba011 bash[31774]: cluster 2021-03-10T15:52:51.689824+0000 mgr.gibba032.whopeo (mgr.24625) 6350 : cluster [DBG] pgmap v3610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:53.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:52 gibba032 bash[24004]: cluster 2021-03-10T15:52:51.689824+0000 mgr.gibba032.whopeo (mgr.24625) 6350 : cluster [DBG] pgmap v3610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:54 gibba032 bash[24004]: cluster 2021-03-10T15:52:53.691471+0000 mgr.gibba032.whopeo (mgr.24625) 6351 : cluster [DBG] pgmap v3611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:54 gibba011 bash[30791]: cluster 2021-03-10T15:52:53.691471+0000 mgr.gibba032.whopeo (mgr.24625) 6351 : cluster [DBG] pgmap v3611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:54 gibba011 bash[31774]: cluster 2021-03-10T15:52:53.691471+0000 mgr.gibba032.whopeo (mgr.24625) 6351 : cluster [DBG] pgmap v3611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:55.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:55 gibba032 bash[24004]: cephadm 2021-03-10T15:52:54.022124+0000 mgr.gibba032.whopeo (mgr.24625) 6352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:55.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:55 gibba011 bash[30791]: cephadm 2021-03-10T15:52:54.022124+0000 mgr.gibba032.whopeo (mgr.24625) 6352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:55.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:55 gibba011 bash[31774]: cephadm 2021-03-10T15:52:54.022124+0000 mgr.gibba032.whopeo (mgr.24625) 6352 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:52:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:56 gibba032 bash[24004]: cluster 2021-03-10T15:52:55.692011+0000 mgr.gibba032.whopeo (mgr.24625) 6353 : cluster [DBG] pgmap v3612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:56 gibba011 bash[30791]: cluster 2021-03-10T15:52:55.692011+0000 mgr.gibba032.whopeo (mgr.24625) 6353 : cluster [DBG] pgmap v3612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:56 gibba011 bash[31774]: cluster 2021-03-10T15:52:55.692011+0000 mgr.gibba032.whopeo (mgr.24625) 6353 : cluster [DBG] pgmap v3612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.741155+0000 mon.a (mon.0) 2353 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:52:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.741774+0000 mon.a (mon.0) 2354 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:52:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.784869+0000 mon.a (mon.0) 2355 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:52:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.785629+0000 mon.a (mon.0) 2356 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:52:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.786309+0000 mon.a (mon.0) 2357 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:52:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:57 gibba032 bash[24004]: audit 2021-03-10T15:52:56.786980+0000 mon.a (mon.0) 2358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:52:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.741155+0000 mon.a (mon.0) 2353 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:52:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.741774+0000 mon.a (mon.0) 2354 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:52:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.784869+0000 mon.a (mon.0) 2355 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:52:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.785629+0000 mon.a (mon.0) 2356 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:52:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.786309+0000 mon.a (mon.0) 2357 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:52:57.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[30791]: audit 2021-03-10T15:52:56.786980+0000 mon.a (mon.0) 2358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:52:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.741155+0000 mon.a (mon.0) 2353 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:52:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.741774+0000 mon.a (mon.0) 2354 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:52:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.784869+0000 mon.a (mon.0) 2355 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T15:52:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.785629+0000 mon.a (mon.0) 2356 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T15:52:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.786309+0000 mon.a (mon.0) 2357 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2021-03-10T15:52:57.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:57 gibba011 bash[31774]: audit 2021-03-10T15:52:56.786980+0000 mon.a (mon.0) 2358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:52:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:52:58 gibba032 bash[24004]: cluster 2021-03-10T15:52:57.693660+0000 mgr.gibba032.whopeo (mgr.24625) 6354 : cluster [DBG] pgmap v3613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:52:58 gibba011 bash[30791]: cluster 2021-03-10T15:52:57.693660+0000 mgr.gibba032.whopeo (mgr.24625) 6354 : cluster [DBG] pgmap v3613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:52:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:52:58 gibba011 bash[31774]: cluster 2021-03-10T15:52:57.693660+0000 mgr.gibba032.whopeo (mgr.24625) 6354 : cluster [DBG] pgmap v3613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:00 gibba011 bash[30791]: cluster 2021-03-10T15:52:59.695242+0000 mgr.gibba032.whopeo (mgr.24625) 6355 : cluster [DBG] pgmap v3614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:00 gibba011 bash[31774]: cluster 2021-03-10T15:52:59.695242+0000 mgr.gibba032.whopeo (mgr.24625) 6355 : cluster [DBG] pgmap v3614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:00 gibba032 bash[24004]: cluster 2021-03-10T15:52:59.695242+0000 mgr.gibba032.whopeo (mgr.24625) 6355 : cluster [DBG] pgmap v3614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:02 gibba011 bash[30791]: cluster 2021-03-10T15:53:01.697053+0000 mgr.gibba032.whopeo (mgr.24625) 6356 : cluster [DBG] pgmap v3615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:02 gibba011 bash[31774]: cluster 2021-03-10T15:53:01.697053+0000 mgr.gibba032.whopeo (mgr.24625) 6356 : cluster [DBG] pgmap v3615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:03.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:02 gibba032 bash[24004]: cluster 2021-03-10T15:53:01.697053+0000 mgr.gibba032.whopeo (mgr.24625) 6356 : cluster [DBG] pgmap v3615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:04 gibba032 bash[24004]: cluster 2021-03-10T15:53:03.698539+0000 mgr.gibba032.whopeo (mgr.24625) 6357 : cluster [DBG] pgmap v3616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:04 gibba011 bash[30791]: cluster 2021-03-10T15:53:03.698539+0000 mgr.gibba032.whopeo (mgr.24625) 6357 : cluster [DBG] pgmap v3616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:04 gibba011 bash[31774]: cluster 2021-03-10T15:53:03.698539+0000 mgr.gibba032.whopeo (mgr.24625) 6357 : cluster [DBG] pgmap v3616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:06 gibba011 bash[30791]: cluster 2021-03-10T15:53:05.699789+0000 mgr.gibba032.whopeo (mgr.24625) 6358 : cluster [DBG] pgmap v3617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:06 gibba011 bash[31774]: cluster 2021-03-10T15:53:05.699789+0000 mgr.gibba032.whopeo (mgr.24625) 6358 : cluster [DBG] pgmap v3617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:06 gibba032 bash[24004]: cluster 2021-03-10T15:53:05.699789+0000 mgr.gibba032.whopeo (mgr.24625) 6358 : cluster [DBG] pgmap v3617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:09.040 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:08 gibba032 bash[24004]: cluster 2021-03-10T15:53:07.701520+0000 mgr.gibba032.whopeo (mgr.24625) 6359 : cluster [DBG] pgmap v3618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:08 gibba011 bash[30791]: cluster 2021-03-10T15:53:07.701520+0000 mgr.gibba032.whopeo (mgr.24625) 6359 : cluster [DBG] pgmap v3618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:08 gibba011 bash[31774]: cluster 2021-03-10T15:53:07.701520+0000 mgr.gibba032.whopeo (mgr.24625) 6359 : cluster [DBG] pgmap v3618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:10 gibba011 bash[30791]: cephadm 2021-03-10T15:53:09.038179+0000 mgr.gibba032.whopeo (mgr.24625) 6360 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:11.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:10 gibba011 bash[30791]: cluster 2021-03-10T15:53:09.703042+0000 mgr.gibba032.whopeo (mgr.24625) 6361 : cluster [DBG] pgmap v3619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:10 gibba011 bash[31774]: cephadm 2021-03-10T15:53:09.038179+0000 mgr.gibba032.whopeo (mgr.24625) 6360 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:11.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:10 gibba011 bash[31774]: cluster 2021-03-10T15:53:09.703042+0000 mgr.gibba032.whopeo (mgr.24625) 6361 : cluster [DBG] pgmap v3619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:10 gibba032 bash[24004]: cephadm 2021-03-10T15:53:09.038179+0000 mgr.gibba032.whopeo (mgr.24625) 6360 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:10 gibba032 bash[24004]: cluster 2021-03-10T15:53:09.703042+0000 mgr.gibba032.whopeo (mgr.24625) 6361 : cluster [DBG] pgmap v3619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:12 gibba011 bash[30791]: cluster 2021-03-10T15:53:11.704762+0000 mgr.gibba032.whopeo (mgr.24625) 6362 : cluster [DBG] pgmap v3620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:12 gibba011 bash[31774]: cluster 2021-03-10T15:53:11.704762+0000 mgr.gibba032.whopeo (mgr.24625) 6362 : cluster [DBG] pgmap v3620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:12 gibba032 bash[24004]: cluster 2021-03-10T15:53:11.704762+0000 mgr.gibba032.whopeo (mgr.24625) 6362 : cluster [DBG] pgmap v3620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:13.905 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:53:14.132 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:53:14.132 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (119m) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:53:14.133 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:53:14.133 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:53:14.133 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:53:14.133 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:53:14.134 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:53:14.134 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (118m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:53:14.134 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:53:14.134 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:53:14.135 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:53:14.135 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (118m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:53:14.135 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:53:14.135 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:53:14.136 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:53:14.136 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:53:14.136 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (116m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:53:14.136 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:53:14.137 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (115m) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:53:14.137 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:53:14.137 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:53:14.380 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:53:14.380 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:53:14.381 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:53:14.381 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:14.381 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:53:14.382 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:53:14.382 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:14.382 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:53:14.382 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:53:14.383 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:14.383 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:53:14.383 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:53:14.383 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:53:14.384 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:53:14.384 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:53:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:14 gibba032 bash[24004]: cluster 2021-03-10T15:53:13.706125+0000 mgr.gibba032.whopeo (mgr.24625) 6363 : cluster [DBG] pgmap v3621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:14 gibba032 bash[24004]: audit 2021-03-10T15:53:13.893987+0000 mgr.gibba032.whopeo (mgr.24625) 6364 : audit [DBG] from='client.48321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:14 gibba032 bash[24004]: audit 2021-03-10T15:53:14.379974+0000 mon.a (mon.0) 2359 : audit [DBG] from='client.? 172.21.2.111:0/2022591411' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[30791]: cluster 2021-03-10T15:53:13.706125+0000 mgr.gibba032.whopeo (mgr.24625) 6363 : cluster [DBG] pgmap v3621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[30791]: audit 2021-03-10T15:53:13.893987+0000 mgr.gibba032.whopeo (mgr.24625) 6364 : audit [DBG] from='client.48321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[30791]: audit 2021-03-10T15:53:14.379974+0000 mon.a (mon.0) 2359 : audit [DBG] from='client.? 172.21.2.111:0/2022591411' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[31774]: cluster 2021-03-10T15:53:13.706125+0000 mgr.gibba032.whopeo (mgr.24625) 6363 : cluster [DBG] pgmap v3621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[31774]: audit 2021-03-10T15:53:13.893987+0000 mgr.gibba032.whopeo (mgr.24625) 6364 : audit [DBG] from='client.48321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:14 gibba011 bash[31774]: audit 2021-03-10T15:53:14.379974+0000 mon.a (mon.0) 2359 : audit [DBG] from='client.? 172.21.2.111:0/2022591411' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:15.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:15 gibba032 bash[24004]: audit 2021-03-10T15:53:14.117946+0000 mgr.gibba032.whopeo (mgr.24625) 6365 : audit [DBG] from='client.58309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:15.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:15 gibba011 bash[30791]: audit 2021-03-10T15:53:14.117946+0000 mgr.gibba032.whopeo (mgr.24625) 6365 : audit [DBG] from='client.58309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:15.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:15 gibba011 bash[31774]: audit 2021-03-10T15:53:14.117946+0000 mgr.gibba032.whopeo (mgr.24625) 6365 : audit [DBG] from='client.58309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:16 gibba032 bash[24004]: cluster 2021-03-10T15:53:15.707472+0000 mgr.gibba032.whopeo (mgr.24625) 6366 : cluster [DBG] pgmap v3622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:16 gibba011 bash[30791]: cluster 2021-03-10T15:53:15.707472+0000 mgr.gibba032.whopeo (mgr.24625) 6366 : cluster [DBG] pgmap v3622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:16 gibba011 bash[31774]: cluster 2021-03-10T15:53:15.707472+0000 mgr.gibba032.whopeo (mgr.24625) 6366 : cluster [DBG] pgmap v3622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:18 gibba011 bash[30791]: cluster 2021-03-10T15:53:17.709178+0000 mgr.gibba032.whopeo (mgr.24625) 6367 : cluster [DBG] pgmap v3623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:18 gibba011 bash[31774]: cluster 2021-03-10T15:53:17.709178+0000 mgr.gibba032.whopeo (mgr.24625) 6367 : cluster [DBG] pgmap v3623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:18 gibba032 bash[24004]: cluster 2021-03-10T15:53:17.709178+0000 mgr.gibba032.whopeo (mgr.24625) 6367 : cluster [DBG] pgmap v3623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:20 gibba011 bash[30791]: cluster 2021-03-10T15:53:19.710663+0000 mgr.gibba032.whopeo (mgr.24625) 6368 : cluster [DBG] pgmap v3624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:20 gibba011 bash[31774]: cluster 2021-03-10T15:53:19.710663+0000 mgr.gibba032.whopeo (mgr.24625) 6368 : cluster [DBG] pgmap v3624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:20 gibba032 bash[24004]: cluster 2021-03-10T15:53:19.710663+0000 mgr.gibba032.whopeo (mgr.24625) 6368 : cluster [DBG] pgmap v3624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:22 gibba011 bash[30791]: cluster 2021-03-10T15:53:21.712455+0000 mgr.gibba032.whopeo (mgr.24625) 6369 : cluster [DBG] pgmap v3625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:22 gibba011 bash[31774]: cluster 2021-03-10T15:53:21.712455+0000 mgr.gibba032.whopeo (mgr.24625) 6369 : cluster [DBG] pgmap v3625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:22 gibba032 bash[24004]: cluster 2021-03-10T15:53:21.712455+0000 mgr.gibba032.whopeo (mgr.24625) 6369 : cluster [DBG] pgmap v3625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:24 gibba032 bash[24004]: cluster 2021-03-10T15:53:23.714056+0000 mgr.gibba032.whopeo (mgr.24625) 6370 : cluster [DBG] pgmap v3626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:24 gibba011 bash[30791]: cluster 2021-03-10T15:53:23.714056+0000 mgr.gibba032.whopeo (mgr.24625) 6370 : cluster [DBG] pgmap v3626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:24 gibba011 bash[31774]: cluster 2021-03-10T15:53:23.714056+0000 mgr.gibba032.whopeo (mgr.24625) 6370 : cluster [DBG] pgmap v3626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:25.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:25 gibba032 bash[24004]: cephadm 2021-03-10T15:53:24.054201+0000 mgr.gibba032.whopeo (mgr.24625) 6371 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:25.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:25 gibba011 bash[30791]: cephadm 2021-03-10T15:53:24.054201+0000 mgr.gibba032.whopeo (mgr.24625) 6371 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:25.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:25 gibba011 bash[31774]: cephadm 2021-03-10T15:53:24.054201+0000 mgr.gibba032.whopeo (mgr.24625) 6371 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:26 gibba032 bash[24004]: cluster 2021-03-10T15:53:25.714989+0000 mgr.gibba032.whopeo (mgr.24625) 6372 : cluster [DBG] pgmap v3627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:26 gibba011 bash[30791]: cluster 2021-03-10T15:53:25.714989+0000 mgr.gibba032.whopeo (mgr.24625) 6372 : cluster [DBG] pgmap v3627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:26 gibba011 bash[31774]: cluster 2021-03-10T15:53:25.714989+0000 mgr.gibba032.whopeo (mgr.24625) 6372 : cluster [DBG] pgmap v3627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:28 gibba011 bash[30791]: cluster 2021-03-10T15:53:27.716744+0000 mgr.gibba032.whopeo (mgr.24625) 6373 : cluster [DBG] pgmap v3628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:28 gibba011 bash[31774]: cluster 2021-03-10T15:53:27.716744+0000 mgr.gibba032.whopeo (mgr.24625) 6373 : cluster [DBG] pgmap v3628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:28 gibba032 bash[24004]: cluster 2021-03-10T15:53:27.716744+0000 mgr.gibba032.whopeo (mgr.24625) 6373 : cluster [DBG] pgmap v3628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:30 gibba011 bash[30791]: cluster 2021-03-10T15:53:29.718413+0000 mgr.gibba032.whopeo (mgr.24625) 6374 : cluster [DBG] pgmap v3629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:30 gibba011 bash[31774]: cluster 2021-03-10T15:53:29.718413+0000 mgr.gibba032.whopeo (mgr.24625) 6374 : cluster [DBG] pgmap v3629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:30 gibba032 bash[24004]: cluster 2021-03-10T15:53:29.718413+0000 mgr.gibba032.whopeo (mgr.24625) 6374 : cluster [DBG] pgmap v3629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:33.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:32 gibba011 bash[30791]: cluster 2021-03-10T15:53:31.720270+0000 mgr.gibba032.whopeo (mgr.24625) 6375 : cluster [DBG] pgmap v3630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:32 gibba011 bash[31774]: cluster 2021-03-10T15:53:31.720270+0000 mgr.gibba032.whopeo (mgr.24625) 6375 : cluster [DBG] pgmap v3630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:32 gibba032 bash[24004]: cluster 2021-03-10T15:53:31.720270+0000 mgr.gibba032.whopeo (mgr.24625) 6375 : cluster [DBG] pgmap v3630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:34 gibba032 bash[24004]: cluster 2021-03-10T15:53:33.722036+0000 mgr.gibba032.whopeo (mgr.24625) 6376 : cluster [DBG] pgmap v3631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:34 gibba011 bash[30791]: cluster 2021-03-10T15:53:33.722036+0000 mgr.gibba032.whopeo (mgr.24625) 6376 : cluster [DBG] pgmap v3631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:34 gibba011 bash[31774]: cluster 2021-03-10T15:53:33.722036+0000 mgr.gibba032.whopeo (mgr.24625) 6376 : cluster [DBG] pgmap v3631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:36 gibba011 bash[30791]: cluster 2021-03-10T15:53:35.723515+0000 mgr.gibba032.whopeo (mgr.24625) 6377 : cluster [DBG] pgmap v3632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:36 gibba011 bash[31774]: cluster 2021-03-10T15:53:35.723515+0000 mgr.gibba032.whopeo (mgr.24625) 6377 : cluster [DBG] pgmap v3632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:36 gibba032 bash[24004]: cluster 2021-03-10T15:53:35.723515+0000 mgr.gibba032.whopeo (mgr.24625) 6377 : cluster [DBG] pgmap v3632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:38 gibba011 bash[30791]: cluster 2021-03-10T15:53:37.725242+0000 mgr.gibba032.whopeo (mgr.24625) 6378 : cluster [DBG] pgmap v3633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:38 gibba011 bash[31774]: cluster 2021-03-10T15:53:37.725242+0000 mgr.gibba032.whopeo (mgr.24625) 6378 : cluster [DBG] pgmap v3633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:38 gibba032 bash[24004]: cluster 2021-03-10T15:53:37.725242+0000 mgr.gibba032.whopeo (mgr.24625) 6378 : cluster [DBG] pgmap v3633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.070993+0000 mgr.gibba032.whopeo (mgr.24625) 6379 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:53:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.072610+0000 mgr.gibba032.whopeo (mgr.24625) 6380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:53:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.284566+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:53:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.285631+0000 mgr.gibba032.whopeo (mgr.24625) 6381 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:53:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.287911+0000 mgr.gibba032.whopeo (mgr.24625) 6382 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:53:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.288254+0000 mon.a (mon.0) 2361 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:53:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.289626+0000 mgr.gibba032.whopeo (mgr.24625) 6383 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:53:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.289681+0000 mgr.gibba032.whopeo (mgr.24625) 6384 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:53:40.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.290075+0000 mon.a (mon.0) 2362 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.291080+0000 mgr.gibba032.whopeo (mgr.24625) 6385 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:53:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.291116+0000 mgr.gibba032.whopeo (mgr.24625) 6386 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:53:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.291752+0000 mon.a (mon.0) 2363 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.292537+0000 mgr.gibba032.whopeo (mgr.24625) 6387 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:53:40.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.293111+0000 mon.a (mon.0) 2364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.294505+0000 mgr.gibba032.whopeo (mgr.24625) 6388 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:53:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.294544+0000 mgr.gibba032.whopeo (mgr.24625) 6389 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:53:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.296151+0000 mgr.gibba032.whopeo (mgr.24625) 6390 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:53:40.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.296556+0000 mgr.gibba032.whopeo (mgr.24625) 6391 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:53:40.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.296726+0000 mon.a (mon.0) 2365 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.298068+0000 mgr.gibba032.whopeo (mgr.24625) 6392 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:53:40.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.298104+0000 mgr.gibba032.whopeo (mgr.24625) 6393 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:53:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.070993+0000 mgr.gibba032.whopeo (mgr.24625) 6379 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:53:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.072610+0000 mgr.gibba032.whopeo (mgr.24625) 6380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:53:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.284566+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:53:40.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.285631+0000 mgr.gibba032.whopeo (mgr.24625) 6381 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:53:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.287911+0000 mgr.gibba032.whopeo (mgr.24625) 6382 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:53:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.288254+0000 mon.a (mon.0) 2361 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:53:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.289626+0000 mgr.gibba032.whopeo (mgr.24625) 6383 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:53:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.289681+0000 mgr.gibba032.whopeo (mgr.24625) 6384 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:53:40.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.290075+0000 mon.a (mon.0) 2362 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.291080+0000 mgr.gibba032.whopeo (mgr.24625) 6385 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:53:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.291116+0000 mgr.gibba032.whopeo (mgr.24625) 6386 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:53:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.291752+0000 mon.a (mon.0) 2363 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.292537+0000 mgr.gibba032.whopeo (mgr.24625) 6387 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:53:40.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.293111+0000 mon.a (mon.0) 2364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.294505+0000 mgr.gibba032.whopeo (mgr.24625) 6388 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:53:40.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.294544+0000 mgr.gibba032.whopeo (mgr.24625) 6389 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:53:40.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.296151+0000 mgr.gibba032.whopeo (mgr.24625) 6390 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:53:40.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.296556+0000 mgr.gibba032.whopeo (mgr.24625) 6391 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:53:40.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.296726+0000 mon.a (mon.0) 2365 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.298068+0000 mgr.gibba032.whopeo (mgr.24625) 6392 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:53:40.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.298104+0000 mgr.gibba032.whopeo (mgr.24625) 6393 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:53:40.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cephadm 2021-03-10T15:53:39.298219+0000 mgr.gibba032.whopeo (mgr.24625) 6394 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:40.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: audit 2021-03-10T15:53:39.298552+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.591 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[30791]: cluster 2021-03-10T15:53:39.726664+0000 mgr.gibba032.whopeo (mgr.24625) 6395 : cluster [DBG] pgmap v3634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:40.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cephadm 2021-03-10T15:53:39.298219+0000 mgr.gibba032.whopeo (mgr.24625) 6394 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:40.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: audit 2021-03-10T15:53:39.298552+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:40 gibba011 bash[31774]: cluster 2021-03-10T15:53:39.726664+0000 mgr.gibba032.whopeo (mgr.24625) 6395 : cluster [DBG] pgmap v3634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.070993+0000 mgr.gibba032.whopeo (mgr.24625) 6379 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:53:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.072610+0000 mgr.gibba032.whopeo (mgr.24625) 6380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:53:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.284566+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:53:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.285631+0000 mgr.gibba032.whopeo (mgr.24625) 6381 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:53:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.287911+0000 mgr.gibba032.whopeo (mgr.24625) 6382 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:53:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.288254+0000 mon.a (mon.0) 2361 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:53:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.289626+0000 mgr.gibba032.whopeo (mgr.24625) 6383 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:53:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.289681+0000 mgr.gibba032.whopeo (mgr.24625) 6384 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:53:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.290075+0000 mon.a (mon.0) 2362 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.291080+0000 mgr.gibba032.whopeo (mgr.24625) 6385 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:53:40.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.291116+0000 mgr.gibba032.whopeo (mgr.24625) 6386 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:53:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.291752+0000 mon.a (mon.0) 2363 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.292537+0000 mgr.gibba032.whopeo (mgr.24625) 6387 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:53:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.293111+0000 mon.a (mon.0) 2364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.294505+0000 mgr.gibba032.whopeo (mgr.24625) 6388 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:53:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.294544+0000 mgr.gibba032.whopeo (mgr.24625) 6389 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:53:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.296151+0000 mgr.gibba032.whopeo (mgr.24625) 6390 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:53:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.296556+0000 mgr.gibba032.whopeo (mgr.24625) 6391 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:53:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.296726+0000 mon.a (mon.0) 2365 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.298068+0000 mgr.gibba032.whopeo (mgr.24625) 6392 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:53:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.298104+0000 mgr.gibba032.whopeo (mgr.24625) 6393 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:53:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cephadm 2021-03-10T15:53:39.298219+0000 mgr.gibba032.whopeo (mgr.24625) 6394 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: audit 2021-03-10T15:53:39.298552+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:40 gibba032 bash[24004]: cluster 2021-03-10T15:53:39.726664+0000 mgr.gibba032.whopeo (mgr.24625) 6395 : cluster [DBG] pgmap v3634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:43.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:42 gibba011 bash[30791]: cluster 2021-03-10T15:53:41.728394+0000 mgr.gibba032.whopeo (mgr.24625) 6396 : cluster [DBG] pgmap v3635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:42 gibba011 bash[31774]: cluster 2021-03-10T15:53:41.728394+0000 mgr.gibba032.whopeo (mgr.24625) 6396 : cluster [DBG] pgmap v3635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:42 gibba032 bash[24004]: cluster 2021-03-10T15:53:41.728394+0000 mgr.gibba032.whopeo (mgr.24625) 6396 : cluster [DBG] pgmap v3635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:44.627 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:53:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:44 gibba032 bash[24004]: cluster 2021-03-10T15:53:43.729985+0000 mgr.gibba032.whopeo (mgr.24625) 6397 : cluster [DBG] pgmap v3636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:44 gibba011 bash[30791]: cluster 2021-03-10T15:53:43.729985+0000 mgr.gibba032.whopeo (mgr.24625) 6397 : cluster [DBG] pgmap v3636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:44 gibba011 bash[31774]: cluster 2021-03-10T15:53:43.729985+0000 mgr.gibba032.whopeo (mgr.24625) 6397 : cluster [DBG] pgmap v3636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:44.858 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:53:44.858 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:53:44.858 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:53:44.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:53:44.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:53:44.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:53:44.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (119m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:53:44.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:53:44.860 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (119m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:53:44.860 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:53:44.860 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:53:44.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:53:44.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:53:44.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:53:44.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:53:44.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:53:44.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:53:44.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:53:44.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (115m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:53:44.862 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:53:44.862 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:53:45.108 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:53:45.109 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:53:45.109 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:53:45.109 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:45.110 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:53:45.110 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:53:45.110 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:45.110 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:53:45.111 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:53:45.111 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:53:45.111 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:53:45.111 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:53:45.111 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:53:45.112 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:53:45.112 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:53:45.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:45 gibba032 bash[24004]: audit 2021-03-10T15:53:44.616092+0000 mgr.gibba032.whopeo (mgr.24625) 6398 : audit [DBG] from='client.48339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:45 gibba032 bash[24004]: audit 2021-03-10T15:53:44.843786+0000 mgr.gibba032.whopeo (mgr.24625) 6399 : audit [DBG] from='client.48345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:45 gibba032 bash[24004]: audit 2021-03-10T15:53:45.108550+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? 172.21.2.111:0/4199015231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:45.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[30791]: audit 2021-03-10T15:53:44.616092+0000 mgr.gibba032.whopeo (mgr.24625) 6398 : audit [DBG] from='client.48339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[30791]: audit 2021-03-10T15:53:44.843786+0000 mgr.gibba032.whopeo (mgr.24625) 6399 : audit [DBG] from='client.48345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[30791]: audit 2021-03-10T15:53:45.108550+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? 172.21.2.111:0/4199015231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:45.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[31774]: audit 2021-03-10T15:53:44.616092+0000 mgr.gibba032.whopeo (mgr.24625) 6398 : audit [DBG] from='client.48339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[31774]: audit 2021-03-10T15:53:44.843786+0000 mgr.gibba032.whopeo (mgr.24625) 6399 : audit [DBG] from='client.48345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:53:45.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:45 gibba011 bash[31774]: audit 2021-03-10T15:53:45.108550+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? 172.21.2.111:0/4199015231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:53:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:46 gibba032 bash[24004]: cluster 2021-03-10T15:53:45.731307+0000 mgr.gibba032.whopeo (mgr.24625) 6400 : cluster [DBG] pgmap v3637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:46 gibba011 bash[30791]: cluster 2021-03-10T15:53:45.731307+0000 mgr.gibba032.whopeo (mgr.24625) 6400 : cluster [DBG] pgmap v3637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:46 gibba011 bash[31774]: cluster 2021-03-10T15:53:45.731307+0000 mgr.gibba032.whopeo (mgr.24625) 6400 : cluster [DBG] pgmap v3637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:48 gibba011 bash[30791]: cluster 2021-03-10T15:53:47.732881+0000 mgr.gibba032.whopeo (mgr.24625) 6401 : cluster [DBG] pgmap v3638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:48 gibba011 bash[31774]: cluster 2021-03-10T15:53:47.732881+0000 mgr.gibba032.whopeo (mgr.24625) 6401 : cluster [DBG] pgmap v3638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:48 gibba032 bash[24004]: cluster 2021-03-10T15:53:47.732881+0000 mgr.gibba032.whopeo (mgr.24625) 6401 : cluster [DBG] pgmap v3638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:50 gibba011 bash[30791]: cluster 2021-03-10T15:53:49.734134+0000 mgr.gibba032.whopeo (mgr.24625) 6402 : cluster [DBG] pgmap v3639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:50 gibba011 bash[31774]: cluster 2021-03-10T15:53:49.734134+0000 mgr.gibba032.whopeo (mgr.24625) 6402 : cluster [DBG] pgmap v3639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:50 gibba032 bash[24004]: cluster 2021-03-10T15:53:49.734134+0000 mgr.gibba032.whopeo (mgr.24625) 6402 : cluster [DBG] pgmap v3639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:52 gibba011 bash[30791]: cluster 2021-03-10T15:53:51.735880+0000 mgr.gibba032.whopeo (mgr.24625) 6403 : cluster [DBG] pgmap v3640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:52 gibba011 bash[31774]: cluster 2021-03-10T15:53:51.735880+0000 mgr.gibba032.whopeo (mgr.24625) 6403 : cluster [DBG] pgmap v3640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:53.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:52 gibba032 bash[24004]: cluster 2021-03-10T15:53:51.735880+0000 mgr.gibba032.whopeo (mgr.24625) 6403 : cluster [DBG] pgmap v3640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:54 gibba032 bash[24004]: cluster 2021-03-10T15:53:53.737319+0000 mgr.gibba032.whopeo (mgr.24625) 6404 : cluster [DBG] pgmap v3641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:54 gibba011 bash[30791]: cluster 2021-03-10T15:53:53.737319+0000 mgr.gibba032.whopeo (mgr.24625) 6404 : cluster [DBG] pgmap v3641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:54 gibba011 bash[31774]: cluster 2021-03-10T15:53:53.737319+0000 mgr.gibba032.whopeo (mgr.24625) 6404 : cluster [DBG] pgmap v3641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:55.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:55 gibba032 bash[24004]: cephadm 2021-03-10T15:53:54.312811+0000 mgr.gibba032.whopeo (mgr.24625) 6405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:55.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:55 gibba011 bash[30791]: cephadm 2021-03-10T15:53:54.312811+0000 mgr.gibba032.whopeo (mgr.24625) 6405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:55.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:55 gibba011 bash[31774]: cephadm 2021-03-10T15:53:54.312811+0000 mgr.gibba032.whopeo (mgr.24625) 6405 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:53:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:56 gibba032 bash[24004]: cluster 2021-03-10T15:53:55.738591+0000 mgr.gibba032.whopeo (mgr.24625) 6406 : cluster [DBG] pgmap v3642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:56 gibba011 bash[30791]: cluster 2021-03-10T15:53:55.738591+0000 mgr.gibba032.whopeo (mgr.24625) 6406 : cluster [DBG] pgmap v3642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:56 gibba011 bash[31774]: cluster 2021-03-10T15:53:55.738591+0000 mgr.gibba032.whopeo (mgr.24625) 6406 : cluster [DBG] pgmap v3642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.747823+0000 mon.a (mon.0) 2368 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:53:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.748552+0000 mon.a (mon.0) 2369 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:53:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.792165+0000 mon.a (mon.0) 2370 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:53:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.792938+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.793600+0000 mon.a (mon.0) 2372 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:57 gibba032 bash[24004]: audit 2021-03-10T15:53:56.794305+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.747823+0000 mon.a (mon.0) 2368 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:53:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.748552+0000 mon.a (mon.0) 2369 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:53:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.792165+0000 mon.a (mon.0) 2370 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:53:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.792938+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.793600+0000 mon.a (mon.0) 2372 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[30791]: audit 2021-03-10T15:53:56.794305+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.747823+0000 mon.a (mon.0) 2368 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:53:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.748552+0000 mon.a (mon.0) 2369 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:53:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.792165+0000 mon.a (mon.0) 2370 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:53:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.792938+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.793600+0000 mon.a (mon.0) 2372 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T15:53:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:57 gibba011 bash[31774]: audit 2021-03-10T15:53:56.794305+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T15:53:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:53:58 gibba032 bash[24004]: cluster 2021-03-10T15:53:57.740285+0000 mgr.gibba032.whopeo (mgr.24625) 6407 : cluster [DBG] pgmap v3643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:53:58 gibba011 bash[30791]: cluster 2021-03-10T15:53:57.740285+0000 mgr.gibba032.whopeo (mgr.24625) 6407 : cluster [DBG] pgmap v3643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:53:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:53:58 gibba011 bash[31774]: cluster 2021-03-10T15:53:57.740285+0000 mgr.gibba032.whopeo (mgr.24625) 6407 : cluster [DBG] pgmap v3643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:00 gibba011 bash[30791]: cluster 2021-03-10T15:53:59.741915+0000 mgr.gibba032.whopeo (mgr.24625) 6408 : cluster [DBG] pgmap v3644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:00 gibba011 bash[31774]: cluster 2021-03-10T15:53:59.741915+0000 mgr.gibba032.whopeo (mgr.24625) 6408 : cluster [DBG] pgmap v3644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:00 gibba032 bash[24004]: cluster 2021-03-10T15:53:59.741915+0000 mgr.gibba032.whopeo (mgr.24625) 6408 : cluster [DBG] pgmap v3644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:02 gibba011 bash[30791]: cluster 2021-03-10T15:54:01.743650+0000 mgr.gibba032.whopeo (mgr.24625) 6409 : cluster [DBG] pgmap v3645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:02 gibba011 bash[31774]: cluster 2021-03-10T15:54:01.743650+0000 mgr.gibba032.whopeo (mgr.24625) 6409 : cluster [DBG] pgmap v3645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:02 gibba032 bash[24004]: cluster 2021-03-10T15:54:01.743650+0000 mgr.gibba032.whopeo (mgr.24625) 6409 : cluster [DBG] pgmap v3645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:04 gibba032 bash[24004]: cluster 2021-03-10T15:54:03.745091+0000 mgr.gibba032.whopeo (mgr.24625) 6410 : cluster [DBG] pgmap v3646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:04 gibba011 bash[30791]: cluster 2021-03-10T15:54:03.745091+0000 mgr.gibba032.whopeo (mgr.24625) 6410 : cluster [DBG] pgmap v3646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:04 gibba011 bash[31774]: cluster 2021-03-10T15:54:03.745091+0000 mgr.gibba032.whopeo (mgr.24625) 6410 : cluster [DBG] pgmap v3646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:06 gibba011 bash[30791]: cluster 2021-03-10T15:54:05.746610+0000 mgr.gibba032.whopeo (mgr.24625) 6411 : cluster [DBG] pgmap v3647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:06 gibba011 bash[31774]: cluster 2021-03-10T15:54:05.746610+0000 mgr.gibba032.whopeo (mgr.24625) 6411 : cluster [DBG] pgmap v3647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:06 gibba032 bash[24004]: cluster 2021-03-10T15:54:05.746610+0000 mgr.gibba032.whopeo (mgr.24625) 6411 : cluster [DBG] pgmap v3647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:08 gibba011 bash[30791]: cluster 2021-03-10T15:54:07.748472+0000 mgr.gibba032.whopeo (mgr.24625) 6412 : cluster [DBG] pgmap v3648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:08 gibba011 bash[31774]: cluster 2021-03-10T15:54:07.748472+0000 mgr.gibba032.whopeo (mgr.24625) 6412 : cluster [DBG] pgmap v3648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:08 gibba032 bash[24004]: cluster 2021-03-10T15:54:07.748472+0000 mgr.gibba032.whopeo (mgr.24625) 6412 : cluster [DBG] pgmap v3648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:10 gibba032 bash[24004]: cephadm 2021-03-10T15:54:09.328834+0000 mgr.gibba032.whopeo (mgr.24625) 6413 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:10 gibba032 bash[24004]: cluster 2021-03-10T15:54:09.750210+0000 mgr.gibba032.whopeo (mgr.24625) 6414 : cluster [DBG] pgmap v3649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:10 gibba011 bash[30791]: cephadm 2021-03-10T15:54:09.328834+0000 mgr.gibba032.whopeo (mgr.24625) 6413 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:10 gibba011 bash[30791]: cluster 2021-03-10T15:54:09.750210+0000 mgr.gibba032.whopeo (mgr.24625) 6414 : cluster [DBG] pgmap v3649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:10 gibba011 bash[31774]: cephadm 2021-03-10T15:54:09.328834+0000 mgr.gibba032.whopeo (mgr.24625) 6413 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:10 gibba011 bash[31774]: cluster 2021-03-10T15:54:09.750210+0000 mgr.gibba032.whopeo (mgr.24625) 6414 : cluster [DBG] pgmap v3649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:12 gibba032 bash[24004]: cluster 2021-03-10T15:54:11.752004+0000 mgr.gibba032.whopeo (mgr.24625) 6415 : cluster [DBG] pgmap v3650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:12 gibba011 bash[30791]: cluster 2021-03-10T15:54:11.752004+0000 mgr.gibba032.whopeo (mgr.24625) 6415 : cluster [DBG] pgmap v3650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:12 gibba011 bash[31774]: cluster 2021-03-10T15:54:11.752004+0000 mgr.gibba032.whopeo (mgr.24625) 6415 : cluster [DBG] pgmap v3650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:14 gibba032 bash[24004]: cluster 2021-03-10T15:54:13.753698+0000 mgr.gibba032.whopeo (mgr.24625) 6416 : cluster [DBG] pgmap v3651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:14 gibba011 bash[30791]: cluster 2021-03-10T15:54:13.753698+0000 mgr.gibba032.whopeo (mgr.24625) 6416 : cluster [DBG] pgmap v3651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:14 gibba011 bash[31774]: cluster 2021-03-10T15:54:13.753698+0000 mgr.gibba032.whopeo (mgr.24625) 6416 : cluster [DBG] pgmap v3651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:15.351 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:54:15.577 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:54:15.577 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:54:15.578 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:54:15.578 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:54:15.578 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:54:15.578 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:54:15.579 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:54:15.579 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (119m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:54:15.579 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:54:15.579 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:54:15.579 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:54:15.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (119m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:54:15.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:54:15.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:54:15.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:54:15.580 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:54:15.581 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (117m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:54:15.581 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:54:15.581 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (116m) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:54:15.581 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:54:15.582 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:54:15.825 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:54:15.826 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:54:15.826 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:54:15.826 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:15.827 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:54:15.827 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:54:15.827 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:15.828 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:54:15.828 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:54:15.828 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:15.829 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:54:15.829 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:54:15.829 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:54:15.830 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:54:15.830 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:54:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[30791]: audit 2021-03-10T15:54:15.340189+0000 mgr.gibba032.whopeo (mgr.24625) 6417 : audit [DBG] from='client.58339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[30791]: audit 2021-03-10T15:54:15.563354+0000 mgr.gibba032.whopeo (mgr.24625) 6418 : audit [DBG] from='client.48363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[30791]: cluster 2021-03-10T15:54:15.755140+0000 mgr.gibba032.whopeo (mgr.24625) 6419 : cluster [DBG] pgmap v3652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[30791]: audit 2021-03-10T15:54:15.825144+0000 mon.a (mon.0) 2374 : audit [DBG] from='client.? 172.21.2.111:0/3938178692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[31774]: audit 2021-03-10T15:54:15.340189+0000 mgr.gibba032.whopeo (mgr.24625) 6417 : audit [DBG] from='client.58339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[31774]: audit 2021-03-10T15:54:15.563354+0000 mgr.gibba032.whopeo (mgr.24625) 6418 : audit [DBG] from='client.48363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[31774]: cluster 2021-03-10T15:54:15.755140+0000 mgr.gibba032.whopeo (mgr.24625) 6419 : cluster [DBG] pgmap v3652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:16 gibba011 bash[31774]: audit 2021-03-10T15:54:15.825144+0000 mon.a (mon.0) 2374 : audit [DBG] from='client.? 172.21.2.111:0/3938178692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:16 gibba032 bash[24004]: audit 2021-03-10T15:54:15.340189+0000 mgr.gibba032.whopeo (mgr.24625) 6417 : audit [DBG] from='client.58339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:16 gibba032 bash[24004]: audit 2021-03-10T15:54:15.563354+0000 mgr.gibba032.whopeo (mgr.24625) 6418 : audit [DBG] from='client.48363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:16 gibba032 bash[24004]: cluster 2021-03-10T15:54:15.755140+0000 mgr.gibba032.whopeo (mgr.24625) 6419 : cluster [DBG] pgmap v3652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:16 gibba032 bash[24004]: audit 2021-03-10T15:54:15.825144+0000 mon.a (mon.0) 2374 : audit [DBG] from='client.? 172.21.2.111:0/3938178692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:18 gibba011 bash[30791]: cluster 2021-03-10T15:54:17.757028+0000 mgr.gibba032.whopeo (mgr.24625) 6420 : cluster [DBG] pgmap v3653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:18 gibba011 bash[31774]: cluster 2021-03-10T15:54:17.757028+0000 mgr.gibba032.whopeo (mgr.24625) 6420 : cluster [DBG] pgmap v3653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:18 gibba032 bash[24004]: cluster 2021-03-10T15:54:17.757028+0000 mgr.gibba032.whopeo (mgr.24625) 6420 : cluster [DBG] pgmap v3653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:20 gibba032 bash[24004]: cluster 2021-03-10T15:54:19.758319+0000 mgr.gibba032.whopeo (mgr.24625) 6421 : cluster [DBG] pgmap v3654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:20 gibba011 bash[30791]: cluster 2021-03-10T15:54:19.758319+0000 mgr.gibba032.whopeo (mgr.24625) 6421 : cluster [DBG] pgmap v3654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:20 gibba011 bash[31774]: cluster 2021-03-10T15:54:19.758319+0000 mgr.gibba032.whopeo (mgr.24625) 6421 : cluster [DBG] pgmap v3654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:23.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:22 gibba032 bash[24004]: cluster 2021-03-10T15:54:21.759840+0000 mgr.gibba032.whopeo (mgr.24625) 6422 : cluster [DBG] pgmap v3655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:22 gibba011 bash[30791]: cluster 2021-03-10T15:54:21.759840+0000 mgr.gibba032.whopeo (mgr.24625) 6422 : cluster [DBG] pgmap v3655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:22 gibba011 bash[31774]: cluster 2021-03-10T15:54:21.759840+0000 mgr.gibba032.whopeo (mgr.24625) 6422 : cluster [DBG] pgmap v3655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:24 gibba032 bash[24004]: cluster 2021-03-10T15:54:23.761484+0000 mgr.gibba032.whopeo (mgr.24625) 6423 : cluster [DBG] pgmap v3656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:24 gibba011 bash[30791]: cluster 2021-03-10T15:54:23.761484+0000 mgr.gibba032.whopeo (mgr.24625) 6423 : cluster [DBG] pgmap v3656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:24 gibba011 bash[31774]: cluster 2021-03-10T15:54:23.761484+0000 mgr.gibba032.whopeo (mgr.24625) 6423 : cluster [DBG] pgmap v3656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:25.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:25 gibba032 bash[24004]: cephadm 2021-03-10T15:54:24.344870+0000 mgr.gibba032.whopeo (mgr.24625) 6424 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:25.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:25 gibba011 bash[30791]: cephadm 2021-03-10T15:54:24.344870+0000 mgr.gibba032.whopeo (mgr.24625) 6424 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:25.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:25 gibba011 bash[31774]: cephadm 2021-03-10T15:54:24.344870+0000 mgr.gibba032.whopeo (mgr.24625) 6424 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:26 gibba032 bash[24004]: cluster 2021-03-10T15:54:25.762784+0000 mgr.gibba032.whopeo (mgr.24625) 6425 : cluster [DBG] pgmap v3657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:26 gibba011 bash[30791]: cluster 2021-03-10T15:54:25.762784+0000 mgr.gibba032.whopeo (mgr.24625) 6425 : cluster [DBG] pgmap v3657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:26 gibba011 bash[31774]: cluster 2021-03-10T15:54:25.762784+0000 mgr.gibba032.whopeo (mgr.24625) 6425 : cluster [DBG] pgmap v3657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:28 gibba032 bash[24004]: cluster 2021-03-10T15:54:27.764329+0000 mgr.gibba032.whopeo (mgr.24625) 6426 : cluster [DBG] pgmap v3658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:28 gibba011 bash[30791]: cluster 2021-03-10T15:54:27.764329+0000 mgr.gibba032.whopeo (mgr.24625) 6426 : cluster [DBG] pgmap v3658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:28 gibba011 bash[31774]: cluster 2021-03-10T15:54:27.764329+0000 mgr.gibba032.whopeo (mgr.24625) 6426 : cluster [DBG] pgmap v3658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:30 gibba032 bash[24004]: cluster 2021-03-10T15:54:29.765826+0000 mgr.gibba032.whopeo (mgr.24625) 6427 : cluster [DBG] pgmap v3659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:30 gibba011 bash[30791]: cluster 2021-03-10T15:54:29.765826+0000 mgr.gibba032.whopeo (mgr.24625) 6427 : cluster [DBG] pgmap v3659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:30 gibba011 bash[31774]: cluster 2021-03-10T15:54:29.765826+0000 mgr.gibba032.whopeo (mgr.24625) 6427 : cluster [DBG] pgmap v3659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:32 gibba032 bash[24004]: cluster 2021-03-10T15:54:31.767604+0000 mgr.gibba032.whopeo (mgr.24625) 6428 : cluster [DBG] pgmap v3660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:32 gibba011 bash[30791]: cluster 2021-03-10T15:54:31.767604+0000 mgr.gibba032.whopeo (mgr.24625) 6428 : cluster [DBG] pgmap v3660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:32 gibba011 bash[31774]: cluster 2021-03-10T15:54:31.767604+0000 mgr.gibba032.whopeo (mgr.24625) 6428 : cluster [DBG] pgmap v3660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:34 gibba032 bash[24004]: cluster 2021-03-10T15:54:33.769131+0000 mgr.gibba032.whopeo (mgr.24625) 6429 : cluster [DBG] pgmap v3661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:34 gibba011 bash[30791]: cluster 2021-03-10T15:54:33.769131+0000 mgr.gibba032.whopeo (mgr.24625) 6429 : cluster [DBG] pgmap v3661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:34 gibba011 bash[31774]: cluster 2021-03-10T15:54:33.769131+0000 mgr.gibba032.whopeo (mgr.24625) 6429 : cluster [DBG] pgmap v3661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:36 gibba032 bash[24004]: cluster 2021-03-10T15:54:35.770488+0000 mgr.gibba032.whopeo (mgr.24625) 6430 : cluster [DBG] pgmap v3662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:36 gibba011 bash[30791]: cluster 2021-03-10T15:54:35.770488+0000 mgr.gibba032.whopeo (mgr.24625) 6430 : cluster [DBG] pgmap v3662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:36 gibba011 bash[31774]: cluster 2021-03-10T15:54:35.770488+0000 mgr.gibba032.whopeo (mgr.24625) 6430 : cluster [DBG] pgmap v3662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:39.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:38 gibba032 bash[24004]: cluster 2021-03-10T15:54:37.772199+0000 mgr.gibba032.whopeo (mgr.24625) 6431 : cluster [DBG] pgmap v3663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:38 gibba011 bash[30791]: cluster 2021-03-10T15:54:37.772199+0000 mgr.gibba032.whopeo (mgr.24625) 6431 : cluster [DBG] pgmap v3663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:38 gibba011 bash[31774]: cluster 2021-03-10T15:54:37.772199+0000 mgr.gibba032.whopeo (mgr.24625) 6431 : cluster [DBG] pgmap v3663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.361415+0000 mgr.gibba032.whopeo (mgr.24625) 6432 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:54:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.363018+0000 mgr.gibba032.whopeo (mgr.24625) 6433 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:54:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.574403+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:54:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.575428+0000 mgr.gibba032.whopeo (mgr.24625) 6434 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:54:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.577638+0000 mgr.gibba032.whopeo (mgr.24625) 6435 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:54:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.577959+0000 mon.a (mon.0) 2376 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:54:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.579100+0000 mgr.gibba032.whopeo (mgr.24625) 6436 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:54:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.579136+0000 mgr.gibba032.whopeo (mgr.24625) 6437 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:54:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.579715+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.580547+0000 mgr.gibba032.whopeo (mgr.24625) 6438 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:54:41.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.580583+0000 mgr.gibba032.whopeo (mgr.24625) 6439 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:54:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.581275+0000 mon.a (mon.0) 2378 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.582106+0000 mgr.gibba032.whopeo (mgr.24625) 6440 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:54:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.582725+0000 mon.a (mon.0) 2379 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.583875+0000 mgr.gibba032.whopeo (mgr.24625) 6441 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:54:41.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.583918+0000 mgr.gibba032.whopeo (mgr.24625) 6442 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:54:41.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.585486+0000 mgr.gibba032.whopeo (mgr.24625) 6443 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:54:41.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.585929+0000 mgr.gibba032.whopeo (mgr.24625) 6444 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:54:41.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.586198+0000 mon.a (mon.0) 2380 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.587316+0000 mgr.gibba032.whopeo (mgr.24625) 6445 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:54:41.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.587350+0000 mgr.gibba032.whopeo (mgr.24625) 6446 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:54:41.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cephadm 2021-03-10T15:54:39.587456+0000 mgr.gibba032.whopeo (mgr.24625) 6447 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:41.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: audit 2021-03-10T15:54:39.588023+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:40 gibba032 bash[24004]: cluster 2021-03-10T15:54:39.773401+0000 mgr.gibba032.whopeo (mgr.24625) 6448 : cluster [DBG] pgmap v3664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.361415+0000 mgr.gibba032.whopeo (mgr.24625) 6432 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:54:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.363018+0000 mgr.gibba032.whopeo (mgr.24625) 6433 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:54:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.574403+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:54:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.575428+0000 mgr.gibba032.whopeo (mgr.24625) 6434 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:54:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.577638+0000 mgr.gibba032.whopeo (mgr.24625) 6435 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:54:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.577959+0000 mon.a (mon.0) 2376 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:54:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.579100+0000 mgr.gibba032.whopeo (mgr.24625) 6436 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:54:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.579136+0000 mgr.gibba032.whopeo (mgr.24625) 6437 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:54:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.579715+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.580547+0000 mgr.gibba032.whopeo (mgr.24625) 6438 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:54:41.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.580583+0000 mgr.gibba032.whopeo (mgr.24625) 6439 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:54:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.581275+0000 mon.a (mon.0) 2378 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.582106+0000 mgr.gibba032.whopeo (mgr.24625) 6440 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:54:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.582725+0000 mon.a (mon.0) 2379 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.583875+0000 mgr.gibba032.whopeo (mgr.24625) 6441 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:54:41.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.583918+0000 mgr.gibba032.whopeo (mgr.24625) 6442 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:54:41.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.585486+0000 mgr.gibba032.whopeo (mgr.24625) 6443 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:54:41.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.585929+0000 mgr.gibba032.whopeo (mgr.24625) 6444 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:54:41.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.586198+0000 mon.a (mon.0) 2380 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.587316+0000 mgr.gibba032.whopeo (mgr.24625) 6445 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:54:41.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.587350+0000 mgr.gibba032.whopeo (mgr.24625) 6446 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:54:41.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cephadm 2021-03-10T15:54:39.587456+0000 mgr.gibba032.whopeo (mgr.24625) 6447 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:41.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: audit 2021-03-10T15:54:39.588023+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[30791]: cluster 2021-03-10T15:54:39.773401+0000 mgr.gibba032.whopeo (mgr.24625) 6448 : cluster [DBG] pgmap v3664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.361415+0000 mgr.gibba032.whopeo (mgr.24625) 6432 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:54:41.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.363018+0000 mgr.gibba032.whopeo (mgr.24625) 6433 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:54:41.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.574403+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:54:41.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.575428+0000 mgr.gibba032.whopeo (mgr.24625) 6434 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:54:41.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.577638+0000 mgr.gibba032.whopeo (mgr.24625) 6435 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:54:41.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.577959+0000 mon.a (mon.0) 2376 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:54:41.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.579100+0000 mgr.gibba032.whopeo (mgr.24625) 6436 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:54:41.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.579136+0000 mgr.gibba032.whopeo (mgr.24625) 6437 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:54:41.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.579715+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.580547+0000 mgr.gibba032.whopeo (mgr.24625) 6438 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:54:41.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.580583+0000 mgr.gibba032.whopeo (mgr.24625) 6439 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:54:41.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.581275+0000 mon.a (mon.0) 2378 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.582106+0000 mgr.gibba032.whopeo (mgr.24625) 6440 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:54:41.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.582725+0000 mon.a (mon.0) 2379 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.583875+0000 mgr.gibba032.whopeo (mgr.24625) 6441 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:54:41.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.583918+0000 mgr.gibba032.whopeo (mgr.24625) 6442 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:54:41.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.585486+0000 mgr.gibba032.whopeo (mgr.24625) 6443 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:54:41.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.585929+0000 mgr.gibba032.whopeo (mgr.24625) 6444 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:54:41.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.586198+0000 mon.a (mon.0) 2380 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.587316+0000 mgr.gibba032.whopeo (mgr.24625) 6445 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:54:41.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.587350+0000 mgr.gibba032.whopeo (mgr.24625) 6446 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:54:41.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cephadm 2021-03-10T15:54:39.587456+0000 mgr.gibba032.whopeo (mgr.24625) 6447 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:41.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: audit 2021-03-10T15:54:39.588023+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:41.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:40 gibba011 bash[31774]: cluster 2021-03-10T15:54:39.773401+0000 mgr.gibba032.whopeo (mgr.24625) 6448 : cluster [DBG] pgmap v3664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:42 gibba032 bash[24004]: cluster 2021-03-10T15:54:41.775047+0000 mgr.gibba032.whopeo (mgr.24625) 6449 : cluster [DBG] pgmap v3665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:42 gibba011 bash[30791]: cluster 2021-03-10T15:54:41.775047+0000 mgr.gibba032.whopeo (mgr.24625) 6449 : cluster [DBG] pgmap v3665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:42 gibba011 bash[31774]: cluster 2021-03-10T15:54:41.775047+0000 mgr.gibba032.whopeo (mgr.24625) 6449 : cluster [DBG] pgmap v3665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:44 gibba032 bash[24004]: cluster 2021-03-10T15:54:43.776673+0000 mgr.gibba032.whopeo (mgr.24625) 6450 : cluster [DBG] pgmap v3666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:44 gibba011 bash[30791]: cluster 2021-03-10T15:54:43.776673+0000 mgr.gibba032.whopeo (mgr.24625) 6450 : cluster [DBG] pgmap v3666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:44 gibba011 bash[31774]: cluster 2021-03-10T15:54:43.776673+0000 mgr.gibba032.whopeo (mgr.24625) 6450 : cluster [DBG] pgmap v3666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:46.066 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:54:46.293 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:54:46.294 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:54:46.294 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:54:46.294 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:54:46.294 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:54:46.295 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:54:46.295 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:54:46.295 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:54:46.295 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:54:46.296 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:54:46.296 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:54:46.296 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:54:46.296 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:54:46.296 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:54:46.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:54:46.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:54:46.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:54:46.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:54:46.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (116m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:54:46.298 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:54:46.298 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:54:46.542 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:54:46.542 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:54:46.543 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:54:46.543 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:46.543 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:54:46.544 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:54:46.544 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:46.544 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:54:46.544 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:54:46.545 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:54:46.545 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:54:46.545 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:54:46.545 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:54:46.546 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:54:46.546 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:54:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:46 gibba032 bash[24004]: cluster 2021-03-10T15:54:45.778021+0000 mgr.gibba032.whopeo (mgr.24625) 6451 : cluster [DBG] pgmap v3667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:46 gibba032 bash[24004]: audit 2021-03-10T15:54:46.542046+0000 mon.a (mon.0) 2382 : audit [DBG] from='client.? 172.21.2.111:0/1208862418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:46 gibba011 bash[30791]: cluster 2021-03-10T15:54:45.778021+0000 mgr.gibba032.whopeo (mgr.24625) 6451 : cluster [DBG] pgmap v3667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:46 gibba011 bash[30791]: audit 2021-03-10T15:54:46.542046+0000 mon.a (mon.0) 2382 : audit [DBG] from='client.? 172.21.2.111:0/1208862418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:46 gibba011 bash[31774]: cluster 2021-03-10T15:54:45.778021+0000 mgr.gibba032.whopeo (mgr.24625) 6451 : cluster [DBG] pgmap v3667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:46 gibba011 bash[31774]: audit 2021-03-10T15:54:46.542046+0000 mon.a (mon.0) 2382 : audit [DBG] from='client.? 172.21.2.111:0/1208862418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:54:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:47 gibba032 bash[24004]: audit 2021-03-10T15:54:46.055603+0000 mgr.gibba032.whopeo (mgr.24625) 6452 : audit [DBG] from='client.48375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:48.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:47 gibba032 bash[24004]: audit 2021-03-10T15:54:46.279676+0000 mgr.gibba032.whopeo (mgr.24625) 6453 : audit [DBG] from='client.58363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:47 gibba011 bash[30791]: audit 2021-03-10T15:54:46.055603+0000 mgr.gibba032.whopeo (mgr.24625) 6452 : audit [DBG] from='client.48375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:48.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:47 gibba011 bash[30791]: audit 2021-03-10T15:54:46.279676+0000 mgr.gibba032.whopeo (mgr.24625) 6453 : audit [DBG] from='client.58363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:47 gibba011 bash[31774]: audit 2021-03-10T15:54:46.055603+0000 mgr.gibba032.whopeo (mgr.24625) 6452 : audit [DBG] from='client.48375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:48.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:47 gibba011 bash[31774]: audit 2021-03-10T15:54:46.279676+0000 mgr.gibba032.whopeo (mgr.24625) 6453 : audit [DBG] from='client.58363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:54:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:48 gibba032 bash[24004]: cluster 2021-03-10T15:54:47.779846+0000 mgr.gibba032.whopeo (mgr.24625) 6454 : cluster [DBG] pgmap v3668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:48 gibba011 bash[30791]: cluster 2021-03-10T15:54:47.779846+0000 mgr.gibba032.whopeo (mgr.24625) 6454 : cluster [DBG] pgmap v3668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:48 gibba011 bash[31774]: cluster 2021-03-10T15:54:47.779846+0000 mgr.gibba032.whopeo (mgr.24625) 6454 : cluster [DBG] pgmap v3668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:50 gibba032 bash[24004]: cluster 2021-03-10T15:54:49.781365+0000 mgr.gibba032.whopeo (mgr.24625) 6455 : cluster [DBG] pgmap v3669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:50 gibba011 bash[30791]: cluster 2021-03-10T15:54:49.781365+0000 mgr.gibba032.whopeo (mgr.24625) 6455 : cluster [DBG] pgmap v3669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:50 gibba011 bash[31774]: cluster 2021-03-10T15:54:49.781365+0000 mgr.gibba032.whopeo (mgr.24625) 6455 : cluster [DBG] pgmap v3669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:52 gibba032 bash[24004]: cluster 2021-03-10T15:54:51.783142+0000 mgr.gibba032.whopeo (mgr.24625) 6456 : cluster [DBG] pgmap v3670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:52 gibba011 bash[30791]: cluster 2021-03-10T15:54:51.783142+0000 mgr.gibba032.whopeo (mgr.24625) 6456 : cluster [DBG] pgmap v3670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:52 gibba011 bash[31774]: cluster 2021-03-10T15:54:51.783142+0000 mgr.gibba032.whopeo (mgr.24625) 6456 : cluster [DBG] pgmap v3670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:54 gibba032 bash[24004]: cluster 2021-03-10T15:54:53.784716+0000 mgr.gibba032.whopeo (mgr.24625) 6457 : cluster [DBG] pgmap v3671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:54 gibba011 bash[30791]: cluster 2021-03-10T15:54:53.784716+0000 mgr.gibba032.whopeo (mgr.24625) 6457 : cluster [DBG] pgmap v3671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:54 gibba011 bash[31774]: cluster 2021-03-10T15:54:53.784716+0000 mgr.gibba032.whopeo (mgr.24625) 6457 : cluster [DBG] pgmap v3671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:55.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:55 gibba032 bash[24004]: cephadm 2021-03-10T15:54:54.602545+0000 mgr.gibba032.whopeo (mgr.24625) 6458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:55.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:55 gibba011 bash[30791]: cephadm 2021-03-10T15:54:54.602545+0000 mgr.gibba032.whopeo (mgr.24625) 6458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:55.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:55 gibba011 bash[31774]: cephadm 2021-03-10T15:54:54.602545+0000 mgr.gibba032.whopeo (mgr.24625) 6458 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:54:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:56 gibba032 bash[24004]: cluster 2021-03-10T15:54:55.786073+0000 mgr.gibba032.whopeo (mgr.24625) 6459 : cluster [DBG] pgmap v3672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:56 gibba011 bash[30791]: cluster 2021-03-10T15:54:55.786073+0000 mgr.gibba032.whopeo (mgr.24625) 6459 : cluster [DBG] pgmap v3672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:56 gibba011 bash[31774]: cluster 2021-03-10T15:54:55.786073+0000 mgr.gibba032.whopeo (mgr.24625) 6459 : cluster [DBG] pgmap v3672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.761691+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:54:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.766026+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:54:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.803711+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:54:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.804447+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:54:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.805174+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T15:54:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:57 gibba032 bash[24004]: audit 2021-03-10T15:54:56.805856+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:54:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.761691+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:54:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.766026+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:54:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.803711+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:54:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.804447+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:54:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.805174+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T15:54:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[30791]: audit 2021-03-10T15:54:56.805856+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:54:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.761691+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:54:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.766026+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:54:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.803711+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T15:54:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.804447+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:54:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.805174+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T15:54:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:57 gibba011 bash[31774]: audit 2021-03-10T15:54:56.805856+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:54:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:54:58 gibba032 bash[24004]: cluster 2021-03-10T15:54:57.787774+0000 mgr.gibba032.whopeo (mgr.24625) 6460 : cluster [DBG] pgmap v3673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:54:58 gibba011 bash[30791]: cluster 2021-03-10T15:54:57.787774+0000 mgr.gibba032.whopeo (mgr.24625) 6460 : cluster [DBG] pgmap v3673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:54:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:54:58 gibba011 bash[31774]: cluster 2021-03-10T15:54:57.787774+0000 mgr.gibba032.whopeo (mgr.24625) 6460 : cluster [DBG] pgmap v3673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:00 gibba032 bash[24004]: cluster 2021-03-10T15:54:59.789014+0000 mgr.gibba032.whopeo (mgr.24625) 6461 : cluster [DBG] pgmap v3674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:00 gibba011 bash[30791]: cluster 2021-03-10T15:54:59.789014+0000 mgr.gibba032.whopeo (mgr.24625) 6461 : cluster [DBG] pgmap v3674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:00 gibba011 bash[31774]: cluster 2021-03-10T15:54:59.789014+0000 mgr.gibba032.whopeo (mgr.24625) 6461 : cluster [DBG] pgmap v3674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:02 gibba032 bash[24004]: cluster 2021-03-10T15:55:01.790847+0000 mgr.gibba032.whopeo (mgr.24625) 6462 : cluster [DBG] pgmap v3675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:02 gibba011 bash[30791]: cluster 2021-03-10T15:55:01.790847+0000 mgr.gibba032.whopeo (mgr.24625) 6462 : cluster [DBG] pgmap v3675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:02 gibba011 bash[31774]: cluster 2021-03-10T15:55:01.790847+0000 mgr.gibba032.whopeo (mgr.24625) 6462 : cluster [DBG] pgmap v3675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:04 gibba032 bash[24004]: cluster 2021-03-10T15:55:03.792529+0000 mgr.gibba032.whopeo (mgr.24625) 6463 : cluster [DBG] pgmap v3676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:04 gibba011 bash[30791]: cluster 2021-03-10T15:55:03.792529+0000 mgr.gibba032.whopeo (mgr.24625) 6463 : cluster [DBG] pgmap v3676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:04 gibba011 bash[31774]: cluster 2021-03-10T15:55:03.792529+0000 mgr.gibba032.whopeo (mgr.24625) 6463 : cluster [DBG] pgmap v3676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:06 gibba032 bash[24004]: cluster 2021-03-10T15:55:05.793842+0000 mgr.gibba032.whopeo (mgr.24625) 6464 : cluster [DBG] pgmap v3677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:06 gibba011 bash[30791]: cluster 2021-03-10T15:55:05.793842+0000 mgr.gibba032.whopeo (mgr.24625) 6464 : cluster [DBG] pgmap v3677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:06 gibba011 bash[31774]: cluster 2021-03-10T15:55:05.793842+0000 mgr.gibba032.whopeo (mgr.24625) 6464 : cluster [DBG] pgmap v3677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:08 gibba032 bash[24004]: cluster 2021-03-10T15:55:07.795645+0000 mgr.gibba032.whopeo (mgr.24625) 6465 : cluster [DBG] pgmap v3678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:08 gibba011 bash[30791]: cluster 2021-03-10T15:55:07.795645+0000 mgr.gibba032.whopeo (mgr.24625) 6465 : cluster [DBG] pgmap v3678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:08 gibba011 bash[31774]: cluster 2021-03-10T15:55:07.795645+0000 mgr.gibba032.whopeo (mgr.24625) 6465 : cluster [DBG] pgmap v3678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:11.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:10 gibba032 bash[24004]: cephadm 2021-03-10T15:55:09.615296+0000 mgr.gibba032.whopeo (mgr.24625) 6466 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:10 gibba032 bash[24004]: cluster 2021-03-10T15:55:09.797116+0000 mgr.gibba032.whopeo (mgr.24625) 6467 : cluster [DBG] pgmap v3679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:10 gibba011 bash[30791]: cephadm 2021-03-10T15:55:09.615296+0000 mgr.gibba032.whopeo (mgr.24625) 6466 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:10 gibba011 bash[30791]: cluster 2021-03-10T15:55:09.797116+0000 mgr.gibba032.whopeo (mgr.24625) 6467 : cluster [DBG] pgmap v3679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:10 gibba011 bash[31774]: cephadm 2021-03-10T15:55:09.615296+0000 mgr.gibba032.whopeo (mgr.24625) 6466 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:10 gibba011 bash[31774]: cluster 2021-03-10T15:55:09.797116+0000 mgr.gibba032.whopeo (mgr.24625) 6467 : cluster [DBG] pgmap v3679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:13.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:12 gibba032 bash[24004]: cluster 2021-03-10T15:55:11.798839+0000 mgr.gibba032.whopeo (mgr.24625) 6468 : cluster [DBG] pgmap v3680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:12 gibba011 bash[30791]: cluster 2021-03-10T15:55:11.798839+0000 mgr.gibba032.whopeo (mgr.24625) 6468 : cluster [DBG] pgmap v3680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:12 gibba011 bash[31774]: cluster 2021-03-10T15:55:11.798839+0000 mgr.gibba032.whopeo (mgr.24625) 6468 : cluster [DBG] pgmap v3680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:14 gibba032 bash[24004]: cluster 2021-03-10T15:55:13.800474+0000 mgr.gibba032.whopeo (mgr.24625) 6469 : cluster [DBG] pgmap v3681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:14 gibba011 bash[30791]: cluster 2021-03-10T15:55:13.800474+0000 mgr.gibba032.whopeo (mgr.24625) 6469 : cluster [DBG] pgmap v3681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:14 gibba011 bash[31774]: cluster 2021-03-10T15:55:13.800474+0000 mgr.gibba032.whopeo (mgr.24625) 6469 : cluster [DBG] pgmap v3681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:16.788 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:55:17.015 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:55:17.015 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:55:17.016 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:55:17.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:55:17.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:55:17.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:55:17.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:55:17.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:55:17.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:55:17.017 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:55:17.017 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:55:17.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:55:17.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:55:17.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:55:17.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:55:17.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:55:17.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (118m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:55:17.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:55:17.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (117m) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:55:17.020 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:55:17.020 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:55:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:16 gibba032 bash[24004]: cluster 2021-03-10T15:55:15.801856+0000 mgr.gibba032.whopeo (mgr.24625) 6470 : cluster [DBG] pgmap v3682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:17.262 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:55:17.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:55:17.262 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:55:17.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:17.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:55:17.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:55:17.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:17.263 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:55:17.264 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:55:17.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:17.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:55:17.264 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:55:17.265 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:55:17.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:55:17.265 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:55:17.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:16 gibba011 bash[30791]: cluster 2021-03-10T15:55:15.801856+0000 mgr.gibba032.whopeo (mgr.24625) 6470 : cluster [DBG] pgmap v3682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:16 gibba011 bash[31774]: cluster 2021-03-10T15:55:15.801856+0000 mgr.gibba032.whopeo (mgr.24625) 6470 : cluster [DBG] pgmap v3682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:17 gibba032 bash[24004]: audit 2021-03-10T15:55:16.777248+0000 mgr.gibba032.whopeo (mgr.24625) 6471 : audit [DBG] from='client.58375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:17 gibba032 bash[24004]: audit 2021-03-10T15:55:17.001322+0000 mgr.gibba032.whopeo (mgr.24625) 6472 : audit [DBG] from='client.48399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:17 gibba032 bash[24004]: audit 2021-03-10T15:55:17.262041+0000 mon.c (mon.1) 122 : audit [DBG] from='client.? 172.21.2.111:0/3927352564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[30791]: audit 2021-03-10T15:55:16.777248+0000 mgr.gibba032.whopeo (mgr.24625) 6471 : audit [DBG] from='client.58375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[30791]: audit 2021-03-10T15:55:17.001322+0000 mgr.gibba032.whopeo (mgr.24625) 6472 : audit [DBG] from='client.48399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[30791]: audit 2021-03-10T15:55:17.262041+0000 mon.c (mon.1) 122 : audit [DBG] from='client.? 172.21.2.111:0/3927352564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[31774]: audit 2021-03-10T15:55:16.777248+0000 mgr.gibba032.whopeo (mgr.24625) 6471 : audit [DBG] from='client.58375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[31774]: audit 2021-03-10T15:55:17.001322+0000 mgr.gibba032.whopeo (mgr.24625) 6472 : audit [DBG] from='client.48399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:18.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:17 gibba011 bash[31774]: audit 2021-03-10T15:55:17.262041+0000 mon.c (mon.1) 122 : audit [DBG] from='client.? 172.21.2.111:0/3927352564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:18 gibba032 bash[24004]: cluster 2021-03-10T15:55:17.803660+0000 mgr.gibba032.whopeo (mgr.24625) 6473 : cluster [DBG] pgmap v3683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:18 gibba011 bash[30791]: cluster 2021-03-10T15:55:17.803660+0000 mgr.gibba032.whopeo (mgr.24625) 6473 : cluster [DBG] pgmap v3683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:18 gibba011 bash[31774]: cluster 2021-03-10T15:55:17.803660+0000 mgr.gibba032.whopeo (mgr.24625) 6473 : cluster [DBG] pgmap v3683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:20 gibba032 bash[24004]: cluster 2021-03-10T15:55:19.805032+0000 mgr.gibba032.whopeo (mgr.24625) 6474 : cluster [DBG] pgmap v3684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:20 gibba011 bash[30791]: cluster 2021-03-10T15:55:19.805032+0000 mgr.gibba032.whopeo (mgr.24625) 6474 : cluster [DBG] pgmap v3684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:20 gibba011 bash[31774]: cluster 2021-03-10T15:55:19.805032+0000 mgr.gibba032.whopeo (mgr.24625) 6474 : cluster [DBG] pgmap v3684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:22 gibba032 bash[24004]: cluster 2021-03-10T15:55:21.806763+0000 mgr.gibba032.whopeo (mgr.24625) 6475 : cluster [DBG] pgmap v3685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:22 gibba011 bash[30791]: cluster 2021-03-10T15:55:21.806763+0000 mgr.gibba032.whopeo (mgr.24625) 6475 : cluster [DBG] pgmap v3685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:22 gibba011 bash[31774]: cluster 2021-03-10T15:55:21.806763+0000 mgr.gibba032.whopeo (mgr.24625) 6475 : cluster [DBG] pgmap v3685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:24 gibba032 bash[24004]: cluster 2021-03-10T15:55:23.808416+0000 mgr.gibba032.whopeo (mgr.24625) 6476 : cluster [DBG] pgmap v3686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:24 gibba011 bash[30791]: cluster 2021-03-10T15:55:23.808416+0000 mgr.gibba032.whopeo (mgr.24625) 6476 : cluster [DBG] pgmap v3686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:24 gibba011 bash[31774]: cluster 2021-03-10T15:55:23.808416+0000 mgr.gibba032.whopeo (mgr.24625) 6476 : cluster [DBG] pgmap v3686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:25.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:25 gibba032 bash[24004]: cephadm 2021-03-10T15:55:24.631312+0000 mgr.gibba032.whopeo (mgr.24625) 6477 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:25.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:25 gibba011 bash[30791]: cephadm 2021-03-10T15:55:24.631312+0000 mgr.gibba032.whopeo (mgr.24625) 6477 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:25.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:25 gibba011 bash[31774]: cephadm 2021-03-10T15:55:24.631312+0000 mgr.gibba032.whopeo (mgr.24625) 6477 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:26 gibba032 bash[24004]: cluster 2021-03-10T15:55:25.809789+0000 mgr.gibba032.whopeo (mgr.24625) 6478 : cluster [DBG] pgmap v3687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:26 gibba011 bash[30791]: cluster 2021-03-10T15:55:25.809789+0000 mgr.gibba032.whopeo (mgr.24625) 6478 : cluster [DBG] pgmap v3687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:26 gibba011 bash[31774]: cluster 2021-03-10T15:55:25.809789+0000 mgr.gibba032.whopeo (mgr.24625) 6478 : cluster [DBG] pgmap v3687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:28 gibba032 bash[24004]: cluster 2021-03-10T15:55:27.811427+0000 mgr.gibba032.whopeo (mgr.24625) 6479 : cluster [DBG] pgmap v3688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:28 gibba011 bash[30791]: cluster 2021-03-10T15:55:27.811427+0000 mgr.gibba032.whopeo (mgr.24625) 6479 : cluster [DBG] pgmap v3688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:28 gibba011 bash[31774]: cluster 2021-03-10T15:55:27.811427+0000 mgr.gibba032.whopeo (mgr.24625) 6479 : cluster [DBG] pgmap v3688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:30 gibba032 bash[24004]: cluster 2021-03-10T15:55:29.812935+0000 mgr.gibba032.whopeo (mgr.24625) 6480 : cluster [DBG] pgmap v3689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:30 gibba011 bash[30791]: cluster 2021-03-10T15:55:29.812935+0000 mgr.gibba032.whopeo (mgr.24625) 6480 : cluster [DBG] pgmap v3689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:30 gibba011 bash[31774]: cluster 2021-03-10T15:55:29.812935+0000 mgr.gibba032.whopeo (mgr.24625) 6480 : cluster [DBG] pgmap v3689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:32 gibba032 bash[24004]: cluster 2021-03-10T15:55:31.814664+0000 mgr.gibba032.whopeo (mgr.24625) 6481 : cluster [DBG] pgmap v3690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:32 gibba011 bash[30791]: cluster 2021-03-10T15:55:31.814664+0000 mgr.gibba032.whopeo (mgr.24625) 6481 : cluster [DBG] pgmap v3690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:32 gibba011 bash[31774]: cluster 2021-03-10T15:55:31.814664+0000 mgr.gibba032.whopeo (mgr.24625) 6481 : cluster [DBG] pgmap v3690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:34 gibba032 bash[24004]: cluster 2021-03-10T15:55:33.816316+0000 mgr.gibba032.whopeo (mgr.24625) 6482 : cluster [DBG] pgmap v3691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:34 gibba011 bash[30791]: cluster 2021-03-10T15:55:33.816316+0000 mgr.gibba032.whopeo (mgr.24625) 6482 : cluster [DBG] pgmap v3691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:34 gibba011 bash[31774]: cluster 2021-03-10T15:55:33.816316+0000 mgr.gibba032.whopeo (mgr.24625) 6482 : cluster [DBG] pgmap v3691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:36 gibba032 bash[24004]: cluster 2021-03-10T15:55:35.817688+0000 mgr.gibba032.whopeo (mgr.24625) 6483 : cluster [DBG] pgmap v3692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:36 gibba011 bash[30791]: cluster 2021-03-10T15:55:35.817688+0000 mgr.gibba032.whopeo (mgr.24625) 6483 : cluster [DBG] pgmap v3692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:36 gibba011 bash[31774]: cluster 2021-03-10T15:55:35.817688+0000 mgr.gibba032.whopeo (mgr.24625) 6483 : cluster [DBG] pgmap v3692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:38 gibba032 bash[24004]: cluster 2021-03-10T15:55:37.819436+0000 mgr.gibba032.whopeo (mgr.24625) 6484 : cluster [DBG] pgmap v3693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:38 gibba011 bash[30791]: cluster 2021-03-10T15:55:37.819436+0000 mgr.gibba032.whopeo (mgr.24625) 6484 : cluster [DBG] pgmap v3693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:38 gibba011 bash[31774]: cluster 2021-03-10T15:55:37.819436+0000 mgr.gibba032.whopeo (mgr.24625) 6484 : cluster [DBG] pgmap v3693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.648198+0000 mgr.gibba032.whopeo (mgr.24625) 6485 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:55:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.649694+0000 mgr.gibba032.whopeo (mgr.24625) 6486 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:55:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cluster 2021-03-10T15:55:39.819943+0000 mgr.gibba032.whopeo (mgr.24625) 6487 : cluster [DBG] pgmap v3694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.854999+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:55:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.855683+0000 mgr.gibba032.whopeo (mgr.24625) 6488 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:55:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.857923+0000 mgr.gibba032.whopeo (mgr.24625) 6489 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:55:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.858289+0000 mon.a (mon.0) 2390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:55:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.860155+0000 mgr.gibba032.whopeo (mgr.24625) 6490 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:55:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.860191+0000 mgr.gibba032.whopeo (mgr.24625) 6491 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:55:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.860263+0000 mon.a (mon.0) 2391 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.862405+0000 mgr.gibba032.whopeo (mgr.24625) 6492 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:55:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.862443+0000 mgr.gibba032.whopeo (mgr.24625) 6493 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:55:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.862513+0000 mon.a (mon.0) 2392 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.864773+0000 mon.a (mon.0) 2393 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.864985+0000 mgr.gibba032.whopeo (mgr.24625) 6494 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:55:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.867208+0000 mgr.gibba032.whopeo (mgr.24625) 6495 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:55:41.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.867243+0000 mgr.gibba032.whopeo (mgr.24625) 6496 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:55:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.869654+0000 mon.a (mon.0) 2394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.869756+0000 mgr.gibba032.whopeo (mgr.24625) 6497 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:55:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.870166+0000 mgr.gibba032.whopeo (mgr.24625) 6498 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:55:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: audit 2021-03-10T15:55:39.872536+0000 mon.a (mon.0) 2395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.872553+0000 mgr.gibba032.whopeo (mgr.24625) 6499 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:55:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.872589+0000 mgr.gibba032.whopeo (mgr.24625) 6500 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:55:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:40 gibba032 bash[24004]: cephadm 2021-03-10T15:55:39.872699+0000 mgr.gibba032.whopeo (mgr.24625) 6501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.648198+0000 mgr.gibba032.whopeo (mgr.24625) 6485 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:55:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.649694+0000 mgr.gibba032.whopeo (mgr.24625) 6486 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:55:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cluster 2021-03-10T15:55:39.819943+0000 mgr.gibba032.whopeo (mgr.24625) 6487 : cluster [DBG] pgmap v3694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.854999+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:55:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.855683+0000 mgr.gibba032.whopeo (mgr.24625) 6488 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:55:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.857923+0000 mgr.gibba032.whopeo (mgr.24625) 6489 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:55:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.858289+0000 mon.a (mon.0) 2390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:55:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.860155+0000 mgr.gibba032.whopeo (mgr.24625) 6490 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:55:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.860191+0000 mgr.gibba032.whopeo (mgr.24625) 6491 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:55:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.860263+0000 mon.a (mon.0) 2391 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.862405+0000 mgr.gibba032.whopeo (mgr.24625) 6492 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:55:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.862443+0000 mgr.gibba032.whopeo (mgr.24625) 6493 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:55:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.862513+0000 mon.a (mon.0) 2392 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.864773+0000 mon.a (mon.0) 2393 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.864985+0000 mgr.gibba032.whopeo (mgr.24625) 6494 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:55:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.867208+0000 mgr.gibba032.whopeo (mgr.24625) 6495 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:55:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.867243+0000 mgr.gibba032.whopeo (mgr.24625) 6496 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:55:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.869654+0000 mon.a (mon.0) 2394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.869756+0000 mgr.gibba032.whopeo (mgr.24625) 6497 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:55:41.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.870166+0000 mgr.gibba032.whopeo (mgr.24625) 6498 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:55:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.648198+0000 mgr.gibba032.whopeo (mgr.24625) 6485 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:55:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.649694+0000 mgr.gibba032.whopeo (mgr.24625) 6486 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:55:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cluster 2021-03-10T15:55:39.819943+0000 mgr.gibba032.whopeo (mgr.24625) 6487 : cluster [DBG] pgmap v3694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.854999+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:55:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.855683+0000 mgr.gibba032.whopeo (mgr.24625) 6488 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:55:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.857923+0000 mgr.gibba032.whopeo (mgr.24625) 6489 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:55:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.858289+0000 mon.a (mon.0) 2390 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:55:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.860155+0000 mgr.gibba032.whopeo (mgr.24625) 6490 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:55:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.860191+0000 mgr.gibba032.whopeo (mgr.24625) 6491 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:55:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.860263+0000 mon.a (mon.0) 2391 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.862405+0000 mgr.gibba032.whopeo (mgr.24625) 6492 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:55:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.862443+0000 mgr.gibba032.whopeo (mgr.24625) 6493 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:55:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.862513+0000 mon.a (mon.0) 2392 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.864773+0000 mon.a (mon.0) 2393 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.864985+0000 mgr.gibba032.whopeo (mgr.24625) 6494 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:55:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.867208+0000 mgr.gibba032.whopeo (mgr.24625) 6495 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:55:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.867243+0000 mgr.gibba032.whopeo (mgr.24625) 6496 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:55:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.869654+0000 mon.a (mon.0) 2394 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.869756+0000 mgr.gibba032.whopeo (mgr.24625) 6497 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:55:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.870166+0000 mgr.gibba032.whopeo (mgr.24625) 6498 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:55:41.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: audit 2021-03-10T15:55:39.872536+0000 mon.a (mon.0) 2395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.872553+0000 mgr.gibba032.whopeo (mgr.24625) 6499 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:55:41.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.872589+0000 mgr.gibba032.whopeo (mgr.24625) 6500 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:55:41.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[30791]: cephadm 2021-03-10T15:55:39.872699+0000 mgr.gibba032.whopeo (mgr.24625) 6501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: audit 2021-03-10T15:55:39.872536+0000 mon.a (mon.0) 2395 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.872553+0000 mgr.gibba032.whopeo (mgr.24625) 6499 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:55:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.872589+0000 mgr.gibba032.whopeo (mgr.24625) 6500 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:55:41.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:40 gibba011 bash[31774]: cephadm 2021-03-10T15:55:39.872699+0000 mgr.gibba032.whopeo (mgr.24625) 6501 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:42 gibba032 bash[24004]: cluster 2021-03-10T15:55:41.821596+0000 mgr.gibba032.whopeo (mgr.24625) 6502 : cluster [DBG] pgmap v3695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:42 gibba011 bash[30791]: cluster 2021-03-10T15:55:41.821596+0000 mgr.gibba032.whopeo (mgr.24625) 6502 : cluster [DBG] pgmap v3695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:42 gibba011 bash[31774]: cluster 2021-03-10T15:55:41.821596+0000 mgr.gibba032.whopeo (mgr.24625) 6502 : cluster [DBG] pgmap v3695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:44 gibba032 bash[24004]: cluster 2021-03-10T15:55:43.823184+0000 mgr.gibba032.whopeo (mgr.24625) 6503 : cluster [DBG] pgmap v3696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:44 gibba011 bash[30791]: cluster 2021-03-10T15:55:43.823184+0000 mgr.gibba032.whopeo (mgr.24625) 6503 : cluster [DBG] pgmap v3696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:44 gibba011 bash[31774]: cluster 2021-03-10T15:55:43.823184+0000 mgr.gibba032.whopeo (mgr.24625) 6503 : cluster [DBG] pgmap v3696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:46 gibba032 bash[24004]: cluster 2021-03-10T15:55:45.824578+0000 mgr.gibba032.whopeo (mgr.24625) 6504 : cluster [DBG] pgmap v3697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:47.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:46 gibba011 bash[30791]: cluster 2021-03-10T15:55:45.824578+0000 mgr.gibba032.whopeo (mgr.24625) 6504 : cluster [DBG] pgmap v3697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:46 gibba011 bash[31774]: cluster 2021-03-10T15:55:45.824578+0000 mgr.gibba032.whopeo (mgr.24625) 6504 : cluster [DBG] pgmap v3697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:47.504 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:55:47.733 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:55:47.733 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:55:47.733 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:55:47.734 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:55:47.734 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:55:47.734 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:55:47.734 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:55:47.735 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:55:47.735 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:55:47.735 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:55:47.735 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:55:47.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:55:47.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:55:47.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:55:47.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:55:47.736 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:55:47.737 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:55:47.737 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:55:47.737 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (117m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:55:47.737 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:55:47.737 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:55:47.982 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:55:47.982 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:55:47.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:55:47.983 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:47.983 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:55:47.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:55:47.983 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:47.984 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:55:47.984 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:55:47.984 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:55:47.984 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:55:47.984 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:55:47.985 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:55:47.985 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:55:47.985 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:55:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:48 gibba032 bash[24004]: audit 2021-03-10T15:55:47.493785+0000 mgr.gibba032.whopeo (mgr.24625) 6505 : audit [DBG] from='client.48411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:48 gibba032 bash[24004]: audit 2021-03-10T15:55:47.718643+0000 mgr.gibba032.whopeo (mgr.24625) 6506 : audit [DBG] from='client.48417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:48 gibba032 bash[24004]: cluster 2021-03-10T15:55:47.826340+0000 mgr.gibba032.whopeo (mgr.24625) 6507 : cluster [DBG] pgmap v3698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:49.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:48 gibba032 bash[24004]: audit 2021-03-10T15:55:47.982328+0000 mon.a (mon.0) 2396 : audit [DBG] from='client.? 172.21.2.111:0/2347194013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[30791]: audit 2021-03-10T15:55:47.493785+0000 mgr.gibba032.whopeo (mgr.24625) 6505 : audit [DBG] from='client.48411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[30791]: audit 2021-03-10T15:55:47.718643+0000 mgr.gibba032.whopeo (mgr.24625) 6506 : audit [DBG] from='client.48417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[30791]: cluster 2021-03-10T15:55:47.826340+0000 mgr.gibba032.whopeo (mgr.24625) 6507 : cluster [DBG] pgmap v3698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:49.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[30791]: audit 2021-03-10T15:55:47.982328+0000 mon.a (mon.0) 2396 : audit [DBG] from='client.? 172.21.2.111:0/2347194013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[31774]: audit 2021-03-10T15:55:47.493785+0000 mgr.gibba032.whopeo (mgr.24625) 6505 : audit [DBG] from='client.48411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[31774]: audit 2021-03-10T15:55:47.718643+0000 mgr.gibba032.whopeo (mgr.24625) 6506 : audit [DBG] from='client.48417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:55:49.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[31774]: cluster 2021-03-10T15:55:47.826340+0000 mgr.gibba032.whopeo (mgr.24625) 6507 : cluster [DBG] pgmap v3698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:49.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:48 gibba011 bash[31774]: audit 2021-03-10T15:55:47.982328+0000 mon.a (mon.0) 2396 : audit [DBG] from='client.? 172.21.2.111:0/2347194013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:55:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:50 gibba032 bash[24004]: cluster 2021-03-10T15:55:49.827829+0000 mgr.gibba032.whopeo (mgr.24625) 6508 : cluster [DBG] pgmap v3699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:50 gibba011 bash[30791]: cluster 2021-03-10T15:55:49.827829+0000 mgr.gibba032.whopeo (mgr.24625) 6508 : cluster [DBG] pgmap v3699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:50 gibba011 bash[31774]: cluster 2021-03-10T15:55:49.827829+0000 mgr.gibba032.whopeo (mgr.24625) 6508 : cluster [DBG] pgmap v3699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:52 gibba032 bash[24004]: cluster 2021-03-10T15:55:51.829346+0000 mgr.gibba032.whopeo (mgr.24625) 6509 : cluster [DBG] pgmap v3700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:52 gibba011 bash[30791]: cluster 2021-03-10T15:55:51.829346+0000 mgr.gibba032.whopeo (mgr.24625) 6509 : cluster [DBG] pgmap v3700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:52 gibba011 bash[31774]: cluster 2021-03-10T15:55:51.829346+0000 mgr.gibba032.whopeo (mgr.24625) 6509 : cluster [DBG] pgmap v3700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:54 gibba032 bash[24004]: cluster 2021-03-10T15:55:53.830832+0000 mgr.gibba032.whopeo (mgr.24625) 6510 : cluster [DBG] pgmap v3701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:54 gibba011 bash[30791]: cluster 2021-03-10T15:55:53.830832+0000 mgr.gibba032.whopeo (mgr.24625) 6510 : cluster [DBG] pgmap v3701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:54 gibba011 bash[31774]: cluster 2021-03-10T15:55:53.830832+0000 mgr.gibba032.whopeo (mgr.24625) 6510 : cluster [DBG] pgmap v3701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:55.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:55 gibba032 bash[24004]: cephadm 2021-03-10T15:55:54.888760+0000 mgr.gibba032.whopeo (mgr.24625) 6511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:55.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:55 gibba011 bash[30791]: cephadm 2021-03-10T15:55:54.888760+0000 mgr.gibba032.whopeo (mgr.24625) 6511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:55.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:55 gibba011 bash[31774]: cephadm 2021-03-10T15:55:54.888760+0000 mgr.gibba032.whopeo (mgr.24625) 6511 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:55:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:56 gibba011 bash[30791]: cluster 2021-03-10T15:55:55.832216+0000 mgr.gibba032.whopeo (mgr.24625) 6512 : cluster [DBG] pgmap v3702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:56.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:56 gibba011 bash[31774]: cluster 2021-03-10T15:55:55.832216+0000 mgr.gibba032.whopeo (mgr.24625) 6512 : cluster [DBG] pgmap v3702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:56 gibba032 bash[24004]: cluster 2021-03-10T15:55:55.832216+0000 mgr.gibba032.whopeo (mgr.24625) 6512 : cluster [DBG] pgmap v3702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.770438+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:55:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.771168+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:55:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.810931+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T15:55:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.811710+0000 mon.a (mon.0) 2400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:55:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.812382+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:55:57.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[30791]: audit 2021-03-10T15:55:56.813097+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:55:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.770438+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:55:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.771168+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:55:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.810931+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T15:55:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.811710+0000 mon.a (mon.0) 2400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:55:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.812382+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:55:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:57 gibba011 bash[31774]: audit 2021-03-10T15:55:56.813097+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:55:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.770438+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:55:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.771168+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:55:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.810931+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T15:55:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.811710+0000 mon.a (mon.0) 2400 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T15:55:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.812382+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:55:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:57 gibba032 bash[24004]: audit 2021-03-10T15:55:56.813097+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:55:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:55:58 gibba011 bash[30791]: cluster 2021-03-10T15:55:57.834087+0000 mgr.gibba032.whopeo (mgr.24625) 6513 : cluster [DBG] pgmap v3703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:55:58 gibba011 bash[31774]: cluster 2021-03-10T15:55:57.834087+0000 mgr.gibba032.whopeo (mgr.24625) 6513 : cluster [DBG] pgmap v3703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:55:59.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:55:58 gibba032 bash[24004]: cluster 2021-03-10T15:55:57.834087+0000 mgr.gibba032.whopeo (mgr.24625) 6513 : cluster [DBG] pgmap v3703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:00 gibba032 bash[24004]: cluster 2021-03-10T15:55:59.835633+0000 mgr.gibba032.whopeo (mgr.24625) 6514 : cluster [DBG] pgmap v3704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:00 gibba011 bash[30791]: cluster 2021-03-10T15:55:59.835633+0000 mgr.gibba032.whopeo (mgr.24625) 6514 : cluster [DBG] pgmap v3704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:00 gibba011 bash[31774]: cluster 2021-03-10T15:55:59.835633+0000 mgr.gibba032.whopeo (mgr.24625) 6514 : cluster [DBG] pgmap v3704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:02 gibba032 bash[24004]: cluster 2021-03-10T15:56:01.837442+0000 mgr.gibba032.whopeo (mgr.24625) 6515 : cluster [DBG] pgmap v3705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:02 gibba011 bash[30791]: cluster 2021-03-10T15:56:01.837442+0000 mgr.gibba032.whopeo (mgr.24625) 6515 : cluster [DBG] pgmap v3705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:02 gibba011 bash[31774]: cluster 2021-03-10T15:56:01.837442+0000 mgr.gibba032.whopeo (mgr.24625) 6515 : cluster [DBG] pgmap v3705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:04 gibba032 bash[24004]: cluster 2021-03-10T15:56:03.839057+0000 mgr.gibba032.whopeo (mgr.24625) 6516 : cluster [DBG] pgmap v3706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:04 gibba011 bash[30791]: cluster 2021-03-10T15:56:03.839057+0000 mgr.gibba032.whopeo (mgr.24625) 6516 : cluster [DBG] pgmap v3706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:04 gibba011 bash[31774]: cluster 2021-03-10T15:56:03.839057+0000 mgr.gibba032.whopeo (mgr.24625) 6516 : cluster [DBG] pgmap v3706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:06 gibba032 bash[24004]: cluster 2021-03-10T15:56:05.840431+0000 mgr.gibba032.whopeo (mgr.24625) 6517 : cluster [DBG] pgmap v3707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:06 gibba011 bash[30791]: cluster 2021-03-10T15:56:05.840431+0000 mgr.gibba032.whopeo (mgr.24625) 6517 : cluster [DBG] pgmap v3707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:06 gibba011 bash[31774]: cluster 2021-03-10T15:56:05.840431+0000 mgr.gibba032.whopeo (mgr.24625) 6517 : cluster [DBG] pgmap v3707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:08 gibba032 bash[24004]: cluster 2021-03-10T15:56:07.842037+0000 mgr.gibba032.whopeo (mgr.24625) 6518 : cluster [DBG] pgmap v3708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:08 gibba011 bash[30791]: cluster 2021-03-10T15:56:07.842037+0000 mgr.gibba032.whopeo (mgr.24625) 6518 : cluster [DBG] pgmap v3708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:08 gibba011 bash[31774]: cluster 2021-03-10T15:56:07.842037+0000 mgr.gibba032.whopeo (mgr.24625) 6518 : cluster [DBG] pgmap v3708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:11.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:10 gibba032 bash[24004]: cluster 2021-03-10T15:56:09.843635+0000 mgr.gibba032.whopeo (mgr.24625) 6519 : cluster [DBG] pgmap v3709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:10 gibba032 bash[24004]: cephadm 2021-03-10T15:56:09.904866+0000 mgr.gibba032.whopeo (mgr.24625) 6520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:10 gibba011 bash[30791]: cluster 2021-03-10T15:56:09.843635+0000 mgr.gibba032.whopeo (mgr.24625) 6519 : cluster [DBG] pgmap v3709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:10 gibba011 bash[30791]: cephadm 2021-03-10T15:56:09.904866+0000 mgr.gibba032.whopeo (mgr.24625) 6520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:10 gibba011 bash[31774]: cluster 2021-03-10T15:56:09.843635+0000 mgr.gibba032.whopeo (mgr.24625) 6519 : cluster [DBG] pgmap v3709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:10 gibba011 bash[31774]: cephadm 2021-03-10T15:56:09.904866+0000 mgr.gibba032.whopeo (mgr.24625) 6520 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:13.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:12 gibba032 bash[24004]: cluster 2021-03-10T15:56:11.845397+0000 mgr.gibba032.whopeo (mgr.24625) 6521 : cluster [DBG] pgmap v3710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:13.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:12 gibba011 bash[30791]: cluster 2021-03-10T15:56:11.845397+0000 mgr.gibba032.whopeo (mgr.24625) 6521 : cluster [DBG] pgmap v3710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:12 gibba011 bash[31774]: cluster 2021-03-10T15:56:11.845397+0000 mgr.gibba032.whopeo (mgr.24625) 6521 : cluster [DBG] pgmap v3710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:14 gibba011 bash[30791]: cluster 2021-03-10T15:56:13.846974+0000 mgr.gibba032.whopeo (mgr.24625) 6522 : cluster [DBG] pgmap v3711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:14 gibba011 bash[31774]: cluster 2021-03-10T15:56:13.846974+0000 mgr.gibba032.whopeo (mgr.24625) 6522 : cluster [DBG] pgmap v3711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:14 gibba032 bash[24004]: cluster 2021-03-10T15:56:13.846974+0000 mgr.gibba032.whopeo (mgr.24625) 6522 : cluster [DBG] pgmap v3711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:16 gibba032 bash[24004]: cluster 2021-03-10T15:56:15.848490+0000 mgr.gibba032.whopeo (mgr.24625) 6523 : cluster [DBG] pgmap v3712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:16 gibba011 bash[30791]: cluster 2021-03-10T15:56:15.848490+0000 mgr.gibba032.whopeo (mgr.24625) 6523 : cluster [DBG] pgmap v3712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:16 gibba011 bash[31774]: cluster 2021-03-10T15:56:15.848490+0000 mgr.gibba032.whopeo (mgr.24625) 6523 : cluster [DBG] pgmap v3712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:18.230 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:56:18.456 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:56:18.457 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:56:18.457 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:56:18.457 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:56:18.457 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:56:18.458 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:56:18.458 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:56:18.458 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:56:18.458 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:56:18.459 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:56:18.459 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:56:18.459 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:56:18.459 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:56:18.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:56:18.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:56:18.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:56:18.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (119m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:56:18.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:56:18.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (118m) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:56:18.461 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:56:18.461 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:56:18.709 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:56:18.709 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:56:18.710 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:56:18.710 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:18.710 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:56:18.711 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:56:18.711 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:18.712 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:56:18.712 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:56:18.712 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:18.713 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:56:18.713 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:56:18.714 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:56:18.714 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:56:18.714 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:56:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:18 gibba032 bash[24004]: cluster 2021-03-10T15:56:17.850261+0000 mgr.gibba032.whopeo (mgr.24625) 6524 : cluster [DBG] pgmap v3713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:18 gibba032 bash[24004]: audit 2021-03-10T15:56:18.709456+0000 mon.a (mon.0) 2403 : audit [DBG] from='client.? 172.21.2.111:0/2014672350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:18 gibba011 bash[30791]: cluster 2021-03-10T15:56:17.850261+0000 mgr.gibba032.whopeo (mgr.24625) 6524 : cluster [DBG] pgmap v3713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:18 gibba011 bash[30791]: audit 2021-03-10T15:56:18.709456+0000 mon.a (mon.0) 2403 : audit [DBG] from='client.? 172.21.2.111:0/2014672350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:18 gibba011 bash[31774]: cluster 2021-03-10T15:56:17.850261+0000 mgr.gibba032.whopeo (mgr.24625) 6524 : cluster [DBG] pgmap v3713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:18 gibba011 bash[31774]: audit 2021-03-10T15:56:18.709456+0000 mon.a (mon.0) 2403 : audit [DBG] from='client.? 172.21.2.111:0/2014672350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:19 gibba032 bash[24004]: audit 2021-03-10T15:56:18.219043+0000 mgr.gibba032.whopeo (mgr.24625) 6525 : audit [DBG] from='client.58411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:20.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:19 gibba032 bash[24004]: audit 2021-03-10T15:56:18.442809+0000 mgr.gibba032.whopeo (mgr.24625) 6526 : audit [DBG] from='client.58417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:19 gibba011 bash[30791]: audit 2021-03-10T15:56:18.219043+0000 mgr.gibba032.whopeo (mgr.24625) 6525 : audit [DBG] from='client.58411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:19 gibba011 bash[30791]: audit 2021-03-10T15:56:18.442809+0000 mgr.gibba032.whopeo (mgr.24625) 6526 : audit [DBG] from='client.58417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:19 gibba011 bash[31774]: audit 2021-03-10T15:56:18.219043+0000 mgr.gibba032.whopeo (mgr.24625) 6525 : audit [DBG] from='client.58411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:19 gibba011 bash[31774]: audit 2021-03-10T15:56:18.442809+0000 mgr.gibba032.whopeo (mgr.24625) 6526 : audit [DBG] from='client.58417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:20 gibba032 bash[24004]: cluster 2021-03-10T15:56:19.851598+0000 mgr.gibba032.whopeo (mgr.24625) 6527 : cluster [DBG] pgmap v3714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:20 gibba011 bash[30791]: cluster 2021-03-10T15:56:19.851598+0000 mgr.gibba032.whopeo (mgr.24625) 6527 : cluster [DBG] pgmap v3714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:20 gibba011 bash[31774]: cluster 2021-03-10T15:56:19.851598+0000 mgr.gibba032.whopeo (mgr.24625) 6527 : cluster [DBG] pgmap v3714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:22 gibba032 bash[24004]: cluster 2021-03-10T15:56:21.853187+0000 mgr.gibba032.whopeo (mgr.24625) 6528 : cluster [DBG] pgmap v3715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:22 gibba011 bash[30791]: cluster 2021-03-10T15:56:21.853187+0000 mgr.gibba032.whopeo (mgr.24625) 6528 : cluster [DBG] pgmap v3715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:22 gibba011 bash[31774]: cluster 2021-03-10T15:56:21.853187+0000 mgr.gibba032.whopeo (mgr.24625) 6528 : cluster [DBG] pgmap v3715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:24 gibba011 bash[30791]: cluster 2021-03-10T15:56:23.854513+0000 mgr.gibba032.whopeo (mgr.24625) 6529 : cluster [DBG] pgmap v3716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:24 gibba011 bash[31774]: cluster 2021-03-10T15:56:23.854513+0000 mgr.gibba032.whopeo (mgr.24625) 6529 : cluster [DBG] pgmap v3716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:24.923 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:24 gibba032 bash[24004]: cluster 2021-03-10T15:56:23.854513+0000 mgr.gibba032.whopeo (mgr.24625) 6529 : cluster [DBG] pgmap v3716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:25.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:25 gibba011 bash[30791]: cephadm 2021-03-10T15:56:24.920873+0000 mgr.gibba032.whopeo (mgr.24625) 6530 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:25.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:25 gibba011 bash[31774]: cephadm 2021-03-10T15:56:24.920873+0000 mgr.gibba032.whopeo (mgr.24625) 6530 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:26.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:25 gibba032 bash[24004]: cephadm 2021-03-10T15:56:24.920873+0000 mgr.gibba032.whopeo (mgr.24625) 6530 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:26 gibba011 bash[30791]: cluster 2021-03-10T15:56:25.855916+0000 mgr.gibba032.whopeo (mgr.24625) 6531 : cluster [DBG] pgmap v3717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:26 gibba011 bash[31774]: cluster 2021-03-10T15:56:25.855916+0000 mgr.gibba032.whopeo (mgr.24625) 6531 : cluster [DBG] pgmap v3717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:26 gibba032 bash[24004]: cluster 2021-03-10T15:56:25.855916+0000 mgr.gibba032.whopeo (mgr.24625) 6531 : cluster [DBG] pgmap v3717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:28 gibba032 bash[24004]: cluster 2021-03-10T15:56:27.857499+0000 mgr.gibba032.whopeo (mgr.24625) 6532 : cluster [DBG] pgmap v3718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:29.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:28 gibba011 bash[30791]: cluster 2021-03-10T15:56:27.857499+0000 mgr.gibba032.whopeo (mgr.24625) 6532 : cluster [DBG] pgmap v3718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:28 gibba011 bash[31774]: cluster 2021-03-10T15:56:27.857499+0000 mgr.gibba032.whopeo (mgr.24625) 6532 : cluster [DBG] pgmap v3718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:30 gibba032 bash[24004]: cluster 2021-03-10T15:56:29.859035+0000 mgr.gibba032.whopeo (mgr.24625) 6533 : cluster [DBG] pgmap v3719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:30 gibba011 bash[30791]: cluster 2021-03-10T15:56:29.859035+0000 mgr.gibba032.whopeo (mgr.24625) 6533 : cluster [DBG] pgmap v3719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:30 gibba011 bash[31774]: cluster 2021-03-10T15:56:29.859035+0000 mgr.gibba032.whopeo (mgr.24625) 6533 : cluster [DBG] pgmap v3719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:32 gibba032 bash[24004]: cluster 2021-03-10T15:56:31.860734+0000 mgr.gibba032.whopeo (mgr.24625) 6534 : cluster [DBG] pgmap v3720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:32 gibba011 bash[30791]: cluster 2021-03-10T15:56:31.860734+0000 mgr.gibba032.whopeo (mgr.24625) 6534 : cluster [DBG] pgmap v3720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:32 gibba011 bash[31774]: cluster 2021-03-10T15:56:31.860734+0000 mgr.gibba032.whopeo (mgr.24625) 6534 : cluster [DBG] pgmap v3720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:34 gibba011 bash[30791]: cluster 2021-03-10T15:56:33.862107+0000 mgr.gibba032.whopeo (mgr.24625) 6535 : cluster [DBG] pgmap v3721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:34 gibba011 bash[31774]: cluster 2021-03-10T15:56:33.862107+0000 mgr.gibba032.whopeo (mgr.24625) 6535 : cluster [DBG] pgmap v3721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:34 gibba032 bash[24004]: cluster 2021-03-10T15:56:33.862107+0000 mgr.gibba032.whopeo (mgr.24625) 6535 : cluster [DBG] pgmap v3721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:36 gibba032 bash[24004]: cluster 2021-03-10T15:56:35.863539+0000 mgr.gibba032.whopeo (mgr.24625) 6536 : cluster [DBG] pgmap v3722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:36 gibba011 bash[30791]: cluster 2021-03-10T15:56:35.863539+0000 mgr.gibba032.whopeo (mgr.24625) 6536 : cluster [DBG] pgmap v3722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:36 gibba011 bash[31774]: cluster 2021-03-10T15:56:35.863539+0000 mgr.gibba032.whopeo (mgr.24625) 6536 : cluster [DBG] pgmap v3722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:38 gibba032 bash[24004]: cluster 2021-03-10T15:56:37.865382+0000 mgr.gibba032.whopeo (mgr.24625) 6537 : cluster [DBG] pgmap v3723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:38 gibba011 bash[30791]: cluster 2021-03-10T15:56:37.865382+0000 mgr.gibba032.whopeo (mgr.24625) 6537 : cluster [DBG] pgmap v3723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:38 gibba011 bash[31774]: cluster 2021-03-10T15:56:37.865382+0000 mgr.gibba032.whopeo (mgr.24625) 6537 : cluster [DBG] pgmap v3723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: cluster 2021-03-10T15:56:39.866878+0000 mgr.gibba032.whopeo (mgr.24625) 6538 : cluster [DBG] pgmap v3724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.123827+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.127680+0000 mon.a (mon.0) 2405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.313852+0000 mon.a (mon.0) 2406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.317158+0000 mon.a (mon.0) 2407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:56:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.318555+0000 mon.a (mon.0) 2408 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.319752+0000 mon.a (mon.0) 2409 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.320914+0000 mon.a (mon.0) 2410 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.323390+0000 mon.a (mon.0) 2411 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:40 gibba032 bash[24004]: audit 2021-03-10T15:56:40.324854+0000 mon.a (mon.0) 2412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: cluster 2021-03-10T15:56:39.866878+0000 mgr.gibba032.whopeo (mgr.24625) 6538 : cluster [DBG] pgmap v3724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.123827+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.127680+0000 mon.a (mon.0) 2405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.313852+0000 mon.a (mon.0) 2406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.317158+0000 mon.a (mon.0) 2407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:56:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.318555+0000 mon.a (mon.0) 2408 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.319752+0000 mon.a (mon.0) 2409 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.320914+0000 mon.a (mon.0) 2410 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.323390+0000 mon.a (mon.0) 2411 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[30791]: audit 2021-03-10T15:56:40.324854+0000 mon.a (mon.0) 2412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: cluster 2021-03-10T15:56:39.866878+0000 mgr.gibba032.whopeo (mgr.24625) 6538 : cluster [DBG] pgmap v3724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.123827+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.127680+0000 mon.a (mon.0) 2405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.313852+0000 mon.a (mon.0) 2406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:56:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.317158+0000 mon.a (mon.0) 2407 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:56:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.318555+0000 mon.a (mon.0) 2408 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.319752+0000 mon.a (mon.0) 2409 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.320914+0000 mon.a (mon.0) 2410 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.323390+0000 mon.a (mon.0) 2411 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:40 gibba011 bash[31774]: audit 2021-03-10T15:56:40.324854+0000 mon.a (mon.0) 2412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.122779+0000 mgr.gibba032.whopeo (mgr.24625) 6539 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:56:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.126604+0000 mgr.gibba032.whopeo (mgr.24625) 6540 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.314624+0000 mgr.gibba032.whopeo (mgr.24625) 6541 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.316399+0000 mgr.gibba032.whopeo (mgr.24625) 6542 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.317483+0000 mgr.gibba032.whopeo (mgr.24625) 6543 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:56:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.317558+0000 mgr.gibba032.whopeo (mgr.24625) 6544 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.318769+0000 mgr.gibba032.whopeo (mgr.24625) 6545 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.318803+0000 mgr.gibba032.whopeo (mgr.24625) 6546 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.320023+0000 mgr.gibba032.whopeo (mgr.24625) 6547 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.321236+0000 mgr.gibba032.whopeo (mgr.24625) 6548 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:56:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.321271+0000 mgr.gibba032.whopeo (mgr.24625) 6549 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:56:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.322510+0000 mgr.gibba032.whopeo (mgr.24625) 6550 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:56:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.322823+0000 mgr.gibba032.whopeo (mgr.24625) 6551 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:56:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.323898+0000 mgr.gibba032.whopeo (mgr.24625) 6552 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:56:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.323936+0000 mgr.gibba032.whopeo (mgr.24625) 6553 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:56:42.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:41 gibba032 bash[24004]: cephadm 2021-03-10T15:56:40.324153+0000 mgr.gibba032.whopeo (mgr.24625) 6554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.122779+0000 mgr.gibba032.whopeo (mgr.24625) 6539 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:56:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.126604+0000 mgr.gibba032.whopeo (mgr.24625) 6540 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.314624+0000 mgr.gibba032.whopeo (mgr.24625) 6541 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.316399+0000 mgr.gibba032.whopeo (mgr.24625) 6542 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.317483+0000 mgr.gibba032.whopeo (mgr.24625) 6543 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:56:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.317558+0000 mgr.gibba032.whopeo (mgr.24625) 6544 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.318769+0000 mgr.gibba032.whopeo (mgr.24625) 6545 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.318803+0000 mgr.gibba032.whopeo (mgr.24625) 6546 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.320023+0000 mgr.gibba032.whopeo (mgr.24625) 6547 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:56:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.321236+0000 mgr.gibba032.whopeo (mgr.24625) 6548 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.321271+0000 mgr.gibba032.whopeo (mgr.24625) 6549 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.322510+0000 mgr.gibba032.whopeo (mgr.24625) 6550 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.322823+0000 mgr.gibba032.whopeo (mgr.24625) 6551 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.323898+0000 mgr.gibba032.whopeo (mgr.24625) 6552 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:56:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.323936+0000 mgr.gibba032.whopeo (mgr.24625) 6553 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:56:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[30791]: cephadm 2021-03-10T15:56:40.324153+0000 mgr.gibba032.whopeo (mgr.24625) 6554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.122779+0000 mgr.gibba032.whopeo (mgr.24625) 6539 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:56:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.126604+0000 mgr.gibba032.whopeo (mgr.24625) 6540 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.314624+0000 mgr.gibba032.whopeo (mgr.24625) 6541 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.316399+0000 mgr.gibba032.whopeo (mgr.24625) 6542 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.317483+0000 mgr.gibba032.whopeo (mgr.24625) 6543 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:56:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.317558+0000 mgr.gibba032.whopeo (mgr.24625) 6544 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.318769+0000 mgr.gibba032.whopeo (mgr.24625) 6545 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.318803+0000 mgr.gibba032.whopeo (mgr.24625) 6546 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.320023+0000 mgr.gibba032.whopeo (mgr.24625) 6547 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:56:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.321236+0000 mgr.gibba032.whopeo (mgr.24625) 6548 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.321271+0000 mgr.gibba032.whopeo (mgr.24625) 6549 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.322510+0000 mgr.gibba032.whopeo (mgr.24625) 6550 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.322823+0000 mgr.gibba032.whopeo (mgr.24625) 6551 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.323898+0000 mgr.gibba032.whopeo (mgr.24625) 6552 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:56:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.323936+0000 mgr.gibba032.whopeo (mgr.24625) 6553 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:56:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:41 gibba011 bash[31774]: cephadm 2021-03-10T15:56:40.324153+0000 mgr.gibba032.whopeo (mgr.24625) 6554 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:42 gibba032 bash[24004]: cluster 2021-03-10T15:56:41.868665+0000 mgr.gibba032.whopeo (mgr.24625) 6555 : cluster [DBG] pgmap v3725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:42 gibba011 bash[30791]: cluster 2021-03-10T15:56:41.868665+0000 mgr.gibba032.whopeo (mgr.24625) 6555 : cluster [DBG] pgmap v3725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:42 gibba011 bash[31774]: cluster 2021-03-10T15:56:41.868665+0000 mgr.gibba032.whopeo (mgr.24625) 6555 : cluster [DBG] pgmap v3725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:44 gibba011 bash[30791]: cluster 2021-03-10T15:56:43.870140+0000 mgr.gibba032.whopeo (mgr.24625) 6556 : cluster [DBG] pgmap v3726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:44 gibba011 bash[31774]: cluster 2021-03-10T15:56:43.870140+0000 mgr.gibba032.whopeo (mgr.24625) 6556 : cluster [DBG] pgmap v3726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:44 gibba032 bash[24004]: cluster 2021-03-10T15:56:43.870140+0000 mgr.gibba032.whopeo (mgr.24625) 6556 : cluster [DBG] pgmap v3726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:46 gibba032 bash[24004]: cluster 2021-03-10T15:56:45.871487+0000 mgr.gibba032.whopeo (mgr.24625) 6557 : cluster [DBG] pgmap v3727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:46 gibba011 bash[30791]: cluster 2021-03-10T15:56:45.871487+0000 mgr.gibba032.whopeo (mgr.24625) 6557 : cluster [DBG] pgmap v3727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:46 gibba011 bash[31774]: cluster 2021-03-10T15:56:45.871487+0000 mgr.gibba032.whopeo (mgr.24625) 6557 : cluster [DBG] pgmap v3727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:48.956 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:56:49.182 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:56:49.183 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:56:49.183 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:56:49.183 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:56:49.183 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:56:49.184 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:56:49.184 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:56:49.184 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:56:49.184 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:56:49.185 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:56:49.185 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:56:49.185 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:56:49.185 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:56:49.186 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:56:49.186 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:56:49.186 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:56:49.186 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:56:49.186 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:56:49.187 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (118m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:56:49.187 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:56:49.187 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:56:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:48 gibba032 bash[24004]: cluster 2021-03-10T15:56:47.873100+0000 mgr.gibba032.whopeo (mgr.24625) 6558 : cluster [DBG] pgmap v3728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:48 gibba011 bash[30791]: cluster 2021-03-10T15:56:47.873100+0000 mgr.gibba032.whopeo (mgr.24625) 6558 : cluster [DBG] pgmap v3728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:49.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:48 gibba011 bash[31774]: cluster 2021-03-10T15:56:47.873100+0000 mgr.gibba032.whopeo (mgr.24625) 6558 : cluster [DBG] pgmap v3728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:49.430 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:56:49.430 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:56:49.430 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:56:49.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:49.431 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:56:49.431 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:56:49.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:49.432 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:56:49.432 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:56:49.432 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:56:49.432 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:56:49.432 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:56:49.433 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:56:49.433 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:56:49.433 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:56:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:49 gibba032 bash[24004]: audit 2021-03-10T15:56:48.945383+0000 mgr.gibba032.whopeo (mgr.24625) 6559 : audit [DBG] from='client.48447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:49 gibba032 bash[24004]: audit 2021-03-10T15:56:49.430454+0000 mon.c (mon.1) 123 : audit [DBG] from='client.? 172.21.2.111:0/973452659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:49 gibba011 bash[30791]: audit 2021-03-10T15:56:48.945383+0000 mgr.gibba032.whopeo (mgr.24625) 6559 : audit [DBG] from='client.48447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:49 gibba011 bash[30791]: audit 2021-03-10T15:56:49.430454+0000 mon.c (mon.1) 123 : audit [DBG] from='client.? 172.21.2.111:0/973452659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:49 gibba011 bash[31774]: audit 2021-03-10T15:56:48.945383+0000 mgr.gibba032.whopeo (mgr.24625) 6559 : audit [DBG] from='client.48447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:50.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:49 gibba011 bash[31774]: audit 2021-03-10T15:56:49.430454+0000 mon.c (mon.1) 123 : audit [DBG] from='client.? 172.21.2.111:0/973452659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:56:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:50 gibba032 bash[24004]: audit 2021-03-10T15:56:49.168961+0000 mgr.gibba032.whopeo (mgr.24625) 6560 : audit [DBG] from='client.58435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:50 gibba032 bash[24004]: cluster 2021-03-10T15:56:49.874506+0000 mgr.gibba032.whopeo (mgr.24625) 6561 : cluster [DBG] pgmap v3729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:50 gibba011 bash[30791]: audit 2021-03-10T15:56:49.168961+0000 mgr.gibba032.whopeo (mgr.24625) 6560 : audit [DBG] from='client.58435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:50 gibba011 bash[30791]: cluster 2021-03-10T15:56:49.874506+0000 mgr.gibba032.whopeo (mgr.24625) 6561 : cluster [DBG] pgmap v3729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:50 gibba011 bash[31774]: audit 2021-03-10T15:56:49.168961+0000 mgr.gibba032.whopeo (mgr.24625) 6560 : audit [DBG] from='client.58435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:56:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:50 gibba011 bash[31774]: cluster 2021-03-10T15:56:49.874506+0000 mgr.gibba032.whopeo (mgr.24625) 6561 : cluster [DBG] pgmap v3729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:52 gibba032 bash[24004]: cluster 2021-03-10T15:56:51.875982+0000 mgr.gibba032.whopeo (mgr.24625) 6562 : cluster [DBG] pgmap v3730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:52 gibba011 bash[30791]: cluster 2021-03-10T15:56:51.875982+0000 mgr.gibba032.whopeo (mgr.24625) 6562 : cluster [DBG] pgmap v3730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:52 gibba011 bash[31774]: cluster 2021-03-10T15:56:51.875982+0000 mgr.gibba032.whopeo (mgr.24625) 6562 : cluster [DBG] pgmap v3730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:54 gibba011 bash[30791]: cluster 2021-03-10T15:56:53.877405+0000 mgr.gibba032.whopeo (mgr.24625) 6563 : cluster [DBG] pgmap v3731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:54 gibba011 bash[31774]: cluster 2021-03-10T15:56:53.877405+0000 mgr.gibba032.whopeo (mgr.24625) 6563 : cluster [DBG] pgmap v3731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:54 gibba032 bash[24004]: cluster 2021-03-10T15:56:53.877405+0000 mgr.gibba032.whopeo (mgr.24625) 6563 : cluster [DBG] pgmap v3731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: cephadm 2021-03-10T15:56:55.340308+0000 mgr.gibba032.whopeo (mgr.24625) 6564 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: cluster 2021-03-10T15:56:55.878732+0000 mgr.gibba032.whopeo (mgr.24625) 6565 : cluster [DBG] pgmap v3732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.774792+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:56:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.775453+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:56:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.820970+0000 mon.a (mon.0) 2415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:56:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.821748+0000 mon.a (mon.0) 2416 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.822439+0000 mon.a (mon.0) 2417 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:56 gibba032 bash[24004]: audit 2021-03-10T15:56:56.823111+0000 mon.a (mon.0) 2418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:56:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: cephadm 2021-03-10T15:56:55.340308+0000 mgr.gibba032.whopeo (mgr.24625) 6564 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: cluster 2021-03-10T15:56:55.878732+0000 mgr.gibba032.whopeo (mgr.24625) 6565 : cluster [DBG] pgmap v3732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.774792+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:56:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.775453+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:56:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.820970+0000 mon.a (mon.0) 2415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:56:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.821748+0000 mon.a (mon.0) 2416 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.822439+0000 mon.a (mon.0) 2417 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[30791]: audit 2021-03-10T15:56:56.823111+0000 mon.a (mon.0) 2418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:56:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: cephadm 2021-03-10T15:56:55.340308+0000 mgr.gibba032.whopeo (mgr.24625) 6564 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:56:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: cluster 2021-03-10T15:56:55.878732+0000 mgr.gibba032.whopeo (mgr.24625) 6565 : cluster [DBG] pgmap v3732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.774792+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:56:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.775453+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:56:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.820970+0000 mon.a (mon.0) 2415 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T15:56:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.821748+0000 mon.a (mon.0) 2416 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.822439+0000 mon.a (mon.0) 2417 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T15:56:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:56 gibba011 bash[31774]: audit 2021-03-10T15:56:56.823111+0000 mon.a (mon.0) 2418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:56:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:56:58 gibba032 bash[24004]: cluster 2021-03-10T15:56:57.880475+0000 mgr.gibba032.whopeo (mgr.24625) 6566 : cluster [DBG] pgmap v3733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:56:58 gibba011 bash[30791]: cluster 2021-03-10T15:56:57.880475+0000 mgr.gibba032.whopeo (mgr.24625) 6566 : cluster [DBG] pgmap v3733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:56:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:56:58 gibba011 bash[31774]: cluster 2021-03-10T15:56:57.880475+0000 mgr.gibba032.whopeo (mgr.24625) 6566 : cluster [DBG] pgmap v3733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:00 gibba032 bash[24004]: cluster 2021-03-10T15:56:59.882000+0000 mgr.gibba032.whopeo (mgr.24625) 6567 : cluster [DBG] pgmap v3734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:00 gibba011 bash[30791]: cluster 2021-03-10T15:56:59.882000+0000 mgr.gibba032.whopeo (mgr.24625) 6567 : cluster [DBG] pgmap v3734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:00 gibba011 bash[31774]: cluster 2021-03-10T15:56:59.882000+0000 mgr.gibba032.whopeo (mgr.24625) 6567 : cluster [DBG] pgmap v3734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:03.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:02 gibba032 bash[24004]: cluster 2021-03-10T15:57:01.883687+0000 mgr.gibba032.whopeo (mgr.24625) 6568 : cluster [DBG] pgmap v3735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:02 gibba011 bash[30791]: cluster 2021-03-10T15:57:01.883687+0000 mgr.gibba032.whopeo (mgr.24625) 6568 : cluster [DBG] pgmap v3735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:02 gibba011 bash[31774]: cluster 2021-03-10T15:57:01.883687+0000 mgr.gibba032.whopeo (mgr.24625) 6568 : cluster [DBG] pgmap v3735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:04 gibba011 bash[30791]: cluster 2021-03-10T15:57:03.885295+0000 mgr.gibba032.whopeo (mgr.24625) 6569 : cluster [DBG] pgmap v3736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:04 gibba011 bash[31774]: cluster 2021-03-10T15:57:03.885295+0000 mgr.gibba032.whopeo (mgr.24625) 6569 : cluster [DBG] pgmap v3736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:04 gibba032 bash[24004]: cluster 2021-03-10T15:57:03.885295+0000 mgr.gibba032.whopeo (mgr.24625) 6569 : cluster [DBG] pgmap v3736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:06 gibba032 bash[24004]: cluster 2021-03-10T15:57:05.886685+0000 mgr.gibba032.whopeo (mgr.24625) 6570 : cluster [DBG] pgmap v3737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:06 gibba011 bash[30791]: cluster 2021-03-10T15:57:05.886685+0000 mgr.gibba032.whopeo (mgr.24625) 6570 : cluster [DBG] pgmap v3737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:06 gibba011 bash[31774]: cluster 2021-03-10T15:57:05.886685+0000 mgr.gibba032.whopeo (mgr.24625) 6570 : cluster [DBG] pgmap v3737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:08 gibba032 bash[24004]: cluster 2021-03-10T15:57:07.888489+0000 mgr.gibba032.whopeo (mgr.24625) 6571 : cluster [DBG] pgmap v3738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:08 gibba011 bash[30791]: cluster 2021-03-10T15:57:07.888489+0000 mgr.gibba032.whopeo (mgr.24625) 6571 : cluster [DBG] pgmap v3738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:08 gibba011 bash[31774]: cluster 2021-03-10T15:57:07.888489+0000 mgr.gibba032.whopeo (mgr.24625) 6571 : cluster [DBG] pgmap v3738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:10 gibba032 bash[24004]: cluster 2021-03-10T15:57:09.890034+0000 mgr.gibba032.whopeo (mgr.24625) 6572 : cluster [DBG] pgmap v3739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:10 gibba011 bash[30791]: cluster 2021-03-10T15:57:09.890034+0000 mgr.gibba032.whopeo (mgr.24625) 6572 : cluster [DBG] pgmap v3739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:10 gibba011 bash[31774]: cluster 2021-03-10T15:57:09.890034+0000 mgr.gibba032.whopeo (mgr.24625) 6572 : cluster [DBG] pgmap v3739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:11 gibba032 bash[24004]: cephadm 2021-03-10T15:57:10.356393+0000 mgr.gibba032.whopeo (mgr.24625) 6573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:11 gibba011 bash[30791]: cephadm 2021-03-10T15:57:10.356393+0000 mgr.gibba032.whopeo (mgr.24625) 6573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:11 gibba011 bash[31774]: cephadm 2021-03-10T15:57:10.356393+0000 mgr.gibba032.whopeo (mgr.24625) 6573 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:13.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:12 gibba032 bash[24004]: cluster 2021-03-10T15:57:11.892103+0000 mgr.gibba032.whopeo (mgr.24625) 6574 : cluster [DBG] pgmap v3740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:13.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:12 gibba011 bash[30791]: cluster 2021-03-10T15:57:11.892103+0000 mgr.gibba032.whopeo (mgr.24625) 6574 : cluster [DBG] pgmap v3740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:12 gibba011 bash[31774]: cluster 2021-03-10T15:57:11.892103+0000 mgr.gibba032.whopeo (mgr.24625) 6574 : cluster [DBG] pgmap v3740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:14 gibba011 bash[30791]: cluster 2021-03-10T15:57:13.893761+0000 mgr.gibba032.whopeo (mgr.24625) 6575 : cluster [DBG] pgmap v3741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:14 gibba011 bash[31774]: cluster 2021-03-10T15:57:13.893761+0000 mgr.gibba032.whopeo (mgr.24625) 6575 : cluster [DBG] pgmap v3741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:14 gibba032 bash[24004]: cluster 2021-03-10T15:57:13.893761+0000 mgr.gibba032.whopeo (mgr.24625) 6575 : cluster [DBG] pgmap v3741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:16 gibba032 bash[24004]: cluster 2021-03-10T15:57:15.895134+0000 mgr.gibba032.whopeo (mgr.24625) 6576 : cluster [DBG] pgmap v3742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:16 gibba011 bash[30791]: cluster 2021-03-10T15:57:15.895134+0000 mgr.gibba032.whopeo (mgr.24625) 6576 : cluster [DBG] pgmap v3742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:16 gibba011 bash[31774]: cluster 2021-03-10T15:57:15.895134+0000 mgr.gibba032.whopeo (mgr.24625) 6576 : cluster [DBG] pgmap v3742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:18 gibba032 bash[24004]: cluster 2021-03-10T15:57:17.896951+0000 mgr.gibba032.whopeo (mgr.24625) 6577 : cluster [DBG] pgmap v3743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:18 gibba011 bash[30791]: cluster 2021-03-10T15:57:17.896951+0000 mgr.gibba032.whopeo (mgr.24625) 6577 : cluster [DBG] pgmap v3743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:18 gibba011 bash[31774]: cluster 2021-03-10T15:57:17.896951+0000 mgr.gibba032.whopeo (mgr.24625) 6577 : cluster [DBG] pgmap v3743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:19.676 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:57:19.904 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:57:19.905 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:57:19.905 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:57:19.905 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:57:19.905 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:57:19.906 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:57:19.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:57:19.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:57:19.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:57:19.907 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:57:19.907 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:57:19.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:57:19.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:57:19.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:57:19.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:57:19.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:57:19.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:57:19.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:57:19.909 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (119m) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:57:19.909 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:57:19.909 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:57:20.156 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:57:20.157 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:57:20.157 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:57:20.157 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:20.157 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:57:20.158 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:57:20.158 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:20.158 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:57:20.158 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:57:20.159 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:20.159 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:57:20.159 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:57:20.159 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:57:20.160 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:57:20.160 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:57:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:20 gibba032 bash[24004]: audit 2021-03-10T15:57:19.665582+0000 mgr.gibba032.whopeo (mgr.24625) 6578 : audit [DBG] from='client.48465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:20 gibba032 bash[24004]: audit 2021-03-10T15:57:19.890632+0000 mgr.gibba032.whopeo (mgr.24625) 6579 : audit [DBG] from='client.58453 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:20 gibba032 bash[24004]: cluster 2021-03-10T15:57:19.897413+0000 mgr.gibba032.whopeo (mgr.24625) 6580 : cluster [DBG] pgmap v3744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:20 gibba032 bash[24004]: audit 2021-03-10T15:57:20.156724+0000 mon.a (mon.0) 2419 : audit [DBG] from='client.? 172.21.2.111:0/4016009451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[30791]: audit 2021-03-10T15:57:19.665582+0000 mgr.gibba032.whopeo (mgr.24625) 6578 : audit [DBG] from='client.48465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[30791]: audit 2021-03-10T15:57:19.890632+0000 mgr.gibba032.whopeo (mgr.24625) 6579 : audit [DBG] from='client.58453 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[30791]: cluster 2021-03-10T15:57:19.897413+0000 mgr.gibba032.whopeo (mgr.24625) 6580 : cluster [DBG] pgmap v3744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:21.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[30791]: audit 2021-03-10T15:57:20.156724+0000 mon.a (mon.0) 2419 : audit [DBG] from='client.? 172.21.2.111:0/4016009451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[31774]: audit 2021-03-10T15:57:19.665582+0000 mgr.gibba032.whopeo (mgr.24625) 6578 : audit [DBG] from='client.48465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[31774]: audit 2021-03-10T15:57:19.890632+0000 mgr.gibba032.whopeo (mgr.24625) 6579 : audit [DBG] from='client.58453 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:21.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[31774]: cluster 2021-03-10T15:57:19.897413+0000 mgr.gibba032.whopeo (mgr.24625) 6580 : cluster [DBG] pgmap v3744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:21.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:20 gibba011 bash[31774]: audit 2021-03-10T15:57:20.156724+0000 mon.a (mon.0) 2419 : audit [DBG] from='client.? 172.21.2.111:0/4016009451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:22 gibba032 bash[24004]: cluster 2021-03-10T15:57:21.899090+0000 mgr.gibba032.whopeo (mgr.24625) 6581 : cluster [DBG] pgmap v3745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:22 gibba011 bash[30791]: cluster 2021-03-10T15:57:21.899090+0000 mgr.gibba032.whopeo (mgr.24625) 6581 : cluster [DBG] pgmap v3745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:22 gibba011 bash[31774]: cluster 2021-03-10T15:57:21.899090+0000 mgr.gibba032.whopeo (mgr.24625) 6581 : cluster [DBG] pgmap v3745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:24 gibba011 bash[30791]: cluster 2021-03-10T15:57:23.900772+0000 mgr.gibba032.whopeo (mgr.24625) 6582 : cluster [DBG] pgmap v3746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:24 gibba011 bash[31774]: cluster 2021-03-10T15:57:23.900772+0000 mgr.gibba032.whopeo (mgr.24625) 6582 : cluster [DBG] pgmap v3746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:24 gibba032 bash[24004]: cluster 2021-03-10T15:57:23.900772+0000 mgr.gibba032.whopeo (mgr.24625) 6582 : cluster [DBG] pgmap v3746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:27.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:26 gibba032 bash[24004]: cephadm 2021-03-10T15:57:25.372411+0000 mgr.gibba032.whopeo (mgr.24625) 6583 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:26 gibba032 bash[24004]: cluster 2021-03-10T15:57:25.902092+0000 mgr.gibba032.whopeo (mgr.24625) 6584 : cluster [DBG] pgmap v3747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:26 gibba011 bash[30791]: cephadm 2021-03-10T15:57:25.372411+0000 mgr.gibba032.whopeo (mgr.24625) 6583 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:26 gibba011 bash[30791]: cluster 2021-03-10T15:57:25.902092+0000 mgr.gibba032.whopeo (mgr.24625) 6584 : cluster [DBG] pgmap v3747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:26 gibba011 bash[31774]: cephadm 2021-03-10T15:57:25.372411+0000 mgr.gibba032.whopeo (mgr.24625) 6583 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:26 gibba011 bash[31774]: cluster 2021-03-10T15:57:25.902092+0000 mgr.gibba032.whopeo (mgr.24625) 6584 : cluster [DBG] pgmap v3747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:28 gibba032 bash[24004]: cluster 2021-03-10T15:57:27.903894+0000 mgr.gibba032.whopeo (mgr.24625) 6585 : cluster [DBG] pgmap v3748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:29.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:28 gibba011 bash[30791]: cluster 2021-03-10T15:57:27.903894+0000 mgr.gibba032.whopeo (mgr.24625) 6585 : cluster [DBG] pgmap v3748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:28 gibba011 bash[31774]: cluster 2021-03-10T15:57:27.903894+0000 mgr.gibba032.whopeo (mgr.24625) 6585 : cluster [DBG] pgmap v3748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:30 gibba032 bash[24004]: cluster 2021-03-10T15:57:29.905414+0000 mgr.gibba032.whopeo (mgr.24625) 6586 : cluster [DBG] pgmap v3749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:30 gibba011 bash[30791]: cluster 2021-03-10T15:57:29.905414+0000 mgr.gibba032.whopeo (mgr.24625) 6586 : cluster [DBG] pgmap v3749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:30 gibba011 bash[31774]: cluster 2021-03-10T15:57:29.905414+0000 mgr.gibba032.whopeo (mgr.24625) 6586 : cluster [DBG] pgmap v3749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:32 gibba032 bash[24004]: cluster 2021-03-10T15:57:31.907066+0000 mgr.gibba032.whopeo (mgr.24625) 6587 : cluster [DBG] pgmap v3750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:32 gibba011 bash[30791]: cluster 2021-03-10T15:57:31.907066+0000 mgr.gibba032.whopeo (mgr.24625) 6587 : cluster [DBG] pgmap v3750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:32 gibba011 bash[31774]: cluster 2021-03-10T15:57:31.907066+0000 mgr.gibba032.whopeo (mgr.24625) 6587 : cluster [DBG] pgmap v3750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:34 gibba011 bash[30791]: cluster 2021-03-10T15:57:33.908742+0000 mgr.gibba032.whopeo (mgr.24625) 6588 : cluster [DBG] pgmap v3751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:34 gibba011 bash[31774]: cluster 2021-03-10T15:57:33.908742+0000 mgr.gibba032.whopeo (mgr.24625) 6588 : cluster [DBG] pgmap v3751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:34 gibba032 bash[24004]: cluster 2021-03-10T15:57:33.908742+0000 mgr.gibba032.whopeo (mgr.24625) 6588 : cluster [DBG] pgmap v3751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:36 gibba032 bash[24004]: cluster 2021-03-10T15:57:35.910129+0000 mgr.gibba032.whopeo (mgr.24625) 6589 : cluster [DBG] pgmap v3752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:36 gibba011 bash[30791]: cluster 2021-03-10T15:57:35.910129+0000 mgr.gibba032.whopeo (mgr.24625) 6589 : cluster [DBG] pgmap v3752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:36 gibba011 bash[31774]: cluster 2021-03-10T15:57:35.910129+0000 mgr.gibba032.whopeo (mgr.24625) 6589 : cluster [DBG] pgmap v3752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:38 gibba032 bash[24004]: cluster 2021-03-10T15:57:37.911960+0000 mgr.gibba032.whopeo (mgr.24625) 6590 : cluster [DBG] pgmap v3753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:38 gibba011 bash[30791]: cluster 2021-03-10T15:57:37.911960+0000 mgr.gibba032.whopeo (mgr.24625) 6590 : cluster [DBG] pgmap v3753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:38 gibba011 bash[31774]: cluster 2021-03-10T15:57:37.911960+0000 mgr.gibba032.whopeo (mgr.24625) 6590 : cluster [DBG] pgmap v3753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: cluster 2021-03-10T15:57:39.913250+0000 mgr.gibba032.whopeo (mgr.24625) 6591 : cluster [DBG] pgmap v3754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.604276+0000 mon.a (mon.0) 2420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:57:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.607877+0000 mon.a (mon.0) 2421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:57:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.609594+0000 mon.a (mon.0) 2422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.611318+0000 mon.a (mon.0) 2423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.612774+0000 mon.a (mon.0) 2424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.615660+0000 mon.a (mon.0) 2425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:40 gibba032 bash[24004]: audit 2021-03-10T15:57:40.617255+0000 mon.a (mon.0) 2426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: cluster 2021-03-10T15:57:39.913250+0000 mgr.gibba032.whopeo (mgr.24625) 6591 : cluster [DBG] pgmap v3754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.604276+0000 mon.a (mon.0) 2420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:57:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.607877+0000 mon.a (mon.0) 2421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:57:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.609594+0000 mon.a (mon.0) 2422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.611318+0000 mon.a (mon.0) 2423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.612774+0000 mon.a (mon.0) 2424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.615660+0000 mon.a (mon.0) 2425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[30791]: audit 2021-03-10T15:57:40.617255+0000 mon.a (mon.0) 2426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: cluster 2021-03-10T15:57:39.913250+0000 mgr.gibba032.whopeo (mgr.24625) 6591 : cluster [DBG] pgmap v3754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.604276+0000 mon.a (mon.0) 2420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:57:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.607877+0000 mon.a (mon.0) 2421 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:57:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.609594+0000 mon.a (mon.0) 2422 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.611318+0000 mon.a (mon.0) 2423 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.612774+0000 mon.a (mon.0) 2424 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.615660+0000 mon.a (mon.0) 2425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:40 gibba011 bash[31774]: audit 2021-03-10T15:57:40.617255+0000 mon.a (mon.0) 2426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.389013+0000 mgr.gibba032.whopeo (mgr.24625) 6592 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:57:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.390600+0000 mgr.gibba032.whopeo (mgr.24625) 6593 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:57:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.605197+0000 mgr.gibba032.whopeo (mgr.24625) 6594 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:57:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.607263+0000 mgr.gibba032.whopeo (mgr.24625) 6595 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:57:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.608935+0000 mgr.gibba032.whopeo (mgr.24625) 6596 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:57:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.608971+0000 mgr.gibba032.whopeo (mgr.24625) 6597 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:57:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.610512+0000 mgr.gibba032.whopeo (mgr.24625) 6598 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:57:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.610548+0000 mgr.gibba032.whopeo (mgr.24625) 6599 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:57:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.611971+0000 mgr.gibba032.whopeo (mgr.24625) 6600 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:57:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.613229+0000 mgr.gibba032.whopeo (mgr.24625) 6601 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:57:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.613264+0000 mgr.gibba032.whopeo (mgr.24625) 6602 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:57:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.614727+0000 mgr.gibba032.whopeo (mgr.24625) 6603 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:57:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.615036+0000 mgr.gibba032.whopeo (mgr.24625) 6604 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:57:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.616341+0000 mgr.gibba032.whopeo (mgr.24625) 6605 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:57:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.616377+0000 mgr.gibba032.whopeo (mgr.24625) 6606 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:57:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:42 gibba032 bash[24004]: cephadm 2021-03-10T15:57:40.616479+0000 mgr.gibba032.whopeo (mgr.24625) 6607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.389013+0000 mgr.gibba032.whopeo (mgr.24625) 6592 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:57:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.390600+0000 mgr.gibba032.whopeo (mgr.24625) 6593 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:57:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.605197+0000 mgr.gibba032.whopeo (mgr.24625) 6594 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:57:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.607263+0000 mgr.gibba032.whopeo (mgr.24625) 6595 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:57:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.608935+0000 mgr.gibba032.whopeo (mgr.24625) 6596 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:57:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.608971+0000 mgr.gibba032.whopeo (mgr.24625) 6597 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:57:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.610512+0000 mgr.gibba032.whopeo (mgr.24625) 6598 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:57:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.610548+0000 mgr.gibba032.whopeo (mgr.24625) 6599 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:57:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.611971+0000 mgr.gibba032.whopeo (mgr.24625) 6600 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:57:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.613229+0000 mgr.gibba032.whopeo (mgr.24625) 6601 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:57:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.613264+0000 mgr.gibba032.whopeo (mgr.24625) 6602 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:57:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.614727+0000 mgr.gibba032.whopeo (mgr.24625) 6603 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:57:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.615036+0000 mgr.gibba032.whopeo (mgr.24625) 6604 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:57:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.616341+0000 mgr.gibba032.whopeo (mgr.24625) 6605 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:57:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.616377+0000 mgr.gibba032.whopeo (mgr.24625) 6606 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:57:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[30791]: cephadm 2021-03-10T15:57:40.616479+0000 mgr.gibba032.whopeo (mgr.24625) 6607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.389013+0000 mgr.gibba032.whopeo (mgr.24625) 6592 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:57:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.390600+0000 mgr.gibba032.whopeo (mgr.24625) 6593 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:57:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.605197+0000 mgr.gibba032.whopeo (mgr.24625) 6594 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:57:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.607263+0000 mgr.gibba032.whopeo (mgr.24625) 6595 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:57:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.608935+0000 mgr.gibba032.whopeo (mgr.24625) 6596 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:57:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.608971+0000 mgr.gibba032.whopeo (mgr.24625) 6597 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:57:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.610512+0000 mgr.gibba032.whopeo (mgr.24625) 6598 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:57:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.610548+0000 mgr.gibba032.whopeo (mgr.24625) 6599 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:57:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.611971+0000 mgr.gibba032.whopeo (mgr.24625) 6600 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:57:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.613229+0000 mgr.gibba032.whopeo (mgr.24625) 6601 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:57:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.613264+0000 mgr.gibba032.whopeo (mgr.24625) 6602 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:57:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.614727+0000 mgr.gibba032.whopeo (mgr.24625) 6603 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:57:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.615036+0000 mgr.gibba032.whopeo (mgr.24625) 6604 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:57:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.616341+0000 mgr.gibba032.whopeo (mgr.24625) 6605 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:57:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.616377+0000 mgr.gibba032.whopeo (mgr.24625) 6606 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:57:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:42 gibba011 bash[31774]: cephadm 2021-03-10T15:57:40.616479+0000 mgr.gibba032.whopeo (mgr.24625) 6607 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:43 gibba011 bash[30791]: cluster 2021-03-10T15:57:41.914957+0000 mgr.gibba032.whopeo (mgr.24625) 6608 : cluster [DBG] pgmap v3755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:43 gibba011 bash[31774]: cluster 2021-03-10T15:57:41.914957+0000 mgr.gibba032.whopeo (mgr.24625) 6608 : cluster [DBG] pgmap v3755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:43 gibba032 bash[24004]: cluster 2021-03-10T15:57:41.914957+0000 mgr.gibba032.whopeo (mgr.24625) 6608 : cluster [DBG] pgmap v3755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:44 gibba011 bash[30791]: cluster 2021-03-10T15:57:43.916576+0000 mgr.gibba032.whopeo (mgr.24625) 6609 : cluster [DBG] pgmap v3756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:44 gibba011 bash[31774]: cluster 2021-03-10T15:57:43.916576+0000 mgr.gibba032.whopeo (mgr.24625) 6609 : cluster [DBG] pgmap v3756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:44 gibba032 bash[24004]: cluster 2021-03-10T15:57:43.916576+0000 mgr.gibba032.whopeo (mgr.24625) 6609 : cluster [DBG] pgmap v3756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:46 gibba032 bash[24004]: cluster 2021-03-10T15:57:45.917956+0000 mgr.gibba032.whopeo (mgr.24625) 6610 : cluster [DBG] pgmap v3757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:46 gibba011 bash[30791]: cluster 2021-03-10T15:57:45.917956+0000 mgr.gibba032.whopeo (mgr.24625) 6610 : cluster [DBG] pgmap v3757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:46 gibba011 bash[31774]: cluster 2021-03-10T15:57:45.917956+0000 mgr.gibba032.whopeo (mgr.24625) 6610 : cluster [DBG] pgmap v3757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:48 gibba032 bash[24004]: cluster 2021-03-10T15:57:47.919693+0000 mgr.gibba032.whopeo (mgr.24625) 6611 : cluster [DBG] pgmap v3758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:48 gibba011 bash[30791]: cluster 2021-03-10T15:57:47.919693+0000 mgr.gibba032.whopeo (mgr.24625) 6611 : cluster [DBG] pgmap v3758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:48 gibba011 bash[31774]: cluster 2021-03-10T15:57:47.919693+0000 mgr.gibba032.whopeo (mgr.24625) 6611 : cluster [DBG] pgmap v3758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:50.400 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:57:50.625 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:57:50.626 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:57:50.626 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:57:50.626 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:57:50.627 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:57:50.627 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:57:50.627 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:57:50.627 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:57:50.627 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:57:50.628 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:57:50.628 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:57:50.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:57:50.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:57:50.629 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:57:50.629 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:57:50.629 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:57:50.629 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:57:50.630 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:57:50.630 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (119m) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:57:50.630 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:57:50.630 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:57:50.886 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:57:50.887 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:57:50.887 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:57:50.887 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:50.887 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:57:50.888 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:57:50.888 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:50.888 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:57:50.888 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:57:50.888 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:57:50.889 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:57:50.889 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:57:50.889 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:57:50.889 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:57:50.890 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:57:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:50 gibba032 bash[24004]: cluster 2021-03-10T15:57:49.921291+0000 mgr.gibba032.whopeo (mgr.24625) 6612 : cluster [DBG] pgmap v3759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:50 gibba032 bash[24004]: audit 2021-03-10T15:57:50.886698+0000 mon.c (mon.1) 124 : audit [DBG] from='client.? 172.21.2.111:0/2118227387' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:50 gibba011 bash[30791]: cluster 2021-03-10T15:57:49.921291+0000 mgr.gibba032.whopeo (mgr.24625) 6612 : cluster [DBG] pgmap v3759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:50 gibba011 bash[30791]: audit 2021-03-10T15:57:50.886698+0000 mon.c (mon.1) 124 : audit [DBG] from='client.? 172.21.2.111:0/2118227387' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:50 gibba011 bash[31774]: cluster 2021-03-10T15:57:49.921291+0000 mgr.gibba032.whopeo (mgr.24625) 6612 : cluster [DBG] pgmap v3759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:50 gibba011 bash[31774]: audit 2021-03-10T15:57:50.886698+0000 mon.c (mon.1) 124 : audit [DBG] from='client.? 172.21.2.111:0/2118227387' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:57:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:52 gibba011 bash[30791]: audit 2021-03-10T15:57:50.389678+0000 mgr.gibba032.whopeo (mgr.24625) 6613 : audit [DBG] from='client.58465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:52 gibba011 bash[30791]: audit 2021-03-10T15:57:50.611993+0000 mgr.gibba032.whopeo (mgr.24625) 6614 : audit [DBG] from='client.48489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:52 gibba011 bash[31774]: audit 2021-03-10T15:57:50.389678+0000 mgr.gibba032.whopeo (mgr.24625) 6613 : audit [DBG] from='client.58465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:52 gibba011 bash[31774]: audit 2021-03-10T15:57:50.611993+0000 mgr.gibba032.whopeo (mgr.24625) 6614 : audit [DBG] from='client.48489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:52 gibba032 bash[24004]: audit 2021-03-10T15:57:50.389678+0000 mgr.gibba032.whopeo (mgr.24625) 6613 : audit [DBG] from='client.58465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:52 gibba032 bash[24004]: audit 2021-03-10T15:57:50.611993+0000 mgr.gibba032.whopeo (mgr.24625) 6614 : audit [DBG] from='client.48489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:57:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:53 gibba011 bash[30791]: cluster 2021-03-10T15:57:51.923015+0000 mgr.gibba032.whopeo (mgr.24625) 6615 : cluster [DBG] pgmap v3760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:53 gibba011 bash[31774]: cluster 2021-03-10T15:57:51.923015+0000 mgr.gibba032.whopeo (mgr.24625) 6615 : cluster [DBG] pgmap v3760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:53 gibba032 bash[24004]: cluster 2021-03-10T15:57:51.923015+0000 mgr.gibba032.whopeo (mgr.24625) 6615 : cluster [DBG] pgmap v3760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:54 gibba011 bash[30791]: cluster 2021-03-10T15:57:53.924662+0000 mgr.gibba032.whopeo (mgr.24625) 6616 : cluster [DBG] pgmap v3761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:54 gibba011 bash[31774]: cluster 2021-03-10T15:57:53.924662+0000 mgr.gibba032.whopeo (mgr.24625) 6616 : cluster [DBG] pgmap v3761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:54 gibba032 bash[24004]: cluster 2021-03-10T15:57:53.924662+0000 mgr.gibba032.whopeo (mgr.24625) 6616 : cluster [DBG] pgmap v3761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: cephadm 2021-03-10T15:57:55.628326+0000 mgr.gibba032.whopeo (mgr.24625) 6617 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: cluster 2021-03-10T15:57:55.926000+0000 mgr.gibba032.whopeo (mgr.24625) 6618 : cluster [DBG] pgmap v3762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.782372+0000 mon.a (mon.0) 2427 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:57:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.783321+0000 mon.a (mon.0) 2428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:57:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.832203+0000 mon.a (mon.0) 2429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:57:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.833337+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:57:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.834392+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:57:57.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:56 gibba032 bash[24004]: audit 2021-03-10T15:57:56.835454+0000 mon.a (mon.0) 2432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:57:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: cephadm 2021-03-10T15:57:55.628326+0000 mgr.gibba032.whopeo (mgr.24625) 6617 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: cluster 2021-03-10T15:57:55.926000+0000 mgr.gibba032.whopeo (mgr.24625) 6618 : cluster [DBG] pgmap v3762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.782372+0000 mon.a (mon.0) 2427 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:57:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.783321+0000 mon.a (mon.0) 2428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:57:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.832203+0000 mon.a (mon.0) 2429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:57:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.833337+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:57:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.834392+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:57:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[30791]: audit 2021-03-10T15:57:56.835454+0000 mon.a (mon.0) 2432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:57:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: cephadm 2021-03-10T15:57:55.628326+0000 mgr.gibba032.whopeo (mgr.24625) 6617 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:57:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: cluster 2021-03-10T15:57:55.926000+0000 mgr.gibba032.whopeo (mgr.24625) 6618 : cluster [DBG] pgmap v3762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.782372+0000 mon.a (mon.0) 2427 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:57:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.783321+0000 mon.a (mon.0) 2428 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:57:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.832203+0000 mon.a (mon.0) 2429 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T15:57:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.833337+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T15:57:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.834392+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T15:57:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:56 gibba011 bash[31774]: audit 2021-03-10T15:57:56.835454+0000 mon.a (mon.0) 2432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T15:57:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:57:58 gibba032 bash[24004]: cluster 2021-03-10T15:57:57.927678+0000 mgr.gibba032.whopeo (mgr.24625) 6619 : cluster [DBG] pgmap v3763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:59.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:57:58 gibba011 bash[30791]: cluster 2021-03-10T15:57:57.927678+0000 mgr.gibba032.whopeo (mgr.24625) 6619 : cluster [DBG] pgmap v3763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:57:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:57:59 gibba011 bash[31774]: cluster 2021-03-10T15:57:57.927678+0000 mgr.gibba032.whopeo (mgr.24625) 6619 : cluster [DBG] pgmap v3763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:01 gibba011 bash[30791]: cluster 2021-03-10T15:57:59.929196+0000 mgr.gibba032.whopeo (mgr.24625) 6620 : cluster [DBG] pgmap v3764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:01 gibba011 bash[31774]: cluster 2021-03-10T15:57:59.929196+0000 mgr.gibba032.whopeo (mgr.24625) 6620 : cluster [DBG] pgmap v3764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:01 gibba032 bash[24004]: cluster 2021-03-10T15:57:59.929196+0000 mgr.gibba032.whopeo (mgr.24625) 6620 : cluster [DBG] pgmap v3764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:03.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:03 gibba011 bash[30791]: cluster 2021-03-10T15:58:01.930930+0000 mgr.gibba032.whopeo (mgr.24625) 6621 : cluster [DBG] pgmap v3765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:03 gibba011 bash[31774]: cluster 2021-03-10T15:58:01.930930+0000 mgr.gibba032.whopeo (mgr.24625) 6621 : cluster [DBG] pgmap v3765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:03 gibba032 bash[24004]: cluster 2021-03-10T15:58:01.930930+0000 mgr.gibba032.whopeo (mgr.24625) 6621 : cluster [DBG] pgmap v3765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:04 gibba011 bash[30791]: cluster 2021-03-10T15:58:03.932349+0000 mgr.gibba032.whopeo (mgr.24625) 6622 : cluster [DBG] pgmap v3766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:04 gibba011 bash[31774]: cluster 2021-03-10T15:58:03.932349+0000 mgr.gibba032.whopeo (mgr.24625) 6622 : cluster [DBG] pgmap v3766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:04 gibba032 bash[24004]: cluster 2021-03-10T15:58:03.932349+0000 mgr.gibba032.whopeo (mgr.24625) 6622 : cluster [DBG] pgmap v3766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:06 gibba032 bash[24004]: cluster 2021-03-10T15:58:05.933709+0000 mgr.gibba032.whopeo (mgr.24625) 6623 : cluster [DBG] pgmap v3767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:06 gibba011 bash[30791]: cluster 2021-03-10T15:58:05.933709+0000 mgr.gibba032.whopeo (mgr.24625) 6623 : cluster [DBG] pgmap v3767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:06 gibba011 bash[31774]: cluster 2021-03-10T15:58:05.933709+0000 mgr.gibba032.whopeo (mgr.24625) 6623 : cluster [DBG] pgmap v3767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:09 gibba011 bash[30791]: cluster 2021-03-10T15:58:07.935644+0000 mgr.gibba032.whopeo (mgr.24625) 6624 : cluster [DBG] pgmap v3768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:09 gibba011 bash[31774]: cluster 2021-03-10T15:58:07.935644+0000 mgr.gibba032.whopeo (mgr.24625) 6624 : cluster [DBG] pgmap v3768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:09.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:09 gibba032 bash[24004]: cluster 2021-03-10T15:58:07.935644+0000 mgr.gibba032.whopeo (mgr.24625) 6624 : cluster [DBG] pgmap v3768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:11 gibba011 bash[30791]: cluster 2021-03-10T15:58:09.937001+0000 mgr.gibba032.whopeo (mgr.24625) 6625 : cluster [DBG] pgmap v3769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:11 gibba011 bash[31774]: cluster 2021-03-10T15:58:09.937001+0000 mgr.gibba032.whopeo (mgr.24625) 6625 : cluster [DBG] pgmap v3769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:11.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:11 gibba032 bash[24004]: cluster 2021-03-10T15:58:09.937001+0000 mgr.gibba032.whopeo (mgr.24625) 6625 : cluster [DBG] pgmap v3769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:12 gibba011 bash[30791]: cephadm 2021-03-10T15:58:10.640950+0000 mgr.gibba032.whopeo (mgr.24625) 6626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:12 gibba011 bash[31774]: cephadm 2021-03-10T15:58:10.640950+0000 mgr.gibba032.whopeo (mgr.24625) 6626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:12 gibba032 bash[24004]: cephadm 2021-03-10T15:58:10.640950+0000 mgr.gibba032.whopeo (mgr.24625) 6626 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:13 gibba011 bash[30791]: cluster 2021-03-10T15:58:11.938725+0000 mgr.gibba032.whopeo (mgr.24625) 6627 : cluster [DBG] pgmap v3770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:13 gibba011 bash[31774]: cluster 2021-03-10T15:58:11.938725+0000 mgr.gibba032.whopeo (mgr.24625) 6627 : cluster [DBG] pgmap v3770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:13 gibba032 bash[24004]: cluster 2021-03-10T15:58:11.938725+0000 mgr.gibba032.whopeo (mgr.24625) 6627 : cluster [DBG] pgmap v3770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:14 gibba011 bash[30791]: cluster 2021-03-10T15:58:13.940349+0000 mgr.gibba032.whopeo (mgr.24625) 6628 : cluster [DBG] pgmap v3771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:14 gibba011 bash[31774]: cluster 2021-03-10T15:58:13.940349+0000 mgr.gibba032.whopeo (mgr.24625) 6628 : cluster [DBG] pgmap v3771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:14 gibba032 bash[24004]: cluster 2021-03-10T15:58:13.940349+0000 mgr.gibba032.whopeo (mgr.24625) 6628 : cluster [DBG] pgmap v3771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:17 gibba032 bash[24004]: cluster 2021-03-10T15:58:15.941687+0000 mgr.gibba032.whopeo (mgr.24625) 6629 : cluster [DBG] pgmap v3772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:17 gibba011 bash[30791]: cluster 2021-03-10T15:58:15.941687+0000 mgr.gibba032.whopeo (mgr.24625) 6629 : cluster [DBG] pgmap v3772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:17.407 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:17 gibba011 bash[31774]: cluster 2021-03-10T15:58:15.941687+0000 mgr.gibba032.whopeo (mgr.24625) 6629 : cluster [DBG] pgmap v3772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:19 gibba011 bash[30791]: cluster 2021-03-10T15:58:17.943559+0000 mgr.gibba032.whopeo (mgr.24625) 6630 : cluster [DBG] pgmap v3773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:19 gibba011 bash[31774]: cluster 2021-03-10T15:58:17.943559+0000 mgr.gibba032.whopeo (mgr.24625) 6630 : cluster [DBG] pgmap v3773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:19 gibba032 bash[24004]: cluster 2021-03-10T15:58:17.943559+0000 mgr.gibba032.whopeo (mgr.24625) 6630 : cluster [DBG] pgmap v3773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:21.132 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:58:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:21 gibba011 bash[30791]: cluster 2021-03-10T15:58:19.944922+0000 mgr.gibba032.whopeo (mgr.24625) 6631 : cluster [DBG] pgmap v3774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:21 gibba011 bash[31774]: cluster 2021-03-10T15:58:19.944922+0000 mgr.gibba032.whopeo (mgr.24625) 6631 : cluster [DBG] pgmap v3774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:21.357 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:58:21.357 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:58:21.358 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:58:21.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:58:21.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:58:21.358 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:58:21.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:58:21.359 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:58:21.360 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:58:21.360 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:58:21.360 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:58:21.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:58:21.360 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:58:21.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:58:21.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:58:21.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:58:21.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:58:21.361 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:58:21.362 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:58:21.362 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:58:21.362 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:58:21.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:21 gibba032 bash[24004]: cluster 2021-03-10T15:58:19.944922+0000 mgr.gibba032.whopeo (mgr.24625) 6631 : cluster [DBG] pgmap v3774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:21.607 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:58:21.607 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:58:21.607 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:58:21.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:21.608 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:58:21.608 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:58:21.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:21.608 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:58:21.609 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:58:21.609 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:21.609 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:58:21.609 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:58:21.610 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:58:21.610 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:58:21.610 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:58:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:22 gibba011 bash[30791]: audit 2021-03-10T15:58:21.607418+0000 mon.c (mon.1) 125 : audit [DBG] from='client.? 172.21.2.111:0/1157562901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:22 gibba011 bash[31774]: audit 2021-03-10T15:58:21.607418+0000 mon.c (mon.1) 125 : audit [DBG] from='client.? 172.21.2.111:0/1157562901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:22 gibba032 bash[24004]: audit 2021-03-10T15:58:21.607418+0000 mon.c (mon.1) 125 : audit [DBG] from='client.? 172.21.2.111:0/1157562901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[30791]: audit 2021-03-10T15:58:21.121656+0000 mgr.gibba032.whopeo (mgr.24625) 6632 : audit [DBG] from='client.58483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[30791]: audit 2021-03-10T15:58:21.344339+0000 mgr.gibba032.whopeo (mgr.24625) 6633 : audit [DBG] from='client.58489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[30791]: cluster 2021-03-10T15:58:21.946394+0000 mgr.gibba032.whopeo (mgr.24625) 6634 : cluster [DBG] pgmap v3775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:23.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[31774]: audit 2021-03-10T15:58:21.121656+0000 mgr.gibba032.whopeo (mgr.24625) 6632 : audit [DBG] from='client.58483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[31774]: audit 2021-03-10T15:58:21.344339+0000 mgr.gibba032.whopeo (mgr.24625) 6633 : audit [DBG] from='client.58489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:23 gibba011 bash[31774]: cluster 2021-03-10T15:58:21.946394+0000 mgr.gibba032.whopeo (mgr.24625) 6634 : cluster [DBG] pgmap v3775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:23 gibba032 bash[24004]: audit 2021-03-10T15:58:21.121656+0000 mgr.gibba032.whopeo (mgr.24625) 6632 : audit [DBG] from='client.58483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:23 gibba032 bash[24004]: audit 2021-03-10T15:58:21.344339+0000 mgr.gibba032.whopeo (mgr.24625) 6633 : audit [DBG] from='client.58489 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:23 gibba032 bash[24004]: cluster 2021-03-10T15:58:21.946394+0000 mgr.gibba032.whopeo (mgr.24625) 6634 : cluster [DBG] pgmap v3775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:24 gibba011 bash[30791]: cluster 2021-03-10T15:58:23.947877+0000 mgr.gibba032.whopeo (mgr.24625) 6635 : cluster [DBG] pgmap v3776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:24 gibba011 bash[31774]: cluster 2021-03-10T15:58:23.947877+0000 mgr.gibba032.whopeo (mgr.24625) 6635 : cluster [DBG] pgmap v3776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:24 gibba032 bash[24004]: cluster 2021-03-10T15:58:23.947877+0000 mgr.gibba032.whopeo (mgr.24625) 6635 : cluster [DBG] pgmap v3776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:27 gibba011 bash[30791]: cephadm 2021-03-10T15:58:25.652965+0000 mgr.gibba032.whopeo (mgr.24625) 6636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:27 gibba011 bash[30791]: cluster 2021-03-10T15:58:25.949191+0000 mgr.gibba032.whopeo (mgr.24625) 6637 : cluster [DBG] pgmap v3777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:27 gibba011 bash[31774]: cephadm 2021-03-10T15:58:25.652965+0000 mgr.gibba032.whopeo (mgr.24625) 6636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:27 gibba011 bash[31774]: cluster 2021-03-10T15:58:25.949191+0000 mgr.gibba032.whopeo (mgr.24625) 6637 : cluster [DBG] pgmap v3777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:27 gibba032 bash[24004]: cephadm 2021-03-10T15:58:25.652965+0000 mgr.gibba032.whopeo (mgr.24625) 6636 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:27 gibba032 bash[24004]: cluster 2021-03-10T15:58:25.949191+0000 mgr.gibba032.whopeo (mgr.24625) 6637 : cluster [DBG] pgmap v3777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:29 gibba011 bash[30791]: cluster 2021-03-10T15:58:27.951137+0000 mgr.gibba032.whopeo (mgr.24625) 6638 : cluster [DBG] pgmap v3778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:29 gibba011 bash[31774]: cluster 2021-03-10T15:58:27.951137+0000 mgr.gibba032.whopeo (mgr.24625) 6638 : cluster [DBG] pgmap v3778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:29 gibba032 bash[24004]: cluster 2021-03-10T15:58:27.951137+0000 mgr.gibba032.whopeo (mgr.24625) 6638 : cluster [DBG] pgmap v3778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:31 gibba011 bash[30791]: cluster 2021-03-10T15:58:29.952395+0000 mgr.gibba032.whopeo (mgr.24625) 6639 : cluster [DBG] pgmap v3779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:31 gibba011 bash[31774]: cluster 2021-03-10T15:58:29.952395+0000 mgr.gibba032.whopeo (mgr.24625) 6639 : cluster [DBG] pgmap v3779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:31 gibba032 bash[24004]: cluster 2021-03-10T15:58:29.952395+0000 mgr.gibba032.whopeo (mgr.24625) 6639 : cluster [DBG] pgmap v3779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:33 gibba011 bash[30791]: cluster 2021-03-10T15:58:31.953923+0000 mgr.gibba032.whopeo (mgr.24625) 6640 : cluster [DBG] pgmap v3780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:33 gibba011 bash[31774]: cluster 2021-03-10T15:58:31.953923+0000 mgr.gibba032.whopeo (mgr.24625) 6640 : cluster [DBG] pgmap v3780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:33 gibba032 bash[24004]: cluster 2021-03-10T15:58:31.953923+0000 mgr.gibba032.whopeo (mgr.24625) 6640 : cluster [DBG] pgmap v3780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:34 gibba032 bash[24004]: cluster 2021-03-10T15:58:33.955561+0000 mgr.gibba032.whopeo (mgr.24625) 6641 : cluster [DBG] pgmap v3781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:34 gibba011 bash[30791]: cluster 2021-03-10T15:58:33.955561+0000 mgr.gibba032.whopeo (mgr.24625) 6641 : cluster [DBG] pgmap v3781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:34 gibba011 bash[31774]: cluster 2021-03-10T15:58:33.955561+0000 mgr.gibba032.whopeo (mgr.24625) 6641 : cluster [DBG] pgmap v3781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:37 gibba011 bash[30791]: cluster 2021-03-10T15:58:35.956944+0000 mgr.gibba032.whopeo (mgr.24625) 6642 : cluster [DBG] pgmap v3782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:37 gibba011 bash[31774]: cluster 2021-03-10T15:58:35.956944+0000 mgr.gibba032.whopeo (mgr.24625) 6642 : cluster [DBG] pgmap v3782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:37 gibba032 bash[24004]: cluster 2021-03-10T15:58:35.956944+0000 mgr.gibba032.whopeo (mgr.24625) 6642 : cluster [DBG] pgmap v3782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:39 gibba011 bash[30791]: cluster 2021-03-10T15:58:37.958594+0000 mgr.gibba032.whopeo (mgr.24625) 6643 : cluster [DBG] pgmap v3783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:39 gibba011 bash[31774]: cluster 2021-03-10T15:58:37.958594+0000 mgr.gibba032.whopeo (mgr.24625) 6643 : cluster [DBG] pgmap v3783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:39.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:39 gibba032 bash[24004]: cluster 2021-03-10T15:58:37.958594+0000 mgr.gibba032.whopeo (mgr.24625) 6643 : cluster [DBG] pgmap v3783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: cluster 2021-03-10T15:58:39.959967+0000 mgr.gibba032.whopeo (mgr.24625) 6644 : cluster [DBG] pgmap v3784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.856629+0000 mon.a (mon.0) 2433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:58:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.860178+0000 mon.a (mon.0) 2434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:58:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.861388+0000 mon.a (mon.0) 2435 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.862553+0000 mon.a (mon.0) 2436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.863533+0000 mon.a (mon.0) 2437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.865574+0000 mon.a (mon.0) 2438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:41 gibba032 bash[24004]: audit 2021-03-10T15:58:40.866891+0000 mon.a (mon.0) 2439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: cluster 2021-03-10T15:58:39.959967+0000 mgr.gibba032.whopeo (mgr.24625) 6644 : cluster [DBG] pgmap v3784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.856629+0000 mon.a (mon.0) 2433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:58:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.860178+0000 mon.a (mon.0) 2434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:58:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.861388+0000 mon.a (mon.0) 2435 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.862553+0000 mon.a (mon.0) 2436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.863533+0000 mon.a (mon.0) 2437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.865574+0000 mon.a (mon.0) 2438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[30791]: audit 2021-03-10T15:58:40.866891+0000 mon.a (mon.0) 2439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: cluster 2021-03-10T15:58:39.959967+0000 mgr.gibba032.whopeo (mgr.24625) 6644 : cluster [DBG] pgmap v3784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.856629+0000 mon.a (mon.0) 2433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:58:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.860178+0000 mon.a (mon.0) 2434 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:58:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.861388+0000 mon.a (mon.0) 2435 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.862553+0000 mon.a (mon.0) 2436 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.863533+0000 mon.a (mon.0) 2437 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.865574+0000 mon.a (mon.0) 2438 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:41.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:41 gibba011 bash[31774]: audit 2021-03-10T15:58:40.866891+0000 mon.a (mon.0) 2439 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.664447+0000 mgr.gibba032.whopeo (mgr.24625) 6645 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:58:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.664769+0000 mgr.gibba032.whopeo (mgr.24625) 6646 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:58:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.857596+0000 mgr.gibba032.whopeo (mgr.24625) 6647 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:58:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.859192+0000 mgr.gibba032.whopeo (mgr.24625) 6648 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:58:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.860365+0000 mgr.gibba032.whopeo (mgr.24625) 6649 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:58:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.860401+0000 mgr.gibba032.whopeo (mgr.24625) 6650 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:58:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.861400+0000 mgr.gibba032.whopeo (mgr.24625) 6651 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:58:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.861438+0000 mgr.gibba032.whopeo (mgr.24625) 6652 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:58:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.862361+0000 mgr.gibba032.whopeo (mgr.24625) 6653 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:58:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.863314+0000 mgr.gibba032.whopeo (mgr.24625) 6654 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:58:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.863350+0000 mgr.gibba032.whopeo (mgr.24625) 6655 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:58:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.864442+0000 mgr.gibba032.whopeo (mgr.24625) 6656 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:58:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.864850+0000 mgr.gibba032.whopeo (mgr.24625) 6657 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:58:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.865695+0000 mgr.gibba032.whopeo (mgr.24625) 6658 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:58:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.865774+0000 mgr.gibba032.whopeo (mgr.24625) 6659 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:58:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[30791]: cephadm 2021-03-10T15:58:40.865931+0000 mgr.gibba032.whopeo (mgr.24625) 6660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.664447+0000 mgr.gibba032.whopeo (mgr.24625) 6645 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:58:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.664769+0000 mgr.gibba032.whopeo (mgr.24625) 6646 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:58:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.857596+0000 mgr.gibba032.whopeo (mgr.24625) 6647 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:58:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.859192+0000 mgr.gibba032.whopeo (mgr.24625) 6648 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:58:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.860365+0000 mgr.gibba032.whopeo (mgr.24625) 6649 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:58:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.860401+0000 mgr.gibba032.whopeo (mgr.24625) 6650 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:58:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.861400+0000 mgr.gibba032.whopeo (mgr.24625) 6651 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:58:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.861438+0000 mgr.gibba032.whopeo (mgr.24625) 6652 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:58:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.862361+0000 mgr.gibba032.whopeo (mgr.24625) 6653 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:58:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.863314+0000 mgr.gibba032.whopeo (mgr.24625) 6654 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:58:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.863350+0000 mgr.gibba032.whopeo (mgr.24625) 6655 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:58:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.864442+0000 mgr.gibba032.whopeo (mgr.24625) 6656 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:58:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.864850+0000 mgr.gibba032.whopeo (mgr.24625) 6657 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:58:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.865695+0000 mgr.gibba032.whopeo (mgr.24625) 6658 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:58:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.865774+0000 mgr.gibba032.whopeo (mgr.24625) 6659 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:58:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:42 gibba011 bash[31774]: cephadm 2021-03-10T15:58:40.865931+0000 mgr.gibba032.whopeo (mgr.24625) 6660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.664447+0000 mgr.gibba032.whopeo (mgr.24625) 6645 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:58:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.664769+0000 mgr.gibba032.whopeo (mgr.24625) 6646 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:58:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.857596+0000 mgr.gibba032.whopeo (mgr.24625) 6647 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:58:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.859192+0000 mgr.gibba032.whopeo (mgr.24625) 6648 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:58:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.860365+0000 mgr.gibba032.whopeo (mgr.24625) 6649 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:58:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.860401+0000 mgr.gibba032.whopeo (mgr.24625) 6650 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:58:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.861400+0000 mgr.gibba032.whopeo (mgr.24625) 6651 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:58:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.861438+0000 mgr.gibba032.whopeo (mgr.24625) 6652 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:58:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.862361+0000 mgr.gibba032.whopeo (mgr.24625) 6653 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:58:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.863314+0000 mgr.gibba032.whopeo (mgr.24625) 6654 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:58:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.863350+0000 mgr.gibba032.whopeo (mgr.24625) 6655 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:58:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.864442+0000 mgr.gibba032.whopeo (mgr.24625) 6656 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:58:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.864850+0000 mgr.gibba032.whopeo (mgr.24625) 6657 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:58:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.865695+0000 mgr.gibba032.whopeo (mgr.24625) 6658 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:58:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.865774+0000 mgr.gibba032.whopeo (mgr.24625) 6659 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:58:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:42 gibba032 bash[24004]: cephadm 2021-03-10T15:58:40.865931+0000 mgr.gibba032.whopeo (mgr.24625) 6660 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:43 gibba011 bash[30791]: cluster 2021-03-10T15:58:41.961724+0000 mgr.gibba032.whopeo (mgr.24625) 6661 : cluster [DBG] pgmap v3785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:43 gibba011 bash[31774]: cluster 2021-03-10T15:58:41.961724+0000 mgr.gibba032.whopeo (mgr.24625) 6661 : cluster [DBG] pgmap v3785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:43 gibba032 bash[24004]: cluster 2021-03-10T15:58:41.961724+0000 mgr.gibba032.whopeo (mgr.24625) 6661 : cluster [DBG] pgmap v3785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:44 gibba032 bash[24004]: cluster 2021-03-10T15:58:43.963319+0000 mgr.gibba032.whopeo (mgr.24625) 6662 : cluster [DBG] pgmap v3786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:44 gibba011 bash[30791]: cluster 2021-03-10T15:58:43.963319+0000 mgr.gibba032.whopeo (mgr.24625) 6662 : cluster [DBG] pgmap v3786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:44 gibba011 bash[31774]: cluster 2021-03-10T15:58:43.963319+0000 mgr.gibba032.whopeo (mgr.24625) 6662 : cluster [DBG] pgmap v3786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:47 gibba011 bash[30791]: cluster 2021-03-10T15:58:45.964730+0000 mgr.gibba032.whopeo (mgr.24625) 6663 : cluster [DBG] pgmap v3787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:47 gibba011 bash[31774]: cluster 2021-03-10T15:58:45.964730+0000 mgr.gibba032.whopeo (mgr.24625) 6663 : cluster [DBG] pgmap v3787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:47 gibba032 bash[24004]: cluster 2021-03-10T15:58:45.964730+0000 mgr.gibba032.whopeo (mgr.24625) 6663 : cluster [DBG] pgmap v3787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:49 gibba011 bash[30791]: cluster 2021-03-10T15:58:47.966384+0000 mgr.gibba032.whopeo (mgr.24625) 6664 : cluster [DBG] pgmap v3788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:49 gibba011 bash[31774]: cluster 2021-03-10T15:58:47.966384+0000 mgr.gibba032.whopeo (mgr.24625) 6664 : cluster [DBG] pgmap v3788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:49 gibba032 bash[24004]: cluster 2021-03-10T15:58:47.966384+0000 mgr.gibba032.whopeo (mgr.24625) 6664 : cluster [DBG] pgmap v3788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:51.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:51 gibba011 bash[30791]: cluster 2021-03-10T15:58:49.967694+0000 mgr.gibba032.whopeo (mgr.24625) 6665 : cluster [DBG] pgmap v3789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:51 gibba011 bash[31774]: cluster 2021-03-10T15:58:49.967694+0000 mgr.gibba032.whopeo (mgr.24625) 6665 : cluster [DBG] pgmap v3789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:51 gibba032 bash[24004]: cluster 2021-03-10T15:58:49.967694+0000 mgr.gibba032.whopeo (mgr.24625) 6665 : cluster [DBG] pgmap v3789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:51.851 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:58:52.079 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:58:52.079 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:58:52.079 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:58:52.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:58:52.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:58:52.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:58:52.080 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:58:52.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:58:52.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:58:52.081 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:58:52.081 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:58:52.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:58:52.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:58:52.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:58:52.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:58:52.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:58:52.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:58:52.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:58:52.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:58:52.084 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:58:52.084 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:58:52.343 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:58:52.343 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:58:52.343 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:58:52.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:52.344 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:58:52.345 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:58:52.345 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:52.346 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:58:52.346 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:58:52.347 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:58:52.347 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:58:52.347 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:58:52.348 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:58:52.348 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:58:52.349 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:58:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[30791]: audit 2021-03-10T15:58:51.840800+0000 mgr.gibba032.whopeo (mgr.24625) 6666 : audit [DBG] from='client.58501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[30791]: cluster 2021-03-10T15:58:51.969561+0000 mgr.gibba032.whopeo (mgr.24625) 6667 : cluster [DBG] pgmap v3790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[30791]: audit 2021-03-10T15:58:52.064997+0000 mgr.gibba032.whopeo (mgr.24625) 6668 : audit [DBG] from='client.58507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[30791]: audit 2021-03-10T15:58:52.343636+0000 mon.c (mon.1) 126 : audit [DBG] from='client.? 172.21.2.111:0/2996466660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[31774]: audit 2021-03-10T15:58:51.840800+0000 mgr.gibba032.whopeo (mgr.24625) 6666 : audit [DBG] from='client.58501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[31774]: cluster 2021-03-10T15:58:51.969561+0000 mgr.gibba032.whopeo (mgr.24625) 6667 : cluster [DBG] pgmap v3790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[31774]: audit 2021-03-10T15:58:52.064997+0000 mgr.gibba032.whopeo (mgr.24625) 6668 : audit [DBG] from='client.58507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:53 gibba011 bash[31774]: audit 2021-03-10T15:58:52.343636+0000 mon.c (mon.1) 126 : audit [DBG] from='client.? 172.21.2.111:0/2996466660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:53 gibba032 bash[24004]: audit 2021-03-10T15:58:51.840800+0000 mgr.gibba032.whopeo (mgr.24625) 6666 : audit [DBG] from='client.58501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:53 gibba032 bash[24004]: cluster 2021-03-10T15:58:51.969561+0000 mgr.gibba032.whopeo (mgr.24625) 6667 : cluster [DBG] pgmap v3790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:53 gibba032 bash[24004]: audit 2021-03-10T15:58:52.064997+0000 mgr.gibba032.whopeo (mgr.24625) 6668 : audit [DBG] from='client.58507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:58:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:53 gibba032 bash[24004]: audit 2021-03-10T15:58:52.343636+0000 mon.c (mon.1) 126 : audit [DBG] from='client.? 172.21.2.111:0/2996466660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:58:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:54 gibba032 bash[24004]: cluster 2021-03-10T15:58:53.971316+0000 mgr.gibba032.whopeo (mgr.24625) 6669 : cluster [DBG] pgmap v3791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:54 gibba011 bash[30791]: cluster 2021-03-10T15:58:53.971316+0000 mgr.gibba032.whopeo (mgr.24625) 6669 : cluster [DBG] pgmap v3791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:54 gibba011 bash[31774]: cluster 2021-03-10T15:58:53.971316+0000 mgr.gibba032.whopeo (mgr.24625) 6669 : cluster [DBG] pgmap v3791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: cephadm 2021-03-10T15:58:55.881748+0000 mgr.gibba032.whopeo (mgr.24625) 6670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: cluster 2021-03-10T15:58:55.973016+0000 mgr.gibba032.whopeo (mgr.24625) 6671 : cluster [DBG] pgmap v3792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.787190+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:58:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.787927+0000 mon.a (mon.0) 2441 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:58:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.843837+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:58:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.844655+0000 mon.a (mon.0) 2443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:58:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.845333+0000 mon.a (mon.0) 2444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:58:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[30791]: audit 2021-03-10T15:58:56.846002+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:58:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: cephadm 2021-03-10T15:58:55.881748+0000 mgr.gibba032.whopeo (mgr.24625) 6670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: cluster 2021-03-10T15:58:55.973016+0000 mgr.gibba032.whopeo (mgr.24625) 6671 : cluster [DBG] pgmap v3792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.787190+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:58:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.787927+0000 mon.a (mon.0) 2441 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:58:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.843837+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:58:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.844655+0000 mon.a (mon.0) 2443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:58:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.845333+0000 mon.a (mon.0) 2444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:58:57.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:57 gibba011 bash[31774]: audit 2021-03-10T15:58:56.846002+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:58:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: cephadm 2021-03-10T15:58:55.881748+0000 mgr.gibba032.whopeo (mgr.24625) 6670 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:58:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: cluster 2021-03-10T15:58:55.973016+0000 mgr.gibba032.whopeo (mgr.24625) 6671 : cluster [DBG] pgmap v3792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.787190+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:58:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.787927+0000 mon.a (mon.0) 2441 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:58:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.843837+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:58:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.844655+0000 mon.a (mon.0) 2443 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T15:58:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.845333+0000 mon.a (mon.0) 2444 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T15:58:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:57 gibba032 bash[24004]: audit 2021-03-10T15:58:56.846002+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T15:58:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:58:59 gibba011 bash[30791]: cluster 2021-03-10T15:58:57.975088+0000 mgr.gibba032.whopeo (mgr.24625) 6672 : cluster [DBG] pgmap v3793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:59.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:58:59 gibba011 bash[31774]: cluster 2021-03-10T15:58:57.975088+0000 mgr.gibba032.whopeo (mgr.24625) 6672 : cluster [DBG] pgmap v3793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:58:59.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:58:59 gibba032 bash[24004]: cluster 2021-03-10T15:58:57.975088+0000 mgr.gibba032.whopeo (mgr.24625) 6672 : cluster [DBG] pgmap v3793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:01 gibba011 bash[30791]: cluster 2021-03-10T15:58:59.976640+0000 mgr.gibba032.whopeo (mgr.24625) 6673 : cluster [DBG] pgmap v3794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:01 gibba011 bash[31774]: cluster 2021-03-10T15:58:59.976640+0000 mgr.gibba032.whopeo (mgr.24625) 6673 : cluster [DBG] pgmap v3794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:01 gibba032 bash[24004]: cluster 2021-03-10T15:58:59.976640+0000 mgr.gibba032.whopeo (mgr.24625) 6673 : cluster [DBG] pgmap v3794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:03.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:03 gibba011 bash[30791]: cluster 2021-03-10T15:59:01.978534+0000 mgr.gibba032.whopeo (mgr.24625) 6674 : cluster [DBG] pgmap v3795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:03.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:03 gibba011 bash[31774]: cluster 2021-03-10T15:59:01.978534+0000 mgr.gibba032.whopeo (mgr.24625) 6674 : cluster [DBG] pgmap v3795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:03 gibba032 bash[24004]: cluster 2021-03-10T15:59:01.978534+0000 mgr.gibba032.whopeo (mgr.24625) 6674 : cluster [DBG] pgmap v3795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:04 gibba032 bash[24004]: cluster 2021-03-10T15:59:03.980194+0000 mgr.gibba032.whopeo (mgr.24625) 6675 : cluster [DBG] pgmap v3796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:04 gibba011 bash[30791]: cluster 2021-03-10T15:59:03.980194+0000 mgr.gibba032.whopeo (mgr.24625) 6675 : cluster [DBG] pgmap v3796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:04 gibba011 bash[31774]: cluster 2021-03-10T15:59:03.980194+0000 mgr.gibba032.whopeo (mgr.24625) 6675 : cluster [DBG] pgmap v3796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:07 gibba011 bash[30791]: cluster 2021-03-10T15:59:05.981666+0000 mgr.gibba032.whopeo (mgr.24625) 6676 : cluster [DBG] pgmap v3797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:07 gibba011 bash[31774]: cluster 2021-03-10T15:59:05.981666+0000 mgr.gibba032.whopeo (mgr.24625) 6676 : cluster [DBG] pgmap v3797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:07 gibba032 bash[24004]: cluster 2021-03-10T15:59:05.981666+0000 mgr.gibba032.whopeo (mgr.24625) 6676 : cluster [DBG] pgmap v3797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:09 gibba011 bash[30791]: cluster 2021-03-10T15:59:07.983482+0000 mgr.gibba032.whopeo (mgr.24625) 6677 : cluster [DBG] pgmap v3798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:09 gibba011 bash[31774]: cluster 2021-03-10T15:59:07.983482+0000 mgr.gibba032.whopeo (mgr.24625) 6677 : cluster [DBG] pgmap v3798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:09.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:09 gibba032 bash[24004]: cluster 2021-03-10T15:59:07.983482+0000 mgr.gibba032.whopeo (mgr.24625) 6677 : cluster [DBG] pgmap v3798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:11.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:11 gibba032 bash[24004]: cluster 2021-03-10T15:59:09.984850+0000 mgr.gibba032.whopeo (mgr.24625) 6678 : cluster [DBG] pgmap v3799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:11 gibba011 bash[30791]: cluster 2021-03-10T15:59:09.984850+0000 mgr.gibba032.whopeo (mgr.24625) 6678 : cluster [DBG] pgmap v3799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:11 gibba011 bash[31774]: cluster 2021-03-10T15:59:09.984850+0000 mgr.gibba032.whopeo (mgr.24625) 6678 : cluster [DBG] pgmap v3799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:12 gibba011 bash[30791]: cephadm 2021-03-10T15:59:10.896793+0000 mgr.gibba032.whopeo (mgr.24625) 6679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:12 gibba011 bash[31774]: cephadm 2021-03-10T15:59:10.896793+0000 mgr.gibba032.whopeo (mgr.24625) 6679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:12 gibba032 bash[24004]: cephadm 2021-03-10T15:59:10.896793+0000 mgr.gibba032.whopeo (mgr.24625) 6679 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:13.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:13 gibba011 bash[30791]: cluster 2021-03-10T15:59:11.986614+0000 mgr.gibba032.whopeo (mgr.24625) 6680 : cluster [DBG] pgmap v3800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:13 gibba011 bash[31774]: cluster 2021-03-10T15:59:11.986614+0000 mgr.gibba032.whopeo (mgr.24625) 6680 : cluster [DBG] pgmap v3800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:13 gibba032 bash[24004]: cluster 2021-03-10T15:59:11.986614+0000 mgr.gibba032.whopeo (mgr.24625) 6680 : cluster [DBG] pgmap v3800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:14 gibba032 bash[24004]: cluster 2021-03-10T15:59:13.988135+0000 mgr.gibba032.whopeo (mgr.24625) 6681 : cluster [DBG] pgmap v3801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:14 gibba011 bash[30791]: cluster 2021-03-10T15:59:13.988135+0000 mgr.gibba032.whopeo (mgr.24625) 6681 : cluster [DBG] pgmap v3801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:14 gibba011 bash[31774]: cluster 2021-03-10T15:59:13.988135+0000 mgr.gibba032.whopeo (mgr.24625) 6681 : cluster [DBG] pgmap v3801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:17.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:17 gibba011 bash[30791]: cluster 2021-03-10T15:59:15.989515+0000 mgr.gibba032.whopeo (mgr.24625) 6682 : cluster [DBG] pgmap v3802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:17 gibba011 bash[31774]: cluster 2021-03-10T15:59:15.989515+0000 mgr.gibba032.whopeo (mgr.24625) 6682 : cluster [DBG] pgmap v3802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:17 gibba032 bash[24004]: cluster 2021-03-10T15:59:15.989515+0000 mgr.gibba032.whopeo (mgr.24625) 6682 : cluster [DBG] pgmap v3802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:19.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:19 gibba011 bash[30791]: cluster 2021-03-10T15:59:17.991279+0000 mgr.gibba032.whopeo (mgr.24625) 6683 : cluster [DBG] pgmap v3803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:19.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:19 gibba011 bash[31774]: cluster 2021-03-10T15:59:17.991279+0000 mgr.gibba032.whopeo (mgr.24625) 6683 : cluster [DBG] pgmap v3803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:19 gibba032 bash[24004]: cluster 2021-03-10T15:59:17.991279+0000 mgr.gibba032.whopeo (mgr.24625) 6683 : cluster [DBG] pgmap v3803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:21 gibba011 bash[30791]: cluster 2021-03-10T15:59:19.992415+0000 mgr.gibba032.whopeo (mgr.24625) 6684 : cluster [DBG] pgmap v3804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:21 gibba011 bash[31774]: cluster 2021-03-10T15:59:19.992415+0000 mgr.gibba032.whopeo (mgr.24625) 6684 : cluster [DBG] pgmap v3804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:21.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:21 gibba032 bash[24004]: cluster 2021-03-10T15:59:19.992415+0000 mgr.gibba032.whopeo (mgr.24625) 6684 : cluster [DBG] pgmap v3804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:22.590 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:59:22.816 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:59:22.816 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:59:22.816 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:59:22.817 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:59:22.817 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:59:22.817 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:59:22.817 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:59:22.818 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:59:22.818 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:59:22.818 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:59:22.818 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:59:22.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:59:22.819 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:59:22.819 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:59:22.819 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:59:22.819 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:59:22.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:59:22.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:59:22.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:59:22.820 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:59:22.820 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:59:23.066 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:59:23.067 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:59:23.067 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:59:23.067 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:23.067 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:59:23.068 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:59:23.068 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:23.068 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:59:23.068 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:59:23.069 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:23.069 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:59:23.069 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:59:23.069 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:59:23.069 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:59:23.070 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:59:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:23 gibba011 bash[30791]: cluster 2021-03-10T15:59:21.994163+0000 mgr.gibba032.whopeo (mgr.24625) 6685 : cluster [DBG] pgmap v3805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:23 gibba011 bash[30791]: audit 2021-03-10T15:59:23.066736+0000 mon.c (mon.1) 127 : audit [DBG] from='client.? 172.21.2.111:0/2702385439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:23 gibba011 bash[31774]: cluster 2021-03-10T15:59:21.994163+0000 mgr.gibba032.whopeo (mgr.24625) 6685 : cluster [DBG] pgmap v3805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:23 gibba011 bash[31774]: audit 2021-03-10T15:59:23.066736+0000 mon.c (mon.1) 127 : audit [DBG] from='client.? 172.21.2.111:0/2702385439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:23 gibba032 bash[24004]: cluster 2021-03-10T15:59:21.994163+0000 mgr.gibba032.whopeo (mgr.24625) 6685 : cluster [DBG] pgmap v3805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:23 gibba032 bash[24004]: audit 2021-03-10T15:59:23.066736+0000 mon.c (mon.1) 127 : audit [DBG] from='client.? 172.21.2.111:0/2702385439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:24 gibba032 bash[24004]: audit 2021-03-10T15:59:22.579621+0000 mgr.gibba032.whopeo (mgr.24625) 6686 : audit [DBG] from='client.48537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:24 gibba032 bash[24004]: audit 2021-03-10T15:59:22.802903+0000 mgr.gibba032.whopeo (mgr.24625) 6687 : audit [DBG] from='client.58525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:24.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:24 gibba011 bash[30791]: audit 2021-03-10T15:59:22.579621+0000 mgr.gibba032.whopeo (mgr.24625) 6686 : audit [DBG] from='client.48537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:24 gibba011 bash[30791]: audit 2021-03-10T15:59:22.802903+0000 mgr.gibba032.whopeo (mgr.24625) 6687 : audit [DBG] from='client.58525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:24 gibba011 bash[31774]: audit 2021-03-10T15:59:22.579621+0000 mgr.gibba032.whopeo (mgr.24625) 6686 : audit [DBG] from='client.48537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:24 gibba011 bash[31774]: audit 2021-03-10T15:59:22.802903+0000 mgr.gibba032.whopeo (mgr.24625) 6687 : audit [DBG] from='client.58525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:25.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:25 gibba032 bash[24004]: cluster 2021-03-10T15:59:23.995450+0000 mgr.gibba032.whopeo (mgr.24625) 6688 : cluster [DBG] pgmap v3806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:25.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:25 gibba011 bash[30791]: cluster 2021-03-10T15:59:23.995450+0000 mgr.gibba032.whopeo (mgr.24625) 6688 : cluster [DBG] pgmap v3806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:25.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:25 gibba011 bash[31774]: cluster 2021-03-10T15:59:23.995450+0000 mgr.gibba032.whopeo (mgr.24625) 6688 : cluster [DBG] pgmap v3806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:27 gibba032 bash[24004]: cephadm 2021-03-10T15:59:25.912600+0000 mgr.gibba032.whopeo (mgr.24625) 6689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:27.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:27 gibba032 bash[24004]: cluster 2021-03-10T15:59:25.996933+0000 mgr.gibba032.whopeo (mgr.24625) 6690 : cluster [DBG] pgmap v3807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:27.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:27 gibba011 bash[30791]: cephadm 2021-03-10T15:59:25.912600+0000 mgr.gibba032.whopeo (mgr.24625) 6689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:27.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:27 gibba011 bash[30791]: cluster 2021-03-10T15:59:25.996933+0000 mgr.gibba032.whopeo (mgr.24625) 6690 : cluster [DBG] pgmap v3807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:27.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:27 gibba011 bash[31774]: cephadm 2021-03-10T15:59:25.912600+0000 mgr.gibba032.whopeo (mgr.24625) 6689 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:27.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:27 gibba011 bash[31774]: cluster 2021-03-10T15:59:25.996933+0000 mgr.gibba032.whopeo (mgr.24625) 6690 : cluster [DBG] pgmap v3807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:28 gibba032 bash[24004]: cluster 2021-03-10T15:59:27.998661+0000 mgr.gibba032.whopeo (mgr.24625) 6691 : cluster [DBG] pgmap v3808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:28 gibba011 bash[30791]: cluster 2021-03-10T15:59:27.998661+0000 mgr.gibba032.whopeo (mgr.24625) 6691 : cluster [DBG] pgmap v3808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:28 gibba011 bash[31774]: cluster 2021-03-10T15:59:27.998661+0000 mgr.gibba032.whopeo (mgr.24625) 6691 : cluster [DBG] pgmap v3808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:31 gibba011 bash[30791]: cluster 2021-03-10T15:59:29.999887+0000 mgr.gibba032.whopeo (mgr.24625) 6692 : cluster [DBG] pgmap v3809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:31 gibba011 bash[31774]: cluster 2021-03-10T15:59:29.999887+0000 mgr.gibba032.whopeo (mgr.24625) 6692 : cluster [DBG] pgmap v3809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:31 gibba032 bash[24004]: cluster 2021-03-10T15:59:29.999887+0000 mgr.gibba032.whopeo (mgr.24625) 6692 : cluster [DBG] pgmap v3809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:33 gibba011 bash[30791]: cluster 2021-03-10T15:59:32.001423+0000 mgr.gibba032.whopeo (mgr.24625) 6693 : cluster [DBG] pgmap v3810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:33.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:33 gibba011 bash[31774]: cluster 2021-03-10T15:59:32.001423+0000 mgr.gibba032.whopeo (mgr.24625) 6693 : cluster [DBG] pgmap v3810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:33.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:33 gibba032 bash[24004]: cluster 2021-03-10T15:59:32.001423+0000 mgr.gibba032.whopeo (mgr.24625) 6693 : cluster [DBG] pgmap v3810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:34 gibba032 bash[24004]: cluster 2021-03-10T15:59:34.002994+0000 mgr.gibba032.whopeo (mgr.24625) 6694 : cluster [DBG] pgmap v3811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:34 gibba011 bash[30791]: cluster 2021-03-10T15:59:34.002994+0000 mgr.gibba032.whopeo (mgr.24625) 6694 : cluster [DBG] pgmap v3811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:34 gibba011 bash[31774]: cluster 2021-03-10T15:59:34.002994+0000 mgr.gibba032.whopeo (mgr.24625) 6694 : cluster [DBG] pgmap v3811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:37 gibba011 bash[30791]: cluster 2021-03-10T15:59:36.004470+0000 mgr.gibba032.whopeo (mgr.24625) 6695 : cluster [DBG] pgmap v3812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:37 gibba011 bash[31774]: cluster 2021-03-10T15:59:36.004470+0000 mgr.gibba032.whopeo (mgr.24625) 6695 : cluster [DBG] pgmap v3812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:37 gibba032 bash[24004]: cluster 2021-03-10T15:59:36.004470+0000 mgr.gibba032.whopeo (mgr.24625) 6695 : cluster [DBG] pgmap v3812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:39.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:39 gibba011 bash[30791]: cluster 2021-03-10T15:59:38.006200+0000 mgr.gibba032.whopeo (mgr.24625) 6696 : cluster [DBG] pgmap v3813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:39 gibba011 bash[31774]: cluster 2021-03-10T15:59:38.006200+0000 mgr.gibba032.whopeo (mgr.24625) 6696 : cluster [DBG] pgmap v3813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:39.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:39 gibba032 bash[24004]: cluster 2021-03-10T15:59:38.006200+0000 mgr.gibba032.whopeo (mgr.24625) 6696 : cluster [DBG] pgmap v3813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:41 gibba032 bash[24004]: cluster 2021-03-10T15:59:40.006669+0000 mgr.gibba032.whopeo (mgr.24625) 6697 : cluster [DBG] pgmap v3814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:41 gibba032 bash[24004]: audit 2021-03-10T15:59:40.932596+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:59:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:41 gibba011 bash[30791]: cluster 2021-03-10T15:59:40.006669+0000 mgr.gibba032.whopeo (mgr.24625) 6697 : cluster [DBG] pgmap v3814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:41.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:41 gibba011 bash[30791]: audit 2021-03-10T15:59:40.932596+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:59:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:41 gibba011 bash[31774]: cluster 2021-03-10T15:59:40.006669+0000 mgr.gibba032.whopeo (mgr.24625) 6697 : cluster [DBG] pgmap v3814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:41 gibba011 bash[31774]: audit 2021-03-10T15:59:40.932596+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T15:59:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:42 gibba032 bash[24004]: cephadm 2021-03-10T15:59:40.930133+0000 mgr.gibba032.whopeo (mgr.24625) 6698 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:59:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:42 gibba011 bash[30791]: cephadm 2021-03-10T15:59:40.930133+0000 mgr.gibba032.whopeo (mgr.24625) 6698 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:59:42.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:42 gibba011 bash[31774]: cephadm 2021-03-10T15:59:40.930133+0000 mgr.gibba032.whopeo (mgr.24625) 6698 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T15:59:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:43 gibba032 bash[24004]: cluster 2021-03-10T15:59:42.008092+0000 mgr.gibba032.whopeo (mgr.24625) 6699 : cluster [DBG] pgmap v3815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:43.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:43 gibba011 bash[30791]: cluster 2021-03-10T15:59:42.008092+0000 mgr.gibba032.whopeo (mgr.24625) 6699 : cluster [DBG] pgmap v3815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:43.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:43 gibba011 bash[31774]: cluster 2021-03-10T15:59:42.008092+0000 mgr.gibba032.whopeo (mgr.24625) 6699 : cluster [DBG] pgmap v3815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:44 gibba032 bash[24004]: cluster 2021-03-10T15:59:44.009493+0000 mgr.gibba032.whopeo (mgr.24625) 6700 : cluster [DBG] pgmap v3816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:44 gibba032 bash[24004]: audit 2021-03-10T15:59:44.473431+0000 mon.a (mon.0) 2447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:44 gibba011 bash[30791]: cluster 2021-03-10T15:59:44.009493+0000 mgr.gibba032.whopeo (mgr.24625) 6700 : cluster [DBG] pgmap v3816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:44 gibba011 bash[30791]: audit 2021-03-10T15:59:44.473431+0000 mon.a (mon.0) 2447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:44 gibba011 bash[31774]: cluster 2021-03-10T15:59:44.009493+0000 mgr.gibba032.whopeo (mgr.24625) 6700 : cluster [DBG] pgmap v3816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:44 gibba011 bash[31774]: audit 2021-03-10T15:59:44.473431+0000 mon.a (mon.0) 2447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.472173+0000 mgr.gibba032.whopeo (mgr.24625) 6701 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:59:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.712458+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.722187+0000 mgr.gibba032.whopeo (mgr.24625) 6702 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:59:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.724374+0000 mgr.gibba032.whopeo (mgr.24625) 6703 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:59:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.725102+0000 mon.a (mon.0) 2449 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:59:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.725861+0000 mgr.gibba032.whopeo (mgr.24625) 6704 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:59:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.725898+0000 mgr.gibba032.whopeo (mgr.24625) 6705 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:59:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.726736+0000 mon.a (mon.0) 2450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.727408+0000 mgr.gibba032.whopeo (mgr.24625) 6706 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:59:46.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.727444+0000 mgr.gibba032.whopeo (mgr.24625) 6707 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:59:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.728477+0000 mon.a (mon.0) 2451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.728869+0000 mgr.gibba032.whopeo (mgr.24625) 6708 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:59:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.729740+0000 mon.a (mon.0) 2452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.730552+0000 mgr.gibba032.whopeo (mgr.24625) 6709 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:59:46.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.730589+0000 mgr.gibba032.whopeo (mgr.24625) 6710 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:59:46.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.732123+0000 mgr.gibba032.whopeo (mgr.24625) 6711 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:59:46.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.732428+0000 mgr.gibba032.whopeo (mgr.24625) 6712 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:59:46.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.733011+0000 mon.a (mon.0) 2453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.733501+0000 mgr.gibba032.whopeo (mgr.24625) 6713 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:59:46.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.733539+0000 mgr.gibba032.whopeo (mgr.24625) 6714 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:59:46.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: cephadm 2021-03-10T15:59:44.733640+0000 mgr.gibba032.whopeo (mgr.24625) 6715 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:46.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:45 gibba032 bash[24004]: audit 2021-03-10T15:59:44.734660+0000 mon.a (mon.0) 2454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.472173+0000 mgr.gibba032.whopeo (mgr.24625) 6701 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:59:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.712458+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.722187+0000 mgr.gibba032.whopeo (mgr.24625) 6702 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:59:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.724374+0000 mgr.gibba032.whopeo (mgr.24625) 6703 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:59:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.725102+0000 mon.a (mon.0) 2449 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:59:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.725861+0000 mgr.gibba032.whopeo (mgr.24625) 6704 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:59:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.725898+0000 mgr.gibba032.whopeo (mgr.24625) 6705 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:59:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.726736+0000 mon.a (mon.0) 2450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.727408+0000 mgr.gibba032.whopeo (mgr.24625) 6706 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:59:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.727444+0000 mgr.gibba032.whopeo (mgr.24625) 6707 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:59:46.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.728477+0000 mon.a (mon.0) 2451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.728869+0000 mgr.gibba032.whopeo (mgr.24625) 6708 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:59:46.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.729740+0000 mon.a (mon.0) 2452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.730552+0000 mgr.gibba032.whopeo (mgr.24625) 6709 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:59:46.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.730589+0000 mgr.gibba032.whopeo (mgr.24625) 6710 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:59:46.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.732123+0000 mgr.gibba032.whopeo (mgr.24625) 6711 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:59:46.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.732428+0000 mgr.gibba032.whopeo (mgr.24625) 6712 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:59:46.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.733011+0000 mon.a (mon.0) 2453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.733501+0000 mgr.gibba032.whopeo (mgr.24625) 6713 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:59:46.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.733539+0000 mgr.gibba032.whopeo (mgr.24625) 6714 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:59:46.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: cephadm 2021-03-10T15:59:44.733640+0000 mgr.gibba032.whopeo (mgr.24625) 6715 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:46.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[30791]: audit 2021-03-10T15:59:44.734660+0000 mon.a (mon.0) 2454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.472173+0000 mgr.gibba032.whopeo (mgr.24625) 6701 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T15:59:46.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.712458+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T15:59:46.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.722187+0000 mgr.gibba032.whopeo (mgr.24625) 6702 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T15:59:46.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.724374+0000 mgr.gibba032.whopeo (mgr.24625) 6703 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T15:59:46.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.725102+0000 mon.a (mon.0) 2449 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T15:59:46.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.725861+0000 mgr.gibba032.whopeo (mgr.24625) 6704 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T15:59:46.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.725898+0000 mgr.gibba032.whopeo (mgr.24625) 6705 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T15:59:46.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.726736+0000 mon.a (mon.0) 2450 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.727408+0000 mgr.gibba032.whopeo (mgr.24625) 6706 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T15:59:46.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.727444+0000 mgr.gibba032.whopeo (mgr.24625) 6707 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T15:59:46.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.728477+0000 mon.a (mon.0) 2451 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.728869+0000 mgr.gibba032.whopeo (mgr.24625) 6708 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T15:59:46.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.729740+0000 mon.a (mon.0) 2452 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.730552+0000 mgr.gibba032.whopeo (mgr.24625) 6709 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T15:59:46.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.730589+0000 mgr.gibba032.whopeo (mgr.24625) 6710 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T15:59:46.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.732123+0000 mgr.gibba032.whopeo (mgr.24625) 6711 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T15:59:46.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.732428+0000 mgr.gibba032.whopeo (mgr.24625) 6712 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T15:59:46.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.733011+0000 mon.a (mon.0) 2453 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:46.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.733501+0000 mgr.gibba032.whopeo (mgr.24625) 6713 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T15:59:46.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.733539+0000 mgr.gibba032.whopeo (mgr.24625) 6714 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T15:59:46.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: cephadm 2021-03-10T15:59:44.733640+0000 mgr.gibba032.whopeo (mgr.24625) 6715 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T15:59:46.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:45 gibba011 bash[31774]: audit 2021-03-10T15:59:44.734660+0000 mon.a (mon.0) 2454 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:47.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:46 gibba032 bash[24004]: cluster 2021-03-10T15:59:46.011051+0000 mgr.gibba032.whopeo (mgr.24625) 6716 : cluster [DBG] pgmap v3817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:46 gibba011 bash[30791]: cluster 2021-03-10T15:59:46.011051+0000 mgr.gibba032.whopeo (mgr.24625) 6716 : cluster [DBG] pgmap v3817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:46 gibba011 bash[31774]: cluster 2021-03-10T15:59:46.011051+0000 mgr.gibba032.whopeo (mgr.24625) 6716 : cluster [DBG] pgmap v3817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:49.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:49 gibba011 bash[30791]: cluster 2021-03-10T15:59:48.013000+0000 mgr.gibba032.whopeo (mgr.24625) 6717 : cluster [DBG] pgmap v3818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:49 gibba011 bash[31774]: cluster 2021-03-10T15:59:48.013000+0000 mgr.gibba032.whopeo (mgr.24625) 6717 : cluster [DBG] pgmap v3818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:49 gibba032 bash[24004]: cluster 2021-03-10T15:59:48.013000+0000 mgr.gibba032.whopeo (mgr.24625) 6717 : cluster [DBG] pgmap v3818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:51 gibba032 bash[24004]: cluster 2021-03-10T15:59:50.013766+0000 mgr.gibba032.whopeo (mgr.24625) 6718 : cluster [DBG] pgmap v3819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:51.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:51 gibba011 bash[30791]: cluster 2021-03-10T15:59:50.013766+0000 mgr.gibba032.whopeo (mgr.24625) 6718 : cluster [DBG] pgmap v3819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:51.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:51 gibba011 bash[31774]: cluster 2021-03-10T15:59:50.013766+0000 mgr.gibba032.whopeo (mgr.24625) 6718 : cluster [DBG] pgmap v3819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:53.313 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T15:59:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:53 gibba032 bash[24004]: cluster 2021-03-10T15:59:52.015191+0000 mgr.gibba032.whopeo (mgr.24625) 6719 : cluster [DBG] pgmap v3820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:53.541 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T15:59:53.542 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T15:59:53.542 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T15:59:53.542 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T15:59:53.542 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T15:59:53.543 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T15:59:53.543 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T15:59:53.543 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T15:59:53.543 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T15:59:53.544 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T15:59:53.544 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T15:59:53.544 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T15:59:53.544 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T15:59:53.544 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T15:59:53.545 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T15:59:53.545 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T15:59:53.545 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T15:59:53.545 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T15:59:53.546 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T15:59:53.546 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T15:59:53.546 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T15:59:53.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:53 gibba011 bash[30791]: cluster 2021-03-10T15:59:52.015191+0000 mgr.gibba032.whopeo (mgr.24625) 6719 : cluster [DBG] pgmap v3820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:53.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:53 gibba011 bash[31774]: cluster 2021-03-10T15:59:52.015191+0000 mgr.gibba032.whopeo (mgr.24625) 6719 : cluster [DBG] pgmap v3820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:53.790 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T15:59:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T15:59:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:59:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T15:59:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T15:59:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T15:59:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T15:59:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T15:59:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T15:59:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T15:59:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T15:59:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T15:59:53.794 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T15:59:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:54 gibba032 bash[24004]: audit 2021-03-10T15:59:53.303269+0000 mgr.gibba032.whopeo (mgr.24625) 6720 : audit [DBG] from='client.58537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:54 gibba032 bash[24004]: audit 2021-03-10T15:59:53.528057+0000 mgr.gibba032.whopeo (mgr.24625) 6721 : audit [DBG] from='client.48561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:54 gibba032 bash[24004]: audit 2021-03-10T15:59:53.791173+0000 mon.a (mon.0) 2455 : audit [DBG] from='client.? 172.21.2.111:0/537286871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:54 gibba032 bash[24004]: cluster 2021-03-10T15:59:54.017074+0000 mgr.gibba032.whopeo (mgr.24625) 6722 : cluster [DBG] pgmap v3821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[30791]: audit 2021-03-10T15:59:53.303269+0000 mgr.gibba032.whopeo (mgr.24625) 6720 : audit [DBG] from='client.58537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[30791]: audit 2021-03-10T15:59:53.528057+0000 mgr.gibba032.whopeo (mgr.24625) 6721 : audit [DBG] from='client.48561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[30791]: audit 2021-03-10T15:59:53.791173+0000 mon.a (mon.0) 2455 : audit [DBG] from='client.? 172.21.2.111:0/537286871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[30791]: cluster 2021-03-10T15:59:54.017074+0000 mgr.gibba032.whopeo (mgr.24625) 6722 : cluster [DBG] pgmap v3821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[31774]: audit 2021-03-10T15:59:53.303269+0000 mgr.gibba032.whopeo (mgr.24625) 6720 : audit [DBG] from='client.58537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[31774]: audit 2021-03-10T15:59:53.528057+0000 mgr.gibba032.whopeo (mgr.24625) 6721 : audit [DBG] from='client.48561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T15:59:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[31774]: audit 2021-03-10T15:59:53.791173+0000 mon.a (mon.0) 2455 : audit [DBG] from='client.? 172.21.2.111:0/537286871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T15:59:55.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:54 gibba011 bash[31774]: cluster 2021-03-10T15:59:54.017074+0000 mgr.gibba032.whopeo (mgr.24625) 6722 : cluster [DBG] pgmap v3821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: cluster 2021-03-10T15:59:56.018669+0000 mgr.gibba032.whopeo (mgr.24625) 6723 : cluster [DBG] pgmap v3822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.795681+0000 mon.a (mon.0) 2456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:59:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.796322+0000 mon.a (mon.0) 2457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:59:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.851318+0000 mon.a (mon.0) 2458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:59:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.852096+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:59:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.852759+0000 mon.a (mon.0) 2460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:59:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:57 gibba032 bash[24004]: audit 2021-03-10T15:59:56.853431+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:59:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: cluster 2021-03-10T15:59:56.018669+0000 mgr.gibba032.whopeo (mgr.24625) 6723 : cluster [DBG] pgmap v3822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.795681+0000 mon.a (mon.0) 2456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:59:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.796322+0000 mon.a (mon.0) 2457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:59:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.851318+0000 mon.a (mon.0) 2458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:59:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.852096+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:59:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.852759+0000 mon.a (mon.0) 2460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:59:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[30791]: audit 2021-03-10T15:59:56.853431+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:59:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: cluster 2021-03-10T15:59:56.018669+0000 mgr.gibba032.whopeo (mgr.24625) 6723 : cluster [DBG] pgmap v3822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.795681+0000 mon.a (mon.0) 2456 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T15:59:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.796322+0000 mon.a (mon.0) 2457 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T15:59:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.851318+0000 mon.a (mon.0) 2458 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T15:59:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.852096+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T15:59:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.852759+0000 mon.a (mon.0) 2460 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T15:59:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:57 gibba011 bash[31774]: audit 2021-03-10T15:59:56.853431+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T15:59:59.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 15:59:59 gibba032 bash[24004]: cluster 2021-03-10T15:59:58.019233+0000 mgr.gibba032.whopeo (mgr.24625) 6724 : cluster [DBG] pgmap v3823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:59.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 15:59:59 gibba011 bash[30791]: cluster 2021-03-10T15:59:58.019233+0000 mgr.gibba032.whopeo (mgr.24625) 6724 : cluster [DBG] pgmap v3823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T15:59:59.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 15:59:59 gibba011 bash[31774]: cluster 2021-03-10T15:59:58.019233+0000 mgr.gibba032.whopeo (mgr.24625) 6724 : cluster [DBG] pgmap v3823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:00.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:00 gibba011 bash[31774]: cluster 2021-03-10T16:00:00.000481+0000 mon.a (mon.0) 2462 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:00:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:00 gibba011 bash[30791]: debug 2021-03-10T16:00:00.000+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:00:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:00 gibba011 bash[30791]: cluster 2021-03-10T16:00:00.000481+0000 mon.a (mon.0) 2462 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:00:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:00 gibba032 bash[24004]: cluster 2021-03-10T16:00:00.000481+0000 mon.a (mon.0) 2462 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:00:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:01 gibba032 bash[24004]: cephadm 2021-03-10T15:59:59.749668+0000 mgr.gibba032.whopeo (mgr.24625) 6725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:01 gibba032 bash[24004]: cluster 2021-03-10T16:00:00.019553+0000 mgr.gibba032.whopeo (mgr.24625) 6726 : cluster [DBG] pgmap v3824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:01 gibba011 bash[30791]: cephadm 2021-03-10T15:59:59.749668+0000 mgr.gibba032.whopeo (mgr.24625) 6725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:01 gibba011 bash[30791]: cluster 2021-03-10T16:00:00.019553+0000 mgr.gibba032.whopeo (mgr.24625) 6726 : cluster [DBG] pgmap v3824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:01.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:01 gibba011 bash[31774]: cephadm 2021-03-10T15:59:59.749668+0000 mgr.gibba032.whopeo (mgr.24625) 6725 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:01 gibba011 bash[31774]: cluster 2021-03-10T16:00:00.019553+0000 mgr.gibba032.whopeo (mgr.24625) 6726 : cluster [DBG] pgmap v3824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:02 gibba032 bash[24004]: cluster 2021-03-10T16:00:02.021134+0000 mgr.gibba032.whopeo (mgr.24625) 6727 : cluster [DBG] pgmap v3825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:02 gibba011 bash[30791]: cluster 2021-03-10T16:00:02.021134+0000 mgr.gibba032.whopeo (mgr.24625) 6727 : cluster [DBG] pgmap v3825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:02 gibba011 bash[31774]: cluster 2021-03-10T16:00:02.021134+0000 mgr.gibba032.whopeo (mgr.24625) 6727 : cluster [DBG] pgmap v3825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:04 gibba032 bash[24004]: cluster 2021-03-10T16:00:04.022772+0000 mgr.gibba032.whopeo (mgr.24625) 6728 : cluster [DBG] pgmap v3826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:04 gibba011 bash[30791]: cluster 2021-03-10T16:00:04.022772+0000 mgr.gibba032.whopeo (mgr.24625) 6728 : cluster [DBG] pgmap v3826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:04 gibba011 bash[31774]: cluster 2021-03-10T16:00:04.022772+0000 mgr.gibba032.whopeo (mgr.24625) 6728 : cluster [DBG] pgmap v3826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:07 gibba032 bash[24004]: cluster 2021-03-10T16:00:06.024269+0000 mgr.gibba032.whopeo (mgr.24625) 6729 : cluster [DBG] pgmap v3827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:07.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:07 gibba011 bash[30791]: cluster 2021-03-10T16:00:06.024269+0000 mgr.gibba032.whopeo (mgr.24625) 6729 : cluster [DBG] pgmap v3827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:07.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:07 gibba011 bash[31774]: cluster 2021-03-10T16:00:06.024269+0000 mgr.gibba032.whopeo (mgr.24625) 6729 : cluster [DBG] pgmap v3827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:09.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:09 gibba032 bash[24004]: cluster 2021-03-10T16:00:08.025013+0000 mgr.gibba032.whopeo (mgr.24625) 6730 : cluster [DBG] pgmap v3828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:09.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:09 gibba011 bash[30791]: cluster 2021-03-10T16:00:08.025013+0000 mgr.gibba032.whopeo (mgr.24625) 6730 : cluster [DBG] pgmap v3828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:09.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:09 gibba011 bash[31774]: cluster 2021-03-10T16:00:08.025013+0000 mgr.gibba032.whopeo (mgr.24625) 6730 : cluster [DBG] pgmap v3828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:10 gibba032 bash[24004]: cluster 2021-03-10T16:00:10.025631+0000 mgr.gibba032.whopeo (mgr.24625) 6731 : cluster [DBG] pgmap v3829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:10 gibba011 bash[30791]: cluster 2021-03-10T16:00:10.025631+0000 mgr.gibba032.whopeo (mgr.24625) 6731 : cluster [DBG] pgmap v3829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:10 gibba011 bash[31774]: cluster 2021-03-10T16:00:10.025631+0000 mgr.gibba032.whopeo (mgr.24625) 6731 : cluster [DBG] pgmap v3829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:13 gibba032 bash[24004]: cluster 2021-03-10T16:00:12.027188+0000 mgr.gibba032.whopeo (mgr.24625) 6732 : cluster [DBG] pgmap v3830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:13.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:13 gibba011 bash[30791]: cluster 2021-03-10T16:00:12.027188+0000 mgr.gibba032.whopeo (mgr.24625) 6732 : cluster [DBG] pgmap v3830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:13.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:13 gibba011 bash[31774]: cluster 2021-03-10T16:00:12.027188+0000 mgr.gibba032.whopeo (mgr.24625) 6732 : cluster [DBG] pgmap v3830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:14 gibba032 bash[24004]: cluster 2021-03-10T16:00:14.028790+0000 mgr.gibba032.whopeo (mgr.24625) 6733 : cluster [DBG] pgmap v3831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:14 gibba011 bash[30791]: cluster 2021-03-10T16:00:14.028790+0000 mgr.gibba032.whopeo (mgr.24625) 6733 : cluster [DBG] pgmap v3831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:14 gibba011 bash[31774]: cluster 2021-03-10T16:00:14.028790+0000 mgr.gibba032.whopeo (mgr.24625) 6733 : cluster [DBG] pgmap v3831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:16.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:15 gibba032 bash[24004]: cephadm 2021-03-10T16:00:14.765729+0000 mgr.gibba032.whopeo (mgr.24625) 6734 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:15 gibba011 bash[30791]: cephadm 2021-03-10T16:00:14.765729+0000 mgr.gibba032.whopeo (mgr.24625) 6734 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:16.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:15 gibba011 bash[31774]: cephadm 2021-03-10T16:00:14.765729+0000 mgr.gibba032.whopeo (mgr.24625) 6734 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:17.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:16 gibba032 bash[24004]: cluster 2021-03-10T16:00:16.030208+0000 mgr.gibba032.whopeo (mgr.24625) 6735 : cluster [DBG] pgmap v3832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:17.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:16 gibba011 bash[30791]: cluster 2021-03-10T16:00:16.030208+0000 mgr.gibba032.whopeo (mgr.24625) 6735 : cluster [DBG] pgmap v3832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:16 gibba011 bash[31774]: cluster 2021-03-10T16:00:16.030208+0000 mgr.gibba032.whopeo (mgr.24625) 6735 : cluster [DBG] pgmap v3832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:19 gibba032 bash[24004]: cluster 2021-03-10T16:00:18.031185+0000 mgr.gibba032.whopeo (mgr.24625) 6736 : cluster [DBG] pgmap v3833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:19.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:19 gibba011 bash[30791]: cluster 2021-03-10T16:00:18.031185+0000 mgr.gibba032.whopeo (mgr.24625) 6736 : cluster [DBG] pgmap v3833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:19.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:19 gibba011 bash[31774]: cluster 2021-03-10T16:00:18.031185+0000 mgr.gibba032.whopeo (mgr.24625) 6736 : cluster [DBG] pgmap v3833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:21.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:21 gibba032 bash[24004]: cluster 2021-03-10T16:00:20.031822+0000 mgr.gibba032.whopeo (mgr.24625) 6737 : cluster [DBG] pgmap v3834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:21.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:21 gibba011 bash[30791]: cluster 2021-03-10T16:00:20.031822+0000 mgr.gibba032.whopeo (mgr.24625) 6737 : cluster [DBG] pgmap v3834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:21.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:21 gibba011 bash[31774]: cluster 2021-03-10T16:00:20.031822+0000 mgr.gibba032.whopeo (mgr.24625) 6737 : cluster [DBG] pgmap v3834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:22 gibba032 bash[24004]: cluster 2021-03-10T16:00:22.033377+0000 mgr.gibba032.whopeo (mgr.24625) 6738 : cluster [DBG] pgmap v3835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:22 gibba011 bash[30791]: cluster 2021-03-10T16:00:22.033377+0000 mgr.gibba032.whopeo (mgr.24625) 6738 : cluster [DBG] pgmap v3835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:22 gibba011 bash[31774]: cluster 2021-03-10T16:00:22.033377+0000 mgr.gibba032.whopeo (mgr.24625) 6738 : cluster [DBG] pgmap v3835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:24.034 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:00:24.260 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:00:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 39s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:00:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:00:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:00:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:00:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:00:24.262 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:00:24.262 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:00:24.262 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:00:24.262 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 39s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:00:24.263 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:00:24.263 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:00:24.263 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:00:24.263 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:00:24.263 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 39s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:00:24.264 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:00:24.264 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:00:24.264 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:00:24.264 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:00:24.265 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:00:24.265 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 39s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:00:24.536 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:00:24.536 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:00:24.537 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:00:24.537 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:24.537 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:00:24.537 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:00:24.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:24.538 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:00:24.538 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:00:24.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:24.539 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:00:24.539 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:00:24.539 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:00:24.539 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:00:24.540 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:00:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:24 gibba032 bash[24004]: audit 2021-03-10T16:00:24.023847+0000 mgr.gibba032.whopeo (mgr.24625) 6739 : audit [DBG] from='client.58555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:24 gibba032 bash[24004]: cluster 2021-03-10T16:00:24.034842+0000 mgr.gibba032.whopeo (mgr.24625) 6740 : cluster [DBG] pgmap v3836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:25.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:24 gibba032 bash[24004]: audit 2021-03-10T16:00:24.536769+0000 mon.a (mon.0) 2463 : audit [DBG] from='client.? 172.21.2.111:0/2571947142' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:25.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[30791]: audit 2021-03-10T16:00:24.023847+0000 mgr.gibba032.whopeo (mgr.24625) 6739 : audit [DBG] from='client.58555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[30791]: cluster 2021-03-10T16:00:24.034842+0000 mgr.gibba032.whopeo (mgr.24625) 6740 : cluster [DBG] pgmap v3836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[30791]: audit 2021-03-10T16:00:24.536769+0000 mon.a (mon.0) 2463 : audit [DBG] from='client.? 172.21.2.111:0/2571947142' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[31774]: audit 2021-03-10T16:00:24.023847+0000 mgr.gibba032.whopeo (mgr.24625) 6739 : audit [DBG] from='client.58555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[31774]: cluster 2021-03-10T16:00:24.034842+0000 mgr.gibba032.whopeo (mgr.24625) 6740 : cluster [DBG] pgmap v3836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:24 gibba011 bash[31774]: audit 2021-03-10T16:00:24.536769+0000 mon.a (mon.0) 2463 : audit [DBG] from='client.? 172.21.2.111:0/2571947142' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:26.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:25 gibba032 bash[24004]: audit 2021-03-10T16:00:24.246666+0000 mgr.gibba032.whopeo (mgr.24625) 6741 : audit [DBG] from='client.58561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:26.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:25 gibba011 bash[30791]: audit 2021-03-10T16:00:24.246666+0000 mgr.gibba032.whopeo (mgr.24625) 6741 : audit [DBG] from='client.58561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:26.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:25 gibba011 bash[31774]: audit 2021-03-10T16:00:24.246666+0000 mgr.gibba032.whopeo (mgr.24625) 6741 : audit [DBG] from='client.58561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:26 gibba032 bash[24004]: cluster 2021-03-10T16:00:26.036483+0000 mgr.gibba032.whopeo (mgr.24625) 6742 : cluster [DBG] pgmap v3837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:27.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:26 gibba011 bash[30791]: cluster 2021-03-10T16:00:26.036483+0000 mgr.gibba032.whopeo (mgr.24625) 6742 : cluster [DBG] pgmap v3837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:26 gibba011 bash[31774]: cluster 2021-03-10T16:00:26.036483+0000 mgr.gibba032.whopeo (mgr.24625) 6742 : cluster [DBG] pgmap v3837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:29 gibba032 bash[24004]: cluster 2021-03-10T16:00:28.037218+0000 mgr.gibba032.whopeo (mgr.24625) 6743 : cluster [DBG] pgmap v3838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:29.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:29 gibba011 bash[30791]: cluster 2021-03-10T16:00:28.037218+0000 mgr.gibba032.whopeo (mgr.24625) 6743 : cluster [DBG] pgmap v3838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:29.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:29 gibba011 bash[31774]: cluster 2021-03-10T16:00:28.037218+0000 mgr.gibba032.whopeo (mgr.24625) 6743 : cluster [DBG] pgmap v3838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:31.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:31 gibba032 bash[24004]: cephadm 2021-03-10T16:00:29.770094+0000 mgr.gibba032.whopeo (mgr.24625) 6744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:31 gibba032 bash[24004]: cluster 2021-03-10T16:00:30.037905+0000 mgr.gibba032.whopeo (mgr.24625) 6745 : cluster [DBG] pgmap v3839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:31.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:31 gibba011 bash[30791]: cephadm 2021-03-10T16:00:29.770094+0000 mgr.gibba032.whopeo (mgr.24625) 6744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:31.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:31 gibba011 bash[30791]: cluster 2021-03-10T16:00:30.037905+0000 mgr.gibba032.whopeo (mgr.24625) 6745 : cluster [DBG] pgmap v3839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:31.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:31 gibba011 bash[31774]: cephadm 2021-03-10T16:00:29.770094+0000 mgr.gibba032.whopeo (mgr.24625) 6744 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:31.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:31 gibba011 bash[31774]: cluster 2021-03-10T16:00:30.037905+0000 mgr.gibba032.whopeo (mgr.24625) 6745 : cluster [DBG] pgmap v3839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:32 gibba032 bash[24004]: cluster 2021-03-10T16:00:32.039190+0000 mgr.gibba032.whopeo (mgr.24625) 6746 : cluster [DBG] pgmap v3840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:32 gibba011 bash[30791]: cluster 2021-03-10T16:00:32.039190+0000 mgr.gibba032.whopeo (mgr.24625) 6746 : cluster [DBG] pgmap v3840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:32 gibba011 bash[31774]: cluster 2021-03-10T16:00:32.039190+0000 mgr.gibba032.whopeo (mgr.24625) 6746 : cluster [DBG] pgmap v3840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:34 gibba032 bash[24004]: cluster 2021-03-10T16:00:34.040846+0000 mgr.gibba032.whopeo (mgr.24625) 6747 : cluster [DBG] pgmap v3841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:34 gibba011 bash[30791]: cluster 2021-03-10T16:00:34.040846+0000 mgr.gibba032.whopeo (mgr.24625) 6747 : cluster [DBG] pgmap v3841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:34 gibba011 bash[31774]: cluster 2021-03-10T16:00:34.040846+0000 mgr.gibba032.whopeo (mgr.24625) 6747 : cluster [DBG] pgmap v3841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:37 gibba032 bash[24004]: cluster 2021-03-10T16:00:36.042289+0000 mgr.gibba032.whopeo (mgr.24625) 6748 : cluster [DBG] pgmap v3842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:37.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:37 gibba011 bash[30791]: cluster 2021-03-10T16:00:36.042289+0000 mgr.gibba032.whopeo (mgr.24625) 6748 : cluster [DBG] pgmap v3842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:37 gibba011 bash[31774]: cluster 2021-03-10T16:00:36.042289+0000 mgr.gibba032.whopeo (mgr.24625) 6748 : cluster [DBG] pgmap v3842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:38 gibba032 bash[24004]: cluster 2021-03-10T16:00:38.043154+0000 mgr.gibba032.whopeo (mgr.24625) 6749 : cluster [DBG] pgmap v3843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:38 gibba011 bash[30791]: cluster 2021-03-10T16:00:38.043154+0000 mgr.gibba032.whopeo (mgr.24625) 6749 : cluster [DBG] pgmap v3843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:38 gibba011 bash[31774]: cluster 2021-03-10T16:00:38.043154+0000 mgr.gibba032.whopeo (mgr.24625) 6749 : cluster [DBG] pgmap v3843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:41 gibba032 bash[24004]: cluster 2021-03-10T16:00:40.043813+0000 mgr.gibba032.whopeo (mgr.24625) 6750 : cluster [DBG] pgmap v3844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:41 gibba011 bash[30791]: cluster 2021-03-10T16:00:40.043813+0000 mgr.gibba032.whopeo (mgr.24625) 6750 : cluster [DBG] pgmap v3844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:41 gibba011 bash[31774]: cluster 2021-03-10T16:00:40.043813+0000 mgr.gibba032.whopeo (mgr.24625) 6750 : cluster [DBG] pgmap v3844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:43 gibba032 bash[24004]: cluster 2021-03-10T16:00:42.045226+0000 mgr.gibba032.whopeo (mgr.24625) 6751 : cluster [DBG] pgmap v3845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:43.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:43 gibba011 bash[30791]: cluster 2021-03-10T16:00:42.045226+0000 mgr.gibba032.whopeo (mgr.24625) 6751 : cluster [DBG] pgmap v3845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:43.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:43 gibba011 bash[31774]: cluster 2021-03-10T16:00:42.045226+0000 mgr.gibba032.whopeo (mgr.24625) 6751 : cluster [DBG] pgmap v3845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:44 gibba032 bash[24004]: cluster 2021-03-10T16:00:44.046811+0000 mgr.gibba032.whopeo (mgr.24625) 6752 : cluster [DBG] pgmap v3846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:44 gibba011 bash[30791]: cluster 2021-03-10T16:00:44.046811+0000 mgr.gibba032.whopeo (mgr.24625) 6752 : cluster [DBG] pgmap v3846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:44 gibba011 bash[31774]: cluster 2021-03-10T16:00:44.046811+0000 mgr.gibba032.whopeo (mgr.24625) 6752 : cluster [DBG] pgmap v3846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:45 gibba032 bash[24004]: cephadm 2021-03-10T16:00:44.786915+0000 mgr.gibba032.whopeo (mgr.24625) 6753 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:00:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:45 gibba032 bash[24004]: audit 2021-03-10T16:00:44.790553+0000 mon.a (mon.0) 2464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:00:46.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:45 gibba011 bash[30791]: cephadm 2021-03-10T16:00:44.786915+0000 mgr.gibba032.whopeo (mgr.24625) 6753 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:00:46.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:45 gibba011 bash[30791]: audit 2021-03-10T16:00:44.790553+0000 mon.a (mon.0) 2464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:00:46.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:45 gibba011 bash[31774]: cephadm 2021-03-10T16:00:44.786915+0000 mgr.gibba032.whopeo (mgr.24625) 6753 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:00:46.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:45 gibba011 bash[31774]: audit 2021-03-10T16:00:44.790553+0000 mon.a (mon.0) 2464 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:00:47.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:46 gibba032 bash[24004]: cluster 2021-03-10T16:00:46.048317+0000 mgr.gibba032.whopeo (mgr.24625) 6754 : cluster [DBG] pgmap v3847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:47.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:46 gibba011 bash[30791]: cluster 2021-03-10T16:00:46.048317+0000 mgr.gibba032.whopeo (mgr.24625) 6754 : cluster [DBG] pgmap v3847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:46 gibba011 bash[31774]: cluster 2021-03-10T16:00:46.048317+0000 mgr.gibba032.whopeo (mgr.24625) 6754 : cluster [DBG] pgmap v3847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: cluster 2021-03-10T16:00:48.048873+0000 mgr.gibba032.whopeo (mgr.24625) 6755 : cluster [DBG] pgmap v3848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.538083+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.740958+0000 mon.a (mon.0) 2466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.744492+0000 mon.a (mon.0) 2467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:00:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.746692+0000 mon.a (mon.0) 2468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.748451+0000 mon.a (mon.0) 2469 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.750275+0000 mon.a (mon.0) 2470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.756274+0000 mon.a (mon.0) 2471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:49 gibba032 bash[24004]: audit 2021-03-10T16:00:48.759076+0000 mon.a (mon.0) 2472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: cluster 2021-03-10T16:00:48.048873+0000 mgr.gibba032.whopeo (mgr.24625) 6755 : cluster [DBG] pgmap v3848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.538083+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.740958+0000 mon.a (mon.0) 2466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.744492+0000 mon.a (mon.0) 2467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:00:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.746692+0000 mon.a (mon.0) 2468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.748451+0000 mon.a (mon.0) 2469 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.750275+0000 mon.a (mon.0) 2470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.756274+0000 mon.a (mon.0) 2471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[30791]: audit 2021-03-10T16:00:48.759076+0000 mon.a (mon.0) 2472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: cluster 2021-03-10T16:00:48.048873+0000 mgr.gibba032.whopeo (mgr.24625) 6755 : cluster [DBG] pgmap v3848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.538083+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.740958+0000 mon.a (mon.0) 2466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:00:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.744492+0000 mon.a (mon.0) 2467 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:00:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.746692+0000 mon.a (mon.0) 2468 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.748451+0000 mon.a (mon.0) 2469 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.750275+0000 mon.a (mon.0) 2470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.756274+0000 mon.a (mon.0) 2471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:49.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:49 gibba011 bash[31774]: audit 2021-03-10T16:00:48.759076+0000 mon.a (mon.0) 2472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.537026+0000 mgr.gibba032.whopeo (mgr.24625) 6756 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:00:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.741639+0000 mgr.gibba032.whopeo (mgr.24625) 6757 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:00:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.744157+0000 mgr.gibba032.whopeo (mgr.24625) 6758 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:00:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.745908+0000 mgr.gibba032.whopeo (mgr.24625) 6759 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:00:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.745946+0000 mgr.gibba032.whopeo (mgr.24625) 6760 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:00:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.747761+0000 mgr.gibba032.whopeo (mgr.24625) 6761 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:00:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.747799+0000 mgr.gibba032.whopeo (mgr.24625) 6762 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:00:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.750373+0000 mgr.gibba032.whopeo (mgr.24625) 6763 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:00:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.753195+0000 mgr.gibba032.whopeo (mgr.24625) 6764 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:00:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.753330+0000 mgr.gibba032.whopeo (mgr.24625) 6765 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:00:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.756414+0000 mgr.gibba032.whopeo (mgr.24625) 6766 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:00:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.756759+0000 mgr.gibba032.whopeo (mgr.24625) 6767 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:00:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.758721+0000 mgr.gibba032.whopeo (mgr.24625) 6768 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:00:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.758761+0000 mgr.gibba032.whopeo (mgr.24625) 6769 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:00:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:50 gibba032 bash[24004]: cephadm 2021-03-10T16:00:48.758953+0000 mgr.gibba032.whopeo (mgr.24625) 6770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.537026+0000 mgr.gibba032.whopeo (mgr.24625) 6756 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:00:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.741639+0000 mgr.gibba032.whopeo (mgr.24625) 6757 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:00:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.744157+0000 mgr.gibba032.whopeo (mgr.24625) 6758 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:00:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.745908+0000 mgr.gibba032.whopeo (mgr.24625) 6759 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:00:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.745946+0000 mgr.gibba032.whopeo (mgr.24625) 6760 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:00:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.747761+0000 mgr.gibba032.whopeo (mgr.24625) 6761 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:00:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.747799+0000 mgr.gibba032.whopeo (mgr.24625) 6762 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:00:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.750373+0000 mgr.gibba032.whopeo (mgr.24625) 6763 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:00:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.753195+0000 mgr.gibba032.whopeo (mgr.24625) 6764 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:00:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.753330+0000 mgr.gibba032.whopeo (mgr.24625) 6765 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:00:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.756414+0000 mgr.gibba032.whopeo (mgr.24625) 6766 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:00:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.756759+0000 mgr.gibba032.whopeo (mgr.24625) 6767 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:00:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.758721+0000 mgr.gibba032.whopeo (mgr.24625) 6768 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:00:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.758761+0000 mgr.gibba032.whopeo (mgr.24625) 6769 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:00:50.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[30791]: cephadm 2021-03-10T16:00:48.758953+0000 mgr.gibba032.whopeo (mgr.24625) 6770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.537026+0000 mgr.gibba032.whopeo (mgr.24625) 6756 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:00:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.741639+0000 mgr.gibba032.whopeo (mgr.24625) 6757 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:00:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.744157+0000 mgr.gibba032.whopeo (mgr.24625) 6758 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:00:50.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.745908+0000 mgr.gibba032.whopeo (mgr.24625) 6759 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:00:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.745946+0000 mgr.gibba032.whopeo (mgr.24625) 6760 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:00:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.747761+0000 mgr.gibba032.whopeo (mgr.24625) 6761 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:00:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.747799+0000 mgr.gibba032.whopeo (mgr.24625) 6762 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:00:50.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.750373+0000 mgr.gibba032.whopeo (mgr.24625) 6763 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:00:50.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.753195+0000 mgr.gibba032.whopeo (mgr.24625) 6764 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:00:50.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.753330+0000 mgr.gibba032.whopeo (mgr.24625) 6765 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:00:50.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.756414+0000 mgr.gibba032.whopeo (mgr.24625) 6766 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:00:50.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.756759+0000 mgr.gibba032.whopeo (mgr.24625) 6767 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:00:50.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.758721+0000 mgr.gibba032.whopeo (mgr.24625) 6768 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:00:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.758761+0000 mgr.gibba032.whopeo (mgr.24625) 6769 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:00:50.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:50 gibba011 bash[31774]: cephadm 2021-03-10T16:00:48.758953+0000 mgr.gibba032.whopeo (mgr.24625) 6770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:00:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:51 gibba032 bash[24004]: cluster 2021-03-10T16:00:50.049426+0000 mgr.gibba032.whopeo (mgr.24625) 6771 : cluster [DBG] pgmap v3849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:51.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:51 gibba011 bash[30791]: cluster 2021-03-10T16:00:50.049426+0000 mgr.gibba032.whopeo (mgr.24625) 6771 : cluster [DBG] pgmap v3849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:51.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:51 gibba011 bash[31774]: cluster 2021-03-10T16:00:50.049426+0000 mgr.gibba032.whopeo (mgr.24625) 6771 : cluster [DBG] pgmap v3849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:52 gibba032 bash[24004]: cluster 2021-03-10T16:00:52.050788+0000 mgr.gibba032.whopeo (mgr.24625) 6772 : cluster [DBG] pgmap v3850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:52 gibba011 bash[30791]: cluster 2021-03-10T16:00:52.050788+0000 mgr.gibba032.whopeo (mgr.24625) 6772 : cluster [DBG] pgmap v3850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:52 gibba011 bash[31774]: cluster 2021-03-10T16:00:52.050788+0000 mgr.gibba032.whopeo (mgr.24625) 6772 : cluster [DBG] pgmap v3850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:54.786 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:00:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:54 gibba032 bash[24004]: cluster 2021-03-10T16:00:54.052149+0000 mgr.gibba032.whopeo (mgr.24625) 6773 : cluster [DBG] pgmap v3851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:55.013 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:00:55.014 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 70s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:00:55.014 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:00:55.014 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:00:55.014 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:00:55.015 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:00:55.015 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:00:55.015 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:00:55.015 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:00:55.015 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 70s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:00:55.016 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:00:55.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:00:55.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:00:55.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:00:55.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 70s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:00:55.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:00:55.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:00:55.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:00:55.017 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:00:55.018 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:00:55.018 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 70s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:00:55.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:54 gibba011 bash[30791]: cluster 2021-03-10T16:00:54.052149+0000 mgr.gibba032.whopeo (mgr.24625) 6773 : cluster [DBG] pgmap v3851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:55.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:54 gibba011 bash[31774]: cluster 2021-03-10T16:00:54.052149+0000 mgr.gibba032.whopeo (mgr.24625) 6773 : cluster [DBG] pgmap v3851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:55.264 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:00:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:00:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:00:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:00:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:00:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:00:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:00:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:00:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:00:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:00:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:00:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:00:55.267 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:00:56.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:55 gibba032 bash[24004]: audit 2021-03-10T16:00:54.776173+0000 mgr.gibba032.whopeo (mgr.24625) 6774 : audit [DBG] from='client.58573 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.097 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:55 gibba032 bash[24004]: audit 2021-03-10T16:00:55.000237+0000 mgr.gibba032.whopeo (mgr.24625) 6775 : audit [DBG] from='client.58579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.097 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:55 gibba032 bash[24004]: audit 2021-03-10T16:00:55.264510+0000 mon.c (mon.1) 128 : audit [DBG] from='client.? 172.21.2.111:0/1968624703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:56.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[30791]: audit 2021-03-10T16:00:54.776173+0000 mgr.gibba032.whopeo (mgr.24625) 6774 : audit [DBG] from='client.58573 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.098 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[30791]: audit 2021-03-10T16:00:55.000237+0000 mgr.gibba032.whopeo (mgr.24625) 6775 : audit [DBG] from='client.58579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.099 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[30791]: audit 2021-03-10T16:00:55.264510+0000 mon.c (mon.1) 128 : audit [DBG] from='client.? 172.21.2.111:0/1968624703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:56.099 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[31774]: audit 2021-03-10T16:00:54.776173+0000 mgr.gibba032.whopeo (mgr.24625) 6774 : audit [DBG] from='client.58573 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.099 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[31774]: audit 2021-03-10T16:00:55.000237+0000 mgr.gibba032.whopeo (mgr.24625) 6775 : audit [DBG] from='client.58579 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:00:56.100 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:55 gibba011 bash[31774]: audit 2021-03-10T16:00:55.264510+0000 mon.c (mon.1) 128 : audit [DBG] from='client.? 172.21.2.111:0/1968624703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:00:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:56 gibba032 bash[24004]: cluster 2021-03-10T16:00:56.053625+0000 mgr.gibba032.whopeo (mgr.24625) 6776 : cluster [DBG] pgmap v3852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:57.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:56 gibba011 bash[30791]: cluster 2021-03-10T16:00:56.053625+0000 mgr.gibba032.whopeo (mgr.24625) 6776 : cluster [DBG] pgmap v3852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:57.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:56 gibba011 bash[31774]: cluster 2021-03-10T16:00:56.053625+0000 mgr.gibba032.whopeo (mgr.24625) 6776 : cluster [DBG] pgmap v3852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.802347+0000 mon.a (mon.0) 2473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:00:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.803011+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:00:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.858459+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:00:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.859208+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:00:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.859921+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:00:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:57 gibba032 bash[24004]: audit 2021-03-10T16:00:56.860636+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:00:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.802347+0000 mon.a (mon.0) 2473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:00:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.803011+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:00:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.858459+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:00:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.859208+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:00:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.859921+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:00:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[30791]: audit 2021-03-10T16:00:56.860636+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:00:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.802347+0000 mon.a (mon.0) 2473 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:00:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.803011+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:00:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.858459+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:00:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.859208+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:00:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.859921+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:00:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:57 gibba011 bash[31774]: audit 2021-03-10T16:00:56.860636+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:00:59.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:00:58 gibba032 bash[24004]: cluster 2021-03-10T16:00:58.054272+0000 mgr.gibba032.whopeo (mgr.24625) 6777 : cluster [DBG] pgmap v3853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:59.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:00:58 gibba011 bash[30791]: cluster 2021-03-10T16:00:58.054272+0000 mgr.gibba032.whopeo (mgr.24625) 6777 : cluster [DBG] pgmap v3853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:00:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:00:58 gibba011 bash[31774]: cluster 2021-03-10T16:00:58.054272+0000 mgr.gibba032.whopeo (mgr.24625) 6777 : cluster [DBG] pgmap v3853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:01 gibba032 bash[24004]: cluster 2021-03-10T16:01:00.054616+0000 mgr.gibba032.whopeo (mgr.24625) 6778 : cluster [DBG] pgmap v3854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:01 gibba011 bash[30791]: cluster 2021-03-10T16:01:00.054616+0000 mgr.gibba032.whopeo (mgr.24625) 6778 : cluster [DBG] pgmap v3854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:01.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:01 gibba011 bash[31774]: cluster 2021-03-10T16:01:00.054616+0000 mgr.gibba032.whopeo (mgr.24625) 6778 : cluster [DBG] pgmap v3854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:02 gibba032 bash[24004]: cluster 2021-03-10T16:01:02.056016+0000 mgr.gibba032.whopeo (mgr.24625) 6779 : cluster [DBG] pgmap v3855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:02 gibba011 bash[30791]: cluster 2021-03-10T16:01:02.056016+0000 mgr.gibba032.whopeo (mgr.24625) 6779 : cluster [DBG] pgmap v3855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:02 gibba011 bash[31774]: cluster 2021-03-10T16:01:02.056016+0000 mgr.gibba032.whopeo (mgr.24625) 6779 : cluster [DBG] pgmap v3855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:04 gibba032 bash[24004]: cephadm 2021-03-10T16:01:03.774742+0000 mgr.gibba032.whopeo (mgr.24625) 6780 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:04 gibba032 bash[24004]: cluster 2021-03-10T16:01:04.057426+0000 mgr.gibba032.whopeo (mgr.24625) 6781 : cluster [DBG] pgmap v3856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:04 gibba011 bash[30791]: cephadm 2021-03-10T16:01:03.774742+0000 mgr.gibba032.whopeo (mgr.24625) 6780 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:04 gibba011 bash[30791]: cluster 2021-03-10T16:01:04.057426+0000 mgr.gibba032.whopeo (mgr.24625) 6781 : cluster [DBG] pgmap v3856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:04 gibba011 bash[31774]: cephadm 2021-03-10T16:01:03.774742+0000 mgr.gibba032.whopeo (mgr.24625) 6780 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:04 gibba011 bash[31774]: cluster 2021-03-10T16:01:04.057426+0000 mgr.gibba032.whopeo (mgr.24625) 6781 : cluster [DBG] pgmap v3856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:06 gibba032 bash[24004]: cluster 2021-03-10T16:01:06.059022+0000 mgr.gibba032.whopeo (mgr.24625) 6782 : cluster [DBG] pgmap v3857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:06 gibba011 bash[30791]: cluster 2021-03-10T16:01:06.059022+0000 mgr.gibba032.whopeo (mgr.24625) 6782 : cluster [DBG] pgmap v3857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:06 gibba011 bash[31774]: cluster 2021-03-10T16:01:06.059022+0000 mgr.gibba032.whopeo (mgr.24625) 6782 : cluster [DBG] pgmap v3857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:09.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:09 gibba032 bash[24004]: cluster 2021-03-10T16:01:08.060695+0000 mgr.gibba032.whopeo (mgr.24625) 6783 : cluster [DBG] pgmap v3858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:09.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:09 gibba011 bash[30791]: cluster 2021-03-10T16:01:08.060695+0000 mgr.gibba032.whopeo (mgr.24625) 6783 : cluster [DBG] pgmap v3858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:09.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:09 gibba011 bash[31774]: cluster 2021-03-10T16:01:08.060695+0000 mgr.gibba032.whopeo (mgr.24625) 6783 : cluster [DBG] pgmap v3858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:10 gibba032 bash[24004]: cluster 2021-03-10T16:01:10.061450+0000 mgr.gibba032.whopeo (mgr.24625) 6784 : cluster [DBG] pgmap v3859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:10 gibba011 bash[30791]: cluster 2021-03-10T16:01:10.061450+0000 mgr.gibba032.whopeo (mgr.24625) 6784 : cluster [DBG] pgmap v3859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:10 gibba011 bash[31774]: cluster 2021-03-10T16:01:10.061450+0000 mgr.gibba032.whopeo (mgr.24625) 6784 : cluster [DBG] pgmap v3859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:12 gibba032 bash[24004]: cluster 2021-03-10T16:01:12.062750+0000 mgr.gibba032.whopeo (mgr.24625) 6785 : cluster [DBG] pgmap v3860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:12 gibba011 bash[30791]: cluster 2021-03-10T16:01:12.062750+0000 mgr.gibba032.whopeo (mgr.24625) 6785 : cluster [DBG] pgmap v3860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:12 gibba011 bash[31774]: cluster 2021-03-10T16:01:12.062750+0000 mgr.gibba032.whopeo (mgr.24625) 6785 : cluster [DBG] pgmap v3860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:14 gibba032 bash[24004]: cluster 2021-03-10T16:01:14.064073+0000 mgr.gibba032.whopeo (mgr.24625) 6786 : cluster [DBG] pgmap v3861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:14 gibba011 bash[30791]: cluster 2021-03-10T16:01:14.064073+0000 mgr.gibba032.whopeo (mgr.24625) 6786 : cluster [DBG] pgmap v3861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:14 gibba011 bash[31774]: cluster 2021-03-10T16:01:14.064073+0000 mgr.gibba032.whopeo (mgr.24625) 6786 : cluster [DBG] pgmap v3861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:16 gibba032 bash[24004]: cluster 2021-03-10T16:01:16.065571+0000 mgr.gibba032.whopeo (mgr.24625) 6787 : cluster [DBG] pgmap v3862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:16 gibba011 bash[30791]: cluster 2021-03-10T16:01:16.065571+0000 mgr.gibba032.whopeo (mgr.24625) 6787 : cluster [DBG] pgmap v3862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:16 gibba011 bash[31774]: cluster 2021-03-10T16:01:16.065571+0000 mgr.gibba032.whopeo (mgr.24625) 6787 : cluster [DBG] pgmap v3862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:18 gibba032 bash[24004]: cluster 2021-03-10T16:01:18.066493+0000 mgr.gibba032.whopeo (mgr.24625) 6788 : cluster [DBG] pgmap v3863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:18 gibba011 bash[30791]: cluster 2021-03-10T16:01:18.066493+0000 mgr.gibba032.whopeo (mgr.24625) 6788 : cluster [DBG] pgmap v3863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:18 gibba011 bash[31774]: cluster 2021-03-10T16:01:18.066493+0000 mgr.gibba032.whopeo (mgr.24625) 6788 : cluster [DBG] pgmap v3863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:20.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:19 gibba032 bash[24004]: cephadm 2021-03-10T16:01:18.790780+0000 mgr.gibba032.whopeo (mgr.24625) 6789 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:20.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:19 gibba011 bash[31774]: cephadm 2021-03-10T16:01:18.790780+0000 mgr.gibba032.whopeo (mgr.24625) 6789 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:19 gibba011 bash[30791]: cephadm 2021-03-10T16:01:18.790780+0000 mgr.gibba032.whopeo (mgr.24625) 6789 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:20 gibba032 bash[24004]: cluster 2021-03-10T16:01:20.067468+0000 mgr.gibba032.whopeo (mgr.24625) 6790 : cluster [DBG] pgmap v3864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:20 gibba011 bash[30791]: cluster 2021-03-10T16:01:20.067468+0000 mgr.gibba032.whopeo (mgr.24625) 6790 : cluster [DBG] pgmap v3864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:21.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:20 gibba011 bash[31774]: cluster 2021-03-10T16:01:20.067468+0000 mgr.gibba032.whopeo (mgr.24625) 6790 : cluster [DBG] pgmap v3864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:22 gibba032 bash[24004]: cluster 2021-03-10T16:01:22.068979+0000 mgr.gibba032.whopeo (mgr.24625) 6791 : cluster [DBG] pgmap v3865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:22 gibba011 bash[30791]: cluster 2021-03-10T16:01:22.068979+0000 mgr.gibba032.whopeo (mgr.24625) 6791 : cluster [DBG] pgmap v3865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:22 gibba011 bash[31774]: cluster 2021-03-10T16:01:22.068979+0000 mgr.gibba032.whopeo (mgr.24625) 6791 : cluster [DBG] pgmap v3865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:24 gibba032 bash[24004]: cluster 2021-03-10T16:01:24.070593+0000 mgr.gibba032.whopeo (mgr.24625) 6792 : cluster [DBG] pgmap v3866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:25.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:24 gibba011 bash[30791]: cluster 2021-03-10T16:01:24.070593+0000 mgr.gibba032.whopeo (mgr.24625) 6792 : cluster [DBG] pgmap v3866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:25.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:24 gibba011 bash[31774]: cluster 2021-03-10T16:01:24.070593+0000 mgr.gibba032.whopeo (mgr.24625) 6792 : cluster [DBG] pgmap v3866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:25.503 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:01:25.731 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:01:25.732 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 101s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:01:25.732 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 37s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:01:25.732 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:01:25.732 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:01:25.733 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:01:25.733 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:01:25.733 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:01:25.733 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:01:25.733 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 101s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:01:25.734 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 37s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:01:25.734 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:01:25.734 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:01:25.734 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:01:25.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:01:25.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:01:25.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:01:25.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:01:25.735 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 37s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:01:25.736 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 37s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:01:25.736 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 101s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:01:25.979 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:01:25.979 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:01:25.979 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:01:25.979 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:25.980 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:01:25.980 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:01:25.980 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:25.980 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:01:25.981 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:01:25.981 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:25.981 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:01:25.981 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:01:25.981 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:01:25.982 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:01:25.982 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:01:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:26 gibba011 bash[30791]: audit 2021-03-10T16:01:25.979251+0000 mon.c (mon.1) 129 : audit [DBG] from='client.? 172.21.2.111:0/3983273162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:26 gibba011 bash[31774]: audit 2021-03-10T16:01:25.979251+0000 mon.c (mon.1) 129 : audit [DBG] from='client.? 172.21.2.111:0/3983273162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:26 gibba032 bash[24004]: audit 2021-03-10T16:01:25.979251+0000 mon.c (mon.1) 129 : audit [DBG] from='client.? 172.21.2.111:0/3983273162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[30791]: audit 2021-03-10T16:01:25.493640+0000 mgr.gibba032.whopeo (mgr.24625) 6793 : audit [DBG] from='client.58591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[30791]: audit 2021-03-10T16:01:25.717034+0000 mgr.gibba032.whopeo (mgr.24625) 6794 : audit [DBG] from='client.58597 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[30791]: cluster 2021-03-10T16:01:26.071255+0000 mgr.gibba032.whopeo (mgr.24625) 6795 : cluster [DBG] pgmap v3867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[31774]: audit 2021-03-10T16:01:25.493640+0000 mgr.gibba032.whopeo (mgr.24625) 6793 : audit [DBG] from='client.58591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[31774]: audit 2021-03-10T16:01:25.717034+0000 mgr.gibba032.whopeo (mgr.24625) 6794 : audit [DBG] from='client.58597 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:27 gibba011 bash[31774]: cluster 2021-03-10T16:01:26.071255+0000 mgr.gibba032.whopeo (mgr.24625) 6795 : cluster [DBG] pgmap v3867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:27 gibba032 bash[24004]: audit 2021-03-10T16:01:25.493640+0000 mgr.gibba032.whopeo (mgr.24625) 6793 : audit [DBG] from='client.58591 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:27 gibba032 bash[24004]: audit 2021-03-10T16:01:25.717034+0000 mgr.gibba032.whopeo (mgr.24625) 6794 : audit [DBG] from='client.58597 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:27 gibba032 bash[24004]: cluster 2021-03-10T16:01:26.071255+0000 mgr.gibba032.whopeo (mgr.24625) 6795 : cluster [DBG] pgmap v3867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:28 gibba032 bash[24004]: cluster 2021-03-10T16:01:28.072063+0000 mgr.gibba032.whopeo (mgr.24625) 6796 : cluster [DBG] pgmap v3868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:28 gibba011 bash[30791]: cluster 2021-03-10T16:01:28.072063+0000 mgr.gibba032.whopeo (mgr.24625) 6796 : cluster [DBG] pgmap v3868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:28 gibba011 bash[31774]: cluster 2021-03-10T16:01:28.072063+0000 mgr.gibba032.whopeo (mgr.24625) 6796 : cluster [DBG] pgmap v3868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:30 gibba032 bash[24004]: cluster 2021-03-10T16:01:30.073052+0000 mgr.gibba032.whopeo (mgr.24625) 6797 : cluster [DBG] pgmap v3869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:30 gibba011 bash[30791]: cluster 2021-03-10T16:01:30.073052+0000 mgr.gibba032.whopeo (mgr.24625) 6797 : cluster [DBG] pgmap v3869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:30 gibba011 bash[31774]: cluster 2021-03-10T16:01:30.073052+0000 mgr.gibba032.whopeo (mgr.24625) 6797 : cluster [DBG] pgmap v3869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:32 gibba032 bash[24004]: cluster 2021-03-10T16:01:32.074376+0000 mgr.gibba032.whopeo (mgr.24625) 6798 : cluster [DBG] pgmap v3870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:32 gibba011 bash[30791]: cluster 2021-03-10T16:01:32.074376+0000 mgr.gibba032.whopeo (mgr.24625) 6798 : cluster [DBG] pgmap v3870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:32 gibba011 bash[31774]: cluster 2021-03-10T16:01:32.074376+0000 mgr.gibba032.whopeo (mgr.24625) 6798 : cluster [DBG] pgmap v3870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:34 gibba032 bash[24004]: cephadm 2021-03-10T16:01:33.806811+0000 mgr.gibba032.whopeo (mgr.24625) 6799 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:34 gibba032 bash[24004]: cluster 2021-03-10T16:01:34.075701+0000 mgr.gibba032.whopeo (mgr.24625) 6800 : cluster [DBG] pgmap v3871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:35.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:34 gibba011 bash[30791]: cephadm 2021-03-10T16:01:33.806811+0000 mgr.gibba032.whopeo (mgr.24625) 6799 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:34 gibba011 bash[30791]: cluster 2021-03-10T16:01:34.075701+0000 mgr.gibba032.whopeo (mgr.24625) 6800 : cluster [DBG] pgmap v3871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:34 gibba011 bash[31774]: cephadm 2021-03-10T16:01:33.806811+0000 mgr.gibba032.whopeo (mgr.24625) 6799 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:34 gibba011 bash[31774]: cluster 2021-03-10T16:01:34.075701+0000 mgr.gibba032.whopeo (mgr.24625) 6800 : cluster [DBG] pgmap v3871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:36 gibba032 bash[24004]: cluster 2021-03-10T16:01:36.077225+0000 mgr.gibba032.whopeo (mgr.24625) 6801 : cluster [DBG] pgmap v3872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:36 gibba011 bash[30791]: cluster 2021-03-10T16:01:36.077225+0000 mgr.gibba032.whopeo (mgr.24625) 6801 : cluster [DBG] pgmap v3872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:36 gibba011 bash[31774]: cluster 2021-03-10T16:01:36.077225+0000 mgr.gibba032.whopeo (mgr.24625) 6801 : cluster [DBG] pgmap v3872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:38 gibba032 bash[24004]: cluster 2021-03-10T16:01:38.078241+0000 mgr.gibba032.whopeo (mgr.24625) 6802 : cluster [DBG] pgmap v3873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:38 gibba011 bash[30791]: cluster 2021-03-10T16:01:38.078241+0000 mgr.gibba032.whopeo (mgr.24625) 6802 : cluster [DBG] pgmap v3873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:38 gibba011 bash[31774]: cluster 2021-03-10T16:01:38.078241+0000 mgr.gibba032.whopeo (mgr.24625) 6802 : cluster [DBG] pgmap v3873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:40 gibba032 bash[24004]: cluster 2021-03-10T16:01:40.079184+0000 mgr.gibba032.whopeo (mgr.24625) 6803 : cluster [DBG] pgmap v3874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:40 gibba011 bash[30791]: cluster 2021-03-10T16:01:40.079184+0000 mgr.gibba032.whopeo (mgr.24625) 6803 : cluster [DBG] pgmap v3874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:40 gibba011 bash[31774]: cluster 2021-03-10T16:01:40.079184+0000 mgr.gibba032.whopeo (mgr.24625) 6803 : cluster [DBG] pgmap v3874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:42 gibba032 bash[24004]: cluster 2021-03-10T16:01:42.080681+0000 mgr.gibba032.whopeo (mgr.24625) 6804 : cluster [DBG] pgmap v3875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:42 gibba011 bash[30791]: cluster 2021-03-10T16:01:42.080681+0000 mgr.gibba032.whopeo (mgr.24625) 6804 : cluster [DBG] pgmap v3875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:42.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:42 gibba011 bash[31774]: cluster 2021-03-10T16:01:42.080681+0000 mgr.gibba032.whopeo (mgr.24625) 6804 : cluster [DBG] pgmap v3875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:44 gibba032 bash[24004]: cluster 2021-03-10T16:01:44.081831+0000 mgr.gibba032.whopeo (mgr.24625) 6805 : cluster [DBG] pgmap v3876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:44 gibba011 bash[30791]: cluster 2021-03-10T16:01:44.081831+0000 mgr.gibba032.whopeo (mgr.24625) 6805 : cluster [DBG] pgmap v3876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:44 gibba011 bash[31774]: cluster 2021-03-10T16:01:44.081831+0000 mgr.gibba032.whopeo (mgr.24625) 6805 : cluster [DBG] pgmap v3876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:46 gibba032 bash[24004]: cluster 2021-03-10T16:01:46.083328+0000 mgr.gibba032.whopeo (mgr.24625) 6806 : cluster [DBG] pgmap v3877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:46 gibba011 bash[30791]: cluster 2021-03-10T16:01:46.083328+0000 mgr.gibba032.whopeo (mgr.24625) 6806 : cluster [DBG] pgmap v3877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:46 gibba011 bash[31774]: cluster 2021-03-10T16:01:46.083328+0000 mgr.gibba032.whopeo (mgr.24625) 6806 : cluster [DBG] pgmap v3877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:48 gibba032 bash[24004]: cluster 2021-03-10T16:01:48.083972+0000 mgr.gibba032.whopeo (mgr.24625) 6807 : cluster [DBG] pgmap v3878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:48 gibba011 bash[30791]: cluster 2021-03-10T16:01:48.083972+0000 mgr.gibba032.whopeo (mgr.24625) 6807 : cluster [DBG] pgmap v3878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:48 gibba011 bash[31774]: cluster 2021-03-10T16:01:48.083972+0000 mgr.gibba032.whopeo (mgr.24625) 6807 : cluster [DBG] pgmap v3878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:48.823590+0000 mgr.gibba032.whopeo (mgr.24625) 6808 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:01:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:48.825163+0000 mgr.gibba032.whopeo (mgr.24625) 6809 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:01:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.048032+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:01:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.048694+0000 mgr.gibba032.whopeo (mgr.24625) 6810 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:01:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.050502+0000 mgr.gibba032.whopeo (mgr.24625) 6811 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:01:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.051206+0000 mon.a (mon.0) 2480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:01:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.052171+0000 mgr.gibba032.whopeo (mgr.24625) 6812 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:01:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.052210+0000 mgr.gibba032.whopeo (mgr.24625) 6813 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:01:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.052970+0000 mon.a (mon.0) 2481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.053544+0000 mgr.gibba032.whopeo (mgr.24625) 6814 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:01:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.053582+0000 mgr.gibba032.whopeo (mgr.24625) 6815 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:01:50.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.054524+0000 mon.a (mon.0) 2482 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.054702+0000 mgr.gibba032.whopeo (mgr.24625) 6816 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:01:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.055660+0000 mon.a (mon.0) 2483 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.055979+0000 mgr.gibba032.whopeo (mgr.24625) 6817 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:01:50.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.056081+0000 mgr.gibba032.whopeo (mgr.24625) 6818 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:01:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.057414+0000 mgr.gibba032.whopeo (mgr.24625) 6819 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:01:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.057730+0000 mgr.gibba032.whopeo (mgr.24625) 6820 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:01:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.058483+0000 mon.a (mon.0) 2484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.058963+0000 mgr.gibba032.whopeo (mgr.24625) 6821 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:01:50.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.059023+0000 mgr.gibba032.whopeo (mgr.24625) 6822 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:01:50.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: cephadm 2021-03-10T16:01:49.059128+0000 mgr.gibba032.whopeo (mgr.24625) 6823 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:50.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[30791]: audit 2021-03-10T16:01:49.059929+0000 mon.a (mon.0) 2485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:48.823590+0000 mgr.gibba032.whopeo (mgr.24625) 6808 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:01:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:48.825163+0000 mgr.gibba032.whopeo (mgr.24625) 6809 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:01:50.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.048032+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:01:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.048694+0000 mgr.gibba032.whopeo (mgr.24625) 6810 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:01:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.050502+0000 mgr.gibba032.whopeo (mgr.24625) 6811 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:01:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.051206+0000 mon.a (mon.0) 2480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:01:50.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.052171+0000 mgr.gibba032.whopeo (mgr.24625) 6812 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:01:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.052210+0000 mgr.gibba032.whopeo (mgr.24625) 6813 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:01:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.052970+0000 mon.a (mon.0) 2481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.053544+0000 mgr.gibba032.whopeo (mgr.24625) 6814 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:01:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.053582+0000 mgr.gibba032.whopeo (mgr.24625) 6815 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:01:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.054524+0000 mon.a (mon.0) 2482 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.054702+0000 mgr.gibba032.whopeo (mgr.24625) 6816 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:01:50.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.055660+0000 mon.a (mon.0) 2483 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.055979+0000 mgr.gibba032.whopeo (mgr.24625) 6817 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:01:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.056081+0000 mgr.gibba032.whopeo (mgr.24625) 6818 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:01:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.057414+0000 mgr.gibba032.whopeo (mgr.24625) 6819 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:01:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.057730+0000 mgr.gibba032.whopeo (mgr.24625) 6820 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:01:50.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.058483+0000 mon.a (mon.0) 2484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.058963+0000 mgr.gibba032.whopeo (mgr.24625) 6821 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:01:50.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.059023+0000 mgr.gibba032.whopeo (mgr.24625) 6822 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:01:50.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: cephadm 2021-03-10T16:01:49.059128+0000 mgr.gibba032.whopeo (mgr.24625) 6823 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:50.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:50 gibba011 bash[31774]: audit 2021-03-10T16:01:49.059929+0000 mon.a (mon.0) 2485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:48.823590+0000 mgr.gibba032.whopeo (mgr.24625) 6808 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:01:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:48.825163+0000 mgr.gibba032.whopeo (mgr.24625) 6809 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:01:50.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.048032+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:01:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.048694+0000 mgr.gibba032.whopeo (mgr.24625) 6810 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:01:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.050502+0000 mgr.gibba032.whopeo (mgr.24625) 6811 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:01:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.051206+0000 mon.a (mon.0) 2480 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:01:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.052171+0000 mgr.gibba032.whopeo (mgr.24625) 6812 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:01:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.052210+0000 mgr.gibba032.whopeo (mgr.24625) 6813 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:01:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.052970+0000 mon.a (mon.0) 2481 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.053544+0000 mgr.gibba032.whopeo (mgr.24625) 6814 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:01:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.053582+0000 mgr.gibba032.whopeo (mgr.24625) 6815 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:01:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.054524+0000 mon.a (mon.0) 2482 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.054702+0000 mgr.gibba032.whopeo (mgr.24625) 6816 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:01:50.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.055660+0000 mon.a (mon.0) 2483 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.055979+0000 mgr.gibba032.whopeo (mgr.24625) 6817 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:01:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.056081+0000 mgr.gibba032.whopeo (mgr.24625) 6818 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:01:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.057414+0000 mgr.gibba032.whopeo (mgr.24625) 6819 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:01:50.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.057730+0000 mgr.gibba032.whopeo (mgr.24625) 6820 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:01:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.058483+0000 mon.a (mon.0) 2484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.058963+0000 mgr.gibba032.whopeo (mgr.24625) 6821 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:01:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.059023+0000 mgr.gibba032.whopeo (mgr.24625) 6822 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:01:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: cephadm 2021-03-10T16:01:49.059128+0000 mgr.gibba032.whopeo (mgr.24625) 6823 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:01:50.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:50 gibba032 bash[24004]: audit 2021-03-10T16:01:49.059929+0000 mon.a (mon.0) 2485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:51.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:51 gibba011 bash[30791]: cluster 2021-03-10T16:01:50.085144+0000 mgr.gibba032.whopeo (mgr.24625) 6824 : cluster [DBG] pgmap v3879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:51 gibba011 bash[31774]: cluster 2021-03-10T16:01:50.085144+0000 mgr.gibba032.whopeo (mgr.24625) 6824 : cluster [DBG] pgmap v3879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:51 gibba032 bash[24004]: cluster 2021-03-10T16:01:50.085144+0000 mgr.gibba032.whopeo (mgr.24625) 6824 : cluster [DBG] pgmap v3879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:52 gibba032 bash[24004]: cluster 2021-03-10T16:01:52.086771+0000 mgr.gibba032.whopeo (mgr.24625) 6825 : cluster [DBG] pgmap v3880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:52 gibba011 bash[30791]: cluster 2021-03-10T16:01:52.086771+0000 mgr.gibba032.whopeo (mgr.24625) 6825 : cluster [DBG] pgmap v3880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:52 gibba011 bash[31774]: cluster 2021-03-10T16:01:52.086771+0000 mgr.gibba032.whopeo (mgr.24625) 6825 : cluster [DBG] pgmap v3880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:54 gibba032 bash[24004]: cluster 2021-03-10T16:01:54.088162+0000 mgr.gibba032.whopeo (mgr.24625) 6826 : cluster [DBG] pgmap v3881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:55.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:54 gibba011 bash[30791]: cluster 2021-03-10T16:01:54.088162+0000 mgr.gibba032.whopeo (mgr.24625) 6826 : cluster [DBG] pgmap v3881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:54 gibba011 bash[31774]: cluster 2021-03-10T16:01:54.088162+0000 mgr.gibba032.whopeo (mgr.24625) 6826 : cluster [DBG] pgmap v3881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:56.223 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:01:56.451 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:01:56.451 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:01:56.452 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 67s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:01:56.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:01:56.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:01:56.452 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:01:56.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:01:56.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:01:56.453 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:01:56.453 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:01:56.454 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 67s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:01:56.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:01:56.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:01:56.454 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:01:56.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:01:56.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:01:56.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:01:56.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:01:56.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 67s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:01:56.456 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 67s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:01:56.456 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:01:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:56 gibba032 bash[24004]: cluster 2021-03-10T16:01:56.089623+0000 mgr.gibba032.whopeo (mgr.24625) 6827 : cluster [DBG] pgmap v3882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:56 gibba011 bash[30791]: cluster 2021-03-10T16:01:56.089623+0000 mgr.gibba032.whopeo (mgr.24625) 6827 : cluster [DBG] pgmap v3882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:56 gibba011 bash[31774]: cluster 2021-03-10T16:01:56.089623+0000 mgr.gibba032.whopeo (mgr.24625) 6827 : cluster [DBG] pgmap v3882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:56.698 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:01:56.699 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:01:56.699 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:01:56.699 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:56.699 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:01:56.700 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:01:56.700 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:56.700 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:01:56.700 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:01:56.701 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:01:56.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:01:56.701 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:01:56.701 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:01:56.701 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:01:56.702 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:01:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.213325+0000 mgr.gibba032.whopeo (mgr.24625) 6828 : audit [DBG] from='client.58609 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.437930+0000 mgr.gibba032.whopeo (mgr.24625) 6829 : audit [DBG] from='client.48633 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.699300+0000 mon.c (mon.1) 130 : audit [DBG] from='client.? 172.21.2.111:0/1066983866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.810244+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:01:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.810598+0000 mon.a (mon.0) 2487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:01:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.865704+0000 mon.a (mon.0) 2488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:01:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.866515+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.867235+0000 mon.a (mon.0) 2490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:57 gibba032 bash[24004]: audit 2021-03-10T16:01:56.867955+0000 mon.a (mon.0) 2491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.213325+0000 mgr.gibba032.whopeo (mgr.24625) 6828 : audit [DBG] from='client.58609 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.437930+0000 mgr.gibba032.whopeo (mgr.24625) 6829 : audit [DBG] from='client.48633 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.699300+0000 mon.c (mon.1) 130 : audit [DBG] from='client.? 172.21.2.111:0/1066983866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.810244+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:01:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.810598+0000 mon.a (mon.0) 2487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:01:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.865704+0000 mon.a (mon.0) 2488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:01:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.866515+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.867235+0000 mon.a (mon.0) 2490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[30791]: audit 2021-03-10T16:01:56.867955+0000 mon.a (mon.0) 2491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.213325+0000 mgr.gibba032.whopeo (mgr.24625) 6828 : audit [DBG] from='client.58609 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.437930+0000 mgr.gibba032.whopeo (mgr.24625) 6829 : audit [DBG] from='client.48633 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:01:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.699300+0000 mon.c (mon.1) 130 : audit [DBG] from='client.? 172.21.2.111:0/1066983866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:01:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.810244+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:01:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.810598+0000 mon.a (mon.0) 2487 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:01:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.865704+0000 mon.a (mon.0) 2488 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:01:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.866515+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.867235+0000 mon.a (mon.0) 2490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:01:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:57 gibba011 bash[31774]: audit 2021-03-10T16:01:56.867955+0000 mon.a (mon.0) 2491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:01:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:01:58 gibba032 bash[24004]: cluster 2021-03-10T16:01:58.090537+0000 mgr.gibba032.whopeo (mgr.24625) 6830 : cluster [DBG] pgmap v3883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:01:58 gibba011 bash[30791]: cluster 2021-03-10T16:01:58.090537+0000 mgr.gibba032.whopeo (mgr.24625) 6830 : cluster [DBG] pgmap v3883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:01:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:01:58 gibba011 bash[31774]: cluster 2021-03-10T16:01:58.090537+0000 mgr.gibba032.whopeo (mgr.24625) 6830 : cluster [DBG] pgmap v3883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:00 gibba032 bash[24004]: cluster 2021-03-10T16:02:00.091283+0000 mgr.gibba032.whopeo (mgr.24625) 6831 : cluster [DBG] pgmap v3884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:00 gibba011 bash[30791]: cluster 2021-03-10T16:02:00.091283+0000 mgr.gibba032.whopeo (mgr.24625) 6831 : cluster [DBG] pgmap v3884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:00 gibba011 bash[31774]: cluster 2021-03-10T16:02:00.091283+0000 mgr.gibba032.whopeo (mgr.24625) 6831 : cluster [DBG] pgmap v3884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:02 gibba032 bash[24004]: cluster 2021-03-10T16:02:02.092730+0000 mgr.gibba032.whopeo (mgr.24625) 6832 : cluster [DBG] pgmap v3885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:02 gibba011 bash[30791]: cluster 2021-03-10T16:02:02.092730+0000 mgr.gibba032.whopeo (mgr.24625) 6832 : cluster [DBG] pgmap v3885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:02 gibba011 bash[31774]: cluster 2021-03-10T16:02:02.092730+0000 mgr.gibba032.whopeo (mgr.24625) 6832 : cluster [DBG] pgmap v3885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:04 gibba032 bash[24004]: cephadm 2021-03-10T16:02:04.066199+0000 mgr.gibba032.whopeo (mgr.24625) 6833 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:04 gibba032 bash[24004]: cluster 2021-03-10T16:02:04.093962+0000 mgr.gibba032.whopeo (mgr.24625) 6834 : cluster [DBG] pgmap v3886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:04 gibba011 bash[30791]: cephadm 2021-03-10T16:02:04.066199+0000 mgr.gibba032.whopeo (mgr.24625) 6833 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:04 gibba011 bash[30791]: cluster 2021-03-10T16:02:04.093962+0000 mgr.gibba032.whopeo (mgr.24625) 6834 : cluster [DBG] pgmap v3886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:04 gibba011 bash[31774]: cephadm 2021-03-10T16:02:04.066199+0000 mgr.gibba032.whopeo (mgr.24625) 6833 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:04 gibba011 bash[31774]: cluster 2021-03-10T16:02:04.093962+0000 mgr.gibba032.whopeo (mgr.24625) 6834 : cluster [DBG] pgmap v3886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:06 gibba032 bash[24004]: cluster 2021-03-10T16:02:06.095535+0000 mgr.gibba032.whopeo (mgr.24625) 6835 : cluster [DBG] pgmap v3887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:06 gibba011 bash[30791]: cluster 2021-03-10T16:02:06.095535+0000 mgr.gibba032.whopeo (mgr.24625) 6835 : cluster [DBG] pgmap v3887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:06 gibba011 bash[31774]: cluster 2021-03-10T16:02:06.095535+0000 mgr.gibba032.whopeo (mgr.24625) 6835 : cluster [DBG] pgmap v3887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:08 gibba032 bash[24004]: cluster 2021-03-10T16:02:08.096494+0000 mgr.gibba032.whopeo (mgr.24625) 6836 : cluster [DBG] pgmap v3888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:08 gibba011 bash[30791]: cluster 2021-03-10T16:02:08.096494+0000 mgr.gibba032.whopeo (mgr.24625) 6836 : cluster [DBG] pgmap v3888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:08 gibba011 bash[31774]: cluster 2021-03-10T16:02:08.096494+0000 mgr.gibba032.whopeo (mgr.24625) 6836 : cluster [DBG] pgmap v3888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:10 gibba032 bash[24004]: cluster 2021-03-10T16:02:10.097325+0000 mgr.gibba032.whopeo (mgr.24625) 6837 : cluster [DBG] pgmap v3889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:10 gibba011 bash[30791]: cluster 2021-03-10T16:02:10.097325+0000 mgr.gibba032.whopeo (mgr.24625) 6837 : cluster [DBG] pgmap v3889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:10 gibba011 bash[31774]: cluster 2021-03-10T16:02:10.097325+0000 mgr.gibba032.whopeo (mgr.24625) 6837 : cluster [DBG] pgmap v3889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:12 gibba032 bash[24004]: cluster 2021-03-10T16:02:12.099039+0000 mgr.gibba032.whopeo (mgr.24625) 6838 : cluster [DBG] pgmap v3890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:12 gibba011 bash[30791]: cluster 2021-03-10T16:02:12.099039+0000 mgr.gibba032.whopeo (mgr.24625) 6838 : cluster [DBG] pgmap v3890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:12 gibba011 bash[31774]: cluster 2021-03-10T16:02:12.099039+0000 mgr.gibba032.whopeo (mgr.24625) 6838 : cluster [DBG] pgmap v3890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:14 gibba032 bash[24004]: cluster 2021-03-10T16:02:14.100535+0000 mgr.gibba032.whopeo (mgr.24625) 6839 : cluster [DBG] pgmap v3891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:14 gibba011 bash[30791]: cluster 2021-03-10T16:02:14.100535+0000 mgr.gibba032.whopeo (mgr.24625) 6839 : cluster [DBG] pgmap v3891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:14 gibba011 bash[31774]: cluster 2021-03-10T16:02:14.100535+0000 mgr.gibba032.whopeo (mgr.24625) 6839 : cluster [DBG] pgmap v3891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:16 gibba032 bash[24004]: cluster 2021-03-10T16:02:16.101988+0000 mgr.gibba032.whopeo (mgr.24625) 6840 : cluster [DBG] pgmap v3892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:16 gibba011 bash[30791]: cluster 2021-03-10T16:02:16.101988+0000 mgr.gibba032.whopeo (mgr.24625) 6840 : cluster [DBG] pgmap v3892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:16 gibba011 bash[31774]: cluster 2021-03-10T16:02:16.101988+0000 mgr.gibba032.whopeo (mgr.24625) 6840 : cluster [DBG] pgmap v3892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:18 gibba032 bash[24004]: cluster 2021-03-10T16:02:18.103321+0000 mgr.gibba032.whopeo (mgr.24625) 6841 : cluster [DBG] pgmap v3893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:18 gibba011 bash[30791]: cluster 2021-03-10T16:02:18.103321+0000 mgr.gibba032.whopeo (mgr.24625) 6841 : cluster [DBG] pgmap v3893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:18 gibba011 bash[31774]: cluster 2021-03-10T16:02:18.103321+0000 mgr.gibba032.whopeo (mgr.24625) 6841 : cluster [DBG] pgmap v3893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:20.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:19 gibba032 bash[24004]: cephadm 2021-03-10T16:02:19.082216+0000 mgr.gibba032.whopeo (mgr.24625) 6842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:19 gibba011 bash[30791]: cephadm 2021-03-10T16:02:19.082216+0000 mgr.gibba032.whopeo (mgr.24625) 6842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:19 gibba011 bash[31774]: cephadm 2021-03-10T16:02:19.082216+0000 mgr.gibba032.whopeo (mgr.24625) 6842 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:20 gibba032 bash[24004]: cluster 2021-03-10T16:02:20.104765+0000 mgr.gibba032.whopeo (mgr.24625) 6843 : cluster [DBG] pgmap v3894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:20 gibba011 bash[30791]: cluster 2021-03-10T16:02:20.104765+0000 mgr.gibba032.whopeo (mgr.24625) 6843 : cluster [DBG] pgmap v3894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:20 gibba011 bash[31774]: cluster 2021-03-10T16:02:20.104765+0000 mgr.gibba032.whopeo (mgr.24625) 6843 : cluster [DBG] pgmap v3894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:22 gibba032 bash[24004]: cluster 2021-03-10T16:02:22.106146+0000 mgr.gibba032.whopeo (mgr.24625) 6844 : cluster [DBG] pgmap v3895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:22 gibba011 bash[30791]: cluster 2021-03-10T16:02:22.106146+0000 mgr.gibba032.whopeo (mgr.24625) 6844 : cluster [DBG] pgmap v3895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:22 gibba011 bash[31774]: cluster 2021-03-10T16:02:22.106146+0000 mgr.gibba032.whopeo (mgr.24625) 6844 : cluster [DBG] pgmap v3895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:24 gibba032 bash[24004]: cluster 2021-03-10T16:02:24.107253+0000 mgr.gibba032.whopeo (mgr.24625) 6845 : cluster [DBG] pgmap v3896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:25.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:24 gibba011 bash[30791]: cluster 2021-03-10T16:02:24.107253+0000 mgr.gibba032.whopeo (mgr.24625) 6845 : cluster [DBG] pgmap v3896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:24 gibba011 bash[31774]: cluster 2021-03-10T16:02:24.107253+0000 mgr.gibba032.whopeo (mgr.24625) 6845 : cluster [DBG] pgmap v3896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:26 gibba032 bash[24004]: cluster 2021-03-10T16:02:26.108812+0000 mgr.gibba032.whopeo (mgr.24625) 6846 : cluster [DBG] pgmap v3897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:26 gibba011 bash[30791]: cluster 2021-03-10T16:02:26.108812+0000 mgr.gibba032.whopeo (mgr.24625) 6846 : cluster [DBG] pgmap v3897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:26 gibba011 bash[31774]: cluster 2021-03-10T16:02:26.108812+0000 mgr.gibba032.whopeo (mgr.24625) 6846 : cluster [DBG] pgmap v3897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:26.939 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:02:27.166 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:02:27.167 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:02:27.167 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 98s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:02:27.167 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:02:27.167 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:02:27.168 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:02:27.168 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:02:27.168 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:02:27.168 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:02:27.169 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:02:27.169 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 98s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:02:27.169 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:02:27.169 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:02:27.170 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:02:27.170 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:02:27.170 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:02:27.170 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:02:27.170 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:02:27.171 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 98s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:02:27.171 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 98s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:02:27.171 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:02:27.417 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:02:27.418 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:02:27.418 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:02:27.418 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:27.418 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:02:27.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:02:27.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:27.419 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:02:27.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:02:27.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:27.420 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:02:27.420 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:02:27.420 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:02:27.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:02:27.421 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:02:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:27 gibba032 bash[24004]: audit 2021-03-10T16:02:26.928909+0000 mgr.gibba032.whopeo (mgr.24625) 6847 : audit [DBG] from='client.48645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:27.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:27 gibba011 bash[30791]: audit 2021-03-10T16:02:26.928909+0000 mgr.gibba032.whopeo (mgr.24625) 6847 : audit [DBG] from='client.48645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:27.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:27 gibba011 bash[31774]: audit 2021-03-10T16:02:26.928909+0000 mgr.gibba032.whopeo (mgr.24625) 6847 : audit [DBG] from='client.48645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:28 gibba032 bash[24004]: audit 2021-03-10T16:02:27.153349+0000 mgr.gibba032.whopeo (mgr.24625) 6848 : audit [DBG] from='client.48651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:28 gibba032 bash[24004]: audit 2021-03-10T16:02:27.418115+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.2.111:0/1354297408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:28 gibba032 bash[24004]: cluster 2021-03-10T16:02:28.109718+0000 mgr.gibba032.whopeo (mgr.24625) 6849 : cluster [DBG] pgmap v3898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[30791]: audit 2021-03-10T16:02:27.153349+0000 mgr.gibba032.whopeo (mgr.24625) 6848 : audit [DBG] from='client.48651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[30791]: audit 2021-03-10T16:02:27.418115+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.2.111:0/1354297408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[30791]: cluster 2021-03-10T16:02:28.109718+0000 mgr.gibba032.whopeo (mgr.24625) 6849 : cluster [DBG] pgmap v3898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[31774]: audit 2021-03-10T16:02:27.153349+0000 mgr.gibba032.whopeo (mgr.24625) 6848 : audit [DBG] from='client.48651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[31774]: audit 2021-03-10T16:02:27.418115+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.2.111:0/1354297408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:28 gibba011 bash[31774]: cluster 2021-03-10T16:02:28.109718+0000 mgr.gibba032.whopeo (mgr.24625) 6849 : cluster [DBG] pgmap v3898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:30 gibba032 bash[24004]: cluster 2021-03-10T16:02:30.111182+0000 mgr.gibba032.whopeo (mgr.24625) 6850 : cluster [DBG] pgmap v3899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:30 gibba011 bash[30791]: cluster 2021-03-10T16:02:30.111182+0000 mgr.gibba032.whopeo (mgr.24625) 6850 : cluster [DBG] pgmap v3899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:30 gibba011 bash[31774]: cluster 2021-03-10T16:02:30.111182+0000 mgr.gibba032.whopeo (mgr.24625) 6850 : cluster [DBG] pgmap v3899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:32 gibba032 bash[24004]: cluster 2021-03-10T16:02:32.112872+0000 mgr.gibba032.whopeo (mgr.24625) 6851 : cluster [DBG] pgmap v3900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:32 gibba011 bash[30791]: cluster 2021-03-10T16:02:32.112872+0000 mgr.gibba032.whopeo (mgr.24625) 6851 : cluster [DBG] pgmap v3900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:32 gibba011 bash[31774]: cluster 2021-03-10T16:02:32.112872+0000 mgr.gibba032.whopeo (mgr.24625) 6851 : cluster [DBG] pgmap v3900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:34 gibba032 bash[24004]: cephadm 2021-03-10T16:02:34.085095+0000 mgr.gibba032.whopeo (mgr.24625) 6852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:34 gibba032 bash[24004]: cluster 2021-03-10T16:02:34.114261+0000 mgr.gibba032.whopeo (mgr.24625) 6853 : cluster [DBG] pgmap v3901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:35.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:34 gibba011 bash[30791]: cephadm 2021-03-10T16:02:34.085095+0000 mgr.gibba032.whopeo (mgr.24625) 6852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:34 gibba011 bash[30791]: cluster 2021-03-10T16:02:34.114261+0000 mgr.gibba032.whopeo (mgr.24625) 6853 : cluster [DBG] pgmap v3901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:34 gibba011 bash[31774]: cephadm 2021-03-10T16:02:34.085095+0000 mgr.gibba032.whopeo (mgr.24625) 6852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:34 gibba011 bash[31774]: cluster 2021-03-10T16:02:34.114261+0000 mgr.gibba032.whopeo (mgr.24625) 6853 : cluster [DBG] pgmap v3901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:36 gibba032 bash[24004]: cluster 2021-03-10T16:02:36.115768+0000 mgr.gibba032.whopeo (mgr.24625) 6854 : cluster [DBG] pgmap v3902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:36 gibba011 bash[30791]: cluster 2021-03-10T16:02:36.115768+0000 mgr.gibba032.whopeo (mgr.24625) 6854 : cluster [DBG] pgmap v3902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:36 gibba011 bash[31774]: cluster 2021-03-10T16:02:36.115768+0000 mgr.gibba032.whopeo (mgr.24625) 6854 : cluster [DBG] pgmap v3902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:38 gibba032 bash[24004]: cluster 2021-03-10T16:02:38.117521+0000 mgr.gibba032.whopeo (mgr.24625) 6855 : cluster [DBG] pgmap v3903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:38 gibba011 bash[30791]: cluster 2021-03-10T16:02:38.117521+0000 mgr.gibba032.whopeo (mgr.24625) 6855 : cluster [DBG] pgmap v3903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:38 gibba011 bash[31774]: cluster 2021-03-10T16:02:38.117521+0000 mgr.gibba032.whopeo (mgr.24625) 6855 : cluster [DBG] pgmap v3903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:40 gibba032 bash[24004]: cluster 2021-03-10T16:02:40.118971+0000 mgr.gibba032.whopeo (mgr.24625) 6856 : cluster [DBG] pgmap v3904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:40 gibba011 bash[30791]: cluster 2021-03-10T16:02:40.118971+0000 mgr.gibba032.whopeo (mgr.24625) 6856 : cluster [DBG] pgmap v3904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:40 gibba011 bash[31774]: cluster 2021-03-10T16:02:40.118971+0000 mgr.gibba032.whopeo (mgr.24625) 6856 : cluster [DBG] pgmap v3904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:42 gibba032 bash[24004]: cluster 2021-03-10T16:02:42.120639+0000 mgr.gibba032.whopeo (mgr.24625) 6857 : cluster [DBG] pgmap v3905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:42 gibba011 bash[30791]: cluster 2021-03-10T16:02:42.120639+0000 mgr.gibba032.whopeo (mgr.24625) 6857 : cluster [DBG] pgmap v3905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:42.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:42 gibba011 bash[31774]: cluster 2021-03-10T16:02:42.120639+0000 mgr.gibba032.whopeo (mgr.24625) 6857 : cluster [DBG] pgmap v3905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:44 gibba032 bash[24004]: cluster 2021-03-10T16:02:44.122136+0000 mgr.gibba032.whopeo (mgr.24625) 6858 : cluster [DBG] pgmap v3906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:45.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:44 gibba011 bash[30791]: cluster 2021-03-10T16:02:44.122136+0000 mgr.gibba032.whopeo (mgr.24625) 6858 : cluster [DBG] pgmap v3906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:45.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:44 gibba011 bash[31774]: cluster 2021-03-10T16:02:44.122136+0000 mgr.gibba032.whopeo (mgr.24625) 6858 : cluster [DBG] pgmap v3906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:46 gibba032 bash[24004]: cluster 2021-03-10T16:02:46.123585+0000 mgr.gibba032.whopeo (mgr.24625) 6859 : cluster [DBG] pgmap v3907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:46 gibba011 bash[30791]: cluster 2021-03-10T16:02:46.123585+0000 mgr.gibba032.whopeo (mgr.24625) 6859 : cluster [DBG] pgmap v3907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:46 gibba011 bash[31774]: cluster 2021-03-10T16:02:46.123585+0000 mgr.gibba032.whopeo (mgr.24625) 6859 : cluster [DBG] pgmap v3907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:48 gibba032 bash[24004]: cluster 2021-03-10T16:02:48.124693+0000 mgr.gibba032.whopeo (mgr.24625) 6860 : cluster [DBG] pgmap v3908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:48 gibba011 bash[30791]: cluster 2021-03-10T16:02:48.124693+0000 mgr.gibba032.whopeo (mgr.24625) 6860 : cluster [DBG] pgmap v3908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:48 gibba011 bash[31774]: cluster 2021-03-10T16:02:48.124693+0000 mgr.gibba032.whopeo (mgr.24625) 6860 : cluster [DBG] pgmap v3908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.093301+0000 mgr.gibba032.whopeo (mgr.24625) 6861 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:02:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.094886+0000 mgr.gibba032.whopeo (mgr.24625) 6862 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:02:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.308454+0000 mon.a (mon.0) 2493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:02:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.312051+0000 mon.a (mon.0) 2494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:02:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.313671+0000 mon.a (mon.0) 2495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.315186+0000 mon.a (mon.0) 2496 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.316359+0000 mon.a (mon.0) 2497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.318696+0000 mon.a (mon.0) 2498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:49 gibba032 bash[24004]: audit 2021-03-10T16:02:49.320367+0000 mon.a (mon.0) 2499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.093301+0000 mgr.gibba032.whopeo (mgr.24625) 6861 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:02:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.094886+0000 mgr.gibba032.whopeo (mgr.24625) 6862 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:02:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.308454+0000 mon.a (mon.0) 2493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:02:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.312051+0000 mon.a (mon.0) 2494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:02:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.313671+0000 mon.a (mon.0) 2495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.315186+0000 mon.a (mon.0) 2496 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.316359+0000 mon.a (mon.0) 2497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.318696+0000 mon.a (mon.0) 2498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[30791]: audit 2021-03-10T16:02:49.320367+0000 mon.a (mon.0) 2499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.093301+0000 mgr.gibba032.whopeo (mgr.24625) 6861 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:02:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.094886+0000 mgr.gibba032.whopeo (mgr.24625) 6862 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:02:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.308454+0000 mon.a (mon.0) 2493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:02:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.312051+0000 mon.a (mon.0) 2494 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:02:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.313671+0000 mon.a (mon.0) 2495 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.315186+0000 mon.a (mon.0) 2496 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.316359+0000 mon.a (mon.0) 2497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.318696+0000 mon.a (mon.0) 2498 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:50.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:49 gibba011 bash[31774]: audit 2021-03-10T16:02:49.320367+0000 mon.a (mon.0) 2499 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.309327+0000 mgr.gibba032.whopeo (mgr.24625) 6863 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:02:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.311140+0000 mgr.gibba032.whopeo (mgr.24625) 6864 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:02:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.312802+0000 mgr.gibba032.whopeo (mgr.24625) 6865 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:02:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.312838+0000 mgr.gibba032.whopeo (mgr.24625) 6866 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:02:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.314114+0000 mgr.gibba032.whopeo (mgr.24625) 6867 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:02:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.314152+0000 mgr.gibba032.whopeo (mgr.24625) 6868 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:02:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.315268+0000 mgr.gibba032.whopeo (mgr.24625) 6869 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:02:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.316262+0000 mgr.gibba032.whopeo (mgr.24625) 6870 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:02:51.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.316301+0000 mgr.gibba032.whopeo (mgr.24625) 6871 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:02:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.317652+0000 mgr.gibba032.whopeo (mgr.24625) 6872 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:02:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.317972+0000 mgr.gibba032.whopeo (mgr.24625) 6873 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:02:51.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.319596+0000 mgr.gibba032.whopeo (mgr.24625) 6874 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:02:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.319634+0000 mgr.gibba032.whopeo (mgr.24625) 6875 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:02:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cephadm 2021-03-10T16:02:49.319740+0000 mgr.gibba032.whopeo (mgr.24625) 6876 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:51.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:50 gibba032 bash[24004]: cluster 2021-03-10T16:02:50.126096+0000 mgr.gibba032.whopeo (mgr.24625) 6877 : cluster [DBG] pgmap v3909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.309327+0000 mgr.gibba032.whopeo (mgr.24625) 6863 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:02:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.311140+0000 mgr.gibba032.whopeo (mgr.24625) 6864 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:02:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.312802+0000 mgr.gibba032.whopeo (mgr.24625) 6865 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:02:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.312838+0000 mgr.gibba032.whopeo (mgr.24625) 6866 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:02:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.314114+0000 mgr.gibba032.whopeo (mgr.24625) 6867 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:02:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.314152+0000 mgr.gibba032.whopeo (mgr.24625) 6868 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:02:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.315268+0000 mgr.gibba032.whopeo (mgr.24625) 6869 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:02:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.316262+0000 mgr.gibba032.whopeo (mgr.24625) 6870 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:02:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.316301+0000 mgr.gibba032.whopeo (mgr.24625) 6871 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:02:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.317652+0000 mgr.gibba032.whopeo (mgr.24625) 6872 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:02:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.317972+0000 mgr.gibba032.whopeo (mgr.24625) 6873 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:02:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.319596+0000 mgr.gibba032.whopeo (mgr.24625) 6874 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:02:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.319634+0000 mgr.gibba032.whopeo (mgr.24625) 6875 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:02:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cephadm 2021-03-10T16:02:49.319740+0000 mgr.gibba032.whopeo (mgr.24625) 6876 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:51.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[30791]: cluster 2021-03-10T16:02:50.126096+0000 mgr.gibba032.whopeo (mgr.24625) 6877 : cluster [DBG] pgmap v3909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.309327+0000 mgr.gibba032.whopeo (mgr.24625) 6863 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:02:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.311140+0000 mgr.gibba032.whopeo (mgr.24625) 6864 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:02:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.312802+0000 mgr.gibba032.whopeo (mgr.24625) 6865 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:02:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.312838+0000 mgr.gibba032.whopeo (mgr.24625) 6866 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:02:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.314114+0000 mgr.gibba032.whopeo (mgr.24625) 6867 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:02:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.314152+0000 mgr.gibba032.whopeo (mgr.24625) 6868 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:02:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.315268+0000 mgr.gibba032.whopeo (mgr.24625) 6869 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:02:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.316262+0000 mgr.gibba032.whopeo (mgr.24625) 6870 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:02:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.316301+0000 mgr.gibba032.whopeo (mgr.24625) 6871 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:02:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.317652+0000 mgr.gibba032.whopeo (mgr.24625) 6872 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:02:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.317972+0000 mgr.gibba032.whopeo (mgr.24625) 6873 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:02:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.319596+0000 mgr.gibba032.whopeo (mgr.24625) 6874 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:02:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.319634+0000 mgr.gibba032.whopeo (mgr.24625) 6875 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:02:51.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cephadm 2021-03-10T16:02:49.319740+0000 mgr.gibba032.whopeo (mgr.24625) 6876 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:02:51.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:50 gibba011 bash[31774]: cluster 2021-03-10T16:02:50.126096+0000 mgr.gibba032.whopeo (mgr.24625) 6877 : cluster [DBG] pgmap v3909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:52 gibba032 bash[24004]: cluster 2021-03-10T16:02:52.127874+0000 mgr.gibba032.whopeo (mgr.24625) 6878 : cluster [DBG] pgmap v3910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:52 gibba011 bash[30791]: cluster 2021-03-10T16:02:52.127874+0000 mgr.gibba032.whopeo (mgr.24625) 6878 : cluster [DBG] pgmap v3910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:52 gibba011 bash[31774]: cluster 2021-03-10T16:02:52.127874+0000 mgr.gibba032.whopeo (mgr.24625) 6878 : cluster [DBG] pgmap v3910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:55.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:54 gibba032 bash[24004]: cluster 2021-03-10T16:02:54.129143+0000 mgr.gibba032.whopeo (mgr.24625) 6879 : cluster [DBG] pgmap v3911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:54 gibba011 bash[30791]: cluster 2021-03-10T16:02:54.129143+0000 mgr.gibba032.whopeo (mgr.24625) 6879 : cluster [DBG] pgmap v3911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:54 gibba011 bash[31774]: cluster 2021-03-10T16:02:54.129143+0000 mgr.gibba032.whopeo (mgr.24625) 6879 : cluster [DBG] pgmap v3911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:56 gibba032 bash[24004]: cluster 2021-03-10T16:02:56.130654+0000 mgr.gibba032.whopeo (mgr.24625) 6880 : cluster [DBG] pgmap v3912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:56 gibba011 bash[30791]: cluster 2021-03-10T16:02:56.130654+0000 mgr.gibba032.whopeo (mgr.24625) 6880 : cluster [DBG] pgmap v3912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:56 gibba011 bash[31774]: cluster 2021-03-10T16:02:56.130654+0000 mgr.gibba032.whopeo (mgr.24625) 6880 : cluster [DBG] pgmap v3912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.814392+0000 mon.a (mon.0) 2500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:02:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.815094+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:02:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.872786+0000 mon.a (mon.0) 2502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:02:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.873575+0000 mon.a (mon.0) 2503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:02:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.874359+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:57 gibba032 bash[24004]: audit 2021-03-10T16:02:56.875123+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.814392+0000 mon.a (mon.0) 2500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:02:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.815094+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:02:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.872786+0000 mon.a (mon.0) 2502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:02:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.873575+0000 mon.a (mon.0) 2503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:02:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.874359+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[30791]: audit 2021-03-10T16:02:56.875123+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.814392+0000 mon.a (mon.0) 2500 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:02:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.815094+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:02:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.872786+0000 mon.a (mon.0) 2502 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:02:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.873575+0000 mon.a (mon.0) 2503 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:02:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.874359+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:57 gibba011 bash[31774]: audit 2021-03-10T16:02:56.875123+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:02:57.659 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:02:57.885 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:02:57.886 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:02:57.886 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:02:57.887 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:02:57.887 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:02:57.887 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:02:57.887 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:02:57.887 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:02:57.888 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:02:57.888 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:02:57.888 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:02:57.888 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:02:57.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:02:57.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:02:57.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:02:57.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:02:57.889 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:02:57.890 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:02:57.890 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:02:57.890 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:02:57.890 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:02:58.170 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:02:58.171 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:02:58.171 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:02:58.171 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:58.172 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:02:58.172 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:02:58.172 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:58.172 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:02:58.173 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:02:58.173 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:02:58.173 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:02:58.173 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:02:58.173 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:02:58.174 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:02:58.174 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:02:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:58 gibba032 bash[24004]: audit 2021-03-10T16:02:57.649142+0000 mgr.gibba032.whopeo (mgr.24625) 6881 : audit [DBG] from='client.58645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:58 gibba032 bash[24004]: audit 2021-03-10T16:02:57.871711+0000 mgr.gibba032.whopeo (mgr.24625) 6882 : audit [DBG] from='client.58651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:58 gibba032 bash[24004]: cluster 2021-03-10T16:02:58.132152+0000 mgr.gibba032.whopeo (mgr.24625) 6883 : cluster [DBG] pgmap v3913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:02:58 gibba032 bash[24004]: audit 2021-03-10T16:02:58.171393+0000 mon.c (mon.1) 131 : audit [DBG] from='client.? 172.21.2.111:0/2241750206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[30791]: audit 2021-03-10T16:02:57.649142+0000 mgr.gibba032.whopeo (mgr.24625) 6881 : audit [DBG] from='client.58645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[30791]: audit 2021-03-10T16:02:57.871711+0000 mgr.gibba032.whopeo (mgr.24625) 6882 : audit [DBG] from='client.58651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[30791]: cluster 2021-03-10T16:02:58.132152+0000 mgr.gibba032.whopeo (mgr.24625) 6883 : cluster [DBG] pgmap v3913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[30791]: audit 2021-03-10T16:02:58.171393+0000 mon.c (mon.1) 131 : audit [DBG] from='client.? 172.21.2.111:0/2241750206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:02:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[31774]: audit 2021-03-10T16:02:57.649142+0000 mgr.gibba032.whopeo (mgr.24625) 6881 : audit [DBG] from='client.58645 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[31774]: audit 2021-03-10T16:02:57.871711+0000 mgr.gibba032.whopeo (mgr.24625) 6882 : audit [DBG] from='client.58651 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:02:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[31774]: cluster 2021-03-10T16:02:58.132152+0000 mgr.gibba032.whopeo (mgr.24625) 6883 : cluster [DBG] pgmap v3913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:02:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:02:58 gibba011 bash[31774]: audit 2021-03-10T16:02:58.171393+0000 mon.c (mon.1) 131 : audit [DBG] from='client.? 172.21.2.111:0/2241750206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:00 gibba032 bash[24004]: cluster 2021-03-10T16:03:00.133084+0000 mgr.gibba032.whopeo (mgr.24625) 6884 : cluster [DBG] pgmap v3914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:00 gibba011 bash[30791]: cluster 2021-03-10T16:03:00.133084+0000 mgr.gibba032.whopeo (mgr.24625) 6884 : cluster [DBG] pgmap v3914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:00 gibba011 bash[31774]: cluster 2021-03-10T16:03:00.133084+0000 mgr.gibba032.whopeo (mgr.24625) 6884 : cluster [DBG] pgmap v3914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:02 gibba032 bash[24004]: cluster 2021-03-10T16:03:02.134641+0000 mgr.gibba032.whopeo (mgr.24625) 6885 : cluster [DBG] pgmap v3915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:02 gibba011 bash[30791]: cluster 2021-03-10T16:03:02.134641+0000 mgr.gibba032.whopeo (mgr.24625) 6885 : cluster [DBG] pgmap v3915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:02 gibba011 bash[31774]: cluster 2021-03-10T16:03:02.134641+0000 mgr.gibba032.whopeo (mgr.24625) 6885 : cluster [DBG] pgmap v3915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:04 gibba032 bash[24004]: cluster 2021-03-10T16:03:04.136129+0000 mgr.gibba032.whopeo (mgr.24625) 6886 : cluster [DBG] pgmap v3916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:04 gibba011 bash[30791]: cluster 2021-03-10T16:03:04.136129+0000 mgr.gibba032.whopeo (mgr.24625) 6886 : cluster [DBG] pgmap v3916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:05.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:04 gibba011 bash[31774]: cluster 2021-03-10T16:03:04.136129+0000 mgr.gibba032.whopeo (mgr.24625) 6886 : cluster [DBG] pgmap v3916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:06.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:05 gibba032 bash[24004]: cephadm 2021-03-10T16:03:04.320941+0000 mgr.gibba032.whopeo (mgr.24625) 6887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:06.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:05 gibba011 bash[30791]: cephadm 2021-03-10T16:03:04.320941+0000 mgr.gibba032.whopeo (mgr.24625) 6887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:06.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:05 gibba011 bash[31774]: cephadm 2021-03-10T16:03:04.320941+0000 mgr.gibba032.whopeo (mgr.24625) 6887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:07.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:06 gibba032 bash[24004]: cluster 2021-03-10T16:03:06.137904+0000 mgr.gibba032.whopeo (mgr.24625) 6888 : cluster [DBG] pgmap v3917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:06 gibba011 bash[30791]: cluster 2021-03-10T16:03:06.137904+0000 mgr.gibba032.whopeo (mgr.24625) 6888 : cluster [DBG] pgmap v3917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:06 gibba011 bash[31774]: cluster 2021-03-10T16:03:06.137904+0000 mgr.gibba032.whopeo (mgr.24625) 6888 : cluster [DBG] pgmap v3917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:08 gibba032 bash[24004]: cluster 2021-03-10T16:03:08.139568+0000 mgr.gibba032.whopeo (mgr.24625) 6889 : cluster [DBG] pgmap v3918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:08 gibba011 bash[30791]: cluster 2021-03-10T16:03:08.139568+0000 mgr.gibba032.whopeo (mgr.24625) 6889 : cluster [DBG] pgmap v3918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:08 gibba011 bash[31774]: cluster 2021-03-10T16:03:08.139568+0000 mgr.gibba032.whopeo (mgr.24625) 6889 : cluster [DBG] pgmap v3918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:10 gibba032 bash[24004]: cluster 2021-03-10T16:03:10.140963+0000 mgr.gibba032.whopeo (mgr.24625) 6890 : cluster [DBG] pgmap v3919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:10 gibba011 bash[30791]: cluster 2021-03-10T16:03:10.140963+0000 mgr.gibba032.whopeo (mgr.24625) 6890 : cluster [DBG] pgmap v3919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:10 gibba011 bash[31774]: cluster 2021-03-10T16:03:10.140963+0000 mgr.gibba032.whopeo (mgr.24625) 6890 : cluster [DBG] pgmap v3919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:12 gibba032 bash[24004]: cluster 2021-03-10T16:03:12.142442+0000 mgr.gibba032.whopeo (mgr.24625) 6891 : cluster [DBG] pgmap v3920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:12 gibba011 bash[30791]: cluster 2021-03-10T16:03:12.142442+0000 mgr.gibba032.whopeo (mgr.24625) 6891 : cluster [DBG] pgmap v3920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:12 gibba011 bash[31774]: cluster 2021-03-10T16:03:12.142442+0000 mgr.gibba032.whopeo (mgr.24625) 6891 : cluster [DBG] pgmap v3920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:14 gibba032 bash[24004]: cluster 2021-03-10T16:03:14.143790+0000 mgr.gibba032.whopeo (mgr.24625) 6892 : cluster [DBG] pgmap v3921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:15.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:14 gibba011 bash[30791]: cluster 2021-03-10T16:03:14.143790+0000 mgr.gibba032.whopeo (mgr.24625) 6892 : cluster [DBG] pgmap v3921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:15.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:14 gibba011 bash[31774]: cluster 2021-03-10T16:03:14.143790+0000 mgr.gibba032.whopeo (mgr.24625) 6892 : cluster [DBG] pgmap v3921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:16 gibba032 bash[24004]: cluster 2021-03-10T16:03:16.145271+0000 mgr.gibba032.whopeo (mgr.24625) 6893 : cluster [DBG] pgmap v3922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:16 gibba011 bash[30791]: cluster 2021-03-10T16:03:16.145271+0000 mgr.gibba032.whopeo (mgr.24625) 6893 : cluster [DBG] pgmap v3922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:16 gibba011 bash[31774]: cluster 2021-03-10T16:03:16.145271+0000 mgr.gibba032.whopeo (mgr.24625) 6893 : cluster [DBG] pgmap v3922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:18 gibba032 bash[24004]: cluster 2021-03-10T16:03:18.146989+0000 mgr.gibba032.whopeo (mgr.24625) 6894 : cluster [DBG] pgmap v3923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:18 gibba011 bash[30791]: cluster 2021-03-10T16:03:18.146989+0000 mgr.gibba032.whopeo (mgr.24625) 6894 : cluster [DBG] pgmap v3923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:18 gibba011 bash[31774]: cluster 2021-03-10T16:03:18.146989+0000 mgr.gibba032.whopeo (mgr.24625) 6894 : cluster [DBG] pgmap v3923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:20 gibba032 bash[24004]: cephadm 2021-03-10T16:03:19.336831+0000 mgr.gibba032.whopeo (mgr.24625) 6895 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:20 gibba032 bash[24004]: cluster 2021-03-10T16:03:20.148481+0000 mgr.gibba032.whopeo (mgr.24625) 6896 : cluster [DBG] pgmap v3924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:20 gibba011 bash[30791]: cephadm 2021-03-10T16:03:19.336831+0000 mgr.gibba032.whopeo (mgr.24625) 6895 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:20 gibba011 bash[30791]: cluster 2021-03-10T16:03:20.148481+0000 mgr.gibba032.whopeo (mgr.24625) 6896 : cluster [DBG] pgmap v3924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:20 gibba011 bash[31774]: cephadm 2021-03-10T16:03:19.336831+0000 mgr.gibba032.whopeo (mgr.24625) 6895 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:20 gibba011 bash[31774]: cluster 2021-03-10T16:03:20.148481+0000 mgr.gibba032.whopeo (mgr.24625) 6896 : cluster [DBG] pgmap v3924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:22 gibba032 bash[24004]: cluster 2021-03-10T16:03:22.149918+0000 mgr.gibba032.whopeo (mgr.24625) 6897 : cluster [DBG] pgmap v3925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:22 gibba011 bash[30791]: cluster 2021-03-10T16:03:22.149918+0000 mgr.gibba032.whopeo (mgr.24625) 6897 : cluster [DBG] pgmap v3925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:22 gibba011 bash[31774]: cluster 2021-03-10T16:03:22.149918+0000 mgr.gibba032.whopeo (mgr.24625) 6897 : cluster [DBG] pgmap v3925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:24 gibba032 bash[24004]: cluster 2021-03-10T16:03:24.151385+0000 mgr.gibba032.whopeo (mgr.24625) 6898 : cluster [DBG] pgmap v3926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:25.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:24 gibba011 bash[30791]: cluster 2021-03-10T16:03:24.151385+0000 mgr.gibba032.whopeo (mgr.24625) 6898 : cluster [DBG] pgmap v3926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:24 gibba011 bash[31774]: cluster 2021-03-10T16:03:24.151385+0000 mgr.gibba032.whopeo (mgr.24625) 6898 : cluster [DBG] pgmap v3926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:26 gibba032 bash[24004]: cluster 2021-03-10T16:03:26.152893+0000 mgr.gibba032.whopeo (mgr.24625) 6899 : cluster [DBG] pgmap v3927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:26 gibba011 bash[30791]: cluster 2021-03-10T16:03:26.152893+0000 mgr.gibba032.whopeo (mgr.24625) 6899 : cluster [DBG] pgmap v3927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:26 gibba011 bash[31774]: cluster 2021-03-10T16:03:26.152893+0000 mgr.gibba032.whopeo (mgr.24625) 6899 : cluster [DBG] pgmap v3927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:28.417 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:03:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:28 gibba032 bash[24004]: cluster 2021-03-10T16:03:28.154420+0000 mgr.gibba032.whopeo (mgr.24625) 6900 : cluster [DBG] pgmap v3928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:28 gibba011 bash[30791]: cluster 2021-03-10T16:03:28.154420+0000 mgr.gibba032.whopeo (mgr.24625) 6900 : cluster [DBG] pgmap v3928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:28 gibba011 bash[31774]: cluster 2021-03-10T16:03:28.154420+0000 mgr.gibba032.whopeo (mgr.24625) 6900 : cluster [DBG] pgmap v3928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:28.644 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:03:28.644 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:03:28.645 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:03:28.645 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:03:28.645 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:03:28.645 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:03:28.645 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:03:28.646 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:03:28.646 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:03:28.646 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:03:28.646 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:03:28.646 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:03:28.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:03:28.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:03:28.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:03:28.647 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:03:28.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:03:28.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:03:28.648 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:03:28.648 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:03:28.648 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:03:28.893 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:03:28.893 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:03:28.893 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:03:28.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:28.894 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:03:28.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:03:28.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:28.894 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:03:28.895 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:03:28.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:28.895 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:03:28.895 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:03:28.896 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:03:28.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:03:28.896 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:03:30.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:29 gibba032 bash[24004]: audit 2021-03-10T16:03:28.407161+0000 mgr.gibba032.whopeo (mgr.24625) 6901 : audit [DBG] from='client.48681 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:29 gibba032 bash[24004]: audit 2021-03-10T16:03:28.630621+0000 mgr.gibba032.whopeo (mgr.24625) 6902 : audit [DBG] from='client.48687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:29 gibba032 bash[24004]: audit 2021-03-10T16:03:28.893779+0000 mon.c (mon.1) 132 : audit [DBG] from='client.? 172.21.2.111:0/1332626962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:30.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[30791]: audit 2021-03-10T16:03:28.407161+0000 mgr.gibba032.whopeo (mgr.24625) 6901 : audit [DBG] from='client.48681 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[30791]: audit 2021-03-10T16:03:28.630621+0000 mgr.gibba032.whopeo (mgr.24625) 6902 : audit [DBG] from='client.48687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[30791]: audit 2021-03-10T16:03:28.893779+0000 mon.c (mon.1) 132 : audit [DBG] from='client.? 172.21.2.111:0/1332626962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[31774]: audit 2021-03-10T16:03:28.407161+0000 mgr.gibba032.whopeo (mgr.24625) 6901 : audit [DBG] from='client.48681 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[31774]: audit 2021-03-10T16:03:28.630621+0000 mgr.gibba032.whopeo (mgr.24625) 6902 : audit [DBG] from='client.48687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:03:30.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:29 gibba011 bash[31774]: audit 2021-03-10T16:03:28.893779+0000 mon.c (mon.1) 132 : audit [DBG] from='client.? 172.21.2.111:0/1332626962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:30 gibba032 bash[24004]: cluster 2021-03-10T16:03:30.155994+0000 mgr.gibba032.whopeo (mgr.24625) 6903 : cluster [DBG] pgmap v3929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:30 gibba011 bash[30791]: cluster 2021-03-10T16:03:30.155994+0000 mgr.gibba032.whopeo (mgr.24625) 6903 : cluster [DBG] pgmap v3929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:31.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:30 gibba011 bash[31774]: cluster 2021-03-10T16:03:30.155994+0000 mgr.gibba032.whopeo (mgr.24625) 6903 : cluster [DBG] pgmap v3929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:32 gibba032 bash[24004]: cluster 2021-03-10T16:03:32.157445+0000 mgr.gibba032.whopeo (mgr.24625) 6904 : cluster [DBG] pgmap v3930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:32 gibba011 bash[30791]: cluster 2021-03-10T16:03:32.157445+0000 mgr.gibba032.whopeo (mgr.24625) 6904 : cluster [DBG] pgmap v3930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:32 gibba011 bash[31774]: cluster 2021-03-10T16:03:32.157445+0000 mgr.gibba032.whopeo (mgr.24625) 6904 : cluster [DBG] pgmap v3930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:34 gibba032 bash[24004]: cluster 2021-03-10T16:03:34.158837+0000 mgr.gibba032.whopeo (mgr.24625) 6905 : cluster [DBG] pgmap v3931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:35.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:34 gibba011 bash[30791]: cluster 2021-03-10T16:03:34.158837+0000 mgr.gibba032.whopeo (mgr.24625) 6905 : cluster [DBG] pgmap v3931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:34 gibba011 bash[31774]: cluster 2021-03-10T16:03:34.158837+0000 mgr.gibba032.whopeo (mgr.24625) 6905 : cluster [DBG] pgmap v3931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:36.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:35 gibba032 bash[24004]: cephadm 2021-03-10T16:03:34.352839+0000 mgr.gibba032.whopeo (mgr.24625) 6906 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:36.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:35 gibba011 bash[30791]: cephadm 2021-03-10T16:03:34.352839+0000 mgr.gibba032.whopeo (mgr.24625) 6906 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:36.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:35 gibba011 bash[31774]: cephadm 2021-03-10T16:03:34.352839+0000 mgr.gibba032.whopeo (mgr.24625) 6906 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:36 gibba032 bash[24004]: cluster 2021-03-10T16:03:36.160381+0000 mgr.gibba032.whopeo (mgr.24625) 6907 : cluster [DBG] pgmap v3932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:36 gibba011 bash[30791]: cluster 2021-03-10T16:03:36.160381+0000 mgr.gibba032.whopeo (mgr.24625) 6907 : cluster [DBG] pgmap v3932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:36 gibba011 bash[31774]: cluster 2021-03-10T16:03:36.160381+0000 mgr.gibba032.whopeo (mgr.24625) 6907 : cluster [DBG] pgmap v3932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:38 gibba032 bash[24004]: cluster 2021-03-10T16:03:38.162123+0000 mgr.gibba032.whopeo (mgr.24625) 6908 : cluster [DBG] pgmap v3933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:38 gibba011 bash[30791]: cluster 2021-03-10T16:03:38.162123+0000 mgr.gibba032.whopeo (mgr.24625) 6908 : cluster [DBG] pgmap v3933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:38 gibba011 bash[31774]: cluster 2021-03-10T16:03:38.162123+0000 mgr.gibba032.whopeo (mgr.24625) 6908 : cluster [DBG] pgmap v3933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:40 gibba032 bash[24004]: cluster 2021-03-10T16:03:40.163719+0000 mgr.gibba032.whopeo (mgr.24625) 6909 : cluster [DBG] pgmap v3934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:40 gibba011 bash[30791]: cluster 2021-03-10T16:03:40.163719+0000 mgr.gibba032.whopeo (mgr.24625) 6909 : cluster [DBG] pgmap v3934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:40 gibba011 bash[31774]: cluster 2021-03-10T16:03:40.163719+0000 mgr.gibba032.whopeo (mgr.24625) 6909 : cluster [DBG] pgmap v3934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:43 gibba032 bash[24004]: cluster 2021-03-10T16:03:42.165349+0000 mgr.gibba032.whopeo (mgr.24625) 6910 : cluster [DBG] pgmap v3935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:43.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:43 gibba011 bash[30791]: cluster 2021-03-10T16:03:42.165349+0000 mgr.gibba032.whopeo (mgr.24625) 6910 : cluster [DBG] pgmap v3935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:43.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:43 gibba011 bash[31774]: cluster 2021-03-10T16:03:42.165349+0000 mgr.gibba032.whopeo (mgr.24625) 6910 : cluster [DBG] pgmap v3935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:45 gibba032 bash[24004]: cluster 2021-03-10T16:03:44.166916+0000 mgr.gibba032.whopeo (mgr.24625) 6911 : cluster [DBG] pgmap v3936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:46.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:45 gibba011 bash[30791]: cluster 2021-03-10T16:03:44.166916+0000 mgr.gibba032.whopeo (mgr.24625) 6911 : cluster [DBG] pgmap v3936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:46.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:45 gibba011 bash[31774]: cluster 2021-03-10T16:03:44.166916+0000 mgr.gibba032.whopeo (mgr.24625) 6911 : cluster [DBG] pgmap v3936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:47 gibba032 bash[24004]: cluster 2021-03-10T16:03:46.168505+0000 mgr.gibba032.whopeo (mgr.24625) 6912 : cluster [DBG] pgmap v3937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:47 gibba011 bash[30791]: cluster 2021-03-10T16:03:46.168505+0000 mgr.gibba032.whopeo (mgr.24625) 6912 : cluster [DBG] pgmap v3937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:48.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:47 gibba011 bash[31774]: cluster 2021-03-10T16:03:46.168505+0000 mgr.gibba032.whopeo (mgr.24625) 6912 : cluster [DBG] pgmap v3937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: cluster 2021-03-10T16:03:48.170114+0000 mgr.gibba032.whopeo (mgr.24625) 6913 : cluster [DBG] pgmap v3938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.572991+0000 mon.a (mon.0) 2506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:03:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.576384+0000 mon.a (mon.0) 2507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:03:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.578176+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.580085+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.581192+0000 mon.a (mon.0) 2510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.583602+0000 mon.a (mon.0) 2511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:49 gibba032 bash[24004]: audit 2021-03-10T16:03:49.585259+0000 mon.a (mon.0) 2512 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: cluster 2021-03-10T16:03:48.170114+0000 mgr.gibba032.whopeo (mgr.24625) 6913 : cluster [DBG] pgmap v3938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.572991+0000 mon.a (mon.0) 2506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:03:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.576384+0000 mon.a (mon.0) 2507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:03:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.578176+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.580085+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.581192+0000 mon.a (mon.0) 2510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.583602+0000 mon.a (mon.0) 2511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[30791]: audit 2021-03-10T16:03:49.585259+0000 mon.a (mon.0) 2512 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: cluster 2021-03-10T16:03:48.170114+0000 mgr.gibba032.whopeo (mgr.24625) 6913 : cluster [DBG] pgmap v3938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.572991+0000 mon.a (mon.0) 2506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:03:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.576384+0000 mon.a (mon.0) 2507 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:03:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.578176+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.580085+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.581192+0000 mon.a (mon.0) 2510 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.583602+0000 mon.a (mon.0) 2511 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:50.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:49 gibba011 bash[31774]: audit 2021-03-10T16:03:49.585259+0000 mon.a (mon.0) 2512 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:03:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.369435+0000 mgr.gibba032.whopeo (mgr.24625) 6914 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:03:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.371008+0000 mgr.gibba032.whopeo (mgr.24625) 6915 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:03:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.573670+0000 mgr.gibba032.whopeo (mgr.24625) 6916 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:03:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.575811+0000 mgr.gibba032.whopeo (mgr.24625) 6917 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:03:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.577611+0000 mgr.gibba032.whopeo (mgr.24625) 6918 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:03:51.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.577653+0000 mgr.gibba032.whopeo (mgr.24625) 6919 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:03:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.579023+0000 mgr.gibba032.whopeo (mgr.24625) 6920 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:03:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.579063+0000 mgr.gibba032.whopeo (mgr.24625) 6921 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:03:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.580003+0000 mgr.gibba032.whopeo (mgr.24625) 6922 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:03:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.581053+0000 mgr.gibba032.whopeo (mgr.24625) 6923 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:03:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.581090+0000 mgr.gibba032.whopeo (mgr.24625) 6924 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:03:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.582575+0000 mgr.gibba032.whopeo (mgr.24625) 6925 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:03:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.582968+0000 mgr.gibba032.whopeo (mgr.24625) 6926 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:03:51.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.584168+0000 mgr.gibba032.whopeo (mgr.24625) 6927 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:03:51.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.584245+0000 mgr.gibba032.whopeo (mgr.24625) 6928 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:03:51.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[30791]: cephadm 2021-03-10T16:03:49.584354+0000 mgr.gibba032.whopeo (mgr.24625) 6929 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:51.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.369435+0000 mgr.gibba032.whopeo (mgr.24625) 6914 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:03:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.371008+0000 mgr.gibba032.whopeo (mgr.24625) 6915 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:03:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.573670+0000 mgr.gibba032.whopeo (mgr.24625) 6916 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:03:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.575811+0000 mgr.gibba032.whopeo (mgr.24625) 6917 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:03:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.577611+0000 mgr.gibba032.whopeo (mgr.24625) 6918 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:03:51.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.577653+0000 mgr.gibba032.whopeo (mgr.24625) 6919 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:03:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.579023+0000 mgr.gibba032.whopeo (mgr.24625) 6920 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:03:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.579063+0000 mgr.gibba032.whopeo (mgr.24625) 6921 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:03:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.580003+0000 mgr.gibba032.whopeo (mgr.24625) 6922 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:03:51.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.581053+0000 mgr.gibba032.whopeo (mgr.24625) 6923 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:03:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.581090+0000 mgr.gibba032.whopeo (mgr.24625) 6924 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:03:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.582575+0000 mgr.gibba032.whopeo (mgr.24625) 6925 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:03:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.582968+0000 mgr.gibba032.whopeo (mgr.24625) 6926 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:03:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.584168+0000 mgr.gibba032.whopeo (mgr.24625) 6927 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:03:51.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.584245+0000 mgr.gibba032.whopeo (mgr.24625) 6928 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:03:51.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:50 gibba011 bash[31774]: cephadm 2021-03-10T16:03:49.584354+0000 mgr.gibba032.whopeo (mgr.24625) 6929 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.369435+0000 mgr.gibba032.whopeo (mgr.24625) 6914 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:03:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.371008+0000 mgr.gibba032.whopeo (mgr.24625) 6915 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:03:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.573670+0000 mgr.gibba032.whopeo (mgr.24625) 6916 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:03:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.575811+0000 mgr.gibba032.whopeo (mgr.24625) 6917 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:03:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.577611+0000 mgr.gibba032.whopeo (mgr.24625) 6918 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:03:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.577653+0000 mgr.gibba032.whopeo (mgr.24625) 6919 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:03:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.579023+0000 mgr.gibba032.whopeo (mgr.24625) 6920 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:03:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.579063+0000 mgr.gibba032.whopeo (mgr.24625) 6921 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:03:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.580003+0000 mgr.gibba032.whopeo (mgr.24625) 6922 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:03:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.581053+0000 mgr.gibba032.whopeo (mgr.24625) 6923 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:03:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.581090+0000 mgr.gibba032.whopeo (mgr.24625) 6924 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:03:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.582575+0000 mgr.gibba032.whopeo (mgr.24625) 6925 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:03:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.582968+0000 mgr.gibba032.whopeo (mgr.24625) 6926 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:03:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.584168+0000 mgr.gibba032.whopeo (mgr.24625) 6927 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:03:51.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.584245+0000 mgr.gibba032.whopeo (mgr.24625) 6928 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:03:51.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:50 gibba032 bash[24004]: cephadm 2021-03-10T16:03:49.584354+0000 mgr.gibba032.whopeo (mgr.24625) 6929 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:03:52.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:51 gibba011 bash[30791]: cluster 2021-03-10T16:03:50.171661+0000 mgr.gibba032.whopeo (mgr.24625) 6930 : cluster [DBG] pgmap v3939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:51 gibba011 bash[31774]: cluster 2021-03-10T16:03:50.171661+0000 mgr.gibba032.whopeo (mgr.24625) 6930 : cluster [DBG] pgmap v3939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:51 gibba032 bash[24004]: cluster 2021-03-10T16:03:50.171661+0000 mgr.gibba032.whopeo (mgr.24625) 6930 : cluster [DBG] pgmap v3939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:54.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:53 gibba011 bash[30791]: cluster 2021-03-10T16:03:52.173283+0000 mgr.gibba032.whopeo (mgr.24625) 6931 : cluster [DBG] pgmap v3940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:54.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:53 gibba011 bash[31774]: cluster 2021-03-10T16:03:52.173283+0000 mgr.gibba032.whopeo (mgr.24625) 6931 : cluster [DBG] pgmap v3940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:53 gibba032 bash[24004]: cluster 2021-03-10T16:03:52.173283+0000 mgr.gibba032.whopeo (mgr.24625) 6931 : cluster [DBG] pgmap v3940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:55 gibba011 bash[30791]: cluster 2021-03-10T16:03:54.174785+0000 mgr.gibba032.whopeo (mgr.24625) 6932 : cluster [DBG] pgmap v3941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:55 gibba011 bash[31774]: cluster 2021-03-10T16:03:54.174785+0000 mgr.gibba032.whopeo (mgr.24625) 6932 : cluster [DBG] pgmap v3941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:55 gibba032 bash[24004]: cluster 2021-03-10T16:03:54.174785+0000 mgr.gibba032.whopeo (mgr.24625) 6932 : cluster [DBG] pgmap v3941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: cluster 2021-03-10T16:03:56.176303+0000 mgr.gibba032.whopeo (mgr.24625) 6933 : cluster [DBG] pgmap v3942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.817807+0000 mon.a (mon.0) 2513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:03:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.817963+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:03:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.881620+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:03:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.882455+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:03:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.883160+0000 mon.a (mon.0) 2517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:03:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[30791]: audit 2021-03-10T16:03:56.883932+0000 mon.a (mon.0) 2518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:03:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: cluster 2021-03-10T16:03:56.176303+0000 mgr.gibba032.whopeo (mgr.24625) 6933 : cluster [DBG] pgmap v3942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.817807+0000 mon.a (mon.0) 2513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:03:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.817963+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:03:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.881620+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:03:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.882455+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:03:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.883160+0000 mon.a (mon.0) 2517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:03:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:57 gibba011 bash[31774]: audit 2021-03-10T16:03:56.883932+0000 mon.a (mon.0) 2518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:03:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: cluster 2021-03-10T16:03:56.176303+0000 mgr.gibba032.whopeo (mgr.24625) 6933 : cluster [DBG] pgmap v3942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:03:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.817807+0000 mon.a (mon.0) 2513 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.817963+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.881620+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.882455+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.883160+0000 mon.a (mon.0) 2517 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:03:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:57 gibba032 bash[24004]: audit 2021-03-10T16:03:56.883932+0000 mon.a (mon.0) 2518 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:03:59.135 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:03:59.360 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:03:59.360 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:03:59.360 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:03:59.361 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:03:59.361 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:03:59.361 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:03:59.361 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:03:59.362 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:03:59.362 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:03:59.362 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:03:59.362 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:03:59.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:03:59.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:03:59.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:03:59.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:03:59.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:03:59.364 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:03:59.364 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:03:59.364 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:03:59.364 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:03:59.365 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:03:59.609 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:03:59.609 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:03:59.609 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:03:59.610 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:59.610 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:03:59.610 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:03:59.610 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:59.611 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:03:59.611 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:03:59.611 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:03:59.611 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:03:59.612 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:03:59.612 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:03:59.612 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:03:59.612 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:04:00.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[30791]: cluster 2021-03-10T16:03:58.177780+0000 mgr.gibba032.whopeo (mgr.24625) 6934 : cluster [DBG] pgmap v3943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:00.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[30791]: audit 2021-03-10T16:03:59.124821+0000 mgr.gibba032.whopeo (mgr.24625) 6935 : audit [DBG] from='client.48699 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:00.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[30791]: audit 2021-03-10T16:03:59.609864+0000 mon.a (mon.0) 2519 : audit [DBG] from='client.? 172.21.2.111:0/520744320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[31774]: cluster 2021-03-10T16:03:58.177780+0000 mgr.gibba032.whopeo (mgr.24625) 6934 : cluster [DBG] pgmap v3943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[31774]: audit 2021-03-10T16:03:59.124821+0000 mgr.gibba032.whopeo (mgr.24625) 6935 : audit [DBG] from='client.48699 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:03:59 gibba011 bash[31774]: audit 2021-03-10T16:03:59.609864+0000 mon.a (mon.0) 2519 : audit [DBG] from='client.? 172.21.2.111:0/520744320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:59 gibba032 bash[24004]: cluster 2021-03-10T16:03:58.177780+0000 mgr.gibba032.whopeo (mgr.24625) 6934 : cluster [DBG] pgmap v3943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:59 gibba032 bash[24004]: audit 2021-03-10T16:03:59.124821+0000 mgr.gibba032.whopeo (mgr.24625) 6935 : audit [DBG] from='client.48699 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:00.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:03:59 gibba032 bash[24004]: audit 2021-03-10T16:03:59.609864+0000 mon.a (mon.0) 2519 : audit [DBG] from='client.? 172.21.2.111:0/520744320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:01.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:00 gibba011 bash[30791]: audit 2021-03-10T16:03:59.346497+0000 mgr.gibba032.whopeo (mgr.24625) 6936 : audit [DBG] from='client.58687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:00 gibba011 bash[31774]: audit 2021-03-10T16:03:59.346497+0000 mgr.gibba032.whopeo (mgr.24625) 6936 : audit [DBG] from='client.58687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:00 gibba032 bash[24004]: audit 2021-03-10T16:03:59.346497+0000 mgr.gibba032.whopeo (mgr.24625) 6936 : audit [DBG] from='client.58687 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:02.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:01 gibba011 bash[30791]: cluster 2021-03-10T16:04:00.179043+0000 mgr.gibba032.whopeo (mgr.24625) 6937 : cluster [DBG] pgmap v3944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:01 gibba011 bash[31774]: cluster 2021-03-10T16:04:00.179043+0000 mgr.gibba032.whopeo (mgr.24625) 6937 : cluster [DBG] pgmap v3944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:01 gibba032 bash[24004]: cluster 2021-03-10T16:04:00.179043+0000 mgr.gibba032.whopeo (mgr.24625) 6937 : cluster [DBG] pgmap v3944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:04.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:03 gibba011 bash[30791]: cluster 2021-03-10T16:04:02.180576+0000 mgr.gibba032.whopeo (mgr.24625) 6938 : cluster [DBG] pgmap v3945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:04.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:03 gibba011 bash[31774]: cluster 2021-03-10T16:04:02.180576+0000 mgr.gibba032.whopeo (mgr.24625) 6938 : cluster [DBG] pgmap v3945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:03 gibba032 bash[24004]: cluster 2021-03-10T16:04:02.180576+0000 mgr.gibba032.whopeo (mgr.24625) 6938 : cluster [DBG] pgmap v3945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:05 gibba032 bash[24004]: cluster 2021-03-10T16:04:04.181955+0000 mgr.gibba032.whopeo (mgr.24625) 6939 : cluster [DBG] pgmap v3946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:05 gibba032 bash[24004]: cephadm 2021-03-10T16:04:04.600467+0000 mgr.gibba032.whopeo (mgr.24625) 6940 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:05 gibba011 bash[30791]: cluster 2021-03-10T16:04:04.181955+0000 mgr.gibba032.whopeo (mgr.24625) 6939 : cluster [DBG] pgmap v3946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:05 gibba011 bash[30791]: cephadm 2021-03-10T16:04:04.600467+0000 mgr.gibba032.whopeo (mgr.24625) 6940 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:05 gibba011 bash[31774]: cluster 2021-03-10T16:04:04.181955+0000 mgr.gibba032.whopeo (mgr.24625) 6939 : cluster [DBG] pgmap v3946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:05 gibba011 bash[31774]: cephadm 2021-03-10T16:04:04.600467+0000 mgr.gibba032.whopeo (mgr.24625) 6940 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:07 gibba032 bash[24004]: cluster 2021-03-10T16:04:06.183551+0000 mgr.gibba032.whopeo (mgr.24625) 6941 : cluster [DBG] pgmap v3947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:07 gibba011 bash[30791]: cluster 2021-03-10T16:04:06.183551+0000 mgr.gibba032.whopeo (mgr.24625) 6941 : cluster [DBG] pgmap v3947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:07 gibba011 bash[31774]: cluster 2021-03-10T16:04:06.183551+0000 mgr.gibba032.whopeo (mgr.24625) 6941 : cluster [DBG] pgmap v3947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:09 gibba032 bash[24004]: cluster 2021-03-10T16:04:08.185372+0000 mgr.gibba032.whopeo (mgr.24625) 6942 : cluster [DBG] pgmap v3948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:09 gibba011 bash[30791]: cluster 2021-03-10T16:04:08.185372+0000 mgr.gibba032.whopeo (mgr.24625) 6942 : cluster [DBG] pgmap v3948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:09 gibba011 bash[31774]: cluster 2021-03-10T16:04:08.185372+0000 mgr.gibba032.whopeo (mgr.24625) 6942 : cluster [DBG] pgmap v3948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:11 gibba032 bash[24004]: cluster 2021-03-10T16:04:10.186714+0000 mgr.gibba032.whopeo (mgr.24625) 6943 : cluster [DBG] pgmap v3949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:11 gibba011 bash[30791]: cluster 2021-03-10T16:04:10.186714+0000 mgr.gibba032.whopeo (mgr.24625) 6943 : cluster [DBG] pgmap v3949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:11 gibba011 bash[31774]: cluster 2021-03-10T16:04:10.186714+0000 mgr.gibba032.whopeo (mgr.24625) 6943 : cluster [DBG] pgmap v3949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:13 gibba032 bash[24004]: cluster 2021-03-10T16:04:12.188253+0000 mgr.gibba032.whopeo (mgr.24625) 6944 : cluster [DBG] pgmap v3950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:13 gibba011 bash[30791]: cluster 2021-03-10T16:04:12.188253+0000 mgr.gibba032.whopeo (mgr.24625) 6944 : cluster [DBG] pgmap v3950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:13 gibba011 bash[31774]: cluster 2021-03-10T16:04:12.188253+0000 mgr.gibba032.whopeo (mgr.24625) 6944 : cluster [DBG] pgmap v3950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:16.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:15 gibba032 bash[24004]: cluster 2021-03-10T16:04:14.189476+0000 mgr.gibba032.whopeo (mgr.24625) 6945 : cluster [DBG] pgmap v3951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:15 gibba011 bash[30791]: cluster 2021-03-10T16:04:14.189476+0000 mgr.gibba032.whopeo (mgr.24625) 6945 : cluster [DBG] pgmap v3951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:16.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:15 gibba011 bash[31774]: cluster 2021-03-10T16:04:14.189476+0000 mgr.gibba032.whopeo (mgr.24625) 6945 : cluster [DBG] pgmap v3951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:17 gibba032 bash[24004]: cluster 2021-03-10T16:04:16.191055+0000 mgr.gibba032.whopeo (mgr.24625) 6946 : cluster [DBG] pgmap v3952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:17 gibba011 bash[30791]: cluster 2021-03-10T16:04:16.191055+0000 mgr.gibba032.whopeo (mgr.24625) 6946 : cluster [DBG] pgmap v3952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:17 gibba011 bash[31774]: cluster 2021-03-10T16:04:16.191055+0000 mgr.gibba032.whopeo (mgr.24625) 6946 : cluster [DBG] pgmap v3952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:19 gibba032 bash[24004]: cluster 2021-03-10T16:04:18.192735+0000 mgr.gibba032.whopeo (mgr.24625) 6947 : cluster [DBG] pgmap v3953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:19 gibba011 bash[30791]: cluster 2021-03-10T16:04:18.192735+0000 mgr.gibba032.whopeo (mgr.24625) 6947 : cluster [DBG] pgmap v3953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:19 gibba011 bash[31774]: cluster 2021-03-10T16:04:18.192735+0000 mgr.gibba032.whopeo (mgr.24625) 6947 : cluster [DBG] pgmap v3953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:20 gibba032 bash[24004]: cephadm 2021-03-10T16:04:19.604818+0000 mgr.gibba032.whopeo (mgr.24625) 6948 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:20 gibba011 bash[30791]: cephadm 2021-03-10T16:04:19.604818+0000 mgr.gibba032.whopeo (mgr.24625) 6948 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:21.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:20 gibba011 bash[31774]: cephadm 2021-03-10T16:04:19.604818+0000 mgr.gibba032.whopeo (mgr.24625) 6948 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:21 gibba032 bash[24004]: cluster 2021-03-10T16:04:20.194331+0000 mgr.gibba032.whopeo (mgr.24625) 6949 : cluster [DBG] pgmap v3954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:21 gibba011 bash[30791]: cluster 2021-03-10T16:04:20.194331+0000 mgr.gibba032.whopeo (mgr.24625) 6949 : cluster [DBG] pgmap v3954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:21 gibba011 bash[31774]: cluster 2021-03-10T16:04:20.194331+0000 mgr.gibba032.whopeo (mgr.24625) 6949 : cluster [DBG] pgmap v3954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:23 gibba032 bash[24004]: cluster 2021-03-10T16:04:22.195968+0000 mgr.gibba032.whopeo (mgr.24625) 6950 : cluster [DBG] pgmap v3955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:23 gibba011 bash[30791]: cluster 2021-03-10T16:04:22.195968+0000 mgr.gibba032.whopeo (mgr.24625) 6950 : cluster [DBG] pgmap v3955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:23 gibba011 bash[31774]: cluster 2021-03-10T16:04:22.195968+0000 mgr.gibba032.whopeo (mgr.24625) 6950 : cluster [DBG] pgmap v3955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:25 gibba032 bash[24004]: cluster 2021-03-10T16:04:24.197438+0000 mgr.gibba032.whopeo (mgr.24625) 6951 : cluster [DBG] pgmap v3956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:25 gibba011 bash[30791]: cluster 2021-03-10T16:04:24.197438+0000 mgr.gibba032.whopeo (mgr.24625) 6951 : cluster [DBG] pgmap v3956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:26.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:25 gibba011 bash[31774]: cluster 2021-03-10T16:04:24.197438+0000 mgr.gibba032.whopeo (mgr.24625) 6951 : cluster [DBG] pgmap v3956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:27 gibba032 bash[24004]: cluster 2021-03-10T16:04:26.199146+0000 mgr.gibba032.whopeo (mgr.24625) 6952 : cluster [DBG] pgmap v3957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:27 gibba011 bash[30791]: cluster 2021-03-10T16:04:26.199146+0000 mgr.gibba032.whopeo (mgr.24625) 6952 : cluster [DBG] pgmap v3957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:27 gibba011 bash[31774]: cluster 2021-03-10T16:04:26.199146+0000 mgr.gibba032.whopeo (mgr.24625) 6952 : cluster [DBG] pgmap v3957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:29.852 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:04:30.079 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:04:30.080 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:04:30.080 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:04:30.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:04:30.080 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:04:30.081 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:04:30.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:04:30.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:04:30.081 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:04:30.082 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:04:30.082 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:04:30.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:04:30.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:04:30.082 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:04:30.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:04:30.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:04:30.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:04:30.083 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:04:30.084 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:04:30.084 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:04:30.084 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:04:30.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:29 gibba032 bash[24004]: cluster 2021-03-10T16:04:28.201005+0000 mgr.gibba032.whopeo (mgr.24625) 6953 : cluster [DBG] pgmap v3958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:30.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:29 gibba011 bash[30791]: cluster 2021-03-10T16:04:28.201005+0000 mgr.gibba032.whopeo (mgr.24625) 6953 : cluster [DBG] pgmap v3958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:30.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:29 gibba011 bash[31774]: cluster 2021-03-10T16:04:28.201005+0000 mgr.gibba032.whopeo (mgr.24625) 6953 : cluster [DBG] pgmap v3958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:30.328 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:04:30.329 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:04:30.329 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:04:30.329 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:04:30.330 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:04:30.330 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:04:30.330 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:04:30.330 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:04:30.331 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:04:30.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:04:30.331 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:04:30.332 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:04:30.332 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:04:30.332 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:04:30.332 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:04:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:30 gibba032 bash[24004]: audit 2021-03-10T16:04:29.842493+0000 mgr.gibba032.whopeo (mgr.24625) 6954 : audit [DBG] from='client.48717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:30 gibba032 bash[24004]: audit 2021-03-10T16:04:30.067216+0000 mgr.gibba032.whopeo (mgr.24625) 6955 : audit [DBG] from='client.58705 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:30 gibba032 bash[24004]: audit 2021-03-10T16:04:30.328630+0000 mon.a (mon.0) 2520 : audit [DBG] from='client.? 172.21.2.111:0/1824776523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[30791]: audit 2021-03-10T16:04:29.842493+0000 mgr.gibba032.whopeo (mgr.24625) 6954 : audit [DBG] from='client.48717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[30791]: audit 2021-03-10T16:04:30.067216+0000 mgr.gibba032.whopeo (mgr.24625) 6955 : audit [DBG] from='client.58705 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[30791]: audit 2021-03-10T16:04:30.328630+0000 mon.a (mon.0) 2520 : audit [DBG] from='client.? 172.21.2.111:0/1824776523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[31774]: audit 2021-03-10T16:04:29.842493+0000 mgr.gibba032.whopeo (mgr.24625) 6954 : audit [DBG] from='client.48717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[31774]: audit 2021-03-10T16:04:30.067216+0000 mgr.gibba032.whopeo (mgr.24625) 6955 : audit [DBG] from='client.58705 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:04:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:30 gibba011 bash[31774]: audit 2021-03-10T16:04:30.328630+0000 mon.a (mon.0) 2520 : audit [DBG] from='client.? 172.21.2.111:0/1824776523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:31 gibba032 bash[24004]: cluster 2021-03-10T16:04:30.202590+0000 mgr.gibba032.whopeo (mgr.24625) 6956 : cluster [DBG] pgmap v3959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:31 gibba011 bash[30791]: cluster 2021-03-10T16:04:30.202590+0000 mgr.gibba032.whopeo (mgr.24625) 6956 : cluster [DBG] pgmap v3959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:31 gibba011 bash[31774]: cluster 2021-03-10T16:04:30.202590+0000 mgr.gibba032.whopeo (mgr.24625) 6956 : cluster [DBG] pgmap v3959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:33 gibba032 bash[24004]: cluster 2021-03-10T16:04:32.204296+0000 mgr.gibba032.whopeo (mgr.24625) 6957 : cluster [DBG] pgmap v3960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:33 gibba011 bash[30791]: cluster 2021-03-10T16:04:32.204296+0000 mgr.gibba032.whopeo (mgr.24625) 6957 : cluster [DBG] pgmap v3960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:33 gibba011 bash[31774]: cluster 2021-03-10T16:04:32.204296+0000 mgr.gibba032.whopeo (mgr.24625) 6957 : cluster [DBG] pgmap v3960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:35 gibba032 bash[24004]: cluster 2021-03-10T16:04:34.205735+0000 mgr.gibba032.whopeo (mgr.24625) 6958 : cluster [DBG] pgmap v3961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:35 gibba032 bash[24004]: cephadm 2021-03-10T16:04:34.620732+0000 mgr.gibba032.whopeo (mgr.24625) 6959 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:35 gibba011 bash[30791]: cluster 2021-03-10T16:04:34.205735+0000 mgr.gibba032.whopeo (mgr.24625) 6958 : cluster [DBG] pgmap v3961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:35 gibba011 bash[30791]: cephadm 2021-03-10T16:04:34.620732+0000 mgr.gibba032.whopeo (mgr.24625) 6959 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:35 gibba011 bash[31774]: cluster 2021-03-10T16:04:34.205735+0000 mgr.gibba032.whopeo (mgr.24625) 6958 : cluster [DBG] pgmap v3961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:35 gibba011 bash[31774]: cephadm 2021-03-10T16:04:34.620732+0000 mgr.gibba032.whopeo (mgr.24625) 6959 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:37 gibba032 bash[24004]: cluster 2021-03-10T16:04:36.207200+0000 mgr.gibba032.whopeo (mgr.24625) 6960 : cluster [DBG] pgmap v3962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:37 gibba011 bash[30791]: cluster 2021-03-10T16:04:36.207200+0000 mgr.gibba032.whopeo (mgr.24625) 6960 : cluster [DBG] pgmap v3962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:37 gibba011 bash[31774]: cluster 2021-03-10T16:04:36.207200+0000 mgr.gibba032.whopeo (mgr.24625) 6960 : cluster [DBG] pgmap v3962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:40.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:39 gibba032 bash[24004]: cluster 2021-03-10T16:04:38.208869+0000 mgr.gibba032.whopeo (mgr.24625) 6961 : cluster [DBG] pgmap v3963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:39 gibba011 bash[30791]: cluster 2021-03-10T16:04:38.208869+0000 mgr.gibba032.whopeo (mgr.24625) 6961 : cluster [DBG] pgmap v3963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:40 gibba011 bash[31774]: cluster 2021-03-10T16:04:38.208869+0000 mgr.gibba032.whopeo (mgr.24625) 6961 : cluster [DBG] pgmap v3963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:42 gibba011 bash[30791]: cluster 2021-03-10T16:04:40.210279+0000 mgr.gibba032.whopeo (mgr.24625) 6962 : cluster [DBG] pgmap v3964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:42 gibba011 bash[31774]: cluster 2021-03-10T16:04:40.210279+0000 mgr.gibba032.whopeo (mgr.24625) 6962 : cluster [DBG] pgmap v3964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:42 gibba032 bash[24004]: cluster 2021-03-10T16:04:40.210279+0000 mgr.gibba032.whopeo (mgr.24625) 6962 : cluster [DBG] pgmap v3964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:44 gibba011 bash[30791]: cluster 2021-03-10T16:04:42.211701+0000 mgr.gibba032.whopeo (mgr.24625) 6963 : cluster [DBG] pgmap v3965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:44 gibba011 bash[31774]: cluster 2021-03-10T16:04:42.211701+0000 mgr.gibba032.whopeo (mgr.24625) 6963 : cluster [DBG] pgmap v3965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:44 gibba032 bash[24004]: cluster 2021-03-10T16:04:42.211701+0000 mgr.gibba032.whopeo (mgr.24625) 6963 : cluster [DBG] pgmap v3965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:46 gibba011 bash[30791]: cluster 2021-03-10T16:04:44.213182+0000 mgr.gibba032.whopeo (mgr.24625) 6964 : cluster [DBG] pgmap v3966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:46 gibba011 bash[31774]: cluster 2021-03-10T16:04:44.213182+0000 mgr.gibba032.whopeo (mgr.24625) 6964 : cluster [DBG] pgmap v3966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:46 gibba032 bash[24004]: cluster 2021-03-10T16:04:44.213182+0000 mgr.gibba032.whopeo (mgr.24625) 6964 : cluster [DBG] pgmap v3966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:48 gibba011 bash[30791]: cluster 2021-03-10T16:04:46.214677+0000 mgr.gibba032.whopeo (mgr.24625) 6965 : cluster [DBG] pgmap v3967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:48 gibba011 bash[31774]: cluster 2021-03-10T16:04:46.214677+0000 mgr.gibba032.whopeo (mgr.24625) 6965 : cluster [DBG] pgmap v3967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:48 gibba032 bash[24004]: cluster 2021-03-10T16:04:46.214677+0000 mgr.gibba032.whopeo (mgr.24625) 6965 : cluster [DBG] pgmap v3967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: cluster 2021-03-10T16:04:48.216390+0000 mgr.gibba032.whopeo (mgr.24625) 6966 : cluster [DBG] pgmap v3968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.845719+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:04:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.848712+0000 mon.a (mon.0) 2522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:04:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.850023+0000 mon.a (mon.0) 2523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.851524+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.852783+0000 mon.a (mon.0) 2525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.854838+0000 mon.a (mon.0) 2526 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:50 gibba032 bash[24004]: audit 2021-03-10T16:04:49.856285+0000 mon.a (mon.0) 2527 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: cluster 2021-03-10T16:04:48.216390+0000 mgr.gibba032.whopeo (mgr.24625) 6966 : cluster [DBG] pgmap v3968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.845719+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:04:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.848712+0000 mon.a (mon.0) 2522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:04:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.850023+0000 mon.a (mon.0) 2523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.851524+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.852783+0000 mon.a (mon.0) 2525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.854838+0000 mon.a (mon.0) 2526 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[30791]: audit 2021-03-10T16:04:49.856285+0000 mon.a (mon.0) 2527 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: cluster 2021-03-10T16:04:48.216390+0000 mgr.gibba032.whopeo (mgr.24625) 6966 : cluster [DBG] pgmap v3968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.845719+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:04:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.848712+0000 mon.a (mon.0) 2522 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:04:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.850023+0000 mon.a (mon.0) 2523 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.851524+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.852783+0000 mon.a (mon.0) 2525 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.854838+0000 mon.a (mon.0) 2526 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:50.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:50 gibba011 bash[31774]: audit 2021-03-10T16:04:49.856285+0000 mon.a (mon.0) 2527 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:04:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.637548+0000 mgr.gibba032.whopeo (mgr.24625) 6967 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:04:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.639072+0000 mgr.gibba032.whopeo (mgr.24625) 6968 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:04:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.846158+0000 mgr.gibba032.whopeo (mgr.24625) 6969 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:04:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.847805+0000 mgr.gibba032.whopeo (mgr.24625) 6970 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:04:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.849260+0000 mgr.gibba032.whopeo (mgr.24625) 6971 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:04:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.849297+0000 mgr.gibba032.whopeo (mgr.24625) 6972 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:04:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.850606+0000 mgr.gibba032.whopeo (mgr.24625) 6973 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:04:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.850645+0000 mgr.gibba032.whopeo (mgr.24625) 6974 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:04:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.851675+0000 mgr.gibba032.whopeo (mgr.24625) 6975 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:04:51.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.852566+0000 mgr.gibba032.whopeo (mgr.24625) 6976 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:04:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.852605+0000 mgr.gibba032.whopeo (mgr.24625) 6977 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:04:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.853720+0000 mgr.gibba032.whopeo (mgr.24625) 6978 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:04:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.854041+0000 mgr.gibba032.whopeo (mgr.24625) 6979 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:04:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.855166+0000 mgr.gibba032.whopeo (mgr.24625) 6980 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:04:51.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.855206+0000 mgr.gibba032.whopeo (mgr.24625) 6981 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:04:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[30791]: cephadm 2021-03-10T16:04:49.855311+0000 mgr.gibba032.whopeo (mgr.24625) 6982 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:51.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.637548+0000 mgr.gibba032.whopeo (mgr.24625) 6967 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:04:51.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.639072+0000 mgr.gibba032.whopeo (mgr.24625) 6968 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:04:51.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.846158+0000 mgr.gibba032.whopeo (mgr.24625) 6969 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:04:51.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.847805+0000 mgr.gibba032.whopeo (mgr.24625) 6970 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:04:51.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.849260+0000 mgr.gibba032.whopeo (mgr.24625) 6971 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:04:51.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.849297+0000 mgr.gibba032.whopeo (mgr.24625) 6972 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:04:51.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.850606+0000 mgr.gibba032.whopeo (mgr.24625) 6973 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:04:51.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.850645+0000 mgr.gibba032.whopeo (mgr.24625) 6974 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:04:51.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.851675+0000 mgr.gibba032.whopeo (mgr.24625) 6975 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:04:51.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.852566+0000 mgr.gibba032.whopeo (mgr.24625) 6976 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:04:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.852605+0000 mgr.gibba032.whopeo (mgr.24625) 6977 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:04:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.853720+0000 mgr.gibba032.whopeo (mgr.24625) 6978 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:04:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.854041+0000 mgr.gibba032.whopeo (mgr.24625) 6979 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:04:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.855166+0000 mgr.gibba032.whopeo (mgr.24625) 6980 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:04:51.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.855206+0000 mgr.gibba032.whopeo (mgr.24625) 6981 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:04:51.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:51 gibba011 bash[31774]: cephadm 2021-03-10T16:04:49.855311+0000 mgr.gibba032.whopeo (mgr.24625) 6982 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.637548+0000 mgr.gibba032.whopeo (mgr.24625) 6967 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:04:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.639072+0000 mgr.gibba032.whopeo (mgr.24625) 6968 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:04:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.846158+0000 mgr.gibba032.whopeo (mgr.24625) 6969 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:04:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.847805+0000 mgr.gibba032.whopeo (mgr.24625) 6970 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:04:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.849260+0000 mgr.gibba032.whopeo (mgr.24625) 6971 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:04:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.849297+0000 mgr.gibba032.whopeo (mgr.24625) 6972 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:04:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.850606+0000 mgr.gibba032.whopeo (mgr.24625) 6973 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:04:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.850645+0000 mgr.gibba032.whopeo (mgr.24625) 6974 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:04:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.851675+0000 mgr.gibba032.whopeo (mgr.24625) 6975 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:04:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.852566+0000 mgr.gibba032.whopeo (mgr.24625) 6976 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:04:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.852605+0000 mgr.gibba032.whopeo (mgr.24625) 6977 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:04:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.853720+0000 mgr.gibba032.whopeo (mgr.24625) 6978 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:04:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.854041+0000 mgr.gibba032.whopeo (mgr.24625) 6979 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:04:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.855166+0000 mgr.gibba032.whopeo (mgr.24625) 6980 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:04:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.855206+0000 mgr.gibba032.whopeo (mgr.24625) 6981 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:04:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:51 gibba032 bash[24004]: cephadm 2021-03-10T16:04:49.855311+0000 mgr.gibba032.whopeo (mgr.24625) 6982 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:04:52.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:52 gibba011 bash[30791]: cluster 2021-03-10T16:04:50.217813+0000 mgr.gibba032.whopeo (mgr.24625) 6983 : cluster [DBG] pgmap v3969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:52 gibba011 bash[31774]: cluster 2021-03-10T16:04:50.217813+0000 mgr.gibba032.whopeo (mgr.24625) 6983 : cluster [DBG] pgmap v3969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:52 gibba032 bash[24004]: cluster 2021-03-10T16:04:50.217813+0000 mgr.gibba032.whopeo (mgr.24625) 6983 : cluster [DBG] pgmap v3969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:54 gibba011 bash[30791]: cluster 2021-03-10T16:04:52.219544+0000 mgr.gibba032.whopeo (mgr.24625) 6984 : cluster [DBG] pgmap v3970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:54 gibba011 bash[31774]: cluster 2021-03-10T16:04:52.219544+0000 mgr.gibba032.whopeo (mgr.24625) 6984 : cluster [DBG] pgmap v3970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:54 gibba032 bash[24004]: cluster 2021-03-10T16:04:52.219544+0000 mgr.gibba032.whopeo (mgr.24625) 6984 : cluster [DBG] pgmap v3970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:56 gibba011 bash[30791]: cluster 2021-03-10T16:04:54.220979+0000 mgr.gibba032.whopeo (mgr.24625) 6985 : cluster [DBG] pgmap v3971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:56 gibba011 bash[31774]: cluster 2021-03-10T16:04:54.220979+0000 mgr.gibba032.whopeo (mgr.24625) 6985 : cluster [DBG] pgmap v3971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:56 gibba032 bash[24004]: cluster 2021-03-10T16:04:54.220979+0000 mgr.gibba032.whopeo (mgr.24625) 6985 : cluster [DBG] pgmap v3971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.835369+0000 mon.a (mon.0) 2528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:04:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.836099+0000 mon.a (mon.0) 2529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:04:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.893293+0000 mon.a (mon.0) 2530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:04:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.894045+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T16:04:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.894816+0000 mon.a (mon.0) 2532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:04:57.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:57 gibba032 bash[24004]: audit 2021-03-10T16:04:56.895550+0000 mon.a (mon.0) 2533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:04:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.835369+0000 mon.a (mon.0) 2528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:04:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.836099+0000 mon.a (mon.0) 2529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:04:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.893293+0000 mon.a (mon.0) 2530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:04:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.894045+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T16:04:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.894816+0000 mon.a (mon.0) 2532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:04:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[30791]: audit 2021-03-10T16:04:56.895550+0000 mon.a (mon.0) 2533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:04:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.835369+0000 mon.a (mon.0) 2528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:04:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.836099+0000 mon.a (mon.0) 2529 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:04:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.893293+0000 mon.a (mon.0) 2530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:04:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.894045+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T16:04:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.894816+0000 mon.a (mon.0) 2532 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:04:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:57 gibba011 bash[31774]: audit 2021-03-10T16:04:56.895550+0000 mon.a (mon.0) 2533 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:04:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:04:58 gibba032 bash[24004]: cluster 2021-03-10T16:04:56.222570+0000 mgr.gibba032.whopeo (mgr.24625) 6986 : cluster [DBG] pgmap v3972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:04:58 gibba011 bash[30791]: cluster 2021-03-10T16:04:56.222570+0000 mgr.gibba032.whopeo (mgr.24625) 6986 : cluster [DBG] pgmap v3972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:04:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:04:58 gibba011 bash[31774]: cluster 2021-03-10T16:04:56.222570+0000 mgr.gibba032.whopeo (mgr.24625) 6986 : cluster [DBG] pgmap v3972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:00 gibba032 bash[24004]: cluster 2021-03-10T16:04:58.224403+0000 mgr.gibba032.whopeo (mgr.24625) 6987 : cluster [DBG] pgmap v3973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:00.573 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:05:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:00 gibba011 bash[30791]: cluster 2021-03-10T16:04:58.224403+0000 mgr.gibba032.whopeo (mgr.24625) 6987 : cluster [DBG] pgmap v3973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:00 gibba011 bash[31774]: cluster 2021-03-10T16:04:58.224403+0000 mgr.gibba032.whopeo (mgr.24625) 6987 : cluster [DBG] pgmap v3973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:00.799 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:05:00.799 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:05:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:05:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:05:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:05:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:05:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:05:00.801 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:05:00.801 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:05:00.801 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:05:00.801 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:05:00.802 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:05:00.802 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:05:00.802 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:05:00.802 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:05:00.802 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:05:00.803 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:05:00.803 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:05:00.803 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:05:00.803 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:05:00.804 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:05:01.046 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:05:01.047 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:05:01.047 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:05:01.047 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:01.047 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:05:01.048 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:05:01.048 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:01.048 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:05:01.048 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:05:01.048 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:01.049 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:05:01.049 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:05:01.049 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:05:01.049 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:05:01.050 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:05:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:01 gibba032 bash[24004]: audit 2021-03-10T16:05:01.047144+0000 mon.c (mon.1) 133 : audit [DBG] from='client.? 172.21.2.111:0/1687899754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:01.540 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:01 gibba011 bash[30791]: audit 2021-03-10T16:05:01.047144+0000 mon.c (mon.1) 133 : audit [DBG] from='client.? 172.21.2.111:0/1687899754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:01.540 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:01 gibba011 bash[31774]: audit 2021-03-10T16:05:01.047144+0000 mon.c (mon.1) 133 : audit [DBG] from='client.? 172.21.2.111:0/1687899754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:02 gibba032 bash[24004]: cluster 2021-03-10T16:05:00.225986+0000 mgr.gibba032.whopeo (mgr.24625) 6988 : cluster [DBG] pgmap v3974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:02 gibba032 bash[24004]: audit 2021-03-10T16:05:00.562822+0000 mgr.gibba032.whopeo (mgr.24625) 6989 : audit [DBG] from='client.58717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:02 gibba032 bash[24004]: audit 2021-03-10T16:05:00.785833+0000 mgr.gibba032.whopeo (mgr.24625) 6990 : audit [DBG] from='client.48741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[30791]: cluster 2021-03-10T16:05:00.225986+0000 mgr.gibba032.whopeo (mgr.24625) 6988 : cluster [DBG] pgmap v3974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[30791]: audit 2021-03-10T16:05:00.562822+0000 mgr.gibba032.whopeo (mgr.24625) 6989 : audit [DBG] from='client.58717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[30791]: audit 2021-03-10T16:05:00.785833+0000 mgr.gibba032.whopeo (mgr.24625) 6990 : audit [DBG] from='client.48741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[31774]: cluster 2021-03-10T16:05:00.225986+0000 mgr.gibba032.whopeo (mgr.24625) 6988 : cluster [DBG] pgmap v3974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[31774]: audit 2021-03-10T16:05:00.562822+0000 mgr.gibba032.whopeo (mgr.24625) 6989 : audit [DBG] from='client.58717 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:02 gibba011 bash[31774]: audit 2021-03-10T16:05:00.785833+0000 mgr.gibba032.whopeo (mgr.24625) 6990 : audit [DBG] from='client.48741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:04 gibba032 bash[24004]: cluster 2021-03-10T16:05:02.227710+0000 mgr.gibba032.whopeo (mgr.24625) 6991 : cluster [DBG] pgmap v3975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:04 gibba011 bash[30791]: cluster 2021-03-10T16:05:02.227710+0000 mgr.gibba032.whopeo (mgr.24625) 6991 : cluster [DBG] pgmap v3975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:04 gibba011 bash[31774]: cluster 2021-03-10T16:05:02.227710+0000 mgr.gibba032.whopeo (mgr.24625) 6991 : cluster [DBG] pgmap v3975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:06 gibba032 bash[24004]: cluster 2021-03-10T16:05:04.229241+0000 mgr.gibba032.whopeo (mgr.24625) 6992 : cluster [DBG] pgmap v3976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:06 gibba032 bash[24004]: cephadm 2021-03-10T16:05:04.871347+0000 mgr.gibba032.whopeo (mgr.24625) 6993 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:06 gibba011 bash[30791]: cluster 2021-03-10T16:05:04.229241+0000 mgr.gibba032.whopeo (mgr.24625) 6992 : cluster [DBG] pgmap v3976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:06 gibba011 bash[30791]: cephadm 2021-03-10T16:05:04.871347+0000 mgr.gibba032.whopeo (mgr.24625) 6993 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:06 gibba011 bash[31774]: cluster 2021-03-10T16:05:04.229241+0000 mgr.gibba032.whopeo (mgr.24625) 6992 : cluster [DBG] pgmap v3976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:06 gibba011 bash[31774]: cephadm 2021-03-10T16:05:04.871347+0000 mgr.gibba032.whopeo (mgr.24625) 6993 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:08 gibba032 bash[24004]: cluster 2021-03-10T16:05:06.231087+0000 mgr.gibba032.whopeo (mgr.24625) 6994 : cluster [DBG] pgmap v3977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:08 gibba011 bash[30791]: cluster 2021-03-10T16:05:06.231087+0000 mgr.gibba032.whopeo (mgr.24625) 6994 : cluster [DBG] pgmap v3977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:08 gibba011 bash[31774]: cluster 2021-03-10T16:05:06.231087+0000 mgr.gibba032.whopeo (mgr.24625) 6994 : cluster [DBG] pgmap v3977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:10 gibba032 bash[24004]: cluster 2021-03-10T16:05:08.232771+0000 mgr.gibba032.whopeo (mgr.24625) 6995 : cluster [DBG] pgmap v3978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:10 gibba011 bash[30791]: cluster 2021-03-10T16:05:08.232771+0000 mgr.gibba032.whopeo (mgr.24625) 6995 : cluster [DBG] pgmap v3978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:10 gibba011 bash[31774]: cluster 2021-03-10T16:05:08.232771+0000 mgr.gibba032.whopeo (mgr.24625) 6995 : cluster [DBG] pgmap v3978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:12 gibba032 bash[24004]: cluster 2021-03-10T16:05:10.234237+0000 mgr.gibba032.whopeo (mgr.24625) 6996 : cluster [DBG] pgmap v3979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:12 gibba011 bash[30791]: cluster 2021-03-10T16:05:10.234237+0000 mgr.gibba032.whopeo (mgr.24625) 6996 : cluster [DBG] pgmap v3979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:12.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:12 gibba011 bash[31774]: cluster 2021-03-10T16:05:10.234237+0000 mgr.gibba032.whopeo (mgr.24625) 6996 : cluster [DBG] pgmap v3979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:14 gibba032 bash[24004]: cluster 2021-03-10T16:05:12.235267+0000 mgr.gibba032.whopeo (mgr.24625) 6997 : cluster [DBG] pgmap v3980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:14.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:14 gibba011 bash[30791]: cluster 2021-03-10T16:05:12.235267+0000 mgr.gibba032.whopeo (mgr.24625) 6997 : cluster [DBG] pgmap v3980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:14 gibba011 bash[31774]: cluster 2021-03-10T16:05:12.235267+0000 mgr.gibba032.whopeo (mgr.24625) 6997 : cluster [DBG] pgmap v3980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:16 gibba032 bash[24004]: cluster 2021-03-10T16:05:14.236291+0000 mgr.gibba032.whopeo (mgr.24625) 6998 : cluster [DBG] pgmap v3981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:16 gibba011 bash[30791]: cluster 2021-03-10T16:05:14.236291+0000 mgr.gibba032.whopeo (mgr.24625) 6998 : cluster [DBG] pgmap v3981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:16 gibba011 bash[31774]: cluster 2021-03-10T16:05:14.236291+0000 mgr.gibba032.whopeo (mgr.24625) 6998 : cluster [DBG] pgmap v3981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:18 gibba032 bash[24004]: cluster 2021-03-10T16:05:16.237119+0000 mgr.gibba032.whopeo (mgr.24625) 6999 : cluster [DBG] pgmap v3982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:18 gibba011 bash[30791]: cluster 2021-03-10T16:05:16.237119+0000 mgr.gibba032.whopeo (mgr.24625) 6999 : cluster [DBG] pgmap v3982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:18 gibba011 bash[31774]: cluster 2021-03-10T16:05:16.237119+0000 mgr.gibba032.whopeo (mgr.24625) 6999 : cluster [DBG] pgmap v3982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:20 gibba032 bash[24004]: cluster 2021-03-10T16:05:18.237972+0000 mgr.gibba032.whopeo (mgr.24625) 7000 : cluster [DBG] pgmap v3983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:20 gibba011 bash[30791]: cluster 2021-03-10T16:05:18.237972+0000 mgr.gibba032.whopeo (mgr.24625) 7000 : cluster [DBG] pgmap v3983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:20 gibba011 bash[31774]: cluster 2021-03-10T16:05:18.237972+0000 mgr.gibba032.whopeo (mgr.24625) 7000 : cluster [DBG] pgmap v3983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:21.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:21 gibba032 bash[24004]: cephadm 2021-03-10T16:05:19.887438+0000 mgr.gibba032.whopeo (mgr.24625) 7001 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:21.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:21 gibba011 bash[30791]: cephadm 2021-03-10T16:05:19.887438+0000 mgr.gibba032.whopeo (mgr.24625) 7001 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:21.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:21 gibba011 bash[31774]: cephadm 2021-03-10T16:05:19.887438+0000 mgr.gibba032.whopeo (mgr.24625) 7001 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:22 gibba032 bash[24004]: cluster 2021-03-10T16:05:20.238675+0000 mgr.gibba032.whopeo (mgr.24625) 7002 : cluster [DBG] pgmap v3984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:22 gibba011 bash[30791]: cluster 2021-03-10T16:05:20.238675+0000 mgr.gibba032.whopeo (mgr.24625) 7002 : cluster [DBG] pgmap v3984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:22 gibba011 bash[31774]: cluster 2021-03-10T16:05:20.238675+0000 mgr.gibba032.whopeo (mgr.24625) 7002 : cluster [DBG] pgmap v3984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:23.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:23 gibba032 bash[24004]: cluster 2021-03-10T16:05:22.239371+0000 mgr.gibba032.whopeo (mgr.24625) 7003 : cluster [DBG] pgmap v3985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:23.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:23 gibba011 bash[30791]: cluster 2021-03-10T16:05:22.239371+0000 mgr.gibba032.whopeo (mgr.24625) 7003 : cluster [DBG] pgmap v3985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:23.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:23 gibba011 bash[31774]: cluster 2021-03-10T16:05:22.239371+0000 mgr.gibba032.whopeo (mgr.24625) 7003 : cluster [DBG] pgmap v3985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:26.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:25 gibba011 bash[30791]: cluster 2021-03-10T16:05:24.240788+0000 mgr.gibba032.whopeo (mgr.24625) 7004 : cluster [DBG] pgmap v3986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:26.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:25 gibba011 bash[31774]: cluster 2021-03-10T16:05:24.240788+0000 mgr.gibba032.whopeo (mgr.24625) 7004 : cluster [DBG] pgmap v3986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:25 gibba032 bash[24004]: cluster 2021-03-10T16:05:24.240788+0000 mgr.gibba032.whopeo (mgr.24625) 7004 : cluster [DBG] pgmap v3986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:28.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:27 gibba011 bash[30791]: cluster 2021-03-10T16:05:26.242499+0000 mgr.gibba032.whopeo (mgr.24625) 7005 : cluster [DBG] pgmap v3987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:28.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:27 gibba011 bash[31774]: cluster 2021-03-10T16:05:26.242499+0000 mgr.gibba032.whopeo (mgr.24625) 7005 : cluster [DBG] pgmap v3987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:27 gibba032 bash[24004]: cluster 2021-03-10T16:05:26.242499+0000 mgr.gibba032.whopeo (mgr.24625) 7005 : cluster [DBG] pgmap v3987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:30.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:29 gibba011 bash[30791]: cluster 2021-03-10T16:05:28.244275+0000 mgr.gibba032.whopeo (mgr.24625) 7006 : cluster [DBG] pgmap v3988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:29 gibba011 bash[31774]: cluster 2021-03-10T16:05:28.244275+0000 mgr.gibba032.whopeo (mgr.24625) 7006 : cluster [DBG] pgmap v3988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:30.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:29 gibba032 bash[24004]: cluster 2021-03-10T16:05:28.244275+0000 mgr.gibba032.whopeo (mgr.24625) 7006 : cluster [DBG] pgmap v3988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:31.290 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:05:31.512 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:05:31.512 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:05:31.512 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:05:31.512 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:05:31.513 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:05:31.513 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:05:31.513 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:05:31.513 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:05:31.514 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:05:31.514 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:05:31.514 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:05:31.514 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:05:31.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:05:31.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:05:31.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:05:31.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:05:31.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:05:31.516 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:05:31.516 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:05:31.516 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:05:31.516 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:05:31.762 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:05:31.762 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:05:31.762 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:05:31.763 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:31.763 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:05:31.763 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:05:31.763 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:31.764 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:05:31.764 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:05:31.764 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:05:31.764 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:05:31.765 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:05:31.765 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:05:31.765 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:05:31.765 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:05:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:31 gibba011 bash[30791]: cluster 2021-03-10T16:05:30.245621+0000 mgr.gibba032.whopeo (mgr.24625) 7007 : cluster [DBG] pgmap v3989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:31 gibba011 bash[30791]: audit 2021-03-10T16:05:31.763098+0000 mon.c (mon.1) 134 : audit [DBG] from='client.? 172.21.2.111:0/1530972437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:31 gibba011 bash[31774]: cluster 2021-03-10T16:05:30.245621+0000 mgr.gibba032.whopeo (mgr.24625) 7007 : cluster [DBG] pgmap v3989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:31 gibba011 bash[31774]: audit 2021-03-10T16:05:31.763098+0000 mon.c (mon.1) 134 : audit [DBG] from='client.? 172.21.2.111:0/1530972437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:31 gibba032 bash[24004]: cluster 2021-03-10T16:05:30.245621+0000 mgr.gibba032.whopeo (mgr.24625) 7007 : cluster [DBG] pgmap v3989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:31 gibba032 bash[24004]: audit 2021-03-10T16:05:31.763098+0000 mon.c (mon.1) 134 : audit [DBG] from='client.? 172.21.2.111:0/1530972437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:32 gibba011 bash[30791]: audit 2021-03-10T16:05:31.280643+0000 mgr.gibba032.whopeo (mgr.24625) 7008 : audit [DBG] from='client.58735 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:32 gibba011 bash[30791]: audit 2021-03-10T16:05:31.502245+0000 mgr.gibba032.whopeo (mgr.24625) 7009 : audit [DBG] from='client.58741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:33.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:32 gibba011 bash[31774]: audit 2021-03-10T16:05:31.280643+0000 mgr.gibba032.whopeo (mgr.24625) 7008 : audit [DBG] from='client.58735 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:32 gibba011 bash[31774]: audit 2021-03-10T16:05:31.502245+0000 mgr.gibba032.whopeo (mgr.24625) 7009 : audit [DBG] from='client.58741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:32 gibba032 bash[24004]: audit 2021-03-10T16:05:31.280643+0000 mgr.gibba032.whopeo (mgr.24625) 7008 : audit [DBG] from='client.58735 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:32 gibba032 bash[24004]: audit 2021-03-10T16:05:31.502245+0000 mgr.gibba032.whopeo (mgr.24625) 7009 : audit [DBG] from='client.58741 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:05:34.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:33 gibba011 bash[30791]: cluster 2021-03-10T16:05:32.247247+0000 mgr.gibba032.whopeo (mgr.24625) 7010 : cluster [DBG] pgmap v3990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:34.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:33 gibba011 bash[31774]: cluster 2021-03-10T16:05:32.247247+0000 mgr.gibba032.whopeo (mgr.24625) 7010 : cluster [DBG] pgmap v3990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:33 gibba032 bash[24004]: cluster 2021-03-10T16:05:32.247247+0000 mgr.gibba032.whopeo (mgr.24625) 7010 : cluster [DBG] pgmap v3990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:36.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:35 gibba032 bash[24004]: cluster 2021-03-10T16:05:34.248775+0000 mgr.gibba032.whopeo (mgr.24625) 7011 : cluster [DBG] pgmap v3991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:36.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:35 gibba032 bash[24004]: cephadm 2021-03-10T16:05:34.902993+0000 mgr.gibba032.whopeo (mgr.24625) 7012 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:35 gibba011 bash[30791]: cluster 2021-03-10T16:05:34.248775+0000 mgr.gibba032.whopeo (mgr.24625) 7011 : cluster [DBG] pgmap v3991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:35 gibba011 bash[30791]: cephadm 2021-03-10T16:05:34.902993+0000 mgr.gibba032.whopeo (mgr.24625) 7012 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:35 gibba011 bash[31774]: cluster 2021-03-10T16:05:34.248775+0000 mgr.gibba032.whopeo (mgr.24625) 7011 : cluster [DBG] pgmap v3991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:35 gibba011 bash[31774]: cephadm 2021-03-10T16:05:34.902993+0000 mgr.gibba032.whopeo (mgr.24625) 7012 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:37 gibba032 bash[24004]: cluster 2021-03-10T16:05:36.250275+0000 mgr.gibba032.whopeo (mgr.24625) 7013 : cluster [DBG] pgmap v3992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:37 gibba011 bash[30791]: cluster 2021-03-10T16:05:36.250275+0000 mgr.gibba032.whopeo (mgr.24625) 7013 : cluster [DBG] pgmap v3992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:37 gibba011 bash[31774]: cluster 2021-03-10T16:05:36.250275+0000 mgr.gibba032.whopeo (mgr.24625) 7013 : cluster [DBG] pgmap v3992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:39 gibba032 bash[24004]: cluster 2021-03-10T16:05:38.251910+0000 mgr.gibba032.whopeo (mgr.24625) 7014 : cluster [DBG] pgmap v3993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:39 gibba011 bash[30791]: cluster 2021-03-10T16:05:38.251910+0000 mgr.gibba032.whopeo (mgr.24625) 7014 : cluster [DBG] pgmap v3993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:40.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:39 gibba011 bash[31774]: cluster 2021-03-10T16:05:38.251910+0000 mgr.gibba032.whopeo (mgr.24625) 7014 : cluster [DBG] pgmap v3993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:41 gibba032 bash[24004]: cluster 2021-03-10T16:05:40.253325+0000 mgr.gibba032.whopeo (mgr.24625) 7015 : cluster [DBG] pgmap v3994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:41 gibba011 bash[30791]: cluster 2021-03-10T16:05:40.253325+0000 mgr.gibba032.whopeo (mgr.24625) 7015 : cluster [DBG] pgmap v3994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:42.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:41 gibba011 bash[31774]: cluster 2021-03-10T16:05:40.253325+0000 mgr.gibba032.whopeo (mgr.24625) 7015 : cluster [DBG] pgmap v3994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:43 gibba032 bash[24004]: cluster 2021-03-10T16:05:42.254992+0000 mgr.gibba032.whopeo (mgr.24625) 7016 : cluster [DBG] pgmap v3995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:43 gibba011 bash[30791]: cluster 2021-03-10T16:05:42.254992+0000 mgr.gibba032.whopeo (mgr.24625) 7016 : cluster [DBG] pgmap v3995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:44.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:43 gibba011 bash[31774]: cluster 2021-03-10T16:05:42.254992+0000 mgr.gibba032.whopeo (mgr.24625) 7016 : cluster [DBG] pgmap v3995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:45 gibba032 bash[24004]: cluster 2021-03-10T16:05:44.256416+0000 mgr.gibba032.whopeo (mgr.24625) 7017 : cluster [DBG] pgmap v3996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:45 gibba011 bash[30791]: cluster 2021-03-10T16:05:44.256416+0000 mgr.gibba032.whopeo (mgr.24625) 7017 : cluster [DBG] pgmap v3996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:46.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:45 gibba011 bash[31774]: cluster 2021-03-10T16:05:44.256416+0000 mgr.gibba032.whopeo (mgr.24625) 7017 : cluster [DBG] pgmap v3996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:48.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:47 gibba032 bash[24004]: cluster 2021-03-10T16:05:46.257921+0000 mgr.gibba032.whopeo (mgr.24625) 7018 : cluster [DBG] pgmap v3997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:47 gibba011 bash[30791]: cluster 2021-03-10T16:05:46.257921+0000 mgr.gibba032.whopeo (mgr.24625) 7018 : cluster [DBG] pgmap v3997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:48.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:47 gibba011 bash[31774]: cluster 2021-03-10T16:05:46.257921+0000 mgr.gibba032.whopeo (mgr.24625) 7018 : cluster [DBG] pgmap v3997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:49 gibba032 bash[24004]: cluster 2021-03-10T16:05:48.259592+0000 mgr.gibba032.whopeo (mgr.24625) 7019 : cluster [DBG] pgmap v3998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:49 gibba011 bash[30791]: cluster 2021-03-10T16:05:48.259592+0000 mgr.gibba032.whopeo (mgr.24625) 7019 : cluster [DBG] pgmap v3998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:49 gibba011 bash[31774]: cluster 2021-03-10T16:05:48.259592+0000 mgr.gibba032.whopeo (mgr.24625) 7019 : cluster [DBG] pgmap v3998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:49.918714+0000 mgr.gibba032.whopeo (mgr.24625) 7020 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:05:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:49.919046+0000 mgr.gibba032.whopeo (mgr.24625) 7021 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:05:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.126696+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:05:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.127540+0000 mgr.gibba032.whopeo (mgr.24625) 7022 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:05:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.129366+0000 mgr.gibba032.whopeo (mgr.24625) 7023 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:05:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.130215+0000 mon.a (mon.0) 2535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:05:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.130649+0000 mgr.gibba032.whopeo (mgr.24625) 7024 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:05:51.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.130695+0000 mgr.gibba032.whopeo (mgr.24625) 7025 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:05:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.131614+0000 mon.a (mon.0) 2536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.131987+0000 mgr.gibba032.whopeo (mgr.24625) 7026 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:05:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.132070+0000 mgr.gibba032.whopeo (mgr.24625) 7027 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:05:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.133085+0000 mon.a (mon.0) 2537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.133206+0000 mgr.gibba032.whopeo (mgr.24625) 7028 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:05:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.134231+0000 mgr.gibba032.whopeo (mgr.24625) 7029 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:05:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.134277+0000 mgr.gibba032.whopeo (mgr.24625) 7030 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:05:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.134341+0000 mon.a (mon.0) 2538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.135526+0000 mgr.gibba032.whopeo (mgr.24625) 7031 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:05:51.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.135951+0000 mgr.gibba032.whopeo (mgr.24625) 7032 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:05:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.136671+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.137164+0000 mgr.gibba032.whopeo (mgr.24625) 7033 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:05:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.137219+0000 mgr.gibba032.whopeo (mgr.24625) 7034 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:05:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: cephadm 2021-03-10T16:05:50.137364+0000 mgr.gibba032.whopeo (mgr.24625) 7035 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:51.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:51 gibba032 bash[24004]: audit 2021-03-10T16:05:50.138236+0000 mon.a (mon.0) 2540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:49.918714+0000 mgr.gibba032.whopeo (mgr.24625) 7020 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:05:51.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:49.919046+0000 mgr.gibba032.whopeo (mgr.24625) 7021 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:05:51.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.126696+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:05:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.127540+0000 mgr.gibba032.whopeo (mgr.24625) 7022 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:05:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.129366+0000 mgr.gibba032.whopeo (mgr.24625) 7023 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:05:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.130215+0000 mon.a (mon.0) 2535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:05:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.130649+0000 mgr.gibba032.whopeo (mgr.24625) 7024 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:05:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.130695+0000 mgr.gibba032.whopeo (mgr.24625) 7025 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:05:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.131614+0000 mon.a (mon.0) 2536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.131987+0000 mgr.gibba032.whopeo (mgr.24625) 7026 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:05:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.132070+0000 mgr.gibba032.whopeo (mgr.24625) 7027 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:05:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.133085+0000 mon.a (mon.0) 2537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.133206+0000 mgr.gibba032.whopeo (mgr.24625) 7028 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:05:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.134231+0000 mgr.gibba032.whopeo (mgr.24625) 7029 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:05:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.134277+0000 mgr.gibba032.whopeo (mgr.24625) 7030 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:05:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.134341+0000 mon.a (mon.0) 2538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.135526+0000 mgr.gibba032.whopeo (mgr.24625) 7031 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:05:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.135951+0000 mgr.gibba032.whopeo (mgr.24625) 7032 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:05:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.136671+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.137164+0000 mgr.gibba032.whopeo (mgr.24625) 7033 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:05:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.137219+0000 mgr.gibba032.whopeo (mgr.24625) 7034 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:05:51.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: cephadm 2021-03-10T16:05:50.137364+0000 mgr.gibba032.whopeo (mgr.24625) 7035 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:51.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[30791]: audit 2021-03-10T16:05:50.138236+0000 mon.a (mon.0) 2540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:49.918714+0000 mgr.gibba032.whopeo (mgr.24625) 7020 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:05:51.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:49.919046+0000 mgr.gibba032.whopeo (mgr.24625) 7021 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:05:51.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.126696+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:05:51.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.127540+0000 mgr.gibba032.whopeo (mgr.24625) 7022 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:05:51.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.129366+0000 mgr.gibba032.whopeo (mgr.24625) 7023 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:05:51.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.130215+0000 mon.a (mon.0) 2535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:05:51.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.130649+0000 mgr.gibba032.whopeo (mgr.24625) 7024 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:05:51.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.130695+0000 mgr.gibba032.whopeo (mgr.24625) 7025 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:05:51.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.131614+0000 mon.a (mon.0) 2536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.131987+0000 mgr.gibba032.whopeo (mgr.24625) 7026 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:05:51.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.132070+0000 mgr.gibba032.whopeo (mgr.24625) 7027 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:05:51.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.133085+0000 mon.a (mon.0) 2537 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.133206+0000 mgr.gibba032.whopeo (mgr.24625) 7028 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:05:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.134231+0000 mgr.gibba032.whopeo (mgr.24625) 7029 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:05:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.134277+0000 mgr.gibba032.whopeo (mgr.24625) 7030 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:05:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.134341+0000 mon.a (mon.0) 2538 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.135526+0000 mgr.gibba032.whopeo (mgr.24625) 7031 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:05:51.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.135951+0000 mgr.gibba032.whopeo (mgr.24625) 7032 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:05:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.136671+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.137164+0000 mgr.gibba032.whopeo (mgr.24625) 7033 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:05:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.137219+0000 mgr.gibba032.whopeo (mgr.24625) 7034 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:05:51.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: cephadm 2021-03-10T16:05:50.137364+0000 mgr.gibba032.whopeo (mgr.24625) 7035 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:05:51.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:51 gibba011 bash[31774]: audit 2021-03-10T16:05:50.138236+0000 mon.a (mon.0) 2540 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:05:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:52 gibba032 bash[24004]: cluster 2021-03-10T16:05:50.261112+0000 mgr.gibba032.whopeo (mgr.24625) 7036 : cluster [DBG] pgmap v3999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:52 gibba011 bash[30791]: cluster 2021-03-10T16:05:50.261112+0000 mgr.gibba032.whopeo (mgr.24625) 7036 : cluster [DBG] pgmap v3999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:52 gibba011 bash[31774]: cluster 2021-03-10T16:05:50.261112+0000 mgr.gibba032.whopeo (mgr.24625) 7036 : cluster [DBG] pgmap v3999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:54 gibba032 bash[24004]: cluster 2021-03-10T16:05:52.262790+0000 mgr.gibba032.whopeo (mgr.24625) 7037 : cluster [DBG] pgmap v4000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:54 gibba011 bash[30791]: cluster 2021-03-10T16:05:52.262790+0000 mgr.gibba032.whopeo (mgr.24625) 7037 : cluster [DBG] pgmap v4000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:54 gibba011 bash[31774]: cluster 2021-03-10T16:05:52.262790+0000 mgr.gibba032.whopeo (mgr.24625) 7037 : cluster [DBG] pgmap v4000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:56 gibba032 bash[24004]: cluster 2021-03-10T16:05:54.264265+0000 mgr.gibba032.whopeo (mgr.24625) 7038 : cluster [DBG] pgmap v4001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:56 gibba011 bash[30791]: cluster 2021-03-10T16:05:54.264265+0000 mgr.gibba032.whopeo (mgr.24625) 7038 : cluster [DBG] pgmap v4001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:56 gibba011 bash[31774]: cluster 2021-03-10T16:05:54.264265+0000 mgr.gibba032.whopeo (mgr.24625) 7038 : cluster [DBG] pgmap v4001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.842248+0000 mon.a (mon.0) 2541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:05:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.842912+0000 mon.a (mon.0) 2542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:05:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.901534+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:05:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.902252+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:05:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.902922+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:05:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:57 gibba032 bash[24004]: audit 2021-03-10T16:05:56.903594+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:05:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.842248+0000 mon.a (mon.0) 2541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:05:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.842912+0000 mon.a (mon.0) 2542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:05:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.901534+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:05:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.902252+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:05:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.902922+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:05:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[30791]: audit 2021-03-10T16:05:56.903594+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:05:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.842248+0000 mon.a (mon.0) 2541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:05:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.842912+0000 mon.a (mon.0) 2542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:05:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.901534+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:05:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.902252+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:05:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.902922+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:05:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:57 gibba011 bash[31774]: audit 2021-03-10T16:05:56.903594+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:05:58 gibba032 bash[24004]: cluster 2021-03-10T16:05:56.265901+0000 mgr.gibba032.whopeo (mgr.24625) 7039 : cluster [DBG] pgmap v4002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:05:58 gibba011 bash[30791]: cluster 2021-03-10T16:05:56.265901+0000 mgr.gibba032.whopeo (mgr.24625) 7039 : cluster [DBG] pgmap v4002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:05:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:05:58 gibba011 bash[31774]: cluster 2021-03-10T16:05:56.265901+0000 mgr.gibba032.whopeo (mgr.24625) 7039 : cluster [DBG] pgmap v4002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:00 gibba032 bash[24004]: cluster 2021-03-10T16:05:58.267538+0000 mgr.gibba032.whopeo (mgr.24625) 7040 : cluster [DBG] pgmap v4003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:00 gibba011 bash[30791]: cluster 2021-03-10T16:05:58.267538+0000 mgr.gibba032.whopeo (mgr.24625) 7040 : cluster [DBG] pgmap v4003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:00 gibba011 bash[31774]: cluster 2021-03-10T16:05:58.267538+0000 mgr.gibba032.whopeo (mgr.24625) 7040 : cluster [DBG] pgmap v4003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:02.006 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:06:02.229 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:06:02.229 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:06:02.229 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:06:02.230 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:06:02.230 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:06:02.230 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:06:02.230 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:06:02.231 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:06:02.231 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:06:02.231 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:06:02.231 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:06:02.231 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:06:02.232 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:06:02.232 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:06:02.232 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:06:02.232 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:06:02.233 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:06:02.233 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:06:02.233 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:06:02.233 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:06:02.234 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:06:02.476 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:06:02.477 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:06:02.477 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:06:02.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:02.477 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:06:02.478 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:06:02.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:02.478 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:06:02.478 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:06:02.479 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:02.479 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:06:02.479 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:06:02.479 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:06:02.480 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:06:02.480 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:06:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:02 gibba032 bash[24004]: cluster 2021-03-10T16:06:00.268603+0000 mgr.gibba032.whopeo (mgr.24625) 7041 : cluster [DBG] pgmap v4004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:02 gibba011 bash[30791]: cluster 2021-03-10T16:06:00.268603+0000 mgr.gibba032.whopeo (mgr.24625) 7041 : cluster [DBG] pgmap v4004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:02.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:02 gibba011 bash[31774]: cluster 2021-03-10T16:06:00.268603+0000 mgr.gibba032.whopeo (mgr.24625) 7041 : cluster [DBG] pgmap v4004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:03 gibba032 bash[24004]: audit 2021-03-10T16:06:01.996232+0000 mgr.gibba032.whopeo (mgr.24625) 7042 : audit [DBG] from='client.58753 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:03 gibba032 bash[24004]: audit 2021-03-10T16:06:02.477404+0000 mon.c (mon.1) 135 : audit [DBG] from='client.? 172.21.2.111:0/1054793465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:03.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:03 gibba011 bash[30791]: audit 2021-03-10T16:06:01.996232+0000 mgr.gibba032.whopeo (mgr.24625) 7042 : audit [DBG] from='client.58753 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:03.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:03 gibba011 bash[30791]: audit 2021-03-10T16:06:02.477404+0000 mon.c (mon.1) 135 : audit [DBG] from='client.? 172.21.2.111:0/1054793465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:03.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:03 gibba011 bash[31774]: audit 2021-03-10T16:06:01.996232+0000 mgr.gibba032.whopeo (mgr.24625) 7042 : audit [DBG] from='client.58753 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:03.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:03 gibba011 bash[31774]: audit 2021-03-10T16:06:02.477404+0000 mon.c (mon.1) 135 : audit [DBG] from='client.? 172.21.2.111:0/1054793465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:04 gibba032 bash[24004]: audit 2021-03-10T16:06:02.219213+0000 mgr.gibba032.whopeo (mgr.24625) 7043 : audit [DBG] from='client.58759 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:04 gibba032 bash[24004]: cluster 2021-03-10T16:06:02.269292+0000 mgr.gibba032.whopeo (mgr.24625) 7044 : cluster [DBG] pgmap v4005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:04 gibba011 bash[30791]: audit 2021-03-10T16:06:02.219213+0000 mgr.gibba032.whopeo (mgr.24625) 7043 : audit [DBG] from='client.58759 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:04 gibba011 bash[30791]: cluster 2021-03-10T16:06:02.269292+0000 mgr.gibba032.whopeo (mgr.24625) 7044 : cluster [DBG] pgmap v4005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:04 gibba011 bash[31774]: audit 2021-03-10T16:06:02.219213+0000 mgr.gibba032.whopeo (mgr.24625) 7043 : audit [DBG] from='client.58759 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:04 gibba011 bash[31774]: cluster 2021-03-10T16:06:02.269292+0000 mgr.gibba032.whopeo (mgr.24625) 7044 : cluster [DBG] pgmap v4005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:05 gibba032 bash[24004]: cluster 2021-03-10T16:06:04.269698+0000 mgr.gibba032.whopeo (mgr.24625) 7045 : cluster [DBG] pgmap v4006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:05 gibba032 bash[24004]: cephadm 2021-03-10T16:06:05.153326+0000 mgr.gibba032.whopeo (mgr.24625) 7046 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:05.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:05 gibba011 bash[30791]: cluster 2021-03-10T16:06:04.269698+0000 mgr.gibba032.whopeo (mgr.24625) 7045 : cluster [DBG] pgmap v4006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:05.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:05 gibba011 bash[30791]: cephadm 2021-03-10T16:06:05.153326+0000 mgr.gibba032.whopeo (mgr.24625) 7046 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:05.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:05 gibba011 bash[31774]: cluster 2021-03-10T16:06:04.269698+0000 mgr.gibba032.whopeo (mgr.24625) 7045 : cluster [DBG] pgmap v4006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:05.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:05 gibba011 bash[31774]: cephadm 2021-03-10T16:06:05.153326+0000 mgr.gibba032.whopeo (mgr.24625) 7046 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:07 gibba032 bash[24004]: cluster 2021-03-10T16:06:06.271034+0000 mgr.gibba032.whopeo (mgr.24625) 7047 : cluster [DBG] pgmap v4007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:07.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:07 gibba011 bash[30791]: cluster 2021-03-10T16:06:06.271034+0000 mgr.gibba032.whopeo (mgr.24625) 7047 : cluster [DBG] pgmap v4007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:07.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:07 gibba011 bash[31774]: cluster 2021-03-10T16:06:06.271034+0000 mgr.gibba032.whopeo (mgr.24625) 7047 : cluster [DBG] pgmap v4007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:10.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:09 gibba011 bash[30791]: cluster 2021-03-10T16:06:08.272841+0000 mgr.gibba032.whopeo (mgr.24625) 7048 : cluster [DBG] pgmap v4008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:10.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:09 gibba011 bash[31774]: cluster 2021-03-10T16:06:08.272841+0000 mgr.gibba032.whopeo (mgr.24625) 7048 : cluster [DBG] pgmap v4008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:09 gibba032 bash[24004]: cluster 2021-03-10T16:06:08.272841+0000 mgr.gibba032.whopeo (mgr.24625) 7048 : cluster [DBG] pgmap v4008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:12.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:11 gibba011 bash[30791]: cluster 2021-03-10T16:06:10.274319+0000 mgr.gibba032.whopeo (mgr.24625) 7049 : cluster [DBG] pgmap v4009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:12.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:11 gibba011 bash[31774]: cluster 2021-03-10T16:06:10.274319+0000 mgr.gibba032.whopeo (mgr.24625) 7049 : cluster [DBG] pgmap v4009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:12.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:11 gibba032 bash[24004]: cluster 2021-03-10T16:06:10.274319+0000 mgr.gibba032.whopeo (mgr.24625) 7049 : cluster [DBG] pgmap v4009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:13 gibba011 bash[30791]: cluster 2021-03-10T16:06:12.275870+0000 mgr.gibba032.whopeo (mgr.24625) 7050 : cluster [DBG] pgmap v4010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:14.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:13 gibba011 bash[31774]: cluster 2021-03-10T16:06:12.275870+0000 mgr.gibba032.whopeo (mgr.24625) 7050 : cluster [DBG] pgmap v4010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:13 gibba032 bash[24004]: cluster 2021-03-10T16:06:12.275870+0000 mgr.gibba032.whopeo (mgr.24625) 7050 : cluster [DBG] pgmap v4010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:15 gibba032 bash[24004]: cluster 2021-03-10T16:06:14.277236+0000 mgr.gibba032.whopeo (mgr.24625) 7051 : cluster [DBG] pgmap v4011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:15 gibba011 bash[30791]: cluster 2021-03-10T16:06:14.277236+0000 mgr.gibba032.whopeo (mgr.24625) 7051 : cluster [DBG] pgmap v4011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:15 gibba011 bash[31774]: cluster 2021-03-10T16:06:14.277236+0000 mgr.gibba032.whopeo (mgr.24625) 7051 : cluster [DBG] pgmap v4011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:17 gibba032 bash[24004]: cluster 2021-03-10T16:06:16.278730+0000 mgr.gibba032.whopeo (mgr.24625) 7052 : cluster [DBG] pgmap v4012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:17 gibba011 bash[30791]: cluster 2021-03-10T16:06:16.278730+0000 mgr.gibba032.whopeo (mgr.24625) 7052 : cluster [DBG] pgmap v4012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:17 gibba011 bash[31774]: cluster 2021-03-10T16:06:16.278730+0000 mgr.gibba032.whopeo (mgr.24625) 7052 : cluster [DBG] pgmap v4012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:20.170 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:19 gibba032 bash[24004]: cluster 2021-03-10T16:06:18.280424+0000 mgr.gibba032.whopeo (mgr.24625) 7053 : cluster [DBG] pgmap v4013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:19 gibba011 bash[30791]: cluster 2021-03-10T16:06:18.280424+0000 mgr.gibba032.whopeo (mgr.24625) 7053 : cluster [DBG] pgmap v4013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:20.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:19 gibba011 bash[31774]: cluster 2021-03-10T16:06:18.280424+0000 mgr.gibba032.whopeo (mgr.24625) 7053 : cluster [DBG] pgmap v4013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:20 gibba032 bash[24004]: cephadm 2021-03-10T16:06:20.169129+0000 mgr.gibba032.whopeo (mgr.24625) 7054 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:21.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:20 gibba011 bash[30791]: cephadm 2021-03-10T16:06:20.169129+0000 mgr.gibba032.whopeo (mgr.24625) 7054 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:21.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:20 gibba011 bash[31774]: cephadm 2021-03-10T16:06:20.169129+0000 mgr.gibba032.whopeo (mgr.24625) 7054 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:21 gibba032 bash[24004]: cluster 2021-03-10T16:06:20.282010+0000 mgr.gibba032.whopeo (mgr.24625) 7055 : cluster [DBG] pgmap v4014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:21 gibba011 bash[30791]: cluster 2021-03-10T16:06:20.282010+0000 mgr.gibba032.whopeo (mgr.24625) 7055 : cluster [DBG] pgmap v4014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:21 gibba011 bash[31774]: cluster 2021-03-10T16:06:20.282010+0000 mgr.gibba032.whopeo (mgr.24625) 7055 : cluster [DBG] pgmap v4014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:23 gibba032 bash[24004]: cluster 2021-03-10T16:06:22.283565+0000 mgr.gibba032.whopeo (mgr.24625) 7056 : cluster [DBG] pgmap v4015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:23 gibba011 bash[30791]: cluster 2021-03-10T16:06:22.283565+0000 mgr.gibba032.whopeo (mgr.24625) 7056 : cluster [DBG] pgmap v4015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:23 gibba011 bash[31774]: cluster 2021-03-10T16:06:22.283565+0000 mgr.gibba032.whopeo (mgr.24625) 7056 : cluster [DBG] pgmap v4015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:25 gibba032 bash[24004]: cluster 2021-03-10T16:06:24.284958+0000 mgr.gibba032.whopeo (mgr.24625) 7057 : cluster [DBG] pgmap v4016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:25 gibba011 bash[30791]: cluster 2021-03-10T16:06:24.284958+0000 mgr.gibba032.whopeo (mgr.24625) 7057 : cluster [DBG] pgmap v4016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:25 gibba011 bash[31774]: cluster 2021-03-10T16:06:24.284958+0000 mgr.gibba032.whopeo (mgr.24625) 7057 : cluster [DBG] pgmap v4016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:27 gibba032 bash[24004]: cluster 2021-03-10T16:06:26.286564+0000 mgr.gibba032.whopeo (mgr.24625) 7058 : cluster [DBG] pgmap v4017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:27 gibba011 bash[30791]: cluster 2021-03-10T16:06:26.286564+0000 mgr.gibba032.whopeo (mgr.24625) 7058 : cluster [DBG] pgmap v4017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:27 gibba011 bash[31774]: cluster 2021-03-10T16:06:26.286564+0000 mgr.gibba032.whopeo (mgr.24625) 7058 : cluster [DBG] pgmap v4017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:29 gibba032 bash[24004]: cluster 2021-03-10T16:06:28.288268+0000 mgr.gibba032.whopeo (mgr.24625) 7059 : cluster [DBG] pgmap v4018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:29 gibba011 bash[30791]: cluster 2021-03-10T16:06:28.288268+0000 mgr.gibba032.whopeo (mgr.24625) 7059 : cluster [DBG] pgmap v4018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:30.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:29 gibba011 bash[31774]: cluster 2021-03-10T16:06:28.288268+0000 mgr.gibba032.whopeo (mgr.24625) 7059 : cluster [DBG] pgmap v4018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:31 gibba032 bash[24004]: cluster 2021-03-10T16:06:30.289706+0000 mgr.gibba032.whopeo (mgr.24625) 7060 : cluster [DBG] pgmap v4019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:31 gibba011 bash[30791]: cluster 2021-03-10T16:06:30.289706+0000 mgr.gibba032.whopeo (mgr.24625) 7060 : cluster [DBG] pgmap v4019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:32.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:31 gibba011 bash[31774]: cluster 2021-03-10T16:06:30.289706+0000 mgr.gibba032.whopeo (mgr.24625) 7060 : cluster [DBG] pgmap v4019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:32.717 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:06:32.944 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:06:32.944 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:06:32.945 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:06:32.945 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:06:32.945 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:06:32.945 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:06:32.946 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:06:32.946 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:06:32.946 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:06:32.946 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:06:32.947 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:06:32.947 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:06:32.947 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:06:32.947 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:06:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:06:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:06:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:06:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:06:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:06:32.949 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:06:32.949 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:06:33.202 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:06:33.203 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:06:33.203 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:06:33.203 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:33.203 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:06:33.204 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:06:33.204 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:33.204 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:06:33.204 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:06:33.204 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:06:33.205 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:06:33.205 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:06:33.205 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:06:33.205 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:06:33.206 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:06:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:33 gibba032 bash[24004]: cluster 2021-03-10T16:06:32.291257+0000 mgr.gibba032.whopeo (mgr.24625) 7061 : cluster [DBG] pgmap v4020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:33 gibba032 bash[24004]: audit 2021-03-10T16:06:32.707396+0000 mgr.gibba032.whopeo (mgr.24625) 7062 : audit [DBG] from='client.48789 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:33 gibba032 bash[24004]: audit 2021-03-10T16:06:32.934232+0000 mgr.gibba032.whopeo (mgr.24625) 7063 : audit [DBG] from='client.48795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:33 gibba032 bash[24004]: audit 2021-03-10T16:06:33.203213+0000 mon.c (mon.1) 136 : audit [DBG] from='client.? 172.21.2.111:0/1263722644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[30791]: cluster 2021-03-10T16:06:32.291257+0000 mgr.gibba032.whopeo (mgr.24625) 7061 : cluster [DBG] pgmap v4020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[30791]: audit 2021-03-10T16:06:32.707396+0000 mgr.gibba032.whopeo (mgr.24625) 7062 : audit [DBG] from='client.48789 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[30791]: audit 2021-03-10T16:06:32.934232+0000 mgr.gibba032.whopeo (mgr.24625) 7063 : audit [DBG] from='client.48795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[30791]: audit 2021-03-10T16:06:33.203213+0000 mon.c (mon.1) 136 : audit [DBG] from='client.? 172.21.2.111:0/1263722644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[31774]: cluster 2021-03-10T16:06:32.291257+0000 mgr.gibba032.whopeo (mgr.24625) 7061 : cluster [DBG] pgmap v4020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[31774]: audit 2021-03-10T16:06:32.707396+0000 mgr.gibba032.whopeo (mgr.24625) 7062 : audit [DBG] from='client.48789 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[31774]: audit 2021-03-10T16:06:32.934232+0000 mgr.gibba032.whopeo (mgr.24625) 7063 : audit [DBG] from='client.48795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:06:34.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:33 gibba011 bash[31774]: audit 2021-03-10T16:06:33.203213+0000 mon.c (mon.1) 136 : audit [DBG] from='client.? 172.21.2.111:0/1263722644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:35 gibba032 bash[24004]: cluster 2021-03-10T16:06:34.292633+0000 mgr.gibba032.whopeo (mgr.24625) 7064 : cluster [DBG] pgmap v4021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:35 gibba032 bash[24004]: cephadm 2021-03-10T16:06:35.185184+0000 mgr.gibba032.whopeo (mgr.24625) 7065 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:35 gibba011 bash[30791]: cluster 2021-03-10T16:06:34.292633+0000 mgr.gibba032.whopeo (mgr.24625) 7064 : cluster [DBG] pgmap v4021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:35 gibba011 bash[30791]: cephadm 2021-03-10T16:06:35.185184+0000 mgr.gibba032.whopeo (mgr.24625) 7065 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:35 gibba011 bash[31774]: cluster 2021-03-10T16:06:34.292633+0000 mgr.gibba032.whopeo (mgr.24625) 7064 : cluster [DBG] pgmap v4021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:35 gibba011 bash[31774]: cephadm 2021-03-10T16:06:35.185184+0000 mgr.gibba032.whopeo (mgr.24625) 7065 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:37 gibba032 bash[24004]: cluster 2021-03-10T16:06:36.294242+0000 mgr.gibba032.whopeo (mgr.24625) 7066 : cluster [DBG] pgmap v4022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:37 gibba011 bash[30791]: cluster 2021-03-10T16:06:36.294242+0000 mgr.gibba032.whopeo (mgr.24625) 7066 : cluster [DBG] pgmap v4022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:37 gibba011 bash[31774]: cluster 2021-03-10T16:06:36.294242+0000 mgr.gibba032.whopeo (mgr.24625) 7066 : cluster [DBG] pgmap v4022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:40.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:39 gibba032 bash[24004]: cluster 2021-03-10T16:06:38.295903+0000 mgr.gibba032.whopeo (mgr.24625) 7067 : cluster [DBG] pgmap v4023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:39 gibba011 bash[30791]: cluster 2021-03-10T16:06:38.295903+0000 mgr.gibba032.whopeo (mgr.24625) 7067 : cluster [DBG] pgmap v4023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:40.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:39 gibba011 bash[31774]: cluster 2021-03-10T16:06:38.295903+0000 mgr.gibba032.whopeo (mgr.24625) 7067 : cluster [DBG] pgmap v4023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:41 gibba032 bash[24004]: cluster 2021-03-10T16:06:40.296642+0000 mgr.gibba032.whopeo (mgr.24625) 7068 : cluster [DBG] pgmap v4024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:41 gibba011 bash[30791]: cluster 2021-03-10T16:06:40.296642+0000 mgr.gibba032.whopeo (mgr.24625) 7068 : cluster [DBG] pgmap v4024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:42.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:41 gibba011 bash[31774]: cluster 2021-03-10T16:06:40.296642+0000 mgr.gibba032.whopeo (mgr.24625) 7068 : cluster [DBG] pgmap v4024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:43 gibba032 bash[24004]: cluster 2021-03-10T16:06:42.297990+0000 mgr.gibba032.whopeo (mgr.24625) 7069 : cluster [DBG] pgmap v4025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:43 gibba011 bash[30791]: cluster 2021-03-10T16:06:42.297990+0000 mgr.gibba032.whopeo (mgr.24625) 7069 : cluster [DBG] pgmap v4025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:43 gibba011 bash[31774]: cluster 2021-03-10T16:06:42.297990+0000 mgr.gibba032.whopeo (mgr.24625) 7069 : cluster [DBG] pgmap v4025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:45 gibba032 bash[24004]: cluster 2021-03-10T16:06:44.299460+0000 mgr.gibba032.whopeo (mgr.24625) 7070 : cluster [DBG] pgmap v4026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:45 gibba011 bash[30791]: cluster 2021-03-10T16:06:44.299460+0000 mgr.gibba032.whopeo (mgr.24625) 7070 : cluster [DBG] pgmap v4026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:46.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:45 gibba011 bash[31774]: cluster 2021-03-10T16:06:44.299460+0000 mgr.gibba032.whopeo (mgr.24625) 7070 : cluster [DBG] pgmap v4026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:47 gibba032 bash[24004]: cluster 2021-03-10T16:06:46.301121+0000 mgr.gibba032.whopeo (mgr.24625) 7071 : cluster [DBG] pgmap v4027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:47 gibba011 bash[30791]: cluster 2021-03-10T16:06:46.301121+0000 mgr.gibba032.whopeo (mgr.24625) 7071 : cluster [DBG] pgmap v4027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:48.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:47 gibba011 bash[31774]: cluster 2021-03-10T16:06:46.301121+0000 mgr.gibba032.whopeo (mgr.24625) 7071 : cluster [DBG] pgmap v4027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:49 gibba032 bash[24004]: cluster 2021-03-10T16:06:48.302843+0000 mgr.gibba032.whopeo (mgr.24625) 7072 : cluster [DBG] pgmap v4028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:50.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:49 gibba011 bash[31774]: cluster 2021-03-10T16:06:48.302843+0000 mgr.gibba032.whopeo (mgr.24625) 7072 : cluster [DBG] pgmap v4028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:49 gibba011 bash[30791]: cluster 2021-03-10T16:06:48.302843+0000 mgr.gibba032.whopeo (mgr.24625) 7072 : cluster [DBG] pgmap v4028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cluster 2021-03-10T16:06:50.303360+0000 mgr.gibba032.whopeo (mgr.24625) 7073 : cluster [DBG] pgmap v4029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.394105+0000 mgr.gibba032.whopeo (mgr.24625) 7074 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:06:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.395325+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.401335+0000 mgr.gibba032.whopeo (mgr.24625) 7075 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:06:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.402651+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.610399+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.611183+0000 mgr.gibba032.whopeo (mgr.24625) 7076 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:06:51.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.613550+0000 mgr.gibba032.whopeo (mgr.24625) 7077 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:06:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.614003+0000 mon.a (mon.0) 2550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:06:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.615280+0000 mgr.gibba032.whopeo (mgr.24625) 7078 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:06:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.615326+0000 mgr.gibba032.whopeo (mgr.24625) 7079 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:06:51.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.615824+0000 mon.a (mon.0) 2551 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.616780+0000 mgr.gibba032.whopeo (mgr.24625) 7080 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:06:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.616822+0000 mgr.gibba032.whopeo (mgr.24625) 7081 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:06:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.617758+0000 mon.a (mon.0) 2552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.618350+0000 mgr.gibba032.whopeo (mgr.24625) 7082 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:06:51.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.619129+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.619700+0000 mgr.gibba032.whopeo (mgr.24625) 7083 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:06:51.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.619746+0000 mgr.gibba032.whopeo (mgr.24625) 7084 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:06:51.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.621311+0000 mgr.gibba032.whopeo (mgr.24625) 7085 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:06:51.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.621816+0000 mgr.gibba032.whopeo (mgr.24625) 7086 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:06:51.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.622250+0000 mon.a (mon.0) 2554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.623246+0000 mgr.gibba032.whopeo (mgr.24625) 7087 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:06:51.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.623295+0000 mgr.gibba032.whopeo (mgr.24625) 7088 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:06:51.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: cephadm 2021-03-10T16:06:50.623455+0000 mgr.gibba032.whopeo (mgr.24625) 7089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:51.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:51 gibba032 bash[24004]: audit 2021-03-10T16:06:50.624121+0000 mon.a (mon.0) 2555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cluster 2021-03-10T16:06:50.303360+0000 mgr.gibba032.whopeo (mgr.24625) 7073 : cluster [DBG] pgmap v4029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.394105+0000 mgr.gibba032.whopeo (mgr.24625) 7074 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:06:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.395325+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.401335+0000 mgr.gibba032.whopeo (mgr.24625) 7075 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:06:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.402651+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.610399+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.611183+0000 mgr.gibba032.whopeo (mgr.24625) 7076 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:06:51.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.613550+0000 mgr.gibba032.whopeo (mgr.24625) 7077 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:06:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.614003+0000 mon.a (mon.0) 2550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:06:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.615280+0000 mgr.gibba032.whopeo (mgr.24625) 7078 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:06:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.615326+0000 mgr.gibba032.whopeo (mgr.24625) 7079 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:06:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.615824+0000 mon.a (mon.0) 2551 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.616780+0000 mgr.gibba032.whopeo (mgr.24625) 7080 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:06:51.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.616822+0000 mgr.gibba032.whopeo (mgr.24625) 7081 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:06:51.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.617758+0000 mon.a (mon.0) 2552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.618350+0000 mgr.gibba032.whopeo (mgr.24625) 7082 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:06:51.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.619129+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.619700+0000 mgr.gibba032.whopeo (mgr.24625) 7083 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:06:51.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.619746+0000 mgr.gibba032.whopeo (mgr.24625) 7084 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:06:51.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.621311+0000 mgr.gibba032.whopeo (mgr.24625) 7085 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:06:51.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.621816+0000 mgr.gibba032.whopeo (mgr.24625) 7086 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:06:51.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cluster 2021-03-10T16:06:50.303360+0000 mgr.gibba032.whopeo (mgr.24625) 7073 : cluster [DBG] pgmap v4029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:51.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.394105+0000 mgr.gibba032.whopeo (mgr.24625) 7074 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:06:51.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.395325+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.401335+0000 mgr.gibba032.whopeo (mgr.24625) 7075 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:06:51.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.402651+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.610399+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:06:51.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.611183+0000 mgr.gibba032.whopeo (mgr.24625) 7076 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:06:51.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.613550+0000 mgr.gibba032.whopeo (mgr.24625) 7077 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:06:51.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.614003+0000 mon.a (mon.0) 2550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:06:51.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.615280+0000 mgr.gibba032.whopeo (mgr.24625) 7078 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:06:51.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.615326+0000 mgr.gibba032.whopeo (mgr.24625) 7079 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:06:51.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.615824+0000 mon.a (mon.0) 2551 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.616780+0000 mgr.gibba032.whopeo (mgr.24625) 7080 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:06:51.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.616822+0000 mgr.gibba032.whopeo (mgr.24625) 7081 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:06:51.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.617758+0000 mon.a (mon.0) 2552 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.618350+0000 mgr.gibba032.whopeo (mgr.24625) 7082 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:06:51.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.619129+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.619700+0000 mgr.gibba032.whopeo (mgr.24625) 7083 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:06:51.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.619746+0000 mgr.gibba032.whopeo (mgr.24625) 7084 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:06:51.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.621311+0000 mgr.gibba032.whopeo (mgr.24625) 7085 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:06:51.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.621816+0000 mgr.gibba032.whopeo (mgr.24625) 7086 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:06:51.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.622250+0000 mon.a (mon.0) 2554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.623246+0000 mgr.gibba032.whopeo (mgr.24625) 7087 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:06:51.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.623295+0000 mgr.gibba032.whopeo (mgr.24625) 7088 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:06:51.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: cephadm 2021-03-10T16:06:50.623455+0000 mgr.gibba032.whopeo (mgr.24625) 7089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:51.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[31774]: audit 2021-03-10T16:06:50.624121+0000 mon.a (mon.0) 2555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.622250+0000 mon.a (mon.0) 2554 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:51.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.623246+0000 mgr.gibba032.whopeo (mgr.24625) 7087 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:06:51.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.623295+0000 mgr.gibba032.whopeo (mgr.24625) 7088 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:06:51.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: cephadm 2021-03-10T16:06:50.623455+0000 mgr.gibba032.whopeo (mgr.24625) 7089 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:06:51.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:51 gibba011 bash[30791]: audit 2021-03-10T16:06:50.624121+0000 mon.a (mon.0) 2555 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:06:53.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:53 gibba032 bash[24004]: cluster 2021-03-10T16:06:52.304960+0000 mgr.gibba032.whopeo (mgr.24625) 7090 : cluster [DBG] pgmap v4030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:53.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:53 gibba011 bash[30791]: cluster 2021-03-10T16:06:52.304960+0000 mgr.gibba032.whopeo (mgr.24625) 7090 : cluster [DBG] pgmap v4030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:53.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:53 gibba011 bash[31774]: cluster 2021-03-10T16:06:52.304960+0000 mgr.gibba032.whopeo (mgr.24625) 7090 : cluster [DBG] pgmap v4030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:55 gibba032 bash[24004]: cluster 2021-03-10T16:06:54.306410+0000 mgr.gibba032.whopeo (mgr.24625) 7091 : cluster [DBG] pgmap v4031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:55 gibba011 bash[30791]: cluster 2021-03-10T16:06:54.306410+0000 mgr.gibba032.whopeo (mgr.24625) 7091 : cluster [DBG] pgmap v4031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:55 gibba011 bash[31774]: cluster 2021-03-10T16:06:54.306410+0000 mgr.gibba032.whopeo (mgr.24625) 7091 : cluster [DBG] pgmap v4031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: cluster 2021-03-10T16:06:56.308073+0000 mgr.gibba032.whopeo (mgr.24625) 7092 : cluster [DBG] pgmap v4032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.847763+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:06:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.848127+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:06:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.909713+0000 mon.a (mon.0) 2558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:06:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.910551+0000 mon.a (mon.0) 2559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.911285+0000 mon.a (mon.0) 2560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:57 gibba032 bash[24004]: audit 2021-03-10T16:06:56.912012+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:06:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: cluster 2021-03-10T16:06:56.308073+0000 mgr.gibba032.whopeo (mgr.24625) 7092 : cluster [DBG] pgmap v4032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.847763+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.848127+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.909713+0000 mon.a (mon.0) 2558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.910551+0000 mon.a (mon.0) 2559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.911285+0000 mon.a (mon.0) 2560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[30791]: audit 2021-03-10T16:06:56.912012+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: cluster 2021-03-10T16:06:56.308073+0000 mgr.gibba032.whopeo (mgr.24625) 7092 : cluster [DBG] pgmap v4032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.847763+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.848127+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.909713+0000 mon.a (mon.0) 2558 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.910551+0000 mon.a (mon.0) 2559 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.911285+0000 mon.a (mon.0) 2560 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:57 gibba011 bash[31774]: audit 2021-03-10T16:06:56.912012+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:07:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:06:59 gibba032 bash[24004]: cluster 2021-03-10T16:06:58.308708+0000 mgr.gibba032.whopeo (mgr.24625) 7093 : cluster [DBG] pgmap v4033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:06:59 gibba011 bash[30791]: cluster 2021-03-10T16:06:58.308708+0000 mgr.gibba032.whopeo (mgr.24625) 7093 : cluster [DBG] pgmap v4033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:00.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:06:59 gibba011 bash[31774]: cluster 2021-03-10T16:06:58.308708+0000 mgr.gibba032.whopeo (mgr.24625) 7093 : cluster [DBG] pgmap v4033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:01 gibba032 bash[24004]: cluster 2021-03-10T16:07:00.309968+0000 mgr.gibba032.whopeo (mgr.24625) 7094 : cluster [DBG] pgmap v4034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:01 gibba011 bash[30791]: cluster 2021-03-10T16:07:00.309968+0000 mgr.gibba032.whopeo (mgr.24625) 7094 : cluster [DBG] pgmap v4034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:02.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:01 gibba011 bash[31774]: cluster 2021-03-10T16:07:00.309968+0000 mgr.gibba032.whopeo (mgr.24625) 7094 : cluster [DBG] pgmap v4034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:03.447 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:07:03.674 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:07:03.674 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:07:03.674 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:07:03.675 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:07:03.675 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:07:03.675 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:07:03.675 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:07:03.676 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:07:03.676 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:07:03.676 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:07:03.676 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:07:03.676 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:07:03.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:07:03.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:07:03.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:07:03.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:07:03.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:07:03.678 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:07:03.678 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:07:03.678 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:07:03.678 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:07:03.926 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:07:03.926 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:07:03.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:07:03.927 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:03.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:07:03.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:07:03.927 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:03.928 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:07:03.928 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:07:03.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:03.928 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:07:03.928 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:07:03.929 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:07:03.929 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:07:03.929 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:07:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:03 gibba032 bash[24004]: cluster 2021-03-10T16:07:02.311413+0000 mgr.gibba032.whopeo (mgr.24625) 7095 : cluster [DBG] pgmap v4035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:03 gibba011 bash[30791]: cluster 2021-03-10T16:07:02.311413+0000 mgr.gibba032.whopeo (mgr.24625) 7095 : cluster [DBG] pgmap v4035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:03 gibba011 bash[31774]: cluster 2021-03-10T16:07:02.311413+0000 mgr.gibba032.whopeo (mgr.24625) 7095 : cluster [DBG] pgmap v4035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:05.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:04 gibba032 bash[24004]: audit 2021-03-10T16:07:03.437679+0000 mgr.gibba032.whopeo (mgr.24625) 7096 : audit [DBG] from='client.48807 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:04 gibba032 bash[24004]: audit 2021-03-10T16:07:03.663622+0000 mgr.gibba032.whopeo (mgr.24625) 7097 : audit [DBG] from='client.58795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:04 gibba032 bash[24004]: audit 2021-03-10T16:07:03.927138+0000 mon.c (mon.1) 137 : audit [DBG] from='client.? 172.21.2.111:0/783839753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:05.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[30791]: audit 2021-03-10T16:07:03.437679+0000 mgr.gibba032.whopeo (mgr.24625) 7096 : audit [DBG] from='client.48807 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[30791]: audit 2021-03-10T16:07:03.663622+0000 mgr.gibba032.whopeo (mgr.24625) 7097 : audit [DBG] from='client.58795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[30791]: audit 2021-03-10T16:07:03.927138+0000 mon.c (mon.1) 137 : audit [DBG] from='client.? 172.21.2.111:0/783839753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:05.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[31774]: audit 2021-03-10T16:07:03.437679+0000 mgr.gibba032.whopeo (mgr.24625) 7096 : audit [DBG] from='client.48807 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[31774]: audit 2021-03-10T16:07:03.663622+0000 mgr.gibba032.whopeo (mgr.24625) 7097 : audit [DBG] from='client.58795 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:04 gibba011 bash[31774]: audit 2021-03-10T16:07:03.927138+0000 mon.c (mon.1) 137 : audit [DBG] from='client.? 172.21.2.111:0/783839753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:06.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:05 gibba032 bash[24004]: cluster 2021-03-10T16:07:04.312642+0000 mgr.gibba032.whopeo (mgr.24625) 7098 : cluster [DBG] pgmap v4036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:05 gibba011 bash[30791]: cluster 2021-03-10T16:07:04.312642+0000 mgr.gibba032.whopeo (mgr.24625) 7098 : cluster [DBG] pgmap v4036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:05 gibba011 bash[31774]: cluster 2021-03-10T16:07:04.312642+0000 mgr.gibba032.whopeo (mgr.24625) 7098 : cluster [DBG] pgmap v4036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:06 gibba032 bash[24004]: cephadm 2021-03-10T16:07:05.639526+0000 mgr.gibba032.whopeo (mgr.24625) 7099 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:06 gibba011 bash[30791]: cephadm 2021-03-10T16:07:05.639526+0000 mgr.gibba032.whopeo (mgr.24625) 7099 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:07.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:06 gibba011 bash[31774]: cephadm 2021-03-10T16:07:05.639526+0000 mgr.gibba032.whopeo (mgr.24625) 7099 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:07 gibba032 bash[24004]: cluster 2021-03-10T16:07:06.314191+0000 mgr.gibba032.whopeo (mgr.24625) 7100 : cluster [DBG] pgmap v4037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:07 gibba011 bash[30791]: cluster 2021-03-10T16:07:06.314191+0000 mgr.gibba032.whopeo (mgr.24625) 7100 : cluster [DBG] pgmap v4037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:07 gibba011 bash[31774]: cluster 2021-03-10T16:07:06.314191+0000 mgr.gibba032.whopeo (mgr.24625) 7100 : cluster [DBG] pgmap v4037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:09 gibba032 bash[24004]: cluster 2021-03-10T16:07:08.315711+0000 mgr.gibba032.whopeo (mgr.24625) 7101 : cluster [DBG] pgmap v4038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:09 gibba011 bash[30791]: cluster 2021-03-10T16:07:08.315711+0000 mgr.gibba032.whopeo (mgr.24625) 7101 : cluster [DBG] pgmap v4038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:09 gibba011 bash[31774]: cluster 2021-03-10T16:07:08.315711+0000 mgr.gibba032.whopeo (mgr.24625) 7101 : cluster [DBG] pgmap v4038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:11 gibba032 bash[24004]: cluster 2021-03-10T16:07:10.317228+0000 mgr.gibba032.whopeo (mgr.24625) 7102 : cluster [DBG] pgmap v4039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:11 gibba011 bash[30791]: cluster 2021-03-10T16:07:10.317228+0000 mgr.gibba032.whopeo (mgr.24625) 7102 : cluster [DBG] pgmap v4039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:11 gibba011 bash[31774]: cluster 2021-03-10T16:07:10.317228+0000 mgr.gibba032.whopeo (mgr.24625) 7102 : cluster [DBG] pgmap v4039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:13 gibba032 bash[24004]: cluster 2021-03-10T16:07:12.318782+0000 mgr.gibba032.whopeo (mgr.24625) 7103 : cluster [DBG] pgmap v4040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:13 gibba011 bash[30791]: cluster 2021-03-10T16:07:12.318782+0000 mgr.gibba032.whopeo (mgr.24625) 7103 : cluster [DBG] pgmap v4040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:13 gibba011 bash[31774]: cluster 2021-03-10T16:07:12.318782+0000 mgr.gibba032.whopeo (mgr.24625) 7103 : cluster [DBG] pgmap v4040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:15 gibba032 bash[24004]: cluster 2021-03-10T16:07:14.320179+0000 mgr.gibba032.whopeo (mgr.24625) 7104 : cluster [DBG] pgmap v4041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:15 gibba011 bash[30791]: cluster 2021-03-10T16:07:14.320179+0000 mgr.gibba032.whopeo (mgr.24625) 7104 : cluster [DBG] pgmap v4041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:15 gibba011 bash[31774]: cluster 2021-03-10T16:07:14.320179+0000 mgr.gibba032.whopeo (mgr.24625) 7104 : cluster [DBG] pgmap v4041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:17 gibba032 bash[24004]: cluster 2021-03-10T16:07:16.321696+0000 mgr.gibba032.whopeo (mgr.24625) 7105 : cluster [DBG] pgmap v4042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:17 gibba011 bash[30791]: cluster 2021-03-10T16:07:16.321696+0000 mgr.gibba032.whopeo (mgr.24625) 7105 : cluster [DBG] pgmap v4042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:17 gibba011 bash[31774]: cluster 2021-03-10T16:07:16.321696+0000 mgr.gibba032.whopeo (mgr.24625) 7105 : cluster [DBG] pgmap v4042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:19 gibba032 bash[24004]: cluster 2021-03-10T16:07:18.323496+0000 mgr.gibba032.whopeo (mgr.24625) 7106 : cluster [DBG] pgmap v4043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:20.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:19 gibba011 bash[30791]: cluster 2021-03-10T16:07:18.323496+0000 mgr.gibba032.whopeo (mgr.24625) 7106 : cluster [DBG] pgmap v4043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:20.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:19 gibba011 bash[31774]: cluster 2021-03-10T16:07:18.323496+0000 mgr.gibba032.whopeo (mgr.24625) 7106 : cluster [DBG] pgmap v4043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:21 gibba032 bash[24004]: cluster 2021-03-10T16:07:20.324887+0000 mgr.gibba032.whopeo (mgr.24625) 7107 : cluster [DBG] pgmap v4044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:21 gibba032 bash[24004]: cephadm 2021-03-10T16:07:20.652952+0000 mgr.gibba032.whopeo (mgr.24625) 7108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:21 gibba011 bash[30791]: cluster 2021-03-10T16:07:20.324887+0000 mgr.gibba032.whopeo (mgr.24625) 7107 : cluster [DBG] pgmap v4044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:21 gibba011 bash[30791]: cephadm 2021-03-10T16:07:20.652952+0000 mgr.gibba032.whopeo (mgr.24625) 7108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:21 gibba011 bash[31774]: cluster 2021-03-10T16:07:20.324887+0000 mgr.gibba032.whopeo (mgr.24625) 7107 : cluster [DBG] pgmap v4044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:21 gibba011 bash[31774]: cephadm 2021-03-10T16:07:20.652952+0000 mgr.gibba032.whopeo (mgr.24625) 7108 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:23 gibba032 bash[24004]: cluster 2021-03-10T16:07:22.326443+0000 mgr.gibba032.whopeo (mgr.24625) 7109 : cluster [DBG] pgmap v4045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:23 gibba011 bash[30791]: cluster 2021-03-10T16:07:22.326443+0000 mgr.gibba032.whopeo (mgr.24625) 7109 : cluster [DBG] pgmap v4045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:23 gibba011 bash[31774]: cluster 2021-03-10T16:07:22.326443+0000 mgr.gibba032.whopeo (mgr.24625) 7109 : cluster [DBG] pgmap v4045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:26.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:25 gibba032 bash[24004]: cluster 2021-03-10T16:07:24.327900+0000 mgr.gibba032.whopeo (mgr.24625) 7110 : cluster [DBG] pgmap v4046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:25 gibba011 bash[30791]: cluster 2021-03-10T16:07:24.327900+0000 mgr.gibba032.whopeo (mgr.24625) 7110 : cluster [DBG] pgmap v4046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:26.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:25 gibba011 bash[31774]: cluster 2021-03-10T16:07:24.327900+0000 mgr.gibba032.whopeo (mgr.24625) 7110 : cluster [DBG] pgmap v4046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:27 gibba032 bash[24004]: cluster 2021-03-10T16:07:26.329421+0000 mgr.gibba032.whopeo (mgr.24625) 7111 : cluster [DBG] pgmap v4047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:27 gibba011 bash[30791]: cluster 2021-03-10T16:07:26.329421+0000 mgr.gibba032.whopeo (mgr.24625) 7111 : cluster [DBG] pgmap v4047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:27 gibba011 bash[31774]: cluster 2021-03-10T16:07:26.329421+0000 mgr.gibba032.whopeo (mgr.24625) 7111 : cluster [DBG] pgmap v4047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:30 gibba032 bash[24004]: cluster 2021-03-10T16:07:28.331072+0000 mgr.gibba032.whopeo (mgr.24625) 7112 : cluster [DBG] pgmap v4048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:30 gibba011 bash[30791]: cluster 2021-03-10T16:07:28.331072+0000 mgr.gibba032.whopeo (mgr.24625) 7112 : cluster [DBG] pgmap v4048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:30.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:30 gibba011 bash[31774]: cluster 2021-03-10T16:07:28.331072+0000 mgr.gibba032.whopeo (mgr.24625) 7112 : cluster [DBG] pgmap v4048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:32 gibba011 bash[30791]: cluster 2021-03-10T16:07:30.332419+0000 mgr.gibba032.whopeo (mgr.24625) 7113 : cluster [DBG] pgmap v4049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:32 gibba011 bash[31774]: cluster 2021-03-10T16:07:30.332419+0000 mgr.gibba032.whopeo (mgr.24625) 7113 : cluster [DBG] pgmap v4049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:32 gibba032 bash[24004]: cluster 2021-03-10T16:07:30.332419+0000 mgr.gibba032.whopeo (mgr.24625) 7113 : cluster [DBG] pgmap v4049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:34.172 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:07:34.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:34 gibba011 bash[30791]: cluster 2021-03-10T16:07:32.333896+0000 mgr.gibba032.whopeo (mgr.24625) 7114 : cluster [DBG] pgmap v4050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:34.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:34 gibba011 bash[31774]: cluster 2021-03-10T16:07:32.333896+0000 mgr.gibba032.whopeo (mgr.24625) 7114 : cluster [DBG] pgmap v4050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:34.400 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:07:34.400 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:07:34.400 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:07:34.400 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:07:34.401 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:07:34.401 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:07:34.402 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:07:34.402 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:07:34.402 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:07:34.402 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:07:34.403 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:07:34.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:07:34.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:07:34.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:07:34.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:07:34.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:07:34.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:07:34.405 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:07:34.405 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:07:34.405 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:07:34.406 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:07:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:34 gibba032 bash[24004]: cluster 2021-03-10T16:07:32.333896+0000 mgr.gibba032.whopeo (mgr.24625) 7114 : cluster [DBG] pgmap v4050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:34.650 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:07:34.650 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:07:34.651 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:07:34.651 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:34.651 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:07:34.652 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:07:34.652 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:34.652 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:07:34.652 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:07:34.652 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:07:34.653 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:07:34.653 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:07:34.653 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:07:34.653 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:07:34.654 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:07:35.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:35 gibba011 bash[30791]: audit 2021-03-10T16:07:34.161990+0000 mgr.gibba032.whopeo (mgr.24625) 7115 : audit [DBG] from='client.48825 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:35.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:35 gibba011 bash[30791]: audit 2021-03-10T16:07:34.649465+0000 mon.c (mon.1) 138 : audit [DBG] from='client.? 172.21.2.111:0/1620931446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:35 gibba011 bash[31774]: audit 2021-03-10T16:07:34.161990+0000 mgr.gibba032.whopeo (mgr.24625) 7115 : audit [DBG] from='client.48825 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:35 gibba011 bash[31774]: audit 2021-03-10T16:07:34.649465+0000 mon.c (mon.1) 138 : audit [DBG] from='client.? 172.21.2.111:0/1620931446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:35 gibba032 bash[24004]: audit 2021-03-10T16:07:34.161990+0000 mgr.gibba032.whopeo (mgr.24625) 7115 : audit [DBG] from='client.48825 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:35 gibba032 bash[24004]: audit 2021-03-10T16:07:34.649465+0000 mon.c (mon.1) 138 : audit [DBG] from='client.? 172.21.2.111:0/1620931446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:36 gibba011 bash[30791]: cluster 2021-03-10T16:07:34.335357+0000 mgr.gibba032.whopeo (mgr.24625) 7116 : cluster [DBG] pgmap v4051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:36.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:36 gibba011 bash[30791]: audit 2021-03-10T16:07:34.387424+0000 mgr.gibba032.whopeo (mgr.24625) 7117 : audit [DBG] from='client.48831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:36 gibba011 bash[31774]: cluster 2021-03-10T16:07:34.335357+0000 mgr.gibba032.whopeo (mgr.24625) 7116 : cluster [DBG] pgmap v4051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:36 gibba011 bash[31774]: audit 2021-03-10T16:07:34.387424+0000 mgr.gibba032.whopeo (mgr.24625) 7117 : audit [DBG] from='client.48831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:36 gibba032 bash[24004]: cluster 2021-03-10T16:07:34.335357+0000 mgr.gibba032.whopeo (mgr.24625) 7116 : cluster [DBG] pgmap v4051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:36 gibba032 bash[24004]: audit 2021-03-10T16:07:34.387424+0000 mgr.gibba032.whopeo (mgr.24625) 7117 : audit [DBG] from='client.48831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:07:37.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:37 gibba011 bash[30791]: cephadm 2021-03-10T16:07:35.664240+0000 mgr.gibba032.whopeo (mgr.24625) 7118 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:37.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:37 gibba011 bash[31774]: cephadm 2021-03-10T16:07:35.664240+0000 mgr.gibba032.whopeo (mgr.24625) 7118 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:37 gibba032 bash[24004]: cephadm 2021-03-10T16:07:35.664240+0000 mgr.gibba032.whopeo (mgr.24625) 7118 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:38 gibba011 bash[30791]: cluster 2021-03-10T16:07:36.337020+0000 mgr.gibba032.whopeo (mgr.24625) 7119 : cluster [DBG] pgmap v4052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:38 gibba011 bash[31774]: cluster 2021-03-10T16:07:36.337020+0000 mgr.gibba032.whopeo (mgr.24625) 7119 : cluster [DBG] pgmap v4052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:38 gibba032 bash[24004]: cluster 2021-03-10T16:07:36.337020+0000 mgr.gibba032.whopeo (mgr.24625) 7119 : cluster [DBG] pgmap v4052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:40 gibba011 bash[30791]: cluster 2021-03-10T16:07:38.338713+0000 mgr.gibba032.whopeo (mgr.24625) 7120 : cluster [DBG] pgmap v4053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:40 gibba011 bash[31774]: cluster 2021-03-10T16:07:38.338713+0000 mgr.gibba032.whopeo (mgr.24625) 7120 : cluster [DBG] pgmap v4053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:40 gibba032 bash[24004]: cluster 2021-03-10T16:07:38.338713+0000 mgr.gibba032.whopeo (mgr.24625) 7120 : cluster [DBG] pgmap v4053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:42 gibba011 bash[30791]: cluster 2021-03-10T16:07:40.340274+0000 mgr.gibba032.whopeo (mgr.24625) 7121 : cluster [DBG] pgmap v4054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:42 gibba011 bash[31774]: cluster 2021-03-10T16:07:40.340274+0000 mgr.gibba032.whopeo (mgr.24625) 7121 : cluster [DBG] pgmap v4054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:42 gibba032 bash[24004]: cluster 2021-03-10T16:07:40.340274+0000 mgr.gibba032.whopeo (mgr.24625) 7121 : cluster [DBG] pgmap v4054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:44 gibba011 bash[30791]: cluster 2021-03-10T16:07:42.341703+0000 mgr.gibba032.whopeo (mgr.24625) 7122 : cluster [DBG] pgmap v4055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:44 gibba011 bash[31774]: cluster 2021-03-10T16:07:42.341703+0000 mgr.gibba032.whopeo (mgr.24625) 7122 : cluster [DBG] pgmap v4055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:44 gibba032 bash[24004]: cluster 2021-03-10T16:07:42.341703+0000 mgr.gibba032.whopeo (mgr.24625) 7122 : cluster [DBG] pgmap v4055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:46 gibba032 bash[24004]: cluster 2021-03-10T16:07:44.343167+0000 mgr.gibba032.whopeo (mgr.24625) 7123 : cluster [DBG] pgmap v4056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:46 gibba011 bash[30791]: cluster 2021-03-10T16:07:44.343167+0000 mgr.gibba032.whopeo (mgr.24625) 7123 : cluster [DBG] pgmap v4056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:46 gibba011 bash[31774]: cluster 2021-03-10T16:07:44.343167+0000 mgr.gibba032.whopeo (mgr.24625) 7123 : cluster [DBG] pgmap v4056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:48 gibba032 bash[24004]: cluster 2021-03-10T16:07:46.345021+0000 mgr.gibba032.whopeo (mgr.24625) 7124 : cluster [DBG] pgmap v4057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:48 gibba011 bash[30791]: cluster 2021-03-10T16:07:46.345021+0000 mgr.gibba032.whopeo (mgr.24625) 7124 : cluster [DBG] pgmap v4057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:48 gibba011 bash[31774]: cluster 2021-03-10T16:07:46.345021+0000 mgr.gibba032.whopeo (mgr.24625) 7124 : cluster [DBG] pgmap v4057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:50 gibba032 bash[24004]: cluster 2021-03-10T16:07:48.346697+0000 mgr.gibba032.whopeo (mgr.24625) 7125 : cluster [DBG] pgmap v4058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:50 gibba011 bash[30791]: cluster 2021-03-10T16:07:48.346697+0000 mgr.gibba032.whopeo (mgr.24625) 7125 : cluster [DBG] pgmap v4058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:50 gibba011 bash[31774]: cluster 2021-03-10T16:07:48.346697+0000 mgr.gibba032.whopeo (mgr.24625) 7125 : cluster [DBG] pgmap v4058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cluster 2021-03-10T16:07:50.348309+0000 mgr.gibba032.whopeo (mgr.24625) 7126 : cluster [DBG] pgmap v4059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.677497+0000 mgr.gibba032.whopeo (mgr.24625) 7127 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:07:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.679047+0000 mgr.gibba032.whopeo (mgr.24625) 7128 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:07:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.903161+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:07:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.903578+0000 mgr.gibba032.whopeo (mgr.24625) 7129 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:07:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.905650+0000 mgr.gibba032.whopeo (mgr.24625) 7130 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:07:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.906189+0000 mon.a (mon.0) 2563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:07:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.907163+0000 mgr.gibba032.whopeo (mgr.24625) 7131 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:07:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.907200+0000 mgr.gibba032.whopeo (mgr.24625) 7132 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:07:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.908046+0000 mon.a (mon.0) 2564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.908690+0000 mgr.gibba032.whopeo (mgr.24625) 7133 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:07:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.908726+0000 mgr.gibba032.whopeo (mgr.24625) 7134 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:07:52.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.909670+0000 mon.a (mon.0) 2565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.910171+0000 mgr.gibba032.whopeo (mgr.24625) 7135 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:07:52.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.911104+0000 mon.a (mon.0) 2566 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.911947+0000 mgr.gibba032.whopeo (mgr.24625) 7136 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:07:52.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.912058+0000 mgr.gibba032.whopeo (mgr.24625) 7137 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:07:52.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.913556+0000 mgr.gibba032.whopeo (mgr.24625) 7138 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:07:52.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.913878+0000 mgr.gibba032.whopeo (mgr.24625) 7139 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:07:52.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.914534+0000 mon.a (mon.0) 2567 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.914921+0000 mgr.gibba032.whopeo (mgr.24625) 7140 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:07:52.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.914959+0000 mgr.gibba032.whopeo (mgr.24625) 7141 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:07:52.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: cephadm 2021-03-10T16:07:50.915063+0000 mgr.gibba032.whopeo (mgr.24625) 7142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:52.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:51 gibba032 bash[24004]: audit 2021-03-10T16:07:50.916038+0000 mon.a (mon.0) 2568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cluster 2021-03-10T16:07:50.348309+0000 mgr.gibba032.whopeo (mgr.24625) 7126 : cluster [DBG] pgmap v4059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.677497+0000 mgr.gibba032.whopeo (mgr.24625) 7127 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:07:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.679047+0000 mgr.gibba032.whopeo (mgr.24625) 7128 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:07:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.903161+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:07:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.903578+0000 mgr.gibba032.whopeo (mgr.24625) 7129 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:07:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.905650+0000 mgr.gibba032.whopeo (mgr.24625) 7130 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:07:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.906189+0000 mon.a (mon.0) 2563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:07:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.907163+0000 mgr.gibba032.whopeo (mgr.24625) 7131 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:07:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.907200+0000 mgr.gibba032.whopeo (mgr.24625) 7132 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:07:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.908046+0000 mon.a (mon.0) 2564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.908690+0000 mgr.gibba032.whopeo (mgr.24625) 7133 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:07:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.908726+0000 mgr.gibba032.whopeo (mgr.24625) 7134 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:07:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.909670+0000 mon.a (mon.0) 2565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.910171+0000 mgr.gibba032.whopeo (mgr.24625) 7135 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:07:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.911104+0000 mon.a (mon.0) 2566 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.911947+0000 mgr.gibba032.whopeo (mgr.24625) 7136 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:07:52.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.912058+0000 mgr.gibba032.whopeo (mgr.24625) 7137 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:07:52.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.913556+0000 mgr.gibba032.whopeo (mgr.24625) 7138 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:07:52.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.913878+0000 mgr.gibba032.whopeo (mgr.24625) 7139 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:07:52.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.914534+0000 mon.a (mon.0) 2567 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.914921+0000 mgr.gibba032.whopeo (mgr.24625) 7140 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:07:52.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cluster 2021-03-10T16:07:50.348309+0000 mgr.gibba032.whopeo (mgr.24625) 7126 : cluster [DBG] pgmap v4059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:52.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.677497+0000 mgr.gibba032.whopeo (mgr.24625) 7127 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:07:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.679047+0000 mgr.gibba032.whopeo (mgr.24625) 7128 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:07:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.903161+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:07:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.903578+0000 mgr.gibba032.whopeo (mgr.24625) 7129 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:07:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.905650+0000 mgr.gibba032.whopeo (mgr.24625) 7130 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:07:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.906189+0000 mon.a (mon.0) 2563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:07:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.907163+0000 mgr.gibba032.whopeo (mgr.24625) 7131 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:07:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.907200+0000 mgr.gibba032.whopeo (mgr.24625) 7132 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:07:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.908046+0000 mon.a (mon.0) 2564 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.908690+0000 mgr.gibba032.whopeo (mgr.24625) 7133 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:07:52.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.908726+0000 mgr.gibba032.whopeo (mgr.24625) 7134 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:07:52.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.909670+0000 mon.a (mon.0) 2565 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.910171+0000 mgr.gibba032.whopeo (mgr.24625) 7135 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:07:52.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.911104+0000 mon.a (mon.0) 2566 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.911947+0000 mgr.gibba032.whopeo (mgr.24625) 7136 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:07:52.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.912058+0000 mgr.gibba032.whopeo (mgr.24625) 7137 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:07:52.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.913556+0000 mgr.gibba032.whopeo (mgr.24625) 7138 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:07:52.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.913878+0000 mgr.gibba032.whopeo (mgr.24625) 7139 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:07:52.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.914534+0000 mon.a (mon.0) 2567 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.914921+0000 mgr.gibba032.whopeo (mgr.24625) 7140 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:07:52.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.914959+0000 mgr.gibba032.whopeo (mgr.24625) 7141 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:07:52.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: cephadm 2021-03-10T16:07:50.915063+0000 mgr.gibba032.whopeo (mgr.24625) 7142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:52.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[30791]: audit 2021-03-10T16:07:50.916038+0000 mon.a (mon.0) 2568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:52.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.914959+0000 mgr.gibba032.whopeo (mgr.24625) 7141 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:07:52.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: cephadm 2021-03-10T16:07:50.915063+0000 mgr.gibba032.whopeo (mgr.24625) 7142 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:07:52.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:51 gibba011 bash[31774]: audit 2021-03-10T16:07:50.916038+0000 mon.a (mon.0) 2568 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:07:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:53 gibba032 bash[24004]: cluster 2021-03-10T16:07:52.349828+0000 mgr.gibba032.whopeo (mgr.24625) 7143 : cluster [DBG] pgmap v4060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:53 gibba011 bash[30791]: cluster 2021-03-10T16:07:52.349828+0000 mgr.gibba032.whopeo (mgr.24625) 7143 : cluster [DBG] pgmap v4060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:53 gibba011 bash[31774]: cluster 2021-03-10T16:07:52.349828+0000 mgr.gibba032.whopeo (mgr.24625) 7143 : cluster [DBG] pgmap v4060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:55 gibba032 bash[24004]: cluster 2021-03-10T16:07:54.351273+0000 mgr.gibba032.whopeo (mgr.24625) 7144 : cluster [DBG] pgmap v4061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:55 gibba011 bash[30791]: cluster 2021-03-10T16:07:54.351273+0000 mgr.gibba032.whopeo (mgr.24625) 7144 : cluster [DBG] pgmap v4061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:55 gibba011 bash[31774]: cluster 2021-03-10T16:07:54.351273+0000 mgr.gibba032.whopeo (mgr.24625) 7144 : cluster [DBG] pgmap v4061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.853714+0000 mon.a (mon.0) 2569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:07:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.854413+0000 mon.a (mon.0) 2570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:07:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.917925+0000 mon.a (mon.0) 2571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:07:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.918699+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:07:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.919468+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:07:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:56 gibba032 bash[24004]: audit 2021-03-10T16:07:56.920183+0000 mon.a (mon.0) 2574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:07:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.853714+0000 mon.a (mon.0) 2569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:07:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.854413+0000 mon.a (mon.0) 2570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:07:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.917925+0000 mon.a (mon.0) 2571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:07:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.918699+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:07:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.919468+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:07:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[30791]: audit 2021-03-10T16:07:56.920183+0000 mon.a (mon.0) 2574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:07:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.853714+0000 mon.a (mon.0) 2569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:07:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.854413+0000 mon.a (mon.0) 2570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:07:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.917925+0000 mon.a (mon.0) 2571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:07:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.918699+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:07:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.919468+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:07:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:56 gibba011 bash[31774]: audit 2021-03-10T16:07:56.920183+0000 mon.a (mon.0) 2574 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:07:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:57 gibba032 bash[24004]: cluster 2021-03-10T16:07:56.353096+0000 mgr.gibba032.whopeo (mgr.24625) 7145 : cluster [DBG] pgmap v4062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:57 gibba011 bash[30791]: cluster 2021-03-10T16:07:56.353096+0000 mgr.gibba032.whopeo (mgr.24625) 7145 : cluster [DBG] pgmap v4062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:07:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:57 gibba011 bash[31774]: cluster 2021-03-10T16:07:56.353096+0000 mgr.gibba032.whopeo (mgr.24625) 7145 : cluster [DBG] pgmap v4062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:07:59 gibba032 bash[24004]: cluster 2021-03-10T16:07:58.354870+0000 mgr.gibba032.whopeo (mgr.24625) 7146 : cluster [DBG] pgmap v4063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:07:59 gibba011 bash[30791]: cluster 2021-03-10T16:07:58.354870+0000 mgr.gibba032.whopeo (mgr.24625) 7146 : cluster [DBG] pgmap v4063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:00.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:07:59 gibba011 bash[31774]: cluster 2021-03-10T16:07:58.354870+0000 mgr.gibba032.whopeo (mgr.24625) 7146 : cluster [DBG] pgmap v4063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:01 gibba032 bash[24004]: cluster 2021-03-10T16:08:00.356387+0000 mgr.gibba032.whopeo (mgr.24625) 7147 : cluster [DBG] pgmap v4064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:01 gibba011 bash[30791]: cluster 2021-03-10T16:08:00.356387+0000 mgr.gibba032.whopeo (mgr.24625) 7147 : cluster [DBG] pgmap v4064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:02.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:01 gibba011 bash[31774]: cluster 2021-03-10T16:08:00.356387+0000 mgr.gibba032.whopeo (mgr.24625) 7147 : cluster [DBG] pgmap v4064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:03 gibba032 bash[24004]: cluster 2021-03-10T16:08:02.357829+0000 mgr.gibba032.whopeo (mgr.24625) 7148 : cluster [DBG] pgmap v4065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:03 gibba011 bash[30791]: cluster 2021-03-10T16:08:02.357829+0000 mgr.gibba032.whopeo (mgr.24625) 7148 : cluster [DBG] pgmap v4065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:03 gibba011 bash[31774]: cluster 2021-03-10T16:08:02.357829+0000 mgr.gibba032.whopeo (mgr.24625) 7148 : cluster [DBG] pgmap v4065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:04.893 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:08:05.125 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:08:05.125 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:08:05.125 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:08:05.125 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:08:05.126 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:08:05.126 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:08:05.126 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:08:05.126 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:08:05.127 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:08:05.127 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:08:05.127 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:08:05.127 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:08:05.128 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:08:05.128 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:08:05.128 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:08:05.128 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:08:05.128 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:08:05.129 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:08:05.129 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:08:05.129 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:08:05.129 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:08:05.375 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:08:05.375 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:08:05.376 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:08:05.376 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:05.376 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:08:05.376 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:08:05.376 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:05.377 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:08:05.377 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:08:05.377 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:05.377 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:08:05.378 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:08:05.378 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:08:05.378 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:08:05.378 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:08:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:05 gibba032 bash[24004]: cluster 2021-03-10T16:08:04.359088+0000 mgr.gibba032.whopeo (mgr.24625) 7149 : cluster [DBG] pgmap v4066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:05 gibba032 bash[24004]: audit 2021-03-10T16:08:04.883397+0000 mgr.gibba032.whopeo (mgr.24625) 7150 : audit [DBG] from='client.48843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:05 gibba032 bash[24004]: audit 2021-03-10T16:08:05.111418+0000 mgr.gibba032.whopeo (mgr.24625) 7151 : audit [DBG] from='client.58831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:05 gibba032 bash[24004]: audit 2021-03-10T16:08:05.376067+0000 mon.c (mon.1) 139 : audit [DBG] from='client.? 172.21.2.111:0/4152174472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[30791]: cluster 2021-03-10T16:08:04.359088+0000 mgr.gibba032.whopeo (mgr.24625) 7149 : cluster [DBG] pgmap v4066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[30791]: audit 2021-03-10T16:08:04.883397+0000 mgr.gibba032.whopeo (mgr.24625) 7150 : audit [DBG] from='client.48843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[30791]: audit 2021-03-10T16:08:05.111418+0000 mgr.gibba032.whopeo (mgr.24625) 7151 : audit [DBG] from='client.58831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[30791]: audit 2021-03-10T16:08:05.376067+0000 mon.c (mon.1) 139 : audit [DBG] from='client.? 172.21.2.111:0/4152174472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[31774]: cluster 2021-03-10T16:08:04.359088+0000 mgr.gibba032.whopeo (mgr.24625) 7149 : cluster [DBG] pgmap v4066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:06.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[31774]: audit 2021-03-10T16:08:04.883397+0000 mgr.gibba032.whopeo (mgr.24625) 7150 : audit [DBG] from='client.48843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[31774]: audit 2021-03-10T16:08:05.111418+0000 mgr.gibba032.whopeo (mgr.24625) 7151 : audit [DBG] from='client.58831 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:06.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:05 gibba011 bash[31774]: audit 2021-03-10T16:08:05.376067+0000 mon.c (mon.1) 139 : audit [DBG] from='client.? 172.21.2.111:0/4152174472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:06 gibba032 bash[24004]: cephadm 2021-03-10T16:08:05.929578+0000 mgr.gibba032.whopeo (mgr.24625) 7152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:06 gibba011 bash[30791]: cephadm 2021-03-10T16:08:05.929578+0000 mgr.gibba032.whopeo (mgr.24625) 7152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:06 gibba011 bash[31774]: cephadm 2021-03-10T16:08:05.929578+0000 mgr.gibba032.whopeo (mgr.24625) 7152 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:07 gibba032 bash[24004]: cluster 2021-03-10T16:08:06.360752+0000 mgr.gibba032.whopeo (mgr.24625) 7153 : cluster [DBG] pgmap v4067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:07 gibba011 bash[30791]: cluster 2021-03-10T16:08:06.360752+0000 mgr.gibba032.whopeo (mgr.24625) 7153 : cluster [DBG] pgmap v4067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:07 gibba011 bash[31774]: cluster 2021-03-10T16:08:06.360752+0000 mgr.gibba032.whopeo (mgr.24625) 7153 : cluster [DBG] pgmap v4067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:09 gibba032 bash[24004]: cluster 2021-03-10T16:08:08.362412+0000 mgr.gibba032.whopeo (mgr.24625) 7154 : cluster [DBG] pgmap v4068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:09 gibba011 bash[30791]: cluster 2021-03-10T16:08:08.362412+0000 mgr.gibba032.whopeo (mgr.24625) 7154 : cluster [DBG] pgmap v4068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:09 gibba011 bash[31774]: cluster 2021-03-10T16:08:08.362412+0000 mgr.gibba032.whopeo (mgr.24625) 7154 : cluster [DBG] pgmap v4068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:12 gibba032 bash[24004]: cluster 2021-03-10T16:08:10.363912+0000 mgr.gibba032.whopeo (mgr.24625) 7155 : cluster [DBG] pgmap v4069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:12 gibba011 bash[30791]: cluster 2021-03-10T16:08:10.363912+0000 mgr.gibba032.whopeo (mgr.24625) 7155 : cluster [DBG] pgmap v4069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:12 gibba011 bash[31774]: cluster 2021-03-10T16:08:10.363912+0000 mgr.gibba032.whopeo (mgr.24625) 7155 : cluster [DBG] pgmap v4069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:14 gibba011 bash[30791]: cluster 2021-03-10T16:08:12.365381+0000 mgr.gibba032.whopeo (mgr.24625) 7156 : cluster [DBG] pgmap v4070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:14 gibba011 bash[31774]: cluster 2021-03-10T16:08:12.365381+0000 mgr.gibba032.whopeo (mgr.24625) 7156 : cluster [DBG] pgmap v4070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:14 gibba032 bash[24004]: cluster 2021-03-10T16:08:12.365381+0000 mgr.gibba032.whopeo (mgr.24625) 7156 : cluster [DBG] pgmap v4070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:16 gibba011 bash[30791]: cluster 2021-03-10T16:08:14.366630+0000 mgr.gibba032.whopeo (mgr.24625) 7157 : cluster [DBG] pgmap v4071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:16 gibba011 bash[31774]: cluster 2021-03-10T16:08:14.366630+0000 mgr.gibba032.whopeo (mgr.24625) 7157 : cluster [DBG] pgmap v4071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:16 gibba032 bash[24004]: cluster 2021-03-10T16:08:14.366630+0000 mgr.gibba032.whopeo (mgr.24625) 7157 : cluster [DBG] pgmap v4071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:18 gibba011 bash[30791]: cluster 2021-03-10T16:08:16.368284+0000 mgr.gibba032.whopeo (mgr.24625) 7158 : cluster [DBG] pgmap v4072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:18 gibba011 bash[31774]: cluster 2021-03-10T16:08:16.368284+0000 mgr.gibba032.whopeo (mgr.24625) 7158 : cluster [DBG] pgmap v4072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:18 gibba032 bash[24004]: cluster 2021-03-10T16:08:16.368284+0000 mgr.gibba032.whopeo (mgr.24625) 7158 : cluster [DBG] pgmap v4072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:20 gibba011 bash[30791]: cluster 2021-03-10T16:08:18.369955+0000 mgr.gibba032.whopeo (mgr.24625) 7159 : cluster [DBG] pgmap v4073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:20.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:20 gibba011 bash[31774]: cluster 2021-03-10T16:08:18.369955+0000 mgr.gibba032.whopeo (mgr.24625) 7159 : cluster [DBG] pgmap v4073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:20 gibba032 bash[24004]: cluster 2021-03-10T16:08:18.369955+0000 mgr.gibba032.whopeo (mgr.24625) 7159 : cluster [DBG] pgmap v4073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:22 gibba011 bash[30791]: cluster 2021-03-10T16:08:20.371298+0000 mgr.gibba032.whopeo (mgr.24625) 7160 : cluster [DBG] pgmap v4074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:22 gibba011 bash[30791]: cephadm 2021-03-10T16:08:20.945286+0000 mgr.gibba032.whopeo (mgr.24625) 7161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:22 gibba011 bash[31774]: cluster 2021-03-10T16:08:20.371298+0000 mgr.gibba032.whopeo (mgr.24625) 7160 : cluster [DBG] pgmap v4074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:22 gibba011 bash[31774]: cephadm 2021-03-10T16:08:20.945286+0000 mgr.gibba032.whopeo (mgr.24625) 7161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:22 gibba032 bash[24004]: cluster 2021-03-10T16:08:20.371298+0000 mgr.gibba032.whopeo (mgr.24625) 7160 : cluster [DBG] pgmap v4074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:22 gibba032 bash[24004]: cephadm 2021-03-10T16:08:20.945286+0000 mgr.gibba032.whopeo (mgr.24625) 7161 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:24 gibba011 bash[30791]: cluster 2021-03-10T16:08:22.372981+0000 mgr.gibba032.whopeo (mgr.24625) 7162 : cluster [DBG] pgmap v4075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:24 gibba011 bash[31774]: cluster 2021-03-10T16:08:22.372981+0000 mgr.gibba032.whopeo (mgr.24625) 7162 : cluster [DBG] pgmap v4075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:24 gibba032 bash[24004]: cluster 2021-03-10T16:08:22.372981+0000 mgr.gibba032.whopeo (mgr.24625) 7162 : cluster [DBG] pgmap v4075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:26 gibba011 bash[30791]: cluster 2021-03-10T16:08:24.374467+0000 mgr.gibba032.whopeo (mgr.24625) 7163 : cluster [DBG] pgmap v4076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:26.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:26 gibba011 bash[31774]: cluster 2021-03-10T16:08:24.374467+0000 mgr.gibba032.whopeo (mgr.24625) 7163 : cluster [DBG] pgmap v4076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:26 gibba032 bash[24004]: cluster 2021-03-10T16:08:24.374467+0000 mgr.gibba032.whopeo (mgr.24625) 7163 : cluster [DBG] pgmap v4076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:28 gibba011 bash[30791]: cluster 2021-03-10T16:08:26.376251+0000 mgr.gibba032.whopeo (mgr.24625) 7164 : cluster [DBG] pgmap v4077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:28 gibba011 bash[31774]: cluster 2021-03-10T16:08:26.376251+0000 mgr.gibba032.whopeo (mgr.24625) 7164 : cluster [DBG] pgmap v4077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:28 gibba032 bash[24004]: cluster 2021-03-10T16:08:26.376251+0000 mgr.gibba032.whopeo (mgr.24625) 7164 : cluster [DBG] pgmap v4077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:30 gibba011 bash[30791]: cluster 2021-03-10T16:08:28.377996+0000 mgr.gibba032.whopeo (mgr.24625) 7165 : cluster [DBG] pgmap v4078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:30 gibba011 bash[31774]: cluster 2021-03-10T16:08:28.377996+0000 mgr.gibba032.whopeo (mgr.24625) 7165 : cluster [DBG] pgmap v4078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:30 gibba032 bash[24004]: cluster 2021-03-10T16:08:28.377996+0000 mgr.gibba032.whopeo (mgr.24625) 7165 : cluster [DBG] pgmap v4078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:32 gibba032 bash[24004]: cluster 2021-03-10T16:08:30.379341+0000 mgr.gibba032.whopeo (mgr.24625) 7166 : cluster [DBG] pgmap v4079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:32 gibba011 bash[30791]: cluster 2021-03-10T16:08:30.379341+0000 mgr.gibba032.whopeo (mgr.24625) 7166 : cluster [DBG] pgmap v4079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:32 gibba011 bash[31774]: cluster 2021-03-10T16:08:30.379341+0000 mgr.gibba032.whopeo (mgr.24625) 7166 : cluster [DBG] pgmap v4079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:34 gibba032 bash[24004]: cluster 2021-03-10T16:08:32.381129+0000 mgr.gibba032.whopeo (mgr.24625) 7167 : cluster [DBG] pgmap v4080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:34.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:34 gibba011 bash[30791]: cluster 2021-03-10T16:08:32.381129+0000 mgr.gibba032.whopeo (mgr.24625) 7167 : cluster [DBG] pgmap v4080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:34 gibba011 bash[31774]: cluster 2021-03-10T16:08:32.381129+0000 mgr.gibba032.whopeo (mgr.24625) 7167 : cluster [DBG] pgmap v4080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:35.619 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:08:35.845 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:08:35.845 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:08:35.845 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:08:35.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:08:35.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:08:35.846 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:08:35.846 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:08:35.846 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:08:35.847 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:08:35.847 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:08:35.847 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:08:35.847 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:08:35.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:08:35.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:08:35.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:08:35.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:08:35.848 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:08:35.849 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:08:35.849 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:08:35.849 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:08:35.849 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:08:36.093 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:08:36.093 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:08:36.094 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:08:36.094 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:36.094 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:08:36.095 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:08:36.095 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:36.095 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:08:36.095 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:08:36.096 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:08:36.096 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:08:36.096 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:08:36.096 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:08:36.097 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:08:36.097 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:08:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:36 gibba032 bash[24004]: cluster 2021-03-10T16:08:34.382553+0000 mgr.gibba032.whopeo (mgr.24625) 7168 : cluster [DBG] pgmap v4081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:36 gibba032 bash[24004]: audit 2021-03-10T16:08:36.094244+0000 mon.a (mon.0) 2575 : audit [DBG] from='client.? 172.21.2.111:0/2806608258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:36 gibba011 bash[30791]: cluster 2021-03-10T16:08:34.382553+0000 mgr.gibba032.whopeo (mgr.24625) 7168 : cluster [DBG] pgmap v4081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:36 gibba011 bash[30791]: audit 2021-03-10T16:08:36.094244+0000 mon.a (mon.0) 2575 : audit [DBG] from='client.? 172.21.2.111:0/2806608258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:36 gibba011 bash[31774]: cluster 2021-03-10T16:08:34.382553+0000 mgr.gibba032.whopeo (mgr.24625) 7168 : cluster [DBG] pgmap v4081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:36 gibba011 bash[31774]: audit 2021-03-10T16:08:36.094244+0000 mon.a (mon.0) 2575 : audit [DBG] from='client.? 172.21.2.111:0/2806608258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:37.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:37 gibba032 bash[24004]: audit 2021-03-10T16:08:35.610071+0000 mgr.gibba032.whopeo (mgr.24625) 7169 : audit [DBG] from='client.58843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:37 gibba032 bash[24004]: audit 2021-03-10T16:08:35.831360+0000 mgr.gibba032.whopeo (mgr.24625) 7170 : audit [DBG] from='client.58849 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:37 gibba032 bash[24004]: cephadm 2021-03-10T16:08:35.960945+0000 mgr.gibba032.whopeo (mgr.24625) 7171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:37.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[30791]: audit 2021-03-10T16:08:35.610071+0000 mgr.gibba032.whopeo (mgr.24625) 7169 : audit [DBG] from='client.58843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[30791]: audit 2021-03-10T16:08:35.831360+0000 mgr.gibba032.whopeo (mgr.24625) 7170 : audit [DBG] from='client.58849 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[30791]: cephadm 2021-03-10T16:08:35.960945+0000 mgr.gibba032.whopeo (mgr.24625) 7171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:37.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[31774]: audit 2021-03-10T16:08:35.610071+0000 mgr.gibba032.whopeo (mgr.24625) 7169 : audit [DBG] from='client.58843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[31774]: audit 2021-03-10T16:08:35.831360+0000 mgr.gibba032.whopeo (mgr.24625) 7170 : audit [DBG] from='client.58849 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:08:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:37 gibba011 bash[31774]: cephadm 2021-03-10T16:08:35.960945+0000 mgr.gibba032.whopeo (mgr.24625) 7171 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:38 gibba032 bash[24004]: cluster 2021-03-10T16:08:36.384393+0000 mgr.gibba032.whopeo (mgr.24625) 7172 : cluster [DBG] pgmap v4082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:38 gibba011 bash[30791]: cluster 2021-03-10T16:08:36.384393+0000 mgr.gibba032.whopeo (mgr.24625) 7172 : cluster [DBG] pgmap v4082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:38 gibba011 bash[31774]: cluster 2021-03-10T16:08:36.384393+0000 mgr.gibba032.whopeo (mgr.24625) 7172 : cluster [DBG] pgmap v4082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:40 gibba032 bash[24004]: cluster 2021-03-10T16:08:38.386196+0000 mgr.gibba032.whopeo (mgr.24625) 7173 : cluster [DBG] pgmap v4083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:40 gibba011 bash[30791]: cluster 2021-03-10T16:08:38.386196+0000 mgr.gibba032.whopeo (mgr.24625) 7173 : cluster [DBG] pgmap v4083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:40 gibba011 bash[31774]: cluster 2021-03-10T16:08:38.386196+0000 mgr.gibba032.whopeo (mgr.24625) 7173 : cluster [DBG] pgmap v4083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:42 gibba032 bash[24004]: cluster 2021-03-10T16:08:40.387683+0000 mgr.gibba032.whopeo (mgr.24625) 7174 : cluster [DBG] pgmap v4084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:42 gibba011 bash[30791]: cluster 2021-03-10T16:08:40.387683+0000 mgr.gibba032.whopeo (mgr.24625) 7174 : cluster [DBG] pgmap v4084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:42 gibba011 bash[31774]: cluster 2021-03-10T16:08:40.387683+0000 mgr.gibba032.whopeo (mgr.24625) 7174 : cluster [DBG] pgmap v4084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:44 gibba032 bash[24004]: cluster 2021-03-10T16:08:42.389298+0000 mgr.gibba032.whopeo (mgr.24625) 7175 : cluster [DBG] pgmap v4085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:44.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:44 gibba011 bash[30791]: cluster 2021-03-10T16:08:42.389298+0000 mgr.gibba032.whopeo (mgr.24625) 7175 : cluster [DBG] pgmap v4085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:44 gibba011 bash[31774]: cluster 2021-03-10T16:08:42.389298+0000 mgr.gibba032.whopeo (mgr.24625) 7175 : cluster [DBG] pgmap v4085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:46 gibba032 bash[24004]: cluster 2021-03-10T16:08:44.390512+0000 mgr.gibba032.whopeo (mgr.24625) 7176 : cluster [DBG] pgmap v4086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:46 gibba011 bash[30791]: cluster 2021-03-10T16:08:44.390512+0000 mgr.gibba032.whopeo (mgr.24625) 7176 : cluster [DBG] pgmap v4086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:46 gibba011 bash[31774]: cluster 2021-03-10T16:08:44.390512+0000 mgr.gibba032.whopeo (mgr.24625) 7176 : cluster [DBG] pgmap v4086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:48 gibba032 bash[24004]: cluster 2021-03-10T16:08:46.392129+0000 mgr.gibba032.whopeo (mgr.24625) 7177 : cluster [DBG] pgmap v4087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:48 gibba011 bash[30791]: cluster 2021-03-10T16:08:46.392129+0000 mgr.gibba032.whopeo (mgr.24625) 7177 : cluster [DBG] pgmap v4087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:48 gibba011 bash[31774]: cluster 2021-03-10T16:08:46.392129+0000 mgr.gibba032.whopeo (mgr.24625) 7177 : cluster [DBG] pgmap v4087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:50 gibba032 bash[24004]: cluster 2021-03-10T16:08:48.393473+0000 mgr.gibba032.whopeo (mgr.24625) 7178 : cluster [DBG] pgmap v4088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:50 gibba011 bash[30791]: cluster 2021-03-10T16:08:48.393473+0000 mgr.gibba032.whopeo (mgr.24625) 7178 : cluster [DBG] pgmap v4088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:50 gibba011 bash[31774]: cluster 2021-03-10T16:08:48.393473+0000 mgr.gibba032.whopeo (mgr.24625) 7178 : cluster [DBG] pgmap v4088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cluster 2021-03-10T16:08:50.395041+0000 mgr.gibba032.whopeo (mgr.24625) 7179 : cluster [DBG] pgmap v4089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:50.973546+0000 mgr.gibba032.whopeo (mgr.24625) 7180 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:08:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:50.975123+0000 mgr.gibba032.whopeo (mgr.24625) 7181 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:08:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.245084+0000 mon.a (mon.0) 2576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:08:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.245803+0000 mgr.gibba032.whopeo (mgr.24625) 7182 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:08:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.248334+0000 mgr.gibba032.whopeo (mgr.24625) 7183 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:08:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.248796+0000 mon.a (mon.0) 2577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:08:52.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.250296+0000 mgr.gibba032.whopeo (mgr.24625) 7184 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:08:52.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.250343+0000 mgr.gibba032.whopeo (mgr.24625) 7185 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:08:52.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.250914+0000 mon.a (mon.0) 2578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.252363+0000 mgr.gibba032.whopeo (mgr.24625) 7186 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:08:52.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.252412+0000 mgr.gibba032.whopeo (mgr.24625) 7187 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:08:52.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.252927+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.253953+0000 mgr.gibba032.whopeo (mgr.24625) 7188 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:08:52.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.254915+0000 mon.a (mon.0) 2580 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.255702+0000 mgr.gibba032.whopeo (mgr.24625) 7189 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:08:52.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.255749+0000 mgr.gibba032.whopeo (mgr.24625) 7190 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:08:52.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.257432+0000 mgr.gibba032.whopeo (mgr.24625) 7191 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:08:52.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.257888+0000 mgr.gibba032.whopeo (mgr.24625) 7192 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:08:52.518 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.258298+0000 mon.a (mon.0) 2581 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.520 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.259605+0000 mgr.gibba032.whopeo (mgr.24625) 7193 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:08:52.521 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.259660+0000 mgr.gibba032.whopeo (mgr.24625) 7194 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:08:52.521 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: cephadm 2021-03-10T16:08:51.259808+0000 mgr.gibba032.whopeo (mgr.24625) 7195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:52.522 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:52 gibba032 bash[24004]: audit 2021-03-10T16:08:51.260265+0000 mon.a (mon.0) 2582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cluster 2021-03-10T16:08:50.395041+0000 mgr.gibba032.whopeo (mgr.24625) 7179 : cluster [DBG] pgmap v4089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:52.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:50.973546+0000 mgr.gibba032.whopeo (mgr.24625) 7180 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:08:52.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:50.975123+0000 mgr.gibba032.whopeo (mgr.24625) 7181 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:08:52.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.245084+0000 mon.a (mon.0) 2576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:08:52.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.245803+0000 mgr.gibba032.whopeo (mgr.24625) 7182 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:08:52.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.248334+0000 mgr.gibba032.whopeo (mgr.24625) 7183 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:08:52.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.248796+0000 mon.a (mon.0) 2577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:08:52.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.250296+0000 mgr.gibba032.whopeo (mgr.24625) 7184 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:08:52.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.250343+0000 mgr.gibba032.whopeo (mgr.24625) 7185 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:08:52.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.250914+0000 mon.a (mon.0) 2578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.252363+0000 mgr.gibba032.whopeo (mgr.24625) 7186 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:08:52.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.252412+0000 mgr.gibba032.whopeo (mgr.24625) 7187 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:08:52.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.252927+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.253953+0000 mgr.gibba032.whopeo (mgr.24625) 7188 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:08:52.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.254915+0000 mon.a (mon.0) 2580 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.255702+0000 mgr.gibba032.whopeo (mgr.24625) 7189 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:08:52.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.255749+0000 mgr.gibba032.whopeo (mgr.24625) 7190 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:08:52.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.257432+0000 mgr.gibba032.whopeo (mgr.24625) 7191 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:08:52.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.257888+0000 mgr.gibba032.whopeo (mgr.24625) 7192 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:08:52.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.258298+0000 mon.a (mon.0) 2581 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.259605+0000 mgr.gibba032.whopeo (mgr.24625) 7193 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:08:52.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cluster 2021-03-10T16:08:50.395041+0000 mgr.gibba032.whopeo (mgr.24625) 7179 : cluster [DBG] pgmap v4089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:52.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:50.973546+0000 mgr.gibba032.whopeo (mgr.24625) 7180 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:08:52.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:50.975123+0000 mgr.gibba032.whopeo (mgr.24625) 7181 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:08:52.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.245084+0000 mon.a (mon.0) 2576 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:08:52.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.245803+0000 mgr.gibba032.whopeo (mgr.24625) 7182 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:08:52.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.248334+0000 mgr.gibba032.whopeo (mgr.24625) 7183 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:08:52.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.248796+0000 mon.a (mon.0) 2577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:08:52.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.250296+0000 mgr.gibba032.whopeo (mgr.24625) 7184 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:08:52.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.250343+0000 mgr.gibba032.whopeo (mgr.24625) 7185 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:08:52.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.250914+0000 mon.a (mon.0) 2578 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.252363+0000 mgr.gibba032.whopeo (mgr.24625) 7186 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:08:52.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.252412+0000 mgr.gibba032.whopeo (mgr.24625) 7187 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:08:52.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.252927+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.253953+0000 mgr.gibba032.whopeo (mgr.24625) 7188 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:08:52.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.254915+0000 mon.a (mon.0) 2580 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.255702+0000 mgr.gibba032.whopeo (mgr.24625) 7189 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:08:52.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.255749+0000 mgr.gibba032.whopeo (mgr.24625) 7190 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:08:52.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.257432+0000 mgr.gibba032.whopeo (mgr.24625) 7191 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:08:52.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.257888+0000 mgr.gibba032.whopeo (mgr.24625) 7192 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:08:52.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.258298+0000 mon.a (mon.0) 2581 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.259605+0000 mgr.gibba032.whopeo (mgr.24625) 7193 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:08:52.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.259660+0000 mgr.gibba032.whopeo (mgr.24625) 7194 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:08:52.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: cephadm 2021-03-10T16:08:51.259808+0000 mgr.gibba032.whopeo (mgr.24625) 7195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:52.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[30791]: audit 2021-03-10T16:08:51.260265+0000 mon.a (mon.0) 2582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:52.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.259660+0000 mgr.gibba032.whopeo (mgr.24625) 7194 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:08:52.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: cephadm 2021-03-10T16:08:51.259808+0000 mgr.gibba032.whopeo (mgr.24625) 7195 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:08:52.596 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:52 gibba011 bash[31774]: audit 2021-03-10T16:08:51.260265+0000 mon.a (mon.0) 2582 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:08:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:54 gibba011 bash[30791]: cluster 2021-03-10T16:08:52.396508+0000 mgr.gibba032.whopeo (mgr.24625) 7196 : cluster [DBG] pgmap v4090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:54 gibba011 bash[31774]: cluster 2021-03-10T16:08:52.396508+0000 mgr.gibba032.whopeo (mgr.24625) 7196 : cluster [DBG] pgmap v4090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:54 gibba032 bash[24004]: cluster 2021-03-10T16:08:52.396508+0000 mgr.gibba032.whopeo (mgr.24625) 7196 : cluster [DBG] pgmap v4090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:56 gibba011 bash[30791]: cluster 2021-03-10T16:08:54.398004+0000 mgr.gibba032.whopeo (mgr.24625) 7197 : cluster [DBG] pgmap v4091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:56 gibba011 bash[31774]: cluster 2021-03-10T16:08:54.398004+0000 mgr.gibba032.whopeo (mgr.24625) 7197 : cluster [DBG] pgmap v4091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:56 gibba032 bash[24004]: cluster 2021-03-10T16:08:54.398004+0000 mgr.gibba032.whopeo (mgr.24625) 7197 : cluster [DBG] pgmap v4091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.861508+0000 mon.a (mon.0) 2583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:08:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.862197+0000 mon.a (mon.0) 2584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:08:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.926512+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:08:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.927303+0000 mon.a (mon.0) 2586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:08:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.928039+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T16:08:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[30791]: audit 2021-03-10T16:08:56.928734+0000 mon.a (mon.0) 2588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:08:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.861508+0000 mon.a (mon.0) 2583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:08:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.862197+0000 mon.a (mon.0) 2584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:08:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.926512+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:08:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.927303+0000 mon.a (mon.0) 2586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:08:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.928039+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T16:08:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:57 gibba011 bash[31774]: audit 2021-03-10T16:08:56.928734+0000 mon.a (mon.0) 2588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:08:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.861508+0000 mon.a (mon.0) 2583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:08:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.862197+0000 mon.a (mon.0) 2584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:08:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.926512+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:08:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.927303+0000 mon.a (mon.0) 2586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:08:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.928039+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T16:08:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:57 gibba032 bash[24004]: audit 2021-03-10T16:08:56.928734+0000 mon.a (mon.0) 2588 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:08:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:58 gibba011 bash[30791]: cluster 2021-03-10T16:08:56.399755+0000 mgr.gibba032.whopeo (mgr.24625) 7198 : cluster [DBG] pgmap v4092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:58 gibba011 bash[31774]: cluster 2021-03-10T16:08:56.399755+0000 mgr.gibba032.whopeo (mgr.24625) 7198 : cluster [DBG] pgmap v4092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:08:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:58 gibba032 bash[24004]: cluster 2021-03-10T16:08:56.399755+0000 mgr.gibba032.whopeo (mgr.24625) 7198 : cluster [DBG] pgmap v4092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:08:59 gibba032 bash[24004]: cluster 2021-03-10T16:08:58.401404+0000 mgr.gibba032.whopeo (mgr.24625) 7199 : cluster [DBG] pgmap v4093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:08:59 gibba011 bash[30791]: cluster 2021-03-10T16:08:58.401404+0000 mgr.gibba032.whopeo (mgr.24625) 7199 : cluster [DBG] pgmap v4093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:00.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:08:59 gibba011 bash[31774]: cluster 2021-03-10T16:08:58.401404+0000 mgr.gibba032.whopeo (mgr.24625) 7199 : cluster [DBG] pgmap v4093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:01 gibba032 bash[24004]: cluster 2021-03-10T16:09:00.402851+0000 mgr.gibba032.whopeo (mgr.24625) 7200 : cluster [DBG] pgmap v4094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:01 gibba011 bash[30791]: cluster 2021-03-10T16:09:00.402851+0000 mgr.gibba032.whopeo (mgr.24625) 7200 : cluster [DBG] pgmap v4094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:01 gibba011 bash[31774]: cluster 2021-03-10T16:09:00.402851+0000 mgr.gibba032.whopeo (mgr.24625) 7200 : cluster [DBG] pgmap v4094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:03 gibba032 bash[24004]: cluster 2021-03-10T16:09:02.404460+0000 mgr.gibba032.whopeo (mgr.24625) 7201 : cluster [DBG] pgmap v4095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:03 gibba011 bash[30791]: cluster 2021-03-10T16:09:02.404460+0000 mgr.gibba032.whopeo (mgr.24625) 7201 : cluster [DBG] pgmap v4095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:03 gibba011 bash[31774]: cluster 2021-03-10T16:09:02.404460+0000 mgr.gibba032.whopeo (mgr.24625) 7201 : cluster [DBG] pgmap v4095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:06.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:05 gibba032 bash[24004]: cluster 2021-03-10T16:09:04.405942+0000 mgr.gibba032.whopeo (mgr.24625) 7202 : cluster [DBG] pgmap v4096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:06.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:05 gibba011 bash[30791]: cluster 2021-03-10T16:09:04.405942+0000 mgr.gibba032.whopeo (mgr.24625) 7202 : cluster [DBG] pgmap v4096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:06.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:05 gibba011 bash[31774]: cluster 2021-03-10T16:09:04.405942+0000 mgr.gibba032.whopeo (mgr.24625) 7202 : cluster [DBG] pgmap v4096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:06.346 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:09:06.590 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:09:06.590 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:09:06.590 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:09:06.590 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:09:06.591 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:09:06.591 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:09:06.591 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:09:06.591 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:09:06.592 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:09:06.592 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:09:06.592 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:09:06.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:09:06.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:09:06.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:09:06.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:09:06.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:09:06.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:09:06.593 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:09:06.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:09:06.594 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:09:06.594 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:09:06.841 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:09:06.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:09:06.842 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:09:06.842 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:06.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:09:06.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:09:06.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:06.843 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:09:06.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:09:06.844 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:06.844 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:09:06.844 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:09:06.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:09:06.844 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:09:06.845 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:09:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:06 gibba032 bash[24004]: cephadm 2021-03-10T16:09:06.273050+0000 mgr.gibba032.whopeo (mgr.24625) 7203 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:06 gibba032 bash[24004]: audit 2021-03-10T16:09:06.842633+0000 mon.a (mon.0) 2589 : audit [DBG] from='client.? 172.21.2.111:0/973872119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:06 gibba011 bash[30791]: cephadm 2021-03-10T16:09:06.273050+0000 mgr.gibba032.whopeo (mgr.24625) 7203 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:06 gibba011 bash[30791]: audit 2021-03-10T16:09:06.842633+0000 mon.a (mon.0) 2589 : audit [DBG] from='client.? 172.21.2.111:0/973872119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:07.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:06 gibba011 bash[31774]: cephadm 2021-03-10T16:09:06.273050+0000 mgr.gibba032.whopeo (mgr.24625) 7203 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:06 gibba011 bash[31774]: audit 2021-03-10T16:09:06.842633+0000 mon.a (mon.0) 2589 : audit [DBG] from='client.? 172.21.2.111:0/973872119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:07 gibba032 bash[24004]: audit 2021-03-10T16:09:06.327129+0000 mgr.gibba032.whopeo (mgr.24625) 7204 : audit [DBG] from='client.58861 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:07 gibba032 bash[24004]: cluster 2021-03-10T16:09:06.407813+0000 mgr.gibba032.whopeo (mgr.24625) 7205 : cluster [DBG] pgmap v4097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:08.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:07 gibba032 bash[24004]: audit 2021-03-10T16:09:06.575539+0000 mgr.gibba032.whopeo (mgr.24625) 7206 : audit [DBG] from='client.58867 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[30791]: audit 2021-03-10T16:09:06.327129+0000 mgr.gibba032.whopeo (mgr.24625) 7204 : audit [DBG] from='client.58861 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[30791]: cluster 2021-03-10T16:09:06.407813+0000 mgr.gibba032.whopeo (mgr.24625) 7205 : cluster [DBG] pgmap v4097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[30791]: audit 2021-03-10T16:09:06.575539+0000 mgr.gibba032.whopeo (mgr.24625) 7206 : audit [DBG] from='client.58867 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[31774]: audit 2021-03-10T16:09:06.327129+0000 mgr.gibba032.whopeo (mgr.24625) 7204 : audit [DBG] from='client.58861 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[31774]: cluster 2021-03-10T16:09:06.407813+0000 mgr.gibba032.whopeo (mgr.24625) 7205 : cluster [DBG] pgmap v4097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:07 gibba011 bash[31774]: audit 2021-03-10T16:09:06.575539+0000 mgr.gibba032.whopeo (mgr.24625) 7206 : audit [DBG] from='client.58867 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:10.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:09 gibba032 bash[24004]: cluster 2021-03-10T16:09:08.409337+0000 mgr.gibba032.whopeo (mgr.24625) 7207 : cluster [DBG] pgmap v4098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:09 gibba011 bash[30791]: cluster 2021-03-10T16:09:08.409337+0000 mgr.gibba032.whopeo (mgr.24625) 7207 : cluster [DBG] pgmap v4098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:10.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:09 gibba011 bash[31774]: cluster 2021-03-10T16:09:08.409337+0000 mgr.gibba032.whopeo (mgr.24625) 7207 : cluster [DBG] pgmap v4098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:12.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:11 gibba032 bash[24004]: cluster 2021-03-10T16:09:10.410805+0000 mgr.gibba032.whopeo (mgr.24625) 7208 : cluster [DBG] pgmap v4099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:11 gibba011 bash[30791]: cluster 2021-03-10T16:09:10.410805+0000 mgr.gibba032.whopeo (mgr.24625) 7208 : cluster [DBG] pgmap v4099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:11 gibba011 bash[31774]: cluster 2021-03-10T16:09:10.410805+0000 mgr.gibba032.whopeo (mgr.24625) 7208 : cluster [DBG] pgmap v4099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:13 gibba032 bash[24004]: cluster 2021-03-10T16:09:12.412568+0000 mgr.gibba032.whopeo (mgr.24625) 7209 : cluster [DBG] pgmap v4100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:13 gibba011 bash[30791]: cluster 2021-03-10T16:09:12.412568+0000 mgr.gibba032.whopeo (mgr.24625) 7209 : cluster [DBG] pgmap v4100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:13 gibba011 bash[31774]: cluster 2021-03-10T16:09:12.412568+0000 mgr.gibba032.whopeo (mgr.24625) 7209 : cluster [DBG] pgmap v4100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:15 gibba032 bash[24004]: cluster 2021-03-10T16:09:14.413859+0000 mgr.gibba032.whopeo (mgr.24625) 7210 : cluster [DBG] pgmap v4101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:15 gibba011 bash[30791]: cluster 2021-03-10T16:09:14.413859+0000 mgr.gibba032.whopeo (mgr.24625) 7210 : cluster [DBG] pgmap v4101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:15 gibba011 bash[31774]: cluster 2021-03-10T16:09:14.413859+0000 mgr.gibba032.whopeo (mgr.24625) 7210 : cluster [DBG] pgmap v4101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:17 gibba032 bash[24004]: cluster 2021-03-10T16:09:16.415764+0000 mgr.gibba032.whopeo (mgr.24625) 7211 : cluster [DBG] pgmap v4102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:17 gibba011 bash[30791]: cluster 2021-03-10T16:09:16.415764+0000 mgr.gibba032.whopeo (mgr.24625) 7211 : cluster [DBG] pgmap v4102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:17 gibba011 bash[31774]: cluster 2021-03-10T16:09:16.415764+0000 mgr.gibba032.whopeo (mgr.24625) 7211 : cluster [DBG] pgmap v4102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:19 gibba032 bash[24004]: cluster 2021-03-10T16:09:18.417339+0000 mgr.gibba032.whopeo (mgr.24625) 7212 : cluster [DBG] pgmap v4103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:19 gibba011 bash[30791]: cluster 2021-03-10T16:09:18.417339+0000 mgr.gibba032.whopeo (mgr.24625) 7212 : cluster [DBG] pgmap v4103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:20.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:20 gibba011 bash[31774]: cluster 2021-03-10T16:09:18.417339+0000 mgr.gibba032.whopeo (mgr.24625) 7212 : cluster [DBG] pgmap v4103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:21 gibba032 bash[24004]: cluster 2021-03-10T16:09:20.418839+0000 mgr.gibba032.whopeo (mgr.24625) 7213 : cluster [DBG] pgmap v4104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:21 gibba011 bash[30791]: cluster 2021-03-10T16:09:20.418839+0000 mgr.gibba032.whopeo (mgr.24625) 7213 : cluster [DBG] pgmap v4104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:21 gibba011 bash[31774]: cluster 2021-03-10T16:09:20.418839+0000 mgr.gibba032.whopeo (mgr.24625) 7213 : cluster [DBG] pgmap v4104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:23.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:23 gibba011 bash[30791]: cephadm 2021-03-10T16:09:21.288765+0000 mgr.gibba032.whopeo (mgr.24625) 7214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:23.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:23 gibba011 bash[31774]: cephadm 2021-03-10T16:09:21.288765+0000 mgr.gibba032.whopeo (mgr.24625) 7214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:23 gibba032 bash[24004]: cephadm 2021-03-10T16:09:21.288765+0000 mgr.gibba032.whopeo (mgr.24625) 7214 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:24 gibba011 bash[30791]: cluster 2021-03-10T16:09:22.420573+0000 mgr.gibba032.whopeo (mgr.24625) 7215 : cluster [DBG] pgmap v4105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:24 gibba011 bash[31774]: cluster 2021-03-10T16:09:22.420573+0000 mgr.gibba032.whopeo (mgr.24625) 7215 : cluster [DBG] pgmap v4105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:24 gibba032 bash[24004]: cluster 2021-03-10T16:09:22.420573+0000 mgr.gibba032.whopeo (mgr.24625) 7215 : cluster [DBG] pgmap v4105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:26.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:26 gibba011 bash[30791]: cluster 2021-03-10T16:09:24.422071+0000 mgr.gibba032.whopeo (mgr.24625) 7216 : cluster [DBG] pgmap v4106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:26.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:26 gibba011 bash[31774]: cluster 2021-03-10T16:09:24.422071+0000 mgr.gibba032.whopeo (mgr.24625) 7216 : cluster [DBG] pgmap v4106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:26 gibba032 bash[24004]: cluster 2021-03-10T16:09:24.422071+0000 mgr.gibba032.whopeo (mgr.24625) 7216 : cluster [DBG] pgmap v4106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:28 gibba011 bash[30791]: cluster 2021-03-10T16:09:26.423853+0000 mgr.gibba032.whopeo (mgr.24625) 7217 : cluster [DBG] pgmap v4107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:28 gibba011 bash[31774]: cluster 2021-03-10T16:09:26.423853+0000 mgr.gibba032.whopeo (mgr.24625) 7217 : cluster [DBG] pgmap v4107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:28 gibba032 bash[24004]: cluster 2021-03-10T16:09:26.423853+0000 mgr.gibba032.whopeo (mgr.24625) 7217 : cluster [DBG] pgmap v4107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:30 gibba011 bash[30791]: cluster 2021-03-10T16:09:28.425562+0000 mgr.gibba032.whopeo (mgr.24625) 7218 : cluster [DBG] pgmap v4108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:30.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:30 gibba011 bash[31774]: cluster 2021-03-10T16:09:28.425562+0000 mgr.gibba032.whopeo (mgr.24625) 7218 : cluster [DBG] pgmap v4108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:30 gibba032 bash[24004]: cluster 2021-03-10T16:09:28.425562+0000 mgr.gibba032.whopeo (mgr.24625) 7218 : cluster [DBG] pgmap v4108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:32 gibba011 bash[30791]: cluster 2021-03-10T16:09:30.427071+0000 mgr.gibba032.whopeo (mgr.24625) 7219 : cluster [DBG] pgmap v4109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:32.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:32 gibba011 bash[31774]: cluster 2021-03-10T16:09:30.427071+0000 mgr.gibba032.whopeo (mgr.24625) 7219 : cluster [DBG] pgmap v4109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:32 gibba032 bash[24004]: cluster 2021-03-10T16:09:30.427071+0000 mgr.gibba032.whopeo (mgr.24625) 7219 : cluster [DBG] pgmap v4109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:34 gibba011 bash[30791]: cluster 2021-03-10T16:09:32.428746+0000 mgr.gibba032.whopeo (mgr.24625) 7220 : cluster [DBG] pgmap v4110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:34.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:34 gibba011 bash[31774]: cluster 2021-03-10T16:09:32.428746+0000 mgr.gibba032.whopeo (mgr.24625) 7220 : cluster [DBG] pgmap v4110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:34 gibba032 bash[24004]: cluster 2021-03-10T16:09:32.428746+0000 mgr.gibba032.whopeo (mgr.24625) 7220 : cluster [DBG] pgmap v4110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:36.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:36 gibba011 bash[30791]: cluster 2021-03-10T16:09:34.430296+0000 mgr.gibba032.whopeo (mgr.24625) 7221 : cluster [DBG] pgmap v4111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:36.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:36 gibba011 bash[31774]: cluster 2021-03-10T16:09:34.430296+0000 mgr.gibba032.whopeo (mgr.24625) 7221 : cluster [DBG] pgmap v4111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:36 gibba032 bash[24004]: cluster 2021-03-10T16:09:34.430296+0000 mgr.gibba032.whopeo (mgr.24625) 7221 : cluster [DBG] pgmap v4111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:37.083 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:09:37.337 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:09:37.337 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:09:37.337 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:09:37.338 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:09:37.338 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:09:37.338 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:09:37.339 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:09:37.339 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:09:37.339 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:09:37.339 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:09:37.340 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:09:37.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:09:37.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:09:37.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:09:37.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:09:37.341 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:09:37.341 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:09:37.341 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:09:37.341 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:09:37.342 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:09:37.342 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:09:37.587 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:09:37.588 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:09:37.588 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:09:37.588 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:37.589 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:09:37.589 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:09:37.589 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:37.589 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:09:37.590 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:09:37.590 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:09:37.590 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:09:37.590 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:09:37.591 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:09:37.591 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:09:37.591 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:09:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[30791]: cephadm 2021-03-10T16:09:36.303236+0000 mgr.gibba032.whopeo (mgr.24625) 7222 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[30791]: cluster 2021-03-10T16:09:36.432152+0000 mgr.gibba032.whopeo (mgr.24625) 7223 : cluster [DBG] pgmap v4112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[30791]: audit 2021-03-10T16:09:37.074280+0000 mgr.gibba032.whopeo (mgr.24625) 7224 : audit [DBG] from='client.58879 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:38.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[30791]: audit 2021-03-10T16:09:37.588715+0000 mon.a (mon.0) 2590 : audit [DBG] from='client.? 172.21.2.111:0/3819856836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[31774]: cephadm 2021-03-10T16:09:36.303236+0000 mgr.gibba032.whopeo (mgr.24625) 7222 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[31774]: cluster 2021-03-10T16:09:36.432152+0000 mgr.gibba032.whopeo (mgr.24625) 7223 : cluster [DBG] pgmap v4112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[31774]: audit 2021-03-10T16:09:37.074280+0000 mgr.gibba032.whopeo (mgr.24625) 7224 : audit [DBG] from='client.58879 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:38.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:38 gibba011 bash[31774]: audit 2021-03-10T16:09:37.588715+0000 mon.a (mon.0) 2590 : audit [DBG] from='client.? 172.21.2.111:0/3819856836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:38 gibba032 bash[24004]: cephadm 2021-03-10T16:09:36.303236+0000 mgr.gibba032.whopeo (mgr.24625) 7222 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:38 gibba032 bash[24004]: cluster 2021-03-10T16:09:36.432152+0000 mgr.gibba032.whopeo (mgr.24625) 7223 : cluster [DBG] pgmap v4112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:38.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:38 gibba032 bash[24004]: audit 2021-03-10T16:09:37.074280+0000 mgr.gibba032.whopeo (mgr.24625) 7224 : audit [DBG] from='client.58879 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:38.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:38 gibba032 bash[24004]: audit 2021-03-10T16:09:37.588715+0000 mon.a (mon.0) 2590 : audit [DBG] from='client.? 172.21.2.111:0/3819856836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:39.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:39 gibba032 bash[24004]: audit 2021-03-10T16:09:37.296691+0000 mgr.gibba032.whopeo (mgr.24625) 7225 : audit [DBG] from='client.48903 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:39.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:39 gibba011 bash[30791]: audit 2021-03-10T16:09:37.296691+0000 mgr.gibba032.whopeo (mgr.24625) 7225 : audit [DBG] from='client.48903 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:39.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:39 gibba011 bash[31774]: audit 2021-03-10T16:09:37.296691+0000 mgr.gibba032.whopeo (mgr.24625) 7225 : audit [DBG] from='client.48903 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:09:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:40 gibba032 bash[24004]: cluster 2021-03-10T16:09:38.433781+0000 mgr.gibba032.whopeo (mgr.24625) 7226 : cluster [DBG] pgmap v4113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:40 gibba011 bash[30791]: cluster 2021-03-10T16:09:38.433781+0000 mgr.gibba032.whopeo (mgr.24625) 7226 : cluster [DBG] pgmap v4113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:40 gibba011 bash[31774]: cluster 2021-03-10T16:09:38.433781+0000 mgr.gibba032.whopeo (mgr.24625) 7226 : cluster [DBG] pgmap v4113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:42 gibba032 bash[24004]: cluster 2021-03-10T16:09:40.435387+0000 mgr.gibba032.whopeo (mgr.24625) 7227 : cluster [DBG] pgmap v4114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:42 gibba011 bash[30791]: cluster 2021-03-10T16:09:40.435387+0000 mgr.gibba032.whopeo (mgr.24625) 7227 : cluster [DBG] pgmap v4114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:42 gibba011 bash[31774]: cluster 2021-03-10T16:09:40.435387+0000 mgr.gibba032.whopeo (mgr.24625) 7227 : cluster [DBG] pgmap v4114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:44 gibba032 bash[24004]: cluster 2021-03-10T16:09:42.436994+0000 mgr.gibba032.whopeo (mgr.24625) 7228 : cluster [DBG] pgmap v4115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:44.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:44 gibba011 bash[30791]: cluster 2021-03-10T16:09:42.436994+0000 mgr.gibba032.whopeo (mgr.24625) 7228 : cluster [DBG] pgmap v4115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:44.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:44 gibba011 bash[31774]: cluster 2021-03-10T16:09:42.436994+0000 mgr.gibba032.whopeo (mgr.24625) 7228 : cluster [DBG] pgmap v4115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:46 gibba032 bash[24004]: cluster 2021-03-10T16:09:44.438528+0000 mgr.gibba032.whopeo (mgr.24625) 7229 : cluster [DBG] pgmap v4116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:46 gibba011 bash[30791]: cluster 2021-03-10T16:09:44.438528+0000 mgr.gibba032.whopeo (mgr.24625) 7229 : cluster [DBG] pgmap v4116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:46 gibba011 bash[31774]: cluster 2021-03-10T16:09:44.438528+0000 mgr.gibba032.whopeo (mgr.24625) 7229 : cluster [DBG] pgmap v4116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:48 gibba032 bash[24004]: cluster 2021-03-10T16:09:46.440216+0000 mgr.gibba032.whopeo (mgr.24625) 7230 : cluster [DBG] pgmap v4117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:48 gibba011 bash[30791]: cluster 2021-03-10T16:09:46.440216+0000 mgr.gibba032.whopeo (mgr.24625) 7230 : cluster [DBG] pgmap v4117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:48 gibba011 bash[31774]: cluster 2021-03-10T16:09:46.440216+0000 mgr.gibba032.whopeo (mgr.24625) 7230 : cluster [DBG] pgmap v4117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:50 gibba032 bash[24004]: cluster 2021-03-10T16:09:48.441886+0000 mgr.gibba032.whopeo (mgr.24625) 7231 : cluster [DBG] pgmap v4118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:50 gibba011 bash[30791]: cluster 2021-03-10T16:09:48.441886+0000 mgr.gibba032.whopeo (mgr.24625) 7231 : cluster [DBG] pgmap v4118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:50 gibba011 bash[31774]: cluster 2021-03-10T16:09:48.441886+0000 mgr.gibba032.whopeo (mgr.24625) 7231 : cluster [DBG] pgmap v4118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:52 gibba032 bash[24004]: cluster 2021-03-10T16:09:50.443527+0000 mgr.gibba032.whopeo (mgr.24625) 7232 : cluster [DBG] pgmap v4119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:52 gibba032 bash[24004]: audit 2021-03-10T16:09:51.316857+0000 mon.a (mon.0) 2591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:09:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:52 gibba011 bash[30791]: cluster 2021-03-10T16:09:50.443527+0000 mgr.gibba032.whopeo (mgr.24625) 7232 : cluster [DBG] pgmap v4119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:52 gibba011 bash[30791]: audit 2021-03-10T16:09:51.316857+0000 mon.a (mon.0) 2591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:09:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:52 gibba011 bash[31774]: cluster 2021-03-10T16:09:50.443527+0000 mgr.gibba032.whopeo (mgr.24625) 7232 : cluster [DBG] pgmap v4119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:52 gibba011 bash[31774]: audit 2021-03-10T16:09:51.316857+0000 mon.a (mon.0) 2591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:09:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:53 gibba032 bash[24004]: cephadm 2021-03-10T16:09:51.314092+0000 mgr.gibba032.whopeo (mgr.24625) 7233 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:09:53.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:53 gibba011 bash[30791]: cephadm 2021-03-10T16:09:51.314092+0000 mgr.gibba032.whopeo (mgr.24625) 7233 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:09:53.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:53 gibba011 bash[31774]: cephadm 2021-03-10T16:09:51.314092+0000 mgr.gibba032.whopeo (mgr.24625) 7233 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:09:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:54 gibba032 bash[24004]: cluster 2021-03-10T16:09:52.445306+0000 mgr.gibba032.whopeo (mgr.24625) 7234 : cluster [DBG] pgmap v4120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:54 gibba011 bash[30791]: cluster 2021-03-10T16:09:52.445306+0000 mgr.gibba032.whopeo (mgr.24625) 7234 : cluster [DBG] pgmap v4120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:54 gibba011 bash[31774]: cluster 2021-03-10T16:09:52.445306+0000 mgr.gibba032.whopeo (mgr.24625) 7234 : cluster [DBG] pgmap v4120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cluster 2021-03-10T16:09:54.446881+0000 mgr.gibba032.whopeo (mgr.24625) 7235 : cluster [DBG] pgmap v4121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:54.927409+0000 mgr.gibba032.whopeo (mgr.24625) 7236 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:09:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:54.928807+0000 mon.a (mon.0) 2592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.138333+0000 mon.a (mon.0) 2593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.143014+0000 mgr.gibba032.whopeo (mgr.24625) 7237 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:09:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.145399+0000 mgr.gibba032.whopeo (mgr.24625) 7238 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:09:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.145783+0000 mon.a (mon.0) 2594 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:09:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.146934+0000 mgr.gibba032.whopeo (mgr.24625) 7239 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:09:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.146974+0000 mgr.gibba032.whopeo (mgr.24625) 7240 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:09:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.147899+0000 mon.a (mon.0) 2595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.148794+0000 mgr.gibba032.whopeo (mgr.24625) 7241 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:09:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.148832+0000 mgr.gibba032.whopeo (mgr.24625) 7242 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:09:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.149808+0000 mon.a (mon.0) 2596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.151410+0000 mgr.gibba032.whopeo (mgr.24625) 7243 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:09:56.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.152268+0000 mon.a (mon.0) 2597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.152589+0000 mgr.gibba032.whopeo (mgr.24625) 7244 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:09:56.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.152629+0000 mgr.gibba032.whopeo (mgr.24625) 7245 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:09:56.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.153916+0000 mgr.gibba032.whopeo (mgr.24625) 7246 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:09:56.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.154378+0000 mgr.gibba032.whopeo (mgr.24625) 7247 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:09:56.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.155032+0000 mon.a (mon.0) 2598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.155476+0000 mgr.gibba032.whopeo (mgr.24625) 7248 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:09:56.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.155571+0000 mgr.gibba032.whopeo (mgr.24625) 7249 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:09:56.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: cephadm 2021-03-10T16:09:55.155702+0000 mgr.gibba032.whopeo (mgr.24625) 7250 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:56.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:55 gibba032 bash[24004]: audit 2021-03-10T16:09:55.156639+0000 mon.a (mon.0) 2599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cluster 2021-03-10T16:09:54.446881+0000 mgr.gibba032.whopeo (mgr.24625) 7235 : cluster [DBG] pgmap v4121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:54.927409+0000 mgr.gibba032.whopeo (mgr.24625) 7236 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:09:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:54.928807+0000 mon.a (mon.0) 2592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.138333+0000 mon.a (mon.0) 2593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.143014+0000 mgr.gibba032.whopeo (mgr.24625) 7237 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:09:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.145399+0000 mgr.gibba032.whopeo (mgr.24625) 7238 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:09:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.145783+0000 mon.a (mon.0) 2594 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:09:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.146934+0000 mgr.gibba032.whopeo (mgr.24625) 7239 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:09:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.146974+0000 mgr.gibba032.whopeo (mgr.24625) 7240 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:09:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.147899+0000 mon.a (mon.0) 2595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.148794+0000 mgr.gibba032.whopeo (mgr.24625) 7241 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:09:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.148832+0000 mgr.gibba032.whopeo (mgr.24625) 7242 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:09:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.149808+0000 mon.a (mon.0) 2596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.151410+0000 mgr.gibba032.whopeo (mgr.24625) 7243 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:09:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.152268+0000 mon.a (mon.0) 2597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.152589+0000 mgr.gibba032.whopeo (mgr.24625) 7244 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:09:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.152629+0000 mgr.gibba032.whopeo (mgr.24625) 7245 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:09:56.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.153916+0000 mgr.gibba032.whopeo (mgr.24625) 7246 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:09:56.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.154378+0000 mgr.gibba032.whopeo (mgr.24625) 7247 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:09:56.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.155032+0000 mon.a (mon.0) 2598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.155476+0000 mgr.gibba032.whopeo (mgr.24625) 7248 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:09:56.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cluster 2021-03-10T16:09:54.446881+0000 mgr.gibba032.whopeo (mgr.24625) 7235 : cluster [DBG] pgmap v4121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:56.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:54.927409+0000 mgr.gibba032.whopeo (mgr.24625) 7236 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:09:56.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:54.928807+0000 mon.a (mon.0) 2592 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.138333+0000 mon.a (mon.0) 2593 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:09:56.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.143014+0000 mgr.gibba032.whopeo (mgr.24625) 7237 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:09:56.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.145399+0000 mgr.gibba032.whopeo (mgr.24625) 7238 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:09:56.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.145783+0000 mon.a (mon.0) 2594 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:09:56.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.146934+0000 mgr.gibba032.whopeo (mgr.24625) 7239 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:09:56.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.146974+0000 mgr.gibba032.whopeo (mgr.24625) 7240 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:09:56.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.147899+0000 mon.a (mon.0) 2595 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.148794+0000 mgr.gibba032.whopeo (mgr.24625) 7241 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:09:56.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.148832+0000 mgr.gibba032.whopeo (mgr.24625) 7242 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:09:56.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.149808+0000 mon.a (mon.0) 2596 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.151410+0000 mgr.gibba032.whopeo (mgr.24625) 7243 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:09:56.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.152268+0000 mon.a (mon.0) 2597 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.152589+0000 mgr.gibba032.whopeo (mgr.24625) 7244 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:09:56.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.152629+0000 mgr.gibba032.whopeo (mgr.24625) 7245 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:09:56.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.153916+0000 mgr.gibba032.whopeo (mgr.24625) 7246 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:09:56.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.154378+0000 mgr.gibba032.whopeo (mgr.24625) 7247 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:09:56.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.155032+0000 mon.a (mon.0) 2598 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.155476+0000 mgr.gibba032.whopeo (mgr.24625) 7248 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:09:56.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.155571+0000 mgr.gibba032.whopeo (mgr.24625) 7249 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:09:56.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: cephadm 2021-03-10T16:09:55.155702+0000 mgr.gibba032.whopeo (mgr.24625) 7250 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:56.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[30791]: audit 2021-03-10T16:09:55.156639+0000 mon.a (mon.0) 2599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:56.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.155571+0000 mgr.gibba032.whopeo (mgr.24625) 7249 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:09:56.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: cephadm 2021-03-10T16:09:55.155702+0000 mgr.gibba032.whopeo (mgr.24625) 7250 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:09:56.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:55 gibba011 bash[31774]: audit 2021-03-10T16:09:55.156639+0000 mon.a (mon.0) 2599 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:09:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:56 gibba032 bash[24004]: audit 2021-03-10T16:09:56.868249+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:09:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:56 gibba032 bash[24004]: audit 2021-03-10T16:09:56.868918+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:09:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:56 gibba011 bash[30791]: audit 2021-03-10T16:09:56.868249+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:09:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:56 gibba011 bash[30791]: audit 2021-03-10T16:09:56.868918+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:09:57.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:56 gibba011 bash[31774]: audit 2021-03-10T16:09:56.868249+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:09:57.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:56 gibba011 bash[31774]: audit 2021-03-10T16:09:56.868918+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:09:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:57 gibba032 bash[24004]: cluster 2021-03-10T16:09:56.448702+0000 mgr.gibba032.whopeo (mgr.24625) 7251 : cluster [DBG] pgmap v4122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:57 gibba032 bash[24004]: audit 2021-03-10T16:09:56.941488+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:09:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:57 gibba032 bash[24004]: audit 2021-03-10T16:09:56.941735+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:09:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:57 gibba032 bash[24004]: audit 2021-03-10T16:09:56.941927+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:09:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:57 gibba032 bash[24004]: audit 2021-03-10T16:09:56.942128+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:09:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[30791]: cluster 2021-03-10T16:09:56.448702+0000 mgr.gibba032.whopeo (mgr.24625) 7251 : cluster [DBG] pgmap v4122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[30791]: audit 2021-03-10T16:09:56.941488+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:09:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[30791]: audit 2021-03-10T16:09:56.941735+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:09:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[30791]: audit 2021-03-10T16:09:56.941927+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:09:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[30791]: audit 2021-03-10T16:09:56.942128+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:09:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[31774]: cluster 2021-03-10T16:09:56.448702+0000 mgr.gibba032.whopeo (mgr.24625) 7251 : cluster [DBG] pgmap v4122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:09:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[31774]: audit 2021-03-10T16:09:56.941488+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:09:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[31774]: audit 2021-03-10T16:09:56.941735+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:09:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[31774]: audit 2021-03-10T16:09:56.941927+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:09:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:57 gibba011 bash[31774]: audit 2021-03-10T16:09:56.942128+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:10:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:09:59 gibba032 bash[24004]: cluster 2021-03-10T16:09:58.450402+0000 mgr.gibba032.whopeo (mgr.24625) 7252 : cluster [DBG] pgmap v4123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:09:59 gibba011 bash[30791]: cluster 2021-03-10T16:09:58.450402+0000 mgr.gibba032.whopeo (mgr.24625) 7252 : cluster [DBG] pgmap v4123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:00 gibba011 bash[30791]: debug 2021-03-10T16:09:59.998+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:10:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:09:59 gibba011 bash[31774]: cluster 2021-03-10T16:09:58.450402+0000 mgr.gibba032.whopeo (mgr.24625) 7252 : cluster [DBG] pgmap v4123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:01.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:00 gibba032 bash[24004]: cluster 2021-03-10T16:10:00.000149+0000 mon.a (mon.0) 2606 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:10:01.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:00 gibba011 bash[30791]: cluster 2021-03-10T16:10:00.000149+0000 mon.a (mon.0) 2606 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:10:01.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:00 gibba011 bash[31774]: cluster 2021-03-10T16:10:00.000149+0000 mon.a (mon.0) 2606 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:10:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:01 gibba032 bash[24004]: cluster 2021-03-10T16:10:00.451968+0000 mgr.gibba032.whopeo (mgr.24625) 7253 : cluster [DBG] pgmap v4124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:01 gibba011 bash[30791]: cluster 2021-03-10T16:10:00.451968+0000 mgr.gibba032.whopeo (mgr.24625) 7253 : cluster [DBG] pgmap v4124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:02.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:01 gibba011 bash[31774]: cluster 2021-03-10T16:10:00.451968+0000 mgr.gibba032.whopeo (mgr.24625) 7253 : cluster [DBG] pgmap v4124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:03 gibba032 bash[24004]: cluster 2021-03-10T16:10:02.453486+0000 mgr.gibba032.whopeo (mgr.24625) 7254 : cluster [DBG] pgmap v4125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:03 gibba011 bash[30791]: cluster 2021-03-10T16:10:02.453486+0000 mgr.gibba032.whopeo (mgr.24625) 7254 : cluster [DBG] pgmap v4125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:03 gibba011 bash[31774]: cluster 2021-03-10T16:10:02.453486+0000 mgr.gibba032.whopeo (mgr.24625) 7254 : cluster [DBG] pgmap v4125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:05 gibba032 bash[24004]: cluster 2021-03-10T16:10:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 7255 : cluster [DBG] pgmap v4126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:05 gibba011 bash[30791]: cluster 2021-03-10T16:10:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 7255 : cluster [DBG] pgmap v4126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:05 gibba011 bash[31774]: cluster 2021-03-10T16:10:04.455006+0000 mgr.gibba032.whopeo (mgr.24625) 7255 : cluster [DBG] pgmap v4126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:07.832 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:10:08.094 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:10:08.095 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 13s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:10:08.095 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:10:08.095 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:10:08.095 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:10:08.096 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:10:08.096 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:10:08.096 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:10:08.096 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:10:08.097 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 13s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:10:08.097 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:10:08.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:10:08.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:10:08.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:10:08.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:10:08.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:10:08.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:10:08.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:10:08.099 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:10:08.099 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:10:08.099 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 13s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:10:08.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:07 gibba032 bash[24004]: cluster 2021-03-10T16:10:06.456803+0000 mgr.gibba032.whopeo (mgr.24625) 7256 : cluster [DBG] pgmap v4127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:08.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:07 gibba011 bash[30791]: cluster 2021-03-10T16:10:06.456803+0000 mgr.gibba032.whopeo (mgr.24625) 7256 : cluster [DBG] pgmap v4127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:07 gibba011 bash[31774]: cluster 2021-03-10T16:10:06.456803+0000 mgr.gibba032.whopeo (mgr.24625) 7256 : cluster [DBG] pgmap v4127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:08.353 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:10:08.353 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:10:08.353 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:10:08.353 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:08.354 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:10:08.354 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:10:08.354 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:08.355 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:10:08.355 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:10:08.355 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:08.355 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:10:08.355 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:10:08.356 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:10:08.356 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:10:08.356 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:10:09.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:08 gibba032 bash[24004]: audit 2021-03-10T16:10:07.822204+0000 mgr.gibba032.whopeo (mgr.24625) 7257 : audit [DBG] from='client.58897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:08 gibba032 bash[24004]: audit 2021-03-10T16:10:08.054361+0000 mgr.gibba032.whopeo (mgr.24625) 7258 : audit [DBG] from='client.48921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:08 gibba032 bash[24004]: audit 2021-03-10T16:10:08.354278+0000 mon.a (mon.0) 2607 : audit [DBG] from='client.? 172.21.2.111:0/974271306' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:09.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:08 gibba011 bash[30791]: audit 2021-03-10T16:10:07.822204+0000 mgr.gibba032.whopeo (mgr.24625) 7257 : audit [DBG] from='client.58897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:08 gibba011 bash[30791]: audit 2021-03-10T16:10:08.054361+0000 mgr.gibba032.whopeo (mgr.24625) 7258 : audit [DBG] from='client.48921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:09 gibba011 bash[30791]: audit 2021-03-10T16:10:08.354278+0000 mon.a (mon.0) 2607 : audit [DBG] from='client.? 172.21.2.111:0/974271306' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:09 gibba011 bash[31774]: audit 2021-03-10T16:10:07.822204+0000 mgr.gibba032.whopeo (mgr.24625) 7257 : audit [DBG] from='client.58897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:09 gibba011 bash[31774]: audit 2021-03-10T16:10:08.054361+0000 mgr.gibba032.whopeo (mgr.24625) 7258 : audit [DBG] from='client.48921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:09.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:09 gibba011 bash[31774]: audit 2021-03-10T16:10:08.354278+0000 mon.a (mon.0) 2607 : audit [DBG] from='client.? 172.21.2.111:0/974271306' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:10 gibba011 bash[30791]: cluster 2021-03-10T16:10:08.458220+0000 mgr.gibba032.whopeo (mgr.24625) 7259 : cluster [DBG] pgmap v4128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:10.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:10 gibba011 bash[31774]: cluster 2021-03-10T16:10:08.458220+0000 mgr.gibba032.whopeo (mgr.24625) 7259 : cluster [DBG] pgmap v4128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:10 gibba032 bash[24004]: cluster 2021-03-10T16:10:08.458220+0000 mgr.gibba032.whopeo (mgr.24625) 7259 : cluster [DBG] pgmap v4128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:11 gibba011 bash[30791]: cephadm 2021-03-10T16:10:10.171616+0000 mgr.gibba032.whopeo (mgr.24625) 7260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:11.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:11 gibba011 bash[31774]: cephadm 2021-03-10T16:10:10.171616+0000 mgr.gibba032.whopeo (mgr.24625) 7260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:11.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:11 gibba032 bash[24004]: cephadm 2021-03-10T16:10:10.171616+0000 mgr.gibba032.whopeo (mgr.24625) 7260 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:12 gibba011 bash[30791]: cluster 2021-03-10T16:10:10.459817+0000 mgr.gibba032.whopeo (mgr.24625) 7261 : cluster [DBG] pgmap v4129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:12 gibba011 bash[31774]: cluster 2021-03-10T16:10:10.459817+0000 mgr.gibba032.whopeo (mgr.24625) 7261 : cluster [DBG] pgmap v4129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:12 gibba032 bash[24004]: cluster 2021-03-10T16:10:10.459817+0000 mgr.gibba032.whopeo (mgr.24625) 7261 : cluster [DBG] pgmap v4129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:14 gibba011 bash[30791]: cluster 2021-03-10T16:10:12.461320+0000 mgr.gibba032.whopeo (mgr.24625) 7262 : cluster [DBG] pgmap v4130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:14 gibba011 bash[31774]: cluster 2021-03-10T16:10:12.461320+0000 mgr.gibba032.whopeo (mgr.24625) 7262 : cluster [DBG] pgmap v4130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:14 gibba032 bash[24004]: cluster 2021-03-10T16:10:12.461320+0000 mgr.gibba032.whopeo (mgr.24625) 7262 : cluster [DBG] pgmap v4130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:16 gibba011 bash[30791]: cluster 2021-03-10T16:10:14.462847+0000 mgr.gibba032.whopeo (mgr.24625) 7263 : cluster [DBG] pgmap v4131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:16 gibba011 bash[31774]: cluster 2021-03-10T16:10:14.462847+0000 mgr.gibba032.whopeo (mgr.24625) 7263 : cluster [DBG] pgmap v4131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:16 gibba032 bash[24004]: cluster 2021-03-10T16:10:14.462847+0000 mgr.gibba032.whopeo (mgr.24625) 7263 : cluster [DBG] pgmap v4131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:18 gibba011 bash[30791]: cluster 2021-03-10T16:10:16.464712+0000 mgr.gibba032.whopeo (mgr.24625) 7264 : cluster [DBG] pgmap v4132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:18 gibba011 bash[31774]: cluster 2021-03-10T16:10:16.464712+0000 mgr.gibba032.whopeo (mgr.24625) 7264 : cluster [DBG] pgmap v4132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:18 gibba032 bash[24004]: cluster 2021-03-10T16:10:16.464712+0000 mgr.gibba032.whopeo (mgr.24625) 7264 : cluster [DBG] pgmap v4132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:20 gibba011 bash[30791]: cluster 2021-03-10T16:10:18.466332+0000 mgr.gibba032.whopeo (mgr.24625) 7265 : cluster [DBG] pgmap v4133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:20 gibba011 bash[31774]: cluster 2021-03-10T16:10:18.466332+0000 mgr.gibba032.whopeo (mgr.24625) 7265 : cluster [DBG] pgmap v4133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:20 gibba032 bash[24004]: cluster 2021-03-10T16:10:18.466332+0000 mgr.gibba032.whopeo (mgr.24625) 7265 : cluster [DBG] pgmap v4133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:22 gibba011 bash[30791]: cluster 2021-03-10T16:10:20.468001+0000 mgr.gibba032.whopeo (mgr.24625) 7266 : cluster [DBG] pgmap v4134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:22.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:22 gibba011 bash[31774]: cluster 2021-03-10T16:10:20.468001+0000 mgr.gibba032.whopeo (mgr.24625) 7266 : cluster [DBG] pgmap v4134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:22 gibba032 bash[24004]: cluster 2021-03-10T16:10:20.468001+0000 mgr.gibba032.whopeo (mgr.24625) 7266 : cluster [DBG] pgmap v4134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:24 gibba011 bash[30791]: cluster 2021-03-10T16:10:22.469605+0000 mgr.gibba032.whopeo (mgr.24625) 7267 : cluster [DBG] pgmap v4135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:24 gibba011 bash[31774]: cluster 2021-03-10T16:10:22.469605+0000 mgr.gibba032.whopeo (mgr.24625) 7267 : cluster [DBG] pgmap v4135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:24 gibba032 bash[24004]: cluster 2021-03-10T16:10:22.469605+0000 mgr.gibba032.whopeo (mgr.24625) 7267 : cluster [DBG] pgmap v4135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:26 gibba032 bash[24004]: cluster 2021-03-10T16:10:24.470880+0000 mgr.gibba032.whopeo (mgr.24625) 7268 : cluster [DBG] pgmap v4136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:26 gibba032 bash[24004]: cephadm 2021-03-10T16:10:25.187517+0000 mgr.gibba032.whopeo (mgr.24625) 7269 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:26 gibba011 bash[30791]: cluster 2021-03-10T16:10:24.470880+0000 mgr.gibba032.whopeo (mgr.24625) 7268 : cluster [DBG] pgmap v4136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:26 gibba011 bash[30791]: cephadm 2021-03-10T16:10:25.187517+0000 mgr.gibba032.whopeo (mgr.24625) 7269 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:26 gibba011 bash[31774]: cluster 2021-03-10T16:10:24.470880+0000 mgr.gibba032.whopeo (mgr.24625) 7268 : cluster [DBG] pgmap v4136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:26 gibba011 bash[31774]: cephadm 2021-03-10T16:10:25.187517+0000 mgr.gibba032.whopeo (mgr.24625) 7269 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:28 gibba032 bash[24004]: cluster 2021-03-10T16:10:26.472702+0000 mgr.gibba032.whopeo (mgr.24625) 7270 : cluster [DBG] pgmap v4137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:28 gibba011 bash[30791]: cluster 2021-03-10T16:10:26.472702+0000 mgr.gibba032.whopeo (mgr.24625) 7270 : cluster [DBG] pgmap v4137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:28 gibba011 bash[31774]: cluster 2021-03-10T16:10:26.472702+0000 mgr.gibba032.whopeo (mgr.24625) 7270 : cluster [DBG] pgmap v4137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:30 gibba032 bash[24004]: cluster 2021-03-10T16:10:28.474348+0000 mgr.gibba032.whopeo (mgr.24625) 7271 : cluster [DBG] pgmap v4138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:30 gibba011 bash[30791]: cluster 2021-03-10T16:10:28.474348+0000 mgr.gibba032.whopeo (mgr.24625) 7271 : cluster [DBG] pgmap v4138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:30 gibba011 bash[31774]: cluster 2021-03-10T16:10:28.474348+0000 mgr.gibba032.whopeo (mgr.24625) 7271 : cluster [DBG] pgmap v4138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:32 gibba032 bash[24004]: cluster 2021-03-10T16:10:30.475846+0000 mgr.gibba032.whopeo (mgr.24625) 7272 : cluster [DBG] pgmap v4139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:32 gibba011 bash[30791]: cluster 2021-03-10T16:10:30.475846+0000 mgr.gibba032.whopeo (mgr.24625) 7272 : cluster [DBG] pgmap v4139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:32 gibba011 bash[31774]: cluster 2021-03-10T16:10:30.475846+0000 mgr.gibba032.whopeo (mgr.24625) 7272 : cluster [DBG] pgmap v4139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:34 gibba032 bash[24004]: cluster 2021-03-10T16:10:32.477625+0000 mgr.gibba032.whopeo (mgr.24625) 7273 : cluster [DBG] pgmap v4140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:34.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:34 gibba011 bash[30791]: cluster 2021-03-10T16:10:32.477625+0000 mgr.gibba032.whopeo (mgr.24625) 7273 : cluster [DBG] pgmap v4140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:34 gibba011 bash[31774]: cluster 2021-03-10T16:10:32.477625+0000 mgr.gibba032.whopeo (mgr.24625) 7273 : cluster [DBG] pgmap v4140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:36.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:36 gibba032 bash[24004]: cluster 2021-03-10T16:10:34.479227+0000 mgr.gibba032.whopeo (mgr.24625) 7274 : cluster [DBG] pgmap v4141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:36 gibba011 bash[30791]: cluster 2021-03-10T16:10:34.479227+0000 mgr.gibba032.whopeo (mgr.24625) 7274 : cluster [DBG] pgmap v4141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:36 gibba011 bash[31774]: cluster 2021-03-10T16:10:34.479227+0000 mgr.gibba032.whopeo (mgr.24625) 7274 : cluster [DBG] pgmap v4141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:38 gibba032 bash[24004]: cluster 2021-03-10T16:10:36.481309+0000 mgr.gibba032.whopeo (mgr.24625) 7275 : cluster [DBG] pgmap v4142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:38 gibba011 bash[30791]: cluster 2021-03-10T16:10:36.481309+0000 mgr.gibba032.whopeo (mgr.24625) 7275 : cluster [DBG] pgmap v4142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:38 gibba011 bash[31774]: cluster 2021-03-10T16:10:36.481309+0000 mgr.gibba032.whopeo (mgr.24625) 7275 : cluster [DBG] pgmap v4142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:38.597 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:10:38.828 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:10:38.828 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 43s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:10:38.829 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:10:38.829 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:10:38.829 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:10:38.830 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:10:38.830 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:10:38.830 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:10:38.831 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:10:38.831 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 43s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:10:38.831 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:10:38.831 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:10:38.832 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:10:38.832 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:10:38.832 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:10:38.832 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:10:38.833 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:10:38.833 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:10:38.833 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:10:38.834 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:10:38.834 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 43s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:10:39.080 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:10:39.081 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:10:39.081 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:10:39.081 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:39.082 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:10:39.082 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:10:39.082 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:39.082 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:10:39.082 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:10:39.083 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:10:39.083 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:10:39.083 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:10:39.083 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:10:39.084 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:10:39.084 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:10:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:39 gibba032 bash[24004]: cluster 2021-03-10T16:10:38.482998+0000 mgr.gibba032.whopeo (mgr.24625) 7276 : cluster [DBG] pgmap v4143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:39 gibba032 bash[24004]: audit 2021-03-10T16:10:38.588461+0000 mgr.gibba032.whopeo (mgr.24625) 7277 : audit [DBG] from='client.58915 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:39 gibba032 bash[24004]: audit 2021-03-10T16:10:38.815022+0000 mgr.gibba032.whopeo (mgr.24625) 7278 : audit [DBG] from='client.58921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:39 gibba032 bash[24004]: audit 2021-03-10T16:10:39.081841+0000 mon.a (mon.0) 2608 : audit [DBG] from='client.? 172.21.2.111:0/1733802465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:40.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[30791]: cluster 2021-03-10T16:10:38.482998+0000 mgr.gibba032.whopeo (mgr.24625) 7276 : cluster [DBG] pgmap v4143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[30791]: audit 2021-03-10T16:10:38.588461+0000 mgr.gibba032.whopeo (mgr.24625) 7277 : audit [DBG] from='client.58915 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[30791]: audit 2021-03-10T16:10:38.815022+0000 mgr.gibba032.whopeo (mgr.24625) 7278 : audit [DBG] from='client.58921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[30791]: audit 2021-03-10T16:10:39.081841+0000 mon.a (mon.0) 2608 : audit [DBG] from='client.? 172.21.2.111:0/1733802465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[31774]: cluster 2021-03-10T16:10:38.482998+0000 mgr.gibba032.whopeo (mgr.24625) 7276 : cluster [DBG] pgmap v4143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[31774]: audit 2021-03-10T16:10:38.588461+0000 mgr.gibba032.whopeo (mgr.24625) 7277 : audit [DBG] from='client.58915 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[31774]: audit 2021-03-10T16:10:38.815022+0000 mgr.gibba032.whopeo (mgr.24625) 7278 : audit [DBG] from='client.58921 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:10:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:39 gibba011 bash[31774]: audit 2021-03-10T16:10:39.081841+0000 mon.a (mon.0) 2608 : audit [DBG] from='client.? 172.21.2.111:0/1733802465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:10:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:40 gibba032 bash[24004]: cephadm 2021-03-10T16:10:40.189046+0000 mgr.gibba032.whopeo (mgr.24625) 7279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:40 gibba011 bash[30791]: cephadm 2021-03-10T16:10:40.189046+0000 mgr.gibba032.whopeo (mgr.24625) 7279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:41.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:40 gibba011 bash[31774]: cephadm 2021-03-10T16:10:40.189046+0000 mgr.gibba032.whopeo (mgr.24625) 7279 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:10:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:41 gibba032 bash[24004]: cluster 2021-03-10T16:10:40.483664+0000 mgr.gibba032.whopeo (mgr.24625) 7280 : cluster [DBG] pgmap v4144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:41 gibba011 bash[30791]: cluster 2021-03-10T16:10:40.483664+0000 mgr.gibba032.whopeo (mgr.24625) 7280 : cluster [DBG] pgmap v4144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:42.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:41 gibba011 bash[31774]: cluster 2021-03-10T16:10:40.483664+0000 mgr.gibba032.whopeo (mgr.24625) 7280 : cluster [DBG] pgmap v4144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:44.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:43 gibba032 bash[24004]: cluster 2021-03-10T16:10:42.485449+0000 mgr.gibba032.whopeo (mgr.24625) 7281 : cluster [DBG] pgmap v4145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:43 gibba011 bash[30791]: cluster 2021-03-10T16:10:42.485449+0000 mgr.gibba032.whopeo (mgr.24625) 7281 : cluster [DBG] pgmap v4145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:44.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:43 gibba011 bash[31774]: cluster 2021-03-10T16:10:42.485449+0000 mgr.gibba032.whopeo (mgr.24625) 7281 : cluster [DBG] pgmap v4145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:46.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:45 gibba032 bash[24004]: cluster 2021-03-10T16:10:44.487030+0000 mgr.gibba032.whopeo (mgr.24625) 7282 : cluster [DBG] pgmap v4146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:45 gibba011 bash[30791]: cluster 2021-03-10T16:10:44.487030+0000 mgr.gibba032.whopeo (mgr.24625) 7282 : cluster [DBG] pgmap v4146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:46.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:45 gibba011 bash[31774]: cluster 2021-03-10T16:10:44.487030+0000 mgr.gibba032.whopeo (mgr.24625) 7282 : cluster [DBG] pgmap v4146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:48.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:47 gibba032 bash[24004]: cluster 2021-03-10T16:10:46.487906+0000 mgr.gibba032.whopeo (mgr.24625) 7283 : cluster [DBG] pgmap v4147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:48.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:47 gibba011 bash[30791]: cluster 2021-03-10T16:10:46.487906+0000 mgr.gibba032.whopeo (mgr.24625) 7283 : cluster [DBG] pgmap v4147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:48.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:47 gibba011 bash[31774]: cluster 2021-03-10T16:10:46.487906+0000 mgr.gibba032.whopeo (mgr.24625) 7283 : cluster [DBG] pgmap v4147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:49 gibba032 bash[24004]: cluster 2021-03-10T16:10:48.489391+0000 mgr.gibba032.whopeo (mgr.24625) 7284 : cluster [DBG] pgmap v4148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:50.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:49 gibba011 bash[30791]: cluster 2021-03-10T16:10:48.489391+0000 mgr.gibba032.whopeo (mgr.24625) 7284 : cluster [DBG] pgmap v4148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:50.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:49 gibba011 bash[31774]: cluster 2021-03-10T16:10:48.489391+0000 mgr.gibba032.whopeo (mgr.24625) 7284 : cluster [DBG] pgmap v4148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:52.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:52 gibba011 bash[30791]: cluster 2021-03-10T16:10:50.490910+0000 mgr.gibba032.whopeo (mgr.24625) 7285 : cluster [DBG] pgmap v4149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:52.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:52 gibba011 bash[31774]: cluster 2021-03-10T16:10:50.490910+0000 mgr.gibba032.whopeo (mgr.24625) 7285 : cluster [DBG] pgmap v4149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:52 gibba032 bash[24004]: cluster 2021-03-10T16:10:50.490910+0000 mgr.gibba032.whopeo (mgr.24625) 7285 : cluster [DBG] pgmap v4149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:54 gibba011 bash[30791]: cluster 2021-03-10T16:10:52.491758+0000 mgr.gibba032.whopeo (mgr.24625) 7286 : cluster [DBG] pgmap v4150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:54.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:54 gibba011 bash[31774]: cluster 2021-03-10T16:10:52.491758+0000 mgr.gibba032.whopeo (mgr.24625) 7286 : cluster [DBG] pgmap v4150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:54 gibba032 bash[24004]: cluster 2021-03-10T16:10:52.491758+0000 mgr.gibba032.whopeo (mgr.24625) 7286 : cluster [DBG] pgmap v4150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[30791]: cluster 2021-03-10T16:10:54.493035+0000 mgr.gibba032.whopeo (mgr.24625) 7287 : cluster [DBG] pgmap v4151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[30791]: cephadm 2021-03-10T16:10:55.205873+0000 mgr.gibba032.whopeo (mgr.24625) 7288 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:10:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[30791]: audit 2021-03-10T16:10:55.209944+0000 mon.a (mon.0) 2609 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:10:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[31774]: cluster 2021-03-10T16:10:54.493035+0000 mgr.gibba032.whopeo (mgr.24625) 7287 : cluster [DBG] pgmap v4151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[31774]: cephadm 2021-03-10T16:10:55.205873+0000 mgr.gibba032.whopeo (mgr.24625) 7288 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:10:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:56 gibba011 bash[31774]: audit 2021-03-10T16:10:55.209944+0000 mon.a (mon.0) 2609 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:10:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:56 gibba032 bash[24004]: cluster 2021-03-10T16:10:54.493035+0000 mgr.gibba032.whopeo (mgr.24625) 7287 : cluster [DBG] pgmap v4151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:56 gibba032 bash[24004]: cephadm 2021-03-10T16:10:55.205873+0000 mgr.gibba032.whopeo (mgr.24625) 7288 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:10:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:56 gibba032 bash[24004]: audit 2021-03-10T16:10:55.209944+0000 mon.a (mon.0) 2609 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:10:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.874415+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:10:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.875073+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:10:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.949043+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:10:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.949833+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:10:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.950485+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:10:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[30791]: audit 2021-03-10T16:10:56.951189+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:10:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.874415+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:10:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.875073+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:10:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.949043+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:10:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.949833+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:10:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.950485+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:10:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:57 gibba011 bash[31774]: audit 2021-03-10T16:10:56.951189+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:10:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.874415+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:10:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.875073+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:10:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.949043+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:10:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.949833+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:10:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.950485+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:10:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:57 gibba032 bash[24004]: audit 2021-03-10T16:10:56.951189+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:10:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:58 gibba011 bash[30791]: cluster 2021-03-10T16:10:56.494698+0000 mgr.gibba032.whopeo (mgr.24625) 7289 : cluster [DBG] pgmap v4152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:58 gibba011 bash[31774]: cluster 2021-03-10T16:10:56.494698+0000 mgr.gibba032.whopeo (mgr.24625) 7289 : cluster [DBG] pgmap v4152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:10:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:58 gibba032 bash[24004]: cluster 2021-03-10T16:10:56.494698+0000 mgr.gibba032.whopeo (mgr.24625) 7289 : cluster [DBG] pgmap v4152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cluster 2021-03-10T16:10:58.495488+0000 mgr.gibba032.whopeo (mgr.24625) 7290 : cluster [DBG] pgmap v4153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:00.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:58.896715+0000 mgr.gibba032.whopeo (mgr.24625) 7291 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:11:00.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:58.897960+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.099554+0000 mon.a (mon.0) 2617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.100026+0000 mgr.gibba032.whopeo (mgr.24625) 7292 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:11:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.102514+0000 mgr.gibba032.whopeo (mgr.24625) 7293 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:11:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.102837+0000 mon.a (mon.0) 2618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:11:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.104263+0000 mgr.gibba032.whopeo (mgr.24625) 7294 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:11:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.104300+0000 mgr.gibba032.whopeo (mgr.24625) 7295 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:11:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.104920+0000 mon.a (mon.0) 2619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.105831+0000 mgr.gibba032.whopeo (mgr.24625) 7296 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:11:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.105868+0000 mgr.gibba032.whopeo (mgr.24625) 7297 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:11:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.106665+0000 mon.a (mon.0) 2620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.107436+0000 mgr.gibba032.whopeo (mgr.24625) 7298 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:11:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.108216+0000 mon.a (mon.0) 2621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.108809+0000 mgr.gibba032.whopeo (mgr.24625) 7299 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:11:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.108845+0000 mgr.gibba032.whopeo (mgr.24625) 7300 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:11:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7301 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:11:00.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.110877+0000 mgr.gibba032.whopeo (mgr.24625) 7302 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:11:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.111272+0000 mon.a (mon.0) 2622 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.113004+0000 mgr.gibba032.whopeo (mgr.24625) 7303 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:11:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.113041+0000 mgr.gibba032.whopeo (mgr.24625) 7304 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:11:00.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: cephadm 2021-03-10T16:10:59.113146+0000 mgr.gibba032.whopeo (mgr.24625) 7305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:00.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:10:59 gibba032 bash[24004]: audit 2021-03-10T16:10:59.113530+0000 mon.a (mon.0) 2623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cluster 2021-03-10T16:10:58.495488+0000 mgr.gibba032.whopeo (mgr.24625) 7290 : cluster [DBG] pgmap v4153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:58.896715+0000 mgr.gibba032.whopeo (mgr.24625) 7291 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:11:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:58.897960+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.099554+0000 mon.a (mon.0) 2617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.100026+0000 mgr.gibba032.whopeo (mgr.24625) 7292 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:11:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.102514+0000 mgr.gibba032.whopeo (mgr.24625) 7293 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:11:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.102837+0000 mon.a (mon.0) 2618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:11:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.104263+0000 mgr.gibba032.whopeo (mgr.24625) 7294 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:11:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.104300+0000 mgr.gibba032.whopeo (mgr.24625) 7295 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:11:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.104920+0000 mon.a (mon.0) 2619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.105831+0000 mgr.gibba032.whopeo (mgr.24625) 7296 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:11:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.105868+0000 mgr.gibba032.whopeo (mgr.24625) 7297 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:11:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.106665+0000 mon.a (mon.0) 2620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.107436+0000 mgr.gibba032.whopeo (mgr.24625) 7298 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:11:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.108216+0000 mon.a (mon.0) 2621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.108809+0000 mgr.gibba032.whopeo (mgr.24625) 7299 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:11:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.108845+0000 mgr.gibba032.whopeo (mgr.24625) 7300 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:11:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7301 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:11:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.110877+0000 mgr.gibba032.whopeo (mgr.24625) 7302 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:11:00.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.111272+0000 mon.a (mon.0) 2622 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.113004+0000 mgr.gibba032.whopeo (mgr.24625) 7303 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:11:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.113041+0000 mgr.gibba032.whopeo (mgr.24625) 7304 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:11:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: cephadm 2021-03-10T16:10:59.113146+0000 mgr.gibba032.whopeo (mgr.24625) 7305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:00.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[30791]: audit 2021-03-10T16:10:59.113530+0000 mon.a (mon.0) 2623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cluster 2021-03-10T16:10:58.495488+0000 mgr.gibba032.whopeo (mgr.24625) 7290 : cluster [DBG] pgmap v4153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:58.896715+0000 mgr.gibba032.whopeo (mgr.24625) 7291 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:11:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:58.897960+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.099554+0000 mon.a (mon.0) 2617 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:11:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.100026+0000 mgr.gibba032.whopeo (mgr.24625) 7292 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:11:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.102514+0000 mgr.gibba032.whopeo (mgr.24625) 7293 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:11:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.102837+0000 mon.a (mon.0) 2618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:11:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.104263+0000 mgr.gibba032.whopeo (mgr.24625) 7294 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:11:00.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.104300+0000 mgr.gibba032.whopeo (mgr.24625) 7295 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:11:00.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.104920+0000 mon.a (mon.0) 2619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.105831+0000 mgr.gibba032.whopeo (mgr.24625) 7296 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:11:00.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.105868+0000 mgr.gibba032.whopeo (mgr.24625) 7297 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:11:00.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.106665+0000 mon.a (mon.0) 2620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.107436+0000 mgr.gibba032.whopeo (mgr.24625) 7298 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:11:00.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.108216+0000 mon.a (mon.0) 2621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.108809+0000 mgr.gibba032.whopeo (mgr.24625) 7299 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:11:00.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.108845+0000 mgr.gibba032.whopeo (mgr.24625) 7300 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:11:00.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7301 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:11:00.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.110877+0000 mgr.gibba032.whopeo (mgr.24625) 7302 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:11:00.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.111272+0000 mon.a (mon.0) 2622 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:00.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.113004+0000 mgr.gibba032.whopeo (mgr.24625) 7303 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:11:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.113041+0000 mgr.gibba032.whopeo (mgr.24625) 7304 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:11:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: cephadm 2021-03-10T16:10:59.113146+0000 mgr.gibba032.whopeo (mgr.24625) 7305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:10:59 gibba011 bash[31774]: audit 2021-03-10T16:10:59.113530+0000 mon.a (mon.0) 2623 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:01 gibba032 bash[24004]: cluster 2021-03-10T16:11:00.496967+0000 mgr.gibba032.whopeo (mgr.24625) 7306 : cluster [DBG] pgmap v4154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:01 gibba011 bash[30791]: cluster 2021-03-10T16:11:00.496967+0000 mgr.gibba032.whopeo (mgr.24625) 7306 : cluster [DBG] pgmap v4154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:02.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:01 gibba011 bash[31774]: cluster 2021-03-10T16:11:00.496967+0000 mgr.gibba032.whopeo (mgr.24625) 7306 : cluster [DBG] pgmap v4154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:03 gibba032 bash[24004]: cluster 2021-03-10T16:11:02.498743+0000 mgr.gibba032.whopeo (mgr.24625) 7307 : cluster [DBG] pgmap v4155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:04.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:03 gibba011 bash[30791]: cluster 2021-03-10T16:11:02.498743+0000 mgr.gibba032.whopeo (mgr.24625) 7307 : cluster [DBG] pgmap v4155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:03 gibba011 bash[31774]: cluster 2021-03-10T16:11:02.498743+0000 mgr.gibba032.whopeo (mgr.24625) 7307 : cluster [DBG] pgmap v4155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:05 gibba032 bash[24004]: cluster 2021-03-10T16:11:04.499471+0000 mgr.gibba032.whopeo (mgr.24625) 7308 : cluster [DBG] pgmap v4156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:06.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:05 gibba011 bash[30791]: cluster 2021-03-10T16:11:04.499471+0000 mgr.gibba032.whopeo (mgr.24625) 7308 : cluster [DBG] pgmap v4156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:06.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:05 gibba011 bash[31774]: cluster 2021-03-10T16:11:04.499471+0000 mgr.gibba032.whopeo (mgr.24625) 7308 : cluster [DBG] pgmap v4156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:07 gibba032 bash[24004]: cluster 2021-03-10T16:11:06.501177+0000 mgr.gibba032.whopeo (mgr.24625) 7309 : cluster [DBG] pgmap v4157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:08.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:07 gibba011 bash[30791]: cluster 2021-03-10T16:11:06.501177+0000 mgr.gibba032.whopeo (mgr.24625) 7309 : cluster [DBG] pgmap v4157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:07 gibba011 bash[31774]: cluster 2021-03-10T16:11:06.501177+0000 mgr.gibba032.whopeo (mgr.24625) 7309 : cluster [DBG] pgmap v4157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:09.324 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:11:09.552 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:11:09.553 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 74s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:11:09.553 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 10s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:11:09.553 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:11:09.553 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:11:09.554 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:11:09.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:11:09.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:11:09.554 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:11:09.554 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 74s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:11:09.555 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 10s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:11:09.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:11:09.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:11:09.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:11:09.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 74s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:11:09.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:11:09.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:11:09.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:11:09.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 10s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:11:09.557 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 10s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:11:09.557 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 74s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:11:09.804 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:11:09.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:11:09.804 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:11:09.804 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:09.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:11:09.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:11:09.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:09.805 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:11:09.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:11:09.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:09.806 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:11:09.806 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:11:09.806 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:11:09.806 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:11:09.807 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:11:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:09 gibba032 bash[24004]: cluster 2021-03-10T16:11:08.502756+0000 mgr.gibba032.whopeo (mgr.24625) 7310 : cluster [DBG] pgmap v4158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:09 gibba032 bash[24004]: audit 2021-03-10T16:11:09.314151+0000 mgr.gibba032.whopeo (mgr.24625) 7311 : audit [DBG] from='client.58933 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:09 gibba032 bash[24004]: audit 2021-03-10T16:11:09.805147+0000 mon.c (mon.1) 140 : audit [DBG] from='client.? 172.21.2.111:0/1409533571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[30791]: cluster 2021-03-10T16:11:08.502756+0000 mgr.gibba032.whopeo (mgr.24625) 7310 : cluster [DBG] pgmap v4158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[30791]: audit 2021-03-10T16:11:09.314151+0000 mgr.gibba032.whopeo (mgr.24625) 7311 : audit [DBG] from='client.58933 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[30791]: audit 2021-03-10T16:11:09.805147+0000 mon.c (mon.1) 140 : audit [DBG] from='client.? 172.21.2.111:0/1409533571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[31774]: cluster 2021-03-10T16:11:08.502756+0000 mgr.gibba032.whopeo (mgr.24625) 7310 : cluster [DBG] pgmap v4158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[31774]: audit 2021-03-10T16:11:09.314151+0000 mgr.gibba032.whopeo (mgr.24625) 7311 : audit [DBG] from='client.58933 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:09 gibba011 bash[31774]: audit 2021-03-10T16:11:09.805147+0000 mon.c (mon.1) 140 : audit [DBG] from='client.? 172.21.2.111:0/1409533571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:11.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:10 gibba032 bash[24004]: audit 2021-03-10T16:11:09.539495+0000 mgr.gibba032.whopeo (mgr.24625) 7312 : audit [DBG] from='client.48957 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:11.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:10 gibba011 bash[30791]: audit 2021-03-10T16:11:09.539495+0000 mgr.gibba032.whopeo (mgr.24625) 7312 : audit [DBG] from='client.48957 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:11.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:10 gibba011 bash[31774]: audit 2021-03-10T16:11:09.539495+0000 mgr.gibba032.whopeo (mgr.24625) 7312 : audit [DBG] from='client.48957 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:11 gibba032 bash[24004]: cluster 2021-03-10T16:11:10.504320+0000 mgr.gibba032.whopeo (mgr.24625) 7313 : cluster [DBG] pgmap v4159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:11 gibba011 bash[30791]: cluster 2021-03-10T16:11:10.504320+0000 mgr.gibba032.whopeo (mgr.24625) 7313 : cluster [DBG] pgmap v4159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:12.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:11 gibba011 bash[31774]: cluster 2021-03-10T16:11:10.504320+0000 mgr.gibba032.whopeo (mgr.24625) 7313 : cluster [DBG] pgmap v4159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:13 gibba032 bash[24004]: cluster 2021-03-10T16:11:12.505788+0000 mgr.gibba032.whopeo (mgr.24625) 7314 : cluster [DBG] pgmap v4160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:14 gibba011 bash[30791]: cluster 2021-03-10T16:11:12.505788+0000 mgr.gibba032.whopeo (mgr.24625) 7314 : cluster [DBG] pgmap v4160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:14.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:14 gibba011 bash[31774]: cluster 2021-03-10T16:11:12.505788+0000 mgr.gibba032.whopeo (mgr.24625) 7314 : cluster [DBG] pgmap v4160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:15.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:15 gibba011 bash[30791]: cephadm 2021-03-10T16:11:14.128935+0000 mgr.gibba032.whopeo (mgr.24625) 7315 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:15.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:15 gibba011 bash[31774]: cephadm 2021-03-10T16:11:14.128935+0000 mgr.gibba032.whopeo (mgr.24625) 7315 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:15.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:15 gibba032 bash[24004]: cephadm 2021-03-10T16:11:14.128935+0000 mgr.gibba032.whopeo (mgr.24625) 7315 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:16 gibba011 bash[30791]: cluster 2021-03-10T16:11:14.507286+0000 mgr.gibba032.whopeo (mgr.24625) 7316 : cluster [DBG] pgmap v4161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:16.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:16 gibba011 bash[31774]: cluster 2021-03-10T16:11:14.507286+0000 mgr.gibba032.whopeo (mgr.24625) 7316 : cluster [DBG] pgmap v4161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:16 gibba032 bash[24004]: cluster 2021-03-10T16:11:14.507286+0000 mgr.gibba032.whopeo (mgr.24625) 7316 : cluster [DBG] pgmap v4161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:18.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:18 gibba011 bash[30791]: cluster 2021-03-10T16:11:16.508010+0000 mgr.gibba032.whopeo (mgr.24625) 7317 : cluster [DBG] pgmap v4162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:18 gibba011 bash[31774]: cluster 2021-03-10T16:11:16.508010+0000 mgr.gibba032.whopeo (mgr.24625) 7317 : cluster [DBG] pgmap v4162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:18 gibba032 bash[24004]: cluster 2021-03-10T16:11:16.508010+0000 mgr.gibba032.whopeo (mgr.24625) 7317 : cluster [DBG] pgmap v4162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:20 gibba011 bash[30791]: cluster 2021-03-10T16:11:18.509256+0000 mgr.gibba032.whopeo (mgr.24625) 7318 : cluster [DBG] pgmap v4163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:20.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:20 gibba011 bash[31774]: cluster 2021-03-10T16:11:18.509256+0000 mgr.gibba032.whopeo (mgr.24625) 7318 : cluster [DBG] pgmap v4163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:20 gibba032 bash[24004]: cluster 2021-03-10T16:11:18.509256+0000 mgr.gibba032.whopeo (mgr.24625) 7318 : cluster [DBG] pgmap v4163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:22 gibba011 bash[30791]: cluster 2021-03-10T16:11:20.510748+0000 mgr.gibba032.whopeo (mgr.24625) 7319 : cluster [DBG] pgmap v4164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:22 gibba011 bash[31774]: cluster 2021-03-10T16:11:20.510748+0000 mgr.gibba032.whopeo (mgr.24625) 7319 : cluster [DBG] pgmap v4164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:22 gibba032 bash[24004]: cluster 2021-03-10T16:11:20.510748+0000 mgr.gibba032.whopeo (mgr.24625) 7319 : cluster [DBG] pgmap v4164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:24 gibba011 bash[30791]: cluster 2021-03-10T16:11:22.511606+0000 mgr.gibba032.whopeo (mgr.24625) 7320 : cluster [DBG] pgmap v4165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:24 gibba011 bash[31774]: cluster 2021-03-10T16:11:22.511606+0000 mgr.gibba032.whopeo (mgr.24625) 7320 : cluster [DBG] pgmap v4165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:24 gibba032 bash[24004]: cluster 2021-03-10T16:11:22.511606+0000 mgr.gibba032.whopeo (mgr.24625) 7320 : cluster [DBG] pgmap v4165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:26 gibba011 bash[30791]: cluster 2021-03-10T16:11:24.512951+0000 mgr.gibba032.whopeo (mgr.24625) 7321 : cluster [DBG] pgmap v4166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:26.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:26 gibba011 bash[31774]: cluster 2021-03-10T16:11:24.512951+0000 mgr.gibba032.whopeo (mgr.24625) 7321 : cluster [DBG] pgmap v4166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:26 gibba032 bash[24004]: cluster 2021-03-10T16:11:24.512951+0000 mgr.gibba032.whopeo (mgr.24625) 7321 : cluster [DBG] pgmap v4166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:28 gibba011 bash[30791]: cluster 2021-03-10T16:11:26.514620+0000 mgr.gibba032.whopeo (mgr.24625) 7322 : cluster [DBG] pgmap v4167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:28 gibba011 bash[31774]: cluster 2021-03-10T16:11:26.514620+0000 mgr.gibba032.whopeo (mgr.24625) 7322 : cluster [DBG] pgmap v4167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:28 gibba032 bash[24004]: cluster 2021-03-10T16:11:26.514620+0000 mgr.gibba032.whopeo (mgr.24625) 7322 : cluster [DBG] pgmap v4167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:30 gibba011 bash[30791]: cluster 2021-03-10T16:11:28.515211+0000 mgr.gibba032.whopeo (mgr.24625) 7323 : cluster [DBG] pgmap v4168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:30 gibba011 bash[30791]: cephadm 2021-03-10T16:11:29.144686+0000 mgr.gibba032.whopeo (mgr.24625) 7324 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:30 gibba011 bash[31774]: cluster 2021-03-10T16:11:28.515211+0000 mgr.gibba032.whopeo (mgr.24625) 7323 : cluster [DBG] pgmap v4168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:30 gibba011 bash[31774]: cephadm 2021-03-10T16:11:29.144686+0000 mgr.gibba032.whopeo (mgr.24625) 7324 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:30 gibba032 bash[24004]: cluster 2021-03-10T16:11:28.515211+0000 mgr.gibba032.whopeo (mgr.24625) 7323 : cluster [DBG] pgmap v4168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:30 gibba032 bash[24004]: cephadm 2021-03-10T16:11:29.144686+0000 mgr.gibba032.whopeo (mgr.24625) 7324 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:32 gibba011 bash[30791]: cluster 2021-03-10T16:11:30.516687+0000 mgr.gibba032.whopeo (mgr.24625) 7325 : cluster [DBG] pgmap v4169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:32.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:32 gibba011 bash[31774]: cluster 2021-03-10T16:11:30.516687+0000 mgr.gibba032.whopeo (mgr.24625) 7325 : cluster [DBG] pgmap v4169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:32 gibba032 bash[24004]: cluster 2021-03-10T16:11:30.516687+0000 mgr.gibba032.whopeo (mgr.24625) 7325 : cluster [DBG] pgmap v4169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:34 gibba032 bash[24004]: cluster 2021-03-10T16:11:32.518169+0000 mgr.gibba032.whopeo (mgr.24625) 7326 : cluster [DBG] pgmap v4170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:34.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:34 gibba011 bash[30791]: cluster 2021-03-10T16:11:32.518169+0000 mgr.gibba032.whopeo (mgr.24625) 7326 : cluster [DBG] pgmap v4170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:34 gibba011 bash[31774]: cluster 2021-03-10T16:11:32.518169+0000 mgr.gibba032.whopeo (mgr.24625) 7326 : cluster [DBG] pgmap v4170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:36 gibba032 bash[24004]: cluster 2021-03-10T16:11:34.519578+0000 mgr.gibba032.whopeo (mgr.24625) 7327 : cluster [DBG] pgmap v4171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:36 gibba011 bash[30791]: cluster 2021-03-10T16:11:34.519578+0000 mgr.gibba032.whopeo (mgr.24625) 7327 : cluster [DBG] pgmap v4171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:36 gibba011 bash[31774]: cluster 2021-03-10T16:11:34.519578+0000 mgr.gibba032.whopeo (mgr.24625) 7327 : cluster [DBG] pgmap v4171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:38 gibba032 bash[24004]: cluster 2021-03-10T16:11:36.521351+0000 mgr.gibba032.whopeo (mgr.24625) 7328 : cluster [DBG] pgmap v4172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:38 gibba011 bash[30791]: cluster 2021-03-10T16:11:36.521351+0000 mgr.gibba032.whopeo (mgr.24625) 7328 : cluster [DBG] pgmap v4172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:38 gibba011 bash[31774]: cluster 2021-03-10T16:11:36.521351+0000 mgr.gibba032.whopeo (mgr.24625) 7328 : cluster [DBG] pgmap v4172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:40.046 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:11:40.274 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:11:40.275 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 105s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:11:40.275 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 41s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:11:40.275 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:11:40.276 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:11:40.276 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:11:40.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:11:40.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:11:40.276 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:11:40.277 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 105s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:11:40.277 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 41s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:11:40.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:11:40.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:11:40.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:11:40.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:11:40.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:11:40.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:11:40.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:11:40.279 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 41s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:11:40.279 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 41s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:11:40.279 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 105s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:11:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:40 gibba032 bash[24004]: cluster 2021-03-10T16:11:38.522860+0000 mgr.gibba032.whopeo (mgr.24625) 7329 : cluster [DBG] pgmap v4173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:40.523 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:11:40.523 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:11:40.524 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:11:40.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:40.524 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:11:40.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:11:40.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:40.525 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:11:40.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:11:40.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:11:40.526 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:11:40.526 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:11:40.526 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:11:40.527 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:11:40.527 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:11:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:40 gibba011 bash[30791]: cluster 2021-03-10T16:11:38.522860+0000 mgr.gibba032.whopeo (mgr.24625) 7329 : cluster [DBG] pgmap v4173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:40 gibba011 bash[31774]: cluster 2021-03-10T16:11:38.522860+0000 mgr.gibba032.whopeo (mgr.24625) 7329 : cluster [DBG] pgmap v4173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:41 gibba032 bash[24004]: audit 2021-03-10T16:11:40.036571+0000 mgr.gibba032.whopeo (mgr.24625) 7330 : audit [DBG] from='client.48969 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:41 gibba032 bash[24004]: audit 2021-03-10T16:11:40.262097+0000 mgr.gibba032.whopeo (mgr.24625) 7331 : audit [DBG] from='client.48975 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:41 gibba032 bash[24004]: audit 2021-03-10T16:11:40.524619+0000 mon.a (mon.0) 2624 : audit [DBG] from='client.? 172.21.2.111:0/3351761243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[30791]: audit 2021-03-10T16:11:40.036571+0000 mgr.gibba032.whopeo (mgr.24625) 7330 : audit [DBG] from='client.48969 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[30791]: audit 2021-03-10T16:11:40.262097+0000 mgr.gibba032.whopeo (mgr.24625) 7331 : audit [DBG] from='client.48975 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[30791]: audit 2021-03-10T16:11:40.524619+0000 mon.a (mon.0) 2624 : audit [DBG] from='client.? 172.21.2.111:0/3351761243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[31774]: audit 2021-03-10T16:11:40.036571+0000 mgr.gibba032.whopeo (mgr.24625) 7330 : audit [DBG] from='client.48969 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[31774]: audit 2021-03-10T16:11:40.262097+0000 mgr.gibba032.whopeo (mgr.24625) 7331 : audit [DBG] from='client.48975 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:11:41.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:41 gibba011 bash[31774]: audit 2021-03-10T16:11:40.524619+0000 mon.a (mon.0) 2624 : audit [DBG] from='client.? 172.21.2.111:0/3351761243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:11:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:42 gibba032 bash[24004]: cluster 2021-03-10T16:11:40.523664+0000 mgr.gibba032.whopeo (mgr.24625) 7332 : cluster [DBG] pgmap v4174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:42 gibba011 bash[30791]: cluster 2021-03-10T16:11:40.523664+0000 mgr.gibba032.whopeo (mgr.24625) 7332 : cluster [DBG] pgmap v4174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:42 gibba011 bash[31774]: cluster 2021-03-10T16:11:40.523664+0000 mgr.gibba032.whopeo (mgr.24625) 7332 : cluster [DBG] pgmap v4174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:44 gibba032 bash[24004]: cluster 2021-03-10T16:11:42.525278+0000 mgr.gibba032.whopeo (mgr.24625) 7333 : cluster [DBG] pgmap v4175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:44.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:44 gibba011 bash[30791]: cluster 2021-03-10T16:11:42.525278+0000 mgr.gibba032.whopeo (mgr.24625) 7333 : cluster [DBG] pgmap v4175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:44 gibba011 bash[31774]: cluster 2021-03-10T16:11:42.525278+0000 mgr.gibba032.whopeo (mgr.24625) 7333 : cluster [DBG] pgmap v4175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:45.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:45 gibba032 bash[24004]: cephadm 2021-03-10T16:11:44.160770+0000 mgr.gibba032.whopeo (mgr.24625) 7334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:45.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:45 gibba011 bash[30791]: cephadm 2021-03-10T16:11:44.160770+0000 mgr.gibba032.whopeo (mgr.24625) 7334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:45.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:45 gibba011 bash[31774]: cephadm 2021-03-10T16:11:44.160770+0000 mgr.gibba032.whopeo (mgr.24625) 7334 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:11:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:46 gibba032 bash[24004]: cluster 2021-03-10T16:11:44.526659+0000 mgr.gibba032.whopeo (mgr.24625) 7335 : cluster [DBG] pgmap v4176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:46 gibba011 bash[30791]: cluster 2021-03-10T16:11:44.526659+0000 mgr.gibba032.whopeo (mgr.24625) 7335 : cluster [DBG] pgmap v4176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:46 gibba011 bash[31774]: cluster 2021-03-10T16:11:44.526659+0000 mgr.gibba032.whopeo (mgr.24625) 7335 : cluster [DBG] pgmap v4176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:48 gibba032 bash[24004]: cluster 2021-03-10T16:11:46.527551+0000 mgr.gibba032.whopeo (mgr.24625) 7336 : cluster [DBG] pgmap v4177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:48 gibba011 bash[30791]: cluster 2021-03-10T16:11:46.527551+0000 mgr.gibba032.whopeo (mgr.24625) 7336 : cluster [DBG] pgmap v4177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:48 gibba011 bash[31774]: cluster 2021-03-10T16:11:46.527551+0000 mgr.gibba032.whopeo (mgr.24625) 7336 : cluster [DBG] pgmap v4177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:50 gibba032 bash[24004]: cluster 2021-03-10T16:11:48.528904+0000 mgr.gibba032.whopeo (mgr.24625) 7337 : cluster [DBG] pgmap v4178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:50 gibba011 bash[30791]: cluster 2021-03-10T16:11:48.528904+0000 mgr.gibba032.whopeo (mgr.24625) 7337 : cluster [DBG] pgmap v4178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:50 gibba011 bash[31774]: cluster 2021-03-10T16:11:48.528904+0000 mgr.gibba032.whopeo (mgr.24625) 7337 : cluster [DBG] pgmap v4178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:52 gibba032 bash[24004]: cluster 2021-03-10T16:11:50.530378+0000 mgr.gibba032.whopeo (mgr.24625) 7338 : cluster [DBG] pgmap v4179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:52 gibba011 bash[30791]: cluster 2021-03-10T16:11:50.530378+0000 mgr.gibba032.whopeo (mgr.24625) 7338 : cluster [DBG] pgmap v4179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:52 gibba011 bash[31774]: cluster 2021-03-10T16:11:50.530378+0000 mgr.gibba032.whopeo (mgr.24625) 7338 : cluster [DBG] pgmap v4179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:54 gibba032 bash[24004]: cluster 2021-03-10T16:11:52.531292+0000 mgr.gibba032.whopeo (mgr.24625) 7339 : cluster [DBG] pgmap v4180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:54 gibba011 bash[30791]: cluster 2021-03-10T16:11:52.531292+0000 mgr.gibba032.whopeo (mgr.24625) 7339 : cluster [DBG] pgmap v4180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:54 gibba011 bash[31774]: cluster 2021-03-10T16:11:52.531292+0000 mgr.gibba032.whopeo (mgr.24625) 7339 : cluster [DBG] pgmap v4180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:56 gibba032 bash[24004]: cluster 2021-03-10T16:11:54.532660+0000 mgr.gibba032.whopeo (mgr.24625) 7340 : cluster [DBG] pgmap v4181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:56 gibba011 bash[30791]: cluster 2021-03-10T16:11:54.532660+0000 mgr.gibba032.whopeo (mgr.24625) 7340 : cluster [DBG] pgmap v4181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:56 gibba011 bash[31774]: cluster 2021-03-10T16:11:54.532660+0000 mgr.gibba032.whopeo (mgr.24625) 7340 : cluster [DBG] pgmap v4181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.881080+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:11:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.881711+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:11:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.958532+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:11:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.959283+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.960040+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:57 gibba032 bash[24004]: audit 2021-03-10T16:11:56.960722+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:11:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.881080+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:11:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.881711+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:11:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.958532+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:11:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.959283+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.960040+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[30791]: audit 2021-03-10T16:11:56.960722+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:11:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.881080+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:11:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.881711+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:11:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.958532+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:11:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.959283+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.960040+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:11:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:57 gibba011 bash[31774]: audit 2021-03-10T16:11:56.960722+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:11:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:11:58 gibba032 bash[24004]: cluster 2021-03-10T16:11:56.534212+0000 mgr.gibba032.whopeo (mgr.24625) 7341 : cluster [DBG] pgmap v4182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:11:58 gibba011 bash[30791]: cluster 2021-03-10T16:11:56.534212+0000 mgr.gibba032.whopeo (mgr.24625) 7341 : cluster [DBG] pgmap v4182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:11:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:11:58 gibba011 bash[31774]: cluster 2021-03-10T16:11:56.534212+0000 mgr.gibba032.whopeo (mgr.24625) 7341 : cluster [DBG] pgmap v4182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: cluster 2021-03-10T16:11:58.535145+0000 mgr.gibba032.whopeo (mgr.24625) 7342 : cluster [DBG] pgmap v4183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.169867+0000 mgr.gibba032.whopeo (mgr.24625) 7343 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:12:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.171426+0000 mgr.gibba032.whopeo (mgr.24625) 7344 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:12:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.392752+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:12:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.396204+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:12:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.397918+0000 mon.a (mon.0) 2633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.399636+0000 mon.a (mon.0) 2634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.401113+0000 mon.a (mon.0) 2635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.403849+0000 mon.a (mon.0) 2636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:00 gibba032 bash[24004]: audit 2021-03-10T16:11:59.405500+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: cluster 2021-03-10T16:11:58.535145+0000 mgr.gibba032.whopeo (mgr.24625) 7342 : cluster [DBG] pgmap v4183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.169867+0000 mgr.gibba032.whopeo (mgr.24625) 7343 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:12:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.171426+0000 mgr.gibba032.whopeo (mgr.24625) 7344 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:12:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.392752+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:12:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.396204+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:12:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.397918+0000 mon.a (mon.0) 2633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.399636+0000 mon.a (mon.0) 2634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.401113+0000 mon.a (mon.0) 2635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.403849+0000 mon.a (mon.0) 2636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[30791]: audit 2021-03-10T16:11:59.405500+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: cluster 2021-03-10T16:11:58.535145+0000 mgr.gibba032.whopeo (mgr.24625) 7342 : cluster [DBG] pgmap v4183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.169867+0000 mgr.gibba032.whopeo (mgr.24625) 7343 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:12:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.171426+0000 mgr.gibba032.whopeo (mgr.24625) 7344 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:12:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.392752+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:12:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.396204+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:12:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.397918+0000 mon.a (mon.0) 2633 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.399636+0000 mon.a (mon.0) 2634 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.401113+0000 mon.a (mon.0) 2635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.403849+0000 mon.a (mon.0) 2636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:00.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:00 gibba011 bash[31774]: audit 2021-03-10T16:11:59.405500+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.393356+0000 mgr.gibba032.whopeo (mgr.24625) 7345 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:12:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.395445+0000 mgr.gibba032.whopeo (mgr.24625) 7346 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:12:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.397276+0000 mgr.gibba032.whopeo (mgr.24625) 7347 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:12:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.397311+0000 mgr.gibba032.whopeo (mgr.24625) 7348 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:12:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.398666+0000 mgr.gibba032.whopeo (mgr.24625) 7349 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:12:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.398703+0000 mgr.gibba032.whopeo (mgr.24625) 7350 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:12:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.399984+0000 mgr.gibba032.whopeo (mgr.24625) 7351 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:12:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.401288+0000 mgr.gibba032.whopeo (mgr.24625) 7352 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:12:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.401323+0000 mgr.gibba032.whopeo (mgr.24625) 7353 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:12:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.402793+0000 mgr.gibba032.whopeo (mgr.24625) 7354 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:12:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.403107+0000 mgr.gibba032.whopeo (mgr.24625) 7355 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:12:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.404491+0000 mgr.gibba032.whopeo (mgr.24625) 7356 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:12:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.404531+0000 mgr.gibba032.whopeo (mgr.24625) 7357 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:12:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:01 gibba032 bash[24004]: cephadm 2021-03-10T16:11:59.404723+0000 mgr.gibba032.whopeo (mgr.24625) 7358 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.393356+0000 mgr.gibba032.whopeo (mgr.24625) 7345 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:12:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.395445+0000 mgr.gibba032.whopeo (mgr.24625) 7346 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:12:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.397276+0000 mgr.gibba032.whopeo (mgr.24625) 7347 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:12:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.397311+0000 mgr.gibba032.whopeo (mgr.24625) 7348 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:12:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.398666+0000 mgr.gibba032.whopeo (mgr.24625) 7349 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:12:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.398703+0000 mgr.gibba032.whopeo (mgr.24625) 7350 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:12:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.399984+0000 mgr.gibba032.whopeo (mgr.24625) 7351 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:12:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.401288+0000 mgr.gibba032.whopeo (mgr.24625) 7352 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:12:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.401323+0000 mgr.gibba032.whopeo (mgr.24625) 7353 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:12:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.402793+0000 mgr.gibba032.whopeo (mgr.24625) 7354 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:12:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.403107+0000 mgr.gibba032.whopeo (mgr.24625) 7355 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:12:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.404491+0000 mgr.gibba032.whopeo (mgr.24625) 7356 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:12:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.404531+0000 mgr.gibba032.whopeo (mgr.24625) 7357 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:12:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[30791]: cephadm 2021-03-10T16:11:59.404723+0000 mgr.gibba032.whopeo (mgr.24625) 7358 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.393356+0000 mgr.gibba032.whopeo (mgr.24625) 7345 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:12:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.395445+0000 mgr.gibba032.whopeo (mgr.24625) 7346 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:12:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.397276+0000 mgr.gibba032.whopeo (mgr.24625) 7347 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:12:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.397311+0000 mgr.gibba032.whopeo (mgr.24625) 7348 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:12:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.398666+0000 mgr.gibba032.whopeo (mgr.24625) 7349 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:12:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.398703+0000 mgr.gibba032.whopeo (mgr.24625) 7350 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:12:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.399984+0000 mgr.gibba032.whopeo (mgr.24625) 7351 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:12:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.401288+0000 mgr.gibba032.whopeo (mgr.24625) 7352 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:12:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.401323+0000 mgr.gibba032.whopeo (mgr.24625) 7353 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:12:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.402793+0000 mgr.gibba032.whopeo (mgr.24625) 7354 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:12:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.403107+0000 mgr.gibba032.whopeo (mgr.24625) 7355 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:12:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.404491+0000 mgr.gibba032.whopeo (mgr.24625) 7356 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:12:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.404531+0000 mgr.gibba032.whopeo (mgr.24625) 7357 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:12:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:01 gibba011 bash[31774]: cephadm 2021-03-10T16:11:59.404723+0000 mgr.gibba032.whopeo (mgr.24625) 7358 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:02 gibba032 bash[24004]: cluster 2021-03-10T16:12:00.536605+0000 mgr.gibba032.whopeo (mgr.24625) 7359 : cluster [DBG] pgmap v4184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:02 gibba011 bash[30791]: cluster 2021-03-10T16:12:00.536605+0000 mgr.gibba032.whopeo (mgr.24625) 7359 : cluster [DBG] pgmap v4184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:02.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:02 gibba011 bash[31774]: cluster 2021-03-10T16:12:00.536605+0000 mgr.gibba032.whopeo (mgr.24625) 7359 : cluster [DBG] pgmap v4184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:04 gibba032 bash[24004]: cluster 2021-03-10T16:12:02.538124+0000 mgr.gibba032.whopeo (mgr.24625) 7360 : cluster [DBG] pgmap v4185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:04 gibba011 bash[30791]: cluster 2021-03-10T16:12:02.538124+0000 mgr.gibba032.whopeo (mgr.24625) 7360 : cluster [DBG] pgmap v4185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:04 gibba011 bash[31774]: cluster 2021-03-10T16:12:02.538124+0000 mgr.gibba032.whopeo (mgr.24625) 7360 : cluster [DBG] pgmap v4185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:06 gibba011 bash[30791]: cluster 2021-03-10T16:12:04.538980+0000 mgr.gibba032.whopeo (mgr.24625) 7361 : cluster [DBG] pgmap v4186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:06 gibba011 bash[31774]: cluster 2021-03-10T16:12:04.538980+0000 mgr.gibba032.whopeo (mgr.24625) 7361 : cluster [DBG] pgmap v4186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:06 gibba032 bash[24004]: cluster 2021-03-10T16:12:04.538980+0000 mgr.gibba032.whopeo (mgr.24625) 7361 : cluster [DBG] pgmap v4186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:08 gibba011 bash[30791]: cluster 2021-03-10T16:12:06.540719+0000 mgr.gibba032.whopeo (mgr.24625) 7362 : cluster [DBG] pgmap v4187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:08 gibba011 bash[31774]: cluster 2021-03-10T16:12:06.540719+0000 mgr.gibba032.whopeo (mgr.24625) 7362 : cluster [DBG] pgmap v4187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:08 gibba032 bash[24004]: cluster 2021-03-10T16:12:06.540719+0000 mgr.gibba032.whopeo (mgr.24625) 7362 : cluster [DBG] pgmap v4187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:10 gibba011 bash[30791]: cluster 2021-03-10T16:12:08.542374+0000 mgr.gibba032.whopeo (mgr.24625) 7363 : cluster [DBG] pgmap v4188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:10 gibba011 bash[31774]: cluster 2021-03-10T16:12:08.542374+0000 mgr.gibba032.whopeo (mgr.24625) 7363 : cluster [DBG] pgmap v4188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:10 gibba032 bash[24004]: cluster 2021-03-10T16:12:08.542374+0000 mgr.gibba032.whopeo (mgr.24625) 7363 : cluster [DBG] pgmap v4188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:10.770 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:12:10.998 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:12:10.999 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:12:10.999 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 72s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:12:10.999 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:12:10.999 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:12:11.000 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:12:11.000 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:12:11.000 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:12:11.000 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:12:11.001 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:12:11.001 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 72s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:12:11.001 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:12:11.001 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:12:11.001 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:12:11.002 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:12:11.002 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:12:11.002 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:12:11.002 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:12:11.003 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 72s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:12:11.003 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 72s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:12:11.003 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:12:11.247 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:12:11.247 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:12:11.247 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:12:11.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:11.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:12:11.248 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:12:11.248 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:11.248 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:12:11.248 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:12:11.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:11.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:12:11.249 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:12:11.249 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:12:11.250 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:12:11.250 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:12:11.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:11 gibba011 bash[30791]: audit 2021-03-10T16:12:11.248020+0000 mon.a (mon.0) 2638 : audit [DBG] from='client.? 172.21.2.111:0/3992154584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:11.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:11 gibba011 bash[31774]: audit 2021-03-10T16:12:11.248020+0000 mon.a (mon.0) 2638 : audit [DBG] from='client.? 172.21.2.111:0/3992154584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:11 gibba032 bash[24004]: audit 2021-03-10T16:12:11.248020+0000 mon.a (mon.0) 2638 : audit [DBG] from='client.? 172.21.2.111:0/3992154584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[30791]: cluster 2021-03-10T16:12:10.543207+0000 mgr.gibba032.whopeo (mgr.24625) 7364 : cluster [DBG] pgmap v4189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[30791]: audit 2021-03-10T16:12:10.761239+0000 mgr.gibba032.whopeo (mgr.24625) 7365 : audit [DBG] from='client.48987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[30791]: audit 2021-03-10T16:12:10.985401+0000 mgr.gibba032.whopeo (mgr.24625) 7366 : audit [DBG] from='client.48993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[31774]: cluster 2021-03-10T16:12:10.543207+0000 mgr.gibba032.whopeo (mgr.24625) 7364 : cluster [DBG] pgmap v4189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[31774]: audit 2021-03-10T16:12:10.761239+0000 mgr.gibba032.whopeo (mgr.24625) 7365 : audit [DBG] from='client.48987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:12 gibba011 bash[31774]: audit 2021-03-10T16:12:10.985401+0000 mgr.gibba032.whopeo (mgr.24625) 7366 : audit [DBG] from='client.48993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:12 gibba032 bash[24004]: cluster 2021-03-10T16:12:10.543207+0000 mgr.gibba032.whopeo (mgr.24625) 7364 : cluster [DBG] pgmap v4189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:12 gibba032 bash[24004]: audit 2021-03-10T16:12:10.761239+0000 mgr.gibba032.whopeo (mgr.24625) 7365 : audit [DBG] from='client.48987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:12 gibba032 bash[24004]: audit 2021-03-10T16:12:10.985401+0000 mgr.gibba032.whopeo (mgr.24625) 7366 : audit [DBG] from='client.48993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:14.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:14 gibba011 bash[30791]: cluster 2021-03-10T16:12:12.544979+0000 mgr.gibba032.whopeo (mgr.24625) 7367 : cluster [DBG] pgmap v4190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:14.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:14 gibba011 bash[31774]: cluster 2021-03-10T16:12:12.544979+0000 mgr.gibba032.whopeo (mgr.24625) 7367 : cluster [DBG] pgmap v4190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:14 gibba032 bash[24004]: cluster 2021-03-10T16:12:12.544979+0000 mgr.gibba032.whopeo (mgr.24625) 7367 : cluster [DBG] pgmap v4190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:16 gibba011 bash[30791]: cephadm 2021-03-10T16:12:14.420795+0000 mgr.gibba032.whopeo (mgr.24625) 7368 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:16 gibba011 bash[30791]: cluster 2021-03-10T16:12:14.546287+0000 mgr.gibba032.whopeo (mgr.24625) 7369 : cluster [DBG] pgmap v4191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:16 gibba011 bash[31774]: cephadm 2021-03-10T16:12:14.420795+0000 mgr.gibba032.whopeo (mgr.24625) 7368 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:16 gibba011 bash[31774]: cluster 2021-03-10T16:12:14.546287+0000 mgr.gibba032.whopeo (mgr.24625) 7369 : cluster [DBG] pgmap v4191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:16 gibba032 bash[24004]: cephadm 2021-03-10T16:12:14.420795+0000 mgr.gibba032.whopeo (mgr.24625) 7368 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:16 gibba032 bash[24004]: cluster 2021-03-10T16:12:14.546287+0000 mgr.gibba032.whopeo (mgr.24625) 7369 : cluster [DBG] pgmap v4191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:18 gibba011 bash[30791]: cluster 2021-03-10T16:12:16.547413+0000 mgr.gibba032.whopeo (mgr.24625) 7370 : cluster [DBG] pgmap v4192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:18 gibba011 bash[31774]: cluster 2021-03-10T16:12:16.547413+0000 mgr.gibba032.whopeo (mgr.24625) 7370 : cluster [DBG] pgmap v4192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:18 gibba032 bash[24004]: cluster 2021-03-10T16:12:16.547413+0000 mgr.gibba032.whopeo (mgr.24625) 7370 : cluster [DBG] pgmap v4192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:20 gibba011 bash[30791]: cluster 2021-03-10T16:12:18.547832+0000 mgr.gibba032.whopeo (mgr.24625) 7371 : cluster [DBG] pgmap v4193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:20 gibba011 bash[31774]: cluster 2021-03-10T16:12:18.547832+0000 mgr.gibba032.whopeo (mgr.24625) 7371 : cluster [DBG] pgmap v4193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:20 gibba032 bash[24004]: cluster 2021-03-10T16:12:18.547832+0000 mgr.gibba032.whopeo (mgr.24625) 7371 : cluster [DBG] pgmap v4193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:22 gibba032 bash[24004]: cluster 2021-03-10T16:12:20.549344+0000 mgr.gibba032.whopeo (mgr.24625) 7372 : cluster [DBG] pgmap v4194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:22 gibba011 bash[30791]: cluster 2021-03-10T16:12:20.549344+0000 mgr.gibba032.whopeo (mgr.24625) 7372 : cluster [DBG] pgmap v4194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:22 gibba011 bash[31774]: cluster 2021-03-10T16:12:20.549344+0000 mgr.gibba032.whopeo (mgr.24625) 7372 : cluster [DBG] pgmap v4194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:24 gibba032 bash[24004]: cluster 2021-03-10T16:12:22.550479+0000 mgr.gibba032.whopeo (mgr.24625) 7373 : cluster [DBG] pgmap v4195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:24 gibba011 bash[30791]: cluster 2021-03-10T16:12:22.550479+0000 mgr.gibba032.whopeo (mgr.24625) 7373 : cluster [DBG] pgmap v4195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:24 gibba011 bash[31774]: cluster 2021-03-10T16:12:22.550479+0000 mgr.gibba032.whopeo (mgr.24625) 7373 : cluster [DBG] pgmap v4195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:26 gibba032 bash[24004]: cluster 2021-03-10T16:12:24.551947+0000 mgr.gibba032.whopeo (mgr.24625) 7374 : cluster [DBG] pgmap v4196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:26 gibba011 bash[30791]: cluster 2021-03-10T16:12:24.551947+0000 mgr.gibba032.whopeo (mgr.24625) 7374 : cluster [DBG] pgmap v4196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:26 gibba011 bash[31774]: cluster 2021-03-10T16:12:24.551947+0000 mgr.gibba032.whopeo (mgr.24625) 7374 : cluster [DBG] pgmap v4196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:28 gibba032 bash[24004]: cluster 2021-03-10T16:12:26.552571+0000 mgr.gibba032.whopeo (mgr.24625) 7375 : cluster [DBG] pgmap v4197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:28.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:28 gibba011 bash[30791]: cluster 2021-03-10T16:12:26.552571+0000 mgr.gibba032.whopeo (mgr.24625) 7375 : cluster [DBG] pgmap v4197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:28 gibba011 bash[31774]: cluster 2021-03-10T16:12:26.552571+0000 mgr.gibba032.whopeo (mgr.24625) 7375 : cluster [DBG] pgmap v4197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:30 gibba032 bash[24004]: cluster 2021-03-10T16:12:28.553474+0000 mgr.gibba032.whopeo (mgr.24625) 7376 : cluster [DBG] pgmap v4198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:30 gibba011 bash[30791]: cluster 2021-03-10T16:12:28.553474+0000 mgr.gibba032.whopeo (mgr.24625) 7376 : cluster [DBG] pgmap v4198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:30 gibba011 bash[31774]: cluster 2021-03-10T16:12:28.553474+0000 mgr.gibba032.whopeo (mgr.24625) 7376 : cluster [DBG] pgmap v4198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:31 gibba032 bash[24004]: cephadm 2021-03-10T16:12:29.436531+0000 mgr.gibba032.whopeo (mgr.24625) 7377 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:31 gibba032 bash[24004]: cluster 2021-03-10T16:12:30.555019+0000 mgr.gibba032.whopeo (mgr.24625) 7378 : cluster [DBG] pgmap v4199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:31 gibba011 bash[30791]: cephadm 2021-03-10T16:12:29.436531+0000 mgr.gibba032.whopeo (mgr.24625) 7377 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:31 gibba011 bash[30791]: cluster 2021-03-10T16:12:30.555019+0000 mgr.gibba032.whopeo (mgr.24625) 7378 : cluster [DBG] pgmap v4199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:31 gibba011 bash[31774]: cephadm 2021-03-10T16:12:29.436531+0000 mgr.gibba032.whopeo (mgr.24625) 7377 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:31 gibba011 bash[31774]: cluster 2021-03-10T16:12:30.555019+0000 mgr.gibba032.whopeo (mgr.24625) 7378 : cluster [DBG] pgmap v4199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:34.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:33 gibba032 bash[24004]: cluster 2021-03-10T16:12:32.556811+0000 mgr.gibba032.whopeo (mgr.24625) 7379 : cluster [DBG] pgmap v4200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:34.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:33 gibba011 bash[30791]: cluster 2021-03-10T16:12:32.556811+0000 mgr.gibba032.whopeo (mgr.24625) 7379 : cluster [DBG] pgmap v4200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:34.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:33 gibba011 bash[31774]: cluster 2021-03-10T16:12:32.556811+0000 mgr.gibba032.whopeo (mgr.24625) 7379 : cluster [DBG] pgmap v4200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:36 gibba011 bash[30791]: cluster 2021-03-10T16:12:34.557470+0000 mgr.gibba032.whopeo (mgr.24625) 7380 : cluster [DBG] pgmap v4201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:36.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:36 gibba011 bash[31774]: cluster 2021-03-10T16:12:34.557470+0000 mgr.gibba032.whopeo (mgr.24625) 7380 : cluster [DBG] pgmap v4201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:36.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:36 gibba032 bash[24004]: cluster 2021-03-10T16:12:34.557470+0000 mgr.gibba032.whopeo (mgr.24625) 7380 : cluster [DBG] pgmap v4201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:38.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:38 gibba011 bash[30791]: cluster 2021-03-10T16:12:36.558999+0000 mgr.gibba032.whopeo (mgr.24625) 7381 : cluster [DBG] pgmap v4202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:38 gibba011 bash[31774]: cluster 2021-03-10T16:12:36.558999+0000 mgr.gibba032.whopeo (mgr.24625) 7381 : cluster [DBG] pgmap v4202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:38 gibba032 bash[24004]: cluster 2021-03-10T16:12:36.558999+0000 mgr.gibba032.whopeo (mgr.24625) 7381 : cluster [DBG] pgmap v4202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:40 gibba011 bash[30791]: cluster 2021-03-10T16:12:38.560558+0000 mgr.gibba032.whopeo (mgr.24625) 7382 : cluster [DBG] pgmap v4203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:40.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:40 gibba011 bash[31774]: cluster 2021-03-10T16:12:38.560558+0000 mgr.gibba032.whopeo (mgr.24625) 7382 : cluster [DBG] pgmap v4203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:40 gibba032 bash[24004]: cluster 2021-03-10T16:12:38.560558+0000 mgr.gibba032.whopeo (mgr.24625) 7382 : cluster [DBG] pgmap v4203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:41.491 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:12:41.719 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:12:41.720 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:12:41.720 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 102s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:12:41.720 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:12:41.720 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:12:41.720 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:12:41.721 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:12:41.721 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:12:41.721 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:12:41.721 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:12:41.721 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 102s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:12:41.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:12:41.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:12:41.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:12:41.722 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:12:41.723 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:12:41.723 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:12:41.723 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:12:41.723 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:12:41.723 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 102s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:12:41.724 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:12:42.002 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:12:42.002 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:12:42.002 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:12:42.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:42.003 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:12:42.003 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:12:42.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:42.004 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:12:42.004 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:12:42.004 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:12:42.004 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:12:42.004 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:12:42.005 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:12:42.005 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:12:42.005 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:12:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:42 gibba011 bash[30791]: cluster 2021-03-10T16:12:40.561771+0000 mgr.gibba032.whopeo (mgr.24625) 7383 : cluster [DBG] pgmap v4204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:42 gibba011 bash[30791]: audit 2021-03-10T16:12:42.002817+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? 172.21.2.111:0/1114248775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:42 gibba011 bash[31774]: cluster 2021-03-10T16:12:40.561771+0000 mgr.gibba032.whopeo (mgr.24625) 7383 : cluster [DBG] pgmap v4204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:42 gibba011 bash[31774]: audit 2021-03-10T16:12:42.002817+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? 172.21.2.111:0/1114248775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:42 gibba032 bash[24004]: cluster 2021-03-10T16:12:40.561771+0000 mgr.gibba032.whopeo (mgr.24625) 7383 : cluster [DBG] pgmap v4204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:42 gibba032 bash[24004]: audit 2021-03-10T16:12:42.002817+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? 172.21.2.111:0/1114248775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:12:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:43 gibba011 bash[30791]: audit 2021-03-10T16:12:41.481743+0000 mgr.gibba032.whopeo (mgr.24625) 7384 : audit [DBG] from='client.58987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:43 gibba011 bash[30791]: audit 2021-03-10T16:12:41.705506+0000 mgr.gibba032.whopeo (mgr.24625) 7385 : audit [DBG] from='client.58993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:43.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:43 gibba011 bash[31774]: audit 2021-03-10T16:12:41.481743+0000 mgr.gibba032.whopeo (mgr.24625) 7384 : audit [DBG] from='client.58987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:43 gibba011 bash[31774]: audit 2021-03-10T16:12:41.705506+0000 mgr.gibba032.whopeo (mgr.24625) 7385 : audit [DBG] from='client.58993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:43.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:43 gibba032 bash[24004]: audit 2021-03-10T16:12:41.481743+0000 mgr.gibba032.whopeo (mgr.24625) 7384 : audit [DBG] from='client.58987 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:43.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:43 gibba032 bash[24004]: audit 2021-03-10T16:12:41.705506+0000 mgr.gibba032.whopeo (mgr.24625) 7385 : audit [DBG] from='client.58993 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:12:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:44 gibba011 bash[30791]: cluster 2021-03-10T16:12:42.562528+0000 mgr.gibba032.whopeo (mgr.24625) 7386 : cluster [DBG] pgmap v4205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:44.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:44 gibba011 bash[31774]: cluster 2021-03-10T16:12:42.562528+0000 mgr.gibba032.whopeo (mgr.24625) 7386 : cluster [DBG] pgmap v4205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:44.453 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:44 gibba032 bash[24004]: cluster 2021-03-10T16:12:42.562528+0000 mgr.gibba032.whopeo (mgr.24625) 7386 : cluster [DBG] pgmap v4205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:46 gibba011 bash[30791]: cephadm 2021-03-10T16:12:44.452519+0000 mgr.gibba032.whopeo (mgr.24625) 7387 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:46 gibba011 bash[30791]: cluster 2021-03-10T16:12:44.563694+0000 mgr.gibba032.whopeo (mgr.24625) 7388 : cluster [DBG] pgmap v4206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:46 gibba011 bash[31774]: cephadm 2021-03-10T16:12:44.452519+0000 mgr.gibba032.whopeo (mgr.24625) 7387 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:46 gibba011 bash[31774]: cluster 2021-03-10T16:12:44.563694+0000 mgr.gibba032.whopeo (mgr.24625) 7388 : cluster [DBG] pgmap v4206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:46 gibba032 bash[24004]: cephadm 2021-03-10T16:12:44.452519+0000 mgr.gibba032.whopeo (mgr.24625) 7387 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:12:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:46 gibba032 bash[24004]: cluster 2021-03-10T16:12:44.563694+0000 mgr.gibba032.whopeo (mgr.24625) 7388 : cluster [DBG] pgmap v4206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:48.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:48 gibba011 bash[30791]: cluster 2021-03-10T16:12:46.565203+0000 mgr.gibba032.whopeo (mgr.24625) 7389 : cluster [DBG] pgmap v4207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:48.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:48 gibba011 bash[31774]: cluster 2021-03-10T16:12:46.565203+0000 mgr.gibba032.whopeo (mgr.24625) 7389 : cluster [DBG] pgmap v4207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:48 gibba032 bash[24004]: cluster 2021-03-10T16:12:46.565203+0000 mgr.gibba032.whopeo (mgr.24625) 7389 : cluster [DBG] pgmap v4207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:50 gibba011 bash[30791]: cluster 2021-03-10T16:12:48.566583+0000 mgr.gibba032.whopeo (mgr.24625) 7390 : cluster [DBG] pgmap v4208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:50.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:50 gibba011 bash[31774]: cluster 2021-03-10T16:12:48.566583+0000 mgr.gibba032.whopeo (mgr.24625) 7390 : cluster [DBG] pgmap v4208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:50 gibba032 bash[24004]: cluster 2021-03-10T16:12:48.566583+0000 mgr.gibba032.whopeo (mgr.24625) 7390 : cluster [DBG] pgmap v4208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:52.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:52 gibba011 bash[30791]: cluster 2021-03-10T16:12:50.567414+0000 mgr.gibba032.whopeo (mgr.24625) 7391 : cluster [DBG] pgmap v4209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:52.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:52 gibba011 bash[31774]: cluster 2021-03-10T16:12:50.567414+0000 mgr.gibba032.whopeo (mgr.24625) 7391 : cluster [DBG] pgmap v4209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:52 gibba032 bash[24004]: cluster 2021-03-10T16:12:50.567414+0000 mgr.gibba032.whopeo (mgr.24625) 7391 : cluster [DBG] pgmap v4209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:54 gibba011 bash[30791]: cluster 2021-03-10T16:12:52.569092+0000 mgr.gibba032.whopeo (mgr.24625) 7392 : cluster [DBG] pgmap v4210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:54.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:54 gibba011 bash[31774]: cluster 2021-03-10T16:12:52.569092+0000 mgr.gibba032.whopeo (mgr.24625) 7392 : cluster [DBG] pgmap v4210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:54 gibba032 bash[24004]: cluster 2021-03-10T16:12:52.569092+0000 mgr.gibba032.whopeo (mgr.24625) 7392 : cluster [DBG] pgmap v4210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:56 gibba032 bash[24004]: cluster 2021-03-10T16:12:54.570482+0000 mgr.gibba032.whopeo (mgr.24625) 7393 : cluster [DBG] pgmap v4211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:56 gibba011 bash[30791]: cluster 2021-03-10T16:12:54.570482+0000 mgr.gibba032.whopeo (mgr.24625) 7393 : cluster [DBG] pgmap v4211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:56 gibba011 bash[31774]: cluster 2021-03-10T16:12:54.570482+0000 mgr.gibba032.whopeo (mgr.24625) 7393 : cluster [DBG] pgmap v4211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:57.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.892726+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:12:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.893377+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:12:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.970779+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:12:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.971566+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:12:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.972270+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:12:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:57 gibba032 bash[24004]: audit 2021-03-10T16:12:56.972955+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:12:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.892726+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:12:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.893377+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:12:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.970779+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:12:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.971566+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:12:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.972270+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:12:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[30791]: audit 2021-03-10T16:12:56.972955+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:12:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.892726+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:12:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.893377+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:12:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.970779+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:12:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.971566+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:12:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.972270+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:12:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:57 gibba011 bash[31774]: audit 2021-03-10T16:12:56.972955+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:12:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:12:58 gibba032 bash[24004]: cluster 2021-03-10T16:12:56.572347+0000 mgr.gibba032.whopeo (mgr.24625) 7394 : cluster [DBG] pgmap v4212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:12:58 gibba011 bash[30791]: cluster 2021-03-10T16:12:56.572347+0000 mgr.gibba032.whopeo (mgr.24625) 7394 : cluster [DBG] pgmap v4212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:12:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:12:58 gibba011 bash[31774]: cluster 2021-03-10T16:12:56.572347+0000 mgr.gibba032.whopeo (mgr.24625) 7394 : cluster [DBG] pgmap v4212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: cluster 2021-03-10T16:12:58.573014+0000 mgr.gibba032.whopeo (mgr.24625) 7395 : cluster [DBG] pgmap v4213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.697017+0000 mon.a (mon.0) 2646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:13:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.700155+0000 mon.a (mon.0) 2647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:13:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.701580+0000 mon.a (mon.0) 2648 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.702955+0000 mon.a (mon.0) 2649 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.703890+0000 mon.a (mon.0) 2650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.705833+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:00 gibba032 bash[24004]: audit 2021-03-10T16:12:59.707149+0000 mon.a (mon.0) 2652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: cluster 2021-03-10T16:12:58.573014+0000 mgr.gibba032.whopeo (mgr.24625) 7395 : cluster [DBG] pgmap v4213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.697017+0000 mon.a (mon.0) 2646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:13:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.700155+0000 mon.a (mon.0) 2647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:13:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.701580+0000 mon.a (mon.0) 2648 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.702955+0000 mon.a (mon.0) 2649 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.703890+0000 mon.a (mon.0) 2650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.705833+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[30791]: audit 2021-03-10T16:12:59.707149+0000 mon.a (mon.0) 2652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: cluster 2021-03-10T16:12:58.573014+0000 mgr.gibba032.whopeo (mgr.24625) 7395 : cluster [DBG] pgmap v4213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.697017+0000 mon.a (mon.0) 2646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:13:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.700155+0000 mon.a (mon.0) 2647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:13:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.701580+0000 mon.a (mon.0) 2648 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.702955+0000 mon.a (mon.0) 2649 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.703890+0000 mon.a (mon.0) 2650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.705833+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:00 gibba011 bash[31774]: audit 2021-03-10T16:12:59.707149+0000 mon.a (mon.0) 2652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.469157+0000 mgr.gibba032.whopeo (mgr.24625) 7396 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:13:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.470691+0000 mgr.gibba032.whopeo (mgr.24625) 7397 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:13:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.697396+0000 mgr.gibba032.whopeo (mgr.24625) 7398 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:13:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.699127+0000 mgr.gibba032.whopeo (mgr.24625) 7399 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:13:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.700395+0000 mgr.gibba032.whopeo (mgr.24625) 7400 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:13:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.700432+0000 mgr.gibba032.whopeo (mgr.24625) 7401 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:13:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.701544+0000 mgr.gibba032.whopeo (mgr.24625) 7402 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:13:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.701581+0000 mgr.gibba032.whopeo (mgr.24625) 7403 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:13:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.702464+0000 mgr.gibba032.whopeo (mgr.24625) 7404 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:13:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.703436+0000 mgr.gibba032.whopeo (mgr.24625) 7405 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:13:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.703472+0000 mgr.gibba032.whopeo (mgr.24625) 7406 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:13:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.704466+0000 mgr.gibba032.whopeo (mgr.24625) 7407 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:13:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.704781+0000 mgr.gibba032.whopeo (mgr.24625) 7408 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:13:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.705967+0000 mgr.gibba032.whopeo (mgr.24625) 7409 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:13:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.706004+0000 mgr.gibba032.whopeo (mgr.24625) 7410 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:13:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:01 gibba032 bash[24004]: cephadm 2021-03-10T16:12:59.706106+0000 mgr.gibba032.whopeo (mgr.24625) 7411 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.469157+0000 mgr.gibba032.whopeo (mgr.24625) 7396 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:13:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.470691+0000 mgr.gibba032.whopeo (mgr.24625) 7397 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:13:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.697396+0000 mgr.gibba032.whopeo (mgr.24625) 7398 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:13:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.699127+0000 mgr.gibba032.whopeo (mgr.24625) 7399 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:13:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.700395+0000 mgr.gibba032.whopeo (mgr.24625) 7400 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:13:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.700432+0000 mgr.gibba032.whopeo (mgr.24625) 7401 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:13:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.701544+0000 mgr.gibba032.whopeo (mgr.24625) 7402 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:13:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.701581+0000 mgr.gibba032.whopeo (mgr.24625) 7403 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:13:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.702464+0000 mgr.gibba032.whopeo (mgr.24625) 7404 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:13:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.703436+0000 mgr.gibba032.whopeo (mgr.24625) 7405 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:13:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.703472+0000 mgr.gibba032.whopeo (mgr.24625) 7406 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:13:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.704466+0000 mgr.gibba032.whopeo (mgr.24625) 7407 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:13:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.704781+0000 mgr.gibba032.whopeo (mgr.24625) 7408 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:13:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.705967+0000 mgr.gibba032.whopeo (mgr.24625) 7409 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:13:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.706004+0000 mgr.gibba032.whopeo (mgr.24625) 7410 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:13:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[30791]: cephadm 2021-03-10T16:12:59.706106+0000 mgr.gibba032.whopeo (mgr.24625) 7411 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.469157+0000 mgr.gibba032.whopeo (mgr.24625) 7396 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:13:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.470691+0000 mgr.gibba032.whopeo (mgr.24625) 7397 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:13:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.697396+0000 mgr.gibba032.whopeo (mgr.24625) 7398 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:13:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.699127+0000 mgr.gibba032.whopeo (mgr.24625) 7399 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:13:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.700395+0000 mgr.gibba032.whopeo (mgr.24625) 7400 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:13:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.700432+0000 mgr.gibba032.whopeo (mgr.24625) 7401 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:13:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.701544+0000 mgr.gibba032.whopeo (mgr.24625) 7402 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:13:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.701581+0000 mgr.gibba032.whopeo (mgr.24625) 7403 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:13:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.702464+0000 mgr.gibba032.whopeo (mgr.24625) 7404 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:13:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.703436+0000 mgr.gibba032.whopeo (mgr.24625) 7405 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:13:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.703472+0000 mgr.gibba032.whopeo (mgr.24625) 7406 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:13:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.704466+0000 mgr.gibba032.whopeo (mgr.24625) 7407 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:13:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.704781+0000 mgr.gibba032.whopeo (mgr.24625) 7408 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:13:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.705967+0000 mgr.gibba032.whopeo (mgr.24625) 7409 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:13:01.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.706004+0000 mgr.gibba032.whopeo (mgr.24625) 7410 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:13:01.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:01 gibba011 bash[31774]: cephadm 2021-03-10T16:12:59.706106+0000 mgr.gibba032.whopeo (mgr.24625) 7411 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:02 gibba032 bash[24004]: cluster 2021-03-10T16:13:00.574491+0000 mgr.gibba032.whopeo (mgr.24625) 7412 : cluster [DBG] pgmap v4214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:02 gibba011 bash[30791]: cluster 2021-03-10T16:13:00.574491+0000 mgr.gibba032.whopeo (mgr.24625) 7412 : cluster [DBG] pgmap v4214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:02.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:02 gibba011 bash[31774]: cluster 2021-03-10T16:13:00.574491+0000 mgr.gibba032.whopeo (mgr.24625) 7412 : cluster [DBG] pgmap v4214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:04 gibba032 bash[24004]: cluster 2021-03-10T16:13:02.576207+0000 mgr.gibba032.whopeo (mgr.24625) 7413 : cluster [DBG] pgmap v4215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:04 gibba011 bash[30791]: cluster 2021-03-10T16:13:02.576207+0000 mgr.gibba032.whopeo (mgr.24625) 7413 : cluster [DBG] pgmap v4215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:04 gibba011 bash[31774]: cluster 2021-03-10T16:13:02.576207+0000 mgr.gibba032.whopeo (mgr.24625) 7413 : cluster [DBG] pgmap v4215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:06 gibba032 bash[24004]: cluster 2021-03-10T16:13:04.577590+0000 mgr.gibba032.whopeo (mgr.24625) 7414 : cluster [DBG] pgmap v4216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:06.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:06 gibba011 bash[30791]: cluster 2021-03-10T16:13:04.577590+0000 mgr.gibba032.whopeo (mgr.24625) 7414 : cluster [DBG] pgmap v4216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:06.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:06 gibba011 bash[31774]: cluster 2021-03-10T16:13:04.577590+0000 mgr.gibba032.whopeo (mgr.24625) 7414 : cluster [DBG] pgmap v4216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:08 gibba032 bash[24004]: cluster 2021-03-10T16:13:06.578613+0000 mgr.gibba032.whopeo (mgr.24625) 7415 : cluster [DBG] pgmap v4217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:08 gibba011 bash[30791]: cluster 2021-03-10T16:13:06.578613+0000 mgr.gibba032.whopeo (mgr.24625) 7415 : cluster [DBG] pgmap v4217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:08 gibba011 bash[31774]: cluster 2021-03-10T16:13:06.578613+0000 mgr.gibba032.whopeo (mgr.24625) 7415 : cluster [DBG] pgmap v4217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:10 gibba032 bash[24004]: cluster 2021-03-10T16:13:08.580157+0000 mgr.gibba032.whopeo (mgr.24625) 7416 : cluster [DBG] pgmap v4218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:10 gibba011 bash[30791]: cluster 2021-03-10T16:13:08.580157+0000 mgr.gibba032.whopeo (mgr.24625) 7416 : cluster [DBG] pgmap v4218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:10 gibba011 bash[31774]: cluster 2021-03-10T16:13:08.580157+0000 mgr.gibba032.whopeo (mgr.24625) 7416 : cluster [DBG] pgmap v4218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:12.246 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:13:12.471 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:13:12.471 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:13:12.471 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:13:12.472 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:13:12.472 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:13:12.472 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:13:12.472 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:13:12.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:13:12.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:13:12.473 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:13:12.473 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:13:12.473 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:13:12.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:13:12.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:13:12.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:13:12.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:13:12.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:13:12.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:13:12.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:13:12.475 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:13:12.475 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:13:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:12 gibba032 bash[24004]: cluster 2021-03-10T16:13:10.581811+0000 mgr.gibba032.whopeo (mgr.24625) 7417 : cluster [DBG] pgmap v4219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:12.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:12 gibba011 bash[30791]: cluster 2021-03-10T16:13:10.581811+0000 mgr.gibba032.whopeo (mgr.24625) 7417 : cluster [DBG] pgmap v4219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:12 gibba011 bash[31774]: cluster 2021-03-10T16:13:10.581811+0000 mgr.gibba032.whopeo (mgr.24625) 7417 : cluster [DBG] pgmap v4219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:12.721 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:13:12.722 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:13:12.722 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:13:12.722 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:12.723 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:13:12.723 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:13:12.723 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:12.723 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:13:12.723 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:13:12.724 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:12.724 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:13:12.724 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:13:12.724 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:13:12.725 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:13:12.725 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:13:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:13 gibba032 bash[24004]: audit 2021-03-10T16:13:12.235477+0000 mgr.gibba032.whopeo (mgr.24625) 7418 : audit [DBG] from='client.59005 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:13.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:13 gibba032 bash[24004]: audit 2021-03-10T16:13:12.722868+0000 mon.c (mon.1) 141 : audit [DBG] from='client.? 172.21.2.111:0/3338139016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:13.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:13 gibba011 bash[30791]: audit 2021-03-10T16:13:12.235477+0000 mgr.gibba032.whopeo (mgr.24625) 7418 : audit [DBG] from='client.59005 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:13.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:13 gibba011 bash[30791]: audit 2021-03-10T16:13:12.722868+0000 mon.c (mon.1) 141 : audit [DBG] from='client.? 172.21.2.111:0/3338139016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:13.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:13 gibba011 bash[31774]: audit 2021-03-10T16:13:12.235477+0000 mgr.gibba032.whopeo (mgr.24625) 7418 : audit [DBG] from='client.59005 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:13.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:13 gibba011 bash[31774]: audit 2021-03-10T16:13:12.722868+0000 mon.c (mon.1) 141 : audit [DBG] from='client.? 172.21.2.111:0/3338139016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:14 gibba032 bash[24004]: audit 2021-03-10T16:13:12.457783+0000 mgr.gibba032.whopeo (mgr.24625) 7419 : audit [DBG] from='client.49029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:14 gibba032 bash[24004]: cluster 2021-03-10T16:13:12.583532+0000 mgr.gibba032.whopeo (mgr.24625) 7420 : cluster [DBG] pgmap v4220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:14 gibba011 bash[30791]: audit 2021-03-10T16:13:12.457783+0000 mgr.gibba032.whopeo (mgr.24625) 7419 : audit [DBG] from='client.49029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:14.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:14 gibba011 bash[30791]: cluster 2021-03-10T16:13:12.583532+0000 mgr.gibba032.whopeo (mgr.24625) 7420 : cluster [DBG] pgmap v4220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:14 gibba011 bash[31774]: audit 2021-03-10T16:13:12.457783+0000 mgr.gibba032.whopeo (mgr.24625) 7419 : audit [DBG] from='client.49029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:14 gibba011 bash[31774]: cluster 2021-03-10T16:13:12.583532+0000 mgr.gibba032.whopeo (mgr.24625) 7420 : cluster [DBG] pgmap v4220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:16 gibba032 bash[24004]: cluster 2021-03-10T16:13:14.584427+0000 mgr.gibba032.whopeo (mgr.24625) 7421 : cluster [DBG] pgmap v4221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:16 gibba032 bash[24004]: cephadm 2021-03-10T16:13:14.721891+0000 mgr.gibba032.whopeo (mgr.24625) 7422 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:16 gibba011 bash[30791]: cluster 2021-03-10T16:13:14.584427+0000 mgr.gibba032.whopeo (mgr.24625) 7421 : cluster [DBG] pgmap v4221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:16 gibba011 bash[30791]: cephadm 2021-03-10T16:13:14.721891+0000 mgr.gibba032.whopeo (mgr.24625) 7422 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:16 gibba011 bash[31774]: cluster 2021-03-10T16:13:14.584427+0000 mgr.gibba032.whopeo (mgr.24625) 7421 : cluster [DBG] pgmap v4221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:16 gibba011 bash[31774]: cephadm 2021-03-10T16:13:14.721891+0000 mgr.gibba032.whopeo (mgr.24625) 7422 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:18 gibba032 bash[24004]: cluster 2021-03-10T16:13:16.585863+0000 mgr.gibba032.whopeo (mgr.24625) 7423 : cluster [DBG] pgmap v4222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:18 gibba011 bash[30791]: cluster 2021-03-10T16:13:16.585863+0000 mgr.gibba032.whopeo (mgr.24625) 7423 : cluster [DBG] pgmap v4222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:18 gibba011 bash[31774]: cluster 2021-03-10T16:13:16.585863+0000 mgr.gibba032.whopeo (mgr.24625) 7423 : cluster [DBG] pgmap v4222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:20 gibba032 bash[24004]: cluster 2021-03-10T16:13:18.587548+0000 mgr.gibba032.whopeo (mgr.24625) 7424 : cluster [DBG] pgmap v4223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:20 gibba011 bash[30791]: cluster 2021-03-10T16:13:18.587548+0000 mgr.gibba032.whopeo (mgr.24625) 7424 : cluster [DBG] pgmap v4223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:20 gibba011 bash[31774]: cluster 2021-03-10T16:13:18.587548+0000 mgr.gibba032.whopeo (mgr.24625) 7424 : cluster [DBG] pgmap v4223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:22 gibba032 bash[24004]: cluster 2021-03-10T16:13:20.589229+0000 mgr.gibba032.whopeo (mgr.24625) 7425 : cluster [DBG] pgmap v4224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:22 gibba011 bash[30791]: cluster 2021-03-10T16:13:20.589229+0000 mgr.gibba032.whopeo (mgr.24625) 7425 : cluster [DBG] pgmap v4224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:22 gibba011 bash[31774]: cluster 2021-03-10T16:13:20.589229+0000 mgr.gibba032.whopeo (mgr.24625) 7425 : cluster [DBG] pgmap v4224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:24 gibba032 bash[24004]: cluster 2021-03-10T16:13:22.590990+0000 mgr.gibba032.whopeo (mgr.24625) 7426 : cluster [DBG] pgmap v4225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:24.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:24 gibba011 bash[30791]: cluster 2021-03-10T16:13:22.590990+0000 mgr.gibba032.whopeo (mgr.24625) 7426 : cluster [DBG] pgmap v4225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:24 gibba011 bash[31774]: cluster 2021-03-10T16:13:22.590990+0000 mgr.gibba032.whopeo (mgr.24625) 7426 : cluster [DBG] pgmap v4225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:26 gibba032 bash[24004]: cluster 2021-03-10T16:13:24.592564+0000 mgr.gibba032.whopeo (mgr.24625) 7427 : cluster [DBG] pgmap v4226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:26 gibba011 bash[30791]: cluster 2021-03-10T16:13:24.592564+0000 mgr.gibba032.whopeo (mgr.24625) 7427 : cluster [DBG] pgmap v4226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:26.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:26 gibba011 bash[31774]: cluster 2021-03-10T16:13:24.592564+0000 mgr.gibba032.whopeo (mgr.24625) 7427 : cluster [DBG] pgmap v4226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:28 gibba032 bash[24004]: cluster 2021-03-10T16:13:26.594403+0000 mgr.gibba032.whopeo (mgr.24625) 7428 : cluster [DBG] pgmap v4227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:28 gibba011 bash[30791]: cluster 2021-03-10T16:13:26.594403+0000 mgr.gibba032.whopeo (mgr.24625) 7428 : cluster [DBG] pgmap v4227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:28 gibba011 bash[31774]: cluster 2021-03-10T16:13:26.594403+0000 mgr.gibba032.whopeo (mgr.24625) 7428 : cluster [DBG] pgmap v4227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:30 gibba032 bash[24004]: cluster 2021-03-10T16:13:28.596007+0000 mgr.gibba032.whopeo (mgr.24625) 7429 : cluster [DBG] pgmap v4228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:30 gibba011 bash[30791]: cluster 2021-03-10T16:13:28.596007+0000 mgr.gibba032.whopeo (mgr.24625) 7429 : cluster [DBG] pgmap v4228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:30.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:30 gibba011 bash[31774]: cluster 2021-03-10T16:13:28.596007+0000 mgr.gibba032.whopeo (mgr.24625) 7429 : cluster [DBG] pgmap v4228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:31.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:31 gibba011 bash[30791]: cephadm 2021-03-10T16:13:29.737884+0000 mgr.gibba032.whopeo (mgr.24625) 7430 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:31.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:31 gibba011 bash[31774]: cephadm 2021-03-10T16:13:29.737884+0000 mgr.gibba032.whopeo (mgr.24625) 7430 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:31 gibba032 bash[24004]: cephadm 2021-03-10T16:13:29.737884+0000 mgr.gibba032.whopeo (mgr.24625) 7430 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:32 gibba011 bash[30791]: cluster 2021-03-10T16:13:30.596683+0000 mgr.gibba032.whopeo (mgr.24625) 7431 : cluster [DBG] pgmap v4229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:32 gibba011 bash[31774]: cluster 2021-03-10T16:13:30.596683+0000 mgr.gibba032.whopeo (mgr.24625) 7431 : cluster [DBG] pgmap v4229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:32 gibba032 bash[24004]: cluster 2021-03-10T16:13:30.596683+0000 mgr.gibba032.whopeo (mgr.24625) 7431 : cluster [DBG] pgmap v4229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:34.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:34 gibba011 bash[30791]: cluster 2021-03-10T16:13:32.598368+0000 mgr.gibba032.whopeo (mgr.24625) 7432 : cluster [DBG] pgmap v4230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:34 gibba011 bash[31774]: cluster 2021-03-10T16:13:32.598368+0000 mgr.gibba032.whopeo (mgr.24625) 7432 : cluster [DBG] pgmap v4230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:34 gibba032 bash[24004]: cluster 2021-03-10T16:13:32.598368+0000 mgr.gibba032.whopeo (mgr.24625) 7432 : cluster [DBG] pgmap v4230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:36 gibba011 bash[30791]: cluster 2021-03-10T16:13:34.599901+0000 mgr.gibba032.whopeo (mgr.24625) 7433 : cluster [DBG] pgmap v4231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:36 gibba011 bash[31774]: cluster 2021-03-10T16:13:34.599901+0000 mgr.gibba032.whopeo (mgr.24625) 7433 : cluster [DBG] pgmap v4231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:36 gibba032 bash[24004]: cluster 2021-03-10T16:13:34.599901+0000 mgr.gibba032.whopeo (mgr.24625) 7433 : cluster [DBG] pgmap v4231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:38 gibba011 bash[30791]: cluster 2021-03-10T16:13:36.601542+0000 mgr.gibba032.whopeo (mgr.24625) 7434 : cluster [DBG] pgmap v4232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:38 gibba011 bash[31774]: cluster 2021-03-10T16:13:36.601542+0000 mgr.gibba032.whopeo (mgr.24625) 7434 : cluster [DBG] pgmap v4232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:38 gibba032 bash[24004]: cluster 2021-03-10T16:13:36.601542+0000 mgr.gibba032.whopeo (mgr.24625) 7434 : cluster [DBG] pgmap v4232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:40 gibba011 bash[30791]: cluster 2021-03-10T16:13:38.602561+0000 mgr.gibba032.whopeo (mgr.24625) 7435 : cluster [DBG] pgmap v4233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:40 gibba011 bash[31774]: cluster 2021-03-10T16:13:38.602561+0000 mgr.gibba032.whopeo (mgr.24625) 7435 : cluster [DBG] pgmap v4233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:40 gibba032 bash[24004]: cluster 2021-03-10T16:13:38.602561+0000 mgr.gibba032.whopeo (mgr.24625) 7435 : cluster [DBG] pgmap v4233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:42 gibba011 bash[30791]: cluster 2021-03-10T16:13:40.604202+0000 mgr.gibba032.whopeo (mgr.24625) 7436 : cluster [DBG] pgmap v4234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:42 gibba011 bash[31774]: cluster 2021-03-10T16:13:40.604202+0000 mgr.gibba032.whopeo (mgr.24625) 7436 : cluster [DBG] pgmap v4234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:42 gibba032 bash[24004]: cluster 2021-03-10T16:13:40.604202+0000 mgr.gibba032.whopeo (mgr.24625) 7436 : cluster [DBG] pgmap v4234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:42.964 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:13:43.224 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:13:43.224 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:13:43.224 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:13:43.225 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:13:43.225 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:13:43.225 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:13:43.225 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:13:43.225 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:13:43.226 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:13:43.226 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:13:43.226 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:13:43.226 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:13:43.227 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:13:43.227 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:13:43.227 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:13:43.227 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:13:43.227 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:13:43.228 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:13:43.228 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:13:43.228 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:13:43.228 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:13:43.480 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:13:43.481 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:13:43.481 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:13:43.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:43.482 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:13:43.482 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:13:43.482 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:43.482 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:13:43.483 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:13:43.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:13:43.483 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:13:43.483 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:13:43.484 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:13:43.484 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:13:43.484 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:13:44.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[30791]: cluster 2021-03-10T16:13:42.606009+0000 mgr.gibba032.whopeo (mgr.24625) 7437 : cluster [DBG] pgmap v4235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:44.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[30791]: audit 2021-03-10T16:13:42.954766+0000 mgr.gibba032.whopeo (mgr.24625) 7438 : audit [DBG] from='client.59023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[30791]: audit 2021-03-10T16:13:43.182919+0000 mgr.gibba032.whopeo (mgr.24625) 7439 : audit [DBG] from='client.59029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[30791]: audit 2021-03-10T16:13:43.481929+0000 mon.c (mon.1) 142 : audit [DBG] from='client.? 172.21.2.111:0/1781162954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:44.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[31774]: cluster 2021-03-10T16:13:42.606009+0000 mgr.gibba032.whopeo (mgr.24625) 7437 : cluster [DBG] pgmap v4235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:44.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[31774]: audit 2021-03-10T16:13:42.954766+0000 mgr.gibba032.whopeo (mgr.24625) 7438 : audit [DBG] from='client.59023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[31774]: audit 2021-03-10T16:13:43.182919+0000 mgr.gibba032.whopeo (mgr.24625) 7439 : audit [DBG] from='client.59029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:44 gibba011 bash[31774]: audit 2021-03-10T16:13:43.481929+0000 mon.c (mon.1) 142 : audit [DBG] from='client.? 172.21.2.111:0/1781162954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:44.751 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:44 gibba032 bash[24004]: cluster 2021-03-10T16:13:42.606009+0000 mgr.gibba032.whopeo (mgr.24625) 7437 : cluster [DBG] pgmap v4235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:44.752 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:44 gibba032 bash[24004]: audit 2021-03-10T16:13:42.954766+0000 mgr.gibba032.whopeo (mgr.24625) 7438 : audit [DBG] from='client.59023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.752 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:44 gibba032 bash[24004]: audit 2021-03-10T16:13:43.182919+0000 mgr.gibba032.whopeo (mgr.24625) 7439 : audit [DBG] from='client.59029 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:13:44.752 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:44 gibba032 bash[24004]: audit 2021-03-10T16:13:43.481929+0000 mon.c (mon.1) 142 : audit [DBG] from='client.? 172.21.2.111:0/1781162954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:13:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:46 gibba011 bash[30791]: cluster 2021-03-10T16:13:44.607479+0000 mgr.gibba032.whopeo (mgr.24625) 7440 : cluster [DBG] pgmap v4236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:46 gibba011 bash[30791]: cephadm 2021-03-10T16:13:44.750562+0000 mgr.gibba032.whopeo (mgr.24625) 7441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:46 gibba011 bash[31774]: cluster 2021-03-10T16:13:44.607479+0000 mgr.gibba032.whopeo (mgr.24625) 7440 : cluster [DBG] pgmap v4236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:46 gibba011 bash[31774]: cephadm 2021-03-10T16:13:44.750562+0000 mgr.gibba032.whopeo (mgr.24625) 7441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:46 gibba032 bash[24004]: cluster 2021-03-10T16:13:44.607479+0000 mgr.gibba032.whopeo (mgr.24625) 7440 : cluster [DBG] pgmap v4236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:46 gibba032 bash[24004]: cephadm 2021-03-10T16:13:44.750562+0000 mgr.gibba032.whopeo (mgr.24625) 7441 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:13:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:48 gibba032 bash[24004]: cluster 2021-03-10T16:13:46.608286+0000 mgr.gibba032.whopeo (mgr.24625) 7442 : cluster [DBG] pgmap v4237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:48 gibba011 bash[30791]: cluster 2021-03-10T16:13:46.608286+0000 mgr.gibba032.whopeo (mgr.24625) 7442 : cluster [DBG] pgmap v4237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:48 gibba011 bash[31774]: cluster 2021-03-10T16:13:46.608286+0000 mgr.gibba032.whopeo (mgr.24625) 7442 : cluster [DBG] pgmap v4237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:50.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:50 gibba032 bash[24004]: cluster 2021-03-10T16:13:48.609849+0000 mgr.gibba032.whopeo (mgr.24625) 7443 : cluster [DBG] pgmap v4238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:50.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:50 gibba011 bash[30791]: cluster 2021-03-10T16:13:48.609849+0000 mgr.gibba032.whopeo (mgr.24625) 7443 : cluster [DBG] pgmap v4238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:50.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:50 gibba011 bash[31774]: cluster 2021-03-10T16:13:48.609849+0000 mgr.gibba032.whopeo (mgr.24625) 7443 : cluster [DBG] pgmap v4238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:52 gibba032 bash[24004]: cluster 2021-03-10T16:13:50.611242+0000 mgr.gibba032.whopeo (mgr.24625) 7444 : cluster [DBG] pgmap v4239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:52 gibba011 bash[30791]: cluster 2021-03-10T16:13:50.611242+0000 mgr.gibba032.whopeo (mgr.24625) 7444 : cluster [DBG] pgmap v4239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:52 gibba011 bash[31774]: cluster 2021-03-10T16:13:50.611242+0000 mgr.gibba032.whopeo (mgr.24625) 7444 : cluster [DBG] pgmap v4239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:54 gibba032 bash[24004]: cluster 2021-03-10T16:13:52.612999+0000 mgr.gibba032.whopeo (mgr.24625) 7445 : cluster [DBG] pgmap v4240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:54 gibba011 bash[30791]: cluster 2021-03-10T16:13:52.612999+0000 mgr.gibba032.whopeo (mgr.24625) 7445 : cluster [DBG] pgmap v4240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:54 gibba011 bash[31774]: cluster 2021-03-10T16:13:52.612999+0000 mgr.gibba032.whopeo (mgr.24625) 7445 : cluster [DBG] pgmap v4240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:56 gibba032 bash[24004]: cluster 2021-03-10T16:13:54.613995+0000 mgr.gibba032.whopeo (mgr.24625) 7446 : cluster [DBG] pgmap v4241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:56.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:56 gibba011 bash[30791]: cluster 2021-03-10T16:13:54.613995+0000 mgr.gibba032.whopeo (mgr.24625) 7446 : cluster [DBG] pgmap v4241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:56 gibba011 bash[31774]: cluster 2021-03-10T16:13:54.613995+0000 mgr.gibba032.whopeo (mgr.24625) 7446 : cluster [DBG] pgmap v4241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.898347+0000 mon.a (mon.0) 2653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:13:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.900409+0000 mon.a (mon.0) 2654 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:13:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.981359+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:13:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.982166+0000 mon.a (mon.0) 2656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:13:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.982816+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:13:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:57 gibba032 bash[24004]: audit 2021-03-10T16:13:56.983578+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:13:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.898347+0000 mon.a (mon.0) 2653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:13:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.900409+0000 mon.a (mon.0) 2654 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:13:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.981359+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:13:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.982166+0000 mon.a (mon.0) 2656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:13:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.982816+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:13:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[30791]: audit 2021-03-10T16:13:56.983578+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:13:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.898347+0000 mon.a (mon.0) 2653 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:13:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.900409+0000 mon.a (mon.0) 2654 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:13:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.981359+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:13:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.982166+0000 mon.a (mon.0) 2656 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:13:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.982816+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:13:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:57 gibba011 bash[31774]: audit 2021-03-10T16:13:56.983578+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:13:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:13:58 gibba032 bash[24004]: cluster 2021-03-10T16:13:56.615681+0000 mgr.gibba032.whopeo (mgr.24625) 7447 : cluster [DBG] pgmap v4242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:13:58 gibba011 bash[30791]: cluster 2021-03-10T16:13:56.615681+0000 mgr.gibba032.whopeo (mgr.24625) 7447 : cluster [DBG] pgmap v4242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:13:58.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:13:58 gibba011 bash[31774]: cluster 2021-03-10T16:13:56.615681+0000 mgr.gibba032.whopeo (mgr.24625) 7447 : cluster [DBG] pgmap v4242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: cluster 2021-03-10T16:13:58.617354+0000 mgr.gibba032.whopeo (mgr.24625) 7448 : cluster [DBG] pgmap v4243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:13:59.997575+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:14:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.000703+0000 mon.a (mon.0) 2660 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:14:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.001910+0000 mon.a (mon.0) 2661 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.003120+0000 mon.a (mon.0) 2662 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.004237+0000 mon.a (mon.0) 2663 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.006159+0000 mon.a (mon.0) 2664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:00 gibba032 bash[24004]: audit 2021-03-10T16:14:00.007666+0000 mon.a (mon.0) 2665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: cluster 2021-03-10T16:13:58.617354+0000 mgr.gibba032.whopeo (mgr.24625) 7448 : cluster [DBG] pgmap v4243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:13:59.997575+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:14:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.000703+0000 mon.a (mon.0) 2660 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:14:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.001910+0000 mon.a (mon.0) 2661 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.003120+0000 mon.a (mon.0) 2662 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.004237+0000 mon.a (mon.0) 2663 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.006159+0000 mon.a (mon.0) 2664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[30791]: audit 2021-03-10T16:14:00.007666+0000 mon.a (mon.0) 2665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: cluster 2021-03-10T16:13:58.617354+0000 mgr.gibba032.whopeo (mgr.24625) 7448 : cluster [DBG] pgmap v4243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:13:59.997575+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:14:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.000703+0000 mon.a (mon.0) 2660 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:14:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.001910+0000 mon.a (mon.0) 2661 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.003120+0000 mon.a (mon.0) 2662 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.004237+0000 mon.a (mon.0) 2663 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.006159+0000 mon.a (mon.0) 2664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:00 gibba011 bash[31774]: audit 2021-03-10T16:14:00.007666+0000 mon.a (mon.0) 2665 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:13:59.767425+0000 mgr.gibba032.whopeo (mgr.24625) 7449 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:14:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:13:59.769032+0000 mgr.gibba032.whopeo (mgr.24625) 7450 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:14:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:13:59.997993+0000 mgr.gibba032.whopeo (mgr.24625) 7451 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:14:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:13:59.999550+0000 mgr.gibba032.whopeo (mgr.24625) 7452 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:14:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.000717+0000 mgr.gibba032.whopeo (mgr.24625) 7453 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:14:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.000753+0000 mgr.gibba032.whopeo (mgr.24625) 7454 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:14:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.001865+0000 mgr.gibba032.whopeo (mgr.24625) 7455 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:14:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.001976+0000 mgr.gibba032.whopeo (mgr.24625) 7456 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:14:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.002963+0000 mgr.gibba032.whopeo (mgr.24625) 7457 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:14:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.003771+0000 mgr.gibba032.whopeo (mgr.24625) 7458 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:14:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.003807+0000 mgr.gibba032.whopeo (mgr.24625) 7459 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:14:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.004927+0000 mgr.gibba032.whopeo (mgr.24625) 7460 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:14:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.005365+0000 mgr.gibba032.whopeo (mgr.24625) 7461 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:14:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.006368+0000 mgr.gibba032.whopeo (mgr.24625) 7462 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:14:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.006442+0000 mgr.gibba032.whopeo (mgr.24625) 7463 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:14:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:01 gibba032 bash[24004]: cephadm 2021-03-10T16:14:00.006549+0000 mgr.gibba032.whopeo (mgr.24625) 7464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:13:59.767425+0000 mgr.gibba032.whopeo (mgr.24625) 7449 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:14:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:13:59.769032+0000 mgr.gibba032.whopeo (mgr.24625) 7450 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:14:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:13:59.997993+0000 mgr.gibba032.whopeo (mgr.24625) 7451 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:14:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:13:59.999550+0000 mgr.gibba032.whopeo (mgr.24625) 7452 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:14:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.000717+0000 mgr.gibba032.whopeo (mgr.24625) 7453 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:14:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.000753+0000 mgr.gibba032.whopeo (mgr.24625) 7454 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:14:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.001865+0000 mgr.gibba032.whopeo (mgr.24625) 7455 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:14:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.001976+0000 mgr.gibba032.whopeo (mgr.24625) 7456 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:14:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.002963+0000 mgr.gibba032.whopeo (mgr.24625) 7457 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:14:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.003771+0000 mgr.gibba032.whopeo (mgr.24625) 7458 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:14:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.003807+0000 mgr.gibba032.whopeo (mgr.24625) 7459 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:14:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.004927+0000 mgr.gibba032.whopeo (mgr.24625) 7460 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:14:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.005365+0000 mgr.gibba032.whopeo (mgr.24625) 7461 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:14:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.006368+0000 mgr.gibba032.whopeo (mgr.24625) 7462 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:14:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.006442+0000 mgr.gibba032.whopeo (mgr.24625) 7463 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:14:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[30791]: cephadm 2021-03-10T16:14:00.006549+0000 mgr.gibba032.whopeo (mgr.24625) 7464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:13:59.767425+0000 mgr.gibba032.whopeo (mgr.24625) 7449 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:14:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:13:59.769032+0000 mgr.gibba032.whopeo (mgr.24625) 7450 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:14:01.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:13:59.997993+0000 mgr.gibba032.whopeo (mgr.24625) 7451 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:14:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:13:59.999550+0000 mgr.gibba032.whopeo (mgr.24625) 7452 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:14:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.000717+0000 mgr.gibba032.whopeo (mgr.24625) 7453 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:14:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.000753+0000 mgr.gibba032.whopeo (mgr.24625) 7454 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:14:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.001865+0000 mgr.gibba032.whopeo (mgr.24625) 7455 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:14:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.001976+0000 mgr.gibba032.whopeo (mgr.24625) 7456 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:14:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.002963+0000 mgr.gibba032.whopeo (mgr.24625) 7457 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:14:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.003771+0000 mgr.gibba032.whopeo (mgr.24625) 7458 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:14:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.003807+0000 mgr.gibba032.whopeo (mgr.24625) 7459 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:14:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.004927+0000 mgr.gibba032.whopeo (mgr.24625) 7460 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:14:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.005365+0000 mgr.gibba032.whopeo (mgr.24625) 7461 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:14:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.006368+0000 mgr.gibba032.whopeo (mgr.24625) 7462 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:14:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.006442+0000 mgr.gibba032.whopeo (mgr.24625) 7463 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:14:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:01 gibba011 bash[31774]: cephadm 2021-03-10T16:14:00.006549+0000 mgr.gibba032.whopeo (mgr.24625) 7464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:02 gibba032 bash[24004]: cluster 2021-03-10T16:14:00.619000+0000 mgr.gibba032.whopeo (mgr.24625) 7465 : cluster [DBG] pgmap v4244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:02 gibba011 bash[30791]: cluster 2021-03-10T16:14:00.619000+0000 mgr.gibba032.whopeo (mgr.24625) 7465 : cluster [DBG] pgmap v4244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:02 gibba011 bash[31774]: cluster 2021-03-10T16:14:00.619000+0000 mgr.gibba032.whopeo (mgr.24625) 7465 : cluster [DBG] pgmap v4244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:03 gibba032 bash[24004]: cluster 2021-03-10T16:14:02.620087+0000 mgr.gibba032.whopeo (mgr.24625) 7466 : cluster [DBG] pgmap v4245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:03 gibba011 bash[30791]: cluster 2021-03-10T16:14:02.620087+0000 mgr.gibba032.whopeo (mgr.24625) 7466 : cluster [DBG] pgmap v4245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:03.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:03 gibba011 bash[31774]: cluster 2021-03-10T16:14:02.620087+0000 mgr.gibba032.whopeo (mgr.24625) 7466 : cluster [DBG] pgmap v4245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:06 gibba011 bash[30791]: cluster 2021-03-10T16:14:04.621557+0000 mgr.gibba032.whopeo (mgr.24625) 7467 : cluster [DBG] pgmap v4246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:06.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:06 gibba011 bash[31774]: cluster 2021-03-10T16:14:04.621557+0000 mgr.gibba032.whopeo (mgr.24625) 7467 : cluster [DBG] pgmap v4246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:06 gibba032 bash[24004]: cluster 2021-03-10T16:14:04.621557+0000 mgr.gibba032.whopeo (mgr.24625) 7467 : cluster [DBG] pgmap v4246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:08.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:08 gibba011 bash[30791]: cluster 2021-03-10T16:14:06.623324+0000 mgr.gibba032.whopeo (mgr.24625) 7468 : cluster [DBG] pgmap v4247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:08.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:08 gibba011 bash[31774]: cluster 2021-03-10T16:14:06.623324+0000 mgr.gibba032.whopeo (mgr.24625) 7468 : cluster [DBG] pgmap v4247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:08 gibba032 bash[24004]: cluster 2021-03-10T16:14:06.623324+0000 mgr.gibba032.whopeo (mgr.24625) 7468 : cluster [DBG] pgmap v4247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:10 gibba011 bash[30791]: cluster 2021-03-10T16:14:08.625000+0000 mgr.gibba032.whopeo (mgr.24625) 7469 : cluster [DBG] pgmap v4248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:10 gibba011 bash[31774]: cluster 2021-03-10T16:14:08.625000+0000 mgr.gibba032.whopeo (mgr.24625) 7469 : cluster [DBG] pgmap v4248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:10 gibba032 bash[24004]: cluster 2021-03-10T16:14:08.625000+0000 mgr.gibba032.whopeo (mgr.24625) 7469 : cluster [DBG] pgmap v4248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:12 gibba032 bash[24004]: cluster 2021-03-10T16:14:10.626485+0000 mgr.gibba032.whopeo (mgr.24625) 7470 : cluster [DBG] pgmap v4249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:12.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:12 gibba011 bash[30791]: cluster 2021-03-10T16:14:10.626485+0000 mgr.gibba032.whopeo (mgr.24625) 7470 : cluster [DBG] pgmap v4249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:12.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:12 gibba011 bash[31774]: cluster 2021-03-10T16:14:10.626485+0000 mgr.gibba032.whopeo (mgr.24625) 7470 : cluster [DBG] pgmap v4249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:13.727 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:14:13.955 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:14:13.956 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:14:13.956 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:14:13.956 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:14:13.956 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:14:13.957 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:14:13.957 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:14:13.957 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:14:13.957 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:14:13.958 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:14:13.958 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:14:13.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:14:13.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:14:13.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:14:13.959 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:14:13.959 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:14:13.959 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:14:13.959 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:14:13.960 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:14:13.960 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:14:13.960 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:14:14.205 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:14:14.206 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:14:14.206 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:14:14.206 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:14.207 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:14:14.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:14:14.207 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:14.207 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:14:14.207 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:14:14.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:14.208 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:14:14.208 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:14:14.208 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:14:14.209 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:14:14.209 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:14:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:14 gibba032 bash[24004]: cluster 2021-03-10T16:14:12.628275+0000 mgr.gibba032.whopeo (mgr.24625) 7471 : cluster [DBG] pgmap v4250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:14 gibba011 bash[30791]: cluster 2021-03-10T16:14:12.628275+0000 mgr.gibba032.whopeo (mgr.24625) 7471 : cluster [DBG] pgmap v4250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:14 gibba011 bash[31774]: cluster 2021-03-10T16:14:12.628275+0000 mgr.gibba032.whopeo (mgr.24625) 7471 : cluster [DBG] pgmap v4250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:15.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:15 gibba032 bash[24004]: audit 2021-03-10T16:14:13.717301+0000 mgr.gibba032.whopeo (mgr.24625) 7472 : audit [DBG] from='client.59041 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:15 gibba032 bash[24004]: audit 2021-03-10T16:14:13.942579+0000 mgr.gibba032.whopeo (mgr.24625) 7473 : audit [DBG] from='client.59047 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:15 gibba032 bash[24004]: audit 2021-03-10T16:14:14.206955+0000 mon.a (mon.0) 2666 : audit [DBG] from='client.? 172.21.2.111:0/4264578754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:15.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[30791]: audit 2021-03-10T16:14:13.717301+0000 mgr.gibba032.whopeo (mgr.24625) 7472 : audit [DBG] from='client.59041 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[30791]: audit 2021-03-10T16:14:13.942579+0000 mgr.gibba032.whopeo (mgr.24625) 7473 : audit [DBG] from='client.59047 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[30791]: audit 2021-03-10T16:14:14.206955+0000 mon.a (mon.0) 2666 : audit [DBG] from='client.? 172.21.2.111:0/4264578754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:15.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[31774]: audit 2021-03-10T16:14:13.717301+0000 mgr.gibba032.whopeo (mgr.24625) 7472 : audit [DBG] from='client.59041 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[31774]: audit 2021-03-10T16:14:13.942579+0000 mgr.gibba032.whopeo (mgr.24625) 7473 : audit [DBG] from='client.59047 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:15.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:15 gibba011 bash[31774]: audit 2021-03-10T16:14:14.206955+0000 mon.a (mon.0) 2666 : audit [DBG] from='client.? 172.21.2.111:0/4264578754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:16 gibba032 bash[24004]: cluster 2021-03-10T16:14:14.629642+0000 mgr.gibba032.whopeo (mgr.24625) 7474 : cluster [DBG] pgmap v4251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:16 gibba032 bash[24004]: cephadm 2021-03-10T16:14:15.022612+0000 mgr.gibba032.whopeo (mgr.24625) 7475 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:16 gibba011 bash[30791]: cluster 2021-03-10T16:14:14.629642+0000 mgr.gibba032.whopeo (mgr.24625) 7474 : cluster [DBG] pgmap v4251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:16 gibba011 bash[30791]: cephadm 2021-03-10T16:14:15.022612+0000 mgr.gibba032.whopeo (mgr.24625) 7475 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:16 gibba011 bash[31774]: cluster 2021-03-10T16:14:14.629642+0000 mgr.gibba032.whopeo (mgr.24625) 7474 : cluster [DBG] pgmap v4251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:16 gibba011 bash[31774]: cephadm 2021-03-10T16:14:15.022612+0000 mgr.gibba032.whopeo (mgr.24625) 7475 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:18 gibba032 bash[24004]: cluster 2021-03-10T16:14:16.631409+0000 mgr.gibba032.whopeo (mgr.24625) 7476 : cluster [DBG] pgmap v4252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:18 gibba011 bash[30791]: cluster 2021-03-10T16:14:16.631409+0000 mgr.gibba032.whopeo (mgr.24625) 7476 : cluster [DBG] pgmap v4252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:18.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:18 gibba011 bash[31774]: cluster 2021-03-10T16:14:16.631409+0000 mgr.gibba032.whopeo (mgr.24625) 7476 : cluster [DBG] pgmap v4252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:20 gibba032 bash[24004]: cluster 2021-03-10T16:14:18.632265+0000 mgr.gibba032.whopeo (mgr.24625) 7477 : cluster [DBG] pgmap v4253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:20 gibba011 bash[30791]: cluster 2021-03-10T16:14:18.632265+0000 mgr.gibba032.whopeo (mgr.24625) 7477 : cluster [DBG] pgmap v4253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:20 gibba011 bash[31774]: cluster 2021-03-10T16:14:18.632265+0000 mgr.gibba032.whopeo (mgr.24625) 7477 : cluster [DBG] pgmap v4253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:22 gibba032 bash[24004]: cluster 2021-03-10T16:14:20.633798+0000 mgr.gibba032.whopeo (mgr.24625) 7478 : cluster [DBG] pgmap v4254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:22.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:22 gibba011 bash[30791]: cluster 2021-03-10T16:14:20.633798+0000 mgr.gibba032.whopeo (mgr.24625) 7478 : cluster [DBG] pgmap v4254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:22 gibba011 bash[31774]: cluster 2021-03-10T16:14:20.633798+0000 mgr.gibba032.whopeo (mgr.24625) 7478 : cluster [DBG] pgmap v4254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:24 gibba032 bash[24004]: cluster 2021-03-10T16:14:22.635550+0000 mgr.gibba032.whopeo (mgr.24625) 7479 : cluster [DBG] pgmap v4255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:24 gibba011 bash[30791]: cluster 2021-03-10T16:14:22.635550+0000 mgr.gibba032.whopeo (mgr.24625) 7479 : cluster [DBG] pgmap v4255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:24 gibba011 bash[31774]: cluster 2021-03-10T16:14:22.635550+0000 mgr.gibba032.whopeo (mgr.24625) 7479 : cluster [DBG] pgmap v4255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:26 gibba032 bash[24004]: cluster 2021-03-10T16:14:24.636996+0000 mgr.gibba032.whopeo (mgr.24625) 7480 : cluster [DBG] pgmap v4256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:26 gibba011 bash[30791]: cluster 2021-03-10T16:14:24.636996+0000 mgr.gibba032.whopeo (mgr.24625) 7480 : cluster [DBG] pgmap v4256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:26 gibba011 bash[31774]: cluster 2021-03-10T16:14:24.636996+0000 mgr.gibba032.whopeo (mgr.24625) 7480 : cluster [DBG] pgmap v4256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:28 gibba032 bash[24004]: cluster 2021-03-10T16:14:26.638773+0000 mgr.gibba032.whopeo (mgr.24625) 7481 : cluster [DBG] pgmap v4257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:28 gibba011 bash[30791]: cluster 2021-03-10T16:14:26.638773+0000 mgr.gibba032.whopeo (mgr.24625) 7481 : cluster [DBG] pgmap v4257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:28 gibba011 bash[31774]: cluster 2021-03-10T16:14:26.638773+0000 mgr.gibba032.whopeo (mgr.24625) 7481 : cluster [DBG] pgmap v4257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:30 gibba032 bash[24004]: cluster 2021-03-10T16:14:28.640334+0000 mgr.gibba032.whopeo (mgr.24625) 7482 : cluster [DBG] pgmap v4258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:30 gibba011 bash[30791]: cluster 2021-03-10T16:14:28.640334+0000 mgr.gibba032.whopeo (mgr.24625) 7482 : cluster [DBG] pgmap v4258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:30 gibba011 bash[31774]: cluster 2021-03-10T16:14:28.640334+0000 mgr.gibba032.whopeo (mgr.24625) 7482 : cluster [DBG] pgmap v4258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:31.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:31 gibba032 bash[24004]: cephadm 2021-03-10T16:14:30.038239+0000 mgr.gibba032.whopeo (mgr.24625) 7483 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:31.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:31 gibba011 bash[30791]: cephadm 2021-03-10T16:14:30.038239+0000 mgr.gibba032.whopeo (mgr.24625) 7483 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:31.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:31 gibba011 bash[31774]: cephadm 2021-03-10T16:14:30.038239+0000 mgr.gibba032.whopeo (mgr.24625) 7483 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:32 gibba032 bash[24004]: cluster 2021-03-10T16:14:30.641699+0000 mgr.gibba032.whopeo (mgr.24625) 7484 : cluster [DBG] pgmap v4259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:32 gibba011 bash[30791]: cluster 2021-03-10T16:14:30.641699+0000 mgr.gibba032.whopeo (mgr.24625) 7484 : cluster [DBG] pgmap v4259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:32 gibba011 bash[31774]: cluster 2021-03-10T16:14:30.641699+0000 mgr.gibba032.whopeo (mgr.24625) 7484 : cluster [DBG] pgmap v4259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:34 gibba032 bash[24004]: cluster 2021-03-10T16:14:32.643526+0000 mgr.gibba032.whopeo (mgr.24625) 7485 : cluster [DBG] pgmap v4260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:34.634 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:34 gibba011 bash[30791]: cluster 2021-03-10T16:14:32.643526+0000 mgr.gibba032.whopeo (mgr.24625) 7485 : cluster [DBG] pgmap v4260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:34.634 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:34 gibba011 bash[31774]: cluster 2021-03-10T16:14:32.643526+0000 mgr.gibba032.whopeo (mgr.24625) 7485 : cluster [DBG] pgmap v4260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:36.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:36 gibba032 bash[24004]: cluster 2021-03-10T16:14:34.645069+0000 mgr.gibba032.whopeo (mgr.24625) 7486 : cluster [DBG] pgmap v4261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:36 gibba011 bash[30791]: cluster 2021-03-10T16:14:34.645069+0000 mgr.gibba032.whopeo (mgr.24625) 7486 : cluster [DBG] pgmap v4261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:36 gibba011 bash[31774]: cluster 2021-03-10T16:14:34.645069+0000 mgr.gibba032.whopeo (mgr.24625) 7486 : cluster [DBG] pgmap v4261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:38 gibba032 bash[24004]: cluster 2021-03-10T16:14:36.646832+0000 mgr.gibba032.whopeo (mgr.24625) 7487 : cluster [DBG] pgmap v4262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:38 gibba011 bash[30791]: cluster 2021-03-10T16:14:36.646832+0000 mgr.gibba032.whopeo (mgr.24625) 7487 : cluster [DBG] pgmap v4262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:38 gibba011 bash[31774]: cluster 2021-03-10T16:14:36.646832+0000 mgr.gibba032.whopeo (mgr.24625) 7487 : cluster [DBG] pgmap v4262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:40 gibba032 bash[24004]: cluster 2021-03-10T16:14:38.648223+0000 mgr.gibba032.whopeo (mgr.24625) 7488 : cluster [DBG] pgmap v4263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:40 gibba011 bash[30791]: cluster 2021-03-10T16:14:38.648223+0000 mgr.gibba032.whopeo (mgr.24625) 7488 : cluster [DBG] pgmap v4263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:40 gibba011 bash[31774]: cluster 2021-03-10T16:14:38.648223+0000 mgr.gibba032.whopeo (mgr.24625) 7488 : cluster [DBG] pgmap v4263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:42 gibba032 bash[24004]: cluster 2021-03-10T16:14:40.649596+0000 mgr.gibba032.whopeo (mgr.24625) 7489 : cluster [DBG] pgmap v4264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:42 gibba011 bash[30791]: cluster 2021-03-10T16:14:40.649596+0000 mgr.gibba032.whopeo (mgr.24625) 7489 : cluster [DBG] pgmap v4264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:42 gibba011 bash[31774]: cluster 2021-03-10T16:14:40.649596+0000 mgr.gibba032.whopeo (mgr.24625) 7489 : cluster [DBG] pgmap v4264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:44.448 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:14:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:44 gibba032 bash[24004]: cluster 2021-03-10T16:14:42.651299+0000 mgr.gibba032.whopeo (mgr.24625) 7490 : cluster [DBG] pgmap v4265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:44 gibba011 bash[30791]: cluster 2021-03-10T16:14:42.651299+0000 mgr.gibba032.whopeo (mgr.24625) 7490 : cluster [DBG] pgmap v4265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:44.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:44 gibba011 bash[31774]: cluster 2021-03-10T16:14:42.651299+0000 mgr.gibba032.whopeo (mgr.24625) 7490 : cluster [DBG] pgmap v4265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:44.676 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:14:44.676 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:14:44.676 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:14:44.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:14:44.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:14:44.677 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:14:44.677 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:14:44.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:14:44.678 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:14:44.678 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:14:44.678 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:14:44.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:14:44.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:14:44.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:14:44.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:14:44.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:14:44.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:14:44.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:14:44.680 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:14:44.680 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:14:44.681 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:14:44.926 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:14:44.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:14:44.927 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:14:44.927 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:44.927 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:14:44.928 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:14:44.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:44.928 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:14:44.928 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:14:44.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:14:44.929 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:14:44.929 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:14:44.929 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:14:44.929 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:14:44.930 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:14:45.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:45 gibba032 bash[24004]: audit 2021-03-10T16:14:44.927332+0000 mon.c (mon.1) 143 : audit [DBG] from='client.? 172.21.2.111:0/1331452984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:45.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:45 gibba011 bash[30791]: audit 2021-03-10T16:14:44.927332+0000 mon.c (mon.1) 143 : audit [DBG] from='client.? 172.21.2.111:0/1331452984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:45.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:45 gibba011 bash[31774]: audit 2021-03-10T16:14:44.927332+0000 mon.c (mon.1) 143 : audit [DBG] from='client.? 172.21.2.111:0/1331452984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:14:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:46 gibba032 bash[24004]: audit 2021-03-10T16:14:44.437833+0000 mgr.gibba032.whopeo (mgr.24625) 7491 : audit [DBG] from='client.59059 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:46 gibba032 bash[24004]: cluster 2021-03-10T16:14:44.652884+0000 mgr.gibba032.whopeo (mgr.24625) 7492 : cluster [DBG] pgmap v4266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:46 gibba032 bash[24004]: audit 2021-03-10T16:14:44.663295+0000 mgr.gibba032.whopeo (mgr.24625) 7493 : audit [DBG] from='client.49083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:46 gibba032 bash[24004]: cephadm 2021-03-10T16:14:45.054017+0000 mgr.gibba032.whopeo (mgr.24625) 7494 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[30791]: audit 2021-03-10T16:14:44.437833+0000 mgr.gibba032.whopeo (mgr.24625) 7491 : audit [DBG] from='client.59059 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[30791]: cluster 2021-03-10T16:14:44.652884+0000 mgr.gibba032.whopeo (mgr.24625) 7492 : cluster [DBG] pgmap v4266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[30791]: audit 2021-03-10T16:14:44.663295+0000 mgr.gibba032.whopeo (mgr.24625) 7493 : audit [DBG] from='client.49083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[30791]: cephadm 2021-03-10T16:14:45.054017+0000 mgr.gibba032.whopeo (mgr.24625) 7494 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[31774]: audit 2021-03-10T16:14:44.437833+0000 mgr.gibba032.whopeo (mgr.24625) 7491 : audit [DBG] from='client.59059 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[31774]: cluster 2021-03-10T16:14:44.652884+0000 mgr.gibba032.whopeo (mgr.24625) 7492 : cluster [DBG] pgmap v4266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[31774]: audit 2021-03-10T16:14:44.663295+0000 mgr.gibba032.whopeo (mgr.24625) 7493 : audit [DBG] from='client.49083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:14:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:46 gibba011 bash[31774]: cephadm 2021-03-10T16:14:45.054017+0000 mgr.gibba032.whopeo (mgr.24625) 7494 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:14:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:48 gibba011 bash[30791]: cluster 2021-03-10T16:14:46.654715+0000 mgr.gibba032.whopeo (mgr.24625) 7495 : cluster [DBG] pgmap v4267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:48 gibba011 bash[31774]: cluster 2021-03-10T16:14:46.654715+0000 mgr.gibba032.whopeo (mgr.24625) 7495 : cluster [DBG] pgmap v4267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:48 gibba032 bash[24004]: cluster 2021-03-10T16:14:46.654715+0000 mgr.gibba032.whopeo (mgr.24625) 7495 : cluster [DBG] pgmap v4267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:50 gibba011 bash[30791]: cluster 2021-03-10T16:14:48.656396+0000 mgr.gibba032.whopeo (mgr.24625) 7496 : cluster [DBG] pgmap v4268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:50 gibba011 bash[31774]: cluster 2021-03-10T16:14:48.656396+0000 mgr.gibba032.whopeo (mgr.24625) 7496 : cluster [DBG] pgmap v4268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:50.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:50 gibba032 bash[24004]: cluster 2021-03-10T16:14:48.656396+0000 mgr.gibba032.whopeo (mgr.24625) 7496 : cluster [DBG] pgmap v4268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:52 gibba011 bash[30791]: cluster 2021-03-10T16:14:50.657934+0000 mgr.gibba032.whopeo (mgr.24625) 7497 : cluster [DBG] pgmap v4269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:52 gibba011 bash[31774]: cluster 2021-03-10T16:14:50.657934+0000 mgr.gibba032.whopeo (mgr.24625) 7497 : cluster [DBG] pgmap v4269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:52 gibba032 bash[24004]: cluster 2021-03-10T16:14:50.657934+0000 mgr.gibba032.whopeo (mgr.24625) 7497 : cluster [DBG] pgmap v4269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:54 gibba011 bash[30791]: cluster 2021-03-10T16:14:52.659772+0000 mgr.gibba032.whopeo (mgr.24625) 7498 : cluster [DBG] pgmap v4270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:54 gibba011 bash[31774]: cluster 2021-03-10T16:14:52.659772+0000 mgr.gibba032.whopeo (mgr.24625) 7498 : cluster [DBG] pgmap v4270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:54 gibba032 bash[24004]: cluster 2021-03-10T16:14:52.659772+0000 mgr.gibba032.whopeo (mgr.24625) 7498 : cluster [DBG] pgmap v4270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:56 gibba011 bash[30791]: cluster 2021-03-10T16:14:54.661220+0000 mgr.gibba032.whopeo (mgr.24625) 7499 : cluster [DBG] pgmap v4271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:56 gibba011 bash[31774]: cluster 2021-03-10T16:14:54.661220+0000 mgr.gibba032.whopeo (mgr.24625) 7499 : cluster [DBG] pgmap v4271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:56 gibba032 bash[24004]: cluster 2021-03-10T16:14:54.661220+0000 mgr.gibba032.whopeo (mgr.24625) 7499 : cluster [DBG] pgmap v4271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.911922+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:14:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.912570+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:14:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.993725+0000 mon.a (mon.0) 2669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:14:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.994442+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.995101+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[30791]: audit 2021-03-10T16:14:56.995791+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:14:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.911922+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:14:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.912570+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:14:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.993725+0000 mon.a (mon.0) 2669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:14:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.994442+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.995101+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:57 gibba011 bash[31774]: audit 2021-03-10T16:14:56.995791+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:14:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.911922+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:14:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.912570+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:14:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.993725+0000 mon.a (mon.0) 2669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:14:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.994442+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.995101+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:14:57.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:57 gibba032 bash[24004]: audit 2021-03-10T16:14:56.995791+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:14:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:14:58 gibba011 bash[30791]: cluster 2021-03-10T16:14:56.662999+0000 mgr.gibba032.whopeo (mgr.24625) 7500 : cluster [DBG] pgmap v4272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:14:58 gibba011 bash[31774]: cluster 2021-03-10T16:14:56.662999+0000 mgr.gibba032.whopeo (mgr.24625) 7500 : cluster [DBG] pgmap v4272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:14:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:14:58 gibba032 bash[24004]: cluster 2021-03-10T16:14:56.662999+0000 mgr.gibba032.whopeo (mgr.24625) 7500 : cluster [DBG] pgmap v4272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: cluster 2021-03-10T16:14:58.664673+0000 mgr.gibba032.whopeo (mgr.24625) 7501 : cluster [DBG] pgmap v4273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.277736+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:15:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.280800+0000 mon.a (mon.0) 2674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:15:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.282396+0000 mon.a (mon.0) 2675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.283950+0000 mon.a (mon.0) 2676 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.285278+0000 mon.a (mon.0) 2677 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.288045+0000 mon.a (mon.0) 2678 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[30791]: audit 2021-03-10T16:15:00.289665+0000 mon.a (mon.0) 2679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: cluster 2021-03-10T16:14:58.664673+0000 mgr.gibba032.whopeo (mgr.24625) 7501 : cluster [DBG] pgmap v4273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.277736+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:15:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.280800+0000 mon.a (mon.0) 2674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:15:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.282396+0000 mon.a (mon.0) 2675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.283950+0000 mon.a (mon.0) 2676 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.285278+0000 mon.a (mon.0) 2677 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.288045+0000 mon.a (mon.0) 2678 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:00 gibba011 bash[31774]: audit 2021-03-10T16:15:00.289665+0000 mon.a (mon.0) 2679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: cluster 2021-03-10T16:14:58.664673+0000 mgr.gibba032.whopeo (mgr.24625) 7501 : cluster [DBG] pgmap v4273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.277736+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:15:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.280800+0000 mon.a (mon.0) 2674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:15:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.282396+0000 mon.a (mon.0) 2675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.283950+0000 mon.a (mon.0) 2676 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.285278+0000 mon.a (mon.0) 2677 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.288045+0000 mon.a (mon.0) 2678 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:00 gibba032 bash[24004]: audit 2021-03-10T16:15:00.289665+0000 mon.a (mon.0) 2679 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.069809+0000 mgr.gibba032.whopeo (mgr.24625) 7502 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:15:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.070267+0000 mgr.gibba032.whopeo (mgr.24625) 7503 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:15:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.277925+0000 mgr.gibba032.whopeo (mgr.24625) 7504 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:15:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.279828+0000 mgr.gibba032.whopeo (mgr.24625) 7505 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:15:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.281293+0000 mgr.gibba032.whopeo (mgr.24625) 7506 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:15:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.281330+0000 mgr.gibba032.whopeo (mgr.24625) 7507 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:15:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.282762+0000 mgr.gibba032.whopeo (mgr.24625) 7508 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:15:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.282800+0000 mgr.gibba032.whopeo (mgr.24625) 7509 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:15:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.284102+0000 mgr.gibba032.whopeo (mgr.24625) 7510 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:15:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.285395+0000 mgr.gibba032.whopeo (mgr.24625) 7511 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:15:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.285432+0000 mgr.gibba032.whopeo (mgr.24625) 7512 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:15:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.286840+0000 mgr.gibba032.whopeo (mgr.24625) 7513 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:15:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.287148+0000 mgr.gibba032.whopeo (mgr.24625) 7514 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:15:01.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.288725+0000 mgr.gibba032.whopeo (mgr.24625) 7515 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:15:01.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.288920+0000 mgr.gibba032.whopeo (mgr.24625) 7516 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:15:01.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:01 gibba032 bash[24004]: cephadm 2021-03-10T16:15:00.289027+0000 mgr.gibba032.whopeo (mgr.24625) 7517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.069809+0000 mgr.gibba032.whopeo (mgr.24625) 7502 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:15:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.070267+0000 mgr.gibba032.whopeo (mgr.24625) 7503 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:15:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.277925+0000 mgr.gibba032.whopeo (mgr.24625) 7504 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:15:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.279828+0000 mgr.gibba032.whopeo (mgr.24625) 7505 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:15:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.281293+0000 mgr.gibba032.whopeo (mgr.24625) 7506 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:15:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.281330+0000 mgr.gibba032.whopeo (mgr.24625) 7507 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:15:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.282762+0000 mgr.gibba032.whopeo (mgr.24625) 7508 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:15:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.282800+0000 mgr.gibba032.whopeo (mgr.24625) 7509 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:15:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.284102+0000 mgr.gibba032.whopeo (mgr.24625) 7510 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:15:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.285395+0000 mgr.gibba032.whopeo (mgr.24625) 7511 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:15:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.285432+0000 mgr.gibba032.whopeo (mgr.24625) 7512 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:15:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.286840+0000 mgr.gibba032.whopeo (mgr.24625) 7513 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:15:01.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.287148+0000 mgr.gibba032.whopeo (mgr.24625) 7514 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:15:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.288725+0000 mgr.gibba032.whopeo (mgr.24625) 7515 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:15:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.288920+0000 mgr.gibba032.whopeo (mgr.24625) 7516 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:15:01.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[30791]: cephadm 2021-03-10T16:15:00.289027+0000 mgr.gibba032.whopeo (mgr.24625) 7517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.069809+0000 mgr.gibba032.whopeo (mgr.24625) 7502 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:15:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.070267+0000 mgr.gibba032.whopeo (mgr.24625) 7503 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:15:01.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.277925+0000 mgr.gibba032.whopeo (mgr.24625) 7504 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:15:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.279828+0000 mgr.gibba032.whopeo (mgr.24625) 7505 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:15:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.281293+0000 mgr.gibba032.whopeo (mgr.24625) 7506 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:15:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.281330+0000 mgr.gibba032.whopeo (mgr.24625) 7507 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:15:01.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.282762+0000 mgr.gibba032.whopeo (mgr.24625) 7508 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:15:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.282800+0000 mgr.gibba032.whopeo (mgr.24625) 7509 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:15:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.284102+0000 mgr.gibba032.whopeo (mgr.24625) 7510 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:15:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.285395+0000 mgr.gibba032.whopeo (mgr.24625) 7511 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:15:01.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.285432+0000 mgr.gibba032.whopeo (mgr.24625) 7512 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:15:01.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.286840+0000 mgr.gibba032.whopeo (mgr.24625) 7513 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:15:01.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.287148+0000 mgr.gibba032.whopeo (mgr.24625) 7514 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:15:01.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.288725+0000 mgr.gibba032.whopeo (mgr.24625) 7515 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:15:01.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.288920+0000 mgr.gibba032.whopeo (mgr.24625) 7516 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:15:01.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:01 gibba011 bash[31774]: cephadm 2021-03-10T16:15:00.289027+0000 mgr.gibba032.whopeo (mgr.24625) 7517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:02 gibba032 bash[24004]: cluster 2021-03-10T16:15:00.665161+0000 mgr.gibba032.whopeo (mgr.24625) 7518 : cluster [DBG] pgmap v4274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:02 gibba011 bash[30791]: cluster 2021-03-10T16:15:00.665161+0000 mgr.gibba032.whopeo (mgr.24625) 7518 : cluster [DBG] pgmap v4274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:02 gibba011 bash[31774]: cluster 2021-03-10T16:15:00.665161+0000 mgr.gibba032.whopeo (mgr.24625) 7518 : cluster [DBG] pgmap v4274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:04 gibba032 bash[24004]: cluster 2021-03-10T16:15:02.666788+0000 mgr.gibba032.whopeo (mgr.24625) 7519 : cluster [DBG] pgmap v4275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:04 gibba011 bash[30791]: cluster 2021-03-10T16:15:02.666788+0000 mgr.gibba032.whopeo (mgr.24625) 7519 : cluster [DBG] pgmap v4275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:04.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:04 gibba011 bash[31774]: cluster 2021-03-10T16:15:02.666788+0000 mgr.gibba032.whopeo (mgr.24625) 7519 : cluster [DBG] pgmap v4275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:06 gibba032 bash[24004]: cluster 2021-03-10T16:15:04.668164+0000 mgr.gibba032.whopeo (mgr.24625) 7520 : cluster [DBG] pgmap v4276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:06 gibba011 bash[30791]: cluster 2021-03-10T16:15:04.668164+0000 mgr.gibba032.whopeo (mgr.24625) 7520 : cluster [DBG] pgmap v4276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:06 gibba011 bash[31774]: cluster 2021-03-10T16:15:04.668164+0000 mgr.gibba032.whopeo (mgr.24625) 7520 : cluster [DBG] pgmap v4276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:08 gibba032 bash[24004]: cluster 2021-03-10T16:15:06.669955+0000 mgr.gibba032.whopeo (mgr.24625) 7521 : cluster [DBG] pgmap v4277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:08 gibba011 bash[30791]: cluster 2021-03-10T16:15:06.669955+0000 mgr.gibba032.whopeo (mgr.24625) 7521 : cluster [DBG] pgmap v4277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:08 gibba011 bash[31774]: cluster 2021-03-10T16:15:06.669955+0000 mgr.gibba032.whopeo (mgr.24625) 7521 : cluster [DBG] pgmap v4277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:10 gibba032 bash[24004]: cluster 2021-03-10T16:15:08.671572+0000 mgr.gibba032.whopeo (mgr.24625) 7522 : cluster [DBG] pgmap v4278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:10 gibba011 bash[30791]: cluster 2021-03-10T16:15:08.671572+0000 mgr.gibba032.whopeo (mgr.24625) 7522 : cluster [DBG] pgmap v4278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:10 gibba011 bash[31774]: cluster 2021-03-10T16:15:08.671572+0000 mgr.gibba032.whopeo (mgr.24625) 7522 : cluster [DBG] pgmap v4278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:12 gibba032 bash[24004]: cluster 2021-03-10T16:15:10.672084+0000 mgr.gibba032.whopeo (mgr.24625) 7523 : cluster [DBG] pgmap v4279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:12 gibba011 bash[30791]: cluster 2021-03-10T16:15:10.672084+0000 mgr.gibba032.whopeo (mgr.24625) 7523 : cluster [DBG] pgmap v4279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:12 gibba011 bash[31774]: cluster 2021-03-10T16:15:10.672084+0000 mgr.gibba032.whopeo (mgr.24625) 7523 : cluster [DBG] pgmap v4279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:14 gibba032 bash[24004]: cluster 2021-03-10T16:15:12.673418+0000 mgr.gibba032.whopeo (mgr.24625) 7524 : cluster [DBG] pgmap v4280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:14 gibba011 bash[30791]: cluster 2021-03-10T16:15:12.673418+0000 mgr.gibba032.whopeo (mgr.24625) 7524 : cluster [DBG] pgmap v4280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:14 gibba011 bash[31774]: cluster 2021-03-10T16:15:12.673418+0000 mgr.gibba032.whopeo (mgr.24625) 7524 : cluster [DBG] pgmap v4280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:15.172 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:15:15.399 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:15:15.400 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:15:15.400 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:15:15.400 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:15:15.401 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:15:15.401 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:15:15.401 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:15:15.401 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:15:15.402 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:15:15.402 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:15:15.402 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:15:15.402 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:15:15.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:15:15.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:15:15.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:15:15.403 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:15:15.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:15:15.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:15:15.404 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:15:15.404 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:15:15.405 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:15:15.647 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:15:15.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:15:15.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:15:15.648 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:15.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:15:15.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:15:15.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:15.649 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:15:15.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:15:15.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:15.650 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:15:15.650 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:15:15.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:15:15.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:15:15.651 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:15:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:16 gibba032 bash[24004]: cluster 2021-03-10T16:15:14.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7525 : cluster [DBG] pgmap v4281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:16 gibba032 bash[24004]: audit 2021-03-10T16:15:15.161706+0000 mgr.gibba032.whopeo (mgr.24625) 7526 : audit [DBG] from='client.49095 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:16 gibba032 bash[24004]: cephadm 2021-03-10T16:15:15.304886+0000 mgr.gibba032.whopeo (mgr.24625) 7527 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:16 gibba032 bash[24004]: audit 2021-03-10T16:15:15.386639+0000 mgr.gibba032.whopeo (mgr.24625) 7528 : audit [DBG] from='client.59083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:16 gibba032 bash[24004]: audit 2021-03-10T16:15:15.648939+0000 mon.c (mon.1) 144 : audit [DBG] from='client.? 172.21.2.111:0/2449341662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[30791]: cluster 2021-03-10T16:15:14.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7525 : cluster [DBG] pgmap v4281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[30791]: audit 2021-03-10T16:15:15.161706+0000 mgr.gibba032.whopeo (mgr.24625) 7526 : audit [DBG] from='client.49095 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[30791]: cephadm 2021-03-10T16:15:15.304886+0000 mgr.gibba032.whopeo (mgr.24625) 7527 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[30791]: audit 2021-03-10T16:15:15.386639+0000 mgr.gibba032.whopeo (mgr.24625) 7528 : audit [DBG] from='client.59083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[30791]: audit 2021-03-10T16:15:15.648939+0000 mon.c (mon.1) 144 : audit [DBG] from='client.? 172.21.2.111:0/2449341662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[31774]: cluster 2021-03-10T16:15:14.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7525 : cluster [DBG] pgmap v4281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[31774]: audit 2021-03-10T16:15:15.161706+0000 mgr.gibba032.whopeo (mgr.24625) 7526 : audit [DBG] from='client.49095 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[31774]: cephadm 2021-03-10T16:15:15.304886+0000 mgr.gibba032.whopeo (mgr.24625) 7527 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[31774]: audit 2021-03-10T16:15:15.386639+0000 mgr.gibba032.whopeo (mgr.24625) 7528 : audit [DBG] from='client.59083 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:16 gibba011 bash[31774]: audit 2021-03-10T16:15:15.648939+0000 mon.c (mon.1) 144 : audit [DBG] from='client.? 172.21.2.111:0/2449341662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:18 gibba032 bash[24004]: cluster 2021-03-10T16:15:16.676471+0000 mgr.gibba032.whopeo (mgr.24625) 7529 : cluster [DBG] pgmap v4282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:18 gibba011 bash[30791]: cluster 2021-03-10T16:15:16.676471+0000 mgr.gibba032.whopeo (mgr.24625) 7529 : cluster [DBG] pgmap v4282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:18 gibba011 bash[31774]: cluster 2021-03-10T16:15:16.676471+0000 mgr.gibba032.whopeo (mgr.24625) 7529 : cluster [DBG] pgmap v4282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:20 gibba032 bash[24004]: cluster 2021-03-10T16:15:18.677793+0000 mgr.gibba032.whopeo (mgr.24625) 7530 : cluster [DBG] pgmap v4283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:20 gibba011 bash[30791]: cluster 2021-03-10T16:15:18.677793+0000 mgr.gibba032.whopeo (mgr.24625) 7530 : cluster [DBG] pgmap v4283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:20.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:20 gibba011 bash[31774]: cluster 2021-03-10T16:15:18.677793+0000 mgr.gibba032.whopeo (mgr.24625) 7530 : cluster [DBG] pgmap v4283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:22 gibba032 bash[24004]: cluster 2021-03-10T16:15:20.679097+0000 mgr.gibba032.whopeo (mgr.24625) 7531 : cluster [DBG] pgmap v4284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:22 gibba011 bash[30791]: cluster 2021-03-10T16:15:20.679097+0000 mgr.gibba032.whopeo (mgr.24625) 7531 : cluster [DBG] pgmap v4284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:22 gibba011 bash[31774]: cluster 2021-03-10T16:15:20.679097+0000 mgr.gibba032.whopeo (mgr.24625) 7531 : cluster [DBG] pgmap v4284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:24 gibba032 bash[24004]: cluster 2021-03-10T16:15:22.680922+0000 mgr.gibba032.whopeo (mgr.24625) 7532 : cluster [DBG] pgmap v4285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:24 gibba011 bash[30791]: cluster 2021-03-10T16:15:22.680922+0000 mgr.gibba032.whopeo (mgr.24625) 7532 : cluster [DBG] pgmap v4285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:24 gibba011 bash[31774]: cluster 2021-03-10T16:15:22.680922+0000 mgr.gibba032.whopeo (mgr.24625) 7532 : cluster [DBG] pgmap v4285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:26 gibba032 bash[24004]: cluster 2021-03-10T16:15:24.682461+0000 mgr.gibba032.whopeo (mgr.24625) 7533 : cluster [DBG] pgmap v4286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:26 gibba011 bash[30791]: cluster 2021-03-10T16:15:24.682461+0000 mgr.gibba032.whopeo (mgr.24625) 7533 : cluster [DBG] pgmap v4286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:26 gibba011 bash[31774]: cluster 2021-03-10T16:15:24.682461+0000 mgr.gibba032.whopeo (mgr.24625) 7533 : cluster [DBG] pgmap v4286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:27.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:27 gibba032 bash[24004]: cluster 2021-03-10T16:15:26.684315+0000 mgr.gibba032.whopeo (mgr.24625) 7534 : cluster [DBG] pgmap v4287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:27.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:27 gibba011 bash[30791]: cluster 2021-03-10T16:15:26.684315+0000 mgr.gibba032.whopeo (mgr.24625) 7534 : cluster [DBG] pgmap v4287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:27.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:27 gibba011 bash[31774]: cluster 2021-03-10T16:15:26.684315+0000 mgr.gibba032.whopeo (mgr.24625) 7534 : cluster [DBG] pgmap v4287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:30 gibba032 bash[24004]: cluster 2021-03-10T16:15:28.685949+0000 mgr.gibba032.whopeo (mgr.24625) 7535 : cluster [DBG] pgmap v4288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:30 gibba011 bash[30791]: cluster 2021-03-10T16:15:28.685949+0000 mgr.gibba032.whopeo (mgr.24625) 7535 : cluster [DBG] pgmap v4288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:30 gibba011 bash[31774]: cluster 2021-03-10T16:15:28.685949+0000 mgr.gibba032.whopeo (mgr.24625) 7535 : cluster [DBG] pgmap v4288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:31.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:31 gibba032 bash[24004]: cephadm 2021-03-10T16:15:30.320975+0000 mgr.gibba032.whopeo (mgr.24625) 7536 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:31.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:31 gibba011 bash[30791]: cephadm 2021-03-10T16:15:30.320975+0000 mgr.gibba032.whopeo (mgr.24625) 7536 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:31.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:31 gibba011 bash[31774]: cephadm 2021-03-10T16:15:30.320975+0000 mgr.gibba032.whopeo (mgr.24625) 7536 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:32 gibba032 bash[24004]: cluster 2021-03-10T16:15:30.687288+0000 mgr.gibba032.whopeo (mgr.24625) 7537 : cluster [DBG] pgmap v4289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:32 gibba011 bash[30791]: cluster 2021-03-10T16:15:30.687288+0000 mgr.gibba032.whopeo (mgr.24625) 7537 : cluster [DBG] pgmap v4289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:32 gibba011 bash[31774]: cluster 2021-03-10T16:15:30.687288+0000 mgr.gibba032.whopeo (mgr.24625) 7537 : cluster [DBG] pgmap v4289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:34 gibba032 bash[24004]: cluster 2021-03-10T16:15:32.689109+0000 mgr.gibba032.whopeo (mgr.24625) 7538 : cluster [DBG] pgmap v4290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:34.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:34 gibba011 bash[30791]: cluster 2021-03-10T16:15:32.689109+0000 mgr.gibba032.whopeo (mgr.24625) 7538 : cluster [DBG] pgmap v4290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:34 gibba011 bash[31774]: cluster 2021-03-10T16:15:32.689109+0000 mgr.gibba032.whopeo (mgr.24625) 7538 : cluster [DBG] pgmap v4290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:36.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:36 gibba032 bash[24004]: cluster 2021-03-10T16:15:34.690594+0000 mgr.gibba032.whopeo (mgr.24625) 7539 : cluster [DBG] pgmap v4291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:36 gibba011 bash[30791]: cluster 2021-03-10T16:15:34.690594+0000 mgr.gibba032.whopeo (mgr.24625) 7539 : cluster [DBG] pgmap v4291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:36 gibba011 bash[31774]: cluster 2021-03-10T16:15:34.690594+0000 mgr.gibba032.whopeo (mgr.24625) 7539 : cluster [DBG] pgmap v4291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:38 gibba032 bash[24004]: cluster 2021-03-10T16:15:36.692420+0000 mgr.gibba032.whopeo (mgr.24625) 7540 : cluster [DBG] pgmap v4292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:38 gibba011 bash[30791]: cluster 2021-03-10T16:15:36.692420+0000 mgr.gibba032.whopeo (mgr.24625) 7540 : cluster [DBG] pgmap v4292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:38 gibba011 bash[31774]: cluster 2021-03-10T16:15:36.692420+0000 mgr.gibba032.whopeo (mgr.24625) 7540 : cluster [DBG] pgmap v4292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:40 gibba032 bash[24004]: cluster 2021-03-10T16:15:38.694034+0000 mgr.gibba032.whopeo (mgr.24625) 7541 : cluster [DBG] pgmap v4293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:40 gibba011 bash[30791]: cluster 2021-03-10T16:15:38.694034+0000 mgr.gibba032.whopeo (mgr.24625) 7541 : cluster [DBG] pgmap v4293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:40 gibba011 bash[31774]: cluster 2021-03-10T16:15:38.694034+0000 mgr.gibba032.whopeo (mgr.24625) 7541 : cluster [DBG] pgmap v4293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:42 gibba032 bash[24004]: cluster 2021-03-10T16:15:40.694642+0000 mgr.gibba032.whopeo (mgr.24625) 7542 : cluster [DBG] pgmap v4294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:42 gibba011 bash[30791]: cluster 2021-03-10T16:15:40.694642+0000 mgr.gibba032.whopeo (mgr.24625) 7542 : cluster [DBG] pgmap v4294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:42 gibba011 bash[31774]: cluster 2021-03-10T16:15:40.694642+0000 mgr.gibba032.whopeo (mgr.24625) 7542 : cluster [DBG] pgmap v4294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:44.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:44 gibba032 bash[24004]: cluster 2021-03-10T16:15:42.696356+0000 mgr.gibba032.whopeo (mgr.24625) 7543 : cluster [DBG] pgmap v4295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:44 gibba011 bash[30791]: cluster 2021-03-10T16:15:42.696356+0000 mgr.gibba032.whopeo (mgr.24625) 7543 : cluster [DBG] pgmap v4295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:44 gibba011 bash[31774]: cluster 2021-03-10T16:15:42.696356+0000 mgr.gibba032.whopeo (mgr.24625) 7543 : cluster [DBG] pgmap v4295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:45.893 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:15:46.117 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:15:46.118 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:15:46.118 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:15:46.118 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:15:46.119 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:15:46.119 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:15:46.119 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:15:46.119 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:15:46.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:15:46.120 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:15:46.120 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:15:46.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:15:46.120 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:15:46.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:15:46.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:15:46.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:15:46.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:15:46.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:15:46.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:15:46.122 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:15:46.122 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:15:46.369 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:15:46.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:15:46.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:15:46.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:46.370 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:15:46.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:15:46.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:46.371 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:15:46.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:15:46.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:15:46.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:15:46.372 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:15:46.372 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:15:46.372 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:15:46.373 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:15:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:46 gibba032 bash[24004]: cluster 2021-03-10T16:15:44.697664+0000 mgr.gibba032.whopeo (mgr.24625) 7544 : cluster [DBG] pgmap v4296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:46 gibba032 bash[24004]: cephadm 2021-03-10T16:15:45.336759+0000 mgr.gibba032.whopeo (mgr.24625) 7545 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:46 gibba011 bash[30791]: cluster 2021-03-10T16:15:44.697664+0000 mgr.gibba032.whopeo (mgr.24625) 7544 : cluster [DBG] pgmap v4296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:46 gibba011 bash[30791]: cephadm 2021-03-10T16:15:45.336759+0000 mgr.gibba032.whopeo (mgr.24625) 7545 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:46 gibba011 bash[31774]: cluster 2021-03-10T16:15:44.697664+0000 mgr.gibba032.whopeo (mgr.24625) 7544 : cluster [DBG] pgmap v4296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:46 gibba011 bash[31774]: cephadm 2021-03-10T16:15:45.336759+0000 mgr.gibba032.whopeo (mgr.24625) 7545 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:15:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:47 gibba032 bash[24004]: audit 2021-03-10T16:15:45.883831+0000 mgr.gibba032.whopeo (mgr.24625) 7546 : audit [DBG] from='client.49113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:47 gibba032 bash[24004]: audit 2021-03-10T16:15:46.104821+0000 mgr.gibba032.whopeo (mgr.24625) 7547 : audit [DBG] from='client.59101 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:47 gibba032 bash[24004]: audit 2021-03-10T16:15:46.370647+0000 mon.a (mon.0) 2680 : audit [DBG] from='client.? 172.21.2.111:0/2138841117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:47.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[30791]: audit 2021-03-10T16:15:45.883831+0000 mgr.gibba032.whopeo (mgr.24625) 7546 : audit [DBG] from='client.49113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[30791]: audit 2021-03-10T16:15:46.104821+0000 mgr.gibba032.whopeo (mgr.24625) 7547 : audit [DBG] from='client.59101 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[30791]: audit 2021-03-10T16:15:46.370647+0000 mon.a (mon.0) 2680 : audit [DBG] from='client.? 172.21.2.111:0/2138841117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:47.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[31774]: audit 2021-03-10T16:15:45.883831+0000 mgr.gibba032.whopeo (mgr.24625) 7546 : audit [DBG] from='client.49113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[31774]: audit 2021-03-10T16:15:46.104821+0000 mgr.gibba032.whopeo (mgr.24625) 7547 : audit [DBG] from='client.59101 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:15:47.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:47 gibba011 bash[31774]: audit 2021-03-10T16:15:46.370647+0000 mon.a (mon.0) 2680 : audit [DBG] from='client.? 172.21.2.111:0/2138841117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:15:48.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:48 gibba032 bash[24004]: cluster 2021-03-10T16:15:46.699442+0000 mgr.gibba032.whopeo (mgr.24625) 7548 : cluster [DBG] pgmap v4297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:48 gibba011 bash[30791]: cluster 2021-03-10T16:15:46.699442+0000 mgr.gibba032.whopeo (mgr.24625) 7548 : cluster [DBG] pgmap v4297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:48 gibba011 bash[31774]: cluster 2021-03-10T16:15:46.699442+0000 mgr.gibba032.whopeo (mgr.24625) 7548 : cluster [DBG] pgmap v4297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:50 gibba032 bash[24004]: cluster 2021-03-10T16:15:48.701078+0000 mgr.gibba032.whopeo (mgr.24625) 7549 : cluster [DBG] pgmap v4298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:50 gibba011 bash[30791]: cluster 2021-03-10T16:15:48.701078+0000 mgr.gibba032.whopeo (mgr.24625) 7549 : cluster [DBG] pgmap v4298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:50 gibba011 bash[31774]: cluster 2021-03-10T16:15:48.701078+0000 mgr.gibba032.whopeo (mgr.24625) 7549 : cluster [DBG] pgmap v4298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:52 gibba032 bash[24004]: cluster 2021-03-10T16:15:50.701802+0000 mgr.gibba032.whopeo (mgr.24625) 7550 : cluster [DBG] pgmap v4299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:52 gibba011 bash[30791]: cluster 2021-03-10T16:15:50.701802+0000 mgr.gibba032.whopeo (mgr.24625) 7550 : cluster [DBG] pgmap v4299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:52 gibba011 bash[31774]: cluster 2021-03-10T16:15:50.701802+0000 mgr.gibba032.whopeo (mgr.24625) 7550 : cluster [DBG] pgmap v4299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:54 gibba032 bash[24004]: cluster 2021-03-10T16:15:52.703453+0000 mgr.gibba032.whopeo (mgr.24625) 7551 : cluster [DBG] pgmap v4300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:54 gibba011 bash[30791]: cluster 2021-03-10T16:15:52.703453+0000 mgr.gibba032.whopeo (mgr.24625) 7551 : cluster [DBG] pgmap v4300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:54.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:54 gibba011 bash[31774]: cluster 2021-03-10T16:15:52.703453+0000 mgr.gibba032.whopeo (mgr.24625) 7551 : cluster [DBG] pgmap v4300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:56 gibba032 bash[24004]: cluster 2021-03-10T16:15:54.704979+0000 mgr.gibba032.whopeo (mgr.24625) 7552 : cluster [DBG] pgmap v4301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:56 gibba011 bash[30791]: cluster 2021-03-10T16:15:54.704979+0000 mgr.gibba032.whopeo (mgr.24625) 7552 : cluster [DBG] pgmap v4301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:56 gibba011 bash[31774]: cluster 2021-03-10T16:15:54.704979+0000 mgr.gibba032.whopeo (mgr.24625) 7552 : cluster [DBG] pgmap v4301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:56.918065+0000 mon.a (mon.0) 2681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:15:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:56.918693+0000 mon.a (mon.0) 2682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:15:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:57.004478+0000 mon.a (mon.0) 2683 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T16:15:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:57.005236+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:57.005929+0000 mon.a (mon.0) 2685 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:57 gibba032 bash[24004]: audit 2021-03-10T16:15:57.006657+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:15:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:56.918065+0000 mon.a (mon.0) 2681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:15:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:56.918693+0000 mon.a (mon.0) 2682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:15:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:57.004478+0000 mon.a (mon.0) 2683 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T16:15:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:57.005236+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:57.005929+0000 mon.a (mon.0) 2685 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[30791]: audit 2021-03-10T16:15:57.006657+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:15:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:56.918065+0000 mon.a (mon.0) 2681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:15:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:56.918693+0000 mon.a (mon.0) 2682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:15:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:57.004478+0000 mon.a (mon.0) 2683 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T16:15:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:57.005236+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:57.005929+0000 mon.a (mon.0) 2685 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:15:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:57 gibba011 bash[31774]: audit 2021-03-10T16:15:57.006657+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:15:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:15:58 gibba032 bash[24004]: cluster 2021-03-10T16:15:56.706477+0000 mgr.gibba032.whopeo (mgr.24625) 7553 : cluster [DBG] pgmap v4302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:15:58 gibba011 bash[31774]: cluster 2021-03-10T16:15:56.706477+0000 mgr.gibba032.whopeo (mgr.24625) 7553 : cluster [DBG] pgmap v4302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:15:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:15:58 gibba011 bash[30791]: cluster 2021-03-10T16:15:56.706477+0000 mgr.gibba032.whopeo (mgr.24625) 7553 : cluster [DBG] pgmap v4302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:00 gibba011 bash[30791]: cluster 2021-03-10T16:15:58.708189+0000 mgr.gibba032.whopeo (mgr.24625) 7554 : cluster [DBG] pgmap v4303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:00 gibba011 bash[31774]: cluster 2021-03-10T16:15:58.708189+0000 mgr.gibba032.whopeo (mgr.24625) 7554 : cluster [DBG] pgmap v4303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:00.601 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:00 gibba032 bash[24004]: cluster 2021-03-10T16:15:58.708189+0000 mgr.gibba032.whopeo (mgr.24625) 7554 : cluster [DBG] pgmap v4303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.353569+0000 mgr.gibba032.whopeo (mgr.24625) 7555 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:16:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.355124+0000 mgr.gibba032.whopeo (mgr.24625) 7556 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:16:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.590032+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:16:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.593601+0000 mon.a (mon.0) 2688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:16:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.595233+0000 mon.a (mon.0) 2689 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.596774+0000 mon.a (mon.0) 2690 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.598110+0000 mon.a (mon.0) 2691 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.600958+0000 mon.a (mon.0) 2692 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[30791]: audit 2021-03-10T16:16:00.602631+0000 mon.a (mon.0) 2693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.353569+0000 mgr.gibba032.whopeo (mgr.24625) 7555 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:16:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.355124+0000 mgr.gibba032.whopeo (mgr.24625) 7556 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:16:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.590032+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:16:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.593601+0000 mon.a (mon.0) 2688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:16:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.595233+0000 mon.a (mon.0) 2689 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.596774+0000 mon.a (mon.0) 2690 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.598110+0000 mon.a (mon.0) 2691 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.600958+0000 mon.a (mon.0) 2692 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:01 gibba011 bash[31774]: audit 2021-03-10T16:16:00.602631+0000 mon.a (mon.0) 2693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.353569+0000 mgr.gibba032.whopeo (mgr.24625) 7555 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:16:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.355124+0000 mgr.gibba032.whopeo (mgr.24625) 7556 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:16:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.590032+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:16:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.593601+0000 mon.a (mon.0) 2688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:16:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.595233+0000 mon.a (mon.0) 2689 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.596774+0000 mon.a (mon.0) 2690 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.598110+0000 mon.a (mon.0) 2691 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.600958+0000 mon.a (mon.0) 2692 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:01 gibba032 bash[24004]: audit 2021-03-10T16:16:00.602631+0000 mon.a (mon.0) 2693 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.590762+0000 mgr.gibba032.whopeo (mgr.24625) 7557 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:16:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.592633+0000 mgr.gibba032.whopeo (mgr.24625) 7558 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:16:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.594131+0000 mgr.gibba032.whopeo (mgr.24625) 7559 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:16:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.594170+0000 mgr.gibba032.whopeo (mgr.24625) 7560 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:16:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.595598+0000 mgr.gibba032.whopeo (mgr.24625) 7561 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:16:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.595636+0000 mgr.gibba032.whopeo (mgr.24625) 7562 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:16:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.596937+0000 mgr.gibba032.whopeo (mgr.24625) 7563 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:16:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.598260+0000 mgr.gibba032.whopeo (mgr.24625) 7564 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:16:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.598297+0000 mgr.gibba032.whopeo (mgr.24625) 7565 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:16:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.599725+0000 mgr.gibba032.whopeo (mgr.24625) 7566 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:16:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.600084+0000 mgr.gibba032.whopeo (mgr.24625) 7567 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:16:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.601403+0000 mgr.gibba032.whopeo (mgr.24625) 7568 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:16:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.601441+0000 mgr.gibba032.whopeo (mgr.24625) 7569 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:16:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cephadm 2021-03-10T16:16:00.601545+0000 mgr.gibba032.whopeo (mgr.24625) 7570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[30791]: cluster 2021-03-10T16:16:00.709551+0000 mgr.gibba032.whopeo (mgr.24625) 7571 : cluster [DBG] pgmap v4304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.590762+0000 mgr.gibba032.whopeo (mgr.24625) 7557 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:16:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.592633+0000 mgr.gibba032.whopeo (mgr.24625) 7558 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:16:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.594131+0000 mgr.gibba032.whopeo (mgr.24625) 7559 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:16:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.594170+0000 mgr.gibba032.whopeo (mgr.24625) 7560 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:16:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.595598+0000 mgr.gibba032.whopeo (mgr.24625) 7561 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:16:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.595636+0000 mgr.gibba032.whopeo (mgr.24625) 7562 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:16:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.596937+0000 mgr.gibba032.whopeo (mgr.24625) 7563 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:16:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.598260+0000 mgr.gibba032.whopeo (mgr.24625) 7564 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:16:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.598297+0000 mgr.gibba032.whopeo (mgr.24625) 7565 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:16:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.599725+0000 mgr.gibba032.whopeo (mgr.24625) 7566 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:16:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.600084+0000 mgr.gibba032.whopeo (mgr.24625) 7567 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:16:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.601403+0000 mgr.gibba032.whopeo (mgr.24625) 7568 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:16:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.601441+0000 mgr.gibba032.whopeo (mgr.24625) 7569 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:16:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cephadm 2021-03-10T16:16:00.601545+0000 mgr.gibba032.whopeo (mgr.24625) 7570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:02 gibba011 bash[31774]: cluster 2021-03-10T16:16:00.709551+0000 mgr.gibba032.whopeo (mgr.24625) 7571 : cluster [DBG] pgmap v4304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.590762+0000 mgr.gibba032.whopeo (mgr.24625) 7557 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:16:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.592633+0000 mgr.gibba032.whopeo (mgr.24625) 7558 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:16:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.594131+0000 mgr.gibba032.whopeo (mgr.24625) 7559 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:16:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.594170+0000 mgr.gibba032.whopeo (mgr.24625) 7560 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:16:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.595598+0000 mgr.gibba032.whopeo (mgr.24625) 7561 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:16:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.595636+0000 mgr.gibba032.whopeo (mgr.24625) 7562 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:16:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.596937+0000 mgr.gibba032.whopeo (mgr.24625) 7563 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:16:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.598260+0000 mgr.gibba032.whopeo (mgr.24625) 7564 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:16:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.598297+0000 mgr.gibba032.whopeo (mgr.24625) 7565 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:16:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.599725+0000 mgr.gibba032.whopeo (mgr.24625) 7566 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:16:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.600084+0000 mgr.gibba032.whopeo (mgr.24625) 7567 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:16:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.601403+0000 mgr.gibba032.whopeo (mgr.24625) 7568 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:16:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.601441+0000 mgr.gibba032.whopeo (mgr.24625) 7569 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:16:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cephadm 2021-03-10T16:16:00.601545+0000 mgr.gibba032.whopeo (mgr.24625) 7570 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:02 gibba032 bash[24004]: cluster 2021-03-10T16:16:00.709551+0000 mgr.gibba032.whopeo (mgr.24625) 7571 : cluster [DBG] pgmap v4304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:04 gibba011 bash[30791]: cluster 2021-03-10T16:16:02.711184+0000 mgr.gibba032.whopeo (mgr.24625) 7572 : cluster [DBG] pgmap v4305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:04 gibba011 bash[31774]: cluster 2021-03-10T16:16:02.711184+0000 mgr.gibba032.whopeo (mgr.24625) 7572 : cluster [DBG] pgmap v4305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:04 gibba032 bash[24004]: cluster 2021-03-10T16:16:02.711184+0000 mgr.gibba032.whopeo (mgr.24625) 7572 : cluster [DBG] pgmap v4305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:06 gibba011 bash[30791]: cluster 2021-03-10T16:16:04.712684+0000 mgr.gibba032.whopeo (mgr.24625) 7573 : cluster [DBG] pgmap v4306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:06.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:06 gibba011 bash[31774]: cluster 2021-03-10T16:16:04.712684+0000 mgr.gibba032.whopeo (mgr.24625) 7573 : cluster [DBG] pgmap v4306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:06 gibba032 bash[24004]: cluster 2021-03-10T16:16:04.712684+0000 mgr.gibba032.whopeo (mgr.24625) 7573 : cluster [DBG] pgmap v4306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:08 gibba011 bash[30791]: cluster 2021-03-10T16:16:06.714425+0000 mgr.gibba032.whopeo (mgr.24625) 7574 : cluster [DBG] pgmap v4307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:08 gibba011 bash[31774]: cluster 2021-03-10T16:16:06.714425+0000 mgr.gibba032.whopeo (mgr.24625) 7574 : cluster [DBG] pgmap v4307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:08 gibba032 bash[24004]: cluster 2021-03-10T16:16:06.714425+0000 mgr.gibba032.whopeo (mgr.24625) 7574 : cluster [DBG] pgmap v4307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:10 gibba011 bash[30791]: cluster 2021-03-10T16:16:08.715784+0000 mgr.gibba032.whopeo (mgr.24625) 7575 : cluster [DBG] pgmap v4308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:10 gibba011 bash[31774]: cluster 2021-03-10T16:16:08.715784+0000 mgr.gibba032.whopeo (mgr.24625) 7575 : cluster [DBG] pgmap v4308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:10 gibba032 bash[24004]: cluster 2021-03-10T16:16:08.715784+0000 mgr.gibba032.whopeo (mgr.24625) 7575 : cluster [DBG] pgmap v4308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:12 gibba011 bash[30791]: cluster 2021-03-10T16:16:10.716803+0000 mgr.gibba032.whopeo (mgr.24625) 7576 : cluster [DBG] pgmap v4309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:12.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:12 gibba011 bash[31774]: cluster 2021-03-10T16:16:10.716803+0000 mgr.gibba032.whopeo (mgr.24625) 7576 : cluster [DBG] pgmap v4309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:12 gibba032 bash[24004]: cluster 2021-03-10T16:16:10.716803+0000 mgr.gibba032.whopeo (mgr.24625) 7576 : cluster [DBG] pgmap v4309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:14 gibba011 bash[30791]: cluster 2021-03-10T16:16:12.718254+0000 mgr.gibba032.whopeo (mgr.24625) 7577 : cluster [DBG] pgmap v4310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:14 gibba011 bash[31774]: cluster 2021-03-10T16:16:12.718254+0000 mgr.gibba032.whopeo (mgr.24625) 7577 : cluster [DBG] pgmap v4310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:14 gibba032 bash[24004]: cluster 2021-03-10T16:16:12.718254+0000 mgr.gibba032.whopeo (mgr.24625) 7577 : cluster [DBG] pgmap v4310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:16.608 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:16:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:16 gibba032 bash[24004]: cluster 2021-03-10T16:16:14.719614+0000 mgr.gibba032.whopeo (mgr.24625) 7578 : cluster [DBG] pgmap v4311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:16 gibba011 bash[30791]: cluster 2021-03-10T16:16:14.719614+0000 mgr.gibba032.whopeo (mgr.24625) 7578 : cluster [DBG] pgmap v4311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:16 gibba011 bash[31774]: cluster 2021-03-10T16:16:14.719614+0000 mgr.gibba032.whopeo (mgr.24625) 7578 : cluster [DBG] pgmap v4311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:16.835 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:16:16.835 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:16:16.835 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:16:16.836 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:16:16.836 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:16:16.836 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:16:16.836 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:16:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:16:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:16:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:16:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:16:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:16:16.838 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:16:16.838 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:16:16.838 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:16:16.838 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:16:16.839 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:16:16.839 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:16:16.839 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:16:16.839 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:16:16.839 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:16:17.082 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:16:17.083 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:16:17.083 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:16:17.083 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:17.083 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:16:17.084 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:16:17.084 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:17.084 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:16:17.084 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:16:17.084 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:17.085 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:16:17.085 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:16:17.085 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:16:17.085 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:16:17.086 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:16:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:17 gibba032 bash[24004]: cephadm 2021-03-10T16:16:15.612983+0000 mgr.gibba032.whopeo (mgr.24625) 7579 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:17 gibba032 bash[24004]: audit 2021-03-10T16:16:17.083873+0000 mon.a (mon.0) 2694 : audit [DBG] from='client.? 172.21.2.111:0/1380912511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:17.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:17 gibba011 bash[30791]: cephadm 2021-03-10T16:16:15.612983+0000 mgr.gibba032.whopeo (mgr.24625) 7579 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:17.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:17 gibba011 bash[30791]: audit 2021-03-10T16:16:17.083873+0000 mon.a (mon.0) 2694 : audit [DBG] from='client.? 172.21.2.111:0/1380912511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:17.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:17 gibba011 bash[31774]: cephadm 2021-03-10T16:16:15.612983+0000 mgr.gibba032.whopeo (mgr.24625) 7579 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:17.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:17 gibba011 bash[31774]: audit 2021-03-10T16:16:17.083873+0000 mon.a (mon.0) 2694 : audit [DBG] from='client.? 172.21.2.111:0/1380912511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:18 gibba032 bash[24004]: audit 2021-03-10T16:16:16.598630+0000 mgr.gibba032.whopeo (mgr.24625) 7580 : audit [DBG] from='client.59113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:18 gibba032 bash[24004]: cluster 2021-03-10T16:16:16.721283+0000 mgr.gibba032.whopeo (mgr.24625) 7581 : cluster [DBG] pgmap v4312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:18 gibba032 bash[24004]: audit 2021-03-10T16:16:16.822889+0000 mgr.gibba032.whopeo (mgr.24625) 7582 : audit [DBG] from='client.59119 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[30791]: audit 2021-03-10T16:16:16.598630+0000 mgr.gibba032.whopeo (mgr.24625) 7580 : audit [DBG] from='client.59113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[30791]: cluster 2021-03-10T16:16:16.721283+0000 mgr.gibba032.whopeo (mgr.24625) 7581 : cluster [DBG] pgmap v4312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[30791]: audit 2021-03-10T16:16:16.822889+0000 mgr.gibba032.whopeo (mgr.24625) 7582 : audit [DBG] from='client.59119 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[31774]: audit 2021-03-10T16:16:16.598630+0000 mgr.gibba032.whopeo (mgr.24625) 7580 : audit [DBG] from='client.59113 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[31774]: cluster 2021-03-10T16:16:16.721283+0000 mgr.gibba032.whopeo (mgr.24625) 7581 : cluster [DBG] pgmap v4312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:18.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:18 gibba011 bash[31774]: audit 2021-03-10T16:16:16.822889+0000 mgr.gibba032.whopeo (mgr.24625) 7582 : audit [DBG] from='client.59119 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:20 gibba032 bash[24004]: cluster 2021-03-10T16:16:18.722872+0000 mgr.gibba032.whopeo (mgr.24625) 7583 : cluster [DBG] pgmap v4313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:20 gibba011 bash[30791]: cluster 2021-03-10T16:16:18.722872+0000 mgr.gibba032.whopeo (mgr.24625) 7583 : cluster [DBG] pgmap v4313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:20.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:20 gibba011 bash[31774]: cluster 2021-03-10T16:16:18.722872+0000 mgr.gibba032.whopeo (mgr.24625) 7583 : cluster [DBG] pgmap v4313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:22 gibba032 bash[24004]: cluster 2021-03-10T16:16:20.724299+0000 mgr.gibba032.whopeo (mgr.24625) 7584 : cluster [DBG] pgmap v4314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:22 gibba011 bash[30791]: cluster 2021-03-10T16:16:20.724299+0000 mgr.gibba032.whopeo (mgr.24625) 7584 : cluster [DBG] pgmap v4314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:22 gibba011 bash[31774]: cluster 2021-03-10T16:16:20.724299+0000 mgr.gibba032.whopeo (mgr.24625) 7584 : cluster [DBG] pgmap v4314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:24 gibba032 bash[24004]: cluster 2021-03-10T16:16:22.726060+0000 mgr.gibba032.whopeo (mgr.24625) 7585 : cluster [DBG] pgmap v4315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:24 gibba011 bash[30791]: cluster 2021-03-10T16:16:22.726060+0000 mgr.gibba032.whopeo (mgr.24625) 7585 : cluster [DBG] pgmap v4315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:24 gibba011 bash[31774]: cluster 2021-03-10T16:16:22.726060+0000 mgr.gibba032.whopeo (mgr.24625) 7585 : cluster [DBG] pgmap v4315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:26 gibba032 bash[24004]: cluster 2021-03-10T16:16:24.727552+0000 mgr.gibba032.whopeo (mgr.24625) 7586 : cluster [DBG] pgmap v4316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:26 gibba011 bash[30791]: cluster 2021-03-10T16:16:24.727552+0000 mgr.gibba032.whopeo (mgr.24625) 7586 : cluster [DBG] pgmap v4316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:26 gibba011 bash[31774]: cluster 2021-03-10T16:16:24.727552+0000 mgr.gibba032.whopeo (mgr.24625) 7586 : cluster [DBG] pgmap v4316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:28 gibba032 bash[24004]: cluster 2021-03-10T16:16:26.729217+0000 mgr.gibba032.whopeo (mgr.24625) 7587 : cluster [DBG] pgmap v4317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:28 gibba011 bash[30791]: cluster 2021-03-10T16:16:26.729217+0000 mgr.gibba032.whopeo (mgr.24625) 7587 : cluster [DBG] pgmap v4317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:28 gibba011 bash[31774]: cluster 2021-03-10T16:16:26.729217+0000 mgr.gibba032.whopeo (mgr.24625) 7587 : cluster [DBG] pgmap v4317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:30 gibba032 bash[24004]: cluster 2021-03-10T16:16:28.730774+0000 mgr.gibba032.whopeo (mgr.24625) 7588 : cluster [DBG] pgmap v4318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:30 gibba011 bash[30791]: cluster 2021-03-10T16:16:28.730774+0000 mgr.gibba032.whopeo (mgr.24625) 7588 : cluster [DBG] pgmap v4318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:30 gibba011 bash[31774]: cluster 2021-03-10T16:16:28.730774+0000 mgr.gibba032.whopeo (mgr.24625) 7588 : cluster [DBG] pgmap v4318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:32 gibba032 bash[24004]: cephadm 2021-03-10T16:16:30.628937+0000 mgr.gibba032.whopeo (mgr.24625) 7589 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:32 gibba032 bash[24004]: cluster 2021-03-10T16:16:30.731547+0000 mgr.gibba032.whopeo (mgr.24625) 7590 : cluster [DBG] pgmap v4319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:32 gibba011 bash[30791]: cephadm 2021-03-10T16:16:30.628937+0000 mgr.gibba032.whopeo (mgr.24625) 7589 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:32 gibba011 bash[30791]: cluster 2021-03-10T16:16:30.731547+0000 mgr.gibba032.whopeo (mgr.24625) 7590 : cluster [DBG] pgmap v4319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:32 gibba011 bash[31774]: cephadm 2021-03-10T16:16:30.628937+0000 mgr.gibba032.whopeo (mgr.24625) 7589 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:32 gibba011 bash[31774]: cluster 2021-03-10T16:16:30.731547+0000 mgr.gibba032.whopeo (mgr.24625) 7590 : cluster [DBG] pgmap v4319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:34 gibba032 bash[24004]: cluster 2021-03-10T16:16:32.733147+0000 mgr.gibba032.whopeo (mgr.24625) 7591 : cluster [DBG] pgmap v4320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:34 gibba011 bash[30791]: cluster 2021-03-10T16:16:32.733147+0000 mgr.gibba032.whopeo (mgr.24625) 7591 : cluster [DBG] pgmap v4320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:34 gibba011 bash[31774]: cluster 2021-03-10T16:16:32.733147+0000 mgr.gibba032.whopeo (mgr.24625) 7591 : cluster [DBG] pgmap v4320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:36 gibba032 bash[24004]: cluster 2021-03-10T16:16:34.734626+0000 mgr.gibba032.whopeo (mgr.24625) 7592 : cluster [DBG] pgmap v4321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:36 gibba011 bash[30791]: cluster 2021-03-10T16:16:34.734626+0000 mgr.gibba032.whopeo (mgr.24625) 7592 : cluster [DBG] pgmap v4321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:36 gibba011 bash[31774]: cluster 2021-03-10T16:16:34.734626+0000 mgr.gibba032.whopeo (mgr.24625) 7592 : cluster [DBG] pgmap v4321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:38 gibba032 bash[24004]: cluster 2021-03-10T16:16:36.736218+0000 mgr.gibba032.whopeo (mgr.24625) 7593 : cluster [DBG] pgmap v4322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:38 gibba011 bash[30791]: cluster 2021-03-10T16:16:36.736218+0000 mgr.gibba032.whopeo (mgr.24625) 7593 : cluster [DBG] pgmap v4322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:38 gibba011 bash[31774]: cluster 2021-03-10T16:16:36.736218+0000 mgr.gibba032.whopeo (mgr.24625) 7593 : cluster [DBG] pgmap v4322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:40 gibba032 bash[24004]: cluster 2021-03-10T16:16:38.737644+0000 mgr.gibba032.whopeo (mgr.24625) 7594 : cluster [DBG] pgmap v4323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:40 gibba011 bash[30791]: cluster 2021-03-10T16:16:38.737644+0000 mgr.gibba032.whopeo (mgr.24625) 7594 : cluster [DBG] pgmap v4323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:40 gibba011 bash[31774]: cluster 2021-03-10T16:16:38.737644+0000 mgr.gibba032.whopeo (mgr.24625) 7594 : cluster [DBG] pgmap v4323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:42 gibba032 bash[24004]: cluster 2021-03-10T16:16:40.738986+0000 mgr.gibba032.whopeo (mgr.24625) 7595 : cluster [DBG] pgmap v4324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:42 gibba011 bash[30791]: cluster 2021-03-10T16:16:40.738986+0000 mgr.gibba032.whopeo (mgr.24625) 7595 : cluster [DBG] pgmap v4324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:42.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:42 gibba011 bash[31774]: cluster 2021-03-10T16:16:40.738986+0000 mgr.gibba032.whopeo (mgr.24625) 7595 : cluster [DBG] pgmap v4324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:44 gibba032 bash[24004]: cluster 2021-03-10T16:16:42.740719+0000 mgr.gibba032.whopeo (mgr.24625) 7596 : cluster [DBG] pgmap v4325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:44 gibba011 bash[30791]: cluster 2021-03-10T16:16:42.740719+0000 mgr.gibba032.whopeo (mgr.24625) 7596 : cluster [DBG] pgmap v4325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:44 gibba011 bash[31774]: cluster 2021-03-10T16:16:42.740719+0000 mgr.gibba032.whopeo (mgr.24625) 7596 : cluster [DBG] pgmap v4325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:46 gibba032 bash[24004]: cluster 2021-03-10T16:16:44.742064+0000 mgr.gibba032.whopeo (mgr.24625) 7597 : cluster [DBG] pgmap v4326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:46 gibba011 bash[30791]: cluster 2021-03-10T16:16:44.742064+0000 mgr.gibba032.whopeo (mgr.24625) 7597 : cluster [DBG] pgmap v4326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:46 gibba011 bash[31774]: cluster 2021-03-10T16:16:44.742064+0000 mgr.gibba032.whopeo (mgr.24625) 7597 : cluster [DBG] pgmap v4326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:47.327 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:16:47.554 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:16:47.554 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:16:47.554 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:16:47.555 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:16:47.555 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:16:47.555 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:16:47.555 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:16:47.555 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:16:47.556 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:16:47.556 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:16:47.556 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:16:47.556 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:16:47.557 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:16:47.557 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:16:47.557 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:16:47.557 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:16:47.557 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:16:47.558 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:16:47.558 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:16:47.558 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:16:47.559 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:16:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:47 gibba032 bash[24004]: cephadm 2021-03-10T16:16:45.644866+0000 mgr.gibba032.whopeo (mgr.24625) 7598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:47.803 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:16:47.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:16:47.804 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:16:47.804 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:47.804 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:16:47.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:16:47.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:47.805 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:16:47.805 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:16:47.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:16:47.806 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:16:47.806 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:16:47.806 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:16:47.806 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:16:47.807 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:16:47.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:47 gibba011 bash[30791]: cephadm 2021-03-10T16:16:45.644866+0000 mgr.gibba032.whopeo (mgr.24625) 7598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:47.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:47 gibba011 bash[31774]: cephadm 2021-03-10T16:16:45.644866+0000 mgr.gibba032.whopeo (mgr.24625) 7598 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:16:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:48 gibba032 bash[24004]: cluster 2021-03-10T16:16:46.743693+0000 mgr.gibba032.whopeo (mgr.24625) 7599 : cluster [DBG] pgmap v4327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:48 gibba032 bash[24004]: audit 2021-03-10T16:16:47.316771+0000 mgr.gibba032.whopeo (mgr.24625) 7600 : audit [DBG] from='client.59131 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:48 gibba032 bash[24004]: audit 2021-03-10T16:16:47.804768+0000 mon.c (mon.1) 145 : audit [DBG] from='client.? 172.21.2.111:0/1765451212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[30791]: cluster 2021-03-10T16:16:46.743693+0000 mgr.gibba032.whopeo (mgr.24625) 7599 : cluster [DBG] pgmap v4327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[30791]: audit 2021-03-10T16:16:47.316771+0000 mgr.gibba032.whopeo (mgr.24625) 7600 : audit [DBG] from='client.59131 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[30791]: audit 2021-03-10T16:16:47.804768+0000 mon.c (mon.1) 145 : audit [DBG] from='client.? 172.21.2.111:0/1765451212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[31774]: cluster 2021-03-10T16:16:46.743693+0000 mgr.gibba032.whopeo (mgr.24625) 7599 : cluster [DBG] pgmap v4327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[31774]: audit 2021-03-10T16:16:47.316771+0000 mgr.gibba032.whopeo (mgr.24625) 7600 : audit [DBG] from='client.59131 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:48 gibba011 bash[31774]: audit 2021-03-10T16:16:47.804768+0000 mon.c (mon.1) 145 : audit [DBG] from='client.? 172.21.2.111:0/1765451212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:16:49.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:49 gibba032 bash[24004]: audit 2021-03-10T16:16:47.541897+0000 mgr.gibba032.whopeo (mgr.24625) 7601 : audit [DBG] from='client.49155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:49.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:49 gibba032 bash[24004]: cluster 2021-03-10T16:16:48.745351+0000 mgr.gibba032.whopeo (mgr.24625) 7602 : cluster [DBG] pgmap v4328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:49.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:49 gibba011 bash[30791]: audit 2021-03-10T16:16:47.541897+0000 mgr.gibba032.whopeo (mgr.24625) 7601 : audit [DBG] from='client.49155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:49.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:49 gibba011 bash[30791]: cluster 2021-03-10T16:16:48.745351+0000 mgr.gibba032.whopeo (mgr.24625) 7602 : cluster [DBG] pgmap v4328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:49.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:49 gibba011 bash[31774]: audit 2021-03-10T16:16:47.541897+0000 mgr.gibba032.whopeo (mgr.24625) 7601 : audit [DBG] from='client.49155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:16:49.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:49 gibba011 bash[31774]: cluster 2021-03-10T16:16:48.745351+0000 mgr.gibba032.whopeo (mgr.24625) 7602 : cluster [DBG] pgmap v4328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:52.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:51 gibba011 bash[30791]: cluster 2021-03-10T16:16:50.746800+0000 mgr.gibba032.whopeo (mgr.24625) 7603 : cluster [DBG] pgmap v4329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:52.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:51 gibba011 bash[31774]: cluster 2021-03-10T16:16:50.746800+0000 mgr.gibba032.whopeo (mgr.24625) 7603 : cluster [DBG] pgmap v4329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:51 gibba032 bash[24004]: cluster 2021-03-10T16:16:50.746800+0000 mgr.gibba032.whopeo (mgr.24625) 7603 : cluster [DBG] pgmap v4329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:54.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:53 gibba011 bash[30791]: cluster 2021-03-10T16:16:52.748635+0000 mgr.gibba032.whopeo (mgr.24625) 7604 : cluster [DBG] pgmap v4330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:54.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:53 gibba011 bash[31774]: cluster 2021-03-10T16:16:52.748635+0000 mgr.gibba032.whopeo (mgr.24625) 7604 : cluster [DBG] pgmap v4330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:53 gibba032 bash[24004]: cluster 2021-03-10T16:16:52.748635+0000 mgr.gibba032.whopeo (mgr.24625) 7604 : cluster [DBG] pgmap v4330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:56.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:56 gibba032 bash[24004]: cluster 2021-03-10T16:16:54.750184+0000 mgr.gibba032.whopeo (mgr.24625) 7605 : cluster [DBG] pgmap v4331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:56 gibba011 bash[30791]: cluster 2021-03-10T16:16:54.750184+0000 mgr.gibba032.whopeo (mgr.24625) 7605 : cluster [DBG] pgmap v4331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:56 gibba011 bash[31774]: cluster 2021-03-10T16:16:54.750184+0000 mgr.gibba032.whopeo (mgr.24625) 7605 : cluster [DBG] pgmap v4331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:56.925775+0000 mon.a (mon.0) 2695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:16:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:56.926465+0000 mon.a (mon.0) 2696 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:16:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:57.015149+0000 mon.a (mon.0) 2697 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:16:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:57.016048+0000 mon.a (mon.0) 2698 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:16:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:57.016808+0000 mon.a (mon.0) 2699 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:16:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:57 gibba032 bash[24004]: audit 2021-03-10T16:16:57.017543+0000 mon.a (mon.0) 2700 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:16:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:56.925775+0000 mon.a (mon.0) 2695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:16:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:56.926465+0000 mon.a (mon.0) 2696 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:16:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:57.015149+0000 mon.a (mon.0) 2697 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:16:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:57.016048+0000 mon.a (mon.0) 2698 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:16:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:57.016808+0000 mon.a (mon.0) 2699 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:16:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[30791]: audit 2021-03-10T16:16:57.017543+0000 mon.a (mon.0) 2700 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:16:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:56.925775+0000 mon.a (mon.0) 2695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:16:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:56.926465+0000 mon.a (mon.0) 2696 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:16:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:57.015149+0000 mon.a (mon.0) 2697 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:16:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:57.016048+0000 mon.a (mon.0) 2698 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:16:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:57.016808+0000 mon.a (mon.0) 2699 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:16:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:57 gibba011 bash[31774]: audit 2021-03-10T16:16:57.017543+0000 mon.a (mon.0) 2700 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:16:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:16:58 gibba032 bash[24004]: cluster 2021-03-10T16:16:56.752137+0000 mgr.gibba032.whopeo (mgr.24625) 7606 : cluster [DBG] pgmap v4332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:16:58 gibba011 bash[30791]: cluster 2021-03-10T16:16:56.752137+0000 mgr.gibba032.whopeo (mgr.24625) 7606 : cluster [DBG] pgmap v4332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:16:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:16:58 gibba011 bash[31774]: cluster 2021-03-10T16:16:56.752137+0000 mgr.gibba032.whopeo (mgr.24625) 7606 : cluster [DBG] pgmap v4332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:00 gibba032 bash[24004]: cluster 2021-03-10T16:16:58.753769+0000 mgr.gibba032.whopeo (mgr.24625) 7607 : cluster [DBG] pgmap v4333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:00 gibba011 bash[30791]: cluster 2021-03-10T16:16:58.753769+0000 mgr.gibba032.whopeo (mgr.24625) 7607 : cluster [DBG] pgmap v4333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:00 gibba011 bash[31774]: cluster 2021-03-10T16:16:58.753769+0000 mgr.gibba032.whopeo (mgr.24625) 7607 : cluster [DBG] pgmap v4333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cluster 2021-03-10T16:17:00.754287+0000 mgr.gibba032.whopeo (mgr.24625) 7608 : cluster [DBG] pgmap v4334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:00.842109+0000 mgr.gibba032.whopeo (mgr.24625) 7609 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:17:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:00.843618+0000 mon.a (mon.0) 2701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:00.845725+0000 mgr.gibba032.whopeo (mgr.24625) 7610 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:17:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:00.846990+0000 mon.a (mon.0) 2702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.047203+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.047658+0000 mgr.gibba032.whopeo (mgr.24625) 7611 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:17:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.049463+0000 mgr.gibba032.whopeo (mgr.24625) 7612 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:17:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.050553+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.050895+0000 mgr.gibba032.whopeo (mgr.24625) 7613 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.050932+0000 mgr.gibba032.whopeo (mgr.24625) 7614 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.051984+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.052727+0000 mgr.gibba032.whopeo (mgr.24625) 7615 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:17:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.052765+0000 mgr.gibba032.whopeo (mgr.24625) 7616 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:17:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.053700+0000 mon.a (mon.0) 2706 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.054603+0000 mgr.gibba032.whopeo (mgr.24625) 7617 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:17:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.055323+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.056275+0000 mgr.gibba032.whopeo (mgr.24625) 7618 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:17:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.056310+0000 mgr.gibba032.whopeo (mgr.24625) 7619 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:17:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.057806+0000 mgr.gibba032.whopeo (mgr.24625) 7620 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:17:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.058140+0000 mgr.gibba032.whopeo (mgr.24625) 7621 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:17:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.059033+0000 mon.a (mon.0) 2708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.059572+0000 mgr.gibba032.whopeo (mgr.24625) 7622 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:17:02.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.059632+0000 mgr.gibba032.whopeo (mgr.24625) 7623 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:17:02.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: cephadm 2021-03-10T16:17:01.059735+0000 mgr.gibba032.whopeo (mgr.24625) 7624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:02.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:01 gibba032 bash[24004]: audit 2021-03-10T16:17:01.060692+0000 mon.a (mon.0) 2709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cluster 2021-03-10T16:17:00.754287+0000 mgr.gibba032.whopeo (mgr.24625) 7608 : cluster [DBG] pgmap v4334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:00.842109+0000 mgr.gibba032.whopeo (mgr.24625) 7609 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:17:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:00.843618+0000 mon.a (mon.0) 2701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:00.845725+0000 mgr.gibba032.whopeo (mgr.24625) 7610 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:17:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:00.846990+0000 mon.a (mon.0) 2702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.047203+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.047658+0000 mgr.gibba032.whopeo (mgr.24625) 7611 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:17:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.049463+0000 mgr.gibba032.whopeo (mgr.24625) 7612 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:17:02.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.050553+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:17:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.050895+0000 mgr.gibba032.whopeo (mgr.24625) 7613 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:17:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.050932+0000 mgr.gibba032.whopeo (mgr.24625) 7614 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:17:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.051984+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.052727+0000 mgr.gibba032.whopeo (mgr.24625) 7615 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:17:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.052765+0000 mgr.gibba032.whopeo (mgr.24625) 7616 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:17:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.053700+0000 mon.a (mon.0) 2706 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.054603+0000 mgr.gibba032.whopeo (mgr.24625) 7617 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:17:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.055323+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.056275+0000 mgr.gibba032.whopeo (mgr.24625) 7618 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:17:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.056310+0000 mgr.gibba032.whopeo (mgr.24625) 7619 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:17:02.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.057806+0000 mgr.gibba032.whopeo (mgr.24625) 7620 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:17:02.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.058140+0000 mgr.gibba032.whopeo (mgr.24625) 7621 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:17:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cluster 2021-03-10T16:17:00.754287+0000 mgr.gibba032.whopeo (mgr.24625) 7608 : cluster [DBG] pgmap v4334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:00.842109+0000 mgr.gibba032.whopeo (mgr.24625) 7609 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:17:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:00.843618+0000 mon.a (mon.0) 2701 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:00.845725+0000 mgr.gibba032.whopeo (mgr.24625) 7610 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:17:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:00.846990+0000 mon.a (mon.0) 2702 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.047203+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:17:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.047658+0000 mgr.gibba032.whopeo (mgr.24625) 7611 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:17:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.049463+0000 mgr.gibba032.whopeo (mgr.24625) 7612 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:17:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.050553+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:17:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.050895+0000 mgr.gibba032.whopeo (mgr.24625) 7613 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:17:02.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.050932+0000 mgr.gibba032.whopeo (mgr.24625) 7614 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:17:02.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.051984+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.052727+0000 mgr.gibba032.whopeo (mgr.24625) 7615 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:17:02.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.052765+0000 mgr.gibba032.whopeo (mgr.24625) 7616 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:17:02.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.053700+0000 mon.a (mon.0) 2706 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.054603+0000 mgr.gibba032.whopeo (mgr.24625) 7617 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:17:02.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.055323+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.056275+0000 mgr.gibba032.whopeo (mgr.24625) 7618 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:17:02.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.056310+0000 mgr.gibba032.whopeo (mgr.24625) 7619 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:17:02.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.057806+0000 mgr.gibba032.whopeo (mgr.24625) 7620 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:17:02.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.058140+0000 mgr.gibba032.whopeo (mgr.24625) 7621 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:17:02.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.059033+0000 mon.a (mon.0) 2708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.059572+0000 mgr.gibba032.whopeo (mgr.24625) 7622 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:17:02.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.059632+0000 mgr.gibba032.whopeo (mgr.24625) 7623 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:17:02.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: cephadm 2021-03-10T16:17:01.059735+0000 mgr.gibba032.whopeo (mgr.24625) 7624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:02.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[30791]: audit 2021-03-10T16:17:01.060692+0000 mon.a (mon.0) 2709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.059033+0000 mon.a (mon.0) 2708 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:02.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.059572+0000 mgr.gibba032.whopeo (mgr.24625) 7622 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:17:02.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.059632+0000 mgr.gibba032.whopeo (mgr.24625) 7623 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:17:02.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: cephadm 2021-03-10T16:17:01.059735+0000 mgr.gibba032.whopeo (mgr.24625) 7624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:02.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:01 gibba011 bash[31774]: audit 2021-03-10T16:17:01.060692+0000 mon.a (mon.0) 2709 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:03 gibba032 bash[24004]: cluster 2021-03-10T16:17:02.755923+0000 mgr.gibba032.whopeo (mgr.24625) 7625 : cluster [DBG] pgmap v4335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:04.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:03 gibba011 bash[30791]: cluster 2021-03-10T16:17:02.755923+0000 mgr.gibba032.whopeo (mgr.24625) 7625 : cluster [DBG] pgmap v4335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:03 gibba011 bash[31774]: cluster 2021-03-10T16:17:02.755923+0000 mgr.gibba032.whopeo (mgr.24625) 7625 : cluster [DBG] pgmap v4335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:06 gibba032 bash[24004]: cluster 2021-03-10T16:17:04.757511+0000 mgr.gibba032.whopeo (mgr.24625) 7626 : cluster [DBG] pgmap v4336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:06.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:06 gibba011 bash[30791]: cluster 2021-03-10T16:17:04.757511+0000 mgr.gibba032.whopeo (mgr.24625) 7626 : cluster [DBG] pgmap v4336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:06 gibba011 bash[31774]: cluster 2021-03-10T16:17:04.757511+0000 mgr.gibba032.whopeo (mgr.24625) 7626 : cluster [DBG] pgmap v4336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:08 gibba032 bash[24004]: cluster 2021-03-10T16:17:06.759233+0000 mgr.gibba032.whopeo (mgr.24625) 7627 : cluster [DBG] pgmap v4337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:08 gibba011 bash[30791]: cluster 2021-03-10T16:17:06.759233+0000 mgr.gibba032.whopeo (mgr.24625) 7627 : cluster [DBG] pgmap v4337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:08 gibba011 bash[31774]: cluster 2021-03-10T16:17:06.759233+0000 mgr.gibba032.whopeo (mgr.24625) 7627 : cluster [DBG] pgmap v4337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:10 gibba032 bash[24004]: cluster 2021-03-10T16:17:08.760908+0000 mgr.gibba032.whopeo (mgr.24625) 7628 : cluster [DBG] pgmap v4338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:10 gibba011 bash[30791]: cluster 2021-03-10T16:17:08.760908+0000 mgr.gibba032.whopeo (mgr.24625) 7628 : cluster [DBG] pgmap v4338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:10 gibba011 bash[31774]: cluster 2021-03-10T16:17:08.760908+0000 mgr.gibba032.whopeo (mgr.24625) 7628 : cluster [DBG] pgmap v4338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:12.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:12 gibba032 bash[24004]: cluster 2021-03-10T16:17:10.762360+0000 mgr.gibba032.whopeo (mgr.24625) 7629 : cluster [DBG] pgmap v4339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:12 gibba011 bash[30791]: cluster 2021-03-10T16:17:10.762360+0000 mgr.gibba032.whopeo (mgr.24625) 7629 : cluster [DBG] pgmap v4339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:12.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:12 gibba011 bash[31774]: cluster 2021-03-10T16:17:10.762360+0000 mgr.gibba032.whopeo (mgr.24625) 7629 : cluster [DBG] pgmap v4339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:14.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:14 gibba032 bash[24004]: cluster 2021-03-10T16:17:12.763876+0000 mgr.gibba032.whopeo (mgr.24625) 7630 : cluster [DBG] pgmap v4340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:14 gibba011 bash[30791]: cluster 2021-03-10T16:17:12.763876+0000 mgr.gibba032.whopeo (mgr.24625) 7630 : cluster [DBG] pgmap v4340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:14 gibba011 bash[31774]: cluster 2021-03-10T16:17:12.763876+0000 mgr.gibba032.whopeo (mgr.24625) 7630 : cluster [DBG] pgmap v4340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:16 gibba032 bash[24004]: cluster 2021-03-10T16:17:14.765278+0000 mgr.gibba032.whopeo (mgr.24625) 7631 : cluster [DBG] pgmap v4341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:16 gibba011 bash[30791]: cluster 2021-03-10T16:17:14.765278+0000 mgr.gibba032.whopeo (mgr.24625) 7631 : cluster [DBG] pgmap v4341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:16 gibba011 bash[31774]: cluster 2021-03-10T16:17:14.765278+0000 mgr.gibba032.whopeo (mgr.24625) 7631 : cluster [DBG] pgmap v4341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:17.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:17 gibba032 bash[24004]: cephadm 2021-03-10T16:17:16.071786+0000 mgr.gibba032.whopeo (mgr.24625) 7632 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:17.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:17 gibba011 bash[30791]: cephadm 2021-03-10T16:17:16.071786+0000 mgr.gibba032.whopeo (mgr.24625) 7632 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:17.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:17 gibba011 bash[31774]: cephadm 2021-03-10T16:17:16.071786+0000 mgr.gibba032.whopeo (mgr.24625) 7632 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:18.046 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:17:18.305 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:17:18.305 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:17:18.306 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:17:18.306 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:17:18.306 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:17:18.306 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:17:18.307 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:17:18.307 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:17:18.307 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:17:18.308 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:17:18.308 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:17:18.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:17:18.308 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:17:18.309 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:17:18.309 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:17:18.309 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:17:18.309 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:17:18.310 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:17:18.310 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:17:18.310 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:17:18.310 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:17:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:18 gibba032 bash[24004]: cluster 2021-03-10T16:17:16.767190+0000 mgr.gibba032.whopeo (mgr.24625) 7633 : cluster [DBG] pgmap v4342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:18.563 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:17:18.563 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:17:18.564 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:17:18.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:18.564 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:17:18.564 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:17:18.565 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:18.565 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:17:18.565 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:17:18.565 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:18.566 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:17:18.566 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:17:18.566 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:17:18.566 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:17:18.567 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:17:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:18 gibba011 bash[30791]: cluster 2021-03-10T16:17:16.767190+0000 mgr.gibba032.whopeo (mgr.24625) 7633 : cluster [DBG] pgmap v4342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:18.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:18 gibba011 bash[31774]: cluster 2021-03-10T16:17:16.767190+0000 mgr.gibba032.whopeo (mgr.24625) 7633 : cluster [DBG] pgmap v4342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:19 gibba032 bash[24004]: audit 2021-03-10T16:17:18.037920+0000 mgr.gibba032.whopeo (mgr.24625) 7634 : audit [DBG] from='client.49167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:19 gibba032 bash[24004]: audit 2021-03-10T16:17:18.265310+0000 mgr.gibba032.whopeo (mgr.24625) 7635 : audit [DBG] from='client.59155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:19 gibba032 bash[24004]: audit 2021-03-10T16:17:18.564888+0000 mon.c (mon.1) 146 : audit [DBG] from='client.? 172.21.2.111:0/3659231637' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:19.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[30791]: audit 2021-03-10T16:17:18.037920+0000 mgr.gibba032.whopeo (mgr.24625) 7634 : audit [DBG] from='client.49167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[30791]: audit 2021-03-10T16:17:18.265310+0000 mgr.gibba032.whopeo (mgr.24625) 7635 : audit [DBG] from='client.59155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[30791]: audit 2021-03-10T16:17:18.564888+0000 mon.c (mon.1) 146 : audit [DBG] from='client.? 172.21.2.111:0/3659231637' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:19.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[31774]: audit 2021-03-10T16:17:18.037920+0000 mgr.gibba032.whopeo (mgr.24625) 7634 : audit [DBG] from='client.49167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[31774]: audit 2021-03-10T16:17:18.265310+0000 mgr.gibba032.whopeo (mgr.24625) 7635 : audit [DBG] from='client.59155 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:19.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:19 gibba011 bash[31774]: audit 2021-03-10T16:17:18.564888+0000 mon.c (mon.1) 146 : audit [DBG] from='client.? 172.21.2.111:0/3659231637' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:20.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:20 gibba032 bash[24004]: cluster 2021-03-10T16:17:18.769005+0000 mgr.gibba032.whopeo (mgr.24625) 7636 : cluster [DBG] pgmap v4343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:20 gibba011 bash[30791]: cluster 2021-03-10T16:17:18.769005+0000 mgr.gibba032.whopeo (mgr.24625) 7636 : cluster [DBG] pgmap v4343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:20 gibba011 bash[31774]: cluster 2021-03-10T16:17:18.769005+0000 mgr.gibba032.whopeo (mgr.24625) 7636 : cluster [DBG] pgmap v4343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:22 gibba032 bash[24004]: cluster 2021-03-10T16:17:20.770407+0000 mgr.gibba032.whopeo (mgr.24625) 7637 : cluster [DBG] pgmap v4344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:22 gibba011 bash[30791]: cluster 2021-03-10T16:17:20.770407+0000 mgr.gibba032.whopeo (mgr.24625) 7637 : cluster [DBG] pgmap v4344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:22 gibba011 bash[31774]: cluster 2021-03-10T16:17:20.770407+0000 mgr.gibba032.whopeo (mgr.24625) 7637 : cluster [DBG] pgmap v4344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:24.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:24 gibba032 bash[24004]: cluster 2021-03-10T16:17:22.772093+0000 mgr.gibba032.whopeo (mgr.24625) 7638 : cluster [DBG] pgmap v4345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:24 gibba011 bash[30791]: cluster 2021-03-10T16:17:22.772093+0000 mgr.gibba032.whopeo (mgr.24625) 7638 : cluster [DBG] pgmap v4345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:24 gibba011 bash[31774]: cluster 2021-03-10T16:17:22.772093+0000 mgr.gibba032.whopeo (mgr.24625) 7638 : cluster [DBG] pgmap v4345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:26 gibba032 bash[24004]: cluster 2021-03-10T16:17:24.773321+0000 mgr.gibba032.whopeo (mgr.24625) 7639 : cluster [DBG] pgmap v4346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:26 gibba011 bash[30791]: cluster 2021-03-10T16:17:24.773321+0000 mgr.gibba032.whopeo (mgr.24625) 7639 : cluster [DBG] pgmap v4346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:26.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:26 gibba011 bash[31774]: cluster 2021-03-10T16:17:24.773321+0000 mgr.gibba032.whopeo (mgr.24625) 7639 : cluster [DBG] pgmap v4346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:28 gibba011 bash[30791]: cluster 2021-03-10T16:17:26.775006+0000 mgr.gibba032.whopeo (mgr.24625) 7640 : cluster [DBG] pgmap v4347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:28 gibba011 bash[31774]: cluster 2021-03-10T16:17:26.775006+0000 mgr.gibba032.whopeo (mgr.24625) 7640 : cluster [DBG] pgmap v4347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:28 gibba032 bash[24004]: cluster 2021-03-10T16:17:26.775006+0000 mgr.gibba032.whopeo (mgr.24625) 7640 : cluster [DBG] pgmap v4347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:30 gibba011 bash[30791]: cluster 2021-03-10T16:17:28.776405+0000 mgr.gibba032.whopeo (mgr.24625) 7641 : cluster [DBG] pgmap v4348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:30.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:30 gibba011 bash[31774]: cluster 2021-03-10T16:17:28.776405+0000 mgr.gibba032.whopeo (mgr.24625) 7641 : cluster [DBG] pgmap v4348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:30 gibba032 bash[24004]: cluster 2021-03-10T16:17:28.776405+0000 mgr.gibba032.whopeo (mgr.24625) 7641 : cluster [DBG] pgmap v4348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:32 gibba011 bash[30791]: cluster 2021-03-10T16:17:30.777697+0000 mgr.gibba032.whopeo (mgr.24625) 7642 : cluster [DBG] pgmap v4349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:32 gibba011 bash[30791]: cephadm 2021-03-10T16:17:31.084774+0000 mgr.gibba032.whopeo (mgr.24625) 7643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:32 gibba011 bash[31774]: cluster 2021-03-10T16:17:30.777697+0000 mgr.gibba032.whopeo (mgr.24625) 7642 : cluster [DBG] pgmap v4349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:32 gibba011 bash[31774]: cephadm 2021-03-10T16:17:31.084774+0000 mgr.gibba032.whopeo (mgr.24625) 7643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:32 gibba032 bash[24004]: cluster 2021-03-10T16:17:30.777697+0000 mgr.gibba032.whopeo (mgr.24625) 7642 : cluster [DBG] pgmap v4349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:32 gibba032 bash[24004]: cephadm 2021-03-10T16:17:31.084774+0000 mgr.gibba032.whopeo (mgr.24625) 7643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:34.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:34 gibba011 bash[30791]: cluster 2021-03-10T16:17:32.779376+0000 mgr.gibba032.whopeo (mgr.24625) 7644 : cluster [DBG] pgmap v4350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:34 gibba011 bash[31774]: cluster 2021-03-10T16:17:32.779376+0000 mgr.gibba032.whopeo (mgr.24625) 7644 : cluster [DBG] pgmap v4350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:34 gibba032 bash[24004]: cluster 2021-03-10T16:17:32.779376+0000 mgr.gibba032.whopeo (mgr.24625) 7644 : cluster [DBG] pgmap v4350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:36 gibba011 bash[30791]: cluster 2021-03-10T16:17:34.780797+0000 mgr.gibba032.whopeo (mgr.24625) 7645 : cluster [DBG] pgmap v4351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:36 gibba011 bash[31774]: cluster 2021-03-10T16:17:34.780797+0000 mgr.gibba032.whopeo (mgr.24625) 7645 : cluster [DBG] pgmap v4351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:36 gibba032 bash[24004]: cluster 2021-03-10T16:17:34.780797+0000 mgr.gibba032.whopeo (mgr.24625) 7645 : cluster [DBG] pgmap v4351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:38 gibba011 bash[30791]: cluster 2021-03-10T16:17:36.782258+0000 mgr.gibba032.whopeo (mgr.24625) 7646 : cluster [DBG] pgmap v4352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:38 gibba011 bash[31774]: cluster 2021-03-10T16:17:36.782258+0000 mgr.gibba032.whopeo (mgr.24625) 7646 : cluster [DBG] pgmap v4352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:38 gibba032 bash[24004]: cluster 2021-03-10T16:17:36.782258+0000 mgr.gibba032.whopeo (mgr.24625) 7646 : cluster [DBG] pgmap v4352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:40 gibba011 bash[30791]: cluster 2021-03-10T16:17:38.783770+0000 mgr.gibba032.whopeo (mgr.24625) 7647 : cluster [DBG] pgmap v4353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:40 gibba011 bash[31774]: cluster 2021-03-10T16:17:38.783770+0000 mgr.gibba032.whopeo (mgr.24625) 7647 : cluster [DBG] pgmap v4353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:40 gibba032 bash[24004]: cluster 2021-03-10T16:17:38.783770+0000 mgr.gibba032.whopeo (mgr.24625) 7647 : cluster [DBG] pgmap v4353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:42 gibba011 bash[30791]: cluster 2021-03-10T16:17:40.785087+0000 mgr.gibba032.whopeo (mgr.24625) 7648 : cluster [DBG] pgmap v4354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:42 gibba011 bash[31774]: cluster 2021-03-10T16:17:40.785087+0000 mgr.gibba032.whopeo (mgr.24625) 7648 : cluster [DBG] pgmap v4354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:42 gibba032 bash[24004]: cluster 2021-03-10T16:17:40.785087+0000 mgr.gibba032.whopeo (mgr.24625) 7648 : cluster [DBG] pgmap v4354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:44 gibba032 bash[24004]: cluster 2021-03-10T16:17:42.786613+0000 mgr.gibba032.whopeo (mgr.24625) 7649 : cluster [DBG] pgmap v4355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:44 gibba011 bash[30791]: cluster 2021-03-10T16:17:42.786613+0000 mgr.gibba032.whopeo (mgr.24625) 7649 : cluster [DBG] pgmap v4355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:44 gibba011 bash[31774]: cluster 2021-03-10T16:17:42.786613+0000 mgr.gibba032.whopeo (mgr.24625) 7649 : cluster [DBG] pgmap v4355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:46 gibba032 bash[24004]: cluster 2021-03-10T16:17:44.787868+0000 mgr.gibba032.whopeo (mgr.24625) 7650 : cluster [DBG] pgmap v4356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:46 gibba011 bash[30791]: cluster 2021-03-10T16:17:44.787868+0000 mgr.gibba032.whopeo (mgr.24625) 7650 : cluster [DBG] pgmap v4356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:46 gibba011 bash[31774]: cluster 2021-03-10T16:17:44.787868+0000 mgr.gibba032.whopeo (mgr.24625) 7650 : cluster [DBG] pgmap v4356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:47 gibba032 bash[24004]: cephadm 2021-03-10T16:17:46.096830+0000 mgr.gibba032.whopeo (mgr.24625) 7651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:47.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:47 gibba011 bash[30791]: cephadm 2021-03-10T16:17:46.096830+0000 mgr.gibba032.whopeo (mgr.24625) 7651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:47.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:47 gibba011 bash[31774]: cephadm 2021-03-10T16:17:46.096830+0000 mgr.gibba032.whopeo (mgr.24625) 7651 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:17:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:48 gibba032 bash[24004]: cluster 2021-03-10T16:17:46.789539+0000 mgr.gibba032.whopeo (mgr.24625) 7652 : cluster [DBG] pgmap v4357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:48.804 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:17:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:48 gibba011 bash[30791]: cluster 2021-03-10T16:17:46.789539+0000 mgr.gibba032.whopeo (mgr.24625) 7652 : cluster [DBG] pgmap v4357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:48.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:48 gibba011 bash[31774]: cluster 2021-03-10T16:17:46.789539+0000 mgr.gibba032.whopeo (mgr.24625) 7652 : cluster [DBG] pgmap v4357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:49.034 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:17:49.035 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:17:49.035 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:17:49.035 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:17:49.035 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:17:49.036 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:17:49.036 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:17:49.036 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:17:49.036 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:17:49.037 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:17:49.037 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:17:49.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:17:49.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:17:49.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:17:49.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:17:49.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:17:49.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:17:49.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:17:49.039 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:17:49.039 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:17:49.039 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:17:49.281 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:17:49.281 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:17:49.281 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:17:49.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:49.282 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:17:49.282 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:17:49.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:49.283 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:17:49.283 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:17:49.283 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:17:49.283 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:17:49.284 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:17:49.284 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:17:49.284 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:17:49.284 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:17:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:50 gibba032 bash[24004]: cluster 2021-03-10T16:17:48.791006+0000 mgr.gibba032.whopeo (mgr.24625) 7653 : cluster [DBG] pgmap v4358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:50 gibba032 bash[24004]: audit 2021-03-10T16:17:48.794712+0000 mgr.gibba032.whopeo (mgr.24625) 7654 : audit [DBG] from='client.59167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:50 gibba032 bash[24004]: audit 2021-03-10T16:17:49.019128+0000 mgr.gibba032.whopeo (mgr.24625) 7655 : audit [DBG] from='client.59173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:50 gibba032 bash[24004]: audit 2021-03-10T16:17:49.282119+0000 mon.a (mon.0) 2710 : audit [DBG] from='client.? 172.21.2.111:0/3749584516' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:50.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[30791]: cluster 2021-03-10T16:17:48.791006+0000 mgr.gibba032.whopeo (mgr.24625) 7653 : cluster [DBG] pgmap v4358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[30791]: audit 2021-03-10T16:17:48.794712+0000 mgr.gibba032.whopeo (mgr.24625) 7654 : audit [DBG] from='client.59167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[30791]: audit 2021-03-10T16:17:49.019128+0000 mgr.gibba032.whopeo (mgr.24625) 7655 : audit [DBG] from='client.59173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[30791]: audit 2021-03-10T16:17:49.282119+0000 mon.a (mon.0) 2710 : audit [DBG] from='client.? 172.21.2.111:0/3749584516' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:50.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[31774]: cluster 2021-03-10T16:17:48.791006+0000 mgr.gibba032.whopeo (mgr.24625) 7653 : cluster [DBG] pgmap v4358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[31774]: audit 2021-03-10T16:17:48.794712+0000 mgr.gibba032.whopeo (mgr.24625) 7654 : audit [DBG] from='client.59167 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[31774]: audit 2021-03-10T16:17:49.019128+0000 mgr.gibba032.whopeo (mgr.24625) 7655 : audit [DBG] from='client.59173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:17:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:50 gibba011 bash[31774]: audit 2021-03-10T16:17:49.282119+0000 mon.a (mon.0) 2710 : audit [DBG] from='client.? 172.21.2.111:0/3749584516' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:17:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:52 gibba032 bash[24004]: cluster 2021-03-10T16:17:50.792359+0000 mgr.gibba032.whopeo (mgr.24625) 7656 : cluster [DBG] pgmap v4359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:52 gibba011 bash[30791]: cluster 2021-03-10T16:17:50.792359+0000 mgr.gibba032.whopeo (mgr.24625) 7656 : cluster [DBG] pgmap v4359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:52 gibba011 bash[31774]: cluster 2021-03-10T16:17:50.792359+0000 mgr.gibba032.whopeo (mgr.24625) 7656 : cluster [DBG] pgmap v4359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:54 gibba032 bash[24004]: cluster 2021-03-10T16:17:52.794070+0000 mgr.gibba032.whopeo (mgr.24625) 7657 : cluster [DBG] pgmap v4360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:54 gibba011 bash[30791]: cluster 2021-03-10T16:17:52.794070+0000 mgr.gibba032.whopeo (mgr.24625) 7657 : cluster [DBG] pgmap v4360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:54.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:54 gibba011 bash[31774]: cluster 2021-03-10T16:17:52.794070+0000 mgr.gibba032.whopeo (mgr.24625) 7657 : cluster [DBG] pgmap v4360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:56 gibba032 bash[24004]: cluster 2021-03-10T16:17:54.795543+0000 mgr.gibba032.whopeo (mgr.24625) 7658 : cluster [DBG] pgmap v4361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:56 gibba011 bash[30791]: cluster 2021-03-10T16:17:54.795543+0000 mgr.gibba032.whopeo (mgr.24625) 7658 : cluster [DBG] pgmap v4361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:56 gibba011 bash[31774]: cluster 2021-03-10T16:17:54.795543+0000 mgr.gibba032.whopeo (mgr.24625) 7658 : cluster [DBG] pgmap v4361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:57.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:56.933261+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:17:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:56.933907+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:17:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:57.024222+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:17:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:57.024968+0000 mon.a (mon.0) 2714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:57.025603+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:57 gibba032 bash[24004]: audit 2021-03-10T16:17:57.026290+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:17:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:56.933261+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:17:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:56.933907+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:17:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:57.024222+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:17:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:57.024968+0000 mon.a (mon.0) 2714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:57.025603+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[30791]: audit 2021-03-10T16:17:57.026290+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:17:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:56.933261+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:17:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:56.933907+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:17:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:57.024222+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:17:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:57.024968+0000 mon.a (mon.0) 2714 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:57.025603+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:17:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:57 gibba011 bash[31774]: audit 2021-03-10T16:17:57.026290+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:17:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:17:58 gibba032 bash[24004]: cluster 2021-03-10T16:17:56.797255+0000 mgr.gibba032.whopeo (mgr.24625) 7659 : cluster [DBG] pgmap v4362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:17:58 gibba011 bash[30791]: cluster 2021-03-10T16:17:56.797255+0000 mgr.gibba032.whopeo (mgr.24625) 7659 : cluster [DBG] pgmap v4362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:17:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:17:58 gibba011 bash[31774]: cluster 2021-03-10T16:17:56.797255+0000 mgr.gibba032.whopeo (mgr.24625) 7659 : cluster [DBG] pgmap v4362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:00 gibba032 bash[24004]: cluster 2021-03-10T16:17:58.798839+0000 mgr.gibba032.whopeo (mgr.24625) 7660 : cluster [DBG] pgmap v4363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:00 gibba011 bash[30791]: cluster 2021-03-10T16:17:58.798839+0000 mgr.gibba032.whopeo (mgr.24625) 7660 : cluster [DBG] pgmap v4363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:00 gibba011 bash[31774]: cluster 2021-03-10T16:17:58.798839+0000 mgr.gibba032.whopeo (mgr.24625) 7660 : cluster [DBG] pgmap v4363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cluster 2021-03-10T16:18:00.800327+0000 mgr.gibba032.whopeo (mgr.24625) 7661 : cluster [DBG] pgmap v4364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.113427+0000 mgr.gibba032.whopeo (mgr.24625) 7662 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:18:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.114971+0000 mgr.gibba032.whopeo (mgr.24625) 7663 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:18:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.330382+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:18:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.330714+0000 mgr.gibba032.whopeo (mgr.24625) 7664 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:18:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.333069+0000 mgr.gibba032.whopeo (mgr.24625) 7665 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:18:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.333903+0000 mon.a (mon.0) 2718 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:18:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.334585+0000 mgr.gibba032.whopeo (mgr.24625) 7666 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:18:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.334673+0000 mgr.gibba032.whopeo (mgr.24625) 7667 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:18:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.335689+0000 mon.a (mon.0) 2719 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.336107+0000 mgr.gibba032.whopeo (mgr.24625) 7668 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:18:02.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.336182+0000 mgr.gibba032.whopeo (mgr.24625) 7669 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:18:02.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.337247+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.337655+0000 mgr.gibba032.whopeo (mgr.24625) 7670 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:18:02.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.338727+0000 mon.a (mon.0) 2721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.339464+0000 mgr.gibba032.whopeo (mgr.24625) 7671 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:18:02.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.339506+0000 mgr.gibba032.whopeo (mgr.24625) 7672 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:18:02.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.341311+0000 mgr.gibba032.whopeo (mgr.24625) 7673 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:18:02.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.341667+0000 mgr.gibba032.whopeo (mgr.24625) 7674 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:18:02.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.342226+0000 mon.a (mon.0) 2722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.343384+0000 mgr.gibba032.whopeo (mgr.24625) 7675 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:18:02.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.343426+0000 mgr.gibba032.whopeo (mgr.24625) 7676 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:18:02.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: cephadm 2021-03-10T16:18:01.343551+0000 mgr.gibba032.whopeo (mgr.24625) 7677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:02.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:02 gibba032 bash[24004]: audit 2021-03-10T16:18:01.344239+0000 mon.a (mon.0) 2723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cluster 2021-03-10T16:18:00.800327+0000 mgr.gibba032.whopeo (mgr.24625) 7661 : cluster [DBG] pgmap v4364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.113427+0000 mgr.gibba032.whopeo (mgr.24625) 7662 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:18:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.114971+0000 mgr.gibba032.whopeo (mgr.24625) 7663 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:18:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.330382+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:18:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.330714+0000 mgr.gibba032.whopeo (mgr.24625) 7664 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:18:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.333069+0000 mgr.gibba032.whopeo (mgr.24625) 7665 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:18:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.333903+0000 mon.a (mon.0) 2718 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:18:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.334585+0000 mgr.gibba032.whopeo (mgr.24625) 7666 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:18:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.334673+0000 mgr.gibba032.whopeo (mgr.24625) 7667 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:18:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.335689+0000 mon.a (mon.0) 2719 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.336107+0000 mgr.gibba032.whopeo (mgr.24625) 7668 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:18:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.336182+0000 mgr.gibba032.whopeo (mgr.24625) 7669 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:18:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.337247+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.337655+0000 mgr.gibba032.whopeo (mgr.24625) 7670 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:18:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.338727+0000 mon.a (mon.0) 2721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.339464+0000 mgr.gibba032.whopeo (mgr.24625) 7671 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:18:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.339506+0000 mgr.gibba032.whopeo (mgr.24625) 7672 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:18:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.341311+0000 mgr.gibba032.whopeo (mgr.24625) 7673 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:18:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.341667+0000 mgr.gibba032.whopeo (mgr.24625) 7674 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:18:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.342226+0000 mon.a (mon.0) 2722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.343384+0000 mgr.gibba032.whopeo (mgr.24625) 7675 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:18:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.343426+0000 mgr.gibba032.whopeo (mgr.24625) 7676 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:18:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: cephadm 2021-03-10T16:18:01.343551+0000 mgr.gibba032.whopeo (mgr.24625) 7677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[30791]: audit 2021-03-10T16:18:01.344239+0000 mon.a (mon.0) 2723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cluster 2021-03-10T16:18:00.800327+0000 mgr.gibba032.whopeo (mgr.24625) 7661 : cluster [DBG] pgmap v4364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.113427+0000 mgr.gibba032.whopeo (mgr.24625) 7662 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:18:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.114971+0000 mgr.gibba032.whopeo (mgr.24625) 7663 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:18:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.330382+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:18:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.330714+0000 mgr.gibba032.whopeo (mgr.24625) 7664 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:18:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.333069+0000 mgr.gibba032.whopeo (mgr.24625) 7665 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:18:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.333903+0000 mon.a (mon.0) 2718 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:18:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.334585+0000 mgr.gibba032.whopeo (mgr.24625) 7666 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:18:02.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.334673+0000 mgr.gibba032.whopeo (mgr.24625) 7667 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:18:02.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.335689+0000 mon.a (mon.0) 2719 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.336107+0000 mgr.gibba032.whopeo (mgr.24625) 7668 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:18:02.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.336182+0000 mgr.gibba032.whopeo (mgr.24625) 7669 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:18:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.337247+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.337655+0000 mgr.gibba032.whopeo (mgr.24625) 7670 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:18:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.338727+0000 mon.a (mon.0) 2721 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.339464+0000 mgr.gibba032.whopeo (mgr.24625) 7671 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:18:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.339506+0000 mgr.gibba032.whopeo (mgr.24625) 7672 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:18:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.341311+0000 mgr.gibba032.whopeo (mgr.24625) 7673 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:18:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.341667+0000 mgr.gibba032.whopeo (mgr.24625) 7674 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:18:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.342226+0000 mon.a (mon.0) 2722 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.343384+0000 mgr.gibba032.whopeo (mgr.24625) 7675 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:18:02.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.343426+0000 mgr.gibba032.whopeo (mgr.24625) 7676 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:18:02.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: cephadm 2021-03-10T16:18:01.343551+0000 mgr.gibba032.whopeo (mgr.24625) 7677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:02.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:02 gibba011 bash[31774]: audit 2021-03-10T16:18:01.344239+0000 mon.a (mon.0) 2723 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:04 gibba032 bash[24004]: cluster 2021-03-10T16:18:02.801930+0000 mgr.gibba032.whopeo (mgr.24625) 7678 : cluster [DBG] pgmap v4365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:04 gibba011 bash[30791]: cluster 2021-03-10T16:18:02.801930+0000 mgr.gibba032.whopeo (mgr.24625) 7678 : cluster [DBG] pgmap v4365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:04 gibba011 bash[31774]: cluster 2021-03-10T16:18:02.801930+0000 mgr.gibba032.whopeo (mgr.24625) 7678 : cluster [DBG] pgmap v4365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:06 gibba032 bash[24004]: cluster 2021-03-10T16:18:04.803169+0000 mgr.gibba032.whopeo (mgr.24625) 7679 : cluster [DBG] pgmap v4366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:06 gibba011 bash[30791]: cluster 2021-03-10T16:18:04.803169+0000 mgr.gibba032.whopeo (mgr.24625) 7679 : cluster [DBG] pgmap v4366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:06 gibba011 bash[31774]: cluster 2021-03-10T16:18:04.803169+0000 mgr.gibba032.whopeo (mgr.24625) 7679 : cluster [DBG] pgmap v4366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:08 gibba032 bash[24004]: cluster 2021-03-10T16:18:06.804870+0000 mgr.gibba032.whopeo (mgr.24625) 7680 : cluster [DBG] pgmap v4367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:08 gibba011 bash[30791]: cluster 2021-03-10T16:18:06.804870+0000 mgr.gibba032.whopeo (mgr.24625) 7680 : cluster [DBG] pgmap v4367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:08 gibba011 bash[31774]: cluster 2021-03-10T16:18:06.804870+0000 mgr.gibba032.whopeo (mgr.24625) 7680 : cluster [DBG] pgmap v4367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:10 gibba032 bash[24004]: cluster 2021-03-10T16:18:08.806242+0000 mgr.gibba032.whopeo (mgr.24625) 7681 : cluster [DBG] pgmap v4368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:10 gibba011 bash[30791]: cluster 2021-03-10T16:18:08.806242+0000 mgr.gibba032.whopeo (mgr.24625) 7681 : cluster [DBG] pgmap v4368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:10 gibba011 bash[31774]: cluster 2021-03-10T16:18:08.806242+0000 mgr.gibba032.whopeo (mgr.24625) 7681 : cluster [DBG] pgmap v4368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:12 gibba032 bash[24004]: cluster 2021-03-10T16:18:10.807613+0000 mgr.gibba032.whopeo (mgr.24625) 7682 : cluster [DBG] pgmap v4369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:12 gibba011 bash[30791]: cluster 2021-03-10T16:18:10.807613+0000 mgr.gibba032.whopeo (mgr.24625) 7682 : cluster [DBG] pgmap v4369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:12 gibba011 bash[31774]: cluster 2021-03-10T16:18:10.807613+0000 mgr.gibba032.whopeo (mgr.24625) 7682 : cluster [DBG] pgmap v4369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:14 gibba032 bash[24004]: cluster 2021-03-10T16:18:12.809135+0000 mgr.gibba032.whopeo (mgr.24625) 7683 : cluster [DBG] pgmap v4370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:14 gibba011 bash[30791]: cluster 2021-03-10T16:18:12.809135+0000 mgr.gibba032.whopeo (mgr.24625) 7683 : cluster [DBG] pgmap v4370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:14 gibba011 bash[31774]: cluster 2021-03-10T16:18:12.809135+0000 mgr.gibba032.whopeo (mgr.24625) 7683 : cluster [DBG] pgmap v4370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:16 gibba032 bash[24004]: cluster 2021-03-10T16:18:14.810611+0000 mgr.gibba032.whopeo (mgr.24625) 7684 : cluster [DBG] pgmap v4371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:16 gibba011 bash[30791]: cluster 2021-03-10T16:18:14.810611+0000 mgr.gibba032.whopeo (mgr.24625) 7684 : cluster [DBG] pgmap v4371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:16 gibba011 bash[31774]: cluster 2021-03-10T16:18:14.810611+0000 mgr.gibba032.whopeo (mgr.24625) 7684 : cluster [DBG] pgmap v4371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:17 gibba032 bash[24004]: cephadm 2021-03-10T16:18:16.359306+0000 mgr.gibba032.whopeo (mgr.24625) 7685 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:17.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:17 gibba011 bash[30791]: cephadm 2021-03-10T16:18:16.359306+0000 mgr.gibba032.whopeo (mgr.24625) 7685 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:17.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:17 gibba011 bash[31774]: cephadm 2021-03-10T16:18:16.359306+0000 mgr.gibba032.whopeo (mgr.24625) 7685 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:18 gibba032 bash[24004]: cluster 2021-03-10T16:18:16.812400+0000 mgr.gibba032.whopeo (mgr.24625) 7686 : cluster [DBG] pgmap v4372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:18 gibba011 bash[30791]: cluster 2021-03-10T16:18:16.812400+0000 mgr.gibba032.whopeo (mgr.24625) 7686 : cluster [DBG] pgmap v4372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:18 gibba011 bash[31774]: cluster 2021-03-10T16:18:16.812400+0000 mgr.gibba032.whopeo (mgr.24625) 7686 : cluster [DBG] pgmap v4372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:19.521 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:18:19.750 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:18:19.750 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:18:19.751 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:18:19.751 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:18:19.751 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:18:19.751 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:18:19.752 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:18:19.752 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:18:19.752 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:18:19.752 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:18:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:18:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:18:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:18:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:18:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:18:19.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:18:19.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:18:19.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:18:19.754 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:18:19.755 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:18:19.755 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:18:19.998 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:18:19.999 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:18:19.999 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:18:19.999 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:19.999 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:18:20.000 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:18:20.000 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:20.000 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:18:20.000 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:18:20.001 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:20.001 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:18:20.001 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:18:20.001 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:18:20.001 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:18:20.002 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:18:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:20 gibba032 bash[24004]: cluster 2021-03-10T16:18:18.813846+0000 mgr.gibba032.whopeo (mgr.24625) 7687 : cluster [DBG] pgmap v4373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:20 gibba032 bash[24004]: audit 2021-03-10T16:18:20.000300+0000 mon.a (mon.0) 2724 : audit [DBG] from='client.? 172.21.2.111:0/2254811992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:20 gibba011 bash[30791]: cluster 2021-03-10T16:18:18.813846+0000 mgr.gibba032.whopeo (mgr.24625) 7687 : cluster [DBG] pgmap v4373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:20 gibba011 bash[30791]: audit 2021-03-10T16:18:20.000300+0000 mon.a (mon.0) 2724 : audit [DBG] from='client.? 172.21.2.111:0/2254811992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:20.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:20 gibba011 bash[31774]: cluster 2021-03-10T16:18:18.813846+0000 mgr.gibba032.whopeo (mgr.24625) 7687 : cluster [DBG] pgmap v4373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:20.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:20 gibba011 bash[31774]: audit 2021-03-10T16:18:20.000300+0000 mon.a (mon.0) 2724 : audit [DBG] from='client.? 172.21.2.111:0/2254811992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:21.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:21 gibba032 bash[24004]: audit 2021-03-10T16:18:19.511917+0000 mgr.gibba032.whopeo (mgr.24625) 7688 : audit [DBG] from='client.49203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:21 gibba032 bash[24004]: audit 2021-03-10T16:18:19.737246+0000 mgr.gibba032.whopeo (mgr.24625) 7689 : audit [DBG] from='client.59191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:21.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:21 gibba011 bash[30791]: audit 2021-03-10T16:18:19.511917+0000 mgr.gibba032.whopeo (mgr.24625) 7688 : audit [DBG] from='client.49203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:21 gibba011 bash[30791]: audit 2021-03-10T16:18:19.737246+0000 mgr.gibba032.whopeo (mgr.24625) 7689 : audit [DBG] from='client.59191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:21.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:21 gibba011 bash[31774]: audit 2021-03-10T16:18:19.511917+0000 mgr.gibba032.whopeo (mgr.24625) 7688 : audit [DBG] from='client.49203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:21.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:21 gibba011 bash[31774]: audit 2021-03-10T16:18:19.737246+0000 mgr.gibba032.whopeo (mgr.24625) 7689 : audit [DBG] from='client.59191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:22 gibba032 bash[24004]: cluster 2021-03-10T16:18:20.815292+0000 mgr.gibba032.whopeo (mgr.24625) 7690 : cluster [DBG] pgmap v4374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:22 gibba011 bash[30791]: cluster 2021-03-10T16:18:20.815292+0000 mgr.gibba032.whopeo (mgr.24625) 7690 : cluster [DBG] pgmap v4374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:22 gibba011 bash[31774]: cluster 2021-03-10T16:18:20.815292+0000 mgr.gibba032.whopeo (mgr.24625) 7690 : cluster [DBG] pgmap v4374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:24 gibba032 bash[24004]: cluster 2021-03-10T16:18:22.816934+0000 mgr.gibba032.whopeo (mgr.24625) 7691 : cluster [DBG] pgmap v4375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:24 gibba011 bash[30791]: cluster 2021-03-10T16:18:22.816934+0000 mgr.gibba032.whopeo (mgr.24625) 7691 : cluster [DBG] pgmap v4375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:24 gibba011 bash[31774]: cluster 2021-03-10T16:18:22.816934+0000 mgr.gibba032.whopeo (mgr.24625) 7691 : cluster [DBG] pgmap v4375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:26 gibba032 bash[24004]: cluster 2021-03-10T16:18:24.818197+0000 mgr.gibba032.whopeo (mgr.24625) 7692 : cluster [DBG] pgmap v4376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:26 gibba011 bash[30791]: cluster 2021-03-10T16:18:24.818197+0000 mgr.gibba032.whopeo (mgr.24625) 7692 : cluster [DBG] pgmap v4376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:26 gibba011 bash[31774]: cluster 2021-03-10T16:18:24.818197+0000 mgr.gibba032.whopeo (mgr.24625) 7692 : cluster [DBG] pgmap v4376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:28 gibba032 bash[24004]: cluster 2021-03-10T16:18:26.819669+0000 mgr.gibba032.whopeo (mgr.24625) 7693 : cluster [DBG] pgmap v4377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:28 gibba011 bash[30791]: cluster 2021-03-10T16:18:26.819669+0000 mgr.gibba032.whopeo (mgr.24625) 7693 : cluster [DBG] pgmap v4377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:28 gibba011 bash[31774]: cluster 2021-03-10T16:18:26.819669+0000 mgr.gibba032.whopeo (mgr.24625) 7693 : cluster [DBG] pgmap v4377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:30 gibba032 bash[24004]: cluster 2021-03-10T16:18:28.821177+0000 mgr.gibba032.whopeo (mgr.24625) 7694 : cluster [DBG] pgmap v4378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:30 gibba011 bash[30791]: cluster 2021-03-10T16:18:28.821177+0000 mgr.gibba032.whopeo (mgr.24625) 7694 : cluster [DBG] pgmap v4378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:30 gibba011 bash[31774]: cluster 2021-03-10T16:18:28.821177+0000 mgr.gibba032.whopeo (mgr.24625) 7694 : cluster [DBG] pgmap v4378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:32 gibba032 bash[24004]: cluster 2021-03-10T16:18:30.822567+0000 mgr.gibba032.whopeo (mgr.24625) 7695 : cluster [DBG] pgmap v4379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:32 gibba032 bash[24004]: cephadm 2021-03-10T16:18:31.375090+0000 mgr.gibba032.whopeo (mgr.24625) 7696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:32 gibba011 bash[30791]: cluster 2021-03-10T16:18:30.822567+0000 mgr.gibba032.whopeo (mgr.24625) 7695 : cluster [DBG] pgmap v4379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:32 gibba011 bash[30791]: cephadm 2021-03-10T16:18:31.375090+0000 mgr.gibba032.whopeo (mgr.24625) 7696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:32 gibba011 bash[31774]: cluster 2021-03-10T16:18:30.822567+0000 mgr.gibba032.whopeo (mgr.24625) 7695 : cluster [DBG] pgmap v4379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:32 gibba011 bash[31774]: cephadm 2021-03-10T16:18:31.375090+0000 mgr.gibba032.whopeo (mgr.24625) 7696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:34 gibba032 bash[24004]: cluster 2021-03-10T16:18:32.824286+0000 mgr.gibba032.whopeo (mgr.24625) 7697 : cluster [DBG] pgmap v4380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:34 gibba011 bash[30791]: cluster 2021-03-10T16:18:32.824286+0000 mgr.gibba032.whopeo (mgr.24625) 7697 : cluster [DBG] pgmap v4380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:34 gibba011 bash[31774]: cluster 2021-03-10T16:18:32.824286+0000 mgr.gibba032.whopeo (mgr.24625) 7697 : cluster [DBG] pgmap v4380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:36 gibba032 bash[24004]: cluster 2021-03-10T16:18:34.825729+0000 mgr.gibba032.whopeo (mgr.24625) 7698 : cluster [DBG] pgmap v4381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:36 gibba011 bash[30791]: cluster 2021-03-10T16:18:34.825729+0000 mgr.gibba032.whopeo (mgr.24625) 7698 : cluster [DBG] pgmap v4381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:36 gibba011 bash[31774]: cluster 2021-03-10T16:18:34.825729+0000 mgr.gibba032.whopeo (mgr.24625) 7698 : cluster [DBG] pgmap v4381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:38 gibba011 bash[30791]: cluster 2021-03-10T16:18:36.827342+0000 mgr.gibba032.whopeo (mgr.24625) 7699 : cluster [DBG] pgmap v4382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:38 gibba011 bash[31774]: cluster 2021-03-10T16:18:36.827342+0000 mgr.gibba032.whopeo (mgr.24625) 7699 : cluster [DBG] pgmap v4382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:38 gibba032 bash[24004]: cluster 2021-03-10T16:18:36.827342+0000 mgr.gibba032.whopeo (mgr.24625) 7699 : cluster [DBG] pgmap v4382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:40 gibba011 bash[30791]: cluster 2021-03-10T16:18:38.828981+0000 mgr.gibba032.whopeo (mgr.24625) 7700 : cluster [DBG] pgmap v4383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:40 gibba011 bash[31774]: cluster 2021-03-10T16:18:38.828981+0000 mgr.gibba032.whopeo (mgr.24625) 7700 : cluster [DBG] pgmap v4383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:40 gibba032 bash[24004]: cluster 2021-03-10T16:18:38.828981+0000 mgr.gibba032.whopeo (mgr.24625) 7700 : cluster [DBG] pgmap v4383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:42 gibba011 bash[30791]: cluster 2021-03-10T16:18:40.830332+0000 mgr.gibba032.whopeo (mgr.24625) 7701 : cluster [DBG] pgmap v4384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:42.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:42 gibba011 bash[31774]: cluster 2021-03-10T16:18:40.830332+0000 mgr.gibba032.whopeo (mgr.24625) 7701 : cluster [DBG] pgmap v4384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:42 gibba032 bash[24004]: cluster 2021-03-10T16:18:40.830332+0000 mgr.gibba032.whopeo (mgr.24625) 7701 : cluster [DBG] pgmap v4384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:43.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:43 gibba011 bash[30791]: cluster 2021-03-10T16:18:42.831976+0000 mgr.gibba032.whopeo (mgr.24625) 7702 : cluster [DBG] pgmap v4385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:43.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:43 gibba011 bash[31774]: cluster 2021-03-10T16:18:42.831976+0000 mgr.gibba032.whopeo (mgr.24625) 7702 : cluster [DBG] pgmap v4385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:43 gibba032 bash[24004]: cluster 2021-03-10T16:18:42.831976+0000 mgr.gibba032.whopeo (mgr.24625) 7702 : cluster [DBG] pgmap v4385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:46 gibba032 bash[24004]: cluster 2021-03-10T16:18:44.833548+0000 mgr.gibba032.whopeo (mgr.24625) 7703 : cluster [DBG] pgmap v4386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:46 gibba011 bash[30791]: cluster 2021-03-10T16:18:44.833548+0000 mgr.gibba032.whopeo (mgr.24625) 7703 : cluster [DBG] pgmap v4386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:46 gibba011 bash[31774]: cluster 2021-03-10T16:18:44.833548+0000 mgr.gibba032.whopeo (mgr.24625) 7703 : cluster [DBG] pgmap v4386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:47.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:47 gibba032 bash[24004]: cephadm 2021-03-10T16:18:46.391113+0000 mgr.gibba032.whopeo (mgr.24625) 7704 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:47.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:47 gibba011 bash[30791]: cephadm 2021-03-10T16:18:46.391113+0000 mgr.gibba032.whopeo (mgr.24625) 7704 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:47.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:47 gibba011 bash[31774]: cephadm 2021-03-10T16:18:46.391113+0000 mgr.gibba032.whopeo (mgr.24625) 7704 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:18:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:48 gibba032 bash[24004]: cluster 2021-03-10T16:18:46.835139+0000 mgr.gibba032.whopeo (mgr.24625) 7705 : cluster [DBG] pgmap v4387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:48 gibba011 bash[30791]: cluster 2021-03-10T16:18:46.835139+0000 mgr.gibba032.whopeo (mgr.24625) 7705 : cluster [DBG] pgmap v4387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:48 gibba011 bash[31774]: cluster 2021-03-10T16:18:46.835139+0000 mgr.gibba032.whopeo (mgr.24625) 7705 : cluster [DBG] pgmap v4387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:50.238 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:18:50.465 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:18:50.465 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:18:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:18:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:18:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:18:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:18:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:18:50.467 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:18:50.467 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:18:50.467 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:18:50.467 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:18:50.468 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:18:50.468 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:18:50.468 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:18:50.468 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:18:50.468 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:18:50.469 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:18:50.469 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:18:50.469 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:18:50.469 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:18:50.470 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:18:50.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:50 gibba032 bash[24004]: cluster 2021-03-10T16:18:48.836679+0000 mgr.gibba032.whopeo (mgr.24625) 7706 : cluster [DBG] pgmap v4388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:50 gibba011 bash[30791]: cluster 2021-03-10T16:18:48.836679+0000 mgr.gibba032.whopeo (mgr.24625) 7706 : cluster [DBG] pgmap v4388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:50.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:50 gibba011 bash[31774]: cluster 2021-03-10T16:18:48.836679+0000 mgr.gibba032.whopeo (mgr.24625) 7706 : cluster [DBG] pgmap v4388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:50.718 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:18:50.718 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:18:50.718 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:18:50.719 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:50.719 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:18:50.719 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:18:50.719 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:50.719 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:18:50.720 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:18:50.720 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:18:50.720 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:18:50.720 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:18:50.721 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:18:50.721 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:18:50.721 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:18:51.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:51 gibba032 bash[24004]: audit 2021-03-10T16:18:50.228952+0000 mgr.gibba032.whopeo (mgr.24625) 7707 : audit [DBG] from='client.59203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:51 gibba032 bash[24004]: audit 2021-03-10T16:18:50.452296+0000 mgr.gibba032.whopeo (mgr.24625) 7708 : audit [DBG] from='client.49227 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:51 gibba032 bash[24004]: audit 2021-03-10T16:18:50.719215+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.2.111:0/2633484164' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:51.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[30791]: audit 2021-03-10T16:18:50.228952+0000 mgr.gibba032.whopeo (mgr.24625) 7707 : audit [DBG] from='client.59203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[30791]: audit 2021-03-10T16:18:50.452296+0000 mgr.gibba032.whopeo (mgr.24625) 7708 : audit [DBG] from='client.49227 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[30791]: audit 2021-03-10T16:18:50.719215+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.2.111:0/2633484164' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:51.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[31774]: audit 2021-03-10T16:18:50.228952+0000 mgr.gibba032.whopeo (mgr.24625) 7707 : audit [DBG] from='client.59203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[31774]: audit 2021-03-10T16:18:50.452296+0000 mgr.gibba032.whopeo (mgr.24625) 7708 : audit [DBG] from='client.49227 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:18:51.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:51 gibba011 bash[31774]: audit 2021-03-10T16:18:50.719215+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.2.111:0/2633484164' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:18:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:52 gibba032 bash[24004]: cluster 2021-03-10T16:18:50.837995+0000 mgr.gibba032.whopeo (mgr.24625) 7709 : cluster [DBG] pgmap v4389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:52 gibba011 bash[30791]: cluster 2021-03-10T16:18:50.837995+0000 mgr.gibba032.whopeo (mgr.24625) 7709 : cluster [DBG] pgmap v4389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:52 gibba011 bash[31774]: cluster 2021-03-10T16:18:50.837995+0000 mgr.gibba032.whopeo (mgr.24625) 7709 : cluster [DBG] pgmap v4389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:54 gibba032 bash[24004]: cluster 2021-03-10T16:18:52.839638+0000 mgr.gibba032.whopeo (mgr.24625) 7710 : cluster [DBG] pgmap v4390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:54 gibba011 bash[30791]: cluster 2021-03-10T16:18:52.839638+0000 mgr.gibba032.whopeo (mgr.24625) 7710 : cluster [DBG] pgmap v4390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:54 gibba011 bash[31774]: cluster 2021-03-10T16:18:52.839638+0000 mgr.gibba032.whopeo (mgr.24625) 7710 : cluster [DBG] pgmap v4390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:56 gibba011 bash[30791]: cluster 2021-03-10T16:18:54.841064+0000 mgr.gibba032.whopeo (mgr.24625) 7711 : cluster [DBG] pgmap v4391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:56 gibba011 bash[31774]: cluster 2021-03-10T16:18:54.841064+0000 mgr.gibba032.whopeo (mgr.24625) 7711 : cluster [DBG] pgmap v4391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:56 gibba032 bash[24004]: cluster 2021-03-10T16:18:54.841064+0000 mgr.gibba032.whopeo (mgr.24625) 7711 : cluster [DBG] pgmap v4391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:56.940670+0000 mon.a (mon.0) 2726 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:18:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:56.941333+0000 mon.a (mon.0) 2727 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:18:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:57.034028+0000 mon.a (mon.0) 2728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:18:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:57.034821+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:57.035534+0000 mon.a (mon.0) 2730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[30791]: audit 2021-03-10T16:18:57.036242+0000 mon.a (mon.0) 2731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:18:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:56.940670+0000 mon.a (mon.0) 2726 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:18:57.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:56.941333+0000 mon.a (mon.0) 2727 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:18:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:57.034028+0000 mon.a (mon.0) 2728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:18:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:57.034821+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:57.035534+0000 mon.a (mon.0) 2730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:57 gibba011 bash[31774]: audit 2021-03-10T16:18:57.036242+0000 mon.a (mon.0) 2731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:18:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:56.940670+0000 mon.a (mon.0) 2726 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:18:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:56.941333+0000 mon.a (mon.0) 2727 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:18:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:57.034028+0000 mon.a (mon.0) 2728 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:18:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:57.034821+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:57.035534+0000 mon.a (mon.0) 2730 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:18:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:57 gibba032 bash[24004]: audit 2021-03-10T16:18:57.036242+0000 mon.a (mon.0) 2731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:18:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:18:58 gibba011 bash[30791]: cluster 2021-03-10T16:18:56.842790+0000 mgr.gibba032.whopeo (mgr.24625) 7712 : cluster [DBG] pgmap v4392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:18:58 gibba011 bash[31774]: cluster 2021-03-10T16:18:56.842790+0000 mgr.gibba032.whopeo (mgr.24625) 7712 : cluster [DBG] pgmap v4392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:18:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:18:58 gibba032 bash[24004]: cluster 2021-03-10T16:18:56.842790+0000 mgr.gibba032.whopeo (mgr.24625) 7712 : cluster [DBG] pgmap v4392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:00 gibba011 bash[30791]: cluster 2021-03-10T16:18:58.844456+0000 mgr.gibba032.whopeo (mgr.24625) 7713 : cluster [DBG] pgmap v4393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:00 gibba011 bash[31774]: cluster 2021-03-10T16:18:58.844456+0000 mgr.gibba032.whopeo (mgr.24625) 7713 : cluster [DBG] pgmap v4393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:00 gibba032 bash[24004]: cluster 2021-03-10T16:18:58.844456+0000 mgr.gibba032.whopeo (mgr.24625) 7713 : cluster [DBG] pgmap v4393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[30791]: cluster 2021-03-10T16:19:00.845799+0000 mgr.gibba032.whopeo (mgr.24625) 7714 : cluster [DBG] pgmap v4394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[30791]: cephadm 2021-03-10T16:19:01.408692+0000 mgr.gibba032.whopeo (mgr.24625) 7715 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:19:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[30791]: audit 2021-03-10T16:19:01.411511+0000 mon.a (mon.0) 2732 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:19:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[31774]: cluster 2021-03-10T16:19:00.845799+0000 mgr.gibba032.whopeo (mgr.24625) 7714 : cluster [DBG] pgmap v4394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[31774]: cephadm 2021-03-10T16:19:01.408692+0000 mgr.gibba032.whopeo (mgr.24625) 7715 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:19:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:02 gibba011 bash[31774]: audit 2021-03-10T16:19:01.411511+0000 mon.a (mon.0) 2732 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:19:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:02 gibba032 bash[24004]: cluster 2021-03-10T16:19:00.845799+0000 mgr.gibba032.whopeo (mgr.24625) 7714 : cluster [DBG] pgmap v4394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:02 gibba032 bash[24004]: cephadm 2021-03-10T16:19:01.408692+0000 mgr.gibba032.whopeo (mgr.24625) 7715 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:19:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:02 gibba032 bash[24004]: audit 2021-03-10T16:19:01.411511+0000 mon.a (mon.0) 2732 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:19:04.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:04 gibba011 bash[30791]: cluster 2021-03-10T16:19:02.847559+0000 mgr.gibba032.whopeo (mgr.24625) 7716 : cluster [DBG] pgmap v4395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:04 gibba011 bash[31774]: cluster 2021-03-10T16:19:02.847559+0000 mgr.gibba032.whopeo (mgr.24625) 7716 : cluster [DBG] pgmap v4395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:04 gibba032 bash[24004]: cluster 2021-03-10T16:19:02.847559+0000 mgr.gibba032.whopeo (mgr.24625) 7716 : cluster [DBG] pgmap v4395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[30791]: cluster 2021-03-10T16:19:04.849810+0000 mgr.gibba032.whopeo (mgr.24625) 7717 : cluster [DBG] pgmap v4396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[30791]: audit 2021-03-10T16:19:06.090795+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:19:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[30791]: audit 2021-03-10T16:19:06.266274+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[31774]: cluster 2021-03-10T16:19:04.849810+0000 mgr.gibba032.whopeo (mgr.24625) 7717 : cluster [DBG] pgmap v4396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[31774]: audit 2021-03-10T16:19:06.090795+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:19:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:06 gibba011 bash[31774]: audit 2021-03-10T16:19:06.266274+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:06 gibba032 bash[24004]: cluster 2021-03-10T16:19:04.849810+0000 mgr.gibba032.whopeo (mgr.24625) 7717 : cluster [DBG] pgmap v4396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:06 gibba032 bash[24004]: audit 2021-03-10T16:19:06.090795+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:19:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:06 gibba032 bash[24004]: audit 2021-03-10T16:19:06.266274+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.265641+0000 mgr.gibba032.whopeo (mgr.24625) 7718 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:19:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.462754+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.468396+0000 mgr.gibba032.whopeo (mgr.24625) 7719 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:19:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.471650+0000 mgr.gibba032.whopeo (mgr.24625) 7720 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:19:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.471829+0000 mon.a (mon.0) 2736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:19:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.474619+0000 mon.a (mon.0) 2737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.474971+0000 mgr.gibba032.whopeo (mgr.24625) 7721 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:19:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.475018+0000 mgr.gibba032.whopeo (mgr.24625) 7722 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:19:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.478061+0000 mon.a (mon.0) 2738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.478113+0000 mgr.gibba032.whopeo (mgr.24625) 7723 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:19:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.478157+0000 mgr.gibba032.whopeo (mgr.24625) 7724 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:19:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.481126+0000 mon.a (mon.0) 2739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.481361+0000 mgr.gibba032.whopeo (mgr.24625) 7725 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:19:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.483366+0000 mgr.gibba032.whopeo (mgr.24625) 7726 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:19:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.483409+0000 mgr.gibba032.whopeo (mgr.24625) 7727 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:19:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.486228+0000 mon.a (mon.0) 2740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.486272+0000 mgr.gibba032.whopeo (mgr.24625) 7728 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:19:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.486795+0000 mgr.gibba032.whopeo (mgr.24625) 7729 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:19:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: audit 2021-03-10T16:19:06.489452+0000 mon.a (mon.0) 2741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.489514+0000 mgr.gibba032.whopeo (mgr.24625) 7730 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:19:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.489568+0000 mgr.gibba032.whopeo (mgr.24625) 7731 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:19:07.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:07 gibba032 bash[24004]: cephadm 2021-03-10T16:19:06.489720+0000 mgr.gibba032.whopeo (mgr.24625) 7732 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.265641+0000 mgr.gibba032.whopeo (mgr.24625) 7718 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:19:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.462754+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.468396+0000 mgr.gibba032.whopeo (mgr.24625) 7719 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:19:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.471650+0000 mgr.gibba032.whopeo (mgr.24625) 7720 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:19:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.471829+0000 mon.a (mon.0) 2736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:19:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.474619+0000 mon.a (mon.0) 2737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.474971+0000 mgr.gibba032.whopeo (mgr.24625) 7721 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:19:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.475018+0000 mgr.gibba032.whopeo (mgr.24625) 7722 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:19:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.478061+0000 mon.a (mon.0) 2738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.478113+0000 mgr.gibba032.whopeo (mgr.24625) 7723 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:19:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.478157+0000 mgr.gibba032.whopeo (mgr.24625) 7724 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:19:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.481126+0000 mon.a (mon.0) 2739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.481361+0000 mgr.gibba032.whopeo (mgr.24625) 7725 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:19:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.483366+0000 mgr.gibba032.whopeo (mgr.24625) 7726 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:19:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.483409+0000 mgr.gibba032.whopeo (mgr.24625) 7727 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:19:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.486228+0000 mon.a (mon.0) 2740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.486272+0000 mgr.gibba032.whopeo (mgr.24625) 7728 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:19:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.486795+0000 mgr.gibba032.whopeo (mgr.24625) 7729 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:19:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: audit 2021-03-10T16:19:06.489452+0000 mon.a (mon.0) 2741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.489514+0000 mgr.gibba032.whopeo (mgr.24625) 7730 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:19:07.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.489568+0000 mgr.gibba032.whopeo (mgr.24625) 7731 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:19:07.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[30791]: cephadm 2021-03-10T16:19:06.489720+0000 mgr.gibba032.whopeo (mgr.24625) 7732 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:07.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.265641+0000 mgr.gibba032.whopeo (mgr.24625) 7718 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:19:07.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.462754+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:19:07.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.468396+0000 mgr.gibba032.whopeo (mgr.24625) 7719 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:19:07.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.471650+0000 mgr.gibba032.whopeo (mgr.24625) 7720 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:19:07.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.471829+0000 mon.a (mon.0) 2736 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:19:07.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.474619+0000 mon.a (mon.0) 2737 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.474971+0000 mgr.gibba032.whopeo (mgr.24625) 7721 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:19:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.475018+0000 mgr.gibba032.whopeo (mgr.24625) 7722 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:19:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.478061+0000 mon.a (mon.0) 2738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.478113+0000 mgr.gibba032.whopeo (mgr.24625) 7723 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:19:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.478157+0000 mgr.gibba032.whopeo (mgr.24625) 7724 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:19:07.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.481126+0000 mon.a (mon.0) 2739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.481361+0000 mgr.gibba032.whopeo (mgr.24625) 7725 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:19:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.483366+0000 mgr.gibba032.whopeo (mgr.24625) 7726 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:19:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.483409+0000 mgr.gibba032.whopeo (mgr.24625) 7727 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:19:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.486228+0000 mon.a (mon.0) 2740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.486272+0000 mgr.gibba032.whopeo (mgr.24625) 7728 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:19:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.486795+0000 mgr.gibba032.whopeo (mgr.24625) 7729 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:19:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: audit 2021-03-10T16:19:06.489452+0000 mon.a (mon.0) 2741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.489514+0000 mgr.gibba032.whopeo (mgr.24625) 7730 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:19:07.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.489568+0000 mgr.gibba032.whopeo (mgr.24625) 7731 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:19:07.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:07 gibba011 bash[31774]: cephadm 2021-03-10T16:19:06.489720+0000 mgr.gibba032.whopeo (mgr.24625) 7732 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:08 gibba032 bash[24004]: cluster 2021-03-10T16:19:06.851527+0000 mgr.gibba032.whopeo (mgr.24625) 7733 : cluster [DBG] pgmap v4397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:08 gibba011 bash[30791]: cluster 2021-03-10T16:19:06.851527+0000 mgr.gibba032.whopeo (mgr.24625) 7733 : cluster [DBG] pgmap v4397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:08 gibba011 bash[31774]: cluster 2021-03-10T16:19:06.851527+0000 mgr.gibba032.whopeo (mgr.24625) 7733 : cluster [DBG] pgmap v4397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:10 gibba032 bash[24004]: cluster 2021-03-10T16:19:08.853130+0000 mgr.gibba032.whopeo (mgr.24625) 7734 : cluster [DBG] pgmap v4398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:10 gibba011 bash[30791]: cluster 2021-03-10T16:19:08.853130+0000 mgr.gibba032.whopeo (mgr.24625) 7734 : cluster [DBG] pgmap v4398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:10 gibba011 bash[31774]: cluster 2021-03-10T16:19:08.853130+0000 mgr.gibba032.whopeo (mgr.24625) 7734 : cluster [DBG] pgmap v4398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:12 gibba032 bash[24004]: cluster 2021-03-10T16:19:10.854455+0000 mgr.gibba032.whopeo (mgr.24625) 7735 : cluster [DBG] pgmap v4399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:12 gibba011 bash[30791]: cluster 2021-03-10T16:19:10.854455+0000 mgr.gibba032.whopeo (mgr.24625) 7735 : cluster [DBG] pgmap v4399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:12 gibba011 bash[31774]: cluster 2021-03-10T16:19:10.854455+0000 mgr.gibba032.whopeo (mgr.24625) 7735 : cluster [DBG] pgmap v4399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:14 gibba032 bash[24004]: cluster 2021-03-10T16:19:12.855011+0000 mgr.gibba032.whopeo (mgr.24625) 7736 : cluster [DBG] pgmap v4400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:14 gibba011 bash[30791]: cluster 2021-03-10T16:19:12.855011+0000 mgr.gibba032.whopeo (mgr.24625) 7736 : cluster [DBG] pgmap v4400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:14 gibba011 bash[31774]: cluster 2021-03-10T16:19:12.855011+0000 mgr.gibba032.whopeo (mgr.24625) 7736 : cluster [DBG] pgmap v4400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:16 gibba011 bash[30791]: cluster 2021-03-10T16:19:14.856169+0000 mgr.gibba032.whopeo (mgr.24625) 7737 : cluster [DBG] pgmap v4401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:16 gibba011 bash[31774]: cluster 2021-03-10T16:19:14.856169+0000 mgr.gibba032.whopeo (mgr.24625) 7737 : cluster [DBG] pgmap v4401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:16 gibba032 bash[24004]: cluster 2021-03-10T16:19:14.856169+0000 mgr.gibba032.whopeo (mgr.24625) 7737 : cluster [DBG] pgmap v4401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:18 gibba011 bash[30791]: cluster 2021-03-10T16:19:16.857641+0000 mgr.gibba032.whopeo (mgr.24625) 7738 : cluster [DBG] pgmap v4402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:18 gibba011 bash[31774]: cluster 2021-03-10T16:19:16.857641+0000 mgr.gibba032.whopeo (mgr.24625) 7738 : cluster [DBG] pgmap v4402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:19.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:18 gibba032 bash[24004]: cluster 2021-03-10T16:19:16.857641+0000 mgr.gibba032.whopeo (mgr.24625) 7738 : cluster [DBG] pgmap v4402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:20 gibba011 bash[30791]: cluster 2021-03-10T16:19:18.859110+0000 mgr.gibba032.whopeo (mgr.24625) 7739 : cluster [DBG] pgmap v4403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:20.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:20 gibba011 bash[31774]: cluster 2021-03-10T16:19:18.859110+0000 mgr.gibba032.whopeo (mgr.24625) 7739 : cluster [DBG] pgmap v4403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:20.965 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:19:21.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:20 gibba032 bash[24004]: cluster 2021-03-10T16:19:18.859110+0000 mgr.gibba032.whopeo (mgr.24625) 7739 : cluster [DBG] pgmap v4403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:21.217 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:19:21.218 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:19:21.218 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:19:21.218 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:19:21.218 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:19:21.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:19:21.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:19:21.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:19:21.219 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:19:21.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:19:21.220 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:19:21.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:19:21.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:19:21.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:19:21.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:19:21.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:19:21.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:19:21.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:19:21.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:19:21.222 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:19:21.223 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:19:21.471 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:19:21.472 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:19:21.472 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:19:21.472 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:21.472 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:19:21.472 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:19:21.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:21.473 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:19:21.473 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:19:21.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:21.474 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:19:21.474 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:19:21.474 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:19:21.474 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:19:21.475 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:19:21.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:21 gibba011 bash[30791]: audit 2021-03-10T16:19:21.472909+0000 mon.c (mon.1) 147 : audit [DBG] from='client.? 172.21.2.111:0/3163056106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:21.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:21 gibba011 bash[31774]: audit 2021-03-10T16:19:21.472909+0000 mon.c (mon.1) 147 : audit [DBG] from='client.? 172.21.2.111:0/3163056106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:22.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:21 gibba032 bash[24004]: audit 2021-03-10T16:19:21.472909+0000 mon.c (mon.1) 147 : audit [DBG] from='client.? 172.21.2.111:0/3163056106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[30791]: cluster 2021-03-10T16:19:20.859628+0000 mgr.gibba032.whopeo (mgr.24625) 7740 : cluster [DBG] pgmap v4404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[30791]: audit 2021-03-10T16:19:20.955306+0000 mgr.gibba032.whopeo (mgr.24625) 7741 : audit [DBG] from='client.49239 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[30791]: audit 2021-03-10T16:19:21.178810+0000 mgr.gibba032.whopeo (mgr.24625) 7742 : audit [DBG] from='client.49245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[30791]: cephadm 2021-03-10T16:19:21.505764+0000 mgr.gibba032.whopeo (mgr.24625) 7743 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[31774]: cluster 2021-03-10T16:19:20.859628+0000 mgr.gibba032.whopeo (mgr.24625) 7740 : cluster [DBG] pgmap v4404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[31774]: audit 2021-03-10T16:19:20.955306+0000 mgr.gibba032.whopeo (mgr.24625) 7741 : audit [DBG] from='client.49239 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[31774]: audit 2021-03-10T16:19:21.178810+0000 mgr.gibba032.whopeo (mgr.24625) 7742 : audit [DBG] from='client.49245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:22 gibba011 bash[31774]: cephadm 2021-03-10T16:19:21.505764+0000 mgr.gibba032.whopeo (mgr.24625) 7743 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:23.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:22 gibba032 bash[24004]: cluster 2021-03-10T16:19:20.859628+0000 mgr.gibba032.whopeo (mgr.24625) 7740 : cluster [DBG] pgmap v4404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:22 gibba032 bash[24004]: audit 2021-03-10T16:19:20.955306+0000 mgr.gibba032.whopeo (mgr.24625) 7741 : audit [DBG] from='client.49239 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:22 gibba032 bash[24004]: audit 2021-03-10T16:19:21.178810+0000 mgr.gibba032.whopeo (mgr.24625) 7742 : audit [DBG] from='client.49245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:22 gibba032 bash[24004]: cephadm 2021-03-10T16:19:21.505764+0000 mgr.gibba032.whopeo (mgr.24625) 7743 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:23.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:23 gibba011 bash[30791]: cluster 2021-03-10T16:19:22.861227+0000 mgr.gibba032.whopeo (mgr.24625) 7744 : cluster [DBG] pgmap v4405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:23.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:23 gibba011 bash[31774]: cluster 2021-03-10T16:19:22.861227+0000 mgr.gibba032.whopeo (mgr.24625) 7744 : cluster [DBG] pgmap v4405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:24.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:23 gibba032 bash[24004]: cluster 2021-03-10T16:19:22.861227+0000 mgr.gibba032.whopeo (mgr.24625) 7744 : cluster [DBG] pgmap v4405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:26.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:26 gibba032 bash[24004]: cluster 2021-03-10T16:19:24.862583+0000 mgr.gibba032.whopeo (mgr.24625) 7745 : cluster [DBG] pgmap v4406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:26 gibba011 bash[30791]: cluster 2021-03-10T16:19:24.862583+0000 mgr.gibba032.whopeo (mgr.24625) 7745 : cluster [DBG] pgmap v4406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:26 gibba011 bash[31774]: cluster 2021-03-10T16:19:24.862583+0000 mgr.gibba032.whopeo (mgr.24625) 7745 : cluster [DBG] pgmap v4406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:28.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:28 gibba032 bash[24004]: cluster 2021-03-10T16:19:26.864306+0000 mgr.gibba032.whopeo (mgr.24625) 7746 : cluster [DBG] pgmap v4407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:28.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:28 gibba011 bash[30791]: cluster 2021-03-10T16:19:26.864306+0000 mgr.gibba032.whopeo (mgr.24625) 7746 : cluster [DBG] pgmap v4407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:28 gibba011 bash[31774]: cluster 2021-03-10T16:19:26.864306+0000 mgr.gibba032.whopeo (mgr.24625) 7746 : cluster [DBG] pgmap v4407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:30.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:30 gibba032 bash[24004]: cluster 2021-03-10T16:19:28.865644+0000 mgr.gibba032.whopeo (mgr.24625) 7747 : cluster [DBG] pgmap v4408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:30 gibba011 bash[30791]: cluster 2021-03-10T16:19:28.865644+0000 mgr.gibba032.whopeo (mgr.24625) 7747 : cluster [DBG] pgmap v4408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:30 gibba011 bash[31774]: cluster 2021-03-10T16:19:28.865644+0000 mgr.gibba032.whopeo (mgr.24625) 7747 : cluster [DBG] pgmap v4408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:32 gibba032 bash[24004]: cluster 2021-03-10T16:19:30.866970+0000 mgr.gibba032.whopeo (mgr.24625) 7748 : cluster [DBG] pgmap v4409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:32 gibba011 bash[30791]: cluster 2021-03-10T16:19:30.866970+0000 mgr.gibba032.whopeo (mgr.24625) 7748 : cluster [DBG] pgmap v4409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:32 gibba011 bash[31774]: cluster 2021-03-10T16:19:30.866970+0000 mgr.gibba032.whopeo (mgr.24625) 7748 : cluster [DBG] pgmap v4409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:34.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:34 gibba032 bash[24004]: cluster 2021-03-10T16:19:32.868737+0000 mgr.gibba032.whopeo (mgr.24625) 7749 : cluster [DBG] pgmap v4410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:34.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:34 gibba011 bash[30791]: cluster 2021-03-10T16:19:32.868737+0000 mgr.gibba032.whopeo (mgr.24625) 7749 : cluster [DBG] pgmap v4410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:34 gibba011 bash[31774]: cluster 2021-03-10T16:19:32.868737+0000 mgr.gibba032.whopeo (mgr.24625) 7749 : cluster [DBG] pgmap v4410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:36.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:36 gibba032 bash[24004]: cluster 2021-03-10T16:19:34.870261+0000 mgr.gibba032.whopeo (mgr.24625) 7750 : cluster [DBG] pgmap v4411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:36 gibba011 bash[30791]: cluster 2021-03-10T16:19:34.870261+0000 mgr.gibba032.whopeo (mgr.24625) 7750 : cluster [DBG] pgmap v4411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:36 gibba011 bash[31774]: cluster 2021-03-10T16:19:34.870261+0000 mgr.gibba032.whopeo (mgr.24625) 7750 : cluster [DBG] pgmap v4411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:37.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:37 gibba011 bash[30791]: cephadm 2021-03-10T16:19:36.516697+0000 mgr.gibba032.whopeo (mgr.24625) 7751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:37.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:37 gibba011 bash[31774]: cephadm 2021-03-10T16:19:36.516697+0000 mgr.gibba032.whopeo (mgr.24625) 7751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:37.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:37 gibba032 bash[24004]: cephadm 2021-03-10T16:19:36.516697+0000 mgr.gibba032.whopeo (mgr.24625) 7751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:38 gibba011 bash[30791]: cluster 2021-03-10T16:19:36.871185+0000 mgr.gibba032.whopeo (mgr.24625) 7752 : cluster [DBG] pgmap v4412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:38 gibba011 bash[31774]: cluster 2021-03-10T16:19:36.871185+0000 mgr.gibba032.whopeo (mgr.24625) 7752 : cluster [DBG] pgmap v4412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:38 gibba032 bash[24004]: cluster 2021-03-10T16:19:36.871185+0000 mgr.gibba032.whopeo (mgr.24625) 7752 : cluster [DBG] pgmap v4412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:40.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:40 gibba011 bash[30791]: cluster 2021-03-10T16:19:38.872670+0000 mgr.gibba032.whopeo (mgr.24625) 7753 : cluster [DBG] pgmap v4413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:40 gibba011 bash[31774]: cluster 2021-03-10T16:19:38.872670+0000 mgr.gibba032.whopeo (mgr.24625) 7753 : cluster [DBG] pgmap v4413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:40 gibba032 bash[24004]: cluster 2021-03-10T16:19:38.872670+0000 mgr.gibba032.whopeo (mgr.24625) 7753 : cluster [DBG] pgmap v4413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:42 gibba011 bash[30791]: cluster 2021-03-10T16:19:40.874103+0000 mgr.gibba032.whopeo (mgr.24625) 7754 : cluster [DBG] pgmap v4414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:42 gibba011 bash[31774]: cluster 2021-03-10T16:19:40.874103+0000 mgr.gibba032.whopeo (mgr.24625) 7754 : cluster [DBG] pgmap v4414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:42 gibba032 bash[24004]: cluster 2021-03-10T16:19:40.874103+0000 mgr.gibba032.whopeo (mgr.24625) 7754 : cluster [DBG] pgmap v4414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:44 gibba011 bash[30791]: cluster 2021-03-10T16:19:42.875861+0000 mgr.gibba032.whopeo (mgr.24625) 7755 : cluster [DBG] pgmap v4415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:44 gibba011 bash[31774]: cluster 2021-03-10T16:19:42.875861+0000 mgr.gibba032.whopeo (mgr.24625) 7755 : cluster [DBG] pgmap v4415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:44 gibba032 bash[24004]: cluster 2021-03-10T16:19:42.875861+0000 mgr.gibba032.whopeo (mgr.24625) 7755 : cluster [DBG] pgmap v4415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:46 gibba011 bash[30791]: cluster 2021-03-10T16:19:44.876810+0000 mgr.gibba032.whopeo (mgr.24625) 7756 : cluster [DBG] pgmap v4416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:46 gibba011 bash[31774]: cluster 2021-03-10T16:19:44.876810+0000 mgr.gibba032.whopeo (mgr.24625) 7756 : cluster [DBG] pgmap v4416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:46 gibba032 bash[24004]: cluster 2021-03-10T16:19:44.876810+0000 mgr.gibba032.whopeo (mgr.24625) 7756 : cluster [DBG] pgmap v4416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:48 gibba011 bash[30791]: cluster 2021-03-10T16:19:46.878170+0000 mgr.gibba032.whopeo (mgr.24625) 7757 : cluster [DBG] pgmap v4417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:48 gibba011 bash[31774]: cluster 2021-03-10T16:19:46.878170+0000 mgr.gibba032.whopeo (mgr.24625) 7757 : cluster [DBG] pgmap v4417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:48 gibba032 bash[24004]: cluster 2021-03-10T16:19:46.878170+0000 mgr.gibba032.whopeo (mgr.24625) 7757 : cluster [DBG] pgmap v4417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:50 gibba011 bash[30791]: cluster 2021-03-10T16:19:48.879626+0000 mgr.gibba032.whopeo (mgr.24625) 7758 : cluster [DBG] pgmap v4418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:50 gibba011 bash[31774]: cluster 2021-03-10T16:19:48.879626+0000 mgr.gibba032.whopeo (mgr.24625) 7758 : cluster [DBG] pgmap v4418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:50.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:50 gibba032 bash[24004]: cluster 2021-03-10T16:19:48.879626+0000 mgr.gibba032.whopeo (mgr.24625) 7758 : cluster [DBG] pgmap v4418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:51.715 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:19:51.946 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:19:51.946 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:19:51.946 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:19:51.947 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:19:51.947 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:19:51.947 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:19:51.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:19:51.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:19:51.948 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:19:51.948 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:19:51.948 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:19:51.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:19:51.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:19:51.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:19:51.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:19:51.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:19:51.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:19:51.950 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:19:51.950 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:19:51.950 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:19:51.950 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:19:52.197 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:19:52.197 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:19:52.198 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:19:52.198 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:52.198 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:19:52.198 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:19:52.198 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:52.199 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:19:52.199 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:19:52.199 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:19:52.199 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:19:52.200 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:19:52.200 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:19:52.200 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:19:52.200 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:19:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:52 gibba032 bash[24004]: cluster 2021-03-10T16:19:50.881094+0000 mgr.gibba032.whopeo (mgr.24625) 7759 : cluster [DBG] pgmap v4419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:52 gibba032 bash[24004]: cephadm 2021-03-10T16:19:51.532739+0000 mgr.gibba032.whopeo (mgr.24625) 7760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:52 gibba032 bash[24004]: audit 2021-03-10T16:19:52.198812+0000 mon.c (mon.1) 148 : audit [DBG] from='client.? 172.21.2.111:0/3112197501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[30791]: cluster 2021-03-10T16:19:50.881094+0000 mgr.gibba032.whopeo (mgr.24625) 7759 : cluster [DBG] pgmap v4419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:52.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[30791]: cephadm 2021-03-10T16:19:51.532739+0000 mgr.gibba032.whopeo (mgr.24625) 7760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:52.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[30791]: audit 2021-03-10T16:19:52.198812+0000 mon.c (mon.1) 148 : audit [DBG] from='client.? 172.21.2.111:0/3112197501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[31774]: cluster 2021-03-10T16:19:50.881094+0000 mgr.gibba032.whopeo (mgr.24625) 7759 : cluster [DBG] pgmap v4419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[31774]: cephadm 2021-03-10T16:19:51.532739+0000 mgr.gibba032.whopeo (mgr.24625) 7760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:19:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:52 gibba011 bash[31774]: audit 2021-03-10T16:19:52.198812+0000 mon.c (mon.1) 148 : audit [DBG] from='client.? 172.21.2.111:0/3112197501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:19:53.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:53 gibba032 bash[24004]: audit 2021-03-10T16:19:51.704897+0000 mgr.gibba032.whopeo (mgr.24625) 7761 : audit [DBG] from='client.49257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:53.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:53 gibba032 bash[24004]: audit 2021-03-10T16:19:51.933027+0000 mgr.gibba032.whopeo (mgr.24625) 7762 : audit [DBG] from='client.59245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:53.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:53 gibba011 bash[30791]: audit 2021-03-10T16:19:51.704897+0000 mgr.gibba032.whopeo (mgr.24625) 7761 : audit [DBG] from='client.49257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:53.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:53 gibba011 bash[30791]: audit 2021-03-10T16:19:51.933027+0000 mgr.gibba032.whopeo (mgr.24625) 7762 : audit [DBG] from='client.59245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:53.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:53 gibba011 bash[31774]: audit 2021-03-10T16:19:51.704897+0000 mgr.gibba032.whopeo (mgr.24625) 7761 : audit [DBG] from='client.49257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:53.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:53 gibba011 bash[31774]: audit 2021-03-10T16:19:51.933027+0000 mgr.gibba032.whopeo (mgr.24625) 7762 : audit [DBG] from='client.59245 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:19:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:54 gibba032 bash[24004]: cluster 2021-03-10T16:19:52.881910+0000 mgr.gibba032.whopeo (mgr.24625) 7763 : cluster [DBG] pgmap v4420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:54 gibba011 bash[30791]: cluster 2021-03-10T16:19:52.881910+0000 mgr.gibba032.whopeo (mgr.24625) 7763 : cluster [DBG] pgmap v4420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:54 gibba011 bash[31774]: cluster 2021-03-10T16:19:52.881910+0000 mgr.gibba032.whopeo (mgr.24625) 7763 : cluster [DBG] pgmap v4420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:56 gibba032 bash[24004]: cluster 2021-03-10T16:19:54.883408+0000 mgr.gibba032.whopeo (mgr.24625) 7764 : cluster [DBG] pgmap v4421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:56 gibba011 bash[30791]: cluster 2021-03-10T16:19:54.883408+0000 mgr.gibba032.whopeo (mgr.24625) 7764 : cluster [DBG] pgmap v4421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:56 gibba011 bash[31774]: cluster 2021-03-10T16:19:54.883408+0000 mgr.gibba032.whopeo (mgr.24625) 7764 : cluster [DBG] pgmap v4421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:56.948643+0000 mon.a (mon.0) 2742 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:19:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:56.949365+0000 mon.a (mon.0) 2743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:19:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:57.043204+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:19:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:57.044087+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:19:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:57.044787+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:19:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:57 gibba032 bash[24004]: audit 2021-03-10T16:19:57.045457+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:19:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:56.948643+0000 mon.a (mon.0) 2742 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:19:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:56.949365+0000 mon.a (mon.0) 2743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:19:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:57.043204+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:19:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:57.044087+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:19:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:57.044787+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:19:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[30791]: audit 2021-03-10T16:19:57.045457+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:19:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:56.948643+0000 mon.a (mon.0) 2742 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:19:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:56.949365+0000 mon.a (mon.0) 2743 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:19:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:57.043204+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:19:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:57.044087+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:19:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:57.044787+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:19:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:57 gibba011 bash[31774]: audit 2021-03-10T16:19:57.045457+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:19:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:19:58 gibba032 bash[24004]: cluster 2021-03-10T16:19:56.885082+0000 mgr.gibba032.whopeo (mgr.24625) 7765 : cluster [DBG] pgmap v4422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:19:58 gibba011 bash[30791]: cluster 2021-03-10T16:19:56.885082+0000 mgr.gibba032.whopeo (mgr.24625) 7765 : cluster [DBG] pgmap v4422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:19:58.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:19:58 gibba011 bash[31774]: cluster 2021-03-10T16:19:56.885082+0000 mgr.gibba032.whopeo (mgr.24625) 7765 : cluster [DBG] pgmap v4422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:00.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:00 gibba011 bash[30791]: debug 2021-03-10T16:19:59.996+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:20:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:00 gibba032 bash[24004]: cluster 2021-03-10T16:19:58.886590+0000 mgr.gibba032.whopeo (mgr.24625) 7766 : cluster [DBG] pgmap v4423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:00 gibba032 bash[24004]: cluster 2021-03-10T16:20:00.000221+0000 mon.a (mon.0) 2748 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:20:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:00 gibba011 bash[30791]: cluster 2021-03-10T16:19:58.886590+0000 mgr.gibba032.whopeo (mgr.24625) 7766 : cluster [DBG] pgmap v4423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:00 gibba011 bash[30791]: cluster 2021-03-10T16:20:00.000221+0000 mon.a (mon.0) 2748 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:20:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:00 gibba011 bash[31774]: cluster 2021-03-10T16:19:58.886590+0000 mgr.gibba032.whopeo (mgr.24625) 7766 : cluster [DBG] pgmap v4423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:00 gibba011 bash[31774]: cluster 2021-03-10T16:20:00.000221+0000 mon.a (mon.0) 2748 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:20:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:02 gibba032 bash[24004]: cluster 2021-03-10T16:20:00.887272+0000 mgr.gibba032.whopeo (mgr.24625) 7767 : cluster [DBG] pgmap v4424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:02 gibba011 bash[30791]: cluster 2021-03-10T16:20:00.887272+0000 mgr.gibba032.whopeo (mgr.24625) 7767 : cluster [DBG] pgmap v4424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:02 gibba011 bash[31774]: cluster 2021-03-10T16:20:00.887272+0000 mgr.gibba032.whopeo (mgr.24625) 7767 : cluster [DBG] pgmap v4424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:04 gibba032 bash[24004]: cluster 2021-03-10T16:20:02.888857+0000 mgr.gibba032.whopeo (mgr.24625) 7768 : cluster [DBG] pgmap v4425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:04 gibba011 bash[30791]: cluster 2021-03-10T16:20:02.888857+0000 mgr.gibba032.whopeo (mgr.24625) 7768 : cluster [DBG] pgmap v4425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:04 gibba011 bash[31774]: cluster 2021-03-10T16:20:02.888857+0000 mgr.gibba032.whopeo (mgr.24625) 7768 : cluster [DBG] pgmap v4425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:06 gibba032 bash[24004]: cluster 2021-03-10T16:20:04.890306+0000 mgr.gibba032.whopeo (mgr.24625) 7769 : cluster [DBG] pgmap v4426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:06 gibba011 bash[30791]: cluster 2021-03-10T16:20:04.890306+0000 mgr.gibba032.whopeo (mgr.24625) 7769 : cluster [DBG] pgmap v4426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:06.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:06 gibba011 bash[31774]: cluster 2021-03-10T16:20:04.890306+0000 mgr.gibba032.whopeo (mgr.24625) 7769 : cluster [DBG] pgmap v4426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:07.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:07 gibba032 bash[24004]: cephadm 2021-03-10T16:20:06.550403+0000 mgr.gibba032.whopeo (mgr.24625) 7770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:20:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:07 gibba032 bash[24004]: audit 2021-03-10T16:20:06.553187+0000 mon.a (mon.0) 2749 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:20:07.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:07 gibba011 bash[30791]: cephadm 2021-03-10T16:20:06.550403+0000 mgr.gibba032.whopeo (mgr.24625) 7770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:20:07.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:07 gibba011 bash[30791]: audit 2021-03-10T16:20:06.553187+0000 mon.a (mon.0) 2749 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:20:07.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:07 gibba011 bash[31774]: cephadm 2021-03-10T16:20:06.550403+0000 mgr.gibba032.whopeo (mgr.24625) 7770 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:20:07.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:07 gibba011 bash[31774]: audit 2021-03-10T16:20:06.553187+0000 mon.a (mon.0) 2749 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:20:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:08 gibba032 bash[24004]: cluster 2021-03-10T16:20:06.891930+0000 mgr.gibba032.whopeo (mgr.24625) 7771 : cluster [DBG] pgmap v4427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:08 gibba011 bash[30791]: cluster 2021-03-10T16:20:06.891930+0000 mgr.gibba032.whopeo (mgr.24625) 7771 : cluster [DBG] pgmap v4427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:08.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:08 gibba011 bash[31774]: cluster 2021-03-10T16:20:06.891930+0000 mgr.gibba032.whopeo (mgr.24625) 7771 : cluster [DBG] pgmap v4427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: cluster 2021-03-10T16:20:08.892635+0000 mgr.gibba032.whopeo (mgr.24625) 7772 : cluster [DBG] pgmap v4428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.126737+0000 mon.a (mon.0) 2750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.328110+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.344440+0000 mon.a (mon.0) 2752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:20:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.349240+0000 mon.a (mon.0) 2753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.353926+0000 mon.a (mon.0) 2754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.357351+0000 mon.a (mon.0) 2755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.364281+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:10 gibba032 bash[24004]: audit 2021-03-10T16:20:10.369557+0000 mon.a (mon.0) 2757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: cluster 2021-03-10T16:20:08.892635+0000 mgr.gibba032.whopeo (mgr.24625) 7772 : cluster [DBG] pgmap v4428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.126737+0000 mon.a (mon.0) 2750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.328110+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.344440+0000 mon.a (mon.0) 2752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:20:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.349240+0000 mon.a (mon.0) 2753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.353926+0000 mon.a (mon.0) 2754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.357351+0000 mon.a (mon.0) 2755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.364281+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[30791]: audit 2021-03-10T16:20:10.369557+0000 mon.a (mon.0) 2757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: cluster 2021-03-10T16:20:08.892635+0000 mgr.gibba032.whopeo (mgr.24625) 7772 : cluster [DBG] pgmap v4428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.126737+0000 mon.a (mon.0) 2750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.328110+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:20:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.344440+0000 mon.a (mon.0) 2752 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:20:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.349240+0000 mon.a (mon.0) 2753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.353926+0000 mon.a (mon.0) 2754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.357351+0000 mon.a (mon.0) 2755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.364281+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:10.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:10 gibba011 bash[31774]: audit 2021-03-10T16:20:10.369557+0000 mon.a (mon.0) 2757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.125200+0000 mgr.gibba032.whopeo (mgr.24625) 7773 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:20:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.339268+0000 mgr.gibba032.whopeo (mgr.24625) 7774 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:20:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.345578+0000 mgr.gibba032.whopeo (mgr.24625) 7775 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:20:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.349877+0000 mgr.gibba032.whopeo (mgr.24625) 7776 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:20:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.350291+0000 mgr.gibba032.whopeo (mgr.24625) 7777 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:20:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.354006+0000 mgr.gibba032.whopeo (mgr.24625) 7778 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:20:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.354423+0000 mgr.gibba032.whopeo (mgr.24625) 7779 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:20:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.357202+0000 mgr.gibba032.whopeo (mgr.24625) 7780 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:20:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.360242+0000 mgr.gibba032.whopeo (mgr.24625) 7781 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:20:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.360643+0000 mgr.gibba032.whopeo (mgr.24625) 7782 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:20:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.364322+0000 mgr.gibba032.whopeo (mgr.24625) 7783 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:20:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.366466+0000 mgr.gibba032.whopeo (mgr.24625) 7784 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:20:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.369629+0000 mgr.gibba032.whopeo (mgr.24625) 7785 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:20:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.370111+0000 mgr.gibba032.whopeo (mgr.24625) 7786 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:20:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:11 gibba032 bash[24004]: cephadm 2021-03-10T16:20:10.370699+0000 mgr.gibba032.whopeo (mgr.24625) 7787 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.125200+0000 mgr.gibba032.whopeo (mgr.24625) 7773 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:20:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.339268+0000 mgr.gibba032.whopeo (mgr.24625) 7774 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:20:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.345578+0000 mgr.gibba032.whopeo (mgr.24625) 7775 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:20:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.349877+0000 mgr.gibba032.whopeo (mgr.24625) 7776 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:20:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.350291+0000 mgr.gibba032.whopeo (mgr.24625) 7777 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:20:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.354006+0000 mgr.gibba032.whopeo (mgr.24625) 7778 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:20:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.354423+0000 mgr.gibba032.whopeo (mgr.24625) 7779 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:20:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.357202+0000 mgr.gibba032.whopeo (mgr.24625) 7780 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:20:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.360242+0000 mgr.gibba032.whopeo (mgr.24625) 7781 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:20:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.360643+0000 mgr.gibba032.whopeo (mgr.24625) 7782 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:20:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.364322+0000 mgr.gibba032.whopeo (mgr.24625) 7783 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:20:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.366466+0000 mgr.gibba032.whopeo (mgr.24625) 7784 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:20:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.369629+0000 mgr.gibba032.whopeo (mgr.24625) 7785 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:20:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.370111+0000 mgr.gibba032.whopeo (mgr.24625) 7786 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:20:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[30791]: cephadm 2021-03-10T16:20:10.370699+0000 mgr.gibba032.whopeo (mgr.24625) 7787 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:11.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.125200+0000 mgr.gibba032.whopeo (mgr.24625) 7773 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:20:11.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.339268+0000 mgr.gibba032.whopeo (mgr.24625) 7774 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:20:11.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.345578+0000 mgr.gibba032.whopeo (mgr.24625) 7775 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:20:11.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.349877+0000 mgr.gibba032.whopeo (mgr.24625) 7776 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:20:11.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.350291+0000 mgr.gibba032.whopeo (mgr.24625) 7777 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:20:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.354006+0000 mgr.gibba032.whopeo (mgr.24625) 7778 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:20:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.354423+0000 mgr.gibba032.whopeo (mgr.24625) 7779 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:20:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.357202+0000 mgr.gibba032.whopeo (mgr.24625) 7780 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:20:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.360242+0000 mgr.gibba032.whopeo (mgr.24625) 7781 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:20:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.360643+0000 mgr.gibba032.whopeo (mgr.24625) 7782 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:20:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.364322+0000 mgr.gibba032.whopeo (mgr.24625) 7783 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:20:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.366466+0000 mgr.gibba032.whopeo (mgr.24625) 7784 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:20:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.369629+0000 mgr.gibba032.whopeo (mgr.24625) 7785 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:20:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.370111+0000 mgr.gibba032.whopeo (mgr.24625) 7786 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:20:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:11 gibba011 bash[31774]: cephadm 2021-03-10T16:20:10.370699+0000 mgr.gibba032.whopeo (mgr.24625) 7787 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:12 gibba032 bash[24004]: cluster 2021-03-10T16:20:10.893978+0000 mgr.gibba032.whopeo (mgr.24625) 7788 : cluster [DBG] pgmap v4429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:12 gibba011 bash[30791]: cluster 2021-03-10T16:20:10.893978+0000 mgr.gibba032.whopeo (mgr.24625) 7788 : cluster [DBG] pgmap v4429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:12 gibba011 bash[31774]: cluster 2021-03-10T16:20:10.893978+0000 mgr.gibba032.whopeo (mgr.24625) 7788 : cluster [DBG] pgmap v4429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:14 gibba032 bash[24004]: cluster 2021-03-10T16:20:12.895914+0000 mgr.gibba032.whopeo (mgr.24625) 7789 : cluster [DBG] pgmap v4430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:14 gibba011 bash[30791]: cluster 2021-03-10T16:20:12.895914+0000 mgr.gibba032.whopeo (mgr.24625) 7789 : cluster [DBG] pgmap v4430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:14 gibba011 bash[31774]: cluster 2021-03-10T16:20:12.895914+0000 mgr.gibba032.whopeo (mgr.24625) 7789 : cluster [DBG] pgmap v4430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:16 gibba032 bash[24004]: cluster 2021-03-10T16:20:14.897423+0000 mgr.gibba032.whopeo (mgr.24625) 7790 : cluster [DBG] pgmap v4431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:16 gibba011 bash[30791]: cluster 2021-03-10T16:20:14.897423+0000 mgr.gibba032.whopeo (mgr.24625) 7790 : cluster [DBG] pgmap v4431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:16 gibba011 bash[31774]: cluster 2021-03-10T16:20:14.897423+0000 mgr.gibba032.whopeo (mgr.24625) 7790 : cluster [DBG] pgmap v4431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:18 gibba032 bash[24004]: cluster 2021-03-10T16:20:16.898310+0000 mgr.gibba032.whopeo (mgr.24625) 7791 : cluster [DBG] pgmap v4432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:18 gibba011 bash[30791]: cluster 2021-03-10T16:20:16.898310+0000 mgr.gibba032.whopeo (mgr.24625) 7791 : cluster [DBG] pgmap v4432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:18 gibba011 bash[31774]: cluster 2021-03-10T16:20:16.898310+0000 mgr.gibba032.whopeo (mgr.24625) 7791 : cluster [DBG] pgmap v4432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:20 gibba032 bash[24004]: cluster 2021-03-10T16:20:18.899733+0000 mgr.gibba032.whopeo (mgr.24625) 7792 : cluster [DBG] pgmap v4433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:20 gibba011 bash[30791]: cluster 2021-03-10T16:20:18.899733+0000 mgr.gibba032.whopeo (mgr.24625) 7792 : cluster [DBG] pgmap v4433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:20.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:20 gibba011 bash[31774]: cluster 2021-03-10T16:20:18.899733+0000 mgr.gibba032.whopeo (mgr.24625) 7792 : cluster [DBG] pgmap v4433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:22.440 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:20:22.669 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:20:22.670 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 12s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:20:22.670 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:20:22.671 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:20:22.671 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:20:22.671 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:20:22.671 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:20:22.672 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:20:22.672 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:20:22.672 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 12s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:20:22.672 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:20:22.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:20:22.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:20:22.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:20:22.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:20:22.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:20:22.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:20:22.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:20:22.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:20:22.675 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:20:22.675 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 12s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:20:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:22 gibba032 bash[24004]: cluster 2021-03-10T16:20:20.901109+0000 mgr.gibba032.whopeo (mgr.24625) 7793 : cluster [DBG] pgmap v4434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:22 gibba011 bash[30791]: cluster 2021-03-10T16:20:20.901109+0000 mgr.gibba032.whopeo (mgr.24625) 7793 : cluster [DBG] pgmap v4434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:22.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:22 gibba011 bash[31774]: cluster 2021-03-10T16:20:20.901109+0000 mgr.gibba032.whopeo (mgr.24625) 7793 : cluster [DBG] pgmap v4434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:22.917 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:20:22.917 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:20:22.918 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:20:22.918 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:22.918 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:20:22.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:20:22.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:22.919 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:20:22.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:20:22.920 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:22.920 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:20:22.920 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:20:22.920 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:20:22.921 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:20:22.921 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:20:23.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:23 gibba032 bash[24004]: audit 2021-03-10T16:20:22.430763+0000 mgr.gibba032.whopeo (mgr.24625) 7794 : audit [DBG] from='client.59257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:23.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:23 gibba032 bash[24004]: audit 2021-03-10T16:20:22.918968+0000 mon.c (mon.1) 149 : audit [DBG] from='client.? 172.21.2.111:0/2524469639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:23.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:23 gibba011 bash[30791]: audit 2021-03-10T16:20:22.430763+0000 mgr.gibba032.whopeo (mgr.24625) 7794 : audit [DBG] from='client.59257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:23.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:23 gibba011 bash[30791]: audit 2021-03-10T16:20:22.918968+0000 mon.c (mon.1) 149 : audit [DBG] from='client.? 172.21.2.111:0/2524469639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:23.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:23 gibba011 bash[31774]: audit 2021-03-10T16:20:22.430763+0000 mgr.gibba032.whopeo (mgr.24625) 7794 : audit [DBG] from='client.59257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:23.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:23 gibba011 bash[31774]: audit 2021-03-10T16:20:22.918968+0000 mon.c (mon.1) 149 : audit [DBG] from='client.? 172.21.2.111:0/2524469639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:24 gibba011 bash[30791]: audit 2021-03-10T16:20:22.658087+0000 mgr.gibba032.whopeo (mgr.24625) 7795 : audit [DBG] from='client.49281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:24 gibba011 bash[30791]: cluster 2021-03-10T16:20:22.902878+0000 mgr.gibba032.whopeo (mgr.24625) 7796 : cluster [DBG] pgmap v4435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:24 gibba011 bash[31774]: audit 2021-03-10T16:20:22.658087+0000 mgr.gibba032.whopeo (mgr.24625) 7795 : audit [DBG] from='client.49281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:24 gibba011 bash[31774]: cluster 2021-03-10T16:20:22.902878+0000 mgr.gibba032.whopeo (mgr.24625) 7796 : cluster [DBG] pgmap v4435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:25.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:24 gibba032 bash[24004]: audit 2021-03-10T16:20:22.658087+0000 mgr.gibba032.whopeo (mgr.24625) 7795 : audit [DBG] from='client.49281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:24 gibba032 bash[24004]: cluster 2021-03-10T16:20:22.902878+0000 mgr.gibba032.whopeo (mgr.24625) 7796 : cluster [DBG] pgmap v4435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:26 gibba011 bash[30791]: cluster 2021-03-10T16:20:24.903579+0000 mgr.gibba032.whopeo (mgr.24625) 7797 : cluster [DBG] pgmap v4436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:26 gibba011 bash[30791]: cephadm 2021-03-10T16:20:25.386798+0000 mgr.gibba032.whopeo (mgr.24625) 7798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:26 gibba011 bash[31774]: cluster 2021-03-10T16:20:24.903579+0000 mgr.gibba032.whopeo (mgr.24625) 7797 : cluster [DBG] pgmap v4436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:26 gibba011 bash[31774]: cephadm 2021-03-10T16:20:25.386798+0000 mgr.gibba032.whopeo (mgr.24625) 7798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:26 gibba032 bash[24004]: cluster 2021-03-10T16:20:24.903579+0000 mgr.gibba032.whopeo (mgr.24625) 7797 : cluster [DBG] pgmap v4436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:26 gibba032 bash[24004]: cephadm 2021-03-10T16:20:25.386798+0000 mgr.gibba032.whopeo (mgr.24625) 7798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:28 gibba011 bash[30791]: cluster 2021-03-10T16:20:26.905090+0000 mgr.gibba032.whopeo (mgr.24625) 7799 : cluster [DBG] pgmap v4437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:28 gibba011 bash[31774]: cluster 2021-03-10T16:20:26.905090+0000 mgr.gibba032.whopeo (mgr.24625) 7799 : cluster [DBG] pgmap v4437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:28 gibba032 bash[24004]: cluster 2021-03-10T16:20:26.905090+0000 mgr.gibba032.whopeo (mgr.24625) 7799 : cluster [DBG] pgmap v4437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:30 gibba011 bash[30791]: cluster 2021-03-10T16:20:28.906657+0000 mgr.gibba032.whopeo (mgr.24625) 7800 : cluster [DBG] pgmap v4438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:30 gibba011 bash[31774]: cluster 2021-03-10T16:20:28.906657+0000 mgr.gibba032.whopeo (mgr.24625) 7800 : cluster [DBG] pgmap v4438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:30 gibba032 bash[24004]: cluster 2021-03-10T16:20:28.906657+0000 mgr.gibba032.whopeo (mgr.24625) 7800 : cluster [DBG] pgmap v4438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:32 gibba011 bash[30791]: cluster 2021-03-10T16:20:30.908105+0000 mgr.gibba032.whopeo (mgr.24625) 7801 : cluster [DBG] pgmap v4439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:32 gibba011 bash[31774]: cluster 2021-03-10T16:20:30.908105+0000 mgr.gibba032.whopeo (mgr.24625) 7801 : cluster [DBG] pgmap v4439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:32 gibba032 bash[24004]: cluster 2021-03-10T16:20:30.908105+0000 mgr.gibba032.whopeo (mgr.24625) 7801 : cluster [DBG] pgmap v4439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:34 gibba011 bash[30791]: cluster 2021-03-10T16:20:32.909135+0000 mgr.gibba032.whopeo (mgr.24625) 7802 : cluster [DBG] pgmap v4440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:34 gibba011 bash[31774]: cluster 2021-03-10T16:20:32.909135+0000 mgr.gibba032.whopeo (mgr.24625) 7802 : cluster [DBG] pgmap v4440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:34 gibba032 bash[24004]: cluster 2021-03-10T16:20:32.909135+0000 mgr.gibba032.whopeo (mgr.24625) 7802 : cluster [DBG] pgmap v4440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:36 gibba011 bash[30791]: cluster 2021-03-10T16:20:34.910305+0000 mgr.gibba032.whopeo (mgr.24625) 7803 : cluster [DBG] pgmap v4441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:36 gibba011 bash[31774]: cluster 2021-03-10T16:20:34.910305+0000 mgr.gibba032.whopeo (mgr.24625) 7803 : cluster [DBG] pgmap v4441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:36 gibba032 bash[24004]: cluster 2021-03-10T16:20:34.910305+0000 mgr.gibba032.whopeo (mgr.24625) 7803 : cluster [DBG] pgmap v4441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:38 gibba011 bash[30791]: cluster 2021-03-10T16:20:36.911932+0000 mgr.gibba032.whopeo (mgr.24625) 7804 : cluster [DBG] pgmap v4442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:38 gibba011 bash[31774]: cluster 2021-03-10T16:20:36.911932+0000 mgr.gibba032.whopeo (mgr.24625) 7804 : cluster [DBG] pgmap v4442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:38 gibba032 bash[24004]: cluster 2021-03-10T16:20:36.911932+0000 mgr.gibba032.whopeo (mgr.24625) 7804 : cluster [DBG] pgmap v4442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:40 gibba011 bash[30791]: cluster 2021-03-10T16:20:38.913334+0000 mgr.gibba032.whopeo (mgr.24625) 7805 : cluster [DBG] pgmap v4443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:40 gibba011 bash[31774]: cluster 2021-03-10T16:20:38.913334+0000 mgr.gibba032.whopeo (mgr.24625) 7805 : cluster [DBG] pgmap v4443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:40 gibba032 bash[24004]: cluster 2021-03-10T16:20:38.913334+0000 mgr.gibba032.whopeo (mgr.24625) 7805 : cluster [DBG] pgmap v4443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:41 gibba011 bash[30791]: cephadm 2021-03-10T16:20:40.402588+0000 mgr.gibba032.whopeo (mgr.24625) 7806 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:41.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:41 gibba011 bash[31774]: cephadm 2021-03-10T16:20:40.402588+0000 mgr.gibba032.whopeo (mgr.24625) 7806 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:41.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:41 gibba032 bash[24004]: cephadm 2021-03-10T16:20:40.402588+0000 mgr.gibba032.whopeo (mgr.24625) 7806 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:42 gibba011 bash[30791]: cluster 2021-03-10T16:20:40.914129+0000 mgr.gibba032.whopeo (mgr.24625) 7807 : cluster [DBG] pgmap v4444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:42 gibba011 bash[31774]: cluster 2021-03-10T16:20:40.914129+0000 mgr.gibba032.whopeo (mgr.24625) 7807 : cluster [DBG] pgmap v4444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:42 gibba032 bash[24004]: cluster 2021-03-10T16:20:40.914129+0000 mgr.gibba032.whopeo (mgr.24625) 7807 : cluster [DBG] pgmap v4444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:44 gibba011 bash[30791]: cluster 2021-03-10T16:20:42.915698+0000 mgr.gibba032.whopeo (mgr.24625) 7808 : cluster [DBG] pgmap v4445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:44 gibba011 bash[31774]: cluster 2021-03-10T16:20:42.915698+0000 mgr.gibba032.whopeo (mgr.24625) 7808 : cluster [DBG] pgmap v4445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:44 gibba032 bash[24004]: cluster 2021-03-10T16:20:42.915698+0000 mgr.gibba032.whopeo (mgr.24625) 7808 : cluster [DBG] pgmap v4445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:46 gibba011 bash[30791]: cluster 2021-03-10T16:20:44.917250+0000 mgr.gibba032.whopeo (mgr.24625) 7809 : cluster [DBG] pgmap v4446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:46 gibba011 bash[31774]: cluster 2021-03-10T16:20:44.917250+0000 mgr.gibba032.whopeo (mgr.24625) 7809 : cluster [DBG] pgmap v4446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:46 gibba032 bash[24004]: cluster 2021-03-10T16:20:44.917250+0000 mgr.gibba032.whopeo (mgr.24625) 7809 : cluster [DBG] pgmap v4446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:48 gibba011 bash[30791]: cluster 2021-03-10T16:20:46.918949+0000 mgr.gibba032.whopeo (mgr.24625) 7810 : cluster [DBG] pgmap v4447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:48 gibba011 bash[31774]: cluster 2021-03-10T16:20:46.918949+0000 mgr.gibba032.whopeo (mgr.24625) 7810 : cluster [DBG] pgmap v4447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:48 gibba032 bash[24004]: cluster 2021-03-10T16:20:46.918949+0000 mgr.gibba032.whopeo (mgr.24625) 7810 : cluster [DBG] pgmap v4447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:50 gibba011 bash[30791]: cluster 2021-03-10T16:20:48.920121+0000 mgr.gibba032.whopeo (mgr.24625) 7811 : cluster [DBG] pgmap v4448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:50 gibba011 bash[31774]: cluster 2021-03-10T16:20:48.920121+0000 mgr.gibba032.whopeo (mgr.24625) 7811 : cluster [DBG] pgmap v4448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:50.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:50 gibba032 bash[24004]: cluster 2021-03-10T16:20:48.920121+0000 mgr.gibba032.whopeo (mgr.24625) 7811 : cluster [DBG] pgmap v4448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:52 gibba011 bash[30791]: cluster 2021-03-10T16:20:50.921466+0000 mgr.gibba032.whopeo (mgr.24625) 7812 : cluster [DBG] pgmap v4449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:52 gibba011 bash[31774]: cluster 2021-03-10T16:20:50.921466+0000 mgr.gibba032.whopeo (mgr.24625) 7812 : cluster [DBG] pgmap v4449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:52 gibba032 bash[24004]: cluster 2021-03-10T16:20:50.921466+0000 mgr.gibba032.whopeo (mgr.24625) 7812 : cluster [DBG] pgmap v4449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:53.167 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:20:53.396 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:20:53.397 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 43s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:20:53.397 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:20:53.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:20:53.397 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:20:53.398 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:20:53.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:20:53.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:20:53.398 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:20:53.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 43s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:20:53.399 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:20:53.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:20:53.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:20:53.399 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:20:53.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 43s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:20:53.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:20:53.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:20:53.400 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:20:53.401 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:20:53.401 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:20:53.401 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 43s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:20:53.647 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:20:53.647 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:20:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:20:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:20:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:20:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:20:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:20:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:20:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:20:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:20:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:20:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:20:53.650 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:20:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[30791]: cluster 2021-03-10T16:20:52.923386+0000 mgr.gibba032.whopeo (mgr.24625) 7813 : cluster [DBG] pgmap v4450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[30791]: audit 2021-03-10T16:20:53.158285+0000 mgr.gibba032.whopeo (mgr.24625) 7814 : audit [DBG] from='client.59275 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[30791]: audit 2021-03-10T16:20:53.384320+0000 mgr.gibba032.whopeo (mgr.24625) 7815 : audit [DBG] from='client.59281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[30791]: audit 2021-03-10T16:20:53.648416+0000 mon.a (mon.0) 2758 : audit [DBG] from='client.? 172.21.2.111:0/2052880777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[31774]: cluster 2021-03-10T16:20:52.923386+0000 mgr.gibba032.whopeo (mgr.24625) 7813 : cluster [DBG] pgmap v4450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[31774]: audit 2021-03-10T16:20:53.158285+0000 mgr.gibba032.whopeo (mgr.24625) 7814 : audit [DBG] from='client.59275 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[31774]: audit 2021-03-10T16:20:53.384320+0000 mgr.gibba032.whopeo (mgr.24625) 7815 : audit [DBG] from='client.59281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:54 gibba011 bash[31774]: audit 2021-03-10T16:20:53.648416+0000 mon.a (mon.0) 2758 : audit [DBG] from='client.? 172.21.2.111:0/2052880777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:54 gibba032 bash[24004]: cluster 2021-03-10T16:20:52.923386+0000 mgr.gibba032.whopeo (mgr.24625) 7813 : cluster [DBG] pgmap v4450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:54 gibba032 bash[24004]: audit 2021-03-10T16:20:53.158285+0000 mgr.gibba032.whopeo (mgr.24625) 7814 : audit [DBG] from='client.59275 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:54 gibba032 bash[24004]: audit 2021-03-10T16:20:53.384320+0000 mgr.gibba032.whopeo (mgr.24625) 7815 : audit [DBG] from='client.59281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:20:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:54 gibba032 bash[24004]: audit 2021-03-10T16:20:53.648416+0000 mon.a (mon.0) 2758 : audit [DBG] from='client.? 172.21.2.111:0/2052880777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:20:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:56 gibba032 bash[24004]: cluster 2021-03-10T16:20:54.925086+0000 mgr.gibba032.whopeo (mgr.24625) 7816 : cluster [DBG] pgmap v4451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:56 gibba032 bash[24004]: cephadm 2021-03-10T16:20:55.418357+0000 mgr.gibba032.whopeo (mgr.24625) 7817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:56 gibba011 bash[30791]: cluster 2021-03-10T16:20:54.925086+0000 mgr.gibba032.whopeo (mgr.24625) 7816 : cluster [DBG] pgmap v4451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:56 gibba011 bash[30791]: cephadm 2021-03-10T16:20:55.418357+0000 mgr.gibba032.whopeo (mgr.24625) 7817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:56 gibba011 bash[31774]: cluster 2021-03-10T16:20:54.925086+0000 mgr.gibba032.whopeo (mgr.24625) 7816 : cluster [DBG] pgmap v4451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:56 gibba011 bash[31774]: cephadm 2021-03-10T16:20:55.418357+0000 mgr.gibba032.whopeo (mgr.24625) 7817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:20:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:56.956075+0000 mon.a (mon.0) 2759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:20:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:56.956701+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:20:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:57.051995+0000 mon.a (mon.0) 2761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:20:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:57.052767+0000 mon.a (mon.0) 2762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:20:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:57.053431+0000 mon.a (mon.0) 2763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:20:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:57 gibba032 bash[24004]: audit 2021-03-10T16:20:57.054107+0000 mon.a (mon.0) 2764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:20:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:56.956075+0000 mon.a (mon.0) 2759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:20:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:56.956701+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:20:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:57.051995+0000 mon.a (mon.0) 2761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:20:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:57.052767+0000 mon.a (mon.0) 2762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:20:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:57.053431+0000 mon.a (mon.0) 2763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:20:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[30791]: audit 2021-03-10T16:20:57.054107+0000 mon.a (mon.0) 2764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:20:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:56.956075+0000 mon.a (mon.0) 2759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:20:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:56.956701+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:20:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:57.051995+0000 mon.a (mon.0) 2761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:20:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:57.052767+0000 mon.a (mon.0) 2762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:20:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:57.053431+0000 mon.a (mon.0) 2763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:20:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:57 gibba011 bash[31774]: audit 2021-03-10T16:20:57.054107+0000 mon.a (mon.0) 2764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:20:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:20:58 gibba032 bash[24004]: cluster 2021-03-10T16:20:56.926950+0000 mgr.gibba032.whopeo (mgr.24625) 7818 : cluster [DBG] pgmap v4452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:20:58 gibba011 bash[30791]: cluster 2021-03-10T16:20:56.926950+0000 mgr.gibba032.whopeo (mgr.24625) 7818 : cluster [DBG] pgmap v4452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:20:58.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:20:58 gibba011 bash[31774]: cluster 2021-03-10T16:20:56.926950+0000 mgr.gibba032.whopeo (mgr.24625) 7818 : cluster [DBG] pgmap v4452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:00 gibba032 bash[24004]: cluster 2021-03-10T16:20:58.928636+0000 mgr.gibba032.whopeo (mgr.24625) 7819 : cluster [DBG] pgmap v4453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:00 gibba011 bash[30791]: cluster 2021-03-10T16:20:58.928636+0000 mgr.gibba032.whopeo (mgr.24625) 7819 : cluster [DBG] pgmap v4453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:00.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:00 gibba011 bash[31774]: cluster 2021-03-10T16:20:58.928636+0000 mgr.gibba032.whopeo (mgr.24625) 7819 : cluster [DBG] pgmap v4453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:02 gibba032 bash[24004]: cluster 2021-03-10T16:21:00.929951+0000 mgr.gibba032.whopeo (mgr.24625) 7820 : cluster [DBG] pgmap v4454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:02 gibba011 bash[30791]: cluster 2021-03-10T16:21:00.929951+0000 mgr.gibba032.whopeo (mgr.24625) 7820 : cluster [DBG] pgmap v4454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:02.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:02 gibba011 bash[31774]: cluster 2021-03-10T16:21:00.929951+0000 mgr.gibba032.whopeo (mgr.24625) 7820 : cluster [DBG] pgmap v4454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:04 gibba032 bash[24004]: cluster 2021-03-10T16:21:02.931617+0000 mgr.gibba032.whopeo (mgr.24625) 7821 : cluster [DBG] pgmap v4455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:04 gibba011 bash[30791]: cluster 2021-03-10T16:21:02.931617+0000 mgr.gibba032.whopeo (mgr.24625) 7821 : cluster [DBG] pgmap v4455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:04 gibba011 bash[31774]: cluster 2021-03-10T16:21:02.931617+0000 mgr.gibba032.whopeo (mgr.24625) 7821 : cluster [DBG] pgmap v4455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:06 gibba032 bash[24004]: cluster 2021-03-10T16:21:04.932712+0000 mgr.gibba032.whopeo (mgr.24625) 7822 : cluster [DBG] pgmap v4456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:06 gibba011 bash[30791]: cluster 2021-03-10T16:21:04.932712+0000 mgr.gibba032.whopeo (mgr.24625) 7822 : cluster [DBG] pgmap v4456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:06 gibba011 bash[31774]: cluster 2021-03-10T16:21:04.932712+0000 mgr.gibba032.whopeo (mgr.24625) 7822 : cluster [DBG] pgmap v4456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:08 gibba032 bash[24004]: cluster 2021-03-10T16:21:06.934455+0000 mgr.gibba032.whopeo (mgr.24625) 7823 : cluster [DBG] pgmap v4457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:08 gibba011 bash[30791]: cluster 2021-03-10T16:21:06.934455+0000 mgr.gibba032.whopeo (mgr.24625) 7823 : cluster [DBG] pgmap v4457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:08 gibba011 bash[31774]: cluster 2021-03-10T16:21:06.934455+0000 mgr.gibba032.whopeo (mgr.24625) 7823 : cluster [DBG] pgmap v4457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:10 gibba032 bash[24004]: cluster 2021-03-10T16:21:08.936074+0000 mgr.gibba032.whopeo (mgr.24625) 7824 : cluster [DBG] pgmap v4458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:10 gibba011 bash[30791]: cluster 2021-03-10T16:21:08.936074+0000 mgr.gibba032.whopeo (mgr.24625) 7824 : cluster [DBG] pgmap v4458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:10.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:10 gibba011 bash[31774]: cluster 2021-03-10T16:21:08.936074+0000 mgr.gibba032.whopeo (mgr.24625) 7824 : cluster [DBG] pgmap v4458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:11 gibba032 bash[24004]: cephadm 2021-03-10T16:21:10.434278+0000 mgr.gibba032.whopeo (mgr.24625) 7825 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:21:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:11 gibba032 bash[24004]: audit 2021-03-10T16:21:10.437508+0000 mon.a (mon.0) 2765 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:11.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:11 gibba011 bash[30791]: cephadm 2021-03-10T16:21:10.434278+0000 mgr.gibba032.whopeo (mgr.24625) 7825 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:21:11.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:11 gibba011 bash[30791]: audit 2021-03-10T16:21:10.437508+0000 mon.a (mon.0) 2765 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:11 gibba011 bash[31774]: cephadm 2021-03-10T16:21:10.434278+0000 mgr.gibba032.whopeo (mgr.24625) 7825 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:21:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:11 gibba011 bash[31774]: audit 2021-03-10T16:21:10.437508+0000 mon.a (mon.0) 2765 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:12 gibba032 bash[24004]: cluster 2021-03-10T16:21:10.937376+0000 mgr.gibba032.whopeo (mgr.24625) 7826 : cluster [DBG] pgmap v4459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:12 gibba011 bash[30791]: cluster 2021-03-10T16:21:10.937376+0000 mgr.gibba032.whopeo (mgr.24625) 7826 : cluster [DBG] pgmap v4459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:12 gibba011 bash[31774]: cluster 2021-03-10T16:21:10.937376+0000 mgr.gibba032.whopeo (mgr.24625) 7826 : cluster [DBG] pgmap v4459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:14 gibba032 bash[24004]: cluster 2021-03-10T16:21:12.938564+0000 mgr.gibba032.whopeo (mgr.24625) 7827 : cluster [DBG] pgmap v4460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:14 gibba032 bash[24004]: audit 2021-03-10T16:21:14.158170+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:14 gibba032 bash[24004]: audit 2021-03-10T16:21:14.159064+0000 mon.a (mon.0) 2767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:21:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[30791]: cluster 2021-03-10T16:21:12.938564+0000 mgr.gibba032.whopeo (mgr.24625) 7827 : cluster [DBG] pgmap v4460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[30791]: audit 2021-03-10T16:21:14.158170+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[30791]: audit 2021-03-10T16:21:14.159064+0000 mon.a (mon.0) 2767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:21:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[31774]: cluster 2021-03-10T16:21:12.938564+0000 mgr.gibba032.whopeo (mgr.24625) 7827 : cluster [DBG] pgmap v4460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[31774]: audit 2021-03-10T16:21:14.158170+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:14 gibba011 bash[31774]: audit 2021-03-10T16:21:14.159064+0000 mon.a (mon.0) 2767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:21:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:16 gibba032 bash[24004]: cluster 2021-03-10T16:21:14.939901+0000 mgr.gibba032.whopeo (mgr.24625) 7828 : cluster [DBG] pgmap v4461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:16 gibba011 bash[30791]: cluster 2021-03-10T16:21:14.939901+0000 mgr.gibba032.whopeo (mgr.24625) 7828 : cluster [DBG] pgmap v4461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:16 gibba011 bash[31774]: cluster 2021-03-10T16:21:14.939901+0000 mgr.gibba032.whopeo (mgr.24625) 7828 : cluster [DBG] pgmap v4461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:18 gibba032 bash[24004]: cluster 2021-03-10T16:21:16.940482+0000 mgr.gibba032.whopeo (mgr.24625) 7829 : cluster [DBG] pgmap v4462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:18 gibba011 bash[30791]: cluster 2021-03-10T16:21:16.940482+0000 mgr.gibba032.whopeo (mgr.24625) 7829 : cluster [DBG] pgmap v4462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:18 gibba011 bash[31774]: cluster 2021-03-10T16:21:16.940482+0000 mgr.gibba032.whopeo (mgr.24625) 7829 : cluster [DBG] pgmap v4462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:19.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:18.749253+0000 mon.a (mon.0) 2768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:19.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:18.917811+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.105294+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.109252+0000 mon.a (mon.0) 2771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:21:19.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.111125+0000 mon.a (mon.0) 2772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.113376+0000 mon.a (mon.0) 2773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.115119+0000 mon.a (mon.0) 2774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.118571+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:19 gibba032 bash[24004]: audit 2021-03-10T16:21:19.120516+0000 mon.a (mon.0) 2776 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:18.749253+0000 mon.a (mon.0) 2768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:19.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:18.917811+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.105294+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.109252+0000 mon.a (mon.0) 2771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:21:19.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.111125+0000 mon.a (mon.0) 2772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.113376+0000 mon.a (mon.0) 2773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.115119+0000 mon.a (mon.0) 2774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.118571+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[30791]: audit 2021-03-10T16:21:19.120516+0000 mon.a (mon.0) 2776 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:18.749253+0000 mon.a (mon.0) 2768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:21:19.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:18.917811+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.105294+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:21:19.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.109252+0000 mon.a (mon.0) 2771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:21:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.111125+0000 mon.a (mon.0) 2772 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.113376+0000 mon.a (mon.0) 2773 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.115119+0000 mon.a (mon.0) 2774 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.118571+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:19.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:19 gibba011 bash[31774]: audit 2021-03-10T16:21:19.120516+0000 mon.a (mon.0) 2776 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:18.916372+0000 mgr.gibba032.whopeo (mgr.24625) 7830 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:21:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cluster 2021-03-10T16:21:18.940896+0000 mgr.gibba032.whopeo (mgr.24625) 7831 : cluster [DBG] pgmap v4463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.106106+0000 mgr.gibba032.whopeo (mgr.24625) 7832 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:21:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.108394+0000 mgr.gibba032.whopeo (mgr.24625) 7833 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:21:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.110523+0000 mgr.gibba032.whopeo (mgr.24625) 7834 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:21:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7835 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:21:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.112407+0000 mgr.gibba032.whopeo (mgr.24625) 7836 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:21:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.112442+0000 mgr.gibba032.whopeo (mgr.24625) 7837 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:21:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.114285+0000 mgr.gibba032.whopeo (mgr.24625) 7838 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:21:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.115835+0000 mgr.gibba032.whopeo (mgr.24625) 7839 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:21:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.115870+0000 mgr.gibba032.whopeo (mgr.24625) 7840 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:21:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.117445+0000 mgr.gibba032.whopeo (mgr.24625) 7841 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:21:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.117752+0000 mgr.gibba032.whopeo (mgr.24625) 7842 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:21:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.119705+0000 mgr.gibba032.whopeo (mgr.24625) 7843 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:21:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.119743+0000 mgr.gibba032.whopeo (mgr.24625) 7844 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:21:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:20 gibba032 bash[24004]: cephadm 2021-03-10T16:21:19.119925+0000 mgr.gibba032.whopeo (mgr.24625) 7845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:18.916372+0000 mgr.gibba032.whopeo (mgr.24625) 7830 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:21:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cluster 2021-03-10T16:21:18.940896+0000 mgr.gibba032.whopeo (mgr.24625) 7831 : cluster [DBG] pgmap v4463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.106106+0000 mgr.gibba032.whopeo (mgr.24625) 7832 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:21:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.108394+0000 mgr.gibba032.whopeo (mgr.24625) 7833 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:21:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.110523+0000 mgr.gibba032.whopeo (mgr.24625) 7834 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:21:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7835 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:21:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.112407+0000 mgr.gibba032.whopeo (mgr.24625) 7836 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:21:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.112442+0000 mgr.gibba032.whopeo (mgr.24625) 7837 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:21:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.114285+0000 mgr.gibba032.whopeo (mgr.24625) 7838 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:21:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.115835+0000 mgr.gibba032.whopeo (mgr.24625) 7839 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:21:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.115870+0000 mgr.gibba032.whopeo (mgr.24625) 7840 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:21:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.117445+0000 mgr.gibba032.whopeo (mgr.24625) 7841 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:21:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.117752+0000 mgr.gibba032.whopeo (mgr.24625) 7842 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:21:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.119705+0000 mgr.gibba032.whopeo (mgr.24625) 7843 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:21:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.119743+0000 mgr.gibba032.whopeo (mgr.24625) 7844 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:21:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[30791]: cephadm 2021-03-10T16:21:19.119925+0000 mgr.gibba032.whopeo (mgr.24625) 7845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:18.916372+0000 mgr.gibba032.whopeo (mgr.24625) 7830 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:21:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cluster 2021-03-10T16:21:18.940896+0000 mgr.gibba032.whopeo (mgr.24625) 7831 : cluster [DBG] pgmap v4463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.106106+0000 mgr.gibba032.whopeo (mgr.24625) 7832 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:21:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.108394+0000 mgr.gibba032.whopeo (mgr.24625) 7833 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:21:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.110523+0000 mgr.gibba032.whopeo (mgr.24625) 7834 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:21:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.110561+0000 mgr.gibba032.whopeo (mgr.24625) 7835 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:21:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.112407+0000 mgr.gibba032.whopeo (mgr.24625) 7836 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:21:20.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.112442+0000 mgr.gibba032.whopeo (mgr.24625) 7837 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:21:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.114285+0000 mgr.gibba032.whopeo (mgr.24625) 7838 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:21:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.115835+0000 mgr.gibba032.whopeo (mgr.24625) 7839 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:21:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.115870+0000 mgr.gibba032.whopeo (mgr.24625) 7840 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:21:20.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.117445+0000 mgr.gibba032.whopeo (mgr.24625) 7841 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:21:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.117752+0000 mgr.gibba032.whopeo (mgr.24625) 7842 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:21:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.119705+0000 mgr.gibba032.whopeo (mgr.24625) 7843 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:21:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.119743+0000 mgr.gibba032.whopeo (mgr.24625) 7844 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:21:20.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:20 gibba011 bash[31774]: cephadm 2021-03-10T16:21:19.119925+0000 mgr.gibba032.whopeo (mgr.24625) 7845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:22 gibba032 bash[24004]: cluster 2021-03-10T16:21:20.942142+0000 mgr.gibba032.whopeo (mgr.24625) 7846 : cluster [DBG] pgmap v4464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:22 gibba011 bash[30791]: cluster 2021-03-10T16:21:20.942142+0000 mgr.gibba032.whopeo (mgr.24625) 7846 : cluster [DBG] pgmap v4464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:22 gibba011 bash[31774]: cluster 2021-03-10T16:21:20.942142+0000 mgr.gibba032.whopeo (mgr.24625) 7846 : cluster [DBG] pgmap v4464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:23.892 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:21:24.119 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:21:24.119 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 73s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:21:24.119 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:21:24.120 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:21:24.120 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:21:24.120 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:21:24.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:21:24.120 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:21:24.121 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:21:24.121 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 73s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:21:24.121 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:21:24.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:21:24.121 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:21:24.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:21:24.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:21:24.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:21:24.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:21:24.122 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:21:24.123 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:21:24.123 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:21:24.123 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 73s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:21:24.386 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:21:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:21:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:21:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:21:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:21:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:21:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:21:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:21:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:21:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:21:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:21:24.389 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:21:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:24 gibba032 bash[24004]: cluster 2021-03-10T16:21:22.943883+0000 mgr.gibba032.whopeo (mgr.24625) 7847 : cluster [DBG] pgmap v4465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:24 gibba032 bash[24004]: audit 2021-03-10T16:21:24.387751+0000 mon.c (mon.1) 150 : audit [DBG] from='client.? 172.21.2.111:0/3108424489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:24 gibba011 bash[30791]: cluster 2021-03-10T16:21:22.943883+0000 mgr.gibba032.whopeo (mgr.24625) 7847 : cluster [DBG] pgmap v4465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:24 gibba011 bash[30791]: audit 2021-03-10T16:21:24.387751+0000 mon.c (mon.1) 150 : audit [DBG] from='client.? 172.21.2.111:0/3108424489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:24 gibba011 bash[31774]: cluster 2021-03-10T16:21:22.943883+0000 mgr.gibba032.whopeo (mgr.24625) 7847 : cluster [DBG] pgmap v4465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:24 gibba011 bash[31774]: audit 2021-03-10T16:21:24.387751+0000 mon.c (mon.1) 150 : audit [DBG] from='client.? 172.21.2.111:0/3108424489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:25.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:25 gibba032 bash[24004]: audit 2021-03-10T16:21:23.882858+0000 mgr.gibba032.whopeo (mgr.24625) 7848 : audit [DBG] from='client.49311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:25.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:25 gibba032 bash[24004]: audit 2021-03-10T16:21:24.106278+0000 mgr.gibba032.whopeo (mgr.24625) 7849 : audit [DBG] from='client.49317 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:25.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:25 gibba011 bash[30791]: audit 2021-03-10T16:21:23.882858+0000 mgr.gibba032.whopeo (mgr.24625) 7848 : audit [DBG] from='client.49311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:25.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:25 gibba011 bash[30791]: audit 2021-03-10T16:21:24.106278+0000 mgr.gibba032.whopeo (mgr.24625) 7849 : audit [DBG] from='client.49317 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:25.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:25 gibba011 bash[31774]: audit 2021-03-10T16:21:23.882858+0000 mgr.gibba032.whopeo (mgr.24625) 7848 : audit [DBG] from='client.49311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:25.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:25 gibba011 bash[31774]: audit 2021-03-10T16:21:24.106278+0000 mgr.gibba032.whopeo (mgr.24625) 7849 : audit [DBG] from='client.49317 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:26 gibba032 bash[24004]: cluster 2021-03-10T16:21:24.945479+0000 mgr.gibba032.whopeo (mgr.24625) 7850 : cluster [DBG] pgmap v4466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:26 gibba011 bash[30791]: cluster 2021-03-10T16:21:24.945479+0000 mgr.gibba032.whopeo (mgr.24625) 7850 : cluster [DBG] pgmap v4466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:26 gibba011 bash[31774]: cluster 2021-03-10T16:21:24.945479+0000 mgr.gibba032.whopeo (mgr.24625) 7850 : cluster [DBG] pgmap v4466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:28 gibba032 bash[24004]: cluster 2021-03-10T16:21:26.947244+0000 mgr.gibba032.whopeo (mgr.24625) 7851 : cluster [DBG] pgmap v4467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:28 gibba011 bash[30791]: cluster 2021-03-10T16:21:26.947244+0000 mgr.gibba032.whopeo (mgr.24625) 7851 : cluster [DBG] pgmap v4467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:28 gibba011 bash[31774]: cluster 2021-03-10T16:21:26.947244+0000 mgr.gibba032.whopeo (mgr.24625) 7851 : cluster [DBG] pgmap v4467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:30 gibba032 bash[24004]: cluster 2021-03-10T16:21:28.948634+0000 mgr.gibba032.whopeo (mgr.24625) 7852 : cluster [DBG] pgmap v4468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:30 gibba011 bash[30791]: cluster 2021-03-10T16:21:28.948634+0000 mgr.gibba032.whopeo (mgr.24625) 7852 : cluster [DBG] pgmap v4468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:30 gibba011 bash[31774]: cluster 2021-03-10T16:21:28.948634+0000 mgr.gibba032.whopeo (mgr.24625) 7852 : cluster [DBG] pgmap v4468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:32 gibba011 bash[30791]: cluster 2021-03-10T16:21:30.949948+0000 mgr.gibba032.whopeo (mgr.24625) 7853 : cluster [DBG] pgmap v4469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:32 gibba011 bash[31774]: cluster 2021-03-10T16:21:30.949948+0000 mgr.gibba032.whopeo (mgr.24625) 7853 : cluster [DBG] pgmap v4469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:32 gibba032 bash[24004]: cluster 2021-03-10T16:21:30.949948+0000 mgr.gibba032.whopeo (mgr.24625) 7853 : cluster [DBG] pgmap v4469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:34 gibba011 bash[30791]: cluster 2021-03-10T16:21:32.951685+0000 mgr.gibba032.whopeo (mgr.24625) 7854 : cluster [DBG] pgmap v4470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:34 gibba011 bash[31774]: cluster 2021-03-10T16:21:32.951685+0000 mgr.gibba032.whopeo (mgr.24625) 7854 : cluster [DBG] pgmap v4470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:34 gibba032 bash[24004]: cluster 2021-03-10T16:21:32.951685+0000 mgr.gibba032.whopeo (mgr.24625) 7854 : cluster [DBG] pgmap v4470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:35.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:35 gibba011 bash[30791]: cephadm 2021-03-10T16:21:34.121023+0000 mgr.gibba032.whopeo (mgr.24625) 7855 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:35.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:35 gibba011 bash[31774]: cephadm 2021-03-10T16:21:34.121023+0000 mgr.gibba032.whopeo (mgr.24625) 7855 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:36.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:35 gibba032 bash[24004]: cephadm 2021-03-10T16:21:34.121023+0000 mgr.gibba032.whopeo (mgr.24625) 7855 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:36 gibba011 bash[30791]: cluster 2021-03-10T16:21:34.953294+0000 mgr.gibba032.whopeo (mgr.24625) 7856 : cluster [DBG] pgmap v4471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:36 gibba011 bash[31774]: cluster 2021-03-10T16:21:34.953294+0000 mgr.gibba032.whopeo (mgr.24625) 7856 : cluster [DBG] pgmap v4471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:36 gibba032 bash[24004]: cluster 2021-03-10T16:21:34.953294+0000 mgr.gibba032.whopeo (mgr.24625) 7856 : cluster [DBG] pgmap v4471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:38 gibba011 bash[30791]: cluster 2021-03-10T16:21:36.955068+0000 mgr.gibba032.whopeo (mgr.24625) 7857 : cluster [DBG] pgmap v4472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:38 gibba011 bash[31774]: cluster 2021-03-10T16:21:36.955068+0000 mgr.gibba032.whopeo (mgr.24625) 7857 : cluster [DBG] pgmap v4472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:38 gibba032 bash[24004]: cluster 2021-03-10T16:21:36.955068+0000 mgr.gibba032.whopeo (mgr.24625) 7857 : cluster [DBG] pgmap v4472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:40 gibba011 bash[30791]: cluster 2021-03-10T16:21:38.956736+0000 mgr.gibba032.whopeo (mgr.24625) 7858 : cluster [DBG] pgmap v4473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:40 gibba011 bash[31774]: cluster 2021-03-10T16:21:38.956736+0000 mgr.gibba032.whopeo (mgr.24625) 7858 : cluster [DBG] pgmap v4473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:40 gibba032 bash[24004]: cluster 2021-03-10T16:21:38.956736+0000 mgr.gibba032.whopeo (mgr.24625) 7858 : cluster [DBG] pgmap v4473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:42 gibba011 bash[30791]: cluster 2021-03-10T16:21:40.958193+0000 mgr.gibba032.whopeo (mgr.24625) 7859 : cluster [DBG] pgmap v4474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:42.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:42 gibba011 bash[31774]: cluster 2021-03-10T16:21:40.958193+0000 mgr.gibba032.whopeo (mgr.24625) 7859 : cluster [DBG] pgmap v4474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:42 gibba032 bash[24004]: cluster 2021-03-10T16:21:40.958193+0000 mgr.gibba032.whopeo (mgr.24625) 7859 : cluster [DBG] pgmap v4474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:44 gibba011 bash[30791]: cluster 2021-03-10T16:21:42.959727+0000 mgr.gibba032.whopeo (mgr.24625) 7860 : cluster [DBG] pgmap v4475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:44 gibba011 bash[31774]: cluster 2021-03-10T16:21:42.959727+0000 mgr.gibba032.whopeo (mgr.24625) 7860 : cluster [DBG] pgmap v4475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:44 gibba032 bash[24004]: cluster 2021-03-10T16:21:42.959727+0000 mgr.gibba032.whopeo (mgr.24625) 7860 : cluster [DBG] pgmap v4475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:46 gibba011 bash[30791]: cluster 2021-03-10T16:21:44.961252+0000 mgr.gibba032.whopeo (mgr.24625) 7861 : cluster [DBG] pgmap v4476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:46 gibba011 bash[31774]: cluster 2021-03-10T16:21:44.961252+0000 mgr.gibba032.whopeo (mgr.24625) 7861 : cluster [DBG] pgmap v4476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:46 gibba032 bash[24004]: cluster 2021-03-10T16:21:44.961252+0000 mgr.gibba032.whopeo (mgr.24625) 7861 : cluster [DBG] pgmap v4476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:48 gibba032 bash[24004]: cluster 2021-03-10T16:21:46.962977+0000 mgr.gibba032.whopeo (mgr.24625) 7862 : cluster [DBG] pgmap v4477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:48 gibba011 bash[30791]: cluster 2021-03-10T16:21:46.962977+0000 mgr.gibba032.whopeo (mgr.24625) 7862 : cluster [DBG] pgmap v4477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:49.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:48 gibba011 bash[31774]: cluster 2021-03-10T16:21:46.962977+0000 mgr.gibba032.whopeo (mgr.24625) 7862 : cluster [DBG] pgmap v4477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:50 gibba032 bash[24004]: cluster 2021-03-10T16:21:48.964324+0000 mgr.gibba032.whopeo (mgr.24625) 7863 : cluster [DBG] pgmap v4478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:50 gibba032 bash[24004]: cephadm 2021-03-10T16:21:49.131082+0000 mgr.gibba032.whopeo (mgr.24625) 7864 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:50 gibba011 bash[30791]: cluster 2021-03-10T16:21:48.964324+0000 mgr.gibba032.whopeo (mgr.24625) 7863 : cluster [DBG] pgmap v4478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:51.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:50 gibba011 bash[30791]: cephadm 2021-03-10T16:21:49.131082+0000 mgr.gibba032.whopeo (mgr.24625) 7864 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:51.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:50 gibba011 bash[31774]: cluster 2021-03-10T16:21:48.964324+0000 mgr.gibba032.whopeo (mgr.24625) 7863 : cluster [DBG] pgmap v4478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:51.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:50 gibba011 bash[31774]: cephadm 2021-03-10T16:21:49.131082+0000 mgr.gibba032.whopeo (mgr.24625) 7864 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:21:52.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:51 gibba032 bash[24004]: cluster 2021-03-10T16:21:50.965938+0000 mgr.gibba032.whopeo (mgr.24625) 7865 : cluster [DBG] pgmap v4479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:52.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:51 gibba011 bash[30791]: cluster 2021-03-10T16:21:50.965938+0000 mgr.gibba032.whopeo (mgr.24625) 7865 : cluster [DBG] pgmap v4479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:52.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:51 gibba011 bash[31774]: cluster 2021-03-10T16:21:50.965938+0000 mgr.gibba032.whopeo (mgr.24625) 7865 : cluster [DBG] pgmap v4479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:54.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:54 gibba011 bash[30791]: cluster 2021-03-10T16:21:52.967703+0000 mgr.gibba032.whopeo (mgr.24625) 7866 : cluster [DBG] pgmap v4480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:54.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:54 gibba011 bash[31774]: cluster 2021-03-10T16:21:52.967703+0000 mgr.gibba032.whopeo (mgr.24625) 7866 : cluster [DBG] pgmap v4480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:54.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:54 gibba032 bash[24004]: cluster 2021-03-10T16:21:52.967703+0000 mgr.gibba032.whopeo (mgr.24625) 7866 : cluster [DBG] pgmap v4480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:54.628 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:21:54.856 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:21:54.856 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 104s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:21:54.857 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 40s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:21:54.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:21:54.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:21:54.857 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:21:54.858 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:21:54.858 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:21:54.858 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:21:54.858 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 104s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:21:54.859 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 40s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:21:54.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:21:54.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:21:54.859 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:21:54.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:21:54.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:21:54.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:21:54.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:21:54.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:21:54.861 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 40s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:21:54.861 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 104s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:21:55.121 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:21:55.121 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:21:55.121 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:21:55.122 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:55.122 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:21:55.122 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:21:55.122 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:55.123 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:21:55.123 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:21:55.123 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:21:55.123 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:21:55.124 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:21:55.124 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:21:55.124 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:21:55.124 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:21:55.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:55 gibba011 bash[30791]: audit 2021-03-10T16:21:55.122528+0000 mon.a (mon.0) 2777 : audit [DBG] from='client.? 172.21.2.111:0/1210395390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:55.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:55 gibba011 bash[31774]: audit 2021-03-10T16:21:55.122528+0000 mon.a (mon.0) 2777 : audit [DBG] from='client.? 172.21.2.111:0/1210395390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:55.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:55 gibba032 bash[24004]: audit 2021-03-10T16:21:55.122528+0000 mon.a (mon.0) 2777 : audit [DBG] from='client.? 172.21.2.111:0/1210395390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:21:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[30791]: audit 2021-03-10T16:21:54.619518+0000 mgr.gibba032.whopeo (mgr.24625) 7867 : audit [DBG] from='client.59311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[30791]: audit 2021-03-10T16:21:54.843855+0000 mgr.gibba032.whopeo (mgr.24625) 7868 : audit [DBG] from='client.49335 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[30791]: cluster 2021-03-10T16:21:54.969146+0000 mgr.gibba032.whopeo (mgr.24625) 7869 : cluster [DBG] pgmap v4481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[31774]: audit 2021-03-10T16:21:54.619518+0000 mgr.gibba032.whopeo (mgr.24625) 7867 : audit [DBG] from='client.59311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[31774]: audit 2021-03-10T16:21:54.843855+0000 mgr.gibba032.whopeo (mgr.24625) 7868 : audit [DBG] from='client.49335 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:56 gibba011 bash[31774]: cluster 2021-03-10T16:21:54.969146+0000 mgr.gibba032.whopeo (mgr.24625) 7869 : cluster [DBG] pgmap v4481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:56 gibba032 bash[24004]: audit 2021-03-10T16:21:54.619518+0000 mgr.gibba032.whopeo (mgr.24625) 7867 : audit [DBG] from='client.59311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:56 gibba032 bash[24004]: audit 2021-03-10T16:21:54.843855+0000 mgr.gibba032.whopeo (mgr.24625) 7868 : audit [DBG] from='client.49335 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:21:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:56 gibba032 bash[24004]: cluster 2021-03-10T16:21:54.969146+0000 mgr.gibba032.whopeo (mgr.24625) 7869 : cluster [DBG] pgmap v4481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:56.963854+0000 mon.a (mon.0) 2778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:21:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:56.964554+0000 mon.a (mon.0) 2779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:21:57.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:57.061787+0000 mon.a (mon.0) 2780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:21:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:57.062573+0000 mon.a (mon.0) 2781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:57.063282+0000 mon.a (mon.0) 2782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[30791]: audit 2021-03-10T16:21:57.064054+0000 mon.a (mon.0) 2783 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:56.963854+0000 mon.a (mon.0) 2778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:21:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:56.964554+0000 mon.a (mon.0) 2779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:21:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:57.061787+0000 mon.a (mon.0) 2780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:21:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:57.062573+0000 mon.a (mon.0) 2781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:57.063282+0000 mon.a (mon.0) 2782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:57 gibba011 bash[31774]: audit 2021-03-10T16:21:57.064054+0000 mon.a (mon.0) 2783 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:56.963854+0000 mon.a (mon.0) 2778 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:21:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:56.964554+0000 mon.a (mon.0) 2779 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:21:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:57.061787+0000 mon.a (mon.0) 2780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:21:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:57.062573+0000 mon.a (mon.0) 2781 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:57.063282+0000 mon.a (mon.0) 2782 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:21:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:57 gibba032 bash[24004]: audit 2021-03-10T16:21:57.064054+0000 mon.a (mon.0) 2783 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:21:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:21:58 gibba011 bash[30791]: cluster 2021-03-10T16:21:56.970016+0000 mgr.gibba032.whopeo (mgr.24625) 7870 : cluster [DBG] pgmap v4482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:21:58 gibba011 bash[31774]: cluster 2021-03-10T16:21:56.970016+0000 mgr.gibba032.whopeo (mgr.24625) 7870 : cluster [DBG] pgmap v4482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:21:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:21:58 gibba032 bash[24004]: cluster 2021-03-10T16:21:56.970016+0000 mgr.gibba032.whopeo (mgr.24625) 7870 : cluster [DBG] pgmap v4482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:00 gibba032 bash[24004]: cluster 2021-03-10T16:21:58.971477+0000 mgr.gibba032.whopeo (mgr.24625) 7871 : cluster [DBG] pgmap v4483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:00 gibba011 bash[30791]: cluster 2021-03-10T16:21:58.971477+0000 mgr.gibba032.whopeo (mgr.24625) 7871 : cluster [DBG] pgmap v4483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:00 gibba011 bash[31774]: cluster 2021-03-10T16:21:58.971477+0000 mgr.gibba032.whopeo (mgr.24625) 7871 : cluster [DBG] pgmap v4483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:02 gibba032 bash[24004]: cluster 2021-03-10T16:22:00.972987+0000 mgr.gibba032.whopeo (mgr.24625) 7872 : cluster [DBG] pgmap v4484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:02 gibba011 bash[30791]: cluster 2021-03-10T16:22:00.972987+0000 mgr.gibba032.whopeo (mgr.24625) 7872 : cluster [DBG] pgmap v4484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:02.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:02 gibba011 bash[31774]: cluster 2021-03-10T16:22:00.972987+0000 mgr.gibba032.whopeo (mgr.24625) 7872 : cluster [DBG] pgmap v4484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:04 gibba032 bash[24004]: cluster 2021-03-10T16:22:02.974730+0000 mgr.gibba032.whopeo (mgr.24625) 7873 : cluster [DBG] pgmap v4485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:04 gibba011 bash[30791]: cluster 2021-03-10T16:22:02.974730+0000 mgr.gibba032.whopeo (mgr.24625) 7873 : cluster [DBG] pgmap v4485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:04 gibba011 bash[31774]: cluster 2021-03-10T16:22:02.974730+0000 mgr.gibba032.whopeo (mgr.24625) 7873 : cluster [DBG] pgmap v4485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:05.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:05 gibba032 bash[24004]: cephadm 2021-03-10T16:22:04.147013+0000 mgr.gibba032.whopeo (mgr.24625) 7874 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:05.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:05 gibba011 bash[30791]: cephadm 2021-03-10T16:22:04.147013+0000 mgr.gibba032.whopeo (mgr.24625) 7874 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:05.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:05 gibba011 bash[31774]: cephadm 2021-03-10T16:22:04.147013+0000 mgr.gibba032.whopeo (mgr.24625) 7874 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:06 gibba032 bash[24004]: cluster 2021-03-10T16:22:04.976294+0000 mgr.gibba032.whopeo (mgr.24625) 7875 : cluster [DBG] pgmap v4486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:06 gibba011 bash[30791]: cluster 2021-03-10T16:22:04.976294+0000 mgr.gibba032.whopeo (mgr.24625) 7875 : cluster [DBG] pgmap v4486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:06 gibba011 bash[31774]: cluster 2021-03-10T16:22:04.976294+0000 mgr.gibba032.whopeo (mgr.24625) 7875 : cluster [DBG] pgmap v4486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:08 gibba032 bash[24004]: cluster 2021-03-10T16:22:06.978034+0000 mgr.gibba032.whopeo (mgr.24625) 7876 : cluster [DBG] pgmap v4487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:08 gibba011 bash[30791]: cluster 2021-03-10T16:22:06.978034+0000 mgr.gibba032.whopeo (mgr.24625) 7876 : cluster [DBG] pgmap v4487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:08.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:08 gibba011 bash[31774]: cluster 2021-03-10T16:22:06.978034+0000 mgr.gibba032.whopeo (mgr.24625) 7876 : cluster [DBG] pgmap v4487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:10 gibba032 bash[24004]: cluster 2021-03-10T16:22:08.979514+0000 mgr.gibba032.whopeo (mgr.24625) 7877 : cluster [DBG] pgmap v4488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:10 gibba011 bash[30791]: cluster 2021-03-10T16:22:08.979514+0000 mgr.gibba032.whopeo (mgr.24625) 7877 : cluster [DBG] pgmap v4488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:10.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:10 gibba011 bash[31774]: cluster 2021-03-10T16:22:08.979514+0000 mgr.gibba032.whopeo (mgr.24625) 7877 : cluster [DBG] pgmap v4488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:12 gibba032 bash[24004]: cluster 2021-03-10T16:22:10.980862+0000 mgr.gibba032.whopeo (mgr.24625) 7878 : cluster [DBG] pgmap v4489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:12 gibba011 bash[30791]: cluster 2021-03-10T16:22:10.980862+0000 mgr.gibba032.whopeo (mgr.24625) 7878 : cluster [DBG] pgmap v4489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:12 gibba011 bash[31774]: cluster 2021-03-10T16:22:10.980862+0000 mgr.gibba032.whopeo (mgr.24625) 7878 : cluster [DBG] pgmap v4489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:14 gibba032 bash[24004]: cluster 2021-03-10T16:22:12.982565+0000 mgr.gibba032.whopeo (mgr.24625) 7879 : cluster [DBG] pgmap v4490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:14 gibba011 bash[30791]: cluster 2021-03-10T16:22:12.982565+0000 mgr.gibba032.whopeo (mgr.24625) 7879 : cluster [DBG] pgmap v4490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:14.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:14 gibba011 bash[31774]: cluster 2021-03-10T16:22:12.982565+0000 mgr.gibba032.whopeo (mgr.24625) 7879 : cluster [DBG] pgmap v4490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:16 gibba032 bash[24004]: cluster 2021-03-10T16:22:14.984076+0000 mgr.gibba032.whopeo (mgr.24625) 7880 : cluster [DBG] pgmap v4491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:16 gibba011 bash[30791]: cluster 2021-03-10T16:22:14.984076+0000 mgr.gibba032.whopeo (mgr.24625) 7880 : cluster [DBG] pgmap v4491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:16 gibba011 bash[31774]: cluster 2021-03-10T16:22:14.984076+0000 mgr.gibba032.whopeo (mgr.24625) 7880 : cluster [DBG] pgmap v4491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:18 gibba032 bash[24004]: cluster 2021-03-10T16:22:16.985524+0000 mgr.gibba032.whopeo (mgr.24625) 7881 : cluster [DBG] pgmap v4492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:18 gibba011 bash[30791]: cluster 2021-03-10T16:22:16.985524+0000 mgr.gibba032.whopeo (mgr.24625) 7881 : cluster [DBG] pgmap v4492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:18 gibba011 bash[31774]: cluster 2021-03-10T16:22:16.985524+0000 mgr.gibba032.whopeo (mgr.24625) 7881 : cluster [DBG] pgmap v4492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cluster 2021-03-10T16:22:18.986980+0000 mgr.gibba032.whopeo (mgr.24625) 7882 : cluster [DBG] pgmap v4493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.163781+0000 mgr.gibba032.whopeo (mgr.24625) 7883 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:22:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.165394+0000 mgr.gibba032.whopeo (mgr.24625) 7884 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:22:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.394804+0000 mon.a (mon.0) 2784 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:22:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.395133+0000 mgr.gibba032.whopeo (mgr.24625) 7885 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:22:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.396724+0000 mgr.gibba032.whopeo (mgr.24625) 7886 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:22:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.397882+0000 mgr.gibba032.whopeo (mgr.24625) 7887 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:22:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.397899+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:22:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.397919+0000 mgr.gibba032.whopeo (mgr.24625) 7888 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:22:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.399016+0000 mgr.gibba032.whopeo (mgr.24625) 7889 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:22:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.399053+0000 mgr.gibba032.whopeo (mgr.24625) 7890 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:22:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.399223+0000 mon.a (mon.0) 2786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.400077+0000 mgr.gibba032.whopeo (mgr.24625) 7891 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:22:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.400404+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.400977+0000 mgr.gibba032.whopeo (mgr.24625) 7892 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:22:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.401012+0000 mgr.gibba032.whopeo (mgr.24625) 7893 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:22:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.401454+0000 mon.a (mon.0) 2788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.402143+0000 mgr.gibba032.whopeo (mgr.24625) 7894 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:22:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.402459+0000 mgr.gibba032.whopeo (mgr.24625) 7895 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:22:20.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.403471+0000 mgr.gibba032.whopeo (mgr.24625) 7896 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:22:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.403508+0000 mgr.gibba032.whopeo (mgr.24625) 7897 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:22:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.403581+0000 mon.a (mon.0) 2789 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: cephadm 2021-03-10T16:22:19.403613+0000 mgr.gibba032.whopeo (mgr.24625) 7898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:20.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:20 gibba032 bash[24004]: audit 2021-03-10T16:22:19.404893+0000 mon.a (mon.0) 2790 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cluster 2021-03-10T16:22:18.986980+0000 mgr.gibba032.whopeo (mgr.24625) 7882 : cluster [DBG] pgmap v4493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.163781+0000 mgr.gibba032.whopeo (mgr.24625) 7883 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:22:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.165394+0000 mgr.gibba032.whopeo (mgr.24625) 7884 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:22:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.394804+0000 mon.a (mon.0) 2784 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:22:20.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.395133+0000 mgr.gibba032.whopeo (mgr.24625) 7885 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:22:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.396724+0000 mgr.gibba032.whopeo (mgr.24625) 7886 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:22:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.397882+0000 mgr.gibba032.whopeo (mgr.24625) 7887 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:22:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.397899+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:22:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.397919+0000 mgr.gibba032.whopeo (mgr.24625) 7888 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:22:20.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.399016+0000 mgr.gibba032.whopeo (mgr.24625) 7889 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:22:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.399053+0000 mgr.gibba032.whopeo (mgr.24625) 7890 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:22:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.399223+0000 mon.a (mon.0) 2786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.400077+0000 mgr.gibba032.whopeo (mgr.24625) 7891 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:22:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.400404+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.400977+0000 mgr.gibba032.whopeo (mgr.24625) 7892 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:22:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.401012+0000 mgr.gibba032.whopeo (mgr.24625) 7893 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:22:20.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.401454+0000 mon.a (mon.0) 2788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.402143+0000 mgr.gibba032.whopeo (mgr.24625) 7894 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:22:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.402459+0000 mgr.gibba032.whopeo (mgr.24625) 7895 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:22:20.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.403471+0000 mgr.gibba032.whopeo (mgr.24625) 7896 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:22:20.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.403508+0000 mgr.gibba032.whopeo (mgr.24625) 7897 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:22:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cluster 2021-03-10T16:22:18.986980+0000 mgr.gibba032.whopeo (mgr.24625) 7882 : cluster [DBG] pgmap v4493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:20.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.163781+0000 mgr.gibba032.whopeo (mgr.24625) 7883 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:22:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.165394+0000 mgr.gibba032.whopeo (mgr.24625) 7884 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:22:20.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.394804+0000 mon.a (mon.0) 2784 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:22:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.395133+0000 mgr.gibba032.whopeo (mgr.24625) 7885 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:22:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.396724+0000 mgr.gibba032.whopeo (mgr.24625) 7886 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:22:20.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.397882+0000 mgr.gibba032.whopeo (mgr.24625) 7887 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:22:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.397899+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:22:20.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.397919+0000 mgr.gibba032.whopeo (mgr.24625) 7888 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:22:20.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.399016+0000 mgr.gibba032.whopeo (mgr.24625) 7889 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:22:20.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.399053+0000 mgr.gibba032.whopeo (mgr.24625) 7890 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:22:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.399223+0000 mon.a (mon.0) 2786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.400077+0000 mgr.gibba032.whopeo (mgr.24625) 7891 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:22:20.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.400404+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.400977+0000 mgr.gibba032.whopeo (mgr.24625) 7892 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:22:20.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.401012+0000 mgr.gibba032.whopeo (mgr.24625) 7893 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:22:20.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.401454+0000 mon.a (mon.0) 2788 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.402143+0000 mgr.gibba032.whopeo (mgr.24625) 7894 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:22:20.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.402459+0000 mgr.gibba032.whopeo (mgr.24625) 7895 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:22:20.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.403471+0000 mgr.gibba032.whopeo (mgr.24625) 7896 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:22:20.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.403508+0000 mgr.gibba032.whopeo (mgr.24625) 7897 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:22:20.847 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.403581+0000 mon.a (mon.0) 2789 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.848 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: cephadm 2021-03-10T16:22:19.403613+0000 mgr.gibba032.whopeo (mgr.24625) 7898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:20.848 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[30791]: audit 2021-03-10T16:22:19.404893+0000 mon.a (mon.0) 2790 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.403581+0000 mon.a (mon.0) 2789 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:20.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: cephadm 2021-03-10T16:22:19.403613+0000 mgr.gibba032.whopeo (mgr.24625) 7898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:20.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:20 gibba011 bash[31774]: audit 2021-03-10T16:22:19.404893+0000 mon.a (mon.0) 2790 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:22 gibba032 bash[24004]: cluster 2021-03-10T16:22:20.988352+0000 mgr.gibba032.whopeo (mgr.24625) 7899 : cluster [DBG] pgmap v4494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:22 gibba011 bash[30791]: cluster 2021-03-10T16:22:20.988352+0000 mgr.gibba032.whopeo (mgr.24625) 7899 : cluster [DBG] pgmap v4494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:22 gibba011 bash[31774]: cluster 2021-03-10T16:22:20.988352+0000 mgr.gibba032.whopeo (mgr.24625) 7899 : cluster [DBG] pgmap v4494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:24 gibba032 bash[24004]: cluster 2021-03-10T16:22:22.990120+0000 mgr.gibba032.whopeo (mgr.24625) 7900 : cluster [DBG] pgmap v4495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:24 gibba011 bash[30791]: cluster 2021-03-10T16:22:22.990120+0000 mgr.gibba032.whopeo (mgr.24625) 7900 : cluster [DBG] pgmap v4495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:24 gibba011 bash[31774]: cluster 2021-03-10T16:22:22.990120+0000 mgr.gibba032.whopeo (mgr.24625) 7900 : cluster [DBG] pgmap v4495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:25.363 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:22:25.591 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:22:25.591 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:22:25.592 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 71s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:22:25.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:22:25.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:22:25.592 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:22:25.592 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:22:25.593 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:22:25.593 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:22:25.593 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:22:25.593 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 71s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:22:25.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:22:25.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:22:25.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:22:25.594 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:22:25.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:22:25.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:22:25.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:22:25.595 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:22:25.595 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 71s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:22:25.596 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:22:25.842 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:22:25.843 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:22:25.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:22:25.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:25.843 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:22:25.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:22:25.844 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:25.844 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:22:25.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:22:25.845 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:25.845 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:22:25.845 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:22:25.845 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:22:25.845 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:22:25.846 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:22:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:26 gibba032 bash[24004]: cluster 2021-03-10T16:22:24.991371+0000 mgr.gibba032.whopeo (mgr.24625) 7901 : cluster [DBG] pgmap v4496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:26 gibba032 bash[24004]: audit 2021-03-10T16:22:25.353684+0000 mgr.gibba032.whopeo (mgr.24625) 7902 : audit [DBG] from='client.59329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:26 gibba032 bash[24004]: audit 2021-03-10T16:22:25.579113+0000 mgr.gibba032.whopeo (mgr.24625) 7903 : audit [DBG] from='client.49353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:26 gibba032 bash[24004]: audit 2021-03-10T16:22:25.844225+0000 mon.a (mon.0) 2791 : audit [DBG] from='client.? 172.21.2.111:0/1746209436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[30791]: cluster 2021-03-10T16:22:24.991371+0000 mgr.gibba032.whopeo (mgr.24625) 7901 : cluster [DBG] pgmap v4496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[30791]: audit 2021-03-10T16:22:25.353684+0000 mgr.gibba032.whopeo (mgr.24625) 7902 : audit [DBG] from='client.59329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[30791]: audit 2021-03-10T16:22:25.579113+0000 mgr.gibba032.whopeo (mgr.24625) 7903 : audit [DBG] from='client.49353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[30791]: audit 2021-03-10T16:22:25.844225+0000 mon.a (mon.0) 2791 : audit [DBG] from='client.? 172.21.2.111:0/1746209436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[31774]: cluster 2021-03-10T16:22:24.991371+0000 mgr.gibba032.whopeo (mgr.24625) 7901 : cluster [DBG] pgmap v4496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[31774]: audit 2021-03-10T16:22:25.353684+0000 mgr.gibba032.whopeo (mgr.24625) 7902 : audit [DBG] from='client.59329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[31774]: audit 2021-03-10T16:22:25.579113+0000 mgr.gibba032.whopeo (mgr.24625) 7903 : audit [DBG] from='client.49353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:26 gibba011 bash[31774]: audit 2021-03-10T16:22:25.844225+0000 mon.a (mon.0) 2791 : audit [DBG] from='client.? 172.21.2.111:0/1746209436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:28 gibba032 bash[24004]: cluster 2021-03-10T16:22:26.993160+0000 mgr.gibba032.whopeo (mgr.24625) 7904 : cluster [DBG] pgmap v4497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:28 gibba011 bash[30791]: cluster 2021-03-10T16:22:26.993160+0000 mgr.gibba032.whopeo (mgr.24625) 7904 : cluster [DBG] pgmap v4497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:28 gibba011 bash[31774]: cluster 2021-03-10T16:22:26.993160+0000 mgr.gibba032.whopeo (mgr.24625) 7904 : cluster [DBG] pgmap v4497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:30 gibba032 bash[24004]: cluster 2021-03-10T16:22:28.994758+0000 mgr.gibba032.whopeo (mgr.24625) 7905 : cluster [DBG] pgmap v4498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:30 gibba011 bash[30791]: cluster 2021-03-10T16:22:28.994758+0000 mgr.gibba032.whopeo (mgr.24625) 7905 : cluster [DBG] pgmap v4498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:30 gibba011 bash[31774]: cluster 2021-03-10T16:22:28.994758+0000 mgr.gibba032.whopeo (mgr.24625) 7905 : cluster [DBG] pgmap v4498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:32 gibba032 bash[24004]: cluster 2021-03-10T16:22:30.996172+0000 mgr.gibba032.whopeo (mgr.24625) 7906 : cluster [DBG] pgmap v4499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:32 gibba011 bash[30791]: cluster 2021-03-10T16:22:30.996172+0000 mgr.gibba032.whopeo (mgr.24625) 7906 : cluster [DBG] pgmap v4499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:32 gibba011 bash[31774]: cluster 2021-03-10T16:22:30.996172+0000 mgr.gibba032.whopeo (mgr.24625) 7906 : cluster [DBG] pgmap v4499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:34 gibba032 bash[24004]: cluster 2021-03-10T16:22:32.997864+0000 mgr.gibba032.whopeo (mgr.24625) 7907 : cluster [DBG] pgmap v4500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:34 gibba011 bash[30791]: cluster 2021-03-10T16:22:32.997864+0000 mgr.gibba032.whopeo (mgr.24625) 7907 : cluster [DBG] pgmap v4500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:34 gibba011 bash[31774]: cluster 2021-03-10T16:22:32.997864+0000 mgr.gibba032.whopeo (mgr.24625) 7907 : cluster [DBG] pgmap v4500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:35.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:35 gibba032 bash[24004]: cephadm 2021-03-10T16:22:34.419658+0000 mgr.gibba032.whopeo (mgr.24625) 7908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:35.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:35 gibba011 bash[30791]: cephadm 2021-03-10T16:22:34.419658+0000 mgr.gibba032.whopeo (mgr.24625) 7908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:35.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:35 gibba011 bash[31774]: cephadm 2021-03-10T16:22:34.419658+0000 mgr.gibba032.whopeo (mgr.24625) 7908 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:36 gibba032 bash[24004]: cluster 2021-03-10T16:22:34.999217+0000 mgr.gibba032.whopeo (mgr.24625) 7909 : cluster [DBG] pgmap v4501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:36 gibba011 bash[30791]: cluster 2021-03-10T16:22:34.999217+0000 mgr.gibba032.whopeo (mgr.24625) 7909 : cluster [DBG] pgmap v4501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:36 gibba011 bash[31774]: cluster 2021-03-10T16:22:34.999217+0000 mgr.gibba032.whopeo (mgr.24625) 7909 : cluster [DBG] pgmap v4501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:38 gibba032 bash[24004]: cluster 2021-03-10T16:22:37.001003+0000 mgr.gibba032.whopeo (mgr.24625) 7910 : cluster [DBG] pgmap v4502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:38 gibba011 bash[30791]: cluster 2021-03-10T16:22:37.001003+0000 mgr.gibba032.whopeo (mgr.24625) 7910 : cluster [DBG] pgmap v4502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:38.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:38 gibba011 bash[31774]: cluster 2021-03-10T16:22:37.001003+0000 mgr.gibba032.whopeo (mgr.24625) 7910 : cluster [DBG] pgmap v4502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:40 gibba032 bash[24004]: cluster 2021-03-10T16:22:39.002571+0000 mgr.gibba032.whopeo (mgr.24625) 7911 : cluster [DBG] pgmap v4503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:40 gibba011 bash[30791]: cluster 2021-03-10T16:22:39.002571+0000 mgr.gibba032.whopeo (mgr.24625) 7911 : cluster [DBG] pgmap v4503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:40 gibba011 bash[31774]: cluster 2021-03-10T16:22:39.002571+0000 mgr.gibba032.whopeo (mgr.24625) 7911 : cluster [DBG] pgmap v4503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:42 gibba032 bash[24004]: cluster 2021-03-10T16:22:41.003937+0000 mgr.gibba032.whopeo (mgr.24625) 7912 : cluster [DBG] pgmap v4504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:42 gibba011 bash[30791]: cluster 2021-03-10T16:22:41.003937+0000 mgr.gibba032.whopeo (mgr.24625) 7912 : cluster [DBG] pgmap v4504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:42.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:42 gibba011 bash[31774]: cluster 2021-03-10T16:22:41.003937+0000 mgr.gibba032.whopeo (mgr.24625) 7912 : cluster [DBG] pgmap v4504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:44 gibba011 bash[30791]: cluster 2021-03-10T16:22:43.005845+0000 mgr.gibba032.whopeo (mgr.24625) 7913 : cluster [DBG] pgmap v4505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:44 gibba011 bash[31774]: cluster 2021-03-10T16:22:43.005845+0000 mgr.gibba032.whopeo (mgr.24625) 7913 : cluster [DBG] pgmap v4505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:44 gibba032 bash[24004]: cluster 2021-03-10T16:22:43.005845+0000 mgr.gibba032.whopeo (mgr.24625) 7913 : cluster [DBG] pgmap v4505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:46 gibba011 bash[30791]: cluster 2021-03-10T16:22:45.007037+0000 mgr.gibba032.whopeo (mgr.24625) 7914 : cluster [DBG] pgmap v4506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:46.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:46 gibba011 bash[31774]: cluster 2021-03-10T16:22:45.007037+0000 mgr.gibba032.whopeo (mgr.24625) 7914 : cluster [DBG] pgmap v4506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:46 gibba032 bash[24004]: cluster 2021-03-10T16:22:45.007037+0000 mgr.gibba032.whopeo (mgr.24625) 7914 : cluster [DBG] pgmap v4506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:48 gibba011 bash[30791]: cluster 2021-03-10T16:22:47.008576+0000 mgr.gibba032.whopeo (mgr.24625) 7915 : cluster [DBG] pgmap v4507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:48 gibba011 bash[31774]: cluster 2021-03-10T16:22:47.008576+0000 mgr.gibba032.whopeo (mgr.24625) 7915 : cluster [DBG] pgmap v4507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:48 gibba032 bash[24004]: cluster 2021-03-10T16:22:47.008576+0000 mgr.gibba032.whopeo (mgr.24625) 7915 : cluster [DBG] pgmap v4507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:50.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:50 gibba011 bash[30791]: cluster 2021-03-10T16:22:49.009969+0000 mgr.gibba032.whopeo (mgr.24625) 7916 : cluster [DBG] pgmap v4508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:50.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:50 gibba011 bash[30791]: cephadm 2021-03-10T16:22:49.435679+0000 mgr.gibba032.whopeo (mgr.24625) 7917 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:50.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:50 gibba011 bash[31774]: cluster 2021-03-10T16:22:49.009969+0000 mgr.gibba032.whopeo (mgr.24625) 7916 : cluster [DBG] pgmap v4508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:50.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:50 gibba011 bash[31774]: cephadm 2021-03-10T16:22:49.435679+0000 mgr.gibba032.whopeo (mgr.24625) 7917 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:50 gibba032 bash[24004]: cluster 2021-03-10T16:22:49.009969+0000 mgr.gibba032.whopeo (mgr.24625) 7916 : cluster [DBG] pgmap v4508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:50 gibba032 bash[24004]: cephadm 2021-03-10T16:22:49.435679+0000 mgr.gibba032.whopeo (mgr.24625) 7917 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:22:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:52 gibba011 bash[30791]: cluster 2021-03-10T16:22:51.011243+0000 mgr.gibba032.whopeo (mgr.24625) 7918 : cluster [DBG] pgmap v4509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:52 gibba011 bash[31774]: cluster 2021-03-10T16:22:51.011243+0000 mgr.gibba032.whopeo (mgr.24625) 7918 : cluster [DBG] pgmap v4509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:52 gibba032 bash[24004]: cluster 2021-03-10T16:22:51.011243+0000 mgr.gibba032.whopeo (mgr.24625) 7918 : cluster [DBG] pgmap v4509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:54 gibba011 bash[30791]: cluster 2021-03-10T16:22:53.013066+0000 mgr.gibba032.whopeo (mgr.24625) 7919 : cluster [DBG] pgmap v4510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:54 gibba011 bash[31774]: cluster 2021-03-10T16:22:53.013066+0000 mgr.gibba032.whopeo (mgr.24625) 7919 : cluster [DBG] pgmap v4510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:54 gibba032 bash[24004]: cluster 2021-03-10T16:22:53.013066+0000 mgr.gibba032.whopeo (mgr.24625) 7919 : cluster [DBG] pgmap v4510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:56.086 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:22:56.313 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:22:56.313 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:22:56.313 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 102s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:22:56.314 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:22:56.314 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:22:56.314 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:22:56.314 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:22:56.314 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:22:56.315 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:22:56.315 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:22:56.315 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 102s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:22:56.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:22:56.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:22:56.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:22:56.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:22:56.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:22:56.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:22:56.317 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:22:56.317 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 102s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:22:56.317 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 102s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:22:56.317 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:22:56.596 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:22:56.596 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:22:56.596 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:22:56.596 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:56.597 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:22:56.597 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:22:56.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:56.597 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:22:56.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:22:56.598 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:22:56.598 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:22:56.598 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:22:56.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:22:56.599 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:22:56.599 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:22:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:56 gibba011 bash[30791]: cluster 2021-03-10T16:22:55.014295+0000 mgr.gibba032.whopeo (mgr.24625) 7920 : cluster [DBG] pgmap v4511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:56 gibba011 bash[31774]: cluster 2021-03-10T16:22:55.014295+0000 mgr.gibba032.whopeo (mgr.24625) 7920 : cluster [DBG] pgmap v4511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:56 gibba032 bash[24004]: cluster 2021-03-10T16:22:55.014295+0000 mgr.gibba032.whopeo (mgr.24625) 7920 : cluster [DBG] pgmap v4511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:56.076860+0000 mgr.gibba032.whopeo (mgr.24625) 7921 : audit [DBG] from='client.59347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:56.300236+0000 mgr.gibba032.whopeo (mgr.24625) 7922 : audit [DBG] from='client.49371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:56.597857+0000 mon.a (mon.0) 2792 : audit [DBG] from='client.? 172.21.2.111:0/1760699218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:56.971794+0000 mon.a (mon.0) 2793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:22:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:56.972438+0000 mon.a (mon.0) 2794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:22:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:57.069413+0000 mon.a (mon.0) 2795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:22:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:57.070140+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:22:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:57.070798+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:22:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[30791]: audit 2021-03-10T16:22:57.071510+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:22:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:56.076860+0000 mgr.gibba032.whopeo (mgr.24625) 7921 : audit [DBG] from='client.59347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:56.300236+0000 mgr.gibba032.whopeo (mgr.24625) 7922 : audit [DBG] from='client.49371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:56.597857+0000 mon.a (mon.0) 2792 : audit [DBG] from='client.? 172.21.2.111:0/1760699218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:56.971794+0000 mon.a (mon.0) 2793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:22:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:56.972438+0000 mon.a (mon.0) 2794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:22:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:57.069413+0000 mon.a (mon.0) 2795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:22:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:57.070140+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:22:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:57.070798+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:22:57.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:57 gibba011 bash[31774]: audit 2021-03-10T16:22:57.071510+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:22:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:56.076860+0000 mgr.gibba032.whopeo (mgr.24625) 7921 : audit [DBG] from='client.59347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:56.300236+0000 mgr.gibba032.whopeo (mgr.24625) 7922 : audit [DBG] from='client.49371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:22:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:56.597857+0000 mon.a (mon.0) 2792 : audit [DBG] from='client.? 172.21.2.111:0/1760699218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:22:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:56.971794+0000 mon.a (mon.0) 2793 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:22:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:56.972438+0000 mon.a (mon.0) 2794 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:22:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:57.069413+0000 mon.a (mon.0) 2795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:22:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:57.070140+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:22:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:57.070798+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:22:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:57 gibba032 bash[24004]: audit 2021-03-10T16:22:57.071510+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:22:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:22:58 gibba032 bash[24004]: cluster 2021-03-10T16:22:57.015978+0000 mgr.gibba032.whopeo (mgr.24625) 7923 : cluster [DBG] pgmap v4512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:22:58 gibba011 bash[30791]: cluster 2021-03-10T16:22:57.015978+0000 mgr.gibba032.whopeo (mgr.24625) 7923 : cluster [DBG] pgmap v4512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:22:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:22:58 gibba011 bash[31774]: cluster 2021-03-10T16:22:57.015978+0000 mgr.gibba032.whopeo (mgr.24625) 7923 : cluster [DBG] pgmap v4512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:00 gibba032 bash[24004]: cluster 2021-03-10T16:22:59.017317+0000 mgr.gibba032.whopeo (mgr.24625) 7924 : cluster [DBG] pgmap v4513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:00 gibba011 bash[30791]: cluster 2021-03-10T16:22:59.017317+0000 mgr.gibba032.whopeo (mgr.24625) 7924 : cluster [DBG] pgmap v4513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:00 gibba011 bash[31774]: cluster 2021-03-10T16:22:59.017317+0000 mgr.gibba032.whopeo (mgr.24625) 7924 : cluster [DBG] pgmap v4513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:02 gibba032 bash[24004]: cluster 2021-03-10T16:23:01.018606+0000 mgr.gibba032.whopeo (mgr.24625) 7925 : cluster [DBG] pgmap v4514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:02 gibba011 bash[30791]: cluster 2021-03-10T16:23:01.018606+0000 mgr.gibba032.whopeo (mgr.24625) 7925 : cluster [DBG] pgmap v4514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:02 gibba011 bash[31774]: cluster 2021-03-10T16:23:01.018606+0000 mgr.gibba032.whopeo (mgr.24625) 7925 : cluster [DBG] pgmap v4514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:04 gibba032 bash[24004]: cluster 2021-03-10T16:23:03.019868+0000 mgr.gibba032.whopeo (mgr.24625) 7926 : cluster [DBG] pgmap v4515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:04 gibba011 bash[30791]: cluster 2021-03-10T16:23:03.019868+0000 mgr.gibba032.whopeo (mgr.24625) 7926 : cluster [DBG] pgmap v4515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:05.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:04 gibba011 bash[31774]: cluster 2021-03-10T16:23:03.019868+0000 mgr.gibba032.whopeo (mgr.24625) 7926 : cluster [DBG] pgmap v4515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:06.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:05 gibba032 bash[24004]: cephadm 2021-03-10T16:23:04.437287+0000 mgr.gibba032.whopeo (mgr.24625) 7927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:06.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:05 gibba011 bash[30791]: cephadm 2021-03-10T16:23:04.437287+0000 mgr.gibba032.whopeo (mgr.24625) 7927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:06.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:05 gibba011 bash[31774]: cephadm 2021-03-10T16:23:04.437287+0000 mgr.gibba032.whopeo (mgr.24625) 7927 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:06 gibba032 bash[24004]: cluster 2021-03-10T16:23:05.021110+0000 mgr.gibba032.whopeo (mgr.24625) 7928 : cluster [DBG] pgmap v4516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:06 gibba011 bash[30791]: cluster 2021-03-10T16:23:05.021110+0000 mgr.gibba032.whopeo (mgr.24625) 7928 : cluster [DBG] pgmap v4516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:06 gibba011 bash[31774]: cluster 2021-03-10T16:23:05.021110+0000 mgr.gibba032.whopeo (mgr.24625) 7928 : cluster [DBG] pgmap v4516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:08 gibba032 bash[24004]: cluster 2021-03-10T16:23:07.022635+0000 mgr.gibba032.whopeo (mgr.24625) 7929 : cluster [DBG] pgmap v4517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:08 gibba011 bash[30791]: cluster 2021-03-10T16:23:07.022635+0000 mgr.gibba032.whopeo (mgr.24625) 7929 : cluster [DBG] pgmap v4517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:08 gibba011 bash[31774]: cluster 2021-03-10T16:23:07.022635+0000 mgr.gibba032.whopeo (mgr.24625) 7929 : cluster [DBG] pgmap v4517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:10 gibba032 bash[24004]: cluster 2021-03-10T16:23:09.024169+0000 mgr.gibba032.whopeo (mgr.24625) 7930 : cluster [DBG] pgmap v4518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:10 gibba011 bash[30791]: cluster 2021-03-10T16:23:09.024169+0000 mgr.gibba032.whopeo (mgr.24625) 7930 : cluster [DBG] pgmap v4518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:10.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:10 gibba011 bash[31774]: cluster 2021-03-10T16:23:09.024169+0000 mgr.gibba032.whopeo (mgr.24625) 7930 : cluster [DBG] pgmap v4518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:12 gibba032 bash[24004]: cluster 2021-03-10T16:23:11.025569+0000 mgr.gibba032.whopeo (mgr.24625) 7931 : cluster [DBG] pgmap v4519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:12 gibba011 bash[30791]: cluster 2021-03-10T16:23:11.025569+0000 mgr.gibba032.whopeo (mgr.24625) 7931 : cluster [DBG] pgmap v4519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:12 gibba011 bash[31774]: cluster 2021-03-10T16:23:11.025569+0000 mgr.gibba032.whopeo (mgr.24625) 7931 : cluster [DBG] pgmap v4519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:14 gibba032 bash[24004]: cluster 2021-03-10T16:23:13.027206+0000 mgr.gibba032.whopeo (mgr.24625) 7932 : cluster [DBG] pgmap v4520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:14 gibba011 bash[30791]: cluster 2021-03-10T16:23:13.027206+0000 mgr.gibba032.whopeo (mgr.24625) 7932 : cluster [DBG] pgmap v4520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:14 gibba011 bash[31774]: cluster 2021-03-10T16:23:13.027206+0000 mgr.gibba032.whopeo (mgr.24625) 7932 : cluster [DBG] pgmap v4520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:16 gibba032 bash[24004]: cluster 2021-03-10T16:23:15.028486+0000 mgr.gibba032.whopeo (mgr.24625) 7933 : cluster [DBG] pgmap v4521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:16 gibba011 bash[30791]: cluster 2021-03-10T16:23:15.028486+0000 mgr.gibba032.whopeo (mgr.24625) 7933 : cluster [DBG] pgmap v4521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:16 gibba011 bash[31774]: cluster 2021-03-10T16:23:15.028486+0000 mgr.gibba032.whopeo (mgr.24625) 7933 : cluster [DBG] pgmap v4521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:18 gibba032 bash[24004]: cluster 2021-03-10T16:23:17.029991+0000 mgr.gibba032.whopeo (mgr.24625) 7934 : cluster [DBG] pgmap v4522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:18 gibba011 bash[30791]: cluster 2021-03-10T16:23:17.029991+0000 mgr.gibba032.whopeo (mgr.24625) 7934 : cluster [DBG] pgmap v4522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:18.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:18 gibba011 bash[31774]: cluster 2021-03-10T16:23:17.029991+0000 mgr.gibba032.whopeo (mgr.24625) 7934 : cluster [DBG] pgmap v4522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: cluster 2021-03-10T16:23:19.031475+0000 mgr.gibba032.whopeo (mgr.24625) 7935 : cluster [DBG] pgmap v4523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.454091+0000 mgr.gibba032.whopeo (mgr.24625) 7936 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:23:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.455634+0000 mgr.gibba032.whopeo (mgr.24625) 7937 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:23:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.669981+0000 mon.a (mon.0) 2799 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:23:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.673263+0000 mon.a (mon.0) 2800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:23:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.674812+0000 mon.a (mon.0) 2801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.675940+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.677351+0000 mon.a (mon.0) 2803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.680009+0000 mon.a (mon.0) 2804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:20 gibba032 bash[24004]: audit 2021-03-10T16:23:19.681634+0000 mon.a (mon.0) 2805 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: cluster 2021-03-10T16:23:19.031475+0000 mgr.gibba032.whopeo (mgr.24625) 7935 : cluster [DBG] pgmap v4523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.454091+0000 mgr.gibba032.whopeo (mgr.24625) 7936 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:23:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.455634+0000 mgr.gibba032.whopeo (mgr.24625) 7937 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:23:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.669981+0000 mon.a (mon.0) 2799 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:23:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.673263+0000 mon.a (mon.0) 2800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:23:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.674812+0000 mon.a (mon.0) 2801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.675940+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.677351+0000 mon.a (mon.0) 2803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.680009+0000 mon.a (mon.0) 2804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[30791]: audit 2021-03-10T16:23:19.681634+0000 mon.a (mon.0) 2805 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: cluster 2021-03-10T16:23:19.031475+0000 mgr.gibba032.whopeo (mgr.24625) 7935 : cluster [DBG] pgmap v4523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.454091+0000 mgr.gibba032.whopeo (mgr.24625) 7936 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:23:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.455634+0000 mgr.gibba032.whopeo (mgr.24625) 7937 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:23:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.669981+0000 mon.a (mon.0) 2799 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:23:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.673263+0000 mon.a (mon.0) 2800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:23:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.674812+0000 mon.a (mon.0) 2801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.675940+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.677351+0000 mon.a (mon.0) 2803 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.680009+0000 mon.a (mon.0) 2804 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:20 gibba011 bash[31774]: audit 2021-03-10T16:23:19.681634+0000 mon.a (mon.0) 2805 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:21.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.670323+0000 mgr.gibba032.whopeo (mgr.24625) 7938 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:23:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.672230+0000 mgr.gibba032.whopeo (mgr.24625) 7939 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:23:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.673480+0000 mgr.gibba032.whopeo (mgr.24625) 7940 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:23:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.673515+0000 mgr.gibba032.whopeo (mgr.24625) 7941 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:23:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7942 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:23:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.674831+0000 mgr.gibba032.whopeo (mgr.24625) 7943 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:23:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.676126+0000 mgr.gibba032.whopeo (mgr.24625) 7944 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:23:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.677304+0000 mgr.gibba032.whopeo (mgr.24625) 7945 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:23:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.677353+0000 mgr.gibba032.whopeo (mgr.24625) 7946 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:23:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.678827+0000 mgr.gibba032.whopeo (mgr.24625) 7947 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:23:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.679131+0000 mgr.gibba032.whopeo (mgr.24625) 7948 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:23:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.680466+0000 mgr.gibba032.whopeo (mgr.24625) 7949 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:23:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.680502+0000 mgr.gibba032.whopeo (mgr.24625) 7950 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:23:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:21 gibba032 bash[24004]: cephadm 2021-03-10T16:23:19.680602+0000 mgr.gibba032.whopeo (mgr.24625) 7951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.670323+0000 mgr.gibba032.whopeo (mgr.24625) 7938 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:23:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.672230+0000 mgr.gibba032.whopeo (mgr.24625) 7939 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:23:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.673480+0000 mgr.gibba032.whopeo (mgr.24625) 7940 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:23:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.673515+0000 mgr.gibba032.whopeo (mgr.24625) 7941 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:23:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7942 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:23:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.674831+0000 mgr.gibba032.whopeo (mgr.24625) 7943 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:23:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.676126+0000 mgr.gibba032.whopeo (mgr.24625) 7944 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:23:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.677304+0000 mgr.gibba032.whopeo (mgr.24625) 7945 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:23:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.677353+0000 mgr.gibba032.whopeo (mgr.24625) 7946 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:23:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.678827+0000 mgr.gibba032.whopeo (mgr.24625) 7947 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:23:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.679131+0000 mgr.gibba032.whopeo (mgr.24625) 7948 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:23:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.680466+0000 mgr.gibba032.whopeo (mgr.24625) 7949 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:23:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.680502+0000 mgr.gibba032.whopeo (mgr.24625) 7950 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:23:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[30791]: cephadm 2021-03-10T16:23:19.680602+0000 mgr.gibba032.whopeo (mgr.24625) 7951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.670323+0000 mgr.gibba032.whopeo (mgr.24625) 7938 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:23:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.672230+0000 mgr.gibba032.whopeo (mgr.24625) 7939 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:23:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.673480+0000 mgr.gibba032.whopeo (mgr.24625) 7940 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:23:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.673515+0000 mgr.gibba032.whopeo (mgr.24625) 7941 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:23:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.674797+0000 mgr.gibba032.whopeo (mgr.24625) 7942 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:23:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.674831+0000 mgr.gibba032.whopeo (mgr.24625) 7943 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:23:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.676126+0000 mgr.gibba032.whopeo (mgr.24625) 7944 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:23:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.677304+0000 mgr.gibba032.whopeo (mgr.24625) 7945 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:23:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.677353+0000 mgr.gibba032.whopeo (mgr.24625) 7946 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:23:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.678827+0000 mgr.gibba032.whopeo (mgr.24625) 7947 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:23:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.679131+0000 mgr.gibba032.whopeo (mgr.24625) 7948 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:23:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.680466+0000 mgr.gibba032.whopeo (mgr.24625) 7949 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:23:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.680502+0000 mgr.gibba032.whopeo (mgr.24625) 7950 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:23:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:21 gibba011 bash[31774]: cephadm 2021-03-10T16:23:19.680602+0000 mgr.gibba032.whopeo (mgr.24625) 7951 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:22 gibba032 bash[24004]: cluster 2021-03-10T16:23:21.032965+0000 mgr.gibba032.whopeo (mgr.24625) 7952 : cluster [DBG] pgmap v4524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:22 gibba011 bash[30791]: cluster 2021-03-10T16:23:21.032965+0000 mgr.gibba032.whopeo (mgr.24625) 7952 : cluster [DBG] pgmap v4524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:22.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:22 gibba011 bash[31774]: cluster 2021-03-10T16:23:21.032965+0000 mgr.gibba032.whopeo (mgr.24625) 7952 : cluster [DBG] pgmap v4524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:24 gibba032 bash[24004]: cluster 2021-03-10T16:23:23.033690+0000 mgr.gibba032.whopeo (mgr.24625) 7953 : cluster [DBG] pgmap v4525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:24 gibba011 bash[30791]: cluster 2021-03-10T16:23:23.033690+0000 mgr.gibba032.whopeo (mgr.24625) 7953 : cluster [DBG] pgmap v4525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:24.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:24 gibba011 bash[31774]: cluster 2021-03-10T16:23:23.033690+0000 mgr.gibba032.whopeo (mgr.24625) 7953 : cluster [DBG] pgmap v4525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:26 gibba032 bash[24004]: cluster 2021-03-10T16:23:25.034842+0000 mgr.gibba032.whopeo (mgr.24625) 7954 : cluster [DBG] pgmap v4526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:26.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:26 gibba011 bash[30791]: cluster 2021-03-10T16:23:25.034842+0000 mgr.gibba032.whopeo (mgr.24625) 7954 : cluster [DBG] pgmap v4526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:26.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:26 gibba011 bash[31774]: cluster 2021-03-10T16:23:25.034842+0000 mgr.gibba032.whopeo (mgr.24625) 7954 : cluster [DBG] pgmap v4526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:26.839 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:23:27.069 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:23:27.069 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:23:27.070 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:23:27.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:23:27.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:23:27.070 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:23:27.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:23:27.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:23:27.071 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:23:27.071 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:23:27.071 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:23:27.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:23:27.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:23:27.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:23:27.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:23:27.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:23:27.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:23:27.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:23:27.073 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:23:27.073 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:23:27.074 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:23:27.318 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:23:27.318 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:23:27.318 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:23:27.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:27.319 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:23:27.319 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:23:27.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:27.319 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:23:27.320 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:23:27.320 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:27.320 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:23:27.320 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:23:27.320 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:23:27.321 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:23:27.321 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:23:27.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:27 gibba032 bash[24004]: audit 2021-03-10T16:23:27.319560+0000 mon.c (mon.1) 151 : audit [DBG] from='client.? 172.21.2.111:0/3127852460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:27.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:27 gibba011 bash[30791]: audit 2021-03-10T16:23:27.319560+0000 mon.c (mon.1) 151 : audit [DBG] from='client.? 172.21.2.111:0/3127852460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:27.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:27 gibba011 bash[31774]: audit 2021-03-10T16:23:27.319560+0000 mon.c (mon.1) 151 : audit [DBG] from='client.? 172.21.2.111:0/3127852460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:28 gibba032 bash[24004]: audit 2021-03-10T16:23:26.829548+0000 mgr.gibba032.whopeo (mgr.24625) 7955 : audit [DBG] from='client.49383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:28 gibba032 bash[24004]: cluster 2021-03-10T16:23:27.036520+0000 mgr.gibba032.whopeo (mgr.24625) 7956 : cluster [DBG] pgmap v4527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:28 gibba032 bash[24004]: audit 2021-03-10T16:23:27.056791+0000 mgr.gibba032.whopeo (mgr.24625) 7957 : audit [DBG] from='client.59371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[30791]: audit 2021-03-10T16:23:26.829548+0000 mgr.gibba032.whopeo (mgr.24625) 7955 : audit [DBG] from='client.49383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[30791]: cluster 2021-03-10T16:23:27.036520+0000 mgr.gibba032.whopeo (mgr.24625) 7956 : cluster [DBG] pgmap v4527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:28.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[30791]: audit 2021-03-10T16:23:27.056791+0000 mgr.gibba032.whopeo (mgr.24625) 7957 : audit [DBG] from='client.59371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[31774]: audit 2021-03-10T16:23:26.829548+0000 mgr.gibba032.whopeo (mgr.24625) 7955 : audit [DBG] from='client.49383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[31774]: cluster 2021-03-10T16:23:27.036520+0000 mgr.gibba032.whopeo (mgr.24625) 7956 : cluster [DBG] pgmap v4527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:28 gibba011 bash[31774]: audit 2021-03-10T16:23:27.056791+0000 mgr.gibba032.whopeo (mgr.24625) 7957 : audit [DBG] from='client.59371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:30 gibba032 bash[24004]: cluster 2021-03-10T16:23:29.037910+0000 mgr.gibba032.whopeo (mgr.24625) 7958 : cluster [DBG] pgmap v4528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:30 gibba011 bash[30791]: cluster 2021-03-10T16:23:29.037910+0000 mgr.gibba032.whopeo (mgr.24625) 7958 : cluster [DBG] pgmap v4528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:30 gibba011 bash[31774]: cluster 2021-03-10T16:23:29.037910+0000 mgr.gibba032.whopeo (mgr.24625) 7958 : cluster [DBG] pgmap v4528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:32 gibba032 bash[24004]: cluster 2021-03-10T16:23:31.039237+0000 mgr.gibba032.whopeo (mgr.24625) 7959 : cluster [DBG] pgmap v4529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:32 gibba011 bash[30791]: cluster 2021-03-10T16:23:31.039237+0000 mgr.gibba032.whopeo (mgr.24625) 7959 : cluster [DBG] pgmap v4529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:32 gibba011 bash[31774]: cluster 2021-03-10T16:23:31.039237+0000 mgr.gibba032.whopeo (mgr.24625) 7959 : cluster [DBG] pgmap v4529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:34 gibba032 bash[24004]: cluster 2021-03-10T16:23:33.039923+0000 mgr.gibba032.whopeo (mgr.24625) 7960 : cluster [DBG] pgmap v4530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:34 gibba011 bash[30791]: cluster 2021-03-10T16:23:33.039923+0000 mgr.gibba032.whopeo (mgr.24625) 7960 : cluster [DBG] pgmap v4530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:34.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:34 gibba011 bash[31774]: cluster 2021-03-10T16:23:33.039923+0000 mgr.gibba032.whopeo (mgr.24625) 7960 : cluster [DBG] pgmap v4530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:36 gibba032 bash[24004]: cephadm 2021-03-10T16:23:34.696642+0000 mgr.gibba032.whopeo (mgr.24625) 7961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:36 gibba032 bash[24004]: cluster 2021-03-10T16:23:35.041054+0000 mgr.gibba032.whopeo (mgr.24625) 7962 : cluster [DBG] pgmap v4531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:36 gibba011 bash[30791]: cephadm 2021-03-10T16:23:34.696642+0000 mgr.gibba032.whopeo (mgr.24625) 7961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:36 gibba011 bash[30791]: cluster 2021-03-10T16:23:35.041054+0000 mgr.gibba032.whopeo (mgr.24625) 7962 : cluster [DBG] pgmap v4531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:36 gibba011 bash[31774]: cephadm 2021-03-10T16:23:34.696642+0000 mgr.gibba032.whopeo (mgr.24625) 7961 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:36 gibba011 bash[31774]: cluster 2021-03-10T16:23:35.041054+0000 mgr.gibba032.whopeo (mgr.24625) 7962 : cluster [DBG] pgmap v4531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:38 gibba032 bash[24004]: cluster 2021-03-10T16:23:37.042679+0000 mgr.gibba032.whopeo (mgr.24625) 7963 : cluster [DBG] pgmap v4532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:38 gibba011 bash[30791]: cluster 2021-03-10T16:23:37.042679+0000 mgr.gibba032.whopeo (mgr.24625) 7963 : cluster [DBG] pgmap v4532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:38.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:38 gibba011 bash[31774]: cluster 2021-03-10T16:23:37.042679+0000 mgr.gibba032.whopeo (mgr.24625) 7963 : cluster [DBG] pgmap v4532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:40 gibba032 bash[24004]: cluster 2021-03-10T16:23:39.044218+0000 mgr.gibba032.whopeo (mgr.24625) 7964 : cluster [DBG] pgmap v4533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:40 gibba011 bash[30791]: cluster 2021-03-10T16:23:39.044218+0000 mgr.gibba032.whopeo (mgr.24625) 7964 : cluster [DBG] pgmap v4533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:40 gibba011 bash[31774]: cluster 2021-03-10T16:23:39.044218+0000 mgr.gibba032.whopeo (mgr.24625) 7964 : cluster [DBG] pgmap v4533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:42 gibba032 bash[24004]: cluster 2021-03-10T16:23:41.045505+0000 mgr.gibba032.whopeo (mgr.24625) 7965 : cluster [DBG] pgmap v4534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:42 gibba011 bash[30791]: cluster 2021-03-10T16:23:41.045505+0000 mgr.gibba032.whopeo (mgr.24625) 7965 : cluster [DBG] pgmap v4534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:42.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:42 gibba011 bash[31774]: cluster 2021-03-10T16:23:41.045505+0000 mgr.gibba032.whopeo (mgr.24625) 7965 : cluster [DBG] pgmap v4534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:44 gibba032 bash[24004]: cluster 2021-03-10T16:23:43.046304+0000 mgr.gibba032.whopeo (mgr.24625) 7966 : cluster [DBG] pgmap v4535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:44 gibba011 bash[30791]: cluster 2021-03-10T16:23:43.046304+0000 mgr.gibba032.whopeo (mgr.24625) 7966 : cluster [DBG] pgmap v4535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:44 gibba011 bash[31774]: cluster 2021-03-10T16:23:43.046304+0000 mgr.gibba032.whopeo (mgr.24625) 7966 : cluster [DBG] pgmap v4535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:46 gibba032 bash[24004]: cluster 2021-03-10T16:23:45.047397+0000 mgr.gibba032.whopeo (mgr.24625) 7967 : cluster [DBG] pgmap v4536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:46 gibba011 bash[30791]: cluster 2021-03-10T16:23:45.047397+0000 mgr.gibba032.whopeo (mgr.24625) 7967 : cluster [DBG] pgmap v4536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:46.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:46 gibba011 bash[31774]: cluster 2021-03-10T16:23:45.047397+0000 mgr.gibba032.whopeo (mgr.24625) 7967 : cluster [DBG] pgmap v4536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:48 gibba011 bash[30791]: cluster 2021-03-10T16:23:47.049166+0000 mgr.gibba032.whopeo (mgr.24625) 7968 : cluster [DBG] pgmap v4537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:48 gibba011 bash[31774]: cluster 2021-03-10T16:23:47.049166+0000 mgr.gibba032.whopeo (mgr.24625) 7968 : cluster [DBG] pgmap v4537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:48 gibba032 bash[24004]: cluster 2021-03-10T16:23:47.049166+0000 mgr.gibba032.whopeo (mgr.24625) 7968 : cluster [DBG] pgmap v4537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:50.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:50 gibba011 bash[30791]: cluster 2021-03-10T16:23:49.050771+0000 mgr.gibba032.whopeo (mgr.24625) 7969 : cluster [DBG] pgmap v4538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:50.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:50 gibba011 bash[31774]: cluster 2021-03-10T16:23:49.050771+0000 mgr.gibba032.whopeo (mgr.24625) 7969 : cluster [DBG] pgmap v4538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:50 gibba032 bash[24004]: cluster 2021-03-10T16:23:49.050771+0000 mgr.gibba032.whopeo (mgr.24625) 7969 : cluster [DBG] pgmap v4538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:51.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:51 gibba011 bash[30791]: cephadm 2021-03-10T16:23:49.706112+0000 mgr.gibba032.whopeo (mgr.24625) 7970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:51.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:51 gibba011 bash[31774]: cephadm 2021-03-10T16:23:49.706112+0000 mgr.gibba032.whopeo (mgr.24625) 7970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:52.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:51 gibba032 bash[24004]: cephadm 2021-03-10T16:23:49.706112+0000 mgr.gibba032.whopeo (mgr.24625) 7970 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:23:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:52 gibba011 bash[30791]: cluster 2021-03-10T16:23:51.052146+0000 mgr.gibba032.whopeo (mgr.24625) 7971 : cluster [DBG] pgmap v4539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:52 gibba011 bash[31774]: cluster 2021-03-10T16:23:51.052146+0000 mgr.gibba032.whopeo (mgr.24625) 7971 : cluster [DBG] pgmap v4539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:52 gibba032 bash[24004]: cluster 2021-03-10T16:23:51.052146+0000 mgr.gibba032.whopeo (mgr.24625) 7971 : cluster [DBG] pgmap v4539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:54 gibba011 bash[30791]: cluster 2021-03-10T16:23:53.053890+0000 mgr.gibba032.whopeo (mgr.24625) 7972 : cluster [DBG] pgmap v4540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:54 gibba011 bash[31774]: cluster 2021-03-10T16:23:53.053890+0000 mgr.gibba032.whopeo (mgr.24625) 7972 : cluster [DBG] pgmap v4540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:54 gibba032 bash[24004]: cluster 2021-03-10T16:23:53.053890+0000 mgr.gibba032.whopeo (mgr.24625) 7972 : cluster [DBG] pgmap v4540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:56 gibba011 bash[30791]: cluster 2021-03-10T16:23:55.055315+0000 mgr.gibba032.whopeo (mgr.24625) 7973 : cluster [DBG] pgmap v4541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:56 gibba011 bash[31774]: cluster 2021-03-10T16:23:55.055315+0000 mgr.gibba032.whopeo (mgr.24625) 7973 : cluster [DBG] pgmap v4541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:56 gibba032 bash[24004]: cluster 2021-03-10T16:23:55.055315+0000 mgr.gibba032.whopeo (mgr.24625) 7973 : cluster [DBG] pgmap v4541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:57.559 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:23:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:23:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:23:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:23:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:23:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:23:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:23:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:23:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:23:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:23:57.790 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:23:57.790 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:23:57.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:23:57.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:23:57.791 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:23:57.791 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:23:57.791 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:23:57.791 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:23:57.792 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:23:57.792 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:23:57.792 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:23:57.792 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:23:57.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:56.979465+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:23:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:56.980195+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:23:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:57.079481+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:23:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:57.080344+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:23:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:57.081121+0000 mon.a (mon.0) 2810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:23:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[30791]: audit 2021-03-10T16:23:57.081953+0000 mon.a (mon.0) 2811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:23:57.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:56.979465+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:23:57.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:56.980195+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:23:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:57.079481+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:23:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:57.080344+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:23:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:57.081121+0000 mon.a (mon.0) 2810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:23:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:57 gibba011 bash[31774]: audit 2021-03-10T16:23:57.081953+0000 mon.a (mon.0) 2811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:23:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:56.979465+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:23:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:56.980195+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:23:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:57.079481+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:23:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:57.080344+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T16:23:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:57.081121+0000 mon.a (mon.0) 2810 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:23:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:57 gibba032 bash[24004]: audit 2021-03-10T16:23:57.081953+0000 mon.a (mon.0) 2811 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:23:58.040 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:23:58.041 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:23:58.041 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:23:58.041 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:58.042 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:23:58.042 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:23:58.042 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:58.043 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:23:58.043 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:23:58.043 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:23:58.044 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:23:58.044 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:23:58.044 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:23:58.045 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:23:58.045 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:23:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[30791]: cluster 2021-03-10T16:23:57.056471+0000 mgr.gibba032.whopeo (mgr.24625) 7974 : cluster [DBG] pgmap v4542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[30791]: audit 2021-03-10T16:23:57.549332+0000 mgr.gibba032.whopeo (mgr.24625) 7975 : audit [DBG] from='client.59383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[30791]: audit 2021-03-10T16:23:58.042883+0000 mon.c (mon.1) 152 : audit [DBG] from='client.? 172.21.2.111:0/1468047978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:58.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[31774]: cluster 2021-03-10T16:23:57.056471+0000 mgr.gibba032.whopeo (mgr.24625) 7974 : cluster [DBG] pgmap v4542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[31774]: audit 2021-03-10T16:23:57.549332+0000 mgr.gibba032.whopeo (mgr.24625) 7975 : audit [DBG] from='client.59383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:58 gibba011 bash[31774]: audit 2021-03-10T16:23:58.042883+0000 mon.c (mon.1) 152 : audit [DBG] from='client.? 172.21.2.111:0/1468047978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:23:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:58 gibba032 bash[24004]: cluster 2021-03-10T16:23:57.056471+0000 mgr.gibba032.whopeo (mgr.24625) 7974 : cluster [DBG] pgmap v4542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:23:59.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:58 gibba032 bash[24004]: audit 2021-03-10T16:23:57.549332+0000 mgr.gibba032.whopeo (mgr.24625) 7975 : audit [DBG] from='client.59383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:23:59.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:58 gibba032 bash[24004]: audit 2021-03-10T16:23:58.042883+0000 mon.c (mon.1) 152 : audit [DBG] from='client.? 172.21.2.111:0/1468047978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:00.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:23:59 gibba032 bash[24004]: audit 2021-03-10T16:23:57.775806+0000 mgr.gibba032.whopeo (mgr.24625) 7976 : audit [DBG] from='client.49407 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:00.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:23:59 gibba011 bash[30791]: audit 2021-03-10T16:23:57.775806+0000 mgr.gibba032.whopeo (mgr.24625) 7976 : audit [DBG] from='client.49407 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:00.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:23:59 gibba011 bash[31774]: audit 2021-03-10T16:23:57.775806+0000 mgr.gibba032.whopeo (mgr.24625) 7976 : audit [DBG] from='client.49407 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:00 gibba032 bash[24004]: cluster 2021-03-10T16:23:59.057946+0000 mgr.gibba032.whopeo (mgr.24625) 7977 : cluster [DBG] pgmap v4543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:00 gibba011 bash[30791]: cluster 2021-03-10T16:23:59.057946+0000 mgr.gibba032.whopeo (mgr.24625) 7977 : cluster [DBG] pgmap v4543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:00 gibba011 bash[31774]: cluster 2021-03-10T16:23:59.057946+0000 mgr.gibba032.whopeo (mgr.24625) 7977 : cluster [DBG] pgmap v4543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:02 gibba032 bash[24004]: cluster 2021-03-10T16:24:01.059530+0000 mgr.gibba032.whopeo (mgr.24625) 7978 : cluster [DBG] pgmap v4544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:02 gibba011 bash[30791]: cluster 2021-03-10T16:24:01.059530+0000 mgr.gibba032.whopeo (mgr.24625) 7978 : cluster [DBG] pgmap v4544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:02 gibba011 bash[31774]: cluster 2021-03-10T16:24:01.059530+0000 mgr.gibba032.whopeo (mgr.24625) 7978 : cluster [DBG] pgmap v4544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:04 gibba032 bash[24004]: cluster 2021-03-10T16:24:03.060447+0000 mgr.gibba032.whopeo (mgr.24625) 7979 : cluster [DBG] pgmap v4545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:04 gibba011 bash[30791]: cluster 2021-03-10T16:24:03.060447+0000 mgr.gibba032.whopeo (mgr.24625) 7979 : cluster [DBG] pgmap v4545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:04 gibba011 bash[31774]: cluster 2021-03-10T16:24:03.060447+0000 mgr.gibba032.whopeo (mgr.24625) 7979 : cluster [DBG] pgmap v4545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:06 gibba032 bash[24004]: cephadm 2021-03-10T16:24:04.721906+0000 mgr.gibba032.whopeo (mgr.24625) 7980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:06 gibba032 bash[24004]: cluster 2021-03-10T16:24:05.061622+0000 mgr.gibba032.whopeo (mgr.24625) 7981 : cluster [DBG] pgmap v4546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:06 gibba011 bash[30791]: cephadm 2021-03-10T16:24:04.721906+0000 mgr.gibba032.whopeo (mgr.24625) 7980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:06 gibba011 bash[30791]: cluster 2021-03-10T16:24:05.061622+0000 mgr.gibba032.whopeo (mgr.24625) 7981 : cluster [DBG] pgmap v4546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:06 gibba011 bash[31774]: cephadm 2021-03-10T16:24:04.721906+0000 mgr.gibba032.whopeo (mgr.24625) 7980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:06 gibba011 bash[31774]: cluster 2021-03-10T16:24:05.061622+0000 mgr.gibba032.whopeo (mgr.24625) 7981 : cluster [DBG] pgmap v4546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:08 gibba032 bash[24004]: cluster 2021-03-10T16:24:07.063238+0000 mgr.gibba032.whopeo (mgr.24625) 7982 : cluster [DBG] pgmap v4547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:08 gibba011 bash[30791]: cluster 2021-03-10T16:24:07.063238+0000 mgr.gibba032.whopeo (mgr.24625) 7982 : cluster [DBG] pgmap v4547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:08 gibba011 bash[31774]: cluster 2021-03-10T16:24:07.063238+0000 mgr.gibba032.whopeo (mgr.24625) 7982 : cluster [DBG] pgmap v4547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:10 gibba032 bash[24004]: cluster 2021-03-10T16:24:09.064753+0000 mgr.gibba032.whopeo (mgr.24625) 7983 : cluster [DBG] pgmap v4548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:10 gibba011 bash[30791]: cluster 2021-03-10T16:24:09.064753+0000 mgr.gibba032.whopeo (mgr.24625) 7983 : cluster [DBG] pgmap v4548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:10 gibba011 bash[31774]: cluster 2021-03-10T16:24:09.064753+0000 mgr.gibba032.whopeo (mgr.24625) 7983 : cluster [DBG] pgmap v4548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:13.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:12 gibba032 bash[24004]: cluster 2021-03-10T16:24:11.066147+0000 mgr.gibba032.whopeo (mgr.24625) 7984 : cluster [DBG] pgmap v4549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:13.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:12 gibba011 bash[30791]: cluster 2021-03-10T16:24:11.066147+0000 mgr.gibba032.whopeo (mgr.24625) 7984 : cluster [DBG] pgmap v4549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:12 gibba011 bash[31774]: cluster 2021-03-10T16:24:11.066147+0000 mgr.gibba032.whopeo (mgr.24625) 7984 : cluster [DBG] pgmap v4549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:14 gibba032 bash[24004]: cluster 2021-03-10T16:24:13.067153+0000 mgr.gibba032.whopeo (mgr.24625) 7985 : cluster [DBG] pgmap v4550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:14 gibba011 bash[30791]: cluster 2021-03-10T16:24:13.067153+0000 mgr.gibba032.whopeo (mgr.24625) 7985 : cluster [DBG] pgmap v4550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:15.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:14 gibba011 bash[31774]: cluster 2021-03-10T16:24:13.067153+0000 mgr.gibba032.whopeo (mgr.24625) 7985 : cluster [DBG] pgmap v4550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:16 gibba032 bash[24004]: cluster 2021-03-10T16:24:15.068324+0000 mgr.gibba032.whopeo (mgr.24625) 7986 : cluster [DBG] pgmap v4551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:16 gibba011 bash[30791]: cluster 2021-03-10T16:24:15.068324+0000 mgr.gibba032.whopeo (mgr.24625) 7986 : cluster [DBG] pgmap v4551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:16 gibba011 bash[31774]: cluster 2021-03-10T16:24:15.068324+0000 mgr.gibba032.whopeo (mgr.24625) 7986 : cluster [DBG] pgmap v4551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:18.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:17 gibba032 bash[24004]: cluster 2021-03-10T16:24:17.069833+0000 mgr.gibba032.whopeo (mgr.24625) 7987 : cluster [DBG] pgmap v4552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:18.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:17 gibba011 bash[30791]: cluster 2021-03-10T16:24:17.069833+0000 mgr.gibba032.whopeo (mgr.24625) 7987 : cluster [DBG] pgmap v4552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:18.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:17 gibba011 bash[31774]: cluster 2021-03-10T16:24:17.069833+0000 mgr.gibba032.whopeo (mgr.24625) 7987 : cluster [DBG] pgmap v4552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: cluster 2021-03-10T16:24:19.071463+0000 mgr.gibba032.whopeo (mgr.24625) 7988 : cluster [DBG] pgmap v4553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.945715+0000 mon.a (mon.0) 2812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:24:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.949183+0000 mon.a (mon.0) 2813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:24:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.950703+0000 mon.a (mon.0) 2814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.952227+0000 mon.a (mon.0) 2815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.953586+0000 mon.a (mon.0) 2816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.956614+0000 mon.a (mon.0) 2817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:20 gibba032 bash[24004]: audit 2021-03-10T16:24:19.958089+0000 mon.a (mon.0) 2818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: cluster 2021-03-10T16:24:19.071463+0000 mgr.gibba032.whopeo (mgr.24625) 7988 : cluster [DBG] pgmap v4553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.945715+0000 mon.a (mon.0) 2812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:24:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.949183+0000 mon.a (mon.0) 2813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:24:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.950703+0000 mon.a (mon.0) 2814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.952227+0000 mon.a (mon.0) 2815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.953586+0000 mon.a (mon.0) 2816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.956614+0000 mon.a (mon.0) 2817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[30791]: audit 2021-03-10T16:24:19.958089+0000 mon.a (mon.0) 2818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: cluster 2021-03-10T16:24:19.071463+0000 mgr.gibba032.whopeo (mgr.24625) 7988 : cluster [DBG] pgmap v4553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.945715+0000 mon.a (mon.0) 2812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:24:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.949183+0000 mon.a (mon.0) 2813 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:24:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.950703+0000 mon.a (mon.0) 2814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.952227+0000 mon.a (mon.0) 2815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.953586+0000 mon.a (mon.0) 2816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.956614+0000 mon.a (mon.0) 2817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:20 gibba011 bash[31774]: audit 2021-03-10T16:24:19.958089+0000 mon.a (mon.0) 2818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.738552+0000 mgr.gibba032.whopeo (mgr.24625) 7989 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:24:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.740120+0000 mgr.gibba032.whopeo (mgr.24625) 7990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:24:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.946238+0000 mgr.gibba032.whopeo (mgr.24625) 7991 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:24:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.948112+0000 mgr.gibba032.whopeo (mgr.24625) 7992 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:24:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.949477+0000 mgr.gibba032.whopeo (mgr.24625) 7993 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:24:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.949570+0000 mgr.gibba032.whopeo (mgr.24625) 7994 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:24:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.951041+0000 mgr.gibba032.whopeo (mgr.24625) 7995 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:24:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.951077+0000 mgr.gibba032.whopeo (mgr.24625) 7996 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:24:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.952386+0000 mgr.gibba032.whopeo (mgr.24625) 7997 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:24:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.953917+0000 mgr.gibba032.whopeo (mgr.24625) 7998 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:24:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.953953+0000 mgr.gibba032.whopeo (mgr.24625) 7999 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:24:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.955369+0000 mgr.gibba032.whopeo (mgr.24625) 8000 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:24:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.955679+0000 mgr.gibba032.whopeo (mgr.24625) 8001 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:24:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.956973+0000 mgr.gibba032.whopeo (mgr.24625) 8002 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:24:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.957008+0000 mgr.gibba032.whopeo (mgr.24625) 8003 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:24:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:21 gibba032 bash[24004]: cephadm 2021-03-10T16:24:19.957110+0000 mgr.gibba032.whopeo (mgr.24625) 8004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.738552+0000 mgr.gibba032.whopeo (mgr.24625) 7989 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:24:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.740120+0000 mgr.gibba032.whopeo (mgr.24625) 7990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:24:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.946238+0000 mgr.gibba032.whopeo (mgr.24625) 7991 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:24:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.948112+0000 mgr.gibba032.whopeo (mgr.24625) 7992 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:24:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.949477+0000 mgr.gibba032.whopeo (mgr.24625) 7993 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:24:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.949570+0000 mgr.gibba032.whopeo (mgr.24625) 7994 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:24:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.951041+0000 mgr.gibba032.whopeo (mgr.24625) 7995 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:24:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.951077+0000 mgr.gibba032.whopeo (mgr.24625) 7996 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:24:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.952386+0000 mgr.gibba032.whopeo (mgr.24625) 7997 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:24:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.953917+0000 mgr.gibba032.whopeo (mgr.24625) 7998 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:24:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.953953+0000 mgr.gibba032.whopeo (mgr.24625) 7999 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:24:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.955369+0000 mgr.gibba032.whopeo (mgr.24625) 8000 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:24:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.955679+0000 mgr.gibba032.whopeo (mgr.24625) 8001 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:24:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.956973+0000 mgr.gibba032.whopeo (mgr.24625) 8002 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:24:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.957008+0000 mgr.gibba032.whopeo (mgr.24625) 8003 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:24:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[30791]: cephadm 2021-03-10T16:24:19.957110+0000 mgr.gibba032.whopeo (mgr.24625) 8004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.738552+0000 mgr.gibba032.whopeo (mgr.24625) 7989 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:24:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.740120+0000 mgr.gibba032.whopeo (mgr.24625) 7990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:24:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.946238+0000 mgr.gibba032.whopeo (mgr.24625) 7991 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:24:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.948112+0000 mgr.gibba032.whopeo (mgr.24625) 7992 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:24:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.949477+0000 mgr.gibba032.whopeo (mgr.24625) 7993 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:24:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.949570+0000 mgr.gibba032.whopeo (mgr.24625) 7994 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:24:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.951041+0000 mgr.gibba032.whopeo (mgr.24625) 7995 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:24:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.951077+0000 mgr.gibba032.whopeo (mgr.24625) 7996 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:24:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.952386+0000 mgr.gibba032.whopeo (mgr.24625) 7997 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:24:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.953917+0000 mgr.gibba032.whopeo (mgr.24625) 7998 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:24:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.953953+0000 mgr.gibba032.whopeo (mgr.24625) 7999 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:24:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.955369+0000 mgr.gibba032.whopeo (mgr.24625) 8000 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:24:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.955679+0000 mgr.gibba032.whopeo (mgr.24625) 8001 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:24:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.956973+0000 mgr.gibba032.whopeo (mgr.24625) 8002 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:24:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.957008+0000 mgr.gibba032.whopeo (mgr.24625) 8003 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:24:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:21 gibba011 bash[31774]: cephadm 2021-03-10T16:24:19.957110+0000 mgr.gibba032.whopeo (mgr.24625) 8004 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:22 gibba032 bash[24004]: cluster 2021-03-10T16:24:21.072857+0000 mgr.gibba032.whopeo (mgr.24625) 8005 : cluster [DBG] pgmap v4554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:22 gibba011 bash[30791]: cluster 2021-03-10T16:24:21.072857+0000 mgr.gibba032.whopeo (mgr.24625) 8005 : cluster [DBG] pgmap v4554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:22.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:22 gibba011 bash[31774]: cluster 2021-03-10T16:24:21.072857+0000 mgr.gibba032.whopeo (mgr.24625) 8005 : cluster [DBG] pgmap v4554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:24 gibba032 bash[24004]: cluster 2021-03-10T16:24:23.073583+0000 mgr.gibba032.whopeo (mgr.24625) 8006 : cluster [DBG] pgmap v4555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:24 gibba011 bash[30791]: cluster 2021-03-10T16:24:23.073583+0000 mgr.gibba032.whopeo (mgr.24625) 8006 : cluster [DBG] pgmap v4555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:24 gibba011 bash[31774]: cluster 2021-03-10T16:24:23.073583+0000 mgr.gibba032.whopeo (mgr.24625) 8006 : cluster [DBG] pgmap v4555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:26 gibba032 bash[24004]: cluster 2021-03-10T16:24:25.074824+0000 mgr.gibba032.whopeo (mgr.24625) 8007 : cluster [DBG] pgmap v4556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:26 gibba011 bash[30791]: cluster 2021-03-10T16:24:25.074824+0000 mgr.gibba032.whopeo (mgr.24625) 8007 : cluster [DBG] pgmap v4556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:26 gibba011 bash[31774]: cluster 2021-03-10T16:24:25.074824+0000 mgr.gibba032.whopeo (mgr.24625) 8007 : cluster [DBG] pgmap v4556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:28.289 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:24:28.511 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:24:28.511 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:24:28.512 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:24:28.512 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:24:28.512 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:24:28.512 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:24:28.513 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:24:28.513 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:24:28.513 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:24:28.513 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:24:28.514 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:24:28.514 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:24:28.514 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:24:28.514 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:24:28.514 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:24:28.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:24:28.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:24:28.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:24:28.515 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:24:28.515 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:24:28.516 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:24:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:28 gibba032 bash[24004]: cluster 2021-03-10T16:24:27.076624+0000 mgr.gibba032.whopeo (mgr.24625) 8008 : cluster [DBG] pgmap v4557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:28.760 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:24:28.760 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:24:28.760 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:24:28.761 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:28.761 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:24:28.761 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:24:28.761 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:28.762 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:24:28.762 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:24:28.762 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:28.762 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:24:28.762 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:24:28.763 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:24:28.763 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:24:28.763 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:24:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:28 gibba011 bash[30791]: cluster 2021-03-10T16:24:27.076624+0000 mgr.gibba032.whopeo (mgr.24625) 8008 : cluster [DBG] pgmap v4557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:28.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:28 gibba011 bash[31774]: cluster 2021-03-10T16:24:27.076624+0000 mgr.gibba032.whopeo (mgr.24625) 8008 : cluster [DBG] pgmap v4557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:29.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:29 gibba032 bash[24004]: audit 2021-03-10T16:24:28.280211+0000 mgr.gibba032.whopeo (mgr.24625) 8009 : audit [DBG] from='client.59401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:29 gibba032 bash[24004]: audit 2021-03-10T16:24:28.502040+0000 mgr.gibba032.whopeo (mgr.24625) 8010 : audit [DBG] from='client.49425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:29 gibba032 bash[24004]: audit 2021-03-10T16:24:28.762082+0000 mon.c (mon.1) 153 : audit [DBG] from='client.? 172.21.2.111:0/605766193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:29.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[30791]: audit 2021-03-10T16:24:28.280211+0000 mgr.gibba032.whopeo (mgr.24625) 8009 : audit [DBG] from='client.59401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[30791]: audit 2021-03-10T16:24:28.502040+0000 mgr.gibba032.whopeo (mgr.24625) 8010 : audit [DBG] from='client.49425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[30791]: audit 2021-03-10T16:24:28.762082+0000 mon.c (mon.1) 153 : audit [DBG] from='client.? 172.21.2.111:0/605766193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:29.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[31774]: audit 2021-03-10T16:24:28.280211+0000 mgr.gibba032.whopeo (mgr.24625) 8009 : audit [DBG] from='client.59401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[31774]: audit 2021-03-10T16:24:28.502040+0000 mgr.gibba032.whopeo (mgr.24625) 8010 : audit [DBG] from='client.49425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:24:29.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:29 gibba011 bash[31774]: audit 2021-03-10T16:24:28.762082+0000 mon.c (mon.1) 153 : audit [DBG] from='client.? 172.21.2.111:0/605766193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:24:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:30 gibba032 bash[24004]: cluster 2021-03-10T16:24:29.078107+0000 mgr.gibba032.whopeo (mgr.24625) 8011 : cluster [DBG] pgmap v4558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:30 gibba011 bash[30791]: cluster 2021-03-10T16:24:29.078107+0000 mgr.gibba032.whopeo (mgr.24625) 8011 : cluster [DBG] pgmap v4558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:30 gibba011 bash[31774]: cluster 2021-03-10T16:24:29.078107+0000 mgr.gibba032.whopeo (mgr.24625) 8011 : cluster [DBG] pgmap v4558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:32 gibba032 bash[24004]: cluster 2021-03-10T16:24:31.079519+0000 mgr.gibba032.whopeo (mgr.24625) 8012 : cluster [DBG] pgmap v4559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:32 gibba011 bash[30791]: cluster 2021-03-10T16:24:31.079519+0000 mgr.gibba032.whopeo (mgr.24625) 8012 : cluster [DBG] pgmap v4559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:32 gibba011 bash[31774]: cluster 2021-03-10T16:24:31.079519+0000 mgr.gibba032.whopeo (mgr.24625) 8012 : cluster [DBG] pgmap v4559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:34 gibba032 bash[24004]: cluster 2021-03-10T16:24:33.080421+0000 mgr.gibba032.whopeo (mgr.24625) 8013 : cluster [DBG] pgmap v4560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:34 gibba011 bash[30791]: cluster 2021-03-10T16:24:33.080421+0000 mgr.gibba032.whopeo (mgr.24625) 8013 : cluster [DBG] pgmap v4560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:34 gibba011 bash[31774]: cluster 2021-03-10T16:24:33.080421+0000 mgr.gibba032.whopeo (mgr.24625) 8013 : cluster [DBG] pgmap v4560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:36 gibba032 bash[24004]: cephadm 2021-03-10T16:24:34.965115+0000 mgr.gibba032.whopeo (mgr.24625) 8014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:36 gibba032 bash[24004]: cluster 2021-03-10T16:24:35.081663+0000 mgr.gibba032.whopeo (mgr.24625) 8015 : cluster [DBG] pgmap v4561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:36 gibba011 bash[30791]: cephadm 2021-03-10T16:24:34.965115+0000 mgr.gibba032.whopeo (mgr.24625) 8014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:36 gibba011 bash[30791]: cluster 2021-03-10T16:24:35.081663+0000 mgr.gibba032.whopeo (mgr.24625) 8015 : cluster [DBG] pgmap v4561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:36 gibba011 bash[31774]: cephadm 2021-03-10T16:24:34.965115+0000 mgr.gibba032.whopeo (mgr.24625) 8014 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:36 gibba011 bash[31774]: cluster 2021-03-10T16:24:35.081663+0000 mgr.gibba032.whopeo (mgr.24625) 8015 : cluster [DBG] pgmap v4561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:38 gibba032 bash[24004]: cluster 2021-03-10T16:24:37.083437+0000 mgr.gibba032.whopeo (mgr.24625) 8016 : cluster [DBG] pgmap v4562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:38 gibba011 bash[30791]: cluster 2021-03-10T16:24:37.083437+0000 mgr.gibba032.whopeo (mgr.24625) 8016 : cluster [DBG] pgmap v4562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:38.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:38 gibba011 bash[31774]: cluster 2021-03-10T16:24:37.083437+0000 mgr.gibba032.whopeo (mgr.24625) 8016 : cluster [DBG] pgmap v4562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:40 gibba032 bash[24004]: cluster 2021-03-10T16:24:39.085048+0000 mgr.gibba032.whopeo (mgr.24625) 8017 : cluster [DBG] pgmap v4563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:40 gibba011 bash[30791]: cluster 2021-03-10T16:24:39.085048+0000 mgr.gibba032.whopeo (mgr.24625) 8017 : cluster [DBG] pgmap v4563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:40.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:40 gibba011 bash[31774]: cluster 2021-03-10T16:24:39.085048+0000 mgr.gibba032.whopeo (mgr.24625) 8017 : cluster [DBG] pgmap v4563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:42 gibba032 bash[24004]: cluster 2021-03-10T16:24:41.086614+0000 mgr.gibba032.whopeo (mgr.24625) 8018 : cluster [DBG] pgmap v4564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:42 gibba011 bash[30791]: cluster 2021-03-10T16:24:41.086614+0000 mgr.gibba032.whopeo (mgr.24625) 8018 : cluster [DBG] pgmap v4564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:42.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:42 gibba011 bash[31774]: cluster 2021-03-10T16:24:41.086614+0000 mgr.gibba032.whopeo (mgr.24625) 8018 : cluster [DBG] pgmap v4564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:44 gibba032 bash[24004]: cluster 2021-03-10T16:24:43.087424+0000 mgr.gibba032.whopeo (mgr.24625) 8019 : cluster [DBG] pgmap v4565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:44 gibba011 bash[30791]: cluster 2021-03-10T16:24:43.087424+0000 mgr.gibba032.whopeo (mgr.24625) 8019 : cluster [DBG] pgmap v4565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:44 gibba011 bash[31774]: cluster 2021-03-10T16:24:43.087424+0000 mgr.gibba032.whopeo (mgr.24625) 8019 : cluster [DBG] pgmap v4565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:46 gibba032 bash[24004]: cluster 2021-03-10T16:24:45.088683+0000 mgr.gibba032.whopeo (mgr.24625) 8020 : cluster [DBG] pgmap v4566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:46 gibba011 bash[30791]: cluster 2021-03-10T16:24:45.088683+0000 mgr.gibba032.whopeo (mgr.24625) 8020 : cluster [DBG] pgmap v4566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:46.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:46 gibba011 bash[31774]: cluster 2021-03-10T16:24:45.088683+0000 mgr.gibba032.whopeo (mgr.24625) 8020 : cluster [DBG] pgmap v4566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:48 gibba032 bash[24004]: cluster 2021-03-10T16:24:47.090408+0000 mgr.gibba032.whopeo (mgr.24625) 8021 : cluster [DBG] pgmap v4567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:48 gibba011 bash[30791]: cluster 2021-03-10T16:24:47.090408+0000 mgr.gibba032.whopeo (mgr.24625) 8021 : cluster [DBG] pgmap v4567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:48 gibba011 bash[31774]: cluster 2021-03-10T16:24:47.090408+0000 mgr.gibba032.whopeo (mgr.24625) 8021 : cluster [DBG] pgmap v4567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:50.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:50 gibba011 bash[30791]: cluster 2021-03-10T16:24:49.091894+0000 mgr.gibba032.whopeo (mgr.24625) 8022 : cluster [DBG] pgmap v4568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:50.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:50 gibba011 bash[31774]: cluster 2021-03-10T16:24:49.091894+0000 mgr.gibba032.whopeo (mgr.24625) 8022 : cluster [DBG] pgmap v4568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:50 gibba032 bash[24004]: cluster 2021-03-10T16:24:49.091894+0000 mgr.gibba032.whopeo (mgr.24625) 8022 : cluster [DBG] pgmap v4568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:51.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:51 gibba011 bash[30791]: cephadm 2021-03-10T16:24:49.981131+0000 mgr.gibba032.whopeo (mgr.24625) 8023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:51.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:51 gibba011 bash[31774]: cephadm 2021-03-10T16:24:49.981131+0000 mgr.gibba032.whopeo (mgr.24625) 8023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:52.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:51 gibba032 bash[24004]: cephadm 2021-03-10T16:24:49.981131+0000 mgr.gibba032.whopeo (mgr.24625) 8023 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:24:52.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:52 gibba011 bash[30791]: cluster 2021-03-10T16:24:51.093337+0000 mgr.gibba032.whopeo (mgr.24625) 8024 : cluster [DBG] pgmap v4569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:52 gibba011 bash[31774]: cluster 2021-03-10T16:24:51.093337+0000 mgr.gibba032.whopeo (mgr.24625) 8024 : cluster [DBG] pgmap v4569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:52 gibba032 bash[24004]: cluster 2021-03-10T16:24:51.093337+0000 mgr.gibba032.whopeo (mgr.24625) 8024 : cluster [DBG] pgmap v4569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:54 gibba011 bash[30791]: cluster 2021-03-10T16:24:53.094358+0000 mgr.gibba032.whopeo (mgr.24625) 8025 : cluster [DBG] pgmap v4570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:54 gibba011 bash[31774]: cluster 2021-03-10T16:24:53.094358+0000 mgr.gibba032.whopeo (mgr.24625) 8025 : cluster [DBG] pgmap v4570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:54 gibba032 bash[24004]: cluster 2021-03-10T16:24:53.094358+0000 mgr.gibba032.whopeo (mgr.24625) 8025 : cluster [DBG] pgmap v4570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:56 gibba011 bash[30791]: cluster 2021-03-10T16:24:55.095486+0000 mgr.gibba032.whopeo (mgr.24625) 8026 : cluster [DBG] pgmap v4571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:56 gibba011 bash[31774]: cluster 2021-03-10T16:24:55.095486+0000 mgr.gibba032.whopeo (mgr.24625) 8026 : cluster [DBG] pgmap v4571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:56 gibba032 bash[24004]: cluster 2021-03-10T16:24:55.095486+0000 mgr.gibba032.whopeo (mgr.24625) 8026 : cluster [DBG] pgmap v4571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:56.986591+0000 mon.a (mon.0) 2819 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:24:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:56.987378+0000 mon.a (mon.0) 2820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:24:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:57.089352+0000 mon.a (mon.0) 2821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:24:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:57.090188+0000 mon.a (mon.0) 2822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:24:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:57.090937+0000 mon.a (mon.0) 2823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:24:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[30791]: audit 2021-03-10T16:24:57.091771+0000 mon.a (mon.0) 2824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:24:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:56.986591+0000 mon.a (mon.0) 2819 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:24:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:56.987378+0000 mon.a (mon.0) 2820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:24:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:57.089352+0000 mon.a (mon.0) 2821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:24:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:57.090188+0000 mon.a (mon.0) 2822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:24:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:57.090937+0000 mon.a (mon.0) 2823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:24:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:57 gibba011 bash[31774]: audit 2021-03-10T16:24:57.091771+0000 mon.a (mon.0) 2824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:24:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:56.986591+0000 mon.a (mon.0) 2819 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:24:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:56.987378+0000 mon.a (mon.0) 2820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:24:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:57.089352+0000 mon.a (mon.0) 2821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:24:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:57.090188+0000 mon.a (mon.0) 2822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:24:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:57.090937+0000 mon.a (mon.0) 2823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:24:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:57 gibba032 bash[24004]: audit 2021-03-10T16:24:57.091771+0000 mon.a (mon.0) 2824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:24:58.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:24:58 gibba011 bash[30791]: cluster 2021-03-10T16:24:57.096012+0000 mgr.gibba032.whopeo (mgr.24625) 8027 : cluster [DBG] pgmap v4572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:58.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:24:58 gibba011 bash[31774]: cluster 2021-03-10T16:24:57.096012+0000 mgr.gibba032.whopeo (mgr.24625) 8027 : cluster [DBG] pgmap v4572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:59.002 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:24:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:24:58 gibba032 bash[24004]: cluster 2021-03-10T16:24:57.096012+0000 mgr.gibba032.whopeo (mgr.24625) 8027 : cluster [DBG] pgmap v4572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:24:59.227 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:24:59.228 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:24:59.228 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:24:59.228 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:24:59.228 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:24:59.229 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:24:59.229 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:24:59.229 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:24:59.229 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:24:59.229 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:24:59.230 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:24:59.230 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:24:59.230 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:24:59.230 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:24:59.231 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:24:59.231 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:24:59.231 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:24:59.231 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:24:59.232 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:24:59.232 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:24:59.232 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:24:59.476 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:24:59.476 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:24:59.477 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:24:59.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:59.477 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:24:59.477 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:24:59.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:59.478 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:24:59.478 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:24:59.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:24:59.479 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:24:59.479 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:24:59.479 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:24:59.479 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:24:59.480 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:25:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:00 gibba032 bash[24004]: audit 2021-03-10T16:24:58.992969+0000 mgr.gibba032.whopeo (mgr.24625) 8028 : audit [DBG] from='client.59419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:00 gibba032 bash[24004]: cluster 2021-03-10T16:24:59.097261+0000 mgr.gibba032.whopeo (mgr.24625) 8029 : cluster [DBG] pgmap v4573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:00.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:00 gibba032 bash[24004]: audit 2021-03-10T16:24:59.214619+0000 mgr.gibba032.whopeo (mgr.24625) 8030 : audit [DBG] from='client.59425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:00 gibba032 bash[24004]: audit 2021-03-10T16:24:59.478327+0000 mon.a (mon.0) 2825 : audit [DBG] from='client.? 172.21.2.111:0/651584110' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:00.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[30791]: audit 2021-03-10T16:24:58.992969+0000 mgr.gibba032.whopeo (mgr.24625) 8028 : audit [DBG] from='client.59419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[30791]: cluster 2021-03-10T16:24:59.097261+0000 mgr.gibba032.whopeo (mgr.24625) 8029 : cluster [DBG] pgmap v4573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[30791]: audit 2021-03-10T16:24:59.214619+0000 mgr.gibba032.whopeo (mgr.24625) 8030 : audit [DBG] from='client.59425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[30791]: audit 2021-03-10T16:24:59.478327+0000 mon.a (mon.0) 2825 : audit [DBG] from='client.? 172.21.2.111:0/651584110' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:00.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[31774]: audit 2021-03-10T16:24:58.992969+0000 mgr.gibba032.whopeo (mgr.24625) 8028 : audit [DBG] from='client.59419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[31774]: cluster 2021-03-10T16:24:59.097261+0000 mgr.gibba032.whopeo (mgr.24625) 8029 : cluster [DBG] pgmap v4573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[31774]: audit 2021-03-10T16:24:59.214619+0000 mgr.gibba032.whopeo (mgr.24625) 8030 : audit [DBG] from='client.59425 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:00 gibba011 bash[31774]: audit 2021-03-10T16:24:59.478327+0000 mon.a (mon.0) 2825 : audit [DBG] from='client.? 172.21.2.111:0/651584110' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:02 gibba032 bash[24004]: cluster 2021-03-10T16:25:01.097795+0000 mgr.gibba032.whopeo (mgr.24625) 8031 : cluster [DBG] pgmap v4574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:02.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:02 gibba011 bash[30791]: cluster 2021-03-10T16:25:01.097795+0000 mgr.gibba032.whopeo (mgr.24625) 8031 : cluster [DBG] pgmap v4574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:02.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:02 gibba011 bash[31774]: cluster 2021-03-10T16:25:01.097795+0000 mgr.gibba032.whopeo (mgr.24625) 8031 : cluster [DBG] pgmap v4574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:04 gibba032 bash[24004]: cluster 2021-03-10T16:25:03.098463+0000 mgr.gibba032.whopeo (mgr.24625) 8032 : cluster [DBG] pgmap v4575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:04.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:04 gibba011 bash[30791]: cluster 2021-03-10T16:25:03.098463+0000 mgr.gibba032.whopeo (mgr.24625) 8032 : cluster [DBG] pgmap v4575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:04 gibba011 bash[31774]: cluster 2021-03-10T16:25:03.098463+0000 mgr.gibba032.whopeo (mgr.24625) 8032 : cluster [DBG] pgmap v4575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:06 gibba032 bash[24004]: cephadm 2021-03-10T16:25:04.996922+0000 mgr.gibba032.whopeo (mgr.24625) 8033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:06 gibba032 bash[24004]: cluster 2021-03-10T16:25:05.099619+0000 mgr.gibba032.whopeo (mgr.24625) 8034 : cluster [DBG] pgmap v4576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:06 gibba011 bash[30791]: cephadm 2021-03-10T16:25:04.996922+0000 mgr.gibba032.whopeo (mgr.24625) 8033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:06 gibba011 bash[30791]: cluster 2021-03-10T16:25:05.099619+0000 mgr.gibba032.whopeo (mgr.24625) 8034 : cluster [DBG] pgmap v4576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:06.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:06 gibba011 bash[31774]: cephadm 2021-03-10T16:25:04.996922+0000 mgr.gibba032.whopeo (mgr.24625) 8033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:06 gibba011 bash[31774]: cluster 2021-03-10T16:25:05.099619+0000 mgr.gibba032.whopeo (mgr.24625) 8034 : cluster [DBG] pgmap v4576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:08 gibba032 bash[24004]: cluster 2021-03-10T16:25:07.101293+0000 mgr.gibba032.whopeo (mgr.24625) 8035 : cluster [DBG] pgmap v4577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:08 gibba011 bash[30791]: cluster 2021-03-10T16:25:07.101293+0000 mgr.gibba032.whopeo (mgr.24625) 8035 : cluster [DBG] pgmap v4577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:08 gibba011 bash[31774]: cluster 2021-03-10T16:25:07.101293+0000 mgr.gibba032.whopeo (mgr.24625) 8035 : cluster [DBG] pgmap v4577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:10 gibba032 bash[24004]: cluster 2021-03-10T16:25:09.102886+0000 mgr.gibba032.whopeo (mgr.24625) 8036 : cluster [DBG] pgmap v4578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:10 gibba011 bash[30791]: cluster 2021-03-10T16:25:09.102886+0000 mgr.gibba032.whopeo (mgr.24625) 8036 : cluster [DBG] pgmap v4578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:10 gibba011 bash[31774]: cluster 2021-03-10T16:25:09.102886+0000 mgr.gibba032.whopeo (mgr.24625) 8036 : cluster [DBG] pgmap v4578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:12 gibba032 bash[24004]: cluster 2021-03-10T16:25:11.104384+0000 mgr.gibba032.whopeo (mgr.24625) 8037 : cluster [DBG] pgmap v4579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:12 gibba011 bash[30791]: cluster 2021-03-10T16:25:11.104384+0000 mgr.gibba032.whopeo (mgr.24625) 8037 : cluster [DBG] pgmap v4579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:12.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:12 gibba011 bash[31774]: cluster 2021-03-10T16:25:11.104384+0000 mgr.gibba032.whopeo (mgr.24625) 8037 : cluster [DBG] pgmap v4579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:14 gibba032 bash[24004]: cluster 2021-03-10T16:25:13.105327+0000 mgr.gibba032.whopeo (mgr.24625) 8038 : cluster [DBG] pgmap v4580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:14 gibba011 bash[30791]: cluster 2021-03-10T16:25:13.105327+0000 mgr.gibba032.whopeo (mgr.24625) 8038 : cluster [DBG] pgmap v4580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:14.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:14 gibba011 bash[31774]: cluster 2021-03-10T16:25:13.105327+0000 mgr.gibba032.whopeo (mgr.24625) 8038 : cluster [DBG] pgmap v4580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:16 gibba032 bash[24004]: cluster 2021-03-10T16:25:15.106588+0000 mgr.gibba032.whopeo (mgr.24625) 8039 : cluster [DBG] pgmap v4581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:16 gibba011 bash[30791]: cluster 2021-03-10T16:25:15.106588+0000 mgr.gibba032.whopeo (mgr.24625) 8039 : cluster [DBG] pgmap v4581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:16 gibba011 bash[31774]: cluster 2021-03-10T16:25:15.106588+0000 mgr.gibba032.whopeo (mgr.24625) 8039 : cluster [DBG] pgmap v4581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:18 gibba032 bash[24004]: cluster 2021-03-10T16:25:17.108371+0000 mgr.gibba032.whopeo (mgr.24625) 8040 : cluster [DBG] pgmap v4582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:18 gibba011 bash[30791]: cluster 2021-03-10T16:25:17.108371+0000 mgr.gibba032.whopeo (mgr.24625) 8040 : cluster [DBG] pgmap v4582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:18.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:18 gibba011 bash[31774]: cluster 2021-03-10T16:25:17.108371+0000 mgr.gibba032.whopeo (mgr.24625) 8040 : cluster [DBG] pgmap v4582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: cluster 2021-03-10T16:25:19.110018+0000 mgr.gibba032.whopeo (mgr.24625) 8041 : cluster [DBG] pgmap v4583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.215944+0000 mon.a (mon.0) 2826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:25:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.219270+0000 mon.a (mon.0) 2827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:25:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.220680+0000 mon.a (mon.0) 2828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.222097+0000 mon.a (mon.0) 2829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.223468+0000 mon.a (mon.0) 2830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.225714+0000 mon.a (mon.0) 2831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:20 gibba032 bash[24004]: audit 2021-03-10T16:25:20.227097+0000 mon.a (mon.0) 2832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: cluster 2021-03-10T16:25:19.110018+0000 mgr.gibba032.whopeo (mgr.24625) 8041 : cluster [DBG] pgmap v4583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.215944+0000 mon.a (mon.0) 2826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:25:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.219270+0000 mon.a (mon.0) 2827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:25:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.220680+0000 mon.a (mon.0) 2828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.222097+0000 mon.a (mon.0) 2829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.223468+0000 mon.a (mon.0) 2830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.225714+0000 mon.a (mon.0) 2831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[30791]: audit 2021-03-10T16:25:20.227097+0000 mon.a (mon.0) 2832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: cluster 2021-03-10T16:25:19.110018+0000 mgr.gibba032.whopeo (mgr.24625) 8041 : cluster [DBG] pgmap v4583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.215944+0000 mon.a (mon.0) 2826 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:25:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.219270+0000 mon.a (mon.0) 2827 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:25:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.220680+0000 mon.a (mon.0) 2828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.222097+0000 mon.a (mon.0) 2829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.223468+0000 mon.a (mon.0) 2830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.225714+0000 mon.a (mon.0) 2831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:20 gibba011 bash[31774]: audit 2021-03-10T16:25:20.227097+0000 mon.a (mon.0) 2832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.012674+0000 mgr.gibba032.whopeo (mgr.24625) 8042 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:25:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.013015+0000 mgr.gibba032.whopeo (mgr.24625) 8043 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:25:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.216433+0000 mgr.gibba032.whopeo (mgr.24625) 8044 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:25:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.218169+0000 mgr.gibba032.whopeo (mgr.24625) 8045 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:25:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.219592+0000 mgr.gibba032.whopeo (mgr.24625) 8046 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:25:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.219631+0000 mgr.gibba032.whopeo (mgr.24625) 8047 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:25:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.220958+0000 mgr.gibba032.whopeo (mgr.24625) 8048 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:25:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.220995+0000 mgr.gibba032.whopeo (mgr.24625) 8049 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:25:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.222090+0000 mgr.gibba032.whopeo (mgr.24625) 8050 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:25:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.223087+0000 mgr.gibba032.whopeo (mgr.24625) 8051 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:25:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.223144+0000 mgr.gibba032.whopeo (mgr.24625) 8052 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:25:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.224381+0000 mgr.gibba032.whopeo (mgr.24625) 8053 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:25:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.224690+0000 mgr.gibba032.whopeo (mgr.24625) 8054 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:25:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.226105+0000 mgr.gibba032.whopeo (mgr.24625) 8055 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:25:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.226143+0000 mgr.gibba032.whopeo (mgr.24625) 8056 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:25:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:21 gibba032 bash[24004]: cephadm 2021-03-10T16:25:20.226245+0000 mgr.gibba032.whopeo (mgr.24625) 8057 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.012674+0000 mgr.gibba032.whopeo (mgr.24625) 8042 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:25:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.013015+0000 mgr.gibba032.whopeo (mgr.24625) 8043 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:25:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.216433+0000 mgr.gibba032.whopeo (mgr.24625) 8044 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:25:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.218169+0000 mgr.gibba032.whopeo (mgr.24625) 8045 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:25:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.219592+0000 mgr.gibba032.whopeo (mgr.24625) 8046 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:25:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.219631+0000 mgr.gibba032.whopeo (mgr.24625) 8047 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:25:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.220958+0000 mgr.gibba032.whopeo (mgr.24625) 8048 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:25:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.220995+0000 mgr.gibba032.whopeo (mgr.24625) 8049 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:25:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.222090+0000 mgr.gibba032.whopeo (mgr.24625) 8050 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:25:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.223087+0000 mgr.gibba032.whopeo (mgr.24625) 8051 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:25:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.223144+0000 mgr.gibba032.whopeo (mgr.24625) 8052 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:25:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.224381+0000 mgr.gibba032.whopeo (mgr.24625) 8053 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:25:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.224690+0000 mgr.gibba032.whopeo (mgr.24625) 8054 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:25:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.226105+0000 mgr.gibba032.whopeo (mgr.24625) 8055 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:25:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.226143+0000 mgr.gibba032.whopeo (mgr.24625) 8056 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:25:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[30791]: cephadm 2021-03-10T16:25:20.226245+0000 mgr.gibba032.whopeo (mgr.24625) 8057 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.012674+0000 mgr.gibba032.whopeo (mgr.24625) 8042 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:25:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.013015+0000 mgr.gibba032.whopeo (mgr.24625) 8043 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:25:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.216433+0000 mgr.gibba032.whopeo (mgr.24625) 8044 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:25:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.218169+0000 mgr.gibba032.whopeo (mgr.24625) 8045 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:25:21.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.219592+0000 mgr.gibba032.whopeo (mgr.24625) 8046 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:25:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.219631+0000 mgr.gibba032.whopeo (mgr.24625) 8047 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:25:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.220958+0000 mgr.gibba032.whopeo (mgr.24625) 8048 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:25:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.220995+0000 mgr.gibba032.whopeo (mgr.24625) 8049 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:25:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.222090+0000 mgr.gibba032.whopeo (mgr.24625) 8050 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:25:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.223087+0000 mgr.gibba032.whopeo (mgr.24625) 8051 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:25:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.223144+0000 mgr.gibba032.whopeo (mgr.24625) 8052 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:25:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.224381+0000 mgr.gibba032.whopeo (mgr.24625) 8053 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:25:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.224690+0000 mgr.gibba032.whopeo (mgr.24625) 8054 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:25:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.226105+0000 mgr.gibba032.whopeo (mgr.24625) 8055 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:25:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.226143+0000 mgr.gibba032.whopeo (mgr.24625) 8056 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:25:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:21 gibba011 bash[31774]: cephadm 2021-03-10T16:25:20.226245+0000 mgr.gibba032.whopeo (mgr.24625) 8057 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:22 gibba032 bash[24004]: cluster 2021-03-10T16:25:21.111844+0000 mgr.gibba032.whopeo (mgr.24625) 8058 : cluster [DBG] pgmap v4584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:22 gibba011 bash[30791]: cluster 2021-03-10T16:25:21.111844+0000 mgr.gibba032.whopeo (mgr.24625) 8058 : cluster [DBG] pgmap v4584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:22.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:22 gibba011 bash[31774]: cluster 2021-03-10T16:25:21.111844+0000 mgr.gibba032.whopeo (mgr.24625) 8058 : cluster [DBG] pgmap v4584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:24 gibba032 bash[24004]: cluster 2021-03-10T16:25:23.113139+0000 mgr.gibba032.whopeo (mgr.24625) 8059 : cluster [DBG] pgmap v4585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:24 gibba011 bash[30791]: cluster 2021-03-10T16:25:23.113139+0000 mgr.gibba032.whopeo (mgr.24625) 8059 : cluster [DBG] pgmap v4585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:24.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:24 gibba011 bash[31774]: cluster 2021-03-10T16:25:23.113139+0000 mgr.gibba032.whopeo (mgr.24625) 8059 : cluster [DBG] pgmap v4585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:26 gibba011 bash[30791]: cluster 2021-03-10T16:25:25.114498+0000 mgr.gibba032.whopeo (mgr.24625) 8060 : cluster [DBG] pgmap v4586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:26 gibba011 bash[31774]: cluster 2021-03-10T16:25:25.114498+0000 mgr.gibba032.whopeo (mgr.24625) 8060 : cluster [DBG] pgmap v4586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:26 gibba032 bash[24004]: cluster 2021-03-10T16:25:25.114498+0000 mgr.gibba032.whopeo (mgr.24625) 8060 : cluster [DBG] pgmap v4586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:28 gibba011 bash[30791]: cluster 2021-03-10T16:25:27.116255+0000 mgr.gibba032.whopeo (mgr.24625) 8061 : cluster [DBG] pgmap v4587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:28 gibba011 bash[31774]: cluster 2021-03-10T16:25:27.116255+0000 mgr.gibba032.whopeo (mgr.24625) 8061 : cluster [DBG] pgmap v4587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:28 gibba032 bash[24004]: cluster 2021-03-10T16:25:27.116255+0000 mgr.gibba032.whopeo (mgr.24625) 8061 : cluster [DBG] pgmap v4587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:29.719 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:25:29.945 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:25:29.945 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:25:29.946 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:25:29.946 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:25:29.946 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:25:29.946 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:25:29.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:25:29.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:25:29.947 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:25:29.947 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:25:29.947 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:25:29.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:25:29.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:25:29.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:25:29.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:25:29.948 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:25:29.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:25:29.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:25:29.949 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:25:29.949 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:25:29.950 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:25:30.193 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:25:30.193 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:25:30.194 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:25:30.194 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:25:30.194 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:25:30.194 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:25:30.195 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:25:30.195 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:25:30.195 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:25:30.195 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:25:30.195 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:25:30.196 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:25:30.196 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:25:30.196 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:25:30.196 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:25:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:30 gibba011 bash[30791]: cluster 2021-03-10T16:25:29.117919+0000 mgr.gibba032.whopeo (mgr.24625) 8062 : cluster [DBG] pgmap v4588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:30 gibba011 bash[30791]: audit 2021-03-10T16:25:30.195638+0000 mon.c (mon.1) 154 : audit [DBG] from='client.? 172.21.2.111:0/1642776570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:30 gibba011 bash[31774]: cluster 2021-03-10T16:25:29.117919+0000 mgr.gibba032.whopeo (mgr.24625) 8062 : cluster [DBG] pgmap v4588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:30 gibba011 bash[31774]: audit 2021-03-10T16:25:30.195638+0000 mon.c (mon.1) 154 : audit [DBG] from='client.? 172.21.2.111:0/1642776570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:30 gibba032 bash[24004]: cluster 2021-03-10T16:25:29.117919+0000 mgr.gibba032.whopeo (mgr.24625) 8062 : cluster [DBG] pgmap v4588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:30 gibba032 bash[24004]: audit 2021-03-10T16:25:30.195638+0000 mon.c (mon.1) 154 : audit [DBG] from='client.? 172.21.2.111:0/1642776570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:25:31.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:31 gibba011 bash[30791]: audit 2021-03-10T16:25:29.709740+0000 mgr.gibba032.whopeo (mgr.24625) 8063 : audit [DBG] from='client.49455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:31.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:31 gibba011 bash[30791]: audit 2021-03-10T16:25:29.932829+0000 mgr.gibba032.whopeo (mgr.24625) 8064 : audit [DBG] from='client.49461 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:31.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:31 gibba011 bash[31774]: audit 2021-03-10T16:25:29.709740+0000 mgr.gibba032.whopeo (mgr.24625) 8063 : audit [DBG] from='client.49455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:31.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:31 gibba011 bash[31774]: audit 2021-03-10T16:25:29.932829+0000 mgr.gibba032.whopeo (mgr.24625) 8064 : audit [DBG] from='client.49461 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:31 gibba032 bash[24004]: audit 2021-03-10T16:25:29.709740+0000 mgr.gibba032.whopeo (mgr.24625) 8063 : audit [DBG] from='client.49455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:31 gibba032 bash[24004]: audit 2021-03-10T16:25:29.932829+0000 mgr.gibba032.whopeo (mgr.24625) 8064 : audit [DBG] from='client.49461 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:25:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:32 gibba011 bash[30791]: cluster 2021-03-10T16:25:31.119487+0000 mgr.gibba032.whopeo (mgr.24625) 8065 : cluster [DBG] pgmap v4589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:32 gibba011 bash[31774]: cluster 2021-03-10T16:25:31.119487+0000 mgr.gibba032.whopeo (mgr.24625) 8065 : cluster [DBG] pgmap v4589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:32 gibba032 bash[24004]: cluster 2021-03-10T16:25:31.119487+0000 mgr.gibba032.whopeo (mgr.24625) 8065 : cluster [DBG] pgmap v4589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:34 gibba011 bash[30791]: cluster 2021-03-10T16:25:33.120850+0000 mgr.gibba032.whopeo (mgr.24625) 8066 : cluster [DBG] pgmap v4590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:34 gibba011 bash[31774]: cluster 2021-03-10T16:25:33.120850+0000 mgr.gibba032.whopeo (mgr.24625) 8066 : cluster [DBG] pgmap v4590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:35.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:34 gibba032 bash[24004]: cluster 2021-03-10T16:25:33.120850+0000 mgr.gibba032.whopeo (mgr.24625) 8066 : cluster [DBG] pgmap v4590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:36 gibba011 bash[30791]: cluster 2021-03-10T16:25:35.122214+0000 mgr.gibba032.whopeo (mgr.24625) 8067 : cluster [DBG] pgmap v4591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:36 gibba011 bash[30791]: cephadm 2021-03-10T16:25:35.242228+0000 mgr.gibba032.whopeo (mgr.24625) 8068 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:36 gibba011 bash[31774]: cluster 2021-03-10T16:25:35.122214+0000 mgr.gibba032.whopeo (mgr.24625) 8067 : cluster [DBG] pgmap v4591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:36 gibba011 bash[31774]: cephadm 2021-03-10T16:25:35.242228+0000 mgr.gibba032.whopeo (mgr.24625) 8068 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:36 gibba032 bash[24004]: cluster 2021-03-10T16:25:35.122214+0000 mgr.gibba032.whopeo (mgr.24625) 8067 : cluster [DBG] pgmap v4591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:36 gibba032 bash[24004]: cephadm 2021-03-10T16:25:35.242228+0000 mgr.gibba032.whopeo (mgr.24625) 8068 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:38 gibba011 bash[30791]: cluster 2021-03-10T16:25:37.123678+0000 mgr.gibba032.whopeo (mgr.24625) 8069 : cluster [DBG] pgmap v4592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:38 gibba011 bash[31774]: cluster 2021-03-10T16:25:37.123678+0000 mgr.gibba032.whopeo (mgr.24625) 8069 : cluster [DBG] pgmap v4592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:38 gibba032 bash[24004]: cluster 2021-03-10T16:25:37.123678+0000 mgr.gibba032.whopeo (mgr.24625) 8069 : cluster [DBG] pgmap v4592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:40 gibba011 bash[30791]: cluster 2021-03-10T16:25:39.125283+0000 mgr.gibba032.whopeo (mgr.24625) 8070 : cluster [DBG] pgmap v4593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:40 gibba011 bash[31774]: cluster 2021-03-10T16:25:39.125283+0000 mgr.gibba032.whopeo (mgr.24625) 8070 : cluster [DBG] pgmap v4593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:41.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:40 gibba032 bash[24004]: cluster 2021-03-10T16:25:39.125283+0000 mgr.gibba032.whopeo (mgr.24625) 8070 : cluster [DBG] pgmap v4593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:42 gibba032 bash[24004]: cluster 2021-03-10T16:25:41.126816+0000 mgr.gibba032.whopeo (mgr.24625) 8071 : cluster [DBG] pgmap v4594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:43.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:42 gibba011 bash[30791]: cluster 2021-03-10T16:25:41.126816+0000 mgr.gibba032.whopeo (mgr.24625) 8071 : cluster [DBG] pgmap v4594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:42 gibba011 bash[31774]: cluster 2021-03-10T16:25:41.126816+0000 mgr.gibba032.whopeo (mgr.24625) 8071 : cluster [DBG] pgmap v4594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:44 gibba032 bash[24004]: cluster 2021-03-10T16:25:43.128593+0000 mgr.gibba032.whopeo (mgr.24625) 8072 : cluster [DBG] pgmap v4595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:44 gibba011 bash[30791]: cluster 2021-03-10T16:25:43.128593+0000 mgr.gibba032.whopeo (mgr.24625) 8072 : cluster [DBG] pgmap v4595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:44 gibba011 bash[31774]: cluster 2021-03-10T16:25:43.128593+0000 mgr.gibba032.whopeo (mgr.24625) 8072 : cluster [DBG] pgmap v4595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:46 gibba032 bash[24004]: cluster 2021-03-10T16:25:45.129989+0000 mgr.gibba032.whopeo (mgr.24625) 8073 : cluster [DBG] pgmap v4596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:46 gibba011 bash[30791]: cluster 2021-03-10T16:25:45.129989+0000 mgr.gibba032.whopeo (mgr.24625) 8073 : cluster [DBG] pgmap v4596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:46 gibba011 bash[31774]: cluster 2021-03-10T16:25:45.129989+0000 mgr.gibba032.whopeo (mgr.24625) 8073 : cluster [DBG] pgmap v4596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:48 gibba032 bash[24004]: cluster 2021-03-10T16:25:47.131513+0000 mgr.gibba032.whopeo (mgr.24625) 8074 : cluster [DBG] pgmap v4597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:48 gibba011 bash[30791]: cluster 2021-03-10T16:25:47.131513+0000 mgr.gibba032.whopeo (mgr.24625) 8074 : cluster [DBG] pgmap v4597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:49.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:48 gibba011 bash[31774]: cluster 2021-03-10T16:25:47.131513+0000 mgr.gibba032.whopeo (mgr.24625) 8074 : cluster [DBG] pgmap v4597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:50 gibba032 bash[24004]: cluster 2021-03-10T16:25:49.133145+0000 mgr.gibba032.whopeo (mgr.24625) 8075 : cluster [DBG] pgmap v4598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:50 gibba011 bash[30791]: cluster 2021-03-10T16:25:49.133145+0000 mgr.gibba032.whopeo (mgr.24625) 8075 : cluster [DBG] pgmap v4598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:50 gibba011 bash[31774]: cluster 2021-03-10T16:25:49.133145+0000 mgr.gibba032.whopeo (mgr.24625) 8075 : cluster [DBG] pgmap v4598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:52.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:51 gibba032 bash[24004]: cephadm 2021-03-10T16:25:50.258137+0000 mgr.gibba032.whopeo (mgr.24625) 8076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:52.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:51 gibba011 bash[30791]: cephadm 2021-03-10T16:25:50.258137+0000 mgr.gibba032.whopeo (mgr.24625) 8076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:52.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:51 gibba011 bash[31774]: cephadm 2021-03-10T16:25:50.258137+0000 mgr.gibba032.whopeo (mgr.24625) 8076 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:25:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:52 gibba032 bash[24004]: cluster 2021-03-10T16:25:51.134863+0000 mgr.gibba032.whopeo (mgr.24625) 8077 : cluster [DBG] pgmap v4599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:52 gibba011 bash[30791]: cluster 2021-03-10T16:25:51.134863+0000 mgr.gibba032.whopeo (mgr.24625) 8077 : cluster [DBG] pgmap v4599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:52 gibba011 bash[31774]: cluster 2021-03-10T16:25:51.134863+0000 mgr.gibba032.whopeo (mgr.24625) 8077 : cluster [DBG] pgmap v4599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:54 gibba032 bash[24004]: cluster 2021-03-10T16:25:53.136716+0000 mgr.gibba032.whopeo (mgr.24625) 8078 : cluster [DBG] pgmap v4600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:54 gibba011 bash[30791]: cluster 2021-03-10T16:25:53.136716+0000 mgr.gibba032.whopeo (mgr.24625) 8078 : cluster [DBG] pgmap v4600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:54 gibba011 bash[31774]: cluster 2021-03-10T16:25:53.136716+0000 mgr.gibba032.whopeo (mgr.24625) 8078 : cluster [DBG] pgmap v4600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:56 gibba032 bash[24004]: cluster 2021-03-10T16:25:55.138222+0000 mgr.gibba032.whopeo (mgr.24625) 8079 : cluster [DBG] pgmap v4601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:56 gibba011 bash[30791]: cluster 2021-03-10T16:25:55.138222+0000 mgr.gibba032.whopeo (mgr.24625) 8079 : cluster [DBG] pgmap v4601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:56 gibba011 bash[31774]: cluster 2021-03-10T16:25:55.138222+0000 mgr.gibba032.whopeo (mgr.24625) 8079 : cluster [DBG] pgmap v4601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:56.994259+0000 mon.a (mon.0) 2833 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:25:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:56.994893+0000 mon.a (mon.0) 2834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:25:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:57.098852+0000 mon.a (mon.0) 2835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:25:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:57.099668+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:57.100353+0000 mon.a (mon.0) 2837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:57 gibba032 bash[24004]: audit 2021-03-10T16:25:57.101021+0000 mon.a (mon.0) 2838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:25:58.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:56.994259+0000 mon.a (mon.0) 2833 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:25:58.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:56.994893+0000 mon.a (mon.0) 2834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:25:58.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:57.098852+0000 mon.a (mon.0) 2835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:25:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:57.099668+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:57.100353+0000 mon.a (mon.0) 2837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[31774]: audit 2021-03-10T16:25:57.101021+0000 mon.a (mon.0) 2838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:25:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:56.994259+0000 mon.a (mon.0) 2833 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:25:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:56.994893+0000 mon.a (mon.0) 2834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:25:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:57.098852+0000 mon.a (mon.0) 2835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:25:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:57.099668+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:57.100353+0000 mon.a (mon.0) 2837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:25:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:57 gibba011 bash[30791]: audit 2021-03-10T16:25:57.101021+0000 mon.a (mon.0) 2838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:25:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:25:58 gibba032 bash[24004]: cluster 2021-03-10T16:25:57.139095+0000 mgr.gibba032.whopeo (mgr.24625) 8080 : cluster [DBG] pgmap v4602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:25:58 gibba011 bash[30791]: cluster 2021-03-10T16:25:57.139095+0000 mgr.gibba032.whopeo (mgr.24625) 8080 : cluster [DBG] pgmap v4602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:25:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:25:58 gibba011 bash[31774]: cluster 2021-03-10T16:25:57.139095+0000 mgr.gibba032.whopeo (mgr.24625) 8080 : cluster [DBG] pgmap v4602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:00.437 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:26:00.663 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:26:00.663 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:26:00.663 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:26:00.664 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:26:00.664 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:26:00.664 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:26:00.664 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:26:00.665 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:26:00.665 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:26:00.665 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:26:00.665 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:26:00.666 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:26:00.666 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:26:00.666 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:26:00.666 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:26:00.666 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:26:00.667 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:26:00.667 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:26:00.667 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:26:00.667 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:26:00.668 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:26:00.916 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:26:00.916 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:26:00.916 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:26:00.917 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:00.917 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:26:00.917 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:26:00.917 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:00.918 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:26:00.918 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:26:00.918 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:00.919 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:26:00.919 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:26:00.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:26:00.919 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:26:00.919 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:26:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:00 gibba032 bash[24004]: cluster 2021-03-10T16:25:59.140687+0000 mgr.gibba032.whopeo (mgr.24625) 8081 : cluster [DBG] pgmap v4603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:00 gibba011 bash[30791]: cluster 2021-03-10T16:25:59.140687+0000 mgr.gibba032.whopeo (mgr.24625) 8081 : cluster [DBG] pgmap v4603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:00 gibba011 bash[31774]: cluster 2021-03-10T16:25:59.140687+0000 mgr.gibba032.whopeo (mgr.24625) 8081 : cluster [DBG] pgmap v4603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:02.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:01 gibba032 bash[24004]: audit 2021-03-10T16:26:00.427033+0000 mgr.gibba032.whopeo (mgr.24625) 8082 : audit [DBG] from='client.49473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:01 gibba032 bash[24004]: audit 2021-03-10T16:26:00.650418+0000 mgr.gibba032.whopeo (mgr.24625) 8083 : audit [DBG] from='client.49479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:01 gibba032 bash[24004]: audit 2021-03-10T16:26:00.918144+0000 mon.a (mon.0) 2839 : audit [DBG] from='client.? 172.21.2.111:0/4114842298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:02.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[30791]: audit 2021-03-10T16:26:00.427033+0000 mgr.gibba032.whopeo (mgr.24625) 8082 : audit [DBG] from='client.49473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[30791]: audit 2021-03-10T16:26:00.650418+0000 mgr.gibba032.whopeo (mgr.24625) 8083 : audit [DBG] from='client.49479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[30791]: audit 2021-03-10T16:26:00.918144+0000 mon.a (mon.0) 2839 : audit [DBG] from='client.? 172.21.2.111:0/4114842298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:02.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[31774]: audit 2021-03-10T16:26:00.427033+0000 mgr.gibba032.whopeo (mgr.24625) 8082 : audit [DBG] from='client.49473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[31774]: audit 2021-03-10T16:26:00.650418+0000 mgr.gibba032.whopeo (mgr.24625) 8083 : audit [DBG] from='client.49479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:02.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:01 gibba011 bash[31774]: audit 2021-03-10T16:26:00.918144+0000 mon.a (mon.0) 2839 : audit [DBG] from='client.? 172.21.2.111:0/4114842298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:02 gibba032 bash[24004]: cluster 2021-03-10T16:26:01.142326+0000 mgr.gibba032.whopeo (mgr.24625) 8084 : cluster [DBG] pgmap v4604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:03.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:02 gibba011 bash[30791]: cluster 2021-03-10T16:26:01.142326+0000 mgr.gibba032.whopeo (mgr.24625) 8084 : cluster [DBG] pgmap v4604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:02 gibba011 bash[31774]: cluster 2021-03-10T16:26:01.142326+0000 mgr.gibba032.whopeo (mgr.24625) 8084 : cluster [DBG] pgmap v4604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:04.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:03 gibba032 bash[24004]: cluster 2021-03-10T16:26:03.144271+0000 mgr.gibba032.whopeo (mgr.24625) 8085 : cluster [DBG] pgmap v4605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:04.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:03 gibba011 bash[30791]: cluster 2021-03-10T16:26:03.144271+0000 mgr.gibba032.whopeo (mgr.24625) 8085 : cluster [DBG] pgmap v4605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:04.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:03 gibba011 bash[31774]: cluster 2021-03-10T16:26:03.144271+0000 mgr.gibba032.whopeo (mgr.24625) 8085 : cluster [DBG] pgmap v4605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:06 gibba032 bash[24004]: cluster 2021-03-10T16:26:05.146031+0000 mgr.gibba032.whopeo (mgr.24625) 8086 : cluster [DBG] pgmap v4606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:06 gibba032 bash[24004]: cephadm 2021-03-10T16:26:05.267561+0000 mgr.gibba032.whopeo (mgr.24625) 8087 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:06 gibba011 bash[30791]: cluster 2021-03-10T16:26:05.146031+0000 mgr.gibba032.whopeo (mgr.24625) 8086 : cluster [DBG] pgmap v4606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:06 gibba011 bash[30791]: cephadm 2021-03-10T16:26:05.267561+0000 mgr.gibba032.whopeo (mgr.24625) 8087 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:06 gibba011 bash[31774]: cluster 2021-03-10T16:26:05.146031+0000 mgr.gibba032.whopeo (mgr.24625) 8086 : cluster [DBG] pgmap v4606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:06 gibba011 bash[31774]: cephadm 2021-03-10T16:26:05.267561+0000 mgr.gibba032.whopeo (mgr.24625) 8087 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:08 gibba032 bash[24004]: cluster 2021-03-10T16:26:07.147767+0000 mgr.gibba032.whopeo (mgr.24625) 8088 : cluster [DBG] pgmap v4607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:08 gibba011 bash[30791]: cluster 2021-03-10T16:26:07.147767+0000 mgr.gibba032.whopeo (mgr.24625) 8088 : cluster [DBG] pgmap v4607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:08.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:08 gibba011 bash[31774]: cluster 2021-03-10T16:26:07.147767+0000 mgr.gibba032.whopeo (mgr.24625) 8088 : cluster [DBG] pgmap v4607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:10 gibba032 bash[24004]: cluster 2021-03-10T16:26:09.149592+0000 mgr.gibba032.whopeo (mgr.24625) 8089 : cluster [DBG] pgmap v4608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:10 gibba011 bash[30791]: cluster 2021-03-10T16:26:09.149592+0000 mgr.gibba032.whopeo (mgr.24625) 8089 : cluster [DBG] pgmap v4608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:10.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:10 gibba011 bash[31774]: cluster 2021-03-10T16:26:09.149592+0000 mgr.gibba032.whopeo (mgr.24625) 8089 : cluster [DBG] pgmap v4608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:12 gibba032 bash[24004]: cluster 2021-03-10T16:26:11.151454+0000 mgr.gibba032.whopeo (mgr.24625) 8090 : cluster [DBG] pgmap v4609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:12 gibba011 bash[30791]: cluster 2021-03-10T16:26:11.151454+0000 mgr.gibba032.whopeo (mgr.24625) 8090 : cluster [DBG] pgmap v4609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:12.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:12 gibba011 bash[31774]: cluster 2021-03-10T16:26:11.151454+0000 mgr.gibba032.whopeo (mgr.24625) 8090 : cluster [DBG] pgmap v4609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:14 gibba032 bash[24004]: cluster 2021-03-10T16:26:13.153254+0000 mgr.gibba032.whopeo (mgr.24625) 8091 : cluster [DBG] pgmap v4610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:14 gibba011 bash[30791]: cluster 2021-03-10T16:26:13.153254+0000 mgr.gibba032.whopeo (mgr.24625) 8091 : cluster [DBG] pgmap v4610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:14.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:14 gibba011 bash[31774]: cluster 2021-03-10T16:26:13.153254+0000 mgr.gibba032.whopeo (mgr.24625) 8091 : cluster [DBG] pgmap v4610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:16 gibba032 bash[24004]: cluster 2021-03-10T16:26:15.154951+0000 mgr.gibba032.whopeo (mgr.24625) 8092 : cluster [DBG] pgmap v4611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:16 gibba011 bash[30791]: cluster 2021-03-10T16:26:15.154951+0000 mgr.gibba032.whopeo (mgr.24625) 8092 : cluster [DBG] pgmap v4611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:16 gibba011 bash[31774]: cluster 2021-03-10T16:26:15.154951+0000 mgr.gibba032.whopeo (mgr.24625) 8092 : cluster [DBG] pgmap v4611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:18 gibba032 bash[24004]: cluster 2021-03-10T16:26:17.156310+0000 mgr.gibba032.whopeo (mgr.24625) 8093 : cluster [DBG] pgmap v4612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:18 gibba011 bash[30791]: cluster 2021-03-10T16:26:17.156310+0000 mgr.gibba032.whopeo (mgr.24625) 8093 : cluster [DBG] pgmap v4612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:18.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:18 gibba011 bash[31774]: cluster 2021-03-10T16:26:17.156310+0000 mgr.gibba032.whopeo (mgr.24625) 8093 : cluster [DBG] pgmap v4612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:20.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:20 gibba032 bash[24004]: cluster 2021-03-10T16:26:19.157808+0000 mgr.gibba032.whopeo (mgr.24625) 8094 : cluster [DBG] pgmap v4613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:20.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:20 gibba011 bash[30791]: cluster 2021-03-10T16:26:19.157808+0000 mgr.gibba032.whopeo (mgr.24625) 8094 : cluster [DBG] pgmap v4613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:20.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:20 gibba011 bash[31774]: cluster 2021-03-10T16:26:19.157808+0000 mgr.gibba032.whopeo (mgr.24625) 8094 : cluster [DBG] pgmap v4613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.284394+0000 mgr.gibba032.whopeo (mgr.24625) 8095 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:26:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.285992+0000 mgr.gibba032.whopeo (mgr.24625) 8096 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:26:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.497522+0000 mon.a (mon.0) 2840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:26:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.497861+0000 mgr.gibba032.whopeo (mgr.24625) 8097 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:26:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.499816+0000 mgr.gibba032.whopeo (mgr.24625) 8098 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:26:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.500877+0000 mon.a (mon.0) 2841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:26:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.501251+0000 mgr.gibba032.whopeo (mgr.24625) 8099 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:26:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.501290+0000 mgr.gibba032.whopeo (mgr.24625) 8100 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:26:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.502416+0000 mon.a (mon.0) 2842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 8101 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:26:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.502895+0000 mgr.gibba032.whopeo (mgr.24625) 8102 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:26:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.504018+0000 mon.a (mon.0) 2843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.504351+0000 mgr.gibba032.whopeo (mgr.24625) 8103 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:26:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.505448+0000 mon.a (mon.0) 2844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.505775+0000 mgr.gibba032.whopeo (mgr.24625) 8104 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:26:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.505816+0000 mgr.gibba032.whopeo (mgr.24625) 8105 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:26:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.507477+0000 mgr.gibba032.whopeo (mgr.24625) 8106 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:26:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.507831+0000 mgr.gibba032.whopeo (mgr.24625) 8107 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:26:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.508535+0000 mon.a (mon.0) 2845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.509191+0000 mgr.gibba032.whopeo (mgr.24625) 8108 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:26:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.509235+0000 mgr.gibba032.whopeo (mgr.24625) 8109 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:26:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: cephadm 2021-03-10T16:26:20.509358+0000 mgr.gibba032.whopeo (mgr.24625) 8110 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:21 gibba032 bash[24004]: audit 2021-03-10T16:26:20.510381+0000 mon.a (mon.0) 2846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.284394+0000 mgr.gibba032.whopeo (mgr.24625) 8095 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:26:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.285992+0000 mgr.gibba032.whopeo (mgr.24625) 8096 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:26:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.497522+0000 mon.a (mon.0) 2840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:26:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.497861+0000 mgr.gibba032.whopeo (mgr.24625) 8097 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:26:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.499816+0000 mgr.gibba032.whopeo (mgr.24625) 8098 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:26:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.500877+0000 mon.a (mon.0) 2841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:26:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.501251+0000 mgr.gibba032.whopeo (mgr.24625) 8099 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:26:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.501290+0000 mgr.gibba032.whopeo (mgr.24625) 8100 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:26:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.502416+0000 mon.a (mon.0) 2842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 8101 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:26:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.502895+0000 mgr.gibba032.whopeo (mgr.24625) 8102 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:26:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.504018+0000 mon.a (mon.0) 2843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.504351+0000 mgr.gibba032.whopeo (mgr.24625) 8103 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:26:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.505448+0000 mon.a (mon.0) 2844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.505775+0000 mgr.gibba032.whopeo (mgr.24625) 8104 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:26:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.505816+0000 mgr.gibba032.whopeo (mgr.24625) 8105 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:26:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.507477+0000 mgr.gibba032.whopeo (mgr.24625) 8106 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:26:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.507831+0000 mgr.gibba032.whopeo (mgr.24625) 8107 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:26:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.508535+0000 mon.a (mon.0) 2845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.509191+0000 mgr.gibba032.whopeo (mgr.24625) 8108 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:26:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.509235+0000 mgr.gibba032.whopeo (mgr.24625) 8109 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:26:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.284394+0000 mgr.gibba032.whopeo (mgr.24625) 8095 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:26:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.285992+0000 mgr.gibba032.whopeo (mgr.24625) 8096 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:26:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.497522+0000 mon.a (mon.0) 2840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:26:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.497861+0000 mgr.gibba032.whopeo (mgr.24625) 8097 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:26:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.499816+0000 mgr.gibba032.whopeo (mgr.24625) 8098 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:26:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.500877+0000 mon.a (mon.0) 2841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:26:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.501251+0000 mgr.gibba032.whopeo (mgr.24625) 8099 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:26:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.501290+0000 mgr.gibba032.whopeo (mgr.24625) 8100 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:26:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.502416+0000 mon.a (mon.0) 2842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 8101 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:26:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.502895+0000 mgr.gibba032.whopeo (mgr.24625) 8102 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:26:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.504018+0000 mon.a (mon.0) 2843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.504351+0000 mgr.gibba032.whopeo (mgr.24625) 8103 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:26:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.505448+0000 mon.a (mon.0) 2844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.505775+0000 mgr.gibba032.whopeo (mgr.24625) 8104 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:26:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.505816+0000 mgr.gibba032.whopeo (mgr.24625) 8105 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:26:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.507477+0000 mgr.gibba032.whopeo (mgr.24625) 8106 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:26:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.507831+0000 mgr.gibba032.whopeo (mgr.24625) 8107 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:26:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.508535+0000 mon.a (mon.0) 2845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.509191+0000 mgr.gibba032.whopeo (mgr.24625) 8108 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:26:21.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.509235+0000 mgr.gibba032.whopeo (mgr.24625) 8109 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:26:21.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: cephadm 2021-03-10T16:26:20.509358+0000 mgr.gibba032.whopeo (mgr.24625) 8110 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:21.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[30791]: audit 2021-03-10T16:26:20.510381+0000 mon.a (mon.0) 2846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:21.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: cephadm 2021-03-10T16:26:20.509358+0000 mgr.gibba032.whopeo (mgr.24625) 8110 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:21.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:21 gibba011 bash[31774]: audit 2021-03-10T16:26:20.510381+0000 mon.a (mon.0) 2846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:22 gibba011 bash[30791]: cluster 2021-03-10T16:26:21.159327+0000 mgr.gibba032.whopeo (mgr.24625) 8111 : cluster [DBG] pgmap v4614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:22 gibba011 bash[31774]: cluster 2021-03-10T16:26:21.159327+0000 mgr.gibba032.whopeo (mgr.24625) 8111 : cluster [DBG] pgmap v4614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:23.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:22 gibba032 bash[24004]: cluster 2021-03-10T16:26:21.159327+0000 mgr.gibba032.whopeo (mgr.24625) 8111 : cluster [DBG] pgmap v4614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:24 gibba011 bash[30791]: cluster 2021-03-10T16:26:23.161036+0000 mgr.gibba032.whopeo (mgr.24625) 8112 : cluster [DBG] pgmap v4615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:24 gibba011 bash[31774]: cluster 2021-03-10T16:26:23.161036+0000 mgr.gibba032.whopeo (mgr.24625) 8112 : cluster [DBG] pgmap v4615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:25.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:24 gibba032 bash[24004]: cluster 2021-03-10T16:26:23.161036+0000 mgr.gibba032.whopeo (mgr.24625) 8112 : cluster [DBG] pgmap v4615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:26 gibba011 bash[30791]: cluster 2021-03-10T16:26:25.162759+0000 mgr.gibba032.whopeo (mgr.24625) 8113 : cluster [DBG] pgmap v4616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:26.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:26 gibba011 bash[31774]: cluster 2021-03-10T16:26:25.162759+0000 mgr.gibba032.whopeo (mgr.24625) 8113 : cluster [DBG] pgmap v4616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:26 gibba032 bash[24004]: cluster 2021-03-10T16:26:25.162759+0000 mgr.gibba032.whopeo (mgr.24625) 8113 : cluster [DBG] pgmap v4616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:28 gibba011 bash[30791]: cluster 2021-03-10T16:26:27.164429+0000 mgr.gibba032.whopeo (mgr.24625) 8114 : cluster [DBG] pgmap v4617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:28 gibba011 bash[31774]: cluster 2021-03-10T16:26:27.164429+0000 mgr.gibba032.whopeo (mgr.24625) 8114 : cluster [DBG] pgmap v4617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:28 gibba032 bash[24004]: cluster 2021-03-10T16:26:27.164429+0000 mgr.gibba032.whopeo (mgr.24625) 8114 : cluster [DBG] pgmap v4617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:30 gibba011 bash[30791]: cluster 2021-03-10T16:26:29.165950+0000 mgr.gibba032.whopeo (mgr.24625) 8115 : cluster [DBG] pgmap v4618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:30 gibba011 bash[31774]: cluster 2021-03-10T16:26:29.165950+0000 mgr.gibba032.whopeo (mgr.24625) 8115 : cluster [DBG] pgmap v4618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:30 gibba032 bash[24004]: cluster 2021-03-10T16:26:29.165950+0000 mgr.gibba032.whopeo (mgr.24625) 8115 : cluster [DBG] pgmap v4618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:31.159 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:26:31.389 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:26:31.389 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:26:31.389 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:26:31.390 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:26:31.390 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:26:31.390 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:26:31.390 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:26:31.391 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:26:31.391 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:26:31.391 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:26:31.391 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:26:31.392 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:26:31.392 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:26:31.392 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:26:31.392 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:26:31.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:26:31.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:26:31.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:26:31.393 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:26:31.394 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:26:31.394 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:26:31.635 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:26:31.635 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:26:31.635 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:26:31.635 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:31.636 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:26:31.636 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:26:31.636 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:31.636 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:26:31.636 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:26:31.637 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:26:31.637 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:26:31.637 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:26:31.637 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:26:31.638 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:26:31.638 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:26:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[30791]: audit 2021-03-10T16:26:31.150495+0000 mgr.gibba032.whopeo (mgr.24625) 8116 : audit [DBG] from='client.49491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[30791]: cluster 2021-03-10T16:26:31.167455+0000 mgr.gibba032.whopeo (mgr.24625) 8117 : cluster [DBG] pgmap v4619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[30791]: audit 2021-03-10T16:26:31.376758+0000 mgr.gibba032.whopeo (mgr.24625) 8118 : audit [DBG] from='client.49497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[30791]: audit 2021-03-10T16:26:31.636806+0000 mon.c (mon.1) 155 : audit [DBG] from='client.? 172.21.2.111:0/2128678885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[31774]: audit 2021-03-10T16:26:31.150495+0000 mgr.gibba032.whopeo (mgr.24625) 8116 : audit [DBG] from='client.49491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[31774]: cluster 2021-03-10T16:26:31.167455+0000 mgr.gibba032.whopeo (mgr.24625) 8117 : cluster [DBG] pgmap v4619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[31774]: audit 2021-03-10T16:26:31.376758+0000 mgr.gibba032.whopeo (mgr.24625) 8118 : audit [DBG] from='client.49497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:32 gibba011 bash[31774]: audit 2021-03-10T16:26:31.636806+0000 mon.c (mon.1) 155 : audit [DBG] from='client.? 172.21.2.111:0/2128678885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:32 gibba032 bash[24004]: audit 2021-03-10T16:26:31.150495+0000 mgr.gibba032.whopeo (mgr.24625) 8116 : audit [DBG] from='client.49491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:32 gibba032 bash[24004]: cluster 2021-03-10T16:26:31.167455+0000 mgr.gibba032.whopeo (mgr.24625) 8117 : cluster [DBG] pgmap v4619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:32 gibba032 bash[24004]: audit 2021-03-10T16:26:31.376758+0000 mgr.gibba032.whopeo (mgr.24625) 8118 : audit [DBG] from='client.49497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:26:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:32 gibba032 bash[24004]: audit 2021-03-10T16:26:31.636806+0000 mon.c (mon.1) 155 : audit [DBG] from='client.? 172.21.2.111:0/2128678885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:26:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:34 gibba011 bash[30791]: cluster 2021-03-10T16:26:33.169140+0000 mgr.gibba032.whopeo (mgr.24625) 8119 : cluster [DBG] pgmap v4620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:34.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:34 gibba011 bash[31774]: cluster 2021-03-10T16:26:33.169140+0000 mgr.gibba032.whopeo (mgr.24625) 8119 : cluster [DBG] pgmap v4620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:34 gibba032 bash[24004]: cluster 2021-03-10T16:26:33.169140+0000 mgr.gibba032.whopeo (mgr.24625) 8119 : cluster [DBG] pgmap v4620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:36 gibba011 bash[30791]: cluster 2021-03-10T16:26:35.170664+0000 mgr.gibba032.whopeo (mgr.24625) 8120 : cluster [DBG] pgmap v4621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:36 gibba011 bash[30791]: cephadm 2021-03-10T16:26:35.525220+0000 mgr.gibba032.whopeo (mgr.24625) 8121 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:36 gibba011 bash[31774]: cluster 2021-03-10T16:26:35.170664+0000 mgr.gibba032.whopeo (mgr.24625) 8120 : cluster [DBG] pgmap v4621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:36 gibba011 bash[31774]: cephadm 2021-03-10T16:26:35.525220+0000 mgr.gibba032.whopeo (mgr.24625) 8121 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:36 gibba032 bash[24004]: cluster 2021-03-10T16:26:35.170664+0000 mgr.gibba032.whopeo (mgr.24625) 8120 : cluster [DBG] pgmap v4621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:36 gibba032 bash[24004]: cephadm 2021-03-10T16:26:35.525220+0000 mgr.gibba032.whopeo (mgr.24625) 8121 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:38 gibba032 bash[24004]: cluster 2021-03-10T16:26:37.172253+0000 mgr.gibba032.whopeo (mgr.24625) 8122 : cluster [DBG] pgmap v4622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:39.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:38 gibba011 bash[30791]: cluster 2021-03-10T16:26:37.172253+0000 mgr.gibba032.whopeo (mgr.24625) 8122 : cluster [DBG] pgmap v4622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:38 gibba011 bash[31774]: cluster 2021-03-10T16:26:37.172253+0000 mgr.gibba032.whopeo (mgr.24625) 8122 : cluster [DBG] pgmap v4622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:40 gibba032 bash[24004]: cluster 2021-03-10T16:26:39.173709+0000 mgr.gibba032.whopeo (mgr.24625) 8123 : cluster [DBG] pgmap v4623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:41.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:40 gibba011 bash[30791]: cluster 2021-03-10T16:26:39.173709+0000 mgr.gibba032.whopeo (mgr.24625) 8123 : cluster [DBG] pgmap v4623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:40 gibba011 bash[31774]: cluster 2021-03-10T16:26:39.173709+0000 mgr.gibba032.whopeo (mgr.24625) 8123 : cluster [DBG] pgmap v4623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:42 gibba032 bash[24004]: cluster 2021-03-10T16:26:41.175392+0000 mgr.gibba032.whopeo (mgr.24625) 8124 : cluster [DBG] pgmap v4624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:42 gibba011 bash[30791]: cluster 2021-03-10T16:26:41.175392+0000 mgr.gibba032.whopeo (mgr.24625) 8124 : cluster [DBG] pgmap v4624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:42 gibba011 bash[31774]: cluster 2021-03-10T16:26:41.175392+0000 mgr.gibba032.whopeo (mgr.24625) 8124 : cluster [DBG] pgmap v4624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:44 gibba032 bash[24004]: cluster 2021-03-10T16:26:43.177003+0000 mgr.gibba032.whopeo (mgr.24625) 8125 : cluster [DBG] pgmap v4625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:44 gibba011 bash[30791]: cluster 2021-03-10T16:26:43.177003+0000 mgr.gibba032.whopeo (mgr.24625) 8125 : cluster [DBG] pgmap v4625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:44 gibba011 bash[31774]: cluster 2021-03-10T16:26:43.177003+0000 mgr.gibba032.whopeo (mgr.24625) 8125 : cluster [DBG] pgmap v4625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:46 gibba032 bash[24004]: cluster 2021-03-10T16:26:45.178625+0000 mgr.gibba032.whopeo (mgr.24625) 8126 : cluster [DBG] pgmap v4626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:46 gibba011 bash[30791]: cluster 2021-03-10T16:26:45.178625+0000 mgr.gibba032.whopeo (mgr.24625) 8126 : cluster [DBG] pgmap v4626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:46 gibba011 bash[31774]: cluster 2021-03-10T16:26:45.178625+0000 mgr.gibba032.whopeo (mgr.24625) 8126 : cluster [DBG] pgmap v4626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:48 gibba032 bash[24004]: cluster 2021-03-10T16:26:47.180122+0000 mgr.gibba032.whopeo (mgr.24625) 8127 : cluster [DBG] pgmap v4627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:48 gibba011 bash[30791]: cluster 2021-03-10T16:26:47.180122+0000 mgr.gibba032.whopeo (mgr.24625) 8127 : cluster [DBG] pgmap v4627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:49.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:48 gibba011 bash[31774]: cluster 2021-03-10T16:26:47.180122+0000 mgr.gibba032.whopeo (mgr.24625) 8127 : cluster [DBG] pgmap v4627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:50 gibba032 bash[24004]: cluster 2021-03-10T16:26:49.181471+0000 mgr.gibba032.whopeo (mgr.24625) 8128 : cluster [DBG] pgmap v4628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:50 gibba011 bash[30791]: cluster 2021-03-10T16:26:49.181471+0000 mgr.gibba032.whopeo (mgr.24625) 8128 : cluster [DBG] pgmap v4628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:51.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:50 gibba011 bash[31774]: cluster 2021-03-10T16:26:49.181471+0000 mgr.gibba032.whopeo (mgr.24625) 8128 : cluster [DBG] pgmap v4628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:52.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:51 gibba032 bash[24004]: cephadm 2021-03-10T16:26:50.541038+0000 mgr.gibba032.whopeo (mgr.24625) 8129 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:52.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:51 gibba011 bash[30791]: cephadm 2021-03-10T16:26:50.541038+0000 mgr.gibba032.whopeo (mgr.24625) 8129 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:52.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:51 gibba011 bash[31774]: cephadm 2021-03-10T16:26:50.541038+0000 mgr.gibba032.whopeo (mgr.24625) 8129 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:26:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:52 gibba032 bash[24004]: cluster 2021-03-10T16:26:51.183225+0000 mgr.gibba032.whopeo (mgr.24625) 8130 : cluster [DBG] pgmap v4629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:52 gibba011 bash[30791]: cluster 2021-03-10T16:26:51.183225+0000 mgr.gibba032.whopeo (mgr.24625) 8130 : cluster [DBG] pgmap v4629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:52 gibba011 bash[31774]: cluster 2021-03-10T16:26:51.183225+0000 mgr.gibba032.whopeo (mgr.24625) 8130 : cluster [DBG] pgmap v4629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:54 gibba032 bash[24004]: cluster 2021-03-10T16:26:53.185235+0000 mgr.gibba032.whopeo (mgr.24625) 8131 : cluster [DBG] pgmap v4630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:54 gibba011 bash[30791]: cluster 2021-03-10T16:26:53.185235+0000 mgr.gibba032.whopeo (mgr.24625) 8131 : cluster [DBG] pgmap v4630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:55.154 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:54 gibba011 bash[31774]: cluster 2021-03-10T16:26:53.185235+0000 mgr.gibba032.whopeo (mgr.24625) 8131 : cluster [DBG] pgmap v4630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:57.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:56 gibba032 bash[24004]: cluster 2021-03-10T16:26:55.186879+0000 mgr.gibba032.whopeo (mgr.24625) 8132 : cluster [DBG] pgmap v4631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:56 gibba011 bash[30791]: cluster 2021-03-10T16:26:55.186879+0000 mgr.gibba032.whopeo (mgr.24625) 8132 : cluster [DBG] pgmap v4631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:56 gibba011 bash[31774]: cluster 2021-03-10T16:26:55.186879+0000 mgr.gibba032.whopeo (mgr.24625) 8132 : cluster [DBG] pgmap v4631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.010918+0000 mon.a (mon.0) 2847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:26:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.011629+0000 mon.a (mon.0) 2848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:26:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.107014+0000 mon.a (mon.0) 2849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:26:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.107788+0000 mon.a (mon.0) 2850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.108460+0000 mon.a (mon.0) 2851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:57 gibba032 bash[24004]: audit 2021-03-10T16:26:57.109122+0000 mon.a (mon.0) 2852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:26:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.010918+0000 mon.a (mon.0) 2847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:26:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.011629+0000 mon.a (mon.0) 2848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:26:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.107014+0000 mon.a (mon.0) 2849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:26:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.107788+0000 mon.a (mon.0) 2850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.108460+0000 mon.a (mon.0) 2851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[30791]: audit 2021-03-10T16:26:57.109122+0000 mon.a (mon.0) 2852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:26:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.010918+0000 mon.a (mon.0) 2847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:26:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.011629+0000 mon.a (mon.0) 2848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:26:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.107014+0000 mon.a (mon.0) 2849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:26:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.107788+0000 mon.a (mon.0) 2850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.108460+0000 mon.a (mon.0) 2851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:26:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:57 gibba011 bash[31774]: audit 2021-03-10T16:26:57.109122+0000 mon.a (mon.0) 2852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:26:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:26:58 gibba032 bash[24004]: cluster 2021-03-10T16:26:57.188209+0000 mgr.gibba032.whopeo (mgr.24625) 8133 : cluster [DBG] pgmap v4632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:26:58 gibba011 bash[30791]: cluster 2021-03-10T16:26:57.188209+0000 mgr.gibba032.whopeo (mgr.24625) 8133 : cluster [DBG] pgmap v4632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:26:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:26:58 gibba011 bash[31774]: cluster 2021-03-10T16:26:57.188209+0000 mgr.gibba032.whopeo (mgr.24625) 8133 : cluster [DBG] pgmap v4632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:00 gibba032 bash[24004]: cluster 2021-03-10T16:26:59.189529+0000 mgr.gibba032.whopeo (mgr.24625) 8134 : cluster [DBG] pgmap v4633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:00 gibba011 bash[30791]: cluster 2021-03-10T16:26:59.189529+0000 mgr.gibba032.whopeo (mgr.24625) 8134 : cluster [DBG] pgmap v4633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:00 gibba011 bash[31774]: cluster 2021-03-10T16:26:59.189529+0000 mgr.gibba032.whopeo (mgr.24625) 8134 : cluster [DBG] pgmap v4633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:01.878 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:27:02.108 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:27:02.109 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:27:02.109 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:27:02.109 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:27:02.109 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:27:02.110 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:27:02.110 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:27:02.110 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:27:02.110 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:27:02.111 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:27:02.111 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:27:02.111 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:27:02.111 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:27:02.111 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:27:02.112 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:27:02.112 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:27:02.112 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:27:02.112 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:27:02.113 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:27:02.113 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:27:02.113 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:27:02.385 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:27:02.385 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:27:02.385 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:27:02.386 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:02.386 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:27:02.386 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:27:02.387 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:02.387 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:27:02.387 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:27:02.387 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:02.388 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:27:02.388 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:27:02.388 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:27:02.388 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:27:02.389 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:27:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:02 gibba032 bash[24004]: cluster 2021-03-10T16:27:01.191304+0000 mgr.gibba032.whopeo (mgr.24625) 8135 : cluster [DBG] pgmap v4634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:02 gibba032 bash[24004]: audit 2021-03-10T16:27:02.386660+0000 mon.c (mon.1) 156 : audit [DBG] from='client.? 172.21.2.111:0/3005504591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:02 gibba011 bash[30791]: cluster 2021-03-10T16:27:01.191304+0000 mgr.gibba032.whopeo (mgr.24625) 8135 : cluster [DBG] pgmap v4634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:02 gibba011 bash[30791]: audit 2021-03-10T16:27:02.386660+0000 mon.c (mon.1) 156 : audit [DBG] from='client.? 172.21.2.111:0/3005504591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:02 gibba011 bash[31774]: cluster 2021-03-10T16:27:01.191304+0000 mgr.gibba032.whopeo (mgr.24625) 8135 : cluster [DBG] pgmap v4634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:03.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:02 gibba011 bash[31774]: audit 2021-03-10T16:27:02.386660+0000 mon.c (mon.1) 156 : audit [DBG] from='client.? 172.21.2.111:0/3005504591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:04.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:03 gibba032 bash[24004]: audit 2021-03-10T16:27:01.869208+0000 mgr.gibba032.whopeo (mgr.24625) 8136 : audit [DBG] from='client.59491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:04.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:03 gibba032 bash[24004]: audit 2021-03-10T16:27:02.095770+0000 mgr.gibba032.whopeo (mgr.24625) 8137 : audit [DBG] from='client.49515 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:04.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:03 gibba011 bash[30791]: audit 2021-03-10T16:27:01.869208+0000 mgr.gibba032.whopeo (mgr.24625) 8136 : audit [DBG] from='client.59491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:04.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:03 gibba011 bash[30791]: audit 2021-03-10T16:27:02.095770+0000 mgr.gibba032.whopeo (mgr.24625) 8137 : audit [DBG] from='client.49515 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:04.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:03 gibba011 bash[31774]: audit 2021-03-10T16:27:01.869208+0000 mgr.gibba032.whopeo (mgr.24625) 8136 : audit [DBG] from='client.59491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:04.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:03 gibba011 bash[31774]: audit 2021-03-10T16:27:02.095770+0000 mgr.gibba032.whopeo (mgr.24625) 8137 : audit [DBG] from='client.49515 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:04 gibba032 bash[24004]: cluster 2021-03-10T16:27:03.193000+0000 mgr.gibba032.whopeo (mgr.24625) 8138 : cluster [DBG] pgmap v4635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:05.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:04 gibba011 bash[30791]: cluster 2021-03-10T16:27:03.193000+0000 mgr.gibba032.whopeo (mgr.24625) 8138 : cluster [DBG] pgmap v4635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:04 gibba011 bash[31774]: cluster 2021-03-10T16:27:03.193000+0000 mgr.gibba032.whopeo (mgr.24625) 8138 : cluster [DBG] pgmap v4635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:07.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:06 gibba032 bash[24004]: cluster 2021-03-10T16:27:05.194582+0000 mgr.gibba032.whopeo (mgr.24625) 8139 : cluster [DBG] pgmap v4636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:06 gibba032 bash[24004]: cephadm 2021-03-10T16:27:05.557090+0000 mgr.gibba032.whopeo (mgr.24625) 8140 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:06 gibba011 bash[30791]: cluster 2021-03-10T16:27:05.194582+0000 mgr.gibba032.whopeo (mgr.24625) 8139 : cluster [DBG] pgmap v4636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:06 gibba011 bash[30791]: cephadm 2021-03-10T16:27:05.557090+0000 mgr.gibba032.whopeo (mgr.24625) 8140 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:06 gibba011 bash[31774]: cluster 2021-03-10T16:27:05.194582+0000 mgr.gibba032.whopeo (mgr.24625) 8139 : cluster [DBG] pgmap v4636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:06 gibba011 bash[31774]: cephadm 2021-03-10T16:27:05.557090+0000 mgr.gibba032.whopeo (mgr.24625) 8140 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:07 gibba032 bash[24004]: cluster 2021-03-10T16:27:07.196225+0000 mgr.gibba032.whopeo (mgr.24625) 8141 : cluster [DBG] pgmap v4637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:08.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:07 gibba011 bash[30791]: cluster 2021-03-10T16:27:07.196225+0000 mgr.gibba032.whopeo (mgr.24625) 8141 : cluster [DBG] pgmap v4637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:08.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:07 gibba011 bash[31774]: cluster 2021-03-10T16:27:07.196225+0000 mgr.gibba032.whopeo (mgr.24625) 8141 : cluster [DBG] pgmap v4637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:10.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:10 gibba032 bash[24004]: cluster 2021-03-10T16:27:09.197531+0000 mgr.gibba032.whopeo (mgr.24625) 8142 : cluster [DBG] pgmap v4638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:10 gibba011 bash[30791]: cluster 2021-03-10T16:27:09.197531+0000 mgr.gibba032.whopeo (mgr.24625) 8142 : cluster [DBG] pgmap v4638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:10.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:10 gibba011 bash[31774]: cluster 2021-03-10T16:27:09.197531+0000 mgr.gibba032.whopeo (mgr.24625) 8142 : cluster [DBG] pgmap v4638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:12 gibba032 bash[24004]: cluster 2021-03-10T16:27:11.199079+0000 mgr.gibba032.whopeo (mgr.24625) 8143 : cluster [DBG] pgmap v4639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:12.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:12 gibba011 bash[30791]: cluster 2021-03-10T16:27:11.199079+0000 mgr.gibba032.whopeo (mgr.24625) 8143 : cluster [DBG] pgmap v4639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:12.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:12 gibba011 bash[31774]: cluster 2021-03-10T16:27:11.199079+0000 mgr.gibba032.whopeo (mgr.24625) 8143 : cluster [DBG] pgmap v4639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:14 gibba032 bash[24004]: cluster 2021-03-10T16:27:13.200942+0000 mgr.gibba032.whopeo (mgr.24625) 8144 : cluster [DBG] pgmap v4640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:14 gibba011 bash[30791]: cluster 2021-03-10T16:27:13.200942+0000 mgr.gibba032.whopeo (mgr.24625) 8144 : cluster [DBG] pgmap v4640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:14.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:14 gibba011 bash[31774]: cluster 2021-03-10T16:27:13.200942+0000 mgr.gibba032.whopeo (mgr.24625) 8144 : cluster [DBG] pgmap v4640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:16 gibba032 bash[24004]: cluster 2021-03-10T16:27:15.202619+0000 mgr.gibba032.whopeo (mgr.24625) 8145 : cluster [DBG] pgmap v4641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:16 gibba011 bash[30791]: cluster 2021-03-10T16:27:15.202619+0000 mgr.gibba032.whopeo (mgr.24625) 8145 : cluster [DBG] pgmap v4641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:16 gibba011 bash[31774]: cluster 2021-03-10T16:27:15.202619+0000 mgr.gibba032.whopeo (mgr.24625) 8145 : cluster [DBG] pgmap v4641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:18 gibba032 bash[24004]: cluster 2021-03-10T16:27:17.204087+0000 mgr.gibba032.whopeo (mgr.24625) 8146 : cluster [DBG] pgmap v4642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:18 gibba011 bash[30791]: cluster 2021-03-10T16:27:17.204087+0000 mgr.gibba032.whopeo (mgr.24625) 8146 : cluster [DBG] pgmap v4642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:18.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:18 gibba011 bash[31774]: cluster 2021-03-10T16:27:17.204087+0000 mgr.gibba032.whopeo (mgr.24625) 8146 : cluster [DBG] pgmap v4642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:20.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:20 gibba032 bash[24004]: cluster 2021-03-10T16:27:19.205349+0000 mgr.gibba032.whopeo (mgr.24625) 8147 : cluster [DBG] pgmap v4643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:20.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:20 gibba011 bash[30791]: cluster 2021-03-10T16:27:19.205349+0000 mgr.gibba032.whopeo (mgr.24625) 8147 : cluster [DBG] pgmap v4643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:20.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:20 gibba011 bash[31774]: cluster 2021-03-10T16:27:19.205349+0000 mgr.gibba032.whopeo (mgr.24625) 8147 : cluster [DBG] pgmap v4643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.777220+0000 mgr.gibba032.whopeo (mgr.24625) 8148 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:27:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.778679+0000 mon.a (mon.0) 2853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.780757+0000 mgr.gibba032.whopeo (mgr.24625) 8149 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:27:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.782396+0000 mon.a (mon.0) 2854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.967806+0000 mon.a (mon.0) 2855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.968218+0000 mgr.gibba032.whopeo (mgr.24625) 8150 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:27:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.970238+0000 mgr.gibba032.whopeo (mgr.24625) 8151 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:27:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.971225+0000 mon.a (mon.0) 2856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:27:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.971708+0000 mgr.gibba032.whopeo (mgr.24625) 8152 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:27:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.971745+0000 mgr.gibba032.whopeo (mgr.24625) 8153 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:27:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.972885+0000 mon.a (mon.0) 2857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.973160+0000 mgr.gibba032.whopeo (mgr.24625) 8154 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:27:22.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.973199+0000 mgr.gibba032.whopeo (mgr.24625) 8155 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:27:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.974211+0000 mgr.gibba032.whopeo (mgr.24625) 8156 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:27:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.974377+0000 mon.a (mon.0) 2858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.974981+0000 mgr.gibba032.whopeo (mgr.24625) 8157 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:27:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.975018+0000 mgr.gibba032.whopeo (mgr.24625) 8158 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:27:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.975650+0000 mon.a (mon.0) 2859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.976153+0000 mgr.gibba032.whopeo (mgr.24625) 8159 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:27:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.976483+0000 mgr.gibba032.whopeo (mgr.24625) 8160 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:27:22.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.977631+0000 mon.a (mon.0) 2860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.777220+0000 mgr.gibba032.whopeo (mgr.24625) 8148 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:27:22.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.778679+0000 mon.a (mon.0) 2853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.780757+0000 mgr.gibba032.whopeo (mgr.24625) 8149 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:27:22.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.782396+0000 mon.a (mon.0) 2854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.967806+0000 mon.a (mon.0) 2855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.968218+0000 mgr.gibba032.whopeo (mgr.24625) 8150 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:27:22.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.970238+0000 mgr.gibba032.whopeo (mgr.24625) 8151 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:27:22.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.971225+0000 mon.a (mon.0) 2856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:27:22.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.971708+0000 mgr.gibba032.whopeo (mgr.24625) 8152 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:27:22.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.971745+0000 mgr.gibba032.whopeo (mgr.24625) 8153 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:27:22.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.972885+0000 mon.a (mon.0) 2857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.973160+0000 mgr.gibba032.whopeo (mgr.24625) 8154 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:27:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.973199+0000 mgr.gibba032.whopeo (mgr.24625) 8155 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:27:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.974211+0000 mgr.gibba032.whopeo (mgr.24625) 8156 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:27:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.974377+0000 mon.a (mon.0) 2858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.974981+0000 mgr.gibba032.whopeo (mgr.24625) 8157 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:27:22.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.975018+0000 mgr.gibba032.whopeo (mgr.24625) 8158 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:27:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.975650+0000 mon.a (mon.0) 2859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.976153+0000 mgr.gibba032.whopeo (mgr.24625) 8159 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:27:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.976483+0000 mgr.gibba032.whopeo (mgr.24625) 8160 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:27:22.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.977631+0000 mon.a (mon.0) 2860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.977814+0000 mgr.gibba032.whopeo (mgr.24625) 8161 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:27:22.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.977853+0000 mgr.gibba032.whopeo (mgr.24625) 8162 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:27:22.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cephadm 2021-03-10T16:27:20.978037+0000 mgr.gibba032.whopeo (mgr.24625) 8163 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:22.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: audit 2021-03-10T16:27:20.978977+0000 mon.a (mon.0) 2861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[30791]: cluster 2021-03-10T16:27:21.207145+0000 mgr.gibba032.whopeo (mgr.24625) 8164 : cluster [DBG] pgmap v4644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.977814+0000 mgr.gibba032.whopeo (mgr.24625) 8161 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:27:22.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.977853+0000 mgr.gibba032.whopeo (mgr.24625) 8162 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:27:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cephadm 2021-03-10T16:27:20.978037+0000 mgr.gibba032.whopeo (mgr.24625) 8163 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: audit 2021-03-10T16:27:20.978977+0000 mon.a (mon.0) 2861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:21 gibba011 bash[31774]: cluster 2021-03-10T16:27:21.207145+0000 mgr.gibba032.whopeo (mgr.24625) 8164 : cluster [DBG] pgmap v4644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.777220+0000 mgr.gibba032.whopeo (mgr.24625) 8148 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:27:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.778679+0000 mon.a (mon.0) 2853 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.780757+0000 mgr.gibba032.whopeo (mgr.24625) 8149 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:27:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.782396+0000 mon.a (mon.0) 2854 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.967806+0000 mon.a (mon.0) 2855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:27:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.968218+0000 mgr.gibba032.whopeo (mgr.24625) 8150 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:27:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.970238+0000 mgr.gibba032.whopeo (mgr.24625) 8151 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:27:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.971225+0000 mon.a (mon.0) 2856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:27:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.971708+0000 mgr.gibba032.whopeo (mgr.24625) 8152 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:27:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.971745+0000 mgr.gibba032.whopeo (mgr.24625) 8153 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:27:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.972885+0000 mon.a (mon.0) 2857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.973160+0000 mgr.gibba032.whopeo (mgr.24625) 8154 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:27:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.973199+0000 mgr.gibba032.whopeo (mgr.24625) 8155 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:27:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.974211+0000 mgr.gibba032.whopeo (mgr.24625) 8156 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:27:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.974377+0000 mon.a (mon.0) 2858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.974981+0000 mgr.gibba032.whopeo (mgr.24625) 8157 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:27:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.975018+0000 mgr.gibba032.whopeo (mgr.24625) 8158 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:27:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.975650+0000 mon.a (mon.0) 2859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.976153+0000 mgr.gibba032.whopeo (mgr.24625) 8159 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:27:22.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.976483+0000 mgr.gibba032.whopeo (mgr.24625) 8160 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:27:22.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.977631+0000 mon.a (mon.0) 2860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.977814+0000 mgr.gibba032.whopeo (mgr.24625) 8161 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:27:22.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.977853+0000 mgr.gibba032.whopeo (mgr.24625) 8162 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:27:22.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cephadm 2021-03-10T16:27:20.978037+0000 mgr.gibba032.whopeo (mgr.24625) 8163 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:22.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: audit 2021-03-10T16:27:20.978977+0000 mon.a (mon.0) 2861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:22.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:21 gibba032 bash[24004]: cluster 2021-03-10T16:27:21.207145+0000 mgr.gibba032.whopeo (mgr.24625) 8164 : cluster [DBG] pgmap v4644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:24 gibba011 bash[30791]: cluster 2021-03-10T16:27:23.208795+0000 mgr.gibba032.whopeo (mgr.24625) 8165 : cluster [DBG] pgmap v4645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:24 gibba011 bash[31774]: cluster 2021-03-10T16:27:23.208795+0000 mgr.gibba032.whopeo (mgr.24625) 8165 : cluster [DBG] pgmap v4645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:24 gibba032 bash[24004]: cluster 2021-03-10T16:27:23.208795+0000 mgr.gibba032.whopeo (mgr.24625) 8165 : cluster [DBG] pgmap v4645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:26 gibba032 bash[24004]: cluster 2021-03-10T16:27:25.210269+0000 mgr.gibba032.whopeo (mgr.24625) 8166 : cluster [DBG] pgmap v4646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:26 gibba011 bash[30791]: cluster 2021-03-10T16:27:25.210269+0000 mgr.gibba032.whopeo (mgr.24625) 8166 : cluster [DBG] pgmap v4646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:26 gibba011 bash[31774]: cluster 2021-03-10T16:27:25.210269+0000 mgr.gibba032.whopeo (mgr.24625) 8166 : cluster [DBG] pgmap v4646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:28 gibba032 bash[24004]: cluster 2021-03-10T16:27:27.212078+0000 mgr.gibba032.whopeo (mgr.24625) 8167 : cluster [DBG] pgmap v4647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:28 gibba011 bash[30791]: cluster 2021-03-10T16:27:27.212078+0000 mgr.gibba032.whopeo (mgr.24625) 8167 : cluster [DBG] pgmap v4647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:28.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:28 gibba011 bash[31774]: cluster 2021-03-10T16:27:27.212078+0000 mgr.gibba032.whopeo (mgr.24625) 8167 : cluster [DBG] pgmap v4647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:30 gibba032 bash[24004]: cluster 2021-03-10T16:27:29.213576+0000 mgr.gibba032.whopeo (mgr.24625) 8168 : cluster [DBG] pgmap v4648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:30 gibba011 bash[30791]: cluster 2021-03-10T16:27:29.213576+0000 mgr.gibba032.whopeo (mgr.24625) 8168 : cluster [DBG] pgmap v4648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:30 gibba011 bash[31774]: cluster 2021-03-10T16:27:29.213576+0000 mgr.gibba032.whopeo (mgr.24625) 8168 : cluster [DBG] pgmap v4648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:32.630 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:27:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:32 gibba032 bash[24004]: cluster 2021-03-10T16:27:31.215046+0000 mgr.gibba032.whopeo (mgr.24625) 8169 : cluster [DBG] pgmap v4649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:32.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:32 gibba011 bash[30791]: cluster 2021-03-10T16:27:31.215046+0000 mgr.gibba032.whopeo (mgr.24625) 8169 : cluster [DBG] pgmap v4649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:32.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:32 gibba011 bash[31774]: cluster 2021-03-10T16:27:31.215046+0000 mgr.gibba032.whopeo (mgr.24625) 8169 : cluster [DBG] pgmap v4649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:32.857 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:27:32.857 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:27:32.858 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:27:32.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:27:32.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:27:32.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:27:32.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:27:32.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:27:32.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:27:32.859 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:27:32.859 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:27:32.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:27:32.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:27:32.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:27:32.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:27:32.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:27:32.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:27:32.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:27:32.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:27:32.861 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:27:32.862 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:27:33.110 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:27:33.110 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:27:33.110 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:27:33.111 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:33.111 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:27:33.111 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:27:33.111 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:33.112 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:27:33.112 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:27:33.112 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:27:33.112 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:27:33.112 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:27:33.113 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:27:33.113 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:27:33.113 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:27:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:33 gibba032 bash[24004]: audit 2021-03-10T16:27:32.620703+0000 mgr.gibba032.whopeo (mgr.24625) 8170 : audit [DBG] from='client.59509 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:33 gibba032 bash[24004]: audit 2021-03-10T16:27:33.112086+0000 mon.c (mon.1) 157 : audit [DBG] from='client.? 172.21.2.111:0/2868582866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:33.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:33 gibba011 bash[30791]: audit 2021-03-10T16:27:32.620703+0000 mgr.gibba032.whopeo (mgr.24625) 8170 : audit [DBG] from='client.59509 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:33.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:33 gibba011 bash[30791]: audit 2021-03-10T16:27:33.112086+0000 mon.c (mon.1) 157 : audit [DBG] from='client.? 172.21.2.111:0/2868582866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:33.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:33 gibba011 bash[31774]: audit 2021-03-10T16:27:32.620703+0000 mgr.gibba032.whopeo (mgr.24625) 8170 : audit [DBG] from='client.59509 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:33.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:33 gibba011 bash[31774]: audit 2021-03-10T16:27:33.112086+0000 mon.c (mon.1) 157 : audit [DBG] from='client.? 172.21.2.111:0/2868582866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:27:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:34 gibba011 bash[30791]: audit 2021-03-10T16:27:32.845672+0000 mgr.gibba032.whopeo (mgr.24625) 8171 : audit [DBG] from='client.49533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:34 gibba011 bash[30791]: cluster 2021-03-10T16:27:33.216881+0000 mgr.gibba032.whopeo (mgr.24625) 8172 : cluster [DBG] pgmap v4650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:34 gibba011 bash[31774]: audit 2021-03-10T16:27:32.845672+0000 mgr.gibba032.whopeo (mgr.24625) 8171 : audit [DBG] from='client.49533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:34 gibba011 bash[31774]: cluster 2021-03-10T16:27:33.216881+0000 mgr.gibba032.whopeo (mgr.24625) 8172 : cluster [DBG] pgmap v4650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:34 gibba032 bash[24004]: audit 2021-03-10T16:27:32.845672+0000 mgr.gibba032.whopeo (mgr.24625) 8171 : audit [DBG] from='client.49533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:27:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:34 gibba032 bash[24004]: cluster 2021-03-10T16:27:33.216881+0000 mgr.gibba032.whopeo (mgr.24625) 8172 : cluster [DBG] pgmap v4650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:36 gibba011 bash[30791]: cluster 2021-03-10T16:27:35.218395+0000 mgr.gibba032.whopeo (mgr.24625) 8173 : cluster [DBG] pgmap v4651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:36 gibba011 bash[31774]: cluster 2021-03-10T16:27:35.218395+0000 mgr.gibba032.whopeo (mgr.24625) 8173 : cluster [DBG] pgmap v4651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:36 gibba032 bash[24004]: cluster 2021-03-10T16:27:35.218395+0000 mgr.gibba032.whopeo (mgr.24625) 8173 : cluster [DBG] pgmap v4651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:37.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:37 gibba011 bash[30791]: cephadm 2021-03-10T16:27:35.988816+0000 mgr.gibba032.whopeo (mgr.24625) 8174 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:37.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:37 gibba011 bash[31774]: cephadm 2021-03-10T16:27:35.988816+0000 mgr.gibba032.whopeo (mgr.24625) 8174 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:38.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:37 gibba032 bash[24004]: cephadm 2021-03-10T16:27:35.988816+0000 mgr.gibba032.whopeo (mgr.24625) 8174 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:38 gibba011 bash[30791]: cluster 2021-03-10T16:27:37.219958+0000 mgr.gibba032.whopeo (mgr.24625) 8175 : cluster [DBG] pgmap v4652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:38.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:38 gibba011 bash[31774]: cluster 2021-03-10T16:27:37.219958+0000 mgr.gibba032.whopeo (mgr.24625) 8175 : cluster [DBG] pgmap v4652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:38 gibba032 bash[24004]: cluster 2021-03-10T16:27:37.219958+0000 mgr.gibba032.whopeo (mgr.24625) 8175 : cluster [DBG] pgmap v4652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:40 gibba011 bash[30791]: cluster 2021-03-10T16:27:39.221232+0000 mgr.gibba032.whopeo (mgr.24625) 8176 : cluster [DBG] pgmap v4653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:40 gibba011 bash[31774]: cluster 2021-03-10T16:27:39.221232+0000 mgr.gibba032.whopeo (mgr.24625) 8176 : cluster [DBG] pgmap v4653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:40 gibba032 bash[24004]: cluster 2021-03-10T16:27:39.221232+0000 mgr.gibba032.whopeo (mgr.24625) 8176 : cluster [DBG] pgmap v4653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:42 gibba011 bash[30791]: cluster 2021-03-10T16:27:41.222910+0000 mgr.gibba032.whopeo (mgr.24625) 8177 : cluster [DBG] pgmap v4654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:42.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:42 gibba011 bash[31774]: cluster 2021-03-10T16:27:41.222910+0000 mgr.gibba032.whopeo (mgr.24625) 8177 : cluster [DBG] pgmap v4654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:42 gibba032 bash[24004]: cluster 2021-03-10T16:27:41.222910+0000 mgr.gibba032.whopeo (mgr.24625) 8177 : cluster [DBG] pgmap v4654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:44 gibba011 bash[30791]: cluster 2021-03-10T16:27:43.224713+0000 mgr.gibba032.whopeo (mgr.24625) 8178 : cluster [DBG] pgmap v4655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:44.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:44 gibba011 bash[31774]: cluster 2021-03-10T16:27:43.224713+0000 mgr.gibba032.whopeo (mgr.24625) 8178 : cluster [DBG] pgmap v4655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:44 gibba032 bash[24004]: cluster 2021-03-10T16:27:43.224713+0000 mgr.gibba032.whopeo (mgr.24625) 8178 : cluster [DBG] pgmap v4655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:46 gibba011 bash[30791]: cluster 2021-03-10T16:27:45.226187+0000 mgr.gibba032.whopeo (mgr.24625) 8179 : cluster [DBG] pgmap v4656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:46.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:46 gibba011 bash[31774]: cluster 2021-03-10T16:27:45.226187+0000 mgr.gibba032.whopeo (mgr.24625) 8179 : cluster [DBG] pgmap v4656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:46 gibba032 bash[24004]: cluster 2021-03-10T16:27:45.226187+0000 mgr.gibba032.whopeo (mgr.24625) 8179 : cluster [DBG] pgmap v4656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:48 gibba011 bash[30791]: cluster 2021-03-10T16:27:47.227872+0000 mgr.gibba032.whopeo (mgr.24625) 8180 : cluster [DBG] pgmap v4657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:48 gibba011 bash[31774]: cluster 2021-03-10T16:27:47.227872+0000 mgr.gibba032.whopeo (mgr.24625) 8180 : cluster [DBG] pgmap v4657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:48 gibba032 bash[24004]: cluster 2021-03-10T16:27:47.227872+0000 mgr.gibba032.whopeo (mgr.24625) 8180 : cluster [DBG] pgmap v4657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:51.002 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:50 gibba032 bash[24004]: cluster 2021-03-10T16:27:49.229281+0000 mgr.gibba032.whopeo (mgr.24625) 8181 : cluster [DBG] pgmap v4658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:50 gibba011 bash[30791]: cluster 2021-03-10T16:27:49.229281+0000 mgr.gibba032.whopeo (mgr.24625) 8181 : cluster [DBG] pgmap v4658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:50 gibba011 bash[31774]: cluster 2021-03-10T16:27:49.229281+0000 mgr.gibba032.whopeo (mgr.24625) 8181 : cluster [DBG] pgmap v4658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:52 gibba032 bash[24004]: cephadm 2021-03-10T16:27:51.000924+0000 mgr.gibba032.whopeo (mgr.24625) 8182 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:52 gibba032 bash[24004]: cluster 2021-03-10T16:27:51.230807+0000 mgr.gibba032.whopeo (mgr.24625) 8183 : cluster [DBG] pgmap v4659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:52 gibba011 bash[30791]: cephadm 2021-03-10T16:27:51.000924+0000 mgr.gibba032.whopeo (mgr.24625) 8182 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:52 gibba011 bash[30791]: cluster 2021-03-10T16:27:51.230807+0000 mgr.gibba032.whopeo (mgr.24625) 8183 : cluster [DBG] pgmap v4659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:53.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:52 gibba011 bash[31774]: cephadm 2021-03-10T16:27:51.000924+0000 mgr.gibba032.whopeo (mgr.24625) 8182 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:27:53.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:52 gibba011 bash[31774]: cluster 2021-03-10T16:27:51.230807+0000 mgr.gibba032.whopeo (mgr.24625) 8183 : cluster [DBG] pgmap v4659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:54 gibba032 bash[24004]: cluster 2021-03-10T16:27:53.232530+0000 mgr.gibba032.whopeo (mgr.24625) 8184 : cluster [DBG] pgmap v4660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:54 gibba011 bash[30791]: cluster 2021-03-10T16:27:53.232530+0000 mgr.gibba032.whopeo (mgr.24625) 8184 : cluster [DBG] pgmap v4660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:54 gibba011 bash[31774]: cluster 2021-03-10T16:27:53.232530+0000 mgr.gibba032.whopeo (mgr.24625) 8184 : cluster [DBG] pgmap v4660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:56 gibba032 bash[24004]: cluster 2021-03-10T16:27:55.233838+0000 mgr.gibba032.whopeo (mgr.24625) 8185 : cluster [DBG] pgmap v4661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:56 gibba011 bash[30791]: cluster 2021-03-10T16:27:55.233838+0000 mgr.gibba032.whopeo (mgr.24625) 8185 : cluster [DBG] pgmap v4661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:56 gibba011 bash[31774]: cluster 2021-03-10T16:27:55.233838+0000 mgr.gibba032.whopeo (mgr.24625) 8185 : cluster [DBG] pgmap v4661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.017478+0000 mon.a (mon.0) 2862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:27:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.018107+0000 mon.a (mon.0) 2863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:27:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.115239+0000 mon.a (mon.0) 2864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:27:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.116048+0000 mon.a (mon.0) 2865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.116752+0000 mon.a (mon.0) 2866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:57 gibba032 bash[24004]: audit 2021-03-10T16:27:57.117423+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:27:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.017478+0000 mon.a (mon.0) 2862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:27:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.018107+0000 mon.a (mon.0) 2863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:27:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.115239+0000 mon.a (mon.0) 2864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:27:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.116048+0000 mon.a (mon.0) 2865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.116752+0000 mon.a (mon.0) 2866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[30791]: audit 2021-03-10T16:27:57.117423+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:27:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.017478+0000 mon.a (mon.0) 2862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:27:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.018107+0000 mon.a (mon.0) 2863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:27:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.115239+0000 mon.a (mon.0) 2864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:27:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.116048+0000 mon.a (mon.0) 2865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.116752+0000 mon.a (mon.0) 2866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:27:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:57 gibba011 bash[31774]: audit 2021-03-10T16:27:57.117423+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:27:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:27:58 gibba032 bash[24004]: cluster 2021-03-10T16:27:57.235641+0000 mgr.gibba032.whopeo (mgr.24625) 8186 : cluster [DBG] pgmap v4662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:27:58 gibba011 bash[30791]: cluster 2021-03-10T16:27:57.235641+0000 mgr.gibba032.whopeo (mgr.24625) 8186 : cluster [DBG] pgmap v4662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:27:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:27:58 gibba011 bash[31774]: cluster 2021-03-10T16:27:57.235641+0000 mgr.gibba032.whopeo (mgr.24625) 8186 : cluster [DBG] pgmap v4662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:00 gibba032 bash[24004]: cluster 2021-03-10T16:27:59.236922+0000 mgr.gibba032.whopeo (mgr.24625) 8187 : cluster [DBG] pgmap v4663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:00 gibba011 bash[30791]: cluster 2021-03-10T16:27:59.236922+0000 mgr.gibba032.whopeo (mgr.24625) 8187 : cluster [DBG] pgmap v4663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:00 gibba011 bash[31774]: cluster 2021-03-10T16:27:59.236922+0000 mgr.gibba032.whopeo (mgr.24625) 8187 : cluster [DBG] pgmap v4663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:02 gibba032 bash[24004]: cluster 2021-03-10T16:28:01.238645+0000 mgr.gibba032.whopeo (mgr.24625) 8188 : cluster [DBG] pgmap v4664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:02 gibba011 bash[30791]: cluster 2021-03-10T16:28:01.238645+0000 mgr.gibba032.whopeo (mgr.24625) 8188 : cluster [DBG] pgmap v4664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:02 gibba011 bash[31774]: cluster 2021-03-10T16:28:01.238645+0000 mgr.gibba032.whopeo (mgr.24625) 8188 : cluster [DBG] pgmap v4664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:03.359 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:28:03.585 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:28:03.585 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:28:03.586 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:28:03.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:28:03.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:28:03.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:28:03.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:28:03.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:28:03.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:28:03.587 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:28:03.588 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:28:03.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:28:03.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:28:03.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:28:03.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:28:03.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:28:03.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:28:03.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:28:03.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:28:03.590 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:28:03.590 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:28:03.841 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:28:03.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:28:03.842 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:28:03.842 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:03.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:28:03.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:28:03.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:03.843 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:28:03.843 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:28:03.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:03.844 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:28:03.844 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:28:03.844 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:28:03.844 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:28:03.844 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:28:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:04 gibba032 bash[24004]: cluster 2021-03-10T16:28:03.240530+0000 mgr.gibba032.whopeo (mgr.24625) 8189 : cluster [DBG] pgmap v4665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:04 gibba032 bash[24004]: audit 2021-03-10T16:28:03.350006+0000 mgr.gibba032.whopeo (mgr.24625) 8190 : audit [DBG] from='client.49545 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:04 gibba032 bash[24004]: audit 2021-03-10T16:28:03.573435+0000 mgr.gibba032.whopeo (mgr.24625) 8191 : audit [DBG] from='client.59533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:04 gibba032 bash[24004]: audit 2021-03-10T16:28:03.843463+0000 mon.c (mon.1) 158 : audit [DBG] from='client.? 172.21.2.111:0/2584568932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[30791]: cluster 2021-03-10T16:28:03.240530+0000 mgr.gibba032.whopeo (mgr.24625) 8189 : cluster [DBG] pgmap v4665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[30791]: audit 2021-03-10T16:28:03.350006+0000 mgr.gibba032.whopeo (mgr.24625) 8190 : audit [DBG] from='client.49545 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[30791]: audit 2021-03-10T16:28:03.573435+0000 mgr.gibba032.whopeo (mgr.24625) 8191 : audit [DBG] from='client.59533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[30791]: audit 2021-03-10T16:28:03.843463+0000 mon.c (mon.1) 158 : audit [DBG] from='client.? 172.21.2.111:0/2584568932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:05.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[31774]: cluster 2021-03-10T16:28:03.240530+0000 mgr.gibba032.whopeo (mgr.24625) 8189 : cluster [DBG] pgmap v4665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[31774]: audit 2021-03-10T16:28:03.350006+0000 mgr.gibba032.whopeo (mgr.24625) 8190 : audit [DBG] from='client.49545 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[31774]: audit 2021-03-10T16:28:03.573435+0000 mgr.gibba032.whopeo (mgr.24625) 8191 : audit [DBG] from='client.59533 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:04 gibba011 bash[31774]: audit 2021-03-10T16:28:03.843463+0000 mon.c (mon.1) 158 : audit [DBG] from='client.? 172.21.2.111:0/2584568932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:06 gibba032 bash[24004]: cluster 2021-03-10T16:28:05.242078+0000 mgr.gibba032.whopeo (mgr.24625) 8192 : cluster [DBG] pgmap v4666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:06 gibba011 bash[30791]: cluster 2021-03-10T16:28:05.242078+0000 mgr.gibba032.whopeo (mgr.24625) 8192 : cluster [DBG] pgmap v4666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:06 gibba011 bash[31774]: cluster 2021-03-10T16:28:05.242078+0000 mgr.gibba032.whopeo (mgr.24625) 8192 : cluster [DBG] pgmap v4666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:07 gibba032 bash[24004]: cephadm 2021-03-10T16:28:06.016760+0000 mgr.gibba032.whopeo (mgr.24625) 8193 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:08.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:07 gibba011 bash[30791]: cephadm 2021-03-10T16:28:06.016760+0000 mgr.gibba032.whopeo (mgr.24625) 8193 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:08.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:07 gibba011 bash[31774]: cephadm 2021-03-10T16:28:06.016760+0000 mgr.gibba032.whopeo (mgr.24625) 8193 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:08 gibba032 bash[24004]: cluster 2021-03-10T16:28:07.243833+0000 mgr.gibba032.whopeo (mgr.24625) 8194 : cluster [DBG] pgmap v4667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:08 gibba011 bash[30791]: cluster 2021-03-10T16:28:07.243833+0000 mgr.gibba032.whopeo (mgr.24625) 8194 : cluster [DBG] pgmap v4667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:08 gibba011 bash[31774]: cluster 2021-03-10T16:28:07.243833+0000 mgr.gibba032.whopeo (mgr.24625) 8194 : cluster [DBG] pgmap v4667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:11.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:10 gibba032 bash[24004]: cluster 2021-03-10T16:28:09.245474+0000 mgr.gibba032.whopeo (mgr.24625) 8195 : cluster [DBG] pgmap v4668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:10 gibba011 bash[30791]: cluster 2021-03-10T16:28:09.245474+0000 mgr.gibba032.whopeo (mgr.24625) 8195 : cluster [DBG] pgmap v4668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:10 gibba011 bash[31774]: cluster 2021-03-10T16:28:09.245474+0000 mgr.gibba032.whopeo (mgr.24625) 8195 : cluster [DBG] pgmap v4668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:13.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:12 gibba032 bash[24004]: cluster 2021-03-10T16:28:11.247170+0000 mgr.gibba032.whopeo (mgr.24625) 8196 : cluster [DBG] pgmap v4669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:12 gibba011 bash[30791]: cluster 2021-03-10T16:28:11.247170+0000 mgr.gibba032.whopeo (mgr.24625) 8196 : cluster [DBG] pgmap v4669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:12 gibba011 bash[31774]: cluster 2021-03-10T16:28:11.247170+0000 mgr.gibba032.whopeo (mgr.24625) 8196 : cluster [DBG] pgmap v4669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:15.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:14 gibba032 bash[24004]: cluster 2021-03-10T16:28:13.249177+0000 mgr.gibba032.whopeo (mgr.24625) 8197 : cluster [DBG] pgmap v4670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:14 gibba011 bash[30791]: cluster 2021-03-10T16:28:13.249177+0000 mgr.gibba032.whopeo (mgr.24625) 8197 : cluster [DBG] pgmap v4670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:14 gibba011 bash[31774]: cluster 2021-03-10T16:28:13.249177+0000 mgr.gibba032.whopeo (mgr.24625) 8197 : cluster [DBG] pgmap v4670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:16 gibba032 bash[24004]: cluster 2021-03-10T16:28:15.250830+0000 mgr.gibba032.whopeo (mgr.24625) 8198 : cluster [DBG] pgmap v4671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:16 gibba011 bash[30791]: cluster 2021-03-10T16:28:15.250830+0000 mgr.gibba032.whopeo (mgr.24625) 8198 : cluster [DBG] pgmap v4671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:16 gibba011 bash[31774]: cluster 2021-03-10T16:28:15.250830+0000 mgr.gibba032.whopeo (mgr.24625) 8198 : cluster [DBG] pgmap v4671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:19.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:18 gibba032 bash[24004]: cluster 2021-03-10T16:28:17.252825+0000 mgr.gibba032.whopeo (mgr.24625) 8199 : cluster [DBG] pgmap v4672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:19.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:18 gibba011 bash[30791]: cluster 2021-03-10T16:28:17.252825+0000 mgr.gibba032.whopeo (mgr.24625) 8199 : cluster [DBG] pgmap v4672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:18 gibba011 bash[31774]: cluster 2021-03-10T16:28:17.252825+0000 mgr.gibba032.whopeo (mgr.24625) 8199 : cluster [DBG] pgmap v4672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:21.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:20 gibba032 bash[24004]: cluster 2021-03-10T16:28:19.254474+0000 mgr.gibba032.whopeo (mgr.24625) 8200 : cluster [DBG] pgmap v4673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:21.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:20 gibba011 bash[30791]: cluster 2021-03-10T16:28:19.254474+0000 mgr.gibba032.whopeo (mgr.24625) 8200 : cluster [DBG] pgmap v4673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:21.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:20 gibba011 bash[31774]: cluster 2021-03-10T16:28:19.254474+0000 mgr.gibba032.whopeo (mgr.24625) 8200 : cluster [DBG] pgmap v4673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.028810+0000 mgr.gibba032.whopeo (mgr.24625) 8201 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:28:22.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.030325+0000 mgr.gibba032.whopeo (mgr.24625) 8202 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:28:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.233628+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:28:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.233870+0000 mgr.gibba032.whopeo (mgr.24625) 8203 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:28:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.235871+0000 mgr.gibba032.whopeo (mgr.24625) 8204 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:28:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.237050+0000 mon.a (mon.0) 2869 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:28:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.237235+0000 mgr.gibba032.whopeo (mgr.24625) 8205 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:28:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.237272+0000 mgr.gibba032.whopeo (mgr.24625) 8206 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:28:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.238447+0000 mon.a (mon.0) 2870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.238657+0000 mgr.gibba032.whopeo (mgr.24625) 8207 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:28:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.238693+0000 mgr.gibba032.whopeo (mgr.24625) 8208 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:28:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.239772+0000 mgr.gibba032.whopeo (mgr.24625) 8209 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:28:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.239943+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.240687+0000 mgr.gibba032.whopeo (mgr.24625) 8210 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:28:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.240722+0000 mgr.gibba032.whopeo (mgr.24625) 8211 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:28:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.241251+0000 mon.a (mon.0) 2872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.241787+0000 mgr.gibba032.whopeo (mgr.24625) 8212 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:28:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.242104+0000 mgr.gibba032.whopeo (mgr.24625) 8213 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:28:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.243055+0000 mgr.gibba032.whopeo (mgr.24625) 8214 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:28:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.243092+0000 mgr.gibba032.whopeo (mgr.24625) 8215 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:28:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cephadm 2021-03-10T16:28:21.243197+0000 mgr.gibba032.whopeo (mgr.24625) 8216 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:22.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.243235+0000 mon.a (mon.0) 2873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: audit 2021-03-10T16:28:21.244575+0000 mon.a (mon.0) 2874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:22 gibba032 bash[24004]: cluster 2021-03-10T16:28:21.255315+0000 mgr.gibba032.whopeo (mgr.24625) 8217 : cluster [DBG] pgmap v4674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.028810+0000 mgr.gibba032.whopeo (mgr.24625) 8201 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:28:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.030325+0000 mgr.gibba032.whopeo (mgr.24625) 8202 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:28:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.233628+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:28:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.233870+0000 mgr.gibba032.whopeo (mgr.24625) 8203 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:28:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.235871+0000 mgr.gibba032.whopeo (mgr.24625) 8204 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:28:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.237050+0000 mon.a (mon.0) 2869 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:28:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.237235+0000 mgr.gibba032.whopeo (mgr.24625) 8205 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:28:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.237272+0000 mgr.gibba032.whopeo (mgr.24625) 8206 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:28:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.238447+0000 mon.a (mon.0) 2870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.238657+0000 mgr.gibba032.whopeo (mgr.24625) 8207 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:28:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.238693+0000 mgr.gibba032.whopeo (mgr.24625) 8208 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:28:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.239772+0000 mgr.gibba032.whopeo (mgr.24625) 8209 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:28:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.239943+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.240687+0000 mgr.gibba032.whopeo (mgr.24625) 8210 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:28:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.240722+0000 mgr.gibba032.whopeo (mgr.24625) 8211 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:28:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.241251+0000 mon.a (mon.0) 2872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.241787+0000 mgr.gibba032.whopeo (mgr.24625) 8212 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:28:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.242104+0000 mgr.gibba032.whopeo (mgr.24625) 8213 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:28:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.243055+0000 mgr.gibba032.whopeo (mgr.24625) 8214 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:28:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.243092+0000 mgr.gibba032.whopeo (mgr.24625) 8215 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:28:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cephadm 2021-03-10T16:28:21.243197+0000 mgr.gibba032.whopeo (mgr.24625) 8216 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.028810+0000 mgr.gibba032.whopeo (mgr.24625) 8201 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:28:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.030325+0000 mgr.gibba032.whopeo (mgr.24625) 8202 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:28:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.233628+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:28:22.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.233870+0000 mgr.gibba032.whopeo (mgr.24625) 8203 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:28:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.235871+0000 mgr.gibba032.whopeo (mgr.24625) 8204 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:28:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.237050+0000 mon.a (mon.0) 2869 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:28:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.237235+0000 mgr.gibba032.whopeo (mgr.24625) 8205 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:28:22.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.237272+0000 mgr.gibba032.whopeo (mgr.24625) 8206 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:28:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.238447+0000 mon.a (mon.0) 2870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.238657+0000 mgr.gibba032.whopeo (mgr.24625) 8207 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:28:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.238693+0000 mgr.gibba032.whopeo (mgr.24625) 8208 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:28:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.239772+0000 mgr.gibba032.whopeo (mgr.24625) 8209 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:28:22.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.239943+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.240687+0000 mgr.gibba032.whopeo (mgr.24625) 8210 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:28:22.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.240722+0000 mgr.gibba032.whopeo (mgr.24625) 8211 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:28:22.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.241251+0000 mon.a (mon.0) 2872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.241787+0000 mgr.gibba032.whopeo (mgr.24625) 8212 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:28:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.242104+0000 mgr.gibba032.whopeo (mgr.24625) 8213 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:28:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.243055+0000 mgr.gibba032.whopeo (mgr.24625) 8214 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:28:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.243092+0000 mgr.gibba032.whopeo (mgr.24625) 8215 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:28:22.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cephadm 2021-03-10T16:28:21.243197+0000 mgr.gibba032.whopeo (mgr.24625) 8216 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:22.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.243235+0000 mon.a (mon.0) 2873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: audit 2021-03-10T16:28:21.244575+0000 mon.a (mon.0) 2874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[30791]: cluster 2021-03-10T16:28:21.255315+0000 mgr.gibba032.whopeo (mgr.24625) 8217 : cluster [DBG] pgmap v4674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:22.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.243235+0000 mon.a (mon.0) 2873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: audit 2021-03-10T16:28:21.244575+0000 mon.a (mon.0) 2874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:22.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:22 gibba011 bash[31774]: cluster 2021-03-10T16:28:21.255315+0000 mgr.gibba032.whopeo (mgr.24625) 8217 : cluster [DBG] pgmap v4674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:24 gibba011 bash[30791]: cluster 2021-03-10T16:28:23.257009+0000 mgr.gibba032.whopeo (mgr.24625) 8218 : cluster [DBG] pgmap v4675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:24 gibba011 bash[31774]: cluster 2021-03-10T16:28:23.257009+0000 mgr.gibba032.whopeo (mgr.24625) 8218 : cluster [DBG] pgmap v4675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:24 gibba032 bash[24004]: cluster 2021-03-10T16:28:23.257009+0000 mgr.gibba032.whopeo (mgr.24625) 8218 : cluster [DBG] pgmap v4675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:26 gibba032 bash[24004]: cluster 2021-03-10T16:28:25.258593+0000 mgr.gibba032.whopeo (mgr.24625) 8219 : cluster [DBG] pgmap v4676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:26.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:26 gibba011 bash[30791]: cluster 2021-03-10T16:28:25.258593+0000 mgr.gibba032.whopeo (mgr.24625) 8219 : cluster [DBG] pgmap v4676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:26.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:26 gibba011 bash[31774]: cluster 2021-03-10T16:28:25.258593+0000 mgr.gibba032.whopeo (mgr.24625) 8219 : cluster [DBG] pgmap v4676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:28 gibba011 bash[30791]: cluster 2021-03-10T16:28:27.260449+0000 mgr.gibba032.whopeo (mgr.24625) 8220 : cluster [DBG] pgmap v4677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:28 gibba011 bash[31774]: cluster 2021-03-10T16:28:27.260449+0000 mgr.gibba032.whopeo (mgr.24625) 8220 : cluster [DBG] pgmap v4677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:28 gibba032 bash[24004]: cluster 2021-03-10T16:28:27.260449+0000 mgr.gibba032.whopeo (mgr.24625) 8220 : cluster [DBG] pgmap v4677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:30 gibba011 bash[30791]: cluster 2021-03-10T16:28:29.261780+0000 mgr.gibba032.whopeo (mgr.24625) 8221 : cluster [DBG] pgmap v4678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:30 gibba011 bash[31774]: cluster 2021-03-10T16:28:29.261780+0000 mgr.gibba032.whopeo (mgr.24625) 8221 : cluster [DBG] pgmap v4678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:31.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:30 gibba032 bash[24004]: cluster 2021-03-10T16:28:29.261780+0000 mgr.gibba032.whopeo (mgr.24625) 8221 : cluster [DBG] pgmap v4678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:32 gibba011 bash[30791]: cluster 2021-03-10T16:28:31.263337+0000 mgr.gibba032.whopeo (mgr.24625) 8222 : cluster [DBG] pgmap v4679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:32.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:32 gibba011 bash[31774]: cluster 2021-03-10T16:28:31.263337+0000 mgr.gibba032.whopeo (mgr.24625) 8222 : cluster [DBG] pgmap v4679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:32 gibba032 bash[24004]: cluster 2021-03-10T16:28:31.263337+0000 mgr.gibba032.whopeo (mgr.24625) 8222 : cluster [DBG] pgmap v4679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:34.085 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:28:34.314 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:28:34.315 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:28:34.315 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:28:34.315 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:28:34.315 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:28:34.316 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:28:34.316 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:28:34.316 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:28:34.316 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:28:34.317 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:28:34.317 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:28:34.317 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:28:34.317 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:28:34.318 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:28:34.318 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:28:34.318 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:28:34.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:28:34.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:28:34.319 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:28:34.320 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:28:34.320 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:28:34.564 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:28:34.564 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:28:34.564 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:28:34.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:34.565 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:28:34.565 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:28:34.565 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:34.565 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:28:34.565 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:28:34.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:28:34.566 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:28:34.566 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:28:34.566 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:28:34.567 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:28:34.567 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:28:34.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:34 gibba011 bash[30791]: cluster 2021-03-10T16:28:33.265155+0000 mgr.gibba032.whopeo (mgr.24625) 8223 : cluster [DBG] pgmap v4680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:34.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:34 gibba011 bash[31774]: cluster 2021-03-10T16:28:33.265155+0000 mgr.gibba032.whopeo (mgr.24625) 8223 : cluster [DBG] pgmap v4680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:34 gibba032 bash[24004]: cluster 2021-03-10T16:28:33.265155+0000 mgr.gibba032.whopeo (mgr.24625) 8223 : cluster [DBG] pgmap v4680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:35.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[30791]: audit 2021-03-10T16:28:34.075913+0000 mgr.gibba032.whopeo (mgr.24625) 8224 : audit [DBG] from='client.49563 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:35.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[30791]: audit 2021-03-10T16:28:34.302288+0000 mgr.gibba032.whopeo (mgr.24625) 8225 : audit [DBG] from='client.49569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:35.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[30791]: audit 2021-03-10T16:28:34.565863+0000 mon.a (mon.0) 2875 : audit [DBG] from='client.? 172.21.2.111:0/3040527080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:35.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[31774]: audit 2021-03-10T16:28:34.075913+0000 mgr.gibba032.whopeo (mgr.24625) 8224 : audit [DBG] from='client.49563 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:35.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[31774]: audit 2021-03-10T16:28:34.302288+0000 mgr.gibba032.whopeo (mgr.24625) 8225 : audit [DBG] from='client.49569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:35.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:35 gibba011 bash[31774]: audit 2021-03-10T16:28:34.565863+0000 mon.a (mon.0) 2875 : audit [DBG] from='client.? 172.21.2.111:0/3040527080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:36.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:35 gibba032 bash[24004]: audit 2021-03-10T16:28:34.075913+0000 mgr.gibba032.whopeo (mgr.24625) 8224 : audit [DBG] from='client.49563 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:36.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:35 gibba032 bash[24004]: audit 2021-03-10T16:28:34.302288+0000 mgr.gibba032.whopeo (mgr.24625) 8225 : audit [DBG] from='client.49569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:28:36.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:35 gibba032 bash[24004]: audit 2021-03-10T16:28:34.565863+0000 mon.a (mon.0) 2875 : audit [DBG] from='client.? 172.21.2.111:0/3040527080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:28:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:36 gibba011 bash[30791]: cluster 2021-03-10T16:28:35.266762+0000 mgr.gibba032.whopeo (mgr.24625) 8226 : cluster [DBG] pgmap v4681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:36.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:36 gibba011 bash[31774]: cluster 2021-03-10T16:28:35.266762+0000 mgr.gibba032.whopeo (mgr.24625) 8226 : cluster [DBG] pgmap v4681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:36 gibba032 bash[24004]: cluster 2021-03-10T16:28:35.266762+0000 mgr.gibba032.whopeo (mgr.24625) 8226 : cluster [DBG] pgmap v4681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:37.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:37 gibba011 bash[30791]: cephadm 2021-03-10T16:28:36.259212+0000 mgr.gibba032.whopeo (mgr.24625) 8227 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:37.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:37 gibba011 bash[31774]: cephadm 2021-03-10T16:28:36.259212+0000 mgr.gibba032.whopeo (mgr.24625) 8227 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:38.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:37 gibba032 bash[24004]: cephadm 2021-03-10T16:28:36.259212+0000 mgr.gibba032.whopeo (mgr.24625) 8227 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:38 gibba011 bash[30791]: cluster 2021-03-10T16:28:37.268597+0000 mgr.gibba032.whopeo (mgr.24625) 8228 : cluster [DBG] pgmap v4682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:38.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:38 gibba011 bash[31774]: cluster 2021-03-10T16:28:37.268597+0000 mgr.gibba032.whopeo (mgr.24625) 8228 : cluster [DBG] pgmap v4682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:39.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:38 gibba032 bash[24004]: cluster 2021-03-10T16:28:37.268597+0000 mgr.gibba032.whopeo (mgr.24625) 8228 : cluster [DBG] pgmap v4682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:40 gibba032 bash[24004]: cluster 2021-03-10T16:28:39.269958+0000 mgr.gibba032.whopeo (mgr.24625) 8229 : cluster [DBG] pgmap v4683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:40 gibba011 bash[30791]: cluster 2021-03-10T16:28:39.269958+0000 mgr.gibba032.whopeo (mgr.24625) 8229 : cluster [DBG] pgmap v4683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:40 gibba011 bash[31774]: cluster 2021-03-10T16:28:39.269958+0000 mgr.gibba032.whopeo (mgr.24625) 8229 : cluster [DBG] pgmap v4683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:42 gibba032 bash[24004]: cluster 2021-03-10T16:28:41.271485+0000 mgr.gibba032.whopeo (mgr.24625) 8230 : cluster [DBG] pgmap v4684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:43.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:42 gibba011 bash[30791]: cluster 2021-03-10T16:28:41.271485+0000 mgr.gibba032.whopeo (mgr.24625) 8230 : cluster [DBG] pgmap v4684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:42 gibba011 bash[31774]: cluster 2021-03-10T16:28:41.271485+0000 mgr.gibba032.whopeo (mgr.24625) 8230 : cluster [DBG] pgmap v4684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:44 gibba032 bash[24004]: cluster 2021-03-10T16:28:43.273340+0000 mgr.gibba032.whopeo (mgr.24625) 8231 : cluster [DBG] pgmap v4685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:44 gibba011 bash[30791]: cluster 2021-03-10T16:28:43.273340+0000 mgr.gibba032.whopeo (mgr.24625) 8231 : cluster [DBG] pgmap v4685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:44 gibba011 bash[31774]: cluster 2021-03-10T16:28:43.273340+0000 mgr.gibba032.whopeo (mgr.24625) 8231 : cluster [DBG] pgmap v4685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:46 gibba032 bash[24004]: cluster 2021-03-10T16:28:45.274943+0000 mgr.gibba032.whopeo (mgr.24625) 8232 : cluster [DBG] pgmap v4686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:46 gibba011 bash[30791]: cluster 2021-03-10T16:28:45.274943+0000 mgr.gibba032.whopeo (mgr.24625) 8232 : cluster [DBG] pgmap v4686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:46 gibba011 bash[31774]: cluster 2021-03-10T16:28:45.274943+0000 mgr.gibba032.whopeo (mgr.24625) 8232 : cluster [DBG] pgmap v4686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:48 gibba032 bash[24004]: cluster 2021-03-10T16:28:47.276491+0000 mgr.gibba032.whopeo (mgr.24625) 8233 : cluster [DBG] pgmap v4687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:48 gibba011 bash[30791]: cluster 2021-03-10T16:28:47.276491+0000 mgr.gibba032.whopeo (mgr.24625) 8233 : cluster [DBG] pgmap v4687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:48 gibba011 bash[31774]: cluster 2021-03-10T16:28:47.276491+0000 mgr.gibba032.whopeo (mgr.24625) 8233 : cluster [DBG] pgmap v4687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:51.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:50 gibba032 bash[24004]: cluster 2021-03-10T16:28:49.277817+0000 mgr.gibba032.whopeo (mgr.24625) 8234 : cluster [DBG] pgmap v4688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:50 gibba011 bash[30791]: cluster 2021-03-10T16:28:49.277817+0000 mgr.gibba032.whopeo (mgr.24625) 8234 : cluster [DBG] pgmap v4688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:50 gibba011 bash[31774]: cluster 2021-03-10T16:28:49.277817+0000 mgr.gibba032.whopeo (mgr.24625) 8234 : cluster [DBG] pgmap v4688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:52 gibba032 bash[24004]: cephadm 2021-03-10T16:28:51.275012+0000 mgr.gibba032.whopeo (mgr.24625) 8235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:52 gibba032 bash[24004]: cluster 2021-03-10T16:28:51.279267+0000 mgr.gibba032.whopeo (mgr.24625) 8236 : cluster [DBG] pgmap v4689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:53.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:52 gibba011 bash[30791]: cephadm 2021-03-10T16:28:51.275012+0000 mgr.gibba032.whopeo (mgr.24625) 8235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:52 gibba011 bash[30791]: cluster 2021-03-10T16:28:51.279267+0000 mgr.gibba032.whopeo (mgr.24625) 8236 : cluster [DBG] pgmap v4689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:52 gibba011 bash[31774]: cephadm 2021-03-10T16:28:51.275012+0000 mgr.gibba032.whopeo (mgr.24625) 8235 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:28:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:52 gibba011 bash[31774]: cluster 2021-03-10T16:28:51.279267+0000 mgr.gibba032.whopeo (mgr.24625) 8236 : cluster [DBG] pgmap v4689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:54 gibba032 bash[24004]: cluster 2021-03-10T16:28:53.281090+0000 mgr.gibba032.whopeo (mgr.24625) 8237 : cluster [DBG] pgmap v4690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:54 gibba011 bash[30791]: cluster 2021-03-10T16:28:53.281090+0000 mgr.gibba032.whopeo (mgr.24625) 8237 : cluster [DBG] pgmap v4690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:54 gibba011 bash[31774]: cluster 2021-03-10T16:28:53.281090+0000 mgr.gibba032.whopeo (mgr.24625) 8237 : cluster [DBG] pgmap v4690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:56 gibba032 bash[24004]: cluster 2021-03-10T16:28:55.282744+0000 mgr.gibba032.whopeo (mgr.24625) 8238 : cluster [DBG] pgmap v4691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:57.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:56 gibba011 bash[30791]: cluster 2021-03-10T16:28:55.282744+0000 mgr.gibba032.whopeo (mgr.24625) 8238 : cluster [DBG] pgmap v4691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:56 gibba011 bash[31774]: cluster 2021-03-10T16:28:55.282744+0000 mgr.gibba032.whopeo (mgr.24625) 8238 : cluster [DBG] pgmap v4691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.034527+0000 mon.a (mon.0) 2876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:28:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.035227+0000 mon.a (mon.0) 2877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:28:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.126168+0000 mon.a (mon.0) 2878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:28:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.126951+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:28:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.127699+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:28:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:57 gibba032 bash[24004]: audit 2021-03-10T16:28:57.128406+0000 mon.a (mon.0) 2881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:28:58.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.034527+0000 mon.a (mon.0) 2876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:28:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.035227+0000 mon.a (mon.0) 2877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:28:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.126168+0000 mon.a (mon.0) 2878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:28:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.126951+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:28:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.127699+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:28:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[30791]: audit 2021-03-10T16:28:57.128406+0000 mon.a (mon.0) 2881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:28:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.034527+0000 mon.a (mon.0) 2876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:28:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.035227+0000 mon.a (mon.0) 2877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:28:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.126168+0000 mon.a (mon.0) 2878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:28:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.126951+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:28:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.127699+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:28:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:57 gibba011 bash[31774]: audit 2021-03-10T16:28:57.128406+0000 mon.a (mon.0) 2881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:28:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:28:58 gibba032 bash[24004]: cluster 2021-03-10T16:28:57.284663+0000 mgr.gibba032.whopeo (mgr.24625) 8239 : cluster [DBG] pgmap v4692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:28:58 gibba011 bash[30791]: cluster 2021-03-10T16:28:57.284663+0000 mgr.gibba032.whopeo (mgr.24625) 8239 : cluster [DBG] pgmap v4692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:28:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:28:58 gibba011 bash[31774]: cluster 2021-03-10T16:28:57.284663+0000 mgr.gibba032.whopeo (mgr.24625) 8239 : cluster [DBG] pgmap v4692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:00 gibba032 bash[24004]: cluster 2021-03-10T16:28:59.286041+0000 mgr.gibba032.whopeo (mgr.24625) 8240 : cluster [DBG] pgmap v4693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:00 gibba011 bash[30791]: cluster 2021-03-10T16:28:59.286041+0000 mgr.gibba032.whopeo (mgr.24625) 8240 : cluster [DBG] pgmap v4693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:00 gibba011 bash[31774]: cluster 2021-03-10T16:28:59.286041+0000 mgr.gibba032.whopeo (mgr.24625) 8240 : cluster [DBG] pgmap v4693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:03.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:02 gibba032 bash[24004]: cluster 2021-03-10T16:29:01.287580+0000 mgr.gibba032.whopeo (mgr.24625) 8241 : cluster [DBG] pgmap v4694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:02 gibba011 bash[30791]: cluster 2021-03-10T16:29:01.287580+0000 mgr.gibba032.whopeo (mgr.24625) 8241 : cluster [DBG] pgmap v4694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:02 gibba011 bash[31774]: cluster 2021-03-10T16:29:01.287580+0000 mgr.gibba032.whopeo (mgr.24625) 8241 : cluster [DBG] pgmap v4694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:04.806 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:29:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:04 gibba032 bash[24004]: cluster 2021-03-10T16:29:03.289369+0000 mgr.gibba032.whopeo (mgr.24625) 8242 : cluster [DBG] pgmap v4695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:05.036 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:29:05.036 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:29:05.036 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:29:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:29:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:29:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:29:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:29:05.037 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:29:05.038 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:29:05.038 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:29:05.038 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:29:05.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:29:05.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:29:05.039 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:29:05.039 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:29:05.039 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:29:05.039 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:29:05.040 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:29:05.040 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:29:05.040 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:29:05.040 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:29:05.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:04 gibba011 bash[30791]: cluster 2021-03-10T16:29:03.289369+0000 mgr.gibba032.whopeo (mgr.24625) 8242 : cluster [DBG] pgmap v4695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:04 gibba011 bash[31774]: cluster 2021-03-10T16:29:03.289369+0000 mgr.gibba032.whopeo (mgr.24625) 8242 : cluster [DBG] pgmap v4695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:05.284 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:29:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:29:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:29:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:29:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:29:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:29:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:29:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:29:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:29:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:29:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:29:05.288 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:29:06.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:05 gibba032 bash[24004]: audit 2021-03-10T16:29:05.286305+0000 mon.c (mon.1) 159 : audit [DBG] from='client.? 172.21.2.111:0/2125945808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:06.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:05 gibba011 bash[30791]: audit 2021-03-10T16:29:05.286305+0000 mon.c (mon.1) 159 : audit [DBG] from='client.? 172.21.2.111:0/2125945808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:06.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:05 gibba011 bash[31774]: audit 2021-03-10T16:29:05.286305+0000 mon.c (mon.1) 159 : audit [DBG] from='client.? 172.21.2.111:0/2125945808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:06 gibba032 bash[24004]: audit 2021-03-10T16:29:04.797461+0000 mgr.gibba032.whopeo (mgr.24625) 8243 : audit [DBG] from='client.49581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:06 gibba032 bash[24004]: audit 2021-03-10T16:29:05.023692+0000 mgr.gibba032.whopeo (mgr.24625) 8244 : audit [DBG] from='client.59569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:06 gibba032 bash[24004]: cluster 2021-03-10T16:29:05.290847+0000 mgr.gibba032.whopeo (mgr.24625) 8245 : cluster [DBG] pgmap v4696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[30791]: audit 2021-03-10T16:29:04.797461+0000 mgr.gibba032.whopeo (mgr.24625) 8243 : audit [DBG] from='client.49581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[30791]: audit 2021-03-10T16:29:05.023692+0000 mgr.gibba032.whopeo (mgr.24625) 8244 : audit [DBG] from='client.59569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[30791]: cluster 2021-03-10T16:29:05.290847+0000 mgr.gibba032.whopeo (mgr.24625) 8245 : cluster [DBG] pgmap v4696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[31774]: audit 2021-03-10T16:29:04.797461+0000 mgr.gibba032.whopeo (mgr.24625) 8243 : audit [DBG] from='client.49581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[31774]: audit 2021-03-10T16:29:05.023692+0000 mgr.gibba032.whopeo (mgr.24625) 8244 : audit [DBG] from='client.59569 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:07.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:06 gibba011 bash[31774]: cluster 2021-03-10T16:29:05.290847+0000 mgr.gibba032.whopeo (mgr.24625) 8245 : cluster [DBG] pgmap v4696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:07 gibba032 bash[24004]: cephadm 2021-03-10T16:29:06.288968+0000 mgr.gibba032.whopeo (mgr.24625) 8246 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:08.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:07 gibba011 bash[30791]: cephadm 2021-03-10T16:29:06.288968+0000 mgr.gibba032.whopeo (mgr.24625) 8246 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:08.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:07 gibba011 bash[31774]: cephadm 2021-03-10T16:29:06.288968+0000 mgr.gibba032.whopeo (mgr.24625) 8246 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:08 gibba032 bash[24004]: cluster 2021-03-10T16:29:07.292664+0000 mgr.gibba032.whopeo (mgr.24625) 8247 : cluster [DBG] pgmap v4697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:08 gibba011 bash[30791]: cluster 2021-03-10T16:29:07.292664+0000 mgr.gibba032.whopeo (mgr.24625) 8247 : cluster [DBG] pgmap v4697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:08 gibba011 bash[31774]: cluster 2021-03-10T16:29:07.292664+0000 mgr.gibba032.whopeo (mgr.24625) 8247 : cluster [DBG] pgmap v4697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:10 gibba032 bash[24004]: cluster 2021-03-10T16:29:09.294123+0000 mgr.gibba032.whopeo (mgr.24625) 8248 : cluster [DBG] pgmap v4698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:10 gibba011 bash[30791]: cluster 2021-03-10T16:29:09.294123+0000 mgr.gibba032.whopeo (mgr.24625) 8248 : cluster [DBG] pgmap v4698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:10 gibba011 bash[31774]: cluster 2021-03-10T16:29:09.294123+0000 mgr.gibba032.whopeo (mgr.24625) 8248 : cluster [DBG] pgmap v4698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:13.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:12 gibba032 bash[24004]: cluster 2021-03-10T16:29:11.295885+0000 mgr.gibba032.whopeo (mgr.24625) 8249 : cluster [DBG] pgmap v4699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:12 gibba011 bash[30791]: cluster 2021-03-10T16:29:11.295885+0000 mgr.gibba032.whopeo (mgr.24625) 8249 : cluster [DBG] pgmap v4699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:12 gibba011 bash[31774]: cluster 2021-03-10T16:29:11.295885+0000 mgr.gibba032.whopeo (mgr.24625) 8249 : cluster [DBG] pgmap v4699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:14 gibba032 bash[24004]: cluster 2021-03-10T16:29:13.297400+0000 mgr.gibba032.whopeo (mgr.24625) 8250 : cluster [DBG] pgmap v4700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:15.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:14 gibba011 bash[30791]: cluster 2021-03-10T16:29:13.297400+0000 mgr.gibba032.whopeo (mgr.24625) 8250 : cluster [DBG] pgmap v4700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:15.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:14 gibba011 bash[31774]: cluster 2021-03-10T16:29:13.297400+0000 mgr.gibba032.whopeo (mgr.24625) 8250 : cluster [DBG] pgmap v4700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:17.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:16 gibba032 bash[24004]: cluster 2021-03-10T16:29:15.299037+0000 mgr.gibba032.whopeo (mgr.24625) 8251 : cluster [DBG] pgmap v4701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:16 gibba011 bash[30791]: cluster 2021-03-10T16:29:15.299037+0000 mgr.gibba032.whopeo (mgr.24625) 8251 : cluster [DBG] pgmap v4701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:16 gibba011 bash[31774]: cluster 2021-03-10T16:29:15.299037+0000 mgr.gibba032.whopeo (mgr.24625) 8251 : cluster [DBG] pgmap v4701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:19.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:18 gibba011 bash[30791]: cluster 2021-03-10T16:29:17.300723+0000 mgr.gibba032.whopeo (mgr.24625) 8252 : cluster [DBG] pgmap v4702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:18 gibba011 bash[31774]: cluster 2021-03-10T16:29:17.300723+0000 mgr.gibba032.whopeo (mgr.24625) 8252 : cluster [DBG] pgmap v4702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:19.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:18 gibba032 bash[24004]: cluster 2021-03-10T16:29:17.300723+0000 mgr.gibba032.whopeo (mgr.24625) 8252 : cluster [DBG] pgmap v4702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:21.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:20 gibba011 bash[30791]: cluster 2021-03-10T16:29:19.301827+0000 mgr.gibba032.whopeo (mgr.24625) 8253 : cluster [DBG] pgmap v4703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:20 gibba011 bash[31774]: cluster 2021-03-10T16:29:19.301827+0000 mgr.gibba032.whopeo (mgr.24625) 8253 : cluster [DBG] pgmap v4703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:21.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:20 gibba032 bash[24004]: cluster 2021-03-10T16:29:19.301827+0000 mgr.gibba032.whopeo (mgr.24625) 8253 : cluster [DBG] pgmap v4703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.301691+0000 mgr.gibba032.whopeo (mgr.24625) 8254 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:29:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.303280+0000 mgr.gibba032.whopeo (mgr.24625) 8255 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:29:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cluster 2021-03-10T16:29:21.303501+0000 mgr.gibba032.whopeo (mgr.24625) 8256 : cluster [DBG] pgmap v4704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.530564+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:29:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.530799+0000 mgr.gibba032.whopeo (mgr.24625) 8257 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:29:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.532406+0000 mgr.gibba032.whopeo (mgr.24625) 8258 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:29:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.533514+0000 mgr.gibba032.whopeo (mgr.24625) 8259 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:29:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.533550+0000 mgr.gibba032.whopeo (mgr.24625) 8260 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:29:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.533721+0000 mon.a (mon.0) 2883 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:29:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.534819+0000 mgr.gibba032.whopeo (mgr.24625) 8261 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:29:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.534857+0000 mgr.gibba032.whopeo (mgr.24625) 8262 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:29:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.534898+0000 mon.a (mon.0) 2884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.535882+0000 mgr.gibba032.whopeo (mgr.24625) 8263 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:29:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.536207+0000 mon.a (mon.0) 2885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.536834+0000 mgr.gibba032.whopeo (mgr.24625) 8264 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:29:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.536883+0000 mgr.gibba032.whopeo (mgr.24625) 8265 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:29:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.537365+0000 mon.a (mon.0) 2886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.538005+0000 mgr.gibba032.whopeo (mgr.24625) 8266 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:29:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.538331+0000 mgr.gibba032.whopeo (mgr.24625) 8267 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:29:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.539289+0000 mgr.gibba032.whopeo (mgr.24625) 8268 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:29:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.539324+0000 mgr.gibba032.whopeo (mgr.24625) 8269 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:29:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.301691+0000 mgr.gibba032.whopeo (mgr.24625) 8254 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:29:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.303280+0000 mgr.gibba032.whopeo (mgr.24625) 8255 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:29:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cluster 2021-03-10T16:29:21.303501+0000 mgr.gibba032.whopeo (mgr.24625) 8256 : cluster [DBG] pgmap v4704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.530564+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:29:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.530799+0000 mgr.gibba032.whopeo (mgr.24625) 8257 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:29:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.532406+0000 mgr.gibba032.whopeo (mgr.24625) 8258 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:29:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.533514+0000 mgr.gibba032.whopeo (mgr.24625) 8259 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:29:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.533550+0000 mgr.gibba032.whopeo (mgr.24625) 8260 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:29:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.533721+0000 mon.a (mon.0) 2883 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:29:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.534819+0000 mgr.gibba032.whopeo (mgr.24625) 8261 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:29:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.534857+0000 mgr.gibba032.whopeo (mgr.24625) 8262 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:29:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.534898+0000 mon.a (mon.0) 2884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.535882+0000 mgr.gibba032.whopeo (mgr.24625) 8263 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:29:22.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.536207+0000 mon.a (mon.0) 2885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.536834+0000 mgr.gibba032.whopeo (mgr.24625) 8264 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:29:22.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.536883+0000 mgr.gibba032.whopeo (mgr.24625) 8265 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:29:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.537365+0000 mon.a (mon.0) 2886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.538005+0000 mgr.gibba032.whopeo (mgr.24625) 8266 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:29:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.538331+0000 mgr.gibba032.whopeo (mgr.24625) 8267 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:29:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.539289+0000 mgr.gibba032.whopeo (mgr.24625) 8268 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:29:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.539324+0000 mgr.gibba032.whopeo (mgr.24625) 8269 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:29:22.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: cephadm 2021-03-10T16:29:21.539423+0000 mgr.gibba032.whopeo (mgr.24625) 8270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:22.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.539538+0000 mon.a (mon.0) 2887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[30791]: audit 2021-03-10T16:29:21.540836+0000 mon.a (mon.0) 2888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: cephadm 2021-03-10T16:29:21.539423+0000 mgr.gibba032.whopeo (mgr.24625) 8270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:22.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.539538+0000 mon.a (mon.0) 2887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:22.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:22 gibba011 bash[31774]: audit 2021-03-10T16:29:21.540836+0000 mon.a (mon.0) 2888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:23.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.301691+0000 mgr.gibba032.whopeo (mgr.24625) 8254 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:29:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.303280+0000 mgr.gibba032.whopeo (mgr.24625) 8255 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:29:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cluster 2021-03-10T16:29:21.303501+0000 mgr.gibba032.whopeo (mgr.24625) 8256 : cluster [DBG] pgmap v4704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.530564+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:29:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.530799+0000 mgr.gibba032.whopeo (mgr.24625) 8257 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:29:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.532406+0000 mgr.gibba032.whopeo (mgr.24625) 8258 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:29:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.533514+0000 mgr.gibba032.whopeo (mgr.24625) 8259 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:29:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.533550+0000 mgr.gibba032.whopeo (mgr.24625) 8260 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:29:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.533721+0000 mon.a (mon.0) 2883 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:29:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.534819+0000 mgr.gibba032.whopeo (mgr.24625) 8261 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:29:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.534857+0000 mgr.gibba032.whopeo (mgr.24625) 8262 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:29:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.534898+0000 mon.a (mon.0) 2884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.535882+0000 mgr.gibba032.whopeo (mgr.24625) 8263 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:29:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.536207+0000 mon.a (mon.0) 2885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.536834+0000 mgr.gibba032.whopeo (mgr.24625) 8264 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:29:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.536883+0000 mgr.gibba032.whopeo (mgr.24625) 8265 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:29:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.537365+0000 mon.a (mon.0) 2886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.538005+0000 mgr.gibba032.whopeo (mgr.24625) 8266 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:29:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.538331+0000 mgr.gibba032.whopeo (mgr.24625) 8267 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:29:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.539289+0000 mgr.gibba032.whopeo (mgr.24625) 8268 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:29:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.539324+0000 mgr.gibba032.whopeo (mgr.24625) 8269 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:29:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: cephadm 2021-03-10T16:29:21.539423+0000 mgr.gibba032.whopeo (mgr.24625) 8270 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.539538+0000 mon.a (mon.0) 2887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:22 gibba032 bash[24004]: audit 2021-03-10T16:29:21.540836+0000 mon.a (mon.0) 2888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:24 gibba011 bash[30791]: cluster 2021-03-10T16:29:23.305125+0000 mgr.gibba032.whopeo (mgr.24625) 8271 : cluster [DBG] pgmap v4705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:24.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:24 gibba011 bash[31774]: cluster 2021-03-10T16:29:23.305125+0000 mgr.gibba032.whopeo (mgr.24625) 8271 : cluster [DBG] pgmap v4705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:25.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:24 gibba032 bash[24004]: cluster 2021-03-10T16:29:23.305125+0000 mgr.gibba032.whopeo (mgr.24625) 8271 : cluster [DBG] pgmap v4705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:26.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:26 gibba011 bash[30791]: cluster 2021-03-10T16:29:25.306681+0000 mgr.gibba032.whopeo (mgr.24625) 8272 : cluster [DBG] pgmap v4706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:26.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:26 gibba011 bash[31774]: cluster 2021-03-10T16:29:25.306681+0000 mgr.gibba032.whopeo (mgr.24625) 8272 : cluster [DBG] pgmap v4706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:26 gibba032 bash[24004]: cluster 2021-03-10T16:29:25.306681+0000 mgr.gibba032.whopeo (mgr.24625) 8272 : cluster [DBG] pgmap v4706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:28 gibba011 bash[30791]: cluster 2021-03-10T16:29:27.308465+0000 mgr.gibba032.whopeo (mgr.24625) 8273 : cluster [DBG] pgmap v4707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:28.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:28 gibba011 bash[31774]: cluster 2021-03-10T16:29:27.308465+0000 mgr.gibba032.whopeo (mgr.24625) 8273 : cluster [DBG] pgmap v4707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:29.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:28 gibba032 bash[24004]: cluster 2021-03-10T16:29:27.308465+0000 mgr.gibba032.whopeo (mgr.24625) 8273 : cluster [DBG] pgmap v4707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:30 gibba011 bash[30791]: cluster 2021-03-10T16:29:29.309819+0000 mgr.gibba032.whopeo (mgr.24625) 8274 : cluster [DBG] pgmap v4708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:30 gibba011 bash[31774]: cluster 2021-03-10T16:29:29.309819+0000 mgr.gibba032.whopeo (mgr.24625) 8274 : cluster [DBG] pgmap v4708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:31.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:30 gibba032 bash[24004]: cluster 2021-03-10T16:29:29.309819+0000 mgr.gibba032.whopeo (mgr.24625) 8274 : cluster [DBG] pgmap v4708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:32 gibba011 bash[30791]: cluster 2021-03-10T16:29:31.311408+0000 mgr.gibba032.whopeo (mgr.24625) 8275 : cluster [DBG] pgmap v4709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:32.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:32 gibba011 bash[31774]: cluster 2021-03-10T16:29:31.311408+0000 mgr.gibba032.whopeo (mgr.24625) 8275 : cluster [DBG] pgmap v4709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:32 gibba032 bash[24004]: cluster 2021-03-10T16:29:31.311408+0000 mgr.gibba032.whopeo (mgr.24625) 8275 : cluster [DBG] pgmap v4709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:34 gibba011 bash[30791]: cluster 2021-03-10T16:29:33.312799+0000 mgr.gibba032.whopeo (mgr.24625) 8276 : cluster [DBG] pgmap v4710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:34.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:34 gibba011 bash[31774]: cluster 2021-03-10T16:29:33.312799+0000 mgr.gibba032.whopeo (mgr.24625) 8276 : cluster [DBG] pgmap v4710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:34 gibba032 bash[24004]: cluster 2021-03-10T16:29:33.312799+0000 mgr.gibba032.whopeo (mgr.24625) 8276 : cluster [DBG] pgmap v4710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:35.530 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:29:35.786 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:29:35.787 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:29:35.787 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:29:35.787 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:29:35.787 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:29:35.788 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:29:35.788 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:29:35.788 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:29:35.788 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:29:35.788 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:29:35.789 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:29:35.789 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:29:35.789 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:29:35.789 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:29:35.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:29:35.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:29:35.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:29:35.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:29:35.790 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:29:35.791 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:29:35.791 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:29:36.041 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:29:36.041 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:29:36.041 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:29:36.042 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:36.042 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:29:36.042 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:29:36.042 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:36.043 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:29:36.043 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:29:36.043 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:29:36.043 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:29:36.043 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:29:36.044 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:29:36.044 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:29:36.044 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:29:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:36 gibba032 bash[24004]: cluster 2021-03-10T16:29:35.314129+0000 mgr.gibba032.whopeo (mgr.24625) 8277 : cluster [DBG] pgmap v4711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:36 gibba032 bash[24004]: audit 2021-03-10T16:29:35.520857+0000 mgr.gibba032.whopeo (mgr.24625) 8278 : audit [DBG] from='client.49599 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:36 gibba032 bash[24004]: audit 2021-03-10T16:29:35.746296+0000 mgr.gibba032.whopeo (mgr.24625) 8279 : audit [DBG] from='client.64697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:36 gibba032 bash[24004]: audit 2021-03-10T16:29:36.043230+0000 mon.a (mon.0) 2889 : audit [DBG] from='client.? 172.21.2.111:0/3003947585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:37.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[30791]: cluster 2021-03-10T16:29:35.314129+0000 mgr.gibba032.whopeo (mgr.24625) 8277 : cluster [DBG] pgmap v4711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:37.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[30791]: audit 2021-03-10T16:29:35.520857+0000 mgr.gibba032.whopeo (mgr.24625) 8278 : audit [DBG] from='client.49599 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[30791]: audit 2021-03-10T16:29:35.746296+0000 mgr.gibba032.whopeo (mgr.24625) 8279 : audit [DBG] from='client.64697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[30791]: audit 2021-03-10T16:29:36.043230+0000 mon.a (mon.0) 2889 : audit [DBG] from='client.? 172.21.2.111:0/3003947585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:37.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[31774]: cluster 2021-03-10T16:29:35.314129+0000 mgr.gibba032.whopeo (mgr.24625) 8277 : cluster [DBG] pgmap v4711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:37.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[31774]: audit 2021-03-10T16:29:35.520857+0000 mgr.gibba032.whopeo (mgr.24625) 8278 : audit [DBG] from='client.49599 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[31774]: audit 2021-03-10T16:29:35.746296+0000 mgr.gibba032.whopeo (mgr.24625) 8279 : audit [DBG] from='client.64697 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:29:37.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:36 gibba011 bash[31774]: audit 2021-03-10T16:29:36.043230+0000 mon.a (mon.0) 2889 : audit [DBG] from='client.? 172.21.2.111:0/3003947585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:29:38.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:37 gibba032 bash[24004]: cephadm 2021-03-10T16:29:36.555516+0000 mgr.gibba032.whopeo (mgr.24625) 8280 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:38.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:37 gibba011 bash[30791]: cephadm 2021-03-10T16:29:36.555516+0000 mgr.gibba032.whopeo (mgr.24625) 8280 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:38.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:37 gibba011 bash[31774]: cephadm 2021-03-10T16:29:36.555516+0000 mgr.gibba032.whopeo (mgr.24625) 8280 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:38 gibba032 bash[24004]: cluster 2021-03-10T16:29:37.315632+0000 mgr.gibba032.whopeo (mgr.24625) 8281 : cluster [DBG] pgmap v4712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:39.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:38 gibba011 bash[30791]: cluster 2021-03-10T16:29:37.315632+0000 mgr.gibba032.whopeo (mgr.24625) 8281 : cluster [DBG] pgmap v4712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:38 gibba011 bash[31774]: cluster 2021-03-10T16:29:37.315632+0000 mgr.gibba032.whopeo (mgr.24625) 8281 : cluster [DBG] pgmap v4712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:41.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:40 gibba032 bash[24004]: cluster 2021-03-10T16:29:39.316905+0000 mgr.gibba032.whopeo (mgr.24625) 8282 : cluster [DBG] pgmap v4713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:41.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:40 gibba011 bash[30791]: cluster 2021-03-10T16:29:39.316905+0000 mgr.gibba032.whopeo (mgr.24625) 8282 : cluster [DBG] pgmap v4713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:40 gibba011 bash[31774]: cluster 2021-03-10T16:29:39.316905+0000 mgr.gibba032.whopeo (mgr.24625) 8282 : cluster [DBG] pgmap v4713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:42 gibba032 bash[24004]: cluster 2021-03-10T16:29:41.318497+0000 mgr.gibba032.whopeo (mgr.24625) 8283 : cluster [DBG] pgmap v4714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:42 gibba011 bash[30791]: cluster 2021-03-10T16:29:41.318497+0000 mgr.gibba032.whopeo (mgr.24625) 8283 : cluster [DBG] pgmap v4714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:42 gibba011 bash[31774]: cluster 2021-03-10T16:29:41.318497+0000 mgr.gibba032.whopeo (mgr.24625) 8283 : cluster [DBG] pgmap v4714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:44 gibba032 bash[24004]: cluster 2021-03-10T16:29:43.320135+0000 mgr.gibba032.whopeo (mgr.24625) 8284 : cluster [DBG] pgmap v4715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:44 gibba011 bash[30791]: cluster 2021-03-10T16:29:43.320135+0000 mgr.gibba032.whopeo (mgr.24625) 8284 : cluster [DBG] pgmap v4715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:44 gibba011 bash[31774]: cluster 2021-03-10T16:29:43.320135+0000 mgr.gibba032.whopeo (mgr.24625) 8284 : cluster [DBG] pgmap v4715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:46 gibba032 bash[24004]: cluster 2021-03-10T16:29:45.321520+0000 mgr.gibba032.whopeo (mgr.24625) 8285 : cluster [DBG] pgmap v4716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:46 gibba011 bash[30791]: cluster 2021-03-10T16:29:45.321520+0000 mgr.gibba032.whopeo (mgr.24625) 8285 : cluster [DBG] pgmap v4716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:46 gibba011 bash[31774]: cluster 2021-03-10T16:29:45.321520+0000 mgr.gibba032.whopeo (mgr.24625) 8285 : cluster [DBG] pgmap v4716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:48 gibba032 bash[24004]: cluster 2021-03-10T16:29:47.323283+0000 mgr.gibba032.whopeo (mgr.24625) 8286 : cluster [DBG] pgmap v4717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:48 gibba011 bash[30791]: cluster 2021-03-10T16:29:47.323283+0000 mgr.gibba032.whopeo (mgr.24625) 8286 : cluster [DBG] pgmap v4717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:48 gibba011 bash[31774]: cluster 2021-03-10T16:29:47.323283+0000 mgr.gibba032.whopeo (mgr.24625) 8286 : cluster [DBG] pgmap v4717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:50 gibba032 bash[24004]: cluster 2021-03-10T16:29:49.324542+0000 mgr.gibba032.whopeo (mgr.24625) 8287 : cluster [DBG] pgmap v4718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:50 gibba011 bash[30791]: cluster 2021-03-10T16:29:49.324542+0000 mgr.gibba032.whopeo (mgr.24625) 8287 : cluster [DBG] pgmap v4718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:51.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:50 gibba011 bash[31774]: cluster 2021-03-10T16:29:49.324542+0000 mgr.gibba032.whopeo (mgr.24625) 8287 : cluster [DBG] pgmap v4718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:52 gibba032 bash[24004]: cluster 2021-03-10T16:29:51.326206+0000 mgr.gibba032.whopeo (mgr.24625) 8288 : cluster [DBG] pgmap v4719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:52 gibba032 bash[24004]: cephadm 2021-03-10T16:29:51.571493+0000 mgr.gibba032.whopeo (mgr.24625) 8289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:53.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:52 gibba011 bash[30791]: cluster 2021-03-10T16:29:51.326206+0000 mgr.gibba032.whopeo (mgr.24625) 8288 : cluster [DBG] pgmap v4719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:52 gibba011 bash[30791]: cephadm 2021-03-10T16:29:51.571493+0000 mgr.gibba032.whopeo (mgr.24625) 8289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:52 gibba011 bash[31774]: cluster 2021-03-10T16:29:51.326206+0000 mgr.gibba032.whopeo (mgr.24625) 8288 : cluster [DBG] pgmap v4719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:52 gibba011 bash[31774]: cephadm 2021-03-10T16:29:51.571493+0000 mgr.gibba032.whopeo (mgr.24625) 8289 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:29:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:54 gibba032 bash[24004]: cluster 2021-03-10T16:29:53.327792+0000 mgr.gibba032.whopeo (mgr.24625) 8290 : cluster [DBG] pgmap v4720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:55.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:54 gibba011 bash[30791]: cluster 2021-03-10T16:29:53.327792+0000 mgr.gibba032.whopeo (mgr.24625) 8290 : cluster [DBG] pgmap v4720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:54 gibba011 bash[31774]: cluster 2021-03-10T16:29:53.327792+0000 mgr.gibba032.whopeo (mgr.24625) 8290 : cluster [DBG] pgmap v4720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:56 gibba032 bash[24004]: cluster 2021-03-10T16:29:55.329507+0000 mgr.gibba032.whopeo (mgr.24625) 8291 : cluster [DBG] pgmap v4721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:57.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:56 gibba011 bash[30791]: cluster 2021-03-10T16:29:55.329507+0000 mgr.gibba032.whopeo (mgr.24625) 8291 : cluster [DBG] pgmap v4721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:56 gibba011 bash[31774]: cluster 2021-03-10T16:29:55.329507+0000 mgr.gibba032.whopeo (mgr.24625) 8291 : cluster [DBG] pgmap v4721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.041868+0000 mon.a (mon.0) 2890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:29:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.042574+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:29:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.135716+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:29:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.136442+0000 mon.a (mon.0) 2893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T16:29:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.137104+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:29:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:57 gibba032 bash[24004]: audit 2021-03-10T16:29:57.137785+0000 mon.a (mon.0) 2895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:29:58.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.041868+0000 mon.a (mon.0) 2890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:29:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.042574+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:29:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.135716+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:29:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.136442+0000 mon.a (mon.0) 2893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T16:29:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.137104+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:29:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[30791]: audit 2021-03-10T16:29:57.137785+0000 mon.a (mon.0) 2895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:29:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.041868+0000 mon.a (mon.0) 2890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:29:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.042574+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:29:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.135716+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:29:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.136442+0000 mon.a (mon.0) 2893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T16:29:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.137104+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:29:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:57 gibba011 bash[31774]: audit 2021-03-10T16:29:57.137785+0000 mon.a (mon.0) 2895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:29:59.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:29:58 gibba032 bash[24004]: cluster 2021-03-10T16:29:57.331277+0000 mgr.gibba032.whopeo (mgr.24625) 8292 : cluster [DBG] pgmap v4722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:29:58 gibba011 bash[30791]: cluster 2021-03-10T16:29:57.331277+0000 mgr.gibba032.whopeo (mgr.24625) 8292 : cluster [DBG] pgmap v4722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:29:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:29:58 gibba011 bash[31774]: cluster 2021-03-10T16:29:57.331277+0000 mgr.gibba032.whopeo (mgr.24625) 8292 : cluster [DBG] pgmap v4722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:00.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:00 gibba011 bash[30791]: debug 2021-03-10T16:29:59.999+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:30:01.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:00 gibba032 bash[24004]: cluster 2021-03-10T16:29:59.332878+0000 mgr.gibba032.whopeo (mgr.24625) 8293 : cluster [DBG] pgmap v4723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:00 gibba032 bash[24004]: cluster 2021-03-10T16:30:00.000286+0000 mon.a (mon.0) 2896 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:30:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:00 gibba011 bash[30791]: cluster 2021-03-10T16:29:59.332878+0000 mgr.gibba032.whopeo (mgr.24625) 8293 : cluster [DBG] pgmap v4723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:00 gibba011 bash[30791]: cluster 2021-03-10T16:30:00.000286+0000 mon.a (mon.0) 2896 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:30:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:00 gibba011 bash[31774]: cluster 2021-03-10T16:29:59.332878+0000 mgr.gibba032.whopeo (mgr.24625) 8293 : cluster [DBG] pgmap v4723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:00 gibba011 bash[31774]: cluster 2021-03-10T16:30:00.000286+0000 mon.a (mon.0) 2896 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:30:03.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:02 gibba032 bash[24004]: cluster 2021-03-10T16:30:01.334534+0000 mgr.gibba032.whopeo (mgr.24625) 8294 : cluster [DBG] pgmap v4724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:02 gibba011 bash[30791]: cluster 2021-03-10T16:30:01.334534+0000 mgr.gibba032.whopeo (mgr.24625) 8294 : cluster [DBG] pgmap v4724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:02 gibba011 bash[31774]: cluster 2021-03-10T16:30:01.334534+0000 mgr.gibba032.whopeo (mgr.24625) 8294 : cluster [DBG] pgmap v4724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:04 gibba032 bash[24004]: cluster 2021-03-10T16:30:03.336345+0000 mgr.gibba032.whopeo (mgr.24625) 8295 : cluster [DBG] pgmap v4725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:04 gibba011 bash[30791]: cluster 2021-03-10T16:30:03.336345+0000 mgr.gibba032.whopeo (mgr.24625) 8295 : cluster [DBG] pgmap v4725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:04 gibba011 bash[31774]: cluster 2021-03-10T16:30:03.336345+0000 mgr.gibba032.whopeo (mgr.24625) 8295 : cluster [DBG] pgmap v4725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:06.284 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:30:06.539 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:30:06.539 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:30:06.540 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:30:06.540 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:30:06.540 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:30:06.540 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:30:06.541 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:30:06.541 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:30:06.541 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:30:06.541 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:30:06.541 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:30:06.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:30:06.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:30:06.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:30:06.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:30:06.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:30:06.543 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:30:06.543 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:30:06.543 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:30:06.543 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:30:06.544 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:30:06.797 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:30:06.797 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:30:06.797 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:30:06.798 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:06.798 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:30:06.798 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:30:06.799 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:06.799 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:30:06.799 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:30:06.799 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:06.799 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:30:06.800 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:30:06.800 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:30:06.800 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:30:06.800 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:30:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:06 gibba032 bash[24004]: cluster 2021-03-10T16:30:05.338128+0000 mgr.gibba032.whopeo (mgr.24625) 8296 : cluster [DBG] pgmap v4726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:06 gibba011 bash[30791]: cluster 2021-03-10T16:30:05.338128+0000 mgr.gibba032.whopeo (mgr.24625) 8296 : cluster [DBG] pgmap v4726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:06 gibba011 bash[31774]: cluster 2021-03-10T16:30:05.338128+0000 mgr.gibba032.whopeo (mgr.24625) 8296 : cluster [DBG] pgmap v4726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:07 gibba032 bash[24004]: audit 2021-03-10T16:30:06.275371+0000 mgr.gibba032.whopeo (mgr.24625) 8297 : audit [DBG] from='client.49617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:07 gibba032 bash[24004]: audit 2021-03-10T16:30:06.499239+0000 mgr.gibba032.whopeo (mgr.24625) 8298 : audit [DBG] from='client.59605 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:07 gibba032 bash[24004]: cephadm 2021-03-10T16:30:06.587585+0000 mgr.gibba032.whopeo (mgr.24625) 8299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:08.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:07 gibba032 bash[24004]: audit 2021-03-10T16:30:06.799417+0000 mon.a (mon.0) 2897 : audit [DBG] from='client.? 172.21.2.111:0/2452553620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:08.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[30791]: audit 2021-03-10T16:30:06.275371+0000 mgr.gibba032.whopeo (mgr.24625) 8297 : audit [DBG] from='client.49617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[30791]: audit 2021-03-10T16:30:06.499239+0000 mgr.gibba032.whopeo (mgr.24625) 8298 : audit [DBG] from='client.59605 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[30791]: cephadm 2021-03-10T16:30:06.587585+0000 mgr.gibba032.whopeo (mgr.24625) 8299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:08.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[30791]: audit 2021-03-10T16:30:06.799417+0000 mon.a (mon.0) 2897 : audit [DBG] from='client.? 172.21.2.111:0/2452553620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:08.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[31774]: audit 2021-03-10T16:30:06.275371+0000 mgr.gibba032.whopeo (mgr.24625) 8297 : audit [DBG] from='client.49617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[31774]: audit 2021-03-10T16:30:06.499239+0000 mgr.gibba032.whopeo (mgr.24625) 8298 : audit [DBG] from='client.59605 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:08.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[31774]: cephadm 2021-03-10T16:30:06.587585+0000 mgr.gibba032.whopeo (mgr.24625) 8299 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:08.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:07 gibba011 bash[31774]: audit 2021-03-10T16:30:06.799417+0000 mon.a (mon.0) 2897 : audit [DBG] from='client.? 172.21.2.111:0/2452553620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:09.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:08 gibba032 bash[24004]: cluster 2021-03-10T16:30:07.340107+0000 mgr.gibba032.whopeo (mgr.24625) 8300 : cluster [DBG] pgmap v4727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:08 gibba011 bash[30791]: cluster 2021-03-10T16:30:07.340107+0000 mgr.gibba032.whopeo (mgr.24625) 8300 : cluster [DBG] pgmap v4727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:08 gibba011 bash[31774]: cluster 2021-03-10T16:30:07.340107+0000 mgr.gibba032.whopeo (mgr.24625) 8300 : cluster [DBG] pgmap v4727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:10 gibba032 bash[24004]: cluster 2021-03-10T16:30:09.341539+0000 mgr.gibba032.whopeo (mgr.24625) 8301 : cluster [DBG] pgmap v4728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:10 gibba011 bash[30791]: cluster 2021-03-10T16:30:09.341539+0000 mgr.gibba032.whopeo (mgr.24625) 8301 : cluster [DBG] pgmap v4728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:10 gibba011 bash[31774]: cluster 2021-03-10T16:30:09.341539+0000 mgr.gibba032.whopeo (mgr.24625) 8301 : cluster [DBG] pgmap v4728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:12 gibba011 bash[30791]: cluster 2021-03-10T16:30:11.343191+0000 mgr.gibba032.whopeo (mgr.24625) 8302 : cluster [DBG] pgmap v4729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:12 gibba011 bash[31774]: cluster 2021-03-10T16:30:11.343191+0000 mgr.gibba032.whopeo (mgr.24625) 8302 : cluster [DBG] pgmap v4729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:12 gibba032 bash[24004]: cluster 2021-03-10T16:30:11.343191+0000 mgr.gibba032.whopeo (mgr.24625) 8302 : cluster [DBG] pgmap v4729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:14 gibba011 bash[30791]: cluster 2021-03-10T16:30:13.344825+0000 mgr.gibba032.whopeo (mgr.24625) 8303 : cluster [DBG] pgmap v4730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:14 gibba011 bash[31774]: cluster 2021-03-10T16:30:13.344825+0000 mgr.gibba032.whopeo (mgr.24625) 8303 : cluster [DBG] pgmap v4730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:15.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:14 gibba032 bash[24004]: cluster 2021-03-10T16:30:13.344825+0000 mgr.gibba032.whopeo (mgr.24625) 8303 : cluster [DBG] pgmap v4730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:16 gibba011 bash[30791]: cluster 2021-03-10T16:30:15.346172+0000 mgr.gibba032.whopeo (mgr.24625) 8304 : cluster [DBG] pgmap v4731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:16 gibba011 bash[31774]: cluster 2021-03-10T16:30:15.346172+0000 mgr.gibba032.whopeo (mgr.24625) 8304 : cluster [DBG] pgmap v4731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:16 gibba032 bash[24004]: cluster 2021-03-10T16:30:15.346172+0000 mgr.gibba032.whopeo (mgr.24625) 8304 : cluster [DBG] pgmap v4731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:19.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:18 gibba011 bash[30791]: cluster 2021-03-10T16:30:17.347971+0000 mgr.gibba032.whopeo (mgr.24625) 8305 : cluster [DBG] pgmap v4732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:18 gibba011 bash[31774]: cluster 2021-03-10T16:30:17.347971+0000 mgr.gibba032.whopeo (mgr.24625) 8305 : cluster [DBG] pgmap v4732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:18 gibba032 bash[24004]: cluster 2021-03-10T16:30:17.347971+0000 mgr.gibba032.whopeo (mgr.24625) 8305 : cluster [DBG] pgmap v4732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:21.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:20 gibba011 bash[30791]: cluster 2021-03-10T16:30:19.349193+0000 mgr.gibba032.whopeo (mgr.24625) 8306 : cluster [DBG] pgmap v4733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:20 gibba011 bash[31774]: cluster 2021-03-10T16:30:19.349193+0000 mgr.gibba032.whopeo (mgr.24625) 8306 : cluster [DBG] pgmap v4733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:21.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:20 gibba032 bash[24004]: cluster 2021-03-10T16:30:19.349193+0000 mgr.gibba032.whopeo (mgr.24625) 8306 : cluster [DBG] pgmap v4733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:22.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[30791]: cluster 2021-03-10T16:30:21.350794+0000 mgr.gibba032.whopeo (mgr.24625) 8307 : cluster [DBG] pgmap v4734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[30791]: cephadm 2021-03-10T16:30:21.605447+0000 mgr.gibba032.whopeo (mgr.24625) 8308 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:30:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[30791]: audit 2021-03-10T16:30:21.608214+0000 mon.a (mon.0) 2898 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:30:22.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[31774]: cluster 2021-03-10T16:30:21.350794+0000 mgr.gibba032.whopeo (mgr.24625) 8307 : cluster [DBG] pgmap v4734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[31774]: cephadm 2021-03-10T16:30:21.605447+0000 mgr.gibba032.whopeo (mgr.24625) 8308 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:30:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:21 gibba011 bash[31774]: audit 2021-03-10T16:30:21.608214+0000 mon.a (mon.0) 2898 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:30:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:21 gibba032 bash[24004]: cluster 2021-03-10T16:30:21.350794+0000 mgr.gibba032.whopeo (mgr.24625) 8307 : cluster [DBG] pgmap v4734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:21 gibba032 bash[24004]: cephadm 2021-03-10T16:30:21.605447+0000 mgr.gibba032.whopeo (mgr.24625) 8308 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:30:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:21 gibba032 bash[24004]: audit 2021-03-10T16:30:21.608214+0000 mon.a (mon.0) 2898 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:30:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:24 gibba032 bash[24004]: cluster 2021-03-10T16:30:23.352301+0000 mgr.gibba032.whopeo (mgr.24625) 8309 : cluster [DBG] pgmap v4735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:24.826 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:24 gibba011 bash[30791]: cluster 2021-03-10T16:30:23.352301+0000 mgr.gibba032.whopeo (mgr.24625) 8309 : cluster [DBG] pgmap v4735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:24.826 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:24 gibba011 bash[31774]: cluster 2021-03-10T16:30:23.352301+0000 mgr.gibba032.whopeo (mgr.24625) 8309 : cluster [DBG] pgmap v4735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.215208+0000 mgr.gibba032.whopeo (mgr.24625) 8310 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:30:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.216800+0000 mon.a (mon.0) 2899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cluster 2021-03-10T16:30:25.353711+0000 mgr.gibba032.whopeo (mgr.24625) 8311 : cluster [DBG] pgmap v4736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.424739+0000 mon.a (mon.0) 2900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.429070+0000 mgr.gibba032.whopeo (mgr.24625) 8312 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:30:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.430738+0000 mgr.gibba032.whopeo (mgr.24625) 8313 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:30:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.431834+0000 mgr.gibba032.whopeo (mgr.24625) 8314 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:30:26.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.431874+0000 mgr.gibba032.whopeo (mgr.24625) 8315 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:30:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.431966+0000 mon.a (mon.0) 2901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:30:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.432948+0000 mgr.gibba032.whopeo (mgr.24625) 8316 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:30:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.432987+0000 mgr.gibba032.whopeo (mgr.24625) 8317 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:30:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.433216+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.434072+0000 mgr.gibba032.whopeo (mgr.24625) 8318 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:30:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.434395+0000 mon.a (mon.0) 2903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.435172+0000 mgr.gibba032.whopeo (mgr.24625) 8319 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:30:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.435214+0000 mgr.gibba032.whopeo (mgr.24625) 8320 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:30:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.435368+0000 mon.a (mon.0) 2904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.436783+0000 mgr.gibba032.whopeo (mgr.24625) 8321 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:30:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.437132+0000 mgr.gibba032.whopeo (mgr.24625) 8322 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:30:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.437907+0000 mon.a (mon.0) 2905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.438392+0000 mgr.gibba032.whopeo (mgr.24625) 8323 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:30:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.438438+0000 mgr.gibba032.whopeo (mgr.24625) 8324 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:30:26.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: cephadm 2021-03-10T16:30:25.438632+0000 mgr.gibba032.whopeo (mgr.24625) 8325 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:26.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:26 gibba032 bash[24004]: audit 2021-03-10T16:30:25.439614+0000 mon.a (mon.0) 2906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.215208+0000 mgr.gibba032.whopeo (mgr.24625) 8310 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:30:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.216800+0000 mon.a (mon.0) 2899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cluster 2021-03-10T16:30:25.353711+0000 mgr.gibba032.whopeo (mgr.24625) 8311 : cluster [DBG] pgmap v4736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.424739+0000 mon.a (mon.0) 2900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.429070+0000 mgr.gibba032.whopeo (mgr.24625) 8312 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:30:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.430738+0000 mgr.gibba032.whopeo (mgr.24625) 8313 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:30:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.431834+0000 mgr.gibba032.whopeo (mgr.24625) 8314 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:30:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.431874+0000 mgr.gibba032.whopeo (mgr.24625) 8315 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:30:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.431966+0000 mon.a (mon.0) 2901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:30:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.432948+0000 mgr.gibba032.whopeo (mgr.24625) 8316 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:30:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.432987+0000 mgr.gibba032.whopeo (mgr.24625) 8317 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:30:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.433216+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.434072+0000 mgr.gibba032.whopeo (mgr.24625) 8318 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:30:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.434395+0000 mon.a (mon.0) 2903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.435172+0000 mgr.gibba032.whopeo (mgr.24625) 8319 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:30:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.435214+0000 mgr.gibba032.whopeo (mgr.24625) 8320 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:30:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.435368+0000 mon.a (mon.0) 2904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.436783+0000 mgr.gibba032.whopeo (mgr.24625) 8321 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:30:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.437132+0000 mgr.gibba032.whopeo (mgr.24625) 8322 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:30:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.437907+0000 mon.a (mon.0) 2905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.438392+0000 mgr.gibba032.whopeo (mgr.24625) 8323 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:30:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.215208+0000 mgr.gibba032.whopeo (mgr.24625) 8310 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:30:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.216800+0000 mon.a (mon.0) 2899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cluster 2021-03-10T16:30:25.353711+0000 mgr.gibba032.whopeo (mgr.24625) 8311 : cluster [DBG] pgmap v4736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.424739+0000 mon.a (mon.0) 2900 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:30:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.429070+0000 mgr.gibba032.whopeo (mgr.24625) 8312 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:30:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.430738+0000 mgr.gibba032.whopeo (mgr.24625) 8313 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:30:26.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.431834+0000 mgr.gibba032.whopeo (mgr.24625) 8314 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:30:26.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.431874+0000 mgr.gibba032.whopeo (mgr.24625) 8315 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:30:26.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.431966+0000 mon.a (mon.0) 2901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:30:26.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.432948+0000 mgr.gibba032.whopeo (mgr.24625) 8316 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:30:26.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.432987+0000 mgr.gibba032.whopeo (mgr.24625) 8317 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:30:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.433216+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.434072+0000 mgr.gibba032.whopeo (mgr.24625) 8318 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:30:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.434395+0000 mon.a (mon.0) 2903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.435172+0000 mgr.gibba032.whopeo (mgr.24625) 8319 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:30:26.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.435214+0000 mgr.gibba032.whopeo (mgr.24625) 8320 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:30:26.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.435368+0000 mon.a (mon.0) 2904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.436783+0000 mgr.gibba032.whopeo (mgr.24625) 8321 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:30:26.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.437132+0000 mgr.gibba032.whopeo (mgr.24625) 8322 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:30:26.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.437907+0000 mon.a (mon.0) 2905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.438392+0000 mgr.gibba032.whopeo (mgr.24625) 8323 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:30:26.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.438438+0000 mgr.gibba032.whopeo (mgr.24625) 8324 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:30:26.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: cephadm 2021-03-10T16:30:25.438632+0000 mgr.gibba032.whopeo (mgr.24625) 8325 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:26.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[30791]: audit 2021-03-10T16:30:25.439614+0000 mon.a (mon.0) 2906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:26.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.438438+0000 mgr.gibba032.whopeo (mgr.24625) 8324 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:30:26.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: cephadm 2021-03-10T16:30:25.438632+0000 mgr.gibba032.whopeo (mgr.24625) 8325 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:26.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:26 gibba011 bash[31774]: audit 2021-03-10T16:30:25.439614+0000 mon.a (mon.0) 2906 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:28 gibba032 bash[24004]: cluster 2021-03-10T16:30:27.355529+0000 mgr.gibba032.whopeo (mgr.24625) 8326 : cluster [DBG] pgmap v4737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:28.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:28 gibba011 bash[30791]: cluster 2021-03-10T16:30:27.355529+0000 mgr.gibba032.whopeo (mgr.24625) 8326 : cluster [DBG] pgmap v4737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:28.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:28 gibba011 bash[31774]: cluster 2021-03-10T16:30:27.355529+0000 mgr.gibba032.whopeo (mgr.24625) 8326 : cluster [DBG] pgmap v4737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:30 gibba011 bash[30791]: cluster 2021-03-10T16:30:29.356999+0000 mgr.gibba032.whopeo (mgr.24625) 8327 : cluster [DBG] pgmap v4738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:30.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:30 gibba011 bash[31774]: cluster 2021-03-10T16:30:29.356999+0000 mgr.gibba032.whopeo (mgr.24625) 8327 : cluster [DBG] pgmap v4738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:31.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:30 gibba032 bash[24004]: cluster 2021-03-10T16:30:29.356999+0000 mgr.gibba032.whopeo (mgr.24625) 8327 : cluster [DBG] pgmap v4738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:32 gibba032 bash[24004]: cluster 2021-03-10T16:30:31.358776+0000 mgr.gibba032.whopeo (mgr.24625) 8328 : cluster [DBG] pgmap v4739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:33.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:32 gibba011 bash[30791]: cluster 2021-03-10T16:30:31.358776+0000 mgr.gibba032.whopeo (mgr.24625) 8328 : cluster [DBG] pgmap v4739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:33.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:32 gibba011 bash[31774]: cluster 2021-03-10T16:30:31.358776+0000 mgr.gibba032.whopeo (mgr.24625) 8328 : cluster [DBG] pgmap v4739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:35.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:34 gibba032 bash[24004]: cluster 2021-03-10T16:30:33.360409+0000 mgr.gibba032.whopeo (mgr.24625) 8329 : cluster [DBG] pgmap v4740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:35.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:34 gibba011 bash[30791]: cluster 2021-03-10T16:30:33.360409+0000 mgr.gibba032.whopeo (mgr.24625) 8329 : cluster [DBG] pgmap v4740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:34 gibba011 bash[31774]: cluster 2021-03-10T16:30:33.360409+0000 mgr.gibba032.whopeo (mgr.24625) 8329 : cluster [DBG] pgmap v4740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:36 gibba032 bash[24004]: cluster 2021-03-10T16:30:35.361746+0000 mgr.gibba032.whopeo (mgr.24625) 8330 : cluster [DBG] pgmap v4741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:37.048 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:30:37.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:36 gibba011 bash[30791]: cluster 2021-03-10T16:30:35.361746+0000 mgr.gibba032.whopeo (mgr.24625) 8330 : cluster [DBG] pgmap v4741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:37.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:36 gibba011 bash[31774]: cluster 2021-03-10T16:30:35.361746+0000 mgr.gibba032.whopeo (mgr.24625) 8330 : cluster [DBG] pgmap v4741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:37.282 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:30:37.282 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 12s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:30:37.282 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:30:37.282 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:30:37.283 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:30:37.283 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:30:37.283 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:30:37.283 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:30:37.284 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:30:37.284 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 12s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:30:37.284 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:30:37.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:30:37.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:30:37.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:30:37.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 12s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:30:37.286 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:30:37.286 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:30:37.286 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:30:37.286 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:30:37.287 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:30:37.287 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 12s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:30:37.531 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:30:37.532 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:30:37.532 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:30:37.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:37.533 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:30:37.533 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:30:37.533 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:37.534 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:30:37.534 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:30:37.535 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:30:37.535 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:30:37.535 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:30:37.536 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:30:37.536 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:30:37.537 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:30:38.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:37 gibba032 bash[24004]: audit 2021-03-10T16:30:37.533137+0000 mon.c (mon.1) 160 : audit [DBG] from='client.? 172.21.2.111:0/1106827489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:38.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:37 gibba011 bash[30791]: audit 2021-03-10T16:30:37.533137+0000 mon.c (mon.1) 160 : audit [DBG] from='client.? 172.21.2.111:0/1106827489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:38.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:37 gibba011 bash[31774]: audit 2021-03-10T16:30:37.533137+0000 mon.c (mon.1) 160 : audit [DBG] from='client.? 172.21.2.111:0/1106827489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:30:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:38 gibba032 bash[24004]: audit 2021-03-10T16:30:37.030466+0000 mgr.gibba032.whopeo (mgr.24625) 8331 : audit [DBG] from='client.59617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:38 gibba032 bash[24004]: audit 2021-03-10T16:30:37.270166+0000 mgr.gibba032.whopeo (mgr.24625) 8332 : audit [DBG] from='client.49641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:38 gibba032 bash[24004]: cluster 2021-03-10T16:30:37.363170+0000 mgr.gibba032.whopeo (mgr.24625) 8333 : cluster [DBG] pgmap v4742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:39.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[30791]: audit 2021-03-10T16:30:37.030466+0000 mgr.gibba032.whopeo (mgr.24625) 8331 : audit [DBG] from='client.59617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[30791]: audit 2021-03-10T16:30:37.270166+0000 mgr.gibba032.whopeo (mgr.24625) 8332 : audit [DBG] from='client.49641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[30791]: cluster 2021-03-10T16:30:37.363170+0000 mgr.gibba032.whopeo (mgr.24625) 8333 : cluster [DBG] pgmap v4742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:39.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[31774]: audit 2021-03-10T16:30:37.030466+0000 mgr.gibba032.whopeo (mgr.24625) 8331 : audit [DBG] from='client.59617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[31774]: audit 2021-03-10T16:30:37.270166+0000 mgr.gibba032.whopeo (mgr.24625) 8332 : audit [DBG] from='client.49641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:30:39.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:38 gibba011 bash[31774]: cluster 2021-03-10T16:30:37.363170+0000 mgr.gibba032.whopeo (mgr.24625) 8333 : cluster [DBG] pgmap v4742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:40.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:40 gibba032 bash[24004]: cluster 2021-03-10T16:30:39.364634+0000 mgr.gibba032.whopeo (mgr.24625) 8334 : cluster [DBG] pgmap v4743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:41.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:40 gibba011 bash[30791]: cluster 2021-03-10T16:30:39.364634+0000 mgr.gibba032.whopeo (mgr.24625) 8334 : cluster [DBG] pgmap v4743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:40 gibba011 bash[31774]: cluster 2021-03-10T16:30:39.364634+0000 mgr.gibba032.whopeo (mgr.24625) 8334 : cluster [DBG] pgmap v4743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:42.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:41 gibba032 bash[24004]: cephadm 2021-03-10T16:30:40.440640+0000 mgr.gibba032.whopeo (mgr.24625) 8335 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:42.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:41 gibba011 bash[30791]: cephadm 2021-03-10T16:30:40.440640+0000 mgr.gibba032.whopeo (mgr.24625) 8335 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:42.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:41 gibba011 bash[31774]: cephadm 2021-03-10T16:30:40.440640+0000 mgr.gibba032.whopeo (mgr.24625) 8335 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:42 gibba032 bash[24004]: cluster 2021-03-10T16:30:41.366126+0000 mgr.gibba032.whopeo (mgr.24625) 8336 : cluster [DBG] pgmap v4744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:42 gibba011 bash[30791]: cluster 2021-03-10T16:30:41.366126+0000 mgr.gibba032.whopeo (mgr.24625) 8336 : cluster [DBG] pgmap v4744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:42 gibba011 bash[31774]: cluster 2021-03-10T16:30:41.366126+0000 mgr.gibba032.whopeo (mgr.24625) 8336 : cluster [DBG] pgmap v4744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:44 gibba032 bash[24004]: cluster 2021-03-10T16:30:43.367546+0000 mgr.gibba032.whopeo (mgr.24625) 8337 : cluster [DBG] pgmap v4745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:44 gibba011 bash[30791]: cluster 2021-03-10T16:30:43.367546+0000 mgr.gibba032.whopeo (mgr.24625) 8337 : cluster [DBG] pgmap v4745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:44 gibba011 bash[31774]: cluster 2021-03-10T16:30:43.367546+0000 mgr.gibba032.whopeo (mgr.24625) 8337 : cluster [DBG] pgmap v4745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:46 gibba032 bash[24004]: cluster 2021-03-10T16:30:45.369114+0000 mgr.gibba032.whopeo (mgr.24625) 8338 : cluster [DBG] pgmap v4746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:46 gibba011 bash[30791]: cluster 2021-03-10T16:30:45.369114+0000 mgr.gibba032.whopeo (mgr.24625) 8338 : cluster [DBG] pgmap v4746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:46 gibba011 bash[31774]: cluster 2021-03-10T16:30:45.369114+0000 mgr.gibba032.whopeo (mgr.24625) 8338 : cluster [DBG] pgmap v4746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:49.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:48 gibba032 bash[24004]: cluster 2021-03-10T16:30:47.370904+0000 mgr.gibba032.whopeo (mgr.24625) 8339 : cluster [DBG] pgmap v4747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:49.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:48 gibba011 bash[30791]: cluster 2021-03-10T16:30:47.370904+0000 mgr.gibba032.whopeo (mgr.24625) 8339 : cluster [DBG] pgmap v4747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:48 gibba011 bash[31774]: cluster 2021-03-10T16:30:47.370904+0000 mgr.gibba032.whopeo (mgr.24625) 8339 : cluster [DBG] pgmap v4747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:51.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:50 gibba032 bash[24004]: cluster 2021-03-10T16:30:49.372518+0000 mgr.gibba032.whopeo (mgr.24625) 8340 : cluster [DBG] pgmap v4748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:50 gibba011 bash[30791]: cluster 2021-03-10T16:30:49.372518+0000 mgr.gibba032.whopeo (mgr.24625) 8340 : cluster [DBG] pgmap v4748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:50 gibba011 bash[31774]: cluster 2021-03-10T16:30:49.372518+0000 mgr.gibba032.whopeo (mgr.24625) 8340 : cluster [DBG] pgmap v4748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:52 gibba032 bash[24004]: cluster 2021-03-10T16:30:51.374413+0000 mgr.gibba032.whopeo (mgr.24625) 8341 : cluster [DBG] pgmap v4749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:52 gibba011 bash[30791]: cluster 2021-03-10T16:30:51.374413+0000 mgr.gibba032.whopeo (mgr.24625) 8341 : cluster [DBG] pgmap v4749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:52 gibba011 bash[31774]: cluster 2021-03-10T16:30:51.374413+0000 mgr.gibba032.whopeo (mgr.24625) 8341 : cluster [DBG] pgmap v4749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:55.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:54 gibba032 bash[24004]: cluster 2021-03-10T16:30:53.376197+0000 mgr.gibba032.whopeo (mgr.24625) 8342 : cluster [DBG] pgmap v4750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:54 gibba011 bash[30791]: cluster 2021-03-10T16:30:53.376197+0000 mgr.gibba032.whopeo (mgr.24625) 8342 : cluster [DBG] pgmap v4750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:54 gibba011 bash[31774]: cluster 2021-03-10T16:30:53.376197+0000 mgr.gibba032.whopeo (mgr.24625) 8342 : cluster [DBG] pgmap v4750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:56 gibba032 bash[24004]: cluster 2021-03-10T16:30:55.377707+0000 mgr.gibba032.whopeo (mgr.24625) 8343 : cluster [DBG] pgmap v4751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:56 gibba032 bash[24004]: cephadm 2021-03-10T16:30:55.456718+0000 mgr.gibba032.whopeo (mgr.24625) 8344 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:56 gibba011 bash[30791]: cluster 2021-03-10T16:30:55.377707+0000 mgr.gibba032.whopeo (mgr.24625) 8343 : cluster [DBG] pgmap v4751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:56 gibba011 bash[30791]: cephadm 2021-03-10T16:30:55.456718+0000 mgr.gibba032.whopeo (mgr.24625) 8344 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:56 gibba011 bash[31774]: cluster 2021-03-10T16:30:55.377707+0000 mgr.gibba032.whopeo (mgr.24625) 8343 : cluster [DBG] pgmap v4751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:57.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:56 gibba011 bash[31774]: cephadm 2021-03-10T16:30:55.456718+0000 mgr.gibba032.whopeo (mgr.24625) 8344 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:30:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.048745+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:30:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.049443+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:30:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.146311+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:30:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.147064+0000 mon.a (mon.0) 2910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:30:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.147819+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:30:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:57 gibba032 bash[24004]: audit 2021-03-10T16:30:57.148553+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:30:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.048745+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:30:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.049443+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:30:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.146311+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:30:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.147064+0000 mon.a (mon.0) 2910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:30:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.147819+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:30:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[30791]: audit 2021-03-10T16:30:57.148553+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:30:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.048745+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:30:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.049443+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:30:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.146311+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:30:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.147064+0000 mon.a (mon.0) 2910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:30:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.147819+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:30:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:57 gibba011 bash[31774]: audit 2021-03-10T16:30:57.148553+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:30:59.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:30:58 gibba032 bash[24004]: cluster 2021-03-10T16:30:57.379595+0000 mgr.gibba032.whopeo (mgr.24625) 8345 : cluster [DBG] pgmap v4752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:30:58 gibba011 bash[30791]: cluster 2021-03-10T16:30:57.379595+0000 mgr.gibba032.whopeo (mgr.24625) 8345 : cluster [DBG] pgmap v4752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:30:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:30:58 gibba011 bash[31774]: cluster 2021-03-10T16:30:57.379595+0000 mgr.gibba032.whopeo (mgr.24625) 8345 : cluster [DBG] pgmap v4752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:01.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:00 gibba032 bash[24004]: cluster 2021-03-10T16:30:59.380979+0000 mgr.gibba032.whopeo (mgr.24625) 8346 : cluster [DBG] pgmap v4753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:00 gibba011 bash[30791]: cluster 2021-03-10T16:30:59.380979+0000 mgr.gibba032.whopeo (mgr.24625) 8346 : cluster [DBG] pgmap v4753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:00 gibba011 bash[31774]: cluster 2021-03-10T16:30:59.380979+0000 mgr.gibba032.whopeo (mgr.24625) 8346 : cluster [DBG] pgmap v4753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:03.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:02 gibba032 bash[24004]: cluster 2021-03-10T16:31:01.382454+0000 mgr.gibba032.whopeo (mgr.24625) 8347 : cluster [DBG] pgmap v4754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:03.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:02 gibba011 bash[30791]: cluster 2021-03-10T16:31:01.382454+0000 mgr.gibba032.whopeo (mgr.24625) 8347 : cluster [DBG] pgmap v4754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:02 gibba011 bash[31774]: cluster 2021-03-10T16:31:01.382454+0000 mgr.gibba032.whopeo (mgr.24625) 8347 : cluster [DBG] pgmap v4754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:05.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:04 gibba032 bash[24004]: cluster 2021-03-10T16:31:03.384084+0000 mgr.gibba032.whopeo (mgr.24625) 8348 : cluster [DBG] pgmap v4755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:04 gibba011 bash[30791]: cluster 2021-03-10T16:31:03.384084+0000 mgr.gibba032.whopeo (mgr.24625) 8348 : cluster [DBG] pgmap v4755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:04 gibba011 bash[31774]: cluster 2021-03-10T16:31:03.384084+0000 mgr.gibba032.whopeo (mgr.24625) 8348 : cluster [DBG] pgmap v4755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:07.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:06 gibba032 bash[24004]: cluster 2021-03-10T16:31:05.385435+0000 mgr.gibba032.whopeo (mgr.24625) 8349 : cluster [DBG] pgmap v4756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:06 gibba011 bash[30791]: cluster 2021-03-10T16:31:05.385435+0000 mgr.gibba032.whopeo (mgr.24625) 8349 : cluster [DBG] pgmap v4756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:07.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:06 gibba011 bash[31774]: cluster 2021-03-10T16:31:05.385435+0000 mgr.gibba032.whopeo (mgr.24625) 8349 : cluster [DBG] pgmap v4756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:07.777 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:31:08.000 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:31:08.001 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 42s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:31:08.001 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:31:08.002 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:31:08.002 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:31:08.002 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:31:08.002 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:31:08.002 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:31:08.003 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:31:08.003 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 42s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:31:08.003 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:31:08.003 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:31:08.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:31:08.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:31:08.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 42s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:31:08.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:31:08.005 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:31:08.005 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:31:08.006 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:31:08.006 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:31:08.007 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 42s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:31:08.262 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:31:08.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:31:08.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:31:08.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:08.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:31:08.263 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:31:08.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:08.264 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:31:08.264 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:31:08.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:08.265 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:31:08.265 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:31:08.265 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:31:08.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:31:08.265 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:31:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[30791]: cluster 2021-03-10T16:31:07.387192+0000 mgr.gibba032.whopeo (mgr.24625) 8350 : cluster [DBG] pgmap v4757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:09.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[30791]: audit 2021-03-10T16:31:07.764604+0000 mgr.gibba032.whopeo (mgr.24625) 8351 : audit [DBG] from='client.59635 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:09.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[30791]: audit 2021-03-10T16:31:08.264625+0000 mon.c (mon.1) 161 : audit [DBG] from='client.? 172.21.2.111:0/3793346977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:09.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[31774]: cluster 2021-03-10T16:31:07.387192+0000 mgr.gibba032.whopeo (mgr.24625) 8350 : cluster [DBG] pgmap v4757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:09.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[31774]: audit 2021-03-10T16:31:07.764604+0000 mgr.gibba032.whopeo (mgr.24625) 8351 : audit [DBG] from='client.59635 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:08 gibba011 bash[31774]: audit 2021-03-10T16:31:08.264625+0000 mon.c (mon.1) 161 : audit [DBG] from='client.? 172.21.2.111:0/3793346977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:09.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:08 gibba032 bash[24004]: cluster 2021-03-10T16:31:07.387192+0000 mgr.gibba032.whopeo (mgr.24625) 8350 : cluster [DBG] pgmap v4757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:09.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:08 gibba032 bash[24004]: audit 2021-03-10T16:31:07.764604+0000 mgr.gibba032.whopeo (mgr.24625) 8351 : audit [DBG] from='client.59635 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:09.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:08 gibba032 bash[24004]: audit 2021-03-10T16:31:08.264625+0000 mon.c (mon.1) 161 : audit [DBG] from='client.? 172.21.2.111:0/3793346977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:10.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:09 gibba011 bash[30791]: audit 2021-03-10T16:31:07.988595+0000 mgr.gibba032.whopeo (mgr.24625) 8352 : audit [DBG] from='client.59641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:10.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:09 gibba011 bash[31774]: audit 2021-03-10T16:31:07.988595+0000 mgr.gibba032.whopeo (mgr.24625) 8352 : audit [DBG] from='client.59641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:10.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:09 gibba032 bash[24004]: audit 2021-03-10T16:31:07.988595+0000 mgr.gibba032.whopeo (mgr.24625) 8352 : audit [DBG] from='client.59641 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:10 gibba011 bash[30791]: cluster 2021-03-10T16:31:09.388700+0000 mgr.gibba032.whopeo (mgr.24625) 8353 : cluster [DBG] pgmap v4758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:11.187 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:10 gibba011 bash[31774]: cluster 2021-03-10T16:31:09.388700+0000 mgr.gibba032.whopeo (mgr.24625) 8353 : cluster [DBG] pgmap v4758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:10 gibba032 bash[24004]: cluster 2021-03-10T16:31:09.388700+0000 mgr.gibba032.whopeo (mgr.24625) 8353 : cluster [DBG] pgmap v4758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:12.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:11 gibba011 bash[30791]: cephadm 2021-03-10T16:31:10.472773+0000 mgr.gibba032.whopeo (mgr.24625) 8354 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:12.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:11 gibba011 bash[31774]: cephadm 2021-03-10T16:31:10.472773+0000 mgr.gibba032.whopeo (mgr.24625) 8354 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:12.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:11 gibba032 bash[24004]: cephadm 2021-03-10T16:31:10.472773+0000 mgr.gibba032.whopeo (mgr.24625) 8354 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:12 gibba011 bash[30791]: cluster 2021-03-10T16:31:11.390430+0000 mgr.gibba032.whopeo (mgr.24625) 8355 : cluster [DBG] pgmap v4759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:12 gibba011 bash[31774]: cluster 2021-03-10T16:31:11.390430+0000 mgr.gibba032.whopeo (mgr.24625) 8355 : cluster [DBG] pgmap v4759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:12 gibba032 bash[24004]: cluster 2021-03-10T16:31:11.390430+0000 mgr.gibba032.whopeo (mgr.24625) 8355 : cluster [DBG] pgmap v4759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:14 gibba011 bash[30791]: cluster 2021-03-10T16:31:13.391988+0000 mgr.gibba032.whopeo (mgr.24625) 8356 : cluster [DBG] pgmap v4760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:15.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:14 gibba011 bash[31774]: cluster 2021-03-10T16:31:13.391988+0000 mgr.gibba032.whopeo (mgr.24625) 8356 : cluster [DBG] pgmap v4760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:15.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:14 gibba032 bash[24004]: cluster 2021-03-10T16:31:13.391988+0000 mgr.gibba032.whopeo (mgr.24625) 8356 : cluster [DBG] pgmap v4760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:16 gibba011 bash[30791]: cluster 2021-03-10T16:31:15.393325+0000 mgr.gibba032.whopeo (mgr.24625) 8357 : cluster [DBG] pgmap v4761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:16 gibba011 bash[31774]: cluster 2021-03-10T16:31:15.393325+0000 mgr.gibba032.whopeo (mgr.24625) 8357 : cluster [DBG] pgmap v4761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:16 gibba032 bash[24004]: cluster 2021-03-10T16:31:15.393325+0000 mgr.gibba032.whopeo (mgr.24625) 8357 : cluster [DBG] pgmap v4761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:19.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:18 gibba011 bash[30791]: cluster 2021-03-10T16:31:17.394787+0000 mgr.gibba032.whopeo (mgr.24625) 8358 : cluster [DBG] pgmap v4762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:18 gibba011 bash[31774]: cluster 2021-03-10T16:31:17.394787+0000 mgr.gibba032.whopeo (mgr.24625) 8358 : cluster [DBG] pgmap v4762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:19.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:18 gibba032 bash[24004]: cluster 2021-03-10T16:31:17.394787+0000 mgr.gibba032.whopeo (mgr.24625) 8358 : cluster [DBG] pgmap v4762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:21.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:20 gibba032 bash[24004]: cluster 2021-03-10T16:31:19.396371+0000 mgr.gibba032.whopeo (mgr.24625) 8359 : cluster [DBG] pgmap v4763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:21.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:20 gibba011 bash[30791]: cluster 2021-03-10T16:31:19.396371+0000 mgr.gibba032.whopeo (mgr.24625) 8359 : cluster [DBG] pgmap v4763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:20 gibba011 bash[31774]: cluster 2021-03-10T16:31:19.396371+0000 mgr.gibba032.whopeo (mgr.24625) 8359 : cluster [DBG] pgmap v4763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:23.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:22 gibba032 bash[24004]: cluster 2021-03-10T16:31:21.398160+0000 mgr.gibba032.whopeo (mgr.24625) 8360 : cluster [DBG] pgmap v4764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:22 gibba011 bash[30791]: cluster 2021-03-10T16:31:21.398160+0000 mgr.gibba032.whopeo (mgr.24625) 8360 : cluster [DBG] pgmap v4764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:23.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:22 gibba011 bash[31774]: cluster 2021-03-10T16:31:21.398160+0000 mgr.gibba032.whopeo (mgr.24625) 8360 : cluster [DBG] pgmap v4764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:25.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:24 gibba032 bash[24004]: cluster 2021-03-10T16:31:23.399773+0000 mgr.gibba032.whopeo (mgr.24625) 8361 : cluster [DBG] pgmap v4765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:25.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:24 gibba011 bash[30791]: cluster 2021-03-10T16:31:23.399773+0000 mgr.gibba032.whopeo (mgr.24625) 8361 : cluster [DBG] pgmap v4765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:24 gibba011 bash[31774]: cluster 2021-03-10T16:31:23.399773+0000 mgr.gibba032.whopeo (mgr.24625) 8361 : cluster [DBG] pgmap v4765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:26.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:25 gibba032 bash[24004]: audit 2021-03-10T16:31:25.487968+0000 mon.a (mon.0) 2913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:31:26.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:25 gibba011 bash[30791]: audit 2021-03-10T16:31:25.487968+0000 mon.a (mon.0) 2913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:31:26.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:25 gibba011 bash[31774]: audit 2021-03-10T16:31:25.487968+0000 mon.a (mon.0) 2913 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:31:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:26 gibba032 bash[24004]: cluster 2021-03-10T16:31:25.401257+0000 mgr.gibba032.whopeo (mgr.24625) 8362 : cluster [DBG] pgmap v4766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:27.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:26 gibba032 bash[24004]: cephadm 2021-03-10T16:31:25.484496+0000 mgr.gibba032.whopeo (mgr.24625) 8363 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:31:27.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:26 gibba011 bash[30791]: cluster 2021-03-10T16:31:25.401257+0000 mgr.gibba032.whopeo (mgr.24625) 8362 : cluster [DBG] pgmap v4766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:27.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:26 gibba011 bash[30791]: cephadm 2021-03-10T16:31:25.484496+0000 mgr.gibba032.whopeo (mgr.24625) 8363 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:31:27.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:26 gibba011 bash[31774]: cluster 2021-03-10T16:31:25.401257+0000 mgr.gibba032.whopeo (mgr.24625) 8362 : cluster [DBG] pgmap v4766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:27.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:26 gibba011 bash[31774]: cephadm 2021-03-10T16:31:25.484496+0000 mgr.gibba032.whopeo (mgr.24625) 8363 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:31:29.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:28 gibba032 bash[24004]: cluster 2021-03-10T16:31:27.402972+0000 mgr.gibba032.whopeo (mgr.24625) 8364 : cluster [DBG] pgmap v4767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:29.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:28 gibba011 bash[30791]: cluster 2021-03-10T16:31:27.402972+0000 mgr.gibba032.whopeo (mgr.24625) 8364 : cluster [DBG] pgmap v4767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:29.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:28 gibba011 bash[31774]: cluster 2021-03-10T16:31:27.402972+0000 mgr.gibba032.whopeo (mgr.24625) 8364 : cluster [DBG] pgmap v4767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.150040+0000 mgr.gibba032.whopeo (mgr.24625) 8365 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:31:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.151645+0000 mon.a (mon.0) 2914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.345968+0000 mon.a (mon.0) 2915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.346815+0000 mgr.gibba032.whopeo (mgr.24625) 8366 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:31:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.349162+0000 mgr.gibba032.whopeo (mgr.24625) 8367 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:31:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.349979+0000 mon.a (mon.0) 2916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:31:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.350861+0000 mgr.gibba032.whopeo (mgr.24625) 8368 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:31:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.350900+0000 mgr.gibba032.whopeo (mgr.24625) 8369 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:31:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.351714+0000 mon.a (mon.0) 2917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.352438+0000 mgr.gibba032.whopeo (mgr.24625) 8370 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:31:30.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.352477+0000 mgr.gibba032.whopeo (mgr.24625) 8371 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:31:30.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.353507+0000 mon.a (mon.0) 2918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.353987+0000 mgr.gibba032.whopeo (mgr.24625) 8372 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:31:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.355161+0000 mon.a (mon.0) 2919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.355735+0000 mgr.gibba032.whopeo (mgr.24625) 8373 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:31:30.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.355773+0000 mgr.gibba032.whopeo (mgr.24625) 8374 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:31:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.357440+0000 mgr.gibba032.whopeo (mgr.24625) 8375 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:31:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.357749+0000 mgr.gibba032.whopeo (mgr.24625) 8376 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:31:30.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.358589+0000 mon.a (mon.0) 2920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.359155+0000 mgr.gibba032.whopeo (mgr.24625) 8377 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:31:30.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.359195+0000 mgr.gibba032.whopeo (mgr.24625) 8378 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:31:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cephadm 2021-03-10T16:31:29.359368+0000 mgr.gibba032.whopeo (mgr.24625) 8379 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: audit 2021-03-10T16:31:29.360299+0000 mon.a (mon.0) 2921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.515 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:30 gibba032 bash[24004]: cluster 2021-03-10T16:31:29.403773+0000 mgr.gibba032.whopeo (mgr.24625) 8380 : cluster [DBG] pgmap v4768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.150040+0000 mgr.gibba032.whopeo (mgr.24625) 8365 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:31:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.151645+0000 mon.a (mon.0) 2914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.345968+0000 mon.a (mon.0) 2915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.346815+0000 mgr.gibba032.whopeo (mgr.24625) 8366 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.349162+0000 mgr.gibba032.whopeo (mgr.24625) 8367 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.349979+0000 mon.a (mon.0) 2916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.350861+0000 mgr.gibba032.whopeo (mgr.24625) 8368 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.350900+0000 mgr.gibba032.whopeo (mgr.24625) 8369 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:31:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.351714+0000 mon.a (mon.0) 2917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.352438+0000 mgr.gibba032.whopeo (mgr.24625) 8370 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.352477+0000 mgr.gibba032.whopeo (mgr.24625) 8371 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.353507+0000 mon.a (mon.0) 2918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.353987+0000 mgr.gibba032.whopeo (mgr.24625) 8372 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.355161+0000 mon.a (mon.0) 2919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.355735+0000 mgr.gibba032.whopeo (mgr.24625) 8373 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:31:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.355773+0000 mgr.gibba032.whopeo (mgr.24625) 8374 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:31:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.357440+0000 mgr.gibba032.whopeo (mgr.24625) 8375 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:31:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.357749+0000 mgr.gibba032.whopeo (mgr.24625) 8376 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:31:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.358589+0000 mon.a (mon.0) 2920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.359155+0000 mgr.gibba032.whopeo (mgr.24625) 8377 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:31:30.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.359195+0000 mgr.gibba032.whopeo (mgr.24625) 8378 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:31:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.150040+0000 mgr.gibba032.whopeo (mgr.24625) 8365 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:31:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.151645+0000 mon.a (mon.0) 2914 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.345968+0000 mon.a (mon.0) 2915 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:31:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.346815+0000 mgr.gibba032.whopeo (mgr.24625) 8366 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:31:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.349162+0000 mgr.gibba032.whopeo (mgr.24625) 8367 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:31:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.349979+0000 mon.a (mon.0) 2916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:31:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.350861+0000 mgr.gibba032.whopeo (mgr.24625) 8368 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:31:30.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.350900+0000 mgr.gibba032.whopeo (mgr.24625) 8369 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:31:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.351714+0000 mon.a (mon.0) 2917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.352438+0000 mgr.gibba032.whopeo (mgr.24625) 8370 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:31:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.352477+0000 mgr.gibba032.whopeo (mgr.24625) 8371 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:31:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.353507+0000 mon.a (mon.0) 2918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.353987+0000 mgr.gibba032.whopeo (mgr.24625) 8372 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:31:30.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.355161+0000 mon.a (mon.0) 2919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.355735+0000 mgr.gibba032.whopeo (mgr.24625) 8373 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:31:30.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.355773+0000 mgr.gibba032.whopeo (mgr.24625) 8374 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:31:30.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.357440+0000 mgr.gibba032.whopeo (mgr.24625) 8375 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:31:30.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.357749+0000 mgr.gibba032.whopeo (mgr.24625) 8376 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:31:30.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.358589+0000 mon.a (mon.0) 2920 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.359155+0000 mgr.gibba032.whopeo (mgr.24625) 8377 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:31:30.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.359195+0000 mgr.gibba032.whopeo (mgr.24625) 8378 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:31:30.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cephadm 2021-03-10T16:31:29.359368+0000 mgr.gibba032.whopeo (mgr.24625) 8379 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:30.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: audit 2021-03-10T16:31:29.360299+0000 mon.a (mon.0) 2921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[30791]: cluster 2021-03-10T16:31:29.403773+0000 mgr.gibba032.whopeo (mgr.24625) 8380 : cluster [DBG] pgmap v4768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:30.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cephadm 2021-03-10T16:31:29.359368+0000 mgr.gibba032.whopeo (mgr.24625) 8379 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:30.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: audit 2021-03-10T16:31:29.360299+0000 mon.a (mon.0) 2921 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:30.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:30 gibba011 bash[31774]: cluster 2021-03-10T16:31:29.403773+0000 mgr.gibba032.whopeo (mgr.24625) 8380 : cluster [DBG] pgmap v4768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:32.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:32 gibba032 bash[24004]: cluster 2021-03-10T16:31:31.405350+0000 mgr.gibba032.whopeo (mgr.24625) 8381 : cluster [DBG] pgmap v4769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:32 gibba011 bash[30791]: cluster 2021-03-10T16:31:31.405350+0000 mgr.gibba032.whopeo (mgr.24625) 8381 : cluster [DBG] pgmap v4769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:32.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:32 gibba011 bash[31774]: cluster 2021-03-10T16:31:31.405350+0000 mgr.gibba032.whopeo (mgr.24625) 8381 : cluster [DBG] pgmap v4769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:34.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:34 gibba032 bash[24004]: cluster 2021-03-10T16:31:33.406936+0000 mgr.gibba032.whopeo (mgr.24625) 8382 : cluster [DBG] pgmap v4770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:34 gibba011 bash[30791]: cluster 2021-03-10T16:31:33.406936+0000 mgr.gibba032.whopeo (mgr.24625) 8382 : cluster [DBG] pgmap v4770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:34.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:34 gibba011 bash[31774]: cluster 2021-03-10T16:31:33.406936+0000 mgr.gibba032.whopeo (mgr.24625) 8382 : cluster [DBG] pgmap v4770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:37.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:36 gibba032 bash[24004]: cluster 2021-03-10T16:31:35.408287+0000 mgr.gibba032.whopeo (mgr.24625) 8383 : cluster [DBG] pgmap v4771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:37.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:36 gibba011 bash[30791]: cluster 2021-03-10T16:31:35.408287+0000 mgr.gibba032.whopeo (mgr.24625) 8383 : cluster [DBG] pgmap v4771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:37.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:36 gibba011 bash[31774]: cluster 2021-03-10T16:31:35.408287+0000 mgr.gibba032.whopeo (mgr.24625) 8383 : cluster [DBG] pgmap v4771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:38.508 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:31:38.740 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:31:38.740 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 73s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:31:38.740 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:31:38.740 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:31:38.741 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:31:38.741 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:31:38.741 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:31:38.741 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:31:38.742 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:31:38.742 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 73s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:31:38.742 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:31:38.742 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:31:38.743 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:31:38.743 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:31:38.743 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 73s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:31:38.743 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:31:38.743 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:31:38.744 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:31:38.744 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:31:38.744 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:31:38.744 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 73s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:31:38.989 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:31:38.989 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:31:38.990 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:31:38.990 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:38.990 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:31:38.990 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:31:38.991 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:38.991 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:31:38.991 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:31:38.991 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:31:38.992 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:31:38.992 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:31:38.992 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:31:38.992 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:31:38.993 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:31:39.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:38 gibba032 bash[24004]: cluster 2021-03-10T16:31:37.410116+0000 mgr.gibba032.whopeo (mgr.24625) 8384 : cluster [DBG] pgmap v4772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:39.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:38 gibba011 bash[30791]: cluster 2021-03-10T16:31:37.410116+0000 mgr.gibba032.whopeo (mgr.24625) 8384 : cluster [DBG] pgmap v4772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:38 gibba011 bash[31774]: cluster 2021-03-10T16:31:37.410116+0000 mgr.gibba032.whopeo (mgr.24625) 8384 : cluster [DBG] pgmap v4772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:40.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:39 gibba032 bash[24004]: audit 2021-03-10T16:31:38.498960+0000 mgr.gibba032.whopeo (mgr.24625) 8385 : audit [DBG] from='client.49671 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:39 gibba032 bash[24004]: audit 2021-03-10T16:31:38.727850+0000 mgr.gibba032.whopeo (mgr.24625) 8386 : audit [DBG] from='client.49677 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:39 gibba032 bash[24004]: audit 2021-03-10T16:31:38.991249+0000 mon.c (mon.1) 162 : audit [DBG] from='client.? 172.21.2.111:0/1448701014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:40.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[30791]: audit 2021-03-10T16:31:38.498960+0000 mgr.gibba032.whopeo (mgr.24625) 8385 : audit [DBG] from='client.49671 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[30791]: audit 2021-03-10T16:31:38.727850+0000 mgr.gibba032.whopeo (mgr.24625) 8386 : audit [DBG] from='client.49677 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[30791]: audit 2021-03-10T16:31:38.991249+0000 mon.c (mon.1) 162 : audit [DBG] from='client.? 172.21.2.111:0/1448701014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:40.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[31774]: audit 2021-03-10T16:31:38.498960+0000 mgr.gibba032.whopeo (mgr.24625) 8385 : audit [DBG] from='client.49671 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[31774]: audit 2021-03-10T16:31:38.727850+0000 mgr.gibba032.whopeo (mgr.24625) 8386 : audit [DBG] from='client.49677 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:31:40.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:39 gibba011 bash[31774]: audit 2021-03-10T16:31:38.991249+0000 mon.c (mon.1) 162 : audit [DBG] from='client.? 172.21.2.111:0/1448701014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:31:41.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:40 gibba032 bash[24004]: cluster 2021-03-10T16:31:39.411699+0000 mgr.gibba032.whopeo (mgr.24625) 8387 : cluster [DBG] pgmap v4773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:40 gibba011 bash[30791]: cluster 2021-03-10T16:31:39.411699+0000 mgr.gibba032.whopeo (mgr.24625) 8387 : cluster [DBG] pgmap v4773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:40 gibba011 bash[31774]: cluster 2021-03-10T16:31:39.411699+0000 mgr.gibba032.whopeo (mgr.24625) 8387 : cluster [DBG] pgmap v4773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:43.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:42 gibba032 bash[24004]: cluster 2021-03-10T16:31:41.413613+0000 mgr.gibba032.whopeo (mgr.24625) 8388 : cluster [DBG] pgmap v4774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:42 gibba011 bash[30791]: cluster 2021-03-10T16:31:41.413613+0000 mgr.gibba032.whopeo (mgr.24625) 8388 : cluster [DBG] pgmap v4774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:42 gibba011 bash[31774]: cluster 2021-03-10T16:31:41.413613+0000 mgr.gibba032.whopeo (mgr.24625) 8388 : cluster [DBG] pgmap v4774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:45.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:44 gibba032 bash[24004]: cluster 2021-03-10T16:31:43.415284+0000 mgr.gibba032.whopeo (mgr.24625) 8389 : cluster [DBG] pgmap v4775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:44 gibba011 bash[30791]: cluster 2021-03-10T16:31:43.415284+0000 mgr.gibba032.whopeo (mgr.24625) 8389 : cluster [DBG] pgmap v4775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:44 gibba011 bash[31774]: cluster 2021-03-10T16:31:43.415284+0000 mgr.gibba032.whopeo (mgr.24625) 8389 : cluster [DBG] pgmap v4775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:46.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:45 gibba032 bash[24004]: cephadm 2021-03-10T16:31:44.375377+0000 mgr.gibba032.whopeo (mgr.24625) 8390 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:46.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:45 gibba011 bash[30791]: cephadm 2021-03-10T16:31:44.375377+0000 mgr.gibba032.whopeo (mgr.24625) 8390 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:46.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:45 gibba011 bash[31774]: cephadm 2021-03-10T16:31:44.375377+0000 mgr.gibba032.whopeo (mgr.24625) 8390 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:31:47.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:46 gibba032 bash[24004]: cluster 2021-03-10T16:31:45.416756+0000 mgr.gibba032.whopeo (mgr.24625) 8391 : cluster [DBG] pgmap v4776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:46 gibba011 bash[30791]: cluster 2021-03-10T16:31:45.416756+0000 mgr.gibba032.whopeo (mgr.24625) 8391 : cluster [DBG] pgmap v4776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:46 gibba011 bash[31774]: cluster 2021-03-10T16:31:45.416756+0000 mgr.gibba032.whopeo (mgr.24625) 8391 : cluster [DBG] pgmap v4776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:49.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:48 gibba032 bash[24004]: cluster 2021-03-10T16:31:47.418507+0000 mgr.gibba032.whopeo (mgr.24625) 8392 : cluster [DBG] pgmap v4777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:49.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:48 gibba011 bash[30791]: cluster 2021-03-10T16:31:47.418507+0000 mgr.gibba032.whopeo (mgr.24625) 8392 : cluster [DBG] pgmap v4777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:48 gibba011 bash[31774]: cluster 2021-03-10T16:31:47.418507+0000 mgr.gibba032.whopeo (mgr.24625) 8392 : cluster [DBG] pgmap v4777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:51.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:50 gibba032 bash[24004]: cluster 2021-03-10T16:31:49.419795+0000 mgr.gibba032.whopeo (mgr.24625) 8393 : cluster [DBG] pgmap v4778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:50 gibba011 bash[30791]: cluster 2021-03-10T16:31:49.419795+0000 mgr.gibba032.whopeo (mgr.24625) 8393 : cluster [DBG] pgmap v4778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:50 gibba011 bash[31774]: cluster 2021-03-10T16:31:49.419795+0000 mgr.gibba032.whopeo (mgr.24625) 8393 : cluster [DBG] pgmap v4778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:53.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:52 gibba032 bash[24004]: cluster 2021-03-10T16:31:51.421551+0000 mgr.gibba032.whopeo (mgr.24625) 8394 : cluster [DBG] pgmap v4779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:53.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:52 gibba011 bash[30791]: cluster 2021-03-10T16:31:51.421551+0000 mgr.gibba032.whopeo (mgr.24625) 8394 : cluster [DBG] pgmap v4779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:52 gibba011 bash[31774]: cluster 2021-03-10T16:31:51.421551+0000 mgr.gibba032.whopeo (mgr.24625) 8394 : cluster [DBG] pgmap v4779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:55.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:54 gibba032 bash[24004]: cluster 2021-03-10T16:31:53.423185+0000 mgr.gibba032.whopeo (mgr.24625) 8395 : cluster [DBG] pgmap v4780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:54 gibba011 bash[30791]: cluster 2021-03-10T16:31:53.423185+0000 mgr.gibba032.whopeo (mgr.24625) 8395 : cluster [DBG] pgmap v4780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:54 gibba011 bash[31774]: cluster 2021-03-10T16:31:53.423185+0000 mgr.gibba032.whopeo (mgr.24625) 8395 : cluster [DBG] pgmap v4780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:57.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:56 gibba032 bash[24004]: cluster 2021-03-10T16:31:55.424633+0000 mgr.gibba032.whopeo (mgr.24625) 8396 : cluster [DBG] pgmap v4781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:56 gibba011 bash[30791]: cluster 2021-03-10T16:31:55.424633+0000 mgr.gibba032.whopeo (mgr.24625) 8396 : cluster [DBG] pgmap v4781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:56 gibba011 bash[31774]: cluster 2021-03-10T16:31:55.424633+0000 mgr.gibba032.whopeo (mgr.24625) 8396 : cluster [DBG] pgmap v4781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.056028+0000 mon.a (mon.0) 2922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:31:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.056824+0000 mon.a (mon.0) 2923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:31:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.157065+0000 mon.a (mon.0) 2924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:31:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.157866+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.158527+0000 mon.a (mon.0) 2926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:57 gibba032 bash[24004]: audit 2021-03-10T16:31:57.159204+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.056028+0000 mon.a (mon.0) 2922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:31:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.056824+0000 mon.a (mon.0) 2923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:31:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.157065+0000 mon.a (mon.0) 2924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:31:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.157866+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.158527+0000 mon.a (mon.0) 2926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[30791]: audit 2021-03-10T16:31:57.159204+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.056028+0000 mon.a (mon.0) 2922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:31:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.056824+0000 mon.a (mon.0) 2923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:31:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.157065+0000 mon.a (mon.0) 2924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T16:31:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.157866+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.158527+0000 mon.a (mon.0) 2926 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:31:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:57 gibba011 bash[31774]: audit 2021-03-10T16:31:57.159204+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:31:59.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:31:58 gibba032 bash[24004]: cluster 2021-03-10T16:31:57.426385+0000 mgr.gibba032.whopeo (mgr.24625) 8397 : cluster [DBG] pgmap v4782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:31:58 gibba011 bash[30791]: cluster 2021-03-10T16:31:57.426385+0000 mgr.gibba032.whopeo (mgr.24625) 8397 : cluster [DBG] pgmap v4782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:31:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:31:58 gibba011 bash[31774]: cluster 2021-03-10T16:31:57.426385+0000 mgr.gibba032.whopeo (mgr.24625) 8397 : cluster [DBG] pgmap v4782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:01.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:00 gibba032 bash[24004]: cephadm 2021-03-10T16:31:59.384218+0000 mgr.gibba032.whopeo (mgr.24625) 8398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:01.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:00 gibba032 bash[24004]: cluster 2021-03-10T16:31:59.427622+0000 mgr.gibba032.whopeo (mgr.24625) 8399 : cluster [DBG] pgmap v4783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:00 gibba011 bash[30791]: cephadm 2021-03-10T16:31:59.384218+0000 mgr.gibba032.whopeo (mgr.24625) 8398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:00 gibba011 bash[30791]: cluster 2021-03-10T16:31:59.427622+0000 mgr.gibba032.whopeo (mgr.24625) 8399 : cluster [DBG] pgmap v4783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:00 gibba011 bash[31774]: cephadm 2021-03-10T16:31:59.384218+0000 mgr.gibba032.whopeo (mgr.24625) 8398 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:00 gibba011 bash[31774]: cluster 2021-03-10T16:31:59.427622+0000 mgr.gibba032.whopeo (mgr.24625) 8399 : cluster [DBG] pgmap v4783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:03.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:02 gibba032 bash[24004]: cluster 2021-03-10T16:32:01.429345+0000 mgr.gibba032.whopeo (mgr.24625) 8400 : cluster [DBG] pgmap v4784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:03.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:02 gibba011 bash[30791]: cluster 2021-03-10T16:32:01.429345+0000 mgr.gibba032.whopeo (mgr.24625) 8400 : cluster [DBG] pgmap v4784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:02 gibba011 bash[31774]: cluster 2021-03-10T16:32:01.429345+0000 mgr.gibba032.whopeo (mgr.24625) 8400 : cluster [DBG] pgmap v4784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:05.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:04 gibba032 bash[24004]: cluster 2021-03-10T16:32:03.430905+0000 mgr.gibba032.whopeo (mgr.24625) 8401 : cluster [DBG] pgmap v4785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:05.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:04 gibba011 bash[30791]: cluster 2021-03-10T16:32:03.430905+0000 mgr.gibba032.whopeo (mgr.24625) 8401 : cluster [DBG] pgmap v4785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:04 gibba011 bash[31774]: cluster 2021-03-10T16:32:03.430905+0000 mgr.gibba032.whopeo (mgr.24625) 8401 : cluster [DBG] pgmap v4785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:07.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:06 gibba032 bash[24004]: cluster 2021-03-10T16:32:05.432423+0000 mgr.gibba032.whopeo (mgr.24625) 8402 : cluster [DBG] pgmap v4786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:07.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:06 gibba011 bash[30791]: cluster 2021-03-10T16:32:05.432423+0000 mgr.gibba032.whopeo (mgr.24625) 8402 : cluster [DBG] pgmap v4786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:06 gibba011 bash[31774]: cluster 2021-03-10T16:32:05.432423+0000 mgr.gibba032.whopeo (mgr.24625) 8402 : cluster [DBG] pgmap v4786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:09.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:08 gibba032 bash[24004]: cluster 2021-03-10T16:32:07.434160+0000 mgr.gibba032.whopeo (mgr.24625) 8403 : cluster [DBG] pgmap v4787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:09.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:08 gibba011 bash[30791]: cluster 2021-03-10T16:32:07.434160+0000 mgr.gibba032.whopeo (mgr.24625) 8403 : cluster [DBG] pgmap v4787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:09.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:08 gibba011 bash[31774]: cluster 2021-03-10T16:32:07.434160+0000 mgr.gibba032.whopeo (mgr.24625) 8403 : cluster [DBG] pgmap v4787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:09.232 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:32:09.458 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:32:09.458 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 104s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:32:09.458 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 40s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:32:09.459 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:32:09.459 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:32:09.459 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:32:09.459 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:32:09.460 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:32:09.460 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:32:09.460 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 104s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:32:09.460 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 40s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:32:09.460 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:32:09.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:32:09.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:32:09.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 104s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:32:09.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:32:09.461 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:32:09.462 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:32:09.462 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 40s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:32:09.462 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 40s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:32:09.462 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 104s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:32:09.707 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:32:09.708 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:32:09.708 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:32:09.708 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:09.708 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:32:09.709 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:32:09.709 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:09.709 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:32:09.709 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:32:09.709 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:09.710 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:32:09.710 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:32:09.710 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:32:09.710 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:32:09.710 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:32:11.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:10 gibba032 bash[24004]: audit 2021-03-10T16:32:09.222761+0000 mgr.gibba032.whopeo (mgr.24625) 8404 : audit [DBG] from='client.49689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:10 gibba032 bash[24004]: cluster 2021-03-10T16:32:09.435428+0000 mgr.gibba032.whopeo (mgr.24625) 8405 : cluster [DBG] pgmap v4788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:10 gibba032 bash[24004]: audit 2021-03-10T16:32:09.446173+0000 mgr.gibba032.whopeo (mgr.24625) 8406 : audit [DBG] from='client.49695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:10 gibba032 bash[24004]: audit 2021-03-10T16:32:09.709892+0000 mon.a (mon.0) 2928 : audit [DBG] from='client.? 172.21.2.111:0/528841239' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[30791]: audit 2021-03-10T16:32:09.222761+0000 mgr.gibba032.whopeo (mgr.24625) 8404 : audit [DBG] from='client.49689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[30791]: cluster 2021-03-10T16:32:09.435428+0000 mgr.gibba032.whopeo (mgr.24625) 8405 : cluster [DBG] pgmap v4788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[30791]: audit 2021-03-10T16:32:09.446173+0000 mgr.gibba032.whopeo (mgr.24625) 8406 : audit [DBG] from='client.49695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[30791]: audit 2021-03-10T16:32:09.709892+0000 mon.a (mon.0) 2928 : audit [DBG] from='client.? 172.21.2.111:0/528841239' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:11.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[31774]: audit 2021-03-10T16:32:09.222761+0000 mgr.gibba032.whopeo (mgr.24625) 8404 : audit [DBG] from='client.49689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[31774]: cluster 2021-03-10T16:32:09.435428+0000 mgr.gibba032.whopeo (mgr.24625) 8405 : cluster [DBG] pgmap v4788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[31774]: audit 2021-03-10T16:32:09.446173+0000 mgr.gibba032.whopeo (mgr.24625) 8406 : audit [DBG] from='client.49695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:10 gibba011 bash[31774]: audit 2021-03-10T16:32:09.709892+0000 mon.a (mon.0) 2928 : audit [DBG] from='client.? 172.21.2.111:0/528841239' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:13.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:12 gibba032 bash[24004]: cluster 2021-03-10T16:32:11.437094+0000 mgr.gibba032.whopeo (mgr.24625) 8407 : cluster [DBG] pgmap v4789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:12 gibba011 bash[30791]: cluster 2021-03-10T16:32:11.437094+0000 mgr.gibba032.whopeo (mgr.24625) 8407 : cluster [DBG] pgmap v4789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:12 gibba011 bash[31774]: cluster 2021-03-10T16:32:11.437094+0000 mgr.gibba032.whopeo (mgr.24625) 8407 : cluster [DBG] pgmap v4789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:14.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:14 gibba032 bash[24004]: cluster 2021-03-10T16:32:13.438659+0000 mgr.gibba032.whopeo (mgr.24625) 8408 : cluster [DBG] pgmap v4790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:14 gibba011 bash[30791]: cluster 2021-03-10T16:32:13.438659+0000 mgr.gibba032.whopeo (mgr.24625) 8408 : cluster [DBG] pgmap v4790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:15.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:14 gibba011 bash[31774]: cluster 2021-03-10T16:32:13.438659+0000 mgr.gibba032.whopeo (mgr.24625) 8408 : cluster [DBG] pgmap v4790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:16.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:15 gibba032 bash[24004]: cephadm 2021-03-10T16:32:14.400197+0000 mgr.gibba032.whopeo (mgr.24625) 8409 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:16.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:15 gibba011 bash[30791]: cephadm 2021-03-10T16:32:14.400197+0000 mgr.gibba032.whopeo (mgr.24625) 8409 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:16.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:15 gibba011 bash[31774]: cephadm 2021-03-10T16:32:14.400197+0000 mgr.gibba032.whopeo (mgr.24625) 8409 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:16 gibba032 bash[24004]: cluster 2021-03-10T16:32:15.440276+0000 mgr.gibba032.whopeo (mgr.24625) 8410 : cluster [DBG] pgmap v4791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:17.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:16 gibba011 bash[30791]: cluster 2021-03-10T16:32:15.440276+0000 mgr.gibba032.whopeo (mgr.24625) 8410 : cluster [DBG] pgmap v4791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:16 gibba011 bash[31774]: cluster 2021-03-10T16:32:15.440276+0000 mgr.gibba032.whopeo (mgr.24625) 8410 : cluster [DBG] pgmap v4791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:19.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:18 gibba032 bash[24004]: cluster 2021-03-10T16:32:17.442093+0000 mgr.gibba032.whopeo (mgr.24625) 8411 : cluster [DBG] pgmap v4792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:19.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:18 gibba011 bash[30791]: cluster 2021-03-10T16:32:17.442093+0000 mgr.gibba032.whopeo (mgr.24625) 8411 : cluster [DBG] pgmap v4792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:18 gibba011 bash[31774]: cluster 2021-03-10T16:32:17.442093+0000 mgr.gibba032.whopeo (mgr.24625) 8411 : cluster [DBG] pgmap v4792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:21.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:20 gibba032 bash[24004]: cluster 2021-03-10T16:32:19.443580+0000 mgr.gibba032.whopeo (mgr.24625) 8412 : cluster [DBG] pgmap v4793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:21.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:20 gibba011 bash[30791]: cluster 2021-03-10T16:32:19.443580+0000 mgr.gibba032.whopeo (mgr.24625) 8412 : cluster [DBG] pgmap v4793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:20 gibba011 bash[31774]: cluster 2021-03-10T16:32:19.443580+0000 mgr.gibba032.whopeo (mgr.24625) 8412 : cluster [DBG] pgmap v4793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:23.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:22 gibba032 bash[24004]: cluster 2021-03-10T16:32:21.445317+0000 mgr.gibba032.whopeo (mgr.24625) 8413 : cluster [DBG] pgmap v4794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:22 gibba011 bash[30791]: cluster 2021-03-10T16:32:21.445317+0000 mgr.gibba032.whopeo (mgr.24625) 8413 : cluster [DBG] pgmap v4794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:23.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:22 gibba011 bash[31774]: cluster 2021-03-10T16:32:21.445317+0000 mgr.gibba032.whopeo (mgr.24625) 8413 : cluster [DBG] pgmap v4794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:25.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:24 gibba032 bash[24004]: cluster 2021-03-10T16:32:23.446907+0000 mgr.gibba032.whopeo (mgr.24625) 8414 : cluster [DBG] pgmap v4795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:25.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:24 gibba011 bash[30791]: cluster 2021-03-10T16:32:23.446907+0000 mgr.gibba032.whopeo (mgr.24625) 8414 : cluster [DBG] pgmap v4795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:24 gibba011 bash[31774]: cluster 2021-03-10T16:32:23.446907+0000 mgr.gibba032.whopeo (mgr.24625) 8414 : cluster [DBG] pgmap v4795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:27.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:26 gibba032 bash[24004]: cluster 2021-03-10T16:32:25.448613+0000 mgr.gibba032.whopeo (mgr.24625) 8415 : cluster [DBG] pgmap v4796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:27.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:26 gibba011 bash[30791]: cluster 2021-03-10T16:32:25.448613+0000 mgr.gibba032.whopeo (mgr.24625) 8415 : cluster [DBG] pgmap v4796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:27.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:26 gibba011 bash[31774]: cluster 2021-03-10T16:32:25.448613+0000 mgr.gibba032.whopeo (mgr.24625) 8415 : cluster [DBG] pgmap v4796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:29.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:28 gibba032 bash[24004]: cluster 2021-03-10T16:32:27.450224+0000 mgr.gibba032.whopeo (mgr.24625) 8416 : cluster [DBG] pgmap v4797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:29.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:28 gibba011 bash[30791]: cluster 2021-03-10T16:32:27.450224+0000 mgr.gibba032.whopeo (mgr.24625) 8416 : cluster [DBG] pgmap v4797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:29.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:28 gibba011 bash[31774]: cluster 2021-03-10T16:32:27.450224+0000 mgr.gibba032.whopeo (mgr.24625) 8416 : cluster [DBG] pgmap v4797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.416978+0000 mgr.gibba032.whopeo (mgr.24625) 8417 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:32:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.418558+0000 mgr.gibba032.whopeo (mgr.24625) 8418 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:32:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cluster 2021-03-10T16:32:29.450707+0000 mgr.gibba032.whopeo (mgr.24625) 8419 : cluster [DBG] pgmap v4798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.640930+0000 mon.a (mon.0) 2929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:32:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.645122+0000 mgr.gibba032.whopeo (mgr.24625) 8420 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:32:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.646670+0000 mgr.gibba032.whopeo (mgr.24625) 8421 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:32:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.647895+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:32:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.648167+0000 mgr.gibba032.whopeo (mgr.24625) 8422 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:32:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.648206+0000 mgr.gibba032.whopeo (mgr.24625) 8423 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:32:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.649136+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.649296+0000 mgr.gibba032.whopeo (mgr.24625) 8424 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:32:31.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.649330+0000 mgr.gibba032.whopeo (mgr.24625) 8425 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:32:31.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.650321+0000 mgr.gibba032.whopeo (mgr.24625) 8426 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:32:31.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.650719+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.651492+0000 mgr.gibba032.whopeo (mgr.24625) 8427 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:32:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.651527+0000 mgr.gibba032.whopeo (mgr.24625) 8428 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:32:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.651673+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.652905+0000 mgr.gibba032.whopeo (mgr.24625) 8429 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:32:31.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.653224+0000 mgr.gibba032.whopeo (mgr.24625) 8430 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:32:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.654002+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.654601+0000 mgr.gibba032.whopeo (mgr.24625) 8431 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:32:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.654640+0000 mgr.gibba032.whopeo (mgr.24625) 8432 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:32:31.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: cephadm 2021-03-10T16:32:29.654745+0000 mgr.gibba032.whopeo (mgr.24625) 8433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:31.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:30 gibba032 bash[24004]: audit 2021-03-10T16:32:29.655766+0000 mon.a (mon.0) 2935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.416978+0000 mgr.gibba032.whopeo (mgr.24625) 8417 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:32:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.418558+0000 mgr.gibba032.whopeo (mgr.24625) 8418 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:32:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cluster 2021-03-10T16:32:29.450707+0000 mgr.gibba032.whopeo (mgr.24625) 8419 : cluster [DBG] pgmap v4798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.640930+0000 mon.a (mon.0) 2929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:32:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.645122+0000 mgr.gibba032.whopeo (mgr.24625) 8420 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:32:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.646670+0000 mgr.gibba032.whopeo (mgr.24625) 8421 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:32:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.647895+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:32:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.648167+0000 mgr.gibba032.whopeo (mgr.24625) 8422 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:32:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.648206+0000 mgr.gibba032.whopeo (mgr.24625) 8423 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:32:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.649136+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.649296+0000 mgr.gibba032.whopeo (mgr.24625) 8424 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:32:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.649330+0000 mgr.gibba032.whopeo (mgr.24625) 8425 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:32:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.650321+0000 mgr.gibba032.whopeo (mgr.24625) 8426 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:32:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.650719+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.651492+0000 mgr.gibba032.whopeo (mgr.24625) 8427 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:32:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.651527+0000 mgr.gibba032.whopeo (mgr.24625) 8428 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:32:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.651673+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.652905+0000 mgr.gibba032.whopeo (mgr.24625) 8429 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:32:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.653224+0000 mgr.gibba032.whopeo (mgr.24625) 8430 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:32:31.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.654002+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.654601+0000 mgr.gibba032.whopeo (mgr.24625) 8431 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:32:31.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.416978+0000 mgr.gibba032.whopeo (mgr.24625) 8417 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:32:31.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.418558+0000 mgr.gibba032.whopeo (mgr.24625) 8418 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:32:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cluster 2021-03-10T16:32:29.450707+0000 mgr.gibba032.whopeo (mgr.24625) 8419 : cluster [DBG] pgmap v4798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.640930+0000 mon.a (mon.0) 2929 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:32:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.645122+0000 mgr.gibba032.whopeo (mgr.24625) 8420 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:32:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.646670+0000 mgr.gibba032.whopeo (mgr.24625) 8421 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:32:31.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.647895+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:32:31.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.648167+0000 mgr.gibba032.whopeo (mgr.24625) 8422 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:32:31.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.648206+0000 mgr.gibba032.whopeo (mgr.24625) 8423 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:32:31.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.649136+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.649296+0000 mgr.gibba032.whopeo (mgr.24625) 8424 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:32:31.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.649330+0000 mgr.gibba032.whopeo (mgr.24625) 8425 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:32:31.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.650321+0000 mgr.gibba032.whopeo (mgr.24625) 8426 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:32:31.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.650719+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.651492+0000 mgr.gibba032.whopeo (mgr.24625) 8427 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:32:31.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.651527+0000 mgr.gibba032.whopeo (mgr.24625) 8428 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:32:31.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.651673+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.652905+0000 mgr.gibba032.whopeo (mgr.24625) 8429 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:32:31.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.653224+0000 mgr.gibba032.whopeo (mgr.24625) 8430 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:32:31.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.654002+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.654601+0000 mgr.gibba032.whopeo (mgr.24625) 8431 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:32:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.654640+0000 mgr.gibba032.whopeo (mgr.24625) 8432 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:32:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: cephadm 2021-03-10T16:32:29.654745+0000 mgr.gibba032.whopeo (mgr.24625) 8433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:31.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[30791]: audit 2021-03-10T16:32:29.655766+0000 mon.a (mon.0) 2935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.654640+0000 mgr.gibba032.whopeo (mgr.24625) 8432 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:32:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: cephadm 2021-03-10T16:32:29.654745+0000 mgr.gibba032.whopeo (mgr.24625) 8433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:31.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:30 gibba011 bash[31774]: audit 2021-03-10T16:32:29.655766+0000 mon.a (mon.0) 2935 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:32 gibba032 bash[24004]: cluster 2021-03-10T16:32:31.452065+0000 mgr.gibba032.whopeo (mgr.24625) 8434 : cluster [DBG] pgmap v4799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:33.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:32 gibba011 bash[30791]: cluster 2021-03-10T16:32:31.452065+0000 mgr.gibba032.whopeo (mgr.24625) 8434 : cluster [DBG] pgmap v4799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:33.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:32 gibba011 bash[31774]: cluster 2021-03-10T16:32:31.452065+0000 mgr.gibba032.whopeo (mgr.24625) 8434 : cluster [DBG] pgmap v4799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:35.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:34 gibba032 bash[24004]: cluster 2021-03-10T16:32:33.453641+0000 mgr.gibba032.whopeo (mgr.24625) 8435 : cluster [DBG] pgmap v4800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:35.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:34 gibba011 bash[30791]: cluster 2021-03-10T16:32:33.453641+0000 mgr.gibba032.whopeo (mgr.24625) 8435 : cluster [DBG] pgmap v4800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:34 gibba011 bash[31774]: cluster 2021-03-10T16:32:33.453641+0000 mgr.gibba032.whopeo (mgr.24625) 8435 : cluster [DBG] pgmap v4800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:36 gibba032 bash[24004]: cluster 2021-03-10T16:32:35.455096+0000 mgr.gibba032.whopeo (mgr.24625) 8436 : cluster [DBG] pgmap v4801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:37.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:36 gibba011 bash[30791]: cluster 2021-03-10T16:32:35.455096+0000 mgr.gibba032.whopeo (mgr.24625) 8436 : cluster [DBG] pgmap v4801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:37.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:36 gibba011 bash[31774]: cluster 2021-03-10T16:32:35.455096+0000 mgr.gibba032.whopeo (mgr.24625) 8436 : cluster [DBG] pgmap v4801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:39.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:38 gibba032 bash[24004]: cluster 2021-03-10T16:32:37.456871+0000 mgr.gibba032.whopeo (mgr.24625) 8437 : cluster [DBG] pgmap v4802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:39.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:38 gibba011 bash[30791]: cluster 2021-03-10T16:32:37.456871+0000 mgr.gibba032.whopeo (mgr.24625) 8437 : cluster [DBG] pgmap v4802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:38 gibba011 bash[31774]: cluster 2021-03-10T16:32:37.456871+0000 mgr.gibba032.whopeo (mgr.24625) 8437 : cluster [DBG] pgmap v4802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:39.948 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:32:40.174 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:32:40.175 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:32:40.175 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 71s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:32:40.175 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:32:40.175 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:32:40.176 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:32:40.176 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:32:40.176 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:32:40.176 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:32:40.176 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:32:40.177 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 71s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:32:40.177 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:32:40.177 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:32:40.177 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:32:40.177 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:32:40.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:32:40.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:32:40.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:32:40.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 71s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:32:40.178 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 71s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:32:40.179 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:32:40.422 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:32:40.423 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:32:40.423 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:32:40.423 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:40.423 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:32:40.424 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:32:40.424 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:40.424 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:32:40.424 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:32:40.424 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:32:40.425 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:32:40.425 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:32:40.425 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:32:40.425 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:32:40.425 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:32:41.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:40 gibba032 bash[24004]: cluster 2021-03-10T16:32:39.458331+0000 mgr.gibba032.whopeo (mgr.24625) 8438 : cluster [DBG] pgmap v4803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:41.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:40 gibba032 bash[24004]: audit 2021-03-10T16:32:40.424523+0000 mon.c (mon.1) 163 : audit [DBG] from='client.? 172.21.2.111:0/849423455' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:40 gibba011 bash[30791]: cluster 2021-03-10T16:32:39.458331+0000 mgr.gibba032.whopeo (mgr.24625) 8438 : cluster [DBG] pgmap v4803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:41.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:40 gibba011 bash[30791]: audit 2021-03-10T16:32:40.424523+0000 mon.c (mon.1) 163 : audit [DBG] from='client.? 172.21.2.111:0/849423455' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:40 gibba011 bash[31774]: cluster 2021-03-10T16:32:39.458331+0000 mgr.gibba032.whopeo (mgr.24625) 8438 : cluster [DBG] pgmap v4803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:40 gibba011 bash[31774]: audit 2021-03-10T16:32:40.424523+0000 mon.c (mon.1) 163 : audit [DBG] from='client.? 172.21.2.111:0/849423455' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:32:42.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:41 gibba032 bash[24004]: audit 2021-03-10T16:32:39.938856+0000 mgr.gibba032.whopeo (mgr.24625) 8439 : audit [DBG] from='client.59689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:42.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:41 gibba032 bash[24004]: audit 2021-03-10T16:32:40.162125+0000 mgr.gibba032.whopeo (mgr.24625) 8440 : audit [DBG] from='client.59695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:42.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:41 gibba011 bash[30791]: audit 2021-03-10T16:32:39.938856+0000 mgr.gibba032.whopeo (mgr.24625) 8439 : audit [DBG] from='client.59689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:42.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:41 gibba011 bash[30791]: audit 2021-03-10T16:32:40.162125+0000 mgr.gibba032.whopeo (mgr.24625) 8440 : audit [DBG] from='client.59695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:42.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:41 gibba011 bash[31774]: audit 2021-03-10T16:32:39.938856+0000 mgr.gibba032.whopeo (mgr.24625) 8439 : audit [DBG] from='client.59689 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:42.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:41 gibba011 bash[31774]: audit 2021-03-10T16:32:40.162125+0000 mgr.gibba032.whopeo (mgr.24625) 8440 : audit [DBG] from='client.59695 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:32:43.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:42 gibba032 bash[24004]: cluster 2021-03-10T16:32:41.460079+0000 mgr.gibba032.whopeo (mgr.24625) 8441 : cluster [DBG] pgmap v4804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:42 gibba011 bash[30791]: cluster 2021-03-10T16:32:41.460079+0000 mgr.gibba032.whopeo (mgr.24625) 8441 : cluster [DBG] pgmap v4804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:43.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:42 gibba011 bash[31774]: cluster 2021-03-10T16:32:41.460079+0000 mgr.gibba032.whopeo (mgr.24625) 8441 : cluster [DBG] pgmap v4804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:45.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:44 gibba032 bash[24004]: cluster 2021-03-10T16:32:43.461661+0000 mgr.gibba032.whopeo (mgr.24625) 8442 : cluster [DBG] pgmap v4805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:45.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:44 gibba011 bash[30791]: cluster 2021-03-10T16:32:43.461661+0000 mgr.gibba032.whopeo (mgr.24625) 8442 : cluster [DBG] pgmap v4805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:44 gibba011 bash[31774]: cluster 2021-03-10T16:32:43.461661+0000 mgr.gibba032.whopeo (mgr.24625) 8442 : cluster [DBG] pgmap v4805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:46.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:45 gibba032 bash[24004]: cephadm 2021-03-10T16:32:44.668602+0000 mgr.gibba032.whopeo (mgr.24625) 8443 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:46.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:45 gibba011 bash[30791]: cephadm 2021-03-10T16:32:44.668602+0000 mgr.gibba032.whopeo (mgr.24625) 8443 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:46.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:45 gibba011 bash[31774]: cephadm 2021-03-10T16:32:44.668602+0000 mgr.gibba032.whopeo (mgr.24625) 8443 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:32:47.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:46 gibba032 bash[24004]: cluster 2021-03-10T16:32:45.463109+0000 mgr.gibba032.whopeo (mgr.24625) 8444 : cluster [DBG] pgmap v4806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:47.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:46 gibba011 bash[31774]: cluster 2021-03-10T16:32:45.463109+0000 mgr.gibba032.whopeo (mgr.24625) 8444 : cluster [DBG] pgmap v4806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:46 gibba011 bash[30791]: cluster 2021-03-10T16:32:45.463109+0000 mgr.gibba032.whopeo (mgr.24625) 8444 : cluster [DBG] pgmap v4806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:49.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:48 gibba032 bash[24004]: cluster 2021-03-10T16:32:47.464882+0000 mgr.gibba032.whopeo (mgr.24625) 8445 : cluster [DBG] pgmap v4807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:49.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:48 gibba011 bash[30791]: cluster 2021-03-10T16:32:47.464882+0000 mgr.gibba032.whopeo (mgr.24625) 8445 : cluster [DBG] pgmap v4807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:48 gibba011 bash[31774]: cluster 2021-03-10T16:32:47.464882+0000 mgr.gibba032.whopeo (mgr.24625) 8445 : cluster [DBG] pgmap v4807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:51.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:50 gibba032 bash[24004]: cluster 2021-03-10T16:32:49.466238+0000 mgr.gibba032.whopeo (mgr.24625) 8446 : cluster [DBG] pgmap v4808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:51.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:50 gibba011 bash[30791]: cluster 2021-03-10T16:32:49.466238+0000 mgr.gibba032.whopeo (mgr.24625) 8446 : cluster [DBG] pgmap v4808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:50 gibba011 bash[31774]: cluster 2021-03-10T16:32:49.466238+0000 mgr.gibba032.whopeo (mgr.24625) 8446 : cluster [DBG] pgmap v4808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:53.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:52 gibba032 bash[24004]: cluster 2021-03-10T16:32:51.467859+0000 mgr.gibba032.whopeo (mgr.24625) 8447 : cluster [DBG] pgmap v4809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:53.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:52 gibba011 bash[30791]: cluster 2021-03-10T16:32:51.467859+0000 mgr.gibba032.whopeo (mgr.24625) 8447 : cluster [DBG] pgmap v4809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:52 gibba011 bash[31774]: cluster 2021-03-10T16:32:51.467859+0000 mgr.gibba032.whopeo (mgr.24625) 8447 : cluster [DBG] pgmap v4809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:55.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:54 gibba011 bash[30791]: cluster 2021-03-10T16:32:53.469470+0000 mgr.gibba032.whopeo (mgr.24625) 8448 : cluster [DBG] pgmap v4810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:54 gibba011 bash[31774]: cluster 2021-03-10T16:32:53.469470+0000 mgr.gibba032.whopeo (mgr.24625) 8448 : cluster [DBG] pgmap v4810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:54 gibba032 bash[24004]: cluster 2021-03-10T16:32:53.469470+0000 mgr.gibba032.whopeo (mgr.24625) 8448 : cluster [DBG] pgmap v4810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:57.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:56 gibba011 bash[30791]: cluster 2021-03-10T16:32:55.470801+0000 mgr.gibba032.whopeo (mgr.24625) 8449 : cluster [DBG] pgmap v4811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:56 gibba011 bash[31774]: cluster 2021-03-10T16:32:55.470801+0000 mgr.gibba032.whopeo (mgr.24625) 8449 : cluster [DBG] pgmap v4811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:56 gibba032 bash[24004]: cluster 2021-03-10T16:32:55.470801+0000 mgr.gibba032.whopeo (mgr.24625) 8449 : cluster [DBG] pgmap v4811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.073170+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:32:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.074432+0000 mon.a (mon.0) 2937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:32:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.168611+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:32:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.169397+0000 mon.a (mon.0) 2939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.170121+0000 mon.a (mon.0) 2940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[30791]: audit 2021-03-10T16:32:57.170915+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.073170+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:32:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.074432+0000 mon.a (mon.0) 2937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:32:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.168611+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:32:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.169397+0000 mon.a (mon.0) 2939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.170121+0000 mon.a (mon.0) 2940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:57 gibba011 bash[31774]: audit 2021-03-10T16:32:57.170915+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.073170+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:32:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.074432+0000 mon.a (mon.0) 2937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:32:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.168611+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:32:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.169397+0000 mon.a (mon.0) 2939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.170121+0000 mon.a (mon.0) 2940 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:32:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:57 gibba032 bash[24004]: audit 2021-03-10T16:32:57.170915+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:32:59.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:32:58 gibba011 bash[30791]: cluster 2021-03-10T16:32:57.472486+0000 mgr.gibba032.whopeo (mgr.24625) 8450 : cluster [DBG] pgmap v4812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:32:58 gibba011 bash[31774]: cluster 2021-03-10T16:32:57.472486+0000 mgr.gibba032.whopeo (mgr.24625) 8450 : cluster [DBG] pgmap v4812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:32:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:32:58 gibba032 bash[24004]: cluster 2021-03-10T16:32:57.472486+0000 mgr.gibba032.whopeo (mgr.24625) 8450 : cluster [DBG] pgmap v4812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:01.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:00 gibba011 bash[30791]: cluster 2021-03-10T16:32:59.473967+0000 mgr.gibba032.whopeo (mgr.24625) 8451 : cluster [DBG] pgmap v4813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:01.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:00 gibba011 bash[30791]: cephadm 2021-03-10T16:32:59.684100+0000 mgr.gibba032.whopeo (mgr.24625) 8452 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:00 gibba011 bash[31774]: cluster 2021-03-10T16:32:59.473967+0000 mgr.gibba032.whopeo (mgr.24625) 8451 : cluster [DBG] pgmap v4813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:00 gibba011 bash[31774]: cephadm 2021-03-10T16:32:59.684100+0000 mgr.gibba032.whopeo (mgr.24625) 8452 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:00 gibba032 bash[24004]: cluster 2021-03-10T16:32:59.473967+0000 mgr.gibba032.whopeo (mgr.24625) 8451 : cluster [DBG] pgmap v4813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:01.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:00 gibba032 bash[24004]: cephadm 2021-03-10T16:32:59.684100+0000 mgr.gibba032.whopeo (mgr.24625) 8452 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:03.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:02 gibba011 bash[30791]: cluster 2021-03-10T16:33:01.475681+0000 mgr.gibba032.whopeo (mgr.24625) 8453 : cluster [DBG] pgmap v4814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:03.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:02 gibba011 bash[31774]: cluster 2021-03-10T16:33:01.475681+0000 mgr.gibba032.whopeo (mgr.24625) 8453 : cluster [DBG] pgmap v4814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:02 gibba032 bash[24004]: cluster 2021-03-10T16:33:01.475681+0000 mgr.gibba032.whopeo (mgr.24625) 8453 : cluster [DBG] pgmap v4814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:05.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:04 gibba011 bash[30791]: cluster 2021-03-10T16:33:03.477273+0000 mgr.gibba032.whopeo (mgr.24625) 8454 : cluster [DBG] pgmap v4815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:05.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:04 gibba011 bash[31774]: cluster 2021-03-10T16:33:03.477273+0000 mgr.gibba032.whopeo (mgr.24625) 8454 : cluster [DBG] pgmap v4815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:04 gibba032 bash[24004]: cluster 2021-03-10T16:33:03.477273+0000 mgr.gibba032.whopeo (mgr.24625) 8454 : cluster [DBG] pgmap v4815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:07.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:06 gibba011 bash[30791]: cluster 2021-03-10T16:33:05.478808+0000 mgr.gibba032.whopeo (mgr.24625) 8455 : cluster [DBG] pgmap v4816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:06 gibba011 bash[31774]: cluster 2021-03-10T16:33:05.478808+0000 mgr.gibba032.whopeo (mgr.24625) 8455 : cluster [DBG] pgmap v4816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:06 gibba032 bash[24004]: cluster 2021-03-10T16:33:05.478808+0000 mgr.gibba032.whopeo (mgr.24625) 8455 : cluster [DBG] pgmap v4816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:09.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:08 gibba011 bash[30791]: cluster 2021-03-10T16:33:07.480367+0000 mgr.gibba032.whopeo (mgr.24625) 8456 : cluster [DBG] pgmap v4817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:08 gibba011 bash[31774]: cluster 2021-03-10T16:33:07.480367+0000 mgr.gibba032.whopeo (mgr.24625) 8456 : cluster [DBG] pgmap v4817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:08 gibba032 bash[24004]: cluster 2021-03-10T16:33:07.480367+0000 mgr.gibba032.whopeo (mgr.24625) 8456 : cluster [DBG] pgmap v4817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:10.663 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:33:10.890 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:33:10.890 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:33:10.890 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 101s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:33:10.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:33:10.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:33:10.891 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:33:10.891 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:33:10.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:33:10.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:33:10.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:33:10.892 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 101s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:33:10.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:33:10.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:33:10.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:33:10.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:33:10.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:33:10.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:33:10.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:33:10.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 101s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:33:10.894 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 101s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:33:10.895 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:33:11.141 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:33:11.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:33:11.141 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:33:11.142 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:11.142 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:33:11.142 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:33:11.142 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:11.143 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:33:11.143 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:33:11.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:11.143 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:33:11.143 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:33:11.144 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:33:11.144 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:33:11.144 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:33:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:10 gibba032 bash[24004]: cluster 2021-03-10T16:33:09.481858+0000 mgr.gibba032.whopeo (mgr.24625) 8457 : cluster [DBG] pgmap v4818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:10 gibba011 bash[30791]: cluster 2021-03-10T16:33:09.481858+0000 mgr.gibba032.whopeo (mgr.24625) 8457 : cluster [DBG] pgmap v4818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:10 gibba011 bash[31774]: cluster 2021-03-10T16:33:09.481858+0000 mgr.gibba032.whopeo (mgr.24625) 8457 : cluster [DBG] pgmap v4818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:12.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:11 gibba032 bash[24004]: audit 2021-03-10T16:33:10.655608+0000 mgr.gibba032.whopeo (mgr.24625) 8458 : audit [DBG] from='client.49725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:12.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:11 gibba032 bash[24004]: audit 2021-03-10T16:33:11.143483+0000 mon.c (mon.1) 164 : audit [DBG] from='client.? 172.21.2.111:0/2853429869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:11 gibba011 bash[30791]: audit 2021-03-10T16:33:10.655608+0000 mgr.gibba032.whopeo (mgr.24625) 8458 : audit [DBG] from='client.49725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:12.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:11 gibba011 bash[30791]: audit 2021-03-10T16:33:11.143483+0000 mon.c (mon.1) 164 : audit [DBG] from='client.? 172.21.2.111:0/2853429869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:12.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:11 gibba011 bash[31774]: audit 2021-03-10T16:33:10.655608+0000 mgr.gibba032.whopeo (mgr.24625) 8458 : audit [DBG] from='client.49725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:12.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:11 gibba011 bash[31774]: audit 2021-03-10T16:33:11.143483+0000 mon.c (mon.1) 164 : audit [DBG] from='client.? 172.21.2.111:0/2853429869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:12 gibba032 bash[24004]: audit 2021-03-10T16:33:10.879428+0000 mgr.gibba032.whopeo (mgr.24625) 8459 : audit [DBG] from='client.49731 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:12 gibba032 bash[24004]: cluster 2021-03-10T16:33:11.483570+0000 mgr.gibba032.whopeo (mgr.24625) 8460 : cluster [DBG] pgmap v4819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:12 gibba011 bash[30791]: audit 2021-03-10T16:33:10.879428+0000 mgr.gibba032.whopeo (mgr.24625) 8459 : audit [DBG] from='client.49731 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:13.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:12 gibba011 bash[30791]: cluster 2021-03-10T16:33:11.483570+0000 mgr.gibba032.whopeo (mgr.24625) 8460 : cluster [DBG] pgmap v4819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:12 gibba011 bash[31774]: audit 2021-03-10T16:33:10.879428+0000 mgr.gibba032.whopeo (mgr.24625) 8459 : audit [DBG] from='client.49731 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:12 gibba011 bash[31774]: cluster 2021-03-10T16:33:11.483570+0000 mgr.gibba032.whopeo (mgr.24625) 8460 : cluster [DBG] pgmap v4819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:14 gibba032 bash[24004]: cluster 2021-03-10T16:33:13.484205+0000 mgr.gibba032.whopeo (mgr.24625) 8461 : cluster [DBG] pgmap v4820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:14 gibba011 bash[30791]: cluster 2021-03-10T16:33:13.484205+0000 mgr.gibba032.whopeo (mgr.24625) 8461 : cluster [DBG] pgmap v4820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:14 gibba011 bash[31774]: cluster 2021-03-10T16:33:13.484205+0000 mgr.gibba032.whopeo (mgr.24625) 8461 : cluster [DBG] pgmap v4820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:16.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:15 gibba032 bash[24004]: cephadm 2021-03-10T16:33:14.696407+0000 mgr.gibba032.whopeo (mgr.24625) 8462 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:16.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:15 gibba011 bash[30791]: cephadm 2021-03-10T16:33:14.696407+0000 mgr.gibba032.whopeo (mgr.24625) 8462 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:16.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:15 gibba011 bash[31774]: cephadm 2021-03-10T16:33:14.696407+0000 mgr.gibba032.whopeo (mgr.24625) 8462 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:16 gibba032 bash[24004]: cluster 2021-03-10T16:33:15.485470+0000 mgr.gibba032.whopeo (mgr.24625) 8463 : cluster [DBG] pgmap v4821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:16 gibba011 bash[30791]: cluster 2021-03-10T16:33:15.485470+0000 mgr.gibba032.whopeo (mgr.24625) 8463 : cluster [DBG] pgmap v4821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:16 gibba011 bash[31774]: cluster 2021-03-10T16:33:15.485470+0000 mgr.gibba032.whopeo (mgr.24625) 8463 : cluster [DBG] pgmap v4821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:18.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:17 gibba032 bash[24004]: cluster 2021-03-10T16:33:17.487177+0000 mgr.gibba032.whopeo (mgr.24625) 8464 : cluster [DBG] pgmap v4822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:18.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:17 gibba011 bash[30791]: cluster 2021-03-10T16:33:17.487177+0000 mgr.gibba032.whopeo (mgr.24625) 8464 : cluster [DBG] pgmap v4822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:18.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:17 gibba011 bash[31774]: cluster 2021-03-10T16:33:17.487177+0000 mgr.gibba032.whopeo (mgr.24625) 8464 : cluster [DBG] pgmap v4822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:21.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:20 gibba032 bash[24004]: cluster 2021-03-10T16:33:19.487951+0000 mgr.gibba032.whopeo (mgr.24625) 8465 : cluster [DBG] pgmap v4823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:21.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:20 gibba011 bash[30791]: cluster 2021-03-10T16:33:19.487951+0000 mgr.gibba032.whopeo (mgr.24625) 8465 : cluster [DBG] pgmap v4823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:20 gibba011 bash[31774]: cluster 2021-03-10T16:33:19.487951+0000 mgr.gibba032.whopeo (mgr.24625) 8465 : cluster [DBG] pgmap v4823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:23.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:22 gibba032 bash[24004]: cluster 2021-03-10T16:33:21.489619+0000 mgr.gibba032.whopeo (mgr.24625) 8466 : cluster [DBG] pgmap v4824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:22 gibba011 bash[30791]: cluster 2021-03-10T16:33:21.489619+0000 mgr.gibba032.whopeo (mgr.24625) 8466 : cluster [DBG] pgmap v4824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:23.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:22 gibba011 bash[31774]: cluster 2021-03-10T16:33:21.489619+0000 mgr.gibba032.whopeo (mgr.24625) 8466 : cluster [DBG] pgmap v4824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:25.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:24 gibba032 bash[24004]: cluster 2021-03-10T16:33:23.491188+0000 mgr.gibba032.whopeo (mgr.24625) 8467 : cluster [DBG] pgmap v4825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:25.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:24 gibba011 bash[30791]: cluster 2021-03-10T16:33:23.491188+0000 mgr.gibba032.whopeo (mgr.24625) 8467 : cluster [DBG] pgmap v4825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:24 gibba011 bash[31774]: cluster 2021-03-10T16:33:23.491188+0000 mgr.gibba032.whopeo (mgr.24625) 8467 : cluster [DBG] pgmap v4825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:27.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:26 gibba032 bash[24004]: cluster 2021-03-10T16:33:25.491971+0000 mgr.gibba032.whopeo (mgr.24625) 8468 : cluster [DBG] pgmap v4826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:27.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:26 gibba011 bash[30791]: cluster 2021-03-10T16:33:25.491971+0000 mgr.gibba032.whopeo (mgr.24625) 8468 : cluster [DBG] pgmap v4826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:27.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:26 gibba011 bash[31774]: cluster 2021-03-10T16:33:25.491971+0000 mgr.gibba032.whopeo (mgr.24625) 8468 : cluster [DBG] pgmap v4826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:29.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:28 gibba032 bash[24004]: cluster 2021-03-10T16:33:27.493604+0000 mgr.gibba032.whopeo (mgr.24625) 8469 : cluster [DBG] pgmap v4827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:29.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:28 gibba011 bash[30791]: cluster 2021-03-10T16:33:27.493604+0000 mgr.gibba032.whopeo (mgr.24625) 8469 : cluster [DBG] pgmap v4827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:29.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:28 gibba011 bash[31774]: cluster 2021-03-10T16:33:27.493604+0000 mgr.gibba032.whopeo (mgr.24625) 8469 : cluster [DBG] pgmap v4827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: cluster 2021-03-10T16:33:29.495138+0000 mgr.gibba032.whopeo (mgr.24625) 8470 : cluster [DBG] pgmap v4828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.712257+0000 mgr.gibba032.whopeo (mgr.24625) 8471 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:33:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.712823+0000 mgr.gibba032.whopeo (mgr.24625) 8472 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:33:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.905931+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:33:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.908965+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:33:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.910335+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.911601+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.912709+0000 mon.a (mon.0) 2946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.915439+0000 mon.a (mon.0) 2947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:30 gibba032 bash[24004]: audit 2021-03-10T16:33:29.917031+0000 mon.a (mon.0) 2948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: cluster 2021-03-10T16:33:29.495138+0000 mgr.gibba032.whopeo (mgr.24625) 8470 : cluster [DBG] pgmap v4828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.712257+0000 mgr.gibba032.whopeo (mgr.24625) 8471 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:33:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.712823+0000 mgr.gibba032.whopeo (mgr.24625) 8472 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:33:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.905931+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:33:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.908965+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:33:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.910335+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.911601+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.912709+0000 mon.a (mon.0) 2946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.915439+0000 mon.a (mon.0) 2947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[30791]: audit 2021-03-10T16:33:29.917031+0000 mon.a (mon.0) 2948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: cluster 2021-03-10T16:33:29.495138+0000 mgr.gibba032.whopeo (mgr.24625) 8470 : cluster [DBG] pgmap v4828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.712257+0000 mgr.gibba032.whopeo (mgr.24625) 8471 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:33:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.712823+0000 mgr.gibba032.whopeo (mgr.24625) 8472 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:33:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.905931+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:33:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.908965+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:33:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.910335+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.911601+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.912709+0000 mon.a (mon.0) 2946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.915439+0000 mon.a (mon.0) 2947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:30 gibba011 bash[31774]: audit 2021-03-10T16:33:29.917031+0000 mon.a (mon.0) 2948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.906239+0000 mgr.gibba032.whopeo (mgr.24625) 8473 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:33:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.907849+0000 mgr.gibba032.whopeo (mgr.24625) 8474 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:33:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.908932+0000 mgr.gibba032.whopeo (mgr.24625) 8475 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:33:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.909007+0000 mgr.gibba032.whopeo (mgr.24625) 8476 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:33:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.910237+0000 mgr.gibba032.whopeo (mgr.24625) 8477 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:33:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.910272+0000 mgr.gibba032.whopeo (mgr.24625) 8478 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:33:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.911428+0000 mgr.gibba032.whopeo (mgr.24625) 8479 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:33:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.912531+0000 mgr.gibba032.whopeo (mgr.24625) 8480 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:33:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.912658+0000 mgr.gibba032.whopeo (mgr.24625) 8481 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:33:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.914122+0000 mgr.gibba032.whopeo (mgr.24625) 8482 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:33:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.914508+0000 mgr.gibba032.whopeo (mgr.24625) 8483 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:33:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.915720+0000 mgr.gibba032.whopeo (mgr.24625) 8484 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:33:32.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.915812+0000 mgr.gibba032.whopeo (mgr.24625) 8485 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:33:32.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:31 gibba032 bash[24004]: cephadm 2021-03-10T16:33:29.915996+0000 mgr.gibba032.whopeo (mgr.24625) 8486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:32.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.906239+0000 mgr.gibba032.whopeo (mgr.24625) 8473 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:33:32.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.907849+0000 mgr.gibba032.whopeo (mgr.24625) 8474 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:33:32.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.908932+0000 mgr.gibba032.whopeo (mgr.24625) 8475 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:33:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.909007+0000 mgr.gibba032.whopeo (mgr.24625) 8476 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:33:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.910237+0000 mgr.gibba032.whopeo (mgr.24625) 8477 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:33:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.910272+0000 mgr.gibba032.whopeo (mgr.24625) 8478 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:33:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.911428+0000 mgr.gibba032.whopeo (mgr.24625) 8479 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:33:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.912531+0000 mgr.gibba032.whopeo (mgr.24625) 8480 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:33:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.912658+0000 mgr.gibba032.whopeo (mgr.24625) 8481 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:33:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.914122+0000 mgr.gibba032.whopeo (mgr.24625) 8482 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:33:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.914508+0000 mgr.gibba032.whopeo (mgr.24625) 8483 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:33:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.915720+0000 mgr.gibba032.whopeo (mgr.24625) 8484 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:33:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.915812+0000 mgr.gibba032.whopeo (mgr.24625) 8485 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:33:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[30791]: cephadm 2021-03-10T16:33:29.915996+0000 mgr.gibba032.whopeo (mgr.24625) 8486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.906239+0000 mgr.gibba032.whopeo (mgr.24625) 8473 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:33:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.907849+0000 mgr.gibba032.whopeo (mgr.24625) 8474 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:33:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.908932+0000 mgr.gibba032.whopeo (mgr.24625) 8475 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:33:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.909007+0000 mgr.gibba032.whopeo (mgr.24625) 8476 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:33:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.910237+0000 mgr.gibba032.whopeo (mgr.24625) 8477 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:33:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.910272+0000 mgr.gibba032.whopeo (mgr.24625) 8478 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:33:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.911428+0000 mgr.gibba032.whopeo (mgr.24625) 8479 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:33:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.912531+0000 mgr.gibba032.whopeo (mgr.24625) 8480 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:33:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.912658+0000 mgr.gibba032.whopeo (mgr.24625) 8481 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:33:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.914122+0000 mgr.gibba032.whopeo (mgr.24625) 8482 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:33:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.914508+0000 mgr.gibba032.whopeo (mgr.24625) 8483 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:33:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.915720+0000 mgr.gibba032.whopeo (mgr.24625) 8484 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:33:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.915812+0000 mgr.gibba032.whopeo (mgr.24625) 8485 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:33:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:31 gibba011 bash[31774]: cephadm 2021-03-10T16:33:29.915996+0000 mgr.gibba032.whopeo (mgr.24625) 8486 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:32 gibba032 bash[24004]: cluster 2021-03-10T16:33:31.495923+0000 mgr.gibba032.whopeo (mgr.24625) 8487 : cluster [DBG] pgmap v4829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:33.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:32 gibba011 bash[30791]: cluster 2021-03-10T16:33:31.495923+0000 mgr.gibba032.whopeo (mgr.24625) 8487 : cluster [DBG] pgmap v4829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:33.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:32 gibba011 bash[31774]: cluster 2021-03-10T16:33:31.495923+0000 mgr.gibba032.whopeo (mgr.24625) 8487 : cluster [DBG] pgmap v4829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:35.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:34 gibba032 bash[24004]: cluster 2021-03-10T16:33:33.497399+0000 mgr.gibba032.whopeo (mgr.24625) 8488 : cluster [DBG] pgmap v4830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:35.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:34 gibba011 bash[30791]: cluster 2021-03-10T16:33:33.497399+0000 mgr.gibba032.whopeo (mgr.24625) 8488 : cluster [DBG] pgmap v4830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:34 gibba011 bash[31774]: cluster 2021-03-10T16:33:33.497399+0000 mgr.gibba032.whopeo (mgr.24625) 8488 : cluster [DBG] pgmap v4830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:37.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:36 gibba032 bash[24004]: cluster 2021-03-10T16:33:35.498886+0000 mgr.gibba032.whopeo (mgr.24625) 8489 : cluster [DBG] pgmap v4831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:37.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:36 gibba011 bash[30791]: cluster 2021-03-10T16:33:35.498886+0000 mgr.gibba032.whopeo (mgr.24625) 8489 : cluster [DBG] pgmap v4831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:37.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:36 gibba011 bash[31774]: cluster 2021-03-10T16:33:35.498886+0000 mgr.gibba032.whopeo (mgr.24625) 8489 : cluster [DBG] pgmap v4831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:39.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:38 gibba032 bash[24004]: cluster 2021-03-10T16:33:37.499745+0000 mgr.gibba032.whopeo (mgr.24625) 8490 : cluster [DBG] pgmap v4832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:39.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:38 gibba011 bash[30791]: cluster 2021-03-10T16:33:37.499745+0000 mgr.gibba032.whopeo (mgr.24625) 8490 : cluster [DBG] pgmap v4832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:38 gibba011 bash[31774]: cluster 2021-03-10T16:33:37.499745+0000 mgr.gibba032.whopeo (mgr.24625) 8490 : cluster [DBG] pgmap v4832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:41.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:40 gibba032 bash[24004]: cluster 2021-03-10T16:33:39.501149+0000 mgr.gibba032.whopeo (mgr.24625) 8491 : cluster [DBG] pgmap v4833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:40 gibba011 bash[30791]: cluster 2021-03-10T16:33:39.501149+0000 mgr.gibba032.whopeo (mgr.24625) 8491 : cluster [DBG] pgmap v4833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:41.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:40 gibba011 bash[31774]: cluster 2021-03-10T16:33:39.501149+0000 mgr.gibba032.whopeo (mgr.24625) 8491 : cluster [DBG] pgmap v4833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:41.383 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:33:41.611 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:33:41.611 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:33:41.612 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:33:41.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:33:41.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:33:41.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:33:41.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:33:41.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:33:41.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:33:41.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:33:41.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:33:41.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:33:41.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:33:41.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:33:41.615 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:33:41.615 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:33:41.860 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:33:41.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:33:41.861 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:33:41.861 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:41.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:33:41.861 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:33:41.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:41.862 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:33:41.862 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:33:41.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:33:41.862 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:33:41.863 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:33:41.863 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:33:41.863 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:33:41.863 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:33:43.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:42 gibba032 bash[24004]: audit 2021-03-10T16:33:41.374604+0000 mgr.gibba032.whopeo (mgr.24625) 8492 : audit [DBG] from='client.59725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:42 gibba032 bash[24004]: cluster 2021-03-10T16:33:41.502796+0000 mgr.gibba032.whopeo (mgr.24625) 8493 : cluster [DBG] pgmap v4834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:42 gibba032 bash[24004]: audit 2021-03-10T16:33:41.599853+0000 mgr.gibba032.whopeo (mgr.24625) 8494 : audit [DBG] from='client.49749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:42 gibba032 bash[24004]: audit 2021-03-10T16:33:41.862860+0000 mon.a (mon.0) 2949 : audit [DBG] from='client.? 172.21.2.111:0/2859298195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[30791]: audit 2021-03-10T16:33:41.374604+0000 mgr.gibba032.whopeo (mgr.24625) 8492 : audit [DBG] from='client.59725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[30791]: cluster 2021-03-10T16:33:41.502796+0000 mgr.gibba032.whopeo (mgr.24625) 8493 : cluster [DBG] pgmap v4834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:43.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[30791]: audit 2021-03-10T16:33:41.599853+0000 mgr.gibba032.whopeo (mgr.24625) 8494 : audit [DBG] from='client.49749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[30791]: audit 2021-03-10T16:33:41.862860+0000 mon.a (mon.0) 2949 : audit [DBG] from='client.? 172.21.2.111:0/2859298195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[31774]: audit 2021-03-10T16:33:41.374604+0000 mgr.gibba032.whopeo (mgr.24625) 8492 : audit [DBG] from='client.59725 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[31774]: cluster 2021-03-10T16:33:41.502796+0000 mgr.gibba032.whopeo (mgr.24625) 8493 : cluster [DBG] pgmap v4834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:43.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[31774]: audit 2021-03-10T16:33:41.599853+0000 mgr.gibba032.whopeo (mgr.24625) 8494 : audit [DBG] from='client.49749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:33:43.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:42 gibba011 bash[31774]: audit 2021-03-10T16:33:41.862860+0000 mon.a (mon.0) 2949 : audit [DBG] from='client.? 172.21.2.111:0/2859298195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:33:45.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:44 gibba032 bash[24004]: cluster 2021-03-10T16:33:43.503532+0000 mgr.gibba032.whopeo (mgr.24625) 8495 : cluster [DBG] pgmap v4835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:45.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:44 gibba011 bash[30791]: cluster 2021-03-10T16:33:43.503532+0000 mgr.gibba032.whopeo (mgr.24625) 8495 : cluster [DBG] pgmap v4835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:44 gibba011 bash[31774]: cluster 2021-03-10T16:33:43.503532+0000 mgr.gibba032.whopeo (mgr.24625) 8495 : cluster [DBG] pgmap v4835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:46 gibba011 bash[30791]: cephadm 2021-03-10T16:33:44.931208+0000 mgr.gibba032.whopeo (mgr.24625) 8496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:46 gibba011 bash[30791]: cluster 2021-03-10T16:33:45.504927+0000 mgr.gibba032.whopeo (mgr.24625) 8497 : cluster [DBG] pgmap v4836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:46 gibba011 bash[31774]: cephadm 2021-03-10T16:33:44.931208+0000 mgr.gibba032.whopeo (mgr.24625) 8496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:47.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:46 gibba011 bash[31774]: cluster 2021-03-10T16:33:45.504927+0000 mgr.gibba032.whopeo (mgr.24625) 8497 : cluster [DBG] pgmap v4836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:46 gibba032 bash[24004]: cephadm 2021-03-10T16:33:44.931208+0000 mgr.gibba032.whopeo (mgr.24625) 8496 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:33:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:46 gibba032 bash[24004]: cluster 2021-03-10T16:33:45.504927+0000 mgr.gibba032.whopeo (mgr.24625) 8497 : cluster [DBG] pgmap v4836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:49.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:48 gibba011 bash[30791]: cluster 2021-03-10T16:33:47.506714+0000 mgr.gibba032.whopeo (mgr.24625) 8498 : cluster [DBG] pgmap v4837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:48 gibba011 bash[31774]: cluster 2021-03-10T16:33:47.506714+0000 mgr.gibba032.whopeo (mgr.24625) 8498 : cluster [DBG] pgmap v4837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:48 gibba032 bash[24004]: cluster 2021-03-10T16:33:47.506714+0000 mgr.gibba032.whopeo (mgr.24625) 8498 : cluster [DBG] pgmap v4837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:51.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:50 gibba011 bash[30791]: cluster 2021-03-10T16:33:49.507406+0000 mgr.gibba032.whopeo (mgr.24625) 8499 : cluster [DBG] pgmap v4838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:51.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:50 gibba011 bash[31774]: cluster 2021-03-10T16:33:49.507406+0000 mgr.gibba032.whopeo (mgr.24625) 8499 : cluster [DBG] pgmap v4838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:50 gibba032 bash[24004]: cluster 2021-03-10T16:33:49.507406+0000 mgr.gibba032.whopeo (mgr.24625) 8499 : cluster [DBG] pgmap v4838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:53.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:52 gibba011 bash[30791]: cluster 2021-03-10T16:33:51.508832+0000 mgr.gibba032.whopeo (mgr.24625) 8500 : cluster [DBG] pgmap v4839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:53.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:52 gibba011 bash[31774]: cluster 2021-03-10T16:33:51.508832+0000 mgr.gibba032.whopeo (mgr.24625) 8500 : cluster [DBG] pgmap v4839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:52 gibba032 bash[24004]: cluster 2021-03-10T16:33:51.508832+0000 mgr.gibba032.whopeo (mgr.24625) 8500 : cluster [DBG] pgmap v4839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:55.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:54 gibba011 bash[30791]: cluster 2021-03-10T16:33:53.510297+0000 mgr.gibba032.whopeo (mgr.24625) 8501 : cluster [DBG] pgmap v4840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:54 gibba011 bash[31774]: cluster 2021-03-10T16:33:53.510297+0000 mgr.gibba032.whopeo (mgr.24625) 8501 : cluster [DBG] pgmap v4840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:54 gibba032 bash[24004]: cluster 2021-03-10T16:33:53.510297+0000 mgr.gibba032.whopeo (mgr.24625) 8501 : cluster [DBG] pgmap v4840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:57.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:56 gibba011 bash[30791]: cluster 2021-03-10T16:33:55.511036+0000 mgr.gibba032.whopeo (mgr.24625) 8502 : cluster [DBG] pgmap v4841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:56 gibba011 bash[31774]: cluster 2021-03-10T16:33:55.511036+0000 mgr.gibba032.whopeo (mgr.24625) 8502 : cluster [DBG] pgmap v4841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:56 gibba032 bash[24004]: cluster 2021-03-10T16:33:55.511036+0000 mgr.gibba032.whopeo (mgr.24625) 8502 : cluster [DBG] pgmap v4841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.079595+0000 mon.a (mon.0) 2950 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:33:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.080266+0000 mon.a (mon.0) 2951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:33:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.180747+0000 mon.a (mon.0) 2952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T16:33:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.181480+0000 mon.a (mon.0) 2953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:33:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.182125+0000 mon.a (mon.0) 2954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:33:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[30791]: audit 2021-03-10T16:33:57.182803+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:33:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.079595+0000 mon.a (mon.0) 2950 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:33:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.080266+0000 mon.a (mon.0) 2951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:33:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.180747+0000 mon.a (mon.0) 2952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T16:33:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.181480+0000 mon.a (mon.0) 2953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:33:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.182125+0000 mon.a (mon.0) 2954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:33:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:57 gibba011 bash[31774]: audit 2021-03-10T16:33:57.182803+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:33:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.079595+0000 mon.a (mon.0) 2950 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:33:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.080266+0000 mon.a (mon.0) 2951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:33:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.180747+0000 mon.a (mon.0) 2952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T16:33:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.181480+0000 mon.a (mon.0) 2953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:33:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.182125+0000 mon.a (mon.0) 2954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:33:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:57 gibba032 bash[24004]: audit 2021-03-10T16:33:57.182803+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:33:59.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:33:58 gibba011 bash[30791]: cluster 2021-03-10T16:33:57.512600+0000 mgr.gibba032.whopeo (mgr.24625) 8503 : cluster [DBG] pgmap v4842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:33:58 gibba011 bash[31774]: cluster 2021-03-10T16:33:57.512600+0000 mgr.gibba032.whopeo (mgr.24625) 8503 : cluster [DBG] pgmap v4842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:33:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:33:58 gibba032 bash[24004]: cluster 2021-03-10T16:33:57.512600+0000 mgr.gibba032.whopeo (mgr.24625) 8503 : cluster [DBG] pgmap v4842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:00 gibba032 bash[24004]: cluster 2021-03-10T16:33:59.513921+0000 mgr.gibba032.whopeo (mgr.24625) 8504 : cluster [DBG] pgmap v4843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:01.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:00 gibba011 bash[30791]: cluster 2021-03-10T16:33:59.513921+0000 mgr.gibba032.whopeo (mgr.24625) 8504 : cluster [DBG] pgmap v4843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:00 gibba011 bash[31774]: cluster 2021-03-10T16:33:59.513921+0000 mgr.gibba032.whopeo (mgr.24625) 8504 : cluster [DBG] pgmap v4843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:02.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:01 gibba032 bash[24004]: cephadm 2021-03-10T16:33:59.947353+0000 mgr.gibba032.whopeo (mgr.24625) 8505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:02.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:01 gibba011 bash[30791]: cephadm 2021-03-10T16:33:59.947353+0000 mgr.gibba032.whopeo (mgr.24625) 8505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:02.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:01 gibba011 bash[31774]: cephadm 2021-03-10T16:33:59.947353+0000 mgr.gibba032.whopeo (mgr.24625) 8505 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:02 gibba032 bash[24004]: cluster 2021-03-10T16:34:01.514682+0000 mgr.gibba032.whopeo (mgr.24625) 8506 : cluster [DBG] pgmap v4844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:02 gibba011 bash[30791]: cluster 2021-03-10T16:34:01.514682+0000 mgr.gibba032.whopeo (mgr.24625) 8506 : cluster [DBG] pgmap v4844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:02 gibba011 bash[31774]: cluster 2021-03-10T16:34:01.514682+0000 mgr.gibba032.whopeo (mgr.24625) 8506 : cluster [DBG] pgmap v4844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:04 gibba032 bash[24004]: cluster 2021-03-10T16:34:03.516147+0000 mgr.gibba032.whopeo (mgr.24625) 8507 : cluster [DBG] pgmap v4845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:04 gibba011 bash[30791]: cluster 2021-03-10T16:34:03.516147+0000 mgr.gibba032.whopeo (mgr.24625) 8507 : cluster [DBG] pgmap v4845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:04 gibba011 bash[31774]: cluster 2021-03-10T16:34:03.516147+0000 mgr.gibba032.whopeo (mgr.24625) 8507 : cluster [DBG] pgmap v4845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:06 gibba032 bash[24004]: cluster 2021-03-10T16:34:05.517717+0000 mgr.gibba032.whopeo (mgr.24625) 8508 : cluster [DBG] pgmap v4846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:07.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:06 gibba011 bash[30791]: cluster 2021-03-10T16:34:05.517717+0000 mgr.gibba032.whopeo (mgr.24625) 8508 : cluster [DBG] pgmap v4846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:06 gibba011 bash[31774]: cluster 2021-03-10T16:34:05.517717+0000 mgr.gibba032.whopeo (mgr.24625) 8508 : cluster [DBG] pgmap v4846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:08 gibba032 bash[24004]: cluster 2021-03-10T16:34:07.518319+0000 mgr.gibba032.whopeo (mgr.24625) 8509 : cluster [DBG] pgmap v4847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:08 gibba011 bash[30791]: cluster 2021-03-10T16:34:07.518319+0000 mgr.gibba032.whopeo (mgr.24625) 8509 : cluster [DBG] pgmap v4847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:08 gibba011 bash[31774]: cluster 2021-03-10T16:34:07.518319+0000 mgr.gibba032.whopeo (mgr.24625) 8509 : cluster [DBG] pgmap v4847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:10 gibba032 bash[24004]: cluster 2021-03-10T16:34:09.519700+0000 mgr.gibba032.whopeo (mgr.24625) 8510 : cluster [DBG] pgmap v4848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:10 gibba011 bash[30791]: cluster 2021-03-10T16:34:09.519700+0000 mgr.gibba032.whopeo (mgr.24625) 8510 : cluster [DBG] pgmap v4848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:11.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:10 gibba011 bash[31774]: cluster 2021-03-10T16:34:09.519700+0000 mgr.gibba032.whopeo (mgr.24625) 8510 : cluster [DBG] pgmap v4848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:12.106 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:34:12.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:11 gibba032 bash[24004]: cluster 2021-03-10T16:34:11.521368+0000 mgr.gibba032.whopeo (mgr.24625) 8511 : cluster [DBG] pgmap v4849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:11 gibba011 bash[30791]: cluster 2021-03-10T16:34:11.521368+0000 mgr.gibba032.whopeo (mgr.24625) 8511 : cluster [DBG] pgmap v4849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:12.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:11 gibba011 bash[31774]: cluster 2021-03-10T16:34:11.521368+0000 mgr.gibba032.whopeo (mgr.24625) 8511 : cluster [DBG] pgmap v4849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:12.333 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:34:12.333 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:34:12.333 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:34:12.334 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:34:12.334 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:34:12.334 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:34:12.334 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:34:12.335 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:34:12.336 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:34:12.336 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:34:12.336 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:34:12.336 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:34:12.336 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:34:12.337 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:34:12.337 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:34:12.337 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:34:12.578 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:34:12.579 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:34:12.579 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:34:12.579 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:12.579 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:34:12.579 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:34:12.580 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:12.580 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:34:12.580 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:34:12.580 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:12.580 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:34:12.581 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:34:12.581 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:34:12.581 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:34:12.581 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:34:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:12 gibba032 bash[24004]: audit 2021-03-10T16:34:12.097320+0000 mgr.gibba032.whopeo (mgr.24625) 8512 : audit [DBG] from='client.49761 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:12 gibba032 bash[24004]: audit 2021-03-10T16:34:12.322220+0000 mgr.gibba032.whopeo (mgr.24625) 8513 : audit [DBG] from='client.59749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:12 gibba032 bash[24004]: audit 2021-03-10T16:34:12.581146+0000 mon.c (mon.1) 165 : audit [DBG] from='client.? 172.21.2.111:0/1840068492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[30791]: audit 2021-03-10T16:34:12.097320+0000 mgr.gibba032.whopeo (mgr.24625) 8512 : audit [DBG] from='client.49761 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[30791]: audit 2021-03-10T16:34:12.322220+0000 mgr.gibba032.whopeo (mgr.24625) 8513 : audit [DBG] from='client.59749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[30791]: audit 2021-03-10T16:34:12.581146+0000 mon.c (mon.1) 165 : audit [DBG] from='client.? 172.21.2.111:0/1840068492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[31774]: audit 2021-03-10T16:34:12.097320+0000 mgr.gibba032.whopeo (mgr.24625) 8512 : audit [DBG] from='client.49761 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[31774]: audit 2021-03-10T16:34:12.322220+0000 mgr.gibba032.whopeo (mgr.24625) 8513 : audit [DBG] from='client.59749 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:12 gibba011 bash[31774]: audit 2021-03-10T16:34:12.581146+0000 mon.c (mon.1) 165 : audit [DBG] from='client.? 172.21.2.111:0/1840068492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:14.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:13 gibba032 bash[24004]: cluster 2021-03-10T16:34:13.522299+0000 mgr.gibba032.whopeo (mgr.24625) 8514 : cluster [DBG] pgmap v4850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:14.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:13 gibba011 bash[30791]: cluster 2021-03-10T16:34:13.522299+0000 mgr.gibba032.whopeo (mgr.24625) 8514 : cluster [DBG] pgmap v4850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:14.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:13 gibba011 bash[31774]: cluster 2021-03-10T16:34:13.522299+0000 mgr.gibba032.whopeo (mgr.24625) 8514 : cluster [DBG] pgmap v4850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:17.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:16 gibba032 bash[24004]: cephadm 2021-03-10T16:34:14.963314+0000 mgr.gibba032.whopeo (mgr.24625) 8515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:16 gibba032 bash[24004]: cluster 2021-03-10T16:34:15.523931+0000 mgr.gibba032.whopeo (mgr.24625) 8516 : cluster [DBG] pgmap v4851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:16 gibba011 bash[30791]: cephadm 2021-03-10T16:34:14.963314+0000 mgr.gibba032.whopeo (mgr.24625) 8515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:16 gibba011 bash[30791]: cluster 2021-03-10T16:34:15.523931+0000 mgr.gibba032.whopeo (mgr.24625) 8516 : cluster [DBG] pgmap v4851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:16 gibba011 bash[31774]: cephadm 2021-03-10T16:34:14.963314+0000 mgr.gibba032.whopeo (mgr.24625) 8515 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:17.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:16 gibba011 bash[31774]: cluster 2021-03-10T16:34:15.523931+0000 mgr.gibba032.whopeo (mgr.24625) 8516 : cluster [DBG] pgmap v4851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:19.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:18 gibba032 bash[24004]: cluster 2021-03-10T16:34:17.525734+0000 mgr.gibba032.whopeo (mgr.24625) 8517 : cluster [DBG] pgmap v4852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:19.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:18 gibba011 bash[30791]: cluster 2021-03-10T16:34:17.525734+0000 mgr.gibba032.whopeo (mgr.24625) 8517 : cluster [DBG] pgmap v4852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:19.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:18 gibba011 bash[31774]: cluster 2021-03-10T16:34:17.525734+0000 mgr.gibba032.whopeo (mgr.24625) 8517 : cluster [DBG] pgmap v4852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:21.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:20 gibba032 bash[24004]: cluster 2021-03-10T16:34:19.526499+0000 mgr.gibba032.whopeo (mgr.24625) 8518 : cluster [DBG] pgmap v4853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:21.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:20 gibba011 bash[30791]: cluster 2021-03-10T16:34:19.526499+0000 mgr.gibba032.whopeo (mgr.24625) 8518 : cluster [DBG] pgmap v4853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:21.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:20 gibba011 bash[31774]: cluster 2021-03-10T16:34:19.526499+0000 mgr.gibba032.whopeo (mgr.24625) 8518 : cluster [DBG] pgmap v4853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:23.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:22 gibba032 bash[24004]: cluster 2021-03-10T16:34:21.528174+0000 mgr.gibba032.whopeo (mgr.24625) 8519 : cluster [DBG] pgmap v4854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:22 gibba011 bash[30791]: cluster 2021-03-10T16:34:21.528174+0000 mgr.gibba032.whopeo (mgr.24625) 8519 : cluster [DBG] pgmap v4854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:23.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:22 gibba011 bash[31774]: cluster 2021-03-10T16:34:21.528174+0000 mgr.gibba032.whopeo (mgr.24625) 8519 : cluster [DBG] pgmap v4854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:25.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:24 gibba032 bash[24004]: cluster 2021-03-10T16:34:23.529520+0000 mgr.gibba032.whopeo (mgr.24625) 8520 : cluster [DBG] pgmap v4855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:25.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:24 gibba011 bash[30791]: cluster 2021-03-10T16:34:23.529520+0000 mgr.gibba032.whopeo (mgr.24625) 8520 : cluster [DBG] pgmap v4855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:24 gibba011 bash[31774]: cluster 2021-03-10T16:34:23.529520+0000 mgr.gibba032.whopeo (mgr.24625) 8520 : cluster [DBG] pgmap v4855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:27.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:26 gibba032 bash[24004]: cluster 2021-03-10T16:34:25.530332+0000 mgr.gibba032.whopeo (mgr.24625) 8521 : cluster [DBG] pgmap v4856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:27.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:26 gibba011 bash[30791]: cluster 2021-03-10T16:34:25.530332+0000 mgr.gibba032.whopeo (mgr.24625) 8521 : cluster [DBG] pgmap v4856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:27.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:26 gibba011 bash[31774]: cluster 2021-03-10T16:34:25.530332+0000 mgr.gibba032.whopeo (mgr.24625) 8521 : cluster [DBG] pgmap v4856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:29.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:28 gibba032 bash[24004]: cluster 2021-03-10T16:34:27.531857+0000 mgr.gibba032.whopeo (mgr.24625) 8522 : cluster [DBG] pgmap v4857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:29.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:28 gibba011 bash[30791]: cluster 2021-03-10T16:34:27.531857+0000 mgr.gibba032.whopeo (mgr.24625) 8522 : cluster [DBG] pgmap v4857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:29.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:28 gibba011 bash[31774]: cluster 2021-03-10T16:34:27.531857+0000 mgr.gibba032.whopeo (mgr.24625) 8522 : cluster [DBG] pgmap v4857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:31.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: cluster 2021-03-10T16:34:29.533219+0000 mgr.gibba032.whopeo (mgr.24625) 8523 : cluster [DBG] pgmap v4858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.178651+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:34:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.182111+0000 mon.a (mon.0) 2957 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:34:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.183594+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.184988+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.186346+0000 mon.a (mon.0) 2960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.189019+0000 mon.a (mon.0) 2961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:30 gibba032 bash[24004]: audit 2021-03-10T16:34:30.190620+0000 mon.a (mon.0) 2962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: cluster 2021-03-10T16:34:29.533219+0000 mgr.gibba032.whopeo (mgr.24625) 8523 : cluster [DBG] pgmap v4858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.178651+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:34:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.182111+0000 mon.a (mon.0) 2957 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:34:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.183594+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.184988+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.186346+0000 mon.a (mon.0) 2960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.189019+0000 mon.a (mon.0) 2961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[30791]: audit 2021-03-10T16:34:30.190620+0000 mon.a (mon.0) 2962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: cluster 2021-03-10T16:34:29.533219+0000 mgr.gibba032.whopeo (mgr.24625) 8523 : cluster [DBG] pgmap v4858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.178651+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:34:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.182111+0000 mon.a (mon.0) 2957 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:34:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.183594+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.184988+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.186346+0000 mon.a (mon.0) 2960 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.189019+0000 mon.a (mon.0) 2961 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:30 gibba011 bash[31774]: audit 2021-03-10T16:34:30.190620+0000 mon.a (mon.0) 2962 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:32.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:29.970345+0000 mgr.gibba032.whopeo (mgr.24625) 8524 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:34:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:29.972060+0000 mgr.gibba032.whopeo (mgr.24625) 8525 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:34:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.179367+0000 mgr.gibba032.whopeo (mgr.24625) 8526 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:34:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.180973+0000 mgr.gibba032.whopeo (mgr.24625) 8527 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:34:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.182401+0000 mgr.gibba032.whopeo (mgr.24625) 8528 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:34:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.182440+0000 mgr.gibba032.whopeo (mgr.24625) 8529 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:34:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.183814+0000 mgr.gibba032.whopeo (mgr.24625) 8530 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:34:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.183852+0000 mgr.gibba032.whopeo (mgr.24625) 8531 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:34:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.185156+0000 mgr.gibba032.whopeo (mgr.24625) 8532 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:34:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.186378+0000 mgr.gibba032.whopeo (mgr.24625) 8533 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:34:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.186414+0000 mgr.gibba032.whopeo (mgr.24625) 8534 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:34:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.187775+0000 mgr.gibba032.whopeo (mgr.24625) 8535 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:34:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.188140+0000 mgr.gibba032.whopeo (mgr.24625) 8536 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:34:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.189724+0000 mgr.gibba032.whopeo (mgr.24625) 8537 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:34:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.189761+0000 mgr.gibba032.whopeo (mgr.24625) 8538 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:34:32.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[30791]: cephadm 2021-03-10T16:34:30.189865+0000 mgr.gibba032.whopeo (mgr.24625) 8539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:29.970345+0000 mgr.gibba032.whopeo (mgr.24625) 8524 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:34:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:29.972060+0000 mgr.gibba032.whopeo (mgr.24625) 8525 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:34:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.179367+0000 mgr.gibba032.whopeo (mgr.24625) 8526 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:34:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.180973+0000 mgr.gibba032.whopeo (mgr.24625) 8527 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:34:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.182401+0000 mgr.gibba032.whopeo (mgr.24625) 8528 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:34:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.182440+0000 mgr.gibba032.whopeo (mgr.24625) 8529 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:34:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.183814+0000 mgr.gibba032.whopeo (mgr.24625) 8530 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:34:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.183852+0000 mgr.gibba032.whopeo (mgr.24625) 8531 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:34:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.185156+0000 mgr.gibba032.whopeo (mgr.24625) 8532 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:34:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.186378+0000 mgr.gibba032.whopeo (mgr.24625) 8533 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:34:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.186414+0000 mgr.gibba032.whopeo (mgr.24625) 8534 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:34:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.187775+0000 mgr.gibba032.whopeo (mgr.24625) 8535 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:34:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.188140+0000 mgr.gibba032.whopeo (mgr.24625) 8536 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:34:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.189724+0000 mgr.gibba032.whopeo (mgr.24625) 8537 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:34:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.189761+0000 mgr.gibba032.whopeo (mgr.24625) 8538 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:34:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:31 gibba011 bash[31774]: cephadm 2021-03-10T16:34:30.189865+0000 mgr.gibba032.whopeo (mgr.24625) 8539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:29.970345+0000 mgr.gibba032.whopeo (mgr.24625) 8524 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:34:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:29.972060+0000 mgr.gibba032.whopeo (mgr.24625) 8525 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:34:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.179367+0000 mgr.gibba032.whopeo (mgr.24625) 8526 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:34:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.180973+0000 mgr.gibba032.whopeo (mgr.24625) 8527 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:34:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.182401+0000 mgr.gibba032.whopeo (mgr.24625) 8528 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:34:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.182440+0000 mgr.gibba032.whopeo (mgr.24625) 8529 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:34:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.183814+0000 mgr.gibba032.whopeo (mgr.24625) 8530 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:34:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.183852+0000 mgr.gibba032.whopeo (mgr.24625) 8531 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:34:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.185156+0000 mgr.gibba032.whopeo (mgr.24625) 8532 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:34:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.186378+0000 mgr.gibba032.whopeo (mgr.24625) 8533 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:34:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.186414+0000 mgr.gibba032.whopeo (mgr.24625) 8534 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:34:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.187775+0000 mgr.gibba032.whopeo (mgr.24625) 8535 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:34:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.188140+0000 mgr.gibba032.whopeo (mgr.24625) 8536 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:34:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.189724+0000 mgr.gibba032.whopeo (mgr.24625) 8537 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:34:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.189761+0000 mgr.gibba032.whopeo (mgr.24625) 8538 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:34:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:31 gibba032 bash[24004]: cephadm 2021-03-10T16:34:30.189865+0000 mgr.gibba032.whopeo (mgr.24625) 8539 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:33.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:32 gibba011 bash[30791]: cluster 2021-03-10T16:34:31.534154+0000 mgr.gibba032.whopeo (mgr.24625) 8540 : cluster [DBG] pgmap v4859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:33.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:32 gibba011 bash[31774]: cluster 2021-03-10T16:34:31.534154+0000 mgr.gibba032.whopeo (mgr.24625) 8540 : cluster [DBG] pgmap v4859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:33.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:32 gibba032 bash[24004]: cluster 2021-03-10T16:34:31.534154+0000 mgr.gibba032.whopeo (mgr.24625) 8540 : cluster [DBG] pgmap v4859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:35.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:34 gibba011 bash[30791]: cluster 2021-03-10T16:34:33.535630+0000 mgr.gibba032.whopeo (mgr.24625) 8541 : cluster [DBG] pgmap v4860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:34 gibba011 bash[31774]: cluster 2021-03-10T16:34:33.535630+0000 mgr.gibba032.whopeo (mgr.24625) 8541 : cluster [DBG] pgmap v4860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:34 gibba032 bash[24004]: cluster 2021-03-10T16:34:33.535630+0000 mgr.gibba032.whopeo (mgr.24625) 8541 : cluster [DBG] pgmap v4860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:37.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:36 gibba011 bash[31774]: cluster 2021-03-10T16:34:35.537149+0000 mgr.gibba032.whopeo (mgr.24625) 8542 : cluster [DBG] pgmap v4861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:37.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:36 gibba011 bash[30791]: cluster 2021-03-10T16:34:35.537149+0000 mgr.gibba032.whopeo (mgr.24625) 8542 : cluster [DBG] pgmap v4861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:36 gibba032 bash[24004]: cluster 2021-03-10T16:34:35.537149+0000 mgr.gibba032.whopeo (mgr.24625) 8542 : cluster [DBG] pgmap v4861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:39.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:38 gibba011 bash[30791]: cluster 2021-03-10T16:34:37.537994+0000 mgr.gibba032.whopeo (mgr.24625) 8543 : cluster [DBG] pgmap v4862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:38 gibba011 bash[31774]: cluster 2021-03-10T16:34:37.537994+0000 mgr.gibba032.whopeo (mgr.24625) 8543 : cluster [DBG] pgmap v4862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:38 gibba032 bash[24004]: cluster 2021-03-10T16:34:37.537994+0000 mgr.gibba032.whopeo (mgr.24625) 8543 : cluster [DBG] pgmap v4862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:41.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:40 gibba011 bash[30791]: cluster 2021-03-10T16:34:39.539398+0000 mgr.gibba032.whopeo (mgr.24625) 8544 : cluster [DBG] pgmap v4863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:40 gibba011 bash[31774]: cluster 2021-03-10T16:34:39.539398+0000 mgr.gibba032.whopeo (mgr.24625) 8544 : cluster [DBG] pgmap v4863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:40 gibba032 bash[24004]: cluster 2021-03-10T16:34:39.539398+0000 mgr.gibba032.whopeo (mgr.24625) 8544 : cluster [DBG] pgmap v4863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:42.823 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:34:43.054 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:34:43.055 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:34:43.055 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:34:43.055 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:34:43.055 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:34:43.056 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:34:43.056 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:34:43.056 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:34:43.056 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:34:43.056 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:34:43.057 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:34:43.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:34:43.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:34:43.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:34:43.057 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:34:43.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:34:43.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:34:43.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:34:43.058 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:34:43.059 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:34:43.059 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:34:43.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:42 gibba011 bash[31774]: cluster 2021-03-10T16:34:41.541191+0000 mgr.gibba032.whopeo (mgr.24625) 8545 : cluster [DBG] pgmap v4864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:42 gibba011 bash[30791]: cluster 2021-03-10T16:34:41.541191+0000 mgr.gibba032.whopeo (mgr.24625) 8545 : cluster [DBG] pgmap v4864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:42 gibba032 bash[24004]: cluster 2021-03-10T16:34:41.541191+0000 mgr.gibba032.whopeo (mgr.24625) 8545 : cluster [DBG] pgmap v4864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:43.307 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:34:43.307 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:34:43.307 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:34:43.307 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:43.308 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:34:43.308 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:34:43.308 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:43.308 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:34:43.309 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:34:43.309 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:34:43.309 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:34:43.309 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:34:43.309 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:34:43.310 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:34:43.310 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:34:44.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:43 gibba011 bash[30791]: audit 2021-03-10T16:34:42.814913+0000 mgr.gibba032.whopeo (mgr.24625) 8546 : audit [DBG] from='client.49779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:44.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:43 gibba011 bash[30791]: audit 2021-03-10T16:34:43.309574+0000 mon.c (mon.1) 166 : audit [DBG] from='client.? 172.21.2.111:0/3681040797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:44.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:43 gibba011 bash[31774]: audit 2021-03-10T16:34:42.814913+0000 mgr.gibba032.whopeo (mgr.24625) 8546 : audit [DBG] from='client.49779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:44.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:43 gibba011 bash[31774]: audit 2021-03-10T16:34:43.309574+0000 mon.c (mon.1) 166 : audit [DBG] from='client.? 172.21.2.111:0/3681040797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:44.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:43 gibba032 bash[24004]: audit 2021-03-10T16:34:42.814913+0000 mgr.gibba032.whopeo (mgr.24625) 8546 : audit [DBG] from='client.49779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:44.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:43 gibba032 bash[24004]: audit 2021-03-10T16:34:43.309574+0000 mon.c (mon.1) 166 : audit [DBG] from='client.? 172.21.2.111:0/3681040797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:34:45.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:44 gibba011 bash[31774]: audit 2021-03-10T16:34:43.044476+0000 mgr.gibba032.whopeo (mgr.24625) 8547 : audit [DBG] from='client.59767 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:44 gibba011 bash[31774]: cluster 2021-03-10T16:34:43.542168+0000 mgr.gibba032.whopeo (mgr.24625) 8548 : cluster [DBG] pgmap v4865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:44 gibba011 bash[30791]: audit 2021-03-10T16:34:43.044476+0000 mgr.gibba032.whopeo (mgr.24625) 8547 : audit [DBG] from='client.59767 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:45.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:44 gibba011 bash[30791]: cluster 2021-03-10T16:34:43.542168+0000 mgr.gibba032.whopeo (mgr.24625) 8548 : cluster [DBG] pgmap v4865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:45.206 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:44 gibba032 bash[24004]: audit 2021-03-10T16:34:43.044476+0000 mgr.gibba032.whopeo (mgr.24625) 8547 : audit [DBG] from='client.59767 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:34:45.206 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:44 gibba032 bash[24004]: cluster 2021-03-10T16:34:43.542168+0000 mgr.gibba032.whopeo (mgr.24625) 8548 : cluster [DBG] pgmap v4865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:47.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:46 gibba011 bash[30791]: cephadm 2021-03-10T16:34:45.205701+0000 mgr.gibba032.whopeo (mgr.24625) 8549 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:46 gibba011 bash[30791]: cluster 2021-03-10T16:34:45.543417+0000 mgr.gibba032.whopeo (mgr.24625) 8550 : cluster [DBG] pgmap v4866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:46 gibba011 bash[31774]: cephadm 2021-03-10T16:34:45.205701+0000 mgr.gibba032.whopeo (mgr.24625) 8549 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:47.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:46 gibba011 bash[31774]: cluster 2021-03-10T16:34:45.543417+0000 mgr.gibba032.whopeo (mgr.24625) 8550 : cluster [DBG] pgmap v4866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:46 gibba032 bash[24004]: cephadm 2021-03-10T16:34:45.205701+0000 mgr.gibba032.whopeo (mgr.24625) 8549 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:34:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:46 gibba032 bash[24004]: cluster 2021-03-10T16:34:45.543417+0000 mgr.gibba032.whopeo (mgr.24625) 8550 : cluster [DBG] pgmap v4866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:48 gibba032 bash[24004]: cluster 2021-03-10T16:34:47.545118+0000 mgr.gibba032.whopeo (mgr.24625) 8551 : cluster [DBG] pgmap v4867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:48 gibba011 bash[30791]: cluster 2021-03-10T16:34:47.545118+0000 mgr.gibba032.whopeo (mgr.24625) 8551 : cluster [DBG] pgmap v4867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:48 gibba011 bash[31774]: cluster 2021-03-10T16:34:47.545118+0000 mgr.gibba032.whopeo (mgr.24625) 8551 : cluster [DBG] pgmap v4867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:50 gibba032 bash[24004]: cluster 2021-03-10T16:34:49.546174+0000 mgr.gibba032.whopeo (mgr.24625) 8552 : cluster [DBG] pgmap v4868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:50 gibba011 bash[30791]: cluster 2021-03-10T16:34:49.546174+0000 mgr.gibba032.whopeo (mgr.24625) 8552 : cluster [DBG] pgmap v4868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:50 gibba011 bash[31774]: cluster 2021-03-10T16:34:49.546174+0000 mgr.gibba032.whopeo (mgr.24625) 8552 : cluster [DBG] pgmap v4868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:52 gibba032 bash[24004]: cluster 2021-03-10T16:34:51.547842+0000 mgr.gibba032.whopeo (mgr.24625) 8553 : cluster [DBG] pgmap v4869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:52 gibba011 bash[30791]: cluster 2021-03-10T16:34:51.547842+0000 mgr.gibba032.whopeo (mgr.24625) 8553 : cluster [DBG] pgmap v4869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:52 gibba011 bash[31774]: cluster 2021-03-10T16:34:51.547842+0000 mgr.gibba032.whopeo (mgr.24625) 8553 : cluster [DBG] pgmap v4869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:54 gibba032 bash[24004]: cluster 2021-03-10T16:34:53.549454+0000 mgr.gibba032.whopeo (mgr.24625) 8554 : cluster [DBG] pgmap v4870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:54 gibba011 bash[30791]: cluster 2021-03-10T16:34:53.549454+0000 mgr.gibba032.whopeo (mgr.24625) 8554 : cluster [DBG] pgmap v4870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:54 gibba011 bash[31774]: cluster 2021-03-10T16:34:53.549454+0000 mgr.gibba032.whopeo (mgr.24625) 8554 : cluster [DBG] pgmap v4870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:56 gibba032 bash[24004]: cluster 2021-03-10T16:34:55.550475+0000 mgr.gibba032.whopeo (mgr.24625) 8555 : cluster [DBG] pgmap v4871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:56 gibba011 bash[30791]: cluster 2021-03-10T16:34:55.550475+0000 mgr.gibba032.whopeo (mgr.24625) 8555 : cluster [DBG] pgmap v4871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:56 gibba011 bash[31774]: cluster 2021-03-10T16:34:55.550475+0000 mgr.gibba032.whopeo (mgr.24625) 8555 : cluster [DBG] pgmap v4871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:58.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.086731+0000 mon.a (mon.0) 2963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:34:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.087511+0000 mon.a (mon.0) 2964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:34:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.192357+0000 mon.a (mon.0) 2965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:34:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.193152+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:34:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.193868+0000 mon.a (mon.0) 2967 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:34:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:57 gibba032 bash[24004]: audit 2021-03-10T16:34:57.194569+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:34:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.086731+0000 mon.a (mon.0) 2963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:34:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.087511+0000 mon.a (mon.0) 2964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:34:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.192357+0000 mon.a (mon.0) 2965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:34:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.193152+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:34:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.193868+0000 mon.a (mon.0) 2967 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:34:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[30791]: audit 2021-03-10T16:34:57.194569+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:34:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.086731+0000 mon.a (mon.0) 2963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:34:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.087511+0000 mon.a (mon.0) 2964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:34:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.192357+0000 mon.a (mon.0) 2965 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:34:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.193152+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:34:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.193868+0000 mon.a (mon.0) 2967 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:34:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:57 gibba011 bash[31774]: audit 2021-03-10T16:34:57.194569+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:34:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:34:58 gibba032 bash[24004]: cluster 2021-03-10T16:34:57.552115+0000 mgr.gibba032.whopeo (mgr.24625) 8556 : cluster [DBG] pgmap v4872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:34:58 gibba011 bash[30791]: cluster 2021-03-10T16:34:57.552115+0000 mgr.gibba032.whopeo (mgr.24625) 8556 : cluster [DBG] pgmap v4872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:34:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:34:58 gibba011 bash[31774]: cluster 2021-03-10T16:34:57.552115+0000 mgr.gibba032.whopeo (mgr.24625) 8556 : cluster [DBG] pgmap v4872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:00 gibba032 bash[24004]: cluster 2021-03-10T16:34:59.553466+0000 mgr.gibba032.whopeo (mgr.24625) 8557 : cluster [DBG] pgmap v4873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:00 gibba011 bash[30791]: cluster 2021-03-10T16:34:59.553466+0000 mgr.gibba032.whopeo (mgr.24625) 8557 : cluster [DBG] pgmap v4873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:01.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:00 gibba011 bash[31774]: cluster 2021-03-10T16:34:59.553466+0000 mgr.gibba032.whopeo (mgr.24625) 8557 : cluster [DBG] pgmap v4873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:02.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:01 gibba032 bash[24004]: cephadm 2021-03-10T16:35:00.213814+0000 mgr.gibba032.whopeo (mgr.24625) 8558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:02.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:01 gibba011 bash[30791]: cephadm 2021-03-10T16:35:00.213814+0000 mgr.gibba032.whopeo (mgr.24625) 8558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:02.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:01 gibba011 bash[31774]: cephadm 2021-03-10T16:35:00.213814+0000 mgr.gibba032.whopeo (mgr.24625) 8558 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:02 gibba032 bash[24004]: cluster 2021-03-10T16:35:01.554405+0000 mgr.gibba032.whopeo (mgr.24625) 8559 : cluster [DBG] pgmap v4874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:02 gibba011 bash[30791]: cluster 2021-03-10T16:35:01.554405+0000 mgr.gibba032.whopeo (mgr.24625) 8559 : cluster [DBG] pgmap v4874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:02 gibba011 bash[31774]: cluster 2021-03-10T16:35:01.554405+0000 mgr.gibba032.whopeo (mgr.24625) 8559 : cluster [DBG] pgmap v4874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:04.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:03 gibba032 bash[24004]: cluster 2021-03-10T16:35:03.555643+0000 mgr.gibba032.whopeo (mgr.24625) 8560 : cluster [DBG] pgmap v4875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:04.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:03 gibba011 bash[30791]: cluster 2021-03-10T16:35:03.555643+0000 mgr.gibba032.whopeo (mgr.24625) 8560 : cluster [DBG] pgmap v4875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:04.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:03 gibba011 bash[31774]: cluster 2021-03-10T16:35:03.555643+0000 mgr.gibba032.whopeo (mgr.24625) 8560 : cluster [DBG] pgmap v4875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:07.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:06 gibba032 bash[24004]: cluster 2021-03-10T16:35:05.557123+0000 mgr.gibba032.whopeo (mgr.24625) 8561 : cluster [DBG] pgmap v4876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:07.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:06 gibba011 bash[30791]: cluster 2021-03-10T16:35:05.557123+0000 mgr.gibba032.whopeo (mgr.24625) 8561 : cluster [DBG] pgmap v4876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:06 gibba011 bash[31774]: cluster 2021-03-10T16:35:05.557123+0000 mgr.gibba032.whopeo (mgr.24625) 8561 : cluster [DBG] pgmap v4876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:09.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:08 gibba032 bash[24004]: cluster 2021-03-10T16:35:07.558427+0000 mgr.gibba032.whopeo (mgr.24625) 8562 : cluster [DBG] pgmap v4877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:09.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:08 gibba011 bash[30791]: cluster 2021-03-10T16:35:07.558427+0000 mgr.gibba032.whopeo (mgr.24625) 8562 : cluster [DBG] pgmap v4877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:09.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:08 gibba011 bash[31774]: cluster 2021-03-10T16:35:07.558427+0000 mgr.gibba032.whopeo (mgr.24625) 8562 : cluster [DBG] pgmap v4877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:11.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:10 gibba032 bash[24004]: cluster 2021-03-10T16:35:09.559854+0000 mgr.gibba032.whopeo (mgr.24625) 8563 : cluster [DBG] pgmap v4878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:11.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:10 gibba011 bash[30791]: cluster 2021-03-10T16:35:09.559854+0000 mgr.gibba032.whopeo (mgr.24625) 8563 : cluster [DBG] pgmap v4878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:10 gibba011 bash[31774]: cluster 2021-03-10T16:35:09.559854+0000 mgr.gibba032.whopeo (mgr.24625) 8563 : cluster [DBG] pgmap v4878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:13.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:12 gibba032 bash[24004]: cluster 2021-03-10T16:35:11.561644+0000 mgr.gibba032.whopeo (mgr.24625) 8564 : cluster [DBG] pgmap v4879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:12 gibba011 bash[30791]: cluster 2021-03-10T16:35:11.561644+0000 mgr.gibba032.whopeo (mgr.24625) 8564 : cluster [DBG] pgmap v4879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:13.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:12 gibba011 bash[31774]: cluster 2021-03-10T16:35:11.561644+0000 mgr.gibba032.whopeo (mgr.24625) 8564 : cluster [DBG] pgmap v4879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:13.550 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:35:13.774 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:35:13.775 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:35:13.775 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:35:13.775 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:35:13.775 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:35:13.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:35:13.776 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:35:13.776 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:35:13.776 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:35:13.776 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:35:13.777 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:35:13.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:35:13.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:35:13.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:35:13.777 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:35:13.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:35:13.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:35:13.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:35:13.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:35:13.779 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:35:13.779 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:35:14.024 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:35:14.025 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:35:14.025 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:35:14.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:14.025 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:35:14.026 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:35:14.026 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:14.026 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:35:14.026 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:35:14.027 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:14.027 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:35:14.027 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:35:14.027 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:35:14.027 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:35:14.028 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:35:15.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:14 gibba032 bash[24004]: audit 2021-03-10T16:35:13.541966+0000 mgr.gibba032.whopeo (mgr.24625) 8565 : audit [DBG] from='client.59779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:14 gibba032 bash[24004]: cluster 2021-03-10T16:35:13.562897+0000 mgr.gibba032.whopeo (mgr.24625) 8566 : cluster [DBG] pgmap v4880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:14 gibba032 bash[24004]: audit 2021-03-10T16:35:13.766290+0000 mgr.gibba032.whopeo (mgr.24625) 8567 : audit [DBG] from='client.59785 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:14 gibba032 bash[24004]: audit 2021-03-10T16:35:14.026696+0000 mon.c (mon.1) 167 : audit [DBG] from='client.? 172.21.2.111:0/3953135278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[30791]: audit 2021-03-10T16:35:13.541966+0000 mgr.gibba032.whopeo (mgr.24625) 8565 : audit [DBG] from='client.59779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[30791]: cluster 2021-03-10T16:35:13.562897+0000 mgr.gibba032.whopeo (mgr.24625) 8566 : cluster [DBG] pgmap v4880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[30791]: audit 2021-03-10T16:35:13.766290+0000 mgr.gibba032.whopeo (mgr.24625) 8567 : audit [DBG] from='client.59785 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[30791]: audit 2021-03-10T16:35:14.026696+0000 mon.c (mon.1) 167 : audit [DBG] from='client.? 172.21.2.111:0/3953135278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[31774]: audit 2021-03-10T16:35:13.541966+0000 mgr.gibba032.whopeo (mgr.24625) 8565 : audit [DBG] from='client.59779 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[31774]: cluster 2021-03-10T16:35:13.562897+0000 mgr.gibba032.whopeo (mgr.24625) 8566 : cluster [DBG] pgmap v4880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[31774]: audit 2021-03-10T16:35:13.766290+0000 mgr.gibba032.whopeo (mgr.24625) 8567 : audit [DBG] from='client.59785 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:15.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:14 gibba011 bash[31774]: audit 2021-03-10T16:35:14.026696+0000 mon.c (mon.1) 167 : audit [DBG] from='client.? 172.21.2.111:0/3953135278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:17.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:16 gibba032 bash[24004]: cephadm 2021-03-10T16:35:15.228479+0000 mgr.gibba032.whopeo (mgr.24625) 8568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:17.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:16 gibba032 bash[24004]: cluster 2021-03-10T16:35:15.564465+0000 mgr.gibba032.whopeo (mgr.24625) 8569 : cluster [DBG] pgmap v4881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:17.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:16 gibba011 bash[30791]: cephadm 2021-03-10T16:35:15.228479+0000 mgr.gibba032.whopeo (mgr.24625) 8568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:17.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:16 gibba011 bash[30791]: cluster 2021-03-10T16:35:15.564465+0000 mgr.gibba032.whopeo (mgr.24625) 8569 : cluster [DBG] pgmap v4881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:16 gibba011 bash[31774]: cephadm 2021-03-10T16:35:15.228479+0000 mgr.gibba032.whopeo (mgr.24625) 8568 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:17.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:16 gibba011 bash[31774]: cluster 2021-03-10T16:35:15.564465+0000 mgr.gibba032.whopeo (mgr.24625) 8569 : cluster [DBG] pgmap v4881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:19.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:18 gibba011 bash[30791]: cluster 2021-03-10T16:35:17.566217+0000 mgr.gibba032.whopeo (mgr.24625) 8570 : cluster [DBG] pgmap v4882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:19.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:18 gibba011 bash[31774]: cluster 2021-03-10T16:35:17.566217+0000 mgr.gibba032.whopeo (mgr.24625) 8570 : cluster [DBG] pgmap v4882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:18 gibba032 bash[24004]: cluster 2021-03-10T16:35:17.566217+0000 mgr.gibba032.whopeo (mgr.24625) 8570 : cluster [DBG] pgmap v4882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:21.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:20 gibba011 bash[30791]: cluster 2021-03-10T16:35:19.567577+0000 mgr.gibba032.whopeo (mgr.24625) 8571 : cluster [DBG] pgmap v4883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:20 gibba011 bash[31774]: cluster 2021-03-10T16:35:19.567577+0000 mgr.gibba032.whopeo (mgr.24625) 8571 : cluster [DBG] pgmap v4883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:20 gibba032 bash[24004]: cluster 2021-03-10T16:35:19.567577+0000 mgr.gibba032.whopeo (mgr.24625) 8571 : cluster [DBG] pgmap v4883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:22 gibba011 bash[30791]: cluster 2021-03-10T16:35:21.569259+0000 mgr.gibba032.whopeo (mgr.24625) 8572 : cluster [DBG] pgmap v4884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:23.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:22 gibba011 bash[31774]: cluster 2021-03-10T16:35:21.569259+0000 mgr.gibba032.whopeo (mgr.24625) 8572 : cluster [DBG] pgmap v4884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:22 gibba032 bash[24004]: cluster 2021-03-10T16:35:21.569259+0000 mgr.gibba032.whopeo (mgr.24625) 8572 : cluster [DBG] pgmap v4884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:25.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:24 gibba011 bash[30791]: cluster 2021-03-10T16:35:23.570816+0000 mgr.gibba032.whopeo (mgr.24625) 8573 : cluster [DBG] pgmap v4885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:24 gibba011 bash[31774]: cluster 2021-03-10T16:35:23.570816+0000 mgr.gibba032.whopeo (mgr.24625) 8573 : cluster [DBG] pgmap v4885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:24 gibba032 bash[24004]: cluster 2021-03-10T16:35:23.570816+0000 mgr.gibba032.whopeo (mgr.24625) 8573 : cluster [DBG] pgmap v4885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:27.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:26 gibba011 bash[30791]: cluster 2021-03-10T16:35:25.572366+0000 mgr.gibba032.whopeo (mgr.24625) 8574 : cluster [DBG] pgmap v4886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:27.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:26 gibba011 bash[31774]: cluster 2021-03-10T16:35:25.572366+0000 mgr.gibba032.whopeo (mgr.24625) 8574 : cluster [DBG] pgmap v4886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:26 gibba032 bash[24004]: cluster 2021-03-10T16:35:25.572366+0000 mgr.gibba032.whopeo (mgr.24625) 8574 : cluster [DBG] pgmap v4886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:29.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:28 gibba011 bash[30791]: cluster 2021-03-10T16:35:27.573888+0000 mgr.gibba032.whopeo (mgr.24625) 8575 : cluster [DBG] pgmap v4887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:29.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:28 gibba011 bash[31774]: cluster 2021-03-10T16:35:27.573888+0000 mgr.gibba032.whopeo (mgr.24625) 8575 : cluster [DBG] pgmap v4887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:28 gibba032 bash[24004]: cluster 2021-03-10T16:35:27.573888+0000 mgr.gibba032.whopeo (mgr.24625) 8575 : cluster [DBG] pgmap v4887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:31.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: cluster 2021-03-10T16:35:29.575259+0000 mgr.gibba032.whopeo (mgr.24625) 8576 : cluster [DBG] pgmap v4888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.475149+0000 mon.a (mon.0) 2969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:35:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.478140+0000 mon.a (mon.0) 2970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:35:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.479687+0000 mon.a (mon.0) 2971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.481160+0000 mon.a (mon.0) 2972 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.482751+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.485983+0000 mon.a (mon.0) 2974 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[30791]: audit 2021-03-10T16:35:30.487828+0000 mon.a (mon.0) 2975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: cluster 2021-03-10T16:35:29.575259+0000 mgr.gibba032.whopeo (mgr.24625) 8576 : cluster [DBG] pgmap v4888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.475149+0000 mon.a (mon.0) 2969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:35:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.478140+0000 mon.a (mon.0) 2970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:35:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.479687+0000 mon.a (mon.0) 2971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.481160+0000 mon.a (mon.0) 2972 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.482751+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.485983+0000 mon.a (mon.0) 2974 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:30 gibba011 bash[31774]: audit 2021-03-10T16:35:30.487828+0000 mon.a (mon.0) 2975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: cluster 2021-03-10T16:35:29.575259+0000 mgr.gibba032.whopeo (mgr.24625) 8576 : cluster [DBG] pgmap v4888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.475149+0000 mon.a (mon.0) 2969 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:35:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.478140+0000 mon.a (mon.0) 2970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:35:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.479687+0000 mon.a (mon.0) 2971 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.481160+0000 mon.a (mon.0) 2972 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.482751+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.485983+0000 mon.a (mon.0) 2974 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:30 gibba032 bash[24004]: audit 2021-03-10T16:35:30.487828+0000 mon.a (mon.0) 2975 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.242850+0000 mgr.gibba032.whopeo (mgr.24625) 8577 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:35:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.244465+0000 mgr.gibba032.whopeo (mgr.24625) 8578 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:35:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.475450+0000 mgr.gibba032.whopeo (mgr.24625) 8579 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:35:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.477085+0000 mgr.gibba032.whopeo (mgr.24625) 8580 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:35:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.478448+0000 mgr.gibba032.whopeo (mgr.24625) 8581 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:35:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.478489+0000 mgr.gibba032.whopeo (mgr.24625) 8582 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:35:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.480239+0000 mgr.gibba032.whopeo (mgr.24625) 8583 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:35:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.480279+0000 mgr.gibba032.whopeo (mgr.24625) 8584 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.481850+0000 mgr.gibba032.whopeo (mgr.24625) 8585 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.483364+0000 mgr.gibba032.whopeo (mgr.24625) 8586 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.483406+0000 mgr.gibba032.whopeo (mgr.24625) 8587 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.485196+0000 mgr.gibba032.whopeo (mgr.24625) 8588 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.485552+0000 mgr.gibba032.whopeo (mgr.24625) 8589 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:35:32.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.486872+0000 mgr.gibba032.whopeo (mgr.24625) 8590 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:35:32.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.486913+0000 mgr.gibba032.whopeo (mgr.24625) 8591 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:35:32.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[30791]: cephadm 2021-03-10T16:35:30.487027+0000 mgr.gibba032.whopeo (mgr.24625) 8592 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.242850+0000 mgr.gibba032.whopeo (mgr.24625) 8577 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:35:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.244465+0000 mgr.gibba032.whopeo (mgr.24625) 8578 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:35:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.475450+0000 mgr.gibba032.whopeo (mgr.24625) 8579 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:35:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.477085+0000 mgr.gibba032.whopeo (mgr.24625) 8580 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:35:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.478448+0000 mgr.gibba032.whopeo (mgr.24625) 8581 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:35:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.478489+0000 mgr.gibba032.whopeo (mgr.24625) 8582 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:35:32.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.480239+0000 mgr.gibba032.whopeo (mgr.24625) 8583 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:35:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.480279+0000 mgr.gibba032.whopeo (mgr.24625) 8584 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:35:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.481850+0000 mgr.gibba032.whopeo (mgr.24625) 8585 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:35:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.483364+0000 mgr.gibba032.whopeo (mgr.24625) 8586 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:35:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.483406+0000 mgr.gibba032.whopeo (mgr.24625) 8587 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:35:32.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.485196+0000 mgr.gibba032.whopeo (mgr.24625) 8588 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:35:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.485552+0000 mgr.gibba032.whopeo (mgr.24625) 8589 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:35:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.486872+0000 mgr.gibba032.whopeo (mgr.24625) 8590 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:35:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.486913+0000 mgr.gibba032.whopeo (mgr.24625) 8591 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:35:32.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:31 gibba011 bash[31774]: cephadm 2021-03-10T16:35:30.487027+0000 mgr.gibba032.whopeo (mgr.24625) 8592 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.242850+0000 mgr.gibba032.whopeo (mgr.24625) 8577 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:35:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.244465+0000 mgr.gibba032.whopeo (mgr.24625) 8578 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:35:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.475450+0000 mgr.gibba032.whopeo (mgr.24625) 8579 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:35:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.477085+0000 mgr.gibba032.whopeo (mgr.24625) 8580 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:35:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.478448+0000 mgr.gibba032.whopeo (mgr.24625) 8581 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:35:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.478489+0000 mgr.gibba032.whopeo (mgr.24625) 8582 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:35:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.480239+0000 mgr.gibba032.whopeo (mgr.24625) 8583 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:35:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.480279+0000 mgr.gibba032.whopeo (mgr.24625) 8584 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:35:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.481850+0000 mgr.gibba032.whopeo (mgr.24625) 8585 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:35:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.483364+0000 mgr.gibba032.whopeo (mgr.24625) 8586 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:35:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.483406+0000 mgr.gibba032.whopeo (mgr.24625) 8587 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:35:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.485196+0000 mgr.gibba032.whopeo (mgr.24625) 8588 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:35:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.485552+0000 mgr.gibba032.whopeo (mgr.24625) 8589 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:35:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.486872+0000 mgr.gibba032.whopeo (mgr.24625) 8590 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:35:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.486913+0000 mgr.gibba032.whopeo (mgr.24625) 8591 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:35:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:31 gibba032 bash[24004]: cephadm 2021-03-10T16:35:30.487027+0000 mgr.gibba032.whopeo (mgr.24625) 8592 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:33.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:32 gibba011 bash[31774]: cluster 2021-03-10T16:35:31.576837+0000 mgr.gibba032.whopeo (mgr.24625) 8593 : cluster [DBG] pgmap v4889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:33.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:32 gibba011 bash[30791]: cluster 2021-03-10T16:35:31.576837+0000 mgr.gibba032.whopeo (mgr.24625) 8593 : cluster [DBG] pgmap v4889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:33.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:32 gibba032 bash[24004]: cluster 2021-03-10T16:35:31.576837+0000 mgr.gibba032.whopeo (mgr.24625) 8593 : cluster [DBG] pgmap v4889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:35.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:34 gibba011 bash[30791]: cluster 2021-03-10T16:35:33.578184+0000 mgr.gibba032.whopeo (mgr.24625) 8594 : cluster [DBG] pgmap v4890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:34 gibba011 bash[31774]: cluster 2021-03-10T16:35:33.578184+0000 mgr.gibba032.whopeo (mgr.24625) 8594 : cluster [DBG] pgmap v4890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:34 gibba032 bash[24004]: cluster 2021-03-10T16:35:33.578184+0000 mgr.gibba032.whopeo (mgr.24625) 8594 : cluster [DBG] pgmap v4890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:36 gibba032 bash[24004]: cluster 2021-03-10T16:35:35.579644+0000 mgr.gibba032.whopeo (mgr.24625) 8595 : cluster [DBG] pgmap v4891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:37.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:36 gibba011 bash[30791]: cluster 2021-03-10T16:35:35.579644+0000 mgr.gibba032.whopeo (mgr.24625) 8595 : cluster [DBG] pgmap v4891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:37.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:36 gibba011 bash[31774]: cluster 2021-03-10T16:35:35.579644+0000 mgr.gibba032.whopeo (mgr.24625) 8595 : cluster [DBG] pgmap v4891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:38 gibba032 bash[24004]: cluster 2021-03-10T16:35:37.581151+0000 mgr.gibba032.whopeo (mgr.24625) 8596 : cluster [DBG] pgmap v4892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:39.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:38 gibba011 bash[30791]: cluster 2021-03-10T16:35:37.581151+0000 mgr.gibba032.whopeo (mgr.24625) 8596 : cluster [DBG] pgmap v4892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:39.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:38 gibba011 bash[31774]: cluster 2021-03-10T16:35:37.581151+0000 mgr.gibba032.whopeo (mgr.24625) 8596 : cluster [DBG] pgmap v4892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:40 gibba032 bash[24004]: cluster 2021-03-10T16:35:39.582625+0000 mgr.gibba032.whopeo (mgr.24625) 8597 : cluster [DBG] pgmap v4893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:40 gibba011 bash[30791]: cluster 2021-03-10T16:35:39.582625+0000 mgr.gibba032.whopeo (mgr.24625) 8597 : cluster [DBG] pgmap v4893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:41.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:40 gibba011 bash[31774]: cluster 2021-03-10T16:35:39.582625+0000 mgr.gibba032.whopeo (mgr.24625) 8597 : cluster [DBG] pgmap v4893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:42 gibba032 bash[24004]: cluster 2021-03-10T16:35:41.584518+0000 mgr.gibba032.whopeo (mgr.24625) 8598 : cluster [DBG] pgmap v4894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:42 gibba011 bash[30791]: cluster 2021-03-10T16:35:41.584518+0000 mgr.gibba032.whopeo (mgr.24625) 8598 : cluster [DBG] pgmap v4894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:42 gibba011 bash[31774]: cluster 2021-03-10T16:35:41.584518+0000 mgr.gibba032.whopeo (mgr.24625) 8598 : cluster [DBG] pgmap v4894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:44.265 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:35:44.490 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:35:44.490 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:35:44.491 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:35:44.491 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:35:44.491 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:35:44.491 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:35:44.491 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:35:44.492 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:35:44.492 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:35:44.492 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:35:44.492 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:35:44.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:35:44.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:35:44.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:35:44.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:35:44.493 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:35:44.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:35:44.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:35:44.494 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:35:44.494 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:35:44.495 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:35:44.740 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:35:44.740 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:35:44.740 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:35:44.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:44.741 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:35:44.741 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:35:44.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:44.741 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:35:44.742 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:35:44.742 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:35:44.742 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:35:44.742 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:35:44.742 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:35:44.743 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:35:44.743 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:35:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:44 gibba032 bash[24004]: cluster 2021-03-10T16:35:43.585950+0000 mgr.gibba032.whopeo (mgr.24625) 8599 : cluster [DBG] pgmap v4895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:44 gibba032 bash[24004]: audit 2021-03-10T16:35:44.742478+0000 mon.c (mon.1) 168 : audit [DBG] from='client.? 172.21.2.111:0/2052966475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:44 gibba011 bash[30791]: cluster 2021-03-10T16:35:43.585950+0000 mgr.gibba032.whopeo (mgr.24625) 8599 : cluster [DBG] pgmap v4895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:45.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:44 gibba011 bash[30791]: audit 2021-03-10T16:35:44.742478+0000 mon.c (mon.1) 168 : audit [DBG] from='client.? 172.21.2.111:0/2052966475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:44 gibba011 bash[31774]: cluster 2021-03-10T16:35:43.585950+0000 mgr.gibba032.whopeo (mgr.24625) 8599 : cluster [DBG] pgmap v4895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:44 gibba011 bash[31774]: audit 2021-03-10T16:35:44.742478+0000 mon.c (mon.1) 168 : audit [DBG] from='client.? 172.21.2.111:0/2052966475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:35:46.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:45 gibba032 bash[24004]: audit 2021-03-10T16:35:44.256480+0000 mgr.gibba032.whopeo (mgr.24625) 8600 : audit [DBG] from='client.59797 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:46.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:45 gibba032 bash[24004]: audit 2021-03-10T16:35:44.478143+0000 mgr.gibba032.whopeo (mgr.24625) 8601 : audit [DBG] from='client.59803 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:46.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:45 gibba011 bash[30791]: audit 2021-03-10T16:35:44.256480+0000 mgr.gibba032.whopeo (mgr.24625) 8600 : audit [DBG] from='client.59797 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:46.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:45 gibba011 bash[30791]: audit 2021-03-10T16:35:44.478143+0000 mgr.gibba032.whopeo (mgr.24625) 8601 : audit [DBG] from='client.59803 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:46.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:45 gibba011 bash[31774]: audit 2021-03-10T16:35:44.256480+0000 mgr.gibba032.whopeo (mgr.24625) 8600 : audit [DBG] from='client.59797 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:46.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:45 gibba011 bash[31774]: audit 2021-03-10T16:35:44.478143+0000 mgr.gibba032.whopeo (mgr.24625) 8601 : audit [DBG] from='client.59803 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:35:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:46 gibba032 bash[24004]: cephadm 2021-03-10T16:35:45.503020+0000 mgr.gibba032.whopeo (mgr.24625) 8602 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:46 gibba032 bash[24004]: cluster 2021-03-10T16:35:45.587419+0000 mgr.gibba032.whopeo (mgr.24625) 8603 : cluster [DBG] pgmap v4896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:46 gibba011 bash[30791]: cephadm 2021-03-10T16:35:45.503020+0000 mgr.gibba032.whopeo (mgr.24625) 8602 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:47.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:46 gibba011 bash[30791]: cluster 2021-03-10T16:35:45.587419+0000 mgr.gibba032.whopeo (mgr.24625) 8603 : cluster [DBG] pgmap v4896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:46 gibba011 bash[31774]: cephadm 2021-03-10T16:35:45.503020+0000 mgr.gibba032.whopeo (mgr.24625) 8602 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:35:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:46 gibba011 bash[31774]: cluster 2021-03-10T16:35:45.587419+0000 mgr.gibba032.whopeo (mgr.24625) 8603 : cluster [DBG] pgmap v4896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:48 gibba032 bash[24004]: cluster 2021-03-10T16:35:47.589011+0000 mgr.gibba032.whopeo (mgr.24625) 8604 : cluster [DBG] pgmap v4897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:48 gibba011 bash[30791]: cluster 2021-03-10T16:35:47.589011+0000 mgr.gibba032.whopeo (mgr.24625) 8604 : cluster [DBG] pgmap v4897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:48 gibba011 bash[31774]: cluster 2021-03-10T16:35:47.589011+0000 mgr.gibba032.whopeo (mgr.24625) 8604 : cluster [DBG] pgmap v4897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:50 gibba032 bash[24004]: cluster 2021-03-10T16:35:49.590467+0000 mgr.gibba032.whopeo (mgr.24625) 8605 : cluster [DBG] pgmap v4898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:50 gibba011 bash[30791]: cluster 2021-03-10T16:35:49.590467+0000 mgr.gibba032.whopeo (mgr.24625) 8605 : cluster [DBG] pgmap v4898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:50 gibba011 bash[31774]: cluster 2021-03-10T16:35:49.590467+0000 mgr.gibba032.whopeo (mgr.24625) 8605 : cluster [DBG] pgmap v4898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:52 gibba032 bash[24004]: cluster 2021-03-10T16:35:51.591978+0000 mgr.gibba032.whopeo (mgr.24625) 8606 : cluster [DBG] pgmap v4899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:52 gibba011 bash[30791]: cluster 2021-03-10T16:35:51.591978+0000 mgr.gibba032.whopeo (mgr.24625) 8606 : cluster [DBG] pgmap v4899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:52 gibba011 bash[31774]: cluster 2021-03-10T16:35:51.591978+0000 mgr.gibba032.whopeo (mgr.24625) 8606 : cluster [DBG] pgmap v4899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:54 gibba032 bash[24004]: cluster 2021-03-10T16:35:53.593391+0000 mgr.gibba032.whopeo (mgr.24625) 8607 : cluster [DBG] pgmap v4900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:54 gibba011 bash[30791]: cluster 2021-03-10T16:35:53.593391+0000 mgr.gibba032.whopeo (mgr.24625) 8607 : cluster [DBG] pgmap v4900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:54 gibba011 bash[31774]: cluster 2021-03-10T16:35:53.593391+0000 mgr.gibba032.whopeo (mgr.24625) 8607 : cluster [DBG] pgmap v4900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:56.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:55 gibba032 bash[24004]: cluster 2021-03-10T16:35:55.594931+0000 mgr.gibba032.whopeo (mgr.24625) 8608 : cluster [DBG] pgmap v4901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:56.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:55 gibba011 bash[30791]: cluster 2021-03-10T16:35:55.594931+0000 mgr.gibba032.whopeo (mgr.24625) 8608 : cluster [DBG] pgmap v4901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:56.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:55 gibba011 bash[31774]: cluster 2021-03-10T16:35:55.594931+0000 mgr.gibba032.whopeo (mgr.24625) 8608 : cluster [DBG] pgmap v4901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:57 gibba032 bash[24004]: audit 2021-03-10T16:35:57.093855+0000 mon.a (mon.0) 2976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:35:57.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:57 gibba032 bash[24004]: audit 2021-03-10T16:35:57.094508+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:35:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:57 gibba011 bash[30791]: audit 2021-03-10T16:35:57.093855+0000 mon.a (mon.0) 2976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:35:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:57 gibba011 bash[30791]: audit 2021-03-10T16:35:57.094508+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:35:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:57 gibba011 bash[31774]: audit 2021-03-10T16:35:57.093855+0000 mon.a (mon.0) 2976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:35:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:57 gibba011 bash[31774]: audit 2021-03-10T16:35:57.094508+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:35:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:58 gibba032 bash[24004]: audit 2021-03-10T16:35:57.203785+0000 mon.a (mon.0) 2978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:35:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:58 gibba032 bash[24004]: audit 2021-03-10T16:35:57.204067+0000 mon.a (mon.0) 2979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:58 gibba032 bash[24004]: audit 2021-03-10T16:35:57.204318+0000 mon.a (mon.0) 2980 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:35:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:58 gibba032 bash[24004]: audit 2021-03-10T16:35:57.204564+0000 mon.a (mon.0) 2981 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:35:58 gibba032 bash[24004]: cluster 2021-03-10T16:35:57.596623+0000 mgr.gibba032.whopeo (mgr.24625) 8609 : cluster [DBG] pgmap v4902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[30791]: audit 2021-03-10T16:35:57.203785+0000 mon.a (mon.0) 2978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:35:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[30791]: audit 2021-03-10T16:35:57.204067+0000 mon.a (mon.0) 2979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[30791]: audit 2021-03-10T16:35:57.204318+0000 mon.a (mon.0) 2980 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:35:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[30791]: audit 2021-03-10T16:35:57.204564+0000 mon.a (mon.0) 2981 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[30791]: cluster 2021-03-10T16:35:57.596623+0000 mgr.gibba032.whopeo (mgr.24625) 8609 : cluster [DBG] pgmap v4902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:35:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[31774]: audit 2021-03-10T16:35:57.203785+0000 mon.a (mon.0) 2978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2021-03-10T16:35:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[31774]: audit 2021-03-10T16:35:57.204067+0000 mon.a (mon.0) 2979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[31774]: audit 2021-03-10T16:35:57.204318+0000 mon.a (mon.0) 2980 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:35:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[31774]: audit 2021-03-10T16:35:57.204564+0000 mon.a (mon.0) 2981 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T16:35:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:35:58 gibba011 bash[31774]: cluster 2021-03-10T16:35:57.596623+0000 mgr.gibba032.whopeo (mgr.24625) 8609 : cluster [DBG] pgmap v4902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:01.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:00 gibba032 bash[24004]: cluster 2021-03-10T16:35:59.598096+0000 mgr.gibba032.whopeo (mgr.24625) 8610 : cluster [DBG] pgmap v4903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:01.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:00 gibba011 bash[30791]: cluster 2021-03-10T16:35:59.598096+0000 mgr.gibba032.whopeo (mgr.24625) 8610 : cluster [DBG] pgmap v4903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:00 gibba011 bash[31774]: cluster 2021-03-10T16:35:59.598096+0000 mgr.gibba032.whopeo (mgr.24625) 8610 : cluster [DBG] pgmap v4903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:02.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:01 gibba032 bash[24004]: cephadm 2021-03-10T16:36:00.518874+0000 mgr.gibba032.whopeo (mgr.24625) 8611 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:02.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:01 gibba011 bash[30791]: cephadm 2021-03-10T16:36:00.518874+0000 mgr.gibba032.whopeo (mgr.24625) 8611 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:02.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:01 gibba011 bash[31774]: cephadm 2021-03-10T16:36:00.518874+0000 mgr.gibba032.whopeo (mgr.24625) 8611 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:03.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:02 gibba011 bash[30791]: cluster 2021-03-10T16:36:01.599708+0000 mgr.gibba032.whopeo (mgr.24625) 8612 : cluster [DBG] pgmap v4904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:03.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:02 gibba011 bash[31774]: cluster 2021-03-10T16:36:01.599708+0000 mgr.gibba032.whopeo (mgr.24625) 8612 : cluster [DBG] pgmap v4904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:02 gibba032 bash[24004]: cluster 2021-03-10T16:36:01.599708+0000 mgr.gibba032.whopeo (mgr.24625) 8612 : cluster [DBG] pgmap v4904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:05.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:04 gibba011 bash[31774]: cluster 2021-03-10T16:36:03.601322+0000 mgr.gibba032.whopeo (mgr.24625) 8613 : cluster [DBG] pgmap v4905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:04 gibba011 bash[30791]: cluster 2021-03-10T16:36:03.601322+0000 mgr.gibba032.whopeo (mgr.24625) 8613 : cluster [DBG] pgmap v4905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:04 gibba032 bash[24004]: cluster 2021-03-10T16:36:03.601322+0000 mgr.gibba032.whopeo (mgr.24625) 8613 : cluster [DBG] pgmap v4905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:07.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:06 gibba011 bash[30791]: cluster 2021-03-10T16:36:05.602778+0000 mgr.gibba032.whopeo (mgr.24625) 8614 : cluster [DBG] pgmap v4906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:07.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:06 gibba011 bash[31774]: cluster 2021-03-10T16:36:05.602778+0000 mgr.gibba032.whopeo (mgr.24625) 8614 : cluster [DBG] pgmap v4906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:06 gibba032 bash[24004]: cluster 2021-03-10T16:36:05.602778+0000 mgr.gibba032.whopeo (mgr.24625) 8614 : cluster [DBG] pgmap v4906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:09.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:08 gibba011 bash[30791]: cluster 2021-03-10T16:36:07.604558+0000 mgr.gibba032.whopeo (mgr.24625) 8615 : cluster [DBG] pgmap v4907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:09.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:08 gibba011 bash[31774]: cluster 2021-03-10T16:36:07.604558+0000 mgr.gibba032.whopeo (mgr.24625) 8615 : cluster [DBG] pgmap v4907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:08 gibba032 bash[24004]: cluster 2021-03-10T16:36:07.604558+0000 mgr.gibba032.whopeo (mgr.24625) 8615 : cluster [DBG] pgmap v4907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:11.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:10 gibba011 bash[30791]: cluster 2021-03-10T16:36:09.605895+0000 mgr.gibba032.whopeo (mgr.24625) 8616 : cluster [DBG] pgmap v4908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:11.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:10 gibba011 bash[31774]: cluster 2021-03-10T16:36:09.605895+0000 mgr.gibba032.whopeo (mgr.24625) 8616 : cluster [DBG] pgmap v4908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:10 gibba032 bash[24004]: cluster 2021-03-10T16:36:09.605895+0000 mgr.gibba032.whopeo (mgr.24625) 8616 : cluster [DBG] pgmap v4908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:13.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:12 gibba011 bash[30791]: cluster 2021-03-10T16:36:11.607699+0000 mgr.gibba032.whopeo (mgr.24625) 8617 : cluster [DBG] pgmap v4909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:13.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:12 gibba011 bash[31774]: cluster 2021-03-10T16:36:11.607699+0000 mgr.gibba032.whopeo (mgr.24625) 8617 : cluster [DBG] pgmap v4909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:12 gibba032 bash[24004]: cluster 2021-03-10T16:36:11.607699+0000 mgr.gibba032.whopeo (mgr.24625) 8617 : cluster [DBG] pgmap v4909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:14.985 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:36:15.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:14 gibba011 bash[30791]: cluster 2021-03-10T16:36:13.609304+0000 mgr.gibba032.whopeo (mgr.24625) 8618 : cluster [DBG] pgmap v4910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:15.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:14 gibba011 bash[31774]: cluster 2021-03-10T16:36:13.609304+0000 mgr.gibba032.whopeo (mgr.24625) 8618 : cluster [DBG] pgmap v4910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:15.215 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:36:15.215 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:36:15.215 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:36:15.215 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:36:15.216 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:36:15.216 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:36:15.216 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:36:15.216 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:36:15.217 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:36:15.217 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:36:15.217 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:36:15.217 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:36:15.217 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:36:15.218 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:36:15.218 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:36:15.218 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:36:15.218 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:36:15.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:36:15.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:36:15.219 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:36:15.219 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:36:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:14 gibba032 bash[24004]: cluster 2021-03-10T16:36:13.609304+0000 mgr.gibba032.whopeo (mgr.24625) 8618 : cluster [DBG] pgmap v4910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:15.460 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:36:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:36:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:36:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:36:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:36:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:36:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:36:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:15.463 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:36:15.463 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:36:15.463 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:36:15.463 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:36:15.463 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:36:16.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:15 gibba011 bash[30791]: audit 2021-03-10T16:36:15.462866+0000 mon.a (mon.0) 2982 : audit [DBG] from='client.? 172.21.2.111:0/1199385040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:16.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:15 gibba011 bash[31774]: audit 2021-03-10T16:36:15.462866+0000 mon.a (mon.0) 2982 : audit [DBG] from='client.? 172.21.2.111:0/1199385040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:16.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:15 gibba032 bash[24004]: audit 2021-03-10T16:36:15.462866+0000 mon.a (mon.0) 2982 : audit [DBG] from='client.? 172.21.2.111:0/1199385040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[30791]: audit 2021-03-10T16:36:14.976647+0000 mgr.gibba032.whopeo (mgr.24625) 8619 : audit [DBG] from='client.59815 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[30791]: audit 2021-03-10T16:36:15.202661+0000 mgr.gibba032.whopeo (mgr.24625) 8620 : audit [DBG] from='client.59821 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[30791]: cephadm 2021-03-10T16:36:15.534223+0000 mgr.gibba032.whopeo (mgr.24625) 8621 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:17.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[30791]: cluster 2021-03-10T16:36:15.610783+0000 mgr.gibba032.whopeo (mgr.24625) 8622 : cluster [DBG] pgmap v4911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:17.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[31774]: audit 2021-03-10T16:36:14.976647+0000 mgr.gibba032.whopeo (mgr.24625) 8619 : audit [DBG] from='client.59815 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[31774]: audit 2021-03-10T16:36:15.202661+0000 mgr.gibba032.whopeo (mgr.24625) 8620 : audit [DBG] from='client.59821 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[31774]: cephadm 2021-03-10T16:36:15.534223+0000 mgr.gibba032.whopeo (mgr.24625) 8621 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:16 gibba011 bash[31774]: cluster 2021-03-10T16:36:15.610783+0000 mgr.gibba032.whopeo (mgr.24625) 8622 : cluster [DBG] pgmap v4911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:16 gibba032 bash[24004]: audit 2021-03-10T16:36:14.976647+0000 mgr.gibba032.whopeo (mgr.24625) 8619 : audit [DBG] from='client.59815 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:16 gibba032 bash[24004]: audit 2021-03-10T16:36:15.202661+0000 mgr.gibba032.whopeo (mgr.24625) 8620 : audit [DBG] from='client.59821 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:16 gibba032 bash[24004]: cephadm 2021-03-10T16:36:15.534223+0000 mgr.gibba032.whopeo (mgr.24625) 8621 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:16 gibba032 bash[24004]: cluster 2021-03-10T16:36:15.610783+0000 mgr.gibba032.whopeo (mgr.24625) 8622 : cluster [DBG] pgmap v4911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:18 gibba032 bash[24004]: cluster 2021-03-10T16:36:17.612601+0000 mgr.gibba032.whopeo (mgr.24625) 8623 : cluster [DBG] pgmap v4912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:18 gibba011 bash[30791]: cluster 2021-03-10T16:36:17.612601+0000 mgr.gibba032.whopeo (mgr.24625) 8623 : cluster [DBG] pgmap v4912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:19.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:18 gibba011 bash[31774]: cluster 2021-03-10T16:36:17.612601+0000 mgr.gibba032.whopeo (mgr.24625) 8623 : cluster [DBG] pgmap v4912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:20 gibba032 bash[24004]: cluster 2021-03-10T16:36:19.613865+0000 mgr.gibba032.whopeo (mgr.24625) 8624 : cluster [DBG] pgmap v4913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:20 gibba011 bash[30791]: cluster 2021-03-10T16:36:19.613865+0000 mgr.gibba032.whopeo (mgr.24625) 8624 : cluster [DBG] pgmap v4913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:20 gibba011 bash[31774]: cluster 2021-03-10T16:36:19.613865+0000 mgr.gibba032.whopeo (mgr.24625) 8624 : cluster [DBG] pgmap v4913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:22 gibba032 bash[24004]: cluster 2021-03-10T16:36:21.615643+0000 mgr.gibba032.whopeo (mgr.24625) 8625 : cluster [DBG] pgmap v4914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:22 gibba011 bash[30791]: cluster 2021-03-10T16:36:21.615643+0000 mgr.gibba032.whopeo (mgr.24625) 8625 : cluster [DBG] pgmap v4914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:22 gibba011 bash[31774]: cluster 2021-03-10T16:36:21.615643+0000 mgr.gibba032.whopeo (mgr.24625) 8625 : cluster [DBG] pgmap v4914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:24 gibba032 bash[24004]: cluster 2021-03-10T16:36:23.617281+0000 mgr.gibba032.whopeo (mgr.24625) 8626 : cluster [DBG] pgmap v4915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:24 gibba011 bash[30791]: cluster 2021-03-10T16:36:23.617281+0000 mgr.gibba032.whopeo (mgr.24625) 8626 : cluster [DBG] pgmap v4915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:25.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:24 gibba011 bash[31774]: cluster 2021-03-10T16:36:23.617281+0000 mgr.gibba032.whopeo (mgr.24625) 8626 : cluster [DBG] pgmap v4915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:26 gibba032 bash[24004]: cluster 2021-03-10T16:36:25.618827+0000 mgr.gibba032.whopeo (mgr.24625) 8627 : cluster [DBG] pgmap v4916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:26 gibba011 bash[30791]: cluster 2021-03-10T16:36:25.618827+0000 mgr.gibba032.whopeo (mgr.24625) 8627 : cluster [DBG] pgmap v4916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:26 gibba011 bash[31774]: cluster 2021-03-10T16:36:25.618827+0000 mgr.gibba032.whopeo (mgr.24625) 8627 : cluster [DBG] pgmap v4916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:28 gibba032 bash[24004]: cluster 2021-03-10T16:36:27.620650+0000 mgr.gibba032.whopeo (mgr.24625) 8628 : cluster [DBG] pgmap v4917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:28 gibba011 bash[30791]: cluster 2021-03-10T16:36:27.620650+0000 mgr.gibba032.whopeo (mgr.24625) 8628 : cluster [DBG] pgmap v4917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:28 gibba011 bash[31774]: cluster 2021-03-10T16:36:27.620650+0000 mgr.gibba032.whopeo (mgr.24625) 8628 : cluster [DBG] pgmap v4917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: cluster 2021-03-10T16:36:29.621881+0000 mgr.gibba032.whopeo (mgr.24625) 8629 : cluster [DBG] pgmap v4918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.782292+0000 mon.a (mon.0) 2983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:36:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.785634+0000 mon.a (mon.0) 2984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:36:31.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.787176+0000 mon.a (mon.0) 2985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.788832+0000 mon.a (mon.0) 2986 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.790217+0000 mon.a (mon.0) 2987 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.793136+0000 mon.a (mon.0) 2988 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:30 gibba032 bash[24004]: audit 2021-03-10T16:36:30.794929+0000 mon.a (mon.0) 2989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: cluster 2021-03-10T16:36:29.621881+0000 mgr.gibba032.whopeo (mgr.24625) 8629 : cluster [DBG] pgmap v4918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.782292+0000 mon.a (mon.0) 2983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:36:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.785634+0000 mon.a (mon.0) 2984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:36:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.787176+0000 mon.a (mon.0) 2985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.788832+0000 mon.a (mon.0) 2986 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.790217+0000 mon.a (mon.0) 2987 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.793136+0000 mon.a (mon.0) 2988 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[30791]: audit 2021-03-10T16:36:30.794929+0000 mon.a (mon.0) 2989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: cluster 2021-03-10T16:36:29.621881+0000 mgr.gibba032.whopeo (mgr.24625) 8629 : cluster [DBG] pgmap v4918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.782292+0000 mon.a (mon.0) 2983 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:36:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.785634+0000 mon.a (mon.0) 2984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:36:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.787176+0000 mon.a (mon.0) 2985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.788832+0000 mon.a (mon.0) 2986 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.790217+0000 mon.a (mon.0) 2987 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.793136+0000 mon.a (mon.0) 2988 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:30 gibba011 bash[31774]: audit 2021-03-10T16:36:30.794929+0000 mon.a (mon.0) 2989 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.550955+0000 mgr.gibba032.whopeo (mgr.24625) 8630 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:36:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.552571+0000 mgr.gibba032.whopeo (mgr.24625) 8631 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:36:32.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.782743+0000 mgr.gibba032.whopeo (mgr.24625) 8632 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:36:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.784694+0000 mgr.gibba032.whopeo (mgr.24625) 8633 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:36:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.786064+0000 mgr.gibba032.whopeo (mgr.24625) 8634 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:36:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.786105+0000 mgr.gibba032.whopeo (mgr.24625) 8635 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:36:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.787755+0000 mgr.gibba032.whopeo (mgr.24625) 8636 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:36:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.787793+0000 mgr.gibba032.whopeo (mgr.24625) 8637 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:36:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.789093+0000 mgr.gibba032.whopeo (mgr.24625) 8638 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:36:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.790693+0000 mgr.gibba032.whopeo (mgr.24625) 8639 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:36:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.790733+0000 mgr.gibba032.whopeo (mgr.24625) 8640 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:36:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.792158+0000 mgr.gibba032.whopeo (mgr.24625) 8641 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:36:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.792542+0000 mgr.gibba032.whopeo (mgr.24625) 8642 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:36:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.793992+0000 mgr.gibba032.whopeo (mgr.24625) 8643 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:36:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.794033+0000 mgr.gibba032.whopeo (mgr.24625) 8644 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:36:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:31 gibba032 bash[24004]: cephadm 2021-03-10T16:36:30.794143+0000 mgr.gibba032.whopeo (mgr.24625) 8645 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.550955+0000 mgr.gibba032.whopeo (mgr.24625) 8630 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:36:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.552571+0000 mgr.gibba032.whopeo (mgr.24625) 8631 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:36:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.782743+0000 mgr.gibba032.whopeo (mgr.24625) 8632 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:36:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.784694+0000 mgr.gibba032.whopeo (mgr.24625) 8633 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:36:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.786064+0000 mgr.gibba032.whopeo (mgr.24625) 8634 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:36:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.786105+0000 mgr.gibba032.whopeo (mgr.24625) 8635 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:36:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.787755+0000 mgr.gibba032.whopeo (mgr.24625) 8636 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:36:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.787793+0000 mgr.gibba032.whopeo (mgr.24625) 8637 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:36:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.789093+0000 mgr.gibba032.whopeo (mgr.24625) 8638 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:36:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.790693+0000 mgr.gibba032.whopeo (mgr.24625) 8639 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:36:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.790733+0000 mgr.gibba032.whopeo (mgr.24625) 8640 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:36:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.792158+0000 mgr.gibba032.whopeo (mgr.24625) 8641 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:36:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.792542+0000 mgr.gibba032.whopeo (mgr.24625) 8642 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:36:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.793992+0000 mgr.gibba032.whopeo (mgr.24625) 8643 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:36:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.794033+0000 mgr.gibba032.whopeo (mgr.24625) 8644 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:36:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[30791]: cephadm 2021-03-10T16:36:30.794143+0000 mgr.gibba032.whopeo (mgr.24625) 8645 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.550955+0000 mgr.gibba032.whopeo (mgr.24625) 8630 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:36:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.552571+0000 mgr.gibba032.whopeo (mgr.24625) 8631 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:36:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.782743+0000 mgr.gibba032.whopeo (mgr.24625) 8632 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:36:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.784694+0000 mgr.gibba032.whopeo (mgr.24625) 8633 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:36:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.786064+0000 mgr.gibba032.whopeo (mgr.24625) 8634 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:36:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.786105+0000 mgr.gibba032.whopeo (mgr.24625) 8635 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:36:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.787755+0000 mgr.gibba032.whopeo (mgr.24625) 8636 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:36:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.787793+0000 mgr.gibba032.whopeo (mgr.24625) 8637 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:36:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.789093+0000 mgr.gibba032.whopeo (mgr.24625) 8638 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:36:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.790693+0000 mgr.gibba032.whopeo (mgr.24625) 8639 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:36:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.790733+0000 mgr.gibba032.whopeo (mgr.24625) 8640 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:36:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.792158+0000 mgr.gibba032.whopeo (mgr.24625) 8641 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:36:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.792542+0000 mgr.gibba032.whopeo (mgr.24625) 8642 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:36:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.793992+0000 mgr.gibba032.whopeo (mgr.24625) 8643 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:36:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.794033+0000 mgr.gibba032.whopeo (mgr.24625) 8644 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:36:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:31 gibba011 bash[31774]: cephadm 2021-03-10T16:36:30.794143+0000 mgr.gibba032.whopeo (mgr.24625) 8645 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:33.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:32 gibba032 bash[24004]: cluster 2021-03-10T16:36:31.623617+0000 mgr.gibba032.whopeo (mgr.24625) 8646 : cluster [DBG] pgmap v4919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:33.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:32 gibba011 bash[30791]: cluster 2021-03-10T16:36:31.623617+0000 mgr.gibba032.whopeo (mgr.24625) 8646 : cluster [DBG] pgmap v4919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:33.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:32 gibba011 bash[31774]: cluster 2021-03-10T16:36:31.623617+0000 mgr.gibba032.whopeo (mgr.24625) 8646 : cluster [DBG] pgmap v4919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:34 gibba032 bash[24004]: cluster 2021-03-10T16:36:33.625231+0000 mgr.gibba032.whopeo (mgr.24625) 8647 : cluster [DBG] pgmap v4920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:34 gibba011 bash[30791]: cluster 2021-03-10T16:36:33.625231+0000 mgr.gibba032.whopeo (mgr.24625) 8647 : cluster [DBG] pgmap v4920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:35.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:34 gibba011 bash[31774]: cluster 2021-03-10T16:36:33.625231+0000 mgr.gibba032.whopeo (mgr.24625) 8647 : cluster [DBG] pgmap v4920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:36 gibba032 bash[24004]: cluster 2021-03-10T16:36:35.626681+0000 mgr.gibba032.whopeo (mgr.24625) 8648 : cluster [DBG] pgmap v4921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:37.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:36 gibba011 bash[30791]: cluster 2021-03-10T16:36:35.626681+0000 mgr.gibba032.whopeo (mgr.24625) 8648 : cluster [DBG] pgmap v4921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:37.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:36 gibba011 bash[31774]: cluster 2021-03-10T16:36:35.626681+0000 mgr.gibba032.whopeo (mgr.24625) 8648 : cluster [DBG] pgmap v4921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:38 gibba032 bash[24004]: cluster 2021-03-10T16:36:37.628283+0000 mgr.gibba032.whopeo (mgr.24625) 8649 : cluster [DBG] pgmap v4922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:39.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:38 gibba011 bash[30791]: cluster 2021-03-10T16:36:37.628283+0000 mgr.gibba032.whopeo (mgr.24625) 8649 : cluster [DBG] pgmap v4922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:39.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:38 gibba011 bash[31774]: cluster 2021-03-10T16:36:37.628283+0000 mgr.gibba032.whopeo (mgr.24625) 8649 : cluster [DBG] pgmap v4922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:40 gibba032 bash[24004]: cluster 2021-03-10T16:36:39.629839+0000 mgr.gibba032.whopeo (mgr.24625) 8650 : cluster [DBG] pgmap v4923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:40 gibba011 bash[30791]: cluster 2021-03-10T16:36:39.629839+0000 mgr.gibba032.whopeo (mgr.24625) 8650 : cluster [DBG] pgmap v4923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:41.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:40 gibba011 bash[31774]: cluster 2021-03-10T16:36:39.629839+0000 mgr.gibba032.whopeo (mgr.24625) 8650 : cluster [DBG] pgmap v4923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:42 gibba032 bash[24004]: cluster 2021-03-10T16:36:41.631443+0000 mgr.gibba032.whopeo (mgr.24625) 8651 : cluster [DBG] pgmap v4924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:43.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:42 gibba011 bash[30791]: cluster 2021-03-10T16:36:41.631443+0000 mgr.gibba032.whopeo (mgr.24625) 8651 : cluster [DBG] pgmap v4924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:43.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:42 gibba011 bash[31774]: cluster 2021-03-10T16:36:41.631443+0000 mgr.gibba032.whopeo (mgr.24625) 8651 : cluster [DBG] pgmap v4924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:44.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:43 gibba032 bash[24004]: cluster 2021-03-10T16:36:43.633150+0000 mgr.gibba032.whopeo (mgr.24625) 8652 : cluster [DBG] pgmap v4925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:44.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:43 gibba011 bash[30791]: cluster 2021-03-10T16:36:43.633150+0000 mgr.gibba032.whopeo (mgr.24625) 8652 : cluster [DBG] pgmap v4925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:44.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:43 gibba011 bash[31774]: cluster 2021-03-10T16:36:43.633150+0000 mgr.gibba032.whopeo (mgr.24625) 8652 : cluster [DBG] pgmap v4925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:45.703 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:36:45.926 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:36:45.927 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:36:45.927 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:36:45.927 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:36:45.928 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:36:45.928 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:36:45.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:36:45.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:36:45.928 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:36:45.929 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:36:45.929 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:36:45.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:36:45.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:36:45.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:36:45.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:36:45.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:36:45.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:36:45.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:36:45.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:36:45.931 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:36:45.931 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:36:46.175 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:36:46.175 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:36:46.176 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:36:46.176 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:46.176 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:36:46.176 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:36:46.177 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:46.177 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:36:46.177 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:36:46.177 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:36:46.178 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:36:46.178 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:36:46.178 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:36:46.178 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:36:46.178 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:36:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[30791]: cluster 2021-03-10T16:36:45.634489+0000 mgr.gibba032.whopeo (mgr.24625) 8653 : cluster [DBG] pgmap v4926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[30791]: audit 2021-03-10T16:36:45.694897+0000 mgr.gibba032.whopeo (mgr.24625) 8654 : audit [DBG] from='client.59833 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[30791]: cephadm 2021-03-10T16:36:45.809830+0000 mgr.gibba032.whopeo (mgr.24625) 8655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:47.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[30791]: audit 2021-03-10T16:36:45.918992+0000 mgr.gibba032.whopeo (mgr.24625) 8656 : audit [DBG] from='client.64718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[30791]: audit 2021-03-10T16:36:46.177721+0000 mon.c (mon.1) 169 : audit [DBG] from='client.? 172.21.2.111:0/4145342885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:47.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[31774]: cluster 2021-03-10T16:36:45.634489+0000 mgr.gibba032.whopeo (mgr.24625) 8653 : cluster [DBG] pgmap v4926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:47.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[31774]: audit 2021-03-10T16:36:45.694897+0000 mgr.gibba032.whopeo (mgr.24625) 8654 : audit [DBG] from='client.59833 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[31774]: cephadm 2021-03-10T16:36:45.809830+0000 mgr.gibba032.whopeo (mgr.24625) 8655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[31774]: audit 2021-03-10T16:36:45.918992+0000 mgr.gibba032.whopeo (mgr.24625) 8656 : audit [DBG] from='client.64718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:46 gibba011 bash[31774]: audit 2021-03-10T16:36:46.177721+0000 mon.c (mon.1) 169 : audit [DBG] from='client.? 172.21.2.111:0/4145342885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:46 gibba032 bash[24004]: cluster 2021-03-10T16:36:45.634489+0000 mgr.gibba032.whopeo (mgr.24625) 8653 : cluster [DBG] pgmap v4926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:46 gibba032 bash[24004]: audit 2021-03-10T16:36:45.694897+0000 mgr.gibba032.whopeo (mgr.24625) 8654 : audit [DBG] from='client.59833 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:46 gibba032 bash[24004]: cephadm 2021-03-10T16:36:45.809830+0000 mgr.gibba032.whopeo (mgr.24625) 8655 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:36:47.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:46 gibba032 bash[24004]: audit 2021-03-10T16:36:45.918992+0000 mgr.gibba032.whopeo (mgr.24625) 8656 : audit [DBG] from='client.64718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:36:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:46 gibba032 bash[24004]: audit 2021-03-10T16:36:46.177721+0000 mon.c (mon.1) 169 : audit [DBG] from='client.? 172.21.2.111:0/4145342885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:36:49.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:48 gibba011 bash[30791]: cluster 2021-03-10T16:36:47.636323+0000 mgr.gibba032.whopeo (mgr.24625) 8657 : cluster [DBG] pgmap v4927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:49.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:48 gibba011 bash[31774]: cluster 2021-03-10T16:36:47.636323+0000 mgr.gibba032.whopeo (mgr.24625) 8657 : cluster [DBG] pgmap v4927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:48 gibba032 bash[24004]: cluster 2021-03-10T16:36:47.636323+0000 mgr.gibba032.whopeo (mgr.24625) 8657 : cluster [DBG] pgmap v4927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:51.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:50 gibba011 bash[30791]: cluster 2021-03-10T16:36:49.637629+0000 mgr.gibba032.whopeo (mgr.24625) 8658 : cluster [DBG] pgmap v4928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:51.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:50 gibba011 bash[31774]: cluster 2021-03-10T16:36:49.637629+0000 mgr.gibba032.whopeo (mgr.24625) 8658 : cluster [DBG] pgmap v4928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:50 gibba032 bash[24004]: cluster 2021-03-10T16:36:49.637629+0000 mgr.gibba032.whopeo (mgr.24625) 8658 : cluster [DBG] pgmap v4928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:53.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:52 gibba011 bash[30791]: cluster 2021-03-10T16:36:51.639402+0000 mgr.gibba032.whopeo (mgr.24625) 8659 : cluster [DBG] pgmap v4929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:53.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:52 gibba011 bash[31774]: cluster 2021-03-10T16:36:51.639402+0000 mgr.gibba032.whopeo (mgr.24625) 8659 : cluster [DBG] pgmap v4929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:52 gibba032 bash[24004]: cluster 2021-03-10T16:36:51.639402+0000 mgr.gibba032.whopeo (mgr.24625) 8659 : cluster [DBG] pgmap v4929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:55.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:54 gibba011 bash[30791]: cluster 2021-03-10T16:36:53.641048+0000 mgr.gibba032.whopeo (mgr.24625) 8660 : cluster [DBG] pgmap v4930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:55.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:54 gibba011 bash[31774]: cluster 2021-03-10T16:36:53.641048+0000 mgr.gibba032.whopeo (mgr.24625) 8660 : cluster [DBG] pgmap v4930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:54 gibba032 bash[24004]: cluster 2021-03-10T16:36:53.641048+0000 mgr.gibba032.whopeo (mgr.24625) 8660 : cluster [DBG] pgmap v4930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:57.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:56 gibba011 bash[30791]: cluster 2021-03-10T16:36:55.642641+0000 mgr.gibba032.whopeo (mgr.24625) 8661 : cluster [DBG] pgmap v4931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:57.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:56 gibba011 bash[31774]: cluster 2021-03-10T16:36:55.642641+0000 mgr.gibba032.whopeo (mgr.24625) 8661 : cluster [DBG] pgmap v4931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:56 gibba032 bash[24004]: cluster 2021-03-10T16:36:55.642641+0000 mgr.gibba032.whopeo (mgr.24625) 8661 : cluster [DBG] pgmap v4931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:58.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.101167+0000 mon.a (mon.0) 2990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:36:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.101816+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:36:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.212537+0000 mon.a (mon.0) 2992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:36:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.213350+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:36:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.214023+0000 mon.a (mon.0) 2994 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:36:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[30791]: audit 2021-03-10T16:36:57.214699+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:36:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.101167+0000 mon.a (mon.0) 2990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:36:58.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.101816+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:36:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.212537+0000 mon.a (mon.0) 2992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:36:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.213350+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:36:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.214023+0000 mon.a (mon.0) 2994 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:36:58.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:57 gibba011 bash[31774]: audit 2021-03-10T16:36:57.214699+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:36:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.101167+0000 mon.a (mon.0) 2990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:36:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.101816+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:36:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.212537+0000 mon.a (mon.0) 2992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:36:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.213350+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:36:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.214023+0000 mon.a (mon.0) 2994 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:36:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:57 gibba032 bash[24004]: audit 2021-03-10T16:36:57.214699+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:36:59.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:36:58 gibba011 bash[30791]: cluster 2021-03-10T16:36:57.644433+0000 mgr.gibba032.whopeo (mgr.24625) 8662 : cluster [DBG] pgmap v4932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:59.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:36:58 gibba011 bash[31774]: cluster 2021-03-10T16:36:57.644433+0000 mgr.gibba032.whopeo (mgr.24625) 8662 : cluster [DBG] pgmap v4932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:36:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:36:58 gibba032 bash[24004]: cluster 2021-03-10T16:36:57.644433+0000 mgr.gibba032.whopeo (mgr.24625) 8662 : cluster [DBG] pgmap v4932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:01.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:00 gibba011 bash[30791]: cluster 2021-03-10T16:36:59.645774+0000 mgr.gibba032.whopeo (mgr.24625) 8663 : cluster [DBG] pgmap v4933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:01.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:00 gibba011 bash[31774]: cluster 2021-03-10T16:36:59.645774+0000 mgr.gibba032.whopeo (mgr.24625) 8663 : cluster [DBG] pgmap v4933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:00 gibba032 bash[24004]: cluster 2021-03-10T16:36:59.645774+0000 mgr.gibba032.whopeo (mgr.24625) 8663 : cluster [DBG] pgmap v4933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:02.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:01 gibba032 bash[24004]: cephadm 2021-03-10T16:37:00.819528+0000 mgr.gibba032.whopeo (mgr.24625) 8664 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:02.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:01 gibba011 bash[30791]: cephadm 2021-03-10T16:37:00.819528+0000 mgr.gibba032.whopeo (mgr.24625) 8664 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:02.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:01 gibba011 bash[31774]: cephadm 2021-03-10T16:37:00.819528+0000 mgr.gibba032.whopeo (mgr.24625) 8664 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:02 gibba032 bash[24004]: cluster 2021-03-10T16:37:01.647494+0000 mgr.gibba032.whopeo (mgr.24625) 8665 : cluster [DBG] pgmap v4934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:02 gibba011 bash[30791]: cluster 2021-03-10T16:37:01.647494+0000 mgr.gibba032.whopeo (mgr.24625) 8665 : cluster [DBG] pgmap v4934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:02 gibba011 bash[31774]: cluster 2021-03-10T16:37:01.647494+0000 mgr.gibba032.whopeo (mgr.24625) 8665 : cluster [DBG] pgmap v4934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:04 gibba032 bash[24004]: cluster 2021-03-10T16:37:03.649089+0000 mgr.gibba032.whopeo (mgr.24625) 8666 : cluster [DBG] pgmap v4935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:04 gibba011 bash[30791]: cluster 2021-03-10T16:37:03.649089+0000 mgr.gibba032.whopeo (mgr.24625) 8666 : cluster [DBG] pgmap v4935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:04 gibba011 bash[31774]: cluster 2021-03-10T16:37:03.649089+0000 mgr.gibba032.whopeo (mgr.24625) 8666 : cluster [DBG] pgmap v4935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:06 gibba032 bash[24004]: cluster 2021-03-10T16:37:05.650453+0000 mgr.gibba032.whopeo (mgr.24625) 8667 : cluster [DBG] pgmap v4936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:07.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:06 gibba011 bash[30791]: cluster 2021-03-10T16:37:05.650453+0000 mgr.gibba032.whopeo (mgr.24625) 8667 : cluster [DBG] pgmap v4936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:06 gibba011 bash[31774]: cluster 2021-03-10T16:37:05.650453+0000 mgr.gibba032.whopeo (mgr.24625) 8667 : cluster [DBG] pgmap v4936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:08 gibba032 bash[24004]: cluster 2021-03-10T16:37:07.652178+0000 mgr.gibba032.whopeo (mgr.24625) 8668 : cluster [DBG] pgmap v4937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:08 gibba011 bash[30791]: cluster 2021-03-10T16:37:07.652178+0000 mgr.gibba032.whopeo (mgr.24625) 8668 : cluster [DBG] pgmap v4937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:08 gibba011 bash[31774]: cluster 2021-03-10T16:37:07.652178+0000 mgr.gibba032.whopeo (mgr.24625) 8668 : cluster [DBG] pgmap v4937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:10 gibba032 bash[24004]: cluster 2021-03-10T16:37:09.653671+0000 mgr.gibba032.whopeo (mgr.24625) 8669 : cluster [DBG] pgmap v4938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:10 gibba011 bash[30791]: cluster 2021-03-10T16:37:09.653671+0000 mgr.gibba032.whopeo (mgr.24625) 8669 : cluster [DBG] pgmap v4938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:10 gibba011 bash[31774]: cluster 2021-03-10T16:37:09.653671+0000 mgr.gibba032.whopeo (mgr.24625) 8669 : cluster [DBG] pgmap v4938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:12 gibba032 bash[24004]: cluster 2021-03-10T16:37:11.655417+0000 mgr.gibba032.whopeo (mgr.24625) 8670 : cluster [DBG] pgmap v4939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:12 gibba011 bash[30791]: cluster 2021-03-10T16:37:11.655417+0000 mgr.gibba032.whopeo (mgr.24625) 8670 : cluster [DBG] pgmap v4939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:12 gibba011 bash[31774]: cluster 2021-03-10T16:37:11.655417+0000 mgr.gibba032.whopeo (mgr.24625) 8670 : cluster [DBG] pgmap v4939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:14 gibba032 bash[24004]: cluster 2021-03-10T16:37:13.657066+0000 mgr.gibba032.whopeo (mgr.24625) 8671 : cluster [DBG] pgmap v4940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:14 gibba011 bash[30791]: cluster 2021-03-10T16:37:13.657066+0000 mgr.gibba032.whopeo (mgr.24625) 8671 : cluster [DBG] pgmap v4940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:14 gibba011 bash[31774]: cluster 2021-03-10T16:37:13.657066+0000 mgr.gibba032.whopeo (mgr.24625) 8671 : cluster [DBG] pgmap v4940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:16.420 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:37:16.646 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:37:16.647 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:37:16.647 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:37:16.647 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:37:16.647 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:37:16.648 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:37:16.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:37:16.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:37:16.648 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:37:16.649 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:37:16.649 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:37:16.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:37:16.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:37:16.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:37:16.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:37:16.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:37:16.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:37:16.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:37:16.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:37:16.653 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:37:16.653 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:37:16.898 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:37:16.898 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:37:16.898 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:37:16.898 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:16.899 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:37:16.899 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:37:16.899 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:16.899 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:37:16.899 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:37:16.900 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:16.900 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:37:16.900 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:37:16.900 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:37:16.901 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:37:16.901 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:37:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:16 gibba032 bash[24004]: cluster 2021-03-10T16:37:15.657555+0000 mgr.gibba032.whopeo (mgr.24625) 8672 : cluster [DBG] pgmap v4941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:17.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:16 gibba032 bash[24004]: cephadm 2021-03-10T16:37:15.835331+0000 mgr.gibba032.whopeo (mgr.24625) 8673 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:16 gibba011 bash[30791]: cluster 2021-03-10T16:37:15.657555+0000 mgr.gibba032.whopeo (mgr.24625) 8672 : cluster [DBG] pgmap v4941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:17.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:16 gibba011 bash[30791]: cephadm 2021-03-10T16:37:15.835331+0000 mgr.gibba032.whopeo (mgr.24625) 8673 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:16 gibba011 bash[31774]: cluster 2021-03-10T16:37:15.657555+0000 mgr.gibba032.whopeo (mgr.24625) 8672 : cluster [DBG] pgmap v4941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:16 gibba011 bash[31774]: cephadm 2021-03-10T16:37:15.835331+0000 mgr.gibba032.whopeo (mgr.24625) 8673 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:18.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:17 gibba032 bash[24004]: audit 2021-03-10T16:37:16.411978+0000 mgr.gibba032.whopeo (mgr.24625) 8674 : audit [DBG] from='client.49869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:17 gibba032 bash[24004]: audit 2021-03-10T16:37:16.635275+0000 mgr.gibba032.whopeo (mgr.24625) 8675 : audit [DBG] from='client.49875 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:17 gibba032 bash[24004]: audit 2021-03-10T16:37:16.900431+0000 mon.a (mon.0) 2996 : audit [DBG] from='client.? 172.21.2.111:0/604197954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:18.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[30791]: audit 2021-03-10T16:37:16.411978+0000 mgr.gibba032.whopeo (mgr.24625) 8674 : audit [DBG] from='client.49869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[30791]: audit 2021-03-10T16:37:16.635275+0000 mgr.gibba032.whopeo (mgr.24625) 8675 : audit [DBG] from='client.49875 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[30791]: audit 2021-03-10T16:37:16.900431+0000 mon.a (mon.0) 2996 : audit [DBG] from='client.? 172.21.2.111:0/604197954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:18.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[31774]: audit 2021-03-10T16:37:16.411978+0000 mgr.gibba032.whopeo (mgr.24625) 8674 : audit [DBG] from='client.49869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[31774]: audit 2021-03-10T16:37:16.635275+0000 mgr.gibba032.whopeo (mgr.24625) 8675 : audit [DBG] from='client.49875 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:17 gibba011 bash[31774]: audit 2021-03-10T16:37:16.900431+0000 mon.a (mon.0) 2996 : audit [DBG] from='client.? 172.21.2.111:0/604197954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:18 gibba032 bash[24004]: cluster 2021-03-10T16:37:17.658960+0000 mgr.gibba032.whopeo (mgr.24625) 8676 : cluster [DBG] pgmap v4942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:18 gibba011 bash[30791]: cluster 2021-03-10T16:37:17.658960+0000 mgr.gibba032.whopeo (mgr.24625) 8676 : cluster [DBG] pgmap v4942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:19.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:18 gibba011 bash[31774]: cluster 2021-03-10T16:37:17.658960+0000 mgr.gibba032.whopeo (mgr.24625) 8676 : cluster [DBG] pgmap v4942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:20 gibba032 bash[24004]: cluster 2021-03-10T16:37:19.660547+0000 mgr.gibba032.whopeo (mgr.24625) 8677 : cluster [DBG] pgmap v4943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:20 gibba011 bash[30791]: cluster 2021-03-10T16:37:19.660547+0000 mgr.gibba032.whopeo (mgr.24625) 8677 : cluster [DBG] pgmap v4943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:20 gibba011 bash[31774]: cluster 2021-03-10T16:37:19.660547+0000 mgr.gibba032.whopeo (mgr.24625) 8677 : cluster [DBG] pgmap v4943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:22 gibba032 bash[24004]: cluster 2021-03-10T16:37:21.662286+0000 mgr.gibba032.whopeo (mgr.24625) 8678 : cluster [DBG] pgmap v4944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:22 gibba011 bash[30791]: cluster 2021-03-10T16:37:21.662286+0000 mgr.gibba032.whopeo (mgr.24625) 8678 : cluster [DBG] pgmap v4944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:22 gibba011 bash[31774]: cluster 2021-03-10T16:37:21.662286+0000 mgr.gibba032.whopeo (mgr.24625) 8678 : cluster [DBG] pgmap v4944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:24 gibba032 bash[24004]: cluster 2021-03-10T16:37:23.663863+0000 mgr.gibba032.whopeo (mgr.24625) 8679 : cluster [DBG] pgmap v4945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:24 gibba011 bash[30791]: cluster 2021-03-10T16:37:23.663863+0000 mgr.gibba032.whopeo (mgr.24625) 8679 : cluster [DBG] pgmap v4945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:25.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:24 gibba011 bash[31774]: cluster 2021-03-10T16:37:23.663863+0000 mgr.gibba032.whopeo (mgr.24625) 8679 : cluster [DBG] pgmap v4945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:26 gibba032 bash[24004]: cluster 2021-03-10T16:37:25.664436+0000 mgr.gibba032.whopeo (mgr.24625) 8680 : cluster [DBG] pgmap v4946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:26 gibba011 bash[30791]: cluster 2021-03-10T16:37:25.664436+0000 mgr.gibba032.whopeo (mgr.24625) 8680 : cluster [DBG] pgmap v4946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:26 gibba011 bash[31774]: cluster 2021-03-10T16:37:25.664436+0000 mgr.gibba032.whopeo (mgr.24625) 8680 : cluster [DBG] pgmap v4946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:28 gibba032 bash[24004]: cluster 2021-03-10T16:37:27.666095+0000 mgr.gibba032.whopeo (mgr.24625) 8681 : cluster [DBG] pgmap v4947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:28 gibba011 bash[30791]: cluster 2021-03-10T16:37:27.666095+0000 mgr.gibba032.whopeo (mgr.24625) 8681 : cluster [DBG] pgmap v4947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:28 gibba011 bash[31774]: cluster 2021-03-10T16:37:27.666095+0000 mgr.gibba032.whopeo (mgr.24625) 8681 : cluster [DBG] pgmap v4947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:31.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:30 gibba011 bash[30791]: cluster 2021-03-10T16:37:29.667592+0000 mgr.gibba032.whopeo (mgr.24625) 8682 : cluster [DBG] pgmap v4948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:31.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:30 gibba011 bash[31774]: cluster 2021-03-10T16:37:29.667592+0000 mgr.gibba032.whopeo (mgr.24625) 8682 : cluster [DBG] pgmap v4948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:31.254 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:30 gibba032 bash[24004]: cluster 2021-03-10T16:37:29.667592+0000 mgr.gibba032.whopeo (mgr.24625) 8682 : cluster [DBG] pgmap v4948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.041422+0000 mgr.gibba032.whopeo (mgr.24625) 8683 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:37:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.042787+0000 mon.a (mon.0) 2997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.044273+0000 mgr.gibba032.whopeo (mgr.24625) 8684 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:37:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.045440+0000 mon.a (mon.0) 2998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.251043+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.251493+0000 mgr.gibba032.whopeo (mgr.24625) 8685 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:37:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.253051+0000 mgr.gibba032.whopeo (mgr.24625) 8686 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:37:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.254128+0000 mgr.gibba032.whopeo (mgr.24625) 8687 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:37:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.254184+0000 mon.a (mon.0) 3000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:37:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.254193+0000 mgr.gibba032.whopeo (mgr.24625) 8688 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:37:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.255158+0000 mgr.gibba032.whopeo (mgr.24625) 8689 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:37:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.255219+0000 mgr.gibba032.whopeo (mgr.24625) 8690 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:37:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.255347+0000 mon.a (mon.0) 3001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.256089+0000 mgr.gibba032.whopeo (mgr.24625) 8691 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:37:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.256510+0000 mon.a (mon.0) 3002 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.256904+0000 mgr.gibba032.whopeo (mgr.24625) 8692 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:37:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.256994+0000 mgr.gibba032.whopeo (mgr.24625) 8693 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:37:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.257435+0000 mon.a (mon.0) 3003 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.257980+0000 mgr.gibba032.whopeo (mgr.24625) 8694 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:37:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.258337+0000 mgr.gibba032.whopeo (mgr.24625) 8695 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:37:32.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.259161+0000 mgr.gibba032.whopeo (mgr.24625) 8696 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:37:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.041422+0000 mgr.gibba032.whopeo (mgr.24625) 8683 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:37:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.042787+0000 mon.a (mon.0) 2997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.044273+0000 mgr.gibba032.whopeo (mgr.24625) 8684 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.045440+0000 mon.a (mon.0) 2998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.251043+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.251493+0000 mgr.gibba032.whopeo (mgr.24625) 8685 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:37:32.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.253051+0000 mgr.gibba032.whopeo (mgr.24625) 8686 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:37:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.254128+0000 mgr.gibba032.whopeo (mgr.24625) 8687 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:37:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.254184+0000 mon.a (mon.0) 3000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:37:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.254193+0000 mgr.gibba032.whopeo (mgr.24625) 8688 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:37:32.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.255158+0000 mgr.gibba032.whopeo (mgr.24625) 8689 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:37:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.255219+0000 mgr.gibba032.whopeo (mgr.24625) 8690 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:37:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.255347+0000 mon.a (mon.0) 3001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.256089+0000 mgr.gibba032.whopeo (mgr.24625) 8691 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:37:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.256510+0000 mon.a (mon.0) 3002 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.256904+0000 mgr.gibba032.whopeo (mgr.24625) 8692 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:37:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.256994+0000 mgr.gibba032.whopeo (mgr.24625) 8693 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:37:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.257435+0000 mon.a (mon.0) 3003 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.257980+0000 mgr.gibba032.whopeo (mgr.24625) 8694 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:37:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.258337+0000 mgr.gibba032.whopeo (mgr.24625) 8695 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:37:32.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.259161+0000 mgr.gibba032.whopeo (mgr.24625) 8696 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:37:32.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.259230+0000 mgr.gibba032.whopeo (mgr.24625) 8697 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:37:32.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.259294+0000 mon.a (mon.0) 3004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cephadm 2021-03-10T16:37:31.259384+0000 mgr.gibba032.whopeo (mgr.24625) 8698 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:32.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: audit 2021-03-10T16:37:31.260517+0000 mon.a (mon.0) 3005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[30791]: cluster 2021-03-10T16:37:31.669317+0000 mgr.gibba032.whopeo (mgr.24625) 8699 : cluster [DBG] pgmap v4949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.259230+0000 mgr.gibba032.whopeo (mgr.24625) 8697 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:37:32.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.259294+0000 mon.a (mon.0) 3004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cephadm 2021-03-10T16:37:31.259384+0000 mgr.gibba032.whopeo (mgr.24625) 8698 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:32.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: audit 2021-03-10T16:37:31.260517+0000 mon.a (mon.0) 3005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:32 gibba011 bash[31774]: cluster 2021-03-10T16:37:31.669317+0000 mgr.gibba032.whopeo (mgr.24625) 8699 : cluster [DBG] pgmap v4949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.041422+0000 mgr.gibba032.whopeo (mgr.24625) 8683 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:37:32.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.042787+0000 mon.a (mon.0) 2997 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.044273+0000 mgr.gibba032.whopeo (mgr.24625) 8684 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:37:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.045440+0000 mon.a (mon.0) 2998 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.251043+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:37:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.251493+0000 mgr.gibba032.whopeo (mgr.24625) 8685 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:37:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.253051+0000 mgr.gibba032.whopeo (mgr.24625) 8686 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:37:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.254128+0000 mgr.gibba032.whopeo (mgr.24625) 8687 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:37:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.254184+0000 mon.a (mon.0) 3000 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:37:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.254193+0000 mgr.gibba032.whopeo (mgr.24625) 8688 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:37:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.255158+0000 mgr.gibba032.whopeo (mgr.24625) 8689 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:37:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.255219+0000 mgr.gibba032.whopeo (mgr.24625) 8690 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:37:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.255347+0000 mon.a (mon.0) 3001 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.256089+0000 mgr.gibba032.whopeo (mgr.24625) 8691 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.256510+0000 mon.a (mon.0) 3002 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.256904+0000 mgr.gibba032.whopeo (mgr.24625) 8692 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.256994+0000 mgr.gibba032.whopeo (mgr.24625) 8693 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.257435+0000 mon.a (mon.0) 3003 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.257980+0000 mgr.gibba032.whopeo (mgr.24625) 8694 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:37:32.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.258337+0000 mgr.gibba032.whopeo (mgr.24625) 8695 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:37:32.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.259161+0000 mgr.gibba032.whopeo (mgr.24625) 8696 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:37:32.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.259230+0000 mgr.gibba032.whopeo (mgr.24625) 8697 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:37:32.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.259294+0000 mon.a (mon.0) 3004 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cephadm 2021-03-10T16:37:31.259384+0000 mgr.gibba032.whopeo (mgr.24625) 8698 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:32.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: audit 2021-03-10T16:37:31.260517+0000 mon.a (mon.0) 3005 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:32.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:32 gibba032 bash[24004]: cluster 2021-03-10T16:37:31.669317+0000 mgr.gibba032.whopeo (mgr.24625) 8699 : cluster [DBG] pgmap v4949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:35.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:34 gibba032 bash[24004]: cluster 2021-03-10T16:37:33.670913+0000 mgr.gibba032.whopeo (mgr.24625) 8700 : cluster [DBG] pgmap v4950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:35.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:34 gibba011 bash[30791]: cluster 2021-03-10T16:37:33.670913+0000 mgr.gibba032.whopeo (mgr.24625) 8700 : cluster [DBG] pgmap v4950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:34 gibba011 bash[31774]: cluster 2021-03-10T16:37:33.670913+0000 mgr.gibba032.whopeo (mgr.24625) 8700 : cluster [DBG] pgmap v4950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:37.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:36 gibba011 bash[30791]: cluster 2021-03-10T16:37:35.671463+0000 mgr.gibba032.whopeo (mgr.24625) 8701 : cluster [DBG] pgmap v4951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:37.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:36 gibba011 bash[31774]: cluster 2021-03-10T16:37:35.671463+0000 mgr.gibba032.whopeo (mgr.24625) 8701 : cluster [DBG] pgmap v4951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:36 gibba032 bash[24004]: cluster 2021-03-10T16:37:35.671463+0000 mgr.gibba032.whopeo (mgr.24625) 8701 : cluster [DBG] pgmap v4951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:39.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:38 gibba011 bash[30791]: cluster 2021-03-10T16:37:37.673072+0000 mgr.gibba032.whopeo (mgr.24625) 8702 : cluster [DBG] pgmap v4952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:38 gibba011 bash[31774]: cluster 2021-03-10T16:37:37.673072+0000 mgr.gibba032.whopeo (mgr.24625) 8702 : cluster [DBG] pgmap v4952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:38 gibba032 bash[24004]: cluster 2021-03-10T16:37:37.673072+0000 mgr.gibba032.whopeo (mgr.24625) 8702 : cluster [DBG] pgmap v4952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:40 gibba011 bash[30791]: cluster 2021-03-10T16:37:39.674568+0000 mgr.gibba032.whopeo (mgr.24625) 8703 : cluster [DBG] pgmap v4953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:40 gibba011 bash[31774]: cluster 2021-03-10T16:37:39.674568+0000 mgr.gibba032.whopeo (mgr.24625) 8703 : cluster [DBG] pgmap v4953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:40 gibba032 bash[24004]: cluster 2021-03-10T16:37:39.674568+0000 mgr.gibba032.whopeo (mgr.24625) 8703 : cluster [DBG] pgmap v4953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:43.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:42 gibba011 bash[30791]: cluster 2021-03-10T16:37:41.676165+0000 mgr.gibba032.whopeo (mgr.24625) 8704 : cluster [DBG] pgmap v4954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:43.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:42 gibba011 bash[31774]: cluster 2021-03-10T16:37:41.676165+0000 mgr.gibba032.whopeo (mgr.24625) 8704 : cluster [DBG] pgmap v4954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:42 gibba032 bash[24004]: cluster 2021-03-10T16:37:41.676165+0000 mgr.gibba032.whopeo (mgr.24625) 8704 : cluster [DBG] pgmap v4954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:45.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:44 gibba011 bash[30791]: cluster 2021-03-10T16:37:43.677835+0000 mgr.gibba032.whopeo (mgr.24625) 8705 : cluster [DBG] pgmap v4955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:45.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:44 gibba011 bash[31774]: cluster 2021-03-10T16:37:43.677835+0000 mgr.gibba032.whopeo (mgr.24625) 8705 : cluster [DBG] pgmap v4955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:44 gibba032 bash[24004]: cluster 2021-03-10T16:37:43.677835+0000 mgr.gibba032.whopeo (mgr.24625) 8705 : cluster [DBG] pgmap v4955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:47.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:46 gibba011 bash[30791]: cluster 2021-03-10T16:37:45.679155+0000 mgr.gibba032.whopeo (mgr.24625) 8706 : cluster [DBG] pgmap v4956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:47.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:46 gibba011 bash[31774]: cluster 2021-03-10T16:37:45.679155+0000 mgr.gibba032.whopeo (mgr.24625) 8706 : cluster [DBG] pgmap v4956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:47.183 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:37:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:46 gibba032 bash[24004]: cluster 2021-03-10T16:37:45.679155+0000 mgr.gibba032.whopeo (mgr.24625) 8706 : cluster [DBG] pgmap v4956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:47.408 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:37:47.409 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:37:47.409 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:37:47.410 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:37:47.410 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:37:47.410 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:37:47.411 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:37:47.411 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:37:47.412 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:37:47.412 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:37:47.413 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:37:47.413 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:37:47.413 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:37:47.414 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:37:47.414 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:37:47.415 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:37:47.415 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:37:47.415 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:37:47.416 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:37:47.416 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:37:47.417 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:37:47.655 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:37:47.655 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:37:47.655 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:37:47.656 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:47.656 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:37:47.656 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:37:47.656 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:47.657 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:37:47.657 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:37:47.657 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:37:47.657 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:37:47.657 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:37:47.658 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:37:47.658 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:37:47.658 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:37:48.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:47 gibba032 bash[24004]: cephadm 2021-03-10T16:37:46.275345+0000 mgr.gibba032.whopeo (mgr.24625) 8707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:48.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:47 gibba032 bash[24004]: audit 2021-03-10T16:37:47.657510+0000 mon.c (mon.1) 170 : audit [DBG] from='client.? 172.21.2.111:0/2659494576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:48.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:47 gibba011 bash[30791]: cephadm 2021-03-10T16:37:46.275345+0000 mgr.gibba032.whopeo (mgr.24625) 8707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:48.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:47 gibba011 bash[30791]: audit 2021-03-10T16:37:47.657510+0000 mon.c (mon.1) 170 : audit [DBG] from='client.? 172.21.2.111:0/2659494576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:48.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:47 gibba011 bash[31774]: cephadm 2021-03-10T16:37:46.275345+0000 mgr.gibba032.whopeo (mgr.24625) 8707 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:37:48.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:47 gibba011 bash[31774]: audit 2021-03-10T16:37:47.657510+0000 mon.c (mon.1) 170 : audit [DBG] from='client.? 172.21.2.111:0/2659494576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:37:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:48 gibba032 bash[24004]: audit 2021-03-10T16:37:47.175401+0000 mgr.gibba032.whopeo (mgr.24625) 8708 : audit [DBG] from='client.59869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:48 gibba032 bash[24004]: audit 2021-03-10T16:37:47.399700+0000 mgr.gibba032.whopeo (mgr.24625) 8709 : audit [DBG] from='client.49893 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:48 gibba032 bash[24004]: cluster 2021-03-10T16:37:47.680690+0000 mgr.gibba032.whopeo (mgr.24625) 8710 : cluster [DBG] pgmap v4957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[30791]: audit 2021-03-10T16:37:47.175401+0000 mgr.gibba032.whopeo (mgr.24625) 8708 : audit [DBG] from='client.59869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[30791]: audit 2021-03-10T16:37:47.399700+0000 mgr.gibba032.whopeo (mgr.24625) 8709 : audit [DBG] from='client.49893 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[30791]: cluster 2021-03-10T16:37:47.680690+0000 mgr.gibba032.whopeo (mgr.24625) 8710 : cluster [DBG] pgmap v4957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[31774]: audit 2021-03-10T16:37:47.175401+0000 mgr.gibba032.whopeo (mgr.24625) 8708 : audit [DBG] from='client.59869 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[31774]: audit 2021-03-10T16:37:47.399700+0000 mgr.gibba032.whopeo (mgr.24625) 8709 : audit [DBG] from='client.49893 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:37:49.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:48 gibba011 bash[31774]: cluster 2021-03-10T16:37:47.680690+0000 mgr.gibba032.whopeo (mgr.24625) 8710 : cluster [DBG] pgmap v4957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:50 gibba032 bash[24004]: cluster 2021-03-10T16:37:49.682157+0000 mgr.gibba032.whopeo (mgr.24625) 8711 : cluster [DBG] pgmap v4958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:50 gibba011 bash[30791]: cluster 2021-03-10T16:37:49.682157+0000 mgr.gibba032.whopeo (mgr.24625) 8711 : cluster [DBG] pgmap v4958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:50 gibba011 bash[31774]: cluster 2021-03-10T16:37:49.682157+0000 mgr.gibba032.whopeo (mgr.24625) 8711 : cluster [DBG] pgmap v4958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:52 gibba032 bash[24004]: cluster 2021-03-10T16:37:51.683920+0000 mgr.gibba032.whopeo (mgr.24625) 8712 : cluster [DBG] pgmap v4959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:52 gibba011 bash[30791]: cluster 2021-03-10T16:37:51.683920+0000 mgr.gibba032.whopeo (mgr.24625) 8712 : cluster [DBG] pgmap v4959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:52 gibba011 bash[31774]: cluster 2021-03-10T16:37:51.683920+0000 mgr.gibba032.whopeo (mgr.24625) 8712 : cluster [DBG] pgmap v4959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:54 gibba032 bash[24004]: cluster 2021-03-10T16:37:53.685324+0000 mgr.gibba032.whopeo (mgr.24625) 8713 : cluster [DBG] pgmap v4960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:54 gibba011 bash[30791]: cluster 2021-03-10T16:37:53.685324+0000 mgr.gibba032.whopeo (mgr.24625) 8713 : cluster [DBG] pgmap v4960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:54 gibba011 bash[31774]: cluster 2021-03-10T16:37:53.685324+0000 mgr.gibba032.whopeo (mgr.24625) 8713 : cluster [DBG] pgmap v4960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:56 gibba032 bash[24004]: cluster 2021-03-10T16:37:55.685900+0000 mgr.gibba032.whopeo (mgr.24625) 8714 : cluster [DBG] pgmap v4961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:56 gibba011 bash[30791]: cluster 2021-03-10T16:37:55.685900+0000 mgr.gibba032.whopeo (mgr.24625) 8714 : cluster [DBG] pgmap v4961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:56 gibba011 bash[31774]: cluster 2021-03-10T16:37:55.685900+0000 mgr.gibba032.whopeo (mgr.24625) 8714 : cluster [DBG] pgmap v4961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:58.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.108232+0000 mon.a (mon.0) 3006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:37:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.108401+0000 mon.a (mon.0) 3007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:37:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.221492+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:37:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.222296+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:37:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.223011+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:37:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:57 gibba032 bash[24004]: audit 2021-03-10T16:37:57.223785+0000 mon.a (mon.0) 3011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:37:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.108232+0000 mon.a (mon.0) 3006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:37:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.108401+0000 mon.a (mon.0) 3007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:37:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.221492+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:37:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.222296+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:37:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.223011+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:37:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[30791]: audit 2021-03-10T16:37:57.223785+0000 mon.a (mon.0) 3011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:37:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.108232+0000 mon.a (mon.0) 3006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:37:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.108401+0000 mon.a (mon.0) 3007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:37:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.221492+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:37:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.222296+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:37:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.223011+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:37:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:57 gibba011 bash[31774]: audit 2021-03-10T16:37:57.223785+0000 mon.a (mon.0) 3011 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:37:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:37:58 gibba032 bash[24004]: cluster 2021-03-10T16:37:57.687532+0000 mgr.gibba032.whopeo (mgr.24625) 8715 : cluster [DBG] pgmap v4962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:59.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:37:58 gibba011 bash[30791]: cluster 2021-03-10T16:37:57.687532+0000 mgr.gibba032.whopeo (mgr.24625) 8715 : cluster [DBG] pgmap v4962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:37:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:37:58 gibba011 bash[31774]: cluster 2021-03-10T16:37:57.687532+0000 mgr.gibba032.whopeo (mgr.24625) 8715 : cluster [DBG] pgmap v4962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:00 gibba032 bash[24004]: cluster 2021-03-10T16:37:59.689006+0000 mgr.gibba032.whopeo (mgr.24625) 8716 : cluster [DBG] pgmap v4963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:01.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:00 gibba011 bash[30791]: cluster 2021-03-10T16:37:59.689006+0000 mgr.gibba032.whopeo (mgr.24625) 8716 : cluster [DBG] pgmap v4963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:00 gibba011 bash[31774]: cluster 2021-03-10T16:37:59.689006+0000 mgr.gibba032.whopeo (mgr.24625) 8716 : cluster [DBG] pgmap v4963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:02 gibba032 bash[24004]: cephadm 2021-03-10T16:38:01.291283+0000 mgr.gibba032.whopeo (mgr.24625) 8717 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:03.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:02 gibba032 bash[24004]: cluster 2021-03-10T16:38:01.690711+0000 mgr.gibba032.whopeo (mgr.24625) 8718 : cluster [DBG] pgmap v4964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:02 gibba011 bash[30791]: cephadm 2021-03-10T16:38:01.291283+0000 mgr.gibba032.whopeo (mgr.24625) 8717 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:03.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:02 gibba011 bash[30791]: cluster 2021-03-10T16:38:01.690711+0000 mgr.gibba032.whopeo (mgr.24625) 8718 : cluster [DBG] pgmap v4964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:02 gibba011 bash[31774]: cephadm 2021-03-10T16:38:01.291283+0000 mgr.gibba032.whopeo (mgr.24625) 8717 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:02 gibba011 bash[31774]: cluster 2021-03-10T16:38:01.690711+0000 mgr.gibba032.whopeo (mgr.24625) 8718 : cluster [DBG] pgmap v4964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:04 gibba032 bash[24004]: cluster 2021-03-10T16:38:03.692300+0000 mgr.gibba032.whopeo (mgr.24625) 8719 : cluster [DBG] pgmap v4965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:04 gibba011 bash[30791]: cluster 2021-03-10T16:38:03.692300+0000 mgr.gibba032.whopeo (mgr.24625) 8719 : cluster [DBG] pgmap v4965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:04 gibba011 bash[31774]: cluster 2021-03-10T16:38:03.692300+0000 mgr.gibba032.whopeo (mgr.24625) 8719 : cluster [DBG] pgmap v4965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:06 gibba032 bash[24004]: cluster 2021-03-10T16:38:05.693589+0000 mgr.gibba032.whopeo (mgr.24625) 8720 : cluster [DBG] pgmap v4966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:07.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:06 gibba011 bash[30791]: cluster 2021-03-10T16:38:05.693589+0000 mgr.gibba032.whopeo (mgr.24625) 8720 : cluster [DBG] pgmap v4966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:06 gibba011 bash[31774]: cluster 2021-03-10T16:38:05.693589+0000 mgr.gibba032.whopeo (mgr.24625) 8720 : cluster [DBG] pgmap v4966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:08 gibba032 bash[24004]: cluster 2021-03-10T16:38:07.695332+0000 mgr.gibba032.whopeo (mgr.24625) 8721 : cluster [DBG] pgmap v4967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:08 gibba011 bash[30791]: cluster 2021-03-10T16:38:07.695332+0000 mgr.gibba032.whopeo (mgr.24625) 8721 : cluster [DBG] pgmap v4967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:08 gibba011 bash[31774]: cluster 2021-03-10T16:38:07.695332+0000 mgr.gibba032.whopeo (mgr.24625) 8721 : cluster [DBG] pgmap v4967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:10 gibba032 bash[24004]: cluster 2021-03-10T16:38:09.696865+0000 mgr.gibba032.whopeo (mgr.24625) 8722 : cluster [DBG] pgmap v4968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:10 gibba011 bash[30791]: cluster 2021-03-10T16:38:09.696865+0000 mgr.gibba032.whopeo (mgr.24625) 8722 : cluster [DBG] pgmap v4968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:10 gibba011 bash[31774]: cluster 2021-03-10T16:38:09.696865+0000 mgr.gibba032.whopeo (mgr.24625) 8722 : cluster [DBG] pgmap v4968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:12 gibba032 bash[24004]: cluster 2021-03-10T16:38:11.698564+0000 mgr.gibba032.whopeo (mgr.24625) 8723 : cluster [DBG] pgmap v4969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:12 gibba011 bash[30791]: cluster 2021-03-10T16:38:11.698564+0000 mgr.gibba032.whopeo (mgr.24625) 8723 : cluster [DBG] pgmap v4969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:12 gibba011 bash[31774]: cluster 2021-03-10T16:38:11.698564+0000 mgr.gibba032.whopeo (mgr.24625) 8723 : cluster [DBG] pgmap v4969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:14 gibba032 bash[24004]: cluster 2021-03-10T16:38:13.700064+0000 mgr.gibba032.whopeo (mgr.24625) 8724 : cluster [DBG] pgmap v4970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:14 gibba011 bash[30791]: cluster 2021-03-10T16:38:13.700064+0000 mgr.gibba032.whopeo (mgr.24625) 8724 : cluster [DBG] pgmap v4970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:14 gibba011 bash[31774]: cluster 2021-03-10T16:38:13.700064+0000 mgr.gibba032.whopeo (mgr.24625) 8724 : cluster [DBG] pgmap v4970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:16.298 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:16 gibba032 bash[24004]: cluster 2021-03-10T16:38:15.700722+0000 mgr.gibba032.whopeo (mgr.24625) 8725 : cluster [DBG] pgmap v4971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:16.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:16 gibba011 bash[30791]: cluster 2021-03-10T16:38:15.700722+0000 mgr.gibba032.whopeo (mgr.24625) 8725 : cluster [DBG] pgmap v4971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:16.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:16 gibba011 bash[31774]: cluster 2021-03-10T16:38:15.700722+0000 mgr.gibba032.whopeo (mgr.24625) 8725 : cluster [DBG] pgmap v4971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:17 gibba011 bash[30791]: cephadm 2021-03-10T16:38:16.299001+0000 mgr.gibba032.whopeo (mgr.24625) 8726 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:17 gibba011 bash[31774]: cephadm 2021-03-10T16:38:16.299001+0000 mgr.gibba032.whopeo (mgr.24625) 8726 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:17.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:17 gibba032 bash[24004]: cephadm 2021-03-10T16:38:16.299001+0000 mgr.gibba032.whopeo (mgr.24625) 8726 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:17.903 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:38:18.139 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:38:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:38:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:38:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:38:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:38:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:38:18.141 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:38:18.141 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:38:18.141 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:38:18.141 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:38:18.142 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:38:18.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:38:18.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:38:18.142 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:38:18.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:38:18.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:38:18.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:38:18.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:38:18.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:38:18.144 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:38:18.144 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:38:18.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:18 gibba011 bash[30791]: cluster 2021-03-10T16:38:17.702186+0000 mgr.gibba032.whopeo (mgr.24625) 8727 : cluster [DBG] pgmap v4972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:18.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:18 gibba011 bash[30791]: audit 2021-03-10T16:38:17.895787+0000 mgr.gibba032.whopeo (mgr.24625) 8728 : audit [DBG] from='client.59887 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:18 gibba011 bash[31774]: cluster 2021-03-10T16:38:17.702186+0000 mgr.gibba032.whopeo (mgr.24625) 8727 : cluster [DBG] pgmap v4972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:18.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:18 gibba011 bash[31774]: audit 2021-03-10T16:38:17.895787+0000 mgr.gibba032.whopeo (mgr.24625) 8728 : audit [DBG] from='client.59887 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:18.389 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:38:18.389 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:38:18.389 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:38:18.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:18.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:38:18.390 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:38:18.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:18.390 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:38:18.391 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:38:18.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:18.391 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:38:18.391 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:38:18.392 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:38:18.392 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:38:18.392 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:38:18.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:18 gibba032 bash[24004]: cluster 2021-03-10T16:38:17.702186+0000 mgr.gibba032.whopeo (mgr.24625) 8727 : cluster [DBG] pgmap v4972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:18.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:18 gibba032 bash[24004]: audit 2021-03-10T16:38:17.895787+0000 mgr.gibba032.whopeo (mgr.24625) 8728 : audit [DBG] from='client.59887 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:19.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:19 gibba011 bash[30791]: audit 2021-03-10T16:38:18.127914+0000 mgr.gibba032.whopeo (mgr.24625) 8729 : audit [DBG] from='client.49911 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:19.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:19 gibba011 bash[30791]: audit 2021-03-10T16:38:18.391426+0000 mon.a (mon.0) 3012 : audit [DBG] from='client.? 172.21.2.111:0/3550448851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:19.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:19 gibba011 bash[31774]: audit 2021-03-10T16:38:18.127914+0000 mgr.gibba032.whopeo (mgr.24625) 8729 : audit [DBG] from='client.49911 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:19.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:19 gibba011 bash[31774]: audit 2021-03-10T16:38:18.391426+0000 mon.a (mon.0) 3012 : audit [DBG] from='client.? 172.21.2.111:0/3550448851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:19.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:19 gibba032 bash[24004]: audit 2021-03-10T16:38:18.127914+0000 mgr.gibba032.whopeo (mgr.24625) 8729 : audit [DBG] from='client.49911 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:19.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:19 gibba032 bash[24004]: audit 2021-03-10T16:38:18.391426+0000 mon.a (mon.0) 3012 : audit [DBG] from='client.? 172.21.2.111:0/3550448851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:21.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:20 gibba011 bash[30791]: cluster 2021-03-10T16:38:19.703441+0000 mgr.gibba032.whopeo (mgr.24625) 8730 : cluster [DBG] pgmap v4973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:21.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:20 gibba011 bash[31774]: cluster 2021-03-10T16:38:19.703441+0000 mgr.gibba032.whopeo (mgr.24625) 8730 : cluster [DBG] pgmap v4973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:20 gibba032 bash[24004]: cluster 2021-03-10T16:38:19.703441+0000 mgr.gibba032.whopeo (mgr.24625) 8730 : cluster [DBG] pgmap v4973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:23.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:22 gibba011 bash[30791]: cluster 2021-03-10T16:38:21.704986+0000 mgr.gibba032.whopeo (mgr.24625) 8731 : cluster [DBG] pgmap v4974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:23.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:22 gibba011 bash[31774]: cluster 2021-03-10T16:38:21.704986+0000 mgr.gibba032.whopeo (mgr.24625) 8731 : cluster [DBG] pgmap v4974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:22 gibba032 bash[24004]: cluster 2021-03-10T16:38:21.704986+0000 mgr.gibba032.whopeo (mgr.24625) 8731 : cluster [DBG] pgmap v4974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:25.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:24 gibba011 bash[30791]: cluster 2021-03-10T16:38:23.706506+0000 mgr.gibba032.whopeo (mgr.24625) 8732 : cluster [DBG] pgmap v4975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:25.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:24 gibba011 bash[31774]: cluster 2021-03-10T16:38:23.706506+0000 mgr.gibba032.whopeo (mgr.24625) 8732 : cluster [DBG] pgmap v4975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:24 gibba032 bash[24004]: cluster 2021-03-10T16:38:23.706506+0000 mgr.gibba032.whopeo (mgr.24625) 8732 : cluster [DBG] pgmap v4975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:27.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:26 gibba011 bash[30791]: cluster 2021-03-10T16:38:25.707860+0000 mgr.gibba032.whopeo (mgr.24625) 8733 : cluster [DBG] pgmap v4976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:27.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:26 gibba011 bash[31774]: cluster 2021-03-10T16:38:25.707860+0000 mgr.gibba032.whopeo (mgr.24625) 8733 : cluster [DBG] pgmap v4976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:26 gibba032 bash[24004]: cluster 2021-03-10T16:38:25.707860+0000 mgr.gibba032.whopeo (mgr.24625) 8733 : cluster [DBG] pgmap v4976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:28 gibba032 bash[24004]: cluster 2021-03-10T16:38:27.709407+0000 mgr.gibba032.whopeo (mgr.24625) 8734 : cluster [DBG] pgmap v4977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:28 gibba011 bash[30791]: cluster 2021-03-10T16:38:27.709407+0000 mgr.gibba032.whopeo (mgr.24625) 8734 : cluster [DBG] pgmap v4977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:28 gibba011 bash[31774]: cluster 2021-03-10T16:38:27.709407+0000 mgr.gibba032.whopeo (mgr.24625) 8734 : cluster [DBG] pgmap v4977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:30 gibba032 bash[24004]: cluster 2021-03-10T16:38:29.710857+0000 mgr.gibba032.whopeo (mgr.24625) 8735 : cluster [DBG] pgmap v4978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:30 gibba011 bash[30791]: cluster 2021-03-10T16:38:29.710857+0000 mgr.gibba032.whopeo (mgr.24625) 8735 : cluster [DBG] pgmap v4978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:30 gibba011 bash[31774]: cluster 2021-03-10T16:38:29.710857+0000 mgr.gibba032.whopeo (mgr.24625) 8735 : cluster [DBG] pgmap v4978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.313566+0000 mgr.gibba032.whopeo (mgr.24625) 8736 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:38:32.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.315140+0000 mgr.gibba032.whopeo (mgr.24625) 8737 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:38:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.546291+0000 mon.a (mon.0) 3013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:38:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.546784+0000 mgr.gibba032.whopeo (mgr.24625) 8738 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:38:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.549130+0000 mgr.gibba032.whopeo (mgr.24625) 8739 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:38:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.549604+0000 mon.a (mon.0) 3014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:38:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.551043+0000 mgr.gibba032.whopeo (mgr.24625) 8740 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:38:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.551087+0000 mgr.gibba032.whopeo (mgr.24625) 8741 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:38:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.551653+0000 mon.a (mon.0) 3015 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.552673+0000 mgr.gibba032.whopeo (mgr.24625) 8742 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:38:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.552718+0000 mgr.gibba032.whopeo (mgr.24625) 8743 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:38:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.553466+0000 mon.a (mon.0) 3016 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.554322+0000 mgr.gibba032.whopeo (mgr.24625) 8744 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:38:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.555090+0000 mon.a (mon.0) 3017 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.555827+0000 mgr.gibba032.whopeo (mgr.24625) 8745 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:38:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.555874+0000 mgr.gibba032.whopeo (mgr.24625) 8746 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:38:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.557420+0000 mgr.gibba032.whopeo (mgr.24625) 8747 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:38:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.557880+0000 mgr.gibba032.whopeo (mgr.24625) 8748 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:38:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.558453+0000 mon.a (mon.0) 3018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.559196+0000 mgr.gibba032.whopeo (mgr.24625) 8749 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:38:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.559239+0000 mgr.gibba032.whopeo (mgr.24625) 8750 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:38:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cephadm 2021-03-10T16:38:31.559383+0000 mgr.gibba032.whopeo (mgr.24625) 8751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: audit 2021-03-10T16:38:31.560259+0000 mon.a (mon.0) 3019 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[30791]: cluster 2021-03-10T16:38:31.712586+0000 mgr.gibba032.whopeo (mgr.24625) 8752 : cluster [DBG] pgmap v4979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.313566+0000 mgr.gibba032.whopeo (mgr.24625) 8736 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:38:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.315140+0000 mgr.gibba032.whopeo (mgr.24625) 8737 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:38:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.546291+0000 mon.a (mon.0) 3013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:38:32.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.546784+0000 mgr.gibba032.whopeo (mgr.24625) 8738 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:38:32.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.549130+0000 mgr.gibba032.whopeo (mgr.24625) 8739 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:38:32.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.549604+0000 mon.a (mon.0) 3014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:38:32.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.551043+0000 mgr.gibba032.whopeo (mgr.24625) 8740 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:38:32.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.551087+0000 mgr.gibba032.whopeo (mgr.24625) 8741 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:38:32.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.551653+0000 mon.a (mon.0) 3015 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.552673+0000 mgr.gibba032.whopeo (mgr.24625) 8742 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:38:32.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.552718+0000 mgr.gibba032.whopeo (mgr.24625) 8743 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:38:32.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.553466+0000 mon.a (mon.0) 3016 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.554322+0000 mgr.gibba032.whopeo (mgr.24625) 8744 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:38:32.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.555090+0000 mon.a (mon.0) 3017 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.555827+0000 mgr.gibba032.whopeo (mgr.24625) 8745 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:38:32.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.555874+0000 mgr.gibba032.whopeo (mgr.24625) 8746 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:38:32.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.557420+0000 mgr.gibba032.whopeo (mgr.24625) 8747 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:38:32.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.557880+0000 mgr.gibba032.whopeo (mgr.24625) 8748 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:38:32.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.558453+0000 mon.a (mon.0) 3018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.559196+0000 mgr.gibba032.whopeo (mgr.24625) 8749 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:38:32.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.559239+0000 mgr.gibba032.whopeo (mgr.24625) 8750 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:38:32.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cephadm 2021-03-10T16:38:31.559383+0000 mgr.gibba032.whopeo (mgr.24625) 8751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:32.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: audit 2021-03-10T16:38:31.560259+0000 mon.a (mon.0) 3019 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:32.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:32 gibba011 bash[31774]: cluster 2021-03-10T16:38:31.712586+0000 mgr.gibba032.whopeo (mgr.24625) 8752 : cluster [DBG] pgmap v4979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.313566+0000 mgr.gibba032.whopeo (mgr.24625) 8736 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:38:33.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.315140+0000 mgr.gibba032.whopeo (mgr.24625) 8737 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:38:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.546291+0000 mon.a (mon.0) 3013 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:38:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.546784+0000 mgr.gibba032.whopeo (mgr.24625) 8738 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:38:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.549130+0000 mgr.gibba032.whopeo (mgr.24625) 8739 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:38:33.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.549604+0000 mon.a (mon.0) 3014 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:38:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.551043+0000 mgr.gibba032.whopeo (mgr.24625) 8740 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:38:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.551087+0000 mgr.gibba032.whopeo (mgr.24625) 8741 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:38:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.551653+0000 mon.a (mon.0) 3015 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.552673+0000 mgr.gibba032.whopeo (mgr.24625) 8742 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:38:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.552718+0000 mgr.gibba032.whopeo (mgr.24625) 8743 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:38:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.553466+0000 mon.a (mon.0) 3016 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.554322+0000 mgr.gibba032.whopeo (mgr.24625) 8744 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:38:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.555090+0000 mon.a (mon.0) 3017 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:33.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.555827+0000 mgr.gibba032.whopeo (mgr.24625) 8745 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:38:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.555874+0000 mgr.gibba032.whopeo (mgr.24625) 8746 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:38:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.557420+0000 mgr.gibba032.whopeo (mgr.24625) 8747 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:38:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.557880+0000 mgr.gibba032.whopeo (mgr.24625) 8748 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:38:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.558453+0000 mon.a (mon.0) 3018 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:33.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.559196+0000 mgr.gibba032.whopeo (mgr.24625) 8749 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:38:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.559239+0000 mgr.gibba032.whopeo (mgr.24625) 8750 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:38:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cephadm 2021-03-10T16:38:31.559383+0000 mgr.gibba032.whopeo (mgr.24625) 8751 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: audit 2021-03-10T16:38:31.560259+0000 mon.a (mon.0) 3019 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:33.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:32 gibba032 bash[24004]: cluster 2021-03-10T16:38:31.712586+0000 mgr.gibba032.whopeo (mgr.24625) 8752 : cluster [DBG] pgmap v4979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:35.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:34 gibba011 bash[30791]: cluster 2021-03-10T16:38:33.714185+0000 mgr.gibba032.whopeo (mgr.24625) 8753 : cluster [DBG] pgmap v4980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:35.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:34 gibba011 bash[31774]: cluster 2021-03-10T16:38:33.714185+0000 mgr.gibba032.whopeo (mgr.24625) 8753 : cluster [DBG] pgmap v4980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:34 gibba032 bash[24004]: cluster 2021-03-10T16:38:33.714185+0000 mgr.gibba032.whopeo (mgr.24625) 8753 : cluster [DBG] pgmap v4980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:37.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:36 gibba011 bash[30791]: cluster 2021-03-10T16:38:35.715499+0000 mgr.gibba032.whopeo (mgr.24625) 8754 : cluster [DBG] pgmap v4981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:37.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:36 gibba011 bash[31774]: cluster 2021-03-10T16:38:35.715499+0000 mgr.gibba032.whopeo (mgr.24625) 8754 : cluster [DBG] pgmap v4981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:36 gibba032 bash[24004]: cluster 2021-03-10T16:38:35.715499+0000 mgr.gibba032.whopeo (mgr.24625) 8754 : cluster [DBG] pgmap v4981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:39.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:38 gibba011 bash[30791]: cluster 2021-03-10T16:38:37.717199+0000 mgr.gibba032.whopeo (mgr.24625) 8755 : cluster [DBG] pgmap v4982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:39.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:38 gibba011 bash[31774]: cluster 2021-03-10T16:38:37.717199+0000 mgr.gibba032.whopeo (mgr.24625) 8755 : cluster [DBG] pgmap v4982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:38 gibba032 bash[24004]: cluster 2021-03-10T16:38:37.717199+0000 mgr.gibba032.whopeo (mgr.24625) 8755 : cluster [DBG] pgmap v4982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:41.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:40 gibba011 bash[30791]: cluster 2021-03-10T16:38:39.718593+0000 mgr.gibba032.whopeo (mgr.24625) 8756 : cluster [DBG] pgmap v4983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:41.077 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:40 gibba011 bash[31774]: cluster 2021-03-10T16:38:39.718593+0000 mgr.gibba032.whopeo (mgr.24625) 8756 : cluster [DBG] pgmap v4983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:40 gibba032 bash[24004]: cluster 2021-03-10T16:38:39.718593+0000 mgr.gibba032.whopeo (mgr.24625) 8756 : cluster [DBG] pgmap v4983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:42 gibba032 bash[24004]: cluster 2021-03-10T16:38:41.720449+0000 mgr.gibba032.whopeo (mgr.24625) 8757 : cluster [DBG] pgmap v4984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:42 gibba011 bash[30791]: cluster 2021-03-10T16:38:41.720449+0000 mgr.gibba032.whopeo (mgr.24625) 8757 : cluster [DBG] pgmap v4984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:42 gibba011 bash[31774]: cluster 2021-03-10T16:38:41.720449+0000 mgr.gibba032.whopeo (mgr.24625) 8757 : cluster [DBG] pgmap v4984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:44 gibba032 bash[24004]: cluster 2021-03-10T16:38:43.722107+0000 mgr.gibba032.whopeo (mgr.24625) 8758 : cluster [DBG] pgmap v4985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:44 gibba011 bash[30791]: cluster 2021-03-10T16:38:43.722107+0000 mgr.gibba032.whopeo (mgr.24625) 8758 : cluster [DBG] pgmap v4985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:44 gibba011 bash[31774]: cluster 2021-03-10T16:38:43.722107+0000 mgr.gibba032.whopeo (mgr.24625) 8758 : cluster [DBG] pgmap v4985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:46 gibba032 bash[24004]: cluster 2021-03-10T16:38:45.723386+0000 mgr.gibba032.whopeo (mgr.24625) 8759 : cluster [DBG] pgmap v4986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:46 gibba011 bash[30791]: cluster 2021-03-10T16:38:45.723386+0000 mgr.gibba032.whopeo (mgr.24625) 8759 : cluster [DBG] pgmap v4986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:46 gibba011 bash[31774]: cluster 2021-03-10T16:38:45.723386+0000 mgr.gibba032.whopeo (mgr.24625) 8759 : cluster [DBG] pgmap v4986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:48.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:47 gibba032 bash[24004]: cephadm 2021-03-10T16:38:46.574057+0000 mgr.gibba032.whopeo (mgr.24625) 8760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:48.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:47 gibba011 bash[30791]: cephadm 2021-03-10T16:38:46.574057+0000 mgr.gibba032.whopeo (mgr.24625) 8760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:48.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:47 gibba011 bash[31774]: cephadm 2021-03-10T16:38:46.574057+0000 mgr.gibba032.whopeo (mgr.24625) 8760 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:38:48.632 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:38:48.860 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:38:48.860 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:38:48.860 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:38:48.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:38:48.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:38:48.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:38:48.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:38:48.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:38:48.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:38:48.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:38:48.863 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:38:48.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:38:48.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:38:48.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:38:48.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:38:48.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:38:48.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:38:48.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:38:48.865 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:38:48.865 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:38:48.865 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:38:49.115 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:38:49.115 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:38:49.115 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:38:49.115 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:49.116 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:38:49.116 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:38:49.116 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:49.116 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:38:49.117 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:38:49.117 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:38:49.117 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:38:49.117 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:38:49.117 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:38:49.118 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:38:49.118 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:38:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:48 gibba032 bash[24004]: cluster 2021-03-10T16:38:47.725177+0000 mgr.gibba032.whopeo (mgr.24625) 8761 : cluster [DBG] pgmap v4987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:48 gibba011 bash[30791]: cluster 2021-03-10T16:38:47.725177+0000 mgr.gibba032.whopeo (mgr.24625) 8761 : cluster [DBG] pgmap v4987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:48 gibba011 bash[31774]: cluster 2021-03-10T16:38:47.725177+0000 mgr.gibba032.whopeo (mgr.24625) 8761 : cluster [DBG] pgmap v4987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:50.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:49 gibba032 bash[24004]: audit 2021-03-10T16:38:48.623659+0000 mgr.gibba032.whopeo (mgr.24625) 8762 : audit [DBG] from='client.49923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:49 gibba032 bash[24004]: audit 2021-03-10T16:38:48.848512+0000 mgr.gibba032.whopeo (mgr.24625) 8763 : audit [DBG] from='client.49929 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:49 gibba032 bash[24004]: audit 2021-03-10T16:38:49.117443+0000 mon.a (mon.0) 3020 : audit [DBG] from='client.? 172.21.2.111:0/2894395240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:50.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[30791]: audit 2021-03-10T16:38:48.623659+0000 mgr.gibba032.whopeo (mgr.24625) 8762 : audit [DBG] from='client.49923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[30791]: audit 2021-03-10T16:38:48.848512+0000 mgr.gibba032.whopeo (mgr.24625) 8763 : audit [DBG] from='client.49929 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[30791]: audit 2021-03-10T16:38:49.117443+0000 mon.a (mon.0) 3020 : audit [DBG] from='client.? 172.21.2.111:0/2894395240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:50.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[31774]: audit 2021-03-10T16:38:48.623659+0000 mgr.gibba032.whopeo (mgr.24625) 8762 : audit [DBG] from='client.49923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[31774]: audit 2021-03-10T16:38:48.848512+0000 mgr.gibba032.whopeo (mgr.24625) 8763 : audit [DBG] from='client.49929 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:38:50.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:49 gibba011 bash[31774]: audit 2021-03-10T16:38:49.117443+0000 mon.a (mon.0) 3020 : audit [DBG] from='client.? 172.21.2.111:0/2894395240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:38:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:50 gibba032 bash[24004]: cluster 2021-03-10T16:38:49.726644+0000 mgr.gibba032.whopeo (mgr.24625) 8764 : cluster [DBG] pgmap v4988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:50 gibba011 bash[30791]: cluster 2021-03-10T16:38:49.726644+0000 mgr.gibba032.whopeo (mgr.24625) 8764 : cluster [DBG] pgmap v4988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:50 gibba011 bash[31774]: cluster 2021-03-10T16:38:49.726644+0000 mgr.gibba032.whopeo (mgr.24625) 8764 : cluster [DBG] pgmap v4988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:52 gibba032 bash[24004]: cluster 2021-03-10T16:38:51.728474+0000 mgr.gibba032.whopeo (mgr.24625) 8765 : cluster [DBG] pgmap v4989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:52 gibba011 bash[30791]: cluster 2021-03-10T16:38:51.728474+0000 mgr.gibba032.whopeo (mgr.24625) 8765 : cluster [DBG] pgmap v4989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:52 gibba011 bash[31774]: cluster 2021-03-10T16:38:51.728474+0000 mgr.gibba032.whopeo (mgr.24625) 8765 : cluster [DBG] pgmap v4989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:54 gibba032 bash[24004]: cluster 2021-03-10T16:38:53.730131+0000 mgr.gibba032.whopeo (mgr.24625) 8766 : cluster [DBG] pgmap v4990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:54 gibba011 bash[30791]: cluster 2021-03-10T16:38:53.730131+0000 mgr.gibba032.whopeo (mgr.24625) 8766 : cluster [DBG] pgmap v4990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:54 gibba011 bash[31774]: cluster 2021-03-10T16:38:53.730131+0000 mgr.gibba032.whopeo (mgr.24625) 8766 : cluster [DBG] pgmap v4990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:56 gibba032 bash[24004]: cluster 2021-03-10T16:38:55.731502+0000 mgr.gibba032.whopeo (mgr.24625) 8767 : cluster [DBG] pgmap v4991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:56 gibba011 bash[30791]: cluster 2021-03-10T16:38:55.731502+0000 mgr.gibba032.whopeo (mgr.24625) 8767 : cluster [DBG] pgmap v4991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:56 gibba011 bash[31774]: cluster 2021-03-10T16:38:55.731502+0000 mgr.gibba032.whopeo (mgr.24625) 8767 : cluster [DBG] pgmap v4991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:58.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.114004+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:38:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.114692+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:38:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.231945+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:38:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.232699+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.233477+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.234240+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [3, 2]}]: dispatch 2021-03-10T16:38:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:57 gibba032 bash[24004]: audit 2021-03-10T16:38:57.234974+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 0]}]: dispatch 2021-03-10T16:38:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.114004+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:38:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.114692+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:38:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.231945+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:38:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.232699+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.233477+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.234240+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [3, 2]}]: dispatch 2021-03-10T16:38:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[30791]: audit 2021-03-10T16:38:57.234974+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 0]}]: dispatch 2021-03-10T16:38:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.114004+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:38:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.114692+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:38:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.231945+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:38:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.232699+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.233477+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:38:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.234240+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [3, 2]}]: dispatch 2021-03-10T16:38:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:57 gibba011 bash[31774]: audit 2021-03-10T16:38:57.234974+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 0]}]: dispatch 2021-03-10T16:38:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:38:58 gibba032 bash[24004]: cluster 2021-03-10T16:38:57.733245+0000 mgr.gibba032.whopeo (mgr.24625) 8768 : cluster [DBG] pgmap v4992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:38:58 gibba011 bash[30791]: cluster 2021-03-10T16:38:57.733245+0000 mgr.gibba032.whopeo (mgr.24625) 8768 : cluster [DBG] pgmap v4992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:38:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:38:58 gibba011 bash[31774]: cluster 2021-03-10T16:38:57.733245+0000 mgr.gibba032.whopeo (mgr.24625) 8768 : cluster [DBG] pgmap v4992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:00 gibba032 bash[24004]: cluster 2021-03-10T16:38:59.734710+0000 mgr.gibba032.whopeo (mgr.24625) 8769 : cluster [DBG] pgmap v4993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:00 gibba011 bash[30791]: cluster 2021-03-10T16:38:59.734710+0000 mgr.gibba032.whopeo (mgr.24625) 8769 : cluster [DBG] pgmap v4993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:00 gibba011 bash[31774]: cluster 2021-03-10T16:38:59.734710+0000 mgr.gibba032.whopeo (mgr.24625) 8769 : cluster [DBG] pgmap v4993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:02 gibba032 bash[24004]: cephadm 2021-03-10T16:39:01.590175+0000 mgr.gibba032.whopeo (mgr.24625) 8770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:02 gibba032 bash[24004]: cluster 2021-03-10T16:39:01.736542+0000 mgr.gibba032.whopeo (mgr.24625) 8771 : cluster [DBG] pgmap v4994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:02 gibba011 bash[30791]: cephadm 2021-03-10T16:39:01.590175+0000 mgr.gibba032.whopeo (mgr.24625) 8770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:03.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:02 gibba011 bash[30791]: cluster 2021-03-10T16:39:01.736542+0000 mgr.gibba032.whopeo (mgr.24625) 8771 : cluster [DBG] pgmap v4994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:02 gibba011 bash[31774]: cephadm 2021-03-10T16:39:01.590175+0000 mgr.gibba032.whopeo (mgr.24625) 8770 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:02 gibba011 bash[31774]: cluster 2021-03-10T16:39:01.736542+0000 mgr.gibba032.whopeo (mgr.24625) 8771 : cluster [DBG] pgmap v4994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:04 gibba032 bash[24004]: cluster 2021-03-10T16:39:03.738162+0000 mgr.gibba032.whopeo (mgr.24625) 8772 : cluster [DBG] pgmap v4995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:04 gibba011 bash[30791]: cluster 2021-03-10T16:39:03.738162+0000 mgr.gibba032.whopeo (mgr.24625) 8772 : cluster [DBG] pgmap v4995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:04 gibba011 bash[31774]: cluster 2021-03-10T16:39:03.738162+0000 mgr.gibba032.whopeo (mgr.24625) 8772 : cluster [DBG] pgmap v4995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:06 gibba032 bash[24004]: cluster 2021-03-10T16:39:05.739617+0000 mgr.gibba032.whopeo (mgr.24625) 8773 : cluster [DBG] pgmap v4996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:07.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:06 gibba011 bash[30791]: cluster 2021-03-10T16:39:05.739617+0000 mgr.gibba032.whopeo (mgr.24625) 8773 : cluster [DBG] pgmap v4996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:07.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:06 gibba011 bash[31774]: cluster 2021-03-10T16:39:05.739617+0000 mgr.gibba032.whopeo (mgr.24625) 8773 : cluster [DBG] pgmap v4996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:09.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:08 gibba032 bash[24004]: cluster 2021-03-10T16:39:07.741413+0000 mgr.gibba032.whopeo (mgr.24625) 8774 : cluster [DBG] pgmap v4997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:08 gibba011 bash[30791]: cluster 2021-03-10T16:39:07.741413+0000 mgr.gibba032.whopeo (mgr.24625) 8774 : cluster [DBG] pgmap v4997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:08 gibba011 bash[31774]: cluster 2021-03-10T16:39:07.741413+0000 mgr.gibba032.whopeo (mgr.24625) 8774 : cluster [DBG] pgmap v4997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:10 gibba032 bash[24004]: cluster 2021-03-10T16:39:09.742931+0000 mgr.gibba032.whopeo (mgr.24625) 8775 : cluster [DBG] pgmap v4998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:10 gibba011 bash[30791]: cluster 2021-03-10T16:39:09.742931+0000 mgr.gibba032.whopeo (mgr.24625) 8775 : cluster [DBG] pgmap v4998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:10 gibba011 bash[31774]: cluster 2021-03-10T16:39:09.742931+0000 mgr.gibba032.whopeo (mgr.24625) 8775 : cluster [DBG] pgmap v4998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:12 gibba032 bash[24004]: cluster 2021-03-10T16:39:11.744740+0000 mgr.gibba032.whopeo (mgr.24625) 8776 : cluster [DBG] pgmap v4999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:13.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:12 gibba011 bash[30791]: cluster 2021-03-10T16:39:11.744740+0000 mgr.gibba032.whopeo (mgr.24625) 8776 : cluster [DBG] pgmap v4999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:13.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:12 gibba011 bash[31774]: cluster 2021-03-10T16:39:11.744740+0000 mgr.gibba032.whopeo (mgr.24625) 8776 : cluster [DBG] pgmap v4999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:14 gibba032 bash[24004]: cluster 2021-03-10T16:39:13.746277+0000 mgr.gibba032.whopeo (mgr.24625) 8777 : cluster [DBG] pgmap v5000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:14 gibba011 bash[30791]: cluster 2021-03-10T16:39:13.746277+0000 mgr.gibba032.whopeo (mgr.24625) 8777 : cluster [DBG] pgmap v5000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:14 gibba011 bash[31774]: cluster 2021-03-10T16:39:13.746277+0000 mgr.gibba032.whopeo (mgr.24625) 8777 : cluster [DBG] pgmap v5000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:16 gibba032 bash[24004]: cluster 2021-03-10T16:39:15.747613+0000 mgr.gibba032.whopeo (mgr.24625) 8778 : cluster [DBG] pgmap v5001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:16 gibba011 bash[30791]: cluster 2021-03-10T16:39:15.747613+0000 mgr.gibba032.whopeo (mgr.24625) 8778 : cluster [DBG] pgmap v5001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:17 gibba011 bash[31774]: cluster 2021-03-10T16:39:15.747613+0000 mgr.gibba032.whopeo (mgr.24625) 8778 : cluster [DBG] pgmap v5001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:18.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:18 gibba032 bash[24004]: cephadm 2021-03-10T16:39:16.606221+0000 mgr.gibba032.whopeo (mgr.24625) 8779 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:18.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:18 gibba032 bash[24004]: cluster 2021-03-10T16:39:17.749320+0000 mgr.gibba032.whopeo (mgr.24625) 8780 : cluster [DBG] pgmap v5002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:18.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:18 gibba011 bash[30791]: cephadm 2021-03-10T16:39:16.606221+0000 mgr.gibba032.whopeo (mgr.24625) 8779 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:18.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:18 gibba011 bash[30791]: cluster 2021-03-10T16:39:17.749320+0000 mgr.gibba032.whopeo (mgr.24625) 8780 : cluster [DBG] pgmap v5002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:18.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:18 gibba011 bash[31774]: cephadm 2021-03-10T16:39:16.606221+0000 mgr.gibba032.whopeo (mgr.24625) 8779 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:18.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:18 gibba011 bash[31774]: cluster 2021-03-10T16:39:17.749320+0000 mgr.gibba032.whopeo (mgr.24625) 8780 : cluster [DBG] pgmap v5002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:19.357 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:39:19.610 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:39:19.610 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:39:19.611 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 7m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:39:19.611 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:39:19.611 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:39:19.612 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:39:19.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:39:19.612 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:39:19.613 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:39:19.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:39:19.613 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 7m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:39:19.613 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:39:19.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:39:19.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:39:19.614 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:39:19.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:39:19.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:39:19.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:39:19.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:39:19.616 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 7m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:39:19.616 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:39:19.872 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:39:19.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:39:19.872 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:39:19.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:19.873 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:39:19.873 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:39:19.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:19.874 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:39:19.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:39:19.874 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:19.874 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:39:19.875 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:39:19.875 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:39:19.875 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:39:19.875 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:39:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:20 gibba032 bash[24004]: audit 2021-03-10T16:39:19.348491+0000 mgr.gibba032.whopeo (mgr.24625) 8781 : audit [DBG] from='client.59923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:20 gibba032 bash[24004]: audit 2021-03-10T16:39:19.570947+0000 mgr.gibba032.whopeo (mgr.24625) 8782 : audit [DBG] from='client.49947 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:20 gibba032 bash[24004]: cluster 2021-03-10T16:39:19.750842+0000 mgr.gibba032.whopeo (mgr.24625) 8783 : cluster [DBG] pgmap v5003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:21.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:20 gibba032 bash[24004]: audit 2021-03-10T16:39:19.874743+0000 mon.c (mon.1) 171 : audit [DBG] from='client.? 172.21.2.111:0/1018457961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:21.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[30791]: audit 2021-03-10T16:39:19.348491+0000 mgr.gibba032.whopeo (mgr.24625) 8781 : audit [DBG] from='client.59923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[30791]: audit 2021-03-10T16:39:19.570947+0000 mgr.gibba032.whopeo (mgr.24625) 8782 : audit [DBG] from='client.49947 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[30791]: cluster 2021-03-10T16:39:19.750842+0000 mgr.gibba032.whopeo (mgr.24625) 8783 : cluster [DBG] pgmap v5003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:21.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[30791]: audit 2021-03-10T16:39:19.874743+0000 mon.c (mon.1) 171 : audit [DBG] from='client.? 172.21.2.111:0/1018457961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:21.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[31774]: audit 2021-03-10T16:39:19.348491+0000 mgr.gibba032.whopeo (mgr.24625) 8781 : audit [DBG] from='client.59923 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[31774]: audit 2021-03-10T16:39:19.570947+0000 mgr.gibba032.whopeo (mgr.24625) 8782 : audit [DBG] from='client.49947 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:21.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[31774]: cluster 2021-03-10T16:39:19.750842+0000 mgr.gibba032.whopeo (mgr.24625) 8783 : cluster [DBG] pgmap v5003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:20 gibba011 bash[31774]: audit 2021-03-10T16:39:19.874743+0000 mon.c (mon.1) 171 : audit [DBG] from='client.? 172.21.2.111:0/1018457961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:22 gibba032 bash[24004]: cluster 2021-03-10T16:39:21.752672+0000 mgr.gibba032.whopeo (mgr.24625) 8784 : cluster [DBG] pgmap v5004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:22 gibba011 bash[30791]: cluster 2021-03-10T16:39:21.752672+0000 mgr.gibba032.whopeo (mgr.24625) 8784 : cluster [DBG] pgmap v5004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:22 gibba011 bash[31774]: cluster 2021-03-10T16:39:21.752672+0000 mgr.gibba032.whopeo (mgr.24625) 8784 : cluster [DBG] pgmap v5004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:24 gibba032 bash[24004]: cluster 2021-03-10T16:39:23.754233+0000 mgr.gibba032.whopeo (mgr.24625) 8785 : cluster [DBG] pgmap v5005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:24 gibba011 bash[30791]: cluster 2021-03-10T16:39:23.754233+0000 mgr.gibba032.whopeo (mgr.24625) 8785 : cluster [DBG] pgmap v5005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:25.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:24 gibba011 bash[31774]: cluster 2021-03-10T16:39:23.754233+0000 mgr.gibba032.whopeo (mgr.24625) 8785 : cluster [DBG] pgmap v5005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:26 gibba032 bash[24004]: cluster 2021-03-10T16:39:25.755575+0000 mgr.gibba032.whopeo (mgr.24625) 8786 : cluster [DBG] pgmap v5006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:26 gibba011 bash[30791]: cluster 2021-03-10T16:39:25.755575+0000 mgr.gibba032.whopeo (mgr.24625) 8786 : cluster [DBG] pgmap v5006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:26 gibba011 bash[31774]: cluster 2021-03-10T16:39:25.755575+0000 mgr.gibba032.whopeo (mgr.24625) 8786 : cluster [DBG] pgmap v5006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:28 gibba032 bash[24004]: cluster 2021-03-10T16:39:27.757275+0000 mgr.gibba032.whopeo (mgr.24625) 8787 : cluster [DBG] pgmap v5007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:28 gibba011 bash[30791]: cluster 2021-03-10T16:39:27.757275+0000 mgr.gibba032.whopeo (mgr.24625) 8787 : cluster [DBG] pgmap v5007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:28 gibba011 bash[31774]: cluster 2021-03-10T16:39:27.757275+0000 mgr.gibba032.whopeo (mgr.24625) 8787 : cluster [DBG] pgmap v5007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:30 gibba032 bash[24004]: cluster 2021-03-10T16:39:29.758536+0000 mgr.gibba032.whopeo (mgr.24625) 8788 : cluster [DBG] pgmap v5008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:30 gibba011 bash[30791]: cluster 2021-03-10T16:39:29.758536+0000 mgr.gibba032.whopeo (mgr.24625) 8788 : cluster [DBG] pgmap v5008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:30 gibba011 bash[31774]: cluster 2021-03-10T16:39:29.758536+0000 mgr.gibba032.whopeo (mgr.24625) 8788 : cluster [DBG] pgmap v5008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:33.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.622803+0000 mgr.gibba032.whopeo (mgr.24625) 8789 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:39:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.624448+0000 mgr.gibba032.whopeo (mgr.24625) 8790 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:39:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cluster 2021-03-10T16:39:31.759156+0000 mgr.gibba032.whopeo (mgr.24625) 8791 : cluster [DBG] pgmap v5009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:33.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.843011+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:39:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.843565+0000 mgr.gibba032.whopeo (mgr.24625) 8792 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:39:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.845393+0000 mgr.gibba032.whopeo (mgr.24625) 8793 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:39:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.846260+0000 mon.a (mon.0) 3029 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:39:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.846772+0000 mgr.gibba032.whopeo (mgr.24625) 8794 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:39:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.846809+0000 mgr.gibba032.whopeo (mgr.24625) 8795 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:39:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.847689+0000 mon.a (mon.0) 3030 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.848241+0000 mgr.gibba032.whopeo (mgr.24625) 8796 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:39:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.848283+0000 mgr.gibba032.whopeo (mgr.24625) 8797 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:39:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.849181+0000 mon.a (mon.0) 3031 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.849426+0000 mgr.gibba032.whopeo (mgr.24625) 8798 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:39:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.850544+0000 mon.a (mon.0) 3032 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.850582+0000 mgr.gibba032.whopeo (mgr.24625) 8799 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:39:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.850621+0000 mgr.gibba032.whopeo (mgr.24625) 8800 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:39:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.851943+0000 mgr.gibba032.whopeo (mgr.24625) 8801 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:39:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.852322+0000 mgr.gibba032.whopeo (mgr.24625) 8802 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:39:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.853055+0000 mon.a (mon.0) 3033 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.853698+0000 mgr.gibba032.whopeo (mgr.24625) 8803 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:39:33.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.853739+0000 mgr.gibba032.whopeo (mgr.24625) 8804 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:39:33.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: cephadm 2021-03-10T16:39:31.853860+0000 mgr.gibba032.whopeo (mgr.24625) 8805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:33.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:32 gibba032 bash[24004]: audit 2021-03-10T16:39:31.854597+0000 mon.a (mon.0) 3034 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.622803+0000 mgr.gibba032.whopeo (mgr.24625) 8789 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:39:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.624448+0000 mgr.gibba032.whopeo (mgr.24625) 8790 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:39:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cluster 2021-03-10T16:39:31.759156+0000 mgr.gibba032.whopeo (mgr.24625) 8791 : cluster [DBG] pgmap v5009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.843011+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:39:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.843565+0000 mgr.gibba032.whopeo (mgr.24625) 8792 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:39:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.845393+0000 mgr.gibba032.whopeo (mgr.24625) 8793 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:39:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.846260+0000 mon.a (mon.0) 3029 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:39:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.846772+0000 mgr.gibba032.whopeo (mgr.24625) 8794 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:39:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.846809+0000 mgr.gibba032.whopeo (mgr.24625) 8795 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:39:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.847689+0000 mon.a (mon.0) 3030 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.848241+0000 mgr.gibba032.whopeo (mgr.24625) 8796 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:39:33.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.848283+0000 mgr.gibba032.whopeo (mgr.24625) 8797 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:39:33.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.849181+0000 mon.a (mon.0) 3031 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.849426+0000 mgr.gibba032.whopeo (mgr.24625) 8798 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:39:33.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.850544+0000 mon.a (mon.0) 3032 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.850582+0000 mgr.gibba032.whopeo (mgr.24625) 8799 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:39:33.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.850621+0000 mgr.gibba032.whopeo (mgr.24625) 8800 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:39:33.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.851943+0000 mgr.gibba032.whopeo (mgr.24625) 8801 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:39:33.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.852322+0000 mgr.gibba032.whopeo (mgr.24625) 8802 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:39:33.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.853055+0000 mon.a (mon.0) 3033 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.853698+0000 mgr.gibba032.whopeo (mgr.24625) 8803 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:39:33.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.622803+0000 mgr.gibba032.whopeo (mgr.24625) 8789 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:39:33.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.624448+0000 mgr.gibba032.whopeo (mgr.24625) 8790 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:39:33.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cluster 2021-03-10T16:39:31.759156+0000 mgr.gibba032.whopeo (mgr.24625) 8791 : cluster [DBG] pgmap v5009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:33.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.843011+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:39:33.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.843565+0000 mgr.gibba032.whopeo (mgr.24625) 8792 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:39:33.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.845393+0000 mgr.gibba032.whopeo (mgr.24625) 8793 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:39:33.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.846260+0000 mon.a (mon.0) 3029 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:39:33.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.846772+0000 mgr.gibba032.whopeo (mgr.24625) 8794 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:39:33.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.846809+0000 mgr.gibba032.whopeo (mgr.24625) 8795 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:39:33.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.847689+0000 mon.a (mon.0) 3030 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.848241+0000 mgr.gibba032.whopeo (mgr.24625) 8796 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:39:33.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.848283+0000 mgr.gibba032.whopeo (mgr.24625) 8797 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:39:33.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.849181+0000 mon.a (mon.0) 3031 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.849426+0000 mgr.gibba032.whopeo (mgr.24625) 8798 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:39:33.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.850544+0000 mon.a (mon.0) 3032 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.850582+0000 mgr.gibba032.whopeo (mgr.24625) 8799 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:39:33.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.850621+0000 mgr.gibba032.whopeo (mgr.24625) 8800 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:39:33.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.851943+0000 mgr.gibba032.whopeo (mgr.24625) 8801 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:39:33.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.852322+0000 mgr.gibba032.whopeo (mgr.24625) 8802 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:39:33.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.853055+0000 mon.a (mon.0) 3033 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.853698+0000 mgr.gibba032.whopeo (mgr.24625) 8803 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:39:33.347 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.853739+0000 mgr.gibba032.whopeo (mgr.24625) 8804 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:39:33.347 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: cephadm 2021-03-10T16:39:31.853860+0000 mgr.gibba032.whopeo (mgr.24625) 8805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:33.347 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[30791]: audit 2021-03-10T16:39:31.854597+0000 mon.a (mon.0) 3034 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:33.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.853739+0000 mgr.gibba032.whopeo (mgr.24625) 8804 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:39:33.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: cephadm 2021-03-10T16:39:31.853860+0000 mgr.gibba032.whopeo (mgr.24625) 8805 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:33.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:32 gibba011 bash[31774]: audit 2021-03-10T16:39:31.854597+0000 mon.a (mon.0) 3034 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:34 gibba032 bash[24004]: cluster 2021-03-10T16:39:33.760841+0000 mgr.gibba032.whopeo (mgr.24625) 8806 : cluster [DBG] pgmap v5010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:34 gibba011 bash[30791]: cluster 2021-03-10T16:39:33.760841+0000 mgr.gibba032.whopeo (mgr.24625) 8806 : cluster [DBG] pgmap v5010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:35.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:34 gibba011 bash[31774]: cluster 2021-03-10T16:39:33.760841+0000 mgr.gibba032.whopeo (mgr.24625) 8806 : cluster [DBG] pgmap v5010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:36 gibba032 bash[24004]: cluster 2021-03-10T16:39:35.762364+0000 mgr.gibba032.whopeo (mgr.24625) 8807 : cluster [DBG] pgmap v5011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:37.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:36 gibba011 bash[30791]: cluster 2021-03-10T16:39:35.762364+0000 mgr.gibba032.whopeo (mgr.24625) 8807 : cluster [DBG] pgmap v5011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:37.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:36 gibba011 bash[31774]: cluster 2021-03-10T16:39:35.762364+0000 mgr.gibba032.whopeo (mgr.24625) 8807 : cluster [DBG] pgmap v5011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:38 gibba032 bash[24004]: cluster 2021-03-10T16:39:37.763982+0000 mgr.gibba032.whopeo (mgr.24625) 8808 : cluster [DBG] pgmap v5012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:39.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:38 gibba011 bash[30791]: cluster 2021-03-10T16:39:37.763982+0000 mgr.gibba032.whopeo (mgr.24625) 8808 : cluster [DBG] pgmap v5012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:39.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:38 gibba011 bash[31774]: cluster 2021-03-10T16:39:37.763982+0000 mgr.gibba032.whopeo (mgr.24625) 8808 : cluster [DBG] pgmap v5012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:40 gibba032 bash[24004]: cluster 2021-03-10T16:39:39.765575+0000 mgr.gibba032.whopeo (mgr.24625) 8809 : cluster [DBG] pgmap v5013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:40 gibba011 bash[30791]: cluster 2021-03-10T16:39:39.765575+0000 mgr.gibba032.whopeo (mgr.24625) 8809 : cluster [DBG] pgmap v5013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:41.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:40 gibba011 bash[31774]: cluster 2021-03-10T16:39:39.765575+0000 mgr.gibba032.whopeo (mgr.24625) 8809 : cluster [DBG] pgmap v5013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:42 gibba032 bash[24004]: cluster 2021-03-10T16:39:41.767320+0000 mgr.gibba032.whopeo (mgr.24625) 8810 : cluster [DBG] pgmap v5014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:42 gibba011 bash[30791]: cluster 2021-03-10T16:39:41.767320+0000 mgr.gibba032.whopeo (mgr.24625) 8810 : cluster [DBG] pgmap v5014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:42 gibba011 bash[31774]: cluster 2021-03-10T16:39:41.767320+0000 mgr.gibba032.whopeo (mgr.24625) 8810 : cluster [DBG] pgmap v5014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:44 gibba032 bash[24004]: cluster 2021-03-10T16:39:43.768976+0000 mgr.gibba032.whopeo (mgr.24625) 8811 : cluster [DBG] pgmap v5015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:44 gibba011 bash[30791]: cluster 2021-03-10T16:39:43.768976+0000 mgr.gibba032.whopeo (mgr.24625) 8811 : cluster [DBG] pgmap v5015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:44 gibba011 bash[31774]: cluster 2021-03-10T16:39:43.768976+0000 mgr.gibba032.whopeo (mgr.24625) 8811 : cluster [DBG] pgmap v5015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:47.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:46 gibba032 bash[24004]: cluster 2021-03-10T16:39:45.770379+0000 mgr.gibba032.whopeo (mgr.24625) 8812 : cluster [DBG] pgmap v5016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:46 gibba011 bash[30791]: cluster 2021-03-10T16:39:45.770379+0000 mgr.gibba032.whopeo (mgr.24625) 8812 : cluster [DBG] pgmap v5016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:46 gibba011 bash[31774]: cluster 2021-03-10T16:39:45.770379+0000 mgr.gibba032.whopeo (mgr.24625) 8812 : cluster [DBG] pgmap v5016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:48.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:47 gibba032 bash[24004]: cephadm 2021-03-10T16:39:46.869734+0000 mgr.gibba032.whopeo (mgr.24625) 8813 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:48.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:47 gibba011 bash[30791]: cephadm 2021-03-10T16:39:46.869734+0000 mgr.gibba032.whopeo (mgr.24625) 8813 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:48.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:47 gibba011 bash[31774]: cephadm 2021-03-10T16:39:46.869734+0000 mgr.gibba032.whopeo (mgr.24625) 8813 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:39:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:48 gibba032 bash[24004]: cluster 2021-03-10T16:39:47.772174+0000 mgr.gibba032.whopeo (mgr.24625) 8814 : cluster [DBG] pgmap v5017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:48 gibba011 bash[30791]: cluster 2021-03-10T16:39:47.772174+0000 mgr.gibba032.whopeo (mgr.24625) 8814 : cluster [DBG] pgmap v5017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:48 gibba011 bash[31774]: cluster 2021-03-10T16:39:47.772174+0000 mgr.gibba032.whopeo (mgr.24625) 8814 : cluster [DBG] pgmap v5017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:50.121 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:39:50.347 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:39:50.348 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:39:50.348 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:39:50.348 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:39:50.348 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:39:50.348 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:39:50.349 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:39:50.349 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:39:50.349 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:39:50.349 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:39:50.349 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:39:50.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:39:50.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:39:50.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:39:50.350 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:39:50.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:39:50.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:39:50.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:39:50.351 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:39:50.351 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:39:50.352 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:39:50.595 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:39:50.596 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:39:50.596 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:39:50.596 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:50.596 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:39:50.597 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:39:50.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:50.597 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:39:50.597 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:39:50.598 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:39:50.598 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:39:50.598 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:39:50.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:39:50.598 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:39:50.599 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:39:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:50 gibba032 bash[24004]: cluster 2021-03-10T16:39:49.773526+0000 mgr.gibba032.whopeo (mgr.24625) 8815 : cluster [DBG] pgmap v5018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:50 gibba032 bash[24004]: audit 2021-03-10T16:39:50.598296+0000 mon.c (mon.1) 172 : audit [DBG] from='client.? 172.21.2.111:0/3286165371' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:50 gibba011 bash[30791]: cluster 2021-03-10T16:39:49.773526+0000 mgr.gibba032.whopeo (mgr.24625) 8815 : cluster [DBG] pgmap v5018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:51.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:50 gibba011 bash[30791]: audit 2021-03-10T16:39:50.598296+0000 mon.c (mon.1) 172 : audit [DBG] from='client.? 172.21.2.111:0/3286165371' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:50 gibba011 bash[31774]: cluster 2021-03-10T16:39:49.773526+0000 mgr.gibba032.whopeo (mgr.24625) 8815 : cluster [DBG] pgmap v5018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:50 gibba011 bash[31774]: audit 2021-03-10T16:39:50.598296+0000 mon.c (mon.1) 172 : audit [DBG] from='client.? 172.21.2.111:0/3286165371' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:39:52.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:51 gibba032 bash[24004]: audit 2021-03-10T16:39:50.112346+0000 mgr.gibba032.whopeo (mgr.24625) 8816 : audit [DBG] from='client.59941 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:52.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:51 gibba032 bash[24004]: audit 2021-03-10T16:39:50.336142+0000 mgr.gibba032.whopeo (mgr.24625) 8817 : audit [DBG] from='client.49965 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:52.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:51 gibba011 bash[30791]: audit 2021-03-10T16:39:50.112346+0000 mgr.gibba032.whopeo (mgr.24625) 8816 : audit [DBG] from='client.59941 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:52.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:51 gibba011 bash[30791]: audit 2021-03-10T16:39:50.336142+0000 mgr.gibba032.whopeo (mgr.24625) 8817 : audit [DBG] from='client.49965 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:52.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:51 gibba011 bash[31774]: audit 2021-03-10T16:39:50.112346+0000 mgr.gibba032.whopeo (mgr.24625) 8816 : audit [DBG] from='client.59941 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:52.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:51 gibba011 bash[31774]: audit 2021-03-10T16:39:50.336142+0000 mgr.gibba032.whopeo (mgr.24625) 8817 : audit [DBG] from='client.49965 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:39:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:52 gibba032 bash[24004]: cluster 2021-03-10T16:39:51.775382+0000 mgr.gibba032.whopeo (mgr.24625) 8818 : cluster [DBG] pgmap v5019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:52 gibba011 bash[30791]: cluster 2021-03-10T16:39:51.775382+0000 mgr.gibba032.whopeo (mgr.24625) 8818 : cluster [DBG] pgmap v5019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:53.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:52 gibba011 bash[31774]: cluster 2021-03-10T16:39:51.775382+0000 mgr.gibba032.whopeo (mgr.24625) 8818 : cluster [DBG] pgmap v5019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:54 gibba032 bash[24004]: cluster 2021-03-10T16:39:53.776963+0000 mgr.gibba032.whopeo (mgr.24625) 8819 : cluster [DBG] pgmap v5020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:54 gibba011 bash[30791]: cluster 2021-03-10T16:39:53.776963+0000 mgr.gibba032.whopeo (mgr.24625) 8819 : cluster [DBG] pgmap v5020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:54 gibba011 bash[31774]: cluster 2021-03-10T16:39:53.776963+0000 mgr.gibba032.whopeo (mgr.24625) 8819 : cluster [DBG] pgmap v5020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:56 gibba032 bash[24004]: cluster 2021-03-10T16:39:55.778300+0000 mgr.gibba032.whopeo (mgr.24625) 8820 : cluster [DBG] pgmap v5021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:56 gibba011 bash[30791]: cluster 2021-03-10T16:39:55.778300+0000 mgr.gibba032.whopeo (mgr.24625) 8820 : cluster [DBG] pgmap v5021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:56 gibba011 bash[31774]: cluster 2021-03-10T16:39:55.778300+0000 mgr.gibba032.whopeo (mgr.24625) 8820 : cluster [DBG] pgmap v5021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:58.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.118979+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:39:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.119655+0000 mon.a (mon.0) 3036 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:39:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.243465+0000 mon.a (mon.0) 3037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:39:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.244212+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:39:58.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.244947+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:39:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:57 gibba032 bash[24004]: audit 2021-03-10T16:39:57.245632+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:39:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.118979+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:39:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.119655+0000 mon.a (mon.0) 3036 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:39:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.243465+0000 mon.a (mon.0) 3037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:39:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.244212+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:39:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.244947+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:39:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[30791]: audit 2021-03-10T16:39:57.245632+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:39:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.118979+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:39:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.119655+0000 mon.a (mon.0) 3036 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:39:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.243465+0000 mon.a (mon.0) 3037 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:39:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.244212+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:39:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.244947+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:39:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:57 gibba011 bash[31774]: audit 2021-03-10T16:39:57.245632+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:39:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:39:58 gibba032 bash[24004]: cluster 2021-03-10T16:39:57.780199+0000 mgr.gibba032.whopeo (mgr.24625) 8821 : cluster [DBG] pgmap v5022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:39:58 gibba011 bash[30791]: cluster 2021-03-10T16:39:57.780199+0000 mgr.gibba032.whopeo (mgr.24625) 8821 : cluster [DBG] pgmap v5022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:39:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:39:58 gibba011 bash[31774]: cluster 2021-03-10T16:39:57.780199+0000 mgr.gibba032.whopeo (mgr.24625) 8821 : cluster [DBG] pgmap v5022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:00.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:00 gibba011 bash[30791]: debug 2021-03-10T16:39:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:40:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:00 gibba032 bash[24004]: cluster 2021-03-10T16:39:59.781733+0000 mgr.gibba032.whopeo (mgr.24625) 8822 : cluster [DBG] pgmap v5023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:00 gibba032 bash[24004]: cluster 2021-03-10T16:40:00.000090+0000 mon.a (mon.0) 3041 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:40:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:00 gibba011 bash[30791]: cluster 2021-03-10T16:39:59.781733+0000 mgr.gibba032.whopeo (mgr.24625) 8822 : cluster [DBG] pgmap v5023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:01.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:00 gibba011 bash[30791]: cluster 2021-03-10T16:40:00.000090+0000 mon.a (mon.0) 3041 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:40:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:00 gibba011 bash[31774]: cluster 2021-03-10T16:39:59.781733+0000 mgr.gibba032.whopeo (mgr.24625) 8822 : cluster [DBG] pgmap v5023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:01.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:00 gibba011 bash[31774]: cluster 2021-03-10T16:40:00.000090+0000 mon.a (mon.0) 3041 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:40:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:02 gibba032 bash[24004]: cluster 2021-03-10T16:40:01.783497+0000 mgr.gibba032.whopeo (mgr.24625) 8823 : cluster [DBG] pgmap v5024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:03.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:02 gibba032 bash[24004]: cephadm 2021-03-10T16:40:01.885775+0000 mgr.gibba032.whopeo (mgr.24625) 8824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:02 gibba011 bash[30791]: cluster 2021-03-10T16:40:01.783497+0000 mgr.gibba032.whopeo (mgr.24625) 8823 : cluster [DBG] pgmap v5024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:03.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:02 gibba011 bash[30791]: cephadm 2021-03-10T16:40:01.885775+0000 mgr.gibba032.whopeo (mgr.24625) 8824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:02 gibba011 bash[31774]: cluster 2021-03-10T16:40:01.783497+0000 mgr.gibba032.whopeo (mgr.24625) 8823 : cluster [DBG] pgmap v5024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:03.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:02 gibba011 bash[31774]: cephadm 2021-03-10T16:40:01.885775+0000 mgr.gibba032.whopeo (mgr.24625) 8824 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:05 gibba032 bash[24004]: cluster 2021-03-10T16:40:03.785134+0000 mgr.gibba032.whopeo (mgr.24625) 8825 : cluster [DBG] pgmap v5025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:05 gibba011 bash[30791]: cluster 2021-03-10T16:40:03.785134+0000 mgr.gibba032.whopeo (mgr.24625) 8825 : cluster [DBG] pgmap v5025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:05 gibba011 bash[31774]: cluster 2021-03-10T16:40:03.785134+0000 mgr.gibba032.whopeo (mgr.24625) 8825 : cluster [DBG] pgmap v5025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:07.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:07 gibba011 bash[30791]: cluster 2021-03-10T16:40:05.786578+0000 mgr.gibba032.whopeo (mgr.24625) 8826 : cluster [DBG] pgmap v5026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:07 gibba011 bash[31774]: cluster 2021-03-10T16:40:05.786578+0000 mgr.gibba032.whopeo (mgr.24625) 8826 : cluster [DBG] pgmap v5026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:07.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:07 gibba032 bash[24004]: cluster 2021-03-10T16:40:05.786578+0000 mgr.gibba032.whopeo (mgr.24625) 8826 : cluster [DBG] pgmap v5026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:08.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:08 gibba011 bash[30791]: cluster 2021-03-10T16:40:07.788348+0000 mgr.gibba032.whopeo (mgr.24625) 8827 : cluster [DBG] pgmap v5027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:08.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:08 gibba011 bash[31774]: cluster 2021-03-10T16:40:07.788348+0000 mgr.gibba032.whopeo (mgr.24625) 8827 : cluster [DBG] pgmap v5027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:08.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:08 gibba032 bash[24004]: cluster 2021-03-10T16:40:07.788348+0000 mgr.gibba032.whopeo (mgr.24625) 8827 : cluster [DBG] pgmap v5027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:10 gibba032 bash[24004]: cluster 2021-03-10T16:40:09.789740+0000 mgr.gibba032.whopeo (mgr.24625) 8828 : cluster [DBG] pgmap v5028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:10 gibba011 bash[30791]: cluster 2021-03-10T16:40:09.789740+0000 mgr.gibba032.whopeo (mgr.24625) 8828 : cluster [DBG] pgmap v5028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:10 gibba011 bash[31774]: cluster 2021-03-10T16:40:09.789740+0000 mgr.gibba032.whopeo (mgr.24625) 8828 : cluster [DBG] pgmap v5028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:13.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:12 gibba032 bash[24004]: cluster 2021-03-10T16:40:11.791562+0000 mgr.gibba032.whopeo (mgr.24625) 8829 : cluster [DBG] pgmap v5029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:12 gibba011 bash[30791]: cluster 2021-03-10T16:40:11.791562+0000 mgr.gibba032.whopeo (mgr.24625) 8829 : cluster [DBG] pgmap v5029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:12 gibba011 bash[31774]: cluster 2021-03-10T16:40:11.791562+0000 mgr.gibba032.whopeo (mgr.24625) 8829 : cluster [DBG] pgmap v5029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:14 gibba032 bash[24004]: cluster 2021-03-10T16:40:13.792934+0000 mgr.gibba032.whopeo (mgr.24625) 8830 : cluster [DBG] pgmap v5030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:14 gibba011 bash[30791]: cluster 2021-03-10T16:40:13.792934+0000 mgr.gibba032.whopeo (mgr.24625) 8830 : cluster [DBG] pgmap v5030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:14 gibba011 bash[31774]: cluster 2021-03-10T16:40:13.792934+0000 mgr.gibba032.whopeo (mgr.24625) 8830 : cluster [DBG] pgmap v5030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:16 gibba032 bash[24004]: cluster 2021-03-10T16:40:15.794373+0000 mgr.gibba032.whopeo (mgr.24625) 8831 : cluster [DBG] pgmap v5031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:16 gibba011 bash[30791]: cluster 2021-03-10T16:40:15.794373+0000 mgr.gibba032.whopeo (mgr.24625) 8831 : cluster [DBG] pgmap v5031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:16 gibba011 bash[31774]: cluster 2021-03-10T16:40:15.794373+0000 mgr.gibba032.whopeo (mgr.24625) 8831 : cluster [DBG] pgmap v5031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:18.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:17 gibba032 bash[24004]: cephadm 2021-03-10T16:40:16.900490+0000 mgr.gibba032.whopeo (mgr.24625) 8832 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:18.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:17 gibba011 bash[30791]: cephadm 2021-03-10T16:40:16.900490+0000 mgr.gibba032.whopeo (mgr.24625) 8832 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:18.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:17 gibba011 bash[31774]: cephadm 2021-03-10T16:40:16.900490+0000 mgr.gibba032.whopeo (mgr.24625) 8832 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:18 gibba032 bash[24004]: cluster 2021-03-10T16:40:17.796320+0000 mgr.gibba032.whopeo (mgr.24625) 8833 : cluster [DBG] pgmap v5032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:18 gibba011 bash[30791]: cluster 2021-03-10T16:40:17.796320+0000 mgr.gibba032.whopeo (mgr.24625) 8833 : cluster [DBG] pgmap v5032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:19.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:18 gibba011 bash[31774]: cluster 2021-03-10T16:40:17.796320+0000 mgr.gibba032.whopeo (mgr.24625) 8833 : cluster [DBG] pgmap v5032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:20.840 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:40:21.069 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:40:21.070 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:40:21.070 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 8m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:40:21.071 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:40:21.071 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:40:21.071 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:40:21.072 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:40:21.072 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:40:21.073 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:40:21.073 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:40:21.073 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 8m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:40:21.074 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:40:21.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:40:21.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:40:21.075 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:40:21.076 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:40:21.076 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:40:21.077 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:40:21.078 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:40:21.078 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 8m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:40:21.079 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:40:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:20 gibba032 bash[24004]: cluster 2021-03-10T16:40:19.797918+0000 mgr.gibba032.whopeo (mgr.24625) 8834 : cluster [DBG] pgmap v5033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:21.321 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:40:21.321 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:40:21.321 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:40:21.322 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:21.322 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:40:21.322 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:40:21.322 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:21.322 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:40:21.323 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:40:21.323 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:21.323 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:40:21.323 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:40:21.324 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:40:21.324 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:40:21.324 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:40:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:20 gibba011 bash[30791]: cluster 2021-03-10T16:40:19.797918+0000 mgr.gibba032.whopeo (mgr.24625) 8834 : cluster [DBG] pgmap v5033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:20 gibba011 bash[31774]: cluster 2021-03-10T16:40:19.797918+0000 mgr.gibba032.whopeo (mgr.24625) 8834 : cluster [DBG] pgmap v5033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:22.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:21 gibba032 bash[24004]: audit 2021-03-10T16:40:20.831911+0000 mgr.gibba032.whopeo (mgr.24625) 8835 : audit [DBG] from='client.49977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:22.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:21 gibba032 bash[24004]: audit 2021-03-10T16:40:21.320754+0000 mon.c (mon.1) 173 : audit [DBG] from='client.? 172.21.2.111:0/1597355285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:22.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:21 gibba011 bash[30791]: audit 2021-03-10T16:40:20.831911+0000 mgr.gibba032.whopeo (mgr.24625) 8835 : audit [DBG] from='client.49977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:22.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:21 gibba011 bash[30791]: audit 2021-03-10T16:40:21.320754+0000 mon.c (mon.1) 173 : audit [DBG] from='client.? 172.21.2.111:0/1597355285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:22.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:21 gibba011 bash[31774]: audit 2021-03-10T16:40:20.831911+0000 mgr.gibba032.whopeo (mgr.24625) 8835 : audit [DBG] from='client.49977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:22.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:21 gibba011 bash[31774]: audit 2021-03-10T16:40:21.320754+0000 mon.c (mon.1) 173 : audit [DBG] from='client.? 172.21.2.111:0/1597355285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:22 gibba032 bash[24004]: audit 2021-03-10T16:40:21.057644+0000 mgr.gibba032.whopeo (mgr.24625) 8836 : audit [DBG] from='client.49983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:22 gibba032 bash[24004]: cluster 2021-03-10T16:40:21.799521+0000 mgr.gibba032.whopeo (mgr.24625) 8837 : cluster [DBG] pgmap v5034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:22 gibba011 bash[30791]: audit 2021-03-10T16:40:21.057644+0000 mgr.gibba032.whopeo (mgr.24625) 8836 : audit [DBG] from='client.49983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:22 gibba011 bash[30791]: cluster 2021-03-10T16:40:21.799521+0000 mgr.gibba032.whopeo (mgr.24625) 8837 : cluster [DBG] pgmap v5034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:22 gibba011 bash[31774]: audit 2021-03-10T16:40:21.057644+0000 mgr.gibba032.whopeo (mgr.24625) 8836 : audit [DBG] from='client.49983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:22 gibba011 bash[31774]: cluster 2021-03-10T16:40:21.799521+0000 mgr.gibba032.whopeo (mgr.24625) 8837 : cluster [DBG] pgmap v5034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:24 gibba032 bash[24004]: cluster 2021-03-10T16:40:23.801067+0000 mgr.gibba032.whopeo (mgr.24625) 8838 : cluster [DBG] pgmap v5035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:24 gibba011 bash[30791]: cluster 2021-03-10T16:40:23.801067+0000 mgr.gibba032.whopeo (mgr.24625) 8838 : cluster [DBG] pgmap v5035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:25.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:24 gibba011 bash[31774]: cluster 2021-03-10T16:40:23.801067+0000 mgr.gibba032.whopeo (mgr.24625) 8838 : cluster [DBG] pgmap v5035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:26 gibba032 bash[24004]: cluster 2021-03-10T16:40:25.802408+0000 mgr.gibba032.whopeo (mgr.24625) 8839 : cluster [DBG] pgmap v5036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:26 gibba011 bash[30791]: cluster 2021-03-10T16:40:25.802408+0000 mgr.gibba032.whopeo (mgr.24625) 8839 : cluster [DBG] pgmap v5036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:26 gibba011 bash[31774]: cluster 2021-03-10T16:40:25.802408+0000 mgr.gibba032.whopeo (mgr.24625) 8839 : cluster [DBG] pgmap v5036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:28 gibba032 bash[24004]: cluster 2021-03-10T16:40:27.804095+0000 mgr.gibba032.whopeo (mgr.24625) 8840 : cluster [DBG] pgmap v5037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:28 gibba011 bash[30791]: cluster 2021-03-10T16:40:27.804095+0000 mgr.gibba032.whopeo (mgr.24625) 8840 : cluster [DBG] pgmap v5037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:28 gibba011 bash[31774]: cluster 2021-03-10T16:40:27.804095+0000 mgr.gibba032.whopeo (mgr.24625) 8840 : cluster [DBG] pgmap v5037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:30 gibba032 bash[24004]: cluster 2021-03-10T16:40:29.805329+0000 mgr.gibba032.whopeo (mgr.24625) 8841 : cluster [DBG] pgmap v5038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:30 gibba011 bash[30791]: cluster 2021-03-10T16:40:29.805329+0000 mgr.gibba032.whopeo (mgr.24625) 8841 : cluster [DBG] pgmap v5038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:30 gibba011 bash[31774]: cluster 2021-03-10T16:40:29.805329+0000 mgr.gibba032.whopeo (mgr.24625) 8841 : cluster [DBG] pgmap v5038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:32.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:31 gibba032 bash[24004]: audit 2021-03-10T16:40:31.908109+0000 mon.a (mon.0) 3042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:40:32.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:31 gibba011 bash[30791]: audit 2021-03-10T16:40:31.908109+0000 mon.a (mon.0) 3042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:40:32.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:31 gibba011 bash[31774]: audit 2021-03-10T16:40:31.908109+0000 mon.a (mon.0) 3042 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:40:33.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:32 gibba032 bash[24004]: cluster 2021-03-10T16:40:31.807073+0000 mgr.gibba032.whopeo (mgr.24625) 8842 : cluster [DBG] pgmap v5039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:33.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:32 gibba032 bash[24004]: cephadm 2021-03-10T16:40:31.905560+0000 mgr.gibba032.whopeo (mgr.24625) 8843 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:40:33.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:32 gibba011 bash[30791]: cluster 2021-03-10T16:40:31.807073+0000 mgr.gibba032.whopeo (mgr.24625) 8842 : cluster [DBG] pgmap v5039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:33.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:32 gibba011 bash[30791]: cephadm 2021-03-10T16:40:31.905560+0000 mgr.gibba032.whopeo (mgr.24625) 8843 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:40:33.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:32 gibba011 bash[31774]: cluster 2021-03-10T16:40:31.807073+0000 mgr.gibba032.whopeo (mgr.24625) 8842 : cluster [DBG] pgmap v5039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:33.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:32 gibba011 bash[31774]: cephadm 2021-03-10T16:40:31.905560+0000 mgr.gibba032.whopeo (mgr.24625) 8843 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:40:35.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:34 gibba032 bash[24004]: cluster 2021-03-10T16:40:33.808854+0000 mgr.gibba032.whopeo (mgr.24625) 8844 : cluster [DBG] pgmap v5040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:34 gibba011 bash[30791]: cluster 2021-03-10T16:40:33.808854+0000 mgr.gibba032.whopeo (mgr.24625) 8844 : cluster [DBG] pgmap v5040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:35.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:34 gibba011 bash[31774]: cluster 2021-03-10T16:40:33.808854+0000 mgr.gibba032.whopeo (mgr.24625) 8844 : cluster [DBG] pgmap v5040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.507085+0000 mgr.gibba032.whopeo (mgr.24625) 8845 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:40:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.508434+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.709782+0000 mon.a (mon.0) 3044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.721665+0000 mgr.gibba032.whopeo (mgr.24625) 8846 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:40:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.726438+0000 mon.a (mon.0) 3045 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:40:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.727292+0000 mgr.gibba032.whopeo (mgr.24625) 8847 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:40:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.730558+0000 mon.a (mon.0) 3046 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.730759+0000 mgr.gibba032.whopeo (mgr.24625) 8848 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:40:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.731019+0000 mgr.gibba032.whopeo (mgr.24625) 8849 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:40:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.733945+0000 mon.a (mon.0) 3047 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.734209+0000 mgr.gibba032.whopeo (mgr.24625) 8850 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:40:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.734472+0000 mgr.gibba032.whopeo (mgr.24625) 8851 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:40:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.737462+0000 mon.a (mon.0) 3048 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.737978+0000 mgr.gibba032.whopeo (mgr.24625) 8852 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:40:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.740988+0000 mgr.gibba032.whopeo (mgr.24625) 8853 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:40:36.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.741209+0000 mgr.gibba032.whopeo (mgr.24625) 8854 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:40:36.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.745047+0000 mon.a (mon.0) 3049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.745453+0000 mgr.gibba032.whopeo (mgr.24625) 8855 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:40:36.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.747373+0000 mgr.gibba032.whopeo (mgr.24625) 8856 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:40:36.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: audit 2021-03-10T16:40:35.750330+0000 mon.a (mon.0) 3050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.750669+0000 mgr.gibba032.whopeo (mgr.24625) 8857 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:40:36.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.507085+0000 mgr.gibba032.whopeo (mgr.24625) 8845 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:40:36.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.508434+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:36.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.709782+0000 mon.a (mon.0) 3044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:36.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.721665+0000 mgr.gibba032.whopeo (mgr.24625) 8846 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:40:36.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.726438+0000 mon.a (mon.0) 3045 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:40:36.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.727292+0000 mgr.gibba032.whopeo (mgr.24625) 8847 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:40:36.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.730558+0000 mon.a (mon.0) 3046 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.730759+0000 mgr.gibba032.whopeo (mgr.24625) 8848 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:40:36.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.731019+0000 mgr.gibba032.whopeo (mgr.24625) 8849 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:40:36.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.733945+0000 mon.a (mon.0) 3047 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.734209+0000 mgr.gibba032.whopeo (mgr.24625) 8850 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:40:36.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.734472+0000 mgr.gibba032.whopeo (mgr.24625) 8851 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:40:36.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.737462+0000 mon.a (mon.0) 3048 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.737978+0000 mgr.gibba032.whopeo (mgr.24625) 8852 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:40:36.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.740988+0000 mgr.gibba032.whopeo (mgr.24625) 8853 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:40:36.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.741209+0000 mgr.gibba032.whopeo (mgr.24625) 8854 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:40:36.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.745047+0000 mon.a (mon.0) 3049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.745453+0000 mgr.gibba032.whopeo (mgr.24625) 8855 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:40:36.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.747373+0000 mgr.gibba032.whopeo (mgr.24625) 8856 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:40:36.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: audit 2021-03-10T16:40:35.750330+0000 mon.a (mon.0) 3050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:36.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.750669+0000 mgr.gibba032.whopeo (mgr.24625) 8857 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:40:36.852 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.751234+0000 mgr.gibba032.whopeo (mgr.24625) 8858 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:40:36.853 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cephadm 2021-03-10T16:40:35.751851+0000 mgr.gibba032.whopeo (mgr.24625) 8859 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:36.853 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[30791]: cluster 2021-03-10T16:40:35.810355+0000 mgr.gibba032.whopeo (mgr.24625) 8860 : cluster [DBG] pgmap v5041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:36.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.751234+0000 mgr.gibba032.whopeo (mgr.24625) 8858 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:40:36.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cephadm 2021-03-10T16:40:35.751851+0000 mgr.gibba032.whopeo (mgr.24625) 8859 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:36.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:36 gibba011 bash[31774]: cluster 2021-03-10T16:40:35.810355+0000 mgr.gibba032.whopeo (mgr.24625) 8860 : cluster [DBG] pgmap v5041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.507085+0000 mgr.gibba032.whopeo (mgr.24625) 8845 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:40:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.508434+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:37.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.709782+0000 mon.a (mon.0) 3044 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:40:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.721665+0000 mgr.gibba032.whopeo (mgr.24625) 8846 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:40:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.726438+0000 mon.a (mon.0) 3045 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:40:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.727292+0000 mgr.gibba032.whopeo (mgr.24625) 8847 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:40:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.730558+0000 mon.a (mon.0) 3046 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.730759+0000 mgr.gibba032.whopeo (mgr.24625) 8848 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:40:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.731019+0000 mgr.gibba032.whopeo (mgr.24625) 8849 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:40:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.733945+0000 mon.a (mon.0) 3047 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.734209+0000 mgr.gibba032.whopeo (mgr.24625) 8850 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:40:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.734472+0000 mgr.gibba032.whopeo (mgr.24625) 8851 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:40:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.737462+0000 mon.a (mon.0) 3048 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.737978+0000 mgr.gibba032.whopeo (mgr.24625) 8852 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:40:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.740988+0000 mgr.gibba032.whopeo (mgr.24625) 8853 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:40:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.741209+0000 mgr.gibba032.whopeo (mgr.24625) 8854 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:40:37.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.745047+0000 mon.a (mon.0) 3049 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:37.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.745453+0000 mgr.gibba032.whopeo (mgr.24625) 8855 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:40:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.747373+0000 mgr.gibba032.whopeo (mgr.24625) 8856 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:40:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: audit 2021-03-10T16:40:35.750330+0000 mon.a (mon.0) 3050 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.750669+0000 mgr.gibba032.whopeo (mgr.24625) 8857 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:40:37.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.751234+0000 mgr.gibba032.whopeo (mgr.24625) 8858 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:40:37.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cephadm 2021-03-10T16:40:35.751851+0000 mgr.gibba032.whopeo (mgr.24625) 8859 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:37.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:36 gibba032 bash[24004]: cluster 2021-03-10T16:40:35.810355+0000 mgr.gibba032.whopeo (mgr.24625) 8860 : cluster [DBG] pgmap v5041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:39.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:38 gibba032 bash[24004]: cluster 2021-03-10T16:40:37.812303+0000 mgr.gibba032.whopeo (mgr.24625) 8861 : cluster [DBG] pgmap v5042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:39.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:38 gibba011 bash[30791]: cluster 2021-03-10T16:40:37.812303+0000 mgr.gibba032.whopeo (mgr.24625) 8861 : cluster [DBG] pgmap v5042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:39.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:38 gibba011 bash[31774]: cluster 2021-03-10T16:40:37.812303+0000 mgr.gibba032.whopeo (mgr.24625) 8861 : cluster [DBG] pgmap v5042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:40 gibba032 bash[24004]: cluster 2021-03-10T16:40:39.813903+0000 mgr.gibba032.whopeo (mgr.24625) 8862 : cluster [DBG] pgmap v5043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:40 gibba011 bash[30791]: cluster 2021-03-10T16:40:39.813903+0000 mgr.gibba032.whopeo (mgr.24625) 8862 : cluster [DBG] pgmap v5043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:41.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:40 gibba011 bash[31774]: cluster 2021-03-10T16:40:39.813903+0000 mgr.gibba032.whopeo (mgr.24625) 8862 : cluster [DBG] pgmap v5043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:42 gibba032 bash[24004]: cluster 2021-03-10T16:40:41.815569+0000 mgr.gibba032.whopeo (mgr.24625) 8863 : cluster [DBG] pgmap v5044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:42 gibba011 bash[30791]: cluster 2021-03-10T16:40:41.815569+0000 mgr.gibba032.whopeo (mgr.24625) 8863 : cluster [DBG] pgmap v5044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:42 gibba011 bash[31774]: cluster 2021-03-10T16:40:41.815569+0000 mgr.gibba032.whopeo (mgr.24625) 8863 : cluster [DBG] pgmap v5044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:44 gibba032 bash[24004]: cluster 2021-03-10T16:40:43.817171+0000 mgr.gibba032.whopeo (mgr.24625) 8864 : cluster [DBG] pgmap v5045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:44 gibba011 bash[30791]: cluster 2021-03-10T16:40:43.817171+0000 mgr.gibba032.whopeo (mgr.24625) 8864 : cluster [DBG] pgmap v5045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:44 gibba011 bash[31774]: cluster 2021-03-10T16:40:43.817171+0000 mgr.gibba032.whopeo (mgr.24625) 8864 : cluster [DBG] pgmap v5045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:46 gibba032 bash[24004]: cluster 2021-03-10T16:40:45.818526+0000 mgr.gibba032.whopeo (mgr.24625) 8865 : cluster [DBG] pgmap v5046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:46 gibba011 bash[30791]: cluster 2021-03-10T16:40:45.818526+0000 mgr.gibba032.whopeo (mgr.24625) 8865 : cluster [DBG] pgmap v5046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:46 gibba011 bash[31774]: cluster 2021-03-10T16:40:45.818526+0000 mgr.gibba032.whopeo (mgr.24625) 8865 : cluster [DBG] pgmap v5046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:48 gibba032 bash[24004]: cluster 2021-03-10T16:40:47.820329+0000 mgr.gibba032.whopeo (mgr.24625) 8866 : cluster [DBG] pgmap v5047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:48 gibba011 bash[30791]: cluster 2021-03-10T16:40:47.820329+0000 mgr.gibba032.whopeo (mgr.24625) 8866 : cluster [DBG] pgmap v5047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:48 gibba011 bash[31774]: cluster 2021-03-10T16:40:47.820329+0000 mgr.gibba032.whopeo (mgr.24625) 8866 : cluster [DBG] pgmap v5047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:50 gibba032 bash[24004]: cluster 2021-03-10T16:40:49.821743+0000 mgr.gibba032.whopeo (mgr.24625) 8867 : cluster [DBG] pgmap v5048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:50 gibba011 bash[30791]: cluster 2021-03-10T16:40:49.821743+0000 mgr.gibba032.whopeo (mgr.24625) 8867 : cluster [DBG] pgmap v5048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:50 gibba011 bash[31774]: cluster 2021-03-10T16:40:49.821743+0000 mgr.gibba032.whopeo (mgr.24625) 8867 : cluster [DBG] pgmap v5048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:51.562 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:40:51.788 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:40:51.788 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 16s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:40:51.789 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:40:51.789 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:40:51.790 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:40:51.790 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:40:51.791 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:40:51.791 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:40:51.791 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:40:51.792 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 16s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:40:51.792 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:40:51.793 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:40:51.793 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:40:51.793 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:40:51.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 16s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:40:51.794 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:40:51.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:40:51.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:40:51.795 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:40:51.796 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:40:51.796 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 16s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:40:52.039 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:40:52.039 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:40:52.040 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:40:52.040 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:52.040 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:40:52.041 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:40:52.041 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:52.041 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:40:52.042 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:40:52.042 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:40:52.043 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:40:52.043 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:40:52.043 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:40:52.044 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:40:52.044 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:40:52.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:51 gibba032 bash[24004]: cephadm 2021-03-10T16:40:50.767608+0000 mgr.gibba032.whopeo (mgr.24625) 8868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:52.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:51 gibba011 bash[30791]: cephadm 2021-03-10T16:40:50.767608+0000 mgr.gibba032.whopeo (mgr.24625) 8868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:52.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:51 gibba011 bash[31774]: cephadm 2021-03-10T16:40:50.767608+0000 mgr.gibba032.whopeo (mgr.24625) 8868 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:40:53.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:52 gibba032 bash[24004]: audit 2021-03-10T16:40:51.553748+0000 mgr.gibba032.whopeo (mgr.24625) 8869 : audit [DBG] from='client.59977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:52 gibba032 bash[24004]: audit 2021-03-10T16:40:51.777115+0000 mgr.gibba032.whopeo (mgr.24625) 8870 : audit [DBG] from='client.59983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:52 gibba032 bash[24004]: cluster 2021-03-10T16:40:51.822706+0000 mgr.gibba032.whopeo (mgr.24625) 8871 : cluster [DBG] pgmap v5049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:53.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:52 gibba032 bash[24004]: audit 2021-03-10T16:40:52.041375+0000 mon.a (mon.0) 3051 : audit [DBG] from='client.? 172.21.2.111:0/870407732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[30791]: audit 2021-03-10T16:40:51.553748+0000 mgr.gibba032.whopeo (mgr.24625) 8869 : audit [DBG] from='client.59977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[30791]: audit 2021-03-10T16:40:51.777115+0000 mgr.gibba032.whopeo (mgr.24625) 8870 : audit [DBG] from='client.59983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[30791]: cluster 2021-03-10T16:40:51.822706+0000 mgr.gibba032.whopeo (mgr.24625) 8871 : cluster [DBG] pgmap v5049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[30791]: audit 2021-03-10T16:40:52.041375+0000 mon.a (mon.0) 3051 : audit [DBG] from='client.? 172.21.2.111:0/870407732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[31774]: audit 2021-03-10T16:40:51.553748+0000 mgr.gibba032.whopeo (mgr.24625) 8869 : audit [DBG] from='client.59977 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[31774]: audit 2021-03-10T16:40:51.777115+0000 mgr.gibba032.whopeo (mgr.24625) 8870 : audit [DBG] from='client.59983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:40:53.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[31774]: cluster 2021-03-10T16:40:51.822706+0000 mgr.gibba032.whopeo (mgr.24625) 8871 : cluster [DBG] pgmap v5049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:53.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:52 gibba011 bash[31774]: audit 2021-03-10T16:40:52.041375+0000 mon.a (mon.0) 3051 : audit [DBG] from='client.? 172.21.2.111:0/870407732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:40:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:54 gibba032 bash[24004]: cluster 2021-03-10T16:40:53.823944+0000 mgr.gibba032.whopeo (mgr.24625) 8872 : cluster [DBG] pgmap v5050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:54 gibba011 bash[30791]: cluster 2021-03-10T16:40:53.823944+0000 mgr.gibba032.whopeo (mgr.24625) 8872 : cluster [DBG] pgmap v5050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:54 gibba011 bash[31774]: cluster 2021-03-10T16:40:53.823944+0000 mgr.gibba032.whopeo (mgr.24625) 8872 : cluster [DBG] pgmap v5050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:57.254 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:56 gibba032 bash[24004]: cluster 2021-03-10T16:40:55.825358+0000 mgr.gibba032.whopeo (mgr.24625) 8873 : cluster [DBG] pgmap v5051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:56 gibba011 bash[30791]: cluster 2021-03-10T16:40:55.825358+0000 mgr.gibba032.whopeo (mgr.24625) 8873 : cluster [DBG] pgmap v5051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:56 gibba011 bash[31774]: cluster 2021-03-10T16:40:55.825358+0000 mgr.gibba032.whopeo (mgr.24625) 8873 : cluster [DBG] pgmap v5051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.125039+0000 mon.a (mon.0) 3052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:40:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.125716+0000 mon.a (mon.0) 3053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:40:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.252130+0000 mon.a (mon.0) 3054 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:40:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.252879+0000 mon.a (mon.0) 3055 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:40:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.253545+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:40:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:57 gibba032 bash[24004]: audit 2021-03-10T16:40:57.254216+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:40:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.125039+0000 mon.a (mon.0) 3052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:40:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.125716+0000 mon.a (mon.0) 3053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:40:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.252130+0000 mon.a (mon.0) 3054 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:40:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.252879+0000 mon.a (mon.0) 3055 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:40:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.253545+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:40:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[30791]: audit 2021-03-10T16:40:57.254216+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:40:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.125039+0000 mon.a (mon.0) 3052 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:40:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.125716+0000 mon.a (mon.0) 3053 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:40:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.252130+0000 mon.a (mon.0) 3054 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:40:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.252879+0000 mon.a (mon.0) 3055 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:40:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.253545+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T16:40:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:57 gibba011 bash[31774]: audit 2021-03-10T16:40:57.254216+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:40:59.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:40:58 gibba032 bash[24004]: cluster 2021-03-10T16:40:57.826836+0000 mgr.gibba032.whopeo (mgr.24625) 8874 : cluster [DBG] pgmap v5052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:40:58 gibba011 bash[30791]: cluster 2021-03-10T16:40:57.826836+0000 mgr.gibba032.whopeo (mgr.24625) 8874 : cluster [DBG] pgmap v5052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:40:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:40:58 gibba011 bash[31774]: cluster 2021-03-10T16:40:57.826836+0000 mgr.gibba032.whopeo (mgr.24625) 8874 : cluster [DBG] pgmap v5052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:01.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:00 gibba032 bash[24004]: cluster 2021-03-10T16:40:59.828348+0000 mgr.gibba032.whopeo (mgr.24625) 8875 : cluster [DBG] pgmap v5053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:00 gibba011 bash[30791]: cluster 2021-03-10T16:40:59.828348+0000 mgr.gibba032.whopeo (mgr.24625) 8875 : cluster [DBG] pgmap v5053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:00 gibba011 bash[31774]: cluster 2021-03-10T16:40:59.828348+0000 mgr.gibba032.whopeo (mgr.24625) 8875 : cluster [DBG] pgmap v5053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:03.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:02 gibba032 bash[24004]: cluster 2021-03-10T16:41:01.830148+0000 mgr.gibba032.whopeo (mgr.24625) 8876 : cluster [DBG] pgmap v5054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:02 gibba011 bash[30791]: cluster 2021-03-10T16:41:01.830148+0000 mgr.gibba032.whopeo (mgr.24625) 8876 : cluster [DBG] pgmap v5054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:02 gibba011 bash[31774]: cluster 2021-03-10T16:41:01.830148+0000 mgr.gibba032.whopeo (mgr.24625) 8876 : cluster [DBG] pgmap v5054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:04 gibba032 bash[24004]: cluster 2021-03-10T16:41:03.831898+0000 mgr.gibba032.whopeo (mgr.24625) 8877 : cluster [DBG] pgmap v5055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:04 gibba011 bash[30791]: cluster 2021-03-10T16:41:03.831898+0000 mgr.gibba032.whopeo (mgr.24625) 8877 : cluster [DBG] pgmap v5055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:04 gibba011 bash[31774]: cluster 2021-03-10T16:41:03.831898+0000 mgr.gibba032.whopeo (mgr.24625) 8877 : cluster [DBG] pgmap v5055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:07 gibba032 bash[24004]: cephadm 2021-03-10T16:41:05.780815+0000 mgr.gibba032.whopeo (mgr.24625) 8878 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:07.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:07 gibba032 bash[24004]: cluster 2021-03-10T16:41:05.833225+0000 mgr.gibba032.whopeo (mgr.24625) 8879 : cluster [DBG] pgmap v5056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:07.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:07 gibba011 bash[30791]: cephadm 2021-03-10T16:41:05.780815+0000 mgr.gibba032.whopeo (mgr.24625) 8878 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:07.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:07 gibba011 bash[30791]: cluster 2021-03-10T16:41:05.833225+0000 mgr.gibba032.whopeo (mgr.24625) 8879 : cluster [DBG] pgmap v5056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:07.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:07 gibba011 bash[31774]: cephadm 2021-03-10T16:41:05.780815+0000 mgr.gibba032.whopeo (mgr.24625) 8878 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:07.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:07 gibba011 bash[31774]: cluster 2021-03-10T16:41:05.833225+0000 mgr.gibba032.whopeo (mgr.24625) 8879 : cluster [DBG] pgmap v5056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:09 gibba011 bash[30791]: cluster 2021-03-10T16:41:07.834880+0000 mgr.gibba032.whopeo (mgr.24625) 8880 : cluster [DBG] pgmap v5057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:09 gibba011 bash[31774]: cluster 2021-03-10T16:41:07.834880+0000 mgr.gibba032.whopeo (mgr.24625) 8880 : cluster [DBG] pgmap v5057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:09.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:09 gibba032 bash[24004]: cluster 2021-03-10T16:41:07.834880+0000 mgr.gibba032.whopeo (mgr.24625) 8880 : cluster [DBG] pgmap v5057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:11.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:11 gibba011 bash[30791]: cluster 2021-03-10T16:41:09.836399+0000 mgr.gibba032.whopeo (mgr.24625) 8881 : cluster [DBG] pgmap v5058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:11.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:11 gibba011 bash[31774]: cluster 2021-03-10T16:41:09.836399+0000 mgr.gibba032.whopeo (mgr.24625) 8881 : cluster [DBG] pgmap v5058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:11 gibba032 bash[24004]: cluster 2021-03-10T16:41:09.836399+0000 mgr.gibba032.whopeo (mgr.24625) 8881 : cluster [DBG] pgmap v5058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:13 gibba011 bash[30791]: cluster 2021-03-10T16:41:11.837836+0000 mgr.gibba032.whopeo (mgr.24625) 8882 : cluster [DBG] pgmap v5059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:13 gibba011 bash[31774]: cluster 2021-03-10T16:41:11.837836+0000 mgr.gibba032.whopeo (mgr.24625) 8882 : cluster [DBG] pgmap v5059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:13.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:13 gibba032 bash[24004]: cluster 2021-03-10T16:41:11.837836+0000 mgr.gibba032.whopeo (mgr.24625) 8882 : cluster [DBG] pgmap v5059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:15 gibba011 bash[30791]: cluster 2021-03-10T16:41:13.839191+0000 mgr.gibba032.whopeo (mgr.24625) 8883 : cluster [DBG] pgmap v5060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:15 gibba011 bash[31774]: cluster 2021-03-10T16:41:13.839191+0000 mgr.gibba032.whopeo (mgr.24625) 8883 : cluster [DBG] pgmap v5060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:15.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:15 gibba032 bash[24004]: cluster 2021-03-10T16:41:13.839191+0000 mgr.gibba032.whopeo (mgr.24625) 8883 : cluster [DBG] pgmap v5060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:16.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:16 gibba032 bash[24004]: cluster 2021-03-10T16:41:15.840470+0000 mgr.gibba032.whopeo (mgr.24625) 8884 : cluster [DBG] pgmap v5061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:16.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:16 gibba011 bash[30791]: cluster 2021-03-10T16:41:15.840470+0000 mgr.gibba032.whopeo (mgr.24625) 8884 : cluster [DBG] pgmap v5061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:16.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:16 gibba011 bash[31774]: cluster 2021-03-10T16:41:15.840470+0000 mgr.gibba032.whopeo (mgr.24625) 8884 : cluster [DBG] pgmap v5061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:19.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:18 gibba032 bash[24004]: cluster 2021-03-10T16:41:17.841945+0000 mgr.gibba032.whopeo (mgr.24625) 8885 : cluster [DBG] pgmap v5062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:18 gibba011 bash[30791]: cluster 2021-03-10T16:41:17.841945+0000 mgr.gibba032.whopeo (mgr.24625) 8885 : cluster [DBG] pgmap v5062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:19.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:18 gibba011 bash[31774]: cluster 2021-03-10T16:41:17.841945+0000 mgr.gibba032.whopeo (mgr.24625) 8885 : cluster [DBG] pgmap v5062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:21.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:20 gibba032 bash[24004]: cluster 2021-03-10T16:41:19.843339+0000 mgr.gibba032.whopeo (mgr.24625) 8886 : cluster [DBG] pgmap v5063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:20 gibba011 bash[30791]: cluster 2021-03-10T16:41:19.843339+0000 mgr.gibba032.whopeo (mgr.24625) 8886 : cluster [DBG] pgmap v5063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:20 gibba011 bash[31774]: cluster 2021-03-10T16:41:19.843339+0000 mgr.gibba032.whopeo (mgr.24625) 8886 : cluster [DBG] pgmap v5063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:22.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:21 gibba032 bash[24004]: cephadm 2021-03-10T16:41:20.792770+0000 mgr.gibba032.whopeo (mgr.24625) 8887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:22.280 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:41:22.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:21 gibba011 bash[30791]: cephadm 2021-03-10T16:41:20.792770+0000 mgr.gibba032.whopeo (mgr.24625) 8887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:22.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:21 gibba011 bash[31774]: cephadm 2021-03-10T16:41:20.792770+0000 mgr.gibba032.whopeo (mgr.24625) 8887 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:22.503 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:41:22.504 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 46s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:41:22.504 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 9m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:41:22.505 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:41:22.505 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:41:22.505 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 47s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:41:22.506 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 47s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:41:22.506 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:41:22.507 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 47s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:41:22.507 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 46s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:41:22.508 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 9m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:41:22.509 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 47s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:41:22.509 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 46s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:41:22.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 46s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:41:22.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 46s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:41:22.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:41:22.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:41:22.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:41:22.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:41:22.512 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 9m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:41:22.512 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 46s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:41:22.756 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:41:22.757 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:41:22.757 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:41:22.757 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:22.757 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:41:22.758 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:41:22.758 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:22.758 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:41:22.758 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:41:22.759 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:22.759 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:41:22.759 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:41:22.759 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:41:22.759 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:41:22.760 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:41:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:22 gibba032 bash[24004]: cluster 2021-03-10T16:41:21.845140+0000 mgr.gibba032.whopeo (mgr.24625) 8888 : cluster [DBG] pgmap v5064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:22 gibba032 bash[24004]: audit 2021-03-10T16:41:22.759177+0000 mon.c (mon.1) 174 : audit [DBG] from='client.? 172.21.2.111:0/3005867327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:22 gibba011 bash[30791]: cluster 2021-03-10T16:41:21.845140+0000 mgr.gibba032.whopeo (mgr.24625) 8888 : cluster [DBG] pgmap v5064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:22 gibba011 bash[30791]: audit 2021-03-10T16:41:22.759177+0000 mon.c (mon.1) 174 : audit [DBG] from='client.? 172.21.2.111:0/3005867327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:22 gibba011 bash[31774]: cluster 2021-03-10T16:41:21.845140+0000 mgr.gibba032.whopeo (mgr.24625) 8888 : cluster [DBG] pgmap v5064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:22 gibba011 bash[31774]: audit 2021-03-10T16:41:22.759177+0000 mon.c (mon.1) 174 : audit [DBG] from='client.? 172.21.2.111:0/3005867327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:24.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:23 gibba032 bash[24004]: audit 2021-03-10T16:41:22.272627+0000 mgr.gibba032.whopeo (mgr.24625) 8889 : audit [DBG] from='client.59995 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:24.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:23 gibba032 bash[24004]: audit 2021-03-10T16:41:22.495841+0000 mgr.gibba032.whopeo (mgr.24625) 8890 : audit [DBG] from='client.50019 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:24.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:23 gibba011 bash[30791]: audit 2021-03-10T16:41:22.272627+0000 mgr.gibba032.whopeo (mgr.24625) 8889 : audit [DBG] from='client.59995 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:24.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:23 gibba011 bash[30791]: audit 2021-03-10T16:41:22.495841+0000 mgr.gibba032.whopeo (mgr.24625) 8890 : audit [DBG] from='client.50019 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:23 gibba011 bash[31774]: audit 2021-03-10T16:41:22.272627+0000 mgr.gibba032.whopeo (mgr.24625) 8889 : audit [DBG] from='client.59995 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:24.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:23 gibba011 bash[31774]: audit 2021-03-10T16:41:22.495841+0000 mgr.gibba032.whopeo (mgr.24625) 8890 : audit [DBG] from='client.50019 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:24 gibba032 bash[24004]: cluster 2021-03-10T16:41:23.846698+0000 mgr.gibba032.whopeo (mgr.24625) 8891 : cluster [DBG] pgmap v5065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:24 gibba011 bash[30791]: cluster 2021-03-10T16:41:23.846698+0000 mgr.gibba032.whopeo (mgr.24625) 8891 : cluster [DBG] pgmap v5065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:25.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:24 gibba011 bash[31774]: cluster 2021-03-10T16:41:23.846698+0000 mgr.gibba032.whopeo (mgr.24625) 8891 : cluster [DBG] pgmap v5065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:26 gibba032 bash[24004]: cluster 2021-03-10T16:41:25.848164+0000 mgr.gibba032.whopeo (mgr.24625) 8892 : cluster [DBG] pgmap v5066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:27.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:26 gibba011 bash[30791]: cluster 2021-03-10T16:41:25.848164+0000 mgr.gibba032.whopeo (mgr.24625) 8892 : cluster [DBG] pgmap v5066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:26 gibba011 bash[31774]: cluster 2021-03-10T16:41:25.848164+0000 mgr.gibba032.whopeo (mgr.24625) 8892 : cluster [DBG] pgmap v5066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:29.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:28 gibba032 bash[24004]: cluster 2021-03-10T16:41:27.849679+0000 mgr.gibba032.whopeo (mgr.24625) 8893 : cluster [DBG] pgmap v5067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:28 gibba011 bash[30791]: cluster 2021-03-10T16:41:27.849679+0000 mgr.gibba032.whopeo (mgr.24625) 8893 : cluster [DBG] pgmap v5067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:28 gibba011 bash[31774]: cluster 2021-03-10T16:41:27.849679+0000 mgr.gibba032.whopeo (mgr.24625) 8893 : cluster [DBG] pgmap v5067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:30 gibba032 bash[24004]: cluster 2021-03-10T16:41:29.851167+0000 mgr.gibba032.whopeo (mgr.24625) 8894 : cluster [DBG] pgmap v5068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:31.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:30 gibba011 bash[30791]: cluster 2021-03-10T16:41:29.851167+0000 mgr.gibba032.whopeo (mgr.24625) 8894 : cluster [DBG] pgmap v5068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:31.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:30 gibba011 bash[31774]: cluster 2021-03-10T16:41:29.851167+0000 mgr.gibba032.whopeo (mgr.24625) 8894 : cluster [DBG] pgmap v5068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:33.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:32 gibba032 bash[24004]: cluster 2021-03-10T16:41:31.852945+0000 mgr.gibba032.whopeo (mgr.24625) 8895 : cluster [DBG] pgmap v5069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:33.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:32 gibba011 bash[30791]: cluster 2021-03-10T16:41:31.852945+0000 mgr.gibba032.whopeo (mgr.24625) 8895 : cluster [DBG] pgmap v5069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:33.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:32 gibba011 bash[31774]: cluster 2021-03-10T16:41:31.852945+0000 mgr.gibba032.whopeo (mgr.24625) 8895 : cluster [DBG] pgmap v5069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:35.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:34 gibba032 bash[24004]: cluster 2021-03-10T16:41:33.854532+0000 mgr.gibba032.whopeo (mgr.24625) 8896 : cluster [DBG] pgmap v5070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:34 gibba011 bash[30791]: cluster 2021-03-10T16:41:33.854532+0000 mgr.gibba032.whopeo (mgr.24625) 8896 : cluster [DBG] pgmap v5070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:35.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:34 gibba011 bash[31774]: cluster 2021-03-10T16:41:33.854532+0000 mgr.gibba032.whopeo (mgr.24625) 8896 : cluster [DBG] pgmap v5070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:36.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:35 gibba032 bash[24004]: audit 2021-03-10T16:41:35.812405+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:41:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:35 gibba011 bash[30791]: audit 2021-03-10T16:41:35.812405+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:41:36.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:35 gibba011 bash[31774]: audit 2021-03-10T16:41:35.812405+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:41:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:36 gibba032 bash[24004]: cephadm 2021-03-10T16:41:35.808408+0000 mgr.gibba032.whopeo (mgr.24625) 8897 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:41:37.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:36 gibba032 bash[24004]: cluster 2021-03-10T16:41:35.855560+0000 mgr.gibba032.whopeo (mgr.24625) 8898 : cluster [DBG] pgmap v5071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:37.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:37 gibba011 bash[30791]: cephadm 2021-03-10T16:41:35.808408+0000 mgr.gibba032.whopeo (mgr.24625) 8897 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:41:37.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:37 gibba011 bash[30791]: cluster 2021-03-10T16:41:35.855560+0000 mgr.gibba032.whopeo (mgr.24625) 8898 : cluster [DBG] pgmap v5071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:37.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:37 gibba011 bash[31774]: cephadm 2021-03-10T16:41:35.808408+0000 mgr.gibba032.whopeo (mgr.24625) 8897 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:41:37.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:37 gibba011 bash[31774]: cluster 2021-03-10T16:41:35.855560+0000 mgr.gibba032.whopeo (mgr.24625) 8898 : cluster [DBG] pgmap v5071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:39.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:39 gibba011 bash[30791]: cluster 2021-03-10T16:41:37.857171+0000 mgr.gibba032.whopeo (mgr.24625) 8899 : cluster [DBG] pgmap v5072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:39.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:39 gibba011 bash[31774]: cluster 2021-03-10T16:41:37.857171+0000 mgr.gibba032.whopeo (mgr.24625) 8899 : cluster [DBG] pgmap v5072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:39.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:39 gibba032 bash[24004]: cluster 2021-03-10T16:41:37.857171+0000 mgr.gibba032.whopeo (mgr.24625) 8899 : cluster [DBG] pgmap v5072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:40.457 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.452302+0000 mgr.gibba032.whopeo (mgr.24625) 8900 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:41:40.457 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.453428+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.457 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.662353+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.452302+0000 mgr.gibba032.whopeo (mgr.24625) 8900 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:41:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.453428+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.662353+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.663094+0000 mgr.gibba032.whopeo (mgr.24625) 8901 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:41:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.665724+0000 mgr.gibba032.whopeo (mgr.24625) 8902 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:41:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.666042+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:41:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.667617+0000 mgr.gibba032.whopeo (mgr.24625) 8903 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:41:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.667653+0000 mgr.gibba032.whopeo (mgr.24625) 8904 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:41:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.668183+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.669715+0000 mgr.gibba032.whopeo (mgr.24625) 8905 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:41:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.669752+0000 mgr.gibba032.whopeo (mgr.24625) 8906 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:41:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.670216+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.671584+0000 mgr.gibba032.whopeo (mgr.24625) 8907 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:41:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.672155+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.673399+0000 mgr.gibba032.whopeo (mgr.24625) 8908 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:41:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.673434+0000 mgr.gibba032.whopeo (mgr.24625) 8909 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:41:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.675152+0000 mgr.gibba032.whopeo (mgr.24625) 8910 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:41:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.675465+0000 mgr.gibba032.whopeo (mgr.24625) 8911 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:41:40.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.675916+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.677225+0000 mgr.gibba032.whopeo (mgr.24625) 8912 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:41:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.677263+0000 mgr.gibba032.whopeo (mgr.24625) 8913 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:41:40.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cephadm 2021-03-10T16:41:39.677438+0000 mgr.gibba032.whopeo (mgr.24625) 8914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: audit 2021-03-10T16:41:39.677889+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:40 gibba032 bash[24004]: cluster 2021-03-10T16:41:39.858567+0000 mgr.gibba032.whopeo (mgr.24625) 8915 : cluster [DBG] pgmap v5073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.663094+0000 mgr.gibba032.whopeo (mgr.24625) 8901 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:41:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.665724+0000 mgr.gibba032.whopeo (mgr.24625) 8902 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:41:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.666042+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:41:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.667617+0000 mgr.gibba032.whopeo (mgr.24625) 8903 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:41:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.667653+0000 mgr.gibba032.whopeo (mgr.24625) 8904 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:41:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.668183+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.669715+0000 mgr.gibba032.whopeo (mgr.24625) 8905 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:41:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.669752+0000 mgr.gibba032.whopeo (mgr.24625) 8906 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:41:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.670216+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.671584+0000 mgr.gibba032.whopeo (mgr.24625) 8907 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:41:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.672155+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.673399+0000 mgr.gibba032.whopeo (mgr.24625) 8908 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:41:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.673434+0000 mgr.gibba032.whopeo (mgr.24625) 8909 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:41:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.675152+0000 mgr.gibba032.whopeo (mgr.24625) 8910 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:41:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.675465+0000 mgr.gibba032.whopeo (mgr.24625) 8911 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:41:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.675916+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.677225+0000 mgr.gibba032.whopeo (mgr.24625) 8912 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:41:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.677263+0000 mgr.gibba032.whopeo (mgr.24625) 8913 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:41:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cephadm 2021-03-10T16:41:39.677438+0000 mgr.gibba032.whopeo (mgr.24625) 8914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:40.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: audit 2021-03-10T16:41:39.677889+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[30791]: cluster 2021-03-10T16:41:39.858567+0000 mgr.gibba032.whopeo (mgr.24625) 8915 : cluster [DBG] pgmap v5073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.452302+0000 mgr.gibba032.whopeo (mgr.24625) 8900 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:41:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.453428+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.662353+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:41:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.663094+0000 mgr.gibba032.whopeo (mgr.24625) 8901 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:41:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.665724+0000 mgr.gibba032.whopeo (mgr.24625) 8902 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:41:40.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.666042+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:41:40.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.667617+0000 mgr.gibba032.whopeo (mgr.24625) 8903 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:41:40.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.667653+0000 mgr.gibba032.whopeo (mgr.24625) 8904 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:41:40.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.668183+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.669715+0000 mgr.gibba032.whopeo (mgr.24625) 8905 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:41:40.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.669752+0000 mgr.gibba032.whopeo (mgr.24625) 8906 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:41:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.670216+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.671584+0000 mgr.gibba032.whopeo (mgr.24625) 8907 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:41:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.672155+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.673399+0000 mgr.gibba032.whopeo (mgr.24625) 8908 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:41:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.673434+0000 mgr.gibba032.whopeo (mgr.24625) 8909 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:41:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.675152+0000 mgr.gibba032.whopeo (mgr.24625) 8910 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:41:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.675465+0000 mgr.gibba032.whopeo (mgr.24625) 8911 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:41:40.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.675916+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.677225+0000 mgr.gibba032.whopeo (mgr.24625) 8912 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:41:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.677263+0000 mgr.gibba032.whopeo (mgr.24625) 8913 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:41:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cephadm 2021-03-10T16:41:39.677438+0000 mgr.gibba032.whopeo (mgr.24625) 8914 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: audit 2021-03-10T16:41:39.677889+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:40.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:40 gibba011 bash[31774]: cluster 2021-03-10T16:41:39.858567+0000 mgr.gibba032.whopeo (mgr.24625) 8915 : cluster [DBG] pgmap v5073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:42 gibba032 bash[24004]: cluster 2021-03-10T16:41:41.860240+0000 mgr.gibba032.whopeo (mgr.24625) 8916 : cluster [DBG] pgmap v5074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:42 gibba011 bash[30791]: cluster 2021-03-10T16:41:41.860240+0000 mgr.gibba032.whopeo (mgr.24625) 8916 : cluster [DBG] pgmap v5074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:42 gibba011 bash[31774]: cluster 2021-03-10T16:41:41.860240+0000 mgr.gibba032.whopeo (mgr.24625) 8916 : cluster [DBG] pgmap v5074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:45.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:44 gibba032 bash[24004]: cluster 2021-03-10T16:41:43.861595+0000 mgr.gibba032.whopeo (mgr.24625) 8917 : cluster [DBG] pgmap v5075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:44 gibba011 bash[30791]: cluster 2021-03-10T16:41:43.861595+0000 mgr.gibba032.whopeo (mgr.24625) 8917 : cluster [DBG] pgmap v5075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:44 gibba011 bash[31774]: cluster 2021-03-10T16:41:43.861595+0000 mgr.gibba032.whopeo (mgr.24625) 8917 : cluster [DBG] pgmap v5075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:47.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:46 gibba032 bash[24004]: cluster 2021-03-10T16:41:45.862990+0000 mgr.gibba032.whopeo (mgr.24625) 8918 : cluster [DBG] pgmap v5076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:46 gibba011 bash[30791]: cluster 2021-03-10T16:41:45.862990+0000 mgr.gibba032.whopeo (mgr.24625) 8918 : cluster [DBG] pgmap v5076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:46 gibba011 bash[31774]: cluster 2021-03-10T16:41:45.862990+0000 mgr.gibba032.whopeo (mgr.24625) 8918 : cluster [DBG] pgmap v5076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:48 gibba032 bash[24004]: cluster 2021-03-10T16:41:47.864715+0000 mgr.gibba032.whopeo (mgr.24625) 8919 : cluster [DBG] pgmap v5077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:48 gibba011 bash[30791]: cluster 2021-03-10T16:41:47.864715+0000 mgr.gibba032.whopeo (mgr.24625) 8919 : cluster [DBG] pgmap v5077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:48 gibba011 bash[31774]: cluster 2021-03-10T16:41:47.864715+0000 mgr.gibba032.whopeo (mgr.24625) 8919 : cluster [DBG] pgmap v5077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:50 gibba032 bash[24004]: cluster 2021-03-10T16:41:49.866167+0000 mgr.gibba032.whopeo (mgr.24625) 8920 : cluster [DBG] pgmap v5078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:50 gibba011 bash[30791]: cluster 2021-03-10T16:41:49.866167+0000 mgr.gibba032.whopeo (mgr.24625) 8920 : cluster [DBG] pgmap v5078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:50 gibba011 bash[31774]: cluster 2021-03-10T16:41:49.866167+0000 mgr.gibba032.whopeo (mgr.24625) 8920 : cluster [DBG] pgmap v5078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:52.999 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:41:53.234 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:41:53.236 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 77s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:41:53.236 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 13s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:41:53.236 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:41:53.237 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:41:53.237 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:41:53.238 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:41:53.239 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:41:53.239 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:41:53.239 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 77s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:41:53.241 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 13s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:41:53.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:41:53.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:41:53.242 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:41:53.244 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 77s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:41:53.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:41:53.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:41:53.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:41:53.246 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 13s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:41:53.246 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 13s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:41:53.246 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 77s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:41:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:52 gibba032 bash[24004]: cluster 2021-03-10T16:41:51.867846+0000 mgr.gibba032.whopeo (mgr.24625) 8921 : cluster [DBG] pgmap v5079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:52 gibba011 bash[30791]: cluster 2021-03-10T16:41:51.867846+0000 mgr.gibba032.whopeo (mgr.24625) 8921 : cluster [DBG] pgmap v5079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:52 gibba011 bash[31774]: cluster 2021-03-10T16:41:51.867846+0000 mgr.gibba032.whopeo (mgr.24625) 8921 : cluster [DBG] pgmap v5079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:53.490 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:41:53.490 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:41:53.491 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:41:53.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:53.492 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:41:53.492 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:41:53.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:53.493 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:41:53.493 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:41:53.494 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:41:53.494 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:41:53.494 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:41:53.495 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:41:53.495 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:41:53.496 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:41:54.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:53 gibba032 bash[24004]: audit 2021-03-10T16:41:52.991509+0000 mgr.gibba032.whopeo (mgr.24625) 8922 : audit [DBG] from='client.60013 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:54.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:53 gibba032 bash[24004]: audit 2021-03-10T16:41:53.492473+0000 mon.c (mon.1) 175 : audit [DBG] from='client.? 172.21.2.111:0/3724543447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:54.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:53 gibba011 bash[30791]: audit 2021-03-10T16:41:52.991509+0000 mgr.gibba032.whopeo (mgr.24625) 8922 : audit [DBG] from='client.60013 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:54.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:53 gibba011 bash[30791]: audit 2021-03-10T16:41:53.492473+0000 mon.c (mon.1) 175 : audit [DBG] from='client.? 172.21.2.111:0/3724543447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:54.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:53 gibba011 bash[31774]: audit 2021-03-10T16:41:52.991509+0000 mgr.gibba032.whopeo (mgr.24625) 8922 : audit [DBG] from='client.60013 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:54.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:53 gibba011 bash[31774]: audit 2021-03-10T16:41:53.492473+0000 mon.c (mon.1) 175 : audit [DBG] from='client.? 172.21.2.111:0/3724543447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:41:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:54 gibba032 bash[24004]: audit 2021-03-10T16:41:53.223171+0000 mgr.gibba032.whopeo (mgr.24625) 8923 : audit [DBG] from='client.50037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:54 gibba032 bash[24004]: cluster 2021-03-10T16:41:53.869348+0000 mgr.gibba032.whopeo (mgr.24625) 8924 : cluster [DBG] pgmap v5080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:54 gibba011 bash[30791]: audit 2021-03-10T16:41:53.223171+0000 mgr.gibba032.whopeo (mgr.24625) 8923 : audit [DBG] from='client.50037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:55.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:54 gibba011 bash[30791]: cluster 2021-03-10T16:41:53.869348+0000 mgr.gibba032.whopeo (mgr.24625) 8924 : cluster [DBG] pgmap v5080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:55.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:54 gibba011 bash[31774]: audit 2021-03-10T16:41:53.223171+0000 mgr.gibba032.whopeo (mgr.24625) 8923 : audit [DBG] from='client.50037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:41:55.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:54 gibba011 bash[31774]: cluster 2021-03-10T16:41:53.869348+0000 mgr.gibba032.whopeo (mgr.24625) 8924 : cluster [DBG] pgmap v5080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:56.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:55 gibba032 bash[24004]: cephadm 2021-03-10T16:41:54.693232+0000 mgr.gibba032.whopeo (mgr.24625) 8925 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:56.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:55 gibba011 bash[30791]: cephadm 2021-03-10T16:41:54.693232+0000 mgr.gibba032.whopeo (mgr.24625) 8925 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:56.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:55 gibba011 bash[31774]: cephadm 2021-03-10T16:41:54.693232+0000 mgr.gibba032.whopeo (mgr.24625) 8925 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:41:57.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:56 gibba032 bash[24004]: cluster 2021-03-10T16:41:55.870710+0000 mgr.gibba032.whopeo (mgr.24625) 8926 : cluster [DBG] pgmap v5081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:57.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:56 gibba011 bash[30791]: cluster 2021-03-10T16:41:55.870710+0000 mgr.gibba032.whopeo (mgr.24625) 8926 : cluster [DBG] pgmap v5081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:57.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:56 gibba011 bash[31774]: cluster 2021-03-10T16:41:55.870710+0000 mgr.gibba032.whopeo (mgr.24625) 8926 : cluster [DBG] pgmap v5081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.132249+0000 mon.a (mon.0) 3067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:41:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.132924+0000 mon.a (mon.0) 3068 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:41:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.263330+0000 mon.a (mon.0) 3069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:41:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.264676+0000 mon.a (mon.0) 3070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:41:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.265554+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:41:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[30791]: audit 2021-03-10T16:41:57.266303+0000 mon.a (mon.0) 3072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:41:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.132249+0000 mon.a (mon.0) 3067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:41:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.132924+0000 mon.a (mon.0) 3068 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:41:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.263330+0000 mon.a (mon.0) 3069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:41:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.264676+0000 mon.a (mon.0) 3070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:41:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.265554+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:41:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:58 gibba011 bash[31774]: audit 2021-03-10T16:41:57.266303+0000 mon.a (mon.0) 3072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:41:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.132249+0000 mon.a (mon.0) 3067 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:41:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.132924+0000 mon.a (mon.0) 3068 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:41:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.263330+0000 mon.a (mon.0) 3069 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T16:41:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.264676+0000 mon.a (mon.0) 3070 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:41:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.265554+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:41:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:58 gibba032 bash[24004]: audit 2021-03-10T16:41:57.266303+0000 mon.a (mon.0) 3072 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T16:41:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:41:59 gibba011 bash[30791]: cluster 2021-03-10T16:41:57.872437+0000 mgr.gibba032.whopeo (mgr.24625) 8927 : cluster [DBG] pgmap v5082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:41:59 gibba011 bash[31774]: cluster 2021-03-10T16:41:57.872437+0000 mgr.gibba032.whopeo (mgr.24625) 8927 : cluster [DBG] pgmap v5082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:41:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:41:59 gibba032 bash[24004]: cluster 2021-03-10T16:41:57.872437+0000 mgr.gibba032.whopeo (mgr.24625) 8927 : cluster [DBG] pgmap v5082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:01 gibba011 bash[30791]: cluster 2021-03-10T16:41:59.874079+0000 mgr.gibba032.whopeo (mgr.24625) 8928 : cluster [DBG] pgmap v5083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:01 gibba011 bash[31774]: cluster 2021-03-10T16:41:59.874079+0000 mgr.gibba032.whopeo (mgr.24625) 8928 : cluster [DBG] pgmap v5083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:01 gibba032 bash[24004]: cluster 2021-03-10T16:41:59.874079+0000 mgr.gibba032.whopeo (mgr.24625) 8928 : cluster [DBG] pgmap v5083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:03 gibba011 bash[30791]: cluster 2021-03-10T16:42:01.876088+0000 mgr.gibba032.whopeo (mgr.24625) 8929 : cluster [DBG] pgmap v5084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:03 gibba011 bash[31774]: cluster 2021-03-10T16:42:01.876088+0000 mgr.gibba032.whopeo (mgr.24625) 8929 : cluster [DBG] pgmap v5084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:03.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:03 gibba032 bash[24004]: cluster 2021-03-10T16:42:01.876088+0000 mgr.gibba032.whopeo (mgr.24625) 8929 : cluster [DBG] pgmap v5084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:05 gibba011 bash[30791]: cluster 2021-03-10T16:42:03.877827+0000 mgr.gibba032.whopeo (mgr.24625) 8930 : cluster [DBG] pgmap v5085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:05.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:05 gibba011 bash[31774]: cluster 2021-03-10T16:42:03.877827+0000 mgr.gibba032.whopeo (mgr.24625) 8930 : cluster [DBG] pgmap v5085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:05.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:05 gibba032 bash[24004]: cluster 2021-03-10T16:42:03.877827+0000 mgr.gibba032.whopeo (mgr.24625) 8930 : cluster [DBG] pgmap v5085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:07.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:07 gibba011 bash[30791]: cluster 2021-03-10T16:42:05.879181+0000 mgr.gibba032.whopeo (mgr.24625) 8931 : cluster [DBG] pgmap v5086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:07 gibba011 bash[31774]: cluster 2021-03-10T16:42:05.879181+0000 mgr.gibba032.whopeo (mgr.24625) 8931 : cluster [DBG] pgmap v5086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:07.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:07 gibba032 bash[24004]: cluster 2021-03-10T16:42:05.879181+0000 mgr.gibba032.whopeo (mgr.24625) 8931 : cluster [DBG] pgmap v5086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:09 gibba011 bash[30791]: cluster 2021-03-10T16:42:07.880910+0000 mgr.gibba032.whopeo (mgr.24625) 8932 : cluster [DBG] pgmap v5087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:09.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:09 gibba011 bash[31774]: cluster 2021-03-10T16:42:07.880910+0000 mgr.gibba032.whopeo (mgr.24625) 8932 : cluster [DBG] pgmap v5087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:09.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:09 gibba032 bash[24004]: cluster 2021-03-10T16:42:07.880910+0000 mgr.gibba032.whopeo (mgr.24625) 8932 : cluster [DBG] pgmap v5087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:11 gibba011 bash[30791]: cephadm 2021-03-10T16:42:09.709307+0000 mgr.gibba032.whopeo (mgr.24625) 8933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:11.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:11 gibba011 bash[30791]: cluster 2021-03-10T16:42:09.882413+0000 mgr.gibba032.whopeo (mgr.24625) 8934 : cluster [DBG] pgmap v5088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:11 gibba011 bash[31774]: cephadm 2021-03-10T16:42:09.709307+0000 mgr.gibba032.whopeo (mgr.24625) 8933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:11 gibba011 bash[31774]: cluster 2021-03-10T16:42:09.882413+0000 mgr.gibba032.whopeo (mgr.24625) 8934 : cluster [DBG] pgmap v5088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:11 gibba032 bash[24004]: cephadm 2021-03-10T16:42:09.709307+0000 mgr.gibba032.whopeo (mgr.24625) 8933 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:11 gibba032 bash[24004]: cluster 2021-03-10T16:42:09.882413+0000 mgr.gibba032.whopeo (mgr.24625) 8934 : cluster [DBG] pgmap v5088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:12 gibba011 bash[30791]: cluster 2021-03-10T16:42:11.883942+0000 mgr.gibba032.whopeo (mgr.24625) 8935 : cluster [DBG] pgmap v5089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:12.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:12 gibba011 bash[31774]: cluster 2021-03-10T16:42:11.883942+0000 mgr.gibba032.whopeo (mgr.24625) 8935 : cluster [DBG] pgmap v5089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:12.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:12 gibba032 bash[24004]: cluster 2021-03-10T16:42:11.883942+0000 mgr.gibba032.whopeo (mgr.24625) 8935 : cluster [DBG] pgmap v5089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:14 gibba032 bash[24004]: cluster 2021-03-10T16:42:13.885680+0000 mgr.gibba032.whopeo (mgr.24625) 8936 : cluster [DBG] pgmap v5090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:14 gibba011 bash[30791]: cluster 2021-03-10T16:42:13.885680+0000 mgr.gibba032.whopeo (mgr.24625) 8936 : cluster [DBG] pgmap v5090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:14 gibba011 bash[31774]: cluster 2021-03-10T16:42:13.885680+0000 mgr.gibba032.whopeo (mgr.24625) 8936 : cluster [DBG] pgmap v5090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:16 gibba032 bash[24004]: cluster 2021-03-10T16:42:15.887056+0000 mgr.gibba032.whopeo (mgr.24625) 8937 : cluster [DBG] pgmap v5091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:16 gibba011 bash[30791]: cluster 2021-03-10T16:42:15.887056+0000 mgr.gibba032.whopeo (mgr.24625) 8937 : cluster [DBG] pgmap v5091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:16 gibba011 bash[31774]: cluster 2021-03-10T16:42:15.887056+0000 mgr.gibba032.whopeo (mgr.24625) 8937 : cluster [DBG] pgmap v5091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:18 gibba032 bash[24004]: cluster 2021-03-10T16:42:17.888889+0000 mgr.gibba032.whopeo (mgr.24625) 8938 : cluster [DBG] pgmap v5092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:18 gibba011 bash[30791]: cluster 2021-03-10T16:42:17.888889+0000 mgr.gibba032.whopeo (mgr.24625) 8938 : cluster [DBG] pgmap v5092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:19.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:18 gibba011 bash[31774]: cluster 2021-03-10T16:42:17.888889+0000 mgr.gibba032.whopeo (mgr.24625) 8938 : cluster [DBG] pgmap v5092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:20 gibba032 bash[24004]: cluster 2021-03-10T16:42:19.890350+0000 mgr.gibba032.whopeo (mgr.24625) 8939 : cluster [DBG] pgmap v5093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:20 gibba011 bash[30791]: cluster 2021-03-10T16:42:19.890350+0000 mgr.gibba032.whopeo (mgr.24625) 8939 : cluster [DBG] pgmap v5093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:20 gibba011 bash[31774]: cluster 2021-03-10T16:42:19.890350+0000 mgr.gibba032.whopeo (mgr.24625) 8939 : cluster [DBG] pgmap v5093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:23.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:22 gibba032 bash[24004]: cluster 2021-03-10T16:42:21.892013+0000 mgr.gibba032.whopeo (mgr.24625) 8940 : cluster [DBG] pgmap v5094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:22 gibba011 bash[30791]: cluster 2021-03-10T16:42:21.892013+0000 mgr.gibba032.whopeo (mgr.24625) 8940 : cluster [DBG] pgmap v5094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:23.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:22 gibba011 bash[31774]: cluster 2021-03-10T16:42:21.892013+0000 mgr.gibba032.whopeo (mgr.24625) 8940 : cluster [DBG] pgmap v5094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:23.740 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:42:23.970 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:42:23.970 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 108s ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:42:23.971 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 44s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:42:23.971 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:42:23.971 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:42:23.971 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:42:23.972 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:42:23.972 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:42:23.972 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:42:23.972 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 108s ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:42:23.973 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 44s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:42:23.973 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:42:23.973 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:42:23.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:42:23.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 108s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:42:23.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:42:23.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:42:23.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:42:23.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 44s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:42:23.976 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 44s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:42:23.976 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 108s ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:42:24.223 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:42:24.223 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:42:24.223 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:42:24.223 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:24.224 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:42:24.224 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:42:24.224 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:24.224 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:42:24.225 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:42:24.225 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:24.225 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:42:24.225 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:42:24.225 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:42:24.226 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:42:24.226 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:42:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:24 gibba032 bash[24004]: audit 2021-03-10T16:42:23.731596+0000 mgr.gibba032.whopeo (mgr.24625) 8941 : audit [DBG] from='client.50049 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:24 gibba032 bash[24004]: cluster 2021-03-10T16:42:23.893696+0000 mgr.gibba032.whopeo (mgr.24625) 8942 : cluster [DBG] pgmap v5095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:24 gibba032 bash[24004]: audit 2021-03-10T16:42:23.959053+0000 mgr.gibba032.whopeo (mgr.24625) 8943 : audit [DBG] from='client.50055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:24 gibba032 bash[24004]: audit 2021-03-10T16:42:24.225541+0000 mon.c (mon.1) 176 : audit [DBG] from='client.? 172.21.2.111:0/178482206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[30791]: audit 2021-03-10T16:42:23.731596+0000 mgr.gibba032.whopeo (mgr.24625) 8941 : audit [DBG] from='client.50049 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[30791]: cluster 2021-03-10T16:42:23.893696+0000 mgr.gibba032.whopeo (mgr.24625) 8942 : cluster [DBG] pgmap v5095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:25.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[30791]: audit 2021-03-10T16:42:23.959053+0000 mgr.gibba032.whopeo (mgr.24625) 8943 : audit [DBG] from='client.50055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[30791]: audit 2021-03-10T16:42:24.225541+0000 mon.c (mon.1) 176 : audit [DBG] from='client.? 172.21.2.111:0/178482206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:25.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[31774]: audit 2021-03-10T16:42:23.731596+0000 mgr.gibba032.whopeo (mgr.24625) 8941 : audit [DBG] from='client.50049 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[31774]: cluster 2021-03-10T16:42:23.893696+0000 mgr.gibba032.whopeo (mgr.24625) 8942 : cluster [DBG] pgmap v5095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:25.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[31774]: audit 2021-03-10T16:42:23.959053+0000 mgr.gibba032.whopeo (mgr.24625) 8943 : audit [DBG] from='client.50055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:25.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:24 gibba011 bash[31774]: audit 2021-03-10T16:42:24.225541+0000 mon.c (mon.1) 176 : audit [DBG] from='client.? 172.21.2.111:0/178482206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:26.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:25 gibba032 bash[24004]: cephadm 2021-03-10T16:42:24.724958+0000 mgr.gibba032.whopeo (mgr.24625) 8944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:26.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:25 gibba011 bash[30791]: cephadm 2021-03-10T16:42:24.724958+0000 mgr.gibba032.whopeo (mgr.24625) 8944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:26.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:25 gibba011 bash[31774]: cephadm 2021-03-10T16:42:24.724958+0000 mgr.gibba032.whopeo (mgr.24625) 8944 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:27.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:27 gibba032 bash[24004]: cluster 2021-03-10T16:42:25.895066+0000 mgr.gibba032.whopeo (mgr.24625) 8945 : cluster [DBG] pgmap v5096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:27 gibba011 bash[30791]: cluster 2021-03-10T16:42:25.895066+0000 mgr.gibba032.whopeo (mgr.24625) 8945 : cluster [DBG] pgmap v5096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:27 gibba011 bash[31774]: cluster 2021-03-10T16:42:25.895066+0000 mgr.gibba032.whopeo (mgr.24625) 8945 : cluster [DBG] pgmap v5096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:29 gibba011 bash[30791]: cluster 2021-03-10T16:42:27.896925+0000 mgr.gibba032.whopeo (mgr.24625) 8946 : cluster [DBG] pgmap v5097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:29 gibba011 bash[31774]: cluster 2021-03-10T16:42:27.896925+0000 mgr.gibba032.whopeo (mgr.24625) 8946 : cluster [DBG] pgmap v5097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:29.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:29 gibba032 bash[24004]: cluster 2021-03-10T16:42:27.896925+0000 mgr.gibba032.whopeo (mgr.24625) 8946 : cluster [DBG] pgmap v5097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:31 gibba011 bash[30791]: cluster 2021-03-10T16:42:29.898194+0000 mgr.gibba032.whopeo (mgr.24625) 8947 : cluster [DBG] pgmap v5098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:31 gibba011 bash[31774]: cluster 2021-03-10T16:42:29.898194+0000 mgr.gibba032.whopeo (mgr.24625) 8947 : cluster [DBG] pgmap v5098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:31 gibba032 bash[24004]: cluster 2021-03-10T16:42:29.898194+0000 mgr.gibba032.whopeo (mgr.24625) 8947 : cluster [DBG] pgmap v5098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:33.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:33 gibba011 bash[30791]: cluster 2021-03-10T16:42:31.899833+0000 mgr.gibba032.whopeo (mgr.24625) 8948 : cluster [DBG] pgmap v5099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:33.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:33 gibba011 bash[31774]: cluster 2021-03-10T16:42:31.899833+0000 mgr.gibba032.whopeo (mgr.24625) 8948 : cluster [DBG] pgmap v5099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:33 gibba032 bash[24004]: cluster 2021-03-10T16:42:31.899833+0000 mgr.gibba032.whopeo (mgr.24625) 8948 : cluster [DBG] pgmap v5099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:35 gibba011 bash[30791]: cluster 2021-03-10T16:42:33.901484+0000 mgr.gibba032.whopeo (mgr.24625) 8949 : cluster [DBG] pgmap v5100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:35.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:35 gibba011 bash[31774]: cluster 2021-03-10T16:42:33.901484+0000 mgr.gibba032.whopeo (mgr.24625) 8949 : cluster [DBG] pgmap v5100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:35.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:35 gibba032 bash[24004]: cluster 2021-03-10T16:42:33.901484+0000 mgr.gibba032.whopeo (mgr.24625) 8949 : cluster [DBG] pgmap v5100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:37.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:37 gibba011 bash[30791]: cluster 2021-03-10T16:42:35.902858+0000 mgr.gibba032.whopeo (mgr.24625) 8950 : cluster [DBG] pgmap v5101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:37.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:37 gibba011 bash[31774]: cluster 2021-03-10T16:42:35.902858+0000 mgr.gibba032.whopeo (mgr.24625) 8950 : cluster [DBG] pgmap v5101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:37 gibba032 bash[24004]: cluster 2021-03-10T16:42:35.902858+0000 mgr.gibba032.whopeo (mgr.24625) 8950 : cluster [DBG] pgmap v5101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:39.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:39 gibba011 bash[30791]: cluster 2021-03-10T16:42:37.904811+0000 mgr.gibba032.whopeo (mgr.24625) 8951 : cluster [DBG] pgmap v5102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:39.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:39 gibba011 bash[31774]: cluster 2021-03-10T16:42:37.904811+0000 mgr.gibba032.whopeo (mgr.24625) 8951 : cluster [DBG] pgmap v5102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:39.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:39 gibba032 bash[24004]: cluster 2021-03-10T16:42:37.904811+0000 mgr.gibba032.whopeo (mgr.24625) 8951 : cluster [DBG] pgmap v5102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.741799+0000 mgr.gibba032.whopeo (mgr.24625) 8952 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:42:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.743411+0000 mgr.gibba032.whopeo (mgr.24625) 8953 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:42:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cluster 2021-03-10T16:42:39.905616+0000 mgr.gibba032.whopeo (mgr.24625) 8954 : cluster [DBG] pgmap v5103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.957637+0000 mon.a (mon.0) 3073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:42:41.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.958530+0000 mgr.gibba032.whopeo (mgr.24625) 8955 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:42:41.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.960085+0000 mgr.gibba032.whopeo (mgr.24625) 8956 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:42:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.961033+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:42:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.961164+0000 mgr.gibba032.whopeo (mgr.24625) 8957 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:42:41.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.961210+0000 mgr.gibba032.whopeo (mgr.24625) 8958 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:42:41.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.962235+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.962354+0000 mgr.gibba032.whopeo (mgr.24625) 8959 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:42:41.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.962398+0000 mgr.gibba032.whopeo (mgr.24625) 8960 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:42:41.275 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.963378+0000 mgr.gibba032.whopeo (mgr.24625) 8961 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:42:41.276 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.963394+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.280 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.964286+0000 mgr.gibba032.whopeo (mgr.24625) 8962 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:42:41.280 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.964330+0000 mgr.gibba032.whopeo (mgr.24625) 8963 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:42:41.281 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.964470+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.286 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.965416+0000 mgr.gibba032.whopeo (mgr.24625) 8964 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:42:41.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.965740+0000 mgr.gibba032.whopeo (mgr.24625) 8965 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:42:41.287 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.966577+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.966594+0000 mgr.gibba032.whopeo (mgr.24625) 8966 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:42:41.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.966640+0000 mgr.gibba032.whopeo (mgr.24625) 8967 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:42:41.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: cephadm 2021-03-10T16:42:39.966759+0000 mgr.gibba032.whopeo (mgr.24625) 8968 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:41.288 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:40 gibba032 bash[24004]: audit 2021-03-10T16:42:39.967789+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.741799+0000 mgr.gibba032.whopeo (mgr.24625) 8952 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:42:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.743411+0000 mgr.gibba032.whopeo (mgr.24625) 8953 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:42:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cluster 2021-03-10T16:42:39.905616+0000 mgr.gibba032.whopeo (mgr.24625) 8954 : cluster [DBG] pgmap v5103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.957637+0000 mon.a (mon.0) 3073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:42:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.958530+0000 mgr.gibba032.whopeo (mgr.24625) 8955 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:42:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.960085+0000 mgr.gibba032.whopeo (mgr.24625) 8956 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:42:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.961033+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:42:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.961164+0000 mgr.gibba032.whopeo (mgr.24625) 8957 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:42:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.961210+0000 mgr.gibba032.whopeo (mgr.24625) 8958 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:42:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.962235+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.962354+0000 mgr.gibba032.whopeo (mgr.24625) 8959 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:42:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.962398+0000 mgr.gibba032.whopeo (mgr.24625) 8960 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:42:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.963378+0000 mgr.gibba032.whopeo (mgr.24625) 8961 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:42:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.963394+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.964286+0000 mgr.gibba032.whopeo (mgr.24625) 8962 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:42:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.964330+0000 mgr.gibba032.whopeo (mgr.24625) 8963 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:42:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.964470+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.965416+0000 mgr.gibba032.whopeo (mgr.24625) 8964 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:42:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.965740+0000 mgr.gibba032.whopeo (mgr.24625) 8965 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:42:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.966577+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.966594+0000 mgr.gibba032.whopeo (mgr.24625) 8966 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:42:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.741799+0000 mgr.gibba032.whopeo (mgr.24625) 8952 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:42:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.743411+0000 mgr.gibba032.whopeo (mgr.24625) 8953 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:42:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cluster 2021-03-10T16:42:39.905616+0000 mgr.gibba032.whopeo (mgr.24625) 8954 : cluster [DBG] pgmap v5103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:41.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.957637+0000 mon.a (mon.0) 3073 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:42:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.958530+0000 mgr.gibba032.whopeo (mgr.24625) 8955 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:42:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.960085+0000 mgr.gibba032.whopeo (mgr.24625) 8956 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:42:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.961033+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:42:41.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.961164+0000 mgr.gibba032.whopeo (mgr.24625) 8957 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:42:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.961210+0000 mgr.gibba032.whopeo (mgr.24625) 8958 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:42:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.962235+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.962354+0000 mgr.gibba032.whopeo (mgr.24625) 8959 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:42:41.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.962398+0000 mgr.gibba032.whopeo (mgr.24625) 8960 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:42:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.963378+0000 mgr.gibba032.whopeo (mgr.24625) 8961 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:42:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.963394+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.964286+0000 mgr.gibba032.whopeo (mgr.24625) 8962 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:42:41.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.964330+0000 mgr.gibba032.whopeo (mgr.24625) 8963 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:42:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.964470+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.965416+0000 mgr.gibba032.whopeo (mgr.24625) 8964 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:42:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.965740+0000 mgr.gibba032.whopeo (mgr.24625) 8965 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:42:41.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.966577+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.966594+0000 mgr.gibba032.whopeo (mgr.24625) 8966 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:42:41.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.966640+0000 mgr.gibba032.whopeo (mgr.24625) 8967 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:42:41.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: cephadm 2021-03-10T16:42:39.966759+0000 mgr.gibba032.whopeo (mgr.24625) 8968 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:41.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[30791]: audit 2021-03-10T16:42:39.967789+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:41.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.966640+0000 mgr.gibba032.whopeo (mgr.24625) 8967 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:42:41.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: cephadm 2021-03-10T16:42:39.966759+0000 mgr.gibba032.whopeo (mgr.24625) 8968 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:41.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:40 gibba011 bash[31774]: audit 2021-03-10T16:42:39.967789+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:42 gibba032 bash[24004]: cluster 2021-03-10T16:42:41.907063+0000 mgr.gibba032.whopeo (mgr.24625) 8969 : cluster [DBG] pgmap v5104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:42 gibba011 bash[30791]: cluster 2021-03-10T16:42:41.907063+0000 mgr.gibba032.whopeo (mgr.24625) 8969 : cluster [DBG] pgmap v5104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:43.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:42 gibba011 bash[31774]: cluster 2021-03-10T16:42:41.907063+0000 mgr.gibba032.whopeo (mgr.24625) 8969 : cluster [DBG] pgmap v5104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:45.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:44 gibba032 bash[24004]: cluster 2021-03-10T16:42:43.908804+0000 mgr.gibba032.whopeo (mgr.24625) 8970 : cluster [DBG] pgmap v5105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:44 gibba011 bash[30791]: cluster 2021-03-10T16:42:43.908804+0000 mgr.gibba032.whopeo (mgr.24625) 8970 : cluster [DBG] pgmap v5105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:45.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:44 gibba011 bash[31774]: cluster 2021-03-10T16:42:43.908804+0000 mgr.gibba032.whopeo (mgr.24625) 8970 : cluster [DBG] pgmap v5105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:47.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:46 gibba032 bash[24004]: cluster 2021-03-10T16:42:45.910115+0000 mgr.gibba032.whopeo (mgr.24625) 8971 : cluster [DBG] pgmap v5106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:46 gibba011 bash[30791]: cluster 2021-03-10T16:42:45.910115+0000 mgr.gibba032.whopeo (mgr.24625) 8971 : cluster [DBG] pgmap v5106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:46 gibba011 bash[31774]: cluster 2021-03-10T16:42:45.910115+0000 mgr.gibba032.whopeo (mgr.24625) 8971 : cluster [DBG] pgmap v5106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:49.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:48 gibba032 bash[24004]: cluster 2021-03-10T16:42:47.911902+0000 mgr.gibba032.whopeo (mgr.24625) 8972 : cluster [DBG] pgmap v5107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:48 gibba011 bash[30791]: cluster 2021-03-10T16:42:47.911902+0000 mgr.gibba032.whopeo (mgr.24625) 8972 : cluster [DBG] pgmap v5107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:49.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:48 gibba011 bash[31774]: cluster 2021-03-10T16:42:47.911902+0000 mgr.gibba032.whopeo (mgr.24625) 8972 : cluster [DBG] pgmap v5107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:51.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:51 gibba032 bash[24004]: cluster 2021-03-10T16:42:49.913274+0000 mgr.gibba032.whopeo (mgr.24625) 8973 : cluster [DBG] pgmap v5108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:51 gibba011 bash[30791]: cluster 2021-03-10T16:42:49.913274+0000 mgr.gibba032.whopeo (mgr.24625) 8973 : cluster [DBG] pgmap v5108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:51 gibba011 bash[31774]: cluster 2021-03-10T16:42:49.913274+0000 mgr.gibba032.whopeo (mgr.24625) 8973 : cluster [DBG] pgmap v5108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:53 gibba011 bash[30791]: cluster 2021-03-10T16:42:51.914969+0000 mgr.gibba032.whopeo (mgr.24625) 8974 : cluster [DBG] pgmap v5109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:53 gibba011 bash[31774]: cluster 2021-03-10T16:42:51.914969+0000 mgr.gibba032.whopeo (mgr.24625) 8974 : cluster [DBG] pgmap v5109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:53.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:53 gibba032 bash[24004]: cluster 2021-03-10T16:42:51.914969+0000 mgr.gibba032.whopeo (mgr.24625) 8974 : cluster [DBG] pgmap v5109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:54.467 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:42:54.694 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:42:54.695 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:42:54.695 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 75s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:42:54.695 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:42:54.696 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:42:54.696 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:42:54.696 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:42:54.696 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:42:54.696 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:42:54.697 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:42:54.697 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 75s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:42:54.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:42:54.697 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:42:54.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:42:54.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:42:54.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:42:54.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:42:54.698 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:42:54.699 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 75s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:42:54.699 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 75s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:42:54.699 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:42:54.946 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:42:54.946 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:42:54.947 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:42:54.947 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:54.948 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:42:54.948 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:42:54.948 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:54.949 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:42:54.949 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:42:54.949 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:42:54.950 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:42:54.951 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:42:54.951 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:42:54.951 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:42:54.952 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:42:55.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:55 gibba032 bash[24004]: cluster 2021-03-10T16:42:53.916638+0000 mgr.gibba032.whopeo (mgr.24625) 8975 : cluster [DBG] pgmap v5110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:55.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:55 gibba032 bash[24004]: audit 2021-03-10T16:42:54.948524+0000 mon.a (mon.0) 3080 : audit [DBG] from='client.? 172.21.2.111:0/2251450376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:55.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:55 gibba011 bash[30791]: cluster 2021-03-10T16:42:53.916638+0000 mgr.gibba032.whopeo (mgr.24625) 8975 : cluster [DBG] pgmap v5110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:55.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:55 gibba011 bash[30791]: audit 2021-03-10T16:42:54.948524+0000 mon.a (mon.0) 3080 : audit [DBG] from='client.? 172.21.2.111:0/2251450376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:55 gibba011 bash[31774]: cluster 2021-03-10T16:42:53.916638+0000 mgr.gibba032.whopeo (mgr.24625) 8975 : cluster [DBG] pgmap v5110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:55.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:55 gibba011 bash[31774]: audit 2021-03-10T16:42:54.948524+0000 mon.a (mon.0) 3080 : audit [DBG] from='client.? 172.21.2.111:0/2251450376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:42:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[30791]: audit 2021-03-10T16:42:54.458819+0000 mgr.gibba032.whopeo (mgr.24625) 8976 : audit [DBG] from='client.50067 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[30791]: audit 2021-03-10T16:42:54.683074+0000 mgr.gibba032.whopeo (mgr.24625) 8977 : audit [DBG] from='client.60055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[30791]: cephadm 2021-03-10T16:42:54.982381+0000 mgr.gibba032.whopeo (mgr.24625) 8978 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[31774]: audit 2021-03-10T16:42:54.458819+0000 mgr.gibba032.whopeo (mgr.24625) 8976 : audit [DBG] from='client.50067 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[31774]: audit 2021-03-10T16:42:54.683074+0000 mgr.gibba032.whopeo (mgr.24625) 8977 : audit [DBG] from='client.60055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:56 gibba011 bash[31774]: cephadm 2021-03-10T16:42:54.982381+0000 mgr.gibba032.whopeo (mgr.24625) 8978 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:56 gibba032 bash[24004]: audit 2021-03-10T16:42:54.458819+0000 mgr.gibba032.whopeo (mgr.24625) 8976 : audit [DBG] from='client.50067 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:56 gibba032 bash[24004]: audit 2021-03-10T16:42:54.683074+0000 mgr.gibba032.whopeo (mgr.24625) 8977 : audit [DBG] from='client.60055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:42:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:56 gibba032 bash[24004]: cephadm 2021-03-10T16:42:54.982381+0000 mgr.gibba032.whopeo (mgr.24625) 8978 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:42:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:57 gibba011 bash[30791]: cluster 2021-03-10T16:42:55.918058+0000 mgr.gibba032.whopeo (mgr.24625) 8979 : cluster [DBG] pgmap v5111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:57 gibba011 bash[31774]: cluster 2021-03-10T16:42:55.918058+0000 mgr.gibba032.whopeo (mgr.24625) 8979 : cluster [DBG] pgmap v5111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:57.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:57 gibba032 bash[24004]: cluster 2021-03-10T16:42:55.918058+0000 mgr.gibba032.whopeo (mgr.24625) 8979 : cluster [DBG] pgmap v5111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.140374+0000 mon.a (mon.0) 3081 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:42:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.141062+0000 mon.a (mon.0) 3082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:42:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.274051+0000 mon.a (mon.0) 3083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:42:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.274831+0000 mon.a (mon.0) 3084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.275596+0000 mon.a (mon.0) 3085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[30791]: audit 2021-03-10T16:42:57.276330+0000 mon.a (mon.0) 3086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T16:42:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.140374+0000 mon.a (mon.0) 3081 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:42:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.141062+0000 mon.a (mon.0) 3082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:42:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.274051+0000 mon.a (mon.0) 3083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:42:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.274831+0000 mon.a (mon.0) 3084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.275596+0000 mon.a (mon.0) 3085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:58 gibba011 bash[31774]: audit 2021-03-10T16:42:57.276330+0000 mon.a (mon.0) 3086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T16:42:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.140374+0000 mon.a (mon.0) 3081 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:42:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.141062+0000 mon.a (mon.0) 3082 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:42:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.274051+0000 mon.a (mon.0) 3083 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:42:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.274831+0000 mon.a (mon.0) 3084 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.275596+0000 mon.a (mon.0) 3085 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:42:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:58 gibba032 bash[24004]: audit 2021-03-10T16:42:57.276330+0000 mon.a (mon.0) 3086 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T16:42:59.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:42:59 gibba011 bash[30791]: cluster 2021-03-10T16:42:57.919620+0000 mgr.gibba032.whopeo (mgr.24625) 8980 : cluster [DBG] pgmap v5112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:42:59 gibba011 bash[31774]: cluster 2021-03-10T16:42:57.919620+0000 mgr.gibba032.whopeo (mgr.24625) 8980 : cluster [DBG] pgmap v5112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:42:59.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:42:59 gibba032 bash[24004]: cluster 2021-03-10T16:42:57.919620+0000 mgr.gibba032.whopeo (mgr.24625) 8980 : cluster [DBG] pgmap v5112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:01 gibba011 bash[30791]: cluster 2021-03-10T16:42:59.921160+0000 mgr.gibba032.whopeo (mgr.24625) 8981 : cluster [DBG] pgmap v5113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:01 gibba011 bash[31774]: cluster 2021-03-10T16:42:59.921160+0000 mgr.gibba032.whopeo (mgr.24625) 8981 : cluster [DBG] pgmap v5113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:01 gibba032 bash[24004]: cluster 2021-03-10T16:42:59.921160+0000 mgr.gibba032.whopeo (mgr.24625) 8981 : cluster [DBG] pgmap v5113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:03 gibba011 bash[30791]: cluster 2021-03-10T16:43:01.922810+0000 mgr.gibba032.whopeo (mgr.24625) 8982 : cluster [DBG] pgmap v5114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:03.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:03 gibba011 bash[31774]: cluster 2021-03-10T16:43:01.922810+0000 mgr.gibba032.whopeo (mgr.24625) 8982 : cluster [DBG] pgmap v5114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:03.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:03 gibba032 bash[24004]: cluster 2021-03-10T16:43:01.922810+0000 mgr.gibba032.whopeo (mgr.24625) 8982 : cluster [DBG] pgmap v5114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:05.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:05 gibba011 bash[30791]: cluster 2021-03-10T16:43:03.924382+0000 mgr.gibba032.whopeo (mgr.24625) 8983 : cluster [DBG] pgmap v5115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:05.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:05 gibba011 bash[31774]: cluster 2021-03-10T16:43:03.924382+0000 mgr.gibba032.whopeo (mgr.24625) 8983 : cluster [DBG] pgmap v5115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:05.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:05 gibba032 bash[24004]: cluster 2021-03-10T16:43:03.924382+0000 mgr.gibba032.whopeo (mgr.24625) 8983 : cluster [DBG] pgmap v5115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:07.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:07 gibba011 bash[30791]: cluster 2021-03-10T16:43:05.925723+0000 mgr.gibba032.whopeo (mgr.24625) 8984 : cluster [DBG] pgmap v5116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:07.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:07 gibba011 bash[31774]: cluster 2021-03-10T16:43:05.925723+0000 mgr.gibba032.whopeo (mgr.24625) 8984 : cluster [DBG] pgmap v5116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:07.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:07 gibba032 bash[24004]: cluster 2021-03-10T16:43:05.925723+0000 mgr.gibba032.whopeo (mgr.24625) 8984 : cluster [DBG] pgmap v5116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:09 gibba032 bash[24004]: cluster 2021-03-10T16:43:07.927665+0000 mgr.gibba032.whopeo (mgr.24625) 8985 : cluster [DBG] pgmap v5117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:09.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:09 gibba011 bash[30791]: cluster 2021-03-10T16:43:07.927665+0000 mgr.gibba032.whopeo (mgr.24625) 8985 : cluster [DBG] pgmap v5117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:09.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:09 gibba011 bash[31774]: cluster 2021-03-10T16:43:07.927665+0000 mgr.gibba032.whopeo (mgr.24625) 8985 : cluster [DBG] pgmap v5117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:10 gibba032 bash[24004]: cluster 2021-03-10T16:43:09.929073+0000 mgr.gibba032.whopeo (mgr.24625) 8986 : cluster [DBG] pgmap v5118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:11.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:10 gibba032 bash[24004]: cephadm 2021-03-10T16:43:09.998016+0000 mgr.gibba032.whopeo (mgr.24625) 8987 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:11.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:10 gibba011 bash[30791]: cluster 2021-03-10T16:43:09.929073+0000 mgr.gibba032.whopeo (mgr.24625) 8986 : cluster [DBG] pgmap v5118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:11.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:10 gibba011 bash[30791]: cephadm 2021-03-10T16:43:09.998016+0000 mgr.gibba032.whopeo (mgr.24625) 8987 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:11.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:10 gibba011 bash[31774]: cluster 2021-03-10T16:43:09.929073+0000 mgr.gibba032.whopeo (mgr.24625) 8986 : cluster [DBG] pgmap v5118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:11.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:10 gibba011 bash[31774]: cephadm 2021-03-10T16:43:09.998016+0000 mgr.gibba032.whopeo (mgr.24625) 8987 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:12 gibba032 bash[24004]: cluster 2021-03-10T16:43:11.930808+0000 mgr.gibba032.whopeo (mgr.24625) 8988 : cluster [DBG] pgmap v5119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:12 gibba011 bash[30791]: cluster 2021-03-10T16:43:11.930808+0000 mgr.gibba032.whopeo (mgr.24625) 8988 : cluster [DBG] pgmap v5119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:13.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:12 gibba011 bash[31774]: cluster 2021-03-10T16:43:11.930808+0000 mgr.gibba032.whopeo (mgr.24625) 8988 : cluster [DBG] pgmap v5119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:15.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:14 gibba032 bash[24004]: cluster 2021-03-10T16:43:13.932318+0000 mgr.gibba032.whopeo (mgr.24625) 8989 : cluster [DBG] pgmap v5120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:14 gibba011 bash[30791]: cluster 2021-03-10T16:43:13.932318+0000 mgr.gibba032.whopeo (mgr.24625) 8989 : cluster [DBG] pgmap v5120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:14 gibba011 bash[31774]: cluster 2021-03-10T16:43:13.932318+0000 mgr.gibba032.whopeo (mgr.24625) 8989 : cluster [DBG] pgmap v5120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:17.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:16 gibba032 bash[24004]: cluster 2021-03-10T16:43:15.933790+0000 mgr.gibba032.whopeo (mgr.24625) 8990 : cluster [DBG] pgmap v5121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:16 gibba011 bash[30791]: cluster 2021-03-10T16:43:15.933790+0000 mgr.gibba032.whopeo (mgr.24625) 8990 : cluster [DBG] pgmap v5121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:17.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:16 gibba011 bash[31774]: cluster 2021-03-10T16:43:15.933790+0000 mgr.gibba032.whopeo (mgr.24625) 8990 : cluster [DBG] pgmap v5121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:19.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:18 gibba032 bash[24004]: cluster 2021-03-10T16:43:17.935747+0000 mgr.gibba032.whopeo (mgr.24625) 8991 : cluster [DBG] pgmap v5122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:19.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:18 gibba011 bash[30791]: cluster 2021-03-10T16:43:17.935747+0000 mgr.gibba032.whopeo (mgr.24625) 8991 : cluster [DBG] pgmap v5122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:19.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:18 gibba011 bash[31774]: cluster 2021-03-10T16:43:17.935747+0000 mgr.gibba032.whopeo (mgr.24625) 8991 : cluster [DBG] pgmap v5122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:21.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:20 gibba032 bash[24004]: cluster 2021-03-10T16:43:19.937170+0000 mgr.gibba032.whopeo (mgr.24625) 8992 : cluster [DBG] pgmap v5123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:21 gibba011 bash[30791]: cluster 2021-03-10T16:43:19.937170+0000 mgr.gibba032.whopeo (mgr.24625) 8992 : cluster [DBG] pgmap v5123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:21 gibba011 bash[31774]: cluster 2021-03-10T16:43:19.937170+0000 mgr.gibba032.whopeo (mgr.24625) 8992 : cluster [DBG] pgmap v5123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:23.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:23 gibba011 bash[30791]: cluster 2021-03-10T16:43:21.938821+0000 mgr.gibba032.whopeo (mgr.24625) 8993 : cluster [DBG] pgmap v5124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:23.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:23 gibba011 bash[31774]: cluster 2021-03-10T16:43:21.938821+0000 mgr.gibba032.whopeo (mgr.24625) 8993 : cluster [DBG] pgmap v5124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:23.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:23 gibba032 bash[24004]: cluster 2021-03-10T16:43:21.938821+0000 mgr.gibba032.whopeo (mgr.24625) 8993 : cluster [DBG] pgmap v5124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:25.195 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:43:25.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:25 gibba032 bash[24004]: cluster 2021-03-10T16:43:23.940532+0000 mgr.gibba032.whopeo (mgr.24625) 8994 : cluster [DBG] pgmap v5125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:25.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:25 gibba011 bash[30791]: cluster 2021-03-10T16:43:23.940532+0000 mgr.gibba032.whopeo (mgr.24625) 8994 : cluster [DBG] pgmap v5125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:25.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:25 gibba011 bash[31774]: cluster 2021-03-10T16:43:23.940532+0000 mgr.gibba032.whopeo (mgr.24625) 8994 : cluster [DBG] pgmap v5125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:25.423 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:43:25.424 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:43:25.424 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 105s ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:43:25.424 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:43:25.424 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:43:25.425 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:43:25.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:43:25.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:43:25.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:43:25.426 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:43:25.426 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 105s ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:43:25.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:43:25.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:43:25.428 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:43:25.428 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:43:25.428 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:43:25.428 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:43:25.429 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:43:25.429 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 105s ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:43:25.429 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 105s ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:43:25.429 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:43:25.671 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:43:25.672 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:43:25.672 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:43:25.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:25.672 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:43:25.672 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:43:25.673 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:25.673 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:43:25.673 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:43:25.673 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:25.674 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:43:25.674 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:43:25.674 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:43:25.675 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:43:25.675 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:43:26.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:26 gibba011 bash[30791]: cephadm 2021-03-10T16:43:25.000672+0000 mgr.gibba032.whopeo (mgr.24625) 8995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:26.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:26 gibba011 bash[30791]: audit 2021-03-10T16:43:25.674266+0000 mon.a (mon.0) 3087 : audit [DBG] from='client.? 172.21.2.111:0/842243258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:26.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:26 gibba011 bash[31774]: cephadm 2021-03-10T16:43:25.000672+0000 mgr.gibba032.whopeo (mgr.24625) 8995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:26.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:26 gibba011 bash[31774]: audit 2021-03-10T16:43:25.674266+0000 mon.a (mon.0) 3087 : audit [DBG] from='client.? 172.21.2.111:0/842243258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:26.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:26 gibba032 bash[24004]: cephadm 2021-03-10T16:43:25.000672+0000 mgr.gibba032.whopeo (mgr.24625) 8995 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:26.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:26 gibba032 bash[24004]: audit 2021-03-10T16:43:25.674266+0000 mon.a (mon.0) 3087 : audit [DBG] from='client.? 172.21.2.111:0/842243258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[30791]: audit 2021-03-10T16:43:25.186722+0000 mgr.gibba032.whopeo (mgr.24625) 8996 : audit [DBG] from='client.50085 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[30791]: audit 2021-03-10T16:43:25.412307+0000 mgr.gibba032.whopeo (mgr.24625) 8997 : audit [DBG] from='client.60073 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[30791]: cluster 2021-03-10T16:43:25.941811+0000 mgr.gibba032.whopeo (mgr.24625) 8998 : cluster [DBG] pgmap v5126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[31774]: audit 2021-03-10T16:43:25.186722+0000 mgr.gibba032.whopeo (mgr.24625) 8996 : audit [DBG] from='client.50085 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[31774]: audit 2021-03-10T16:43:25.412307+0000 mgr.gibba032.whopeo (mgr.24625) 8997 : audit [DBG] from='client.60073 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:27 gibba011 bash[31774]: cluster 2021-03-10T16:43:25.941811+0000 mgr.gibba032.whopeo (mgr.24625) 8998 : cluster [DBG] pgmap v5126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:27 gibba032 bash[24004]: audit 2021-03-10T16:43:25.186722+0000 mgr.gibba032.whopeo (mgr.24625) 8996 : audit [DBG] from='client.50085 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:27 gibba032 bash[24004]: audit 2021-03-10T16:43:25.412307+0000 mgr.gibba032.whopeo (mgr.24625) 8997 : audit [DBG] from='client.60073 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:27 gibba032 bash[24004]: cluster 2021-03-10T16:43:25.941811+0000 mgr.gibba032.whopeo (mgr.24625) 8998 : cluster [DBG] pgmap v5126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:29.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:29 gibba011 bash[30791]: cluster 2021-03-10T16:43:27.943768+0000 mgr.gibba032.whopeo (mgr.24625) 8999 : cluster [DBG] pgmap v5127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:29.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:29 gibba011 bash[31774]: cluster 2021-03-10T16:43:27.943768+0000 mgr.gibba032.whopeo (mgr.24625) 8999 : cluster [DBG] pgmap v5127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:29.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:29 gibba032 bash[24004]: cluster 2021-03-10T16:43:27.943768+0000 mgr.gibba032.whopeo (mgr.24625) 8999 : cluster [DBG] pgmap v5127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:31 gibba011 bash[30791]: cluster 2021-03-10T16:43:29.945116+0000 mgr.gibba032.whopeo (mgr.24625) 9000 : cluster [DBG] pgmap v5128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:31 gibba011 bash[31774]: cluster 2021-03-10T16:43:29.945116+0000 mgr.gibba032.whopeo (mgr.24625) 9000 : cluster [DBG] pgmap v5128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:31 gibba032 bash[24004]: cluster 2021-03-10T16:43:29.945116+0000 mgr.gibba032.whopeo (mgr.24625) 9000 : cluster [DBG] pgmap v5128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:33.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:33 gibba011 bash[30791]: cluster 2021-03-10T16:43:31.946727+0000 mgr.gibba032.whopeo (mgr.24625) 9001 : cluster [DBG] pgmap v5129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:33.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:33 gibba011 bash[31774]: cluster 2021-03-10T16:43:31.946727+0000 mgr.gibba032.whopeo (mgr.24625) 9001 : cluster [DBG] pgmap v5129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:33.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:33 gibba032 bash[24004]: cluster 2021-03-10T16:43:31.946727+0000 mgr.gibba032.whopeo (mgr.24625) 9001 : cluster [DBG] pgmap v5129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:35.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:35 gibba011 bash[30791]: cluster 2021-03-10T16:43:33.948269+0000 mgr.gibba032.whopeo (mgr.24625) 9002 : cluster [DBG] pgmap v5130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:35.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:35 gibba011 bash[31774]: cluster 2021-03-10T16:43:33.948269+0000 mgr.gibba032.whopeo (mgr.24625) 9002 : cluster [DBG] pgmap v5130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:35.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:35 gibba032 bash[24004]: cluster 2021-03-10T16:43:33.948269+0000 mgr.gibba032.whopeo (mgr.24625) 9002 : cluster [DBG] pgmap v5130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:37.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:37 gibba011 bash[30791]: cluster 2021-03-10T16:43:35.949625+0000 mgr.gibba032.whopeo (mgr.24625) 9003 : cluster [DBG] pgmap v5131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:37.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:37 gibba011 bash[31774]: cluster 2021-03-10T16:43:35.949625+0000 mgr.gibba032.whopeo (mgr.24625) 9003 : cluster [DBG] pgmap v5131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:37 gibba032 bash[24004]: cluster 2021-03-10T16:43:35.949625+0000 mgr.gibba032.whopeo (mgr.24625) 9003 : cluster [DBG] pgmap v5131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:39.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:39 gibba032 bash[24004]: cluster 2021-03-10T16:43:37.951555+0000 mgr.gibba032.whopeo (mgr.24625) 9004 : cluster [DBG] pgmap v5132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:39.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:39 gibba011 bash[30791]: cluster 2021-03-10T16:43:37.951555+0000 mgr.gibba032.whopeo (mgr.24625) 9004 : cluster [DBG] pgmap v5132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:39.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:39 gibba011 bash[31774]: cluster 2021-03-10T16:43:37.951555+0000 mgr.gibba032.whopeo (mgr.24625) 9004 : cluster [DBG] pgmap v5132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: cluster 2021-03-10T16:43:39.952814+0000 mgr.gibba032.whopeo (mgr.24625) 9005 : cluster [DBG] pgmap v5133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.012790+0000 mgr.gibba032.whopeo (mgr.24625) 9006 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:43:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.013207+0000 mgr.gibba032.whopeo (mgr.24625) 9007 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:43:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.227315+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:43:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.230878+0000 mon.a (mon.0) 3089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:43:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.232303+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.233732+0000 mon.a (mon.0) 3091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.234964+0000 mon.a (mon.0) 3092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.237470+0000 mon.a (mon.0) 3093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:41 gibba032 bash[24004]: audit 2021-03-10T16:43:40.239314+0000 mon.a (mon.0) 3094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: cluster 2021-03-10T16:43:39.952814+0000 mgr.gibba032.whopeo (mgr.24625) 9005 : cluster [DBG] pgmap v5133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.012790+0000 mgr.gibba032.whopeo (mgr.24625) 9006 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:43:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.013207+0000 mgr.gibba032.whopeo (mgr.24625) 9007 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:43:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.227315+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:43:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.230878+0000 mon.a (mon.0) 3089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:43:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.232303+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.233732+0000 mon.a (mon.0) 3091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.234964+0000 mon.a (mon.0) 3092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.237470+0000 mon.a (mon.0) 3093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[30791]: audit 2021-03-10T16:43:40.239314+0000 mon.a (mon.0) 3094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: cluster 2021-03-10T16:43:39.952814+0000 mgr.gibba032.whopeo (mgr.24625) 9005 : cluster [DBG] pgmap v5133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.012790+0000 mgr.gibba032.whopeo (mgr.24625) 9006 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:43:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.013207+0000 mgr.gibba032.whopeo (mgr.24625) 9007 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:43:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.227315+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:43:41.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.230878+0000 mon.a (mon.0) 3089 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:43:41.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.232303+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.233732+0000 mon.a (mon.0) 3091 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.234964+0000 mon.a (mon.0) 3092 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.237470+0000 mon.a (mon.0) 3093 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:41.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:41 gibba011 bash[31774]: audit 2021-03-10T16:43:40.239314+0000 mon.a (mon.0) 3094 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.228276+0000 mgr.gibba032.whopeo (mgr.24625) 9008 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:43:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.230071+0000 mgr.gibba032.whopeo (mgr.24625) 9009 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:43:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.231455+0000 mgr.gibba032.whopeo (mgr.24625) 9010 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:43:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.231495+0000 mgr.gibba032.whopeo (mgr.24625) 9011 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:43:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.232762+0000 mgr.gibba032.whopeo (mgr.24625) 9012 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:43:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.232800+0000 mgr.gibba032.whopeo (mgr.24625) 9013 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:43:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.233891+0000 mgr.gibba032.whopeo (mgr.24625) 9014 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:43:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.235112+0000 mgr.gibba032.whopeo (mgr.24625) 9015 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:43:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.235149+0000 mgr.gibba032.whopeo (mgr.24625) 9016 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:43:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.236823+0000 mgr.gibba032.whopeo (mgr.24625) 9017 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:43:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.237140+0000 mgr.gibba032.whopeo (mgr.24625) 9018 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:43:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.238510+0000 mgr.gibba032.whopeo (mgr.24625) 9019 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:43:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.238550+0000 mgr.gibba032.whopeo (mgr.24625) 9020 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:43:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cephadm 2021-03-10T16:43:40.238656+0000 mgr.gibba032.whopeo (mgr.24625) 9021 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:42 gibba032 bash[24004]: cluster 2021-03-10T16:43:41.954294+0000 mgr.gibba032.whopeo (mgr.24625) 9022 : cluster [DBG] pgmap v5134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.228276+0000 mgr.gibba032.whopeo (mgr.24625) 9008 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:43:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.230071+0000 mgr.gibba032.whopeo (mgr.24625) 9009 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:43:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.231455+0000 mgr.gibba032.whopeo (mgr.24625) 9010 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:43:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.231495+0000 mgr.gibba032.whopeo (mgr.24625) 9011 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:43:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.232762+0000 mgr.gibba032.whopeo (mgr.24625) 9012 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:43:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.232800+0000 mgr.gibba032.whopeo (mgr.24625) 9013 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:43:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.233891+0000 mgr.gibba032.whopeo (mgr.24625) 9014 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:43:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.235112+0000 mgr.gibba032.whopeo (mgr.24625) 9015 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:43:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.235149+0000 mgr.gibba032.whopeo (mgr.24625) 9016 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:43:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.236823+0000 mgr.gibba032.whopeo (mgr.24625) 9017 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:43:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.237140+0000 mgr.gibba032.whopeo (mgr.24625) 9018 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:43:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.238510+0000 mgr.gibba032.whopeo (mgr.24625) 9019 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:43:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.238550+0000 mgr.gibba032.whopeo (mgr.24625) 9020 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:43:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cephadm 2021-03-10T16:43:40.238656+0000 mgr.gibba032.whopeo (mgr.24625) 9021 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[30791]: cluster 2021-03-10T16:43:41.954294+0000 mgr.gibba032.whopeo (mgr.24625) 9022 : cluster [DBG] pgmap v5134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.228276+0000 mgr.gibba032.whopeo (mgr.24625) 9008 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:43:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.230071+0000 mgr.gibba032.whopeo (mgr.24625) 9009 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:43:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.231455+0000 mgr.gibba032.whopeo (mgr.24625) 9010 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:43:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.231495+0000 mgr.gibba032.whopeo (mgr.24625) 9011 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:43:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.232762+0000 mgr.gibba032.whopeo (mgr.24625) 9012 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:43:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.232800+0000 mgr.gibba032.whopeo (mgr.24625) 9013 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:43:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.233891+0000 mgr.gibba032.whopeo (mgr.24625) 9014 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:43:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.235112+0000 mgr.gibba032.whopeo (mgr.24625) 9015 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:43:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.235149+0000 mgr.gibba032.whopeo (mgr.24625) 9016 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:43:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.236823+0000 mgr.gibba032.whopeo (mgr.24625) 9017 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:43:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.237140+0000 mgr.gibba032.whopeo (mgr.24625) 9018 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:43:42.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.238510+0000 mgr.gibba032.whopeo (mgr.24625) 9019 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:43:42.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.238550+0000 mgr.gibba032.whopeo (mgr.24625) 9020 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:43:42.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cephadm 2021-03-10T16:43:40.238656+0000 mgr.gibba032.whopeo (mgr.24625) 9021 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:42.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:42 gibba011 bash[31774]: cluster 2021-03-10T16:43:41.954294+0000 mgr.gibba032.whopeo (mgr.24625) 9022 : cluster [DBG] pgmap v5134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:45.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:45 gibba011 bash[30791]: cluster 2021-03-10T16:43:43.955901+0000 mgr.gibba032.whopeo (mgr.24625) 9023 : cluster [DBG] pgmap v5135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:45.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:45 gibba011 bash[31774]: cluster 2021-03-10T16:43:43.955901+0000 mgr.gibba032.whopeo (mgr.24625) 9023 : cluster [DBG] pgmap v5135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:45.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:45 gibba032 bash[24004]: cluster 2021-03-10T16:43:43.955901+0000 mgr.gibba032.whopeo (mgr.24625) 9023 : cluster [DBG] pgmap v5135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:47.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:47 gibba011 bash[30791]: cluster 2021-03-10T16:43:45.957222+0000 mgr.gibba032.whopeo (mgr.24625) 9024 : cluster [DBG] pgmap v5136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:47.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:47 gibba011 bash[31774]: cluster 2021-03-10T16:43:45.957222+0000 mgr.gibba032.whopeo (mgr.24625) 9024 : cluster [DBG] pgmap v5136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:47.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:47 gibba032 bash[24004]: cluster 2021-03-10T16:43:45.957222+0000 mgr.gibba032.whopeo (mgr.24625) 9024 : cluster [DBG] pgmap v5136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:49.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:49 gibba011 bash[30791]: cluster 2021-03-10T16:43:47.959060+0000 mgr.gibba032.whopeo (mgr.24625) 9025 : cluster [DBG] pgmap v5137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:49.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:49 gibba011 bash[31774]: cluster 2021-03-10T16:43:47.959060+0000 mgr.gibba032.whopeo (mgr.24625) 9025 : cluster [DBG] pgmap v5137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:49.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:49 gibba032 bash[24004]: cluster 2021-03-10T16:43:47.959060+0000 mgr.gibba032.whopeo (mgr.24625) 9025 : cluster [DBG] pgmap v5137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:51 gibba011 bash[30791]: cluster 2021-03-10T16:43:49.960449+0000 mgr.gibba032.whopeo (mgr.24625) 9026 : cluster [DBG] pgmap v5138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:51 gibba011 bash[31774]: cluster 2021-03-10T16:43:49.960449+0000 mgr.gibba032.whopeo (mgr.24625) 9026 : cluster [DBG] pgmap v5138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:51 gibba032 bash[24004]: cluster 2021-03-10T16:43:49.960449+0000 mgr.gibba032.whopeo (mgr.24625) 9026 : cluster [DBG] pgmap v5138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:53 gibba011 bash[30791]: cluster 2021-03-10T16:43:51.962038+0000 mgr.gibba032.whopeo (mgr.24625) 9027 : cluster [DBG] pgmap v5139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:53.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:53 gibba011 bash[31774]: cluster 2021-03-10T16:43:51.962038+0000 mgr.gibba032.whopeo (mgr.24625) 9027 : cluster [DBG] pgmap v5139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:53.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:53 gibba032 bash[24004]: cluster 2021-03-10T16:43:51.962038+0000 mgr.gibba032.whopeo (mgr.24625) 9027 : cluster [DBG] pgmap v5139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:55.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:55 gibba011 bash[30791]: cluster 2021-03-10T16:43:53.963631+0000 mgr.gibba032.whopeo (mgr.24625) 9028 : cluster [DBG] pgmap v5140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:55.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:55 gibba011 bash[31774]: cluster 2021-03-10T16:43:53.963631+0000 mgr.gibba032.whopeo (mgr.24625) 9028 : cluster [DBG] pgmap v5140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:55.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:55 gibba032 bash[24004]: cluster 2021-03-10T16:43:53.963631+0000 mgr.gibba032.whopeo (mgr.24625) 9028 : cluster [DBG] pgmap v5140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:55.918 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:43:56.147 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:43:56.148 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:43:56.148 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:43:56.148 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:43:56.148 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:43:56.149 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:43:56.149 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:43:56.149 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:43:56.149 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:43:56.149 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:43:56.150 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:43:56.150 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:43:56.150 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:43:56.150 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:43:56.151 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:43:56.151 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:43:56.152 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:43:56.152 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:43:56.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:43:56.156 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:43:56.156 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:43:56.396 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:43:56.396 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:43:56.397 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:43:56.397 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:56.397 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:43:56.397 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:43:56.398 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:56.398 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:43:56.398 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:43:56.398 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:43:56.398 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:43:56.399 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:43:56.399 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:43:56.399 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:43:56.399 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:43:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[30791]: cephadm 2021-03-10T16:43:55.253078+0000 mgr.gibba032.whopeo (mgr.24625) 9029 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:57.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[30791]: audit 2021-03-10T16:43:55.909941+0000 mgr.gibba032.whopeo (mgr.24625) 9030 : audit [DBG] from='client.50103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:57.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[30791]: cluster 2021-03-10T16:43:55.965001+0000 mgr.gibba032.whopeo (mgr.24625) 9031 : cluster [DBG] pgmap v5141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:57.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[30791]: audit 2021-03-10T16:43:56.399088+0000 mon.c (mon.1) 177 : audit [DBG] from='client.? 172.21.2.111:0/878627779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:57.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[31774]: cephadm 2021-03-10T16:43:55.253078+0000 mgr.gibba032.whopeo (mgr.24625) 9029 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:57.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[31774]: audit 2021-03-10T16:43:55.909941+0000 mgr.gibba032.whopeo (mgr.24625) 9030 : audit [DBG] from='client.50103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:57.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[31774]: cluster 2021-03-10T16:43:55.965001+0000 mgr.gibba032.whopeo (mgr.24625) 9031 : cluster [DBG] pgmap v5141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:57.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:57 gibba011 bash[31774]: audit 2021-03-10T16:43:56.399088+0000 mon.c (mon.1) 177 : audit [DBG] from='client.? 172.21.2.111:0/878627779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:57 gibba032 bash[24004]: cephadm 2021-03-10T16:43:55.253078+0000 mgr.gibba032.whopeo (mgr.24625) 9029 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:43:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:57 gibba032 bash[24004]: audit 2021-03-10T16:43:55.909941+0000 mgr.gibba032.whopeo (mgr.24625) 9030 : audit [DBG] from='client.50103 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:57 gibba032 bash[24004]: cluster 2021-03-10T16:43:55.965001+0000 mgr.gibba032.whopeo (mgr.24625) 9031 : cluster [DBG] pgmap v5141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:57 gibba032 bash[24004]: audit 2021-03-10T16:43:56.399088+0000 mon.c (mon.1) 177 : audit [DBG] from='client.? 172.21.2.111:0/878627779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:43:58.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:56.136289+0000 mgr.gibba032.whopeo (mgr.24625) 9032 : audit [DBG] from='client.60091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.146469+0000 mon.a (mon.0) 3095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:43:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.147473+0000 mon.a (mon.0) 3096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:43:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.283230+0000 mon.a (mon.0) 3097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:43:58.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.284006+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.284668+0000 mon.a (mon.0) 3099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[30791]: audit 2021-03-10T16:43:57.285331+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:43:58.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:56.136289+0000 mgr.gibba032.whopeo (mgr.24625) 9032 : audit [DBG] from='client.60091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.146469+0000 mon.a (mon.0) 3095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:43:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.147473+0000 mon.a (mon.0) 3096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:43:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.283230+0000 mon.a (mon.0) 3097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:43:58.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.284006+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.284668+0000 mon.a (mon.0) 3099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:58 gibba011 bash[31774]: audit 2021-03-10T16:43:57.285331+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:43:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:56.136289+0000 mgr.gibba032.whopeo (mgr.24625) 9032 : audit [DBG] from='client.60091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:43:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.146469+0000 mon.a (mon.0) 3095 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:43:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.147473+0000 mon.a (mon.0) 3096 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:43:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.283230+0000 mon.a (mon.0) 3097 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:43:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.284006+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.284668+0000 mon.a (mon.0) 3099 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:43:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:58 gibba032 bash[24004]: audit 2021-03-10T16:43:57.285331+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:43:59.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:43:59 gibba011 bash[30791]: cluster 2021-03-10T16:43:57.966863+0000 mgr.gibba032.whopeo (mgr.24625) 9033 : cluster [DBG] pgmap v5142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:59.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:43:59 gibba011 bash[31774]: cluster 2021-03-10T16:43:57.966863+0000 mgr.gibba032.whopeo (mgr.24625) 9033 : cluster [DBG] pgmap v5142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:43:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:43:59 gibba032 bash[24004]: cluster 2021-03-10T16:43:57.966863+0000 mgr.gibba032.whopeo (mgr.24625) 9033 : cluster [DBG] pgmap v5142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:01 gibba032 bash[24004]: cluster 2021-03-10T16:43:59.968251+0000 mgr.gibba032.whopeo (mgr.24625) 9034 : cluster [DBG] pgmap v5143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:01.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:01 gibba011 bash[30791]: cluster 2021-03-10T16:43:59.968251+0000 mgr.gibba032.whopeo (mgr.24625) 9034 : cluster [DBG] pgmap v5143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:01.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:01 gibba011 bash[31774]: cluster 2021-03-10T16:43:59.968251+0000 mgr.gibba032.whopeo (mgr.24625) 9034 : cluster [DBG] pgmap v5143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:03 gibba032 bash[24004]: cluster 2021-03-10T16:44:01.969994+0000 mgr.gibba032.whopeo (mgr.24625) 9035 : cluster [DBG] pgmap v5144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:03.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:03 gibba011 bash[30791]: cluster 2021-03-10T16:44:01.969994+0000 mgr.gibba032.whopeo (mgr.24625) 9035 : cluster [DBG] pgmap v5144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:03.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:03 gibba011 bash[31774]: cluster 2021-03-10T16:44:01.969994+0000 mgr.gibba032.whopeo (mgr.24625) 9035 : cluster [DBG] pgmap v5144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:05.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:05 gibba032 bash[24004]: cluster 2021-03-10T16:44:03.971457+0000 mgr.gibba032.whopeo (mgr.24625) 9036 : cluster [DBG] pgmap v5145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:05.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:05 gibba011 bash[30791]: cluster 2021-03-10T16:44:03.971457+0000 mgr.gibba032.whopeo (mgr.24625) 9036 : cluster [DBG] pgmap v5145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:05.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:05 gibba011 bash[31774]: cluster 2021-03-10T16:44:03.971457+0000 mgr.gibba032.whopeo (mgr.24625) 9036 : cluster [DBG] pgmap v5145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:06.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:06 gibba032 bash[24004]: cluster 2021-03-10T16:44:05.972978+0000 mgr.gibba032.whopeo (mgr.24625) 9037 : cluster [DBG] pgmap v5146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:06 gibba011 bash[30791]: cluster 2021-03-10T16:44:05.972978+0000 mgr.gibba032.whopeo (mgr.24625) 9037 : cluster [DBG] pgmap v5146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:06 gibba011 bash[31774]: cluster 2021-03-10T16:44:05.972978+0000 mgr.gibba032.whopeo (mgr.24625) 9037 : cluster [DBG] pgmap v5146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:09.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:09 gibba011 bash[30791]: cluster 2021-03-10T16:44:07.974652+0000 mgr.gibba032.whopeo (mgr.24625) 9038 : cluster [DBG] pgmap v5147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:09.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:09 gibba011 bash[31774]: cluster 2021-03-10T16:44:07.974652+0000 mgr.gibba032.whopeo (mgr.24625) 9038 : cluster [DBG] pgmap v5147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:09 gibba032 bash[24004]: cluster 2021-03-10T16:44:07.974652+0000 mgr.gibba032.whopeo (mgr.24625) 9038 : cluster [DBG] pgmap v5147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:11 gibba011 bash[30791]: cluster 2021-03-10T16:44:09.976003+0000 mgr.gibba032.whopeo (mgr.24625) 9039 : cluster [DBG] pgmap v5148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:11.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:11 gibba011 bash[31774]: cluster 2021-03-10T16:44:09.976003+0000 mgr.gibba032.whopeo (mgr.24625) 9039 : cluster [DBG] pgmap v5148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:11.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:11 gibba032 bash[24004]: cluster 2021-03-10T16:44:09.976003+0000 mgr.gibba032.whopeo (mgr.24625) 9039 : cluster [DBG] pgmap v5148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:12 gibba011 bash[30791]: cephadm 2021-03-10T16:44:10.257180+0000 mgr.gibba032.whopeo (mgr.24625) 9040 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:12.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:12 gibba011 bash[31774]: cephadm 2021-03-10T16:44:10.257180+0000 mgr.gibba032.whopeo (mgr.24625) 9040 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:12.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:12 gibba032 bash[24004]: cephadm 2021-03-10T16:44:10.257180+0000 mgr.gibba032.whopeo (mgr.24625) 9040 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:13 gibba011 bash[30791]: cluster 2021-03-10T16:44:11.977562+0000 mgr.gibba032.whopeo (mgr.24625) 9041 : cluster [DBG] pgmap v5149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:13.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:13 gibba011 bash[31774]: cluster 2021-03-10T16:44:11.977562+0000 mgr.gibba032.whopeo (mgr.24625) 9041 : cluster [DBG] pgmap v5149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:13.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:13 gibba032 bash[24004]: cluster 2021-03-10T16:44:11.977562+0000 mgr.gibba032.whopeo (mgr.24625) 9041 : cluster [DBG] pgmap v5149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:15.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:15 gibba011 bash[30791]: cluster 2021-03-10T16:44:13.978897+0000 mgr.gibba032.whopeo (mgr.24625) 9042 : cluster [DBG] pgmap v5150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:15.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:15 gibba011 bash[31774]: cluster 2021-03-10T16:44:13.978897+0000 mgr.gibba032.whopeo (mgr.24625) 9042 : cluster [DBG] pgmap v5150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:15.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:15 gibba032 bash[24004]: cluster 2021-03-10T16:44:13.978897+0000 mgr.gibba032.whopeo (mgr.24625) 9042 : cluster [DBG] pgmap v5150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:17.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:17 gibba011 bash[30791]: cluster 2021-03-10T16:44:15.980268+0000 mgr.gibba032.whopeo (mgr.24625) 9043 : cluster [DBG] pgmap v5151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:17.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:17 gibba011 bash[31774]: cluster 2021-03-10T16:44:15.980268+0000 mgr.gibba032.whopeo (mgr.24625) 9043 : cluster [DBG] pgmap v5151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:17.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:17 gibba032 bash[24004]: cluster 2021-03-10T16:44:15.980268+0000 mgr.gibba032.whopeo (mgr.24625) 9043 : cluster [DBG] pgmap v5151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:19.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:19 gibba032 bash[24004]: cluster 2021-03-10T16:44:17.982179+0000 mgr.gibba032.whopeo (mgr.24625) 9044 : cluster [DBG] pgmap v5152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:19.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:19 gibba011 bash[30791]: cluster 2021-03-10T16:44:17.982179+0000 mgr.gibba032.whopeo (mgr.24625) 9044 : cluster [DBG] pgmap v5152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:19.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:19 gibba011 bash[31774]: cluster 2021-03-10T16:44:17.982179+0000 mgr.gibba032.whopeo (mgr.24625) 9044 : cluster [DBG] pgmap v5152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:21.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:21 gibba032 bash[24004]: cluster 2021-03-10T16:44:19.983490+0000 mgr.gibba032.whopeo (mgr.24625) 9045 : cluster [DBG] pgmap v5153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:21.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:21 gibba011 bash[30791]: cluster 2021-03-10T16:44:19.983490+0000 mgr.gibba032.whopeo (mgr.24625) 9045 : cluster [DBG] pgmap v5153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:21.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:21 gibba011 bash[31774]: cluster 2021-03-10T16:44:19.983490+0000 mgr.gibba032.whopeo (mgr.24625) 9045 : cluster [DBG] pgmap v5153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:23.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:23 gibba032 bash[24004]: cluster 2021-03-10T16:44:21.985254+0000 mgr.gibba032.whopeo (mgr.24625) 9046 : cluster [DBG] pgmap v5154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:23.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:23 gibba011 bash[30791]: cluster 2021-03-10T16:44:21.985254+0000 mgr.gibba032.whopeo (mgr.24625) 9046 : cluster [DBG] pgmap v5154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:23.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:23 gibba011 bash[31774]: cluster 2021-03-10T16:44:21.985254+0000 mgr.gibba032.whopeo (mgr.24625) 9046 : cluster [DBG] pgmap v5154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:24.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:24 gibba032 bash[24004]: cluster 2021-03-10T16:44:23.986823+0000 mgr.gibba032.whopeo (mgr.24625) 9047 : cluster [DBG] pgmap v5155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:24 gibba011 bash[30791]: cluster 2021-03-10T16:44:23.986823+0000 mgr.gibba032.whopeo (mgr.24625) 9047 : cluster [DBG] pgmap v5155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:24 gibba011 bash[31774]: cluster 2021-03-10T16:44:23.986823+0000 mgr.gibba032.whopeo (mgr.24625) 9047 : cluster [DBG] pgmap v5155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:26.642 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:44:26.904 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:44:26.905 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 3m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:44:26.905 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 2m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:44:26.905 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:44:26.906 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:44:26.906 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:44:26.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:44:26.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:44:26.906 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:44:26.907 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:44:26.907 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 2m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:44:26.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:44:26.907 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:44:26.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:44:26.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:44:26.908 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:44:26.917 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:44:26.918 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:44:26.918 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:44:26.918 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 2m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:44:26.918 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:44:27.167 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:44:27.168 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:44:27.168 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:44:27.168 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:27.168 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:44:27.169 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:44:27.169 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:27.169 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:44:27.169 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:44:27.170 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:27.170 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:44:27.171 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:44:27.171 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:44:27.171 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:44:27.171 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:44:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:27 gibba011 bash[30791]: cephadm 2021-03-10T16:44:25.273244+0000 mgr.gibba032.whopeo (mgr.24625) 9048 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:27.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:27 gibba011 bash[30791]: cluster 2021-03-10T16:44:25.988219+0000 mgr.gibba032.whopeo (mgr.24625) 9049 : cluster [DBG] pgmap v5156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:27 gibba011 bash[31774]: cephadm 2021-03-10T16:44:25.273244+0000 mgr.gibba032.whopeo (mgr.24625) 9048 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:27.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:27 gibba011 bash[31774]: cluster 2021-03-10T16:44:25.988219+0000 mgr.gibba032.whopeo (mgr.24625) 9049 : cluster [DBG] pgmap v5156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:27 gibba032 bash[24004]: cephadm 2021-03-10T16:44:25.273244+0000 mgr.gibba032.whopeo (mgr.24625) 9048 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:27 gibba032 bash[24004]: cluster 2021-03-10T16:44:25.988219+0000 mgr.gibba032.whopeo (mgr.24625) 9049 : cluster [DBG] pgmap v5156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:28.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[30791]: audit 2021-03-10T16:44:26.634035+0000 mgr.gibba032.whopeo (mgr.24625) 9050 : audit [DBG] from='client.50121 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[30791]: audit 2021-03-10T16:44:26.865473+0000 mgr.gibba032.whopeo (mgr.24625) 9051 : audit [DBG] from='client.60109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[30791]: audit 2021-03-10T16:44:27.170477+0000 mon.a (mon.0) 3101 : audit [DBG] from='client.? 172.21.2.111:0/928848856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[31774]: audit 2021-03-10T16:44:26.634035+0000 mgr.gibba032.whopeo (mgr.24625) 9050 : audit [DBG] from='client.50121 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[31774]: audit 2021-03-10T16:44:26.865473+0000 mgr.gibba032.whopeo (mgr.24625) 9051 : audit [DBG] from='client.60109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:28 gibba011 bash[31774]: audit 2021-03-10T16:44:27.170477+0000 mon.a (mon.0) 3101 : audit [DBG] from='client.? 172.21.2.111:0/928848856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:28.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:28 gibba032 bash[24004]: audit 2021-03-10T16:44:26.634035+0000 mgr.gibba032.whopeo (mgr.24625) 9050 : audit [DBG] from='client.50121 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:28 gibba032 bash[24004]: audit 2021-03-10T16:44:26.865473+0000 mgr.gibba032.whopeo (mgr.24625) 9051 : audit [DBG] from='client.60109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:28.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:28 gibba032 bash[24004]: audit 2021-03-10T16:44:27.170477+0000 mon.a (mon.0) 3101 : audit [DBG] from='client.? 172.21.2.111:0/928848856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:29.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:29 gibba011 bash[30791]: cluster 2021-03-10T16:44:27.990091+0000 mgr.gibba032.whopeo (mgr.24625) 9052 : cluster [DBG] pgmap v5157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:29.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:29 gibba011 bash[31774]: cluster 2021-03-10T16:44:27.990091+0000 mgr.gibba032.whopeo (mgr.24625) 9052 : cluster [DBG] pgmap v5157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:29.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:29 gibba032 bash[24004]: cluster 2021-03-10T16:44:27.990091+0000 mgr.gibba032.whopeo (mgr.24625) 9052 : cluster [DBG] pgmap v5157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:31.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:31 gibba032 bash[24004]: cluster 2021-03-10T16:44:29.991147+0000 mgr.gibba032.whopeo (mgr.24625) 9053 : cluster [DBG] pgmap v5158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:31 gibba011 bash[30791]: cluster 2021-03-10T16:44:29.991147+0000 mgr.gibba032.whopeo (mgr.24625) 9053 : cluster [DBG] pgmap v5158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:31.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:31 gibba011 bash[31774]: cluster 2021-03-10T16:44:29.991147+0000 mgr.gibba032.whopeo (mgr.24625) 9053 : cluster [DBG] pgmap v5158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:33.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:33 gibba011 bash[30791]: cluster 2021-03-10T16:44:32.000852+0000 mgr.gibba032.whopeo (mgr.24625) 9054 : cluster [DBG] pgmap v5159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:33.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:33 gibba011 bash[31774]: cluster 2021-03-10T16:44:32.000852+0000 mgr.gibba032.whopeo (mgr.24625) 9054 : cluster [DBG] pgmap v5159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:33 gibba032 bash[24004]: cluster 2021-03-10T16:44:32.000852+0000 mgr.gibba032.whopeo (mgr.24625) 9054 : cluster [DBG] pgmap v5159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:35.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:35 gibba032 bash[24004]: cluster 2021-03-10T16:44:34.002532+0000 mgr.gibba032.whopeo (mgr.24625) 9055 : cluster [DBG] pgmap v5160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:35.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:35 gibba011 bash[30791]: cluster 2021-03-10T16:44:34.002532+0000 mgr.gibba032.whopeo (mgr.24625) 9055 : cluster [DBG] pgmap v5160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:35.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:35 gibba011 bash[31774]: cluster 2021-03-10T16:44:34.002532+0000 mgr.gibba032.whopeo (mgr.24625) 9055 : cluster [DBG] pgmap v5160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:37 gibba032 bash[24004]: cluster 2021-03-10T16:44:36.004106+0000 mgr.gibba032.whopeo (mgr.24625) 9056 : cluster [DBG] pgmap v5161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:37.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:37 gibba011 bash[30791]: cluster 2021-03-10T16:44:36.004106+0000 mgr.gibba032.whopeo (mgr.24625) 9056 : cluster [DBG] pgmap v5161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:37.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:37 gibba011 bash[31774]: cluster 2021-03-10T16:44:36.004106+0000 mgr.gibba032.whopeo (mgr.24625) 9056 : cluster [DBG] pgmap v5161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:39.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:39 gibba032 bash[24004]: cluster 2021-03-10T16:44:38.005977+0000 mgr.gibba032.whopeo (mgr.24625) 9057 : cluster [DBG] pgmap v5162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:39.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:39 gibba011 bash[30791]: cluster 2021-03-10T16:44:38.005977+0000 mgr.gibba032.whopeo (mgr.24625) 9057 : cluster [DBG] pgmap v5162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:39.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:39 gibba011 bash[31774]: cluster 2021-03-10T16:44:38.005977+0000 mgr.gibba032.whopeo (mgr.24625) 9057 : cluster [DBG] pgmap v5162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: cluster 2021-03-10T16:44:40.006570+0000 mgr.gibba032.whopeo (mgr.24625) 9058 : cluster [DBG] pgmap v5163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.517835+0000 mon.a (mon.0) 3102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:44:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.521353+0000 mon.a (mon.0) 3103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:44:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.523066+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.524508+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.526121+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.529473+0000 mon.a (mon.0) 3107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[30791]: audit 2021-03-10T16:44:40.531757+0000 mon.a (mon.0) 3108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: cluster 2021-03-10T16:44:40.006570+0000 mgr.gibba032.whopeo (mgr.24625) 9058 : cluster [DBG] pgmap v5163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.517835+0000 mon.a (mon.0) 3102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:44:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.521353+0000 mon.a (mon.0) 3103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:44:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.523066+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.524508+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.526121+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.529473+0000 mon.a (mon.0) 3107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:41 gibba011 bash[31774]: audit 2021-03-10T16:44:40.531757+0000 mon.a (mon.0) 3108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: cluster 2021-03-10T16:44:40.006570+0000 mgr.gibba032.whopeo (mgr.24625) 9058 : cluster [DBG] pgmap v5163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:41.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.517835+0000 mon.a (mon.0) 3102 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:44:41.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.521353+0000 mon.a (mon.0) 3103 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:44:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.523066+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.524508+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.526121+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.529473+0000 mon.a (mon.0) 3107 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:41.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:41 gibba032 bash[24004]: audit 2021-03-10T16:44:40.531757+0000 mon.a (mon.0) 3108 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.289874+0000 mgr.gibba032.whopeo (mgr.24625) 9059 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:44:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.291423+0000 mgr.gibba032.whopeo (mgr.24625) 9060 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:44:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.518702+0000 mgr.gibba032.whopeo (mgr.24625) 9061 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:44:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.520808+0000 mgr.gibba032.whopeo (mgr.24625) 9062 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:44:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.522256+0000 mgr.gibba032.whopeo (mgr.24625) 9063 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:44:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.522303+0000 mgr.gibba032.whopeo (mgr.24625) 9064 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:44:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.523689+0000 mgr.gibba032.whopeo (mgr.24625) 9065 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:44:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.523736+0000 mgr.gibba032.whopeo (mgr.24625) 9066 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:44:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.525660+0000 mgr.gibba032.whopeo (mgr.24625) 9067 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:44:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.526866+0000 mgr.gibba032.whopeo (mgr.24625) 9068 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:44:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.526916+0000 mgr.gibba032.whopeo (mgr.24625) 9069 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:44:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.528916+0000 mgr.gibba032.whopeo (mgr.24625) 9070 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:44:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.529326+0000 mgr.gibba032.whopeo (mgr.24625) 9071 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:44:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.530905+0000 mgr.gibba032.whopeo (mgr.24625) 9072 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:44:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.530957+0000 mgr.gibba032.whopeo (mgr.24625) 9073 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:44:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[30791]: cephadm 2021-03-10T16:44:40.531092+0000 mgr.gibba032.whopeo (mgr.24625) 9074 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.289874+0000 mgr.gibba032.whopeo (mgr.24625) 9059 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:44:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.291423+0000 mgr.gibba032.whopeo (mgr.24625) 9060 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:44:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.518702+0000 mgr.gibba032.whopeo (mgr.24625) 9061 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:44:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.520808+0000 mgr.gibba032.whopeo (mgr.24625) 9062 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:44:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.522256+0000 mgr.gibba032.whopeo (mgr.24625) 9063 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:44:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.522303+0000 mgr.gibba032.whopeo (mgr.24625) 9064 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:44:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.523689+0000 mgr.gibba032.whopeo (mgr.24625) 9065 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:44:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.523736+0000 mgr.gibba032.whopeo (mgr.24625) 9066 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:44:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.525660+0000 mgr.gibba032.whopeo (mgr.24625) 9067 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:44:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.526866+0000 mgr.gibba032.whopeo (mgr.24625) 9068 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:44:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.526916+0000 mgr.gibba032.whopeo (mgr.24625) 9069 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:44:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.528916+0000 mgr.gibba032.whopeo (mgr.24625) 9070 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:44:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.529326+0000 mgr.gibba032.whopeo (mgr.24625) 9071 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:44:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.530905+0000 mgr.gibba032.whopeo (mgr.24625) 9072 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:44:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.530957+0000 mgr.gibba032.whopeo (mgr.24625) 9073 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:44:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:42 gibba011 bash[31774]: cephadm 2021-03-10T16:44:40.531092+0000 mgr.gibba032.whopeo (mgr.24625) 9074 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.289874+0000 mgr.gibba032.whopeo (mgr.24625) 9059 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:44:42.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.291423+0000 mgr.gibba032.whopeo (mgr.24625) 9060 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:44:42.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.518702+0000 mgr.gibba032.whopeo (mgr.24625) 9061 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:44:42.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.520808+0000 mgr.gibba032.whopeo (mgr.24625) 9062 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:44:42.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.522256+0000 mgr.gibba032.whopeo (mgr.24625) 9063 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:44:42.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.522303+0000 mgr.gibba032.whopeo (mgr.24625) 9064 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:44:42.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.523689+0000 mgr.gibba032.whopeo (mgr.24625) 9065 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:44:42.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.523736+0000 mgr.gibba032.whopeo (mgr.24625) 9066 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:44:42.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.525660+0000 mgr.gibba032.whopeo (mgr.24625) 9067 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:44:42.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.526866+0000 mgr.gibba032.whopeo (mgr.24625) 9068 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:44:42.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.526916+0000 mgr.gibba032.whopeo (mgr.24625) 9069 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:44:42.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.528916+0000 mgr.gibba032.whopeo (mgr.24625) 9070 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:44:42.516 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.529326+0000 mgr.gibba032.whopeo (mgr.24625) 9071 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:44:42.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.530905+0000 mgr.gibba032.whopeo (mgr.24625) 9072 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:44:42.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.530957+0000 mgr.gibba032.whopeo (mgr.24625) 9073 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:44:42.517 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:42 gibba032 bash[24004]: cephadm 2021-03-10T16:44:40.531092+0000 mgr.gibba032.whopeo (mgr.24625) 9074 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:43 gibba011 bash[30791]: cluster 2021-03-10T16:44:42.008265+0000 mgr.gibba032.whopeo (mgr.24625) 9075 : cluster [DBG] pgmap v5164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:43.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:43 gibba011 bash[31774]: cluster 2021-03-10T16:44:42.008265+0000 mgr.gibba032.whopeo (mgr.24625) 9075 : cluster [DBG] pgmap v5164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:43.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:43 gibba032 bash[24004]: cluster 2021-03-10T16:44:42.008265+0000 mgr.gibba032.whopeo (mgr.24625) 9075 : cluster [DBG] pgmap v5164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:45.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:45 gibba011 bash[30791]: cluster 2021-03-10T16:44:44.009569+0000 mgr.gibba032.whopeo (mgr.24625) 9076 : cluster [DBG] pgmap v5165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:45.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:45 gibba011 bash[31774]: cluster 2021-03-10T16:44:44.009569+0000 mgr.gibba032.whopeo (mgr.24625) 9076 : cluster [DBG] pgmap v5165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:45.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:45 gibba032 bash[24004]: cluster 2021-03-10T16:44:44.009569+0000 mgr.gibba032.whopeo (mgr.24625) 9076 : cluster [DBG] pgmap v5165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:47.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:47 gibba032 bash[24004]: cluster 2021-03-10T16:44:46.010902+0000 mgr.gibba032.whopeo (mgr.24625) 9077 : cluster [DBG] pgmap v5166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:47.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:47 gibba011 bash[30791]: cluster 2021-03-10T16:44:46.010902+0000 mgr.gibba032.whopeo (mgr.24625) 9077 : cluster [DBG] pgmap v5166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:47.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:47 gibba011 bash[31774]: cluster 2021-03-10T16:44:46.010902+0000 mgr.gibba032.whopeo (mgr.24625) 9077 : cluster [DBG] pgmap v5166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:49.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:49 gibba032 bash[24004]: cluster 2021-03-10T16:44:48.012737+0000 mgr.gibba032.whopeo (mgr.24625) 9078 : cluster [DBG] pgmap v5167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:49.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:49 gibba011 bash[30791]: cluster 2021-03-10T16:44:48.012737+0000 mgr.gibba032.whopeo (mgr.24625) 9078 : cluster [DBG] pgmap v5167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:49.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:49 gibba011 bash[31774]: cluster 2021-03-10T16:44:48.012737+0000 mgr.gibba032.whopeo (mgr.24625) 9078 : cluster [DBG] pgmap v5167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:51 gibba011 bash[30791]: cluster 2021-03-10T16:44:50.013474+0000 mgr.gibba032.whopeo (mgr.24625) 9079 : cluster [DBG] pgmap v5168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:51.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:51 gibba011 bash[31774]: cluster 2021-03-10T16:44:50.013474+0000 mgr.gibba032.whopeo (mgr.24625) 9079 : cluster [DBG] pgmap v5168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:51.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:51 gibba032 bash[24004]: cluster 2021-03-10T16:44:50.013474+0000 mgr.gibba032.whopeo (mgr.24625) 9079 : cluster [DBG] pgmap v5168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:53 gibba011 bash[30791]: cluster 2021-03-10T16:44:52.014975+0000 mgr.gibba032.whopeo (mgr.24625) 9080 : cluster [DBG] pgmap v5169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:53.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:53 gibba011 bash[31774]: cluster 2021-03-10T16:44:52.014975+0000 mgr.gibba032.whopeo (mgr.24625) 9080 : cluster [DBG] pgmap v5169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:53.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:53 gibba032 bash[24004]: cluster 2021-03-10T16:44:52.014975+0000 mgr.gibba032.whopeo (mgr.24625) 9080 : cluster [DBG] pgmap v5169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:55.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:55 gibba032 bash[24004]: cluster 2021-03-10T16:44:54.016647+0000 mgr.gibba032.whopeo (mgr.24625) 9081 : cluster [DBG] pgmap v5170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:55.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:55 gibba011 bash[30791]: cluster 2021-03-10T16:44:54.016647+0000 mgr.gibba032.whopeo (mgr.24625) 9081 : cluster [DBG] pgmap v5170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:55.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:55 gibba011 bash[31774]: cluster 2021-03-10T16:44:54.016647+0000 mgr.gibba032.whopeo (mgr.24625) 9081 : cluster [DBG] pgmap v5170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:57.412 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:44:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:57 gibba032 bash[24004]: cephadm 2021-03-10T16:44:55.546715+0000 mgr.gibba032.whopeo (mgr.24625) 9082 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:57 gibba032 bash[24004]: cluster 2021-03-10T16:44:56.018025+0000 mgr.gibba032.whopeo (mgr.24625) 9083 : cluster [DBG] pgmap v5171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:57.575 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:57 gibba011 bash[30791]: cephadm 2021-03-10T16:44:55.546715+0000 mgr.gibba032.whopeo (mgr.24625) 9082 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:57 gibba011 bash[30791]: cluster 2021-03-10T16:44:56.018025+0000 mgr.gibba032.whopeo (mgr.24625) 9083 : cluster [DBG] pgmap v5171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:57.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:57 gibba011 bash[31774]: cephadm 2021-03-10T16:44:55.546715+0000 mgr.gibba032.whopeo (mgr.24625) 9082 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:44:57.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:57 gibba011 bash[31774]: cluster 2021-03-10T16:44:56.018025+0000 mgr.gibba032.whopeo (mgr.24625) 9083 : cluster [DBG] pgmap v5171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:57.641 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:44:57.642 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:44:57.642 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:44:57.642 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:44:57.642 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:44:57.643 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:44:57.643 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:44:57.643 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:44:57.643 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:44:57.643 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:44:57.644 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:44:57.644 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:44:57.644 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:44:57.644 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:44:57.644 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:44:57.645 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:44:57.645 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:44:57.645 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:44:57.645 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:44:57.645 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:44:57.646 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:44:57.893 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:44:57.893 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:44:57.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:44:57.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:57.894 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:44:57.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:44:57.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:57.895 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:44:57.895 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:44:57.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:44:57.895 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:44:57.896 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:44:57.896 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:44:57.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:44:57.896 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:44:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.151495+0000 mon.a (mon.0) 3109 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:44:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.151719+0000 mon.a (mon.0) 3110 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:44:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.292773+0000 mon.a (mon.0) 3111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:44:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.292960+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:44:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.293155+0000 mon.a (mon.0) 3113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.293343+0000 mon.a (mon.0) 3114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:58 gibba032 bash[24004]: audit 2021-03-10T16:44:57.896190+0000 mon.a (mon.0) 3115 : audit [DBG] from='client.? 172.21.2.111:0/491555692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.151495+0000 mon.a (mon.0) 3109 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.151719+0000 mon.a (mon.0) 3110 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.292773+0000 mon.a (mon.0) 3111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.292960+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.293155+0000 mon.a (mon.0) 3113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.293343+0000 mon.a (mon.0) 3114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[30791]: audit 2021-03-10T16:44:57.896190+0000 mon.a (mon.0) 3115 : audit [DBG] from='client.? 172.21.2.111:0/491555692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.151495+0000 mon.a (mon.0) 3109 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:44:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.151719+0000 mon.a (mon.0) 3110 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.292773+0000 mon.a (mon.0) 3111 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.292960+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.293155+0000 mon.a (mon.0) 3113 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.293343+0000 mon.a (mon.0) 3114 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:44:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:58 gibba011 bash[31774]: audit 2021-03-10T16:44:57.896190+0000 mon.a (mon.0) 3115 : audit [DBG] from='client.? 172.21.2.111:0/491555692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:44:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:59 gibba032 bash[24004]: audit 2021-03-10T16:44:57.403842+0000 mgr.gibba032.whopeo (mgr.24625) 9084 : audit [DBG] from='client.50139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:59 gibba032 bash[24004]: audit 2021-03-10T16:44:57.630237+0000 mgr.gibba032.whopeo (mgr.24625) 9085 : audit [DBG] from='client.50145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:44:59 gibba032 bash[24004]: cluster 2021-03-10T16:44:58.019935+0000 mgr.gibba032.whopeo (mgr.24625) 9086 : cluster [DBG] pgmap v5172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:59.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[30791]: audit 2021-03-10T16:44:57.403842+0000 mgr.gibba032.whopeo (mgr.24625) 9084 : audit [DBG] from='client.50139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[30791]: audit 2021-03-10T16:44:57.630237+0000 mgr.gibba032.whopeo (mgr.24625) 9085 : audit [DBG] from='client.50145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[30791]: cluster 2021-03-10T16:44:58.019935+0000 mgr.gibba032.whopeo (mgr.24625) 9086 : cluster [DBG] pgmap v5172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:44:59.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[31774]: audit 2021-03-10T16:44:57.403842+0000 mgr.gibba032.whopeo (mgr.24625) 9084 : audit [DBG] from='client.50139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[31774]: audit 2021-03-10T16:44:57.630237+0000 mgr.gibba032.whopeo (mgr.24625) 9085 : audit [DBG] from='client.50145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:44:59.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:44:59 gibba011 bash[31774]: cluster 2021-03-10T16:44:58.019935+0000 mgr.gibba032.whopeo (mgr.24625) 9086 : cluster [DBG] pgmap v5172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:01 gibba011 bash[30791]: cluster 2021-03-10T16:45:00.020406+0000 mgr.gibba032.whopeo (mgr.24625) 9087 : cluster [DBG] pgmap v5173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:01.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:01 gibba011 bash[31774]: cluster 2021-03-10T16:45:00.020406+0000 mgr.gibba032.whopeo (mgr.24625) 9087 : cluster [DBG] pgmap v5173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:01 gibba032 bash[24004]: cluster 2021-03-10T16:45:00.020406+0000 mgr.gibba032.whopeo (mgr.24625) 9087 : cluster [DBG] pgmap v5173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:03.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:03 gibba032 bash[24004]: cluster 2021-03-10T16:45:02.021929+0000 mgr.gibba032.whopeo (mgr.24625) 9088 : cluster [DBG] pgmap v5174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:03.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:03 gibba011 bash[30791]: cluster 2021-03-10T16:45:02.021929+0000 mgr.gibba032.whopeo (mgr.24625) 9088 : cluster [DBG] pgmap v5174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:03.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:03 gibba011 bash[31774]: cluster 2021-03-10T16:45:02.021929+0000 mgr.gibba032.whopeo (mgr.24625) 9088 : cluster [DBG] pgmap v5174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:05.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:05 gibba032 bash[24004]: cluster 2021-03-10T16:45:04.023567+0000 mgr.gibba032.whopeo (mgr.24625) 9089 : cluster [DBG] pgmap v5175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:05.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:05 gibba011 bash[30791]: cluster 2021-03-10T16:45:04.023567+0000 mgr.gibba032.whopeo (mgr.24625) 9089 : cluster [DBG] pgmap v5175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:05.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:05 gibba011 bash[31774]: cluster 2021-03-10T16:45:04.023567+0000 mgr.gibba032.whopeo (mgr.24625) 9089 : cluster [DBG] pgmap v5175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:07.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:07 gibba032 bash[24004]: cluster 2021-03-10T16:45:06.025066+0000 mgr.gibba032.whopeo (mgr.24625) 9090 : cluster [DBG] pgmap v5176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:07.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:07 gibba011 bash[30791]: cluster 2021-03-10T16:45:06.025066+0000 mgr.gibba032.whopeo (mgr.24625) 9090 : cluster [DBG] pgmap v5176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:07.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:07 gibba011 bash[31774]: cluster 2021-03-10T16:45:06.025066+0000 mgr.gibba032.whopeo (mgr.24625) 9090 : cluster [DBG] pgmap v5176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:09.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:09 gibba032 bash[24004]: cluster 2021-03-10T16:45:08.026706+0000 mgr.gibba032.whopeo (mgr.24625) 9091 : cluster [DBG] pgmap v5177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:09.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:09 gibba011 bash[30791]: cluster 2021-03-10T16:45:08.026706+0000 mgr.gibba032.whopeo (mgr.24625) 9091 : cluster [DBG] pgmap v5177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:09.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:09 gibba011 bash[31774]: cluster 2021-03-10T16:45:08.026706+0000 mgr.gibba032.whopeo (mgr.24625) 9091 : cluster [DBG] pgmap v5177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:11 gibba011 bash[30791]: cluster 2021-03-10T16:45:10.027425+0000 mgr.gibba032.whopeo (mgr.24625) 9092 : cluster [DBG] pgmap v5178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:11.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:11 gibba011 bash[31774]: cluster 2021-03-10T16:45:10.027425+0000 mgr.gibba032.whopeo (mgr.24625) 9092 : cluster [DBG] pgmap v5178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:11 gibba032 bash[24004]: cluster 2021-03-10T16:45:10.027425+0000 mgr.gibba032.whopeo (mgr.24625) 9092 : cluster [DBG] pgmap v5178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:12 gibba011 bash[30791]: cephadm 2021-03-10T16:45:10.562590+0000 mgr.gibba032.whopeo (mgr.24625) 9093 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:12.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:12 gibba011 bash[31774]: cephadm 2021-03-10T16:45:10.562590+0000 mgr.gibba032.whopeo (mgr.24625) 9093 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:12.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:12 gibba032 bash[24004]: cephadm 2021-03-10T16:45:10.562590+0000 mgr.gibba032.whopeo (mgr.24625) 9093 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:13 gibba011 bash[30791]: cluster 2021-03-10T16:45:12.028714+0000 mgr.gibba032.whopeo (mgr.24625) 9094 : cluster [DBG] pgmap v5179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:13.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:13 gibba011 bash[31774]: cluster 2021-03-10T16:45:12.028714+0000 mgr.gibba032.whopeo (mgr.24625) 9094 : cluster [DBG] pgmap v5179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:13.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:13 gibba032 bash[24004]: cluster 2021-03-10T16:45:12.028714+0000 mgr.gibba032.whopeo (mgr.24625) 9094 : cluster [DBG] pgmap v5179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:15.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:15 gibba032 bash[24004]: cluster 2021-03-10T16:45:14.030314+0000 mgr.gibba032.whopeo (mgr.24625) 9095 : cluster [DBG] pgmap v5180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:15.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:15 gibba011 bash[30791]: cluster 2021-03-10T16:45:14.030314+0000 mgr.gibba032.whopeo (mgr.24625) 9095 : cluster [DBG] pgmap v5180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:15.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:15 gibba011 bash[31774]: cluster 2021-03-10T16:45:14.030314+0000 mgr.gibba032.whopeo (mgr.24625) 9095 : cluster [DBG] pgmap v5180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:17.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:17 gibba032 bash[24004]: cluster 2021-03-10T16:45:16.031789+0000 mgr.gibba032.whopeo (mgr.24625) 9096 : cluster [DBG] pgmap v5181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:17.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:17 gibba011 bash[30791]: cluster 2021-03-10T16:45:16.031789+0000 mgr.gibba032.whopeo (mgr.24625) 9096 : cluster [DBG] pgmap v5181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:17.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:17 gibba011 bash[31774]: cluster 2021-03-10T16:45:16.031789+0000 mgr.gibba032.whopeo (mgr.24625) 9096 : cluster [DBG] pgmap v5181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:19.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:19 gibba032 bash[24004]: cluster 2021-03-10T16:45:18.032622+0000 mgr.gibba032.whopeo (mgr.24625) 9097 : cluster [DBG] pgmap v5182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:19.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:19 gibba011 bash[30791]: cluster 2021-03-10T16:45:18.032622+0000 mgr.gibba032.whopeo (mgr.24625) 9097 : cluster [DBG] pgmap v5182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:19.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:19 gibba011 bash[31774]: cluster 2021-03-10T16:45:18.032622+0000 mgr.gibba032.whopeo (mgr.24625) 9097 : cluster [DBG] pgmap v5182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:21 gibba011 bash[30791]: cluster 2021-03-10T16:45:20.033303+0000 mgr.gibba032.whopeo (mgr.24625) 9098 : cluster [DBG] pgmap v5183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:21.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:21 gibba011 bash[31774]: cluster 2021-03-10T16:45:20.033303+0000 mgr.gibba032.whopeo (mgr.24625) 9098 : cluster [DBG] pgmap v5183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:21.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:21 gibba032 bash[24004]: cluster 2021-03-10T16:45:20.033303+0000 mgr.gibba032.whopeo (mgr.24625) 9098 : cluster [DBG] pgmap v5183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:23.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:23 gibba032 bash[24004]: cluster 2021-03-10T16:45:22.034720+0000 mgr.gibba032.whopeo (mgr.24625) 9099 : cluster [DBG] pgmap v5184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:23.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:23 gibba011 bash[30791]: cluster 2021-03-10T16:45:22.034720+0000 mgr.gibba032.whopeo (mgr.24625) 9099 : cluster [DBG] pgmap v5184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:23.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:23 gibba011 bash[31774]: cluster 2021-03-10T16:45:22.034720+0000 mgr.gibba032.whopeo (mgr.24625) 9099 : cluster [DBG] pgmap v5184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:25.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:25 gibba032 bash[24004]: cluster 2021-03-10T16:45:24.036313+0000 mgr.gibba032.whopeo (mgr.24625) 9100 : cluster [DBG] pgmap v5185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:25.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:25 gibba011 bash[30791]: cluster 2021-03-10T16:45:24.036313+0000 mgr.gibba032.whopeo (mgr.24625) 9100 : cluster [DBG] pgmap v5185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:25.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:25 gibba011 bash[31774]: cluster 2021-03-10T16:45:24.036313+0000 mgr.gibba032.whopeo (mgr.24625) 9100 : cluster [DBG] pgmap v5185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:27 gibba032 bash[24004]: cephadm 2021-03-10T16:45:25.576756+0000 mgr.gibba032.whopeo (mgr.24625) 9101 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:27 gibba032 bash[24004]: cluster 2021-03-10T16:45:26.037677+0000 mgr.gibba032.whopeo (mgr.24625) 9102 : cluster [DBG] pgmap v5186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:27 gibba011 bash[30791]: cephadm 2021-03-10T16:45:25.576756+0000 mgr.gibba032.whopeo (mgr.24625) 9101 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:27 gibba011 bash[30791]: cluster 2021-03-10T16:45:26.037677+0000 mgr.gibba032.whopeo (mgr.24625) 9102 : cluster [DBG] pgmap v5186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:27.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:27 gibba011 bash[31774]: cephadm 2021-03-10T16:45:25.576756+0000 mgr.gibba032.whopeo (mgr.24625) 9101 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:27.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:27 gibba011 bash[31774]: cluster 2021-03-10T16:45:26.037677+0000 mgr.gibba032.whopeo (mgr.24625) 9102 : cluster [DBG] pgmap v5186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:28.140 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:45:28.375 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:45:28.376 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 4m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:45:28.376 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 3m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:45:28.377 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:45:28.377 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:45:28.377 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:45:28.377 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:45:28.377 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:45:28.378 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:45:28.378 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:45:28.378 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 3m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:45:28.379 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:45:28.379 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:45:28.379 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:45:28.379 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:45:28.380 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:45:28.380 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:45:28.380 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:45:28.380 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:45:28.381 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 3m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:45:28.381 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:45:28.624 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:45:28.624 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:45:28.625 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:45:28.625 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:28.625 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:45:28.625 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:45:28.626 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:28.626 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:45:28.626 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:45:28.626 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:28.626 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:45:28.627 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:45:28.627 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:45:28.627 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:45:28.627 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:45:29.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:29 gibba032 bash[24004]: cluster 2021-03-10T16:45:28.038531+0000 mgr.gibba032.whopeo (mgr.24625) 9103 : cluster [DBG] pgmap v5187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:29.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:29 gibba032 bash[24004]: audit 2021-03-10T16:45:28.627067+0000 mon.c (mon.1) 178 : audit [DBG] from='client.? 172.21.2.111:0/750199773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:29.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:29 gibba011 bash[30791]: cluster 2021-03-10T16:45:28.038531+0000 mgr.gibba032.whopeo (mgr.24625) 9103 : cluster [DBG] pgmap v5187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:29.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:29 gibba011 bash[30791]: audit 2021-03-10T16:45:28.627067+0000 mon.c (mon.1) 178 : audit [DBG] from='client.? 172.21.2.111:0/750199773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:29.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:29 gibba011 bash[31774]: cluster 2021-03-10T16:45:28.038531+0000 mgr.gibba032.whopeo (mgr.24625) 9103 : cluster [DBG] pgmap v5187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:29.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:29 gibba011 bash[31774]: audit 2021-03-10T16:45:28.627067+0000 mon.c (mon.1) 178 : audit [DBG] from='client.? 172.21.2.111:0/750199773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:30.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:30 gibba032 bash[24004]: audit 2021-03-10T16:45:28.133645+0000 mgr.gibba032.whopeo (mgr.24625) 9104 : audit [DBG] from='client.60139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:30.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:30 gibba032 bash[24004]: audit 2021-03-10T16:45:28.364808+0000 mgr.gibba032.whopeo (mgr.24625) 9105 : audit [DBG] from='client.60145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:30.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:30 gibba011 bash[30791]: audit 2021-03-10T16:45:28.133645+0000 mgr.gibba032.whopeo (mgr.24625) 9104 : audit [DBG] from='client.60139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:30 gibba011 bash[30791]: audit 2021-03-10T16:45:28.364808+0000 mgr.gibba032.whopeo (mgr.24625) 9105 : audit [DBG] from='client.60145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:30.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:30 gibba011 bash[31774]: audit 2021-03-10T16:45:28.133645+0000 mgr.gibba032.whopeo (mgr.24625) 9104 : audit [DBG] from='client.60139 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:30 gibba011 bash[31774]: audit 2021-03-10T16:45:28.364808+0000 mgr.gibba032.whopeo (mgr.24625) 9105 : audit [DBG] from='client.60145 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:31 gibba032 bash[24004]: cluster 2021-03-10T16:45:30.039042+0000 mgr.gibba032.whopeo (mgr.24625) 9106 : cluster [DBG] pgmap v5188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:31.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:31 gibba011 bash[30791]: cluster 2021-03-10T16:45:30.039042+0000 mgr.gibba032.whopeo (mgr.24625) 9106 : cluster [DBG] pgmap v5188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:31.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:31 gibba011 bash[31774]: cluster 2021-03-10T16:45:30.039042+0000 mgr.gibba032.whopeo (mgr.24625) 9106 : cluster [DBG] pgmap v5188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:32.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:32 gibba032 bash[24004]: cluster 2021-03-10T16:45:32.040522+0000 mgr.gibba032.whopeo (mgr.24625) 9107 : cluster [DBG] pgmap v5189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:32.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:32 gibba011 bash[30791]: cluster 2021-03-10T16:45:32.040522+0000 mgr.gibba032.whopeo (mgr.24625) 9107 : cluster [DBG] pgmap v5189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:32.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:32 gibba011 bash[31774]: cluster 2021-03-10T16:45:32.040522+0000 mgr.gibba032.whopeo (mgr.24625) 9107 : cluster [DBG] pgmap v5189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:35.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:35 gibba032 bash[24004]: cluster 2021-03-10T16:45:34.042108+0000 mgr.gibba032.whopeo (mgr.24625) 9108 : cluster [DBG] pgmap v5190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:35.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:35 gibba011 bash[30791]: cluster 2021-03-10T16:45:34.042108+0000 mgr.gibba032.whopeo (mgr.24625) 9108 : cluster [DBG] pgmap v5190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:35.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:35 gibba011 bash[31774]: cluster 2021-03-10T16:45:34.042108+0000 mgr.gibba032.whopeo (mgr.24625) 9108 : cluster [DBG] pgmap v5190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:37 gibba032 bash[24004]: cluster 2021-03-10T16:45:36.043493+0000 mgr.gibba032.whopeo (mgr.24625) 9109 : cluster [DBG] pgmap v5191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:37.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:37 gibba011 bash[30791]: cluster 2021-03-10T16:45:36.043493+0000 mgr.gibba032.whopeo (mgr.24625) 9109 : cluster [DBG] pgmap v5191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:37.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:37 gibba011 bash[31774]: cluster 2021-03-10T16:45:36.043493+0000 mgr.gibba032.whopeo (mgr.24625) 9109 : cluster [DBG] pgmap v5191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:39.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:39 gibba032 bash[24004]: cluster 2021-03-10T16:45:38.044114+0000 mgr.gibba032.whopeo (mgr.24625) 9110 : cluster [DBG] pgmap v5192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:39.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:39 gibba011 bash[30791]: cluster 2021-03-10T16:45:38.044114+0000 mgr.gibba032.whopeo (mgr.24625) 9110 : cluster [DBG] pgmap v5192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:39.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:39 gibba011 bash[31774]: cluster 2021-03-10T16:45:38.044114+0000 mgr.gibba032.whopeo (mgr.24625) 9110 : cluster [DBG] pgmap v5192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: cluster 2021-03-10T16:45:40.044718+0000 mgr.gibba032.whopeo (mgr.24625) 9111 : cluster [DBG] pgmap v5193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:41.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.817893+0000 mon.a (mon.0) 3116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:45:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.821077+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:45:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.822525+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.823662+0000 mon.a (mon.0) 3119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.824693+0000 mon.a (mon.0) 3120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.826817+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:41 gibba032 bash[24004]: audit 2021-03-10T16:45:40.828331+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: cluster 2021-03-10T16:45:40.044718+0000 mgr.gibba032.whopeo (mgr.24625) 9111 : cluster [DBG] pgmap v5193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.817893+0000 mon.a (mon.0) 3116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:45:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.821077+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:45:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.822525+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.823662+0000 mon.a (mon.0) 3119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.824693+0000 mon.a (mon.0) 3120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.826817+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[30791]: audit 2021-03-10T16:45:40.828331+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: cluster 2021-03-10T16:45:40.044718+0000 mgr.gibba032.whopeo (mgr.24625) 9111 : cluster [DBG] pgmap v5193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.817893+0000 mon.a (mon.0) 3116 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:45:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.821077+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:45:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.822525+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.823662+0000 mon.a (mon.0) 3119 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.824693+0000 mon.a (mon.0) 3120 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.826817+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:41 gibba011 bash[31774]: audit 2021-03-10T16:45:40.828331+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:45:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.593516+0000 mgr.gibba032.whopeo (mgr.24625) 9112 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:45:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.595285+0000 mgr.gibba032.whopeo (mgr.24625) 9113 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:45:42.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.818611+0000 mgr.gibba032.whopeo (mgr.24625) 9114 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:45:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.820461+0000 mgr.gibba032.whopeo (mgr.24625) 9115 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:45:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.821552+0000 mgr.gibba032.whopeo (mgr.24625) 9116 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:45:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.821591+0000 mgr.gibba032.whopeo (mgr.24625) 9117 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:45:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.822674+0000 mgr.gibba032.whopeo (mgr.24625) 9118 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:45:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.822714+0000 mgr.gibba032.whopeo (mgr.24625) 9119 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:45:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.823648+0000 mgr.gibba032.whopeo (mgr.24625) 9120 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:45:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.824556+0000 mgr.gibba032.whopeo (mgr.24625) 9121 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:45:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.824593+0000 mgr.gibba032.whopeo (mgr.24625) 9122 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:45:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.825867+0000 mgr.gibba032.whopeo (mgr.24625) 9123 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:45:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.826190+0000 mgr.gibba032.whopeo (mgr.24625) 9124 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:45:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.827499+0000 mgr.gibba032.whopeo (mgr.24625) 9125 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:45:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.827539+0000 mgr.gibba032.whopeo (mgr.24625) 9126 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:45:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[30791]: cephadm 2021-03-10T16:45:40.827646+0000 mgr.gibba032.whopeo (mgr.24625) 9127 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.593516+0000 mgr.gibba032.whopeo (mgr.24625) 9112 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:45:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.595285+0000 mgr.gibba032.whopeo (mgr.24625) 9113 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:45:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.818611+0000 mgr.gibba032.whopeo (mgr.24625) 9114 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:45:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.820461+0000 mgr.gibba032.whopeo (mgr.24625) 9115 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:45:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.821552+0000 mgr.gibba032.whopeo (mgr.24625) 9116 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:45:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.821591+0000 mgr.gibba032.whopeo (mgr.24625) 9117 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:45:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.822674+0000 mgr.gibba032.whopeo (mgr.24625) 9118 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:45:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.822714+0000 mgr.gibba032.whopeo (mgr.24625) 9119 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:45:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.823648+0000 mgr.gibba032.whopeo (mgr.24625) 9120 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:45:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.824556+0000 mgr.gibba032.whopeo (mgr.24625) 9121 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:45:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.824593+0000 mgr.gibba032.whopeo (mgr.24625) 9122 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:45:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.825867+0000 mgr.gibba032.whopeo (mgr.24625) 9123 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:45:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.826190+0000 mgr.gibba032.whopeo (mgr.24625) 9124 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:45:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.827499+0000 mgr.gibba032.whopeo (mgr.24625) 9125 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:45:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.827539+0000 mgr.gibba032.whopeo (mgr.24625) 9126 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:45:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:42 gibba011 bash[31774]: cephadm 2021-03-10T16:45:40.827646+0000 mgr.gibba032.whopeo (mgr.24625) 9127 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.593516+0000 mgr.gibba032.whopeo (mgr.24625) 9112 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:45:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.595285+0000 mgr.gibba032.whopeo (mgr.24625) 9113 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:45:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.818611+0000 mgr.gibba032.whopeo (mgr.24625) 9114 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:45:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.820461+0000 mgr.gibba032.whopeo (mgr.24625) 9115 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:45:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.821552+0000 mgr.gibba032.whopeo (mgr.24625) 9116 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:45:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.821591+0000 mgr.gibba032.whopeo (mgr.24625) 9117 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.822674+0000 mgr.gibba032.whopeo (mgr.24625) 9118 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.822714+0000 mgr.gibba032.whopeo (mgr.24625) 9119 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.823648+0000 mgr.gibba032.whopeo (mgr.24625) 9120 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.824556+0000 mgr.gibba032.whopeo (mgr.24625) 9121 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:45:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.824593+0000 mgr.gibba032.whopeo (mgr.24625) 9122 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.825867+0000 mgr.gibba032.whopeo (mgr.24625) 9123 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.826190+0000 mgr.gibba032.whopeo (mgr.24625) 9124 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.827499+0000 mgr.gibba032.whopeo (mgr.24625) 9125 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.827539+0000 mgr.gibba032.whopeo (mgr.24625) 9126 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:45:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:42 gibba032 bash[24004]: cephadm 2021-03-10T16:45:40.827646+0000 mgr.gibba032.whopeo (mgr.24625) 9127 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:43.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:43 gibba011 bash[30791]: cluster 2021-03-10T16:45:42.045067+0000 mgr.gibba032.whopeo (mgr.24625) 9128 : cluster [DBG] pgmap v5194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:43.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:43 gibba011 bash[31774]: cluster 2021-03-10T16:45:42.045067+0000 mgr.gibba032.whopeo (mgr.24625) 9128 : cluster [DBG] pgmap v5194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:43.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:43 gibba032 bash[24004]: cluster 2021-03-10T16:45:42.045067+0000 mgr.gibba032.whopeo (mgr.24625) 9128 : cluster [DBG] pgmap v5194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:45.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:45 gibba032 bash[24004]: cluster 2021-03-10T16:45:44.046314+0000 mgr.gibba032.whopeo (mgr.24625) 9129 : cluster [DBG] pgmap v5195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:45.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:45 gibba011 bash[30791]: cluster 2021-03-10T16:45:44.046314+0000 mgr.gibba032.whopeo (mgr.24625) 9129 : cluster [DBG] pgmap v5195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:45.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:45 gibba011 bash[31774]: cluster 2021-03-10T16:45:44.046314+0000 mgr.gibba032.whopeo (mgr.24625) 9129 : cluster [DBG] pgmap v5195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:47.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:47 gibba032 bash[24004]: cluster 2021-03-10T16:45:46.047736+0000 mgr.gibba032.whopeo (mgr.24625) 9130 : cluster [DBG] pgmap v5196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:47.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:47 gibba011 bash[30791]: cluster 2021-03-10T16:45:46.047736+0000 mgr.gibba032.whopeo (mgr.24625) 9130 : cluster [DBG] pgmap v5196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:47.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:47 gibba011 bash[31774]: cluster 2021-03-10T16:45:46.047736+0000 mgr.gibba032.whopeo (mgr.24625) 9130 : cluster [DBG] pgmap v5196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:49.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:49 gibba032 bash[24004]: cluster 2021-03-10T16:45:48.048673+0000 mgr.gibba032.whopeo (mgr.24625) 9131 : cluster [DBG] pgmap v5197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:49.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:49 gibba011 bash[30791]: cluster 2021-03-10T16:45:48.048673+0000 mgr.gibba032.whopeo (mgr.24625) 9131 : cluster [DBG] pgmap v5197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:49.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:49 gibba011 bash[31774]: cluster 2021-03-10T16:45:48.048673+0000 mgr.gibba032.whopeo (mgr.24625) 9131 : cluster [DBG] pgmap v5197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:51 gibba011 bash[30791]: cluster 2021-03-10T16:45:50.049365+0000 mgr.gibba032.whopeo (mgr.24625) 9132 : cluster [DBG] pgmap v5198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:51.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:51 gibba011 bash[31774]: cluster 2021-03-10T16:45:50.049365+0000 mgr.gibba032.whopeo (mgr.24625) 9132 : cluster [DBG] pgmap v5198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:51.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:51 gibba032 bash[24004]: cluster 2021-03-10T16:45:50.049365+0000 mgr.gibba032.whopeo (mgr.24625) 9132 : cluster [DBG] pgmap v5198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:53.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:53 gibba032 bash[24004]: cluster 2021-03-10T16:45:52.050556+0000 mgr.gibba032.whopeo (mgr.24625) 9133 : cluster [DBG] pgmap v5199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:53.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:53 gibba011 bash[30791]: cluster 2021-03-10T16:45:52.050556+0000 mgr.gibba032.whopeo (mgr.24625) 9133 : cluster [DBG] pgmap v5199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:53.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:53 gibba011 bash[31774]: cluster 2021-03-10T16:45:52.050556+0000 mgr.gibba032.whopeo (mgr.24625) 9133 : cluster [DBG] pgmap v5199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:55.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:55 gibba032 bash[24004]: cluster 2021-03-10T16:45:54.052060+0000 mgr.gibba032.whopeo (mgr.24625) 9134 : cluster [DBG] pgmap v5200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:55.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:55 gibba011 bash[30791]: cluster 2021-03-10T16:45:54.052060+0000 mgr.gibba032.whopeo (mgr.24625) 9134 : cluster [DBG] pgmap v5200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:55.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:55 gibba011 bash[31774]: cluster 2021-03-10T16:45:54.052060+0000 mgr.gibba032.whopeo (mgr.24625) 9134 : cluster [DBG] pgmap v5200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:57.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:57 gibba032 bash[24004]: cephadm 2021-03-10T16:45:55.840720+0000 mgr.gibba032.whopeo (mgr.24625) 9135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:57 gibba032 bash[24004]: cluster 2021-03-10T16:45:56.053351+0000 mgr.gibba032.whopeo (mgr.24625) 9136 : cluster [DBG] pgmap v5201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:57 gibba011 bash[30791]: cephadm 2021-03-10T16:45:55.840720+0000 mgr.gibba032.whopeo (mgr.24625) 9135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:57 gibba011 bash[30791]: cluster 2021-03-10T16:45:56.053351+0000 mgr.gibba032.whopeo (mgr.24625) 9136 : cluster [DBG] pgmap v5201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:57 gibba011 bash[31774]: cephadm 2021-03-10T16:45:55.840720+0000 mgr.gibba032.whopeo (mgr.24625) 9135 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:45:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:57 gibba011 bash[31774]: cluster 2021-03-10T16:45:56.053351+0000 mgr.gibba032.whopeo (mgr.24625) 9136 : cluster [DBG] pgmap v5201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.154991+0000 mon.a (mon.0) 3123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:45:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.155146+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:45:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.298932+0000 mon.a (mon.0) 3125 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:45:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.299717+0000 mon.a (mon.0) 3126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:45:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.300400+0000 mon.a (mon.0) 3127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:58 gibba032 bash[24004]: audit 2021-03-10T16:45:57.301110+0000 mon.a (mon.0) 3128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.154991+0000 mon.a (mon.0) 3123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:45:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.155146+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:45:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.298932+0000 mon.a (mon.0) 3125 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:45:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.299717+0000 mon.a (mon.0) 3126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:45:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.300400+0000 mon.a (mon.0) 3127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[30791]: audit 2021-03-10T16:45:57.301110+0000 mon.a (mon.0) 3128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.154991+0000 mon.a (mon.0) 3123 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:45:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.155146+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:45:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.298932+0000 mon.a (mon.0) 3125 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T16:45:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.299717+0000 mon.a (mon.0) 3126 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:45:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.300400+0000 mon.a (mon.0) 3127 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:58 gibba011 bash[31774]: audit 2021-03-10T16:45:57.301110+0000 mon.a (mon.0) 3128 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:45:58.867 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:45:59.093 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:45:59.093 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:45:59.094 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:45:59.094 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:45:59.094 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:45:59.094 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:45:59.095 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:45:59.095 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:45:59.095 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:45:59.095 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:45:59.095 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:45:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:45:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:45:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:45:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:45:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:45:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:45:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:45:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:45:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:45:59.098 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:45:59.345 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:45:59.345 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:45:59.345 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:45:59.346 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:59.346 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:45:59.346 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:45:59.346 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:59.346 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:45:59.347 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:45:59.347 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:45:59.347 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:45:59.347 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:45:59.347 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:45:59.348 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:45:59.348 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:45:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:59 gibba032 bash[24004]: cluster 2021-03-10T16:45:58.054184+0000 mgr.gibba032.whopeo (mgr.24625) 9137 : cluster [DBG] pgmap v5202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:59 gibba032 bash[24004]: audit 2021-03-10T16:45:58.858764+0000 mgr.gibba032.whopeo (mgr.24625) 9138 : audit [DBG] from='client.60157 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:59.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:45:59 gibba032 bash[24004]: audit 2021-03-10T16:45:59.082090+0000 mgr.gibba032.whopeo (mgr.24625) 9139 : audit [DBG] from='client.60163 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:59.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[30791]: cluster 2021-03-10T16:45:58.054184+0000 mgr.gibba032.whopeo (mgr.24625) 9137 : cluster [DBG] pgmap v5202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:59.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[30791]: audit 2021-03-10T16:45:58.858764+0000 mgr.gibba032.whopeo (mgr.24625) 9138 : audit [DBG] from='client.60157 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:59.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[30791]: audit 2021-03-10T16:45:59.082090+0000 mgr.gibba032.whopeo (mgr.24625) 9139 : audit [DBG] from='client.60163 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:59.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[31774]: cluster 2021-03-10T16:45:58.054184+0000 mgr.gibba032.whopeo (mgr.24625) 9137 : cluster [DBG] pgmap v5202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:45:59.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[31774]: audit 2021-03-10T16:45:58.858764+0000 mgr.gibba032.whopeo (mgr.24625) 9138 : audit [DBG] from='client.60157 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:45:59.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:45:59 gibba011 bash[31774]: audit 2021-03-10T16:45:59.082090+0000 mgr.gibba032.whopeo (mgr.24625) 9139 : audit [DBG] from='client.60163 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:00.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:00 gibba032 bash[24004]: audit 2021-03-10T16:45:59.346330+0000 mon.a (mon.0) 3129 : audit [DBG] from='client.? 172.21.2.111:0/3785541564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:00.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:00 gibba032 bash[24004]: cluster 2021-03-10T16:46:00.054521+0000 mgr.gibba032.whopeo (mgr.24625) 9140 : cluster [DBG] pgmap v5203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:00.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:00 gibba011 bash[30791]: audit 2021-03-10T16:45:59.346330+0000 mon.a (mon.0) 3129 : audit [DBG] from='client.? 172.21.2.111:0/3785541564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:00 gibba011 bash[30791]: cluster 2021-03-10T16:46:00.054521+0000 mgr.gibba032.whopeo (mgr.24625) 9140 : cluster [DBG] pgmap v5203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:00.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:00 gibba011 bash[31774]: audit 2021-03-10T16:45:59.346330+0000 mon.a (mon.0) 3129 : audit [DBG] from='client.? 172.21.2.111:0/3785541564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:00.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:00 gibba011 bash[31774]: cluster 2021-03-10T16:46:00.054521+0000 mgr.gibba032.whopeo (mgr.24625) 9140 : cluster [DBG] pgmap v5203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:03.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:03 gibba032 bash[24004]: cluster 2021-03-10T16:46:02.055907+0000 mgr.gibba032.whopeo (mgr.24625) 9141 : cluster [DBG] pgmap v5204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:03.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:03 gibba011 bash[30791]: cluster 2021-03-10T16:46:02.055907+0000 mgr.gibba032.whopeo (mgr.24625) 9141 : cluster [DBG] pgmap v5204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:03.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:03 gibba011 bash[31774]: cluster 2021-03-10T16:46:02.055907+0000 mgr.gibba032.whopeo (mgr.24625) 9141 : cluster [DBG] pgmap v5204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:05.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:05 gibba032 bash[24004]: cluster 2021-03-10T16:46:04.057330+0000 mgr.gibba032.whopeo (mgr.24625) 9142 : cluster [DBG] pgmap v5205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:05.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:05 gibba011 bash[30791]: cluster 2021-03-10T16:46:04.057330+0000 mgr.gibba032.whopeo (mgr.24625) 9142 : cluster [DBG] pgmap v5205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:05.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:05 gibba011 bash[31774]: cluster 2021-03-10T16:46:04.057330+0000 mgr.gibba032.whopeo (mgr.24625) 9142 : cluster [DBG] pgmap v5205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:07.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:07 gibba032 bash[24004]: cluster 2021-03-10T16:46:06.058763+0000 mgr.gibba032.whopeo (mgr.24625) 9143 : cluster [DBG] pgmap v5206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:07.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:07 gibba011 bash[30791]: cluster 2021-03-10T16:46:06.058763+0000 mgr.gibba032.whopeo (mgr.24625) 9143 : cluster [DBG] pgmap v5206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:07.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:07 gibba011 bash[31774]: cluster 2021-03-10T16:46:06.058763+0000 mgr.gibba032.whopeo (mgr.24625) 9143 : cluster [DBG] pgmap v5206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:08.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:08 gibba032 bash[24004]: cluster 2021-03-10T16:46:08.059691+0000 mgr.gibba032.whopeo (mgr.24625) 9144 : cluster [DBG] pgmap v5207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:08.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:08 gibba011 bash[30791]: cluster 2021-03-10T16:46:08.059691+0000 mgr.gibba032.whopeo (mgr.24625) 9144 : cluster [DBG] pgmap v5207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:08.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:08 gibba011 bash[31774]: cluster 2021-03-10T16:46:08.059691+0000 mgr.gibba032.whopeo (mgr.24625) 9144 : cluster [DBG] pgmap v5207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:11.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:11 gibba032 bash[24004]: cluster 2021-03-10T16:46:10.060259+0000 mgr.gibba032.whopeo (mgr.24625) 9145 : cluster [DBG] pgmap v5208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:11 gibba011 bash[30791]: cluster 2021-03-10T16:46:10.060259+0000 mgr.gibba032.whopeo (mgr.24625) 9145 : cluster [DBG] pgmap v5208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:11.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:11 gibba011 bash[31774]: cluster 2021-03-10T16:46:10.060259+0000 mgr.gibba032.whopeo (mgr.24625) 9145 : cluster [DBG] pgmap v5208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:12.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:12 gibba011 bash[30791]: cephadm 2021-03-10T16:46:10.854541+0000 mgr.gibba032.whopeo (mgr.24625) 9146 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:12.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:12 gibba011 bash[31774]: cephadm 2021-03-10T16:46:10.854541+0000 mgr.gibba032.whopeo (mgr.24625) 9146 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:12.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:12 gibba032 bash[24004]: cephadm 2021-03-10T16:46:10.854541+0000 mgr.gibba032.whopeo (mgr.24625) 9146 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:13.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:13 gibba011 bash[30791]: cluster 2021-03-10T16:46:12.060654+0000 mgr.gibba032.whopeo (mgr.24625) 9147 : cluster [DBG] pgmap v5209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:13.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:13 gibba011 bash[31774]: cluster 2021-03-10T16:46:12.060654+0000 mgr.gibba032.whopeo (mgr.24625) 9147 : cluster [DBG] pgmap v5209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:13.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:13 gibba032 bash[24004]: cluster 2021-03-10T16:46:12.060654+0000 mgr.gibba032.whopeo (mgr.24625) 9147 : cluster [DBG] pgmap v5209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:15.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:15 gibba032 bash[24004]: cluster 2021-03-10T16:46:14.062073+0000 mgr.gibba032.whopeo (mgr.24625) 9148 : cluster [DBG] pgmap v5210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:15.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:15 gibba011 bash[30791]: cluster 2021-03-10T16:46:14.062073+0000 mgr.gibba032.whopeo (mgr.24625) 9148 : cluster [DBG] pgmap v5210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:15.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:15 gibba011 bash[31774]: cluster 2021-03-10T16:46:14.062073+0000 mgr.gibba032.whopeo (mgr.24625) 9148 : cluster [DBG] pgmap v5210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:17.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:17 gibba032 bash[24004]: cluster 2021-03-10T16:46:16.063495+0000 mgr.gibba032.whopeo (mgr.24625) 9149 : cluster [DBG] pgmap v5211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:17.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:17 gibba011 bash[30791]: cluster 2021-03-10T16:46:16.063495+0000 mgr.gibba032.whopeo (mgr.24625) 9149 : cluster [DBG] pgmap v5211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:17.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:17 gibba011 bash[31774]: cluster 2021-03-10T16:46:16.063495+0000 mgr.gibba032.whopeo (mgr.24625) 9149 : cluster [DBG] pgmap v5211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:19.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:19 gibba032 bash[24004]: cluster 2021-03-10T16:46:18.064427+0000 mgr.gibba032.whopeo (mgr.24625) 9150 : cluster [DBG] pgmap v5212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:19.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:19 gibba011 bash[30791]: cluster 2021-03-10T16:46:18.064427+0000 mgr.gibba032.whopeo (mgr.24625) 9150 : cluster [DBG] pgmap v5212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:19.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:19 gibba011 bash[31774]: cluster 2021-03-10T16:46:18.064427+0000 mgr.gibba032.whopeo (mgr.24625) 9150 : cluster [DBG] pgmap v5212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:21 gibba011 bash[30791]: cluster 2021-03-10T16:46:20.065278+0000 mgr.gibba032.whopeo (mgr.24625) 9151 : cluster [DBG] pgmap v5213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:21.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:21 gibba011 bash[31774]: cluster 2021-03-10T16:46:20.065278+0000 mgr.gibba032.whopeo (mgr.24625) 9151 : cluster [DBG] pgmap v5213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:21.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:21 gibba032 bash[24004]: cluster 2021-03-10T16:46:20.065278+0000 mgr.gibba032.whopeo (mgr.24625) 9151 : cluster [DBG] pgmap v5213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:23.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:23 gibba032 bash[24004]: cluster 2021-03-10T16:46:22.066556+0000 mgr.gibba032.whopeo (mgr.24625) 9152 : cluster [DBG] pgmap v5214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:23.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:23 gibba011 bash[30791]: cluster 2021-03-10T16:46:22.066556+0000 mgr.gibba032.whopeo (mgr.24625) 9152 : cluster [DBG] pgmap v5214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:23.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:23 gibba011 bash[31774]: cluster 2021-03-10T16:46:22.066556+0000 mgr.gibba032.whopeo (mgr.24625) 9152 : cluster [DBG] pgmap v5214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:25.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:25 gibba032 bash[24004]: cluster 2021-03-10T16:46:24.068127+0000 mgr.gibba032.whopeo (mgr.24625) 9153 : cluster [DBG] pgmap v5215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:25.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:25 gibba011 bash[30791]: cluster 2021-03-10T16:46:24.068127+0000 mgr.gibba032.whopeo (mgr.24625) 9153 : cluster [DBG] pgmap v5215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:25.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:25 gibba011 bash[31774]: cluster 2021-03-10T16:46:24.068127+0000 mgr.gibba032.whopeo (mgr.24625) 9153 : cluster [DBG] pgmap v5215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:27 gibba032 bash[24004]: cephadm 2021-03-10T16:46:25.866696+0000 mgr.gibba032.whopeo (mgr.24625) 9154 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:27 gibba032 bash[24004]: cluster 2021-03-10T16:46:26.069421+0000 mgr.gibba032.whopeo (mgr.24625) 9155 : cluster [DBG] pgmap v5216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:27 gibba011 bash[30791]: cephadm 2021-03-10T16:46:25.866696+0000 mgr.gibba032.whopeo (mgr.24625) 9154 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:27.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:27 gibba011 bash[30791]: cluster 2021-03-10T16:46:26.069421+0000 mgr.gibba032.whopeo (mgr.24625) 9155 : cluster [DBG] pgmap v5216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:27.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:27 gibba011 bash[31774]: cephadm 2021-03-10T16:46:25.866696+0000 mgr.gibba032.whopeo (mgr.24625) 9154 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:27.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:27 gibba011 bash[31774]: cluster 2021-03-10T16:46:26.069421+0000 mgr.gibba032.whopeo (mgr.24625) 9155 : cluster [DBG] pgmap v5216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:28.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:28 gibba032 bash[24004]: cluster 2021-03-10T16:46:28.070399+0000 mgr.gibba032.whopeo (mgr.24625) 9156 : cluster [DBG] pgmap v5217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:28 gibba011 bash[30791]: cluster 2021-03-10T16:46:28.070399+0000 mgr.gibba032.whopeo (mgr.24625) 9156 : cluster [DBG] pgmap v5217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:28 gibba011 bash[31774]: cluster 2021-03-10T16:46:28.070399+0000 mgr.gibba032.whopeo (mgr.24625) 9156 : cluster [DBG] pgmap v5217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:29.585 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:46:29.812 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:46:29.812 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 5m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:46:29.813 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 4m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:46:29.813 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:46:29.814 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:46:29.814 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:46:29.815 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:46:29.815 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:46:29.815 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:46:29.816 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:46:29.816 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 4m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:46:29.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:46:29.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:46:29.817 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:46:29.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:46:29.818 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:46:29.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:46:29.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:46:29.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:46:29.821 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 4m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:46:29.821 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:46:30.089 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:46:30.089 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:46:30.090 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:46:30.090 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:46:30.091 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:46:30.091 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:46:30.091 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:46:30.092 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:46:30.092 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:46:30.093 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:46:30.093 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:46:30.093 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:46:30.094 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:46:30.094 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:46:30.095 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:46:31.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[30791]: audit 2021-03-10T16:46:29.577649+0000 mgr.gibba032.whopeo (mgr.24625) 9157 : audit [DBG] from='client.50193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[30791]: audit 2021-03-10T16:46:29.801048+0000 mgr.gibba032.whopeo (mgr.24625) 9158 : audit [DBG] from='client.60181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[30791]: cluster 2021-03-10T16:46:30.071079+0000 mgr.gibba032.whopeo (mgr.24625) 9159 : cluster [DBG] pgmap v5218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:31.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[30791]: audit 2021-03-10T16:46:30.090824+0000 mon.c (mon.1) 179 : audit [DBG] from='client.? 172.21.2.111:0/3348065551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:31.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[31774]: audit 2021-03-10T16:46:29.577649+0000 mgr.gibba032.whopeo (mgr.24625) 9157 : audit [DBG] from='client.50193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[31774]: audit 2021-03-10T16:46:29.801048+0000 mgr.gibba032.whopeo (mgr.24625) 9158 : audit [DBG] from='client.60181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[31774]: cluster 2021-03-10T16:46:30.071079+0000 mgr.gibba032.whopeo (mgr.24625) 9159 : cluster [DBG] pgmap v5218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:31.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:31 gibba011 bash[31774]: audit 2021-03-10T16:46:30.090824+0000 mon.c (mon.1) 179 : audit [DBG] from='client.? 172.21.2.111:0/3348065551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:31 gibba032 bash[24004]: audit 2021-03-10T16:46:29.577649+0000 mgr.gibba032.whopeo (mgr.24625) 9157 : audit [DBG] from='client.50193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:31 gibba032 bash[24004]: audit 2021-03-10T16:46:29.801048+0000 mgr.gibba032.whopeo (mgr.24625) 9158 : audit [DBG] from='client.60181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:46:31.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:31 gibba032 bash[24004]: cluster 2021-03-10T16:46:30.071079+0000 mgr.gibba032.whopeo (mgr.24625) 9159 : cluster [DBG] pgmap v5218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:31.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:31 gibba032 bash[24004]: audit 2021-03-10T16:46:30.090824+0000 mon.c (mon.1) 179 : audit [DBG] from='client.? 172.21.2.111:0/3348065551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:33.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:33 gibba032 bash[24004]: cluster 2021-03-10T16:46:32.072381+0000 mgr.gibba032.whopeo (mgr.24625) 9160 : cluster [DBG] pgmap v5219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:33 gibba011 bash[30791]: cluster 2021-03-10T16:46:32.072381+0000 mgr.gibba032.whopeo (mgr.24625) 9160 : cluster [DBG] pgmap v5219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:33.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:33 gibba011 bash[31774]: cluster 2021-03-10T16:46:32.072381+0000 mgr.gibba032.whopeo (mgr.24625) 9160 : cluster [DBG] pgmap v5219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:35.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:35 gibba032 bash[24004]: cluster 2021-03-10T16:46:34.074047+0000 mgr.gibba032.whopeo (mgr.24625) 9161 : cluster [DBG] pgmap v5220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:35.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:35 gibba011 bash[30791]: cluster 2021-03-10T16:46:34.074047+0000 mgr.gibba032.whopeo (mgr.24625) 9161 : cluster [DBG] pgmap v5220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:35.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:35 gibba011 bash[31774]: cluster 2021-03-10T16:46:34.074047+0000 mgr.gibba032.whopeo (mgr.24625) 9161 : cluster [DBG] pgmap v5220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:37 gibba032 bash[24004]: cluster 2021-03-10T16:46:36.075449+0000 mgr.gibba032.whopeo (mgr.24625) 9162 : cluster [DBG] pgmap v5221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:37.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:37 gibba011 bash[31774]: cluster 2021-03-10T16:46:36.075449+0000 mgr.gibba032.whopeo (mgr.24625) 9162 : cluster [DBG] pgmap v5221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:37.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:37 gibba011 bash[30791]: cluster 2021-03-10T16:46:36.075449+0000 mgr.gibba032.whopeo (mgr.24625) 9162 : cluster [DBG] pgmap v5221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:38.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:38 gibba032 bash[24004]: cluster 2021-03-10T16:46:38.076537+0000 mgr.gibba032.whopeo (mgr.24625) 9163 : cluster [DBG] pgmap v5222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:38 gibba011 bash[30791]: cluster 2021-03-10T16:46:38.076537+0000 mgr.gibba032.whopeo (mgr.24625) 9163 : cluster [DBG] pgmap v5222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:38.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:38 gibba011 bash[31774]: cluster 2021-03-10T16:46:38.076537+0000 mgr.gibba032.whopeo (mgr.24625) 9163 : cluster [DBG] pgmap v5222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:41 gibba011 bash[30791]: cluster 2021-03-10T16:46:40.077622+0000 mgr.gibba032.whopeo (mgr.24625) 9164 : cluster [DBG] pgmap v5223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:41.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:41 gibba011 bash[31774]: cluster 2021-03-10T16:46:40.077622+0000 mgr.gibba032.whopeo (mgr.24625) 9164 : cluster [DBG] pgmap v5223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:41.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:41 gibba032 bash[24004]: cluster 2021-03-10T16:46:40.077622+0000 mgr.gibba032.whopeo (mgr.24625) 9164 : cluster [DBG] pgmap v5223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:40.881598+0000 mgr.gibba032.whopeo (mgr.24625) 9165 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:46:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:40.883150+0000 mgr.gibba032.whopeo (mgr.24625) 9166 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:46:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.121257+0000 mon.a (mon.0) 3130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:46:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.122471+0000 mgr.gibba032.whopeo (mgr.24625) 9167 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:46:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.124565+0000 mgr.gibba032.whopeo (mgr.24625) 9168 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:46:42.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.125025+0000 mon.a (mon.0) 3131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:46:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.125971+0000 mgr.gibba032.whopeo (mgr.24625) 9169 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:46:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.126020+0000 mgr.gibba032.whopeo (mgr.24625) 9170 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:46:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.126663+0000 mon.a (mon.0) 3132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.127477+0000 mgr.gibba032.whopeo (mgr.24625) 9171 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:46:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.127527+0000 mgr.gibba032.whopeo (mgr.24625) 9172 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:46:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.128150+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.128880+0000 mgr.gibba032.whopeo (mgr.24625) 9173 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:46:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.129552+0000 mon.a (mon.0) 3134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.130065+0000 mgr.gibba032.whopeo (mgr.24625) 9174 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:46:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.130116+0000 mgr.gibba032.whopeo (mgr.24625) 9175 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.131684+0000 mgr.gibba032.whopeo (mgr.24625) 9176 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.132244+0000 mgr.gibba032.whopeo (mgr.24625) 9177 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.132448+0000 mon.a (mon.0) 3135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.133876+0000 mgr.gibba032.whopeo (mgr.24625) 9178 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.133931+0000 mgr.gibba032.whopeo (mgr.24625) 9179 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:46:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: cephadm 2021-03-10T16:46:41.134085+0000 mgr.gibba032.whopeo (mgr.24625) 9180 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:42 gibba032 bash[24004]: audit 2021-03-10T16:46:41.134382+0000 mon.a (mon.0) 3136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:40.881598+0000 mgr.gibba032.whopeo (mgr.24625) 9165 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:46:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:40.883150+0000 mgr.gibba032.whopeo (mgr.24625) 9166 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:46:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.121257+0000 mon.a (mon.0) 3130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:46:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.122471+0000 mgr.gibba032.whopeo (mgr.24625) 9167 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:46:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.124565+0000 mgr.gibba032.whopeo (mgr.24625) 9168 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:46:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.125025+0000 mon.a (mon.0) 3131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:46:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.125971+0000 mgr.gibba032.whopeo (mgr.24625) 9169 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:46:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.126020+0000 mgr.gibba032.whopeo (mgr.24625) 9170 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:46:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.126663+0000 mon.a (mon.0) 3132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.127477+0000 mgr.gibba032.whopeo (mgr.24625) 9171 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:46:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.127527+0000 mgr.gibba032.whopeo (mgr.24625) 9172 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:46:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.128150+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.128880+0000 mgr.gibba032.whopeo (mgr.24625) 9173 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:46:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.129552+0000 mon.a (mon.0) 3134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.130065+0000 mgr.gibba032.whopeo (mgr.24625) 9174 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:46:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.130116+0000 mgr.gibba032.whopeo (mgr.24625) 9175 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:46:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.131684+0000 mgr.gibba032.whopeo (mgr.24625) 9176 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.132244+0000 mgr.gibba032.whopeo (mgr.24625) 9177 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.132448+0000 mon.a (mon.0) 3135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.133876+0000 mgr.gibba032.whopeo (mgr.24625) 9178 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.133931+0000 mgr.gibba032.whopeo (mgr.24625) 9179 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: cephadm 2021-03-10T16:46:41.134085+0000 mgr.gibba032.whopeo (mgr.24625) 9180 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[30791]: audit 2021-03-10T16:46:41.134382+0000 mon.a (mon.0) 3136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:40.881598+0000 mgr.gibba032.whopeo (mgr.24625) 9165 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:46:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:40.883150+0000 mgr.gibba032.whopeo (mgr.24625) 9166 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:46:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.121257+0000 mon.a (mon.0) 3130 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:46:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.122471+0000 mgr.gibba032.whopeo (mgr.24625) 9167 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:46:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.124565+0000 mgr.gibba032.whopeo (mgr.24625) 9168 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:46:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.125025+0000 mon.a (mon.0) 3131 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:46:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.125971+0000 mgr.gibba032.whopeo (mgr.24625) 9169 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:46:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.126020+0000 mgr.gibba032.whopeo (mgr.24625) 9170 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:46:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.126663+0000 mon.a (mon.0) 3132 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.127477+0000 mgr.gibba032.whopeo (mgr.24625) 9171 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:46:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.127527+0000 mgr.gibba032.whopeo (mgr.24625) 9172 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:46:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.128150+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.128880+0000 mgr.gibba032.whopeo (mgr.24625) 9173 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:46:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.129552+0000 mon.a (mon.0) 3134 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.130065+0000 mgr.gibba032.whopeo (mgr.24625) 9174 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:46:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.130116+0000 mgr.gibba032.whopeo (mgr.24625) 9175 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:46:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.131684+0000 mgr.gibba032.whopeo (mgr.24625) 9176 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:46:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.132244+0000 mgr.gibba032.whopeo (mgr.24625) 9177 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:46:42.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.132448+0000 mon.a (mon.0) 3135 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:42.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.133876+0000 mgr.gibba032.whopeo (mgr.24625) 9178 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:46:42.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.133931+0000 mgr.gibba032.whopeo (mgr.24625) 9179 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:46:42.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: cephadm 2021-03-10T16:46:41.134085+0000 mgr.gibba032.whopeo (mgr.24625) 9180 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:42.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:42 gibba011 bash[31774]: audit 2021-03-10T16:46:41.134382+0000 mon.a (mon.0) 3136 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:46:43.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:43 gibba032 bash[24004]: cluster 2021-03-10T16:46:42.079047+0000 mgr.gibba032.whopeo (mgr.24625) 9181 : cluster [DBG] pgmap v5224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:43.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:43 gibba011 bash[30791]: cluster 2021-03-10T16:46:42.079047+0000 mgr.gibba032.whopeo (mgr.24625) 9181 : cluster [DBG] pgmap v5224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:43.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:43 gibba011 bash[31774]: cluster 2021-03-10T16:46:42.079047+0000 mgr.gibba032.whopeo (mgr.24625) 9181 : cluster [DBG] pgmap v5224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:45.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:45 gibba032 bash[24004]: cluster 2021-03-10T16:46:44.080530+0000 mgr.gibba032.whopeo (mgr.24625) 9182 : cluster [DBG] pgmap v5225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:45.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:45 gibba011 bash[30791]: cluster 2021-03-10T16:46:44.080530+0000 mgr.gibba032.whopeo (mgr.24625) 9182 : cluster [DBG] pgmap v5225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:45.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:45 gibba011 bash[31774]: cluster 2021-03-10T16:46:44.080530+0000 mgr.gibba032.whopeo (mgr.24625) 9182 : cluster [DBG] pgmap v5225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:46.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:46 gibba032 bash[24004]: cluster 2021-03-10T16:46:46.081914+0000 mgr.gibba032.whopeo (mgr.24625) 9183 : cluster [DBG] pgmap v5226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:46 gibba011 bash[30791]: cluster 2021-03-10T16:46:46.081914+0000 mgr.gibba032.whopeo (mgr.24625) 9183 : cluster [DBG] pgmap v5226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:46.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:46 gibba011 bash[31774]: cluster 2021-03-10T16:46:46.081914+0000 mgr.gibba032.whopeo (mgr.24625) 9183 : cluster [DBG] pgmap v5226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:49.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:49 gibba032 bash[24004]: cluster 2021-03-10T16:46:48.082993+0000 mgr.gibba032.whopeo (mgr.24625) 9184 : cluster [DBG] pgmap v5227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:49.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:49 gibba011 bash[30791]: cluster 2021-03-10T16:46:48.082993+0000 mgr.gibba032.whopeo (mgr.24625) 9184 : cluster [DBG] pgmap v5227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:49.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:49 gibba011 bash[31774]: cluster 2021-03-10T16:46:48.082993+0000 mgr.gibba032.whopeo (mgr.24625) 9184 : cluster [DBG] pgmap v5227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:51.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:51 gibba011 bash[30791]: cluster 2021-03-10T16:46:50.084400+0000 mgr.gibba032.whopeo (mgr.24625) 9185 : cluster [DBG] pgmap v5228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:51.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:51 gibba011 bash[31774]: cluster 2021-03-10T16:46:50.084400+0000 mgr.gibba032.whopeo (mgr.24625) 9185 : cluster [DBG] pgmap v5228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:51.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:51 gibba032 bash[24004]: cluster 2021-03-10T16:46:50.084400+0000 mgr.gibba032.whopeo (mgr.24625) 9185 : cluster [DBG] pgmap v5228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:53.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:53 gibba032 bash[24004]: cluster 2021-03-10T16:46:52.085829+0000 mgr.gibba032.whopeo (mgr.24625) 9186 : cluster [DBG] pgmap v5229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:53.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:53 gibba011 bash[30791]: cluster 2021-03-10T16:46:52.085829+0000 mgr.gibba032.whopeo (mgr.24625) 9186 : cluster [DBG] pgmap v5229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:53.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:53 gibba011 bash[31774]: cluster 2021-03-10T16:46:52.085829+0000 mgr.gibba032.whopeo (mgr.24625) 9186 : cluster [DBG] pgmap v5229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:55.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:55 gibba032 bash[24004]: cluster 2021-03-10T16:46:54.087299+0000 mgr.gibba032.whopeo (mgr.24625) 9187 : cluster [DBG] pgmap v5230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:55.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:55 gibba011 bash[30791]: cluster 2021-03-10T16:46:54.087299+0000 mgr.gibba032.whopeo (mgr.24625) 9187 : cluster [DBG] pgmap v5230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:55.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:55 gibba011 bash[31774]: cluster 2021-03-10T16:46:54.087299+0000 mgr.gibba032.whopeo (mgr.24625) 9187 : cluster [DBG] pgmap v5230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:56.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:56 gibba032 bash[24004]: cluster 2021-03-10T16:46:56.088645+0000 mgr.gibba032.whopeo (mgr.24625) 9188 : cluster [DBG] pgmap v5231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:56 gibba032 bash[24004]: cephadm 2021-03-10T16:46:56.148699+0000 mgr.gibba032.whopeo (mgr.24625) 9189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:56.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:56 gibba011 bash[30791]: cluster 2021-03-10T16:46:56.088645+0000 mgr.gibba032.whopeo (mgr.24625) 9188 : cluster [DBG] pgmap v5231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:56 gibba011 bash[30791]: cephadm 2021-03-10T16:46:56.148699+0000 mgr.gibba032.whopeo (mgr.24625) 9189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:56 gibba011 bash[31774]: cluster 2021-03-10T16:46:56.088645+0000 mgr.gibba032.whopeo (mgr.24625) 9188 : cluster [DBG] pgmap v5231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:56.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:56 gibba011 bash[31774]: cephadm 2021-03-10T16:46:56.148699+0000 mgr.gibba032.whopeo (mgr.24625) 9189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:46:57.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:57 gibba032 bash[24004]: audit 2021-03-10T16:46:57.171850+0000 mon.a (mon.0) 3137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:46:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:57 gibba032 bash[24004]: audit 2021-03-10T16:46:57.172569+0000 mon.a (mon.0) 3138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:46:57.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:57 gibba011 bash[30791]: audit 2021-03-10T16:46:57.171850+0000 mon.a (mon.0) 3137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:46:57.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:57 gibba011 bash[30791]: audit 2021-03-10T16:46:57.172569+0000 mon.a (mon.0) 3138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:46:57.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:57 gibba011 bash[31774]: audit 2021-03-10T16:46:57.171850+0000 mon.a (mon.0) 3137 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:46:57.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:57 gibba011 bash[31774]: audit 2021-03-10T16:46:57.172569+0000 mon.a (mon.0) 3138 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:46:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:58 gibba032 bash[24004]: audit 2021-03-10T16:46:57.306572+0000 mon.a (mon.0) 3139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:46:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:58 gibba032 bash[24004]: audit 2021-03-10T16:46:57.306856+0000 mon.a (mon.0) 3140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:58 gibba032 bash[24004]: audit 2021-03-10T16:46:57.307114+0000 mon.a (mon.0) 3141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:58 gibba032 bash[24004]: audit 2021-03-10T16:46:57.307382+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T16:46:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:46:58 gibba032 bash[24004]: cluster 2021-03-10T16:46:58.089692+0000 mgr.gibba032.whopeo (mgr.24625) 9190 : cluster [DBG] pgmap v5232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[30791]: audit 2021-03-10T16:46:57.306572+0000 mon.a (mon.0) 3139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:46:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[30791]: audit 2021-03-10T16:46:57.306856+0000 mon.a (mon.0) 3140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[30791]: audit 2021-03-10T16:46:57.307114+0000 mon.a (mon.0) 3141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[30791]: audit 2021-03-10T16:46:57.307382+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T16:46:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[30791]: cluster 2021-03-10T16:46:58.089692+0000 mgr.gibba032.whopeo (mgr.24625) 9190 : cluster [DBG] pgmap v5232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:46:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[31774]: audit 2021-03-10T16:46:57.306572+0000 mon.a (mon.0) 3139 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:46:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[31774]: audit 2021-03-10T16:46:57.306856+0000 mon.a (mon.0) 3140 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[31774]: audit 2021-03-10T16:46:57.307114+0000 mon.a (mon.0) 3141 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T16:46:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[31774]: audit 2021-03-10T16:46:57.307382+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T16:46:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:46:58 gibba011 bash[31774]: cluster 2021-03-10T16:46:58.089692+0000 mgr.gibba032.whopeo (mgr.24625) 9190 : cluster [DBG] pgmap v5232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:00.324 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:47:00.549 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:47:00.549 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:47:00.549 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:47:00.549 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:47:00.550 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:47:00.550 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:47:00.550 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:47:00.550 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:47:00.550 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:47:00.551 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:47:00.551 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:47:00.551 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:47:00.551 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:47:00.551 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:47:00.552 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:47:00.552 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:47:00.552 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:47:00.552 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:47:00.553 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 2h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:47:00.553 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:47:00.553 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:47:00.795 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:47:00.795 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:47:00.796 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:47:00.796 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:00.797 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:47:00.797 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:47:00.797 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:00.797 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:47:00.798 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:47:00.798 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:00.799 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:47:00.799 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:47:00.799 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:47:00.800 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:47:00.800 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:47:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:01 gibba011 bash[30791]: cluster 2021-03-10T16:47:00.090361+0000 mgr.gibba032.whopeo (mgr.24625) 9191 : cluster [DBG] pgmap v5233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:01.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:01 gibba011 bash[30791]: audit 2021-03-10T16:47:00.798096+0000 mon.a (mon.0) 3143 : audit [DBG] from='client.? 172.21.2.111:0/2046629021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:01 gibba011 bash[31774]: cluster 2021-03-10T16:47:00.090361+0000 mgr.gibba032.whopeo (mgr.24625) 9191 : cluster [DBG] pgmap v5233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:01.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:01 gibba011 bash[31774]: audit 2021-03-10T16:47:00.798096+0000 mon.a (mon.0) 3143 : audit [DBG] from='client.? 172.21.2.111:0/2046629021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:01 gibba032 bash[24004]: cluster 2021-03-10T16:47:00.090361+0000 mgr.gibba032.whopeo (mgr.24625) 9191 : cluster [DBG] pgmap v5233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:01.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:01 gibba032 bash[24004]: audit 2021-03-10T16:47:00.798096+0000 mon.a (mon.0) 3143 : audit [DBG] from='client.? 172.21.2.111:0/2046629021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:02.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:02 gibba011 bash[30791]: audit 2021-03-10T16:47:00.317109+0000 mgr.gibba032.whopeo (mgr.24625) 9192 : audit [DBG] from='client.60193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:02.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:02 gibba011 bash[30791]: audit 2021-03-10T16:47:00.539437+0000 mgr.gibba032.whopeo (mgr.24625) 9193 : audit [DBG] from='client.60199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:02.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:02 gibba011 bash[31774]: audit 2021-03-10T16:47:00.317109+0000 mgr.gibba032.whopeo (mgr.24625) 9192 : audit [DBG] from='client.60193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:02.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:02 gibba011 bash[31774]: audit 2021-03-10T16:47:00.539437+0000 mgr.gibba032.whopeo (mgr.24625) 9193 : audit [DBG] from='client.60199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:02 gibba032 bash[24004]: audit 2021-03-10T16:47:00.317109+0000 mgr.gibba032.whopeo (mgr.24625) 9192 : audit [DBG] from='client.60193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:02 gibba032 bash[24004]: audit 2021-03-10T16:47:00.539437+0000 mgr.gibba032.whopeo (mgr.24625) 9193 : audit [DBG] from='client.60199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:03.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:03 gibba011 bash[30791]: cluster 2021-03-10T16:47:02.091011+0000 mgr.gibba032.whopeo (mgr.24625) 9194 : cluster [DBG] pgmap v5234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:03.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:03 gibba011 bash[31774]: cluster 2021-03-10T16:47:02.091011+0000 mgr.gibba032.whopeo (mgr.24625) 9194 : cluster [DBG] pgmap v5234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:03.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:03 gibba032 bash[24004]: cluster 2021-03-10T16:47:02.091011+0000 mgr.gibba032.whopeo (mgr.24625) 9194 : cluster [DBG] pgmap v5234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:05.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:05 gibba032 bash[24004]: cluster 2021-03-10T16:47:04.092261+0000 mgr.gibba032.whopeo (mgr.24625) 9195 : cluster [DBG] pgmap v5235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:05.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:05 gibba011 bash[30791]: cluster 2021-03-10T16:47:04.092261+0000 mgr.gibba032.whopeo (mgr.24625) 9195 : cluster [DBG] pgmap v5235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:05.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:05 gibba011 bash[31774]: cluster 2021-03-10T16:47:04.092261+0000 mgr.gibba032.whopeo (mgr.24625) 9195 : cluster [DBG] pgmap v5235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:06.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:06 gibba032 bash[24004]: cluster 2021-03-10T16:47:06.093609+0000 mgr.gibba032.whopeo (mgr.24625) 9196 : cluster [DBG] pgmap v5236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:06 gibba011 bash[30791]: cluster 2021-03-10T16:47:06.093609+0000 mgr.gibba032.whopeo (mgr.24625) 9196 : cluster [DBG] pgmap v5236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:06 gibba011 bash[31774]: cluster 2021-03-10T16:47:06.093609+0000 mgr.gibba032.whopeo (mgr.24625) 9196 : cluster [DBG] pgmap v5236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:09.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:09 gibba032 bash[24004]: cluster 2021-03-10T16:47:08.094726+0000 mgr.gibba032.whopeo (mgr.24625) 9197 : cluster [DBG] pgmap v5237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:09.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:09 gibba011 bash[30791]: cluster 2021-03-10T16:47:08.094726+0000 mgr.gibba032.whopeo (mgr.24625) 9197 : cluster [DBG] pgmap v5237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:09.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:09 gibba011 bash[31774]: cluster 2021-03-10T16:47:08.094726+0000 mgr.gibba032.whopeo (mgr.24625) 9197 : cluster [DBG] pgmap v5237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:11.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:11 gibba011 bash[30791]: cluster 2021-03-10T16:47:10.095510+0000 mgr.gibba032.whopeo (mgr.24625) 9198 : cluster [DBG] pgmap v5238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:11.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:11 gibba011 bash[31774]: cluster 2021-03-10T16:47:10.095510+0000 mgr.gibba032.whopeo (mgr.24625) 9198 : cluster [DBG] pgmap v5238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:11.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:11 gibba032 bash[24004]: cluster 2021-03-10T16:47:10.095510+0000 mgr.gibba032.whopeo (mgr.24625) 9198 : cluster [DBG] pgmap v5238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:13.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:13 gibba032 bash[24004]: cephadm 2021-03-10T16:47:11.161004+0000 mgr.gibba032.whopeo (mgr.24625) 9199 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:13.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:13 gibba032 bash[24004]: cluster 2021-03-10T16:47:12.096979+0000 mgr.gibba032.whopeo (mgr.24625) 9200 : cluster [DBG] pgmap v5239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:13.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:13 gibba011 bash[30791]: cephadm 2021-03-10T16:47:11.161004+0000 mgr.gibba032.whopeo (mgr.24625) 9199 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:13.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:13 gibba011 bash[30791]: cluster 2021-03-10T16:47:12.096979+0000 mgr.gibba032.whopeo (mgr.24625) 9200 : cluster [DBG] pgmap v5239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:13.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:13 gibba011 bash[31774]: cephadm 2021-03-10T16:47:11.161004+0000 mgr.gibba032.whopeo (mgr.24625) 9199 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:13.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:13 gibba011 bash[31774]: cluster 2021-03-10T16:47:12.096979+0000 mgr.gibba032.whopeo (mgr.24625) 9200 : cluster [DBG] pgmap v5239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:14.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:14 gibba032 bash[24004]: cluster 2021-03-10T16:47:14.098388+0000 mgr.gibba032.whopeo (mgr.24625) 9201 : cluster [DBG] pgmap v5240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:14.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:14 gibba011 bash[30791]: cluster 2021-03-10T16:47:14.098388+0000 mgr.gibba032.whopeo (mgr.24625) 9201 : cluster [DBG] pgmap v5240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:14.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:14 gibba011 bash[31774]: cluster 2021-03-10T16:47:14.098388+0000 mgr.gibba032.whopeo (mgr.24625) 9201 : cluster [DBG] pgmap v5240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:17.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:17 gibba032 bash[24004]: cluster 2021-03-10T16:47:16.099770+0000 mgr.gibba032.whopeo (mgr.24625) 9202 : cluster [DBG] pgmap v5241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:17.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:17 gibba011 bash[30791]: cluster 2021-03-10T16:47:16.099770+0000 mgr.gibba032.whopeo (mgr.24625) 9202 : cluster [DBG] pgmap v5241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:17.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:17 gibba011 bash[31774]: cluster 2021-03-10T16:47:16.099770+0000 mgr.gibba032.whopeo (mgr.24625) 9202 : cluster [DBG] pgmap v5241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:18.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:18 gibba032 bash[24004]: cluster 2021-03-10T16:47:18.100693+0000 mgr.gibba032.whopeo (mgr.24625) 9203 : cluster [DBG] pgmap v5242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:18.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:18 gibba011 bash[30791]: cluster 2021-03-10T16:47:18.100693+0000 mgr.gibba032.whopeo (mgr.24625) 9203 : cluster [DBG] pgmap v5242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:18.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:18 gibba011 bash[31774]: cluster 2021-03-10T16:47:18.100693+0000 mgr.gibba032.whopeo (mgr.24625) 9203 : cluster [DBG] pgmap v5242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:21.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:21 gibba011 bash[30791]: cluster 2021-03-10T16:47:20.101253+0000 mgr.gibba032.whopeo (mgr.24625) 9204 : cluster [DBG] pgmap v5243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:21.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:21 gibba011 bash[31774]: cluster 2021-03-10T16:47:20.101253+0000 mgr.gibba032.whopeo (mgr.24625) 9204 : cluster [DBG] pgmap v5243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:21.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:21 gibba032 bash[24004]: cluster 2021-03-10T16:47:20.101253+0000 mgr.gibba032.whopeo (mgr.24625) 9204 : cluster [DBG] pgmap v5243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:23.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:23 gibba032 bash[24004]: cluster 2021-03-10T16:47:22.102562+0000 mgr.gibba032.whopeo (mgr.24625) 9205 : cluster [DBG] pgmap v5244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:23.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:23 gibba011 bash[30791]: cluster 2021-03-10T16:47:22.102562+0000 mgr.gibba032.whopeo (mgr.24625) 9205 : cluster [DBG] pgmap v5244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:23.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:23 gibba011 bash[31774]: cluster 2021-03-10T16:47:22.102562+0000 mgr.gibba032.whopeo (mgr.24625) 9205 : cluster [DBG] pgmap v5244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:24.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:24 gibba032 bash[24004]: cluster 2021-03-10T16:47:24.104006+0000 mgr.gibba032.whopeo (mgr.24625) 9206 : cluster [DBG] pgmap v5245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:24 gibba011 bash[30791]: cluster 2021-03-10T16:47:24.104006+0000 mgr.gibba032.whopeo (mgr.24625) 9206 : cluster [DBG] pgmap v5245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:24.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:24 gibba011 bash[31774]: cluster 2021-03-10T16:47:24.104006+0000 mgr.gibba032.whopeo (mgr.24625) 9206 : cluster [DBG] pgmap v5245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:27.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:27 gibba032 bash[24004]: cluster 2021-03-10T16:47:26.105417+0000 mgr.gibba032.whopeo (mgr.24625) 9207 : cluster [DBG] pgmap v5246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:27 gibba011 bash[30791]: cluster 2021-03-10T16:47:26.105417+0000 mgr.gibba032.whopeo (mgr.24625) 9207 : cluster [DBG] pgmap v5246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:27.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:27 gibba011 bash[31774]: cluster 2021-03-10T16:47:26.105417+0000 mgr.gibba032.whopeo (mgr.24625) 9207 : cluster [DBG] pgmap v5246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:28.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:28 gibba032 bash[24004]: cephadm 2021-03-10T16:47:26.176180+0000 mgr.gibba032.whopeo (mgr.24625) 9208 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:28.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:28 gibba032 bash[24004]: cluster 2021-03-10T16:47:28.106495+0000 mgr.gibba032.whopeo (mgr.24625) 9209 : cluster [DBG] pgmap v5247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:28 gibba011 bash[30791]: cephadm 2021-03-10T16:47:26.176180+0000 mgr.gibba032.whopeo (mgr.24625) 9208 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:28 gibba011 bash[30791]: cluster 2021-03-10T16:47:28.106495+0000 mgr.gibba032.whopeo (mgr.24625) 9209 : cluster [DBG] pgmap v5247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:28 gibba011 bash[31774]: cephadm 2021-03-10T16:47:26.176180+0000 mgr.gibba032.whopeo (mgr.24625) 9208 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:28 gibba011 bash[31774]: cluster 2021-03-10T16:47:28.106495+0000 mgr.gibba032.whopeo (mgr.24625) 9209 : cluster [DBG] pgmap v5247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:31.079 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:47:31.308 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:47:31.308 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 6m ago 2h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:47:31.308 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 5m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:47:31.308 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:47:31.309 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:47:31.309 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:47:31.309 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:47:31.309 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:47:31.310 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:47:31.310 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (2h) 6m ago 2h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:47:31.310 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (2h) 5m ago 2h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:47:31.310 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:47:31.311 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:47:31.311 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:47:31.311 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:47:31.311 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:47:31.312 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:47:31.312 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:47:31.312 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:47:31.312 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 5m ago 2h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:47:31.313 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:47:31.325 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:31 gibba011 bash[30791]: cluster 2021-03-10T16:47:30.107191+0000 mgr.gibba032.whopeo (mgr.24625) 9210 : cluster [DBG] pgmap v5248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:31.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:31 gibba011 bash[31774]: cluster 2021-03-10T16:47:30.107191+0000 mgr.gibba032.whopeo (mgr.24625) 9210 : cluster [DBG] pgmap v5248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:31 gibba032 bash[24004]: cluster 2021-03-10T16:47:30.107191+0000 mgr.gibba032.whopeo (mgr.24625) 9210 : cluster [DBG] pgmap v5248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:31.558 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:47:31.558 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:47:31.559 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:47:31.559 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:31.559 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:47:31.560 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:47:31.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:31.560 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:47:31.560 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:47:31.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:47:31.561 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:47:31.561 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:47:31.561 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:47:31.561 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:47:31.561 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:47:32.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:32 gibba011 bash[30791]: audit 2021-03-10T16:47:31.071402+0000 mgr.gibba032.whopeo (mgr.24625) 9211 : audit [DBG] from='client.60211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:32.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:32 gibba011 bash[30791]: audit 2021-03-10T16:47:31.561309+0000 mon.c (mon.1) 180 : audit [DBG] from='client.? 172.21.2.111:0/3647857744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:32.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:32 gibba011 bash[31774]: audit 2021-03-10T16:47:31.071402+0000 mgr.gibba032.whopeo (mgr.24625) 9211 : audit [DBG] from='client.60211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:32.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:32 gibba011 bash[31774]: audit 2021-03-10T16:47:31.561309+0000 mon.c (mon.1) 180 : audit [DBG] from='client.? 172.21.2.111:0/3647857744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:32.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:32 gibba032 bash[24004]: audit 2021-03-10T16:47:31.071402+0000 mgr.gibba032.whopeo (mgr.24625) 9211 : audit [DBG] from='client.60211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:32.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:32 gibba032 bash[24004]: audit 2021-03-10T16:47:31.561309+0000 mon.c (mon.1) 180 : audit [DBG] from='client.? 172.21.2.111:0/3647857744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:33 gibba032 bash[24004]: audit 2021-03-10T16:47:31.296525+0000 mgr.gibba032.whopeo (mgr.24625) 9212 : audit [DBG] from='client.60217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:33 gibba032 bash[24004]: cluster 2021-03-10T16:47:32.107863+0000 mgr.gibba032.whopeo (mgr.24625) 9213 : cluster [DBG] pgmap v5249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:33 gibba011 bash[30791]: audit 2021-03-10T16:47:31.296525+0000 mgr.gibba032.whopeo (mgr.24625) 9212 : audit [DBG] from='client.60217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:33 gibba011 bash[30791]: cluster 2021-03-10T16:47:32.107863+0000 mgr.gibba032.whopeo (mgr.24625) 9213 : cluster [DBG] pgmap v5249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:33 gibba011 bash[31774]: audit 2021-03-10T16:47:31.296525+0000 mgr.gibba032.whopeo (mgr.24625) 9212 : audit [DBG] from='client.60217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:47:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:33 gibba011 bash[31774]: cluster 2021-03-10T16:47:32.107863+0000 mgr.gibba032.whopeo (mgr.24625) 9213 : cluster [DBG] pgmap v5249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:34.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:34 gibba032 bash[24004]: cluster 2021-03-10T16:47:34.109188+0000 mgr.gibba032.whopeo (mgr.24625) 9214 : cluster [DBG] pgmap v5250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:34.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:34 gibba011 bash[31774]: cluster 2021-03-10T16:47:34.109188+0000 mgr.gibba032.whopeo (mgr.24625) 9214 : cluster [DBG] pgmap v5250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:34.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:34 gibba011 bash[30791]: cluster 2021-03-10T16:47:34.109188+0000 mgr.gibba032.whopeo (mgr.24625) 9214 : cluster [DBG] pgmap v5250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:36.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:36 gibba032 bash[24004]: cluster 2021-03-10T16:47:36.110501+0000 mgr.gibba032.whopeo (mgr.24625) 9215 : cluster [DBG] pgmap v5251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:36.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:36 gibba011 bash[30791]: cluster 2021-03-10T16:47:36.110501+0000 mgr.gibba032.whopeo (mgr.24625) 9215 : cluster [DBG] pgmap v5251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:36.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:36 gibba011 bash[31774]: cluster 2021-03-10T16:47:36.110501+0000 mgr.gibba032.whopeo (mgr.24625) 9215 : cluster [DBG] pgmap v5251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:38 gibba032 bash[24004]: cluster 2021-03-10T16:47:38.111647+0000 mgr.gibba032.whopeo (mgr.24625) 9216 : cluster [DBG] pgmap v5252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:38 gibba011 bash[30791]: cluster 2021-03-10T16:47:38.111647+0000 mgr.gibba032.whopeo (mgr.24625) 9216 : cluster [DBG] pgmap v5252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:38.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:38 gibba011 bash[31774]: cluster 2021-03-10T16:47:38.111647+0000 mgr.gibba032.whopeo (mgr.24625) 9216 : cluster [DBG] pgmap v5252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:41.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:41 gibba011 bash[30791]: cluster 2021-03-10T16:47:40.112336+0000 mgr.gibba032.whopeo (mgr.24625) 9217 : cluster [DBG] pgmap v5253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:41.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:41 gibba011 bash[31774]: cluster 2021-03-10T16:47:40.112336+0000 mgr.gibba032.whopeo (mgr.24625) 9217 : cluster [DBG] pgmap v5253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:41.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:41 gibba032 bash[24004]: cluster 2021-03-10T16:47:40.112336+0000 mgr.gibba032.whopeo (mgr.24625) 9217 : cluster [DBG] pgmap v5253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.391915+0000 mgr.gibba032.whopeo (mgr.24625) 9218 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:47:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.393134+0000 mon.a (mon.0) 3144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.395098+0000 mgr.gibba032.whopeo (mgr.24625) 9219 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:47:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.396212+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.602387+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.603289+0000 mgr.gibba032.whopeo (mgr.24625) 9220 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:47:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.605195+0000 mgr.gibba032.whopeo (mgr.24625) 9221 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:47:42.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.605836+0000 mon.a (mon.0) 3147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:47:42.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.606778+0000 mgr.gibba032.whopeo (mgr.24625) 9222 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:47:42.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.606818+0000 mgr.gibba032.whopeo (mgr.24625) 9223 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:47:42.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.607459+0000 mon.a (mon.0) 3148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.608409+0000 mgr.gibba032.whopeo (mgr.24625) 9224 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:47:42.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.608449+0000 mgr.gibba032.whopeo (mgr.24625) 9225 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:47:42.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.609025+0000 mon.a (mon.0) 3149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.609611+0000 mgr.gibba032.whopeo (mgr.24625) 9226 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:47:42.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.610489+0000 mon.a (mon.0) 3150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.610698+0000 mgr.gibba032.whopeo (mgr.24625) 9227 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:47:42.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.610739+0000 mgr.gibba032.whopeo (mgr.24625) 9228 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:47:42.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.612184+0000 mgr.gibba032.whopeo (mgr.24625) 9229 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:47:42.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.612534+0000 mgr.gibba032.whopeo (mgr.24625) 9230 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:47:42.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.612878+0000 mon.a (mon.0) 3151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.768 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.613882+0000 mgr.gibba032.whopeo (mgr.24625) 9231 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:47:42.769 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.613920+0000 mgr.gibba032.whopeo (mgr.24625) 9232 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:47:42.769 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cephadm 2021-03-10T16:47:41.614022+0000 mgr.gibba032.whopeo (mgr.24625) 9233 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:42.770 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: audit 2021-03-10T16:47:41.614653+0000 mon.a (mon.0) 3152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.770 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:42 gibba032 bash[24004]: cluster 2021-03-10T16:47:42.113626+0000 mgr.gibba032.whopeo (mgr.24625) 9234 : cluster [DBG] pgmap v5254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.391915+0000 mgr.gibba032.whopeo (mgr.24625) 9218 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:47:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.393134+0000 mon.a (mon.0) 3144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.395098+0000 mgr.gibba032.whopeo (mgr.24625) 9219 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:47:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.396212+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.602387+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.603289+0000 mgr.gibba032.whopeo (mgr.24625) 9220 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:47:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.605195+0000 mgr.gibba032.whopeo (mgr.24625) 9221 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:47:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.605836+0000 mon.a (mon.0) 3147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:47:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.606778+0000 mgr.gibba032.whopeo (mgr.24625) 9222 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:47:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.606818+0000 mgr.gibba032.whopeo (mgr.24625) 9223 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:47:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.607459+0000 mon.a (mon.0) 3148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.608409+0000 mgr.gibba032.whopeo (mgr.24625) 9224 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:47:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.608449+0000 mgr.gibba032.whopeo (mgr.24625) 9225 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:47:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.609025+0000 mon.a (mon.0) 3149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.609611+0000 mgr.gibba032.whopeo (mgr.24625) 9226 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:47:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.610489+0000 mon.a (mon.0) 3150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.610698+0000 mgr.gibba032.whopeo (mgr.24625) 9227 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:47:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.610739+0000 mgr.gibba032.whopeo (mgr.24625) 9228 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:47:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.612184+0000 mgr.gibba032.whopeo (mgr.24625) 9229 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:47:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.612534+0000 mgr.gibba032.whopeo (mgr.24625) 9230 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:47:42.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.612878+0000 mon.a (mon.0) 3151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.391915+0000 mgr.gibba032.whopeo (mgr.24625) 9218 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:47:42.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.393134+0000 mon.a (mon.0) 3144 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.395098+0000 mgr.gibba032.whopeo (mgr.24625) 9219 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:47:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.396212+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.602387+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:47:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.603289+0000 mgr.gibba032.whopeo (mgr.24625) 9220 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:47:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.605195+0000 mgr.gibba032.whopeo (mgr.24625) 9221 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:47:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.605836+0000 mon.a (mon.0) 3147 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:47:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.606778+0000 mgr.gibba032.whopeo (mgr.24625) 9222 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:47:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.606818+0000 mgr.gibba032.whopeo (mgr.24625) 9223 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:47:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.607459+0000 mon.a (mon.0) 3148 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.608409+0000 mgr.gibba032.whopeo (mgr.24625) 9224 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:47:42.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.608449+0000 mgr.gibba032.whopeo (mgr.24625) 9225 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:47:42.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.609025+0000 mon.a (mon.0) 3149 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.609611+0000 mgr.gibba032.whopeo (mgr.24625) 9226 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:47:42.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.610489+0000 mon.a (mon.0) 3150 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.610698+0000 mgr.gibba032.whopeo (mgr.24625) 9227 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:47:42.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.610739+0000 mgr.gibba032.whopeo (mgr.24625) 9228 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:47:42.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.612184+0000 mgr.gibba032.whopeo (mgr.24625) 9229 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:47:42.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.612534+0000 mgr.gibba032.whopeo (mgr.24625) 9230 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:47:42.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.612878+0000 mon.a (mon.0) 3151 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.613882+0000 mgr.gibba032.whopeo (mgr.24625) 9231 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:47:42.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.613920+0000 mgr.gibba032.whopeo (mgr.24625) 9232 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:47:42.848 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cephadm 2021-03-10T16:47:41.614022+0000 mgr.gibba032.whopeo (mgr.24625) 9233 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:42.848 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: audit 2021-03-10T16:47:41.614653+0000 mon.a (mon.0) 3152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.849 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[30791]: cluster 2021-03-10T16:47:42.113626+0000 mgr.gibba032.whopeo (mgr.24625) 9234 : cluster [DBG] pgmap v5254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:42.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.613882+0000 mgr.gibba032.whopeo (mgr.24625) 9231 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:47:42.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.613920+0000 mgr.gibba032.whopeo (mgr.24625) 9232 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:47:42.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cephadm 2021-03-10T16:47:41.614022+0000 mgr.gibba032.whopeo (mgr.24625) 9233 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:42.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: audit 2021-03-10T16:47:41.614653+0000 mon.a (mon.0) 3152 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:47:42.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:42 gibba011 bash[31774]: cluster 2021-03-10T16:47:42.113626+0000 mgr.gibba032.whopeo (mgr.24625) 9234 : cluster [DBG] pgmap v5254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:44.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:44 gibba032 bash[24004]: cluster 2021-03-10T16:47:44.115047+0000 mgr.gibba032.whopeo (mgr.24625) 9235 : cluster [DBG] pgmap v5255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:44.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:44 gibba011 bash[30791]: cluster 2021-03-10T16:47:44.115047+0000 mgr.gibba032.whopeo (mgr.24625) 9235 : cluster [DBG] pgmap v5255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:44.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:44 gibba011 bash[31774]: cluster 2021-03-10T16:47:44.115047+0000 mgr.gibba032.whopeo (mgr.24625) 9235 : cluster [DBG] pgmap v5255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:46.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:46 gibba032 bash[24004]: cluster 2021-03-10T16:47:46.116453+0000 mgr.gibba032.whopeo (mgr.24625) 9236 : cluster [DBG] pgmap v5256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:46 gibba011 bash[30791]: cluster 2021-03-10T16:47:46.116453+0000 mgr.gibba032.whopeo (mgr.24625) 9236 : cluster [DBG] pgmap v5256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:46.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:46 gibba011 bash[31774]: cluster 2021-03-10T16:47:46.116453+0000 mgr.gibba032.whopeo (mgr.24625) 9236 : cluster [DBG] pgmap v5256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:48.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:48 gibba032 bash[24004]: cluster 2021-03-10T16:47:48.117725+0000 mgr.gibba032.whopeo (mgr.24625) 9237 : cluster [DBG] pgmap v5257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:48.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:48 gibba011 bash[30791]: cluster 2021-03-10T16:47:48.117725+0000 mgr.gibba032.whopeo (mgr.24625) 9237 : cluster [DBG] pgmap v5257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:48.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:48 gibba011 bash[31774]: cluster 2021-03-10T16:47:48.117725+0000 mgr.gibba032.whopeo (mgr.24625) 9237 : cluster [DBG] pgmap v5257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:51.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:51 gibba032 bash[24004]: cluster 2021-03-10T16:47:50.118398+0000 mgr.gibba032.whopeo (mgr.24625) 9238 : cluster [DBG] pgmap v5258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:51.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:51 gibba011 bash[30791]: cluster 2021-03-10T16:47:50.118398+0000 mgr.gibba032.whopeo (mgr.24625) 9238 : cluster [DBG] pgmap v5258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:51.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:51 gibba011 bash[31774]: cluster 2021-03-10T16:47:50.118398+0000 mgr.gibba032.whopeo (mgr.24625) 9238 : cluster [DBG] pgmap v5258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:52.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:52 gibba032 bash[24004]: cluster 2021-03-10T16:47:52.119653+0000 mgr.gibba032.whopeo (mgr.24625) 9239 : cluster [DBG] pgmap v5259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:52.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:52 gibba011 bash[30791]: cluster 2021-03-10T16:47:52.119653+0000 mgr.gibba032.whopeo (mgr.24625) 9239 : cluster [DBG] pgmap v5259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:52.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:52 gibba011 bash[31774]: cluster 2021-03-10T16:47:52.119653+0000 mgr.gibba032.whopeo (mgr.24625) 9239 : cluster [DBG] pgmap v5259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:54.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:54 gibba032 bash[24004]: cluster 2021-03-10T16:47:54.121153+0000 mgr.gibba032.whopeo (mgr.24625) 9240 : cluster [DBG] pgmap v5260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:54.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:54 gibba011 bash[30791]: cluster 2021-03-10T16:47:54.121153+0000 mgr.gibba032.whopeo (mgr.24625) 9240 : cluster [DBG] pgmap v5260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:54.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:54 gibba011 bash[31774]: cluster 2021-03-10T16:47:54.121153+0000 mgr.gibba032.whopeo (mgr.24625) 9240 : cluster [DBG] pgmap v5260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:56.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:56 gibba032 bash[24004]: cluster 2021-03-10T16:47:56.122602+0000 mgr.gibba032.whopeo (mgr.24625) 9241 : cluster [DBG] pgmap v5261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:56.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:56 gibba011 bash[30791]: cluster 2021-03-10T16:47:56.122602+0000 mgr.gibba032.whopeo (mgr.24625) 9241 : cluster [DBG] pgmap v5261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:56.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:56 gibba011 bash[31774]: cluster 2021-03-10T16:47:56.122602+0000 mgr.gibba032.whopeo (mgr.24625) 9241 : cluster [DBG] pgmap v5261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:57.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:57 gibba032 bash[24004]: cephadm 2021-03-10T16:47:56.617156+0000 mgr.gibba032.whopeo (mgr.24625) 9242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:57 gibba032 bash[24004]: audit 2021-03-10T16:47:57.178011+0000 mon.a (mon.0) 3153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:47:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:57 gibba032 bash[24004]: audit 2021-03-10T16:47:57.178756+0000 mon.a (mon.0) 3154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:47:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[30791]: cephadm 2021-03-10T16:47:56.617156+0000 mgr.gibba032.whopeo (mgr.24625) 9242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[30791]: audit 2021-03-10T16:47:57.178011+0000 mon.a (mon.0) 3153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:47:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[30791]: audit 2021-03-10T16:47:57.178756+0000 mon.a (mon.0) 3154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:47:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[31774]: cephadm 2021-03-10T16:47:56.617156+0000 mgr.gibba032.whopeo (mgr.24625) 9242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:47:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[31774]: audit 2021-03-10T16:47:57.178011+0000 mon.a (mon.0) 3153 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:47:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:57 gibba011 bash[31774]: audit 2021-03-10T16:47:57.178756+0000 mon.a (mon.0) 3154 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:47:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:58 gibba032 bash[24004]: audit 2021-03-10T16:47:57.315315+0000 mon.a (mon.0) 3155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:47:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:58 gibba032 bash[24004]: audit 2021-03-10T16:47:57.316240+0000 mon.a (mon.0) 3156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:47:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:58 gibba032 bash[24004]: audit 2021-03-10T16:47:57.317046+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:58 gibba032 bash[24004]: audit 2021-03-10T16:47:57.317888+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:47:58 gibba032 bash[24004]: cluster 2021-03-10T16:47:58.123884+0000 mgr.gibba032.whopeo (mgr.24625) 9243 : cluster [DBG] pgmap v5262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[30791]: audit 2021-03-10T16:47:57.315315+0000 mon.a (mon.0) 3155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:47:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[30791]: audit 2021-03-10T16:47:57.316240+0000 mon.a (mon.0) 3156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:47:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[30791]: audit 2021-03-10T16:47:57.317046+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[30791]: audit 2021-03-10T16:47:57.317888+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[30791]: cluster 2021-03-10T16:47:58.123884+0000 mgr.gibba032.whopeo (mgr.24625) 9243 : cluster [DBG] pgmap v5262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:47:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[31774]: audit 2021-03-10T16:47:57.315315+0000 mon.a (mon.0) 3155 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:47:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[31774]: audit 2021-03-10T16:47:57.316240+0000 mon.a (mon.0) 3156 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:47:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[31774]: audit 2021-03-10T16:47:57.317046+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[31774]: audit 2021-03-10T16:47:57.317888+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:47:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:47:58 gibba011 bash[31774]: cluster 2021-03-10T16:47:58.123884+0000 mgr.gibba032.whopeo (mgr.24625) 9243 : cluster [DBG] pgmap v5262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:01 gibba032 bash[24004]: cluster 2021-03-10T16:48:00.124337+0000 mgr.gibba032.whopeo (mgr.24625) 9244 : cluster [DBG] pgmap v5263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:01.575 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:01 gibba011 bash[30791]: cluster 2021-03-10T16:48:00.124337+0000 mgr.gibba032.whopeo (mgr.24625) 9244 : cluster [DBG] pgmap v5263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:01.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:01 gibba011 bash[31774]: cluster 2021-03-10T16:48:00.124337+0000 mgr.gibba032.whopeo (mgr.24625) 9244 : cluster [DBG] pgmap v5263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:01.804 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:48:02.027 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:48:02.028 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:48:02.028 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (2h) 6m ago 2h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:48:02.028 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:48:02.028 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:48:02.029 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:48:02.029 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:48:02.029 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:48:02.029 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:48:02.030 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:48:02.030 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:48:02.030 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:48:02.030 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:48:02.030 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:48:02.031 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:48:02.031 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:48:02.031 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:48:02.031 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:48:02.032 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:48:02.032 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (2h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:48:02.032 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:48:02.275 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:48:02.276 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:48:02.277 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:48:02.278 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:02.279 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:48:02.279 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:48:02.280 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:02.280 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:48:02.280 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:48:02.280 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:02.280 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:48:02.281 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:48:02.281 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:48:02.282 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:48:02.282 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:48:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:02 gibba032 bash[24004]: audit 2021-03-10T16:48:01.796276+0000 mgr.gibba032.whopeo (mgr.24625) 9245 : audit [DBG] from='client.50247 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:02 gibba032 bash[24004]: audit 2021-03-10T16:48:02.017273+0000 mgr.gibba032.whopeo (mgr.24625) 9246 : audit [DBG] from='client.50253 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:02 gibba032 bash[24004]: cluster 2021-03-10T16:48:02.125871+0000 mgr.gibba032.whopeo (mgr.24625) 9247 : cluster [DBG] pgmap v5264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:02.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[30791]: audit 2021-03-10T16:48:01.796276+0000 mgr.gibba032.whopeo (mgr.24625) 9245 : audit [DBG] from='client.50247 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[30791]: audit 2021-03-10T16:48:02.017273+0000 mgr.gibba032.whopeo (mgr.24625) 9246 : audit [DBG] from='client.50253 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[30791]: cluster 2021-03-10T16:48:02.125871+0000 mgr.gibba032.whopeo (mgr.24625) 9247 : cluster [DBG] pgmap v5264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[31774]: audit 2021-03-10T16:48:01.796276+0000 mgr.gibba032.whopeo (mgr.24625) 9245 : audit [DBG] from='client.50247 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[31774]: audit 2021-03-10T16:48:02.017273+0000 mgr.gibba032.whopeo (mgr.24625) 9246 : audit [DBG] from='client.50253 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:02 gibba011 bash[31774]: cluster 2021-03-10T16:48:02.125871+0000 mgr.gibba032.whopeo (mgr.24625) 9247 : cluster [DBG] pgmap v5264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:03.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:03 gibba032 bash[24004]: audit 2021-03-10T16:48:02.278072+0000 mon.a (mon.0) 3159 : audit [DBG] from='client.? 172.21.2.111:0/3166545799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:03.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:03 gibba011 bash[30791]: audit 2021-03-10T16:48:02.278072+0000 mon.a (mon.0) 3159 : audit [DBG] from='client.? 172.21.2.111:0/3166545799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:03.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:03 gibba011 bash[31774]: audit 2021-03-10T16:48:02.278072+0000 mon.a (mon.0) 3159 : audit [DBG] from='client.? 172.21.2.111:0/3166545799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:04.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:04 gibba032 bash[24004]: cluster 2021-03-10T16:48:04.127365+0000 mgr.gibba032.whopeo (mgr.24625) 9248 : cluster [DBG] pgmap v5265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:04.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:04 gibba011 bash[30791]: cluster 2021-03-10T16:48:04.127365+0000 mgr.gibba032.whopeo (mgr.24625) 9248 : cluster [DBG] pgmap v5265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:04.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:04 gibba011 bash[31774]: cluster 2021-03-10T16:48:04.127365+0000 mgr.gibba032.whopeo (mgr.24625) 9248 : cluster [DBG] pgmap v5265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:06.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:06 gibba032 bash[24004]: cluster 2021-03-10T16:48:06.128748+0000 mgr.gibba032.whopeo (mgr.24625) 9249 : cluster [DBG] pgmap v5266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:06 gibba011 bash[30791]: cluster 2021-03-10T16:48:06.128748+0000 mgr.gibba032.whopeo (mgr.24625) 9249 : cluster [DBG] pgmap v5266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:06.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:06 gibba011 bash[31774]: cluster 2021-03-10T16:48:06.128748+0000 mgr.gibba032.whopeo (mgr.24625) 9249 : cluster [DBG] pgmap v5266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:08.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:08 gibba032 bash[24004]: cluster 2021-03-10T16:48:08.130629+0000 mgr.gibba032.whopeo (mgr.24625) 9250 : cluster [DBG] pgmap v5267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:08.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:08 gibba011 bash[30791]: cluster 2021-03-10T16:48:08.130629+0000 mgr.gibba032.whopeo (mgr.24625) 9250 : cluster [DBG] pgmap v5267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:08.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:08 gibba011 bash[31774]: cluster 2021-03-10T16:48:08.130629+0000 mgr.gibba032.whopeo (mgr.24625) 9250 : cluster [DBG] pgmap v5267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:11 gibba032 bash[24004]: cluster 2021-03-10T16:48:10.132178+0000 mgr.gibba032.whopeo (mgr.24625) 9251 : cluster [DBG] pgmap v5268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:11.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:11 gibba011 bash[30791]: cluster 2021-03-10T16:48:10.132178+0000 mgr.gibba032.whopeo (mgr.24625) 9251 : cluster [DBG] pgmap v5268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:11.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:11 gibba011 bash[31774]: cluster 2021-03-10T16:48:10.132178+0000 mgr.gibba032.whopeo (mgr.24625) 9251 : cluster [DBG] pgmap v5268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:12.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:12 gibba032 bash[24004]: cephadm 2021-03-10T16:48:11.633010+0000 mgr.gibba032.whopeo (mgr.24625) 9252 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:12.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:12 gibba032 bash[24004]: cluster 2021-03-10T16:48:12.133577+0000 mgr.gibba032.whopeo (mgr.24625) 9253 : cluster [DBG] pgmap v5269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:12 gibba011 bash[30791]: cephadm 2021-03-10T16:48:11.633010+0000 mgr.gibba032.whopeo (mgr.24625) 9252 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:12 gibba011 bash[30791]: cluster 2021-03-10T16:48:12.133577+0000 mgr.gibba032.whopeo (mgr.24625) 9253 : cluster [DBG] pgmap v5269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:12 gibba011 bash[31774]: cephadm 2021-03-10T16:48:11.633010+0000 mgr.gibba032.whopeo (mgr.24625) 9252 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:12 gibba011 bash[31774]: cluster 2021-03-10T16:48:12.133577+0000 mgr.gibba032.whopeo (mgr.24625) 9253 : cluster [DBG] pgmap v5269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:14.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:14 gibba032 bash[24004]: cluster 2021-03-10T16:48:14.134733+0000 mgr.gibba032.whopeo (mgr.24625) 9254 : cluster [DBG] pgmap v5270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:14.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:14 gibba011 bash[30791]: cluster 2021-03-10T16:48:14.134733+0000 mgr.gibba032.whopeo (mgr.24625) 9254 : cluster [DBG] pgmap v5270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:14 gibba011 bash[31774]: cluster 2021-03-10T16:48:14.134733+0000 mgr.gibba032.whopeo (mgr.24625) 9254 : cluster [DBG] pgmap v5270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:16.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:16 gibba032 bash[24004]: cluster 2021-03-10T16:48:16.136195+0000 mgr.gibba032.whopeo (mgr.24625) 9255 : cluster [DBG] pgmap v5271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:16.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:16 gibba011 bash[30791]: cluster 2021-03-10T16:48:16.136195+0000 mgr.gibba032.whopeo (mgr.24625) 9255 : cluster [DBG] pgmap v5271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:16.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:16 gibba011 bash[31774]: cluster 2021-03-10T16:48:16.136195+0000 mgr.gibba032.whopeo (mgr.24625) 9255 : cluster [DBG] pgmap v5271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:18.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:18 gibba032 bash[24004]: cluster 2021-03-10T16:48:18.137998+0000 mgr.gibba032.whopeo (mgr.24625) 9256 : cluster [DBG] pgmap v5272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:18.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:18 gibba011 bash[30791]: cluster 2021-03-10T16:48:18.137998+0000 mgr.gibba032.whopeo (mgr.24625) 9256 : cluster [DBG] pgmap v5272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:18.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:18 gibba011 bash[31774]: cluster 2021-03-10T16:48:18.137998+0000 mgr.gibba032.whopeo (mgr.24625) 9256 : cluster [DBG] pgmap v5272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:21.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:21 gibba032 bash[24004]: cluster 2021-03-10T16:48:20.138792+0000 mgr.gibba032.whopeo (mgr.24625) 9257 : cluster [DBG] pgmap v5273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:21.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:21 gibba011 bash[30791]: cluster 2021-03-10T16:48:20.138792+0000 mgr.gibba032.whopeo (mgr.24625) 9257 : cluster [DBG] pgmap v5273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:21.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:21 gibba011 bash[31774]: cluster 2021-03-10T16:48:20.138792+0000 mgr.gibba032.whopeo (mgr.24625) 9257 : cluster [DBG] pgmap v5273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:22.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:22 gibba032 bash[24004]: cluster 2021-03-10T16:48:22.140356+0000 mgr.gibba032.whopeo (mgr.24625) 9258 : cluster [DBG] pgmap v5274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:22.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:22 gibba011 bash[30791]: cluster 2021-03-10T16:48:22.140356+0000 mgr.gibba032.whopeo (mgr.24625) 9258 : cluster [DBG] pgmap v5274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:22 gibba011 bash[31774]: cluster 2021-03-10T16:48:22.140356+0000 mgr.gibba032.whopeo (mgr.24625) 9258 : cluster [DBG] pgmap v5274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:24.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:24 gibba032 bash[24004]: cluster 2021-03-10T16:48:24.141725+0000 mgr.gibba032.whopeo (mgr.24625) 9259 : cluster [DBG] pgmap v5275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:24 gibba011 bash[30791]: cluster 2021-03-10T16:48:24.141725+0000 mgr.gibba032.whopeo (mgr.24625) 9259 : cluster [DBG] pgmap v5275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:24.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:24 gibba011 bash[31774]: cluster 2021-03-10T16:48:24.141725+0000 mgr.gibba032.whopeo (mgr.24625) 9259 : cluster [DBG] pgmap v5275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:26.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:26 gibba032 bash[24004]: cluster 2021-03-10T16:48:26.143066+0000 mgr.gibba032.whopeo (mgr.24625) 9260 : cluster [DBG] pgmap v5276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:26.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:26 gibba011 bash[30791]: cluster 2021-03-10T16:48:26.143066+0000 mgr.gibba032.whopeo (mgr.24625) 9260 : cluster [DBG] pgmap v5276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:26.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:26 gibba011 bash[31774]: cluster 2021-03-10T16:48:26.143066+0000 mgr.gibba032.whopeo (mgr.24625) 9260 : cluster [DBG] pgmap v5276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:27.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:27 gibba032 bash[24004]: cephadm 2021-03-10T16:48:26.649036+0000 mgr.gibba032.whopeo (mgr.24625) 9261 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:27 gibba011 bash[30791]: cephadm 2021-03-10T16:48:26.649036+0000 mgr.gibba032.whopeo (mgr.24625) 9261 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:27.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:27 gibba011 bash[31774]: cephadm 2021-03-10T16:48:26.649036+0000 mgr.gibba032.whopeo (mgr.24625) 9261 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:28 gibba011 bash[30791]: cluster 2021-03-10T16:48:28.144880+0000 mgr.gibba032.whopeo (mgr.24625) 9262 : cluster [DBG] pgmap v5277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:28 gibba011 bash[31774]: cluster 2021-03-10T16:48:28.144880+0000 mgr.gibba032.whopeo (mgr.24625) 9262 : cluster [DBG] pgmap v5277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:28.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:28 gibba032 bash[24004]: cluster 2021-03-10T16:48:28.144880+0000 mgr.gibba032.whopeo (mgr.24625) 9262 : cluster [DBG] pgmap v5277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:31 gibba032 bash[24004]: cluster 2021-03-10T16:48:30.145752+0000 mgr.gibba032.whopeo (mgr.24625) 9263 : cluster [DBG] pgmap v5278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:31.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:31 gibba011 bash[30791]: cluster 2021-03-10T16:48:30.145752+0000 mgr.gibba032.whopeo (mgr.24625) 9263 : cluster [DBG] pgmap v5278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:31.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:31 gibba011 bash[31774]: cluster 2021-03-10T16:48:30.145752+0000 mgr.gibba032.whopeo (mgr.24625) 9263 : cluster [DBG] pgmap v5278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:32.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:32 gibba032 bash[24004]: cluster 2021-03-10T16:48:32.147412+0000 mgr.gibba032.whopeo (mgr.24625) 9264 : cluster [DBG] pgmap v5279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:32.520 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:48:32.575 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:32 gibba011 bash[30791]: cluster 2021-03-10T16:48:32.147412+0000 mgr.gibba032.whopeo (mgr.24625) 9264 : cluster [DBG] pgmap v5279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:32.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:32 gibba011 bash[31774]: cluster 2021-03-10T16:48:32.147412+0000 mgr.gibba032.whopeo (mgr.24625) 9264 : cluster [DBG] pgmap v5279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:32.749 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:48:32.749 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:48:32.750 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:48:32.750 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:48:32.750 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:48:32.750 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:48:32.750 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:48:32.751 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:48:32.751 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:48:32.751 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:48:32.751 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:48:32.752 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:48:32.752 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:48:32.752 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:48:32.752 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:48:32.752 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:48:32.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:48:32.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:48:32.753 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:48:32.753 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:48:32.753 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:48:33.011 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:48:33.012 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:48:33.012 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:48:33.013 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:33.013 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:48:33.013 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:48:33.014 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:33.014 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:48:33.015 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:48:33.015 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:48:33.015 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:48:33.016 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:48:33.016 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:48:33.017 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:48:33.017 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:48:33.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:33 gibba032 bash[24004]: audit 2021-03-10T16:48:32.511425+0000 mgr.gibba032.whopeo (mgr.24625) 9265 : audit [DBG] from='client.50265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:33 gibba032 bash[24004]: audit 2021-03-10T16:48:32.737768+0000 mgr.gibba032.whopeo (mgr.24625) 9266 : audit [DBG] from='client.50271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:33 gibba032 bash[24004]: audit 2021-03-10T16:48:33.014297+0000 mon.a (mon.0) 3160 : audit [DBG] from='client.? 172.21.2.111:0/2339466156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[30791]: audit 2021-03-10T16:48:32.511425+0000 mgr.gibba032.whopeo (mgr.24625) 9265 : audit [DBG] from='client.50265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[30791]: audit 2021-03-10T16:48:32.737768+0000 mgr.gibba032.whopeo (mgr.24625) 9266 : audit [DBG] from='client.50271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[30791]: audit 2021-03-10T16:48:33.014297+0000 mon.a (mon.0) 3160 : audit [DBG] from='client.? 172.21.2.111:0/2339466156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:33.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[31774]: audit 2021-03-10T16:48:32.511425+0000 mgr.gibba032.whopeo (mgr.24625) 9265 : audit [DBG] from='client.50265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[31774]: audit 2021-03-10T16:48:32.737768+0000 mgr.gibba032.whopeo (mgr.24625) 9266 : audit [DBG] from='client.50271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:48:33.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:33 gibba011 bash[31774]: audit 2021-03-10T16:48:33.014297+0000 mon.a (mon.0) 3160 : audit [DBG] from='client.? 172.21.2.111:0/2339466156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:34.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:34 gibba011 bash[30791]: cluster 2021-03-10T16:48:34.149047+0000 mgr.gibba032.whopeo (mgr.24625) 9267 : cluster [DBG] pgmap v5280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:34.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:34 gibba011 bash[31774]: cluster 2021-03-10T16:48:34.149047+0000 mgr.gibba032.whopeo (mgr.24625) 9267 : cluster [DBG] pgmap v5280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:34.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:34 gibba032 bash[24004]: cluster 2021-03-10T16:48:34.149047+0000 mgr.gibba032.whopeo (mgr.24625) 9267 : cluster [DBG] pgmap v5280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:36.004 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 16:48:35 gibba032 bash[19417]: level=info ts=2021-03-10T16:48:35.704Z caller=compact.go:507 component=tsdb msg="write block" mint=1615384112999 maxt=1615384800000 ulid=01F0EGMHB50RZTXFFHX05Y3T8D duration=18.529655ms 2021-03-10T16:48:36.005 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 16:48:35 gibba032 bash[19417]: level=info ts=2021-03-10T16:48:35.705Z caller=head.go:847 component=tsdb msg="Head GC completed" duration=1.252306ms 2021-03-10T16:48:36.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:36 gibba032 bash[24004]: cluster 2021-03-10T16:48:36.150716+0000 mgr.gibba032.whopeo (mgr.24625) 9268 : cluster [DBG] pgmap v5281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:36.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:36 gibba011 bash[30791]: cluster 2021-03-10T16:48:36.150716+0000 mgr.gibba032.whopeo (mgr.24625) 9268 : cluster [DBG] pgmap v5281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:36.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:36 gibba011 bash[31774]: cluster 2021-03-10T16:48:36.150716+0000 mgr.gibba032.whopeo (mgr.24625) 9268 : cluster [DBG] pgmap v5281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:38.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:38 gibba032 bash[24004]: cluster 2021-03-10T16:48:38.152741+0000 mgr.gibba032.whopeo (mgr.24625) 9269 : cluster [DBG] pgmap v5282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:38 gibba011 bash[30791]: cluster 2021-03-10T16:48:38.152741+0000 mgr.gibba032.whopeo (mgr.24625) 9269 : cluster [DBG] pgmap v5282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:38.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:38 gibba011 bash[31774]: cluster 2021-03-10T16:48:38.152741+0000 mgr.gibba032.whopeo (mgr.24625) 9269 : cluster [DBG] pgmap v5282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:41.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:41 gibba032 bash[24004]: cluster 2021-03-10T16:48:40.153713+0000 mgr.gibba032.whopeo (mgr.24625) 9270 : cluster [DBG] pgmap v5283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:41.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:41 gibba011 bash[30791]: cluster 2021-03-10T16:48:40.153713+0000 mgr.gibba032.whopeo (mgr.24625) 9270 : cluster [DBG] pgmap v5283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:41.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:41 gibba011 bash[31774]: cluster 2021-03-10T16:48:40.153713+0000 mgr.gibba032.whopeo (mgr.24625) 9270 : cluster [DBG] pgmap v5283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:43.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.665905+0000 mgr.gibba032.whopeo (mgr.24625) 9271 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:48:43.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.667436+0000 mgr.gibba032.whopeo (mgr.24625) 9272 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:48:43.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.897990+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:48:43.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.898795+0000 mgr.gibba032.whopeo (mgr.24625) 9273 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:48:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.901504+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:48:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.901691+0000 mgr.gibba032.whopeo (mgr.24625) 9274 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:48:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.903977+0000 mon.a (mon.0) 3163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.904205+0000 mgr.gibba032.whopeo (mgr.24625) 9275 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:48:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.904240+0000 mgr.gibba032.whopeo (mgr.24625) 9276 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:48:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.906680+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.906748+0000 mgr.gibba032.whopeo (mgr.24625) 9277 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:48:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.906787+0000 mgr.gibba032.whopeo (mgr.24625) 9278 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:48:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.909245+0000 mon.a (mon.0) 3165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.909427+0000 mgr.gibba032.whopeo (mgr.24625) 9279 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:48:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.911926+0000 mgr.gibba032.whopeo (mgr.24625) 9280 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:48:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.911977+0000 mgr.gibba032.whopeo (mgr.24625) 9281 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:48:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.914406+0000 mgr.gibba032.whopeo (mgr.24625) 9282 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:48:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.914466+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.914746+0000 mgr.gibba032.whopeo (mgr.24625) 9283 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:48:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.916814+0000 mgr.gibba032.whopeo (mgr.24625) 9284 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:48:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.916855+0000 mgr.gibba032.whopeo (mgr.24625) 9285 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:48:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: audit 2021-03-10T16:48:41.916943+0000 mon.a (mon.0) 3167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cephadm 2021-03-10T16:48:41.916967+0000 mgr.gibba032.whopeo (mgr.24625) 9286 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:42 gibba032 bash[24004]: cluster 2021-03-10T16:48:42.155152+0000 mgr.gibba032.whopeo (mgr.24625) 9287 : cluster [DBG] pgmap v5284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:43.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.665905+0000 mgr.gibba032.whopeo (mgr.24625) 9271 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:48:43.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.667436+0000 mgr.gibba032.whopeo (mgr.24625) 9272 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:48:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.897990+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:48:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.898795+0000 mgr.gibba032.whopeo (mgr.24625) 9273 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:48:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.901504+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:48:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.901691+0000 mgr.gibba032.whopeo (mgr.24625) 9274 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:48:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.903977+0000 mon.a (mon.0) 3163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.904205+0000 mgr.gibba032.whopeo (mgr.24625) 9275 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:48:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.904240+0000 mgr.gibba032.whopeo (mgr.24625) 9276 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:48:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.906680+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.906748+0000 mgr.gibba032.whopeo (mgr.24625) 9277 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:48:43.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.906787+0000 mgr.gibba032.whopeo (mgr.24625) 9278 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:48:43.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.909245+0000 mon.a (mon.0) 3165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.909427+0000 mgr.gibba032.whopeo (mgr.24625) 9279 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:48:43.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.911926+0000 mgr.gibba032.whopeo (mgr.24625) 9280 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:48:43.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.911977+0000 mgr.gibba032.whopeo (mgr.24625) 9281 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:48:43.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.914406+0000 mgr.gibba032.whopeo (mgr.24625) 9282 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:48:43.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.914466+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.914746+0000 mgr.gibba032.whopeo (mgr.24625) 9283 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:48:43.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.916814+0000 mgr.gibba032.whopeo (mgr.24625) 9284 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:48:43.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.916855+0000 mgr.gibba032.whopeo (mgr.24625) 9285 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:48:43.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.665905+0000 mgr.gibba032.whopeo (mgr.24625) 9271 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:48:43.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.667436+0000 mgr.gibba032.whopeo (mgr.24625) 9272 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:48:43.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.897990+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:48:43.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.898795+0000 mgr.gibba032.whopeo (mgr.24625) 9273 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:48:43.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.901504+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:48:43.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.901691+0000 mgr.gibba032.whopeo (mgr.24625) 9274 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:48:43.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.903977+0000 mon.a (mon.0) 3163 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.904205+0000 mgr.gibba032.whopeo (mgr.24625) 9275 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:48:43.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.904240+0000 mgr.gibba032.whopeo (mgr.24625) 9276 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:48:43.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.906680+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.906748+0000 mgr.gibba032.whopeo (mgr.24625) 9277 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:48:43.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.906787+0000 mgr.gibba032.whopeo (mgr.24625) 9278 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:48:43.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.909245+0000 mon.a (mon.0) 3165 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.909427+0000 mgr.gibba032.whopeo (mgr.24625) 9279 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:48:43.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.911926+0000 mgr.gibba032.whopeo (mgr.24625) 9280 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:48:43.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.911977+0000 mgr.gibba032.whopeo (mgr.24625) 9281 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:48:43.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.914406+0000 mgr.gibba032.whopeo (mgr.24625) 9282 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:48:43.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.914466+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.914746+0000 mgr.gibba032.whopeo (mgr.24625) 9283 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:48:43.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.916814+0000 mgr.gibba032.whopeo (mgr.24625) 9284 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:48:43.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.916855+0000 mgr.gibba032.whopeo (mgr.24625) 9285 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:48:43.345 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: audit 2021-03-10T16:48:41.916943+0000 mon.a (mon.0) 3167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cephadm 2021-03-10T16:48:41.916967+0000 mgr.gibba032.whopeo (mgr.24625) 9286 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:43.346 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[30791]: cluster 2021-03-10T16:48:42.155152+0000 mgr.gibba032.whopeo (mgr.24625) 9287 : cluster [DBG] pgmap v5284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:43.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: audit 2021-03-10T16:48:41.916943+0000 mon.a (mon.0) 3167 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:48:43.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cephadm 2021-03-10T16:48:41.916967+0000 mgr.gibba032.whopeo (mgr.24625) 9286 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:43.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:42 gibba011 bash[31774]: cluster 2021-03-10T16:48:42.155152+0000 mgr.gibba032.whopeo (mgr.24625) 9287 : cluster [DBG] pgmap v5284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:44.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:44 gibba032 bash[24004]: cluster 2021-03-10T16:48:44.156799+0000 mgr.gibba032.whopeo (mgr.24625) 9288 : cluster [DBG] pgmap v5285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:44.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:44 gibba011 bash[30791]: cluster 2021-03-10T16:48:44.156799+0000 mgr.gibba032.whopeo (mgr.24625) 9288 : cluster [DBG] pgmap v5285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:44.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:44 gibba011 bash[31774]: cluster 2021-03-10T16:48:44.156799+0000 mgr.gibba032.whopeo (mgr.24625) 9288 : cluster [DBG] pgmap v5285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:46.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:46 gibba032 bash[24004]: cluster 2021-03-10T16:48:46.158156+0000 mgr.gibba032.whopeo (mgr.24625) 9289 : cluster [DBG] pgmap v5286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:46 gibba011 bash[30791]: cluster 2021-03-10T16:48:46.158156+0000 mgr.gibba032.whopeo (mgr.24625) 9289 : cluster [DBG] pgmap v5286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:46.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:46 gibba011 bash[31774]: cluster 2021-03-10T16:48:46.158156+0000 mgr.gibba032.whopeo (mgr.24625) 9289 : cluster [DBG] pgmap v5286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:48.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:48 gibba032 bash[24004]: cluster 2021-03-10T16:48:48.160115+0000 mgr.gibba032.whopeo (mgr.24625) 9290 : cluster [DBG] pgmap v5287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:48.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:48 gibba011 bash[30791]: cluster 2021-03-10T16:48:48.160115+0000 mgr.gibba032.whopeo (mgr.24625) 9290 : cluster [DBG] pgmap v5287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:48.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:48 gibba011 bash[31774]: cluster 2021-03-10T16:48:48.160115+0000 mgr.gibba032.whopeo (mgr.24625) 9290 : cluster [DBG] pgmap v5287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:51.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:51 gibba032 bash[24004]: cluster 2021-03-10T16:48:50.160818+0000 mgr.gibba032.whopeo (mgr.24625) 9291 : cluster [DBG] pgmap v5288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:51.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:51 gibba011 bash[30791]: cluster 2021-03-10T16:48:50.160818+0000 mgr.gibba032.whopeo (mgr.24625) 9291 : cluster [DBG] pgmap v5288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:51.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:51 gibba011 bash[31774]: cluster 2021-03-10T16:48:50.160818+0000 mgr.gibba032.whopeo (mgr.24625) 9291 : cluster [DBG] pgmap v5288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:52.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:52 gibba032 bash[24004]: cluster 2021-03-10T16:48:52.162141+0000 mgr.gibba032.whopeo (mgr.24625) 9292 : cluster [DBG] pgmap v5289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:52.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:52 gibba011 bash[30791]: cluster 2021-03-10T16:48:52.162141+0000 mgr.gibba032.whopeo (mgr.24625) 9292 : cluster [DBG] pgmap v5289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:52.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:52 gibba011 bash[31774]: cluster 2021-03-10T16:48:52.162141+0000 mgr.gibba032.whopeo (mgr.24625) 9292 : cluster [DBG] pgmap v5289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:54.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:54 gibba032 bash[24004]: cluster 2021-03-10T16:48:54.163582+0000 mgr.gibba032.whopeo (mgr.24625) 9293 : cluster [DBG] pgmap v5290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:54.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:54 gibba011 bash[30791]: cluster 2021-03-10T16:48:54.163582+0000 mgr.gibba032.whopeo (mgr.24625) 9293 : cluster [DBG] pgmap v5290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:54 gibba011 bash[31774]: cluster 2021-03-10T16:48:54.163582+0000 mgr.gibba032.whopeo (mgr.24625) 9293 : cluster [DBG] pgmap v5290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:56.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:56 gibba032 bash[24004]: cluster 2021-03-10T16:48:56.164943+0000 mgr.gibba032.whopeo (mgr.24625) 9294 : cluster [DBG] pgmap v5291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:56.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:56 gibba011 bash[30791]: cluster 2021-03-10T16:48:56.164943+0000 mgr.gibba032.whopeo (mgr.24625) 9294 : cluster [DBG] pgmap v5291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:56 gibba011 bash[31774]: cluster 2021-03-10T16:48:56.164943+0000 mgr.gibba032.whopeo (mgr.24625) 9294 : cluster [DBG] pgmap v5291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[30791]: cephadm 2021-03-10T16:48:56.932739+0000 mgr.gibba032.whopeo (mgr.24625) 9295 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[30791]: audit 2021-03-10T16:48:57.185270+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:48:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[30791]: audit 2021-03-10T16:48:57.185973+0000 mon.a (mon.0) 3169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:48:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[31774]: cephadm 2021-03-10T16:48:56.932739+0000 mgr.gibba032.whopeo (mgr.24625) 9295 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[31774]: audit 2021-03-10T16:48:57.185270+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:48:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:57 gibba011 bash[31774]: audit 2021-03-10T16:48:57.185973+0000 mon.a (mon.0) 3169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:48:57.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:57 gibba032 bash[24004]: cephadm 2021-03-10T16:48:56.932739+0000 mgr.gibba032.whopeo (mgr.24625) 9295 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:48:57.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:57 gibba032 bash[24004]: audit 2021-03-10T16:48:57.185270+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:48:57.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:57 gibba032 bash[24004]: audit 2021-03-10T16:48:57.185973+0000 mon.a (mon.0) 3169 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:48:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[30791]: audit 2021-03-10T16:48:57.325350+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:48:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[30791]: audit 2021-03-10T16:48:57.325787+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[30791]: audit 2021-03-10T16:48:57.326170+0000 mon.a (mon.0) 3172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:48:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[30791]: audit 2021-03-10T16:48:57.326556+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[30791]: cluster 2021-03-10T16:48:58.166799+0000 mgr.gibba032.whopeo (mgr.24625) 9296 : cluster [DBG] pgmap v5292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[31774]: audit 2021-03-10T16:48:57.325350+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:48:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[31774]: audit 2021-03-10T16:48:57.325787+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[31774]: audit 2021-03-10T16:48:57.326170+0000 mon.a (mon.0) 3172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:48:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[31774]: audit 2021-03-10T16:48:57.326556+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:48:58 gibba011 bash[31774]: cluster 2021-03-10T16:48:58.166799+0000 mgr.gibba032.whopeo (mgr.24625) 9296 : cluster [DBG] pgmap v5292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:48:58.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:58 gibba032 bash[24004]: audit 2021-03-10T16:48:57.325350+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T16:48:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:58 gibba032 bash[24004]: audit 2021-03-10T16:48:57.325787+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:58 gibba032 bash[24004]: audit 2021-03-10T16:48:57.326170+0000 mon.a (mon.0) 3172 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T16:48:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:58 gibba032 bash[24004]: audit 2021-03-10T16:48:57.326556+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:48:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:48:58 gibba032 bash[24004]: cluster 2021-03-10T16:48:58.166799+0000 mgr.gibba032.whopeo (mgr.24625) 9296 : cluster [DBG] pgmap v5292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:01 gibba032 bash[24004]: cluster 2021-03-10T16:49:00.167822+0000 mgr.gibba032.whopeo (mgr.24625) 9297 : cluster [DBG] pgmap v5293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:01.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:01 gibba011 bash[30791]: cluster 2021-03-10T16:49:00.167822+0000 mgr.gibba032.whopeo (mgr.24625) 9297 : cluster [DBG] pgmap v5293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:01.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:01 gibba011 bash[31774]: cluster 2021-03-10T16:49:00.167822+0000 mgr.gibba032.whopeo (mgr.24625) 9297 : cluster [DBG] pgmap v5293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:02.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:02 gibba011 bash[30791]: cluster 2021-03-10T16:49:02.169380+0000 mgr.gibba032.whopeo (mgr.24625) 9298 : cluster [DBG] pgmap v5294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:02 gibba011 bash[31774]: cluster 2021-03-10T16:49:02.169380+0000 mgr.gibba032.whopeo (mgr.24625) 9298 : cluster [DBG] pgmap v5294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:02.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:02 gibba032 bash[24004]: cluster 2021-03-10T16:49:02.169380+0000 mgr.gibba032.whopeo (mgr.24625) 9298 : cluster [DBG] pgmap v5294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:03.256 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:49:03.485 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:49:03.486 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:49:03.486 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:49:03.486 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:49:03.487 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:49:03.487 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:49:03.487 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:49:03.487 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:49:03.487 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:49:03.488 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:49:03.488 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:49:03.488 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:49:03.488 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:49:03.489 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:49:03.489 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:49:03.489 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:49:03.489 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:49:03.489 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:49:03.490 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:49:03.490 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:49:03.490 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:49:03.733 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:49:03.733 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:49:03.734 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:49:03.734 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:03.734 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:49:03.735 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:49:03.735 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:03.736 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:49:03.736 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:49:03.736 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:03.736 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:49:03.736 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:49:03.737 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:49:03.737 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:49:03.737 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:49:04.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:03 gibba011 bash[30791]: audit 2021-03-10T16:49:03.735882+0000 mon.a (mon.0) 3174 : audit [DBG] from='client.? 172.21.2.111:0/2666874015' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:04.076 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:03 gibba011 bash[31774]: audit 2021-03-10T16:49:03.735882+0000 mon.a (mon.0) 3174 : audit [DBG] from='client.? 172.21.2.111:0/2666874015' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:04.254 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:03 gibba032 bash[24004]: audit 2021-03-10T16:49:03.735882+0000 mon.a (mon.0) 3174 : audit [DBG] from='client.? 172.21.2.111:0/2666874015' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:05.076 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[30791]: audit 2021-03-10T16:49:03.249332+0000 mgr.gibba032.whopeo (mgr.24625) 9299 : audit [DBG] from='client.60265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[30791]: audit 2021-03-10T16:49:03.474561+0000 mgr.gibba032.whopeo (mgr.24625) 9300 : audit [DBG] from='client.60271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[30791]: cluster 2021-03-10T16:49:04.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9301 : cluster [DBG] pgmap v5295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:05.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[31774]: audit 2021-03-10T16:49:03.249332+0000 mgr.gibba032.whopeo (mgr.24625) 9299 : audit [DBG] from='client.60265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[31774]: audit 2021-03-10T16:49:03.474561+0000 mgr.gibba032.whopeo (mgr.24625) 9300 : audit [DBG] from='client.60271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:04 gibba011 bash[31774]: cluster 2021-03-10T16:49:04.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9301 : cluster [DBG] pgmap v5295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:04 gibba032 bash[24004]: audit 2021-03-10T16:49:03.249332+0000 mgr.gibba032.whopeo (mgr.24625) 9299 : audit [DBG] from='client.60265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:04 gibba032 bash[24004]: audit 2021-03-10T16:49:03.474561+0000 mgr.gibba032.whopeo (mgr.24625) 9300 : audit [DBG] from='client.60271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:05.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:04 gibba032 bash[24004]: cluster 2021-03-10T16:49:04.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9301 : cluster [DBG] pgmap v5295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:06 gibba011 bash[30791]: cluster 2021-03-10T16:49:06.172345+0000 mgr.gibba032.whopeo (mgr.24625) 9302 : cluster [DBG] pgmap v5296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:06.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:06 gibba011 bash[31774]: cluster 2021-03-10T16:49:06.172345+0000 mgr.gibba032.whopeo (mgr.24625) 9302 : cluster [DBG] pgmap v5296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:06.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:06 gibba032 bash[24004]: cluster 2021-03-10T16:49:06.172345+0000 mgr.gibba032.whopeo (mgr.24625) 9302 : cluster [DBG] pgmap v5296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:08.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:08 gibba011 bash[30791]: cluster 2021-03-10T16:49:08.174038+0000 mgr.gibba032.whopeo (mgr.24625) 9303 : cluster [DBG] pgmap v5297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:08.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:08 gibba011 bash[31774]: cluster 2021-03-10T16:49:08.174038+0000 mgr.gibba032.whopeo (mgr.24625) 9303 : cluster [DBG] pgmap v5297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:08.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:08 gibba032 bash[24004]: cluster 2021-03-10T16:49:08.174038+0000 mgr.gibba032.whopeo (mgr.24625) 9303 : cluster [DBG] pgmap v5297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:11.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:11 gibba032 bash[24004]: cluster 2021-03-10T16:49:10.174728+0000 mgr.gibba032.whopeo (mgr.24625) 9304 : cluster [DBG] pgmap v5298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:11.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:11 gibba011 bash[30791]: cluster 2021-03-10T16:49:10.174728+0000 mgr.gibba032.whopeo (mgr.24625) 9304 : cluster [DBG] pgmap v5298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:11.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:11 gibba011 bash[31774]: cluster 2021-03-10T16:49:10.174728+0000 mgr.gibba032.whopeo (mgr.24625) 9304 : cluster [DBG] pgmap v5298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:12 gibba011 bash[30791]: cephadm 2021-03-10T16:49:11.948647+0000 mgr.gibba032.whopeo (mgr.24625) 9305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:12 gibba011 bash[30791]: cluster 2021-03-10T16:49:12.176010+0000 mgr.gibba032.whopeo (mgr.24625) 9306 : cluster [DBG] pgmap v5299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:12.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:12 gibba011 bash[31774]: cephadm 2021-03-10T16:49:11.948647+0000 mgr.gibba032.whopeo (mgr.24625) 9305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:12 gibba011 bash[31774]: cluster 2021-03-10T16:49:12.176010+0000 mgr.gibba032.whopeo (mgr.24625) 9306 : cluster [DBG] pgmap v5299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:12.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:12 gibba032 bash[24004]: cephadm 2021-03-10T16:49:11.948647+0000 mgr.gibba032.whopeo (mgr.24625) 9305 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:12.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:12 gibba032 bash[24004]: cluster 2021-03-10T16:49:12.176010+0000 mgr.gibba032.whopeo (mgr.24625) 9306 : cluster [DBG] pgmap v5299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:14.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:14 gibba011 bash[30791]: cluster 2021-03-10T16:49:14.177564+0000 mgr.gibba032.whopeo (mgr.24625) 9307 : cluster [DBG] pgmap v5300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:14 gibba011 bash[31774]: cluster 2021-03-10T16:49:14.177564+0000 mgr.gibba032.whopeo (mgr.24625) 9307 : cluster [DBG] pgmap v5300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:14.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:14 gibba032 bash[24004]: cluster 2021-03-10T16:49:14.177564+0000 mgr.gibba032.whopeo (mgr.24625) 9307 : cluster [DBG] pgmap v5300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:16.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:16 gibba011 bash[30791]: cluster 2021-03-10T16:49:16.178979+0000 mgr.gibba032.whopeo (mgr.24625) 9308 : cluster [DBG] pgmap v5301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:16.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:16 gibba011 bash[31774]: cluster 2021-03-10T16:49:16.178979+0000 mgr.gibba032.whopeo (mgr.24625) 9308 : cluster [DBG] pgmap v5301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:16.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:16 gibba032 bash[24004]: cluster 2021-03-10T16:49:16.178979+0000 mgr.gibba032.whopeo (mgr.24625) 9308 : cluster [DBG] pgmap v5301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:18.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:18 gibba011 bash[30791]: cluster 2021-03-10T16:49:18.180788+0000 mgr.gibba032.whopeo (mgr.24625) 9309 : cluster [DBG] pgmap v5302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:18.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:18 gibba011 bash[31774]: cluster 2021-03-10T16:49:18.180788+0000 mgr.gibba032.whopeo (mgr.24625) 9309 : cluster [DBG] pgmap v5302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:18.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:18 gibba032 bash[24004]: cluster 2021-03-10T16:49:18.180788+0000 mgr.gibba032.whopeo (mgr.24625) 9309 : cluster [DBG] pgmap v5302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:21.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:21 gibba032 bash[24004]: cluster 2021-03-10T16:49:20.182261+0000 mgr.gibba032.whopeo (mgr.24625) 9310 : cluster [DBG] pgmap v5303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:21.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:21 gibba011 bash[30791]: cluster 2021-03-10T16:49:20.182261+0000 mgr.gibba032.whopeo (mgr.24625) 9310 : cluster [DBG] pgmap v5303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:21.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:21 gibba011 bash[31774]: cluster 2021-03-10T16:49:20.182261+0000 mgr.gibba032.whopeo (mgr.24625) 9310 : cluster [DBG] pgmap v5303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:22.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:22 gibba011 bash[30791]: cluster 2021-03-10T16:49:22.183891+0000 mgr.gibba032.whopeo (mgr.24625) 9311 : cluster [DBG] pgmap v5304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:22 gibba011 bash[31774]: cluster 2021-03-10T16:49:22.183891+0000 mgr.gibba032.whopeo (mgr.24625) 9311 : cluster [DBG] pgmap v5304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:22.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:22 gibba032 bash[24004]: cluster 2021-03-10T16:49:22.183891+0000 mgr.gibba032.whopeo (mgr.24625) 9311 : cluster [DBG] pgmap v5304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:24 gibba011 bash[30791]: cluster 2021-03-10T16:49:24.185163+0000 mgr.gibba032.whopeo (mgr.24625) 9312 : cluster [DBG] pgmap v5305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:24 gibba011 bash[31774]: cluster 2021-03-10T16:49:24.185163+0000 mgr.gibba032.whopeo (mgr.24625) 9312 : cluster [DBG] pgmap v5305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:24.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:24 gibba032 bash[24004]: cluster 2021-03-10T16:49:24.185163+0000 mgr.gibba032.whopeo (mgr.24625) 9312 : cluster [DBG] pgmap v5305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:26.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:26 gibba011 bash[30791]: cluster 2021-03-10T16:49:26.186872+0000 mgr.gibba032.whopeo (mgr.24625) 9313 : cluster [DBG] pgmap v5306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:26.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:26 gibba011 bash[31774]: cluster 2021-03-10T16:49:26.186872+0000 mgr.gibba032.whopeo (mgr.24625) 9313 : cluster [DBG] pgmap v5306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:26.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:26 gibba032 bash[24004]: cluster 2021-03-10T16:49:26.186872+0000 mgr.gibba032.whopeo (mgr.24625) 9313 : cluster [DBG] pgmap v5306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:27.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:27 gibba011 bash[30791]: cephadm 2021-03-10T16:49:26.958656+0000 mgr.gibba032.whopeo (mgr.24625) 9314 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:27.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:27 gibba011 bash[31774]: cephadm 2021-03-10T16:49:26.958656+0000 mgr.gibba032.whopeo (mgr.24625) 9314 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:27.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:27 gibba032 bash[24004]: cephadm 2021-03-10T16:49:26.958656+0000 mgr.gibba032.whopeo (mgr.24625) 9314 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:28 gibba011 bash[30791]: cluster 2021-03-10T16:49:28.188685+0000 mgr.gibba032.whopeo (mgr.24625) 9315 : cluster [DBG] pgmap v5307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:28 gibba011 bash[31774]: cluster 2021-03-10T16:49:28.188685+0000 mgr.gibba032.whopeo (mgr.24625) 9315 : cluster [DBG] pgmap v5307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:28.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:28 gibba032 bash[24004]: cluster 2021-03-10T16:49:28.188685+0000 mgr.gibba032.whopeo (mgr.24625) 9315 : cluster [DBG] pgmap v5307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:31.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:31 gibba032 bash[24004]: cluster 2021-03-10T16:49:30.190479+0000 mgr.gibba032.whopeo (mgr.24625) 9316 : cluster [DBG] pgmap v5308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:31.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:31 gibba011 bash[30791]: cluster 2021-03-10T16:49:30.190479+0000 mgr.gibba032.whopeo (mgr.24625) 9316 : cluster [DBG] pgmap v5308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:31.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:31 gibba011 bash[31774]: cluster 2021-03-10T16:49:30.190479+0000 mgr.gibba032.whopeo (mgr.24625) 9316 : cluster [DBG] pgmap v5308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:32 gibba011 bash[30791]: cluster 2021-03-10T16:49:32.192385+0000 mgr.gibba032.whopeo (mgr.24625) 9317 : cluster [DBG] pgmap v5309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:32.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:32 gibba011 bash[31774]: cluster 2021-03-10T16:49:32.192385+0000 mgr.gibba032.whopeo (mgr.24625) 9317 : cluster [DBG] pgmap v5309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:32.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:32 gibba032 bash[24004]: cluster 2021-03-10T16:49:32.192385+0000 mgr.gibba032.whopeo (mgr.24625) 9317 : cluster [DBG] pgmap v5309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:33.979 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:49:34.206 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:49:34.207 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:49:34.207 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:49:34.207 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:49:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:49:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:49:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:49:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:49:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:49:34.209 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:49:34.209 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:49:34.209 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:49:34.209 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:49:34.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:49:34.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:49:34.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:49:34.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:49:34.210 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:49:34.211 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:49:34.211 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:49:34.211 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 2h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:49:34.457 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:49:34.457 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:49:34.458 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:49:34.458 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:34.458 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:49:34.458 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:49:34.459 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:34.459 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:49:34.459 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:49:34.459 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:49:34.459 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:49:34.460 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:49:34.460 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:49:34.460 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:49:34.460 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:49:34.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[30791]: audit 2021-03-10T16:49:33.972397+0000 mgr.gibba032.whopeo (mgr.24625) 9318 : audit [DBG] from='client.60283 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:34.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[30791]: cluster 2021-03-10T16:49:34.194083+0000 mgr.gibba032.whopeo (mgr.24625) 9319 : cluster [DBG] pgmap v5310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:34.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[30791]: audit 2021-03-10T16:49:34.195873+0000 mgr.gibba032.whopeo (mgr.24625) 9320 : audit [DBG] from='client.60289 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[31774]: audit 2021-03-10T16:49:33.972397+0000 mgr.gibba032.whopeo (mgr.24625) 9318 : audit [DBG] from='client.60283 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:34.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[31774]: cluster 2021-03-10T16:49:34.194083+0000 mgr.gibba032.whopeo (mgr.24625) 9319 : cluster [DBG] pgmap v5310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:34 gibba011 bash[31774]: audit 2021-03-10T16:49:34.195873+0000 mgr.gibba032.whopeo (mgr.24625) 9320 : audit [DBG] from='client.60289 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:34.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:34 gibba032 bash[24004]: audit 2021-03-10T16:49:33.972397+0000 mgr.gibba032.whopeo (mgr.24625) 9318 : audit [DBG] from='client.60283 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:34.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:34 gibba032 bash[24004]: cluster 2021-03-10T16:49:34.194083+0000 mgr.gibba032.whopeo (mgr.24625) 9319 : cluster [DBG] pgmap v5310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:34.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:34 gibba032 bash[24004]: audit 2021-03-10T16:49:34.195873+0000 mgr.gibba032.whopeo (mgr.24625) 9320 : audit [DBG] from='client.60289 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:49:35.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:35 gibba011 bash[30791]: audit 2021-03-10T16:49:34.459907+0000 mon.a (mon.0) 3175 : audit [DBG] from='client.? 172.21.2.111:0/3032977163' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:35.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:35 gibba011 bash[31774]: audit 2021-03-10T16:49:34.459907+0000 mon.a (mon.0) 3175 : audit [DBG] from='client.? 172.21.2.111:0/3032977163' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:35.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:35 gibba032 bash[24004]: audit 2021-03-10T16:49:34.459907+0000 mon.a (mon.0) 3175 : audit [DBG] from='client.? 172.21.2.111:0/3032977163' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:36.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:36 gibba011 bash[30791]: cluster 2021-03-10T16:49:36.195740+0000 mgr.gibba032.whopeo (mgr.24625) 9321 : cluster [DBG] pgmap v5311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:36.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:36 gibba011 bash[31774]: cluster 2021-03-10T16:49:36.195740+0000 mgr.gibba032.whopeo (mgr.24625) 9321 : cluster [DBG] pgmap v5311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:36.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:36 gibba032 bash[24004]: cluster 2021-03-10T16:49:36.195740+0000 mgr.gibba032.whopeo (mgr.24625) 9321 : cluster [DBG] pgmap v5311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:38 gibba011 bash[30791]: cluster 2021-03-10T16:49:38.197658+0000 mgr.gibba032.whopeo (mgr.24625) 9322 : cluster [DBG] pgmap v5312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:38 gibba011 bash[31774]: cluster 2021-03-10T16:49:38.197658+0000 mgr.gibba032.whopeo (mgr.24625) 9322 : cluster [DBG] pgmap v5312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:38.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:38 gibba032 bash[24004]: cluster 2021-03-10T16:49:38.197658+0000 mgr.gibba032.whopeo (mgr.24625) 9322 : cluster [DBG] pgmap v5312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:41.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:41 gibba032 bash[24004]: cluster 2021-03-10T16:49:40.198672+0000 mgr.gibba032.whopeo (mgr.24625) 9323 : cluster [DBG] pgmap v5313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:41.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:41 gibba011 bash[30791]: cluster 2021-03-10T16:49:40.198672+0000 mgr.gibba032.whopeo (mgr.24625) 9323 : cluster [DBG] pgmap v5313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:41.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:41 gibba011 bash[31774]: cluster 2021-03-10T16:49:40.198672+0000 mgr.gibba032.whopeo (mgr.24625) 9323 : cluster [DBG] pgmap v5313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:42.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:42 gibba032 bash[24004]: audit 2021-03-10T16:49:41.978618+0000 mon.a (mon.0) 3176 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:49:42.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:42 gibba011 bash[30791]: audit 2021-03-10T16:49:41.978618+0000 mon.a (mon.0) 3176 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:49:42.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:42 gibba011 bash[31774]: audit 2021-03-10T16:49:41.978618+0000 mon.a (mon.0) 3176 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:49:43.408 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:43 gibba011 bash[30791]: cephadm 2021-03-10T16:49:41.976401+0000 mgr.gibba032.whopeo (mgr.24625) 9324 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:49:43.409 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:43 gibba011 bash[30791]: cluster 2021-03-10T16:49:42.199254+0000 mgr.gibba032.whopeo (mgr.24625) 9325 : cluster [DBG] pgmap v5314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:43.410 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:43 gibba011 bash[31774]: cephadm 2021-03-10T16:49:41.976401+0000 mgr.gibba032.whopeo (mgr.24625) 9324 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:49:43.410 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:43 gibba011 bash[31774]: cluster 2021-03-10T16:49:42.199254+0000 mgr.gibba032.whopeo (mgr.24625) 9325 : cluster [DBG] pgmap v5314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:43.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:43 gibba032 bash[24004]: cephadm 2021-03-10T16:49:41.976401+0000 mgr.gibba032.whopeo (mgr.24625) 9324 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:49:43.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:43 gibba032 bash[24004]: cluster 2021-03-10T16:49:42.199254+0000 mgr.gibba032.whopeo (mgr.24625) 9325 : cluster [DBG] pgmap v5314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:44.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:44 gibba011 bash[30791]: cluster 2021-03-10T16:49:44.200494+0000 mgr.gibba032.whopeo (mgr.24625) 9326 : cluster [DBG] pgmap v5315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:44.326 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:44 gibba011 bash[31774]: cluster 2021-03-10T16:49:44.200494+0000 mgr.gibba032.whopeo (mgr.24625) 9326 : cluster [DBG] pgmap v5315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:44.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:44 gibba032 bash[24004]: cluster 2021-03-10T16:49:44.200494+0000 mgr.gibba032.whopeo (mgr.24625) 9326 : cluster [DBG] pgmap v5315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:46 gibba011 bash[30791]: cluster 2021-03-10T16:49:46.202232+0000 mgr.gibba032.whopeo (mgr.24625) 9327 : cluster [DBG] pgmap v5316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:46.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:46 gibba011 bash[31774]: cluster 2021-03-10T16:49:46.202232+0000 mgr.gibba032.whopeo (mgr.24625) 9327 : cluster [DBG] pgmap v5316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:46.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:46 gibba032 bash[24004]: cluster 2021-03-10T16:49:46.202232+0000 mgr.gibba032.whopeo (mgr.24625) 9327 : cluster [DBG] pgmap v5316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:47.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[30791]: audit 2021-03-10T16:49:46.886182+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:49:47.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.098145+0000 mgr.gibba032.whopeo (mgr.24625) 9328 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:49:47.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[30791]: audit 2021-03-10T16:49:47.098196+0000 mon.a (mon.0) 3178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:47.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[31774]: audit 2021-03-10T16:49:46.886182+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:49:47.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.098145+0000 mgr.gibba032.whopeo (mgr.24625) 9328 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:49:47.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:47 gibba011 bash[31774]: audit 2021-03-10T16:49:47.098196+0000 mon.a (mon.0) 3178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:47.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:47 gibba032 bash[24004]: audit 2021-03-10T16:49:46.886182+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:49:47.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:47 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.098145+0000 mgr.gibba032.whopeo (mgr.24625) 9328 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:49:47.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:47 gibba032 bash[24004]: audit 2021-03-10T16:49:47.098196+0000 mon.a (mon.0) 3178 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:48.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.331154+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:48.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.334117+0000 mgr.gibba032.whopeo (mgr.24625) 9329 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:49:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.337082+0000 mgr.gibba032.whopeo (mgr.24625) 9330 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:49:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.337263+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:49:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.339070+0000 mgr.gibba032.whopeo (mgr.24625) 9331 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:49:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.339277+0000 mgr.gibba032.whopeo (mgr.24625) 9332 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:49:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.339450+0000 mon.a (mon.0) 3181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.341083+0000 mgr.gibba032.whopeo (mgr.24625) 9333 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:49:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.341170+0000 mgr.gibba032.whopeo (mgr.24625) 9334 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:49:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.341763+0000 mon.a (mon.0) 3182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.342843+0000 mgr.gibba032.whopeo (mgr.24625) 9335 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:49:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.343343+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.344260+0000 mgr.gibba032.whopeo (mgr.24625) 9336 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:49:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.344517+0000 mgr.gibba032.whopeo (mgr.24625) 9337 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:49:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.346911+0000 mgr.gibba032.whopeo (mgr.24625) 9338 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:49:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.347254+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.348298+0000 mgr.gibba032.whopeo (mgr.24625) 9339 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:49:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: audit 2021-03-10T16:49:47.350820+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.351463+0000 mgr.gibba032.whopeo (mgr.24625) 9340 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:49:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.351789+0000 mgr.gibba032.whopeo (mgr.24625) 9341 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:49:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cephadm 2021-03-10T16:49:47.352314+0000 mgr.gibba032.whopeo (mgr.24625) 9342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:48.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:48 gibba032 bash[24004]: cluster 2021-03-10T16:49:48.203967+0000 mgr.gibba032.whopeo (mgr.24625) 9343 : cluster [DBG] pgmap v5317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.331154+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.334117+0000 mgr.gibba032.whopeo (mgr.24625) 9329 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:49:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.337082+0000 mgr.gibba032.whopeo (mgr.24625) 9330 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:49:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.337263+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:49:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.339070+0000 mgr.gibba032.whopeo (mgr.24625) 9331 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:49:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.339277+0000 mgr.gibba032.whopeo (mgr.24625) 9332 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:49:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.339450+0000 mon.a (mon.0) 3181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.341083+0000 mgr.gibba032.whopeo (mgr.24625) 9333 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:49:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.341170+0000 mgr.gibba032.whopeo (mgr.24625) 9334 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:49:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.341763+0000 mon.a (mon.0) 3182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.342843+0000 mgr.gibba032.whopeo (mgr.24625) 9335 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:49:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.343343+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.344260+0000 mgr.gibba032.whopeo (mgr.24625) 9336 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:49:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.344517+0000 mgr.gibba032.whopeo (mgr.24625) 9337 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:49:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.346911+0000 mgr.gibba032.whopeo (mgr.24625) 9338 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:49:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.347254+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.348298+0000 mgr.gibba032.whopeo (mgr.24625) 9339 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: audit 2021-03-10T16:49:47.350820+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.351463+0000 mgr.gibba032.whopeo (mgr.24625) 9340 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.351789+0000 mgr.gibba032.whopeo (mgr.24625) 9341 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cephadm 2021-03-10T16:49:47.352314+0000 mgr.gibba032.whopeo (mgr.24625) 9342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[30791]: cluster 2021-03-10T16:49:48.203967+0000 mgr.gibba032.whopeo (mgr.24625) 9343 : cluster [DBG] pgmap v5317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.331154+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:49:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.334117+0000 mgr.gibba032.whopeo (mgr.24625) 9329 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:49:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.337082+0000 mgr.gibba032.whopeo (mgr.24625) 9330 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:49:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.337263+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:49:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.339070+0000 mgr.gibba032.whopeo (mgr.24625) 9331 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:49:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.339277+0000 mgr.gibba032.whopeo (mgr.24625) 9332 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:49:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.339450+0000 mon.a (mon.0) 3181 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.341083+0000 mgr.gibba032.whopeo (mgr.24625) 9333 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:49:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.341170+0000 mgr.gibba032.whopeo (mgr.24625) 9334 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:49:48.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.341763+0000 mon.a (mon.0) 3182 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.342843+0000 mgr.gibba032.whopeo (mgr.24625) 9335 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:49:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.343343+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.344260+0000 mgr.gibba032.whopeo (mgr.24625) 9336 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:49:48.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.344517+0000 mgr.gibba032.whopeo (mgr.24625) 9337 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:49:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.346911+0000 mgr.gibba032.whopeo (mgr.24625) 9338 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:49:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.347254+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.348298+0000 mgr.gibba032.whopeo (mgr.24625) 9339 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:49:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: audit 2021-03-10T16:49:47.350820+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:49:48.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.351463+0000 mgr.gibba032.whopeo (mgr.24625) 9340 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:49:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.351789+0000 mgr.gibba032.whopeo (mgr.24625) 9341 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:49:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cephadm 2021-03-10T16:49:47.352314+0000 mgr.gibba032.whopeo (mgr.24625) 9342 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:49:48.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:48 gibba011 bash[31774]: cluster 2021-03-10T16:49:48.203967+0000 mgr.gibba032.whopeo (mgr.24625) 9343 : cluster [DBG] pgmap v5317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:51.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:51 gibba032 bash[24004]: cluster 2021-03-10T16:49:50.205473+0000 mgr.gibba032.whopeo (mgr.24625) 9344 : cluster [DBG] pgmap v5318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:51.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:51 gibba011 bash[30791]: cluster 2021-03-10T16:49:50.205473+0000 mgr.gibba032.whopeo (mgr.24625) 9344 : cluster [DBG] pgmap v5318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:51.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:51 gibba011 bash[31774]: cluster 2021-03-10T16:49:50.205473+0000 mgr.gibba032.whopeo (mgr.24625) 9344 : cluster [DBG] pgmap v5318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:52.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:52 gibba011 bash[30791]: cluster 2021-03-10T16:49:52.207000+0000 mgr.gibba032.whopeo (mgr.24625) 9345 : cluster [DBG] pgmap v5319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:52.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:52 gibba011 bash[31774]: cluster 2021-03-10T16:49:52.207000+0000 mgr.gibba032.whopeo (mgr.24625) 9345 : cluster [DBG] pgmap v5319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:52.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:52 gibba032 bash[24004]: cluster 2021-03-10T16:49:52.207000+0000 mgr.gibba032.whopeo (mgr.24625) 9345 : cluster [DBG] pgmap v5319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:54.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:54 gibba011 bash[30791]: cluster 2021-03-10T16:49:54.208474+0000 mgr.gibba032.whopeo (mgr.24625) 9346 : cluster [DBG] pgmap v5320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:54.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:54 gibba011 bash[31774]: cluster 2021-03-10T16:49:54.208474+0000 mgr.gibba032.whopeo (mgr.24625) 9346 : cluster [DBG] pgmap v5320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:54.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:54 gibba032 bash[24004]: cluster 2021-03-10T16:49:54.208474+0000 mgr.gibba032.whopeo (mgr.24625) 9346 : cluster [DBG] pgmap v5320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:56.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:56 gibba011 bash[30791]: cluster 2021-03-10T16:49:56.210134+0000 mgr.gibba032.whopeo (mgr.24625) 9347 : cluster [DBG] pgmap v5321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:56 gibba011 bash[31774]: cluster 2021-03-10T16:49:56.210134+0000 mgr.gibba032.whopeo (mgr.24625) 9347 : cluster [DBG] pgmap v5321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:56.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:56 gibba032 bash[24004]: cluster 2021-03-10T16:49:56.210134+0000 mgr.gibba032.whopeo (mgr.24625) 9347 : cluster [DBG] pgmap v5321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:57.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:57 gibba011 bash[30791]: audit 2021-03-10T16:49:57.192145+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:49:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:57 gibba011 bash[30791]: audit 2021-03-10T16:49:57.192805+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:49:57.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:57 gibba011 bash[31774]: audit 2021-03-10T16:49:57.192145+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:49:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:57 gibba011 bash[31774]: audit 2021-03-10T16:49:57.192805+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:49:57.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:57 gibba032 bash[24004]: audit 2021-03-10T16:49:57.192145+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:49:57.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:57 gibba032 bash[24004]: audit 2021-03-10T16:49:57.192805+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:49:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[30791]: audit 2021-03-10T16:49:57.330976+0000 mon.a (mon.0) 3188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:49:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[30791]: audit 2021-03-10T16:49:57.331870+0000 mon.a (mon.0) 3189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[30791]: audit 2021-03-10T16:49:57.332588+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:49:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[30791]: audit 2021-03-10T16:49:57.333306+0000 mon.a (mon.0) 3191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[30791]: cluster 2021-03-10T16:49:58.211955+0000 mgr.gibba032.whopeo (mgr.24625) 9348 : cluster [DBG] pgmap v5322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:58.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[31774]: audit 2021-03-10T16:49:57.330976+0000 mon.a (mon.0) 3188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:49:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[31774]: audit 2021-03-10T16:49:57.331870+0000 mon.a (mon.0) 3189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[31774]: audit 2021-03-10T16:49:57.332588+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:49:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[31774]: audit 2021-03-10T16:49:57.333306+0000 mon.a (mon.0) 3191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:49:58 gibba011 bash[31774]: cluster 2021-03-10T16:49:58.211955+0000 mgr.gibba032.whopeo (mgr.24625) 9348 : cluster [DBG] pgmap v5322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:49:58.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:58 gibba032 bash[24004]: audit 2021-03-10T16:49:57.330976+0000 mon.a (mon.0) 3188 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:49:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:58 gibba032 bash[24004]: audit 2021-03-10T16:49:57.331870+0000 mon.a (mon.0) 3189 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:58 gibba032 bash[24004]: audit 2021-03-10T16:49:57.332588+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:49:58.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:58 gibba032 bash[24004]: audit 2021-03-10T16:49:57.333306+0000 mon.a (mon.0) 3191 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T16:49:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:49:58 gibba032 bash[24004]: cluster 2021-03-10T16:49:58.211955+0000 mgr.gibba032.whopeo (mgr.24625) 9348 : cluster [DBG] pgmap v5322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:00 gibba011 bash[30791]: debug 2021-03-10T16:49:59.996+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:50:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:00 gibba011 bash[30791]: cluster 2021-03-10T16:50:00.000351+0000 mon.a (mon.0) 3192 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:50:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:00 gibba011 bash[31774]: cluster 2021-03-10T16:50:00.000351+0000 mon.a (mon.0) 3192 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:50:00.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:00 gibba032 bash[24004]: cluster 2021-03-10T16:50:00.000351+0000 mon.a (mon.0) 3192 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T16:50:01.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:01 gibba032 bash[24004]: cluster 2021-03-10T16:50:00.213307+0000 mgr.gibba032.whopeo (mgr.24625) 9349 : cluster [DBG] pgmap v5323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:01.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:01 gibba011 bash[30791]: cluster 2021-03-10T16:50:00.213307+0000 mgr.gibba032.whopeo (mgr.24625) 9349 : cluster [DBG] pgmap v5323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:01.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:01 gibba011 bash[31774]: cluster 2021-03-10T16:50:00.213307+0000 mgr.gibba032.whopeo (mgr.24625) 9349 : cluster [DBG] pgmap v5323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:02.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:02 gibba011 bash[30791]: cluster 2021-03-10T16:50:02.214948+0000 mgr.gibba032.whopeo (mgr.24625) 9350 : cluster [DBG] pgmap v5324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:02.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:02 gibba011 bash[31774]: cluster 2021-03-10T16:50:02.214948+0000 mgr.gibba032.whopeo (mgr.24625) 9350 : cluster [DBG] pgmap v5324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:02.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:02 gibba032 bash[24004]: cluster 2021-03-10T16:50:02.214948+0000 mgr.gibba032.whopeo (mgr.24625) 9350 : cluster [DBG] pgmap v5324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:03.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:03 gibba011 bash[30791]: cephadm 2021-03-10T16:50:02.368430+0000 mgr.gibba032.whopeo (mgr.24625) 9351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:03.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:03 gibba011 bash[31774]: cephadm 2021-03-10T16:50:02.368430+0000 mgr.gibba032.whopeo (mgr.24625) 9351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:03.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:03 gibba032 bash[24004]: cephadm 2021-03-10T16:50:02.368430+0000 mgr.gibba032.whopeo (mgr.24625) 9351 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:04.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:04 gibba011 bash[30791]: cluster 2021-03-10T16:50:04.216409+0000 mgr.gibba032.whopeo (mgr.24625) 9352 : cluster [DBG] pgmap v5325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:04.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:04 gibba011 bash[31774]: cluster 2021-03-10T16:50:04.216409+0000 mgr.gibba032.whopeo (mgr.24625) 9352 : cluster [DBG] pgmap v5325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:04.702 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:50:04.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:04 gibba032 bash[24004]: cluster 2021-03-10T16:50:04.216409+0000 mgr.gibba032.whopeo (mgr.24625) 9352 : cluster [DBG] pgmap v5325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:50:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:50:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:50:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:50:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:50:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:50:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:50:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:50:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:50:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:50:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:50:04.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:50:04.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:50:04.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:50:04.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:50:04.931 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:50:04.932 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:50:04.932 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:50:04.932 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:50:04.932 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:50:04.933 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:50:05.178 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:50:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:50:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:50:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:50:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:50:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:50:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:50:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:50:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:50:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:50:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:50:05.181 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:50:06.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:06 gibba032 bash[24004]: audit 2021-03-10T16:50:04.694913+0000 mgr.gibba032.whopeo (mgr.24625) 9353 : audit [DBG] from='client.50319 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:06 gibba032 bash[24004]: audit 2021-03-10T16:50:04.917172+0000 mgr.gibba032.whopeo (mgr.24625) 9354 : audit [DBG] from='client.50325 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:06 gibba032 bash[24004]: audit 2021-03-10T16:50:05.180952+0000 mon.a (mon.0) 3193 : audit [DBG] from='client.? 172.21.2.111:0/2645598128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[30791]: audit 2021-03-10T16:50:04.694913+0000 mgr.gibba032.whopeo (mgr.24625) 9353 : audit [DBG] from='client.50319 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[30791]: audit 2021-03-10T16:50:04.917172+0000 mgr.gibba032.whopeo (mgr.24625) 9354 : audit [DBG] from='client.50325 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[30791]: audit 2021-03-10T16:50:05.180952+0000 mon.a (mon.0) 3193 : audit [DBG] from='client.? 172.21.2.111:0/2645598128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[31774]: audit 2021-03-10T16:50:04.694913+0000 mgr.gibba032.whopeo (mgr.24625) 9353 : audit [DBG] from='client.50319 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[31774]: audit 2021-03-10T16:50:04.917172+0000 mgr.gibba032.whopeo (mgr.24625) 9354 : audit [DBG] from='client.50325 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:06 gibba011 bash[31774]: audit 2021-03-10T16:50:05.180952+0000 mon.a (mon.0) 3193 : audit [DBG] from='client.? 172.21.2.111:0/2645598128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:07.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:07 gibba032 bash[24004]: cluster 2021-03-10T16:50:06.217986+0000 mgr.gibba032.whopeo (mgr.24625) 9355 : cluster [DBG] pgmap v5326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:07.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:07 gibba011 bash[30791]: cluster 2021-03-10T16:50:06.217986+0000 mgr.gibba032.whopeo (mgr.24625) 9355 : cluster [DBG] pgmap v5326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:07.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:07 gibba011 bash[31774]: cluster 2021-03-10T16:50:06.217986+0000 mgr.gibba032.whopeo (mgr.24625) 9355 : cluster [DBG] pgmap v5326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:08.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:08 gibba011 bash[30791]: cluster 2021-03-10T16:50:08.219481+0000 mgr.gibba032.whopeo (mgr.24625) 9356 : cluster [DBG] pgmap v5327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:08.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:08 gibba011 bash[31774]: cluster 2021-03-10T16:50:08.219481+0000 mgr.gibba032.whopeo (mgr.24625) 9356 : cluster [DBG] pgmap v5327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:08.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:08 gibba032 bash[24004]: cluster 2021-03-10T16:50:08.219481+0000 mgr.gibba032.whopeo (mgr.24625) 9356 : cluster [DBG] pgmap v5327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:11.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:11 gibba032 bash[24004]: cluster 2021-03-10T16:50:10.221091+0000 mgr.gibba032.whopeo (mgr.24625) 9357 : cluster [DBG] pgmap v5328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:11.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:11 gibba011 bash[30791]: cluster 2021-03-10T16:50:10.221091+0000 mgr.gibba032.whopeo (mgr.24625) 9357 : cluster [DBG] pgmap v5328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:11.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:11 gibba011 bash[31774]: cluster 2021-03-10T16:50:10.221091+0000 mgr.gibba032.whopeo (mgr.24625) 9357 : cluster [DBG] pgmap v5328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:12 gibba011 bash[30791]: cluster 2021-03-10T16:50:12.222644+0000 mgr.gibba032.whopeo (mgr.24625) 9358 : cluster [DBG] pgmap v5329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:12 gibba011 bash[31774]: cluster 2021-03-10T16:50:12.222644+0000 mgr.gibba032.whopeo (mgr.24625) 9358 : cluster [DBG] pgmap v5329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:12.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:12 gibba032 bash[24004]: cluster 2021-03-10T16:50:12.222644+0000 mgr.gibba032.whopeo (mgr.24625) 9358 : cluster [DBG] pgmap v5329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:14.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:14 gibba011 bash[30791]: cluster 2021-03-10T16:50:14.223923+0000 mgr.gibba032.whopeo (mgr.24625) 9359 : cluster [DBG] pgmap v5330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:14 gibba011 bash[31774]: cluster 2021-03-10T16:50:14.223923+0000 mgr.gibba032.whopeo (mgr.24625) 9359 : cluster [DBG] pgmap v5330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:14.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:14 gibba032 bash[24004]: cluster 2021-03-10T16:50:14.223923+0000 mgr.gibba032.whopeo (mgr.24625) 9359 : cluster [DBG] pgmap v5330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:16.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:16 gibba011 bash[30791]: cluster 2021-03-10T16:50:16.225665+0000 mgr.gibba032.whopeo (mgr.24625) 9360 : cluster [DBG] pgmap v5331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:16.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:16 gibba011 bash[31774]: cluster 2021-03-10T16:50:16.225665+0000 mgr.gibba032.whopeo (mgr.24625) 9360 : cluster [DBG] pgmap v5331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:16.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:16 gibba032 bash[24004]: cluster 2021-03-10T16:50:16.225665+0000 mgr.gibba032.whopeo (mgr.24625) 9360 : cluster [DBG] pgmap v5331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:18.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:18 gibba011 bash[30791]: cephadm 2021-03-10T16:50:17.384529+0000 mgr.gibba032.whopeo (mgr.24625) 9361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:18 gibba011 bash[30791]: cluster 2021-03-10T16:50:18.227408+0000 mgr.gibba032.whopeo (mgr.24625) 9362 : cluster [DBG] pgmap v5332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:18 gibba011 bash[31774]: cephadm 2021-03-10T16:50:17.384529+0000 mgr.gibba032.whopeo (mgr.24625) 9361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:18 gibba011 bash[31774]: cluster 2021-03-10T16:50:18.227408+0000 mgr.gibba032.whopeo (mgr.24625) 9362 : cluster [DBG] pgmap v5332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:18.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:18 gibba032 bash[24004]: cephadm 2021-03-10T16:50:17.384529+0000 mgr.gibba032.whopeo (mgr.24625) 9361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:18.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:18 gibba032 bash[24004]: cluster 2021-03-10T16:50:18.227408+0000 mgr.gibba032.whopeo (mgr.24625) 9362 : cluster [DBG] pgmap v5332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:21.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:21 gibba032 bash[24004]: cluster 2021-03-10T16:50:20.229024+0000 mgr.gibba032.whopeo (mgr.24625) 9363 : cluster [DBG] pgmap v5333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:21.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:21 gibba011 bash[30791]: cluster 2021-03-10T16:50:20.229024+0000 mgr.gibba032.whopeo (mgr.24625) 9363 : cluster [DBG] pgmap v5333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:21.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:21 gibba011 bash[31774]: cluster 2021-03-10T16:50:20.229024+0000 mgr.gibba032.whopeo (mgr.24625) 9363 : cluster [DBG] pgmap v5333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:22.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:22 gibba011 bash[30791]: cluster 2021-03-10T16:50:22.230439+0000 mgr.gibba032.whopeo (mgr.24625) 9364 : cluster [DBG] pgmap v5334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:22 gibba011 bash[31774]: cluster 2021-03-10T16:50:22.230439+0000 mgr.gibba032.whopeo (mgr.24625) 9364 : cluster [DBG] pgmap v5334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:22.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:22 gibba032 bash[24004]: cluster 2021-03-10T16:50:22.230439+0000 mgr.gibba032.whopeo (mgr.24625) 9364 : cluster [DBG] pgmap v5334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:24 gibba011 bash[30791]: cluster 2021-03-10T16:50:24.231900+0000 mgr.gibba032.whopeo (mgr.24625) 9365 : cluster [DBG] pgmap v5335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:24 gibba011 bash[31774]: cluster 2021-03-10T16:50:24.231900+0000 mgr.gibba032.whopeo (mgr.24625) 9365 : cluster [DBG] pgmap v5335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:24.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:24 gibba032 bash[24004]: cluster 2021-03-10T16:50:24.231900+0000 mgr.gibba032.whopeo (mgr.24625) 9365 : cluster [DBG] pgmap v5335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:26.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:26 gibba011 bash[30791]: cluster 2021-03-10T16:50:26.233734+0000 mgr.gibba032.whopeo (mgr.24625) 9366 : cluster [DBG] pgmap v5336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:26 gibba011 bash[31774]: cluster 2021-03-10T16:50:26.233734+0000 mgr.gibba032.whopeo (mgr.24625) 9366 : cluster [DBG] pgmap v5336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:26.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:26 gibba032 bash[24004]: cluster 2021-03-10T16:50:26.233734+0000 mgr.gibba032.whopeo (mgr.24625) 9366 : cluster [DBG] pgmap v5336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:28.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:28 gibba011 bash[30791]: cluster 2021-03-10T16:50:28.235365+0000 mgr.gibba032.whopeo (mgr.24625) 9367 : cluster [DBG] pgmap v5337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:28 gibba011 bash[31774]: cluster 2021-03-10T16:50:28.235365+0000 mgr.gibba032.whopeo (mgr.24625) 9367 : cluster [DBG] pgmap v5337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:28.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:28 gibba032 bash[24004]: cluster 2021-03-10T16:50:28.235365+0000 mgr.gibba032.whopeo (mgr.24625) 9367 : cluster [DBG] pgmap v5337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:31.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:31 gibba032 bash[24004]: cluster 2021-03-10T16:50:30.236664+0000 mgr.gibba032.whopeo (mgr.24625) 9368 : cluster [DBG] pgmap v5338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:31.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:31 gibba011 bash[30791]: cluster 2021-03-10T16:50:30.236664+0000 mgr.gibba032.whopeo (mgr.24625) 9368 : cluster [DBG] pgmap v5338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:31.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:31 gibba011 bash[31774]: cluster 2021-03-10T16:50:30.236664+0000 mgr.gibba032.whopeo (mgr.24625) 9368 : cluster [DBG] pgmap v5338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:33.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:33 gibba011 bash[30791]: cluster 2021-03-10T16:50:32.238133+0000 mgr.gibba032.whopeo (mgr.24625) 9369 : cluster [DBG] pgmap v5339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:33 gibba011 bash[30791]: cephadm 2021-03-10T16:50:32.400603+0000 mgr.gibba032.whopeo (mgr.24625) 9370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:33.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:33 gibba011 bash[31774]: cluster 2021-03-10T16:50:32.238133+0000 mgr.gibba032.whopeo (mgr.24625) 9369 : cluster [DBG] pgmap v5339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:33.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:33 gibba011 bash[31774]: cephadm 2021-03-10T16:50:32.400603+0000 mgr.gibba032.whopeo (mgr.24625) 9370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:33.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:33 gibba032 bash[24004]: cluster 2021-03-10T16:50:32.238133+0000 mgr.gibba032.whopeo (mgr.24625) 9369 : cluster [DBG] pgmap v5339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:33.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:33 gibba032 bash[24004]: cephadm 2021-03-10T16:50:32.400603+0000 mgr.gibba032.whopeo (mgr.24625) 9370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:35.417 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:50:35.643 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:50:35.644 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 10m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:50:35.644 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:50:35.644 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:50:35.645 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:50:35.646 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:50:35.646 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:50:35.647 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:50:35.647 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:50:35.648 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 10m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:50:35.649 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:50:35.649 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:50:35.650 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:50:35.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:50:35.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:50:35.651 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:50:35.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:50:35.652 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:50:35.653 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:50:35.653 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:50:35.653 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:50:35.891 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:50:35.892 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:50:35.892 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:50:35.893 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:35.893 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:50:35.893 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:50:35.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:35.894 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:50:35.894 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:50:35.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:50:35.895 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:50:35.895 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:50:35.896 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:50:35.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:50:35.897 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:50:36.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:36 gibba032 bash[24004]: cluster 2021-03-10T16:50:34.239606+0000 mgr.gibba032.whopeo (mgr.24625) 9371 : cluster [DBG] pgmap v5340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:36.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:36 gibba032 bash[24004]: audit 2021-03-10T16:50:35.894258+0000 mon.a (mon.0) 3194 : audit [DBG] from='client.? 172.21.2.111:0/294544788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:36.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:36 gibba011 bash[30791]: cluster 2021-03-10T16:50:34.239606+0000 mgr.gibba032.whopeo (mgr.24625) 9371 : cluster [DBG] pgmap v5340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:36 gibba011 bash[30791]: audit 2021-03-10T16:50:35.894258+0000 mon.a (mon.0) 3194 : audit [DBG] from='client.? 172.21.2.111:0/294544788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:36.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:36 gibba011 bash[31774]: cluster 2021-03-10T16:50:34.239606+0000 mgr.gibba032.whopeo (mgr.24625) 9371 : cluster [DBG] pgmap v5340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:36.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:36 gibba011 bash[31774]: audit 2021-03-10T16:50:35.894258+0000 mon.a (mon.0) 3194 : audit [DBG] from='client.? 172.21.2.111:0/294544788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:37 gibba032 bash[24004]: audit 2021-03-10T16:50:35.409547+0000 mgr.gibba032.whopeo (mgr.24625) 9372 : audit [DBG] from='client.50337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:37.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:37 gibba032 bash[24004]: audit 2021-03-10T16:50:35.633402+0000 mgr.gibba032.whopeo (mgr.24625) 9373 : audit [DBG] from='client.50343 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:37.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:37 gibba011 bash[30791]: audit 2021-03-10T16:50:35.409547+0000 mgr.gibba032.whopeo (mgr.24625) 9372 : audit [DBG] from='client.50337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:37.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:37 gibba011 bash[30791]: audit 2021-03-10T16:50:35.633402+0000 mgr.gibba032.whopeo (mgr.24625) 9373 : audit [DBG] from='client.50343 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:37.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:37 gibba011 bash[31774]: audit 2021-03-10T16:50:35.409547+0000 mgr.gibba032.whopeo (mgr.24625) 9372 : audit [DBG] from='client.50337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:37.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:37 gibba011 bash[31774]: audit 2021-03-10T16:50:35.633402+0000 mgr.gibba032.whopeo (mgr.24625) 9373 : audit [DBG] from='client.50343 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:50:38.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:38 gibba032 bash[24004]: cluster 2021-03-10T16:50:36.241117+0000 mgr.gibba032.whopeo (mgr.24625) 9374 : cluster [DBG] pgmap v5341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:38 gibba011 bash[30791]: cluster 2021-03-10T16:50:36.241117+0000 mgr.gibba032.whopeo (mgr.24625) 9374 : cluster [DBG] pgmap v5341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:38 gibba011 bash[31774]: cluster 2021-03-10T16:50:36.241117+0000 mgr.gibba032.whopeo (mgr.24625) 9374 : cluster [DBG] pgmap v5341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:40.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:40 gibba032 bash[24004]: cluster 2021-03-10T16:50:38.242621+0000 mgr.gibba032.whopeo (mgr.24625) 9375 : cluster [DBG] pgmap v5342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:40 gibba011 bash[30791]: cluster 2021-03-10T16:50:38.242621+0000 mgr.gibba032.whopeo (mgr.24625) 9375 : cluster [DBG] pgmap v5342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:40 gibba011 bash[31774]: cluster 2021-03-10T16:50:38.242621+0000 mgr.gibba032.whopeo (mgr.24625) 9375 : cluster [DBG] pgmap v5342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:42 gibba032 bash[24004]: cluster 2021-03-10T16:50:40.244251+0000 mgr.gibba032.whopeo (mgr.24625) 9376 : cluster [DBG] pgmap v5343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:42.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:42 gibba011 bash[30791]: cluster 2021-03-10T16:50:40.244251+0000 mgr.gibba032.whopeo (mgr.24625) 9376 : cluster [DBG] pgmap v5343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:42.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:42 gibba011 bash[31774]: cluster 2021-03-10T16:50:40.244251+0000 mgr.gibba032.whopeo (mgr.24625) 9376 : cluster [DBG] pgmap v5343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:44.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:44 gibba032 bash[24004]: cluster 2021-03-10T16:50:42.245919+0000 mgr.gibba032.whopeo (mgr.24625) 9377 : cluster [DBG] pgmap v5344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:44.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:44 gibba011 bash[30791]: cluster 2021-03-10T16:50:42.245919+0000 mgr.gibba032.whopeo (mgr.24625) 9377 : cluster [DBG] pgmap v5344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:44.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:44 gibba011 bash[31774]: cluster 2021-03-10T16:50:42.245919+0000 mgr.gibba032.whopeo (mgr.24625) 9377 : cluster [DBG] pgmap v5344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:46 gibba032 bash[24004]: cluster 2021-03-10T16:50:44.246544+0000 mgr.gibba032.whopeo (mgr.24625) 9378 : cluster [DBG] pgmap v5345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:46 gibba011 bash[30791]: cluster 2021-03-10T16:50:44.246544+0000 mgr.gibba032.whopeo (mgr.24625) 9378 : cluster [DBG] pgmap v5345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:46.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:46 gibba011 bash[31774]: cluster 2021-03-10T16:50:44.246544+0000 mgr.gibba032.whopeo (mgr.24625) 9378 : cluster [DBG] pgmap v5345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:48.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:48 gibba032 bash[24004]: cluster 2021-03-10T16:50:46.247237+0000 mgr.gibba032.whopeo (mgr.24625) 9379 : cluster [DBG] pgmap v5346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:48.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:48 gibba032 bash[24004]: audit 2021-03-10T16:50:47.408330+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:50:48.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:48 gibba011 bash[30791]: cluster 2021-03-10T16:50:46.247237+0000 mgr.gibba032.whopeo (mgr.24625) 9379 : cluster [DBG] pgmap v5346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:48 gibba011 bash[30791]: audit 2021-03-10T16:50:47.408330+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:50:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:48 gibba011 bash[31774]: cluster 2021-03-10T16:50:46.247237+0000 mgr.gibba032.whopeo (mgr.24625) 9379 : cluster [DBG] pgmap v5346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:48 gibba011 bash[31774]: audit 2021-03-10T16:50:47.408330+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:50:49.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:49 gibba032 bash[24004]: cephadm 2021-03-10T16:50:47.405958+0000 mgr.gibba032.whopeo (mgr.24625) 9380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:50:49.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:49 gibba011 bash[30791]: cephadm 2021-03-10T16:50:47.405958+0000 mgr.gibba032.whopeo (mgr.24625) 9380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:50:49.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:49 gibba011 bash[31774]: cephadm 2021-03-10T16:50:47.405958+0000 mgr.gibba032.whopeo (mgr.24625) 9380 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:50:50.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:50 gibba032 bash[24004]: cluster 2021-03-10T16:50:48.247912+0000 mgr.gibba032.whopeo (mgr.24625) 9381 : cluster [DBG] pgmap v5347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:50.575 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:50 gibba011 bash[31774]: cluster 2021-03-10T16:50:48.247912+0000 mgr.gibba032.whopeo (mgr.24625) 9381 : cluster [DBG] pgmap v5347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:50.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:50 gibba011 bash[30791]: cluster 2021-03-10T16:50:48.247912+0000 mgr.gibba032.whopeo (mgr.24625) 9381 : cluster [DBG] pgmap v5347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:52.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: cluster 2021-03-10T16:50:50.249108+0000 mgr.gibba032.whopeo (mgr.24625) 9382 : cluster [DBG] pgmap v5348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:52.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.034284+0000 mgr.gibba032.whopeo (mgr.24625) 9383 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:50:52.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.034539+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.283902+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.291161+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:50:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.292409+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.293586+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.294944+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.297701+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[30791]: audit 2021-03-10T16:50:51.299402+0000 mon.a (mon.0) 3203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: cluster 2021-03-10T16:50:50.249108+0000 mgr.gibba032.whopeo (mgr.24625) 9382 : cluster [DBG] pgmap v5348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:52.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.034284+0000 mgr.gibba032.whopeo (mgr.24625) 9383 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:50:52.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.034539+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.283902+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.291161+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:50:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.292409+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.293586+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.294944+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.297701+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:52 gibba011 bash[31774]: audit 2021-03-10T16:50:51.299402+0000 mon.a (mon.0) 3203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: cluster 2021-03-10T16:50:50.249108+0000 mgr.gibba032.whopeo (mgr.24625) 9382 : cluster [DBG] pgmap v5348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:52.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.034284+0000 mgr.gibba032.whopeo (mgr.24625) 9383 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:50:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.034539+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.283902+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:50:52.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.291161+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:50:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.292409+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.293586+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.294944+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.297701+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:52 gibba032 bash[24004]: audit 2021-03-10T16:50:51.299402+0000 mon.a (mon.0) 3203 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:50:53.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.288604+0000 mgr.gibba032.whopeo (mgr.24625) 9384 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:50:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.290379+0000 mgr.gibba032.whopeo (mgr.24625) 9385 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:50:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.291524+0000 mgr.gibba032.whopeo (mgr.24625) 9386 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:50:53.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.291563+0000 mgr.gibba032.whopeo (mgr.24625) 9387 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:50:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.292822+0000 mgr.gibba032.whopeo (mgr.24625) 9388 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:50:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.292861+0000 mgr.gibba032.whopeo (mgr.24625) 9389 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:50:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.294190+0000 mgr.gibba032.whopeo (mgr.24625) 9390 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:50:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.295432+0000 mgr.gibba032.whopeo (mgr.24625) 9391 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:50:53.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.295471+0000 mgr.gibba032.whopeo (mgr.24625) 9392 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:50:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.296922+0000 mgr.gibba032.whopeo (mgr.24625) 9393 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:50:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.297276+0000 mgr.gibba032.whopeo (mgr.24625) 9394 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:50:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.298777+0000 mgr.gibba032.whopeo (mgr.24625) 9395 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:50:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.298819+0000 mgr.gibba032.whopeo (mgr.24625) 9396 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:50:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[30791]: cephadm 2021-03-10T16:50:51.298936+0000 mgr.gibba032.whopeo (mgr.24625) 9397 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.288604+0000 mgr.gibba032.whopeo (mgr.24625) 9384 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:50:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.290379+0000 mgr.gibba032.whopeo (mgr.24625) 9385 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:50:53.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.291524+0000 mgr.gibba032.whopeo (mgr.24625) 9386 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:50:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.291563+0000 mgr.gibba032.whopeo (mgr.24625) 9387 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:50:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.292822+0000 mgr.gibba032.whopeo (mgr.24625) 9388 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:50:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.292861+0000 mgr.gibba032.whopeo (mgr.24625) 9389 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:50:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.294190+0000 mgr.gibba032.whopeo (mgr.24625) 9390 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:50:53.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.295432+0000 mgr.gibba032.whopeo (mgr.24625) 9391 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:50:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.295471+0000 mgr.gibba032.whopeo (mgr.24625) 9392 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:50:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.296922+0000 mgr.gibba032.whopeo (mgr.24625) 9393 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:50:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.297276+0000 mgr.gibba032.whopeo (mgr.24625) 9394 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:50:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.298777+0000 mgr.gibba032.whopeo (mgr.24625) 9395 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:50:53.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.298819+0000 mgr.gibba032.whopeo (mgr.24625) 9396 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:50:53.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:53 gibba011 bash[31774]: cephadm 2021-03-10T16:50:51.298936+0000 mgr.gibba032.whopeo (mgr.24625) 9397 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:53.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.288604+0000 mgr.gibba032.whopeo (mgr.24625) 9384 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:50:53.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.290379+0000 mgr.gibba032.whopeo (mgr.24625) 9385 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:50:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.291524+0000 mgr.gibba032.whopeo (mgr.24625) 9386 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:50:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.291563+0000 mgr.gibba032.whopeo (mgr.24625) 9387 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:50:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.292822+0000 mgr.gibba032.whopeo (mgr.24625) 9388 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:50:53.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.292861+0000 mgr.gibba032.whopeo (mgr.24625) 9389 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:50:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.294190+0000 mgr.gibba032.whopeo (mgr.24625) 9390 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:50:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.295432+0000 mgr.gibba032.whopeo (mgr.24625) 9391 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:50:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.295471+0000 mgr.gibba032.whopeo (mgr.24625) 9392 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:50:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.296922+0000 mgr.gibba032.whopeo (mgr.24625) 9393 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:50:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.297276+0000 mgr.gibba032.whopeo (mgr.24625) 9394 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:50:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.298777+0000 mgr.gibba032.whopeo (mgr.24625) 9395 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:50:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.298819+0000 mgr.gibba032.whopeo (mgr.24625) 9396 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:50:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:53 gibba032 bash[24004]: cephadm 2021-03-10T16:50:51.298936+0000 mgr.gibba032.whopeo (mgr.24625) 9397 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:50:54.326 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:54 gibba011 bash[30791]: cluster 2021-03-10T16:50:52.250523+0000 mgr.gibba032.whopeo (mgr.24625) 9398 : cluster [DBG] pgmap v5349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:54.327 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:54 gibba011 bash[31774]: cluster 2021-03-10T16:50:52.250523+0000 mgr.gibba032.whopeo (mgr.24625) 9398 : cluster [DBG] pgmap v5349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:54.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:54 gibba032 bash[24004]: cluster 2021-03-10T16:50:52.250523+0000 mgr.gibba032.whopeo (mgr.24625) 9398 : cluster [DBG] pgmap v5349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:56 gibba032 bash[24004]: cluster 2021-03-10T16:50:54.251796+0000 mgr.gibba032.whopeo (mgr.24625) 9399 : cluster [DBG] pgmap v5350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:56.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:56 gibba011 bash[30791]: cluster 2021-03-10T16:50:54.251796+0000 mgr.gibba032.whopeo (mgr.24625) 9399 : cluster [DBG] pgmap v5350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:56 gibba011 bash[31774]: cluster 2021-03-10T16:50:54.251796+0000 mgr.gibba032.whopeo (mgr.24625) 9399 : cluster [DBG] pgmap v5350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: cluster 2021-03-10T16:50:56.253406+0000 mgr.gibba032.whopeo (mgr.24625) 9400 : cluster [DBG] pgmap v5351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.194844+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:50:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.194977+0000 mon.a (mon.0) 3205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:50:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.341686+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:50:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.342464+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:50:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.343137+0000 mon.a (mon.0) 3208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:50:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:50:58 gibba032 bash[24004]: audit 2021-03-10T16:50:57.343832+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:50:58.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: cluster 2021-03-10T16:50:56.253406+0000 mgr.gibba032.whopeo (mgr.24625) 9400 : cluster [DBG] pgmap v5351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.194844+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:50:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.194977+0000 mon.a (mon.0) 3205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:50:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.341686+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:50:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.342464+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:50:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.343137+0000 mon.a (mon.0) 3208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:50:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[30791]: audit 2021-03-10T16:50:57.343832+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:50:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: cluster 2021-03-10T16:50:56.253406+0000 mgr.gibba032.whopeo (mgr.24625) 9400 : cluster [DBG] pgmap v5351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:50:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.194844+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:50:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.194977+0000 mon.a (mon.0) 3205 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:50:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.341686+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:50:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.342464+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:50:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.343137+0000 mon.a (mon.0) 3208 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:50:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:50:58 gibba011 bash[31774]: audit 2021-03-10T16:50:57.343832+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T16:51:00.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:00 gibba032 bash[24004]: cluster 2021-03-10T16:50:58.255114+0000 mgr.gibba032.whopeo (mgr.24625) 9401 : cluster [DBG] pgmap v5352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:00.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:00 gibba011 bash[30791]: cluster 2021-03-10T16:50:58.255114+0000 mgr.gibba032.whopeo (mgr.24625) 9401 : cluster [DBG] pgmap v5352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:00.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:00 gibba011 bash[31774]: cluster 2021-03-10T16:50:58.255114+0000 mgr.gibba032.whopeo (mgr.24625) 9401 : cluster [DBG] pgmap v5352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:02 gibba032 bash[24004]: cluster 2021-03-10T16:51:00.256655+0000 mgr.gibba032.whopeo (mgr.24625) 9402 : cluster [DBG] pgmap v5353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:02.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:02 gibba011 bash[30791]: cluster 2021-03-10T16:51:00.256655+0000 mgr.gibba032.whopeo (mgr.24625) 9402 : cluster [DBG] pgmap v5353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:02 gibba011 bash[31774]: cluster 2021-03-10T16:51:00.256655+0000 mgr.gibba032.whopeo (mgr.24625) 9402 : cluster [DBG] pgmap v5353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:04.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:04 gibba032 bash[24004]: cluster 2021-03-10T16:51:02.257383+0000 mgr.gibba032.whopeo (mgr.24625) 9403 : cluster [DBG] pgmap v5354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:04.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:04 gibba011 bash[30791]: cluster 2021-03-10T16:51:02.257383+0000 mgr.gibba032.whopeo (mgr.24625) 9403 : cluster [DBG] pgmap v5354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:04.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:04 gibba011 bash[31774]: cluster 2021-03-10T16:51:02.257383+0000 mgr.gibba032.whopeo (mgr.24625) 9403 : cluster [DBG] pgmap v5354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:06.133 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:51:06.359 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:51:06.359 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 15s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:51:06.360 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:51:06.360 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:51:06.360 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:51:06.361 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:51:06.361 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:51:06.361 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:51:06.361 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:51:06.362 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 15s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:51:06.362 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:51:06.362 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:51:06.362 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:51:06.363 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:51:06.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:51:06.371 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:51:06.372 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:51:06.372 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:51:06.372 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:51:06.373 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:51:06.373 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 15s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:51:06.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:06 gibba032 bash[24004]: cluster 2021-03-10T16:51:04.258015+0000 mgr.gibba032.whopeo (mgr.24625) 9404 : cluster [DBG] pgmap v5355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:06 gibba011 bash[30791]: cluster 2021-03-10T16:51:04.258015+0000 mgr.gibba032.whopeo (mgr.24625) 9404 : cluster [DBG] pgmap v5355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:06.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:06 gibba011 bash[31774]: cluster 2021-03-10T16:51:04.258015+0000 mgr.gibba032.whopeo (mgr.24625) 9404 : cluster [DBG] pgmap v5355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:06.606 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:51:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:51:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:51:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:51:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:51:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:51:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:51:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:51:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:51:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:51:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:51:06.610 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:51:07.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:07 gibba032 bash[24004]: audit 2021-03-10T16:51:06.124340+0000 mgr.gibba032.whopeo (mgr.24625) 9405 : audit [DBG] from='client.60337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:07.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:07 gibba032 bash[24004]: audit 2021-03-10T16:51:06.608945+0000 mon.c (mon.1) 181 : audit [DBG] from='client.? 172.21.2.111:0/3741784310' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:07.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:07 gibba011 bash[30791]: audit 2021-03-10T16:51:06.124340+0000 mgr.gibba032.whopeo (mgr.24625) 9405 : audit [DBG] from='client.60337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:07.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:07 gibba011 bash[30791]: audit 2021-03-10T16:51:06.608945+0000 mon.c (mon.1) 181 : audit [DBG] from='client.? 172.21.2.111:0/3741784310' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:07.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:07 gibba011 bash[31774]: audit 2021-03-10T16:51:06.124340+0000 mgr.gibba032.whopeo (mgr.24625) 9405 : audit [DBG] from='client.60337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:07.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:07 gibba011 bash[31774]: audit 2021-03-10T16:51:06.608945+0000 mon.c (mon.1) 181 : audit [DBG] from='client.? 172.21.2.111:0/3741784310' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:08.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:08 gibba032 bash[24004]: cluster 2021-03-10T16:51:06.259403+0000 mgr.gibba032.whopeo (mgr.24625) 9406 : cluster [DBG] pgmap v5356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:08.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:08 gibba032 bash[24004]: cephadm 2021-03-10T16:51:06.312859+0000 mgr.gibba032.whopeo (mgr.24625) 9407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:08.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:08 gibba032 bash[24004]: audit 2021-03-10T16:51:06.347522+0000 mgr.gibba032.whopeo (mgr.24625) 9408 : audit [DBG] from='client.50361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:08.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[30791]: cluster 2021-03-10T16:51:06.259403+0000 mgr.gibba032.whopeo (mgr.24625) 9406 : cluster [DBG] pgmap v5356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[30791]: cephadm 2021-03-10T16:51:06.312859+0000 mgr.gibba032.whopeo (mgr.24625) 9407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[30791]: audit 2021-03-10T16:51:06.347522+0000 mgr.gibba032.whopeo (mgr.24625) 9408 : audit [DBG] from='client.50361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:08.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[31774]: cluster 2021-03-10T16:51:06.259403+0000 mgr.gibba032.whopeo (mgr.24625) 9406 : cluster [DBG] pgmap v5356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[31774]: cephadm 2021-03-10T16:51:06.312859+0000 mgr.gibba032.whopeo (mgr.24625) 9407 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:08 gibba011 bash[31774]: audit 2021-03-10T16:51:06.347522+0000 mgr.gibba032.whopeo (mgr.24625) 9408 : audit [DBG] from='client.50361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:10.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:10 gibba032 bash[24004]: cluster 2021-03-10T16:51:08.261065+0000 mgr.gibba032.whopeo (mgr.24625) 9409 : cluster [DBG] pgmap v5357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:10.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:10 gibba011 bash[30791]: cluster 2021-03-10T16:51:08.261065+0000 mgr.gibba032.whopeo (mgr.24625) 9409 : cluster [DBG] pgmap v5357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:10.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:10 gibba011 bash[31774]: cluster 2021-03-10T16:51:08.261065+0000 mgr.gibba032.whopeo (mgr.24625) 9409 : cluster [DBG] pgmap v5357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:12.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:12 gibba032 bash[24004]: cluster 2021-03-10T16:51:10.262407+0000 mgr.gibba032.whopeo (mgr.24625) 9410 : cluster [DBG] pgmap v5358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:12.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:12 gibba011 bash[30791]: cluster 2021-03-10T16:51:10.262407+0000 mgr.gibba032.whopeo (mgr.24625) 9410 : cluster [DBG] pgmap v5358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:12 gibba011 bash[31774]: cluster 2021-03-10T16:51:10.262407+0000 mgr.gibba032.whopeo (mgr.24625) 9410 : cluster [DBG] pgmap v5358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:14.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:14 gibba011 bash[30791]: cluster 2021-03-10T16:51:12.263188+0000 mgr.gibba032.whopeo (mgr.24625) 9411 : cluster [DBG] pgmap v5359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:14 gibba011 bash[31774]: cluster 2021-03-10T16:51:12.263188+0000 mgr.gibba032.whopeo (mgr.24625) 9411 : cluster [DBG] pgmap v5359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:14.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:14 gibba032 bash[24004]: cluster 2021-03-10T16:51:12.263188+0000 mgr.gibba032.whopeo (mgr.24625) 9411 : cluster [DBG] pgmap v5359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:16.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:16 gibba032 bash[24004]: cluster 2021-03-10T16:51:14.263567+0000 mgr.gibba032.whopeo (mgr.24625) 9412 : cluster [DBG] pgmap v5360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:16.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:16 gibba011 bash[30791]: cluster 2021-03-10T16:51:14.263567+0000 mgr.gibba032.whopeo (mgr.24625) 9412 : cluster [DBG] pgmap v5360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:16.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:16 gibba011 bash[31774]: cluster 2021-03-10T16:51:14.263567+0000 mgr.gibba032.whopeo (mgr.24625) 9412 : cluster [DBG] pgmap v5360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:18.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:18 gibba032 bash[24004]: cluster 2021-03-10T16:51:16.264964+0000 mgr.gibba032.whopeo (mgr.24625) 9413 : cluster [DBG] pgmap v5361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:18 gibba011 bash[30791]: cluster 2021-03-10T16:51:16.264964+0000 mgr.gibba032.whopeo (mgr.24625) 9413 : cluster [DBG] pgmap v5361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:18 gibba011 bash[31774]: cluster 2021-03-10T16:51:16.264964+0000 mgr.gibba032.whopeo (mgr.24625) 9413 : cluster [DBG] pgmap v5361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:20.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:20 gibba032 bash[24004]: cluster 2021-03-10T16:51:18.266461+0000 mgr.gibba032.whopeo (mgr.24625) 9414 : cluster [DBG] pgmap v5362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:20.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:20 gibba011 bash[30791]: cluster 2021-03-10T16:51:18.266461+0000 mgr.gibba032.whopeo (mgr.24625) 9414 : cluster [DBG] pgmap v5362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:20.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:20 gibba011 bash[31774]: cluster 2021-03-10T16:51:18.266461+0000 mgr.gibba032.whopeo (mgr.24625) 9414 : cluster [DBG] pgmap v5362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:22.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:22 gibba032 bash[24004]: cluster 2021-03-10T16:51:20.298458+0000 mgr.gibba032.whopeo (mgr.24625) 9415 : cluster [DBG] pgmap v5363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:22 gibba011 bash[30791]: cluster 2021-03-10T16:51:20.298458+0000 mgr.gibba032.whopeo (mgr.24625) 9415 : cluster [DBG] pgmap v5363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:22 gibba011 bash[31774]: cluster 2021-03-10T16:51:20.298458+0000 mgr.gibba032.whopeo (mgr.24625) 9415 : cluster [DBG] pgmap v5363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:23.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:23 gibba032 bash[24004]: cephadm 2021-03-10T16:51:21.328704+0000 mgr.gibba032.whopeo (mgr.24625) 9416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:23.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:23 gibba011 bash[30791]: cephadm 2021-03-10T16:51:21.328704+0000 mgr.gibba032.whopeo (mgr.24625) 9416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:23.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:23 gibba011 bash[31774]: cephadm 2021-03-10T16:51:21.328704+0000 mgr.gibba032.whopeo (mgr.24625) 9416 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:24.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:24 gibba032 bash[24004]: cluster 2021-03-10T16:51:22.299951+0000 mgr.gibba032.whopeo (mgr.24625) 9417 : cluster [DBG] pgmap v5364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:24.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:24 gibba011 bash[30791]: cluster 2021-03-10T16:51:22.299951+0000 mgr.gibba032.whopeo (mgr.24625) 9417 : cluster [DBG] pgmap v5364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:24 gibba011 bash[31774]: cluster 2021-03-10T16:51:22.299951+0000 mgr.gibba032.whopeo (mgr.24625) 9417 : cluster [DBG] pgmap v5364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:26.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:26 gibba032 bash[24004]: cluster 2021-03-10T16:51:24.300812+0000 mgr.gibba032.whopeo (mgr.24625) 9418 : cluster [DBG] pgmap v5365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:26.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:26 gibba011 bash[30791]: cluster 2021-03-10T16:51:24.300812+0000 mgr.gibba032.whopeo (mgr.24625) 9418 : cluster [DBG] pgmap v5365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:26 gibba011 bash[31774]: cluster 2021-03-10T16:51:24.300812+0000 mgr.gibba032.whopeo (mgr.24625) 9418 : cluster [DBG] pgmap v5365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:28.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:28 gibba032 bash[24004]: cluster 2021-03-10T16:51:26.301911+0000 mgr.gibba032.whopeo (mgr.24625) 9419 : cluster [DBG] pgmap v5366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:28.576 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:28 gibba011 bash[31774]: cluster 2021-03-10T16:51:26.301911+0000 mgr.gibba032.whopeo (mgr.24625) 9419 : cluster [DBG] pgmap v5366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:28 gibba011 bash[30791]: cluster 2021-03-10T16:51:26.301911+0000 mgr.gibba032.whopeo (mgr.24625) 9419 : cluster [DBG] pgmap v5366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:30.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:30 gibba032 bash[24004]: cluster 2021-03-10T16:51:28.303472+0000 mgr.gibba032.whopeo (mgr.24625) 9420 : cluster [DBG] pgmap v5367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:30 gibba011 bash[30791]: cluster 2021-03-10T16:51:28.303472+0000 mgr.gibba032.whopeo (mgr.24625) 9420 : cluster [DBG] pgmap v5367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:30.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:30 gibba011 bash[31774]: cluster 2021-03-10T16:51:28.303472+0000 mgr.gibba032.whopeo (mgr.24625) 9420 : cluster [DBG] pgmap v5367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:32.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:32 gibba011 bash[30791]: cluster 2021-03-10T16:51:30.305075+0000 mgr.gibba032.whopeo (mgr.24625) 9421 : cluster [DBG] pgmap v5368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:32.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:32 gibba011 bash[31774]: cluster 2021-03-10T16:51:30.305075+0000 mgr.gibba032.whopeo (mgr.24625) 9421 : cluster [DBG] pgmap v5368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:32.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:32 gibba032 bash[24004]: cluster 2021-03-10T16:51:30.305075+0000 mgr.gibba032.whopeo (mgr.24625) 9421 : cluster [DBG] pgmap v5368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:34.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:34 gibba011 bash[30791]: cluster 2021-03-10T16:51:32.305920+0000 mgr.gibba032.whopeo (mgr.24625) 9422 : cluster [DBG] pgmap v5369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:34.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:34 gibba011 bash[31774]: cluster 2021-03-10T16:51:32.305920+0000 mgr.gibba032.whopeo (mgr.24625) 9422 : cluster [DBG] pgmap v5369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:34.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:34 gibba032 bash[24004]: cluster 2021-03-10T16:51:32.305920+0000 mgr.gibba032.whopeo (mgr.24625) 9422 : cluster [DBG] pgmap v5369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:36.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:36 gibba032 bash[24004]: cluster 2021-03-10T16:51:34.306532+0000 mgr.gibba032.whopeo (mgr.24625) 9423 : cluster [DBG] pgmap v5370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:36.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:36 gibba011 bash[30791]: cluster 2021-03-10T16:51:34.306532+0000 mgr.gibba032.whopeo (mgr.24625) 9423 : cluster [DBG] pgmap v5370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:36.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:36 gibba011 bash[31774]: cluster 2021-03-10T16:51:34.306532+0000 mgr.gibba032.whopeo (mgr.24625) 9423 : cluster [DBG] pgmap v5370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:36.851 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:51:37.075 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:51:37.076 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 46s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:51:37.076 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:51:37.076 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:51:37.076 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:51:37.077 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:51:37.077 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:51:37.077 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:51:37.077 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:51:37.078 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 46s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:51:37.078 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:51:37.078 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:51:37.078 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:51:37.079 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:51:37.079 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:51:37.079 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:51:37.079 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:51:37.080 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:51:37.080 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:51:37.080 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:51:37.080 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 46s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:51:37.329 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:51:37.330 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:51:37.330 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:51:37.330 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:37.330 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:51:37.331 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:51:37.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:37.331 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:51:37.331 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:51:37.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:51:37.332 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:51:37.332 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:51:37.332 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:51:37.332 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:51:37.333 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:51:38.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:38 gibba032 bash[24004]: cluster 2021-03-10T16:51:36.307923+0000 mgr.gibba032.whopeo (mgr.24625) 9424 : cluster [DBG] pgmap v5371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:38.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:38 gibba032 bash[24004]: cephadm 2021-03-10T16:51:36.344685+0000 mgr.gibba032.whopeo (mgr.24625) 9425 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:38.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:38 gibba032 bash[24004]: audit 2021-03-10T16:51:36.842926+0000 mgr.gibba032.whopeo (mgr.24625) 9426 : audit [DBG] from='client.50373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:38 gibba032 bash[24004]: audit 2021-03-10T16:51:37.065756+0000 mgr.gibba032.whopeo (mgr.24625) 9427 : audit [DBG] from='client.60361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:38 gibba032 bash[24004]: audit 2021-03-10T16:51:37.332176+0000 mon.a (mon.0) 3210 : audit [DBG] from='client.? 172.21.2.111:0/3453693355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[30791]: cluster 2021-03-10T16:51:36.307923+0000 mgr.gibba032.whopeo (mgr.24625) 9424 : cluster [DBG] pgmap v5371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[30791]: cephadm 2021-03-10T16:51:36.344685+0000 mgr.gibba032.whopeo (mgr.24625) 9425 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[30791]: audit 2021-03-10T16:51:36.842926+0000 mgr.gibba032.whopeo (mgr.24625) 9426 : audit [DBG] from='client.50373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[30791]: audit 2021-03-10T16:51:37.065756+0000 mgr.gibba032.whopeo (mgr.24625) 9427 : audit [DBG] from='client.60361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[30791]: audit 2021-03-10T16:51:37.332176+0000 mon.a (mon.0) 3210 : audit [DBG] from='client.? 172.21.2.111:0/3453693355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[31774]: cluster 2021-03-10T16:51:36.307923+0000 mgr.gibba032.whopeo (mgr.24625) 9424 : cluster [DBG] pgmap v5371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[31774]: cephadm 2021-03-10T16:51:36.344685+0000 mgr.gibba032.whopeo (mgr.24625) 9425 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:51:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[31774]: audit 2021-03-10T16:51:36.842926+0000 mgr.gibba032.whopeo (mgr.24625) 9426 : audit [DBG] from='client.50373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[31774]: audit 2021-03-10T16:51:37.065756+0000 mgr.gibba032.whopeo (mgr.24625) 9427 : audit [DBG] from='client.60361 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:51:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:38 gibba011 bash[31774]: audit 2021-03-10T16:51:37.332176+0000 mon.a (mon.0) 3210 : audit [DBG] from='client.? 172.21.2.111:0/3453693355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:51:40.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:40 gibba032 bash[24004]: cluster 2021-03-10T16:51:38.309678+0000 mgr.gibba032.whopeo (mgr.24625) 9428 : cluster [DBG] pgmap v5372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:40.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:40 gibba011 bash[30791]: cluster 2021-03-10T16:51:38.309678+0000 mgr.gibba032.whopeo (mgr.24625) 9428 : cluster [DBG] pgmap v5372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:40.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:40 gibba011 bash[31774]: cluster 2021-03-10T16:51:38.309678+0000 mgr.gibba032.whopeo (mgr.24625) 9428 : cluster [DBG] pgmap v5372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:42 gibba032 bash[24004]: cluster 2021-03-10T16:51:40.311254+0000 mgr.gibba032.whopeo (mgr.24625) 9429 : cluster [DBG] pgmap v5373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:42.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:42 gibba011 bash[30791]: cluster 2021-03-10T16:51:40.311254+0000 mgr.gibba032.whopeo (mgr.24625) 9429 : cluster [DBG] pgmap v5373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:42.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:42 gibba011 bash[31774]: cluster 2021-03-10T16:51:40.311254+0000 mgr.gibba032.whopeo (mgr.24625) 9429 : cluster [DBG] pgmap v5373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:44.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:44 gibba032 bash[24004]: cluster 2021-03-10T16:51:42.312677+0000 mgr.gibba032.whopeo (mgr.24625) 9430 : cluster [DBG] pgmap v5374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:44 gibba011 bash[30791]: cluster 2021-03-10T16:51:42.312677+0000 mgr.gibba032.whopeo (mgr.24625) 9430 : cluster [DBG] pgmap v5374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:44 gibba011 bash[31774]: cluster 2021-03-10T16:51:42.312677+0000 mgr.gibba032.whopeo (mgr.24625) 9430 : cluster [DBG] pgmap v5374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:46.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:46 gibba032 bash[24004]: cluster 2021-03-10T16:51:44.313596+0000 mgr.gibba032.whopeo (mgr.24625) 9431 : cluster [DBG] pgmap v5375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:46.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:46 gibba011 bash[30791]: cluster 2021-03-10T16:51:44.313596+0000 mgr.gibba032.whopeo (mgr.24625) 9431 : cluster [DBG] pgmap v5375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:46.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:46 gibba011 bash[31774]: cluster 2021-03-10T16:51:44.313596+0000 mgr.gibba032.whopeo (mgr.24625) 9431 : cluster [DBG] pgmap v5375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:48.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:48 gibba032 bash[24004]: cluster 2021-03-10T16:51:46.315043+0000 mgr.gibba032.whopeo (mgr.24625) 9432 : cluster [DBG] pgmap v5376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:48.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:48 gibba011 bash[30791]: cluster 2021-03-10T16:51:46.315043+0000 mgr.gibba032.whopeo (mgr.24625) 9432 : cluster [DBG] pgmap v5376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:48 gibba011 bash[31774]: cluster 2021-03-10T16:51:46.315043+0000 mgr.gibba032.whopeo (mgr.24625) 9432 : cluster [DBG] pgmap v5376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:50.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:50 gibba032 bash[24004]: cluster 2021-03-10T16:51:48.316003+0000 mgr.gibba032.whopeo (mgr.24625) 9433 : cluster [DBG] pgmap v5377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:50.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:50 gibba011 bash[30791]: cluster 2021-03-10T16:51:48.316003+0000 mgr.gibba032.whopeo (mgr.24625) 9433 : cluster [DBG] pgmap v5377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:50.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:50 gibba011 bash[31774]: cluster 2021-03-10T16:51:48.316003+0000 mgr.gibba032.whopeo (mgr.24625) 9433 : cluster [DBG] pgmap v5377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:52.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:52 gibba032 bash[24004]: cluster 2021-03-10T16:51:50.317537+0000 mgr.gibba032.whopeo (mgr.24625) 9434 : cluster [DBG] pgmap v5378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:52.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:52 gibba032 bash[24004]: audit 2021-03-10T16:51:51.364976+0000 mon.a (mon.0) 3211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:51:52.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:52 gibba011 bash[30791]: cluster 2021-03-10T16:51:50.317537+0000 mgr.gibba032.whopeo (mgr.24625) 9434 : cluster [DBG] pgmap v5378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:52 gibba011 bash[30791]: audit 2021-03-10T16:51:51.364976+0000 mon.a (mon.0) 3211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:51:52.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:52 gibba011 bash[31774]: cluster 2021-03-10T16:51:50.317537+0000 mgr.gibba032.whopeo (mgr.24625) 9434 : cluster [DBG] pgmap v5378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:52 gibba011 bash[31774]: audit 2021-03-10T16:51:51.364976+0000 mon.a (mon.0) 3211 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:51:53.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:53 gibba011 bash[30791]: cephadm 2021-03-10T16:51:51.361495+0000 mgr.gibba032.whopeo (mgr.24625) 9435 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:51:53.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:53 gibba011 bash[31774]: cephadm 2021-03-10T16:51:51.361495+0000 mgr.gibba032.whopeo (mgr.24625) 9435 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:51:53.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:53 gibba032 bash[24004]: cephadm 2021-03-10T16:51:51.361495+0000 mgr.gibba032.whopeo (mgr.24625) 9435 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:51:54.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:54 gibba011 bash[30791]: cluster 2021-03-10T16:51:52.318466+0000 mgr.gibba032.whopeo (mgr.24625) 9436 : cluster [DBG] pgmap v5379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:54.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:54 gibba011 bash[31774]: cluster 2021-03-10T16:51:52.318466+0000 mgr.gibba032.whopeo (mgr.24625) 9436 : cluster [DBG] pgmap v5379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:54.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:54 gibba032 bash[24004]: cluster 2021-03-10T16:51:52.318466+0000 mgr.gibba032.whopeo (mgr.24625) 9436 : cluster [DBG] pgmap v5379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:56.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:56 gibba032 bash[24004]: cluster 2021-03-10T16:51:54.318888+0000 mgr.gibba032.whopeo (mgr.24625) 9437 : cluster [DBG] pgmap v5380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:56.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:56 gibba032 bash[24004]: audit 2021-03-10T16:51:55.038856+0000 mon.a (mon.0) 3212 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:51:56.307 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:56 gibba032 bash[24004]: audit 2021-03-10T16:51:55.040892+0000 mon.a (mon.0) 3213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:51:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[30791]: cluster 2021-03-10T16:51:54.318888+0000 mgr.gibba032.whopeo (mgr.24625) 9437 : cluster [DBG] pgmap v5380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[30791]: audit 2021-03-10T16:51:55.038856+0000 mon.a (mon.0) 3212 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:51:56.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[30791]: audit 2021-03-10T16:51:55.040892+0000 mon.a (mon.0) 3213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:51:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[31774]: cluster 2021-03-10T16:51:54.318888+0000 mgr.gibba032.whopeo (mgr.24625) 9437 : cluster [DBG] pgmap v5380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[31774]: audit 2021-03-10T16:51:55.038856+0000 mon.a (mon.0) 3212 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:51:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:56 gibba011 bash[31774]: audit 2021-03-10T16:51:55.040892+0000 mon.a (mon.0) 3213 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T16:51:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: cluster 2021-03-10T16:51:56.319460+0000 mgr.gibba032.whopeo (mgr.24625) 9438 : cluster [DBG] pgmap v5381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.201815+0000 mon.a (mon.0) 3214 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:51:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.202508+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:51:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.350563+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:51:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.351287+0000 mon.a (mon.0) 3217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.352009+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[30791]: audit 2021-03-10T16:51:57.352693+0000 mon.a (mon.0) 3219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:51:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: cluster 2021-03-10T16:51:56.319460+0000 mgr.gibba032.whopeo (mgr.24625) 9438 : cluster [DBG] pgmap v5381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.201815+0000 mon.a (mon.0) 3214 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:51:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.202508+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:51:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.350563+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:51:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.351287+0000 mon.a (mon.0) 3217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.352009+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:51:58 gibba011 bash[31774]: audit 2021-03-10T16:51:57.352693+0000 mon.a (mon.0) 3219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:51:58.504 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: cluster 2021-03-10T16:51:56.319460+0000 mgr.gibba032.whopeo (mgr.24625) 9438 : cluster [DBG] pgmap v5381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:51:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.201815+0000 mon.a (mon.0) 3214 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:51:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.202508+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:51:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.350563+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:51:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.351287+0000 mon.a (mon.0) 3217 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.352009+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T16:51:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:51:58 gibba032 bash[24004]: audit 2021-03-10T16:51:57.352693+0000 mon.a (mon.0) 3219 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:52:00.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: cluster 2021-03-10T16:51:58.320217+0000 mgr.gibba032.whopeo (mgr.24625) 9439 : cluster [DBG] pgmap v5382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:00.255 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.604385+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:52:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.774291+0000 mon.a (mon.0) 3221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.965211+0000 mon.a (mon.0) 3222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.969106+0000 mon.a (mon.0) 3223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:52:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.970854+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.972250+0000 mon.a (mon.0) 3225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.974126+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.977502+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:00 gibba032 bash[24004]: audit 2021-03-10T16:51:59.979202+0000 mon.a (mon.0) 3228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: cluster 2021-03-10T16:51:58.320217+0000 mgr.gibba032.whopeo (mgr.24625) 9439 : cluster [DBG] pgmap v5382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.604385+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:52:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.774291+0000 mon.a (mon.0) 3221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.965211+0000 mon.a (mon.0) 3222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.969106+0000 mon.a (mon.0) 3223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:52:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.970854+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.972250+0000 mon.a (mon.0) 3225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.974126+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.977502+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[30791]: audit 2021-03-10T16:51:59.979202+0000 mon.a (mon.0) 3228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: cluster 2021-03-10T16:51:58.320217+0000 mgr.gibba032.whopeo (mgr.24625) 9439 : cluster [DBG] pgmap v5382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.604385+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T16:52:00.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.774291+0000 mon.a (mon.0) 3221 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.965211+0000 mon.a (mon.0) 3222 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:52:00.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.969106+0000 mon.a (mon.0) 3223 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:52:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.970854+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.972250+0000 mon.a (mon.0) 3225 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.974126+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.977502+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:00.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:00 gibba011 bash[31774]: audit 2021-03-10T16:51:59.979202+0000 mon.a (mon.0) 3228 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:01.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.773183+0000 mgr.gibba032.whopeo (mgr.24625) 9440 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:52:01.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.966394+0000 mgr.gibba032.whopeo (mgr.24625) 9441 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:52:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.968588+0000 mgr.gibba032.whopeo (mgr.24625) 9442 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:52:01.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.969918+0000 mgr.gibba032.whopeo (mgr.24625) 9443 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:52:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.969956+0000 mgr.gibba032.whopeo (mgr.24625) 9444 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:52:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.971840+0000 mgr.gibba032.whopeo (mgr.24625) 9445 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:52:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.971877+0000 mgr.gibba032.whopeo (mgr.24625) 9446 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:52:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.973533+0000 mgr.gibba032.whopeo (mgr.24625) 9447 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:52:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.975084+0000 mgr.gibba032.whopeo (mgr.24625) 9448 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:52:01.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.975123+0000 mgr.gibba032.whopeo (mgr.24625) 9449 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:52:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.976795+0000 mgr.gibba032.whopeo (mgr.24625) 9450 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:52:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.977125+0000 mgr.gibba032.whopeo (mgr.24625) 9451 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:52:01.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.978297+0000 mgr.gibba032.whopeo (mgr.24625) 9452 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:52:01.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.978346+0000 mgr.gibba032.whopeo (mgr.24625) 9453 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:52:01.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:01 gibba032 bash[24004]: cephadm 2021-03-10T16:51:59.978467+0000 mgr.gibba032.whopeo (mgr.24625) 9454 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:01.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.773183+0000 mgr.gibba032.whopeo (mgr.24625) 9440 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:52:01.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.966394+0000 mgr.gibba032.whopeo (mgr.24625) 9441 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:52:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.968588+0000 mgr.gibba032.whopeo (mgr.24625) 9442 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:52:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.969918+0000 mgr.gibba032.whopeo (mgr.24625) 9443 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:52:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.969956+0000 mgr.gibba032.whopeo (mgr.24625) 9444 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:52:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.971840+0000 mgr.gibba032.whopeo (mgr.24625) 9445 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:52:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.971877+0000 mgr.gibba032.whopeo (mgr.24625) 9446 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:52:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.973533+0000 mgr.gibba032.whopeo (mgr.24625) 9447 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:52:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.975084+0000 mgr.gibba032.whopeo (mgr.24625) 9448 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:52:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.975123+0000 mgr.gibba032.whopeo (mgr.24625) 9449 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:52:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.976795+0000 mgr.gibba032.whopeo (mgr.24625) 9450 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:52:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.977125+0000 mgr.gibba032.whopeo (mgr.24625) 9451 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:52:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.978297+0000 mgr.gibba032.whopeo (mgr.24625) 9452 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:52:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.978346+0000 mgr.gibba032.whopeo (mgr.24625) 9453 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:52:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[30791]: cephadm 2021-03-10T16:51:59.978467+0000 mgr.gibba032.whopeo (mgr.24625) 9454 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.773183+0000 mgr.gibba032.whopeo (mgr.24625) 9440 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:52:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.966394+0000 mgr.gibba032.whopeo (mgr.24625) 9441 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:52:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.968588+0000 mgr.gibba032.whopeo (mgr.24625) 9442 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:52:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.969918+0000 mgr.gibba032.whopeo (mgr.24625) 9443 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:52:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.969956+0000 mgr.gibba032.whopeo (mgr.24625) 9444 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:52:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.971840+0000 mgr.gibba032.whopeo (mgr.24625) 9445 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:52:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.971877+0000 mgr.gibba032.whopeo (mgr.24625) 9446 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:52:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.973533+0000 mgr.gibba032.whopeo (mgr.24625) 9447 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:52:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.975084+0000 mgr.gibba032.whopeo (mgr.24625) 9448 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:52:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.975123+0000 mgr.gibba032.whopeo (mgr.24625) 9449 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:52:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.976795+0000 mgr.gibba032.whopeo (mgr.24625) 9450 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:52:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.977125+0000 mgr.gibba032.whopeo (mgr.24625) 9451 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:52:01.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.978297+0000 mgr.gibba032.whopeo (mgr.24625) 9452 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:52:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.978346+0000 mgr.gibba032.whopeo (mgr.24625) 9453 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:52:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:01 gibba011 bash[31774]: cephadm 2021-03-10T16:51:59.978467+0000 mgr.gibba032.whopeo (mgr.24625) 9454 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:02.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:02 gibba032 bash[24004]: cluster 2021-03-10T16:52:00.321709+0000 mgr.gibba032.whopeo (mgr.24625) 9455 : cluster [DBG] pgmap v5383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:02 gibba011 bash[30791]: cluster 2021-03-10T16:52:00.321709+0000 mgr.gibba032.whopeo (mgr.24625) 9455 : cluster [DBG] pgmap v5383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:02 gibba011 bash[31774]: cluster 2021-03-10T16:52:00.321709+0000 mgr.gibba032.whopeo (mgr.24625) 9455 : cluster [DBG] pgmap v5383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:04.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:04 gibba032 bash[24004]: cluster 2021-03-10T16:52:02.323353+0000 mgr.gibba032.whopeo (mgr.24625) 9456 : cluster [DBG] pgmap v5384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:04 gibba011 bash[30791]: cluster 2021-03-10T16:52:02.323353+0000 mgr.gibba032.whopeo (mgr.24625) 9456 : cluster [DBG] pgmap v5384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:04 gibba011 bash[31774]: cluster 2021-03-10T16:52:02.323353+0000 mgr.gibba032.whopeo (mgr.24625) 9456 : cluster [DBG] pgmap v5384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:06.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:06 gibba032 bash[24004]: cluster 2021-03-10T16:52:04.324736+0000 mgr.gibba032.whopeo (mgr.24625) 9457 : cluster [DBG] pgmap v5385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:06.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:06 gibba011 bash[30791]: cluster 2021-03-10T16:52:04.324736+0000 mgr.gibba032.whopeo (mgr.24625) 9457 : cluster [DBG] pgmap v5385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:06 gibba011 bash[31774]: cluster 2021-03-10T16:52:04.324736+0000 mgr.gibba032.whopeo (mgr.24625) 9457 : cluster [DBG] pgmap v5385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:07.574 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:52:07.801 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:52:07.801 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 76s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:52:07.802 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 12s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:52:07.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:52:07.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:52:07.802 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:52:07.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:52:07.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:52:07.803 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:52:07.803 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 76s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:52:07.804 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 12s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:52:07.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:52:07.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:52:07.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:52:07.804 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:52:07.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:52:07.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:52:07.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:52:07.805 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:52:07.806 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 12s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:52:07.806 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 76s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:52:08.060 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:52:08.061 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:52:08.061 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:52:08.061 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:08.061 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:52:08.062 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:52:08.062 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:08.062 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:52:08.062 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:52:08.063 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:08.063 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:52:08.063 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:52:08.063 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:52:08.064 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:52:08.064 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:52:08.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:08 gibba032 bash[24004]: cluster 2021-03-10T16:52:06.326190+0000 mgr.gibba032.whopeo (mgr.24625) 9458 : cluster [DBG] pgmap v5386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:08.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:08 gibba032 bash[24004]: audit 2021-03-10T16:52:08.062855+0000 mon.c (mon.1) 182 : audit [DBG] from='client.? 172.21.2.111:0/2775731040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:08 gibba011 bash[30791]: cluster 2021-03-10T16:52:06.326190+0000 mgr.gibba032.whopeo (mgr.24625) 9458 : cluster [DBG] pgmap v5386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:08 gibba011 bash[30791]: audit 2021-03-10T16:52:08.062855+0000 mon.c (mon.1) 182 : audit [DBG] from='client.? 172.21.2.111:0/2775731040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:08 gibba011 bash[31774]: cluster 2021-03-10T16:52:06.326190+0000 mgr.gibba032.whopeo (mgr.24625) 9458 : cluster [DBG] pgmap v5386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:08 gibba011 bash[31774]: audit 2021-03-10T16:52:08.062855+0000 mon.c (mon.1) 182 : audit [DBG] from='client.? 172.21.2.111:0/2775731040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:09.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:09 gibba032 bash[24004]: audit 2021-03-10T16:52:07.566177+0000 mgr.gibba032.whopeo (mgr.24625) 9459 : audit [DBG] from='client.60373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:09.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:09 gibba032 bash[24004]: audit 2021-03-10T16:52:07.789342+0000 mgr.gibba032.whopeo (mgr.24625) 9460 : audit [DBG] from='client.60379 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:09.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:09 gibba011 bash[30791]: audit 2021-03-10T16:52:07.566177+0000 mgr.gibba032.whopeo (mgr.24625) 9459 : audit [DBG] from='client.60373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:09.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:09 gibba011 bash[30791]: audit 2021-03-10T16:52:07.789342+0000 mgr.gibba032.whopeo (mgr.24625) 9460 : audit [DBG] from='client.60379 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:09.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:09 gibba011 bash[31774]: audit 2021-03-10T16:52:07.566177+0000 mgr.gibba032.whopeo (mgr.24625) 9459 : audit [DBG] from='client.60373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:09.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:09 gibba011 bash[31774]: audit 2021-03-10T16:52:07.789342+0000 mgr.gibba032.whopeo (mgr.24625) 9460 : audit [DBG] from='client.60379 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:10 gibba011 bash[30791]: cluster 2021-03-10T16:52:08.327454+0000 mgr.gibba032.whopeo (mgr.24625) 9461 : cluster [DBG] pgmap v5387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:10.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:10 gibba011 bash[31774]: cluster 2021-03-10T16:52:08.327454+0000 mgr.gibba032.whopeo (mgr.24625) 9461 : cluster [DBG] pgmap v5387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:10.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:10 gibba032 bash[24004]: cluster 2021-03-10T16:52:08.327454+0000 mgr.gibba032.whopeo (mgr.24625) 9461 : cluster [DBG] pgmap v5387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:12.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:12 gibba011 bash[30791]: cluster 2021-03-10T16:52:10.329048+0000 mgr.gibba032.whopeo (mgr.24625) 9462 : cluster [DBG] pgmap v5388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:12.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:12 gibba011 bash[31774]: cluster 2021-03-10T16:52:10.329048+0000 mgr.gibba032.whopeo (mgr.24625) 9462 : cluster [DBG] pgmap v5388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:12 gibba032 bash[24004]: cluster 2021-03-10T16:52:10.329048+0000 mgr.gibba032.whopeo (mgr.24625) 9462 : cluster [DBG] pgmap v5388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:14 gibba011 bash[30791]: cluster 2021-03-10T16:52:12.330130+0000 mgr.gibba032.whopeo (mgr.24625) 9463 : cluster [DBG] pgmap v5389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:14.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:14 gibba011 bash[31774]: cluster 2021-03-10T16:52:12.330130+0000 mgr.gibba032.whopeo (mgr.24625) 9463 : cluster [DBG] pgmap v5389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:14.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:14 gibba032 bash[24004]: cluster 2021-03-10T16:52:12.330130+0000 mgr.gibba032.whopeo (mgr.24625) 9463 : cluster [DBG] pgmap v5389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:16.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:16 gibba032 bash[24004]: cluster 2021-03-10T16:52:14.330722+0000 mgr.gibba032.whopeo (mgr.24625) 9464 : cluster [DBG] pgmap v5390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:16 gibba032 bash[24004]: cephadm 2021-03-10T16:52:14.994316+0000 mgr.gibba032.whopeo (mgr.24625) 9465 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:16.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:16 gibba011 bash[30791]: cluster 2021-03-10T16:52:14.330722+0000 mgr.gibba032.whopeo (mgr.24625) 9464 : cluster [DBG] pgmap v5390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:16 gibba011 bash[30791]: cephadm 2021-03-10T16:52:14.994316+0000 mgr.gibba032.whopeo (mgr.24625) 9465 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:16 gibba011 bash[31774]: cluster 2021-03-10T16:52:14.330722+0000 mgr.gibba032.whopeo (mgr.24625) 9464 : cluster [DBG] pgmap v5390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:16 gibba011 bash[31774]: cephadm 2021-03-10T16:52:14.994316+0000 mgr.gibba032.whopeo (mgr.24625) 9465 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:18.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:18 gibba032 bash[24004]: cluster 2021-03-10T16:52:16.332397+0000 mgr.gibba032.whopeo (mgr.24625) 9466 : cluster [DBG] pgmap v5391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:18 gibba011 bash[30791]: cluster 2021-03-10T16:52:16.332397+0000 mgr.gibba032.whopeo (mgr.24625) 9466 : cluster [DBG] pgmap v5391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:18.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:18 gibba011 bash[31774]: cluster 2021-03-10T16:52:16.332397+0000 mgr.gibba032.whopeo (mgr.24625) 9466 : cluster [DBG] pgmap v5391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:20.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:20 gibba032 bash[24004]: cluster 2021-03-10T16:52:18.334095+0000 mgr.gibba032.whopeo (mgr.24625) 9467 : cluster [DBG] pgmap v5392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:20.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:20 gibba011 bash[30791]: cluster 2021-03-10T16:52:18.334095+0000 mgr.gibba032.whopeo (mgr.24625) 9467 : cluster [DBG] pgmap v5392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:20.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:20 gibba011 bash[31774]: cluster 2021-03-10T16:52:18.334095+0000 mgr.gibba032.whopeo (mgr.24625) 9467 : cluster [DBG] pgmap v5392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:22.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:22 gibba032 bash[24004]: cluster 2021-03-10T16:52:20.335683+0000 mgr.gibba032.whopeo (mgr.24625) 9468 : cluster [DBG] pgmap v5393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:22 gibba011 bash[30791]: cluster 2021-03-10T16:52:20.335683+0000 mgr.gibba032.whopeo (mgr.24625) 9468 : cluster [DBG] pgmap v5393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:22.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:22 gibba011 bash[31774]: cluster 2021-03-10T16:52:20.335683+0000 mgr.gibba032.whopeo (mgr.24625) 9468 : cluster [DBG] pgmap v5393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:24.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:24 gibba032 bash[24004]: cluster 2021-03-10T16:52:22.337305+0000 mgr.gibba032.whopeo (mgr.24625) 9469 : cluster [DBG] pgmap v5394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:24 gibba011 bash[30791]: cluster 2021-03-10T16:52:22.337305+0000 mgr.gibba032.whopeo (mgr.24625) 9469 : cluster [DBG] pgmap v5394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:24.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:24 gibba011 bash[31774]: cluster 2021-03-10T16:52:22.337305+0000 mgr.gibba032.whopeo (mgr.24625) 9469 : cluster [DBG] pgmap v5394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:26.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:26 gibba032 bash[24004]: cluster 2021-03-10T16:52:24.338783+0000 mgr.gibba032.whopeo (mgr.24625) 9470 : cluster [DBG] pgmap v5395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:26.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:26 gibba011 bash[30791]: cluster 2021-03-10T16:52:24.338783+0000 mgr.gibba032.whopeo (mgr.24625) 9470 : cluster [DBG] pgmap v5395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:26 gibba011 bash[31774]: cluster 2021-03-10T16:52:24.338783+0000 mgr.gibba032.whopeo (mgr.24625) 9470 : cluster [DBG] pgmap v5395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:28 gibba011 bash[30791]: cluster 2021-03-10T16:52:26.340578+0000 mgr.gibba032.whopeo (mgr.24625) 9471 : cluster [DBG] pgmap v5396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:28 gibba011 bash[31774]: cluster 2021-03-10T16:52:26.340578+0000 mgr.gibba032.whopeo (mgr.24625) 9471 : cluster [DBG] pgmap v5396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:28.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:28 gibba032 bash[24004]: cluster 2021-03-10T16:52:26.340578+0000 mgr.gibba032.whopeo (mgr.24625) 9471 : cluster [DBG] pgmap v5396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:30.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:30 gibba011 bash[30791]: cluster 2021-03-10T16:52:28.342181+0000 mgr.gibba032.whopeo (mgr.24625) 9472 : cluster [DBG] pgmap v5397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:30.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:30 gibba011 bash[31774]: cluster 2021-03-10T16:52:28.342181+0000 mgr.gibba032.whopeo (mgr.24625) 9472 : cluster [DBG] pgmap v5397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:30.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:30 gibba032 bash[24004]: cluster 2021-03-10T16:52:28.342181+0000 mgr.gibba032.whopeo (mgr.24625) 9472 : cluster [DBG] pgmap v5397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:31.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:31 gibba011 bash[30791]: cephadm 2021-03-10T16:52:30.008202+0000 mgr.gibba032.whopeo (mgr.24625) 9473 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:31.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:31 gibba011 bash[31774]: cephadm 2021-03-10T16:52:30.008202+0000 mgr.gibba032.whopeo (mgr.24625) 9473 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:31 gibba032 bash[24004]: cephadm 2021-03-10T16:52:30.008202+0000 mgr.gibba032.whopeo (mgr.24625) 9473 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:32 gibba011 bash[30791]: cluster 2021-03-10T16:52:30.343789+0000 mgr.gibba032.whopeo (mgr.24625) 9474 : cluster [DBG] pgmap v5398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:32.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:32 gibba011 bash[31774]: cluster 2021-03-10T16:52:30.343789+0000 mgr.gibba032.whopeo (mgr.24625) 9474 : cluster [DBG] pgmap v5398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:32.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:32 gibba032 bash[24004]: cluster 2021-03-10T16:52:30.343789+0000 mgr.gibba032.whopeo (mgr.24625) 9474 : cluster [DBG] pgmap v5398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:33 gibba011 bash[30791]: cluster 2021-03-10T16:52:32.344795+0000 mgr.gibba032.whopeo (mgr.24625) 9475 : cluster [DBG] pgmap v5399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:33 gibba011 bash[31774]: cluster 2021-03-10T16:52:32.344795+0000 mgr.gibba032.whopeo (mgr.24625) 9475 : cluster [DBG] pgmap v5399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:33.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:33 gibba032 bash[24004]: cluster 2021-03-10T16:52:32.344795+0000 mgr.gibba032.whopeo (mgr.24625) 9475 : cluster [DBG] pgmap v5399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:36.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:36 gibba032 bash[24004]: cluster 2021-03-10T16:52:34.346100+0000 mgr.gibba032.whopeo (mgr.24625) 9476 : cluster [DBG] pgmap v5400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:36 gibba011 bash[30791]: cluster 2021-03-10T16:52:34.346100+0000 mgr.gibba032.whopeo (mgr.24625) 9476 : cluster [DBG] pgmap v5400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:36 gibba011 bash[31774]: cluster 2021-03-10T16:52:34.346100+0000 mgr.gibba032.whopeo (mgr.24625) 9476 : cluster [DBG] pgmap v5400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:38.310 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:52:38.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:38 gibba032 bash[24004]: cluster 2021-03-10T16:52:36.347786+0000 mgr.gibba032.whopeo (mgr.24625) 9477 : cluster [DBG] pgmap v5401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:38.536 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:52:38.536 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 107s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:52:38.537 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 43s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:52:38.537 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:52:38.537 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:52:38.537 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:52:38.537 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:52:38.538 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:52:38.538 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:52:38.538 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 107s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:52:38.538 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 43s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:52:38.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:52:38.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:52:38.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:52:38.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:52:38.539 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:52:38.540 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:52:38.540 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:52:38.540 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:52:38.543 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 43s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:52:38.543 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 107s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:52:38.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:38 gibba011 bash[30791]: cluster 2021-03-10T16:52:36.347786+0000 mgr.gibba032.whopeo (mgr.24625) 9477 : cluster [DBG] pgmap v5401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:38.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:38 gibba011 bash[31774]: cluster 2021-03-10T16:52:36.347786+0000 mgr.gibba032.whopeo (mgr.24625) 9477 : cluster [DBG] pgmap v5401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:38.784 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:52:38.785 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:52:38.785 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:52:38.785 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:38.786 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:52:38.786 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:52:38.786 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:38.786 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:52:38.786 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:52:38.787 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:52:38.787 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:52:38.787 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:52:38.787 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:52:38.788 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:52:38.788 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:52:39.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:39 gibba032 bash[24004]: audit 2021-03-10T16:52:38.786257+0000 mon.c (mon.1) 183 : audit [DBG] from='client.? 172.21.2.111:0/3640606000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:39.576 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:39 gibba011 bash[30791]: audit 2021-03-10T16:52:38.786257+0000 mon.c (mon.1) 183 : audit [DBG] from='client.? 172.21.2.111:0/3640606000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:39.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:39 gibba011 bash[31774]: audit 2021-03-10T16:52:38.786257+0000 mon.c (mon.1) 183 : audit [DBG] from='client.? 172.21.2.111:0/3640606000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:52:40.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:40 gibba032 bash[24004]: audit 2021-03-10T16:52:38.302624+0000 mgr.gibba032.whopeo (mgr.24625) 9478 : audit [DBG] from='client.50409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:40.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:40 gibba032 bash[24004]: cluster 2021-03-10T16:52:38.349490+0000 mgr.gibba032.whopeo (mgr.24625) 9479 : cluster [DBG] pgmap v5402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:40.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:40 gibba032 bash[24004]: audit 2021-03-10T16:52:38.524524+0000 mgr.gibba032.whopeo (mgr.24625) 9480 : audit [DBG] from='client.50415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[30791]: audit 2021-03-10T16:52:38.302624+0000 mgr.gibba032.whopeo (mgr.24625) 9478 : audit [DBG] from='client.50409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[30791]: cluster 2021-03-10T16:52:38.349490+0000 mgr.gibba032.whopeo (mgr.24625) 9479 : cluster [DBG] pgmap v5402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[30791]: audit 2021-03-10T16:52:38.524524+0000 mgr.gibba032.whopeo (mgr.24625) 9480 : audit [DBG] from='client.50415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[31774]: audit 2021-03-10T16:52:38.302624+0000 mgr.gibba032.whopeo (mgr.24625) 9478 : audit [DBG] from='client.50409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[31774]: cluster 2021-03-10T16:52:38.349490+0000 mgr.gibba032.whopeo (mgr.24625) 9479 : cluster [DBG] pgmap v5402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:40 gibba011 bash[31774]: audit 2021-03-10T16:52:38.524524+0000 mgr.gibba032.whopeo (mgr.24625) 9480 : audit [DBG] from='client.50415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:52:42.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:42 gibba032 bash[24004]: cluster 2021-03-10T16:52:40.350951+0000 mgr.gibba032.whopeo (mgr.24625) 9481 : cluster [DBG] pgmap v5403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:42 gibba011 bash[30791]: cluster 2021-03-10T16:52:40.350951+0000 mgr.gibba032.whopeo (mgr.24625) 9481 : cluster [DBG] pgmap v5403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:42.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:42 gibba011 bash[31774]: cluster 2021-03-10T16:52:40.350951+0000 mgr.gibba032.whopeo (mgr.24625) 9481 : cluster [DBG] pgmap v5403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:44 gibba011 bash[30791]: cluster 2021-03-10T16:52:42.352694+0000 mgr.gibba032.whopeo (mgr.24625) 9482 : cluster [DBG] pgmap v5404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:44.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:44 gibba011 bash[31774]: cluster 2021-03-10T16:52:42.352694+0000 mgr.gibba032.whopeo (mgr.24625) 9482 : cluster [DBG] pgmap v5404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:44.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:44 gibba032 bash[24004]: cluster 2021-03-10T16:52:42.352694+0000 mgr.gibba032.whopeo (mgr.24625) 9482 : cluster [DBG] pgmap v5404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:46 gibba011 bash[30791]: cluster 2021-03-10T16:52:44.353922+0000 mgr.gibba032.whopeo (mgr.24625) 9483 : cluster [DBG] pgmap v5405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:46 gibba011 bash[30791]: cephadm 2021-03-10T16:52:45.024329+0000 mgr.gibba032.whopeo (mgr.24625) 9484 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:46.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:46 gibba011 bash[31774]: cluster 2021-03-10T16:52:44.353922+0000 mgr.gibba032.whopeo (mgr.24625) 9483 : cluster [DBG] pgmap v5405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:46 gibba011 bash[31774]: cephadm 2021-03-10T16:52:45.024329+0000 mgr.gibba032.whopeo (mgr.24625) 9484 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:46.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:46 gibba032 bash[24004]: cluster 2021-03-10T16:52:44.353922+0000 mgr.gibba032.whopeo (mgr.24625) 9483 : cluster [DBG] pgmap v5405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:46 gibba032 bash[24004]: cephadm 2021-03-10T16:52:45.024329+0000 mgr.gibba032.whopeo (mgr.24625) 9484 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:52:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:48 gibba011 bash[30791]: cluster 2021-03-10T16:52:46.355643+0000 mgr.gibba032.whopeo (mgr.24625) 9485 : cluster [DBG] pgmap v5406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:48 gibba011 bash[31774]: cluster 2021-03-10T16:52:46.355643+0000 mgr.gibba032.whopeo (mgr.24625) 9485 : cluster [DBG] pgmap v5406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:48.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:48 gibba032 bash[24004]: cluster 2021-03-10T16:52:46.355643+0000 mgr.gibba032.whopeo (mgr.24625) 9485 : cluster [DBG] pgmap v5406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:50 gibba011 bash[30791]: cluster 2021-03-10T16:52:48.356751+0000 mgr.gibba032.whopeo (mgr.24625) 9486 : cluster [DBG] pgmap v5407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:50.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:50 gibba011 bash[31774]: cluster 2021-03-10T16:52:48.356751+0000 mgr.gibba032.whopeo (mgr.24625) 9486 : cluster [DBG] pgmap v5407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:50.754 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:50 gibba032 bash[24004]: cluster 2021-03-10T16:52:48.356751+0000 mgr.gibba032.whopeo (mgr.24625) 9486 : cluster [DBG] pgmap v5407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:51.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:51 gibba032 bash[24004]: cluster 2021-03-10T16:52:50.358248+0000 mgr.gibba032.whopeo (mgr.24625) 9487 : cluster [DBG] pgmap v5408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:51.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:51 gibba011 bash[30791]: cluster 2021-03-10T16:52:50.358248+0000 mgr.gibba032.whopeo (mgr.24625) 9487 : cluster [DBG] pgmap v5408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:51.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:51 gibba011 bash[31774]: cluster 2021-03-10T16:52:50.358248+0000 mgr.gibba032.whopeo (mgr.24625) 9487 : cluster [DBG] pgmap v5408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:53.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:53 gibba032 bash[24004]: cluster 2021-03-10T16:52:52.359820+0000 mgr.gibba032.whopeo (mgr.24625) 9488 : cluster [DBG] pgmap v5409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:53.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:53 gibba011 bash[30791]: cluster 2021-03-10T16:52:52.359820+0000 mgr.gibba032.whopeo (mgr.24625) 9488 : cluster [DBG] pgmap v5409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:53.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:53 gibba011 bash[31774]: cluster 2021-03-10T16:52:52.359820+0000 mgr.gibba032.whopeo (mgr.24625) 9488 : cluster [DBG] pgmap v5409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:56.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:56 gibba032 bash[24004]: cluster 2021-03-10T16:52:54.361336+0000 mgr.gibba032.whopeo (mgr.24625) 9489 : cluster [DBG] pgmap v5410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:56 gibba011 bash[30791]: cluster 2021-03-10T16:52:54.361336+0000 mgr.gibba032.whopeo (mgr.24625) 9489 : cluster [DBG] pgmap v5410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:56 gibba011 bash[31774]: cluster 2021-03-10T16:52:54.361336+0000 mgr.gibba032.whopeo (mgr.24625) 9489 : cluster [DBG] pgmap v5410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:57.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:57 gibba032 bash[24004]: audit 2021-03-10T16:52:57.208188+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:52:57.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:57 gibba032 bash[24004]: audit 2021-03-10T16:52:57.208817+0000 mon.a (mon.0) 3230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:52:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:57 gibba011 bash[30791]: audit 2021-03-10T16:52:57.208188+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:52:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:57 gibba011 bash[30791]: audit 2021-03-10T16:52:57.208817+0000 mon.a (mon.0) 3230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:52:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:57 gibba011 bash[31774]: audit 2021-03-10T16:52:57.208188+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:52:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:57 gibba011 bash[31774]: audit 2021-03-10T16:52:57.208817+0000 mon.a (mon.0) 3230 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:52:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:58 gibba032 bash[24004]: cluster 2021-03-10T16:52:56.363144+0000 mgr.gibba032.whopeo (mgr.24625) 9490 : cluster [DBG] pgmap v5411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:58 gibba032 bash[24004]: audit 2021-03-10T16:52:57.358881+0000 mon.a (mon.0) 3231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:52:58.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:58 gibba032 bash[24004]: audit 2021-03-10T16:52:57.359682+0000 mon.a (mon.0) 3232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:52:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:58 gibba032 bash[24004]: audit 2021-03-10T16:52:57.360395+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:52:58.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:52:58 gibba032 bash[24004]: audit 2021-03-10T16:52:57.361065+0000 mon.a (mon.0) 3234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:52:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[30791]: cluster 2021-03-10T16:52:56.363144+0000 mgr.gibba032.whopeo (mgr.24625) 9490 : cluster [DBG] pgmap v5411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[30791]: audit 2021-03-10T16:52:57.358881+0000 mon.a (mon.0) 3231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:52:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[30791]: audit 2021-03-10T16:52:57.359682+0000 mon.a (mon.0) 3232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:52:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[30791]: audit 2021-03-10T16:52:57.360395+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:52:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[30791]: audit 2021-03-10T16:52:57.361065+0000 mon.a (mon.0) 3234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:52:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[31774]: cluster 2021-03-10T16:52:56.363144+0000 mgr.gibba032.whopeo (mgr.24625) 9490 : cluster [DBG] pgmap v5411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:52:58.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[31774]: audit 2021-03-10T16:52:57.358881+0000 mon.a (mon.0) 3231 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T16:52:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[31774]: audit 2021-03-10T16:52:57.359682+0000 mon.a (mon.0) 3232 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:52:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[31774]: audit 2021-03-10T16:52:57.360395+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:52:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:52:58 gibba011 bash[31774]: audit 2021-03-10T16:52:57.361065+0000 mon.a (mon.0) 3234 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:53:00.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:00 gibba032 bash[24004]: cluster 2021-03-10T16:52:58.364674+0000 mgr.gibba032.whopeo (mgr.24625) 9491 : cluster [DBG] pgmap v5412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:00 gibba032 bash[24004]: audit 2021-03-10T16:53:00.244683+0000 mon.a (mon.0) 3235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:53:00.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:00 gibba032 bash[24004]: audit 2021-03-10T16:53:00.248034+0000 mon.a (mon.0) 3236 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:53:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[30791]: cluster 2021-03-10T16:52:58.364674+0000 mgr.gibba032.whopeo (mgr.24625) 9491 : cluster [DBG] pgmap v5412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[30791]: audit 2021-03-10T16:53:00.244683+0000 mon.a (mon.0) 3235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:53:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[30791]: audit 2021-03-10T16:53:00.248034+0000 mon.a (mon.0) 3236 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:53:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[31774]: cluster 2021-03-10T16:52:58.364674+0000 mgr.gibba032.whopeo (mgr.24625) 9491 : cluster [DBG] pgmap v5412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[31774]: audit 2021-03-10T16:53:00.244683+0000 mon.a (mon.0) 3235 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:53:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:00 gibba011 bash[31774]: audit 2021-03-10T16:53:00.248034+0000 mon.a (mon.0) 3236 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:53:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.036449+0000 mgr.gibba032.whopeo (mgr.24625) 9492 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:53:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.036767+0000 mgr.gibba032.whopeo (mgr.24625) 9493 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:53:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.245523+0000 mgr.gibba032.whopeo (mgr.24625) 9494 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:53:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.247421+0000 mgr.gibba032.whopeo (mgr.24625) 9495 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:53:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.249182+0000 mgr.gibba032.whopeo (mgr.24625) 9496 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:53:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.249221+0000 mgr.gibba032.whopeo (mgr.24625) 9497 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:53:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: audit 2021-03-10T16:53:00.250014+0000 mon.a (mon.0) 3237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.251853+0000 mgr.gibba032.whopeo (mgr.24625) 9498 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:53:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.251893+0000 mgr.gibba032.whopeo (mgr.24625) 9499 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:53:01.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: audit 2021-03-10T16:53:00.252767+0000 mon.a (mon.0) 3238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.252775+0000 mgr.gibba032.whopeo (mgr.24625) 9500 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:53:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.253564+0000 mgr.gibba032.whopeo (mgr.24625) 9501 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:53:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.253601+0000 mgr.gibba032.whopeo (mgr.24625) 9502 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:53:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: audit 2021-03-10T16:53:00.253786+0000 mon.a (mon.0) 3239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.254682+0000 mgr.gibba032.whopeo (mgr.24625) 9503 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:53:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.255010+0000 mgr.gibba032.whopeo (mgr.24625) 9504 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:53:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: audit 2021-03-10T16:53:00.255680+0000 mon.a (mon.0) 3240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.255930+0000 mgr.gibba032.whopeo (mgr.24625) 9505 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:53:01.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.255974+0000 mgr.gibba032.whopeo (mgr.24625) 9506 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:53:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: cephadm 2021-03-10T16:53:00.256142+0000 mgr.gibba032.whopeo (mgr.24625) 9507 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:01.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[30791]: audit 2021-03-10T16:53:00.256950+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.036449+0000 mgr.gibba032.whopeo (mgr.24625) 9492 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:53:01.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.036767+0000 mgr.gibba032.whopeo (mgr.24625) 9493 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:53:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.245523+0000 mgr.gibba032.whopeo (mgr.24625) 9494 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:53:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.247421+0000 mgr.gibba032.whopeo (mgr.24625) 9495 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:53:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.249182+0000 mgr.gibba032.whopeo (mgr.24625) 9496 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:53:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.249221+0000 mgr.gibba032.whopeo (mgr.24625) 9497 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:53:01.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: audit 2021-03-10T16:53:00.250014+0000 mon.a (mon.0) 3237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.251853+0000 mgr.gibba032.whopeo (mgr.24625) 9498 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:53:01.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.251893+0000 mgr.gibba032.whopeo (mgr.24625) 9499 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:53:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: audit 2021-03-10T16:53:00.252767+0000 mon.a (mon.0) 3238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.252775+0000 mgr.gibba032.whopeo (mgr.24625) 9500 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:53:01.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.253564+0000 mgr.gibba032.whopeo (mgr.24625) 9501 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:53:01.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.253601+0000 mgr.gibba032.whopeo (mgr.24625) 9502 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:53:01.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: audit 2021-03-10T16:53:00.253786+0000 mon.a (mon.0) 3239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.254682+0000 mgr.gibba032.whopeo (mgr.24625) 9503 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:53:01.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.255010+0000 mgr.gibba032.whopeo (mgr.24625) 9504 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:53:01.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: audit 2021-03-10T16:53:00.255680+0000 mon.a (mon.0) 3240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.255930+0000 mgr.gibba032.whopeo (mgr.24625) 9505 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:53:01.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.255974+0000 mgr.gibba032.whopeo (mgr.24625) 9506 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:53:01.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: cephadm 2021-03-10T16:53:00.256142+0000 mgr.gibba032.whopeo (mgr.24625) 9507 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:01.595 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:01 gibba011 bash[31774]: audit 2021-03-10T16:53:00.256950+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.036449+0000 mgr.gibba032.whopeo (mgr.24625) 9492 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:53:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.036767+0000 mgr.gibba032.whopeo (mgr.24625) 9493 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:53:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.245523+0000 mgr.gibba032.whopeo (mgr.24625) 9494 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:53:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.247421+0000 mgr.gibba032.whopeo (mgr.24625) 9495 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:53:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.249182+0000 mgr.gibba032.whopeo (mgr.24625) 9496 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:53:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.249221+0000 mgr.gibba032.whopeo (mgr.24625) 9497 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:53:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: audit 2021-03-10T16:53:00.250014+0000 mon.a (mon.0) 3237 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.251853+0000 mgr.gibba032.whopeo (mgr.24625) 9498 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:53:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.251893+0000 mgr.gibba032.whopeo (mgr.24625) 9499 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:53:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: audit 2021-03-10T16:53:00.252767+0000 mon.a (mon.0) 3238 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.252775+0000 mgr.gibba032.whopeo (mgr.24625) 9500 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:53:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.253564+0000 mgr.gibba032.whopeo (mgr.24625) 9501 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:53:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.253601+0000 mgr.gibba032.whopeo (mgr.24625) 9502 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:53:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: audit 2021-03-10T16:53:00.253786+0000 mon.a (mon.0) 3239 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.254682+0000 mgr.gibba032.whopeo (mgr.24625) 9503 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:53:01.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.255010+0000 mgr.gibba032.whopeo (mgr.24625) 9504 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:53:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: audit 2021-03-10T16:53:00.255680+0000 mon.a (mon.0) 3240 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.255930+0000 mgr.gibba032.whopeo (mgr.24625) 9505 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:53:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.255974+0000 mgr.gibba032.whopeo (mgr.24625) 9506 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:53:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: cephadm 2021-03-10T16:53:00.256142+0000 mgr.gibba032.whopeo (mgr.24625) 9507 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:01.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:01 gibba032 bash[24004]: audit 2021-03-10T16:53:00.256950+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:02.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:02 gibba011 bash[30791]: cluster 2021-03-10T16:53:00.366119+0000 mgr.gibba032.whopeo (mgr.24625) 9508 : cluster [DBG] pgmap v5413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:02.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:02 gibba011 bash[31774]: cluster 2021-03-10T16:53:00.366119+0000 mgr.gibba032.whopeo (mgr.24625) 9508 : cluster [DBG] pgmap v5413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:02.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:02 gibba032 bash[24004]: cluster 2021-03-10T16:53:00.366119+0000 mgr.gibba032.whopeo (mgr.24625) 9508 : cluster [DBG] pgmap v5413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:04.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:04 gibba011 bash[30791]: cluster 2021-03-10T16:53:02.367654+0000 mgr.gibba032.whopeo (mgr.24625) 9509 : cluster [DBG] pgmap v5414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:04.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:04 gibba011 bash[31774]: cluster 2021-03-10T16:53:02.367654+0000 mgr.gibba032.whopeo (mgr.24625) 9509 : cluster [DBG] pgmap v5414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:04.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:04 gibba032 bash[24004]: cluster 2021-03-10T16:53:02.367654+0000 mgr.gibba032.whopeo (mgr.24625) 9509 : cluster [DBG] pgmap v5414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:06.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:06 gibba011 bash[30791]: cluster 2021-03-10T16:53:04.368958+0000 mgr.gibba032.whopeo (mgr.24625) 9510 : cluster [DBG] pgmap v5415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:06.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:06 gibba011 bash[31774]: cluster 2021-03-10T16:53:04.368958+0000 mgr.gibba032.whopeo (mgr.24625) 9510 : cluster [DBG] pgmap v5415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:06.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:06 gibba032 bash[24004]: cluster 2021-03-10T16:53:04.368958+0000 mgr.gibba032.whopeo (mgr.24625) 9510 : cluster [DBG] pgmap v5415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:08 gibba011 bash[30791]: cluster 2021-03-10T16:53:06.370247+0000 mgr.gibba032.whopeo (mgr.24625) 9511 : cluster [DBG] pgmap v5416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:08.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:08 gibba011 bash[31774]: cluster 2021-03-10T16:53:06.370247+0000 mgr.gibba032.whopeo (mgr.24625) 9511 : cluster [DBG] pgmap v5416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:08 gibba032 bash[24004]: cluster 2021-03-10T16:53:06.370247+0000 mgr.gibba032.whopeo (mgr.24625) 9511 : cluster [DBG] pgmap v5416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:09.027 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:53:09.253 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:53:09.254 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (2h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:53:09.254 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 74s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:53:09.255 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:53:09.255 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:53:09.255 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:53:09.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:53:09.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:53:09.256 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:53:09.257 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:53:09.257 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 74s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:53:09.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:53:09.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:53:09.258 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:53:09.259 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:53:09.259 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:53:09.259 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:53:09.260 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:53:09.261 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 74s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:53:09.261 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 74s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:53:09.261 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:53:09.507 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:53:09.508 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:53:09.508 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:53:09.508 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:09.509 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:53:09.509 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:53:09.509 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:09.510 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:53:09.510 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:53:09.511 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:09.511 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:53:09.511 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:53:09.512 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:53:09.512 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:53:09.512 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:53:09.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:09 gibba032 bash[24004]: cluster 2021-03-10T16:53:08.371360+0000 mgr.gibba032.whopeo (mgr.24625) 9512 : cluster [DBG] pgmap v5417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:09.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:09 gibba032 bash[24004]: audit 2021-03-10T16:53:09.018762+0000 mgr.gibba032.whopeo (mgr.24625) 9513 : audit [DBG] from='client.50427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:09.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:09 gibba032 bash[24004]: audit 2021-03-10T16:53:09.241737+0000 mgr.gibba032.whopeo (mgr.24625) 9514 : audit [DBG] from='client.60415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:09.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[30791]: cluster 2021-03-10T16:53:08.371360+0000 mgr.gibba032.whopeo (mgr.24625) 9512 : cluster [DBG] pgmap v5417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:09.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[30791]: audit 2021-03-10T16:53:09.018762+0000 mgr.gibba032.whopeo (mgr.24625) 9513 : audit [DBG] from='client.50427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:09.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[30791]: audit 2021-03-10T16:53:09.241737+0000 mgr.gibba032.whopeo (mgr.24625) 9514 : audit [DBG] from='client.60415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:09.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[31774]: cluster 2021-03-10T16:53:08.371360+0000 mgr.gibba032.whopeo (mgr.24625) 9512 : cluster [DBG] pgmap v5417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:09.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[31774]: audit 2021-03-10T16:53:09.018762+0000 mgr.gibba032.whopeo (mgr.24625) 9513 : audit [DBG] from='client.50427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:09.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:09 gibba011 bash[31774]: audit 2021-03-10T16:53:09.241737+0000 mgr.gibba032.whopeo (mgr.24625) 9514 : audit [DBG] from='client.60415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:10.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:10 gibba032 bash[24004]: audit 2021-03-10T16:53:09.509474+0000 mon.c (mon.1) 184 : audit [DBG] from='client.? 172.21.2.111:0/1335002380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:10 gibba011 bash[30791]: audit 2021-03-10T16:53:09.509474+0000 mon.c (mon.1) 184 : audit [DBG] from='client.? 172.21.2.111:0/1335002380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:10 gibba011 bash[31774]: audit 2021-03-10T16:53:09.509474+0000 mon.c (mon.1) 184 : audit [DBG] from='client.? 172.21.2.111:0/1335002380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:11.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:11 gibba032 bash[24004]: cluster 2021-03-10T16:53:10.372031+0000 mgr.gibba032.whopeo (mgr.24625) 9515 : cluster [DBG] pgmap v5418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:11.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:11 gibba011 bash[30791]: cluster 2021-03-10T16:53:10.372031+0000 mgr.gibba032.whopeo (mgr.24625) 9515 : cluster [DBG] pgmap v5418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:11 gibba011 bash[31774]: cluster 2021-03-10T16:53:10.372031+0000 mgr.gibba032.whopeo (mgr.24625) 9515 : cluster [DBG] pgmap v5418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:13.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:13 gibba032 bash[24004]: cluster 2021-03-10T16:53:12.373578+0000 mgr.gibba032.whopeo (mgr.24625) 9516 : cluster [DBG] pgmap v5419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:13.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:13 gibba011 bash[30791]: cluster 2021-03-10T16:53:12.373578+0000 mgr.gibba032.whopeo (mgr.24625) 9516 : cluster [DBG] pgmap v5419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:13.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:13 gibba011 bash[31774]: cluster 2021-03-10T16:53:12.373578+0000 mgr.gibba032.whopeo (mgr.24625) 9516 : cluster [DBG] pgmap v5419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:16.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:16 gibba032 bash[24004]: cluster 2021-03-10T16:53:14.374910+0000 mgr.gibba032.whopeo (mgr.24625) 9517 : cluster [DBG] pgmap v5420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:16.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:16 gibba032 bash[24004]: cephadm 2021-03-10T16:53:15.271781+0000 mgr.gibba032.whopeo (mgr.24625) 9518 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:16.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:16 gibba011 bash[30791]: cluster 2021-03-10T16:53:14.374910+0000 mgr.gibba032.whopeo (mgr.24625) 9517 : cluster [DBG] pgmap v5420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:16.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:16 gibba011 bash[30791]: cephadm 2021-03-10T16:53:15.271781+0000 mgr.gibba032.whopeo (mgr.24625) 9518 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:16 gibba011 bash[31774]: cluster 2021-03-10T16:53:14.374910+0000 mgr.gibba032.whopeo (mgr.24625) 9517 : cluster [DBG] pgmap v5420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:16 gibba011 bash[31774]: cephadm 2021-03-10T16:53:15.271781+0000 mgr.gibba032.whopeo (mgr.24625) 9518 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:18.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:18 gibba032 bash[24004]: cluster 2021-03-10T16:53:16.376649+0000 mgr.gibba032.whopeo (mgr.24625) 9519 : cluster [DBG] pgmap v5421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:18 gibba011 bash[30791]: cluster 2021-03-10T16:53:16.376649+0000 mgr.gibba032.whopeo (mgr.24625) 9519 : cluster [DBG] pgmap v5421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:18 gibba011 bash[31774]: cluster 2021-03-10T16:53:16.376649+0000 mgr.gibba032.whopeo (mgr.24625) 9519 : cluster [DBG] pgmap v5421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:20.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:20 gibba032 bash[24004]: cluster 2021-03-10T16:53:18.378374+0000 mgr.gibba032.whopeo (mgr.24625) 9520 : cluster [DBG] pgmap v5422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:20 gibba011 bash[30791]: cluster 2021-03-10T16:53:18.378374+0000 mgr.gibba032.whopeo (mgr.24625) 9520 : cluster [DBG] pgmap v5422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:20.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:20 gibba011 bash[31774]: cluster 2021-03-10T16:53:18.378374+0000 mgr.gibba032.whopeo (mgr.24625) 9520 : cluster [DBG] pgmap v5422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:22 gibba011 bash[30791]: cluster 2021-03-10T16:53:20.379943+0000 mgr.gibba032.whopeo (mgr.24625) 9521 : cluster [DBG] pgmap v5423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:22 gibba011 bash[31774]: cluster 2021-03-10T16:53:20.379943+0000 mgr.gibba032.whopeo (mgr.24625) 9521 : cluster [DBG] pgmap v5423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:22.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:22 gibba032 bash[24004]: cluster 2021-03-10T16:53:20.379943+0000 mgr.gibba032.whopeo (mgr.24625) 9521 : cluster [DBG] pgmap v5423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:24 gibba011 bash[30791]: cluster 2021-03-10T16:53:22.381478+0000 mgr.gibba032.whopeo (mgr.24625) 9522 : cluster [DBG] pgmap v5424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:24 gibba011 bash[31774]: cluster 2021-03-10T16:53:22.381478+0000 mgr.gibba032.whopeo (mgr.24625) 9522 : cluster [DBG] pgmap v5424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:24.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:24 gibba032 bash[24004]: cluster 2021-03-10T16:53:22.381478+0000 mgr.gibba032.whopeo (mgr.24625) 9522 : cluster [DBG] pgmap v5424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:26 gibba011 bash[30791]: cluster 2021-03-10T16:53:24.382967+0000 mgr.gibba032.whopeo (mgr.24625) 9523 : cluster [DBG] pgmap v5425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:26.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:26 gibba011 bash[31774]: cluster 2021-03-10T16:53:24.382967+0000 mgr.gibba032.whopeo (mgr.24625) 9523 : cluster [DBG] pgmap v5425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:26.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:26 gibba032 bash[24004]: cluster 2021-03-10T16:53:24.382967+0000 mgr.gibba032.whopeo (mgr.24625) 9523 : cluster [DBG] pgmap v5425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:28 gibba011 bash[30791]: cluster 2021-03-10T16:53:26.384761+0000 mgr.gibba032.whopeo (mgr.24625) 9524 : cluster [DBG] pgmap v5426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:28.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:28 gibba011 bash[31774]: cluster 2021-03-10T16:53:26.384761+0000 mgr.gibba032.whopeo (mgr.24625) 9524 : cluster [DBG] pgmap v5426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:28.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:28 gibba032 bash[24004]: cluster 2021-03-10T16:53:26.384761+0000 mgr.gibba032.whopeo (mgr.24625) 9524 : cluster [DBG] pgmap v5426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:30 gibba011 bash[30791]: cluster 2021-03-10T16:53:28.386548+0000 mgr.gibba032.whopeo (mgr.24625) 9525 : cluster [DBG] pgmap v5427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:30 gibba011 bash[31774]: cluster 2021-03-10T16:53:28.386548+0000 mgr.gibba032.whopeo (mgr.24625) 9525 : cluster [DBG] pgmap v5427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:30.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:30 gibba032 bash[24004]: cluster 2021-03-10T16:53:28.386548+0000 mgr.gibba032.whopeo (mgr.24625) 9525 : cluster [DBG] pgmap v5427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:31.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:31 gibba011 bash[30791]: cephadm 2021-03-10T16:53:30.279595+0000 mgr.gibba032.whopeo (mgr.24625) 9526 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:31.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:31 gibba011 bash[31774]: cephadm 2021-03-10T16:53:30.279595+0000 mgr.gibba032.whopeo (mgr.24625) 9526 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:31.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:31 gibba032 bash[24004]: cephadm 2021-03-10T16:53:30.279595+0000 mgr.gibba032.whopeo (mgr.24625) 9526 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:32 gibba011 bash[30791]: cluster 2021-03-10T16:53:30.388318+0000 mgr.gibba032.whopeo (mgr.24625) 9527 : cluster [DBG] pgmap v5428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:32 gibba011 bash[31774]: cluster 2021-03-10T16:53:30.388318+0000 mgr.gibba032.whopeo (mgr.24625) 9527 : cluster [DBG] pgmap v5428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:32.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:32 gibba032 bash[24004]: cluster 2021-03-10T16:53:30.388318+0000 mgr.gibba032.whopeo (mgr.24625) 9527 : cluster [DBG] pgmap v5428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:33.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:33 gibba011 bash[30791]: cluster 2021-03-10T16:53:32.389605+0000 mgr.gibba032.whopeo (mgr.24625) 9528 : cluster [DBG] pgmap v5429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:33.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:33 gibba011 bash[31774]: cluster 2021-03-10T16:53:32.389605+0000 mgr.gibba032.whopeo (mgr.24625) 9528 : cluster [DBG] pgmap v5429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:33.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:33 gibba032 bash[24004]: cluster 2021-03-10T16:53:32.389605+0000 mgr.gibba032.whopeo (mgr.24625) 9528 : cluster [DBG] pgmap v5429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:36.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:36 gibba032 bash[24004]: cluster 2021-03-10T16:53:34.391134+0000 mgr.gibba032.whopeo (mgr.24625) 9529 : cluster [DBG] pgmap v5430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:36 gibba011 bash[30791]: cluster 2021-03-10T16:53:34.391134+0000 mgr.gibba032.whopeo (mgr.24625) 9529 : cluster [DBG] pgmap v5430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:36 gibba011 bash[31774]: cluster 2021-03-10T16:53:34.391134+0000 mgr.gibba032.whopeo (mgr.24625) 9529 : cluster [DBG] pgmap v5430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:38 gibba011 bash[30791]: cluster 2021-03-10T16:53:36.393118+0000 mgr.gibba032.whopeo (mgr.24625) 9530 : cluster [DBG] pgmap v5431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:38 gibba011 bash[31774]: cluster 2021-03-10T16:53:36.393118+0000 mgr.gibba032.whopeo (mgr.24625) 9530 : cluster [DBG] pgmap v5431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:38.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:38 gibba032 bash[24004]: cluster 2021-03-10T16:53:36.393118+0000 mgr.gibba032.whopeo (mgr.24625) 9530 : cluster [DBG] pgmap v5431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:39.749 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:53:39.975 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:53:39.976 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:53:39.976 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 104s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:53:39.976 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:53:39.976 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:53:39.977 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:53:39.977 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:53:39.977 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:53:39.978 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:53:39.978 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:53:39.978 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 104s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:53:39.978 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:53:39.979 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:53:39.979 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:53:39.979 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:53:39.979 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:53:39.979 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:53:39.980 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:53:39.980 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:53:39.980 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 104s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:53:39.980 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:53:40.228 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:53:40.228 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:53:40.228 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:53:40.229 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:40.229 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:53:40.229 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:53:40.229 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:40.229 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:53:40.230 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:53:40.230 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:53:40.230 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:53:40.230 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:53:40.232 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:53:40.232 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:53:40.233 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:53:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:40 gibba011 bash[30791]: cluster 2021-03-10T16:53:38.394903+0000 mgr.gibba032.whopeo (mgr.24625) 9531 : cluster [DBG] pgmap v5432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:40 gibba011 bash[30791]: audit 2021-03-10T16:53:40.229964+0000 mon.c (mon.1) 185 : audit [DBG] from='client.? 172.21.2.111:0/3295782231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:40 gibba011 bash[31774]: cluster 2021-03-10T16:53:38.394903+0000 mgr.gibba032.whopeo (mgr.24625) 9531 : cluster [DBG] pgmap v5432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:40 gibba011 bash[31774]: audit 2021-03-10T16:53:40.229964+0000 mon.c (mon.1) 185 : audit [DBG] from='client.? 172.21.2.111:0/3295782231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:40.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:40 gibba032 bash[24004]: cluster 2021-03-10T16:53:38.394903+0000 mgr.gibba032.whopeo (mgr.24625) 9531 : cluster [DBG] pgmap v5432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:40.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:40 gibba032 bash[24004]: audit 2021-03-10T16:53:40.229964+0000 mon.c (mon.1) 185 : audit [DBG] from='client.? 172.21.2.111:0/3295782231' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:53:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:41 gibba011 bash[30791]: audit 2021-03-10T16:53:39.740769+0000 mgr.gibba032.whopeo (mgr.24625) 9532 : audit [DBG] from='client.50445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:41.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:41 gibba011 bash[30791]: audit 2021-03-10T16:53:39.964224+0000 mgr.gibba032.whopeo (mgr.24625) 9533 : audit [DBG] from='client.50451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:41.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:41 gibba011 bash[31774]: audit 2021-03-10T16:53:39.740769+0000 mgr.gibba032.whopeo (mgr.24625) 9532 : audit [DBG] from='client.50445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:41.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:41 gibba011 bash[31774]: audit 2021-03-10T16:53:39.964224+0000 mgr.gibba032.whopeo (mgr.24625) 9533 : audit [DBG] from='client.50451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:41.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:41 gibba032 bash[24004]: audit 2021-03-10T16:53:39.740769+0000 mgr.gibba032.whopeo (mgr.24625) 9532 : audit [DBG] from='client.50445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:41.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:41 gibba032 bash[24004]: audit 2021-03-10T16:53:39.964224+0000 mgr.gibba032.whopeo (mgr.24625) 9533 : audit [DBG] from='client.50451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:53:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:42 gibba011 bash[30791]: cluster 2021-03-10T16:53:40.396534+0000 mgr.gibba032.whopeo (mgr.24625) 9534 : cluster [DBG] pgmap v5433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:42.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:42 gibba011 bash[31774]: cluster 2021-03-10T16:53:40.396534+0000 mgr.gibba032.whopeo (mgr.24625) 9534 : cluster [DBG] pgmap v5433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:42.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:42 gibba032 bash[24004]: cluster 2021-03-10T16:53:40.396534+0000 mgr.gibba032.whopeo (mgr.24625) 9534 : cluster [DBG] pgmap v5433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:44 gibba011 bash[30791]: cluster 2021-03-10T16:53:42.398036+0000 mgr.gibba032.whopeo (mgr.24625) 9535 : cluster [DBG] pgmap v5434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:44 gibba011 bash[31774]: cluster 2021-03-10T16:53:42.398036+0000 mgr.gibba032.whopeo (mgr.24625) 9535 : cluster [DBG] pgmap v5434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:44.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:44 gibba032 bash[24004]: cluster 2021-03-10T16:53:42.398036+0000 mgr.gibba032.whopeo (mgr.24625) 9535 : cluster [DBG] pgmap v5434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:46 gibba011 bash[30791]: cluster 2021-03-10T16:53:44.399407+0000 mgr.gibba032.whopeo (mgr.24625) 9536 : cluster [DBG] pgmap v5435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:46 gibba011 bash[30791]: cephadm 2021-03-10T16:53:45.285048+0000 mgr.gibba032.whopeo (mgr.24625) 9537 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:46 gibba011 bash[31774]: cluster 2021-03-10T16:53:44.399407+0000 mgr.gibba032.whopeo (mgr.24625) 9536 : cluster [DBG] pgmap v5435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:46 gibba011 bash[31774]: cephadm 2021-03-10T16:53:45.285048+0000 mgr.gibba032.whopeo (mgr.24625) 9537 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:46.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:46 gibba032 bash[24004]: cluster 2021-03-10T16:53:44.399407+0000 mgr.gibba032.whopeo (mgr.24625) 9536 : cluster [DBG] pgmap v5435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:46 gibba032 bash[24004]: cephadm 2021-03-10T16:53:45.285048+0000 mgr.gibba032.whopeo (mgr.24625) 9537 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:53:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:48 gibba011 bash[30791]: cluster 2021-03-10T16:53:46.401108+0000 mgr.gibba032.whopeo (mgr.24625) 9538 : cluster [DBG] pgmap v5436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:48.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:48 gibba011 bash[31774]: cluster 2021-03-10T16:53:46.401108+0000 mgr.gibba032.whopeo (mgr.24625) 9538 : cluster [DBG] pgmap v5436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:48 gibba032 bash[24004]: cluster 2021-03-10T16:53:46.401108+0000 mgr.gibba032.whopeo (mgr.24625) 9538 : cluster [DBG] pgmap v5436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:49.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:49 gibba011 bash[30791]: cluster 2021-03-10T16:53:48.402756+0000 mgr.gibba032.whopeo (mgr.24625) 9539 : cluster [DBG] pgmap v5437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:49.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:49 gibba011 bash[31774]: cluster 2021-03-10T16:53:48.402756+0000 mgr.gibba032.whopeo (mgr.24625) 9539 : cluster [DBG] pgmap v5437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:49.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:49 gibba032 bash[24004]: cluster 2021-03-10T16:53:48.402756+0000 mgr.gibba032.whopeo (mgr.24625) 9539 : cluster [DBG] pgmap v5437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:52 gibba011 bash[30791]: cluster 2021-03-10T16:53:50.404117+0000 mgr.gibba032.whopeo (mgr.24625) 9540 : cluster [DBG] pgmap v5438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:52 gibba011 bash[31774]: cluster 2021-03-10T16:53:50.404117+0000 mgr.gibba032.whopeo (mgr.24625) 9540 : cluster [DBG] pgmap v5438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:52.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:52 gibba032 bash[24004]: cluster 2021-03-10T16:53:50.404117+0000 mgr.gibba032.whopeo (mgr.24625) 9540 : cluster [DBG] pgmap v5438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:54 gibba011 bash[30791]: cluster 2021-03-10T16:53:52.405801+0000 mgr.gibba032.whopeo (mgr.24625) 9541 : cluster [DBG] pgmap v5439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:54.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:54 gibba011 bash[31774]: cluster 2021-03-10T16:53:52.405801+0000 mgr.gibba032.whopeo (mgr.24625) 9541 : cluster [DBG] pgmap v5439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:54.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:54 gibba032 bash[24004]: cluster 2021-03-10T16:53:52.405801+0000 mgr.gibba032.whopeo (mgr.24625) 9541 : cluster [DBG] pgmap v5439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:56.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:56 gibba011 bash[30791]: cluster 2021-03-10T16:53:54.407024+0000 mgr.gibba032.whopeo (mgr.24625) 9542 : cluster [DBG] pgmap v5440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:56.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:56 gibba011 bash[31774]: cluster 2021-03-10T16:53:54.407024+0000 mgr.gibba032.whopeo (mgr.24625) 9542 : cluster [DBG] pgmap v5440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:56.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:56 gibba032 bash[24004]: cluster 2021-03-10T16:53:54.407024+0000 mgr.gibba032.whopeo (mgr.24625) 9542 : cluster [DBG] pgmap v5440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:57 gibba011 bash[30791]: audit 2021-03-10T16:53:57.215779+0000 mon.a (mon.0) 3242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:53:57.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:57 gibba011 bash[30791]: audit 2021-03-10T16:53:57.216450+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:53:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:57 gibba011 bash[31774]: audit 2021-03-10T16:53:57.215779+0000 mon.a (mon.0) 3242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:53:57.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:57 gibba011 bash[31774]: audit 2021-03-10T16:53:57.216450+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:53:57.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:57 gibba032 bash[24004]: audit 2021-03-10T16:53:57.215779+0000 mon.a (mon.0) 3242 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:53:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:57 gibba032 bash[24004]: audit 2021-03-10T16:53:57.216450+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:53:58.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[30791]: cluster 2021-03-10T16:53:56.408820+0000 mgr.gibba032.whopeo (mgr.24625) 9543 : cluster [DBG] pgmap v5441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[30791]: audit 2021-03-10T16:53:57.369184+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:53:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[30791]: audit 2021-03-10T16:53:57.369956+0000 mon.a (mon.0) 3245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[30791]: audit 2021-03-10T16:53:57.370651+0000 mon.a (mon.0) 3246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[30791]: audit 2021-03-10T16:53:57.371377+0000 mon.a (mon.0) 3247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:53:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[31774]: cluster 2021-03-10T16:53:56.408820+0000 mgr.gibba032.whopeo (mgr.24625) 9543 : cluster [DBG] pgmap v5441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[31774]: audit 2021-03-10T16:53:57.369184+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:53:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[31774]: audit 2021-03-10T16:53:57.369956+0000 mon.a (mon.0) 3245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[31774]: audit 2021-03-10T16:53:57.370651+0000 mon.a (mon.0) 3246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:53:58 gibba011 bash[31774]: audit 2021-03-10T16:53:57.371377+0000 mon.a (mon.0) 3247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:53:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:58 gibba032 bash[24004]: cluster 2021-03-10T16:53:56.408820+0000 mgr.gibba032.whopeo (mgr.24625) 9543 : cluster [DBG] pgmap v5441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:53:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:58 gibba032 bash[24004]: audit 2021-03-10T16:53:57.369184+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T16:53:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:58 gibba032 bash[24004]: audit 2021-03-10T16:53:57.369956+0000 mon.a (mon.0) 3245 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:58 gibba032 bash[24004]: audit 2021-03-10T16:53:57.370651+0000 mon.a (mon.0) 3246 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T16:53:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:53:58 gibba032 bash[24004]: audit 2021-03-10T16:53:57.371377+0000 mon.a (mon.0) 3247 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T16:54:00.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:00 gibba011 bash[30791]: cluster 2021-03-10T16:53:58.410411+0000 mgr.gibba032.whopeo (mgr.24625) 9544 : cluster [DBG] pgmap v5442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:00.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:00 gibba011 bash[31774]: cluster 2021-03-10T16:53:58.410411+0000 mgr.gibba032.whopeo (mgr.24625) 9544 : cluster [DBG] pgmap v5442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:00.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:00 gibba032 bash[24004]: cluster 2021-03-10T16:53:58.410411+0000 mgr.gibba032.whopeo (mgr.24625) 9544 : cluster [DBG] pgmap v5442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:01.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.300499+0000 mgr.gibba032.whopeo (mgr.24625) 9545 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:54:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.300909+0000 mgr.gibba032.whopeo (mgr.24625) 9546 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:54:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.516211+0000 mon.a (mon.0) 3248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:54:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.519802+0000 mon.a (mon.0) 3249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:54:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.521030+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.522441+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.523632+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.525677+0000 mon.a (mon.0) 3253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[30791]: audit 2021-03-10T16:54:00.526997+0000 mon.a (mon.0) 3254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.300499+0000 mgr.gibba032.whopeo (mgr.24625) 9545 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:54:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.300909+0000 mgr.gibba032.whopeo (mgr.24625) 9546 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:54:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.516211+0000 mon.a (mon.0) 3248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:54:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.519802+0000 mon.a (mon.0) 3249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:54:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.521030+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.522441+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.523632+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.525677+0000 mon.a (mon.0) 3253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:01 gibba011 bash[31774]: audit 2021-03-10T16:54:00.526997+0000 mon.a (mon.0) 3254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.300499+0000 mgr.gibba032.whopeo (mgr.24625) 9545 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:54:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.300909+0000 mgr.gibba032.whopeo (mgr.24625) 9546 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:54:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.516211+0000 mon.a (mon.0) 3248 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:54:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.519802+0000 mon.a (mon.0) 3249 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:54:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.521030+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.522441+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.523632+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.525677+0000 mon.a (mon.0) 3253 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:01.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:01 gibba032 bash[24004]: audit 2021-03-10T16:54:00.526997+0000 mon.a (mon.0) 3254 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cluster 2021-03-10T16:54:00.410937+0000 mgr.gibba032.whopeo (mgr.24625) 9547 : cluster [DBG] pgmap v5443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.517468+0000 mgr.gibba032.whopeo (mgr.24625) 9548 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:54:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.518996+0000 mgr.gibba032.whopeo (mgr.24625) 9549 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:54:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.520251+0000 mgr.gibba032.whopeo (mgr.24625) 9550 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:54:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.520290+0000 mgr.gibba032.whopeo (mgr.24625) 9551 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:54:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.521669+0000 mgr.gibba032.whopeo (mgr.24625) 9552 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:54:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.521708+0000 mgr.gibba032.whopeo (mgr.24625) 9553 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:54:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.522648+0000 mgr.gibba032.whopeo (mgr.24625) 9554 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:54:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.523498+0000 mgr.gibba032.whopeo (mgr.24625) 9555 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:54:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.523535+0000 mgr.gibba032.whopeo (mgr.24625) 9556 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:54:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.524680+0000 mgr.gibba032.whopeo (mgr.24625) 9557 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:54:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.525004+0000 mgr.gibba032.whopeo (mgr.24625) 9558 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:54:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.526037+0000 mgr.gibba032.whopeo (mgr.24625) 9559 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:54:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.526076+0000 mgr.gibba032.whopeo (mgr.24625) 9560 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:54:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[30791]: cephadm 2021-03-10T16:54:00.526182+0000 mgr.gibba032.whopeo (mgr.24625) 9561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cluster 2021-03-10T16:54:00.410937+0000 mgr.gibba032.whopeo (mgr.24625) 9547 : cluster [DBG] pgmap v5443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.517468+0000 mgr.gibba032.whopeo (mgr.24625) 9548 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:54:02.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.518996+0000 mgr.gibba032.whopeo (mgr.24625) 9549 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:54:02.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.520251+0000 mgr.gibba032.whopeo (mgr.24625) 9550 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:54:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.520290+0000 mgr.gibba032.whopeo (mgr.24625) 9551 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:54:02.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.521669+0000 mgr.gibba032.whopeo (mgr.24625) 9552 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:54:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.521708+0000 mgr.gibba032.whopeo (mgr.24625) 9553 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:54:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.522648+0000 mgr.gibba032.whopeo (mgr.24625) 9554 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:54:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.523498+0000 mgr.gibba032.whopeo (mgr.24625) 9555 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:54:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.523535+0000 mgr.gibba032.whopeo (mgr.24625) 9556 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:54:02.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.524680+0000 mgr.gibba032.whopeo (mgr.24625) 9557 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:54:02.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.525004+0000 mgr.gibba032.whopeo (mgr.24625) 9558 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:54:02.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.526037+0000 mgr.gibba032.whopeo (mgr.24625) 9559 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:54:02.620 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.526076+0000 mgr.gibba032.whopeo (mgr.24625) 9560 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:54:02.621 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:02 gibba011 bash[31774]: cephadm 2021-03-10T16:54:00.526182+0000 mgr.gibba032.whopeo (mgr.24625) 9561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cluster 2021-03-10T16:54:00.410937+0000 mgr.gibba032.whopeo (mgr.24625) 9547 : cluster [DBG] pgmap v5443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.517468+0000 mgr.gibba032.whopeo (mgr.24625) 9548 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:54:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.518996+0000 mgr.gibba032.whopeo (mgr.24625) 9549 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:54:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.520251+0000 mgr.gibba032.whopeo (mgr.24625) 9550 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:54:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.520290+0000 mgr.gibba032.whopeo (mgr.24625) 9551 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:54:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.521669+0000 mgr.gibba032.whopeo (mgr.24625) 9552 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:54:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.521708+0000 mgr.gibba032.whopeo (mgr.24625) 9553 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:54:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.522648+0000 mgr.gibba032.whopeo (mgr.24625) 9554 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:54:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.523498+0000 mgr.gibba032.whopeo (mgr.24625) 9555 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:54:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.523535+0000 mgr.gibba032.whopeo (mgr.24625) 9556 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:54:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.524680+0000 mgr.gibba032.whopeo (mgr.24625) 9557 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:54:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.525004+0000 mgr.gibba032.whopeo (mgr.24625) 9558 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:54:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.526037+0000 mgr.gibba032.whopeo (mgr.24625) 9559 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:54:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.526076+0000 mgr.gibba032.whopeo (mgr.24625) 9560 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:54:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:02 gibba032 bash[24004]: cephadm 2021-03-10T16:54:00.526182+0000 mgr.gibba032.whopeo (mgr.24625) 9561 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:03.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:03 gibba011 bash[30791]: cluster 2021-03-10T16:54:02.412645+0000 mgr.gibba032.whopeo (mgr.24625) 9562 : cluster [DBG] pgmap v5444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:03.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:03 gibba011 bash[31774]: cluster 2021-03-10T16:54:02.412645+0000 mgr.gibba032.whopeo (mgr.24625) 9562 : cluster [DBG] pgmap v5444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:03.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:03 gibba032 bash[24004]: cluster 2021-03-10T16:54:02.412645+0000 mgr.gibba032.whopeo (mgr.24625) 9562 : cluster [DBG] pgmap v5444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:06 gibba011 bash[30791]: cluster 2021-03-10T16:54:04.413923+0000 mgr.gibba032.whopeo (mgr.24625) 9563 : cluster [DBG] pgmap v5445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:06 gibba011 bash[31774]: cluster 2021-03-10T16:54:04.413923+0000 mgr.gibba032.whopeo (mgr.24625) 9563 : cluster [DBG] pgmap v5445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:06 gibba032 bash[24004]: cluster 2021-03-10T16:54:04.413923+0000 mgr.gibba032.whopeo (mgr.24625) 9563 : cluster [DBG] pgmap v5445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:08 gibba011 bash[30791]: cluster 2021-03-10T16:54:06.415899+0000 mgr.gibba032.whopeo (mgr.24625) 9564 : cluster [DBG] pgmap v5446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:08 gibba011 bash[31774]: cluster 2021-03-10T16:54:06.415899+0000 mgr.gibba032.whopeo (mgr.24625) 9564 : cluster [DBG] pgmap v5446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:08.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:08 gibba032 bash[24004]: cluster 2021-03-10T16:54:06.415899+0000 mgr.gibba032.whopeo (mgr.24625) 9564 : cluster [DBG] pgmap v5446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:10.475 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:54:10.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:10 gibba011 bash[30791]: cluster 2021-03-10T16:54:08.417535+0000 mgr.gibba032.whopeo (mgr.24625) 9565 : cluster [DBG] pgmap v5447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:10 gibba011 bash[31774]: cluster 2021-03-10T16:54:08.417535+0000 mgr.gibba032.whopeo (mgr.24625) 9565 : cluster [DBG] pgmap v5447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:10.722 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:54:10.723 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:54:10.723 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:54:10.724 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:54:10.724 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:54:10.725 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:54:10.725 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:54:10.725 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:54:10.726 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:54:10.726 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:54:10.726 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:54:10.726 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:54:10.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:54:10.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:54:10.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:54:10.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:54:10.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:54:10.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:54:10.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:54:10.728 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:54:10.728 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:54:10.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:10 gibba032 bash[24004]: cluster 2021-03-10T16:54:08.417535+0000 mgr.gibba032.whopeo (mgr.24625) 9565 : cluster [DBG] pgmap v5447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:10.974 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:54:10.974 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:54:10.975 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:54:10.975 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:10.976 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:54:10.976 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:54:10.977 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:10.977 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:54:10.977 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:54:10.978 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:10.982 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:54:10.983 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:54:10.983 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:54:10.983 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:54:10.984 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:54:11.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:11 gibba011 bash[30791]: audit 2021-03-10T16:54:10.975645+0000 mon.c (mon.1) 186 : audit [DBG] from='client.? 172.21.2.111:0/2583157776' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:11.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:11 gibba011 bash[31774]: audit 2021-03-10T16:54:10.975645+0000 mon.c (mon.1) 186 : audit [DBG] from='client.? 172.21.2.111:0/2583157776' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:11 gibba032 bash[24004]: audit 2021-03-10T16:54:10.975645+0000 mon.c (mon.1) 186 : audit [DBG] from='client.? 172.21.2.111:0/2583157776' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:12.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[30791]: cluster 2021-03-10T16:54:10.419033+0000 mgr.gibba032.whopeo (mgr.24625) 9566 : cluster [DBG] pgmap v5448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[30791]: audit 2021-03-10T16:54:10.466729+0000 mgr.gibba032.whopeo (mgr.24625) 9567 : audit [DBG] from='client.60445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:12.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[30791]: audit 2021-03-10T16:54:10.692879+0000 mgr.gibba032.whopeo (mgr.24625) 9568 : audit [DBG] from='client.60451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[31774]: cluster 2021-03-10T16:54:10.419033+0000 mgr.gibba032.whopeo (mgr.24625) 9566 : cluster [DBG] pgmap v5448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[31774]: audit 2021-03-10T16:54:10.466729+0000 mgr.gibba032.whopeo (mgr.24625) 9567 : audit [DBG] from='client.60445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:12 gibba011 bash[31774]: audit 2021-03-10T16:54:10.692879+0000 mgr.gibba032.whopeo (mgr.24625) 9568 : audit [DBG] from='client.60451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:12 gibba032 bash[24004]: cluster 2021-03-10T16:54:10.419033+0000 mgr.gibba032.whopeo (mgr.24625) 9566 : cluster [DBG] pgmap v5448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:12 gibba032 bash[24004]: audit 2021-03-10T16:54:10.466729+0000 mgr.gibba032.whopeo (mgr.24625) 9567 : audit [DBG] from='client.60445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:12 gibba032 bash[24004]: audit 2021-03-10T16:54:10.692879+0000 mgr.gibba032.whopeo (mgr.24625) 9568 : audit [DBG] from='client.60451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:14.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:14 gibba011 bash[30791]: cluster 2021-03-10T16:54:12.420750+0000 mgr.gibba032.whopeo (mgr.24625) 9569 : cluster [DBG] pgmap v5449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:14.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:14 gibba011 bash[31774]: cluster 2021-03-10T16:54:12.420750+0000 mgr.gibba032.whopeo (mgr.24625) 9569 : cluster [DBG] pgmap v5449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:14 gibba032 bash[24004]: cluster 2021-03-10T16:54:12.420750+0000 mgr.gibba032.whopeo (mgr.24625) 9569 : cluster [DBG] pgmap v5449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:16.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:16 gibba011 bash[30791]: cluster 2021-03-10T16:54:14.422004+0000 mgr.gibba032.whopeo (mgr.24625) 9570 : cluster [DBG] pgmap v5450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:16 gibba011 bash[31774]: cluster 2021-03-10T16:54:14.422004+0000 mgr.gibba032.whopeo (mgr.24625) 9570 : cluster [DBG] pgmap v5450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:16 gibba032 bash[24004]: cluster 2021-03-10T16:54:14.422004+0000 mgr.gibba032.whopeo (mgr.24625) 9570 : cluster [DBG] pgmap v5450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:17.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:17 gibba011 bash[30791]: cephadm 2021-03-10T16:54:15.542177+0000 mgr.gibba032.whopeo (mgr.24625) 9571 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:17.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:17 gibba011 bash[31774]: cephadm 2021-03-10T16:54:15.542177+0000 mgr.gibba032.whopeo (mgr.24625) 9571 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:17 gibba032 bash[24004]: cephadm 2021-03-10T16:54:15.542177+0000 mgr.gibba032.whopeo (mgr.24625) 9571 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:18.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:18 gibba011 bash[30791]: cluster 2021-03-10T16:54:16.423881+0000 mgr.gibba032.whopeo (mgr.24625) 9572 : cluster [DBG] pgmap v5451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:18.639 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:18 gibba011 bash[31774]: cluster 2021-03-10T16:54:16.423881+0000 mgr.gibba032.whopeo (mgr.24625) 9572 : cluster [DBG] pgmap v5451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:18.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:18 gibba032 bash[24004]: cluster 2021-03-10T16:54:16.423881+0000 mgr.gibba032.whopeo (mgr.24625) 9572 : cluster [DBG] pgmap v5451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:20.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:20 gibba011 bash[30791]: cluster 2021-03-10T16:54:18.425609+0000 mgr.gibba032.whopeo (mgr.24625) 9573 : cluster [DBG] pgmap v5452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:20.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:20 gibba011 bash[31774]: cluster 2021-03-10T16:54:18.425609+0000 mgr.gibba032.whopeo (mgr.24625) 9573 : cluster [DBG] pgmap v5452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:20.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:20 gibba032 bash[24004]: cluster 2021-03-10T16:54:18.425609+0000 mgr.gibba032.whopeo (mgr.24625) 9573 : cluster [DBG] pgmap v5452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:22.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:22 gibba011 bash[30791]: cluster 2021-03-10T16:54:20.427029+0000 mgr.gibba032.whopeo (mgr.24625) 9574 : cluster [DBG] pgmap v5453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:22.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:22 gibba011 bash[31774]: cluster 2021-03-10T16:54:20.427029+0000 mgr.gibba032.whopeo (mgr.24625) 9574 : cluster [DBG] pgmap v5453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:22 gibba032 bash[24004]: cluster 2021-03-10T16:54:20.427029+0000 mgr.gibba032.whopeo (mgr.24625) 9574 : cluster [DBG] pgmap v5453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:24.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:24 gibba011 bash[30791]: cluster 2021-03-10T16:54:22.428845+0000 mgr.gibba032.whopeo (mgr.24625) 9575 : cluster [DBG] pgmap v5454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:24.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:24 gibba011 bash[31774]: cluster 2021-03-10T16:54:22.428845+0000 mgr.gibba032.whopeo (mgr.24625) 9575 : cluster [DBG] pgmap v5454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:24.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:24 gibba032 bash[24004]: cluster 2021-03-10T16:54:22.428845+0000 mgr.gibba032.whopeo (mgr.24625) 9575 : cluster [DBG] pgmap v5454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:26 gibba011 bash[30791]: cluster 2021-03-10T16:54:24.430151+0000 mgr.gibba032.whopeo (mgr.24625) 9576 : cluster [DBG] pgmap v5455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:26.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:26 gibba011 bash[31774]: cluster 2021-03-10T16:54:24.430151+0000 mgr.gibba032.whopeo (mgr.24625) 9576 : cluster [DBG] pgmap v5455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:26 gibba032 bash[24004]: cluster 2021-03-10T16:54:24.430151+0000 mgr.gibba032.whopeo (mgr.24625) 9576 : cluster [DBG] pgmap v5455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:28 gibba011 bash[30791]: cluster 2021-03-10T16:54:26.431827+0000 mgr.gibba032.whopeo (mgr.24625) 9577 : cluster [DBG] pgmap v5456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:28 gibba011 bash[31774]: cluster 2021-03-10T16:54:26.431827+0000 mgr.gibba032.whopeo (mgr.24625) 9577 : cluster [DBG] pgmap v5456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:28 gibba032 bash[24004]: cluster 2021-03-10T16:54:26.431827+0000 mgr.gibba032.whopeo (mgr.24625) 9577 : cluster [DBG] pgmap v5456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:29.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:29 gibba032 bash[24004]: cluster 2021-03-10T16:54:28.433329+0000 mgr.gibba032.whopeo (mgr.24625) 9578 : cluster [DBG] pgmap v5457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:29.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:29 gibba011 bash[30791]: cluster 2021-03-10T16:54:28.433329+0000 mgr.gibba032.whopeo (mgr.24625) 9578 : cluster [DBG] pgmap v5457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:29.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:29 gibba011 bash[31774]: cluster 2021-03-10T16:54:28.433329+0000 mgr.gibba032.whopeo (mgr.24625) 9578 : cluster [DBG] pgmap v5457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:32 gibba011 bash[30791]: cluster 2021-03-10T16:54:30.434837+0000 mgr.gibba032.whopeo (mgr.24625) 9579 : cluster [DBG] pgmap v5458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:32 gibba011 bash[30791]: cephadm 2021-03-10T16:54:30.558170+0000 mgr.gibba032.whopeo (mgr.24625) 9580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:32 gibba011 bash[31774]: cluster 2021-03-10T16:54:30.434837+0000 mgr.gibba032.whopeo (mgr.24625) 9579 : cluster [DBG] pgmap v5458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:32 gibba011 bash[31774]: cephadm 2021-03-10T16:54:30.558170+0000 mgr.gibba032.whopeo (mgr.24625) 9580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:32 gibba032 bash[24004]: cluster 2021-03-10T16:54:30.434837+0000 mgr.gibba032.whopeo (mgr.24625) 9579 : cluster [DBG] pgmap v5458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:32 gibba032 bash[24004]: cephadm 2021-03-10T16:54:30.558170+0000 mgr.gibba032.whopeo (mgr.24625) 9580 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:34.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:34 gibba011 bash[30791]: cluster 2021-03-10T16:54:32.436605+0000 mgr.gibba032.whopeo (mgr.24625) 9581 : cluster [DBG] pgmap v5459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:34 gibba011 bash[31774]: cluster 2021-03-10T16:54:32.436605+0000 mgr.gibba032.whopeo (mgr.24625) 9581 : cluster [DBG] pgmap v5459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:34.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:34 gibba032 bash[24004]: cluster 2021-03-10T16:54:32.436605+0000 mgr.gibba032.whopeo (mgr.24625) 9581 : cluster [DBG] pgmap v5459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:36.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:36 gibba011 bash[30791]: cluster 2021-03-10T16:54:34.437963+0000 mgr.gibba032.whopeo (mgr.24625) 9582 : cluster [DBG] pgmap v5460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:36.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:36 gibba011 bash[31774]: cluster 2021-03-10T16:54:34.437963+0000 mgr.gibba032.whopeo (mgr.24625) 9582 : cluster [DBG] pgmap v5460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:36 gibba032 bash[24004]: cluster 2021-03-10T16:54:34.437963+0000 mgr.gibba032.whopeo (mgr.24625) 9582 : cluster [DBG] pgmap v5460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:38.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:38 gibba011 bash[30791]: cluster 2021-03-10T16:54:36.439688+0000 mgr.gibba032.whopeo (mgr.24625) 9583 : cluster [DBG] pgmap v5461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:38.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:38 gibba011 bash[31774]: cluster 2021-03-10T16:54:36.439688+0000 mgr.gibba032.whopeo (mgr.24625) 9583 : cluster [DBG] pgmap v5461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:38.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:38 gibba032 bash[24004]: cluster 2021-03-10T16:54:36.439688+0000 mgr.gibba032.whopeo (mgr.24625) 9583 : cluster [DBG] pgmap v5461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:40.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:40 gibba011 bash[30791]: cluster 2021-03-10T16:54:38.441303+0000 mgr.gibba032.whopeo (mgr.24625) 9584 : cluster [DBG] pgmap v5462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:40.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:40 gibba011 bash[31774]: cluster 2021-03-10T16:54:38.441303+0000 mgr.gibba032.whopeo (mgr.24625) 9584 : cluster [DBG] pgmap v5462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:40.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:40 gibba032 bash[24004]: cluster 2021-03-10T16:54:38.441303+0000 mgr.gibba032.whopeo (mgr.24625) 9584 : cluster [DBG] pgmap v5462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:41.222 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:54:41.453 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:54:41.453 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:54:41.453 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:54:41.454 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:54:41.454 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:54:41.454 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:54:41.454 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:54:41.455 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:54:41.455 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:54:41.455 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:54:41.455 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:54:41.455 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:54:41.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:54:41.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:54:41.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:54:41.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:54:41.456 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:54:41.457 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:54:41.458 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:54:41.458 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:54:41.458 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:54:41.700 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:54:41.700 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:54:41.700 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:54:41.700 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:41.701 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:54:41.701 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:54:41.701 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:41.701 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:54:41.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:54:41.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:54:41.702 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:54:41.702 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:54:41.702 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:54:41.703 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:54:41.703 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:54:42.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[30791]: cluster 2021-03-10T16:54:40.442805+0000 mgr.gibba032.whopeo (mgr.24625) 9585 : cluster [DBG] pgmap v5463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[30791]: audit 2021-03-10T16:54:41.212762+0000 mgr.gibba032.whopeo (mgr.24625) 9586 : audit [DBG] from='client.50481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[30791]: audit 2021-03-10T16:54:41.701796+0000 mon.a (mon.0) 3255 : audit [DBG] from='client.? 172.21.2.111:0/3840635139' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[31774]: cluster 2021-03-10T16:54:40.442805+0000 mgr.gibba032.whopeo (mgr.24625) 9585 : cluster [DBG] pgmap v5463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[31774]: audit 2021-03-10T16:54:41.212762+0000 mgr.gibba032.whopeo (mgr.24625) 9586 : audit [DBG] from='client.50481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:42.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:42 gibba011 bash[31774]: audit 2021-03-10T16:54:41.701796+0000 mon.a (mon.0) 3255 : audit [DBG] from='client.? 172.21.2.111:0/3840635139' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:42 gibba032 bash[24004]: cluster 2021-03-10T16:54:40.442805+0000 mgr.gibba032.whopeo (mgr.24625) 9585 : cluster [DBG] pgmap v5463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:42 gibba032 bash[24004]: audit 2021-03-10T16:54:41.212762+0000 mgr.gibba032.whopeo (mgr.24625) 9586 : audit [DBG] from='client.50481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:42 gibba032 bash[24004]: audit 2021-03-10T16:54:41.701796+0000 mon.a (mon.0) 3255 : audit [DBG] from='client.? 172.21.2.111:0/3840635139' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:54:43.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:43 gibba011 bash[30791]: audit 2021-03-10T16:54:41.441273+0000 mgr.gibba032.whopeo (mgr.24625) 9587 : audit [DBG] from='client.50487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:43.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:43 gibba011 bash[31774]: audit 2021-03-10T16:54:41.441273+0000 mgr.gibba032.whopeo (mgr.24625) 9587 : audit [DBG] from='client.50487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:43.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:43 gibba032 bash[24004]: audit 2021-03-10T16:54:41.441273+0000 mgr.gibba032.whopeo (mgr.24625) 9587 : audit [DBG] from='client.50487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:54:44.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:44 gibba011 bash[30791]: cluster 2021-03-10T16:54:42.444530+0000 mgr.gibba032.whopeo (mgr.24625) 9588 : cluster [DBG] pgmap v5464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:44.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:44 gibba011 bash[31774]: cluster 2021-03-10T16:54:42.444530+0000 mgr.gibba032.whopeo (mgr.24625) 9588 : cluster [DBG] pgmap v5464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:44.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:44 gibba032 bash[24004]: cluster 2021-03-10T16:54:42.444530+0000 mgr.gibba032.whopeo (mgr.24625) 9588 : cluster [DBG] pgmap v5464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:46.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:46 gibba011 bash[30791]: cluster 2021-03-10T16:54:44.445823+0000 mgr.gibba032.whopeo (mgr.24625) 9589 : cluster [DBG] pgmap v5465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:46.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:46 gibba011 bash[31774]: cluster 2021-03-10T16:54:44.445823+0000 mgr.gibba032.whopeo (mgr.24625) 9589 : cluster [DBG] pgmap v5465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:46 gibba032 bash[24004]: cluster 2021-03-10T16:54:44.445823+0000 mgr.gibba032.whopeo (mgr.24625) 9589 : cluster [DBG] pgmap v5465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:47.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:47 gibba011 bash[30791]: cephadm 2021-03-10T16:54:45.573951+0000 mgr.gibba032.whopeo (mgr.24625) 9590 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:47.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:47 gibba011 bash[31774]: cephadm 2021-03-10T16:54:45.573951+0000 mgr.gibba032.whopeo (mgr.24625) 9590 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:47 gibba032 bash[24004]: cephadm 2021-03-10T16:54:45.573951+0000 mgr.gibba032.whopeo (mgr.24625) 9590 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:54:48.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:48 gibba011 bash[30791]: cluster 2021-03-10T16:54:46.447481+0000 mgr.gibba032.whopeo (mgr.24625) 9591 : cluster [DBG] pgmap v5466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:48.577 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:48 gibba011 bash[31774]: cluster 2021-03-10T16:54:46.447481+0000 mgr.gibba032.whopeo (mgr.24625) 9591 : cluster [DBG] pgmap v5466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:48 gibba032 bash[24004]: cluster 2021-03-10T16:54:46.447481+0000 mgr.gibba032.whopeo (mgr.24625) 9591 : cluster [DBG] pgmap v5466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:50.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:50 gibba011 bash[30791]: cluster 2021-03-10T16:54:48.449035+0000 mgr.gibba032.whopeo (mgr.24625) 9592 : cluster [DBG] pgmap v5467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:50.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:50 gibba011 bash[31774]: cluster 2021-03-10T16:54:48.449035+0000 mgr.gibba032.whopeo (mgr.24625) 9592 : cluster [DBG] pgmap v5467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:50.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:50 gibba032 bash[24004]: cluster 2021-03-10T16:54:48.449035+0000 mgr.gibba032.whopeo (mgr.24625) 9592 : cluster [DBG] pgmap v5467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:52.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:52 gibba011 bash[30791]: cluster 2021-03-10T16:54:50.450624+0000 mgr.gibba032.whopeo (mgr.24625) 9593 : cluster [DBG] pgmap v5468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:52 gibba011 bash[31774]: cluster 2021-03-10T16:54:50.450624+0000 mgr.gibba032.whopeo (mgr.24625) 9593 : cluster [DBG] pgmap v5468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:52.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:52 gibba032 bash[24004]: cluster 2021-03-10T16:54:50.450624+0000 mgr.gibba032.whopeo (mgr.24625) 9593 : cluster [DBG] pgmap v5468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:54.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:54 gibba011 bash[30791]: cluster 2021-03-10T16:54:52.452429+0000 mgr.gibba032.whopeo (mgr.24625) 9594 : cluster [DBG] pgmap v5469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:54.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:54 gibba011 bash[31774]: cluster 2021-03-10T16:54:52.452429+0000 mgr.gibba032.whopeo (mgr.24625) 9594 : cluster [DBG] pgmap v5469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:54 gibba032 bash[24004]: cluster 2021-03-10T16:54:52.452429+0000 mgr.gibba032.whopeo (mgr.24625) 9594 : cluster [DBG] pgmap v5469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:56 gibba032 bash[24004]: cluster 2021-03-10T16:54:54.453836+0000 mgr.gibba032.whopeo (mgr.24625) 9595 : cluster [DBG] pgmap v5470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:56 gibba011 bash[30791]: cluster 2021-03-10T16:54:54.453836+0000 mgr.gibba032.whopeo (mgr.24625) 9595 : cluster [DBG] pgmap v5470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:56 gibba011 bash[31774]: cluster 2021-03-10T16:54:54.453836+0000 mgr.gibba032.whopeo (mgr.24625) 9595 : cluster [DBG] pgmap v5470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:57 gibba032 bash[24004]: cluster 2021-03-10T16:54:56.455508+0000 mgr.gibba032.whopeo (mgr.24625) 9596 : cluster [DBG] pgmap v5471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:57 gibba032 bash[24004]: audit 2021-03-10T16:54:57.232658+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:54:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:57 gibba032 bash[24004]: audit 2021-03-10T16:54:57.233388+0000 mon.a (mon.0) 3257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:54:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[30791]: cluster 2021-03-10T16:54:56.455508+0000 mgr.gibba032.whopeo (mgr.24625) 9596 : cluster [DBG] pgmap v5471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[30791]: audit 2021-03-10T16:54:57.232658+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:54:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[30791]: audit 2021-03-10T16:54:57.233388+0000 mon.a (mon.0) 3257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:54:57.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[31774]: cluster 2021-03-10T16:54:56.455508+0000 mgr.gibba032.whopeo (mgr.24625) 9596 : cluster [DBG] pgmap v5471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[31774]: audit 2021-03-10T16:54:57.232658+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:54:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:57 gibba011 bash[31774]: audit 2021-03-10T16:54:57.233388+0000 mon.a (mon.0) 3257 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:54:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:58 gibba032 bash[24004]: audit 2021-03-10T16:54:57.375349+0000 mon.a (mon.0) 3258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:54:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:58 gibba032 bash[24004]: audit 2021-03-10T16:54:57.375616+0000 mon.a (mon.0) 3259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:58 gibba032 bash[24004]: audit 2021-03-10T16:54:57.375860+0000 mon.a (mon.0) 3260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:58 gibba032 bash[24004]: audit 2021-03-10T16:54:57.376106+0000 mon.a (mon.0) 3261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:54:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[30791]: audit 2021-03-10T16:54:57.375349+0000 mon.a (mon.0) 3258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:54:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[30791]: audit 2021-03-10T16:54:57.375616+0000 mon.a (mon.0) 3259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[30791]: audit 2021-03-10T16:54:57.375860+0000 mon.a (mon.0) 3260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[30791]: audit 2021-03-10T16:54:57.376106+0000 mon.a (mon.0) 3261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:54:58.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[31774]: audit 2021-03-10T16:54:57.375349+0000 mon.a (mon.0) 3258 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T16:54:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[31774]: audit 2021-03-10T16:54:57.375616+0000 mon.a (mon.0) 3259 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[31774]: audit 2021-03-10T16:54:57.375860+0000 mon.a (mon.0) 3260 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T16:54:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:58 gibba011 bash[31774]: audit 2021-03-10T16:54:57.376106+0000 mon.a (mon.0) 3261 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:54:59.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:54:59 gibba032 bash[24004]: cluster 2021-03-10T16:54:58.457085+0000 mgr.gibba032.whopeo (mgr.24625) 9597 : cluster [DBG] pgmap v5472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:59.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:54:59 gibba011 bash[30791]: cluster 2021-03-10T16:54:58.457085+0000 mgr.gibba032.whopeo (mgr.24625) 9597 : cluster [DBG] pgmap v5472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:54:59.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:54:59 gibba011 bash[31774]: cluster 2021-03-10T16:54:58.457085+0000 mgr.gibba032.whopeo (mgr.24625) 9597 : cluster [DBG] pgmap v5472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:02.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cluster 2021-03-10T16:55:00.458567+0000 mgr.gibba032.whopeo (mgr.24625) 9598 : cluster [DBG] pgmap v5473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.590486+0000 mgr.gibba032.whopeo (mgr.24625) 9599 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:55:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.592100+0000 mgr.gibba032.whopeo (mgr.24625) 9600 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:55:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.817935+0000 mon.a (mon.0) 3262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:55:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.818910+0000 mgr.gibba032.whopeo (mgr.24625) 9601 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:55:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.820523+0000 mgr.gibba032.whopeo (mgr.24625) 9602 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:55:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.821266+0000 mon.a (mon.0) 3263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:55:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.821626+0000 mgr.gibba032.whopeo (mgr.24625) 9603 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:55:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.821671+0000 mgr.gibba032.whopeo (mgr.24625) 9604 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:55:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.822532+0000 mon.a (mon.0) 3264 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.822800+0000 mgr.gibba032.whopeo (mgr.24625) 9605 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:55:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.822842+0000 mgr.gibba032.whopeo (mgr.24625) 9606 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:55:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.823672+0000 mon.a (mon.0) 3265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.823923+0000 mgr.gibba032.whopeo (mgr.24625) 9607 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:55:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.824796+0000 mon.a (mon.0) 3266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.824930+0000 mgr.gibba032.whopeo (mgr.24625) 9608 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:55:02.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.824976+0000 mgr.gibba032.whopeo (mgr.24625) 9609 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:55:02.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.826362+0000 mgr.gibba032.whopeo (mgr.24625) 9610 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:55:02.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.826708+0000 mgr.gibba032.whopeo (mgr.24625) 9611 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:55:02.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.827088+0000 mon.a (mon.0) 3267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.827976+0000 mgr.gibba032.whopeo (mgr.24625) 9612 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:55:02.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cluster 2021-03-10T16:55:00.458567+0000 mgr.gibba032.whopeo (mgr.24625) 9598 : cluster [DBG] pgmap v5473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:02.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.590486+0000 mgr.gibba032.whopeo (mgr.24625) 9599 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:55:02.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.592100+0000 mgr.gibba032.whopeo (mgr.24625) 9600 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:55:02.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.817935+0000 mon.a (mon.0) 3262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:55:02.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.818910+0000 mgr.gibba032.whopeo (mgr.24625) 9601 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:55:02.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.820523+0000 mgr.gibba032.whopeo (mgr.24625) 9602 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:55:02.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.821266+0000 mon.a (mon.0) 3263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:55:02.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.821626+0000 mgr.gibba032.whopeo (mgr.24625) 9603 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:55:02.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.821671+0000 mgr.gibba032.whopeo (mgr.24625) 9604 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:55:02.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.822532+0000 mon.a (mon.0) 3264 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.822800+0000 mgr.gibba032.whopeo (mgr.24625) 9605 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:55:02.095 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.822842+0000 mgr.gibba032.whopeo (mgr.24625) 9606 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:55:02.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.823672+0000 mon.a (mon.0) 3265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.823923+0000 mgr.gibba032.whopeo (mgr.24625) 9607 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:55:02.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.824796+0000 mon.a (mon.0) 3266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.824930+0000 mgr.gibba032.whopeo (mgr.24625) 9608 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:55:02.096 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.824976+0000 mgr.gibba032.whopeo (mgr.24625) 9609 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:55:02.106 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.826362+0000 mgr.gibba032.whopeo (mgr.24625) 9610 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:55:02.107 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.826708+0000 mgr.gibba032.whopeo (mgr.24625) 9611 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:55:02.107 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.827088+0000 mon.a (mon.0) 3267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.107 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.827976+0000 mgr.gibba032.whopeo (mgr.24625) 9612 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:55:02.108 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.828049+0000 mgr.gibba032.whopeo (mgr.24625) 9613 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:55:02.108 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: cephadm 2021-03-10T16:55:00.828173+0000 mgr.gibba032.whopeo (mgr.24625) 9614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:02.120 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[30791]: audit 2021-03-10T16:55:00.828769+0000 mon.a (mon.0) 3268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.121 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.828049+0000 mgr.gibba032.whopeo (mgr.24625) 9613 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:55:02.121 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: cephadm 2021-03-10T16:55:00.828173+0000 mgr.gibba032.whopeo (mgr.24625) 9614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:02.121 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:01 gibba011 bash[31774]: audit 2021-03-10T16:55:00.828769+0000 mon.a (mon.0) 3268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cluster 2021-03-10T16:55:00.458567+0000 mgr.gibba032.whopeo (mgr.24625) 9598 : cluster [DBG] pgmap v5473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.590486+0000 mgr.gibba032.whopeo (mgr.24625) 9599 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:55:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.592100+0000 mgr.gibba032.whopeo (mgr.24625) 9600 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:55:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.817935+0000 mon.a (mon.0) 3262 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:55:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.818910+0000 mgr.gibba032.whopeo (mgr.24625) 9601 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:55:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.820523+0000 mgr.gibba032.whopeo (mgr.24625) 9602 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:55:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.821266+0000 mon.a (mon.0) 3263 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:55:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.821626+0000 mgr.gibba032.whopeo (mgr.24625) 9603 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:55:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.821671+0000 mgr.gibba032.whopeo (mgr.24625) 9604 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:55:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.822532+0000 mon.a (mon.0) 3264 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.822800+0000 mgr.gibba032.whopeo (mgr.24625) 9605 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:55:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.822842+0000 mgr.gibba032.whopeo (mgr.24625) 9606 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:55:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.823672+0000 mon.a (mon.0) 3265 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.823923+0000 mgr.gibba032.whopeo (mgr.24625) 9607 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:55:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.824796+0000 mon.a (mon.0) 3266 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.824930+0000 mgr.gibba032.whopeo (mgr.24625) 9608 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:55:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.824976+0000 mgr.gibba032.whopeo (mgr.24625) 9609 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:55:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.826362+0000 mgr.gibba032.whopeo (mgr.24625) 9610 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:55:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.826708+0000 mgr.gibba032.whopeo (mgr.24625) 9611 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:55:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.827088+0000 mon.a (mon.0) 3267 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.827976+0000 mgr.gibba032.whopeo (mgr.24625) 9612 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:55:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.828049+0000 mgr.gibba032.whopeo (mgr.24625) 9613 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:55:02.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: cephadm 2021-03-10T16:55:00.828173+0000 mgr.gibba032.whopeo (mgr.24625) 9614 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:02.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:01 gibba032 bash[24004]: audit 2021-03-10T16:55:00.828769+0000 mon.a (mon.0) 3268 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:03 gibba032 bash[24004]: cluster 2021-03-10T16:55:02.460457+0000 mgr.gibba032.whopeo (mgr.24625) 9615 : cluster [DBG] pgmap v5474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:04.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:03 gibba011 bash[30791]: cluster 2021-03-10T16:55:02.460457+0000 mgr.gibba032.whopeo (mgr.24625) 9615 : cluster [DBG] pgmap v5474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:03 gibba011 bash[31774]: cluster 2021-03-10T16:55:02.460457+0000 mgr.gibba032.whopeo (mgr.24625) 9615 : cluster [DBG] pgmap v5474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:06.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:06 gibba011 bash[30791]: cluster 2021-03-10T16:55:04.461934+0000 mgr.gibba032.whopeo (mgr.24625) 9616 : cluster [DBG] pgmap v5475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:06.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:06 gibba011 bash[31774]: cluster 2021-03-10T16:55:04.461934+0000 mgr.gibba032.whopeo (mgr.24625) 9616 : cluster [DBG] pgmap v5475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:06 gibba032 bash[24004]: cluster 2021-03-10T16:55:04.461934+0000 mgr.gibba032.whopeo (mgr.24625) 9616 : cluster [DBG] pgmap v5475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:08.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:08 gibba011 bash[30791]: cluster 2021-03-10T16:55:06.463727+0000 mgr.gibba032.whopeo (mgr.24625) 9617 : cluster [DBG] pgmap v5476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:08 gibba011 bash[31774]: cluster 2021-03-10T16:55:06.463727+0000 mgr.gibba032.whopeo (mgr.24625) 9617 : cluster [DBG] pgmap v5476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:08.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:08 gibba032 bash[24004]: cluster 2021-03-10T16:55:06.463727+0000 mgr.gibba032.whopeo (mgr.24625) 9617 : cluster [DBG] pgmap v5476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:10 gibba011 bash[30791]: cluster 2021-03-10T16:55:08.465422+0000 mgr.gibba032.whopeo (mgr.24625) 9618 : cluster [DBG] pgmap v5477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:10 gibba011 bash[31774]: cluster 2021-03-10T16:55:08.465422+0000 mgr.gibba032.whopeo (mgr.24625) 9618 : cluster [DBG] pgmap v5477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:10.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:10 gibba032 bash[24004]: cluster 2021-03-10T16:55:08.465422+0000 mgr.gibba032.whopeo (mgr.24625) 9618 : cluster [DBG] pgmap v5477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:11.945 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:55:12.173 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:55:12.173 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:55:12.173 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:55:12.174 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:55:12.175 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:55:12.176 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:55:12.176 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:55:12.176 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:55:12.177 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:55:12.177 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:55:12.178 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:55:12.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:55:12.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:55:12.178 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:55:12.179 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:55:12.179 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:55:12.179 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:55:12.179 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:55:12.180 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:55:12.180 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:55:12.180 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:55:12.428 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:55:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:55:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:55:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:55:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:55:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:55:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:55:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:55:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:55:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:55:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:55:12.432 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:55:12.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:12 gibba011 bash[30791]: cluster 2021-03-10T16:55:10.466965+0000 mgr.gibba032.whopeo (mgr.24625) 9619 : cluster [DBG] pgmap v5478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:12.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:12 gibba011 bash[31774]: cluster 2021-03-10T16:55:10.466965+0000 mgr.gibba032.whopeo (mgr.24625) 9619 : cluster [DBG] pgmap v5478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:12 gibba032 bash[24004]: cluster 2021-03-10T16:55:10.466965+0000 mgr.gibba032.whopeo (mgr.24625) 9619 : cluster [DBG] pgmap v5478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:13.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:13 gibba032 bash[24004]: audit 2021-03-10T16:55:11.936701+0000 mgr.gibba032.whopeo (mgr.24625) 9620 : audit [DBG] from='client.60481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:13 gibba032 bash[24004]: audit 2021-03-10T16:55:12.160363+0000 mgr.gibba032.whopeo (mgr.24625) 9621 : audit [DBG] from='client.60487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:13 gibba032 bash[24004]: audit 2021-03-10T16:55:12.430472+0000 mon.c (mon.1) 187 : audit [DBG] from='client.? 172.21.2.111:0/2470504661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:13.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[30791]: audit 2021-03-10T16:55:11.936701+0000 mgr.gibba032.whopeo (mgr.24625) 9620 : audit [DBG] from='client.60481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[30791]: audit 2021-03-10T16:55:12.160363+0000 mgr.gibba032.whopeo (mgr.24625) 9621 : audit [DBG] from='client.60487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[30791]: audit 2021-03-10T16:55:12.430472+0000 mon.c (mon.1) 187 : audit [DBG] from='client.? 172.21.2.111:0/2470504661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:13.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[31774]: audit 2021-03-10T16:55:11.936701+0000 mgr.gibba032.whopeo (mgr.24625) 9620 : audit [DBG] from='client.60481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[31774]: audit 2021-03-10T16:55:12.160363+0000 mgr.gibba032.whopeo (mgr.24625) 9621 : audit [DBG] from='client.60487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:13.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:13 gibba011 bash[31774]: audit 2021-03-10T16:55:12.430472+0000 mon.c (mon.1) 187 : audit [DBG] from='client.? 172.21.2.111:0/2470504661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:14 gibba032 bash[24004]: cluster 2021-03-10T16:55:12.468691+0000 mgr.gibba032.whopeo (mgr.24625) 9622 : cluster [DBG] pgmap v5479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:14 gibba011 bash[30791]: cluster 2021-03-10T16:55:12.468691+0000 mgr.gibba032.whopeo (mgr.24625) 9622 : cluster [DBG] pgmap v5479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:14 gibba011 bash[31774]: cluster 2021-03-10T16:55:12.468691+0000 mgr.gibba032.whopeo (mgr.24625) 9622 : cluster [DBG] pgmap v5479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:16.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:16 gibba011 bash[30791]: cluster 2021-03-10T16:55:14.470058+0000 mgr.gibba032.whopeo (mgr.24625) 9623 : cluster [DBG] pgmap v5480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:16.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:16 gibba011 bash[31774]: cluster 2021-03-10T16:55:14.470058+0000 mgr.gibba032.whopeo (mgr.24625) 9623 : cluster [DBG] pgmap v5480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:16 gibba032 bash[24004]: cluster 2021-03-10T16:55:14.470058+0000 mgr.gibba032.whopeo (mgr.24625) 9623 : cluster [DBG] pgmap v5480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:17.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:17 gibba011 bash[30791]: cephadm 2021-03-10T16:55:15.844059+0000 mgr.gibba032.whopeo (mgr.24625) 9624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:17.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:17 gibba011 bash[31774]: cephadm 2021-03-10T16:55:15.844059+0000 mgr.gibba032.whopeo (mgr.24625) 9624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:17 gibba032 bash[24004]: cephadm 2021-03-10T16:55:15.844059+0000 mgr.gibba032.whopeo (mgr.24625) 9624 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:18.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:18 gibba011 bash[30791]: cluster 2021-03-10T16:55:16.471860+0000 mgr.gibba032.whopeo (mgr.24625) 9625 : cluster [DBG] pgmap v5481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:18.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:18 gibba011 bash[31774]: cluster 2021-03-10T16:55:16.471860+0000 mgr.gibba032.whopeo (mgr.24625) 9625 : cluster [DBG] pgmap v5481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:18.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:18 gibba032 bash[24004]: cluster 2021-03-10T16:55:16.471860+0000 mgr.gibba032.whopeo (mgr.24625) 9625 : cluster [DBG] pgmap v5481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:20.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:20 gibba011 bash[30791]: cluster 2021-03-10T16:55:18.473585+0000 mgr.gibba032.whopeo (mgr.24625) 9626 : cluster [DBG] pgmap v5482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:20.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:20 gibba011 bash[31774]: cluster 2021-03-10T16:55:18.473585+0000 mgr.gibba032.whopeo (mgr.24625) 9626 : cluster [DBG] pgmap v5482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:20.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:20 gibba032 bash[24004]: cluster 2021-03-10T16:55:18.473585+0000 mgr.gibba032.whopeo (mgr.24625) 9626 : cluster [DBG] pgmap v5482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:22 gibba032 bash[24004]: cluster 2021-03-10T16:55:20.475147+0000 mgr.gibba032.whopeo (mgr.24625) 9627 : cluster [DBG] pgmap v5483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:22 gibba011 bash[30791]: cluster 2021-03-10T16:55:20.475147+0000 mgr.gibba032.whopeo (mgr.24625) 9627 : cluster [DBG] pgmap v5483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:22 gibba011 bash[31774]: cluster 2021-03-10T16:55:20.475147+0000 mgr.gibba032.whopeo (mgr.24625) 9627 : cluster [DBG] pgmap v5483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:24 gibba032 bash[24004]: cluster 2021-03-10T16:55:22.476916+0000 mgr.gibba032.whopeo (mgr.24625) 9628 : cluster [DBG] pgmap v5484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:24 gibba011 bash[30791]: cluster 2021-03-10T16:55:22.476916+0000 mgr.gibba032.whopeo (mgr.24625) 9628 : cluster [DBG] pgmap v5484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:24 gibba011 bash[31774]: cluster 2021-03-10T16:55:22.476916+0000 mgr.gibba032.whopeo (mgr.24625) 9628 : cluster [DBG] pgmap v5484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:26.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:26 gibba011 bash[30791]: cluster 2021-03-10T16:55:24.478225+0000 mgr.gibba032.whopeo (mgr.24625) 9629 : cluster [DBG] pgmap v5485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:26.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:26 gibba011 bash[31774]: cluster 2021-03-10T16:55:24.478225+0000 mgr.gibba032.whopeo (mgr.24625) 9629 : cluster [DBG] pgmap v5485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:26.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:26 gibba032 bash[24004]: cluster 2021-03-10T16:55:24.478225+0000 mgr.gibba032.whopeo (mgr.24625) 9629 : cluster [DBG] pgmap v5485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:28.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:28 gibba011 bash[30791]: cluster 2021-03-10T16:55:26.480107+0000 mgr.gibba032.whopeo (mgr.24625) 9630 : cluster [DBG] pgmap v5486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:28.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:28 gibba011 bash[31774]: cluster 2021-03-10T16:55:26.480107+0000 mgr.gibba032.whopeo (mgr.24625) 9630 : cluster [DBG] pgmap v5486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:28 gibba032 bash[24004]: cluster 2021-03-10T16:55:26.480107+0000 mgr.gibba032.whopeo (mgr.24625) 9630 : cluster [DBG] pgmap v5486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:30.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:30 gibba011 bash[30791]: cluster 2021-03-10T16:55:28.481543+0000 mgr.gibba032.whopeo (mgr.24625) 9631 : cluster [DBG] pgmap v5487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:30.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:30 gibba011 bash[31774]: cluster 2021-03-10T16:55:28.481543+0000 mgr.gibba032.whopeo (mgr.24625) 9631 : cluster [DBG] pgmap v5487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:30.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:30 gibba032 bash[24004]: cluster 2021-03-10T16:55:28.481543+0000 mgr.gibba032.whopeo (mgr.24625) 9631 : cluster [DBG] pgmap v5487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:32.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:32 gibba011 bash[30791]: cluster 2021-03-10T16:55:30.482898+0000 mgr.gibba032.whopeo (mgr.24625) 9632 : cluster [DBG] pgmap v5488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:32.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:32 gibba011 bash[30791]: cephadm 2021-03-10T16:55:30.857092+0000 mgr.gibba032.whopeo (mgr.24625) 9633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:32 gibba011 bash[31774]: cluster 2021-03-10T16:55:30.482898+0000 mgr.gibba032.whopeo (mgr.24625) 9632 : cluster [DBG] pgmap v5488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:32.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:32 gibba011 bash[31774]: cephadm 2021-03-10T16:55:30.857092+0000 mgr.gibba032.whopeo (mgr.24625) 9633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:32 gibba032 bash[24004]: cluster 2021-03-10T16:55:30.482898+0000 mgr.gibba032.whopeo (mgr.24625) 9632 : cluster [DBG] pgmap v5488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:32 gibba032 bash[24004]: cephadm 2021-03-10T16:55:30.857092+0000 mgr.gibba032.whopeo (mgr.24625) 9633 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:34 gibba032 bash[24004]: cluster 2021-03-10T16:55:32.484717+0000 mgr.gibba032.whopeo (mgr.24625) 9634 : cluster [DBG] pgmap v5489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:34 gibba011 bash[30791]: cluster 2021-03-10T16:55:32.484717+0000 mgr.gibba032.whopeo (mgr.24625) 9634 : cluster [DBG] pgmap v5489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:34 gibba011 bash[31774]: cluster 2021-03-10T16:55:32.484717+0000 mgr.gibba032.whopeo (mgr.24625) 9634 : cluster [DBG] pgmap v5489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:36 gibba032 bash[24004]: cluster 2021-03-10T16:55:34.485176+0000 mgr.gibba032.whopeo (mgr.24625) 9635 : cluster [DBG] pgmap v5490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:36 gibba011 bash[30791]: cluster 2021-03-10T16:55:34.485176+0000 mgr.gibba032.whopeo (mgr.24625) 9635 : cluster [DBG] pgmap v5490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:36 gibba011 bash[31774]: cluster 2021-03-10T16:55:34.485176+0000 mgr.gibba032.whopeo (mgr.24625) 9635 : cluster [DBG] pgmap v5490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:37.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:37 gibba032 bash[24004]: cluster 2021-03-10T16:55:36.486794+0000 mgr.gibba032.whopeo (mgr.24625) 9636 : cluster [DBG] pgmap v5491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:37.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:37 gibba011 bash[30791]: cluster 2021-03-10T16:55:36.486794+0000 mgr.gibba032.whopeo (mgr.24625) 9636 : cluster [DBG] pgmap v5491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:37.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:37 gibba011 bash[31774]: cluster 2021-03-10T16:55:36.486794+0000 mgr.gibba032.whopeo (mgr.24625) 9636 : cluster [DBG] pgmap v5491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:39.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:39 gibba011 bash[30791]: cluster 2021-03-10T16:55:38.488480+0000 mgr.gibba032.whopeo (mgr.24625) 9637 : cluster [DBG] pgmap v5492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:39.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:39 gibba011 bash[31774]: cluster 2021-03-10T16:55:38.488480+0000 mgr.gibba032.whopeo (mgr.24625) 9637 : cluster [DBG] pgmap v5492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:40.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:39 gibba032 bash[24004]: cluster 2021-03-10T16:55:38.488480+0000 mgr.gibba032.whopeo (mgr.24625) 9637 : cluster [DBG] pgmap v5492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:42.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:42 gibba011 bash[30791]: cluster 2021-03-10T16:55:40.489135+0000 mgr.gibba032.whopeo (mgr.24625) 9638 : cluster [DBG] pgmap v5493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:42.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:42 gibba011 bash[31774]: cluster 2021-03-10T16:55:40.489135+0000 mgr.gibba032.whopeo (mgr.24625) 9638 : cluster [DBG] pgmap v5493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:42.672 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:55:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:42 gibba032 bash[24004]: cluster 2021-03-10T16:55:40.489135+0000 mgr.gibba032.whopeo (mgr.24625) 9638 : cluster [DBG] pgmap v5493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:42.900 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:55:42.901 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:55:42.901 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:55:42.901 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:55:42.901 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:55:42.901 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:55:42.902 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:55:42.902 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:55:42.902 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:55:42.902 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:55:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:55:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:55:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:55:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:55:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:55:42.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:55:42.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:55:42.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:55:42.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:55:42.905 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:55:42.905 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:55:43.150 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:55:43.150 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:55:43.151 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:55:43.151 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:43.151 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:55:43.151 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:55:43.152 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:43.152 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:55:43.152 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:55:43.152 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:55:43.152 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:55:43.153 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:55:43.153 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:55:43.153 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:55:43.153 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:55:43.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:43 gibba011 bash[30791]: audit 2021-03-10T16:55:43.151781+0000 mon.c (mon.1) 188 : audit [DBG] from='client.? 172.21.2.111:0/1330461326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:43.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:43 gibba011 bash[31774]: audit 2021-03-10T16:55:43.151781+0000 mon.c (mon.1) 188 : audit [DBG] from='client.? 172.21.2.111:0/1330461326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:43.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:43 gibba032 bash[24004]: audit 2021-03-10T16:55:43.151781+0000 mon.c (mon.1) 188 : audit [DBG] from='client.? 172.21.2.111:0/1330461326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:55:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:44 gibba032 bash[24004]: cluster 2021-03-10T16:55:42.490775+0000 mgr.gibba032.whopeo (mgr.24625) 9639 : cluster [DBG] pgmap v5494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:44 gibba032 bash[24004]: audit 2021-03-10T16:55:42.663281+0000 mgr.gibba032.whopeo (mgr.24625) 9640 : audit [DBG] from='client.60499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:44 gibba032 bash[24004]: audit 2021-03-10T16:55:42.888338+0000 mgr.gibba032.whopeo (mgr.24625) 9641 : audit [DBG] from='client.60505 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:44.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[30791]: cluster 2021-03-10T16:55:42.490775+0000 mgr.gibba032.whopeo (mgr.24625) 9639 : cluster [DBG] pgmap v5494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:44.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[30791]: audit 2021-03-10T16:55:42.663281+0000 mgr.gibba032.whopeo (mgr.24625) 9640 : audit [DBG] from='client.60499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:44.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[30791]: audit 2021-03-10T16:55:42.888338+0000 mgr.gibba032.whopeo (mgr.24625) 9641 : audit [DBG] from='client.60505 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[31774]: cluster 2021-03-10T16:55:42.490775+0000 mgr.gibba032.whopeo (mgr.24625) 9639 : cluster [DBG] pgmap v5494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[31774]: audit 2021-03-10T16:55:42.663281+0000 mgr.gibba032.whopeo (mgr.24625) 9640 : audit [DBG] from='client.60499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:44 gibba011 bash[31774]: audit 2021-03-10T16:55:42.888338+0000 mgr.gibba032.whopeo (mgr.24625) 9641 : audit [DBG] from='client.60505 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:55:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:46 gibba032 bash[24004]: cluster 2021-03-10T16:55:44.492350+0000 mgr.gibba032.whopeo (mgr.24625) 9642 : cluster [DBG] pgmap v5495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:46.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:46 gibba011 bash[30791]: cluster 2021-03-10T16:55:44.492350+0000 mgr.gibba032.whopeo (mgr.24625) 9642 : cluster [DBG] pgmap v5495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:46 gibba011 bash[31774]: cluster 2021-03-10T16:55:44.492350+0000 mgr.gibba032.whopeo (mgr.24625) 9642 : cluster [DBG] pgmap v5495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:47 gibba032 bash[24004]: cephadm 2021-03-10T16:55:45.872766+0000 mgr.gibba032.whopeo (mgr.24625) 9643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:47.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:47 gibba011 bash[30791]: cephadm 2021-03-10T16:55:45.872766+0000 mgr.gibba032.whopeo (mgr.24625) 9643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:47.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:47 gibba011 bash[31774]: cephadm 2021-03-10T16:55:45.872766+0000 mgr.gibba032.whopeo (mgr.24625) 9643 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:55:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:48 gibba032 bash[24004]: cluster 2021-03-10T16:55:46.493122+0000 mgr.gibba032.whopeo (mgr.24625) 9644 : cluster [DBG] pgmap v5496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:48.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:48 gibba011 bash[30791]: cluster 2021-03-10T16:55:46.493122+0000 mgr.gibba032.whopeo (mgr.24625) 9644 : cluster [DBG] pgmap v5496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:48.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:48 gibba011 bash[31774]: cluster 2021-03-10T16:55:46.493122+0000 mgr.gibba032.whopeo (mgr.24625) 9644 : cluster [DBG] pgmap v5496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:49.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:49 gibba032 bash[24004]: cluster 2021-03-10T16:55:48.494642+0000 mgr.gibba032.whopeo (mgr.24625) 9645 : cluster [DBG] pgmap v5497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:49.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:49 gibba011 bash[30791]: cluster 2021-03-10T16:55:48.494642+0000 mgr.gibba032.whopeo (mgr.24625) 9645 : cluster [DBG] pgmap v5497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:49.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:49 gibba011 bash[31774]: cluster 2021-03-10T16:55:48.494642+0000 mgr.gibba032.whopeo (mgr.24625) 9645 : cluster [DBG] pgmap v5497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:52.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:52 gibba011 bash[30791]: cluster 2021-03-10T16:55:50.496286+0000 mgr.gibba032.whopeo (mgr.24625) 9646 : cluster [DBG] pgmap v5498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:52.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:52 gibba011 bash[31774]: cluster 2021-03-10T16:55:50.496286+0000 mgr.gibba032.whopeo (mgr.24625) 9646 : cluster [DBG] pgmap v5498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:52 gibba032 bash[24004]: cluster 2021-03-10T16:55:50.496286+0000 mgr.gibba032.whopeo (mgr.24625) 9646 : cluster [DBG] pgmap v5498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:54 gibba032 bash[24004]: cluster 2021-03-10T16:55:52.497821+0000 mgr.gibba032.whopeo (mgr.24625) 9647 : cluster [DBG] pgmap v5499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:54 gibba011 bash[30791]: cluster 2021-03-10T16:55:52.497821+0000 mgr.gibba032.whopeo (mgr.24625) 9647 : cluster [DBG] pgmap v5499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:54 gibba011 bash[31774]: cluster 2021-03-10T16:55:52.497821+0000 mgr.gibba032.whopeo (mgr.24625) 9647 : cluster [DBG] pgmap v5499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:56 gibba032 bash[24004]: cluster 2021-03-10T16:55:54.499368+0000 mgr.gibba032.whopeo (mgr.24625) 9648 : cluster [DBG] pgmap v5500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:56.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:56 gibba011 bash[30791]: cluster 2021-03-10T16:55:54.499368+0000 mgr.gibba032.whopeo (mgr.24625) 9648 : cluster [DBG] pgmap v5500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:56 gibba011 bash[31774]: cluster 2021-03-10T16:55:54.499368+0000 mgr.gibba032.whopeo (mgr.24625) 9648 : cluster [DBG] pgmap v5500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:57 gibba032 bash[24004]: audit 2021-03-10T16:55:57.238962+0000 mon.a (mon.0) 3269 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:55:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:57 gibba032 bash[24004]: audit 2021-03-10T16:55:57.239621+0000 mon.a (mon.0) 3270 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:55:57.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:57 gibba011 bash[30791]: audit 2021-03-10T16:55:57.238962+0000 mon.a (mon.0) 3269 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:55:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:57 gibba011 bash[30791]: audit 2021-03-10T16:55:57.239621+0000 mon.a (mon.0) 3270 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:55:57.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:57 gibba011 bash[31774]: audit 2021-03-10T16:55:57.238962+0000 mon.a (mon.0) 3269 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:55:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:57 gibba011 bash[31774]: audit 2021-03-10T16:55:57.239621+0000 mon.a (mon.0) 3270 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:55:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:58 gibba032 bash[24004]: cluster 2021-03-10T16:55:56.501192+0000 mgr.gibba032.whopeo (mgr.24625) 9649 : cluster [DBG] pgmap v5501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:58 gibba032 bash[24004]: audit 2021-03-10T16:55:57.379780+0000 mon.a (mon.0) 3271 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:55:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:58 gibba032 bash[24004]: audit 2021-03-10T16:55:57.379981+0000 mon.a (mon.0) 3272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:55:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:58 gibba032 bash[24004]: audit 2021-03-10T16:55:57.380158+0000 mon.a (mon.0) 3273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:55:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:58 gibba032 bash[24004]: audit 2021-03-10T16:55:57.380338+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:55:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[30791]: cluster 2021-03-10T16:55:56.501192+0000 mgr.gibba032.whopeo (mgr.24625) 9649 : cluster [DBG] pgmap v5501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[30791]: audit 2021-03-10T16:55:57.379780+0000 mon.a (mon.0) 3271 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:55:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[30791]: audit 2021-03-10T16:55:57.379981+0000 mon.a (mon.0) 3272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:55:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[30791]: audit 2021-03-10T16:55:57.380158+0000 mon.a (mon.0) 3273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:55:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[30791]: audit 2021-03-10T16:55:57.380338+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:55:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[31774]: cluster 2021-03-10T16:55:56.501192+0000 mgr.gibba032.whopeo (mgr.24625) 9649 : cluster [DBG] pgmap v5501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[31774]: audit 2021-03-10T16:55:57.379780+0000 mon.a (mon.0) 3271 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T16:55:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[31774]: audit 2021-03-10T16:55:57.379981+0000 mon.a (mon.0) 3272 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T16:55:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[31774]: audit 2021-03-10T16:55:57.380158+0000 mon.a (mon.0) 3273 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:55:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:58 gibba011 bash[31774]: audit 2021-03-10T16:55:57.380338+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T16:55:59.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:55:59 gibba032 bash[24004]: cluster 2021-03-10T16:55:58.501932+0000 mgr.gibba032.whopeo (mgr.24625) 9650 : cluster [DBG] pgmap v5502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:59.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:55:59 gibba011 bash[30791]: cluster 2021-03-10T16:55:58.501932+0000 mgr.gibba032.whopeo (mgr.24625) 9650 : cluster [DBG] pgmap v5502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:55:59.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:55:59 gibba011 bash[31774]: cluster 2021-03-10T16:55:58.501932+0000 mgr.gibba032.whopeo (mgr.24625) 9650 : cluster [DBG] pgmap v5502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cluster 2021-03-10T16:56:00.503413+0000 mgr.gibba032.whopeo (mgr.24625) 9651 : cluster [DBG] pgmap v5503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:00.885644+0000 mgr.gibba032.whopeo (mgr.24625) 9652 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:56:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:00.887183+0000 mgr.gibba032.whopeo (mgr.24625) 9653 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:56:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.094554+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:56:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.095300+0000 mgr.gibba032.whopeo (mgr.24625) 9654 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:56:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.097413+0000 mgr.gibba032.whopeo (mgr.24625) 9655 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:56:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.097656+0000 mon.a (mon.0) 3276 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:56:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.098967+0000 mgr.gibba032.whopeo (mgr.24625) 9656 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:56:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.099007+0000 mgr.gibba032.whopeo (mgr.24625) 9657 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:56:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.099611+0000 mon.a (mon.0) 3277 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.100642+0000 mgr.gibba032.whopeo (mgr.24625) 9658 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:56:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.100678+0000 mgr.gibba032.whopeo (mgr.24625) 9659 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:56:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.101202+0000 mon.a (mon.0) 3278 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.102086+0000 mgr.gibba032.whopeo (mgr.24625) 9660 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:56:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.102755+0000 mon.a (mon.0) 3279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.103295+0000 mgr.gibba032.whopeo (mgr.24625) 9661 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:56:02.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.103332+0000 mgr.gibba032.whopeo (mgr.24625) 9662 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:56:02.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.104756+0000 mgr.gibba032.whopeo (mgr.24625) 9663 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:56:02.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.105163+0000 mgr.gibba032.whopeo (mgr.24625) 9664 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:56:02.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.105476+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.106458+0000 mgr.gibba032.whopeo (mgr.24625) 9665 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:56:02.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.106497+0000 mgr.gibba032.whopeo (mgr.24625) 9666 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:56:02.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: cephadm 2021-03-10T16:56:01.106607+0000 mgr.gibba032.whopeo (mgr.24625) 9667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:02.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:02 gibba032 bash[24004]: audit 2021-03-10T16:56:01.107195+0000 mon.a (mon.0) 3281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cluster 2021-03-10T16:56:00.503413+0000 mgr.gibba032.whopeo (mgr.24625) 9651 : cluster [DBG] pgmap v5503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:00.885644+0000 mgr.gibba032.whopeo (mgr.24625) 9652 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:56:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:00.887183+0000 mgr.gibba032.whopeo (mgr.24625) 9653 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:56:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.094554+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:56:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.095300+0000 mgr.gibba032.whopeo (mgr.24625) 9654 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:56:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.097413+0000 mgr.gibba032.whopeo (mgr.24625) 9655 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:56:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.097656+0000 mon.a (mon.0) 3276 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:56:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.098967+0000 mgr.gibba032.whopeo (mgr.24625) 9656 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:56:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.099007+0000 mgr.gibba032.whopeo (mgr.24625) 9657 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:56:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.099611+0000 mon.a (mon.0) 3277 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.100642+0000 mgr.gibba032.whopeo (mgr.24625) 9658 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:56:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.100678+0000 mgr.gibba032.whopeo (mgr.24625) 9659 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:56:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.101202+0000 mon.a (mon.0) 3278 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.102086+0000 mgr.gibba032.whopeo (mgr.24625) 9660 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:56:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.102755+0000 mon.a (mon.0) 3279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.103295+0000 mgr.gibba032.whopeo (mgr.24625) 9661 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:56:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.103332+0000 mgr.gibba032.whopeo (mgr.24625) 9662 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:56:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.104756+0000 mgr.gibba032.whopeo (mgr.24625) 9663 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:56:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.105163+0000 mgr.gibba032.whopeo (mgr.24625) 9664 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:56:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.105476+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.106458+0000 mgr.gibba032.whopeo (mgr.24625) 9665 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:56:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cluster 2021-03-10T16:56:00.503413+0000 mgr.gibba032.whopeo (mgr.24625) 9651 : cluster [DBG] pgmap v5503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:00.885644+0000 mgr.gibba032.whopeo (mgr.24625) 9652 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:56:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:00.887183+0000 mgr.gibba032.whopeo (mgr.24625) 9653 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:56:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.094554+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:56:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.095300+0000 mgr.gibba032.whopeo (mgr.24625) 9654 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:56:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.097413+0000 mgr.gibba032.whopeo (mgr.24625) 9655 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:56:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.097656+0000 mon.a (mon.0) 3276 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:56:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.098967+0000 mgr.gibba032.whopeo (mgr.24625) 9656 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:56:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.099007+0000 mgr.gibba032.whopeo (mgr.24625) 9657 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:56:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.099611+0000 mon.a (mon.0) 3277 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.100642+0000 mgr.gibba032.whopeo (mgr.24625) 9658 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:56:02.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.100678+0000 mgr.gibba032.whopeo (mgr.24625) 9659 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:56:02.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.101202+0000 mon.a (mon.0) 3278 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.102086+0000 mgr.gibba032.whopeo (mgr.24625) 9660 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:56:02.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.102755+0000 mon.a (mon.0) 3279 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.103295+0000 mgr.gibba032.whopeo (mgr.24625) 9661 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:56:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.103332+0000 mgr.gibba032.whopeo (mgr.24625) 9662 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:56:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.104756+0000 mgr.gibba032.whopeo (mgr.24625) 9663 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:56:02.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.105163+0000 mgr.gibba032.whopeo (mgr.24625) 9664 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:56:02.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.105476+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.106458+0000 mgr.gibba032.whopeo (mgr.24625) 9665 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:56:02.589 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.106497+0000 mgr.gibba032.whopeo (mgr.24625) 9666 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:56:02.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: cephadm 2021-03-10T16:56:01.106607+0000 mgr.gibba032.whopeo (mgr.24625) 9667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:02.590 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[30791]: audit 2021-03-10T16:56:01.107195+0000 mon.a (mon.0) 3281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:02.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.106497+0000 mgr.gibba032.whopeo (mgr.24625) 9666 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:56:02.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: cephadm 2021-03-10T16:56:01.106607+0000 mgr.gibba032.whopeo (mgr.24625) 9667 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:02.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:02 gibba011 bash[31774]: audit 2021-03-10T16:56:01.107195+0000 mon.a (mon.0) 3281 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:04.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:04 gibba032 bash[24004]: cluster 2021-03-10T16:56:02.505362+0000 mgr.gibba032.whopeo (mgr.24625) 9668 : cluster [DBG] pgmap v5504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:04.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:04 gibba011 bash[30791]: cluster 2021-03-10T16:56:02.505362+0000 mgr.gibba032.whopeo (mgr.24625) 9668 : cluster [DBG] pgmap v5504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:04 gibba011 bash[31774]: cluster 2021-03-10T16:56:02.505362+0000 mgr.gibba032.whopeo (mgr.24625) 9668 : cluster [DBG] pgmap v5504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:06.577 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:06 gibba011 bash[30791]: cluster 2021-03-10T16:56:04.506017+0000 mgr.gibba032.whopeo (mgr.24625) 9669 : cluster [DBG] pgmap v5505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:06.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:06 gibba011 bash[31774]: cluster 2021-03-10T16:56:04.506017+0000 mgr.gibba032.whopeo (mgr.24625) 9669 : cluster [DBG] pgmap v5505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:06.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:06 gibba032 bash[24004]: cluster 2021-03-10T16:56:04.506017+0000 mgr.gibba032.whopeo (mgr.24625) 9669 : cluster [DBG] pgmap v5505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:08 gibba032 bash[24004]: cluster 2021-03-10T16:56:06.507823+0000 mgr.gibba032.whopeo (mgr.24625) 9670 : cluster [DBG] pgmap v5506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:08 gibba011 bash[30791]: cluster 2021-03-10T16:56:06.507823+0000 mgr.gibba032.whopeo (mgr.24625) 9670 : cluster [DBG] pgmap v5506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:08 gibba011 bash[31774]: cluster 2021-03-10T16:56:06.507823+0000 mgr.gibba032.whopeo (mgr.24625) 9670 : cluster [DBG] pgmap v5506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:10.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:10 gibba032 bash[24004]: cluster 2021-03-10T16:56:08.509455+0000 mgr.gibba032.whopeo (mgr.24625) 9671 : cluster [DBG] pgmap v5507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:10 gibba011 bash[30791]: cluster 2021-03-10T16:56:08.509455+0000 mgr.gibba032.whopeo (mgr.24625) 9671 : cluster [DBG] pgmap v5507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:10 gibba011 bash[31774]: cluster 2021-03-10T16:56:08.509455+0000 mgr.gibba032.whopeo (mgr.24625) 9671 : cluster [DBG] pgmap v5507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:12.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:12 gibba032 bash[24004]: cluster 2021-03-10T16:56:10.510419+0000 mgr.gibba032.whopeo (mgr.24625) 9672 : cluster [DBG] pgmap v5508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:12 gibba011 bash[30791]: cluster 2021-03-10T16:56:10.510419+0000 mgr.gibba032.whopeo (mgr.24625) 9672 : cluster [DBG] pgmap v5508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:12 gibba011 bash[31774]: cluster 2021-03-10T16:56:10.510419+0000 mgr.gibba032.whopeo (mgr.24625) 9672 : cluster [DBG] pgmap v5508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:13.393 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:56:13.624 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:56:13.625 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:56:13.625 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:56:13.625 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:56:13.625 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:56:13.626 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:56:13.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:56:13.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:56:13.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:56:13.626 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:56:13.627 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:56:13.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:56:13.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:56:13.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:56:13.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:56:13.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:56:13.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:56:13.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:56:13.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:56:13.629 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:56:13.629 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:56:13.872 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:56:13.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:56:13.873 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:56:13.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:13.874 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:56:13.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:56:13.874 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:13.874 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:56:13.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:56:13.875 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:13.875 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:56:13.875 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:56:13.875 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:56:13.875 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:56:13.876 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:56:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:14 gibba032 bash[24004]: cluster 2021-03-10T16:56:12.512292+0000 mgr.gibba032.whopeo (mgr.24625) 9673 : cluster [DBG] pgmap v5509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:14.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:14 gibba032 bash[24004]: audit 2021-03-10T16:56:13.871471+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.2.111:0/246021318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:14.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:14 gibba011 bash[30791]: cluster 2021-03-10T16:56:12.512292+0000 mgr.gibba032.whopeo (mgr.24625) 9673 : cluster [DBG] pgmap v5509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:14 gibba011 bash[30791]: audit 2021-03-10T16:56:13.871471+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.2.111:0/246021318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:14 gibba011 bash[31774]: cluster 2021-03-10T16:56:12.512292+0000 mgr.gibba032.whopeo (mgr.24625) 9673 : cluster [DBG] pgmap v5509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:14 gibba011 bash[31774]: audit 2021-03-10T16:56:13.871471+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.2.111:0/246021318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:15 gibba032 bash[24004]: audit 2021-03-10T16:56:13.384595+0000 mgr.gibba032.whopeo (mgr.24625) 9674 : audit [DBG] from='client.50535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:15.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:15 gibba032 bash[24004]: audit 2021-03-10T16:56:13.611546+0000 mgr.gibba032.whopeo (mgr.24625) 9675 : audit [DBG] from='client.60523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:15.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:15 gibba011 bash[30791]: audit 2021-03-10T16:56:13.384595+0000 mgr.gibba032.whopeo (mgr.24625) 9674 : audit [DBG] from='client.50535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:15.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:15 gibba011 bash[30791]: audit 2021-03-10T16:56:13.611546+0000 mgr.gibba032.whopeo (mgr.24625) 9675 : audit [DBG] from='client.60523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:15.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:15 gibba011 bash[31774]: audit 2021-03-10T16:56:13.384595+0000 mgr.gibba032.whopeo (mgr.24625) 9674 : audit [DBG] from='client.50535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:15.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:15 gibba011 bash[31774]: audit 2021-03-10T16:56:13.611546+0000 mgr.gibba032.whopeo (mgr.24625) 9675 : audit [DBG] from='client.60523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:16.505 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:16 gibba032 bash[24004]: cluster 2021-03-10T16:56:14.513969+0000 mgr.gibba032.whopeo (mgr.24625) 9676 : cluster [DBG] pgmap v5510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:16.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:16 gibba032 bash[24004]: cephadm 2021-03-10T16:56:16.120808+0000 mgr.gibba032.whopeo (mgr.24625) 9677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:16 gibba011 bash[30791]: cluster 2021-03-10T16:56:14.513969+0000 mgr.gibba032.whopeo (mgr.24625) 9676 : cluster [DBG] pgmap v5510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:16.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:16 gibba011 bash[30791]: cephadm 2021-03-10T16:56:16.120808+0000 mgr.gibba032.whopeo (mgr.24625) 9677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:16 gibba011 bash[31774]: cluster 2021-03-10T16:56:14.513969+0000 mgr.gibba032.whopeo (mgr.24625) 9676 : cluster [DBG] pgmap v5510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:16.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:16 gibba011 bash[31774]: cephadm 2021-03-10T16:56:16.120808+0000 mgr.gibba032.whopeo (mgr.24625) 9677 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:17 gibba032 bash[24004]: cluster 2021-03-10T16:56:16.514732+0000 mgr.gibba032.whopeo (mgr.24625) 9678 : cluster [DBG] pgmap v5511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:17.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:17 gibba011 bash[30791]: cluster 2021-03-10T16:56:16.514732+0000 mgr.gibba032.whopeo (mgr.24625) 9678 : cluster [DBG] pgmap v5511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:17.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:17 gibba011 bash[31774]: cluster 2021-03-10T16:56:16.514732+0000 mgr.gibba032.whopeo (mgr.24625) 9678 : cluster [DBG] pgmap v5511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:20.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:19 gibba032 bash[24004]: cluster 2021-03-10T16:56:18.516211+0000 mgr.gibba032.whopeo (mgr.24625) 9679 : cluster [DBG] pgmap v5512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:20.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:19 gibba011 bash[30791]: cluster 2021-03-10T16:56:18.516211+0000 mgr.gibba032.whopeo (mgr.24625) 9679 : cluster [DBG] pgmap v5512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:20.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:19 gibba011 bash[31774]: cluster 2021-03-10T16:56:18.516211+0000 mgr.gibba032.whopeo (mgr.24625) 9679 : cluster [DBG] pgmap v5512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:22 gibba032 bash[24004]: cluster 2021-03-10T16:56:20.517895+0000 mgr.gibba032.whopeo (mgr.24625) 9680 : cluster [DBG] pgmap v5513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:22 gibba011 bash[30791]: cluster 2021-03-10T16:56:20.517895+0000 mgr.gibba032.whopeo (mgr.24625) 9680 : cluster [DBG] pgmap v5513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:22 gibba011 bash[31774]: cluster 2021-03-10T16:56:20.517895+0000 mgr.gibba032.whopeo (mgr.24625) 9680 : cluster [DBG] pgmap v5513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:24.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:24 gibba032 bash[24004]: cluster 2021-03-10T16:56:22.518904+0000 mgr.gibba032.whopeo (mgr.24625) 9681 : cluster [DBG] pgmap v5514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:24.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:24 gibba011 bash[30791]: cluster 2021-03-10T16:56:22.518904+0000 mgr.gibba032.whopeo (mgr.24625) 9681 : cluster [DBG] pgmap v5514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:24.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:24 gibba011 bash[31774]: cluster 2021-03-10T16:56:22.518904+0000 mgr.gibba032.whopeo (mgr.24625) 9681 : cluster [DBG] pgmap v5514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:26 gibba032 bash[24004]: cluster 2021-03-10T16:56:24.520451+0000 mgr.gibba032.whopeo (mgr.24625) 9682 : cluster [DBG] pgmap v5515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:26 gibba011 bash[30791]: cluster 2021-03-10T16:56:24.520451+0000 mgr.gibba032.whopeo (mgr.24625) 9682 : cluster [DBG] pgmap v5515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:26 gibba011 bash[31774]: cluster 2021-03-10T16:56:24.520451+0000 mgr.gibba032.whopeo (mgr.24625) 9682 : cluster [DBG] pgmap v5515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:27.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:27 gibba032 bash[24004]: cluster 2021-03-10T16:56:26.522040+0000 mgr.gibba032.whopeo (mgr.24625) 9683 : cluster [DBG] pgmap v5516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:27.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:27 gibba011 bash[30791]: cluster 2021-03-10T16:56:26.522040+0000 mgr.gibba032.whopeo (mgr.24625) 9683 : cluster [DBG] pgmap v5516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:27.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:27 gibba011 bash[31774]: cluster 2021-03-10T16:56:26.522040+0000 mgr.gibba032.whopeo (mgr.24625) 9683 : cluster [DBG] pgmap v5516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:30.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:29 gibba032 bash[24004]: cluster 2021-03-10T16:56:28.522865+0000 mgr.gibba032.whopeo (mgr.24625) 9684 : cluster [DBG] pgmap v5517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:30.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:29 gibba011 bash[30791]: cluster 2021-03-10T16:56:28.522865+0000 mgr.gibba032.whopeo (mgr.24625) 9684 : cluster [DBG] pgmap v5517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:30.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:29 gibba011 bash[31774]: cluster 2021-03-10T16:56:28.522865+0000 mgr.gibba032.whopeo (mgr.24625) 9684 : cluster [DBG] pgmap v5517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:32 gibba032 bash[24004]: cluster 2021-03-10T16:56:30.524420+0000 mgr.gibba032.whopeo (mgr.24625) 9685 : cluster [DBG] pgmap v5518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:32 gibba032 bash[24004]: cephadm 2021-03-10T16:56:31.136642+0000 mgr.gibba032.whopeo (mgr.24625) 9686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:32 gibba011 bash[30791]: cluster 2021-03-10T16:56:30.524420+0000 mgr.gibba032.whopeo (mgr.24625) 9685 : cluster [DBG] pgmap v5518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:32 gibba011 bash[30791]: cephadm 2021-03-10T16:56:31.136642+0000 mgr.gibba032.whopeo (mgr.24625) 9686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:32 gibba011 bash[31774]: cluster 2021-03-10T16:56:30.524420+0000 mgr.gibba032.whopeo (mgr.24625) 9685 : cluster [DBG] pgmap v5518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:32 gibba011 bash[31774]: cephadm 2021-03-10T16:56:31.136642+0000 mgr.gibba032.whopeo (mgr.24625) 9686 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:34.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:34 gibba032 bash[24004]: cluster 2021-03-10T16:56:32.526214+0000 mgr.gibba032.whopeo (mgr.24625) 9687 : cluster [DBG] pgmap v5519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:34.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:34 gibba011 bash[30791]: cluster 2021-03-10T16:56:32.526214+0000 mgr.gibba032.whopeo (mgr.24625) 9687 : cluster [DBG] pgmap v5519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:34.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:34 gibba011 bash[31774]: cluster 2021-03-10T16:56:32.526214+0000 mgr.gibba032.whopeo (mgr.24625) 9687 : cluster [DBG] pgmap v5519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:36 gibba032 bash[24004]: cluster 2021-03-10T16:56:34.527757+0000 mgr.gibba032.whopeo (mgr.24625) 9688 : cluster [DBG] pgmap v5520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:36.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:36 gibba011 bash[30791]: cluster 2021-03-10T16:56:34.527757+0000 mgr.gibba032.whopeo (mgr.24625) 9688 : cluster [DBG] pgmap v5520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:36 gibba011 bash[31774]: cluster 2021-03-10T16:56:34.527757+0000 mgr.gibba032.whopeo (mgr.24625) 9688 : cluster [DBG] pgmap v5520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:37.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:37 gibba032 bash[24004]: cluster 2021-03-10T16:56:36.529415+0000 mgr.gibba032.whopeo (mgr.24625) 9689 : cluster [DBG] pgmap v5521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:37.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:37 gibba011 bash[30791]: cluster 2021-03-10T16:56:36.529415+0000 mgr.gibba032.whopeo (mgr.24625) 9689 : cluster [DBG] pgmap v5521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:37.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:37 gibba011 bash[31774]: cluster 2021-03-10T16:56:36.529415+0000 mgr.gibba032.whopeo (mgr.24625) 9689 : cluster [DBG] pgmap v5521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:40.005 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:39 gibba032 bash[24004]: cluster 2021-03-10T16:56:38.531175+0000 mgr.gibba032.whopeo (mgr.24625) 9690 : cluster [DBG] pgmap v5522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:40.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:39 gibba011 bash[30791]: cluster 2021-03-10T16:56:38.531175+0000 mgr.gibba032.whopeo (mgr.24625) 9690 : cluster [DBG] pgmap v5522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:40.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:39 gibba011 bash[31774]: cluster 2021-03-10T16:56:38.531175+0000 mgr.gibba032.whopeo (mgr.24625) 9690 : cluster [DBG] pgmap v5522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:42.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:42 gibba032 bash[24004]: cluster 2021-03-10T16:56:40.532356+0000 mgr.gibba032.whopeo (mgr.24625) 9691 : cluster [DBG] pgmap v5523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:42.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:42 gibba011 bash[30791]: cluster 2021-03-10T16:56:40.532356+0000 mgr.gibba032.whopeo (mgr.24625) 9691 : cluster [DBG] pgmap v5523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:42.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:42 gibba011 bash[31774]: cluster 2021-03-10T16:56:40.532356+0000 mgr.gibba032.whopeo (mgr.24625) 9691 : cluster [DBG] pgmap v5523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:44.112 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:56:44.336 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:56:44.336 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:56:44.337 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:56:44.337 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:56:44.337 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:56:44.337 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:56:44.337 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:56:44.338 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:56:44.338 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:56:44.338 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:56:44.338 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:56:44.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:56:44.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:56:44.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:56:44.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:56:44.339 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:56:44.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:56:44.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:56:44.340 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:56:44.340 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:56:44.340 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:56:44.590 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:56:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:56:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:56:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:56:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:56:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:56:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:56:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:56:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:56:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:56:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:56:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:56:44.593 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:56:44.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:44 gibba032 bash[24004]: cluster 2021-03-10T16:56:42.534194+0000 mgr.gibba032.whopeo (mgr.24625) 9692 : cluster [DBG] pgmap v5524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:44.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:44 gibba011 bash[30791]: cluster 2021-03-10T16:56:42.534194+0000 mgr.gibba032.whopeo (mgr.24625) 9692 : cluster [DBG] pgmap v5524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:44.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:44 gibba011 bash[31774]: cluster 2021-03-10T16:56:42.534194+0000 mgr.gibba032.whopeo (mgr.24625) 9692 : cluster [DBG] pgmap v5524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:45.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:45 gibba032 bash[24004]: audit 2021-03-10T16:56:44.101387+0000 mgr.gibba032.whopeo (mgr.24625) 9693 : audit [DBG] from='client.50553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:45 gibba032 bash[24004]: audit 2021-03-10T16:56:44.324261+0000 mgr.gibba032.whopeo (mgr.24625) 9694 : audit [DBG] from='client.50559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:45 gibba032 bash[24004]: audit 2021-03-10T16:56:44.591375+0000 mon.a (mon.0) 3283 : audit [DBG] from='client.? 172.21.2.111:0/2906129964' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[30791]: audit 2021-03-10T16:56:44.101387+0000 mgr.gibba032.whopeo (mgr.24625) 9693 : audit [DBG] from='client.50553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[30791]: audit 2021-03-10T16:56:44.324261+0000 mgr.gibba032.whopeo (mgr.24625) 9694 : audit [DBG] from='client.50559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[30791]: audit 2021-03-10T16:56:44.591375+0000 mon.a (mon.0) 3283 : audit [DBG] from='client.? 172.21.2.111:0/2906129964' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[31774]: audit 2021-03-10T16:56:44.101387+0000 mgr.gibba032.whopeo (mgr.24625) 9693 : audit [DBG] from='client.50553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[31774]: audit 2021-03-10T16:56:44.324261+0000 mgr.gibba032.whopeo (mgr.24625) 9694 : audit [DBG] from='client.50559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:56:45.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:45 gibba011 bash[31774]: audit 2021-03-10T16:56:44.591375+0000 mon.a (mon.0) 3283 : audit [DBG] from='client.? 172.21.2.111:0/2906129964' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:56:46.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:46 gibba032 bash[24004]: cluster 2021-03-10T16:56:44.535763+0000 mgr.gibba032.whopeo (mgr.24625) 9695 : cluster [DBG] pgmap v5525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:46 gibba011 bash[30791]: cluster 2021-03-10T16:56:44.535763+0000 mgr.gibba032.whopeo (mgr.24625) 9695 : cluster [DBG] pgmap v5525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:46 gibba011 bash[31774]: cluster 2021-03-10T16:56:44.535763+0000 mgr.gibba032.whopeo (mgr.24625) 9695 : cluster [DBG] pgmap v5525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:47 gibba032 bash[24004]: cephadm 2021-03-10T16:56:46.149713+0000 mgr.gibba032.whopeo (mgr.24625) 9696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:47.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:47 gibba011 bash[30791]: cephadm 2021-03-10T16:56:46.149713+0000 mgr.gibba032.whopeo (mgr.24625) 9696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:47.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:47 gibba011 bash[31774]: cephadm 2021-03-10T16:56:46.149713+0000 mgr.gibba032.whopeo (mgr.24625) 9696 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:56:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:48 gibba032 bash[24004]: cluster 2021-03-10T16:56:46.537494+0000 mgr.gibba032.whopeo (mgr.24625) 9697 : cluster [DBG] pgmap v5526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:48.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:48 gibba011 bash[30791]: cluster 2021-03-10T16:56:46.537494+0000 mgr.gibba032.whopeo (mgr.24625) 9697 : cluster [DBG] pgmap v5526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:48.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:48 gibba011 bash[31774]: cluster 2021-03-10T16:56:46.537494+0000 mgr.gibba032.whopeo (mgr.24625) 9697 : cluster [DBG] pgmap v5526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:49.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:49 gibba032 bash[24004]: cluster 2021-03-10T16:56:48.538977+0000 mgr.gibba032.whopeo (mgr.24625) 9698 : cluster [DBG] pgmap v5527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:49.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:49 gibba011 bash[30791]: cluster 2021-03-10T16:56:48.538977+0000 mgr.gibba032.whopeo (mgr.24625) 9698 : cluster [DBG] pgmap v5527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:49.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:49 gibba011 bash[31774]: cluster 2021-03-10T16:56:48.538977+0000 mgr.gibba032.whopeo (mgr.24625) 9698 : cluster [DBG] pgmap v5527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:52 gibba032 bash[24004]: cluster 2021-03-10T16:56:50.540539+0000 mgr.gibba032.whopeo (mgr.24625) 9699 : cluster [DBG] pgmap v5528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:52.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:52 gibba011 bash[30791]: cluster 2021-03-10T16:56:50.540539+0000 mgr.gibba032.whopeo (mgr.24625) 9699 : cluster [DBG] pgmap v5528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:52.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:52 gibba011 bash[31774]: cluster 2021-03-10T16:56:50.540539+0000 mgr.gibba032.whopeo (mgr.24625) 9699 : cluster [DBG] pgmap v5528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:54 gibba032 bash[24004]: cluster 2021-03-10T16:56:52.542343+0000 mgr.gibba032.whopeo (mgr.24625) 9700 : cluster [DBG] pgmap v5529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:54.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:54 gibba011 bash[30791]: cluster 2021-03-10T16:56:52.542343+0000 mgr.gibba032.whopeo (mgr.24625) 9700 : cluster [DBG] pgmap v5529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:54 gibba011 bash[31774]: cluster 2021-03-10T16:56:52.542343+0000 mgr.gibba032.whopeo (mgr.24625) 9700 : cluster [DBG] pgmap v5529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:56 gibba032 bash[24004]: cluster 2021-03-10T16:56:54.543823+0000 mgr.gibba032.whopeo (mgr.24625) 9701 : cluster [DBG] pgmap v5530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:56.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:56 gibba011 bash[30791]: cluster 2021-03-10T16:56:54.543823+0000 mgr.gibba032.whopeo (mgr.24625) 9701 : cluster [DBG] pgmap v5530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:56 gibba011 bash[31774]: cluster 2021-03-10T16:56:54.543823+0000 mgr.gibba032.whopeo (mgr.24625) 9701 : cluster [DBG] pgmap v5530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:57 gibba032 bash[24004]: cluster 2021-03-10T16:56:56.545492+0000 mgr.gibba032.whopeo (mgr.24625) 9702 : cluster [DBG] pgmap v5531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:57 gibba032 bash[24004]: audit 2021-03-10T16:56:57.246184+0000 mon.a (mon.0) 3284 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:56:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:57 gibba032 bash[24004]: audit 2021-03-10T16:56:57.246810+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:56:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[30791]: cluster 2021-03-10T16:56:56.545492+0000 mgr.gibba032.whopeo (mgr.24625) 9702 : cluster [DBG] pgmap v5531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[30791]: audit 2021-03-10T16:56:57.246184+0000 mon.a (mon.0) 3284 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:56:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[30791]: audit 2021-03-10T16:56:57.246810+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:56:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[31774]: cluster 2021-03-10T16:56:56.545492+0000 mgr.gibba032.whopeo (mgr.24625) 9702 : cluster [DBG] pgmap v5531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[31774]: audit 2021-03-10T16:56:57.246184+0000 mon.a (mon.0) 3284 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:56:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:57 gibba011 bash[31774]: audit 2021-03-10T16:56:57.246810+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:56:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:58 gibba032 bash[24004]: audit 2021-03-10T16:56:57.385385+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:56:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:58 gibba032 bash[24004]: audit 2021-03-10T16:56:57.385580+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T16:56:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:58 gibba032 bash[24004]: audit 2021-03-10T16:56:57.385734+0000 mon.a (mon.0) 3288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:56:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:58 gibba032 bash[24004]: audit 2021-03-10T16:56:57.385862+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:56:58.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[30791]: audit 2021-03-10T16:56:57.385385+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:56:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[30791]: audit 2021-03-10T16:56:57.385580+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T16:56:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[30791]: audit 2021-03-10T16:56:57.385734+0000 mon.a (mon.0) 3288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:56:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[30791]: audit 2021-03-10T16:56:57.385862+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:56:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[31774]: audit 2021-03-10T16:56:57.385385+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2021-03-10T16:56:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[31774]: audit 2021-03-10T16:56:57.385580+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2021-03-10T16:56:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[31774]: audit 2021-03-10T16:56:57.385734+0000 mon.a (mon.0) 3288 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:56:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:58 gibba011 bash[31774]: audit 2021-03-10T16:56:57.385862+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2021-03-10T16:56:59.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:56:59 gibba032 bash[24004]: cluster 2021-03-10T16:56:58.546433+0000 mgr.gibba032.whopeo (mgr.24625) 9703 : cluster [DBG] pgmap v5532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:59.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:56:59 gibba011 bash[30791]: cluster 2021-03-10T16:56:58.546433+0000 mgr.gibba032.whopeo (mgr.24625) 9703 : cluster [DBG] pgmap v5532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:56:59.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:56:59 gibba011 bash[31774]: cluster 2021-03-10T16:56:58.546433+0000 mgr.gibba032.whopeo (mgr.24625) 9703 : cluster [DBG] pgmap v5532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cluster 2021-03-10T16:57:00.547991+0000 mgr.gibba032.whopeo (mgr.24625) 9704 : cluster [DBG] pgmap v5533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.160714+0000 mgr.gibba032.whopeo (mgr.24625) 9705 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:57:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.162313+0000 mgr.gibba032.whopeo (mgr.24625) 9706 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:57:02.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.361935+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:57:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.362687+0000 mgr.gibba032.whopeo (mgr.24625) 9707 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:57:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.364773+0000 mgr.gibba032.whopeo (mgr.24625) 9708 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:57:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.365175+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:57:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.366530+0000 mgr.gibba032.whopeo (mgr.24625) 9709 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:57:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.366567+0000 mgr.gibba032.whopeo (mgr.24625) 9710 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:57:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.366924+0000 mon.a (mon.0) 3292 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.367995+0000 mgr.gibba032.whopeo (mgr.24625) 9711 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:57:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.368065+0000 mgr.gibba032.whopeo (mgr.24625) 9712 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:57:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.368718+0000 mon.a (mon.0) 3293 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.370163+0000 mon.a (mon.0) 3294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.373382+0000 mon.a (mon.0) 3295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:02 gibba032 bash[24004]: audit 2021-03-10T16:57:01.375030+0000 mon.a (mon.0) 3296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cluster 2021-03-10T16:57:00.547991+0000 mgr.gibba032.whopeo (mgr.24625) 9704 : cluster [DBG] pgmap v5533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.160714+0000 mgr.gibba032.whopeo (mgr.24625) 9705 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:57:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.162313+0000 mgr.gibba032.whopeo (mgr.24625) 9706 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:57:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.361935+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:57:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.362687+0000 mgr.gibba032.whopeo (mgr.24625) 9707 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:57:02.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.364773+0000 mgr.gibba032.whopeo (mgr.24625) 9708 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:57:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.365175+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:57:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.366530+0000 mgr.gibba032.whopeo (mgr.24625) 9709 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:57:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.366567+0000 mgr.gibba032.whopeo (mgr.24625) 9710 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:57:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.366924+0000 mon.a (mon.0) 3292 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.367995+0000 mgr.gibba032.whopeo (mgr.24625) 9711 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:57:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.368065+0000 mgr.gibba032.whopeo (mgr.24625) 9712 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:57:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.368718+0000 mon.a (mon.0) 3293 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.370163+0000 mon.a (mon.0) 3294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.373382+0000 mon.a (mon.0) 3295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[30791]: audit 2021-03-10T16:57:01.375030+0000 mon.a (mon.0) 3296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cluster 2021-03-10T16:57:00.547991+0000 mgr.gibba032.whopeo (mgr.24625) 9704 : cluster [DBG] pgmap v5533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.160714+0000 mgr.gibba032.whopeo (mgr.24625) 9705 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:57:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.162313+0000 mgr.gibba032.whopeo (mgr.24625) 9706 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:57:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.361935+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:57:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.362687+0000 mgr.gibba032.whopeo (mgr.24625) 9707 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:57:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.364773+0000 mgr.gibba032.whopeo (mgr.24625) 9708 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:57:02.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.365175+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:57:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.366530+0000 mgr.gibba032.whopeo (mgr.24625) 9709 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:57:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.366567+0000 mgr.gibba032.whopeo (mgr.24625) 9710 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:57:02.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.366924+0000 mon.a (mon.0) 3292 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.367995+0000 mgr.gibba032.whopeo (mgr.24625) 9711 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:57:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.368065+0000 mgr.gibba032.whopeo (mgr.24625) 9712 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:57:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.368718+0000 mon.a (mon.0) 3293 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.370163+0000 mon.a (mon.0) 3294 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.373382+0000 mon.a (mon.0) 3295 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:02.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:02 gibba011 bash[31774]: audit 2021-03-10T16:57:01.375030+0000 mon.a (mon.0) 3296 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:03.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.369443+0000 mgr.gibba032.whopeo (mgr.24625) 9713 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:57:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.371118+0000 mgr.gibba032.whopeo (mgr.24625) 9714 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:57:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.371156+0000 mgr.gibba032.whopeo (mgr.24625) 9715 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:57:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.372648+0000 mgr.gibba032.whopeo (mgr.24625) 9716 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:57:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.372968+0000 mgr.gibba032.whopeo (mgr.24625) 9717 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:57:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.374569+0000 mgr.gibba032.whopeo (mgr.24625) 9718 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:57:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.374607+0000 mgr.gibba032.whopeo (mgr.24625) 9719 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:57:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:03 gibba032 bash[24004]: cephadm 2021-03-10T16:57:01.374709+0000 mgr.gibba032.whopeo (mgr.24625) 9720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.369443+0000 mgr.gibba032.whopeo (mgr.24625) 9713 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:57:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.371118+0000 mgr.gibba032.whopeo (mgr.24625) 9714 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:57:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.371156+0000 mgr.gibba032.whopeo (mgr.24625) 9715 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:57:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.372648+0000 mgr.gibba032.whopeo (mgr.24625) 9716 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:57:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.372968+0000 mgr.gibba032.whopeo (mgr.24625) 9717 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:57:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.374569+0000 mgr.gibba032.whopeo (mgr.24625) 9718 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:57:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.374607+0000 mgr.gibba032.whopeo (mgr.24625) 9719 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:57:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[30791]: cephadm 2021-03-10T16:57:01.374709+0000 mgr.gibba032.whopeo (mgr.24625) 9720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.369443+0000 mgr.gibba032.whopeo (mgr.24625) 9713 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:57:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.371118+0000 mgr.gibba032.whopeo (mgr.24625) 9714 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:57:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.371156+0000 mgr.gibba032.whopeo (mgr.24625) 9715 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:57:03.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.372648+0000 mgr.gibba032.whopeo (mgr.24625) 9716 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:57:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.372968+0000 mgr.gibba032.whopeo (mgr.24625) 9717 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:57:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.374569+0000 mgr.gibba032.whopeo (mgr.24625) 9718 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:57:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.374607+0000 mgr.gibba032.whopeo (mgr.24625) 9719 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:57:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:03 gibba011 bash[31774]: cephadm 2021-03-10T16:57:01.374709+0000 mgr.gibba032.whopeo (mgr.24625) 9720 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:04 gibba032 bash[24004]: cluster 2021-03-10T16:57:02.549812+0000 mgr.gibba032.whopeo (mgr.24625) 9721 : cluster [DBG] pgmap v5534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:04.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:04 gibba011 bash[30791]: cluster 2021-03-10T16:57:02.549812+0000 mgr.gibba032.whopeo (mgr.24625) 9721 : cluster [DBG] pgmap v5534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:04.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:04 gibba011 bash[31774]: cluster 2021-03-10T16:57:02.549812+0000 mgr.gibba032.whopeo (mgr.24625) 9721 : cluster [DBG] pgmap v5534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:06 gibba032 bash[24004]: cluster 2021-03-10T16:57:04.550752+0000 mgr.gibba032.whopeo (mgr.24625) 9722 : cluster [DBG] pgmap v5535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:06.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:06 gibba011 bash[30791]: cluster 2021-03-10T16:57:04.550752+0000 mgr.gibba032.whopeo (mgr.24625) 9722 : cluster [DBG] pgmap v5535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:06 gibba011 bash[31774]: cluster 2021-03-10T16:57:04.550752+0000 mgr.gibba032.whopeo (mgr.24625) 9722 : cluster [DBG] pgmap v5535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:08 gibba032 bash[24004]: cluster 2021-03-10T16:57:06.552332+0000 mgr.gibba032.whopeo (mgr.24625) 9723 : cluster [DBG] pgmap v5536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:08.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:08 gibba011 bash[30791]: cluster 2021-03-10T16:57:06.552332+0000 mgr.gibba032.whopeo (mgr.24625) 9723 : cluster [DBG] pgmap v5536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:08 gibba011 bash[31774]: cluster 2021-03-10T16:57:06.552332+0000 mgr.gibba032.whopeo (mgr.24625) 9723 : cluster [DBG] pgmap v5536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:10 gibba032 bash[24004]: cluster 2021-03-10T16:57:08.553955+0000 mgr.gibba032.whopeo (mgr.24625) 9724 : cluster [DBG] pgmap v5537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:10 gibba011 bash[30791]: cluster 2021-03-10T16:57:08.553955+0000 mgr.gibba032.whopeo (mgr.24625) 9724 : cluster [DBG] pgmap v5537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:10 gibba011 bash[31774]: cluster 2021-03-10T16:57:08.553955+0000 mgr.gibba032.whopeo (mgr.24625) 9724 : cluster [DBG] pgmap v5537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:11 gibba032 bash[24004]: cluster 2021-03-10T16:57:10.555054+0000 mgr.gibba032.whopeo (mgr.24625) 9725 : cluster [DBG] pgmap v5538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:11 gibba011 bash[30791]: cluster 2021-03-10T16:57:10.555054+0000 mgr.gibba032.whopeo (mgr.24625) 9725 : cluster [DBG] pgmap v5538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:11 gibba011 bash[31774]: cluster 2021-03-10T16:57:10.555054+0000 mgr.gibba032.whopeo (mgr.24625) 9725 : cluster [DBG] pgmap v5538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:14.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:13 gibba032 bash[24004]: cluster 2021-03-10T16:57:12.556816+0000 mgr.gibba032.whopeo (mgr.24625) 9726 : cluster [DBG] pgmap v5539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:13 gibba011 bash[30791]: cluster 2021-03-10T16:57:12.556816+0000 mgr.gibba032.whopeo (mgr.24625) 9726 : cluster [DBG] pgmap v5539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:14.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:13 gibba011 bash[31774]: cluster 2021-03-10T16:57:12.556816+0000 mgr.gibba032.whopeo (mgr.24625) 9726 : cluster [DBG] pgmap v5539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:14.838 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:57:15.094 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:57:15.095 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:57:15.095 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:57:15.096 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:57:15.096 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:57:15.096 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:57:15.096 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:57:15.097 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:57:15.097 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:57:15.097 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:57:15.097 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:57:15.097 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:57:15.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:57:15.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:57:15.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:57:15.098 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:57:15.099 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:57:15.099 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:57:15.099 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:57:15.099 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:57:15.099 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:57:15.351 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:57:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:57:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:57:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:57:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:57:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:57:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:57:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:57:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:57:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:57:15.354 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:57:15.354 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:57:16.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:15 gibba032 bash[24004]: cluster 2021-03-10T16:57:14.558357+0000 mgr.gibba032.whopeo (mgr.24625) 9727 : cluster [DBG] pgmap v5540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:16.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:15 gibba032 bash[24004]: audit 2021-03-10T16:57:14.828965+0000 mgr.gibba032.whopeo (mgr.24625) 9728 : audit [DBG] from='client.60553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:15 gibba032 bash[24004]: audit 2021-03-10T16:57:15.053768+0000 mgr.gibba032.whopeo (mgr.24625) 9729 : audit [DBG] from='client.50577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:15 gibba032 bash[24004]: audit 2021-03-10T16:57:15.352020+0000 mon.c (mon.1) 189 : audit [DBG] from='client.? 172.21.2.111:0/2095666789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[30791]: cluster 2021-03-10T16:57:14.558357+0000 mgr.gibba032.whopeo (mgr.24625) 9727 : cluster [DBG] pgmap v5540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[30791]: audit 2021-03-10T16:57:14.828965+0000 mgr.gibba032.whopeo (mgr.24625) 9728 : audit [DBG] from='client.60553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[30791]: audit 2021-03-10T16:57:15.053768+0000 mgr.gibba032.whopeo (mgr.24625) 9729 : audit [DBG] from='client.50577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[30791]: audit 2021-03-10T16:57:15.352020+0000 mon.c (mon.1) 189 : audit [DBG] from='client.? 172.21.2.111:0/2095666789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:16.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[31774]: cluster 2021-03-10T16:57:14.558357+0000 mgr.gibba032.whopeo (mgr.24625) 9727 : cluster [DBG] pgmap v5540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:16.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[31774]: audit 2021-03-10T16:57:14.828965+0000 mgr.gibba032.whopeo (mgr.24625) 9728 : audit [DBG] from='client.60553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[31774]: audit 2021-03-10T16:57:15.053768+0000 mgr.gibba032.whopeo (mgr.24625) 9729 : audit [DBG] from='client.50577 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:16.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:15 gibba011 bash[31774]: audit 2021-03-10T16:57:15.352020+0000 mon.c (mon.1) 189 : audit [DBG] from='client.? 172.21.2.111:0/2095666789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:18.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:17 gibba032 bash[24004]: cephadm 2021-03-10T16:57:16.390798+0000 mgr.gibba032.whopeo (mgr.24625) 9730 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:17 gibba032 bash[24004]: cluster 2021-03-10T16:57:16.559701+0000 mgr.gibba032.whopeo (mgr.24625) 9731 : cluster [DBG] pgmap v5541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:18.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:17 gibba011 bash[30791]: cephadm 2021-03-10T16:57:16.390798+0000 mgr.gibba032.whopeo (mgr.24625) 9730 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:18.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:17 gibba011 bash[30791]: cluster 2021-03-10T16:57:16.559701+0000 mgr.gibba032.whopeo (mgr.24625) 9731 : cluster [DBG] pgmap v5541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:18.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:17 gibba011 bash[31774]: cephadm 2021-03-10T16:57:16.390798+0000 mgr.gibba032.whopeo (mgr.24625) 9730 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:18.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:17 gibba011 bash[31774]: cluster 2021-03-10T16:57:16.559701+0000 mgr.gibba032.whopeo (mgr.24625) 9731 : cluster [DBG] pgmap v5541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:20.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:19 gibba032 bash[24004]: cluster 2021-03-10T16:57:18.561311+0000 mgr.gibba032.whopeo (mgr.24625) 9732 : cluster [DBG] pgmap v5542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:19 gibba011 bash[30791]: cluster 2021-03-10T16:57:18.561311+0000 mgr.gibba032.whopeo (mgr.24625) 9732 : cluster [DBG] pgmap v5542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:20.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:19 gibba011 bash[31774]: cluster 2021-03-10T16:57:18.561311+0000 mgr.gibba032.whopeo (mgr.24625) 9732 : cluster [DBG] pgmap v5542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:22.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:22 gibba032 bash[24004]: cluster 2021-03-10T16:57:20.562697+0000 mgr.gibba032.whopeo (mgr.24625) 9733 : cluster [DBG] pgmap v5543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:22.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:22 gibba011 bash[30791]: cluster 2021-03-10T16:57:20.562697+0000 mgr.gibba032.whopeo (mgr.24625) 9733 : cluster [DBG] pgmap v5543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:22.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:22 gibba011 bash[31774]: cluster 2021-03-10T16:57:20.562697+0000 mgr.gibba032.whopeo (mgr.24625) 9733 : cluster [DBG] pgmap v5543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:23.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:23 gibba032 bash[24004]: cluster 2021-03-10T16:57:22.564493+0000 mgr.gibba032.whopeo (mgr.24625) 9734 : cluster [DBG] pgmap v5544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:23.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:23 gibba011 bash[30791]: cluster 2021-03-10T16:57:22.564493+0000 mgr.gibba032.whopeo (mgr.24625) 9734 : cluster [DBG] pgmap v5544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:23.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:23 gibba011 bash[31774]: cluster 2021-03-10T16:57:22.564493+0000 mgr.gibba032.whopeo (mgr.24625) 9734 : cluster [DBG] pgmap v5544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:26 gibba032 bash[24004]: cluster 2021-03-10T16:57:24.566079+0000 mgr.gibba032.whopeo (mgr.24625) 9735 : cluster [DBG] pgmap v5545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:26 gibba011 bash[30791]: cluster 2021-03-10T16:57:24.566079+0000 mgr.gibba032.whopeo (mgr.24625) 9735 : cluster [DBG] pgmap v5545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:26 gibba011 bash[31774]: cluster 2021-03-10T16:57:24.566079+0000 mgr.gibba032.whopeo (mgr.24625) 9735 : cluster [DBG] pgmap v5545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:28.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:28 gibba032 bash[24004]: cluster 2021-03-10T16:57:26.567901+0000 mgr.gibba032.whopeo (mgr.24625) 9736 : cluster [DBG] pgmap v5546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:28.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:28 gibba011 bash[30791]: cluster 2021-03-10T16:57:26.567901+0000 mgr.gibba032.whopeo (mgr.24625) 9736 : cluster [DBG] pgmap v5546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:28.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:28 gibba011 bash[31774]: cluster 2021-03-10T16:57:26.567901+0000 mgr.gibba032.whopeo (mgr.24625) 9736 : cluster [DBG] pgmap v5546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:30.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:30 gibba032 bash[24004]: cluster 2021-03-10T16:57:28.569487+0000 mgr.gibba032.whopeo (mgr.24625) 9737 : cluster [DBG] pgmap v5547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:30.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:30 gibba011 bash[30791]: cluster 2021-03-10T16:57:28.569487+0000 mgr.gibba032.whopeo (mgr.24625) 9737 : cluster [DBG] pgmap v5547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:30.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:30 gibba011 bash[31774]: cluster 2021-03-10T16:57:28.569487+0000 mgr.gibba032.whopeo (mgr.24625) 9737 : cluster [DBG] pgmap v5547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:31 gibba032 bash[24004]: cluster 2021-03-10T16:57:30.570958+0000 mgr.gibba032.whopeo (mgr.24625) 9738 : cluster [DBG] pgmap v5548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:31 gibba011 bash[30791]: cluster 2021-03-10T16:57:30.570958+0000 mgr.gibba032.whopeo (mgr.24625) 9738 : cluster [DBG] pgmap v5548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:31.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:31 gibba011 bash[31774]: cluster 2021-03-10T16:57:30.570958+0000 mgr.gibba032.whopeo (mgr.24625) 9738 : cluster [DBG] pgmap v5548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:32 gibba032 bash[24004]: cephadm 2021-03-10T16:57:31.404557+0000 mgr.gibba032.whopeo (mgr.24625) 9739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:32 gibba011 bash[30791]: cephadm 2021-03-10T16:57:31.404557+0000 mgr.gibba032.whopeo (mgr.24625) 9739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:32 gibba011 bash[31774]: cephadm 2021-03-10T16:57:31.404557+0000 mgr.gibba032.whopeo (mgr.24625) 9739 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:33 gibba032 bash[24004]: cluster 2021-03-10T16:57:32.572623+0000 mgr.gibba032.whopeo (mgr.24625) 9740 : cluster [DBG] pgmap v5549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:33.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:33 gibba011 bash[30791]: cluster 2021-03-10T16:57:32.572623+0000 mgr.gibba032.whopeo (mgr.24625) 9740 : cluster [DBG] pgmap v5549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:33.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:33 gibba011 bash[31774]: cluster 2021-03-10T16:57:32.572623+0000 mgr.gibba032.whopeo (mgr.24625) 9740 : cluster [DBG] pgmap v5549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:36 gibba032 bash[24004]: cluster 2021-03-10T16:57:34.574120+0000 mgr.gibba032.whopeo (mgr.24625) 9741 : cluster [DBG] pgmap v5550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:36 gibba011 bash[30791]: cluster 2021-03-10T16:57:34.574120+0000 mgr.gibba032.whopeo (mgr.24625) 9741 : cluster [DBG] pgmap v5550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:36.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:36 gibba011 bash[31774]: cluster 2021-03-10T16:57:34.574120+0000 mgr.gibba032.whopeo (mgr.24625) 9741 : cluster [DBG] pgmap v5550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:38.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:38 gibba032 bash[24004]: cluster 2021-03-10T16:57:36.575931+0000 mgr.gibba032.whopeo (mgr.24625) 9742 : cluster [DBG] pgmap v5551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:38.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:38 gibba011 bash[30791]: cluster 2021-03-10T16:57:36.575931+0000 mgr.gibba032.whopeo (mgr.24625) 9742 : cluster [DBG] pgmap v5551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:38.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:38 gibba011 bash[31774]: cluster 2021-03-10T16:57:36.575931+0000 mgr.gibba032.whopeo (mgr.24625) 9742 : cluster [DBG] pgmap v5551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:40.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:40 gibba032 bash[24004]: cluster 2021-03-10T16:57:38.577746+0000 mgr.gibba032.whopeo (mgr.24625) 9743 : cluster [DBG] pgmap v5552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:40.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:40 gibba011 bash[30791]: cluster 2021-03-10T16:57:38.577746+0000 mgr.gibba032.whopeo (mgr.24625) 9743 : cluster [DBG] pgmap v5552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:40.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:40 gibba011 bash[31774]: cluster 2021-03-10T16:57:38.577746+0000 mgr.gibba032.whopeo (mgr.24625) 9743 : cluster [DBG] pgmap v5552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:41.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:41 gibba032 bash[24004]: cluster 2021-03-10T16:57:40.579192+0000 mgr.gibba032.whopeo (mgr.24625) 9744 : cluster [DBG] pgmap v5553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:41 gibba011 bash[30791]: cluster 2021-03-10T16:57:40.579192+0000 mgr.gibba032.whopeo (mgr.24625) 9744 : cluster [DBG] pgmap v5553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:41.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:41 gibba011 bash[31774]: cluster 2021-03-10T16:57:40.579192+0000 mgr.gibba032.whopeo (mgr.24625) 9744 : cluster [DBG] pgmap v5553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:43 gibba032 bash[24004]: cluster 2021-03-10T16:57:42.580806+0000 mgr.gibba032.whopeo (mgr.24625) 9745 : cluster [DBG] pgmap v5554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:44.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:43 gibba011 bash[30791]: cluster 2021-03-10T16:57:42.580806+0000 mgr.gibba032.whopeo (mgr.24625) 9745 : cluster [DBG] pgmap v5554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:44.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:43 gibba011 bash[31774]: cluster 2021-03-10T16:57:42.580806+0000 mgr.gibba032.whopeo (mgr.24625) 9745 : cluster [DBG] pgmap v5554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:45.599 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:57:45.849 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:57:45.849 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:57:45.849 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:57:45.850 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:57:45.850 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:57:45.850 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:57:45.850 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:57:45.851 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:57:45.851 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:57:45.851 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:57:45.851 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:57:45.851 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:57:45.852 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:57:45.852 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:57:45.852 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:57:45.852 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:57:45.852 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:57:45.853 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:57:45.853 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (2h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:57:45.853 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:57:45.853 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:57:46.100 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:57:46.100 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:57:46.100 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:57:46.101 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:46.101 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:57:46.101 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:57:46.101 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:46.101 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:57:46.102 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:57:46.102 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:57:46.102 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:57:46.102 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:57:46.103 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:57:46.103 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:57:46.103 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:57:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:46 gibba032 bash[24004]: cluster 2021-03-10T16:57:44.582441+0000 mgr.gibba032.whopeo (mgr.24625) 9746 : cluster [DBG] pgmap v5555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:46.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:46 gibba032 bash[24004]: audit 2021-03-10T16:57:46.101325+0000 mon.c (mon.1) 190 : audit [DBG] from='client.? 172.21.2.111:0/222274622' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:46.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:46 gibba011 bash[30791]: cluster 2021-03-10T16:57:44.582441+0000 mgr.gibba032.whopeo (mgr.24625) 9746 : cluster [DBG] pgmap v5555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:46.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:46 gibba011 bash[30791]: audit 2021-03-10T16:57:46.101325+0000 mon.c (mon.1) 190 : audit [DBG] from='client.? 172.21.2.111:0/222274622' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:46 gibba011 bash[31774]: cluster 2021-03-10T16:57:44.582441+0000 mgr.gibba032.whopeo (mgr.24625) 9746 : cluster [DBG] pgmap v5555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:46.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:46 gibba011 bash[31774]: audit 2021-03-10T16:57:46.101325+0000 mon.c (mon.1) 190 : audit [DBG] from='client.? 172.21.2.111:0/222274622' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:57:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:47 gibba032 bash[24004]: audit 2021-03-10T16:57:45.588622+0000 mgr.gibba032.whopeo (mgr.24625) 9747 : audit [DBG] from='client.50589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:47.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:47 gibba032 bash[24004]: audit 2021-03-10T16:57:45.810970+0000 mgr.gibba032.whopeo (mgr.24625) 9748 : audit [DBG] from='client.50595 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:47.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:47 gibba011 bash[30791]: audit 2021-03-10T16:57:45.588622+0000 mgr.gibba032.whopeo (mgr.24625) 9747 : audit [DBG] from='client.50589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:47.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:47 gibba011 bash[30791]: audit 2021-03-10T16:57:45.810970+0000 mgr.gibba032.whopeo (mgr.24625) 9748 : audit [DBG] from='client.50595 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:47.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:47 gibba011 bash[31774]: audit 2021-03-10T16:57:45.588622+0000 mgr.gibba032.whopeo (mgr.24625) 9747 : audit [DBG] from='client.50589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:47.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:47 gibba011 bash[31774]: audit 2021-03-10T16:57:45.810970+0000 mgr.gibba032.whopeo (mgr.24625) 9748 : audit [DBG] from='client.50595 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:57:48.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:48 gibba032 bash[24004]: cephadm 2021-03-10T16:57:46.411907+0000 mgr.gibba032.whopeo (mgr.24625) 9749 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:48 gibba032 bash[24004]: cluster 2021-03-10T16:57:46.584218+0000 mgr.gibba032.whopeo (mgr.24625) 9750 : cluster [DBG] pgmap v5556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:48 gibba011 bash[30791]: cephadm 2021-03-10T16:57:46.411907+0000 mgr.gibba032.whopeo (mgr.24625) 9749 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:48.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:48 gibba011 bash[30791]: cluster 2021-03-10T16:57:46.584218+0000 mgr.gibba032.whopeo (mgr.24625) 9750 : cluster [DBG] pgmap v5556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:48 gibba011 bash[31774]: cephadm 2021-03-10T16:57:46.411907+0000 mgr.gibba032.whopeo (mgr.24625) 9749 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:57:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:48 gibba011 bash[31774]: cluster 2021-03-10T16:57:46.584218+0000 mgr.gibba032.whopeo (mgr.24625) 9750 : cluster [DBG] pgmap v5556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:49.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:49 gibba032 bash[24004]: cluster 2021-03-10T16:57:48.585621+0000 mgr.gibba032.whopeo (mgr.24625) 9751 : cluster [DBG] pgmap v5557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:49.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:49 gibba011 bash[30791]: cluster 2021-03-10T16:57:48.585621+0000 mgr.gibba032.whopeo (mgr.24625) 9751 : cluster [DBG] pgmap v5557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:49.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:49 gibba011 bash[31774]: cluster 2021-03-10T16:57:48.585621+0000 mgr.gibba032.whopeo (mgr.24625) 9751 : cluster [DBG] pgmap v5557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:52.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:52 gibba032 bash[24004]: cluster 2021-03-10T16:57:50.587138+0000 mgr.gibba032.whopeo (mgr.24625) 9752 : cluster [DBG] pgmap v5558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:52.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:52 gibba011 bash[30791]: cluster 2021-03-10T16:57:50.587138+0000 mgr.gibba032.whopeo (mgr.24625) 9752 : cluster [DBG] pgmap v5558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:52 gibba011 bash[31774]: cluster 2021-03-10T16:57:50.587138+0000 mgr.gibba032.whopeo (mgr.24625) 9752 : cluster [DBG] pgmap v5558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:54.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:54 gibba032 bash[24004]: cluster 2021-03-10T16:57:52.588900+0000 mgr.gibba032.whopeo (mgr.24625) 9753 : cluster [DBG] pgmap v5559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:54.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:54 gibba011 bash[30791]: cluster 2021-03-10T16:57:52.588900+0000 mgr.gibba032.whopeo (mgr.24625) 9753 : cluster [DBG] pgmap v5559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:54.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:54 gibba011 bash[31774]: cluster 2021-03-10T16:57:52.588900+0000 mgr.gibba032.whopeo (mgr.24625) 9753 : cluster [DBG] pgmap v5559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:56.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:56 gibba032 bash[24004]: cluster 2021-03-10T16:57:54.590309+0000 mgr.gibba032.whopeo (mgr.24625) 9754 : cluster [DBG] pgmap v5560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:56.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:56 gibba011 bash[30791]: cluster 2021-03-10T16:57:54.590309+0000 mgr.gibba032.whopeo (mgr.24625) 9754 : cluster [DBG] pgmap v5560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:56.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:56 gibba011 bash[31774]: cluster 2021-03-10T16:57:54.590309+0000 mgr.gibba032.whopeo (mgr.24625) 9754 : cluster [DBG] pgmap v5560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:57.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:57 gibba032 bash[24004]: audit 2021-03-10T16:57:57.253216+0000 mon.a (mon.0) 3297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:57:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:57 gibba032 bash[24004]: audit 2021-03-10T16:57:57.253941+0000 mon.a (mon.0) 3298 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:57:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:57 gibba011 bash[30791]: audit 2021-03-10T16:57:57.253216+0000 mon.a (mon.0) 3297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:57:57.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:57 gibba011 bash[30791]: audit 2021-03-10T16:57:57.253941+0000 mon.a (mon.0) 3298 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:57:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:57 gibba011 bash[31774]: audit 2021-03-10T16:57:57.253216+0000 mon.a (mon.0) 3297 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:57:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:57 gibba011 bash[31774]: audit 2021-03-10T16:57:57.253941+0000 mon.a (mon.0) 3298 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:57:58.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:58 gibba032 bash[24004]: cluster 2021-03-10T16:57:56.591979+0000 mgr.gibba032.whopeo (mgr.24625) 9755 : cluster [DBG] pgmap v5561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:58.805 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:58 gibba032 bash[24004]: audit 2021-03-10T16:57:57.391912+0000 mon.a (mon.0) 3299 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:57:58.805 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:58 gibba032 bash[24004]: audit 2021-03-10T16:57:57.392032+0000 mon.a (mon.0) 3300 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:57:58.806 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:58 gibba032 bash[24004]: audit 2021-03-10T16:57:57.392112+0000 mon.a (mon.0) 3301 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:57:58.806 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:58 gibba032 bash[24004]: audit 2021-03-10T16:57:57.393805+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:57:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[30791]: cluster 2021-03-10T16:57:56.591979+0000 mgr.gibba032.whopeo (mgr.24625) 9755 : cluster [DBG] pgmap v5561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[30791]: audit 2021-03-10T16:57:57.391912+0000 mon.a (mon.0) 3299 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:57:58.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[30791]: audit 2021-03-10T16:57:57.392032+0000 mon.a (mon.0) 3300 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:57:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[30791]: audit 2021-03-10T16:57:57.392112+0000 mon.a (mon.0) 3301 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:57:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[30791]: audit 2021-03-10T16:57:57.393805+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:57:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[31774]: cluster 2021-03-10T16:57:56.591979+0000 mgr.gibba032.whopeo (mgr.24625) 9755 : cluster [DBG] pgmap v5561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:58.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[31774]: audit 2021-03-10T16:57:57.391912+0000 mon.a (mon.0) 3299 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T16:57:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[31774]: audit 2021-03-10T16:57:57.392032+0000 mon.a (mon.0) 3300 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T16:57:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[31774]: audit 2021-03-10T16:57:57.392112+0000 mon.a (mon.0) 3301 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:57:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:58 gibba011 bash[31774]: audit 2021-03-10T16:57:57.393805+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T16:57:59.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:57:59 gibba032 bash[24004]: cluster 2021-03-10T16:57:58.593777+0000 mgr.gibba032.whopeo (mgr.24625) 9756 : cluster [DBG] pgmap v5562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:59.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:57:59 gibba011 bash[30791]: cluster 2021-03-10T16:57:58.593777+0000 mgr.gibba032.whopeo (mgr.24625) 9756 : cluster [DBG] pgmap v5562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:57:59.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:57:59 gibba011 bash[31774]: cluster 2021-03-10T16:57:58.593777+0000 mgr.gibba032.whopeo (mgr.24625) 9756 : cluster [DBG] pgmap v5562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:01.755 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:01 gibba032 bash[24004]: cluster 2021-03-10T16:58:00.595136+0000 mgr.gibba032.whopeo (mgr.24625) 9757 : cluster [DBG] pgmap v5563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:01.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:01 gibba011 bash[30791]: cluster 2021-03-10T16:58:00.595136+0000 mgr.gibba032.whopeo (mgr.24625) 9757 : cluster [DBG] pgmap v5563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:01.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:01 gibba011 bash[31774]: cluster 2021-03-10T16:58:00.595136+0000 mgr.gibba032.whopeo (mgr.24625) 9757 : cluster [DBG] pgmap v5563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:03.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.624558+0000 mgr.gibba032.whopeo (mgr.24625) 9758 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:58:03.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.625695+0000 mon.a (mon.0) 3303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.628071+0000 mgr.gibba032.whopeo (mgr.24625) 9759 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:58:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.629063+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.829841+0000 mon.a (mon.0) 3305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.830630+0000 mgr.gibba032.whopeo (mgr.24625) 9760 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:58:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.832339+0000 mgr.gibba032.whopeo (mgr.24625) 9761 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:58:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.833007+0000 mon.a (mon.0) 3306 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:58:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.833497+0000 mgr.gibba032.whopeo (mgr.24625) 9762 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:58:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.833536+0000 mgr.gibba032.whopeo (mgr.24625) 9763 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:58:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.834311+0000 mon.a (mon.0) 3307 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.834625+0000 mgr.gibba032.whopeo (mgr.24625) 9764 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:58:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.834664+0000 mgr.gibba032.whopeo (mgr.24625) 9765 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:58:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.835500+0000 mon.a (mon.0) 3308 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.835739+0000 mgr.gibba032.whopeo (mgr.24625) 9766 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:58:03.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.836567+0000 mon.a (mon.0) 3309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.836777+0000 mgr.gibba032.whopeo (mgr.24625) 9767 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:58:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.836814+0000 mgr.gibba032.whopeo (mgr.24625) 9768 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:58:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.838096+0000 mgr.gibba032.whopeo (mgr.24625) 9769 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:58:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.838419+0000 mgr.gibba032.whopeo (mgr.24625) 9770 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:58:03.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.838870+0000 mon.a (mon.0) 3310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.839508+0000 mgr.gibba032.whopeo (mgr.24625) 9771 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:58:03.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.839548+0000 mgr.gibba032.whopeo (mgr.24625) 9772 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:58:03.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: cephadm 2021-03-10T16:58:01.839655+0000 mgr.gibba032.whopeo (mgr.24625) 9773 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:03.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:02 gibba032 bash[24004]: audit 2021-03-10T16:58:01.840335+0000 mon.a (mon.0) 3311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.624558+0000 mgr.gibba032.whopeo (mgr.24625) 9758 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:58:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.625695+0000 mon.a (mon.0) 3303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.628071+0000 mgr.gibba032.whopeo (mgr.24625) 9759 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:58:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.629063+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.829841+0000 mon.a (mon.0) 3305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.830630+0000 mgr.gibba032.whopeo (mgr.24625) 9760 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:58:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.832339+0000 mgr.gibba032.whopeo (mgr.24625) 9761 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:58:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.833007+0000 mon.a (mon.0) 3306 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:58:03.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.833497+0000 mgr.gibba032.whopeo (mgr.24625) 9762 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:58:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.833536+0000 mgr.gibba032.whopeo (mgr.24625) 9763 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:58:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.834311+0000 mon.a (mon.0) 3307 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.834625+0000 mgr.gibba032.whopeo (mgr.24625) 9764 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:58:03.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.834664+0000 mgr.gibba032.whopeo (mgr.24625) 9765 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:58:03.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.835500+0000 mon.a (mon.0) 3308 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.835739+0000 mgr.gibba032.whopeo (mgr.24625) 9766 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:58:03.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.836567+0000 mon.a (mon.0) 3309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.836777+0000 mgr.gibba032.whopeo (mgr.24625) 9767 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:58:03.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.836814+0000 mgr.gibba032.whopeo (mgr.24625) 9768 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:58:03.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.838096+0000 mgr.gibba032.whopeo (mgr.24625) 9769 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:58:03.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.838419+0000 mgr.gibba032.whopeo (mgr.24625) 9770 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:58:03.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.838870+0000 mon.a (mon.0) 3310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.839508+0000 mgr.gibba032.whopeo (mgr.24625) 9771 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:58:03.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.839548+0000 mgr.gibba032.whopeo (mgr.24625) 9772 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:58:03.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: cephadm 2021-03-10T16:58:01.839655+0000 mgr.gibba032.whopeo (mgr.24625) 9773 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:03.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[30791]: audit 2021-03-10T16:58:01.840335+0000 mon.a (mon.0) 3311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.624558+0000 mgr.gibba032.whopeo (mgr.24625) 9758 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:58:03.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.625695+0000 mon.a (mon.0) 3303 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.628071+0000 mgr.gibba032.whopeo (mgr.24625) 9759 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:58:03.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.629063+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.829841+0000 mon.a (mon.0) 3305 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:58:03.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.830630+0000 mgr.gibba032.whopeo (mgr.24625) 9760 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:58:03.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.832339+0000 mgr.gibba032.whopeo (mgr.24625) 9761 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:58:03.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.833007+0000 mon.a (mon.0) 3306 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:58:03.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.833497+0000 mgr.gibba032.whopeo (mgr.24625) 9762 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:58:03.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.833536+0000 mgr.gibba032.whopeo (mgr.24625) 9763 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:58:03.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.834311+0000 mon.a (mon.0) 3307 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.834625+0000 mgr.gibba032.whopeo (mgr.24625) 9764 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:58:03.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.834664+0000 mgr.gibba032.whopeo (mgr.24625) 9765 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:58:03.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.835500+0000 mon.a (mon.0) 3308 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.835739+0000 mgr.gibba032.whopeo (mgr.24625) 9766 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:58:03.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.836567+0000 mon.a (mon.0) 3309 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.836777+0000 mgr.gibba032.whopeo (mgr.24625) 9767 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:58:03.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.836814+0000 mgr.gibba032.whopeo (mgr.24625) 9768 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:58:03.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.838096+0000 mgr.gibba032.whopeo (mgr.24625) 9769 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:58:03.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.838419+0000 mgr.gibba032.whopeo (mgr.24625) 9770 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:58:03.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.838870+0000 mon.a (mon.0) 3310 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:03.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.839508+0000 mgr.gibba032.whopeo (mgr.24625) 9771 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:58:03.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.839548+0000 mgr.gibba032.whopeo (mgr.24625) 9772 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:58:03.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: cephadm 2021-03-10T16:58:01.839655+0000 mgr.gibba032.whopeo (mgr.24625) 9773 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:03.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:02 gibba011 bash[31774]: audit 2021-03-10T16:58:01.840335+0000 mon.a (mon.0) 3311 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:04.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:03 gibba032 bash[24004]: cluster 2021-03-10T16:58:02.596848+0000 mgr.gibba032.whopeo (mgr.24625) 9774 : cluster [DBG] pgmap v5564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:04.327 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:03 gibba011 bash[30791]: cluster 2021-03-10T16:58:02.596848+0000 mgr.gibba032.whopeo (mgr.24625) 9774 : cluster [DBG] pgmap v5564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:04.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:03 gibba011 bash[31774]: cluster 2021-03-10T16:58:02.596848+0000 mgr.gibba032.whopeo (mgr.24625) 9774 : cluster [DBG] pgmap v5564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:06 gibba032 bash[24004]: cluster 2021-03-10T16:58:04.598223+0000 mgr.gibba032.whopeo (mgr.24625) 9775 : cluster [DBG] pgmap v5565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:06.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:06 gibba011 bash[30791]: cluster 2021-03-10T16:58:04.598223+0000 mgr.gibba032.whopeo (mgr.24625) 9775 : cluster [DBG] pgmap v5565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:06 gibba011 bash[31774]: cluster 2021-03-10T16:58:04.598223+0000 mgr.gibba032.whopeo (mgr.24625) 9775 : cluster [DBG] pgmap v5565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:07.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:07 gibba032 bash[24004]: cluster 2021-03-10T16:58:06.599749+0000 mgr.gibba032.whopeo (mgr.24625) 9776 : cluster [DBG] pgmap v5566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:07.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:07 gibba011 bash[30791]: cluster 2021-03-10T16:58:06.599749+0000 mgr.gibba032.whopeo (mgr.24625) 9776 : cluster [DBG] pgmap v5566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:07.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:07 gibba011 bash[31774]: cluster 2021-03-10T16:58:06.599749+0000 mgr.gibba032.whopeo (mgr.24625) 9776 : cluster [DBG] pgmap v5566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:10.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:09 gibba032 bash[24004]: cluster 2021-03-10T16:58:08.601389+0000 mgr.gibba032.whopeo (mgr.24625) 9777 : cluster [DBG] pgmap v5567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:10.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:09 gibba011 bash[30791]: cluster 2021-03-10T16:58:08.601389+0000 mgr.gibba032.whopeo (mgr.24625) 9777 : cluster [DBG] pgmap v5567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:10.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:09 gibba011 bash[31774]: cluster 2021-03-10T16:58:08.601389+0000 mgr.gibba032.whopeo (mgr.24625) 9777 : cluster [DBG] pgmap v5567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:11 gibba032 bash[24004]: cluster 2021-03-10T16:58:10.602878+0000 mgr.gibba032.whopeo (mgr.24625) 9778 : cluster [DBG] pgmap v5568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:11 gibba011 bash[30791]: cluster 2021-03-10T16:58:10.602878+0000 mgr.gibba032.whopeo (mgr.24625) 9778 : cluster [DBG] pgmap v5568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:11 gibba011 bash[31774]: cluster 2021-03-10T16:58:10.602878+0000 mgr.gibba032.whopeo (mgr.24625) 9778 : cluster [DBG] pgmap v5568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:14.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:13 gibba032 bash[24004]: cluster 2021-03-10T16:58:12.604588+0000 mgr.gibba032.whopeo (mgr.24625) 9779 : cluster [DBG] pgmap v5569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:13 gibba011 bash[30791]: cluster 2021-03-10T16:58:12.604588+0000 mgr.gibba032.whopeo (mgr.24625) 9779 : cluster [DBG] pgmap v5569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:14.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:13 gibba011 bash[31774]: cluster 2021-03-10T16:58:12.604588+0000 mgr.gibba032.whopeo (mgr.24625) 9779 : cluster [DBG] pgmap v5569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:16.341 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:58:16.570 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:58:16.570 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:58:16.571 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:58:16.571 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:58:16.571 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:58:16.571 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:58:16.572 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:58:16.572 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:58:16.572 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:58:16.572 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:58:16.573 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:58:16.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:58:16.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:58:16.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:58:16.573 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:58:16.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:58:16.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:58:16.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:58:16.574 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:58:16.575 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:58:16.575 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:58:16.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:16 gibba032 bash[24004]: cluster 2021-03-10T16:58:14.605880+0000 mgr.gibba032.whopeo (mgr.24625) 9780 : cluster [DBG] pgmap v5570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:16.823 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:58:16.823 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:58:16.823 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:58:16.823 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:16.824 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:58:16.824 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:58:16.824 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:16.824 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:58:16.824 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:58:16.825 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:16.825 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:58:16.825 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:58:16.825 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:58:16.826 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:58:16.826 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:58:16.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:16 gibba011 bash[30791]: cluster 2021-03-10T16:58:14.605880+0000 mgr.gibba032.whopeo (mgr.24625) 9780 : cluster [DBG] pgmap v5570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:16.827 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:16 gibba011 bash[31774]: cluster 2021-03-10T16:58:14.605880+0000 mgr.gibba032.whopeo (mgr.24625) 9780 : cluster [DBG] pgmap v5570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:17 gibba032 bash[24004]: audit 2021-03-10T16:58:16.331347+0000 mgr.gibba032.whopeo (mgr.24625) 9781 : audit [DBG] from='client.50607 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:17.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:17 gibba032 bash[24004]: audit 2021-03-10T16:58:16.824049+0000 mon.a (mon.0) 3312 : audit [DBG] from='client.? 172.21.2.111:0/1994016768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:17.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:17 gibba011 bash[30791]: audit 2021-03-10T16:58:16.331347+0000 mgr.gibba032.whopeo (mgr.24625) 9781 : audit [DBG] from='client.50607 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:17.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:17 gibba011 bash[30791]: audit 2021-03-10T16:58:16.824049+0000 mon.a (mon.0) 3312 : audit [DBG] from='client.? 172.21.2.111:0/1994016768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:17.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:17 gibba011 bash[31774]: audit 2021-03-10T16:58:16.331347+0000 mgr.gibba032.whopeo (mgr.24625) 9781 : audit [DBG] from='client.50607 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:17.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:17 gibba011 bash[31774]: audit 2021-03-10T16:58:16.824049+0000 mon.a (mon.0) 3312 : audit [DBG] from='client.? 172.21.2.111:0/1994016768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:18 gibba032 bash[24004]: audit 2021-03-10T16:58:16.557678+0000 mgr.gibba032.whopeo (mgr.24625) 9782 : audit [DBG] from='client.50613 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:18.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:18 gibba032 bash[24004]: cluster 2021-03-10T16:58:16.606800+0000 mgr.gibba032.whopeo (mgr.24625) 9783 : cluster [DBG] pgmap v5571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:18 gibba032 bash[24004]: cephadm 2021-03-10T16:58:16.854291+0000 mgr.gibba032.whopeo (mgr.24625) 9784 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:18.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[30791]: audit 2021-03-10T16:58:16.557678+0000 mgr.gibba032.whopeo (mgr.24625) 9782 : audit [DBG] from='client.50613 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[30791]: cluster 2021-03-10T16:58:16.606800+0000 mgr.gibba032.whopeo (mgr.24625) 9783 : cluster [DBG] pgmap v5571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[30791]: cephadm 2021-03-10T16:58:16.854291+0000 mgr.gibba032.whopeo (mgr.24625) 9784 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[31774]: audit 2021-03-10T16:58:16.557678+0000 mgr.gibba032.whopeo (mgr.24625) 9782 : audit [DBG] from='client.50613 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[31774]: cluster 2021-03-10T16:58:16.606800+0000 mgr.gibba032.whopeo (mgr.24625) 9783 : cluster [DBG] pgmap v5571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:18 gibba011 bash[31774]: cephadm 2021-03-10T16:58:16.854291+0000 mgr.gibba032.whopeo (mgr.24625) 9784 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:19.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:19 gibba032 bash[24004]: cluster 2021-03-10T16:58:18.608466+0000 mgr.gibba032.whopeo (mgr.24625) 9785 : cluster [DBG] pgmap v5572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:19.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:19 gibba011 bash[30791]: cluster 2021-03-10T16:58:18.608466+0000 mgr.gibba032.whopeo (mgr.24625) 9785 : cluster [DBG] pgmap v5572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:19.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:19 gibba011 bash[31774]: cluster 2021-03-10T16:58:18.608466+0000 mgr.gibba032.whopeo (mgr.24625) 9785 : cluster [DBG] pgmap v5572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:21.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:21 gibba032 bash[24004]: cluster 2021-03-10T16:58:20.609864+0000 mgr.gibba032.whopeo (mgr.24625) 9786 : cluster [DBG] pgmap v5573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:21 gibba011 bash[30791]: cluster 2021-03-10T16:58:20.609864+0000 mgr.gibba032.whopeo (mgr.24625) 9786 : cluster [DBG] pgmap v5573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:21.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:21 gibba011 bash[31774]: cluster 2021-03-10T16:58:20.609864+0000 mgr.gibba032.whopeo (mgr.24625) 9786 : cluster [DBG] pgmap v5573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:24.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:23 gibba032 bash[24004]: cluster 2021-03-10T16:58:22.611636+0000 mgr.gibba032.whopeo (mgr.24625) 9787 : cluster [DBG] pgmap v5574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:24.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:23 gibba011 bash[30791]: cluster 2021-03-10T16:58:22.611636+0000 mgr.gibba032.whopeo (mgr.24625) 9787 : cluster [DBG] pgmap v5574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:24.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:23 gibba011 bash[31774]: cluster 2021-03-10T16:58:22.611636+0000 mgr.gibba032.whopeo (mgr.24625) 9787 : cluster [DBG] pgmap v5574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:26.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:26 gibba032 bash[24004]: cluster 2021-03-10T16:58:24.613286+0000 mgr.gibba032.whopeo (mgr.24625) 9788 : cluster [DBG] pgmap v5575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:26 gibba011 bash[30791]: cluster 2021-03-10T16:58:24.613286+0000 mgr.gibba032.whopeo (mgr.24625) 9788 : cluster [DBG] pgmap v5575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:26.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:26 gibba011 bash[31774]: cluster 2021-03-10T16:58:24.613286+0000 mgr.gibba032.whopeo (mgr.24625) 9788 : cluster [DBG] pgmap v5575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:27.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:27 gibba032 bash[24004]: cluster 2021-03-10T16:58:26.614965+0000 mgr.gibba032.whopeo (mgr.24625) 9789 : cluster [DBG] pgmap v5576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:27.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:27 gibba011 bash[30791]: cluster 2021-03-10T16:58:26.614965+0000 mgr.gibba032.whopeo (mgr.24625) 9789 : cluster [DBG] pgmap v5576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:27.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:27 gibba011 bash[31774]: cluster 2021-03-10T16:58:26.614965+0000 mgr.gibba032.whopeo (mgr.24625) 9789 : cluster [DBG] pgmap v5576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:30.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:29 gibba032 bash[24004]: cluster 2021-03-10T16:58:28.616654+0000 mgr.gibba032.whopeo (mgr.24625) 9790 : cluster [DBG] pgmap v5577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:30.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:29 gibba011 bash[30791]: cluster 2021-03-10T16:58:28.616654+0000 mgr.gibba032.whopeo (mgr.24625) 9790 : cluster [DBG] pgmap v5577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:30.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:29 gibba011 bash[31774]: cluster 2021-03-10T16:58:28.616654+0000 mgr.gibba032.whopeo (mgr.24625) 9790 : cluster [DBG] pgmap v5577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:31 gibba032 bash[24004]: cluster 2021-03-10T16:58:30.618031+0000 mgr.gibba032.whopeo (mgr.24625) 9791 : cluster [DBG] pgmap v5578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:31 gibba011 bash[30791]: cluster 2021-03-10T16:58:30.618031+0000 mgr.gibba032.whopeo (mgr.24625) 9791 : cluster [DBG] pgmap v5578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:31.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:31 gibba011 bash[31774]: cluster 2021-03-10T16:58:30.618031+0000 mgr.gibba032.whopeo (mgr.24625) 9791 : cluster [DBG] pgmap v5578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:32 gibba032 bash[24004]: cephadm 2021-03-10T16:58:31.860605+0000 mgr.gibba032.whopeo (mgr.24625) 9792 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:32 gibba011 bash[30791]: cephadm 2021-03-10T16:58:31.860605+0000 mgr.gibba032.whopeo (mgr.24625) 9792 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:32 gibba011 bash[31774]: cephadm 2021-03-10T16:58:31.860605+0000 mgr.gibba032.whopeo (mgr.24625) 9792 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:33 gibba032 bash[24004]: cluster 2021-03-10T16:58:32.619786+0000 mgr.gibba032.whopeo (mgr.24625) 9793 : cluster [DBG] pgmap v5579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:33.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:33 gibba011 bash[30791]: cluster 2021-03-10T16:58:32.619786+0000 mgr.gibba032.whopeo (mgr.24625) 9793 : cluster [DBG] pgmap v5579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:33.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:33 gibba011 bash[31774]: cluster 2021-03-10T16:58:32.619786+0000 mgr.gibba032.whopeo (mgr.24625) 9793 : cluster [DBG] pgmap v5579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:36.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:36 gibba032 bash[24004]: cluster 2021-03-10T16:58:34.621439+0000 mgr.gibba032.whopeo (mgr.24625) 9794 : cluster [DBG] pgmap v5580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:36.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:36 gibba011 bash[30791]: cluster 2021-03-10T16:58:34.621439+0000 mgr.gibba032.whopeo (mgr.24625) 9794 : cluster [DBG] pgmap v5580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:36 gibba011 bash[31774]: cluster 2021-03-10T16:58:34.621439+0000 mgr.gibba032.whopeo (mgr.24625) 9794 : cluster [DBG] pgmap v5580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:37.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:37 gibba032 bash[24004]: cluster 2021-03-10T16:58:36.623232+0000 mgr.gibba032.whopeo (mgr.24625) 9795 : cluster [DBG] pgmap v5581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:37.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:37 gibba011 bash[30791]: cluster 2021-03-10T16:58:36.623232+0000 mgr.gibba032.whopeo (mgr.24625) 9795 : cluster [DBG] pgmap v5581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:37.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:37 gibba011 bash[31774]: cluster 2021-03-10T16:58:36.623232+0000 mgr.gibba032.whopeo (mgr.24625) 9795 : cluster [DBG] pgmap v5581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:40.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:39 gibba032 bash[24004]: cluster 2021-03-10T16:58:38.624739+0000 mgr.gibba032.whopeo (mgr.24625) 9796 : cluster [DBG] pgmap v5582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:40.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:39 gibba011 bash[30791]: cluster 2021-03-10T16:58:38.624739+0000 mgr.gibba032.whopeo (mgr.24625) 9796 : cluster [DBG] pgmap v5582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:40.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:39 gibba011 bash[31774]: cluster 2021-03-10T16:58:38.624739+0000 mgr.gibba032.whopeo (mgr.24625) 9796 : cluster [DBG] pgmap v5582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:41.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:41 gibba032 bash[24004]: cluster 2021-03-10T16:58:40.626313+0000 mgr.gibba032.whopeo (mgr.24625) 9797 : cluster [DBG] pgmap v5583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:41 gibba011 bash[30791]: cluster 2021-03-10T16:58:40.626313+0000 mgr.gibba032.whopeo (mgr.24625) 9797 : cluster [DBG] pgmap v5583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:41.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:41 gibba011 bash[31774]: cluster 2021-03-10T16:58:40.626313+0000 mgr.gibba032.whopeo (mgr.24625) 9797 : cluster [DBG] pgmap v5583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:43 gibba032 bash[24004]: cluster 2021-03-10T16:58:42.627970+0000 mgr.gibba032.whopeo (mgr.24625) 9798 : cluster [DBG] pgmap v5584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:44.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:43 gibba011 bash[30791]: cluster 2021-03-10T16:58:42.627970+0000 mgr.gibba032.whopeo (mgr.24625) 9798 : cluster [DBG] pgmap v5584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:44.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:43 gibba011 bash[31774]: cluster 2021-03-10T16:58:42.627970+0000 mgr.gibba032.whopeo (mgr.24625) 9798 : cluster [DBG] pgmap v5584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:46.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:45 gibba032 bash[24004]: cluster 2021-03-10T16:58:44.629534+0000 mgr.gibba032.whopeo (mgr.24625) 9799 : cluster [DBG] pgmap v5585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:46.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:45 gibba011 bash[30791]: cluster 2021-03-10T16:58:44.629534+0000 mgr.gibba032.whopeo (mgr.24625) 9799 : cluster [DBG] pgmap v5585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:46.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:45 gibba011 bash[31774]: cluster 2021-03-10T16:58:44.629534+0000 mgr.gibba032.whopeo (mgr.24625) 9799 : cluster [DBG] pgmap v5585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:47.067 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:58:47.292 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:58:47.292 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:58:47.292 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:58:47.292 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:58:47.293 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:58:47.293 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:58:47.293 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:58:47.293 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:58:47.294 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:58:47.294 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:58:47.294 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:58:47.294 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:58:47.294 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:58:47.295 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:58:47.295 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:58:47.295 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:58:47.295 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:58:47.296 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:58:47.296 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:58:47.296 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:58:47.296 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:58:47.540 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:58:47.540 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:58:47.540 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:58:47.541 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:47.541 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:58:47.541 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:58:47.541 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:47.542 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:58:47.542 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:58:47.542 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:58:47.542 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:58:47.543 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:58:47.543 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:58:47.543 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:58:47.543 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:58:48.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:47 gibba032 bash[24004]: cluster 2021-03-10T16:58:46.631180+0000 mgr.gibba032.whopeo (mgr.24625) 9800 : cluster [DBG] pgmap v5586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:47 gibba032 bash[24004]: cephadm 2021-03-10T16:58:46.874933+0000 mgr.gibba032.whopeo (mgr.24625) 9801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:47 gibba032 bash[24004]: audit 2021-03-10T16:58:47.057371+0000 mgr.gibba032.whopeo (mgr.24625) 9802 : audit [DBG] from='client.50625 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:47 gibba032 bash[24004]: audit 2021-03-10T16:58:47.278687+0000 mgr.gibba032.whopeo (mgr.24625) 9803 : audit [DBG] from='client.50631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:47 gibba032 bash[24004]: audit 2021-03-10T16:58:47.540994+0000 mon.a (mon.0) 3313 : audit [DBG] from='client.? 172.21.2.111:0/2069235359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:48.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[30791]: cluster 2021-03-10T16:58:46.631180+0000 mgr.gibba032.whopeo (mgr.24625) 9800 : cluster [DBG] pgmap v5586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[30791]: cephadm 2021-03-10T16:58:46.874933+0000 mgr.gibba032.whopeo (mgr.24625) 9801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[30791]: audit 2021-03-10T16:58:47.057371+0000 mgr.gibba032.whopeo (mgr.24625) 9802 : audit [DBG] from='client.50625 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[30791]: audit 2021-03-10T16:58:47.278687+0000 mgr.gibba032.whopeo (mgr.24625) 9803 : audit [DBG] from='client.50631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[30791]: audit 2021-03-10T16:58:47.540994+0000 mon.a (mon.0) 3313 : audit [DBG] from='client.? 172.21.2.111:0/2069235359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[31774]: cluster 2021-03-10T16:58:46.631180+0000 mgr.gibba032.whopeo (mgr.24625) 9800 : cluster [DBG] pgmap v5586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[31774]: cephadm 2021-03-10T16:58:46.874933+0000 mgr.gibba032.whopeo (mgr.24625) 9801 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:58:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[31774]: audit 2021-03-10T16:58:47.057371+0000 mgr.gibba032.whopeo (mgr.24625) 9802 : audit [DBG] from='client.50625 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[31774]: audit 2021-03-10T16:58:47.278687+0000 mgr.gibba032.whopeo (mgr.24625) 9803 : audit [DBG] from='client.50631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:58:48.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:47 gibba011 bash[31774]: audit 2021-03-10T16:58:47.540994+0000 mon.a (mon.0) 3313 : audit [DBG] from='client.? 172.21.2.111:0/2069235359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:58:50.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:49 gibba032 bash[24004]: cluster 2021-03-10T16:58:48.632729+0000 mgr.gibba032.whopeo (mgr.24625) 9804 : cluster [DBG] pgmap v5587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:50.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:49 gibba011 bash[30791]: cluster 2021-03-10T16:58:48.632729+0000 mgr.gibba032.whopeo (mgr.24625) 9804 : cluster [DBG] pgmap v5587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:50.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:49 gibba011 bash[31774]: cluster 2021-03-10T16:58:48.632729+0000 mgr.gibba032.whopeo (mgr.24625) 9804 : cluster [DBG] pgmap v5587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:51 gibba032 bash[24004]: cluster 2021-03-10T16:58:50.634097+0000 mgr.gibba032.whopeo (mgr.24625) 9805 : cluster [DBG] pgmap v5588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:51 gibba011 bash[30791]: cluster 2021-03-10T16:58:50.634097+0000 mgr.gibba032.whopeo (mgr.24625) 9805 : cluster [DBG] pgmap v5588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:51.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:51 gibba011 bash[31774]: cluster 2021-03-10T16:58:50.634097+0000 mgr.gibba032.whopeo (mgr.24625) 9805 : cluster [DBG] pgmap v5588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:54.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:53 gibba032 bash[24004]: cluster 2021-03-10T16:58:52.635729+0000 mgr.gibba032.whopeo (mgr.24625) 9806 : cluster [DBG] pgmap v5589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:54.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:53 gibba011 bash[30791]: cluster 2021-03-10T16:58:52.635729+0000 mgr.gibba032.whopeo (mgr.24625) 9806 : cluster [DBG] pgmap v5589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:54.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:53 gibba011 bash[31774]: cluster 2021-03-10T16:58:52.635729+0000 mgr.gibba032.whopeo (mgr.24625) 9806 : cluster [DBG] pgmap v5589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:56.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:55 gibba032 bash[24004]: cluster 2021-03-10T16:58:54.637212+0000 mgr.gibba032.whopeo (mgr.24625) 9807 : cluster [DBG] pgmap v5590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:55 gibba011 bash[30791]: cluster 2021-03-10T16:58:54.637212+0000 mgr.gibba032.whopeo (mgr.24625) 9807 : cluster [DBG] pgmap v5590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:56.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:55 gibba011 bash[31774]: cluster 2021-03-10T16:58:54.637212+0000 mgr.gibba032.whopeo (mgr.24625) 9807 : cluster [DBG] pgmap v5590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:58.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: cluster 2021-03-10T16:58:56.638860+0000 mgr.gibba032.whopeo (mgr.24625) 9808 : cluster [DBG] pgmap v5591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.261189+0000 mon.a (mon.0) 3314 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:58:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.261890+0000 mon.a (mon.0) 3315 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:58:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.401514+0000 mon.a (mon.0) 3316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:58:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.402259+0000 mon.a (mon.0) 3317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:58:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.402940+0000 mon.a (mon.0) 3318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:58:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:57 gibba032 bash[24004]: audit 2021-03-10T16:58:57.403706+0000 mon.a (mon.0) 3319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:58:58.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: cluster 2021-03-10T16:58:56.638860+0000 mgr.gibba032.whopeo (mgr.24625) 9808 : cluster [DBG] pgmap v5591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.261189+0000 mon.a (mon.0) 3314 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:58:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.261890+0000 mon.a (mon.0) 3315 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:58:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.401514+0000 mon.a (mon.0) 3316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:58:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.402259+0000 mon.a (mon.0) 3317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:58:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.402940+0000 mon.a (mon.0) 3318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:58:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[30791]: audit 2021-03-10T16:58:57.403706+0000 mon.a (mon.0) 3319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:58:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: cluster 2021-03-10T16:58:56.638860+0000 mgr.gibba032.whopeo (mgr.24625) 9808 : cluster [DBG] pgmap v5591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:58:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.261189+0000 mon.a (mon.0) 3314 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:58:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.261890+0000 mon.a (mon.0) 3315 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:58:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.401514+0000 mon.a (mon.0) 3316 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T16:58:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.402259+0000 mon.a (mon.0) 3317 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T16:58:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.402940+0000 mon.a (mon.0) 3318 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:58:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:57 gibba011 bash[31774]: audit 2021-03-10T16:58:57.403706+0000 mon.a (mon.0) 3319 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T16:59:00.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:58:59 gibba032 bash[24004]: cluster 2021-03-10T16:58:58.640422+0000 mgr.gibba032.whopeo (mgr.24625) 9809 : cluster [DBG] pgmap v5592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:00.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:58:59 gibba011 bash[30791]: cluster 2021-03-10T16:58:58.640422+0000 mgr.gibba032.whopeo (mgr.24625) 9809 : cluster [DBG] pgmap v5592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:58:59 gibba011 bash[31774]: cluster 2021-03-10T16:58:58.640422+0000 mgr.gibba032.whopeo (mgr.24625) 9809 : cluster [DBG] pgmap v5592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:01 gibba032 bash[24004]: cluster 2021-03-10T16:59:00.641788+0000 mgr.gibba032.whopeo (mgr.24625) 9810 : cluster [DBG] pgmap v5593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:01 gibba011 bash[30791]: cluster 2021-03-10T16:59:00.641788+0000 mgr.gibba032.whopeo (mgr.24625) 9810 : cluster [DBG] pgmap v5593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:01.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:01 gibba011 bash[31774]: cluster 2021-03-10T16:59:00.641788+0000 mgr.gibba032.whopeo (mgr.24625) 9810 : cluster [DBG] pgmap v5593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:01.878409+0000 mgr.gibba032.whopeo (mgr.24625) 9811 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:59:03.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:01.879959+0000 mgr.gibba032.whopeo (mgr.24625) 9812 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:59:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.096731+0000 mon.a (mon.0) 3320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:59:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.097611+0000 mgr.gibba032.whopeo (mgr.24625) 9813 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:59:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.099229+0000 mgr.gibba032.whopeo (mgr.24625) 9814 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:59:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.099974+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:59:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.100359+0000 mgr.gibba032.whopeo (mgr.24625) 9815 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:59:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.100395+0000 mgr.gibba032.whopeo (mgr.24625) 9816 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:59:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.101241+0000 mon.a (mon.0) 3322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.101467+0000 mgr.gibba032.whopeo (mgr.24625) 9817 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:59:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.101504+0000 mgr.gibba032.whopeo (mgr.24625) 9818 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:59:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.102386+0000 mon.a (mon.0) 3323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.102769+0000 mgr.gibba032.whopeo (mgr.24625) 9819 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:59:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.103507+0000 mon.a (mon.0) 3324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.103828+0000 mgr.gibba032.whopeo (mgr.24625) 9820 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:59:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.103869+0000 mgr.gibba032.whopeo (mgr.24625) 9821 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:59:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.105230+0000 mgr.gibba032.whopeo (mgr.24625) 9822 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:59:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.105541+0000 mgr.gibba032.whopeo (mgr.24625) 9823 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:59:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.106154+0000 mon.a (mon.0) 3325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.106781+0000 mgr.gibba032.whopeo (mgr.24625) 9824 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:59:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.106820+0000 mgr.gibba032.whopeo (mgr.24625) 9825 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:59:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: cephadm 2021-03-10T16:59:02.106970+0000 mgr.gibba032.whopeo (mgr.24625) 9826 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:03 gibba032 bash[24004]: audit 2021-03-10T16:59:02.107622+0000 mon.a (mon.0) 3326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:01.878409+0000 mgr.gibba032.whopeo (mgr.24625) 9811 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:59:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:01.879959+0000 mgr.gibba032.whopeo (mgr.24625) 9812 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:59:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.096731+0000 mon.a (mon.0) 3320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:59:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.097611+0000 mgr.gibba032.whopeo (mgr.24625) 9813 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:59:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.099229+0000 mgr.gibba032.whopeo (mgr.24625) 9814 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:59:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.099974+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:59:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.100359+0000 mgr.gibba032.whopeo (mgr.24625) 9815 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:59:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.100395+0000 mgr.gibba032.whopeo (mgr.24625) 9816 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:59:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.101241+0000 mon.a (mon.0) 3322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.101467+0000 mgr.gibba032.whopeo (mgr.24625) 9817 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:59:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.101504+0000 mgr.gibba032.whopeo (mgr.24625) 9818 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:59:03.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.102386+0000 mon.a (mon.0) 3323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.102769+0000 mgr.gibba032.whopeo (mgr.24625) 9819 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:59:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.103507+0000 mon.a (mon.0) 3324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.103828+0000 mgr.gibba032.whopeo (mgr.24625) 9820 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:59:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.103869+0000 mgr.gibba032.whopeo (mgr.24625) 9821 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:59:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.105230+0000 mgr.gibba032.whopeo (mgr.24625) 9822 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:59:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.105541+0000 mgr.gibba032.whopeo (mgr.24625) 9823 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:59:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.106154+0000 mon.a (mon.0) 3325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.106781+0000 mgr.gibba032.whopeo (mgr.24625) 9824 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:59:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.106820+0000 mgr.gibba032.whopeo (mgr.24625) 9825 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:59:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: cephadm 2021-03-10T16:59:02.106970+0000 mgr.gibba032.whopeo (mgr.24625) 9826 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[30791]: audit 2021-03-10T16:59:02.107622+0000 mon.a (mon.0) 3326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:01.878409+0000 mgr.gibba032.whopeo (mgr.24625) 9811 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T16:59:03.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:01.879959+0000 mgr.gibba032.whopeo (mgr.24625) 9812 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T16:59:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.096731+0000 mon.a (mon.0) 3320 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T16:59:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.097611+0000 mgr.gibba032.whopeo (mgr.24625) 9813 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T16:59:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.099229+0000 mgr.gibba032.whopeo (mgr.24625) 9814 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T16:59:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.099974+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T16:59:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.100359+0000 mgr.gibba032.whopeo (mgr.24625) 9815 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T16:59:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.100395+0000 mgr.gibba032.whopeo (mgr.24625) 9816 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T16:59:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.101241+0000 mon.a (mon.0) 3322 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.101467+0000 mgr.gibba032.whopeo (mgr.24625) 9817 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T16:59:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.101504+0000 mgr.gibba032.whopeo (mgr.24625) 9818 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T16:59:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.102386+0000 mon.a (mon.0) 3323 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.102769+0000 mgr.gibba032.whopeo (mgr.24625) 9819 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T16:59:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.103507+0000 mon.a (mon.0) 3324 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.103828+0000 mgr.gibba032.whopeo (mgr.24625) 9820 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T16:59:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.103869+0000 mgr.gibba032.whopeo (mgr.24625) 9821 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T16:59:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.105230+0000 mgr.gibba032.whopeo (mgr.24625) 9822 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T16:59:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.105541+0000 mgr.gibba032.whopeo (mgr.24625) 9823 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T16:59:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.106154+0000 mon.a (mon.0) 3325 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.106781+0000 mgr.gibba032.whopeo (mgr.24625) 9824 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T16:59:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.106820+0000 mgr.gibba032.whopeo (mgr.24625) 9825 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T16:59:03.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: cephadm 2021-03-10T16:59:02.106970+0000 mgr.gibba032.whopeo (mgr.24625) 9826 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:03.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:03 gibba011 bash[31774]: audit 2021-03-10T16:59:02.107622+0000 mon.a (mon.0) 3326 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:04.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:04 gibba032 bash[24004]: cluster 2021-03-10T16:59:02.643459+0000 mgr.gibba032.whopeo (mgr.24625) 9827 : cluster [DBG] pgmap v5594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:04.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:04 gibba011 bash[30791]: cluster 2021-03-10T16:59:02.643459+0000 mgr.gibba032.whopeo (mgr.24625) 9827 : cluster [DBG] pgmap v5594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:04.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:04 gibba011 bash[31774]: cluster 2021-03-10T16:59:02.643459+0000 mgr.gibba032.whopeo (mgr.24625) 9827 : cluster [DBG] pgmap v5594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:06.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:06 gibba032 bash[24004]: cluster 2021-03-10T16:59:04.644805+0000 mgr.gibba032.whopeo (mgr.24625) 9828 : cluster [DBG] pgmap v5595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:06 gibba011 bash[30791]: cluster 2021-03-10T16:59:04.644805+0000 mgr.gibba032.whopeo (mgr.24625) 9828 : cluster [DBG] pgmap v5595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:06.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:06 gibba011 bash[31774]: cluster 2021-03-10T16:59:04.644805+0000 mgr.gibba032.whopeo (mgr.24625) 9828 : cluster [DBG] pgmap v5595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:08 gibba032 bash[24004]: cluster 2021-03-10T16:59:06.646420+0000 mgr.gibba032.whopeo (mgr.24625) 9829 : cluster [DBG] pgmap v5596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:08 gibba011 bash[30791]: cluster 2021-03-10T16:59:06.646420+0000 mgr.gibba032.whopeo (mgr.24625) 9829 : cluster [DBG] pgmap v5596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:08.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:08 gibba011 bash[31774]: cluster 2021-03-10T16:59:06.646420+0000 mgr.gibba032.whopeo (mgr.24625) 9829 : cluster [DBG] pgmap v5596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:10 gibba032 bash[24004]: cluster 2021-03-10T16:59:08.647911+0000 mgr.gibba032.whopeo (mgr.24625) 9830 : cluster [DBG] pgmap v5597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:10 gibba011 bash[30791]: cluster 2021-03-10T16:59:08.647911+0000 mgr.gibba032.whopeo (mgr.24625) 9830 : cluster [DBG] pgmap v5597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:10 gibba011 bash[31774]: cluster 2021-03-10T16:59:08.647911+0000 mgr.gibba032.whopeo (mgr.24625) 9830 : cluster [DBG] pgmap v5597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:11 gibba032 bash[24004]: cluster 2021-03-10T16:59:10.649318+0000 mgr.gibba032.whopeo (mgr.24625) 9831 : cluster [DBG] pgmap v5598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:11 gibba011 bash[30791]: cluster 2021-03-10T16:59:10.649318+0000 mgr.gibba032.whopeo (mgr.24625) 9831 : cluster [DBG] pgmap v5598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:11 gibba011 bash[31774]: cluster 2021-03-10T16:59:10.649318+0000 mgr.gibba032.whopeo (mgr.24625) 9831 : cluster [DBG] pgmap v5598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:14.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:13 gibba032 bash[24004]: cluster 2021-03-10T16:59:12.650862+0000 mgr.gibba032.whopeo (mgr.24625) 9832 : cluster [DBG] pgmap v5599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:13 gibba011 bash[30791]: cluster 2021-03-10T16:59:12.650862+0000 mgr.gibba032.whopeo (mgr.24625) 9832 : cluster [DBG] pgmap v5599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:14.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:13 gibba011 bash[31774]: cluster 2021-03-10T16:59:12.650862+0000 mgr.gibba032.whopeo (mgr.24625) 9832 : cluster [DBG] pgmap v5599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:16.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:15 gibba032 bash[24004]: cluster 2021-03-10T16:59:14.652064+0000 mgr.gibba032.whopeo (mgr.24625) 9833 : cluster [DBG] pgmap v5600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:15 gibba011 bash[30791]: cluster 2021-03-10T16:59:14.652064+0000 mgr.gibba032.whopeo (mgr.24625) 9833 : cluster [DBG] pgmap v5600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:16.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:15 gibba011 bash[31774]: cluster 2021-03-10T16:59:14.652064+0000 mgr.gibba032.whopeo (mgr.24625) 9833 : cluster [DBG] pgmap v5600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:17.781 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:59:18.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:17 gibba032 bash[24004]: cluster 2021-03-10T16:59:16.653506+0000 mgr.gibba032.whopeo (mgr.24625) 9834 : cluster [DBG] pgmap v5601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:18.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:17 gibba032 bash[24004]: cephadm 2021-03-10T16:59:17.122957+0000 mgr.gibba032.whopeo (mgr.24625) 9835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:18.012 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:59:18.012 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:59:18.012 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:59:18.012 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:59:18.013 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:59:18.013 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:59:18.013 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:59:18.013 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:59:18.014 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:59:18.014 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:59:18.014 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:59:18.014 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:59:18.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:59:18.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:59:18.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:59:18.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:59:18.015 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:59:18.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:59:18.016 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:59:18.016 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:59:18.016 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:59:18.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:17 gibba011 bash[30791]: cluster 2021-03-10T16:59:16.653506+0000 mgr.gibba032.whopeo (mgr.24625) 9834 : cluster [DBG] pgmap v5601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:18.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:17 gibba011 bash[30791]: cephadm 2021-03-10T16:59:17.122957+0000 mgr.gibba032.whopeo (mgr.24625) 9835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:18.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:17 gibba011 bash[31774]: cluster 2021-03-10T16:59:16.653506+0000 mgr.gibba032.whopeo (mgr.24625) 9834 : cluster [DBG] pgmap v5601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:18.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:17 gibba011 bash[31774]: cephadm 2021-03-10T16:59:17.122957+0000 mgr.gibba032.whopeo (mgr.24625) 9835 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:18.268 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:59:18.268 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:59:18.269 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:59:18.269 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:18.269 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:59:18.269 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:59:18.269 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:18.270 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:59:18.270 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:59:18.270 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:18.270 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:59:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:59:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:59:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:59:18.271 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:59:19.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:18 gibba032 bash[24004]: audit 2021-03-10T16:59:17.772605+0000 mgr.gibba032.whopeo (mgr.24625) 9836 : audit [DBG] from='client.50643 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:18 gibba032 bash[24004]: audit 2021-03-10T16:59:17.998551+0000 mgr.gibba032.whopeo (mgr.24625) 9837 : audit [DBG] from='client.60631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:18 gibba032 bash[24004]: audit 2021-03-10T16:59:18.269253+0000 mon.c (mon.1) 191 : audit [DBG] from='client.? 172.21.2.111:0/2833936487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:19.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[30791]: audit 2021-03-10T16:59:17.772605+0000 mgr.gibba032.whopeo (mgr.24625) 9836 : audit [DBG] from='client.50643 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[30791]: audit 2021-03-10T16:59:17.998551+0000 mgr.gibba032.whopeo (mgr.24625) 9837 : audit [DBG] from='client.60631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[30791]: audit 2021-03-10T16:59:18.269253+0000 mon.c (mon.1) 191 : audit [DBG] from='client.? 172.21.2.111:0/2833936487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[31774]: audit 2021-03-10T16:59:17.772605+0000 mgr.gibba032.whopeo (mgr.24625) 9836 : audit [DBG] from='client.50643 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[31774]: audit 2021-03-10T16:59:17.998551+0000 mgr.gibba032.whopeo (mgr.24625) 9837 : audit [DBG] from='client.60631 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:18 gibba011 bash[31774]: audit 2021-03-10T16:59:18.269253+0000 mon.c (mon.1) 191 : audit [DBG] from='client.? 172.21.2.111:0/2833936487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:20.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:19 gibba032 bash[24004]: cluster 2021-03-10T16:59:18.654882+0000 mgr.gibba032.whopeo (mgr.24625) 9838 : cluster [DBG] pgmap v5602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:20.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:19 gibba011 bash[30791]: cluster 2021-03-10T16:59:18.654882+0000 mgr.gibba032.whopeo (mgr.24625) 9838 : cluster [DBG] pgmap v5602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:20.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:19 gibba011 bash[31774]: cluster 2021-03-10T16:59:18.654882+0000 mgr.gibba032.whopeo (mgr.24625) 9838 : cluster [DBG] pgmap v5602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:21.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:21 gibba032 bash[24004]: cluster 2021-03-10T16:59:20.656483+0000 mgr.gibba032.whopeo (mgr.24625) 9839 : cluster [DBG] pgmap v5603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:21 gibba011 bash[30791]: cluster 2021-03-10T16:59:20.656483+0000 mgr.gibba032.whopeo (mgr.24625) 9839 : cluster [DBG] pgmap v5603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:21.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:21 gibba011 bash[31774]: cluster 2021-03-10T16:59:20.656483+0000 mgr.gibba032.whopeo (mgr.24625) 9839 : cluster [DBG] pgmap v5603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:24.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:23 gibba032 bash[24004]: cluster 2021-03-10T16:59:22.658017+0000 mgr.gibba032.whopeo (mgr.24625) 9840 : cluster [DBG] pgmap v5604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:24.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:23 gibba011 bash[30791]: cluster 2021-03-10T16:59:22.658017+0000 mgr.gibba032.whopeo (mgr.24625) 9840 : cluster [DBG] pgmap v5604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:24.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:23 gibba011 bash[31774]: cluster 2021-03-10T16:59:22.658017+0000 mgr.gibba032.whopeo (mgr.24625) 9840 : cluster [DBG] pgmap v5604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:26.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:25 gibba032 bash[24004]: cluster 2021-03-10T16:59:24.659207+0000 mgr.gibba032.whopeo (mgr.24625) 9841 : cluster [DBG] pgmap v5605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:26.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:25 gibba011 bash[30791]: cluster 2021-03-10T16:59:24.659207+0000 mgr.gibba032.whopeo (mgr.24625) 9841 : cluster [DBG] pgmap v5605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:26.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:25 gibba011 bash[31774]: cluster 2021-03-10T16:59:24.659207+0000 mgr.gibba032.whopeo (mgr.24625) 9841 : cluster [DBG] pgmap v5605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:28.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:27 gibba032 bash[24004]: cluster 2021-03-10T16:59:26.660726+0000 mgr.gibba032.whopeo (mgr.24625) 9842 : cluster [DBG] pgmap v5606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:28.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:27 gibba011 bash[30791]: cluster 2021-03-10T16:59:26.660726+0000 mgr.gibba032.whopeo (mgr.24625) 9842 : cluster [DBG] pgmap v5606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:28.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:27 gibba011 bash[31774]: cluster 2021-03-10T16:59:26.660726+0000 mgr.gibba032.whopeo (mgr.24625) 9842 : cluster [DBG] pgmap v5606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:30.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:29 gibba032 bash[24004]: cluster 2021-03-10T16:59:28.661995+0000 mgr.gibba032.whopeo (mgr.24625) 9843 : cluster [DBG] pgmap v5607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:30.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:29 gibba011 bash[30791]: cluster 2021-03-10T16:59:28.661995+0000 mgr.gibba032.whopeo (mgr.24625) 9843 : cluster [DBG] pgmap v5607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:30.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:29 gibba011 bash[31774]: cluster 2021-03-10T16:59:28.661995+0000 mgr.gibba032.whopeo (mgr.24625) 9843 : cluster [DBG] pgmap v5607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:31 gibba032 bash[24004]: cluster 2021-03-10T16:59:30.663302+0000 mgr.gibba032.whopeo (mgr.24625) 9844 : cluster [DBG] pgmap v5608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:31 gibba011 bash[30791]: cluster 2021-03-10T16:59:30.663302+0000 mgr.gibba032.whopeo (mgr.24625) 9844 : cluster [DBG] pgmap v5608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:31.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:31 gibba011 bash[31774]: cluster 2021-03-10T16:59:30.663302+0000 mgr.gibba032.whopeo (mgr.24625) 9844 : cluster [DBG] pgmap v5608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:32.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:32 gibba032 bash[24004]: cephadm 2021-03-10T16:59:32.138978+0000 mgr.gibba032.whopeo (mgr.24625) 9845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:32 gibba011 bash[30791]: cephadm 2021-03-10T16:59:32.138978+0000 mgr.gibba032.whopeo (mgr.24625) 9845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:32.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:32 gibba011 bash[31774]: cephadm 2021-03-10T16:59:32.138978+0000 mgr.gibba032.whopeo (mgr.24625) 9845 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:33.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:33 gibba032 bash[24004]: cluster 2021-03-10T16:59:32.664972+0000 mgr.gibba032.whopeo (mgr.24625) 9846 : cluster [DBG] pgmap v5609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:33.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:33 gibba011 bash[30791]: cluster 2021-03-10T16:59:32.664972+0000 mgr.gibba032.whopeo (mgr.24625) 9846 : cluster [DBG] pgmap v5609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:33.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:33 gibba011 bash[31774]: cluster 2021-03-10T16:59:32.664972+0000 mgr.gibba032.whopeo (mgr.24625) 9846 : cluster [DBG] pgmap v5609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:36.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:35 gibba032 bash[24004]: cluster 2021-03-10T16:59:34.666250+0000 mgr.gibba032.whopeo (mgr.24625) 9847 : cluster [DBG] pgmap v5610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:36.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:35 gibba011 bash[30791]: cluster 2021-03-10T16:59:34.666250+0000 mgr.gibba032.whopeo (mgr.24625) 9847 : cluster [DBG] pgmap v5610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:36.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:35 gibba011 bash[31774]: cluster 2021-03-10T16:59:34.666250+0000 mgr.gibba032.whopeo (mgr.24625) 9847 : cluster [DBG] pgmap v5610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:38.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:37 gibba032 bash[24004]: cluster 2021-03-10T16:59:36.668058+0000 mgr.gibba032.whopeo (mgr.24625) 9848 : cluster [DBG] pgmap v5611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:38.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:37 gibba011 bash[30791]: cluster 2021-03-10T16:59:36.668058+0000 mgr.gibba032.whopeo (mgr.24625) 9848 : cluster [DBG] pgmap v5611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:38.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:37 gibba011 bash[31774]: cluster 2021-03-10T16:59:36.668058+0000 mgr.gibba032.whopeo (mgr.24625) 9848 : cluster [DBG] pgmap v5611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:40.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:39 gibba032 bash[24004]: cluster 2021-03-10T16:59:38.669672+0000 mgr.gibba032.whopeo (mgr.24625) 9849 : cluster [DBG] pgmap v5612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:40.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:39 gibba011 bash[30791]: cluster 2021-03-10T16:59:38.669672+0000 mgr.gibba032.whopeo (mgr.24625) 9849 : cluster [DBG] pgmap v5612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:40.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:39 gibba011 bash[31774]: cluster 2021-03-10T16:59:38.669672+0000 mgr.gibba032.whopeo (mgr.24625) 9849 : cluster [DBG] pgmap v5612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:41.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:41 gibba032 bash[24004]: cluster 2021-03-10T16:59:40.671108+0000 mgr.gibba032.whopeo (mgr.24625) 9850 : cluster [DBG] pgmap v5613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:41 gibba011 bash[30791]: cluster 2021-03-10T16:59:40.671108+0000 mgr.gibba032.whopeo (mgr.24625) 9850 : cluster [DBG] pgmap v5613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:41.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:41 gibba011 bash[31774]: cluster 2021-03-10T16:59:40.671108+0000 mgr.gibba032.whopeo (mgr.24625) 9850 : cluster [DBG] pgmap v5613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:44.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:43 gibba032 bash[24004]: cluster 2021-03-10T16:59:42.672921+0000 mgr.gibba032.whopeo (mgr.24625) 9851 : cluster [DBG] pgmap v5614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:44.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:43 gibba011 bash[30791]: cluster 2021-03-10T16:59:42.672921+0000 mgr.gibba032.whopeo (mgr.24625) 9851 : cluster [DBG] pgmap v5614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:44.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:43 gibba011 bash[31774]: cluster 2021-03-10T16:59:42.672921+0000 mgr.gibba032.whopeo (mgr.24625) 9851 : cluster [DBG] pgmap v5614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:46.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:45 gibba032 bash[24004]: cluster 2021-03-10T16:59:44.674351+0000 mgr.gibba032.whopeo (mgr.24625) 9852 : cluster [DBG] pgmap v5615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:46.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:45 gibba011 bash[30791]: cluster 2021-03-10T16:59:44.674351+0000 mgr.gibba032.whopeo (mgr.24625) 9852 : cluster [DBG] pgmap v5615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:46.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:45 gibba011 bash[31774]: cluster 2021-03-10T16:59:44.674351+0000 mgr.gibba032.whopeo (mgr.24625) 9852 : cluster [DBG] pgmap v5615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:48.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:47 gibba032 bash[24004]: cluster 2021-03-10T16:59:46.676194+0000 mgr.gibba032.whopeo (mgr.24625) 9853 : cluster [DBG] pgmap v5616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:47 gibba032 bash[24004]: cephadm 2021-03-10T16:59:47.154994+0000 mgr.gibba032.whopeo (mgr.24625) 9854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:47 gibba011 bash[30791]: cluster 2021-03-10T16:59:46.676194+0000 mgr.gibba032.whopeo (mgr.24625) 9853 : cluster [DBG] pgmap v5616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:47 gibba011 bash[30791]: cephadm 2021-03-10T16:59:47.154994+0000 mgr.gibba032.whopeo (mgr.24625) 9854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:48.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:47 gibba011 bash[31774]: cluster 2021-03-10T16:59:46.676194+0000 mgr.gibba032.whopeo (mgr.24625) 9853 : cluster [DBG] pgmap v5616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:47 gibba011 bash[31774]: cephadm 2021-03-10T16:59:47.154994+0000 mgr.gibba032.whopeo (mgr.24625) 9854 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T16:59:48.516 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T16:59:48.742 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T16:59:48.743 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T16:59:48.743 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T16:59:48.743 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T16:59:48.743 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T16:59:48.744 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T16:59:48.744 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T16:59:48.744 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T16:59:48.744 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T16:59:48.744 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T16:59:48.745 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T16:59:48.745 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T16:59:48.745 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T16:59:48.745 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T16:59:48.745 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T16:59:48.746 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T16:59:48.746 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T16:59:48.746 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T16:59:48.746 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T16:59:48.747 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T16:59:48.747 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T16:59:49.003 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T16:59:49.004 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T16:59:49.004 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:59:49.004 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:49.004 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T16:59:49.005 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T16:59:49.005 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:49.005 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T16:59:49.005 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T16:59:49.006 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T16:59:49.006 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T16:59:49.006 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T16:59:49.006 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T16:59:49.006 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T16:59:49.007 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T16:59:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[30791]: audit 2021-03-10T16:59:48.506246+0000 mgr.gibba032.whopeo (mgr.24625) 9855 : audit [DBG] from='client.50661 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[30791]: cluster 2021-03-10T16:59:48.677540+0000 mgr.gibba032.whopeo (mgr.24625) 9856 : cluster [DBG] pgmap v5617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[30791]: audit 2021-03-10T16:59:48.729137+0000 mgr.gibba032.whopeo (mgr.24625) 9857 : audit [DBG] from='client.60649 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[30791]: audit 2021-03-10T16:59:49.004655+0000 mon.c (mon.1) 192 : audit [DBG] from='client.? 172.21.2.111:0/1180128324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:50.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[31774]: audit 2021-03-10T16:59:48.506246+0000 mgr.gibba032.whopeo (mgr.24625) 9855 : audit [DBG] from='client.50661 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[31774]: cluster 2021-03-10T16:59:48.677540+0000 mgr.gibba032.whopeo (mgr.24625) 9856 : cluster [DBG] pgmap v5617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[31774]: audit 2021-03-10T16:59:48.729137+0000 mgr.gibba032.whopeo (mgr.24625) 9857 : audit [DBG] from='client.60649 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:49 gibba011 bash[31774]: audit 2021-03-10T16:59:49.004655+0000 mon.c (mon.1) 192 : audit [DBG] from='client.? 172.21.2.111:0/1180128324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:49 gibba032 bash[24004]: audit 2021-03-10T16:59:48.506246+0000 mgr.gibba032.whopeo (mgr.24625) 9855 : audit [DBG] from='client.50661 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:49 gibba032 bash[24004]: cluster 2021-03-10T16:59:48.677540+0000 mgr.gibba032.whopeo (mgr.24625) 9856 : cluster [DBG] pgmap v5617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:49 gibba032 bash[24004]: audit 2021-03-10T16:59:48.729137+0000 mgr.gibba032.whopeo (mgr.24625) 9857 : audit [DBG] from='client.60649 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T16:59:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:49 gibba032 bash[24004]: audit 2021-03-10T16:59:49.004655+0000 mon.c (mon.1) 192 : audit [DBG] from='client.? 172.21.2.111:0/1180128324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T16:59:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:51 gibba032 bash[24004]: cluster 2021-03-10T16:59:50.678121+0000 mgr.gibba032.whopeo (mgr.24625) 9858 : cluster [DBG] pgmap v5618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:51 gibba011 bash[30791]: cluster 2021-03-10T16:59:50.678121+0000 mgr.gibba032.whopeo (mgr.24625) 9858 : cluster [DBG] pgmap v5618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:51.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:51 gibba011 bash[31774]: cluster 2021-03-10T16:59:50.678121+0000 mgr.gibba032.whopeo (mgr.24625) 9858 : cluster [DBG] pgmap v5618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:54.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:53 gibba032 bash[24004]: cluster 2021-03-10T16:59:52.679653+0000 mgr.gibba032.whopeo (mgr.24625) 9859 : cluster [DBG] pgmap v5619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:54.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:53 gibba011 bash[30791]: cluster 2021-03-10T16:59:52.679653+0000 mgr.gibba032.whopeo (mgr.24625) 9859 : cluster [DBG] pgmap v5619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:54.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:53 gibba011 bash[31774]: cluster 2021-03-10T16:59:52.679653+0000 mgr.gibba032.whopeo (mgr.24625) 9859 : cluster [DBG] pgmap v5619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:56.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:55 gibba032 bash[24004]: cluster 2021-03-10T16:59:54.680952+0000 mgr.gibba032.whopeo (mgr.24625) 9860 : cluster [DBG] pgmap v5620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:56.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:55 gibba011 bash[30791]: cluster 2021-03-10T16:59:54.680952+0000 mgr.gibba032.whopeo (mgr.24625) 9860 : cluster [DBG] pgmap v5620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:56.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:55 gibba011 bash[31774]: cluster 2021-03-10T16:59:54.680952+0000 mgr.gibba032.whopeo (mgr.24625) 9860 : cluster [DBG] pgmap v5620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: cluster 2021-03-10T16:59:56.682543+0000 mgr.gibba032.whopeo (mgr.24625) 9861 : cluster [DBG] pgmap v5621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.269216+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:59:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.269891+0000 mon.a (mon.0) 3328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:59:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.411712+0000 mon.a (mon.0) 3329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:59:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.412428+0000 mon.a (mon.0) 3330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:59:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.413102+0000 mon.a (mon.0) 3331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:59:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[30791]: audit 2021-03-10T16:59:57.413805+0000 mon.a (mon.0) 3332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:59:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: cluster 2021-03-10T16:59:56.682543+0000 mgr.gibba032.whopeo (mgr.24625) 9861 : cluster [DBG] pgmap v5621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.269216+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:59:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.269891+0000 mon.a (mon.0) 3328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:59:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.411712+0000 mon.a (mon.0) 3329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:59:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.412428+0000 mon.a (mon.0) 3330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:59:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.413102+0000 mon.a (mon.0) 3331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:59:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:57 gibba011 bash[31774]: audit 2021-03-10T16:59:57.413805+0000 mon.a (mon.0) 3332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T16:59:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: cluster 2021-03-10T16:59:56.682543+0000 mgr.gibba032.whopeo (mgr.24625) 9861 : cluster [DBG] pgmap v5621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T16:59:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.269216+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T16:59:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.269891+0000 mon.a (mon.0) 3328 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T16:59:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.411712+0000 mon.a (mon.0) 3329 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T16:59:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.412428+0000 mon.a (mon.0) 3330 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T16:59:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.413102+0000 mon.a (mon.0) 3331 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T16:59:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:57 gibba032 bash[24004]: audit 2021-03-10T16:59:57.413805+0000 mon.a (mon.0) 3332 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:00:00.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 16:59:59 gibba011 bash[30791]: cluster 2021-03-10T16:59:58.684134+0000 mgr.gibba032.whopeo (mgr.24625) 9862 : cluster [DBG] pgmap v5622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:00.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:00 gibba011 bash[30791]: debug 2021-03-10T16:59:59.998+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:00:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 16:59:59 gibba011 bash[31774]: cluster 2021-03-10T16:59:58.684134+0000 mgr.gibba032.whopeo (mgr.24625) 9862 : cluster [DBG] pgmap v5622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 16:59:59 gibba032 bash[24004]: cluster 2021-03-10T16:59:58.684134+0000 mgr.gibba032.whopeo (mgr.24625) 9862 : cluster [DBG] pgmap v5622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:01.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:00 gibba011 bash[30791]: cluster 2021-03-10T17:00:00.000213+0000 mon.a (mon.0) 3333 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:00:01.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:00 gibba011 bash[31774]: cluster 2021-03-10T17:00:00.000213+0000 mon.a (mon.0) 3333 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:00:01.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:00 gibba032 bash[24004]: cluster 2021-03-10T17:00:00.000213+0000 mon.a (mon.0) 3333 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:00:02.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:01 gibba011 bash[30791]: cluster 2021-03-10T17:00:00.684627+0000 mgr.gibba032.whopeo (mgr.24625) 9863 : cluster [DBG] pgmap v5623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:01 gibba011 bash[31774]: cluster 2021-03-10T17:00:00.684627+0000 mgr.gibba032.whopeo (mgr.24625) 9863 : cluster [DBG] pgmap v5623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:02.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:01 gibba032 bash[24004]: cluster 2021-03-10T17:00:00.684627+0000 mgr.gibba032.whopeo (mgr.24625) 9863 : cluster [DBG] pgmap v5623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:03.402 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[19417]: level=info ts=2021-03-10T17:00:03.090Z caller=compact.go:507 component=tsdb msg="write block" mint=1615384800647 maxt=1615392000000 ulid=01F0EH9GK90D6NSTC11986WNQF duration=41.217293ms 2021-03-10T17:00:03.402 INFO:journalctl@ceph.prometheus.a.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[19417]: level=info ts=2021-03-10T17:00:03.092Z caller=head.go:847 component=tsdb msg="Head GC completed" duration=1.660869ms 2021-03-10T17:00:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.171692+0000 mgr.gibba032.whopeo (mgr.24625) 9864 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:00:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.173314+0000 mgr.gibba032.whopeo (mgr.24625) 9865 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:00:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.396579+0000 mon.a (mon.0) 3334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:00:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.397431+0000 mgr.gibba032.whopeo (mgr.24625) 9866 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:00:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.399250+0000 mgr.gibba032.whopeo (mgr.24625) 9867 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:00:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.399820+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:00:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.401081+0000 mgr.gibba032.whopeo (mgr.24625) 9868 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:00:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.401117+0000 mgr.gibba032.whopeo (mgr.24625) 9869 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:00:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.401572+0000 mon.a (mon.0) 3336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.402993+0000 mgr.gibba032.whopeo (mgr.24625) 9870 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:00:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.403032+0000 mgr.gibba032.whopeo (mgr.24625) 9871 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:00:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.403489+0000 mon.a (mon.0) 3337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.404487+0000 mgr.gibba032.whopeo (mgr.24625) 9872 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:00:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.405142+0000 mon.a (mon.0) 3338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.406027+0000 mgr.gibba032.whopeo (mgr.24625) 9873 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:00:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.406064+0000 mgr.gibba032.whopeo (mgr.24625) 9874 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:00:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.407563+0000 mgr.gibba032.whopeo (mgr.24625) 9875 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:00:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.407879+0000 mgr.gibba032.whopeo (mgr.24625) 9876 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:00:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.408256+0000 mon.a (mon.0) 3339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.409198+0000 mgr.gibba032.whopeo (mgr.24625) 9877 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:00:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.409234+0000 mgr.gibba032.whopeo (mgr.24625) 9878 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:00:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: cephadm 2021-03-10T17:00:02.409338+0000 mgr.gibba032.whopeo (mgr.24625) 9879 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:03.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:03 gibba032 bash[24004]: audit 2021-03-10T17:00:02.409849+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.171692+0000 mgr.gibba032.whopeo (mgr.24625) 9864 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:00:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.173314+0000 mgr.gibba032.whopeo (mgr.24625) 9865 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:00:03.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.396579+0000 mon.a (mon.0) 3334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:00:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.397431+0000 mgr.gibba032.whopeo (mgr.24625) 9866 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:00:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.399250+0000 mgr.gibba032.whopeo (mgr.24625) 9867 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:00:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.399820+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:00:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.401081+0000 mgr.gibba032.whopeo (mgr.24625) 9868 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:00:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.401117+0000 mgr.gibba032.whopeo (mgr.24625) 9869 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:00:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.401572+0000 mon.a (mon.0) 3336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.402993+0000 mgr.gibba032.whopeo (mgr.24625) 9870 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:00:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.403032+0000 mgr.gibba032.whopeo (mgr.24625) 9871 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:00:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.403489+0000 mon.a (mon.0) 3337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.404487+0000 mgr.gibba032.whopeo (mgr.24625) 9872 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:00:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.405142+0000 mon.a (mon.0) 3338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.406027+0000 mgr.gibba032.whopeo (mgr.24625) 9873 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:00:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.406064+0000 mgr.gibba032.whopeo (mgr.24625) 9874 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:00:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.407563+0000 mgr.gibba032.whopeo (mgr.24625) 9875 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:00:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.407879+0000 mgr.gibba032.whopeo (mgr.24625) 9876 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:00:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.408256+0000 mon.a (mon.0) 3339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.409198+0000 mgr.gibba032.whopeo (mgr.24625) 9877 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:00:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.409234+0000 mgr.gibba032.whopeo (mgr.24625) 9878 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:00:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: cephadm 2021-03-10T17:00:02.409338+0000 mgr.gibba032.whopeo (mgr.24625) 9879 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[30791]: audit 2021-03-10T17:00:02.409849+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.171692+0000 mgr.gibba032.whopeo (mgr.24625) 9864 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:00:03.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.173314+0000 mgr.gibba032.whopeo (mgr.24625) 9865 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:00:03.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.396579+0000 mon.a (mon.0) 3334 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:00:03.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.397431+0000 mgr.gibba032.whopeo (mgr.24625) 9866 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:00:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.399250+0000 mgr.gibba032.whopeo (mgr.24625) 9867 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:00:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.399820+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:00:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.401081+0000 mgr.gibba032.whopeo (mgr.24625) 9868 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:00:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.401117+0000 mgr.gibba032.whopeo (mgr.24625) 9869 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:00:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.401572+0000 mon.a (mon.0) 3336 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.402993+0000 mgr.gibba032.whopeo (mgr.24625) 9870 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:00:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.403032+0000 mgr.gibba032.whopeo (mgr.24625) 9871 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:00:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.403489+0000 mon.a (mon.0) 3337 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.404487+0000 mgr.gibba032.whopeo (mgr.24625) 9872 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:00:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.405142+0000 mon.a (mon.0) 3338 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.406027+0000 mgr.gibba032.whopeo (mgr.24625) 9873 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:00:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.406064+0000 mgr.gibba032.whopeo (mgr.24625) 9874 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:00:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.407563+0000 mgr.gibba032.whopeo (mgr.24625) 9875 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:00:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.407879+0000 mgr.gibba032.whopeo (mgr.24625) 9876 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:00:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.408256+0000 mon.a (mon.0) 3339 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.409198+0000 mgr.gibba032.whopeo (mgr.24625) 9877 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:00:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.409234+0000 mgr.gibba032.whopeo (mgr.24625) 9878 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:00:03.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: cephadm 2021-03-10T17:00:02.409338+0000 mgr.gibba032.whopeo (mgr.24625) 9879 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:03.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:03 gibba011 bash[31774]: audit 2021-03-10T17:00:02.409849+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:04.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:04 gibba032 bash[24004]: cluster 2021-03-10T17:00:02.685967+0000 mgr.gibba032.whopeo (mgr.24625) 9880 : cluster [DBG] pgmap v5624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:04.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:04 gibba011 bash[30791]: cluster 2021-03-10T17:00:02.685967+0000 mgr.gibba032.whopeo (mgr.24625) 9880 : cluster [DBG] pgmap v5624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:04 gibba011 bash[31774]: cluster 2021-03-10T17:00:02.685967+0000 mgr.gibba032.whopeo (mgr.24625) 9880 : cluster [DBG] pgmap v5624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:06.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:06 gibba032 bash[24004]: cluster 2021-03-10T17:00:04.687249+0000 mgr.gibba032.whopeo (mgr.24625) 9881 : cluster [DBG] pgmap v5625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:06.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:06 gibba011 bash[30791]: cluster 2021-03-10T17:00:04.687249+0000 mgr.gibba032.whopeo (mgr.24625) 9881 : cluster [DBG] pgmap v5625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:06.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:06 gibba011 bash[31774]: cluster 2021-03-10T17:00:04.687249+0000 mgr.gibba032.whopeo (mgr.24625) 9881 : cluster [DBG] pgmap v5625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:08.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:08 gibba032 bash[24004]: cluster 2021-03-10T17:00:06.688905+0000 mgr.gibba032.whopeo (mgr.24625) 9882 : cluster [DBG] pgmap v5626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:08.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:08 gibba011 bash[30791]: cluster 2021-03-10T17:00:06.688905+0000 mgr.gibba032.whopeo (mgr.24625) 9882 : cluster [DBG] pgmap v5626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:08.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:08 gibba011 bash[31774]: cluster 2021-03-10T17:00:06.688905+0000 mgr.gibba032.whopeo (mgr.24625) 9882 : cluster [DBG] pgmap v5626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:10.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:10 gibba032 bash[24004]: cluster 2021-03-10T17:00:08.690508+0000 mgr.gibba032.whopeo (mgr.24625) 9883 : cluster [DBG] pgmap v5627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:10.827 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:10 gibba011 bash[30791]: cluster 2021-03-10T17:00:08.690508+0000 mgr.gibba032.whopeo (mgr.24625) 9883 : cluster [DBG] pgmap v5627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:10.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:10 gibba011 bash[31774]: cluster 2021-03-10T17:00:08.690508+0000 mgr.gibba032.whopeo (mgr.24625) 9883 : cluster [DBG] pgmap v5627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:11 gibba032 bash[24004]: cluster 2021-03-10T17:00:10.691158+0000 mgr.gibba032.whopeo (mgr.24625) 9884 : cluster [DBG] pgmap v5628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:11.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:11 gibba011 bash[30791]: cluster 2021-03-10T17:00:10.691158+0000 mgr.gibba032.whopeo (mgr.24625) 9884 : cluster [DBG] pgmap v5628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:11.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:11 gibba011 bash[31774]: cluster 2021-03-10T17:00:10.691158+0000 mgr.gibba032.whopeo (mgr.24625) 9884 : cluster [DBG] pgmap v5628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:13 gibba011 bash[30791]: cluster 2021-03-10T17:00:12.692722+0000 mgr.gibba032.whopeo (mgr.24625) 9885 : cluster [DBG] pgmap v5629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:14.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:13 gibba011 bash[31774]: cluster 2021-03-10T17:00:12.692722+0000 mgr.gibba032.whopeo (mgr.24625) 9885 : cluster [DBG] pgmap v5629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:13 gibba032 bash[24004]: cluster 2021-03-10T17:00:12.692722+0000 mgr.gibba032.whopeo (mgr.24625) 9885 : cluster [DBG] pgmap v5629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:15 gibba011 bash[30791]: cluster 2021-03-10T17:00:14.694229+0000 mgr.gibba032.whopeo (mgr.24625) 9886 : cluster [DBG] pgmap v5630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:16.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:15 gibba011 bash[31774]: cluster 2021-03-10T17:00:14.694229+0000 mgr.gibba032.whopeo (mgr.24625) 9886 : cluster [DBG] pgmap v5630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:15 gibba032 bash[24004]: cluster 2021-03-10T17:00:14.694229+0000 mgr.gibba032.whopeo (mgr.24625) 9886 : cluster [DBG] pgmap v5630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:18.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:17 gibba011 bash[30791]: cluster 2021-03-10T17:00:16.695979+0000 mgr.gibba032.whopeo (mgr.24625) 9887 : cluster [DBG] pgmap v5631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:18.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:17 gibba011 bash[30791]: cephadm 2021-03-10T17:00:17.425114+0000 mgr.gibba032.whopeo (mgr.24625) 9888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:18.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:17 gibba011 bash[31774]: cluster 2021-03-10T17:00:16.695979+0000 mgr.gibba032.whopeo (mgr.24625) 9887 : cluster [DBG] pgmap v5631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:18.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:17 gibba011 bash[31774]: cephadm 2021-03-10T17:00:17.425114+0000 mgr.gibba032.whopeo (mgr.24625) 9888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:17 gibba032 bash[24004]: cluster 2021-03-10T17:00:16.695979+0000 mgr.gibba032.whopeo (mgr.24625) 9887 : cluster [DBG] pgmap v5631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:17 gibba032 bash[24004]: cephadm 2021-03-10T17:00:17.425114+0000 mgr.gibba032.whopeo (mgr.24625) 9888 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:19.247 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:00:19.475 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:00:19.475 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:00:19.476 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:00:19.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:00:19.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:00:19.477 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:00:19.477 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:00:19.477 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:00:19.478 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:00:19.478 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:00:19.478 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:00:19.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:00:19.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:00:19.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:00:19.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:00:19.480 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:00:19.480 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:00:19.480 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:00:19.480 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:00:19.481 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:00:19.481 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:00:19.749 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:00:19.750 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:00:19.750 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:00:19.750 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:19.750 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:00:19.751 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:00:19.751 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:19.751 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:00:19.751 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:00:19.752 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:19.752 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:00:19.752 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:00:19.752 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:00:19.752 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:00:19.753 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:00:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[30791]: cluster 2021-03-10T17:00:18.697374+0000 mgr.gibba032.whopeo (mgr.24625) 9889 : cluster [DBG] pgmap v5632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[30791]: audit 2021-03-10T17:00:19.237029+0000 mgr.gibba032.whopeo (mgr.24625) 9890 : audit [DBG] from='client.50679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[30791]: audit 2021-03-10T17:00:19.750419+0000 mon.c (mon.1) 193 : audit [DBG] from='client.? 172.21.2.111:0/1453654855' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[31774]: cluster 2021-03-10T17:00:18.697374+0000 mgr.gibba032.whopeo (mgr.24625) 9889 : cluster [DBG] pgmap v5632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[31774]: audit 2021-03-10T17:00:19.237029+0000 mgr.gibba032.whopeo (mgr.24625) 9890 : audit [DBG] from='client.50679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:20.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:19 gibba011 bash[31774]: audit 2021-03-10T17:00:19.750419+0000 mon.c (mon.1) 193 : audit [DBG] from='client.? 172.21.2.111:0/1453654855' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:20.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:19 gibba032 bash[24004]: cluster 2021-03-10T17:00:18.697374+0000 mgr.gibba032.whopeo (mgr.24625) 9889 : cluster [DBG] pgmap v5632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:19 gibba032 bash[24004]: audit 2021-03-10T17:00:19.237029+0000 mgr.gibba032.whopeo (mgr.24625) 9890 : audit [DBG] from='client.50679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:19 gibba032 bash[24004]: audit 2021-03-10T17:00:19.750419+0000 mon.c (mon.1) 193 : audit [DBG] from='client.? 172.21.2.111:0/1453654855' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:20 gibba011 bash[30791]: audit 2021-03-10T17:00:19.462188+0000 mgr.gibba032.whopeo (mgr.24625) 9891 : audit [DBG] from='client.50685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:20 gibba011 bash[31774]: audit 2021-03-10T17:00:19.462188+0000 mgr.gibba032.whopeo (mgr.24625) 9891 : audit [DBG] from='client.50685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:21.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:20 gibba032 bash[24004]: audit 2021-03-10T17:00:19.462188+0000 mgr.gibba032.whopeo (mgr.24625) 9891 : audit [DBG] from='client.50685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:21 gibba011 bash[30791]: cluster 2021-03-10T17:00:20.698701+0000 mgr.gibba032.whopeo (mgr.24625) 9892 : cluster [DBG] pgmap v5633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:21 gibba011 bash[31774]: cluster 2021-03-10T17:00:20.698701+0000 mgr.gibba032.whopeo (mgr.24625) 9892 : cluster [DBG] pgmap v5633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:22.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:21 gibba032 bash[24004]: cluster 2021-03-10T17:00:20.698701+0000 mgr.gibba032.whopeo (mgr.24625) 9892 : cluster [DBG] pgmap v5633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:24.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:23 gibba011 bash[30791]: cluster 2021-03-10T17:00:22.700414+0000 mgr.gibba032.whopeo (mgr.24625) 9893 : cluster [DBG] pgmap v5634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:24.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:23 gibba011 bash[31774]: cluster 2021-03-10T17:00:22.700414+0000 mgr.gibba032.whopeo (mgr.24625) 9893 : cluster [DBG] pgmap v5634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:23 gibba032 bash[24004]: cluster 2021-03-10T17:00:22.700414+0000 mgr.gibba032.whopeo (mgr.24625) 9893 : cluster [DBG] pgmap v5634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:26.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:25 gibba011 bash[30791]: cluster 2021-03-10T17:00:24.701886+0000 mgr.gibba032.whopeo (mgr.24625) 9894 : cluster [DBG] pgmap v5635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:26.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:25 gibba011 bash[31774]: cluster 2021-03-10T17:00:24.701886+0000 mgr.gibba032.whopeo (mgr.24625) 9894 : cluster [DBG] pgmap v5635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:25 gibba032 bash[24004]: cluster 2021-03-10T17:00:24.701886+0000 mgr.gibba032.whopeo (mgr.24625) 9894 : cluster [DBG] pgmap v5635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:28.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:27 gibba011 bash[30791]: cluster 2021-03-10T17:00:26.703487+0000 mgr.gibba032.whopeo (mgr.24625) 9895 : cluster [DBG] pgmap v5636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:28.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:27 gibba011 bash[31774]: cluster 2021-03-10T17:00:26.703487+0000 mgr.gibba032.whopeo (mgr.24625) 9895 : cluster [DBG] pgmap v5636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:28.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:27 gibba032 bash[24004]: cluster 2021-03-10T17:00:26.703487+0000 mgr.gibba032.whopeo (mgr.24625) 9895 : cluster [DBG] pgmap v5636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:30.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:29 gibba011 bash[30791]: cluster 2021-03-10T17:00:28.705134+0000 mgr.gibba032.whopeo (mgr.24625) 9896 : cluster [DBG] pgmap v5637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:29 gibba011 bash[31774]: cluster 2021-03-10T17:00:28.705134+0000 mgr.gibba032.whopeo (mgr.24625) 9896 : cluster [DBG] pgmap v5637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:30.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:29 gibba032 bash[24004]: cluster 2021-03-10T17:00:28.705134+0000 mgr.gibba032.whopeo (mgr.24625) 9896 : cluster [DBG] pgmap v5637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:31.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:31 gibba032 bash[24004]: cluster 2021-03-10T17:00:30.705686+0000 mgr.gibba032.whopeo (mgr.24625) 9897 : cluster [DBG] pgmap v5638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:31 gibba011 bash[30791]: cluster 2021-03-10T17:00:30.705686+0000 mgr.gibba032.whopeo (mgr.24625) 9897 : cluster [DBG] pgmap v5638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:31 gibba011 bash[31774]: cluster 2021-03-10T17:00:30.705686+0000 mgr.gibba032.whopeo (mgr.24625) 9897 : cluster [DBG] pgmap v5638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:32.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:32 gibba032 bash[24004]: cephadm 2021-03-10T17:00:32.441041+0000 mgr.gibba032.whopeo (mgr.24625) 9898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:32.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:32 gibba011 bash[30791]: cephadm 2021-03-10T17:00:32.441041+0000 mgr.gibba032.whopeo (mgr.24625) 9898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:32.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:32 gibba011 bash[31774]: cephadm 2021-03-10T17:00:32.441041+0000 mgr.gibba032.whopeo (mgr.24625) 9898 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:33.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:33 gibba032 bash[24004]: cluster 2021-03-10T17:00:32.707306+0000 mgr.gibba032.whopeo (mgr.24625) 9899 : cluster [DBG] pgmap v5639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:33.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:33 gibba011 bash[30791]: cluster 2021-03-10T17:00:32.707306+0000 mgr.gibba032.whopeo (mgr.24625) 9899 : cluster [DBG] pgmap v5639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:33.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:33 gibba011 bash[31774]: cluster 2021-03-10T17:00:32.707306+0000 mgr.gibba032.whopeo (mgr.24625) 9899 : cluster [DBG] pgmap v5639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:36.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:35 gibba011 bash[30791]: cluster 2021-03-10T17:00:34.708826+0000 mgr.gibba032.whopeo (mgr.24625) 9900 : cluster [DBG] pgmap v5640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:36.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:35 gibba011 bash[31774]: cluster 2021-03-10T17:00:34.708826+0000 mgr.gibba032.whopeo (mgr.24625) 9900 : cluster [DBG] pgmap v5640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:36.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:35 gibba032 bash[24004]: cluster 2021-03-10T17:00:34.708826+0000 mgr.gibba032.whopeo (mgr.24625) 9900 : cluster [DBG] pgmap v5640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:38.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:37 gibba011 bash[30791]: cluster 2021-03-10T17:00:36.710554+0000 mgr.gibba032.whopeo (mgr.24625) 9901 : cluster [DBG] pgmap v5641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:38.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:37 gibba011 bash[31774]: cluster 2021-03-10T17:00:36.710554+0000 mgr.gibba032.whopeo (mgr.24625) 9901 : cluster [DBG] pgmap v5641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:38.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:37 gibba032 bash[24004]: cluster 2021-03-10T17:00:36.710554+0000 mgr.gibba032.whopeo (mgr.24625) 9901 : cluster [DBG] pgmap v5641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:40.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:39 gibba011 bash[30791]: cluster 2021-03-10T17:00:38.712130+0000 mgr.gibba032.whopeo (mgr.24625) 9902 : cluster [DBG] pgmap v5642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:40.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:39 gibba011 bash[31774]: cluster 2021-03-10T17:00:38.712130+0000 mgr.gibba032.whopeo (mgr.24625) 9902 : cluster [DBG] pgmap v5642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:39 gibba032 bash[24004]: cluster 2021-03-10T17:00:38.712130+0000 mgr.gibba032.whopeo (mgr.24625) 9902 : cluster [DBG] pgmap v5642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:41.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:41 gibba032 bash[24004]: cluster 2021-03-10T17:00:40.713466+0000 mgr.gibba032.whopeo (mgr.24625) 9903 : cluster [DBG] pgmap v5643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:41 gibba011 bash[30791]: cluster 2021-03-10T17:00:40.713466+0000 mgr.gibba032.whopeo (mgr.24625) 9903 : cluster [DBG] pgmap v5643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:41.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:41 gibba011 bash[31774]: cluster 2021-03-10T17:00:40.713466+0000 mgr.gibba032.whopeo (mgr.24625) 9903 : cluster [DBG] pgmap v5643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:44.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:43 gibba011 bash[30791]: cluster 2021-03-10T17:00:42.714939+0000 mgr.gibba032.whopeo (mgr.24625) 9904 : cluster [DBG] pgmap v5644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:44.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:43 gibba011 bash[31774]: cluster 2021-03-10T17:00:42.714939+0000 mgr.gibba032.whopeo (mgr.24625) 9904 : cluster [DBG] pgmap v5644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:43 gibba032 bash[24004]: cluster 2021-03-10T17:00:42.714939+0000 mgr.gibba032.whopeo (mgr.24625) 9904 : cluster [DBG] pgmap v5644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:46.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:45 gibba011 bash[30791]: cluster 2021-03-10T17:00:44.716493+0000 mgr.gibba032.whopeo (mgr.24625) 9905 : cluster [DBG] pgmap v5645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:46.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:45 gibba011 bash[31774]: cluster 2021-03-10T17:00:44.716493+0000 mgr.gibba032.whopeo (mgr.24625) 9905 : cluster [DBG] pgmap v5645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:46.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:45 gibba032 bash[24004]: cluster 2021-03-10T17:00:44.716493+0000 mgr.gibba032.whopeo (mgr.24625) 9905 : cluster [DBG] pgmap v5645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:48.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:47 gibba011 bash[30791]: cluster 2021-03-10T17:00:46.718091+0000 mgr.gibba032.whopeo (mgr.24625) 9906 : cluster [DBG] pgmap v5646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:48.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:47 gibba011 bash[31774]: cluster 2021-03-10T17:00:46.718091+0000 mgr.gibba032.whopeo (mgr.24625) 9906 : cluster [DBG] pgmap v5646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:47 gibba032 bash[24004]: cluster 2021-03-10T17:00:46.718091+0000 mgr.gibba032.whopeo (mgr.24625) 9906 : cluster [DBG] pgmap v5646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:49.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:48 gibba011 bash[30791]: cephadm 2021-03-10T17:00:47.457133+0000 mgr.gibba032.whopeo (mgr.24625) 9907 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:48 gibba011 bash[31774]: cephadm 2021-03-10T17:00:47.457133+0000 mgr.gibba032.whopeo (mgr.24625) 9907 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:49.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:48 gibba032 bash[24004]: cephadm 2021-03-10T17:00:47.457133+0000 mgr.gibba032.whopeo (mgr.24625) 9907 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:00:49.994 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:00:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:49 gibba011 bash[30791]: cluster 2021-03-10T17:00:48.719706+0000 mgr.gibba032.whopeo (mgr.24625) 9908 : cluster [DBG] pgmap v5647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:50.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:49 gibba011 bash[31774]: cluster 2021-03-10T17:00:48.719706+0000 mgr.gibba032.whopeo (mgr.24625) 9908 : cluster [DBG] pgmap v5647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:50.220 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:00:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:00:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:00:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:00:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:00:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:00:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:00:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:00:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:00:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:00:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:00:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:00:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:00:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:00:50.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:00:50.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:00:50.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:00:50.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:00:50.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:00:50.225 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:00:50.225 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:00:50.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:49 gibba032 bash[24004]: cluster 2021-03-10T17:00:48.719706+0000 mgr.gibba032.whopeo (mgr.24625) 9908 : cluster [DBG] pgmap v5647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:50.472 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:00:50.472 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:00:50.472 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:00:50.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:50.473 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:00:50.473 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:00:50.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:50.473 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:00:50.474 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:00:50.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:00:50.474 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:00:50.474 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:00:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:00:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:00:50.475 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:00:51.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[30791]: audit 2021-03-10T17:00:49.983795+0000 mgr.gibba032.whopeo (mgr.24625) 9909 : audit [DBG] from='client.50697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[30791]: audit 2021-03-10T17:00:50.208158+0000 mgr.gibba032.whopeo (mgr.24625) 9910 : audit [DBG] from='client.60685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[30791]: audit 2021-03-10T17:00:50.472721+0000 mon.a (mon.0) 3341 : audit [DBG] from='client.? 172.21.2.111:0/796314082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[31774]: audit 2021-03-10T17:00:49.983795+0000 mgr.gibba032.whopeo (mgr.24625) 9909 : audit [DBG] from='client.50697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[31774]: audit 2021-03-10T17:00:50.208158+0000 mgr.gibba032.whopeo (mgr.24625) 9910 : audit [DBG] from='client.60685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:50 gibba011 bash[31774]: audit 2021-03-10T17:00:50.472721+0000 mon.a (mon.0) 3341 : audit [DBG] from='client.? 172.21.2.111:0/796314082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:51.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:50 gibba032 bash[24004]: audit 2021-03-10T17:00:49.983795+0000 mgr.gibba032.whopeo (mgr.24625) 9909 : audit [DBG] from='client.50697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:50 gibba032 bash[24004]: audit 2021-03-10T17:00:50.208158+0000 mgr.gibba032.whopeo (mgr.24625) 9910 : audit [DBG] from='client.60685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:00:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:50 gibba032 bash[24004]: audit 2021-03-10T17:00:50.472721+0000 mon.a (mon.0) 3341 : audit [DBG] from='client.? 172.21.2.111:0/796314082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:00:52.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:51 gibba011 bash[30791]: cluster 2021-03-10T17:00:50.721079+0000 mgr.gibba032.whopeo (mgr.24625) 9911 : cluster [DBG] pgmap v5648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:52.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:51 gibba011 bash[31774]: cluster 2021-03-10T17:00:50.721079+0000 mgr.gibba032.whopeo (mgr.24625) 9911 : cluster [DBG] pgmap v5648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:51 gibba032 bash[24004]: cluster 2021-03-10T17:00:50.721079+0000 mgr.gibba032.whopeo (mgr.24625) 9911 : cluster [DBG] pgmap v5648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:54.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:53 gibba011 bash[30791]: cluster 2021-03-10T17:00:52.722701+0000 mgr.gibba032.whopeo (mgr.24625) 9912 : cluster [DBG] pgmap v5649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:54.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:53 gibba011 bash[31774]: cluster 2021-03-10T17:00:52.722701+0000 mgr.gibba032.whopeo (mgr.24625) 9912 : cluster [DBG] pgmap v5649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:54.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:53 gibba032 bash[24004]: cluster 2021-03-10T17:00:52.722701+0000 mgr.gibba032.whopeo (mgr.24625) 9912 : cluster [DBG] pgmap v5649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:56.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:55 gibba032 bash[24004]: cluster 2021-03-10T17:00:54.723959+0000 mgr.gibba032.whopeo (mgr.24625) 9913 : cluster [DBG] pgmap v5650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:55 gibba011 bash[30791]: cluster 2021-03-10T17:00:54.723959+0000 mgr.gibba032.whopeo (mgr.24625) 9913 : cluster [DBG] pgmap v5650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:55 gibba011 bash[31774]: cluster 2021-03-10T17:00:54.723959+0000 mgr.gibba032.whopeo (mgr.24625) 9913 : cluster [DBG] pgmap v5650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: cluster 2021-03-10T17:00:56.725426+0000 mgr.gibba032.whopeo (mgr.24625) 9914 : cluster [DBG] pgmap v5651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.276134+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:00:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.276780+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:00:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.421176+0000 mon.a (mon.0) 3344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:00:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.421959+0000 mon.a (mon.0) 3345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.422654+0000 mon.a (mon.0) 3346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:57 gibba032 bash[24004]: audit 2021-03-10T17:00:57.423363+0000 mon.a (mon.0) 3347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:00:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: cluster 2021-03-10T17:00:56.725426+0000 mgr.gibba032.whopeo (mgr.24625) 9914 : cluster [DBG] pgmap v5651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.276134+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:00:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.276780+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:00:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.421176+0000 mon.a (mon.0) 3344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:00:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.421959+0000 mon.a (mon.0) 3345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.422654+0000 mon.a (mon.0) 3346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[30791]: audit 2021-03-10T17:00:57.423363+0000 mon.a (mon.0) 3347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:00:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: cluster 2021-03-10T17:00:56.725426+0000 mgr.gibba032.whopeo (mgr.24625) 9914 : cluster [DBG] pgmap v5651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:00:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.276134+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:00:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.276780+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:00:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.421176+0000 mon.a (mon.0) 3344 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:00:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.421959+0000 mon.a (mon.0) 3345 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.422654+0000 mon.a (mon.0) 3346 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:00:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:57 gibba011 bash[31774]: audit 2021-03-10T17:00:57.423363+0000 mon.a (mon.0) 3347 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:01:00.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:00:59 gibba032 bash[24004]: cluster 2021-03-10T17:00:58.727052+0000 mgr.gibba032.whopeo (mgr.24625) 9915 : cluster [DBG] pgmap v5652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:00:59 gibba011 bash[30791]: cluster 2021-03-10T17:00:58.727052+0000 mgr.gibba032.whopeo (mgr.24625) 9915 : cluster [DBG] pgmap v5652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:00.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:00:59 gibba011 bash[31774]: cluster 2021-03-10T17:00:58.727052+0000 mgr.gibba032.whopeo (mgr.24625) 9915 : cluster [DBG] pgmap v5652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:01.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:01 gibba032 bash[24004]: cluster 2021-03-10T17:01:00.728435+0000 mgr.gibba032.whopeo (mgr.24625) 9916 : cluster [DBG] pgmap v5653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:01 gibba011 bash[30791]: cluster 2021-03-10T17:01:00.728435+0000 mgr.gibba032.whopeo (mgr.24625) 9916 : cluster [DBG] pgmap v5653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:01.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:01 gibba011 bash[31774]: cluster 2021-03-10T17:01:00.728435+0000 mgr.gibba032.whopeo (mgr.24625) 9916 : cluster [DBG] pgmap v5653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:02.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:02 gibba032 bash[24004]: audit 2021-03-10T17:01:02.476930+0000 mon.a (mon.0) 3348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:01:02.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:02 gibba011 bash[30791]: audit 2021-03-10T17:01:02.476930+0000 mon.a (mon.0) 3348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:01:02.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:02 gibba011 bash[31774]: audit 2021-03-10T17:01:02.476930+0000 mon.a (mon.0) 3348 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:01:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:03 gibba032 bash[24004]: cephadm 2021-03-10T17:01:02.474560+0000 mgr.gibba032.whopeo (mgr.24625) 9917 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:01:03.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:03 gibba032 bash[24004]: cluster 2021-03-10T17:01:02.729211+0000 mgr.gibba032.whopeo (mgr.24625) 9918 : cluster [DBG] pgmap v5654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:03 gibba011 bash[30791]: cephadm 2021-03-10T17:01:02.474560+0000 mgr.gibba032.whopeo (mgr.24625) 9917 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:01:03.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:03 gibba011 bash[30791]: cluster 2021-03-10T17:01:02.729211+0000 mgr.gibba032.whopeo (mgr.24625) 9918 : cluster [DBG] pgmap v5654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:03.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:03 gibba011 bash[31774]: cephadm 2021-03-10T17:01:02.474560+0000 mgr.gibba032.whopeo (mgr.24625) 9917 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:01:03.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:03 gibba011 bash[31774]: cluster 2021-03-10T17:01:02.729211+0000 mgr.gibba032.whopeo (mgr.24625) 9918 : cluster [DBG] pgmap v5654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:06.077 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:05 gibba011 bash[30791]: cluster 2021-03-10T17:01:04.730609+0000 mgr.gibba032.whopeo (mgr.24625) 9919 : cluster [DBG] pgmap v5655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:06.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:05 gibba011 bash[31774]: cluster 2021-03-10T17:01:04.730609+0000 mgr.gibba032.whopeo (mgr.24625) 9919 : cluster [DBG] pgmap v5655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:06.248 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:05 gibba032 bash[24004]: cluster 2021-03-10T17:01:04.730609+0000 mgr.gibba032.whopeo (mgr.24625) 9919 : cluster [DBG] pgmap v5655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.033490+0000 mgr.gibba032.whopeo (mgr.24625) 9920 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:01:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.033520+0000 mon.a (mon.0) 3349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.231696+0000 mon.a (mon.0) 3350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.237651+0000 mgr.gibba032.whopeo (mgr.24625) 9921 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:01:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.239690+0000 mgr.gibba032.whopeo (mgr.24625) 9922 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:01:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.240181+0000 mon.a (mon.0) 3351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:01:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.240971+0000 mgr.gibba032.whopeo (mgr.24625) 9923 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:01:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.241007+0000 mgr.gibba032.whopeo (mgr.24625) 9924 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:01:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.241840+0000 mon.a (mon.0) 3352 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.242246+0000 mgr.gibba032.whopeo (mgr.24625) 9925 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:01:07.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.242282+0000 mgr.gibba032.whopeo (mgr.24625) 9926 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:01:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.243145+0000 mon.a (mon.0) 3353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.243515+0000 mgr.gibba032.whopeo (mgr.24625) 9927 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:01:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.244233+0000 mon.a (mon.0) 3354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.244814+0000 mgr.gibba032.whopeo (mgr.24625) 9928 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:01:07.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.244855+0000 mgr.gibba032.whopeo (mgr.24625) 9929 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:01:07.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.246134+0000 mgr.gibba032.whopeo (mgr.24625) 9930 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:01:07.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.246542+0000 mgr.gibba032.whopeo (mgr.24625) 9931 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:01:07.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.247072+0000 mon.a (mon.0) 3355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.247858+0000 mgr.gibba032.whopeo (mgr.24625) 9932 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:01:07.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.247903+0000 mgr.gibba032.whopeo (mgr.24625) 9933 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:01:07.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: cephadm 2021-03-10T17:01:06.248044+0000 mgr.gibba032.whopeo (mgr.24625) 9934 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:07.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[30791]: audit 2021-03-10T17:01:06.248596+0000 mon.a (mon.0) 3356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.033490+0000 mgr.gibba032.whopeo (mgr.24625) 9920 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:01:07.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.033520+0000 mon.a (mon.0) 3349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.231696+0000 mon.a (mon.0) 3350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.237651+0000 mgr.gibba032.whopeo (mgr.24625) 9921 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:01:07.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.239690+0000 mgr.gibba032.whopeo (mgr.24625) 9922 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:01:07.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.240181+0000 mon.a (mon.0) 3351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:01:07.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.240971+0000 mgr.gibba032.whopeo (mgr.24625) 9923 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:01:07.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.241007+0000 mgr.gibba032.whopeo (mgr.24625) 9924 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:01:07.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.241840+0000 mon.a (mon.0) 3352 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.242246+0000 mgr.gibba032.whopeo (mgr.24625) 9925 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:01:07.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.242282+0000 mgr.gibba032.whopeo (mgr.24625) 9926 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:01:07.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.243145+0000 mon.a (mon.0) 3353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.243515+0000 mgr.gibba032.whopeo (mgr.24625) 9927 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:01:07.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.244233+0000 mon.a (mon.0) 3354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.244814+0000 mgr.gibba032.whopeo (mgr.24625) 9928 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:01:07.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.244855+0000 mgr.gibba032.whopeo (mgr.24625) 9929 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:01:07.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.246134+0000 mgr.gibba032.whopeo (mgr.24625) 9930 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:01:07.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.246542+0000 mgr.gibba032.whopeo (mgr.24625) 9931 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:01:07.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.247072+0000 mon.a (mon.0) 3355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.247858+0000 mgr.gibba032.whopeo (mgr.24625) 9932 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:01:07.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.247903+0000 mgr.gibba032.whopeo (mgr.24625) 9933 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:01:07.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: cephadm 2021-03-10T17:01:06.248044+0000 mgr.gibba032.whopeo (mgr.24625) 9934 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:07.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:07 gibba011 bash[31774]: audit 2021-03-10T17:01:06.248596+0000 mon.a (mon.0) 3356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.033490+0000 mgr.gibba032.whopeo (mgr.24625) 9920 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:01:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.033520+0000 mon.a (mon.0) 3349 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.231696+0000 mon.a (mon.0) 3350 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:01:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.237651+0000 mgr.gibba032.whopeo (mgr.24625) 9921 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:01:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.239690+0000 mgr.gibba032.whopeo (mgr.24625) 9922 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:01:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.240181+0000 mon.a (mon.0) 3351 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:01:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.240971+0000 mgr.gibba032.whopeo (mgr.24625) 9923 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:01:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.241007+0000 mgr.gibba032.whopeo (mgr.24625) 9924 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:01:07.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.241840+0000 mon.a (mon.0) 3352 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.242246+0000 mgr.gibba032.whopeo (mgr.24625) 9925 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:01:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.242282+0000 mgr.gibba032.whopeo (mgr.24625) 9926 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:01:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.243145+0000 mon.a (mon.0) 3353 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.243515+0000 mgr.gibba032.whopeo (mgr.24625) 9927 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:01:07.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.244233+0000 mon.a (mon.0) 3354 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.244814+0000 mgr.gibba032.whopeo (mgr.24625) 9928 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:01:07.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.244855+0000 mgr.gibba032.whopeo (mgr.24625) 9929 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:01:07.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.246134+0000 mgr.gibba032.whopeo (mgr.24625) 9930 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:01:07.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.246542+0000 mgr.gibba032.whopeo (mgr.24625) 9931 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:01:07.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.247072+0000 mon.a (mon.0) 3355 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:07.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.247858+0000 mgr.gibba032.whopeo (mgr.24625) 9932 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:01:07.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.247903+0000 mgr.gibba032.whopeo (mgr.24625) 9933 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:01:07.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: cephadm 2021-03-10T17:01:06.248044+0000 mgr.gibba032.whopeo (mgr.24625) 9934 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:07.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:07 gibba032 bash[24004]: audit 2021-03-10T17:01:06.248596+0000 mon.a (mon.0) 3356 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:08.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:08 gibba032 bash[24004]: cluster 2021-03-10T17:01:06.732420+0000 mgr.gibba032.whopeo (mgr.24625) 9935 : cluster [DBG] pgmap v5656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:08 gibba011 bash[30791]: cluster 2021-03-10T17:01:06.732420+0000 mgr.gibba032.whopeo (mgr.24625) 9935 : cluster [DBG] pgmap v5656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:08 gibba011 bash[31774]: cluster 2021-03-10T17:01:06.732420+0000 mgr.gibba032.whopeo (mgr.24625) 9935 : cluster [DBG] pgmap v5656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:10.506 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:10 gibba032 bash[24004]: cluster 2021-03-10T17:01:08.733993+0000 mgr.gibba032.whopeo (mgr.24625) 9936 : cluster [DBG] pgmap v5657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:10 gibba011 bash[30791]: cluster 2021-03-10T17:01:08.733993+0000 mgr.gibba032.whopeo (mgr.24625) 9936 : cluster [DBG] pgmap v5657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:10.578 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:10 gibba011 bash[31774]: cluster 2021-03-10T17:01:08.733993+0000 mgr.gibba032.whopeo (mgr.24625) 9936 : cluster [DBG] pgmap v5657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:11.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:11 gibba032 bash[24004]: cluster 2021-03-10T17:01:10.735323+0000 mgr.gibba032.whopeo (mgr.24625) 9937 : cluster [DBG] pgmap v5658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:11.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:11 gibba011 bash[31774]: cluster 2021-03-10T17:01:10.735323+0000 mgr.gibba032.whopeo (mgr.24625) 9937 : cluster [DBG] pgmap v5658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:11 gibba011 bash[30791]: cluster 2021-03-10T17:01:10.735323+0000 mgr.gibba032.whopeo (mgr.24625) 9937 : cluster [DBG] pgmap v5658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:14.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:13 gibba011 bash[30791]: cluster 2021-03-10T17:01:12.737139+0000 mgr.gibba032.whopeo (mgr.24625) 9938 : cluster [DBG] pgmap v5659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:14.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:13 gibba011 bash[31774]: cluster 2021-03-10T17:01:12.737139+0000 mgr.gibba032.whopeo (mgr.24625) 9938 : cluster [DBG] pgmap v5659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:14.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:13 gibba032 bash[24004]: cluster 2021-03-10T17:01:12.737139+0000 mgr.gibba032.whopeo (mgr.24625) 9938 : cluster [DBG] pgmap v5659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:16.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:15 gibba011 bash[30791]: cluster 2021-03-10T17:01:14.738656+0000 mgr.gibba032.whopeo (mgr.24625) 9939 : cluster [DBG] pgmap v5660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:16.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:15 gibba011 bash[31774]: cluster 2021-03-10T17:01:14.738656+0000 mgr.gibba032.whopeo (mgr.24625) 9939 : cluster [DBG] pgmap v5660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:15 gibba032 bash[24004]: cluster 2021-03-10T17:01:14.738656+0000 mgr.gibba032.whopeo (mgr.24625) 9939 : cluster [DBG] pgmap v5660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:18.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:17 gibba011 bash[30791]: cluster 2021-03-10T17:01:16.740405+0000 mgr.gibba032.whopeo (mgr.24625) 9940 : cluster [DBG] pgmap v5661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:18.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:17 gibba011 bash[31774]: cluster 2021-03-10T17:01:16.740405+0000 mgr.gibba032.whopeo (mgr.24625) 9940 : cluster [DBG] pgmap v5661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:17 gibba032 bash[24004]: cluster 2021-03-10T17:01:16.740405+0000 mgr.gibba032.whopeo (mgr.24625) 9940 : cluster [DBG] pgmap v5661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:20.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:19 gibba011 bash[30791]: cluster 2021-03-10T17:01:18.741814+0000 mgr.gibba032.whopeo (mgr.24625) 9941 : cluster [DBG] pgmap v5662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:19 gibba011 bash[31774]: cluster 2021-03-10T17:01:18.741814+0000 mgr.gibba032.whopeo (mgr.24625) 9941 : cluster [DBG] pgmap v5662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:20.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:19 gibba032 bash[24004]: cluster 2021-03-10T17:01:18.741814+0000 mgr.gibba032.whopeo (mgr.24625) 9941 : cluster [DBG] pgmap v5662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:20.715 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:01:20.971 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:01:20.971 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 14s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:01:20.972 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:01:20.972 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:01:20.972 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:01:20.972 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:01:20.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:01:20.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:01:20.973 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:01:20.973 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 14s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:01:20.973 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:01:20.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:01:20.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:01:20.974 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:01:20.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 14s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:01:20.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:01:20.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:01:20.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:01:20.975 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:01:20.976 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:01:20.976 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 14s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:01:21.231 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:01:21.231 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:01:21.231 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:01:21.231 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:21.232 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:01:21.232 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:01:21.232 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:21.232 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:01:21.232 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:01:21.233 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:21.233 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:01:21.233 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:01:21.233 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:01:21.234 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:01:21.234 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:01:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:21 gibba032 bash[24004]: audit 2021-03-10T17:01:20.705008+0000 mgr.gibba032.whopeo (mgr.24625) 9942 : audit [DBG] from='client.60697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:21 gibba032 bash[24004]: cluster 2021-03-10T17:01:20.743181+0000 mgr.gibba032.whopeo (mgr.24625) 9943 : cluster [DBG] pgmap v5663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:21 gibba032 bash[24004]: audit 2021-03-10T17:01:20.930965+0000 mgr.gibba032.whopeo (mgr.24625) 9944 : audit [DBG] from='client.50721 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:21 gibba032 bash[24004]: audit 2021-03-10T17:01:21.231562+0000 mon.c (mon.1) 194 : audit [DBG] from='client.? 172.21.2.111:0/587509810' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:21 gibba032 bash[24004]: cephadm 2021-03-10T17:01:21.263918+0000 mgr.gibba032.whopeo (mgr.24625) 9945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[30791]: audit 2021-03-10T17:01:20.705008+0000 mgr.gibba032.whopeo (mgr.24625) 9942 : audit [DBG] from='client.60697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[30791]: cluster 2021-03-10T17:01:20.743181+0000 mgr.gibba032.whopeo (mgr.24625) 9943 : cluster [DBG] pgmap v5663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[30791]: audit 2021-03-10T17:01:20.930965+0000 mgr.gibba032.whopeo (mgr.24625) 9944 : audit [DBG] from='client.50721 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[30791]: audit 2021-03-10T17:01:21.231562+0000 mon.c (mon.1) 194 : audit [DBG] from='client.? 172.21.2.111:0/587509810' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[30791]: cephadm 2021-03-10T17:01:21.263918+0000 mgr.gibba032.whopeo (mgr.24625) 9945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:21.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[31774]: audit 2021-03-10T17:01:20.705008+0000 mgr.gibba032.whopeo (mgr.24625) 9942 : audit [DBG] from='client.60697 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[31774]: cluster 2021-03-10T17:01:20.743181+0000 mgr.gibba032.whopeo (mgr.24625) 9943 : cluster [DBG] pgmap v5663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:21.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[31774]: audit 2021-03-10T17:01:20.930965+0000 mgr.gibba032.whopeo (mgr.24625) 9944 : audit [DBG] from='client.50721 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[31774]: audit 2021-03-10T17:01:21.231562+0000 mon.c (mon.1) 194 : audit [DBG] from='client.? 172.21.2.111:0/587509810' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:21 gibba011 bash[31774]: cephadm 2021-03-10T17:01:21.263918+0000 mgr.gibba032.whopeo (mgr.24625) 9945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:24.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:23 gibba011 bash[30791]: cluster 2021-03-10T17:01:22.744959+0000 mgr.gibba032.whopeo (mgr.24625) 9946 : cluster [DBG] pgmap v5664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:24.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:23 gibba011 bash[31774]: cluster 2021-03-10T17:01:22.744959+0000 mgr.gibba032.whopeo (mgr.24625) 9946 : cluster [DBG] pgmap v5664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:24.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:23 gibba032 bash[24004]: cluster 2021-03-10T17:01:22.744959+0000 mgr.gibba032.whopeo (mgr.24625) 9946 : cluster [DBG] pgmap v5664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:26.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:25 gibba011 bash[30791]: cluster 2021-03-10T17:01:24.746351+0000 mgr.gibba032.whopeo (mgr.24625) 9947 : cluster [DBG] pgmap v5665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:26.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:25 gibba011 bash[31774]: cluster 2021-03-10T17:01:24.746351+0000 mgr.gibba032.whopeo (mgr.24625) 9947 : cluster [DBG] pgmap v5665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:26.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:25 gibba032 bash[24004]: cluster 2021-03-10T17:01:24.746351+0000 mgr.gibba032.whopeo (mgr.24625) 9947 : cluster [DBG] pgmap v5665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:28.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:27 gibba011 bash[31774]: cluster 2021-03-10T17:01:26.748012+0000 mgr.gibba032.whopeo (mgr.24625) 9948 : cluster [DBG] pgmap v5666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:28.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:27 gibba011 bash[30791]: cluster 2021-03-10T17:01:26.748012+0000 mgr.gibba032.whopeo (mgr.24625) 9948 : cluster [DBG] pgmap v5666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:28.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:27 gibba032 bash[24004]: cluster 2021-03-10T17:01:26.748012+0000 mgr.gibba032.whopeo (mgr.24625) 9948 : cluster [DBG] pgmap v5666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:29 gibba032 bash[24004]: cluster 2021-03-10T17:01:28.749580+0000 mgr.gibba032.whopeo (mgr.24625) 9949 : cluster [DBG] pgmap v5667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:29 gibba011 bash[30791]: cluster 2021-03-10T17:01:28.749580+0000 mgr.gibba032.whopeo (mgr.24625) 9949 : cluster [DBG] pgmap v5667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:29 gibba011 bash[31774]: cluster 2021-03-10T17:01:28.749580+0000 mgr.gibba032.whopeo (mgr.24625) 9949 : cluster [DBG] pgmap v5667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:31.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:31 gibba032 bash[24004]: cluster 2021-03-10T17:01:30.750898+0000 mgr.gibba032.whopeo (mgr.24625) 9950 : cluster [DBG] pgmap v5668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:31 gibba011 bash[30791]: cluster 2021-03-10T17:01:30.750898+0000 mgr.gibba032.whopeo (mgr.24625) 9950 : cluster [DBG] pgmap v5668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:31 gibba011 bash[31774]: cluster 2021-03-10T17:01:30.750898+0000 mgr.gibba032.whopeo (mgr.24625) 9950 : cluster [DBG] pgmap v5668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:34.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:33 gibba011 bash[30791]: cluster 2021-03-10T17:01:32.752600+0000 mgr.gibba032.whopeo (mgr.24625) 9951 : cluster [DBG] pgmap v5669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:34.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:33 gibba011 bash[31774]: cluster 2021-03-10T17:01:32.752600+0000 mgr.gibba032.whopeo (mgr.24625) 9951 : cluster [DBG] pgmap v5669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:33 gibba032 bash[24004]: cluster 2021-03-10T17:01:32.752600+0000 mgr.gibba032.whopeo (mgr.24625) 9951 : cluster [DBG] pgmap v5669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:36.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:35 gibba011 bash[30791]: cluster 2021-03-10T17:01:34.754029+0000 mgr.gibba032.whopeo (mgr.24625) 9952 : cluster [DBG] pgmap v5670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:36.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:35 gibba011 bash[31774]: cluster 2021-03-10T17:01:34.754029+0000 mgr.gibba032.whopeo (mgr.24625) 9952 : cluster [DBG] pgmap v5670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:36.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:35 gibba032 bash[24004]: cluster 2021-03-10T17:01:34.754029+0000 mgr.gibba032.whopeo (mgr.24625) 9952 : cluster [DBG] pgmap v5670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:37.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:36 gibba032 bash[24004]: cephadm 2021-03-10T17:01:36.276863+0000 mgr.gibba032.whopeo (mgr.24625) 9953 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:37.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:36 gibba011 bash[30791]: cephadm 2021-03-10T17:01:36.276863+0000 mgr.gibba032.whopeo (mgr.24625) 9953 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:36 gibba011 bash[31774]: cephadm 2021-03-10T17:01:36.276863+0000 mgr.gibba032.whopeo (mgr.24625) 9953 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:37 gibba032 bash[24004]: cluster 2021-03-10T17:01:36.755671+0000 mgr.gibba032.whopeo (mgr.24625) 9954 : cluster [DBG] pgmap v5671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:37 gibba011 bash[30791]: cluster 2021-03-10T17:01:36.755671+0000 mgr.gibba032.whopeo (mgr.24625) 9954 : cluster [DBG] pgmap v5671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:38.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:37 gibba011 bash[31774]: cluster 2021-03-10T17:01:36.755671+0000 mgr.gibba032.whopeo (mgr.24625) 9954 : cluster [DBG] pgmap v5671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:39 gibba032 bash[24004]: cluster 2021-03-10T17:01:38.757251+0000 mgr.gibba032.whopeo (mgr.24625) 9955 : cluster [DBG] pgmap v5672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:39 gibba011 bash[30791]: cluster 2021-03-10T17:01:38.757251+0000 mgr.gibba032.whopeo (mgr.24625) 9955 : cluster [DBG] pgmap v5672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:39 gibba011 bash[31774]: cluster 2021-03-10T17:01:38.757251+0000 mgr.gibba032.whopeo (mgr.24625) 9955 : cluster [DBG] pgmap v5672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:41.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:41 gibba032 bash[24004]: cluster 2021-03-10T17:01:40.758632+0000 mgr.gibba032.whopeo (mgr.24625) 9956 : cluster [DBG] pgmap v5673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:41 gibba011 bash[30791]: cluster 2021-03-10T17:01:40.758632+0000 mgr.gibba032.whopeo (mgr.24625) 9956 : cluster [DBG] pgmap v5673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:41.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:41 gibba011 bash[31774]: cluster 2021-03-10T17:01:40.758632+0000 mgr.gibba032.whopeo (mgr.24625) 9956 : cluster [DBG] pgmap v5673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:44.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:43 gibba011 bash[30791]: cluster 2021-03-10T17:01:42.760246+0000 mgr.gibba032.whopeo (mgr.24625) 9957 : cluster [DBG] pgmap v5674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:44.119 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:43 gibba011 bash[31774]: cluster 2021-03-10T17:01:42.760246+0000 mgr.gibba032.whopeo (mgr.24625) 9957 : cluster [DBG] pgmap v5674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:43 gibba032 bash[24004]: cluster 2021-03-10T17:01:42.760246+0000 mgr.gibba032.whopeo (mgr.24625) 9957 : cluster [DBG] pgmap v5674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:45 gibba032 bash[24004]: cluster 2021-03-10T17:01:44.761553+0000 mgr.gibba032.whopeo (mgr.24625) 9958 : cluster [DBG] pgmap v5675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:45 gibba011 bash[30791]: cluster 2021-03-10T17:01:44.761553+0000 mgr.gibba032.whopeo (mgr.24625) 9958 : cluster [DBG] pgmap v5675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:45 gibba011 bash[31774]: cluster 2021-03-10T17:01:44.761553+0000 mgr.gibba032.whopeo (mgr.24625) 9958 : cluster [DBG] pgmap v5675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:47 gibba032 bash[24004]: cluster 2021-03-10T17:01:46.763177+0000 mgr.gibba032.whopeo (mgr.24625) 9959 : cluster [DBG] pgmap v5676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:47 gibba011 bash[30791]: cluster 2021-03-10T17:01:46.763177+0000 mgr.gibba032.whopeo (mgr.24625) 9959 : cluster [DBG] pgmap v5676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:47 gibba011 bash[31774]: cluster 2021-03-10T17:01:46.763177+0000 mgr.gibba032.whopeo (mgr.24625) 9959 : cluster [DBG] pgmap v5676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:49 gibba032 bash[24004]: cluster 2021-03-10T17:01:48.764743+0000 mgr.gibba032.whopeo (mgr.24625) 9960 : cluster [DBG] pgmap v5677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:49 gibba011 bash[30791]: cluster 2021-03-10T17:01:48.764743+0000 mgr.gibba032.whopeo (mgr.24625) 9960 : cluster [DBG] pgmap v5677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:49 gibba011 bash[31774]: cluster 2021-03-10T17:01:48.764743+0000 mgr.gibba032.whopeo (mgr.24625) 9960 : cluster [DBG] pgmap v5677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:51.473 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:01:51.701 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:01:51.701 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 45s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:01:51.701 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:01:51.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:01:51.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:01:51.702 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:01:51.702 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:01:51.703 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:01:51.703 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:01:51.703 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 45s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:01:51.703 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:01:51.703 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:01:51.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:01:51.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:01:51.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:01:51.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:01:51.704 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:01:51.705 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:01:51.705 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:01:51.705 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:01:51.705 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 45s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:01:51.756 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:51 gibba032 bash[24004]: cluster 2021-03-10T17:01:50.766143+0000 mgr.gibba032.whopeo (mgr.24625) 9961 : cluster [DBG] pgmap v5678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:51 gibba032 bash[24004]: cephadm 2021-03-10T17:01:51.288843+0000 mgr.gibba032.whopeo (mgr.24625) 9962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:51.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:51 gibba032 bash[24004]: audit 2021-03-10T17:01:51.463414+0000 mgr.gibba032.whopeo (mgr.24625) 9963 : audit [DBG] from='client.60715 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[30791]: cluster 2021-03-10T17:01:50.766143+0000 mgr.gibba032.whopeo (mgr.24625) 9961 : cluster [DBG] pgmap v5678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[30791]: cephadm 2021-03-10T17:01:51.288843+0000 mgr.gibba032.whopeo (mgr.24625) 9962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[30791]: audit 2021-03-10T17:01:51.463414+0000 mgr.gibba032.whopeo (mgr.24625) 9963 : audit [DBG] from='client.60715 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[31774]: cluster 2021-03-10T17:01:50.766143+0000 mgr.gibba032.whopeo (mgr.24625) 9961 : cluster [DBG] pgmap v5678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[31774]: cephadm 2021-03-10T17:01:51.288843+0000 mgr.gibba032.whopeo (mgr.24625) 9962 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:01:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:51 gibba011 bash[31774]: audit 2021-03-10T17:01:51.463414+0000 mgr.gibba032.whopeo (mgr.24625) 9963 : audit [DBG] from='client.60715 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:51.952 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:01:51.952 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:01:51.952 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:01:51.953 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:51.953 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:01:51.953 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:01:51.953 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:51.953 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:01:51.954 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:01:51.954 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:01:51.954 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:01:51.954 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:01:51.954 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:01:51.955 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:01:51.955 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:01:52.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:52 gibba011 bash[30791]: audit 2021-03-10T17:01:51.687692+0000 mgr.gibba032.whopeo (mgr.24625) 9964 : audit [DBG] from='client.50739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:52 gibba011 bash[30791]: audit 2021-03-10T17:01:51.952097+0000 mon.a (mon.0) 3357 : audit [DBG] from='client.? 172.21.2.111:0/2694437410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:52 gibba011 bash[31774]: audit 2021-03-10T17:01:51.687692+0000 mgr.gibba032.whopeo (mgr.24625) 9964 : audit [DBG] from='client.50739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:52.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:52 gibba011 bash[31774]: audit 2021-03-10T17:01:51.952097+0000 mon.a (mon.0) 3357 : audit [DBG] from='client.? 172.21.2.111:0/2694437410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:52 gibba032 bash[24004]: audit 2021-03-10T17:01:51.687692+0000 mgr.gibba032.whopeo (mgr.24625) 9964 : audit [DBG] from='client.50739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:01:53.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:52 gibba032 bash[24004]: audit 2021-03-10T17:01:51.952097+0000 mon.a (mon.0) 3357 : audit [DBG] from='client.? 172.21.2.111:0/2694437410' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:01:53.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:53 gibba011 bash[30791]: cluster 2021-03-10T17:01:52.767834+0000 mgr.gibba032.whopeo (mgr.24625) 9965 : cluster [DBG] pgmap v5679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:53.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:53 gibba011 bash[31774]: cluster 2021-03-10T17:01:52.767834+0000 mgr.gibba032.whopeo (mgr.24625) 9965 : cluster [DBG] pgmap v5679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:54.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:53 gibba032 bash[24004]: cluster 2021-03-10T17:01:52.767834+0000 mgr.gibba032.whopeo (mgr.24625) 9965 : cluster [DBG] pgmap v5679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:55 gibba032 bash[24004]: cluster 2021-03-10T17:01:54.769337+0000 mgr.gibba032.whopeo (mgr.24625) 9966 : cluster [DBG] pgmap v5680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:55 gibba011 bash[30791]: cluster 2021-03-10T17:01:54.769337+0000 mgr.gibba032.whopeo (mgr.24625) 9966 : cluster [DBG] pgmap v5680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:55 gibba011 bash[31774]: cluster 2021-03-10T17:01:54.769337+0000 mgr.gibba032.whopeo (mgr.24625) 9966 : cluster [DBG] pgmap v5680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: cluster 2021-03-10T17:01:56.771176+0000 mgr.gibba032.whopeo (mgr.24625) 9967 : cluster [DBG] pgmap v5681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.283666+0000 mon.a (mon.0) 3358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:01:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.284334+0000 mon.a (mon.0) 3359 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:01:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.432377+0000 mon.a (mon.0) 3360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:01:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.433164+0000 mon.a (mon.0) 3361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:01:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.433843+0000 mon.a (mon.0) 3362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T17:01:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:57 gibba032 bash[24004]: audit 2021-03-10T17:01:57.434527+0000 mon.a (mon.0) 3363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:01:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: cluster 2021-03-10T17:01:56.771176+0000 mgr.gibba032.whopeo (mgr.24625) 9967 : cluster [DBG] pgmap v5681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.283666+0000 mon.a (mon.0) 3358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:01:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.284334+0000 mon.a (mon.0) 3359 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:01:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.432377+0000 mon.a (mon.0) 3360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:01:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.433164+0000 mon.a (mon.0) 3361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:01:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.433843+0000 mon.a (mon.0) 3362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T17:01:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[30791]: audit 2021-03-10T17:01:57.434527+0000 mon.a (mon.0) 3363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:01:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: cluster 2021-03-10T17:01:56.771176+0000 mgr.gibba032.whopeo (mgr.24625) 9967 : cluster [DBG] pgmap v5681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:01:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.283666+0000 mon.a (mon.0) 3358 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:01:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.284334+0000 mon.a (mon.0) 3359 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:01:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.432377+0000 mon.a (mon.0) 3360 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:01:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.433164+0000 mon.a (mon.0) 3361 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:01:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.433843+0000 mon.a (mon.0) 3362 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 5]}]: dispatch 2021-03-10T17:01:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:57 gibba011 bash[31774]: audit 2021-03-10T17:01:57.434527+0000 mon.a (mon.0) 3363 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:02:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:01:59 gibba032 bash[24004]: cluster 2021-03-10T17:01:58.772757+0000 mgr.gibba032.whopeo (mgr.24625) 9968 : cluster [DBG] pgmap v5682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:01:59 gibba011 bash[30791]: cluster 2021-03-10T17:01:58.772757+0000 mgr.gibba032.whopeo (mgr.24625) 9968 : cluster [DBG] pgmap v5682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:00.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:01:59 gibba011 bash[31774]: cluster 2021-03-10T17:01:58.772757+0000 mgr.gibba032.whopeo (mgr.24625) 9968 : cluster [DBG] pgmap v5682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:01 gibba011 bash[30791]: cluster 2021-03-10T17:02:00.774176+0000 mgr.gibba032.whopeo (mgr.24625) 9969 : cluster [DBG] pgmap v5683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:01.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:01 gibba011 bash[31774]: cluster 2021-03-10T17:02:00.774176+0000 mgr.gibba032.whopeo (mgr.24625) 9969 : cluster [DBG] pgmap v5683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:01 gibba032 bash[24004]: cluster 2021-03-10T17:02:00.774176+0000 mgr.gibba032.whopeo (mgr.24625) 9969 : cluster [DBG] pgmap v5683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:03 gibba032 bash[24004]: cluster 2021-03-10T17:02:02.775964+0000 mgr.gibba032.whopeo (mgr.24625) 9970 : cluster [DBG] pgmap v5684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:03 gibba011 bash[30791]: cluster 2021-03-10T17:02:02.775964+0000 mgr.gibba032.whopeo (mgr.24625) 9970 : cluster [DBG] pgmap v5684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:03 gibba011 bash[31774]: cluster 2021-03-10T17:02:02.775964+0000 mgr.gibba032.whopeo (mgr.24625) 9970 : cluster [DBG] pgmap v5684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:05 gibba032 bash[24004]: cluster 2021-03-10T17:02:04.777286+0000 mgr.gibba032.whopeo (mgr.24625) 9971 : cluster [DBG] pgmap v5685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:05 gibba011 bash[30791]: cluster 2021-03-10T17:02:04.777286+0000 mgr.gibba032.whopeo (mgr.24625) 9971 : cluster [DBG] pgmap v5685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:06.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:05 gibba011 bash[31774]: cluster 2021-03-10T17:02:04.777286+0000 mgr.gibba032.whopeo (mgr.24625) 9971 : cluster [DBG] pgmap v5685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:07.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:06 gibba032 bash[24004]: cephadm 2021-03-10T17:02:06.305600+0000 mgr.gibba032.whopeo (mgr.24625) 9972 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:02:07.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:06 gibba032 bash[24004]: audit 2021-03-10T17:02:06.309182+0000 mon.a (mon.0) 3364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:02:07.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:06 gibba011 bash[30791]: cephadm 2021-03-10T17:02:06.305600+0000 mgr.gibba032.whopeo (mgr.24625) 9972 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:02:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:06 gibba011 bash[30791]: audit 2021-03-10T17:02:06.309182+0000 mon.a (mon.0) 3364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:02:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:06 gibba011 bash[31774]: cephadm 2021-03-10T17:02:06.305600+0000 mgr.gibba032.whopeo (mgr.24625) 9972 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:02:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:06 gibba011 bash[31774]: audit 2021-03-10T17:02:06.309182+0000 mon.a (mon.0) 3364 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:02:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:07 gibba032 bash[24004]: cluster 2021-03-10T17:02:06.778932+0000 mgr.gibba032.whopeo (mgr.24625) 9973 : cluster [DBG] pgmap v5686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:07 gibba011 bash[30791]: cluster 2021-03-10T17:02:06.778932+0000 mgr.gibba032.whopeo (mgr.24625) 9973 : cluster [DBG] pgmap v5686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:07 gibba011 bash[31774]: cluster 2021-03-10T17:02:06.778932+0000 mgr.gibba032.whopeo (mgr.24625) 9973 : cluster [DBG] pgmap v5686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:10.177 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:09 gibba032 bash[24004]: cluster 2021-03-10T17:02:08.780387+0000 mgr.gibba032.whopeo (mgr.24625) 9974 : cluster [DBG] pgmap v5687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:09 gibba011 bash[30791]: cluster 2021-03-10T17:02:08.780387+0000 mgr.gibba032.whopeo (mgr.24625) 9974 : cluster [DBG] pgmap v5687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:09 gibba011 bash[31774]: cluster 2021-03-10T17:02:08.780387+0000 mgr.gibba032.whopeo (mgr.24625) 9974 : cluster [DBG] pgmap v5687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:11.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:09.939240+0000 mgr.gibba032.whopeo (mgr.24625) 9975 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:02:11.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:09.940118+0000 mon.a (mon.0) 3365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.163073+0000 mon.a (mon.0) 3366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.164188+0000 mgr.gibba032.whopeo (mgr.24625) 9976 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:02:11.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.165949+0000 mgr.gibba032.whopeo (mgr.24625) 9977 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:02:11.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.166582+0000 mon.a (mon.0) 3367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:02:11.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.167675+0000 mgr.gibba032.whopeo (mgr.24625) 9978 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:02:11.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.167713+0000 mgr.gibba032.whopeo (mgr.24625) 9979 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:02:11.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.168273+0000 mon.a (mon.0) 3368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.169400+0000 mgr.gibba032.whopeo (mgr.24625) 9980 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:02:11.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.169438+0000 mgr.gibba032.whopeo (mgr.24625) 9981 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:02:11.266 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.170017+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9982 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:02:11.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.171608+0000 mon.a (mon.0) 3370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.172425+0000 mgr.gibba032.whopeo (mgr.24625) 9983 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:02:11.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.172461+0000 mgr.gibba032.whopeo (mgr.24625) 9984 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:02:11.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.174102+0000 mgr.gibba032.whopeo (mgr.24625) 9985 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:02:11.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.174455+0000 mgr.gibba032.whopeo (mgr.24625) 9986 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:02:11.269 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.174866+0000 mon.a (mon.0) 3371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.176173+0000 mgr.gibba032.whopeo (mgr.24625) 9987 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:02:11.270 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.176212+0000 mgr.gibba032.whopeo (mgr.24625) 9988 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:02:11.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: cephadm 2021-03-10T17:02:10.176316+0000 mgr.gibba032.whopeo (mgr.24625) 9989 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:11.271 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:10 gibba032 bash[24004]: audit 2021-03-10T17:02:10.176758+0000 mon.a (mon.0) 3372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:09.939240+0000 mgr.gibba032.whopeo (mgr.24625) 9975 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:02:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:09.940118+0000 mon.a (mon.0) 3365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.163073+0000 mon.a (mon.0) 3366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.164188+0000 mgr.gibba032.whopeo (mgr.24625) 9976 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:02:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.165949+0000 mgr.gibba032.whopeo (mgr.24625) 9977 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:02:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.166582+0000 mon.a (mon.0) 3367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:02:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.167675+0000 mgr.gibba032.whopeo (mgr.24625) 9978 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:02:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.167713+0000 mgr.gibba032.whopeo (mgr.24625) 9979 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:02:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.168273+0000 mon.a (mon.0) 3368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.169400+0000 mgr.gibba032.whopeo (mgr.24625) 9980 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:02:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.169438+0000 mgr.gibba032.whopeo (mgr.24625) 9981 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:02:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.170017+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9982 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:02:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.171608+0000 mon.a (mon.0) 3370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.172425+0000 mgr.gibba032.whopeo (mgr.24625) 9983 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:02:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.172461+0000 mgr.gibba032.whopeo (mgr.24625) 9984 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:02:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.174102+0000 mgr.gibba032.whopeo (mgr.24625) 9985 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:02:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.174455+0000 mgr.gibba032.whopeo (mgr.24625) 9986 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:02:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.174866+0000 mon.a (mon.0) 3371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.176173+0000 mgr.gibba032.whopeo (mgr.24625) 9987 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:02:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.176212+0000 mgr.gibba032.whopeo (mgr.24625) 9988 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:02:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: cephadm 2021-03-10T17:02:10.176316+0000 mgr.gibba032.whopeo (mgr.24625) 9989 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:11.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[30791]: audit 2021-03-10T17:02:10.176758+0000 mon.a (mon.0) 3372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:09.939240+0000 mgr.gibba032.whopeo (mgr.24625) 9975 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:02:11.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:09.940118+0000 mon.a (mon.0) 3365 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.163073+0000 mon.a (mon.0) 3366 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:02:11.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.164188+0000 mgr.gibba032.whopeo (mgr.24625) 9976 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:02:11.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.165949+0000 mgr.gibba032.whopeo (mgr.24625) 9977 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:02:11.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.166582+0000 mon.a (mon.0) 3367 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:02:11.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.167675+0000 mgr.gibba032.whopeo (mgr.24625) 9978 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:02:11.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.167713+0000 mgr.gibba032.whopeo (mgr.24625) 9979 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:02:11.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.168273+0000 mon.a (mon.0) 3368 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.169400+0000 mgr.gibba032.whopeo (mgr.24625) 9980 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:02:11.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.169438+0000 mgr.gibba032.whopeo (mgr.24625) 9981 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:02:11.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.170017+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.170826+0000 mgr.gibba032.whopeo (mgr.24625) 9982 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:02:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.171608+0000 mon.a (mon.0) 3370 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.172425+0000 mgr.gibba032.whopeo (mgr.24625) 9983 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:02:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.172461+0000 mgr.gibba032.whopeo (mgr.24625) 9984 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:02:11.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.174102+0000 mgr.gibba032.whopeo (mgr.24625) 9985 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:02:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.174455+0000 mgr.gibba032.whopeo (mgr.24625) 9986 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:02:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.174866+0000 mon.a (mon.0) 3371 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.176173+0000 mgr.gibba032.whopeo (mgr.24625) 9987 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:02:11.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.176212+0000 mgr.gibba032.whopeo (mgr.24625) 9988 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:02:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: cephadm 2021-03-10T17:02:10.176316+0000 mgr.gibba032.whopeo (mgr.24625) 9989 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:11.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:10 gibba011 bash[31774]: audit 2021-03-10T17:02:10.176758+0000 mon.a (mon.0) 3372 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:11 gibba032 bash[24004]: cluster 2021-03-10T17:02:10.781754+0000 mgr.gibba032.whopeo (mgr.24625) 9990 : cluster [DBG] pgmap v5688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:11 gibba011 bash[30791]: cluster 2021-03-10T17:02:10.781754+0000 mgr.gibba032.whopeo (mgr.24625) 9990 : cluster [DBG] pgmap v5688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:11 gibba011 bash[31774]: cluster 2021-03-10T17:02:10.781754+0000 mgr.gibba032.whopeo (mgr.24625) 9990 : cluster [DBG] pgmap v5688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:13 gibba032 bash[24004]: cluster 2021-03-10T17:02:12.783291+0000 mgr.gibba032.whopeo (mgr.24625) 9991 : cluster [DBG] pgmap v5689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:13 gibba011 bash[30791]: cluster 2021-03-10T17:02:12.783291+0000 mgr.gibba032.whopeo (mgr.24625) 9991 : cluster [DBG] pgmap v5689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:13 gibba011 bash[31774]: cluster 2021-03-10T17:02:12.783291+0000 mgr.gibba032.whopeo (mgr.24625) 9991 : cluster [DBG] pgmap v5689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:15 gibba032 bash[24004]: cluster 2021-03-10T17:02:14.784830+0000 mgr.gibba032.whopeo (mgr.24625) 9992 : cluster [DBG] pgmap v5690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:15 gibba011 bash[30791]: cluster 2021-03-10T17:02:14.784830+0000 mgr.gibba032.whopeo (mgr.24625) 9992 : cluster [DBG] pgmap v5690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:16.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:15 gibba011 bash[31774]: cluster 2021-03-10T17:02:14.784830+0000 mgr.gibba032.whopeo (mgr.24625) 9992 : cluster [DBG] pgmap v5690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:18.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:17 gibba032 bash[24004]: cluster 2021-03-10T17:02:16.786358+0000 mgr.gibba032.whopeo (mgr.24625) 9993 : cluster [DBG] pgmap v5691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:17 gibba011 bash[30791]: cluster 2021-03-10T17:02:16.786358+0000 mgr.gibba032.whopeo (mgr.24625) 9993 : cluster [DBG] pgmap v5691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:18.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:17 gibba011 bash[31774]: cluster 2021-03-10T17:02:16.786358+0000 mgr.gibba032.whopeo (mgr.24625) 9993 : cluster [DBG] pgmap v5691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:19 gibba032 bash[24004]: cluster 2021-03-10T17:02:18.787858+0000 mgr.gibba032.whopeo (mgr.24625) 9994 : cluster [DBG] pgmap v5692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:19 gibba011 bash[30791]: cluster 2021-03-10T17:02:18.787858+0000 mgr.gibba032.whopeo (mgr.24625) 9994 : cluster [DBG] pgmap v5692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:19 gibba011 bash[31774]: cluster 2021-03-10T17:02:18.787858+0000 mgr.gibba032.whopeo (mgr.24625) 9994 : cluster [DBG] pgmap v5692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:21 gibba011 bash[30791]: cluster 2021-03-10T17:02:20.789206+0000 mgr.gibba032.whopeo (mgr.24625) 9995 : cluster [DBG] pgmap v5693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:21.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:21 gibba011 bash[31774]: cluster 2021-03-10T17:02:20.789206+0000 mgr.gibba032.whopeo (mgr.24625) 9995 : cluster [DBG] pgmap v5693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:21 gibba032 bash[24004]: cluster 2021-03-10T17:02:20.789206+0000 mgr.gibba032.whopeo (mgr.24625) 9995 : cluster [DBG] pgmap v5693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:22.193 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:02:22.420 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:02:22.420 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 76s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:02:22.420 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 12s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:02:22.421 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:02:22.421 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:02:22.421 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:02:22.421 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:02:22.422 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:02:22.422 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:02:22.422 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 76s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:02:22.422 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 12s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:02:22.422 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:02:22.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:02:22.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:02:22.423 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:02:22.424 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:02:22.424 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:02:22.424 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:02:22.424 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 12s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:02:22.424 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 12s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:02:22.425 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 76s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:02:22.671 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:02:22.671 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:02:22.671 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:02:22.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:22.672 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:02:22.672 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:02:22.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:22.673 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:02:22.673 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:02:22.673 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:22.674 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:02:22.674 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:02:22.674 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:02:22.674 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:02:22.674 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:02:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:22 gibba011 bash[30791]: audit 2021-03-10T17:02:22.184280+0000 mgr.gibba032.whopeo (mgr.24625) 9996 : audit [DBG] from='client.50751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:22 gibba011 bash[30791]: audit 2021-03-10T17:02:22.406872+0000 mgr.gibba032.whopeo (mgr.24625) 9997 : audit [DBG] from='client.60739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:22 gibba011 bash[31774]: audit 2021-03-10T17:02:22.184280+0000 mgr.gibba032.whopeo (mgr.24625) 9996 : audit [DBG] from='client.50751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:22 gibba011 bash[31774]: audit 2021-03-10T17:02:22.406872+0000 mgr.gibba032.whopeo (mgr.24625) 9997 : audit [DBG] from='client.60739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:22 gibba032 bash[24004]: audit 2021-03-10T17:02:22.184280+0000 mgr.gibba032.whopeo (mgr.24625) 9996 : audit [DBG] from='client.50751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:22 gibba032 bash[24004]: audit 2021-03-10T17:02:22.406872+0000 mgr.gibba032.whopeo (mgr.24625) 9997 : audit [DBG] from='client.60739 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:23.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:23 gibba011 bash[30791]: audit 2021-03-10T17:02:22.671758+0000 mon.a (mon.0) 3373 : audit [DBG] from='client.? 172.21.2.111:0/1376142725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:23.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:23 gibba011 bash[30791]: cluster 2021-03-10T17:02:22.791027+0000 mgr.gibba032.whopeo (mgr.24625) 9998 : cluster [DBG] pgmap v5694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:23.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:23 gibba011 bash[31774]: audit 2021-03-10T17:02:22.671758+0000 mon.a (mon.0) 3373 : audit [DBG] from='client.? 172.21.2.111:0/1376142725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:23.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:23 gibba011 bash[31774]: cluster 2021-03-10T17:02:22.791027+0000 mgr.gibba032.whopeo (mgr.24625) 9998 : cluster [DBG] pgmap v5694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:24.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:23 gibba032 bash[24004]: audit 2021-03-10T17:02:22.671758+0000 mon.a (mon.0) 3373 : audit [DBG] from='client.? 172.21.2.111:0/1376142725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:24.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:23 gibba032 bash[24004]: cluster 2021-03-10T17:02:22.791027+0000 mgr.gibba032.whopeo (mgr.24625) 9998 : cluster [DBG] pgmap v5694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:25 gibba032 bash[24004]: cluster 2021-03-10T17:02:24.792543+0000 mgr.gibba032.whopeo (mgr.24625) 9999 : cluster [DBG] pgmap v5695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:25 gibba032 bash[24004]: cephadm 2021-03-10T17:02:25.192422+0000 mgr.gibba032.whopeo (mgr.24625) 10000 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:25 gibba011 bash[30791]: cluster 2021-03-10T17:02:24.792543+0000 mgr.gibba032.whopeo (mgr.24625) 9999 : cluster [DBG] pgmap v5695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:25 gibba011 bash[30791]: cephadm 2021-03-10T17:02:25.192422+0000 mgr.gibba032.whopeo (mgr.24625) 10000 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:25 gibba011 bash[31774]: cluster 2021-03-10T17:02:24.792543+0000 mgr.gibba032.whopeo (mgr.24625) 9999 : cluster [DBG] pgmap v5695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:25 gibba011 bash[31774]: cephadm 2021-03-10T17:02:25.192422+0000 mgr.gibba032.whopeo (mgr.24625) 10000 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:27 gibba032 bash[24004]: cluster 2021-03-10T17:02:26.794091+0000 mgr.gibba032.whopeo (mgr.24625) 10001 : cluster [DBG] pgmap v5696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:27 gibba011 bash[30791]: cluster 2021-03-10T17:02:26.794091+0000 mgr.gibba032.whopeo (mgr.24625) 10001 : cluster [DBG] pgmap v5696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:27 gibba011 bash[31774]: cluster 2021-03-10T17:02:26.794091+0000 mgr.gibba032.whopeo (mgr.24625) 10001 : cluster [DBG] pgmap v5696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:29 gibba032 bash[24004]: cluster 2021-03-10T17:02:28.795582+0000 mgr.gibba032.whopeo (mgr.24625) 10002 : cluster [DBG] pgmap v5697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:29 gibba011 bash[30791]: cluster 2021-03-10T17:02:28.795582+0000 mgr.gibba032.whopeo (mgr.24625) 10002 : cluster [DBG] pgmap v5697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:29 gibba011 bash[31774]: cluster 2021-03-10T17:02:28.795582+0000 mgr.gibba032.whopeo (mgr.24625) 10002 : cluster [DBG] pgmap v5697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:31.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:31 gibba011 bash[30791]: cluster 2021-03-10T17:02:30.797071+0000 mgr.gibba032.whopeo (mgr.24625) 10003 : cluster [DBG] pgmap v5698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:31 gibba011 bash[31774]: cluster 2021-03-10T17:02:30.797071+0000 mgr.gibba032.whopeo (mgr.24625) 10003 : cluster [DBG] pgmap v5698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:31 gibba032 bash[24004]: cluster 2021-03-10T17:02:30.797071+0000 mgr.gibba032.whopeo (mgr.24625) 10003 : cluster [DBG] pgmap v5698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:33 gibba032 bash[24004]: cluster 2021-03-10T17:02:32.798924+0000 mgr.gibba032.whopeo (mgr.24625) 10004 : cluster [DBG] pgmap v5699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:33 gibba011 bash[30791]: cluster 2021-03-10T17:02:32.798924+0000 mgr.gibba032.whopeo (mgr.24625) 10004 : cluster [DBG] pgmap v5699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:33 gibba011 bash[31774]: cluster 2021-03-10T17:02:32.798924+0000 mgr.gibba032.whopeo (mgr.24625) 10004 : cluster [DBG] pgmap v5699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:35 gibba032 bash[24004]: cluster 2021-03-10T17:02:34.800262+0000 mgr.gibba032.whopeo (mgr.24625) 10005 : cluster [DBG] pgmap v5700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:35 gibba011 bash[30791]: cluster 2021-03-10T17:02:34.800262+0000 mgr.gibba032.whopeo (mgr.24625) 10005 : cluster [DBG] pgmap v5700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:35 gibba011 bash[31774]: cluster 2021-03-10T17:02:34.800262+0000 mgr.gibba032.whopeo (mgr.24625) 10005 : cluster [DBG] pgmap v5700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:37 gibba032 bash[24004]: cluster 2021-03-10T17:02:36.801803+0000 mgr.gibba032.whopeo (mgr.24625) 10006 : cluster [DBG] pgmap v5701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:37 gibba011 bash[30791]: cluster 2021-03-10T17:02:36.801803+0000 mgr.gibba032.whopeo (mgr.24625) 10006 : cluster [DBG] pgmap v5701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:37 gibba011 bash[31774]: cluster 2021-03-10T17:02:36.801803+0000 mgr.gibba032.whopeo (mgr.24625) 10006 : cluster [DBG] pgmap v5701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:40.210 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:39 gibba032 bash[24004]: cluster 2021-03-10T17:02:38.803408+0000 mgr.gibba032.whopeo (mgr.24625) 10007 : cluster [DBG] pgmap v5702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:39 gibba011 bash[30791]: cluster 2021-03-10T17:02:38.803408+0000 mgr.gibba032.whopeo (mgr.24625) 10007 : cluster [DBG] pgmap v5702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:39 gibba011 bash[31774]: cluster 2021-03-10T17:02:38.803408+0000 mgr.gibba032.whopeo (mgr.24625) 10007 : cluster [DBG] pgmap v5702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:41.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:40 gibba032 bash[24004]: cephadm 2021-03-10T17:02:40.208430+0000 mgr.gibba032.whopeo (mgr.24625) 10008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:40 gibba011 bash[30791]: cephadm 2021-03-10T17:02:40.208430+0000 mgr.gibba032.whopeo (mgr.24625) 10008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:40 gibba011 bash[31774]: cephadm 2021-03-10T17:02:40.208430+0000 mgr.gibba032.whopeo (mgr.24625) 10008 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:41 gibba032 bash[24004]: cluster 2021-03-10T17:02:40.804833+0000 mgr.gibba032.whopeo (mgr.24625) 10009 : cluster [DBG] pgmap v5703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:41 gibba011 bash[30791]: cluster 2021-03-10T17:02:40.804833+0000 mgr.gibba032.whopeo (mgr.24625) 10009 : cluster [DBG] pgmap v5703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:41 gibba011 bash[31774]: cluster 2021-03-10T17:02:40.804833+0000 mgr.gibba032.whopeo (mgr.24625) 10009 : cluster [DBG] pgmap v5703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:43 gibba032 bash[24004]: cluster 2021-03-10T17:02:42.806562+0000 mgr.gibba032.whopeo (mgr.24625) 10010 : cluster [DBG] pgmap v5704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:43 gibba011 bash[30791]: cluster 2021-03-10T17:02:42.806562+0000 mgr.gibba032.whopeo (mgr.24625) 10010 : cluster [DBG] pgmap v5704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:43 gibba011 bash[31774]: cluster 2021-03-10T17:02:42.806562+0000 mgr.gibba032.whopeo (mgr.24625) 10010 : cluster [DBG] pgmap v5704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:46.256 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:45 gibba032 bash[24004]: cluster 2021-03-10T17:02:44.808133+0000 mgr.gibba032.whopeo (mgr.24625) 10011 : cluster [DBG] pgmap v5705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:45 gibba011 bash[30791]: cluster 2021-03-10T17:02:44.808133+0000 mgr.gibba032.whopeo (mgr.24625) 10011 : cluster [DBG] pgmap v5705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:45 gibba011 bash[31774]: cluster 2021-03-10T17:02:44.808133+0000 mgr.gibba032.whopeo (mgr.24625) 10011 : cluster [DBG] pgmap v5705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:47 gibba032 bash[24004]: cluster 2021-03-10T17:02:46.810017+0000 mgr.gibba032.whopeo (mgr.24625) 10012 : cluster [DBG] pgmap v5706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:47 gibba011 bash[30791]: cluster 2021-03-10T17:02:46.810017+0000 mgr.gibba032.whopeo (mgr.24625) 10012 : cluster [DBG] pgmap v5706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:47 gibba011 bash[31774]: cluster 2021-03-10T17:02:46.810017+0000 mgr.gibba032.whopeo (mgr.24625) 10012 : cluster [DBG] pgmap v5706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:49 gibba032 bash[24004]: cluster 2021-03-10T17:02:48.811651+0000 mgr.gibba032.whopeo (mgr.24625) 10013 : cluster [DBG] pgmap v5707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:49 gibba011 bash[30791]: cluster 2021-03-10T17:02:48.811651+0000 mgr.gibba032.whopeo (mgr.24625) 10013 : cluster [DBG] pgmap v5707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:49 gibba011 bash[31774]: cluster 2021-03-10T17:02:48.811651+0000 mgr.gibba032.whopeo (mgr.24625) 10013 : cluster [DBG] pgmap v5707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:51 gibba011 bash[30791]: cluster 2021-03-10T17:02:50.812216+0000 mgr.gibba032.whopeo (mgr.24625) 10014 : cluster [DBG] pgmap v5708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:51 gibba011 bash[31774]: cluster 2021-03-10T17:02:50.812216+0000 mgr.gibba032.whopeo (mgr.24625) 10014 : cluster [DBG] pgmap v5708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:52.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:51 gibba032 bash[24004]: cluster 2021-03-10T17:02:50.812216+0000 mgr.gibba032.whopeo (mgr.24625) 10014 : cluster [DBG] pgmap v5708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:52.911 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:02:53.141 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:02:53.142 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 107s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:02:53.142 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 43s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:02:53.142 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:02:53.142 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:02:53.143 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:02:53.143 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:02:53.143 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:02:53.143 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:02:53.144 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 107s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:02:53.144 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 43s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:02:53.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:02:53.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:02:53.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:02:53.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:02:53.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:02:53.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:02:53.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:02:53.146 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 43s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:02:53.146 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 43s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:02:53.146 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 107s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:02:53.400 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:02:53.401 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:02:53.401 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:02:53.401 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:53.401 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:02:53.402 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:02:53.402 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:53.402 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:02:53.402 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:02:53.403 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:02:53.403 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:02:53.403 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:02:53.403 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:02:53.403 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:02:53.404 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:02:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:53 gibba032 bash[24004]: cluster 2021-03-10T17:02:52.813590+0000 mgr.gibba032.whopeo (mgr.24625) 10015 : cluster [DBG] pgmap v5709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:53 gibba032 bash[24004]: audit 2021-03-10T17:02:52.902176+0000 mgr.gibba032.whopeo (mgr.24625) 10016 : audit [DBG] from='client.60751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:53 gibba032 bash[24004]: audit 2021-03-10T17:02:53.128341+0000 mgr.gibba032.whopeo (mgr.24625) 10017 : audit [DBG] from='client.60757 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:53 gibba032 bash[24004]: audit 2021-03-10T17:02:53.401279+0000 mon.a (mon.0) 3374 : audit [DBG] from='client.? 172.21.2.111:0/1945435693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[30791]: cluster 2021-03-10T17:02:52.813590+0000 mgr.gibba032.whopeo (mgr.24625) 10015 : cluster [DBG] pgmap v5709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[30791]: audit 2021-03-10T17:02:52.902176+0000 mgr.gibba032.whopeo (mgr.24625) 10016 : audit [DBG] from='client.60751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[30791]: audit 2021-03-10T17:02:53.128341+0000 mgr.gibba032.whopeo (mgr.24625) 10017 : audit [DBG] from='client.60757 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[30791]: audit 2021-03-10T17:02:53.401279+0000 mon.a (mon.0) 3374 : audit [DBG] from='client.? 172.21.2.111:0/1945435693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[31774]: cluster 2021-03-10T17:02:52.813590+0000 mgr.gibba032.whopeo (mgr.24625) 10015 : cluster [DBG] pgmap v5709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[31774]: audit 2021-03-10T17:02:52.902176+0000 mgr.gibba032.whopeo (mgr.24625) 10016 : audit [DBG] from='client.60751 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[31774]: audit 2021-03-10T17:02:53.128341+0000 mgr.gibba032.whopeo (mgr.24625) 10017 : audit [DBG] from='client.60757 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:02:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:53 gibba011 bash[31774]: audit 2021-03-10T17:02:53.401279+0000 mon.a (mon.0) 3374 : audit [DBG] from='client.? 172.21.2.111:0/1945435693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:02:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:55 gibba032 bash[24004]: cluster 2021-03-10T17:02:54.815289+0000 mgr.gibba032.whopeo (mgr.24625) 10018 : cluster [DBG] pgmap v5710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:55 gibba032 bash[24004]: cephadm 2021-03-10T17:02:55.224545+0000 mgr.gibba032.whopeo (mgr.24625) 10019 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:55 gibba011 bash[30791]: cluster 2021-03-10T17:02:54.815289+0000 mgr.gibba032.whopeo (mgr.24625) 10018 : cluster [DBG] pgmap v5710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:55 gibba011 bash[30791]: cephadm 2021-03-10T17:02:55.224545+0000 mgr.gibba032.whopeo (mgr.24625) 10019 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:55 gibba011 bash[31774]: cluster 2021-03-10T17:02:54.815289+0000 mgr.gibba032.whopeo (mgr.24625) 10018 : cluster [DBG] pgmap v5710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:55 gibba011 bash[31774]: cephadm 2021-03-10T17:02:55.224545+0000 mgr.gibba032.whopeo (mgr.24625) 10019 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:02:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: cluster 2021-03-10T17:02:56.817152+0000 mgr.gibba032.whopeo (mgr.24625) 10020 : cluster [DBG] pgmap v5711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.291046+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:02:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.291736+0000 mon.a (mon.0) 3376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:02:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.442304+0000 mon.a (mon.0) 3377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:02:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.443090+0000 mon.a (mon.0) 3378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.443847+0000 mon.a (mon.0) 3379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:57 gibba032 bash[24004]: audit 2021-03-10T17:02:57.444535+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:02:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: cluster 2021-03-10T17:02:56.817152+0000 mgr.gibba032.whopeo (mgr.24625) 10020 : cluster [DBG] pgmap v5711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.291046+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:02:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.291736+0000 mon.a (mon.0) 3376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:02:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.442304+0000 mon.a (mon.0) 3377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:02:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.443090+0000 mon.a (mon.0) 3378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.443847+0000 mon.a (mon.0) 3379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[30791]: audit 2021-03-10T17:02:57.444535+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:02:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: cluster 2021-03-10T17:02:56.817152+0000 mgr.gibba032.whopeo (mgr.24625) 10020 : cluster [DBG] pgmap v5711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:02:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.291046+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:02:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.291736+0000 mon.a (mon.0) 3376 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:02:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.442304+0000 mon.a (mon.0) 3377 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:02:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.443090+0000 mon.a (mon.0) 3378 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.443847+0000 mon.a (mon.0) 3379 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:02:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:57 gibba011 bash[31774]: audit 2021-03-10T17:02:57.444535+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:03:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:02:59 gibba032 bash[24004]: cluster 2021-03-10T17:02:58.817658+0000 mgr.gibba032.whopeo (mgr.24625) 10021 : cluster [DBG] pgmap v5712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:02:59 gibba011 bash[30791]: cluster 2021-03-10T17:02:58.817658+0000 mgr.gibba032.whopeo (mgr.24625) 10021 : cluster [DBG] pgmap v5712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:02:59 gibba011 bash[31774]: cluster 2021-03-10T17:02:58.817658+0000 mgr.gibba032.whopeo (mgr.24625) 10021 : cluster [DBG] pgmap v5712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:01.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:01 gibba011 bash[30791]: cluster 2021-03-10T17:03:00.818923+0000 mgr.gibba032.whopeo (mgr.24625) 10022 : cluster [DBG] pgmap v5713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:01.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:01 gibba011 bash[31774]: cluster 2021-03-10T17:03:00.818923+0000 mgr.gibba032.whopeo (mgr.24625) 10022 : cluster [DBG] pgmap v5713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:01 gibba032 bash[24004]: cluster 2021-03-10T17:03:00.818923+0000 mgr.gibba032.whopeo (mgr.24625) 10022 : cluster [DBG] pgmap v5713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:03 gibba032 bash[24004]: cluster 2021-03-10T17:03:02.820734+0000 mgr.gibba032.whopeo (mgr.24625) 10023 : cluster [DBG] pgmap v5714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:03 gibba011 bash[30791]: cluster 2021-03-10T17:03:02.820734+0000 mgr.gibba032.whopeo (mgr.24625) 10023 : cluster [DBG] pgmap v5714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:03 gibba011 bash[31774]: cluster 2021-03-10T17:03:02.820734+0000 mgr.gibba032.whopeo (mgr.24625) 10023 : cluster [DBG] pgmap v5714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:05 gibba032 bash[24004]: cluster 2021-03-10T17:03:04.822248+0000 mgr.gibba032.whopeo (mgr.24625) 10024 : cluster [DBG] pgmap v5715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:05 gibba011 bash[30791]: cluster 2021-03-10T17:03:04.822248+0000 mgr.gibba032.whopeo (mgr.24625) 10024 : cluster [DBG] pgmap v5715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:05 gibba011 bash[31774]: cluster 2021-03-10T17:03:04.822248+0000 mgr.gibba032.whopeo (mgr.24625) 10024 : cluster [DBG] pgmap v5715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:07 gibba032 bash[24004]: cluster 2021-03-10T17:03:06.822911+0000 mgr.gibba032.whopeo (mgr.24625) 10025 : cluster [DBG] pgmap v5716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:07 gibba011 bash[30791]: cluster 2021-03-10T17:03:06.822911+0000 mgr.gibba032.whopeo (mgr.24625) 10025 : cluster [DBG] pgmap v5716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:07 gibba011 bash[31774]: cluster 2021-03-10T17:03:06.822911+0000 mgr.gibba032.whopeo (mgr.24625) 10025 : cluster [DBG] pgmap v5716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:09 gibba032 bash[24004]: cluster 2021-03-10T17:03:08.824467+0000 mgr.gibba032.whopeo (mgr.24625) 10026 : cluster [DBG] pgmap v5717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:09 gibba011 bash[30791]: cluster 2021-03-10T17:03:08.824467+0000 mgr.gibba032.whopeo (mgr.24625) 10026 : cluster [DBG] pgmap v5717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:09 gibba011 bash[31774]: cluster 2021-03-10T17:03:08.824467+0000 mgr.gibba032.whopeo (mgr.24625) 10026 : cluster [DBG] pgmap v5717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.241429+0000 mgr.gibba032.whopeo (mgr.24625) 10027 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:03:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.243003+0000 mgr.gibba032.whopeo (mgr.24625) 10028 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:03:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.452588+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:03:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.453277+0000 mgr.gibba032.whopeo (mgr.24625) 10029 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:03:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.454971+0000 mgr.gibba032.whopeo (mgr.24625) 10030 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:03:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.455679+0000 mon.a (mon.0) 3382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:03:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.456188+0000 mgr.gibba032.whopeo (mgr.24625) 10031 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:03:11.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.456227+0000 mgr.gibba032.whopeo (mgr.24625) 10032 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:03:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.457159+0000 mon.a (mon.0) 3383 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.457710+0000 mgr.gibba032.whopeo (mgr.24625) 10033 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:03:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.457749+0000 mgr.gibba032.whopeo (mgr.24625) 10034 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:03:11.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.458488+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.459033+0000 mgr.gibba032.whopeo (mgr.24625) 10035 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:03:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.459865+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.460117+0000 mgr.gibba032.whopeo (mgr.24625) 10036 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:03:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.460152+0000 mgr.gibba032.whopeo (mgr.24625) 10037 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:03:11.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.461651+0000 mgr.gibba032.whopeo (mgr.24625) 10038 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:03:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.461972+0000 mgr.gibba032.whopeo (mgr.24625) 10039 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:03:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.462478+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.463281+0000 mgr.gibba032.whopeo (mgr.24625) 10040 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:03:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.463319+0000 mgr.gibba032.whopeo (mgr.24625) 10041 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:03:11.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: cephadm 2021-03-10T17:03:10.463426+0000 mgr.gibba032.whopeo (mgr.24625) 10042 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:11.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:11 gibba032 bash[24004]: audit 2021-03-10T17:03:10.464076+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.241429+0000 mgr.gibba032.whopeo (mgr.24625) 10027 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:03:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.243003+0000 mgr.gibba032.whopeo (mgr.24625) 10028 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:03:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.452588+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:03:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.453277+0000 mgr.gibba032.whopeo (mgr.24625) 10029 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:03:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.454971+0000 mgr.gibba032.whopeo (mgr.24625) 10030 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:03:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.455679+0000 mon.a (mon.0) 3382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:03:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.456188+0000 mgr.gibba032.whopeo (mgr.24625) 10031 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:03:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.456227+0000 mgr.gibba032.whopeo (mgr.24625) 10032 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:03:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.457159+0000 mon.a (mon.0) 3383 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.457710+0000 mgr.gibba032.whopeo (mgr.24625) 10033 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:03:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.457749+0000 mgr.gibba032.whopeo (mgr.24625) 10034 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:03:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.458488+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.459033+0000 mgr.gibba032.whopeo (mgr.24625) 10035 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:03:11.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.459865+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.460117+0000 mgr.gibba032.whopeo (mgr.24625) 10036 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:03:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.460152+0000 mgr.gibba032.whopeo (mgr.24625) 10037 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:03:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.461651+0000 mgr.gibba032.whopeo (mgr.24625) 10038 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:03:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.461972+0000 mgr.gibba032.whopeo (mgr.24625) 10039 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:03:11.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.462478+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.463281+0000 mgr.gibba032.whopeo (mgr.24625) 10040 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:03:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.463319+0000 mgr.gibba032.whopeo (mgr.24625) 10041 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:03:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: cephadm 2021-03-10T17:03:10.463426+0000 mgr.gibba032.whopeo (mgr.24625) 10042 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:11.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[30791]: audit 2021-03-10T17:03:10.464076+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.241429+0000 mgr.gibba032.whopeo (mgr.24625) 10027 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:03:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.243003+0000 mgr.gibba032.whopeo (mgr.24625) 10028 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:03:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.452588+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:03:11.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.453277+0000 mgr.gibba032.whopeo (mgr.24625) 10029 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:03:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.454971+0000 mgr.gibba032.whopeo (mgr.24625) 10030 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:03:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.455679+0000 mon.a (mon.0) 3382 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:03:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.456188+0000 mgr.gibba032.whopeo (mgr.24625) 10031 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:03:11.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.456227+0000 mgr.gibba032.whopeo (mgr.24625) 10032 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:03:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.457159+0000 mon.a (mon.0) 3383 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.457710+0000 mgr.gibba032.whopeo (mgr.24625) 10033 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:03:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.457749+0000 mgr.gibba032.whopeo (mgr.24625) 10034 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:03:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.458488+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.459033+0000 mgr.gibba032.whopeo (mgr.24625) 10035 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:03:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.459865+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.460117+0000 mgr.gibba032.whopeo (mgr.24625) 10036 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:03:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.460152+0000 mgr.gibba032.whopeo (mgr.24625) 10037 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:03:11.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.461651+0000 mgr.gibba032.whopeo (mgr.24625) 10038 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:03:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.461972+0000 mgr.gibba032.whopeo (mgr.24625) 10039 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:03:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.462478+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.463281+0000 mgr.gibba032.whopeo (mgr.24625) 10040 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:03:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.463319+0000 mgr.gibba032.whopeo (mgr.24625) 10041 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:03:11.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: cephadm 2021-03-10T17:03:10.463426+0000 mgr.gibba032.whopeo (mgr.24625) 10042 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:11.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:11 gibba011 bash[31774]: audit 2021-03-10T17:03:10.464076+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:12 gibba032 bash[24004]: cluster 2021-03-10T17:03:10.825760+0000 mgr.gibba032.whopeo (mgr.24625) 10043 : cluster [DBG] pgmap v5718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:12.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:12 gibba011 bash[30791]: cluster 2021-03-10T17:03:10.825760+0000 mgr.gibba032.whopeo (mgr.24625) 10043 : cluster [DBG] pgmap v5718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:12 gibba011 bash[31774]: cluster 2021-03-10T17:03:10.825760+0000 mgr.gibba032.whopeo (mgr.24625) 10043 : cluster [DBG] pgmap v5718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:14 gibba032 bash[24004]: cluster 2021-03-10T17:03:12.827599+0000 mgr.gibba032.whopeo (mgr.24625) 10044 : cluster [DBG] pgmap v5719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:14 gibba011 bash[30791]: cluster 2021-03-10T17:03:12.827599+0000 mgr.gibba032.whopeo (mgr.24625) 10044 : cluster [DBG] pgmap v5719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:14.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:14 gibba011 bash[31774]: cluster 2021-03-10T17:03:12.827599+0000 mgr.gibba032.whopeo (mgr.24625) 10044 : cluster [DBG] pgmap v5719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:16 gibba032 bash[24004]: cluster 2021-03-10T17:03:14.828367+0000 mgr.gibba032.whopeo (mgr.24625) 10045 : cluster [DBG] pgmap v5720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:16 gibba011 bash[30791]: cluster 2021-03-10T17:03:14.828367+0000 mgr.gibba032.whopeo (mgr.24625) 10045 : cluster [DBG] pgmap v5720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:16.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:16 gibba011 bash[31774]: cluster 2021-03-10T17:03:14.828367+0000 mgr.gibba032.whopeo (mgr.24625) 10045 : cluster [DBG] pgmap v5720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:18 gibba032 bash[24004]: cluster 2021-03-10T17:03:16.829761+0000 mgr.gibba032.whopeo (mgr.24625) 10046 : cluster [DBG] pgmap v5721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:18.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:18 gibba011 bash[30791]: cluster 2021-03-10T17:03:16.829761+0000 mgr.gibba032.whopeo (mgr.24625) 10046 : cluster [DBG] pgmap v5721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:18 gibba011 bash[31774]: cluster 2021-03-10T17:03:16.829761+0000 mgr.gibba032.whopeo (mgr.24625) 10046 : cluster [DBG] pgmap v5721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:20 gibba032 bash[24004]: cluster 2021-03-10T17:03:18.831361+0000 mgr.gibba032.whopeo (mgr.24625) 10047 : cluster [DBG] pgmap v5722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:20 gibba011 bash[30791]: cluster 2021-03-10T17:03:18.831361+0000 mgr.gibba032.whopeo (mgr.24625) 10047 : cluster [DBG] pgmap v5722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:20 gibba011 bash[31774]: cluster 2021-03-10T17:03:18.831361+0000 mgr.gibba032.whopeo (mgr.24625) 10047 : cluster [DBG] pgmap v5722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:21.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:21 gibba011 bash[30791]: cluster 2021-03-10T17:03:20.832866+0000 mgr.gibba032.whopeo (mgr.24625) 10048 : cluster [DBG] pgmap v5723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:21.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:21 gibba011 bash[31774]: cluster 2021-03-10T17:03:20.832866+0000 mgr.gibba032.whopeo (mgr.24625) 10048 : cluster [DBG] pgmap v5723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:21 gibba032 bash[24004]: cluster 2021-03-10T17:03:20.832866+0000 mgr.gibba032.whopeo (mgr.24625) 10048 : cluster [DBG] pgmap v5723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:23.648 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:03:23.876 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:03:23.876 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:03:23.877 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 73s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:03:23.877 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:03:23.877 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:03:23.877 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:03:23.878 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:03:23.878 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:03:23.878 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:03:23.878 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:03:23.878 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 73s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:03:23.879 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:03:23.879 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:03:23.879 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:03:23.879 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:03:23.879 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:03:23.880 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:03:23.880 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:03:23.880 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 73s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:03:23.880 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 73s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:03:23.881 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:03:24.142 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:03:24.143 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:03:24.143 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:03:24.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:24.143 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:03:24.144 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:03:24.144 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:24.144 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:03:24.144 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:03:24.145 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:24.145 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:03:24.145 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:03:24.145 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:03:24.145 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:03:24.146 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:03:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:23 gibba032 bash[24004]: cluster 2021-03-10T17:03:22.833780+0000 mgr.gibba032.whopeo (mgr.24625) 10049 : cluster [DBG] pgmap v5724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:23 gibba011 bash[30791]: cluster 2021-03-10T17:03:22.833780+0000 mgr.gibba032.whopeo (mgr.24625) 10049 : cluster [DBG] pgmap v5724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:23 gibba011 bash[31774]: cluster 2021-03-10T17:03:22.833780+0000 mgr.gibba032.whopeo (mgr.24625) 10049 : cluster [DBG] pgmap v5724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:24 gibba032 bash[24004]: audit 2021-03-10T17:03:23.637828+0000 mgr.gibba032.whopeo (mgr.24625) 10050 : audit [DBG] from='client.50787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:24 gibba032 bash[24004]: audit 2021-03-10T17:03:23.862102+0000 mgr.gibba032.whopeo (mgr.24625) 10051 : audit [DBG] from='client.60775 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:24 gibba032 bash[24004]: audit 2021-03-10T17:03:24.142827+0000 mon.a (mon.0) 3388 : audit [DBG] from='client.? 172.21.2.111:0/370150591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:25.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[30791]: audit 2021-03-10T17:03:23.637828+0000 mgr.gibba032.whopeo (mgr.24625) 10050 : audit [DBG] from='client.50787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[30791]: audit 2021-03-10T17:03:23.862102+0000 mgr.gibba032.whopeo (mgr.24625) 10051 : audit [DBG] from='client.60775 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[30791]: audit 2021-03-10T17:03:24.142827+0000 mon.a (mon.0) 3388 : audit [DBG] from='client.? 172.21.2.111:0/370150591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[31774]: audit 2021-03-10T17:03:23.637828+0000 mgr.gibba032.whopeo (mgr.24625) 10050 : audit [DBG] from='client.50787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[31774]: audit 2021-03-10T17:03:23.862102+0000 mgr.gibba032.whopeo (mgr.24625) 10051 : audit [DBG] from='client.60775 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:24 gibba011 bash[31774]: audit 2021-03-10T17:03:24.142827+0000 mon.a (mon.0) 3388 : audit [DBG] from='client.? 172.21.2.111:0/370150591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:25 gibba032 bash[24004]: cluster 2021-03-10T17:03:24.835072+0000 mgr.gibba032.whopeo (mgr.24625) 10052 : cluster [DBG] pgmap v5725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:25 gibba032 bash[24004]: cephadm 2021-03-10T17:03:25.479189+0000 mgr.gibba032.whopeo (mgr.24625) 10053 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:25 gibba011 bash[30791]: cluster 2021-03-10T17:03:24.835072+0000 mgr.gibba032.whopeo (mgr.24625) 10052 : cluster [DBG] pgmap v5725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:25 gibba011 bash[30791]: cephadm 2021-03-10T17:03:25.479189+0000 mgr.gibba032.whopeo (mgr.24625) 10053 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:25 gibba011 bash[31774]: cluster 2021-03-10T17:03:24.835072+0000 mgr.gibba032.whopeo (mgr.24625) 10052 : cluster [DBG] pgmap v5725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:25 gibba011 bash[31774]: cephadm 2021-03-10T17:03:25.479189+0000 mgr.gibba032.whopeo (mgr.24625) 10053 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:27 gibba032 bash[24004]: cluster 2021-03-10T17:03:26.836780+0000 mgr.gibba032.whopeo (mgr.24625) 10054 : cluster [DBG] pgmap v5726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:27 gibba011 bash[30791]: cluster 2021-03-10T17:03:26.836780+0000 mgr.gibba032.whopeo (mgr.24625) 10054 : cluster [DBG] pgmap v5726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:27 gibba011 bash[31774]: cluster 2021-03-10T17:03:26.836780+0000 mgr.gibba032.whopeo (mgr.24625) 10054 : cluster [DBG] pgmap v5726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:29 gibba032 bash[24004]: cluster 2021-03-10T17:03:28.838267+0000 mgr.gibba032.whopeo (mgr.24625) 10055 : cluster [DBG] pgmap v5727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:29 gibba011 bash[30791]: cluster 2021-03-10T17:03:28.838267+0000 mgr.gibba032.whopeo (mgr.24625) 10055 : cluster [DBG] pgmap v5727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:29 gibba011 bash[31774]: cluster 2021-03-10T17:03:28.838267+0000 mgr.gibba032.whopeo (mgr.24625) 10055 : cluster [DBG] pgmap v5727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:31.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:31 gibba011 bash[30791]: cluster 2021-03-10T17:03:30.838853+0000 mgr.gibba032.whopeo (mgr.24625) 10056 : cluster [DBG] pgmap v5728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:31.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:31 gibba011 bash[31774]: cluster 2021-03-10T17:03:30.838853+0000 mgr.gibba032.whopeo (mgr.24625) 10056 : cluster [DBG] pgmap v5728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:32.006 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:31 gibba032 bash[24004]: cluster 2021-03-10T17:03:30.838853+0000 mgr.gibba032.whopeo (mgr.24625) 10056 : cluster [DBG] pgmap v5728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:33 gibba032 bash[24004]: cluster 2021-03-10T17:03:32.840433+0000 mgr.gibba032.whopeo (mgr.24625) 10057 : cluster [DBG] pgmap v5729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:33 gibba011 bash[30791]: cluster 2021-03-10T17:03:32.840433+0000 mgr.gibba032.whopeo (mgr.24625) 10057 : cluster [DBG] pgmap v5729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:33 gibba011 bash[31774]: cluster 2021-03-10T17:03:32.840433+0000 mgr.gibba032.whopeo (mgr.24625) 10057 : cluster [DBG] pgmap v5729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:35 gibba032 bash[24004]: cluster 2021-03-10T17:03:34.841700+0000 mgr.gibba032.whopeo (mgr.24625) 10058 : cluster [DBG] pgmap v5730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:35 gibba011 bash[30791]: cluster 2021-03-10T17:03:34.841700+0000 mgr.gibba032.whopeo (mgr.24625) 10058 : cluster [DBG] pgmap v5730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:35 gibba011 bash[31774]: cluster 2021-03-10T17:03:34.841700+0000 mgr.gibba032.whopeo (mgr.24625) 10058 : cluster [DBG] pgmap v5730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:37 gibba032 bash[24004]: cluster 2021-03-10T17:03:36.843457+0000 mgr.gibba032.whopeo (mgr.24625) 10059 : cluster [DBG] pgmap v5731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:37 gibba011 bash[30791]: cluster 2021-03-10T17:03:36.843457+0000 mgr.gibba032.whopeo (mgr.24625) 10059 : cluster [DBG] pgmap v5731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:37 gibba011 bash[31774]: cluster 2021-03-10T17:03:36.843457+0000 mgr.gibba032.whopeo (mgr.24625) 10059 : cluster [DBG] pgmap v5731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:39 gibba032 bash[24004]: cluster 2021-03-10T17:03:38.844305+0000 mgr.gibba032.whopeo (mgr.24625) 10060 : cluster [DBG] pgmap v5732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:39 gibba011 bash[30791]: cluster 2021-03-10T17:03:38.844305+0000 mgr.gibba032.whopeo (mgr.24625) 10060 : cluster [DBG] pgmap v5732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:39 gibba011 bash[31774]: cluster 2021-03-10T17:03:38.844305+0000 mgr.gibba032.whopeo (mgr.24625) 10060 : cluster [DBG] pgmap v5732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:40 gibba032 bash[24004]: cephadm 2021-03-10T17:03:40.492464+0000 mgr.gibba032.whopeo (mgr.24625) 10061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:40 gibba011 bash[30791]: cephadm 2021-03-10T17:03:40.492464+0000 mgr.gibba032.whopeo (mgr.24625) 10061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:40 gibba011 bash[31774]: cephadm 2021-03-10T17:03:40.492464+0000 mgr.gibba032.whopeo (mgr.24625) 10061 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:41 gibba032 bash[24004]: cluster 2021-03-10T17:03:40.845563+0000 mgr.gibba032.whopeo (mgr.24625) 10062 : cluster [DBG] pgmap v5733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:42.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:41 gibba011 bash[30791]: cluster 2021-03-10T17:03:40.845563+0000 mgr.gibba032.whopeo (mgr.24625) 10062 : cluster [DBG] pgmap v5733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:41 gibba011 bash[31774]: cluster 2021-03-10T17:03:40.845563+0000 mgr.gibba032.whopeo (mgr.24625) 10062 : cluster [DBG] pgmap v5733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:43 gibba032 bash[24004]: cluster 2021-03-10T17:03:42.847343+0000 mgr.gibba032.whopeo (mgr.24625) 10063 : cluster [DBG] pgmap v5734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:43 gibba011 bash[30791]: cluster 2021-03-10T17:03:42.847343+0000 mgr.gibba032.whopeo (mgr.24625) 10063 : cluster [DBG] pgmap v5734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:43 gibba011 bash[31774]: cluster 2021-03-10T17:03:42.847343+0000 mgr.gibba032.whopeo (mgr.24625) 10063 : cluster [DBG] pgmap v5734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:45 gibba032 bash[24004]: cluster 2021-03-10T17:03:44.848871+0000 mgr.gibba032.whopeo (mgr.24625) 10064 : cluster [DBG] pgmap v5735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:45 gibba011 bash[30791]: cluster 2021-03-10T17:03:44.848871+0000 mgr.gibba032.whopeo (mgr.24625) 10064 : cluster [DBG] pgmap v5735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:45 gibba011 bash[31774]: cluster 2021-03-10T17:03:44.848871+0000 mgr.gibba032.whopeo (mgr.24625) 10064 : cluster [DBG] pgmap v5735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:47 gibba032 bash[24004]: cluster 2021-03-10T17:03:46.850399+0000 mgr.gibba032.whopeo (mgr.24625) 10065 : cluster [DBG] pgmap v5736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:47 gibba011 bash[30791]: cluster 2021-03-10T17:03:46.850399+0000 mgr.gibba032.whopeo (mgr.24625) 10065 : cluster [DBG] pgmap v5736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:47 gibba011 bash[31774]: cluster 2021-03-10T17:03:46.850399+0000 mgr.gibba032.whopeo (mgr.24625) 10065 : cluster [DBG] pgmap v5736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:49 gibba032 bash[24004]: cluster 2021-03-10T17:03:48.851931+0000 mgr.gibba032.whopeo (mgr.24625) 10066 : cluster [DBG] pgmap v5737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:49 gibba011 bash[30791]: cluster 2021-03-10T17:03:48.851931+0000 mgr.gibba032.whopeo (mgr.24625) 10066 : cluster [DBG] pgmap v5737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:49 gibba011 bash[31774]: cluster 2021-03-10T17:03:48.851931+0000 mgr.gibba032.whopeo (mgr.24625) 10066 : cluster [DBG] pgmap v5737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:51.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:51 gibba011 bash[30791]: cluster 2021-03-10T17:03:50.853304+0000 mgr.gibba032.whopeo (mgr.24625) 10067 : cluster [DBG] pgmap v5738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:51 gibba011 bash[31774]: cluster 2021-03-10T17:03:50.853304+0000 mgr.gibba032.whopeo (mgr.24625) 10067 : cluster [DBG] pgmap v5738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:52.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:51 gibba032 bash[24004]: cluster 2021-03-10T17:03:50.853304+0000 mgr.gibba032.whopeo (mgr.24625) 10067 : cluster [DBG] pgmap v5738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:53 gibba032 bash[24004]: cluster 2021-03-10T17:03:52.854950+0000 mgr.gibba032.whopeo (mgr.24625) 10068 : cluster [DBG] pgmap v5739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:53 gibba011 bash[30791]: cluster 2021-03-10T17:03:52.854950+0000 mgr.gibba032.whopeo (mgr.24625) 10068 : cluster [DBG] pgmap v5739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:53 gibba011 bash[31774]: cluster 2021-03-10T17:03:52.854950+0000 mgr.gibba032.whopeo (mgr.24625) 10068 : cluster [DBG] pgmap v5739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:54.385 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:03:54.613 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:03:54.613 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:03:54.613 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 104s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:03:54.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:03:54.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:03:54.614 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:03:54.614 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:03:54.615 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:03:54.615 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:03:54.615 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:03:54.615 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 104s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:03:54.615 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:03:54.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:03:54.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:03:54.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:03:54.616 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:03:54.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:03:54.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:03:54.617 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 104s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:03:54.617 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 104s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:03:54.617 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:03:54.863 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:03:54.863 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:03:54.864 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:03:54.864 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:54.865 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:03:54.865 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:03:54.865 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:54.866 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:03:54.866 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:03:54.866 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:03:54.867 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:03:54.867 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:03:54.867 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:03:54.867 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:03:54.868 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:03:55.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:54 gibba032 bash[24004]: audit 2021-03-10T17:03:54.375288+0000 mgr.gibba032.whopeo (mgr.24625) 10069 : audit [DBG] from='client.60787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:55.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:54 gibba032 bash[24004]: audit 2021-03-10T17:03:54.863819+0000 mon.a (mon.0) 3389 : audit [DBG] from='client.? 172.21.2.111:0/2048092287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:54 gibba011 bash[30791]: audit 2021-03-10T17:03:54.375288+0000 mgr.gibba032.whopeo (mgr.24625) 10069 : audit [DBG] from='client.60787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:54 gibba011 bash[30791]: audit 2021-03-10T17:03:54.863819+0000 mon.a (mon.0) 3389 : audit [DBG] from='client.? 172.21.2.111:0/2048092287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:54 gibba011 bash[31774]: audit 2021-03-10T17:03:54.375288+0000 mgr.gibba032.whopeo (mgr.24625) 10069 : audit [DBG] from='client.60787 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:54 gibba011 bash[31774]: audit 2021-03-10T17:03:54.863819+0000 mon.a (mon.0) 3389 : audit [DBG] from='client.? 172.21.2.111:0/2048092287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:03:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:55 gibba032 bash[24004]: audit 2021-03-10T17:03:54.599440+0000 mgr.gibba032.whopeo (mgr.24625) 10070 : audit [DBG] from='client.60793 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:55 gibba032 bash[24004]: cluster 2021-03-10T17:03:54.855764+0000 mgr.gibba032.whopeo (mgr.24625) 10071 : cluster [DBG] pgmap v5740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:55 gibba032 bash[24004]: cephadm 2021-03-10T17:03:55.508592+0000 mgr.gibba032.whopeo (mgr.24625) 10072 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[30791]: audit 2021-03-10T17:03:54.599440+0000 mgr.gibba032.whopeo (mgr.24625) 10070 : audit [DBG] from='client.60793 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[30791]: cluster 2021-03-10T17:03:54.855764+0000 mgr.gibba032.whopeo (mgr.24625) 10071 : cluster [DBG] pgmap v5740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[30791]: cephadm 2021-03-10T17:03:55.508592+0000 mgr.gibba032.whopeo (mgr.24625) 10072 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[31774]: audit 2021-03-10T17:03:54.599440+0000 mgr.gibba032.whopeo (mgr.24625) 10070 : audit [DBG] from='client.60793 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:03:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[31774]: cluster 2021-03-10T17:03:54.855764+0000 mgr.gibba032.whopeo (mgr.24625) 10071 : cluster [DBG] pgmap v5740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:55 gibba011 bash[31774]: cephadm 2021-03-10T17:03:55.508592+0000 mgr.gibba032.whopeo (mgr.24625) 10072 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:03:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: cluster 2021-03-10T17:03:56.857475+0000 mgr.gibba032.whopeo (mgr.24625) 10073 : cluster [DBG] pgmap v5741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.299213+0000 mon.a (mon.0) 3390 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.299982+0000 mon.a (mon.0) 3391 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.450414+0000 mon.a (mon.0) 3392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.451297+0000 mon.a (mon.0) 3393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.452112+0000 mon.a (mon.0) 3394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:57 gibba032 bash[24004]: audit 2021-03-10T17:03:57.452917+0000 mon.a (mon.0) 3395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:03:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: cluster 2021-03-10T17:03:56.857475+0000 mgr.gibba032.whopeo (mgr.24625) 10073 : cluster [DBG] pgmap v5741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.299213+0000 mon.a (mon.0) 3390 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:03:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.299982+0000 mon.a (mon.0) 3391 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:03:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.450414+0000 mon.a (mon.0) 3392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:03:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.451297+0000 mon.a (mon.0) 3393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.452112+0000 mon.a (mon.0) 3394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[30791]: audit 2021-03-10T17:03:57.452917+0000 mon.a (mon.0) 3395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:03:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: cluster 2021-03-10T17:03:56.857475+0000 mgr.gibba032.whopeo (mgr.24625) 10073 : cluster [DBG] pgmap v5741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:03:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.299213+0000 mon.a (mon.0) 3390 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:03:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.299982+0000 mon.a (mon.0) 3391 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:03:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.450414+0000 mon.a (mon.0) 3392 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:03:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.451297+0000 mon.a (mon.0) 3393 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.452112+0000 mon.a (mon.0) 3394 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:03:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:57 gibba011 bash[31774]: audit 2021-03-10T17:03:57.452917+0000 mon.a (mon.0) 3395 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:04:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:03:59 gibba032 bash[24004]: cluster 2021-03-10T17:03:58.859120+0000 mgr.gibba032.whopeo (mgr.24625) 10074 : cluster [DBG] pgmap v5742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:03:59 gibba011 bash[30791]: cluster 2021-03-10T17:03:58.859120+0000 mgr.gibba032.whopeo (mgr.24625) 10074 : cluster [DBG] pgmap v5742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:03:59 gibba011 bash[31774]: cluster 2021-03-10T17:03:58.859120+0000 mgr.gibba032.whopeo (mgr.24625) 10074 : cluster [DBG] pgmap v5742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:01.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:01 gibba011 bash[30791]: cluster 2021-03-10T17:04:00.860501+0000 mgr.gibba032.whopeo (mgr.24625) 10075 : cluster [DBG] pgmap v5743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:01.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:01 gibba011 bash[31774]: cluster 2021-03-10T17:04:00.860501+0000 mgr.gibba032.whopeo (mgr.24625) 10075 : cluster [DBG] pgmap v5743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:01 gibba032 bash[24004]: cluster 2021-03-10T17:04:00.860501+0000 mgr.gibba032.whopeo (mgr.24625) 10075 : cluster [DBG] pgmap v5743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:03 gibba032 bash[24004]: cluster 2021-03-10T17:04:02.861494+0000 mgr.gibba032.whopeo (mgr.24625) 10076 : cluster [DBG] pgmap v5744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:03 gibba011 bash[30791]: cluster 2021-03-10T17:04:02.861494+0000 mgr.gibba032.whopeo (mgr.24625) 10076 : cluster [DBG] pgmap v5744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:03 gibba011 bash[31774]: cluster 2021-03-10T17:04:02.861494+0000 mgr.gibba032.whopeo (mgr.24625) 10076 : cluster [DBG] pgmap v5744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:05 gibba032 bash[24004]: cluster 2021-03-10T17:04:04.862660+0000 mgr.gibba032.whopeo (mgr.24625) 10077 : cluster [DBG] pgmap v5745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:05 gibba011 bash[30791]: cluster 2021-03-10T17:04:04.862660+0000 mgr.gibba032.whopeo (mgr.24625) 10077 : cluster [DBG] pgmap v5745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:05 gibba011 bash[31774]: cluster 2021-03-10T17:04:04.862660+0000 mgr.gibba032.whopeo (mgr.24625) 10077 : cluster [DBG] pgmap v5745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:07 gibba032 bash[24004]: cluster 2021-03-10T17:04:06.864242+0000 mgr.gibba032.whopeo (mgr.24625) 10078 : cluster [DBG] pgmap v5746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:08.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:07 gibba011 bash[30791]: cluster 2021-03-10T17:04:06.864242+0000 mgr.gibba032.whopeo (mgr.24625) 10078 : cluster [DBG] pgmap v5746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:07 gibba011 bash[31774]: cluster 2021-03-10T17:04:06.864242+0000 mgr.gibba032.whopeo (mgr.24625) 10078 : cluster [DBG] pgmap v5746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:09 gibba032 bash[24004]: cluster 2021-03-10T17:04:08.865872+0000 mgr.gibba032.whopeo (mgr.24625) 10079 : cluster [DBG] pgmap v5747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:09 gibba011 bash[30791]: cluster 2021-03-10T17:04:08.865872+0000 mgr.gibba032.whopeo (mgr.24625) 10079 : cluster [DBG] pgmap v5747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:09 gibba011 bash[31774]: cluster 2021-03-10T17:04:08.865872+0000 mgr.gibba032.whopeo (mgr.24625) 10079 : cluster [DBG] pgmap v5747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.524512+0000 mgr.gibba032.whopeo (mgr.24625) 10080 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:04:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.524997+0000 mgr.gibba032.whopeo (mgr.24625) 10081 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:04:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.721632+0000 mon.a (mon.0) 3396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:04:11.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.725004+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:04:11.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.727743+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.730196+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.732495+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.737035+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:10 gibba032 bash[24004]: audit 2021-03-10T17:04:10.739435+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.524512+0000 mgr.gibba032.whopeo (mgr.24625) 10080 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:04:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.524997+0000 mgr.gibba032.whopeo (mgr.24625) 10081 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:04:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.721632+0000 mon.a (mon.0) 3396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:04:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.725004+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:04:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.727743+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.730196+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.732495+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.737035+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[30791]: audit 2021-03-10T17:04:10.739435+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.524512+0000 mgr.gibba032.whopeo (mgr.24625) 10080 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:04:11.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.524997+0000 mgr.gibba032.whopeo (mgr.24625) 10081 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:04:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.721632+0000 mon.a (mon.0) 3396 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:04:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.725004+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:04:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.727743+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.730196+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.732495+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.737035+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:11.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:10 gibba011 bash[31774]: audit 2021-03-10T17:04:10.739435+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.722344+0000 mgr.gibba032.whopeo (mgr.24625) 10082 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:04:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.725414+0000 mgr.gibba032.whopeo (mgr.24625) 10083 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:04:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.727742+0000 mgr.gibba032.whopeo (mgr.24625) 10084 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:04:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.727782+0000 mgr.gibba032.whopeo (mgr.24625) 10085 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:04:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.730247+0000 mgr.gibba032.whopeo (mgr.24625) 10086 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:04:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.730284+0000 mgr.gibba032.whopeo (mgr.24625) 10087 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:04:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.732303+0000 mgr.gibba032.whopeo (mgr.24625) 10088 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:04:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.734577+0000 mgr.gibba032.whopeo (mgr.24625) 10089 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:04:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.734624+0000 mgr.gibba032.whopeo (mgr.24625) 10090 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:04:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.736770+0000 mgr.gibba032.whopeo (mgr.24625) 10091 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:04:12.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.737186+0000 mgr.gibba032.whopeo (mgr.24625) 10092 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:04:12.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.739395+0000 mgr.gibba032.whopeo (mgr.24625) 10093 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:04:12.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.739435+0000 mgr.gibba032.whopeo (mgr.24625) 10094 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:04:12.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cephadm 2021-03-10T17:04:10.739567+0000 mgr.gibba032.whopeo (mgr.24625) 10095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:12.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:11 gibba032 bash[24004]: cluster 2021-03-10T17:04:10.866780+0000 mgr.gibba032.whopeo (mgr.24625) 10096 : cluster [DBG] pgmap v5748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.722344+0000 mgr.gibba032.whopeo (mgr.24625) 10082 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:04:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.725414+0000 mgr.gibba032.whopeo (mgr.24625) 10083 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:04:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.727742+0000 mgr.gibba032.whopeo (mgr.24625) 10084 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:04:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.727782+0000 mgr.gibba032.whopeo (mgr.24625) 10085 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:04:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.730247+0000 mgr.gibba032.whopeo (mgr.24625) 10086 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:04:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.730284+0000 mgr.gibba032.whopeo (mgr.24625) 10087 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:04:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.732303+0000 mgr.gibba032.whopeo (mgr.24625) 10088 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:04:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.734577+0000 mgr.gibba032.whopeo (mgr.24625) 10089 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:04:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.734624+0000 mgr.gibba032.whopeo (mgr.24625) 10090 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:04:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.736770+0000 mgr.gibba032.whopeo (mgr.24625) 10091 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:04:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.737186+0000 mgr.gibba032.whopeo (mgr.24625) 10092 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:04:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.739395+0000 mgr.gibba032.whopeo (mgr.24625) 10093 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:04:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.739435+0000 mgr.gibba032.whopeo (mgr.24625) 10094 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:04:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cephadm 2021-03-10T17:04:10.739567+0000 mgr.gibba032.whopeo (mgr.24625) 10095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[30791]: cluster 2021-03-10T17:04:10.866780+0000 mgr.gibba032.whopeo (mgr.24625) 10096 : cluster [DBG] pgmap v5748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.722344+0000 mgr.gibba032.whopeo (mgr.24625) 10082 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:04:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.725414+0000 mgr.gibba032.whopeo (mgr.24625) 10083 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:04:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.727742+0000 mgr.gibba032.whopeo (mgr.24625) 10084 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:04:12.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.727782+0000 mgr.gibba032.whopeo (mgr.24625) 10085 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:04:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.730247+0000 mgr.gibba032.whopeo (mgr.24625) 10086 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:04:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.730284+0000 mgr.gibba032.whopeo (mgr.24625) 10087 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:04:12.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.732303+0000 mgr.gibba032.whopeo (mgr.24625) 10088 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:04:12.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.734577+0000 mgr.gibba032.whopeo (mgr.24625) 10089 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:04:12.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.734624+0000 mgr.gibba032.whopeo (mgr.24625) 10090 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:04:12.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.736770+0000 mgr.gibba032.whopeo (mgr.24625) 10091 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:04:12.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.737186+0000 mgr.gibba032.whopeo (mgr.24625) 10092 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:04:12.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.739395+0000 mgr.gibba032.whopeo (mgr.24625) 10093 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:04:12.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.739435+0000 mgr.gibba032.whopeo (mgr.24625) 10094 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:04:12.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cephadm 2021-03-10T17:04:10.739567+0000 mgr.gibba032.whopeo (mgr.24625) 10095 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:12.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:11 gibba011 bash[31774]: cluster 2021-03-10T17:04:10.866780+0000 mgr.gibba032.whopeo (mgr.24625) 10096 : cluster [DBG] pgmap v5748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:13 gibba032 bash[24004]: cluster 2021-03-10T17:04:12.868572+0000 mgr.gibba032.whopeo (mgr.24625) 10097 : cluster [DBG] pgmap v5749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:13 gibba011 bash[30791]: cluster 2021-03-10T17:04:12.868572+0000 mgr.gibba032.whopeo (mgr.24625) 10097 : cluster [DBG] pgmap v5749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:13 gibba011 bash[31774]: cluster 2021-03-10T17:04:12.868572+0000 mgr.gibba032.whopeo (mgr.24625) 10097 : cluster [DBG] pgmap v5749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:16.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:15 gibba032 bash[24004]: cluster 2021-03-10T17:04:14.870096+0000 mgr.gibba032.whopeo (mgr.24625) 10098 : cluster [DBG] pgmap v5750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:15 gibba011 bash[30791]: cluster 2021-03-10T17:04:14.870096+0000 mgr.gibba032.whopeo (mgr.24625) 10098 : cluster [DBG] pgmap v5750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:16.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:15 gibba011 bash[31774]: cluster 2021-03-10T17:04:14.870096+0000 mgr.gibba032.whopeo (mgr.24625) 10098 : cluster [DBG] pgmap v5750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:18.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:17 gibba032 bash[24004]: cluster 2021-03-10T17:04:16.871733+0000 mgr.gibba032.whopeo (mgr.24625) 10099 : cluster [DBG] pgmap v5751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:17 gibba011 bash[30791]: cluster 2021-03-10T17:04:16.871733+0000 mgr.gibba032.whopeo (mgr.24625) 10099 : cluster [DBG] pgmap v5751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:17 gibba011 bash[31774]: cluster 2021-03-10T17:04:16.871733+0000 mgr.gibba032.whopeo (mgr.24625) 10099 : cluster [DBG] pgmap v5751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:20.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:19 gibba032 bash[24004]: cluster 2021-03-10T17:04:18.872614+0000 mgr.gibba032.whopeo (mgr.24625) 10100 : cluster [DBG] pgmap v5752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:20.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:19 gibba011 bash[30791]: cluster 2021-03-10T17:04:18.872614+0000 mgr.gibba032.whopeo (mgr.24625) 10100 : cluster [DBG] pgmap v5752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:19 gibba011 bash[31774]: cluster 2021-03-10T17:04:18.872614+0000 mgr.gibba032.whopeo (mgr.24625) 10100 : cluster [DBG] pgmap v5752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:21 gibba032 bash[24004]: cluster 2021-03-10T17:04:20.873860+0000 mgr.gibba032.whopeo (mgr.24625) 10101 : cluster [DBG] pgmap v5753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:21 gibba011 bash[30791]: cluster 2021-03-10T17:04:20.873860+0000 mgr.gibba032.whopeo (mgr.24625) 10101 : cluster [DBG] pgmap v5753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:22.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:21 gibba011 bash[31774]: cluster 2021-03-10T17:04:20.873860+0000 mgr.gibba032.whopeo (mgr.24625) 10101 : cluster [DBG] pgmap v5753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:23 gibba032 bash[24004]: cluster 2021-03-10T17:04:22.875536+0000 mgr.gibba032.whopeo (mgr.24625) 10102 : cluster [DBG] pgmap v5754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:23 gibba011 bash[30791]: cluster 2021-03-10T17:04:22.875536+0000 mgr.gibba032.whopeo (mgr.24625) 10102 : cluster [DBG] pgmap v5754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:23 gibba011 bash[31774]: cluster 2021-03-10T17:04:22.875536+0000 mgr.gibba032.whopeo (mgr.24625) 10102 : cluster [DBG] pgmap v5754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:25.106 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:04:25.363 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:04:25.363 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:04:25.364 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:04:25.364 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:04:25.364 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:04:25.364 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:04:25.364 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:04:25.365 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:04:25.365 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:04:25.365 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:04:25.365 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:04:25.365 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:04:25.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:04:25.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:04:25.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:04:25.366 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:04:25.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:04:25.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:04:25.367 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:04:25.367 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:04:25.367 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:04:25.619 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:04:25.620 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:04:25.620 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:04:25.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:04:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:04:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:04:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:04:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:04:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:04:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:04:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:04:25.623 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:04:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:25 gibba032 bash[24004]: cluster 2021-03-10T17:04:24.877055+0000 mgr.gibba032.whopeo (mgr.24625) 10103 : cluster [DBG] pgmap v5755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:25 gibba032 bash[24004]: audit 2021-03-10T17:04:25.098025+0000 mgr.gibba032.whopeo (mgr.24625) 10104 : audit [DBG] from='client.50823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:25 gibba032 bash[24004]: audit 2021-03-10T17:04:25.321785+0000 mgr.gibba032.whopeo (mgr.24625) 10105 : audit [DBG] from='client.50829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:25 gibba032 bash[24004]: audit 2021-03-10T17:04:25.620261+0000 mon.a (mon.0) 3403 : audit [DBG] from='client.? 172.21.2.111:0/3037439706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[30791]: cluster 2021-03-10T17:04:24.877055+0000 mgr.gibba032.whopeo (mgr.24625) 10103 : cluster [DBG] pgmap v5755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[30791]: audit 2021-03-10T17:04:25.098025+0000 mgr.gibba032.whopeo (mgr.24625) 10104 : audit [DBG] from='client.50823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[30791]: audit 2021-03-10T17:04:25.321785+0000 mgr.gibba032.whopeo (mgr.24625) 10105 : audit [DBG] from='client.50829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[30791]: audit 2021-03-10T17:04:25.620261+0000 mon.a (mon.0) 3403 : audit [DBG] from='client.? 172.21.2.111:0/3037439706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[31774]: cluster 2021-03-10T17:04:24.877055+0000 mgr.gibba032.whopeo (mgr.24625) 10103 : cluster [DBG] pgmap v5755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[31774]: audit 2021-03-10T17:04:25.098025+0000 mgr.gibba032.whopeo (mgr.24625) 10104 : audit [DBG] from='client.50823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[31774]: audit 2021-03-10T17:04:25.321785+0000 mgr.gibba032.whopeo (mgr.24625) 10105 : audit [DBG] from='client.50829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:25 gibba011 bash[31774]: audit 2021-03-10T17:04:25.620261+0000 mon.a (mon.0) 3403 : audit [DBG] from='client.? 172.21.2.111:0/3037439706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:27.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:26 gibba032 bash[24004]: cephadm 2021-03-10T17:04:25.755312+0000 mgr.gibba032.whopeo (mgr.24625) 10106 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:27.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:26 gibba011 bash[30791]: cephadm 2021-03-10T17:04:25.755312+0000 mgr.gibba032.whopeo (mgr.24625) 10106 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:26 gibba011 bash[31774]: cephadm 2021-03-10T17:04:25.755312+0000 mgr.gibba032.whopeo (mgr.24625) 10106 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:27 gibba032 bash[24004]: cluster 2021-03-10T17:04:26.878139+0000 mgr.gibba032.whopeo (mgr.24625) 10107 : cluster [DBG] pgmap v5756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:27 gibba011 bash[30791]: cluster 2021-03-10T17:04:26.878139+0000 mgr.gibba032.whopeo (mgr.24625) 10107 : cluster [DBG] pgmap v5756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:27 gibba011 bash[31774]: cluster 2021-03-10T17:04:26.878139+0000 mgr.gibba032.whopeo (mgr.24625) 10107 : cluster [DBG] pgmap v5756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:30.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:29 gibba032 bash[24004]: cluster 2021-03-10T17:04:28.879597+0000 mgr.gibba032.whopeo (mgr.24625) 10108 : cluster [DBG] pgmap v5757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:30.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:29 gibba011 bash[30791]: cluster 2021-03-10T17:04:28.879597+0000 mgr.gibba032.whopeo (mgr.24625) 10108 : cluster [DBG] pgmap v5757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:29 gibba011 bash[31774]: cluster 2021-03-10T17:04:28.879597+0000 mgr.gibba032.whopeo (mgr.24625) 10108 : cluster [DBG] pgmap v5757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:31 gibba032 bash[24004]: cluster 2021-03-10T17:04:30.880966+0000 mgr.gibba032.whopeo (mgr.24625) 10109 : cluster [DBG] pgmap v5758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:31 gibba011 bash[30791]: cluster 2021-03-10T17:04:30.880966+0000 mgr.gibba032.whopeo (mgr.24625) 10109 : cluster [DBG] pgmap v5758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:31 gibba011 bash[31774]: cluster 2021-03-10T17:04:30.880966+0000 mgr.gibba032.whopeo (mgr.24625) 10109 : cluster [DBG] pgmap v5758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:33 gibba032 bash[24004]: cluster 2021-03-10T17:04:32.882753+0000 mgr.gibba032.whopeo (mgr.24625) 10110 : cluster [DBG] pgmap v5759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:34.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:33 gibba011 bash[30791]: cluster 2021-03-10T17:04:32.882753+0000 mgr.gibba032.whopeo (mgr.24625) 10110 : cluster [DBG] pgmap v5759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:33 gibba011 bash[31774]: cluster 2021-03-10T17:04:32.882753+0000 mgr.gibba032.whopeo (mgr.24625) 10110 : cluster [DBG] pgmap v5759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:36.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:35 gibba032 bash[24004]: cluster 2021-03-10T17:04:34.883794+0000 mgr.gibba032.whopeo (mgr.24625) 10111 : cluster [DBG] pgmap v5760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:35 gibba011 bash[30791]: cluster 2021-03-10T17:04:34.883794+0000 mgr.gibba032.whopeo (mgr.24625) 10111 : cluster [DBG] pgmap v5760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:35 gibba011 bash[31774]: cluster 2021-03-10T17:04:34.883794+0000 mgr.gibba032.whopeo (mgr.24625) 10111 : cluster [DBG] pgmap v5760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:38.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:37 gibba032 bash[24004]: cluster 2021-03-10T17:04:36.885452+0000 mgr.gibba032.whopeo (mgr.24625) 10112 : cluster [DBG] pgmap v5761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:37 gibba011 bash[30791]: cluster 2021-03-10T17:04:36.885452+0000 mgr.gibba032.whopeo (mgr.24625) 10112 : cluster [DBG] pgmap v5761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:37 gibba011 bash[31774]: cluster 2021-03-10T17:04:36.885452+0000 mgr.gibba032.whopeo (mgr.24625) 10112 : cluster [DBG] pgmap v5761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:40.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:39 gibba032 bash[24004]: cluster 2021-03-10T17:04:38.887055+0000 mgr.gibba032.whopeo (mgr.24625) 10113 : cluster [DBG] pgmap v5762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:39 gibba011 bash[30791]: cluster 2021-03-10T17:04:38.887055+0000 mgr.gibba032.whopeo (mgr.24625) 10113 : cluster [DBG] pgmap v5762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:39 gibba011 bash[31774]: cluster 2021-03-10T17:04:38.887055+0000 mgr.gibba032.whopeo (mgr.24625) 10113 : cluster [DBG] pgmap v5762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:41 gibba032 bash[24004]: cephadm 2021-03-10T17:04:40.769129+0000 mgr.gibba032.whopeo (mgr.24625) 10114 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:41 gibba032 bash[24004]: cluster 2021-03-10T17:04:40.888440+0000 mgr.gibba032.whopeo (mgr.24625) 10115 : cluster [DBG] pgmap v5763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:41 gibba011 bash[30791]: cephadm 2021-03-10T17:04:40.769129+0000 mgr.gibba032.whopeo (mgr.24625) 10114 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:41 gibba011 bash[30791]: cluster 2021-03-10T17:04:40.888440+0000 mgr.gibba032.whopeo (mgr.24625) 10115 : cluster [DBG] pgmap v5763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:41 gibba011 bash[31774]: cephadm 2021-03-10T17:04:40.769129+0000 mgr.gibba032.whopeo (mgr.24625) 10114 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:41 gibba011 bash[31774]: cluster 2021-03-10T17:04:40.888440+0000 mgr.gibba032.whopeo (mgr.24625) 10115 : cluster [DBG] pgmap v5763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:43 gibba032 bash[24004]: cluster 2021-03-10T17:04:42.889734+0000 mgr.gibba032.whopeo (mgr.24625) 10116 : cluster [DBG] pgmap v5764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:43 gibba011 bash[30791]: cluster 2021-03-10T17:04:42.889734+0000 mgr.gibba032.whopeo (mgr.24625) 10116 : cluster [DBG] pgmap v5764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:43 gibba011 bash[31774]: cluster 2021-03-10T17:04:42.889734+0000 mgr.gibba032.whopeo (mgr.24625) 10116 : cluster [DBG] pgmap v5764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:46.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:45 gibba032 bash[24004]: cluster 2021-03-10T17:04:44.891227+0000 mgr.gibba032.whopeo (mgr.24625) 10117 : cluster [DBG] pgmap v5765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:45 gibba011 bash[30791]: cluster 2021-03-10T17:04:44.891227+0000 mgr.gibba032.whopeo (mgr.24625) 10117 : cluster [DBG] pgmap v5765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:45 gibba011 bash[31774]: cluster 2021-03-10T17:04:44.891227+0000 mgr.gibba032.whopeo (mgr.24625) 10117 : cluster [DBG] pgmap v5765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:48.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:47 gibba032 bash[24004]: cluster 2021-03-10T17:04:46.893033+0000 mgr.gibba032.whopeo (mgr.24625) 10118 : cluster [DBG] pgmap v5766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:47 gibba011 bash[30791]: cluster 2021-03-10T17:04:46.893033+0000 mgr.gibba032.whopeo (mgr.24625) 10118 : cluster [DBG] pgmap v5766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:47 gibba011 bash[31774]: cluster 2021-03-10T17:04:46.893033+0000 mgr.gibba032.whopeo (mgr.24625) 10118 : cluster [DBG] pgmap v5766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:50.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:49 gibba032 bash[24004]: cluster 2021-03-10T17:04:48.894656+0000 mgr.gibba032.whopeo (mgr.24625) 10119 : cluster [DBG] pgmap v5767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:49 gibba011 bash[30791]: cluster 2021-03-10T17:04:48.894656+0000 mgr.gibba032.whopeo (mgr.24625) 10119 : cluster [DBG] pgmap v5767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:49 gibba011 bash[31774]: cluster 2021-03-10T17:04:48.894656+0000 mgr.gibba032.whopeo (mgr.24625) 10119 : cluster [DBG] pgmap v5767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:52.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:51 gibba032 bash[24004]: cluster 2021-03-10T17:04:50.896248+0000 mgr.gibba032.whopeo (mgr.24625) 10120 : cluster [DBG] pgmap v5768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:52.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:51 gibba011 bash[30791]: cluster 2021-03-10T17:04:50.896248+0000 mgr.gibba032.whopeo (mgr.24625) 10120 : cluster [DBG] pgmap v5768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:51 gibba011 bash[31774]: cluster 2021-03-10T17:04:50.896248+0000 mgr.gibba032.whopeo (mgr.24625) 10120 : cluster [DBG] pgmap v5768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:54.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:53 gibba032 bash[24004]: cluster 2021-03-10T17:04:52.898179+0000 mgr.gibba032.whopeo (mgr.24625) 10121 : cluster [DBG] pgmap v5769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:53 gibba011 bash[30791]: cluster 2021-03-10T17:04:52.898179+0000 mgr.gibba032.whopeo (mgr.24625) 10121 : cluster [DBG] pgmap v5769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:53 gibba011 bash[31774]: cluster 2021-03-10T17:04:52.898179+0000 mgr.gibba032.whopeo (mgr.24625) 10121 : cluster [DBG] pgmap v5769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:55.860 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:04:56.090 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:04:56.090 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:04:56.091 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:04:56.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:04:56.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:04:56.091 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:04:56.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:04:56.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:04:56.092 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:04:56.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:04:56.092 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:04:56.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:04:56.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:04:56.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:04:56.093 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:04:56.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:04:56.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:04:56.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:04:56.094 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:04:56.094 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:04:56.095 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:04:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:55 gibba032 bash[24004]: cluster 2021-03-10T17:04:54.899742+0000 mgr.gibba032.whopeo (mgr.24625) 10122 : cluster [DBG] pgmap v5770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:55 gibba011 bash[30791]: cluster 2021-03-10T17:04:54.899742+0000 mgr.gibba032.whopeo (mgr.24625) 10122 : cluster [DBG] pgmap v5770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:56.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:55 gibba011 bash[31774]: cluster 2021-03-10T17:04:54.899742+0000 mgr.gibba032.whopeo (mgr.24625) 10122 : cluster [DBG] pgmap v5770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:56.337 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:04:56.337 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:04:56.338 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:04:56.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:56.338 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:04:56.338 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:04:56.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:56.339 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:04:56.339 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:04:56.339 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:04:56.339 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:04:56.339 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:04:56.340 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:04:56.340 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:04:56.340 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:04:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:56 gibba032 bash[24004]: cephadm 2021-03-10T17:04:55.770482+0000 mgr.gibba032.whopeo (mgr.24625) 10123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:56 gibba032 bash[24004]: audit 2021-03-10T17:04:55.849502+0000 mgr.gibba032.whopeo (mgr.24625) 10124 : audit [DBG] from='client.60823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:56 gibba032 bash[24004]: audit 2021-03-10T17:04:56.076525+0000 mgr.gibba032.whopeo (mgr.24625) 10125 : audit [DBG] from='client.60829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:56 gibba032 bash[24004]: audit 2021-03-10T17:04:56.337817+0000 mon.a (mon.0) 3404 : audit [DBG] from='client.? 172.21.2.111:0/1985925262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[30791]: cephadm 2021-03-10T17:04:55.770482+0000 mgr.gibba032.whopeo (mgr.24625) 10123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[30791]: audit 2021-03-10T17:04:55.849502+0000 mgr.gibba032.whopeo (mgr.24625) 10124 : audit [DBG] from='client.60823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[30791]: audit 2021-03-10T17:04:56.076525+0000 mgr.gibba032.whopeo (mgr.24625) 10125 : audit [DBG] from='client.60829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[30791]: audit 2021-03-10T17:04:56.337817+0000 mon.a (mon.0) 3404 : audit [DBG] from='client.? 172.21.2.111:0/1985925262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[31774]: cephadm 2021-03-10T17:04:55.770482+0000 mgr.gibba032.whopeo (mgr.24625) 10123 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:04:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[31774]: audit 2021-03-10T17:04:55.849502+0000 mgr.gibba032.whopeo (mgr.24625) 10124 : audit [DBG] from='client.60823 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[31774]: audit 2021-03-10T17:04:56.076525+0000 mgr.gibba032.whopeo (mgr.24625) 10125 : audit [DBG] from='client.60829 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:04:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:56 gibba011 bash[31774]: audit 2021-03-10T17:04:56.337817+0000 mon.a (mon.0) 3404 : audit [DBG] from='client.? 172.21.2.111:0/1985925262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:04:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: cluster 2021-03-10T17:04:56.901467+0000 mgr.gibba032.whopeo (mgr.24625) 10126 : cluster [DBG] pgmap v5771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:58.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.304821+0000 mon.a (mon.0) 3405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:04:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.305774+0000 mon.a (mon.0) 3406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:04:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.461632+0000 mon.a (mon.0) 3407 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:04:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.462742+0000 mon.a (mon.0) 3408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:04:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.463807+0000 mon.a (mon.0) 3409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:04:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:57 gibba032 bash[24004]: audit 2021-03-10T17:04:57.464848+0000 mon.a (mon.0) 3410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:04:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: cluster 2021-03-10T17:04:56.901467+0000 mgr.gibba032.whopeo (mgr.24625) 10126 : cluster [DBG] pgmap v5771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.304821+0000 mon.a (mon.0) 3405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:04:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.305774+0000 mon.a (mon.0) 3406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:04:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.461632+0000 mon.a (mon.0) 3407 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:04:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.462742+0000 mon.a (mon.0) 3408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:04:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.463807+0000 mon.a (mon.0) 3409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:04:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[30791]: audit 2021-03-10T17:04:57.464848+0000 mon.a (mon.0) 3410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:04:58.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: cluster 2021-03-10T17:04:56.901467+0000 mgr.gibba032.whopeo (mgr.24625) 10126 : cluster [DBG] pgmap v5771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:04:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.304821+0000 mon.a (mon.0) 3405 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:04:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.305774+0000 mon.a (mon.0) 3406 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:04:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.461632+0000 mon.a (mon.0) 3407 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:04:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.462742+0000 mon.a (mon.0) 3408 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:04:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.463807+0000 mon.a (mon.0) 3409 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:04:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:57 gibba011 bash[31774]: audit 2021-03-10T17:04:57.464848+0000 mon.a (mon.0) 3410 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:05:00.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:04:59 gibba032 bash[24004]: cluster 2021-03-10T17:04:58.903004+0000 mgr.gibba032.whopeo (mgr.24625) 10127 : cluster [DBG] pgmap v5772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:04:59 gibba011 bash[30791]: cluster 2021-03-10T17:04:58.903004+0000 mgr.gibba032.whopeo (mgr.24625) 10127 : cluster [DBG] pgmap v5772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:04:59 gibba011 bash[31774]: cluster 2021-03-10T17:04:58.903004+0000 mgr.gibba032.whopeo (mgr.24625) 10127 : cluster [DBG] pgmap v5772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:01 gibba032 bash[24004]: cluster 2021-03-10T17:05:00.904421+0000 mgr.gibba032.whopeo (mgr.24625) 10128 : cluster [DBG] pgmap v5773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:02.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:01 gibba011 bash[30791]: cluster 2021-03-10T17:05:00.904421+0000 mgr.gibba032.whopeo (mgr.24625) 10128 : cluster [DBG] pgmap v5773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:01 gibba011 bash[31774]: cluster 2021-03-10T17:05:00.904421+0000 mgr.gibba032.whopeo (mgr.24625) 10128 : cluster [DBG] pgmap v5773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:04.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:03 gibba032 bash[24004]: cluster 2021-03-10T17:05:02.905953+0000 mgr.gibba032.whopeo (mgr.24625) 10129 : cluster [DBG] pgmap v5774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:03 gibba011 bash[30791]: cluster 2021-03-10T17:05:02.905953+0000 mgr.gibba032.whopeo (mgr.24625) 10129 : cluster [DBG] pgmap v5774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:03 gibba011 bash[31774]: cluster 2021-03-10T17:05:02.905953+0000 mgr.gibba032.whopeo (mgr.24625) 10129 : cluster [DBG] pgmap v5774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:06.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:05 gibba032 bash[24004]: cluster 2021-03-10T17:05:04.907364+0000 mgr.gibba032.whopeo (mgr.24625) 10130 : cluster [DBG] pgmap v5775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:05 gibba011 bash[30791]: cluster 2021-03-10T17:05:04.907364+0000 mgr.gibba032.whopeo (mgr.24625) 10130 : cluster [DBG] pgmap v5775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:05 gibba011 bash[31774]: cluster 2021-03-10T17:05:04.907364+0000 mgr.gibba032.whopeo (mgr.24625) 10130 : cluster [DBG] pgmap v5775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:08.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:07 gibba032 bash[24004]: cluster 2021-03-10T17:05:06.909099+0000 mgr.gibba032.whopeo (mgr.24625) 10131 : cluster [DBG] pgmap v5776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:07 gibba011 bash[30791]: cluster 2021-03-10T17:05:06.909099+0000 mgr.gibba032.whopeo (mgr.24625) 10131 : cluster [DBG] pgmap v5776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:07 gibba011 bash[31774]: cluster 2021-03-10T17:05:06.909099+0000 mgr.gibba032.whopeo (mgr.24625) 10131 : cluster [DBG] pgmap v5776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:10.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:09 gibba032 bash[24004]: cluster 2021-03-10T17:05:08.910616+0000 mgr.gibba032.whopeo (mgr.24625) 10132 : cluster [DBG] pgmap v5777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:10.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:09 gibba011 bash[30791]: cluster 2021-03-10T17:05:08.910616+0000 mgr.gibba032.whopeo (mgr.24625) 10132 : cluster [DBG] pgmap v5777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:09 gibba011 bash[31774]: cluster 2021-03-10T17:05:08.910616+0000 mgr.gibba032.whopeo (mgr.24625) 10132 : cluster [DBG] pgmap v5777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:10.785553+0000 mgr.gibba032.whopeo (mgr.24625) 10133 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:05:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:10.787106+0000 mgr.gibba032.whopeo (mgr.24625) 10134 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:05:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cluster 2021-03-10T17:05:10.911075+0000 mgr.gibba032.whopeo (mgr.24625) 10135 : cluster [DBG] pgmap v5778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.039677+0000 mon.a (mon.0) 3411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:05:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.040444+0000 mgr.gibba032.whopeo (mgr.24625) 10136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:05:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.042223+0000 mgr.gibba032.whopeo (mgr.24625) 10137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:05:12.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.042893+0000 mon.a (mon.0) 3412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:05:12.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.043725+0000 mgr.gibba032.whopeo (mgr.24625) 10138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:05:12.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.043763+0000 mgr.gibba032.whopeo (mgr.24625) 10139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:05:12.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.044436+0000 mon.a (mon.0) 3413 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.045431+0000 mgr.gibba032.whopeo (mgr.24625) 10140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:05:12.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.045468+0000 mgr.gibba032.whopeo (mgr.24625) 10141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:05:12.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.045916+0000 mon.a (mon.0) 3414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.046838+0000 mgr.gibba032.whopeo (mgr.24625) 10142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:05:12.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.047615+0000 mon.a (mon.0) 3415 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.048015+0000 mgr.gibba032.whopeo (mgr.24625) 10143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:05:12.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.048170+0000 mgr.gibba032.whopeo (mgr.24625) 10144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:05:12.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.049448+0000 mgr.gibba032.whopeo (mgr.24625) 10145 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:05:12.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.049764+0000 mgr.gibba032.whopeo (mgr.24625) 10146 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:05:12.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.050397+0000 mon.a (mon.0) 3416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.050782+0000 mgr.gibba032.whopeo (mgr.24625) 10147 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:05:12.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:10.785553+0000 mgr.gibba032.whopeo (mgr.24625) 10133 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:05:12.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:10.787106+0000 mgr.gibba032.whopeo (mgr.24625) 10134 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:05:12.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cluster 2021-03-10T17:05:10.911075+0000 mgr.gibba032.whopeo (mgr.24625) 10135 : cluster [DBG] pgmap v5778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:12.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.039677+0000 mon.a (mon.0) 3411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:05:12.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.040444+0000 mgr.gibba032.whopeo (mgr.24625) 10136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:05:12.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.042223+0000 mgr.gibba032.whopeo (mgr.24625) 10137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:05:12.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.042893+0000 mon.a (mon.0) 3412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:05:12.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.043725+0000 mgr.gibba032.whopeo (mgr.24625) 10138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:05:12.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.043763+0000 mgr.gibba032.whopeo (mgr.24625) 10139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:05:12.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.044436+0000 mon.a (mon.0) 3413 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.045431+0000 mgr.gibba032.whopeo (mgr.24625) 10140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:05:12.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.045468+0000 mgr.gibba032.whopeo (mgr.24625) 10141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:05:12.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.045916+0000 mon.a (mon.0) 3414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.046838+0000 mgr.gibba032.whopeo (mgr.24625) 10142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:05:12.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.047615+0000 mon.a (mon.0) 3415 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.048015+0000 mgr.gibba032.whopeo (mgr.24625) 10143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:05:12.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.048170+0000 mgr.gibba032.whopeo (mgr.24625) 10144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:05:12.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.049448+0000 mgr.gibba032.whopeo (mgr.24625) 10145 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:05:12.347 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.049764+0000 mgr.gibba032.whopeo (mgr.24625) 10146 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:05:12.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.050397+0000 mon.a (mon.0) 3416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.050782+0000 mgr.gibba032.whopeo (mgr.24625) 10147 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:05:12.349 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.050821+0000 mgr.gibba032.whopeo (mgr.24625) 10148 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:05:12.349 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: cephadm 2021-03-10T17:05:11.050925+0000 mgr.gibba032.whopeo (mgr.24625) 10149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:12.350 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[30791]: audit 2021-03-10T17:05:11.051719+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.350 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.050821+0000 mgr.gibba032.whopeo (mgr.24625) 10148 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:05:12.351 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: cephadm 2021-03-10T17:05:11.050925+0000 mgr.gibba032.whopeo (mgr.24625) 10149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:12.351 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:12 gibba011 bash[31774]: audit 2021-03-10T17:05:11.051719+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:10.785553+0000 mgr.gibba032.whopeo (mgr.24625) 10133 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:05:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:10.787106+0000 mgr.gibba032.whopeo (mgr.24625) 10134 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:05:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cluster 2021-03-10T17:05:10.911075+0000 mgr.gibba032.whopeo (mgr.24625) 10135 : cluster [DBG] pgmap v5778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.039677+0000 mon.a (mon.0) 3411 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:05:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.040444+0000 mgr.gibba032.whopeo (mgr.24625) 10136 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:05:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.042223+0000 mgr.gibba032.whopeo (mgr.24625) 10137 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:05:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.042893+0000 mon.a (mon.0) 3412 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:05:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.043725+0000 mgr.gibba032.whopeo (mgr.24625) 10138 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:05:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.043763+0000 mgr.gibba032.whopeo (mgr.24625) 10139 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:05:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.044436+0000 mon.a (mon.0) 3413 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.045431+0000 mgr.gibba032.whopeo (mgr.24625) 10140 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:05:12.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.045468+0000 mgr.gibba032.whopeo (mgr.24625) 10141 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:05:12.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.045916+0000 mon.a (mon.0) 3414 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.046838+0000 mgr.gibba032.whopeo (mgr.24625) 10142 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:05:12.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.047615+0000 mon.a (mon.0) 3415 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.048015+0000 mgr.gibba032.whopeo (mgr.24625) 10143 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:05:12.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.048170+0000 mgr.gibba032.whopeo (mgr.24625) 10144 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:05:12.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.049448+0000 mgr.gibba032.whopeo (mgr.24625) 10145 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:05:12.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.049764+0000 mgr.gibba032.whopeo (mgr.24625) 10146 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:05:12.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.050397+0000 mon.a (mon.0) 3416 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:12.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.050782+0000 mgr.gibba032.whopeo (mgr.24625) 10147 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:05:12.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.050821+0000 mgr.gibba032.whopeo (mgr.24625) 10148 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:05:12.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: cephadm 2021-03-10T17:05:11.050925+0000 mgr.gibba032.whopeo (mgr.24625) 10149 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:12.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:12 gibba032 bash[24004]: audit 2021-03-10T17:05:11.051719+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:14 gibba011 bash[30791]: cluster 2021-03-10T17:05:12.912681+0000 mgr.gibba032.whopeo (mgr.24625) 10150 : cluster [DBG] pgmap v5779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:14 gibba011 bash[31774]: cluster 2021-03-10T17:05:12.912681+0000 mgr.gibba032.whopeo (mgr.24625) 10150 : cluster [DBG] pgmap v5779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:14.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:14 gibba032 bash[24004]: cluster 2021-03-10T17:05:12.912681+0000 mgr.gibba032.whopeo (mgr.24625) 10150 : cluster [DBG] pgmap v5779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:16.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:16 gibba011 bash[30791]: cluster 2021-03-10T17:05:14.913976+0000 mgr.gibba032.whopeo (mgr.24625) 10151 : cluster [DBG] pgmap v5780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:16.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:16 gibba011 bash[31774]: cluster 2021-03-10T17:05:14.913976+0000 mgr.gibba032.whopeo (mgr.24625) 10151 : cluster [DBG] pgmap v5780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:16 gibba032 bash[24004]: cluster 2021-03-10T17:05:14.913976+0000 mgr.gibba032.whopeo (mgr.24625) 10151 : cluster [DBG] pgmap v5780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:18.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:18 gibba011 bash[30791]: cluster 2021-03-10T17:05:16.915640+0000 mgr.gibba032.whopeo (mgr.24625) 10152 : cluster [DBG] pgmap v5781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:18.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:18 gibba011 bash[31774]: cluster 2021-03-10T17:05:16.915640+0000 mgr.gibba032.whopeo (mgr.24625) 10152 : cluster [DBG] pgmap v5781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:18 gibba032 bash[24004]: cluster 2021-03-10T17:05:16.915640+0000 mgr.gibba032.whopeo (mgr.24625) 10152 : cluster [DBG] pgmap v5781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:20.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:20 gibba011 bash[30791]: cluster 2021-03-10T17:05:18.917217+0000 mgr.gibba032.whopeo (mgr.24625) 10153 : cluster [DBG] pgmap v5782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:20.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:20 gibba011 bash[31774]: cluster 2021-03-10T17:05:18.917217+0000 mgr.gibba032.whopeo (mgr.24625) 10153 : cluster [DBG] pgmap v5782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:20 gibba032 bash[24004]: cluster 2021-03-10T17:05:18.917217+0000 mgr.gibba032.whopeo (mgr.24625) 10153 : cluster [DBG] pgmap v5782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:21 gibba032 bash[24004]: cluster 2021-03-10T17:05:20.918546+0000 mgr.gibba032.whopeo (mgr.24625) 10154 : cluster [DBG] pgmap v5783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:21 gibba011 bash[30791]: cluster 2021-03-10T17:05:20.918546+0000 mgr.gibba032.whopeo (mgr.24625) 10154 : cluster [DBG] pgmap v5783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:21 gibba011 bash[31774]: cluster 2021-03-10T17:05:20.918546+0000 mgr.gibba032.whopeo (mgr.24625) 10154 : cluster [DBG] pgmap v5783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:24.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:23 gibba032 bash[24004]: cluster 2021-03-10T17:05:22.920258+0000 mgr.gibba032.whopeo (mgr.24625) 10155 : cluster [DBG] pgmap v5784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:23 gibba011 bash[30791]: cluster 2021-03-10T17:05:22.920258+0000 mgr.gibba032.whopeo (mgr.24625) 10155 : cluster [DBG] pgmap v5784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:23 gibba011 bash[31774]: cluster 2021-03-10T17:05:22.920258+0000 mgr.gibba032.whopeo (mgr.24625) 10155 : cluster [DBG] pgmap v5784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:25 gibba032 bash[24004]: cluster 2021-03-10T17:05:24.921802+0000 mgr.gibba032.whopeo (mgr.24625) 10156 : cluster [DBG] pgmap v5785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:26.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:25 gibba011 bash[30791]: cluster 2021-03-10T17:05:24.921802+0000 mgr.gibba032.whopeo (mgr.24625) 10156 : cluster [DBG] pgmap v5785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:25 gibba011 bash[31774]: cluster 2021-03-10T17:05:24.921802+0000 mgr.gibba032.whopeo (mgr.24625) 10156 : cluster [DBG] pgmap v5785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:26.581 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:05:26.811 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:05:26.811 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:05:26.812 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:05:26.812 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:05:26.812 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:05:26.812 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:05:26.813 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:05:26.813 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:05:26.813 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:05:26.813 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:05:26.813 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:05:26.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:05:26.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:05:26.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:05:26.814 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:05:26.815 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:05:26.815 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:05:26.816 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:05:26.816 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:05:26.816 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:05:26.817 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:05:27.062 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:05:27.063 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:05:27.063 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:05:27.063 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:27.064 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:05:27.064 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:05:27.064 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:27.064 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:05:27.064 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:05:27.065 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:27.065 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:05:27.065 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:05:27.065 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:05:27.065 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:05:27.066 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:05:27.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:27 gibba032 bash[24004]: cephadm 2021-03-10T17:05:26.066621+0000 mgr.gibba032.whopeo (mgr.24625) 10157 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:27.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:27 gibba011 bash[30791]: cephadm 2021-03-10T17:05:26.066621+0000 mgr.gibba032.whopeo (mgr.24625) 10157 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:27 gibba011 bash[31774]: cephadm 2021-03-10T17:05:26.066621+0000 mgr.gibba032.whopeo (mgr.24625) 10157 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[30791]: audit 2021-03-10T17:05:26.570761+0000 mgr.gibba032.whopeo (mgr.24625) 10158 : audit [DBG] from='client.50859 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[30791]: audit 2021-03-10T17:05:26.798189+0000 mgr.gibba032.whopeo (mgr.24625) 10159 : audit [DBG] from='client.60847 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[30791]: cluster 2021-03-10T17:05:26.923566+0000 mgr.gibba032.whopeo (mgr.24625) 10160 : cluster [DBG] pgmap v5786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[30791]: audit 2021-03-10T17:05:27.063283+0000 mon.c (mon.1) 195 : audit [DBG] from='client.? 172.21.2.111:0/2066756848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[31774]: audit 2021-03-10T17:05:26.570761+0000 mgr.gibba032.whopeo (mgr.24625) 10158 : audit [DBG] from='client.50859 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[31774]: audit 2021-03-10T17:05:26.798189+0000 mgr.gibba032.whopeo (mgr.24625) 10159 : audit [DBG] from='client.60847 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[31774]: cluster 2021-03-10T17:05:26.923566+0000 mgr.gibba032.whopeo (mgr.24625) 10160 : cluster [DBG] pgmap v5786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:28 gibba011 bash[31774]: audit 2021-03-10T17:05:27.063283+0000 mon.c (mon.1) 195 : audit [DBG] from='client.? 172.21.2.111:0/2066756848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:28 gibba032 bash[24004]: audit 2021-03-10T17:05:26.570761+0000 mgr.gibba032.whopeo (mgr.24625) 10158 : audit [DBG] from='client.50859 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:28 gibba032 bash[24004]: audit 2021-03-10T17:05:26.798189+0000 mgr.gibba032.whopeo (mgr.24625) 10159 : audit [DBG] from='client.60847 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:28 gibba032 bash[24004]: cluster 2021-03-10T17:05:26.923566+0000 mgr.gibba032.whopeo (mgr.24625) 10160 : cluster [DBG] pgmap v5786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:28 gibba032 bash[24004]: audit 2021-03-10T17:05:27.063283+0000 mon.c (mon.1) 195 : audit [DBG] from='client.? 172.21.2.111:0/2066756848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:30 gibba011 bash[30791]: cluster 2021-03-10T17:05:28.925186+0000 mgr.gibba032.whopeo (mgr.24625) 10161 : cluster [DBG] pgmap v5787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:30 gibba011 bash[31774]: cluster 2021-03-10T17:05:28.925186+0000 mgr.gibba032.whopeo (mgr.24625) 10161 : cluster [DBG] pgmap v5787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:30 gibba032 bash[24004]: cluster 2021-03-10T17:05:28.925186+0000 mgr.gibba032.whopeo (mgr.24625) 10161 : cluster [DBG] pgmap v5787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:31 gibba032 bash[24004]: cluster 2021-03-10T17:05:30.926527+0000 mgr.gibba032.whopeo (mgr.24625) 10162 : cluster [DBG] pgmap v5788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:31 gibba011 bash[30791]: cluster 2021-03-10T17:05:30.926527+0000 mgr.gibba032.whopeo (mgr.24625) 10162 : cluster [DBG] pgmap v5788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:31 gibba011 bash[31774]: cluster 2021-03-10T17:05:30.926527+0000 mgr.gibba032.whopeo (mgr.24625) 10162 : cluster [DBG] pgmap v5788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:34.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:33 gibba032 bash[24004]: cluster 2021-03-10T17:05:32.928395+0000 mgr.gibba032.whopeo (mgr.24625) 10163 : cluster [DBG] pgmap v5789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:33 gibba011 bash[30791]: cluster 2021-03-10T17:05:32.928395+0000 mgr.gibba032.whopeo (mgr.24625) 10163 : cluster [DBG] pgmap v5789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:33 gibba011 bash[31774]: cluster 2021-03-10T17:05:32.928395+0000 mgr.gibba032.whopeo (mgr.24625) 10163 : cluster [DBG] pgmap v5789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:36.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:36 gibba032 bash[24004]: cluster 2021-03-10T17:05:34.929689+0000 mgr.gibba032.whopeo (mgr.24625) 10164 : cluster [DBG] pgmap v5790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:36 gibba011 bash[30791]: cluster 2021-03-10T17:05:34.929689+0000 mgr.gibba032.whopeo (mgr.24625) 10164 : cluster [DBG] pgmap v5790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:36 gibba011 bash[31774]: cluster 2021-03-10T17:05:34.929689+0000 mgr.gibba032.whopeo (mgr.24625) 10164 : cluster [DBG] pgmap v5790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:38 gibba011 bash[30791]: cluster 2021-03-10T17:05:36.931501+0000 mgr.gibba032.whopeo (mgr.24625) 10165 : cluster [DBG] pgmap v5791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:38 gibba011 bash[31774]: cluster 2021-03-10T17:05:36.931501+0000 mgr.gibba032.whopeo (mgr.24625) 10165 : cluster [DBG] pgmap v5791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:38 gibba032 bash[24004]: cluster 2021-03-10T17:05:36.931501+0000 mgr.gibba032.whopeo (mgr.24625) 10165 : cluster [DBG] pgmap v5791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:40.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:40 gibba011 bash[30791]: cluster 2021-03-10T17:05:38.933061+0000 mgr.gibba032.whopeo (mgr.24625) 10166 : cluster [DBG] pgmap v5792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:40 gibba011 bash[31774]: cluster 2021-03-10T17:05:38.933061+0000 mgr.gibba032.whopeo (mgr.24625) 10166 : cluster [DBG] pgmap v5792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:40 gibba032 bash[24004]: cluster 2021-03-10T17:05:38.933061+0000 mgr.gibba032.whopeo (mgr.24625) 10166 : cluster [DBG] pgmap v5792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:41 gibba032 bash[24004]: cluster 2021-03-10T17:05:40.934460+0000 mgr.gibba032.whopeo (mgr.24625) 10167 : cluster [DBG] pgmap v5793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:41 gibba032 bash[24004]: cephadm 2021-03-10T17:05:41.080792+0000 mgr.gibba032.whopeo (mgr.24625) 10168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:42.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:41 gibba011 bash[30791]: cluster 2021-03-10T17:05:40.934460+0000 mgr.gibba032.whopeo (mgr.24625) 10167 : cluster [DBG] pgmap v5793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:41 gibba011 bash[30791]: cephadm 2021-03-10T17:05:41.080792+0000 mgr.gibba032.whopeo (mgr.24625) 10168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:41 gibba011 bash[31774]: cluster 2021-03-10T17:05:40.934460+0000 mgr.gibba032.whopeo (mgr.24625) 10167 : cluster [DBG] pgmap v5793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:41 gibba011 bash[31774]: cephadm 2021-03-10T17:05:41.080792+0000 mgr.gibba032.whopeo (mgr.24625) 10168 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:44 gibba032 bash[24004]: cluster 2021-03-10T17:05:42.936262+0000 mgr.gibba032.whopeo (mgr.24625) 10169 : cluster [DBG] pgmap v5794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:44 gibba011 bash[30791]: cluster 2021-03-10T17:05:42.936262+0000 mgr.gibba032.whopeo (mgr.24625) 10169 : cluster [DBG] pgmap v5794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:44 gibba011 bash[31774]: cluster 2021-03-10T17:05:42.936262+0000 mgr.gibba032.whopeo (mgr.24625) 10169 : cluster [DBG] pgmap v5794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:46.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:46 gibba011 bash[30791]: cluster 2021-03-10T17:05:44.937489+0000 mgr.gibba032.whopeo (mgr.24625) 10170 : cluster [DBG] pgmap v5795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:46 gibba011 bash[31774]: cluster 2021-03-10T17:05:44.937489+0000 mgr.gibba032.whopeo (mgr.24625) 10170 : cluster [DBG] pgmap v5795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:46 gibba032 bash[24004]: cluster 2021-03-10T17:05:44.937489+0000 mgr.gibba032.whopeo (mgr.24625) 10170 : cluster [DBG] pgmap v5795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:48 gibba011 bash[30791]: cluster 2021-03-10T17:05:46.939160+0000 mgr.gibba032.whopeo (mgr.24625) 10171 : cluster [DBG] pgmap v5796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:48 gibba011 bash[31774]: cluster 2021-03-10T17:05:46.939160+0000 mgr.gibba032.whopeo (mgr.24625) 10171 : cluster [DBG] pgmap v5796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:48 gibba032 bash[24004]: cluster 2021-03-10T17:05:46.939160+0000 mgr.gibba032.whopeo (mgr.24625) 10171 : cluster [DBG] pgmap v5796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:50 gibba011 bash[30791]: cluster 2021-03-10T17:05:48.940701+0000 mgr.gibba032.whopeo (mgr.24625) 10172 : cluster [DBG] pgmap v5797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:50 gibba011 bash[31774]: cluster 2021-03-10T17:05:48.940701+0000 mgr.gibba032.whopeo (mgr.24625) 10172 : cluster [DBG] pgmap v5797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:50 gibba032 bash[24004]: cluster 2021-03-10T17:05:48.940701+0000 mgr.gibba032.whopeo (mgr.24625) 10172 : cluster [DBG] pgmap v5797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:52.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:51 gibba032 bash[24004]: cluster 2021-03-10T17:05:50.942073+0000 mgr.gibba032.whopeo (mgr.24625) 10173 : cluster [DBG] pgmap v5798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:51 gibba011 bash[30791]: cluster 2021-03-10T17:05:50.942073+0000 mgr.gibba032.whopeo (mgr.24625) 10173 : cluster [DBG] pgmap v5798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:51 gibba011 bash[31774]: cluster 2021-03-10T17:05:50.942073+0000 mgr.gibba032.whopeo (mgr.24625) 10173 : cluster [DBG] pgmap v5798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:54.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:54 gibba011 bash[30791]: cluster 2021-03-10T17:05:52.943919+0000 mgr.gibba032.whopeo (mgr.24625) 10174 : cluster [DBG] pgmap v5799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:54 gibba011 bash[31774]: cluster 2021-03-10T17:05:52.943919+0000 mgr.gibba032.whopeo (mgr.24625) 10174 : cluster [DBG] pgmap v5799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:54 gibba032 bash[24004]: cluster 2021-03-10T17:05:52.943919+0000 mgr.gibba032.whopeo (mgr.24625) 10174 : cluster [DBG] pgmap v5799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:56.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:56 gibba011 bash[30791]: cluster 2021-03-10T17:05:54.945241+0000 mgr.gibba032.whopeo (mgr.24625) 10175 : cluster [DBG] pgmap v5800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:56 gibba011 bash[31774]: cluster 2021-03-10T17:05:54.945241+0000 mgr.gibba032.whopeo (mgr.24625) 10175 : cluster [DBG] pgmap v5800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:56 gibba032 bash[24004]: cluster 2021-03-10T17:05:54.945241+0000 mgr.gibba032.whopeo (mgr.24625) 10175 : cluster [DBG] pgmap v5800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:57.304 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:05:57.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:57 gibba011 bash[30791]: cephadm 2021-03-10T17:05:56.096735+0000 mgr.gibba032.whopeo (mgr.24625) 10176 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:57.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:57 gibba011 bash[31774]: cephadm 2021-03-10T17:05:56.096735+0000 mgr.gibba032.whopeo (mgr.24625) 10176 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:57 gibba032 bash[24004]: cephadm 2021-03-10T17:05:56.096735+0000 mgr.gibba032.whopeo (mgr.24625) 10176 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:05:57.532 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:05:57.532 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:05:57.533 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:05:57.533 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:05:57.533 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:05:57.533 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:05:57.534 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:05:57.534 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:05:57.534 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:05:57.534 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:05:57.534 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:05:57.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:05:57.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:05:57.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:05:57.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:05:57.535 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:05:57.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:05:57.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:05:57.536 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:05:57.536 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:05:57.536 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:05:57.781 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:05:57.781 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:05:57.782 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:05:57.782 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:57.782 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:05:57.782 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:05:57.782 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:57.783 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:05:57.783 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:05:57.783 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:05:57.783 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:05:57.784 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:05:57.784 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:05:57.784 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:05:57.784 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:05:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: cluster 2021-03-10T17:05:56.946980+0000 mgr.gibba032.whopeo (mgr.24625) 10177 : cluster [DBG] pgmap v5801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.294118+0000 mgr.gibba032.whopeo (mgr.24625) 10178 : audit [DBG] from='client.50877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.320269+0000 mon.a (mon.0) 3418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:05:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.320528+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:05:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.472631+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:05:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.472853+0000 mon.a (mon.0) 3421 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.473045+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.473248+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T17:05:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.520710+0000 mgr.gibba032.whopeo (mgr.24625) 10179 : audit [DBG] from='client.60865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[30791]: audit 2021-03-10T17:05:57.781579+0000 mon.c (mon.1) 196 : audit [DBG] from='client.? 172.21.2.111:0/128247482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: cluster 2021-03-10T17:05:56.946980+0000 mgr.gibba032.whopeo (mgr.24625) 10177 : cluster [DBG] pgmap v5801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.294118+0000 mgr.gibba032.whopeo (mgr.24625) 10178 : audit [DBG] from='client.50877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.320269+0000 mon.a (mon.0) 3418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:05:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.320528+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:05:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.472631+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:05:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.472853+0000 mon.a (mon.0) 3421 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.473045+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.473248+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T17:05:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.520710+0000 mgr.gibba032.whopeo (mgr.24625) 10179 : audit [DBG] from='client.60865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:05:58 gibba011 bash[31774]: audit 2021-03-10T17:05:57.781579+0000 mon.c (mon.1) 196 : audit [DBG] from='client.? 172.21.2.111:0/128247482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: cluster 2021-03-10T17:05:56.946980+0000 mgr.gibba032.whopeo (mgr.24625) 10177 : cluster [DBG] pgmap v5801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.294118+0000 mgr.gibba032.whopeo (mgr.24625) 10178 : audit [DBG] from='client.50877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.320269+0000 mon.a (mon.0) 3418 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:05:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.320528+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:05:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.472631+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:05:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.472853+0000 mon.a (mon.0) 3421 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.473045+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:05:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.473248+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2021-03-10T17:05:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.520710+0000 mgr.gibba032.whopeo (mgr.24625) 10179 : audit [DBG] from='client.60865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:05:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:05:58 gibba032 bash[24004]: audit 2021-03-10T17:05:57.781579+0000 mon.c (mon.1) 196 : audit [DBG] from='client.? 172.21.2.111:0/128247482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:00.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:00 gibba011 bash[30791]: cluster 2021-03-10T17:05:58.948404+0000 mgr.gibba032.whopeo (mgr.24625) 10180 : cluster [DBG] pgmap v5802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:00.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:00 gibba011 bash[31774]: cluster 2021-03-10T17:05:58.948404+0000 mgr.gibba032.whopeo (mgr.24625) 10180 : cluster [DBG] pgmap v5802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:00 gibba032 bash[24004]: cluster 2021-03-10T17:05:58.948404+0000 mgr.gibba032.whopeo (mgr.24625) 10180 : cluster [DBG] pgmap v5802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:01 gibba032 bash[24004]: cluster 2021-03-10T17:06:00.949760+0000 mgr.gibba032.whopeo (mgr.24625) 10181 : cluster [DBG] pgmap v5803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:01 gibba011 bash[30791]: cluster 2021-03-10T17:06:00.949760+0000 mgr.gibba032.whopeo (mgr.24625) 10181 : cluster [DBG] pgmap v5803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:01 gibba011 bash[31774]: cluster 2021-03-10T17:06:00.949760+0000 mgr.gibba032.whopeo (mgr.24625) 10181 : cluster [DBG] pgmap v5803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:04 gibba011 bash[30791]: cluster 2021-03-10T17:06:02.951309+0000 mgr.gibba032.whopeo (mgr.24625) 10182 : cluster [DBG] pgmap v5804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:04 gibba011 bash[31774]: cluster 2021-03-10T17:06:02.951309+0000 mgr.gibba032.whopeo (mgr.24625) 10182 : cluster [DBG] pgmap v5804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:04 gibba032 bash[24004]: cluster 2021-03-10T17:06:02.951309+0000 mgr.gibba032.whopeo (mgr.24625) 10182 : cluster [DBG] pgmap v5804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:06.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:06 gibba011 bash[30791]: cluster 2021-03-10T17:06:04.952871+0000 mgr.gibba032.whopeo (mgr.24625) 10183 : cluster [DBG] pgmap v5805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:06 gibba011 bash[31774]: cluster 2021-03-10T17:06:04.952871+0000 mgr.gibba032.whopeo (mgr.24625) 10183 : cluster [DBG] pgmap v5805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:06 gibba032 bash[24004]: cluster 2021-03-10T17:06:04.952871+0000 mgr.gibba032.whopeo (mgr.24625) 10183 : cluster [DBG] pgmap v5805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:08.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:08 gibba011 bash[30791]: cluster 2021-03-10T17:06:06.954614+0000 mgr.gibba032.whopeo (mgr.24625) 10184 : cluster [DBG] pgmap v5806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:08.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:08 gibba011 bash[31774]: cluster 2021-03-10T17:06:06.954614+0000 mgr.gibba032.whopeo (mgr.24625) 10184 : cluster [DBG] pgmap v5806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:08 gibba032 bash[24004]: cluster 2021-03-10T17:06:06.954614+0000 mgr.gibba032.whopeo (mgr.24625) 10184 : cluster [DBG] pgmap v5806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:10.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:10 gibba011 bash[30791]: cluster 2021-03-10T17:06:08.956258+0000 mgr.gibba032.whopeo (mgr.24625) 10185 : cluster [DBG] pgmap v5807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:10.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:10 gibba011 bash[31774]: cluster 2021-03-10T17:06:08.956258+0000 mgr.gibba032.whopeo (mgr.24625) 10185 : cluster [DBG] pgmap v5807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:10 gibba032 bash[24004]: cluster 2021-03-10T17:06:08.956258+0000 mgr.gibba032.whopeo (mgr.24625) 10185 : cluster [DBG] pgmap v5807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cluster 2021-03-10T17:06:10.957571+0000 mgr.gibba032.whopeo (mgr.24625) 10186 : cluster [DBG] pgmap v5808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.109989+0000 mgr.gibba032.whopeo (mgr.24625) 10187 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:06:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.111542+0000 mgr.gibba032.whopeo (mgr.24625) 10188 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:06:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.374334+0000 mon.a (mon.0) 3424 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:06:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.379520+0000 mgr.gibba032.whopeo (mgr.24625) 10189 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:06:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.382549+0000 mon.a (mon.0) 3425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:06:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.382912+0000 mgr.gibba032.whopeo (mgr.24625) 10190 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:06:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.385409+0000 mon.a (mon.0) 3426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.385723+0000 mgr.gibba032.whopeo (mgr.24625) 10191 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:06:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.385784+0000 mgr.gibba032.whopeo (mgr.24625) 10192 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:06:12.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.388348+0000 mon.a (mon.0) 3427 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.388673+0000 mgr.gibba032.whopeo (mgr.24625) 10193 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:06:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.388710+0000 mgr.gibba032.whopeo (mgr.24625) 10194 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:06:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.391064+0000 mgr.gibba032.whopeo (mgr.24625) 10195 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:06:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.391114+0000 mon.a (mon.0) 3428 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.393093+0000 mgr.gibba032.whopeo (mgr.24625) 10196 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:06:12.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.393128+0000 mgr.gibba032.whopeo (mgr.24625) 10197 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:06:12.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.395555+0000 mgr.gibba032.whopeo (mgr.24625) 10198 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:06:12.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.395616+0000 mon.a (mon.0) 3429 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.395880+0000 mgr.gibba032.whopeo (mgr.24625) 10199 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:06:12.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: audit 2021-03-10T17:06:11.398179+0000 mon.a (mon.0) 3430 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.398578+0000 mgr.gibba032.whopeo (mgr.24625) 10200 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:06:12.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.398622+0000 mgr.gibba032.whopeo (mgr.24625) 10201 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:06:12.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:12 gibba032 bash[24004]: cephadm 2021-03-10T17:06:11.398747+0000 mgr.gibba032.whopeo (mgr.24625) 10202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cluster 2021-03-10T17:06:10.957571+0000 mgr.gibba032.whopeo (mgr.24625) 10186 : cluster [DBG] pgmap v5808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.109989+0000 mgr.gibba032.whopeo (mgr.24625) 10187 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:06:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.111542+0000 mgr.gibba032.whopeo (mgr.24625) 10188 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:06:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.374334+0000 mon.a (mon.0) 3424 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:06:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.379520+0000 mgr.gibba032.whopeo (mgr.24625) 10189 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:06:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.382549+0000 mon.a (mon.0) 3425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:06:12.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.382912+0000 mgr.gibba032.whopeo (mgr.24625) 10190 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:06:12.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.385409+0000 mon.a (mon.0) 3426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.385723+0000 mgr.gibba032.whopeo (mgr.24625) 10191 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:06:12.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.385784+0000 mgr.gibba032.whopeo (mgr.24625) 10192 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:06:12.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.388348+0000 mon.a (mon.0) 3427 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.388673+0000 mgr.gibba032.whopeo (mgr.24625) 10193 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:06:12.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.388710+0000 mgr.gibba032.whopeo (mgr.24625) 10194 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:06:12.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.391064+0000 mgr.gibba032.whopeo (mgr.24625) 10195 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:06:12.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.391114+0000 mon.a (mon.0) 3428 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.393093+0000 mgr.gibba032.whopeo (mgr.24625) 10196 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:06:12.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.393128+0000 mgr.gibba032.whopeo (mgr.24625) 10197 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:06:12.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.395555+0000 mgr.gibba032.whopeo (mgr.24625) 10198 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:06:12.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.395616+0000 mon.a (mon.0) 3429 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.395880+0000 mgr.gibba032.whopeo (mgr.24625) 10199 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:06:12.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cluster 2021-03-10T17:06:10.957571+0000 mgr.gibba032.whopeo (mgr.24625) 10186 : cluster [DBG] pgmap v5808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:12.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.109989+0000 mgr.gibba032.whopeo (mgr.24625) 10187 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:06:12.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.111542+0000 mgr.gibba032.whopeo (mgr.24625) 10188 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:06:12.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.374334+0000 mon.a (mon.0) 3424 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:06:12.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.379520+0000 mgr.gibba032.whopeo (mgr.24625) 10189 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:06:12.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.382549+0000 mon.a (mon.0) 3425 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:06:12.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.382912+0000 mgr.gibba032.whopeo (mgr.24625) 10190 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:06:12.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.385409+0000 mon.a (mon.0) 3426 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.385723+0000 mgr.gibba032.whopeo (mgr.24625) 10191 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:06:12.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.385784+0000 mgr.gibba032.whopeo (mgr.24625) 10192 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:06:12.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.388348+0000 mon.a (mon.0) 3427 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.388673+0000 mgr.gibba032.whopeo (mgr.24625) 10193 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:06:12.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.388710+0000 mgr.gibba032.whopeo (mgr.24625) 10194 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:06:12.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.391064+0000 mgr.gibba032.whopeo (mgr.24625) 10195 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:06:12.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.391114+0000 mon.a (mon.0) 3428 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.393093+0000 mgr.gibba032.whopeo (mgr.24625) 10196 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:06:12.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.393128+0000 mgr.gibba032.whopeo (mgr.24625) 10197 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:06:12.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.395555+0000 mgr.gibba032.whopeo (mgr.24625) 10198 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:06:12.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.395616+0000 mon.a (mon.0) 3429 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.395880+0000 mgr.gibba032.whopeo (mgr.24625) 10199 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:06:12.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: audit 2021-03-10T17:06:11.398179+0000 mon.a (mon.0) 3430 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.398578+0000 mgr.gibba032.whopeo (mgr.24625) 10200 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:06:12.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.398622+0000 mgr.gibba032.whopeo (mgr.24625) 10201 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:06:12.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[30791]: cephadm 2021-03-10T17:06:11.398747+0000 mgr.gibba032.whopeo (mgr.24625) 10202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:12.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: audit 2021-03-10T17:06:11.398179+0000 mon.a (mon.0) 3430 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:12.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.398578+0000 mgr.gibba032.whopeo (mgr.24625) 10200 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:06:12.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.398622+0000 mgr.gibba032.whopeo (mgr.24625) 10201 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:06:12.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:12 gibba011 bash[31774]: cephadm 2021-03-10T17:06:11.398747+0000 mgr.gibba032.whopeo (mgr.24625) 10202 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:14 gibba032 bash[24004]: cluster 2021-03-10T17:06:12.959434+0000 mgr.gibba032.whopeo (mgr.24625) 10203 : cluster [DBG] pgmap v5809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:14 gibba011 bash[30791]: cluster 2021-03-10T17:06:12.959434+0000 mgr.gibba032.whopeo (mgr.24625) 10203 : cluster [DBG] pgmap v5809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:14 gibba011 bash[31774]: cluster 2021-03-10T17:06:12.959434+0000 mgr.gibba032.whopeo (mgr.24625) 10203 : cluster [DBG] pgmap v5809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:16 gibba032 bash[24004]: cluster 2021-03-10T17:06:14.960760+0000 mgr.gibba032.whopeo (mgr.24625) 10204 : cluster [DBG] pgmap v5810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:16 gibba011 bash[30791]: cluster 2021-03-10T17:06:14.960760+0000 mgr.gibba032.whopeo (mgr.24625) 10204 : cluster [DBG] pgmap v5810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:16 gibba011 bash[31774]: cluster 2021-03-10T17:06:14.960760+0000 mgr.gibba032.whopeo (mgr.24625) 10204 : cluster [DBG] pgmap v5810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:18 gibba032 bash[24004]: cluster 2021-03-10T17:06:16.962536+0000 mgr.gibba032.whopeo (mgr.24625) 10205 : cluster [DBG] pgmap v5811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:18.828 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:18 gibba011 bash[31774]: cluster 2021-03-10T17:06:16.962536+0000 mgr.gibba032.whopeo (mgr.24625) 10205 : cluster [DBG] pgmap v5811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:18 gibba011 bash[30791]: cluster 2021-03-10T17:06:16.962536+0000 mgr.gibba032.whopeo (mgr.24625) 10205 : cluster [DBG] pgmap v5811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:20 gibba032 bash[24004]: cluster 2021-03-10T17:06:18.964233+0000 mgr.gibba032.whopeo (mgr.24625) 10206 : cluster [DBG] pgmap v5812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:20 gibba011 bash[30791]: cluster 2021-03-10T17:06:18.964233+0000 mgr.gibba032.whopeo (mgr.24625) 10206 : cluster [DBG] pgmap v5812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:20 gibba011 bash[31774]: cluster 2021-03-10T17:06:18.964233+0000 mgr.gibba032.whopeo (mgr.24625) 10206 : cluster [DBG] pgmap v5812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:21 gibba032 bash[24004]: cluster 2021-03-10T17:06:20.965670+0000 mgr.gibba032.whopeo (mgr.24625) 10207 : cluster [DBG] pgmap v5813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:21 gibba011 bash[30791]: cluster 2021-03-10T17:06:20.965670+0000 mgr.gibba032.whopeo (mgr.24625) 10207 : cluster [DBG] pgmap v5813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:21 gibba011 bash[31774]: cluster 2021-03-10T17:06:20.965670+0000 mgr.gibba032.whopeo (mgr.24625) 10207 : cluster [DBG] pgmap v5813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:24 gibba011 bash[30791]: cluster 2021-03-10T17:06:22.967445+0000 mgr.gibba032.whopeo (mgr.24625) 10208 : cluster [DBG] pgmap v5814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:24 gibba011 bash[31774]: cluster 2021-03-10T17:06:22.967445+0000 mgr.gibba032.whopeo (mgr.24625) 10208 : cluster [DBG] pgmap v5814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:24 gibba032 bash[24004]: cluster 2021-03-10T17:06:22.967445+0000 mgr.gibba032.whopeo (mgr.24625) 10208 : cluster [DBG] pgmap v5814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:26 gibba011 bash[30791]: cluster 2021-03-10T17:06:24.968969+0000 mgr.gibba032.whopeo (mgr.24625) 10209 : cluster [DBG] pgmap v5815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:26 gibba011 bash[31774]: cluster 2021-03-10T17:06:24.968969+0000 mgr.gibba032.whopeo (mgr.24625) 10209 : cluster [DBG] pgmap v5815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:26.411 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:26 gibba032 bash[24004]: cluster 2021-03-10T17:06:24.968969+0000 mgr.gibba032.whopeo (mgr.24625) 10209 : cluster [DBG] pgmap v5815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:27.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:27 gibba011 bash[30791]: cephadm 2021-03-10T17:06:26.409337+0000 mgr.gibba032.whopeo (mgr.24625) 10210 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:27.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:27 gibba011 bash[31774]: cephadm 2021-03-10T17:06:26.409337+0000 mgr.gibba032.whopeo (mgr.24625) 10210 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:27.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:27 gibba032 bash[24004]: cephadm 2021-03-10T17:06:26.409337+0000 mgr.gibba032.whopeo (mgr.24625) 10210 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:28.024 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:06:28.288 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:06:28.288 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:06:28.288 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:06:28.289 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:06:28.289 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:06:28.289 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:06:28.289 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:06:28.290 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:06:28.290 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:06:28.290 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:06:28.290 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:06:28.290 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:06:28.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:06:28.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:06:28.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:06:28.291 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:06:28.292 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:06:28.292 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:06:28.292 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:06:28.292 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:06:28.292 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:06:28.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:28 gibba011 bash[30791]: cluster 2021-03-10T17:06:26.970513+0000 mgr.gibba032.whopeo (mgr.24625) 10211 : cluster [DBG] pgmap v5816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:28.328 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:28 gibba011 bash[31774]: cluster 2021-03-10T17:06:26.970513+0000 mgr.gibba032.whopeo (mgr.24625) 10211 : cluster [DBG] pgmap v5816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:28 gibba032 bash[24004]: cluster 2021-03-10T17:06:26.970513+0000 mgr.gibba032.whopeo (mgr.24625) 10211 : cluster [DBG] pgmap v5816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:28.536 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:06:28.537 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:06:28.537 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:06:28.537 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:28.537 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:06:28.538 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:06:28.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:28.538 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:06:28.538 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:06:28.539 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:28.539 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:06:28.539 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:06:28.539 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:06:28.539 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:06:28.540 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:06:29.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[30791]: audit 2021-03-10T17:06:28.014458+0000 mgr.gibba032.whopeo (mgr.24625) 10212 : audit [DBG] from='client.60877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[30791]: audit 2021-03-10T17:06:28.247198+0000 mgr.gibba032.whopeo (mgr.24625) 10213 : audit [DBG] from='client.50901 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[30791]: audit 2021-03-10T17:06:28.536980+0000 mon.a (mon.0) 3431 : audit [DBG] from='client.? 172.21.2.111:0/137756320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[31774]: audit 2021-03-10T17:06:28.014458+0000 mgr.gibba032.whopeo (mgr.24625) 10212 : audit [DBG] from='client.60877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[31774]: audit 2021-03-10T17:06:28.247198+0000 mgr.gibba032.whopeo (mgr.24625) 10213 : audit [DBG] from='client.50901 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:29 gibba011 bash[31774]: audit 2021-03-10T17:06:28.536980+0000 mon.a (mon.0) 3431 : audit [DBG] from='client.? 172.21.2.111:0/137756320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:29 gibba032 bash[24004]: audit 2021-03-10T17:06:28.014458+0000 mgr.gibba032.whopeo (mgr.24625) 10212 : audit [DBG] from='client.60877 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:29 gibba032 bash[24004]: audit 2021-03-10T17:06:28.247198+0000 mgr.gibba032.whopeo (mgr.24625) 10213 : audit [DBG] from='client.50901 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:06:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:29 gibba032 bash[24004]: audit 2021-03-10T17:06:28.536980+0000 mon.a (mon.0) 3431 : audit [DBG] from='client.? 172.21.2.111:0/137756320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:06:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:30 gibba011 bash[30791]: cluster 2021-03-10T17:06:28.972071+0000 mgr.gibba032.whopeo (mgr.24625) 10214 : cluster [DBG] pgmap v5817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:30.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:30 gibba011 bash[31774]: cluster 2021-03-10T17:06:28.972071+0000 mgr.gibba032.whopeo (mgr.24625) 10214 : cluster [DBG] pgmap v5817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:30 gibba032 bash[24004]: cluster 2021-03-10T17:06:28.972071+0000 mgr.gibba032.whopeo (mgr.24625) 10214 : cluster [DBG] pgmap v5817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:31 gibba032 bash[24004]: cluster 2021-03-10T17:06:30.973402+0000 mgr.gibba032.whopeo (mgr.24625) 10215 : cluster [DBG] pgmap v5818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:31 gibba011 bash[30791]: cluster 2021-03-10T17:06:30.973402+0000 mgr.gibba032.whopeo (mgr.24625) 10215 : cluster [DBG] pgmap v5818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:31 gibba011 bash[31774]: cluster 2021-03-10T17:06:30.973402+0000 mgr.gibba032.whopeo (mgr.24625) 10215 : cluster [DBG] pgmap v5818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:34 gibba011 bash[30791]: cluster 2021-03-10T17:06:32.975091+0000 mgr.gibba032.whopeo (mgr.24625) 10216 : cluster [DBG] pgmap v5819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:34.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:34 gibba011 bash[31774]: cluster 2021-03-10T17:06:32.975091+0000 mgr.gibba032.whopeo (mgr.24625) 10216 : cluster [DBG] pgmap v5819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:34 gibba032 bash[24004]: cluster 2021-03-10T17:06:32.975091+0000 mgr.gibba032.whopeo (mgr.24625) 10216 : cluster [DBG] pgmap v5819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:36.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:36 gibba011 bash[30791]: cluster 2021-03-10T17:06:34.976646+0000 mgr.gibba032.whopeo (mgr.24625) 10217 : cluster [DBG] pgmap v5820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:36.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:36 gibba011 bash[31774]: cluster 2021-03-10T17:06:34.976646+0000 mgr.gibba032.whopeo (mgr.24625) 10217 : cluster [DBG] pgmap v5820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:36 gibba032 bash[24004]: cluster 2021-03-10T17:06:34.976646+0000 mgr.gibba032.whopeo (mgr.24625) 10217 : cluster [DBG] pgmap v5820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:38.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:38 gibba011 bash[30791]: cluster 2021-03-10T17:06:36.978170+0000 mgr.gibba032.whopeo (mgr.24625) 10218 : cluster [DBG] pgmap v5821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:38.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:38 gibba011 bash[31774]: cluster 2021-03-10T17:06:36.978170+0000 mgr.gibba032.whopeo (mgr.24625) 10218 : cluster [DBG] pgmap v5821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:38 gibba032 bash[24004]: cluster 2021-03-10T17:06:36.978170+0000 mgr.gibba032.whopeo (mgr.24625) 10218 : cluster [DBG] pgmap v5821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:40.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:40 gibba011 bash[30791]: cluster 2021-03-10T17:06:38.979831+0000 mgr.gibba032.whopeo (mgr.24625) 10219 : cluster [DBG] pgmap v5822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:40.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:40 gibba011 bash[31774]: cluster 2021-03-10T17:06:38.979831+0000 mgr.gibba032.whopeo (mgr.24625) 10219 : cluster [DBG] pgmap v5822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:40 gibba032 bash[24004]: cluster 2021-03-10T17:06:38.979831+0000 mgr.gibba032.whopeo (mgr.24625) 10219 : cluster [DBG] pgmap v5822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:41.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:41 gibba032 bash[24004]: cluster 2021-03-10T17:06:40.981177+0000 mgr.gibba032.whopeo (mgr.24625) 10220 : cluster [DBG] pgmap v5823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:41.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:41 gibba032 bash[24004]: cephadm 2021-03-10T17:06:41.424938+0000 mgr.gibba032.whopeo (mgr.24625) 10221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:41 gibba011 bash[30791]: cluster 2021-03-10T17:06:40.981177+0000 mgr.gibba032.whopeo (mgr.24625) 10220 : cluster [DBG] pgmap v5823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:41 gibba011 bash[30791]: cephadm 2021-03-10T17:06:41.424938+0000 mgr.gibba032.whopeo (mgr.24625) 10221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:41 gibba011 bash[31774]: cluster 2021-03-10T17:06:40.981177+0000 mgr.gibba032.whopeo (mgr.24625) 10220 : cluster [DBG] pgmap v5823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:41 gibba011 bash[31774]: cephadm 2021-03-10T17:06:41.424938+0000 mgr.gibba032.whopeo (mgr.24625) 10221 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:44.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:44 gibba011 bash[30791]: cluster 2021-03-10T17:06:42.982873+0000 mgr.gibba032.whopeo (mgr.24625) 10222 : cluster [DBG] pgmap v5824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:44.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:44 gibba011 bash[31774]: cluster 2021-03-10T17:06:42.982873+0000 mgr.gibba032.whopeo (mgr.24625) 10222 : cluster [DBG] pgmap v5824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:44 gibba032 bash[24004]: cluster 2021-03-10T17:06:42.982873+0000 mgr.gibba032.whopeo (mgr.24625) 10222 : cluster [DBG] pgmap v5824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:46.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:46 gibba011 bash[30791]: cluster 2021-03-10T17:06:44.984308+0000 mgr.gibba032.whopeo (mgr.24625) 10223 : cluster [DBG] pgmap v5825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:46.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:46 gibba011 bash[31774]: cluster 2021-03-10T17:06:44.984308+0000 mgr.gibba032.whopeo (mgr.24625) 10223 : cluster [DBG] pgmap v5825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:46 gibba032 bash[24004]: cluster 2021-03-10T17:06:44.984308+0000 mgr.gibba032.whopeo (mgr.24625) 10223 : cluster [DBG] pgmap v5825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:48.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:48 gibba011 bash[30791]: cluster 2021-03-10T17:06:46.985913+0000 mgr.gibba032.whopeo (mgr.24625) 10224 : cluster [DBG] pgmap v5826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:48.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:48 gibba011 bash[31774]: cluster 2021-03-10T17:06:46.985913+0000 mgr.gibba032.whopeo (mgr.24625) 10224 : cluster [DBG] pgmap v5826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:48 gibba032 bash[24004]: cluster 2021-03-10T17:06:46.985913+0000 mgr.gibba032.whopeo (mgr.24625) 10224 : cluster [DBG] pgmap v5826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:50.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:50 gibba011 bash[30791]: cluster 2021-03-10T17:06:48.987458+0000 mgr.gibba032.whopeo (mgr.24625) 10225 : cluster [DBG] pgmap v5827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:50.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:50 gibba011 bash[31774]: cluster 2021-03-10T17:06:48.987458+0000 mgr.gibba032.whopeo (mgr.24625) 10225 : cluster [DBG] pgmap v5827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:50 gibba032 bash[24004]: cluster 2021-03-10T17:06:48.987458+0000 mgr.gibba032.whopeo (mgr.24625) 10225 : cluster [DBG] pgmap v5827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:52.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:51 gibba032 bash[24004]: cluster 2021-03-10T17:06:50.988850+0000 mgr.gibba032.whopeo (mgr.24625) 10226 : cluster [DBG] pgmap v5828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:51 gibba011 bash[30791]: cluster 2021-03-10T17:06:50.988850+0000 mgr.gibba032.whopeo (mgr.24625) 10226 : cluster [DBG] pgmap v5828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:51 gibba011 bash[31774]: cluster 2021-03-10T17:06:50.988850+0000 mgr.gibba032.whopeo (mgr.24625) 10226 : cluster [DBG] pgmap v5828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:54 gibba011 bash[30791]: cluster 2021-03-10T17:06:52.990651+0000 mgr.gibba032.whopeo (mgr.24625) 10227 : cluster [DBG] pgmap v5829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:54 gibba011 bash[31774]: cluster 2021-03-10T17:06:52.990651+0000 mgr.gibba032.whopeo (mgr.24625) 10227 : cluster [DBG] pgmap v5829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:54 gibba032 bash[24004]: cluster 2021-03-10T17:06:52.990651+0000 mgr.gibba032.whopeo (mgr.24625) 10227 : cluster [DBG] pgmap v5829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:56 gibba011 bash[30791]: cluster 2021-03-10T17:06:54.991757+0000 mgr.gibba032.whopeo (mgr.24625) 10228 : cluster [DBG] pgmap v5830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:56.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:56 gibba011 bash[31774]: cluster 2021-03-10T17:06:54.991757+0000 mgr.gibba032.whopeo (mgr.24625) 10228 : cluster [DBG] pgmap v5830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:56.442 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:56 gibba032 bash[24004]: cluster 2021-03-10T17:06:54.991757+0000 mgr.gibba032.whopeo (mgr.24625) 10228 : cluster [DBG] pgmap v5830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:57 gibba011 bash[30791]: cephadm 2021-03-10T17:06:56.440992+0000 mgr.gibba032.whopeo (mgr.24625) 10229 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:57 gibba011 bash[31774]: cephadm 2021-03-10T17:06:56.440992+0000 mgr.gibba032.whopeo (mgr.24625) 10229 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:57.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:57 gibba032 bash[24004]: cephadm 2021-03-10T17:06:56.440992+0000 mgr.gibba032.whopeo (mgr.24625) 10229 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:06:58.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: cluster 2021-03-10T17:06:56.993421+0000 mgr.gibba032.whopeo (mgr.24625) 10230 : cluster [DBG] pgmap v5831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.327322+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.328016+0000 mon.a (mon.0) 3433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:06:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.481784+0000 mon.a (mon.0) 3434 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:06:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.482588+0000 mon.a (mon.0) 3435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:06:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.483299+0000 mon.a (mon.0) 3436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[30791]: audit 2021-03-10T17:06:57.484029+0000 mon.a (mon.0) 3437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: cluster 2021-03-10T17:06:56.993421+0000 mgr.gibba032.whopeo (mgr.24625) 10230 : cluster [DBG] pgmap v5831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.327322+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.328016+0000 mon.a (mon.0) 3433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:06:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.481784+0000 mon.a (mon.0) 3434 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.482588+0000 mon.a (mon.0) 3435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.483299+0000 mon.a (mon.0) 3436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:06:58 gibba011 bash[31774]: audit 2021-03-10T17:06:57.484029+0000 mon.a (mon.0) 3437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: cluster 2021-03-10T17:06:56.993421+0000 mgr.gibba032.whopeo (mgr.24625) 10230 : cluster [DBG] pgmap v5831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:06:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.327322+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:06:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.328016+0000 mon.a (mon.0) 3433 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:06:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.481784+0000 mon.a (mon.0) 3434 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:06:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.482588+0000 mon.a (mon.0) 3435 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:06:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.483299+0000 mon.a (mon.0) 3436 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:06:58 gibba032 bash[24004]: audit 2021-03-10T17:06:57.484029+0000 mon.a (mon.0) 3437 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:06:58.780 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:06:59.039 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:06:59.039 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:06:59.039 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:06:59.039 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:06:59.040 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:06:59.040 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:06:59.040 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:06:59.040 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:06:59.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:06:59.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:06:59.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:06:59.042 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:06:59.043 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:06:59.043 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:06:59.043 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:06:59.043 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:06:59.295 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:06:59.295 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:06:59.295 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:06:59.295 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:59.296 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:06:59.296 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:06:59.296 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:59.296 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:06:59.296 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:06:59.297 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:06:59.297 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:06:59.297 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:06:59.297 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:06:59.298 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:06:59.298 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:07:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:00 gibba032 bash[24004]: audit 2021-03-10T17:06:58.769837+0000 mgr.gibba032.whopeo (mgr.24625) 10231 : audit [DBG] from='client.50913 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:00 gibba032 bash[24004]: cluster 2021-03-10T17:06:58.994928+0000 mgr.gibba032.whopeo (mgr.24625) 10232 : cluster [DBG] pgmap v5832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:00 gibba032 bash[24004]: audit 2021-03-10T17:06:58.998384+0000 mgr.gibba032.whopeo (mgr.24625) 10233 : audit [DBG] from='client.50919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:00 gibba032 bash[24004]: audit 2021-03-10T17:06:59.295271+0000 mon.a (mon.0) 3438 : audit [DBG] from='client.? 172.21.2.111:0/2451370841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[30791]: audit 2021-03-10T17:06:58.769837+0000 mgr.gibba032.whopeo (mgr.24625) 10231 : audit [DBG] from='client.50913 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[30791]: cluster 2021-03-10T17:06:58.994928+0000 mgr.gibba032.whopeo (mgr.24625) 10232 : cluster [DBG] pgmap v5832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[30791]: audit 2021-03-10T17:06:58.998384+0000 mgr.gibba032.whopeo (mgr.24625) 10233 : audit [DBG] from='client.50919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[30791]: audit 2021-03-10T17:06:59.295271+0000 mon.a (mon.0) 3438 : audit [DBG] from='client.? 172.21.2.111:0/2451370841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[31774]: audit 2021-03-10T17:06:58.769837+0000 mgr.gibba032.whopeo (mgr.24625) 10231 : audit [DBG] from='client.50913 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[31774]: cluster 2021-03-10T17:06:58.994928+0000 mgr.gibba032.whopeo (mgr.24625) 10232 : cluster [DBG] pgmap v5832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[31774]: audit 2021-03-10T17:06:58.998384+0000 mgr.gibba032.whopeo (mgr.24625) 10233 : audit [DBG] from='client.50919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:00 gibba011 bash[31774]: audit 2021-03-10T17:06:59.295271+0000 mon.a (mon.0) 3438 : audit [DBG] from='client.? 172.21.2.111:0/2451370841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:01 gibba032 bash[24004]: cluster 2021-03-10T17:07:00.996337+0000 mgr.gibba032.whopeo (mgr.24625) 10234 : cluster [DBG] pgmap v5833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:01 gibba011 bash[30791]: cluster 2021-03-10T17:07:00.996337+0000 mgr.gibba032.whopeo (mgr.24625) 10234 : cluster [DBG] pgmap v5833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:01 gibba011 bash[31774]: cluster 2021-03-10T17:07:00.996337+0000 mgr.gibba032.whopeo (mgr.24625) 10234 : cluster [DBG] pgmap v5833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:04.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:04 gibba011 bash[30791]: cluster 2021-03-10T17:07:02.998039+0000 mgr.gibba032.whopeo (mgr.24625) 10235 : cluster [DBG] pgmap v5834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:04.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:04 gibba011 bash[31774]: cluster 2021-03-10T17:07:02.998039+0000 mgr.gibba032.whopeo (mgr.24625) 10235 : cluster [DBG] pgmap v5834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:04 gibba032 bash[24004]: cluster 2021-03-10T17:07:02.998039+0000 mgr.gibba032.whopeo (mgr.24625) 10235 : cluster [DBG] pgmap v5834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:06.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:06 gibba011 bash[30791]: cluster 2021-03-10T17:07:04.999432+0000 mgr.gibba032.whopeo (mgr.24625) 10236 : cluster [DBG] pgmap v5835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:06.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:06 gibba011 bash[31774]: cluster 2021-03-10T17:07:04.999432+0000 mgr.gibba032.whopeo (mgr.24625) 10236 : cluster [DBG] pgmap v5835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:06 gibba032 bash[24004]: cluster 2021-03-10T17:07:04.999432+0000 mgr.gibba032.whopeo (mgr.24625) 10236 : cluster [DBG] pgmap v5835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:08 gibba032 bash[24004]: cluster 2021-03-10T17:07:07.001123+0000 mgr.gibba032.whopeo (mgr.24625) 10237 : cluster [DBG] pgmap v5836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:08 gibba011 bash[30791]: cluster 2021-03-10T17:07:07.001123+0000 mgr.gibba032.whopeo (mgr.24625) 10237 : cluster [DBG] pgmap v5836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:08 gibba011 bash[31774]: cluster 2021-03-10T17:07:07.001123+0000 mgr.gibba032.whopeo (mgr.24625) 10237 : cluster [DBG] pgmap v5836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:10 gibba032 bash[24004]: cluster 2021-03-10T17:07:09.002710+0000 mgr.gibba032.whopeo (mgr.24625) 10238 : cluster [DBG] pgmap v5837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:10 gibba011 bash[30791]: cluster 2021-03-10T17:07:09.002710+0000 mgr.gibba032.whopeo (mgr.24625) 10238 : cluster [DBG] pgmap v5837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:10 gibba011 bash[31774]: cluster 2021-03-10T17:07:09.002710+0000 mgr.gibba032.whopeo (mgr.24625) 10238 : cluster [DBG] pgmap v5837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:12.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:11 gibba032 bash[24004]: cluster 2021-03-10T17:07:11.004452+0000 mgr.gibba032.whopeo (mgr.24625) 10239 : cluster [DBG] pgmap v5838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:12.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:11 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.457539+0000 mgr.gibba032.whopeo (mgr.24625) 10240 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:07:12.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:11 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.459123+0000 mgr.gibba032.whopeo (mgr.24625) 10241 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:07:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[30791]: cluster 2021-03-10T17:07:11.004452+0000 mgr.gibba032.whopeo (mgr.24625) 10239 : cluster [DBG] pgmap v5838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.457539+0000 mgr.gibba032.whopeo (mgr.24625) 10240 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:07:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.459123+0000 mgr.gibba032.whopeo (mgr.24625) 10241 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:07:12.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[31774]: cluster 2021-03-10T17:07:11.004452+0000 mgr.gibba032.whopeo (mgr.24625) 10239 : cluster [DBG] pgmap v5838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:12.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.457539+0000 mgr.gibba032.whopeo (mgr.24625) 10240 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:07:12.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:11 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.459123+0000 mgr.gibba032.whopeo (mgr.24625) 10241 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:07:13.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.697399+0000 mon.a (mon.0) 3439 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:07:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.702076+0000 mgr.gibba032.whopeo (mgr.24625) 10242 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:07:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.704131+0000 mgr.gibba032.whopeo (mgr.24625) 10243 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:07:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.704563+0000 mon.a (mon.0) 3440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:07:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.706093+0000 mgr.gibba032.whopeo (mgr.24625) 10244 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:07:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.706137+0000 mgr.gibba032.whopeo (mgr.24625) 10245 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:07:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.706385+0000 mon.a (mon.0) 3441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.707603+0000 mgr.gibba032.whopeo (mgr.24625) 10246 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:07:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.707641+0000 mgr.gibba032.whopeo (mgr.24625) 10247 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:07:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.708390+0000 mon.a (mon.0) 3442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.709425+0000 mgr.gibba032.whopeo (mgr.24625) 10248 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:07:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.709872+0000 mon.a (mon.0) 3443 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.710635+0000 mgr.gibba032.whopeo (mgr.24625) 10249 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:07:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.710677+0000 mgr.gibba032.whopeo (mgr.24625) 10250 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:07:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.712076+0000 mgr.gibba032.whopeo (mgr.24625) 10251 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:07:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.712529+0000 mgr.gibba032.whopeo (mgr.24625) 10252 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:07:13.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.712966+0000 mon.a (mon.0) 3444 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.713733+0000 mgr.gibba032.whopeo (mgr.24625) 10253 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:07:13.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.713779+0000 mgr.gibba032.whopeo (mgr.24625) 10254 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:07:13.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: cephadm 2021-03-10T17:07:11.713903+0000 mgr.gibba032.whopeo (mgr.24625) 10255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:13.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:12 gibba032 bash[24004]: audit 2021-03-10T17:07:11.714630+0000 mon.a (mon.0) 3445 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.697399+0000 mon.a (mon.0) 3439 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:07:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.702076+0000 mgr.gibba032.whopeo (mgr.24625) 10242 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:07:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.704131+0000 mgr.gibba032.whopeo (mgr.24625) 10243 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:07:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.704563+0000 mon.a (mon.0) 3440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:07:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.706093+0000 mgr.gibba032.whopeo (mgr.24625) 10244 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.706137+0000 mgr.gibba032.whopeo (mgr.24625) 10245 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.706385+0000 mon.a (mon.0) 3441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.707603+0000 mgr.gibba032.whopeo (mgr.24625) 10246 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.707641+0000 mgr.gibba032.whopeo (mgr.24625) 10247 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:07:13.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.708390+0000 mon.a (mon.0) 3442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.709425+0000 mgr.gibba032.whopeo (mgr.24625) 10248 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:07:13.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.709872+0000 mon.a (mon.0) 3443 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.710635+0000 mgr.gibba032.whopeo (mgr.24625) 10249 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:07:13.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.710677+0000 mgr.gibba032.whopeo (mgr.24625) 10250 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:07:13.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.712076+0000 mgr.gibba032.whopeo (mgr.24625) 10251 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:07:13.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.712529+0000 mgr.gibba032.whopeo (mgr.24625) 10252 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:07:13.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.712966+0000 mon.a (mon.0) 3444 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.713733+0000 mgr.gibba032.whopeo (mgr.24625) 10253 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:07:13.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.713779+0000 mgr.gibba032.whopeo (mgr.24625) 10254 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:07:13.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: cephadm 2021-03-10T17:07:11.713903+0000 mgr.gibba032.whopeo (mgr.24625) 10255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:13.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[30791]: audit 2021-03-10T17:07:11.714630+0000 mon.a (mon.0) 3445 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.697399+0000 mon.a (mon.0) 3439 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:07:13.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.702076+0000 mgr.gibba032.whopeo (mgr.24625) 10242 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:07:13.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.704131+0000 mgr.gibba032.whopeo (mgr.24625) 10243 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:07:13.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.704563+0000 mon.a (mon.0) 3440 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:07:13.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.706093+0000 mgr.gibba032.whopeo (mgr.24625) 10244 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:07:13.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.706137+0000 mgr.gibba032.whopeo (mgr.24625) 10245 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:07:13.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.706385+0000 mon.a (mon.0) 3441 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.707603+0000 mgr.gibba032.whopeo (mgr.24625) 10246 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:07:13.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.707641+0000 mgr.gibba032.whopeo (mgr.24625) 10247 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:07:13.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.708390+0000 mon.a (mon.0) 3442 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.709425+0000 mgr.gibba032.whopeo (mgr.24625) 10248 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:07:13.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.709872+0000 mon.a (mon.0) 3443 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.710635+0000 mgr.gibba032.whopeo (mgr.24625) 10249 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:07:13.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.710677+0000 mgr.gibba032.whopeo (mgr.24625) 10250 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:07:13.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.712076+0000 mgr.gibba032.whopeo (mgr.24625) 10251 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:07:13.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.712529+0000 mgr.gibba032.whopeo (mgr.24625) 10252 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:07:13.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.712966+0000 mon.a (mon.0) 3444 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:13.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.713733+0000 mgr.gibba032.whopeo (mgr.24625) 10253 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:07:13.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.713779+0000 mgr.gibba032.whopeo (mgr.24625) 10254 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:07:13.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: cephadm 2021-03-10T17:07:11.713903+0000 mgr.gibba032.whopeo (mgr.24625) 10255 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:13.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:12 gibba011 bash[31774]: audit 2021-03-10T17:07:11.714630+0000 mon.a (mon.0) 3445 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:14.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:13 gibba032 bash[24004]: cluster 2021-03-10T17:07:13.006312+0000 mgr.gibba032.whopeo (mgr.24625) 10256 : cluster [DBG] pgmap v5839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:14.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:13 gibba011 bash[30791]: cluster 2021-03-10T17:07:13.006312+0000 mgr.gibba032.whopeo (mgr.24625) 10256 : cluster [DBG] pgmap v5839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:14.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:13 gibba011 bash[31774]: cluster 2021-03-10T17:07:13.006312+0000 mgr.gibba032.whopeo (mgr.24625) 10256 : cluster [DBG] pgmap v5839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:16.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:16 gibba011 bash[30791]: cluster 2021-03-10T17:07:15.007690+0000 mgr.gibba032.whopeo (mgr.24625) 10257 : cluster [DBG] pgmap v5840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:16.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:16 gibba011 bash[31774]: cluster 2021-03-10T17:07:15.007690+0000 mgr.gibba032.whopeo (mgr.24625) 10257 : cluster [DBG] pgmap v5840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:16 gibba032 bash[24004]: cluster 2021-03-10T17:07:15.007690+0000 mgr.gibba032.whopeo (mgr.24625) 10257 : cluster [DBG] pgmap v5840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:18 gibba032 bash[24004]: cluster 2021-03-10T17:07:17.009490+0000 mgr.gibba032.whopeo (mgr.24625) 10258 : cluster [DBG] pgmap v5841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:18 gibba011 bash[30791]: cluster 2021-03-10T17:07:17.009490+0000 mgr.gibba032.whopeo (mgr.24625) 10258 : cluster [DBG] pgmap v5841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:18 gibba011 bash[31774]: cluster 2021-03-10T17:07:17.009490+0000 mgr.gibba032.whopeo (mgr.24625) 10258 : cluster [DBG] pgmap v5841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:20 gibba032 bash[24004]: cluster 2021-03-10T17:07:19.011092+0000 mgr.gibba032.whopeo (mgr.24625) 10259 : cluster [DBG] pgmap v5842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:20 gibba011 bash[30791]: cluster 2021-03-10T17:07:19.011092+0000 mgr.gibba032.whopeo (mgr.24625) 10259 : cluster [DBG] pgmap v5842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:20 gibba011 bash[31774]: cluster 2021-03-10T17:07:19.011092+0000 mgr.gibba032.whopeo (mgr.24625) 10259 : cluster [DBG] pgmap v5842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:21 gibba032 bash[24004]: cluster 2021-03-10T17:07:21.012498+0000 mgr.gibba032.whopeo (mgr.24625) 10260 : cluster [DBG] pgmap v5843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:21 gibba011 bash[30791]: cluster 2021-03-10T17:07:21.012498+0000 mgr.gibba032.whopeo (mgr.24625) 10260 : cluster [DBG] pgmap v5843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:21 gibba011 bash[31774]: cluster 2021-03-10T17:07:21.012498+0000 mgr.gibba032.whopeo (mgr.24625) 10260 : cluster [DBG] pgmap v5843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:24.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:24 gibba011 bash[30791]: cluster 2021-03-10T17:07:23.014172+0000 mgr.gibba032.whopeo (mgr.24625) 10261 : cluster [DBG] pgmap v5844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:24.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:24 gibba011 bash[31774]: cluster 2021-03-10T17:07:23.014172+0000 mgr.gibba032.whopeo (mgr.24625) 10261 : cluster [DBG] pgmap v5844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:24 gibba032 bash[24004]: cluster 2021-03-10T17:07:23.014172+0000 mgr.gibba032.whopeo (mgr.24625) 10261 : cluster [DBG] pgmap v5844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:26 gibba032 bash[24004]: cluster 2021-03-10T17:07:25.015284+0000 mgr.gibba032.whopeo (mgr.24625) 10262 : cluster [DBG] pgmap v5845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:26.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:26 gibba011 bash[30791]: cluster 2021-03-10T17:07:25.015284+0000 mgr.gibba032.whopeo (mgr.24625) 10262 : cluster [DBG] pgmap v5845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:26 gibba011 bash[31774]: cluster 2021-03-10T17:07:25.015284+0000 mgr.gibba032.whopeo (mgr.24625) 10262 : cluster [DBG] pgmap v5845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:28 gibba032 bash[24004]: cephadm 2021-03-10T17:07:26.729727+0000 mgr.gibba032.whopeo (mgr.24625) 10263 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:28 gibba032 bash[24004]: cluster 2021-03-10T17:07:27.017005+0000 mgr.gibba032.whopeo (mgr.24625) 10264 : cluster [DBG] pgmap v5846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:28 gibba011 bash[30791]: cephadm 2021-03-10T17:07:26.729727+0000 mgr.gibba032.whopeo (mgr.24625) 10263 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:28 gibba011 bash[30791]: cluster 2021-03-10T17:07:27.017005+0000 mgr.gibba032.whopeo (mgr.24625) 10264 : cluster [DBG] pgmap v5846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:28 gibba011 bash[31774]: cephadm 2021-03-10T17:07:26.729727+0000 mgr.gibba032.whopeo (mgr.24625) 10263 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:28 gibba011 bash[31774]: cluster 2021-03-10T17:07:27.017005+0000 mgr.gibba032.whopeo (mgr.24625) 10264 : cluster [DBG] pgmap v5846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:29.536 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:07:29.762 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:07:29.762 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:07:29.762 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:07:29.763 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:07:29.763 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:07:29.763 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:07:29.763 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:07:29.763 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:07:29.764 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:07:29.764 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:07:29.764 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:07:29.764 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:07:29.764 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:07:29.765 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:07:29.765 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:07:29.765 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:07:29.765 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:07:29.766 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:07:29.766 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:07:29.766 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:07:29.766 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:07:30.011 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:07:30.012 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:07:30.012 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:07:30.012 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:07:30.012 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:07:30.013 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:07:30.013 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:07:30.013 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:07:30.013 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:07:30.013 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:07:30.014 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:07:30.014 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:07:30.014 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:07:30.014 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:07:30.014 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:07:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:30 gibba032 bash[24004]: cluster 2021-03-10T17:07:29.018485+0000 mgr.gibba032.whopeo (mgr.24625) 10265 : cluster [DBG] pgmap v5847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:30 gibba032 bash[24004]: audit 2021-03-10T17:07:29.525497+0000 mgr.gibba032.whopeo (mgr.24625) 10266 : audit [DBG] from='client.50931 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:30 gibba032 bash[24004]: audit 2021-03-10T17:07:30.011825+0000 mon.c (mon.1) 197 : audit [DBG] from='client.? 172.21.2.111:0/2158282289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[30791]: cluster 2021-03-10T17:07:29.018485+0000 mgr.gibba032.whopeo (mgr.24625) 10265 : cluster [DBG] pgmap v5847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[30791]: audit 2021-03-10T17:07:29.525497+0000 mgr.gibba032.whopeo (mgr.24625) 10266 : audit [DBG] from='client.50931 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[30791]: audit 2021-03-10T17:07:30.011825+0000 mon.c (mon.1) 197 : audit [DBG] from='client.? 172.21.2.111:0/2158282289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[31774]: cluster 2021-03-10T17:07:29.018485+0000 mgr.gibba032.whopeo (mgr.24625) 10265 : cluster [DBG] pgmap v5847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[31774]: audit 2021-03-10T17:07:29.525497+0000 mgr.gibba032.whopeo (mgr.24625) 10266 : audit [DBG] from='client.50931 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:30 gibba011 bash[31774]: audit 2021-03-10T17:07:30.011825+0000 mon.c (mon.1) 197 : audit [DBG] from='client.? 172.21.2.111:0/2158282289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:07:31.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:31 gibba032 bash[24004]: audit 2021-03-10T17:07:29.748853+0000 mgr.gibba032.whopeo (mgr.24625) 10267 : audit [DBG] from='client.60919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:31.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:31 gibba011 bash[30791]: audit 2021-03-10T17:07:29.748853+0000 mgr.gibba032.whopeo (mgr.24625) 10267 : audit [DBG] from='client.60919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:31.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:31 gibba011 bash[31774]: audit 2021-03-10T17:07:29.748853+0000 mgr.gibba032.whopeo (mgr.24625) 10267 : audit [DBG] from='client.60919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:07:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:32 gibba032 bash[24004]: cluster 2021-03-10T17:07:31.019872+0000 mgr.gibba032.whopeo (mgr.24625) 10268 : cluster [DBG] pgmap v5848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:32 gibba011 bash[30791]: cluster 2021-03-10T17:07:31.019872+0000 mgr.gibba032.whopeo (mgr.24625) 10268 : cluster [DBG] pgmap v5848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:32 gibba011 bash[31774]: cluster 2021-03-10T17:07:31.019872+0000 mgr.gibba032.whopeo (mgr.24625) 10268 : cluster [DBG] pgmap v5848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:34 gibba032 bash[24004]: cluster 2021-03-10T17:07:33.021812+0000 mgr.gibba032.whopeo (mgr.24625) 10269 : cluster [DBG] pgmap v5849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:34 gibba011 bash[30791]: cluster 2021-03-10T17:07:33.021812+0000 mgr.gibba032.whopeo (mgr.24625) 10269 : cluster [DBG] pgmap v5849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:34 gibba011 bash[31774]: cluster 2021-03-10T17:07:33.021812+0000 mgr.gibba032.whopeo (mgr.24625) 10269 : cluster [DBG] pgmap v5849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:36 gibba032 bash[24004]: cluster 2021-03-10T17:07:35.023150+0000 mgr.gibba032.whopeo (mgr.24625) 10270 : cluster [DBG] pgmap v5850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:36.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:36 gibba011 bash[30791]: cluster 2021-03-10T17:07:35.023150+0000 mgr.gibba032.whopeo (mgr.24625) 10270 : cluster [DBG] pgmap v5850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:36 gibba011 bash[31774]: cluster 2021-03-10T17:07:35.023150+0000 mgr.gibba032.whopeo (mgr.24625) 10270 : cluster [DBG] pgmap v5850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:38 gibba032 bash[24004]: cluster 2021-03-10T17:07:37.024972+0000 mgr.gibba032.whopeo (mgr.24625) 10271 : cluster [DBG] pgmap v5851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:38 gibba011 bash[30791]: cluster 2021-03-10T17:07:37.024972+0000 mgr.gibba032.whopeo (mgr.24625) 10271 : cluster [DBG] pgmap v5851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:38 gibba011 bash[31774]: cluster 2021-03-10T17:07:37.024972+0000 mgr.gibba032.whopeo (mgr.24625) 10271 : cluster [DBG] pgmap v5851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:40 gibba032 bash[24004]: cluster 2021-03-10T17:07:39.026407+0000 mgr.gibba032.whopeo (mgr.24625) 10272 : cluster [DBG] pgmap v5852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:40 gibba011 bash[30791]: cluster 2021-03-10T17:07:39.026407+0000 mgr.gibba032.whopeo (mgr.24625) 10272 : cluster [DBG] pgmap v5852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:40 gibba011 bash[31774]: cluster 2021-03-10T17:07:39.026407+0000 mgr.gibba032.whopeo (mgr.24625) 10272 : cluster [DBG] pgmap v5852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:41 gibba032 bash[24004]: cluster 2021-03-10T17:07:41.028183+0000 mgr.gibba032.whopeo (mgr.24625) 10273 : cluster [DBG] pgmap v5853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:41 gibba011 bash[30791]: cluster 2021-03-10T17:07:41.028183+0000 mgr.gibba032.whopeo (mgr.24625) 10273 : cluster [DBG] pgmap v5853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:41 gibba011 bash[31774]: cluster 2021-03-10T17:07:41.028183+0000 mgr.gibba032.whopeo (mgr.24625) 10273 : cluster [DBG] pgmap v5853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:43.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:42 gibba032 bash[24004]: cephadm 2021-03-10T17:07:41.740581+0000 mgr.gibba032.whopeo (mgr.24625) 10274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:42 gibba011 bash[30791]: cephadm 2021-03-10T17:07:41.740581+0000 mgr.gibba032.whopeo (mgr.24625) 10274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:43.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:42 gibba011 bash[31774]: cephadm 2021-03-10T17:07:41.740581+0000 mgr.gibba032.whopeo (mgr.24625) 10274 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:44.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:43 gibba032 bash[24004]: cluster 2021-03-10T17:07:43.030066+0000 mgr.gibba032.whopeo (mgr.24625) 10275 : cluster [DBG] pgmap v5854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:44.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:43 gibba011 bash[30791]: cluster 2021-03-10T17:07:43.030066+0000 mgr.gibba032.whopeo (mgr.24625) 10275 : cluster [DBG] pgmap v5854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:44.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:43 gibba011 bash[31774]: cluster 2021-03-10T17:07:43.030066+0000 mgr.gibba032.whopeo (mgr.24625) 10275 : cluster [DBG] pgmap v5854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:46 gibba032 bash[24004]: cluster 2021-03-10T17:07:45.031352+0000 mgr.gibba032.whopeo (mgr.24625) 10276 : cluster [DBG] pgmap v5855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:46 gibba011 bash[30791]: cluster 2021-03-10T17:07:45.031352+0000 mgr.gibba032.whopeo (mgr.24625) 10276 : cluster [DBG] pgmap v5855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:46 gibba011 bash[31774]: cluster 2021-03-10T17:07:45.031352+0000 mgr.gibba032.whopeo (mgr.24625) 10276 : cluster [DBG] pgmap v5855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:48 gibba032 bash[24004]: cluster 2021-03-10T17:07:47.033088+0000 mgr.gibba032.whopeo (mgr.24625) 10277 : cluster [DBG] pgmap v5856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:48.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:48 gibba011 bash[30791]: cluster 2021-03-10T17:07:47.033088+0000 mgr.gibba032.whopeo (mgr.24625) 10277 : cluster [DBG] pgmap v5856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:48 gibba011 bash[31774]: cluster 2021-03-10T17:07:47.033088+0000 mgr.gibba032.whopeo (mgr.24625) 10277 : cluster [DBG] pgmap v5856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:50 gibba032 bash[24004]: cluster 2021-03-10T17:07:49.034568+0000 mgr.gibba032.whopeo (mgr.24625) 10278 : cluster [DBG] pgmap v5857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:50 gibba011 bash[30791]: cluster 2021-03-10T17:07:49.034568+0000 mgr.gibba032.whopeo (mgr.24625) 10278 : cluster [DBG] pgmap v5857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:50 gibba011 bash[31774]: cluster 2021-03-10T17:07:49.034568+0000 mgr.gibba032.whopeo (mgr.24625) 10278 : cluster [DBG] pgmap v5857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:52.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:51 gibba032 bash[24004]: cluster 2021-03-10T17:07:51.035951+0000 mgr.gibba032.whopeo (mgr.24625) 10279 : cluster [DBG] pgmap v5858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:52.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:51 gibba011 bash[30791]: cluster 2021-03-10T17:07:51.035951+0000 mgr.gibba032.whopeo (mgr.24625) 10279 : cluster [DBG] pgmap v5858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:51 gibba011 bash[31774]: cluster 2021-03-10T17:07:51.035951+0000 mgr.gibba032.whopeo (mgr.24625) 10279 : cluster [DBG] pgmap v5858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:54 gibba032 bash[24004]: cluster 2021-03-10T17:07:53.037618+0000 mgr.gibba032.whopeo (mgr.24625) 10280 : cluster [DBG] pgmap v5859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:54 gibba011 bash[30791]: cluster 2021-03-10T17:07:53.037618+0000 mgr.gibba032.whopeo (mgr.24625) 10280 : cluster [DBG] pgmap v5859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:54 gibba011 bash[31774]: cluster 2021-03-10T17:07:53.037618+0000 mgr.gibba032.whopeo (mgr.24625) 10280 : cluster [DBG] pgmap v5859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:56 gibba032 bash[24004]: cluster 2021-03-10T17:07:55.038742+0000 mgr.gibba032.whopeo (mgr.24625) 10281 : cluster [DBG] pgmap v5860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:56.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:56 gibba011 bash[30791]: cluster 2021-03-10T17:07:55.038742+0000 mgr.gibba032.whopeo (mgr.24625) 10281 : cluster [DBG] pgmap v5860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:56 gibba011 bash[31774]: cluster 2021-03-10T17:07:55.038742+0000 mgr.gibba032.whopeo (mgr.24625) 10281 : cluster [DBG] pgmap v5860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: cephadm 2021-03-10T17:07:56.756395+0000 mgr.gibba032.whopeo (mgr.24625) 10282 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: cluster 2021-03-10T17:07:57.040570+0000 mgr.gibba032.whopeo (mgr.24625) 10283 : cluster [DBG] pgmap v5861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.334823+0000 mon.a (mon.0) 3446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:07:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.335564+0000 mon.a (mon.0) 3447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:07:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.494574+0000 mon.a (mon.0) 3448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:07:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.495313+0000 mon.a (mon.0) 3449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:07:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.496160+0000 mon.a (mon.0) 3450 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:07:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:07:58 gibba032 bash[24004]: audit 2021-03-10T17:07:57.496921+0000 mon.a (mon.0) 3451 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:07:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: cephadm 2021-03-10T17:07:56.756395+0000 mgr.gibba032.whopeo (mgr.24625) 10282 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: cluster 2021-03-10T17:07:57.040570+0000 mgr.gibba032.whopeo (mgr.24625) 10283 : cluster [DBG] pgmap v5861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.334823+0000 mon.a (mon.0) 3446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:07:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.335564+0000 mon.a (mon.0) 3447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:07:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.494574+0000 mon.a (mon.0) 3448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:07:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.495313+0000 mon.a (mon.0) 3449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:07:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.496160+0000 mon.a (mon.0) 3450 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:07:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[30791]: audit 2021-03-10T17:07:57.496921+0000 mon.a (mon.0) 3451 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:07:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: cephadm 2021-03-10T17:07:56.756395+0000 mgr.gibba032.whopeo (mgr.24625) 10282 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:07:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: cluster 2021-03-10T17:07:57.040570+0000 mgr.gibba032.whopeo (mgr.24625) 10283 : cluster [DBG] pgmap v5861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:07:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.334823+0000 mon.a (mon.0) 3446 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:07:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.335564+0000 mon.a (mon.0) 3447 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:07:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.494574+0000 mon.a (mon.0) 3448 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:07:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.495313+0000 mon.a (mon.0) 3449 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:07:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.496160+0000 mon.a (mon.0) 3450 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:07:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:07:58 gibba011 bash[31774]: audit 2021-03-10T17:07:57.496921+0000 mon.a (mon.0) 3451 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:08:00.251 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:08:00.480 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:08:00.480 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:08:00.481 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:08:00.481 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:08:00.481 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:08:00.481 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:08:00.482 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:08:00.482 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:08:00.482 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:08:00.482 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:08:00.483 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:08:00.483 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:08:00.483 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:08:00.483 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:08:00.484 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:08:00.484 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:08:00.484 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:08:00.484 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:08:00.484 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:08:00.485 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:08:00.485 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:08:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:00 gibba032 bash[24004]: cluster 2021-03-10T17:07:59.041944+0000 mgr.gibba032.whopeo (mgr.24625) 10284 : cluster [DBG] pgmap v5862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:00 gibba011 bash[30791]: cluster 2021-03-10T17:07:59.041944+0000 mgr.gibba032.whopeo (mgr.24625) 10284 : cluster [DBG] pgmap v5862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:00 gibba011 bash[31774]: cluster 2021-03-10T17:07:59.041944+0000 mgr.gibba032.whopeo (mgr.24625) 10284 : cluster [DBG] pgmap v5862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:00.730 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:08:00.731 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:08:00.731 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:08:00.731 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:00.731 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:08:00.731 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:08:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:08:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:08:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:08:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:08:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:08:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:08:00.733 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:08:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:01 gibba032 bash[24004]: audit 2021-03-10T17:08:00.240761+0000 mgr.gibba032.whopeo (mgr.24625) 10285 : audit [DBG] from='client.50949 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:01 gibba032 bash[24004]: audit 2021-03-10T17:08:00.466117+0000 mgr.gibba032.whopeo (mgr.24625) 10286 : audit [DBG] from='client.50955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:01 gibba032 bash[24004]: audit 2021-03-10T17:08:00.730770+0000 mon.a (mon.0) 3452 : audit [DBG] from='client.? 172.21.2.111:0/2052660143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:01.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[30791]: audit 2021-03-10T17:08:00.240761+0000 mgr.gibba032.whopeo (mgr.24625) 10285 : audit [DBG] from='client.50949 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[30791]: audit 2021-03-10T17:08:00.466117+0000 mgr.gibba032.whopeo (mgr.24625) 10286 : audit [DBG] from='client.50955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[30791]: audit 2021-03-10T17:08:00.730770+0000 mon.a (mon.0) 3452 : audit [DBG] from='client.? 172.21.2.111:0/2052660143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[31774]: audit 2021-03-10T17:08:00.240761+0000 mgr.gibba032.whopeo (mgr.24625) 10285 : audit [DBG] from='client.50949 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[31774]: audit 2021-03-10T17:08:00.466117+0000 mgr.gibba032.whopeo (mgr.24625) 10286 : audit [DBG] from='client.50955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:01 gibba011 bash[31774]: audit 2021-03-10T17:08:00.730770+0000 mon.a (mon.0) 3452 : audit [DBG] from='client.? 172.21.2.111:0/2052660143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:02 gibba032 bash[24004]: cluster 2021-03-10T17:08:01.043334+0000 mgr.gibba032.whopeo (mgr.24625) 10287 : cluster [DBG] pgmap v5863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:02 gibba011 bash[30791]: cluster 2021-03-10T17:08:01.043334+0000 mgr.gibba032.whopeo (mgr.24625) 10287 : cluster [DBG] pgmap v5863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:02 gibba011 bash[31774]: cluster 2021-03-10T17:08:01.043334+0000 mgr.gibba032.whopeo (mgr.24625) 10287 : cluster [DBG] pgmap v5863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:04 gibba032 bash[24004]: cluster 2021-03-10T17:08:03.045010+0000 mgr.gibba032.whopeo (mgr.24625) 10288 : cluster [DBG] pgmap v5864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:04 gibba011 bash[30791]: cluster 2021-03-10T17:08:03.045010+0000 mgr.gibba032.whopeo (mgr.24625) 10288 : cluster [DBG] pgmap v5864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:04 gibba011 bash[31774]: cluster 2021-03-10T17:08:03.045010+0000 mgr.gibba032.whopeo (mgr.24625) 10288 : cluster [DBG] pgmap v5864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:06 gibba032 bash[24004]: cluster 2021-03-10T17:08:05.046367+0000 mgr.gibba032.whopeo (mgr.24625) 10289 : cluster [DBG] pgmap v5865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:06 gibba011 bash[30791]: cluster 2021-03-10T17:08:05.046367+0000 mgr.gibba032.whopeo (mgr.24625) 10289 : cluster [DBG] pgmap v5865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:06 gibba011 bash[31774]: cluster 2021-03-10T17:08:05.046367+0000 mgr.gibba032.whopeo (mgr.24625) 10289 : cluster [DBG] pgmap v5865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:08 gibba032 bash[24004]: cluster 2021-03-10T17:08:07.048156+0000 mgr.gibba032.whopeo (mgr.24625) 10290 : cluster [DBG] pgmap v5866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:08 gibba011 bash[30791]: cluster 2021-03-10T17:08:07.048156+0000 mgr.gibba032.whopeo (mgr.24625) 10290 : cluster [DBG] pgmap v5866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:08 gibba011 bash[31774]: cluster 2021-03-10T17:08:07.048156+0000 mgr.gibba032.whopeo (mgr.24625) 10290 : cluster [DBG] pgmap v5866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:10 gibba032 bash[24004]: cluster 2021-03-10T17:08:09.049478+0000 mgr.gibba032.whopeo (mgr.24625) 10291 : cluster [DBG] pgmap v5867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:10 gibba011 bash[30791]: cluster 2021-03-10T17:08:09.049478+0000 mgr.gibba032.whopeo (mgr.24625) 10291 : cluster [DBG] pgmap v5867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:10 gibba011 bash[31774]: cluster 2021-03-10T17:08:09.049478+0000 mgr.gibba032.whopeo (mgr.24625) 10291 : cluster [DBG] pgmap v5867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:12.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:11 gibba032 bash[24004]: cluster 2021-03-10T17:08:11.050855+0000 mgr.gibba032.whopeo (mgr.24625) 10292 : cluster [DBG] pgmap v5868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:12.078 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:11 gibba011 bash[31774]: cluster 2021-03-10T17:08:11.050855+0000 mgr.gibba032.whopeo (mgr.24625) 10292 : cluster [DBG] pgmap v5868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:11 gibba011 bash[30791]: cluster 2021-03-10T17:08:11.050855+0000 mgr.gibba032.whopeo (mgr.24625) 10292 : cluster [DBG] pgmap v5868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:11.959251+0000 mgr.gibba032.whopeo (mgr.24625) 10293 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:08:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:11.960353+0000 mon.a (mon.0) 3453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:11.972158+0000 mgr.gibba032.whopeo (mgr.24625) 10294 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:08:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:11.973301+0000 mon.a (mon.0) 3454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.177503+0000 mon.a (mon.0) 3455 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.178330+0000 mgr.gibba032.whopeo (mgr.24625) 10295 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:08:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.180168+0000 mgr.gibba032.whopeo (mgr.24625) 10296 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:08:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.180796+0000 mon.a (mon.0) 3456 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:08:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.181395+0000 mgr.gibba032.whopeo (mgr.24625) 10297 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:08:13.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.181442+0000 mgr.gibba032.whopeo (mgr.24625) 10298 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.182272+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.182796+0000 mgr.gibba032.whopeo (mgr.24625) 10299 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.182842+0000 mgr.gibba032.whopeo (mgr.24625) 10300 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.183640+0000 mon.a (mon.0) 3458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.184051+0000 mgr.gibba032.whopeo (mgr.24625) 10301 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:08:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.184787+0000 mgr.gibba032.whopeo (mgr.24625) 10302 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:08:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.184828+0000 mgr.gibba032.whopeo (mgr.24625) 10303 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:08:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.185056+0000 mon.a (mon.0) 3459 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.186052+0000 mgr.gibba032.whopeo (mgr.24625) 10304 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:08:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.186464+0000 mgr.gibba032.whopeo (mgr.24625) 10305 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:08:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.187044+0000 mon.a (mon.0) 3460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.187627+0000 mgr.gibba032.whopeo (mgr.24625) 10306 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:08:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.187677+0000 mgr.gibba032.whopeo (mgr.24625) 10307 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:08:13.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: cephadm 2021-03-10T17:08:12.187813+0000 mgr.gibba032.whopeo (mgr.24625) 10308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:13.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:12 gibba032 bash[24004]: audit 2021-03-10T17:08:12.188602+0000 mon.a (mon.0) 3461 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:11.959251+0000 mgr.gibba032.whopeo (mgr.24625) 10293 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:08:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:11.960353+0000 mon.a (mon.0) 3453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:11.972158+0000 mgr.gibba032.whopeo (mgr.24625) 10294 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:08:13.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:11.973301+0000 mon.a (mon.0) 3454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.177503+0000 mon.a (mon.0) 3455 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.178330+0000 mgr.gibba032.whopeo (mgr.24625) 10295 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:08:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.180168+0000 mgr.gibba032.whopeo (mgr.24625) 10296 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:08:13.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.180796+0000 mon.a (mon.0) 3456 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:08:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.181395+0000 mgr.gibba032.whopeo (mgr.24625) 10297 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:08:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.181442+0000 mgr.gibba032.whopeo (mgr.24625) 10298 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:08:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.182272+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.182796+0000 mgr.gibba032.whopeo (mgr.24625) 10299 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:08:13.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.182842+0000 mgr.gibba032.whopeo (mgr.24625) 10300 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:08:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.183640+0000 mon.a (mon.0) 3458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.184051+0000 mgr.gibba032.whopeo (mgr.24625) 10301 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:08:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.184787+0000 mgr.gibba032.whopeo (mgr.24625) 10302 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:08:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.184828+0000 mgr.gibba032.whopeo (mgr.24625) 10303 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:08:13.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.185056+0000 mon.a (mon.0) 3459 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.186052+0000 mgr.gibba032.whopeo (mgr.24625) 10304 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:08:13.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.186464+0000 mgr.gibba032.whopeo (mgr.24625) 10305 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:08:13.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.187044+0000 mon.a (mon.0) 3460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:11.959251+0000 mgr.gibba032.whopeo (mgr.24625) 10293 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:08:13.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:11.960353+0000 mon.a (mon.0) 3453 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:11.972158+0000 mgr.gibba032.whopeo (mgr.24625) 10294 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:08:13.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:11.973301+0000 mon.a (mon.0) 3454 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.177503+0000 mon.a (mon.0) 3455 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:08:13.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.178330+0000 mgr.gibba032.whopeo (mgr.24625) 10295 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:08:13.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.180168+0000 mgr.gibba032.whopeo (mgr.24625) 10296 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:08:13.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.180796+0000 mon.a (mon.0) 3456 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:08:13.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.181395+0000 mgr.gibba032.whopeo (mgr.24625) 10297 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:08:13.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.181442+0000 mgr.gibba032.whopeo (mgr.24625) 10298 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:08:13.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.182272+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.182796+0000 mgr.gibba032.whopeo (mgr.24625) 10299 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:08:13.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.182842+0000 mgr.gibba032.whopeo (mgr.24625) 10300 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:08:13.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.183640+0000 mon.a (mon.0) 3458 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.184051+0000 mgr.gibba032.whopeo (mgr.24625) 10301 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:08:13.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.184787+0000 mgr.gibba032.whopeo (mgr.24625) 10302 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:08:13.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.184828+0000 mgr.gibba032.whopeo (mgr.24625) 10303 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:08:13.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.185056+0000 mon.a (mon.0) 3459 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.186052+0000 mgr.gibba032.whopeo (mgr.24625) 10304 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:08:13.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.186464+0000 mgr.gibba032.whopeo (mgr.24625) 10305 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:08:13.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.187044+0000 mon.a (mon.0) 3460 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.187627+0000 mgr.gibba032.whopeo (mgr.24625) 10306 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:08:13.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.187677+0000 mgr.gibba032.whopeo (mgr.24625) 10307 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:08:13.340 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: cephadm 2021-03-10T17:08:12.187813+0000 mgr.gibba032.whopeo (mgr.24625) 10308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:13.341 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[30791]: audit 2021-03-10T17:08:12.188602+0000 mon.a (mon.0) 3461 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:13.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.187627+0000 mgr.gibba032.whopeo (mgr.24625) 10306 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:08:13.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.187677+0000 mgr.gibba032.whopeo (mgr.24625) 10307 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:08:13.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: cephadm 2021-03-10T17:08:12.187813+0000 mgr.gibba032.whopeo (mgr.24625) 10308 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:13.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:12 gibba011 bash[31774]: audit 2021-03-10T17:08:12.188602+0000 mon.a (mon.0) 3461 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:13 gibba032 bash[24004]: cluster 2021-03-10T17:08:13.052701+0000 mgr.gibba032.whopeo (mgr.24625) 10309 : cluster [DBG] pgmap v5869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:13 gibba011 bash[30791]: cluster 2021-03-10T17:08:13.052701+0000 mgr.gibba032.whopeo (mgr.24625) 10309 : cluster [DBG] pgmap v5869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:13 gibba011 bash[31774]: cluster 2021-03-10T17:08:13.052701+0000 mgr.gibba032.whopeo (mgr.24625) 10309 : cluster [DBG] pgmap v5869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:16 gibba032 bash[24004]: cluster 2021-03-10T17:08:15.053880+0000 mgr.gibba032.whopeo (mgr.24625) 10310 : cluster [DBG] pgmap v5870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:16 gibba011 bash[30791]: cluster 2021-03-10T17:08:15.053880+0000 mgr.gibba032.whopeo (mgr.24625) 10310 : cluster [DBG] pgmap v5870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:16 gibba011 bash[31774]: cluster 2021-03-10T17:08:15.053880+0000 mgr.gibba032.whopeo (mgr.24625) 10310 : cluster [DBG] pgmap v5870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:18 gibba032 bash[24004]: cluster 2021-03-10T17:08:17.055551+0000 mgr.gibba032.whopeo (mgr.24625) 10311 : cluster [DBG] pgmap v5871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:18 gibba011 bash[30791]: cluster 2021-03-10T17:08:17.055551+0000 mgr.gibba032.whopeo (mgr.24625) 10311 : cluster [DBG] pgmap v5871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:18 gibba011 bash[31774]: cluster 2021-03-10T17:08:17.055551+0000 mgr.gibba032.whopeo (mgr.24625) 10311 : cluster [DBG] pgmap v5871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:20 gibba032 bash[24004]: cluster 2021-03-10T17:08:19.057173+0000 mgr.gibba032.whopeo (mgr.24625) 10312 : cluster [DBG] pgmap v5872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:20.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:20 gibba011 bash[30791]: cluster 2021-03-10T17:08:19.057173+0000 mgr.gibba032.whopeo (mgr.24625) 10312 : cluster [DBG] pgmap v5872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:20 gibba011 bash[31774]: cluster 2021-03-10T17:08:19.057173+0000 mgr.gibba032.whopeo (mgr.24625) 10312 : cluster [DBG] pgmap v5872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:21 gibba032 bash[24004]: cluster 2021-03-10T17:08:21.058477+0000 mgr.gibba032.whopeo (mgr.24625) 10313 : cluster [DBG] pgmap v5873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:21 gibba011 bash[30791]: cluster 2021-03-10T17:08:21.058477+0000 mgr.gibba032.whopeo (mgr.24625) 10313 : cluster [DBG] pgmap v5873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:21 gibba011 bash[31774]: cluster 2021-03-10T17:08:21.058477+0000 mgr.gibba032.whopeo (mgr.24625) 10313 : cluster [DBG] pgmap v5873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:24 gibba032 bash[24004]: cluster 2021-03-10T17:08:23.059371+0000 mgr.gibba032.whopeo (mgr.24625) 10314 : cluster [DBG] pgmap v5874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:24.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:24 gibba011 bash[30791]: cluster 2021-03-10T17:08:23.059371+0000 mgr.gibba032.whopeo (mgr.24625) 10314 : cluster [DBG] pgmap v5874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:24 gibba011 bash[31774]: cluster 2021-03-10T17:08:23.059371+0000 mgr.gibba032.whopeo (mgr.24625) 10314 : cluster [DBG] pgmap v5874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:26 gibba032 bash[24004]: cluster 2021-03-10T17:08:25.060713+0000 mgr.gibba032.whopeo (mgr.24625) 10315 : cluster [DBG] pgmap v5875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:26 gibba011 bash[30791]: cluster 2021-03-10T17:08:25.060713+0000 mgr.gibba032.whopeo (mgr.24625) 10315 : cluster [DBG] pgmap v5875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:26 gibba011 bash[31774]: cluster 2021-03-10T17:08:25.060713+0000 mgr.gibba032.whopeo (mgr.24625) 10315 : cluster [DBG] pgmap v5875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:28 gibba032 bash[24004]: cluster 2021-03-10T17:08:27.062565+0000 mgr.gibba032.whopeo (mgr.24625) 10316 : cluster [DBG] pgmap v5876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:28 gibba032 bash[24004]: cephadm 2021-03-10T17:08:27.203590+0000 mgr.gibba032.whopeo (mgr.24625) 10317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:28 gibba011 bash[30791]: cluster 2021-03-10T17:08:27.062565+0000 mgr.gibba032.whopeo (mgr.24625) 10316 : cluster [DBG] pgmap v5876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:28 gibba011 bash[30791]: cephadm 2021-03-10T17:08:27.203590+0000 mgr.gibba032.whopeo (mgr.24625) 10317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:28 gibba011 bash[31774]: cluster 2021-03-10T17:08:27.062565+0000 mgr.gibba032.whopeo (mgr.24625) 10316 : cluster [DBG] pgmap v5876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:28 gibba011 bash[31774]: cephadm 2021-03-10T17:08:27.203590+0000 mgr.gibba032.whopeo (mgr.24625) 10317 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:30 gibba032 bash[24004]: cluster 2021-03-10T17:08:29.064081+0000 mgr.gibba032.whopeo (mgr.24625) 10318 : cluster [DBG] pgmap v5877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:30 gibba011 bash[30791]: cluster 2021-03-10T17:08:29.064081+0000 mgr.gibba032.whopeo (mgr.24625) 10318 : cluster [DBG] pgmap v5877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:30 gibba011 bash[31774]: cluster 2021-03-10T17:08:29.064081+0000 mgr.gibba032.whopeo (mgr.24625) 10318 : cluster [DBG] pgmap v5877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:30.968 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:08:31.221 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:08:31.221 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:08:31.221 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:08:31.222 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:08:31.222 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:08:31.222 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:08:31.222 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:08:31.223 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:08:31.223 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:08:31.223 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:08:31.223 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:08:31.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:08:31.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:08:31.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:08:31.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:08:31.224 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:08:31.225 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:08:31.225 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:08:31.225 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:08:31.225 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:08:31.226 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:08:31.472 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:08:31.473 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:08:31.473 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:08:31.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:31.474 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:08:31.474 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:08:31.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:31.474 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:08:31.475 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:08:31.475 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:08:31.475 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:08:31.475 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:08:31.476 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:08:31.476 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:08:31.476 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:08:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:31 gibba032 bash[24004]: audit 2021-03-10T17:08:30.958439+0000 mgr.gibba032.whopeo (mgr.24625) 10319 : audit [DBG] from='client.50967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:31 gibba032 bash[24004]: cluster 2021-03-10T17:08:31.065380+0000 mgr.gibba032.whopeo (mgr.24625) 10320 : cluster [DBG] pgmap v5878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:31 gibba032 bash[24004]: audit 2021-03-10T17:08:31.181758+0000 mgr.gibba032.whopeo (mgr.24625) 10321 : audit [DBG] from='client.50973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:31 gibba032 bash[24004]: audit 2021-03-10T17:08:31.472812+0000 mon.a (mon.0) 3462 : audit [DBG] from='client.? 172.21.2.111:0/1947418479' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:32.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[30791]: audit 2021-03-10T17:08:30.958439+0000 mgr.gibba032.whopeo (mgr.24625) 10319 : audit [DBG] from='client.50967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[30791]: cluster 2021-03-10T17:08:31.065380+0000 mgr.gibba032.whopeo (mgr.24625) 10320 : cluster [DBG] pgmap v5878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[30791]: audit 2021-03-10T17:08:31.181758+0000 mgr.gibba032.whopeo (mgr.24625) 10321 : audit [DBG] from='client.50973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[30791]: audit 2021-03-10T17:08:31.472812+0000 mon.a (mon.0) 3462 : audit [DBG] from='client.? 172.21.2.111:0/1947418479' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[31774]: audit 2021-03-10T17:08:30.958439+0000 mgr.gibba032.whopeo (mgr.24625) 10319 : audit [DBG] from='client.50967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[31774]: cluster 2021-03-10T17:08:31.065380+0000 mgr.gibba032.whopeo (mgr.24625) 10320 : cluster [DBG] pgmap v5878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[31774]: audit 2021-03-10T17:08:31.181758+0000 mgr.gibba032.whopeo (mgr.24625) 10321 : audit [DBG] from='client.50973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:08:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:31 gibba011 bash[31774]: audit 2021-03-10T17:08:31.472812+0000 mon.a (mon.0) 3462 : audit [DBG] from='client.? 172.21.2.111:0/1947418479' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:08:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:34 gibba032 bash[24004]: cluster 2021-03-10T17:08:33.066101+0000 mgr.gibba032.whopeo (mgr.24625) 10322 : cluster [DBG] pgmap v5879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:34 gibba011 bash[30791]: cluster 2021-03-10T17:08:33.066101+0000 mgr.gibba032.whopeo (mgr.24625) 10322 : cluster [DBG] pgmap v5879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:34 gibba011 bash[31774]: cluster 2021-03-10T17:08:33.066101+0000 mgr.gibba032.whopeo (mgr.24625) 10322 : cluster [DBG] pgmap v5879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:36 gibba032 bash[24004]: cluster 2021-03-10T17:08:35.067223+0000 mgr.gibba032.whopeo (mgr.24625) 10323 : cluster [DBG] pgmap v5880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:36 gibba011 bash[30791]: cluster 2021-03-10T17:08:35.067223+0000 mgr.gibba032.whopeo (mgr.24625) 10323 : cluster [DBG] pgmap v5880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:36 gibba011 bash[31774]: cluster 2021-03-10T17:08:35.067223+0000 mgr.gibba032.whopeo (mgr.24625) 10323 : cluster [DBG] pgmap v5880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:38 gibba032 bash[24004]: cluster 2021-03-10T17:08:37.068953+0000 mgr.gibba032.whopeo (mgr.24625) 10324 : cluster [DBG] pgmap v5881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:38 gibba011 bash[30791]: cluster 2021-03-10T17:08:37.068953+0000 mgr.gibba032.whopeo (mgr.24625) 10324 : cluster [DBG] pgmap v5881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:38 gibba011 bash[31774]: cluster 2021-03-10T17:08:37.068953+0000 mgr.gibba032.whopeo (mgr.24625) 10324 : cluster [DBG] pgmap v5881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:40 gibba032 bash[24004]: cluster 2021-03-10T17:08:39.070561+0000 mgr.gibba032.whopeo (mgr.24625) 10325 : cluster [DBG] pgmap v5882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:40 gibba011 bash[30791]: cluster 2021-03-10T17:08:39.070561+0000 mgr.gibba032.whopeo (mgr.24625) 10325 : cluster [DBG] pgmap v5882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:40 gibba011 bash[31774]: cluster 2021-03-10T17:08:39.070561+0000 mgr.gibba032.whopeo (mgr.24625) 10325 : cluster [DBG] pgmap v5882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:41 gibba032 bash[24004]: cluster 2021-03-10T17:08:41.071907+0000 mgr.gibba032.whopeo (mgr.24625) 10326 : cluster [DBG] pgmap v5883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:41 gibba011 bash[30791]: cluster 2021-03-10T17:08:41.071907+0000 mgr.gibba032.whopeo (mgr.24625) 10326 : cluster [DBG] pgmap v5883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:41 gibba011 bash[31774]: cluster 2021-03-10T17:08:41.071907+0000 mgr.gibba032.whopeo (mgr.24625) 10326 : cluster [DBG] pgmap v5883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:43.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:42 gibba032 bash[24004]: cephadm 2021-03-10T17:08:42.207929+0000 mgr.gibba032.whopeo (mgr.24625) 10327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:43.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:42 gibba011 bash[30791]: cephadm 2021-03-10T17:08:42.207929+0000 mgr.gibba032.whopeo (mgr.24625) 10327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:42 gibba011 bash[31774]: cephadm 2021-03-10T17:08:42.207929+0000 mgr.gibba032.whopeo (mgr.24625) 10327 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:44.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:43 gibba032 bash[24004]: cluster 2021-03-10T17:08:43.073593+0000 mgr.gibba032.whopeo (mgr.24625) 10328 : cluster [DBG] pgmap v5884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:44.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:43 gibba011 bash[30791]: cluster 2021-03-10T17:08:43.073593+0000 mgr.gibba032.whopeo (mgr.24625) 10328 : cluster [DBG] pgmap v5884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:44.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:43 gibba011 bash[31774]: cluster 2021-03-10T17:08:43.073593+0000 mgr.gibba032.whopeo (mgr.24625) 10328 : cluster [DBG] pgmap v5884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:46 gibba032 bash[24004]: cluster 2021-03-10T17:08:45.074970+0000 mgr.gibba032.whopeo (mgr.24625) 10329 : cluster [DBG] pgmap v5885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:46 gibba011 bash[30791]: cluster 2021-03-10T17:08:45.074970+0000 mgr.gibba032.whopeo (mgr.24625) 10329 : cluster [DBG] pgmap v5885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:46 gibba011 bash[31774]: cluster 2021-03-10T17:08:45.074970+0000 mgr.gibba032.whopeo (mgr.24625) 10329 : cluster [DBG] pgmap v5885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:48 gibba032 bash[24004]: cluster 2021-03-10T17:08:47.076811+0000 mgr.gibba032.whopeo (mgr.24625) 10330 : cluster [DBG] pgmap v5886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:48 gibba011 bash[30791]: cluster 2021-03-10T17:08:47.076811+0000 mgr.gibba032.whopeo (mgr.24625) 10330 : cluster [DBG] pgmap v5886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:48 gibba011 bash[31774]: cluster 2021-03-10T17:08:47.076811+0000 mgr.gibba032.whopeo (mgr.24625) 10330 : cluster [DBG] pgmap v5886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:50 gibba032 bash[24004]: cluster 2021-03-10T17:08:49.078413+0000 mgr.gibba032.whopeo (mgr.24625) 10331 : cluster [DBG] pgmap v5887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:50 gibba011 bash[30791]: cluster 2021-03-10T17:08:49.078413+0000 mgr.gibba032.whopeo (mgr.24625) 10331 : cluster [DBG] pgmap v5887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:50 gibba011 bash[31774]: cluster 2021-03-10T17:08:49.078413+0000 mgr.gibba032.whopeo (mgr.24625) 10331 : cluster [DBG] pgmap v5887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:52.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:51 gibba032 bash[24004]: cluster 2021-03-10T17:08:51.079877+0000 mgr.gibba032.whopeo (mgr.24625) 10332 : cluster [DBG] pgmap v5888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:51 gibba011 bash[30791]: cluster 2021-03-10T17:08:51.079877+0000 mgr.gibba032.whopeo (mgr.24625) 10332 : cluster [DBG] pgmap v5888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:51 gibba011 bash[31774]: cluster 2021-03-10T17:08:51.079877+0000 mgr.gibba032.whopeo (mgr.24625) 10332 : cluster [DBG] pgmap v5888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:54 gibba032 bash[24004]: cluster 2021-03-10T17:08:53.080759+0000 mgr.gibba032.whopeo (mgr.24625) 10333 : cluster [DBG] pgmap v5889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:54 gibba011 bash[30791]: cluster 2021-03-10T17:08:53.080759+0000 mgr.gibba032.whopeo (mgr.24625) 10333 : cluster [DBG] pgmap v5889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:54 gibba011 bash[31774]: cluster 2021-03-10T17:08:53.080759+0000 mgr.gibba032.whopeo (mgr.24625) 10333 : cluster [DBG] pgmap v5889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:56 gibba032 bash[24004]: cluster 2021-03-10T17:08:55.081935+0000 mgr.gibba032.whopeo (mgr.24625) 10334 : cluster [DBG] pgmap v5890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:56 gibba011 bash[30791]: cluster 2021-03-10T17:08:55.081935+0000 mgr.gibba032.whopeo (mgr.24625) 10334 : cluster [DBG] pgmap v5890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:56 gibba011 bash[31774]: cluster 2021-03-10T17:08:55.081935+0000 mgr.gibba032.whopeo (mgr.24625) 10334 : cluster [DBG] pgmap v5890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: cluster 2021-03-10T17:08:57.083700+0000 mgr.gibba032.whopeo (mgr.24625) 10335 : cluster [DBG] pgmap v5891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: cephadm 2021-03-10T17:08:57.223987+0000 mgr.gibba032.whopeo (mgr.24625) 10336 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.342228+0000 mon.a (mon.0) 3463 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:08:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.342908+0000 mon.a (mon.0) 3464 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:08:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.506746+0000 mon.a (mon.0) 3465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:08:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.507546+0000 mon.a (mon.0) 3466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.508247+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:08:58 gibba032 bash[24004]: audit 2021-03-10T17:08:57.508926+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:08:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: cluster 2021-03-10T17:08:57.083700+0000 mgr.gibba032.whopeo (mgr.24625) 10335 : cluster [DBG] pgmap v5891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: cephadm 2021-03-10T17:08:57.223987+0000 mgr.gibba032.whopeo (mgr.24625) 10336 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.342228+0000 mon.a (mon.0) 3463 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:08:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.342908+0000 mon.a (mon.0) 3464 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:08:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.506746+0000 mon.a (mon.0) 3465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:08:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.507546+0000 mon.a (mon.0) 3466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.508247+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[30791]: audit 2021-03-10T17:08:57.508926+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:08:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: cluster 2021-03-10T17:08:57.083700+0000 mgr.gibba032.whopeo (mgr.24625) 10335 : cluster [DBG] pgmap v5891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:08:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: cephadm 2021-03-10T17:08:57.223987+0000 mgr.gibba032.whopeo (mgr.24625) 10336 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:08:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.342228+0000 mon.a (mon.0) 3463 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:08:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.342908+0000 mon.a (mon.0) 3464 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:08:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.506746+0000 mon.a (mon.0) 3465 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:08:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.507546+0000 mon.a (mon.0) 3466 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.508247+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:08:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:08:58 gibba011 bash[31774]: audit 2021-03-10T17:08:57.508926+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:09:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:00 gibba032 bash[24004]: cluster 2021-03-10T17:08:59.085269+0000 mgr.gibba032.whopeo (mgr.24625) 10337 : cluster [DBG] pgmap v5892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:00 gibba011 bash[30791]: cluster 2021-03-10T17:08:59.085269+0000 mgr.gibba032.whopeo (mgr.24625) 10337 : cluster [DBG] pgmap v5892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:00 gibba011 bash[31774]: cluster 2021-03-10T17:08:59.085269+0000 mgr.gibba032.whopeo (mgr.24625) 10337 : cluster [DBG] pgmap v5892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:01.724 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:09:01.985 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:09:01.986 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:09:01.986 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:09:01.986 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:09:01.987 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:09:01.987 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:09:01.988 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:09:01.988 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:09:01.988 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:09:01.989 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:09:01.989 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:09:01.990 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:09:01.990 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:09:01.991 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:09:01.991 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:09:01.991 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:09:01.992 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:09:01.992 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:09:01.993 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:09:01.993 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:09:01.993 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:09:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:01 gibba032 bash[24004]: cluster 2021-03-10T17:09:01.086713+0000 mgr.gibba032.whopeo (mgr.24625) 10338 : cluster [DBG] pgmap v5893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:01 gibba011 bash[30791]: cluster 2021-03-10T17:09:01.086713+0000 mgr.gibba032.whopeo (mgr.24625) 10338 : cluster [DBG] pgmap v5893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:02.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:01 gibba011 bash[31774]: cluster 2021-03-10T17:09:01.086713+0000 mgr.gibba032.whopeo (mgr.24625) 10338 : cluster [DBG] pgmap v5893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:02.238 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:09:02.238 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:09:02.238 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:09:02.239 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:02.239 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:09:02.239 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:09:02.239 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:02.239 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:09:02.240 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:09:02.240 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:02.240 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:09:02.240 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:09:02.240 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:09:02.241 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:09:02.241 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:09:03.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:02 gibba032 bash[24004]: audit 2021-03-10T17:09:01.705684+0000 mgr.gibba032.whopeo (mgr.24625) 10339 : audit [DBG] from='client.50985 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:02 gibba032 bash[24004]: audit 2021-03-10T17:09:01.944616+0000 mgr.gibba032.whopeo (mgr.24625) 10340 : audit [DBG] from='client.60973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:02 gibba032 bash[24004]: audit 2021-03-10T17:09:02.238068+0000 mon.c (mon.1) 198 : audit [DBG] from='client.? 172.21.2.111:0/2438553951' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:03.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[30791]: audit 2021-03-10T17:09:01.705684+0000 mgr.gibba032.whopeo (mgr.24625) 10339 : audit [DBG] from='client.50985 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[30791]: audit 2021-03-10T17:09:01.944616+0000 mgr.gibba032.whopeo (mgr.24625) 10340 : audit [DBG] from='client.60973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[30791]: audit 2021-03-10T17:09:02.238068+0000 mon.c (mon.1) 198 : audit [DBG] from='client.? 172.21.2.111:0/2438553951' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:03.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[31774]: audit 2021-03-10T17:09:01.705684+0000 mgr.gibba032.whopeo (mgr.24625) 10339 : audit [DBG] from='client.50985 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[31774]: audit 2021-03-10T17:09:01.944616+0000 mgr.gibba032.whopeo (mgr.24625) 10340 : audit [DBG] from='client.60973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:03.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:02 gibba011 bash[31774]: audit 2021-03-10T17:09:02.238068+0000 mon.c (mon.1) 198 : audit [DBG] from='client.? 172.21.2.111:0/2438553951' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:04.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:03 gibba032 bash[24004]: cluster 2021-03-10T17:09:03.088501+0000 mgr.gibba032.whopeo (mgr.24625) 10341 : cluster [DBG] pgmap v5894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:04.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:03 gibba011 bash[30791]: cluster 2021-03-10T17:09:03.088501+0000 mgr.gibba032.whopeo (mgr.24625) 10341 : cluster [DBG] pgmap v5894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:04.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:03 gibba011 bash[31774]: cluster 2021-03-10T17:09:03.088501+0000 mgr.gibba032.whopeo (mgr.24625) 10341 : cluster [DBG] pgmap v5894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:06 gibba032 bash[24004]: cluster 2021-03-10T17:09:05.089934+0000 mgr.gibba032.whopeo (mgr.24625) 10342 : cluster [DBG] pgmap v5895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:06 gibba011 bash[30791]: cluster 2021-03-10T17:09:05.089934+0000 mgr.gibba032.whopeo (mgr.24625) 10342 : cluster [DBG] pgmap v5895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:06 gibba011 bash[31774]: cluster 2021-03-10T17:09:05.089934+0000 mgr.gibba032.whopeo (mgr.24625) 10342 : cluster [DBG] pgmap v5895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:08 gibba032 bash[24004]: cluster 2021-03-10T17:09:07.091695+0000 mgr.gibba032.whopeo (mgr.24625) 10343 : cluster [DBG] pgmap v5896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:08 gibba011 bash[30791]: cluster 2021-03-10T17:09:07.091695+0000 mgr.gibba032.whopeo (mgr.24625) 10343 : cluster [DBG] pgmap v5896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:08 gibba011 bash[31774]: cluster 2021-03-10T17:09:07.091695+0000 mgr.gibba032.whopeo (mgr.24625) 10343 : cluster [DBG] pgmap v5896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:10 gibba032 bash[24004]: cluster 2021-03-10T17:09:09.093348+0000 mgr.gibba032.whopeo (mgr.24625) 10344 : cluster [DBG] pgmap v5897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:10 gibba011 bash[30791]: cluster 2021-03-10T17:09:09.093348+0000 mgr.gibba032.whopeo (mgr.24625) 10344 : cluster [DBG] pgmap v5897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:10 gibba011 bash[31774]: cluster 2021-03-10T17:09:09.093348+0000 mgr.gibba032.whopeo (mgr.24625) 10344 : cluster [DBG] pgmap v5897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:12.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:11 gibba032 bash[24004]: cluster 2021-03-10T17:09:11.094884+0000 mgr.gibba032.whopeo (mgr.24625) 10345 : cluster [DBG] pgmap v5898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:11 gibba011 bash[30791]: cluster 2021-03-10T17:09:11.094884+0000 mgr.gibba032.whopeo (mgr.24625) 10345 : cluster [DBG] pgmap v5898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:12.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:11 gibba011 bash[31774]: cluster 2021-03-10T17:09:11.094884+0000 mgr.gibba032.whopeo (mgr.24625) 10345 : cluster [DBG] pgmap v5898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.240883+0000 mgr.gibba032.whopeo (mgr.24625) 10346 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:09:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.242442+0000 mgr.gibba032.whopeo (mgr.24625) 10347 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:09:13.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.454619+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:09:13.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.455889+0000 mgr.gibba032.whopeo (mgr.24625) 10348 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:09:13.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.457824+0000 mgr.gibba032.whopeo (mgr.24625) 10349 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:09:13.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.458474+0000 mon.a (mon.0) 3470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:09:13.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.459389+0000 mgr.gibba032.whopeo (mgr.24625) 10350 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:09:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.459443+0000 mgr.gibba032.whopeo (mgr.24625) 10351 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:09:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.460286+0000 mon.a (mon.0) 3471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.460909+0000 mgr.gibba032.whopeo (mgr.24625) 10352 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:09:13.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.460966+0000 mgr.gibba032.whopeo (mgr.24625) 10353 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:09:13.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.461702+0000 mon.a (mon.0) 3472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.462213+0000 mgr.gibba032.whopeo (mgr.24625) 10354 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:09:13.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.463168+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.463621+0000 mgr.gibba032.whopeo (mgr.24625) 10355 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:09:13.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.463679+0000 mgr.gibba032.whopeo (mgr.24625) 10356 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:09:13.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.465311+0000 mgr.gibba032.whopeo (mgr.24625) 10357 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:09:13.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.465741+0000 mgr.gibba032.whopeo (mgr.24625) 10358 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:09:13.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.466139+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.467152+0000 mgr.gibba032.whopeo (mgr.24625) 10359 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:09:13.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.467212+0000 mgr.gibba032.whopeo (mgr.24625) 10360 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:09:13.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: cephadm 2021-03-10T17:09:12.467383+0000 mgr.gibba032.whopeo (mgr.24625) 10361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:13.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:13 gibba032 bash[24004]: audit 2021-03-10T17:09:12.467971+0000 mon.a (mon.0) 3475 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.240883+0000 mgr.gibba032.whopeo (mgr.24625) 10346 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:09:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.242442+0000 mgr.gibba032.whopeo (mgr.24625) 10347 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:09:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.454619+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:09:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.455889+0000 mgr.gibba032.whopeo (mgr.24625) 10348 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:09:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.457824+0000 mgr.gibba032.whopeo (mgr.24625) 10349 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:09:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.458474+0000 mon.a (mon.0) 3470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:09:13.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.459389+0000 mgr.gibba032.whopeo (mgr.24625) 10350 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:09:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.459443+0000 mgr.gibba032.whopeo (mgr.24625) 10351 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:09:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.460286+0000 mon.a (mon.0) 3471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.460909+0000 mgr.gibba032.whopeo (mgr.24625) 10352 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:09:13.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.460966+0000 mgr.gibba032.whopeo (mgr.24625) 10353 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:09:13.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.461702+0000 mon.a (mon.0) 3472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.462213+0000 mgr.gibba032.whopeo (mgr.24625) 10354 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:09:13.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.463168+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.463621+0000 mgr.gibba032.whopeo (mgr.24625) 10355 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:09:13.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.463679+0000 mgr.gibba032.whopeo (mgr.24625) 10356 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:09:13.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.465311+0000 mgr.gibba032.whopeo (mgr.24625) 10357 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:09:13.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.465741+0000 mgr.gibba032.whopeo (mgr.24625) 10358 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:09:13.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.466139+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.467152+0000 mgr.gibba032.whopeo (mgr.24625) 10359 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:09:13.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.467212+0000 mgr.gibba032.whopeo (mgr.24625) 10360 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:09:13.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: cephadm 2021-03-10T17:09:12.467383+0000 mgr.gibba032.whopeo (mgr.24625) 10361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:13.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[30791]: audit 2021-03-10T17:09:12.467971+0000 mon.a (mon.0) 3475 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.240883+0000 mgr.gibba032.whopeo (mgr.24625) 10346 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:09:13.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.242442+0000 mgr.gibba032.whopeo (mgr.24625) 10347 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:09:13.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.454619+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:09:13.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.455889+0000 mgr.gibba032.whopeo (mgr.24625) 10348 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:09:13.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.457824+0000 mgr.gibba032.whopeo (mgr.24625) 10349 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:09:13.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.458474+0000 mon.a (mon.0) 3470 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:09:13.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.459389+0000 mgr.gibba032.whopeo (mgr.24625) 10350 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:09:13.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.459443+0000 mgr.gibba032.whopeo (mgr.24625) 10351 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:09:13.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.460286+0000 mon.a (mon.0) 3471 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.460909+0000 mgr.gibba032.whopeo (mgr.24625) 10352 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:09:13.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.460966+0000 mgr.gibba032.whopeo (mgr.24625) 10353 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:09:13.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.461702+0000 mon.a (mon.0) 3472 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.462213+0000 mgr.gibba032.whopeo (mgr.24625) 10354 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:09:13.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.463168+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.463621+0000 mgr.gibba032.whopeo (mgr.24625) 10355 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:09:13.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.463679+0000 mgr.gibba032.whopeo (mgr.24625) 10356 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:09:13.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.465311+0000 mgr.gibba032.whopeo (mgr.24625) 10357 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:09:13.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.465741+0000 mgr.gibba032.whopeo (mgr.24625) 10358 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:09:13.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.466139+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:13.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.467152+0000 mgr.gibba032.whopeo (mgr.24625) 10359 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:09:13.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.467212+0000 mgr.gibba032.whopeo (mgr.24625) 10360 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:09:13.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: cephadm 2021-03-10T17:09:12.467383+0000 mgr.gibba032.whopeo (mgr.24625) 10361 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:13.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:13 gibba011 bash[31774]: audit 2021-03-10T17:09:12.467971+0000 mon.a (mon.0) 3475 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:14 gibba032 bash[24004]: cluster 2021-03-10T17:09:13.096307+0000 mgr.gibba032.whopeo (mgr.24625) 10362 : cluster [DBG] pgmap v5899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:14 gibba011 bash[30791]: cluster 2021-03-10T17:09:13.096307+0000 mgr.gibba032.whopeo (mgr.24625) 10362 : cluster [DBG] pgmap v5899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:14 gibba011 bash[31774]: cluster 2021-03-10T17:09:13.096307+0000 mgr.gibba032.whopeo (mgr.24625) 10362 : cluster [DBG] pgmap v5899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:16 gibba032 bash[24004]: cluster 2021-03-10T17:09:15.097633+0000 mgr.gibba032.whopeo (mgr.24625) 10363 : cluster [DBG] pgmap v5900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:16 gibba011 bash[30791]: cluster 2021-03-10T17:09:15.097633+0000 mgr.gibba032.whopeo (mgr.24625) 10363 : cluster [DBG] pgmap v5900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:16 gibba011 bash[31774]: cluster 2021-03-10T17:09:15.097633+0000 mgr.gibba032.whopeo (mgr.24625) 10363 : cluster [DBG] pgmap v5900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:18 gibba032 bash[24004]: cluster 2021-03-10T17:09:17.099314+0000 mgr.gibba032.whopeo (mgr.24625) 10364 : cluster [DBG] pgmap v5901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:18 gibba011 bash[30791]: cluster 2021-03-10T17:09:17.099314+0000 mgr.gibba032.whopeo (mgr.24625) 10364 : cluster [DBG] pgmap v5901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:18 gibba011 bash[31774]: cluster 2021-03-10T17:09:17.099314+0000 mgr.gibba032.whopeo (mgr.24625) 10364 : cluster [DBG] pgmap v5901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:20 gibba032 bash[24004]: cluster 2021-03-10T17:09:19.100999+0000 mgr.gibba032.whopeo (mgr.24625) 10365 : cluster [DBG] pgmap v5902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:20 gibba011 bash[30791]: cluster 2021-03-10T17:09:19.100999+0000 mgr.gibba032.whopeo (mgr.24625) 10365 : cluster [DBG] pgmap v5902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:20 gibba011 bash[31774]: cluster 2021-03-10T17:09:19.100999+0000 mgr.gibba032.whopeo (mgr.24625) 10365 : cluster [DBG] pgmap v5902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:22.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:21 gibba032 bash[24004]: cluster 2021-03-10T17:09:21.102532+0000 mgr.gibba032.whopeo (mgr.24625) 10366 : cluster [DBG] pgmap v5903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:21 gibba011 bash[30791]: cluster 2021-03-10T17:09:21.102532+0000 mgr.gibba032.whopeo (mgr.24625) 10366 : cluster [DBG] pgmap v5903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:21 gibba011 bash[31774]: cluster 2021-03-10T17:09:21.102532+0000 mgr.gibba032.whopeo (mgr.24625) 10366 : cluster [DBG] pgmap v5903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:24 gibba032 bash[24004]: cluster 2021-03-10T17:09:23.104070+0000 mgr.gibba032.whopeo (mgr.24625) 10367 : cluster [DBG] pgmap v5904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:24 gibba011 bash[30791]: cluster 2021-03-10T17:09:23.104070+0000 mgr.gibba032.whopeo (mgr.24625) 10367 : cluster [DBG] pgmap v5904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:24 gibba011 bash[31774]: cluster 2021-03-10T17:09:23.104070+0000 mgr.gibba032.whopeo (mgr.24625) 10367 : cluster [DBG] pgmap v5904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:26 gibba032 bash[24004]: cluster 2021-03-10T17:09:25.105468+0000 mgr.gibba032.whopeo (mgr.24625) 10368 : cluster [DBG] pgmap v5905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:26 gibba011 bash[30791]: cluster 2021-03-10T17:09:25.105468+0000 mgr.gibba032.whopeo (mgr.24625) 10368 : cluster [DBG] pgmap v5905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:26 gibba011 bash[31774]: cluster 2021-03-10T17:09:25.105468+0000 mgr.gibba032.whopeo (mgr.24625) 10368 : cluster [DBG] pgmap v5905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:28 gibba032 bash[24004]: cluster 2021-03-10T17:09:27.107154+0000 mgr.gibba032.whopeo (mgr.24625) 10369 : cluster [DBG] pgmap v5906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:28 gibba032 bash[24004]: cephadm 2021-03-10T17:09:27.472486+0000 mgr.gibba032.whopeo (mgr.24625) 10370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:28 gibba011 bash[30791]: cluster 2021-03-10T17:09:27.107154+0000 mgr.gibba032.whopeo (mgr.24625) 10369 : cluster [DBG] pgmap v5906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:28 gibba011 bash[30791]: cephadm 2021-03-10T17:09:27.472486+0000 mgr.gibba032.whopeo (mgr.24625) 10370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:28 gibba011 bash[31774]: cluster 2021-03-10T17:09:27.107154+0000 mgr.gibba032.whopeo (mgr.24625) 10369 : cluster [DBG] pgmap v5906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:28 gibba011 bash[31774]: cephadm 2021-03-10T17:09:27.472486+0000 mgr.gibba032.whopeo (mgr.24625) 10370 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:30 gibba032 bash[24004]: cluster 2021-03-10T17:09:29.108600+0000 mgr.gibba032.whopeo (mgr.24625) 10371 : cluster [DBG] pgmap v5907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:30 gibba011 bash[30791]: cluster 2021-03-10T17:09:29.108600+0000 mgr.gibba032.whopeo (mgr.24625) 10371 : cluster [DBG] pgmap v5907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:30 gibba011 bash[31774]: cluster 2021-03-10T17:09:29.108600+0000 mgr.gibba032.whopeo (mgr.24625) 10371 : cluster [DBG] pgmap v5907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:31 gibba032 bash[24004]: cluster 2021-03-10T17:09:31.110020+0000 mgr.gibba032.whopeo (mgr.24625) 10372 : cluster [DBG] pgmap v5908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:31 gibba011 bash[30791]: cluster 2021-03-10T17:09:31.110020+0000 mgr.gibba032.whopeo (mgr.24625) 10372 : cluster [DBG] pgmap v5908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:31 gibba011 bash[31774]: cluster 2021-03-10T17:09:31.110020+0000 mgr.gibba032.whopeo (mgr.24625) 10372 : cluster [DBG] pgmap v5908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:32.482 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:09:32.711 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:09:32.712 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:09:32.712 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:09:32.712 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:09:32.712 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:09:32.712 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:09:32.713 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:09:32.713 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:09:32.713 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:09:32.713 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:09:32.714 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:09:32.714 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:09:32.714 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:09:32.714 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:09:32.714 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:09:32.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:09:32.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:09:32.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:09:32.715 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:09:32.715 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:09:32.716 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:09:32.965 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:09:32.966 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:09:32.966 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:09:32.966 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:32.966 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:09:32.967 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:09:32.967 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:32.967 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:09:32.967 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:09:32.967 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:09:32.968 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:09:32.968 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:09:32.968 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:09:32.968 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:09:32.969 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:09:33.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:32 gibba032 bash[24004]: audit 2021-03-10T17:09:32.471283+0000 mgr.gibba032.whopeo (mgr.24625) 10373 : audit [DBG] from='client.51003 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:33.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:32 gibba011 bash[30791]: audit 2021-03-10T17:09:32.471283+0000 mgr.gibba032.whopeo (mgr.24625) 10373 : audit [DBG] from='client.51003 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:33.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:32 gibba011 bash[31774]: audit 2021-03-10T17:09:32.471283+0000 mgr.gibba032.whopeo (mgr.24625) 10373 : audit [DBG] from='client.51003 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:34.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:33 gibba032 bash[24004]: audit 2021-03-10T17:09:32.697415+0000 mgr.gibba032.whopeo (mgr.24625) 10374 : audit [DBG] from='client.60991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:34.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:33 gibba032 bash[24004]: audit 2021-03-10T17:09:32.965870+0000 mon.c (mon.1) 199 : audit [DBG] from='client.? 172.21.2.111:0/1876338453' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:34.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:33 gibba032 bash[24004]: cluster 2021-03-10T17:09:33.111696+0000 mgr.gibba032.whopeo (mgr.24625) 10375 : cluster [DBG] pgmap v5909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:34.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[30791]: audit 2021-03-10T17:09:32.697415+0000 mgr.gibba032.whopeo (mgr.24625) 10374 : audit [DBG] from='client.60991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:34.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[30791]: audit 2021-03-10T17:09:32.965870+0000 mon.c (mon.1) 199 : audit [DBG] from='client.? 172.21.2.111:0/1876338453' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:34.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[30791]: cluster 2021-03-10T17:09:33.111696+0000 mgr.gibba032.whopeo (mgr.24625) 10375 : cluster [DBG] pgmap v5909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:34.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[31774]: audit 2021-03-10T17:09:32.697415+0000 mgr.gibba032.whopeo (mgr.24625) 10374 : audit [DBG] from='client.60991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:09:34.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[31774]: audit 2021-03-10T17:09:32.965870+0000 mon.c (mon.1) 199 : audit [DBG] from='client.? 172.21.2.111:0/1876338453' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:09:34.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:33 gibba011 bash[31774]: cluster 2021-03-10T17:09:33.111696+0000 mgr.gibba032.whopeo (mgr.24625) 10375 : cluster [DBG] pgmap v5909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:36 gibba032 bash[24004]: cluster 2021-03-10T17:09:35.112978+0000 mgr.gibba032.whopeo (mgr.24625) 10376 : cluster [DBG] pgmap v5910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:36 gibba011 bash[30791]: cluster 2021-03-10T17:09:35.112978+0000 mgr.gibba032.whopeo (mgr.24625) 10376 : cluster [DBG] pgmap v5910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:36 gibba011 bash[31774]: cluster 2021-03-10T17:09:35.112978+0000 mgr.gibba032.whopeo (mgr.24625) 10376 : cluster [DBG] pgmap v5910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:38.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:38 gibba032 bash[24004]: cluster 2021-03-10T17:09:37.114596+0000 mgr.gibba032.whopeo (mgr.24625) 10377 : cluster [DBG] pgmap v5911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:38.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:38 gibba011 bash[30791]: cluster 2021-03-10T17:09:37.114596+0000 mgr.gibba032.whopeo (mgr.24625) 10377 : cluster [DBG] pgmap v5911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:38 gibba011 bash[31774]: cluster 2021-03-10T17:09:37.114596+0000 mgr.gibba032.whopeo (mgr.24625) 10377 : cluster [DBG] pgmap v5911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:40 gibba032 bash[24004]: cluster 2021-03-10T17:09:39.116204+0000 mgr.gibba032.whopeo (mgr.24625) 10378 : cluster [DBG] pgmap v5912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:40 gibba011 bash[30791]: cluster 2021-03-10T17:09:39.116204+0000 mgr.gibba032.whopeo (mgr.24625) 10378 : cluster [DBG] pgmap v5912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:40 gibba011 bash[31774]: cluster 2021-03-10T17:09:39.116204+0000 mgr.gibba032.whopeo (mgr.24625) 10378 : cluster [DBG] pgmap v5912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:42.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:41 gibba032 bash[24004]: cluster 2021-03-10T17:09:41.117695+0000 mgr.gibba032.whopeo (mgr.24625) 10379 : cluster [DBG] pgmap v5913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:41 gibba011 bash[30791]: cluster 2021-03-10T17:09:41.117695+0000 mgr.gibba032.whopeo (mgr.24625) 10379 : cluster [DBG] pgmap v5913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:41 gibba011 bash[31774]: cluster 2021-03-10T17:09:41.117695+0000 mgr.gibba032.whopeo (mgr.24625) 10379 : cluster [DBG] pgmap v5913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:43.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:42 gibba011 bash[30791]: cephadm 2021-03-10T17:09:42.488400+0000 mgr.gibba032.whopeo (mgr.24625) 10380 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:43.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:42 gibba011 bash[31774]: cephadm 2021-03-10T17:09:42.488400+0000 mgr.gibba032.whopeo (mgr.24625) 10380 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:42 gibba032 bash[24004]: cephadm 2021-03-10T17:09:42.488400+0000 mgr.gibba032.whopeo (mgr.24625) 10380 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:44.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:43 gibba011 bash[30791]: cluster 2021-03-10T17:09:43.118884+0000 mgr.gibba032.whopeo (mgr.24625) 10381 : cluster [DBG] pgmap v5914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:44.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:43 gibba011 bash[31774]: cluster 2021-03-10T17:09:43.118884+0000 mgr.gibba032.whopeo (mgr.24625) 10381 : cluster [DBG] pgmap v5914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:43 gibba032 bash[24004]: cluster 2021-03-10T17:09:43.118884+0000 mgr.gibba032.whopeo (mgr.24625) 10381 : cluster [DBG] pgmap v5914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:46 gibba032 bash[24004]: cluster 2021-03-10T17:09:45.119240+0000 mgr.gibba032.whopeo (mgr.24625) 10382 : cluster [DBG] pgmap v5915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:46.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:46 gibba011 bash[30791]: cluster 2021-03-10T17:09:45.119240+0000 mgr.gibba032.whopeo (mgr.24625) 10382 : cluster [DBG] pgmap v5915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:46 gibba011 bash[31774]: cluster 2021-03-10T17:09:45.119240+0000 mgr.gibba032.whopeo (mgr.24625) 10382 : cluster [DBG] pgmap v5915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:48.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:48 gibba032 bash[24004]: cluster 2021-03-10T17:09:47.120829+0000 mgr.gibba032.whopeo (mgr.24625) 10383 : cluster [DBG] pgmap v5916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:48 gibba011 bash[30791]: cluster 2021-03-10T17:09:47.120829+0000 mgr.gibba032.whopeo (mgr.24625) 10383 : cluster [DBG] pgmap v5916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:48 gibba011 bash[31774]: cluster 2021-03-10T17:09:47.120829+0000 mgr.gibba032.whopeo (mgr.24625) 10383 : cluster [DBG] pgmap v5916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:50 gibba032 bash[24004]: cluster 2021-03-10T17:09:49.122373+0000 mgr.gibba032.whopeo (mgr.24625) 10384 : cluster [DBG] pgmap v5917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:50 gibba011 bash[30791]: cluster 2021-03-10T17:09:49.122373+0000 mgr.gibba032.whopeo (mgr.24625) 10384 : cluster [DBG] pgmap v5917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:50 gibba011 bash[31774]: cluster 2021-03-10T17:09:49.122373+0000 mgr.gibba032.whopeo (mgr.24625) 10384 : cluster [DBG] pgmap v5917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:52.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:51 gibba032 bash[24004]: cluster 2021-03-10T17:09:51.123887+0000 mgr.gibba032.whopeo (mgr.24625) 10385 : cluster [DBG] pgmap v5918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:51 gibba011 bash[30791]: cluster 2021-03-10T17:09:51.123887+0000 mgr.gibba032.whopeo (mgr.24625) 10385 : cluster [DBG] pgmap v5918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:52.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:51 gibba011 bash[31774]: cluster 2021-03-10T17:09:51.123887+0000 mgr.gibba032.whopeo (mgr.24625) 10385 : cluster [DBG] pgmap v5918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:54 gibba032 bash[24004]: cluster 2021-03-10T17:09:53.124445+0000 mgr.gibba032.whopeo (mgr.24625) 10386 : cluster [DBG] pgmap v5919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:54.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:54 gibba011 bash[30791]: cluster 2021-03-10T17:09:53.124445+0000 mgr.gibba032.whopeo (mgr.24625) 10386 : cluster [DBG] pgmap v5919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:54 gibba011 bash[31774]: cluster 2021-03-10T17:09:53.124445+0000 mgr.gibba032.whopeo (mgr.24625) 10386 : cluster [DBG] pgmap v5919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:56 gibba032 bash[24004]: cluster 2021-03-10T17:09:55.125773+0000 mgr.gibba032.whopeo (mgr.24625) 10387 : cluster [DBG] pgmap v5920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:56 gibba011 bash[30791]: cluster 2021-03-10T17:09:55.125773+0000 mgr.gibba032.whopeo (mgr.24625) 10387 : cluster [DBG] pgmap v5920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:56 gibba011 bash[31774]: cluster 2021-03-10T17:09:55.125773+0000 mgr.gibba032.whopeo (mgr.24625) 10387 : cluster [DBG] pgmap v5920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: cluster 2021-03-10T17:09:57.127489+0000 mgr.gibba032.whopeo (mgr.24625) 10388 : cluster [DBG] pgmap v5921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.349654+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:09:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.350392+0000 mon.a (mon.0) 3477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:09:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: cephadm 2021-03-10T17:09:57.499612+0000 mgr.gibba032.whopeo (mgr.24625) 10389 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.517740+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:09:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.518544+0000 mon.a (mon.0) 3479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:09:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.519330+0000 mon.a (mon.0) 3480 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:09:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:09:58 gibba032 bash[24004]: audit 2021-03-10T17:09:57.520189+0000 mon.a (mon.0) 3481 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:09:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: cluster 2021-03-10T17:09:57.127489+0000 mgr.gibba032.whopeo (mgr.24625) 10388 : cluster [DBG] pgmap v5921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.349654+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:09:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.350392+0000 mon.a (mon.0) 3477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:09:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: cephadm 2021-03-10T17:09:57.499612+0000 mgr.gibba032.whopeo (mgr.24625) 10389 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.517740+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:09:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.518544+0000 mon.a (mon.0) 3479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:09:58.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.519330+0000 mon.a (mon.0) 3480 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:09:58.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[30791]: audit 2021-03-10T17:09:57.520189+0000 mon.a (mon.0) 3481 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:09:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: cluster 2021-03-10T17:09:57.127489+0000 mgr.gibba032.whopeo (mgr.24625) 10388 : cluster [DBG] pgmap v5921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:09:58.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.349654+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:09:58.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.350392+0000 mon.a (mon.0) 3477 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:09:58.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: cephadm 2021-03-10T17:09:57.499612+0000 mgr.gibba032.whopeo (mgr.24625) 10389 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:09:58.592 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.517740+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:09:58.593 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.518544+0000 mon.a (mon.0) 3479 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:09:58.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.519330+0000 mon.a (mon.0) 3480 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:09:58.594 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:09:58 gibba011 bash[31774]: audit 2021-03-10T17:09:57.520189+0000 mon.a (mon.0) 3481 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:10:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:00 gibba011 bash[30791]: debug 2021-03-10T17:09:59.996+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:10:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:00 gibba011 bash[30791]: cluster 2021-03-10T17:09:59.129134+0000 mgr.gibba032.whopeo (mgr.24625) 10390 : cluster [DBG] pgmap v5922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:00 gibba011 bash[30791]: cluster 2021-03-10T17:10:00.000215+0000 mon.a (mon.0) 3482 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:10:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:00 gibba011 bash[31774]: cluster 2021-03-10T17:09:59.129134+0000 mgr.gibba032.whopeo (mgr.24625) 10390 : cluster [DBG] pgmap v5922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:00 gibba011 bash[31774]: cluster 2021-03-10T17:10:00.000215+0000 mon.a (mon.0) 3482 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:10:00.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:00 gibba032 bash[24004]: cluster 2021-03-10T17:09:59.129134+0000 mgr.gibba032.whopeo (mgr.24625) 10390 : cluster [DBG] pgmap v5922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:00 gibba032 bash[24004]: cluster 2021-03-10T17:10:00.000215+0000 mon.a (mon.0) 3482 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:10:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:01 gibba011 bash[30791]: cluster 2021-03-10T17:10:01.129677+0000 mgr.gibba032.whopeo (mgr.24625) 10391 : cluster [DBG] pgmap v5923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:02.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:01 gibba011 bash[31774]: cluster 2021-03-10T17:10:01.129677+0000 mgr.gibba032.whopeo (mgr.24625) 10391 : cluster [DBG] pgmap v5923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:01 gibba032 bash[24004]: cluster 2021-03-10T17:10:01.129677+0000 mgr.gibba032.whopeo (mgr.24625) 10391 : cluster [DBG] pgmap v5923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:03.246 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:10:03.475 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:10:03.475 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:10:03.475 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:10:03.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:10:03.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:10:03.476 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:10:03.476 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:10:03.476 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:10:03.477 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:10:03.477 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:10:03.477 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:10:03.477 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:10:03.477 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:10:03.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:10:03.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:10:03.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:10:03.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:10:03.478 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:10:03.479 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:10:03.479 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:10:03.479 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:10:03.723 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:10:03.724 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:10:03.724 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:10:03.724 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:03.724 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:10:03.724 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:10:03.725 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:03.725 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:10:03.725 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:10:03.725 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:03.726 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:10:03.726 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:10:03.726 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:10:03.726 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:10:03.726 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:10:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:04 gibba032 bash[24004]: cluster 2021-03-10T17:10:03.131318+0000 mgr.gibba032.whopeo (mgr.24625) 10392 : cluster [DBG] pgmap v5924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:04 gibba032 bash[24004]: audit 2021-03-10T17:10:03.236109+0000 mgr.gibba032.whopeo (mgr.24625) 10393 : audit [DBG] from='client.51021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:04 gibba032 bash[24004]: audit 2021-03-10T17:10:03.461510+0000 mgr.gibba032.whopeo (mgr.24625) 10394 : audit [DBG] from='client.51027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:04 gibba032 bash[24004]: audit 2021-03-10T17:10:03.723570+0000 mon.c (mon.1) 200 : audit [DBG] from='client.? 172.21.2.111:0/2479252862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[30791]: cluster 2021-03-10T17:10:03.131318+0000 mgr.gibba032.whopeo (mgr.24625) 10392 : cluster [DBG] pgmap v5924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[30791]: audit 2021-03-10T17:10:03.236109+0000 mgr.gibba032.whopeo (mgr.24625) 10393 : audit [DBG] from='client.51021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[30791]: audit 2021-03-10T17:10:03.461510+0000 mgr.gibba032.whopeo (mgr.24625) 10394 : audit [DBG] from='client.51027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[30791]: audit 2021-03-10T17:10:03.723570+0000 mon.c (mon.1) 200 : audit [DBG] from='client.? 172.21.2.111:0/2479252862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[31774]: cluster 2021-03-10T17:10:03.131318+0000 mgr.gibba032.whopeo (mgr.24625) 10392 : cluster [DBG] pgmap v5924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:04.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[31774]: audit 2021-03-10T17:10:03.236109+0000 mgr.gibba032.whopeo (mgr.24625) 10393 : audit [DBG] from='client.51021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[31774]: audit 2021-03-10T17:10:03.461510+0000 mgr.gibba032.whopeo (mgr.24625) 10394 : audit [DBG] from='client.51027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:04.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:04 gibba011 bash[31774]: audit 2021-03-10T17:10:03.723570+0000 mon.c (mon.1) 200 : audit [DBG] from='client.? 172.21.2.111:0/2479252862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:06 gibba032 bash[24004]: cluster 2021-03-10T17:10:05.132767+0000 mgr.gibba032.whopeo (mgr.24625) 10395 : cluster [DBG] pgmap v5925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:06 gibba011 bash[30791]: cluster 2021-03-10T17:10:05.132767+0000 mgr.gibba032.whopeo (mgr.24625) 10395 : cluster [DBG] pgmap v5925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:06 gibba011 bash[31774]: cluster 2021-03-10T17:10:05.132767+0000 mgr.gibba032.whopeo (mgr.24625) 10395 : cluster [DBG] pgmap v5925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:08.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:08 gibba032 bash[24004]: cluster 2021-03-10T17:10:07.134344+0000 mgr.gibba032.whopeo (mgr.24625) 10396 : cluster [DBG] pgmap v5926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:08.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:08 gibba011 bash[30791]: cluster 2021-03-10T17:10:07.134344+0000 mgr.gibba032.whopeo (mgr.24625) 10396 : cluster [DBG] pgmap v5926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:08 gibba011 bash[31774]: cluster 2021-03-10T17:10:07.134344+0000 mgr.gibba032.whopeo (mgr.24625) 10396 : cluster [DBG] pgmap v5926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:10.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:10 gibba032 bash[24004]: cluster 2021-03-10T17:10:09.134912+0000 mgr.gibba032.whopeo (mgr.24625) 10397 : cluster [DBG] pgmap v5927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:10.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:10 gibba011 bash[30791]: cluster 2021-03-10T17:10:09.134912+0000 mgr.gibba032.whopeo (mgr.24625) 10397 : cluster [DBG] pgmap v5927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:10 gibba011 bash[31774]: cluster 2021-03-10T17:10:09.134912+0000 mgr.gibba032.whopeo (mgr.24625) 10397 : cluster [DBG] pgmap v5927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:11 gibba011 bash[30791]: cluster 2021-03-10T17:10:11.136564+0000 mgr.gibba032.whopeo (mgr.24625) 10398 : cluster [DBG] pgmap v5928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:12.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:11 gibba011 bash[31774]: cluster 2021-03-10T17:10:11.136564+0000 mgr.gibba032.whopeo (mgr.24625) 10398 : cluster [DBG] pgmap v5928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:11 gibba032 bash[24004]: cluster 2021-03-10T17:10:11.136564+0000 mgr.gibba032.whopeo (mgr.24625) 10398 : cluster [DBG] pgmap v5928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:13.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.516385+0000 mgr.gibba032.whopeo (mgr.24625) 10399 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:10:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.517968+0000 mgr.gibba032.whopeo (mgr.24625) 10400 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:10:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.742943+0000 mon.a (mon.0) 3483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:10:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.746317+0000 mon.a (mon.0) 3484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:10:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.747803+0000 mon.a (mon.0) 3485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.748967+0000 mon.a (mon.0) 3486 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.750154+0000 mon.a (mon.0) 3487 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.752484+0000 mon.a (mon.0) 3488 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:12 gibba032 bash[24004]: audit 2021-03-10T17:10:12.753867+0000 mon.a (mon.0) 3489 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.516385+0000 mgr.gibba032.whopeo (mgr.24625) 10399 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:10:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.517968+0000 mgr.gibba032.whopeo (mgr.24625) 10400 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:10:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.742943+0000 mon.a (mon.0) 3483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:10:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.746317+0000 mon.a (mon.0) 3484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:10:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.747803+0000 mon.a (mon.0) 3485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.748967+0000 mon.a (mon.0) 3486 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.750154+0000 mon.a (mon.0) 3487 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.752484+0000 mon.a (mon.0) 3488 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[30791]: audit 2021-03-10T17:10:12.753867+0000 mon.a (mon.0) 3489 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.516385+0000 mgr.gibba032.whopeo (mgr.24625) 10399 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:10:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.517968+0000 mgr.gibba032.whopeo (mgr.24625) 10400 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:10:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.742943+0000 mon.a (mon.0) 3483 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:10:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.746317+0000 mon.a (mon.0) 3484 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:10:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.747803+0000 mon.a (mon.0) 3485 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.748967+0000 mon.a (mon.0) 3486 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.750154+0000 mon.a (mon.0) 3487 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.752484+0000 mon.a (mon.0) 3488 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:13.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:12 gibba011 bash[31774]: audit 2021-03-10T17:10:12.753867+0000 mon.a (mon.0) 3489 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:14.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.743709+0000 mgr.gibba032.whopeo (mgr.24625) 10401 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:10:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.745526+0000 mgr.gibba032.whopeo (mgr.24625) 10402 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:10:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.746797+0000 mgr.gibba032.whopeo (mgr.24625) 10403 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:10:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.746836+0000 mgr.gibba032.whopeo (mgr.24625) 10404 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:10:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.747955+0000 mgr.gibba032.whopeo (mgr.24625) 10405 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:10:14.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.748006+0000 mgr.gibba032.whopeo (mgr.24625) 10406 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:10:14.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.749220+0000 mgr.gibba032.whopeo (mgr.24625) 10407 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:10:14.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.750245+0000 mgr.gibba032.whopeo (mgr.24625) 10408 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:10:14.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.750282+0000 mgr.gibba032.whopeo (mgr.24625) 10409 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:10:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.751488+0000 mgr.gibba032.whopeo (mgr.24625) 10410 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:10:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.751822+0000 mgr.gibba032.whopeo (mgr.24625) 10411 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:10:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.753292+0000 mgr.gibba032.whopeo (mgr.24625) 10412 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:10:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.753328+0000 mgr.gibba032.whopeo (mgr.24625) 10413 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:10:14.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cephadm 2021-03-10T17:10:12.753430+0000 mgr.gibba032.whopeo (mgr.24625) 10414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:14.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[30791]: cluster 2021-03-10T17:10:13.138046+0000 mgr.gibba032.whopeo (mgr.24625) 10415 : cluster [DBG] pgmap v5929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:14.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.743709+0000 mgr.gibba032.whopeo (mgr.24625) 10401 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:10:14.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.745526+0000 mgr.gibba032.whopeo (mgr.24625) 10402 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:10:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.746797+0000 mgr.gibba032.whopeo (mgr.24625) 10403 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:10:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.746836+0000 mgr.gibba032.whopeo (mgr.24625) 10404 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:10:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.747955+0000 mgr.gibba032.whopeo (mgr.24625) 10405 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:10:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.748006+0000 mgr.gibba032.whopeo (mgr.24625) 10406 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:10:14.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.749220+0000 mgr.gibba032.whopeo (mgr.24625) 10407 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:10:14.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.750245+0000 mgr.gibba032.whopeo (mgr.24625) 10408 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:10:14.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.750282+0000 mgr.gibba032.whopeo (mgr.24625) 10409 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:10:14.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.751488+0000 mgr.gibba032.whopeo (mgr.24625) 10410 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:10:14.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.751822+0000 mgr.gibba032.whopeo (mgr.24625) 10411 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:10:14.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.753292+0000 mgr.gibba032.whopeo (mgr.24625) 10412 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:10:14.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.753328+0000 mgr.gibba032.whopeo (mgr.24625) 10413 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:10:14.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cephadm 2021-03-10T17:10:12.753430+0000 mgr.gibba032.whopeo (mgr.24625) 10414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:14.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:13 gibba011 bash[31774]: cluster 2021-03-10T17:10:13.138046+0000 mgr.gibba032.whopeo (mgr.24625) 10415 : cluster [DBG] pgmap v5929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.743709+0000 mgr.gibba032.whopeo (mgr.24625) 10401 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:10:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.745526+0000 mgr.gibba032.whopeo (mgr.24625) 10402 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:10:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.746797+0000 mgr.gibba032.whopeo (mgr.24625) 10403 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:10:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.746836+0000 mgr.gibba032.whopeo (mgr.24625) 10404 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:10:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.747955+0000 mgr.gibba032.whopeo (mgr.24625) 10405 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:10:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.748006+0000 mgr.gibba032.whopeo (mgr.24625) 10406 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:10:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.749220+0000 mgr.gibba032.whopeo (mgr.24625) 10407 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:10:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.750245+0000 mgr.gibba032.whopeo (mgr.24625) 10408 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:10:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.750282+0000 mgr.gibba032.whopeo (mgr.24625) 10409 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:10:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.751488+0000 mgr.gibba032.whopeo (mgr.24625) 10410 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:10:14.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.751822+0000 mgr.gibba032.whopeo (mgr.24625) 10411 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.753292+0000 mgr.gibba032.whopeo (mgr.24625) 10412 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.753328+0000 mgr.gibba032.whopeo (mgr.24625) 10413 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cephadm 2021-03-10T17:10:12.753430+0000 mgr.gibba032.whopeo (mgr.24625) 10414 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:14.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:13 gibba032 bash[24004]: cluster 2021-03-10T17:10:13.138046+0000 mgr.gibba032.whopeo (mgr.24625) 10415 : cluster [DBG] pgmap v5929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:16 gibba032 bash[24004]: cluster 2021-03-10T17:10:15.139510+0000 mgr.gibba032.whopeo (mgr.24625) 10416 : cluster [DBG] pgmap v5930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:16 gibba011 bash[30791]: cluster 2021-03-10T17:10:15.139510+0000 mgr.gibba032.whopeo (mgr.24625) 10416 : cluster [DBG] pgmap v5930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:16 gibba011 bash[31774]: cluster 2021-03-10T17:10:15.139510+0000 mgr.gibba032.whopeo (mgr.24625) 10416 : cluster [DBG] pgmap v5930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:18.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:18 gibba032 bash[24004]: cluster 2021-03-10T17:10:17.140107+0000 mgr.gibba032.whopeo (mgr.24625) 10417 : cluster [DBG] pgmap v5931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:18.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:18 gibba011 bash[30791]: cluster 2021-03-10T17:10:17.140107+0000 mgr.gibba032.whopeo (mgr.24625) 10417 : cluster [DBG] pgmap v5931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:18.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:18 gibba011 bash[31774]: cluster 2021-03-10T17:10:17.140107+0000 mgr.gibba032.whopeo (mgr.24625) 10417 : cluster [DBG] pgmap v5931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:20.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:20 gibba032 bash[24004]: cluster 2021-03-10T17:10:19.141361+0000 mgr.gibba032.whopeo (mgr.24625) 10418 : cluster [DBG] pgmap v5932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:20 gibba011 bash[30791]: cluster 2021-03-10T17:10:19.141361+0000 mgr.gibba032.whopeo (mgr.24625) 10418 : cluster [DBG] pgmap v5932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:20 gibba011 bash[31774]: cluster 2021-03-10T17:10:19.141361+0000 mgr.gibba032.whopeo (mgr.24625) 10418 : cluster [DBG] pgmap v5932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:22.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:21 gibba011 bash[30791]: cluster 2021-03-10T17:10:21.143100+0000 mgr.gibba032.whopeo (mgr.24625) 10419 : cluster [DBG] pgmap v5933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:21 gibba011 bash[31774]: cluster 2021-03-10T17:10:21.143100+0000 mgr.gibba032.whopeo (mgr.24625) 10419 : cluster [DBG] pgmap v5933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:21 gibba032 bash[24004]: cluster 2021-03-10T17:10:21.143100+0000 mgr.gibba032.whopeo (mgr.24625) 10419 : cluster [DBG] pgmap v5933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:24 gibba032 bash[24004]: cluster 2021-03-10T17:10:23.144929+0000 mgr.gibba032.whopeo (mgr.24625) 10420 : cluster [DBG] pgmap v5934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:24 gibba011 bash[30791]: cluster 2021-03-10T17:10:23.144929+0000 mgr.gibba032.whopeo (mgr.24625) 10420 : cluster [DBG] pgmap v5934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:24 gibba011 bash[31774]: cluster 2021-03-10T17:10:23.144929+0000 mgr.gibba032.whopeo (mgr.24625) 10420 : cluster [DBG] pgmap v5934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:26.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:26 gibba032 bash[24004]: cluster 2021-03-10T17:10:25.146209+0000 mgr.gibba032.whopeo (mgr.24625) 10421 : cluster [DBG] pgmap v5935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:26 gibba011 bash[30791]: cluster 2021-03-10T17:10:25.146209+0000 mgr.gibba032.whopeo (mgr.24625) 10421 : cluster [DBG] pgmap v5935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:26.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:26 gibba011 bash[31774]: cluster 2021-03-10T17:10:25.146209+0000 mgr.gibba032.whopeo (mgr.24625) 10421 : cluster [DBG] pgmap v5935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:28.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:28 gibba032 bash[24004]: cluster 2021-03-10T17:10:27.147626+0000 mgr.gibba032.whopeo (mgr.24625) 10422 : cluster [DBG] pgmap v5936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:28 gibba011 bash[30791]: cluster 2021-03-10T17:10:27.147626+0000 mgr.gibba032.whopeo (mgr.24625) 10422 : cluster [DBG] pgmap v5936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:28 gibba011 bash[31774]: cluster 2021-03-10T17:10:27.147626+0000 mgr.gibba032.whopeo (mgr.24625) 10422 : cluster [DBG] pgmap v5936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:29.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:29 gibba032 bash[24004]: cephadm 2021-03-10T17:10:27.769225+0000 mgr.gibba032.whopeo (mgr.24625) 10423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:29.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:29 gibba011 bash[30791]: cephadm 2021-03-10T17:10:27.769225+0000 mgr.gibba032.whopeo (mgr.24625) 10423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:29.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:29 gibba011 bash[31774]: cephadm 2021-03-10T17:10:27.769225+0000 mgr.gibba032.whopeo (mgr.24625) 10423 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:30.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:30 gibba032 bash[24004]: cluster 2021-03-10T17:10:29.148933+0000 mgr.gibba032.whopeo (mgr.24625) 10424 : cluster [DBG] pgmap v5937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:30 gibba011 bash[30791]: cluster 2021-03-10T17:10:29.148933+0000 mgr.gibba032.whopeo (mgr.24625) 10424 : cluster [DBG] pgmap v5937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:30 gibba011 bash[31774]: cluster 2021-03-10T17:10:29.148933+0000 mgr.gibba032.whopeo (mgr.24625) 10424 : cluster [DBG] pgmap v5937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:31 gibba011 bash[30791]: cluster 2021-03-10T17:10:31.150450+0000 mgr.gibba032.whopeo (mgr.24625) 10425 : cluster [DBG] pgmap v5938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:32.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:31 gibba011 bash[31774]: cluster 2021-03-10T17:10:31.150450+0000 mgr.gibba032.whopeo (mgr.24625) 10425 : cluster [DBG] pgmap v5938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:31 gibba032 bash[24004]: cluster 2021-03-10T17:10:31.150450+0000 mgr.gibba032.whopeo (mgr.24625) 10425 : cluster [DBG] pgmap v5938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:33.968 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:10:34.199 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:10:34.199 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:10:34.200 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:10:34.200 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:10:34.201 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:10:34.201 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:10:34.201 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:10:34.202 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:10:34.202 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:10:34.202 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:10:34.203 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:10:34.203 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:10:34.204 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:10:34.204 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:10:34.204 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:10:34.205 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:10:34.206 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:10:34.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:10:34.207 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:10:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:10:34.208 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:10:34.448 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:10:34.449 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:10:34.449 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:10:34.449 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:34.450 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:10:34.450 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:10:34.450 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:34.450 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:10:34.450 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:10:34.451 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:10:34.451 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:10:34.451 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:10:34.451 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:10:34.452 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:10:34.452 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:10:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:34 gibba032 bash[24004]: cluster 2021-03-10T17:10:33.152135+0000 mgr.gibba032.whopeo (mgr.24625) 10426 : cluster [DBG] pgmap v5939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:34 gibba011 bash[30791]: cluster 2021-03-10T17:10:33.152135+0000 mgr.gibba032.whopeo (mgr.24625) 10426 : cluster [DBG] pgmap v5939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:34 gibba011 bash[31774]: cluster 2021-03-10T17:10:33.152135+0000 mgr.gibba032.whopeo (mgr.24625) 10426 : cluster [DBG] pgmap v5939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:35.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:35 gibba032 bash[24004]: audit 2021-03-10T17:10:33.957389+0000 mgr.gibba032.whopeo (mgr.24625) 10427 : audit [DBG] from='client.61021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:35 gibba032 bash[24004]: audit 2021-03-10T17:10:34.185087+0000 mgr.gibba032.whopeo (mgr.24625) 10428 : audit [DBG] from='client.51045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:35 gibba032 bash[24004]: audit 2021-03-10T17:10:34.448698+0000 mon.c (mon.1) 201 : audit [DBG] from='client.? 172.21.2.111:0/3622271446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[30791]: audit 2021-03-10T17:10:33.957389+0000 mgr.gibba032.whopeo (mgr.24625) 10427 : audit [DBG] from='client.61021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[30791]: audit 2021-03-10T17:10:34.185087+0000 mgr.gibba032.whopeo (mgr.24625) 10428 : audit [DBG] from='client.51045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[30791]: audit 2021-03-10T17:10:34.448698+0000 mon.c (mon.1) 201 : audit [DBG] from='client.? 172.21.2.111:0/3622271446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[31774]: audit 2021-03-10T17:10:33.957389+0000 mgr.gibba032.whopeo (mgr.24625) 10427 : audit [DBG] from='client.61021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[31774]: audit 2021-03-10T17:10:34.185087+0000 mgr.gibba032.whopeo (mgr.24625) 10428 : audit [DBG] from='client.51045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:10:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:35 gibba011 bash[31774]: audit 2021-03-10T17:10:34.448698+0000 mon.c (mon.1) 201 : audit [DBG] from='client.? 172.21.2.111:0/3622271446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:10:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:36 gibba032 bash[24004]: cluster 2021-03-10T17:10:35.153411+0000 mgr.gibba032.whopeo (mgr.24625) 10429 : cluster [DBG] pgmap v5940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:36 gibba011 bash[30791]: cluster 2021-03-10T17:10:35.153411+0000 mgr.gibba032.whopeo (mgr.24625) 10429 : cluster [DBG] pgmap v5940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:36.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:36 gibba011 bash[31774]: cluster 2021-03-10T17:10:35.153411+0000 mgr.gibba032.whopeo (mgr.24625) 10429 : cluster [DBG] pgmap v5940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:38 gibba032 bash[24004]: cluster 2021-03-10T17:10:37.154970+0000 mgr.gibba032.whopeo (mgr.24625) 10430 : cluster [DBG] pgmap v5941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:38 gibba011 bash[30791]: cluster 2021-03-10T17:10:37.154970+0000 mgr.gibba032.whopeo (mgr.24625) 10430 : cluster [DBG] pgmap v5941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:38.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:38 gibba011 bash[31774]: cluster 2021-03-10T17:10:37.154970+0000 mgr.gibba032.whopeo (mgr.24625) 10430 : cluster [DBG] pgmap v5941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:40.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:40 gibba032 bash[24004]: cluster 2021-03-10T17:10:39.156451+0000 mgr.gibba032.whopeo (mgr.24625) 10431 : cluster [DBG] pgmap v5942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:40 gibba011 bash[30791]: cluster 2021-03-10T17:10:39.156451+0000 mgr.gibba032.whopeo (mgr.24625) 10431 : cluster [DBG] pgmap v5942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:40 gibba011 bash[31774]: cluster 2021-03-10T17:10:39.156451+0000 mgr.gibba032.whopeo (mgr.24625) 10431 : cluster [DBG] pgmap v5942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:41 gibba011 bash[30791]: cluster 2021-03-10T17:10:41.157992+0000 mgr.gibba032.whopeo (mgr.24625) 10432 : cluster [DBG] pgmap v5943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:42.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:41 gibba011 bash[31774]: cluster 2021-03-10T17:10:41.157992+0000 mgr.gibba032.whopeo (mgr.24625) 10432 : cluster [DBG] pgmap v5943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:41 gibba032 bash[24004]: cluster 2021-03-10T17:10:41.157992+0000 mgr.gibba032.whopeo (mgr.24625) 10432 : cluster [DBG] pgmap v5943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:44.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:44 gibba032 bash[24004]: cephadm 2021-03-10T17:10:42.784996+0000 mgr.gibba032.whopeo (mgr.24625) 10433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:44.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:44 gibba032 bash[24004]: cluster 2021-03-10T17:10:43.159502+0000 mgr.gibba032.whopeo (mgr.24625) 10434 : cluster [DBG] pgmap v5944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:44 gibba011 bash[30791]: cephadm 2021-03-10T17:10:42.784996+0000 mgr.gibba032.whopeo (mgr.24625) 10433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:44 gibba011 bash[30791]: cluster 2021-03-10T17:10:43.159502+0000 mgr.gibba032.whopeo (mgr.24625) 10434 : cluster [DBG] pgmap v5944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:44 gibba011 bash[31774]: cephadm 2021-03-10T17:10:42.784996+0000 mgr.gibba032.whopeo (mgr.24625) 10433 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:44 gibba011 bash[31774]: cluster 2021-03-10T17:10:43.159502+0000 mgr.gibba032.whopeo (mgr.24625) 10434 : cluster [DBG] pgmap v5944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:46.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:46 gibba032 bash[24004]: cluster 2021-03-10T17:10:45.161123+0000 mgr.gibba032.whopeo (mgr.24625) 10435 : cluster [DBG] pgmap v5945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:46 gibba011 bash[30791]: cluster 2021-03-10T17:10:45.161123+0000 mgr.gibba032.whopeo (mgr.24625) 10435 : cluster [DBG] pgmap v5945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:46 gibba011 bash[31774]: cluster 2021-03-10T17:10:45.161123+0000 mgr.gibba032.whopeo (mgr.24625) 10435 : cluster [DBG] pgmap v5945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:48.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:48 gibba032 bash[24004]: cluster 2021-03-10T17:10:47.162542+0000 mgr.gibba032.whopeo (mgr.24625) 10436 : cluster [DBG] pgmap v5946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:48 gibba011 bash[30791]: cluster 2021-03-10T17:10:47.162542+0000 mgr.gibba032.whopeo (mgr.24625) 10436 : cluster [DBG] pgmap v5946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:48 gibba011 bash[31774]: cluster 2021-03-10T17:10:47.162542+0000 mgr.gibba032.whopeo (mgr.24625) 10436 : cluster [DBG] pgmap v5946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:50.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:50 gibba032 bash[24004]: cluster 2021-03-10T17:10:49.163423+0000 mgr.gibba032.whopeo (mgr.24625) 10437 : cluster [DBG] pgmap v5947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:50 gibba011 bash[30791]: cluster 2021-03-10T17:10:49.163423+0000 mgr.gibba032.whopeo (mgr.24625) 10437 : cluster [DBG] pgmap v5947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:50 gibba011 bash[31774]: cluster 2021-03-10T17:10:49.163423+0000 mgr.gibba032.whopeo (mgr.24625) 10437 : cluster [DBG] pgmap v5947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:51 gibba011 bash[30791]: cluster 2021-03-10T17:10:51.165115+0000 mgr.gibba032.whopeo (mgr.24625) 10438 : cluster [DBG] pgmap v5948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:51 gibba011 bash[31774]: cluster 2021-03-10T17:10:51.165115+0000 mgr.gibba032.whopeo (mgr.24625) 10438 : cluster [DBG] pgmap v5948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:51 gibba032 bash[24004]: cluster 2021-03-10T17:10:51.165115+0000 mgr.gibba032.whopeo (mgr.24625) 10438 : cluster [DBG] pgmap v5948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:54 gibba032 bash[24004]: cluster 2021-03-10T17:10:53.166851+0000 mgr.gibba032.whopeo (mgr.24625) 10439 : cluster [DBG] pgmap v5949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:54 gibba011 bash[30791]: cluster 2021-03-10T17:10:53.166851+0000 mgr.gibba032.whopeo (mgr.24625) 10439 : cluster [DBG] pgmap v5949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:54 gibba011 bash[31774]: cluster 2021-03-10T17:10:53.166851+0000 mgr.gibba032.whopeo (mgr.24625) 10439 : cluster [DBG] pgmap v5949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:56.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:56 gibba032 bash[24004]: cluster 2021-03-10T17:10:55.168353+0000 mgr.gibba032.whopeo (mgr.24625) 10440 : cluster [DBG] pgmap v5950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:56 gibba011 bash[30791]: cluster 2021-03-10T17:10:55.168353+0000 mgr.gibba032.whopeo (mgr.24625) 10440 : cluster [DBG] pgmap v5950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:56.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:56 gibba011 bash[31774]: cluster 2021-03-10T17:10:55.168353+0000 mgr.gibba032.whopeo (mgr.24625) 10440 : cluster [DBG] pgmap v5950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: cluster 2021-03-10T17:10:57.169055+0000 mgr.gibba032.whopeo (mgr.24625) 10441 : cluster [DBG] pgmap v5951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:58.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.354525+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:10:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.355219+0000 mon.a (mon.0) 3491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:10:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.527709+0000 mon.a (mon.0) 3492 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:10:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.528533+0000 mon.a (mon.0) 3493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:10:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.529321+0000 mon.a (mon.0) 3494 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:10:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:58 gibba032 bash[24004]: audit 2021-03-10T17:10:57.530033+0000 mon.a (mon.0) 3495 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:10:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: cluster 2021-03-10T17:10:57.169055+0000 mgr.gibba032.whopeo (mgr.24625) 10441 : cluster [DBG] pgmap v5951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.354525+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:10:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.355219+0000 mon.a (mon.0) 3491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:10:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.527709+0000 mon.a (mon.0) 3492 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:10:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.528533+0000 mon.a (mon.0) 3493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:10:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.529321+0000 mon.a (mon.0) 3494 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:10:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[30791]: audit 2021-03-10T17:10:57.530033+0000 mon.a (mon.0) 3495 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:10:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: cluster 2021-03-10T17:10:57.169055+0000 mgr.gibba032.whopeo (mgr.24625) 10441 : cluster [DBG] pgmap v5951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:10:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.354525+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:10:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.355219+0000 mon.a (mon.0) 3491 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:10:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.527709+0000 mon.a (mon.0) 3492 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:10:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.528533+0000 mon.a (mon.0) 3493 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:10:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.529321+0000 mon.a (mon.0) 3494 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:10:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:58 gibba011 bash[31774]: audit 2021-03-10T17:10:57.530033+0000 mon.a (mon.0) 3495 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:10:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:10:59 gibba011 bash[30791]: cephadm 2021-03-10T17:10:57.801064+0000 mgr.gibba032.whopeo (mgr.24625) 10442 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:59.756 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:10:59 gibba011 bash[31774]: cephadm 2021-03-10T17:10:57.801064+0000 mgr.gibba032.whopeo (mgr.24625) 10442 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:10:59.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:10:59 gibba032 bash[24004]: cephadm 2021-03-10T17:10:57.801064+0000 mgr.gibba032.whopeo (mgr.24625) 10442 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:00.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:00 gibba011 bash[30791]: cluster 2021-03-10T17:10:59.170330+0000 mgr.gibba032.whopeo (mgr.24625) 10443 : cluster [DBG] pgmap v5952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:00 gibba011 bash[31774]: cluster 2021-03-10T17:10:59.170330+0000 mgr.gibba032.whopeo (mgr.24625) 10443 : cluster [DBG] pgmap v5952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:00 gibba032 bash[24004]: cluster 2021-03-10T17:10:59.170330+0000 mgr.gibba032.whopeo (mgr.24625) 10443 : cluster [DBG] pgmap v5952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:02.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:01 gibba011 bash[30791]: cluster 2021-03-10T17:11:01.171953+0000 mgr.gibba032.whopeo (mgr.24625) 10444 : cluster [DBG] pgmap v5953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:02.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:01 gibba011 bash[31774]: cluster 2021-03-10T17:11:01.171953+0000 mgr.gibba032.whopeo (mgr.24625) 10444 : cluster [DBG] pgmap v5953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:01 gibba032 bash[24004]: cluster 2021-03-10T17:11:01.171953+0000 mgr.gibba032.whopeo (mgr.24625) 10444 : cluster [DBG] pgmap v5953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:04 gibba032 bash[24004]: cluster 2021-03-10T17:11:03.173494+0000 mgr.gibba032.whopeo (mgr.24625) 10445 : cluster [DBG] pgmap v5954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:04.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:04 gibba011 bash[30791]: cluster 2021-03-10T17:11:03.173494+0000 mgr.gibba032.whopeo (mgr.24625) 10445 : cluster [DBG] pgmap v5954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:04.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:04 gibba011 bash[31774]: cluster 2021-03-10T17:11:03.173494+0000 mgr.gibba032.whopeo (mgr.24625) 10445 : cluster [DBG] pgmap v5954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:04.695 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:11:04.923 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:11:04.924 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:11:04.924 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:11:04.925 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:11:04.925 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:11:04.925 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:11:04.926 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:11:04.926 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:11:04.927 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:11:04.927 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:11:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:11:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:11:04.928 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:11:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:11:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:11:04.929 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:11:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:11:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:11:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:11:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:11:04.930 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:11:05.178 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:11:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:11:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:11:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:11:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:11:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:11:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:11:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:11:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:11:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:11:05.183 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:11:05.183 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:11:05.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:05 gibba032 bash[24004]: audit 2021-03-10T17:11:04.684162+0000 mgr.gibba032.whopeo (mgr.24625) 10446 : audit [DBG] from='client.51057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:05.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:05 gibba032 bash[24004]: audit 2021-03-10T17:11:05.178186+0000 mon.a (mon.0) 3496 : audit [DBG] from='client.? 172.21.2.111:0/2749216443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:05 gibba011 bash[30791]: audit 2021-03-10T17:11:04.684162+0000 mgr.gibba032.whopeo (mgr.24625) 10446 : audit [DBG] from='client.51057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:05 gibba011 bash[30791]: audit 2021-03-10T17:11:05.178186+0000 mon.a (mon.0) 3496 : audit [DBG] from='client.? 172.21.2.111:0/2749216443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:05.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:05 gibba011 bash[31774]: audit 2021-03-10T17:11:04.684162+0000 mgr.gibba032.whopeo (mgr.24625) 10446 : audit [DBG] from='client.51057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:05 gibba011 bash[31774]: audit 2021-03-10T17:11:05.178186+0000 mon.a (mon.0) 3496 : audit [DBG] from='client.? 172.21.2.111:0/2749216443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:06.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:06 gibba032 bash[24004]: audit 2021-03-10T17:11:04.909469+0000 mgr.gibba032.whopeo (mgr.24625) 10447 : audit [DBG] from='client.51063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:06.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:06 gibba032 bash[24004]: cluster 2021-03-10T17:11:05.174468+0000 mgr.gibba032.whopeo (mgr.24625) 10448 : cluster [DBG] pgmap v5955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:06 gibba011 bash[30791]: audit 2021-03-10T17:11:04.909469+0000 mgr.gibba032.whopeo (mgr.24625) 10447 : audit [DBG] from='client.51063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:06 gibba011 bash[30791]: cluster 2021-03-10T17:11:05.174468+0000 mgr.gibba032.whopeo (mgr.24625) 10448 : cluster [DBG] pgmap v5955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:06 gibba011 bash[31774]: audit 2021-03-10T17:11:04.909469+0000 mgr.gibba032.whopeo (mgr.24625) 10447 : audit [DBG] from='client.51063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:06 gibba011 bash[31774]: cluster 2021-03-10T17:11:05.174468+0000 mgr.gibba032.whopeo (mgr.24625) 10448 : cluster [DBG] pgmap v5955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:08 gibba011 bash[30791]: cluster 2021-03-10T17:11:07.175790+0000 mgr.gibba032.whopeo (mgr.24625) 10449 : cluster [DBG] pgmap v5956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:08 gibba011 bash[31774]: cluster 2021-03-10T17:11:07.175790+0000 mgr.gibba032.whopeo (mgr.24625) 10449 : cluster [DBG] pgmap v5956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:08 gibba032 bash[24004]: cluster 2021-03-10T17:11:07.175790+0000 mgr.gibba032.whopeo (mgr.24625) 10449 : cluster [DBG] pgmap v5956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:10.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:10 gibba011 bash[30791]: cluster 2021-03-10T17:11:09.177201+0000 mgr.gibba032.whopeo (mgr.24625) 10450 : cluster [DBG] pgmap v5957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:10.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:10 gibba011 bash[31774]: cluster 2021-03-10T17:11:09.177201+0000 mgr.gibba032.whopeo (mgr.24625) 10450 : cluster [DBG] pgmap v5957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:10 gibba032 bash[24004]: cluster 2021-03-10T17:11:09.177201+0000 mgr.gibba032.whopeo (mgr.24625) 10450 : cluster [DBG] pgmap v5957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:11 gibba011 bash[30791]: cluster 2021-03-10T17:11:11.178712+0000 mgr.gibba032.whopeo (mgr.24625) 10451 : cluster [DBG] pgmap v5958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:12.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:11 gibba011 bash[31774]: cluster 2021-03-10T17:11:11.178712+0000 mgr.gibba032.whopeo (mgr.24625) 10451 : cluster [DBG] pgmap v5958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:11 gibba032 bash[24004]: cluster 2021-03-10T17:11:11.178712+0000 mgr.gibba032.whopeo (mgr.24625) 10451 : cluster [DBG] pgmap v5958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:13.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:12 gibba032 bash[24004]: audit 2021-03-10T17:11:12.820731+0000 mon.a (mon.0) 3497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:11:13.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:12 gibba011 bash[30791]: audit 2021-03-10T17:11:12.820731+0000 mon.a (mon.0) 3497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:11:13.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:12 gibba011 bash[31774]: audit 2021-03-10T17:11:12.820731+0000 mon.a (mon.0) 3497 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:11:14.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:13 gibba032 bash[24004]: cephadm 2021-03-10T17:11:12.818505+0000 mgr.gibba032.whopeo (mgr.24625) 10452 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:11:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:13 gibba032 bash[24004]: cluster 2021-03-10T17:11:13.180373+0000 mgr.gibba032.whopeo (mgr.24625) 10453 : cluster [DBG] pgmap v5959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:14.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:13 gibba011 bash[30791]: cephadm 2021-03-10T17:11:12.818505+0000 mgr.gibba032.whopeo (mgr.24625) 10452 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:11:14.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:13 gibba011 bash[30791]: cluster 2021-03-10T17:11:13.180373+0000 mgr.gibba032.whopeo (mgr.24625) 10453 : cluster [DBG] pgmap v5959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:14.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:13 gibba011 bash[31774]: cephadm 2021-03-10T17:11:12.818505+0000 mgr.gibba032.whopeo (mgr.24625) 10452 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:11:14.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:13 gibba011 bash[31774]: cluster 2021-03-10T17:11:13.180373+0000 mgr.gibba032.whopeo (mgr.24625) 10453 : cluster [DBG] pgmap v5959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:16.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:16 gibba032 bash[24004]: cluster 2021-03-10T17:11:15.181970+0000 mgr.gibba032.whopeo (mgr.24625) 10454 : cluster [DBG] pgmap v5960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:16.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:16 gibba011 bash[30791]: cluster 2021-03-10T17:11:15.181970+0000 mgr.gibba032.whopeo (mgr.24625) 10454 : cluster [DBG] pgmap v5960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:16 gibba011 bash[31774]: cluster 2021-03-10T17:11:15.181970+0000 mgr.gibba032.whopeo (mgr.24625) 10454 : cluster [DBG] pgmap v5960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:17.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.361173+0000 mgr.gibba032.whopeo (mgr.24625) 10455 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:11:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.362453+0000 mon.a (mon.0) 3498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.572332+0000 mon.a (mon.0) 3499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.584456+0000 mgr.gibba032.whopeo (mgr.24625) 10456 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:11:17.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.589393+0000 mon.a (mon.0) 3500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:11:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.591626+0000 mgr.gibba032.whopeo (mgr.24625) 10457 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:11:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.595080+0000 mon.a (mon.0) 3501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.596479+0000 mgr.gibba032.whopeo (mgr.24625) 10458 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:11:17.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.596716+0000 mgr.gibba032.whopeo (mgr.24625) 10459 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:11:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.600357+0000 mon.a (mon.0) 3502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.601572+0000 mgr.gibba032.whopeo (mgr.24625) 10460 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:11:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.601837+0000 mgr.gibba032.whopeo (mgr.24625) 10461 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:11:17.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.604958+0000 mon.a (mon.0) 3503 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.606427+0000 mgr.gibba032.whopeo (mgr.24625) 10462 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:11:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.611051+0000 mgr.gibba032.whopeo (mgr.24625) 10463 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:11:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.611661+0000 mgr.gibba032.whopeo (mgr.24625) 10464 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:11:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.615388+0000 mon.a (mon.0) 3504 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.616570+0000 mgr.gibba032.whopeo (mgr.24625) 10465 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:11:17.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.618635+0000 mgr.gibba032.whopeo (mgr.24625) 10466 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:11:17.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: audit 2021-03-10T17:11:16.621779+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.622993+0000 mgr.gibba032.whopeo (mgr.24625) 10467 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:11:17.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.623518+0000 mgr.gibba032.whopeo (mgr.24625) 10468 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:11:17.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:17 gibba032 bash[24004]: cephadm 2021-03-10T17:11:16.624177+0000 mgr.gibba032.whopeo (mgr.24625) 10469 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.361173+0000 mgr.gibba032.whopeo (mgr.24625) 10455 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:11:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.362453+0000 mon.a (mon.0) 3498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.572332+0000 mon.a (mon.0) 3499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.584456+0000 mgr.gibba032.whopeo (mgr.24625) 10456 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:11:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.589393+0000 mon.a (mon.0) 3500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:11:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.591626+0000 mgr.gibba032.whopeo (mgr.24625) 10457 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:11:17.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.595080+0000 mon.a (mon.0) 3501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.596479+0000 mgr.gibba032.whopeo (mgr.24625) 10458 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:11:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.596716+0000 mgr.gibba032.whopeo (mgr.24625) 10459 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:11:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.600357+0000 mon.a (mon.0) 3502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.601572+0000 mgr.gibba032.whopeo (mgr.24625) 10460 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:11:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.601837+0000 mgr.gibba032.whopeo (mgr.24625) 10461 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:11:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.604958+0000 mon.a (mon.0) 3503 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.606427+0000 mgr.gibba032.whopeo (mgr.24625) 10462 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:11:17.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.611051+0000 mgr.gibba032.whopeo (mgr.24625) 10463 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:11:17.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.611661+0000 mgr.gibba032.whopeo (mgr.24625) 10464 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:11:17.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.615388+0000 mon.a (mon.0) 3504 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.616570+0000 mgr.gibba032.whopeo (mgr.24625) 10465 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:11:17.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.618635+0000 mgr.gibba032.whopeo (mgr.24625) 10466 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:11:17.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: audit 2021-03-10T17:11:16.621779+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.622993+0000 mgr.gibba032.whopeo (mgr.24625) 10467 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:11:17.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.623518+0000 mgr.gibba032.whopeo (mgr.24625) 10468 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:11:17.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[30791]: cephadm 2021-03-10T17:11:16.624177+0000 mgr.gibba032.whopeo (mgr.24625) 10469 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.361173+0000 mgr.gibba032.whopeo (mgr.24625) 10455 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:11:17.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.362453+0000 mon.a (mon.0) 3498 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.572332+0000 mon.a (mon.0) 3499 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:11:17.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.584456+0000 mgr.gibba032.whopeo (mgr.24625) 10456 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:11:17.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.589393+0000 mon.a (mon.0) 3500 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:11:17.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.591626+0000 mgr.gibba032.whopeo (mgr.24625) 10457 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:11:17.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.595080+0000 mon.a (mon.0) 3501 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.596479+0000 mgr.gibba032.whopeo (mgr.24625) 10458 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:11:17.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.596716+0000 mgr.gibba032.whopeo (mgr.24625) 10459 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:11:17.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.600357+0000 mon.a (mon.0) 3502 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.601572+0000 mgr.gibba032.whopeo (mgr.24625) 10460 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:11:17.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.601837+0000 mgr.gibba032.whopeo (mgr.24625) 10461 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:11:17.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.604958+0000 mon.a (mon.0) 3503 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.606427+0000 mgr.gibba032.whopeo (mgr.24625) 10462 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:11:17.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.611051+0000 mgr.gibba032.whopeo (mgr.24625) 10463 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:11:17.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.611661+0000 mgr.gibba032.whopeo (mgr.24625) 10464 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:11:17.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.615388+0000 mon.a (mon.0) 3504 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.616570+0000 mgr.gibba032.whopeo (mgr.24625) 10465 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:11:17.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.618635+0000 mgr.gibba032.whopeo (mgr.24625) 10466 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:11:17.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: audit 2021-03-10T17:11:16.621779+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:17.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.622993+0000 mgr.gibba032.whopeo (mgr.24625) 10467 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:11:17.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.623518+0000 mgr.gibba032.whopeo (mgr.24625) 10468 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:11:17.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:17 gibba011 bash[31774]: cephadm 2021-03-10T17:11:16.624177+0000 mgr.gibba032.whopeo (mgr.24625) 10469 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:19.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:18 gibba032 bash[24004]: cluster 2021-03-10T17:11:17.183255+0000 mgr.gibba032.whopeo (mgr.24625) 10470 : cluster [DBG] pgmap v5961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:18 gibba011 bash[30791]: cluster 2021-03-10T17:11:17.183255+0000 mgr.gibba032.whopeo (mgr.24625) 10470 : cluster [DBG] pgmap v5961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:18 gibba011 bash[31774]: cluster 2021-03-10T17:11:17.183255+0000 mgr.gibba032.whopeo (mgr.24625) 10470 : cluster [DBG] pgmap v5961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:20 gibba032 bash[24004]: cluster 2021-03-10T17:11:19.184874+0000 mgr.gibba032.whopeo (mgr.24625) 10471 : cluster [DBG] pgmap v5962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:20 gibba011 bash[30791]: cluster 2021-03-10T17:11:19.184874+0000 mgr.gibba032.whopeo (mgr.24625) 10471 : cluster [DBG] pgmap v5962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:20 gibba011 bash[31774]: cluster 2021-03-10T17:11:19.184874+0000 mgr.gibba032.whopeo (mgr.24625) 10471 : cluster [DBG] pgmap v5962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:21 gibba011 bash[30791]: cluster 2021-03-10T17:11:21.186343+0000 mgr.gibba032.whopeo (mgr.24625) 10472 : cluster [DBG] pgmap v5963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:22.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:21 gibba011 bash[31774]: cluster 2021-03-10T17:11:21.186343+0000 mgr.gibba032.whopeo (mgr.24625) 10472 : cluster [DBG] pgmap v5963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:22.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:21 gibba032 bash[24004]: cluster 2021-03-10T17:11:21.186343+0000 mgr.gibba032.whopeo (mgr.24625) 10472 : cluster [DBG] pgmap v5963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:24 gibba032 bash[24004]: cluster 2021-03-10T17:11:23.187999+0000 mgr.gibba032.whopeo (mgr.24625) 10473 : cluster [DBG] pgmap v5964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:24 gibba011 bash[30791]: cluster 2021-03-10T17:11:23.187999+0000 mgr.gibba032.whopeo (mgr.24625) 10473 : cluster [DBG] pgmap v5964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:24 gibba011 bash[31774]: cluster 2021-03-10T17:11:23.187999+0000 mgr.gibba032.whopeo (mgr.24625) 10473 : cluster [DBG] pgmap v5964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:26.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:26 gibba011 bash[30791]: cluster 2021-03-10T17:11:25.189605+0000 mgr.gibba032.whopeo (mgr.24625) 10474 : cluster [DBG] pgmap v5965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:26 gibba011 bash[31774]: cluster 2021-03-10T17:11:25.189605+0000 mgr.gibba032.whopeo (mgr.24625) 10474 : cluster [DBG] pgmap v5965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:26 gibba032 bash[24004]: cluster 2021-03-10T17:11:25.189605+0000 mgr.gibba032.whopeo (mgr.24625) 10474 : cluster [DBG] pgmap v5965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:28.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:28 gibba011 bash[30791]: cluster 2021-03-10T17:11:27.191125+0000 mgr.gibba032.whopeo (mgr.24625) 10475 : cluster [DBG] pgmap v5966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:28.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:28 gibba011 bash[31774]: cluster 2021-03-10T17:11:27.191125+0000 mgr.gibba032.whopeo (mgr.24625) 10475 : cluster [DBG] pgmap v5966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:28 gibba032 bash[24004]: cluster 2021-03-10T17:11:27.191125+0000 mgr.gibba032.whopeo (mgr.24625) 10475 : cluster [DBG] pgmap v5966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:30.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:30 gibba011 bash[30791]: cluster 2021-03-10T17:11:29.192252+0000 mgr.gibba032.whopeo (mgr.24625) 10476 : cluster [DBG] pgmap v5967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:30.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:30 gibba011 bash[31774]: cluster 2021-03-10T17:11:29.192252+0000 mgr.gibba032.whopeo (mgr.24625) 10476 : cluster [DBG] pgmap v5967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:30 gibba032 bash[24004]: cluster 2021-03-10T17:11:29.192252+0000 mgr.gibba032.whopeo (mgr.24625) 10476 : cluster [DBG] pgmap v5967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:32.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:31 gibba032 bash[24004]: cluster 2021-03-10T17:11:31.193868+0000 mgr.gibba032.whopeo (mgr.24625) 10477 : cluster [DBG] pgmap v5968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:32.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:31 gibba032 bash[24004]: cephadm 2021-03-10T17:11:31.640248+0000 mgr.gibba032.whopeo (mgr.24625) 10478 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:31 gibba011 bash[30791]: cluster 2021-03-10T17:11:31.193868+0000 mgr.gibba032.whopeo (mgr.24625) 10477 : cluster [DBG] pgmap v5968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:32.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:31 gibba011 bash[30791]: cephadm 2021-03-10T17:11:31.640248+0000 mgr.gibba032.whopeo (mgr.24625) 10478 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:31 gibba011 bash[31774]: cluster 2021-03-10T17:11:31.193868+0000 mgr.gibba032.whopeo (mgr.24625) 10477 : cluster [DBG] pgmap v5968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:32.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:31 gibba011 bash[31774]: cephadm 2021-03-10T17:11:31.640248+0000 mgr.gibba032.whopeo (mgr.24625) 10478 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:34 gibba011 bash[30791]: cluster 2021-03-10T17:11:33.195614+0000 mgr.gibba032.whopeo (mgr.24625) 10479 : cluster [DBG] pgmap v5969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:34 gibba011 bash[31774]: cluster 2021-03-10T17:11:33.195614+0000 mgr.gibba032.whopeo (mgr.24625) 10479 : cluster [DBG] pgmap v5969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:34 gibba032 bash[24004]: cluster 2021-03-10T17:11:33.195614+0000 mgr.gibba032.whopeo (mgr.24625) 10479 : cluster [DBG] pgmap v5969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:35.425 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:11:35.655 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:11:35.655 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 19s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:11:35.656 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:11:35.656 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:11:35.656 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:11:35.656 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:11:35.656 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:11:35.657 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:11:35.657 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:11:35.657 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 19s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:11:35.657 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:11:35.658 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:11:35.658 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:11:35.658 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:11:35.658 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:11:35.658 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:11:35.659 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:11:35.659 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:11:35.659 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:11:35.659 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:11:35.659 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 19s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:11:35.904 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:11:35.905 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:11:35.905 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:11:35.905 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:35.906 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:11:35.906 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:11:35.906 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:35.906 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:11:35.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:11:35.907 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:11:35.907 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:11:35.907 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:11:35.908 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:11:35.908 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:11:35.908 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:11:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[30791]: cluster 2021-03-10T17:11:35.197161+0000 mgr.gibba032.whopeo (mgr.24625) 10480 : cluster [DBG] pgmap v5970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[30791]: audit 2021-03-10T17:11:35.414273+0000 mgr.gibba032.whopeo (mgr.24625) 10481 : audit [DBG] from='client.51075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[30791]: audit 2021-03-10T17:11:35.639613+0000 mgr.gibba032.whopeo (mgr.24625) 10482 : audit [DBG] from='client.51081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[30791]: audit 2021-03-10T17:11:35.904893+0000 mon.c (mon.1) 202 : audit [DBG] from='client.? 172.21.2.111:0/18238487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[31774]: cluster 2021-03-10T17:11:35.197161+0000 mgr.gibba032.whopeo (mgr.24625) 10480 : cluster [DBG] pgmap v5970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[31774]: audit 2021-03-10T17:11:35.414273+0000 mgr.gibba032.whopeo (mgr.24625) 10481 : audit [DBG] from='client.51075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[31774]: audit 2021-03-10T17:11:35.639613+0000 mgr.gibba032.whopeo (mgr.24625) 10482 : audit [DBG] from='client.51081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:36 gibba011 bash[31774]: audit 2021-03-10T17:11:35.904893+0000 mon.c (mon.1) 202 : audit [DBG] from='client.? 172.21.2.111:0/18238487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:36 gibba032 bash[24004]: cluster 2021-03-10T17:11:35.197161+0000 mgr.gibba032.whopeo (mgr.24625) 10480 : cluster [DBG] pgmap v5970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:36 gibba032 bash[24004]: audit 2021-03-10T17:11:35.414273+0000 mgr.gibba032.whopeo (mgr.24625) 10481 : audit [DBG] from='client.51075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:36 gibba032 bash[24004]: audit 2021-03-10T17:11:35.639613+0000 mgr.gibba032.whopeo (mgr.24625) 10482 : audit [DBG] from='client.51081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:11:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:36 gibba032 bash[24004]: audit 2021-03-10T17:11:35.904893+0000 mon.c (mon.1) 202 : audit [DBG] from='client.? 172.21.2.111:0/18238487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:11:38.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:38 gibba011 bash[30791]: cluster 2021-03-10T17:11:37.198425+0000 mgr.gibba032.whopeo (mgr.24625) 10483 : cluster [DBG] pgmap v5971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:38.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:38 gibba011 bash[31774]: cluster 2021-03-10T17:11:37.198425+0000 mgr.gibba032.whopeo (mgr.24625) 10483 : cluster [DBG] pgmap v5971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:38 gibba032 bash[24004]: cluster 2021-03-10T17:11:37.198425+0000 mgr.gibba032.whopeo (mgr.24625) 10483 : cluster [DBG] pgmap v5971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:40.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:40 gibba011 bash[30791]: cluster 2021-03-10T17:11:39.199748+0000 mgr.gibba032.whopeo (mgr.24625) 10484 : cluster [DBG] pgmap v5972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:40.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:40 gibba011 bash[31774]: cluster 2021-03-10T17:11:39.199748+0000 mgr.gibba032.whopeo (mgr.24625) 10484 : cluster [DBG] pgmap v5972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:40 gibba032 bash[24004]: cluster 2021-03-10T17:11:39.199748+0000 mgr.gibba032.whopeo (mgr.24625) 10484 : cluster [DBG] pgmap v5972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:42.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:41 gibba011 bash[30791]: cluster 2021-03-10T17:11:41.201416+0000 mgr.gibba032.whopeo (mgr.24625) 10485 : cluster [DBG] pgmap v5973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:41 gibba011 bash[31774]: cluster 2021-03-10T17:11:41.201416+0000 mgr.gibba032.whopeo (mgr.24625) 10485 : cluster [DBG] pgmap v5973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:41 gibba032 bash[24004]: cluster 2021-03-10T17:11:41.201416+0000 mgr.gibba032.whopeo (mgr.24625) 10485 : cluster [DBG] pgmap v5973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:44 gibba011 bash[30791]: cluster 2021-03-10T17:11:43.203265+0000 mgr.gibba032.whopeo (mgr.24625) 10486 : cluster [DBG] pgmap v5974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:44 gibba011 bash[31774]: cluster 2021-03-10T17:11:43.203265+0000 mgr.gibba032.whopeo (mgr.24625) 10486 : cluster [DBG] pgmap v5974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:44 gibba032 bash[24004]: cluster 2021-03-10T17:11:43.203265+0000 mgr.gibba032.whopeo (mgr.24625) 10486 : cluster [DBG] pgmap v5974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:46 gibba011 bash[30791]: cluster 2021-03-10T17:11:45.204771+0000 mgr.gibba032.whopeo (mgr.24625) 10487 : cluster [DBG] pgmap v5975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:46 gibba011 bash[31774]: cluster 2021-03-10T17:11:45.204771+0000 mgr.gibba032.whopeo (mgr.24625) 10487 : cluster [DBG] pgmap v5975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:46.658 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:46 gibba032 bash[24004]: cluster 2021-03-10T17:11:45.204771+0000 mgr.gibba032.whopeo (mgr.24625) 10487 : cluster [DBG] pgmap v5975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:47.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:47 gibba011 bash[30791]: cephadm 2021-03-10T17:11:46.656369+0000 mgr.gibba032.whopeo (mgr.24625) 10488 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:47.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:47 gibba011 bash[31774]: cephadm 2021-03-10T17:11:46.656369+0000 mgr.gibba032.whopeo (mgr.24625) 10488 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:47.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:47 gibba032 bash[24004]: cephadm 2021-03-10T17:11:46.656369+0000 mgr.gibba032.whopeo (mgr.24625) 10488 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:11:48.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:48 gibba011 bash[30791]: cluster 2021-03-10T17:11:47.206484+0000 mgr.gibba032.whopeo (mgr.24625) 10489 : cluster [DBG] pgmap v5976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:48.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:48 gibba011 bash[31774]: cluster 2021-03-10T17:11:47.206484+0000 mgr.gibba032.whopeo (mgr.24625) 10489 : cluster [DBG] pgmap v5976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:48 gibba032 bash[24004]: cluster 2021-03-10T17:11:47.206484+0000 mgr.gibba032.whopeo (mgr.24625) 10489 : cluster [DBG] pgmap v5976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:50.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:50 gibba011 bash[30791]: cluster 2021-03-10T17:11:49.207994+0000 mgr.gibba032.whopeo (mgr.24625) 10490 : cluster [DBG] pgmap v5977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:50.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:50 gibba011 bash[31774]: cluster 2021-03-10T17:11:49.207994+0000 mgr.gibba032.whopeo (mgr.24625) 10490 : cluster [DBG] pgmap v5977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:50 gibba032 bash[24004]: cluster 2021-03-10T17:11:49.207994+0000 mgr.gibba032.whopeo (mgr.24625) 10490 : cluster [DBG] pgmap v5977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:51 gibba011 bash[30791]: cluster 2021-03-10T17:11:51.209465+0000 mgr.gibba032.whopeo (mgr.24625) 10491 : cluster [DBG] pgmap v5978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:52.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:51 gibba011 bash[31774]: cluster 2021-03-10T17:11:51.209465+0000 mgr.gibba032.whopeo (mgr.24625) 10491 : cluster [DBG] pgmap v5978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:51 gibba032 bash[24004]: cluster 2021-03-10T17:11:51.209465+0000 mgr.gibba032.whopeo (mgr.24625) 10491 : cluster [DBG] pgmap v5978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:54.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:54 gibba011 bash[30791]: cluster 2021-03-10T17:11:53.210966+0000 mgr.gibba032.whopeo (mgr.24625) 10492 : cluster [DBG] pgmap v5979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:54.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:54 gibba011 bash[31774]: cluster 2021-03-10T17:11:53.210966+0000 mgr.gibba032.whopeo (mgr.24625) 10492 : cluster [DBG] pgmap v5979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:54 gibba032 bash[24004]: cluster 2021-03-10T17:11:53.210966+0000 mgr.gibba032.whopeo (mgr.24625) 10492 : cluster [DBG] pgmap v5979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:56.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:56 gibba011 bash[30791]: cluster 2021-03-10T17:11:55.212577+0000 mgr.gibba032.whopeo (mgr.24625) 10493 : cluster [DBG] pgmap v5980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:56 gibba011 bash[31774]: cluster 2021-03-10T17:11:55.212577+0000 mgr.gibba032.whopeo (mgr.24625) 10493 : cluster [DBG] pgmap v5980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:56 gibba032 bash[24004]: cluster 2021-03-10T17:11:55.212577+0000 mgr.gibba032.whopeo (mgr.24625) 10493 : cluster [DBG] pgmap v5980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:58.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: cluster 2021-03-10T17:11:57.214245+0000 mgr.gibba032.whopeo (mgr.24625) 10494 : cluster [DBG] pgmap v5981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.359726+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:11:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.360049+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:11:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.538359+0000 mon.a (mon.0) 3508 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:11:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.539098+0000 mon.a (mon.0) 3509 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:11:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.539828+0000 mon.a (mon.0) 3510 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:11:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[30791]: audit 2021-03-10T17:11:57.540502+0000 mon.a (mon.0) 3511 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:11:58.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: cluster 2021-03-10T17:11:57.214245+0000 mgr.gibba032.whopeo (mgr.24625) 10494 : cluster [DBG] pgmap v5981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.359726+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:11:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.360049+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:11:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.538359+0000 mon.a (mon.0) 3508 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:11:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.539098+0000 mon.a (mon.0) 3509 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:11:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.539828+0000 mon.a (mon.0) 3510 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:11:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:11:58 gibba011 bash[31774]: audit 2021-03-10T17:11:57.540502+0000 mon.a (mon.0) 3511 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:11:58.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: cluster 2021-03-10T17:11:57.214245+0000 mgr.gibba032.whopeo (mgr.24625) 10494 : cluster [DBG] pgmap v5981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:11:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.359726+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:11:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.360049+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:11:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.538359+0000 mon.a (mon.0) 3508 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:11:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.539098+0000 mon.a (mon.0) 3509 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:11:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.539828+0000 mon.a (mon.0) 3510 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:11:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:11:58 gibba032 bash[24004]: audit 2021-03-10T17:11:57.540502+0000 mon.a (mon.0) 3511 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:12:00.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:00 gibba011 bash[30791]: cluster 2021-03-10T17:11:59.215723+0000 mgr.gibba032.whopeo (mgr.24625) 10495 : cluster [DBG] pgmap v5982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:00.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:00 gibba011 bash[31774]: cluster 2021-03-10T17:11:59.215723+0000 mgr.gibba032.whopeo (mgr.24625) 10495 : cluster [DBG] pgmap v5982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:00 gibba032 bash[24004]: cluster 2021-03-10T17:11:59.215723+0000 mgr.gibba032.whopeo (mgr.24625) 10495 : cluster [DBG] pgmap v5982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:02.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:01 gibba032 bash[24004]: cluster 2021-03-10T17:12:01.217512+0000 mgr.gibba032.whopeo (mgr.24625) 10496 : cluster [DBG] pgmap v5983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:02.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:01 gibba032 bash[24004]: cephadm 2021-03-10T17:12:01.672451+0000 mgr.gibba032.whopeo (mgr.24625) 10497 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:01 gibba011 bash[30791]: cluster 2021-03-10T17:12:01.217512+0000 mgr.gibba032.whopeo (mgr.24625) 10496 : cluster [DBG] pgmap v5983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:02.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:01 gibba011 bash[30791]: cephadm 2021-03-10T17:12:01.672451+0000 mgr.gibba032.whopeo (mgr.24625) 10497 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:02.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:01 gibba011 bash[31774]: cluster 2021-03-10T17:12:01.217512+0000 mgr.gibba032.whopeo (mgr.24625) 10496 : cluster [DBG] pgmap v5983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:02.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:01 gibba011 bash[31774]: cephadm 2021-03-10T17:12:01.672451+0000 mgr.gibba032.whopeo (mgr.24625) 10497 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:04 gibba011 bash[30791]: cluster 2021-03-10T17:12:03.219251+0000 mgr.gibba032.whopeo (mgr.24625) 10498 : cluster [DBG] pgmap v5984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:04.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:04 gibba011 bash[31774]: cluster 2021-03-10T17:12:03.219251+0000 mgr.gibba032.whopeo (mgr.24625) 10498 : cluster [DBG] pgmap v5984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:04 gibba032 bash[24004]: cluster 2021-03-10T17:12:03.219251+0000 mgr.gibba032.whopeo (mgr.24625) 10498 : cluster [DBG] pgmap v5984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:06.150 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:12:06.378 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:12:06.379 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 50s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:12:06.379 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:12:06.379 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:12:06.379 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:12:06.380 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:12:06.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:12:06.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:12:06.380 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:12:06.381 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 50s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:12:06.381 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:12:06.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:12:06.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:12:06.381 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:12:06.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:12:06.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:12:06.382 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:12:06.383 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:12:06.383 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:12:06.383 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:12:06.383 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 50s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:12:06.578 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:06 gibba011 bash[30791]: cluster 2021-03-10T17:12:05.220988+0000 mgr.gibba032.whopeo (mgr.24625) 10499 : cluster [DBG] pgmap v5985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:06.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:06 gibba011 bash[31774]: cluster 2021-03-10T17:12:05.220988+0000 mgr.gibba032.whopeo (mgr.24625) 10499 : cluster [DBG] pgmap v5985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:06.628 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:12:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:12:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:12:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:12:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:12:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:12:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:12:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:12:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:12:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:12:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:12:06.631 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:12:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:06 gibba032 bash[24004]: cluster 2021-03-10T17:12:05.220988+0000 mgr.gibba032.whopeo (mgr.24625) 10499 : cluster [DBG] pgmap v5985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:07.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[30791]: audit 2021-03-10T17:12:06.139606+0000 mgr.gibba032.whopeo (mgr.24625) 10500 : audit [DBG] from='client.51093 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[30791]: audit 2021-03-10T17:12:06.364874+0000 mgr.gibba032.whopeo (mgr.24625) 10501 : audit [DBG] from='client.61081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[30791]: audit 2021-03-10T17:12:06.628552+0000 mon.a (mon.0) 3512 : audit [DBG] from='client.? 172.21.2.111:0/1976226849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[31774]: audit 2021-03-10T17:12:06.139606+0000 mgr.gibba032.whopeo (mgr.24625) 10500 : audit [DBG] from='client.51093 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[31774]: audit 2021-03-10T17:12:06.364874+0000 mgr.gibba032.whopeo (mgr.24625) 10501 : audit [DBG] from='client.61081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:07 gibba011 bash[31774]: audit 2021-03-10T17:12:06.628552+0000 mon.a (mon.0) 3512 : audit [DBG] from='client.? 172.21.2.111:0/1976226849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:07 gibba032 bash[24004]: audit 2021-03-10T17:12:06.139606+0000 mgr.gibba032.whopeo (mgr.24625) 10500 : audit [DBG] from='client.51093 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:07 gibba032 bash[24004]: audit 2021-03-10T17:12:06.364874+0000 mgr.gibba032.whopeo (mgr.24625) 10501 : audit [DBG] from='client.61081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:07 gibba032 bash[24004]: audit 2021-03-10T17:12:06.628552+0000 mon.a (mon.0) 3512 : audit [DBG] from='client.? 172.21.2.111:0/1976226849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:08.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:08 gibba011 bash[30791]: cluster 2021-03-10T17:12:07.222548+0000 mgr.gibba032.whopeo (mgr.24625) 10502 : cluster [DBG] pgmap v5986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:08.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:08 gibba011 bash[31774]: cluster 2021-03-10T17:12:07.222548+0000 mgr.gibba032.whopeo (mgr.24625) 10502 : cluster [DBG] pgmap v5986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:08 gibba032 bash[24004]: cluster 2021-03-10T17:12:07.222548+0000 mgr.gibba032.whopeo (mgr.24625) 10502 : cluster [DBG] pgmap v5986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:10 gibba011 bash[30791]: cluster 2021-03-10T17:12:09.224016+0000 mgr.gibba032.whopeo (mgr.24625) 10503 : cluster [DBG] pgmap v5987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:10.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:10 gibba011 bash[31774]: cluster 2021-03-10T17:12:09.224016+0000 mgr.gibba032.whopeo (mgr.24625) 10503 : cluster [DBG] pgmap v5987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:10 gibba032 bash[24004]: cluster 2021-03-10T17:12:09.224016+0000 mgr.gibba032.whopeo (mgr.24625) 10503 : cluster [DBG] pgmap v5987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:12.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:11 gibba011 bash[30791]: cluster 2021-03-10T17:12:11.225852+0000 mgr.gibba032.whopeo (mgr.24625) 10504 : cluster [DBG] pgmap v5988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:12.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:11 gibba011 bash[31774]: cluster 2021-03-10T17:12:11.225852+0000 mgr.gibba032.whopeo (mgr.24625) 10504 : cluster [DBG] pgmap v5988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:11 gibba032 bash[24004]: cluster 2021-03-10T17:12:11.225852+0000 mgr.gibba032.whopeo (mgr.24625) 10504 : cluster [DBG] pgmap v5988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:14 gibba011 bash[30791]: cluster 2021-03-10T17:12:13.227649+0000 mgr.gibba032.whopeo (mgr.24625) 10505 : cluster [DBG] pgmap v5989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:14 gibba011 bash[31774]: cluster 2021-03-10T17:12:13.227649+0000 mgr.gibba032.whopeo (mgr.24625) 10505 : cluster [DBG] pgmap v5989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:14 gibba032 bash[24004]: cluster 2021-03-10T17:12:13.227649+0000 mgr.gibba032.whopeo (mgr.24625) 10505 : cluster [DBG] pgmap v5989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:16.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:16 gibba011 bash[30791]: cluster 2021-03-10T17:12:15.229189+0000 mgr.gibba032.whopeo (mgr.24625) 10506 : cluster [DBG] pgmap v5990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:16.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:16 gibba011 bash[31774]: cluster 2021-03-10T17:12:15.229189+0000 mgr.gibba032.whopeo (mgr.24625) 10506 : cluster [DBG] pgmap v5990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:16 gibba032 bash[24004]: cluster 2021-03-10T17:12:15.229189+0000 mgr.gibba032.whopeo (mgr.24625) 10506 : cluster [DBG] pgmap v5990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:17.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:17 gibba011 bash[30791]: cephadm 2021-03-10T17:12:16.689355+0000 mgr.gibba032.whopeo (mgr.24625) 10507 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:12:17.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:17 gibba011 bash[30791]: audit 2021-03-10T17:12:16.693173+0000 mon.a (mon.0) 3513 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:12:17.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:17 gibba011 bash[31774]: cephadm 2021-03-10T17:12:16.689355+0000 mgr.gibba032.whopeo (mgr.24625) 10507 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:12:17.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:17 gibba011 bash[31774]: audit 2021-03-10T17:12:16.693173+0000 mon.a (mon.0) 3513 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:12:17.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:17 gibba032 bash[24004]: cephadm 2021-03-10T17:12:16.689355+0000 mgr.gibba032.whopeo (mgr.24625) 10507 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:12:17.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:17 gibba032 bash[24004]: audit 2021-03-10T17:12:16.693173+0000 mon.a (mon.0) 3513 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:12:18.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:18 gibba011 bash[30791]: cluster 2021-03-10T17:12:17.230847+0000 mgr.gibba032.whopeo (mgr.24625) 10508 : cluster [DBG] pgmap v5991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:18 gibba011 bash[31774]: cluster 2021-03-10T17:12:17.230847+0000 mgr.gibba032.whopeo (mgr.24625) 10508 : cluster [DBG] pgmap v5991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:18 gibba032 bash[24004]: cluster 2021-03-10T17:12:17.230847+0000 mgr.gibba032.whopeo (mgr.24625) 10508 : cluster [DBG] pgmap v5991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:20.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:20 gibba011 bash[30791]: cluster 2021-03-10T17:12:19.232134+0000 mgr.gibba032.whopeo (mgr.24625) 10509 : cluster [DBG] pgmap v5992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:20.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:20 gibba011 bash[31774]: cluster 2021-03-10T17:12:19.232134+0000 mgr.gibba032.whopeo (mgr.24625) 10509 : cluster [DBG] pgmap v5992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:20.670 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:20 gibba032 bash[24004]: cluster 2021-03-10T17:12:19.232134+0000 mgr.gibba032.whopeo (mgr.24625) 10509 : cluster [DBG] pgmap v5992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.453826+0000 mgr.gibba032.whopeo (mgr.24625) 10510 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:12:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.454939+0000 mon.a (mon.0) 3514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.655864+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.656736+0000 mgr.gibba032.whopeo (mgr.24625) 10511 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:12:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.658989+0000 mgr.gibba032.whopeo (mgr.24625) 10512 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:12:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.659563+0000 mon.a (mon.0) 3516 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:12:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.660462+0000 mgr.gibba032.whopeo (mgr.24625) 10513 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:12:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.660499+0000 mgr.gibba032.whopeo (mgr.24625) 10514 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:12:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.661455+0000 mon.a (mon.0) 3517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.662318+0000 mgr.gibba032.whopeo (mgr.24625) 10515 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:12:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.662371+0000 mgr.gibba032.whopeo (mgr.24625) 10516 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:12:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.663025+0000 mon.a (mon.0) 3518 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.663938+0000 mgr.gibba032.whopeo (mgr.24625) 10517 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:12:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.664823+0000 mon.a (mon.0) 3519 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.665287+0000 mgr.gibba032.whopeo (mgr.24625) 10518 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:12:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.665324+0000 mgr.gibba032.whopeo (mgr.24625) 10519 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:12:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.666690+0000 mgr.gibba032.whopeo (mgr.24625) 10520 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:12:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.667043+0000 mgr.gibba032.whopeo (mgr.24625) 10521 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:12:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.667599+0000 mon.a (mon.0) 3520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.668122+0000 mgr.gibba032.whopeo (mgr.24625) 10522 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:12:21.771 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.668180+0000 mgr.gibba032.whopeo (mgr.24625) 10523 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:12:21.771 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: cephadm 2021-03-10T17:12:20.668309+0000 mgr.gibba032.whopeo (mgr.24625) 10524 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:21.772 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:21 gibba032 bash[24004]: audit 2021-03-10T17:12:20.669252+0000 mon.a (mon.0) 3521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.453826+0000 mgr.gibba032.whopeo (mgr.24625) 10510 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:12:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.454939+0000 mon.a (mon.0) 3514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.655864+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.656736+0000 mgr.gibba032.whopeo (mgr.24625) 10511 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:12:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.658989+0000 mgr.gibba032.whopeo (mgr.24625) 10512 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:12:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.659563+0000 mon.a (mon.0) 3516 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:12:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.660462+0000 mgr.gibba032.whopeo (mgr.24625) 10513 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:12:21.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.660499+0000 mgr.gibba032.whopeo (mgr.24625) 10514 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:12:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.661455+0000 mon.a (mon.0) 3517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.662318+0000 mgr.gibba032.whopeo (mgr.24625) 10515 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:12:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.662371+0000 mgr.gibba032.whopeo (mgr.24625) 10516 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:12:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.663025+0000 mon.a (mon.0) 3518 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.663938+0000 mgr.gibba032.whopeo (mgr.24625) 10517 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:12:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.664823+0000 mon.a (mon.0) 3519 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.665287+0000 mgr.gibba032.whopeo (mgr.24625) 10518 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:12:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.665324+0000 mgr.gibba032.whopeo (mgr.24625) 10519 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:12:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.666690+0000 mgr.gibba032.whopeo (mgr.24625) 10520 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:12:21.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.667043+0000 mgr.gibba032.whopeo (mgr.24625) 10521 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:12:21.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.667599+0000 mon.a (mon.0) 3520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.668122+0000 mgr.gibba032.whopeo (mgr.24625) 10522 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:12:21.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.668180+0000 mgr.gibba032.whopeo (mgr.24625) 10523 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:12:21.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: cephadm 2021-03-10T17:12:20.668309+0000 mgr.gibba032.whopeo (mgr.24625) 10524 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:21.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[30791]: audit 2021-03-10T17:12:20.669252+0000 mon.a (mon.0) 3521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.453826+0000 mgr.gibba032.whopeo (mgr.24625) 10510 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:12:21.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.454939+0000 mon.a (mon.0) 3514 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.655864+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:12:21.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.656736+0000 mgr.gibba032.whopeo (mgr.24625) 10511 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:12:21.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.658989+0000 mgr.gibba032.whopeo (mgr.24625) 10512 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:12:21.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.659563+0000 mon.a (mon.0) 3516 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:12:21.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.660462+0000 mgr.gibba032.whopeo (mgr.24625) 10513 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:12:21.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.660499+0000 mgr.gibba032.whopeo (mgr.24625) 10514 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:12:21.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.661455+0000 mon.a (mon.0) 3517 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.662318+0000 mgr.gibba032.whopeo (mgr.24625) 10515 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:12:21.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.662371+0000 mgr.gibba032.whopeo (mgr.24625) 10516 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:12:21.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.663025+0000 mon.a (mon.0) 3518 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.663938+0000 mgr.gibba032.whopeo (mgr.24625) 10517 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:12:21.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.664823+0000 mon.a (mon.0) 3519 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.665287+0000 mgr.gibba032.whopeo (mgr.24625) 10518 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:12:21.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.665324+0000 mgr.gibba032.whopeo (mgr.24625) 10519 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:12:21.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.666690+0000 mgr.gibba032.whopeo (mgr.24625) 10520 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:12:21.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.667043+0000 mgr.gibba032.whopeo (mgr.24625) 10521 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:12:21.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.667599+0000 mon.a (mon.0) 3520 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:21.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.668122+0000 mgr.gibba032.whopeo (mgr.24625) 10522 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:12:21.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.668180+0000 mgr.gibba032.whopeo (mgr.24625) 10523 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:12:21.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: cephadm 2021-03-10T17:12:20.668309+0000 mgr.gibba032.whopeo (mgr.24625) 10524 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:21.845 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:21 gibba011 bash[31774]: audit 2021-03-10T17:12:20.669252+0000 mon.a (mon.0) 3521 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:22.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:22 gibba032 bash[24004]: cluster 2021-03-10T17:12:21.233647+0000 mgr.gibba032.whopeo (mgr.24625) 10525 : cluster [DBG] pgmap v5993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:22.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:22 gibba011 bash[30791]: cluster 2021-03-10T17:12:21.233647+0000 mgr.gibba032.whopeo (mgr.24625) 10525 : cluster [DBG] pgmap v5993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:22.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:22 gibba011 bash[31774]: cluster 2021-03-10T17:12:21.233647+0000 mgr.gibba032.whopeo (mgr.24625) 10525 : cluster [DBG] pgmap v5993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:24 gibba032 bash[24004]: cluster 2021-03-10T17:12:23.235531+0000 mgr.gibba032.whopeo (mgr.24625) 10526 : cluster [DBG] pgmap v5994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:24.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:24 gibba011 bash[30791]: cluster 2021-03-10T17:12:23.235531+0000 mgr.gibba032.whopeo (mgr.24625) 10526 : cluster [DBG] pgmap v5994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:24 gibba011 bash[31774]: cluster 2021-03-10T17:12:23.235531+0000 mgr.gibba032.whopeo (mgr.24625) 10526 : cluster [DBG] pgmap v5994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:26 gibba032 bash[24004]: cluster 2021-03-10T17:12:25.237261+0000 mgr.gibba032.whopeo (mgr.24625) 10527 : cluster [DBG] pgmap v5995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:26 gibba011 bash[30791]: cluster 2021-03-10T17:12:25.237261+0000 mgr.gibba032.whopeo (mgr.24625) 10527 : cluster [DBG] pgmap v5995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:26 gibba011 bash[31774]: cluster 2021-03-10T17:12:25.237261+0000 mgr.gibba032.whopeo (mgr.24625) 10527 : cluster [DBG] pgmap v5995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:28 gibba032 bash[24004]: cluster 2021-03-10T17:12:27.238814+0000 mgr.gibba032.whopeo (mgr.24625) 10528 : cluster [DBG] pgmap v5996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:28 gibba011 bash[30791]: cluster 2021-03-10T17:12:27.238814+0000 mgr.gibba032.whopeo (mgr.24625) 10528 : cluster [DBG] pgmap v5996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:28 gibba011 bash[31774]: cluster 2021-03-10T17:12:27.238814+0000 mgr.gibba032.whopeo (mgr.24625) 10528 : cluster [DBG] pgmap v5996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:30 gibba032 bash[24004]: cluster 2021-03-10T17:12:29.240328+0000 mgr.gibba032.whopeo (mgr.24625) 10529 : cluster [DBG] pgmap v5997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:30.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:30 gibba011 bash[30791]: cluster 2021-03-10T17:12:29.240328+0000 mgr.gibba032.whopeo (mgr.24625) 10529 : cluster [DBG] pgmap v5997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:30 gibba011 bash[31774]: cluster 2021-03-10T17:12:29.240328+0000 mgr.gibba032.whopeo (mgr.24625) 10529 : cluster [DBG] pgmap v5997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:31 gibba032 bash[24004]: cluster 2021-03-10T17:12:31.242121+0000 mgr.gibba032.whopeo (mgr.24625) 10530 : cluster [DBG] pgmap v5998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:32.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:31 gibba011 bash[30791]: cluster 2021-03-10T17:12:31.242121+0000 mgr.gibba032.whopeo (mgr.24625) 10530 : cluster [DBG] pgmap v5998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:31 gibba011 bash[31774]: cluster 2021-03-10T17:12:31.242121+0000 mgr.gibba032.whopeo (mgr.24625) 10530 : cluster [DBG] pgmap v5998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:34 gibba011 bash[30791]: cluster 2021-03-10T17:12:33.243881+0000 mgr.gibba032.whopeo (mgr.24625) 10531 : cluster [DBG] pgmap v5999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:34 gibba011 bash[31774]: cluster 2021-03-10T17:12:33.243881+0000 mgr.gibba032.whopeo (mgr.24625) 10531 : cluster [DBG] pgmap v5999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:34 gibba032 bash[24004]: cluster 2021-03-10T17:12:33.243881+0000 mgr.gibba032.whopeo (mgr.24625) 10531 : cluster [DBG] pgmap v5999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:36 gibba011 bash[30791]: cluster 2021-03-10T17:12:35.245476+0000 mgr.gibba032.whopeo (mgr.24625) 10532 : cluster [DBG] pgmap v6000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:36.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:36 gibba011 bash[30791]: cephadm 2021-03-10T17:12:35.681512+0000 mgr.gibba032.whopeo (mgr.24625) 10533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:36 gibba011 bash[31774]: cluster 2021-03-10T17:12:35.245476+0000 mgr.gibba032.whopeo (mgr.24625) 10532 : cluster [DBG] pgmap v6000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:36 gibba011 bash[31774]: cephadm 2021-03-10T17:12:35.681512+0000 mgr.gibba032.whopeo (mgr.24625) 10533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:36 gibba032 bash[24004]: cluster 2021-03-10T17:12:35.245476+0000 mgr.gibba032.whopeo (mgr.24625) 10532 : cluster [DBG] pgmap v6000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:36 gibba032 bash[24004]: cephadm 2021-03-10T17:12:35.681512+0000 mgr.gibba032.whopeo (mgr.24625) 10533 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:36.871 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:12:37.105 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:12:37.105 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 80s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:12:37.105 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 16s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:12:37.105 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:12:37.106 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:12:37.106 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:12:37.106 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:12:37.106 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:12:37.107 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:12:37.107 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 80s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:12:37.107 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 16s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:12:37.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:12:37.107 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:12:37.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:12:37.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:12:37.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:12:37.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:12:37.108 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:12:37.109 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 16s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:12:37.109 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 16s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:12:37.109 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 80s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:12:37.356 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:12:37.356 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:12:37.356 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:12:37.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:37.357 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:12:37.357 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:12:37.357 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:37.357 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:12:37.358 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:12:37.358 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:12:37.358 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:12:37.358 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:12:37.359 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:12:37.359 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:12:37.359 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:12:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[30791]: audit 2021-03-10T17:12:36.860390+0000 mgr.gibba032.whopeo (mgr.24625) 10534 : audit [DBG] from='client.51111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[30791]: audit 2021-03-10T17:12:37.090445+0000 mgr.gibba032.whopeo (mgr.24625) 10535 : audit [DBG] from='client.51117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[30791]: cluster 2021-03-10T17:12:37.247139+0000 mgr.gibba032.whopeo (mgr.24625) 10536 : cluster [DBG] pgmap v6001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[30791]: audit 2021-03-10T17:12:37.356151+0000 mon.a (mon.0) 3522 : audit [DBG] from='client.? 172.21.2.111:0/2844994247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:38.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[31774]: audit 2021-03-10T17:12:36.860390+0000 mgr.gibba032.whopeo (mgr.24625) 10534 : audit [DBG] from='client.51111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[31774]: audit 2021-03-10T17:12:37.090445+0000 mgr.gibba032.whopeo (mgr.24625) 10535 : audit [DBG] from='client.51117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[31774]: cluster 2021-03-10T17:12:37.247139+0000 mgr.gibba032.whopeo (mgr.24625) 10536 : cluster [DBG] pgmap v6001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:38 gibba011 bash[31774]: audit 2021-03-10T17:12:37.356151+0000 mon.a (mon.0) 3522 : audit [DBG] from='client.? 172.21.2.111:0/2844994247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:38 gibba032 bash[24004]: audit 2021-03-10T17:12:36.860390+0000 mgr.gibba032.whopeo (mgr.24625) 10534 : audit [DBG] from='client.51111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:38 gibba032 bash[24004]: audit 2021-03-10T17:12:37.090445+0000 mgr.gibba032.whopeo (mgr.24625) 10535 : audit [DBG] from='client.51117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:12:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:38 gibba032 bash[24004]: cluster 2021-03-10T17:12:37.247139+0000 mgr.gibba032.whopeo (mgr.24625) 10536 : cluster [DBG] pgmap v6001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:38 gibba032 bash[24004]: audit 2021-03-10T17:12:37.356151+0000 mon.a (mon.0) 3522 : audit [DBG] from='client.? 172.21.2.111:0/2844994247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:12:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:40 gibba032 bash[24004]: cluster 2021-03-10T17:12:39.248690+0000 mgr.gibba032.whopeo (mgr.24625) 10537 : cluster [DBG] pgmap v6002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:40.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:40 gibba011 bash[30791]: cluster 2021-03-10T17:12:39.248690+0000 mgr.gibba032.whopeo (mgr.24625) 10537 : cluster [DBG] pgmap v6002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:40 gibba011 bash[31774]: cluster 2021-03-10T17:12:39.248690+0000 mgr.gibba032.whopeo (mgr.24625) 10537 : cluster [DBG] pgmap v6002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:41 gibba032 bash[24004]: cluster 2021-03-10T17:12:41.250177+0000 mgr.gibba032.whopeo (mgr.24625) 10538 : cluster [DBG] pgmap v6003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:41 gibba011 bash[30791]: cluster 2021-03-10T17:12:41.250177+0000 mgr.gibba032.whopeo (mgr.24625) 10538 : cluster [DBG] pgmap v6003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:41 gibba011 bash[31774]: cluster 2021-03-10T17:12:41.250177+0000 mgr.gibba032.whopeo (mgr.24625) 10538 : cluster [DBG] pgmap v6003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:44 gibba011 bash[30791]: cluster 2021-03-10T17:12:43.251886+0000 mgr.gibba032.whopeo (mgr.24625) 10539 : cluster [DBG] pgmap v6004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:44.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:44 gibba011 bash[31774]: cluster 2021-03-10T17:12:43.251886+0000 mgr.gibba032.whopeo (mgr.24625) 10539 : cluster [DBG] pgmap v6004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:44 gibba032 bash[24004]: cluster 2021-03-10T17:12:43.251886+0000 mgr.gibba032.whopeo (mgr.24625) 10539 : cluster [DBG] pgmap v6004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:46.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:46 gibba011 bash[30791]: cluster 2021-03-10T17:12:45.253570+0000 mgr.gibba032.whopeo (mgr.24625) 10540 : cluster [DBG] pgmap v6005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:46.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:46 gibba011 bash[31774]: cluster 2021-03-10T17:12:45.253570+0000 mgr.gibba032.whopeo (mgr.24625) 10540 : cluster [DBG] pgmap v6005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:46 gibba032 bash[24004]: cluster 2021-03-10T17:12:45.253570+0000 mgr.gibba032.whopeo (mgr.24625) 10540 : cluster [DBG] pgmap v6005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:48 gibba032 bash[24004]: cluster 2021-03-10T17:12:47.255298+0000 mgr.gibba032.whopeo (mgr.24625) 10541 : cluster [DBG] pgmap v6006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:48 gibba011 bash[30791]: cluster 2021-03-10T17:12:47.255298+0000 mgr.gibba032.whopeo (mgr.24625) 10541 : cluster [DBG] pgmap v6006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:48 gibba011 bash[31774]: cluster 2021-03-10T17:12:47.255298+0000 mgr.gibba032.whopeo (mgr.24625) 10541 : cluster [DBG] pgmap v6006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:50.699 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:50 gibba032 bash[24004]: cluster 2021-03-10T17:12:49.256701+0000 mgr.gibba032.whopeo (mgr.24625) 10542 : cluster [DBG] pgmap v6007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:50 gibba011 bash[30791]: cluster 2021-03-10T17:12:49.256701+0000 mgr.gibba032.whopeo (mgr.24625) 10542 : cluster [DBG] pgmap v6007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:50.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:50 gibba011 bash[31774]: cluster 2021-03-10T17:12:49.256701+0000 mgr.gibba032.whopeo (mgr.24625) 10542 : cluster [DBG] pgmap v6007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:51.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:51 gibba032 bash[24004]: cephadm 2021-03-10T17:12:50.696876+0000 mgr.gibba032.whopeo (mgr.24625) 10543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:51.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:51 gibba011 bash[30791]: cephadm 2021-03-10T17:12:50.696876+0000 mgr.gibba032.whopeo (mgr.24625) 10543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:51.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:51 gibba011 bash[31774]: cephadm 2021-03-10T17:12:50.696876+0000 mgr.gibba032.whopeo (mgr.24625) 10543 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:12:52.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:52 gibba032 bash[24004]: cluster 2021-03-10T17:12:51.258330+0000 mgr.gibba032.whopeo (mgr.24625) 10544 : cluster [DBG] pgmap v6008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:52.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:52 gibba011 bash[30791]: cluster 2021-03-10T17:12:51.258330+0000 mgr.gibba032.whopeo (mgr.24625) 10544 : cluster [DBG] pgmap v6008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:52 gibba011 bash[31774]: cluster 2021-03-10T17:12:51.258330+0000 mgr.gibba032.whopeo (mgr.24625) 10544 : cluster [DBG] pgmap v6008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:54 gibba032 bash[24004]: cluster 2021-03-10T17:12:53.260001+0000 mgr.gibba032.whopeo (mgr.24625) 10545 : cluster [DBG] pgmap v6009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:54 gibba011 bash[30791]: cluster 2021-03-10T17:12:53.260001+0000 mgr.gibba032.whopeo (mgr.24625) 10545 : cluster [DBG] pgmap v6009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:54 gibba011 bash[31774]: cluster 2021-03-10T17:12:53.260001+0000 mgr.gibba032.whopeo (mgr.24625) 10545 : cluster [DBG] pgmap v6009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:56 gibba032 bash[24004]: cluster 2021-03-10T17:12:55.261650+0000 mgr.gibba032.whopeo (mgr.24625) 10546 : cluster [DBG] pgmap v6010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:56 gibba011 bash[30791]: cluster 2021-03-10T17:12:55.261650+0000 mgr.gibba032.whopeo (mgr.24625) 10546 : cluster [DBG] pgmap v6010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:56.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:56 gibba011 bash[31774]: cluster 2021-03-10T17:12:55.261650+0000 mgr.gibba032.whopeo (mgr.24625) 10546 : cluster [DBG] pgmap v6010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:57 gibba032 bash[24004]: audit 2021-03-10T17:12:57.368198+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:12:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:57 gibba032 bash[24004]: audit 2021-03-10T17:12:57.377866+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:12:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:57 gibba011 bash[30791]: audit 2021-03-10T17:12:57.368198+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:12:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:57 gibba011 bash[30791]: audit 2021-03-10T17:12:57.377866+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:12:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:57 gibba011 bash[31774]: audit 2021-03-10T17:12:57.368198+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:12:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:57 gibba011 bash[31774]: audit 2021-03-10T17:12:57.377866+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:12:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:58 gibba032 bash[24004]: cluster 2021-03-10T17:12:57.263435+0000 mgr.gibba032.whopeo (mgr.24625) 10547 : cluster [DBG] pgmap v6011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:58 gibba032 bash[24004]: audit 2021-03-10T17:12:57.549639+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:12:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:58 gibba032 bash[24004]: audit 2021-03-10T17:12:57.550404+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:58 gibba032 bash[24004]: audit 2021-03-10T17:12:57.551104+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:12:58 gibba032 bash[24004]: audit 2021-03-10T17:12:57.551865+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:12:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[30791]: cluster 2021-03-10T17:12:57.263435+0000 mgr.gibba032.whopeo (mgr.24625) 10547 : cluster [DBG] pgmap v6011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[30791]: audit 2021-03-10T17:12:57.549639+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:12:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[30791]: audit 2021-03-10T17:12:57.550404+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[30791]: audit 2021-03-10T17:12:57.551104+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[30791]: audit 2021-03-10T17:12:57.551865+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:12:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[31774]: cluster 2021-03-10T17:12:57.263435+0000 mgr.gibba032.whopeo (mgr.24625) 10547 : cluster [DBG] pgmap v6011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:12:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[31774]: audit 2021-03-10T17:12:57.549639+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:12:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[31774]: audit 2021-03-10T17:12:57.550404+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[31774]: audit 2021-03-10T17:12:57.551104+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:12:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:12:58 gibba011 bash[31774]: audit 2021-03-10T17:12:57.551865+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:13:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:00 gibba032 bash[24004]: cluster 2021-03-10T17:12:59.264867+0000 mgr.gibba032.whopeo (mgr.24625) 10548 : cluster [DBG] pgmap v6012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:00 gibba011 bash[30791]: cluster 2021-03-10T17:12:59.264867+0000 mgr.gibba032.whopeo (mgr.24625) 10548 : cluster [DBG] pgmap v6012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:00 gibba011 bash[31774]: cluster 2021-03-10T17:12:59.264867+0000 mgr.gibba032.whopeo (mgr.24625) 10548 : cluster [DBG] pgmap v6012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:01 gibba032 bash[24004]: cluster 2021-03-10T17:13:01.266493+0000 mgr.gibba032.whopeo (mgr.24625) 10549 : cluster [DBG] pgmap v6013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:01 gibba011 bash[30791]: cluster 2021-03-10T17:13:01.266493+0000 mgr.gibba032.whopeo (mgr.24625) 10549 : cluster [DBG] pgmap v6013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:02.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:01 gibba011 bash[31774]: cluster 2021-03-10T17:13:01.266493+0000 mgr.gibba032.whopeo (mgr.24625) 10549 : cluster [DBG] pgmap v6013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:04 gibba032 bash[24004]: cluster 2021-03-10T17:13:03.268011+0000 mgr.gibba032.whopeo (mgr.24625) 10550 : cluster [DBG] pgmap v6014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:04 gibba011 bash[30791]: cluster 2021-03-10T17:13:03.268011+0000 mgr.gibba032.whopeo (mgr.24625) 10550 : cluster [DBG] pgmap v6014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:04 gibba011 bash[31774]: cluster 2021-03-10T17:13:03.268011+0000 mgr.gibba032.whopeo (mgr.24625) 10550 : cluster [DBG] pgmap v6014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:06 gibba032 bash[24004]: cluster 2021-03-10T17:13:05.269659+0000 mgr.gibba032.whopeo (mgr.24625) 10551 : cluster [DBG] pgmap v6015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:06 gibba032 bash[24004]: cephadm 2021-03-10T17:13:05.708486+0000 mgr.gibba032.whopeo (mgr.24625) 10552 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:06 gibba011 bash[30791]: cluster 2021-03-10T17:13:05.269659+0000 mgr.gibba032.whopeo (mgr.24625) 10551 : cluster [DBG] pgmap v6015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:06 gibba011 bash[30791]: cephadm 2021-03-10T17:13:05.708486+0000 mgr.gibba032.whopeo (mgr.24625) 10552 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:06 gibba011 bash[31774]: cluster 2021-03-10T17:13:05.269659+0000 mgr.gibba032.whopeo (mgr.24625) 10551 : cluster [DBG] pgmap v6015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:06 gibba011 bash[31774]: cephadm 2021-03-10T17:13:05.708486+0000 mgr.gibba032.whopeo (mgr.24625) 10552 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:07.598 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:13:07.827 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:13:07.827 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 111s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:13:07.827 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 47s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:13:07.827 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:13:07.828 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:13:07.828 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:13:07.828 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:13:07.828 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:13:07.829 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:13:07.829 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 111s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:13:07.829 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 47s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:13:07.829 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:13:07.829 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:13:07.830 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:13:07.830 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:13:07.830 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:13:07.830 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:13:07.831 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:13:07.831 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 47s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:13:07.831 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 47s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:13:07.831 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 111s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:13:08.084 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:13:08.084 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:13:08.084 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:13:08.084 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:08.085 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:13:08.085 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:13:08.085 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:08.085 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:13:08.086 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:13:08.086 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:08.086 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:13:08.086 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:13:08.086 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:13:08.087 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:13:08.087 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:13:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:08 gibba032 bash[24004]: cluster 2021-03-10T17:13:07.271412+0000 mgr.gibba032.whopeo (mgr.24625) 10553 : cluster [DBG] pgmap v6016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:08 gibba032 bash[24004]: audit 2021-03-10T17:13:07.587307+0000 mgr.gibba032.whopeo (mgr.24625) 10554 : audit [DBG] from='client.51129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:08 gibba032 bash[24004]: audit 2021-03-10T17:13:08.084050+0000 mon.a (mon.0) 3529 : audit [DBG] from='client.? 172.21.2.111:0/621794216' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[30791]: cluster 2021-03-10T17:13:07.271412+0000 mgr.gibba032.whopeo (mgr.24625) 10553 : cluster [DBG] pgmap v6016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[30791]: audit 2021-03-10T17:13:07.587307+0000 mgr.gibba032.whopeo (mgr.24625) 10554 : audit [DBG] from='client.51129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[30791]: audit 2021-03-10T17:13:08.084050+0000 mon.a (mon.0) 3529 : audit [DBG] from='client.? 172.21.2.111:0/621794216' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[31774]: cluster 2021-03-10T17:13:07.271412+0000 mgr.gibba032.whopeo (mgr.24625) 10553 : cluster [DBG] pgmap v6016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[31774]: audit 2021-03-10T17:13:07.587307+0000 mgr.gibba032.whopeo (mgr.24625) 10554 : audit [DBG] from='client.51129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:08 gibba011 bash[31774]: audit 2021-03-10T17:13:08.084050+0000 mon.a (mon.0) 3529 : audit [DBG] from='client.? 172.21.2.111:0/621794216' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:09.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:09 gibba032 bash[24004]: audit 2021-03-10T17:13:07.813235+0000 mgr.gibba032.whopeo (mgr.24625) 10555 : audit [DBG] from='client.51135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:09.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:09 gibba011 bash[30791]: audit 2021-03-10T17:13:07.813235+0000 mgr.gibba032.whopeo (mgr.24625) 10555 : audit [DBG] from='client.51135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:09.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:09 gibba011 bash[31774]: audit 2021-03-10T17:13:07.813235+0000 mgr.gibba032.whopeo (mgr.24625) 10555 : audit [DBG] from='client.51135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:10 gibba032 bash[24004]: cluster 2021-03-10T17:13:09.272803+0000 mgr.gibba032.whopeo (mgr.24625) 10556 : cluster [DBG] pgmap v6017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:10 gibba011 bash[30791]: cluster 2021-03-10T17:13:09.272803+0000 mgr.gibba032.whopeo (mgr.24625) 10556 : cluster [DBG] pgmap v6017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:10 gibba011 bash[31774]: cluster 2021-03-10T17:13:09.272803+0000 mgr.gibba032.whopeo (mgr.24625) 10556 : cluster [DBG] pgmap v6017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:11 gibba032 bash[24004]: cluster 2021-03-10T17:13:11.274480+0000 mgr.gibba032.whopeo (mgr.24625) 10557 : cluster [DBG] pgmap v6018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:11 gibba011 bash[30791]: cluster 2021-03-10T17:13:11.274480+0000 mgr.gibba032.whopeo (mgr.24625) 10557 : cluster [DBG] pgmap v6018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:11 gibba011 bash[31774]: cluster 2021-03-10T17:13:11.274480+0000 mgr.gibba032.whopeo (mgr.24625) 10557 : cluster [DBG] pgmap v6018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:14 gibba032 bash[24004]: cluster 2021-03-10T17:13:13.276242+0000 mgr.gibba032.whopeo (mgr.24625) 10558 : cluster [DBG] pgmap v6019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:14 gibba011 bash[30791]: cluster 2021-03-10T17:13:13.276242+0000 mgr.gibba032.whopeo (mgr.24625) 10558 : cluster [DBG] pgmap v6019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:14 gibba011 bash[31774]: cluster 2021-03-10T17:13:13.276242+0000 mgr.gibba032.whopeo (mgr.24625) 10558 : cluster [DBG] pgmap v6019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:16 gibba032 bash[24004]: cluster 2021-03-10T17:13:15.277674+0000 mgr.gibba032.whopeo (mgr.24625) 10559 : cluster [DBG] pgmap v6020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:16 gibba011 bash[30791]: cluster 2021-03-10T17:13:15.277674+0000 mgr.gibba032.whopeo (mgr.24625) 10559 : cluster [DBG] pgmap v6020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:16 gibba011 bash[31774]: cluster 2021-03-10T17:13:15.277674+0000 mgr.gibba032.whopeo (mgr.24625) 10559 : cluster [DBG] pgmap v6020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:18 gibba032 bash[24004]: cluster 2021-03-10T17:13:17.279357+0000 mgr.gibba032.whopeo (mgr.24625) 10560 : cluster [DBG] pgmap v6021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:18 gibba011 bash[30791]: cluster 2021-03-10T17:13:17.279357+0000 mgr.gibba032.whopeo (mgr.24625) 10560 : cluster [DBG] pgmap v6021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:18 gibba011 bash[31774]: cluster 2021-03-10T17:13:17.279357+0000 mgr.gibba032.whopeo (mgr.24625) 10560 : cluster [DBG] pgmap v6021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:20 gibba032 bash[24004]: cluster 2021-03-10T17:13:19.280723+0000 mgr.gibba032.whopeo (mgr.24625) 10561 : cluster [DBG] pgmap v6022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:20.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:20 gibba011 bash[30791]: cluster 2021-03-10T17:13:19.280723+0000 mgr.gibba032.whopeo (mgr.24625) 10561 : cluster [DBG] pgmap v6022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:20 gibba011 bash[31774]: cluster 2021-03-10T17:13:19.280723+0000 mgr.gibba032.whopeo (mgr.24625) 10561 : cluster [DBG] pgmap v6022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.723819+0000 mgr.gibba032.whopeo (mgr.24625) 10562 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:13:21.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.725410+0000 mgr.gibba032.whopeo (mgr.24625) 10563 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:13:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.939098+0000 mon.a (mon.0) 3530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:13:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.942370+0000 mon.a (mon.0) 3531 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:13:21.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.943619+0000 mon.a (mon.0) 3532 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.944891+0000 mon.a (mon.0) 3533 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.945928+0000 mon.a (mon.0) 3534 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.947690+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:21 gibba032 bash[24004]: audit 2021-03-10T17:13:20.948969+0000 mon.a (mon.0) 3536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.723819+0000 mgr.gibba032.whopeo (mgr.24625) 10562 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:13:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.725410+0000 mgr.gibba032.whopeo (mgr.24625) 10563 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:13:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.939098+0000 mon.a (mon.0) 3530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:13:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.942370+0000 mon.a (mon.0) 3531 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:13:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.943619+0000 mon.a (mon.0) 3532 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.944891+0000 mon.a (mon.0) 3533 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.945928+0000 mon.a (mon.0) 3534 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.947690+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[30791]: audit 2021-03-10T17:13:20.948969+0000 mon.a (mon.0) 3536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.723819+0000 mgr.gibba032.whopeo (mgr.24625) 10562 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:13:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.725410+0000 mgr.gibba032.whopeo (mgr.24625) 10563 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:13:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.939098+0000 mon.a (mon.0) 3530 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:13:21.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.942370+0000 mon.a (mon.0) 3531 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:13:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.943619+0000 mon.a (mon.0) 3532 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.944891+0000 mon.a (mon.0) 3533 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.945928+0000 mon.a (mon.0) 3534 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.947690+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:21.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:21 gibba011 bash[31774]: audit 2021-03-10T17:13:20.948969+0000 mon.a (mon.0) 3536 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.939624+0000 mgr.gibba032.whopeo (mgr.24625) 10564 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:13:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.941434+0000 mgr.gibba032.whopeo (mgr.24625) 10565 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:13:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.942570+0000 mgr.gibba032.whopeo (mgr.24625) 10566 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:13:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.942611+0000 mgr.gibba032.whopeo (mgr.24625) 10567 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:13:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.943808+0000 mgr.gibba032.whopeo (mgr.24625) 10568 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:13:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.943847+0000 mgr.gibba032.whopeo (mgr.24625) 10569 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:13:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.944685+0000 mgr.gibba032.whopeo (mgr.24625) 10570 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:13:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.945433+0000 mgr.gibba032.whopeo (mgr.24625) 10571 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:13:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.945470+0000 mgr.gibba032.whopeo (mgr.24625) 10572 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:13:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.946446+0000 mgr.gibba032.whopeo (mgr.24625) 10573 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:13:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.946768+0000 mgr.gibba032.whopeo (mgr.24625) 10574 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:13:22.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.947621+0000 mgr.gibba032.whopeo (mgr.24625) 10575 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:13:22.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.947661+0000 mgr.gibba032.whopeo (mgr.24625) 10576 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:13:22.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cephadm 2021-03-10T17:13:20.947767+0000 mgr.gibba032.whopeo (mgr.24625) 10577 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:22.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:22 gibba032 bash[24004]: cluster 2021-03-10T17:13:21.282255+0000 mgr.gibba032.whopeo (mgr.24625) 10578 : cluster [DBG] pgmap v6023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:22.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.939624+0000 mgr.gibba032.whopeo (mgr.24625) 10564 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:13:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.941434+0000 mgr.gibba032.whopeo (mgr.24625) 10565 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:13:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.942570+0000 mgr.gibba032.whopeo (mgr.24625) 10566 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:13:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.942611+0000 mgr.gibba032.whopeo (mgr.24625) 10567 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:13:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.943808+0000 mgr.gibba032.whopeo (mgr.24625) 10568 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:13:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.943847+0000 mgr.gibba032.whopeo (mgr.24625) 10569 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:13:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.944685+0000 mgr.gibba032.whopeo (mgr.24625) 10570 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:13:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.945433+0000 mgr.gibba032.whopeo (mgr.24625) 10571 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:13:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.945470+0000 mgr.gibba032.whopeo (mgr.24625) 10572 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:13:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.946446+0000 mgr.gibba032.whopeo (mgr.24625) 10573 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:13:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.946768+0000 mgr.gibba032.whopeo (mgr.24625) 10574 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:13:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.947621+0000 mgr.gibba032.whopeo (mgr.24625) 10575 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:13:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.947661+0000 mgr.gibba032.whopeo (mgr.24625) 10576 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:13:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cephadm 2021-03-10T17:13:20.947767+0000 mgr.gibba032.whopeo (mgr.24625) 10577 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[30791]: cluster 2021-03-10T17:13:21.282255+0000 mgr.gibba032.whopeo (mgr.24625) 10578 : cluster [DBG] pgmap v6023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.939624+0000 mgr.gibba032.whopeo (mgr.24625) 10564 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:13:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.941434+0000 mgr.gibba032.whopeo (mgr.24625) 10565 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:13:22.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.942570+0000 mgr.gibba032.whopeo (mgr.24625) 10566 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:13:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.942611+0000 mgr.gibba032.whopeo (mgr.24625) 10567 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:13:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.943808+0000 mgr.gibba032.whopeo (mgr.24625) 10568 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:13:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.943847+0000 mgr.gibba032.whopeo (mgr.24625) 10569 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:13:22.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.944685+0000 mgr.gibba032.whopeo (mgr.24625) 10570 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:13:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.945433+0000 mgr.gibba032.whopeo (mgr.24625) 10571 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:13:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.945470+0000 mgr.gibba032.whopeo (mgr.24625) 10572 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:13:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.946446+0000 mgr.gibba032.whopeo (mgr.24625) 10573 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:13:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.946768+0000 mgr.gibba032.whopeo (mgr.24625) 10574 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:13:22.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.947621+0000 mgr.gibba032.whopeo (mgr.24625) 10575 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:13:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.947661+0000 mgr.gibba032.whopeo (mgr.24625) 10576 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:13:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cephadm 2021-03-10T17:13:20.947767+0000 mgr.gibba032.whopeo (mgr.24625) 10577 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:22.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:22 gibba011 bash[31774]: cluster 2021-03-10T17:13:21.282255+0000 mgr.gibba032.whopeo (mgr.24625) 10578 : cluster [DBG] pgmap v6023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:24 gibba032 bash[24004]: cluster 2021-03-10T17:13:23.284081+0000 mgr.gibba032.whopeo (mgr.24625) 10579 : cluster [DBG] pgmap v6024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:24 gibba011 bash[30791]: cluster 2021-03-10T17:13:23.284081+0000 mgr.gibba032.whopeo (mgr.24625) 10579 : cluster [DBG] pgmap v6024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:24 gibba011 bash[31774]: cluster 2021-03-10T17:13:23.284081+0000 mgr.gibba032.whopeo (mgr.24625) 10579 : cluster [DBG] pgmap v6024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:26 gibba032 bash[24004]: cluster 2021-03-10T17:13:25.285762+0000 mgr.gibba032.whopeo (mgr.24625) 10580 : cluster [DBG] pgmap v6025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:26.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:26 gibba011 bash[30791]: cluster 2021-03-10T17:13:25.285762+0000 mgr.gibba032.whopeo (mgr.24625) 10580 : cluster [DBG] pgmap v6025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:26 gibba011 bash[31774]: cluster 2021-03-10T17:13:25.285762+0000 mgr.gibba032.whopeo (mgr.24625) 10580 : cluster [DBG] pgmap v6025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:28 gibba032 bash[24004]: cluster 2021-03-10T17:13:27.287311+0000 mgr.gibba032.whopeo (mgr.24625) 10581 : cluster [DBG] pgmap v6026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:28.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:28 gibba011 bash[30791]: cluster 2021-03-10T17:13:27.287311+0000 mgr.gibba032.whopeo (mgr.24625) 10581 : cluster [DBG] pgmap v6026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:28.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:28 gibba011 bash[31774]: cluster 2021-03-10T17:13:27.287311+0000 mgr.gibba032.whopeo (mgr.24625) 10581 : cluster [DBG] pgmap v6026: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:30 gibba032 bash[24004]: cluster 2021-03-10T17:13:29.288627+0000 mgr.gibba032.whopeo (mgr.24625) 10582 : cluster [DBG] pgmap v6027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:30 gibba011 bash[30791]: cluster 2021-03-10T17:13:29.288627+0000 mgr.gibba032.whopeo (mgr.24625) 10582 : cluster [DBG] pgmap v6027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:30 gibba011 bash[31774]: cluster 2021-03-10T17:13:29.288627+0000 mgr.gibba032.whopeo (mgr.24625) 10582 : cluster [DBG] pgmap v6027: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:31 gibba032 bash[24004]: cluster 2021-03-10T17:13:31.290266+0000 mgr.gibba032.whopeo (mgr.24625) 10583 : cluster [DBG] pgmap v6028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:31 gibba011 bash[30791]: cluster 2021-03-10T17:13:31.290266+0000 mgr.gibba032.whopeo (mgr.24625) 10583 : cluster [DBG] pgmap v6028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:32.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:31 gibba011 bash[31774]: cluster 2021-03-10T17:13:31.290266+0000 mgr.gibba032.whopeo (mgr.24625) 10583 : cluster [DBG] pgmap v6028: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:34 gibba032 bash[24004]: cluster 2021-03-10T17:13:33.292059+0000 mgr.gibba032.whopeo (mgr.24625) 10584 : cluster [DBG] pgmap v6029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:34 gibba011 bash[30791]: cluster 2021-03-10T17:13:33.292059+0000 mgr.gibba032.whopeo (mgr.24625) 10584 : cluster [DBG] pgmap v6029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:34 gibba011 bash[31774]: cluster 2021-03-10T17:13:33.292059+0000 mgr.gibba032.whopeo (mgr.24625) 10584 : cluster [DBG] pgmap v6029: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:36 gibba032 bash[24004]: cluster 2021-03-10T17:13:35.293740+0000 mgr.gibba032.whopeo (mgr.24625) 10585 : cluster [DBG] pgmap v6030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:36 gibba011 bash[30791]: cluster 2021-03-10T17:13:35.293740+0000 mgr.gibba032.whopeo (mgr.24625) 10585 : cluster [DBG] pgmap v6030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:36.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:36 gibba011 bash[31774]: cluster 2021-03-10T17:13:35.293740+0000 mgr.gibba032.whopeo (mgr.24625) 10585 : cluster [DBG] pgmap v6030: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:37 gibba032 bash[24004]: cephadm 2021-03-10T17:13:35.960935+0000 mgr.gibba032.whopeo (mgr.24625) 10586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:37.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:37 gibba011 bash[30791]: cephadm 2021-03-10T17:13:35.960935+0000 mgr.gibba032.whopeo (mgr.24625) 10586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:37.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:37 gibba011 bash[31774]: cephadm 2021-03-10T17:13:35.960935+0000 mgr.gibba032.whopeo (mgr.24625) 10586 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:38.333 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:13:38.563 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:13:38.564 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:13:38.564 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 78s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:13:38.564 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:13:38.564 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:13:38.564 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:13:38.565 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:13:38.565 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:13:38.565 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:13:38.565 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:13:38.566 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 78s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:13:38.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:13:38.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:13:38.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:13:38.566 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:13:38.567 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:13:38.567 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:13:38.567 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:13:38.567 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 78s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:13:38.567 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 78s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:13:38.568 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:13:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:38 gibba032 bash[24004]: cluster 2021-03-10T17:13:37.295414+0000 mgr.gibba032.whopeo (mgr.24625) 10587 : cluster [DBG] pgmap v6031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:38.814 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:13:38.814 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:13:38.815 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:13:38.815 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:38.815 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:13:38.815 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:13:38.816 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:38.816 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:13:38.816 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:13:38.816 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:13:38.817 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:13:38.817 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:13:38.817 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:13:38.817 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:13:38.818 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:13:38.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:38 gibba011 bash[30791]: cluster 2021-03-10T17:13:37.295414+0000 mgr.gibba032.whopeo (mgr.24625) 10587 : cluster [DBG] pgmap v6031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:38 gibba011 bash[31774]: cluster 2021-03-10T17:13:37.295414+0000 mgr.gibba032.whopeo (mgr.24625) 10587 : cluster [DBG] pgmap v6031: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:39.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:39 gibba032 bash[24004]: audit 2021-03-10T17:13:38.322543+0000 mgr.gibba032.whopeo (mgr.24625) 10588 : audit [DBG] from='client.51147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:39 gibba032 bash[24004]: audit 2021-03-10T17:13:38.551172+0000 mgr.gibba032.whopeo (mgr.24625) 10589 : audit [DBG] from='client.51153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:39 gibba032 bash[24004]: audit 2021-03-10T17:13:38.814229+0000 mon.a (mon.0) 3537 : audit [DBG] from='client.? 172.21.2.111:0/947653641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:39.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[30791]: audit 2021-03-10T17:13:38.322543+0000 mgr.gibba032.whopeo (mgr.24625) 10588 : audit [DBG] from='client.51147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[30791]: audit 2021-03-10T17:13:38.551172+0000 mgr.gibba032.whopeo (mgr.24625) 10589 : audit [DBG] from='client.51153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[30791]: audit 2021-03-10T17:13:38.814229+0000 mon.a (mon.0) 3537 : audit [DBG] from='client.? 172.21.2.111:0/947653641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:39.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[31774]: audit 2021-03-10T17:13:38.322543+0000 mgr.gibba032.whopeo (mgr.24625) 10588 : audit [DBG] from='client.51147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[31774]: audit 2021-03-10T17:13:38.551172+0000 mgr.gibba032.whopeo (mgr.24625) 10589 : audit [DBG] from='client.51153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:13:39.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:39 gibba011 bash[31774]: audit 2021-03-10T17:13:38.814229+0000 mon.a (mon.0) 3537 : audit [DBG] from='client.? 172.21.2.111:0/947653641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:13:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:40 gibba032 bash[24004]: cluster 2021-03-10T17:13:39.296557+0000 mgr.gibba032.whopeo (mgr.24625) 10590 : cluster [DBG] pgmap v6032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:40 gibba011 bash[30791]: cluster 2021-03-10T17:13:39.296557+0000 mgr.gibba032.whopeo (mgr.24625) 10590 : cluster [DBG] pgmap v6032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:40 gibba011 bash[31774]: cluster 2021-03-10T17:13:39.296557+0000 mgr.gibba032.whopeo (mgr.24625) 10590 : cluster [DBG] pgmap v6032: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:41 gibba032 bash[24004]: cluster 2021-03-10T17:13:41.298312+0000 mgr.gibba032.whopeo (mgr.24625) 10591 : cluster [DBG] pgmap v6033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:41 gibba011 bash[30791]: cluster 2021-03-10T17:13:41.298312+0000 mgr.gibba032.whopeo (mgr.24625) 10591 : cluster [DBG] pgmap v6033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:41 gibba011 bash[31774]: cluster 2021-03-10T17:13:41.298312+0000 mgr.gibba032.whopeo (mgr.24625) 10591 : cluster [DBG] pgmap v6033: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:44 gibba032 bash[24004]: cluster 2021-03-10T17:13:43.299952+0000 mgr.gibba032.whopeo (mgr.24625) 10592 : cluster [DBG] pgmap v6034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:44 gibba011 bash[30791]: cluster 2021-03-10T17:13:43.299952+0000 mgr.gibba032.whopeo (mgr.24625) 10592 : cluster [DBG] pgmap v6034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:44 gibba011 bash[31774]: cluster 2021-03-10T17:13:43.299952+0000 mgr.gibba032.whopeo (mgr.24625) 10592 : cluster [DBG] pgmap v6034: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:46 gibba032 bash[24004]: cluster 2021-03-10T17:13:45.301587+0000 mgr.gibba032.whopeo (mgr.24625) 10593 : cluster [DBG] pgmap v6035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:46 gibba011 bash[30791]: cluster 2021-03-10T17:13:45.301587+0000 mgr.gibba032.whopeo (mgr.24625) 10593 : cluster [DBG] pgmap v6035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:46 gibba011 bash[31774]: cluster 2021-03-10T17:13:45.301587+0000 mgr.gibba032.whopeo (mgr.24625) 10593 : cluster [DBG] pgmap v6035: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:48 gibba032 bash[24004]: cluster 2021-03-10T17:13:47.303375+0000 mgr.gibba032.whopeo (mgr.24625) 10594 : cluster [DBG] pgmap v6036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:48 gibba011 bash[30791]: cluster 2021-03-10T17:13:47.303375+0000 mgr.gibba032.whopeo (mgr.24625) 10594 : cluster [DBG] pgmap v6036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:48 gibba011 bash[31774]: cluster 2021-03-10T17:13:47.303375+0000 mgr.gibba032.whopeo (mgr.24625) 10594 : cluster [DBG] pgmap v6036: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:50 gibba032 bash[24004]: cluster 2021-03-10T17:13:49.304782+0000 mgr.gibba032.whopeo (mgr.24625) 10595 : cluster [DBG] pgmap v6037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:50 gibba011 bash[30791]: cluster 2021-03-10T17:13:49.304782+0000 mgr.gibba032.whopeo (mgr.24625) 10595 : cluster [DBG] pgmap v6037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:50 gibba011 bash[31774]: cluster 2021-03-10T17:13:49.304782+0000 mgr.gibba032.whopeo (mgr.24625) 10595 : cluster [DBG] pgmap v6037: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:51 gibba032 bash[24004]: cephadm 2021-03-10T17:13:50.976655+0000 mgr.gibba032.whopeo (mgr.24625) 10596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:51 gibba032 bash[24004]: cluster 2021-03-10T17:13:51.306426+0000 mgr.gibba032.whopeo (mgr.24625) 10597 : cluster [DBG] pgmap v6038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:51 gibba011 bash[30791]: cephadm 2021-03-10T17:13:50.976655+0000 mgr.gibba032.whopeo (mgr.24625) 10596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:51 gibba011 bash[30791]: cluster 2021-03-10T17:13:51.306426+0000 mgr.gibba032.whopeo (mgr.24625) 10597 : cluster [DBG] pgmap v6038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:51 gibba011 bash[31774]: cephadm 2021-03-10T17:13:50.976655+0000 mgr.gibba032.whopeo (mgr.24625) 10596 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:13:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:51 gibba011 bash[31774]: cluster 2021-03-10T17:13:51.306426+0000 mgr.gibba032.whopeo (mgr.24625) 10597 : cluster [DBG] pgmap v6038: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:54 gibba032 bash[24004]: cluster 2021-03-10T17:13:53.307790+0000 mgr.gibba032.whopeo (mgr.24625) 10598 : cluster [DBG] pgmap v6039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:54 gibba011 bash[30791]: cluster 2021-03-10T17:13:53.307790+0000 mgr.gibba032.whopeo (mgr.24625) 10598 : cluster [DBG] pgmap v6039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:54.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:54 gibba011 bash[31774]: cluster 2021-03-10T17:13:53.307790+0000 mgr.gibba032.whopeo (mgr.24625) 10598 : cluster [DBG] pgmap v6039: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:56 gibba032 bash[24004]: cluster 2021-03-10T17:13:55.309377+0000 mgr.gibba032.whopeo (mgr.24625) 10599 : cluster [DBG] pgmap v6040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:56 gibba011 bash[30791]: cluster 2021-03-10T17:13:55.309377+0000 mgr.gibba032.whopeo (mgr.24625) 10599 : cluster [DBG] pgmap v6040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:56 gibba011 bash[31774]: cluster 2021-03-10T17:13:55.309377+0000 mgr.gibba032.whopeo (mgr.24625) 10599 : cluster [DBG] pgmap v6040: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: cluster 2021-03-10T17:13:57.311175+0000 mgr.gibba032.whopeo (mgr.24625) 10600 : cluster [DBG] pgmap v6041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.388807+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:13:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.389111+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:13:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.558503+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:13:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.559294+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:13:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.560127+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:13:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:13:58 gibba032 bash[24004]: audit 2021-03-10T17:13:57.560814+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:13:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: cluster 2021-03-10T17:13:57.311175+0000 mgr.gibba032.whopeo (mgr.24625) 10600 : cluster [DBG] pgmap v6041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.388807+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:13:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.389111+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:13:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.558503+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:13:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.559294+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:13:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.560127+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:13:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[30791]: audit 2021-03-10T17:13:57.560814+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:13:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: cluster 2021-03-10T17:13:57.311175+0000 mgr.gibba032.whopeo (mgr.24625) 10600 : cluster [DBG] pgmap v6041: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:13:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.388807+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:13:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.389111+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:13:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.558503+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:13:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.559294+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:13:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.560127+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:13:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:13:58 gibba011 bash[31774]: audit 2021-03-10T17:13:57.560814+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:14:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:00 gibba032 bash[24004]: cluster 2021-03-10T17:13:59.312407+0000 mgr.gibba032.whopeo (mgr.24625) 10601 : cluster [DBG] pgmap v6042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:00 gibba011 bash[30791]: cluster 2021-03-10T17:13:59.312407+0000 mgr.gibba032.whopeo (mgr.24625) 10601 : cluster [DBG] pgmap v6042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:00 gibba011 bash[31774]: cluster 2021-03-10T17:13:59.312407+0000 mgr.gibba032.whopeo (mgr.24625) 10601 : cluster [DBG] pgmap v6042: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:01 gibba032 bash[24004]: cluster 2021-03-10T17:14:01.314037+0000 mgr.gibba032.whopeo (mgr.24625) 10602 : cluster [DBG] pgmap v6043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:01 gibba011 bash[30791]: cluster 2021-03-10T17:14:01.314037+0000 mgr.gibba032.whopeo (mgr.24625) 10602 : cluster [DBG] pgmap v6043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:01 gibba011 bash[31774]: cluster 2021-03-10T17:14:01.314037+0000 mgr.gibba032.whopeo (mgr.24625) 10602 : cluster [DBG] pgmap v6043: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:04 gibba032 bash[24004]: cluster 2021-03-10T17:14:03.315541+0000 mgr.gibba032.whopeo (mgr.24625) 10603 : cluster [DBG] pgmap v6044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:04 gibba011 bash[30791]: cluster 2021-03-10T17:14:03.315541+0000 mgr.gibba032.whopeo (mgr.24625) 10603 : cluster [DBG] pgmap v6044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:04 gibba011 bash[31774]: cluster 2021-03-10T17:14:03.315541+0000 mgr.gibba032.whopeo (mgr.24625) 10603 : cluster [DBG] pgmap v6044: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:06 gibba032 bash[24004]: cluster 2021-03-10T17:14:05.317095+0000 mgr.gibba032.whopeo (mgr.24625) 10604 : cluster [DBG] pgmap v6045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:06 gibba011 bash[30791]: cluster 2021-03-10T17:14:05.317095+0000 mgr.gibba032.whopeo (mgr.24625) 10604 : cluster [DBG] pgmap v6045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:06 gibba011 bash[31774]: cluster 2021-03-10T17:14:05.317095+0000 mgr.gibba032.whopeo (mgr.24625) 10604 : cluster [DBG] pgmap v6045: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:07 gibba032 bash[24004]: cephadm 2021-03-10T17:14:05.988814+0000 mgr.gibba032.whopeo (mgr.24625) 10605 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:07 gibba011 bash[30791]: cephadm 2021-03-10T17:14:05.988814+0000 mgr.gibba032.whopeo (mgr.24625) 10605 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:07.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:07 gibba011 bash[31774]: cephadm 2021-03-10T17:14:05.988814+0000 mgr.gibba032.whopeo (mgr.24625) 10605 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:08 gibba032 bash[24004]: cluster 2021-03-10T17:14:07.318859+0000 mgr.gibba032.whopeo (mgr.24625) 10606 : cluster [DBG] pgmap v6046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:08 gibba011 bash[30791]: cluster 2021-03-10T17:14:07.318859+0000 mgr.gibba032.whopeo (mgr.24625) 10606 : cluster [DBG] pgmap v6046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:08 gibba011 bash[31774]: cluster 2021-03-10T17:14:07.318859+0000 mgr.gibba032.whopeo (mgr.24625) 10606 : cluster [DBG] pgmap v6046: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:09.060 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:14:09.295 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:14:09.295 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:14:09.295 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 108s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:14:09.295 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:14:09.296 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:14:09.296 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:14:09.296 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:14:09.296 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:14:09.297 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:14:09.297 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:14:09.297 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 108s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:14:09.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:14:09.297 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:14:09.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:14:09.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:14:09.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:14:09.298 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:14:09.299 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:14:09.299 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:14:09.299 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 108s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:14:09.299 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:14:09.545 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:14:09.545 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:14:09.546 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:14:09.546 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:09.546 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:14:09.546 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:14:09.547 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:09.547 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:14:09.547 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:14:09.547 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:09.547 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:14:09.548 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:14:09.548 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:14:09.548 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:14:09.548 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:14:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:10 gibba032 bash[24004]: audit 2021-03-10T17:14:09.049730+0000 mgr.gibba032.whopeo (mgr.24625) 10607 : audit [DBG] from='client.61147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:10 gibba032 bash[24004]: audit 2021-03-10T17:14:09.280992+0000 mgr.gibba032.whopeo (mgr.24625) 10608 : audit [DBG] from='client.51171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:10 gibba032 bash[24004]: cluster 2021-03-10T17:14:09.319600+0000 mgr.gibba032.whopeo (mgr.24625) 10609 : cluster [DBG] pgmap v6047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:10 gibba032 bash[24004]: audit 2021-03-10T17:14:09.545550+0000 mon.c (mon.1) 203 : audit [DBG] from='client.? 172.21.2.111:0/2781422545' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[30791]: audit 2021-03-10T17:14:09.049730+0000 mgr.gibba032.whopeo (mgr.24625) 10607 : audit [DBG] from='client.61147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[30791]: audit 2021-03-10T17:14:09.280992+0000 mgr.gibba032.whopeo (mgr.24625) 10608 : audit [DBG] from='client.51171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[30791]: cluster 2021-03-10T17:14:09.319600+0000 mgr.gibba032.whopeo (mgr.24625) 10609 : cluster [DBG] pgmap v6047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:10.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[30791]: audit 2021-03-10T17:14:09.545550+0000 mon.c (mon.1) 203 : audit [DBG] from='client.? 172.21.2.111:0/2781422545' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[31774]: audit 2021-03-10T17:14:09.049730+0000 mgr.gibba032.whopeo (mgr.24625) 10607 : audit [DBG] from='client.61147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[31774]: audit 2021-03-10T17:14:09.280992+0000 mgr.gibba032.whopeo (mgr.24625) 10608 : audit [DBG] from='client.51171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[31774]: cluster 2021-03-10T17:14:09.319600+0000 mgr.gibba032.whopeo (mgr.24625) 10609 : cluster [DBG] pgmap v6047: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:10 gibba011 bash[31774]: audit 2021-03-10T17:14:09.545550+0000 mon.c (mon.1) 203 : audit [DBG] from='client.? 172.21.2.111:0/2781422545' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:11 gibba032 bash[24004]: cluster 2021-03-10T17:14:11.320947+0000 mgr.gibba032.whopeo (mgr.24625) 10610 : cluster [DBG] pgmap v6048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:12.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:11 gibba011 bash[30791]: cluster 2021-03-10T17:14:11.320947+0000 mgr.gibba032.whopeo (mgr.24625) 10610 : cluster [DBG] pgmap v6048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:12.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:11 gibba011 bash[31774]: cluster 2021-03-10T17:14:11.320947+0000 mgr.gibba032.whopeo (mgr.24625) 10610 : cluster [DBG] pgmap v6048: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:14 gibba032 bash[24004]: cluster 2021-03-10T17:14:13.322291+0000 mgr.gibba032.whopeo (mgr.24625) 10611 : cluster [DBG] pgmap v6049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:14 gibba011 bash[30791]: cluster 2021-03-10T17:14:13.322291+0000 mgr.gibba032.whopeo (mgr.24625) 10611 : cluster [DBG] pgmap v6049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:14 gibba011 bash[31774]: cluster 2021-03-10T17:14:13.322291+0000 mgr.gibba032.whopeo (mgr.24625) 10611 : cluster [DBG] pgmap v6049: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:16.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:16 gibba032 bash[24004]: cluster 2021-03-10T17:14:15.323746+0000 mgr.gibba032.whopeo (mgr.24625) 10612 : cluster [DBG] pgmap v6050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:16 gibba011 bash[30791]: cluster 2021-03-10T17:14:15.323746+0000 mgr.gibba032.whopeo (mgr.24625) 10612 : cluster [DBG] pgmap v6050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:16 gibba011 bash[31774]: cluster 2021-03-10T17:14:15.323746+0000 mgr.gibba032.whopeo (mgr.24625) 10612 : cluster [DBG] pgmap v6050: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:18.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:18 gibba032 bash[24004]: cluster 2021-03-10T17:14:17.325587+0000 mgr.gibba032.whopeo (mgr.24625) 10613 : cluster [DBG] pgmap v6051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:18 gibba011 bash[30791]: cluster 2021-03-10T17:14:17.325587+0000 mgr.gibba032.whopeo (mgr.24625) 10613 : cluster [DBG] pgmap v6051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:18 gibba011 bash[31774]: cluster 2021-03-10T17:14:17.325587+0000 mgr.gibba032.whopeo (mgr.24625) 10613 : cluster [DBG] pgmap v6051: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:20 gibba032 bash[24004]: cluster 2021-03-10T17:14:19.327089+0000 mgr.gibba032.whopeo (mgr.24625) 10614 : cluster [DBG] pgmap v6052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:20 gibba011 bash[30791]: cluster 2021-03-10T17:14:19.327089+0000 mgr.gibba032.whopeo (mgr.24625) 10614 : cluster [DBG] pgmap v6052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:20.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:20 gibba011 bash[31774]: cluster 2021-03-10T17:14:19.327089+0000 mgr.gibba032.whopeo (mgr.24625) 10614 : cluster [DBG] pgmap v6052: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.005414+0000 mgr.gibba032.whopeo (mgr.24625) 10615 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:14:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.006949+0000 mgr.gibba032.whopeo (mgr.24625) 10616 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:14:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.220872+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:14:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.221424+0000 mgr.gibba032.whopeo (mgr.24625) 10617 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:14:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.223491+0000 mgr.gibba032.whopeo (mgr.24625) 10618 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:14:22.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.224312+0000 mon.a (mon.0) 3545 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:14:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.224855+0000 mgr.gibba032.whopeo (mgr.24625) 10619 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:14:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.224893+0000 mgr.gibba032.whopeo (mgr.24625) 10620 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:14:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.225827+0000 mon.a (mon.0) 3546 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.226566+0000 mgr.gibba032.whopeo (mgr.24625) 10621 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:14:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.226605+0000 mgr.gibba032.whopeo (mgr.24625) 10622 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:14:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.227405+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.227849+0000 mgr.gibba032.whopeo (mgr.24625) 10623 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:14:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.228837+0000 mgr.gibba032.whopeo (mgr.24625) 10624 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:14:22.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.228875+0000 mgr.gibba032.whopeo (mgr.24625) 10625 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:14:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.228890+0000 mon.a (mon.0) 3548 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.230265+0000 mgr.gibba032.whopeo (mgr.24625) 10626 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:14:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.230602+0000 mgr.gibba032.whopeo (mgr.24625) 10627 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:14:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.231226+0000 mon.a (mon.0) 3549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.231836+0000 mgr.gibba032.whopeo (mgr.24625) 10628 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:14:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.231875+0000 mgr.gibba032.whopeo (mgr.24625) 10629 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:14:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cephadm 2021-03-10T17:14:21.231979+0000 mgr.gibba032.whopeo (mgr.24625) 10630 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: audit 2021-03-10T17:14:21.232871+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:22 gibba032 bash[24004]: cluster 2021-03-10T17:14:21.328962+0000 mgr.gibba032.whopeo (mgr.24625) 10631 : cluster [DBG] pgmap v6053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.005414+0000 mgr.gibba032.whopeo (mgr.24625) 10615 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:14:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.006949+0000 mgr.gibba032.whopeo (mgr.24625) 10616 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:14:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.220872+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:14:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.221424+0000 mgr.gibba032.whopeo (mgr.24625) 10617 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:14:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.223491+0000 mgr.gibba032.whopeo (mgr.24625) 10618 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:14:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.224312+0000 mon.a (mon.0) 3545 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:14:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.224855+0000 mgr.gibba032.whopeo (mgr.24625) 10619 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:14:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.224893+0000 mgr.gibba032.whopeo (mgr.24625) 10620 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:14:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.225827+0000 mon.a (mon.0) 3546 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.226566+0000 mgr.gibba032.whopeo (mgr.24625) 10621 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:14:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.226605+0000 mgr.gibba032.whopeo (mgr.24625) 10622 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:14:22.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.227405+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.227849+0000 mgr.gibba032.whopeo (mgr.24625) 10623 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:14:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.228837+0000 mgr.gibba032.whopeo (mgr.24625) 10624 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:14:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.228875+0000 mgr.gibba032.whopeo (mgr.24625) 10625 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:14:22.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.228890+0000 mon.a (mon.0) 3548 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.230265+0000 mgr.gibba032.whopeo (mgr.24625) 10626 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:14:22.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.230602+0000 mgr.gibba032.whopeo (mgr.24625) 10627 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:14:22.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.231226+0000 mon.a (mon.0) 3549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.231836+0000 mgr.gibba032.whopeo (mgr.24625) 10628 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:14:22.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.231875+0000 mgr.gibba032.whopeo (mgr.24625) 10629 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:14:22.594 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cephadm 2021-03-10T17:14:21.231979+0000 mgr.gibba032.whopeo (mgr.24625) 10630 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:22.595 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: audit 2021-03-10T17:14:21.232871+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.597 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[30791]: cluster 2021-03-10T17:14:21.328962+0000 mgr.gibba032.whopeo (mgr.24625) 10631 : cluster [DBG] pgmap v6053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:22.598 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.005414+0000 mgr.gibba032.whopeo (mgr.24625) 10615 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:14:22.598 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.006949+0000 mgr.gibba032.whopeo (mgr.24625) 10616 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:14:22.599 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.220872+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:14:22.599 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.221424+0000 mgr.gibba032.whopeo (mgr.24625) 10617 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:14:22.599 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.223491+0000 mgr.gibba032.whopeo (mgr.24625) 10618 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:14:22.599 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.224312+0000 mon.a (mon.0) 3545 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:14:22.600 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.224855+0000 mgr.gibba032.whopeo (mgr.24625) 10619 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:14:22.600 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.224893+0000 mgr.gibba032.whopeo (mgr.24625) 10620 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:14:22.600 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.225827+0000 mon.a (mon.0) 3546 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.600 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.226566+0000 mgr.gibba032.whopeo (mgr.24625) 10621 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:14:22.600 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.226605+0000 mgr.gibba032.whopeo (mgr.24625) 10622 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:14:22.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.227405+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.227849+0000 mgr.gibba032.whopeo (mgr.24625) 10623 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:14:22.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.228837+0000 mgr.gibba032.whopeo (mgr.24625) 10624 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:14:22.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.228875+0000 mgr.gibba032.whopeo (mgr.24625) 10625 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:14:22.601 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.228890+0000 mon.a (mon.0) 3548 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.230265+0000 mgr.gibba032.whopeo (mgr.24625) 10626 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:14:22.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.230602+0000 mgr.gibba032.whopeo (mgr.24625) 10627 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:14:22.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.231226+0000 mon.a (mon.0) 3549 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.231836+0000 mgr.gibba032.whopeo (mgr.24625) 10628 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:14:22.602 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.231875+0000 mgr.gibba032.whopeo (mgr.24625) 10629 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:14:22.603 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cephadm 2021-03-10T17:14:21.231979+0000 mgr.gibba032.whopeo (mgr.24625) 10630 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:22.603 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: audit 2021-03-10T17:14:21.232871+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:22.603 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:22 gibba011 bash[31774]: cluster 2021-03-10T17:14:21.328962+0000 mgr.gibba032.whopeo (mgr.24625) 10631 : cluster [DBG] pgmap v6053: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:24 gibba032 bash[24004]: cluster 2021-03-10T17:14:23.330554+0000 mgr.gibba032.whopeo (mgr.24625) 10632 : cluster [DBG] pgmap v6054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:24 gibba011 bash[30791]: cluster 2021-03-10T17:14:23.330554+0000 mgr.gibba032.whopeo (mgr.24625) 10632 : cluster [DBG] pgmap v6054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:24 gibba011 bash[31774]: cluster 2021-03-10T17:14:23.330554+0000 mgr.gibba032.whopeo (mgr.24625) 10632 : cluster [DBG] pgmap v6054: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:26.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:26 gibba032 bash[24004]: cluster 2021-03-10T17:14:25.332147+0000 mgr.gibba032.whopeo (mgr.24625) 10633 : cluster [DBG] pgmap v6055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:26 gibba011 bash[30791]: cluster 2021-03-10T17:14:25.332147+0000 mgr.gibba032.whopeo (mgr.24625) 10633 : cluster [DBG] pgmap v6055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:26 gibba011 bash[31774]: cluster 2021-03-10T17:14:25.332147+0000 mgr.gibba032.whopeo (mgr.24625) 10633 : cluster [DBG] pgmap v6055: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:28.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:28 gibba032 bash[24004]: cluster 2021-03-10T17:14:27.333892+0000 mgr.gibba032.whopeo (mgr.24625) 10634 : cluster [DBG] pgmap v6056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:28 gibba011 bash[30791]: cluster 2021-03-10T17:14:27.333892+0000 mgr.gibba032.whopeo (mgr.24625) 10634 : cluster [DBG] pgmap v6056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:28 gibba011 bash[31774]: cluster 2021-03-10T17:14:27.333892+0000 mgr.gibba032.whopeo (mgr.24625) 10634 : cluster [DBG] pgmap v6056: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:30 gibba032 bash[24004]: cluster 2021-03-10T17:14:29.335404+0000 mgr.gibba032.whopeo (mgr.24625) 10635 : cluster [DBG] pgmap v6057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:30 gibba011 bash[30791]: cluster 2021-03-10T17:14:29.335404+0000 mgr.gibba032.whopeo (mgr.24625) 10635 : cluster [DBG] pgmap v6057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:30 gibba011 bash[31774]: cluster 2021-03-10T17:14:29.335404+0000 mgr.gibba032.whopeo (mgr.24625) 10635 : cluster [DBG] pgmap v6057: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:31 gibba032 bash[24004]: cluster 2021-03-10T17:14:31.336867+0000 mgr.gibba032.whopeo (mgr.24625) 10636 : cluster [DBG] pgmap v6058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:31 gibba011 bash[30791]: cluster 2021-03-10T17:14:31.336867+0000 mgr.gibba032.whopeo (mgr.24625) 10636 : cluster [DBG] pgmap v6058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:31 gibba011 bash[31774]: cluster 2021-03-10T17:14:31.336867+0000 mgr.gibba032.whopeo (mgr.24625) 10636 : cluster [DBG] pgmap v6058: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:34 gibba032 bash[24004]: cluster 2021-03-10T17:14:33.338462+0000 mgr.gibba032.whopeo (mgr.24625) 10637 : cluster [DBG] pgmap v6059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:34 gibba011 bash[30791]: cluster 2021-03-10T17:14:33.338462+0000 mgr.gibba032.whopeo (mgr.24625) 10637 : cluster [DBG] pgmap v6059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:34.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:34 gibba011 bash[31774]: cluster 2021-03-10T17:14:33.338462+0000 mgr.gibba032.whopeo (mgr.24625) 10637 : cluster [DBG] pgmap v6059: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:36.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:36 gibba032 bash[24004]: cluster 2021-03-10T17:14:35.339937+0000 mgr.gibba032.whopeo (mgr.24625) 10638 : cluster [DBG] pgmap v6060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:36 gibba011 bash[30791]: cluster 2021-03-10T17:14:35.339937+0000 mgr.gibba032.whopeo (mgr.24625) 10638 : cluster [DBG] pgmap v6060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:36 gibba011 bash[31774]: cluster 2021-03-10T17:14:35.339937+0000 mgr.gibba032.whopeo (mgr.24625) 10638 : cluster [DBG] pgmap v6060: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:37 gibba032 bash[24004]: cephadm 2021-03-10T17:14:36.248072+0000 mgr.gibba032.whopeo (mgr.24625) 10639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:37.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:37 gibba011 bash[30791]: cephadm 2021-03-10T17:14:36.248072+0000 mgr.gibba032.whopeo (mgr.24625) 10639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:37.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:37 gibba011 bash[31774]: cephadm 2021-03-10T17:14:36.248072+0000 mgr.gibba032.whopeo (mgr.24625) 10639 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:38.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:38 gibba032 bash[24004]: cluster 2021-03-10T17:14:37.341720+0000 mgr.gibba032.whopeo (mgr.24625) 10640 : cluster [DBG] pgmap v6061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:38 gibba011 bash[30791]: cluster 2021-03-10T17:14:37.341720+0000 mgr.gibba032.whopeo (mgr.24625) 10640 : cluster [DBG] pgmap v6061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:38 gibba011 bash[31774]: cluster 2021-03-10T17:14:37.341720+0000 mgr.gibba032.whopeo (mgr.24625) 10640 : cluster [DBG] pgmap v6061: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:39.790 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:14:40.015 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:14:40.016 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:14:40.016 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:14:40.016 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:14:40.017 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:14:40.017 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:14:40.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:14:40.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:14:40.017 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:14:40.018 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:14:40.018 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:14:40.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:14:40.018 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:14:40.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:14:40.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:14:40.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:14:40.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:14:40.019 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:14:40.020 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:14:40.020 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:14:40.020 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:14:40.267 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:14:40.268 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:14:40.268 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:14:40.268 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:40.269 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:14:40.269 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:14:40.269 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:40.269 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:14:40.269 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:14:40.270 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:14:40.270 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:14:40.270 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:14:40.270 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:14:40.270 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:14:40.271 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:14:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:40 gibba032 bash[24004]: cluster 2021-03-10T17:14:39.343092+0000 mgr.gibba032.whopeo (mgr.24625) 10641 : cluster [DBG] pgmap v6062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:40 gibba032 bash[24004]: audit 2021-03-10T17:14:39.779576+0000 mgr.gibba032.whopeo (mgr.24625) 10642 : audit [DBG] from='client.51183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:40 gibba032 bash[24004]: audit 2021-03-10T17:14:40.267790+0000 mon.c (mon.1) 204 : audit [DBG] from='client.? 172.21.2.111:0/174162756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[30791]: cluster 2021-03-10T17:14:39.343092+0000 mgr.gibba032.whopeo (mgr.24625) 10641 : cluster [DBG] pgmap v6062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[30791]: audit 2021-03-10T17:14:39.779576+0000 mgr.gibba032.whopeo (mgr.24625) 10642 : audit [DBG] from='client.51183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[30791]: audit 2021-03-10T17:14:40.267790+0000 mon.c (mon.1) 204 : audit [DBG] from='client.? 172.21.2.111:0/174162756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[31774]: cluster 2021-03-10T17:14:39.343092+0000 mgr.gibba032.whopeo (mgr.24625) 10641 : cluster [DBG] pgmap v6062: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[31774]: audit 2021-03-10T17:14:39.779576+0000 mgr.gibba032.whopeo (mgr.24625) 10642 : audit [DBG] from='client.51183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:40.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:40 gibba011 bash[31774]: audit 2021-03-10T17:14:40.267790+0000 mon.c (mon.1) 204 : audit [DBG] from='client.? 172.21.2.111:0/174162756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:14:41.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:41 gibba032 bash[24004]: audit 2021-03-10T17:14:40.005305+0000 mgr.gibba032.whopeo (mgr.24625) 10643 : audit [DBG] from='client.61171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:41.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:41 gibba011 bash[30791]: audit 2021-03-10T17:14:40.005305+0000 mgr.gibba032.whopeo (mgr.24625) 10643 : audit [DBG] from='client.61171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:41.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:41 gibba011 bash[31774]: audit 2021-03-10T17:14:40.005305+0000 mgr.gibba032.whopeo (mgr.24625) 10643 : audit [DBG] from='client.61171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:14:42.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:42 gibba032 bash[24004]: cluster 2021-03-10T17:14:41.344842+0000 mgr.gibba032.whopeo (mgr.24625) 10644 : cluster [DBG] pgmap v6063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:42.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:42 gibba011 bash[30791]: cluster 2021-03-10T17:14:41.344842+0000 mgr.gibba032.whopeo (mgr.24625) 10644 : cluster [DBG] pgmap v6063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:42 gibba011 bash[31774]: cluster 2021-03-10T17:14:41.344842+0000 mgr.gibba032.whopeo (mgr.24625) 10644 : cluster [DBG] pgmap v6063: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:44 gibba032 bash[24004]: cluster 2021-03-10T17:14:43.346425+0000 mgr.gibba032.whopeo (mgr.24625) 10645 : cluster [DBG] pgmap v6064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:44 gibba011 bash[30791]: cluster 2021-03-10T17:14:43.346425+0000 mgr.gibba032.whopeo (mgr.24625) 10645 : cluster [DBG] pgmap v6064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:44 gibba011 bash[31774]: cluster 2021-03-10T17:14:43.346425+0000 mgr.gibba032.whopeo (mgr.24625) 10645 : cluster [DBG] pgmap v6064: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:46 gibba032 bash[24004]: cluster 2021-03-10T17:14:45.348209+0000 mgr.gibba032.whopeo (mgr.24625) 10646 : cluster [DBG] pgmap v6065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:46 gibba011 bash[30791]: cluster 2021-03-10T17:14:45.348209+0000 mgr.gibba032.whopeo (mgr.24625) 10646 : cluster [DBG] pgmap v6065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:46 gibba011 bash[31774]: cluster 2021-03-10T17:14:45.348209+0000 mgr.gibba032.whopeo (mgr.24625) 10646 : cluster [DBG] pgmap v6065: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:48.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:48 gibba032 bash[24004]: cluster 2021-03-10T17:14:47.350129+0000 mgr.gibba032.whopeo (mgr.24625) 10647 : cluster [DBG] pgmap v6066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:48 gibba011 bash[30791]: cluster 2021-03-10T17:14:47.350129+0000 mgr.gibba032.whopeo (mgr.24625) 10647 : cluster [DBG] pgmap v6066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:48 gibba011 bash[31774]: cluster 2021-03-10T17:14:47.350129+0000 mgr.gibba032.whopeo (mgr.24625) 10647 : cluster [DBG] pgmap v6066: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:50 gibba032 bash[24004]: cluster 2021-03-10T17:14:49.351656+0000 mgr.gibba032.whopeo (mgr.24625) 10648 : cluster [DBG] pgmap v6067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:50 gibba011 bash[30791]: cluster 2021-03-10T17:14:49.351656+0000 mgr.gibba032.whopeo (mgr.24625) 10648 : cluster [DBG] pgmap v6067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:50 gibba011 bash[31774]: cluster 2021-03-10T17:14:49.351656+0000 mgr.gibba032.whopeo (mgr.24625) 10648 : cluster [DBG] pgmap v6067: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:51 gibba032 bash[24004]: cephadm 2021-03-10T17:14:51.261361+0000 mgr.gibba032.whopeo (mgr.24625) 10649 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:51 gibba032 bash[24004]: cluster 2021-03-10T17:14:51.353345+0000 mgr.gibba032.whopeo (mgr.24625) 10650 : cluster [DBG] pgmap v6068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:51 gibba011 bash[30791]: cephadm 2021-03-10T17:14:51.261361+0000 mgr.gibba032.whopeo (mgr.24625) 10649 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:52.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:51 gibba011 bash[30791]: cluster 2021-03-10T17:14:51.353345+0000 mgr.gibba032.whopeo (mgr.24625) 10650 : cluster [DBG] pgmap v6068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:51 gibba011 bash[31774]: cephadm 2021-03-10T17:14:51.261361+0000 mgr.gibba032.whopeo (mgr.24625) 10649 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:14:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:51 gibba011 bash[31774]: cluster 2021-03-10T17:14:51.353345+0000 mgr.gibba032.whopeo (mgr.24625) 10650 : cluster [DBG] pgmap v6068: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:54 gibba032 bash[24004]: cluster 2021-03-10T17:14:53.354702+0000 mgr.gibba032.whopeo (mgr.24625) 10651 : cluster [DBG] pgmap v6069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:54 gibba011 bash[30791]: cluster 2021-03-10T17:14:53.354702+0000 mgr.gibba032.whopeo (mgr.24625) 10651 : cluster [DBG] pgmap v6069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:54.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:54 gibba011 bash[31774]: cluster 2021-03-10T17:14:53.354702+0000 mgr.gibba032.whopeo (mgr.24625) 10651 : cluster [DBG] pgmap v6069: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:56.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:56 gibba032 bash[24004]: cluster 2021-03-10T17:14:55.356453+0000 mgr.gibba032.whopeo (mgr.24625) 10652 : cluster [DBG] pgmap v6070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:56 gibba011 bash[30791]: cluster 2021-03-10T17:14:55.356453+0000 mgr.gibba032.whopeo (mgr.24625) 10652 : cluster [DBG] pgmap v6070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:56.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:56 gibba011 bash[31774]: cluster 2021-03-10T17:14:55.356453+0000 mgr.gibba032.whopeo (mgr.24625) 10652 : cluster [DBG] pgmap v6070: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:57 gibba032 bash[24004]: audit 2021-03-10T17:14:57.400504+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:14:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:57 gibba032 bash[24004]: audit 2021-03-10T17:14:57.401141+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:14:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:57 gibba011 bash[30791]: audit 2021-03-10T17:14:57.400504+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:14:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:57 gibba011 bash[30791]: audit 2021-03-10T17:14:57.401141+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:14:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:57 gibba011 bash[31774]: audit 2021-03-10T17:14:57.400504+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:14:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:57 gibba011 bash[31774]: audit 2021-03-10T17:14:57.401141+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:14:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:58 gibba032 bash[24004]: cluster 2021-03-10T17:14:57.358278+0000 mgr.gibba032.whopeo (mgr.24625) 10653 : cluster [DBG] pgmap v6071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:58 gibba032 bash[24004]: audit 2021-03-10T17:14:57.568700+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:14:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:58 gibba032 bash[24004]: audit 2021-03-10T17:14:57.569440+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:14:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:58 gibba032 bash[24004]: audit 2021-03-10T17:14:57.570139+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:14:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:14:58 gibba032 bash[24004]: audit 2021-03-10T17:14:57.570821+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:14:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[30791]: cluster 2021-03-10T17:14:57.358278+0000 mgr.gibba032.whopeo (mgr.24625) 10653 : cluster [DBG] pgmap v6071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[30791]: audit 2021-03-10T17:14:57.568700+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:14:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[30791]: audit 2021-03-10T17:14:57.569440+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:14:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[30791]: audit 2021-03-10T17:14:57.570139+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:14:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[30791]: audit 2021-03-10T17:14:57.570821+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:14:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[31774]: cluster 2021-03-10T17:14:57.358278+0000 mgr.gibba032.whopeo (mgr.24625) 10653 : cluster [DBG] pgmap v6071: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:14:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[31774]: audit 2021-03-10T17:14:57.568700+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:14:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[31774]: audit 2021-03-10T17:14:57.569440+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:14:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[31774]: audit 2021-03-10T17:14:57.570139+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:14:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:14:58 gibba011 bash[31774]: audit 2021-03-10T17:14:57.570821+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:15:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:00 gibba032 bash[24004]: cluster 2021-03-10T17:14:59.359910+0000 mgr.gibba032.whopeo (mgr.24625) 10654 : cluster [DBG] pgmap v6072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:00 gibba011 bash[30791]: cluster 2021-03-10T17:14:59.359910+0000 mgr.gibba032.whopeo (mgr.24625) 10654 : cluster [DBG] pgmap v6072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:00.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:00 gibba011 bash[31774]: cluster 2021-03-10T17:14:59.359910+0000 mgr.gibba032.whopeo (mgr.24625) 10654 : cluster [DBG] pgmap v6072: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:01 gibba032 bash[24004]: cluster 2021-03-10T17:15:01.361675+0000 mgr.gibba032.whopeo (mgr.24625) 10655 : cluster [DBG] pgmap v6073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:01 gibba011 bash[30791]: cluster 2021-03-10T17:15:01.361675+0000 mgr.gibba032.whopeo (mgr.24625) 10655 : cluster [DBG] pgmap v6073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:01 gibba011 bash[31774]: cluster 2021-03-10T17:15:01.361675+0000 mgr.gibba032.whopeo (mgr.24625) 10655 : cluster [DBG] pgmap v6073: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:04.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:04 gibba032 bash[24004]: cluster 2021-03-10T17:15:03.363057+0000 mgr.gibba032.whopeo (mgr.24625) 10656 : cluster [DBG] pgmap v6074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:04 gibba011 bash[30791]: cluster 2021-03-10T17:15:03.363057+0000 mgr.gibba032.whopeo (mgr.24625) 10656 : cluster [DBG] pgmap v6074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:04 gibba011 bash[31774]: cluster 2021-03-10T17:15:03.363057+0000 mgr.gibba032.whopeo (mgr.24625) 10656 : cluster [DBG] pgmap v6074: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:06.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:06 gibba032 bash[24004]: cluster 2021-03-10T17:15:05.364672+0000 mgr.gibba032.whopeo (mgr.24625) 10657 : cluster [DBG] pgmap v6075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:06 gibba011 bash[30791]: cluster 2021-03-10T17:15:05.364672+0000 mgr.gibba032.whopeo (mgr.24625) 10657 : cluster [DBG] pgmap v6075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:06 gibba011 bash[31774]: cluster 2021-03-10T17:15:05.364672+0000 mgr.gibba032.whopeo (mgr.24625) 10657 : cluster [DBG] pgmap v6075: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:07.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:07 gibba032 bash[24004]: cephadm 2021-03-10T17:15:06.276801+0000 mgr.gibba032.whopeo (mgr.24625) 10658 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:07 gibba011 bash[30791]: cephadm 2021-03-10T17:15:06.276801+0000 mgr.gibba032.whopeo (mgr.24625) 10658 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:07.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:07 gibba011 bash[31774]: cephadm 2021-03-10T17:15:06.276801+0000 mgr.gibba032.whopeo (mgr.24625) 10658 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:08 gibba032 bash[24004]: cluster 2021-03-10T17:15:07.366319+0000 mgr.gibba032.whopeo (mgr.24625) 10659 : cluster [DBG] pgmap v6076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:08 gibba011 bash[30791]: cluster 2021-03-10T17:15:07.366319+0000 mgr.gibba032.whopeo (mgr.24625) 10659 : cluster [DBG] pgmap v6076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:08 gibba011 bash[31774]: cluster 2021-03-10T17:15:07.366319+0000 mgr.gibba032.whopeo (mgr.24625) 10659 : cluster [DBG] pgmap v6076: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:10.554 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:15:10.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:10 gibba032 bash[24004]: cluster 2021-03-10T17:15:09.367746+0000 mgr.gibba032.whopeo (mgr.24625) 10660 : cluster [DBG] pgmap v6077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:10.803 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:15:10.804 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:15:10.804 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:15:10.804 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:15:10.804 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:15:10.805 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:15:10.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:15:10.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:15:10.805 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:15:10.806 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:15:10.806 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:15:10.806 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:15:10.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:15:10.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:15:10.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:15:10.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:15:10.807 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:15:10.808 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:15:10.808 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:15:10.808 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:15:10.808 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:15:10.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:10 gibba011 bash[30791]: cluster 2021-03-10T17:15:09.367746+0000 mgr.gibba032.whopeo (mgr.24625) 10660 : cluster [DBG] pgmap v6077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:10 gibba011 bash[31774]: cluster 2021-03-10T17:15:09.367746+0000 mgr.gibba032.whopeo (mgr.24625) 10660 : cluster [DBG] pgmap v6077: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:11.054 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:15:11.055 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:15:11.055 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:15:11.055 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:11.055 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:15:11.056 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:15:11.056 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:11.056 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:15:11.056 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:15:11.056 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:11.057 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:15:11.057 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:15:11.057 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:15:11.057 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:15:11.057 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:15:11.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:11 gibba032 bash[24004]: audit 2021-03-10T17:15:10.543332+0000 mgr.gibba032.whopeo (mgr.24625) 10661 : audit [DBG] from='client.51201 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:11 gibba032 bash[24004]: audit 2021-03-10T17:15:10.765158+0000 mgr.gibba032.whopeo (mgr.24625) 10662 : audit [DBG] from='client.51207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:11 gibba032 bash[24004]: audit 2021-03-10T17:15:11.054649+0000 mon.c (mon.1) 205 : audit [DBG] from='client.? 172.21.2.111:0/2622058273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[30791]: audit 2021-03-10T17:15:10.543332+0000 mgr.gibba032.whopeo (mgr.24625) 10661 : audit [DBG] from='client.51201 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[30791]: audit 2021-03-10T17:15:10.765158+0000 mgr.gibba032.whopeo (mgr.24625) 10662 : audit [DBG] from='client.51207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[30791]: audit 2021-03-10T17:15:11.054649+0000 mon.c (mon.1) 205 : audit [DBG] from='client.? 172.21.2.111:0/2622058273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[31774]: audit 2021-03-10T17:15:10.543332+0000 mgr.gibba032.whopeo (mgr.24625) 10661 : audit [DBG] from='client.51201 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[31774]: audit 2021-03-10T17:15:10.765158+0000 mgr.gibba032.whopeo (mgr.24625) 10662 : audit [DBG] from='client.51207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:11.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:11 gibba011 bash[31774]: audit 2021-03-10T17:15:11.054649+0000 mon.c (mon.1) 205 : audit [DBG] from='client.? 172.21.2.111:0/2622058273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:12.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:12 gibba032 bash[24004]: cluster 2021-03-10T17:15:11.369613+0000 mgr.gibba032.whopeo (mgr.24625) 10663 : cluster [DBG] pgmap v6078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:12.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:12 gibba011 bash[30791]: cluster 2021-03-10T17:15:11.369613+0000 mgr.gibba032.whopeo (mgr.24625) 10663 : cluster [DBG] pgmap v6078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:12.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:12 gibba011 bash[31774]: cluster 2021-03-10T17:15:11.369613+0000 mgr.gibba032.whopeo (mgr.24625) 10663 : cluster [DBG] pgmap v6078: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:14.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:14 gibba032 bash[24004]: cluster 2021-03-10T17:15:13.371196+0000 mgr.gibba032.whopeo (mgr.24625) 10664 : cluster [DBG] pgmap v6079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:14 gibba011 bash[30791]: cluster 2021-03-10T17:15:13.371196+0000 mgr.gibba032.whopeo (mgr.24625) 10664 : cluster [DBG] pgmap v6079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:14 gibba011 bash[31774]: cluster 2021-03-10T17:15:13.371196+0000 mgr.gibba032.whopeo (mgr.24625) 10664 : cluster [DBG] pgmap v6079: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:16 gibba032 bash[24004]: cluster 2021-03-10T17:15:15.372895+0000 mgr.gibba032.whopeo (mgr.24625) 10665 : cluster [DBG] pgmap v6080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:16 gibba011 bash[30791]: cluster 2021-03-10T17:15:15.372895+0000 mgr.gibba032.whopeo (mgr.24625) 10665 : cluster [DBG] pgmap v6080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:16 gibba011 bash[31774]: cluster 2021-03-10T17:15:15.372895+0000 mgr.gibba032.whopeo (mgr.24625) 10665 : cluster [DBG] pgmap v6080: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:18 gibba032 bash[24004]: cluster 2021-03-10T17:15:17.374776+0000 mgr.gibba032.whopeo (mgr.24625) 10666 : cluster [DBG] pgmap v6081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:18 gibba011 bash[30791]: cluster 2021-03-10T17:15:17.374776+0000 mgr.gibba032.whopeo (mgr.24625) 10666 : cluster [DBG] pgmap v6081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:18 gibba011 bash[31774]: cluster 2021-03-10T17:15:17.374776+0000 mgr.gibba032.whopeo (mgr.24625) 10666 : cluster [DBG] pgmap v6081: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:20 gibba032 bash[24004]: cluster 2021-03-10T17:15:19.376458+0000 mgr.gibba032.whopeo (mgr.24625) 10667 : cluster [DBG] pgmap v6082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:20 gibba011 bash[30791]: cluster 2021-03-10T17:15:19.376458+0000 mgr.gibba032.whopeo (mgr.24625) 10667 : cluster [DBG] pgmap v6082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:20 gibba011 bash[31774]: cluster 2021-03-10T17:15:19.376458+0000 mgr.gibba032.whopeo (mgr.24625) 10667 : cluster [DBG] pgmap v6082: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.293507+0000 mgr.gibba032.whopeo (mgr.24625) 10668 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:15:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.295038+0000 mgr.gibba032.whopeo (mgr.24625) 10669 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:15:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cluster 2021-03-10T17:15:21.377035+0000 mgr.gibba032.whopeo (mgr.24625) 10670 : cluster [DBG] pgmap v6083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.509782+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:15:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.510277+0000 mgr.gibba032.whopeo (mgr.24625) 10671 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:15:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.512175+0000 mgr.gibba032.whopeo (mgr.24625) 10672 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:15:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.513082+0000 mon.a (mon.0) 3558 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:15:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.513390+0000 mgr.gibba032.whopeo (mgr.24625) 10673 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:15:22.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.513428+0000 mgr.gibba032.whopeo (mgr.24625) 10674 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:15:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.514438+0000 mon.a (mon.0) 3559 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.514557+0000 mgr.gibba032.whopeo (mgr.24625) 10675 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:15:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.514602+0000 mgr.gibba032.whopeo (mgr.24625) 10676 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:15:22.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.515642+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.515759+0000 mgr.gibba032.whopeo (mgr.24625) 10677 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:15:22.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.516874+0000 mon.a (mon.0) 3561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.516921+0000 mgr.gibba032.whopeo (mgr.24625) 10678 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:15:22.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.516961+0000 mgr.gibba032.whopeo (mgr.24625) 10679 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:15:22.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.518347+0000 mgr.gibba032.whopeo (mgr.24625) 10680 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:15:22.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.518665+0000 mgr.gibba032.whopeo (mgr.24625) 10681 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:15:22.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.519432+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.520238+0000 mgr.gibba032.whopeo (mgr.24625) 10682 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:15:22.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.293507+0000 mgr.gibba032.whopeo (mgr.24625) 10668 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:15:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.295038+0000 mgr.gibba032.whopeo (mgr.24625) 10669 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:15:22.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cluster 2021-03-10T17:15:21.377035+0000 mgr.gibba032.whopeo (mgr.24625) 10670 : cluster [DBG] pgmap v6083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:22.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.509782+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:15:22.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.510277+0000 mgr.gibba032.whopeo (mgr.24625) 10671 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:15:22.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.512175+0000 mgr.gibba032.whopeo (mgr.24625) 10672 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:15:22.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.513082+0000 mon.a (mon.0) 3558 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:15:22.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.513390+0000 mgr.gibba032.whopeo (mgr.24625) 10673 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:15:22.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.513428+0000 mgr.gibba032.whopeo (mgr.24625) 10674 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:15:22.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.514438+0000 mon.a (mon.0) 3559 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.514557+0000 mgr.gibba032.whopeo (mgr.24625) 10675 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:15:22.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.514602+0000 mgr.gibba032.whopeo (mgr.24625) 10676 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:15:22.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.515642+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.515759+0000 mgr.gibba032.whopeo (mgr.24625) 10677 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:15:22.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.516874+0000 mon.a (mon.0) 3561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.516921+0000 mgr.gibba032.whopeo (mgr.24625) 10678 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:15:22.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.516961+0000 mgr.gibba032.whopeo (mgr.24625) 10679 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:15:22.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.518347+0000 mgr.gibba032.whopeo (mgr.24625) 10680 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:15:22.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.518665+0000 mgr.gibba032.whopeo (mgr.24625) 10681 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:15:22.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.519432+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.520238+0000 mgr.gibba032.whopeo (mgr.24625) 10682 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:15:22.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.520287+0000 mgr.gibba032.whopeo (mgr.24625) 10683 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:15:22.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: cephadm 2021-03-10T17:15:21.520393+0000 mgr.gibba032.whopeo (mgr.24625) 10684 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:22.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[30791]: audit 2021-03-10T17:15:21.521104+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:22.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.520287+0000 mgr.gibba032.whopeo (mgr.24625) 10683 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:15:22.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: cephadm 2021-03-10T17:15:21.520393+0000 mgr.gibba032.whopeo (mgr.24625) 10684 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:22.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:22 gibba011 bash[31774]: audit 2021-03-10T17:15:21.521104+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:23.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.293507+0000 mgr.gibba032.whopeo (mgr.24625) 10668 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:15:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.295038+0000 mgr.gibba032.whopeo (mgr.24625) 10669 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:15:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cluster 2021-03-10T17:15:21.377035+0000 mgr.gibba032.whopeo (mgr.24625) 10670 : cluster [DBG] pgmap v6083: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.509782+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:15:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.510277+0000 mgr.gibba032.whopeo (mgr.24625) 10671 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:15:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.512175+0000 mgr.gibba032.whopeo (mgr.24625) 10672 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:15:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.513082+0000 mon.a (mon.0) 3558 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:15:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.513390+0000 mgr.gibba032.whopeo (mgr.24625) 10673 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:15:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.513428+0000 mgr.gibba032.whopeo (mgr.24625) 10674 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:15:23.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.514438+0000 mon.a (mon.0) 3559 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.514557+0000 mgr.gibba032.whopeo (mgr.24625) 10675 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:15:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.514602+0000 mgr.gibba032.whopeo (mgr.24625) 10676 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:15:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.515642+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.515759+0000 mgr.gibba032.whopeo (mgr.24625) 10677 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:15:23.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.516874+0000 mon.a (mon.0) 3561 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.516921+0000 mgr.gibba032.whopeo (mgr.24625) 10678 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:15:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.516961+0000 mgr.gibba032.whopeo (mgr.24625) 10679 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:15:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.518347+0000 mgr.gibba032.whopeo (mgr.24625) 10680 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:15:23.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.518665+0000 mgr.gibba032.whopeo (mgr.24625) 10681 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:15:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.519432+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.520238+0000 mgr.gibba032.whopeo (mgr.24625) 10682 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:15:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.520287+0000 mgr.gibba032.whopeo (mgr.24625) 10683 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:15:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: cephadm 2021-03-10T17:15:21.520393+0000 mgr.gibba032.whopeo (mgr.24625) 10684 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:23.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:22 gibba032 bash[24004]: audit 2021-03-10T17:15:21.521104+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:24 gibba011 bash[30791]: cluster 2021-03-10T17:15:23.378297+0000 mgr.gibba032.whopeo (mgr.24625) 10685 : cluster [DBG] pgmap v6084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:24.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:24 gibba011 bash[31774]: cluster 2021-03-10T17:15:23.378297+0000 mgr.gibba032.whopeo (mgr.24625) 10685 : cluster [DBG] pgmap v6084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:24 gibba032 bash[24004]: cluster 2021-03-10T17:15:23.378297+0000 mgr.gibba032.whopeo (mgr.24625) 10685 : cluster [DBG] pgmap v6084: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:26.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:26 gibba011 bash[30791]: cluster 2021-03-10T17:15:25.379796+0000 mgr.gibba032.whopeo (mgr.24625) 10686 : cluster [DBG] pgmap v6085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:26.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:26 gibba011 bash[31774]: cluster 2021-03-10T17:15:25.379796+0000 mgr.gibba032.whopeo (mgr.24625) 10686 : cluster [DBG] pgmap v6085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:26 gibba032 bash[24004]: cluster 2021-03-10T17:15:25.379796+0000 mgr.gibba032.whopeo (mgr.24625) 10686 : cluster [DBG] pgmap v6085: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:28 gibba011 bash[30791]: cluster 2021-03-10T17:15:27.381545+0000 mgr.gibba032.whopeo (mgr.24625) 10687 : cluster [DBG] pgmap v6086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:28 gibba011 bash[31774]: cluster 2021-03-10T17:15:27.381545+0000 mgr.gibba032.whopeo (mgr.24625) 10687 : cluster [DBG] pgmap v6086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:29.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:28 gibba032 bash[24004]: cluster 2021-03-10T17:15:27.381545+0000 mgr.gibba032.whopeo (mgr.24625) 10687 : cluster [DBG] pgmap v6086: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:30 gibba011 bash[30791]: cluster 2021-03-10T17:15:29.383024+0000 mgr.gibba032.whopeo (mgr.24625) 10688 : cluster [DBG] pgmap v6087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:30 gibba011 bash[31774]: cluster 2021-03-10T17:15:29.383024+0000 mgr.gibba032.whopeo (mgr.24625) 10688 : cluster [DBG] pgmap v6087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:30 gibba032 bash[24004]: cluster 2021-03-10T17:15:29.383024+0000 mgr.gibba032.whopeo (mgr.24625) 10688 : cluster [DBG] pgmap v6087: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:31 gibba032 bash[24004]: cluster 2021-03-10T17:15:31.384682+0000 mgr.gibba032.whopeo (mgr.24625) 10689 : cluster [DBG] pgmap v6088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:31 gibba011 bash[30791]: cluster 2021-03-10T17:15:31.384682+0000 mgr.gibba032.whopeo (mgr.24625) 10689 : cluster [DBG] pgmap v6088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:31 gibba011 bash[31774]: cluster 2021-03-10T17:15:31.384682+0000 mgr.gibba032.whopeo (mgr.24625) 10689 : cluster [DBG] pgmap v6088: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:34.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:34 gibba032 bash[24004]: cluster 2021-03-10T17:15:33.386265+0000 mgr.gibba032.whopeo (mgr.24625) 10690 : cluster [DBG] pgmap v6089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:34 gibba011 bash[30791]: cluster 2021-03-10T17:15:33.386265+0000 mgr.gibba032.whopeo (mgr.24625) 10690 : cluster [DBG] pgmap v6089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:34 gibba011 bash[31774]: cluster 2021-03-10T17:15:33.386265+0000 mgr.gibba032.whopeo (mgr.24625) 10690 : cluster [DBG] pgmap v6089: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:36.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:36 gibba032 bash[24004]: cluster 2021-03-10T17:15:35.387765+0000 mgr.gibba032.whopeo (mgr.24625) 10691 : cluster [DBG] pgmap v6090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:36 gibba011 bash[30791]: cluster 2021-03-10T17:15:35.387765+0000 mgr.gibba032.whopeo (mgr.24625) 10691 : cluster [DBG] pgmap v6090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:36 gibba011 bash[31774]: cluster 2021-03-10T17:15:35.387765+0000 mgr.gibba032.whopeo (mgr.24625) 10691 : cluster [DBG] pgmap v6090: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:37.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:37 gibba032 bash[24004]: cephadm 2021-03-10T17:15:36.536127+0000 mgr.gibba032.whopeo (mgr.24625) 10692 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:37.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:37 gibba011 bash[30791]: cephadm 2021-03-10T17:15:36.536127+0000 mgr.gibba032.whopeo (mgr.24625) 10692 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:37.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:37 gibba011 bash[31774]: cephadm 2021-03-10T17:15:36.536127+0000 mgr.gibba032.whopeo (mgr.24625) 10692 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:38.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:38 gibba032 bash[24004]: cluster 2021-03-10T17:15:37.389298+0000 mgr.gibba032.whopeo (mgr.24625) 10693 : cluster [DBG] pgmap v6091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:38 gibba011 bash[30791]: cluster 2021-03-10T17:15:37.389298+0000 mgr.gibba032.whopeo (mgr.24625) 10693 : cluster [DBG] pgmap v6091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:38 gibba011 bash[31774]: cluster 2021-03-10T17:15:37.389298+0000 mgr.gibba032.whopeo (mgr.24625) 10693 : cluster [DBG] pgmap v6091: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:40.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:40 gibba032 bash[24004]: cluster 2021-03-10T17:15:39.390760+0000 mgr.gibba032.whopeo (mgr.24625) 10694 : cluster [DBG] pgmap v6092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:40 gibba011 bash[30791]: cluster 2021-03-10T17:15:39.390760+0000 mgr.gibba032.whopeo (mgr.24625) 10694 : cluster [DBG] pgmap v6092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:40 gibba011 bash[31774]: cluster 2021-03-10T17:15:39.390760+0000 mgr.gibba032.whopeo (mgr.24625) 10694 : cluster [DBG] pgmap v6092: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:41.298 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:15:41.527 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:15:41.528 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:15:41.528 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:15:41.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:15:41.528 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:15:41.529 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:15:41.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:15:41.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:15:41.529 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:15:41.529 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:15:41.530 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:15:41.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:15:41.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:15:41.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:15:41.530 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:15:41.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:15:41.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:15:41.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:15:41.531 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:15:41.532 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:15:41.532 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:15:41.772 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:15:41.772 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:15:41.773 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:15:41.773 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:41.773 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:15:41.774 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:15:41.774 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:41.774 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:15:41.774 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:15:41.775 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:15:41.775 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:15:41.775 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:15:41.776 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:15:41.776 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:15:41.776 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:15:42.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:41 gibba032 bash[24004]: audit 2021-03-10T17:15:41.288762+0000 mgr.gibba032.whopeo (mgr.24625) 10695 : audit [DBG] from='client.51219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:41 gibba032 bash[24004]: cluster 2021-03-10T17:15:41.392397+0000 mgr.gibba032.whopeo (mgr.24625) 10696 : cluster [DBG] pgmap v6093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:41 gibba032 bash[24004]: audit 2021-03-10T17:15:41.514472+0000 mgr.gibba032.whopeo (mgr.24625) 10697 : audit [DBG] from='client.61207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:41 gibba032 bash[24004]: audit 2021-03-10T17:15:41.771827+0000 mon.a (mon.0) 3564 : audit [DBG] from='client.? 172.21.2.111:0/3294055451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[30791]: audit 2021-03-10T17:15:41.288762+0000 mgr.gibba032.whopeo (mgr.24625) 10695 : audit [DBG] from='client.51219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[30791]: cluster 2021-03-10T17:15:41.392397+0000 mgr.gibba032.whopeo (mgr.24625) 10696 : cluster [DBG] pgmap v6093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[30791]: audit 2021-03-10T17:15:41.514472+0000 mgr.gibba032.whopeo (mgr.24625) 10697 : audit [DBG] from='client.61207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[30791]: audit 2021-03-10T17:15:41.771827+0000 mon.a (mon.0) 3564 : audit [DBG] from='client.? 172.21.2.111:0/3294055451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[31774]: audit 2021-03-10T17:15:41.288762+0000 mgr.gibba032.whopeo (mgr.24625) 10695 : audit [DBG] from='client.51219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[31774]: cluster 2021-03-10T17:15:41.392397+0000 mgr.gibba032.whopeo (mgr.24625) 10696 : cluster [DBG] pgmap v6093: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[31774]: audit 2021-03-10T17:15:41.514472+0000 mgr.gibba032.whopeo (mgr.24625) 10697 : audit [DBG] from='client.61207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:15:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:41 gibba011 bash[31774]: audit 2021-03-10T17:15:41.771827+0000 mon.a (mon.0) 3564 : audit [DBG] from='client.? 172.21.2.111:0/3294055451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:15:44.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:44 gibba032 bash[24004]: cluster 2021-03-10T17:15:43.393776+0000 mgr.gibba032.whopeo (mgr.24625) 10698 : cluster [DBG] pgmap v6094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:44 gibba011 bash[30791]: cluster 2021-03-10T17:15:43.393776+0000 mgr.gibba032.whopeo (mgr.24625) 10698 : cluster [DBG] pgmap v6094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:44 gibba011 bash[31774]: cluster 2021-03-10T17:15:43.393776+0000 mgr.gibba032.whopeo (mgr.24625) 10698 : cluster [DBG] pgmap v6094: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:46 gibba032 bash[24004]: cluster 2021-03-10T17:15:45.395108+0000 mgr.gibba032.whopeo (mgr.24625) 10699 : cluster [DBG] pgmap v6095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:46 gibba011 bash[30791]: cluster 2021-03-10T17:15:45.395108+0000 mgr.gibba032.whopeo (mgr.24625) 10699 : cluster [DBG] pgmap v6095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:46.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:46 gibba011 bash[31774]: cluster 2021-03-10T17:15:45.395108+0000 mgr.gibba032.whopeo (mgr.24625) 10699 : cluster [DBG] pgmap v6095: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:48 gibba032 bash[24004]: cluster 2021-03-10T17:15:47.396919+0000 mgr.gibba032.whopeo (mgr.24625) 10700 : cluster [DBG] pgmap v6096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:48 gibba011 bash[30791]: cluster 2021-03-10T17:15:47.396919+0000 mgr.gibba032.whopeo (mgr.24625) 10700 : cluster [DBG] pgmap v6096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:48 gibba011 bash[31774]: cluster 2021-03-10T17:15:47.396919+0000 mgr.gibba032.whopeo (mgr.24625) 10700 : cluster [DBG] pgmap v6096: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:50.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:50 gibba032 bash[24004]: cluster 2021-03-10T17:15:49.398400+0000 mgr.gibba032.whopeo (mgr.24625) 10701 : cluster [DBG] pgmap v6097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:50 gibba011 bash[30791]: cluster 2021-03-10T17:15:49.398400+0000 mgr.gibba032.whopeo (mgr.24625) 10701 : cluster [DBG] pgmap v6097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:50 gibba011 bash[31774]: cluster 2021-03-10T17:15:49.398400+0000 mgr.gibba032.whopeo (mgr.24625) 10701 : cluster [DBG] pgmap v6097: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:51 gibba032 bash[24004]: cluster 2021-03-10T17:15:51.400197+0000 mgr.gibba032.whopeo (mgr.24625) 10702 : cluster [DBG] pgmap v6098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:51 gibba032 bash[24004]: cephadm 2021-03-10T17:15:51.552163+0000 mgr.gibba032.whopeo (mgr.24625) 10703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:51 gibba011 bash[30791]: cluster 2021-03-10T17:15:51.400197+0000 mgr.gibba032.whopeo (mgr.24625) 10702 : cluster [DBG] pgmap v6098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:51 gibba011 bash[30791]: cephadm 2021-03-10T17:15:51.552163+0000 mgr.gibba032.whopeo (mgr.24625) 10703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:51 gibba011 bash[31774]: cluster 2021-03-10T17:15:51.400197+0000 mgr.gibba032.whopeo (mgr.24625) 10702 : cluster [DBG] pgmap v6098: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:51 gibba011 bash[31774]: cephadm 2021-03-10T17:15:51.552163+0000 mgr.gibba032.whopeo (mgr.24625) 10703 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:15:54.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:54 gibba032 bash[24004]: cluster 2021-03-10T17:15:53.401502+0000 mgr.gibba032.whopeo (mgr.24625) 10704 : cluster [DBG] pgmap v6099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:54.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:54 gibba011 bash[30791]: cluster 2021-03-10T17:15:53.401502+0000 mgr.gibba032.whopeo (mgr.24625) 10704 : cluster [DBG] pgmap v6099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:54.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:54 gibba011 bash[31774]: cluster 2021-03-10T17:15:53.401502+0000 mgr.gibba032.whopeo (mgr.24625) 10704 : cluster [DBG] pgmap v6099: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:56 gibba032 bash[24004]: cluster 2021-03-10T17:15:55.402958+0000 mgr.gibba032.whopeo (mgr.24625) 10705 : cluster [DBG] pgmap v6100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:56 gibba011 bash[30791]: cluster 2021-03-10T17:15:55.402958+0000 mgr.gibba032.whopeo (mgr.24625) 10705 : cluster [DBG] pgmap v6100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:56.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:56 gibba011 bash[31774]: cluster 2021-03-10T17:15:55.402958+0000 mgr.gibba032.whopeo (mgr.24625) 10705 : cluster [DBG] pgmap v6100: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:57.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:57 gibba032 bash[24004]: audit 2021-03-10T17:15:57.408231+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:15:57.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:57 gibba032 bash[24004]: audit 2021-03-10T17:15:57.408840+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:15:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:57 gibba011 bash[30791]: audit 2021-03-10T17:15:57.408231+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:15:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:57 gibba011 bash[30791]: audit 2021-03-10T17:15:57.408840+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:15:57.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:57 gibba011 bash[31774]: audit 2021-03-10T17:15:57.408231+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:15:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:57 gibba011 bash[31774]: audit 2021-03-10T17:15:57.408840+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:15:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:58 gibba032 bash[24004]: cluster 2021-03-10T17:15:57.403520+0000 mgr.gibba032.whopeo (mgr.24625) 10706 : cluster [DBG] pgmap v6101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:58.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:58 gibba032 bash[24004]: audit 2021-03-10T17:15:57.576321+0000 mon.a (mon.0) 3567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:15:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:58 gibba032 bash[24004]: audit 2021-03-10T17:15:57.577106+0000 mon.a (mon.0) 3568 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:15:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:58 gibba032 bash[24004]: audit 2021-03-10T17:15:57.577846+0000 mon.a (mon.0) 3569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:15:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:15:58 gibba032 bash[24004]: audit 2021-03-10T17:15:57.578519+0000 mon.a (mon.0) 3570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:15:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[30791]: cluster 2021-03-10T17:15:57.403520+0000 mgr.gibba032.whopeo (mgr.24625) 10706 : cluster [DBG] pgmap v6101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[30791]: audit 2021-03-10T17:15:57.576321+0000 mon.a (mon.0) 3567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:15:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[30791]: audit 2021-03-10T17:15:57.577106+0000 mon.a (mon.0) 3568 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:15:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[30791]: audit 2021-03-10T17:15:57.577846+0000 mon.a (mon.0) 3569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:15:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[30791]: audit 2021-03-10T17:15:57.578519+0000 mon.a (mon.0) 3570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:15:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[31774]: cluster 2021-03-10T17:15:57.403520+0000 mgr.gibba032.whopeo (mgr.24625) 10706 : cluster [DBG] pgmap v6101: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:15:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[31774]: audit 2021-03-10T17:15:57.576321+0000 mon.a (mon.0) 3567 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:15:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[31774]: audit 2021-03-10T17:15:57.577106+0000 mon.a (mon.0) 3568 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:15:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[31774]: audit 2021-03-10T17:15:57.577846+0000 mon.a (mon.0) 3569 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:15:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:15:58 gibba011 bash[31774]: audit 2021-03-10T17:15:57.578519+0000 mon.a (mon.0) 3570 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:16:00.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:00 gibba032 bash[24004]: cluster 2021-03-10T17:15:59.405012+0000 mgr.gibba032.whopeo (mgr.24625) 10707 : cluster [DBG] pgmap v6102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:00 gibba011 bash[30791]: cluster 2021-03-10T17:15:59.405012+0000 mgr.gibba032.whopeo (mgr.24625) 10707 : cluster [DBG] pgmap v6102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:00 gibba011 bash[31774]: cluster 2021-03-10T17:15:59.405012+0000 mgr.gibba032.whopeo (mgr.24625) 10707 : cluster [DBG] pgmap v6102: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:02.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:01 gibba032 bash[24004]: cluster 2021-03-10T17:16:01.406765+0000 mgr.gibba032.whopeo (mgr.24625) 10708 : cluster [DBG] pgmap v6103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:01 gibba011 bash[30791]: cluster 2021-03-10T17:16:01.406765+0000 mgr.gibba032.whopeo (mgr.24625) 10708 : cluster [DBG] pgmap v6103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:01 gibba011 bash[31774]: cluster 2021-03-10T17:16:01.406765+0000 mgr.gibba032.whopeo (mgr.24625) 10708 : cluster [DBG] pgmap v6103: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:04 gibba032 bash[24004]: cluster 2021-03-10T17:16:03.408410+0000 mgr.gibba032.whopeo (mgr.24625) 10709 : cluster [DBG] pgmap v6104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:04 gibba011 bash[30791]: cluster 2021-03-10T17:16:03.408410+0000 mgr.gibba032.whopeo (mgr.24625) 10709 : cluster [DBG] pgmap v6104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:04 gibba011 bash[31774]: cluster 2021-03-10T17:16:03.408410+0000 mgr.gibba032.whopeo (mgr.24625) 10709 : cluster [DBG] pgmap v6104: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:06.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:06 gibba032 bash[24004]: cluster 2021-03-10T17:16:05.409757+0000 mgr.gibba032.whopeo (mgr.24625) 10710 : cluster [DBG] pgmap v6105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:06 gibba011 bash[30791]: cluster 2021-03-10T17:16:05.409757+0000 mgr.gibba032.whopeo (mgr.24625) 10710 : cluster [DBG] pgmap v6105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:06 gibba011 bash[31774]: cluster 2021-03-10T17:16:05.409757+0000 mgr.gibba032.whopeo (mgr.24625) 10710 : cluster [DBG] pgmap v6105: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:07 gibba032 bash[24004]: cephadm 2021-03-10T17:16:06.567901+0000 mgr.gibba032.whopeo (mgr.24625) 10711 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:07 gibba011 bash[30791]: cephadm 2021-03-10T17:16:06.567901+0000 mgr.gibba032.whopeo (mgr.24625) 10711 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:07.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:07 gibba011 bash[31774]: cephadm 2021-03-10T17:16:06.567901+0000 mgr.gibba032.whopeo (mgr.24625) 10711 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:08.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:08 gibba032 bash[24004]: cluster 2021-03-10T17:16:07.411566+0000 mgr.gibba032.whopeo (mgr.24625) 10712 : cluster [DBG] pgmap v6106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:08 gibba011 bash[30791]: cluster 2021-03-10T17:16:07.411566+0000 mgr.gibba032.whopeo (mgr.24625) 10712 : cluster [DBG] pgmap v6106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:08.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:08 gibba011 bash[31774]: cluster 2021-03-10T17:16:07.411566+0000 mgr.gibba032.whopeo (mgr.24625) 10712 : cluster [DBG] pgmap v6106: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:10 gibba032 bash[24004]: cluster 2021-03-10T17:16:09.413088+0000 mgr.gibba032.whopeo (mgr.24625) 10713 : cluster [DBG] pgmap v6107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:10.949 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:10 gibba011 bash[30791]: cluster 2021-03-10T17:16:09.413088+0000 mgr.gibba032.whopeo (mgr.24625) 10713 : cluster [DBG] pgmap v6107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:10.950 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:10 gibba011 bash[31774]: cluster 2021-03-10T17:16:09.413088+0000 mgr.gibba032.whopeo (mgr.24625) 10713 : cluster [DBG] pgmap v6107: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:12.014 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:16:12.241 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:16:12.242 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:16:12.242 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:16:12.242 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:16:12.242 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:16:12.242 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:16:12.243 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:16:12.243 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:16:12.243 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:16:12.243 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:16:12.244 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:16:12.244 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:16:12.244 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:16:12.244 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:16:12.244 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:16:12.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:16:12.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:16:12.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:16:12.245 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:16:12.245 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:16:12.246 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:16:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:11 gibba032 bash[24004]: cluster 2021-03-10T17:16:11.414700+0000 mgr.gibba032.whopeo (mgr.24625) 10714 : cluster [DBG] pgmap v6108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:12.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:11 gibba011 bash[30791]: cluster 2021-03-10T17:16:11.414700+0000 mgr.gibba032.whopeo (mgr.24625) 10714 : cluster [DBG] pgmap v6108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:11 gibba011 bash[31774]: cluster 2021-03-10T17:16:11.414700+0000 mgr.gibba032.whopeo (mgr.24625) 10714 : cluster [DBG] pgmap v6108: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:16:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:12 gibba032 bash[24004]: audit 2021-03-10T17:16:12.003845+0000 mgr.gibba032.whopeo (mgr.24625) 10715 : audit [DBG] from='client.61219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:12 gibba032 bash[24004]: audit 2021-03-10T17:16:12.227255+0000 mgr.gibba032.whopeo (mgr.24625) 10716 : audit [DBG] from='client.51243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:12 gibba032 bash[24004]: audit 2021-03-10T17:16:12.490457+0000 mon.c (mon.1) 206 : audit [DBG] from='client.? 172.21.2.111:0/3110310867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[30791]: audit 2021-03-10T17:16:12.003845+0000 mgr.gibba032.whopeo (mgr.24625) 10715 : audit [DBG] from='client.61219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[30791]: audit 2021-03-10T17:16:12.227255+0000 mgr.gibba032.whopeo (mgr.24625) 10716 : audit [DBG] from='client.51243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[30791]: audit 2021-03-10T17:16:12.490457+0000 mon.c (mon.1) 206 : audit [DBG] from='client.? 172.21.2.111:0/3110310867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[31774]: audit 2021-03-10T17:16:12.003845+0000 mgr.gibba032.whopeo (mgr.24625) 10715 : audit [DBG] from='client.61219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[31774]: audit 2021-03-10T17:16:12.227255+0000 mgr.gibba032.whopeo (mgr.24625) 10716 : audit [DBG] from='client.51243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:13.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:12 gibba011 bash[31774]: audit 2021-03-10T17:16:12.490457+0000 mon.c (mon.1) 206 : audit [DBG] from='client.? 172.21.2.111:0/3110310867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:14.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:13 gibba032 bash[24004]: cluster 2021-03-10T17:16:13.416353+0000 mgr.gibba032.whopeo (mgr.24625) 10717 : cluster [DBG] pgmap v6109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:13 gibba011 bash[30791]: cluster 2021-03-10T17:16:13.416353+0000 mgr.gibba032.whopeo (mgr.24625) 10717 : cluster [DBG] pgmap v6109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:13 gibba011 bash[31774]: cluster 2021-03-10T17:16:13.416353+0000 mgr.gibba032.whopeo (mgr.24625) 10717 : cluster [DBG] pgmap v6109: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:16 gibba032 bash[24004]: cluster 2021-03-10T17:16:15.417990+0000 mgr.gibba032.whopeo (mgr.24625) 10718 : cluster [DBG] pgmap v6110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:16 gibba011 bash[30791]: cluster 2021-03-10T17:16:15.417990+0000 mgr.gibba032.whopeo (mgr.24625) 10718 : cluster [DBG] pgmap v6110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:16 gibba011 bash[31774]: cluster 2021-03-10T17:16:15.417990+0000 mgr.gibba032.whopeo (mgr.24625) 10718 : cluster [DBG] pgmap v6110: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:18 gibba032 bash[24004]: cluster 2021-03-10T17:16:17.419731+0000 mgr.gibba032.whopeo (mgr.24625) 10719 : cluster [DBG] pgmap v6111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:18.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:18 gibba011 bash[30791]: cluster 2021-03-10T17:16:17.419731+0000 mgr.gibba032.whopeo (mgr.24625) 10719 : cluster [DBG] pgmap v6111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:18.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:18 gibba011 bash[31774]: cluster 2021-03-10T17:16:17.419731+0000 mgr.gibba032.whopeo (mgr.24625) 10719 : cluster [DBG] pgmap v6111: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:20.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:20 gibba032 bash[24004]: cluster 2021-03-10T17:16:19.421127+0000 mgr.gibba032.whopeo (mgr.24625) 10720 : cluster [DBG] pgmap v6112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:20 gibba011 bash[30791]: cluster 2021-03-10T17:16:19.421127+0000 mgr.gibba032.whopeo (mgr.24625) 10720 : cluster [DBG] pgmap v6112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:20 gibba011 bash[31774]: cluster 2021-03-10T17:16:19.421127+0000 mgr.gibba032.whopeo (mgr.24625) 10720 : cluster [DBG] pgmap v6112: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:23.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cluster 2021-03-10T17:16:21.422858+0000 mgr.gibba032.whopeo (mgr.24625) 10721 : cluster [DBG] pgmap v6113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.584632+0000 mgr.gibba032.whopeo (mgr.24625) 10722 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:16:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.586216+0000 mgr.gibba032.whopeo (mgr.24625) 10723 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:16:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.825127+0000 mon.a (mon.0) 3571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:16:23.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.826020+0000 mgr.gibba032.whopeo (mgr.24625) 10724 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:16:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.827972+0000 mgr.gibba032.whopeo (mgr.24625) 10725 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:16:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.828670+0000 mon.a (mon.0) 3572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:16:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.829283+0000 mgr.gibba032.whopeo (mgr.24625) 10726 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:16:23.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.829328+0000 mgr.gibba032.whopeo (mgr.24625) 10727 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:16:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.830211+0000 mon.a (mon.0) 3573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.830637+0000 mgr.gibba032.whopeo (mgr.24625) 10728 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:16:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.830686+0000 mgr.gibba032.whopeo (mgr.24625) 10729 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:16:23.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.831522+0000 mon.a (mon.0) 3574 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.831991+0000 mgr.gibba032.whopeo (mgr.24625) 10730 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:16:23.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.832830+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.832959+0000 mgr.gibba032.whopeo (mgr.24625) 10731 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:16:23.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.833006+0000 mgr.gibba032.whopeo (mgr.24625) 10732 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:16:23.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.834426+0000 mgr.gibba032.whopeo (mgr.24625) 10733 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:16:23.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.834872+0000 mgr.gibba032.whopeo (mgr.24625) 10734 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:16:23.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.835302+0000 mon.a (mon.0) 3576 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:22 gibba032 bash[24004]: audit 2021-03-10T17:16:21.837118+0000 mon.a (mon.0) 3577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cluster 2021-03-10T17:16:21.422858+0000 mgr.gibba032.whopeo (mgr.24625) 10721 : cluster [DBG] pgmap v6113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:23.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.584632+0000 mgr.gibba032.whopeo (mgr.24625) 10722 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:16:23.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.586216+0000 mgr.gibba032.whopeo (mgr.24625) 10723 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:16:23.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.825127+0000 mon.a (mon.0) 3571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:16:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.826020+0000 mgr.gibba032.whopeo (mgr.24625) 10724 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:16:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.827972+0000 mgr.gibba032.whopeo (mgr.24625) 10725 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:16:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.828670+0000 mon.a (mon.0) 3572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:16:23.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.829283+0000 mgr.gibba032.whopeo (mgr.24625) 10726 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:16:23.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.829328+0000 mgr.gibba032.whopeo (mgr.24625) 10727 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:16:23.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.830211+0000 mon.a (mon.0) 3573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.830637+0000 mgr.gibba032.whopeo (mgr.24625) 10728 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:16:23.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.830686+0000 mgr.gibba032.whopeo (mgr.24625) 10729 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:16:23.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.831522+0000 mon.a (mon.0) 3574 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.831991+0000 mgr.gibba032.whopeo (mgr.24625) 10730 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:16:23.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.832830+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.832959+0000 mgr.gibba032.whopeo (mgr.24625) 10731 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:16:23.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.833006+0000 mgr.gibba032.whopeo (mgr.24625) 10732 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:16:23.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.834426+0000 mgr.gibba032.whopeo (mgr.24625) 10733 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:16:23.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.834872+0000 mgr.gibba032.whopeo (mgr.24625) 10734 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:16:23.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.835302+0000 mon.a (mon.0) 3576 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[30791]: audit 2021-03-10T17:16:21.837118+0000 mon.a (mon.0) 3577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cluster 2021-03-10T17:16:21.422858+0000 mgr.gibba032.whopeo (mgr.24625) 10721 : cluster [DBG] pgmap v6113: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:23.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.584632+0000 mgr.gibba032.whopeo (mgr.24625) 10722 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:16:23.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.586216+0000 mgr.gibba032.whopeo (mgr.24625) 10723 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:16:23.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.825127+0000 mon.a (mon.0) 3571 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:16:23.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.826020+0000 mgr.gibba032.whopeo (mgr.24625) 10724 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:16:23.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.827972+0000 mgr.gibba032.whopeo (mgr.24625) 10725 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:16:23.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.828670+0000 mon.a (mon.0) 3572 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:16:23.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.829283+0000 mgr.gibba032.whopeo (mgr.24625) 10726 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:16:23.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.829328+0000 mgr.gibba032.whopeo (mgr.24625) 10727 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:16:23.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.830211+0000 mon.a (mon.0) 3573 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.830637+0000 mgr.gibba032.whopeo (mgr.24625) 10728 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:16:23.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.830686+0000 mgr.gibba032.whopeo (mgr.24625) 10729 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:16:23.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.831522+0000 mon.a (mon.0) 3574 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.831991+0000 mgr.gibba032.whopeo (mgr.24625) 10730 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:16:23.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.832830+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.832959+0000 mgr.gibba032.whopeo (mgr.24625) 10731 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:16:23.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.833006+0000 mgr.gibba032.whopeo (mgr.24625) 10732 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:16:23.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.834426+0000 mgr.gibba032.whopeo (mgr.24625) 10733 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:16:23.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.834872+0000 mgr.gibba032.whopeo (mgr.24625) 10734 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:16:23.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.835302+0000 mon.a (mon.0) 3576 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:23.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:22 gibba011 bash[31774]: audit 2021-03-10T17:16:21.837118+0000 mon.a (mon.0) 3577 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:23 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.836216+0000 mgr.gibba032.whopeo (mgr.24625) 10735 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:16:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:23 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.836272+0000 mgr.gibba032.whopeo (mgr.24625) 10736 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:16:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:23 gibba032 bash[24004]: cephadm 2021-03-10T17:16:21.836418+0000 mgr.gibba032.whopeo (mgr.24625) 10737 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.836216+0000 mgr.gibba032.whopeo (mgr.24625) 10735 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:16:24.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.836272+0000 mgr.gibba032.whopeo (mgr.24625) 10736 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:16:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[30791]: cephadm 2021-03-10T17:16:21.836418+0000 mgr.gibba032.whopeo (mgr.24625) 10737 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:24.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.836216+0000 mgr.gibba032.whopeo (mgr.24625) 10735 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:16:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.836272+0000 mgr.gibba032.whopeo (mgr.24625) 10736 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:16:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:23 gibba011 bash[31774]: cephadm 2021-03-10T17:16:21.836418+0000 mgr.gibba032.whopeo (mgr.24625) 10737 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:24 gibba032 bash[24004]: cluster 2021-03-10T17:16:23.424369+0000 mgr.gibba032.whopeo (mgr.24625) 10738 : cluster [DBG] pgmap v6114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:24 gibba011 bash[30791]: cluster 2021-03-10T17:16:23.424369+0000 mgr.gibba032.whopeo (mgr.24625) 10738 : cluster [DBG] pgmap v6114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:24 gibba011 bash[31774]: cluster 2021-03-10T17:16:23.424369+0000 mgr.gibba032.whopeo (mgr.24625) 10738 : cluster [DBG] pgmap v6114: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:25 gibba032 bash[24004]: cluster 2021-03-10T17:16:25.425712+0000 mgr.gibba032.whopeo (mgr.24625) 10739 : cluster [DBG] pgmap v6115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:25 gibba011 bash[30791]: cluster 2021-03-10T17:16:25.425712+0000 mgr.gibba032.whopeo (mgr.24625) 10739 : cluster [DBG] pgmap v6115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:25 gibba011 bash[31774]: cluster 2021-03-10T17:16:25.425712+0000 mgr.gibba032.whopeo (mgr.24625) 10739 : cluster [DBG] pgmap v6115: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:28 gibba032 bash[24004]: cluster 2021-03-10T17:16:27.427523+0000 mgr.gibba032.whopeo (mgr.24625) 10740 : cluster [DBG] pgmap v6116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:28 gibba011 bash[30791]: cluster 2021-03-10T17:16:27.427523+0000 mgr.gibba032.whopeo (mgr.24625) 10740 : cluster [DBG] pgmap v6116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:28 gibba011 bash[31774]: cluster 2021-03-10T17:16:27.427523+0000 mgr.gibba032.whopeo (mgr.24625) 10740 : cluster [DBG] pgmap v6116: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:30.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:30 gibba032 bash[24004]: cluster 2021-03-10T17:16:29.428996+0000 mgr.gibba032.whopeo (mgr.24625) 10741 : cluster [DBG] pgmap v6117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:30 gibba011 bash[30791]: cluster 2021-03-10T17:16:29.428996+0000 mgr.gibba032.whopeo (mgr.24625) 10741 : cluster [DBG] pgmap v6117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:30 gibba011 bash[31774]: cluster 2021-03-10T17:16:29.428996+0000 mgr.gibba032.whopeo (mgr.24625) 10741 : cluster [DBG] pgmap v6117: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:32.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:31 gibba032 bash[24004]: cluster 2021-03-10T17:16:31.430608+0000 mgr.gibba032.whopeo (mgr.24625) 10742 : cluster [DBG] pgmap v6118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:31 gibba011 bash[30791]: cluster 2021-03-10T17:16:31.430608+0000 mgr.gibba032.whopeo (mgr.24625) 10742 : cluster [DBG] pgmap v6118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:31 gibba011 bash[31774]: cluster 2021-03-10T17:16:31.430608+0000 mgr.gibba032.whopeo (mgr.24625) 10742 : cluster [DBG] pgmap v6118: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:34 gibba032 bash[24004]: cluster 2021-03-10T17:16:33.432309+0000 mgr.gibba032.whopeo (mgr.24625) 10743 : cluster [DBG] pgmap v6119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:34 gibba011 bash[30791]: cluster 2021-03-10T17:16:33.432309+0000 mgr.gibba032.whopeo (mgr.24625) 10743 : cluster [DBG] pgmap v6119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:34 gibba011 bash[31774]: cluster 2021-03-10T17:16:33.432309+0000 mgr.gibba032.whopeo (mgr.24625) 10743 : cluster [DBG] pgmap v6119: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:36 gibba032 bash[24004]: cluster 2021-03-10T17:16:35.433840+0000 mgr.gibba032.whopeo (mgr.24625) 10744 : cluster [DBG] pgmap v6120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:36 gibba011 bash[30791]: cluster 2021-03-10T17:16:35.433840+0000 mgr.gibba032.whopeo (mgr.24625) 10744 : cluster [DBG] pgmap v6120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:36 gibba011 bash[31774]: cluster 2021-03-10T17:16:35.433840+0000 mgr.gibba032.whopeo (mgr.24625) 10744 : cluster [DBG] pgmap v6120: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:37.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:37 gibba011 bash[30791]: cephadm 2021-03-10T17:16:36.839960+0000 mgr.gibba032.whopeo (mgr.24625) 10745 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:37.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:37 gibba011 bash[31774]: cephadm 2021-03-10T17:16:36.839960+0000 mgr.gibba032.whopeo (mgr.24625) 10745 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:38.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:37 gibba032 bash[24004]: cephadm 2021-03-10T17:16:36.839960+0000 mgr.gibba032.whopeo (mgr.24625) 10745 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:38 gibba011 bash[30791]: cluster 2021-03-10T17:16:37.435574+0000 mgr.gibba032.whopeo (mgr.24625) 10746 : cluster [DBG] pgmap v6121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:38.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:38 gibba011 bash[31774]: cluster 2021-03-10T17:16:37.435574+0000 mgr.gibba032.whopeo (mgr.24625) 10746 : cluster [DBG] pgmap v6121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:39.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:38 gibba032 bash[24004]: cluster 2021-03-10T17:16:37.435574+0000 mgr.gibba032.whopeo (mgr.24625) 10746 : cluster [DBG] pgmap v6121: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:40.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:40 gibba011 bash[30791]: cluster 2021-03-10T17:16:39.436943+0000 mgr.gibba032.whopeo (mgr.24625) 10747 : cluster [DBG] pgmap v6122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:40.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:40 gibba011 bash[31774]: cluster 2021-03-10T17:16:39.436943+0000 mgr.gibba032.whopeo (mgr.24625) 10747 : cluster [DBG] pgmap v6122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:40 gibba032 bash[24004]: cluster 2021-03-10T17:16:39.436943+0000 mgr.gibba032.whopeo (mgr.24625) 10747 : cluster [DBG] pgmap v6122: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:41 gibba032 bash[24004]: cluster 2021-03-10T17:16:41.438565+0000 mgr.gibba032.whopeo (mgr.24625) 10748 : cluster [DBG] pgmap v6123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:41 gibba011 bash[30791]: cluster 2021-03-10T17:16:41.438565+0000 mgr.gibba032.whopeo (mgr.24625) 10748 : cluster [DBG] pgmap v6123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:41 gibba011 bash[31774]: cluster 2021-03-10T17:16:41.438565+0000 mgr.gibba032.whopeo (mgr.24625) 10748 : cluster [DBG] pgmap v6123: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:42.734 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:16:42.964 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:16:42.964 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:16:42.965 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:16:42.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:16:42.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:16:42.965 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:16:42.965 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:16:42.966 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:16:42.966 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:16:42.966 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:16:42.966 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:16:42.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:16:42.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:16:42.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:16:42.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:16:42.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:16:42.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:16:42.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:16:42.968 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:16:42.968 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:16:42.968 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:16:43.220 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:16:43.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:16:43.220 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:16:43.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:43.221 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:16:43.221 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:16:43.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:43.222 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:16:43.222 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:16:43.222 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:16:43.222 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:16:43.223 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:16:43.223 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:16:43.223 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:16:43.223 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:16:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:42 gibba032 bash[24004]: audit 2021-03-10T17:16:42.723908+0000 mgr.gibba032.whopeo (mgr.24625) 10749 : audit [DBG] from='client.51255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:42 gibba011 bash[30791]: audit 2021-03-10T17:16:42.723908+0000 mgr.gibba032.whopeo (mgr.24625) 10749 : audit [DBG] from='client.51255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:42 gibba011 bash[31774]: audit 2021-03-10T17:16:42.723908+0000 mgr.gibba032.whopeo (mgr.24625) 10749 : audit [DBG] from='client.51255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:44.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:43 gibba032 bash[24004]: audit 2021-03-10T17:16:42.949947+0000 mgr.gibba032.whopeo (mgr.24625) 10750 : audit [DBG] from='client.51261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:44.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:43 gibba032 bash[24004]: audit 2021-03-10T17:16:43.220107+0000 mon.a (mon.0) 3578 : audit [DBG] from='client.? 172.21.2.111:0/1115600929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:44.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:43 gibba032 bash[24004]: cluster 2021-03-10T17:16:43.440227+0000 mgr.gibba032.whopeo (mgr.24625) 10751 : cluster [DBG] pgmap v6124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:44.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[30791]: audit 2021-03-10T17:16:42.949947+0000 mgr.gibba032.whopeo (mgr.24625) 10750 : audit [DBG] from='client.51261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[30791]: audit 2021-03-10T17:16:43.220107+0000 mon.a (mon.0) 3578 : audit [DBG] from='client.? 172.21.2.111:0/1115600929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[30791]: cluster 2021-03-10T17:16:43.440227+0000 mgr.gibba032.whopeo (mgr.24625) 10751 : cluster [DBG] pgmap v6124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[31774]: audit 2021-03-10T17:16:42.949947+0000 mgr.gibba032.whopeo (mgr.24625) 10750 : audit [DBG] from='client.51261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:16:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[31774]: audit 2021-03-10T17:16:43.220107+0000 mon.a (mon.0) 3578 : audit [DBG] from='client.? 172.21.2.111:0/1115600929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:16:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:43 gibba011 bash[31774]: cluster 2021-03-10T17:16:43.440227+0000 mgr.gibba032.whopeo (mgr.24625) 10751 : cluster [DBG] pgmap v6124: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:46.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:46 gibba032 bash[24004]: cluster 2021-03-10T17:16:45.441675+0000 mgr.gibba032.whopeo (mgr.24625) 10752 : cluster [DBG] pgmap v6125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:46 gibba011 bash[30791]: cluster 2021-03-10T17:16:45.441675+0000 mgr.gibba032.whopeo (mgr.24625) 10752 : cluster [DBG] pgmap v6125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:46 gibba011 bash[31774]: cluster 2021-03-10T17:16:45.441675+0000 mgr.gibba032.whopeo (mgr.24625) 10752 : cluster [DBG] pgmap v6125: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:48 gibba011 bash[30791]: cluster 2021-03-10T17:16:47.443298+0000 mgr.gibba032.whopeo (mgr.24625) 10753 : cluster [DBG] pgmap v6126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:48 gibba011 bash[31774]: cluster 2021-03-10T17:16:47.443298+0000 mgr.gibba032.whopeo (mgr.24625) 10753 : cluster [DBG] pgmap v6126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:49.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:48 gibba032 bash[24004]: cluster 2021-03-10T17:16:47.443298+0000 mgr.gibba032.whopeo (mgr.24625) 10753 : cluster [DBG] pgmap v6126: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:50.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:50 gibba011 bash[30791]: cluster 2021-03-10T17:16:49.444812+0000 mgr.gibba032.whopeo (mgr.24625) 10754 : cluster [DBG] pgmap v6127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:50.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:50 gibba011 bash[31774]: cluster 2021-03-10T17:16:49.444812+0000 mgr.gibba032.whopeo (mgr.24625) 10754 : cluster [DBG] pgmap v6127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:50 gibba032 bash[24004]: cluster 2021-03-10T17:16:49.444812+0000 mgr.gibba032.whopeo (mgr.24625) 10754 : cluster [DBG] pgmap v6127: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:51 gibba032 bash[24004]: cluster 2021-03-10T17:16:51.446332+0000 mgr.gibba032.whopeo (mgr.24625) 10755 : cluster [DBG] pgmap v6128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:51 gibba011 bash[30791]: cluster 2021-03-10T17:16:51.446332+0000 mgr.gibba032.whopeo (mgr.24625) 10755 : cluster [DBG] pgmap v6128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:51 gibba011 bash[31774]: cluster 2021-03-10T17:16:51.446332+0000 mgr.gibba032.whopeo (mgr.24625) 10755 : cluster [DBG] pgmap v6128: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:52 gibba032 bash[24004]: cephadm 2021-03-10T17:16:51.855982+0000 mgr.gibba032.whopeo (mgr.24625) 10756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:52 gibba011 bash[30791]: cephadm 2021-03-10T17:16:51.855982+0000 mgr.gibba032.whopeo (mgr.24625) 10756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:52 gibba011 bash[31774]: cephadm 2021-03-10T17:16:51.855982+0000 mgr.gibba032.whopeo (mgr.24625) 10756 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:16:54.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:53 gibba032 bash[24004]: cluster 2021-03-10T17:16:53.447816+0000 mgr.gibba032.whopeo (mgr.24625) 10757 : cluster [DBG] pgmap v6129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:53 gibba011 bash[30791]: cluster 2021-03-10T17:16:53.447816+0000 mgr.gibba032.whopeo (mgr.24625) 10757 : cluster [DBG] pgmap v6129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:54.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:53 gibba011 bash[31774]: cluster 2021-03-10T17:16:53.447816+0000 mgr.gibba032.whopeo (mgr.24625) 10757 : cluster [DBG] pgmap v6129: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:56 gibba011 bash[30791]: cluster 2021-03-10T17:16:55.449166+0000 mgr.gibba032.whopeo (mgr.24625) 10758 : cluster [DBG] pgmap v6130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:56 gibba011 bash[31774]: cluster 2021-03-10T17:16:55.449166+0000 mgr.gibba032.whopeo (mgr.24625) 10758 : cluster [DBG] pgmap v6130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:57.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:56 gibba032 bash[24004]: cluster 2021-03-10T17:16:55.449166+0000 mgr.gibba032.whopeo (mgr.24625) 10758 : cluster [DBG] pgmap v6130: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:57 gibba011 bash[30791]: audit 2021-03-10T17:16:57.414302+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:16:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:57 gibba011 bash[30791]: audit 2021-03-10T17:16:57.414980+0000 mon.a (mon.0) 3580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:16:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:57 gibba011 bash[31774]: audit 2021-03-10T17:16:57.414302+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:16:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:57 gibba011 bash[31774]: audit 2021-03-10T17:16:57.414980+0000 mon.a (mon.0) 3580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:16:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:57 gibba032 bash[24004]: audit 2021-03-10T17:16:57.414302+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:16:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:57 gibba032 bash[24004]: audit 2021-03-10T17:16:57.414980+0000 mon.a (mon.0) 3580 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:16:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[30791]: cluster 2021-03-10T17:16:57.450101+0000 mgr.gibba032.whopeo (mgr.24625) 10759 : cluster [DBG] pgmap v6131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[30791]: audit 2021-03-10T17:16:57.582994+0000 mon.a (mon.0) 3581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:16:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[30791]: audit 2021-03-10T17:16:57.583426+0000 mon.a (mon.0) 3582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:16:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[30791]: audit 2021-03-10T17:16:57.583790+0000 mon.a (mon.0) 3583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:16:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[30791]: audit 2021-03-10T17:16:57.584193+0000 mon.a (mon.0) 3584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:16:58.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[31774]: cluster 2021-03-10T17:16:57.450101+0000 mgr.gibba032.whopeo (mgr.24625) 10759 : cluster [DBG] pgmap v6131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[31774]: audit 2021-03-10T17:16:57.582994+0000 mon.a (mon.0) 3581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:16:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[31774]: audit 2021-03-10T17:16:57.583426+0000 mon.a (mon.0) 3582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:16:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[31774]: audit 2021-03-10T17:16:57.583790+0000 mon.a (mon.0) 3583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:16:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:16:58 gibba011 bash[31774]: audit 2021-03-10T17:16:57.584193+0000 mon.a (mon.0) 3584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:16:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:58 gibba032 bash[24004]: cluster 2021-03-10T17:16:57.450101+0000 mgr.gibba032.whopeo (mgr.24625) 10759 : cluster [DBG] pgmap v6131: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:16:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:58 gibba032 bash[24004]: audit 2021-03-10T17:16:57.582994+0000 mon.a (mon.0) 3581 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:16:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:58 gibba032 bash[24004]: audit 2021-03-10T17:16:57.583426+0000 mon.a (mon.0) 3582 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:16:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:58 gibba032 bash[24004]: audit 2021-03-10T17:16:57.583790+0000 mon.a (mon.0) 3583 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:16:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:16:58 gibba032 bash[24004]: audit 2021-03-10T17:16:57.584193+0000 mon.a (mon.0) 3584 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:17:00.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:00 gibba011 bash[30791]: cluster 2021-03-10T17:16:59.451491+0000 mgr.gibba032.whopeo (mgr.24625) 10760 : cluster [DBG] pgmap v6132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:00.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:00 gibba011 bash[31774]: cluster 2021-03-10T17:16:59.451491+0000 mgr.gibba032.whopeo (mgr.24625) 10760 : cluster [DBG] pgmap v6132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:00 gibba032 bash[24004]: cluster 2021-03-10T17:16:59.451491+0000 mgr.gibba032.whopeo (mgr.24625) 10760 : cluster [DBG] pgmap v6132: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:02.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:01 gibba032 bash[24004]: cluster 2021-03-10T17:17:01.453188+0000 mgr.gibba032.whopeo (mgr.24625) 10761 : cluster [DBG] pgmap v6133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:01 gibba011 bash[30791]: cluster 2021-03-10T17:17:01.453188+0000 mgr.gibba032.whopeo (mgr.24625) 10761 : cluster [DBG] pgmap v6133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:01 gibba011 bash[31774]: cluster 2021-03-10T17:17:01.453188+0000 mgr.gibba032.whopeo (mgr.24625) 10761 : cluster [DBG] pgmap v6133: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:04 gibba011 bash[30791]: cluster 2021-03-10T17:17:03.454779+0000 mgr.gibba032.whopeo (mgr.24625) 10762 : cluster [DBG] pgmap v6134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:04 gibba011 bash[31774]: cluster 2021-03-10T17:17:03.454779+0000 mgr.gibba032.whopeo (mgr.24625) 10762 : cluster [DBG] pgmap v6134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:04 gibba032 bash[24004]: cluster 2021-03-10T17:17:03.454779+0000 mgr.gibba032.whopeo (mgr.24625) 10762 : cluster [DBG] pgmap v6134: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:06 gibba011 bash[30791]: cluster 2021-03-10T17:17:05.456329+0000 mgr.gibba032.whopeo (mgr.24625) 10763 : cluster [DBG] pgmap v6135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:06.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:06 gibba011 bash[31774]: cluster 2021-03-10T17:17:05.456329+0000 mgr.gibba032.whopeo (mgr.24625) 10763 : cluster [DBG] pgmap v6135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:06.874 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:06 gibba032 bash[24004]: cluster 2021-03-10T17:17:05.456329+0000 mgr.gibba032.whopeo (mgr.24625) 10763 : cluster [DBG] pgmap v6135: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:08 gibba011 bash[30791]: cephadm 2021-03-10T17:17:06.871920+0000 mgr.gibba032.whopeo (mgr.24625) 10764 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:08 gibba011 bash[30791]: cluster 2021-03-10T17:17:07.457871+0000 mgr.gibba032.whopeo (mgr.24625) 10765 : cluster [DBG] pgmap v6136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:08 gibba011 bash[31774]: cephadm 2021-03-10T17:17:06.871920+0000 mgr.gibba032.whopeo (mgr.24625) 10764 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:08 gibba011 bash[31774]: cluster 2021-03-10T17:17:07.457871+0000 mgr.gibba032.whopeo (mgr.24625) 10765 : cluster [DBG] pgmap v6136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:08 gibba032 bash[24004]: cephadm 2021-03-10T17:17:06.871920+0000 mgr.gibba032.whopeo (mgr.24625) 10764 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:08 gibba032 bash[24004]: cluster 2021-03-10T17:17:07.457871+0000 mgr.gibba032.whopeo (mgr.24625) 10765 : cluster [DBG] pgmap v6136: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:10.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:10 gibba011 bash[30791]: cluster 2021-03-10T17:17:09.459260+0000 mgr.gibba032.whopeo (mgr.24625) 10766 : cluster [DBG] pgmap v6137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:10.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:10 gibba011 bash[31774]: cluster 2021-03-10T17:17:09.459260+0000 mgr.gibba032.whopeo (mgr.24625) 10766 : cluster [DBG] pgmap v6137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:10 gibba032 bash[24004]: cluster 2021-03-10T17:17:09.459260+0000 mgr.gibba032.whopeo (mgr.24625) 10766 : cluster [DBG] pgmap v6137: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:11 gibba032 bash[24004]: cluster 2021-03-10T17:17:11.460918+0000 mgr.gibba032.whopeo (mgr.24625) 10767 : cluster [DBG] pgmap v6138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:11 gibba011 bash[30791]: cluster 2021-03-10T17:17:11.460918+0000 mgr.gibba032.whopeo (mgr.24625) 10767 : cluster [DBG] pgmap v6138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:11 gibba011 bash[31774]: cluster 2021-03-10T17:17:11.460918+0000 mgr.gibba032.whopeo (mgr.24625) 10767 : cluster [DBG] pgmap v6138: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:13.466 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:17:13.697 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:17:13.697 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:17:13.697 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:17:13.698 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:17:13.698 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:17:13.698 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:17:13.698 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:17:13.698 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:17:13.699 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:17:13.699 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:17:13.699 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:17:13.699 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:17:13.699 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:17:13.700 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:17:13.700 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:17:13.700 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:17:13.700 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:17:13.700 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:17:13.701 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:17:13.701 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:17:13.701 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:17:13.948 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:17:13.949 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:17:13.949 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:17:13.949 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:13.949 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:17:13.949 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:17:13.950 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:13.950 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:17:13.950 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:17:13.950 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:13.951 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:17:13.951 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:17:13.951 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:17:13.951 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:17:13.951 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:17:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[30791]: audit 2021-03-10T17:17:13.455429+0000 mgr.gibba032.whopeo (mgr.24625) 10768 : audit [DBG] from='client.51273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[30791]: cluster 2021-03-10T17:17:13.462414+0000 mgr.gibba032.whopeo (mgr.24625) 10769 : cluster [DBG] pgmap v6139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[30791]: audit 2021-03-10T17:17:13.682809+0000 mgr.gibba032.whopeo (mgr.24625) 10770 : audit [DBG] from='client.51279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:14.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[30791]: audit 2021-03-10T17:17:13.948544+0000 mon.a (mon.0) 3585 : audit [DBG] from='client.? 172.21.2.111:0/2353110318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[31774]: audit 2021-03-10T17:17:13.455429+0000 mgr.gibba032.whopeo (mgr.24625) 10768 : audit [DBG] from='client.51273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[31774]: cluster 2021-03-10T17:17:13.462414+0000 mgr.gibba032.whopeo (mgr.24625) 10769 : cluster [DBG] pgmap v6139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[31774]: audit 2021-03-10T17:17:13.682809+0000 mgr.gibba032.whopeo (mgr.24625) 10770 : audit [DBG] from='client.51279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:14 gibba011 bash[31774]: audit 2021-03-10T17:17:13.948544+0000 mon.a (mon.0) 3585 : audit [DBG] from='client.? 172.21.2.111:0/2353110318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:14 gibba032 bash[24004]: audit 2021-03-10T17:17:13.455429+0000 mgr.gibba032.whopeo (mgr.24625) 10768 : audit [DBG] from='client.51273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:14 gibba032 bash[24004]: cluster 2021-03-10T17:17:13.462414+0000 mgr.gibba032.whopeo (mgr.24625) 10769 : cluster [DBG] pgmap v6139: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:14 gibba032 bash[24004]: audit 2021-03-10T17:17:13.682809+0000 mgr.gibba032.whopeo (mgr.24625) 10770 : audit [DBG] from='client.51279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:14 gibba032 bash[24004]: audit 2021-03-10T17:17:13.948544+0000 mon.a (mon.0) 3585 : audit [DBG] from='client.? 172.21.2.111:0/2353110318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:16.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:16 gibba011 bash[30791]: cluster 2021-03-10T17:17:15.463875+0000 mgr.gibba032.whopeo (mgr.24625) 10771 : cluster [DBG] pgmap v6140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:16.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:16 gibba011 bash[31774]: cluster 2021-03-10T17:17:15.463875+0000 mgr.gibba032.whopeo (mgr.24625) 10771 : cluster [DBG] pgmap v6140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:16 gibba032 bash[24004]: cluster 2021-03-10T17:17:15.463875+0000 mgr.gibba032.whopeo (mgr.24625) 10771 : cluster [DBG] pgmap v6140: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:18.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:18 gibba011 bash[30791]: cluster 2021-03-10T17:17:17.465379+0000 mgr.gibba032.whopeo (mgr.24625) 10772 : cluster [DBG] pgmap v6141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:18.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:18 gibba011 bash[31774]: cluster 2021-03-10T17:17:17.465379+0000 mgr.gibba032.whopeo (mgr.24625) 10772 : cluster [DBG] pgmap v6141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:18 gibba032 bash[24004]: cluster 2021-03-10T17:17:17.465379+0000 mgr.gibba032.whopeo (mgr.24625) 10772 : cluster [DBG] pgmap v6141: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:20.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:20 gibba011 bash[30791]: cluster 2021-03-10T17:17:19.466665+0000 mgr.gibba032.whopeo (mgr.24625) 10773 : cluster [DBG] pgmap v6142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:20.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:20 gibba011 bash[31774]: cluster 2021-03-10T17:17:19.466665+0000 mgr.gibba032.whopeo (mgr.24625) 10773 : cluster [DBG] pgmap v6142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:20 gibba032 bash[24004]: cluster 2021-03-10T17:17:19.466665+0000 mgr.gibba032.whopeo (mgr.24625) 10773 : cluster [DBG] pgmap v6142: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:21 gibba032 bash[24004]: cluster 2021-03-10T17:17:21.468250+0000 mgr.gibba032.whopeo (mgr.24625) 10774 : cluster [DBG] pgmap v6143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:21 gibba011 bash[30791]: cluster 2021-03-10T17:17:21.468250+0000 mgr.gibba032.whopeo (mgr.24625) 10774 : cluster [DBG] pgmap v6143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:21 gibba011 bash[31774]: cluster 2021-03-10T17:17:21.468250+0000 mgr.gibba032.whopeo (mgr.24625) 10774 : cluster [DBG] pgmap v6143: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:21.888732+0000 mgr.gibba032.whopeo (mgr.24625) 10775 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:17:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:21.890270+0000 mgr.gibba032.whopeo (mgr.24625) 10776 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:17:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.118939+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:17:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.119454+0000 mgr.gibba032.whopeo (mgr.24625) 10777 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:17:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.121538+0000 mgr.gibba032.whopeo (mgr.24625) 10778 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:17:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.122047+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:17:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.123239+0000 mgr.gibba032.whopeo (mgr.24625) 10779 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:17:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.123280+0000 mgr.gibba032.whopeo (mgr.24625) 10780 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:17:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.123806+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.124640+0000 mgr.gibba032.whopeo (mgr.24625) 10781 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:17:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.124679+0000 mgr.gibba032.whopeo (mgr.24625) 10782 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:17:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.125537+0000 mon.a (mon.0) 3589 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.125837+0000 mgr.gibba032.whopeo (mgr.24625) 10783 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:17:23.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.126834+0000 mon.a (mon.0) 3590 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.127109+0000 mgr.gibba032.whopeo (mgr.24625) 10784 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:17:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.127148+0000 mgr.gibba032.whopeo (mgr.24625) 10785 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:17:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.128628+0000 mgr.gibba032.whopeo (mgr.24625) 10786 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:17:23.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.129035+0000 mgr.gibba032.whopeo (mgr.24625) 10787 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:17:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.129433+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.130766+0000 mgr.gibba032.whopeo (mgr.24625) 10788 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:17:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.130806+0000 mgr.gibba032.whopeo (mgr.24625) 10789 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:17:23.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: cephadm 2021-03-10T17:17:22.130918+0000 mgr.gibba032.whopeo (mgr.24625) 10790 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:23.514 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:23 gibba032 bash[24004]: audit 2021-03-10T17:17:22.131301+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:21.888732+0000 mgr.gibba032.whopeo (mgr.24625) 10775 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:17:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:21.890270+0000 mgr.gibba032.whopeo (mgr.24625) 10776 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:17:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.118939+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:17:23.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.119454+0000 mgr.gibba032.whopeo (mgr.24625) 10777 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:17:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.121538+0000 mgr.gibba032.whopeo (mgr.24625) 10778 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:17:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.122047+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:17:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.123239+0000 mgr.gibba032.whopeo (mgr.24625) 10779 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:17:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.123280+0000 mgr.gibba032.whopeo (mgr.24625) 10780 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:17:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.123806+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.124640+0000 mgr.gibba032.whopeo (mgr.24625) 10781 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:17:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.124679+0000 mgr.gibba032.whopeo (mgr.24625) 10782 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:17:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.125537+0000 mon.a (mon.0) 3589 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.125837+0000 mgr.gibba032.whopeo (mgr.24625) 10783 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:17:23.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.126834+0000 mon.a (mon.0) 3590 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.127109+0000 mgr.gibba032.whopeo (mgr.24625) 10784 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:17:23.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.127148+0000 mgr.gibba032.whopeo (mgr.24625) 10785 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:17:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.128628+0000 mgr.gibba032.whopeo (mgr.24625) 10786 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:17:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.129035+0000 mgr.gibba032.whopeo (mgr.24625) 10787 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:17:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.129433+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.130766+0000 mgr.gibba032.whopeo (mgr.24625) 10788 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:17:23.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.130806+0000 mgr.gibba032.whopeo (mgr.24625) 10789 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:17:23.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: cephadm 2021-03-10T17:17:22.130918+0000 mgr.gibba032.whopeo (mgr.24625) 10790 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:23.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[30791]: audit 2021-03-10T17:17:22.131301+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:21.888732+0000 mgr.gibba032.whopeo (mgr.24625) 10775 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:17:23.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:21.890270+0000 mgr.gibba032.whopeo (mgr.24625) 10776 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:17:23.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.118939+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:17:23.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.119454+0000 mgr.gibba032.whopeo (mgr.24625) 10777 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:17:23.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.121538+0000 mgr.gibba032.whopeo (mgr.24625) 10778 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:17:23.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.122047+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:17:23.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.123239+0000 mgr.gibba032.whopeo (mgr.24625) 10779 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:17:23.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.123280+0000 mgr.gibba032.whopeo (mgr.24625) 10780 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:17:23.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.123806+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.124640+0000 mgr.gibba032.whopeo (mgr.24625) 10781 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:17:23.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.124679+0000 mgr.gibba032.whopeo (mgr.24625) 10782 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:17:23.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.125537+0000 mon.a (mon.0) 3589 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.125837+0000 mgr.gibba032.whopeo (mgr.24625) 10783 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:17:23.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.126834+0000 mon.a (mon.0) 3590 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.127109+0000 mgr.gibba032.whopeo (mgr.24625) 10784 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:17:23.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.127148+0000 mgr.gibba032.whopeo (mgr.24625) 10785 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:17:23.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.128628+0000 mgr.gibba032.whopeo (mgr.24625) 10786 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:17:23.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.129035+0000 mgr.gibba032.whopeo (mgr.24625) 10787 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:17:23.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.129433+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:23.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.130766+0000 mgr.gibba032.whopeo (mgr.24625) 10788 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:17:23.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.130806+0000 mgr.gibba032.whopeo (mgr.24625) 10789 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:17:23.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: cephadm 2021-03-10T17:17:22.130918+0000 mgr.gibba032.whopeo (mgr.24625) 10790 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:23.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:23 gibba011 bash[31774]: audit 2021-03-10T17:17:22.131301+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:24 gibba032 bash[24004]: cluster 2021-03-10T17:17:23.469642+0000 mgr.gibba032.whopeo (mgr.24625) 10791 : cluster [DBG] pgmap v6144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:24 gibba011 bash[30791]: cluster 2021-03-10T17:17:23.469642+0000 mgr.gibba032.whopeo (mgr.24625) 10791 : cluster [DBG] pgmap v6144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:24.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:24 gibba011 bash[31774]: cluster 2021-03-10T17:17:23.469642+0000 mgr.gibba032.whopeo (mgr.24625) 10791 : cluster [DBG] pgmap v6144: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:26 gibba011 bash[30791]: cluster 2021-03-10T17:17:25.471129+0000 mgr.gibba032.whopeo (mgr.24625) 10792 : cluster [DBG] pgmap v6145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:26 gibba011 bash[31774]: cluster 2021-03-10T17:17:25.471129+0000 mgr.gibba032.whopeo (mgr.24625) 10792 : cluster [DBG] pgmap v6145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:26 gibba032 bash[24004]: cluster 2021-03-10T17:17:25.471129+0000 mgr.gibba032.whopeo (mgr.24625) 10792 : cluster [DBG] pgmap v6145: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:28.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:28 gibba011 bash[30791]: cluster 2021-03-10T17:17:27.472942+0000 mgr.gibba032.whopeo (mgr.24625) 10793 : cluster [DBG] pgmap v6146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:28.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:28 gibba011 bash[31774]: cluster 2021-03-10T17:17:27.472942+0000 mgr.gibba032.whopeo (mgr.24625) 10793 : cluster [DBG] pgmap v6146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:29.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:28 gibba032 bash[24004]: cluster 2021-03-10T17:17:27.472942+0000 mgr.gibba032.whopeo (mgr.24625) 10793 : cluster [DBG] pgmap v6146: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:30.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:30 gibba011 bash[30791]: cluster 2021-03-10T17:17:29.474182+0000 mgr.gibba032.whopeo (mgr.24625) 10794 : cluster [DBG] pgmap v6147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:30.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:30 gibba011 bash[31774]: cluster 2021-03-10T17:17:29.474182+0000 mgr.gibba032.whopeo (mgr.24625) 10794 : cluster [DBG] pgmap v6147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:30 gibba032 bash[24004]: cluster 2021-03-10T17:17:29.474182+0000 mgr.gibba032.whopeo (mgr.24625) 10794 : cluster [DBG] pgmap v6147: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:32.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:31 gibba032 bash[24004]: cluster 2021-03-10T17:17:31.475991+0000 mgr.gibba032.whopeo (mgr.24625) 10795 : cluster [DBG] pgmap v6148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:31 gibba011 bash[30791]: cluster 2021-03-10T17:17:31.475991+0000 mgr.gibba032.whopeo (mgr.24625) 10795 : cluster [DBG] pgmap v6148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:31 gibba011 bash[31774]: cluster 2021-03-10T17:17:31.475991+0000 mgr.gibba032.whopeo (mgr.24625) 10795 : cluster [DBG] pgmap v6148: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:34.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:34 gibba011 bash[30791]: cluster 2021-03-10T17:17:33.477653+0000 mgr.gibba032.whopeo (mgr.24625) 10796 : cluster [DBG] pgmap v6149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:34 gibba011 bash[31774]: cluster 2021-03-10T17:17:33.477653+0000 mgr.gibba032.whopeo (mgr.24625) 10796 : cluster [DBG] pgmap v6149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:34 gibba032 bash[24004]: cluster 2021-03-10T17:17:33.477653+0000 mgr.gibba032.whopeo (mgr.24625) 10796 : cluster [DBG] pgmap v6149: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:36.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:36 gibba011 bash[30791]: cluster 2021-03-10T17:17:35.479000+0000 mgr.gibba032.whopeo (mgr.24625) 10797 : cluster [DBG] pgmap v6150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:36.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:36 gibba011 bash[31774]: cluster 2021-03-10T17:17:35.479000+0000 mgr.gibba032.whopeo (mgr.24625) 10797 : cluster [DBG] pgmap v6150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:36 gibba032 bash[24004]: cluster 2021-03-10T17:17:35.479000+0000 mgr.gibba032.whopeo (mgr.24625) 10797 : cluster [DBG] pgmap v6150: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:38.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:38 gibba011 bash[30791]: cephadm 2021-03-10T17:17:37.146835+0000 mgr.gibba032.whopeo (mgr.24625) 10798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:38 gibba011 bash[30791]: cluster 2021-03-10T17:17:37.480515+0000 mgr.gibba032.whopeo (mgr.24625) 10799 : cluster [DBG] pgmap v6151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:38 gibba011 bash[31774]: cephadm 2021-03-10T17:17:37.146835+0000 mgr.gibba032.whopeo (mgr.24625) 10798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:38 gibba011 bash[31774]: cluster 2021-03-10T17:17:37.480515+0000 mgr.gibba032.whopeo (mgr.24625) 10799 : cluster [DBG] pgmap v6151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:39.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:38 gibba032 bash[24004]: cephadm 2021-03-10T17:17:37.146835+0000 mgr.gibba032.whopeo (mgr.24625) 10798 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:38 gibba032 bash[24004]: cluster 2021-03-10T17:17:37.480515+0000 mgr.gibba032.whopeo (mgr.24625) 10799 : cluster [DBG] pgmap v6151: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:40.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:40 gibba011 bash[30791]: cluster 2021-03-10T17:17:39.481893+0000 mgr.gibba032.whopeo (mgr.24625) 10800 : cluster [DBG] pgmap v6152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:40.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:40 gibba011 bash[31774]: cluster 2021-03-10T17:17:39.481893+0000 mgr.gibba032.whopeo (mgr.24625) 10800 : cluster [DBG] pgmap v6152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:41.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:40 gibba032 bash[24004]: cluster 2021-03-10T17:17:39.481893+0000 mgr.gibba032.whopeo (mgr.24625) 10800 : cluster [DBG] pgmap v6152: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:41 gibba032 bash[24004]: cluster 2021-03-10T17:17:41.483540+0000 mgr.gibba032.whopeo (mgr.24625) 10801 : cluster [DBG] pgmap v6153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:41 gibba011 bash[30791]: cluster 2021-03-10T17:17:41.483540+0000 mgr.gibba032.whopeo (mgr.24625) 10801 : cluster [DBG] pgmap v6153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:41 gibba011 bash[31774]: cluster 2021-03-10T17:17:41.483540+0000 mgr.gibba032.whopeo (mgr.24625) 10801 : cluster [DBG] pgmap v6153: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:44.197 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:17:44.423 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:17:44.424 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:17:44.424 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:17:44.424 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:17:44.424 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:17:44.425 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:17:44.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:17:44.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:17:44.425 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:17:44.425 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:17:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:17:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:17:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:17:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:17:44.426 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:17:44.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:17:44.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:17:44.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:17:44.427 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:17:44.427 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:17:44.428 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:17:44.671 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:17:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:17:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:17:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:17:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:17:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:17:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:17:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:17:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:17:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:17:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:17:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:17:44.675 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:17:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:44 gibba011 bash[30791]: cluster 2021-03-10T17:17:43.484052+0000 mgr.gibba032.whopeo (mgr.24625) 10802 : cluster [DBG] pgmap v6154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:44.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:44 gibba011 bash[31774]: cluster 2021-03-10T17:17:43.484052+0000 mgr.gibba032.whopeo (mgr.24625) 10802 : cluster [DBG] pgmap v6154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:44 gibba032 bash[24004]: cluster 2021-03-10T17:17:43.484052+0000 mgr.gibba032.whopeo (mgr.24625) 10802 : cluster [DBG] pgmap v6154: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:45.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[30791]: audit 2021-03-10T17:17:44.186275+0000 mgr.gibba032.whopeo (mgr.24625) 10803 : audit [DBG] from='client.51291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[30791]: audit 2021-03-10T17:17:44.409916+0000 mgr.gibba032.whopeo (mgr.24625) 10804 : audit [DBG] from='client.51297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:45.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[30791]: audit 2021-03-10T17:17:44.671739+0000 mon.a (mon.0) 3593 : audit [DBG] from='client.? 172.21.2.111:0/388741193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[31774]: audit 2021-03-10T17:17:44.186275+0000 mgr.gibba032.whopeo (mgr.24625) 10803 : audit [DBG] from='client.51291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[31774]: audit 2021-03-10T17:17:44.409916+0000 mgr.gibba032.whopeo (mgr.24625) 10804 : audit [DBG] from='client.51297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:45.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:45 gibba011 bash[31774]: audit 2021-03-10T17:17:44.671739+0000 mon.a (mon.0) 3593 : audit [DBG] from='client.? 172.21.2.111:0/388741193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:45 gibba032 bash[24004]: audit 2021-03-10T17:17:44.186275+0000 mgr.gibba032.whopeo (mgr.24625) 10803 : audit [DBG] from='client.51291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:46.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:45 gibba032 bash[24004]: audit 2021-03-10T17:17:44.409916+0000 mgr.gibba032.whopeo (mgr.24625) 10804 : audit [DBG] from='client.51297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:17:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:45 gibba032 bash[24004]: audit 2021-03-10T17:17:44.671739+0000 mon.a (mon.0) 3593 : audit [DBG] from='client.? 172.21.2.111:0/388741193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:17:46.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:46 gibba011 bash[30791]: cluster 2021-03-10T17:17:45.485588+0000 mgr.gibba032.whopeo (mgr.24625) 10805 : cluster [DBG] pgmap v6155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:46.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:46 gibba011 bash[31774]: cluster 2021-03-10T17:17:45.485588+0000 mgr.gibba032.whopeo (mgr.24625) 10805 : cluster [DBG] pgmap v6155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:47.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:46 gibba032 bash[24004]: cluster 2021-03-10T17:17:45.485588+0000 mgr.gibba032.whopeo (mgr.24625) 10805 : cluster [DBG] pgmap v6155: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:48.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:48 gibba011 bash[30791]: cluster 2021-03-10T17:17:47.487239+0000 mgr.gibba032.whopeo (mgr.24625) 10806 : cluster [DBG] pgmap v6156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:48.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:48 gibba011 bash[31774]: cluster 2021-03-10T17:17:47.487239+0000 mgr.gibba032.whopeo (mgr.24625) 10806 : cluster [DBG] pgmap v6156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:48 gibba032 bash[24004]: cluster 2021-03-10T17:17:47.487239+0000 mgr.gibba032.whopeo (mgr.24625) 10806 : cluster [DBG] pgmap v6156: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:50 gibba032 bash[24004]: cluster 2021-03-10T17:17:49.487984+0000 mgr.gibba032.whopeo (mgr.24625) 10807 : cluster [DBG] pgmap v6157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:50 gibba011 bash[30791]: cluster 2021-03-10T17:17:49.487984+0000 mgr.gibba032.whopeo (mgr.24625) 10807 : cluster [DBG] pgmap v6157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:50 gibba011 bash[31774]: cluster 2021-03-10T17:17:49.487984+0000 mgr.gibba032.whopeo (mgr.24625) 10807 : cluster [DBG] pgmap v6157: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:52.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:51 gibba032 bash[24004]: cluster 2021-03-10T17:17:51.489583+0000 mgr.gibba032.whopeo (mgr.24625) 10808 : cluster [DBG] pgmap v6158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:51 gibba011 bash[30791]: cluster 2021-03-10T17:17:51.489583+0000 mgr.gibba032.whopeo (mgr.24625) 10808 : cluster [DBG] pgmap v6158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:52.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:51 gibba011 bash[31774]: cluster 2021-03-10T17:17:51.489583+0000 mgr.gibba032.whopeo (mgr.24625) 10808 : cluster [DBG] pgmap v6158: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:53.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:53 gibba032 bash[24004]: cephadm 2021-03-10T17:17:52.160799+0000 mgr.gibba032.whopeo (mgr.24625) 10809 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:53 gibba011 bash[30791]: cephadm 2021-03-10T17:17:52.160799+0000 mgr.gibba032.whopeo (mgr.24625) 10809 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:53 gibba011 bash[31774]: cephadm 2021-03-10T17:17:52.160799+0000 mgr.gibba032.whopeo (mgr.24625) 10809 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:17:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:54 gibba011 bash[30791]: cluster 2021-03-10T17:17:53.491194+0000 mgr.gibba032.whopeo (mgr.24625) 10810 : cluster [DBG] pgmap v6159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:54 gibba011 bash[31774]: cluster 2021-03-10T17:17:53.491194+0000 mgr.gibba032.whopeo (mgr.24625) 10810 : cluster [DBG] pgmap v6159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:54 gibba032 bash[24004]: cluster 2021-03-10T17:17:53.491194+0000 mgr.gibba032.whopeo (mgr.24625) 10810 : cluster [DBG] pgmap v6159: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:56.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:56 gibba011 bash[30791]: cluster 2021-03-10T17:17:55.491752+0000 mgr.gibba032.whopeo (mgr.24625) 10811 : cluster [DBG] pgmap v6160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:56.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:56 gibba011 bash[31774]: cluster 2021-03-10T17:17:55.491752+0000 mgr.gibba032.whopeo (mgr.24625) 10811 : cluster [DBG] pgmap v6160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:57.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:56 gibba032 bash[24004]: cluster 2021-03-10T17:17:55.491752+0000 mgr.gibba032.whopeo (mgr.24625) 10811 : cluster [DBG] pgmap v6160: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:57 gibba011 bash[30791]: audit 2021-03-10T17:17:57.421911+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:17:57.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:57 gibba011 bash[30791]: audit 2021-03-10T17:17:57.422568+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:17:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:57 gibba011 bash[31774]: audit 2021-03-10T17:17:57.421911+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:17:57.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:57 gibba011 bash[31774]: audit 2021-03-10T17:17:57.422568+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:17:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:57 gibba032 bash[24004]: audit 2021-03-10T17:17:57.421911+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:17:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:57 gibba032 bash[24004]: audit 2021-03-10T17:17:57.422568+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:17:58.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[30791]: cluster 2021-03-10T17:17:57.492985+0000 mgr.gibba032.whopeo (mgr.24625) 10812 : cluster [DBG] pgmap v6161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[30791]: audit 2021-03-10T17:17:57.587787+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:17:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[30791]: audit 2021-03-10T17:17:57.587980+0000 mon.a (mon.0) 3597 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:17:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[30791]: audit 2021-03-10T17:17:57.588149+0000 mon.a (mon.0) 3598 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:17:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[30791]: audit 2021-03-10T17:17:57.588315+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:17:58.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[31774]: cluster 2021-03-10T17:17:57.492985+0000 mgr.gibba032.whopeo (mgr.24625) 10812 : cluster [DBG] pgmap v6161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[31774]: audit 2021-03-10T17:17:57.587787+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:17:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[31774]: audit 2021-03-10T17:17:57.587980+0000 mon.a (mon.0) 3597 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:17:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[31774]: audit 2021-03-10T17:17:57.588149+0000 mon.a (mon.0) 3598 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:17:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:17:58 gibba011 bash[31774]: audit 2021-03-10T17:17:57.588315+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:17:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:58 gibba032 bash[24004]: cluster 2021-03-10T17:17:57.492985+0000 mgr.gibba032.whopeo (mgr.24625) 10812 : cluster [DBG] pgmap v6161: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:17:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:58 gibba032 bash[24004]: audit 2021-03-10T17:17:57.587787+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:17:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:58 gibba032 bash[24004]: audit 2021-03-10T17:17:57.587980+0000 mon.a (mon.0) 3597 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:17:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:58 gibba032 bash[24004]: audit 2021-03-10T17:17:57.588149+0000 mon.a (mon.0) 3598 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2021-03-10T17:17:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:17:58 gibba032 bash[24004]: audit 2021-03-10T17:17:57.588315+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:18:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:00 gibba032 bash[24004]: cluster 2021-03-10T17:17:59.494262+0000 mgr.gibba032.whopeo (mgr.24625) 10813 : cluster [DBG] pgmap v6162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:00 gibba011 bash[30791]: cluster 2021-03-10T17:17:59.494262+0000 mgr.gibba032.whopeo (mgr.24625) 10813 : cluster [DBG] pgmap v6162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:00 gibba011 bash[31774]: cluster 2021-03-10T17:17:59.494262+0000 mgr.gibba032.whopeo (mgr.24625) 10813 : cluster [DBG] pgmap v6162: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:01 gibba032 bash[24004]: cluster 2021-03-10T17:18:01.494983+0000 mgr.gibba032.whopeo (mgr.24625) 10814 : cluster [DBG] pgmap v6163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:02 gibba011 bash[30791]: cluster 2021-03-10T17:18:01.494983+0000 mgr.gibba032.whopeo (mgr.24625) 10814 : cluster [DBG] pgmap v6163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:02 gibba011 bash[31774]: cluster 2021-03-10T17:18:01.494983+0000 mgr.gibba032.whopeo (mgr.24625) 10814 : cluster [DBG] pgmap v6163: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:04 gibba011 bash[30791]: cluster 2021-03-10T17:18:03.496320+0000 mgr.gibba032.whopeo (mgr.24625) 10815 : cluster [DBG] pgmap v6164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:04.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:04 gibba011 bash[31774]: cluster 2021-03-10T17:18:03.496320+0000 mgr.gibba032.whopeo (mgr.24625) 10815 : cluster [DBG] pgmap v6164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:05.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:04 gibba032 bash[24004]: cluster 2021-03-10T17:18:03.496320+0000 mgr.gibba032.whopeo (mgr.24625) 10815 : cluster [DBG] pgmap v6164: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:06.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:06 gibba011 bash[30791]: cluster 2021-03-10T17:18:05.497807+0000 mgr.gibba032.whopeo (mgr.24625) 10816 : cluster [DBG] pgmap v6165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:06 gibba011 bash[31774]: cluster 2021-03-10T17:18:05.497807+0000 mgr.gibba032.whopeo (mgr.24625) 10816 : cluster [DBG] pgmap v6165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:07.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:06 gibba032 bash[24004]: cluster 2021-03-10T17:18:05.497807+0000 mgr.gibba032.whopeo (mgr.24625) 10816 : cluster [DBG] pgmap v6165: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:08.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:08 gibba011 bash[30791]: cephadm 2021-03-10T17:18:07.176744+0000 mgr.gibba032.whopeo (mgr.24625) 10817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:08 gibba011 bash[30791]: cluster 2021-03-10T17:18:07.498597+0000 mgr.gibba032.whopeo (mgr.24625) 10818 : cluster [DBG] pgmap v6166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:08.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:08 gibba011 bash[31774]: cephadm 2021-03-10T17:18:07.176744+0000 mgr.gibba032.whopeo (mgr.24625) 10817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:08 gibba011 bash[31774]: cluster 2021-03-10T17:18:07.498597+0000 mgr.gibba032.whopeo (mgr.24625) 10818 : cluster [DBG] pgmap v6166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:09.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:08 gibba032 bash[24004]: cephadm 2021-03-10T17:18:07.176744+0000 mgr.gibba032.whopeo (mgr.24625) 10817 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:08 gibba032 bash[24004]: cluster 2021-03-10T17:18:07.498597+0000 mgr.gibba032.whopeo (mgr.24625) 10818 : cluster [DBG] pgmap v6166: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:11.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:10 gibba032 bash[24004]: cluster 2021-03-10T17:18:09.499874+0000 mgr.gibba032.whopeo (mgr.24625) 10819 : cluster [DBG] pgmap v6167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:10 gibba011 bash[30791]: cluster 2021-03-10T17:18:09.499874+0000 mgr.gibba032.whopeo (mgr.24625) 10819 : cluster [DBG] pgmap v6167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:10 gibba011 bash[31774]: cluster 2021-03-10T17:18:09.499874+0000 mgr.gibba032.whopeo (mgr.24625) 10819 : cluster [DBG] pgmap v6167: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:12.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:12 gibba032 bash[24004]: cluster 2021-03-10T17:18:11.501536+0000 mgr.gibba032.whopeo (mgr.24625) 10820 : cluster [DBG] pgmap v6168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:12 gibba011 bash[30791]: cluster 2021-03-10T17:18:11.501536+0000 mgr.gibba032.whopeo (mgr.24625) 10820 : cluster [DBG] pgmap v6168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:12.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:12 gibba011 bash[31774]: cluster 2021-03-10T17:18:11.501536+0000 mgr.gibba032.whopeo (mgr.24625) 10820 : cluster [DBG] pgmap v6168: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:14.828 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:14 gibba011 bash[30791]: cluster 2021-03-10T17:18:13.502329+0000 mgr.gibba032.whopeo (mgr.24625) 10821 : cluster [DBG] pgmap v6169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:14.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:14 gibba011 bash[31774]: cluster 2021-03-10T17:18:13.502329+0000 mgr.gibba032.whopeo (mgr.24625) 10821 : cluster [DBG] pgmap v6169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:14.914 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:18:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:14 gibba032 bash[24004]: cluster 2021-03-10T17:18:13.502329+0000 mgr.gibba032.whopeo (mgr.24625) 10821 : cluster [DBG] pgmap v6169: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:15.141 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:18:15.141 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:18:15.142 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:18:15.142 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:18:15.142 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:18:15.142 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:18:15.142 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:18:15.143 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:18:15.143 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:18:15.143 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:18:15.143 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:18:15.143 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:18:15.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:18:15.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:18:15.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:18:15.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:18:15.144 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:18:15.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:18:15.145 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:18:15.145 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:18:15.146 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:18:15.392 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:18:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:18:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:18:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:18:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:18:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:18:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:18:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:18:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:18:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:18:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:18:15.396 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:18:15.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:15 gibba011 bash[30791]: audit 2021-03-10T17:18:15.391874+0000 mon.c (mon.1) 207 : audit [DBG] from='client.? 172.21.2.111:0/2344055972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:15.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:15 gibba011 bash[31774]: audit 2021-03-10T17:18:15.391874+0000 mon.c (mon.1) 207 : audit [DBG] from='client.? 172.21.2.111:0/2344055972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:16.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:15 gibba032 bash[24004]: audit 2021-03-10T17:18:15.391874+0000 mon.c (mon.1) 207 : audit [DBG] from='client.? 172.21.2.111:0/2344055972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:16 gibba032 bash[24004]: audit 2021-03-10T17:18:14.903565+0000 mgr.gibba032.whopeo (mgr.24625) 10822 : audit [DBG] from='client.61291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:16 gibba032 bash[24004]: audit 2021-03-10T17:18:15.127225+0000 mgr.gibba032.whopeo (mgr.24625) 10823 : audit [DBG] from='client.51315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:16 gibba032 bash[24004]: cluster 2021-03-10T17:18:15.503865+0000 mgr.gibba032.whopeo (mgr.24625) 10824 : cluster [DBG] pgmap v6170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[30791]: audit 2021-03-10T17:18:14.903565+0000 mgr.gibba032.whopeo (mgr.24625) 10822 : audit [DBG] from='client.61291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[30791]: audit 2021-03-10T17:18:15.127225+0000 mgr.gibba032.whopeo (mgr.24625) 10823 : audit [DBG] from='client.51315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[30791]: cluster 2021-03-10T17:18:15.503865+0000 mgr.gibba032.whopeo (mgr.24625) 10824 : cluster [DBG] pgmap v6170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[31774]: audit 2021-03-10T17:18:14.903565+0000 mgr.gibba032.whopeo (mgr.24625) 10822 : audit [DBG] from='client.61291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[31774]: audit 2021-03-10T17:18:15.127225+0000 mgr.gibba032.whopeo (mgr.24625) 10823 : audit [DBG] from='client.51315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:16 gibba011 bash[31774]: cluster 2021-03-10T17:18:15.503865+0000 mgr.gibba032.whopeo (mgr.24625) 10824 : cluster [DBG] pgmap v6170: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:19.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:18 gibba032 bash[24004]: cluster 2021-03-10T17:18:17.505682+0000 mgr.gibba032.whopeo (mgr.24625) 10825 : cluster [DBG] pgmap v6171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:18 gibba011 bash[30791]: cluster 2021-03-10T17:18:17.505682+0000 mgr.gibba032.whopeo (mgr.24625) 10825 : cluster [DBG] pgmap v6171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:18 gibba011 bash[31774]: cluster 2021-03-10T17:18:17.505682+0000 mgr.gibba032.whopeo (mgr.24625) 10825 : cluster [DBG] pgmap v6171: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:20 gibba032 bash[24004]: cluster 2021-03-10T17:18:19.506223+0000 mgr.gibba032.whopeo (mgr.24625) 10826 : cluster [DBG] pgmap v6172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:20 gibba011 bash[30791]: cluster 2021-03-10T17:18:19.506223+0000 mgr.gibba032.whopeo (mgr.24625) 10826 : cluster [DBG] pgmap v6172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:20 gibba011 bash[31774]: cluster 2021-03-10T17:18:19.506223+0000 mgr.gibba032.whopeo (mgr.24625) 10826 : cluster [DBG] pgmap v6172: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:22.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:22 gibba011 bash[30791]: cluster 2021-03-10T17:18:21.507813+0000 mgr.gibba032.whopeo (mgr.24625) 10827 : cluster [DBG] pgmap v6173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:22 gibba011 bash[31774]: cluster 2021-03-10T17:18:21.507813+0000 mgr.gibba032.whopeo (mgr.24625) 10827 : cluster [DBG] pgmap v6173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:22 gibba032 bash[24004]: cluster 2021-03-10T17:18:21.507813+0000 mgr.gibba032.whopeo (mgr.24625) 10827 : cluster [DBG] pgmap v6173: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:23.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.394735+0000 mgr.gibba032.whopeo (mgr.24625) 10828 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:18:23.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.395908+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.401515+0000 mgr.gibba032.whopeo (mgr.24625) 10829 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:18:23.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.402867+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.605596+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.606439+0000 mgr.gibba032.whopeo (mgr.24625) 10830 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:18:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.608096+0000 mgr.gibba032.whopeo (mgr.24625) 10831 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:18:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.608985+0000 mon.a (mon.0) 3603 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:18:23.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.609311+0000 mgr.gibba032.whopeo (mgr.24625) 10832 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:18:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.609347+0000 mgr.gibba032.whopeo (mgr.24625) 10833 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:18:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.610451+0000 mon.a (mon.0) 3604 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.610563+0000 mgr.gibba032.whopeo (mgr.24625) 10834 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:18:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.610598+0000 mgr.gibba032.whopeo (mgr.24625) 10835 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:18:23.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.611674+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.612941+0000 mgr.gibba032.whopeo (mgr.24625) 10836 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:18:23.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.613048+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.615231+0000 mgr.gibba032.whopeo (mgr.24625) 10837 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:18:23.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.615276+0000 mgr.gibba032.whopeo (mgr.24625) 10838 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:18:23.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.617822+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.617877+0000 mgr.gibba032.whopeo (mgr.24625) 10839 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:18:23.766 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.618380+0000 mgr.gibba032.whopeo (mgr.24625) 10840 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:18:23.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: audit 2021-03-10T17:18:22.620781+0000 mon.a (mon.0) 3608 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.621041+0000 mgr.gibba032.whopeo (mgr.24625) 10841 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:18:23.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.621093+0000 mgr.gibba032.whopeo (mgr.24625) 10842 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:18:23.767 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:23 gibba032 bash[24004]: cephadm 2021-03-10T17:18:22.621242+0000 mgr.gibba032.whopeo (mgr.24625) 10843 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:23.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.394735+0000 mgr.gibba032.whopeo (mgr.24625) 10828 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:18:23.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.395908+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.401515+0000 mgr.gibba032.whopeo (mgr.24625) 10829 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:18:23.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.402867+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.605596+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.606439+0000 mgr.gibba032.whopeo (mgr.24625) 10830 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:18:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.608096+0000 mgr.gibba032.whopeo (mgr.24625) 10831 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:18:23.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.608985+0000 mon.a (mon.0) 3603 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:18:23.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.609311+0000 mgr.gibba032.whopeo (mgr.24625) 10832 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:18:23.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.609347+0000 mgr.gibba032.whopeo (mgr.24625) 10833 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:18:23.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.610451+0000 mon.a (mon.0) 3604 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.610563+0000 mgr.gibba032.whopeo (mgr.24625) 10834 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:18:23.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.610598+0000 mgr.gibba032.whopeo (mgr.24625) 10835 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:18:23.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.611674+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.612941+0000 mgr.gibba032.whopeo (mgr.24625) 10836 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:18:23.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.613048+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.615231+0000 mgr.gibba032.whopeo (mgr.24625) 10837 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:18:23.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.615276+0000 mgr.gibba032.whopeo (mgr.24625) 10838 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:18:23.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.617822+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.617877+0000 mgr.gibba032.whopeo (mgr.24625) 10839 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:18:23.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.618380+0000 mgr.gibba032.whopeo (mgr.24625) 10840 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:18:23.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.394735+0000 mgr.gibba032.whopeo (mgr.24625) 10828 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:18:23.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.395908+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.401515+0000 mgr.gibba032.whopeo (mgr.24625) 10829 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:18:23.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.402867+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.605596+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:18:23.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.606439+0000 mgr.gibba032.whopeo (mgr.24625) 10830 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:18:23.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.608096+0000 mgr.gibba032.whopeo (mgr.24625) 10831 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:18:23.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.608985+0000 mon.a (mon.0) 3603 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:18:23.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.609311+0000 mgr.gibba032.whopeo (mgr.24625) 10832 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:18:23.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.609347+0000 mgr.gibba032.whopeo (mgr.24625) 10833 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:18:23.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.610451+0000 mon.a (mon.0) 3604 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.610563+0000 mgr.gibba032.whopeo (mgr.24625) 10834 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:18:23.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.610598+0000 mgr.gibba032.whopeo (mgr.24625) 10835 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:18:23.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.611674+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.612941+0000 mgr.gibba032.whopeo (mgr.24625) 10836 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:18:23.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.613048+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.615231+0000 mgr.gibba032.whopeo (mgr.24625) 10837 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:18:23.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.615276+0000 mgr.gibba032.whopeo (mgr.24625) 10838 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:18:23.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.617822+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.617877+0000 mgr.gibba032.whopeo (mgr.24625) 10839 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:18:23.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.618380+0000 mgr.gibba032.whopeo (mgr.24625) 10840 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:18:23.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: audit 2021-03-10T17:18:22.620781+0000 mon.a (mon.0) 3608 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.845 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.621041+0000 mgr.gibba032.whopeo (mgr.24625) 10841 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:18:23.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.621093+0000 mgr.gibba032.whopeo (mgr.24625) 10842 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:18:23.846 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[30791]: cephadm 2021-03-10T17:18:22.621242+0000 mgr.gibba032.whopeo (mgr.24625) 10843 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:23.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: audit 2021-03-10T17:18:22.620781+0000 mon.a (mon.0) 3608 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:23.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.621041+0000 mgr.gibba032.whopeo (mgr.24625) 10841 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:18:23.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.621093+0000 mgr.gibba032.whopeo (mgr.24625) 10842 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:18:23.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:23 gibba011 bash[31774]: cephadm 2021-03-10T17:18:22.621242+0000 mgr.gibba032.whopeo (mgr.24625) 10843 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:24.757 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:24 gibba032 bash[24004]: cluster 2021-03-10T17:18:23.509433+0000 mgr.gibba032.whopeo (mgr.24625) 10844 : cluster [DBG] pgmap v6174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:24.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:24 gibba011 bash[30791]: cluster 2021-03-10T17:18:23.509433+0000 mgr.gibba032.whopeo (mgr.24625) 10844 : cluster [DBG] pgmap v6174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:24.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:24 gibba011 bash[31774]: cluster 2021-03-10T17:18:23.509433+0000 mgr.gibba032.whopeo (mgr.24625) 10844 : cluster [DBG] pgmap v6174: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:26.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:26 gibba011 bash[30791]: cluster 2021-03-10T17:18:25.510829+0000 mgr.gibba032.whopeo (mgr.24625) 10845 : cluster [DBG] pgmap v6175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:26.829 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:26 gibba011 bash[31774]: cluster 2021-03-10T17:18:25.510829+0000 mgr.gibba032.whopeo (mgr.24625) 10845 : cluster [DBG] pgmap v6175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:26 gibba032 bash[24004]: cluster 2021-03-10T17:18:25.510829+0000 mgr.gibba032.whopeo (mgr.24625) 10845 : cluster [DBG] pgmap v6175: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:29.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:28 gibba032 bash[24004]: cluster 2021-03-10T17:18:27.512418+0000 mgr.gibba032.whopeo (mgr.24625) 10846 : cluster [DBG] pgmap v6176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:28 gibba011 bash[30791]: cluster 2021-03-10T17:18:27.512418+0000 mgr.gibba032.whopeo (mgr.24625) 10846 : cluster [DBG] pgmap v6176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:28 gibba011 bash[31774]: cluster 2021-03-10T17:18:27.512418+0000 mgr.gibba032.whopeo (mgr.24625) 10846 : cluster [DBG] pgmap v6176: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:30 gibba032 bash[24004]: cluster 2021-03-10T17:18:29.513943+0000 mgr.gibba032.whopeo (mgr.24625) 10847 : cluster [DBG] pgmap v6177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:30 gibba011 bash[30791]: cluster 2021-03-10T17:18:29.513943+0000 mgr.gibba032.whopeo (mgr.24625) 10847 : cluster [DBG] pgmap v6177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:30 gibba011 bash[31774]: cluster 2021-03-10T17:18:29.513943+0000 mgr.gibba032.whopeo (mgr.24625) 10847 : cluster [DBG] pgmap v6177: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:32 gibba011 bash[30791]: cluster 2021-03-10T17:18:31.514854+0000 mgr.gibba032.whopeo (mgr.24625) 10848 : cluster [DBG] pgmap v6178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:32 gibba011 bash[31774]: cluster 2021-03-10T17:18:31.514854+0000 mgr.gibba032.whopeo (mgr.24625) 10848 : cluster [DBG] pgmap v6178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:32 gibba032 bash[24004]: cluster 2021-03-10T17:18:31.514854+0000 mgr.gibba032.whopeo (mgr.24625) 10848 : cluster [DBG] pgmap v6178: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:35.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:34 gibba032 bash[24004]: cluster 2021-03-10T17:18:33.516283+0000 mgr.gibba032.whopeo (mgr.24625) 10849 : cluster [DBG] pgmap v6179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:34 gibba011 bash[30791]: cluster 2021-03-10T17:18:33.516283+0000 mgr.gibba032.whopeo (mgr.24625) 10849 : cluster [DBG] pgmap v6179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:34 gibba011 bash[31774]: cluster 2021-03-10T17:18:33.516283+0000 mgr.gibba032.whopeo (mgr.24625) 10849 : cluster [DBG] pgmap v6179: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:36 gibba032 bash[24004]: cluster 2021-03-10T17:18:35.518039+0000 mgr.gibba032.whopeo (mgr.24625) 10850 : cluster [DBG] pgmap v6180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:36 gibba011 bash[30791]: cluster 2021-03-10T17:18:35.518039+0000 mgr.gibba032.whopeo (mgr.24625) 10850 : cluster [DBG] pgmap v6180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:36 gibba011 bash[31774]: cluster 2021-03-10T17:18:35.518039+0000 mgr.gibba032.whopeo (mgr.24625) 10850 : cluster [DBG] pgmap v6180: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:39.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:38 gibba032 bash[24004]: cluster 2021-03-10T17:18:37.519100+0000 mgr.gibba032.whopeo (mgr.24625) 10851 : cluster [DBG] pgmap v6181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:38 gibba032 bash[24004]: cephadm 2021-03-10T17:18:37.636732+0000 mgr.gibba032.whopeo (mgr.24625) 10852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:38 gibba011 bash[30791]: cluster 2021-03-10T17:18:37.519100+0000 mgr.gibba032.whopeo (mgr.24625) 10851 : cluster [DBG] pgmap v6181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:38 gibba011 bash[30791]: cephadm 2021-03-10T17:18:37.636732+0000 mgr.gibba032.whopeo (mgr.24625) 10852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:38 gibba011 bash[31774]: cluster 2021-03-10T17:18:37.519100+0000 mgr.gibba032.whopeo (mgr.24625) 10851 : cluster [DBG] pgmap v6181: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:38 gibba011 bash[31774]: cephadm 2021-03-10T17:18:37.636732+0000 mgr.gibba032.whopeo (mgr.24625) 10852 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:41.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:40 gibba032 bash[24004]: cluster 2021-03-10T17:18:39.520751+0000 mgr.gibba032.whopeo (mgr.24625) 10853 : cluster [DBG] pgmap v6182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:40 gibba011 bash[30791]: cluster 2021-03-10T17:18:39.520751+0000 mgr.gibba032.whopeo (mgr.24625) 10853 : cluster [DBG] pgmap v6182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:40 gibba011 bash[31774]: cluster 2021-03-10T17:18:39.520751+0000 mgr.gibba032.whopeo (mgr.24625) 10853 : cluster [DBG] pgmap v6182: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:42 gibba011 bash[30791]: cluster 2021-03-10T17:18:41.522590+0000 mgr.gibba032.whopeo (mgr.24625) 10854 : cluster [DBG] pgmap v6183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:42.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:42 gibba011 bash[31774]: cluster 2021-03-10T17:18:41.522590+0000 mgr.gibba032.whopeo (mgr.24625) 10854 : cluster [DBG] pgmap v6183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:42 gibba032 bash[24004]: cluster 2021-03-10T17:18:41.522590+0000 mgr.gibba032.whopeo (mgr.24625) 10854 : cluster [DBG] pgmap v6183: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:44 gibba032 bash[24004]: cluster 2021-03-10T17:18:43.523462+0000 mgr.gibba032.whopeo (mgr.24625) 10855 : cluster [DBG] pgmap v6184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:44 gibba011 bash[30791]: cluster 2021-03-10T17:18:43.523462+0000 mgr.gibba032.whopeo (mgr.24625) 10855 : cluster [DBG] pgmap v6184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:44 gibba011 bash[31774]: cluster 2021-03-10T17:18:43.523462+0000 mgr.gibba032.whopeo (mgr.24625) 10855 : cluster [DBG] pgmap v6184: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:45.633 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:18:45.859 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:18:45.860 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:18:45.860 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:18:45.860 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:18:45.860 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:18:45.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:18:45.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:18:45.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:18:45.861 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:18:45.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:18:45.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:18:45.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:18:45.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:18:45.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:18:45.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:18:45.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:18:45.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:18:45.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:18:45.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:18:45.864 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:18:45.864 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:18:46.114 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:18:46.115 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:18:46.115 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:18:46.115 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:46.115 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:18:46.116 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:18:46.116 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:46.116 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:18:46.116 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:18:46.116 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:18:46.117 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:18:46.117 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:18:46.117 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:18:46.117 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:18:46.118 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:18:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:46 gibba032 bash[24004]: cluster 2021-03-10T17:18:45.525004+0000 mgr.gibba032.whopeo (mgr.24625) 10856 : cluster [DBG] pgmap v6185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:46 gibba032 bash[24004]: audit 2021-03-10T17:18:45.623146+0000 mgr.gibba032.whopeo (mgr.24625) 10857 : audit [DBG] from='client.51327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:46 gibba032 bash[24004]: audit 2021-03-10T17:18:45.845605+0000 mgr.gibba032.whopeo (mgr.24625) 10858 : audit [DBG] from='client.61315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:46 gibba032 bash[24004]: audit 2021-03-10T17:18:46.114692+0000 mon.c (mon.1) 208 : audit [DBG] from='client.? 172.21.2.111:0/3607266812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[30791]: cluster 2021-03-10T17:18:45.525004+0000 mgr.gibba032.whopeo (mgr.24625) 10856 : cluster [DBG] pgmap v6185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:47.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[30791]: audit 2021-03-10T17:18:45.623146+0000 mgr.gibba032.whopeo (mgr.24625) 10857 : audit [DBG] from='client.51327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[30791]: audit 2021-03-10T17:18:45.845605+0000 mgr.gibba032.whopeo (mgr.24625) 10858 : audit [DBG] from='client.61315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[30791]: audit 2021-03-10T17:18:46.114692+0000 mon.c (mon.1) 208 : audit [DBG] from='client.? 172.21.2.111:0/3607266812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:47.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[31774]: cluster 2021-03-10T17:18:45.525004+0000 mgr.gibba032.whopeo (mgr.24625) 10856 : cluster [DBG] pgmap v6185: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:47.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[31774]: audit 2021-03-10T17:18:45.623146+0000 mgr.gibba032.whopeo (mgr.24625) 10857 : audit [DBG] from='client.51327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[31774]: audit 2021-03-10T17:18:45.845605+0000 mgr.gibba032.whopeo (mgr.24625) 10858 : audit [DBG] from='client.61315 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:18:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:46 gibba011 bash[31774]: audit 2021-03-10T17:18:46.114692+0000 mon.c (mon.1) 208 : audit [DBG] from='client.? 172.21.2.111:0/3607266812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:18:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:48 gibba032 bash[24004]: cluster 2021-03-10T17:18:47.526756+0000 mgr.gibba032.whopeo (mgr.24625) 10859 : cluster [DBG] pgmap v6186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:48 gibba011 bash[30791]: cluster 2021-03-10T17:18:47.526756+0000 mgr.gibba032.whopeo (mgr.24625) 10859 : cluster [DBG] pgmap v6186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:48 gibba011 bash[31774]: cluster 2021-03-10T17:18:47.526756+0000 mgr.gibba032.whopeo (mgr.24625) 10859 : cluster [DBG] pgmap v6186: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:50 gibba032 bash[24004]: cluster 2021-03-10T17:18:49.528215+0000 mgr.gibba032.whopeo (mgr.24625) 10860 : cluster [DBG] pgmap v6187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:50 gibba011 bash[30791]: cluster 2021-03-10T17:18:49.528215+0000 mgr.gibba032.whopeo (mgr.24625) 10860 : cluster [DBG] pgmap v6187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:50 gibba011 bash[31774]: cluster 2021-03-10T17:18:49.528215+0000 mgr.gibba032.whopeo (mgr.24625) 10860 : cluster [DBG] pgmap v6187: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:52 gibba011 bash[30791]: cluster 2021-03-10T17:18:51.529698+0000 mgr.gibba032.whopeo (mgr.24625) 10861 : cluster [DBG] pgmap v6188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:52 gibba011 bash[31774]: cluster 2021-03-10T17:18:51.529698+0000 mgr.gibba032.whopeo (mgr.24625) 10861 : cluster [DBG] pgmap v6188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:52 gibba032 bash[24004]: cluster 2021-03-10T17:18:51.529698+0000 mgr.gibba032.whopeo (mgr.24625) 10861 : cluster [DBG] pgmap v6188: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:53.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:53 gibba011 bash[30791]: cephadm 2021-03-10T17:18:52.652792+0000 mgr.gibba032.whopeo (mgr.24625) 10862 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:53.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:53 gibba011 bash[31774]: cephadm 2021-03-10T17:18:52.652792+0000 mgr.gibba032.whopeo (mgr.24625) 10862 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:53.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:53 gibba032 bash[24004]: cephadm 2021-03-10T17:18:52.652792+0000 mgr.gibba032.whopeo (mgr.24625) 10862 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:18:54.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:54 gibba011 bash[30791]: cluster 2021-03-10T17:18:53.531351+0000 mgr.gibba032.whopeo (mgr.24625) 10863 : cluster [DBG] pgmap v6189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:54.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:54 gibba011 bash[31774]: cluster 2021-03-10T17:18:53.531351+0000 mgr.gibba032.whopeo (mgr.24625) 10863 : cluster [DBG] pgmap v6189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:54 gibba032 bash[24004]: cluster 2021-03-10T17:18:53.531351+0000 mgr.gibba032.whopeo (mgr.24625) 10863 : cluster [DBG] pgmap v6189: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:56 gibba032 bash[24004]: cluster 2021-03-10T17:18:55.532449+0000 mgr.gibba032.whopeo (mgr.24625) 10864 : cluster [DBG] pgmap v6190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:56 gibba011 bash[30791]: cluster 2021-03-10T17:18:55.532449+0000 mgr.gibba032.whopeo (mgr.24625) 10864 : cluster [DBG] pgmap v6190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:56 gibba011 bash[31774]: cluster 2021-03-10T17:18:55.532449+0000 mgr.gibba032.whopeo (mgr.24625) 10864 : cluster [DBG] pgmap v6190: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:58.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:57 gibba032 bash[24004]: audit 2021-03-10T17:18:57.429272+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:18:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:57 gibba032 bash[24004]: audit 2021-03-10T17:18:57.430005+0000 mon.a (mon.0) 3610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:18:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:57 gibba032 bash[24004]: audit 2021-03-10T17:18:57.596042+0000 mon.a (mon.0) 3611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:18:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[30791]: audit 2021-03-10T17:18:57.429272+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:18:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[30791]: audit 2021-03-10T17:18:57.430005+0000 mon.a (mon.0) 3610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:18:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[30791]: audit 2021-03-10T17:18:57.596042+0000 mon.a (mon.0) 3611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:18:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[31774]: audit 2021-03-10T17:18:57.429272+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:18:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[31774]: audit 2021-03-10T17:18:57.430005+0000 mon.a (mon.0) 3610 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:18:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:57 gibba011 bash[31774]: audit 2021-03-10T17:18:57.596042+0000 mon.a (mon.0) 3611 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:18:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:58 gibba032 bash[24004]: cluster 2021-03-10T17:18:57.534370+0000 mgr.gibba032.whopeo (mgr.24625) 10865 : cluster [DBG] pgmap v6191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:58 gibba032 bash[24004]: audit 2021-03-10T17:18:57.598256+0000 mon.a (mon.0) 3612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:18:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:58 gibba032 bash[24004]: audit 2021-03-10T17:18:57.599021+0000 mon.a (mon.0) 3613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:18:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:18:58 gibba032 bash[24004]: audit 2021-03-10T17:18:57.599822+0000 mon.a (mon.0) 3614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:18:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[30791]: cluster 2021-03-10T17:18:57.534370+0000 mgr.gibba032.whopeo (mgr.24625) 10865 : cluster [DBG] pgmap v6191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:59.243 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[30791]: audit 2021-03-10T17:18:57.598256+0000 mon.a (mon.0) 3612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:18:59.244 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[30791]: audit 2021-03-10T17:18:57.599021+0000 mon.a (mon.0) 3613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:18:59.245 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[30791]: audit 2021-03-10T17:18:57.599822+0000 mon.a (mon.0) 3614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:18:59.246 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[31774]: cluster 2021-03-10T17:18:57.534370+0000 mgr.gibba032.whopeo (mgr.24625) 10865 : cluster [DBG] pgmap v6191: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:18:59.246 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[31774]: audit 2021-03-10T17:18:57.598256+0000 mon.a (mon.0) 3612 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:18:59.247 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[31774]: audit 2021-03-10T17:18:57.599021+0000 mon.a (mon.0) 3613 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:18:59.248 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:18:58 gibba011 bash[31774]: audit 2021-03-10T17:18:57.599822+0000 mon.a (mon.0) 3614 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:19:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:00 gibba032 bash[24004]: cluster 2021-03-10T17:18:59.535868+0000 mgr.gibba032.whopeo (mgr.24625) 10866 : cluster [DBG] pgmap v6192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:00 gibba011 bash[30791]: cluster 2021-03-10T17:18:59.535868+0000 mgr.gibba032.whopeo (mgr.24625) 10866 : cluster [DBG] pgmap v6192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:00 gibba011 bash[31774]: cluster 2021-03-10T17:18:59.535868+0000 mgr.gibba032.whopeo (mgr.24625) 10866 : cluster [DBG] pgmap v6192: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:02 gibba011 bash[30791]: cluster 2021-03-10T17:19:01.537026+0000 mgr.gibba032.whopeo (mgr.24625) 10867 : cluster [DBG] pgmap v6193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:02.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:02 gibba011 bash[31774]: cluster 2021-03-10T17:19:01.537026+0000 mgr.gibba032.whopeo (mgr.24625) 10867 : cluster [DBG] pgmap v6193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:02 gibba032 bash[24004]: cluster 2021-03-10T17:19:01.537026+0000 mgr.gibba032.whopeo (mgr.24625) 10867 : cluster [DBG] pgmap v6193: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:04 gibba032 bash[24004]: cluster 2021-03-10T17:19:03.538537+0000 mgr.gibba032.whopeo (mgr.24625) 10868 : cluster [DBG] pgmap v6194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:04 gibba011 bash[30791]: cluster 2021-03-10T17:19:03.538537+0000 mgr.gibba032.whopeo (mgr.24625) 10868 : cluster [DBG] pgmap v6194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:04 gibba011 bash[31774]: cluster 2021-03-10T17:19:03.538537+0000 mgr.gibba032.whopeo (mgr.24625) 10868 : cluster [DBG] pgmap v6194: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:06 gibba032 bash[24004]: cluster 2021-03-10T17:19:05.540124+0000 mgr.gibba032.whopeo (mgr.24625) 10869 : cluster [DBG] pgmap v6195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:06 gibba011 bash[30791]: cluster 2021-03-10T17:19:05.540124+0000 mgr.gibba032.whopeo (mgr.24625) 10869 : cluster [DBG] pgmap v6195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:06 gibba011 bash[31774]: cluster 2021-03-10T17:19:05.540124+0000 mgr.gibba032.whopeo (mgr.24625) 10869 : cluster [DBG] pgmap v6195: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:08 gibba032 bash[24004]: cluster 2021-03-10T17:19:07.541297+0000 mgr.gibba032.whopeo (mgr.24625) 10870 : cluster [DBG] pgmap v6196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:08 gibba032 bash[24004]: cephadm 2021-03-10T17:19:07.664540+0000 mgr.gibba032.whopeo (mgr.24625) 10871 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:08 gibba011 bash[30791]: cluster 2021-03-10T17:19:07.541297+0000 mgr.gibba032.whopeo (mgr.24625) 10870 : cluster [DBG] pgmap v6196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:08 gibba011 bash[30791]: cephadm 2021-03-10T17:19:07.664540+0000 mgr.gibba032.whopeo (mgr.24625) 10871 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:08 gibba011 bash[31774]: cluster 2021-03-10T17:19:07.541297+0000 mgr.gibba032.whopeo (mgr.24625) 10870 : cluster [DBG] pgmap v6196: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:08 gibba011 bash[31774]: cephadm 2021-03-10T17:19:07.664540+0000 mgr.gibba032.whopeo (mgr.24625) 10871 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:10 gibba032 bash[24004]: cluster 2021-03-10T17:19:09.542744+0000 mgr.gibba032.whopeo (mgr.24625) 10872 : cluster [DBG] pgmap v6197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:11.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:10 gibba011 bash[30791]: cluster 2021-03-10T17:19:09.542744+0000 mgr.gibba032.whopeo (mgr.24625) 10872 : cluster [DBG] pgmap v6197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:10 gibba011 bash[31774]: cluster 2021-03-10T17:19:09.542744+0000 mgr.gibba032.whopeo (mgr.24625) 10872 : cluster [DBG] pgmap v6197: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:12 gibba011 bash[30791]: cluster 2021-03-10T17:19:11.544520+0000 mgr.gibba032.whopeo (mgr.24625) 10873 : cluster [DBG] pgmap v6198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:12 gibba011 bash[31774]: cluster 2021-03-10T17:19:11.544520+0000 mgr.gibba032.whopeo (mgr.24625) 10873 : cluster [DBG] pgmap v6198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:12 gibba032 bash[24004]: cluster 2021-03-10T17:19:11.544520+0000 mgr.gibba032.whopeo (mgr.24625) 10873 : cluster [DBG] pgmap v6198: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:14 gibba032 bash[24004]: cluster 2021-03-10T17:19:13.545592+0000 mgr.gibba032.whopeo (mgr.24625) 10874 : cluster [DBG] pgmap v6199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:14 gibba011 bash[30791]: cluster 2021-03-10T17:19:13.545592+0000 mgr.gibba032.whopeo (mgr.24625) 10874 : cluster [DBG] pgmap v6199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:14 gibba011 bash[31774]: cluster 2021-03-10T17:19:13.545592+0000 mgr.gibba032.whopeo (mgr.24625) 10874 : cluster [DBG] pgmap v6199: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:16.360 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:19:16.585 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:19:16.585 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:19:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:19:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:19:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:19:16.586 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:19:16.587 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:19:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:19:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:19:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:19:16.588 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:19:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:19:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:19:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:19:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:19:16.589 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:19:16.837 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:19:16.837 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:19:16.838 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:19:16.838 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:16.838 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:19:16.838 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:19:16.838 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:16.839 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:19:16.839 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:19:16.839 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:16.839 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:19:16.839 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:19:16.840 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:19:16.840 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:19:16.840 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:19:17.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:16 gibba032 bash[24004]: cluster 2021-03-10T17:19:15.547250+0000 mgr.gibba032.whopeo (mgr.24625) 10875 : cluster [DBG] pgmap v6200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:16 gibba011 bash[30791]: cluster 2021-03-10T17:19:15.547250+0000 mgr.gibba032.whopeo (mgr.24625) 10875 : cluster [DBG] pgmap v6200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:16 gibba011 bash[31774]: cluster 2021-03-10T17:19:15.547250+0000 mgr.gibba032.whopeo (mgr.24625) 10875 : cluster [DBG] pgmap v6200: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:18.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:17 gibba032 bash[24004]: audit 2021-03-10T17:19:16.349189+0000 mgr.gibba032.whopeo (mgr.24625) 10876 : audit [DBG] from='client.51345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:17 gibba032 bash[24004]: audit 2021-03-10T17:19:16.571210+0000 mgr.gibba032.whopeo (mgr.24625) 10877 : audit [DBG] from='client.61333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:17 gibba032 bash[24004]: audit 2021-03-10T17:19:16.837100+0000 mon.c (mon.1) 209 : audit [DBG] from='client.? 172.21.2.111:0/1414043229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[30791]: audit 2021-03-10T17:19:16.349189+0000 mgr.gibba032.whopeo (mgr.24625) 10876 : audit [DBG] from='client.51345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[30791]: audit 2021-03-10T17:19:16.571210+0000 mgr.gibba032.whopeo (mgr.24625) 10877 : audit [DBG] from='client.61333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[30791]: audit 2021-03-10T17:19:16.837100+0000 mon.c (mon.1) 209 : audit [DBG] from='client.? 172.21.2.111:0/1414043229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:18.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[31774]: audit 2021-03-10T17:19:16.349189+0000 mgr.gibba032.whopeo (mgr.24625) 10876 : audit [DBG] from='client.51345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[31774]: audit 2021-03-10T17:19:16.571210+0000 mgr.gibba032.whopeo (mgr.24625) 10877 : audit [DBG] from='client.61333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:18.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:17 gibba011 bash[31774]: audit 2021-03-10T17:19:16.837100+0000 mon.c (mon.1) 209 : audit [DBG] from='client.? 172.21.2.111:0/1414043229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:18 gibba032 bash[24004]: cluster 2021-03-10T17:19:17.548957+0000 mgr.gibba032.whopeo (mgr.24625) 10878 : cluster [DBG] pgmap v6201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:18 gibba011 bash[30791]: cluster 2021-03-10T17:19:17.548957+0000 mgr.gibba032.whopeo (mgr.24625) 10878 : cluster [DBG] pgmap v6201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:18 gibba011 bash[31774]: cluster 2021-03-10T17:19:17.548957+0000 mgr.gibba032.whopeo (mgr.24625) 10878 : cluster [DBG] pgmap v6201: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:20 gibba032 bash[24004]: cluster 2021-03-10T17:19:19.549891+0000 mgr.gibba032.whopeo (mgr.24625) 10879 : cluster [DBG] pgmap v6202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:20 gibba011 bash[30791]: cluster 2021-03-10T17:19:19.549891+0000 mgr.gibba032.whopeo (mgr.24625) 10879 : cluster [DBG] pgmap v6202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:20 gibba011 bash[31774]: cluster 2021-03-10T17:19:19.549891+0000 mgr.gibba032.whopeo (mgr.24625) 10879 : cluster [DBG] pgmap v6202: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:22 gibba011 bash[30791]: cluster 2021-03-10T17:19:21.551496+0000 mgr.gibba032.whopeo (mgr.24625) 10880 : cluster [DBG] pgmap v6203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:22.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:22 gibba011 bash[31774]: cluster 2021-03-10T17:19:21.551496+0000 mgr.gibba032.whopeo (mgr.24625) 10880 : cluster [DBG] pgmap v6203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:22 gibba032 bash[24004]: cluster 2021-03-10T17:19:21.551496+0000 mgr.gibba032.whopeo (mgr.24625) 10880 : cluster [DBG] pgmap v6203: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.681337+0000 mgr.gibba032.whopeo (mgr.24625) 10881 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:19:24.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.682886+0000 mgr.gibba032.whopeo (mgr.24625) 10882 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:19:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.905449+0000 mon.a (mon.0) 3615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:19:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.906133+0000 mgr.gibba032.whopeo (mgr.24625) 10883 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:19:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.908883+0000 mon.a (mon.0) 3616 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:19:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.910375+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.911661+0000 mon.a (mon.0) 3618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.912868+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.915637+0000 mon.a (mon.0) 3620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:23 gibba032 bash[24004]: audit 2021-03-10T17:19:22.917173+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.681337+0000 mgr.gibba032.whopeo (mgr.24625) 10881 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:19:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.682886+0000 mgr.gibba032.whopeo (mgr.24625) 10882 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:19:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.905449+0000 mon.a (mon.0) 3615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:19:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.906133+0000 mgr.gibba032.whopeo (mgr.24625) 10883 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:19:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.908883+0000 mon.a (mon.0) 3616 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:19:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.910375+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.911661+0000 mon.a (mon.0) 3618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.912868+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.915637+0000 mon.a (mon.0) 3620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[30791]: audit 2021-03-10T17:19:22.917173+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.681337+0000 mgr.gibba032.whopeo (mgr.24625) 10881 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:19:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.682886+0000 mgr.gibba032.whopeo (mgr.24625) 10882 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:19:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.905449+0000 mon.a (mon.0) 3615 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:19:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.906133+0000 mgr.gibba032.whopeo (mgr.24625) 10883 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:19:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.908883+0000 mon.a (mon.0) 3616 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:19:24.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.910375+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.911661+0000 mon.a (mon.0) 3618 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.912868+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.915637+0000 mon.a (mon.0) 3620 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:24.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:23 gibba011 bash[31774]: audit 2021-03-10T17:19:22.917173+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.908057+0000 mgr.gibba032.whopeo (mgr.24625) 10884 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:19:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.909253+0000 mgr.gibba032.whopeo (mgr.24625) 10885 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:19:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.909293+0000 mgr.gibba032.whopeo (mgr.24625) 10886 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:19:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.910672+0000 mgr.gibba032.whopeo (mgr.24625) 10887 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:19:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.910710+0000 mgr.gibba032.whopeo (mgr.24625) 10888 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:19:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.911829+0000 mgr.gibba032.whopeo (mgr.24625) 10889 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:19:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.913259+0000 mgr.gibba032.whopeo (mgr.24625) 10890 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:19:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.913297+0000 mgr.gibba032.whopeo (mgr.24625) 10891 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:19:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.914562+0000 mgr.gibba032.whopeo (mgr.24625) 10892 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:19:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.914896+0000 mgr.gibba032.whopeo (mgr.24625) 10893 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:19:25.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.916246+0000 mgr.gibba032.whopeo (mgr.24625) 10894 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:19:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.916289+0000 mgr.gibba032.whopeo (mgr.24625) 10895 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:19:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cephadm 2021-03-10T17:19:22.916405+0000 mgr.gibba032.whopeo (mgr.24625) 10896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:25.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:24 gibba032 bash[24004]: cluster 2021-03-10T17:19:23.553173+0000 mgr.gibba032.whopeo (mgr.24625) 10897 : cluster [DBG] pgmap v6204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.908057+0000 mgr.gibba032.whopeo (mgr.24625) 10884 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:19:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.909253+0000 mgr.gibba032.whopeo (mgr.24625) 10885 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:19:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.909293+0000 mgr.gibba032.whopeo (mgr.24625) 10886 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:19:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.910672+0000 mgr.gibba032.whopeo (mgr.24625) 10887 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:19:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.910710+0000 mgr.gibba032.whopeo (mgr.24625) 10888 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:19:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.911829+0000 mgr.gibba032.whopeo (mgr.24625) 10889 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:19:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.913259+0000 mgr.gibba032.whopeo (mgr.24625) 10890 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:19:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.913297+0000 mgr.gibba032.whopeo (mgr.24625) 10891 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:19:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.914562+0000 mgr.gibba032.whopeo (mgr.24625) 10892 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:19:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.914896+0000 mgr.gibba032.whopeo (mgr.24625) 10893 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:19:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.916246+0000 mgr.gibba032.whopeo (mgr.24625) 10894 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:19:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.916289+0000 mgr.gibba032.whopeo (mgr.24625) 10895 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:19:25.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cephadm 2021-03-10T17:19:22.916405+0000 mgr.gibba032.whopeo (mgr.24625) 10896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:25.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[30791]: cluster 2021-03-10T17:19:23.553173+0000 mgr.gibba032.whopeo (mgr.24625) 10897 : cluster [DBG] pgmap v6204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:25.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.908057+0000 mgr.gibba032.whopeo (mgr.24625) 10884 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:19:25.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.909253+0000 mgr.gibba032.whopeo (mgr.24625) 10885 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:19:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.909293+0000 mgr.gibba032.whopeo (mgr.24625) 10886 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:19:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.910672+0000 mgr.gibba032.whopeo (mgr.24625) 10887 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:19:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.910710+0000 mgr.gibba032.whopeo (mgr.24625) 10888 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:19:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.911829+0000 mgr.gibba032.whopeo (mgr.24625) 10889 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:19:25.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.913259+0000 mgr.gibba032.whopeo (mgr.24625) 10890 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:19:25.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.913297+0000 mgr.gibba032.whopeo (mgr.24625) 10891 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:19:25.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.914562+0000 mgr.gibba032.whopeo (mgr.24625) 10892 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:19:25.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.914896+0000 mgr.gibba032.whopeo (mgr.24625) 10893 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:19:25.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.916246+0000 mgr.gibba032.whopeo (mgr.24625) 10894 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:19:25.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.916289+0000 mgr.gibba032.whopeo (mgr.24625) 10895 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:19:25.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cephadm 2021-03-10T17:19:22.916405+0000 mgr.gibba032.whopeo (mgr.24625) 10896 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:25.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:24 gibba011 bash[31774]: cluster 2021-03-10T17:19:23.553173+0000 mgr.gibba032.whopeo (mgr.24625) 10897 : cluster [DBG] pgmap v6204: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:26.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:25 gibba032 bash[24004]: cluster 2021-03-10T17:19:25.554249+0000 mgr.gibba032.whopeo (mgr.24625) 10898 : cluster [DBG] pgmap v6205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:25 gibba011 bash[30791]: cluster 2021-03-10T17:19:25.554249+0000 mgr.gibba032.whopeo (mgr.24625) 10898 : cluster [DBG] pgmap v6205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:26.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:25 gibba011 bash[31774]: cluster 2021-03-10T17:19:25.554249+0000 mgr.gibba032.whopeo (mgr.24625) 10898 : cluster [DBG] pgmap v6205: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:28 gibba032 bash[24004]: cluster 2021-03-10T17:19:27.555914+0000 mgr.gibba032.whopeo (mgr.24625) 10899 : cluster [DBG] pgmap v6206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:28 gibba011 bash[30791]: cluster 2021-03-10T17:19:27.555914+0000 mgr.gibba032.whopeo (mgr.24625) 10899 : cluster [DBG] pgmap v6206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:28 gibba011 bash[31774]: cluster 2021-03-10T17:19:27.555914+0000 mgr.gibba032.whopeo (mgr.24625) 10899 : cluster [DBG] pgmap v6206: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:30 gibba032 bash[24004]: cluster 2021-03-10T17:19:29.557408+0000 mgr.gibba032.whopeo (mgr.24625) 10900 : cluster [DBG] pgmap v6207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:30 gibba011 bash[30791]: cluster 2021-03-10T17:19:29.557408+0000 mgr.gibba032.whopeo (mgr.24625) 10900 : cluster [DBG] pgmap v6207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:30 gibba011 bash[31774]: cluster 2021-03-10T17:19:29.557408+0000 mgr.gibba032.whopeo (mgr.24625) 10900 : cluster [DBG] pgmap v6207: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:32 gibba011 bash[30791]: cluster 2021-03-10T17:19:31.558672+0000 mgr.gibba032.whopeo (mgr.24625) 10901 : cluster [DBG] pgmap v6208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:32 gibba011 bash[31774]: cluster 2021-03-10T17:19:31.558672+0000 mgr.gibba032.whopeo (mgr.24625) 10901 : cluster [DBG] pgmap v6208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:32 gibba032 bash[24004]: cluster 2021-03-10T17:19:31.558672+0000 mgr.gibba032.whopeo (mgr.24625) 10901 : cluster [DBG] pgmap v6208: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:34 gibba032 bash[24004]: cluster 2021-03-10T17:19:33.560162+0000 mgr.gibba032.whopeo (mgr.24625) 10902 : cluster [DBG] pgmap v6209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:34 gibba011 bash[30791]: cluster 2021-03-10T17:19:33.560162+0000 mgr.gibba032.whopeo (mgr.24625) 10902 : cluster [DBG] pgmap v6209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:35.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:34 gibba011 bash[31774]: cluster 2021-03-10T17:19:33.560162+0000 mgr.gibba032.whopeo (mgr.24625) 10902 : cluster [DBG] pgmap v6209: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:37.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:36 gibba032 bash[24004]: cluster 2021-03-10T17:19:35.561469+0000 mgr.gibba032.whopeo (mgr.24625) 10903 : cluster [DBG] pgmap v6210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:36 gibba011 bash[30791]: cluster 2021-03-10T17:19:35.561469+0000 mgr.gibba032.whopeo (mgr.24625) 10903 : cluster [DBG] pgmap v6210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:36 gibba011 bash[31774]: cluster 2021-03-10T17:19:35.561469+0000 mgr.gibba032.whopeo (mgr.24625) 10903 : cluster [DBG] pgmap v6210: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:39.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:38 gibba032 bash[24004]: cluster 2021-03-10T17:19:37.563277+0000 mgr.gibba032.whopeo (mgr.24625) 10904 : cluster [DBG] pgmap v6211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:38 gibba011 bash[30791]: cluster 2021-03-10T17:19:37.563277+0000 mgr.gibba032.whopeo (mgr.24625) 10904 : cluster [DBG] pgmap v6211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:39.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:38 gibba011 bash[31774]: cluster 2021-03-10T17:19:37.563277+0000 mgr.gibba032.whopeo (mgr.24625) 10904 : cluster [DBG] pgmap v6211: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:40.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:39 gibba032 bash[24004]: cephadm 2021-03-10T17:19:37.932175+0000 mgr.gibba032.whopeo (mgr.24625) 10905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:40.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:39 gibba011 bash[30791]: cephadm 2021-03-10T17:19:37.932175+0000 mgr.gibba032.whopeo (mgr.24625) 10905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:40.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:39 gibba011 bash[31774]: cephadm 2021-03-10T17:19:37.932175+0000 mgr.gibba032.whopeo (mgr.24625) 10905 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:41.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:40 gibba032 bash[24004]: cluster 2021-03-10T17:19:39.564849+0000 mgr.gibba032.whopeo (mgr.24625) 10906 : cluster [DBG] pgmap v6212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:40 gibba011 bash[30791]: cluster 2021-03-10T17:19:39.564849+0000 mgr.gibba032.whopeo (mgr.24625) 10906 : cluster [DBG] pgmap v6212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:40 gibba011 bash[31774]: cluster 2021-03-10T17:19:39.564849+0000 mgr.gibba032.whopeo (mgr.24625) 10906 : cluster [DBG] pgmap v6212: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:42 gibba011 bash[30791]: cluster 2021-03-10T17:19:41.566572+0000 mgr.gibba032.whopeo (mgr.24625) 10907 : cluster [DBG] pgmap v6213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:42.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:42 gibba011 bash[31774]: cluster 2021-03-10T17:19:41.566572+0000 mgr.gibba032.whopeo (mgr.24625) 10907 : cluster [DBG] pgmap v6213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:42 gibba032 bash[24004]: cluster 2021-03-10T17:19:41.566572+0000 mgr.gibba032.whopeo (mgr.24625) 10907 : cluster [DBG] pgmap v6213: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:45.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:44 gibba032 bash[24004]: cluster 2021-03-10T17:19:43.567913+0000 mgr.gibba032.whopeo (mgr.24625) 10908 : cluster [DBG] pgmap v6214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:44 gibba011 bash[30791]: cluster 2021-03-10T17:19:43.567913+0000 mgr.gibba032.whopeo (mgr.24625) 10908 : cluster [DBG] pgmap v6214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:44 gibba011 bash[31774]: cluster 2021-03-10T17:19:43.567913+0000 mgr.gibba032.whopeo (mgr.24625) 10908 : cluster [DBG] pgmap v6214: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:46 gibba032 bash[24004]: cluster 2021-03-10T17:19:45.569230+0000 mgr.gibba032.whopeo (mgr.24625) 10909 : cluster [DBG] pgmap v6215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:46 gibba011 bash[30791]: cluster 2021-03-10T17:19:45.569230+0000 mgr.gibba032.whopeo (mgr.24625) 10909 : cluster [DBG] pgmap v6215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:46 gibba011 bash[31774]: cluster 2021-03-10T17:19:45.569230+0000 mgr.gibba032.whopeo (mgr.24625) 10909 : cluster [DBG] pgmap v6215: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:47.080 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:19:47.310 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:19:47.310 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:19:47.311 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:19:47.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:19:47.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:19:47.311 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:19:47.311 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:19:47.312 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:19:47.312 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:19:47.312 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:19:47.312 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:19:47.313 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:19:47.313 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:19:47.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:19:47.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:19:47.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:19:47.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:19:47.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:19:47.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:19:47.317 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:19:47.317 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:19:47.561 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:19:47.561 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:19:47.561 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:19:47.562 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:47.562 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:19:47.562 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:19:47.562 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:47.562 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:19:47.563 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:19:47.563 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:19:47.563 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:19:47.563 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:19:47.563 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:19:47.564 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:19:47.564 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:19:48.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:47 gibba032 bash[24004]: audit 2021-03-10T17:19:47.560981+0000 mon.a (mon.0) 3622 : audit [DBG] from='client.? 172.21.2.111:0/9018468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:48.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:47 gibba011 bash[30791]: audit 2021-03-10T17:19:47.560981+0000 mon.a (mon.0) 3622 : audit [DBG] from='client.? 172.21.2.111:0/9018468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:48.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:47 gibba011 bash[31774]: audit 2021-03-10T17:19:47.560981+0000 mon.a (mon.0) 3622 : audit [DBG] from='client.? 172.21.2.111:0/9018468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:19:49.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:48 gibba032 bash[24004]: audit 2021-03-10T17:19:47.069539+0000 mgr.gibba032.whopeo (mgr.24625) 10910 : audit [DBG] from='client.61345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:48 gibba032 bash[24004]: audit 2021-03-10T17:19:47.295972+0000 mgr.gibba032.whopeo (mgr.24625) 10911 : audit [DBG] from='client.51369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:48 gibba032 bash[24004]: cluster 2021-03-10T17:19:47.570715+0000 mgr.gibba032.whopeo (mgr.24625) 10912 : cluster [DBG] pgmap v6216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[30791]: audit 2021-03-10T17:19:47.069539+0000 mgr.gibba032.whopeo (mgr.24625) 10910 : audit [DBG] from='client.61345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[30791]: audit 2021-03-10T17:19:47.295972+0000 mgr.gibba032.whopeo (mgr.24625) 10911 : audit [DBG] from='client.51369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[30791]: cluster 2021-03-10T17:19:47.570715+0000 mgr.gibba032.whopeo (mgr.24625) 10912 : cluster [DBG] pgmap v6216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:49.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[31774]: audit 2021-03-10T17:19:47.069539+0000 mgr.gibba032.whopeo (mgr.24625) 10910 : audit [DBG] from='client.61345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[31774]: audit 2021-03-10T17:19:47.295972+0000 mgr.gibba032.whopeo (mgr.24625) 10911 : audit [DBG] from='client.51369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:19:49.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:48 gibba011 bash[31774]: cluster 2021-03-10T17:19:47.570715+0000 mgr.gibba032.whopeo (mgr.24625) 10912 : cluster [DBG] pgmap v6216: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:50 gibba032 bash[24004]: cluster 2021-03-10T17:19:49.572102+0000 mgr.gibba032.whopeo (mgr.24625) 10913 : cluster [DBG] pgmap v6217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:50 gibba011 bash[30791]: cluster 2021-03-10T17:19:49.572102+0000 mgr.gibba032.whopeo (mgr.24625) 10913 : cluster [DBG] pgmap v6217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:51.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:50 gibba011 bash[31774]: cluster 2021-03-10T17:19:49.572102+0000 mgr.gibba032.whopeo (mgr.24625) 10913 : cluster [DBG] pgmap v6217: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:52.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:52 gibba011 bash[30791]: cluster 2021-03-10T17:19:51.573796+0000 mgr.gibba032.whopeo (mgr.24625) 10914 : cluster [DBG] pgmap v6218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:52.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:52 gibba011 bash[31774]: cluster 2021-03-10T17:19:51.573796+0000 mgr.gibba032.whopeo (mgr.24625) 10914 : cluster [DBG] pgmap v6218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:52 gibba032 bash[24004]: cluster 2021-03-10T17:19:51.573796+0000 mgr.gibba032.whopeo (mgr.24625) 10914 : cluster [DBG] pgmap v6218: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:54 gibba032 bash[24004]: cephadm 2021-03-10T17:19:52.948158+0000 mgr.gibba032.whopeo (mgr.24625) 10915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:54 gibba032 bash[24004]: cluster 2021-03-10T17:19:53.575276+0000 mgr.gibba032.whopeo (mgr.24625) 10916 : cluster [DBG] pgmap v6219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:54 gibba011 bash[30791]: cephadm 2021-03-10T17:19:52.948158+0000 mgr.gibba032.whopeo (mgr.24625) 10915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:54 gibba011 bash[30791]: cluster 2021-03-10T17:19:53.575276+0000 mgr.gibba032.whopeo (mgr.24625) 10916 : cluster [DBG] pgmap v6219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:54 gibba011 bash[31774]: cephadm 2021-03-10T17:19:52.948158+0000 mgr.gibba032.whopeo (mgr.24625) 10915 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:19:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:54 gibba011 bash[31774]: cluster 2021-03-10T17:19:53.575276+0000 mgr.gibba032.whopeo (mgr.24625) 10916 : cluster [DBG] pgmap v6219: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:56 gibba032 bash[24004]: cluster 2021-03-10T17:19:55.576992+0000 mgr.gibba032.whopeo (mgr.24625) 10917 : cluster [DBG] pgmap v6220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:56 gibba011 bash[30791]: cluster 2021-03-10T17:19:55.576992+0000 mgr.gibba032.whopeo (mgr.24625) 10917 : cluster [DBG] pgmap v6220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:56 gibba011 bash[31774]: cluster 2021-03-10T17:19:55.576992+0000 mgr.gibba032.whopeo (mgr.24625) 10917 : cluster [DBG] pgmap v6220: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.437251+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:19:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.437979+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:19:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.611896+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:19:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.612808+0000 mon.a (mon.0) 3626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:19:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.613565+0000 mon.a (mon.0) 3627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:19:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:57 gibba032 bash[24004]: audit 2021-03-10T17:19:57.614298+0000 mon.a (mon.0) 3628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:19:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.437251+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:19:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.437979+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:19:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.611896+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:19:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.612808+0000 mon.a (mon.0) 3626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:19:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.613565+0000 mon.a (mon.0) 3627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:19:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[30791]: audit 2021-03-10T17:19:57.614298+0000 mon.a (mon.0) 3628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:19:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.437251+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:19:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.437979+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:19:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.611896+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:19:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.612808+0000 mon.a (mon.0) 3626 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:19:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.613565+0000 mon.a (mon.0) 3627 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:19:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:57 gibba011 bash[31774]: audit 2021-03-10T17:19:57.614298+0000 mon.a (mon.0) 3628 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:19:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:19:58 gibba032 bash[24004]: cluster 2021-03-10T17:19:57.578730+0000 mgr.gibba032.whopeo (mgr.24625) 10918 : cluster [DBG] pgmap v6221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:19:58 gibba011 bash[30791]: cluster 2021-03-10T17:19:57.578730+0000 mgr.gibba032.whopeo (mgr.24625) 10918 : cluster [DBG] pgmap v6221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:19:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:19:58 gibba011 bash[31774]: cluster 2021-03-10T17:19:57.578730+0000 mgr.gibba032.whopeo (mgr.24625) 10918 : cluster [DBG] pgmap v6221: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:00 gibba011 bash[30791]: debug 2021-03-10T17:19:59.999+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:20:01.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:00 gibba032 bash[24004]: cluster 2021-03-10T17:19:59.580351+0000 mgr.gibba032.whopeo (mgr.24625) 10919 : cluster [DBG] pgmap v6222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:01.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:00 gibba032 bash[24004]: cluster 2021-03-10T17:20:00.000498+0000 mon.a (mon.0) 3629 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:20:01.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:00 gibba011 bash[30791]: cluster 2021-03-10T17:19:59.580351+0000 mgr.gibba032.whopeo (mgr.24625) 10919 : cluster [DBG] pgmap v6222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:00 gibba011 bash[30791]: cluster 2021-03-10T17:20:00.000498+0000 mon.a (mon.0) 3629 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:20:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:00 gibba011 bash[31774]: cluster 2021-03-10T17:19:59.580351+0000 mgr.gibba032.whopeo (mgr.24625) 10919 : cluster [DBG] pgmap v6222: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:00 gibba011 bash[31774]: cluster 2021-03-10T17:20:00.000498+0000 mon.a (mon.0) 3629 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:20:02.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:02 gibba011 bash[30791]: cluster 2021-03-10T17:20:01.582100+0000 mgr.gibba032.whopeo (mgr.24625) 10920 : cluster [DBG] pgmap v6223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:02.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:02 gibba011 bash[31774]: cluster 2021-03-10T17:20:01.582100+0000 mgr.gibba032.whopeo (mgr.24625) 10920 : cluster [DBG] pgmap v6223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:02 gibba032 bash[24004]: cluster 2021-03-10T17:20:01.582100+0000 mgr.gibba032.whopeo (mgr.24625) 10920 : cluster [DBG] pgmap v6223: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:04 gibba032 bash[24004]: cluster 2021-03-10T17:20:03.583755+0000 mgr.gibba032.whopeo (mgr.24625) 10921 : cluster [DBG] pgmap v6224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:04 gibba011 bash[30791]: cluster 2021-03-10T17:20:03.583755+0000 mgr.gibba032.whopeo (mgr.24625) 10921 : cluster [DBG] pgmap v6224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:04 gibba011 bash[31774]: cluster 2021-03-10T17:20:03.583755+0000 mgr.gibba032.whopeo (mgr.24625) 10921 : cluster [DBG] pgmap v6224: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:07.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:06 gibba032 bash[24004]: cluster 2021-03-10T17:20:05.585365+0000 mgr.gibba032.whopeo (mgr.24625) 10922 : cluster [DBG] pgmap v6225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:06 gibba011 bash[30791]: cluster 2021-03-10T17:20:05.585365+0000 mgr.gibba032.whopeo (mgr.24625) 10922 : cluster [DBG] pgmap v6225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:06 gibba011 bash[31774]: cluster 2021-03-10T17:20:05.585365+0000 mgr.gibba032.whopeo (mgr.24625) 10922 : cluster [DBG] pgmap v6225: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:08 gibba032 bash[24004]: cluster 2021-03-10T17:20:07.587100+0000 mgr.gibba032.whopeo (mgr.24625) 10923 : cluster [DBG] pgmap v6226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:08 gibba011 bash[30791]: cluster 2021-03-10T17:20:07.587100+0000 mgr.gibba032.whopeo (mgr.24625) 10923 : cluster [DBG] pgmap v6226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:08 gibba011 bash[31774]: cluster 2021-03-10T17:20:07.587100+0000 mgr.gibba032.whopeo (mgr.24625) 10923 : cluster [DBG] pgmap v6226: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:10.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:09 gibba032 bash[24004]: cephadm 2021-03-10T17:20:07.964283+0000 mgr.gibba032.whopeo (mgr.24625) 10924 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:10.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:09 gibba011 bash[30791]: cephadm 2021-03-10T17:20:07.964283+0000 mgr.gibba032.whopeo (mgr.24625) 10924 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:10.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:09 gibba011 bash[31774]: cephadm 2021-03-10T17:20:07.964283+0000 mgr.gibba032.whopeo (mgr.24625) 10924 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:10 gibba032 bash[24004]: cluster 2021-03-10T17:20:09.588687+0000 mgr.gibba032.whopeo (mgr.24625) 10925 : cluster [DBG] pgmap v6227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:10 gibba011 bash[30791]: cluster 2021-03-10T17:20:09.588687+0000 mgr.gibba032.whopeo (mgr.24625) 10925 : cluster [DBG] pgmap v6227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:10 gibba011 bash[31774]: cluster 2021-03-10T17:20:09.588687+0000 mgr.gibba032.whopeo (mgr.24625) 10925 : cluster [DBG] pgmap v6227: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:12.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:12 gibba011 bash[30791]: cluster 2021-03-10T17:20:11.590312+0000 mgr.gibba032.whopeo (mgr.24625) 10926 : cluster [DBG] pgmap v6228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:12.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:12 gibba011 bash[31774]: cluster 2021-03-10T17:20:11.590312+0000 mgr.gibba032.whopeo (mgr.24625) 10926 : cluster [DBG] pgmap v6228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:12 gibba032 bash[24004]: cluster 2021-03-10T17:20:11.590312+0000 mgr.gibba032.whopeo (mgr.24625) 10926 : cluster [DBG] pgmap v6228: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:14 gibba032 bash[24004]: cluster 2021-03-10T17:20:13.591890+0000 mgr.gibba032.whopeo (mgr.24625) 10927 : cluster [DBG] pgmap v6229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:14 gibba011 bash[30791]: cluster 2021-03-10T17:20:13.591890+0000 mgr.gibba032.whopeo (mgr.24625) 10927 : cluster [DBG] pgmap v6229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:14 gibba011 bash[31774]: cluster 2021-03-10T17:20:13.591890+0000 mgr.gibba032.whopeo (mgr.24625) 10927 : cluster [DBG] pgmap v6229: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:17.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:16 gibba032 bash[24004]: cluster 2021-03-10T17:20:15.593414+0000 mgr.gibba032.whopeo (mgr.24625) 10928 : cluster [DBG] pgmap v6230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:16 gibba011 bash[30791]: cluster 2021-03-10T17:20:15.593414+0000 mgr.gibba032.whopeo (mgr.24625) 10928 : cluster [DBG] pgmap v6230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:16 gibba011 bash[31774]: cluster 2021-03-10T17:20:15.593414+0000 mgr.gibba032.whopeo (mgr.24625) 10928 : cluster [DBG] pgmap v6230: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:17.804 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:20:18.033 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:20:18.034 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:20:18.034 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:20:18.034 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:20:18.035 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:20:18.035 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:20:18.035 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:20:18.035 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:20:18.036 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:20:18.036 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:20:18.036 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:20:18.036 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:20:18.036 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:20:18.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:20:18.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:20:18.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:20:18.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:20:18.037 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:20:18.038 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:20:18.038 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:20:18.038 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:20:18.301 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:20:18.302 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:20:18.302 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:20:18.302 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:18.302 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:20:18.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:20:18.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:18.303 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:20:18.303 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:20:18.303 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:18.304 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:20:18.304 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:20:18.304 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:20:18.304 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:20:18.304 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:20:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:18 gibba032 bash[24004]: cluster 2021-03-10T17:20:17.594973+0000 mgr.gibba032.whopeo (mgr.24625) 10929 : cluster [DBG] pgmap v6231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:18 gibba032 bash[24004]: audit 2021-03-10T17:20:17.793421+0000 mgr.gibba032.whopeo (mgr.24625) 10930 : audit [DBG] from='client.51381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:19.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:18 gibba032 bash[24004]: audit 2021-03-10T17:20:18.301398+0000 mon.a (mon.0) 3630 : audit [DBG] from='client.? 172.21.2.111:0/709991613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[30791]: cluster 2021-03-10T17:20:17.594973+0000 mgr.gibba032.whopeo (mgr.24625) 10929 : cluster [DBG] pgmap v6231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[30791]: audit 2021-03-10T17:20:17.793421+0000 mgr.gibba032.whopeo (mgr.24625) 10930 : audit [DBG] from='client.51381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:19.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[30791]: audit 2021-03-10T17:20:18.301398+0000 mon.a (mon.0) 3630 : audit [DBG] from='client.? 172.21.2.111:0/709991613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[31774]: cluster 2021-03-10T17:20:17.594973+0000 mgr.gibba032.whopeo (mgr.24625) 10929 : cluster [DBG] pgmap v6231: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[31774]: audit 2021-03-10T17:20:17.793421+0000 mgr.gibba032.whopeo (mgr.24625) 10930 : audit [DBG] from='client.51381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:19.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:18 gibba011 bash[31774]: audit 2021-03-10T17:20:18.301398+0000 mon.a (mon.0) 3630 : audit [DBG] from='client.? 172.21.2.111:0/709991613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:20.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:19 gibba032 bash[24004]: audit 2021-03-10T17:20:18.019811+0000 mgr.gibba032.whopeo (mgr.24625) 10931 : audit [DBG] from='client.61369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:20.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:19 gibba011 bash[30791]: audit 2021-03-10T17:20:18.019811+0000 mgr.gibba032.whopeo (mgr.24625) 10931 : audit [DBG] from='client.61369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:20.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:19 gibba011 bash[31774]: audit 2021-03-10T17:20:18.019811+0000 mgr.gibba032.whopeo (mgr.24625) 10931 : audit [DBG] from='client.61369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:20 gibba032 bash[24004]: cluster 2021-03-10T17:20:19.596418+0000 mgr.gibba032.whopeo (mgr.24625) 10932 : cluster [DBG] pgmap v6232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:20 gibba011 bash[30791]: cluster 2021-03-10T17:20:19.596418+0000 mgr.gibba032.whopeo (mgr.24625) 10932 : cluster [DBG] pgmap v6232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:20 gibba011 bash[31774]: cluster 2021-03-10T17:20:19.596418+0000 mgr.gibba032.whopeo (mgr.24625) 10932 : cluster [DBG] pgmap v6232: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:22.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:22 gibba011 bash[30791]: cluster 2021-03-10T17:20:21.598217+0000 mgr.gibba032.whopeo (mgr.24625) 10933 : cluster [DBG] pgmap v6233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:22.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:22 gibba011 bash[31774]: cluster 2021-03-10T17:20:21.598217+0000 mgr.gibba032.whopeo (mgr.24625) 10933 : cluster [DBG] pgmap v6233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:22 gibba032 bash[24004]: cluster 2021-03-10T17:20:21.598217+0000 mgr.gibba032.whopeo (mgr.24625) 10933 : cluster [DBG] pgmap v6233: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:23.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:23 gibba011 bash[30791]: audit 2021-03-10T17:20:22.984514+0000 mon.a (mon.0) 3631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:20:23.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:23 gibba011 bash[31774]: audit 2021-03-10T17:20:22.984514+0000 mon.a (mon.0) 3631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:20:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:23 gibba032 bash[24004]: audit 2021-03-10T17:20:22.984514+0000 mon.a (mon.0) 3631 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:20:24.359 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:24 gibba011 bash[30791]: cephadm 2021-03-10T17:20:22.982251+0000 mgr.gibba032.whopeo (mgr.24625) 10934 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:20:24.359 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:24 gibba011 bash[30791]: cluster 2021-03-10T17:20:23.599813+0000 mgr.gibba032.whopeo (mgr.24625) 10935 : cluster [DBG] pgmap v6234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:24.359 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:24 gibba011 bash[31774]: cephadm 2021-03-10T17:20:22.982251+0000 mgr.gibba032.whopeo (mgr.24625) 10934 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:20:24.360 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:24 gibba011 bash[31774]: cluster 2021-03-10T17:20:23.599813+0000 mgr.gibba032.whopeo (mgr.24625) 10935 : cluster [DBG] pgmap v6234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:24 gibba032 bash[24004]: cephadm 2021-03-10T17:20:22.982251+0000 mgr.gibba032.whopeo (mgr.24625) 10934 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:20:24.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:24 gibba032 bash[24004]: cluster 2021-03-10T17:20:23.599813+0000 mgr.gibba032.whopeo (mgr.24625) 10935 : cluster [DBG] pgmap v6234: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:27.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:26 gibba032 bash[24004]: cluster 2021-03-10T17:20:25.601160+0000 mgr.gibba032.whopeo (mgr.24625) 10936 : cluster [DBG] pgmap v6235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:27.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:26 gibba011 bash[30791]: cluster 2021-03-10T17:20:25.601160+0000 mgr.gibba032.whopeo (mgr.24625) 10936 : cluster [DBG] pgmap v6235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:26 gibba011 bash[31774]: cluster 2021-03-10T17:20:25.601160+0000 mgr.gibba032.whopeo (mgr.24625) 10936 : cluster [DBG] pgmap v6235: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: cluster 2021-03-10T17:20:27.602949+0000 mgr.gibba032.whopeo (mgr.24625) 10937 : cluster [DBG] pgmap v6236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:29.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:27.764108+0000 mon.a (mon.0) 3632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:20:29.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:27.949508+0000 mon.a (mon.0) 3633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.151093+0000 mon.a (mon.0) 3634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.156048+0000 mon.a (mon.0) 3635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:20:29.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.158493+0000 mon.a (mon.0) 3636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.160994+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.163486+0000 mon.a (mon.0) 3638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.167491+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:28 gibba032 bash[24004]: audit 2021-03-10T17:20:28.170301+0000 mon.a (mon.0) 3640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: cluster 2021-03-10T17:20:27.602949+0000 mgr.gibba032.whopeo (mgr.24625) 10937 : cluster [DBG] pgmap v6236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:29.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:27.764108+0000 mon.a (mon.0) 3632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:20:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:27.949508+0000 mon.a (mon.0) 3633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.151093+0000 mon.a (mon.0) 3634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.156048+0000 mon.a (mon.0) 3635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:20:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.158493+0000 mon.a (mon.0) 3636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.160994+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.163486+0000 mon.a (mon.0) 3638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.167491+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[30791]: audit 2021-03-10T17:20:28.170301+0000 mon.a (mon.0) 3640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: cluster 2021-03-10T17:20:27.602949+0000 mgr.gibba032.whopeo (mgr.24625) 10937 : cluster [DBG] pgmap v6236: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:29.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:27.764108+0000 mon.a (mon.0) 3632 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:20:29.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:27.949508+0000 mon.a (mon.0) 3633 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.151093+0000 mon.a (mon.0) 3634 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:20:29.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.156048+0000 mon.a (mon.0) 3635 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:20:29.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.158493+0000 mon.a (mon.0) 3636 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.160994+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.163486+0000 mon.a (mon.0) 3638 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.167491+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:29.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:28 gibba011 bash[31774]: audit 2021-03-10T17:20:28.170301+0000 mon.a (mon.0) 3640 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:30.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:27.949167+0000 mgr.gibba032.whopeo (mgr.24625) 10938 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:20:30.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.153190+0000 mgr.gibba032.whopeo (mgr.24625) 10939 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:20:30.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.156124+0000 mgr.gibba032.whopeo (mgr.24625) 10940 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:20:30.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.158552+0000 mgr.gibba032.whopeo (mgr.24625) 10941 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:20:30.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.158594+0000 mgr.gibba032.whopeo (mgr.24625) 10942 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:20:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.161027+0000 mgr.gibba032.whopeo (mgr.24625) 10943 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:20:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.161073+0000 mgr.gibba032.whopeo (mgr.24625) 10944 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:20:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.163106+0000 mgr.gibba032.whopeo (mgr.24625) 10945 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:20:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.164867+0000 mgr.gibba032.whopeo (mgr.24625) 10946 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:20:30.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.164910+0000 mgr.gibba032.whopeo (mgr.24625) 10947 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:20:30.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.167318+0000 mgr.gibba032.whopeo (mgr.24625) 10948 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:20:30.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.167899+0000 mgr.gibba032.whopeo (mgr.24625) 10949 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:20:30.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.170331+0000 mgr.gibba032.whopeo (mgr.24625) 10950 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:20:30.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.170524+0000 mgr.gibba032.whopeo (mgr.24625) 10951 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:20:30.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:29 gibba032 bash[24004]: cephadm 2021-03-10T17:20:28.170680+0000 mgr.gibba032.whopeo (mgr.24625) 10952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:27.949167+0000 mgr.gibba032.whopeo (mgr.24625) 10938 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:20:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.153190+0000 mgr.gibba032.whopeo (mgr.24625) 10939 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:20:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.156124+0000 mgr.gibba032.whopeo (mgr.24625) 10940 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:20:30.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.158552+0000 mgr.gibba032.whopeo (mgr.24625) 10941 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:20:30.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.158594+0000 mgr.gibba032.whopeo (mgr.24625) 10942 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:20:30.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.161027+0000 mgr.gibba032.whopeo (mgr.24625) 10943 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:20:30.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.161073+0000 mgr.gibba032.whopeo (mgr.24625) 10944 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:20:30.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.163106+0000 mgr.gibba032.whopeo (mgr.24625) 10945 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:20:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.164867+0000 mgr.gibba032.whopeo (mgr.24625) 10946 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:20:30.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.164910+0000 mgr.gibba032.whopeo (mgr.24625) 10947 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:20:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.167318+0000 mgr.gibba032.whopeo (mgr.24625) 10948 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:20:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.167899+0000 mgr.gibba032.whopeo (mgr.24625) 10949 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:20:30.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.170331+0000 mgr.gibba032.whopeo (mgr.24625) 10950 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:20:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.170524+0000 mgr.gibba032.whopeo (mgr.24625) 10951 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:20:30.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[30791]: cephadm 2021-03-10T17:20:28.170680+0000 mgr.gibba032.whopeo (mgr.24625) 10952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:30.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:27.949167+0000 mgr.gibba032.whopeo (mgr.24625) 10938 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:20:30.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.153190+0000 mgr.gibba032.whopeo (mgr.24625) 10939 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:20:30.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.156124+0000 mgr.gibba032.whopeo (mgr.24625) 10940 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:20:30.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.158552+0000 mgr.gibba032.whopeo (mgr.24625) 10941 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:20:30.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.158594+0000 mgr.gibba032.whopeo (mgr.24625) 10942 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:20:30.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.161027+0000 mgr.gibba032.whopeo (mgr.24625) 10943 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:20:30.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.161073+0000 mgr.gibba032.whopeo (mgr.24625) 10944 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:20:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.163106+0000 mgr.gibba032.whopeo (mgr.24625) 10945 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:20:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.164867+0000 mgr.gibba032.whopeo (mgr.24625) 10946 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:20:30.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.164910+0000 mgr.gibba032.whopeo (mgr.24625) 10947 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:20:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.167318+0000 mgr.gibba032.whopeo (mgr.24625) 10948 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:20:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.167899+0000 mgr.gibba032.whopeo (mgr.24625) 10949 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:20:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.170331+0000 mgr.gibba032.whopeo (mgr.24625) 10950 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:20:30.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.170524+0000 mgr.gibba032.whopeo (mgr.24625) 10951 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:20:30.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:29 gibba011 bash[31774]: cephadm 2021-03-10T17:20:28.170680+0000 mgr.gibba032.whopeo (mgr.24625) 10952 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:31.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:30 gibba032 bash[24004]: cluster 2021-03-10T17:20:29.604523+0000 mgr.gibba032.whopeo (mgr.24625) 10953 : cluster [DBG] pgmap v6237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:30 gibba011 bash[30791]: cluster 2021-03-10T17:20:29.604523+0000 mgr.gibba032.whopeo (mgr.24625) 10953 : cluster [DBG] pgmap v6237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:31.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:30 gibba011 bash[31774]: cluster 2021-03-10T17:20:29.604523+0000 mgr.gibba032.whopeo (mgr.24625) 10953 : cluster [DBG] pgmap v6237: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:32.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:32 gibba011 bash[30791]: cluster 2021-03-10T17:20:31.606043+0000 mgr.gibba032.whopeo (mgr.24625) 10954 : cluster [DBG] pgmap v6238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:32.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:32 gibba011 bash[31774]: cluster 2021-03-10T17:20:31.606043+0000 mgr.gibba032.whopeo (mgr.24625) 10954 : cluster [DBG] pgmap v6238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:32 gibba032 bash[24004]: cluster 2021-03-10T17:20:31.606043+0000 mgr.gibba032.whopeo (mgr.24625) 10954 : cluster [DBG] pgmap v6238: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:35.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:34 gibba032 bash[24004]: cluster 2021-03-10T17:20:33.607637+0000 mgr.gibba032.whopeo (mgr.24625) 10955 : cluster [DBG] pgmap v6239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:34 gibba011 bash[30791]: cluster 2021-03-10T17:20:33.607637+0000 mgr.gibba032.whopeo (mgr.24625) 10955 : cluster [DBG] pgmap v6239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:34 gibba011 bash[31774]: cluster 2021-03-10T17:20:33.607637+0000 mgr.gibba032.whopeo (mgr.24625) 10955 : cluster [DBG] pgmap v6239: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:37.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:36 gibba032 bash[24004]: cluster 2021-03-10T17:20:35.609237+0000 mgr.gibba032.whopeo (mgr.24625) 10956 : cluster [DBG] pgmap v6240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:36 gibba011 bash[30791]: cluster 2021-03-10T17:20:35.609237+0000 mgr.gibba032.whopeo (mgr.24625) 10956 : cluster [DBG] pgmap v6240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:36 gibba011 bash[31774]: cluster 2021-03-10T17:20:35.609237+0000 mgr.gibba032.whopeo (mgr.24625) 10956 : cluster [DBG] pgmap v6240: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:39.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:38 gibba032 bash[24004]: cluster 2021-03-10T17:20:37.610908+0000 mgr.gibba032.whopeo (mgr.24625) 10957 : cluster [DBG] pgmap v6241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:38 gibba011 bash[30791]: cluster 2021-03-10T17:20:37.610908+0000 mgr.gibba032.whopeo (mgr.24625) 10957 : cluster [DBG] pgmap v6241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:39.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:38 gibba011 bash[31774]: cluster 2021-03-10T17:20:37.610908+0000 mgr.gibba032.whopeo (mgr.24625) 10957 : cluster [DBG] pgmap v6241: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:40 gibba032 bash[24004]: cluster 2021-03-10T17:20:39.612426+0000 mgr.gibba032.whopeo (mgr.24625) 10958 : cluster [DBG] pgmap v6242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:41.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:40 gibba011 bash[30791]: cluster 2021-03-10T17:20:39.612426+0000 mgr.gibba032.whopeo (mgr.24625) 10958 : cluster [DBG] pgmap v6242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:41.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:40 gibba011 bash[31774]: cluster 2021-03-10T17:20:39.612426+0000 mgr.gibba032.whopeo (mgr.24625) 10958 : cluster [DBG] pgmap v6242: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:42.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:42 gibba032 bash[24004]: cluster 2021-03-10T17:20:41.614144+0000 mgr.gibba032.whopeo (mgr.24625) 10959 : cluster [DBG] pgmap v6243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:42 gibba011 bash[30791]: cluster 2021-03-10T17:20:41.614144+0000 mgr.gibba032.whopeo (mgr.24625) 10959 : cluster [DBG] pgmap v6243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:42 gibba011 bash[31774]: cluster 2021-03-10T17:20:41.614144+0000 mgr.gibba032.whopeo (mgr.24625) 10959 : cluster [DBG] pgmap v6243: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:44 gibba032 bash[24004]: cephadm 2021-03-10T17:20:43.185025+0000 mgr.gibba032.whopeo (mgr.24625) 10960 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:44 gibba032 bash[24004]: cluster 2021-03-10T17:20:43.615748+0000 mgr.gibba032.whopeo (mgr.24625) 10961 : cluster [DBG] pgmap v6244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:44 gibba011 bash[30791]: cephadm 2021-03-10T17:20:43.185025+0000 mgr.gibba032.whopeo (mgr.24625) 10960 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:44 gibba011 bash[30791]: cluster 2021-03-10T17:20:43.615748+0000 mgr.gibba032.whopeo (mgr.24625) 10961 : cluster [DBG] pgmap v6244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:44 gibba011 bash[31774]: cephadm 2021-03-10T17:20:43.185025+0000 mgr.gibba032.whopeo (mgr.24625) 10960 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:20:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:44 gibba011 bash[31774]: cluster 2021-03-10T17:20:43.615748+0000 mgr.gibba032.whopeo (mgr.24625) 10961 : cluster [DBG] pgmap v6244: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:46 gibba032 bash[24004]: cluster 2021-03-10T17:20:45.617223+0000 mgr.gibba032.whopeo (mgr.24625) 10962 : cluster [DBG] pgmap v6245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:46 gibba011 bash[30791]: cluster 2021-03-10T17:20:45.617223+0000 mgr.gibba032.whopeo (mgr.24625) 10962 : cluster [DBG] pgmap v6245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:47.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:46 gibba011 bash[31774]: cluster 2021-03-10T17:20:45.617223+0000 mgr.gibba032.whopeo (mgr.24625) 10962 : cluster [DBG] pgmap v6245: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:48.548 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:20:48.776 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:20:48.777 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:20:48.777 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:20:48.777 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:20:48.777 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:20:48.778 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:20:48.778 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:20:48.778 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:20:48.778 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:20:48.779 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:20:48.779 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:20:48.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:20:48.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:20:48.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:20:48.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:20:48.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:20:48.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:20:48.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:20:48.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:20:48.781 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:20:48.781 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:20:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:48 gibba032 bash[24004]: cluster 2021-03-10T17:20:47.619052+0000 mgr.gibba032.whopeo (mgr.24625) 10963 : cluster [DBG] pgmap v6246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:49.026 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:20:49.026 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:20:49.026 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:20:49.027 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:49.027 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:20:49.027 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:20:49.027 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:49.028 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:20:49.028 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:20:49.028 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:20:49.028 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:20:49.028 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:20:49.029 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:20:49.029 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:20:49.029 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:20:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:48 gibba011 bash[30791]: cluster 2021-03-10T17:20:47.619052+0000 mgr.gibba032.whopeo (mgr.24625) 10963 : cluster [DBG] pgmap v6246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:49.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:48 gibba011 bash[31774]: cluster 2021-03-10T17:20:47.619052+0000 mgr.gibba032.whopeo (mgr.24625) 10963 : cluster [DBG] pgmap v6246: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:49 gibba032 bash[24004]: audit 2021-03-10T17:20:48.537393+0000 mgr.gibba032.whopeo (mgr.24625) 10964 : audit [DBG] from='client.51399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:49 gibba032 bash[24004]: audit 2021-03-10T17:20:48.763720+0000 mgr.gibba032.whopeo (mgr.24625) 10965 : audit [DBG] from='client.51405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:49 gibba032 bash[24004]: audit 2021-03-10T17:20:49.025908+0000 mon.c (mon.1) 210 : audit [DBG] from='client.? 172.21.2.111:0/3079129575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:50.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[30791]: audit 2021-03-10T17:20:48.537393+0000 mgr.gibba032.whopeo (mgr.24625) 10964 : audit [DBG] from='client.51399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[30791]: audit 2021-03-10T17:20:48.763720+0000 mgr.gibba032.whopeo (mgr.24625) 10965 : audit [DBG] from='client.51405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[30791]: audit 2021-03-10T17:20:49.025908+0000 mon.c (mon.1) 210 : audit [DBG] from='client.? 172.21.2.111:0/3079129575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[31774]: audit 2021-03-10T17:20:48.537393+0000 mgr.gibba032.whopeo (mgr.24625) 10964 : audit [DBG] from='client.51399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[31774]: audit 2021-03-10T17:20:48.763720+0000 mgr.gibba032.whopeo (mgr.24625) 10965 : audit [DBG] from='client.51405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:20:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:49 gibba011 bash[31774]: audit 2021-03-10T17:20:49.025908+0000 mon.c (mon.1) 210 : audit [DBG] from='client.? 172.21.2.111:0/3079129575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:20:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:50 gibba032 bash[24004]: cluster 2021-03-10T17:20:49.620494+0000 mgr.gibba032.whopeo (mgr.24625) 10966 : cluster [DBG] pgmap v6247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:50 gibba011 bash[30791]: cluster 2021-03-10T17:20:49.620494+0000 mgr.gibba032.whopeo (mgr.24625) 10966 : cluster [DBG] pgmap v6247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:50 gibba011 bash[31774]: cluster 2021-03-10T17:20:49.620494+0000 mgr.gibba032.whopeo (mgr.24625) 10966 : cluster [DBG] pgmap v6247: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:52 gibba032 bash[24004]: cluster 2021-03-10T17:20:51.622224+0000 mgr.gibba032.whopeo (mgr.24625) 10967 : cluster [DBG] pgmap v6248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:52 gibba011 bash[30791]: cluster 2021-03-10T17:20:51.622224+0000 mgr.gibba032.whopeo (mgr.24625) 10967 : cluster [DBG] pgmap v6248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:52 gibba011 bash[31774]: cluster 2021-03-10T17:20:51.622224+0000 mgr.gibba032.whopeo (mgr.24625) 10967 : cluster [DBG] pgmap v6248: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:54 gibba032 bash[24004]: cluster 2021-03-10T17:20:53.623770+0000 mgr.gibba032.whopeo (mgr.24625) 10968 : cluster [DBG] pgmap v6249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:54 gibba011 bash[30791]: cluster 2021-03-10T17:20:53.623770+0000 mgr.gibba032.whopeo (mgr.24625) 10968 : cluster [DBG] pgmap v6249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:54 gibba011 bash[31774]: cluster 2021-03-10T17:20:53.623770+0000 mgr.gibba032.whopeo (mgr.24625) 10968 : cluster [DBG] pgmap v6249: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:57.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:56 gibba032 bash[24004]: cluster 2021-03-10T17:20:55.625280+0000 mgr.gibba032.whopeo (mgr.24625) 10969 : cluster [DBG] pgmap v6250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:56 gibba011 bash[30791]: cluster 2021-03-10T17:20:55.625280+0000 mgr.gibba032.whopeo (mgr.24625) 10969 : cluster [DBG] pgmap v6250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:56 gibba011 bash[31774]: cluster 2021-03-10T17:20:55.625280+0000 mgr.gibba032.whopeo (mgr.24625) 10969 : cluster [DBG] pgmap v6250: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.444106+0000 mon.a (mon.0) 3641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:20:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.444765+0000 mon.a (mon.0) 3642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:20:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.620768+0000 mon.a (mon.0) 3643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:20:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.621529+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:20:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.622218+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:20:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:57 gibba032 bash[24004]: audit 2021-03-10T17:20:57.622895+0000 mon.a (mon.0) 3646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:20:58.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.444106+0000 mon.a (mon.0) 3641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:20:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.444765+0000 mon.a (mon.0) 3642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:20:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.620768+0000 mon.a (mon.0) 3643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:20:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.621529+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:20:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.622218+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:20:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[30791]: audit 2021-03-10T17:20:57.622895+0000 mon.a (mon.0) 3646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:20:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.444106+0000 mon.a (mon.0) 3641 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:20:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.444765+0000 mon.a (mon.0) 3642 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:20:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.620768+0000 mon.a (mon.0) 3643 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:20:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.621529+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:20:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.622218+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:20:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:57 gibba011 bash[31774]: audit 2021-03-10T17:20:57.622895+0000 mon.a (mon.0) 3646 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:20:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:58 gibba032 bash[24004]: cluster 2021-03-10T17:20:57.626744+0000 mgr.gibba032.whopeo (mgr.24625) 10970 : cluster [DBG] pgmap v6251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:58 gibba011 bash[30791]: cluster 2021-03-10T17:20:57.626744+0000 mgr.gibba032.whopeo (mgr.24625) 10970 : cluster [DBG] pgmap v6251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:20:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:58 gibba011 bash[31774]: cluster 2021-03-10T17:20:57.626744+0000 mgr.gibba032.whopeo (mgr.24625) 10970 : cluster [DBG] pgmap v6251: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:00.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:20:59 gibba032 bash[24004]: cephadm 2021-03-10T17:20:58.201115+0000 mgr.gibba032.whopeo (mgr.24625) 10971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:00.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:20:59 gibba011 bash[30791]: cephadm 2021-03-10T17:20:58.201115+0000 mgr.gibba032.whopeo (mgr.24625) 10971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:00.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:20:59 gibba011 bash[31774]: cephadm 2021-03-10T17:20:58.201115+0000 mgr.gibba032.whopeo (mgr.24625) 10971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:01.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:00 gibba032 bash[24004]: cluster 2021-03-10T17:20:59.628266+0000 mgr.gibba032.whopeo (mgr.24625) 10972 : cluster [DBG] pgmap v6252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:00 gibba011 bash[30791]: cluster 2021-03-10T17:20:59.628266+0000 mgr.gibba032.whopeo (mgr.24625) 10972 : cluster [DBG] pgmap v6252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:00 gibba011 bash[31774]: cluster 2021-03-10T17:20:59.628266+0000 mgr.gibba032.whopeo (mgr.24625) 10972 : cluster [DBG] pgmap v6252: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:02 gibba032 bash[24004]: cluster 2021-03-10T17:21:01.629759+0000 mgr.gibba032.whopeo (mgr.24625) 10973 : cluster [DBG] pgmap v6253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:02 gibba011 bash[30791]: cluster 2021-03-10T17:21:01.629759+0000 mgr.gibba032.whopeo (mgr.24625) 10973 : cluster [DBG] pgmap v6253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:02 gibba011 bash[31774]: cluster 2021-03-10T17:21:01.629759+0000 mgr.gibba032.whopeo (mgr.24625) 10973 : cluster [DBG] pgmap v6253: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:05.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:04 gibba032 bash[24004]: cluster 2021-03-10T17:21:03.631341+0000 mgr.gibba032.whopeo (mgr.24625) 10974 : cluster [DBG] pgmap v6254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:04 gibba011 bash[30791]: cluster 2021-03-10T17:21:03.631341+0000 mgr.gibba032.whopeo (mgr.24625) 10974 : cluster [DBG] pgmap v6254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:05.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:04 gibba011 bash[31774]: cluster 2021-03-10T17:21:03.631341+0000 mgr.gibba032.whopeo (mgr.24625) 10974 : cluster [DBG] pgmap v6254: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:07.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:06 gibba032 bash[24004]: cluster 2021-03-10T17:21:05.632845+0000 mgr.gibba032.whopeo (mgr.24625) 10975 : cluster [DBG] pgmap v6255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:06 gibba011 bash[30791]: cluster 2021-03-10T17:21:05.632845+0000 mgr.gibba032.whopeo (mgr.24625) 10975 : cluster [DBG] pgmap v6255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:06 gibba011 bash[31774]: cluster 2021-03-10T17:21:05.632845+0000 mgr.gibba032.whopeo (mgr.24625) 10975 : cluster [DBG] pgmap v6255: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:08 gibba032 bash[24004]: cluster 2021-03-10T17:21:07.634459+0000 mgr.gibba032.whopeo (mgr.24625) 10976 : cluster [DBG] pgmap v6256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:08 gibba011 bash[30791]: cluster 2021-03-10T17:21:07.634459+0000 mgr.gibba032.whopeo (mgr.24625) 10976 : cluster [DBG] pgmap v6256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:08 gibba011 bash[31774]: cluster 2021-03-10T17:21:07.634459+0000 mgr.gibba032.whopeo (mgr.24625) 10976 : cluster [DBG] pgmap v6256: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:11.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:10 gibba032 bash[24004]: cluster 2021-03-10T17:21:09.636062+0000 mgr.gibba032.whopeo (mgr.24625) 10977 : cluster [DBG] pgmap v6257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:10 gibba011 bash[30791]: cluster 2021-03-10T17:21:09.636062+0000 mgr.gibba032.whopeo (mgr.24625) 10977 : cluster [DBG] pgmap v6257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:10 gibba011 bash[31774]: cluster 2021-03-10T17:21:09.636062+0000 mgr.gibba032.whopeo (mgr.24625) 10977 : cluster [DBG] pgmap v6257: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:12 gibba032 bash[24004]: cluster 2021-03-10T17:21:11.637608+0000 mgr.gibba032.whopeo (mgr.24625) 10978 : cluster [DBG] pgmap v6258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:12 gibba011 bash[30791]: cluster 2021-03-10T17:21:11.637608+0000 mgr.gibba032.whopeo (mgr.24625) 10978 : cluster [DBG] pgmap v6258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:12 gibba011 bash[31774]: cluster 2021-03-10T17:21:11.637608+0000 mgr.gibba032.whopeo (mgr.24625) 10978 : cluster [DBG] pgmap v6258: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:14 gibba032 bash[24004]: cephadm 2021-03-10T17:21:13.216658+0000 mgr.gibba032.whopeo (mgr.24625) 10979 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:15.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:14 gibba032 bash[24004]: cluster 2021-03-10T17:21:13.639071+0000 mgr.gibba032.whopeo (mgr.24625) 10980 : cluster [DBG] pgmap v6259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:14 gibba011 bash[30791]: cephadm 2021-03-10T17:21:13.216658+0000 mgr.gibba032.whopeo (mgr.24625) 10979 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:14 gibba011 bash[30791]: cluster 2021-03-10T17:21:13.639071+0000 mgr.gibba032.whopeo (mgr.24625) 10980 : cluster [DBG] pgmap v6259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:14 gibba011 bash[31774]: cephadm 2021-03-10T17:21:13.216658+0000 mgr.gibba032.whopeo (mgr.24625) 10979 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:14 gibba011 bash[31774]: cluster 2021-03-10T17:21:13.639071+0000 mgr.gibba032.whopeo (mgr.24625) 10980 : cluster [DBG] pgmap v6259: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:17.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:16 gibba032 bash[24004]: cluster 2021-03-10T17:21:15.640727+0000 mgr.gibba032.whopeo (mgr.24625) 10981 : cluster [DBG] pgmap v6260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:16 gibba011 bash[30791]: cluster 2021-03-10T17:21:15.640727+0000 mgr.gibba032.whopeo (mgr.24625) 10981 : cluster [DBG] pgmap v6260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:17.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:16 gibba011 bash[31774]: cluster 2021-03-10T17:21:15.640727+0000 mgr.gibba032.whopeo (mgr.24625) 10981 : cluster [DBG] pgmap v6260: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:19.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:18 gibba032 bash[24004]: cluster 2021-03-10T17:21:17.642502+0000 mgr.gibba032.whopeo (mgr.24625) 10982 : cluster [DBG] pgmap v6261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:19.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:18 gibba011 bash[30791]: cluster 2021-03-10T17:21:17.642502+0000 mgr.gibba032.whopeo (mgr.24625) 10982 : cluster [DBG] pgmap v6261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:19.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:18 gibba011 bash[31774]: cluster 2021-03-10T17:21:17.642502+0000 mgr.gibba032.whopeo (mgr.24625) 10982 : cluster [DBG] pgmap v6261: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:19.271 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:21:19.494 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:21:19.494 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 10m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:21:19.495 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:21:19.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:21:19.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:21:19.495 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:21:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:21:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:21:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:21:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 10m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:21:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:21:19.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:21:19.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:21:19.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:21:19.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:21:19.497 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:21:19.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:21:19.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:21:19.498 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:21:19.498 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:21:19.499 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:21:19.745 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:21:19.746 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:21:19.746 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:21:19.746 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:19.747 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:21:19.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:21:19.747 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:19.747 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:21:19.747 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:21:19.748 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:19.748 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:21:19.748 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:21:19.748 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:21:19.749 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:21:19.749 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:21:21.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:20 gibba032 bash[24004]: audit 2021-03-10T17:21:19.260223+0000 mgr.gibba032.whopeo (mgr.24625) 10983 : audit [DBG] from='client.51417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:20 gibba032 bash[24004]: audit 2021-03-10T17:21:19.484247+0000 mgr.gibba032.whopeo (mgr.24625) 10984 : audit [DBG] from='client.61405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:20 gibba032 bash[24004]: cluster 2021-03-10T17:21:19.643985+0000 mgr.gibba032.whopeo (mgr.24625) 10985 : cluster [DBG] pgmap v6262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:21.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:20 gibba032 bash[24004]: audit 2021-03-10T17:21:19.745655+0000 mon.c (mon.1) 211 : audit [DBG] from='client.? 172.21.2.111:0/1277548469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[30791]: audit 2021-03-10T17:21:19.260223+0000 mgr.gibba032.whopeo (mgr.24625) 10983 : audit [DBG] from='client.51417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[30791]: audit 2021-03-10T17:21:19.484247+0000 mgr.gibba032.whopeo (mgr.24625) 10984 : audit [DBG] from='client.61405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[30791]: cluster 2021-03-10T17:21:19.643985+0000 mgr.gibba032.whopeo (mgr.24625) 10985 : cluster [DBG] pgmap v6262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:21.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[30791]: audit 2021-03-10T17:21:19.745655+0000 mon.c (mon.1) 211 : audit [DBG] from='client.? 172.21.2.111:0/1277548469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:21.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[31774]: audit 2021-03-10T17:21:19.260223+0000 mgr.gibba032.whopeo (mgr.24625) 10983 : audit [DBG] from='client.51417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[31774]: audit 2021-03-10T17:21:19.484247+0000 mgr.gibba032.whopeo (mgr.24625) 10984 : audit [DBG] from='client.61405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:21.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[31774]: cluster 2021-03-10T17:21:19.643985+0000 mgr.gibba032.whopeo (mgr.24625) 10985 : cluster [DBG] pgmap v6262: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:21.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:20 gibba011 bash[31774]: audit 2021-03-10T17:21:19.745655+0000 mon.c (mon.1) 211 : audit [DBG] from='client.? 172.21.2.111:0/1277548469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:22 gibba032 bash[24004]: cluster 2021-03-10T17:21:21.645590+0000 mgr.gibba032.whopeo (mgr.24625) 10986 : cluster [DBG] pgmap v6263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:22 gibba011 bash[30791]: cluster 2021-03-10T17:21:21.645590+0000 mgr.gibba032.whopeo (mgr.24625) 10986 : cluster [DBG] pgmap v6263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:22 gibba011 bash[31774]: cluster 2021-03-10T17:21:21.645590+0000 mgr.gibba032.whopeo (mgr.24625) 10986 : cluster [DBG] pgmap v6263: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:25.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:24 gibba032 bash[24004]: cluster 2021-03-10T17:21:23.647047+0000 mgr.gibba032.whopeo (mgr.24625) 10987 : cluster [DBG] pgmap v6264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:24 gibba011 bash[30791]: cluster 2021-03-10T17:21:23.647047+0000 mgr.gibba032.whopeo (mgr.24625) 10987 : cluster [DBG] pgmap v6264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:25.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:24 gibba011 bash[31774]: cluster 2021-03-10T17:21:23.647047+0000 mgr.gibba032.whopeo (mgr.24625) 10987 : cluster [DBG] pgmap v6264: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:27.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:26 gibba032 bash[24004]: cluster 2021-03-10T17:21:25.648876+0000 mgr.gibba032.whopeo (mgr.24625) 10988 : cluster [DBG] pgmap v6265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:26 gibba011 bash[30791]: cluster 2021-03-10T17:21:25.648876+0000 mgr.gibba032.whopeo (mgr.24625) 10988 : cluster [DBG] pgmap v6265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:26 gibba011 bash[31774]: cluster 2021-03-10T17:21:25.648876+0000 mgr.gibba032.whopeo (mgr.24625) 10988 : cluster [DBG] pgmap v6265: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:28 gibba032 bash[24004]: cluster 2021-03-10T17:21:27.650635+0000 mgr.gibba032.whopeo (mgr.24625) 10989 : cluster [DBG] pgmap v6266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:29.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:28 gibba032 bash[24004]: audit 2021-03-10T17:21:28.236588+0000 mon.a (mon.0) 3647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:21:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:28 gibba011 bash[30791]: cluster 2021-03-10T17:21:27.650635+0000 mgr.gibba032.whopeo (mgr.24625) 10989 : cluster [DBG] pgmap v6266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:28 gibba011 bash[30791]: audit 2021-03-10T17:21:28.236588+0000 mon.a (mon.0) 3647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:21:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:28 gibba011 bash[31774]: cluster 2021-03-10T17:21:27.650635+0000 mgr.gibba032.whopeo (mgr.24625) 10989 : cluster [DBG] pgmap v6266: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:29.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:28 gibba011 bash[31774]: audit 2021-03-10T17:21:28.236588+0000 mon.a (mon.0) 3647 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:21:30.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:29 gibba032 bash[24004]: cephadm 2021-03-10T17:21:28.234243+0000 mgr.gibba032.whopeo (mgr.24625) 10990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:21:30.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:29 gibba011 bash[30791]: cephadm 2021-03-10T17:21:28.234243+0000 mgr.gibba032.whopeo (mgr.24625) 10990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:21:30.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:29 gibba011 bash[31774]: cephadm 2021-03-10T17:21:28.234243+0000 mgr.gibba032.whopeo (mgr.24625) 10990 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:21:31.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:30 gibba032 bash[24004]: cluster 2021-03-10T17:21:29.652301+0000 mgr.gibba032.whopeo (mgr.24625) 10991 : cluster [DBG] pgmap v6267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:31.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:30 gibba011 bash[30791]: cluster 2021-03-10T17:21:29.652301+0000 mgr.gibba032.whopeo (mgr.24625) 10991 : cluster [DBG] pgmap v6267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:30 gibba011 bash[31774]: cluster 2021-03-10T17:21:29.652301+0000 mgr.gibba032.whopeo (mgr.24625) 10991 : cluster [DBG] pgmap v6267: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: cluster 2021-03-10T17:21:31.653823+0000 mgr.gibba032.whopeo (mgr.24625) 10992 : cluster [DBG] pgmap v6268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: cephadm 2021-03-10T17:21:31.852368+0000 mgr.gibba032.whopeo (mgr.24625) 10993 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:21:33.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:31.853666+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.090499+0000 mon.a (mon.0) 3649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.099591+0000 mon.a (mon.0) 3650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:21:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.100843+0000 mon.a (mon.0) 3651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.101899+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.102828+0000 mon.a (mon.0) 3653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.104614+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:32 gibba032 bash[24004]: audit 2021-03-10T17:21:32.105660+0000 mon.a (mon.0) 3655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: cluster 2021-03-10T17:21:31.653823+0000 mgr.gibba032.whopeo (mgr.24625) 10992 : cluster [DBG] pgmap v6268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:33.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: cephadm 2021-03-10T17:21:31.852368+0000 mgr.gibba032.whopeo (mgr.24625) 10993 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:21:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:31.853666+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.090499+0000 mon.a (mon.0) 3649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.099591+0000 mon.a (mon.0) 3650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:21:33.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.100843+0000 mon.a (mon.0) 3651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.101899+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.102828+0000 mon.a (mon.0) 3653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.104614+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[30791]: audit 2021-03-10T17:21:32.105660+0000 mon.a (mon.0) 3655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: cluster 2021-03-10T17:21:31.653823+0000 mgr.gibba032.whopeo (mgr.24625) 10992 : cluster [DBG] pgmap v6268: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: cephadm 2021-03-10T17:21:31.852368+0000 mgr.gibba032.whopeo (mgr.24625) 10993 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:21:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:31.853666+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.090499+0000 mon.a (mon.0) 3649 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:21:33.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.099591+0000 mon.a (mon.0) 3650 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:21:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.100843+0000 mon.a (mon.0) 3651 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.101899+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.102828+0000 mon.a (mon.0) 3653 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.104614+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:33.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:32 gibba011 bash[31774]: audit 2021-03-10T17:21:32.105660+0000 mon.a (mon.0) 3655 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:34.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.096037+0000 mgr.gibba032.whopeo (mgr.24625) 10994 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:21:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.098687+0000 mgr.gibba032.whopeo (mgr.24625) 10995 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:21:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.099719+0000 mgr.gibba032.whopeo (mgr.24625) 10996 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:21:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.099759+0000 mgr.gibba032.whopeo (mgr.24625) 10997 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:21:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.100712+0000 mgr.gibba032.whopeo (mgr.24625) 10998 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:21:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.100756+0000 mgr.gibba032.whopeo (mgr.24625) 10999 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:21:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.101619+0000 mgr.gibba032.whopeo (mgr.24625) 11000 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:21:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.102423+0000 mgr.gibba032.whopeo (mgr.24625) 11001 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:21:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.102459+0000 mgr.gibba032.whopeo (mgr.24625) 11002 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:21:34.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.103320+0000 mgr.gibba032.whopeo (mgr.24625) 11003 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:21:34.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.103639+0000 mgr.gibba032.whopeo (mgr.24625) 11004 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:21:34.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.104393+0000 mgr.gibba032.whopeo (mgr.24625) 11005 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:21:34.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.104435+0000 mgr.gibba032.whopeo (mgr.24625) 11006 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:21:34.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:33 gibba032 bash[24004]: cephadm 2021-03-10T17:21:32.104539+0000 mgr.gibba032.whopeo (mgr.24625) 11007 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:34.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.096037+0000 mgr.gibba032.whopeo (mgr.24625) 10994 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:21:34.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.098687+0000 mgr.gibba032.whopeo (mgr.24625) 10995 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:21:34.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.099719+0000 mgr.gibba032.whopeo (mgr.24625) 10996 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:21:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.099759+0000 mgr.gibba032.whopeo (mgr.24625) 10997 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:21:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.100712+0000 mgr.gibba032.whopeo (mgr.24625) 10998 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:21:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.100756+0000 mgr.gibba032.whopeo (mgr.24625) 10999 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:21:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.101619+0000 mgr.gibba032.whopeo (mgr.24625) 11000 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:21:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.102423+0000 mgr.gibba032.whopeo (mgr.24625) 11001 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:21:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.102459+0000 mgr.gibba032.whopeo (mgr.24625) 11002 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:21:34.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.103320+0000 mgr.gibba032.whopeo (mgr.24625) 11003 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:21:34.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.103639+0000 mgr.gibba032.whopeo (mgr.24625) 11004 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:21:34.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.104393+0000 mgr.gibba032.whopeo (mgr.24625) 11005 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:21:34.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.104435+0000 mgr.gibba032.whopeo (mgr.24625) 11006 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:21:34.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[30791]: cephadm 2021-03-10T17:21:32.104539+0000 mgr.gibba032.whopeo (mgr.24625) 11007 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:34.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.096037+0000 mgr.gibba032.whopeo (mgr.24625) 10994 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:21:34.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.098687+0000 mgr.gibba032.whopeo (mgr.24625) 10995 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:21:34.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.099719+0000 mgr.gibba032.whopeo (mgr.24625) 10996 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:21:34.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.099759+0000 mgr.gibba032.whopeo (mgr.24625) 10997 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:21:34.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.100712+0000 mgr.gibba032.whopeo (mgr.24625) 10998 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:21:34.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.100756+0000 mgr.gibba032.whopeo (mgr.24625) 10999 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:21:34.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.101619+0000 mgr.gibba032.whopeo (mgr.24625) 11000 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:21:34.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.102423+0000 mgr.gibba032.whopeo (mgr.24625) 11001 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:21:34.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.102459+0000 mgr.gibba032.whopeo (mgr.24625) 11002 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:21:34.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.103320+0000 mgr.gibba032.whopeo (mgr.24625) 11003 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:21:34.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.103639+0000 mgr.gibba032.whopeo (mgr.24625) 11004 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:21:34.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.104393+0000 mgr.gibba032.whopeo (mgr.24625) 11005 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:21:34.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.104435+0000 mgr.gibba032.whopeo (mgr.24625) 11006 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:21:34.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:33 gibba011 bash[31774]: cephadm 2021-03-10T17:21:32.104539+0000 mgr.gibba032.whopeo (mgr.24625) 11007 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:35.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:34 gibba032 bash[24004]: cluster 2021-03-10T17:21:33.655412+0000 mgr.gibba032.whopeo (mgr.24625) 11008 : cluster [DBG] pgmap v6269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:34 gibba011 bash[30791]: cluster 2021-03-10T17:21:33.655412+0000 mgr.gibba032.whopeo (mgr.24625) 11008 : cluster [DBG] pgmap v6269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:34 gibba011 bash[31774]: cluster 2021-03-10T17:21:33.655412+0000 mgr.gibba032.whopeo (mgr.24625) 11008 : cluster [DBG] pgmap v6269: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:36 gibba032 bash[24004]: cluster 2021-03-10T17:21:35.656996+0000 mgr.gibba032.whopeo (mgr.24625) 11009 : cluster [DBG] pgmap v6270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:36 gibba011 bash[30791]: cluster 2021-03-10T17:21:35.656996+0000 mgr.gibba032.whopeo (mgr.24625) 11009 : cluster [DBG] pgmap v6270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:37.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:36 gibba011 bash[31774]: cluster 2021-03-10T17:21:35.656996+0000 mgr.gibba032.whopeo (mgr.24625) 11009 : cluster [DBG] pgmap v6270: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:39.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:38 gibba032 bash[24004]: cluster 2021-03-10T17:21:37.658799+0000 mgr.gibba032.whopeo (mgr.24625) 11010 : cluster [DBG] pgmap v6271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:38 gibba011 bash[30791]: cluster 2021-03-10T17:21:37.658799+0000 mgr.gibba032.whopeo (mgr.24625) 11010 : cluster [DBG] pgmap v6271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:38 gibba011 bash[31774]: cluster 2021-03-10T17:21:37.658799+0000 mgr.gibba032.whopeo (mgr.24625) 11010 : cluster [DBG] pgmap v6271: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:40 gibba032 bash[24004]: cluster 2021-03-10T17:21:39.660338+0000 mgr.gibba032.whopeo (mgr.24625) 11011 : cluster [DBG] pgmap v6272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:40 gibba011 bash[30791]: cluster 2021-03-10T17:21:39.660338+0000 mgr.gibba032.whopeo (mgr.24625) 11011 : cluster [DBG] pgmap v6272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:40 gibba011 bash[31774]: cluster 2021-03-10T17:21:39.660338+0000 mgr.gibba032.whopeo (mgr.24625) 11011 : cluster [DBG] pgmap v6272: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:42 gibba032 bash[24004]: cluster 2021-03-10T17:21:41.661939+0000 mgr.gibba032.whopeo (mgr.24625) 11012 : cluster [DBG] pgmap v6273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:42 gibba011 bash[30791]: cluster 2021-03-10T17:21:41.661939+0000 mgr.gibba032.whopeo (mgr.24625) 11012 : cluster [DBG] pgmap v6273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:42 gibba011 bash[31774]: cluster 2021-03-10T17:21:41.661939+0000 mgr.gibba032.whopeo (mgr.24625) 11012 : cluster [DBG] pgmap v6273: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:44 gibba032 bash[24004]: cluster 2021-03-10T17:21:43.663428+0000 mgr.gibba032.whopeo (mgr.24625) 11013 : cluster [DBG] pgmap v6274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:45.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:44 gibba011 bash[30791]: cluster 2021-03-10T17:21:43.663428+0000 mgr.gibba032.whopeo (mgr.24625) 11013 : cluster [DBG] pgmap v6274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:45.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:44 gibba011 bash[31774]: cluster 2021-03-10T17:21:43.663428+0000 mgr.gibba032.whopeo (mgr.24625) 11013 : cluster [DBG] pgmap v6274: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:46 gibba032 bash[24004]: cluster 2021-03-10T17:21:45.663941+0000 mgr.gibba032.whopeo (mgr.24625) 11014 : cluster [DBG] pgmap v6275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:47.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:46 gibba011 bash[30791]: cluster 2021-03-10T17:21:45.663941+0000 mgr.gibba032.whopeo (mgr.24625) 11014 : cluster [DBG] pgmap v6275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:47.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:46 gibba011 bash[31774]: cluster 2021-03-10T17:21:45.663941+0000 mgr.gibba032.whopeo (mgr.24625) 11014 : cluster [DBG] pgmap v6275: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:48 gibba032 bash[24004]: cephadm 2021-03-10T17:21:47.117009+0000 mgr.gibba032.whopeo (mgr.24625) 11015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:48 gibba032 bash[24004]: cluster 2021-03-10T17:21:47.665611+0000 mgr.gibba032.whopeo (mgr.24625) 11016 : cluster [DBG] pgmap v6276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:49.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:48 gibba011 bash[30791]: cephadm 2021-03-10T17:21:47.117009+0000 mgr.gibba032.whopeo (mgr.24625) 11015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:49.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:48 gibba011 bash[30791]: cluster 2021-03-10T17:21:47.665611+0000 mgr.gibba032.whopeo (mgr.24625) 11016 : cluster [DBG] pgmap v6276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:49.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:48 gibba011 bash[31774]: cephadm 2021-03-10T17:21:47.117009+0000 mgr.gibba032.whopeo (mgr.24625) 11015 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:21:49.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:48 gibba011 bash[31774]: cluster 2021-03-10T17:21:47.665611+0000 mgr.gibba032.whopeo (mgr.24625) 11016 : cluster [DBG] pgmap v6276: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:49.986 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:21:50.218 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:21:50.219 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 18s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:21:50.219 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:21:50.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:21:50.219 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:21:50.220 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:21:50.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:21:50.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:21:50.220 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:21:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 18s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:21:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:21:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:21:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:21:50.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:21:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 18s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:21:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:21:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:21:50.222 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:21:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:21:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:21:50.223 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 18s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:21:50.466 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:21:50.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:21:50.467 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:21:50.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:50.468 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:21:50.469 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:21:50.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:50.469 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:21:50.470 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:21:50.470 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:21:50.471 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:21:50.471 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:21:50.471 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:21:50.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:21:50.472 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:21:51.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:50 gibba032 bash[24004]: cluster 2021-03-10T17:21:49.667115+0000 mgr.gibba032.whopeo (mgr.24625) 11017 : cluster [DBG] pgmap v6277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:51.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:50 gibba032 bash[24004]: audit 2021-03-10T17:21:50.466111+0000 mon.a (mon.0) 3656 : audit [DBG] from='client.? 172.21.2.111:0/851442672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:50 gibba011 bash[30791]: cluster 2021-03-10T17:21:49.667115+0000 mgr.gibba032.whopeo (mgr.24625) 11017 : cluster [DBG] pgmap v6277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:51.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:50 gibba011 bash[30791]: audit 2021-03-10T17:21:50.466111+0000 mon.a (mon.0) 3656 : audit [DBG] from='client.? 172.21.2.111:0/851442672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:50 gibba011 bash[31774]: cluster 2021-03-10T17:21:49.667115+0000 mgr.gibba032.whopeo (mgr.24625) 11017 : cluster [DBG] pgmap v6277: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:51.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:50 gibba011 bash[31774]: audit 2021-03-10T17:21:50.466111+0000 mon.a (mon.0) 3656 : audit [DBG] from='client.? 172.21.2.111:0/851442672' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:21:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:51 gibba011 bash[30791]: audit 2021-03-10T17:21:49.975166+0000 mgr.gibba032.whopeo (mgr.24625) 11018 : audit [DBG] from='client.51435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:52.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:51 gibba011 bash[30791]: audit 2021-03-10T17:21:50.204759+0000 mgr.gibba032.whopeo (mgr.24625) 11019 : audit [DBG] from='client.61423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:52.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:51 gibba011 bash[31774]: audit 2021-03-10T17:21:49.975166+0000 mgr.gibba032.whopeo (mgr.24625) 11018 : audit [DBG] from='client.51435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:52.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:51 gibba011 bash[31774]: audit 2021-03-10T17:21:50.204759+0000 mgr.gibba032.whopeo (mgr.24625) 11019 : audit [DBG] from='client.61423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:51 gibba032 bash[24004]: audit 2021-03-10T17:21:49.975166+0000 mgr.gibba032.whopeo (mgr.24625) 11018 : audit [DBG] from='client.51435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:52.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:51 gibba032 bash[24004]: audit 2021-03-10T17:21:50.204759+0000 mgr.gibba032.whopeo (mgr.24625) 11019 : audit [DBG] from='client.61423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:21:53.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:52 gibba011 bash[30791]: cluster 2021-03-10T17:21:51.668809+0000 mgr.gibba032.whopeo (mgr.24625) 11020 : cluster [DBG] pgmap v6278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:53.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:52 gibba011 bash[31774]: cluster 2021-03-10T17:21:51.668809+0000 mgr.gibba032.whopeo (mgr.24625) 11020 : cluster [DBG] pgmap v6278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:53.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:52 gibba032 bash[24004]: cluster 2021-03-10T17:21:51.668809+0000 mgr.gibba032.whopeo (mgr.24625) 11020 : cluster [DBG] pgmap v6278: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:54 gibba011 bash[30791]: cluster 2021-03-10T17:21:53.670403+0000 mgr.gibba032.whopeo (mgr.24625) 11021 : cluster [DBG] pgmap v6279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:55.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:54 gibba011 bash[31774]: cluster 2021-03-10T17:21:53.670403+0000 mgr.gibba032.whopeo (mgr.24625) 11021 : cluster [DBG] pgmap v6279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:54 gibba032 bash[24004]: cluster 2021-03-10T17:21:53.670403+0000 mgr.gibba032.whopeo (mgr.24625) 11021 : cluster [DBG] pgmap v6279: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:56 gibba011 bash[30791]: cluster 2021-03-10T17:21:55.670936+0000 mgr.gibba032.whopeo (mgr.24625) 11022 : cluster [DBG] pgmap v6280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:56 gibba011 bash[31774]: cluster 2021-03-10T17:21:55.670936+0000 mgr.gibba032.whopeo (mgr.24625) 11022 : cluster [DBG] pgmap v6280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:56 gibba032 bash[24004]: cluster 2021-03-10T17:21:55.670936+0000 mgr.gibba032.whopeo (mgr.24625) 11022 : cluster [DBG] pgmap v6280: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.452382+0000 mon.a (mon.0) 3657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:21:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.453091+0000 mon.a (mon.0) 3658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:21:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.630966+0000 mon.a (mon.0) 3659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:21:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.631821+0000 mon.a (mon.0) 3660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:21:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.632486+0000 mon.a (mon.0) 3661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:21:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[30791]: audit 2021-03-10T17:21:57.633169+0000 mon.a (mon.0) 3662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:21:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.452382+0000 mon.a (mon.0) 3657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:21:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.453091+0000 mon.a (mon.0) 3658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:21:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.630966+0000 mon.a (mon.0) 3659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:21:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.631821+0000 mon.a (mon.0) 3660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:21:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.632486+0000 mon.a (mon.0) 3661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:21:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:57 gibba011 bash[31774]: audit 2021-03-10T17:21:57.633169+0000 mon.a (mon.0) 3662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:21:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.452382+0000 mon.a (mon.0) 3657 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:21:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.453091+0000 mon.a (mon.0) 3658 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:21:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.630966+0000 mon.a (mon.0) 3659 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:21:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.631821+0000 mon.a (mon.0) 3660 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:21:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.632486+0000 mon.a (mon.0) 3661 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:21:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:57 gibba032 bash[24004]: audit 2021-03-10T17:21:57.633169+0000 mon.a (mon.0) 3662 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:21:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:21:58 gibba011 bash[30791]: cluster 2021-03-10T17:21:57.672514+0000 mgr.gibba032.whopeo (mgr.24625) 11023 : cluster [DBG] pgmap v6281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:59.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:21:58 gibba011 bash[31774]: cluster 2021-03-10T17:21:57.672514+0000 mgr.gibba032.whopeo (mgr.24625) 11023 : cluster [DBG] pgmap v6281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:21:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:21:58 gibba032 bash[24004]: cluster 2021-03-10T17:21:57.672514+0000 mgr.gibba032.whopeo (mgr.24625) 11023 : cluster [DBG] pgmap v6281: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:00 gibba011 bash[30791]: cluster 2021-03-10T17:21:59.673986+0000 mgr.gibba032.whopeo (mgr.24625) 11024 : cluster [DBG] pgmap v6282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:01.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:00 gibba011 bash[31774]: cluster 2021-03-10T17:21:59.673986+0000 mgr.gibba032.whopeo (mgr.24625) 11024 : cluster [DBG] pgmap v6282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:00 gibba032 bash[24004]: cluster 2021-03-10T17:21:59.673986+0000 mgr.gibba032.whopeo (mgr.24625) 11024 : cluster [DBG] pgmap v6282: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:02 gibba032 bash[24004]: cluster 2021-03-10T17:22:01.675745+0000 mgr.gibba032.whopeo (mgr.24625) 11025 : cluster [DBG] pgmap v6283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:02 gibba011 bash[30791]: cluster 2021-03-10T17:22:01.675745+0000 mgr.gibba032.whopeo (mgr.24625) 11025 : cluster [DBG] pgmap v6283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:02 gibba011 bash[31774]: cluster 2021-03-10T17:22:01.675745+0000 mgr.gibba032.whopeo (mgr.24625) 11025 : cluster [DBG] pgmap v6283: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:03 gibba032 bash[24004]: cephadm 2021-03-10T17:22:02.132561+0000 mgr.gibba032.whopeo (mgr.24625) 11026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:03.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:03 gibba011 bash[30791]: cephadm 2021-03-10T17:22:02.132561+0000 mgr.gibba032.whopeo (mgr.24625) 11026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:03.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:03 gibba011 bash[31774]: cephadm 2021-03-10T17:22:02.132561+0000 mgr.gibba032.whopeo (mgr.24625) 11026 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:04.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:04 gibba032 bash[24004]: cluster 2021-03-10T17:22:03.677337+0000 mgr.gibba032.whopeo (mgr.24625) 11027 : cluster [DBG] pgmap v6284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:04.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:04 gibba011 bash[30791]: cluster 2021-03-10T17:22:03.677337+0000 mgr.gibba032.whopeo (mgr.24625) 11027 : cluster [DBG] pgmap v6284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:04.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:04 gibba011 bash[31774]: cluster 2021-03-10T17:22:03.677337+0000 mgr.gibba032.whopeo (mgr.24625) 11027 : cluster [DBG] pgmap v6284: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:07.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:06 gibba032 bash[24004]: cluster 2021-03-10T17:22:05.677986+0000 mgr.gibba032.whopeo (mgr.24625) 11028 : cluster [DBG] pgmap v6285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:06 gibba011 bash[30791]: cluster 2021-03-10T17:22:05.677986+0000 mgr.gibba032.whopeo (mgr.24625) 11028 : cluster [DBG] pgmap v6285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:06 gibba011 bash[31774]: cluster 2021-03-10T17:22:05.677986+0000 mgr.gibba032.whopeo (mgr.24625) 11028 : cluster [DBG] pgmap v6285: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:09.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:08 gibba032 bash[24004]: cluster 2021-03-10T17:22:07.679567+0000 mgr.gibba032.whopeo (mgr.24625) 11029 : cluster [DBG] pgmap v6286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:08 gibba011 bash[30791]: cluster 2021-03-10T17:22:07.679567+0000 mgr.gibba032.whopeo (mgr.24625) 11029 : cluster [DBG] pgmap v6286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:09.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:08 gibba011 bash[31774]: cluster 2021-03-10T17:22:07.679567+0000 mgr.gibba032.whopeo (mgr.24625) 11029 : cluster [DBG] pgmap v6286: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:10 gibba011 bash[30791]: cluster 2021-03-10T17:22:09.680904+0000 mgr.gibba032.whopeo (mgr.24625) 11030 : cluster [DBG] pgmap v6287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:11.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:10 gibba011 bash[31774]: cluster 2021-03-10T17:22:09.680904+0000 mgr.gibba032.whopeo (mgr.24625) 11030 : cluster [DBG] pgmap v6287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:11.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:10 gibba032 bash[24004]: cluster 2021-03-10T17:22:09.680904+0000 mgr.gibba032.whopeo (mgr.24625) 11030 : cluster [DBG] pgmap v6287: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:12.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:12 gibba032 bash[24004]: cluster 2021-03-10T17:22:11.682644+0000 mgr.gibba032.whopeo (mgr.24625) 11031 : cluster [DBG] pgmap v6288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:12 gibba011 bash[30791]: cluster 2021-03-10T17:22:11.682644+0000 mgr.gibba032.whopeo (mgr.24625) 11031 : cluster [DBG] pgmap v6288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:12.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:12 gibba011 bash[31774]: cluster 2021-03-10T17:22:11.682644+0000 mgr.gibba032.whopeo (mgr.24625) 11031 : cluster [DBG] pgmap v6288: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:15.007 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:14 gibba032 bash[24004]: cluster 2021-03-10T17:22:13.684211+0000 mgr.gibba032.whopeo (mgr.24625) 11032 : cluster [DBG] pgmap v6289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:15.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:14 gibba011 bash[30791]: cluster 2021-03-10T17:22:13.684211+0000 mgr.gibba032.whopeo (mgr.24625) 11032 : cluster [DBG] pgmap v6289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:15.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:14 gibba011 bash[31774]: cluster 2021-03-10T17:22:13.684211+0000 mgr.gibba032.whopeo (mgr.24625) 11032 : cluster [DBG] pgmap v6289: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:17.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:16 gibba011 bash[30791]: cluster 2021-03-10T17:22:15.685488+0000 mgr.gibba032.whopeo (mgr.24625) 11033 : cluster [DBG] pgmap v6290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:17.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:16 gibba011 bash[31774]: cluster 2021-03-10T17:22:15.685488+0000 mgr.gibba032.whopeo (mgr.24625) 11033 : cluster [DBG] pgmap v6290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:17.151 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:16 gibba032 bash[24004]: cluster 2021-03-10T17:22:15.685488+0000 mgr.gibba032.whopeo (mgr.24625) 11033 : cluster [DBG] pgmap v6290: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:18 gibba011 bash[30791]: cephadm 2021-03-10T17:22:17.148577+0000 mgr.gibba032.whopeo (mgr.24625) 11034 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:19.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:18 gibba011 bash[30791]: cluster 2021-03-10T17:22:17.687231+0000 mgr.gibba032.whopeo (mgr.24625) 11035 : cluster [DBG] pgmap v6291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:18 gibba011 bash[31774]: cephadm 2021-03-10T17:22:17.148577+0000 mgr.gibba032.whopeo (mgr.24625) 11034 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:19.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:18 gibba011 bash[31774]: cluster 2021-03-10T17:22:17.687231+0000 mgr.gibba032.whopeo (mgr.24625) 11035 : cluster [DBG] pgmap v6291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:19.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:18 gibba032 bash[24004]: cephadm 2021-03-10T17:22:17.148577+0000 mgr.gibba032.whopeo (mgr.24625) 11034 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:18 gibba032 bash[24004]: cluster 2021-03-10T17:22:17.687231+0000 mgr.gibba032.whopeo (mgr.24625) 11035 : cluster [DBG] pgmap v6291: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:20.709 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:22:20.937 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:22:20.938 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 49s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:22:20.938 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 10m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:22:20.938 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:22:20.938 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:22:20.939 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:22:20.939 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:22:20.939 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:22:20.939 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:22:20.940 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 49s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:22:20.940 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 10m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:22:20.940 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:22:20.940 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:22:20.940 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:22:20.941 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 49s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:22:20.941 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:22:20.941 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:22:20.941 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:22:20.941 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:22:20.942 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 10m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:22:20.942 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 49s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:22:21.078 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:20 gibba011 bash[30791]: cluster 2021-03-10T17:22:19.688646+0000 mgr.gibba032.whopeo (mgr.24625) 11036 : cluster [DBG] pgmap v6292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:21.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:20 gibba011 bash[31774]: cluster 2021-03-10T17:22:19.688646+0000 mgr.gibba032.whopeo (mgr.24625) 11036 : cluster [DBG] pgmap v6292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:21.185 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:22:21.185 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:22:21.186 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:22:21.186 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:21.186 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:22:21.186 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:22:21.186 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:21.187 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:22:21.187 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:22:21.187 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:21.187 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:22:21.188 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:22:21.188 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:22:21.188 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:22:21.188 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:22:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:20 gibba032 bash[24004]: cluster 2021-03-10T17:22:19.688646+0000 mgr.gibba032.whopeo (mgr.24625) 11036 : cluster [DBG] pgmap v6292: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:22.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[30791]: audit 2021-03-10T17:22:20.697698+0000 mgr.gibba032.whopeo (mgr.24625) 11037 : audit [DBG] from='client.51453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[30791]: audit 2021-03-10T17:22:20.923434+0000 mgr.gibba032.whopeo (mgr.24625) 11038 : audit [DBG] from='client.51459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[30791]: audit 2021-03-10T17:22:21.185245+0000 mon.a (mon.0) 3663 : audit [DBG] from='client.? 172.21.2.111:0/343926360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:22.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[31774]: audit 2021-03-10T17:22:20.697698+0000 mgr.gibba032.whopeo (mgr.24625) 11037 : audit [DBG] from='client.51453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[31774]: audit 2021-03-10T17:22:20.923434+0000 mgr.gibba032.whopeo (mgr.24625) 11038 : audit [DBG] from='client.51459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:21 gibba011 bash[31774]: audit 2021-03-10T17:22:21.185245+0000 mon.a (mon.0) 3663 : audit [DBG] from='client.? 172.21.2.111:0/343926360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:21 gibba032 bash[24004]: audit 2021-03-10T17:22:20.697698+0000 mgr.gibba032.whopeo (mgr.24625) 11037 : audit [DBG] from='client.51453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:21 gibba032 bash[24004]: audit 2021-03-10T17:22:20.923434+0000 mgr.gibba032.whopeo (mgr.24625) 11038 : audit [DBG] from='client.51459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:22.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:21 gibba032 bash[24004]: audit 2021-03-10T17:22:21.185245+0000 mon.a (mon.0) 3663 : audit [DBG] from='client.? 172.21.2.111:0/343926360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:23.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:22 gibba011 bash[30791]: cluster 2021-03-10T17:22:21.690430+0000 mgr.gibba032.whopeo (mgr.24625) 11039 : cluster [DBG] pgmap v6293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:23.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:22 gibba011 bash[31774]: cluster 2021-03-10T17:22:21.690430+0000 mgr.gibba032.whopeo (mgr.24625) 11039 : cluster [DBG] pgmap v6293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:23.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:22 gibba032 bash[24004]: cluster 2021-03-10T17:22:21.690430+0000 mgr.gibba032.whopeo (mgr.24625) 11039 : cluster [DBG] pgmap v6293: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:25.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:24 gibba011 bash[30791]: cluster 2021-03-10T17:22:23.692082+0000 mgr.gibba032.whopeo (mgr.24625) 11040 : cluster [DBG] pgmap v6294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:25.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:24 gibba011 bash[31774]: cluster 2021-03-10T17:22:23.692082+0000 mgr.gibba032.whopeo (mgr.24625) 11040 : cluster [DBG] pgmap v6294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:24 gibba032 bash[24004]: cluster 2021-03-10T17:22:23.692082+0000 mgr.gibba032.whopeo (mgr.24625) 11040 : cluster [DBG] pgmap v6294: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:26 gibba011 bash[30791]: cluster 2021-03-10T17:22:25.692775+0000 mgr.gibba032.whopeo (mgr.24625) 11041 : cluster [DBG] pgmap v6295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:27.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:26 gibba011 bash[31774]: cluster 2021-03-10T17:22:25.692775+0000 mgr.gibba032.whopeo (mgr.24625) 11041 : cluster [DBG] pgmap v6295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:26 gibba032 bash[24004]: cluster 2021-03-10T17:22:25.692775+0000 mgr.gibba032.whopeo (mgr.24625) 11041 : cluster [DBG] pgmap v6295: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:28 gibba011 bash[30791]: cluster 2021-03-10T17:22:27.694442+0000 mgr.gibba032.whopeo (mgr.24625) 11042 : cluster [DBG] pgmap v6296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:28 gibba011 bash[31774]: cluster 2021-03-10T17:22:27.694442+0000 mgr.gibba032.whopeo (mgr.24625) 11042 : cluster [DBG] pgmap v6296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:28 gibba032 bash[24004]: cluster 2021-03-10T17:22:27.694442+0000 mgr.gibba032.whopeo (mgr.24625) 11042 : cluster [DBG] pgmap v6296: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:31.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:30 gibba011 bash[30791]: cluster 2021-03-10T17:22:29.695837+0000 mgr.gibba032.whopeo (mgr.24625) 11043 : cluster [DBG] pgmap v6297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:31.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:30 gibba011 bash[31774]: cluster 2021-03-10T17:22:29.695837+0000 mgr.gibba032.whopeo (mgr.24625) 11043 : cluster [DBG] pgmap v6297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:30 gibba032 bash[24004]: cluster 2021-03-10T17:22:29.695837+0000 mgr.gibba032.whopeo (mgr.24625) 11043 : cluster [DBG] pgmap v6297: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:32 gibba032 bash[24004]: cluster 2021-03-10T17:22:31.697591+0000 mgr.gibba032.whopeo (mgr.24625) 11044 : cluster [DBG] pgmap v6298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:32 gibba011 bash[30791]: cluster 2021-03-10T17:22:31.697591+0000 mgr.gibba032.whopeo (mgr.24625) 11044 : cluster [DBG] pgmap v6298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:32.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:32 gibba011 bash[31774]: cluster 2021-03-10T17:22:31.697591+0000 mgr.gibba032.whopeo (mgr.24625) 11044 : cluster [DBG] pgmap v6298: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:33 gibba032 bash[24004]: cephadm 2021-03-10T17:22:32.164595+0000 mgr.gibba032.whopeo (mgr.24625) 11045 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:22:33.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:33 gibba032 bash[24004]: audit 2021-03-10T17:22:32.167466+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:33.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:33 gibba011 bash[30791]: cephadm 2021-03-10T17:22:32.164595+0000 mgr.gibba032.whopeo (mgr.24625) 11045 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:22:33.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:33 gibba011 bash[30791]: audit 2021-03-10T17:22:32.167466+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:33.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:33 gibba011 bash[31774]: cephadm 2021-03-10T17:22:32.164595+0000 mgr.gibba032.whopeo (mgr.24625) 11045 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:22:33.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:33 gibba011 bash[31774]: audit 2021-03-10T17:22:32.167466+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:34.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:34 gibba032 bash[24004]: cluster 2021-03-10T17:22:33.699072+0000 mgr.gibba032.whopeo (mgr.24625) 11046 : cluster [DBG] pgmap v6299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:34.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:34 gibba011 bash[30791]: cluster 2021-03-10T17:22:33.699072+0000 mgr.gibba032.whopeo (mgr.24625) 11046 : cluster [DBG] pgmap v6299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:34.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:34 gibba011 bash[31774]: cluster 2021-03-10T17:22:33.699072+0000 mgr.gibba032.whopeo (mgr.24625) 11046 : cluster [DBG] pgmap v6299: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:37.122 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:36 gibba032 bash[24004]: cluster 2021-03-10T17:22:35.699557+0000 mgr.gibba032.whopeo (mgr.24625) 11047 : cluster [DBG] pgmap v6300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:37.123 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:36 gibba032 bash[24004]: audit 2021-03-10T17:22:35.845496+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:37.123 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:36 gibba032 bash[24004]: audit 2021-03-10T17:22:35.846358+0000 mon.a (mon.0) 3666 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:22:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[30791]: cluster 2021-03-10T17:22:35.699557+0000 mgr.gibba032.whopeo (mgr.24625) 11047 : cluster [DBG] pgmap v6300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[30791]: audit 2021-03-10T17:22:35.845496+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[30791]: audit 2021-03-10T17:22:35.846358+0000 mon.a (mon.0) 3666 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:22:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[31774]: cluster 2021-03-10T17:22:35.699557+0000 mgr.gibba032.whopeo (mgr.24625) 11047 : cluster [DBG] pgmap v6300: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[31774]: audit 2021-03-10T17:22:35.845496+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:36 gibba011 bash[31774]: audit 2021-03-10T17:22:35.846358+0000 mon.a (mon.0) 3666 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "osd","key": "container_image"}]: dispatch 2021-03-10T17:22:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:38 gibba032 bash[24004]: cluster 2021-03-10T17:22:37.701224+0000 mgr.gibba032.whopeo (mgr.24625) 11048 : cluster [DBG] pgmap v6301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:38 gibba011 bash[30791]: cluster 2021-03-10T17:22:37.701224+0000 mgr.gibba032.whopeo (mgr.24625) 11048 : cluster [DBG] pgmap v6301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:39.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:38 gibba011 bash[31774]: cluster 2021-03-10T17:22:37.701224+0000 mgr.gibba032.whopeo (mgr.24625) 11048 : cluster [DBG] pgmap v6301: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:41.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:40 gibba032 bash[24004]: cluster 2021-03-10T17:22:39.702779+0000 mgr.gibba032.whopeo (mgr.24625) 11049 : cluster [DBG] pgmap v6302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:40 gibba032 bash[24004]: audit 2021-03-10T17:22:40.523935+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:40 gibba032 bash[24004]: audit 2021-03-10T17:22:40.697702+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[30791]: cluster 2021-03-10T17:22:39.702779+0000 mgr.gibba032.whopeo (mgr.24625) 11049 : cluster [DBG] pgmap v6302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[30791]: audit 2021-03-10T17:22:40.523935+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:41.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[30791]: audit 2021-03-10T17:22:40.697702+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[31774]: cluster 2021-03-10T17:22:39.702779+0000 mgr.gibba032.whopeo (mgr.24625) 11049 : cluster [DBG] pgmap v6302: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[31774]: audit 2021-03-10T17:22:40.523935+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:22:41.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:40 gibba011 bash[31774]: audit 2021-03-10T17:22:40.697702+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.696745+0000 mgr.gibba032.whopeo (mgr.24625) 11050 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:22:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.911259+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.911852+0000 mgr.gibba032.whopeo (mgr.24625) 11051 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:22:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.914397+0000 mgr.gibba032.whopeo (mgr.24625) 11052 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:22:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.914563+0000 mon.a (mon.0) 3670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:22:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.916016+0000 mgr.gibba032.whopeo (mgr.24625) 11053 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:22:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.916154+0000 mgr.gibba032.whopeo (mgr.24625) 11054 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:22:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.916704+0000 mon.a (mon.0) 3671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.917702+0000 mgr.gibba032.whopeo (mgr.24625) 11055 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:22:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.917740+0000 mgr.gibba032.whopeo (mgr.24625) 11056 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:22:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.918439+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.919147+0000 mgr.gibba032.whopeo (mgr.24625) 11057 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:22:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.919933+0000 mon.a (mon.0) 3673 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.920393+0000 mgr.gibba032.whopeo (mgr.24625) 11058 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:22:42.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.920430+0000 mgr.gibba032.whopeo (mgr.24625) 11059 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:22:42.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.922155+0000 mgr.gibba032.whopeo (mgr.24625) 11060 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:22:42.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.922569+0000 mgr.gibba032.whopeo (mgr.24625) 11061 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:22:42.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.922804+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.924262+0000 mgr.gibba032.whopeo (mgr.24625) 11062 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:22:42.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.924302+0000 mgr.gibba032.whopeo (mgr.24625) 11063 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:22:42.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: cephadm 2021-03-10T17:22:40.924415+0000 mgr.gibba032.whopeo (mgr.24625) 11064 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:42.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:41 gibba032 bash[24004]: audit 2021-03-10T17:22:40.924849+0000 mon.a (mon.0) 3675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.696745+0000 mgr.gibba032.whopeo (mgr.24625) 11050 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:22:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.911259+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.911852+0000 mgr.gibba032.whopeo (mgr.24625) 11051 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:22:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.914397+0000 mgr.gibba032.whopeo (mgr.24625) 11052 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:22:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.914563+0000 mon.a (mon.0) 3670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:22:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.916016+0000 mgr.gibba032.whopeo (mgr.24625) 11053 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:22:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.916154+0000 mgr.gibba032.whopeo (mgr.24625) 11054 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:22:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.916704+0000 mon.a (mon.0) 3671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.917702+0000 mgr.gibba032.whopeo (mgr.24625) 11055 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:22:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.917740+0000 mgr.gibba032.whopeo (mgr.24625) 11056 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:22:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.918439+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.919147+0000 mgr.gibba032.whopeo (mgr.24625) 11057 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:22:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.919933+0000 mon.a (mon.0) 3673 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.920393+0000 mgr.gibba032.whopeo (mgr.24625) 11058 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:22:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.920430+0000 mgr.gibba032.whopeo (mgr.24625) 11059 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:22:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.922155+0000 mgr.gibba032.whopeo (mgr.24625) 11060 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:22:42.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.922569+0000 mgr.gibba032.whopeo (mgr.24625) 11061 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:22:42.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.922804+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.924262+0000 mgr.gibba032.whopeo (mgr.24625) 11062 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:22:42.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.924302+0000 mgr.gibba032.whopeo (mgr.24625) 11063 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:22:42.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: cephadm 2021-03-10T17:22:40.924415+0000 mgr.gibba032.whopeo (mgr.24625) 11064 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:42.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[30791]: audit 2021-03-10T17:22:40.924849+0000 mon.a (mon.0) 3675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.696745+0000 mgr.gibba032.whopeo (mgr.24625) 11050 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:22:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.911259+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:22:42.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.911852+0000 mgr.gibba032.whopeo (mgr.24625) 11051 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:22:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.914397+0000 mgr.gibba032.whopeo (mgr.24625) 11052 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:22:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.914563+0000 mon.a (mon.0) 3670 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:22:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.916016+0000 mgr.gibba032.whopeo (mgr.24625) 11053 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:22:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.916154+0000 mgr.gibba032.whopeo (mgr.24625) 11054 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:22:42.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.916704+0000 mon.a (mon.0) 3671 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.917702+0000 mgr.gibba032.whopeo (mgr.24625) 11055 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:22:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.917740+0000 mgr.gibba032.whopeo (mgr.24625) 11056 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:22:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.918439+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.919147+0000 mgr.gibba032.whopeo (mgr.24625) 11057 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:22:42.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.919933+0000 mon.a (mon.0) 3673 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.920393+0000 mgr.gibba032.whopeo (mgr.24625) 11058 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:22:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.920430+0000 mgr.gibba032.whopeo (mgr.24625) 11059 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:22:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.922155+0000 mgr.gibba032.whopeo (mgr.24625) 11060 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:22:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.922569+0000 mgr.gibba032.whopeo (mgr.24625) 11061 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:22:42.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.922804+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:42.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.924262+0000 mgr.gibba032.whopeo (mgr.24625) 11062 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:22:42.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.924302+0000 mgr.gibba032.whopeo (mgr.24625) 11063 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:22:42.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: cephadm 2021-03-10T17:22:40.924415+0000 mgr.gibba032.whopeo (mgr.24625) 11064 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:42.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:41 gibba011 bash[31774]: audit 2021-03-10T17:22:40.924849+0000 mon.a (mon.0) 3675 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:43.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:42 gibba032 bash[24004]: cluster 2021-03-10T17:22:41.704517+0000 mgr.gibba032.whopeo (mgr.24625) 11065 : cluster [DBG] pgmap v6303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:42 gibba011 bash[30791]: cluster 2021-03-10T17:22:41.704517+0000 mgr.gibba032.whopeo (mgr.24625) 11065 : cluster [DBG] pgmap v6303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:43.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:42 gibba011 bash[31774]: cluster 2021-03-10T17:22:41.704517+0000 mgr.gibba032.whopeo (mgr.24625) 11065 : cluster [DBG] pgmap v6303: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:44 gibba032 bash[24004]: cluster 2021-03-10T17:22:43.706104+0000 mgr.gibba032.whopeo (mgr.24625) 11066 : cluster [DBG] pgmap v6304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:44 gibba011 bash[30791]: cluster 2021-03-10T17:22:43.706104+0000 mgr.gibba032.whopeo (mgr.24625) 11066 : cluster [DBG] pgmap v6304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:45.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:44 gibba011 bash[31774]: cluster 2021-03-10T17:22:43.706104+0000 mgr.gibba032.whopeo (mgr.24625) 11066 : cluster [DBG] pgmap v6304: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:46 gibba032 bash[24004]: cluster 2021-03-10T17:22:45.706887+0000 mgr.gibba032.whopeo (mgr.24625) 11067 : cluster [DBG] pgmap v6305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:46 gibba011 bash[30791]: cluster 2021-03-10T17:22:45.706887+0000 mgr.gibba032.whopeo (mgr.24625) 11067 : cluster [DBG] pgmap v6305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:46 gibba011 bash[31774]: cluster 2021-03-10T17:22:45.706887+0000 mgr.gibba032.whopeo (mgr.24625) 11067 : cluster [DBG] pgmap v6305: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:48 gibba032 bash[24004]: cluster 2021-03-10T17:22:47.708587+0000 mgr.gibba032.whopeo (mgr.24625) 11068 : cluster [DBG] pgmap v6306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:48 gibba011 bash[30791]: cluster 2021-03-10T17:22:47.708587+0000 mgr.gibba032.whopeo (mgr.24625) 11068 : cluster [DBG] pgmap v6306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:48 gibba011 bash[31774]: cluster 2021-03-10T17:22:47.708587+0000 mgr.gibba032.whopeo (mgr.24625) 11068 : cluster [DBG] pgmap v6306: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:50 gibba032 bash[24004]: cluster 2021-03-10T17:22:49.709956+0000 mgr.gibba032.whopeo (mgr.24625) 11069 : cluster [DBG] pgmap v6307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:51.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:50 gibba011 bash[30791]: cluster 2021-03-10T17:22:49.709956+0000 mgr.gibba032.whopeo (mgr.24625) 11069 : cluster [DBG] pgmap v6307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:50 gibba011 bash[31774]: cluster 2021-03-10T17:22:49.709956+0000 mgr.gibba032.whopeo (mgr.24625) 11069 : cluster [DBG] pgmap v6307: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:51.460 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:22:51.685 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:22:51.685 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 79s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:22:51.685 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 15s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:22:51.685 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:22:51.686 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:22:51.686 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:22:51.686 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:22:51.686 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:22:51.687 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:22:51.687 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 79s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:22:51.687 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 15s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:22:51.687 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:22:51.687 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:22:51.688 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:22:51.688 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:22:51.688 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:22:51.688 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:22:51.688 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:22:51.689 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:22:51.689 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 15s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:22:51.689 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 79s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:22:51.935 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:22:51.935 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:22:51.936 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:22:51.936 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:51.936 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:22:51.936 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:22:51.937 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:51.937 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:22:51.937 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:22:51.937 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:22:51.937 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:22:51.938 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:22:51.938 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:22:51.938 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:22:51.938 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:22:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:52 gibba032 bash[24004]: audit 2021-03-10T17:22:51.448244+0000 mgr.gibba032.whopeo (mgr.24625) 11070 : audit [DBG] from='client.61453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:52 gibba032 bash[24004]: audit 2021-03-10T17:22:51.670738+0000 mgr.gibba032.whopeo (mgr.24625) 11071 : audit [DBG] from='client.61459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:52 gibba032 bash[24004]: cluster 2021-03-10T17:22:51.710845+0000 mgr.gibba032.whopeo (mgr.24625) 11072 : cluster [DBG] pgmap v6308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:52 gibba032 bash[24004]: audit 2021-03-10T17:22:51.935101+0000 mon.c (mon.1) 212 : audit [DBG] from='client.? 172.21.2.111:0/4209841292' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[30791]: audit 2021-03-10T17:22:51.448244+0000 mgr.gibba032.whopeo (mgr.24625) 11070 : audit [DBG] from='client.61453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[30791]: audit 2021-03-10T17:22:51.670738+0000 mgr.gibba032.whopeo (mgr.24625) 11071 : audit [DBG] from='client.61459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[30791]: cluster 2021-03-10T17:22:51.710845+0000 mgr.gibba032.whopeo (mgr.24625) 11072 : cluster [DBG] pgmap v6308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[30791]: audit 2021-03-10T17:22:51.935101+0000 mon.c (mon.1) 212 : audit [DBG] from='client.? 172.21.2.111:0/4209841292' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[31774]: audit 2021-03-10T17:22:51.448244+0000 mgr.gibba032.whopeo (mgr.24625) 11070 : audit [DBG] from='client.61453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[31774]: audit 2021-03-10T17:22:51.670738+0000 mgr.gibba032.whopeo (mgr.24625) 11071 : audit [DBG] from='client.61459 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:22:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[31774]: cluster 2021-03-10T17:22:51.710845+0000 mgr.gibba032.whopeo (mgr.24625) 11072 : cluster [DBG] pgmap v6308: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:52 gibba011 bash[31774]: audit 2021-03-10T17:22:51.935101+0000 mon.c (mon.1) 212 : audit [DBG] from='client.? 172.21.2.111:0/4209841292' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:22:55.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:54 gibba011 bash[30791]: cluster 2021-03-10T17:22:53.712252+0000 mgr.gibba032.whopeo (mgr.24625) 11073 : cluster [DBG] pgmap v6309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:55.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:54 gibba011 bash[31774]: cluster 2021-03-10T17:22:53.712252+0000 mgr.gibba032.whopeo (mgr.24625) 11073 : cluster [DBG] pgmap v6309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:54 gibba032 bash[24004]: cluster 2021-03-10T17:22:53.712252+0000 mgr.gibba032.whopeo (mgr.24625) 11073 : cluster [DBG] pgmap v6309: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:56 gibba011 bash[30791]: cluster 2021-03-10T17:22:55.713553+0000 mgr.gibba032.whopeo (mgr.24625) 11074 : cluster [DBG] pgmap v6310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:57.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:56 gibba011 bash[30791]: cephadm 2021-03-10T17:22:55.926093+0000 mgr.gibba032.whopeo (mgr.24625) 11075 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:56 gibba011 bash[31774]: cluster 2021-03-10T17:22:55.713553+0000 mgr.gibba032.whopeo (mgr.24625) 11074 : cluster [DBG] pgmap v6310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:56 gibba011 bash[31774]: cephadm 2021-03-10T17:22:55.926093+0000 mgr.gibba032.whopeo (mgr.24625) 11075 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:56 gibba032 bash[24004]: cluster 2021-03-10T17:22:55.713553+0000 mgr.gibba032.whopeo (mgr.24625) 11074 : cluster [DBG] pgmap v6310: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:56 gibba032 bash[24004]: cephadm 2021-03-10T17:22:55.926093+0000 mgr.gibba032.whopeo (mgr.24625) 11075 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:22:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.460363+0000 mon.a (mon.0) 3676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:22:58.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.461023+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:22:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.639873+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T17:22:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.640648+0000 mon.a (mon.0) 3679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:22:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.641347+0000 mon.a (mon.0) 3680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:22:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[30791]: audit 2021-03-10T17:22:57.642068+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:22:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.460363+0000 mon.a (mon.0) 3676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:22:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.461023+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:22:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.639873+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T17:22:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.640648+0000 mon.a (mon.0) 3679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:22:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.641347+0000 mon.a (mon.0) 3680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:22:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:57 gibba011 bash[31774]: audit 2021-03-10T17:22:57.642068+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:22:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.460363+0000 mon.a (mon.0) 3676 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:22:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.461023+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:22:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.639873+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [7, 2]}]: dispatch 2021-03-10T17:22:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.640648+0000 mon.a (mon.0) 3679 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:22:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.641347+0000 mon.a (mon.0) 3680 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:22:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:57 gibba032 bash[24004]: audit 2021-03-10T17:22:57.642068+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:22:59.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:22:58 gibba011 bash[30791]: cluster 2021-03-10T17:22:57.714538+0000 mgr.gibba032.whopeo (mgr.24625) 11076 : cluster [DBG] pgmap v6311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:59.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:22:58 gibba011 bash[31774]: cluster 2021-03-10T17:22:57.714538+0000 mgr.gibba032.whopeo (mgr.24625) 11076 : cluster [DBG] pgmap v6311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:22:59.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:22:58 gibba032 bash[24004]: cluster 2021-03-10T17:22:57.714538+0000 mgr.gibba032.whopeo (mgr.24625) 11076 : cluster [DBG] pgmap v6311: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:01.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:00 gibba011 bash[30791]: cluster 2021-03-10T17:22:59.716054+0000 mgr.gibba032.whopeo (mgr.24625) 11077 : cluster [DBG] pgmap v6312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:01.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:00 gibba011 bash[31774]: cluster 2021-03-10T17:22:59.716054+0000 mgr.gibba032.whopeo (mgr.24625) 11077 : cluster [DBG] pgmap v6312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:00 gibba032 bash[24004]: cluster 2021-03-10T17:22:59.716054+0000 mgr.gibba032.whopeo (mgr.24625) 11077 : cluster [DBG] pgmap v6312: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:02 gibba032 bash[24004]: cluster 2021-03-10T17:23:01.717860+0000 mgr.gibba032.whopeo (mgr.24625) 11078 : cluster [DBG] pgmap v6313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:02 gibba011 bash[30791]: cluster 2021-03-10T17:23:01.717860+0000 mgr.gibba032.whopeo (mgr.24625) 11078 : cluster [DBG] pgmap v6313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:02 gibba011 bash[31774]: cluster 2021-03-10T17:23:01.717860+0000 mgr.gibba032.whopeo (mgr.24625) 11078 : cluster [DBG] pgmap v6313: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:05.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:04 gibba011 bash[30791]: cluster 2021-03-10T17:23:03.719489+0000 mgr.gibba032.whopeo (mgr.24625) 11079 : cluster [DBG] pgmap v6314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:05.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:04 gibba011 bash[31774]: cluster 2021-03-10T17:23:03.719489+0000 mgr.gibba032.whopeo (mgr.24625) 11079 : cluster [DBG] pgmap v6314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:04 gibba032 bash[24004]: cluster 2021-03-10T17:23:03.719489+0000 mgr.gibba032.whopeo (mgr.24625) 11079 : cluster [DBG] pgmap v6314: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:07.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:06 gibba011 bash[30791]: cluster 2021-03-10T17:23:05.720845+0000 mgr.gibba032.whopeo (mgr.24625) 11080 : cluster [DBG] pgmap v6315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:07.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:06 gibba011 bash[31774]: cluster 2021-03-10T17:23:05.720845+0000 mgr.gibba032.whopeo (mgr.24625) 11080 : cluster [DBG] pgmap v6315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:06 gibba032 bash[24004]: cluster 2021-03-10T17:23:05.720845+0000 mgr.gibba032.whopeo (mgr.24625) 11080 : cluster [DBG] pgmap v6315: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:09.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:08 gibba011 bash[30791]: cluster 2021-03-10T17:23:07.722555+0000 mgr.gibba032.whopeo (mgr.24625) 11081 : cluster [DBG] pgmap v6316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:09.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:08 gibba011 bash[31774]: cluster 2021-03-10T17:23:07.722555+0000 mgr.gibba032.whopeo (mgr.24625) 11081 : cluster [DBG] pgmap v6316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:08 gibba032 bash[24004]: cluster 2021-03-10T17:23:07.722555+0000 mgr.gibba032.whopeo (mgr.24625) 11081 : cluster [DBG] pgmap v6316: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:11.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:10 gibba011 bash[30791]: cluster 2021-03-10T17:23:09.723934+0000 mgr.gibba032.whopeo (mgr.24625) 11082 : cluster [DBG] pgmap v6317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:11.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:10 gibba011 bash[31774]: cluster 2021-03-10T17:23:09.723934+0000 mgr.gibba032.whopeo (mgr.24625) 11082 : cluster [DBG] pgmap v6317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:10 gibba032 bash[24004]: cluster 2021-03-10T17:23:09.723934+0000 mgr.gibba032.whopeo (mgr.24625) 11082 : cluster [DBG] pgmap v6317: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:12.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:11 gibba011 bash[30791]: cephadm 2021-03-10T17:23:10.941888+0000 mgr.gibba032.whopeo (mgr.24625) 11083 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:12.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:11 gibba011 bash[31774]: cephadm 2021-03-10T17:23:10.941888+0000 mgr.gibba032.whopeo (mgr.24625) 11083 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:12.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:11 gibba032 bash[24004]: cephadm 2021-03-10T17:23:10.941888+0000 mgr.gibba032.whopeo (mgr.24625) 11083 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:13.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:12 gibba011 bash[30791]: cluster 2021-03-10T17:23:11.725725+0000 mgr.gibba032.whopeo (mgr.24625) 11084 : cluster [DBG] pgmap v6318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:13.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:12 gibba011 bash[31774]: cluster 2021-03-10T17:23:11.725725+0000 mgr.gibba032.whopeo (mgr.24625) 11084 : cluster [DBG] pgmap v6318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:13.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:12 gibba032 bash[24004]: cluster 2021-03-10T17:23:11.725725+0000 mgr.gibba032.whopeo (mgr.24625) 11084 : cluster [DBG] pgmap v6318: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:14 gibba032 bash[24004]: cluster 2021-03-10T17:23:13.727348+0000 mgr.gibba032.whopeo (mgr.24625) 11085 : cluster [DBG] pgmap v6319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:14 gibba011 bash[30791]: cluster 2021-03-10T17:23:13.727348+0000 mgr.gibba032.whopeo (mgr.24625) 11085 : cluster [DBG] pgmap v6319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:14 gibba011 bash[31774]: cluster 2021-03-10T17:23:13.727348+0000 mgr.gibba032.whopeo (mgr.24625) 11085 : cluster [DBG] pgmap v6319: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:16 gibba032 bash[24004]: cluster 2021-03-10T17:23:15.728863+0000 mgr.gibba032.whopeo (mgr.24625) 11086 : cluster [DBG] pgmap v6320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:16 gibba011 bash[30791]: cluster 2021-03-10T17:23:15.728863+0000 mgr.gibba032.whopeo (mgr.24625) 11086 : cluster [DBG] pgmap v6320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:17.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:16 gibba011 bash[31774]: cluster 2021-03-10T17:23:15.728863+0000 mgr.gibba032.whopeo (mgr.24625) 11086 : cluster [DBG] pgmap v6320: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:18 gibba032 bash[24004]: cluster 2021-03-10T17:23:17.730602+0000 mgr.gibba032.whopeo (mgr.24625) 11087 : cluster [DBG] pgmap v6321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:18 gibba011 bash[30791]: cluster 2021-03-10T17:23:17.730602+0000 mgr.gibba032.whopeo (mgr.24625) 11087 : cluster [DBG] pgmap v6321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:18 gibba011 bash[31774]: cluster 2021-03-10T17:23:17.730602+0000 mgr.gibba032.whopeo (mgr.24625) 11087 : cluster [DBG] pgmap v6321: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:20 gibba032 bash[24004]: cluster 2021-03-10T17:23:19.731990+0000 mgr.gibba032.whopeo (mgr.24625) 11088 : cluster [DBG] pgmap v6322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:20 gibba011 bash[30791]: cluster 2021-03-10T17:23:19.731990+0000 mgr.gibba032.whopeo (mgr.24625) 11088 : cluster [DBG] pgmap v6322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:20 gibba011 bash[31774]: cluster 2021-03-10T17:23:19.731990+0000 mgr.gibba032.whopeo (mgr.24625) 11088 : cluster [DBG] pgmap v6322: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:22.176 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:23:22.433 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:23:22.434 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 110s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:23:22.434 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 46s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:23:22.434 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:23:22.435 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:23:22.435 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:23:22.435 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:23:22.436 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:23:22.436 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:23:22.437 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 110s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:23:22.437 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 46s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:23:22.438 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:23:22.438 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:23:22.438 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:23:22.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 110s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:23:22.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:23:22.440 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:23:22.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:23:22.441 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 46s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:23:22.442 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 46s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:23:22.442 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 110s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:23:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:22 gibba032 bash[24004]: cluster 2021-03-10T17:23:21.733840+0000 mgr.gibba032.whopeo (mgr.24625) 11089 : cluster [DBG] pgmap v6323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:22 gibba011 bash[30791]: cluster 2021-03-10T17:23:21.733840+0000 mgr.gibba032.whopeo (mgr.24625) 11089 : cluster [DBG] pgmap v6323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:22.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:22 gibba011 bash[31774]: cluster 2021-03-10T17:23:21.733840+0000 mgr.gibba032.whopeo (mgr.24625) 11089 : cluster [DBG] pgmap v6323: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:22.685 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:23:22.686 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:23:22.686 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:23:22.686 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:22.687 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:23:22.687 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:23:22.687 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:22.688 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:23:22.688 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:23:22.688 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:22.689 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:23:22.689 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:23:22.689 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:23:22.690 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:23:22.690 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:23:23.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:23 gibba032 bash[24004]: audit 2021-03-10T17:23:22.165269+0000 mgr.gibba032.whopeo (mgr.24625) 11090 : audit [DBG] from='client.51489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:23 gibba032 bash[24004]: audit 2021-03-10T17:23:22.391522+0000 mgr.gibba032.whopeo (mgr.24625) 11091 : audit [DBG] from='client.51495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:23 gibba032 bash[24004]: audit 2021-03-10T17:23:22.685366+0000 mon.c (mon.1) 213 : audit [DBG] from='client.? 172.21.2.111:0/3708099820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[30791]: audit 2021-03-10T17:23:22.165269+0000 mgr.gibba032.whopeo (mgr.24625) 11090 : audit [DBG] from='client.51489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[30791]: audit 2021-03-10T17:23:22.391522+0000 mgr.gibba032.whopeo (mgr.24625) 11091 : audit [DBG] from='client.51495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[30791]: audit 2021-03-10T17:23:22.685366+0000 mon.c (mon.1) 213 : audit [DBG] from='client.? 172.21.2.111:0/3708099820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[31774]: audit 2021-03-10T17:23:22.165269+0000 mgr.gibba032.whopeo (mgr.24625) 11090 : audit [DBG] from='client.51489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[31774]: audit 2021-03-10T17:23:22.391522+0000 mgr.gibba032.whopeo (mgr.24625) 11091 : audit [DBG] from='client.51495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:23.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:23 gibba011 bash[31774]: audit 2021-03-10T17:23:22.685366+0000 mon.c (mon.1) 213 : audit [DBG] from='client.? 172.21.2.111:0/3708099820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:24.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:24 gibba032 bash[24004]: cluster 2021-03-10T17:23:23.735430+0000 mgr.gibba032.whopeo (mgr.24625) 11092 : cluster [DBG] pgmap v6324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:24.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:24 gibba011 bash[30791]: cluster 2021-03-10T17:23:23.735430+0000 mgr.gibba032.whopeo (mgr.24625) 11092 : cluster [DBG] pgmap v6324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:24.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:24 gibba011 bash[31774]: cluster 2021-03-10T17:23:23.735430+0000 mgr.gibba032.whopeo (mgr.24625) 11092 : cluster [DBG] pgmap v6324: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:27.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:26 gibba011 bash[30791]: cluster 2021-03-10T17:23:25.736946+0000 mgr.gibba032.whopeo (mgr.24625) 11093 : cluster [DBG] pgmap v6325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:27.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:26 gibba011 bash[30791]: cephadm 2021-03-10T17:23:25.956813+0000 mgr.gibba032.whopeo (mgr.24625) 11094 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:27.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:26 gibba011 bash[31774]: cluster 2021-03-10T17:23:25.736946+0000 mgr.gibba032.whopeo (mgr.24625) 11093 : cluster [DBG] pgmap v6325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:27.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:26 gibba011 bash[31774]: cephadm 2021-03-10T17:23:25.956813+0000 mgr.gibba032.whopeo (mgr.24625) 11094 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:27.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:26 gibba032 bash[24004]: cluster 2021-03-10T17:23:25.736946+0000 mgr.gibba032.whopeo (mgr.24625) 11093 : cluster [DBG] pgmap v6325: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:26 gibba032 bash[24004]: cephadm 2021-03-10T17:23:25.956813+0000 mgr.gibba032.whopeo (mgr.24625) 11094 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:29.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:28 gibba011 bash[30791]: cluster 2021-03-10T17:23:27.738585+0000 mgr.gibba032.whopeo (mgr.24625) 11095 : cluster [DBG] pgmap v6326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:29.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:28 gibba011 bash[31774]: cluster 2021-03-10T17:23:27.738585+0000 mgr.gibba032.whopeo (mgr.24625) 11095 : cluster [DBG] pgmap v6326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:29.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:28 gibba032 bash[24004]: cluster 2021-03-10T17:23:27.738585+0000 mgr.gibba032.whopeo (mgr.24625) 11095 : cluster [DBG] pgmap v6326: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:31.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:30 gibba011 bash[30791]: cluster 2021-03-10T17:23:29.740060+0000 mgr.gibba032.whopeo (mgr.24625) 11096 : cluster [DBG] pgmap v6327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:31.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:30 gibba011 bash[31774]: cluster 2021-03-10T17:23:29.740060+0000 mgr.gibba032.whopeo (mgr.24625) 11096 : cluster [DBG] pgmap v6327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:30 gibba032 bash[24004]: cluster 2021-03-10T17:23:29.740060+0000 mgr.gibba032.whopeo (mgr.24625) 11096 : cluster [DBG] pgmap v6327: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:32 gibba032 bash[24004]: cluster 2021-03-10T17:23:31.741853+0000 mgr.gibba032.whopeo (mgr.24625) 11097 : cluster [DBG] pgmap v6328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:32 gibba011 bash[30791]: cluster 2021-03-10T17:23:31.741853+0000 mgr.gibba032.whopeo (mgr.24625) 11097 : cluster [DBG] pgmap v6328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:32 gibba011 bash[31774]: cluster 2021-03-10T17:23:31.741853+0000 mgr.gibba032.whopeo (mgr.24625) 11097 : cluster [DBG] pgmap v6328: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:35.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:34 gibba011 bash[30791]: cluster 2021-03-10T17:23:33.743303+0000 mgr.gibba032.whopeo (mgr.24625) 11098 : cluster [DBG] pgmap v6329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:35.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:34 gibba011 bash[31774]: cluster 2021-03-10T17:23:33.743303+0000 mgr.gibba032.whopeo (mgr.24625) 11098 : cluster [DBG] pgmap v6329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:35.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:34 gibba032 bash[24004]: cluster 2021-03-10T17:23:33.743303+0000 mgr.gibba032.whopeo (mgr.24625) 11098 : cluster [DBG] pgmap v6329: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:37.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:36 gibba011 bash[30791]: cluster 2021-03-10T17:23:35.744574+0000 mgr.gibba032.whopeo (mgr.24625) 11099 : cluster [DBG] pgmap v6330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:37.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:36 gibba011 bash[31774]: cluster 2021-03-10T17:23:35.744574+0000 mgr.gibba032.whopeo (mgr.24625) 11099 : cluster [DBG] pgmap v6330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:36 gibba032 bash[24004]: cluster 2021-03-10T17:23:35.744574+0000 mgr.gibba032.whopeo (mgr.24625) 11099 : cluster [DBG] pgmap v6330: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:39.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:38 gibba011 bash[30791]: cluster 2021-03-10T17:23:37.746077+0000 mgr.gibba032.whopeo (mgr.24625) 11100 : cluster [DBG] pgmap v6331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:39.079 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:38 gibba011 bash[31774]: cluster 2021-03-10T17:23:37.746077+0000 mgr.gibba032.whopeo (mgr.24625) 11100 : cluster [DBG] pgmap v6331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:38 gibba032 bash[24004]: cluster 2021-03-10T17:23:37.746077+0000 mgr.gibba032.whopeo (mgr.24625) 11100 : cluster [DBG] pgmap v6331: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:41.221 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:40 gibba032 bash[24004]: cluster 2021-03-10T17:23:39.747333+0000 mgr.gibba032.whopeo (mgr.24625) 11101 : cluster [DBG] pgmap v6332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:40 gibba011 bash[30791]: cluster 2021-03-10T17:23:39.747333+0000 mgr.gibba032.whopeo (mgr.24625) 11101 : cluster [DBG] pgmap v6332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:40 gibba011 bash[31774]: cluster 2021-03-10T17:23:39.747333+0000 mgr.gibba032.whopeo (mgr.24625) 11101 : cluster [DBG] pgmap v6332: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: cephadm 2021-03-10T17:23:40.973409+0000 mgr.gibba032.whopeo (mgr.24625) 11102 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:23:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: cephadm 2021-03-10T17:23:40.974963+0000 mgr.gibba032.whopeo (mgr.24625) 11103 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:23:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.206782+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:23:42.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.210563+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:23:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.212035+0000 mon.a (mon.0) 3684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.213489+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.214787+0000 mon.a (mon.0) 3686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.218112+0000 mon.a (mon.0) 3687 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:41 gibba032 bash[24004]: audit 2021-03-10T17:23:41.220157+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: cephadm 2021-03-10T17:23:40.973409+0000 mgr.gibba032.whopeo (mgr.24625) 11102 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:23:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: cephadm 2021-03-10T17:23:40.974963+0000 mgr.gibba032.whopeo (mgr.24625) 11103 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:23:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.206782+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:23:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.210563+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:23:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.212035+0000 mon.a (mon.0) 3684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.213489+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.214787+0000 mon.a (mon.0) 3686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.218112+0000 mon.a (mon.0) 3687 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[30791]: audit 2021-03-10T17:23:41.220157+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: cephadm 2021-03-10T17:23:40.973409+0000 mgr.gibba032.whopeo (mgr.24625) 11102 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:23:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: cephadm 2021-03-10T17:23:40.974963+0000 mgr.gibba032.whopeo (mgr.24625) 11103 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:23:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.206782+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:23:42.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.210563+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:23:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.212035+0000 mon.a (mon.0) 3684 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.213489+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.214787+0000 mon.a (mon.0) 3686 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.218112+0000 mon.a (mon.0) 3687 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:42.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:41 gibba011 bash[31774]: audit 2021-03-10T17:23:41.220157+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:43.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.207900+0000 mgr.gibba032.whopeo (mgr.24625) 11104 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.209713+0000 mgr.gibba032.whopeo (mgr.24625) 11105 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.211103+0000 mgr.gibba032.whopeo (mgr.24625) 11106 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.211155+0000 mgr.gibba032.whopeo (mgr.24625) 11107 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.212576+0000 mgr.gibba032.whopeo (mgr.24625) 11108 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:23:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.212626+0000 mgr.gibba032.whopeo (mgr.24625) 11109 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:23:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.214050+0000 mgr.gibba032.whopeo (mgr.24625) 11110 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:23:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.215561+0000 mgr.gibba032.whopeo (mgr.24625) 11111 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:23:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.215616+0000 mgr.gibba032.whopeo (mgr.24625) 11112 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:23:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.217438+0000 mgr.gibba032.whopeo (mgr.24625) 11113 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:23:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.217905+0000 mgr.gibba032.whopeo (mgr.24625) 11114 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:23:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.219525+0000 mgr.gibba032.whopeo (mgr.24625) 11115 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:23:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.219589+0000 mgr.gibba032.whopeo (mgr.24625) 11116 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:23:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cephadm 2021-03-10T17:23:41.219758+0000 mgr.gibba032.whopeo (mgr.24625) 11117 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:42 gibba032 bash[24004]: cluster 2021-03-10T17:23:41.748801+0000 mgr.gibba032.whopeo (mgr.24625) 11118 : cluster [DBG] pgmap v6333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:43.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.207900+0000 mgr.gibba032.whopeo (mgr.24625) 11104 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:23:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.209713+0000 mgr.gibba032.whopeo (mgr.24625) 11105 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:23:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.211103+0000 mgr.gibba032.whopeo (mgr.24625) 11106 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:23:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.211155+0000 mgr.gibba032.whopeo (mgr.24625) 11107 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:23:43.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.212576+0000 mgr.gibba032.whopeo (mgr.24625) 11108 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:23:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.212626+0000 mgr.gibba032.whopeo (mgr.24625) 11109 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:23:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.214050+0000 mgr.gibba032.whopeo (mgr.24625) 11110 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:23:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.215561+0000 mgr.gibba032.whopeo (mgr.24625) 11111 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:23:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.215616+0000 mgr.gibba032.whopeo (mgr.24625) 11112 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:23:43.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.217438+0000 mgr.gibba032.whopeo (mgr.24625) 11113 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:23:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.217905+0000 mgr.gibba032.whopeo (mgr.24625) 11114 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:23:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.219525+0000 mgr.gibba032.whopeo (mgr.24625) 11115 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:23:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.219589+0000 mgr.gibba032.whopeo (mgr.24625) 11116 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:23:43.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cephadm 2021-03-10T17:23:41.219758+0000 mgr.gibba032.whopeo (mgr.24625) 11117 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:43.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[30791]: cluster 2021-03-10T17:23:41.748801+0000 mgr.gibba032.whopeo (mgr.24625) 11118 : cluster [DBG] pgmap v6333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:43.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.207900+0000 mgr.gibba032.whopeo (mgr.24625) 11104 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:23:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.209713+0000 mgr.gibba032.whopeo (mgr.24625) 11105 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:23:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.211103+0000 mgr.gibba032.whopeo (mgr.24625) 11106 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:23:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.211155+0000 mgr.gibba032.whopeo (mgr.24625) 11107 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:23:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.212576+0000 mgr.gibba032.whopeo (mgr.24625) 11108 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:23:43.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.212626+0000 mgr.gibba032.whopeo (mgr.24625) 11109 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:23:43.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.214050+0000 mgr.gibba032.whopeo (mgr.24625) 11110 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:23:43.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.215561+0000 mgr.gibba032.whopeo (mgr.24625) 11111 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:23:43.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.215616+0000 mgr.gibba032.whopeo (mgr.24625) 11112 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:23:43.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.217438+0000 mgr.gibba032.whopeo (mgr.24625) 11113 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:23:43.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.217905+0000 mgr.gibba032.whopeo (mgr.24625) 11114 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:23:43.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.219525+0000 mgr.gibba032.whopeo (mgr.24625) 11115 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:23:43.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.219589+0000 mgr.gibba032.whopeo (mgr.24625) 11116 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:23:43.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cephadm 2021-03-10T17:23:41.219758+0000 mgr.gibba032.whopeo (mgr.24625) 11117 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:43.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:42 gibba011 bash[31774]: cluster 2021-03-10T17:23:41.748801+0000 mgr.gibba032.whopeo (mgr.24625) 11118 : cluster [DBG] pgmap v6333: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:44 gibba032 bash[24004]: cluster 2021-03-10T17:23:43.750252+0000 mgr.gibba032.whopeo (mgr.24625) 11119 : cluster [DBG] pgmap v6334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:44 gibba011 bash[30791]: cluster 2021-03-10T17:23:43.750252+0000 mgr.gibba032.whopeo (mgr.24625) 11119 : cluster [DBG] pgmap v6334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:44 gibba011 bash[31774]: cluster 2021-03-10T17:23:43.750252+0000 mgr.gibba032.whopeo (mgr.24625) 11119 : cluster [DBG] pgmap v6334: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:46 gibba032 bash[24004]: cluster 2021-03-10T17:23:45.751594+0000 mgr.gibba032.whopeo (mgr.24625) 11120 : cluster [DBG] pgmap v6335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:46 gibba011 bash[30791]: cluster 2021-03-10T17:23:45.751594+0000 mgr.gibba032.whopeo (mgr.24625) 11120 : cluster [DBG] pgmap v6335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:46 gibba011 bash[31774]: cluster 2021-03-10T17:23:45.751594+0000 mgr.gibba032.whopeo (mgr.24625) 11120 : cluster [DBG] pgmap v6335: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:48 gibba032 bash[24004]: cluster 2021-03-10T17:23:47.753393+0000 mgr.gibba032.whopeo (mgr.24625) 11121 : cluster [DBG] pgmap v6336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:48 gibba011 bash[30791]: cluster 2021-03-10T17:23:47.753393+0000 mgr.gibba032.whopeo (mgr.24625) 11121 : cluster [DBG] pgmap v6336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:49.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:48 gibba011 bash[31774]: cluster 2021-03-10T17:23:47.753393+0000 mgr.gibba032.whopeo (mgr.24625) 11121 : cluster [DBG] pgmap v6336: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:50 gibba032 bash[24004]: cluster 2021-03-10T17:23:49.754887+0000 mgr.gibba032.whopeo (mgr.24625) 11122 : cluster [DBG] pgmap v6337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:50 gibba011 bash[30791]: cluster 2021-03-10T17:23:49.754887+0000 mgr.gibba032.whopeo (mgr.24625) 11122 : cluster [DBG] pgmap v6337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:50 gibba011 bash[31774]: cluster 2021-03-10T17:23:49.754887+0000 mgr.gibba032.whopeo (mgr.24625) 11122 : cluster [DBG] pgmap v6337: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:52 gibba032 bash[24004]: cluster 2021-03-10T17:23:51.756644+0000 mgr.gibba032.whopeo (mgr.24625) 11123 : cluster [DBG] pgmap v6338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:52 gibba011 bash[30791]: cluster 2021-03-10T17:23:51.756644+0000 mgr.gibba032.whopeo (mgr.24625) 11123 : cluster [DBG] pgmap v6338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:52.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:52 gibba011 bash[31774]: cluster 2021-03-10T17:23:51.756644+0000 mgr.gibba032.whopeo (mgr.24625) 11123 : cluster [DBG] pgmap v6338: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:52.930 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:23:53.159 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:23:53.160 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:23:53.160 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 77s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:23:53.160 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:23:53.160 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:23:53.160 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:23:53.161 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:23:53.161 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:23:53.161 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:23:53.161 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:23:53.162 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 77s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:23:53.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:23:53.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:23:53.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:23:53.162 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:23:53.163 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:23:53.163 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:23:53.163 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:23:53.163 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 77s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:23:53.164 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 77s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:23:53.164 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:23:53.415 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:23:53.415 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:23:53.416 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:23:53.416 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:53.416 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:23:53.416 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:23:53.417 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:53.417 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:23:53.417 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:23:53.417 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:23:53.418 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:23:53.418 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:23:53.418 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:23:53.418 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:23:53.419 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:23:53.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:53 gibba032 bash[24004]: audit 2021-03-10T17:23:52.919218+0000 mgr.gibba032.whopeo (mgr.24625) 11124 : audit [DBG] from='client.51507 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:53 gibba011 bash[30791]: audit 2021-03-10T17:23:52.919218+0000 mgr.gibba032.whopeo (mgr.24625) 11124 : audit [DBG] from='client.51507 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:53.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:53 gibba011 bash[31774]: audit 2021-03-10T17:23:52.919218+0000 mgr.gibba032.whopeo (mgr.24625) 11124 : audit [DBG] from='client.51507 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:54.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:54 gibba032 bash[24004]: audit 2021-03-10T17:23:53.145202+0000 mgr.gibba032.whopeo (mgr.24625) 11125 : audit [DBG] from='client.61495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:54.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:54 gibba032 bash[24004]: audit 2021-03-10T17:23:53.414984+0000 mon.c (mon.1) 214 : audit [DBG] from='client.? 172.21.2.111:0/1986714081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:54.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:54 gibba032 bash[24004]: cluster 2021-03-10T17:23:53.758240+0000 mgr.gibba032.whopeo (mgr.24625) 11126 : cluster [DBG] pgmap v6339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[30791]: audit 2021-03-10T17:23:53.145202+0000 mgr.gibba032.whopeo (mgr.24625) 11125 : audit [DBG] from='client.61495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[30791]: audit 2021-03-10T17:23:53.414984+0000 mon.c (mon.1) 214 : audit [DBG] from='client.? 172.21.2.111:0/1986714081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[30791]: cluster 2021-03-10T17:23:53.758240+0000 mgr.gibba032.whopeo (mgr.24625) 11126 : cluster [DBG] pgmap v6339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[31774]: audit 2021-03-10T17:23:53.145202+0000 mgr.gibba032.whopeo (mgr.24625) 11125 : audit [DBG] from='client.61495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:23:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[31774]: audit 2021-03-10T17:23:53.414984+0000 mon.c (mon.1) 214 : audit [DBG] from='client.? 172.21.2.111:0/1986714081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:23:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:54 gibba011 bash[31774]: cluster 2021-03-10T17:23:53.758240+0000 mgr.gibba032.whopeo (mgr.24625) 11126 : cluster [DBG] pgmap v6339: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:57.079 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:56 gibba011 bash[30791]: cluster 2021-03-10T17:23:55.759590+0000 mgr.gibba032.whopeo (mgr.24625) 11127 : cluster [DBG] pgmap v6340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:57.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:56 gibba011 bash[31774]: cluster 2021-03-10T17:23:55.759590+0000 mgr.gibba032.whopeo (mgr.24625) 11127 : cluster [DBG] pgmap v6340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:56 gibba032 bash[24004]: cluster 2021-03-10T17:23:55.759590+0000 mgr.gibba032.whopeo (mgr.24625) 11127 : cluster [DBG] pgmap v6340: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: cephadm 2021-03-10T17:23:56.235554+0000 mgr.gibba032.whopeo (mgr.24625) 11128 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.467957+0000 mon.a (mon.0) 3689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:23:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.468553+0000 mon.a (mon.0) 3690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:23:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.649890+0000 mon.a (mon.0) 3691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:23:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.650669+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.651373+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:57 gibba032 bash[24004]: audit 2021-03-10T17:23:57.652088+0000 mon.a (mon.0) 3694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:23:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: cephadm 2021-03-10T17:23:56.235554+0000 mgr.gibba032.whopeo (mgr.24625) 11128 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.467957+0000 mon.a (mon.0) 3689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:23:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.468553+0000 mon.a (mon.0) 3690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:23:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.649890+0000 mon.a (mon.0) 3691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:23:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.650669+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.651373+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[30791]: audit 2021-03-10T17:23:57.652088+0000 mon.a (mon.0) 3694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:23:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: cephadm 2021-03-10T17:23:56.235554+0000 mgr.gibba032.whopeo (mgr.24625) 11128 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:23:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.467957+0000 mon.a (mon.0) 3689 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:23:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.468553+0000 mon.a (mon.0) 3690 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:23:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.649890+0000 mon.a (mon.0) 3691 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:23:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.650669+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.651373+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:23:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:57 gibba011 bash[31774]: audit 2021-03-10T17:23:57.652088+0000 mon.a (mon.0) 3694 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:23:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:23:58 gibba032 bash[24004]: cluster 2021-03-10T17:23:57.761326+0000 mgr.gibba032.whopeo (mgr.24625) 11129 : cluster [DBG] pgmap v6341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:23:58 gibba011 bash[30791]: cluster 2021-03-10T17:23:57.761326+0000 mgr.gibba032.whopeo (mgr.24625) 11129 : cluster [DBG] pgmap v6341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:23:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:23:58 gibba011 bash[31774]: cluster 2021-03-10T17:23:57.761326+0000 mgr.gibba032.whopeo (mgr.24625) 11129 : cluster [DBG] pgmap v6341: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:01.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:00 gibba032 bash[24004]: cluster 2021-03-10T17:23:59.762843+0000 mgr.gibba032.whopeo (mgr.24625) 11130 : cluster [DBG] pgmap v6342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:00 gibba011 bash[30791]: cluster 2021-03-10T17:23:59.762843+0000 mgr.gibba032.whopeo (mgr.24625) 11130 : cluster [DBG] pgmap v6342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:00 gibba011 bash[31774]: cluster 2021-03-10T17:23:59.762843+0000 mgr.gibba032.whopeo (mgr.24625) 11130 : cluster [DBG] pgmap v6342: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:02 gibba032 bash[24004]: cluster 2021-03-10T17:24:01.764543+0000 mgr.gibba032.whopeo (mgr.24625) 11131 : cluster [DBG] pgmap v6343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:02 gibba011 bash[30791]: cluster 2021-03-10T17:24:01.764543+0000 mgr.gibba032.whopeo (mgr.24625) 11131 : cluster [DBG] pgmap v6343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:02.579 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:02 gibba011 bash[31774]: cluster 2021-03-10T17:24:01.764543+0000 mgr.gibba032.whopeo (mgr.24625) 11131 : cluster [DBG] pgmap v6343: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:05.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:04 gibba032 bash[24004]: cluster 2021-03-10T17:24:03.766087+0000 mgr.gibba032.whopeo (mgr.24625) 11132 : cluster [DBG] pgmap v6344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:04 gibba011 bash[30791]: cluster 2021-03-10T17:24:03.766087+0000 mgr.gibba032.whopeo (mgr.24625) 11132 : cluster [DBG] pgmap v6344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:05.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:04 gibba011 bash[31774]: cluster 2021-03-10T17:24:03.766087+0000 mgr.gibba032.whopeo (mgr.24625) 11132 : cluster [DBG] pgmap v6344: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:07.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:06 gibba032 bash[24004]: cluster 2021-03-10T17:24:05.767460+0000 mgr.gibba032.whopeo (mgr.24625) 11133 : cluster [DBG] pgmap v6345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:06 gibba011 bash[30791]: cluster 2021-03-10T17:24:05.767460+0000 mgr.gibba032.whopeo (mgr.24625) 11133 : cluster [DBG] pgmap v6345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:06 gibba011 bash[31774]: cluster 2021-03-10T17:24:05.767460+0000 mgr.gibba032.whopeo (mgr.24625) 11133 : cluster [DBG] pgmap v6345: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:08 gibba032 bash[24004]: cluster 2021-03-10T17:24:07.769192+0000 mgr.gibba032.whopeo (mgr.24625) 11134 : cluster [DBG] pgmap v6346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:08 gibba011 bash[30791]: cluster 2021-03-10T17:24:07.769192+0000 mgr.gibba032.whopeo (mgr.24625) 11134 : cluster [DBG] pgmap v6346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:08 gibba011 bash[31774]: cluster 2021-03-10T17:24:07.769192+0000 mgr.gibba032.whopeo (mgr.24625) 11134 : cluster [DBG] pgmap v6346: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:11.254 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:10 gibba032 bash[24004]: cluster 2021-03-10T17:24:09.770571+0000 mgr.gibba032.whopeo (mgr.24625) 11135 : cluster [DBG] pgmap v6347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:10 gibba011 bash[30791]: cluster 2021-03-10T17:24:09.770571+0000 mgr.gibba032.whopeo (mgr.24625) 11135 : cluster [DBG] pgmap v6347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:10 gibba011 bash[31774]: cluster 2021-03-10T17:24:09.770571+0000 mgr.gibba032.whopeo (mgr.24625) 11135 : cluster [DBG] pgmap v6347: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:12 gibba032 bash[24004]: cephadm 2021-03-10T17:24:11.251596+0000 mgr.gibba032.whopeo (mgr.24625) 11136 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:12 gibba032 bash[24004]: cluster 2021-03-10T17:24:11.772350+0000 mgr.gibba032.whopeo (mgr.24625) 11137 : cluster [DBG] pgmap v6348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:12 gibba011 bash[30791]: cephadm 2021-03-10T17:24:11.251596+0000 mgr.gibba032.whopeo (mgr.24625) 11136 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:12 gibba011 bash[30791]: cluster 2021-03-10T17:24:11.772350+0000 mgr.gibba032.whopeo (mgr.24625) 11137 : cluster [DBG] pgmap v6348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:12 gibba011 bash[31774]: cephadm 2021-03-10T17:24:11.251596+0000 mgr.gibba032.whopeo (mgr.24625) 11136 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:12 gibba011 bash[31774]: cluster 2021-03-10T17:24:11.772350+0000 mgr.gibba032.whopeo (mgr.24625) 11137 : cluster [DBG] pgmap v6348: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:14 gibba032 bash[24004]: cluster 2021-03-10T17:24:13.773772+0000 mgr.gibba032.whopeo (mgr.24625) 11138 : cluster [DBG] pgmap v6349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:14 gibba011 bash[30791]: cluster 2021-03-10T17:24:13.773772+0000 mgr.gibba032.whopeo (mgr.24625) 11138 : cluster [DBG] pgmap v6349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:14 gibba011 bash[31774]: cluster 2021-03-10T17:24:13.773772+0000 mgr.gibba032.whopeo (mgr.24625) 11138 : cluster [DBG] pgmap v6349: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:17.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:16 gibba032 bash[24004]: cluster 2021-03-10T17:24:15.775224+0000 mgr.gibba032.whopeo (mgr.24625) 11139 : cluster [DBG] pgmap v6350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:16 gibba011 bash[30791]: cluster 2021-03-10T17:24:15.775224+0000 mgr.gibba032.whopeo (mgr.24625) 11139 : cluster [DBG] pgmap v6350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:16 gibba011 bash[31774]: cluster 2021-03-10T17:24:15.775224+0000 mgr.gibba032.whopeo (mgr.24625) 11139 : cluster [DBG] pgmap v6350: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:18 gibba032 bash[24004]: cluster 2021-03-10T17:24:17.777054+0000 mgr.gibba032.whopeo (mgr.24625) 11140 : cluster [DBG] pgmap v6351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:18 gibba011 bash[30791]: cluster 2021-03-10T17:24:17.777054+0000 mgr.gibba032.whopeo (mgr.24625) 11140 : cluster [DBG] pgmap v6351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:18 gibba011 bash[31774]: cluster 2021-03-10T17:24:17.777054+0000 mgr.gibba032.whopeo (mgr.24625) 11140 : cluster [DBG] pgmap v6351: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:20 gibba032 bash[24004]: cluster 2021-03-10T17:24:19.778569+0000 mgr.gibba032.whopeo (mgr.24625) 11141 : cluster [DBG] pgmap v6352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:20 gibba011 bash[30791]: cluster 2021-03-10T17:24:19.778569+0000 mgr.gibba032.whopeo (mgr.24625) 11141 : cluster [DBG] pgmap v6352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:21.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:20 gibba011 bash[31774]: cluster 2021-03-10T17:24:19.778569+0000 mgr.gibba032.whopeo (mgr.24625) 11141 : cluster [DBG] pgmap v6352: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:22 gibba032 bash[24004]: cluster 2021-03-10T17:24:21.780359+0000 mgr.gibba032.whopeo (mgr.24625) 11142 : cluster [DBG] pgmap v6353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:22 gibba011 bash[30791]: cluster 2021-03-10T17:24:21.780359+0000 mgr.gibba032.whopeo (mgr.24625) 11142 : cluster [DBG] pgmap v6353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:22 gibba011 bash[31774]: cluster 2021-03-10T17:24:21.780359+0000 mgr.gibba032.whopeo (mgr.24625) 11142 : cluster [DBG] pgmap v6353: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:23.671 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:24:23.898 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:24:23.898 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:24:23.899 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 108s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:24:23.899 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:24:23.900 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:24:23.900 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:24:23.901 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:24:23.901 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:24:23.901 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:24:23.902 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:24:23.902 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 108s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:24:23.902 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:24:23.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:24:23.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:24:23.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:24:23.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:24:23.903 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:24:23.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:24:23.904 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 108s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:24:23.904 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 108s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:24:23.904 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:24:24.151 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:24:24.151 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:24:24.152 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:24:24.152 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:24.152 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:24:24.152 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:24:24.152 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:24.153 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:24:24.153 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:24:24.153 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:24.153 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:24:24.154 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:24:24.154 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:24:24.154 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:24:24.154 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:24:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:24 gibba032 bash[24004]: audit 2021-03-10T17:24:23.651271+0000 mgr.gibba032.whopeo (mgr.24625) 11143 : audit [DBG] from='client.51525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:24 gibba032 bash[24004]: cluster 2021-03-10T17:24:23.781840+0000 mgr.gibba032.whopeo (mgr.24625) 11144 : cluster [DBG] pgmap v6354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:24 gibba032 bash[24004]: audit 2021-03-10T17:24:23.886274+0000 mgr.gibba032.whopeo (mgr.24625) 11145 : audit [DBG] from='client.61513 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:24 gibba032 bash[24004]: audit 2021-03-10T17:24:24.150926+0000 mon.c (mon.1) 215 : audit [DBG] from='client.? 172.21.2.111:0/4017355235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[30791]: audit 2021-03-10T17:24:23.651271+0000 mgr.gibba032.whopeo (mgr.24625) 11143 : audit [DBG] from='client.51525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[30791]: cluster 2021-03-10T17:24:23.781840+0000 mgr.gibba032.whopeo (mgr.24625) 11144 : cluster [DBG] pgmap v6354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[30791]: audit 2021-03-10T17:24:23.886274+0000 mgr.gibba032.whopeo (mgr.24625) 11145 : audit [DBG] from='client.61513 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[30791]: audit 2021-03-10T17:24:24.150926+0000 mon.c (mon.1) 215 : audit [DBG] from='client.? 172.21.2.111:0/4017355235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:25.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[31774]: audit 2021-03-10T17:24:23.651271+0000 mgr.gibba032.whopeo (mgr.24625) 11143 : audit [DBG] from='client.51525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[31774]: cluster 2021-03-10T17:24:23.781840+0000 mgr.gibba032.whopeo (mgr.24625) 11144 : cluster [DBG] pgmap v6354: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[31774]: audit 2021-03-10T17:24:23.886274+0000 mgr.gibba032.whopeo (mgr.24625) 11145 : audit [DBG] from='client.61513 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:25.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:24 gibba011 bash[31774]: audit 2021-03-10T17:24:24.150926+0000 mon.c (mon.1) 215 : audit [DBG] from='client.? 172.21.2.111:0/4017355235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:26 gibba032 bash[24004]: cluster 2021-03-10T17:24:25.783175+0000 mgr.gibba032.whopeo (mgr.24625) 11146 : cluster [DBG] pgmap v6355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:26 gibba011 bash[30791]: cluster 2021-03-10T17:24:25.783175+0000 mgr.gibba032.whopeo (mgr.24625) 11146 : cluster [DBG] pgmap v6355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:26 gibba011 bash[31774]: cluster 2021-03-10T17:24:25.783175+0000 mgr.gibba032.whopeo (mgr.24625) 11146 : cluster [DBG] pgmap v6355: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:28.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:27 gibba032 bash[24004]: cephadm 2021-03-10T17:24:26.267542+0000 mgr.gibba032.whopeo (mgr.24625) 11147 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:27 gibba011 bash[30791]: cephadm 2021-03-10T17:24:26.267542+0000 mgr.gibba032.whopeo (mgr.24625) 11147 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:28.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:27 gibba011 bash[31774]: cephadm 2021-03-10T17:24:26.267542+0000 mgr.gibba032.whopeo (mgr.24625) 11147 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:28 gibba032 bash[24004]: cluster 2021-03-10T17:24:27.784990+0000 mgr.gibba032.whopeo (mgr.24625) 11148 : cluster [DBG] pgmap v6356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:28 gibba011 bash[30791]: cluster 2021-03-10T17:24:27.784990+0000 mgr.gibba032.whopeo (mgr.24625) 11148 : cluster [DBG] pgmap v6356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:28 gibba011 bash[31774]: cluster 2021-03-10T17:24:27.784990+0000 mgr.gibba032.whopeo (mgr.24625) 11148 : cluster [DBG] pgmap v6356: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:31.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:30 gibba032 bash[24004]: cluster 2021-03-10T17:24:29.786589+0000 mgr.gibba032.whopeo (mgr.24625) 11149 : cluster [DBG] pgmap v6357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:30 gibba011 bash[30791]: cluster 2021-03-10T17:24:29.786589+0000 mgr.gibba032.whopeo (mgr.24625) 11149 : cluster [DBG] pgmap v6357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:31.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:30 gibba011 bash[31774]: cluster 2021-03-10T17:24:29.786589+0000 mgr.gibba032.whopeo (mgr.24625) 11149 : cluster [DBG] pgmap v6357: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:32.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:32 gibba032 bash[24004]: cluster 2021-03-10T17:24:31.788273+0000 mgr.gibba032.whopeo (mgr.24625) 11150 : cluster [DBG] pgmap v6358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:32 gibba011 bash[30791]: cluster 2021-03-10T17:24:31.788273+0000 mgr.gibba032.whopeo (mgr.24625) 11150 : cluster [DBG] pgmap v6358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:32 gibba011 bash[31774]: cluster 2021-03-10T17:24:31.788273+0000 mgr.gibba032.whopeo (mgr.24625) 11150 : cluster [DBG] pgmap v6358: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:34 gibba032 bash[24004]: cluster 2021-03-10T17:24:33.789851+0000 mgr.gibba032.whopeo (mgr.24625) 11151 : cluster [DBG] pgmap v6359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:34 gibba011 bash[30791]: cluster 2021-03-10T17:24:33.789851+0000 mgr.gibba032.whopeo (mgr.24625) 11151 : cluster [DBG] pgmap v6359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:35.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:34 gibba011 bash[31774]: cluster 2021-03-10T17:24:33.789851+0000 mgr.gibba032.whopeo (mgr.24625) 11151 : cluster [DBG] pgmap v6359: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:36 gibba032 bash[24004]: cluster 2021-03-10T17:24:35.791187+0000 mgr.gibba032.whopeo (mgr.24625) 11152 : cluster [DBG] pgmap v6360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:36 gibba011 bash[30791]: cluster 2021-03-10T17:24:35.791187+0000 mgr.gibba032.whopeo (mgr.24625) 11152 : cluster [DBG] pgmap v6360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:36 gibba011 bash[31774]: cluster 2021-03-10T17:24:35.791187+0000 mgr.gibba032.whopeo (mgr.24625) 11152 : cluster [DBG] pgmap v6360: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:39.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:38 gibba032 bash[24004]: cluster 2021-03-10T17:24:37.792996+0000 mgr.gibba032.whopeo (mgr.24625) 11153 : cluster [DBG] pgmap v6361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:38 gibba011 bash[30791]: cluster 2021-03-10T17:24:37.792996+0000 mgr.gibba032.whopeo (mgr.24625) 11153 : cluster [DBG] pgmap v6361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:38 gibba011 bash[31774]: cluster 2021-03-10T17:24:37.792996+0000 mgr.gibba032.whopeo (mgr.24625) 11153 : cluster [DBG] pgmap v6361: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:40 gibba032 bash[24004]: cluster 2021-03-10T17:24:39.794668+0000 mgr.gibba032.whopeo (mgr.24625) 11154 : cluster [DBG] pgmap v6362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:41.328 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:40 gibba011 bash[30791]: cluster 2021-03-10T17:24:39.794668+0000 mgr.gibba032.whopeo (mgr.24625) 11154 : cluster [DBG] pgmap v6362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:41.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:40 gibba011 bash[31774]: cluster 2021-03-10T17:24:39.794668+0000 mgr.gibba032.whopeo (mgr.24625) 11154 : cluster [DBG] pgmap v6362: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.281775+0000 mgr.gibba032.whopeo (mgr.24625) 11155 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:24:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.283339+0000 mgr.gibba032.whopeo (mgr.24625) 11156 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:24:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.502944+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:24:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.503907+0000 mgr.gibba032.whopeo (mgr.24625) 11157 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:24:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.505953+0000 mgr.gibba032.whopeo (mgr.24625) 11158 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:24:42.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.506493+0000 mon.a (mon.0) 3696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:24:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.507432+0000 mgr.gibba032.whopeo (mgr.24625) 11159 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:24:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.507480+0000 mgr.gibba032.whopeo (mgr.24625) 11160 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:24:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.508313+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.509064+0000 mgr.gibba032.whopeo (mgr.24625) 11161 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:24:42.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.509114+0000 mgr.gibba032.whopeo (mgr.24625) 11162 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:24:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.509805+0000 mon.a (mon.0) 3698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.510446+0000 mgr.gibba032.whopeo (mgr.24625) 11163 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:24:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.511371+0000 mon.a (mon.0) 3699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.511733+0000 mgr.gibba032.whopeo (mgr.24625) 11164 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:24:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.511783+0000 mgr.gibba032.whopeo (mgr.24625) 11165 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:24:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.513516+0000 mgr.gibba032.whopeo (mgr.24625) 11166 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:24:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.513958+0000 mgr.gibba032.whopeo (mgr.24625) 11167 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:24:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.514229+0000 mon.a (mon.0) 3700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.515215+0000 mgr.gibba032.whopeo (mgr.24625) 11168 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:24:42.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.515266+0000 mgr.gibba032.whopeo (mgr.24625) 11169 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:24:42.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cephadm 2021-03-10T17:24:41.515400+0000 mgr.gibba032.whopeo (mgr.24625) 11170 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:42.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: audit 2021-03-10T17:24:41.516204+0000 mon.a (mon.0) 3701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[30791]: cluster 2021-03-10T17:24:41.796522+0000 mgr.gibba032.whopeo (mgr.24625) 11171 : cluster [DBG] pgmap v6363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:42.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.281775+0000 mgr.gibba032.whopeo (mgr.24625) 11155 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:24:42.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.283339+0000 mgr.gibba032.whopeo (mgr.24625) 11156 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:24:42.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.502944+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:24:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.503907+0000 mgr.gibba032.whopeo (mgr.24625) 11157 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:24:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.505953+0000 mgr.gibba032.whopeo (mgr.24625) 11158 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:24:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.506493+0000 mon.a (mon.0) 3696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:24:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.507432+0000 mgr.gibba032.whopeo (mgr.24625) 11159 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:24:42.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.507480+0000 mgr.gibba032.whopeo (mgr.24625) 11160 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:24:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.508313+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.509064+0000 mgr.gibba032.whopeo (mgr.24625) 11161 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:24:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.509114+0000 mgr.gibba032.whopeo (mgr.24625) 11162 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:24:42.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.509805+0000 mon.a (mon.0) 3698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.510446+0000 mgr.gibba032.whopeo (mgr.24625) 11163 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:24:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.511371+0000 mon.a (mon.0) 3699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.511733+0000 mgr.gibba032.whopeo (mgr.24625) 11164 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:24:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.511783+0000 mgr.gibba032.whopeo (mgr.24625) 11165 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:24:42.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.513516+0000 mgr.gibba032.whopeo (mgr.24625) 11166 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:24:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.513958+0000 mgr.gibba032.whopeo (mgr.24625) 11167 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:24:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.514229+0000 mon.a (mon.0) 3700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.515215+0000 mgr.gibba032.whopeo (mgr.24625) 11168 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:24:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.515266+0000 mgr.gibba032.whopeo (mgr.24625) 11169 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:24:42.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cephadm 2021-03-10T17:24:41.515400+0000 mgr.gibba032.whopeo (mgr.24625) 11170 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:42.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: audit 2021-03-10T17:24:41.516204+0000 mon.a (mon.0) 3701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:42.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:42 gibba011 bash[31774]: cluster 2021-03-10T17:24:41.796522+0000 mgr.gibba032.whopeo (mgr.24625) 11171 : cluster [DBG] pgmap v6363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.281775+0000 mgr.gibba032.whopeo (mgr.24625) 11155 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:24:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.283339+0000 mgr.gibba032.whopeo (mgr.24625) 11156 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:24:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.502944+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:24:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.503907+0000 mgr.gibba032.whopeo (mgr.24625) 11157 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:24:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.505953+0000 mgr.gibba032.whopeo (mgr.24625) 11158 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:24:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.506493+0000 mon.a (mon.0) 3696 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:24:43.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.507432+0000 mgr.gibba032.whopeo (mgr.24625) 11159 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:24:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.507480+0000 mgr.gibba032.whopeo (mgr.24625) 11160 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:24:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.508313+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.509064+0000 mgr.gibba032.whopeo (mgr.24625) 11161 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:24:43.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.509114+0000 mgr.gibba032.whopeo (mgr.24625) 11162 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:24:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.509805+0000 mon.a (mon.0) 3698 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.510446+0000 mgr.gibba032.whopeo (mgr.24625) 11163 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:24:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.511371+0000 mon.a (mon.0) 3699 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.511733+0000 mgr.gibba032.whopeo (mgr.24625) 11164 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:24:43.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.511783+0000 mgr.gibba032.whopeo (mgr.24625) 11165 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:24:43.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.513516+0000 mgr.gibba032.whopeo (mgr.24625) 11166 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:24:43.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.513958+0000 mgr.gibba032.whopeo (mgr.24625) 11167 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:24:43.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.514229+0000 mon.a (mon.0) 3700 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:43.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.515215+0000 mgr.gibba032.whopeo (mgr.24625) 11168 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:24:43.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.515266+0000 mgr.gibba032.whopeo (mgr.24625) 11169 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:24:43.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cephadm 2021-03-10T17:24:41.515400+0000 mgr.gibba032.whopeo (mgr.24625) 11170 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:43.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: audit 2021-03-10T17:24:41.516204+0000 mon.a (mon.0) 3701 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:43.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:42 gibba032 bash[24004]: cluster 2021-03-10T17:24:41.796522+0000 mgr.gibba032.whopeo (mgr.24625) 11171 : cluster [DBG] pgmap v6363: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:45.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:44 gibba032 bash[24004]: cluster 2021-03-10T17:24:43.797914+0000 mgr.gibba032.whopeo (mgr.24625) 11172 : cluster [DBG] pgmap v6364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:45.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:44 gibba011 bash[30791]: cluster 2021-03-10T17:24:43.797914+0000 mgr.gibba032.whopeo (mgr.24625) 11172 : cluster [DBG] pgmap v6364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:45.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:44 gibba011 bash[31774]: cluster 2021-03-10T17:24:43.797914+0000 mgr.gibba032.whopeo (mgr.24625) 11172 : cluster [DBG] pgmap v6364: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:46 gibba032 bash[24004]: cluster 2021-03-10T17:24:45.799392+0000 mgr.gibba032.whopeo (mgr.24625) 11173 : cluster [DBG] pgmap v6365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:46 gibba011 bash[30791]: cluster 2021-03-10T17:24:45.799392+0000 mgr.gibba032.whopeo (mgr.24625) 11173 : cluster [DBG] pgmap v6365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:47.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:46 gibba011 bash[31774]: cluster 2021-03-10T17:24:45.799392+0000 mgr.gibba032.whopeo (mgr.24625) 11173 : cluster [DBG] pgmap v6365: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:48 gibba032 bash[24004]: cluster 2021-03-10T17:24:47.801254+0000 mgr.gibba032.whopeo (mgr.24625) 11174 : cluster [DBG] pgmap v6366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:48 gibba011 bash[30791]: cluster 2021-03-10T17:24:47.801254+0000 mgr.gibba032.whopeo (mgr.24625) 11174 : cluster [DBG] pgmap v6366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:48 gibba011 bash[31774]: cluster 2021-03-10T17:24:47.801254+0000 mgr.gibba032.whopeo (mgr.24625) 11174 : cluster [DBG] pgmap v6366: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:50 gibba032 bash[24004]: cluster 2021-03-10T17:24:49.802729+0000 mgr.gibba032.whopeo (mgr.24625) 11175 : cluster [DBG] pgmap v6367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:50 gibba011 bash[30791]: cluster 2021-03-10T17:24:49.802729+0000 mgr.gibba032.whopeo (mgr.24625) 11175 : cluster [DBG] pgmap v6367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:50 gibba011 bash[31774]: cluster 2021-03-10T17:24:49.802729+0000 mgr.gibba032.whopeo (mgr.24625) 11175 : cluster [DBG] pgmap v6367: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:52 gibba032 bash[24004]: cluster 2021-03-10T17:24:51.804526+0000 mgr.gibba032.whopeo (mgr.24625) 11176 : cluster [DBG] pgmap v6368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:52 gibba011 bash[30791]: cluster 2021-03-10T17:24:51.804526+0000 mgr.gibba032.whopeo (mgr.24625) 11176 : cluster [DBG] pgmap v6368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:52 gibba011 bash[31774]: cluster 2021-03-10T17:24:51.804526+0000 mgr.gibba032.whopeo (mgr.24625) 11176 : cluster [DBG] pgmap v6368: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:54.399 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:24:54.628 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:24:54.629 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:24:54.629 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:24:54.629 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:24:54.630 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:24:54.630 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:24:54.630 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:24:54.630 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:24:54.631 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:24:54.631 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:24:54.631 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:24:54.631 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:24:54.632 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:24:54.632 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:24:54.632 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:24:54.632 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:24:54.633 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:24:54.633 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:24:54.633 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:24:54.633 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:24:54.634 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:24:54.873 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:24:54.873 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:24:54.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:24:54.874 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:54.874 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:24:54.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:24:54.875 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:54.875 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:24:54.875 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:24:54.875 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:24:54.876 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:24:54.876 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:24:54.876 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:24:54.876 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:24:54.877 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:24:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:54 gibba032 bash[24004]: cluster 2021-03-10T17:24:53.806141+0000 mgr.gibba032.whopeo (mgr.24625) 11177 : cluster [DBG] pgmap v6369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:54 gibba011 bash[30791]: cluster 2021-03-10T17:24:53.806141+0000 mgr.gibba032.whopeo (mgr.24625) 11177 : cluster [DBG] pgmap v6369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:54 gibba011 bash[31774]: cluster 2021-03-10T17:24:53.806141+0000 mgr.gibba032.whopeo (mgr.24625) 11177 : cluster [DBG] pgmap v6369: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:56.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:55 gibba032 bash[24004]: audit 2021-03-10T17:24:54.388527+0000 mgr.gibba032.whopeo (mgr.24625) 11178 : audit [DBG] from='client.61525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:55 gibba032 bash[24004]: audit 2021-03-10T17:24:54.610920+0000 mgr.gibba032.whopeo (mgr.24625) 11179 : audit [DBG] from='client.51549 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:55 gibba032 bash[24004]: audit 2021-03-10T17:24:54.873028+0000 mon.a (mon.0) 3702 : audit [DBG] from='client.? 172.21.2.111:0/2429392561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:56.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[30791]: audit 2021-03-10T17:24:54.388527+0000 mgr.gibba032.whopeo (mgr.24625) 11178 : audit [DBG] from='client.61525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[30791]: audit 2021-03-10T17:24:54.610920+0000 mgr.gibba032.whopeo (mgr.24625) 11179 : audit [DBG] from='client.51549 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[30791]: audit 2021-03-10T17:24:54.873028+0000 mon.a (mon.0) 3702 : audit [DBG] from='client.? 172.21.2.111:0/2429392561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[31774]: audit 2021-03-10T17:24:54.388527+0000 mgr.gibba032.whopeo (mgr.24625) 11178 : audit [DBG] from='client.61525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[31774]: audit 2021-03-10T17:24:54.610920+0000 mgr.gibba032.whopeo (mgr.24625) 11179 : audit [DBG] from='client.51549 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:24:56.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:55 gibba011 bash[31774]: audit 2021-03-10T17:24:54.873028+0000 mon.a (mon.0) 3702 : audit [DBG] from='client.? 172.21.2.111:0/2429392561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:24:57.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:56 gibba032 bash[24004]: cluster 2021-03-10T17:24:55.807515+0000 mgr.gibba032.whopeo (mgr.24625) 11180 : cluster [DBG] pgmap v6370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:56 gibba011 bash[30791]: cluster 2021-03-10T17:24:55.807515+0000 mgr.gibba032.whopeo (mgr.24625) 11180 : cluster [DBG] pgmap v6370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:56 gibba011 bash[31774]: cluster 2021-03-10T17:24:55.807515+0000 mgr.gibba032.whopeo (mgr.24625) 11180 : cluster [DBG] pgmap v6370: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: cephadm 2021-03-10T17:24:56.522705+0000 mgr.gibba032.whopeo (mgr.24625) 11181 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.476139+0000 mon.a (mon.0) 3703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:24:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.476773+0000 mon.a (mon.0) 3704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:24:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.659717+0000 mon.a (mon.0) 3705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:24:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.660479+0000 mon.a (mon.0) 3706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.661139+0000 mon.a (mon.0) 3707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:57 gibba032 bash[24004]: audit 2021-03-10T17:24:57.661809+0000 mon.a (mon.0) 3708 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:24:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: cephadm 2021-03-10T17:24:56.522705+0000 mgr.gibba032.whopeo (mgr.24625) 11181 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.476139+0000 mon.a (mon.0) 3703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:24:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.476773+0000 mon.a (mon.0) 3704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:24:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.659717+0000 mon.a (mon.0) 3705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:24:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.660479+0000 mon.a (mon.0) 3706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.661139+0000 mon.a (mon.0) 3707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[30791]: audit 2021-03-10T17:24:57.661809+0000 mon.a (mon.0) 3708 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:24:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: cephadm 2021-03-10T17:24:56.522705+0000 mgr.gibba032.whopeo (mgr.24625) 11181 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:24:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.476139+0000 mon.a (mon.0) 3703 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.476773+0000 mon.a (mon.0) 3704 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.659717+0000 mon.a (mon.0) 3705 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.660479+0000 mon.a (mon.0) 3706 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.661139+0000 mon.a (mon.0) 3707 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:24:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:57 gibba011 bash[31774]: audit 2021-03-10T17:24:57.661809+0000 mon.a (mon.0) 3708 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:24:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:24:58 gibba032 bash[24004]: cluster 2021-03-10T17:24:57.809128+0000 mgr.gibba032.whopeo (mgr.24625) 11182 : cluster [DBG] pgmap v6371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:24:58 gibba011 bash[30791]: cluster 2021-03-10T17:24:57.809128+0000 mgr.gibba032.whopeo (mgr.24625) 11182 : cluster [DBG] pgmap v6371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:24:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:24:58 gibba011 bash[31774]: cluster 2021-03-10T17:24:57.809128+0000 mgr.gibba032.whopeo (mgr.24625) 11182 : cluster [DBG] pgmap v6371: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:00 gibba032 bash[24004]: cluster 2021-03-10T17:24:59.810395+0000 mgr.gibba032.whopeo (mgr.24625) 11183 : cluster [DBG] pgmap v6372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:00 gibba011 bash[30791]: cluster 2021-03-10T17:24:59.810395+0000 mgr.gibba032.whopeo (mgr.24625) 11183 : cluster [DBG] pgmap v6372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:01.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:00 gibba011 bash[31774]: cluster 2021-03-10T17:24:59.810395+0000 mgr.gibba032.whopeo (mgr.24625) 11183 : cluster [DBG] pgmap v6372: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:02.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:02 gibba032 bash[24004]: cluster 2021-03-10T17:25:01.812165+0000 mgr.gibba032.whopeo (mgr.24625) 11184 : cluster [DBG] pgmap v6373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:02 gibba011 bash[30791]: cluster 2021-03-10T17:25:01.812165+0000 mgr.gibba032.whopeo (mgr.24625) 11184 : cluster [DBG] pgmap v6373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:02 gibba011 bash[31774]: cluster 2021-03-10T17:25:01.812165+0000 mgr.gibba032.whopeo (mgr.24625) 11184 : cluster [DBG] pgmap v6373: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:04 gibba032 bash[24004]: cluster 2021-03-10T17:25:03.813767+0000 mgr.gibba032.whopeo (mgr.24625) 11185 : cluster [DBG] pgmap v6374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:04 gibba011 bash[30791]: cluster 2021-03-10T17:25:03.813767+0000 mgr.gibba032.whopeo (mgr.24625) 11185 : cluster [DBG] pgmap v6374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:04 gibba011 bash[31774]: cluster 2021-03-10T17:25:03.813767+0000 mgr.gibba032.whopeo (mgr.24625) 11185 : cluster [DBG] pgmap v6374: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:06 gibba032 bash[24004]: cluster 2021-03-10T17:25:05.815091+0000 mgr.gibba032.whopeo (mgr.24625) 11186 : cluster [DBG] pgmap v6375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:06 gibba011 bash[30791]: cluster 2021-03-10T17:25:05.815091+0000 mgr.gibba032.whopeo (mgr.24625) 11186 : cluster [DBG] pgmap v6375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:07.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:06 gibba011 bash[31774]: cluster 2021-03-10T17:25:05.815091+0000 mgr.gibba032.whopeo (mgr.24625) 11186 : cluster [DBG] pgmap v6375: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:08 gibba032 bash[24004]: cluster 2021-03-10T17:25:07.816895+0000 mgr.gibba032.whopeo (mgr.24625) 11187 : cluster [DBG] pgmap v6376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:08 gibba011 bash[30791]: cluster 2021-03-10T17:25:07.816895+0000 mgr.gibba032.whopeo (mgr.24625) 11187 : cluster [DBG] pgmap v6376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:08 gibba011 bash[31774]: cluster 2021-03-10T17:25:07.816895+0000 mgr.gibba032.whopeo (mgr.24625) 11187 : cluster [DBG] pgmap v6376: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:11.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:10 gibba032 bash[24004]: cluster 2021-03-10T17:25:09.818487+0000 mgr.gibba032.whopeo (mgr.24625) 11188 : cluster [DBG] pgmap v6377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:10 gibba011 bash[30791]: cluster 2021-03-10T17:25:09.818487+0000 mgr.gibba032.whopeo (mgr.24625) 11188 : cluster [DBG] pgmap v6377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:10 gibba011 bash[31774]: cluster 2021-03-10T17:25:09.818487+0000 mgr.gibba032.whopeo (mgr.24625) 11188 : cluster [DBG] pgmap v6377: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:12 gibba032 bash[24004]: cephadm 2021-03-10T17:25:11.538689+0000 mgr.gibba032.whopeo (mgr.24625) 11189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:12 gibba032 bash[24004]: cluster 2021-03-10T17:25:11.820267+0000 mgr.gibba032.whopeo (mgr.24625) 11190 : cluster [DBG] pgmap v6378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:12 gibba011 bash[30791]: cephadm 2021-03-10T17:25:11.538689+0000 mgr.gibba032.whopeo (mgr.24625) 11189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:12 gibba011 bash[30791]: cluster 2021-03-10T17:25:11.820267+0000 mgr.gibba032.whopeo (mgr.24625) 11190 : cluster [DBG] pgmap v6378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:12 gibba011 bash[31774]: cephadm 2021-03-10T17:25:11.538689+0000 mgr.gibba032.whopeo (mgr.24625) 11189 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:12 gibba011 bash[31774]: cluster 2021-03-10T17:25:11.820267+0000 mgr.gibba032.whopeo (mgr.24625) 11190 : cluster [DBG] pgmap v6378: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:14 gibba032 bash[24004]: cluster 2021-03-10T17:25:13.821912+0000 mgr.gibba032.whopeo (mgr.24625) 11191 : cluster [DBG] pgmap v6379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:14 gibba011 bash[30791]: cluster 2021-03-10T17:25:13.821912+0000 mgr.gibba032.whopeo (mgr.24625) 11191 : cluster [DBG] pgmap v6379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:15.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:14 gibba011 bash[31774]: cluster 2021-03-10T17:25:13.821912+0000 mgr.gibba032.whopeo (mgr.24625) 11191 : cluster [DBG] pgmap v6379: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:16 gibba032 bash[24004]: cluster 2021-03-10T17:25:15.823201+0000 mgr.gibba032.whopeo (mgr.24625) 11192 : cluster [DBG] pgmap v6380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:16 gibba011 bash[30791]: cluster 2021-03-10T17:25:15.823201+0000 mgr.gibba032.whopeo (mgr.24625) 11192 : cluster [DBG] pgmap v6380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:16 gibba011 bash[31774]: cluster 2021-03-10T17:25:15.823201+0000 mgr.gibba032.whopeo (mgr.24625) 11192 : cluster [DBG] pgmap v6380: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:18 gibba032 bash[24004]: cluster 2021-03-10T17:25:17.825022+0000 mgr.gibba032.whopeo (mgr.24625) 11193 : cluster [DBG] pgmap v6381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:18 gibba011 bash[30791]: cluster 2021-03-10T17:25:17.825022+0000 mgr.gibba032.whopeo (mgr.24625) 11193 : cluster [DBG] pgmap v6381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:18 gibba011 bash[31774]: cluster 2021-03-10T17:25:17.825022+0000 mgr.gibba032.whopeo (mgr.24625) 11193 : cluster [DBG] pgmap v6381: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:21.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:20 gibba032 bash[24004]: cluster 2021-03-10T17:25:19.826588+0000 mgr.gibba032.whopeo (mgr.24625) 11194 : cluster [DBG] pgmap v6382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:20 gibba011 bash[30791]: cluster 2021-03-10T17:25:19.826588+0000 mgr.gibba032.whopeo (mgr.24625) 11194 : cluster [DBG] pgmap v6382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:20 gibba011 bash[31774]: cluster 2021-03-10T17:25:19.826588+0000 mgr.gibba032.whopeo (mgr.24625) 11194 : cluster [DBG] pgmap v6382: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:22 gibba032 bash[24004]: cluster 2021-03-10T17:25:21.828217+0000 mgr.gibba032.whopeo (mgr.24625) 11195 : cluster [DBG] pgmap v6383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:22 gibba011 bash[30791]: cluster 2021-03-10T17:25:21.828217+0000 mgr.gibba032.whopeo (mgr.24625) 11195 : cluster [DBG] pgmap v6383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:22 gibba011 bash[31774]: cluster 2021-03-10T17:25:21.828217+0000 mgr.gibba032.whopeo (mgr.24625) 11195 : cluster [DBG] pgmap v6383: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:25.130 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:25:25.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:24 gibba032 bash[24004]: cluster 2021-03-10T17:25:23.829811+0000 mgr.gibba032.whopeo (mgr.24625) 11196 : cluster [DBG] pgmap v6384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:24 gibba011 bash[30791]: cluster 2021-03-10T17:25:23.829811+0000 mgr.gibba032.whopeo (mgr.24625) 11196 : cluster [DBG] pgmap v6384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:25.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:24 gibba011 bash[31774]: cluster 2021-03-10T17:25:23.829811+0000 mgr.gibba032.whopeo (mgr.24625) 11196 : cluster [DBG] pgmap v6384: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:25.372 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:25:25.372 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:25:25.372 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:25:25.373 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:25:25.373 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:25:25.373 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:25:25.373 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:25:25.374 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:25:25.374 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:25:25.374 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:25:25.374 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:25:25.374 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:25:25.375 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:25:25.375 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:25:25.375 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:25:25.375 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:25:25.376 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:25:25.376 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:25:25.376 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:25:25.377 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:25:25.377 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:25:25.620 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:25:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:25:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:25:25.621 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:25:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:25:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:25:25.622 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:25:25.623 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:25.623 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:25:25.623 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:25:25.623 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:25:25.624 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:25:25.624 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:25:26.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:25 gibba032 bash[24004]: audit 2021-03-10T17:25:25.620376+0000 mon.a (mon.0) 3709 : audit [DBG] from='client.? 172.21.2.111:0/2826804207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:26.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:25 gibba011 bash[30791]: audit 2021-03-10T17:25:25.620376+0000 mon.a (mon.0) 3709 : audit [DBG] from='client.? 172.21.2.111:0/2826804207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:26.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:25 gibba011 bash[31774]: audit 2021-03-10T17:25:25.620376+0000 mon.a (mon.0) 3709 : audit [DBG] from='client.? 172.21.2.111:0/2826804207' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:26 gibba032 bash[24004]: audit 2021-03-10T17:25:25.110863+0000 mgr.gibba032.whopeo (mgr.24625) 11197 : audit [DBG] from='client.51561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:26 gibba032 bash[24004]: audit 2021-03-10T17:25:25.357473+0000 mgr.gibba032.whopeo (mgr.24625) 11198 : audit [DBG] from='client.51567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:26 gibba032 bash[24004]: cluster 2021-03-10T17:25:25.831252+0000 mgr.gibba032.whopeo (mgr.24625) 11199 : cluster [DBG] pgmap v6385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[30791]: audit 2021-03-10T17:25:25.110863+0000 mgr.gibba032.whopeo (mgr.24625) 11197 : audit [DBG] from='client.51561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[30791]: audit 2021-03-10T17:25:25.357473+0000 mgr.gibba032.whopeo (mgr.24625) 11198 : audit [DBG] from='client.51567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[30791]: cluster 2021-03-10T17:25:25.831252+0000 mgr.gibba032.whopeo (mgr.24625) 11199 : cluster [DBG] pgmap v6385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[31774]: audit 2021-03-10T17:25:25.110863+0000 mgr.gibba032.whopeo (mgr.24625) 11197 : audit [DBG] from='client.51561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[31774]: audit 2021-03-10T17:25:25.357473+0000 mgr.gibba032.whopeo (mgr.24625) 11198 : audit [DBG] from='client.51567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:27.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:26 gibba011 bash[31774]: cluster 2021-03-10T17:25:25.831252+0000 mgr.gibba032.whopeo (mgr.24625) 11199 : cluster [DBG] pgmap v6385: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:27 gibba032 bash[24004]: cephadm 2021-03-10T17:25:26.554747+0000 mgr.gibba032.whopeo (mgr.24625) 11200 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:27 gibba011 bash[30791]: cephadm 2021-03-10T17:25:26.554747+0000 mgr.gibba032.whopeo (mgr.24625) 11200 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:27 gibba011 bash[31774]: cephadm 2021-03-10T17:25:26.554747+0000 mgr.gibba032.whopeo (mgr.24625) 11200 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:28 gibba032 bash[24004]: cluster 2021-03-10T17:25:27.833049+0000 mgr.gibba032.whopeo (mgr.24625) 11201 : cluster [DBG] pgmap v6386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:28 gibba011 bash[30791]: cluster 2021-03-10T17:25:27.833049+0000 mgr.gibba032.whopeo (mgr.24625) 11201 : cluster [DBG] pgmap v6386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:29.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:28 gibba011 bash[31774]: cluster 2021-03-10T17:25:27.833049+0000 mgr.gibba032.whopeo (mgr.24625) 11201 : cluster [DBG] pgmap v6386: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:31.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:30 gibba032 bash[24004]: cluster 2021-03-10T17:25:29.834524+0000 mgr.gibba032.whopeo (mgr.24625) 11202 : cluster [DBG] pgmap v6387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:30 gibba011 bash[30791]: cluster 2021-03-10T17:25:29.834524+0000 mgr.gibba032.whopeo (mgr.24625) 11202 : cluster [DBG] pgmap v6387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:30 gibba011 bash[31774]: cluster 2021-03-10T17:25:29.834524+0000 mgr.gibba032.whopeo (mgr.24625) 11202 : cluster [DBG] pgmap v6387: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:32 gibba032 bash[24004]: cluster 2021-03-10T17:25:31.836249+0000 mgr.gibba032.whopeo (mgr.24625) 11203 : cluster [DBG] pgmap v6388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:32 gibba011 bash[30791]: cluster 2021-03-10T17:25:31.836249+0000 mgr.gibba032.whopeo (mgr.24625) 11203 : cluster [DBG] pgmap v6388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:32 gibba011 bash[31774]: cluster 2021-03-10T17:25:31.836249+0000 mgr.gibba032.whopeo (mgr.24625) 11203 : cluster [DBG] pgmap v6388: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:34 gibba032 bash[24004]: cluster 2021-03-10T17:25:33.837809+0000 mgr.gibba032.whopeo (mgr.24625) 11204 : cluster [DBG] pgmap v6389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:35.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:34 gibba011 bash[30791]: cluster 2021-03-10T17:25:33.837809+0000 mgr.gibba032.whopeo (mgr.24625) 11204 : cluster [DBG] pgmap v6389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:34 gibba011 bash[31774]: cluster 2021-03-10T17:25:33.837809+0000 mgr.gibba032.whopeo (mgr.24625) 11204 : cluster [DBG] pgmap v6389: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:36 gibba032 bash[24004]: cluster 2021-03-10T17:25:35.839127+0000 mgr.gibba032.whopeo (mgr.24625) 11205 : cluster [DBG] pgmap v6390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:36 gibba011 bash[30791]: cluster 2021-03-10T17:25:35.839127+0000 mgr.gibba032.whopeo (mgr.24625) 11205 : cluster [DBG] pgmap v6390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:36 gibba011 bash[31774]: cluster 2021-03-10T17:25:35.839127+0000 mgr.gibba032.whopeo (mgr.24625) 11205 : cluster [DBG] pgmap v6390: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:39.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:38 gibba032 bash[24004]: cluster 2021-03-10T17:25:37.840895+0000 mgr.gibba032.whopeo (mgr.24625) 11206 : cluster [DBG] pgmap v6391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:38 gibba011 bash[30791]: cluster 2021-03-10T17:25:37.840895+0000 mgr.gibba032.whopeo (mgr.24625) 11206 : cluster [DBG] pgmap v6391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:38 gibba011 bash[31774]: cluster 2021-03-10T17:25:37.840895+0000 mgr.gibba032.whopeo (mgr.24625) 11206 : cluster [DBG] pgmap v6391: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:40 gibba032 bash[24004]: cluster 2021-03-10T17:25:39.842427+0000 mgr.gibba032.whopeo (mgr.24625) 11207 : cluster [DBG] pgmap v6392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:40 gibba011 bash[30791]: cluster 2021-03-10T17:25:39.842427+0000 mgr.gibba032.whopeo (mgr.24625) 11207 : cluster [DBG] pgmap v6392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:40 gibba011 bash[31774]: cluster 2021-03-10T17:25:39.842427+0000 mgr.gibba032.whopeo (mgr.24625) 11207 : cluster [DBG] pgmap v6392: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.569691+0000 mgr.gibba032.whopeo (mgr.24625) 11208 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:25:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.571215+0000 mgr.gibba032.whopeo (mgr.24625) 11209 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:25:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.795974+0000 mon.a (mon.0) 3710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:25:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.796746+0000 mgr.gibba032.whopeo (mgr.24625) 11210 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:25:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.798484+0000 mgr.gibba032.whopeo (mgr.24625) 11211 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:25:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.799302+0000 mon.a (mon.0) 3711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:25:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.799841+0000 mgr.gibba032.whopeo (mgr.24625) 11212 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:25:43.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.799903+0000 mgr.gibba032.whopeo (mgr.24625) 11213 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:25:43.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.800781+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.801635+0000 mgr.gibba032.whopeo (mgr.24625) 11214 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:25:43.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.801676+0000 mgr.gibba032.whopeo (mgr.24625) 11215 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:25:43.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.802283+0000 mon.a (mon.0) 3713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.803158+0000 mgr.gibba032.whopeo (mgr.24625) 11216 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:25:43.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.803995+0000 mon.a (mon.0) 3714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.804484+0000 mgr.gibba032.whopeo (mgr.24625) 11217 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:25:43.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.804522+0000 mgr.gibba032.whopeo (mgr.24625) 11218 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:25:43.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.805986+0000 mgr.gibba032.whopeo (mgr.24625) 11219 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:25:43.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.806394+0000 mgr.gibba032.whopeo (mgr.24625) 11220 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:25:43.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.806944+0000 mon.a (mon.0) 3715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.807932+0000 mgr.gibba032.whopeo (mgr.24625) 11221 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:25:43.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.807974+0000 mgr.gibba032.whopeo (mgr.24625) 11222 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:25:43.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.569691+0000 mgr.gibba032.whopeo (mgr.24625) 11208 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:25:43.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.571215+0000 mgr.gibba032.whopeo (mgr.24625) 11209 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:25:43.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.795974+0000 mon.a (mon.0) 3710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:25:43.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.796746+0000 mgr.gibba032.whopeo (mgr.24625) 11210 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:25:43.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.798484+0000 mgr.gibba032.whopeo (mgr.24625) 11211 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:25:43.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.799302+0000 mon.a (mon.0) 3711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:25:43.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.799841+0000 mgr.gibba032.whopeo (mgr.24625) 11212 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:25:43.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.799903+0000 mgr.gibba032.whopeo (mgr.24625) 11213 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:25:43.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.800781+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.801635+0000 mgr.gibba032.whopeo (mgr.24625) 11214 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:25:43.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.801676+0000 mgr.gibba032.whopeo (mgr.24625) 11215 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:25:43.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.802283+0000 mon.a (mon.0) 3713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.803158+0000 mgr.gibba032.whopeo (mgr.24625) 11216 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:25:43.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.803995+0000 mon.a (mon.0) 3714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.804484+0000 mgr.gibba032.whopeo (mgr.24625) 11217 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:25:43.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.804522+0000 mgr.gibba032.whopeo (mgr.24625) 11218 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:25:43.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.805986+0000 mgr.gibba032.whopeo (mgr.24625) 11219 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:25:43.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.806394+0000 mgr.gibba032.whopeo (mgr.24625) 11220 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:25:43.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.806944+0000 mon.a (mon.0) 3715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.807932+0000 mgr.gibba032.whopeo (mgr.24625) 11221 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:25:43.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.807974+0000 mgr.gibba032.whopeo (mgr.24625) 11222 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:25:43.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cephadm 2021-03-10T17:25:41.808177+0000 mgr.gibba032.whopeo (mgr.24625) 11223 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:43.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: audit 2021-03-10T17:25:41.808637+0000 mon.a (mon.0) 3716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[30791]: cluster 2021-03-10T17:25:41.843253+0000 mgr.gibba032.whopeo (mgr.24625) 11224 : cluster [DBG] pgmap v6393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:43.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cephadm 2021-03-10T17:25:41.808177+0000 mgr.gibba032.whopeo (mgr.24625) 11223 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:43.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: audit 2021-03-10T17:25:41.808637+0000 mon.a (mon.0) 3716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:42 gibba011 bash[31774]: cluster 2021-03-10T17:25:41.843253+0000 mgr.gibba032.whopeo (mgr.24625) 11224 : cluster [DBG] pgmap v6393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.569691+0000 mgr.gibba032.whopeo (mgr.24625) 11208 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:25:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.571215+0000 mgr.gibba032.whopeo (mgr.24625) 11209 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:25:43.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.795974+0000 mon.a (mon.0) 3710 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:25:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.796746+0000 mgr.gibba032.whopeo (mgr.24625) 11210 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:25:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.798484+0000 mgr.gibba032.whopeo (mgr.24625) 11211 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:25:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.799302+0000 mon.a (mon.0) 3711 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:25:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.799841+0000 mgr.gibba032.whopeo (mgr.24625) 11212 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:25:43.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.799903+0000 mgr.gibba032.whopeo (mgr.24625) 11213 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:25:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.800781+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.801635+0000 mgr.gibba032.whopeo (mgr.24625) 11214 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:25:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.801676+0000 mgr.gibba032.whopeo (mgr.24625) 11215 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:25:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.802283+0000 mon.a (mon.0) 3713 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.803158+0000 mgr.gibba032.whopeo (mgr.24625) 11216 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:25:43.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.803995+0000 mon.a (mon.0) 3714 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.804484+0000 mgr.gibba032.whopeo (mgr.24625) 11217 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:25:43.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.804522+0000 mgr.gibba032.whopeo (mgr.24625) 11218 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:25:43.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.805986+0000 mgr.gibba032.whopeo (mgr.24625) 11219 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:25:43.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.806394+0000 mgr.gibba032.whopeo (mgr.24625) 11220 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:25:43.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.806944+0000 mon.a (mon.0) 3715 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.807932+0000 mgr.gibba032.whopeo (mgr.24625) 11221 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:25:43.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.807974+0000 mgr.gibba032.whopeo (mgr.24625) 11222 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:25:43.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cephadm 2021-03-10T17:25:41.808177+0000 mgr.gibba032.whopeo (mgr.24625) 11223 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:43.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: audit 2021-03-10T17:25:41.808637+0000 mon.a (mon.0) 3716 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:43.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:42 gibba032 bash[24004]: cluster 2021-03-10T17:25:41.843253+0000 mgr.gibba032.whopeo (mgr.24625) 11224 : cluster [DBG] pgmap v6393: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:45.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:44 gibba032 bash[24004]: cluster 2021-03-10T17:25:43.844758+0000 mgr.gibba032.whopeo (mgr.24625) 11225 : cluster [DBG] pgmap v6394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:45.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:44 gibba011 bash[30791]: cluster 2021-03-10T17:25:43.844758+0000 mgr.gibba032.whopeo (mgr.24625) 11225 : cluster [DBG] pgmap v6394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:45.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:44 gibba011 bash[31774]: cluster 2021-03-10T17:25:43.844758+0000 mgr.gibba032.whopeo (mgr.24625) 11225 : cluster [DBG] pgmap v6394: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:46 gibba032 bash[24004]: cluster 2021-03-10T17:25:45.846234+0000 mgr.gibba032.whopeo (mgr.24625) 11226 : cluster [DBG] pgmap v6395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:46 gibba011 bash[30791]: cluster 2021-03-10T17:25:45.846234+0000 mgr.gibba032.whopeo (mgr.24625) 11226 : cluster [DBG] pgmap v6395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:46 gibba011 bash[31774]: cluster 2021-03-10T17:25:45.846234+0000 mgr.gibba032.whopeo (mgr.24625) 11226 : cluster [DBG] pgmap v6395: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:49.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:48 gibba032 bash[24004]: cluster 2021-03-10T17:25:47.847975+0000 mgr.gibba032.whopeo (mgr.24625) 11227 : cluster [DBG] pgmap v6396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:48 gibba011 bash[30791]: cluster 2021-03-10T17:25:47.847975+0000 mgr.gibba032.whopeo (mgr.24625) 11227 : cluster [DBG] pgmap v6396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:48 gibba011 bash[31774]: cluster 2021-03-10T17:25:47.847975+0000 mgr.gibba032.whopeo (mgr.24625) 11227 : cluster [DBG] pgmap v6396: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:51.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:50 gibba032 bash[24004]: cluster 2021-03-10T17:25:49.849530+0000 mgr.gibba032.whopeo (mgr.24625) 11228 : cluster [DBG] pgmap v6397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:50 gibba011 bash[30791]: cluster 2021-03-10T17:25:49.849530+0000 mgr.gibba032.whopeo (mgr.24625) 11228 : cluster [DBG] pgmap v6397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:51.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:50 gibba011 bash[31774]: cluster 2021-03-10T17:25:49.849530+0000 mgr.gibba032.whopeo (mgr.24625) 11228 : cluster [DBG] pgmap v6397: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:52.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:52 gibba032 bash[24004]: cluster 2021-03-10T17:25:51.851302+0000 mgr.gibba032.whopeo (mgr.24625) 11229 : cluster [DBG] pgmap v6398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:52 gibba011 bash[30791]: cluster 2021-03-10T17:25:51.851302+0000 mgr.gibba032.whopeo (mgr.24625) 11229 : cluster [DBG] pgmap v6398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:52 gibba011 bash[31774]: cluster 2021-03-10T17:25:51.851302+0000 mgr.gibba032.whopeo (mgr.24625) 11229 : cluster [DBG] pgmap v6398: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:54 gibba032 bash[24004]: cluster 2021-03-10T17:25:53.852961+0000 mgr.gibba032.whopeo (mgr.24625) 11230 : cluster [DBG] pgmap v6399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:54 gibba011 bash[30791]: cluster 2021-03-10T17:25:53.852961+0000 mgr.gibba032.whopeo (mgr.24625) 11230 : cluster [DBG] pgmap v6399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:54 gibba011 bash[31774]: cluster 2021-03-10T17:25:53.852961+0000 mgr.gibba032.whopeo (mgr.24625) 11230 : cluster [DBG] pgmap v6399: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:55.861 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:25:56.100 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:25:56.100 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:25:56.101 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:25:56.101 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:25:56.130 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:25:56.130 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:25:56.130 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:25:56.131 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:25:56.131 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:25:56.132 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:25:56.132 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:25:56.132 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:25:56.133 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:25:56.133 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:25:56.133 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:25:56.134 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:25:56.134 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:25:56.134 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:25:56.135 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:25:56.135 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:25:56.135 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:25:56.350 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:25:56.350 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:25:56.350 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:25:56.350 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:56.351 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:25:56.351 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:25:56.351 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:56.351 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:25:56.352 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:25:56.352 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:25:56.352 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:25:56.352 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:25:56.352 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:25:56.353 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:25:56.353 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:25:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:56 gibba032 bash[24004]: audit 2021-03-10T17:25:55.849720+0000 mgr.gibba032.whopeo (mgr.24625) 11231 : audit [DBG] from='client.61561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:56 gibba032 bash[24004]: cluster 2021-03-10T17:25:55.854190+0000 mgr.gibba032.whopeo (mgr.24625) 11232 : cluster [DBG] pgmap v6400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:56 gibba032 bash[24004]: audit 2021-03-10T17:25:56.349630+0000 mon.c (mon.1) 216 : audit [DBG] from='client.? 172.21.2.111:0/3249629270' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[30791]: audit 2021-03-10T17:25:55.849720+0000 mgr.gibba032.whopeo (mgr.24625) 11231 : audit [DBG] from='client.61561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[30791]: cluster 2021-03-10T17:25:55.854190+0000 mgr.gibba032.whopeo (mgr.24625) 11232 : cluster [DBG] pgmap v6400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[30791]: audit 2021-03-10T17:25:56.349630+0000 mon.c (mon.1) 216 : audit [DBG] from='client.? 172.21.2.111:0/3249629270' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[31774]: audit 2021-03-10T17:25:55.849720+0000 mgr.gibba032.whopeo (mgr.24625) 11231 : audit [DBG] from='client.61561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[31774]: cluster 2021-03-10T17:25:55.854190+0000 mgr.gibba032.whopeo (mgr.24625) 11232 : cluster [DBG] pgmap v6400: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:57.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:56 gibba011 bash[31774]: audit 2021-03-10T17:25:56.349630+0000 mon.c (mon.1) 216 : audit [DBG] from='client.? 172.21.2.111:0/3249629270' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:25:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:56.085050+0000 mgr.gibba032.whopeo (mgr.24625) 11233 : audit [DBG] from='client.61567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: cephadm 2021-03-10T17:25:56.824218+0000 mgr.gibba032.whopeo (mgr.24625) 11234 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.482872+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:25:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.483635+0000 mon.a (mon.0) 3718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:25:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.670131+0000 mon.a (mon.0) 3719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:25:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.670978+0000 mon.a (mon.0) 3720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.671863+0000 mon.a (mon.0) 3721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:57 gibba032 bash[24004]: audit 2021-03-10T17:25:57.672647+0000 mon.a (mon.0) 3722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:25:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:56.085050+0000 mgr.gibba032.whopeo (mgr.24625) 11233 : audit [DBG] from='client.61567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: cephadm 2021-03-10T17:25:56.824218+0000 mgr.gibba032.whopeo (mgr.24625) 11234 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.482872+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:25:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.483635+0000 mon.a (mon.0) 3718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:25:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.670131+0000 mon.a (mon.0) 3719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:25:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.670978+0000 mon.a (mon.0) 3720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.671863+0000 mon.a (mon.0) 3721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[30791]: audit 2021-03-10T17:25:57.672647+0000 mon.a (mon.0) 3722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:25:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:56.085050+0000 mgr.gibba032.whopeo (mgr.24625) 11233 : audit [DBG] from='client.61567 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:25:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: cephadm 2021-03-10T17:25:56.824218+0000 mgr.gibba032.whopeo (mgr.24625) 11234 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:25:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.482872+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:25:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.483635+0000 mon.a (mon.0) 3718 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:25:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.670131+0000 mon.a (mon.0) 3719 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:25:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.670978+0000 mon.a (mon.0) 3720 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.671863+0000 mon.a (mon.0) 3721 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:25:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:57 gibba011 bash[31774]: audit 2021-03-10T17:25:57.672647+0000 mon.a (mon.0) 3722 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:25:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:25:58 gibba032 bash[24004]: cluster 2021-03-10T17:25:57.855872+0000 mgr.gibba032.whopeo (mgr.24625) 11235 : cluster [DBG] pgmap v6401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:59.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:25:58 gibba011 bash[30791]: cluster 2021-03-10T17:25:57.855872+0000 mgr.gibba032.whopeo (mgr.24625) 11235 : cluster [DBG] pgmap v6401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:25:59.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:25:58 gibba011 bash[31774]: cluster 2021-03-10T17:25:57.855872+0000 mgr.gibba032.whopeo (mgr.24625) 11235 : cluster [DBG] pgmap v6401: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:00 gibba032 bash[24004]: cluster 2021-03-10T17:25:59.857504+0000 mgr.gibba032.whopeo (mgr.24625) 11236 : cluster [DBG] pgmap v6402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:00 gibba011 bash[30791]: cluster 2021-03-10T17:25:59.857504+0000 mgr.gibba032.whopeo (mgr.24625) 11236 : cluster [DBG] pgmap v6402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:00 gibba011 bash[31774]: cluster 2021-03-10T17:25:59.857504+0000 mgr.gibba032.whopeo (mgr.24625) 11236 : cluster [DBG] pgmap v6402: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:02 gibba032 bash[24004]: cluster 2021-03-10T17:26:01.859270+0000 mgr.gibba032.whopeo (mgr.24625) 11237 : cluster [DBG] pgmap v6403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:02 gibba011 bash[30791]: cluster 2021-03-10T17:26:01.859270+0000 mgr.gibba032.whopeo (mgr.24625) 11237 : cluster [DBG] pgmap v6403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:02 gibba011 bash[31774]: cluster 2021-03-10T17:26:01.859270+0000 mgr.gibba032.whopeo (mgr.24625) 11237 : cluster [DBG] pgmap v6403: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:04 gibba032 bash[24004]: cluster 2021-03-10T17:26:03.860938+0000 mgr.gibba032.whopeo (mgr.24625) 11238 : cluster [DBG] pgmap v6404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:04 gibba011 bash[30791]: cluster 2021-03-10T17:26:03.860938+0000 mgr.gibba032.whopeo (mgr.24625) 11238 : cluster [DBG] pgmap v6404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:04 gibba011 bash[31774]: cluster 2021-03-10T17:26:03.860938+0000 mgr.gibba032.whopeo (mgr.24625) 11238 : cluster [DBG] pgmap v6404: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:06 gibba032 bash[24004]: cluster 2021-03-10T17:26:05.862391+0000 mgr.gibba032.whopeo (mgr.24625) 11239 : cluster [DBG] pgmap v6405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:06 gibba011 bash[30791]: cluster 2021-03-10T17:26:05.862391+0000 mgr.gibba032.whopeo (mgr.24625) 11239 : cluster [DBG] pgmap v6405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:06 gibba011 bash[31774]: cluster 2021-03-10T17:26:05.862391+0000 mgr.gibba032.whopeo (mgr.24625) 11239 : cluster [DBG] pgmap v6405: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:08 gibba032 bash[24004]: cluster 2021-03-10T17:26:07.864252+0000 mgr.gibba032.whopeo (mgr.24625) 11240 : cluster [DBG] pgmap v6406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:08 gibba011 bash[30791]: cluster 2021-03-10T17:26:07.864252+0000 mgr.gibba032.whopeo (mgr.24625) 11240 : cluster [DBG] pgmap v6406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:08 gibba011 bash[31774]: cluster 2021-03-10T17:26:07.864252+0000 mgr.gibba032.whopeo (mgr.24625) 11240 : cluster [DBG] pgmap v6406: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:10 gibba032 bash[24004]: cluster 2021-03-10T17:26:09.865507+0000 mgr.gibba032.whopeo (mgr.24625) 11241 : cluster [DBG] pgmap v6407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:10 gibba011 bash[30791]: cluster 2021-03-10T17:26:09.865507+0000 mgr.gibba032.whopeo (mgr.24625) 11241 : cluster [DBG] pgmap v6407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:11.329 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:10 gibba011 bash[31774]: cluster 2021-03-10T17:26:09.865507+0000 mgr.gibba032.whopeo (mgr.24625) 11241 : cluster [DBG] pgmap v6407: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:12 gibba032 bash[24004]: cephadm 2021-03-10T17:26:11.840319+0000 mgr.gibba032.whopeo (mgr.24625) 11242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:12 gibba032 bash[24004]: cluster 2021-03-10T17:26:11.867140+0000 mgr.gibba032.whopeo (mgr.24625) 11243 : cluster [DBG] pgmap v6408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:12 gibba011 bash[30791]: cephadm 2021-03-10T17:26:11.840319+0000 mgr.gibba032.whopeo (mgr.24625) 11242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:12 gibba011 bash[30791]: cluster 2021-03-10T17:26:11.867140+0000 mgr.gibba032.whopeo (mgr.24625) 11243 : cluster [DBG] pgmap v6408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:12 gibba011 bash[31774]: cephadm 2021-03-10T17:26:11.840319+0000 mgr.gibba032.whopeo (mgr.24625) 11242 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:12 gibba011 bash[31774]: cluster 2021-03-10T17:26:11.867140+0000 mgr.gibba032.whopeo (mgr.24625) 11243 : cluster [DBG] pgmap v6408: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:15.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:14 gibba032 bash[24004]: cluster 2021-03-10T17:26:13.868783+0000 mgr.gibba032.whopeo (mgr.24625) 11244 : cluster [DBG] pgmap v6409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:15.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:14 gibba011 bash[30791]: cluster 2021-03-10T17:26:13.868783+0000 mgr.gibba032.whopeo (mgr.24625) 11244 : cluster [DBG] pgmap v6409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:15.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:14 gibba011 bash[31774]: cluster 2021-03-10T17:26:13.868783+0000 mgr.gibba032.whopeo (mgr.24625) 11244 : cluster [DBG] pgmap v6409: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:16 gibba032 bash[24004]: cluster 2021-03-10T17:26:15.870064+0000 mgr.gibba032.whopeo (mgr.24625) 11245 : cluster [DBG] pgmap v6410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:17.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:16 gibba011 bash[30791]: cluster 2021-03-10T17:26:15.870064+0000 mgr.gibba032.whopeo (mgr.24625) 11245 : cluster [DBG] pgmap v6410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:16 gibba011 bash[31774]: cluster 2021-03-10T17:26:15.870064+0000 mgr.gibba032.whopeo (mgr.24625) 11245 : cluster [DBG] pgmap v6410: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:18 gibba032 bash[24004]: cluster 2021-03-10T17:26:17.871923+0000 mgr.gibba032.whopeo (mgr.24625) 11246 : cluster [DBG] pgmap v6411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:18 gibba011 bash[30791]: cluster 2021-03-10T17:26:17.871923+0000 mgr.gibba032.whopeo (mgr.24625) 11246 : cluster [DBG] pgmap v6411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:18 gibba011 bash[31774]: cluster 2021-03-10T17:26:17.871923+0000 mgr.gibba032.whopeo (mgr.24625) 11246 : cluster [DBG] pgmap v6411: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:20 gibba032 bash[24004]: cluster 2021-03-10T17:26:19.873350+0000 mgr.gibba032.whopeo (mgr.24625) 11247 : cluster [DBG] pgmap v6412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:21.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:20 gibba011 bash[30791]: cluster 2021-03-10T17:26:19.873350+0000 mgr.gibba032.whopeo (mgr.24625) 11247 : cluster [DBG] pgmap v6412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:20 gibba011 bash[31774]: cluster 2021-03-10T17:26:19.873350+0000 mgr.gibba032.whopeo (mgr.24625) 11247 : cluster [DBG] pgmap v6412: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:22.507 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:22 gibba032 bash[24004]: cluster 2021-03-10T17:26:21.875005+0000 mgr.gibba032.whopeo (mgr.24625) 11248 : cluster [DBG] pgmap v6413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:22.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:22 gibba011 bash[30791]: cluster 2021-03-10T17:26:21.875005+0000 mgr.gibba032.whopeo (mgr.24625) 11248 : cluster [DBG] pgmap v6413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:22 gibba011 bash[31774]: cluster 2021-03-10T17:26:21.875005+0000 mgr.gibba032.whopeo (mgr.24625) 11248 : cluster [DBG] pgmap v6413: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:24 gibba032 bash[24004]: cluster 2021-03-10T17:26:23.876533+0000 mgr.gibba032.whopeo (mgr.24625) 11249 : cluster [DBG] pgmap v6414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:24 gibba011 bash[30791]: cluster 2021-03-10T17:26:23.876533+0000 mgr.gibba032.whopeo (mgr.24625) 11249 : cluster [DBG] pgmap v6414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:24 gibba011 bash[31774]: cluster 2021-03-10T17:26:23.876533+0000 mgr.gibba032.whopeo (mgr.24625) 11249 : cluster [DBG] pgmap v6414: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:26.592 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:26:26.820 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:26:26.821 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:26:26.821 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:26:26.821 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:26:26.821 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:26:26.822 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:26:26.822 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:26:26.822 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:26:26.823 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:26:26.823 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:26:26.823 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:26:26.823 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:26:26.823 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:26:26.824 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:26:26.824 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:26:26.824 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:26:26.824 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:26:26.824 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:26:26.825 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:26:26.825 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:26:26.825 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:26:27.075 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:26:27.075 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:26:27.075 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:26:27.076 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:27.076 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:26:27.076 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:26:27.076 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:27.077 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:26:27.077 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:26:27.077 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:27.077 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:26:27.077 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:26:27.078 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:26:27.078 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:26:27.078 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:26:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:26 gibba032 bash[24004]: cluster 2021-03-10T17:26:25.877900+0000 mgr.gibba032.whopeo (mgr.24625) 11250 : cluster [DBG] pgmap v6415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:27.407 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:26 gibba011 bash[30791]: cluster 2021-03-10T17:26:25.877900+0000 mgr.gibba032.whopeo (mgr.24625) 11250 : cluster [DBG] pgmap v6415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:27.408 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:26 gibba011 bash[31774]: cluster 2021-03-10T17:26:25.877900+0000 mgr.gibba032.whopeo (mgr.24625) 11250 : cluster [DBG] pgmap v6415: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:27 gibba032 bash[24004]: audit 2021-03-10T17:26:26.581159+0000 mgr.gibba032.whopeo (mgr.24625) 11251 : audit [DBG] from='client.51597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:27 gibba032 bash[24004]: audit 2021-03-10T17:26:26.806059+0000 mgr.gibba032.whopeo (mgr.24625) 11252 : audit [DBG] from='client.61585 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:27 gibba032 bash[24004]: cephadm 2021-03-10T17:26:26.855980+0000 mgr.gibba032.whopeo (mgr.24625) 11253 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:27 gibba032 bash[24004]: audit 2021-03-10T17:26:27.074703+0000 mon.a (mon.0) 3723 : audit [DBG] from='client.? 172.21.2.111:0/969583982' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:28.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[30791]: audit 2021-03-10T17:26:26.581159+0000 mgr.gibba032.whopeo (mgr.24625) 11251 : audit [DBG] from='client.51597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[30791]: audit 2021-03-10T17:26:26.806059+0000 mgr.gibba032.whopeo (mgr.24625) 11252 : audit [DBG] from='client.61585 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[30791]: cephadm 2021-03-10T17:26:26.855980+0000 mgr.gibba032.whopeo (mgr.24625) 11253 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[30791]: audit 2021-03-10T17:26:27.074703+0000 mon.a (mon.0) 3723 : audit [DBG] from='client.? 172.21.2.111:0/969583982' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:28.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[31774]: audit 2021-03-10T17:26:26.581159+0000 mgr.gibba032.whopeo (mgr.24625) 11251 : audit [DBG] from='client.51597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[31774]: audit 2021-03-10T17:26:26.806059+0000 mgr.gibba032.whopeo (mgr.24625) 11252 : audit [DBG] from='client.61585 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[31774]: cephadm 2021-03-10T17:26:26.855980+0000 mgr.gibba032.whopeo (mgr.24625) 11253 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:27 gibba011 bash[31774]: audit 2021-03-10T17:26:27.074703+0000 mon.a (mon.0) 3723 : audit [DBG] from='client.? 172.21.2.111:0/969583982' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:28 gibba032 bash[24004]: cluster 2021-03-10T17:26:27.879556+0000 mgr.gibba032.whopeo (mgr.24625) 11254 : cluster [DBG] pgmap v6416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:28 gibba011 bash[30791]: cluster 2021-03-10T17:26:27.879556+0000 mgr.gibba032.whopeo (mgr.24625) 11254 : cluster [DBG] pgmap v6416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:28 gibba011 bash[31774]: cluster 2021-03-10T17:26:27.879556+0000 mgr.gibba032.whopeo (mgr.24625) 11254 : cluster [DBG] pgmap v6416: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:31.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:30 gibba032 bash[24004]: cluster 2021-03-10T17:26:29.881049+0000 mgr.gibba032.whopeo (mgr.24625) 11255 : cluster [DBG] pgmap v6417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:30 gibba011 bash[30791]: cluster 2021-03-10T17:26:29.881049+0000 mgr.gibba032.whopeo (mgr.24625) 11255 : cluster [DBG] pgmap v6417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:30 gibba011 bash[31774]: cluster 2021-03-10T17:26:29.881049+0000 mgr.gibba032.whopeo (mgr.24625) 11255 : cluster [DBG] pgmap v6417: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:32 gibba032 bash[24004]: cluster 2021-03-10T17:26:31.882511+0000 mgr.gibba032.whopeo (mgr.24625) 11256 : cluster [DBG] pgmap v6418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:32 gibba011 bash[30791]: cluster 2021-03-10T17:26:31.882511+0000 mgr.gibba032.whopeo (mgr.24625) 11256 : cluster [DBG] pgmap v6418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:32 gibba011 bash[31774]: cluster 2021-03-10T17:26:31.882511+0000 mgr.gibba032.whopeo (mgr.24625) 11256 : cluster [DBG] pgmap v6418: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:34 gibba032 bash[24004]: cluster 2021-03-10T17:26:33.884296+0000 mgr.gibba032.whopeo (mgr.24625) 11257 : cluster [DBG] pgmap v6419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:34 gibba011 bash[30791]: cluster 2021-03-10T17:26:33.884296+0000 mgr.gibba032.whopeo (mgr.24625) 11257 : cluster [DBG] pgmap v6419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:34 gibba011 bash[31774]: cluster 2021-03-10T17:26:33.884296+0000 mgr.gibba032.whopeo (mgr.24625) 11257 : cluster [DBG] pgmap v6419: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:37.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:36 gibba032 bash[24004]: cluster 2021-03-10T17:26:35.885700+0000 mgr.gibba032.whopeo (mgr.24625) 11258 : cluster [DBG] pgmap v6420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:37.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:36 gibba011 bash[30791]: cluster 2021-03-10T17:26:35.885700+0000 mgr.gibba032.whopeo (mgr.24625) 11258 : cluster [DBG] pgmap v6420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:36 gibba011 bash[31774]: cluster 2021-03-10T17:26:35.885700+0000 mgr.gibba032.whopeo (mgr.24625) 11258 : cluster [DBG] pgmap v6420: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:39.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:38 gibba032 bash[24004]: cluster 2021-03-10T17:26:37.887436+0000 mgr.gibba032.whopeo (mgr.24625) 11259 : cluster [DBG] pgmap v6421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:38 gibba011 bash[30791]: cluster 2021-03-10T17:26:37.887436+0000 mgr.gibba032.whopeo (mgr.24625) 11259 : cluster [DBG] pgmap v6421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:38 gibba011 bash[31774]: cluster 2021-03-10T17:26:37.887436+0000 mgr.gibba032.whopeo (mgr.24625) 11259 : cluster [DBG] pgmap v6421: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:41.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:40 gibba032 bash[24004]: cluster 2021-03-10T17:26:39.888827+0000 mgr.gibba032.whopeo (mgr.24625) 11260 : cluster [DBG] pgmap v6422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:40 gibba011 bash[30791]: cluster 2021-03-10T17:26:39.888827+0000 mgr.gibba032.whopeo (mgr.24625) 11260 : cluster [DBG] pgmap v6422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:40 gibba011 bash[31774]: cluster 2021-03-10T17:26:39.888827+0000 mgr.gibba032.whopeo (mgr.24625) 11260 : cluster [DBG] pgmap v6422: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:42.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: cephadm 2021-03-10T17:26:41.872575+0000 mgr.gibba032.whopeo (mgr.24625) 11261 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:26:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: cephadm 2021-03-10T17:26:41.874117+0000 mgr.gibba032.whopeo (mgr.24625) 11262 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:26:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: cluster 2021-03-10T17:26:41.889376+0000 mgr.gibba032.whopeo (mgr.24625) 11263 : cluster [DBG] pgmap v6423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.088660+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:26:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.092019+0000 mon.a (mon.0) 3725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:26:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.093969+0000 mon.a (mon.0) 3726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.095565+0000 mon.a (mon.0) 3727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.096890+0000 mon.a (mon.0) 3728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.100838+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:42 gibba032 bash[24004]: audit 2021-03-10T17:26:42.102937+0000 mon.a (mon.0) 3730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: cephadm 2021-03-10T17:26:41.872575+0000 mgr.gibba032.whopeo (mgr.24625) 11261 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:26:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: cephadm 2021-03-10T17:26:41.874117+0000 mgr.gibba032.whopeo (mgr.24625) 11262 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:26:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: cluster 2021-03-10T17:26:41.889376+0000 mgr.gibba032.whopeo (mgr.24625) 11263 : cluster [DBG] pgmap v6423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.088660+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:26:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.092019+0000 mon.a (mon.0) 3725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:26:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.093969+0000 mon.a (mon.0) 3726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.095565+0000 mon.a (mon.0) 3727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.096890+0000 mon.a (mon.0) 3728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.100838+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[30791]: audit 2021-03-10T17:26:42.102937+0000 mon.a (mon.0) 3730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: cephadm 2021-03-10T17:26:41.872575+0000 mgr.gibba032.whopeo (mgr.24625) 11261 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:26:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: cephadm 2021-03-10T17:26:41.874117+0000 mgr.gibba032.whopeo (mgr.24625) 11262 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:26:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: cluster 2021-03-10T17:26:41.889376+0000 mgr.gibba032.whopeo (mgr.24625) 11263 : cluster [DBG] pgmap v6423: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.088660+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:26:42.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.092019+0000 mon.a (mon.0) 3725 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:26:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.093969+0000 mon.a (mon.0) 3726 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.095565+0000 mon.a (mon.0) 3727 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.096890+0000 mon.a (mon.0) 3728 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.100838+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:42.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:42 gibba011 bash[31774]: audit 2021-03-10T17:26:42.102937+0000 mon.a (mon.0) 3730 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.089388+0000 mgr.gibba032.whopeo (mgr.24625) 11264 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:26:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.091528+0000 mgr.gibba032.whopeo (mgr.24625) 11265 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:26:43.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.093252+0000 mgr.gibba032.whopeo (mgr.24625) 11266 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:26:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.093291+0000 mgr.gibba032.whopeo (mgr.24625) 11267 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:26:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.094669+0000 mgr.gibba032.whopeo (mgr.24625) 11268 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:26:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.094708+0000 mgr.gibba032.whopeo (mgr.24625) 11269 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:26:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.096339+0000 mgr.gibba032.whopeo (mgr.24625) 11270 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:26:43.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.098222+0000 mgr.gibba032.whopeo (mgr.24625) 11271 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:26:43.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.098261+0000 mgr.gibba032.whopeo (mgr.24625) 11272 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:26:43.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.100216+0000 mgr.gibba032.whopeo (mgr.24625) 11273 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:26:43.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.100538+0000 mgr.gibba032.whopeo (mgr.24625) 11274 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:26:43.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.102071+0000 mgr.gibba032.whopeo (mgr.24625) 11275 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:26:43.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.102110+0000 mgr.gibba032.whopeo (mgr.24625) 11276 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:26:43.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[30791]: cephadm 2021-03-10T17:26:42.102263+0000 mgr.gibba032.whopeo (mgr.24625) 11277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:43.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.089388+0000 mgr.gibba032.whopeo (mgr.24625) 11264 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:26:43.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.091528+0000 mgr.gibba032.whopeo (mgr.24625) 11265 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:26:43.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.093252+0000 mgr.gibba032.whopeo (mgr.24625) 11266 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:26:43.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.093291+0000 mgr.gibba032.whopeo (mgr.24625) 11267 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:26:43.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.094669+0000 mgr.gibba032.whopeo (mgr.24625) 11268 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:26:43.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.094708+0000 mgr.gibba032.whopeo (mgr.24625) 11269 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:26:43.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.096339+0000 mgr.gibba032.whopeo (mgr.24625) 11270 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:26:43.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.098222+0000 mgr.gibba032.whopeo (mgr.24625) 11271 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:26:43.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.098261+0000 mgr.gibba032.whopeo (mgr.24625) 11272 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:26:43.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.100216+0000 mgr.gibba032.whopeo (mgr.24625) 11273 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:26:43.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.100538+0000 mgr.gibba032.whopeo (mgr.24625) 11274 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:26:43.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.102071+0000 mgr.gibba032.whopeo (mgr.24625) 11275 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:26:43.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.102110+0000 mgr.gibba032.whopeo (mgr.24625) 11276 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:26:43.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:43 gibba011 bash[31774]: cephadm 2021-03-10T17:26:42.102263+0000 mgr.gibba032.whopeo (mgr.24625) 11277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:43.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.089388+0000 mgr.gibba032.whopeo (mgr.24625) 11264 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:26:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.091528+0000 mgr.gibba032.whopeo (mgr.24625) 11265 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:26:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.093252+0000 mgr.gibba032.whopeo (mgr.24625) 11266 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:26:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.093291+0000 mgr.gibba032.whopeo (mgr.24625) 11267 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:26:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.094669+0000 mgr.gibba032.whopeo (mgr.24625) 11268 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:26:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.094708+0000 mgr.gibba032.whopeo (mgr.24625) 11269 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:26:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.096339+0000 mgr.gibba032.whopeo (mgr.24625) 11270 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:26:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.098222+0000 mgr.gibba032.whopeo (mgr.24625) 11271 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:26:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.098261+0000 mgr.gibba032.whopeo (mgr.24625) 11272 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:26:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.100216+0000 mgr.gibba032.whopeo (mgr.24625) 11273 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:26:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.100538+0000 mgr.gibba032.whopeo (mgr.24625) 11274 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:26:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.102071+0000 mgr.gibba032.whopeo (mgr.24625) 11275 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:26:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.102110+0000 mgr.gibba032.whopeo (mgr.24625) 11276 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:26:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:43 gibba032 bash[24004]: cephadm 2021-03-10T17:26:42.102263+0000 mgr.gibba032.whopeo (mgr.24625) 11277 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:44.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:44 gibba011 bash[30791]: cluster 2021-03-10T17:26:43.890755+0000 mgr.gibba032.whopeo (mgr.24625) 11278 : cluster [DBG] pgmap v6424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:44.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:44 gibba011 bash[31774]: cluster 2021-03-10T17:26:43.890755+0000 mgr.gibba032.whopeo (mgr.24625) 11278 : cluster [DBG] pgmap v6424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:44 gibba032 bash[24004]: cluster 2021-03-10T17:26:43.890755+0000 mgr.gibba032.whopeo (mgr.24625) 11278 : cluster [DBG] pgmap v6424: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:46 gibba032 bash[24004]: cluster 2021-03-10T17:26:45.892230+0000 mgr.gibba032.whopeo (mgr.24625) 11279 : cluster [DBG] pgmap v6425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:46 gibba011 bash[30791]: cluster 2021-03-10T17:26:45.892230+0000 mgr.gibba032.whopeo (mgr.24625) 11279 : cluster [DBG] pgmap v6425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:46 gibba011 bash[31774]: cluster 2021-03-10T17:26:45.892230+0000 mgr.gibba032.whopeo (mgr.24625) 11279 : cluster [DBG] pgmap v6425: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:49.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:48 gibba032 bash[24004]: cluster 2021-03-10T17:26:47.893960+0000 mgr.gibba032.whopeo (mgr.24625) 11280 : cluster [DBG] pgmap v6426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:48 gibba011 bash[30791]: cluster 2021-03-10T17:26:47.893960+0000 mgr.gibba032.whopeo (mgr.24625) 11280 : cluster [DBG] pgmap v6426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:48 gibba011 bash[31774]: cluster 2021-03-10T17:26:47.893960+0000 mgr.gibba032.whopeo (mgr.24625) 11280 : cluster [DBG] pgmap v6426: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:51.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:50 gibba032 bash[24004]: cluster 2021-03-10T17:26:49.895393+0000 mgr.gibba032.whopeo (mgr.24625) 11281 : cluster [DBG] pgmap v6427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:51.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:50 gibba011 bash[30791]: cluster 2021-03-10T17:26:49.895393+0000 mgr.gibba032.whopeo (mgr.24625) 11281 : cluster [DBG] pgmap v6427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:50 gibba011 bash[31774]: cluster 2021-03-10T17:26:49.895393+0000 mgr.gibba032.whopeo (mgr.24625) 11281 : cluster [DBG] pgmap v6427: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:52 gibba011 bash[30791]: cluster 2021-03-10T17:26:51.896952+0000 mgr.gibba032.whopeo (mgr.24625) 11282 : cluster [DBG] pgmap v6428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:52 gibba011 bash[31774]: cluster 2021-03-10T17:26:51.896952+0000 mgr.gibba032.whopeo (mgr.24625) 11282 : cluster [DBG] pgmap v6428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:52 gibba032 bash[24004]: cluster 2021-03-10T17:26:51.896952+0000 mgr.gibba032.whopeo (mgr.24625) 11282 : cluster [DBG] pgmap v6428: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:55.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:54 gibba032 bash[24004]: cluster 2021-03-10T17:26:53.898530+0000 mgr.gibba032.whopeo (mgr.24625) 11283 : cluster [DBG] pgmap v6429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:54 gibba011 bash[30791]: cluster 2021-03-10T17:26:53.898530+0000 mgr.gibba032.whopeo (mgr.24625) 11283 : cluster [DBG] pgmap v6429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:54 gibba011 bash[31774]: cluster 2021-03-10T17:26:53.898530+0000 mgr.gibba032.whopeo (mgr.24625) 11283 : cluster [DBG] pgmap v6429: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:57.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:56 gibba032 bash[24004]: cluster 2021-03-10T17:26:55.899893+0000 mgr.gibba032.whopeo (mgr.24625) 11284 : cluster [DBG] pgmap v6430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:57.325 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:26:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:56 gibba011 bash[30791]: cluster 2021-03-10T17:26:55.899893+0000 mgr.gibba032.whopeo (mgr.24625) 11284 : cluster [DBG] pgmap v6430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:56 gibba011 bash[31774]: cluster 2021-03-10T17:26:55.899893+0000 mgr.gibba032.whopeo (mgr.24625) 11284 : cluster [DBG] pgmap v6430: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:57.551 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:26:57.551 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:26:57.552 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:26:57.552 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:26:57.552 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:26:57.552 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:26:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:26:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:26:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:26:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:26:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:26:57.554 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:26:57.554 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:26:57.554 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:26:57.554 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:26:57.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:26:57.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:26:57.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:26:57.555 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:26:57.555 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:26:57.556 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:26:57.800 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:26:57.903 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:26:57.903 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:26:57.904 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:57.904 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:26:57.905 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:26:57.905 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:57.905 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:26:57.906 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:26:57.906 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:26:57.906 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:26:57.907 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:26:57.907 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:26:57.907 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:26:57.908 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:26:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.489483+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:26:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.489675+0000 mon.a (mon.0) 3732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:26:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.680204+0000 mon.a (mon.0) 3733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:26:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.680406+0000 mon.a (mon.0) 3734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.680592+0000 mon.a (mon.0) 3735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:26:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.680739+0000 mon.a (mon.0) 3736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:57 gibba032 bash[24004]: audit 2021-03-10T17:26:57.798864+0000 mon.c (mon.1) 217 : audit [DBG] from='client.? 172.21.2.111:0/3485074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.489483+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:26:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.489675+0000 mon.a (mon.0) 3732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:26:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.680204+0000 mon.a (mon.0) 3733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:26:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.680406+0000 mon.a (mon.0) 3734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.680592+0000 mon.a (mon.0) 3735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:26:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.680739+0000 mon.a (mon.0) 3736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[30791]: audit 2021-03-10T17:26:57.798864+0000 mon.c (mon.1) 217 : audit [DBG] from='client.? 172.21.2.111:0/3485074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.489483+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.489675+0000 mon.a (mon.0) 3732 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.680204+0000 mon.a (mon.0) 3733 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.680406+0000 mon.a (mon.0) 3734 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.680592+0000 mon.a (mon.0) 3735 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:26:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.680739+0000 mon.a (mon.0) 3736 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:26:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:57 gibba011 bash[31774]: audit 2021-03-10T17:26:57.798864+0000 mon.c (mon.1) 217 : audit [DBG] from='client.? 172.21.2.111:0/3485074611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:26:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:58 gibba032 bash[24004]: cephadm 2021-03-10T17:26:57.107665+0000 mgr.gibba032.whopeo (mgr.24625) 11285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:59.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:58 gibba032 bash[24004]: audit 2021-03-10T17:26:57.313377+0000 mgr.gibba032.whopeo (mgr.24625) 11286 : audit [DBG] from='client.61597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:58 gibba032 bash[24004]: audit 2021-03-10T17:26:57.537895+0000 mgr.gibba032.whopeo (mgr.24625) 11287 : audit [DBG] from='client.51621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:26:58 gibba032 bash[24004]: cluster 2021-03-10T17:26:57.901683+0000 mgr.gibba032.whopeo (mgr.24625) 11288 : cluster [DBG] pgmap v6431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[30791]: cephadm 2021-03-10T17:26:57.107665+0000 mgr.gibba032.whopeo (mgr.24625) 11285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[30791]: audit 2021-03-10T17:26:57.313377+0000 mgr.gibba032.whopeo (mgr.24625) 11286 : audit [DBG] from='client.61597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[30791]: audit 2021-03-10T17:26:57.537895+0000 mgr.gibba032.whopeo (mgr.24625) 11287 : audit [DBG] from='client.51621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[30791]: cluster 2021-03-10T17:26:57.901683+0000 mgr.gibba032.whopeo (mgr.24625) 11288 : cluster [DBG] pgmap v6431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:26:59.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[31774]: cephadm 2021-03-10T17:26:57.107665+0000 mgr.gibba032.whopeo (mgr.24625) 11285 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:26:59.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[31774]: audit 2021-03-10T17:26:57.313377+0000 mgr.gibba032.whopeo (mgr.24625) 11286 : audit [DBG] from='client.61597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[31774]: audit 2021-03-10T17:26:57.537895+0000 mgr.gibba032.whopeo (mgr.24625) 11287 : audit [DBG] from='client.51621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:26:59.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:26:58 gibba011 bash[31774]: cluster 2021-03-10T17:26:57.901683+0000 mgr.gibba032.whopeo (mgr.24625) 11288 : cluster [DBG] pgmap v6431: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:01.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:00 gibba032 bash[24004]: cluster 2021-03-10T17:26:59.903203+0000 mgr.gibba032.whopeo (mgr.24625) 11289 : cluster [DBG] pgmap v6432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:00 gibba011 bash[30791]: cluster 2021-03-10T17:26:59.903203+0000 mgr.gibba032.whopeo (mgr.24625) 11289 : cluster [DBG] pgmap v6432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:00 gibba011 bash[31774]: cluster 2021-03-10T17:26:59.903203+0000 mgr.gibba032.whopeo (mgr.24625) 11289 : cluster [DBG] pgmap v6432: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:02 gibba011 bash[30791]: cluster 2021-03-10T17:27:01.904796+0000 mgr.gibba032.whopeo (mgr.24625) 11290 : cluster [DBG] pgmap v6433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:02 gibba011 bash[31774]: cluster 2021-03-10T17:27:01.904796+0000 mgr.gibba032.whopeo (mgr.24625) 11290 : cluster [DBG] pgmap v6433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:02 gibba032 bash[24004]: cluster 2021-03-10T17:27:01.904796+0000 mgr.gibba032.whopeo (mgr.24625) 11290 : cluster [DBG] pgmap v6433: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:05.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:04 gibba032 bash[24004]: cluster 2021-03-10T17:27:03.906460+0000 mgr.gibba032.whopeo (mgr.24625) 11291 : cluster [DBG] pgmap v6434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:04 gibba011 bash[30791]: cluster 2021-03-10T17:27:03.906460+0000 mgr.gibba032.whopeo (mgr.24625) 11291 : cluster [DBG] pgmap v6434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:04 gibba011 bash[31774]: cluster 2021-03-10T17:27:03.906460+0000 mgr.gibba032.whopeo (mgr.24625) 11291 : cluster [DBG] pgmap v6434: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:07.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:06 gibba032 bash[24004]: cluster 2021-03-10T17:27:05.907807+0000 mgr.gibba032.whopeo (mgr.24625) 11292 : cluster [DBG] pgmap v6435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:07.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:06 gibba011 bash[30791]: cluster 2021-03-10T17:27:05.907807+0000 mgr.gibba032.whopeo (mgr.24625) 11292 : cluster [DBG] pgmap v6435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:06 gibba011 bash[31774]: cluster 2021-03-10T17:27:05.907807+0000 mgr.gibba032.whopeo (mgr.24625) 11292 : cluster [DBG] pgmap v6435: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:09.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:08 gibba032 bash[24004]: cluster 2021-03-10T17:27:07.909607+0000 mgr.gibba032.whopeo (mgr.24625) 11293 : cluster [DBG] pgmap v6436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:08 gibba011 bash[30791]: cluster 2021-03-10T17:27:07.909607+0000 mgr.gibba032.whopeo (mgr.24625) 11293 : cluster [DBG] pgmap v6436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:08 gibba011 bash[31774]: cluster 2021-03-10T17:27:07.909607+0000 mgr.gibba032.whopeo (mgr.24625) 11293 : cluster [DBG] pgmap v6436: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:11.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:10 gibba032 bash[24004]: cluster 2021-03-10T17:27:09.911119+0000 mgr.gibba032.whopeo (mgr.24625) 11294 : cluster [DBG] pgmap v6437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:11.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:10 gibba011 bash[30791]: cluster 2021-03-10T17:27:09.911119+0000 mgr.gibba032.whopeo (mgr.24625) 11294 : cluster [DBG] pgmap v6437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:10 gibba011 bash[31774]: cluster 2021-03-10T17:27:09.911119+0000 mgr.gibba032.whopeo (mgr.24625) 11294 : cluster [DBG] pgmap v6437: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:12.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:12 gibba032 bash[24004]: cluster 2021-03-10T17:27:11.912622+0000 mgr.gibba032.whopeo (mgr.24625) 11295 : cluster [DBG] pgmap v6438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:12.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:12 gibba011 bash[30791]: cluster 2021-03-10T17:27:11.912622+0000 mgr.gibba032.whopeo (mgr.24625) 11295 : cluster [DBG] pgmap v6438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:12.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:12 gibba011 bash[31774]: cluster 2021-03-10T17:27:11.912622+0000 mgr.gibba032.whopeo (mgr.24625) 11295 : cluster [DBG] pgmap v6438: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:13.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:13 gibba011 bash[30791]: cephadm 2021-03-10T17:27:12.123690+0000 mgr.gibba032.whopeo (mgr.24625) 11296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:13.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:13 gibba011 bash[31774]: cephadm 2021-03-10T17:27:12.123690+0000 mgr.gibba032.whopeo (mgr.24625) 11296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:13 gibba032 bash[24004]: cephadm 2021-03-10T17:27:12.123690+0000 mgr.gibba032.whopeo (mgr.24625) 11296 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:14.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:14 gibba011 bash[30791]: cluster 2021-03-10T17:27:13.914271+0000 mgr.gibba032.whopeo (mgr.24625) 11297 : cluster [DBG] pgmap v6439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:14 gibba011 bash[31774]: cluster 2021-03-10T17:27:13.914271+0000 mgr.gibba032.whopeo (mgr.24625) 11297 : cluster [DBG] pgmap v6439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:14 gibba032 bash[24004]: cluster 2021-03-10T17:27:13.914271+0000 mgr.gibba032.whopeo (mgr.24625) 11297 : cluster [DBG] pgmap v6439: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:17.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:16 gibba032 bash[24004]: cluster 2021-03-10T17:27:15.915911+0000 mgr.gibba032.whopeo (mgr.24625) 11298 : cluster [DBG] pgmap v6440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:16 gibba011 bash[30791]: cluster 2021-03-10T17:27:15.915911+0000 mgr.gibba032.whopeo (mgr.24625) 11298 : cluster [DBG] pgmap v6440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:16 gibba011 bash[31774]: cluster 2021-03-10T17:27:15.915911+0000 mgr.gibba032.whopeo (mgr.24625) 11298 : cluster [DBG] pgmap v6440: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:19.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:18 gibba032 bash[24004]: cluster 2021-03-10T17:27:17.917627+0000 mgr.gibba032.whopeo (mgr.24625) 11299 : cluster [DBG] pgmap v6441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:18 gibba011 bash[30791]: cluster 2021-03-10T17:27:17.917627+0000 mgr.gibba032.whopeo (mgr.24625) 11299 : cluster [DBG] pgmap v6441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:18 gibba011 bash[31774]: cluster 2021-03-10T17:27:17.917627+0000 mgr.gibba032.whopeo (mgr.24625) 11299 : cluster [DBG] pgmap v6441: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:21.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:20 gibba032 bash[24004]: cluster 2021-03-10T17:27:19.919146+0000 mgr.gibba032.whopeo (mgr.24625) 11300 : cluster [DBG] pgmap v6442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:21.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:20 gibba011 bash[30791]: cluster 2021-03-10T17:27:19.919146+0000 mgr.gibba032.whopeo (mgr.24625) 11300 : cluster [DBG] pgmap v6442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:21.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:21 gibba011 bash[31774]: cluster 2021-03-10T17:27:19.919146+0000 mgr.gibba032.whopeo (mgr.24625) 11300 : cluster [DBG] pgmap v6442: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:22 gibba011 bash[30791]: cluster 2021-03-10T17:27:21.920987+0000 mgr.gibba032.whopeo (mgr.24625) 11301 : cluster [DBG] pgmap v6443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:22.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:22 gibba011 bash[31774]: cluster 2021-03-10T17:27:21.920987+0000 mgr.gibba032.whopeo (mgr.24625) 11301 : cluster [DBG] pgmap v6443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:22 gibba032 bash[24004]: cluster 2021-03-10T17:27:21.920987+0000 mgr.gibba032.whopeo (mgr.24625) 11301 : cluster [DBG] pgmap v6443: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:25.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:24 gibba032 bash[24004]: cluster 2021-03-10T17:27:23.922373+0000 mgr.gibba032.whopeo (mgr.24625) 11302 : cluster [DBG] pgmap v6444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:25.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:24 gibba011 bash[30791]: cluster 2021-03-10T17:27:23.922373+0000 mgr.gibba032.whopeo (mgr.24625) 11302 : cluster [DBG] pgmap v6444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:24 gibba011 bash[31774]: cluster 2021-03-10T17:27:23.922373+0000 mgr.gibba032.whopeo (mgr.24625) 11302 : cluster [DBG] pgmap v6444: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:27.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:26 gibba032 bash[24004]: cluster 2021-03-10T17:27:25.923704+0000 mgr.gibba032.whopeo (mgr.24625) 11303 : cluster [DBG] pgmap v6445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:26 gibba011 bash[30791]: cluster 2021-03-10T17:27:25.923704+0000 mgr.gibba032.whopeo (mgr.24625) 11303 : cluster [DBG] pgmap v6445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:26 gibba011 bash[31774]: cluster 2021-03-10T17:27:25.923704+0000 mgr.gibba032.whopeo (mgr.24625) 11303 : cluster [DBG] pgmap v6445: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:28.047 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:27:28.281 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:27:28.281 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:27:28.282 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:27:28.282 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:27:28.282 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:27:28.282 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:27:28.283 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:27:28.283 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:27:28.283 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:27:28.283 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:27:28.283 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:27:28.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:27:28.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:27:28.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:27:28.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:27:28.284 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:27:28.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:27:28.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:27:28.285 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:27:28.285 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:27:28.286 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:27:28.529 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:27:28.530 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:27:28.530 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:27:28.530 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:28.530 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:27:28.531 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:27:28.531 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:28.531 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:27:28.531 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:27:28.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:28.532 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:27:28.532 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:27:28.532 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:27:28.532 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:27:28.533 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:27:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:29 gibba032 bash[24004]: cephadm 2021-03-10T17:27:27.139740+0000 mgr.gibba032.whopeo (mgr.24625) 11304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:29 gibba032 bash[24004]: cluster 2021-03-10T17:27:27.925424+0000 mgr.gibba032.whopeo (mgr.24625) 11305 : cluster [DBG] pgmap v6446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:29.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:29 gibba032 bash[24004]: audit 2021-03-10T17:27:28.035443+0000 mgr.gibba032.whopeo (mgr.24625) 11306 : audit [DBG] from='client.61615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:29.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:29 gibba032 bash[24004]: audit 2021-03-10T17:27:28.528952+0000 mon.c (mon.1) 218 : audit [DBG] from='client.? 172.21.2.111:0/3906530872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:29.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[30791]: cephadm 2021-03-10T17:27:27.139740+0000 mgr.gibba032.whopeo (mgr.24625) 11304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:29.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[30791]: cluster 2021-03-10T17:27:27.925424+0000 mgr.gibba032.whopeo (mgr.24625) 11305 : cluster [DBG] pgmap v6446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:29.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[30791]: audit 2021-03-10T17:27:28.035443+0000 mgr.gibba032.whopeo (mgr.24625) 11306 : audit [DBG] from='client.61615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:29.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[30791]: audit 2021-03-10T17:27:28.528952+0000 mon.c (mon.1) 218 : audit [DBG] from='client.? 172.21.2.111:0/3906530872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:29.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[31774]: cephadm 2021-03-10T17:27:27.139740+0000 mgr.gibba032.whopeo (mgr.24625) 11304 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:29.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[31774]: cluster 2021-03-10T17:27:27.925424+0000 mgr.gibba032.whopeo (mgr.24625) 11305 : cluster [DBG] pgmap v6446: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:29.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[31774]: audit 2021-03-10T17:27:28.035443+0000 mgr.gibba032.whopeo (mgr.24625) 11306 : audit [DBG] from='client.61615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:29.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:29 gibba011 bash[31774]: audit 2021-03-10T17:27:28.528952+0000 mon.c (mon.1) 218 : audit [DBG] from='client.? 172.21.2.111:0/3906530872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:30.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:30 gibba011 bash[30791]: audit 2021-03-10T17:27:28.266730+0000 mgr.gibba032.whopeo (mgr.24625) 11307 : audit [DBG] from='client.61621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:30.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:30 gibba011 bash[31774]: audit 2021-03-10T17:27:28.266730+0000 mgr.gibba032.whopeo (mgr.24625) 11307 : audit [DBG] from='client.61621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:30 gibba032 bash[24004]: audit 2021-03-10T17:27:28.266730+0000 mgr.gibba032.whopeo (mgr.24625) 11307 : audit [DBG] from='client.61621 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:27:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:31 gibba011 bash[30791]: cluster 2021-03-10T17:27:29.926721+0000 mgr.gibba032.whopeo (mgr.24625) 11308 : cluster [DBG] pgmap v6447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:31 gibba011 bash[31774]: cluster 2021-03-10T17:27:29.926721+0000 mgr.gibba032.whopeo (mgr.24625) 11308 : cluster [DBG] pgmap v6447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:31 gibba032 bash[24004]: cluster 2021-03-10T17:27:29.926721+0000 mgr.gibba032.whopeo (mgr.24625) 11308 : cluster [DBG] pgmap v6447: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:32.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:32 gibba011 bash[30791]: cluster 2021-03-10T17:27:31.928407+0000 mgr.gibba032.whopeo (mgr.24625) 11309 : cluster [DBG] pgmap v6448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:32.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:32 gibba011 bash[31774]: cluster 2021-03-10T17:27:31.928407+0000 mgr.gibba032.whopeo (mgr.24625) 11309 : cluster [DBG] pgmap v6448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:32 gibba032 bash[24004]: cluster 2021-03-10T17:27:31.928407+0000 mgr.gibba032.whopeo (mgr.24625) 11309 : cluster [DBG] pgmap v6448: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:35.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:34 gibba032 bash[24004]: cluster 2021-03-10T17:27:33.930022+0000 mgr.gibba032.whopeo (mgr.24625) 11310 : cluster [DBG] pgmap v6449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:34 gibba011 bash[30791]: cluster 2021-03-10T17:27:33.930022+0000 mgr.gibba032.whopeo (mgr.24625) 11310 : cluster [DBG] pgmap v6449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:34 gibba011 bash[31774]: cluster 2021-03-10T17:27:33.930022+0000 mgr.gibba032.whopeo (mgr.24625) 11310 : cluster [DBG] pgmap v6449: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:37.257 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:37 gibba032 bash[24004]: cluster 2021-03-10T17:27:35.931471+0000 mgr.gibba032.whopeo (mgr.24625) 11311 : cluster [DBG] pgmap v6450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:37 gibba011 bash[30791]: cluster 2021-03-10T17:27:35.931471+0000 mgr.gibba032.whopeo (mgr.24625) 11311 : cluster [DBG] pgmap v6450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:37 gibba011 bash[31774]: cluster 2021-03-10T17:27:35.931471+0000 mgr.gibba032.whopeo (mgr.24625) 11311 : cluster [DBG] pgmap v6450: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:39.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:39 gibba011 bash[30791]: cluster 2021-03-10T17:27:37.933247+0000 mgr.gibba032.whopeo (mgr.24625) 11312 : cluster [DBG] pgmap v6451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:39 gibba011 bash[31774]: cluster 2021-03-10T17:27:37.933247+0000 mgr.gibba032.whopeo (mgr.24625) 11312 : cluster [DBG] pgmap v6451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:39.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:39 gibba032 bash[24004]: cluster 2021-03-10T17:27:37.933247+0000 mgr.gibba032.whopeo (mgr.24625) 11312 : cluster [DBG] pgmap v6451: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:41 gibba011 bash[30791]: cluster 2021-03-10T17:27:39.934818+0000 mgr.gibba032.whopeo (mgr.24625) 11313 : cluster [DBG] pgmap v6452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:41 gibba011 bash[31774]: cluster 2021-03-10T17:27:39.934818+0000 mgr.gibba032.whopeo (mgr.24625) 11313 : cluster [DBG] pgmap v6452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:41 gibba032 bash[24004]: cluster 2021-03-10T17:27:39.934818+0000 mgr.gibba032.whopeo (mgr.24625) 11313 : cluster [DBG] pgmap v6452: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:42 gibba011 bash[30791]: cluster 2021-03-10T17:27:41.936322+0000 mgr.gibba032.whopeo (mgr.24625) 11314 : cluster [DBG] pgmap v6453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:42 gibba011 bash[31774]: cluster 2021-03-10T17:27:41.936322+0000 mgr.gibba032.whopeo (mgr.24625) 11314 : cluster [DBG] pgmap v6453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:42 gibba032 bash[24004]: cluster 2021-03-10T17:27:41.936322+0000 mgr.gibba032.whopeo (mgr.24625) 11314 : cluster [DBG] pgmap v6453: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.156339+0000 mgr.gibba032.whopeo (mgr.24625) 11315 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:27:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.157875+0000 mgr.gibba032.whopeo (mgr.24625) 11316 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:27:43.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.372686+0000 mon.a (mon.0) 3737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:27:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.373289+0000 mgr.gibba032.whopeo (mgr.24625) 11317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:27:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.375607+0000 mgr.gibba032.whopeo (mgr.24625) 11318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:27:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.375973+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:27:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.377631+0000 mgr.gibba032.whopeo (mgr.24625) 11319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:27:43.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.377670+0000 mgr.gibba032.whopeo (mgr.24625) 11320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:27:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.378025+0000 mon.a (mon.0) 3739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.379581+0000 mgr.gibba032.whopeo (mgr.24625) 11321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:27:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.379621+0000 mgr.gibba032.whopeo (mgr.24625) 11322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:27:43.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.380078+0000 mon.a (mon.0) 3740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.381416+0000 mgr.gibba032.whopeo (mgr.24625) 11323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:27:43.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.382027+0000 mon.a (mon.0) 3741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.383240+0000 mgr.gibba032.whopeo (mgr.24625) 11324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:27:43.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.383280+0000 mgr.gibba032.whopeo (mgr.24625) 11325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:27:43.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.385050+0000 mgr.gibba032.whopeo (mgr.24625) 11326 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:27:43.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.385371+0000 mgr.gibba032.whopeo (mgr.24625) 11327 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:27:43.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.385731+0000 mon.a (mon.0) 3742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.387148+0000 mgr.gibba032.whopeo (mgr.24625) 11328 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:27:43.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.387186+0000 mgr.gibba032.whopeo (mgr.24625) 11329 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:27:43.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: cephadm 2021-03-10T17:27:42.387293+0000 mgr.gibba032.whopeo (mgr.24625) 11330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:43.765 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:43 gibba032 bash[24004]: audit 2021-03-10T17:27:42.387610+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.156339+0000 mgr.gibba032.whopeo (mgr.24625) 11315 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:27:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.157875+0000 mgr.gibba032.whopeo (mgr.24625) 11316 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:27:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.372686+0000 mon.a (mon.0) 3737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:27:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.373289+0000 mgr.gibba032.whopeo (mgr.24625) 11317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:27:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.375607+0000 mgr.gibba032.whopeo (mgr.24625) 11318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:27:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.375973+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:27:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.377631+0000 mgr.gibba032.whopeo (mgr.24625) 11319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:27:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.377670+0000 mgr.gibba032.whopeo (mgr.24625) 11320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:27:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.378025+0000 mon.a (mon.0) 3739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.379581+0000 mgr.gibba032.whopeo (mgr.24625) 11321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:27:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.379621+0000 mgr.gibba032.whopeo (mgr.24625) 11322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:27:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.380078+0000 mon.a (mon.0) 3740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.381416+0000 mgr.gibba032.whopeo (mgr.24625) 11323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:27:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.382027+0000 mon.a (mon.0) 3741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.383240+0000 mgr.gibba032.whopeo (mgr.24625) 11324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:27:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.383280+0000 mgr.gibba032.whopeo (mgr.24625) 11325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:27:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.385050+0000 mgr.gibba032.whopeo (mgr.24625) 11326 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:27:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.385371+0000 mgr.gibba032.whopeo (mgr.24625) 11327 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:27:43.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.385731+0000 mon.a (mon.0) 3742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.387148+0000 mgr.gibba032.whopeo (mgr.24625) 11328 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:27:43.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.387186+0000 mgr.gibba032.whopeo (mgr.24625) 11329 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:27:43.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: cephadm 2021-03-10T17:27:42.387293+0000 mgr.gibba032.whopeo (mgr.24625) 11330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:43.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[30791]: audit 2021-03-10T17:27:42.387610+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.156339+0000 mgr.gibba032.whopeo (mgr.24625) 11315 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:27:43.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.157875+0000 mgr.gibba032.whopeo (mgr.24625) 11316 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:27:43.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.372686+0000 mon.a (mon.0) 3737 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:27:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.373289+0000 mgr.gibba032.whopeo (mgr.24625) 11317 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:27:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.375607+0000 mgr.gibba032.whopeo (mgr.24625) 11318 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:27:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.375973+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:27:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.377631+0000 mgr.gibba032.whopeo (mgr.24625) 11319 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:27:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.377670+0000 mgr.gibba032.whopeo (mgr.24625) 11320 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:27:43.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.378025+0000 mon.a (mon.0) 3739 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.379581+0000 mgr.gibba032.whopeo (mgr.24625) 11321 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:27:43.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.379621+0000 mgr.gibba032.whopeo (mgr.24625) 11322 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:27:43.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.380078+0000 mon.a (mon.0) 3740 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.381416+0000 mgr.gibba032.whopeo (mgr.24625) 11323 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:27:43.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.382027+0000 mon.a (mon.0) 3741 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.383240+0000 mgr.gibba032.whopeo (mgr.24625) 11324 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:27:43.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.383280+0000 mgr.gibba032.whopeo (mgr.24625) 11325 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:27:43.840 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.385050+0000 mgr.gibba032.whopeo (mgr.24625) 11326 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:27:43.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.385371+0000 mgr.gibba032.whopeo (mgr.24625) 11327 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:27:43.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.385731+0000 mon.a (mon.0) 3742 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:43.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.387148+0000 mgr.gibba032.whopeo (mgr.24625) 11328 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:27:43.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.387186+0000 mgr.gibba032.whopeo (mgr.24625) 11329 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:27:43.841 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: cephadm 2021-03-10T17:27:42.387293+0000 mgr.gibba032.whopeo (mgr.24625) 11330 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:43.842 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:43 gibba011 bash[31774]: audit 2021-03-10T17:27:42.387610+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:27:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:44 gibba032 bash[24004]: cluster 2021-03-10T17:27:43.937766+0000 mgr.gibba032.whopeo (mgr.24625) 11331 : cluster [DBG] pgmap v6454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:44.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:44 gibba011 bash[30791]: cluster 2021-03-10T17:27:43.937766+0000 mgr.gibba032.whopeo (mgr.24625) 11331 : cluster [DBG] pgmap v6454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:44.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:44 gibba011 bash[31774]: cluster 2021-03-10T17:27:43.937766+0000 mgr.gibba032.whopeo (mgr.24625) 11331 : cluster [DBG] pgmap v6454: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:47.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:47 gibba032 bash[24004]: cluster 2021-03-10T17:27:45.939103+0000 mgr.gibba032.whopeo (mgr.24625) 11332 : cluster [DBG] pgmap v6455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:47.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:47 gibba011 bash[30791]: cluster 2021-03-10T17:27:45.939103+0000 mgr.gibba032.whopeo (mgr.24625) 11332 : cluster [DBG] pgmap v6455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:47 gibba011 bash[31774]: cluster 2021-03-10T17:27:45.939103+0000 mgr.gibba032.whopeo (mgr.24625) 11332 : cluster [DBG] pgmap v6455: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:49 gibba011 bash[30791]: cluster 2021-03-10T17:27:47.940746+0000 mgr.gibba032.whopeo (mgr.24625) 11333 : cluster [DBG] pgmap v6456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:49 gibba011 bash[31774]: cluster 2021-03-10T17:27:47.940746+0000 mgr.gibba032.whopeo (mgr.24625) 11333 : cluster [DBG] pgmap v6456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:49 gibba032 bash[24004]: cluster 2021-03-10T17:27:47.940746+0000 mgr.gibba032.whopeo (mgr.24625) 11333 : cluster [DBG] pgmap v6456: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:51.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:51 gibba011 bash[30791]: cluster 2021-03-10T17:27:49.942048+0000 mgr.gibba032.whopeo (mgr.24625) 11334 : cluster [DBG] pgmap v6457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:51.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:51 gibba011 bash[31774]: cluster 2021-03-10T17:27:49.942048+0000 mgr.gibba032.whopeo (mgr.24625) 11334 : cluster [DBG] pgmap v6457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:51 gibba032 bash[24004]: cluster 2021-03-10T17:27:49.942048+0000 mgr.gibba032.whopeo (mgr.24625) 11334 : cluster [DBG] pgmap v6457: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:52.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:52 gibba011 bash[30791]: cluster 2021-03-10T17:27:51.943532+0000 mgr.gibba032.whopeo (mgr.24625) 11335 : cluster [DBG] pgmap v6458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:52 gibba011 bash[31774]: cluster 2021-03-10T17:27:51.943532+0000 mgr.gibba032.whopeo (mgr.24625) 11335 : cluster [DBG] pgmap v6458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:52 gibba032 bash[24004]: cluster 2021-03-10T17:27:51.943532+0000 mgr.gibba032.whopeo (mgr.24625) 11335 : cluster [DBG] pgmap v6458: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:55.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:55 gibba011 bash[30791]: cluster 2021-03-10T17:27:53.945142+0000 mgr.gibba032.whopeo (mgr.24625) 11336 : cluster [DBG] pgmap v6459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:55.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:55 gibba011 bash[31774]: cluster 2021-03-10T17:27:53.945142+0000 mgr.gibba032.whopeo (mgr.24625) 11336 : cluster [DBG] pgmap v6459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:55.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:55 gibba032 bash[24004]: cluster 2021-03-10T17:27:53.945142+0000 mgr.gibba032.whopeo (mgr.24625) 11336 : cluster [DBG] pgmap v6459: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:57.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:57 gibba011 bash[30791]: cluster 2021-03-10T17:27:55.946458+0000 mgr.gibba032.whopeo (mgr.24625) 11337 : cluster [DBG] pgmap v6460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:57 gibba011 bash[31774]: cluster 2021-03-10T17:27:55.946458+0000 mgr.gibba032.whopeo (mgr.24625) 11337 : cluster [DBG] pgmap v6460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:57.406 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:57 gibba032 bash[24004]: cluster 2021-03-10T17:27:55.946458+0000 mgr.gibba032.whopeo (mgr.24625) 11337 : cluster [DBG] pgmap v6460: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:58.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.496976+0000 mon.a (mon.0) 3744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:27:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.497676+0000 mon.a (mon.0) 3745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:27:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.688191+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:27:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.688976+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.689673+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[30791]: audit 2021-03-10T17:27:57.690399+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T17:27:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.496976+0000 mon.a (mon.0) 3744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:27:58.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.497676+0000 mon.a (mon.0) 3745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:27:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.688191+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:27:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.688976+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.689673+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:58 gibba011 bash[31774]: audit 2021-03-10T17:27:57.690399+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T17:27:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.496976+0000 mon.a (mon.0) 3744 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:27:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.497676+0000 mon.a (mon.0) 3745 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:27:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.688191+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2021-03-10T17:27:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.688976+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.689673+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:27:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:58 gibba032 bash[24004]: audit 2021-03-10T17:27:57.690399+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2021-03-10T17:27:58.772 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:27:59.000 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:27:59.001 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:27:59.001 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:27:59.001 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:27:59.002 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:27:59.002 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:27:59.002 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:27:59.003 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:27:59.003 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:27:59.003 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:27:59.004 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:27:59.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:27:59.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:27:59.004 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:27:59.005 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:27:59.005 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:27:59.005 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:27:59.006 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:27:59.006 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:27:59.007 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:27:59.007 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:27:59.250 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:27:59.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:27:59.250 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:27:59.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:59.251 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:27:59.251 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:27:59.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:59.252 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:27:59.252 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:27:59.252 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:27:59.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:27:59.252 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:27:59.253 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:27:59.253 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:27:59.253 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:27:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:59 gibba011 bash[30791]: cephadm 2021-03-10T17:27:57.403357+0000 mgr.gibba032.whopeo (mgr.24625) 11338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:59.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:27:59 gibba011 bash[30791]: cluster 2021-03-10T17:27:57.948268+0000 mgr.gibba032.whopeo (mgr.24625) 11339 : cluster [DBG] pgmap v6461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:59.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:59 gibba011 bash[31774]: cephadm 2021-03-10T17:27:57.403357+0000 mgr.gibba032.whopeo (mgr.24625) 11338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:59.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:27:59 gibba011 bash[31774]: cluster 2021-03-10T17:27:57.948268+0000 mgr.gibba032.whopeo (mgr.24625) 11339 : cluster [DBG] pgmap v6461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:27:59.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:59 gibba032 bash[24004]: cephadm 2021-03-10T17:27:57.403357+0000 mgr.gibba032.whopeo (mgr.24625) 11338 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:27:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:27:59 gibba032 bash[24004]: cluster 2021-03-10T17:27:57.948268+0000 mgr.gibba032.whopeo (mgr.24625) 11339 : cluster [DBG] pgmap v6461: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:00.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[30791]: audit 2021-03-10T17:27:58.761183+0000 mgr.gibba032.whopeo (mgr.24625) 11340 : audit [DBG] from='client.51651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[30791]: audit 2021-03-10T17:27:58.985451+0000 mgr.gibba032.whopeo (mgr.24625) 11341 : audit [DBG] from='client.61639 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[30791]: audit 2021-03-10T17:27:59.249370+0000 mon.c (mon.1) 219 : audit [DBG] from='client.? 172.21.2.111:0/1619638947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:00.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[31774]: audit 2021-03-10T17:27:58.761183+0000 mgr.gibba032.whopeo (mgr.24625) 11340 : audit [DBG] from='client.51651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[31774]: audit 2021-03-10T17:27:58.985451+0000 mgr.gibba032.whopeo (mgr.24625) 11341 : audit [DBG] from='client.61639 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:00 gibba011 bash[31774]: audit 2021-03-10T17:27:59.249370+0000 mon.c (mon.1) 219 : audit [DBG] from='client.? 172.21.2.111:0/1619638947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:00 gibba032 bash[24004]: audit 2021-03-10T17:27:58.761183+0000 mgr.gibba032.whopeo (mgr.24625) 11340 : audit [DBG] from='client.51651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:00 gibba032 bash[24004]: audit 2021-03-10T17:27:58.985451+0000 mgr.gibba032.whopeo (mgr.24625) 11341 : audit [DBG] from='client.61639 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:00 gibba032 bash[24004]: audit 2021-03-10T17:27:59.249370+0000 mon.c (mon.1) 219 : audit [DBG] from='client.? 172.21.2.111:0/1619638947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:01.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:01 gibba011 bash[30791]: cluster 2021-03-10T17:27:59.949555+0000 mgr.gibba032.whopeo (mgr.24625) 11342 : cluster [DBG] pgmap v6462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:01.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:01 gibba011 bash[31774]: cluster 2021-03-10T17:27:59.949555+0000 mgr.gibba032.whopeo (mgr.24625) 11342 : cluster [DBG] pgmap v6462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:01 gibba032 bash[24004]: cluster 2021-03-10T17:27:59.949555+0000 mgr.gibba032.whopeo (mgr.24625) 11342 : cluster [DBG] pgmap v6462: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:02 gibba011 bash[30791]: cluster 2021-03-10T17:28:01.951329+0000 mgr.gibba032.whopeo (mgr.24625) 11343 : cluster [DBG] pgmap v6463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:02 gibba011 bash[31774]: cluster 2021-03-10T17:28:01.951329+0000 mgr.gibba032.whopeo (mgr.24625) 11343 : cluster [DBG] pgmap v6463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:02 gibba032 bash[24004]: cluster 2021-03-10T17:28:01.951329+0000 mgr.gibba032.whopeo (mgr.24625) 11343 : cluster [DBG] pgmap v6463: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:05.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:05 gibba011 bash[30791]: cluster 2021-03-10T17:28:03.952987+0000 mgr.gibba032.whopeo (mgr.24625) 11344 : cluster [DBG] pgmap v6464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:05.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:05 gibba011 bash[31774]: cluster 2021-03-10T17:28:03.952987+0000 mgr.gibba032.whopeo (mgr.24625) 11344 : cluster [DBG] pgmap v6464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:05.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:05 gibba032 bash[24004]: cluster 2021-03-10T17:28:03.952987+0000 mgr.gibba032.whopeo (mgr.24625) 11344 : cluster [DBG] pgmap v6464: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:07.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:07 gibba011 bash[30791]: cluster 2021-03-10T17:28:05.954453+0000 mgr.gibba032.whopeo (mgr.24625) 11345 : cluster [DBG] pgmap v6465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:07.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:07 gibba011 bash[31774]: cluster 2021-03-10T17:28:05.954453+0000 mgr.gibba032.whopeo (mgr.24625) 11345 : cluster [DBG] pgmap v6465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:07 gibba032 bash[24004]: cluster 2021-03-10T17:28:05.954453+0000 mgr.gibba032.whopeo (mgr.24625) 11345 : cluster [DBG] pgmap v6465: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:09.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:09 gibba011 bash[30791]: cluster 2021-03-10T17:28:07.956472+0000 mgr.gibba032.whopeo (mgr.24625) 11346 : cluster [DBG] pgmap v6466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:09.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:09 gibba011 bash[31774]: cluster 2021-03-10T17:28:07.956472+0000 mgr.gibba032.whopeo (mgr.24625) 11346 : cluster [DBG] pgmap v6466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:09 gibba032 bash[24004]: cluster 2021-03-10T17:28:07.956472+0000 mgr.gibba032.whopeo (mgr.24625) 11346 : cluster [DBG] pgmap v6466: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:11.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:11 gibba011 bash[30791]: cluster 2021-03-10T17:28:09.957897+0000 mgr.gibba032.whopeo (mgr.24625) 11347 : cluster [DBG] pgmap v6467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:11.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:11 gibba011 bash[31774]: cluster 2021-03-10T17:28:09.957897+0000 mgr.gibba032.whopeo (mgr.24625) 11347 : cluster [DBG] pgmap v6467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:11 gibba032 bash[24004]: cluster 2021-03-10T17:28:09.957897+0000 mgr.gibba032.whopeo (mgr.24625) 11347 : cluster [DBG] pgmap v6467: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:12 gibba011 bash[30791]: cluster 2021-03-10T17:28:11.959589+0000 mgr.gibba032.whopeo (mgr.24625) 11348 : cluster [DBG] pgmap v6468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:12.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:12 gibba011 bash[31774]: cluster 2021-03-10T17:28:11.959589+0000 mgr.gibba032.whopeo (mgr.24625) 11348 : cluster [DBG] pgmap v6468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:12 gibba032 bash[24004]: cluster 2021-03-10T17:28:11.959589+0000 mgr.gibba032.whopeo (mgr.24625) 11348 : cluster [DBG] pgmap v6468: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:13.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:13 gibba011 bash[30791]: cephadm 2021-03-10T17:28:12.419380+0000 mgr.gibba032.whopeo (mgr.24625) 11349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:13.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:13 gibba011 bash[31774]: cephadm 2021-03-10T17:28:12.419380+0000 mgr.gibba032.whopeo (mgr.24625) 11349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:13 gibba032 bash[24004]: cephadm 2021-03-10T17:28:12.419380+0000 mgr.gibba032.whopeo (mgr.24625) 11349 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:14 gibba011 bash[30791]: cluster 2021-03-10T17:28:13.961223+0000 mgr.gibba032.whopeo (mgr.24625) 11350 : cluster [DBG] pgmap v6469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:14 gibba011 bash[31774]: cluster 2021-03-10T17:28:13.961223+0000 mgr.gibba032.whopeo (mgr.24625) 11350 : cluster [DBG] pgmap v6469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:14 gibba032 bash[24004]: cluster 2021-03-10T17:28:13.961223+0000 mgr.gibba032.whopeo (mgr.24625) 11350 : cluster [DBG] pgmap v6469: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:17 gibba011 bash[30791]: cluster 2021-03-10T17:28:15.962582+0000 mgr.gibba032.whopeo (mgr.24625) 11351 : cluster [DBG] pgmap v6470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:17.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:17 gibba011 bash[31774]: cluster 2021-03-10T17:28:15.962582+0000 mgr.gibba032.whopeo (mgr.24625) 11351 : cluster [DBG] pgmap v6470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:17 gibba032 bash[24004]: cluster 2021-03-10T17:28:15.962582+0000 mgr.gibba032.whopeo (mgr.24625) 11351 : cluster [DBG] pgmap v6470: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:19.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:19 gibba011 bash[30791]: cluster 2021-03-10T17:28:17.964596+0000 mgr.gibba032.whopeo (mgr.24625) 11352 : cluster [DBG] pgmap v6471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:19.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:19 gibba011 bash[31774]: cluster 2021-03-10T17:28:17.964596+0000 mgr.gibba032.whopeo (mgr.24625) 11352 : cluster [DBG] pgmap v6471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:19 gibba032 bash[24004]: cluster 2021-03-10T17:28:17.964596+0000 mgr.gibba032.whopeo (mgr.24625) 11352 : cluster [DBG] pgmap v6471: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:21.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:21 gibba011 bash[30791]: cluster 2021-03-10T17:28:19.965729+0000 mgr.gibba032.whopeo (mgr.24625) 11353 : cluster [DBG] pgmap v6472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:21.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:21 gibba011 bash[31774]: cluster 2021-03-10T17:28:19.965729+0000 mgr.gibba032.whopeo (mgr.24625) 11353 : cluster [DBG] pgmap v6472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:21 gibba032 bash[24004]: cluster 2021-03-10T17:28:19.965729+0000 mgr.gibba032.whopeo (mgr.24625) 11353 : cluster [DBG] pgmap v6472: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:22 gibba011 bash[30791]: cluster 2021-03-10T17:28:21.967217+0000 mgr.gibba032.whopeo (mgr.24625) 11354 : cluster [DBG] pgmap v6473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:22 gibba011 bash[31774]: cluster 2021-03-10T17:28:21.967217+0000 mgr.gibba032.whopeo (mgr.24625) 11354 : cluster [DBG] pgmap v6473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:22 gibba032 bash[24004]: cluster 2021-03-10T17:28:21.967217+0000 mgr.gibba032.whopeo (mgr.24625) 11354 : cluster [DBG] pgmap v6473: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:25 gibba011 bash[30791]: cluster 2021-03-10T17:28:23.968865+0000 mgr.gibba032.whopeo (mgr.24625) 11355 : cluster [DBG] pgmap v6474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:25.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:25 gibba011 bash[31774]: cluster 2021-03-10T17:28:23.968865+0000 mgr.gibba032.whopeo (mgr.24625) 11355 : cluster [DBG] pgmap v6474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:25.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:25 gibba032 bash[24004]: cluster 2021-03-10T17:28:23.968865+0000 mgr.gibba032.whopeo (mgr.24625) 11355 : cluster [DBG] pgmap v6474: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:27.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:27 gibba011 bash[30791]: cluster 2021-03-10T17:28:25.970224+0000 mgr.gibba032.whopeo (mgr.24625) 11356 : cluster [DBG] pgmap v6475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:27.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:27 gibba011 bash[31774]: cluster 2021-03-10T17:28:25.970224+0000 mgr.gibba032.whopeo (mgr.24625) 11356 : cluster [DBG] pgmap v6475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:27.438 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:27 gibba032 bash[24004]: cluster 2021-03-10T17:28:25.970224+0000 mgr.gibba032.whopeo (mgr.24625) 11356 : cluster [DBG] pgmap v6475: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:29 gibba011 bash[30791]: cephadm 2021-03-10T17:28:27.435404+0000 mgr.gibba032.whopeo (mgr.24625) 11357 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:29.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:29 gibba011 bash[30791]: cluster 2021-03-10T17:28:27.972198+0000 mgr.gibba032.whopeo (mgr.24625) 11358 : cluster [DBG] pgmap v6476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:29 gibba011 bash[31774]: cephadm 2021-03-10T17:28:27.435404+0000 mgr.gibba032.whopeo (mgr.24625) 11357 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:29.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:29 gibba011 bash[31774]: cluster 2021-03-10T17:28:27.972198+0000 mgr.gibba032.whopeo (mgr.24625) 11358 : cluster [DBG] pgmap v6476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:29.495 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:28:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:29 gibba032 bash[24004]: cephadm 2021-03-10T17:28:27.435404+0000 mgr.gibba032.whopeo (mgr.24625) 11357 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:29 gibba032 bash[24004]: cluster 2021-03-10T17:28:27.972198+0000 mgr.gibba032.whopeo (mgr.24625) 11358 : cluster [DBG] pgmap v6476: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:29.724 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:28:29.725 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:28:29.725 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:28:29.725 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:28:29.725 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:28:29.726 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:28:29.726 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:28:29.726 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:28:29.726 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:28:29.727 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:28:29.727 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:28:29.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:28:29.727 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:28:29.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:28:29.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:28:29.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:28:29.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:28:29.728 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:28:29.729 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:28:29.729 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:28:29.729 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:28:29.971 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:28:29.971 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:28:29.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:28:29.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:28:29.972 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:28:29.972 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:28:29.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:28:29.973 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:28:29.973 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:28:29.973 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:28:29.973 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:28:29.974 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:28:29.974 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:28:29.974 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:28:29.974 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:28:30.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:30 gibba011 bash[30791]: audit 2021-03-10T17:28:29.970564+0000 mon.c (mon.1) 220 : audit [DBG] from='client.? 172.21.2.111:0/690542553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:30.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:30 gibba011 bash[31774]: audit 2021-03-10T17:28:29.970564+0000 mon.c (mon.1) 220 : audit [DBG] from='client.? 172.21.2.111:0/690542553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:30 gibba032 bash[24004]: audit 2021-03-10T17:28:29.970564+0000 mon.c (mon.1) 220 : audit [DBG] from='client.? 172.21.2.111:0/690542553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:31.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[30791]: audit 2021-03-10T17:28:29.484231+0000 mgr.gibba032.whopeo (mgr.24625) 11359 : audit [DBG] from='client.61651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[30791]: audit 2021-03-10T17:28:29.709377+0000 mgr.gibba032.whopeo (mgr.24625) 11360 : audit [DBG] from='client.51675 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[30791]: cluster 2021-03-10T17:28:29.973444+0000 mgr.gibba032.whopeo (mgr.24625) 11361 : cluster [DBG] pgmap v6477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:31.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[31774]: audit 2021-03-10T17:28:29.484231+0000 mgr.gibba032.whopeo (mgr.24625) 11359 : audit [DBG] from='client.61651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[31774]: audit 2021-03-10T17:28:29.709377+0000 mgr.gibba032.whopeo (mgr.24625) 11360 : audit [DBG] from='client.51675 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:31 gibba011 bash[31774]: cluster 2021-03-10T17:28:29.973444+0000 mgr.gibba032.whopeo (mgr.24625) 11361 : cluster [DBG] pgmap v6477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:31 gibba032 bash[24004]: audit 2021-03-10T17:28:29.484231+0000 mgr.gibba032.whopeo (mgr.24625) 11359 : audit [DBG] from='client.61651 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:31 gibba032 bash[24004]: audit 2021-03-10T17:28:29.709377+0000 mgr.gibba032.whopeo (mgr.24625) 11360 : audit [DBG] from='client.51675 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:28:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:31 gibba032 bash[24004]: cluster 2021-03-10T17:28:29.973444+0000 mgr.gibba032.whopeo (mgr.24625) 11361 : cluster [DBG] pgmap v6477: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:32 gibba011 bash[30791]: cluster 2021-03-10T17:28:31.975209+0000 mgr.gibba032.whopeo (mgr.24625) 11362 : cluster [DBG] pgmap v6478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:32 gibba011 bash[31774]: cluster 2021-03-10T17:28:31.975209+0000 mgr.gibba032.whopeo (mgr.24625) 11362 : cluster [DBG] pgmap v6478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:32 gibba032 bash[24004]: cluster 2021-03-10T17:28:31.975209+0000 mgr.gibba032.whopeo (mgr.24625) 11362 : cluster [DBG] pgmap v6478: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:35.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:35 gibba011 bash[30791]: cluster 2021-03-10T17:28:33.976931+0000 mgr.gibba032.whopeo (mgr.24625) 11363 : cluster [DBG] pgmap v6479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:35.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:35 gibba011 bash[31774]: cluster 2021-03-10T17:28:33.976931+0000 mgr.gibba032.whopeo (mgr.24625) 11363 : cluster [DBG] pgmap v6479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:35 gibba032 bash[24004]: cluster 2021-03-10T17:28:33.976931+0000 mgr.gibba032.whopeo (mgr.24625) 11363 : cluster [DBG] pgmap v6479: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:37.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:37 gibba011 bash[30791]: cluster 2021-03-10T17:28:35.978411+0000 mgr.gibba032.whopeo (mgr.24625) 11364 : cluster [DBG] pgmap v6480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:37.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:37 gibba011 bash[31774]: cluster 2021-03-10T17:28:35.978411+0000 mgr.gibba032.whopeo (mgr.24625) 11364 : cluster [DBG] pgmap v6480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:37 gibba032 bash[24004]: cluster 2021-03-10T17:28:35.978411+0000 mgr.gibba032.whopeo (mgr.24625) 11364 : cluster [DBG] pgmap v6480: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:39.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:39 gibba011 bash[30791]: cluster 2021-03-10T17:28:37.980093+0000 mgr.gibba032.whopeo (mgr.24625) 11365 : cluster [DBG] pgmap v6481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:39.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:39 gibba011 bash[31774]: cluster 2021-03-10T17:28:37.980093+0000 mgr.gibba032.whopeo (mgr.24625) 11365 : cluster [DBG] pgmap v6481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:39.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:39 gibba032 bash[24004]: cluster 2021-03-10T17:28:37.980093+0000 mgr.gibba032.whopeo (mgr.24625) 11365 : cluster [DBG] pgmap v6481: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:41.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:41 gibba011 bash[30791]: cluster 2021-03-10T17:28:39.981475+0000 mgr.gibba032.whopeo (mgr.24625) 11366 : cluster [DBG] pgmap v6482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:41.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:41 gibba011 bash[31774]: cluster 2021-03-10T17:28:39.981475+0000 mgr.gibba032.whopeo (mgr.24625) 11366 : cluster [DBG] pgmap v6482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:41 gibba032 bash[24004]: cluster 2021-03-10T17:28:39.981475+0000 mgr.gibba032.whopeo (mgr.24625) 11366 : cluster [DBG] pgmap v6482: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:42.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:42 gibba011 bash[30791]: cluster 2021-03-10T17:28:41.983097+0000 mgr.gibba032.whopeo (mgr.24625) 11367 : cluster [DBG] pgmap v6483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:42.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:42 gibba011 bash[31774]: cluster 2021-03-10T17:28:41.983097+0000 mgr.gibba032.whopeo (mgr.24625) 11367 : cluster [DBG] pgmap v6483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:42 gibba032 bash[24004]: cluster 2021-03-10T17:28:41.983097+0000 mgr.gibba032.whopeo (mgr.24625) 11367 : cluster [DBG] pgmap v6483: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:44.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.650163+0000 mgr.gibba032.whopeo (mgr.24625) 11368 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:28:44.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.651283+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.658278+0000 mgr.gibba032.whopeo (mgr.24625) 11369 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:28:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.659497+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.856135+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.856795+0000 mgr.gibba032.whopeo (mgr.24625) 11370 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:28:44.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.858518+0000 mgr.gibba032.whopeo (mgr.24625) 11371 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:28:44.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.859330+0000 mon.a (mon.0) 3753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:28:44.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.859963+0000 mgr.gibba032.whopeo (mgr.24625) 11372 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:28:44.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.860014+0000 mgr.gibba032.whopeo (mgr.24625) 11373 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:28:44.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.860670+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.861670+0000 mgr.gibba032.whopeo (mgr.24625) 11374 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:28:44.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.861709+0000 mgr.gibba032.whopeo (mgr.24625) 11375 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:28:44.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.862455+0000 mon.a (mon.0) 3755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.863143+0000 mgr.gibba032.whopeo (mgr.24625) 11376 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:28:44.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.863954+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.864355+0000 mgr.gibba032.whopeo (mgr.24625) 11377 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:28:44.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.864391+0000 mgr.gibba032.whopeo (mgr.24625) 11378 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:28:44.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.866033+0000 mgr.gibba032.whopeo (mgr.24625) 11379 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:28:44.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.866356+0000 mgr.gibba032.whopeo (mgr.24625) 11380 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:28:44.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.866766+0000 mon.a (mon.0) 3757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.867749+0000 mgr.gibba032.whopeo (mgr.24625) 11381 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:28:44.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.867790+0000 mgr.gibba032.whopeo (mgr.24625) 11382 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:28:44.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: cephadm 2021-03-10T17:28:42.867897+0000 mgr.gibba032.whopeo (mgr.24625) 11383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:44.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:43 gibba032 bash[24004]: audit 2021-03-10T17:28:42.868589+0000 mon.a (mon.0) 3758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.650163+0000 mgr.gibba032.whopeo (mgr.24625) 11368 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:28:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.651283+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.658278+0000 mgr.gibba032.whopeo (mgr.24625) 11369 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:28:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.659497+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.856135+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.856795+0000 mgr.gibba032.whopeo (mgr.24625) 11370 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:28:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.858518+0000 mgr.gibba032.whopeo (mgr.24625) 11371 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:28:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.859330+0000 mon.a (mon.0) 3753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:28:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.859963+0000 mgr.gibba032.whopeo (mgr.24625) 11372 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:28:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.860014+0000 mgr.gibba032.whopeo (mgr.24625) 11373 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:28:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.860670+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.861670+0000 mgr.gibba032.whopeo (mgr.24625) 11374 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:28:44.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.861709+0000 mgr.gibba032.whopeo (mgr.24625) 11375 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:28:44.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.862455+0000 mon.a (mon.0) 3755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.863143+0000 mgr.gibba032.whopeo (mgr.24625) 11376 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:28:44.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.863954+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.864355+0000 mgr.gibba032.whopeo (mgr.24625) 11377 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:28:44.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.864391+0000 mgr.gibba032.whopeo (mgr.24625) 11378 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:28:44.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.866033+0000 mgr.gibba032.whopeo (mgr.24625) 11379 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:28:44.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.866356+0000 mgr.gibba032.whopeo (mgr.24625) 11380 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:28:44.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.866766+0000 mon.a (mon.0) 3757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.867749+0000 mgr.gibba032.whopeo (mgr.24625) 11381 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:28:44.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.867790+0000 mgr.gibba032.whopeo (mgr.24625) 11382 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:28:44.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: cephadm 2021-03-10T17:28:42.867897+0000 mgr.gibba032.whopeo (mgr.24625) 11383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:44.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[31774]: audit 2021-03-10T17:28:42.868589+0000 mon.a (mon.0) 3758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.650163+0000 mgr.gibba032.whopeo (mgr.24625) 11368 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:28:44.087 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.651283+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.658278+0000 mgr.gibba032.whopeo (mgr.24625) 11369 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:28:44.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.659497+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.856135+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:28:44.088 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.856795+0000 mgr.gibba032.whopeo (mgr.24625) 11370 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:28:44.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.858518+0000 mgr.gibba032.whopeo (mgr.24625) 11371 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:28:44.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.859330+0000 mon.a (mon.0) 3753 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:28:44.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.859963+0000 mgr.gibba032.whopeo (mgr.24625) 11372 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:28:44.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.860014+0000 mgr.gibba032.whopeo (mgr.24625) 11373 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:28:44.089 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.860670+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.861670+0000 mgr.gibba032.whopeo (mgr.24625) 11374 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:28:44.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.861709+0000 mgr.gibba032.whopeo (mgr.24625) 11375 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:28:44.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.862455+0000 mon.a (mon.0) 3755 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.863143+0000 mgr.gibba032.whopeo (mgr.24625) 11376 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:28:44.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.863954+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.864355+0000 mgr.gibba032.whopeo (mgr.24625) 11377 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:28:44.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.864391+0000 mgr.gibba032.whopeo (mgr.24625) 11378 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:28:44.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.866033+0000 mgr.gibba032.whopeo (mgr.24625) 11379 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:28:44.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.866356+0000 mgr.gibba032.whopeo (mgr.24625) 11380 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:28:44.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.866766+0000 mon.a (mon.0) 3757 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:44.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.867749+0000 mgr.gibba032.whopeo (mgr.24625) 11381 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:28:44.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.867790+0000 mgr.gibba032.whopeo (mgr.24625) 11382 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:28:44.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: cephadm 2021-03-10T17:28:42.867897+0000 mgr.gibba032.whopeo (mgr.24625) 11383 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:44.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:43 gibba011 bash[30791]: audit 2021-03-10T17:28:42.868589+0000 mon.a (mon.0) 3758 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:28:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:44 gibba032 bash[24004]: cluster 2021-03-10T17:28:43.984645+0000 mgr.gibba032.whopeo (mgr.24625) 11384 : cluster [DBG] pgmap v6484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:44 gibba011 bash[30791]: cluster 2021-03-10T17:28:43.984645+0000 mgr.gibba032.whopeo (mgr.24625) 11384 : cluster [DBG] pgmap v6484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:45.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:44 gibba011 bash[31774]: cluster 2021-03-10T17:28:43.984645+0000 mgr.gibba032.whopeo (mgr.24625) 11384 : cluster [DBG] pgmap v6484: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:47.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:47 gibba011 bash[30791]: cluster 2021-03-10T17:28:45.985976+0000 mgr.gibba032.whopeo (mgr.24625) 11385 : cluster [DBG] pgmap v6485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:47.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:47 gibba011 bash[31774]: cluster 2021-03-10T17:28:45.985976+0000 mgr.gibba032.whopeo (mgr.24625) 11385 : cluster [DBG] pgmap v6485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:47.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:47 gibba032 bash[24004]: cluster 2021-03-10T17:28:45.985976+0000 mgr.gibba032.whopeo (mgr.24625) 11385 : cluster [DBG] pgmap v6485: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:49.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:49 gibba011 bash[30791]: cluster 2021-03-10T17:28:47.987777+0000 mgr.gibba032.whopeo (mgr.24625) 11386 : cluster [DBG] pgmap v6486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:49.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:49 gibba011 bash[31774]: cluster 2021-03-10T17:28:47.987777+0000 mgr.gibba032.whopeo (mgr.24625) 11386 : cluster [DBG] pgmap v6486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:49 gibba032 bash[24004]: cluster 2021-03-10T17:28:47.987777+0000 mgr.gibba032.whopeo (mgr.24625) 11386 : cluster [DBG] pgmap v6486: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:51.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:51 gibba011 bash[30791]: cluster 2021-03-10T17:28:49.989076+0000 mgr.gibba032.whopeo (mgr.24625) 11387 : cluster [DBG] pgmap v6487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:51.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:51 gibba011 bash[31774]: cluster 2021-03-10T17:28:49.989076+0000 mgr.gibba032.whopeo (mgr.24625) 11387 : cluster [DBG] pgmap v6487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:51 gibba032 bash[24004]: cluster 2021-03-10T17:28:49.989076+0000 mgr.gibba032.whopeo (mgr.24625) 11387 : cluster [DBG] pgmap v6487: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:52 gibba011 bash[30791]: cluster 2021-03-10T17:28:51.990864+0000 mgr.gibba032.whopeo (mgr.24625) 11388 : cluster [DBG] pgmap v6488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:52.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:52 gibba011 bash[31774]: cluster 2021-03-10T17:28:51.990864+0000 mgr.gibba032.whopeo (mgr.24625) 11388 : cluster [DBG] pgmap v6488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:52 gibba032 bash[24004]: cluster 2021-03-10T17:28:51.990864+0000 mgr.gibba032.whopeo (mgr.24625) 11388 : cluster [DBG] pgmap v6488: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:55.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:55 gibba011 bash[30791]: cluster 2021-03-10T17:28:53.992590+0000 mgr.gibba032.whopeo (mgr.24625) 11389 : cluster [DBG] pgmap v6489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:55.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:55 gibba011 bash[31774]: cluster 2021-03-10T17:28:53.992590+0000 mgr.gibba032.whopeo (mgr.24625) 11389 : cluster [DBG] pgmap v6489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:55.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:55 gibba032 bash[24004]: cluster 2021-03-10T17:28:53.992590+0000 mgr.gibba032.whopeo (mgr.24625) 11389 : cluster [DBG] pgmap v6489: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:57.329 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:57 gibba011 bash[30791]: cluster 2021-03-10T17:28:55.993977+0000 mgr.gibba032.whopeo (mgr.24625) 11390 : cluster [DBG] pgmap v6490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:57.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:57 gibba011 bash[31774]: cluster 2021-03-10T17:28:55.993977+0000 mgr.gibba032.whopeo (mgr.24625) 11390 : cluster [DBG] pgmap v6490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:57 gibba032 bash[24004]: cluster 2021-03-10T17:28:55.993977+0000 mgr.gibba032.whopeo (mgr.24625) 11390 : cluster [DBG] pgmap v6490: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:58.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.505185+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:28:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.505907+0000 mon.a (mon.0) 3760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:28:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.696714+0000 mon.a (mon.0) 3761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:28:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.697573+0000 mon.a (mon.0) 3762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.698377+0000 mon.a (mon.0) 3763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:58 gibba032 bash[24004]: audit 2021-03-10T17:28:57.699183+0000 mon.a (mon.0) 3764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:28:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.505185+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:28:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.505907+0000 mon.a (mon.0) 3760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:28:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.696714+0000 mon.a (mon.0) 3761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:28:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.697573+0000 mon.a (mon.0) 3762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.698377+0000 mon.a (mon.0) 3763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[30791]: audit 2021-03-10T17:28:57.699183+0000 mon.a (mon.0) 3764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:28:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.505185+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:28:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.505907+0000 mon.a (mon.0) 3760 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:28:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.696714+0000 mon.a (mon.0) 3761 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:28:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.697573+0000 mon.a (mon.0) 3762 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.698377+0000 mon.a (mon.0) 3763 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:28:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:58 gibba011 bash[31774]: audit 2021-03-10T17:28:57.699183+0000 mon.a (mon.0) 3764 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:28:59.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:59 gibba032 bash[24004]: cephadm 2021-03-10T17:28:57.883918+0000 mgr.gibba032.whopeo (mgr.24625) 11391 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:28:59 gibba032 bash[24004]: cluster 2021-03-10T17:28:57.995855+0000 mgr.gibba032.whopeo (mgr.24625) 11392 : cluster [DBG] pgmap v6491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:59 gibba011 bash[30791]: cephadm 2021-03-10T17:28:57.883918+0000 mgr.gibba032.whopeo (mgr.24625) 11391 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:28:59 gibba011 bash[30791]: cluster 2021-03-10T17:28:57.995855+0000 mgr.gibba032.whopeo (mgr.24625) 11392 : cluster [DBG] pgmap v6491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:28:59.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:59 gibba011 bash[31774]: cephadm 2021-03-10T17:28:57.883918+0000 mgr.gibba032.whopeo (mgr.24625) 11391 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:28:59.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:28:59 gibba011 bash[31774]: cluster 2021-03-10T17:28:57.995855+0000 mgr.gibba032.whopeo (mgr.24625) 11392 : cluster [DBG] pgmap v6491: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:00.217 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:29:00.472 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:29:00.472 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:29:00.472 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:29:00.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:29:00.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:29:00.473 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:29:00.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:29:00.473 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:29:00.474 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:29:00.474 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:29:00.474 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:29:00.474 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:29:00.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:29:00.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:29:00.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:29:00.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:29:00.475 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:29:00.476 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:29:00.476 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:29:00.476 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:29:00.476 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:29:00.732 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:29:00.732 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:29:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:29:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:29:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:29:00.733 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:00.734 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:29:00.734 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:29:00.734 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:00.734 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:29:00.735 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:29:00.735 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:29:00.735 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:29:00.735 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:29:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:01 gibba032 bash[24004]: cluster 2021-03-10T17:28:59.997161+0000 mgr.gibba032.whopeo (mgr.24625) 11393 : cluster [DBG] pgmap v6492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:01 gibba032 bash[24004]: audit 2021-03-10T17:29:00.731032+0000 mon.c (mon.1) 221 : audit [DBG] from='client.? 172.21.2.111:0/1699833658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:01 gibba011 bash[30791]: cluster 2021-03-10T17:28:59.997161+0000 mgr.gibba032.whopeo (mgr.24625) 11393 : cluster [DBG] pgmap v6492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:01 gibba011 bash[30791]: audit 2021-03-10T17:29:00.731032+0000 mon.c (mon.1) 221 : audit [DBG] from='client.? 172.21.2.111:0/1699833658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:01 gibba011 bash[31774]: cluster 2021-03-10T17:28:59.997161+0000 mgr.gibba032.whopeo (mgr.24625) 11393 : cluster [DBG] pgmap v6492: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:01.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:01 gibba011 bash[31774]: audit 2021-03-10T17:29:00.731032+0000 mon.c (mon.1) 221 : audit [DBG] from='client.? 172.21.2.111:0/1699833658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:02.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:02 gibba032 bash[24004]: audit 2021-03-10T17:29:00.205697+0000 mgr.gibba032.whopeo (mgr.24625) 11394 : audit [DBG] from='client.61669 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:02 gibba032 bash[24004]: audit 2021-03-10T17:29:00.429549+0000 mgr.gibba032.whopeo (mgr.24625) 11395 : audit [DBG] from='client.51693 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:02.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:02 gibba011 bash[30791]: audit 2021-03-10T17:29:00.205697+0000 mgr.gibba032.whopeo (mgr.24625) 11394 : audit [DBG] from='client.61669 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:02.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:02 gibba011 bash[30791]: audit 2021-03-10T17:29:00.429549+0000 mgr.gibba032.whopeo (mgr.24625) 11395 : audit [DBG] from='client.51693 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:02 gibba011 bash[31774]: audit 2021-03-10T17:29:00.205697+0000 mgr.gibba032.whopeo (mgr.24625) 11394 : audit [DBG] from='client.61669 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:02.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:02 gibba011 bash[31774]: audit 2021-03-10T17:29:00.429549+0000 mgr.gibba032.whopeo (mgr.24625) 11395 : audit [DBG] from='client.51693 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:03.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:03 gibba032 bash[24004]: cluster 2021-03-10T17:29:01.998750+0000 mgr.gibba032.whopeo (mgr.24625) 11396 : cluster [DBG] pgmap v6493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:03.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:03 gibba011 bash[30791]: cluster 2021-03-10T17:29:01.998750+0000 mgr.gibba032.whopeo (mgr.24625) 11396 : cluster [DBG] pgmap v6493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:03.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:03 gibba011 bash[31774]: cluster 2021-03-10T17:29:01.998750+0000 mgr.gibba032.whopeo (mgr.24625) 11396 : cluster [DBG] pgmap v6493: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:05.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:05 gibba032 bash[24004]: cluster 2021-03-10T17:29:04.000550+0000 mgr.gibba032.whopeo (mgr.24625) 11397 : cluster [DBG] pgmap v6494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:05.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:05 gibba011 bash[30791]: cluster 2021-03-10T17:29:04.000550+0000 mgr.gibba032.whopeo (mgr.24625) 11397 : cluster [DBG] pgmap v6494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:05 gibba011 bash[31774]: cluster 2021-03-10T17:29:04.000550+0000 mgr.gibba032.whopeo (mgr.24625) 11397 : cluster [DBG] pgmap v6494: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:07 gibba032 bash[24004]: cluster 2021-03-10T17:29:06.002136+0000 mgr.gibba032.whopeo (mgr.24625) 11398 : cluster [DBG] pgmap v6495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:07 gibba011 bash[30791]: cluster 2021-03-10T17:29:06.002136+0000 mgr.gibba032.whopeo (mgr.24625) 11398 : cluster [DBG] pgmap v6495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:07 gibba011 bash[31774]: cluster 2021-03-10T17:29:06.002136+0000 mgr.gibba032.whopeo (mgr.24625) 11398 : cluster [DBG] pgmap v6495: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:09 gibba032 bash[24004]: cluster 2021-03-10T17:29:08.004048+0000 mgr.gibba032.whopeo (mgr.24625) 11399 : cluster [DBG] pgmap v6496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:09 gibba011 bash[30791]: cluster 2021-03-10T17:29:08.004048+0000 mgr.gibba032.whopeo (mgr.24625) 11399 : cluster [DBG] pgmap v6496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:09.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:09 gibba011 bash[31774]: cluster 2021-03-10T17:29:08.004048+0000 mgr.gibba032.whopeo (mgr.24625) 11399 : cluster [DBG] pgmap v6496: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:11 gibba032 bash[24004]: cluster 2021-03-10T17:29:10.005417+0000 mgr.gibba032.whopeo (mgr.24625) 11400 : cluster [DBG] pgmap v6497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:11.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:11 gibba011 bash[30791]: cluster 2021-03-10T17:29:10.005417+0000 mgr.gibba032.whopeo (mgr.24625) 11400 : cluster [DBG] pgmap v6497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:11.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:11 gibba011 bash[31774]: cluster 2021-03-10T17:29:10.005417+0000 mgr.gibba032.whopeo (mgr.24625) 11400 : cluster [DBG] pgmap v6497: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:12 gibba032 bash[24004]: cluster 2021-03-10T17:29:12.007217+0000 mgr.gibba032.whopeo (mgr.24625) 11401 : cluster [DBG] pgmap v6498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:12 gibba011 bash[30791]: cluster 2021-03-10T17:29:12.007217+0000 mgr.gibba032.whopeo (mgr.24625) 11401 : cluster [DBG] pgmap v6498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:12 gibba011 bash[31774]: cluster 2021-03-10T17:29:12.007217+0000 mgr.gibba032.whopeo (mgr.24625) 11401 : cluster [DBG] pgmap v6498: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:13.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:13 gibba032 bash[24004]: cephadm 2021-03-10T17:29:12.899973+0000 mgr.gibba032.whopeo (mgr.24625) 11402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:13 gibba011 bash[30791]: cephadm 2021-03-10T17:29:12.899973+0000 mgr.gibba032.whopeo (mgr.24625) 11402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:13.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:13 gibba011 bash[31774]: cephadm 2021-03-10T17:29:12.899973+0000 mgr.gibba032.whopeo (mgr.24625) 11402 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:14 gibba032 bash[24004]: cluster 2021-03-10T17:29:14.008875+0000 mgr.gibba032.whopeo (mgr.24625) 11403 : cluster [DBG] pgmap v6499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:14.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:14 gibba011 bash[30791]: cluster 2021-03-10T17:29:14.008875+0000 mgr.gibba032.whopeo (mgr.24625) 11403 : cluster [DBG] pgmap v6499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:14.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:14 gibba011 bash[31774]: cluster 2021-03-10T17:29:14.008875+0000 mgr.gibba032.whopeo (mgr.24625) 11403 : cluster [DBG] pgmap v6499: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:17.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:17 gibba011 bash[30791]: cluster 2021-03-10T17:29:16.010252+0000 mgr.gibba032.whopeo (mgr.24625) 11404 : cluster [DBG] pgmap v6500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:17.330 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:17 gibba011 bash[31774]: cluster 2021-03-10T17:29:16.010252+0000 mgr.gibba032.whopeo (mgr.24625) 11404 : cluster [DBG] pgmap v6500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:17 gibba032 bash[24004]: cluster 2021-03-10T17:29:16.010252+0000 mgr.gibba032.whopeo (mgr.24625) 11404 : cluster [DBG] pgmap v6500: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:19 gibba032 bash[24004]: cluster 2021-03-10T17:29:18.011885+0000 mgr.gibba032.whopeo (mgr.24625) 11405 : cluster [DBG] pgmap v6501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:19 gibba011 bash[30791]: cluster 2021-03-10T17:29:18.011885+0000 mgr.gibba032.whopeo (mgr.24625) 11405 : cluster [DBG] pgmap v6501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:19.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:19 gibba011 bash[31774]: cluster 2021-03-10T17:29:18.011885+0000 mgr.gibba032.whopeo (mgr.24625) 11405 : cluster [DBG] pgmap v6501: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:21 gibba032 bash[24004]: cluster 2021-03-10T17:29:20.012376+0000 mgr.gibba032.whopeo (mgr.24625) 11406 : cluster [DBG] pgmap v6502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:21.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:21 gibba011 bash[30791]: cluster 2021-03-10T17:29:20.012376+0000 mgr.gibba032.whopeo (mgr.24625) 11406 : cluster [DBG] pgmap v6502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:21 gibba011 bash[31774]: cluster 2021-03-10T17:29:20.012376+0000 mgr.gibba032.whopeo (mgr.24625) 11406 : cluster [DBG] pgmap v6502: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:22 gibba032 bash[24004]: cluster 2021-03-10T17:29:22.013964+0000 mgr.gibba032.whopeo (mgr.24625) 11407 : cluster [DBG] pgmap v6503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:22 gibba011 bash[30791]: cluster 2021-03-10T17:29:22.013964+0000 mgr.gibba032.whopeo (mgr.24625) 11407 : cluster [DBG] pgmap v6503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:22 gibba011 bash[31774]: cluster 2021-03-10T17:29:22.013964+0000 mgr.gibba032.whopeo (mgr.24625) 11407 : cluster [DBG] pgmap v6503: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:25.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:25 gibba032 bash[24004]: cluster 2021-03-10T17:29:24.015550+0000 mgr.gibba032.whopeo (mgr.24625) 11408 : cluster [DBG] pgmap v6504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:25.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:25 gibba011 bash[30791]: cluster 2021-03-10T17:29:24.015550+0000 mgr.gibba032.whopeo (mgr.24625) 11408 : cluster [DBG] pgmap v6504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:25.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:25 gibba011 bash[31774]: cluster 2021-03-10T17:29:24.015550+0000 mgr.gibba032.whopeo (mgr.24625) 11408 : cluster [DBG] pgmap v6504: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:27.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:27 gibba032 bash[24004]: cluster 2021-03-10T17:29:26.016887+0000 mgr.gibba032.whopeo (mgr.24625) 11409 : cluster [DBG] pgmap v6505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:27 gibba011 bash[30791]: cluster 2021-03-10T17:29:26.016887+0000 mgr.gibba032.whopeo (mgr.24625) 11409 : cluster [DBG] pgmap v6505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:27.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:27 gibba011 bash[31774]: cluster 2021-03-10T17:29:26.016887+0000 mgr.gibba032.whopeo (mgr.24625) 11409 : cluster [DBG] pgmap v6505: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:29 gibba032 bash[24004]: cephadm 2021-03-10T17:29:27.915885+0000 mgr.gibba032.whopeo (mgr.24625) 11410 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:29.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:29 gibba032 bash[24004]: cluster 2021-03-10T17:29:28.018802+0000 mgr.gibba032.whopeo (mgr.24625) 11411 : cluster [DBG] pgmap v6506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:29 gibba011 bash[30791]: cephadm 2021-03-10T17:29:27.915885+0000 mgr.gibba032.whopeo (mgr.24625) 11410 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:29 gibba011 bash[30791]: cluster 2021-03-10T17:29:28.018802+0000 mgr.gibba032.whopeo (mgr.24625) 11411 : cluster [DBG] pgmap v6506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:29.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:29 gibba011 bash[31774]: cephadm 2021-03-10T17:29:27.915885+0000 mgr.gibba032.whopeo (mgr.24625) 11410 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:29.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:29 gibba011 bash[31774]: cluster 2021-03-10T17:29:28.018802+0000 mgr.gibba032.whopeo (mgr.24625) 11411 : cluster [DBG] pgmap v6506: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:30.973 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:29:31.230 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:29:31.231 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:29:31.231 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:29:31.232 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:29:31.232 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:29:31.232 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:29:31.233 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:29:31.233 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:29:31.234 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:29:31.234 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:29:31.234 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:29:31.235 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:29:31.235 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:29:31.236 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:29:31.236 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:29:31.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:29:31.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:29:31.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:29:31.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:29:31.238 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:29:31.239 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:29:31.483 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:29:31.483 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:29:31.484 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:29:31.484 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:31.485 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:29:31.485 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:29:31.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:31.486 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:29:31.486 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:29:31.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:29:31.487 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:29:31.488 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:29:31.488 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:29:31.488 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:29:31.489 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:29:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:31 gibba032 bash[24004]: cluster 2021-03-10T17:29:30.019540+0000 mgr.gibba032.whopeo (mgr.24625) 11412 : cluster [DBG] pgmap v6507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:31 gibba011 bash[30791]: cluster 2021-03-10T17:29:30.019540+0000 mgr.gibba032.whopeo (mgr.24625) 11412 : cluster [DBG] pgmap v6507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:31 gibba011 bash[31774]: cluster 2021-03-10T17:29:30.019540+0000 mgr.gibba032.whopeo (mgr.24625) 11412 : cluster [DBG] pgmap v6507: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:32.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:32 gibba032 bash[24004]: audit 2021-03-10T17:29:30.962438+0000 mgr.gibba032.whopeo (mgr.24625) 11413 : audit [DBG] from='client.61687 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:32 gibba032 bash[24004]: audit 2021-03-10T17:29:31.482170+0000 mon.c (mon.1) 222 : audit [DBG] from='client.? 172.21.2.111:0/1738832763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:32 gibba011 bash[30791]: audit 2021-03-10T17:29:30.962438+0000 mgr.gibba032.whopeo (mgr.24625) 11413 : audit [DBG] from='client.61687 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:32.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:32 gibba011 bash[30791]: audit 2021-03-10T17:29:31.482170+0000 mon.c (mon.1) 222 : audit [DBG] from='client.? 172.21.2.111:0/1738832763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:32 gibba011 bash[31774]: audit 2021-03-10T17:29:30.962438+0000 mgr.gibba032.whopeo (mgr.24625) 11413 : audit [DBG] from='client.61687 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:32 gibba011 bash[31774]: audit 2021-03-10T17:29:31.482170+0000 mon.c (mon.1) 222 : audit [DBG] from='client.? 172.21.2.111:0/1738832763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:33 gibba032 bash[24004]: audit 2021-03-10T17:29:31.187816+0000 mgr.gibba032.whopeo (mgr.24625) 11414 : audit [DBG] from='client.51711 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:33.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:33 gibba032 bash[24004]: cluster 2021-03-10T17:29:32.021219+0000 mgr.gibba032.whopeo (mgr.24625) 11415 : cluster [DBG] pgmap v6508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:33.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:33 gibba011 bash[30791]: audit 2021-03-10T17:29:31.187816+0000 mgr.gibba032.whopeo (mgr.24625) 11414 : audit [DBG] from='client.51711 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:33.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:33 gibba011 bash[30791]: cluster 2021-03-10T17:29:32.021219+0000 mgr.gibba032.whopeo (mgr.24625) 11415 : cluster [DBG] pgmap v6508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:33 gibba011 bash[31774]: audit 2021-03-10T17:29:31.187816+0000 mgr.gibba032.whopeo (mgr.24625) 11414 : audit [DBG] from='client.51711 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:29:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:33 gibba011 bash[31774]: cluster 2021-03-10T17:29:32.021219+0000 mgr.gibba032.whopeo (mgr.24625) 11415 : cluster [DBG] pgmap v6508: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:35 gibba032 bash[24004]: cluster 2021-03-10T17:29:34.022860+0000 mgr.gibba032.whopeo (mgr.24625) 11416 : cluster [DBG] pgmap v6509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:35 gibba011 bash[30791]: cluster 2021-03-10T17:29:34.022860+0000 mgr.gibba032.whopeo (mgr.24625) 11416 : cluster [DBG] pgmap v6509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:35.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:35 gibba011 bash[31774]: cluster 2021-03-10T17:29:34.022860+0000 mgr.gibba032.whopeo (mgr.24625) 11416 : cluster [DBG] pgmap v6509: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:37 gibba032 bash[24004]: cluster 2021-03-10T17:29:36.024249+0000 mgr.gibba032.whopeo (mgr.24625) 11417 : cluster [DBG] pgmap v6510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:37.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:37 gibba011 bash[30791]: cluster 2021-03-10T17:29:36.024249+0000 mgr.gibba032.whopeo (mgr.24625) 11417 : cluster [DBG] pgmap v6510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:37 gibba011 bash[31774]: cluster 2021-03-10T17:29:36.024249+0000 mgr.gibba032.whopeo (mgr.24625) 11417 : cluster [DBG] pgmap v6510: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:39.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:39 gibba032 bash[24004]: cluster 2021-03-10T17:29:38.026171+0000 mgr.gibba032.whopeo (mgr.24625) 11418 : cluster [DBG] pgmap v6511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:39.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:39 gibba011 bash[30791]: cluster 2021-03-10T17:29:38.026171+0000 mgr.gibba032.whopeo (mgr.24625) 11418 : cluster [DBG] pgmap v6511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:39.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:39 gibba011 bash[31774]: cluster 2021-03-10T17:29:38.026171+0000 mgr.gibba032.whopeo (mgr.24625) 11418 : cluster [DBG] pgmap v6511: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:40 gibba032 bash[24004]: cluster 2021-03-10T17:29:40.026922+0000 mgr.gibba032.whopeo (mgr.24625) 11419 : cluster [DBG] pgmap v6512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:40 gibba011 bash[30791]: cluster 2021-03-10T17:29:40.026922+0000 mgr.gibba032.whopeo (mgr.24625) 11419 : cluster [DBG] pgmap v6512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:40.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:40 gibba011 bash[31774]: cluster 2021-03-10T17:29:40.026922+0000 mgr.gibba032.whopeo (mgr.24625) 11419 : cluster [DBG] pgmap v6512: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:42 gibba032 bash[24004]: cluster 2021-03-10T17:29:42.028433+0000 mgr.gibba032.whopeo (mgr.24625) 11420 : cluster [DBG] pgmap v6513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:42 gibba011 bash[30791]: cluster 2021-03-10T17:29:42.028433+0000 mgr.gibba032.whopeo (mgr.24625) 11420 : cluster [DBG] pgmap v6513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:42 gibba011 bash[31774]: cluster 2021-03-10T17:29:42.028433+0000 mgr.gibba032.whopeo (mgr.24625) 11420 : cluster [DBG] pgmap v6513: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:43.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: cephadm 2021-03-10T17:29:42.925792+0000 mgr.gibba032.whopeo (mgr.24625) 11421 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:29:43.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: cephadm 2021-03-10T17:29:42.927329+0000 mgr.gibba032.whopeo (mgr.24625) 11422 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:29:43.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.151677+0000 mon.a (mon.0) 3765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:29:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.154751+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:29:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.155984+0000 mon.a (mon.0) 3767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.157132+0000 mon.a (mon.0) 3768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.158220+0000 mon.a (mon.0) 3769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.160397+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:43 gibba032 bash[24004]: audit 2021-03-10T17:29:43.161700+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: cephadm 2021-03-10T17:29:42.925792+0000 mgr.gibba032.whopeo (mgr.24625) 11421 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:29:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: cephadm 2021-03-10T17:29:42.927329+0000 mgr.gibba032.whopeo (mgr.24625) 11422 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:29:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.151677+0000 mon.a (mon.0) 3765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:29:43.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.154751+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:29:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.155984+0000 mon.a (mon.0) 3767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.157132+0000 mon.a (mon.0) 3768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.158220+0000 mon.a (mon.0) 3769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.160397+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[30791]: audit 2021-03-10T17:29:43.161700+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: cephadm 2021-03-10T17:29:42.925792+0000 mgr.gibba032.whopeo (mgr.24625) 11421 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:29:43.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: cephadm 2021-03-10T17:29:42.927329+0000 mgr.gibba032.whopeo (mgr.24625) 11422 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:29:43.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.151677+0000 mon.a (mon.0) 3765 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:29:43.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.154751+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:29:43.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.155984+0000 mon.a (mon.0) 3767 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.157132+0000 mon.a (mon.0) 3768 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.158220+0000 mon.a (mon.0) 3769 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.160397+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:43.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:43 gibba011 bash[31774]: audit 2021-03-10T17:29:43.161700+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:29:44.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.152298+0000 mgr.gibba032.whopeo (mgr.24625) 11423 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:29:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.153898+0000 mgr.gibba032.whopeo (mgr.24625) 11424 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:29:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.154933+0000 mgr.gibba032.whopeo (mgr.24625) 11425 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:29:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.154969+0000 mgr.gibba032.whopeo (mgr.24625) 11426 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:29:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.156151+0000 mgr.gibba032.whopeo (mgr.24625) 11427 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:29:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.156190+0000 mgr.gibba032.whopeo (mgr.24625) 11428 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:29:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.157168+0000 mgr.gibba032.whopeo (mgr.24625) 11429 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:29:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.158134+0000 mgr.gibba032.whopeo (mgr.24625) 11430 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:29:44.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.158202+0000 mgr.gibba032.whopeo (mgr.24625) 11431 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:29:44.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.159380+0000 mgr.gibba032.whopeo (mgr.24625) 11432 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:29:44.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.159695+0000 mgr.gibba032.whopeo (mgr.24625) 11433 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:29:44.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.160898+0000 mgr.gibba032.whopeo (mgr.24625) 11434 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:29:44.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.160934+0000 mgr.gibba032.whopeo (mgr.24625) 11435 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:29:44.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cephadm 2021-03-10T17:29:43.161039+0000 mgr.gibba032.whopeo (mgr.24625) 11436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:44.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:44 gibba032 bash[24004]: cluster 2021-03-10T17:29:44.029789+0000 mgr.gibba032.whopeo (mgr.24625) 11437 : cluster [DBG] pgmap v6514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:44.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.152298+0000 mgr.gibba032.whopeo (mgr.24625) 11423 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:29:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.153898+0000 mgr.gibba032.whopeo (mgr.24625) 11424 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:29:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.154933+0000 mgr.gibba032.whopeo (mgr.24625) 11425 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:29:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.154969+0000 mgr.gibba032.whopeo (mgr.24625) 11426 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:29:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.156151+0000 mgr.gibba032.whopeo (mgr.24625) 11427 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:29:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.156190+0000 mgr.gibba032.whopeo (mgr.24625) 11428 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:29:44.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.157168+0000 mgr.gibba032.whopeo (mgr.24625) 11429 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:29:44.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.158134+0000 mgr.gibba032.whopeo (mgr.24625) 11430 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:29:44.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.158202+0000 mgr.gibba032.whopeo (mgr.24625) 11431 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:29:44.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.159380+0000 mgr.gibba032.whopeo (mgr.24625) 11432 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:29:44.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.159695+0000 mgr.gibba032.whopeo (mgr.24625) 11433 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:29:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.160898+0000 mgr.gibba032.whopeo (mgr.24625) 11434 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:29:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.160934+0000 mgr.gibba032.whopeo (mgr.24625) 11435 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:29:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cephadm 2021-03-10T17:29:43.161039+0000 mgr.gibba032.whopeo (mgr.24625) 11436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[30791]: cluster 2021-03-10T17:29:44.029789+0000 mgr.gibba032.whopeo (mgr.24625) 11437 : cluster [DBG] pgmap v6514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:44.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.152298+0000 mgr.gibba032.whopeo (mgr.24625) 11423 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:29:44.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.153898+0000 mgr.gibba032.whopeo (mgr.24625) 11424 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:29:44.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.154933+0000 mgr.gibba032.whopeo (mgr.24625) 11425 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:29:44.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.154969+0000 mgr.gibba032.whopeo (mgr.24625) 11426 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:29:44.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.156151+0000 mgr.gibba032.whopeo (mgr.24625) 11427 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:29:44.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.156190+0000 mgr.gibba032.whopeo (mgr.24625) 11428 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:29:44.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.157168+0000 mgr.gibba032.whopeo (mgr.24625) 11429 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:29:44.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.158134+0000 mgr.gibba032.whopeo (mgr.24625) 11430 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:29:44.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.158202+0000 mgr.gibba032.whopeo (mgr.24625) 11431 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:29:44.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.159380+0000 mgr.gibba032.whopeo (mgr.24625) 11432 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:29:44.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.159695+0000 mgr.gibba032.whopeo (mgr.24625) 11433 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:29:44.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.160898+0000 mgr.gibba032.whopeo (mgr.24625) 11434 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:29:44.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.160934+0000 mgr.gibba032.whopeo (mgr.24625) 11435 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:29:44.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cephadm 2021-03-10T17:29:43.161039+0000 mgr.gibba032.whopeo (mgr.24625) 11436 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:29:44.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:44 gibba011 bash[31774]: cluster 2021-03-10T17:29:44.029789+0000 mgr.gibba032.whopeo (mgr.24625) 11437 : cluster [DBG] pgmap v6514: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:47.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:47 gibba032 bash[24004]: cluster 2021-03-10T17:29:46.031189+0000 mgr.gibba032.whopeo (mgr.24625) 11438 : cluster [DBG] pgmap v6515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:47 gibba011 bash[30791]: cluster 2021-03-10T17:29:46.031189+0000 mgr.gibba032.whopeo (mgr.24625) 11438 : cluster [DBG] pgmap v6515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:47.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:47 gibba011 bash[31774]: cluster 2021-03-10T17:29:46.031189+0000 mgr.gibba032.whopeo (mgr.24625) 11438 : cluster [DBG] pgmap v6515: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:49 gibba032 bash[24004]: cluster 2021-03-10T17:29:48.033154+0000 mgr.gibba032.whopeo (mgr.24625) 11439 : cluster [DBG] pgmap v6516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:49 gibba011 bash[30791]: cluster 2021-03-10T17:29:48.033154+0000 mgr.gibba032.whopeo (mgr.24625) 11439 : cluster [DBG] pgmap v6516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:49.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:49 gibba011 bash[31774]: cluster 2021-03-10T17:29:48.033154+0000 mgr.gibba032.whopeo (mgr.24625) 11439 : cluster [DBG] pgmap v6516: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:51 gibba032 bash[24004]: cluster 2021-03-10T17:29:50.033639+0000 mgr.gibba032.whopeo (mgr.24625) 11440 : cluster [DBG] pgmap v6517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:51 gibba011 bash[30791]: cluster 2021-03-10T17:29:50.033639+0000 mgr.gibba032.whopeo (mgr.24625) 11440 : cluster [DBG] pgmap v6517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:51 gibba011 bash[31774]: cluster 2021-03-10T17:29:50.033639+0000 mgr.gibba032.whopeo (mgr.24625) 11440 : cluster [DBG] pgmap v6517: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:52 gibba032 bash[24004]: cluster 2021-03-10T17:29:52.035062+0000 mgr.gibba032.whopeo (mgr.24625) 11441 : cluster [DBG] pgmap v6518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:52 gibba011 bash[30791]: cluster 2021-03-10T17:29:52.035062+0000 mgr.gibba032.whopeo (mgr.24625) 11441 : cluster [DBG] pgmap v6518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:52 gibba011 bash[31774]: cluster 2021-03-10T17:29:52.035062+0000 mgr.gibba032.whopeo (mgr.24625) 11441 : cluster [DBG] pgmap v6518: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:55.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:55 gibba032 bash[24004]: cluster 2021-03-10T17:29:54.036617+0000 mgr.gibba032.whopeo (mgr.24625) 11442 : cluster [DBG] pgmap v6519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:55 gibba011 bash[30791]: cluster 2021-03-10T17:29:54.036617+0000 mgr.gibba032.whopeo (mgr.24625) 11442 : cluster [DBG] pgmap v6519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:55 gibba011 bash[31774]: cluster 2021-03-10T17:29:54.036617+0000 mgr.gibba032.whopeo (mgr.24625) 11442 : cluster [DBG] pgmap v6519: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:57 gibba032 bash[24004]: cluster 2021-03-10T17:29:56.037916+0000 mgr.gibba032.whopeo (mgr.24625) 11443 : cluster [DBG] pgmap v6520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:57 gibba011 bash[30791]: cluster 2021-03-10T17:29:56.037916+0000 mgr.gibba032.whopeo (mgr.24625) 11443 : cluster [DBG] pgmap v6520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:57 gibba011 bash[31774]: cluster 2021-03-10T17:29:56.037916+0000 mgr.gibba032.whopeo (mgr.24625) 11443 : cluster [DBG] pgmap v6520: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.511708+0000 mon.a (mon.0) 3772 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:29:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.512448+0000 mon.a (mon.0) 3773 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:29:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.707507+0000 mon.a (mon.0) 3774 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:29:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.708255+0000 mon.a (mon.0) 3775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.708972+0000 mon.a (mon.0) 3776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:58 gibba032 bash[24004]: audit 2021-03-10T17:29:57.709677+0000 mon.a (mon.0) 3777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.511708+0000 mon.a (mon.0) 3772 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:29:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.512448+0000 mon.a (mon.0) 3773 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:29:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.707507+0000 mon.a (mon.0) 3774 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:29:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.708255+0000 mon.a (mon.0) 3775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.708972+0000 mon.a (mon.0) 3776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[30791]: audit 2021-03-10T17:29:57.709677+0000 mon.a (mon.0) 3777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.511708+0000 mon.a (mon.0) 3772 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:29:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.512448+0000 mon.a (mon.0) 3773 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:29:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.707507+0000 mon.a (mon.0) 3774 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2021-03-10T17:29:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.708255+0000 mon.a (mon.0) 3775 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.708972+0000 mon.a (mon.0) 3776 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2021-03-10T17:29:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:58 gibba011 bash[31774]: audit 2021-03-10T17:29:57.709677+0000 mon.a (mon.0) 3777 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:29:59.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:29:59 gibba032 bash[24004]: cluster 2021-03-10T17:29:58.038713+0000 mgr.gibba032.whopeo (mgr.24625) 11444 : cluster [DBG] pgmap v6521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:29:59 gibba011 bash[30791]: cluster 2021-03-10T17:29:58.038713+0000 mgr.gibba032.whopeo (mgr.24625) 11444 : cluster [DBG] pgmap v6521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:29:59.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:29:59 gibba011 bash[31774]: cluster 2021-03-10T17:29:58.038713+0000 mgr.gibba032.whopeo (mgr.24625) 11444 : cluster [DBG] pgmap v6521: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:00 gibba011 bash[30791]: debug 2021-03-10T17:29:59.997+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:30:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:00 gibba011 bash[30791]: cephadm 2021-03-10T17:29:58.176512+0000 mgr.gibba032.whopeo (mgr.24625) 11445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:00 gibba011 bash[30791]: cluster 2021-03-10T17:30:00.000289+0000 mon.a (mon.0) 3778 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:30:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:00 gibba011 bash[31774]: cephadm 2021-03-10T17:29:58.176512+0000 mgr.gibba032.whopeo (mgr.24625) 11445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:00 gibba011 bash[31774]: cluster 2021-03-10T17:30:00.000289+0000 mon.a (mon.0) 3778 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:30:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:00 gibba032 bash[24004]: cephadm 2021-03-10T17:29:58.176512+0000 mgr.gibba032.whopeo (mgr.24625) 11445 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:00 gibba032 bash[24004]: cluster 2021-03-10T17:30:00.000289+0000 mon.a (mon.0) 3778 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:30:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:01 gibba032 bash[24004]: cluster 2021-03-10T17:30:00.039152+0000 mgr.gibba032.whopeo (mgr.24625) 11446 : cluster [DBG] pgmap v6522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:01.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:01 gibba011 bash[30791]: cluster 2021-03-10T17:30:00.039152+0000 mgr.gibba032.whopeo (mgr.24625) 11446 : cluster [DBG] pgmap v6522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:01 gibba011 bash[31774]: cluster 2021-03-10T17:30:00.039152+0000 mgr.gibba032.whopeo (mgr.24625) 11446 : cluster [DBG] pgmap v6522: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:01.726 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:30:01.954 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:30:01.954 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:30:01.955 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:30:01.955 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:30:01.955 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:30:01.955 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:30:01.955 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:30:01.956 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:30:01.956 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:30:01.956 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:30:01.956 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:30:01.957 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:30:01.957 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:30:01.957 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:30:01.957 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:30:01.957 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:30:01.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:30:01.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:30:01.958 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:30:01.958 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:30:01.958 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:30:02.200 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:30:02.200 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:30:02.200 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:30:02.201 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:02.201 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:30:02.201 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:30:02.201 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:02.202 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:30:02.202 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:30:02.202 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:02.202 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:30:02.202 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:30:02.203 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:30:02.203 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:30:02.203 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:30:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:02 gibba032 bash[24004]: audit 2021-03-10T17:30:01.713972+0000 mgr.gibba032.whopeo (mgr.24625) 11447 : audit [DBG] from='client.61705 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:02 gibba032 bash[24004]: audit 2021-03-10T17:30:01.939137+0000 mgr.gibba032.whopeo (mgr.24625) 11448 : audit [DBG] from='client.51729 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:02 gibba032 bash[24004]: cluster 2021-03-10T17:30:02.040671+0000 mgr.gibba032.whopeo (mgr.24625) 11449 : cluster [DBG] pgmap v6523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:02 gibba032 bash[24004]: audit 2021-03-10T17:30:02.199168+0000 mon.c (mon.1) 223 : audit [DBG] from='client.? 172.21.2.111:0/537931466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[30791]: audit 2021-03-10T17:30:01.713972+0000 mgr.gibba032.whopeo (mgr.24625) 11447 : audit [DBG] from='client.61705 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[30791]: audit 2021-03-10T17:30:01.939137+0000 mgr.gibba032.whopeo (mgr.24625) 11448 : audit [DBG] from='client.51729 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[30791]: cluster 2021-03-10T17:30:02.040671+0000 mgr.gibba032.whopeo (mgr.24625) 11449 : cluster [DBG] pgmap v6523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[30791]: audit 2021-03-10T17:30:02.199168+0000 mon.c (mon.1) 223 : audit [DBG] from='client.? 172.21.2.111:0/537931466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[31774]: audit 2021-03-10T17:30:01.713972+0000 mgr.gibba032.whopeo (mgr.24625) 11447 : audit [DBG] from='client.61705 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[31774]: audit 2021-03-10T17:30:01.939137+0000 mgr.gibba032.whopeo (mgr.24625) 11448 : audit [DBG] from='client.51729 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[31774]: cluster 2021-03-10T17:30:02.040671+0000 mgr.gibba032.whopeo (mgr.24625) 11449 : cluster [DBG] pgmap v6523: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:02 gibba011 bash[31774]: audit 2021-03-10T17:30:02.199168+0000 mon.c (mon.1) 223 : audit [DBG] from='client.? 172.21.2.111:0/537931466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:05.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:05 gibba032 bash[24004]: cluster 2021-03-10T17:30:04.042205+0000 mgr.gibba032.whopeo (mgr.24625) 11450 : cluster [DBG] pgmap v6524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:05.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:05 gibba011 bash[30791]: cluster 2021-03-10T17:30:04.042205+0000 mgr.gibba032.whopeo (mgr.24625) 11450 : cluster [DBG] pgmap v6524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:05.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:05 gibba011 bash[31774]: cluster 2021-03-10T17:30:04.042205+0000 mgr.gibba032.whopeo (mgr.24625) 11450 : cluster [DBG] pgmap v6524: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:07 gibba032 bash[24004]: cluster 2021-03-10T17:30:06.043667+0000 mgr.gibba032.whopeo (mgr.24625) 11451 : cluster [DBG] pgmap v6525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:07 gibba011 bash[30791]: cluster 2021-03-10T17:30:06.043667+0000 mgr.gibba032.whopeo (mgr.24625) 11451 : cluster [DBG] pgmap v6525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:07.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:07 gibba011 bash[31774]: cluster 2021-03-10T17:30:06.043667+0000 mgr.gibba032.whopeo (mgr.24625) 11451 : cluster [DBG] pgmap v6525: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:09 gibba032 bash[24004]: cluster 2021-03-10T17:30:08.044505+0000 mgr.gibba032.whopeo (mgr.24625) 11452 : cluster [DBG] pgmap v6526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:09 gibba011 bash[30791]: cluster 2021-03-10T17:30:08.044505+0000 mgr.gibba032.whopeo (mgr.24625) 11452 : cluster [DBG] pgmap v6526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:09.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:09 gibba011 bash[31774]: cluster 2021-03-10T17:30:08.044505+0000 mgr.gibba032.whopeo (mgr.24625) 11452 : cluster [DBG] pgmap v6526: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:11 gibba032 bash[24004]: cluster 2021-03-10T17:30:10.045277+0000 mgr.gibba032.whopeo (mgr.24625) 11453 : cluster [DBG] pgmap v6527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:11 gibba011 bash[30791]: cluster 2021-03-10T17:30:10.045277+0000 mgr.gibba032.whopeo (mgr.24625) 11453 : cluster [DBG] pgmap v6527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:11.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:11 gibba011 bash[31774]: cluster 2021-03-10T17:30:10.045277+0000 mgr.gibba032.whopeo (mgr.24625) 11453 : cluster [DBG] pgmap v6527: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:12 gibba032 bash[24004]: cluster 2021-03-10T17:30:12.046852+0000 mgr.gibba032.whopeo (mgr.24625) 11454 : cluster [DBG] pgmap v6528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:12 gibba011 bash[30791]: cluster 2021-03-10T17:30:12.046852+0000 mgr.gibba032.whopeo (mgr.24625) 11454 : cluster [DBG] pgmap v6528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:12 gibba011 bash[31774]: cluster 2021-03-10T17:30:12.046852+0000 mgr.gibba032.whopeo (mgr.24625) 11454 : cluster [DBG] pgmap v6528: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:15.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:15 gibba032 bash[24004]: cephadm 2021-03-10T17:30:13.192549+0000 mgr.gibba032.whopeo (mgr.24625) 11455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:15 gibba032 bash[24004]: cluster 2021-03-10T17:30:14.047354+0000 mgr.gibba032.whopeo (mgr.24625) 11456 : cluster [DBG] pgmap v6529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:15 gibba011 bash[30791]: cephadm 2021-03-10T17:30:13.192549+0000 mgr.gibba032.whopeo (mgr.24625) 11455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:15 gibba011 bash[30791]: cluster 2021-03-10T17:30:14.047354+0000 mgr.gibba032.whopeo (mgr.24625) 11456 : cluster [DBG] pgmap v6529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:15 gibba011 bash[31774]: cephadm 2021-03-10T17:30:13.192549+0000 mgr.gibba032.whopeo (mgr.24625) 11455 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:15 gibba011 bash[31774]: cluster 2021-03-10T17:30:14.047354+0000 mgr.gibba032.whopeo (mgr.24625) 11456 : cluster [DBG] pgmap v6529: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:17 gibba032 bash[24004]: cluster 2021-03-10T17:30:16.048648+0000 mgr.gibba032.whopeo (mgr.24625) 11457 : cluster [DBG] pgmap v6530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:17 gibba011 bash[30791]: cluster 2021-03-10T17:30:16.048648+0000 mgr.gibba032.whopeo (mgr.24625) 11457 : cluster [DBG] pgmap v6530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:17.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:17 gibba011 bash[31774]: cluster 2021-03-10T17:30:16.048648+0000 mgr.gibba032.whopeo (mgr.24625) 11457 : cluster [DBG] pgmap v6530: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:19 gibba032 bash[24004]: cluster 2021-03-10T17:30:18.049441+0000 mgr.gibba032.whopeo (mgr.24625) 11458 : cluster [DBG] pgmap v6531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:19 gibba011 bash[30791]: cluster 2021-03-10T17:30:18.049441+0000 mgr.gibba032.whopeo (mgr.24625) 11458 : cluster [DBG] pgmap v6531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:19.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:19 gibba011 bash[31774]: cluster 2021-03-10T17:30:18.049441+0000 mgr.gibba032.whopeo (mgr.24625) 11458 : cluster [DBG] pgmap v6531: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:21 gibba032 bash[24004]: cluster 2021-03-10T17:30:20.050049+0000 mgr.gibba032.whopeo (mgr.24625) 11459 : cluster [DBG] pgmap v6532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:21.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:21 gibba011 bash[30791]: cluster 2021-03-10T17:30:20.050049+0000 mgr.gibba032.whopeo (mgr.24625) 11459 : cluster [DBG] pgmap v6532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:21 gibba011 bash[31774]: cluster 2021-03-10T17:30:20.050049+0000 mgr.gibba032.whopeo (mgr.24625) 11459 : cluster [DBG] pgmap v6532: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:22 gibba032 bash[24004]: cluster 2021-03-10T17:30:22.051319+0000 mgr.gibba032.whopeo (mgr.24625) 11460 : cluster [DBG] pgmap v6533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:22 gibba011 bash[30791]: cluster 2021-03-10T17:30:22.051319+0000 mgr.gibba032.whopeo (mgr.24625) 11460 : cluster [DBG] pgmap v6533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:22 gibba011 bash[31774]: cluster 2021-03-10T17:30:22.051319+0000 mgr.gibba032.whopeo (mgr.24625) 11460 : cluster [DBG] pgmap v6533: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:25.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:25 gibba032 bash[24004]: cluster 2021-03-10T17:30:24.052718+0000 mgr.gibba032.whopeo (mgr.24625) 11461 : cluster [DBG] pgmap v6534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:25.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:25 gibba011 bash[30791]: cluster 2021-03-10T17:30:24.052718+0000 mgr.gibba032.whopeo (mgr.24625) 11461 : cluster [DBG] pgmap v6534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:25.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:25 gibba011 bash[31774]: cluster 2021-03-10T17:30:24.052718+0000 mgr.gibba032.whopeo (mgr.24625) 11461 : cluster [DBG] pgmap v6534: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:27.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:27 gibba032 bash[24004]: cluster 2021-03-10T17:30:26.054103+0000 mgr.gibba032.whopeo (mgr.24625) 11462 : cluster [DBG] pgmap v6535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:27 gibba011 bash[30791]: cluster 2021-03-10T17:30:26.054103+0000 mgr.gibba032.whopeo (mgr.24625) 11462 : cluster [DBG] pgmap v6535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:27.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:27 gibba011 bash[31774]: cluster 2021-03-10T17:30:26.054103+0000 mgr.gibba032.whopeo (mgr.24625) 11462 : cluster [DBG] pgmap v6535: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:29 gibba032 bash[24004]: cluster 2021-03-10T17:30:28.055021+0000 mgr.gibba032.whopeo (mgr.24625) 11463 : cluster [DBG] pgmap v6536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:29 gibba011 bash[30791]: cluster 2021-03-10T17:30:28.055021+0000 mgr.gibba032.whopeo (mgr.24625) 11463 : cluster [DBG] pgmap v6536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:29.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:29 gibba011 bash[31774]: cluster 2021-03-10T17:30:28.055021+0000 mgr.gibba032.whopeo (mgr.24625) 11463 : cluster [DBG] pgmap v6536: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:30 gibba032 bash[24004]: cephadm 2021-03-10T17:30:28.208310+0000 mgr.gibba032.whopeo (mgr.24625) 11464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:30 gibba011 bash[30791]: cephadm 2021-03-10T17:30:28.208310+0000 mgr.gibba032.whopeo (mgr.24625) 11464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:30 gibba011 bash[31774]: cephadm 2021-03-10T17:30:28.208310+0000 mgr.gibba032.whopeo (mgr.24625) 11464 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:31 gibba032 bash[24004]: cluster 2021-03-10T17:30:30.055673+0000 mgr.gibba032.whopeo (mgr.24625) 11465 : cluster [DBG] pgmap v6537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:31 gibba011 bash[30791]: cluster 2021-03-10T17:30:30.055673+0000 mgr.gibba032.whopeo (mgr.24625) 11465 : cluster [DBG] pgmap v6537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:31 gibba011 bash[31774]: cluster 2021-03-10T17:30:30.055673+0000 mgr.gibba032.whopeo (mgr.24625) 11465 : cluster [DBG] pgmap v6537: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:32.440 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:30:32.668 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:30:32.669 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:30:32.669 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:30:32.670 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:30:32.670 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:30:32.671 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:30:32.671 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:30:32.671 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:30:32.672 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:30:32.672 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:30:32.673 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:30:32.673 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:30:32.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:30:32.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:30:32.674 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:30:32.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:30:32.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:30:32.676 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:30:32.676 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:30:32.677 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:30:32.677 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:30:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:32 gibba032 bash[24004]: cluster 2021-03-10T17:30:32.057221+0000 mgr.gibba032.whopeo (mgr.24625) 11466 : cluster [DBG] pgmap v6538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:32.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:32 gibba011 bash[30791]: cluster 2021-03-10T17:30:32.057221+0000 mgr.gibba032.whopeo (mgr.24625) 11466 : cluster [DBG] pgmap v6538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:32 gibba011 bash[31774]: cluster 2021-03-10T17:30:32.057221+0000 mgr.gibba032.whopeo (mgr.24625) 11466 : cluster [DBG] pgmap v6538: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:32.918 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:30:32.918 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:30:32.918 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:30:32.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:32.919 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:30:32.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:30:32.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:32.920 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:30:32.920 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:30:32.920 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:30:32.920 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:30:32.920 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:30:32.921 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:30:32.921 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:30:32.921 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:30:33.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:33 gibba032 bash[24004]: audit 2021-03-10T17:30:32.429396+0000 mgr.gibba032.whopeo (mgr.24625) 11467 : audit [DBG] from='client.51741 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:33 gibba032 bash[24004]: audit 2021-03-10T17:30:32.653762+0000 mgr.gibba032.whopeo (mgr.24625) 11468 : audit [DBG] from='client.51747 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:33 gibba032 bash[24004]: audit 2021-03-10T17:30:32.916923+0000 mon.a (mon.0) 3779 : audit [DBG] from='client.? 172.21.2.111:0/2332148723' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[30791]: audit 2021-03-10T17:30:32.429396+0000 mgr.gibba032.whopeo (mgr.24625) 11467 : audit [DBG] from='client.51741 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[30791]: audit 2021-03-10T17:30:32.653762+0000 mgr.gibba032.whopeo (mgr.24625) 11468 : audit [DBG] from='client.51747 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[30791]: audit 2021-03-10T17:30:32.916923+0000 mon.a (mon.0) 3779 : audit [DBG] from='client.? 172.21.2.111:0/2332148723' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[31774]: audit 2021-03-10T17:30:32.429396+0000 mgr.gibba032.whopeo (mgr.24625) 11467 : audit [DBG] from='client.51741 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[31774]: audit 2021-03-10T17:30:32.653762+0000 mgr.gibba032.whopeo (mgr.24625) 11468 : audit [DBG] from='client.51747 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:30:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:33 gibba011 bash[31774]: audit 2021-03-10T17:30:32.916923+0000 mon.a (mon.0) 3779 : audit [DBG] from='client.? 172.21.2.111:0/2332148723' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:34 gibba032 bash[24004]: cluster 2021-03-10T17:30:34.058826+0000 mgr.gibba032.whopeo (mgr.24625) 11469 : cluster [DBG] pgmap v6539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:34 gibba011 bash[30791]: cluster 2021-03-10T17:30:34.058826+0000 mgr.gibba032.whopeo (mgr.24625) 11469 : cluster [DBG] pgmap v6539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:34 gibba011 bash[31774]: cluster 2021-03-10T17:30:34.058826+0000 mgr.gibba032.whopeo (mgr.24625) 11469 : cluster [DBG] pgmap v6539: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:37 gibba032 bash[24004]: cluster 2021-03-10T17:30:36.060227+0000 mgr.gibba032.whopeo (mgr.24625) 11470 : cluster [DBG] pgmap v6540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:37.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:37 gibba011 bash[30791]: cluster 2021-03-10T17:30:36.060227+0000 mgr.gibba032.whopeo (mgr.24625) 11470 : cluster [DBG] pgmap v6540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:37.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:37 gibba011 bash[31774]: cluster 2021-03-10T17:30:36.060227+0000 mgr.gibba032.whopeo (mgr.24625) 11470 : cluster [DBG] pgmap v6540: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:39.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:39 gibba032 bash[24004]: cluster 2021-03-10T17:30:38.060834+0000 mgr.gibba032.whopeo (mgr.24625) 11471 : cluster [DBG] pgmap v6541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:39.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:39 gibba011 bash[30791]: cluster 2021-03-10T17:30:38.060834+0000 mgr.gibba032.whopeo (mgr.24625) 11471 : cluster [DBG] pgmap v6541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:39.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:39 gibba011 bash[31774]: cluster 2021-03-10T17:30:38.060834+0000 mgr.gibba032.whopeo (mgr.24625) 11471 : cluster [DBG] pgmap v6541: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:41 gibba032 bash[24004]: cluster 2021-03-10T17:30:40.061628+0000 mgr.gibba032.whopeo (mgr.24625) 11472 : cluster [DBG] pgmap v6542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:41.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:41 gibba011 bash[30791]: cluster 2021-03-10T17:30:40.061628+0000 mgr.gibba032.whopeo (mgr.24625) 11472 : cluster [DBG] pgmap v6542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:41 gibba011 bash[31774]: cluster 2021-03-10T17:30:40.061628+0000 mgr.gibba032.whopeo (mgr.24625) 11472 : cluster [DBG] pgmap v6542: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:42 gibba032 bash[24004]: cluster 2021-03-10T17:30:42.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11473 : cluster [DBG] pgmap v6543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:42 gibba011 bash[30791]: cluster 2021-03-10T17:30:42.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11473 : cluster [DBG] pgmap v6543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:42 gibba011 bash[31774]: cluster 2021-03-10T17:30:42.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11473 : cluster [DBG] pgmap v6543: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:44.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.225149+0000 mgr.gibba032.whopeo (mgr.24625) 11474 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:30:44.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.226683+0000 mgr.gibba032.whopeo (mgr.24625) 11475 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:30:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.508895+0000 mon.a (mon.0) 3780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:30:44.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.509490+0000 mgr.gibba032.whopeo (mgr.24625) 11476 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:30:44.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.511651+0000 mgr.gibba032.whopeo (mgr.24625) 11477 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:30:44.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.512271+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:30:44.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.513398+0000 mgr.gibba032.whopeo (mgr.24625) 11478 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:30:44.836 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.513434+0000 mgr.gibba032.whopeo (mgr.24625) 11479 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:30:44.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.514082+0000 mon.a (mon.0) 3782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.514979+0000 mgr.gibba032.whopeo (mgr.24625) 11480 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:30:44.837 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.515016+0000 mgr.gibba032.whopeo (mgr.24625) 11481 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:30:44.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.515697+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.838 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.516286+0000 mgr.gibba032.whopeo (mgr.24625) 11482 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:30:44.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.517176+0000 mon.a (mon.0) 3784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.517867+0000 mgr.gibba032.whopeo (mgr.24625) 11483 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:30:44.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.517906+0000 mgr.gibba032.whopeo (mgr.24625) 11484 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:30:44.839 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.519102+0000 mgr.gibba032.whopeo (mgr.24625) 11485 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:30:44.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.519424+0000 mgr.gibba032.whopeo (mgr.24625) 11486 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:30:44.840 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.520241+0000 mon.a (mon.0) 3785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.520839+0000 mgr.gibba032.whopeo (mgr.24625) 11487 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:30:44.841 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.520877+0000 mgr.gibba032.whopeo (mgr.24625) 11488 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:30:44.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.225149+0000 mgr.gibba032.whopeo (mgr.24625) 11474 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:30:44.843 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.226683+0000 mgr.gibba032.whopeo (mgr.24625) 11475 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:30:44.844 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.508895+0000 mon.a (mon.0) 3780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:30:44.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.509490+0000 mgr.gibba032.whopeo (mgr.24625) 11476 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:30:44.846 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.511651+0000 mgr.gibba032.whopeo (mgr.24625) 11477 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:30:44.847 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.512271+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:30:44.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.513398+0000 mgr.gibba032.whopeo (mgr.24625) 11478 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:30:44.848 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.513434+0000 mgr.gibba032.whopeo (mgr.24625) 11479 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:30:44.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.514082+0000 mon.a (mon.0) 3782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.849 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.514979+0000 mgr.gibba032.whopeo (mgr.24625) 11480 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:30:44.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.515016+0000 mgr.gibba032.whopeo (mgr.24625) 11481 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:30:44.850 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.515697+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.516286+0000 mgr.gibba032.whopeo (mgr.24625) 11482 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:30:44.851 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.517176+0000 mon.a (mon.0) 3784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.517867+0000 mgr.gibba032.whopeo (mgr.24625) 11483 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:30:44.852 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.517906+0000 mgr.gibba032.whopeo (mgr.24625) 11484 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:30:44.853 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.519102+0000 mgr.gibba032.whopeo (mgr.24625) 11485 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:30:44.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.519424+0000 mgr.gibba032.whopeo (mgr.24625) 11486 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:30:44.854 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.520241+0000 mon.a (mon.0) 3785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.520839+0000 mgr.gibba032.whopeo (mgr.24625) 11487 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:30:44.855 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.520877+0000 mgr.gibba032.whopeo (mgr.24625) 11488 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:30:44.857 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cephadm 2021-03-10T17:30:43.520984+0000 mgr.gibba032.whopeo (mgr.24625) 11489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:44.858 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: audit 2021-03-10T17:30:43.521617+0000 mon.a (mon.0) 3786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.858 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[30791]: cluster 2021-03-10T17:30:44.064822+0000 mgr.gibba032.whopeo (mgr.24625) 11490 : cluster [DBG] pgmap v6544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:44.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cephadm 2021-03-10T17:30:43.520984+0000 mgr.gibba032.whopeo (mgr.24625) 11489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:44.860 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: audit 2021-03-10T17:30:43.521617+0000 mon.a (mon.0) 3786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:44.861 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:44 gibba011 bash[31774]: cluster 2021-03-10T17:30:44.064822+0000 mgr.gibba032.whopeo (mgr.24625) 11490 : cluster [DBG] pgmap v6544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:45.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.225149+0000 mgr.gibba032.whopeo (mgr.24625) 11474 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:30:45.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.226683+0000 mgr.gibba032.whopeo (mgr.24625) 11475 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:30:45.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.508895+0000 mon.a (mon.0) 3780 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:30:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.509490+0000 mgr.gibba032.whopeo (mgr.24625) 11476 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:30:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.511651+0000 mgr.gibba032.whopeo (mgr.24625) 11477 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:30:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.512271+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:30:45.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.513398+0000 mgr.gibba032.whopeo (mgr.24625) 11478 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:30:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.513434+0000 mgr.gibba032.whopeo (mgr.24625) 11479 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:30:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.514082+0000 mon.a (mon.0) 3782 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.514979+0000 mgr.gibba032.whopeo (mgr.24625) 11480 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:30:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.515016+0000 mgr.gibba032.whopeo (mgr.24625) 11481 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:30:45.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.515697+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:45.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.516286+0000 mgr.gibba032.whopeo (mgr.24625) 11482 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:30:45.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.517176+0000 mon.a (mon.0) 3784 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:45.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.517867+0000 mgr.gibba032.whopeo (mgr.24625) 11483 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:30:45.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.517906+0000 mgr.gibba032.whopeo (mgr.24625) 11484 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:30:45.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.519102+0000 mgr.gibba032.whopeo (mgr.24625) 11485 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:30:45.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.519424+0000 mgr.gibba032.whopeo (mgr.24625) 11486 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:30:45.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.520241+0000 mon.a (mon.0) 3785 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:45.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.520839+0000 mgr.gibba032.whopeo (mgr.24625) 11487 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:30:45.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.520877+0000 mgr.gibba032.whopeo (mgr.24625) 11488 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:30:45.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cephadm 2021-03-10T17:30:43.520984+0000 mgr.gibba032.whopeo (mgr.24625) 11489 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:30:45.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: audit 2021-03-10T17:30:43.521617+0000 mon.a (mon.0) 3786 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:30:45.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:44 gibba032 bash[24004]: cluster 2021-03-10T17:30:44.064822+0000 mgr.gibba032.whopeo (mgr.24625) 11490 : cluster [DBG] pgmap v6544: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:47.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:47 gibba032 bash[24004]: cluster 2021-03-10T17:30:46.066137+0000 mgr.gibba032.whopeo (mgr.24625) 11491 : cluster [DBG] pgmap v6545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:47 gibba011 bash[30791]: cluster 2021-03-10T17:30:46.066137+0000 mgr.gibba032.whopeo (mgr.24625) 11491 : cluster [DBG] pgmap v6545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:47.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:47 gibba011 bash[31774]: cluster 2021-03-10T17:30:46.066137+0000 mgr.gibba032.whopeo (mgr.24625) 11491 : cluster [DBG] pgmap v6545: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:49.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:49 gibba032 bash[24004]: cluster 2021-03-10T17:30:48.068091+0000 mgr.gibba032.whopeo (mgr.24625) 11492 : cluster [DBG] pgmap v6546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:49.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:49 gibba011 bash[30791]: cluster 2021-03-10T17:30:48.068091+0000 mgr.gibba032.whopeo (mgr.24625) 11492 : cluster [DBG] pgmap v6546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:49.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:49 gibba011 bash[31774]: cluster 2021-03-10T17:30:48.068091+0000 mgr.gibba032.whopeo (mgr.24625) 11492 : cluster [DBG] pgmap v6546: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:51 gibba032 bash[24004]: cluster 2021-03-10T17:30:50.068880+0000 mgr.gibba032.whopeo (mgr.24625) 11493 : cluster [DBG] pgmap v6547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:51 gibba011 bash[30791]: cluster 2021-03-10T17:30:50.068880+0000 mgr.gibba032.whopeo (mgr.24625) 11493 : cluster [DBG] pgmap v6547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:51 gibba011 bash[31774]: cluster 2021-03-10T17:30:50.068880+0000 mgr.gibba032.whopeo (mgr.24625) 11493 : cluster [DBG] pgmap v6547: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:52.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:52 gibba032 bash[24004]: cluster 2021-03-10T17:30:52.070344+0000 mgr.gibba032.whopeo (mgr.24625) 11494 : cluster [DBG] pgmap v6548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:52 gibba011 bash[30791]: cluster 2021-03-10T17:30:52.070344+0000 mgr.gibba032.whopeo (mgr.24625) 11494 : cluster [DBG] pgmap v6548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:52.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:52 gibba011 bash[31774]: cluster 2021-03-10T17:30:52.070344+0000 mgr.gibba032.whopeo (mgr.24625) 11494 : cluster [DBG] pgmap v6548: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:55 gibba032 bash[24004]: cluster 2021-03-10T17:30:54.072057+0000 mgr.gibba032.whopeo (mgr.24625) 11495 : cluster [DBG] pgmap v6549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:55 gibba011 bash[30791]: cluster 2021-03-10T17:30:54.072057+0000 mgr.gibba032.whopeo (mgr.24625) 11495 : cluster [DBG] pgmap v6549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:55 gibba011 bash[31774]: cluster 2021-03-10T17:30:54.072057+0000 mgr.gibba032.whopeo (mgr.24625) 11495 : cluster [DBG] pgmap v6549: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:57 gibba032 bash[24004]: cluster 2021-03-10T17:30:56.073611+0000 mgr.gibba032.whopeo (mgr.24625) 11496 : cluster [DBG] pgmap v6550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:57 gibba011 bash[30791]: cluster 2021-03-10T17:30:56.073611+0000 mgr.gibba032.whopeo (mgr.24625) 11496 : cluster [DBG] pgmap v6550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:57 gibba011 bash[31774]: cluster 2021-03-10T17:30:56.073611+0000 mgr.gibba032.whopeo (mgr.24625) 11496 : cluster [DBG] pgmap v6550: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.518664+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:30:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.519420+0000 mon.a (mon.0) 3788 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:30:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.715690+0000 mon.a (mon.0) 3789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:30:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.716473+0000 mon.a (mon.0) 3790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:30:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.717220+0000 mon.a (mon.0) 3791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:30:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:58 gibba032 bash[24004]: audit 2021-03-10T17:30:57.717938+0000 mon.a (mon.0) 3792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:30:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.518664+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:30:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.519420+0000 mon.a (mon.0) 3788 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:30:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.715690+0000 mon.a (mon.0) 3789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:30:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.716473+0000 mon.a (mon.0) 3790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:30:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.717220+0000 mon.a (mon.0) 3791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:30:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[30791]: audit 2021-03-10T17:30:57.717938+0000 mon.a (mon.0) 3792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:30:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.518664+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:30:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.519420+0000 mon.a (mon.0) 3788 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:30:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.715690+0000 mon.a (mon.0) 3789 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:30:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.716473+0000 mon.a (mon.0) 3790 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:30:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.717220+0000 mon.a (mon.0) 3791 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:30:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:58 gibba011 bash[31774]: audit 2021-03-10T17:30:57.717938+0000 mon.a (mon.0) 3792 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:30:59.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:30:59 gibba032 bash[24004]: cluster 2021-03-10T17:30:58.075463+0000 mgr.gibba032.whopeo (mgr.24625) 11497 : cluster [DBG] pgmap v6551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:30:59 gibba011 bash[30791]: cluster 2021-03-10T17:30:58.075463+0000 mgr.gibba032.whopeo (mgr.24625) 11497 : cluster [DBG] pgmap v6551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:30:59.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:30:59 gibba011 bash[31774]: cluster 2021-03-10T17:30:58.075463+0000 mgr.gibba032.whopeo (mgr.24625) 11497 : cluster [DBG] pgmap v6551: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:00.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:00 gibba032 bash[24004]: cephadm 2021-03-10T17:30:58.534831+0000 mgr.gibba032.whopeo (mgr.24625) 11498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:00 gibba011 bash[30791]: cephadm 2021-03-10T17:30:58.534831+0000 mgr.gibba032.whopeo (mgr.24625) 11498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:00 gibba011 bash[31774]: cephadm 2021-03-10T17:30:58.534831+0000 mgr.gibba032.whopeo (mgr.24625) 11498 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:01 gibba032 bash[24004]: cluster 2021-03-10T17:31:00.076337+0000 mgr.gibba032.whopeo (mgr.24625) 11499 : cluster [DBG] pgmap v6552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:01 gibba011 bash[30791]: cluster 2021-03-10T17:31:00.076337+0000 mgr.gibba032.whopeo (mgr.24625) 11499 : cluster [DBG] pgmap v6552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:01 gibba011 bash[31774]: cluster 2021-03-10T17:31:00.076337+0000 mgr.gibba032.whopeo (mgr.24625) 11499 : cluster [DBG] pgmap v6552: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:02 gibba032 bash[24004]: cluster 2021-03-10T17:31:02.077632+0000 mgr.gibba032.whopeo (mgr.24625) 11500 : cluster [DBG] pgmap v6553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:02 gibba011 bash[30791]: cluster 2021-03-10T17:31:02.077632+0000 mgr.gibba032.whopeo (mgr.24625) 11500 : cluster [DBG] pgmap v6553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:02.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:02 gibba011 bash[31774]: cluster 2021-03-10T17:31:02.077632+0000 mgr.gibba032.whopeo (mgr.24625) 11500 : cluster [DBG] pgmap v6553: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:03.167 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:31:03.408 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:31:03.408 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:31:03.409 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:31:03.409 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:31:03.409 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:31:03.409 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:31:03.410 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:31:03.410 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:31:03.410 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:31:03.410 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:31:03.411 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:31:03.411 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:31:03.411 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:31:03.411 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:31:03.412 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:31:03.412 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:31:03.412 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:31:03.412 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:31:03.413 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:31:03.413 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:31:03.413 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:31:03.653 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:31:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:31:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:31:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:31:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:31:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:31:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:31:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:31:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:31:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:31:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:31:03.879 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:31:03.880 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:03 gibba032 bash[24004]: audit 2021-03-10T17:31:03.156233+0000 mgr.gibba032.whopeo (mgr.24625) 11501 : audit [DBG] from='client.51759 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:03.880 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:03 gibba011 bash[30791]: audit 2021-03-10T17:31:03.156233+0000 mgr.gibba032.whopeo (mgr.24625) 11501 : audit [DBG] from='client.51759 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:03.881 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:03 gibba011 bash[31774]: audit 2021-03-10T17:31:03.156233+0000 mgr.gibba032.whopeo (mgr.24625) 11501 : audit [DBG] from='client.51759 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:04 gibba032 bash[24004]: audit 2021-03-10T17:31:03.380594+0000 mgr.gibba032.whopeo (mgr.24625) 11502 : audit [DBG] from='client.51765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:04 gibba032 bash[24004]: audit 2021-03-10T17:31:03.652222+0000 mon.a (mon.0) 3793 : audit [DBG] from='client.? 172.21.2.111:0/3638141517' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:04 gibba032 bash[24004]: cluster 2021-03-10T17:31:04.079095+0000 mgr.gibba032.whopeo (mgr.24625) 11503 : cluster [DBG] pgmap v6554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[30791]: audit 2021-03-10T17:31:03.380594+0000 mgr.gibba032.whopeo (mgr.24625) 11502 : audit [DBG] from='client.51765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[30791]: audit 2021-03-10T17:31:03.652222+0000 mon.a (mon.0) 3793 : audit [DBG] from='client.? 172.21.2.111:0/3638141517' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:04.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[30791]: cluster 2021-03-10T17:31:04.079095+0000 mgr.gibba032.whopeo (mgr.24625) 11503 : cluster [DBG] pgmap v6554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[31774]: audit 2021-03-10T17:31:03.380594+0000 mgr.gibba032.whopeo (mgr.24625) 11502 : audit [DBG] from='client.51765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[31774]: audit 2021-03-10T17:31:03.652222+0000 mon.a (mon.0) 3793 : audit [DBG] from='client.? 172.21.2.111:0/3638141517' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:04 gibba011 bash[31774]: cluster 2021-03-10T17:31:04.079095+0000 mgr.gibba032.whopeo (mgr.24625) 11503 : cluster [DBG] pgmap v6554: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:07.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:07 gibba032 bash[24004]: cluster 2021-03-10T17:31:06.080668+0000 mgr.gibba032.whopeo (mgr.24625) 11504 : cluster [DBG] pgmap v6555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:07.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:07 gibba011 bash[30791]: cluster 2021-03-10T17:31:06.080668+0000 mgr.gibba032.whopeo (mgr.24625) 11504 : cluster [DBG] pgmap v6555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:07.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:07 gibba011 bash[31774]: cluster 2021-03-10T17:31:06.080668+0000 mgr.gibba032.whopeo (mgr.24625) 11504 : cluster [DBG] pgmap v6555: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:09.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:09 gibba032 bash[24004]: cluster 2021-03-10T17:31:08.081874+0000 mgr.gibba032.whopeo (mgr.24625) 11505 : cluster [DBG] pgmap v6556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:09.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:09 gibba011 bash[30791]: cluster 2021-03-10T17:31:08.081874+0000 mgr.gibba032.whopeo (mgr.24625) 11505 : cluster [DBG] pgmap v6556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:09.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:09 gibba011 bash[31774]: cluster 2021-03-10T17:31:08.081874+0000 mgr.gibba032.whopeo (mgr.24625) 11505 : cluster [DBG] pgmap v6556: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:11 gibba032 bash[24004]: cluster 2021-03-10T17:31:10.083380+0000 mgr.gibba032.whopeo (mgr.24625) 11506 : cluster [DBG] pgmap v6557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:11 gibba011 bash[30791]: cluster 2021-03-10T17:31:10.083380+0000 mgr.gibba032.whopeo (mgr.24625) 11506 : cluster [DBG] pgmap v6557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:11.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:11 gibba011 bash[31774]: cluster 2021-03-10T17:31:10.083380+0000 mgr.gibba032.whopeo (mgr.24625) 11506 : cluster [DBG] pgmap v6557: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:12 gibba032 bash[24004]: cluster 2021-03-10T17:31:12.084893+0000 mgr.gibba032.whopeo (mgr.24625) 11507 : cluster [DBG] pgmap v6558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:12 gibba011 bash[30791]: cluster 2021-03-10T17:31:12.084893+0000 mgr.gibba032.whopeo (mgr.24625) 11507 : cluster [DBG] pgmap v6558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:12 gibba011 bash[31774]: cluster 2021-03-10T17:31:12.084893+0000 mgr.gibba032.whopeo (mgr.24625) 11507 : cluster [DBG] pgmap v6558: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:15.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:15 gibba032 bash[24004]: cephadm 2021-03-10T17:31:13.550775+0000 mgr.gibba032.whopeo (mgr.24625) 11508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:15.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:15 gibba032 bash[24004]: cluster 2021-03-10T17:31:14.086705+0000 mgr.gibba032.whopeo (mgr.24625) 11509 : cluster [DBG] pgmap v6559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:15 gibba011 bash[30791]: cephadm 2021-03-10T17:31:13.550775+0000 mgr.gibba032.whopeo (mgr.24625) 11508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:15.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:15 gibba011 bash[30791]: cluster 2021-03-10T17:31:14.086705+0000 mgr.gibba032.whopeo (mgr.24625) 11509 : cluster [DBG] pgmap v6559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:15 gibba011 bash[31774]: cephadm 2021-03-10T17:31:13.550775+0000 mgr.gibba032.whopeo (mgr.24625) 11508 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:15.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:15 gibba011 bash[31774]: cluster 2021-03-10T17:31:14.086705+0000 mgr.gibba032.whopeo (mgr.24625) 11509 : cluster [DBG] pgmap v6559: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:17.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:17 gibba032 bash[24004]: cluster 2021-03-10T17:31:16.088180+0000 mgr.gibba032.whopeo (mgr.24625) 11510 : cluster [DBG] pgmap v6560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:17 gibba011 bash[30791]: cluster 2021-03-10T17:31:16.088180+0000 mgr.gibba032.whopeo (mgr.24625) 11510 : cluster [DBG] pgmap v6560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:17.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:17 gibba011 bash[31774]: cluster 2021-03-10T17:31:16.088180+0000 mgr.gibba032.whopeo (mgr.24625) 11510 : cluster [DBG] pgmap v6560: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:19.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:19 gibba032 bash[24004]: cluster 2021-03-10T17:31:18.089080+0000 mgr.gibba032.whopeo (mgr.24625) 11511 : cluster [DBG] pgmap v6561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:19.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:19 gibba011 bash[30791]: cluster 2021-03-10T17:31:18.089080+0000 mgr.gibba032.whopeo (mgr.24625) 11511 : cluster [DBG] pgmap v6561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:19.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:19 gibba011 bash[31774]: cluster 2021-03-10T17:31:18.089080+0000 mgr.gibba032.whopeo (mgr.24625) 11511 : cluster [DBG] pgmap v6561: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:21.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:21 gibba032 bash[24004]: cluster 2021-03-10T17:31:20.090149+0000 mgr.gibba032.whopeo (mgr.24625) 11512 : cluster [DBG] pgmap v6562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:21.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:21 gibba011 bash[30791]: cluster 2021-03-10T17:31:20.090149+0000 mgr.gibba032.whopeo (mgr.24625) 11512 : cluster [DBG] pgmap v6562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:21 gibba011 bash[31774]: cluster 2021-03-10T17:31:20.090149+0000 mgr.gibba032.whopeo (mgr.24625) 11512 : cluster [DBG] pgmap v6562: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:22 gibba032 bash[24004]: cluster 2021-03-10T17:31:22.091357+0000 mgr.gibba032.whopeo (mgr.24625) 11513 : cluster [DBG] pgmap v6563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:22 gibba011 bash[30791]: cluster 2021-03-10T17:31:22.091357+0000 mgr.gibba032.whopeo (mgr.24625) 11513 : cluster [DBG] pgmap v6563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:22 gibba011 bash[31774]: cluster 2021-03-10T17:31:22.091357+0000 mgr.gibba032.whopeo (mgr.24625) 11513 : cluster [DBG] pgmap v6563: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:25.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:25 gibba032 bash[24004]: cluster 2021-03-10T17:31:24.092983+0000 mgr.gibba032.whopeo (mgr.24625) 11514 : cluster [DBG] pgmap v6564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:25.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:25 gibba011 bash[30791]: cluster 2021-03-10T17:31:24.092983+0000 mgr.gibba032.whopeo (mgr.24625) 11514 : cluster [DBG] pgmap v6564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:25.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:25 gibba011 bash[31774]: cluster 2021-03-10T17:31:24.092983+0000 mgr.gibba032.whopeo (mgr.24625) 11514 : cluster [DBG] pgmap v6564: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:27.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:27 gibba032 bash[24004]: cluster 2021-03-10T17:31:26.094313+0000 mgr.gibba032.whopeo (mgr.24625) 11515 : cluster [DBG] pgmap v6565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:27.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:27 gibba011 bash[30791]: cluster 2021-03-10T17:31:26.094313+0000 mgr.gibba032.whopeo (mgr.24625) 11515 : cluster [DBG] pgmap v6565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:27.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:27 gibba011 bash[31774]: cluster 2021-03-10T17:31:26.094313+0000 mgr.gibba032.whopeo (mgr.24625) 11515 : cluster [DBG] pgmap v6565: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:29 gibba032 bash[24004]: cluster 2021-03-10T17:31:28.095673+0000 mgr.gibba032.whopeo (mgr.24625) 11516 : cluster [DBG] pgmap v6566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:29 gibba011 bash[30791]: cluster 2021-03-10T17:31:28.095673+0000 mgr.gibba032.whopeo (mgr.24625) 11516 : cluster [DBG] pgmap v6566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:29.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:29 gibba011 bash[31774]: cluster 2021-03-10T17:31:28.095673+0000 mgr.gibba032.whopeo (mgr.24625) 11516 : cluster [DBG] pgmap v6566: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:30.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:30 gibba032 bash[24004]: cephadm 2021-03-10T17:31:28.555753+0000 mgr.gibba032.whopeo (mgr.24625) 11517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:30 gibba011 bash[30791]: cephadm 2021-03-10T17:31:28.555753+0000 mgr.gibba032.whopeo (mgr.24625) 11517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:30 gibba011 bash[31774]: cephadm 2021-03-10T17:31:28.555753+0000 mgr.gibba032.whopeo (mgr.24625) 11517 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:31.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:31 gibba032 bash[24004]: cluster 2021-03-10T17:31:30.096894+0000 mgr.gibba032.whopeo (mgr.24625) 11518 : cluster [DBG] pgmap v6567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:31.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:31 gibba011 bash[30791]: cluster 2021-03-10T17:31:30.096894+0000 mgr.gibba032.whopeo (mgr.24625) 11518 : cluster [DBG] pgmap v6567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:31.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:31 gibba011 bash[31774]: cluster 2021-03-10T17:31:30.096894+0000 mgr.gibba032.whopeo (mgr.24625) 11518 : cluster [DBG] pgmap v6567: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:32 gibba032 bash[24004]: cluster 2021-03-10T17:31:32.098518+0000 mgr.gibba032.whopeo (mgr.24625) 11519 : cluster [DBG] pgmap v6568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:32 gibba011 bash[30791]: cluster 2021-03-10T17:31:32.098518+0000 mgr.gibba032.whopeo (mgr.24625) 11519 : cluster [DBG] pgmap v6568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:32 gibba011 bash[31774]: cluster 2021-03-10T17:31:32.098518+0000 mgr.gibba032.whopeo (mgr.24625) 11519 : cluster [DBG] pgmap v6568: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:33.896 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:31:34.126 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:31:34.127 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 10m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:31:34.127 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:31:34.128 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:31:34.128 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:31:34.128 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:31:34.128 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:31:34.128 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:31:34.129 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:31:34.129 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 10m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:31:34.129 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:31:34.129 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:31:34.130 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:31:34.130 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:31:34.130 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:31:34.131 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:31:34.131 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:31:34.131 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:31:34.132 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:31:34.132 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:31:34.132 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:31:34.378 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:31:34.378 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:31:34.378 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:31:34.379 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:34.379 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:31:34.379 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:31:34.379 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:34.379 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:31:34.380 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:31:34.380 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:31:34.380 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:31:34.380 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:31:34.381 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:31:34.381 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:31:34.381 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:31:35.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:35 gibba032 bash[24004]: audit 2021-03-10T17:31:33.884316+0000 mgr.gibba032.whopeo (mgr.24625) 11520 : audit [DBG] from='client.51777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:35 gibba032 bash[24004]: cluster 2021-03-10T17:31:34.100182+0000 mgr.gibba032.whopeo (mgr.24625) 11521 : cluster [DBG] pgmap v6569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:35 gibba032 bash[24004]: audit 2021-03-10T17:31:34.111779+0000 mgr.gibba032.whopeo (mgr.24625) 11522 : audit [DBG] from='client.61765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:35 gibba032 bash[24004]: audit 2021-03-10T17:31:34.376924+0000 mon.c (mon.1) 224 : audit [DBG] from='client.? 172.21.2.111:0/3001885107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[30791]: audit 2021-03-10T17:31:33.884316+0000 mgr.gibba032.whopeo (mgr.24625) 11520 : audit [DBG] from='client.51777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[30791]: cluster 2021-03-10T17:31:34.100182+0000 mgr.gibba032.whopeo (mgr.24625) 11521 : cluster [DBG] pgmap v6569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:35.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[30791]: audit 2021-03-10T17:31:34.111779+0000 mgr.gibba032.whopeo (mgr.24625) 11522 : audit [DBG] from='client.61765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[30791]: audit 2021-03-10T17:31:34.376924+0000 mon.c (mon.1) 224 : audit [DBG] from='client.? 172.21.2.111:0/3001885107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[31774]: audit 2021-03-10T17:31:33.884316+0000 mgr.gibba032.whopeo (mgr.24625) 11520 : audit [DBG] from='client.51777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[31774]: cluster 2021-03-10T17:31:34.100182+0000 mgr.gibba032.whopeo (mgr.24625) 11521 : cluster [DBG] pgmap v6569: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:35.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[31774]: audit 2021-03-10T17:31:34.111779+0000 mgr.gibba032.whopeo (mgr.24625) 11522 : audit [DBG] from='client.61765 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:31:35.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:35 gibba011 bash[31774]: audit 2021-03-10T17:31:34.376924+0000 mon.c (mon.1) 224 : audit [DBG] from='client.? 172.21.2.111:0/3001885107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:37.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:37 gibba032 bash[24004]: cluster 2021-03-10T17:31:36.101512+0000 mgr.gibba032.whopeo (mgr.24625) 11523 : cluster [DBG] pgmap v6570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:37.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:37 gibba011 bash[30791]: cluster 2021-03-10T17:31:36.101512+0000 mgr.gibba032.whopeo (mgr.24625) 11523 : cluster [DBG] pgmap v6570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:37.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:37 gibba011 bash[31774]: cluster 2021-03-10T17:31:36.101512+0000 mgr.gibba032.whopeo (mgr.24625) 11523 : cluster [DBG] pgmap v6570: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:39.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:39 gibba032 bash[24004]: cluster 2021-03-10T17:31:38.102734+0000 mgr.gibba032.whopeo (mgr.24625) 11524 : cluster [DBG] pgmap v6571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:39.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:39 gibba011 bash[30791]: cluster 2021-03-10T17:31:38.102734+0000 mgr.gibba032.whopeo (mgr.24625) 11524 : cluster [DBG] pgmap v6571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:39.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:39 gibba011 bash[31774]: cluster 2021-03-10T17:31:38.102734+0000 mgr.gibba032.whopeo (mgr.24625) 11524 : cluster [DBG] pgmap v6571: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:41.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:41 gibba032 bash[24004]: cluster 2021-03-10T17:31:40.104258+0000 mgr.gibba032.whopeo (mgr.24625) 11525 : cluster [DBG] pgmap v6572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:41.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:41 gibba011 bash[30791]: cluster 2021-03-10T17:31:40.104258+0000 mgr.gibba032.whopeo (mgr.24625) 11525 : cluster [DBG] pgmap v6572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:41.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:41 gibba011 bash[31774]: cluster 2021-03-10T17:31:40.104258+0000 mgr.gibba032.whopeo (mgr.24625) 11525 : cluster [DBG] pgmap v6572: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:42 gibba032 bash[24004]: cluster 2021-03-10T17:31:42.105724+0000 mgr.gibba032.whopeo (mgr.24625) 11526 : cluster [DBG] pgmap v6573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:42 gibba011 bash[30791]: cluster 2021-03-10T17:31:42.105724+0000 mgr.gibba032.whopeo (mgr.24625) 11526 : cluster [DBG] pgmap v6573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:42.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:42 gibba011 bash[31774]: cluster 2021-03-10T17:31:42.105724+0000 mgr.gibba032.whopeo (mgr.24625) 11526 : cluster [DBG] pgmap v6573: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:44.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:43 gibba032 bash[24004]: audit 2021-03-10T17:31:43.575950+0000 mon.a (mon.0) 3794 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:31:44.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:43 gibba011 bash[30791]: audit 2021-03-10T17:31:43.575950+0000 mon.a (mon.0) 3794 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:31:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:43 gibba011 bash[31774]: audit 2021-03-10T17:31:43.575950+0000 mon.a (mon.0) 3794 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:31:45.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:44 gibba032 bash[24004]: cephadm 2021-03-10T17:31:43.573460+0000 mgr.gibba032.whopeo (mgr.24625) 11527 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:31:45.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:44 gibba032 bash[24004]: cluster 2021-03-10T17:31:44.107188+0000 mgr.gibba032.whopeo (mgr.24625) 11528 : cluster [DBG] pgmap v6574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:44 gibba011 bash[30791]: cephadm 2021-03-10T17:31:43.573460+0000 mgr.gibba032.whopeo (mgr.24625) 11527 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:31:45.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:44 gibba011 bash[30791]: cluster 2021-03-10T17:31:44.107188+0000 mgr.gibba032.whopeo (mgr.24625) 11528 : cluster [DBG] pgmap v6574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:45.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:44 gibba011 bash[31774]: cephadm 2021-03-10T17:31:43.573460+0000 mgr.gibba032.whopeo (mgr.24625) 11527 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:31:45.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:44 gibba011 bash[31774]: cluster 2021-03-10T17:31:44.107188+0000 mgr.gibba032.whopeo (mgr.24625) 11528 : cluster [DBG] pgmap v6574: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:47.432 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:47 gibba032 bash[24004]: cluster 2021-03-10T17:31:46.108653+0000 mgr.gibba032.whopeo (mgr.24625) 11529 : cluster [DBG] pgmap v6575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:47.432 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:47 gibba032 bash[24004]: audit 2021-03-10T17:31:47.130588+0000 mon.a (mon.0) 3795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:47 gibba011 bash[30791]: cluster 2021-03-10T17:31:46.108653+0000 mgr.gibba032.whopeo (mgr.24625) 11529 : cluster [DBG] pgmap v6575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:47.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:47 gibba011 bash[30791]: audit 2021-03-10T17:31:47.130588+0000 mon.a (mon.0) 3795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:47 gibba011 bash[31774]: cluster 2021-03-10T17:31:46.108653+0000 mgr.gibba032.whopeo (mgr.24625) 11529 : cluster [DBG] pgmap v6575: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:47.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:47 gibba011 bash[31774]: audit 2021-03-10T17:31:47.130588+0000 mon.a (mon.0) 3795 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.130393+0000 mgr.gibba032.whopeo (mgr.24625) 11530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:31:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.412475+0000 mon.a (mon.0) 3796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.419793+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:31:48.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.421394+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.423442+0000 mon.a (mon.0) 3799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.425140+0000 mon.a (mon.0) 3800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.428212+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:48 gibba032 bash[24004]: audit 2021-03-10T17:31:47.430273+0000 mon.a (mon.0) 3802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.586 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.130393+0000 mgr.gibba032.whopeo (mgr.24625) 11530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:31:48.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.412475+0000 mon.a (mon.0) 3796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:48.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.419793+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:31:48.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.421394+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.587 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.423442+0000 mon.a (mon.0) 3799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.425140+0000 mon.a (mon.0) 3800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.428212+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.588 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[30791]: audit 2021-03-10T17:31:47.430273+0000 mon.a (mon.0) 3802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.130393+0000 mgr.gibba032.whopeo (mgr.24625) 11530 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:31:48.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.412475+0000 mon.a (mon.0) 3796 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:31:48.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.419793+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:31:48.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.421394+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.423442+0000 mon.a (mon.0) 3799 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.425140+0000 mon.a (mon.0) 3800 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.428212+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:48.591 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:48 gibba011 bash[31774]: audit 2021-03-10T17:31:47.430273+0000 mon.a (mon.0) 3802 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:31:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.417229+0000 mgr.gibba032.whopeo (mgr.24625) 11531 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:31:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.419096+0000 mgr.gibba032.whopeo (mgr.24625) 11532 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:31:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.421022+0000 mgr.gibba032.whopeo (mgr.24625) 11533 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:31:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.421063+0000 mgr.gibba032.whopeo (mgr.24625) 11534 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:31:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.422854+0000 mgr.gibba032.whopeo (mgr.24625) 11535 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:31:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.422896+0000 mgr.gibba032.whopeo (mgr.24625) 11536 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:31:49.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.424395+0000 mgr.gibba032.whopeo (mgr.24625) 11537 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:31:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.425787+0000 mgr.gibba032.whopeo (mgr.24625) 11538 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:31:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.425826+0000 mgr.gibba032.whopeo (mgr.24625) 11539 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:31:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.427480+0000 mgr.gibba032.whopeo (mgr.24625) 11540 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:31:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.427952+0000 mgr.gibba032.whopeo (mgr.24625) 11541 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:31:49.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.429490+0000 mgr.gibba032.whopeo (mgr.24625) 11542 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:31:49.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.429535+0000 mgr.gibba032.whopeo (mgr.24625) 11543 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:31:49.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cephadm 2021-03-10T17:31:47.429662+0000 mgr.gibba032.whopeo (mgr.24625) 11544 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:49.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:49 gibba032 bash[24004]: cluster 2021-03-10T17:31:48.110493+0000 mgr.gibba032.whopeo (mgr.24625) 11545 : cluster [DBG] pgmap v6576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.417229+0000 mgr.gibba032.whopeo (mgr.24625) 11531 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:31:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.419096+0000 mgr.gibba032.whopeo (mgr.24625) 11532 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:31:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.421022+0000 mgr.gibba032.whopeo (mgr.24625) 11533 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:31:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.421063+0000 mgr.gibba032.whopeo (mgr.24625) 11534 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:31:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.422854+0000 mgr.gibba032.whopeo (mgr.24625) 11535 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:31:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.422896+0000 mgr.gibba032.whopeo (mgr.24625) 11536 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:31:49.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.424395+0000 mgr.gibba032.whopeo (mgr.24625) 11537 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:31:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.425787+0000 mgr.gibba032.whopeo (mgr.24625) 11538 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:31:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.425826+0000 mgr.gibba032.whopeo (mgr.24625) 11539 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:31:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.427480+0000 mgr.gibba032.whopeo (mgr.24625) 11540 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:31:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.427952+0000 mgr.gibba032.whopeo (mgr.24625) 11541 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:31:49.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.429490+0000 mgr.gibba032.whopeo (mgr.24625) 11542 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:31:49.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.429535+0000 mgr.gibba032.whopeo (mgr.24625) 11543 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:31:49.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cephadm 2021-03-10T17:31:47.429662+0000 mgr.gibba032.whopeo (mgr.24625) 11544 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:49.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[30791]: cluster 2021-03-10T17:31:48.110493+0000 mgr.gibba032.whopeo (mgr.24625) 11545 : cluster [DBG] pgmap v6576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:49.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.417229+0000 mgr.gibba032.whopeo (mgr.24625) 11531 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:31:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.419096+0000 mgr.gibba032.whopeo (mgr.24625) 11532 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:31:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.421022+0000 mgr.gibba032.whopeo (mgr.24625) 11533 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:31:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.421063+0000 mgr.gibba032.whopeo (mgr.24625) 11534 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:31:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.422854+0000 mgr.gibba032.whopeo (mgr.24625) 11535 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:31:49.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.422896+0000 mgr.gibba032.whopeo (mgr.24625) 11536 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:31:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.424395+0000 mgr.gibba032.whopeo (mgr.24625) 11537 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:31:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.425787+0000 mgr.gibba032.whopeo (mgr.24625) 11538 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:31:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.425826+0000 mgr.gibba032.whopeo (mgr.24625) 11539 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:31:49.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.427480+0000 mgr.gibba032.whopeo (mgr.24625) 11540 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:31:49.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.427952+0000 mgr.gibba032.whopeo (mgr.24625) 11541 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:31:49.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.429490+0000 mgr.gibba032.whopeo (mgr.24625) 11542 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:31:49.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.429535+0000 mgr.gibba032.whopeo (mgr.24625) 11543 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:31:49.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cephadm 2021-03-10T17:31:47.429662+0000 mgr.gibba032.whopeo (mgr.24625) 11544 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:31:49.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:49 gibba011 bash[31774]: cluster 2021-03-10T17:31:48.110493+0000 mgr.gibba032.whopeo (mgr.24625) 11545 : cluster [DBG] pgmap v6576: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:51.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:51 gibba032 bash[24004]: cluster 2021-03-10T17:31:50.112192+0000 mgr.gibba032.whopeo (mgr.24625) 11546 : cluster [DBG] pgmap v6577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:51.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:51 gibba011 bash[30791]: cluster 2021-03-10T17:31:50.112192+0000 mgr.gibba032.whopeo (mgr.24625) 11546 : cluster [DBG] pgmap v6577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:51.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:51 gibba011 bash[31774]: cluster 2021-03-10T17:31:50.112192+0000 mgr.gibba032.whopeo (mgr.24625) 11546 : cluster [DBG] pgmap v6577: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:52 gibba032 bash[24004]: cluster 2021-03-10T17:31:52.113558+0000 mgr.gibba032.whopeo (mgr.24625) 11547 : cluster [DBG] pgmap v6578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:52 gibba011 bash[30791]: cluster 2021-03-10T17:31:52.113558+0000 mgr.gibba032.whopeo (mgr.24625) 11547 : cluster [DBG] pgmap v6578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:52 gibba011 bash[31774]: cluster 2021-03-10T17:31:52.113558+0000 mgr.gibba032.whopeo (mgr.24625) 11547 : cluster [DBG] pgmap v6578: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:55.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:55 gibba032 bash[24004]: cluster 2021-03-10T17:31:54.115191+0000 mgr.gibba032.whopeo (mgr.24625) 11548 : cluster [DBG] pgmap v6579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:55 gibba011 bash[30791]: cluster 2021-03-10T17:31:54.115191+0000 mgr.gibba032.whopeo (mgr.24625) 11548 : cluster [DBG] pgmap v6579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:55.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:55 gibba011 bash[31774]: cluster 2021-03-10T17:31:54.115191+0000 mgr.gibba032.whopeo (mgr.24625) 11548 : cluster [DBG] pgmap v6579: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:57 gibba032 bash[24004]: cluster 2021-03-10T17:31:56.116575+0000 mgr.gibba032.whopeo (mgr.24625) 11549 : cluster [DBG] pgmap v6580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:57 gibba011 bash[30791]: cluster 2021-03-10T17:31:56.116575+0000 mgr.gibba032.whopeo (mgr.24625) 11549 : cluster [DBG] pgmap v6580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:57.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:57 gibba011 bash[31774]: cluster 2021-03-10T17:31:56.116575+0000 mgr.gibba032.whopeo (mgr.24625) 11549 : cluster [DBG] pgmap v6580: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:58.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.527731+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.528762+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.725267+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.726400+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.727475+0000 mon.a (mon.0) 3807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:31:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:58 gibba032 bash[24004]: audit 2021-03-10T17:31:57.728564+0000 mon.a (mon.0) 3808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:31:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.527731+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:31:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.528762+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:31:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.725267+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:31:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.726400+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:31:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.727475+0000 mon.a (mon.0) 3807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:31:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[30791]: audit 2021-03-10T17:31:57.728564+0000 mon.a (mon.0) 3808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:31:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.527731+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:31:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.528762+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:31:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.725267+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2021-03-10T17:31:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.726400+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:31:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.727475+0000 mon.a (mon.0) 3807 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:31:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:58 gibba011 bash[31774]: audit 2021-03-10T17:31:57.728564+0000 mon.a (mon.0) 3808 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:31:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:31:59 gibba032 bash[24004]: cluster 2021-03-10T17:31:58.118309+0000 mgr.gibba032.whopeo (mgr.24625) 11550 : cluster [DBG] pgmap v6581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:31:59 gibba011 bash[30791]: cluster 2021-03-10T17:31:58.118309+0000 mgr.gibba032.whopeo (mgr.24625) 11550 : cluster [DBG] pgmap v6581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:31:59.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:31:59 gibba011 bash[31774]: cluster 2021-03-10T17:31:58.118309+0000 mgr.gibba032.whopeo (mgr.24625) 11550 : cluster [DBG] pgmap v6581: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:00 gibba032 bash[24004]: cluster 2021-03-10T17:32:00.119238+0000 mgr.gibba032.whopeo (mgr.24625) 11551 : cluster [DBG] pgmap v6582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:00 gibba011 bash[30791]: cluster 2021-03-10T17:32:00.119238+0000 mgr.gibba032.whopeo (mgr.24625) 11551 : cluster [DBG] pgmap v6582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:00.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:00 gibba011 bash[31774]: cluster 2021-03-10T17:32:00.119238+0000 mgr.gibba032.whopeo (mgr.24625) 11551 : cluster [DBG] pgmap v6582: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:02 gibba032 bash[24004]: cluster 2021-03-10T17:32:02.120689+0000 mgr.gibba032.whopeo (mgr.24625) 11552 : cluster [DBG] pgmap v6583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:02 gibba011 bash[30791]: cluster 2021-03-10T17:32:02.120689+0000 mgr.gibba032.whopeo (mgr.24625) 11552 : cluster [DBG] pgmap v6583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:02.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:02 gibba011 bash[31774]: cluster 2021-03-10T17:32:02.120689+0000 mgr.gibba032.whopeo (mgr.24625) 11552 : cluster [DBG] pgmap v6583: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:03.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:03 gibba032 bash[24004]: cephadm 2021-03-10T17:32:02.444514+0000 mgr.gibba032.whopeo (mgr.24625) 11553 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:03 gibba011 bash[30791]: cephadm 2021-03-10T17:32:02.444514+0000 mgr.gibba032.whopeo (mgr.24625) 11553 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:03 gibba011 bash[31774]: cephadm 2021-03-10T17:32:02.444514+0000 mgr.gibba032.whopeo (mgr.24625) 11553 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:04.627 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:32:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:04 gibba032 bash[24004]: cluster 2021-03-10T17:32:04.122148+0000 mgr.gibba032.whopeo (mgr.24625) 11554 : cluster [DBG] pgmap v6584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:04.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:04 gibba011 bash[30791]: cluster 2021-03-10T17:32:04.122148+0000 mgr.gibba032.whopeo (mgr.24625) 11554 : cluster [DBG] pgmap v6584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:04.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:04 gibba011 bash[31774]: cluster 2021-03-10T17:32:04.122148+0000 mgr.gibba032.whopeo (mgr.24625) 11554 : cluster [DBG] pgmap v6584: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:04.857 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:32:04.858 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 17s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:32:04.858 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:32:04.858 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:32:04.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:32:04.859 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:32:04.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:32:04.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:32:04.859 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:32:04.860 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 17s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:32:04.860 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:32:04.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:32:04.860 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:32:04.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:32:04.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 17s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:32:04.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:32:04.861 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:32:04.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:32:04.862 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:32:04.862 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:32:04.862 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 17s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:32:05.111 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:32:05.111 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:32:05.112 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:32:05.112 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:05.113 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:32:05.113 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:32:05.113 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:05.114 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:32:05.114 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:32:05.115 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:05.115 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:32:05.115 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:32:05.116 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:32:05.116 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:32:05.116 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:32:05.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:05 gibba032 bash[24004]: audit 2021-03-10T17:32:04.616202+0000 mgr.gibba032.whopeo (mgr.24625) 11555 : audit [DBG] from='client.61777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:05 gibba032 bash[24004]: audit 2021-03-10T17:32:04.842715+0000 mgr.gibba032.whopeo (mgr.24625) 11556 : audit [DBG] from='client.51801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:05 gibba032 bash[24004]: audit 2021-03-10T17:32:05.109897+0000 mon.a (mon.0) 3809 : audit [DBG] from='client.? 172.21.2.111:0/3793950767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:05.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[30791]: audit 2021-03-10T17:32:04.616202+0000 mgr.gibba032.whopeo (mgr.24625) 11555 : audit [DBG] from='client.61777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[30791]: audit 2021-03-10T17:32:04.842715+0000 mgr.gibba032.whopeo (mgr.24625) 11556 : audit [DBG] from='client.51801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[30791]: audit 2021-03-10T17:32:05.109897+0000 mon.a (mon.0) 3809 : audit [DBG] from='client.? 172.21.2.111:0/3793950767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[31774]: audit 2021-03-10T17:32:04.616202+0000 mgr.gibba032.whopeo (mgr.24625) 11555 : audit [DBG] from='client.61777 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[31774]: audit 2021-03-10T17:32:04.842715+0000 mgr.gibba032.whopeo (mgr.24625) 11556 : audit [DBG] from='client.51801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:05 gibba011 bash[31774]: audit 2021-03-10T17:32:05.109897+0000 mon.a (mon.0) 3809 : audit [DBG] from='client.? 172.21.2.111:0/3793950767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:06 gibba032 bash[24004]: cluster 2021-03-10T17:32:06.123692+0000 mgr.gibba032.whopeo (mgr.24625) 11557 : cluster [DBG] pgmap v6585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:06 gibba011 bash[30791]: cluster 2021-03-10T17:32:06.123692+0000 mgr.gibba032.whopeo (mgr.24625) 11557 : cluster [DBG] pgmap v6585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:06.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:06 gibba011 bash[31774]: cluster 2021-03-10T17:32:06.123692+0000 mgr.gibba032.whopeo (mgr.24625) 11557 : cluster [DBG] pgmap v6585: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:09.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:09 gibba032 bash[24004]: cluster 2021-03-10T17:32:08.125505+0000 mgr.gibba032.whopeo (mgr.24625) 11558 : cluster [DBG] pgmap v6586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:09.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:09 gibba011 bash[30791]: cluster 2021-03-10T17:32:08.125505+0000 mgr.gibba032.whopeo (mgr.24625) 11558 : cluster [DBG] pgmap v6586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:09.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:09 gibba011 bash[31774]: cluster 2021-03-10T17:32:08.125505+0000 mgr.gibba032.whopeo (mgr.24625) 11558 : cluster [DBG] pgmap v6586: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:11.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:11 gibba032 bash[24004]: cluster 2021-03-10T17:32:10.127086+0000 mgr.gibba032.whopeo (mgr.24625) 11559 : cluster [DBG] pgmap v6587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:11.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:11 gibba011 bash[30791]: cluster 2021-03-10T17:32:10.127086+0000 mgr.gibba032.whopeo (mgr.24625) 11559 : cluster [DBG] pgmap v6587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:11.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:11 gibba011 bash[31774]: cluster 2021-03-10T17:32:10.127086+0000 mgr.gibba032.whopeo (mgr.24625) 11559 : cluster [DBG] pgmap v6587: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:12 gibba032 bash[24004]: cluster 2021-03-10T17:32:12.128643+0000 mgr.gibba032.whopeo (mgr.24625) 11560 : cluster [DBG] pgmap v6588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:12 gibba011 bash[30791]: cluster 2021-03-10T17:32:12.128643+0000 mgr.gibba032.whopeo (mgr.24625) 11560 : cluster [DBG] pgmap v6588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:12.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:12 gibba011 bash[31774]: cluster 2021-03-10T17:32:12.128643+0000 mgr.gibba032.whopeo (mgr.24625) 11560 : cluster [DBG] pgmap v6588: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:15.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:15 gibba032 bash[24004]: cluster 2021-03-10T17:32:14.130239+0000 mgr.gibba032.whopeo (mgr.24625) 11561 : cluster [DBG] pgmap v6589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:15.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:15 gibba011 bash[30791]: cluster 2021-03-10T17:32:14.130239+0000 mgr.gibba032.whopeo (mgr.24625) 11561 : cluster [DBG] pgmap v6589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:15.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:15 gibba011 bash[31774]: cluster 2021-03-10T17:32:14.130239+0000 mgr.gibba032.whopeo (mgr.24625) 11561 : cluster [DBG] pgmap v6589: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:17.461 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:17 gibba032 bash[24004]: cluster 2021-03-10T17:32:16.131600+0000 mgr.gibba032.whopeo (mgr.24625) 11562 : cluster [DBG] pgmap v6590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:17.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:17 gibba011 bash[30791]: cluster 2021-03-10T17:32:16.131600+0000 mgr.gibba032.whopeo (mgr.24625) 11562 : cluster [DBG] pgmap v6590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:17.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:17 gibba011 bash[31774]: cluster 2021-03-10T17:32:16.131600+0000 mgr.gibba032.whopeo (mgr.24625) 11562 : cluster [DBG] pgmap v6590: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:18.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:18 gibba032 bash[24004]: cephadm 2021-03-10T17:32:17.457798+0000 mgr.gibba032.whopeo (mgr.24625) 11563 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:18 gibba032 bash[24004]: cluster 2021-03-10T17:32:18.133209+0000 mgr.gibba032.whopeo (mgr.24625) 11564 : cluster [DBG] pgmap v6591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:18 gibba011 bash[30791]: cephadm 2021-03-10T17:32:17.457798+0000 mgr.gibba032.whopeo (mgr.24625) 11563 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:18 gibba011 bash[30791]: cluster 2021-03-10T17:32:18.133209+0000 mgr.gibba032.whopeo (mgr.24625) 11564 : cluster [DBG] pgmap v6591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:18 gibba011 bash[31774]: cephadm 2021-03-10T17:32:17.457798+0000 mgr.gibba032.whopeo (mgr.24625) 11563 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:18 gibba011 bash[31774]: cluster 2021-03-10T17:32:18.133209+0000 mgr.gibba032.whopeo (mgr.24625) 11564 : cluster [DBG] pgmap v6591: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:21.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:21 gibba032 bash[24004]: cluster 2021-03-10T17:32:20.134770+0000 mgr.gibba032.whopeo (mgr.24625) 11565 : cluster [DBG] pgmap v6592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:21.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:21 gibba011 bash[30791]: cluster 2021-03-10T17:32:20.134770+0000 mgr.gibba032.whopeo (mgr.24625) 11565 : cluster [DBG] pgmap v6592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:21.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:21 gibba011 bash[31774]: cluster 2021-03-10T17:32:20.134770+0000 mgr.gibba032.whopeo (mgr.24625) 11565 : cluster [DBG] pgmap v6592: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:22.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:22 gibba032 bash[24004]: cluster 2021-03-10T17:32:22.136327+0000 mgr.gibba032.whopeo (mgr.24625) 11566 : cluster [DBG] pgmap v6593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:22 gibba011 bash[30791]: cluster 2021-03-10T17:32:22.136327+0000 mgr.gibba032.whopeo (mgr.24625) 11566 : cluster [DBG] pgmap v6593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:22 gibba011 bash[31774]: cluster 2021-03-10T17:32:22.136327+0000 mgr.gibba032.whopeo (mgr.24625) 11566 : cluster [DBG] pgmap v6593: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:25.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:25 gibba032 bash[24004]: cluster 2021-03-10T17:32:24.137932+0000 mgr.gibba032.whopeo (mgr.24625) 11567 : cluster [DBG] pgmap v6594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:25.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:25 gibba011 bash[30791]: cluster 2021-03-10T17:32:24.137932+0000 mgr.gibba032.whopeo (mgr.24625) 11567 : cluster [DBG] pgmap v6594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:25.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:25 gibba011 bash[31774]: cluster 2021-03-10T17:32:24.137932+0000 mgr.gibba032.whopeo (mgr.24625) 11567 : cluster [DBG] pgmap v6594: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:26 gibba011 bash[30791]: cluster 2021-03-10T17:32:26.139298+0000 mgr.gibba032.whopeo (mgr.24625) 11568 : cluster [DBG] pgmap v6595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:26.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:26 gibba011 bash[31774]: cluster 2021-03-10T17:32:26.139298+0000 mgr.gibba032.whopeo (mgr.24625) 11568 : cluster [DBG] pgmap v6595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:26.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:26 gibba032 bash[24004]: cluster 2021-03-10T17:32:26.139298+0000 mgr.gibba032.whopeo (mgr.24625) 11568 : cluster [DBG] pgmap v6595: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:29.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:29 gibba032 bash[24004]: cluster 2021-03-10T17:32:28.141251+0000 mgr.gibba032.whopeo (mgr.24625) 11569 : cluster [DBG] pgmap v6596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:29.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:29 gibba011 bash[30791]: cluster 2021-03-10T17:32:28.141251+0000 mgr.gibba032.whopeo (mgr.24625) 11569 : cluster [DBG] pgmap v6596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:29.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:29 gibba011 bash[31774]: cluster 2021-03-10T17:32:28.141251+0000 mgr.gibba032.whopeo (mgr.24625) 11569 : cluster [DBG] pgmap v6596: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:30 gibba011 bash[30791]: cluster 2021-03-10T17:32:30.142812+0000 mgr.gibba032.whopeo (mgr.24625) 11570 : cluster [DBG] pgmap v6597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:30.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:30 gibba011 bash[31774]: cluster 2021-03-10T17:32:30.142812+0000 mgr.gibba032.whopeo (mgr.24625) 11570 : cluster [DBG] pgmap v6597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:30 gibba032 bash[24004]: cluster 2021-03-10T17:32:30.142812+0000 mgr.gibba032.whopeo (mgr.24625) 11570 : cluster [DBG] pgmap v6597: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:32 gibba032 bash[24004]: cluster 2021-03-10T17:32:32.144181+0000 mgr.gibba032.whopeo (mgr.24625) 11571 : cluster [DBG] pgmap v6598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:32 gibba011 bash[30791]: cluster 2021-03-10T17:32:32.144181+0000 mgr.gibba032.whopeo (mgr.24625) 11571 : cluster [DBG] pgmap v6598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:32.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:32 gibba011 bash[31774]: cluster 2021-03-10T17:32:32.144181+0000 mgr.gibba032.whopeo (mgr.24625) 11571 : cluster [DBG] pgmap v6598: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:33 gibba032 bash[24004]: cephadm 2021-03-10T17:32:32.473325+0000 mgr.gibba032.whopeo (mgr.24625) 11572 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:33.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:33 gibba011 bash[30791]: cephadm 2021-03-10T17:32:32.473325+0000 mgr.gibba032.whopeo (mgr.24625) 11572 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:33.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:33 gibba011 bash[31774]: cephadm 2021-03-10T17:32:32.473325+0000 mgr.gibba032.whopeo (mgr.24625) 11572 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:34.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:34 gibba032 bash[24004]: cluster 2021-03-10T17:32:34.145858+0000 mgr.gibba032.whopeo (mgr.24625) 11573 : cluster [DBG] pgmap v6599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:34 gibba011 bash[30791]: cluster 2021-03-10T17:32:34.145858+0000 mgr.gibba032.whopeo (mgr.24625) 11573 : cluster [DBG] pgmap v6599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:34.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:34 gibba011 bash[31774]: cluster 2021-03-10T17:32:34.145858+0000 mgr.gibba032.whopeo (mgr.24625) 11573 : cluster [DBG] pgmap v6599: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:35.395 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:32:35.624 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:32:35.625 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 48s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:32:35.625 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:32:35.625 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:32:35.625 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:32:35.625 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:32:35.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:32:35.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:32:35.626 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:32:35.626 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 48s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:32:35.627 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:32:35.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:32:35.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:32:35.627 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:32:35.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 48s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:32:35.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:32:35.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:32:35.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:32:35.628 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:32:35.629 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:32:35.629 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 48s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:32:35.871 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:32:35.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:32:35.872 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:32:35.872 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:35.872 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:32:35.872 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:32:35.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:35.873 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:32:35.873 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:32:35.873 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:32:35.874 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:32:35.874 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:32:35.874 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:32:35.874 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:32:35.875 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:32:36.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:35 gibba032 bash[24004]: audit 2021-03-10T17:32:35.870316+0000 mon.a (mon.0) 3810 : audit [DBG] from='client.? 172.21.2.111:0/3025452021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:35 gibba011 bash[30791]: audit 2021-03-10T17:32:35.870316+0000 mon.a (mon.0) 3810 : audit [DBG] from='client.? 172.21.2.111:0/3025452021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:35 gibba011 bash[31774]: audit 2021-03-10T17:32:35.870316+0000 mon.a (mon.0) 3810 : audit [DBG] from='client.? 172.21.2.111:0/3025452021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:37.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:36 gibba032 bash[24004]: audit 2021-03-10T17:32:35.383738+0000 mgr.gibba032.whopeo (mgr.24625) 11574 : audit [DBG] from='client.51813 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:36 gibba032 bash[24004]: audit 2021-03-10T17:32:35.609167+0000 mgr.gibba032.whopeo (mgr.24625) 11575 : audit [DBG] from='client.61801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:36 gibba032 bash[24004]: cluster 2021-03-10T17:32:36.147224+0000 mgr.gibba032.whopeo (mgr.24625) 11576 : cluster [DBG] pgmap v6600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:37.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[30791]: audit 2021-03-10T17:32:35.383738+0000 mgr.gibba032.whopeo (mgr.24625) 11574 : audit [DBG] from='client.51813 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[30791]: audit 2021-03-10T17:32:35.609167+0000 mgr.gibba032.whopeo (mgr.24625) 11575 : audit [DBG] from='client.61801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[30791]: cluster 2021-03-10T17:32:36.147224+0000 mgr.gibba032.whopeo (mgr.24625) 11576 : cluster [DBG] pgmap v6600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:37.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[31774]: audit 2021-03-10T17:32:35.383738+0000 mgr.gibba032.whopeo (mgr.24625) 11574 : audit [DBG] from='client.51813 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[31774]: audit 2021-03-10T17:32:35.609167+0000 mgr.gibba032.whopeo (mgr.24625) 11575 : audit [DBG] from='client.61801 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:32:37.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:36 gibba011 bash[31774]: cluster 2021-03-10T17:32:36.147224+0000 mgr.gibba032.whopeo (mgr.24625) 11576 : cluster [DBG] pgmap v6600: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:39.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:39 gibba032 bash[24004]: cluster 2021-03-10T17:32:38.149154+0000 mgr.gibba032.whopeo (mgr.24625) 11577 : cluster [DBG] pgmap v6601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:39.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:39 gibba011 bash[30791]: cluster 2021-03-10T17:32:38.149154+0000 mgr.gibba032.whopeo (mgr.24625) 11577 : cluster [DBG] pgmap v6601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:39.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:39 gibba011 bash[31774]: cluster 2021-03-10T17:32:38.149154+0000 mgr.gibba032.whopeo (mgr.24625) 11577 : cluster [DBG] pgmap v6601: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:40 gibba032 bash[24004]: cluster 2021-03-10T17:32:40.150795+0000 mgr.gibba032.whopeo (mgr.24625) 11578 : cluster [DBG] pgmap v6602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:40 gibba011 bash[30791]: cluster 2021-03-10T17:32:40.150795+0000 mgr.gibba032.whopeo (mgr.24625) 11578 : cluster [DBG] pgmap v6602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:40 gibba011 bash[31774]: cluster 2021-03-10T17:32:40.150795+0000 mgr.gibba032.whopeo (mgr.24625) 11578 : cluster [DBG] pgmap v6602: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:42 gibba032 bash[24004]: cluster 2021-03-10T17:32:42.152119+0000 mgr.gibba032.whopeo (mgr.24625) 11579 : cluster [DBG] pgmap v6603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:42 gibba011 bash[30791]: cluster 2021-03-10T17:32:42.152119+0000 mgr.gibba032.whopeo (mgr.24625) 11579 : cluster [DBG] pgmap v6603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:42 gibba011 bash[31774]: cluster 2021-03-10T17:32:42.152119+0000 mgr.gibba032.whopeo (mgr.24625) 11579 : cluster [DBG] pgmap v6603: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:44 gibba011 bash[30791]: cluster 2021-03-10T17:32:44.153723+0000 mgr.gibba032.whopeo (mgr.24625) 11580 : cluster [DBG] pgmap v6604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:44 gibba011 bash[31774]: cluster 2021-03-10T17:32:44.153723+0000 mgr.gibba032.whopeo (mgr.24625) 11580 : cluster [DBG] pgmap v6604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:44 gibba032 bash[24004]: cluster 2021-03-10T17:32:44.153723+0000 mgr.gibba032.whopeo (mgr.24625) 11580 : cluster [DBG] pgmap v6604: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:46 gibba011 bash[30791]: cluster 2021-03-10T17:32:46.155191+0000 mgr.gibba032.whopeo (mgr.24625) 11581 : cluster [DBG] pgmap v6605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:46 gibba011 bash[31774]: cluster 2021-03-10T17:32:46.155191+0000 mgr.gibba032.whopeo (mgr.24625) 11581 : cluster [DBG] pgmap v6605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:46 gibba032 bash[24004]: cluster 2021-03-10T17:32:46.155191+0000 mgr.gibba032.whopeo (mgr.24625) 11581 : cluster [DBG] pgmap v6605: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:47.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:47 gibba011 bash[30791]: audit 2021-03-10T17:32:47.481002+0000 mon.a (mon.0) 3811 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:32:47.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:47 gibba011 bash[31774]: audit 2021-03-10T17:32:47.481002+0000 mon.a (mon.0) 3811 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:32:48.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:47 gibba032 bash[24004]: audit 2021-03-10T17:32:47.481002+0000 mon.a (mon.0) 3811 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:32:48.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:48 gibba011 bash[30791]: cephadm 2021-03-10T17:32:47.477263+0000 mgr.gibba032.whopeo (mgr.24625) 11582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:32:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:48 gibba011 bash[30791]: cluster 2021-03-10T17:32:48.157062+0000 mgr.gibba032.whopeo (mgr.24625) 11583 : cluster [DBG] pgmap v6606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:48 gibba011 bash[31774]: cephadm 2021-03-10T17:32:47.477263+0000 mgr.gibba032.whopeo (mgr.24625) 11582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:32:48.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:48 gibba011 bash[31774]: cluster 2021-03-10T17:32:48.157062+0000 mgr.gibba032.whopeo (mgr.24625) 11583 : cluster [DBG] pgmap v6606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:49.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:48 gibba032 bash[24004]: cephadm 2021-03-10T17:32:47.477263+0000 mgr.gibba032.whopeo (mgr.24625) 11582 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:32:49.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:48 gibba032 bash[24004]: cluster 2021-03-10T17:32:48.157062+0000 mgr.gibba032.whopeo (mgr.24625) 11583 : cluster [DBG] pgmap v6606: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:50 gibba011 bash[30791]: cluster 2021-03-10T17:32:50.158587+0000 mgr.gibba032.whopeo (mgr.24625) 11584 : cluster [DBG] pgmap v6607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:50 gibba011 bash[31774]: cluster 2021-03-10T17:32:50.158587+0000 mgr.gibba032.whopeo (mgr.24625) 11584 : cluster [DBG] pgmap v6607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:50 gibba032 bash[24004]: cluster 2021-03-10T17:32:50.158587+0000 mgr.gibba032.whopeo (mgr.24625) 11584 : cluster [DBG] pgmap v6607: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:52.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.170039+0000 mgr.gibba032.whopeo (mgr.24625) 11585 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:32:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.170596+0000 mon.a (mon.0) 3812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.369440+0000 mon.a (mon.0) 3813 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.373155+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:32:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.375252+0000 mon.a (mon.0) 3815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.377229+0000 mon.a (mon.0) 3816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.378567+0000 mon.a (mon.0) 3817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.381466+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:52 gibba032 bash[24004]: audit 2021-03-10T17:32:51.383085+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.170039+0000 mgr.gibba032.whopeo (mgr.24625) 11585 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:32:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.170596+0000 mon.a (mon.0) 3812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.369440+0000 mon.a (mon.0) 3813 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.373155+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:32:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.375252+0000 mon.a (mon.0) 3815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.377229+0000 mon.a (mon.0) 3816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.378567+0000 mon.a (mon.0) 3817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.381466+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[30791]: audit 2021-03-10T17:32:51.383085+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.170039+0000 mgr.gibba032.whopeo (mgr.24625) 11585 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:32:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.170596+0000 mon.a (mon.0) 3812 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.369440+0000 mon.a (mon.0) 3813 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:32:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.373155+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:32:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.375252+0000 mon.a (mon.0) 3815 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.377229+0000 mon.a (mon.0) 3816 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.378567+0000 mon.a (mon.0) 3817 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.381466+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:52.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:52 gibba011 bash[31774]: audit 2021-03-10T17:32:51.383085+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:32:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.370249+0000 mgr.gibba032.whopeo (mgr.24625) 11586 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:32:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.372861+0000 mgr.gibba032.whopeo (mgr.24625) 11587 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:32:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.374755+0000 mgr.gibba032.whopeo (mgr.24625) 11588 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:32:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.374794+0000 mgr.gibba032.whopeo (mgr.24625) 11589 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:32:53.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.376359+0000 mgr.gibba032.whopeo (mgr.24625) 11590 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:32:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.376395+0000 mgr.gibba032.whopeo (mgr.24625) 11591 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:32:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.377787+0000 mgr.gibba032.whopeo (mgr.24625) 11592 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:32:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.379000+0000 mgr.gibba032.whopeo (mgr.24625) 11593 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:32:53.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.379077+0000 mgr.gibba032.whopeo (mgr.24625) 11594 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:32:53.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.380580+0000 mgr.gibba032.whopeo (mgr.24625) 11595 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:32:53.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.380891+0000 mgr.gibba032.whopeo (mgr.24625) 11596 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:32:53.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.382249+0000 mgr.gibba032.whopeo (mgr.24625) 11597 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:32:53.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.382287+0000 mgr.gibba032.whopeo (mgr.24625) 11598 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:32:53.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cephadm 2021-03-10T17:32:51.382389+0000 mgr.gibba032.whopeo (mgr.24625) 11599 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:53.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:53 gibba032 bash[24004]: cluster 2021-03-10T17:32:52.160143+0000 mgr.gibba032.whopeo (mgr.24625) 11600 : cluster [DBG] pgmap v6608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.370249+0000 mgr.gibba032.whopeo (mgr.24625) 11586 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:32:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.372861+0000 mgr.gibba032.whopeo (mgr.24625) 11587 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:32:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.374755+0000 mgr.gibba032.whopeo (mgr.24625) 11588 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:32:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.374794+0000 mgr.gibba032.whopeo (mgr.24625) 11589 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:32:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.376359+0000 mgr.gibba032.whopeo (mgr.24625) 11590 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:32:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.376395+0000 mgr.gibba032.whopeo (mgr.24625) 11591 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:32:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.377787+0000 mgr.gibba032.whopeo (mgr.24625) 11592 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:32:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.379000+0000 mgr.gibba032.whopeo (mgr.24625) 11593 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:32:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.379077+0000 mgr.gibba032.whopeo (mgr.24625) 11594 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:32:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.380580+0000 mgr.gibba032.whopeo (mgr.24625) 11595 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:32:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.380891+0000 mgr.gibba032.whopeo (mgr.24625) 11596 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:32:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.382249+0000 mgr.gibba032.whopeo (mgr.24625) 11597 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:32:53.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.382287+0000 mgr.gibba032.whopeo (mgr.24625) 11598 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:32:53.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cephadm 2021-03-10T17:32:51.382389+0000 mgr.gibba032.whopeo (mgr.24625) 11599 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:53.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[30791]: cluster 2021-03-10T17:32:52.160143+0000 mgr.gibba032.whopeo (mgr.24625) 11600 : cluster [DBG] pgmap v6608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.370249+0000 mgr.gibba032.whopeo (mgr.24625) 11586 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:32:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.372861+0000 mgr.gibba032.whopeo (mgr.24625) 11587 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:32:53.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.374755+0000 mgr.gibba032.whopeo (mgr.24625) 11588 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:32:53.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.374794+0000 mgr.gibba032.whopeo (mgr.24625) 11589 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:32:53.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.376359+0000 mgr.gibba032.whopeo (mgr.24625) 11590 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:32:53.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.376395+0000 mgr.gibba032.whopeo (mgr.24625) 11591 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:32:53.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.377787+0000 mgr.gibba032.whopeo (mgr.24625) 11592 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:32:53.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.379000+0000 mgr.gibba032.whopeo (mgr.24625) 11593 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:32:53.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.379077+0000 mgr.gibba032.whopeo (mgr.24625) 11594 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:32:53.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.380580+0000 mgr.gibba032.whopeo (mgr.24625) 11595 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:32:53.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.380891+0000 mgr.gibba032.whopeo (mgr.24625) 11596 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:32:53.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.382249+0000 mgr.gibba032.whopeo (mgr.24625) 11597 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:32:53.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.382287+0000 mgr.gibba032.whopeo (mgr.24625) 11598 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:32:53.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cephadm 2021-03-10T17:32:51.382389+0000 mgr.gibba032.whopeo (mgr.24625) 11599 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:32:53.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:53 gibba011 bash[31774]: cluster 2021-03-10T17:32:52.160143+0000 mgr.gibba032.whopeo (mgr.24625) 11600 : cluster [DBG] pgmap v6608: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:55.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:55 gibba032 bash[24004]: cluster 2021-03-10T17:32:54.161444+0000 mgr.gibba032.whopeo (mgr.24625) 11601 : cluster [DBG] pgmap v6609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:55 gibba011 bash[30791]: cluster 2021-03-10T17:32:54.161444+0000 mgr.gibba032.whopeo (mgr.24625) 11601 : cluster [DBG] pgmap v6609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:55 gibba011 bash[31774]: cluster 2021-03-10T17:32:54.161444+0000 mgr.gibba032.whopeo (mgr.24625) 11601 : cluster [DBG] pgmap v6609: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:57.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:57 gibba032 bash[24004]: cluster 2021-03-10T17:32:56.163071+0000 mgr.gibba032.whopeo (mgr.24625) 11602 : cluster [DBG] pgmap v6610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:57.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:57 gibba011 bash[30791]: cluster 2021-03-10T17:32:56.163071+0000 mgr.gibba032.whopeo (mgr.24625) 11602 : cluster [DBG] pgmap v6610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:57.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:57 gibba011 bash[31774]: cluster 2021-03-10T17:32:56.163071+0000 mgr.gibba032.whopeo (mgr.24625) 11602 : cluster [DBG] pgmap v6610: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.540787+0000 mon.a (mon.0) 3820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:32:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.541481+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:32:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.735073+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T17:32:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.735959+0000 mon.a (mon.0) 3823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.736746+0000 mon.a (mon.0) 3824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:58 gibba032 bash[24004]: audit 2021-03-10T17:32:57.737459+0000 mon.a (mon.0) 3825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:32:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.540787+0000 mon.a (mon.0) 3820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:32:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.541481+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:32:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.735073+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T17:32:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.735959+0000 mon.a (mon.0) 3823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.736746+0000 mon.a (mon.0) 3824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[30791]: audit 2021-03-10T17:32:57.737459+0000 mon.a (mon.0) 3825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:32:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.540787+0000 mon.a (mon.0) 3820 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:32:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.541481+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:32:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.735073+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2021-03-10T17:32:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.735959+0000 mon.a (mon.0) 3823 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.736746+0000 mon.a (mon.0) 3824 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:32:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:58 gibba011 bash[31774]: audit 2021-03-10T17:32:57.737459+0000 mon.a (mon.0) 3825 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:32:59.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:32:59 gibba032 bash[24004]: cluster 2021-03-10T17:32:58.164836+0000 mgr.gibba032.whopeo (mgr.24625) 11603 : cluster [DBG] pgmap v6611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:59.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:32:59 gibba011 bash[30791]: cluster 2021-03-10T17:32:58.164836+0000 mgr.gibba032.whopeo (mgr.24625) 11603 : cluster [DBG] pgmap v6611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:32:59.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:32:59 gibba011 bash[31774]: cluster 2021-03-10T17:32:58.164836+0000 mgr.gibba032.whopeo (mgr.24625) 11603 : cluster [DBG] pgmap v6611: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:01.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:01 gibba032 bash[24004]: cluster 2021-03-10T17:33:00.165616+0000 mgr.gibba032.whopeo (mgr.24625) 11604 : cluster [DBG] pgmap v6612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:01.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:01 gibba011 bash[30791]: cluster 2021-03-10T17:33:00.165616+0000 mgr.gibba032.whopeo (mgr.24625) 11604 : cluster [DBG] pgmap v6612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:01.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:01 gibba011 bash[31774]: cluster 2021-03-10T17:33:00.165616+0000 mgr.gibba032.whopeo (mgr.24625) 11604 : cluster [DBG] pgmap v6612: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:02 gibba032 bash[24004]: cluster 2021-03-10T17:33:02.166964+0000 mgr.gibba032.whopeo (mgr.24625) 11605 : cluster [DBG] pgmap v6613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:02 gibba011 bash[30791]: cluster 2021-03-10T17:33:02.166964+0000 mgr.gibba032.whopeo (mgr.24625) 11605 : cluster [DBG] pgmap v6613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:02.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:02 gibba011 bash[31774]: cluster 2021-03-10T17:33:02.166964+0000 mgr.gibba032.whopeo (mgr.24625) 11605 : cluster [DBG] pgmap v6613: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:04 gibba011 bash[30791]: cluster 2021-03-10T17:33:04.168472+0000 mgr.gibba032.whopeo (mgr.24625) 11606 : cluster [DBG] pgmap v6614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:04 gibba011 bash[31774]: cluster 2021-03-10T17:33:04.168472+0000 mgr.gibba032.whopeo (mgr.24625) 11606 : cluster [DBG] pgmap v6614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:04 gibba032 bash[24004]: cluster 2021-03-10T17:33:04.168472+0000 mgr.gibba032.whopeo (mgr.24625) 11606 : cluster [DBG] pgmap v6614: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:06.115 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:33:06.343 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:33:06.344 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 79s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:33:06.344 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 15s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:33:06.344 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:33:06.344 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:33:06.345 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:33:06.345 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:33:06.345 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:33:06.345 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:33:06.346 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 79s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:33:06.346 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 15s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:33:06.346 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:33:06.346 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:33:06.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:33:06.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 79s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:33:06.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:33:06.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:33:06.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:33:06.348 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 15s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:33:06.348 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 15s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:33:06.348 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 79s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:33:06.579 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:06 gibba011 bash[30791]: audit 2021-03-10T17:33:06.103279+0000 mgr.gibba032.whopeo (mgr.24625) 11607 : audit [DBG] from='client.51831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:06 gibba011 bash[30791]: cluster 2021-03-10T17:33:06.169989+0000 mgr.gibba032.whopeo (mgr.24625) 11608 : cluster [DBG] pgmap v6615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:06.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:06 gibba011 bash[31774]: audit 2021-03-10T17:33:06.103279+0000 mgr.gibba032.whopeo (mgr.24625) 11607 : audit [DBG] from='client.51831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:06 gibba011 bash[31774]: cluster 2021-03-10T17:33:06.169989+0000 mgr.gibba032.whopeo (mgr.24625) 11608 : cluster [DBG] pgmap v6615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:06.595 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:33:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:33:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:33:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:33:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:33:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:33:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:33:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:33:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:33:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:33:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:33:06.598 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:33:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:06 gibba032 bash[24004]: audit 2021-03-10T17:33:06.103279+0000 mgr.gibba032.whopeo (mgr.24625) 11607 : audit [DBG] from='client.51831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:06 gibba032 bash[24004]: cluster 2021-03-10T17:33:06.169989+0000 mgr.gibba032.whopeo (mgr.24625) 11608 : cluster [DBG] pgmap v6615: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:07 gibba032 bash[24004]: audit 2021-03-10T17:33:06.329825+0000 mgr.gibba032.whopeo (mgr.24625) 11609 : audit [DBG] from='client.61819 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:07.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:07 gibba032 bash[24004]: cephadm 2021-03-10T17:33:06.398408+0000 mgr.gibba032.whopeo (mgr.24625) 11610 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:07.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:07 gibba032 bash[24004]: audit 2021-03-10T17:33:06.594062+0000 mon.c (mon.1) 225 : audit [DBG] from='client.? 172.21.2.111:0/4130663226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[30791]: audit 2021-03-10T17:33:06.329825+0000 mgr.gibba032.whopeo (mgr.24625) 11609 : audit [DBG] from='client.61819 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[30791]: cephadm 2021-03-10T17:33:06.398408+0000 mgr.gibba032.whopeo (mgr.24625) 11610 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:07.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[30791]: audit 2021-03-10T17:33:06.594062+0000 mon.c (mon.1) 225 : audit [DBG] from='client.? 172.21.2.111:0/4130663226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[31774]: audit 2021-03-10T17:33:06.329825+0000 mgr.gibba032.whopeo (mgr.24625) 11609 : audit [DBG] from='client.61819 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[31774]: cephadm 2021-03-10T17:33:06.398408+0000 mgr.gibba032.whopeo (mgr.24625) 11610 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:07 gibba011 bash[31774]: audit 2021-03-10T17:33:06.594062+0000 mon.c (mon.1) 225 : audit [DBG] from='client.? 172.21.2.111:0/4130663226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:08.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:08 gibba032 bash[24004]: cluster 2021-03-10T17:33:08.171505+0000 mgr.gibba032.whopeo (mgr.24625) 11611 : cluster [DBG] pgmap v6616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:08 gibba011 bash[30791]: cluster 2021-03-10T17:33:08.171505+0000 mgr.gibba032.whopeo (mgr.24625) 11611 : cluster [DBG] pgmap v6616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:08 gibba011 bash[31774]: cluster 2021-03-10T17:33:08.171505+0000 mgr.gibba032.whopeo (mgr.24625) 11611 : cluster [DBG] pgmap v6616: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:10 gibba011 bash[30791]: cluster 2021-03-10T17:33:10.172977+0000 mgr.gibba032.whopeo (mgr.24625) 11612 : cluster [DBG] pgmap v6617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:10 gibba011 bash[31774]: cluster 2021-03-10T17:33:10.172977+0000 mgr.gibba032.whopeo (mgr.24625) 11612 : cluster [DBG] pgmap v6617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:10 gibba032 bash[24004]: cluster 2021-03-10T17:33:10.172977+0000 mgr.gibba032.whopeo (mgr.24625) 11612 : cluster [DBG] pgmap v6617: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:12.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:12 gibba032 bash[24004]: cluster 2021-03-10T17:33:12.174257+0000 mgr.gibba032.whopeo (mgr.24625) 11613 : cluster [DBG] pgmap v6618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:12 gibba011 bash[30791]: cluster 2021-03-10T17:33:12.174257+0000 mgr.gibba032.whopeo (mgr.24625) 11613 : cluster [DBG] pgmap v6618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:12 gibba011 bash[31774]: cluster 2021-03-10T17:33:12.174257+0000 mgr.gibba032.whopeo (mgr.24625) 11613 : cluster [DBG] pgmap v6618: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:14 gibba011 bash[30791]: cluster 2021-03-10T17:33:14.175769+0000 mgr.gibba032.whopeo (mgr.24625) 11614 : cluster [DBG] pgmap v6619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:14 gibba011 bash[31774]: cluster 2021-03-10T17:33:14.175769+0000 mgr.gibba032.whopeo (mgr.24625) 11614 : cluster [DBG] pgmap v6619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:14 gibba032 bash[24004]: cluster 2021-03-10T17:33:14.175769+0000 mgr.gibba032.whopeo (mgr.24625) 11614 : cluster [DBG] pgmap v6619: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:16 gibba011 bash[30791]: cluster 2021-03-10T17:33:16.177223+0000 mgr.gibba032.whopeo (mgr.24625) 11615 : cluster [DBG] pgmap v6620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:16.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:16 gibba011 bash[31774]: cluster 2021-03-10T17:33:16.177223+0000 mgr.gibba032.whopeo (mgr.24625) 11615 : cluster [DBG] pgmap v6620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:16.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:16 gibba032 bash[24004]: cluster 2021-03-10T17:33:16.177223+0000 mgr.gibba032.whopeo (mgr.24625) 11615 : cluster [DBG] pgmap v6620: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:18 gibba011 bash[30791]: cluster 2021-03-10T17:33:18.179021+0000 mgr.gibba032.whopeo (mgr.24625) 11616 : cluster [DBG] pgmap v6621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:18 gibba011 bash[31774]: cluster 2021-03-10T17:33:18.179021+0000 mgr.gibba032.whopeo (mgr.24625) 11616 : cluster [DBG] pgmap v6621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:18.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:18 gibba032 bash[24004]: cluster 2021-03-10T17:33:18.179021+0000 mgr.gibba032.whopeo (mgr.24625) 11616 : cluster [DBG] pgmap v6621: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:20 gibba011 bash[30791]: cluster 2021-03-10T17:33:20.180703+0000 mgr.gibba032.whopeo (mgr.24625) 11617 : cluster [DBG] pgmap v6622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:20.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:20 gibba011 bash[31774]: cluster 2021-03-10T17:33:20.180703+0000 mgr.gibba032.whopeo (mgr.24625) 11617 : cluster [DBG] pgmap v6622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:20 gibba032 bash[24004]: cluster 2021-03-10T17:33:20.180703+0000 mgr.gibba032.whopeo (mgr.24625) 11617 : cluster [DBG] pgmap v6622: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:22 gibba032 bash[24004]: cephadm 2021-03-10T17:33:21.413992+0000 mgr.gibba032.whopeo (mgr.24625) 11618 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:22 gibba032 bash[24004]: cluster 2021-03-10T17:33:22.181989+0000 mgr.gibba032.whopeo (mgr.24625) 11619 : cluster [DBG] pgmap v6623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:22 gibba011 bash[30791]: cephadm 2021-03-10T17:33:21.413992+0000 mgr.gibba032.whopeo (mgr.24625) 11618 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:22 gibba011 bash[30791]: cluster 2021-03-10T17:33:22.181989+0000 mgr.gibba032.whopeo (mgr.24625) 11619 : cluster [DBG] pgmap v6623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:22 gibba011 bash[31774]: cephadm 2021-03-10T17:33:21.413992+0000 mgr.gibba032.whopeo (mgr.24625) 11618 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:22 gibba011 bash[31774]: cluster 2021-03-10T17:33:22.181989+0000 mgr.gibba032.whopeo (mgr.24625) 11619 : cluster [DBG] pgmap v6623: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:24 gibba011 bash[30791]: cluster 2021-03-10T17:33:24.183597+0000 mgr.gibba032.whopeo (mgr.24625) 11620 : cluster [DBG] pgmap v6624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:24 gibba011 bash[31774]: cluster 2021-03-10T17:33:24.183597+0000 mgr.gibba032.whopeo (mgr.24625) 11620 : cluster [DBG] pgmap v6624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:24 gibba032 bash[24004]: cluster 2021-03-10T17:33:24.183597+0000 mgr.gibba032.whopeo (mgr.24625) 11620 : cluster [DBG] pgmap v6624: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:26 gibba011 bash[30791]: cluster 2021-03-10T17:33:26.185134+0000 mgr.gibba032.whopeo (mgr.24625) 11621 : cluster [DBG] pgmap v6625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:26 gibba011 bash[31774]: cluster 2021-03-10T17:33:26.185134+0000 mgr.gibba032.whopeo (mgr.24625) 11621 : cluster [DBG] pgmap v6625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:26 gibba032 bash[24004]: cluster 2021-03-10T17:33:26.185134+0000 mgr.gibba032.whopeo (mgr.24625) 11621 : cluster [DBG] pgmap v6625: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:28 gibba011 bash[30791]: cluster 2021-03-10T17:33:28.186759+0000 mgr.gibba032.whopeo (mgr.24625) 11622 : cluster [DBG] pgmap v6626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:28 gibba011 bash[31774]: cluster 2021-03-10T17:33:28.186759+0000 mgr.gibba032.whopeo (mgr.24625) 11622 : cluster [DBG] pgmap v6626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:28.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:28 gibba032 bash[24004]: cluster 2021-03-10T17:33:28.186759+0000 mgr.gibba032.whopeo (mgr.24625) 11622 : cluster [DBG] pgmap v6626: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:30 gibba011 bash[30791]: cluster 2021-03-10T17:33:30.188181+0000 mgr.gibba032.whopeo (mgr.24625) 11623 : cluster [DBG] pgmap v6627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:30 gibba011 bash[31774]: cluster 2021-03-10T17:33:30.188181+0000 mgr.gibba032.whopeo (mgr.24625) 11623 : cluster [DBG] pgmap v6627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:30 gibba032 bash[24004]: cluster 2021-03-10T17:33:30.188181+0000 mgr.gibba032.whopeo (mgr.24625) 11623 : cluster [DBG] pgmap v6627: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:32.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:32 gibba032 bash[24004]: cluster 2021-03-10T17:33:32.189433+0000 mgr.gibba032.whopeo (mgr.24625) 11624 : cluster [DBG] pgmap v6628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:32 gibba011 bash[30791]: cluster 2021-03-10T17:33:32.189433+0000 mgr.gibba032.whopeo (mgr.24625) 11624 : cluster [DBG] pgmap v6628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:32 gibba011 bash[31774]: cluster 2021-03-10T17:33:32.189433+0000 mgr.gibba032.whopeo (mgr.24625) 11624 : cluster [DBG] pgmap v6628: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:34 gibba011 bash[30791]: cluster 2021-03-10T17:33:34.190918+0000 mgr.gibba032.whopeo (mgr.24625) 11625 : cluster [DBG] pgmap v6629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:34 gibba011 bash[31774]: cluster 2021-03-10T17:33:34.190918+0000 mgr.gibba032.whopeo (mgr.24625) 11625 : cluster [DBG] pgmap v6629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:34 gibba032 bash[24004]: cluster 2021-03-10T17:33:34.190918+0000 mgr.gibba032.whopeo (mgr.24625) 11625 : cluster [DBG] pgmap v6629: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:36 gibba011 bash[30791]: cluster 2021-03-10T17:33:36.192436+0000 mgr.gibba032.whopeo (mgr.24625) 11626 : cluster [DBG] pgmap v6630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:36.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:36 gibba011 bash[31774]: cluster 2021-03-10T17:33:36.192436+0000 mgr.gibba032.whopeo (mgr.24625) 11626 : cluster [DBG] pgmap v6630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:36.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:36 gibba032 bash[24004]: cluster 2021-03-10T17:33:36.192436+0000 mgr.gibba032.whopeo (mgr.24625) 11626 : cluster [DBG] pgmap v6630: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:36.841 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:33:37.065 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:33:37.066 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 109s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:33:37.066 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 45s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:33:37.066 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:33:37.066 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:33:37.067 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:33:37.067 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:33:37.067 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:33:37.068 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:33:37.068 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 109s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:33:37.068 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 45s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:33:37.069 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:33:37.069 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:33:37.069 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:33:37.070 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 109s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:33:37.070 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:33:37.071 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:33:37.071 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:33:37.072 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 45s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:33:37.072 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 45s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:33:37.072 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 109s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:33:37.314 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:33:37.314 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:33:37.315 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:33:37.315 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:37.315 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:33:37.315 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:33:37.316 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:37.316 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:33:37.316 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:33:37.316 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:33:37.316 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:33:37.317 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:33:37.317 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:33:37.317 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:33:37.317 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:33:37.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:37 gibba032 bash[24004]: cephadm 2021-03-10T17:33:36.429908+0000 mgr.gibba032.whopeo (mgr.24625) 11627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:37.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:37 gibba032 bash[24004]: audit 2021-03-10T17:33:36.829544+0000 mgr.gibba032.whopeo (mgr.24625) 11628 : audit [DBG] from='client.61831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:37 gibba032 bash[24004]: audit 2021-03-10T17:33:37.052602+0000 mgr.gibba032.whopeo (mgr.24625) 11629 : audit [DBG] from='client.51855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:37 gibba032 bash[24004]: audit 2021-03-10T17:33:37.313232+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? 172.21.2.111:0/1019654109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[30791]: cephadm 2021-03-10T17:33:36.429908+0000 mgr.gibba032.whopeo (mgr.24625) 11627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[30791]: audit 2021-03-10T17:33:36.829544+0000 mgr.gibba032.whopeo (mgr.24625) 11628 : audit [DBG] from='client.61831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[30791]: audit 2021-03-10T17:33:37.052602+0000 mgr.gibba032.whopeo (mgr.24625) 11629 : audit [DBG] from='client.51855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[30791]: audit 2021-03-10T17:33:37.313232+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? 172.21.2.111:0/1019654109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[31774]: cephadm 2021-03-10T17:33:36.429908+0000 mgr.gibba032.whopeo (mgr.24625) 11627 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[31774]: audit 2021-03-10T17:33:36.829544+0000 mgr.gibba032.whopeo (mgr.24625) 11628 : audit [DBG] from='client.61831 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[31774]: audit 2021-03-10T17:33:37.052602+0000 mgr.gibba032.whopeo (mgr.24625) 11629 : audit [DBG] from='client.51855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:33:37.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:37 gibba011 bash[31774]: audit 2021-03-10T17:33:37.313232+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? 172.21.2.111:0/1019654109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:38 gibba032 bash[24004]: cluster 2021-03-10T17:33:38.193974+0000 mgr.gibba032.whopeo (mgr.24625) 11630 : cluster [DBG] pgmap v6631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:38 gibba011 bash[30791]: cluster 2021-03-10T17:33:38.193974+0000 mgr.gibba032.whopeo (mgr.24625) 11630 : cluster [DBG] pgmap v6631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:38 gibba011 bash[31774]: cluster 2021-03-10T17:33:38.193974+0000 mgr.gibba032.whopeo (mgr.24625) 11630 : cluster [DBG] pgmap v6631: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:40.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:40 gibba011 bash[30791]: cluster 2021-03-10T17:33:40.195609+0000 mgr.gibba032.whopeo (mgr.24625) 11631 : cluster [DBG] pgmap v6632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:40 gibba011 bash[31774]: cluster 2021-03-10T17:33:40.195609+0000 mgr.gibba032.whopeo (mgr.24625) 11631 : cluster [DBG] pgmap v6632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:40.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:40 gibba032 bash[24004]: cluster 2021-03-10T17:33:40.195609+0000 mgr.gibba032.whopeo (mgr.24625) 11631 : cluster [DBG] pgmap v6632: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:42 gibba032 bash[24004]: cluster 2021-03-10T17:33:42.197199+0000 mgr.gibba032.whopeo (mgr.24625) 11632 : cluster [DBG] pgmap v6633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:42 gibba011 bash[30791]: cluster 2021-03-10T17:33:42.197199+0000 mgr.gibba032.whopeo (mgr.24625) 11632 : cluster [DBG] pgmap v6633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:42 gibba011 bash[31774]: cluster 2021-03-10T17:33:42.197199+0000 mgr.gibba032.whopeo (mgr.24625) 11632 : cluster [DBG] pgmap v6633: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:44 gibba011 bash[30791]: cluster 2021-03-10T17:33:44.198687+0000 mgr.gibba032.whopeo (mgr.24625) 11633 : cluster [DBG] pgmap v6634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:44 gibba011 bash[31774]: cluster 2021-03-10T17:33:44.198687+0000 mgr.gibba032.whopeo (mgr.24625) 11633 : cluster [DBG] pgmap v6634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:44 gibba032 bash[24004]: cluster 2021-03-10T17:33:44.198687+0000 mgr.gibba032.whopeo (mgr.24625) 11633 : cluster [DBG] pgmap v6634: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:46 gibba011 bash[30791]: cluster 2021-03-10T17:33:46.200228+0000 mgr.gibba032.whopeo (mgr.24625) 11634 : cluster [DBG] pgmap v6635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:46 gibba011 bash[31774]: cluster 2021-03-10T17:33:46.200228+0000 mgr.gibba032.whopeo (mgr.24625) 11634 : cluster [DBG] pgmap v6635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:46.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:46 gibba032 bash[24004]: cluster 2021-03-10T17:33:46.200228+0000 mgr.gibba032.whopeo (mgr.24625) 11634 : cluster [DBG] pgmap v6635: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:48 gibba011 bash[30791]: cluster 2021-03-10T17:33:48.201962+0000 mgr.gibba032.whopeo (mgr.24625) 11635 : cluster [DBG] pgmap v6636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:48 gibba011 bash[31774]: cluster 2021-03-10T17:33:48.201962+0000 mgr.gibba032.whopeo (mgr.24625) 11635 : cluster [DBG] pgmap v6636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:48 gibba032 bash[24004]: cluster 2021-03-10T17:33:48.201962+0000 mgr.gibba032.whopeo (mgr.24625) 11635 : cluster [DBG] pgmap v6636: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:50 gibba011 bash[30791]: cluster 2021-03-10T17:33:50.203556+0000 mgr.gibba032.whopeo (mgr.24625) 11636 : cluster [DBG] pgmap v6637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:50 gibba011 bash[31774]: cluster 2021-03-10T17:33:50.203556+0000 mgr.gibba032.whopeo (mgr.24625) 11636 : cluster [DBG] pgmap v6637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:50 gibba032 bash[24004]: cluster 2021-03-10T17:33:50.203556+0000 mgr.gibba032.whopeo (mgr.24625) 11636 : cluster [DBG] pgmap v6637: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:53.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.445747+0000 mgr.gibba032.whopeo (mgr.24625) 11637 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:33:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.447360+0000 mgr.gibba032.whopeo (mgr.24625) 11638 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:33:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.689663+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:33:53.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.691410+0000 mgr.gibba032.whopeo (mgr.24625) 11639 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:33:53.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.694382+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:33:53.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.694610+0000 mgr.gibba032.whopeo (mgr.24625) 11640 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:33:53.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.697272+0000 mon.a (mon.0) 3829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.698578+0000 mgr.gibba032.whopeo (mgr.24625) 11641 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:33:53.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.699091+0000 mgr.gibba032.whopeo (mgr.24625) 11642 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:33:53.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.702542+0000 mon.a (mon.0) 3830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.704462+0000 mgr.gibba032.whopeo (mgr.24625) 11643 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:33:53.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.704701+0000 mgr.gibba032.whopeo (mgr.24625) 11644 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:33:53.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.708009+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.709487+0000 mgr.gibba032.whopeo (mgr.24625) 11645 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:33:53.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.713395+0000 mgr.gibba032.whopeo (mgr.24625) 11646 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:33:53.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.713656+0000 mgr.gibba032.whopeo (mgr.24625) 11647 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:33:53.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.717609+0000 mon.a (mon.0) 3832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.718854+0000 mgr.gibba032.whopeo (mgr.24625) 11648 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:33:53.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.721087+0000 mgr.gibba032.whopeo (mgr.24625) 11649 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:33:53.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: audit 2021-03-10T17:33:51.724077+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.725340+0000 mgr.gibba032.whopeo (mgr.24625) 11650 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:33:53.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.725838+0000 mgr.gibba032.whopeo (mgr.24625) 11651 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:33:53.016 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cephadm 2021-03-10T17:33:51.726515+0000 mgr.gibba032.whopeo (mgr.24625) 11652 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:53.017 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:52 gibba032 bash[24004]: cluster 2021-03-10T17:33:52.205030+0000 mgr.gibba032.whopeo (mgr.24625) 11653 : cluster [DBG] pgmap v6638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:53.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.445747+0000 mgr.gibba032.whopeo (mgr.24625) 11637 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:33:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.447360+0000 mgr.gibba032.whopeo (mgr.24625) 11638 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:33:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.689663+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:33:53.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.691410+0000 mgr.gibba032.whopeo (mgr.24625) 11639 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:33:53.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.694382+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:33:53.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.694610+0000 mgr.gibba032.whopeo (mgr.24625) 11640 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:33:53.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.697272+0000 mon.a (mon.0) 3829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.698578+0000 mgr.gibba032.whopeo (mgr.24625) 11641 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:33:53.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.699091+0000 mgr.gibba032.whopeo (mgr.24625) 11642 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:33:53.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.702542+0000 mon.a (mon.0) 3830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.704462+0000 mgr.gibba032.whopeo (mgr.24625) 11643 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:33:53.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.704701+0000 mgr.gibba032.whopeo (mgr.24625) 11644 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:33:53.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.708009+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.709487+0000 mgr.gibba032.whopeo (mgr.24625) 11645 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:33:53.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.713395+0000 mgr.gibba032.whopeo (mgr.24625) 11646 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:33:53.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.713656+0000 mgr.gibba032.whopeo (mgr.24625) 11647 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:33:53.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.717609+0000 mon.a (mon.0) 3832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.085 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.718854+0000 mgr.gibba032.whopeo (mgr.24625) 11648 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:33:53.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.721087+0000 mgr.gibba032.whopeo (mgr.24625) 11649 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:33:53.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: audit 2021-03-10T17:33:51.724077+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.086 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.725340+0000 mgr.gibba032.whopeo (mgr.24625) 11650 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:33:53.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.445747+0000 mgr.gibba032.whopeo (mgr.24625) 11637 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:33:53.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.447360+0000 mgr.gibba032.whopeo (mgr.24625) 11638 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:33:53.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.689663+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:33:53.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.691410+0000 mgr.gibba032.whopeo (mgr.24625) 11639 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:33:53.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.694382+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:33:53.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.694610+0000 mgr.gibba032.whopeo (mgr.24625) 11640 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:33:53.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.697272+0000 mon.a (mon.0) 3829 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.698578+0000 mgr.gibba032.whopeo (mgr.24625) 11641 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:33:53.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.699091+0000 mgr.gibba032.whopeo (mgr.24625) 11642 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:33:53.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.702542+0000 mon.a (mon.0) 3830 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.704462+0000 mgr.gibba032.whopeo (mgr.24625) 11643 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:33:53.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.704701+0000 mgr.gibba032.whopeo (mgr.24625) 11644 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:33:53.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.708009+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.709487+0000 mgr.gibba032.whopeo (mgr.24625) 11645 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:33:53.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.713395+0000 mgr.gibba032.whopeo (mgr.24625) 11646 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:33:53.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.713656+0000 mgr.gibba032.whopeo (mgr.24625) 11647 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:33:53.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.717609+0000 mon.a (mon.0) 3832 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.718854+0000 mgr.gibba032.whopeo (mgr.24625) 11648 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:33:53.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.721087+0000 mgr.gibba032.whopeo (mgr.24625) 11649 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:33:53.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: audit 2021-03-10T17:33:51.724077+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:33:53.092 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.725340+0000 mgr.gibba032.whopeo (mgr.24625) 11650 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:33:53.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.725838+0000 mgr.gibba032.whopeo (mgr.24625) 11651 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:33:53.092 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cephadm 2021-03-10T17:33:51.726515+0000 mgr.gibba032.whopeo (mgr.24625) 11652 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:53.093 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[30791]: cluster 2021-03-10T17:33:52.205030+0000 mgr.gibba032.whopeo (mgr.24625) 11653 : cluster [DBG] pgmap v6638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:53.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.725838+0000 mgr.gibba032.whopeo (mgr.24625) 11651 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:33:53.093 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cephadm 2021-03-10T17:33:51.726515+0000 mgr.gibba032.whopeo (mgr.24625) 11652 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:33:53.094 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:52 gibba011 bash[31774]: cluster 2021-03-10T17:33:52.205030+0000 mgr.gibba032.whopeo (mgr.24625) 11653 : cluster [DBG] pgmap v6638: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:54 gibba011 bash[30791]: cluster 2021-03-10T17:33:54.206518+0000 mgr.gibba032.whopeo (mgr.24625) 11654 : cluster [DBG] pgmap v6639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:54 gibba011 bash[31774]: cluster 2021-03-10T17:33:54.206518+0000 mgr.gibba032.whopeo (mgr.24625) 11654 : cluster [DBG] pgmap v6639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:54 gibba032 bash[24004]: cluster 2021-03-10T17:33:54.206518+0000 mgr.gibba032.whopeo (mgr.24625) 11654 : cluster [DBG] pgmap v6639: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:56 gibba011 bash[30791]: cluster 2021-03-10T17:33:56.207967+0000 mgr.gibba032.whopeo (mgr.24625) 11655 : cluster [DBG] pgmap v6640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:56 gibba011 bash[31774]: cluster 2021-03-10T17:33:56.207967+0000 mgr.gibba032.whopeo (mgr.24625) 11655 : cluster [DBG] pgmap v6640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:56 gibba032 bash[24004]: cluster 2021-03-10T17:33:56.207967+0000 mgr.gibba032.whopeo (mgr.24625) 11655 : cluster [DBG] pgmap v6640: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:57 gibba032 bash[24004]: audit 2021-03-10T17:33:57.545273+0000 mon.a (mon.0) 3834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:33:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:57 gibba032 bash[24004]: audit 2021-03-10T17:33:57.545935+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:33:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:57 gibba011 bash[30791]: audit 2021-03-10T17:33:57.545273+0000 mon.a (mon.0) 3834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:33:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:57 gibba011 bash[30791]: audit 2021-03-10T17:33:57.545935+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:33:58.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:57 gibba011 bash[31774]: audit 2021-03-10T17:33:57.545273+0000 mon.a (mon.0) 3834 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:33:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:57 gibba011 bash[31774]: audit 2021-03-10T17:33:57.545935+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:33:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:58 gibba032 bash[24004]: audit 2021-03-10T17:33:57.745373+0000 mon.a (mon.0) 3836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:33:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:58 gibba032 bash[24004]: audit 2021-03-10T17:33:57.746149+0000 mon.a (mon.0) 3837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:58 gibba032 bash[24004]: audit 2021-03-10T17:33:57.746825+0000 mon.a (mon.0) 3838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:58 gibba032 bash[24004]: audit 2021-03-10T17:33:57.747524+0000 mon.a (mon.0) 3839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:33:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:33:58 gibba032 bash[24004]: cluster 2021-03-10T17:33:58.209517+0000 mgr.gibba032.whopeo (mgr.24625) 11656 : cluster [DBG] pgmap v6641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[30791]: audit 2021-03-10T17:33:57.745373+0000 mon.a (mon.0) 3836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:33:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[30791]: audit 2021-03-10T17:33:57.746149+0000 mon.a (mon.0) 3837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[30791]: audit 2021-03-10T17:33:57.746825+0000 mon.a (mon.0) 3838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[30791]: audit 2021-03-10T17:33:57.747524+0000 mon.a (mon.0) 3839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:33:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[30791]: cluster 2021-03-10T17:33:58.209517+0000 mgr.gibba032.whopeo (mgr.24625) 11656 : cluster [DBG] pgmap v6641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:33:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[31774]: audit 2021-03-10T17:33:57.745373+0000 mon.a (mon.0) 3836 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2021-03-10T17:33:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[31774]: audit 2021-03-10T17:33:57.746149+0000 mon.a (mon.0) 3837 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[31774]: audit 2021-03-10T17:33:57.746825+0000 mon.a (mon.0) 3838 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:33:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[31774]: audit 2021-03-10T17:33:57.747524+0000 mon.a (mon.0) 3839 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:33:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:33:58 gibba011 bash[31774]: cluster 2021-03-10T17:33:58.209517+0000 mgr.gibba032.whopeo (mgr.24625) 11656 : cluster [DBG] pgmap v6641: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:00 gibba011 bash[30791]: cluster 2021-03-10T17:34:00.210956+0000 mgr.gibba032.whopeo (mgr.24625) 11657 : cluster [DBG] pgmap v6642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:00 gibba011 bash[31774]: cluster 2021-03-10T17:34:00.210956+0000 mgr.gibba032.whopeo (mgr.24625) 11657 : cluster [DBG] pgmap v6642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:00.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:00 gibba032 bash[24004]: cluster 2021-03-10T17:34:00.210956+0000 mgr.gibba032.whopeo (mgr.24625) 11657 : cluster [DBG] pgmap v6642: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:02.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:02 gibba032 bash[24004]: cluster 2021-03-10T17:34:02.212449+0000 mgr.gibba032.whopeo (mgr.24625) 11658 : cluster [DBG] pgmap v6643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:02 gibba011 bash[30791]: cluster 2021-03-10T17:34:02.212449+0000 mgr.gibba032.whopeo (mgr.24625) 11658 : cluster [DBG] pgmap v6643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:02 gibba011 bash[31774]: cluster 2021-03-10T17:34:02.212449+0000 mgr.gibba032.whopeo (mgr.24625) 11658 : cluster [DBG] pgmap v6643: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:04 gibba011 bash[30791]: cluster 2021-03-10T17:34:04.214020+0000 mgr.gibba032.whopeo (mgr.24625) 11659 : cluster [DBG] pgmap v6644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:04 gibba011 bash[31774]: cluster 2021-03-10T17:34:04.214020+0000 mgr.gibba032.whopeo (mgr.24625) 11659 : cluster [DBG] pgmap v6644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:04 gibba032 bash[24004]: cluster 2021-03-10T17:34:04.214020+0000 mgr.gibba032.whopeo (mgr.24625) 11659 : cluster [DBG] pgmap v6644: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:06 gibba011 bash[30791]: cluster 2021-03-10T17:34:06.215942+0000 mgr.gibba032.whopeo (mgr.24625) 11660 : cluster [DBG] pgmap v6645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:06 gibba011 bash[31774]: cluster 2021-03-10T17:34:06.215942+0000 mgr.gibba032.whopeo (mgr.24625) 11660 : cluster [DBG] pgmap v6645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:06.746 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:06 gibba032 bash[24004]: cluster 2021-03-10T17:34:06.215942+0000 mgr.gibba032.whopeo (mgr.24625) 11660 : cluster [DBG] pgmap v6645: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:07.556 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:34:07.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:07 gibba032 bash[24004]: cephadm 2021-03-10T17:34:06.742512+0000 mgr.gibba032.whopeo (mgr.24625) 11661 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:07.782 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:34:07.782 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:34:07.783 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 76s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:34:07.783 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:34:07.783 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:34:07.783 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:34:07.784 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:34:07.784 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:34:07.784 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:34:07.784 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:34:07.784 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 76s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:34:07.785 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:34:07.785 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:34:07.785 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:34:07.785 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:34:07.785 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:34:07.786 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:34:07.786 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:34:07.786 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 76s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:34:07.786 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 76s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:34:07.787 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:34:07.829 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:07 gibba011 bash[30791]: cephadm 2021-03-10T17:34:06.742512+0000 mgr.gibba032.whopeo (mgr.24625) 11661 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:07.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:07 gibba011 bash[31774]: cephadm 2021-03-10T17:34:06.742512+0000 mgr.gibba032.whopeo (mgr.24625) 11661 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:08.033 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:34:08.034 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:34:08.034 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:34:08.034 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:08.035 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:34:08.035 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:34:08.035 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:08.035 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:34:08.036 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:34:08.036 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:08.036 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:34:08.036 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:34:08.036 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:34:08.037 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:34:08.037 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:34:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:08 gibba032 bash[24004]: audit 2021-03-10T17:34:07.545196+0000 mgr.gibba032.whopeo (mgr.24625) 11662 : audit [DBG] from='client.61849 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:08 gibba032 bash[24004]: audit 2021-03-10T17:34:07.767038+0000 mgr.gibba032.whopeo (mgr.24625) 11663 : audit [DBG] from='client.61855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:08 gibba032 bash[24004]: audit 2021-03-10T17:34:08.032359+0000 mon.c (mon.1) 226 : audit [DBG] from='client.? 172.21.2.111:0/2704044304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:08.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:08 gibba032 bash[24004]: cluster 2021-03-10T17:34:08.217533+0000 mgr.gibba032.whopeo (mgr.24625) 11664 : cluster [DBG] pgmap v6646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:08.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[30791]: audit 2021-03-10T17:34:07.545196+0000 mgr.gibba032.whopeo (mgr.24625) 11662 : audit [DBG] from='client.61849 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[30791]: audit 2021-03-10T17:34:07.767038+0000 mgr.gibba032.whopeo (mgr.24625) 11663 : audit [DBG] from='client.61855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[30791]: audit 2021-03-10T17:34:08.032359+0000 mon.c (mon.1) 226 : audit [DBG] from='client.? 172.21.2.111:0/2704044304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[30791]: cluster 2021-03-10T17:34:08.217533+0000 mgr.gibba032.whopeo (mgr.24625) 11664 : cluster [DBG] pgmap v6646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[31774]: audit 2021-03-10T17:34:07.545196+0000 mgr.gibba032.whopeo (mgr.24625) 11662 : audit [DBG] from='client.61849 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[31774]: audit 2021-03-10T17:34:07.767038+0000 mgr.gibba032.whopeo (mgr.24625) 11663 : audit [DBG] from='client.61855 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[31774]: audit 2021-03-10T17:34:08.032359+0000 mon.c (mon.1) 226 : audit [DBG] from='client.? 172.21.2.111:0/2704044304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:08.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:08 gibba011 bash[31774]: cluster 2021-03-10T17:34:08.217533+0000 mgr.gibba032.whopeo (mgr.24625) 11664 : cluster [DBG] pgmap v6646: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:10.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:10 gibba011 bash[30791]: cluster 2021-03-10T17:34:10.219022+0000 mgr.gibba032.whopeo (mgr.24625) 11665 : cluster [DBG] pgmap v6647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:10 gibba011 bash[31774]: cluster 2021-03-10T17:34:10.219022+0000 mgr.gibba032.whopeo (mgr.24625) 11665 : cluster [DBG] pgmap v6647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:10.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:10 gibba032 bash[24004]: cluster 2021-03-10T17:34:10.219022+0000 mgr.gibba032.whopeo (mgr.24625) 11665 : cluster [DBG] pgmap v6647: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:12 gibba032 bash[24004]: cluster 2021-03-10T17:34:12.220589+0000 mgr.gibba032.whopeo (mgr.24625) 11666 : cluster [DBG] pgmap v6648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:12.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:12 gibba011 bash[30791]: cluster 2021-03-10T17:34:12.220589+0000 mgr.gibba032.whopeo (mgr.24625) 11666 : cluster [DBG] pgmap v6648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:12 gibba011 bash[31774]: cluster 2021-03-10T17:34:12.220589+0000 mgr.gibba032.whopeo (mgr.24625) 11666 : cluster [DBG] pgmap v6648: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:14 gibba011 bash[30791]: cluster 2021-03-10T17:34:14.221951+0000 mgr.gibba032.whopeo (mgr.24625) 11667 : cluster [DBG] pgmap v6649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:14 gibba011 bash[31774]: cluster 2021-03-10T17:34:14.221951+0000 mgr.gibba032.whopeo (mgr.24625) 11667 : cluster [DBG] pgmap v6649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:14.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:14 gibba032 bash[24004]: cluster 2021-03-10T17:34:14.221951+0000 mgr.gibba032.whopeo (mgr.24625) 11667 : cluster [DBG] pgmap v6649: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:16.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:16 gibba011 bash[30791]: cluster 2021-03-10T17:34:16.223457+0000 mgr.gibba032.whopeo (mgr.24625) 11668 : cluster [DBG] pgmap v6650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:16 gibba011 bash[31774]: cluster 2021-03-10T17:34:16.223457+0000 mgr.gibba032.whopeo (mgr.24625) 11668 : cluster [DBG] pgmap v6650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:16 gibba032 bash[24004]: cluster 2021-03-10T17:34:16.223457+0000 mgr.gibba032.whopeo (mgr.24625) 11668 : cluster [DBG] pgmap v6650: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:18 gibba011 bash[30791]: cluster 2021-03-10T17:34:18.224995+0000 mgr.gibba032.whopeo (mgr.24625) 11669 : cluster [DBG] pgmap v6651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:18.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:18 gibba011 bash[31774]: cluster 2021-03-10T17:34:18.224995+0000 mgr.gibba032.whopeo (mgr.24625) 11669 : cluster [DBG] pgmap v6651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:18 gibba032 bash[24004]: cluster 2021-03-10T17:34:18.224995+0000 mgr.gibba032.whopeo (mgr.24625) 11669 : cluster [DBG] pgmap v6651: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:20 gibba011 bash[30791]: cluster 2021-03-10T17:34:20.226362+0000 mgr.gibba032.whopeo (mgr.24625) 11670 : cluster [DBG] pgmap v6652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:20 gibba011 bash[31774]: cluster 2021-03-10T17:34:20.226362+0000 mgr.gibba032.whopeo (mgr.24625) 11670 : cluster [DBG] pgmap v6652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:20.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:20 gibba032 bash[24004]: cluster 2021-03-10T17:34:20.226362+0000 mgr.gibba032.whopeo (mgr.24625) 11670 : cluster [DBG] pgmap v6652: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:22 gibba032 bash[24004]: cephadm 2021-03-10T17:34:21.745347+0000 mgr.gibba032.whopeo (mgr.24625) 11671 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:22 gibba032 bash[24004]: cluster 2021-03-10T17:34:22.227633+0000 mgr.gibba032.whopeo (mgr.24625) 11672 : cluster [DBG] pgmap v6653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:22 gibba011 bash[30791]: cephadm 2021-03-10T17:34:21.745347+0000 mgr.gibba032.whopeo (mgr.24625) 11671 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:22 gibba011 bash[30791]: cluster 2021-03-10T17:34:22.227633+0000 mgr.gibba032.whopeo (mgr.24625) 11672 : cluster [DBG] pgmap v6653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:22 gibba011 bash[31774]: cephadm 2021-03-10T17:34:21.745347+0000 mgr.gibba032.whopeo (mgr.24625) 11671 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:22 gibba011 bash[31774]: cluster 2021-03-10T17:34:22.227633+0000 mgr.gibba032.whopeo (mgr.24625) 11672 : cluster [DBG] pgmap v6653: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:24.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:24 gibba011 bash[30791]: cluster 2021-03-10T17:34:24.229131+0000 mgr.gibba032.whopeo (mgr.24625) 11673 : cluster [DBG] pgmap v6654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:24 gibba011 bash[31774]: cluster 2021-03-10T17:34:24.229131+0000 mgr.gibba032.whopeo (mgr.24625) 11673 : cluster [DBG] pgmap v6654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:24 gibba032 bash[24004]: cluster 2021-03-10T17:34:24.229131+0000 mgr.gibba032.whopeo (mgr.24625) 11673 : cluster [DBG] pgmap v6654: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:26 gibba011 bash[30791]: cluster 2021-03-10T17:34:26.230644+0000 mgr.gibba032.whopeo (mgr.24625) 11674 : cluster [DBG] pgmap v6655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:26 gibba011 bash[31774]: cluster 2021-03-10T17:34:26.230644+0000 mgr.gibba032.whopeo (mgr.24625) 11674 : cluster [DBG] pgmap v6655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:26 gibba032 bash[24004]: cluster 2021-03-10T17:34:26.230644+0000 mgr.gibba032.whopeo (mgr.24625) 11674 : cluster [DBG] pgmap v6655: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:28.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:28 gibba011 bash[30791]: cluster 2021-03-10T17:34:28.232331+0000 mgr.gibba032.whopeo (mgr.24625) 11675 : cluster [DBG] pgmap v6656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:28.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:28 gibba011 bash[31774]: cluster 2021-03-10T17:34:28.232331+0000 mgr.gibba032.whopeo (mgr.24625) 11675 : cluster [DBG] pgmap v6656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:28 gibba032 bash[24004]: cluster 2021-03-10T17:34:28.232331+0000 mgr.gibba032.whopeo (mgr.24625) 11675 : cluster [DBG] pgmap v6656: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:30.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:30 gibba011 bash[30791]: cluster 2021-03-10T17:34:30.233949+0000 mgr.gibba032.whopeo (mgr.24625) 11676 : cluster [DBG] pgmap v6657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:30 gibba011 bash[31774]: cluster 2021-03-10T17:34:30.233949+0000 mgr.gibba032.whopeo (mgr.24625) 11676 : cluster [DBG] pgmap v6657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:30.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:30 gibba032 bash[24004]: cluster 2021-03-10T17:34:30.233949+0000 mgr.gibba032.whopeo (mgr.24625) 11676 : cluster [DBG] pgmap v6657: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:32 gibba032 bash[24004]: cluster 2021-03-10T17:34:32.235386+0000 mgr.gibba032.whopeo (mgr.24625) 11677 : cluster [DBG] pgmap v6658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:32.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:32 gibba011 bash[30791]: cluster 2021-03-10T17:34:32.235386+0000 mgr.gibba032.whopeo (mgr.24625) 11677 : cluster [DBG] pgmap v6658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:32 gibba011 bash[31774]: cluster 2021-03-10T17:34:32.235386+0000 mgr.gibba032.whopeo (mgr.24625) 11677 : cluster [DBG] pgmap v6658: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:34.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:34 gibba011 bash[30791]: cluster 2021-03-10T17:34:34.237063+0000 mgr.gibba032.whopeo (mgr.24625) 11678 : cluster [DBG] pgmap v6659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:34 gibba011 bash[31774]: cluster 2021-03-10T17:34:34.237063+0000 mgr.gibba032.whopeo (mgr.24625) 11678 : cluster [DBG] pgmap v6659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:34 gibba032 bash[24004]: cluster 2021-03-10T17:34:34.237063+0000 mgr.gibba032.whopeo (mgr.24625) 11678 : cluster [DBG] pgmap v6659: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:36.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:36 gibba011 bash[30791]: cluster 2021-03-10T17:34:36.238608+0000 mgr.gibba032.whopeo (mgr.24625) 11679 : cluster [DBG] pgmap v6660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:36 gibba011 bash[31774]: cluster 2021-03-10T17:34:36.238608+0000 mgr.gibba032.whopeo (mgr.24625) 11679 : cluster [DBG] pgmap v6660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:36 gibba032 bash[24004]: cluster 2021-03-10T17:34:36.238608+0000 mgr.gibba032.whopeo (mgr.24625) 11679 : cluster [DBG] pgmap v6660: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:37.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:37 gibba032 bash[24004]: cephadm 2021-03-10T17:34:36.760876+0000 mgr.gibba032.whopeo (mgr.24625) 11680 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:37.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:37 gibba011 bash[30791]: cephadm 2021-03-10T17:34:36.760876+0000 mgr.gibba032.whopeo (mgr.24625) 11680 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:37.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:37 gibba011 bash[31774]: cephadm 2021-03-10T17:34:36.760876+0000 mgr.gibba032.whopeo (mgr.24625) 11680 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:38.281 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:34:38.507 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:34:38.507 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:34:38.508 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 107s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:34:38.508 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:34:38.508 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:34:38.508 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:34:38.509 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:34:38.509 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:34:38.509 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:34:38.509 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:34:38.509 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 107s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:34:38.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:34:38.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:34:38.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:34:38.510 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:34:38.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:34:38.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:34:38.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:34:38.511 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 107s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:34:38.511 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 107s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:34:38.512 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:34:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:38 gibba032 bash[24004]: cluster 2021-03-10T17:34:38.240265+0000 mgr.gibba032.whopeo (mgr.24625) 11681 : cluster [DBG] pgmap v6661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:38.762 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:34:38.762 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:34:38.763 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:34:38.763 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:38.763 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:34:38.763 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:34:38.764 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:38.764 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:34:38.764 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:34:38.764 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:34:38.764 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:34:38.765 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:34:38.765 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:34:38.765 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:34:38.765 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:34:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:38 gibba011 bash[30791]: cluster 2021-03-10T17:34:38.240265+0000 mgr.gibba032.whopeo (mgr.24625) 11681 : cluster [DBG] pgmap v6661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:38.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:38 gibba011 bash[31774]: cluster 2021-03-10T17:34:38.240265+0000 mgr.gibba032.whopeo (mgr.24625) 11681 : cluster [DBG] pgmap v6661: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:39.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[30791]: audit 2021-03-10T17:34:38.269651+0000 mgr.gibba032.whopeo (mgr.24625) 11682 : audit [DBG] from='client.61867 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[30791]: audit 2021-03-10T17:34:38.492541+0000 mgr.gibba032.whopeo (mgr.24625) 11683 : audit [DBG] from='client.51891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:39.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[30791]: audit 2021-03-10T17:34:38.761385+0000 mon.c (mon.1) 227 : audit [DBG] from='client.? 172.21.2.111:0/2994972763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:39.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[31774]: audit 2021-03-10T17:34:38.269651+0000 mgr.gibba032.whopeo (mgr.24625) 11682 : audit [DBG] from='client.61867 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[31774]: audit 2021-03-10T17:34:38.492541+0000 mgr.gibba032.whopeo (mgr.24625) 11683 : audit [DBG] from='client.51891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:39.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:39 gibba011 bash[31774]: audit 2021-03-10T17:34:38.761385+0000 mon.c (mon.1) 227 : audit [DBG] from='client.? 172.21.2.111:0/2994972763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:39 gibba032 bash[24004]: audit 2021-03-10T17:34:38.269651+0000 mgr.gibba032.whopeo (mgr.24625) 11682 : audit [DBG] from='client.61867 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:39 gibba032 bash[24004]: audit 2021-03-10T17:34:38.492541+0000 mgr.gibba032.whopeo (mgr.24625) 11683 : audit [DBG] from='client.51891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:34:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:39 gibba032 bash[24004]: audit 2021-03-10T17:34:38.761385+0000 mon.c (mon.1) 227 : audit [DBG] from='client.? 172.21.2.111:0/2994972763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:40.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:40 gibba011 bash[30791]: cluster 2021-03-10T17:34:40.241726+0000 mgr.gibba032.whopeo (mgr.24625) 11684 : cluster [DBG] pgmap v6662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:40 gibba011 bash[31774]: cluster 2021-03-10T17:34:40.241726+0000 mgr.gibba032.whopeo (mgr.24625) 11684 : cluster [DBG] pgmap v6662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:41.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:40 gibba032 bash[24004]: cluster 2021-03-10T17:34:40.241726+0000 mgr.gibba032.whopeo (mgr.24625) 11684 : cluster [DBG] pgmap v6662: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:42.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:42 gibba032 bash[24004]: cluster 2021-03-10T17:34:42.243208+0000 mgr.gibba032.whopeo (mgr.24625) 11685 : cluster [DBG] pgmap v6663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:42.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:42 gibba011 bash[30791]: cluster 2021-03-10T17:34:42.243208+0000 mgr.gibba032.whopeo (mgr.24625) 11685 : cluster [DBG] pgmap v6663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:42.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:42 gibba011 bash[31774]: cluster 2021-03-10T17:34:42.243208+0000 mgr.gibba032.whopeo (mgr.24625) 11685 : cluster [DBG] pgmap v6663: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:44.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:44 gibba011 bash[30791]: cluster 2021-03-10T17:34:44.244929+0000 mgr.gibba032.whopeo (mgr.24625) 11686 : cluster [DBG] pgmap v6664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:44 gibba011 bash[31774]: cluster 2021-03-10T17:34:44.244929+0000 mgr.gibba032.whopeo (mgr.24625) 11686 : cluster [DBG] pgmap v6664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:44 gibba032 bash[24004]: cluster 2021-03-10T17:34:44.244929+0000 mgr.gibba032.whopeo (mgr.24625) 11686 : cluster [DBG] pgmap v6664: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:46 gibba011 bash[30791]: cluster 2021-03-10T17:34:46.246475+0000 mgr.gibba032.whopeo (mgr.24625) 11687 : cluster [DBG] pgmap v6665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:46.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:46 gibba011 bash[31774]: cluster 2021-03-10T17:34:46.246475+0000 mgr.gibba032.whopeo (mgr.24625) 11687 : cluster [DBG] pgmap v6665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:46.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:46 gibba032 bash[24004]: cluster 2021-03-10T17:34:46.246475+0000 mgr.gibba032.whopeo (mgr.24625) 11687 : cluster [DBG] pgmap v6665: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:48 gibba011 bash[30791]: cluster 2021-03-10T17:34:48.248066+0000 mgr.gibba032.whopeo (mgr.24625) 11688 : cluster [DBG] pgmap v6666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:48.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:48 gibba011 bash[31774]: cluster 2021-03-10T17:34:48.248066+0000 mgr.gibba032.whopeo (mgr.24625) 11688 : cluster [DBG] pgmap v6666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:48.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:48 gibba032 bash[24004]: cluster 2021-03-10T17:34:48.248066+0000 mgr.gibba032.whopeo (mgr.24625) 11688 : cluster [DBG] pgmap v6666: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:50 gibba011 bash[30791]: cluster 2021-03-10T17:34:50.249397+0000 mgr.gibba032.whopeo (mgr.24625) 11689 : cluster [DBG] pgmap v6667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:50 gibba011 bash[31774]: cluster 2021-03-10T17:34:50.249397+0000 mgr.gibba032.whopeo (mgr.24625) 11689 : cluster [DBG] pgmap v6667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:50.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:50 gibba032 bash[24004]: cluster 2021-03-10T17:34:50.249397+0000 mgr.gibba032.whopeo (mgr.24625) 11689 : cluster [DBG] pgmap v6667: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:53.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.777705+0000 mgr.gibba032.whopeo (mgr.24625) 11690 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:34:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.779237+0000 mgr.gibba032.whopeo (mgr.24625) 11691 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:34:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:51.985873+0000 mon.a (mon.0) 3840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:34:53.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.986613+0000 mgr.gibba032.whopeo (mgr.24625) 11692 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:34:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.988581+0000 mgr.gibba032.whopeo (mgr.24625) 11693 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:34:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:51.989223+0000 mon.a (mon.0) 3841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:34:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:51.991082+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.991615+0000 mgr.gibba032.whopeo (mgr.24625) 11694 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:34:53.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.991749+0000 mgr.gibba032.whopeo (mgr.24625) 11695 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:34:53.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:51.994136+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.994451+0000 mgr.gibba032.whopeo (mgr.24625) 11696 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:34:53.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.994525+0000 mgr.gibba032.whopeo (mgr.24625) 11697 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:34:53.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:51.996849+0000 mon.a (mon.0) 3844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.997011+0000 mgr.gibba032.whopeo (mgr.24625) 11698 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:34:53.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.999026+0000 mgr.gibba032.whopeo (mgr.24625) 11699 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:34:53.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:51.999065+0000 mgr.gibba032.whopeo (mgr.24625) 11700 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:34:53.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:52.001512+0000 mon.a (mon.0) 3845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:52.001683+0000 mgr.gibba032.whopeo (mgr.24625) 11701 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:34:53.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:52.002179+0000 mgr.gibba032.whopeo (mgr.24625) 11702 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:34:53.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: audit 2021-03-10T17:34:52.004448+0000 mon.a (mon.0) 3846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:52.004519+0000 mgr.gibba032.whopeo (mgr.24625) 11703 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:34:53.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:52.004555+0000 mgr.gibba032.whopeo (mgr.24625) 11704 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:34:53.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cephadm 2021-03-10T17:34:52.004662+0000 mgr.gibba032.whopeo (mgr.24625) 11705 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:53.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:52 gibba032 bash[24004]: cluster 2021-03-10T17:34:52.250698+0000 mgr.gibba032.whopeo (mgr.24625) 11706 : cluster [DBG] pgmap v6668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.777705+0000 mgr.gibba032.whopeo (mgr.24625) 11690 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:34:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.779237+0000 mgr.gibba032.whopeo (mgr.24625) 11691 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:34:53.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:51.985873+0000 mon.a (mon.0) 3840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:34:53.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.986613+0000 mgr.gibba032.whopeo (mgr.24625) 11692 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:34:53.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.988581+0000 mgr.gibba032.whopeo (mgr.24625) 11693 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:34:53.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:51.989223+0000 mon.a (mon.0) 3841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:34:53.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:51.991082+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.991615+0000 mgr.gibba032.whopeo (mgr.24625) 11694 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:34:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.991749+0000 mgr.gibba032.whopeo (mgr.24625) 11695 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:34:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:51.994136+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.994451+0000 mgr.gibba032.whopeo (mgr.24625) 11696 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:34:53.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.994525+0000 mgr.gibba032.whopeo (mgr.24625) 11697 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:34:53.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:51.996849+0000 mon.a (mon.0) 3844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.997011+0000 mgr.gibba032.whopeo (mgr.24625) 11698 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:34:53.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.999026+0000 mgr.gibba032.whopeo (mgr.24625) 11699 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:34:53.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:51.999065+0000 mgr.gibba032.whopeo (mgr.24625) 11700 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:34:53.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:52.001512+0000 mon.a (mon.0) 3845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:52.001683+0000 mgr.gibba032.whopeo (mgr.24625) 11701 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:34:53.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:52.002179+0000 mgr.gibba032.whopeo (mgr.24625) 11702 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:34:53.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: audit 2021-03-10T17:34:52.004448+0000 mon.a (mon.0) 3846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:52.004519+0000 mgr.gibba032.whopeo (mgr.24625) 11703 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:34:53.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:52.004555+0000 mgr.gibba032.whopeo (mgr.24625) 11704 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:34:53.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cephadm 2021-03-10T17:34:52.004662+0000 mgr.gibba032.whopeo (mgr.24625) 11705 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:53.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[30791]: cluster 2021-03-10T17:34:52.250698+0000 mgr.gibba032.whopeo (mgr.24625) 11706 : cluster [DBG] pgmap v6668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:53.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.777705+0000 mgr.gibba032.whopeo (mgr.24625) 11690 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:34:53.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.779237+0000 mgr.gibba032.whopeo (mgr.24625) 11691 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:34:53.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:51.985873+0000 mon.a (mon.0) 3840 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:34:53.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.986613+0000 mgr.gibba032.whopeo (mgr.24625) 11692 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:34:53.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.988581+0000 mgr.gibba032.whopeo (mgr.24625) 11693 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:34:53.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:51.989223+0000 mon.a (mon.0) 3841 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:34:53.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:51.991082+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.991615+0000 mgr.gibba032.whopeo (mgr.24625) 11694 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:34:53.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.991749+0000 mgr.gibba032.whopeo (mgr.24625) 11695 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:34:53.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:51.994136+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.994451+0000 mgr.gibba032.whopeo (mgr.24625) 11696 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:34:53.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.994525+0000 mgr.gibba032.whopeo (mgr.24625) 11697 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:34:53.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:51.996849+0000 mon.a (mon.0) 3844 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.997011+0000 mgr.gibba032.whopeo (mgr.24625) 11698 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:34:53.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.999026+0000 mgr.gibba032.whopeo (mgr.24625) 11699 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:34:53.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:51.999065+0000 mgr.gibba032.whopeo (mgr.24625) 11700 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:34:53.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:52.001512+0000 mon.a (mon.0) 3845 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:52.001683+0000 mgr.gibba032.whopeo (mgr.24625) 11701 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:34:53.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:52.002179+0000 mgr.gibba032.whopeo (mgr.24625) 11702 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:34:53.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: audit 2021-03-10T17:34:52.004448+0000 mon.a (mon.0) 3846 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:34:53.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:52.004519+0000 mgr.gibba032.whopeo (mgr.24625) 11703 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:34:53.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:52.004555+0000 mgr.gibba032.whopeo (mgr.24625) 11704 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:34:53.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cephadm 2021-03-10T17:34:52.004662+0000 mgr.gibba032.whopeo (mgr.24625) 11705 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:34:53.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:52 gibba011 bash[31774]: cluster 2021-03-10T17:34:52.250698+0000 mgr.gibba032.whopeo (mgr.24625) 11706 : cluster [DBG] pgmap v6668: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:54.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:54 gibba011 bash[30791]: cluster 2021-03-10T17:34:54.252201+0000 mgr.gibba032.whopeo (mgr.24625) 11707 : cluster [DBG] pgmap v6669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:54.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:54 gibba011 bash[31774]: cluster 2021-03-10T17:34:54.252201+0000 mgr.gibba032.whopeo (mgr.24625) 11707 : cluster [DBG] pgmap v6669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:54.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:54 gibba032 bash[24004]: cluster 2021-03-10T17:34:54.252201+0000 mgr.gibba032.whopeo (mgr.24625) 11707 : cluster [DBG] pgmap v6669: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:56 gibba011 bash[30791]: cluster 2021-03-10T17:34:56.253672+0000 mgr.gibba032.whopeo (mgr.24625) 11708 : cluster [DBG] pgmap v6670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:56.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:56 gibba011 bash[31774]: cluster 2021-03-10T17:34:56.253672+0000 mgr.gibba032.whopeo (mgr.24625) 11708 : cluster [DBG] pgmap v6670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:56.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:56 gibba032 bash[24004]: cluster 2021-03-10T17:34:56.253672+0000 mgr.gibba032.whopeo (mgr.24625) 11708 : cluster [DBG] pgmap v6670: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:57 gibba032 bash[24004]: audit 2021-03-10T17:34:57.551170+0000 mon.a (mon.0) 3847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:34:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:57 gibba032 bash[24004]: audit 2021-03-10T17:34:57.551831+0000 mon.a (mon.0) 3848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:34:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:57 gibba011 bash[30791]: audit 2021-03-10T17:34:57.551170+0000 mon.a (mon.0) 3847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:34:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:57 gibba011 bash[30791]: audit 2021-03-10T17:34:57.551831+0000 mon.a (mon.0) 3848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:34:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:57 gibba011 bash[31774]: audit 2021-03-10T17:34:57.551170+0000 mon.a (mon.0) 3847 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:34:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:57 gibba011 bash[31774]: audit 2021-03-10T17:34:57.551831+0000 mon.a (mon.0) 3848 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:34:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:58 gibba032 bash[24004]: audit 2021-03-10T17:34:57.755512+0000 mon.a (mon.0) 3849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:34:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:58 gibba032 bash[24004]: audit 2021-03-10T17:34:57.756263+0000 mon.a (mon.0) 3850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:34:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:58 gibba032 bash[24004]: audit 2021-03-10T17:34:57.756942+0000 mon.a (mon.0) 3851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:58 gibba032 bash[24004]: audit 2021-03-10T17:34:57.757691+0000 mon.a (mon.0) 3852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:34:58 gibba032 bash[24004]: cluster 2021-03-10T17:34:58.255468+0000 mgr.gibba032.whopeo (mgr.24625) 11709 : cluster [DBG] pgmap v6671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:59.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[30791]: audit 2021-03-10T17:34:57.755512+0000 mon.a (mon.0) 3849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:34:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[30791]: audit 2021-03-10T17:34:57.756263+0000 mon.a (mon.0) 3850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:34:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[30791]: audit 2021-03-10T17:34:57.756942+0000 mon.a (mon.0) 3851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[30791]: audit 2021-03-10T17:34:57.757691+0000 mon.a (mon.0) 3852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[30791]: cluster 2021-03-10T17:34:58.255468+0000 mgr.gibba032.whopeo (mgr.24625) 11709 : cluster [DBG] pgmap v6671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:34:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[31774]: audit 2021-03-10T17:34:57.755512+0000 mon.a (mon.0) 3849 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2021-03-10T17:34:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[31774]: audit 2021-03-10T17:34:57.756263+0000 mon.a (mon.0) 3850 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 5]}]: dispatch 2021-03-10T17:34:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[31774]: audit 2021-03-10T17:34:57.756942+0000 mon.a (mon.0) 3851 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[31774]: audit 2021-03-10T17:34:57.757691+0000 mon.a (mon.0) 3852 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:34:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:34:58 gibba011 bash[31774]: cluster 2021-03-10T17:34:58.255468+0000 mgr.gibba032.whopeo (mgr.24625) 11709 : cluster [DBG] pgmap v6671: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:00 gibba011 bash[30791]: cluster 2021-03-10T17:35:00.256976+0000 mgr.gibba032.whopeo (mgr.24625) 11710 : cluster [DBG] pgmap v6672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:00 gibba011 bash[31774]: cluster 2021-03-10T17:35:00.256976+0000 mgr.gibba032.whopeo (mgr.24625) 11710 : cluster [DBG] pgmap v6672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:00 gibba032 bash[24004]: cluster 2021-03-10T17:35:00.256976+0000 mgr.gibba032.whopeo (mgr.24625) 11710 : cluster [DBG] pgmap v6672: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:02.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:02 gibba011 bash[30791]: cluster 2021-03-10T17:35:02.258442+0000 mgr.gibba032.whopeo (mgr.24625) 11711 : cluster [DBG] pgmap v6673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:02.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:02 gibba011 bash[31774]: cluster 2021-03-10T17:35:02.258442+0000 mgr.gibba032.whopeo (mgr.24625) 11711 : cluster [DBG] pgmap v6673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:03.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:02 gibba032 bash[24004]: cluster 2021-03-10T17:35:02.258442+0000 mgr.gibba032.whopeo (mgr.24625) 11711 : cluster [DBG] pgmap v6673: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:04 gibba011 bash[30791]: cluster 2021-03-10T17:35:04.259968+0000 mgr.gibba032.whopeo (mgr.24625) 11712 : cluster [DBG] pgmap v6674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:04 gibba011 bash[31774]: cluster 2021-03-10T17:35:04.259968+0000 mgr.gibba032.whopeo (mgr.24625) 11712 : cluster [DBG] pgmap v6674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:04.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:04 gibba032 bash[24004]: cluster 2021-03-10T17:35:04.259968+0000 mgr.gibba032.whopeo (mgr.24625) 11712 : cluster [DBG] pgmap v6674: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:06 gibba011 bash[30791]: cluster 2021-03-10T17:35:06.261567+0000 mgr.gibba032.whopeo (mgr.24625) 11713 : cluster [DBG] pgmap v6675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:06 gibba011 bash[31774]: cluster 2021-03-10T17:35:06.261567+0000 mgr.gibba032.whopeo (mgr.24625) 11713 : cluster [DBG] pgmap v6675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:06 gibba032 bash[24004]: cluster 2021-03-10T17:35:06.261567+0000 mgr.gibba032.whopeo (mgr.24625) 11713 : cluster [DBG] pgmap v6675: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:07.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:07 gibba011 bash[30791]: cephadm 2021-03-10T17:35:07.020102+0000 mgr.gibba032.whopeo (mgr.24625) 11714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:07.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:07 gibba011 bash[31774]: cephadm 2021-03-10T17:35:07.020102+0000 mgr.gibba032.whopeo (mgr.24625) 11714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:08.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:07 gibba032 bash[24004]: cephadm 2021-03-10T17:35:07.020102+0000 mgr.gibba032.whopeo (mgr.24625) 11714 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:09.006 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:35:09.234 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:35:09.234 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:35:09.234 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:35:09.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:35:09.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:35:09.235 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:35:09.235 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:35:09.236 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:35:09.236 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:35:09.236 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:35:09.236 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:35:09.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:35:09.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:35:09.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:35:09.237 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:35:09.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:35:09.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:35:09.238 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:35:09.239 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:35:09.239 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:35:09.239 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:35:09.488 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:35:09.488 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:35:09.489 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:35:09.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:09.490 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:35:09.490 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:35:09.490 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:09.491 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:35:09.491 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:35:09.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:09.492 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:35:09.492 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:35:09.493 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:35:09.494 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:35:09.495 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:35:09.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[30791]: cluster 2021-03-10T17:35:08.263213+0000 mgr.gibba032.whopeo (mgr.24625) 11715 : cluster [DBG] pgmap v6676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[30791]: audit 2021-03-10T17:35:08.994152+0000 mgr.gibba032.whopeo (mgr.24625) 11716 : audit [DBG] from='client.51903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[30791]: audit 2021-03-10T17:35:09.219883+0000 mgr.gibba032.whopeo (mgr.24625) 11717 : audit [DBG] from='client.61891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:09.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[30791]: audit 2021-03-10T17:35:09.486742+0000 mon.a (mon.0) 3853 : audit [DBG] from='client.? 172.21.2.111:0/3030089206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:09.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[31774]: cluster 2021-03-10T17:35:08.263213+0000 mgr.gibba032.whopeo (mgr.24625) 11715 : cluster [DBG] pgmap v6676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:09.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[31774]: audit 2021-03-10T17:35:08.994152+0000 mgr.gibba032.whopeo (mgr.24625) 11716 : audit [DBG] from='client.51903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:09.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[31774]: audit 2021-03-10T17:35:09.219883+0000 mgr.gibba032.whopeo (mgr.24625) 11717 : audit [DBG] from='client.61891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:09.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:09 gibba011 bash[31774]: audit 2021-03-10T17:35:09.486742+0000 mon.a (mon.0) 3853 : audit [DBG] from='client.? 172.21.2.111:0/3030089206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:09 gibba032 bash[24004]: cluster 2021-03-10T17:35:08.263213+0000 mgr.gibba032.whopeo (mgr.24625) 11715 : cluster [DBG] pgmap v6676: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:09 gibba032 bash[24004]: audit 2021-03-10T17:35:08.994152+0000 mgr.gibba032.whopeo (mgr.24625) 11716 : audit [DBG] from='client.51903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:09 gibba032 bash[24004]: audit 2021-03-10T17:35:09.219883+0000 mgr.gibba032.whopeo (mgr.24625) 11717 : audit [DBG] from='client.61891 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:09 gibba032 bash[24004]: audit 2021-03-10T17:35:09.486742+0000 mon.a (mon.0) 3853 : audit [DBG] from='client.? 172.21.2.111:0/3030089206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:11.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:11 gibba011 bash[30791]: cluster 2021-03-10T17:35:10.264585+0000 mgr.gibba032.whopeo (mgr.24625) 11718 : cluster [DBG] pgmap v6677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:11.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:11 gibba011 bash[31774]: cluster 2021-03-10T17:35:10.264585+0000 mgr.gibba032.whopeo (mgr.24625) 11718 : cluster [DBG] pgmap v6677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:12.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:11 gibba032 bash[24004]: cluster 2021-03-10T17:35:10.264585+0000 mgr.gibba032.whopeo (mgr.24625) 11718 : cluster [DBG] pgmap v6677: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:13.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:13 gibba011 bash[30791]: cluster 2021-03-10T17:35:12.266013+0000 mgr.gibba032.whopeo (mgr.24625) 11719 : cluster [DBG] pgmap v6678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:13.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:13 gibba011 bash[31774]: cluster 2021-03-10T17:35:12.266013+0000 mgr.gibba032.whopeo (mgr.24625) 11719 : cluster [DBG] pgmap v6678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:14.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:13 gibba032 bash[24004]: cluster 2021-03-10T17:35:12.266013+0000 mgr.gibba032.whopeo (mgr.24625) 11719 : cluster [DBG] pgmap v6678: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:15.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:15 gibba011 bash[30791]: cluster 2021-03-10T17:35:14.267619+0000 mgr.gibba032.whopeo (mgr.24625) 11720 : cluster [DBG] pgmap v6679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:15.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:15 gibba011 bash[31774]: cluster 2021-03-10T17:35:14.267619+0000 mgr.gibba032.whopeo (mgr.24625) 11720 : cluster [DBG] pgmap v6679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:16.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:15 gibba032 bash[24004]: cluster 2021-03-10T17:35:14.267619+0000 mgr.gibba032.whopeo (mgr.24625) 11720 : cluster [DBG] pgmap v6679: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:17.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:17 gibba011 bash[30791]: cluster 2021-03-10T17:35:16.269137+0000 mgr.gibba032.whopeo (mgr.24625) 11721 : cluster [DBG] pgmap v6680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:17.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:17 gibba011 bash[31774]: cluster 2021-03-10T17:35:16.269137+0000 mgr.gibba032.whopeo (mgr.24625) 11721 : cluster [DBG] pgmap v6680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:17 gibba032 bash[24004]: cluster 2021-03-10T17:35:16.269137+0000 mgr.gibba032.whopeo (mgr.24625) 11721 : cluster [DBG] pgmap v6680: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:19.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:19 gibba011 bash[30791]: cluster 2021-03-10T17:35:18.270806+0000 mgr.gibba032.whopeo (mgr.24625) 11722 : cluster [DBG] pgmap v6681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:19.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:19 gibba011 bash[31774]: cluster 2021-03-10T17:35:18.270806+0000 mgr.gibba032.whopeo (mgr.24625) 11722 : cluster [DBG] pgmap v6681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:20.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:19 gibba032 bash[24004]: cluster 2021-03-10T17:35:18.270806+0000 mgr.gibba032.whopeo (mgr.24625) 11722 : cluster [DBG] pgmap v6681: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:21.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:21 gibba011 bash[30791]: cluster 2021-03-10T17:35:20.272329+0000 mgr.gibba032.whopeo (mgr.24625) 11723 : cluster [DBG] pgmap v6682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:21.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:21 gibba011 bash[31774]: cluster 2021-03-10T17:35:20.272329+0000 mgr.gibba032.whopeo (mgr.24625) 11723 : cluster [DBG] pgmap v6682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:22.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:21 gibba032 bash[24004]: cluster 2021-03-10T17:35:20.272329+0000 mgr.gibba032.whopeo (mgr.24625) 11723 : cluster [DBG] pgmap v6682: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:22.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:22 gibba011 bash[30791]: cephadm 2021-03-10T17:35:22.032967+0000 mgr.gibba032.whopeo (mgr.24625) 11724 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:22.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:22 gibba011 bash[31774]: cephadm 2021-03-10T17:35:22.032967+0000 mgr.gibba032.whopeo (mgr.24625) 11724 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:23.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:22 gibba032 bash[24004]: cephadm 2021-03-10T17:35:22.032967+0000 mgr.gibba032.whopeo (mgr.24625) 11724 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:23 gibba032 bash[24004]: cluster 2021-03-10T17:35:22.274022+0000 mgr.gibba032.whopeo (mgr.24625) 11725 : cluster [DBG] pgmap v6683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:23 gibba011 bash[30791]: cluster 2021-03-10T17:35:22.274022+0000 mgr.gibba032.whopeo (mgr.24625) 11725 : cluster [DBG] pgmap v6683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:23 gibba011 bash[31774]: cluster 2021-03-10T17:35:22.274022+0000 mgr.gibba032.whopeo (mgr.24625) 11725 : cluster [DBG] pgmap v6683: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:25 gibba032 bash[24004]: cluster 2021-03-10T17:35:24.275779+0000 mgr.gibba032.whopeo (mgr.24625) 11726 : cluster [DBG] pgmap v6684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:26.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:25 gibba011 bash[30791]: cluster 2021-03-10T17:35:24.275779+0000 mgr.gibba032.whopeo (mgr.24625) 11726 : cluster [DBG] pgmap v6684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:26.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:25 gibba011 bash[31774]: cluster 2021-03-10T17:35:24.275779+0000 mgr.gibba032.whopeo (mgr.24625) 11726 : cluster [DBG] pgmap v6684: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:27 gibba032 bash[24004]: cluster 2021-03-10T17:35:26.277462+0000 mgr.gibba032.whopeo (mgr.24625) 11727 : cluster [DBG] pgmap v6685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:27 gibba011 bash[30791]: cluster 2021-03-10T17:35:26.277462+0000 mgr.gibba032.whopeo (mgr.24625) 11727 : cluster [DBG] pgmap v6685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:28.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:27 gibba011 bash[31774]: cluster 2021-03-10T17:35:26.277462+0000 mgr.gibba032.whopeo (mgr.24625) 11727 : cluster [DBG] pgmap v6685: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:30.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:29 gibba032 bash[24004]: cluster 2021-03-10T17:35:28.279285+0000 mgr.gibba032.whopeo (mgr.24625) 11728 : cluster [DBG] pgmap v6686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:29 gibba011 bash[30791]: cluster 2021-03-10T17:35:28.279285+0000 mgr.gibba032.whopeo (mgr.24625) 11728 : cluster [DBG] pgmap v6686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:30.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:29 gibba011 bash[31774]: cluster 2021-03-10T17:35:28.279285+0000 mgr.gibba032.whopeo (mgr.24625) 11728 : cluster [DBG] pgmap v6686: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:32.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:31 gibba032 bash[24004]: cluster 2021-03-10T17:35:30.281047+0000 mgr.gibba032.whopeo (mgr.24625) 11729 : cluster [DBG] pgmap v6687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:31 gibba011 bash[30791]: cluster 2021-03-10T17:35:30.281047+0000 mgr.gibba032.whopeo (mgr.24625) 11729 : cluster [DBG] pgmap v6687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:31 gibba011 bash[31774]: cluster 2021-03-10T17:35:30.281047+0000 mgr.gibba032.whopeo (mgr.24625) 11729 : cluster [DBG] pgmap v6687: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:34.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:33 gibba032 bash[24004]: cluster 2021-03-10T17:35:32.282562+0000 mgr.gibba032.whopeo (mgr.24625) 11730 : cluster [DBG] pgmap v6688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:34.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:33 gibba011 bash[30791]: cluster 2021-03-10T17:35:32.282562+0000 mgr.gibba032.whopeo (mgr.24625) 11730 : cluster [DBG] pgmap v6688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:34.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:33 gibba011 bash[31774]: cluster 2021-03-10T17:35:32.282562+0000 mgr.gibba032.whopeo (mgr.24625) 11730 : cluster [DBG] pgmap v6688: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:36.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:35 gibba032 bash[24004]: cluster 2021-03-10T17:35:34.283893+0000 mgr.gibba032.whopeo (mgr.24625) 11731 : cluster [DBG] pgmap v6689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:36.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:35 gibba011 bash[30791]: cluster 2021-03-10T17:35:34.283893+0000 mgr.gibba032.whopeo (mgr.24625) 11731 : cluster [DBG] pgmap v6689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:36.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:35 gibba011 bash[31774]: cluster 2021-03-10T17:35:34.283893+0000 mgr.gibba032.whopeo (mgr.24625) 11731 : cluster [DBG] pgmap v6689: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:37 gibba032 bash[24004]: cluster 2021-03-10T17:35:36.285841+0000 mgr.gibba032.whopeo (mgr.24625) 11732 : cluster [DBG] pgmap v6690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:37 gibba032 bash[24004]: cephadm 2021-03-10T17:35:37.048949+0000 mgr.gibba032.whopeo (mgr.24625) 11733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:37 gibba011 bash[30791]: cluster 2021-03-10T17:35:36.285841+0000 mgr.gibba032.whopeo (mgr.24625) 11732 : cluster [DBG] pgmap v6690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:37 gibba011 bash[30791]: cephadm 2021-03-10T17:35:37.048949+0000 mgr.gibba032.whopeo (mgr.24625) 11733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:37 gibba011 bash[31774]: cluster 2021-03-10T17:35:36.285841+0000 mgr.gibba032.whopeo (mgr.24625) 11732 : cluster [DBG] pgmap v6690: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:37 gibba011 bash[31774]: cephadm 2021-03-10T17:35:37.048949+0000 mgr.gibba032.whopeo (mgr.24625) 11733 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:39.733 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:35:39.963 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:35:39.963 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:35:39.963 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:35:39.964 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:35:39.964 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:35:39.964 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:35:39.964 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:35:39.965 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:35:39.965 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:35:39.965 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:35:39.965 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:35:39.965 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:35:39.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:35:39.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:35:39.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:35:39.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:35:39.966 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:35:39.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:35:39.967 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:35:39.967 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:35:39.968 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:35:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:39 gibba032 bash[24004]: cluster 2021-03-10T17:35:38.287845+0000 mgr.gibba032.whopeo (mgr.24625) 11734 : cluster [DBG] pgmap v6691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:39 gibba011 bash[30791]: cluster 2021-03-10T17:35:38.287845+0000 mgr.gibba032.whopeo (mgr.24625) 11734 : cluster [DBG] pgmap v6691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:40.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:39 gibba011 bash[31774]: cluster 2021-03-10T17:35:38.287845+0000 mgr.gibba032.whopeo (mgr.24625) 11734 : cluster [DBG] pgmap v6691: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:40.212 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:35:40.213 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:35:40.213 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:35:40.213 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:40.213 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:35:40.214 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:35:40.214 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:40.214 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:35:40.214 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:35:40.215 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:35:40.215 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:35:40.215 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:35:40.215 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:35:40.215 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:35:40.216 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:35:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:40 gibba032 bash[24004]: audit 2021-03-10T17:35:39.721441+0000 mgr.gibba032.whopeo (mgr.24625) 11735 : audit [DBG] from='client.61903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:40 gibba032 bash[24004]: audit 2021-03-10T17:35:39.947796+0000 mgr.gibba032.whopeo (mgr.24625) 11736 : audit [DBG] from='client.61909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:40 gibba032 bash[24004]: audit 2021-03-10T17:35:40.211303+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.2.111:0/1433398104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:41.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[30791]: audit 2021-03-10T17:35:39.721441+0000 mgr.gibba032.whopeo (mgr.24625) 11735 : audit [DBG] from='client.61903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[30791]: audit 2021-03-10T17:35:39.947796+0000 mgr.gibba032.whopeo (mgr.24625) 11736 : audit [DBG] from='client.61909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[30791]: audit 2021-03-10T17:35:40.211303+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.2.111:0/1433398104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:41.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[31774]: audit 2021-03-10T17:35:39.721441+0000 mgr.gibba032.whopeo (mgr.24625) 11735 : audit [DBG] from='client.61903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[31774]: audit 2021-03-10T17:35:39.947796+0000 mgr.gibba032.whopeo (mgr.24625) 11736 : audit [DBG] from='client.61909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:35:41.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:40 gibba011 bash[31774]: audit 2021-03-10T17:35:40.211303+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.2.111:0/1433398104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:42.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:41 gibba032 bash[24004]: cluster 2021-03-10T17:35:40.289637+0000 mgr.gibba032.whopeo (mgr.24625) 11737 : cluster [DBG] pgmap v6692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:41 gibba011 bash[30791]: cluster 2021-03-10T17:35:40.289637+0000 mgr.gibba032.whopeo (mgr.24625) 11737 : cluster [DBG] pgmap v6692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:42.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:41 gibba011 bash[31774]: cluster 2021-03-10T17:35:40.289637+0000 mgr.gibba032.whopeo (mgr.24625) 11737 : cluster [DBG] pgmap v6692: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:44.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:43 gibba032 bash[24004]: cluster 2021-03-10T17:35:42.291322+0000 mgr.gibba032.whopeo (mgr.24625) 11738 : cluster [DBG] pgmap v6693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:44.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:43 gibba011 bash[30791]: cluster 2021-03-10T17:35:42.291322+0000 mgr.gibba032.whopeo (mgr.24625) 11738 : cluster [DBG] pgmap v6693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:43 gibba011 bash[31774]: cluster 2021-03-10T17:35:42.291322+0000 mgr.gibba032.whopeo (mgr.24625) 11738 : cluster [DBG] pgmap v6693: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:46.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:45 gibba032 bash[24004]: cluster 2021-03-10T17:35:44.292996+0000 mgr.gibba032.whopeo (mgr.24625) 11739 : cluster [DBG] pgmap v6694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:45 gibba011 bash[30791]: cluster 2021-03-10T17:35:44.292996+0000 mgr.gibba032.whopeo (mgr.24625) 11739 : cluster [DBG] pgmap v6694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:45 gibba011 bash[31774]: cluster 2021-03-10T17:35:44.292996+0000 mgr.gibba032.whopeo (mgr.24625) 11739 : cluster [DBG] pgmap v6694: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:48.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:47 gibba032 bash[24004]: cluster 2021-03-10T17:35:46.294626+0000 mgr.gibba032.whopeo (mgr.24625) 11740 : cluster [DBG] pgmap v6695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:47 gibba011 bash[30791]: cluster 2021-03-10T17:35:46.294626+0000 mgr.gibba032.whopeo (mgr.24625) 11740 : cluster [DBG] pgmap v6695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:48.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:47 gibba011 bash[31774]: cluster 2021-03-10T17:35:46.294626+0000 mgr.gibba032.whopeo (mgr.24625) 11740 : cluster [DBG] pgmap v6695: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:50.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:49 gibba032 bash[24004]: cluster 2021-03-10T17:35:48.296345+0000 mgr.gibba032.whopeo (mgr.24625) 11741 : cluster [DBG] pgmap v6696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:49 gibba011 bash[30791]: cluster 2021-03-10T17:35:48.296345+0000 mgr.gibba032.whopeo (mgr.24625) 11741 : cluster [DBG] pgmap v6696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:50.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:49 gibba011 bash[31774]: cluster 2021-03-10T17:35:48.296345+0000 mgr.gibba032.whopeo (mgr.24625) 11741 : cluster [DBG] pgmap v6696: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:52.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:51 gibba032 bash[24004]: cluster 2021-03-10T17:35:50.297794+0000 mgr.gibba032.whopeo (mgr.24625) 11742 : cluster [DBG] pgmap v6697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:52.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:51 gibba011 bash[30791]: cluster 2021-03-10T17:35:50.297794+0000 mgr.gibba032.whopeo (mgr.24625) 11742 : cluster [DBG] pgmap v6697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:52.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:51 gibba011 bash[31774]: cluster 2021-03-10T17:35:50.297794+0000 mgr.gibba032.whopeo (mgr.24625) 11742 : cluster [DBG] pgmap v6697: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.061624+0000 mgr.gibba032.whopeo (mgr.24625) 11743 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:35:53.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11744 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:35:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.280261+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:35:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.284010+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:35:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.285269+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.286585+0000 mon.a (mon.0) 3858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.287681+0000 mon.a (mon.0) 3859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.290168+0000 mon.a (mon.0) 3860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[30791]: audit 2021-03-10T17:35:52.291558+0000 mon.a (mon.0) 3861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.061624+0000 mgr.gibba032.whopeo (mgr.24625) 11743 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:35:53.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11744 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:35:53.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.280261+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:35:53.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.284010+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:35:53.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.285269+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.286585+0000 mon.a (mon.0) 3858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.287681+0000 mon.a (mon.0) 3859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.290168+0000 mon.a (mon.0) 3860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:53 gibba011 bash[31774]: audit 2021-03-10T17:35:52.291558+0000 mon.a (mon.0) 3861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.061624+0000 mgr.gibba032.whopeo (mgr.24625) 11743 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:35:53.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.063190+0000 mgr.gibba032.whopeo (mgr.24625) 11744 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:35:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.280261+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:35:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.284010+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:35:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.285269+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.286585+0000 mon.a (mon.0) 3858 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.287681+0000 mon.a (mon.0) 3859 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.290168+0000 mon.a (mon.0) 3860 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:53.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:53 gibba032 bash[24004]: audit 2021-03-10T17:35:52.291558+0000 mon.a (mon.0) 3861 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:35:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.281251+0000 mgr.gibba032.whopeo (mgr.24625) 11745 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:35:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.283304+0000 mgr.gibba032.whopeo (mgr.24625) 11746 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:35:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.284403+0000 mgr.gibba032.whopeo (mgr.24625) 11747 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:35:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.284488+0000 mgr.gibba032.whopeo (mgr.24625) 11748 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:35:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.285684+0000 mgr.gibba032.whopeo (mgr.24625) 11749 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:35:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.285722+0000 mgr.gibba032.whopeo (mgr.24625) 11750 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:35:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.286784+0000 mgr.gibba032.whopeo (mgr.24625) 11751 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:35:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.287829+0000 mgr.gibba032.whopeo (mgr.24625) 11752 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:35:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.287866+0000 mgr.gibba032.whopeo (mgr.24625) 11753 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:35:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.289222+0000 mgr.gibba032.whopeo (mgr.24625) 11754 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:35:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.289542+0000 mgr.gibba032.whopeo (mgr.24625) 11755 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:35:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.290570+0000 mgr.gibba032.whopeo (mgr.24625) 11756 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:35:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.290609+0000 mgr.gibba032.whopeo (mgr.24625) 11757 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:35:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cephadm 2021-03-10T17:35:52.290713+0000 mgr.gibba032.whopeo (mgr.24625) 11758 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[30791]: cluster 2021-03-10T17:35:52.298268+0000 mgr.gibba032.whopeo (mgr.24625) 11759 : cluster [DBG] pgmap v6698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.281251+0000 mgr.gibba032.whopeo (mgr.24625) 11745 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:35:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.283304+0000 mgr.gibba032.whopeo (mgr.24625) 11746 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:35:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.284403+0000 mgr.gibba032.whopeo (mgr.24625) 11747 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:35:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.284488+0000 mgr.gibba032.whopeo (mgr.24625) 11748 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:35:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.285684+0000 mgr.gibba032.whopeo (mgr.24625) 11749 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:35:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.285722+0000 mgr.gibba032.whopeo (mgr.24625) 11750 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:35:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.286784+0000 mgr.gibba032.whopeo (mgr.24625) 11751 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:35:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.287829+0000 mgr.gibba032.whopeo (mgr.24625) 11752 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:35:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.287866+0000 mgr.gibba032.whopeo (mgr.24625) 11753 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:35:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.289222+0000 mgr.gibba032.whopeo (mgr.24625) 11754 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:35:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.289542+0000 mgr.gibba032.whopeo (mgr.24625) 11755 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:35:54.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.290570+0000 mgr.gibba032.whopeo (mgr.24625) 11756 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:35:54.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.290609+0000 mgr.gibba032.whopeo (mgr.24625) 11757 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:35:54.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cephadm 2021-03-10T17:35:52.290713+0000 mgr.gibba032.whopeo (mgr.24625) 11758 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:54.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:54 gibba011 bash[31774]: cluster 2021-03-10T17:35:52.298268+0000 mgr.gibba032.whopeo (mgr.24625) 11759 : cluster [DBG] pgmap v6698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.281251+0000 mgr.gibba032.whopeo (mgr.24625) 11745 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:35:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.283304+0000 mgr.gibba032.whopeo (mgr.24625) 11746 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:35:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.284403+0000 mgr.gibba032.whopeo (mgr.24625) 11747 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:35:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.284488+0000 mgr.gibba032.whopeo (mgr.24625) 11748 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:35:54.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.285684+0000 mgr.gibba032.whopeo (mgr.24625) 11749 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:35:54.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.285722+0000 mgr.gibba032.whopeo (mgr.24625) 11750 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:35:54.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.286784+0000 mgr.gibba032.whopeo (mgr.24625) 11751 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:35:54.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.287829+0000 mgr.gibba032.whopeo (mgr.24625) 11752 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:35:54.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.287866+0000 mgr.gibba032.whopeo (mgr.24625) 11753 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:35:54.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.289222+0000 mgr.gibba032.whopeo (mgr.24625) 11754 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:35:54.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.289542+0000 mgr.gibba032.whopeo (mgr.24625) 11755 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:35:54.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.290570+0000 mgr.gibba032.whopeo (mgr.24625) 11756 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:35:54.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.290609+0000 mgr.gibba032.whopeo (mgr.24625) 11757 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:35:54.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cephadm 2021-03-10T17:35:52.290713+0000 mgr.gibba032.whopeo (mgr.24625) 11758 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:35:54.764 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:54 gibba032 bash[24004]: cluster 2021-03-10T17:35:52.298268+0000 mgr.gibba032.whopeo (mgr.24625) 11759 : cluster [DBG] pgmap v6698: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:55.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:55 gibba011 bash[30791]: cluster 2021-03-10T17:35:54.299451+0000 mgr.gibba032.whopeo (mgr.24625) 11760 : cluster [DBG] pgmap v6699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:55.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:55 gibba011 bash[31774]: cluster 2021-03-10T17:35:54.299451+0000 mgr.gibba032.whopeo (mgr.24625) 11760 : cluster [DBG] pgmap v6699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:55.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:55 gibba032 bash[24004]: cluster 2021-03-10T17:35:54.299451+0000 mgr.gibba032.whopeo (mgr.24625) 11760 : cluster [DBG] pgmap v6699: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:57.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:57 gibba011 bash[30791]: cluster 2021-03-10T17:35:56.301188+0000 mgr.gibba032.whopeo (mgr.24625) 11761 : cluster [DBG] pgmap v6700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:57.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:57 gibba011 bash[31774]: cluster 2021-03-10T17:35:56.301188+0000 mgr.gibba032.whopeo (mgr.24625) 11761 : cluster [DBG] pgmap v6700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:58.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:57 gibba032 bash[24004]: cluster 2021-03-10T17:35:56.301188+0000 mgr.gibba032.whopeo (mgr.24625) 11761 : cluster [DBG] pgmap v6700: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:58.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.553984+0000 mon.a (mon.0) 3862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:35:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.554135+0000 mon.a (mon.0) 3863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:35:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.764694+0000 mon.a (mon.0) 3864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T17:35:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.765460+0000 mon.a (mon.0) 3865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:35:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.766152+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:35:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[30791]: audit 2021-03-10T17:35:57.766912+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:35:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.553984+0000 mon.a (mon.0) 3862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:35:58.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.554135+0000 mon.a (mon.0) 3863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:35:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.764694+0000 mon.a (mon.0) 3864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T17:35:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.765460+0000 mon.a (mon.0) 3865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:35:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.766152+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:35:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:58 gibba011 bash[31774]: audit 2021-03-10T17:35:57.766912+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:35:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.553984+0000 mon.a (mon.0) 3862 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:35:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.554135+0000 mon.a (mon.0) 3863 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:35:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.764694+0000 mon.a (mon.0) 3864 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2021-03-10T17:35:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.765460+0000 mon.a (mon.0) 3865 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:35:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.766152+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2021-03-10T17:35:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:58 gibba032 bash[24004]: audit 2021-03-10T17:35:57.766912+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2021-03-10T17:35:59.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:35:59 gibba011 bash[30791]: cluster 2021-03-10T17:35:58.302949+0000 mgr.gibba032.whopeo (mgr.24625) 11762 : cluster [DBG] pgmap v6701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:35:59.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:35:59 gibba011 bash[31774]: cluster 2021-03-10T17:35:58.302949+0000 mgr.gibba032.whopeo (mgr.24625) 11762 : cluster [DBG] pgmap v6701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:35:59 gibba032 bash[24004]: cluster 2021-03-10T17:35:58.302949+0000 mgr.gibba032.whopeo (mgr.24625) 11762 : cluster [DBG] pgmap v6701: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:01.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:01 gibba011 bash[30791]: cluster 2021-03-10T17:36:00.304450+0000 mgr.gibba032.whopeo (mgr.24625) 11763 : cluster [DBG] pgmap v6702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:01.830 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:01 gibba011 bash[31774]: cluster 2021-03-10T17:36:00.304450+0000 mgr.gibba032.whopeo (mgr.24625) 11763 : cluster [DBG] pgmap v6702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:02.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:01 gibba032 bash[24004]: cluster 2021-03-10T17:36:00.304450+0000 mgr.gibba032.whopeo (mgr.24625) 11763 : cluster [DBG] pgmap v6702: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:03.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:03 gibba011 bash[30791]: cluster 2021-03-10T17:36:02.305980+0000 mgr.gibba032.whopeo (mgr.24625) 11764 : cluster [DBG] pgmap v6703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:03.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:03 gibba011 bash[31774]: cluster 2021-03-10T17:36:02.305980+0000 mgr.gibba032.whopeo (mgr.24625) 11764 : cluster [DBG] pgmap v6703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:04.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:03 gibba032 bash[24004]: cluster 2021-03-10T17:36:02.305980+0000 mgr.gibba032.whopeo (mgr.24625) 11764 : cluster [DBG] pgmap v6703: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:05.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:05 gibba011 bash[30791]: cluster 2021-03-10T17:36:04.307416+0000 mgr.gibba032.whopeo (mgr.24625) 11765 : cluster [DBG] pgmap v6704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:05.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:05 gibba011 bash[31774]: cluster 2021-03-10T17:36:04.307416+0000 mgr.gibba032.whopeo (mgr.24625) 11765 : cluster [DBG] pgmap v6704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:06.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:05 gibba032 bash[24004]: cluster 2021-03-10T17:36:04.307416+0000 mgr.gibba032.whopeo (mgr.24625) 11765 : cluster [DBG] pgmap v6704: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:08.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:07 gibba032 bash[24004]: cluster 2021-03-10T17:36:06.309116+0000 mgr.gibba032.whopeo (mgr.24625) 11766 : cluster [DBG] pgmap v6705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:07 gibba011 bash[30791]: cluster 2021-03-10T17:36:06.309116+0000 mgr.gibba032.whopeo (mgr.24625) 11766 : cluster [DBG] pgmap v6705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:07 gibba011 bash[31774]: cluster 2021-03-10T17:36:06.309116+0000 mgr.gibba032.whopeo (mgr.24625) 11766 : cluster [DBG] pgmap v6705: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:08 gibba032 bash[24004]: cephadm 2021-03-10T17:36:07.297104+0000 mgr.gibba032.whopeo (mgr.24625) 11767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:08 gibba011 bash[30791]: cephadm 2021-03-10T17:36:07.297104+0000 mgr.gibba032.whopeo (mgr.24625) 11767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:09.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:08 gibba011 bash[31774]: cephadm 2021-03-10T17:36:07.297104+0000 mgr.gibba032.whopeo (mgr.24625) 11767 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:10.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:09 gibba032 bash[24004]: cluster 2021-03-10T17:36:08.310837+0000 mgr.gibba032.whopeo (mgr.24625) 11768 : cluster [DBG] pgmap v6706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:09 gibba011 bash[30791]: cluster 2021-03-10T17:36:08.310837+0000 mgr.gibba032.whopeo (mgr.24625) 11768 : cluster [DBG] pgmap v6706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:10.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:09 gibba011 bash[31774]: cluster 2021-03-10T17:36:08.310837+0000 mgr.gibba032.whopeo (mgr.24625) 11768 : cluster [DBG] pgmap v6706: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:10.460 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:36:10.690 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:36:10.690 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:36:10.690 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:36:10.691 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:36:10.691 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:36:10.691 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:36:10.691 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:36:10.691 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:36:10.692 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:36:10.692 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:36:10.692 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:36:10.692 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:36:10.692 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:36:10.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:36:10.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:36:10.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:36:10.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:36:10.693 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:36:10.694 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:36:10.694 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:36:10.694 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:36:10.937 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:36:10.937 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:36:10.937 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:36:10.937 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:10.938 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:36:10.938 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:36:10.938 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:10.938 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:36:10.939 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:36:10.939 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:10.939 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:36:10.939 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:36:10.939 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:36:10.940 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:36:10.940 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:36:12.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:11 gibba032 bash[24004]: cluster 2021-03-10T17:36:10.312392+0000 mgr.gibba032.whopeo (mgr.24625) 11769 : cluster [DBG] pgmap v6707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:12.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:11 gibba032 bash[24004]: audit 2021-03-10T17:36:10.447502+0000 mgr.gibba032.whopeo (mgr.24625) 11770 : audit [DBG] from='client.51939 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:11 gibba032 bash[24004]: audit 2021-03-10T17:36:10.674282+0000 mgr.gibba032.whopeo (mgr.24625) 11771 : audit [DBG] from='client.61927 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:11 gibba032 bash[24004]: audit 2021-03-10T17:36:10.935156+0000 mon.a (mon.0) 3868 : audit [DBG] from='client.? 172.21.2.111:0/1277006904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:12.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[30791]: cluster 2021-03-10T17:36:10.312392+0000 mgr.gibba032.whopeo (mgr.24625) 11769 : cluster [DBG] pgmap v6707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:12.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[30791]: audit 2021-03-10T17:36:10.447502+0000 mgr.gibba032.whopeo (mgr.24625) 11770 : audit [DBG] from='client.51939 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[30791]: audit 2021-03-10T17:36:10.674282+0000 mgr.gibba032.whopeo (mgr.24625) 11771 : audit [DBG] from='client.61927 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[30791]: audit 2021-03-10T17:36:10.935156+0000 mon.a (mon.0) 3868 : audit [DBG] from='client.? 172.21.2.111:0/1277006904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:12.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[31774]: cluster 2021-03-10T17:36:10.312392+0000 mgr.gibba032.whopeo (mgr.24625) 11769 : cluster [DBG] pgmap v6707: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:12.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[31774]: audit 2021-03-10T17:36:10.447502+0000 mgr.gibba032.whopeo (mgr.24625) 11770 : audit [DBG] from='client.51939 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[31774]: audit 2021-03-10T17:36:10.674282+0000 mgr.gibba032.whopeo (mgr.24625) 11771 : audit [DBG] from='client.61927 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:12.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:11 gibba011 bash[31774]: audit 2021-03-10T17:36:10.935156+0000 mon.a (mon.0) 3868 : audit [DBG] from='client.? 172.21.2.111:0/1277006904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:14.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:13 gibba032 bash[24004]: cluster 2021-03-10T17:36:12.313896+0000 mgr.gibba032.whopeo (mgr.24625) 11772 : cluster [DBG] pgmap v6708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:13 gibba011 bash[30791]: cluster 2021-03-10T17:36:12.313896+0000 mgr.gibba032.whopeo (mgr.24625) 11772 : cluster [DBG] pgmap v6708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:14.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:13 gibba011 bash[31774]: cluster 2021-03-10T17:36:12.313896+0000 mgr.gibba032.whopeo (mgr.24625) 11772 : cluster [DBG] pgmap v6708: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:16.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:15 gibba032 bash[24004]: cluster 2021-03-10T17:36:14.315243+0000 mgr.gibba032.whopeo (mgr.24625) 11773 : cluster [DBG] pgmap v6709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:16.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:15 gibba011 bash[30791]: cluster 2021-03-10T17:36:14.315243+0000 mgr.gibba032.whopeo (mgr.24625) 11773 : cluster [DBG] pgmap v6709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:16.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:15 gibba011 bash[31774]: cluster 2021-03-10T17:36:14.315243+0000 mgr.gibba032.whopeo (mgr.24625) 11773 : cluster [DBG] pgmap v6709: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:18.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:17 gibba032 bash[24004]: cluster 2021-03-10T17:36:16.316767+0000 mgr.gibba032.whopeo (mgr.24625) 11774 : cluster [DBG] pgmap v6710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:17 gibba011 bash[30791]: cluster 2021-03-10T17:36:16.316767+0000 mgr.gibba032.whopeo (mgr.24625) 11774 : cluster [DBG] pgmap v6710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:18.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:17 gibba011 bash[31774]: cluster 2021-03-10T17:36:16.316767+0000 mgr.gibba032.whopeo (mgr.24625) 11774 : cluster [DBG] pgmap v6710: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:20.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:19 gibba032 bash[24004]: cluster 2021-03-10T17:36:18.318301+0000 mgr.gibba032.whopeo (mgr.24625) 11775 : cluster [DBG] pgmap v6711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:19 gibba011 bash[30791]: cluster 2021-03-10T17:36:18.318301+0000 mgr.gibba032.whopeo (mgr.24625) 11775 : cluster [DBG] pgmap v6711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:20.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:19 gibba011 bash[31774]: cluster 2021-03-10T17:36:18.318301+0000 mgr.gibba032.whopeo (mgr.24625) 11775 : cluster [DBG] pgmap v6711: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:22.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:21 gibba032 bash[24004]: cluster 2021-03-10T17:36:20.319858+0000 mgr.gibba032.whopeo (mgr.24625) 11776 : cluster [DBG] pgmap v6712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:22.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:21 gibba011 bash[30791]: cluster 2021-03-10T17:36:20.319858+0000 mgr.gibba032.whopeo (mgr.24625) 11776 : cluster [DBG] pgmap v6712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:22.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:21 gibba011 bash[31774]: cluster 2021-03-10T17:36:20.319858+0000 mgr.gibba032.whopeo (mgr.24625) 11776 : cluster [DBG] pgmap v6712: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:23 gibba032 bash[24004]: cephadm 2021-03-10T17:36:22.305172+0000 mgr.gibba032.whopeo (mgr.24625) 11777 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:23 gibba032 bash[24004]: cluster 2021-03-10T17:36:22.321373+0000 mgr.gibba032.whopeo (mgr.24625) 11778 : cluster [DBG] pgmap v6713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:23 gibba011 bash[30791]: cephadm 2021-03-10T17:36:22.305172+0000 mgr.gibba032.whopeo (mgr.24625) 11777 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:23 gibba011 bash[30791]: cluster 2021-03-10T17:36:22.321373+0000 mgr.gibba032.whopeo (mgr.24625) 11778 : cluster [DBG] pgmap v6713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:23 gibba011 bash[31774]: cephadm 2021-03-10T17:36:22.305172+0000 mgr.gibba032.whopeo (mgr.24625) 11777 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:23 gibba011 bash[31774]: cluster 2021-03-10T17:36:22.321373+0000 mgr.gibba032.whopeo (mgr.24625) 11778 : cluster [DBG] pgmap v6713: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:26.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:25 gibba032 bash[24004]: cluster 2021-03-10T17:36:24.322610+0000 mgr.gibba032.whopeo (mgr.24625) 11779 : cluster [DBG] pgmap v6714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:26.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:25 gibba011 bash[30791]: cluster 2021-03-10T17:36:24.322610+0000 mgr.gibba032.whopeo (mgr.24625) 11779 : cluster [DBG] pgmap v6714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:26.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:25 gibba011 bash[31774]: cluster 2021-03-10T17:36:24.322610+0000 mgr.gibba032.whopeo (mgr.24625) 11779 : cluster [DBG] pgmap v6714: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:28.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:27 gibba032 bash[24004]: cluster 2021-03-10T17:36:26.324175+0000 mgr.gibba032.whopeo (mgr.24625) 11780 : cluster [DBG] pgmap v6715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:28.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:27 gibba011 bash[30791]: cluster 2021-03-10T17:36:26.324175+0000 mgr.gibba032.whopeo (mgr.24625) 11780 : cluster [DBG] pgmap v6715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:28.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:27 gibba011 bash[31774]: cluster 2021-03-10T17:36:26.324175+0000 mgr.gibba032.whopeo (mgr.24625) 11780 : cluster [DBG] pgmap v6715: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:30.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:29 gibba032 bash[24004]: cluster 2021-03-10T17:36:28.325912+0000 mgr.gibba032.whopeo (mgr.24625) 11781 : cluster [DBG] pgmap v6716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:29 gibba011 bash[30791]: cluster 2021-03-10T17:36:28.325912+0000 mgr.gibba032.whopeo (mgr.24625) 11781 : cluster [DBG] pgmap v6716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:30.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:29 gibba011 bash[31774]: cluster 2021-03-10T17:36:28.325912+0000 mgr.gibba032.whopeo (mgr.24625) 11781 : cluster [DBG] pgmap v6716: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:32.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:31 gibba032 bash[24004]: cluster 2021-03-10T17:36:30.327551+0000 mgr.gibba032.whopeo (mgr.24625) 11782 : cluster [DBG] pgmap v6717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:31 gibba011 bash[30791]: cluster 2021-03-10T17:36:30.327551+0000 mgr.gibba032.whopeo (mgr.24625) 11782 : cluster [DBG] pgmap v6717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:31 gibba011 bash[31774]: cluster 2021-03-10T17:36:30.327551+0000 mgr.gibba032.whopeo (mgr.24625) 11782 : cluster [DBG] pgmap v6717: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:34.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:33 gibba032 bash[24004]: cluster 2021-03-10T17:36:32.329184+0000 mgr.gibba032.whopeo (mgr.24625) 11783 : cluster [DBG] pgmap v6718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:34.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:33 gibba011 bash[30791]: cluster 2021-03-10T17:36:32.329184+0000 mgr.gibba032.whopeo (mgr.24625) 11783 : cluster [DBG] pgmap v6718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:34.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:33 gibba011 bash[31774]: cluster 2021-03-10T17:36:32.329184+0000 mgr.gibba032.whopeo (mgr.24625) 11783 : cluster [DBG] pgmap v6718: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:36.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:35 gibba032 bash[24004]: cluster 2021-03-10T17:36:34.330657+0000 mgr.gibba032.whopeo (mgr.24625) 11784 : cluster [DBG] pgmap v6719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:36.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:35 gibba011 bash[30791]: cluster 2021-03-10T17:36:34.330657+0000 mgr.gibba032.whopeo (mgr.24625) 11784 : cluster [DBG] pgmap v6719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:36.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:35 gibba011 bash[31774]: cluster 2021-03-10T17:36:34.330657+0000 mgr.gibba032.whopeo (mgr.24625) 11784 : cluster [DBG] pgmap v6719: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:37 gibba032 bash[24004]: cluster 2021-03-10T17:36:36.332304+0000 mgr.gibba032.whopeo (mgr.24625) 11785 : cluster [DBG] pgmap v6720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:37 gibba011 bash[30791]: cluster 2021-03-10T17:36:36.332304+0000 mgr.gibba032.whopeo (mgr.24625) 11785 : cluster [DBG] pgmap v6720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:38.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:37 gibba011 bash[31774]: cluster 2021-03-10T17:36:36.332304+0000 mgr.gibba032.whopeo (mgr.24625) 11785 : cluster [DBG] pgmap v6720: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:39.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:38 gibba032 bash[24004]: cephadm 2021-03-10T17:36:37.320913+0000 mgr.gibba032.whopeo (mgr.24625) 11786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:38 gibba011 bash[30791]: cephadm 2021-03-10T17:36:37.320913+0000 mgr.gibba032.whopeo (mgr.24625) 11786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:39.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:38 gibba011 bash[31774]: cephadm 2021-03-10T17:36:37.320913+0000 mgr.gibba032.whopeo (mgr.24625) 11786 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:40.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:39 gibba032 bash[24004]: cluster 2021-03-10T17:36:38.334083+0000 mgr.gibba032.whopeo (mgr.24625) 11787 : cluster [DBG] pgmap v6721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:39 gibba011 bash[30791]: cluster 2021-03-10T17:36:38.334083+0000 mgr.gibba032.whopeo (mgr.24625) 11787 : cluster [DBG] pgmap v6721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:40.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:39 gibba011 bash[31774]: cluster 2021-03-10T17:36:38.334083+0000 mgr.gibba032.whopeo (mgr.24625) 11787 : cluster [DBG] pgmap v6721: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:41.177 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:36:41.433 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:36:41.433 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:36:41.433 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:36:41.434 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:36:41.434 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:36:41.434 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:36:41.434 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:36:41.435 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:36:41.435 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:36:41.435 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:36:41.435 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:36:41.435 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:36:41.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:36:41.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:36:41.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:36:41.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:36:41.436 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:36:41.437 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:36:41.437 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:36:41.437 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:36:41.437 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:36:41.686 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:36:41.687 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:36:41.687 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:36:41.687 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:41.687 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:36:41.688 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:36:41.688 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:41.688 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:36:41.688 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:36:41.688 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:36:41.689 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:36:41.689 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:36:41.689 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:36:41.689 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:36:41.689 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:36:42.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:41 gibba032 bash[24004]: cluster 2021-03-10T17:36:40.335537+0000 mgr.gibba032.whopeo (mgr.24625) 11788 : cluster [DBG] pgmap v6722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:42.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:41 gibba032 bash[24004]: audit 2021-03-10T17:36:41.165319+0000 mgr.gibba032.whopeo (mgr.24625) 11789 : audit [DBG] from='client.51957 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:42.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:41 gibba032 bash[24004]: audit 2021-03-10T17:36:41.685247+0000 mon.c (mon.1) 228 : audit [DBG] from='client.? 172.21.2.111:0/2864299936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:42.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[30791]: cluster 2021-03-10T17:36:40.335537+0000 mgr.gibba032.whopeo (mgr.24625) 11788 : cluster [DBG] pgmap v6722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[30791]: audit 2021-03-10T17:36:41.165319+0000 mgr.gibba032.whopeo (mgr.24625) 11789 : audit [DBG] from='client.51957 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[30791]: audit 2021-03-10T17:36:41.685247+0000 mon.c (mon.1) 228 : audit [DBG] from='client.? 172.21.2.111:0/2864299936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:42.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[31774]: cluster 2021-03-10T17:36:40.335537+0000 mgr.gibba032.whopeo (mgr.24625) 11788 : cluster [DBG] pgmap v6722: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[31774]: audit 2021-03-10T17:36:41.165319+0000 mgr.gibba032.whopeo (mgr.24625) 11789 : audit [DBG] from='client.51957 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:41 gibba011 bash[31774]: audit 2021-03-10T17:36:41.685247+0000 mon.c (mon.1) 228 : audit [DBG] from='client.? 172.21.2.111:0/2864299936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:43.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:42 gibba032 bash[24004]: audit 2021-03-10T17:36:41.390055+0000 mgr.gibba032.whopeo (mgr.24625) 11790 : audit [DBG] from='client.61945 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:43.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:42 gibba011 bash[30791]: audit 2021-03-10T17:36:41.390055+0000 mgr.gibba032.whopeo (mgr.24625) 11790 : audit [DBG] from='client.61945 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:43.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:42 gibba011 bash[31774]: audit 2021-03-10T17:36:41.390055+0000 mgr.gibba032.whopeo (mgr.24625) 11790 : audit [DBG] from='client.61945 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:36:44.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:43 gibba011 bash[30791]: cluster 2021-03-10T17:36:42.337156+0000 mgr.gibba032.whopeo (mgr.24625) 11791 : cluster [DBG] pgmap v6723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:44.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:43 gibba011 bash[31774]: cluster 2021-03-10T17:36:42.337156+0000 mgr.gibba032.whopeo (mgr.24625) 11791 : cluster [DBG] pgmap v6723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:43 gibba032 bash[24004]: cluster 2021-03-10T17:36:42.337156+0000 mgr.gibba032.whopeo (mgr.24625) 11791 : cluster [DBG] pgmap v6723: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:45 gibba011 bash[30791]: cluster 2021-03-10T17:36:44.338578+0000 mgr.gibba032.whopeo (mgr.24625) 11792 : cluster [DBG] pgmap v6724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:46.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:45 gibba011 bash[31774]: cluster 2021-03-10T17:36:44.338578+0000 mgr.gibba032.whopeo (mgr.24625) 11792 : cluster [DBG] pgmap v6724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:46.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:45 gibba032 bash[24004]: cluster 2021-03-10T17:36:44.338578+0000 mgr.gibba032.whopeo (mgr.24625) 11792 : cluster [DBG] pgmap v6724: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:47 gibba011 bash[30791]: cluster 2021-03-10T17:36:46.340204+0000 mgr.gibba032.whopeo (mgr.24625) 11793 : cluster [DBG] pgmap v6725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:47 gibba011 bash[31774]: cluster 2021-03-10T17:36:46.340204+0000 mgr.gibba032.whopeo (mgr.24625) 11793 : cluster [DBG] pgmap v6725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:48.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:47 gibba032 bash[24004]: cluster 2021-03-10T17:36:46.340204+0000 mgr.gibba032.whopeo (mgr.24625) 11793 : cluster [DBG] pgmap v6725: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:50.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:49 gibba011 bash[30791]: cluster 2021-03-10T17:36:48.341798+0000 mgr.gibba032.whopeo (mgr.24625) 11794 : cluster [DBG] pgmap v6726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:49 gibba011 bash[31774]: cluster 2021-03-10T17:36:48.341798+0000 mgr.gibba032.whopeo (mgr.24625) 11794 : cluster [DBG] pgmap v6726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:50.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:49 gibba032 bash[24004]: cluster 2021-03-10T17:36:48.341798+0000 mgr.gibba032.whopeo (mgr.24625) 11794 : cluster [DBG] pgmap v6726: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:52.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:51 gibba011 bash[30791]: cluster 2021-03-10T17:36:50.343252+0000 mgr.gibba032.whopeo (mgr.24625) 11795 : cluster [DBG] pgmap v6727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:52.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:51 gibba011 bash[31774]: cluster 2021-03-10T17:36:50.343252+0000 mgr.gibba032.whopeo (mgr.24625) 11795 : cluster [DBG] pgmap v6727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:51 gibba032 bash[24004]: cluster 2021-03-10T17:36:50.343252+0000 mgr.gibba032.whopeo (mgr.24625) 11795 : cluster [DBG] pgmap v6727: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:54.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.337711+0000 mgr.gibba032.whopeo (mgr.24625) 11796 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:36:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.339320+0000 mgr.gibba032.whopeo (mgr.24625) 11797 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:36:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cluster 2021-03-10T17:36:52.344670+0000 mgr.gibba032.whopeo (mgr.24625) 11798 : cluster [DBG] pgmap v6728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.588224+0000 mon.a (mon.0) 3869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:36:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.589114+0000 mgr.gibba032.whopeo (mgr.24625) 11799 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:36:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.590897+0000 mgr.gibba032.whopeo (mgr.24625) 11800 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:36:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.591499+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:36:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.592208+0000 mgr.gibba032.whopeo (mgr.24625) 11801 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:36:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.592246+0000 mgr.gibba032.whopeo (mgr.24625) 11802 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:36:54.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.592951+0000 mon.a (mon.0) 3871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.593624+0000 mgr.gibba032.whopeo (mgr.24625) 11803 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:36:54.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.593660+0000 mgr.gibba032.whopeo (mgr.24625) 11804 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:36:54.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.594371+0000 mon.a (mon.0) 3872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.595023+0000 mgr.gibba032.whopeo (mgr.24625) 11805 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:36:54.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.595719+0000 mon.a (mon.0) 3873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.596226+0000 mgr.gibba032.whopeo (mgr.24625) 11806 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:36:54.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.596263+0000 mgr.gibba032.whopeo (mgr.24625) 11807 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:36:54.013 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.597847+0000 mgr.gibba032.whopeo (mgr.24625) 11808 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:36:54.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.598160+0000 mgr.gibba032.whopeo (mgr.24625) 11809 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:36:54.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.598522+0000 mon.a (mon.0) 3874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.599582+0000 mgr.gibba032.whopeo (mgr.24625) 11810 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:36:54.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.599619+0000 mgr.gibba032.whopeo (mgr.24625) 11811 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:36:54.014 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: cephadm 2021-03-10T17:36:52.599721+0000 mgr.gibba032.whopeo (mgr.24625) 11812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:54.015 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:53 gibba032 bash[24004]: audit 2021-03-10T17:36:52.600225+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.337711+0000 mgr.gibba032.whopeo (mgr.24625) 11796 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:36:54.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.339320+0000 mgr.gibba032.whopeo (mgr.24625) 11797 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:36:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cluster 2021-03-10T17:36:52.344670+0000 mgr.gibba032.whopeo (mgr.24625) 11798 : cluster [DBG] pgmap v6728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.588224+0000 mon.a (mon.0) 3869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:36:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.589114+0000 mgr.gibba032.whopeo (mgr.24625) 11799 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:36:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.590897+0000 mgr.gibba032.whopeo (mgr.24625) 11800 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:36:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.591499+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:36:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.592208+0000 mgr.gibba032.whopeo (mgr.24625) 11801 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:36:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.592246+0000 mgr.gibba032.whopeo (mgr.24625) 11802 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:36:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.592951+0000 mon.a (mon.0) 3871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.593624+0000 mgr.gibba032.whopeo (mgr.24625) 11803 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:36:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.593660+0000 mgr.gibba032.whopeo (mgr.24625) 11804 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:36:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.594371+0000 mon.a (mon.0) 3872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.595023+0000 mgr.gibba032.whopeo (mgr.24625) 11805 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:36:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.595719+0000 mon.a (mon.0) 3873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.596226+0000 mgr.gibba032.whopeo (mgr.24625) 11806 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:36:54.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.596263+0000 mgr.gibba032.whopeo (mgr.24625) 11807 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:36:54.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.597847+0000 mgr.gibba032.whopeo (mgr.24625) 11808 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:36:54.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.598160+0000 mgr.gibba032.whopeo (mgr.24625) 11809 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:36:54.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.598522+0000 mon.a (mon.0) 3874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.599582+0000 mgr.gibba032.whopeo (mgr.24625) 11810 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:36:54.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.337711+0000 mgr.gibba032.whopeo (mgr.24625) 11796 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:36:54.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.339320+0000 mgr.gibba032.whopeo (mgr.24625) 11797 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:36:54.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cluster 2021-03-10T17:36:52.344670+0000 mgr.gibba032.whopeo (mgr.24625) 11798 : cluster [DBG] pgmap v6728: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:54.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.588224+0000 mon.a (mon.0) 3869 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:36:54.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.589114+0000 mgr.gibba032.whopeo (mgr.24625) 11799 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:36:54.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.590897+0000 mgr.gibba032.whopeo (mgr.24625) 11800 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:36:54.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.591499+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:36:54.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.592208+0000 mgr.gibba032.whopeo (mgr.24625) 11801 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:36:54.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.592246+0000 mgr.gibba032.whopeo (mgr.24625) 11802 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:36:54.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.592951+0000 mon.a (mon.0) 3871 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.593624+0000 mgr.gibba032.whopeo (mgr.24625) 11803 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:36:54.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.593660+0000 mgr.gibba032.whopeo (mgr.24625) 11804 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:36:54.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.594371+0000 mon.a (mon.0) 3872 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.595023+0000 mgr.gibba032.whopeo (mgr.24625) 11805 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:36:54.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.595719+0000 mon.a (mon.0) 3873 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.596226+0000 mgr.gibba032.whopeo (mgr.24625) 11806 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:36:54.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.596263+0000 mgr.gibba032.whopeo (mgr.24625) 11807 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:36:54.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.597847+0000 mgr.gibba032.whopeo (mgr.24625) 11808 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:36:54.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.598160+0000 mgr.gibba032.whopeo (mgr.24625) 11809 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:36:54.089 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.598522+0000 mon.a (mon.0) 3874 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.090 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.599582+0000 mgr.gibba032.whopeo (mgr.24625) 11810 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:36:54.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.599619+0000 mgr.gibba032.whopeo (mgr.24625) 11811 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:36:54.090 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: cephadm 2021-03-10T17:36:52.599721+0000 mgr.gibba032.whopeo (mgr.24625) 11812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:54.091 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[30791]: audit 2021-03-10T17:36:52.600225+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:54.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.599619+0000 mgr.gibba032.whopeo (mgr.24625) 11811 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:36:54.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: cephadm 2021-03-10T17:36:52.599721+0000 mgr.gibba032.whopeo (mgr.24625) 11812 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:36:54.091 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:53 gibba011 bash[31774]: audit 2021-03-10T17:36:52.600225+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:36:56.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:55 gibba032 bash[24004]: cluster 2021-03-10T17:36:54.346163+0000 mgr.gibba032.whopeo (mgr.24625) 11813 : cluster [DBG] pgmap v6729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:55 gibba011 bash[30791]: cluster 2021-03-10T17:36:54.346163+0000 mgr.gibba032.whopeo (mgr.24625) 11813 : cluster [DBG] pgmap v6729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:55 gibba011 bash[31774]: cluster 2021-03-10T17:36:54.346163+0000 mgr.gibba032.whopeo (mgr.24625) 11813 : cluster [DBG] pgmap v6729: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:57 gibba032 bash[24004]: cluster 2021-03-10T17:36:56.347644+0000 mgr.gibba032.whopeo (mgr.24625) 11814 : cluster [DBG] pgmap v6730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:57 gibba032 bash[24004]: audit 2021-03-10T17:36:57.561827+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:36:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:57 gibba032 bash[24004]: audit 2021-03-10T17:36:57.562491+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:36:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[30791]: cluster 2021-03-10T17:36:56.347644+0000 mgr.gibba032.whopeo (mgr.24625) 11814 : cluster [DBG] pgmap v6730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[30791]: audit 2021-03-10T17:36:57.561827+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:36:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[30791]: audit 2021-03-10T17:36:57.562491+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:36:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[31774]: cluster 2021-03-10T17:36:56.347644+0000 mgr.gibba032.whopeo (mgr.24625) 11814 : cluster [DBG] pgmap v6730: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:36:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[31774]: audit 2021-03-10T17:36:57.561827+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:36:58.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:57 gibba011 bash[31774]: audit 2021-03-10T17:36:57.562491+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:36:59.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:58 gibba032 bash[24004]: audit 2021-03-10T17:36:57.773412+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:36:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:58 gibba032 bash[24004]: audit 2021-03-10T17:36:57.774176+0000 mon.a (mon.0) 3879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:58 gibba032 bash[24004]: audit 2021-03-10T17:36:57.774838+0000 mon.a (mon.0) 3880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:58 gibba032 bash[24004]: audit 2021-03-10T17:36:57.775564+0000 mon.a (mon.0) 3881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:36:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[30791]: audit 2021-03-10T17:36:57.773412+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:36:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[30791]: audit 2021-03-10T17:36:57.774176+0000 mon.a (mon.0) 3879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[30791]: audit 2021-03-10T17:36:57.774838+0000 mon.a (mon.0) 3880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[30791]: audit 2021-03-10T17:36:57.775564+0000 mon.a (mon.0) 3881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:36:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[31774]: audit 2021-03-10T17:36:57.773412+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2021-03-10T17:36:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[31774]: audit 2021-03-10T17:36:57.774176+0000 mon.a (mon.0) 3879 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[31774]: audit 2021-03-10T17:36:57.774838+0000 mon.a (mon.0) 3880 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2021-03-10T17:36:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:58 gibba011 bash[31774]: audit 2021-03-10T17:36:57.775564+0000 mon.a (mon.0) 3881 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2021-03-10T17:37:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:36:59 gibba032 bash[24004]: cluster 2021-03-10T17:36:58.349412+0000 mgr.gibba032.whopeo (mgr.24625) 11815 : cluster [DBG] pgmap v6731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:00.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:36:59 gibba011 bash[30791]: cluster 2021-03-10T17:36:58.349412+0000 mgr.gibba032.whopeo (mgr.24625) 11815 : cluster [DBG] pgmap v6731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:00.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:36:59 gibba011 bash[31774]: cluster 2021-03-10T17:36:58.349412+0000 mgr.gibba032.whopeo (mgr.24625) 11815 : cluster [DBG] pgmap v6731: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:02.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:01 gibba032 bash[24004]: cluster 2021-03-10T17:37:00.350982+0000 mgr.gibba032.whopeo (mgr.24625) 11816 : cluster [DBG] pgmap v6732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:02.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:01 gibba011 bash[31774]: cluster 2021-03-10T17:37:00.350982+0000 mgr.gibba032.whopeo (mgr.24625) 11816 : cluster [DBG] pgmap v6732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:02.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:01 gibba011 bash[30791]: cluster 2021-03-10T17:37:00.350982+0000 mgr.gibba032.whopeo (mgr.24625) 11816 : cluster [DBG] pgmap v6732: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:04.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:03 gibba032 bash[24004]: cluster 2021-03-10T17:37:02.352722+0000 mgr.gibba032.whopeo (mgr.24625) 11817 : cluster [DBG] pgmap v6733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:04.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:03 gibba011 bash[30791]: cluster 2021-03-10T17:37:02.352722+0000 mgr.gibba032.whopeo (mgr.24625) 11817 : cluster [DBG] pgmap v6733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:04.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:03 gibba011 bash[31774]: cluster 2021-03-10T17:37:02.352722+0000 mgr.gibba032.whopeo (mgr.24625) 11817 : cluster [DBG] pgmap v6733: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:06.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:05 gibba032 bash[24004]: cluster 2021-03-10T17:37:04.354028+0000 mgr.gibba032.whopeo (mgr.24625) 11818 : cluster [DBG] pgmap v6734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:06.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:05 gibba011 bash[30791]: cluster 2021-03-10T17:37:04.354028+0000 mgr.gibba032.whopeo (mgr.24625) 11818 : cluster [DBG] pgmap v6734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:06.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:05 gibba011 bash[31774]: cluster 2021-03-10T17:37:04.354028+0000 mgr.gibba032.whopeo (mgr.24625) 11818 : cluster [DBG] pgmap v6734: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:08.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:07 gibba032 bash[24004]: cluster 2021-03-10T17:37:06.355493+0000 mgr.gibba032.whopeo (mgr.24625) 11819 : cluster [DBG] pgmap v6735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:07 gibba011 bash[30791]: cluster 2021-03-10T17:37:06.355493+0000 mgr.gibba032.whopeo (mgr.24625) 11819 : cluster [DBG] pgmap v6735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:07 gibba011 bash[31774]: cluster 2021-03-10T17:37:06.355493+0000 mgr.gibba032.whopeo (mgr.24625) 11819 : cluster [DBG] pgmap v6735: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:09.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:08 gibba032 bash[24004]: cephadm 2021-03-10T17:37:07.615693+0000 mgr.gibba032.whopeo (mgr.24625) 11820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:09.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:08 gibba011 bash[30791]: cephadm 2021-03-10T17:37:07.615693+0000 mgr.gibba032.whopeo (mgr.24625) 11820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:09.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:08 gibba011 bash[31774]: cephadm 2021-03-10T17:37:07.615693+0000 mgr.gibba032.whopeo (mgr.24625) 11820 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:10.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:09 gibba032 bash[24004]: cluster 2021-03-10T17:37:08.357246+0000 mgr.gibba032.whopeo (mgr.24625) 11821 : cluster [DBG] pgmap v6736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:10.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:09 gibba011 bash[30791]: cluster 2021-03-10T17:37:08.357246+0000 mgr.gibba032.whopeo (mgr.24625) 11821 : cluster [DBG] pgmap v6736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:09 gibba011 bash[31774]: cluster 2021-03-10T17:37:08.357246+0000 mgr.gibba032.whopeo (mgr.24625) 11821 : cluster [DBG] pgmap v6736: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:11.923 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:37:12.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:11 gibba032 bash[24004]: cluster 2021-03-10T17:37:10.358542+0000 mgr.gibba032.whopeo (mgr.24625) 11822 : cluster [DBG] pgmap v6737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:12.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:11 gibba011 bash[30791]: cluster 2021-03-10T17:37:10.358542+0000 mgr.gibba032.whopeo (mgr.24625) 11822 : cluster [DBG] pgmap v6737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:12.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:11 gibba011 bash[31774]: cluster 2021-03-10T17:37:10.358542+0000 mgr.gibba032.whopeo (mgr.24625) 11822 : cluster [DBG] pgmap v6737: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:12.150 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:37:12.150 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:37:12.150 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:37:12.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:37:12.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:37:12.151 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:37:12.151 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:37:12.151 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:37:12.152 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:37:12.152 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:37:12.152 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:37:12.152 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:37:12.152 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:37:12.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:37:12.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:37:12.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:37:12.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:37:12.153 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:37:12.154 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:37:12.154 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:37:12.154 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:37:12.397 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:37:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:37:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:37:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:37:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:37:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:37:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:37:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:37:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:37:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:37:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:37:12.400 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:37:13.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:12 gibba032 bash[24004]: audit 2021-03-10T17:37:11.911293+0000 mgr.gibba032.whopeo (mgr.24625) 11823 : audit [DBG] from='client.51975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:12 gibba032 bash[24004]: audit 2021-03-10T17:37:12.134796+0000 mgr.gibba032.whopeo (mgr.24625) 11824 : audit [DBG] from='client.61963 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:12 gibba032 bash[24004]: audit 2021-03-10T17:37:12.396039+0000 mon.a (mon.0) 3882 : audit [DBG] from='client.? 172.21.2.111:0/4196649144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[30791]: audit 2021-03-10T17:37:11.911293+0000 mgr.gibba032.whopeo (mgr.24625) 11823 : audit [DBG] from='client.51975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[30791]: audit 2021-03-10T17:37:12.134796+0000 mgr.gibba032.whopeo (mgr.24625) 11824 : audit [DBG] from='client.61963 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[30791]: audit 2021-03-10T17:37:12.396039+0000 mon.a (mon.0) 3882 : audit [DBG] from='client.? 172.21.2.111:0/4196649144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[31774]: audit 2021-03-10T17:37:11.911293+0000 mgr.gibba032.whopeo (mgr.24625) 11823 : audit [DBG] from='client.51975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[31774]: audit 2021-03-10T17:37:12.134796+0000 mgr.gibba032.whopeo (mgr.24625) 11824 : audit [DBG] from='client.61963 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:13.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:12 gibba011 bash[31774]: audit 2021-03-10T17:37:12.396039+0000 mon.a (mon.0) 3882 : audit [DBG] from='client.? 172.21.2.111:0/4196649144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:14.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:13 gibba032 bash[24004]: cluster 2021-03-10T17:37:12.359900+0000 mgr.gibba032.whopeo (mgr.24625) 11825 : cluster [DBG] pgmap v6738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:13 gibba011 bash[30791]: cluster 2021-03-10T17:37:12.359900+0000 mgr.gibba032.whopeo (mgr.24625) 11825 : cluster [DBG] pgmap v6738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:14.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:13 gibba011 bash[31774]: cluster 2021-03-10T17:37:12.359900+0000 mgr.gibba032.whopeo (mgr.24625) 11825 : cluster [DBG] pgmap v6738: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:16.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:15 gibba032 bash[24004]: cluster 2021-03-10T17:37:14.361173+0000 mgr.gibba032.whopeo (mgr.24625) 11826 : cluster [DBG] pgmap v6739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:16.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:15 gibba011 bash[30791]: cluster 2021-03-10T17:37:14.361173+0000 mgr.gibba032.whopeo (mgr.24625) 11826 : cluster [DBG] pgmap v6739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:16.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:15 gibba011 bash[31774]: cluster 2021-03-10T17:37:14.361173+0000 mgr.gibba032.whopeo (mgr.24625) 11826 : cluster [DBG] pgmap v6739: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:18.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:17 gibba032 bash[24004]: cluster 2021-03-10T17:37:16.362667+0000 mgr.gibba032.whopeo (mgr.24625) 11827 : cluster [DBG] pgmap v6740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:18.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:17 gibba011 bash[31774]: cluster 2021-03-10T17:37:16.362667+0000 mgr.gibba032.whopeo (mgr.24625) 11827 : cluster [DBG] pgmap v6740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:17 gibba011 bash[30791]: cluster 2021-03-10T17:37:16.362667+0000 mgr.gibba032.whopeo (mgr.24625) 11827 : cluster [DBG] pgmap v6740: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:20.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:19 gibba032 bash[24004]: cluster 2021-03-10T17:37:18.364466+0000 mgr.gibba032.whopeo (mgr.24625) 11828 : cluster [DBG] pgmap v6741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:20.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:19 gibba011 bash[30791]: cluster 2021-03-10T17:37:18.364466+0000 mgr.gibba032.whopeo (mgr.24625) 11828 : cluster [DBG] pgmap v6741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:20.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:19 gibba011 bash[31774]: cluster 2021-03-10T17:37:18.364466+0000 mgr.gibba032.whopeo (mgr.24625) 11828 : cluster [DBG] pgmap v6741: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:22.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:21 gibba032 bash[24004]: cluster 2021-03-10T17:37:20.365907+0000 mgr.gibba032.whopeo (mgr.24625) 11829 : cluster [DBG] pgmap v6742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:22.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:21 gibba011 bash[30791]: cluster 2021-03-10T17:37:20.365907+0000 mgr.gibba032.whopeo (mgr.24625) 11829 : cluster [DBG] pgmap v6742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:22.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:21 gibba011 bash[31774]: cluster 2021-03-10T17:37:20.365907+0000 mgr.gibba032.whopeo (mgr.24625) 11829 : cluster [DBG] pgmap v6742: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:23 gibba032 bash[24004]: cluster 2021-03-10T17:37:22.367457+0000 mgr.gibba032.whopeo (mgr.24625) 11830 : cluster [DBG] pgmap v6743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:23 gibba032 bash[24004]: cephadm 2021-03-10T17:37:22.631708+0000 mgr.gibba032.whopeo (mgr.24625) 11831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:24.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:23 gibba011 bash[30791]: cluster 2021-03-10T17:37:22.367457+0000 mgr.gibba032.whopeo (mgr.24625) 11830 : cluster [DBG] pgmap v6743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:24.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:23 gibba011 bash[30791]: cephadm 2021-03-10T17:37:22.631708+0000 mgr.gibba032.whopeo (mgr.24625) 11831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:23 gibba011 bash[31774]: cluster 2021-03-10T17:37:22.367457+0000 mgr.gibba032.whopeo (mgr.24625) 11830 : cluster [DBG] pgmap v6743: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:24.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:23 gibba011 bash[31774]: cephadm 2021-03-10T17:37:22.631708+0000 mgr.gibba032.whopeo (mgr.24625) 11831 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:26.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:25 gibba011 bash[30791]: cluster 2021-03-10T17:37:24.368948+0000 mgr.gibba032.whopeo (mgr.24625) 11832 : cluster [DBG] pgmap v6744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:26.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:25 gibba011 bash[31774]: cluster 2021-03-10T17:37:24.368948+0000 mgr.gibba032.whopeo (mgr.24625) 11832 : cluster [DBG] pgmap v6744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:26.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:25 gibba032 bash[24004]: cluster 2021-03-10T17:37:24.368948+0000 mgr.gibba032.whopeo (mgr.24625) 11832 : cluster [DBG] pgmap v6744: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:27 gibba011 bash[30791]: cluster 2021-03-10T17:37:26.370642+0000 mgr.gibba032.whopeo (mgr.24625) 11833 : cluster [DBG] pgmap v6745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:27 gibba011 bash[31774]: cluster 2021-03-10T17:37:26.370642+0000 mgr.gibba032.whopeo (mgr.24625) 11833 : cluster [DBG] pgmap v6745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:27 gibba032 bash[24004]: cluster 2021-03-10T17:37:26.370642+0000 mgr.gibba032.whopeo (mgr.24625) 11833 : cluster [DBG] pgmap v6745: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:30.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:29 gibba011 bash[30791]: cluster 2021-03-10T17:37:28.372442+0000 mgr.gibba032.whopeo (mgr.24625) 11834 : cluster [DBG] pgmap v6746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:30.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:29 gibba011 bash[31774]: cluster 2021-03-10T17:37:28.372442+0000 mgr.gibba032.whopeo (mgr.24625) 11834 : cluster [DBG] pgmap v6746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:30.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:29 gibba032 bash[24004]: cluster 2021-03-10T17:37:28.372442+0000 mgr.gibba032.whopeo (mgr.24625) 11834 : cluster [DBG] pgmap v6746: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:32.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:31 gibba011 bash[30791]: cluster 2021-03-10T17:37:30.373815+0000 mgr.gibba032.whopeo (mgr.24625) 11835 : cluster [DBG] pgmap v6747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:32.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:31 gibba011 bash[31774]: cluster 2021-03-10T17:37:30.373815+0000 mgr.gibba032.whopeo (mgr.24625) 11835 : cluster [DBG] pgmap v6747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:31 gibba032 bash[24004]: cluster 2021-03-10T17:37:30.373815+0000 mgr.gibba032.whopeo (mgr.24625) 11835 : cluster [DBG] pgmap v6747: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:34.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:33 gibba011 bash[30791]: cluster 2021-03-10T17:37:32.375457+0000 mgr.gibba032.whopeo (mgr.24625) 11836 : cluster [DBG] pgmap v6748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:34.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:33 gibba011 bash[31774]: cluster 2021-03-10T17:37:32.375457+0000 mgr.gibba032.whopeo (mgr.24625) 11836 : cluster [DBG] pgmap v6748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:33 gibba032 bash[24004]: cluster 2021-03-10T17:37:32.375457+0000 mgr.gibba032.whopeo (mgr.24625) 11836 : cluster [DBG] pgmap v6748: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:36.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:35 gibba011 bash[30791]: cluster 2021-03-10T17:37:34.376931+0000 mgr.gibba032.whopeo (mgr.24625) 11837 : cluster [DBG] pgmap v6749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:36.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:35 gibba011 bash[31774]: cluster 2021-03-10T17:37:34.376931+0000 mgr.gibba032.whopeo (mgr.24625) 11837 : cluster [DBG] pgmap v6749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:36.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:35 gibba032 bash[24004]: cluster 2021-03-10T17:37:34.376931+0000 mgr.gibba032.whopeo (mgr.24625) 11837 : cluster [DBG] pgmap v6749: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:37 gibba032 bash[24004]: cluster 2021-03-10T17:37:36.378629+0000 mgr.gibba032.whopeo (mgr.24625) 11838 : cluster [DBG] pgmap v6750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:37 gibba011 bash[30791]: cluster 2021-03-10T17:37:36.378629+0000 mgr.gibba032.whopeo (mgr.24625) 11838 : cluster [DBG] pgmap v6750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:37 gibba011 bash[31774]: cluster 2021-03-10T17:37:36.378629+0000 mgr.gibba032.whopeo (mgr.24625) 11838 : cluster [DBG] pgmap v6750: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:39.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:38 gibba011 bash[30791]: cephadm 2021-03-10T17:37:37.647813+0000 mgr.gibba032.whopeo (mgr.24625) 11839 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:39.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:38 gibba011 bash[31774]: cephadm 2021-03-10T17:37:37.647813+0000 mgr.gibba032.whopeo (mgr.24625) 11839 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:39.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:38 gibba032 bash[24004]: cephadm 2021-03-10T17:37:37.647813+0000 mgr.gibba032.whopeo (mgr.24625) 11839 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:39 gibba011 bash[30791]: cluster 2021-03-10T17:37:38.380427+0000 mgr.gibba032.whopeo (mgr.24625) 11840 : cluster [DBG] pgmap v6751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:40.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:39 gibba011 bash[31774]: cluster 2021-03-10T17:37:38.380427+0000 mgr.gibba032.whopeo (mgr.24625) 11840 : cluster [DBG] pgmap v6751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:40.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:39 gibba032 bash[24004]: cluster 2021-03-10T17:37:38.380427+0000 mgr.gibba032.whopeo (mgr.24625) 11840 : cluster [DBG] pgmap v6751: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:41 gibba032 bash[24004]: cluster 2021-03-10T17:37:40.381939+0000 mgr.gibba032.whopeo (mgr.24625) 11841 : cluster [DBG] pgmap v6752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:41 gibba011 bash[30791]: cluster 2021-03-10T17:37:40.381939+0000 mgr.gibba032.whopeo (mgr.24625) 11841 : cluster [DBG] pgmap v6752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:41 gibba011 bash[31774]: cluster 2021-03-10T17:37:40.381939+0000 mgr.gibba032.whopeo (mgr.24625) 11841 : cluster [DBG] pgmap v6752: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:42.639 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:37:42.860 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:37:42.860 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 5m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:37:42.861 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 4m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:37:42.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:37:42.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:37:42.861 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:37:42.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:37:42.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:37:42.862 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:37:42.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:37:42.862 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:37:42.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:37:42.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:37:42.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:37:42.863 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:37:42.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:37:42.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:37:42.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:37:42.864 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 4m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:37:42.865 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 4m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:37:42.865 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 5m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:37:43.112 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:37:43.113 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:37:43.113 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:37:43.113 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:43.113 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:37:43.114 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:37:43.114 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:43.114 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:37:43.114 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:37:43.114 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:37:43.115 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:37:43.115 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:37:43.115 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:37:43.115 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:37:43.116 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:37:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:43 gibba032 bash[24004]: cluster 2021-03-10T17:37:42.383611+0000 mgr.gibba032.whopeo (mgr.24625) 11842 : cluster [DBG] pgmap v6753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:43 gibba032 bash[24004]: audit 2021-03-10T17:37:42.626395+0000 mgr.gibba032.whopeo (mgr.24625) 11843 : audit [DBG] from='client.61975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:43 gibba032 bash[24004]: audit 2021-03-10T17:37:42.848778+0000 mgr.gibba032.whopeo (mgr.24625) 11844 : audit [DBG] from='client.51999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:43 gibba032 bash[24004]: audit 2021-03-10T17:37:43.110960+0000 mon.c (mon.1) 229 : audit [DBG] from='client.? 172.21.2.111:0/957845237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[30791]: cluster 2021-03-10T17:37:42.383611+0000 mgr.gibba032.whopeo (mgr.24625) 11842 : cluster [DBG] pgmap v6753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[30791]: audit 2021-03-10T17:37:42.626395+0000 mgr.gibba032.whopeo (mgr.24625) 11843 : audit [DBG] from='client.61975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[30791]: audit 2021-03-10T17:37:42.848778+0000 mgr.gibba032.whopeo (mgr.24625) 11844 : audit [DBG] from='client.51999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[30791]: audit 2021-03-10T17:37:43.110960+0000 mon.c (mon.1) 229 : audit [DBG] from='client.? 172.21.2.111:0/957845237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[31774]: cluster 2021-03-10T17:37:42.383611+0000 mgr.gibba032.whopeo (mgr.24625) 11842 : cluster [DBG] pgmap v6753: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[31774]: audit 2021-03-10T17:37:42.626395+0000 mgr.gibba032.whopeo (mgr.24625) 11843 : audit [DBG] from='client.61975 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[31774]: audit 2021-03-10T17:37:42.848778+0000 mgr.gibba032.whopeo (mgr.24625) 11844 : audit [DBG] from='client.51999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:37:44.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:43 gibba011 bash[31774]: audit 2021-03-10T17:37:43.110960+0000 mon.c (mon.1) 229 : audit [DBG] from='client.? 172.21.2.111:0/957845237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:46.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:45 gibba032 bash[24004]: cluster 2021-03-10T17:37:44.385109+0000 mgr.gibba032.whopeo (mgr.24625) 11845 : cluster [DBG] pgmap v6754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:45 gibba011 bash[30791]: cluster 2021-03-10T17:37:44.385109+0000 mgr.gibba032.whopeo (mgr.24625) 11845 : cluster [DBG] pgmap v6754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:45 gibba011 bash[31774]: cluster 2021-03-10T17:37:44.385109+0000 mgr.gibba032.whopeo (mgr.24625) 11845 : cluster [DBG] pgmap v6754: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:48.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:47 gibba032 bash[24004]: cluster 2021-03-10T17:37:46.386935+0000 mgr.gibba032.whopeo (mgr.24625) 11846 : cluster [DBG] pgmap v6755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:48.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:47 gibba011 bash[30791]: cluster 2021-03-10T17:37:46.386935+0000 mgr.gibba032.whopeo (mgr.24625) 11846 : cluster [DBG] pgmap v6755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:48.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:47 gibba011 bash[31774]: cluster 2021-03-10T17:37:46.386935+0000 mgr.gibba032.whopeo (mgr.24625) 11846 : cluster [DBG] pgmap v6755: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:49 gibba032 bash[24004]: cluster 2021-03-10T17:37:48.388609+0000 mgr.gibba032.whopeo (mgr.24625) 11847 : cluster [DBG] pgmap v6756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:50.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:49 gibba011 bash[30791]: cluster 2021-03-10T17:37:48.388609+0000 mgr.gibba032.whopeo (mgr.24625) 11847 : cluster [DBG] pgmap v6756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:49 gibba011 bash[31774]: cluster 2021-03-10T17:37:48.388609+0000 mgr.gibba032.whopeo (mgr.24625) 11847 : cluster [DBG] pgmap v6756: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:51 gibba032 bash[24004]: cluster 2021-03-10T17:37:50.389956+0000 mgr.gibba032.whopeo (mgr.24625) 11848 : cluster [DBG] pgmap v6757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:51 gibba011 bash[30791]: cluster 2021-03-10T17:37:50.389956+0000 mgr.gibba032.whopeo (mgr.24625) 11848 : cluster [DBG] pgmap v6757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:51 gibba011 bash[31774]: cluster 2021-03-10T17:37:50.389956+0000 mgr.gibba032.whopeo (mgr.24625) 11848 : cluster [DBG] pgmap v6757: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cluster 2021-03-10T17:37:52.391567+0000 mgr.gibba032.whopeo (mgr.24625) 11849 : cluster [DBG] pgmap v6758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.654024+0000 mgr.gibba032.whopeo (mgr.24625) 11850 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:37:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.655563+0000 mgr.gibba032.whopeo (mgr.24625) 11851 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:37:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.883062+0000 mon.a (mon.0) 3883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.883777+0000 mgr.gibba032.whopeo (mgr.24625) 11852 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.885741+0000 mgr.gibba032.whopeo (mgr.24625) 11853 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.886252+0000 mon.a (mon.0) 3884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:37:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.886934+0000 mgr.gibba032.whopeo (mgr.24625) 11854 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.886973+0000 mgr.gibba032.whopeo (mgr.24625) 11855 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.887771+0000 mon.a (mon.0) 3885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.888223+0000 mgr.gibba032.whopeo (mgr.24625) 11856 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.888261+0000 mgr.gibba032.whopeo (mgr.24625) 11857 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:37:54.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.888991+0000 mon.a (mon.0) 3886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.889331+0000 mgr.gibba032.whopeo (mgr.24625) 11858 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.890211+0000 mon.a (mon.0) 3887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.890489+0000 mgr.gibba032.whopeo (mgr.24625) 11859 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.890527+0000 mgr.gibba032.whopeo (mgr.24625) 11860 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:37:54.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.891911+0000 mgr.gibba032.whopeo (mgr.24625) 11861 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.892298+0000 mgr.gibba032.whopeo (mgr.24625) 11862 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.892732+0000 mon.a (mon.0) 3888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.893525+0000 mgr.gibba032.whopeo (mgr.24625) 11863 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:37:54.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.893564+0000 mgr.gibba032.whopeo (mgr.24625) 11864 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: cephadm 2021-03-10T17:37:52.893669+0000 mgr.gibba032.whopeo (mgr.24625) 11865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:54.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:53 gibba032 bash[24004]: audit 2021-03-10T17:37:52.894352+0000 mon.a (mon.0) 3889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cluster 2021-03-10T17:37:52.391567+0000 mgr.gibba032.whopeo (mgr.24625) 11849 : cluster [DBG] pgmap v6758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.654024+0000 mgr.gibba032.whopeo (mgr.24625) 11850 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.655563+0000 mgr.gibba032.whopeo (mgr.24625) 11851 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:37:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.883062+0000 mon.a (mon.0) 3883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.883777+0000 mgr.gibba032.whopeo (mgr.24625) 11852 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:37:54.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.885741+0000 mgr.gibba032.whopeo (mgr.24625) 11853 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.886252+0000 mon.a (mon.0) 3884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:37:54.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.886934+0000 mgr.gibba032.whopeo (mgr.24625) 11854 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:37:54.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.886973+0000 mgr.gibba032.whopeo (mgr.24625) 11855 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:37:54.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.887771+0000 mon.a (mon.0) 3885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.888223+0000 mgr.gibba032.whopeo (mgr.24625) 11856 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:37:54.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.888261+0000 mgr.gibba032.whopeo (mgr.24625) 11857 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:37:54.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.888991+0000 mon.a (mon.0) 3886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.889331+0000 mgr.gibba032.whopeo (mgr.24625) 11858 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:37:54.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.890211+0000 mon.a (mon.0) 3887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.337 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.890489+0000 mgr.gibba032.whopeo (mgr.24625) 11859 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:37:54.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.890527+0000 mgr.gibba032.whopeo (mgr.24625) 11860 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:37:54.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.891911+0000 mgr.gibba032.whopeo (mgr.24625) 11861 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:37:54.338 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.892298+0000 mgr.gibba032.whopeo (mgr.24625) 11862 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:37:54.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.892732+0000 mon.a (mon.0) 3888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.339 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.893525+0000 mgr.gibba032.whopeo (mgr.24625) 11863 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:37:54.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cluster 2021-03-10T17:37:52.391567+0000 mgr.gibba032.whopeo (mgr.24625) 11849 : cluster [DBG] pgmap v6758: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:54.345 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.654024+0000 mgr.gibba032.whopeo (mgr.24625) 11850 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:37:54.346 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.655563+0000 mgr.gibba032.whopeo (mgr.24625) 11851 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:37:54.348 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.883062+0000 mon.a (mon.0) 3883 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:37:54.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.883777+0000 mgr.gibba032.whopeo (mgr.24625) 11852 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:37:54.349 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.885741+0000 mgr.gibba032.whopeo (mgr.24625) 11853 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:37:54.362 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.886252+0000 mon.a (mon.0) 3884 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:37:54.363 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.886934+0000 mgr.gibba032.whopeo (mgr.24625) 11854 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:37:54.363 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.886973+0000 mgr.gibba032.whopeo (mgr.24625) 11855 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:37:54.363 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.887771+0000 mon.a (mon.0) 3885 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.364 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.888223+0000 mgr.gibba032.whopeo (mgr.24625) 11856 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:37:54.364 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.888261+0000 mgr.gibba032.whopeo (mgr.24625) 11857 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:37:54.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.888991+0000 mon.a (mon.0) 3886 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.889331+0000 mgr.gibba032.whopeo (mgr.24625) 11858 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:37:54.365 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.890211+0000 mon.a (mon.0) 3887 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.379 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.890489+0000 mgr.gibba032.whopeo (mgr.24625) 11859 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:37:54.379 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.890527+0000 mgr.gibba032.whopeo (mgr.24625) 11860 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:37:54.380 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.891911+0000 mgr.gibba032.whopeo (mgr.24625) 11861 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:37:54.380 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.892298+0000 mgr.gibba032.whopeo (mgr.24625) 11862 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:37:54.381 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.892732+0000 mon.a (mon.0) 3888 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.381 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.893525+0000 mgr.gibba032.whopeo (mgr.24625) 11863 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:37:54.382 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.893564+0000 mgr.gibba032.whopeo (mgr.24625) 11864 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:37:54.383 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: cephadm 2021-03-10T17:37:52.893669+0000 mgr.gibba032.whopeo (mgr.24625) 11865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:54.383 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[30791]: audit 2021-03-10T17:37:52.894352+0000 mon.a (mon.0) 3889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:54.383 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.893564+0000 mgr.gibba032.whopeo (mgr.24625) 11864 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:37:54.384 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: cephadm 2021-03-10T17:37:52.893669+0000 mgr.gibba032.whopeo (mgr.24625) 11865 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:37:54.384 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:53 gibba011 bash[31774]: audit 2021-03-10T17:37:52.894352+0000 mon.a (mon.0) 3889 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:37:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:55 gibba032 bash[24004]: cluster 2021-03-10T17:37:54.393043+0000 mgr.gibba032.whopeo (mgr.24625) 11866 : cluster [DBG] pgmap v6759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:55 gibba011 bash[30791]: cluster 2021-03-10T17:37:54.393043+0000 mgr.gibba032.whopeo (mgr.24625) 11866 : cluster [DBG] pgmap v6759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:56.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:55 gibba011 bash[31774]: cluster 2021-03-10T17:37:54.393043+0000 mgr.gibba032.whopeo (mgr.24625) 11866 : cluster [DBG] pgmap v6759: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: cluster 2021-03-10T17:37:56.394656+0000 mgr.gibba032.whopeo (mgr.24625) 11867 : cluster [DBG] pgmap v6760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.568563+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:37:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.569168+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:37:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.782826+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:37:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.783606+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.784263+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:57 gibba032 bash[24004]: audit 2021-03-10T17:37:57.784935+0000 mon.a (mon.0) 3895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:37:58.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: cluster 2021-03-10T17:37:56.394656+0000 mgr.gibba032.whopeo (mgr.24625) 11867 : cluster [DBG] pgmap v6760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.568563+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:37:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.569168+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:37:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.782826+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:37:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.783606+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.784263+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[30791]: audit 2021-03-10T17:37:57.784935+0000 mon.a (mon.0) 3895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:37:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: cluster 2021-03-10T17:37:56.394656+0000 mgr.gibba032.whopeo (mgr.24625) 11867 : cluster [DBG] pgmap v6760: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:37:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.568563+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:37:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.569168+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:37:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.782826+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2021-03-10T17:37:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.783606+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.784263+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:37:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:57 gibba011 bash[31774]: audit 2021-03-10T17:37:57.784935+0000 mon.a (mon.0) 3895 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:38:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:37:59 gibba032 bash[24004]: cluster 2021-03-10T17:37:58.396335+0000 mgr.gibba032.whopeo (mgr.24625) 11868 : cluster [DBG] pgmap v6761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:37:59 gibba011 bash[30791]: cluster 2021-03-10T17:37:58.396335+0000 mgr.gibba032.whopeo (mgr.24625) 11868 : cluster [DBG] pgmap v6761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:37:59 gibba011 bash[31774]: cluster 2021-03-10T17:37:58.396335+0000 mgr.gibba032.whopeo (mgr.24625) 11868 : cluster [DBG] pgmap v6761: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:01 gibba032 bash[24004]: cluster 2021-03-10T17:38:00.397747+0000 mgr.gibba032.whopeo (mgr.24625) 11869 : cluster [DBG] pgmap v6762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:01 gibba011 bash[30791]: cluster 2021-03-10T17:38:00.397747+0000 mgr.gibba032.whopeo (mgr.24625) 11869 : cluster [DBG] pgmap v6762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:01 gibba011 bash[31774]: cluster 2021-03-10T17:38:00.397747+0000 mgr.gibba032.whopeo (mgr.24625) 11869 : cluster [DBG] pgmap v6762: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:04.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:03 gibba032 bash[24004]: cluster 2021-03-10T17:38:02.399261+0000 mgr.gibba032.whopeo (mgr.24625) 11870 : cluster [DBG] pgmap v6763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:04.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:03 gibba011 bash[30791]: cluster 2021-03-10T17:38:02.399261+0000 mgr.gibba032.whopeo (mgr.24625) 11870 : cluster [DBG] pgmap v6763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:03 gibba011 bash[31774]: cluster 2021-03-10T17:38:02.399261+0000 mgr.gibba032.whopeo (mgr.24625) 11870 : cluster [DBG] pgmap v6763: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:06.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:05 gibba032 bash[24004]: cluster 2021-03-10T17:38:04.400820+0000 mgr.gibba032.whopeo (mgr.24625) 11871 : cluster [DBG] pgmap v6764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:05 gibba011 bash[30791]: cluster 2021-03-10T17:38:04.400820+0000 mgr.gibba032.whopeo (mgr.24625) 11871 : cluster [DBG] pgmap v6764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:06.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:05 gibba011 bash[31774]: cluster 2021-03-10T17:38:04.400820+0000 mgr.gibba032.whopeo (mgr.24625) 11871 : cluster [DBG] pgmap v6764: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:08.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:07 gibba032 bash[24004]: cluster 2021-03-10T17:38:06.402554+0000 mgr.gibba032.whopeo (mgr.24625) 11872 : cluster [DBG] pgmap v6765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:07 gibba011 bash[30791]: cluster 2021-03-10T17:38:06.402554+0000 mgr.gibba032.whopeo (mgr.24625) 11872 : cluster [DBG] pgmap v6765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:07 gibba011 bash[31774]: cluster 2021-03-10T17:38:06.402554+0000 mgr.gibba032.whopeo (mgr.24625) 11872 : cluster [DBG] pgmap v6765: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:09.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:08 gibba032 bash[24004]: cephadm 2021-03-10T17:38:07.908804+0000 mgr.gibba032.whopeo (mgr.24625) 11873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:09.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:08 gibba011 bash[30791]: cephadm 2021-03-10T17:38:07.908804+0000 mgr.gibba032.whopeo (mgr.24625) 11873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:09.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:08 gibba011 bash[31774]: cephadm 2021-03-10T17:38:07.908804+0000 mgr.gibba032.whopeo (mgr.24625) 11873 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:09 gibba032 bash[24004]: cluster 2021-03-10T17:38:08.404190+0000 mgr.gibba032.whopeo (mgr.24625) 11874 : cluster [DBG] pgmap v6766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:10.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:09 gibba011 bash[30791]: cluster 2021-03-10T17:38:08.404190+0000 mgr.gibba032.whopeo (mgr.24625) 11874 : cluster [DBG] pgmap v6766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:09 gibba011 bash[31774]: cluster 2021-03-10T17:38:08.404190+0000 mgr.gibba032.whopeo (mgr.24625) 11874 : cluster [DBG] pgmap v6766: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:11 gibba032 bash[24004]: cluster 2021-03-10T17:38:10.405516+0000 mgr.gibba032.whopeo (mgr.24625) 11875 : cluster [DBG] pgmap v6767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:12.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:11 gibba011 bash[30791]: cluster 2021-03-10T17:38:10.405516+0000 mgr.gibba032.whopeo (mgr.24625) 11875 : cluster [DBG] pgmap v6767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:11 gibba011 bash[31774]: cluster 2021-03-10T17:38:10.405516+0000 mgr.gibba032.whopeo (mgr.24625) 11875 : cluster [DBG] pgmap v6767: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:13.361 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:38:13.588 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:38:13.589 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:38:13.589 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:38:13.589 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:38:13.589 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:38:13.590 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:38:13.590 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:38:13.590 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:38:13.590 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:38:13.591 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:38:13.591 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:38:13.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:38:13.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:38:13.591 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:38:13.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:38:13.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:38:13.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:38:13.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:38:13.592 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:38:13.593 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:38:13.593 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:38:13.839 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:38:13.839 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:38:13.840 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:38:13.840 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:13.840 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:38:13.840 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:38:13.841 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:13.841 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:38:13.841 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:38:13.841 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:13.841 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:38:13.842 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:38:13.842 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:38:13.842 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:38:13.842 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:38:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:13 gibba032 bash[24004]: cluster 2021-03-10T17:38:12.407041+0000 mgr.gibba032.whopeo (mgr.24625) 11876 : cluster [DBG] pgmap v6768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:13 gibba032 bash[24004]: audit 2021-03-10T17:38:13.837791+0000 mon.a (mon.0) 3896 : audit [DBG] from='client.? 172.21.2.111:0/3819008741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:13 gibba011 bash[30791]: cluster 2021-03-10T17:38:12.407041+0000 mgr.gibba032.whopeo (mgr.24625) 11876 : cluster [DBG] pgmap v6768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:13 gibba011 bash[30791]: audit 2021-03-10T17:38:13.837791+0000 mon.a (mon.0) 3896 : audit [DBG] from='client.? 172.21.2.111:0/3819008741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:13 gibba011 bash[31774]: cluster 2021-03-10T17:38:12.407041+0000 mgr.gibba032.whopeo (mgr.24625) 11876 : cluster [DBG] pgmap v6768: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:13 gibba011 bash[31774]: audit 2021-03-10T17:38:13.837791+0000 mon.a (mon.0) 3896 : audit [DBG] from='client.? 172.21.2.111:0/3819008741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:15.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:14 gibba032 bash[24004]: audit 2021-03-10T17:38:13.347296+0000 mgr.gibba032.whopeo (mgr.24625) 11877 : audit [DBG] from='client.52011 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:15.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:14 gibba032 bash[24004]: audit 2021-03-10T17:38:13.573925+0000 mgr.gibba032.whopeo (mgr.24625) 11878 : audit [DBG] from='client.61999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:15.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:14 gibba011 bash[30791]: audit 2021-03-10T17:38:13.347296+0000 mgr.gibba032.whopeo (mgr.24625) 11877 : audit [DBG] from='client.52011 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:15.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:14 gibba011 bash[30791]: audit 2021-03-10T17:38:13.573925+0000 mgr.gibba032.whopeo (mgr.24625) 11878 : audit [DBG] from='client.61999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:15.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:14 gibba011 bash[31774]: audit 2021-03-10T17:38:13.347296+0000 mgr.gibba032.whopeo (mgr.24625) 11877 : audit [DBG] from='client.52011 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:15.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:14 gibba011 bash[31774]: audit 2021-03-10T17:38:13.573925+0000 mgr.gibba032.whopeo (mgr.24625) 11878 : audit [DBG] from='client.61999 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:16.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:16 gibba032 bash[24004]: cluster 2021-03-10T17:38:14.408331+0000 mgr.gibba032.whopeo (mgr.24625) 11879 : cluster [DBG] pgmap v6769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:16 gibba011 bash[30791]: cluster 2021-03-10T17:38:14.408331+0000 mgr.gibba032.whopeo (mgr.24625) 11879 : cluster [DBG] pgmap v6769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:16 gibba011 bash[31774]: cluster 2021-03-10T17:38:14.408331+0000 mgr.gibba032.whopeo (mgr.24625) 11879 : cluster [DBG] pgmap v6769: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:18.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:18 gibba011 bash[30791]: cluster 2021-03-10T17:38:16.410099+0000 mgr.gibba032.whopeo (mgr.24625) 11880 : cluster [DBG] pgmap v6770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:18 gibba011 bash[31774]: cluster 2021-03-10T17:38:16.410099+0000 mgr.gibba032.whopeo (mgr.24625) 11880 : cluster [DBG] pgmap v6770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:18 gibba032 bash[24004]: cluster 2021-03-10T17:38:16.410099+0000 mgr.gibba032.whopeo (mgr.24625) 11880 : cluster [DBG] pgmap v6770: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:20 gibba011 bash[30791]: cluster 2021-03-10T17:38:18.411671+0000 mgr.gibba032.whopeo (mgr.24625) 11881 : cluster [DBG] pgmap v6771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:20 gibba011 bash[31774]: cluster 2021-03-10T17:38:18.411671+0000 mgr.gibba032.whopeo (mgr.24625) 11881 : cluster [DBG] pgmap v6771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:20.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:20 gibba032 bash[24004]: cluster 2021-03-10T17:38:18.411671+0000 mgr.gibba032.whopeo (mgr.24625) 11881 : cluster [DBG] pgmap v6771: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:22.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:22 gibba011 bash[30791]: cluster 2021-03-10T17:38:20.413189+0000 mgr.gibba032.whopeo (mgr.24625) 11882 : cluster [DBG] pgmap v6772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:22 gibba011 bash[31774]: cluster 2021-03-10T17:38:20.413189+0000 mgr.gibba032.whopeo (mgr.24625) 11882 : cluster [DBG] pgmap v6772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:22 gibba032 bash[24004]: cluster 2021-03-10T17:38:20.413189+0000 mgr.gibba032.whopeo (mgr.24625) 11882 : cluster [DBG] pgmap v6772: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:24 gibba011 bash[30791]: cluster 2021-03-10T17:38:22.414701+0000 mgr.gibba032.whopeo (mgr.24625) 11883 : cluster [DBG] pgmap v6773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:24 gibba011 bash[30791]: cephadm 2021-03-10T17:38:22.924699+0000 mgr.gibba032.whopeo (mgr.24625) 11884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:24 gibba011 bash[31774]: cluster 2021-03-10T17:38:22.414701+0000 mgr.gibba032.whopeo (mgr.24625) 11883 : cluster [DBG] pgmap v6773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:24.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:24 gibba011 bash[31774]: cephadm 2021-03-10T17:38:22.924699+0000 mgr.gibba032.whopeo (mgr.24625) 11884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:24 gibba032 bash[24004]: cluster 2021-03-10T17:38:22.414701+0000 mgr.gibba032.whopeo (mgr.24625) 11883 : cluster [DBG] pgmap v6773: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:24.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:24 gibba032 bash[24004]: cephadm 2021-03-10T17:38:22.924699+0000 mgr.gibba032.whopeo (mgr.24625) 11884 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:26.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:26 gibba011 bash[30791]: cluster 2021-03-10T17:38:24.416152+0000 mgr.gibba032.whopeo (mgr.24625) 11885 : cluster [DBG] pgmap v6774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:26 gibba011 bash[31774]: cluster 2021-03-10T17:38:24.416152+0000 mgr.gibba032.whopeo (mgr.24625) 11885 : cluster [DBG] pgmap v6774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:26 gibba032 bash[24004]: cluster 2021-03-10T17:38:24.416152+0000 mgr.gibba032.whopeo (mgr.24625) 11885 : cluster [DBG] pgmap v6774: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:28 gibba011 bash[30791]: cluster 2021-03-10T17:38:26.417758+0000 mgr.gibba032.whopeo (mgr.24625) 11886 : cluster [DBG] pgmap v6775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:28 gibba011 bash[31774]: cluster 2021-03-10T17:38:26.417758+0000 mgr.gibba032.whopeo (mgr.24625) 11886 : cluster [DBG] pgmap v6775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:28.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:28 gibba032 bash[24004]: cluster 2021-03-10T17:38:26.417758+0000 mgr.gibba032.whopeo (mgr.24625) 11886 : cluster [DBG] pgmap v6775: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:30 gibba011 bash[30791]: cluster 2021-03-10T17:38:28.419299+0000 mgr.gibba032.whopeo (mgr.24625) 11887 : cluster [DBG] pgmap v6776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:30 gibba011 bash[31774]: cluster 2021-03-10T17:38:28.419299+0000 mgr.gibba032.whopeo (mgr.24625) 11887 : cluster [DBG] pgmap v6776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:30 gibba032 bash[24004]: cluster 2021-03-10T17:38:28.419299+0000 mgr.gibba032.whopeo (mgr.24625) 11887 : cluster [DBG] pgmap v6776: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:32 gibba011 bash[30791]: cluster 2021-03-10T17:38:30.420951+0000 mgr.gibba032.whopeo (mgr.24625) 11888 : cluster [DBG] pgmap v6777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:32 gibba011 bash[31774]: cluster 2021-03-10T17:38:30.420951+0000 mgr.gibba032.whopeo (mgr.24625) 11888 : cluster [DBG] pgmap v6777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:32 gibba032 bash[24004]: cluster 2021-03-10T17:38:30.420951+0000 mgr.gibba032.whopeo (mgr.24625) 11888 : cluster [DBG] pgmap v6777: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:34 gibba011 bash[30791]: cluster 2021-03-10T17:38:32.422694+0000 mgr.gibba032.whopeo (mgr.24625) 11889 : cluster [DBG] pgmap v6778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:34 gibba011 bash[31774]: cluster 2021-03-10T17:38:32.422694+0000 mgr.gibba032.whopeo (mgr.24625) 11889 : cluster [DBG] pgmap v6778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:34.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:34 gibba032 bash[24004]: cluster 2021-03-10T17:38:32.422694+0000 mgr.gibba032.whopeo (mgr.24625) 11889 : cluster [DBG] pgmap v6778: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:36 gibba011 bash[30791]: cluster 2021-03-10T17:38:34.423983+0000 mgr.gibba032.whopeo (mgr.24625) 11890 : cluster [DBG] pgmap v6779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:36 gibba011 bash[31774]: cluster 2021-03-10T17:38:34.423983+0000 mgr.gibba032.whopeo (mgr.24625) 11890 : cluster [DBG] pgmap v6779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:36 gibba032 bash[24004]: cluster 2021-03-10T17:38:34.423983+0000 mgr.gibba032.whopeo (mgr.24625) 11890 : cluster [DBG] pgmap v6779: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:38.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:38 gibba032 bash[24004]: cluster 2021-03-10T17:38:36.425623+0000 mgr.gibba032.whopeo (mgr.24625) 11891 : cluster [DBG] pgmap v6780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:38 gibba011 bash[30791]: cluster 2021-03-10T17:38:36.425623+0000 mgr.gibba032.whopeo (mgr.24625) 11891 : cluster [DBG] pgmap v6780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:38 gibba011 bash[31774]: cluster 2021-03-10T17:38:36.425623+0000 mgr.gibba032.whopeo (mgr.24625) 11891 : cluster [DBG] pgmap v6780: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:39.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:39 gibba032 bash[24004]: cephadm 2021-03-10T17:38:37.940658+0000 mgr.gibba032.whopeo (mgr.24625) 11892 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:39.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:39 gibba011 bash[30791]: cephadm 2021-03-10T17:38:37.940658+0000 mgr.gibba032.whopeo (mgr.24625) 11892 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:39.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:39 gibba011 bash[31774]: cephadm 2021-03-10T17:38:37.940658+0000 mgr.gibba032.whopeo (mgr.24625) 11892 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:40.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:40 gibba032 bash[24004]: cluster 2021-03-10T17:38:38.427088+0000 mgr.gibba032.whopeo (mgr.24625) 11893 : cluster [DBG] pgmap v6781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:40 gibba011 bash[30791]: cluster 2021-03-10T17:38:38.427088+0000 mgr.gibba032.whopeo (mgr.24625) 11893 : cluster [DBG] pgmap v6781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:40 gibba011 bash[31774]: cluster 2021-03-10T17:38:38.427088+0000 mgr.gibba032.whopeo (mgr.24625) 11893 : cluster [DBG] pgmap v6781: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:42 gibba032 bash[24004]: cluster 2021-03-10T17:38:40.428646+0000 mgr.gibba032.whopeo (mgr.24625) 11894 : cluster [DBG] pgmap v6782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:42 gibba011 bash[30791]: cluster 2021-03-10T17:38:40.428646+0000 mgr.gibba032.whopeo (mgr.24625) 11894 : cluster [DBG] pgmap v6782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:42 gibba011 bash[31774]: cluster 2021-03-10T17:38:40.428646+0000 mgr.gibba032.whopeo (mgr.24625) 11894 : cluster [DBG] pgmap v6782: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:44.081 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:38:44.312 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:38:44.312 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 6m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:38:44.312 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 5m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:38:44.313 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:38:44.313 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:38:44.313 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:38:44.313 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:38:44.313 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:38:44.314 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:38:44.314 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:38:44.314 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 5m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:38:44.314 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:38:44.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:38:44.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:38:44.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:38:44.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:38:44.315 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:38:44.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:38:44.316 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 5m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:38:44.316 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 5m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:38:44.316 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 6m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:38:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:44 gibba032 bash[24004]: cluster 2021-03-10T17:38:42.430262+0000 mgr.gibba032.whopeo (mgr.24625) 11895 : cluster [DBG] pgmap v6783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:44.558 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:38:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:38:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:38:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:38:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:38:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:38:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:38:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:38:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:38:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:38:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:38:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:38:44.562 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:38:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:44 gibba011 bash[30791]: cluster 2021-03-10T17:38:42.430262+0000 mgr.gibba032.whopeo (mgr.24625) 11895 : cluster [DBG] pgmap v6783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:44 gibba011 bash[31774]: cluster 2021-03-10T17:38:42.430262+0000 mgr.gibba032.whopeo (mgr.24625) 11895 : cluster [DBG] pgmap v6783: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:45.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:45 gibba032 bash[24004]: audit 2021-03-10T17:38:44.069502+0000 mgr.gibba032.whopeo (mgr.24625) 11896 : audit [DBG] from='client.52029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:45 gibba032 bash[24004]: audit 2021-03-10T17:38:44.296836+0000 mgr.gibba032.whopeo (mgr.24625) 11897 : audit [DBG] from='client.62017 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:45 gibba032 bash[24004]: audit 2021-03-10T17:38:44.557013+0000 mon.c (mon.1) 230 : audit [DBG] from='client.? 172.21.2.111:0/934740895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[30791]: audit 2021-03-10T17:38:44.069502+0000 mgr.gibba032.whopeo (mgr.24625) 11896 : audit [DBG] from='client.52029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[30791]: audit 2021-03-10T17:38:44.296836+0000 mgr.gibba032.whopeo (mgr.24625) 11897 : audit [DBG] from='client.62017 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[30791]: audit 2021-03-10T17:38:44.557013+0000 mon.c (mon.1) 230 : audit [DBG] from='client.? 172.21.2.111:0/934740895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[31774]: audit 2021-03-10T17:38:44.069502+0000 mgr.gibba032.whopeo (mgr.24625) 11896 : audit [DBG] from='client.52029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[31774]: audit 2021-03-10T17:38:44.296836+0000 mgr.gibba032.whopeo (mgr.24625) 11897 : audit [DBG] from='client.62017 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:38:45.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:45 gibba011 bash[31774]: audit 2021-03-10T17:38:44.557013+0000 mon.c (mon.1) 230 : audit [DBG] from='client.? 172.21.2.111:0/934740895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:46.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:46 gibba032 bash[24004]: cluster 2021-03-10T17:38:44.431469+0000 mgr.gibba032.whopeo (mgr.24625) 11898 : cluster [DBG] pgmap v6784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:46 gibba011 bash[30791]: cluster 2021-03-10T17:38:44.431469+0000 mgr.gibba032.whopeo (mgr.24625) 11898 : cluster [DBG] pgmap v6784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:46 gibba011 bash[31774]: cluster 2021-03-10T17:38:44.431469+0000 mgr.gibba032.whopeo (mgr.24625) 11898 : cluster [DBG] pgmap v6784: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:48 gibba032 bash[24004]: cluster 2021-03-10T17:38:46.433140+0000 mgr.gibba032.whopeo (mgr.24625) 11899 : cluster [DBG] pgmap v6785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:48 gibba011 bash[30791]: cluster 2021-03-10T17:38:46.433140+0000 mgr.gibba032.whopeo (mgr.24625) 11899 : cluster [DBG] pgmap v6785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:48 gibba011 bash[31774]: cluster 2021-03-10T17:38:46.433140+0000 mgr.gibba032.whopeo (mgr.24625) 11899 : cluster [DBG] pgmap v6785: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:50 gibba032 bash[24004]: cluster 2021-03-10T17:38:48.434722+0000 mgr.gibba032.whopeo (mgr.24625) 11900 : cluster [DBG] pgmap v6786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:50.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:50 gibba011 bash[30791]: cluster 2021-03-10T17:38:48.434722+0000 mgr.gibba032.whopeo (mgr.24625) 11900 : cluster [DBG] pgmap v6786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:50 gibba011 bash[31774]: cluster 2021-03-10T17:38:48.434722+0000 mgr.gibba032.whopeo (mgr.24625) 11900 : cluster [DBG] pgmap v6786: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:52 gibba032 bash[24004]: cluster 2021-03-10T17:38:50.436389+0000 mgr.gibba032.whopeo (mgr.24625) 11901 : cluster [DBG] pgmap v6787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:52.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:52 gibba011 bash[30791]: cluster 2021-03-10T17:38:50.436389+0000 mgr.gibba032.whopeo (mgr.24625) 11901 : cluster [DBG] pgmap v6787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:52.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:52 gibba011 bash[31774]: cluster 2021-03-10T17:38:50.436389+0000 mgr.gibba032.whopeo (mgr.24625) 11901 : cluster [DBG] pgmap v6787: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: cluster 2021-03-10T17:38:52.438144+0000 mgr.gibba032.whopeo (mgr.24625) 11902 : cluster [DBG] pgmap v6788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.148786+0000 mgr.gibba032.whopeo (mgr.24625) 11903 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.149934+0000 mon.a (mon.0) 3897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.157040+0000 mgr.gibba032.whopeo (mgr.24625) 11904 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:38:54.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.158173+0000 mon.a (mon.0) 3898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.352294+0000 mon.a (mon.0) 3899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.355552+0000 mon.a (mon.0) 3900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.357096+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.358538+0000 mon.a (mon.0) 3902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.359882+0000 mon.a (mon.0) 3903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.362556+0000 mon.a (mon.0) 3904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:54 gibba032 bash[24004]: audit 2021-03-10T17:38:53.364080+0000 mon.a (mon.0) 3905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: cluster 2021-03-10T17:38:52.438144+0000 mgr.gibba032.whopeo (mgr.24625) 11902 : cluster [DBG] pgmap v6788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.148786+0000 mgr.gibba032.whopeo (mgr.24625) 11903 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:38:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.149934+0000 mon.a (mon.0) 3897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.157040+0000 mgr.gibba032.whopeo (mgr.24625) 11904 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:38:54.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.158173+0000 mon.a (mon.0) 3898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.352294+0000 mon.a (mon.0) 3899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.355552+0000 mon.a (mon.0) 3900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:38:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.357096+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.358538+0000 mon.a (mon.0) 3902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.359882+0000 mon.a (mon.0) 3903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.362556+0000 mon.a (mon.0) 3904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[30791]: audit 2021-03-10T17:38:53.364080+0000 mon.a (mon.0) 3905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: cluster 2021-03-10T17:38:52.438144+0000 mgr.gibba032.whopeo (mgr.24625) 11902 : cluster [DBG] pgmap v6788: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.148786+0000 mgr.gibba032.whopeo (mgr.24625) 11903 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:38:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.149934+0000 mon.a (mon.0) 3897 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.157040+0000 mgr.gibba032.whopeo (mgr.24625) 11904 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:38:54.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.158173+0000 mon.a (mon.0) 3898 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.352294+0000 mon.a (mon.0) 3899 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:38:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.355552+0000 mon.a (mon.0) 3900 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:38:54.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.357096+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.358538+0000 mon.a (mon.0) 3902 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.359882+0000 mon.a (mon.0) 3903 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.362556+0000 mon.a (mon.0) 3904 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:54.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:54 gibba011 bash[31774]: audit 2021-03-10T17:38:53.364080+0000 mon.a (mon.0) 3905 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:38:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.353171+0000 mgr.gibba032.whopeo (mgr.24625) 11905 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:38:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.355107+0000 mgr.gibba032.whopeo (mgr.24625) 11906 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:38:55.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.356429+0000 mgr.gibba032.whopeo (mgr.24625) 11907 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:38:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.356467+0000 mgr.gibba032.whopeo (mgr.24625) 11908 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:38:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.357854+0000 mgr.gibba032.whopeo (mgr.24625) 11909 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:38:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.357891+0000 mgr.gibba032.whopeo (mgr.24625) 11910 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:38:55.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.359096+0000 mgr.gibba032.whopeo (mgr.24625) 11911 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:38:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.360379+0000 mgr.gibba032.whopeo (mgr.24625) 11912 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:38:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.360415+0000 mgr.gibba032.whopeo (mgr.24625) 11913 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:38:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.361809+0000 mgr.gibba032.whopeo (mgr.24625) 11914 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:38:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.362116+0000 mgr.gibba032.whopeo (mgr.24625) 11915 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:38:55.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.363338+0000 mgr.gibba032.whopeo (mgr.24625) 11916 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:38:55.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.363373+0000 mgr.gibba032.whopeo (mgr.24625) 11917 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:38:55.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:55 gibba032 bash[24004]: cephadm 2021-03-10T17:38:53.363472+0000 mgr.gibba032.whopeo (mgr.24625) 11918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.353171+0000 mgr.gibba032.whopeo (mgr.24625) 11905 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:38:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.355107+0000 mgr.gibba032.whopeo (mgr.24625) 11906 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:38:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.356429+0000 mgr.gibba032.whopeo (mgr.24625) 11907 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:38:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.356467+0000 mgr.gibba032.whopeo (mgr.24625) 11908 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:38:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.357854+0000 mgr.gibba032.whopeo (mgr.24625) 11909 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:38:55.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.357891+0000 mgr.gibba032.whopeo (mgr.24625) 11910 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:38:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.359096+0000 mgr.gibba032.whopeo (mgr.24625) 11911 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:38:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.360379+0000 mgr.gibba032.whopeo (mgr.24625) 11912 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:38:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.360415+0000 mgr.gibba032.whopeo (mgr.24625) 11913 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:38:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.361809+0000 mgr.gibba032.whopeo (mgr.24625) 11914 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:38:55.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.362116+0000 mgr.gibba032.whopeo (mgr.24625) 11915 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:38:55.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.363338+0000 mgr.gibba032.whopeo (mgr.24625) 11916 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:38:55.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.363373+0000 mgr.gibba032.whopeo (mgr.24625) 11917 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:38:55.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[30791]: cephadm 2021-03-10T17:38:53.363472+0000 mgr.gibba032.whopeo (mgr.24625) 11918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:55.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.353171+0000 mgr.gibba032.whopeo (mgr.24625) 11905 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:38:55.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.355107+0000 mgr.gibba032.whopeo (mgr.24625) 11906 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:38:55.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.356429+0000 mgr.gibba032.whopeo (mgr.24625) 11907 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:38:55.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.356467+0000 mgr.gibba032.whopeo (mgr.24625) 11908 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:38:55.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.357854+0000 mgr.gibba032.whopeo (mgr.24625) 11909 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:38:55.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.357891+0000 mgr.gibba032.whopeo (mgr.24625) 11910 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:38:55.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.359096+0000 mgr.gibba032.whopeo (mgr.24625) 11911 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:38:55.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.360379+0000 mgr.gibba032.whopeo (mgr.24625) 11912 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:38:55.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.360415+0000 mgr.gibba032.whopeo (mgr.24625) 11913 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:38:55.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.361809+0000 mgr.gibba032.whopeo (mgr.24625) 11914 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:38:55.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.362116+0000 mgr.gibba032.whopeo (mgr.24625) 11915 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:38:55.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.363338+0000 mgr.gibba032.whopeo (mgr.24625) 11916 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:38:55.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.363373+0000 mgr.gibba032.whopeo (mgr.24625) 11917 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:38:55.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:55 gibba011 bash[31774]: cephadm 2021-03-10T17:38:53.363472+0000 mgr.gibba032.whopeo (mgr.24625) 11918 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:38:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:56 gibba032 bash[24004]: cluster 2021-03-10T17:38:54.439472+0000 mgr.gibba032.whopeo (mgr.24625) 11919 : cluster [DBG] pgmap v6789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:56 gibba011 bash[30791]: cluster 2021-03-10T17:38:54.439472+0000 mgr.gibba032.whopeo (mgr.24625) 11919 : cluster [DBG] pgmap v6789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:56.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:56 gibba011 bash[31774]: cluster 2021-03-10T17:38:54.439472+0000 mgr.gibba032.whopeo (mgr.24625) 11919 : cluster [DBG] pgmap v6789: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: cluster 2021-03-10T17:38:56.441126+0000 mgr.gibba032.whopeo (mgr.24625) 11920 : cluster [DBG] pgmap v6790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.575322+0000 mon.a (mon.0) 3906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:38:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.576100+0000 mon.a (mon.0) 3907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:38:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.792407+0000 mon.a (mon.0) 3908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:38:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.793274+0000 mon.a (mon.0) 3909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.794032+0000 mon.a (mon.0) 3910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:38:58 gibba032 bash[24004]: audit 2021-03-10T17:38:57.794721+0000 mon.a (mon.0) 3911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:38:58.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: cluster 2021-03-10T17:38:56.441126+0000 mgr.gibba032.whopeo (mgr.24625) 11920 : cluster [DBG] pgmap v6790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.575322+0000 mon.a (mon.0) 3906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.576100+0000 mon.a (mon.0) 3907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:38:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.792407+0000 mon.a (mon.0) 3908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.793274+0000 mon.a (mon.0) 3909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.794032+0000 mon.a (mon.0) 3910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[30791]: audit 2021-03-10T17:38:57.794721+0000 mon.a (mon.0) 3911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:38:58.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: cluster 2021-03-10T17:38:56.441126+0000 mgr.gibba032.whopeo (mgr.24625) 11920 : cluster [DBG] pgmap v6790: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:38:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.575322+0000 mon.a (mon.0) 3906 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:38:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.576100+0000 mon.a (mon.0) 3907 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:38:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.792407+0000 mon.a (mon.0) 3908 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:38:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.793274+0000 mon.a (mon.0) 3909 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.794032+0000 mon.a (mon.0) 3910 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2021-03-10T17:38:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:38:58 gibba011 bash[31774]: audit 2021-03-10T17:38:57.794721+0000 mon.a (mon.0) 3911 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:39:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:00 gibba032 bash[24004]: cluster 2021-03-10T17:38:58.442752+0000 mgr.gibba032.whopeo (mgr.24625) 11921 : cluster [DBG] pgmap v6791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:00 gibba011 bash[30791]: cluster 2021-03-10T17:38:58.442752+0000 mgr.gibba032.whopeo (mgr.24625) 11921 : cluster [DBG] pgmap v6791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:00 gibba011 bash[31774]: cluster 2021-03-10T17:38:58.442752+0000 mgr.gibba032.whopeo (mgr.24625) 11921 : cluster [DBG] pgmap v6791: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:02 gibba032 bash[24004]: cluster 2021-03-10T17:39:00.444436+0000 mgr.gibba032.whopeo (mgr.24625) 11922 : cluster [DBG] pgmap v6792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:02 gibba011 bash[30791]: cluster 2021-03-10T17:39:00.444436+0000 mgr.gibba032.whopeo (mgr.24625) 11922 : cluster [DBG] pgmap v6792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:02 gibba011 bash[31774]: cluster 2021-03-10T17:39:00.444436+0000 mgr.gibba032.whopeo (mgr.24625) 11922 : cluster [DBG] pgmap v6792: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:04 gibba032 bash[24004]: cluster 2021-03-10T17:39:02.446184+0000 mgr.gibba032.whopeo (mgr.24625) 11923 : cluster [DBG] pgmap v6793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:04.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:04 gibba011 bash[30791]: cluster 2021-03-10T17:39:02.446184+0000 mgr.gibba032.whopeo (mgr.24625) 11923 : cluster [DBG] pgmap v6793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:04 gibba011 bash[31774]: cluster 2021-03-10T17:39:02.446184+0000 mgr.gibba032.whopeo (mgr.24625) 11923 : cluster [DBG] pgmap v6793: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:06 gibba032 bash[24004]: cluster 2021-03-10T17:39:04.447546+0000 mgr.gibba032.whopeo (mgr.24625) 11924 : cluster [DBG] pgmap v6794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:06.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:06 gibba011 bash[30791]: cluster 2021-03-10T17:39:04.447546+0000 mgr.gibba032.whopeo (mgr.24625) 11924 : cluster [DBG] pgmap v6794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:06 gibba011 bash[31774]: cluster 2021-03-10T17:39:04.447546+0000 mgr.gibba032.whopeo (mgr.24625) 11924 : cluster [DBG] pgmap v6794: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:08 gibba032 bash[24004]: cluster 2021-03-10T17:39:06.449373+0000 mgr.gibba032.whopeo (mgr.24625) 11925 : cluster [DBG] pgmap v6795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:08 gibba011 bash[30791]: cluster 2021-03-10T17:39:06.449373+0000 mgr.gibba032.whopeo (mgr.24625) 11925 : cluster [DBG] pgmap v6795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:08 gibba011 bash[31774]: cluster 2021-03-10T17:39:06.449373+0000 mgr.gibba032.whopeo (mgr.24625) 11925 : cluster [DBG] pgmap v6795: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:10 gibba032 bash[24004]: cephadm 2021-03-10T17:39:08.378163+0000 mgr.gibba032.whopeo (mgr.24625) 11926 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:10 gibba032 bash[24004]: cluster 2021-03-10T17:39:08.450929+0000 mgr.gibba032.whopeo (mgr.24625) 11927 : cluster [DBG] pgmap v6796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:10 gibba011 bash[30791]: cephadm 2021-03-10T17:39:08.378163+0000 mgr.gibba032.whopeo (mgr.24625) 11926 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:10 gibba011 bash[30791]: cluster 2021-03-10T17:39:08.450929+0000 mgr.gibba032.whopeo (mgr.24625) 11927 : cluster [DBG] pgmap v6796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:10 gibba011 bash[31774]: cephadm 2021-03-10T17:39:08.378163+0000 mgr.gibba032.whopeo (mgr.24625) 11926 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:10.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:10 gibba011 bash[31774]: cluster 2021-03-10T17:39:08.450929+0000 mgr.gibba032.whopeo (mgr.24625) 11927 : cluster [DBG] pgmap v6796: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:12 gibba032 bash[24004]: cluster 2021-03-10T17:39:10.452457+0000 mgr.gibba032.whopeo (mgr.24625) 11928 : cluster [DBG] pgmap v6797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:12.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:12 gibba011 bash[30791]: cluster 2021-03-10T17:39:10.452457+0000 mgr.gibba032.whopeo (mgr.24625) 11928 : cluster [DBG] pgmap v6797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:12 gibba011 bash[31774]: cluster 2021-03-10T17:39:10.452457+0000 mgr.gibba032.whopeo (mgr.24625) 11928 : cluster [DBG] pgmap v6797: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:14.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:14 gibba011 bash[30791]: cluster 2021-03-10T17:39:12.454014+0000 mgr.gibba032.whopeo (mgr.24625) 11929 : cluster [DBG] pgmap v6798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:14.580 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:14 gibba011 bash[31774]: cluster 2021-03-10T17:39:12.454014+0000 mgr.gibba032.whopeo (mgr.24625) 11929 : cluster [DBG] pgmap v6798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:14 gibba032 bash[24004]: cluster 2021-03-10T17:39:12.454014+0000 mgr.gibba032.whopeo (mgr.24625) 11929 : cluster [DBG] pgmap v6798: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:14.800 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:39:15.023 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:39:15.023 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:39:15.024 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:39:15.024 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:39:15.024 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:39:15.024 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:39:15.024 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:39:15.025 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:39:15.025 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:39:15.025 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:39:15.025 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:39:15.026 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:39:15.026 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:39:15.026 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:39:15.026 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:39:15.026 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:39:15.027 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:39:15.028 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:39:15.028 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:39:15.029 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:39:15.029 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:39:15.274 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:39:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:39:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:39:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:39:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:39:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:39:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:39:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:39:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:39:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:39:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:39:15.278 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:39:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[30791]: cluster 2021-03-10T17:39:14.455239+0000 mgr.gibba032.whopeo (mgr.24625) 11930 : cluster [DBG] pgmap v6799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[30791]: audit 2021-03-10T17:39:14.787897+0000 mgr.gibba032.whopeo (mgr.24625) 11931 : audit [DBG] from='client.62029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[30791]: audit 2021-03-10T17:39:15.010166+0000 mgr.gibba032.whopeo (mgr.24625) 11932 : audit [DBG] from='client.62035 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[30791]: audit 2021-03-10T17:39:15.272987+0000 mon.c (mon.1) 231 : audit [DBG] from='client.? 172.21.2.111:0/755293225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[31774]: cluster 2021-03-10T17:39:14.455239+0000 mgr.gibba032.whopeo (mgr.24625) 11930 : cluster [DBG] pgmap v6799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[31774]: audit 2021-03-10T17:39:14.787897+0000 mgr.gibba032.whopeo (mgr.24625) 11931 : audit [DBG] from='client.62029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[31774]: audit 2021-03-10T17:39:15.010166+0000 mgr.gibba032.whopeo (mgr.24625) 11932 : audit [DBG] from='client.62035 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:16 gibba011 bash[31774]: audit 2021-03-10T17:39:15.272987+0000 mon.c (mon.1) 231 : audit [DBG] from='client.? 172.21.2.111:0/755293225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:16 gibba032 bash[24004]: cluster 2021-03-10T17:39:14.455239+0000 mgr.gibba032.whopeo (mgr.24625) 11930 : cluster [DBG] pgmap v6799: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:16 gibba032 bash[24004]: audit 2021-03-10T17:39:14.787897+0000 mgr.gibba032.whopeo (mgr.24625) 11931 : audit [DBG] from='client.62029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:16 gibba032 bash[24004]: audit 2021-03-10T17:39:15.010166+0000 mgr.gibba032.whopeo (mgr.24625) 11932 : audit [DBG] from='client.62035 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:16.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:16 gibba032 bash[24004]: audit 2021-03-10T17:39:15.272987+0000 mon.c (mon.1) 231 : audit [DBG] from='client.? 172.21.2.111:0/755293225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:18 gibba011 bash[30791]: cluster 2021-03-10T17:39:16.457252+0000 mgr.gibba032.whopeo (mgr.24625) 11933 : cluster [DBG] pgmap v6800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:18 gibba011 bash[31774]: cluster 2021-03-10T17:39:16.457252+0000 mgr.gibba032.whopeo (mgr.24625) 11933 : cluster [DBG] pgmap v6800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:18 gibba032 bash[24004]: cluster 2021-03-10T17:39:16.457252+0000 mgr.gibba032.whopeo (mgr.24625) 11933 : cluster [DBG] pgmap v6800: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:20 gibba011 bash[30791]: cluster 2021-03-10T17:39:18.458873+0000 mgr.gibba032.whopeo (mgr.24625) 11934 : cluster [DBG] pgmap v6801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:20 gibba011 bash[31774]: cluster 2021-03-10T17:39:18.458873+0000 mgr.gibba032.whopeo (mgr.24625) 11934 : cluster [DBG] pgmap v6801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:20 gibba032 bash[24004]: cluster 2021-03-10T17:39:18.458873+0000 mgr.gibba032.whopeo (mgr.24625) 11934 : cluster [DBG] pgmap v6801: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:22.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:22 gibba011 bash[30791]: cluster 2021-03-10T17:39:20.460484+0000 mgr.gibba032.whopeo (mgr.24625) 11935 : cluster [DBG] pgmap v6802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:22.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:22 gibba011 bash[31774]: cluster 2021-03-10T17:39:20.460484+0000 mgr.gibba032.whopeo (mgr.24625) 11935 : cluster [DBG] pgmap v6802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:22 gibba032 bash[24004]: cluster 2021-03-10T17:39:20.460484+0000 mgr.gibba032.whopeo (mgr.24625) 11935 : cluster [DBG] pgmap v6802: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:24 gibba011 bash[30791]: cluster 2021-03-10T17:39:22.462353+0000 mgr.gibba032.whopeo (mgr.24625) 11936 : cluster [DBG] pgmap v6803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:24 gibba011 bash[31774]: cluster 2021-03-10T17:39:22.462353+0000 mgr.gibba032.whopeo (mgr.24625) 11936 : cluster [DBG] pgmap v6803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:24 gibba032 bash[24004]: cluster 2021-03-10T17:39:22.462353+0000 mgr.gibba032.whopeo (mgr.24625) 11936 : cluster [DBG] pgmap v6803: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:25.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:25 gibba011 bash[30791]: cephadm 2021-03-10T17:39:23.394064+0000 mgr.gibba032.whopeo (mgr.24625) 11937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:25.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:25 gibba011 bash[31774]: cephadm 2021-03-10T17:39:23.394064+0000 mgr.gibba032.whopeo (mgr.24625) 11937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:25.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:25 gibba032 bash[24004]: cephadm 2021-03-10T17:39:23.394064+0000 mgr.gibba032.whopeo (mgr.24625) 11937 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:26.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:26 gibba011 bash[30791]: cluster 2021-03-10T17:39:24.463677+0000 mgr.gibba032.whopeo (mgr.24625) 11938 : cluster [DBG] pgmap v6804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:26 gibba011 bash[31774]: cluster 2021-03-10T17:39:24.463677+0000 mgr.gibba032.whopeo (mgr.24625) 11938 : cluster [DBG] pgmap v6804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:26 gibba032 bash[24004]: cluster 2021-03-10T17:39:24.463677+0000 mgr.gibba032.whopeo (mgr.24625) 11938 : cluster [DBG] pgmap v6804: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:28 gibba011 bash[30791]: cluster 2021-03-10T17:39:26.465596+0000 mgr.gibba032.whopeo (mgr.24625) 11939 : cluster [DBG] pgmap v6805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:28 gibba011 bash[31774]: cluster 2021-03-10T17:39:26.465596+0000 mgr.gibba032.whopeo (mgr.24625) 11939 : cluster [DBG] pgmap v6805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:28 gibba032 bash[24004]: cluster 2021-03-10T17:39:26.465596+0000 mgr.gibba032.whopeo (mgr.24625) 11939 : cluster [DBG] pgmap v6805: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:30 gibba032 bash[24004]: cluster 2021-03-10T17:39:28.467148+0000 mgr.gibba032.whopeo (mgr.24625) 11940 : cluster [DBG] pgmap v6806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:30 gibba011 bash[30791]: cluster 2021-03-10T17:39:28.467148+0000 mgr.gibba032.whopeo (mgr.24625) 11940 : cluster [DBG] pgmap v6806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:30 gibba011 bash[31774]: cluster 2021-03-10T17:39:28.467148+0000 mgr.gibba032.whopeo (mgr.24625) 11940 : cluster [DBG] pgmap v6806: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:32 gibba032 bash[24004]: cluster 2021-03-10T17:39:30.468810+0000 mgr.gibba032.whopeo (mgr.24625) 11941 : cluster [DBG] pgmap v6807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:32 gibba011 bash[30791]: cluster 2021-03-10T17:39:30.468810+0000 mgr.gibba032.whopeo (mgr.24625) 11941 : cluster [DBG] pgmap v6807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:32.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:32 gibba011 bash[31774]: cluster 2021-03-10T17:39:30.468810+0000 mgr.gibba032.whopeo (mgr.24625) 11941 : cluster [DBG] pgmap v6807: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:34 gibba032 bash[24004]: cluster 2021-03-10T17:39:32.470573+0000 mgr.gibba032.whopeo (mgr.24625) 11942 : cluster [DBG] pgmap v6808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:34.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:34 gibba011 bash[30791]: cluster 2021-03-10T17:39:32.470573+0000 mgr.gibba032.whopeo (mgr.24625) 11942 : cluster [DBG] pgmap v6808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:34.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:34 gibba011 bash[31774]: cluster 2021-03-10T17:39:32.470573+0000 mgr.gibba032.whopeo (mgr.24625) 11942 : cluster [DBG] pgmap v6808: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:36 gibba032 bash[24004]: cluster 2021-03-10T17:39:34.471993+0000 mgr.gibba032.whopeo (mgr.24625) 11943 : cluster [DBG] pgmap v6809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:36.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:36 gibba011 bash[30791]: cluster 2021-03-10T17:39:34.471993+0000 mgr.gibba032.whopeo (mgr.24625) 11943 : cluster [DBG] pgmap v6809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:36.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:36 gibba011 bash[31774]: cluster 2021-03-10T17:39:34.471993+0000 mgr.gibba032.whopeo (mgr.24625) 11943 : cluster [DBG] pgmap v6809: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:38 gibba032 bash[24004]: cluster 2021-03-10T17:39:36.474029+0000 mgr.gibba032.whopeo (mgr.24625) 11944 : cluster [DBG] pgmap v6810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:38.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:38 gibba011 bash[30791]: cluster 2021-03-10T17:39:36.474029+0000 mgr.gibba032.whopeo (mgr.24625) 11944 : cluster [DBG] pgmap v6810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:38.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:38 gibba011 bash[31774]: cluster 2021-03-10T17:39:36.474029+0000 mgr.gibba032.whopeo (mgr.24625) 11944 : cluster [DBG] pgmap v6810: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:40 gibba032 bash[24004]: cephadm 2021-03-10T17:39:38.408529+0000 mgr.gibba032.whopeo (mgr.24625) 11945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:40 gibba032 bash[24004]: cluster 2021-03-10T17:39:38.475402+0000 mgr.gibba032.whopeo (mgr.24625) 11946 : cluster [DBG] pgmap v6811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:40 gibba011 bash[30791]: cephadm 2021-03-10T17:39:38.408529+0000 mgr.gibba032.whopeo (mgr.24625) 11945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:40.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:40 gibba011 bash[30791]: cluster 2021-03-10T17:39:38.475402+0000 mgr.gibba032.whopeo (mgr.24625) 11946 : cluster [DBG] pgmap v6811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:40.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:40 gibba011 bash[31774]: cephadm 2021-03-10T17:39:38.408529+0000 mgr.gibba032.whopeo (mgr.24625) 11945 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:40.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:40 gibba011 bash[31774]: cluster 2021-03-10T17:39:38.475402+0000 mgr.gibba032.whopeo (mgr.24625) 11946 : cluster [DBG] pgmap v6811: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:41.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:41 gibba032 bash[24004]: cluster 2021-03-10T17:39:40.477046+0000 mgr.gibba032.whopeo (mgr.24625) 11947 : cluster [DBG] pgmap v6812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:41.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:41 gibba011 bash[30791]: cluster 2021-03-10T17:39:40.477046+0000 mgr.gibba032.whopeo (mgr.24625) 11947 : cluster [DBG] pgmap v6812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:41.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:41 gibba011 bash[31774]: cluster 2021-03-10T17:39:40.477046+0000 mgr.gibba032.whopeo (mgr.24625) 11947 : cluster [DBG] pgmap v6812: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:44.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:43 gibba032 bash[24004]: cluster 2021-03-10T17:39:42.478551+0000 mgr.gibba032.whopeo (mgr.24625) 11948 : cluster [DBG] pgmap v6813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:44.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:43 gibba011 bash[30791]: cluster 2021-03-10T17:39:42.478551+0000 mgr.gibba032.whopeo (mgr.24625) 11948 : cluster [DBG] pgmap v6813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:44.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:43 gibba011 bash[31774]: cluster 2021-03-10T17:39:42.478551+0000 mgr.gibba032.whopeo (mgr.24625) 11948 : cluster [DBG] pgmap v6813: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:45.517 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:39:45.775 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:39:45.776 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 7m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:39:45.776 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 6m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:39:45.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:39:45.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:39:45.776 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:39:45.777 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:39:45.777 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:39:45.777 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:39:45.777 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:39:45.778 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 6m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:39:45.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:39:45.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:39:45.778 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:39:45.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:39:45.779 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:39:45.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:39:45.780 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:39:45.781 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 6m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:39:45.781 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 6m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:39:45.781 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 7m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:39:46.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:45 gibba032 bash[24004]: cluster 2021-03-10T17:39:44.479951+0000 mgr.gibba032.whopeo (mgr.24625) 11949 : cluster [DBG] pgmap v6814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:46.027 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:39:46.027 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:39:46.027 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:39:46.028 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:46.028 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:39:46.028 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:39:46.028 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:46.029 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:39:46.029 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:39:46.029 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:39:46.029 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:39:46.029 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:39:46.030 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:39:46.030 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:39:46.030 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:39:46.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:45 gibba011 bash[30791]: cluster 2021-03-10T17:39:44.479951+0000 mgr.gibba032.whopeo (mgr.24625) 11949 : cluster [DBG] pgmap v6814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:45 gibba011 bash[31774]: cluster 2021-03-10T17:39:44.479951+0000 mgr.gibba032.whopeo (mgr.24625) 11949 : cluster [DBG] pgmap v6814: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:47.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:46 gibba032 bash[24004]: audit 2021-03-10T17:39:45.505320+0000 mgr.gibba032.whopeo (mgr.24625) 11950 : audit [DBG] from='client.52065 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:46 gibba032 bash[24004]: audit 2021-03-10T17:39:45.731052+0000 mgr.gibba032.whopeo (mgr.24625) 11951 : audit [DBG] from='client.62053 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:46 gibba032 bash[24004]: audit 2021-03-10T17:39:46.025433+0000 mon.a (mon.0) 3912 : audit [DBG] from='client.? 172.21.2.111:0/3862273682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[30791]: audit 2021-03-10T17:39:45.505320+0000 mgr.gibba032.whopeo (mgr.24625) 11950 : audit [DBG] from='client.52065 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[30791]: audit 2021-03-10T17:39:45.731052+0000 mgr.gibba032.whopeo (mgr.24625) 11951 : audit [DBG] from='client.62053 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[30791]: audit 2021-03-10T17:39:46.025433+0000 mon.a (mon.0) 3912 : audit [DBG] from='client.? 172.21.2.111:0/3862273682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[31774]: audit 2021-03-10T17:39:45.505320+0000 mgr.gibba032.whopeo (mgr.24625) 11950 : audit [DBG] from='client.52065 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[31774]: audit 2021-03-10T17:39:45.731052+0000 mgr.gibba032.whopeo (mgr.24625) 11951 : audit [DBG] from='client.62053 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:39:47.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:46 gibba011 bash[31774]: audit 2021-03-10T17:39:46.025433+0000 mon.a (mon.0) 3912 : audit [DBG] from='client.? 172.21.2.111:0/3862273682' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:48.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:47 gibba032 bash[24004]: cluster 2021-03-10T17:39:46.480883+0000 mgr.gibba032.whopeo (mgr.24625) 11952 : cluster [DBG] pgmap v6815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:47 gibba011 bash[30791]: cluster 2021-03-10T17:39:46.480883+0000 mgr.gibba032.whopeo (mgr.24625) 11952 : cluster [DBG] pgmap v6815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:47 gibba011 bash[31774]: cluster 2021-03-10T17:39:46.480883+0000 mgr.gibba032.whopeo (mgr.24625) 11952 : cluster [DBG] pgmap v6815: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:50.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:49 gibba032 bash[24004]: cluster 2021-03-10T17:39:48.482349+0000 mgr.gibba032.whopeo (mgr.24625) 11953 : cluster [DBG] pgmap v6816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:49 gibba011 bash[30791]: cluster 2021-03-10T17:39:48.482349+0000 mgr.gibba032.whopeo (mgr.24625) 11953 : cluster [DBG] pgmap v6816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:49 gibba011 bash[31774]: cluster 2021-03-10T17:39:48.482349+0000 mgr.gibba032.whopeo (mgr.24625) 11953 : cluster [DBG] pgmap v6816: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:52.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:51 gibba032 bash[24004]: cluster 2021-03-10T17:39:50.483917+0000 mgr.gibba032.whopeo (mgr.24625) 11954 : cluster [DBG] pgmap v6817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:52.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:51 gibba011 bash[30791]: cluster 2021-03-10T17:39:50.483917+0000 mgr.gibba032.whopeo (mgr.24625) 11954 : cluster [DBG] pgmap v6817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:52.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:51 gibba011 bash[31774]: cluster 2021-03-10T17:39:50.483917+0000 mgr.gibba032.whopeo (mgr.24625) 11954 : cluster [DBG] pgmap v6817: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:54.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: cluster 2021-03-10T17:39:52.484543+0000 mgr.gibba032.whopeo (mgr.24625) 11955 : cluster [DBG] pgmap v6818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:54.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.623271+0000 mon.a (mon.0) 3913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:39:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.626522+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:39:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.628287+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.629702+0000 mon.a (mon.0) 3916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.631028+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.633221+0000 mon.a (mon.0) 3918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:53 gibba032 bash[24004]: audit 2021-03-10T17:39:53.635124+0000 mon.a (mon.0) 3919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: cluster 2021-03-10T17:39:52.484543+0000 mgr.gibba032.whopeo (mgr.24625) 11955 : cluster [DBG] pgmap v6818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.623271+0000 mon.a (mon.0) 3913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:39:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.626522+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:39:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.628287+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.629702+0000 mon.a (mon.0) 3916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.631028+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.633221+0000 mon.a (mon.0) 3918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[30791]: audit 2021-03-10T17:39:53.635124+0000 mon.a (mon.0) 3919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: cluster 2021-03-10T17:39:52.484543+0000 mgr.gibba032.whopeo (mgr.24625) 11955 : cluster [DBG] pgmap v6818: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:54.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.623271+0000 mon.a (mon.0) 3913 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:39:54.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.626522+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:39:54.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.628287+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.629702+0000 mon.a (mon.0) 3916 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.631028+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.633221+0000 mon.a (mon.0) 3918 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:54.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:53 gibba011 bash[31774]: audit 2021-03-10T17:39:53.635124+0000 mon.a (mon.0) 3919 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:39:55.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.413439+0000 mgr.gibba032.whopeo (mgr.24625) 11956 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:39:55.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.414968+0000 mgr.gibba032.whopeo (mgr.24625) 11957 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:39:55.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.624194+0000 mgr.gibba032.whopeo (mgr.24625) 11958 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:39:55.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.626358+0000 mgr.gibba032.whopeo (mgr.24625) 11959 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:39:55.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.627593+0000 mgr.gibba032.whopeo (mgr.24625) 11960 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:39:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.627629+0000 mgr.gibba032.whopeo (mgr.24625) 11961 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:39:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.629088+0000 mgr.gibba032.whopeo (mgr.24625) 11962 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:39:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.629125+0000 mgr.gibba032.whopeo (mgr.24625) 11963 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:39:55.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.630168+0000 mgr.gibba032.whopeo (mgr.24625) 11964 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:39:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.631042+0000 mgr.gibba032.whopeo (mgr.24625) 11965 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:39:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.631077+0000 mgr.gibba032.whopeo (mgr.24625) 11966 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:39:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.632708+0000 mgr.gibba032.whopeo (mgr.24625) 11967 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:39:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.633086+0000 mgr.gibba032.whopeo (mgr.24625) 11968 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:39:55.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.634681+0000 mgr.gibba032.whopeo (mgr.24625) 11969 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:39:55.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.634720+0000 mgr.gibba032.whopeo (mgr.24625) 11970 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:39:55.012 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:54 gibba032 bash[24004]: cephadm 2021-03-10T17:39:53.634826+0000 mgr.gibba032.whopeo (mgr.24625) 11971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:55.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.413439+0000 mgr.gibba032.whopeo (mgr.24625) 11956 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:39:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.414968+0000 mgr.gibba032.whopeo (mgr.24625) 11957 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:39:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.624194+0000 mgr.gibba032.whopeo (mgr.24625) 11958 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:39:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.626358+0000 mgr.gibba032.whopeo (mgr.24625) 11959 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:39:55.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.627593+0000 mgr.gibba032.whopeo (mgr.24625) 11960 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:39:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.627629+0000 mgr.gibba032.whopeo (mgr.24625) 11961 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:39:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.629088+0000 mgr.gibba032.whopeo (mgr.24625) 11962 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:39:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.629125+0000 mgr.gibba032.whopeo (mgr.24625) 11963 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:39:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.630168+0000 mgr.gibba032.whopeo (mgr.24625) 11964 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:39:55.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.631042+0000 mgr.gibba032.whopeo (mgr.24625) 11965 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:39:55.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.631077+0000 mgr.gibba032.whopeo (mgr.24625) 11966 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:39:55.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.632708+0000 mgr.gibba032.whopeo (mgr.24625) 11967 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:39:55.083 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.633086+0000 mgr.gibba032.whopeo (mgr.24625) 11968 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:39:55.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.634681+0000 mgr.gibba032.whopeo (mgr.24625) 11969 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:39:55.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.634720+0000 mgr.gibba032.whopeo (mgr.24625) 11970 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:39:55.084 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[30791]: cephadm 2021-03-10T17:39:53.634826+0000 mgr.gibba032.whopeo (mgr.24625) 11971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:55.084 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.413439+0000 mgr.gibba032.whopeo (mgr.24625) 11956 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:39:55.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.414968+0000 mgr.gibba032.whopeo (mgr.24625) 11957 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:39:55.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.624194+0000 mgr.gibba032.whopeo (mgr.24625) 11958 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:39:55.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.626358+0000 mgr.gibba032.whopeo (mgr.24625) 11959 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:39:55.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.627593+0000 mgr.gibba032.whopeo (mgr.24625) 11960 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:39:55.085 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.627629+0000 mgr.gibba032.whopeo (mgr.24625) 11961 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:39:55.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.629088+0000 mgr.gibba032.whopeo (mgr.24625) 11962 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:39:55.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.629125+0000 mgr.gibba032.whopeo (mgr.24625) 11963 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:39:55.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.630168+0000 mgr.gibba032.whopeo (mgr.24625) 11964 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:39:55.086 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.631042+0000 mgr.gibba032.whopeo (mgr.24625) 11965 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:39:55.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.631077+0000 mgr.gibba032.whopeo (mgr.24625) 11966 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:39:55.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.632708+0000 mgr.gibba032.whopeo (mgr.24625) 11967 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:39:55.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.633086+0000 mgr.gibba032.whopeo (mgr.24625) 11968 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:39:55.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.634681+0000 mgr.gibba032.whopeo (mgr.24625) 11969 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:39:55.087 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.634720+0000 mgr.gibba032.whopeo (mgr.24625) 11970 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:39:55.088 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:54 gibba011 bash[31774]: cephadm 2021-03-10T17:39:53.634826+0000 mgr.gibba032.whopeo (mgr.24625) 11971 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:39:56.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:55 gibba032 bash[24004]: cluster 2021-03-10T17:39:54.485836+0000 mgr.gibba032.whopeo (mgr.24625) 11972 : cluster [DBG] pgmap v6819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:56.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:55 gibba011 bash[30791]: cluster 2021-03-10T17:39:54.485836+0000 mgr.gibba032.whopeo (mgr.24625) 11972 : cluster [DBG] pgmap v6819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:55 gibba011 bash[31774]: cluster 2021-03-10T17:39:54.485836+0000 mgr.gibba032.whopeo (mgr.24625) 11972 : cluster [DBG] pgmap v6819: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:57 gibba032 bash[24004]: cluster 2021-03-10T17:39:56.487733+0000 mgr.gibba032.whopeo (mgr.24625) 11973 : cluster [DBG] pgmap v6820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:58.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:57 gibba032 bash[24004]: audit 2021-03-10T17:39:57.582092+0000 mon.a (mon.0) 3920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:39:58.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:57 gibba032 bash[24004]: audit 2021-03-10T17:39:57.582806+0000 mon.a (mon.0) 3921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:39:58.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[30791]: cluster 2021-03-10T17:39:56.487733+0000 mgr.gibba032.whopeo (mgr.24625) 11973 : cluster [DBG] pgmap v6820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[30791]: audit 2021-03-10T17:39:57.582092+0000 mon.a (mon.0) 3920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:39:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[30791]: audit 2021-03-10T17:39:57.582806+0000 mon.a (mon.0) 3921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:39:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[31774]: cluster 2021-03-10T17:39:56.487733+0000 mgr.gibba032.whopeo (mgr.24625) 11973 : cluster [DBG] pgmap v6820: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:39:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[31774]: audit 2021-03-10T17:39:57.582092+0000 mon.a (mon.0) 3920 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:39:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:57 gibba011 bash[31774]: audit 2021-03-10T17:39:57.582806+0000 mon.a (mon.0) 3921 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:39:59.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:58 gibba032 bash[24004]: audit 2021-03-10T17:39:57.801451+0000 mon.a (mon.0) 3922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:39:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:58 gibba032 bash[24004]: audit 2021-03-10T17:39:57.801862+0000 mon.a (mon.0) 3923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:39:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:58 gibba032 bash[24004]: audit 2021-03-10T17:39:57.802236+0000 mon.a (mon.0) 3924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:39:59.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:58 gibba032 bash[24004]: audit 2021-03-10T17:39:57.802618+0000 mon.a (mon.0) 3925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:39:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[30791]: audit 2021-03-10T17:39:57.801451+0000 mon.a (mon.0) 3922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:39:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[30791]: audit 2021-03-10T17:39:57.801862+0000 mon.a (mon.0) 3923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:39:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[30791]: audit 2021-03-10T17:39:57.802236+0000 mon.a (mon.0) 3924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:39:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[30791]: audit 2021-03-10T17:39:57.802618+0000 mon.a (mon.0) 3925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:39:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[31774]: audit 2021-03-10T17:39:57.801451+0000 mon.a (mon.0) 3922 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:39:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[31774]: audit 2021-03-10T17:39:57.801862+0000 mon.a (mon.0) 3923 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 5]}]: dispatch 2021-03-10T17:39:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[31774]: audit 2021-03-10T17:39:57.802236+0000 mon.a (mon.0) 3924 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:39:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:58 gibba011 bash[31774]: audit 2021-03-10T17:39:57.802618+0000 mon.a (mon.0) 3925 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2021-03-10T17:40:00.002 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:39:59 gibba011 bash[30791]: cluster 2021-03-10T17:39:58.488424+0000 mgr.gibba032.whopeo (mgr.24625) 11974 : cluster [DBG] pgmap v6821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:00.003 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:39:59 gibba011 bash[31774]: cluster 2021-03-10T17:39:58.488424+0000 mgr.gibba032.whopeo (mgr.24625) 11974 : cluster [DBG] pgmap v6821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:00.008 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:39:59 gibba032 bash[24004]: cluster 2021-03-10T17:39:58.488424+0000 mgr.gibba032.whopeo (mgr.24625) 11974 : cluster [DBG] pgmap v6821: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:00.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:00 gibba011 bash[30791]: debug 2021-03-10T17:39:59.995+0000 7efe9d847700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:40:01.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:00 gibba032 bash[24004]: cluster 2021-03-10T17:40:00.000219+0000 mon.a (mon.0) 3926 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:40:01.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:00 gibba011 bash[30791]: cluster 2021-03-10T17:40:00.000219+0000 mon.a (mon.0) 3926 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:40:01.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:00 gibba011 bash[31774]: cluster 2021-03-10T17:40:00.000219+0000 mon.a (mon.0) 3926 : cluster [ERR] overall HEALTH_ERR 1 failed cephadm daemon(s); Module 'dashboard' has failed: Timeout('Port 8443 not free on ::.',) 2021-03-10T17:40:02.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:01 gibba032 bash[24004]: cluster 2021-03-10T17:40:00.489837+0000 mgr.gibba032.whopeo (mgr.24625) 11975 : cluster [DBG] pgmap v6822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:02.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:01 gibba011 bash[30791]: cluster 2021-03-10T17:40:00.489837+0000 mgr.gibba032.whopeo (mgr.24625) 11975 : cluster [DBG] pgmap v6822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:02.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:01 gibba011 bash[31774]: cluster 2021-03-10T17:40:00.489837+0000 mgr.gibba032.whopeo (mgr.24625) 11975 : cluster [DBG] pgmap v6822: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:04.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:03 gibba011 bash[30791]: cluster 2021-03-10T17:40:02.491575+0000 mgr.gibba032.whopeo (mgr.24625) 11976 : cluster [DBG] pgmap v6823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:04.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:03 gibba011 bash[31774]: cluster 2021-03-10T17:40:02.491575+0000 mgr.gibba032.whopeo (mgr.24625) 11976 : cluster [DBG] pgmap v6823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:04.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:03 gibba032 bash[24004]: cluster 2021-03-10T17:40:02.491575+0000 mgr.gibba032.whopeo (mgr.24625) 11976 : cluster [DBG] pgmap v6823: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:06.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:05 gibba011 bash[30791]: cluster 2021-03-10T17:40:04.492257+0000 mgr.gibba032.whopeo (mgr.24625) 11977 : cluster [DBG] pgmap v6824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:06.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:05 gibba011 bash[31774]: cluster 2021-03-10T17:40:04.492257+0000 mgr.gibba032.whopeo (mgr.24625) 11977 : cluster [DBG] pgmap v6824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:06.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:05 gibba032 bash[24004]: cluster 2021-03-10T17:40:04.492257+0000 mgr.gibba032.whopeo (mgr.24625) 11977 : cluster [DBG] pgmap v6824: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:08.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:07 gibba011 bash[30791]: cluster 2021-03-10T17:40:06.493775+0000 mgr.gibba032.whopeo (mgr.24625) 11978 : cluster [DBG] pgmap v6825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:08.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:07 gibba011 bash[31774]: cluster 2021-03-10T17:40:06.493775+0000 mgr.gibba032.whopeo (mgr.24625) 11978 : cluster [DBG] pgmap v6825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:08.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:07 gibba032 bash[24004]: cluster 2021-03-10T17:40:06.493775+0000 mgr.gibba032.whopeo (mgr.24625) 11978 : cluster [DBG] pgmap v6825: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:10.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:09 gibba011 bash[30791]: cluster 2021-03-10T17:40:08.495270+0000 mgr.gibba032.whopeo (mgr.24625) 11979 : cluster [DBG] pgmap v6826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:10.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:09 gibba011 bash[30791]: cephadm 2021-03-10T17:40:08.650554+0000 mgr.gibba032.whopeo (mgr.24625) 11980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:10.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:09 gibba011 bash[31774]: cluster 2021-03-10T17:40:08.495270+0000 mgr.gibba032.whopeo (mgr.24625) 11979 : cluster [DBG] pgmap v6826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:10.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:09 gibba011 bash[31774]: cephadm 2021-03-10T17:40:08.650554+0000 mgr.gibba032.whopeo (mgr.24625) 11980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:09 gibba032 bash[24004]: cluster 2021-03-10T17:40:08.495270+0000 mgr.gibba032.whopeo (mgr.24625) 11979 : cluster [DBG] pgmap v6826: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:09 gibba032 bash[24004]: cephadm 2021-03-10T17:40:08.650554+0000 mgr.gibba032.whopeo (mgr.24625) 11980 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:12.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:11 gibba011 bash[30791]: cluster 2021-03-10T17:40:10.495906+0000 mgr.gibba032.whopeo (mgr.24625) 11981 : cluster [DBG] pgmap v6827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:12.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:11 gibba011 bash[31774]: cluster 2021-03-10T17:40:10.495906+0000 mgr.gibba032.whopeo (mgr.24625) 11981 : cluster [DBG] pgmap v6827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:11 gibba032 bash[24004]: cluster 2021-03-10T17:40:10.495906+0000 mgr.gibba032.whopeo (mgr.24625) 11981 : cluster [DBG] pgmap v6827: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:14.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:13 gibba011 bash[30791]: cluster 2021-03-10T17:40:12.497564+0000 mgr.gibba032.whopeo (mgr.24625) 11982 : cluster [DBG] pgmap v6828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:14.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:13 gibba011 bash[31774]: cluster 2021-03-10T17:40:12.497564+0000 mgr.gibba032.whopeo (mgr.24625) 11982 : cluster [DBG] pgmap v6828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:13 gibba032 bash[24004]: cluster 2021-03-10T17:40:12.497564+0000 mgr.gibba032.whopeo (mgr.24625) 11982 : cluster [DBG] pgmap v6828: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:16.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:15 gibba011 bash[30791]: cluster 2021-03-10T17:40:14.498947+0000 mgr.gibba032.whopeo (mgr.24625) 11983 : cluster [DBG] pgmap v6829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:16.080 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:15 gibba011 bash[31774]: cluster 2021-03-10T17:40:14.498947+0000 mgr.gibba032.whopeo (mgr.24625) 11983 : cluster [DBG] pgmap v6829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:16.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:15 gibba032 bash[24004]: cluster 2021-03-10T17:40:14.498947+0000 mgr.gibba032.whopeo (mgr.24625) 11983 : cluster [DBG] pgmap v6829: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:16.271 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:40:16.496 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:40:16.496 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 8m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:40:16.497 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:40:16.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:40:16.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:40:16.497 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:40:16.497 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:40:16.498 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:40:16.498 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:40:16.498 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:40:16.498 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:40:16.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:40:16.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:40:16.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:40:16.499 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:40:16.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:40:16.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:40:16.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:40:16.500 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:40:16.500 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:40:16.501 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 8m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:40:16.749 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:40:16.749 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:40:16.749 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:40:16.750 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:16.750 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:40:16.750 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:40:16.750 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:16.751 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:40:16.751 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:40:16.751 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:16.751 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:40:16.751 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:40:16.752 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:40:16.752 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:40:16.752 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:40:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:16 gibba011 bash[30791]: audit 2021-03-10T17:40:16.259131+0000 mgr.gibba032.whopeo (mgr.24625) 11984 : audit [DBG] from='client.52083 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:17.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:16 gibba011 bash[30791]: audit 2021-03-10T17:40:16.747576+0000 mon.c (mon.1) 232 : audit [DBG] from='client.? 172.21.2.111:0/2431433056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:17.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:16 gibba011 bash[31774]: audit 2021-03-10T17:40:16.259131+0000 mgr.gibba032.whopeo (mgr.24625) 11984 : audit [DBG] from='client.52083 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:17.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:16 gibba011 bash[31774]: audit 2021-03-10T17:40:16.747576+0000 mon.c (mon.1) 232 : audit [DBG] from='client.? 172.21.2.111:0/2431433056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:16 gibba032 bash[24004]: audit 2021-03-10T17:40:16.259131+0000 mgr.gibba032.whopeo (mgr.24625) 11984 : audit [DBG] from='client.52083 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:17.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:16 gibba032 bash[24004]: audit 2021-03-10T17:40:16.747576+0000 mon.c (mon.1) 232 : audit [DBG] from='client.? 172.21.2.111:0/2431433056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:18.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:17 gibba011 bash[30791]: audit 2021-03-10T17:40:16.484663+0000 mgr.gibba032.whopeo (mgr.24625) 11985 : audit [DBG] from='client.62071 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:18.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:17 gibba011 bash[30791]: cluster 2021-03-10T17:40:16.499771+0000 mgr.gibba032.whopeo (mgr.24625) 11986 : cluster [DBG] pgmap v6830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:17 gibba011 bash[31774]: audit 2021-03-10T17:40:16.484663+0000 mgr.gibba032.whopeo (mgr.24625) 11985 : audit [DBG] from='client.62071 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:18.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:17 gibba011 bash[31774]: cluster 2021-03-10T17:40:16.499771+0000 mgr.gibba032.whopeo (mgr.24625) 11986 : cluster [DBG] pgmap v6830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:17 gibba032 bash[24004]: audit 2021-03-10T17:40:16.484663+0000 mgr.gibba032.whopeo (mgr.24625) 11985 : audit [DBG] from='client.62071 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:17 gibba032 bash[24004]: cluster 2021-03-10T17:40:16.499771+0000 mgr.gibba032.whopeo (mgr.24625) 11986 : cluster [DBG] pgmap v6830: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:20.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:19 gibba032 bash[24004]: cluster 2021-03-10T17:40:18.501302+0000 mgr.gibba032.whopeo (mgr.24625) 11987 : cluster [DBG] pgmap v6831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:20.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:19 gibba011 bash[30791]: cluster 2021-03-10T17:40:18.501302+0000 mgr.gibba032.whopeo (mgr.24625) 11987 : cluster [DBG] pgmap v6831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:19 gibba011 bash[31774]: cluster 2021-03-10T17:40:18.501302+0000 mgr.gibba032.whopeo (mgr.24625) 11987 : cluster [DBG] pgmap v6831: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:21 gibba032 bash[24004]: cluster 2021-03-10T17:40:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 11988 : cluster [DBG] pgmap v6832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:21 gibba011 bash[30791]: cluster 2021-03-10T17:40:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 11988 : cluster [DBG] pgmap v6832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:21 gibba011 bash[31774]: cluster 2021-03-10T17:40:20.502856+0000 mgr.gibba032.whopeo (mgr.24625) 11988 : cluster [DBG] pgmap v6832: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:24.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:23 gibba032 bash[24004]: cluster 2021-03-10T17:40:22.503667+0000 mgr.gibba032.whopeo (mgr.24625) 11989 : cluster [DBG] pgmap v6833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:24.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:23 gibba011 bash[30791]: cluster 2021-03-10T17:40:22.503667+0000 mgr.gibba032.whopeo (mgr.24625) 11989 : cluster [DBG] pgmap v6833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:23 gibba011 bash[31774]: cluster 2021-03-10T17:40:22.503667+0000 mgr.gibba032.whopeo (mgr.24625) 11989 : cluster [DBG] pgmap v6833: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:25.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:24 gibba032 bash[24004]: cephadm 2021-03-10T17:40:23.666485+0000 mgr.gibba032.whopeo (mgr.24625) 11990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:25.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:24 gibba011 bash[30791]: cephadm 2021-03-10T17:40:23.666485+0000 mgr.gibba032.whopeo (mgr.24625) 11990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:25.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:24 gibba011 bash[31774]: cephadm 2021-03-10T17:40:23.666485+0000 mgr.gibba032.whopeo (mgr.24625) 11990 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:26.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:25 gibba032 bash[24004]: cluster 2021-03-10T17:40:24.505018+0000 mgr.gibba032.whopeo (mgr.24625) 11991 : cluster [DBG] pgmap v6834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:26.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:25 gibba011 bash[30791]: cluster 2021-03-10T17:40:24.505018+0000 mgr.gibba032.whopeo (mgr.24625) 11991 : cluster [DBG] pgmap v6834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:25 gibba011 bash[31774]: cluster 2021-03-10T17:40:24.505018+0000 mgr.gibba032.whopeo (mgr.24625) 11991 : cluster [DBG] pgmap v6834: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:27 gibba032 bash[24004]: cluster 2021-03-10T17:40:26.506719+0000 mgr.gibba032.whopeo (mgr.24625) 11992 : cluster [DBG] pgmap v6835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:27 gibba011 bash[30791]: cluster 2021-03-10T17:40:26.506719+0000 mgr.gibba032.whopeo (mgr.24625) 11992 : cluster [DBG] pgmap v6835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:27 gibba011 bash[31774]: cluster 2021-03-10T17:40:26.506719+0000 mgr.gibba032.whopeo (mgr.24625) 11992 : cluster [DBG] pgmap v6835: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:30.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:29 gibba032 bash[24004]: cluster 2021-03-10T17:40:28.507496+0000 mgr.gibba032.whopeo (mgr.24625) 11993 : cluster [DBG] pgmap v6836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:29 gibba011 bash[30791]: cluster 2021-03-10T17:40:28.507496+0000 mgr.gibba032.whopeo (mgr.24625) 11993 : cluster [DBG] pgmap v6836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:29 gibba011 bash[31774]: cluster 2021-03-10T17:40:28.507496+0000 mgr.gibba032.whopeo (mgr.24625) 11993 : cluster [DBG] pgmap v6836: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:31 gibba032 bash[24004]: cluster 2021-03-10T17:40:30.509042+0000 mgr.gibba032.whopeo (mgr.24625) 11994 : cluster [DBG] pgmap v6837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:32.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:31 gibba011 bash[30791]: cluster 2021-03-10T17:40:30.509042+0000 mgr.gibba032.whopeo (mgr.24625) 11994 : cluster [DBG] pgmap v6837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:31 gibba011 bash[31774]: cluster 2021-03-10T17:40:30.509042+0000 mgr.gibba032.whopeo (mgr.24625) 11994 : cluster [DBG] pgmap v6837: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:33 gibba032 bash[24004]: cluster 2021-03-10T17:40:32.510748+0000 mgr.gibba032.whopeo (mgr.24625) 11995 : cluster [DBG] pgmap v6838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:33 gibba011 bash[30791]: cluster 2021-03-10T17:40:32.510748+0000 mgr.gibba032.whopeo (mgr.24625) 11995 : cluster [DBG] pgmap v6838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:33 gibba011 bash[31774]: cluster 2021-03-10T17:40:32.510748+0000 mgr.gibba032.whopeo (mgr.24625) 11995 : cluster [DBG] pgmap v6838: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:36.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:35 gibba032 bash[24004]: cluster 2021-03-10T17:40:34.511629+0000 mgr.gibba032.whopeo (mgr.24625) 11996 : cluster [DBG] pgmap v6839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:36.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:35 gibba011 bash[30791]: cluster 2021-03-10T17:40:34.511629+0000 mgr.gibba032.whopeo (mgr.24625) 11996 : cluster [DBG] pgmap v6839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:35 gibba011 bash[31774]: cluster 2021-03-10T17:40:34.511629+0000 mgr.gibba032.whopeo (mgr.24625) 11996 : cluster [DBG] pgmap v6839: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:38.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:37 gibba032 bash[24004]: cluster 2021-03-10T17:40:36.513259+0000 mgr.gibba032.whopeo (mgr.24625) 11997 : cluster [DBG] pgmap v6840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:37 gibba011 bash[30791]: cluster 2021-03-10T17:40:36.513259+0000 mgr.gibba032.whopeo (mgr.24625) 11997 : cluster [DBG] pgmap v6840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:37 gibba011 bash[31774]: cluster 2021-03-10T17:40:36.513259+0000 mgr.gibba032.whopeo (mgr.24625) 11997 : cluster [DBG] pgmap v6840: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:40.258 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:39 gibba032 bash[24004]: cluster 2021-03-10T17:40:38.514742+0000 mgr.gibba032.whopeo (mgr.24625) 11998 : cluster [DBG] pgmap v6841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:40.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:39 gibba032 bash[24004]: cephadm 2021-03-10T17:40:38.682404+0000 mgr.gibba032.whopeo (mgr.24625) 11999 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:40.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:39 gibba011 bash[30791]: cluster 2021-03-10T17:40:38.514742+0000 mgr.gibba032.whopeo (mgr.24625) 11998 : cluster [DBG] pgmap v6841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:39 gibba011 bash[30791]: cephadm 2021-03-10T17:40:38.682404+0000 mgr.gibba032.whopeo (mgr.24625) 11999 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:39 gibba011 bash[31774]: cluster 2021-03-10T17:40:38.514742+0000 mgr.gibba032.whopeo (mgr.24625) 11998 : cluster [DBG] pgmap v6841: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:39 gibba011 bash[31774]: cephadm 2021-03-10T17:40:38.682404+0000 mgr.gibba032.whopeo (mgr.24625) 11999 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:41 gibba032 bash[24004]: cluster 2021-03-10T17:40:40.515514+0000 mgr.gibba032.whopeo (mgr.24625) 12000 : cluster [DBG] pgmap v6842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:42.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:41 gibba011 bash[30791]: cluster 2021-03-10T17:40:40.515514+0000 mgr.gibba032.whopeo (mgr.24625) 12000 : cluster [DBG] pgmap v6842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:41 gibba011 bash[31774]: cluster 2021-03-10T17:40:40.515514+0000 mgr.gibba032.whopeo (mgr.24625) 12000 : cluster [DBG] pgmap v6842: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:43 gibba032 bash[24004]: cluster 2021-03-10T17:40:42.517174+0000 mgr.gibba032.whopeo (mgr.24625) 12001 : cluster [DBG] pgmap v6843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:44.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:43 gibba011 bash[30791]: cluster 2021-03-10T17:40:42.517174+0000 mgr.gibba032.whopeo (mgr.24625) 12001 : cluster [DBG] pgmap v6843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:43 gibba011 bash[31774]: cluster 2021-03-10T17:40:42.517174+0000 mgr.gibba032.whopeo (mgr.24625) 12001 : cluster [DBG] pgmap v6843: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:46.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:45 gibba032 bash[24004]: cluster 2021-03-10T17:40:44.518480+0000 mgr.gibba032.whopeo (mgr.24625) 12002 : cluster [DBG] pgmap v6844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:45 gibba011 bash[30791]: cluster 2021-03-10T17:40:44.518480+0000 mgr.gibba032.whopeo (mgr.24625) 12002 : cluster [DBG] pgmap v6844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:45 gibba011 bash[31774]: cluster 2021-03-10T17:40:44.518480+0000 mgr.gibba032.whopeo (mgr.24625) 12002 : cluster [DBG] pgmap v6844: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:46.988 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:40:47.216 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:40:47.217 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:40:47.217 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 7m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:40:47.217 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:40:47.217 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:40:47.218 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:40:47.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:40:47.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:40:47.218 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:40:47.219 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:40:47.219 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 7m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:40:47.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:40:47.219 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:40:47.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:40:47.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:40:47.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:40:47.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:40:47.220 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:40:47.221 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 7m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:40:47.221 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 7m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:40:47.221 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:40:47.463 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:40:47.463 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:40:47.463 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:40:47.463 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:47.464 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:40:47.464 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:40:47.464 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:47.464 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:40:47.465 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:40:47.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:40:47.465 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:40:47.465 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:40:47.466 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:40:47.466 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:40:47.466 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:40:48.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:47 gibba032 bash[24004]: cluster 2021-03-10T17:40:46.520269+0000 mgr.gibba032.whopeo (mgr.24625) 12003 : cluster [DBG] pgmap v6845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:47 gibba032 bash[24004]: audit 2021-03-10T17:40:46.976863+0000 mgr.gibba032.whopeo (mgr.24625) 12004 : audit [DBG] from='client.52101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:47 gibba032 bash[24004]: audit 2021-03-10T17:40:47.201452+0000 mgr.gibba032.whopeo (mgr.24625) 12005 : audit [DBG] from='client.62089 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:47 gibba032 bash[24004]: audit 2021-03-10T17:40:47.461154+0000 mon.c (mon.1) 233 : audit [DBG] from='client.? 172.21.2.111:0/2433538503' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[30791]: cluster 2021-03-10T17:40:46.520269+0000 mgr.gibba032.whopeo (mgr.24625) 12003 : cluster [DBG] pgmap v6845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[30791]: audit 2021-03-10T17:40:46.976863+0000 mgr.gibba032.whopeo (mgr.24625) 12004 : audit [DBG] from='client.52101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[30791]: audit 2021-03-10T17:40:47.201452+0000 mgr.gibba032.whopeo (mgr.24625) 12005 : audit [DBG] from='client.62089 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[30791]: audit 2021-03-10T17:40:47.461154+0000 mon.c (mon.1) 233 : audit [DBG] from='client.? 172.21.2.111:0/2433538503' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[31774]: cluster 2021-03-10T17:40:46.520269+0000 mgr.gibba032.whopeo (mgr.24625) 12003 : cluster [DBG] pgmap v6845: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[31774]: audit 2021-03-10T17:40:46.976863+0000 mgr.gibba032.whopeo (mgr.24625) 12004 : audit [DBG] from='client.52101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[31774]: audit 2021-03-10T17:40:47.201452+0000 mgr.gibba032.whopeo (mgr.24625) 12005 : audit [DBG] from='client.62089 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:40:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:47 gibba011 bash[31774]: audit 2021-03-10T17:40:47.461154+0000 mon.c (mon.1) 233 : audit [DBG] from='client.? 172.21.2.111:0/2433538503' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:49 gibba032 bash[24004]: cluster 2021-03-10T17:40:48.521708+0000 mgr.gibba032.whopeo (mgr.24625) 12006 : cluster [DBG] pgmap v6846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:49 gibba011 bash[30791]: cluster 2021-03-10T17:40:48.521708+0000 mgr.gibba032.whopeo (mgr.24625) 12006 : cluster [DBG] pgmap v6846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:49 gibba011 bash[31774]: cluster 2021-03-10T17:40:48.521708+0000 mgr.gibba032.whopeo (mgr.24625) 12006 : cluster [DBG] pgmap v6846: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:51 gibba032 bash[24004]: cluster 2021-03-10T17:40:50.523364+0000 mgr.gibba032.whopeo (mgr.24625) 12007 : cluster [DBG] pgmap v6847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:51 gibba011 bash[30791]: cluster 2021-03-10T17:40:50.523364+0000 mgr.gibba032.whopeo (mgr.24625) 12007 : cluster [DBG] pgmap v6847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:51 gibba011 bash[31774]: cluster 2021-03-10T17:40:50.523364+0000 mgr.gibba032.whopeo (mgr.24625) 12007 : cluster [DBG] pgmap v6847: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:54.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: cluster 2021-03-10T17:40:52.524327+0000 mgr.gibba032.whopeo (mgr.24625) 12008 : cluster [DBG] pgmap v6848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.921510+0000 mon.a (mon.0) 3927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:40:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.924574+0000 mon.a (mon.0) 3928 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:40:54.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.926399+0000 mon.a (mon.0) 3929 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.927875+0000 mon.a (mon.0) 3930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.929237+0000 mon.a (mon.0) 3931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.932491+0000 mon.a (mon.0) 3932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:53 gibba032 bash[24004]: audit 2021-03-10T17:40:53.934147+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: cluster 2021-03-10T17:40:52.524327+0000 mgr.gibba032.whopeo (mgr.24625) 12008 : cluster [DBG] pgmap v6848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.921510+0000 mon.a (mon.0) 3927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:40:54.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.924574+0000 mon.a (mon.0) 3928 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:40:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.926399+0000 mon.a (mon.0) 3929 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.927875+0000 mon.a (mon.0) 3930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.929237+0000 mon.a (mon.0) 3931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.932491+0000 mon.a (mon.0) 3932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[30791]: audit 2021-03-10T17:40:53.934147+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: cluster 2021-03-10T17:40:52.524327+0000 mgr.gibba032.whopeo (mgr.24625) 12008 : cluster [DBG] pgmap v6848: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.921510+0000 mon.a (mon.0) 3927 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:40:54.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.924574+0000 mon.a (mon.0) 3928 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:40:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.926399+0000 mon.a (mon.0) 3929 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.927875+0000 mon.a (mon.0) 3930 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.929237+0000 mon.a (mon.0) 3931 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.932491+0000 mon.a (mon.0) 3932 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:54.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:53 gibba011 bash[31774]: audit 2021-03-10T17:40:53.934147+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:40:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.699171+0000 mgr.gibba032.whopeo (mgr.24625) 12009 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:40:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.700737+0000 mgr.gibba032.whopeo (mgr.24625) 12010 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:40:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.922274+0000 mgr.gibba032.whopeo (mgr.24625) 12011 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:40:55.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.924296+0000 mgr.gibba032.whopeo (mgr.24625) 12012 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:40:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.925771+0000 mgr.gibba032.whopeo (mgr.24625) 12013 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:40:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.925809+0000 mgr.gibba032.whopeo (mgr.24625) 12014 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:40:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.927284+0000 mgr.gibba032.whopeo (mgr.24625) 12015 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:40:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.927321+0000 mgr.gibba032.whopeo (mgr.24625) 12016 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:40:55.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.928721+0000 mgr.gibba032.whopeo (mgr.24625) 12017 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:40:55.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.930253+0000 mgr.gibba032.whopeo (mgr.24625) 12018 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:40:55.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.930291+0000 mgr.gibba032.whopeo (mgr.24625) 12019 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:40:55.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.931812+0000 mgr.gibba032.whopeo (mgr.24625) 12020 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:40:55.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.932180+0000 mgr.gibba032.whopeo (mgr.24625) 12021 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:40:55.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.933536+0000 mgr.gibba032.whopeo (mgr.24625) 12022 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:40:55.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.933575+0000 mgr.gibba032.whopeo (mgr.24625) 12023 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:40:55.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:54 gibba032 bash[24004]: cephadm 2021-03-10T17:40:53.933680+0000 mgr.gibba032.whopeo (mgr.24625) 12024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.699171+0000 mgr.gibba032.whopeo (mgr.24625) 12009 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:40:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.700737+0000 mgr.gibba032.whopeo (mgr.24625) 12010 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:40:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.922274+0000 mgr.gibba032.whopeo (mgr.24625) 12011 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:40:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.924296+0000 mgr.gibba032.whopeo (mgr.24625) 12012 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:40:55.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.925771+0000 mgr.gibba032.whopeo (mgr.24625) 12013 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:40:55.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.925809+0000 mgr.gibba032.whopeo (mgr.24625) 12014 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:40:55.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.927284+0000 mgr.gibba032.whopeo (mgr.24625) 12015 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:40:55.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.927321+0000 mgr.gibba032.whopeo (mgr.24625) 12016 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:40:55.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.928721+0000 mgr.gibba032.whopeo (mgr.24625) 12017 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:40:55.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.930253+0000 mgr.gibba032.whopeo (mgr.24625) 12018 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:40:55.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.930291+0000 mgr.gibba032.whopeo (mgr.24625) 12019 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:40:55.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.931812+0000 mgr.gibba032.whopeo (mgr.24625) 12020 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:40:55.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.932180+0000 mgr.gibba032.whopeo (mgr.24625) 12021 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:40:55.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.933536+0000 mgr.gibba032.whopeo (mgr.24625) 12022 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:40:55.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.933575+0000 mgr.gibba032.whopeo (mgr.24625) 12023 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:40:55.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[30791]: cephadm 2021-03-10T17:40:53.933680+0000 mgr.gibba032.whopeo (mgr.24625) 12024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:55.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.699171+0000 mgr.gibba032.whopeo (mgr.24625) 12009 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:40:55.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.700737+0000 mgr.gibba032.whopeo (mgr.24625) 12010 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:40:55.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.922274+0000 mgr.gibba032.whopeo (mgr.24625) 12011 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:40:55.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.924296+0000 mgr.gibba032.whopeo (mgr.24625) 12012 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:40:55.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.925771+0000 mgr.gibba032.whopeo (mgr.24625) 12013 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:40:55.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.925809+0000 mgr.gibba032.whopeo (mgr.24625) 12014 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:40:55.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.927284+0000 mgr.gibba032.whopeo (mgr.24625) 12015 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:40:55.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.927321+0000 mgr.gibba032.whopeo (mgr.24625) 12016 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:40:55.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.928721+0000 mgr.gibba032.whopeo (mgr.24625) 12017 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:40:55.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.930253+0000 mgr.gibba032.whopeo (mgr.24625) 12018 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:40:55.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.930291+0000 mgr.gibba032.whopeo (mgr.24625) 12019 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:40:55.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.931812+0000 mgr.gibba032.whopeo (mgr.24625) 12020 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:40:55.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.932180+0000 mgr.gibba032.whopeo (mgr.24625) 12021 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:40:55.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.933536+0000 mgr.gibba032.whopeo (mgr.24625) 12022 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:40:55.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.933575+0000 mgr.gibba032.whopeo (mgr.24625) 12023 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:40:55.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:54 gibba011 bash[31774]: cephadm 2021-03-10T17:40:53.933680+0000 mgr.gibba032.whopeo (mgr.24625) 12024 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:40:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:56 gibba032 bash[24004]: cluster 2021-03-10T17:40:54.525720+0000 mgr.gibba032.whopeo (mgr.24625) 12025 : cluster [DBG] pgmap v6849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:56.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:56 gibba011 bash[30791]: cluster 2021-03-10T17:40:54.525720+0000 mgr.gibba032.whopeo (mgr.24625) 12025 : cluster [DBG] pgmap v6849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:56.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:56 gibba011 bash[31774]: cluster 2021-03-10T17:40:54.525720+0000 mgr.gibba032.whopeo (mgr.24625) 12025 : cluster [DBG] pgmap v6849: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: cluster 2021-03-10T17:40:56.527211+0000 mgr.gibba032.whopeo (mgr.24625) 12026 : cluster [DBG] pgmap v6850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.588724+0000 mon.a (mon.0) 3934 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:40:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.589400+0000 mon.a (mon.0) 3935 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:40:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.809981+0000 mon.a (mon.0) 3936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:40:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.810745+0000 mon.a (mon.0) 3937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:40:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.811474+0000 mon.a (mon.0) 3938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:40:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[30791]: audit 2021-03-10T17:40:57.812154+0000 mon.a (mon.0) 3939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:40:58.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: cluster 2021-03-10T17:40:56.527211+0000 mgr.gibba032.whopeo (mgr.24625) 12026 : cluster [DBG] pgmap v6850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.588724+0000 mon.a (mon.0) 3934 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:40:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.589400+0000 mon.a (mon.0) 3935 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:40:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.809981+0000 mon.a (mon.0) 3936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:40:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.810745+0000 mon.a (mon.0) 3937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:40:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.811474+0000 mon.a (mon.0) 3938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:40:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:40:58 gibba011 bash[31774]: audit 2021-03-10T17:40:57.812154+0000 mon.a (mon.0) 3939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:40:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: cluster 2021-03-10T17:40:56.527211+0000 mgr.gibba032.whopeo (mgr.24625) 12026 : cluster [DBG] pgmap v6850: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:40:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.588724+0000 mon.a (mon.0) 3934 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:40:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.589400+0000 mon.a (mon.0) 3935 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:40:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.809981+0000 mon.a (mon.0) 3936 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2021-03-10T17:40:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.810745+0000 mon.a (mon.0) 3937 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2021-03-10T17:40:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.811474+0000 mon.a (mon.0) 3938 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:40:58.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:40:58 gibba032 bash[24004]: audit 2021-03-10T17:40:57.812154+0000 mon.a (mon.0) 3939 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:41:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:00 gibba011 bash[30791]: cluster 2021-03-10T17:40:58.528033+0000 mgr.gibba032.whopeo (mgr.24625) 12027 : cluster [DBG] pgmap v6851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:00 gibba011 bash[31774]: cluster 2021-03-10T17:40:58.528033+0000 mgr.gibba032.whopeo (mgr.24625) 12027 : cluster [DBG] pgmap v6851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:00 gibba032 bash[24004]: cluster 2021-03-10T17:40:58.528033+0000 mgr.gibba032.whopeo (mgr.24625) 12027 : cluster [DBG] pgmap v6851: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:02 gibba011 bash[30791]: cluster 2021-03-10T17:41:00.529184+0000 mgr.gibba032.whopeo (mgr.24625) 12028 : cluster [DBG] pgmap v6852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:02 gibba011 bash[31774]: cluster 2021-03-10T17:41:00.529184+0000 mgr.gibba032.whopeo (mgr.24625) 12028 : cluster [DBG] pgmap v6852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:02.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:02 gibba032 bash[24004]: cluster 2021-03-10T17:41:00.529184+0000 mgr.gibba032.whopeo (mgr.24625) 12028 : cluster [DBG] pgmap v6852: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:04.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:04 gibba011 bash[30791]: cluster 2021-03-10T17:41:02.530795+0000 mgr.gibba032.whopeo (mgr.24625) 12029 : cluster [DBG] pgmap v6853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:04.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:04 gibba011 bash[31774]: cluster 2021-03-10T17:41:02.530795+0000 mgr.gibba032.whopeo (mgr.24625) 12029 : cluster [DBG] pgmap v6853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:04 gibba032 bash[24004]: cluster 2021-03-10T17:41:02.530795+0000 mgr.gibba032.whopeo (mgr.24625) 12029 : cluster [DBG] pgmap v6853: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:06 gibba011 bash[30791]: cluster 2021-03-10T17:41:04.531633+0000 mgr.gibba032.whopeo (mgr.24625) 12030 : cluster [DBG] pgmap v6854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:06.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:06 gibba011 bash[31774]: cluster 2021-03-10T17:41:04.531633+0000 mgr.gibba032.whopeo (mgr.24625) 12030 : cluster [DBG] pgmap v6854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:06 gibba032 bash[24004]: cluster 2021-03-10T17:41:04.531633+0000 mgr.gibba032.whopeo (mgr.24625) 12030 : cluster [DBG] pgmap v6854: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:08.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:08 gibba011 bash[30791]: cluster 2021-03-10T17:41:06.533238+0000 mgr.gibba032.whopeo (mgr.24625) 12031 : cluster [DBG] pgmap v6855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:08 gibba011 bash[31774]: cluster 2021-03-10T17:41:06.533238+0000 mgr.gibba032.whopeo (mgr.24625) 12031 : cluster [DBG] pgmap v6855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:08 gibba032 bash[24004]: cluster 2021-03-10T17:41:06.533238+0000 mgr.gibba032.whopeo (mgr.24625) 12031 : cluster [DBG] pgmap v6855: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:10 gibba011 bash[30791]: cluster 2021-03-10T17:41:08.534716+0000 mgr.gibba032.whopeo (mgr.24625) 12032 : cluster [DBG] pgmap v6856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:10 gibba011 bash[30791]: cephadm 2021-03-10T17:41:08.942887+0000 mgr.gibba032.whopeo (mgr.24625) 12033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:10 gibba011 bash[31774]: cluster 2021-03-10T17:41:08.534716+0000 mgr.gibba032.whopeo (mgr.24625) 12032 : cluster [DBG] pgmap v6856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:10.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:10 gibba011 bash[31774]: cephadm 2021-03-10T17:41:08.942887+0000 mgr.gibba032.whopeo (mgr.24625) 12033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:10 gibba032 bash[24004]: cluster 2021-03-10T17:41:08.534716+0000 mgr.gibba032.whopeo (mgr.24625) 12032 : cluster [DBG] pgmap v6856: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:10 gibba032 bash[24004]: cephadm 2021-03-10T17:41:08.942887+0000 mgr.gibba032.whopeo (mgr.24625) 12033 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:12 gibba011 bash[30791]: cluster 2021-03-10T17:41:10.535601+0000 mgr.gibba032.whopeo (mgr.24625) 12034 : cluster [DBG] pgmap v6857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:12 gibba011 bash[31774]: cluster 2021-03-10T17:41:10.535601+0000 mgr.gibba032.whopeo (mgr.24625) 12034 : cluster [DBG] pgmap v6857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:12 gibba032 bash[24004]: cluster 2021-03-10T17:41:10.535601+0000 mgr.gibba032.whopeo (mgr.24625) 12034 : cluster [DBG] pgmap v6857: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:14.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:14 gibba011 bash[30791]: cluster 2021-03-10T17:41:12.537215+0000 mgr.gibba032.whopeo (mgr.24625) 12035 : cluster [DBG] pgmap v6858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:14 gibba011 bash[31774]: cluster 2021-03-10T17:41:12.537215+0000 mgr.gibba032.whopeo (mgr.24625) 12035 : cluster [DBG] pgmap v6858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:14 gibba032 bash[24004]: cluster 2021-03-10T17:41:12.537215+0000 mgr.gibba032.whopeo (mgr.24625) 12035 : cluster [DBG] pgmap v6858: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:16.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:16 gibba011 bash[30791]: cluster 2021-03-10T17:41:14.538596+0000 mgr.gibba032.whopeo (mgr.24625) 12036 : cluster [DBG] pgmap v6859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:16 gibba011 bash[31774]: cluster 2021-03-10T17:41:14.538596+0000 mgr.gibba032.whopeo (mgr.24625) 12036 : cluster [DBG] pgmap v6859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:16.508 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:16 gibba032 bash[24004]: cluster 2021-03-10T17:41:14.538596+0000 mgr.gibba032.whopeo (mgr.24625) 12036 : cluster [DBG] pgmap v6859: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:17.708 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:41:17.934 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:41:17.934 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 9m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:41:17.934 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:41:17.935 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:41:17.935 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:41:17.935 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:41:17.935 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:41:17.936 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:41:17.936 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:41:17.936 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:41:17.936 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:41:17.936 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:41:17.937 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:41:17.937 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:41:17.937 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:41:17.937 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:41:17.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:41:17.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:41:17.938 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:41:17.938 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:41:17.938 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 9m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:41:18.196 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:41:18.196 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:41:18.196 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:41:18.197 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:18.197 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:41:18.197 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:41:18.197 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:18.197 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:41:18.198 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:41:18.198 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:18.198 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:41:18.198 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:41:18.199 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:41:18.199 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:41:18.199 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:41:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:18 gibba032 bash[24004]: cluster 2021-03-10T17:41:16.539560+0000 mgr.gibba032.whopeo (mgr.24625) 12037 : cluster [DBG] pgmap v6860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:18.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:18 gibba011 bash[30791]: cluster 2021-03-10T17:41:16.539560+0000 mgr.gibba032.whopeo (mgr.24625) 12037 : cluster [DBG] pgmap v6860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:18.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:18 gibba011 bash[31774]: cluster 2021-03-10T17:41:16.539560+0000 mgr.gibba032.whopeo (mgr.24625) 12037 : cluster [DBG] pgmap v6860: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:19 gibba032 bash[24004]: audit 2021-03-10T17:41:17.695835+0000 mgr.gibba032.whopeo (mgr.24625) 12038 : audit [DBG] from='client.62101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:19 gibba032 bash[24004]: audit 2021-03-10T17:41:17.918268+0000 mgr.gibba032.whopeo (mgr.24625) 12039 : audit [DBG] from='client.52125 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:19 gibba032 bash[24004]: audit 2021-03-10T17:41:18.194280+0000 mon.c (mon.1) 234 : audit [DBG] from='client.? 172.21.2.111:0/3070973459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:19.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[30791]: audit 2021-03-10T17:41:17.695835+0000 mgr.gibba032.whopeo (mgr.24625) 12038 : audit [DBG] from='client.62101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[30791]: audit 2021-03-10T17:41:17.918268+0000 mgr.gibba032.whopeo (mgr.24625) 12039 : audit [DBG] from='client.52125 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[30791]: audit 2021-03-10T17:41:18.194280+0000 mon.c (mon.1) 234 : audit [DBG] from='client.? 172.21.2.111:0/3070973459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:19.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[31774]: audit 2021-03-10T17:41:17.695835+0000 mgr.gibba032.whopeo (mgr.24625) 12038 : audit [DBG] from='client.62101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[31774]: audit 2021-03-10T17:41:17.918268+0000 mgr.gibba032.whopeo (mgr.24625) 12039 : audit [DBG] from='client.52125 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:19.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:19 gibba011 bash[31774]: audit 2021-03-10T17:41:18.194280+0000 mon.c (mon.1) 234 : audit [DBG] from='client.? 172.21.2.111:0/3070973459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:20 gibba032 bash[24004]: cluster 2021-03-10T17:41:18.541020+0000 mgr.gibba032.whopeo (mgr.24625) 12040 : cluster [DBG] pgmap v6861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:20.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:20 gibba011 bash[30791]: cluster 2021-03-10T17:41:18.541020+0000 mgr.gibba032.whopeo (mgr.24625) 12040 : cluster [DBG] pgmap v6861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:20.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:20 gibba011 bash[31774]: cluster 2021-03-10T17:41:18.541020+0000 mgr.gibba032.whopeo (mgr.24625) 12040 : cluster [DBG] pgmap v6861: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:22 gibba032 bash[24004]: cluster 2021-03-10T17:41:20.542599+0000 mgr.gibba032.whopeo (mgr.24625) 12041 : cluster [DBG] pgmap v6862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:22 gibba011 bash[30791]: cluster 2021-03-10T17:41:20.542599+0000 mgr.gibba032.whopeo (mgr.24625) 12041 : cluster [DBG] pgmap v6862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:22 gibba011 bash[31774]: cluster 2021-03-10T17:41:20.542599+0000 mgr.gibba032.whopeo (mgr.24625) 12041 : cluster [DBG] pgmap v6862: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:24 gibba032 bash[24004]: cluster 2021-03-10T17:41:22.543847+0000 mgr.gibba032.whopeo (mgr.24625) 12042 : cluster [DBG] pgmap v6863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:24 gibba011 bash[30791]: cluster 2021-03-10T17:41:22.543847+0000 mgr.gibba032.whopeo (mgr.24625) 12042 : cluster [DBG] pgmap v6863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:24 gibba011 bash[31774]: cluster 2021-03-10T17:41:22.543847+0000 mgr.gibba032.whopeo (mgr.24625) 12042 : cluster [DBG] pgmap v6863: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:25.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:25 gibba032 bash[24004]: cephadm 2021-03-10T17:41:23.958941+0000 mgr.gibba032.whopeo (mgr.24625) 12043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:25.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:25 gibba011 bash[30791]: cephadm 2021-03-10T17:41:23.958941+0000 mgr.gibba032.whopeo (mgr.24625) 12043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:25.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:25 gibba011 bash[31774]: cephadm 2021-03-10T17:41:23.958941+0000 mgr.gibba032.whopeo (mgr.24625) 12043 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:26 gibba032 bash[24004]: cluster 2021-03-10T17:41:24.545073+0000 mgr.gibba032.whopeo (mgr.24625) 12044 : cluster [DBG] pgmap v6864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:26 gibba011 bash[30791]: cluster 2021-03-10T17:41:24.545073+0000 mgr.gibba032.whopeo (mgr.24625) 12044 : cluster [DBG] pgmap v6864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:26 gibba011 bash[31774]: cluster 2021-03-10T17:41:24.545073+0000 mgr.gibba032.whopeo (mgr.24625) 12044 : cluster [DBG] pgmap v6864: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:28 gibba032 bash[24004]: cluster 2021-03-10T17:41:26.546856+0000 mgr.gibba032.whopeo (mgr.24625) 12045 : cluster [DBG] pgmap v6865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:28 gibba011 bash[30791]: cluster 2021-03-10T17:41:26.546856+0000 mgr.gibba032.whopeo (mgr.24625) 12045 : cluster [DBG] pgmap v6865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:28.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:28 gibba011 bash[31774]: cluster 2021-03-10T17:41:26.546856+0000 mgr.gibba032.whopeo (mgr.24625) 12045 : cluster [DBG] pgmap v6865: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:30 gibba032 bash[24004]: cluster 2021-03-10T17:41:28.548350+0000 mgr.gibba032.whopeo (mgr.24625) 12046 : cluster [DBG] pgmap v6866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:30 gibba011 bash[30791]: cluster 2021-03-10T17:41:28.548350+0000 mgr.gibba032.whopeo (mgr.24625) 12046 : cluster [DBG] pgmap v6866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:30 gibba011 bash[31774]: cluster 2021-03-10T17:41:28.548350+0000 mgr.gibba032.whopeo (mgr.24625) 12046 : cluster [DBG] pgmap v6866: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:32 gibba032 bash[24004]: cluster 2021-03-10T17:41:30.549731+0000 mgr.gibba032.whopeo (mgr.24625) 12047 : cluster [DBG] pgmap v6867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:32 gibba011 bash[30791]: cluster 2021-03-10T17:41:30.549731+0000 mgr.gibba032.whopeo (mgr.24625) 12047 : cluster [DBG] pgmap v6867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:32 gibba011 bash[31774]: cluster 2021-03-10T17:41:30.549731+0000 mgr.gibba032.whopeo (mgr.24625) 12047 : cluster [DBG] pgmap v6867: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:34 gibba032 bash[24004]: cluster 2021-03-10T17:41:32.551386+0000 mgr.gibba032.whopeo (mgr.24625) 12048 : cluster [DBG] pgmap v6868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:34 gibba011 bash[30791]: cluster 2021-03-10T17:41:32.551386+0000 mgr.gibba032.whopeo (mgr.24625) 12048 : cluster [DBG] pgmap v6868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:34 gibba011 bash[31774]: cluster 2021-03-10T17:41:32.551386+0000 mgr.gibba032.whopeo (mgr.24625) 12048 : cluster [DBG] pgmap v6868: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:36 gibba032 bash[24004]: cluster 2021-03-10T17:41:34.552334+0000 mgr.gibba032.whopeo (mgr.24625) 12049 : cluster [DBG] pgmap v6869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:36 gibba011 bash[30791]: cluster 2021-03-10T17:41:34.552334+0000 mgr.gibba032.whopeo (mgr.24625) 12049 : cluster [DBG] pgmap v6869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:36 gibba011 bash[31774]: cluster 2021-03-10T17:41:34.552334+0000 mgr.gibba032.whopeo (mgr.24625) 12049 : cluster [DBG] pgmap v6869: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:38 gibba032 bash[24004]: cluster 2021-03-10T17:41:36.553952+0000 mgr.gibba032.whopeo (mgr.24625) 12050 : cluster [DBG] pgmap v6870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:38.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:38 gibba011 bash[30791]: cluster 2021-03-10T17:41:36.553952+0000 mgr.gibba032.whopeo (mgr.24625) 12050 : cluster [DBG] pgmap v6870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:38 gibba011 bash[31774]: cluster 2021-03-10T17:41:36.553952+0000 mgr.gibba032.whopeo (mgr.24625) 12050 : cluster [DBG] pgmap v6870: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:40 gibba032 bash[24004]: cluster 2021-03-10T17:41:38.555491+0000 mgr.gibba032.whopeo (mgr.24625) 12051 : cluster [DBG] pgmap v6871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:40 gibba032 bash[24004]: cephadm 2021-03-10T17:41:38.969925+0000 mgr.gibba032.whopeo (mgr.24625) 12052 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:40 gibba011 bash[30791]: cluster 2021-03-10T17:41:38.555491+0000 mgr.gibba032.whopeo (mgr.24625) 12051 : cluster [DBG] pgmap v6871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:40 gibba011 bash[30791]: cephadm 2021-03-10T17:41:38.969925+0000 mgr.gibba032.whopeo (mgr.24625) 12052 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:40 gibba011 bash[31774]: cluster 2021-03-10T17:41:38.555491+0000 mgr.gibba032.whopeo (mgr.24625) 12051 : cluster [DBG] pgmap v6871: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:40 gibba011 bash[31774]: cephadm 2021-03-10T17:41:38.969925+0000 mgr.gibba032.whopeo (mgr.24625) 12052 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:42 gibba032 bash[24004]: cluster 2021-03-10T17:41:40.556377+0000 mgr.gibba032.whopeo (mgr.24625) 12053 : cluster [DBG] pgmap v6872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:42.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:42 gibba011 bash[30791]: cluster 2021-03-10T17:41:40.556377+0000 mgr.gibba032.whopeo (mgr.24625) 12053 : cluster [DBG] pgmap v6872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:42 gibba011 bash[31774]: cluster 2021-03-10T17:41:40.556377+0000 mgr.gibba032.whopeo (mgr.24625) 12053 : cluster [DBG] pgmap v6872: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:44 gibba032 bash[24004]: cluster 2021-03-10T17:41:42.557849+0000 mgr.gibba032.whopeo (mgr.24625) 12054 : cluster [DBG] pgmap v6873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:44 gibba011 bash[30791]: cluster 2021-03-10T17:41:42.557849+0000 mgr.gibba032.whopeo (mgr.24625) 12054 : cluster [DBG] pgmap v6873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:44 gibba011 bash[31774]: cluster 2021-03-10T17:41:42.557849+0000 mgr.gibba032.whopeo (mgr.24625) 12054 : cluster [DBG] pgmap v6873: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:46 gibba032 bash[24004]: cluster 2021-03-10T17:41:44.559097+0000 mgr.gibba032.whopeo (mgr.24625) 12055 : cluster [DBG] pgmap v6874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:46.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:46 gibba011 bash[30791]: cluster 2021-03-10T17:41:44.559097+0000 mgr.gibba032.whopeo (mgr.24625) 12055 : cluster [DBG] pgmap v6874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:46 gibba011 bash[31774]: cluster 2021-03-10T17:41:44.559097+0000 mgr.gibba032.whopeo (mgr.24625) 12055 : cluster [DBG] pgmap v6874: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:48.444 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:41:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:48 gibba032 bash[24004]: cluster 2021-03-10T17:41:46.560303+0000 mgr.gibba032.whopeo (mgr.24625) 12056 : cluster [DBG] pgmap v6875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:48.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:48 gibba011 bash[30791]: cluster 2021-03-10T17:41:46.560303+0000 mgr.gibba032.whopeo (mgr.24625) 12056 : cluster [DBG] pgmap v6875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:48 gibba011 bash[31774]: cluster 2021-03-10T17:41:46.560303+0000 mgr.gibba032.whopeo (mgr.24625) 12056 : cluster [DBG] pgmap v6875: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:48.670 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:41:48.671 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 10m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:41:48.672 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 8m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:41:48.672 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:41:48.672 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:41:48.673 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:41:48.673 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:41:48.674 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:41:48.674 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:41:48.675 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 10m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:41:48.675 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 8m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:41:48.675 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:41:48.676 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:41:48.676 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:41:48.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 10m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:41:48.677 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:41:48.678 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:41:48.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:41:48.679 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 8m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:41:48.679 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 8m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:41:48.679 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 10m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:41:48.918 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:41:48.918 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:41:48.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:41:48.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:48.919 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:41:48.919 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:41:48.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:48.920 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:41:48.920 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:41:48.920 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:41:48.920 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:41:48.920 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:41:48.921 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:41:48.921 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:41:48.921 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:41:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:49 gibba032 bash[24004]: audit 2021-03-10T17:41:48.431556+0000 mgr.gibba032.whopeo (mgr.24625) 12057 : audit [DBG] from='client.62119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:49 gibba032 bash[24004]: audit 2021-03-10T17:41:48.916357+0000 mon.c (mon.1) 235 : audit [DBG] from='client.? 172.21.2.111:0/4068376570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:49 gibba011 bash[30791]: audit 2021-03-10T17:41:48.431556+0000 mgr.gibba032.whopeo (mgr.24625) 12057 : audit [DBG] from='client.62119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:49 gibba011 bash[30791]: audit 2021-03-10T17:41:48.916357+0000 mon.c (mon.1) 235 : audit [DBG] from='client.? 172.21.2.111:0/4068376570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:49 gibba011 bash[31774]: audit 2021-03-10T17:41:48.431556+0000 mgr.gibba032.whopeo (mgr.24625) 12057 : audit [DBG] from='client.62119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:49 gibba011 bash[31774]: audit 2021-03-10T17:41:48.916357+0000 mon.c (mon.1) 235 : audit [DBG] from='client.? 172.21.2.111:0/4068376570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:50 gibba032 bash[24004]: cluster 2021-03-10T17:41:48.561680+0000 mgr.gibba032.whopeo (mgr.24625) 12058 : cluster [DBG] pgmap v6876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:50 gibba032 bash[24004]: audit 2021-03-10T17:41:48.655338+0000 mgr.gibba032.whopeo (mgr.24625) 12059 : audit [DBG] from='client.52143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:50 gibba011 bash[30791]: cluster 2021-03-10T17:41:48.561680+0000 mgr.gibba032.whopeo (mgr.24625) 12058 : cluster [DBG] pgmap v6876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:50.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:50 gibba011 bash[30791]: audit 2021-03-10T17:41:48.655338+0000 mgr.gibba032.whopeo (mgr.24625) 12059 : audit [DBG] from='client.52143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:50.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:50 gibba011 bash[31774]: cluster 2021-03-10T17:41:48.561680+0000 mgr.gibba032.whopeo (mgr.24625) 12058 : cluster [DBG] pgmap v6876: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:50.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:50 gibba011 bash[31774]: audit 2021-03-10T17:41:48.655338+0000 mgr.gibba032.whopeo (mgr.24625) 12059 : audit [DBG] from='client.52143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:41:52.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:52 gibba011 bash[30791]: cluster 2021-03-10T17:41:50.563140+0000 mgr.gibba032.whopeo (mgr.24625) 12060 : cluster [DBG] pgmap v6877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:52.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:52 gibba011 bash[31774]: cluster 2021-03-10T17:41:50.563140+0000 mgr.gibba032.whopeo (mgr.24625) 12060 : cluster [DBG] pgmap v6877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:52 gibba032 bash[24004]: cluster 2021-03-10T17:41:50.563140+0000 mgr.gibba032.whopeo (mgr.24625) 12060 : cluster [DBG] pgmap v6877: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:54 gibba011 bash[30791]: cluster 2021-03-10T17:41:52.564828+0000 mgr.gibba032.whopeo (mgr.24625) 12061 : cluster [DBG] pgmap v6878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:54 gibba011 bash[30791]: audit 2021-03-10T17:41:53.989971+0000 mon.a (mon.0) 3940 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:41:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:54 gibba011 bash[31774]: cluster 2021-03-10T17:41:52.564828+0000 mgr.gibba032.whopeo (mgr.24625) 12061 : cluster [DBG] pgmap v6878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:54.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:54 gibba011 bash[31774]: audit 2021-03-10T17:41:53.989971+0000 mon.a (mon.0) 3940 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:41:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:54 gibba032 bash[24004]: cluster 2021-03-10T17:41:52.564828+0000 mgr.gibba032.whopeo (mgr.24625) 12061 : cluster [DBG] pgmap v6878: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:54 gibba032 bash[24004]: audit 2021-03-10T17:41:53.989971+0000 mon.a (mon.0) 3940 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:41:55.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:55 gibba011 bash[30791]: cephadm 2021-03-10T17:41:53.987603+0000 mgr.gibba032.whopeo (mgr.24625) 12062 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:41:55.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:55 gibba011 bash[31774]: cephadm 2021-03-10T17:41:53.987603+0000 mgr.gibba032.whopeo (mgr.24625) 12062 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:41:55.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:55 gibba032 bash[24004]: cephadm 2021-03-10T17:41:53.987603+0000 mgr.gibba032.whopeo (mgr.24625) 12062 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:41:56.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:56 gibba011 bash[30791]: cluster 2021-03-10T17:41:54.566069+0000 mgr.gibba032.whopeo (mgr.24625) 12063 : cluster [DBG] pgmap v6879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:56 gibba011 bash[31774]: cluster 2021-03-10T17:41:54.566069+0000 mgr.gibba032.whopeo (mgr.24625) 12063 : cluster [DBG] pgmap v6879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:56 gibba032 bash[24004]: cluster 2021-03-10T17:41:54.566069+0000 mgr.gibba032.whopeo (mgr.24625) 12063 : cluster [DBG] pgmap v6879: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: cluster 2021-03-10T17:41:56.567822+0000 mgr.gibba032.whopeo (mgr.24625) 12064 : cluster [DBG] pgmap v6880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.498735+0000 mon.a (mon.0) 3941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.595296+0000 mon.a (mon.0) 3942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.595439+0000 mon.a (mon.0) 3943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.695604+0000 mon.a (mon.0) 3944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.711546+0000 mon.a (mon.0) 3945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:41:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.716289+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.720718+0000 mon.a (mon.0) 3947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.724428+0000 mon.a (mon.0) 3948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.732294+0000 mon.a (mon.0) 3949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.737683+0000 mon.a (mon.0) 3950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.819140+0000 mon.a (mon.0) 3951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:41:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.820028+0000 mon.a (mon.0) 3952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:41:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.820799+0000 mon.a (mon.0) 3953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:41:58.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[30791]: audit 2021-03-10T17:41:57.821623+0000 mon.a (mon.0) 3954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:41:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: cluster 2021-03-10T17:41:56.567822+0000 mgr.gibba032.whopeo (mgr.24625) 12064 : cluster [DBG] pgmap v6880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.498735+0000 mon.a (mon.0) 3941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.595296+0000 mon.a (mon.0) 3942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:41:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.595439+0000 mon.a (mon.0) 3943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:41:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.695604+0000 mon.a (mon.0) 3944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.711546+0000 mon.a (mon.0) 3945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:41:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.716289+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.720718+0000 mon.a (mon.0) 3947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.724428+0000 mon.a (mon.0) 3948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.732294+0000 mon.a (mon.0) 3949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.737683+0000 mon.a (mon.0) 3950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.819140+0000 mon.a (mon.0) 3951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:41:58.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.820028+0000 mon.a (mon.0) 3952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:41:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.820799+0000 mon.a (mon.0) 3953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:41:58.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:58 gibba011 bash[31774]: audit 2021-03-10T17:41:57.821623+0000 mon.a (mon.0) 3954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:41:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: cluster 2021-03-10T17:41:56.567822+0000 mgr.gibba032.whopeo (mgr.24625) 12064 : cluster [DBG] pgmap v6880: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.498735+0000 mon.a (mon.0) 3941 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.595296+0000 mon.a (mon.0) 3942 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.595439+0000 mon.a (mon.0) 3943 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:41:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.695604+0000 mon.a (mon.0) 3944 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.711546+0000 mon.a (mon.0) 3945 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.716289+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.720718+0000 mon.a (mon.0) 3947 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.724428+0000 mon.a (mon.0) 3948 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.732294+0000 mon.a (mon.0) 3949 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.737683+0000 mon.a (mon.0) 3950 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:41:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.819140+0000 mon.a (mon.0) 3951 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [7, 4]}]: dispatch 2021-03-10T17:41:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.820028+0000 mon.a (mon.0) 3952 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2021-03-10T17:41:58.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.820799+0000 mon.a (mon.0) 3953 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 5]}]: dispatch 2021-03-10T17:41:58.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:58 gibba032 bash[24004]: audit 2021-03-10T17:41:57.821623+0000 mon.a (mon.0) 3954 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2021-03-10T17:41:59.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.497709+0000 mgr.gibba032.whopeo (mgr.24625) 12065 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.707088+0000 mgr.gibba032.whopeo (mgr.24625) 12066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.713117+0000 mgr.gibba032.whopeo (mgr.24625) 12067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.717094+0000 mgr.gibba032.whopeo (mgr.24625) 12068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:41:59.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.717582+0000 mgr.gibba032.whopeo (mgr.24625) 12069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.721363+0000 mgr.gibba032.whopeo (mgr.24625) 12070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.721982+0000 mgr.gibba032.whopeo (mgr.24625) 12071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.725020+0000 mgr.gibba032.whopeo (mgr.24625) 12072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:41:59.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.728383+0000 mgr.gibba032.whopeo (mgr.24625) 12073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:41:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.728942+0000 mgr.gibba032.whopeo (mgr.24625) 12074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:41:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.732998+0000 mgr.gibba032.whopeo (mgr.24625) 12075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:41:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.735100+0000 mgr.gibba032.whopeo (mgr.24625) 12076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:41:59.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.738184+0000 mgr.gibba032.whopeo (mgr.24625) 12077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:41:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.738464+0000 mgr.gibba032.whopeo (mgr.24625) 12078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:41:59.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[30791]: cephadm 2021-03-10T17:41:57.739062+0000 mgr.gibba032.whopeo (mgr.24625) 12079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.497709+0000 mgr.gibba032.whopeo (mgr.24625) 12065 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:41:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.707088+0000 mgr.gibba032.whopeo (mgr.24625) 12066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:41:59.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.713117+0000 mgr.gibba032.whopeo (mgr.24625) 12067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:41:59.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.717094+0000 mgr.gibba032.whopeo (mgr.24625) 12068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:41:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.717582+0000 mgr.gibba032.whopeo (mgr.24625) 12069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:41:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.721363+0000 mgr.gibba032.whopeo (mgr.24625) 12070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:41:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.721982+0000 mgr.gibba032.whopeo (mgr.24625) 12071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:41:59.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.725020+0000 mgr.gibba032.whopeo (mgr.24625) 12072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:41:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.728383+0000 mgr.gibba032.whopeo (mgr.24625) 12073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:41:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.728942+0000 mgr.gibba032.whopeo (mgr.24625) 12074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:41:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.732998+0000 mgr.gibba032.whopeo (mgr.24625) 12075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:41:59.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.735100+0000 mgr.gibba032.whopeo (mgr.24625) 12076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:41:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.738184+0000 mgr.gibba032.whopeo (mgr.24625) 12077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:41:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.738464+0000 mgr.gibba032.whopeo (mgr.24625) 12078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:41:59.590 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:41:59 gibba011 bash[31774]: cephadm 2021-03-10T17:41:57.739062+0000 mgr.gibba032.whopeo (mgr.24625) 12079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:41:59.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.497709+0000 mgr.gibba032.whopeo (mgr.24625) 12065 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.707088+0000 mgr.gibba032.whopeo (mgr.24625) 12066 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.713117+0000 mgr.gibba032.whopeo (mgr.24625) 12067 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.717094+0000 mgr.gibba032.whopeo (mgr.24625) 12068 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:41:59.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.717582+0000 mgr.gibba032.whopeo (mgr.24625) 12069 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.721363+0000 mgr.gibba032.whopeo (mgr.24625) 12070 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.721982+0000 mgr.gibba032.whopeo (mgr.24625) 12071 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.725020+0000 mgr.gibba032.whopeo (mgr.24625) 12072 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:41:59.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.728383+0000 mgr.gibba032.whopeo (mgr.24625) 12073 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.728942+0000 mgr.gibba032.whopeo (mgr.24625) 12074 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.732998+0000 mgr.gibba032.whopeo (mgr.24625) 12075 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.735100+0000 mgr.gibba032.whopeo (mgr.24625) 12076 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.738184+0000 mgr.gibba032.whopeo (mgr.24625) 12077 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:41:59.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.738464+0000 mgr.gibba032.whopeo (mgr.24625) 12078 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:41:59.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:41:59 gibba032 bash[24004]: cephadm 2021-03-10T17:41:57.739062+0000 mgr.gibba032.whopeo (mgr.24625) 12079 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:00.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:00 gibba011 bash[30791]: cluster 2021-03-10T17:41:58.569206+0000 mgr.gibba032.whopeo (mgr.24625) 12080 : cluster [DBG] pgmap v6881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:00 gibba011 bash[31774]: cluster 2021-03-10T17:41:58.569206+0000 mgr.gibba032.whopeo (mgr.24625) 12080 : cluster [DBG] pgmap v6881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:00 gibba032 bash[24004]: cluster 2021-03-10T17:41:58.569206+0000 mgr.gibba032.whopeo (mgr.24625) 12080 : cluster [DBG] pgmap v6881: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:02.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:02 gibba011 bash[30791]: cluster 2021-03-10T17:42:00.570745+0000 mgr.gibba032.whopeo (mgr.24625) 12081 : cluster [DBG] pgmap v6882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:02.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:02 gibba011 bash[31774]: cluster 2021-03-10T17:42:00.570745+0000 mgr.gibba032.whopeo (mgr.24625) 12081 : cluster [DBG] pgmap v6882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:02 gibba032 bash[24004]: cluster 2021-03-10T17:42:00.570745+0000 mgr.gibba032.whopeo (mgr.24625) 12081 : cluster [DBG] pgmap v6882: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:04 gibba011 bash[30791]: cluster 2021-03-10T17:42:02.572554+0000 mgr.gibba032.whopeo (mgr.24625) 12082 : cluster [DBG] pgmap v6883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:04 gibba011 bash[31774]: cluster 2021-03-10T17:42:02.572554+0000 mgr.gibba032.whopeo (mgr.24625) 12082 : cluster [DBG] pgmap v6883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:04 gibba032 bash[24004]: cluster 2021-03-10T17:42:02.572554+0000 mgr.gibba032.whopeo (mgr.24625) 12082 : cluster [DBG] pgmap v6883: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:06.758 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:06 gibba032 bash[24004]: cluster 2021-03-10T17:42:04.574010+0000 mgr.gibba032.whopeo (mgr.24625) 12083 : cluster [DBG] pgmap v6884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:06.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:06 gibba011 bash[30791]: cluster 2021-03-10T17:42:04.574010+0000 mgr.gibba032.whopeo (mgr.24625) 12083 : cluster [DBG] pgmap v6884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:06.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:06 gibba011 bash[31774]: cluster 2021-03-10T17:42:04.574010+0000 mgr.gibba032.whopeo (mgr.24625) 12083 : cluster [DBG] pgmap v6884: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:08 gibba032 bash[24004]: cluster 2021-03-10T17:42:06.575553+0000 mgr.gibba032.whopeo (mgr.24625) 12084 : cluster [DBG] pgmap v6885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:08 gibba011 bash[30791]: cluster 2021-03-10T17:42:06.575553+0000 mgr.gibba032.whopeo (mgr.24625) 12084 : cluster [DBG] pgmap v6885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:08 gibba011 bash[31774]: cluster 2021-03-10T17:42:06.575553+0000 mgr.gibba032.whopeo (mgr.24625) 12084 : cluster [DBG] pgmap v6885: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:10 gibba032 bash[24004]: cluster 2021-03-10T17:42:08.577157+0000 mgr.gibba032.whopeo (mgr.24625) 12085 : cluster [DBG] pgmap v6886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:10 gibba011 bash[30791]: cluster 2021-03-10T17:42:08.577157+0000 mgr.gibba032.whopeo (mgr.24625) 12085 : cluster [DBG] pgmap v6886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:10 gibba011 bash[31774]: cluster 2021-03-10T17:42:08.577157+0000 mgr.gibba032.whopeo (mgr.24625) 12085 : cluster [DBG] pgmap v6886: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:12.748 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:12 gibba032 bash[24004]: cluster 2021-03-10T17:42:10.578615+0000 mgr.gibba032.whopeo (mgr.24625) 12086 : cluster [DBG] pgmap v6887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:12 gibba011 bash[30791]: cluster 2021-03-10T17:42:10.578615+0000 mgr.gibba032.whopeo (mgr.24625) 12086 : cluster [DBG] pgmap v6887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:12.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:12 gibba011 bash[31774]: cluster 2021-03-10T17:42:10.578615+0000 mgr.gibba032.whopeo (mgr.24625) 12086 : cluster [DBG] pgmap v6887: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:14 gibba032 bash[24004]: cluster 2021-03-10T17:42:12.580213+0000 mgr.gibba032.whopeo (mgr.24625) 12087 : cluster [DBG] pgmap v6888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:14.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:14 gibba032 bash[24004]: cephadm 2021-03-10T17:42:12.744854+0000 mgr.gibba032.whopeo (mgr.24625) 12088 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:14 gibba011 bash[30791]: cluster 2021-03-10T17:42:12.580213+0000 mgr.gibba032.whopeo (mgr.24625) 12087 : cluster [DBG] pgmap v6888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:14 gibba011 bash[30791]: cephadm 2021-03-10T17:42:12.744854+0000 mgr.gibba032.whopeo (mgr.24625) 12088 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:14 gibba011 bash[31774]: cluster 2021-03-10T17:42:12.580213+0000 mgr.gibba032.whopeo (mgr.24625) 12087 : cluster [DBG] pgmap v6888: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:14.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:14 gibba011 bash[31774]: cephadm 2021-03-10T17:42:12.744854+0000 mgr.gibba032.whopeo (mgr.24625) 12088 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:16 gibba032 bash[24004]: cluster 2021-03-10T17:42:14.581487+0000 mgr.gibba032.whopeo (mgr.24625) 12089 : cluster [DBG] pgmap v6889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:16.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:16 gibba011 bash[30791]: cluster 2021-03-10T17:42:14.581487+0000 mgr.gibba032.whopeo (mgr.24625) 12089 : cluster [DBG] pgmap v6889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:16 gibba011 bash[31774]: cluster 2021-03-10T17:42:14.581487+0000 mgr.gibba032.whopeo (mgr.24625) 12089 : cluster [DBG] pgmap v6889: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:18 gibba032 bash[24004]: cluster 2021-03-10T17:42:16.583275+0000 mgr.gibba032.whopeo (mgr.24625) 12090 : cluster [DBG] pgmap v6890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:18 gibba011 bash[30791]: cluster 2021-03-10T17:42:16.583275+0000 mgr.gibba032.whopeo (mgr.24625) 12090 : cluster [DBG] pgmap v6890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:18 gibba011 bash[31774]: cluster 2021-03-10T17:42:16.583275+0000 mgr.gibba032.whopeo (mgr.24625) 12090 : cluster [DBG] pgmap v6890: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:19.166 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:42:19.392 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:42:19.393 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 21s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:42:19.393 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:42:19.393 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:42:19.394 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:42:19.394 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:42:19.394 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:42:19.394 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:42:19.395 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:42:19.395 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 21s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:42:19.395 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:42:19.395 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:42:19.395 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:42:19.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:42:19.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 21s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:42:19.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:42:19.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:42:19.396 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:42:19.397 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:42:19.397 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:42:19.397 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 21s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:42:19.646 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:42:19.647 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:42:19.647 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:42:19.647 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:19.647 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:42:19.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:42:19.648 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:19.648 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:42:19.648 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:42:19.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:19.649 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:42:19.649 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:42:19.649 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:42:19.649 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:42:19.650 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:42:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:20 gibba032 bash[24004]: cluster 2021-03-10T17:42:18.584902+0000 mgr.gibba032.whopeo (mgr.24625) 12091 : cluster [DBG] pgmap v6891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:20 gibba032 bash[24004]: audit 2021-03-10T17:42:19.153261+0000 mgr.gibba032.whopeo (mgr.24625) 12092 : audit [DBG] from='client.62137 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:20 gibba032 bash[24004]: audit 2021-03-10T17:42:19.377605+0000 mgr.gibba032.whopeo (mgr.24625) 12093 : audit [DBG] from='client.62143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:20 gibba032 bash[24004]: audit 2021-03-10T17:42:19.644800+0000 mon.a (mon.0) 3955 : audit [DBG] from='client.? 172.21.2.111:0/3619817750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[30791]: cluster 2021-03-10T17:42:18.584902+0000 mgr.gibba032.whopeo (mgr.24625) 12091 : cluster [DBG] pgmap v6891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[30791]: audit 2021-03-10T17:42:19.153261+0000 mgr.gibba032.whopeo (mgr.24625) 12092 : audit [DBG] from='client.62137 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[30791]: audit 2021-03-10T17:42:19.377605+0000 mgr.gibba032.whopeo (mgr.24625) 12093 : audit [DBG] from='client.62143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[30791]: audit 2021-03-10T17:42:19.644800+0000 mon.a (mon.0) 3955 : audit [DBG] from='client.? 172.21.2.111:0/3619817750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[31774]: cluster 2021-03-10T17:42:18.584902+0000 mgr.gibba032.whopeo (mgr.24625) 12091 : cluster [DBG] pgmap v6891: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[31774]: audit 2021-03-10T17:42:19.153261+0000 mgr.gibba032.whopeo (mgr.24625) 12092 : audit [DBG] from='client.62137 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[31774]: audit 2021-03-10T17:42:19.377605+0000 mgr.gibba032.whopeo (mgr.24625) 12093 : audit [DBG] from='client.62143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:20.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:20 gibba011 bash[31774]: audit 2021-03-10T17:42:19.644800+0000 mon.a (mon.0) 3955 : audit [DBG] from='client.? 172.21.2.111:0/3619817750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:22 gibba032 bash[24004]: cluster 2021-03-10T17:42:20.586261+0000 mgr.gibba032.whopeo (mgr.24625) 12094 : cluster [DBG] pgmap v6892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:22 gibba011 bash[30791]: cluster 2021-03-10T17:42:20.586261+0000 mgr.gibba032.whopeo (mgr.24625) 12094 : cluster [DBG] pgmap v6892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:22 gibba011 bash[31774]: cluster 2021-03-10T17:42:20.586261+0000 mgr.gibba032.whopeo (mgr.24625) 12094 : cluster [DBG] pgmap v6892: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:24.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:24 gibba032 bash[24004]: cluster 2021-03-10T17:42:22.587921+0000 mgr.gibba032.whopeo (mgr.24625) 12095 : cluster [DBG] pgmap v6893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:24 gibba011 bash[30791]: cluster 2021-03-10T17:42:22.587921+0000 mgr.gibba032.whopeo (mgr.24625) 12095 : cluster [DBG] pgmap v6893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:24 gibba011 bash[31774]: cluster 2021-03-10T17:42:22.587921+0000 mgr.gibba032.whopeo (mgr.24625) 12095 : cluster [DBG] pgmap v6893: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:26.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:26 gibba032 bash[24004]: cluster 2021-03-10T17:42:24.589435+0000 mgr.gibba032.whopeo (mgr.24625) 12096 : cluster [DBG] pgmap v6894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:26.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:26 gibba011 bash[30791]: cluster 2021-03-10T17:42:24.589435+0000 mgr.gibba032.whopeo (mgr.24625) 12096 : cluster [DBG] pgmap v6894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:26.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:26 gibba011 bash[31774]: cluster 2021-03-10T17:42:24.589435+0000 mgr.gibba032.whopeo (mgr.24625) 12096 : cluster [DBG] pgmap v6894: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:28.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:28 gibba032 bash[24004]: cluster 2021-03-10T17:42:26.591213+0000 mgr.gibba032.whopeo (mgr.24625) 12097 : cluster [DBG] pgmap v6895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:28.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:28 gibba011 bash[30791]: cluster 2021-03-10T17:42:26.591213+0000 mgr.gibba032.whopeo (mgr.24625) 12097 : cluster [DBG] pgmap v6895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:28.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:28 gibba011 bash[31774]: cluster 2021-03-10T17:42:26.591213+0000 mgr.gibba032.whopeo (mgr.24625) 12097 : cluster [DBG] pgmap v6895: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:29.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:29 gibba032 bash[24004]: cephadm 2021-03-10T17:42:27.752818+0000 mgr.gibba032.whopeo (mgr.24625) 12098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:29.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:29 gibba011 bash[30791]: cephadm 2021-03-10T17:42:27.752818+0000 mgr.gibba032.whopeo (mgr.24625) 12098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:29.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:29 gibba011 bash[31774]: cephadm 2021-03-10T17:42:27.752818+0000 mgr.gibba032.whopeo (mgr.24625) 12098 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:30.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:30 gibba032 bash[24004]: cluster 2021-03-10T17:42:28.592638+0000 mgr.gibba032.whopeo (mgr.24625) 12099 : cluster [DBG] pgmap v6896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:30.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:30 gibba011 bash[30791]: cluster 2021-03-10T17:42:28.592638+0000 mgr.gibba032.whopeo (mgr.24625) 12099 : cluster [DBG] pgmap v6896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:30.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:30 gibba011 bash[31774]: cluster 2021-03-10T17:42:28.592638+0000 mgr.gibba032.whopeo (mgr.24625) 12099 : cluster [DBG] pgmap v6896: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:32.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:32 gibba032 bash[24004]: cluster 2021-03-10T17:42:30.594224+0000 mgr.gibba032.whopeo (mgr.24625) 12100 : cluster [DBG] pgmap v6897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:32.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:32 gibba011 bash[30791]: cluster 2021-03-10T17:42:30.594224+0000 mgr.gibba032.whopeo (mgr.24625) 12100 : cluster [DBG] pgmap v6897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:32.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:32 gibba011 bash[31774]: cluster 2021-03-10T17:42:30.594224+0000 mgr.gibba032.whopeo (mgr.24625) 12100 : cluster [DBG] pgmap v6897: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:33.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:33 gibba011 bash[30791]: cluster 2021-03-10T17:42:32.595961+0000 mgr.gibba032.whopeo (mgr.24625) 12101 : cluster [DBG] pgmap v6898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:33.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:33 gibba011 bash[31774]: cluster 2021-03-10T17:42:32.595961+0000 mgr.gibba032.whopeo (mgr.24625) 12101 : cluster [DBG] pgmap v6898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:34.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:33 gibba032 bash[24004]: cluster 2021-03-10T17:42:32.595961+0000 mgr.gibba032.whopeo (mgr.24625) 12101 : cluster [DBG] pgmap v6898: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:36.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:35 gibba032 bash[24004]: cluster 2021-03-10T17:42:34.597464+0000 mgr.gibba032.whopeo (mgr.24625) 12102 : cluster [DBG] pgmap v6899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:36.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:35 gibba011 bash[30791]: cluster 2021-03-10T17:42:34.597464+0000 mgr.gibba032.whopeo (mgr.24625) 12102 : cluster [DBG] pgmap v6899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:36.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:35 gibba011 bash[31774]: cluster 2021-03-10T17:42:34.597464+0000 mgr.gibba032.whopeo (mgr.24625) 12102 : cluster [DBG] pgmap v6899: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:38.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:37 gibba032 bash[24004]: cluster 2021-03-10T17:42:36.599251+0000 mgr.gibba032.whopeo (mgr.24625) 12103 : cluster [DBG] pgmap v6900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:38.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:37 gibba011 bash[30791]: cluster 2021-03-10T17:42:36.599251+0000 mgr.gibba032.whopeo (mgr.24625) 12103 : cluster [DBG] pgmap v6900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:38.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:37 gibba011 bash[31774]: cluster 2021-03-10T17:42:36.599251+0000 mgr.gibba032.whopeo (mgr.24625) 12103 : cluster [DBG] pgmap v6900: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:40.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:39 gibba032 bash[24004]: cluster 2021-03-10T17:42:38.600910+0000 mgr.gibba032.whopeo (mgr.24625) 12104 : cluster [DBG] pgmap v6901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:40.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:39 gibba011 bash[30791]: cluster 2021-03-10T17:42:38.600910+0000 mgr.gibba032.whopeo (mgr.24625) 12104 : cluster [DBG] pgmap v6901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:40.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:39 gibba011 bash[31774]: cluster 2021-03-10T17:42:38.600910+0000 mgr.gibba032.whopeo (mgr.24625) 12104 : cluster [DBG] pgmap v6901: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:42.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:41 gibba032 bash[24004]: cluster 2021-03-10T17:42:40.602514+0000 mgr.gibba032.whopeo (mgr.24625) 12105 : cluster [DBG] pgmap v6902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:42.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:41 gibba011 bash[30791]: cluster 2021-03-10T17:42:40.602514+0000 mgr.gibba032.whopeo (mgr.24625) 12105 : cluster [DBG] pgmap v6902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:42.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:41 gibba011 bash[31774]: cluster 2021-03-10T17:42:40.602514+0000 mgr.gibba032.whopeo (mgr.24625) 12105 : cluster [DBG] pgmap v6902: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:44.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:43 gibba011 bash[30791]: cluster 2021-03-10T17:42:42.604348+0000 mgr.gibba032.whopeo (mgr.24625) 12106 : cluster [DBG] pgmap v6903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:44.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:43 gibba011 bash[30791]: cephadm 2021-03-10T17:42:42.760799+0000 mgr.gibba032.whopeo (mgr.24625) 12107 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:43 gibba011 bash[31774]: cluster 2021-03-10T17:42:42.604348+0000 mgr.gibba032.whopeo (mgr.24625) 12106 : cluster [DBG] pgmap v6903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:44.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:43 gibba011 bash[31774]: cephadm 2021-03-10T17:42:42.760799+0000 mgr.gibba032.whopeo (mgr.24625) 12107 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:43 gibba032 bash[24004]: cluster 2021-03-10T17:42:42.604348+0000 mgr.gibba032.whopeo (mgr.24625) 12106 : cluster [DBG] pgmap v6903: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:44.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:43 gibba032 bash[24004]: cephadm 2021-03-10T17:42:42.760799+0000 mgr.gibba032.whopeo (mgr.24625) 12107 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:42:46.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:45 gibba011 bash[30791]: cluster 2021-03-10T17:42:44.605894+0000 mgr.gibba032.whopeo (mgr.24625) 12108 : cluster [DBG] pgmap v6904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:46.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:45 gibba011 bash[31774]: cluster 2021-03-10T17:42:44.605894+0000 mgr.gibba032.whopeo (mgr.24625) 12108 : cluster [DBG] pgmap v6904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:46.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:45 gibba032 bash[24004]: cluster 2021-03-10T17:42:44.605894+0000 mgr.gibba032.whopeo (mgr.24625) 12108 : cluster [DBG] pgmap v6904: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:48.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:47 gibba011 bash[30791]: cluster 2021-03-10T17:42:46.607530+0000 mgr.gibba032.whopeo (mgr.24625) 12109 : cluster [DBG] pgmap v6905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:48.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:47 gibba011 bash[31774]: cluster 2021-03-10T17:42:46.607530+0000 mgr.gibba032.whopeo (mgr.24625) 12109 : cluster [DBG] pgmap v6905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:48.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:47 gibba032 bash[24004]: cluster 2021-03-10T17:42:46.607530+0000 mgr.gibba032.whopeo (mgr.24625) 12109 : cluster [DBG] pgmap v6905: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:49.894 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:42:50.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:49 gibba011 bash[30791]: cluster 2021-03-10T17:42:48.608932+0000 mgr.gibba032.whopeo (mgr.24625) 12110 : cluster [DBG] pgmap v6906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:50.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:49 gibba011 bash[31774]: cluster 2021-03-10T17:42:48.608932+0000 mgr.gibba032.whopeo (mgr.24625) 12110 : cluster [DBG] pgmap v6906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:50.123 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:42:50.123 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 52s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:42:50.124 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 9m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:42:50.124 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:42:50.124 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:42:50.124 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:42:50.125 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 52s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:42:50.125 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:42:50.125 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:42:50.125 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 52s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:42:50.125 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 9m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:42:50.126 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:42:50.126 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:42:50.126 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:42:50.126 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 52s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:42:50.127 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:42:50.127 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:42:50.127 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:42:50.127 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 9m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:42:50.127 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 9m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:42:50.128 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 52s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:42:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:49 gibba032 bash[24004]: cluster 2021-03-10T17:42:48.608932+0000 mgr.gibba032.whopeo (mgr.24625) 12110 : cluster [DBG] pgmap v6906: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:50.387 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:42:50.388 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:42:50.388 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:42:50.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:50.388 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:42:50.389 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:42:50.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:50.389 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:42:50.389 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:42:50.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:42:50.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:42:50.390 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:42:50.390 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:42:50.390 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:42:50.391 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:42:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[30791]: audit 2021-03-10T17:42:49.881232+0000 mgr.gibba032.whopeo (mgr.24625) 12111 : audit [DBG] from='client.62155 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[30791]: audit 2021-03-10T17:42:50.108232+0000 mgr.gibba032.whopeo (mgr.24625) 12112 : audit [DBG] from='client.62161 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[30791]: audit 2021-03-10T17:42:50.386004+0000 mon.c (mon.1) 236 : audit [DBG] from='client.? 172.21.2.111:0/3918247781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[31774]: audit 2021-03-10T17:42:49.881232+0000 mgr.gibba032.whopeo (mgr.24625) 12111 : audit [DBG] from='client.62155 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[31774]: audit 2021-03-10T17:42:50.108232+0000 mgr.gibba032.whopeo (mgr.24625) 12112 : audit [DBG] from='client.62161 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:50 gibba011 bash[31774]: audit 2021-03-10T17:42:50.386004+0000 mon.c (mon.1) 236 : audit [DBG] from='client.? 172.21.2.111:0/3918247781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:50 gibba032 bash[24004]: audit 2021-03-10T17:42:49.881232+0000 mgr.gibba032.whopeo (mgr.24625) 12111 : audit [DBG] from='client.62155 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:50 gibba032 bash[24004]: audit 2021-03-10T17:42:50.108232+0000 mgr.gibba032.whopeo (mgr.24625) 12112 : audit [DBG] from='client.62161 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:42:51.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:50 gibba032 bash[24004]: audit 2021-03-10T17:42:50.386004+0000 mon.c (mon.1) 236 : audit [DBG] from='client.? 172.21.2.111:0/3918247781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:42:52.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:51 gibba011 bash[30791]: cluster 2021-03-10T17:42:50.610496+0000 mgr.gibba032.whopeo (mgr.24625) 12113 : cluster [DBG] pgmap v6907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:52.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:51 gibba011 bash[31774]: cluster 2021-03-10T17:42:50.610496+0000 mgr.gibba032.whopeo (mgr.24625) 12113 : cluster [DBG] pgmap v6907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:51 gibba032 bash[24004]: cluster 2021-03-10T17:42:50.610496+0000 mgr.gibba032.whopeo (mgr.24625) 12113 : cluster [DBG] pgmap v6907: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:54.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:53 gibba011 bash[30791]: cluster 2021-03-10T17:42:52.612403+0000 mgr.gibba032.whopeo (mgr.24625) 12114 : cluster [DBG] pgmap v6908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:54.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:53 gibba011 bash[31774]: cluster 2021-03-10T17:42:52.612403+0000 mgr.gibba032.whopeo (mgr.24625) 12114 : cluster [DBG] pgmap v6908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:54.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:53 gibba032 bash[24004]: cluster 2021-03-10T17:42:52.612403+0000 mgr.gibba032.whopeo (mgr.24625) 12114 : cluster [DBG] pgmap v6908: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:56.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:55 gibba011 bash[30791]: cluster 2021-03-10T17:42:54.613959+0000 mgr.gibba032.whopeo (mgr.24625) 12115 : cluster [DBG] pgmap v6909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:56.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:55 gibba011 bash[31774]: cluster 2021-03-10T17:42:54.613959+0000 mgr.gibba032.whopeo (mgr.24625) 12115 : cluster [DBG] pgmap v6909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:55 gibba032 bash[24004]: cluster 2021-03-10T17:42:54.613959+0000 mgr.gibba032.whopeo (mgr.24625) 12115 : cluster [DBG] pgmap v6909: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[30791]: cluster 2021-03-10T17:42:56.615690+0000 mgr.gibba032.whopeo (mgr.24625) 12116 : cluster [DBG] pgmap v6910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[30791]: audit 2021-03-10T17:42:57.603873+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:42:58.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[30791]: audit 2021-03-10T17:42:57.604539+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:42:58.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[30791]: audit 2021-03-10T17:42:57.781247+0000 mon.a (mon.0) 3958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:42:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[31774]: cluster 2021-03-10T17:42:56.615690+0000 mgr.gibba032.whopeo (mgr.24625) 12116 : cluster [DBG] pgmap v6910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:58.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[31774]: audit 2021-03-10T17:42:57.603873+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:42:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[31774]: audit 2021-03-10T17:42:57.604539+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:42:58.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:57 gibba011 bash[31774]: audit 2021-03-10T17:42:57.781247+0000 mon.a (mon.0) 3958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:42:58.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:57 gibba032 bash[24004]: cluster 2021-03-10T17:42:56.615690+0000 mgr.gibba032.whopeo (mgr.24625) 12116 : cluster [DBG] pgmap v6910: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:42:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:57 gibba032 bash[24004]: audit 2021-03-10T17:42:57.603873+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:42:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:57 gibba032 bash[24004]: audit 2021-03-10T17:42:57.604539+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:42:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:57 gibba032 bash[24004]: audit 2021-03-10T17:42:57.781247+0000 mon.a (mon.0) 3958 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config get","who": "mon","key": "container_image"}]: dispatch 2021-03-10T17:42:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[30791]: cephadm 2021-03-10T17:42:57.777614+0000 mgr.gibba032.whopeo (mgr.24625) 12117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:42:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[30791]: audit 2021-03-10T17:42:57.825593+0000 mon.a (mon.0) 3959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:42:59.081 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[30791]: audit 2021-03-10T17:42:57.825882+0000 mon.a (mon.0) 3960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:42:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[30791]: audit 2021-03-10T17:42:57.825998+0000 mon.a (mon.0) 3961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:42:59.082 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[30791]: audit 2021-03-10T17:42:57.826120+0000 mon.a (mon.0) 3962 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:42:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[31774]: cephadm 2021-03-10T17:42:57.777614+0000 mgr.gibba032.whopeo (mgr.24625) 12117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:42:59.082 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[31774]: audit 2021-03-10T17:42:57.825593+0000 mon.a (mon.0) 3959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:42:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[31774]: audit 2021-03-10T17:42:57.825882+0000 mon.a (mon.0) 3960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:42:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[31774]: audit 2021-03-10T17:42:57.825998+0000 mon.a (mon.0) 3961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:42:59.083 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:58 gibba011 bash[31774]: audit 2021-03-10T17:42:57.826120+0000 mon.a (mon.0) 3962 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:42:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:58 gibba032 bash[24004]: cephadm 2021-03-10T17:42:57.777614+0000 mgr.gibba032.whopeo (mgr.24625) 12117 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:42:59.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:58 gibba032 bash[24004]: audit 2021-03-10T17:42:57.825593+0000 mon.a (mon.0) 3959 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2021-03-10T17:42:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:58 gibba032 bash[24004]: audit 2021-03-10T17:42:57.825882+0000 mon.a (mon.0) 3960 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:42:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:58 gibba032 bash[24004]: audit 2021-03-10T17:42:57.825998+0000 mon.a (mon.0) 3961 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2021-03-10T17:42:59.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:58 gibba032 bash[24004]: audit 2021-03-10T17:42:57.826120+0000 mon.a (mon.0) 3962 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2021-03-10T17:43:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:42:59 gibba032 bash[24004]: cluster 2021-03-10T17:42:58.617388+0000 mgr.gibba032.whopeo (mgr.24625) 12118 : cluster [DBG] pgmap v6911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:42:59 gibba011 bash[30791]: cluster 2021-03-10T17:42:58.617388+0000 mgr.gibba032.whopeo (mgr.24625) 12118 : cluster [DBG] pgmap v6911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:42:59 gibba011 bash[31774]: cluster 2021-03-10T17:42:58.617388+0000 mgr.gibba032.whopeo (mgr.24625) 12118 : cluster [DBG] pgmap v6911: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:02.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: cluster 2021-03-10T17:43:00.617908+0000 mgr.gibba032.whopeo (mgr.24625) 12119 : cluster [DBG] pgmap v6912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:02.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.516890+0000 mon.a (mon.0) 3963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.708121+0000 mon.a (mon.0) 3964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.711685+0000 mon.a (mon.0) 3965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:43:02.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.713243+0000 mon.a (mon.0) 3966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.714823+0000 mon.a (mon.0) 3967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.716424+0000 mon.a (mon.0) 3968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.719514+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.011 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:01 gibba032 bash[24004]: audit 2021-03-10T17:43:01.721339+0000 mon.a (mon.0) 3970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: cluster 2021-03-10T17:43:00.617908+0000 mgr.gibba032.whopeo (mgr.24625) 12119 : cluster [DBG] pgmap v6912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.516890+0000 mon.a (mon.0) 3963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.708121+0000 mon.a (mon.0) 3964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.711685+0000 mon.a (mon.0) 3965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:43:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.713243+0000 mon.a (mon.0) 3966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.714823+0000 mon.a (mon.0) 3967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.716424+0000 mon.a (mon.0) 3968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.719514+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[30791]: audit 2021-03-10T17:43:01.721339+0000 mon.a (mon.0) 3970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: cluster 2021-03-10T17:43:00.617908+0000 mgr.gibba032.whopeo (mgr.24625) 12119 : cluster [DBG] pgmap v6912: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.516890+0000 mon.a (mon.0) 3963 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.708121+0000 mon.a (mon.0) 3964 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:43:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.711685+0000 mon.a (mon.0) 3965 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:43:02.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.713243+0000 mon.a (mon.0) 3966 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.714823+0000 mon.a (mon.0) 3967 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.716424+0000 mon.a (mon.0) 3968 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.719514+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:02.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:01 gibba011 bash[31774]: audit 2021-03-10T17:43:01.721339+0000 mon.a (mon.0) 3970 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.516564+0000 mgr.gibba032.whopeo (mgr.24625) 12120 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:43:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.709336+0000 mgr.gibba032.whopeo (mgr.24625) 12121 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:43:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.711176+0000 mgr.gibba032.whopeo (mgr.24625) 12122 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:43:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.712778+0000 mgr.gibba032.whopeo (mgr.24625) 12123 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:43:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.712816+0000 mgr.gibba032.whopeo (mgr.24625) 12124 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:43:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.714429+0000 mgr.gibba032.whopeo (mgr.24625) 12125 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:43:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.714532+0000 mgr.gibba032.whopeo (mgr.24625) 12126 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:43:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.715732+0000 mgr.gibba032.whopeo (mgr.24625) 12127 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:43:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.717328+0000 mgr.gibba032.whopeo (mgr.24625) 12128 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:43:03.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.717483+0000 mgr.gibba032.whopeo (mgr.24625) 12129 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:43:03.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.718954+0000 mgr.gibba032.whopeo (mgr.24625) 12130 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:43:03.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.719356+0000 mgr.gibba032.whopeo (mgr.24625) 12131 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:43:03.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.720958+0000 mgr.gibba032.whopeo (mgr.24625) 12132 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:43:03.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.720993+0000 mgr.gibba032.whopeo (mgr.24625) 12133 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:43:03.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:02 gibba032 bash[24004]: cephadm 2021-03-10T17:43:01.721103+0000 mgr.gibba032.whopeo (mgr.24625) 12134 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.516564+0000 mgr.gibba032.whopeo (mgr.24625) 12120 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:43:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.709336+0000 mgr.gibba032.whopeo (mgr.24625) 12121 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:43:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.711176+0000 mgr.gibba032.whopeo (mgr.24625) 12122 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:43:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.712778+0000 mgr.gibba032.whopeo (mgr.24625) 12123 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:43:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.712816+0000 mgr.gibba032.whopeo (mgr.24625) 12124 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:43:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.714429+0000 mgr.gibba032.whopeo (mgr.24625) 12125 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:43:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.714532+0000 mgr.gibba032.whopeo (mgr.24625) 12126 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:43:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.715732+0000 mgr.gibba032.whopeo (mgr.24625) 12127 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:43:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.717328+0000 mgr.gibba032.whopeo (mgr.24625) 12128 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:43:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.717483+0000 mgr.gibba032.whopeo (mgr.24625) 12129 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:43:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.718954+0000 mgr.gibba032.whopeo (mgr.24625) 12130 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:43:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.719356+0000 mgr.gibba032.whopeo (mgr.24625) 12131 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:43:03.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.720958+0000 mgr.gibba032.whopeo (mgr.24625) 12132 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:43:03.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.720993+0000 mgr.gibba032.whopeo (mgr.24625) 12133 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:43:03.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[30791]: cephadm 2021-03-10T17:43:01.721103+0000 mgr.gibba032.whopeo (mgr.24625) 12134 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:03.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.516564+0000 mgr.gibba032.whopeo (mgr.24625) 12120 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:43:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.709336+0000 mgr.gibba032.whopeo (mgr.24625) 12121 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:43:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.711176+0000 mgr.gibba032.whopeo (mgr.24625) 12122 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:43:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.712778+0000 mgr.gibba032.whopeo (mgr.24625) 12123 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:43:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.712816+0000 mgr.gibba032.whopeo (mgr.24625) 12124 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:43:03.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.714429+0000 mgr.gibba032.whopeo (mgr.24625) 12125 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:43:03.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.714532+0000 mgr.gibba032.whopeo (mgr.24625) 12126 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:43:03.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.715732+0000 mgr.gibba032.whopeo (mgr.24625) 12127 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:43:03.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.717328+0000 mgr.gibba032.whopeo (mgr.24625) 12128 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:43:03.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.717483+0000 mgr.gibba032.whopeo (mgr.24625) 12129 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:43:03.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.718954+0000 mgr.gibba032.whopeo (mgr.24625) 12130 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:43:03.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.719356+0000 mgr.gibba032.whopeo (mgr.24625) 12131 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:43:03.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.720958+0000 mgr.gibba032.whopeo (mgr.24625) 12132 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:43:03.337 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.720993+0000 mgr.gibba032.whopeo (mgr.24625) 12133 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:43:03.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:02 gibba011 bash[31774]: cephadm 2021-03-10T17:43:01.721103+0000 mgr.gibba032.whopeo (mgr.24625) 12134 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:04.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:03 gibba032 bash[24004]: cluster 2021-03-10T17:43:02.619541+0000 mgr.gibba032.whopeo (mgr.24625) 12135 : cluster [DBG] pgmap v6913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:04.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:03 gibba011 bash[30791]: cluster 2021-03-10T17:43:02.619541+0000 mgr.gibba032.whopeo (mgr.24625) 12135 : cluster [DBG] pgmap v6913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:04.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:03 gibba011 bash[31774]: cluster 2021-03-10T17:43:02.619541+0000 mgr.gibba032.whopeo (mgr.24625) 12135 : cluster [DBG] pgmap v6913: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:06.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:05 gibba032 bash[24004]: cluster 2021-03-10T17:43:04.621134+0000 mgr.gibba032.whopeo (mgr.24625) 12136 : cluster [DBG] pgmap v6914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:05 gibba011 bash[30791]: cluster 2021-03-10T17:43:04.621134+0000 mgr.gibba032.whopeo (mgr.24625) 12136 : cluster [DBG] pgmap v6914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:06.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:05 gibba011 bash[31774]: cluster 2021-03-10T17:43:04.621134+0000 mgr.gibba032.whopeo (mgr.24625) 12136 : cluster [DBG] pgmap v6914: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:08.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:07 gibba032 bash[24004]: cluster 2021-03-10T17:43:06.622768+0000 mgr.gibba032.whopeo (mgr.24625) 12137 : cluster [DBG] pgmap v6915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:07 gibba011 bash[30791]: cluster 2021-03-10T17:43:06.622768+0000 mgr.gibba032.whopeo (mgr.24625) 12137 : cluster [DBG] pgmap v6915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:08.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:07 gibba011 bash[31774]: cluster 2021-03-10T17:43:06.622768+0000 mgr.gibba032.whopeo (mgr.24625) 12137 : cluster [DBG] pgmap v6915: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:10.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:09 gibba032 bash[24004]: cluster 2021-03-10T17:43:08.624360+0000 mgr.gibba032.whopeo (mgr.24625) 12138 : cluster [DBG] pgmap v6916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:09 gibba011 bash[30791]: cluster 2021-03-10T17:43:08.624360+0000 mgr.gibba032.whopeo (mgr.24625) 12138 : cluster [DBG] pgmap v6916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:09 gibba011 bash[31774]: cluster 2021-03-10T17:43:08.624360+0000 mgr.gibba032.whopeo (mgr.24625) 12138 : cluster [DBG] pgmap v6916: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:12.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:11 gibba032 bash[24004]: cluster 2021-03-10T17:43:10.625689+0000 mgr.gibba032.whopeo (mgr.24625) 12139 : cluster [DBG] pgmap v6917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:11 gibba011 bash[30791]: cluster 2021-03-10T17:43:10.625689+0000 mgr.gibba032.whopeo (mgr.24625) 12139 : cluster [DBG] pgmap v6917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:11 gibba011 bash[31774]: cluster 2021-03-10T17:43:10.625689+0000 mgr.gibba032.whopeo (mgr.24625) 12139 : cluster [DBG] pgmap v6917: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:14.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:13 gibba032 bash[24004]: cluster 2021-03-10T17:43:12.627323+0000 mgr.gibba032.whopeo (mgr.24625) 12140 : cluster [DBG] pgmap v6918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:14.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:13 gibba011 bash[30791]: cluster 2021-03-10T17:43:12.627323+0000 mgr.gibba032.whopeo (mgr.24625) 12140 : cluster [DBG] pgmap v6918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:14.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:13 gibba011 bash[31774]: cluster 2021-03-10T17:43:12.627323+0000 mgr.gibba032.whopeo (mgr.24625) 12140 : cluster [DBG] pgmap v6918: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:16.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:15 gibba032 bash[24004]: cluster 2021-03-10T17:43:14.628856+0000 mgr.gibba032.whopeo (mgr.24625) 12141 : cluster [DBG] pgmap v6919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:16.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:15 gibba011 bash[30791]: cluster 2021-03-10T17:43:14.628856+0000 mgr.gibba032.whopeo (mgr.24625) 12141 : cluster [DBG] pgmap v6919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:15 gibba011 bash[31774]: cluster 2021-03-10T17:43:14.628856+0000 mgr.gibba032.whopeo (mgr.24625) 12141 : cluster [DBG] pgmap v6919: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:17 gibba032 bash[24004]: cluster 2021-03-10T17:43:16.630394+0000 mgr.gibba032.whopeo (mgr.24625) 12142 : cluster [DBG] pgmap v6920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:18.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:17 gibba032 bash[24004]: cephadm 2021-03-10T17:43:16.728721+0000 mgr.gibba032.whopeo (mgr.24625) 12143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:17 gibba011 bash[30791]: cluster 2021-03-10T17:43:16.630394+0000 mgr.gibba032.whopeo (mgr.24625) 12142 : cluster [DBG] pgmap v6920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:17 gibba011 bash[30791]: cephadm 2021-03-10T17:43:16.728721+0000 mgr.gibba032.whopeo (mgr.24625) 12143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:17 gibba011 bash[31774]: cluster 2021-03-10T17:43:16.630394+0000 mgr.gibba032.whopeo (mgr.24625) 12142 : cluster [DBG] pgmap v6920: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:18.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:17 gibba011 bash[31774]: cephadm 2021-03-10T17:43:16.728721+0000 mgr.gibba032.whopeo (mgr.24625) 12143 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:20.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:19 gibba032 bash[24004]: cluster 2021-03-10T17:43:18.631909+0000 mgr.gibba032.whopeo (mgr.24625) 12144 : cluster [DBG] pgmap v6921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:19 gibba011 bash[30791]: cluster 2021-03-10T17:43:18.631909+0000 mgr.gibba032.whopeo (mgr.24625) 12144 : cluster [DBG] pgmap v6921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:19 gibba011 bash[31774]: cluster 2021-03-10T17:43:18.631909+0000 mgr.gibba032.whopeo (mgr.24625) 12144 : cluster [DBG] pgmap v6921: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:20.634 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:43:20.891 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:43:20.891 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 83s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:43:20.891 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 19s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:43:20.892 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:43:20.892 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:43:20.892 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 83s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:43:20.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 83s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:43:20.892 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:43:20.893 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 83s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:43:20.893 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 83s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:43:20.893 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 19s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:43:20.893 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 83s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:43:20.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 83s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:43:20.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 83s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:43:20.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 83s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:43:20.894 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:43:20.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:43:20.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:43:20.895 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 19s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:43:20.895 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 19s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:43:20.895 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 83s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:43:21.144 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:43:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:43:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:43:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:43:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:43:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:43:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:43:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:43:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:43:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:43:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:43:21.147 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:43:22.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:21 gibba032 bash[24004]: audit 2021-03-10T17:43:20.621606+0000 mgr.gibba032.whopeo (mgr.24625) 12145 : audit [DBG] from='client.52188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:21 gibba032 bash[24004]: cluster 2021-03-10T17:43:20.633291+0000 mgr.gibba032.whopeo (mgr.24625) 12146 : cluster [DBG] pgmap v6922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:21 gibba032 bash[24004]: audit 2021-03-10T17:43:20.846628+0000 mgr.gibba032.whopeo (mgr.24625) 12147 : audit [DBG] from='client.62179 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:21 gibba032 bash[24004]: audit 2021-03-10T17:43:21.142592+0000 mon.a (mon.0) 3971 : audit [DBG] from='client.? 172.21.2.111:0/625280458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[30791]: audit 2021-03-10T17:43:20.621606+0000 mgr.gibba032.whopeo (mgr.24625) 12145 : audit [DBG] from='client.52188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[30791]: cluster 2021-03-10T17:43:20.633291+0000 mgr.gibba032.whopeo (mgr.24625) 12146 : cluster [DBG] pgmap v6922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[30791]: audit 2021-03-10T17:43:20.846628+0000 mgr.gibba032.whopeo (mgr.24625) 12147 : audit [DBG] from='client.62179 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[30791]: audit 2021-03-10T17:43:21.142592+0000 mon.a (mon.0) 3971 : audit [DBG] from='client.? 172.21.2.111:0/625280458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:22.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[31774]: audit 2021-03-10T17:43:20.621606+0000 mgr.gibba032.whopeo (mgr.24625) 12145 : audit [DBG] from='client.52188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[31774]: cluster 2021-03-10T17:43:20.633291+0000 mgr.gibba032.whopeo (mgr.24625) 12146 : cluster [DBG] pgmap v6922: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:22.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[31774]: audit 2021-03-10T17:43:20.846628+0000 mgr.gibba032.whopeo (mgr.24625) 12147 : audit [DBG] from='client.62179 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:22.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:21 gibba011 bash[31774]: audit 2021-03-10T17:43:21.142592+0000 mon.a (mon.0) 3971 : audit [DBG] from='client.? 172.21.2.111:0/625280458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:24.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:23 gibba032 bash[24004]: cluster 2021-03-10T17:43:22.635093+0000 mgr.gibba032.whopeo (mgr.24625) 12148 : cluster [DBG] pgmap v6923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:24.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:23 gibba011 bash[30791]: cluster 2021-03-10T17:43:22.635093+0000 mgr.gibba032.whopeo (mgr.24625) 12148 : cluster [DBG] pgmap v6923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:24.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:23 gibba011 bash[31774]: cluster 2021-03-10T17:43:22.635093+0000 mgr.gibba032.whopeo (mgr.24625) 12148 : cluster [DBG] pgmap v6923: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:26.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:25 gibba032 bash[24004]: cluster 2021-03-10T17:43:24.636693+0000 mgr.gibba032.whopeo (mgr.24625) 12149 : cluster [DBG] pgmap v6924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:26.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:25 gibba011 bash[30791]: cluster 2021-03-10T17:43:24.636693+0000 mgr.gibba032.whopeo (mgr.24625) 12149 : cluster [DBG] pgmap v6924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:26.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:25 gibba011 bash[31774]: cluster 2021-03-10T17:43:24.636693+0000 mgr.gibba032.whopeo (mgr.24625) 12149 : cluster [DBG] pgmap v6924: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:27 gibba032 bash[24004]: cluster 2021-03-10T17:43:26.638446+0000 mgr.gibba032.whopeo (mgr.24625) 12150 : cluster [DBG] pgmap v6925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:28.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:27 gibba011 bash[30791]: cluster 2021-03-10T17:43:26.638446+0000 mgr.gibba032.whopeo (mgr.24625) 12150 : cluster [DBG] pgmap v6925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:28.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:27 gibba011 bash[31774]: cluster 2021-03-10T17:43:26.638446+0000 mgr.gibba032.whopeo (mgr.24625) 12150 : cluster [DBG] pgmap v6925: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:30.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:29 gibba032 bash[24004]: cluster 2021-03-10T17:43:28.639958+0000 mgr.gibba032.whopeo (mgr.24625) 12151 : cluster [DBG] pgmap v6926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:29 gibba011 bash[30791]: cluster 2021-03-10T17:43:28.639958+0000 mgr.gibba032.whopeo (mgr.24625) 12151 : cluster [DBG] pgmap v6926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:30.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:29 gibba011 bash[31774]: cluster 2021-03-10T17:43:28.639958+0000 mgr.gibba032.whopeo (mgr.24625) 12151 : cluster [DBG] pgmap v6926: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:32.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:31 gibba032 bash[24004]: cluster 2021-03-10T17:43:30.641381+0000 mgr.gibba032.whopeo (mgr.24625) 12152 : cluster [DBG] pgmap v6927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:31 gibba011 bash[30791]: cluster 2021-03-10T17:43:30.641381+0000 mgr.gibba032.whopeo (mgr.24625) 12152 : cluster [DBG] pgmap v6927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:31 gibba011 bash[31774]: cluster 2021-03-10T17:43:30.641381+0000 mgr.gibba032.whopeo (mgr.24625) 12152 : cluster [DBG] pgmap v6927: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:32 gibba032 bash[24004]: cephadm 2021-03-10T17:43:31.741995+0000 mgr.gibba032.whopeo (mgr.24625) 12153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:32 gibba011 bash[30791]: cephadm 2021-03-10T17:43:31.741995+0000 mgr.gibba032.whopeo (mgr.24625) 12153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:33 gibba011 bash[31774]: cephadm 2021-03-10T17:43:31.741995+0000 mgr.gibba032.whopeo (mgr.24625) 12153 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:34 gibba032 bash[24004]: cluster 2021-03-10T17:43:32.643145+0000 mgr.gibba032.whopeo (mgr.24625) 12154 : cluster [DBG] pgmap v6928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:34 gibba011 bash[30791]: cluster 2021-03-10T17:43:32.643145+0000 mgr.gibba032.whopeo (mgr.24625) 12154 : cluster [DBG] pgmap v6928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:34 gibba011 bash[31774]: cluster 2021-03-10T17:43:32.643145+0000 mgr.gibba032.whopeo (mgr.24625) 12154 : cluster [DBG] pgmap v6928: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:36.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:36 gibba011 bash[30791]: cluster 2021-03-10T17:43:34.644879+0000 mgr.gibba032.whopeo (mgr.24625) 12155 : cluster [DBG] pgmap v6929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:36 gibba011 bash[31774]: cluster 2021-03-10T17:43:34.644879+0000 mgr.gibba032.whopeo (mgr.24625) 12155 : cluster [DBG] pgmap v6929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:36 gibba032 bash[24004]: cluster 2021-03-10T17:43:34.644879+0000 mgr.gibba032.whopeo (mgr.24625) 12155 : cluster [DBG] pgmap v6929: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:38 gibba011 bash[30791]: cluster 2021-03-10T17:43:36.646808+0000 mgr.gibba032.whopeo (mgr.24625) 12156 : cluster [DBG] pgmap v6930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:38 gibba011 bash[31774]: cluster 2021-03-10T17:43:36.646808+0000 mgr.gibba032.whopeo (mgr.24625) 12156 : cluster [DBG] pgmap v6930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:38 gibba032 bash[24004]: cluster 2021-03-10T17:43:36.646808+0000 mgr.gibba032.whopeo (mgr.24625) 12156 : cluster [DBG] pgmap v6930: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:40 gibba011 bash[30791]: cluster 2021-03-10T17:43:38.648480+0000 mgr.gibba032.whopeo (mgr.24625) 12157 : cluster [DBG] pgmap v6931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:40 gibba011 bash[31774]: cluster 2021-03-10T17:43:38.648480+0000 mgr.gibba032.whopeo (mgr.24625) 12157 : cluster [DBG] pgmap v6931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:40 gibba032 bash[24004]: cluster 2021-03-10T17:43:38.648480+0000 mgr.gibba032.whopeo (mgr.24625) 12157 : cluster [DBG] pgmap v6931: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:42 gibba011 bash[30791]: cluster 2021-03-10T17:43:40.649789+0000 mgr.gibba032.whopeo (mgr.24625) 12158 : cluster [DBG] pgmap v6932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:42.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:42 gibba011 bash[31774]: cluster 2021-03-10T17:43:40.649789+0000 mgr.gibba032.whopeo (mgr.24625) 12158 : cluster [DBG] pgmap v6932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:42 gibba032 bash[24004]: cluster 2021-03-10T17:43:40.649789+0000 mgr.gibba032.whopeo (mgr.24625) 12158 : cluster [DBG] pgmap v6932: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:44 gibba011 bash[30791]: cluster 2021-03-10T17:43:42.651485+0000 mgr.gibba032.whopeo (mgr.24625) 12159 : cluster [DBG] pgmap v6933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:44 gibba011 bash[31774]: cluster 2021-03-10T17:43:42.651485+0000 mgr.gibba032.whopeo (mgr.24625) 12159 : cluster [DBG] pgmap v6933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:44 gibba032 bash[24004]: cluster 2021-03-10T17:43:42.651485+0000 mgr.gibba032.whopeo (mgr.24625) 12159 : cluster [DBG] pgmap v6933: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:46.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:46 gibba011 bash[30791]: cluster 2021-03-10T17:43:44.652957+0000 mgr.gibba032.whopeo (mgr.24625) 12160 : cluster [DBG] pgmap v6934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:46 gibba011 bash[31774]: cluster 2021-03-10T17:43:44.652957+0000 mgr.gibba032.whopeo (mgr.24625) 12160 : cluster [DBG] pgmap v6934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:46 gibba032 bash[24004]: cluster 2021-03-10T17:43:44.652957+0000 mgr.gibba032.whopeo (mgr.24625) 12160 : cluster [DBG] pgmap v6934: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:48 gibba011 bash[30791]: cluster 2021-03-10T17:43:46.654769+0000 mgr.gibba032.whopeo (mgr.24625) 12161 : cluster [DBG] pgmap v6935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:48 gibba011 bash[30791]: cephadm 2021-03-10T17:43:46.757896+0000 mgr.gibba032.whopeo (mgr.24625) 12162 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:48.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:48 gibba011 bash[31774]: cluster 2021-03-10T17:43:46.654769+0000 mgr.gibba032.whopeo (mgr.24625) 12161 : cluster [DBG] pgmap v6935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:48 gibba011 bash[31774]: cephadm 2021-03-10T17:43:46.757896+0000 mgr.gibba032.whopeo (mgr.24625) 12162 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:48 gibba032 bash[24004]: cluster 2021-03-10T17:43:46.654769+0000 mgr.gibba032.whopeo (mgr.24625) 12161 : cluster [DBG] pgmap v6935: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:48 gibba032 bash[24004]: cephadm 2021-03-10T17:43:46.757896+0000 mgr.gibba032.whopeo (mgr.24625) 12162 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:43:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:50 gibba011 bash[30791]: cluster 2021-03-10T17:43:48.656466+0000 mgr.gibba032.whopeo (mgr.24625) 12163 : cluster [DBG] pgmap v6936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:50 gibba011 bash[31774]: cluster 2021-03-10T17:43:48.656466+0000 mgr.gibba032.whopeo (mgr.24625) 12163 : cluster [DBG] pgmap v6936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:50.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:50 gibba032 bash[24004]: cluster 2021-03-10T17:43:48.656466+0000 mgr.gibba032.whopeo (mgr.24625) 12163 : cluster [DBG] pgmap v6936: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:51.388 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:43:51.615 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:43:51.615 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 114s ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:43:51.616 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 50s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:43:51.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:43:51.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:43:51.616 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 114s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:43:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 114s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:43:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:43:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 114s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:43:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 114s ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:43:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 50s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:43:51.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 114s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:43:51.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 114s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:43:51.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 114s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:43:51.618 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 114s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:43:51.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:43:51.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:43:51.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:43:51.619 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 50s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:43:51.619 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 50s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:43:51.620 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 114s ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:43:51.863 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:43:51.864 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:43:51.864 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:43:51.864 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:51.864 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:43:51.865 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:43:51.865 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:51.865 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:43:51.865 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:43:51.866 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:43:51.866 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:43:51.866 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:43:51.866 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:43:51.867 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:43:51.867 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:43:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[30791]: cluster 2021-03-10T17:43:50.656952+0000 mgr.gibba032.whopeo (mgr.24625) 12164 : cluster [DBG] pgmap v6937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[30791]: audit 2021-03-10T17:43:51.375527+0000 mgr.gibba032.whopeo (mgr.24625) 12165 : audit [DBG] from='client.62191 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[30791]: audit 2021-03-10T17:43:51.861976+0000 mon.a (mon.0) 3972 : audit [DBG] from='client.? 172.21.2.111:0/2331052763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[31774]: cluster 2021-03-10T17:43:50.656952+0000 mgr.gibba032.whopeo (mgr.24625) 12164 : cluster [DBG] pgmap v6937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[31774]: audit 2021-03-10T17:43:51.375527+0000 mgr.gibba032.whopeo (mgr.24625) 12165 : audit [DBG] from='client.62191 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:52.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:52 gibba011 bash[31774]: audit 2021-03-10T17:43:51.861976+0000 mon.a (mon.0) 3972 : audit [DBG] from='client.? 172.21.2.111:0/2331052763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:52 gibba032 bash[24004]: cluster 2021-03-10T17:43:50.656952+0000 mgr.gibba032.whopeo (mgr.24625) 12164 : cluster [DBG] pgmap v6937: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:52 gibba032 bash[24004]: audit 2021-03-10T17:43:51.375527+0000 mgr.gibba032.whopeo (mgr.24625) 12165 : audit [DBG] from='client.62191 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:52.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:52 gibba032 bash[24004]: audit 2021-03-10T17:43:51.861976+0000 mon.a (mon.0) 3972 : audit [DBG] from='client.? 172.21.2.111:0/2331052763' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:43:53.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:53 gibba032 bash[24004]: audit 2021-03-10T17:43:51.599733+0000 mgr.gibba032.whopeo (mgr.24625) 12166 : audit [DBG] from='client.52212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:53.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:53 gibba011 bash[30791]: audit 2021-03-10T17:43:51.599733+0000 mgr.gibba032.whopeo (mgr.24625) 12166 : audit [DBG] from='client.52212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:53.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:53 gibba011 bash[31774]: audit 2021-03-10T17:43:51.599733+0000 mgr.gibba032.whopeo (mgr.24625) 12166 : audit [DBG] from='client.52212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:43:54.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:54 gibba032 bash[24004]: cluster 2021-03-10T17:43:52.658563+0000 mgr.gibba032.whopeo (mgr.24625) 12167 : cluster [DBG] pgmap v6938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:54.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:54 gibba011 bash[30791]: cluster 2021-03-10T17:43:52.658563+0000 mgr.gibba032.whopeo (mgr.24625) 12167 : cluster [DBG] pgmap v6938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:54.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:54 gibba011 bash[31774]: cluster 2021-03-10T17:43:52.658563+0000 mgr.gibba032.whopeo (mgr.24625) 12167 : cluster [DBG] pgmap v6938: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:56.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:56 gibba032 bash[24004]: cluster 2021-03-10T17:43:54.660109+0000 mgr.gibba032.whopeo (mgr.24625) 12168 : cluster [DBG] pgmap v6939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:56.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:56 gibba011 bash[30791]: cluster 2021-03-10T17:43:54.660109+0000 mgr.gibba032.whopeo (mgr.24625) 12168 : cluster [DBG] pgmap v6939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:56.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:56 gibba011 bash[31774]: cluster 2021-03-10T17:43:54.660109+0000 mgr.gibba032.whopeo (mgr.24625) 12168 : cluster [DBG] pgmap v6939: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: cluster 2021-03-10T17:43:56.661914+0000 mgr.gibba032.whopeo (mgr.24625) 12169 : cluster [DBG] pgmap v6940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.619820+0000 mon.a (mon.0) 3973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:43:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.620440+0000 mon.a (mon.0) 3974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:43:58.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.833248+0000 mon.a (mon.0) 3975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:43:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.833999+0000 mon.a (mon.0) 3976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.834677+0000 mon.a (mon.0) 3977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:43:58 gibba032 bash[24004]: audit 2021-03-10T17:43:57.835400+0000 mon.a (mon.0) 3978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:43:58.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: cluster 2021-03-10T17:43:56.661914+0000 mgr.gibba032.whopeo (mgr.24625) 12169 : cluster [DBG] pgmap v6940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.619820+0000 mon.a (mon.0) 3973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:43:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.620440+0000 mon.a (mon.0) 3974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:43:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.833248+0000 mon.a (mon.0) 3975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:43:58.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.833999+0000 mon.a (mon.0) 3976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.834677+0000 mon.a (mon.0) 3977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[30791]: audit 2021-03-10T17:43:57.835400+0000 mon.a (mon.0) 3978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:43:58.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: cluster 2021-03-10T17:43:56.661914+0000 mgr.gibba032.whopeo (mgr.24625) 12169 : cluster [DBG] pgmap v6940: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:43:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.619820+0000 mon.a (mon.0) 3973 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:43:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.620440+0000 mon.a (mon.0) 3974 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:43:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.833248+0000 mon.a (mon.0) 3975 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2021-03-10T17:43:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.833999+0000 mon.a (mon.0) 3976 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.834677+0000 mon.a (mon.0) 3977 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2021-03-10T17:43:58.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:43:58 gibba011 bash[31774]: audit 2021-03-10T17:43:57.835400+0000 mon.a (mon.0) 3978 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:44:00.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:00 gibba032 bash[24004]: cluster 2021-03-10T17:43:58.663586+0000 mgr.gibba032.whopeo (mgr.24625) 12170 : cluster [DBG] pgmap v6941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:00.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:00 gibba011 bash[30791]: cluster 2021-03-10T17:43:58.663586+0000 mgr.gibba032.whopeo (mgr.24625) 12170 : cluster [DBG] pgmap v6941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:00.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:00 gibba011 bash[31774]: cluster 2021-03-10T17:43:58.663586+0000 mgr.gibba032.whopeo (mgr.24625) 12170 : cluster [DBG] pgmap v6941: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: cluster 2021-03-10T17:44:00.664090+0000 mgr.gibba032.whopeo (mgr.24625) 12171 : cluster [DBG] pgmap v6942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.981175+0000 mon.a (mon.0) 3979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:44:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.984414+0000 mon.a (mon.0) 3980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:44:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.985952+0000 mon.a (mon.0) 3981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.987534+0000 mon.a (mon.0) 3982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.989016+0000 mon.a (mon.0) 3983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.991497+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:02 gibba032 bash[24004]: audit 2021-03-10T17:44:01.993283+0000 mon.a (mon.0) 3985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: cluster 2021-03-10T17:44:00.664090+0000 mgr.gibba032.whopeo (mgr.24625) 12171 : cluster [DBG] pgmap v6942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.981175+0000 mon.a (mon.0) 3979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:44:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.984414+0000 mon.a (mon.0) 3980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:44:02.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.985952+0000 mon.a (mon.0) 3981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.987534+0000 mon.a (mon.0) 3982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.989016+0000 mon.a (mon.0) 3983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.991497+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[30791]: audit 2021-03-10T17:44:01.993283+0000 mon.a (mon.0) 3985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: cluster 2021-03-10T17:44:00.664090+0000 mgr.gibba032.whopeo (mgr.24625) 12171 : cluster [DBG] pgmap v6942: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.981175+0000 mon.a (mon.0) 3979 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:44:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.984414+0000 mon.a (mon.0) 3980 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:44:02.584 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.985952+0000 mon.a (mon.0) 3981 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.987534+0000 mon.a (mon.0) 3982 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.989016+0000 mon.a (mon.0) 3983 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.991497+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:02.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:02 gibba011 bash[31774]: audit 2021-03-10T17:44:01.993283+0000 mon.a (mon.0) 3985 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.774571+0000 mgr.gibba032.whopeo (mgr.24625) 12172 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:44:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.777458+0000 mgr.gibba032.whopeo (mgr.24625) 12173 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:44:03.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.982027+0000 mgr.gibba032.whopeo (mgr.24625) 12174 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:44:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.983992+0000 mgr.gibba032.whopeo (mgr.24625) 12175 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:44:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.985377+0000 mgr.gibba032.whopeo (mgr.24625) 12176 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:44:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.985425+0000 mgr.gibba032.whopeo (mgr.24625) 12177 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:44:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.987006+0000 mgr.gibba032.whopeo (mgr.24625) 12178 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:44:03.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.987052+0000 mgr.gibba032.whopeo (mgr.24625) 12179 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:44:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.988355+0000 mgr.gibba032.whopeo (mgr.24625) 12180 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:44:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.989442+0000 mgr.gibba032.whopeo (mgr.24625) 12181 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:44:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.989487+0000 mgr.gibba032.whopeo (mgr.24625) 12182 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:44:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.990984+0000 mgr.gibba032.whopeo (mgr.24625) 12183 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:44:03.512 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.991299+0000 mgr.gibba032.whopeo (mgr.24625) 12184 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:44:03.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.992771+0000 mgr.gibba032.whopeo (mgr.24625) 12185 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:44:03.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.992818+0000 mgr.gibba032.whopeo (mgr.24625) 12186 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:44:03.513 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:03 gibba032 bash[24004]: cephadm 2021-03-10T17:44:01.992934+0000 mgr.gibba032.whopeo (mgr.24625) 12187 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.774571+0000 mgr.gibba032.whopeo (mgr.24625) 12172 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:44:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.777458+0000 mgr.gibba032.whopeo (mgr.24625) 12173 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:44:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.982027+0000 mgr.gibba032.whopeo (mgr.24625) 12174 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:44:03.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.983992+0000 mgr.gibba032.whopeo (mgr.24625) 12175 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:44:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.985377+0000 mgr.gibba032.whopeo (mgr.24625) 12176 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:44:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.985425+0000 mgr.gibba032.whopeo (mgr.24625) 12177 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:44:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.987006+0000 mgr.gibba032.whopeo (mgr.24625) 12178 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:44:03.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.987052+0000 mgr.gibba032.whopeo (mgr.24625) 12179 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:44:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.988355+0000 mgr.gibba032.whopeo (mgr.24625) 12180 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:44:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.989442+0000 mgr.gibba032.whopeo (mgr.24625) 12181 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:44:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.989487+0000 mgr.gibba032.whopeo (mgr.24625) 12182 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:44:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.990984+0000 mgr.gibba032.whopeo (mgr.24625) 12183 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:44:03.584 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.991299+0000 mgr.gibba032.whopeo (mgr.24625) 12184 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:44:03.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.992771+0000 mgr.gibba032.whopeo (mgr.24625) 12185 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:44:03.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.992818+0000 mgr.gibba032.whopeo (mgr.24625) 12186 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:44:03.585 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[30791]: cephadm 2021-03-10T17:44:01.992934+0000 mgr.gibba032.whopeo (mgr.24625) 12187 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:03.585 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.774571+0000 mgr.gibba032.whopeo (mgr.24625) 12172 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:44:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.777458+0000 mgr.gibba032.whopeo (mgr.24625) 12173 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:44:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.982027+0000 mgr.gibba032.whopeo (mgr.24625) 12174 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:44:03.586 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.983992+0000 mgr.gibba032.whopeo (mgr.24625) 12175 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:44:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.985377+0000 mgr.gibba032.whopeo (mgr.24625) 12176 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:44:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.985425+0000 mgr.gibba032.whopeo (mgr.24625) 12177 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:44:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.987006+0000 mgr.gibba032.whopeo (mgr.24625) 12178 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:44:03.587 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.987052+0000 mgr.gibba032.whopeo (mgr.24625) 12179 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:44:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.988355+0000 mgr.gibba032.whopeo (mgr.24625) 12180 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:44:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.989442+0000 mgr.gibba032.whopeo (mgr.24625) 12181 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:44:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.989487+0000 mgr.gibba032.whopeo (mgr.24625) 12182 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:44:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.990984+0000 mgr.gibba032.whopeo (mgr.24625) 12183 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:44:03.588 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.991299+0000 mgr.gibba032.whopeo (mgr.24625) 12184 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:44:03.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.992771+0000 mgr.gibba032.whopeo (mgr.24625) 12185 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:44:03.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.992818+0000 mgr.gibba032.whopeo (mgr.24625) 12186 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:44:03.589 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:03 gibba011 bash[31774]: cephadm 2021-03-10T17:44:01.992934+0000 mgr.gibba032.whopeo (mgr.24625) 12187 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:04.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:04 gibba032 bash[24004]: cluster 2021-03-10T17:44:02.665585+0000 mgr.gibba032.whopeo (mgr.24625) 12188 : cluster [DBG] pgmap v6943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:04.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:04 gibba011 bash[30791]: cluster 2021-03-10T17:44:02.665585+0000 mgr.gibba032.whopeo (mgr.24625) 12188 : cluster [DBG] pgmap v6943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:04.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:04 gibba011 bash[31774]: cluster 2021-03-10T17:44:02.665585+0000 mgr.gibba032.whopeo (mgr.24625) 12188 : cluster [DBG] pgmap v6943: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:06 gibba032 bash[24004]: cluster 2021-03-10T17:44:04.667136+0000 mgr.gibba032.whopeo (mgr.24625) 12189 : cluster [DBG] pgmap v6944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:06.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:06 gibba011 bash[30791]: cluster 2021-03-10T17:44:04.667136+0000 mgr.gibba032.whopeo (mgr.24625) 12189 : cluster [DBG] pgmap v6944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:06.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:06 gibba011 bash[31774]: cluster 2021-03-10T17:44:04.667136+0000 mgr.gibba032.whopeo (mgr.24625) 12189 : cluster [DBG] pgmap v6944: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:08 gibba032 bash[24004]: cluster 2021-03-10T17:44:06.668987+0000 mgr.gibba032.whopeo (mgr.24625) 12190 : cluster [DBG] pgmap v6945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:08.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:08 gibba011 bash[30791]: cluster 2021-03-10T17:44:06.668987+0000 mgr.gibba032.whopeo (mgr.24625) 12190 : cluster [DBG] pgmap v6945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:08.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:08 gibba011 bash[31774]: cluster 2021-03-10T17:44:06.668987+0000 mgr.gibba032.whopeo (mgr.24625) 12190 : cluster [DBG] pgmap v6945: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:10 gibba032 bash[24004]: cluster 2021-03-10T17:44:08.669490+0000 mgr.gibba032.whopeo (mgr.24625) 12191 : cluster [DBG] pgmap v6946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:10.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:10 gibba011 bash[30791]: cluster 2021-03-10T17:44:08.669490+0000 mgr.gibba032.whopeo (mgr.24625) 12191 : cluster [DBG] pgmap v6946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:10.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:10 gibba011 bash[31774]: cluster 2021-03-10T17:44:08.669490+0000 mgr.gibba032.whopeo (mgr.24625) 12191 : cluster [DBG] pgmap v6946: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:12 gibba032 bash[24004]: cluster 2021-03-10T17:44:10.670705+0000 mgr.gibba032.whopeo (mgr.24625) 12192 : cluster [DBG] pgmap v6947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:12.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:12 gibba011 bash[30791]: cluster 2021-03-10T17:44:10.670705+0000 mgr.gibba032.whopeo (mgr.24625) 12192 : cluster [DBG] pgmap v6947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:12.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:12 gibba011 bash[31774]: cluster 2021-03-10T17:44:10.670705+0000 mgr.gibba032.whopeo (mgr.24625) 12192 : cluster [DBG] pgmap v6947: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:14 gibba032 bash[24004]: cluster 2021-03-10T17:44:12.672455+0000 mgr.gibba032.whopeo (mgr.24625) 12193 : cluster [DBG] pgmap v6948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:14.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:14 gibba011 bash[30791]: cluster 2021-03-10T17:44:12.672455+0000 mgr.gibba032.whopeo (mgr.24625) 12193 : cluster [DBG] pgmap v6948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:14.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:14 gibba011 bash[31774]: cluster 2021-03-10T17:44:12.672455+0000 mgr.gibba032.whopeo (mgr.24625) 12193 : cluster [DBG] pgmap v6948: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:16 gibba032 bash[24004]: cluster 2021-03-10T17:44:14.674002+0000 mgr.gibba032.whopeo (mgr.24625) 12194 : cluster [DBG] pgmap v6949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:16.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:16 gibba011 bash[30791]: cluster 2021-03-10T17:44:14.674002+0000 mgr.gibba032.whopeo (mgr.24625) 12194 : cluster [DBG] pgmap v6949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:16.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:16 gibba011 bash[31774]: cluster 2021-03-10T17:44:14.674002+0000 mgr.gibba032.whopeo (mgr.24625) 12194 : cluster [DBG] pgmap v6949: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:18 gibba032 bash[24004]: cluster 2021-03-10T17:44:16.675686+0000 mgr.gibba032.whopeo (mgr.24625) 12195 : cluster [DBG] pgmap v6950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:18 gibba032 bash[24004]: cephadm 2021-03-10T17:44:17.008914+0000 mgr.gibba032.whopeo (mgr.24625) 12196 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:18 gibba011 bash[30791]: cluster 2021-03-10T17:44:16.675686+0000 mgr.gibba032.whopeo (mgr.24625) 12195 : cluster [DBG] pgmap v6950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:18.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:18 gibba011 bash[30791]: cephadm 2021-03-10T17:44:17.008914+0000 mgr.gibba032.whopeo (mgr.24625) 12196 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:18 gibba011 bash[31774]: cluster 2021-03-10T17:44:16.675686+0000 mgr.gibba032.whopeo (mgr.24625) 12195 : cluster [DBG] pgmap v6950: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:18.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:18 gibba011 bash[31774]: cephadm 2021-03-10T17:44:17.008914+0000 mgr.gibba032.whopeo (mgr.24625) 12196 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:20.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:20 gibba032 bash[24004]: cluster 2021-03-10T17:44:18.677245+0000 mgr.gibba032.whopeo (mgr.24625) 12197 : cluster [DBG] pgmap v6951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:20.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:20 gibba011 bash[30791]: cluster 2021-03-10T17:44:18.677245+0000 mgr.gibba032.whopeo (mgr.24625) 12197 : cluster [DBG] pgmap v6951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:20.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:20 gibba011 bash[31774]: cluster 2021-03-10T17:44:18.677245+0000 mgr.gibba032.whopeo (mgr.24625) 12197 : cluster [DBG] pgmap v6951: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:22.109 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:44:22.341 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:44:22.342 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:44:22.342 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 80s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:44:22.342 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:44:22.343 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:44:22.343 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:44:22.343 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:44:22.344 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 80s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:44:22.344 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:44:22.344 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:44:22.344 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 80s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:44:22.345 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:44:22.345 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:44:22.345 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:44:22.346 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:44:22.346 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:44:22.346 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:44:22.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:44:22.347 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 80s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:44:22.347 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 80s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:44:22.347 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:44:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:22 gibba032 bash[24004]: cluster 2021-03-10T17:44:20.677802+0000 mgr.gibba032.whopeo (mgr.24625) 12198 : cluster [DBG] pgmap v6952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:22.580 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:22 gibba011 bash[30791]: cluster 2021-03-10T17:44:20.677802+0000 mgr.gibba032.whopeo (mgr.24625) 12198 : cluster [DBG] pgmap v6952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:22.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:22 gibba011 bash[31774]: cluster 2021-03-10T17:44:20.677802+0000 mgr.gibba032.whopeo (mgr.24625) 12198 : cluster [DBG] pgmap v6952: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:22.590 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:44:22.590 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:44:22.590 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:44:22.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:22.591 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:44:22.591 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:44:22.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:22.591 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:44:22.592 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:44:22.592 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:22.592 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:44:22.592 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:44:22.593 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:44:22.593 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:44:22.593 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:44:23.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:23 gibba032 bash[24004]: audit 2021-03-10T17:44:22.096997+0000 mgr.gibba032.whopeo (mgr.24625) 12199 : audit [DBG] from='client.62209 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:23 gibba032 bash[24004]: audit 2021-03-10T17:44:22.326070+0000 mgr.gibba032.whopeo (mgr.24625) 12200 : audit [DBG] from='client.62215 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:23 gibba032 bash[24004]: audit 2021-03-10T17:44:22.588167+0000 mon.c (mon.1) 237 : audit [DBG] from='client.? 172.21.2.111:0/3168062816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[30791]: audit 2021-03-10T17:44:22.096997+0000 mgr.gibba032.whopeo (mgr.24625) 12199 : audit [DBG] from='client.62209 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[30791]: audit 2021-03-10T17:44:22.326070+0000 mgr.gibba032.whopeo (mgr.24625) 12200 : audit [DBG] from='client.62215 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[30791]: audit 2021-03-10T17:44:22.588167+0000 mon.c (mon.1) 237 : audit [DBG] from='client.? 172.21.2.111:0/3168062816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[31774]: audit 2021-03-10T17:44:22.096997+0000 mgr.gibba032.whopeo (mgr.24625) 12199 : audit [DBG] from='client.62209 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[31774]: audit 2021-03-10T17:44:22.326070+0000 mgr.gibba032.whopeo (mgr.24625) 12200 : audit [DBG] from='client.62215 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:23.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:23 gibba011 bash[31774]: audit 2021-03-10T17:44:22.588167+0000 mon.c (mon.1) 237 : audit [DBG] from='client.? 172.21.2.111:0/3168062816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:24 gibba032 bash[24004]: cluster 2021-03-10T17:44:22.679228+0000 mgr.gibba032.whopeo (mgr.24625) 12201 : cluster [DBG] pgmap v6953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:24.583 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:24 gibba011 bash[30791]: cluster 2021-03-10T17:44:22.679228+0000 mgr.gibba032.whopeo (mgr.24625) 12201 : cluster [DBG] pgmap v6953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:24.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:24 gibba011 bash[31774]: cluster 2021-03-10T17:44:22.679228+0000 mgr.gibba032.whopeo (mgr.24625) 12201 : cluster [DBG] pgmap v6953: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:26.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:26 gibba032 bash[24004]: cluster 2021-03-10T17:44:24.680793+0000 mgr.gibba032.whopeo (mgr.24625) 12202 : cluster [DBG] pgmap v6954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:26 gibba011 bash[30791]: cluster 2021-03-10T17:44:24.680793+0000 mgr.gibba032.whopeo (mgr.24625) 12202 : cluster [DBG] pgmap v6954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:26.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:26 gibba011 bash[31774]: cluster 2021-03-10T17:44:24.680793+0000 mgr.gibba032.whopeo (mgr.24625) 12202 : cluster [DBG] pgmap v6954: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:28 gibba032 bash[24004]: cluster 2021-03-10T17:44:26.682502+0000 mgr.gibba032.whopeo (mgr.24625) 12203 : cluster [DBG] pgmap v6955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:28 gibba011 bash[30791]: cluster 2021-03-10T17:44:26.682502+0000 mgr.gibba032.whopeo (mgr.24625) 12203 : cluster [DBG] pgmap v6955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:28 gibba011 bash[31774]: cluster 2021-03-10T17:44:26.682502+0000 mgr.gibba032.whopeo (mgr.24625) 12203 : cluster [DBG] pgmap v6955: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:30 gibba032 bash[24004]: cluster 2021-03-10T17:44:28.684099+0000 mgr.gibba032.whopeo (mgr.24625) 12204 : cluster [DBG] pgmap v6956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:30 gibba011 bash[30791]: cluster 2021-03-10T17:44:28.684099+0000 mgr.gibba032.whopeo (mgr.24625) 12204 : cluster [DBG] pgmap v6956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:30.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:30 gibba011 bash[31774]: cluster 2021-03-10T17:44:28.684099+0000 mgr.gibba032.whopeo (mgr.24625) 12204 : cluster [DBG] pgmap v6956: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:32 gibba032 bash[24004]: cluster 2021-03-10T17:44:30.684630+0000 mgr.gibba032.whopeo (mgr.24625) 12205 : cluster [DBG] pgmap v6957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:32 gibba011 bash[30791]: cluster 2021-03-10T17:44:30.684630+0000 mgr.gibba032.whopeo (mgr.24625) 12205 : cluster [DBG] pgmap v6957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:32 gibba011 bash[31774]: cluster 2021-03-10T17:44:30.684630+0000 mgr.gibba032.whopeo (mgr.24625) 12205 : cluster [DBG] pgmap v6957: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:33.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:33 gibba011 bash[30791]: cephadm 2021-03-10T17:44:32.025041+0000 mgr.gibba032.whopeo (mgr.24625) 12206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:33.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:33 gibba011 bash[31774]: cephadm 2021-03-10T17:44:32.025041+0000 mgr.gibba032.whopeo (mgr.24625) 12206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:33.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:33 gibba032 bash[24004]: cephadm 2021-03-10T17:44:32.025041+0000 mgr.gibba032.whopeo (mgr.24625) 12206 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:34.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:34 gibba011 bash[30791]: cluster 2021-03-10T17:44:32.686133+0000 mgr.gibba032.whopeo (mgr.24625) 12207 : cluster [DBG] pgmap v6958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:34.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:34 gibba011 bash[31774]: cluster 2021-03-10T17:44:32.686133+0000 mgr.gibba032.whopeo (mgr.24625) 12207 : cluster [DBG] pgmap v6958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:34.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:34 gibba032 bash[24004]: cluster 2021-03-10T17:44:32.686133+0000 mgr.gibba032.whopeo (mgr.24625) 12207 : cluster [DBG] pgmap v6958: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:36 gibba011 bash[30791]: cluster 2021-03-10T17:44:34.687471+0000 mgr.gibba032.whopeo (mgr.24625) 12208 : cluster [DBG] pgmap v6959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:36 gibba011 bash[31774]: cluster 2021-03-10T17:44:34.687471+0000 mgr.gibba032.whopeo (mgr.24625) 12208 : cluster [DBG] pgmap v6959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:36.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:36 gibba032 bash[24004]: cluster 2021-03-10T17:44:34.687471+0000 mgr.gibba032.whopeo (mgr.24625) 12208 : cluster [DBG] pgmap v6959: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:38 gibba011 bash[30791]: cluster 2021-03-10T17:44:36.689122+0000 mgr.gibba032.whopeo (mgr.24625) 12209 : cluster [DBG] pgmap v6960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:38 gibba011 bash[31774]: cluster 2021-03-10T17:44:36.689122+0000 mgr.gibba032.whopeo (mgr.24625) 12209 : cluster [DBG] pgmap v6960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:38.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:38 gibba032 bash[24004]: cluster 2021-03-10T17:44:36.689122+0000 mgr.gibba032.whopeo (mgr.24625) 12209 : cluster [DBG] pgmap v6960: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:40 gibba011 bash[30791]: cluster 2021-03-10T17:44:38.690589+0000 mgr.gibba032.whopeo (mgr.24625) 12210 : cluster [DBG] pgmap v6961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:40.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:40 gibba011 bash[31774]: cluster 2021-03-10T17:44:38.690589+0000 mgr.gibba032.whopeo (mgr.24625) 12210 : cluster [DBG] pgmap v6961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:40.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:40 gibba032 bash[24004]: cluster 2021-03-10T17:44:38.690589+0000 mgr.gibba032.whopeo (mgr.24625) 12210 : cluster [DBG] pgmap v6961: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:42 gibba011 bash[30791]: cluster 2021-03-10T17:44:40.691136+0000 mgr.gibba032.whopeo (mgr.24625) 12211 : cluster [DBG] pgmap v6962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:42.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:42 gibba011 bash[31774]: cluster 2021-03-10T17:44:40.691136+0000 mgr.gibba032.whopeo (mgr.24625) 12211 : cluster [DBG] pgmap v6962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:42.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:42 gibba032 bash[24004]: cluster 2021-03-10T17:44:40.691136+0000 mgr.gibba032.whopeo (mgr.24625) 12211 : cluster [DBG] pgmap v6962: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:44 gibba011 bash[30791]: cluster 2021-03-10T17:44:42.692790+0000 mgr.gibba032.whopeo (mgr.24625) 12212 : cluster [DBG] pgmap v6963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:44 gibba011 bash[31774]: cluster 2021-03-10T17:44:42.692790+0000 mgr.gibba032.whopeo (mgr.24625) 12212 : cluster [DBG] pgmap v6963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:44.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:44 gibba032 bash[24004]: cluster 2021-03-10T17:44:42.692790+0000 mgr.gibba032.whopeo (mgr.24625) 12212 : cluster [DBG] pgmap v6963: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:46.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:46 gibba032 bash[24004]: cluster 2021-03-10T17:44:44.694063+0000 mgr.gibba032.whopeo (mgr.24625) 12213 : cluster [DBG] pgmap v6964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:46.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:46 gibba011 bash[30791]: cluster 2021-03-10T17:44:44.694063+0000 mgr.gibba032.whopeo (mgr.24625) 12213 : cluster [DBG] pgmap v6964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:46.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:46 gibba011 bash[31774]: cluster 2021-03-10T17:44:44.694063+0000 mgr.gibba032.whopeo (mgr.24625) 12213 : cluster [DBG] pgmap v6964: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:48 gibba032 bash[24004]: cluster 2021-03-10T17:44:46.695713+0000 mgr.gibba032.whopeo (mgr.24625) 12214 : cluster [DBG] pgmap v6965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:48.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:48 gibba032 bash[24004]: cephadm 2021-03-10T17:44:47.040934+0000 mgr.gibba032.whopeo (mgr.24625) 12215 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:48 gibba011 bash[30791]: cluster 2021-03-10T17:44:46.695713+0000 mgr.gibba032.whopeo (mgr.24625) 12214 : cluster [DBG] pgmap v6965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:48.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:48 gibba011 bash[30791]: cephadm 2021-03-10T17:44:47.040934+0000 mgr.gibba032.whopeo (mgr.24625) 12215 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:48 gibba011 bash[31774]: cluster 2021-03-10T17:44:46.695713+0000 mgr.gibba032.whopeo (mgr.24625) 12214 : cluster [DBG] pgmap v6965: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:48.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:48 gibba011 bash[31774]: cephadm 2021-03-10T17:44:47.040934+0000 mgr.gibba032.whopeo (mgr.24625) 12215 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:44:50.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:50 gibba032 bash[24004]: cluster 2021-03-10T17:44:48.697304+0000 mgr.gibba032.whopeo (mgr.24625) 12216 : cluster [DBG] pgmap v6966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:50.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:50 gibba011 bash[30791]: cluster 2021-03-10T17:44:48.697304+0000 mgr.gibba032.whopeo (mgr.24625) 12216 : cluster [DBG] pgmap v6966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:50.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:50 gibba011 bash[31774]: cluster 2021-03-10T17:44:48.697304+0000 mgr.gibba032.whopeo (mgr.24625) 12216 : cluster [DBG] pgmap v6966: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:52.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:52 gibba032 bash[24004]: cluster 2021-03-10T17:44:50.698675+0000 mgr.gibba032.whopeo (mgr.24625) 12217 : cluster [DBG] pgmap v6967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:52.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:52 gibba011 bash[30791]: cluster 2021-03-10T17:44:50.698675+0000 mgr.gibba032.whopeo (mgr.24625) 12217 : cluster [DBG] pgmap v6967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:52.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:52 gibba011 bash[31774]: cluster 2021-03-10T17:44:50.698675+0000 mgr.gibba032.whopeo (mgr.24625) 12217 : cluster [DBG] pgmap v6967: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:52.834 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:44:53.064 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:44:53.064 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 2m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:44:53.065 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 111s ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:44:53.065 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 111s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:44:53.065 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 111s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:44:53.065 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:44:53.066 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:44:53.066 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 111s ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:44:53.066 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:44:53.066 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:44:53.066 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 111s ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:44:53.067 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:44:53.067 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:44:53.067 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:44:53.067 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:44:53.068 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:44:53.068 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:44:53.068 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:44:53.068 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 111s ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:44:53.068 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 111s ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:44:53.069 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 2m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:44:53.341 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:44:53.341 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:44:53.341 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:44:53.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:53.342 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:44:53.342 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:44:53.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:53.342 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:44:53.343 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:44:53.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:44:53.343 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:44:53.343 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:44:53.344 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:44:53.344 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:44:53.344 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:44:53.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:53 gibba032 bash[24004]: audit 2021-03-10T17:44:53.339069+0000 mon.a (mon.0) 3986 : audit [DBG] from='client.? 172.21.2.111:0/2040234974' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:53.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:53 gibba011 bash[30791]: audit 2021-03-10T17:44:53.339069+0000 mon.a (mon.0) 3986 : audit [DBG] from='client.? 172.21.2.111:0/2040234974' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:53.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:53 gibba011 bash[31774]: audit 2021-03-10T17:44:53.339069+0000 mon.a (mon.0) 3986 : audit [DBG] from='client.? 172.21.2.111:0/2040234974' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:44:54.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:54 gibba032 bash[24004]: cluster 2021-03-10T17:44:52.700297+0000 mgr.gibba032.whopeo (mgr.24625) 12218 : cluster [DBG] pgmap v6968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:54 gibba032 bash[24004]: audit 2021-03-10T17:44:52.821735+0000 mgr.gibba032.whopeo (mgr.24625) 12219 : audit [DBG] from='client.62227 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:54.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:54 gibba032 bash[24004]: audit 2021-03-10T17:44:53.051490+0000 mgr.gibba032.whopeo (mgr.24625) 12220 : audit [DBG] from='client.62233 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[30791]: cluster 2021-03-10T17:44:52.700297+0000 mgr.gibba032.whopeo (mgr.24625) 12218 : cluster [DBG] pgmap v6968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:54.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[30791]: audit 2021-03-10T17:44:52.821735+0000 mgr.gibba032.whopeo (mgr.24625) 12219 : audit [DBG] from='client.62227 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:54.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[30791]: audit 2021-03-10T17:44:53.051490+0000 mgr.gibba032.whopeo (mgr.24625) 12220 : audit [DBG] from='client.62233 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[31774]: cluster 2021-03-10T17:44:52.700297+0000 mgr.gibba032.whopeo (mgr.24625) 12218 : cluster [DBG] pgmap v6968: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:54.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[31774]: audit 2021-03-10T17:44:52.821735+0000 mgr.gibba032.whopeo (mgr.24625) 12219 : audit [DBG] from='client.62227 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:54.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:54 gibba011 bash[31774]: audit 2021-03-10T17:44:53.051490+0000 mgr.gibba032.whopeo (mgr.24625) 12220 : audit [DBG] from='client.62233 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:44:56.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:56 gibba032 bash[24004]: cluster 2021-03-10T17:44:54.701825+0000 mgr.gibba032.whopeo (mgr.24625) 12221 : cluster [DBG] pgmap v6969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:56.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:56 gibba011 bash[30791]: cluster 2021-03-10T17:44:54.701825+0000 mgr.gibba032.whopeo (mgr.24625) 12221 : cluster [DBG] pgmap v6969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:56.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:56 gibba011 bash[31774]: cluster 2021-03-10T17:44:54.701825+0000 mgr.gibba032.whopeo (mgr.24625) 12221 : cluster [DBG] pgmap v6969: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:58.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: cluster 2021-03-10T17:44:56.703595+0000 mgr.gibba032.whopeo (mgr.24625) 12222 : cluster [DBG] pgmap v6970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.627031+0000 mon.a (mon.0) 3987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:44:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.627777+0000 mon.a (mon.0) 3988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:44:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.841913+0000 mon.a (mon.0) 3989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:44:58.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.842680+0000 mon.a (mon.0) 3990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.843306+0000 mon.a (mon.0) 3991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:44:58 gibba032 bash[24004]: audit 2021-03-10T17:44:57.844014+0000 mon.a (mon.0) 3992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T17:44:58.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: cluster 2021-03-10T17:44:56.703595+0000 mgr.gibba032.whopeo (mgr.24625) 12222 : cluster [DBG] pgmap v6970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.627031+0000 mon.a (mon.0) 3987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:44:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.627777+0000 mon.a (mon.0) 3988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:44:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.841913+0000 mon.a (mon.0) 3989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:44:58.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.842680+0000 mon.a (mon.0) 3990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.843306+0000 mon.a (mon.0) 3991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[30791]: audit 2021-03-10T17:44:57.844014+0000 mon.a (mon.0) 3992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T17:44:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: cluster 2021-03-10T17:44:56.703595+0000 mgr.gibba032.whopeo (mgr.24625) 12222 : cluster [DBG] pgmap v6970: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:44:58.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.627031+0000 mon.a (mon.0) 3987 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:44:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.627777+0000 mon.a (mon.0) 3988 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:44:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.841913+0000 mon.a (mon.0) 3989 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2021-03-10T17:44:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.842680+0000 mon.a (mon.0) 3990 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.843306+0000 mon.a (mon.0) 3991 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2021-03-10T17:44:58.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:44:58 gibba011 bash[31774]: audit 2021-03-10T17:44:57.844014+0000 mon.a (mon.0) 3992 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2021-03-10T17:45:00.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:00 gibba032 bash[24004]: cluster 2021-03-10T17:44:58.705252+0000 mgr.gibba032.whopeo (mgr.24625) 12223 : cluster [DBG] pgmap v6971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:00.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:00 gibba011 bash[30791]: cluster 2021-03-10T17:44:58.705252+0000 mgr.gibba032.whopeo (mgr.24625) 12223 : cluster [DBG] pgmap v6971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:00.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:00 gibba011 bash[31774]: cluster 2021-03-10T17:44:58.705252+0000 mgr.gibba032.whopeo (mgr.24625) 12223 : cluster [DBG] pgmap v6971: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:02.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: cluster 2021-03-10T17:45:00.706282+0000 mgr.gibba032.whopeo (mgr.24625) 12224 : cluster [DBG] pgmap v6972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.260775+0000 mon.a (mon.0) 3993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:45:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.264158+0000 mon.a (mon.0) 3994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:45:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.265401+0000 mon.a (mon.0) 3995 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.266796+0000 mon.a (mon.0) 3996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.267880+0000 mon.a (mon.0) 3997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.270483+0000 mon.a (mon.0) 3998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:02 gibba032 bash[24004]: audit 2021-03-10T17:45:02.272086+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: cluster 2021-03-10T17:45:00.706282+0000 mgr.gibba032.whopeo (mgr.24625) 12224 : cluster [DBG] pgmap v6972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.260775+0000 mon.a (mon.0) 3993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:45:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.264158+0000 mon.a (mon.0) 3994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:45:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.265401+0000 mon.a (mon.0) 3995 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.266796+0000 mon.a (mon.0) 3996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.267880+0000 mon.a (mon.0) 3997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.270483+0000 mon.a (mon.0) 3998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[30791]: audit 2021-03-10T17:45:02.272086+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: cluster 2021-03-10T17:45:00.706282+0000 mgr.gibba032.whopeo (mgr.24625) 12224 : cluster [DBG] pgmap v6972: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.260775+0000 mon.a (mon.0) 3993 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:45:02.834 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.264158+0000 mon.a (mon.0) 3994 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:45:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.265401+0000 mon.a (mon.0) 3995 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.266796+0000 mon.a (mon.0) 3996 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.267880+0000 mon.a (mon.0) 3997 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.270483+0000 mon.a (mon.0) 3998 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:02.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:02 gibba011 bash[31774]: audit 2021-03-10T17:45:02.272086+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.057757+0000 mgr.gibba032.whopeo (mgr.24625) 12225 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:45:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.059311+0000 mgr.gibba032.whopeo (mgr.24625) 12226 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:45:03.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.261878+0000 mgr.gibba032.whopeo (mgr.24625) 12227 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:45:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.263483+0000 mgr.gibba032.whopeo (mgr.24625) 12228 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:45:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.264805+0000 mgr.gibba032.whopeo (mgr.24625) 12229 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:45:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.264881+0000 mgr.gibba032.whopeo (mgr.24625) 12230 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:45:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.265965+0000 mgr.gibba032.whopeo (mgr.24625) 12231 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:45:03.761 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.266044+0000 mgr.gibba032.whopeo (mgr.24625) 12232 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:45:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.267129+0000 mgr.gibba032.whopeo (mgr.24625) 12233 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:45:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.268322+0000 mgr.gibba032.whopeo (mgr.24625) 12234 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:45:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.268412+0000 mgr.gibba032.whopeo (mgr.24625) 12235 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:45:03.762 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.269786+0000 mgr.gibba032.whopeo (mgr.24625) 12236 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:45:03.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.270119+0000 mgr.gibba032.whopeo (mgr.24625) 12237 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:45:03.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.271372+0000 mgr.gibba032.whopeo (mgr.24625) 12238 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:45:03.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.271418+0000 mgr.gibba032.whopeo (mgr.24625) 12239 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:45:03.763 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:03 gibba032 bash[24004]: cephadm 2021-03-10T17:45:02.271532+0000 mgr.gibba032.whopeo (mgr.24625) 12240 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:03.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.057757+0000 mgr.gibba032.whopeo (mgr.24625) 12225 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:45:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.059311+0000 mgr.gibba032.whopeo (mgr.24625) 12226 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:45:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.261878+0000 mgr.gibba032.whopeo (mgr.24625) 12227 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:45:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.263483+0000 mgr.gibba032.whopeo (mgr.24625) 12228 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:45:03.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.264805+0000 mgr.gibba032.whopeo (mgr.24625) 12229 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:45:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.264881+0000 mgr.gibba032.whopeo (mgr.24625) 12230 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:45:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.265965+0000 mgr.gibba032.whopeo (mgr.24625) 12231 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:45:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.266044+0000 mgr.gibba032.whopeo (mgr.24625) 12232 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:45:03.833 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.267129+0000 mgr.gibba032.whopeo (mgr.24625) 12233 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:45:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.268322+0000 mgr.gibba032.whopeo (mgr.24625) 12234 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:45:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.268412+0000 mgr.gibba032.whopeo (mgr.24625) 12235 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:45:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.269786+0000 mgr.gibba032.whopeo (mgr.24625) 12236 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:45:03.834 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.270119+0000 mgr.gibba032.whopeo (mgr.24625) 12237 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:45:03.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.271372+0000 mgr.gibba032.whopeo (mgr.24625) 12238 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:45:03.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.271418+0000 mgr.gibba032.whopeo (mgr.24625) 12239 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:45:03.835 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[30791]: cephadm 2021-03-10T17:45:02.271532+0000 mgr.gibba032.whopeo (mgr.24625) 12240 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:03.835 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.057757+0000 mgr.gibba032.whopeo (mgr.24625) 12225 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:45:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.059311+0000 mgr.gibba032.whopeo (mgr.24625) 12226 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:45:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.261878+0000 mgr.gibba032.whopeo (mgr.24625) 12227 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:45:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.263483+0000 mgr.gibba032.whopeo (mgr.24625) 12228 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:45:03.836 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.264805+0000 mgr.gibba032.whopeo (mgr.24625) 12229 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:45:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.264881+0000 mgr.gibba032.whopeo (mgr.24625) 12230 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:45:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.265965+0000 mgr.gibba032.whopeo (mgr.24625) 12231 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:45:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.266044+0000 mgr.gibba032.whopeo (mgr.24625) 12232 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:45:03.837 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.267129+0000 mgr.gibba032.whopeo (mgr.24625) 12233 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:45:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.268322+0000 mgr.gibba032.whopeo (mgr.24625) 12234 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:45:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.268412+0000 mgr.gibba032.whopeo (mgr.24625) 12235 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:45:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.269786+0000 mgr.gibba032.whopeo (mgr.24625) 12236 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:45:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.270119+0000 mgr.gibba032.whopeo (mgr.24625) 12237 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:45:03.838 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.271372+0000 mgr.gibba032.whopeo (mgr.24625) 12238 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:45:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.271418+0000 mgr.gibba032.whopeo (mgr.24625) 12239 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:45:03.839 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:03 gibba011 bash[31774]: cephadm 2021-03-10T17:45:02.271532+0000 mgr.gibba032.whopeo (mgr.24625) 12240 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:04.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:04 gibba032 bash[24004]: cluster 2021-03-10T17:45:02.708129+0000 mgr.gibba032.whopeo (mgr.24625) 12241 : cluster [DBG] pgmap v6973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:04.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:04 gibba011 bash[30791]: cluster 2021-03-10T17:45:02.708129+0000 mgr.gibba032.whopeo (mgr.24625) 12241 : cluster [DBG] pgmap v6973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:04.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:04 gibba011 bash[31774]: cluster 2021-03-10T17:45:02.708129+0000 mgr.gibba032.whopeo (mgr.24625) 12241 : cluster [DBG] pgmap v6973: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:06.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:06 gibba032 bash[24004]: cluster 2021-03-10T17:45:04.709656+0000 mgr.gibba032.whopeo (mgr.24625) 12242 : cluster [DBG] pgmap v6974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:06.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:06 gibba011 bash[30791]: cluster 2021-03-10T17:45:04.709656+0000 mgr.gibba032.whopeo (mgr.24625) 12242 : cluster [DBG] pgmap v6974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:06.857 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:06 gibba011 bash[31774]: cluster 2021-03-10T17:45:04.709656+0000 mgr.gibba032.whopeo (mgr.24625) 12242 : cluster [DBG] pgmap v6974: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:08.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:08 gibba032 bash[24004]: cluster 2021-03-10T17:45:06.711283+0000 mgr.gibba032.whopeo (mgr.24625) 12243 : cluster [DBG] pgmap v6975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:08.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:08 gibba011 bash[30791]: cluster 2021-03-10T17:45:06.711283+0000 mgr.gibba032.whopeo (mgr.24625) 12243 : cluster [DBG] pgmap v6975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:08.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:08 gibba011 bash[31774]: cluster 2021-03-10T17:45:06.711283+0000 mgr.gibba032.whopeo (mgr.24625) 12243 : cluster [DBG] pgmap v6975: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:10.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:10 gibba032 bash[24004]: cluster 2021-03-10T17:45:08.712826+0000 mgr.gibba032.whopeo (mgr.24625) 12244 : cluster [DBG] pgmap v6976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:10.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:10 gibba011 bash[30791]: cluster 2021-03-10T17:45:08.712826+0000 mgr.gibba032.whopeo (mgr.24625) 12244 : cluster [DBG] pgmap v6976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:10.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:10 gibba011 bash[31774]: cluster 2021-03-10T17:45:08.712826+0000 mgr.gibba032.whopeo (mgr.24625) 12244 : cluster [DBG] pgmap v6976: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:12.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:12 gibba032 bash[24004]: cluster 2021-03-10T17:45:10.714189+0000 mgr.gibba032.whopeo (mgr.24625) 12245 : cluster [DBG] pgmap v6977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:12.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:12 gibba011 bash[30791]: cluster 2021-03-10T17:45:10.714189+0000 mgr.gibba032.whopeo (mgr.24625) 12245 : cluster [DBG] pgmap v6977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:12.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:12 gibba011 bash[31774]: cluster 2021-03-10T17:45:10.714189+0000 mgr.gibba032.whopeo (mgr.24625) 12245 : cluster [DBG] pgmap v6977: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:14.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:14 gibba032 bash[24004]: cluster 2021-03-10T17:45:12.715984+0000 mgr.gibba032.whopeo (mgr.24625) 12246 : cluster [DBG] pgmap v6978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:14.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:14 gibba011 bash[30791]: cluster 2021-03-10T17:45:12.715984+0000 mgr.gibba032.whopeo (mgr.24625) 12246 : cluster [DBG] pgmap v6978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:14.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:14 gibba011 bash[31774]: cluster 2021-03-10T17:45:12.715984+0000 mgr.gibba032.whopeo (mgr.24625) 12246 : cluster [DBG] pgmap v6978: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:16.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:16 gibba032 bash[24004]: cluster 2021-03-10T17:45:14.717462+0000 mgr.gibba032.whopeo (mgr.24625) 12247 : cluster [DBG] pgmap v6979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:16.830 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:16 gibba011 bash[30791]: cluster 2021-03-10T17:45:14.717462+0000 mgr.gibba032.whopeo (mgr.24625) 12247 : cluster [DBG] pgmap v6979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:16.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:16 gibba011 bash[31774]: cluster 2021-03-10T17:45:14.717462+0000 mgr.gibba032.whopeo (mgr.24625) 12247 : cluster [DBG] pgmap v6979: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:18.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:18 gibba032 bash[24004]: cluster 2021-03-10T17:45:16.719060+0000 mgr.gibba032.whopeo (mgr.24625) 12248 : cluster [DBG] pgmap v6980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:18.760 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:18 gibba032 bash[24004]: cephadm 2021-03-10T17:45:17.287559+0000 mgr.gibba032.whopeo (mgr.24625) 12249 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:18 gibba011 bash[30791]: cluster 2021-03-10T17:45:16.719060+0000 mgr.gibba032.whopeo (mgr.24625) 12248 : cluster [DBG] pgmap v6980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:18.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:18 gibba011 bash[30791]: cephadm 2021-03-10T17:45:17.287559+0000 mgr.gibba032.whopeo (mgr.24625) 12249 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:18.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:18 gibba011 bash[31774]: cluster 2021-03-10T17:45:16.719060+0000 mgr.gibba032.whopeo (mgr.24625) 12248 : cluster [DBG] pgmap v6980: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:18.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:18 gibba011 bash[31774]: cephadm 2021-03-10T17:45:17.287559+0000 mgr.gibba032.whopeo (mgr.24625) 12249 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:20.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:20 gibba032 bash[24004]: cluster 2021-03-10T17:45:18.720839+0000 mgr.gibba032.whopeo (mgr.24625) 12250 : cluster [DBG] pgmap v6981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:20.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:20 gibba011 bash[30791]: cluster 2021-03-10T17:45:18.720839+0000 mgr.gibba032.whopeo (mgr.24625) 12250 : cluster [DBG] pgmap v6981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:20.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:20 gibba011 bash[31774]: cluster 2021-03-10T17:45:18.720839+0000 mgr.gibba032.whopeo (mgr.24625) 12250 : cluster [DBG] pgmap v6981: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:22.759 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:22 gibba032 bash[24004]: cluster 2021-03-10T17:45:20.722349+0000 mgr.gibba032.whopeo (mgr.24625) 12251 : cluster [DBG] pgmap v6982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:22.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:22 gibba011 bash[30791]: cluster 2021-03-10T17:45:20.722349+0000 mgr.gibba032.whopeo (mgr.24625) 12251 : cluster [DBG] pgmap v6982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:22.831 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:22 gibba011 bash[31774]: cluster 2021-03-10T17:45:20.722349+0000 mgr.gibba032.whopeo (mgr.24625) 12251 : cluster [DBG] pgmap v6982: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:23.587 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:45:23.817 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:45:23.817 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:45:23.818 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:45:23.818 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:45:23.818 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:45:23.818 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:45:23.819 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:45:23.819 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:45:23.819 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:45:23.819 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:45:23.819 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:45:23.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:45:23.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:45:23.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:45:23.820 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:45:23.821 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:45:23.821 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:45:23.821 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:45:23.821 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:45:23.821 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:45:23.822 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:45:24.065 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:45:24.065 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:45:24.065 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:45:24.066 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:24.066 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:45:24.066 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:45:24.066 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:24.067 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:45:24.067 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:45:24.067 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:24.067 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:45:24.067 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:45:24.068 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:45:24.068 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:45:24.068 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:45:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:24 gibba011 bash[30791]: cluster 2021-03-10T17:45:22.724288+0000 mgr.gibba032.whopeo (mgr.24625) 12252 : cluster [DBG] pgmap v6983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:24.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:24 gibba011 bash[30791]: audit 2021-03-10T17:45:24.063106+0000 mon.c (mon.1) 238 : audit [DBG] from='client.? 172.21.2.111:0/700641679' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:24 gibba011 bash[31774]: cluster 2021-03-10T17:45:22.724288+0000 mgr.gibba032.whopeo (mgr.24625) 12252 : cluster [DBG] pgmap v6983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:24.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:24 gibba011 bash[31774]: audit 2021-03-10T17:45:24.063106+0000 mon.c (mon.1) 238 : audit [DBG] from='client.? 172.21.2.111:0/700641679' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:25.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:24 gibba032 bash[24004]: cluster 2021-03-10T17:45:22.724288+0000 mgr.gibba032.whopeo (mgr.24625) 12252 : cluster [DBG] pgmap v6983: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:25.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:24 gibba032 bash[24004]: audit 2021-03-10T17:45:24.063106+0000 mon.c (mon.1) 238 : audit [DBG] from='client.? 172.21.2.111:0/700641679' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[30791]: audit 2021-03-10T17:45:23.574925+0000 mgr.gibba032.whopeo (mgr.24625) 12253 : audit [DBG] from='client.52260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:25.831 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[30791]: audit 2021-03-10T17:45:23.801758+0000 mgr.gibba032.whopeo (mgr.24625) 12254 : audit [DBG] from='client.62251 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:25.832 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[30791]: cluster 2021-03-10T17:45:24.725966+0000 mgr.gibba032.whopeo (mgr.24625) 12255 : cluster [DBG] pgmap v6984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[31774]: audit 2021-03-10T17:45:23.574925+0000 mgr.gibba032.whopeo (mgr.24625) 12253 : audit [DBG] from='client.52260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:25.832 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[31774]: audit 2021-03-10T17:45:23.801758+0000 mgr.gibba032.whopeo (mgr.24625) 12254 : audit [DBG] from='client.62251 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:25.833 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:25 gibba011 bash[31774]: cluster 2021-03-10T17:45:24.725966+0000 mgr.gibba032.whopeo (mgr.24625) 12255 : cluster [DBG] pgmap v6984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:25 gibba032 bash[24004]: audit 2021-03-10T17:45:23.574925+0000 mgr.gibba032.whopeo (mgr.24625) 12253 : audit [DBG] from='client.52260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:26.009 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:25 gibba032 bash[24004]: audit 2021-03-10T17:45:23.801758+0000 mgr.gibba032.whopeo (mgr.24625) 12254 : audit [DBG] from='client.62251 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:26.010 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:25 gibba032 bash[24004]: cluster 2021-03-10T17:45:24.725966+0000 mgr.gibba032.whopeo (mgr.24625) 12255 : cluster [DBG] pgmap v6984: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:28.080 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:27 gibba011 bash[30791]: cluster 2021-03-10T17:45:26.728128+0000 mgr.gibba032.whopeo (mgr.24625) 12256 : cluster [DBG] pgmap v6985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:28.081 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:27 gibba011 bash[31774]: cluster 2021-03-10T17:45:26.728128+0000 mgr.gibba032.whopeo (mgr.24625) 12256 : cluster [DBG] pgmap v6985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:28.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:27 gibba032 bash[24004]: cluster 2021-03-10T17:45:26.728128+0000 mgr.gibba032.whopeo (mgr.24625) 12256 : cluster [DBG] pgmap v6985: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:30.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:29 gibba032 bash[24004]: cluster 2021-03-10T17:45:28.729734+0000 mgr.gibba032.whopeo (mgr.24625) 12257 : cluster [DBG] pgmap v6986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:30.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:29 gibba011 bash[30791]: cluster 2021-03-10T17:45:28.729734+0000 mgr.gibba032.whopeo (mgr.24625) 12257 : cluster [DBG] pgmap v6986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:30.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:29 gibba011 bash[31774]: cluster 2021-03-10T17:45:28.729734+0000 mgr.gibba032.whopeo (mgr.24625) 12257 : cluster [DBG] pgmap v6986: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:32.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:31 gibba032 bash[24004]: cluster 2021-03-10T17:45:30.731059+0000 mgr.gibba032.whopeo (mgr.24625) 12258 : cluster [DBG] pgmap v6987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:32.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:31 gibba011 bash[30791]: cluster 2021-03-10T17:45:30.731059+0000 mgr.gibba032.whopeo (mgr.24625) 12258 : cluster [DBG] pgmap v6987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:32.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:31 gibba011 bash[31774]: cluster 2021-03-10T17:45:30.731059+0000 mgr.gibba032.whopeo (mgr.24625) 12258 : cluster [DBG] pgmap v6987: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:33.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:32 gibba032 bash[24004]: cephadm 2021-03-10T17:45:32.298065+0000 mgr.gibba032.whopeo (mgr.24625) 12259 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:33.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:32 gibba011 bash[30791]: cephadm 2021-03-10T17:45:32.298065+0000 mgr.gibba032.whopeo (mgr.24625) 12259 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:33.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:32 gibba011 bash[31774]: cephadm 2021-03-10T17:45:32.298065+0000 mgr.gibba032.whopeo (mgr.24625) 12259 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:34.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:33 gibba032 bash[24004]: cluster 2021-03-10T17:45:32.732921+0000 mgr.gibba032.whopeo (mgr.24625) 12260 : cluster [DBG] pgmap v6988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:34.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:33 gibba011 bash[30791]: cluster 2021-03-10T17:45:32.732921+0000 mgr.gibba032.whopeo (mgr.24625) 12260 : cluster [DBG] pgmap v6988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:34.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:33 gibba011 bash[31774]: cluster 2021-03-10T17:45:32.732921+0000 mgr.gibba032.whopeo (mgr.24625) 12260 : cluster [DBG] pgmap v6988: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:36.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:35 gibba032 bash[24004]: cluster 2021-03-10T17:45:34.734412+0000 mgr.gibba032.whopeo (mgr.24625) 12261 : cluster [DBG] pgmap v6989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:36.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:35 gibba011 bash[30791]: cluster 2021-03-10T17:45:34.734412+0000 mgr.gibba032.whopeo (mgr.24625) 12261 : cluster [DBG] pgmap v6989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:36.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:35 gibba011 bash[31774]: cluster 2021-03-10T17:45:34.734412+0000 mgr.gibba032.whopeo (mgr.24625) 12261 : cluster [DBG] pgmap v6989: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:38.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:37 gibba032 bash[24004]: cluster 2021-03-10T17:45:36.735925+0000 mgr.gibba032.whopeo (mgr.24625) 12262 : cluster [DBG] pgmap v6990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:38.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:37 gibba011 bash[30791]: cluster 2021-03-10T17:45:36.735925+0000 mgr.gibba032.whopeo (mgr.24625) 12262 : cluster [DBG] pgmap v6990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:38.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:37 gibba011 bash[31774]: cluster 2021-03-10T17:45:36.735925+0000 mgr.gibba032.whopeo (mgr.24625) 12262 : cluster [DBG] pgmap v6990: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:40.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:39 gibba032 bash[24004]: cluster 2021-03-10T17:45:38.737581+0000 mgr.gibba032.whopeo (mgr.24625) 12263 : cluster [DBG] pgmap v6991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:40.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:39 gibba011 bash[30791]: cluster 2021-03-10T17:45:38.737581+0000 mgr.gibba032.whopeo (mgr.24625) 12263 : cluster [DBG] pgmap v6991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:40.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:39 gibba011 bash[31774]: cluster 2021-03-10T17:45:38.737581+0000 mgr.gibba032.whopeo (mgr.24625) 12263 : cluster [DBG] pgmap v6991: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:42.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:41 gibba032 bash[24004]: cluster 2021-03-10T17:45:40.738929+0000 mgr.gibba032.whopeo (mgr.24625) 12264 : cluster [DBG] pgmap v6992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:42.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:41 gibba011 bash[30791]: cluster 2021-03-10T17:45:40.738929+0000 mgr.gibba032.whopeo (mgr.24625) 12264 : cluster [DBG] pgmap v6992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:42.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:41 gibba011 bash[31774]: cluster 2021-03-10T17:45:40.738929+0000 mgr.gibba032.whopeo (mgr.24625) 12264 : cluster [DBG] pgmap v6992: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:44.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:43 gibba032 bash[24004]: cluster 2021-03-10T17:45:42.740701+0000 mgr.gibba032.whopeo (mgr.24625) 12265 : cluster [DBG] pgmap v6993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:44.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:43 gibba011 bash[30791]: cluster 2021-03-10T17:45:42.740701+0000 mgr.gibba032.whopeo (mgr.24625) 12265 : cluster [DBG] pgmap v6993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:44.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:43 gibba011 bash[31774]: cluster 2021-03-10T17:45:42.740701+0000 mgr.gibba032.whopeo (mgr.24625) 12265 : cluster [DBG] pgmap v6993: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:46.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:45 gibba032 bash[24004]: cluster 2021-03-10T17:45:44.742198+0000 mgr.gibba032.whopeo (mgr.24625) 12266 : cluster [DBG] pgmap v6994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:46.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:45 gibba011 bash[30791]: cluster 2021-03-10T17:45:44.742198+0000 mgr.gibba032.whopeo (mgr.24625) 12266 : cluster [DBG] pgmap v6994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:46.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:45 gibba011 bash[31774]: cluster 2021-03-10T17:45:44.742198+0000 mgr.gibba032.whopeo (mgr.24625) 12266 : cluster [DBG] pgmap v6994: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:47 gibba032 bash[24004]: cluster 2021-03-10T17:45:46.743903+0000 mgr.gibba032.whopeo (mgr.24625) 12267 : cluster [DBG] pgmap v6995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:48.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:47 gibba032 bash[24004]: cephadm 2021-03-10T17:45:47.313964+0000 mgr.gibba032.whopeo (mgr.24625) 12268 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:47 gibba011 bash[30791]: cluster 2021-03-10T17:45:46.743903+0000 mgr.gibba032.whopeo (mgr.24625) 12267 : cluster [DBG] pgmap v6995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:48.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:47 gibba011 bash[30791]: cephadm 2021-03-10T17:45:47.313964+0000 mgr.gibba032.whopeo (mgr.24625) 12268 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:47 gibba011 bash[31774]: cluster 2021-03-10T17:45:46.743903+0000 mgr.gibba032.whopeo (mgr.24625) 12267 : cluster [DBG] pgmap v6995: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:48.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:47 gibba011 bash[31774]: cephadm 2021-03-10T17:45:47.313964+0000 mgr.gibba032.whopeo (mgr.24625) 12268 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:45:50.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:49 gibba032 bash[24004]: cluster 2021-03-10T17:45:48.745393+0000 mgr.gibba032.whopeo (mgr.24625) 12269 : cluster [DBG] pgmap v6996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:50.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:49 gibba011 bash[30791]: cluster 2021-03-10T17:45:48.745393+0000 mgr.gibba032.whopeo (mgr.24625) 12269 : cluster [DBG] pgmap v6996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:50.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:49 gibba011 bash[31774]: cluster 2021-03-10T17:45:48.745393+0000 mgr.gibba032.whopeo (mgr.24625) 12269 : cluster [DBG] pgmap v6996: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:52.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:51 gibba032 bash[24004]: cluster 2021-03-10T17:45:50.746715+0000 mgr.gibba032.whopeo (mgr.24625) 12270 : cluster [DBG] pgmap v6997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:52.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:51 gibba011 bash[30791]: cluster 2021-03-10T17:45:50.746715+0000 mgr.gibba032.whopeo (mgr.24625) 12270 : cluster [DBG] pgmap v6997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:52.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:51 gibba011 bash[31774]: cluster 2021-03-10T17:45:50.746715+0000 mgr.gibba032.whopeo (mgr.24625) 12270 : cluster [DBG] pgmap v6997: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:54.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:53 gibba032 bash[24004]: cluster 2021-03-10T17:45:52.748404+0000 mgr.gibba032.whopeo (mgr.24625) 12271 : cluster [DBG] pgmap v6998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:54.309 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:45:54.330 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:53 gibba011 bash[30791]: cluster 2021-03-10T17:45:52.748404+0000 mgr.gibba032.whopeo (mgr.24625) 12271 : cluster [DBG] pgmap v6998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:54.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:53 gibba011 bash[31774]: cluster 2021-03-10T17:45:52.748404+0000 mgr.gibba032.whopeo (mgr.24625) 12271 : cluster [DBG] pgmap v6998: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:54.538 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:45:54.538 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 3m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:45:54.538 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 2m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:45:54.539 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:45:54.539 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:45:54.539 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:45:54.539 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:45:54.539 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 2m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:45:54.540 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:45:54.540 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:45:54.540 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 2m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:45:54.540 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:45:54.541 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:45:54.541 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:45:54.541 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:45:54.541 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:45:54.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:45:54.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:45:54.542 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 2m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:45:54.542 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 2m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:45:54.542 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 3m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:45:54.795 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:45:54.795 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:45:54.795 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:45:54.796 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:54.796 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:45:54.796 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:45:54.796 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:54.797 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:45:54.797 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:45:54.797 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:45:54.797 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:45:54.797 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:45:54.798 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:45:54.798 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:45:54.798 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:45:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:54 gibba032 bash[24004]: audit 2021-03-10T17:45:54.295993+0000 mgr.gibba032.whopeo (mgr.24625) 12272 : audit [DBG] from='client.62263 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:55.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:54 gibba032 bash[24004]: audit 2021-03-10T17:45:54.793281+0000 mon.a (mon.0) 4000 : audit [DBG] from='client.? 172.21.2.111:0/2186360680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:54 gibba011 bash[30791]: audit 2021-03-10T17:45:54.295993+0000 mgr.gibba032.whopeo (mgr.24625) 12272 : audit [DBG] from='client.62263 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:55.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:54 gibba011 bash[30791]: audit 2021-03-10T17:45:54.793281+0000 mon.a (mon.0) 4000 : audit [DBG] from='client.? 172.21.2.111:0/2186360680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:54 gibba011 bash[31774]: audit 2021-03-10T17:45:54.295993+0000 mgr.gibba032.whopeo (mgr.24625) 12272 : audit [DBG] from='client.62263 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:55.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:54 gibba011 bash[31774]: audit 2021-03-10T17:45:54.793281+0000 mon.a (mon.0) 4000 : audit [DBG] from='client.? 172.21.2.111:0/2186360680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:45:56.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:55 gibba032 bash[24004]: audit 2021-03-10T17:45:54.522349+0000 mgr.gibba032.whopeo (mgr.24625) 12273 : audit [DBG] from='client.62269 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:56.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:55 gibba032 bash[24004]: cluster 2021-03-10T17:45:54.749928+0000 mgr.gibba032.whopeo (mgr.24625) 12274 : cluster [DBG] pgmap v6999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:55 gibba011 bash[30791]: audit 2021-03-10T17:45:54.522349+0000 mgr.gibba032.whopeo (mgr.24625) 12273 : audit [DBG] from='client.62269 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:56.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:55 gibba011 bash[30791]: cluster 2021-03-10T17:45:54.749928+0000 mgr.gibba032.whopeo (mgr.24625) 12274 : cluster [DBG] pgmap v6999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:55 gibba011 bash[31774]: audit 2021-03-10T17:45:54.522349+0000 mgr.gibba032.whopeo (mgr.24625) 12273 : audit [DBG] from='client.62269 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:45:56.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:55 gibba011 bash[31774]: cluster 2021-03-10T17:45:54.749928+0000 mgr.gibba032.whopeo (mgr.24625) 12274 : cluster [DBG] pgmap v6999: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: cluster 2021-03-10T17:45:56.751681+0000 mgr.gibba032.whopeo (mgr.24625) 12275 : cluster [DBG] pgmap v7000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:58.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.635011+0000 mon.a (mon.0) 4001 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:45:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.635695+0000 mon.a (mon.0) 4002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:45:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.852243+0000 mon.a (mon.0) 4003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:45:58.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.852979+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.853610+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:57 gibba032 bash[24004]: audit 2021-03-10T17:45:57.854275+0000 mon.a (mon.0) 4006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:45:58.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: cluster 2021-03-10T17:45:56.751681+0000 mgr.gibba032.whopeo (mgr.24625) 12275 : cluster [DBG] pgmap v7000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.635011+0000 mon.a (mon.0) 4001 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:45:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.635695+0000 mon.a (mon.0) 4002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:45:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.852243+0000 mon.a (mon.0) 4003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:45:58.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.852979+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.853610+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[30791]: audit 2021-03-10T17:45:57.854275+0000 mon.a (mon.0) 4006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:45:58.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: cluster 2021-03-10T17:45:56.751681+0000 mgr.gibba032.whopeo (mgr.24625) 12275 : cluster [DBG] pgmap v7000: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:45:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.635011+0000 mon.a (mon.0) 4001 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/trash_purge_schedule"}]: dispatch 2021-03-10T17:45:58.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.635695+0000 mon.a (mon.0) 4002 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/gibba032.whopeo/mirror_snapshot_schedule"}]: dispatch 2021-03-10T17:45:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.852243+0000 mon.a (mon.0) 4003 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2021-03-10T17:45:58.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.852979+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.853610+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2021-03-10T17:45:58.336 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:57 gibba011 bash[31774]: audit 2021-03-10T17:45:57.854275+0000 mon.a (mon.0) 4006 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2021-03-10T17:46:00.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:45:59 gibba032 bash[24004]: cluster 2021-03-10T17:45:58.753239+0000 mgr.gibba032.whopeo (mgr.24625) 12276 : cluster [DBG] pgmap v7001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:00.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:45:59 gibba011 bash[30791]: cluster 2021-03-10T17:45:58.753239+0000 mgr.gibba032.whopeo (mgr.24625) 12276 : cluster [DBG] pgmap v7001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:00.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:45:59 gibba011 bash[31774]: cluster 2021-03-10T17:45:58.753239+0000 mgr.gibba032.whopeo (mgr.24625) 12276 : cluster [DBG] pgmap v7001: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:02.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:01 gibba032 bash[24004]: cluster 2021-03-10T17:46:00.754574+0000 mgr.gibba032.whopeo (mgr.24625) 12277 : cluster [DBG] pgmap v7002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:02.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:01 gibba011 bash[30791]: cluster 2021-03-10T17:46:00.754574+0000 mgr.gibba032.whopeo (mgr.24625) 12277 : cluster [DBG] pgmap v7002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:02.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:01 gibba011 bash[31774]: cluster 2021-03-10T17:46:00.754574+0000 mgr.gibba032.whopeo (mgr.24625) 12277 : cluster [DBG] pgmap v7002: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:03.259 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:02 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.321811+0000 mgr.gibba032.whopeo (mgr.24625) 12278 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:46:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:02 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.323372+0000 mgr.gibba032.whopeo (mgr.24625) 12279 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:46:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:02 gibba032 bash[24004]: audit 2021-03-10T17:46:02.533778+0000 mon.a (mon.0) 4007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:46:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.536955+0000 mon.a (mon.0) 4008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:46:03.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.538039+0000 mon.a (mon.0) 4009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.539091+0000 mon.a (mon.0) 4010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.540382+0000 mon.a (mon.0) 4011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.542766+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:03 gibba032 bash[24004]: audit 2021-03-10T17:46:02.544442+0000 mon.a (mon.0) 4013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:02 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.321811+0000 mgr.gibba032.whopeo (mgr.24625) 12278 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:46:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:02 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.323372+0000 mgr.gibba032.whopeo (mgr.24625) 12279 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:46:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.533778+0000 mon.a (mon.0) 4007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:46:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.536955+0000 mon.a (mon.0) 4008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:46:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.538039+0000 mon.a (mon.0) 4009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.539091+0000 mon.a (mon.0) 4010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.540382+0000 mon.a (mon.0) 4011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.542766+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[30791]: audit 2021-03-10T17:46:02.544442+0000 mon.a (mon.0) 4013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.333 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.321811+0000 mgr.gibba032.whopeo (mgr.24625) 12278 : cephadm [INF] Refreshing gibba011 facts 2021-03-10T17:46:03.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.323372+0000 mgr.gibba032.whopeo (mgr.24625) 12279 : cephadm [INF] Refreshing gibba032 facts 2021-03-10T17:46:03.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.533778+0000 mon.a (mon.0) 4007 : audit [INF] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' 2021-03-10T17:46:03.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.536955+0000 mon.a (mon.0) 4008 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-03-10T17:46:03.334 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.538039+0000 mon.a (mon.0) 4009 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.539091+0000 mon.a (mon.0) 4010 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.540382+0000 mon.a (mon.0) 4011 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.542766+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:03.335 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:03 gibba011 bash[31774]: audit 2021-03-10T17:46:02.544442+0000 mon.a (mon.0) 4013 : audit [DBG] from='mgr.24625 172.21.2.132:0/587223463' entity='mgr.gibba032.whopeo' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.534727+0000 mgr.gibba032.whopeo (mgr.24625) 12280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:46:04.260 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.536233+0000 mgr.gibba032.whopeo (mgr.24625) 12281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:46:04.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.537230+0000 mgr.gibba032.whopeo (mgr.24625) 12282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:46:04.261 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.537267+0000 mgr.gibba032.whopeo (mgr.24625) 12283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:46:04.262 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.538334+0000 mgr.gibba032.whopeo (mgr.24625) 12284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:46:04.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.538482+0000 mgr.gibba032.whopeo (mgr.24625) 12285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:46:04.263 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.539437+0000 mgr.gibba032.whopeo (mgr.24625) 12286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:46:04.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.540661+0000 mgr.gibba032.whopeo (mgr.24625) 12287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:46:04.264 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.540695+0000 mgr.gibba032.whopeo (mgr.24625) 12288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:46:04.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.542220+0000 mgr.gibba032.whopeo (mgr.24625) 12289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:46:04.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.542529+0000 mgr.gibba032.whopeo (mgr.24625) 12290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:46:04.265 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.543642+0000 mgr.gibba032.whopeo (mgr.24625) 12291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:46:04.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.543678+0000 mgr.gibba032.whopeo (mgr.24625) 12292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:46:04.267 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cephadm 2021-03-10T17:46:02.543782+0000 mgr.gibba032.whopeo (mgr.24625) 12293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:04.268 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:04 gibba032 bash[24004]: cluster 2021-03-10T17:46:02.756343+0000 mgr.gibba032.whopeo (mgr.24625) 12294 : cluster [DBG] pgmap v7003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.534727+0000 mgr.gibba032.whopeo (mgr.24625) 12280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:46:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.536233+0000 mgr.gibba032.whopeo (mgr.24625) 12281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:46:04.332 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.537230+0000 mgr.gibba032.whopeo (mgr.24625) 12282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:46:04.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.537267+0000 mgr.gibba032.whopeo (mgr.24625) 12283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:46:04.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.538334+0000 mgr.gibba032.whopeo (mgr.24625) 12284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:46:04.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.538482+0000 mgr.gibba032.whopeo (mgr.24625) 12285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:46:04.333 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.539437+0000 mgr.gibba032.whopeo (mgr.24625) 12286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:46:04.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.540661+0000 mgr.gibba032.whopeo (mgr.24625) 12287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:46:04.334 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.540695+0000 mgr.gibba032.whopeo (mgr.24625) 12288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:46:04.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.542220+0000 mgr.gibba032.whopeo (mgr.24625) 12289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:46:04.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.542529+0000 mgr.gibba032.whopeo (mgr.24625) 12290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:46:04.335 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.543642+0000 mgr.gibba032.whopeo (mgr.24625) 12291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:46:04.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.543678+0000 mgr.gibba032.whopeo (mgr.24625) 12292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:46:04.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cephadm 2021-03-10T17:46:02.543782+0000 mgr.gibba032.whopeo (mgr.24625) 12293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:04.336 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[30791]: cluster 2021-03-10T17:46:02.756343+0000 mgr.gibba032.whopeo (mgr.24625) 12294 : cluster [DBG] pgmap v7003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:04.338 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.534727+0000 mgr.gibba032.whopeo (mgr.24625) 12280 : cephadm [INF] Upgrade: Target is version 17.0.0-1749-gb26fc080 (quincy), container quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a digests ['quay.ceph.io/ceph-ci/ceph@sha256:530c3c162b28671b21709357145385150b49dbeaa7896ba51de446d049615c54'] 2021-03-10T17:46:04.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.536233+0000 mgr.gibba032.whopeo (mgr.24625) 12281 : cephadm [INF] Upgrade: Checking mgr daemons 2021-03-10T17:46:04.339 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.537230+0000 mgr.gibba032.whopeo (mgr.24625) 12282 : cephadm [INF] Upgrade: All mgr daemons are up to date. 2021-03-10T17:46:04.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.537267+0000 mgr.gibba032.whopeo (mgr.24625) 12283 : cephadm [INF] Upgrade: Checking mon daemons 2021-03-10T17:46:04.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.538334+0000 mgr.gibba032.whopeo (mgr.24625) 12284 : cephadm [INF] Upgrade: All mon daemons are up to date. 2021-03-10T17:46:04.340 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.538482+0000 mgr.gibba032.whopeo (mgr.24625) 12285 : cephadm [INF] Upgrade: Checking crash daemons 2021-03-10T17:46:04.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.539437+0000 mgr.gibba032.whopeo (mgr.24625) 12286 : cephadm [INF] Upgrade: Setting container_image for all crash 2021-03-10T17:46:04.341 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.540661+0000 mgr.gibba032.whopeo (mgr.24625) 12287 : cephadm [INF] Upgrade: All crash daemons are up to date. 2021-03-10T17:46:04.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.540695+0000 mgr.gibba032.whopeo (mgr.24625) 12288 : cephadm [INF] Upgrade: Checking osd daemons 2021-03-10T17:46:04.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.542220+0000 mgr.gibba032.whopeo (mgr.24625) 12289 : cephadm [INF] Upgrade: All osd daemons are up to date. 2021-03-10T17:46:04.342 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.542529+0000 mgr.gibba032.whopeo (mgr.24625) 12290 : cephadm [INF] Upgrade: Checking mds daemons 2021-03-10T17:46:04.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.543642+0000 mgr.gibba032.whopeo (mgr.24625) 12291 : cephadm [INF] Upgrade: All mds daemons are up to date. 2021-03-10T17:46:04.343 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.543678+0000 mgr.gibba032.whopeo (mgr.24625) 12292 : cephadm [INF] Upgrade: Checking rgw daemons 2021-03-10T17:46:04.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cephadm 2021-03-10T17:46:02.543782+0000 mgr.gibba032.whopeo (mgr.24625) 12293 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:04.344 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:04 gibba011 bash[31774]: cluster 2021-03-10T17:46:02.756343+0000 mgr.gibba032.whopeo (mgr.24625) 12294 : cluster [DBG] pgmap v7003: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:06.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:06 gibba011 bash[30791]: cluster 2021-03-10T17:46:04.757742+0000 mgr.gibba032.whopeo (mgr.24625) 12295 : cluster [DBG] pgmap v7004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:06.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:06 gibba011 bash[31774]: cluster 2021-03-10T17:46:04.757742+0000 mgr.gibba032.whopeo (mgr.24625) 12295 : cluster [DBG] pgmap v7004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:06.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:06 gibba032 bash[24004]: cluster 2021-03-10T17:46:04.757742+0000 mgr.gibba032.whopeo (mgr.24625) 12295 : cluster [DBG] pgmap v7004: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:08.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:08 gibba011 bash[30791]: cluster 2021-03-10T17:46:06.759543+0000 mgr.gibba032.whopeo (mgr.24625) 12296 : cluster [DBG] pgmap v7005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:08.332 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:08 gibba011 bash[31774]: cluster 2021-03-10T17:46:06.759543+0000 mgr.gibba032.whopeo (mgr.24625) 12296 : cluster [DBG] pgmap v7005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:08.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:08 gibba032 bash[24004]: cluster 2021-03-10T17:46:06.759543+0000 mgr.gibba032.whopeo (mgr.24625) 12296 : cluster [DBG] pgmap v7005: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:10.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:10 gibba011 bash[30791]: cluster 2021-03-10T17:46:08.761193+0000 mgr.gibba032.whopeo (mgr.24625) 12297 : cluster [DBG] pgmap v7006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:10.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:10 gibba011 bash[31774]: cluster 2021-03-10T17:46:08.761193+0000 mgr.gibba032.whopeo (mgr.24625) 12297 : cluster [DBG] pgmap v7006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:10.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:10 gibba032 bash[24004]: cluster 2021-03-10T17:46:08.761193+0000 mgr.gibba032.whopeo (mgr.24625) 12297 : cluster [DBG] pgmap v7006: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:12.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:12 gibba011 bash[30791]: cluster 2021-03-10T17:46:10.762551+0000 mgr.gibba032.whopeo (mgr.24625) 12298 : cluster [DBG] pgmap v7007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:12.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:12 gibba011 bash[31774]: cluster 2021-03-10T17:46:10.762551+0000 mgr.gibba032.whopeo (mgr.24625) 12298 : cluster [DBG] pgmap v7007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:12.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:12 gibba032 bash[24004]: cluster 2021-03-10T17:46:10.762551+0000 mgr.gibba032.whopeo (mgr.24625) 12298 : cluster [DBG] pgmap v7007: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:14.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:14 gibba011 bash[30791]: cluster 2021-03-10T17:46:12.764089+0000 mgr.gibba032.whopeo (mgr.24625) 12299 : cluster [DBG] pgmap v7008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:14.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:14 gibba011 bash[31774]: cluster 2021-03-10T17:46:12.764089+0000 mgr.gibba032.whopeo (mgr.24625) 12299 : cluster [DBG] pgmap v7008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:14.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:14 gibba032 bash[24004]: cluster 2021-03-10T17:46:12.764089+0000 mgr.gibba032.whopeo (mgr.24625) 12299 : cluster [DBG] pgmap v7008: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:16.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:16 gibba011 bash[30791]: cluster 2021-03-10T17:46:14.765288+0000 mgr.gibba032.whopeo (mgr.24625) 12300 : cluster [DBG] pgmap v7009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:16.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:16 gibba011 bash[31774]: cluster 2021-03-10T17:46:14.765288+0000 mgr.gibba032.whopeo (mgr.24625) 12300 : cluster [DBG] pgmap v7009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:16.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:16 gibba032 bash[24004]: cluster 2021-03-10T17:46:14.765288+0000 mgr.gibba032.whopeo (mgr.24625) 12300 : cluster [DBG] pgmap v7009: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:18.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:18 gibba011 bash[30791]: cluster 2021-03-10T17:46:16.767007+0000 mgr.gibba032.whopeo (mgr.24625) 12301 : cluster [DBG] pgmap v7010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:18.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:18 gibba011 bash[31774]: cluster 2021-03-10T17:46:16.767007+0000 mgr.gibba032.whopeo (mgr.24625) 12301 : cluster [DBG] pgmap v7010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:18.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:18 gibba032 bash[24004]: cluster 2021-03-10T17:46:16.767007+0000 mgr.gibba032.whopeo (mgr.24625) 12301 : cluster [DBG] pgmap v7010: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:19.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:19 gibba011 bash[30791]: cephadm 2021-03-10T17:46:17.559797+0000 mgr.gibba032.whopeo (mgr.24625) 12302 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:19.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:19 gibba011 bash[31774]: cephadm 2021-03-10T17:46:17.559797+0000 mgr.gibba032.whopeo (mgr.24625) 12302 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:19.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:19 gibba032 bash[24004]: cephadm 2021-03-10T17:46:17.559797+0000 mgr.gibba032.whopeo (mgr.24625) 12302 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:20.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:20 gibba011 bash[30791]: cluster 2021-03-10T17:46:18.768561+0000 mgr.gibba032.whopeo (mgr.24625) 12303 : cluster [DBG] pgmap v7011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:20.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:20 gibba011 bash[31774]: cluster 2021-03-10T17:46:18.768561+0000 mgr.gibba032.whopeo (mgr.24625) 12303 : cluster [DBG] pgmap v7011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:20.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:20 gibba032 bash[24004]: cluster 2021-03-10T17:46:18.768561+0000 mgr.gibba032.whopeo (mgr.24625) 12303 : cluster [DBG] pgmap v7011: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:22.331 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:22 gibba011 bash[30791]: cluster 2021-03-10T17:46:20.769881+0000 mgr.gibba032.whopeo (mgr.24625) 12304 : cluster [DBG] pgmap v7012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:22.331 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:22 gibba011 bash[31774]: cluster 2021-03-10T17:46:20.769881+0000 mgr.gibba032.whopeo (mgr.24625) 12304 : cluster [DBG] pgmap v7012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:22.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:22 gibba032 bash[24004]: cluster 2021-03-10T17:46:20.769881+0000 mgr.gibba032.whopeo (mgr.24625) 12304 : cluster [DBG] pgmap v7012: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:24.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:24 gibba032 bash[24004]: cluster 2021-03-10T17:46:22.771419+0000 mgr.gibba032.whopeo (mgr.24625) 12305 : cluster [DBG] pgmap v7013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:24.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:24 gibba011 bash[30791]: cluster 2021-03-10T17:46:22.771419+0000 mgr.gibba032.whopeo (mgr.24625) 12305 : cluster [DBG] pgmap v7013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:24.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:24 gibba011 bash[31774]: cluster 2021-03-10T17:46:22.771419+0000 mgr.gibba032.whopeo (mgr.24625) 12305 : cluster [DBG] pgmap v7013: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:25.040 INFO:teuthology.orchestra.run.gibba011.stdout:true 2021-03-10T17:46:25.269 INFO:teuthology.orchestra.run.gibba011.stdout:NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID 2021-03-10T17:46:25.270 INFO:teuthology.orchestra.run.gibba011.stdout:alertmanager.a gibba011 running (3h) 4m ago 3h 0.21.0 prom/alertmanager c876f5897d7b 6c1e707dd209 2021-03-10T17:46:25.270 INFO:teuthology.orchestra.run.gibba011.stdout:grafana.a gibba032 running (3h) 3m ago 3h 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c c8432c2a7a94 2021-03-10T17:46:25.271 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.gibba032.whopeo gibba032 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da f91cc2266d59 2021-03-10T17:46:25.271 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.x gibba032 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d6016a02942a 2021-03-10T17:46:25.272 INFO:teuthology.orchestra.run.gibba011.stdout:mgr.y gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da db0a8bf316f6 2021-03-10T17:46:25.272 INFO:teuthology.orchestra.run.gibba011.stdout:mon.a gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 4c594a5db324 2021-03-10T17:46:25.272 INFO:teuthology.orchestra.run.gibba011.stdout:mon.b gibba032 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 170c1129bacc 2021-03-10T17:46:25.274 INFO:teuthology.orchestra.run.gibba011.stdout:mon.c gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 6edd7f37c961 2021-03-10T17:46:25.275 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.a gibba011 running (3h) 4m ago 3h 1.1.2 prom/node-exporter c19ae228f069 c7b51a252180 2021-03-10T17:46:25.276 INFO:teuthology.orchestra.run.gibba011.stdout:node-exporter.b gibba032 running (3h) 3m ago 3h 1.1.2 prom/node-exporter c19ae228f069 f2425e6af6e6 2021-03-10T17:46:25.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.0 gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da fdc098aa5d9a 2021-03-10T17:46:25.276 INFO:teuthology.orchestra.run.gibba011.stdout:osd.1 gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da ab3ed8814311 2021-03-10T17:46:25.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.2 gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 3761338e06ef 2021-03-10T17:46:25.277 INFO:teuthology.orchestra.run.gibba011.stdout:osd.3 gibba011 running (3h) 4m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da d711aae9bc24 2021-03-10T17:46:25.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.4 gibba032 running (3h) 3m ago 4h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 627b2b27442d 2021-03-10T17:46:25.278 INFO:teuthology.orchestra.run.gibba011.stdout:osd.5 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 460ef12c5ac7 2021-03-10T17:46:25.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.6 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 02f37043ad95 2021-03-10T17:46:25.281 INFO:teuthology.orchestra.run.gibba011.stdout:osd.7 gibba032 running (3h) 3m ago 3h 17.0.0-1749-gb26fc080 quay.ceph.io/ceph-ci/ceph:b26fc080937a021ce826cdc94d1adb52c064381a d4615e7085da 41fce4a6b542 2021-03-10T17:46:25.281 INFO:teuthology.orchestra.run.gibba011.stdout:prometheus.a gibba032 running (3h) 3m ago 3h 2.25.0 prom/prometheus:latest a618f5685492 6a603218f777 2021-03-10T17:46:25.282 INFO:teuthology.orchestra.run.gibba011.stdout:rgw.r.z.gibba011.xbljmx gibba011 error 4m ago 3h docker.io/ceph/ceph:v15.2.0 2021-03-10T17:46:25.524 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-03-10T17:46:25.524 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": { 2021-03-10T17:46:25.524 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:46:25.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:46:25.525 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": { 2021-03-10T17:46:25.525 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 3 2021-03-10T17:46:25.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:46:25.526 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": { 2021-03-10T17:46:25.526 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 8 2021-03-10T17:46:25.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-03-10T17:46:25.526 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": {}, 2021-03-10T17:46:25.527 INFO:teuthology.orchestra.run.gibba011.stdout: "overall": { 2021-03-10T17:46:25.527 INFO:teuthology.orchestra.run.gibba011.stdout: "ceph version 17.0.0-1749-gb26fc080 (b26fc080937a021ce826cdc94d1adb52c064381a) quincy (dev)": 14 2021-03-10T17:46:25.527 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-03-10T17:46:25.527 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-03-10T17:46:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:26 gibba032 bash[24004]: cluster 2021-03-10T17:46:24.772971+0000 mgr.gibba032.whopeo (mgr.24625) 12306 : cluster [DBG] pgmap v7014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:26 gibba032 bash[24004]: audit 2021-03-10T17:46:25.027598+0000 mgr.gibba032.whopeo (mgr.24625) 12307 : audit [DBG] from='client.62281 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.510 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:26 gibba032 bash[24004]: audit 2021-03-10T17:46:25.254325+0000 mgr.gibba032.whopeo (mgr.24625) 12308 : audit [DBG] from='client.52302 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.511 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:26 gibba032 bash[24004]: audit 2021-03-10T17:46:25.522078+0000 mon.c (mon.1) 239 : audit [DBG] from='client.? 172.21.2.111:0/427325281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:26.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[30791]: cluster 2021-03-10T17:46:24.772971+0000 mgr.gibba032.whopeo (mgr.24625) 12306 : cluster [DBG] pgmap v7014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[30791]: audit 2021-03-10T17:46:25.027598+0000 mgr.gibba032.whopeo (mgr.24625) 12307 : audit [DBG] from='client.62281 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[30791]: audit 2021-03-10T17:46:25.254325+0000 mgr.gibba032.whopeo (mgr.24625) 12308 : audit [DBG] from='client.52302 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[30791]: audit 2021-03-10T17:46:25.522078+0000 mon.c (mon.1) 239 : audit [DBG] from='client.? 172.21.2.111:0/427325281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:26.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[31774]: cluster 2021-03-10T17:46:24.772971+0000 mgr.gibba032.whopeo (mgr.24625) 12306 : cluster [DBG] pgmap v7014: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[31774]: audit 2021-03-10T17:46:25.027598+0000 mgr.gibba032.whopeo (mgr.24625) 12307 : audit [DBG] from='client.62281 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[31774]: audit 2021-03-10T17:46:25.254325+0000 mgr.gibba032.whopeo (mgr.24625) 12308 : audit [DBG] from='client.52302 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-03-10T17:46:26.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:26 gibba011 bash[31774]: audit 2021-03-10T17:46:25.522078+0000 mon.c (mon.1) 239 : audit [DBG] from='client.? 172.21.2.111:0/427325281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-03-10T17:46:28.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:28 gibba032 bash[24004]: cluster 2021-03-10T17:46:26.774748+0000 mgr.gibba032.whopeo (mgr.24625) 12309 : cluster [DBG] pgmap v7015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:28.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:28 gibba011 bash[30791]: cluster 2021-03-10T17:46:26.774748+0000 mgr.gibba032.whopeo (mgr.24625) 12309 : cluster [DBG] pgmap v7015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:28.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:28 gibba011 bash[31774]: cluster 2021-03-10T17:46:26.774748+0000 mgr.gibba032.whopeo (mgr.24625) 12309 : cluster [DBG] pgmap v7015: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:30.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:30 gibba032 bash[24004]: cluster 2021-03-10T17:46:28.776400+0000 mgr.gibba032.whopeo (mgr.24625) 12310 : cluster [DBG] pgmap v7016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:30.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:30 gibba011 bash[30791]: cluster 2021-03-10T17:46:28.776400+0000 mgr.gibba032.whopeo (mgr.24625) 12310 : cluster [DBG] pgmap v7016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:30.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:30 gibba011 bash[31774]: cluster 2021-03-10T17:46:28.776400+0000 mgr.gibba032.whopeo (mgr.24625) 12310 : cluster [DBG] pgmap v7016: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:32.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:32 gibba032 bash[24004]: cluster 2021-03-10T17:46:30.777718+0000 mgr.gibba032.whopeo (mgr.24625) 12311 : cluster [DBG] pgmap v7017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:32.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:32 gibba011 bash[30791]: cluster 2021-03-10T17:46:30.777718+0000 mgr.gibba032.whopeo (mgr.24625) 12311 : cluster [DBG] pgmap v7017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:32.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:32 gibba011 bash[31774]: cluster 2021-03-10T17:46:30.777718+0000 mgr.gibba032.whopeo (mgr.24625) 12311 : cluster [DBG] pgmap v7017: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:34 gibba032 bash[24004]: cephadm 2021-03-10T17:46:32.573424+0000 mgr.gibba032.whopeo (mgr.24625) 12312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:34.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:34 gibba032 bash[24004]: cluster 2021-03-10T17:46:32.779339+0000 mgr.gibba032.whopeo (mgr.24625) 12313 : cluster [DBG] pgmap v7018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:34 gibba011 bash[30791]: cephadm 2021-03-10T17:46:32.573424+0000 mgr.gibba032.whopeo (mgr.24625) 12312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:34.582 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:34 gibba011 bash[30791]: cluster 2021-03-10T17:46:32.779339+0000 mgr.gibba032.whopeo (mgr.24625) 12313 : cluster [DBG] pgmap v7018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:34.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:34 gibba011 bash[31774]: cephadm 2021-03-10T17:46:32.573424+0000 mgr.gibba032.whopeo (mgr.24625) 12312 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:34.583 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:34 gibba011 bash[31774]: cluster 2021-03-10T17:46:32.779339+0000 mgr.gibba032.whopeo (mgr.24625) 12313 : cluster [DBG] pgmap v7018: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:36.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:36 gibba032 bash[24004]: cluster 2021-03-10T17:46:34.780878+0000 mgr.gibba032.whopeo (mgr.24625) 12314 : cluster [DBG] pgmap v7019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:36.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:36 gibba011 bash[30791]: cluster 2021-03-10T17:46:34.780878+0000 mgr.gibba032.whopeo (mgr.24625) 12314 : cluster [DBG] pgmap v7019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:36.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:36 gibba011 bash[31774]: cluster 2021-03-10T17:46:34.780878+0000 mgr.gibba032.whopeo (mgr.24625) 12314 : cluster [DBG] pgmap v7019: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:38.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:38 gibba032 bash[24004]: cluster 2021-03-10T17:46:36.782610+0000 mgr.gibba032.whopeo (mgr.24625) 12315 : cluster [DBG] pgmap v7020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:38.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:38 gibba011 bash[30791]: cluster 2021-03-10T17:46:36.782610+0000 mgr.gibba032.whopeo (mgr.24625) 12315 : cluster [DBG] pgmap v7020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:38.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:38 gibba011 bash[31774]: cluster 2021-03-10T17:46:36.782610+0000 mgr.gibba032.whopeo (mgr.24625) 12315 : cluster [DBG] pgmap v7020: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:40.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:40 gibba032 bash[24004]: cluster 2021-03-10T17:46:38.784267+0000 mgr.gibba032.whopeo (mgr.24625) 12316 : cluster [DBG] pgmap v7021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:40.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:40 gibba011 bash[30791]: cluster 2021-03-10T17:46:38.784267+0000 mgr.gibba032.whopeo (mgr.24625) 12316 : cluster [DBG] pgmap v7021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:40.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:40 gibba011 bash[31774]: cluster 2021-03-10T17:46:38.784267+0000 mgr.gibba032.whopeo (mgr.24625) 12316 : cluster [DBG] pgmap v7021: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:42.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:42 gibba032 bash[24004]: cluster 2021-03-10T17:46:40.785585+0000 mgr.gibba032.whopeo (mgr.24625) 12317 : cluster [DBG] pgmap v7022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:42.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:42 gibba011 bash[30791]: cluster 2021-03-10T17:46:40.785585+0000 mgr.gibba032.whopeo (mgr.24625) 12317 : cluster [DBG] pgmap v7022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:42.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:42 gibba011 bash[31774]: cluster 2021-03-10T17:46:40.785585+0000 mgr.gibba032.whopeo (mgr.24625) 12317 : cluster [DBG] pgmap v7022: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:44.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:44 gibba032 bash[24004]: cluster 2021-03-10T17:46:42.787054+0000 mgr.gibba032.whopeo (mgr.24625) 12318 : cluster [DBG] pgmap v7023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:44.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:44 gibba011 bash[30791]: cluster 2021-03-10T17:46:42.787054+0000 mgr.gibba032.whopeo (mgr.24625) 12318 : cluster [DBG] pgmap v7023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:44.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:44 gibba011 bash[31774]: cluster 2021-03-10T17:46:42.787054+0000 mgr.gibba032.whopeo (mgr.24625) 12318 : cluster [DBG] pgmap v7023: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:46.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:46 gibba032 bash[24004]: cluster 2021-03-10T17:46:44.788341+0000 mgr.gibba032.whopeo (mgr.24625) 12319 : cluster [DBG] pgmap v7024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:46.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:46 gibba011 bash[30791]: cluster 2021-03-10T17:46:44.788341+0000 mgr.gibba032.whopeo (mgr.24625) 12319 : cluster [DBG] pgmap v7024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:46.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:46 gibba011 bash[31774]: cluster 2021-03-10T17:46:44.788341+0000 mgr.gibba032.whopeo (mgr.24625) 12319 : cluster [DBG] pgmap v7024: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:48.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:48 gibba032 bash[24004]: cluster 2021-03-10T17:46:46.790157+0000 mgr.gibba032.whopeo (mgr.24625) 12320 : cluster [DBG] pgmap v7025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:48.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:48 gibba011 bash[30791]: cluster 2021-03-10T17:46:46.790157+0000 mgr.gibba032.whopeo (mgr.24625) 12320 : cluster [DBG] pgmap v7025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:48.581 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:48 gibba011 bash[31774]: cluster 2021-03-10T17:46:46.790157+0000 mgr.gibba032.whopeo (mgr.24625) 12320 : cluster [DBG] pgmap v7025: 129 pgs: 129 active+clean; 7.0 KiB data, 54 MiB used, 672 GiB / 672 GiB avail 2021-03-10T17:46:49.509 INFO:journalctl@ceph.mon.b.gibba032.stdout:Mar 10 17:46:49 gibba032 bash[24004]: cephadm 2021-03-10T17:46:47.579098+0000 mgr.gibba032.whopeo (mgr.24625) 12321 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:49.581 INFO:journalctl@ceph.mon.a.gibba011.stdout:Mar 10 17:46:49 gibba011 bash[30791]: cephadm 2021-03-10T17:46:47.579098+0000 mgr.gibba032.whopeo (mgr.24625) 12321 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:49.582 INFO:journalctl@ceph.mon.c.gibba011.stdout:Mar 10 17:46:49 gibba011 bash[31774]: cephadm 2021-03-10T17:46:47.579098+0000 mgr.gibba032.whopeo (mgr.24625) 12321 : cephadm [ERR] Upgrade: ALERT: Cannot stop ['rgw.r.z.gibba011.xbljmx'] in RGW service. Not enough remaining RGW daemons. Please deploy at least 2 RGW daemons before stopping ['rgw.r.z.gibba011.xbljmx']. 2021-03-10T17:46:50.158 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-03-10T17:46:50.159 DEBUG:teuthology.task.console_log:Killing console logger for gibba011 2021-03-10T17:46:50.160 DEBUG:teuthology.task.console_log:Killing console logger for gibba032 2021-03-10T17:46:50.160 DEBUG:teuthology.exit:Finished running handlers